Hi Eric,
> On 30 Mar 2021, at 12:57, Eric Gibbs <[log in to unmask]> wrote:
>
> The micrograph looping then proceeded at a pace that would have taken 80 hours to complete. I tweaked the input to be 1 process per node and 24 cpus per process and that reduced the time to 50-60 hours, but still much slower than a single node.
What sort of file system are the movies stored on? Polishing has to read the movies and this can mean that with multiple nodes all hitting the file system I/O is the bottleneck and most of the CPUs are idle. If on your cluster you are allowed to ssh into the compute nodes you can easily check this in top.
Best regards,
Huw
########################################################################
To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1
This message was issued to members of www.jiscmail.ac.uk/CCPEM, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
|