Hi David,
Most of the stochasticity you observe is probably down to file access
patterns and network. In order to make sure your benchmarks are
completely the same you e.g. have to purge the RAM-cache before each
run, otherwise the second run might be faster. However, running the
standard benchmarks on our system, variation is nowhere near 10%. I
would guess that network-speed or influence from other processes could
cause larger distortions on some systems and for some tests, perhaps you
are seeing such effects? To exclude them, I'd run a small enough
benchmark that it can be repeated, and where I know it fit's in RAM and
on minimal resources, so that network traffic is omitted from
consideration.
RELION also perturbs the oriention-sampling each iteration to avoid
finite-sampling effects. You probably want to turn that off by using
--perturb 0.
Using double precision is more reproducible, but slower and irrelevant
for most benchmarks. Since RELION distributes images to it's MPI-slaves
on-demand, there's no way of enforcing strict reproducibility in terms
of the order in which images are used and summed into a reconstruction.
This applies to the performance as well.
If you have more specific questions or consideration in mind, feel free
to email me directly.
/Björn
On 02/20/2018 05:44 PM, David Hoover wrote:
> Hey all,
>
> Is there a way of running 2D or 3D classification such that the job runs exactly the same and takes exactly the same time? In my benchmarking tests, using the same --seed value, the elapsed time for the RELION job can have standard deviations of +/- 12%, and it makes it hard to compare the effect of hardware on run times.
>
> David
>
> ----------------------------------------------------------------
> David Hoover, HPC @ NIH
> [log in to unmask]
|