Hi David,
We are running a box size of 600 pixels on a 16 core Xeon (two chips, total 32 cores), 384 GB workstation. To get this to run, I split the job over the minimum three MPI processes with 128 GB per process, using more than 2 threads doesn’t seem to work for us but this could be our set up. These refinements get through with using 68 GB per MPI process in the final converged maximisation step (double precision in relion-1.4).
Looks like it could handle bigger boxes but I have not tested to see precisely how large we can go. I do know that a 900 pixel box eats up the memory and crashes at the final maximisation.
Hope our real world test case might help you predict what you need!
All the best,
Kyle
> On 9 Dec 2015, at 11:16, David Bhella <[log in to unmask]> wrote:
>
> I am trying to put together a funding request for a cluster and I would be interested to know what people’s thoughts and experiences are regarding optimal cluster configuration for Relion. In particular I am wondering whether lots of 8-core nodes (with ~64 GB RAM) or fewer higher core-density nodes would be preferable for example with 4x E7-8420 cores and ~512 GB RAM? Is 8-12 GB/core an appropriate amount of memory (considering we sometimes work with quite large viruses)? It seems to me that most of the heavy lifting in terms of memory use is done by the MPI master - is this true, and if so is it possible to configure SGE to address MPI jobs to designated high-memory nodes?
>
> Also any comments on where bottlenecks are would be helpful - what is the best networking option (10Gb?), is parallel disk access beneficial?
>
> I would be most grateful for any guidance or recent experience. Finally, should I just forget local hardware and go for a cloud computing option? (What worries me about this is that we then pay for data processing from our grants for ever after rather than a one-off capital equipment award to cover several years of number crunching.)
>
> Many thanks,
> D.
>
> Dr David Bhella
> MRC-University of Glasgow Centre for Virus Research
> Sir Michael Stoker Building
> Garscube Campus
> 464 Bearsden Road
> Glasgow G61 1QH
> Scotland (UK)
>
> Telephone: 0141-330-3685
> Skype: d.bhella
>
> Virus structure group on Facebook: https://www.facebook.com/CVRstructure
> Molecular Machines - Images from Virus Research: http://www.molecularmachines.org.uk
>
> CVR website: http://www.cvr.ac.uk
> CVR on Facebook: https://www.facebook.com/centreforvirusresearch
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|