Hi Alberto,
Odd number of cards will be used equally efficiency as even numbers for
2D and 3D classification. It's only the 3D auto-refine that makes better
use of even number of cards (as each half-set is processed by a
different MPI process). If you need 1 card for something else, I would
probably choose to run relion with 3 cards rather than complicating your
setup with 5 cards....
HTH,
Sjors
On 07/18/2016 10:42 AM, Alberto Riera wrote:
> Thank you for all the advice!
>
> I was thinking a little bit more about this computer and I wonder if
> it would be advantageous for us to install an additional GPU. I know
> that Relion uses the GPUs better in pairs, so the idea would be to let
> 4 cards just for it and use the spare one for some other tasks if
> needed. Alternatively, we could use the fifth card with Relion as
> well, although it would be less efficient than having six of them.
>
>
> On 14/07/16 16:01, Craig Yoshioka wrote:
>> Hi Alberto,
>>
>> I’d like to add, it might be worthwhile to keep a few of those CPUs
>> and RAM on the side for ZFS. I’d highly recommend turning on ZFS LZ4
>> compression, especially on raw frame stacks it’ll save a lot of space
>> and be faster I/O.
>>
>>
>>
>>> On Jul 14, 2016, at 5:09 AM, Erik Lindahl <[log in to unmask]
>>> <mailto:[log in to unmask]>> wrote:
>>>
>>> Hi Alberto,
>>>
>>> On vacation, so sorry if I'm a bit brief.
>>>
>>> I think your machine will work great if it's intended as a staging
>>> server where you do all/most processing. We have a highly similar
>>> one, where we also got an external 44-disk enclosure for storage.
>>>
>>> However, if you want to get more than one machine, we prefer to
>>> separate CPU-heavy and GPU-heavy nodes. The reason is that even the
>>> GPU code will use CPUs, so you will hurt your GPU performance a lot
>>> if you start to run a CPU job on the same machine - and if you
>>> don't, you waste those expensive CPUs. For this reason we bought
>>> one set of CPU-only nodes with fast processors and then cheap CPUs
>>> for the GPU nodes - on the whole I think that provides better value
>>> for cluster nodes.
>>>
>>> We no longer use RAID cards, but have moved to ZFS instead. If you
>>> use ubuntu 16.04 it even comes built-in.
>>>
>>> The released version of oculus rift unfortunately no longer supports
>>> plain display mode, so unless a program has been adapted to their
>>> devkit, you won't be able to use it.
>>>
>>> Cheers,
>>>
>>> Erik
>>>
>>>
>>>
>>> Erik Lindahl <[log in to unmask]
>>> <mailto:[log in to unmask]>>
>>> Professor of Biophysics
>>> Science for Life Laboratory
>>> Stockholm University & KTH
>>> Office (SciLifeLab): +46 8 524 81567
>>> Cell (Sweden): +46 73 4618050
>>> Cell (US): +1 267 3078746
>>>
>>>
>>> On 14 juli 2016, at 07:24, Alberto Riera
>>> <[log in to unmask] <mailto:[log in to unmask]>>
>>> wrote:
>>>
>>>> Hi guys!
>>>>
>>>> After a lot of days working on this we finally have defined a
>>>> system to work with the GPU implementation of Relion and it would
>>>> be great it you could tell us your opinion. Currently it is a bit
>>>> on the overkill side from the CPU point of view, but we managed to
>>>> obtain a very good price and, on the other hand, it is future proof
>>>> in this regard and in case we need to install more RAM (it goes up
>>>> to 2TB).
>>>>
>>>> So, it is going to be a Supermicro 4U GPU chassis (workstation
>>>> format) with the following:
>>>>
>>>> 2x Intel E5-2697v4 2.3Ghz Processors (36 cores)
>>>> 8x Supermicro 16GB DDR4 ECC REG Memory
>>>> 1x 960GB SATA Read Intensive Enterprise Level SSD
>>>> 4x 2TB SATA Mixed Workload Enterprise Level SSD (RAID 5)
>>>> 1x LSI 9361-8i RAID Controller inc CacheVault
>>>> 4x NVidia GTX 1080
>>>> 1x Mellanox Single Port 10Gb SFP+ NIC
>>>>
>>>> Any opinions on it?
>>>>
>>>> On the other hand, we are looking to do some 3D visualization with
>>>> the system and I wonder if any of you has experience with Linux and
>>>> the new VR headsets. They seem very promising but I still did not
>>>> have the chance to test them. If you have any other preferred
>>>> option, suggestions are very welcome :).
>>>>
>>>> Thanks!
>>>>
>>>> Alberto
>>>>
>>>> --
>>>> Alberto Riera PhD
>>>> Imperial College London
>>>> Faculty of Medicine
>>>> Institute of Clinical Sciences (CRB room 3006)
>>>> Hammersmith Hospital Campus
>>>> Du Cane Road
>>>> London W12 0NN
>>>>
>>>> [log in to unmask]
>>
>
--
Sjors Scheres
MRC Laboratory of Molecular Biology
Francis Crick Avenue, Cambridge Biomedical Campus
Cambridge CB2 0QH, U.K.
tel: +44 (0)1223 267061
http://www2.mrc-lmb.cam.ac.uk/groups/scheres
|