Hi,
> it should be not the GPU memory
It IS the GPU memory shortage. MultiBody refinement needs much
more GPU memory in each GPU. In the worse case, where
each body overlaps with all others, it requires N*N times
memory of the standard Refine3D (N is the number of bodies).
Please use 'skip padding: Yes', which reduces memory
consumption to 1/8.
Best regards,
Takanori Nakane
On 2019/09/07 21:25, Lifei wrote:
> Hey all,
>
> I tried to test the Muti-body refinement in Relion 3, and download the test data spliceosomal B-complex (accession number 10180) from the EMPIAR data base.
>
> However, when I started to run the Muti-body refinement job, I have came across an error indicating as below:
> ERROR: out of memory in /home/greg/relion/src/acc/acc_projector_impl.h at line 63 (error-code 2)
> in: /home/relion/src/acc/cuda/cuda_settings.h, line 67
> === Backtrace ===
> /usr/local/relion/bin/relion_refine_mpi(_ZN11RelionErrorC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_l+0x6d) [0x44256d]
> /usr/local/relion/bin/relion_refine_mpi() [0x61649a]
> /usr/local/relion/bin/relion_refine_mpi(_ZN12AccProjector9setMdlDimEiiiiiii+0x1e3) [0x616c43]
> /usr/local/relion/bin/relion_refine_mpi(_ZN14MlDeviceBundle22setupFixedSizedObjectsEv+0x324) [0x61f314]
> /usr/local/relion/bin/relion_refine_mpi(_ZN14MlOptimiserMpi11expectationEv+0x186d) [0x4467dd]
> /usr/local/relion/bin/relion_refine_mpi(_ZN14MlOptimiserMpi7iterateEv+0xca) [0x458e7a]
> /usr/local/relion/bin/relion_refine_mpi(main+0x2091) [0x436c31]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f3b08560830]
> /usr/local/relion/bin/relion_refine_mpi(_start+0x29) [0x4391c9]
> ==================
> ERROR:
> A GPU-function failed to execute.
> If this occured at the start of a run, you might have GPUs which
> are incompatible with either the data or your installation of relion.
> If you
> -> INSTALLED RELION YOURSELF: if you e.g. specified -DCUDA_ARCH=50
> and are trying ot run on a compute 3.5 GPU (-DCUDA_ARCH=3.5),
> this may happen.
> -> HAVE MULTIPLE GPUS OF DIFFERNT VERSIONS: relion needs GPUS with
> at least compute 3.5. You may be trying to use a GPU older than
> this. If you have multiple generations, try specifying --gpu <X>
> with X=0. Then try X=1 in a new run, and so on. The numbering of
> GPUs may not be obvious from the driver or intuition. For a list
> of GPU compute generations, see
> en.wikipedia.org/wiki/CUDA#Version_features_and_specifications
> -> ARE USING DOUBLE-PRECISION GPU CODE: relion was been written so
> as to not require this, and may thus have unforeseen requirements
> when run in this mode. If you think it is nonetheless necessary,
> please consult the developers with this error.
> If this occurred at the middle or end of a run, it might be that
> -> YOUR DATA OR PARAMETERS WERE UNEXPECTED: execution on GPUs is
> subject to many restrictions, and relion is written to work within
> common restraints. If you have exotic data or settings, unexpected
> configurations may occur. See also above point regarding
> double precision.
> If none of the above applies, please report the error to the relion
> developers at github.com/3dem/relion/issues
>
> I had this error message from the 1st iteration regarding to GPU, but there are 4 GTX 1080 GPUs with 8 GB each (CUDA 10), CPU 32 Cores with 128 GB, after changed the MPI number, still have the same issue, it should be not the GPU memory. Any idea about how to fix it?
>
> Thanks in advance,
> Lifei
>
> ########################################################################
>
> To unsubscribe from the CCPEM list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
>
########################################################################
To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
|