Hi,
> libcufft.so.9.2 => /usr/local/cuda-9.2/lib64/libcufft.so.9.2
> (0x00007f2a6b5b7000)
> libcurand.so.9.2 => /usr/local/cuda-9.2/lib64/libcurand.so.9.2
> (0x00007f2a66808000)
> libcudart.so.9.2 => /usr/local/cuda-9.2/lib64/libcudart.so.9.2
> (0x00007f2a6659e000)
This means that you can copy /usr/local/cuda-9.2/lib64 to
somewhere (say ~/local/cuda-lib) and set LD_LIBRARY_PATH to there.
> Ok, I think the best is the second install, I'll try it out and see if I
> can get 100 MPI's running. Is there an optimum number of threads/MPI given
> that memory is not a limiting factor?
The number of frames should be divisible by the number of threads for
efficiency.
For example, 1, 2, 4, 8, 16, 32 or 64 threads are good for 64 frames.
32 threads for 65 frames are very inefficient, because
- 1st cycle: frames 1 -32 are processed by threads 1-32
- 2nd cycle: frames 33-64 are processed by threads 1-32
- 3rd cycle: frame 65 is processed by thread 1, other 30 threads being idle.
If the memory is not limiting, having more MPIs is better than having more
threads. That is, on a 32 core machine, 8 MPI x 4 threads are
usually faster than 1 MPI x 32 threads .
Best regards,
Takanori Nakane
> Thanks,
>
> Mike.
> ________________________________________
> From: Takanori Nakane [[log in to unmask]]
> Sent: 21 February 2019 12:46
> To: Michael Swan
> Cc: CCPEM
> Subject: RE: [ccpem] After beam-induced motion correction in Relion3: what
> are the units of the accumulated motions in the output graphs/histograms?
> Pixels?
>
> Hi,
>
>> -bash-4.2$ ldd 'which relion_run_motioncorr_mpi'
>> ldd: ./which relion_run_motioncorr_mpi: No such file or directory
>
> Use backquote "`", not single quote "'".
>
>> Anyway, it is a large cluster with many nodes so installing cuda on all
>> the cpu only ones is not feasible.
>
> As I said, you need only the CUDA runtime, not a driver.
> You can put it in your home directory or wherever accessible from
> all nodes, just as you have installed RELION. Then set LD_LIBRARY_PATH
> to there.
>
>> Then I did compile with -DCUDA=ON which I need for the downstream
>> classification and refinement so is the only way to have a second
>> version
>> with it off just for motioncor?
>
> If you don't want to install CUDA runtime, this is the only option.
> Make a new build directory, compile with -DCUDA=OFF, write a new
> cluster submission script that sets PATH accordingly,
> and use it for CPU jobs.
>
> Best regards,
>
> Takanori Nakane
>
> P.S. Please CC to CCPEM when replying.
>
>> Thanks again,
>>
>> Mike.
>> ________________________________________
>> From: Takanori Nakane [[log in to unmask]]
>> Sent: 21 February 2019 08:41
>> To: Michael Swan; CCPEM
>> Subject: Re: [ccpem] After beam-induced motion correction in Relion3:
>> what
>> are the units of the accumulated motions in the output
>> graphs/histograms?
>> Pixels?
>>
>> Hi,
>>
>> Please show me your command line and the result of
>> "ldd `which relion_run_motioncorr`".
>>
>> Probably your binary has been linked against CUDA runtime.
>> If so, the program fails to start even if you don't actually use CUDA.
>> The easiest fix is to build RELION without CUDA (-DCUDA=OFF).
>>
>> Alternatively you can install CUDA runtime on CPU nodes; you don't need
>> drivers, just runtime. So you don't need root privileges.
>>
>> Best regards,
>>
>> Takanori Nakane
>>
>> On 2019/02/21 8:30, Michael Swan wrote:
>>> Hi Takanori,
>>>
>>> quick question I hope; I am trying to run motioncor using the Relion
>>> implementation on a cluster sge queue and it fails with an error
>>> relating to loading cuda libraries. I thought this was a cpu only
>>> implementation so why does it need to find any cuda stuff? I want to
>>> run
>>> say 100MPIs which will be split between several nodes and if I try
>>> including the cuda modules it tends to fail as not all nodes are cuda
>>> capable.
>>>
>>> Any suggestions would be much appreciated.
>>>
>>> Best regards,
>>>
>>> Mike.
>>> ________________________________________
>>> From: Collaborative Computational Project in Electron cryo-Microscopy
>>> [[log in to unmask]] on behalf of Takanori Nakane
>>> [[log in to unmask]]
>>> Sent: 20 February 2019 15:24
>>> To: [log in to unmask]
>>> Subject: Re: [ccpem] After beam-induced motion correction in Relion3:
>>> what are the units of the accumulated motions in the output
>>> graphs/histograms? Pixels?
>>>
>>> Hi,
>>>
>>>> what are the units of the motions in the file? Pixels?
>>>
>>> Yes, pixels before binning (unless you specified --early_binning).
>>>
>>>> After my first attempt at motion correction,
>>>> the majority seem to have small estimated motions but about 30 or so
>>>> have
>>>> estimated Total accumulated motions between 200 and 1200 units (which
>>>> is
>>>> the max)
>>>> so I am considering excluding them from this very first pass/look at
>>>> the
>>>> data.
>>>
>>> This is mostly personal preference but I don't remove any movies using
>>> these statistics. These numbers are not reliable especially when the
>>> trajectories are noisy (these are merely line integrals along
>>> trajectories).
>>>
>>> If motion estimation diverged, the output becomes very bad and
>>> particles
>>> on them will be rejected during classification.
>>> Although some people are very concerned about outliers and spend long
>>> time manually culling them, in my experience, the final resolution
>>> rarely
>>> changes,
>>> or only one or two shells if any, provided that you follow the standard
>>> procedure
>>> of Class2D and Class3D.
>>>
>>> Best regards,
>>>
>>> Takanori Nakane
>>>
>>>> After my first attempt at motion correction,
>>>> the majority seem to have small estimated motions but about 30 or so
>>>> have
>>>> estimated Total accumulated motions between 200 and 1200 units (which
>>>> is
>>>> the max)
>>>> so I am considering excluding them from this very first pass/look at
>>>> the
>>>> data.
>>>>
>>>> Does this sound sensible? I imagine these decisions depend on what one
>>>> is
>>>> trying to achieve at each stage of the process,
>>>> and I am aware that I can always come back and revisit them at a later
>>>> stage in the data processing,
>>>> still I am interested in hearing what people's established procedures
>>>> are
>>>> regarding this.
>>>>
>>>> Thank you,
>>>>
>>>> Pietro
>>>>
>>>>
>>>>
>>>> Pietro Roversi
>>>>
>>>> LISCB Wellcome Trust ISSF Fellow
>>>>
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bit.ly_2I4Wm5Z&d=DwIDJg&c=TzEZu9LIcihmW37vx9Ah6w&r=MGqTYVjznyJeo11cyogL6dk-zNlLgCG7sodeNan77aw&m=7kIcmexTtrKzIQaJPb1rMa0D-XywN0qTI6ztZgvs1h8&s=Gs0TalF0eXYAgcC5dFVd61vk6t61w6ORTnu-J_ki6yQ&e=
>>>>
>>>>
>>>> Leicester Institute of Structural and Chemical Biology
>>>> Department of Molecular and Cell Biology, University of Leicester
>>>> Henry Wellcome Building
>>>> Lancaster Road, Leicester, LE1 7HB
>>>> England, United Kingdom
>>>>
>>>> Tel. +44 (0)116 2297237
>>>>
>>>>
>>>> ########################################################################
>>>>
>>>> To unsubscribe from the CCPEM list, click the following link:
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCPEM-26A-3D1&d=DwIDJg&c=TzEZu9LIcihmW37vx9Ah6w&r=MGqTYVjznyJeo11cyogL6dk-zNlLgCG7sodeNan77aw&m=7kIcmexTtrKzIQaJPb1rMa0D-XywN0qTI6ztZgvs1h8&s=jxLSToGw6UFwQ4h6xVRmPrJfPMcf5hEJ7_NEEXWSnu0&e=
>>>>
>>>
>>> ########################################################################
>>>
>>> To unsubscribe from the CCPEM list, click the following link:
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCPEM-26A-3D1&d=DwIDJg&c=TzEZu9LIcihmW37vx9Ah6w&r=MGqTYVjznyJeo11cyogL6dk-zNlLgCG7sodeNan77aw&m=7kIcmexTtrKzIQaJPb1rMa0D-XywN0qTI6ztZgvs1h8&s=jxLSToGw6UFwQ4h6xVRmPrJfPMcf5hEJ7_NEEXWSnu0&e=
>>>
>>>
>>> This email message and any attachments are confidential and intended
>>> for
>>> use by the addressee(s) only. If you are not the intended recipient,
>>> please notify me immediately by replying to this message, and destroy
>>> all copies of this message and any attachments. Thank you.
>>>
>>
>>
>>
>> This email message and any attachments are confidential and intended for
>> use by the addressee(s) only. If you are not the intended recipient,
>> please notify me immediately by replying to this message, and destroy
>> all
>> copies of this message and any attachments. Thank you.
>>
>
>
>
>
> This email message and any attachments are confidential and intended for
> use by the addressee(s) only. If you are not the intended recipient,
> please notify me immediately by replying to this message, and destroy all
> copies of this message and any attachments. Thank you.
>
########################################################################
To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
|