Agreed. Do classification on the parts that are “variable” and see if anything can be identified.
Best wishes,
Reza
Reza Khayat, PhD
Assistant Professor
Department of Chemistry
City College of New York
85 Saint Nicholas Terrace, CDI 2.318
New York, NY 10031
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.khayatlab.org_&d=DgIGaQ&c=4NmamNZG3KTnUCoC6InoLJ6KV1tbVKrkZXHRwtIMGmo&r=vqUYK5Lmyk_tQCp60dUyDsN43c2sMB12ww1nOK_Ac-M&m=k8QGlNmL6Q38R3EM9NxCR5vhBJpJa_0nkQOp4CH5pAQ&s=B0gQYkBcMr-bqgGVF2cLZE2cSeLIhdzqBpyAkl9N--Q&e=
212-650-6070
From: Collaborative Computational Project in Electron cryo-Microscopy [mailto:[log in to unmask]] On Behalf Of Ludtke, Steven J
Sent: Wednesday, April 19, 2017 12:39 PM
To: [log in to unmask]
Subject: Re: [ccpem] Sphire sx3dvariability -important note
Variance is nonlinear. I would suggest starting with Penczek's first paper on the subject (he first introduced the concept in CryoEM):
Penczek, P. A., Yang, C., Frank, J. & Spahn, C. M. (2006) Estimation of variance in single-particle reconstruction using the bootstrap technique. J. Struct. Biol. 154, 168-183.
IMHO, variance maps are of somewhat limited utility. They can tell you qualitatively if one portion of the map is less stable or uniform than another part, but that really just tells you that you should be doing additional analysis to determine the details...
On Apr 18, 2017, at 3:57 PM, Zack Berndsen <[log in to unmask]<mailto:[log in to unmask]>> wrote:
***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.
________________________________
Good point, Thanks Steven. On that note, how should one interpret/report resolution of a variance map ? I'm not sure I see the distinction between the two situations you mentioned
Zachary Berndsen, PhD
The Scripps Research Institute
Hazen Building (HL 104)
Ward Lab
________________________________
From: Ludtke, Steven J <[log in to unmask]<mailto:[log in to unmask]>>
Sent: Tuesday, April 18, 2017 1:40:33 PM
To: Zack Berndsen
Cc: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [ccpem] Sphire sx3dvariability -important note
Just a quick note that, (independent of software) a low-pass filtered variance map is NOT the same as a variance map computed on low-pass filtered (or downsampled data). There may be a good reason to look at both of these things, but they are not interchangeable!
----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept. of Biochemistry and Mol. Biol. Those who do
Co-Director National Center For Macromolecular Imaging
ARE
Baylor College of Medicine The converse
[log in to unmask]<mailto:[log in to unmask]> -or- [log in to unmask]<mailto:[log in to unmask]> also applies
https://urldefense.proofpoint.com/v2/url?u=http-3A__ncmi.bcm.edu_-7Estevel&d=DgIGaQ&c=4NmamNZG3KTnUCoC6InoLJ6KV1tbVKrkZXHRwtIMGmo&r=vqUYK5Lmyk_tQCp60dUyDsN43c2sMB12ww1nOK_Ac-M&m=k8QGlNmL6Q38R3EM9NxCR5vhBJpJa_0nkQOp4CH5pAQ&s=xzP5s8j5hbjnguJZa1Sr7kVOvI9bCBTCbW96Ab4VJBY&e= <https://urldefense.proofpoint.com/v2/url?u=http-3A__ncmi.bcm.edu_-7Estevel&d=DgMGaQ&c=4NmamNZG3KTnUCoC6InoLJ6KV1tbVKrkZXHRwtIMGmo&r=1DzJFW0v6TgEhkW1gy_-ke-RbtvS1fzEbD5_hcb9Up0&m=pqBhNnDCon8OGlje-IX2wwOkrIfBH8oJFMi29dna0Ag&s=-fsvEUEWzblTd9bNVyig4YQ9DYkRczl2b7txBGajHRY&e=>
On Apr 18, 2017, at 3:27 PM, Zack Berndsen <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Hi Reza,
In my experience the 3dvariability maps need to be low-pass filtered before they can be interpreted, so binning your data will not limit you so long as the binned pixel size is not larger than FilterFrequency/2. Say you need to filter to 8A, with a pixel size of 1.16A your binned by 2 Nyquist frequency will be 4.64 , so binning your data will not limit you at all and will speed up processing.
Binning is accomplished by the --decimate flag and low-pass filtering can be implemented with the --fl and --aa flags. I use the EMAN/SPARX daily build and a Relion 2.0 refinement data.star file as my input and run the following script on 16x16cpu nodes:
sxrelion2sparx.py Refine3D/job###/run_data.star --output_dir=Sparx_3DVar --create_stack
cd Sparx_3DVar
sxcpy.py sparx_stack.hdf bdb:sparx_stack
sxheader.py bdb:sparx_stack --import=sparx_stack_proj3d.txt --params=xform.projection
sxheader.py bdb:sparx_stack --import=sparx_stack_ctf.txt --params=ctf
mpirun -np 256 sx3dvariability.py bdb:sparx_stack --CTF --VERBOSE --img_per_grp=100 --fl=0.40 --aa=0.1 \
--ave3D=3Dvol_SPX.hdf --var3D=3Dvar_SPX.hdf --decimate=2
I usually have to filter my variability maps even further, but the hdf files do not contain the right pixel size so you will have to adjust that. I usually convert them to mrc files with the correct pixel size (here because we decimate by 2 the pixel size should be 2.32) then use relion_image_handler to filter to the desired frequency. Or you can adjust the pixel size and use the gaussian filter in Chimera with a stdv of 2 or 3 usually works for me. Depending on how isotropic your angular distribution is and the number of particles in the data set you may want to play around with the --img_per_group flag, maybe vary it between 50 and 200 and compare the results. The --ave3D produces a 3D reconstruction from the grouped/averaged images and gives you a good idea of whether or not you are getting any strange rotational artifacts from grouping.
With 68K particles decimated by 2 and 40 cpus the job should not take too long, maybe a few hours, but its hard to say. with ~150K particles and 256 cpus my jobs do not take more than an hour. You can always run it on a smaller subset to benchmark the speed.
Hope this helps,
Zachary Berndsen, PhD
The Scripps Research Institute
Hazen Building (HL 104)
Ward Lab
________________________________
From: Collaborative Computational Project in Electron cryo-Microscopy <[log in to unmask]<mailto:[log in to unmask]>> on behalf of Reza Khayat <[log in to unmask]<mailto:[log in to unmask]>>
Sent: Monday, April 17, 2017 8:37:36 AM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: [ccpem] Sphire sx3dvariability
Hi,
Can anyone suggest some starting parameters for using sx3dvariability in Sphire/Sparx? I have about 68,000 particles, 1.16Å pixel size, 256x256 boxsize, and a resolution of 4.3Å. Should I bin the images, how many img_per_grp (default is 10 but example uses 100)... Thanks.
Also, how long will this take if I only have 40 cores for the job?
Best wishes,
Reza
Reza Khayat, PhD
Assistant Professor
City College of New York
Department of Chemistry
New York, NY 10031
________________________________
From: Collaborative Computational Project in Electron cryo-Microscopy <[log in to unmask]<mailto:[log in to unmask]>> on behalf of Shawn Zheng <[log in to unmask]<mailto:[log in to unmask]>>
Sent: Thursday, April 13, 2017 3:23 PM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [ccpem] Motion Correction of Low Dose Datasets
Hi Joshua,
The rule of thumb is to use a group number as small as possible to get reliable motion correction. When you see many times of message of repeating measurements on larger patch, this is a sign of unreliable motion correction.
Someone else also suggested to use larger b factor. The default is 100. You can double it.
Best
Shawn
On Thu, Apr 13, 2017 at 10:00 AM Joshua Lobo <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Hi CCPEM
When performing motion-correction of low-dose data sets what would be the best method to optimize the group parameter ( to group frames in order to increase signal).Is there a rule of thumb based on dose accumulated or a particular metric used to track the effect of grouping on the actual signal . Any inputs or suggestions on how this is usually done would be really helpful
Sincerely
Joshua Lobo
----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept of Biochemistry and Mol. Biol. (www.bcm.edu/biochem<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.bcm.edu_biochem&d=DgMGaQ&c=4NmamNZG3KTnUCoC6InoLJ6KV1tbVKrkZXHRwtIMGmo&r=1DzJFW0v6TgEhkW1gy_-ke-RbtvS1fzEbD5_hcb9Up0&m=pqBhNnDCon8OGlje-IX2wwOkrIfBH8oJFMi29dna0Ag&s=ofN7kHTU1KBnIg59SP5msoegvFMwY2vwGJdd-ELGuAI&e=>)
Co-Director National Center For Macromolecular Imaging (ncmi.bcm.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ncmi.bcm.edu&d=DgMGaQ&c=4NmamNZG3KTnUCoC6InoLJ6KV1tbVKrkZXHRwtIMGmo&r=1DzJFW0v6TgEhkW1gy_-ke-RbtvS1fzEbD5_hcb9Up0&m=pqBhNnDCon8OGlje-IX2wwOkrIfBH8oJFMi29dna0Ag&s=sqW2VCxNKa1nUucq9v4PLSGf4ogQ_eQMLkEcWfM_HDY&e=>)
Co-Director CIBR Center (www.bcm.edu/research/cibr<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.bcm.edu_research_cibr&d=DgMGaQ&c=4NmamNZG3KTnUCoC6InoLJ6KV1tbVKrkZXHRwtIMGmo&r=1DzJFW0v6TgEhkW1gy_-ke-RbtvS1fzEbD5_hcb9Up0&m=pqBhNnDCon8OGlje-IX2wwOkrIfBH8oJFMi29dna0Ag&s=Pp3rpqHP73z03HhFO7Q9Qa-pv4hasIbSNYaeEZFv4c0&e=>)
Baylor College of Medicine
[log in to unmask]<mailto:[log in to unmask]>
