JISCMail - CCPEM Archives

On 31 Jul 2022, at 13:37, Ludtke, Steven J. <[log in to unmask]> wrote:

e2proc2d doesn't support reading JPG intentionally due to the impact JPG encoding has on many algorithms. At present, we only support JPG for writing.

Having said that, I would suggest testing 2 things. If either of these produces something like the contrast enhancement you are looking for, I can give you equivalent command-line processes to do them in batch (very quickly, as they are quite simple procedures):

run e2display.py and open one of the MRC files with the contrast you have issues with.

middle-click for a control panel, then where it says "Normal" try "Hist Flat" or "Hist Gauss".

or

run

e2boxer.py micrograph.mrc --gui

and press the "Filter Disp." toggle

Both of these processes tend to improve the contrast for particle picking substantially via straightforward filtration processes. Nothing against deep learning for visualization, we use that extensively for other purposes.

--------------------------------------------------------------------------------------

Steven Ludtke, Ph.D. <[log in to unmask]> Baylor College of Medicine

Charles C. Bell Jr., Professor of Structural Biology Dept. of Biochemistry

Deputy Director, Advanced Technical Cores and Molecular Biology

Academic Director, CryoEM Core

Co-Director CIBR Center

On Jul 31, 2022, at 7:16 AM, Samuel Haysom [RPG] <[log in to unmask]> wrote:

***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.
Hi Pranav and Takanori

Thanks for your suggestions, I will try running some demonising if I can’t get this to work. I was trying to use the jpg’s because they are automatically generated during collection so I thought I could save on some compute time by using them instead of running denoising jobs.

I normally do my picking with cryolo’s general model but for my current dataset I need to do some picking to train my own model. I was planning to do that on the jpgs to increase my chances of separating good particles from junk in the training data. Of course I would not be extracting particles directly from the jpegs, but converting the coordinates and then using them to extract from the motion corrected micrographs.

Kind regards,

Sam

On 31 Jul 2022, at 12:55, Pranav Shah <[log in to unmask]> wrote:

Btw, i second takanori’s comments.

You could pick a small subset of your particles promiscuously(i.e large false postives) using the LoG picker in relion -> 2d classify -> select good classes for template based 2d picking OR train a neural net to pick from the remaining mics.

On Sun, 31 Jul 2022 at 12:53, Pranav Shah <[log in to unmask]> wrote:

If particle visualisation is a problem you can use topaz to denoise the images for you.

I reckon that a wrapper to do that trivially in relion may be provided. If not you will have to call the relevant topaz routines externally. You could also consider janni from cryolo which will give you similar results….

On Sun, 31 Jul 2022 at 12:45, Samuel Haysom [RPG] <[log in to unmask]> wrote:

Hi Pranav,

I see, thanks for the info. To be clear, I wasn’t intending to extract from the .jpg files, I just find that the filtering EPU applies makes it much easier to see the particles than anything I can achieve in RELION. I was planning to try picking on the jpg files then scaling the coordinates and using them to extract from my motion corrected full scale micrographs.

Do you know of a particle picking tool that would display my micrographs with similar filtering to that used by EPU when generating the .jpg files?

I’m not sure this is an issue with metadata as I also cannot convert to simpler image formats such as png which I assume would not require metadata.

Kind regards,
Sam

> On 31 Jul 2022, at 12:37, Pranav Shah <[log in to unmask]> wrote:
>
> HI Samuel,
> Jpegs' cannot trivially be converted to mrc files as they do not store
> a lot of the image metadata information that is stored in an actual
> mrc file as such the jpg files are useful for only quick visualisation
> of the collected data. You should use the mrc files generated by your
> microscope data collection software to pick particles and all down
> stream processing.
> Best,
> Pranav
> --
> Pranav Shah
> Postdoctoral Research Fellow.
>
> Division of Structural Biology,
> Wellcome Trust Centre for Human Genetics,
> University of Oxford,
> Roosevelt Drive, Oxford OX3 7BN,
> UK
>
>
> On Sun, Jul 31, 2022 at 12:27 PM Samuel Haysom [RPG] <[log in to unmask]> wrote:
>>
>> Hello,
>>
>> I am trying to convert the .jpg images that EPU generates for each micrograph during a collection into .mrc files so I can view them and pick on them in RELION (my installation cannot seem to natively open .jpg files the, at least using relion_display). However, e2proc2d.py is giving me an error when I try to perform the conversion. The command I am using (with eman2 v2.3) is:
>>
>> e2proc2d.py FoilHole_27553537_Data_27541781_27541783_20220728_072412.jpg FoilHole_27553537_Data_27541781_27541783_20220728_072412.mrc
>>
>> And I get the following error:
>>
>> Traceback (most recent call last):
>> File "/absl/EM/emsoftware2/env-modules/install/eman2/2.3/bin/e2proc2d.py", line 1173, in <module>
>> main()
>> File "/absl/EM/emsoftware2/env-modules/install/eman2/2.3/bin/e2proc2d.py", line 450, in main
>> nimg = EMUtil.get_image_count(infile)
>> File "/fbs/emsoftware2/env-modules/install/eman2/2.3/lib/python2.7/site-packages/EMAN2db.py", line 545, in db_get_image_count
>> raise Exception(fsp)
>> Exception: FoilHole_27553537_Data_27541781_27541783_20220728_072412.jpg
>>
>> Has anyone seen this error or know how to fix it? Some googling indicates it's to do with e2proc2d.py not properly recognising the file header but I couldn’t find any solution. Alternatively, can anyone recommend an alternative file conversion tool I could use?
>>
>> Kind regards,
>> Sam
>>
>> ########################################################################
>>
>> To unsubscribe from the CCPEM list, click the following link:
>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fcgi-bin%2FWA-JISC.exe%3FSUBED1%3DCCPEM%26A%3D1&data=05%7C01%7Cbssfh%40leeds.ac.uk%7Ccb2d5343b58e42f0105308da72e92afa%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C1%7C0%7C637948643022107048%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MJFxVSjhQt68HghudADo%2BdS12dii0jvWEOidLS2Wk3g%3D&reserved=0
>>
>> This message was issued to members of https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2FCCPEM&data=05%7C01%7Cbssfh%40leeds.ac.uk%7Ccb2d5343b58e42f0105308da72e92afa%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C1%7C0%7C637948643022107048%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Uv6QlDtmlVH3zd9yaXkkgbxMJ6B7zndBX%2B1n5G5ev4k%3D&reserved=0, a mailing list hosted by https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.jiscmail.ac.uk%2F&data=05%7C01%7Cbssfh%40leeds.ac.uk%7Ccb2d5343b58e42f0105308da72e92afa%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C1%7C0%7C637948643022107048%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jFwGifUOxZM0XkNvawbjScGv3ELGzcBlHLICkanbFn0%3D&reserved=0, terms & conditions are available at https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fpolicyandsecurity%2F&data=05%7C01%7Cbssfh%40leeds.ac.uk%7Ccb2d5343b58e42f0105308da72e92afa%7Cbdeaeda8c81d45ce863e5232a535b7cb%7C1%7C0%7C637948643022107048%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mzqlhJ%2BIN%2BILYJ6HIKc5HGXdRc3ve89hbrJ2ANq8I6s%3D&reserved=0

--

Best,
Pranav
--
Pranav Shah
Postdoctoral Research Fellow.

Division of Structural Biology,
Wellcome Trust Centre for Human Genetics,
University of Oxford,
Roosevelt Drive, Oxford OX3 7BN,
UK

--

Best,
Pranav
--
Pranav Shah
Postdoctoral Research Fellow.

Division of Structural Biology,
Wellcome Trust Centre for Human Genetics,
University of Oxford,
Roosevelt Drive, Oxford OX3 7BN,
UK

To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1

To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1