There is certainly a significant difference in my hands, just a bit sub-linear.

$ time bash -c 'for i in Raw/*.dm4; do e2proc2d.py ${i} Micrographs/$(basename ${i%.dm4}.mrc) > /dev/null;done;'
> real    0m7.337s

$ time parallel -j 12 e2proc2d.py {} Micrographs/{/.}.mrc > /dev/null ::: Raw/*.dm4
> real    0m1.776s

There are 12 4k images in Raw, and the workstation has 8 physical cores and a RAID10 with 6 consumer hard drives. I dropped all the file caches before each command. (It's 0m2.107s with -j 8).

Best,
-da

On Wed, Dec 12, 2018 at 11:03 AM Ludtke, Steven J <[log in to unmask]> wrote:
I would just point out that format conversion requires virtually no CPU resources at all. It is entirely I/O limited, so, on most machines this sort of parallel operation will actually make the conversion take a bit longer, and lead to more disk fragmentation. If you are using SSDs or a RAID of SSDs, then the performance difference should be negligible, but you are still unlikely to see a significant speed boost this way.


--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <[log in to unmask]>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)



On Dec 12, 2018, at 12:17 PM, Daniel Asarnow <[log in to unmask]> wrote:

***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.
I am impatient, and I have many CPUs.

parallel -j 20 e2proc2d.py {} Micrographs/{/.}.mrc ::: Raw/*.hdf

(Using GNU parallel, similar to - but much better than - xargs, available for all Linux distros).

Best,
-da

On Wed, Dec 12, 2018 at 3:36 AM Sonya James <[log in to unmask]> wrote:
Hello,

I would like to convert Eman .hdf files to .mrc files. I can use;

e2proc2d.py folder/image.hdf image.mrc

to change one image - can I use it to convert all the images in a folder and what do I need to type (I'm a newbie to all this so need step by step instructions!)?

Thanks
Sonya

########################################################################

To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1


To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1




To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1