Hi,

 > 1) how does relion generate particles.mrcs file and how does it copy 
it from storage to scratch directory? The facility managers of the hpc 
on which I am working followed the storage to scratch transfer and they 
found it was quite slower compared to the actual capacity of the two 
storages and the wired connection to transfer data (i.e. copying relion 
mrcs particles is slower than copying a different file with a comparable 
size). It looks like relion could not efficiently exploit hpc 
potentiality. This issue causes a long copying process (40 minutes for 
44 Gb) before the lassification can start.

RELION copies particles slice by slice (because previous subset
selection jobs could have excluded some particles). This generates a
lot of small IO operations (only hundreds of KBs each). Some
network file system is very poor at it.
The current implementation can use only a single thread for the copy.

Sometimes it is faster to copy particles outside RELION
using rsync or tools dedicated for the particular file system.
In this case, you need to modify paths in a STAR file
before and after the job. Someone in this mailing list
wrote such a script but I cannot remember who it was.

 > 2) when a job that uses scratch starts, relion look for the free 
available space on the directory that I indicated as a scratch in order 
to decide whether this space is enough for the whole set of particles or 
not. Is that right? I was wondering if there is a flag to pass to relion 
in order to tell the software to use only a percentage of the available 
disk space.

You cannot use percentages but we have this option:

  --keep_free_scratch (10) : Space available for copying particle stacks 
(in Gb)

Best regards,

Takanori Nakane

On 2023/10/31 8:43, Omar De Bei wrote:
> Dear all,
> 
> I would greatly appreciate it if you could provide me with some specific details regarding the use of scratch in Relion.
> 
> Indeed, I am working with a huge dataset. The disk quota that I have on my hpc account only allows me to keep data saved on slow hard drives and to launch RELION from there.
> 
> It was a game changer to me using scratch command on relion to copy particles on the fast hard drive of the processing node before classification.
> 
> However, I have some questions that I would like to submit to you:
> 
> 1) how does relion generate particles.mrcs file and how does it copy it from storage to scratch directory? The facility managers of the hpc on which I am working followed the storage to scratch transfer and they found it was quite slower compared to the actual capacity of the two storages and the wired connection to transfer data (i.e. copying relion mrcs particles is slower than copying a different file with a comparable size). It looks like relion could not efficiently exploit hpc potentiality. This issue causes a long copying process (40 minutes for 44 Gb) before the lassification can start.
> 
> 2) when a job that uses scratch starts, relion look for the free available space on the directory that I indicated as a scratch in order to decide whether this space is enough for the whole set of particles or not. Is that right? I was wondering if there is a flag to pass to relion in order to tell the software to use only a percentage of the available disk space. Indeed, most of the time the classification job that I launch will go on a computing node that is shared with other users. Sometimes, those users need to copy file on the fast hard drives on the node as well. Therefore I could not occupy the entire disk quota of the node. In case a flag exists, this will safe both my job and the one of other users of the same node.
> 
> Thank you in advance for your help.
> Best wishes,
> Omar
> 
> ########################################################################
> 
> To unsubscribe from the CCPEM list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCPEM, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/

########################################################################

To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1

This message was issued to members of www.jiscmail.ac.uk/CCPEM, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/