Hi Lekshmi,
bedpostx_preproc_gpu is just the name of the job. It executes the binary
file $FSLDIR/bin/split_parts_gpu.
Yes, you only need to modify bedpostx_gpu to adapt it to SLURM. This script
support job submissions for two kind of queue systems: TORQUE (PBS)
and Grid Engine using fsl_sub.
You can take these submission commands as reference, but you will need to
modify them.
It seems that you do not have the CUDA binaries files.
What distribution are you using ?
The binaries are only included in the CentOS 6 distribution,
but you can compile the CUDA source code:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind1512&L=FSL&F=&S=&P=180733
Moises.
On 22 December 2015 at 12:32, Lekshmi <[log in to unmask]> wrote:
> Dear all,
>
> Currently, I am in the process of porting and optimising FSL for use
> with our new supercomputer which uses Easy build for software installation
> and SLURM as the resource manager.
>
> I was able to install FSL 5.0.9 using easy build and added the new
> bedpostx patch (
> http://fsl.fmrib.ox.ac.uk/fsldownloads/patches/bedpostx-patch-fsl-5.0.9.tar.gz
> ).
>
> I came across some issues when customising the bedpostx_gpu script.
>
> 1. in the following code (line number:251), the preprocessing job name is
> bedpostx_preproc_gpu. But I could see only bedpostx_postproc_gpu under the
> ..../FSL/5.0.9/fsl/bin/ directory. Where can I find this script?
>
> torque_command="qsub -V $queue -l nodes=1:ppn=1:gpus=1,walltime=00:40:00
> -N bedpostx_prepr\
> oc_gpu -o ${subjdir}.bedpostX/logs -e ${subjdir}.bedpostX/logs"
>
> Following post processing code looks good in terms of the job name.
> torque_command="qsub -V $queue -l nodes=1:ppn=1:gpus=1,walltime=00:40:00
> -N bedpostx_postproc_g\
> pu -o ${subjdir}.bedpostX/logs -e ${subjdir}.bedpostX/logs -W
> depend=afterokarray:$bedpostid"
>
> 2. #280
>
> echo "${FSLDIR}/bin/xfibres_gpu --data=${subjdir}.bedpostX/data_$part
> --mask=$subjdir.bedp\
> ostX/nodif_brain_mask -b ${subjdir}.bedpostX/bvals -r
> ${subjdir}.bedpostX/bvecs --forcedir --logdi\
> r=$subjdir.bedpostX/diff_parts/data_part_$partzp $gopts ${subjdir} $part
> $njobs $nvox" >> ${subjdi\
> r}.bedpostX/commands.txt
>
> Here there is a reference to ${FSLDIR}/bin/xfibres_gpu. But I have only
> ${FSLDIR}/bin/xfibres
>
>
> 3. Hope I need to do the changes only in bedpostx_gpu script for
> optimising it for SLURM. I got confused after seeing the reference to
> fsl_sub in the following line:
>
> #SGE
> bedpostid=`${FSLDIR}/bin/fsl_sub $queue -l
> ${subjdir}.bedpostX/logs -N bedpostx_gpu -j $splitID\
> -t ${subjdir}.bedpostX/commands.txt`
>
> Could some one explain to me which all scripts needs to be changed for
> SLURM?
>
> Many thanks in advance,
> Lekshmi
>
