Print

Print


Hello,

We have FSL 6.0.1 on a HPC cluster with access to an Nvidia v100 Volta GPU. I am consistently having issues trying to run the following eddy_cuda command line, which includes slice-to-volume correction:

eddy_cuda9.1 --imain=HARDI_b1200.nii --mask=my_hifi_b0_LR_mean_brain_betf0.2robust_mask.nii.gz --acqp=acqparams.txt --index=index.txt --bvecs=bvecs.bvec --bvals=bvals.bval --topup=my_topup_results --out=eddy_simple_of_topup_corrected_data_LR --niter=8 --fwhm=10,6,4,2,0,0,0,0 --repol --ol_type=both --mporder=8 --s2v_niter=8 --slspec=my_slspec.txt --verbose

It starts to run and there is a short burst of GPU activity within the first ten minutes and then it goes back to CPU and just keeps running but without any further updates/outputs. It seems to get stuck. The last time I tried it ran for a few days! I manually killed it in the end.

Any ideas, please? I don't suppose there would be any way of implementing slice-to-volume correction on CPU, instead of GPU?

On entering the command line above, this is the verbose output in the terminal:

Reading images
Performing volume-to-volume registration
Running Register
 ...................Allocated GPU # 0...................
Loading prediction maker
Evaluating prediction maker model
 error: chol(): failed to converge

 error: chol(): failed to converge
 Checking for outliers
Calculating parameter updates
Iter: 0, Total mss = 1.65129
Loading prediction maker
Evaluating prediction maker model
Checking for outliers
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 1, Total mss = 0.678474
Loading prediction maker
Evaluating prediction maker model
Checking for outliers
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 2, Total mss = 0.980167
Loading prediction maker
Evaluating prediction maker model
Checking for outliers
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 3, Total mss = 2.493
Loading prediction maker
Evaluating prediction maker model
Checking for outliers
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 4, Total mss = 4.92218
Loading prediction maker
Evaluating prediction maker model
Checking for outliers
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 5, Total mss = 4.92738
Loading prediction maker
Evaluating prediction maker model
Checking for outliers
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 6, Total mss = 5.33343
Loading prediction maker
Evaluating prediction maker model
Checking for outliers
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 7, Total mss = 5.62952
Setting scan 38 as shell shape-reference for shell 0 with b-value= 1200
Running sm.ApplyDWILocationReference
Checking shell alignment along PE-direction (running PostEddyAlignShellsAlongPE)
Aligning shells (running PostEddyAlignShells)
Performing slice-to-volume registration
Setting slice-to-volume order to 8
Running Register
Loading prediction maker
Evaluating prediction maker model
Calculating parameter updates
Iter: 0, Total mss = 11.2026
Loading prediction maker
Evaluating prediction maker model


What does "error: chol(): failed to converge" mean? Is this serious?

Any help would be hugely appreciated.

Many thanks.

Best wishes,

Hamied


Dr Hamied A Haroon, MInstP, MSc, PhD | Research Associate in Biomedical Magnetic Resonance Imaging
Division of Informatics, Imaging & Data Sciences | School of Health Sciences | Faculty of Biology, Medicine and Health
The University of Manchester
Room G.522 | Centre for Imaging Sciences | Stopford Building | Oxford Road | Manchester M13 9PT | England | United Kingdom


########################################################################

To unsubscribe from the FSL list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1