Hello,

We have FSL 6.0.1 on a HPC cluster with access to an Nvidia v100 Volta GPU. I am consistently having issues trying to run the following eddy_cuda command line, which includes slice-to-volume correction:

eddy_cuda9.1 --imain=HARDI_b1200.nii --mask=my_hifi_b0_LR_mean_brain_betf0.2robust_mask.nii.gz --acqp=acqparams.txt --index=index.txt --bvecs=bvecs.bvec --bvals=bvals.bval --topup=my_topup_results --out=eddy_simple_of_topup_corrected_data_LR --niter=8 --fwhm=10,6,4,2,0,0,0,0 --repol --ol_type=both --mporder=8 --s2v_niter=8 --slspec=my_slspec.txt --verbose

It starts to run and there is a short burst of GPU activity within the first ten minutes and then it goes back to CPU and just keeps running but without any further updates/outputs. It seems to get stuck. The last time I tried it ran for a few days! I manually killed it in the end.

Any ideas, please? I don’t suppose there would be any way of implementing slice-to-volume correction on CPU, instead of GPU?

On entering the command line above, this is the verbose output in the terminal:

Reading images

Performing volume-to-volume registration

Running Register

...................Allocated GPU # 0...................

Loading prediction maker

Evaluating prediction maker model

error: chol(): failed to converge

Checking for outliers

Calculating parameter updates

Iter: 0, Total mss = 1.65129

Loading prediction maker

Evaluating prediction maker model

Checking for outliers

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 1, Total mss = 0.678474

Loading prediction maker

Evaluating prediction maker model

Checking for outliers

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 2, Total mss = 0.980167

Loading prediction maker

Evaluating prediction maker model

Checking for outliers

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 3, Total mss = 2.493

Loading prediction maker

Evaluating prediction maker model

Checking for outliers

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 4, Total mss = 4.92218

Loading prediction maker

Evaluating prediction maker model

Checking for outliers

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 5, Total mss = 4.92738

Loading prediction maker

Evaluating prediction maker model

Checking for outliers

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 6, Total mss = 5.33343

Loading prediction maker

Evaluating prediction maker model

Checking for outliers

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 7, Total mss = 5.62952

Setting scan 38 as shell shape-reference for shell 0 with b-value= 1200

Running sm.ApplyDWILocationReference

Checking shell alignment along PE-direction (running PostEddyAlignShellsAlongPE)

Aligning shells (running PostEddyAlignShells)

Performing slice-to-volume registration

Setting slice-to-volume order to 8

Running Register

Loading prediction maker

Evaluating prediction maker model

Calculating parameter updates

Iter: 0, Total mss = 11.2026

Loading prediction maker

Evaluating prediction maker model

What does “error: chol(): failed to converge” mean? Is this serious?

Any help would be hugely appreciated.

Many thanks.

Best wishes,

Hamied

Dr Hamied A Haroon, MInstP, MSc, PhD | Research Associate in Biomedical Magnetic Resonance Imaging

Division of Informatics, Imaging & Data Sciences | School of Health Sciences | Faculty of Biology, Medicine and Health

The University of Manchester