Print

Print


Dear Kelvin,

I have seen this thing before with the GTX1080.

Normally when you build a “fat” CUDA executable it will be able to run on any GPU with a compute capability that is in the range of capabilities built into the executable. Say for example you have a GPU with compute capability 3 and an executable with capabilities 2-5, when running that executable the system should simply pick the instructions suitable for 3 and execute those. This seems to work for all GPUs I have come across, except the GTX1080. With the GTX1080 it seems one has to build a separate executable with _only_ the compute capability suitable for the GTX1080 (6 something if I remember correctly).

As far as I understand (which might well be wrong) this is an Nvidia bug, and it is possible it has been resolved with CUDA 8.0. We are in the process of compiling a new release of eddy, so I will put on the list to build a special executable for the 1080.

Jesper

On 25 Aug 2017, at 00:34, Kelvin Sarink <[log in to unmask]<mailto:[log in to unmask]>> wrote:

Hello,

I have a problem with eddy_cuda. When I call it, it provides this error:

###Error, beginning###############################
Reading images
Performing volume-to-volume registration
Running Register
Entering EddyGpuUtils::LoadPredictionMaker

...................Allocated GPU # 0...................
thrust::system_error thrown in CudaVolume::common_assignment_from_newimage_vol after resize() with message: function_attributes(): after cudaFuncGetAttributes: invalid device function
terminate called after throwing an instance of 'thrust::system::system_error'
  what():  function_attributes(): after cudaFuncGetAttributes: invalid device function
Aborted (core dumped)
###Error, end##################################

I already searched in this mailing list for people who are had the same issue but I only found this one (https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=fsl;49bc910f.1606) and he has the same problem because his GPU was too old. The thing is I have a GTX1080 and since it is the a GPU of the current generation, it cannot be the problem (I guess?).

Additional debug info:
FSL version: 5.0.10 (centos6 version)
OS: Ubuntu 16.04.3 LTS
Cuda: release 7.5, V7.5.17
Parameters of eddy command: fsl5.0.10-eddy_cuda --imain=mergedDWI.nii.gz --mask=BETB0_mask.nii.gz --acqp=acqp --index=index --bvecs=bvec.bvec --bvals=bval.bval --field=Fieldmap_Hz --out=eddy_tit_cpu
Nvidia driver: 375.82
nvidia-smi and nvidia-smi -a outputs are attached


I would appreciate any kind of help. If somebody needs additional debug info, just ask and I will try to get it :)



With kind regards,

Kelvin Sarink
<nvidia-smi.txt><nvidia-smi_all.txt>