 |
 |
Hi Josh, I am not sure about this issue, Can you check that GPUs are set to exclusive mode and there are not several jobs assigned to the same GPU? if that is not the problem, can you share all the log files? I can try to investigate. What FSL version / GPU / CUDA version are you using? Moises On Fri, 5 Jul 2019 at 04:45, Josh Robinson <[log in to unmask]> wrote: > Hi all, > > We recently started having a problem with bedpostx_gpu where it will print > some (but not all) expected output files that are empty or near empty. It > seems that one of the preprocessing job and the first gpu job error out > immediately while the second gpu job runs to completion, and we are unsure > why this is. I checked a logfile and found the following: > > Error: setup_randoms_kernel: out of memory > > We know this is routine in the cuda version of xfibres but we are unsure > of how to fix it and get it back up and running. This had originally > started after we updated a cuda library a few weeks ago, so we figure it > has something to do with that. However, we updated the shell path to the > new library and remounted gpu nodes and we are unsure of what else to try. > > Has anyone else had this error or know how it can be fixed? > > Thank you, > > Josh > > ######################################################################## > > To unsubscribe from the FSL list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1 > ######################################################################## To unsubscribe from the FSL list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1