 |
 |
Hi, Most of this has now been answered by others: On 9 Oct 2007, at 20:17, Jeremy Bronson wrote: > Hi, > > I tried to get FSL working with XGrid during the summer, but wasn't > quite successful, and was recommended to use SGE in its place by > Steve and > others. I'm giving it another go, but I've run into a few questions > while setting up SGE. Hopefully they're easy enough to answer... > > -The SGE manual says that each host must have the same user account > names and passwords, which is the alternate to XGrid's nobody:nobody > permissions, but seems equally impractical. If SGE is really > designed to be > implemented in large grids, I'm not sure how it could scale beyond a > handful of managed machines. My question is whether this is how the > lab is configured at Oxford or any other site, or alternately are all > jobs submitted under a single username? > > -All scan data resides on an XServe, automounted via NFS. What kind of > permissions are necessary on this share? > > -Is anyone using the multiple queues available in SGE, or only using > one for large jobs? We use multiple queues - but that's just a matter of choice, and either option is easy to setup. fsl_sub just needs customising to reflect whatever you setup. Cheers. > My basic goal is to allow users to login, start multiple large jobs on > the grid, then log out and retrieve the results later. The FEAT > first-level analyses tend to tie up all the machines for a time, > so I'd love > to speed them up by running the jobs in parallel and in the > background. > > > Thanks in advance! > > Jeremy > > > > > > > On Tue, 14 Aug 2007 06:24:00 +0100, Steve Smith <[log in to unmask]> > wrote: >> Hi Jeremy, >> >> I agree with Andrew, most people seem much happier with SGE than with > >> XGrid, so if it's not too late I would consider that. >> >> Anyway - yes, hopefully FSL 4.0 should be fairly easy to setup with >> either (though much easier with SGE as that's what we have so should > >> need much less customising). >> >> First, see the brief intro at: >> http://www.fmrib.ox.ac.uk/fsl/fsl/downloading.html#sge >> So if the user runs any of these programs on a machine which can >> submit to the cluster then that should happen automatically, after >> the sysadmin has: >> >> Setup the central cluster-controlling script >> $FSLDIR/bin/fsl_sub >> >> This is a heavily commented shell script and hopefully should be >> reasonably easy to follow and customise..... >> >> >> So hopefully the user can just run FSL programs on their normal >> working machine, and if it's setup to be a cluster submission host, >> then whenever a "big" job is run that will automatically get sent to > >> the queue. >> >> Cheers, Steve. >> >> >> >> >> >> On 14 Aug 2007, at 05:59, Jeremy Bronson wrote: >> >>> Hi All, >>> >>> I'm the new administrator of a neuroimaging research lab, and I'm >>> working on getting FSL and other MRI-analysis tools running on >>> XGrid. I'm not yet intimately familiar with how FSL works, so I'm > >>> hoping there are others out there who have already figured out how > >>> to run it on a cluster, so I don't have to reinvent the wheel. >>> I've heard of several existing clusters that run FSL, albeit a >>> modified version. I'm hoping it can be done with the off-the-shelf > >>> version, perhaps it's now possible with the just-released 4.0? If > >>> anybody could point me in the direction of some specifics, I'd be >>> most grateful. >>> >>> We've got an XServe with RAID that houses all the data, and FSL is > >>> installed and configured on all host (agent) machines. Most users > >>> use the GUI, and manually point the tools to the appropriate data, > >>> so I assume they'll need to familiarize themselves with the > command- >>> line tools and specifying data directories on the CLI (or via >>> GridStuffer). I'm thinking that each machine will need the data >>> volume to be auto-mounted (NFS?) at startup with appropriate >>> directories having read/write access for the 'nobody' user. Does >>> this sound correct? >>> >>> Additionally, each tool that's part of the FSL package seems to >>> launch a number of other UNIX commands during analysis, to copy, >>> move and otherwise manipulate the result data. Will this confuse >>> XGrid, or will the job and all sub-commands run until the original > >>> command completes? (e.g. A complete FEAT analysis) >>> >>> Hopefully this isn't too difficult, and afterwards I'd like to take > >>> the time to post a HOWTO on macresearch.org or the like, so others > >>> might take advantage of the info. Thanks in advance to anyone who > >>> might be able to help. >>> >>> >>> Jeremy Bronson >>> Systems Administrator >>> Frey Research Lab >>> University of Oregon >> >> >> > > ---------------------------------------------------------------------- > -- >> --- >> Stephen M. Smith, Professor of Biomedical Engineering >> Associate Director, Oxford University FMRIB Centre >> >> FMRIB, JR Hospital, Headington, Oxford OX3 9DU, UK >> +44 (0) 1865 222726 (fax 222717) >> [log in to unmask] http://www.fmrib.ox.ac.uk/~steve >> > > ---------------------------------------------------------------------- > -- >> --- >> > > > > > > Hi, > > I tried to get FSL working with XGrid during the summer, but wasn't > quite successful, and was recommended to use SGE in its place by > Steve and > others. I'm giving it another go, but I've run into a few questions > while setting up SGE. Hopefully they're easy enough to answer... > > -The SGE manual says that each host must have the same user account > names and passwords, which is the alternate to XGrid's nobody:nobody > permissions, but seems equally impractical. If SGE is really > designed to be > implemented in large grids, I'm not sure how it could scale beyond a > handful of managed machines. My question is whether this is how the > lab is configured at Oxford or any other site, or alternately are all > jobs submitted under a single username? > > -All scan data resides on an XServe, automounted via NFS. What kind of > permissions are necessary on this share? > > -Is anyone using the multiple queues available in SGE, or only using > one for large jobs? > > My basic goal is to allow users to login, start multiple large jobs on > the grid, then log out and retrieve the results later. The FEAT > first-level analyses tend to tie up all the machines for a time, > so I'd love > to speed them up by running the jobs in parallel and in the > background. > > > Thanks in advance! > > Jeremy > > > > > > > On Tue, 14 Aug 2007 06:24:00 +0100, Steve Smith <[log in to unmask]> > wrote: >> Hi Jeremy, >> >> I agree with Andrew, most people seem much happier with SGE than with > >> XGrid, so if it's not too late I would consider that. >> >> Anyway - yes, hopefully FSL 4.0 should be fairly easy to setup with >> either (though much easier with SGE as that's what we have so should > >> need much less customising). >> >> First, see the brief intro at: >> http://www.fmrib.ox.ac.uk/fsl/fsl/downloading.html#sge >> So if the user runs any of these programs on a machine which can >> submit to the cluster then that should happen automatically, after >> the sysadmin has: >> >> Setup the central cluster-controlling script >> $FSLDIR/bin/fsl_sub >> >> This is a heavily commented shell script and hopefully should be >> reasonably easy to follow and customise..... >> >> >> So hopefully the user can just run FSL programs on their normal >> working machine, and if it's setup to be a cluster submission host, >> then whenever a "big" job is run that will automatically get sent to > >> the queue. >> >> Cheers, Steve. >> >> >> >> >> >> On 14 Aug 2007, at 05:59, Jeremy Bronson wrote: >> >>> Hi All, >>> >>> I'm the new administrator of a neuroimaging research lab, and I'm >>> working on getting FSL and other MRI-analysis tools running on >>> XGrid. I'm not yet intimately familiar with how FSL works, so I'm > >>> hoping there are others out there who have already figured out how > >>> to run it on a cluster, so I don't have to reinvent the wheel. >>> I've heard of several existing clusters that run FSL, albeit a >>> modified version. I'm hoping it can be done with the off-the-shelf > >>> version, perhaps it's now possible with the just-released 4.0? If > >>> anybody could point me in the direction of some specifics, I'd be >>> most grateful. >>> >>> We've got an XServe with RAID that houses all the data, and FSL is > >>> installed and configured on all host (agent) machines. Most users > >>> use the GUI, and manually point the tools to the appropriate data, > >>> so I assume they'll need to familiarize themselves with the > command- >>> line tools and specifying data directories on the CLI (or via >>> GridStuffer). I'm thinking that each machine will need the data >>> volume to be auto-mounted (NFS?) at startup with appropriate >>> directories having read/write access for the 'nobody' user. Does >>> this sound correct? >>> >>> Additionally, each tool that's part of the FSL package seems to >>> launch a number of other UNIX commands during analysis, to copy, >>> move and otherwise manipulate the result data. Will this confuse >>> XGrid, or will the job and all sub-commands run until the original > >>> command completes? (e.g. A complete FEAT analysis) >>> >>> Hopefully this isn't too difficult, and afterwards I'd like to take > >>> the time to post a HOWTO on macresearch.org or the like, so others > >>> might take advantage of the info. Thanks in advance to anyone who > >>> might be able to help. >>> >>> >>> Jeremy Bronson >>> Systems Administrator >>> Frey Research Lab >>> University of Oregon >> >> >> > > ---------------------------------------------------------------------- > -- >> --- >> Stephen M. Smith, Professor of Biomedical Engineering >> Associate Director, Oxford University FMRIB Centre >> >> FMRIB, JR Hospital, Headington, Oxford OX3 9DU, UK >> +44 (0) 1865 222726 (fax 222717) >> [log in to unmask] http://www.fmrib.ox.ac.uk/~steve >> > > ---------------------------------------------------------------------- > -- >> --- >> > > > > > > ------------------------------------------------------------------------ --- Stephen M. Smith, Professor of Biomedical Engineering Associate Director, Oxford University FMRIB Centre FMRIB, JR Hospital, Headington, Oxford OX3 9DU, UK +44 (0) 1865 222726 (fax 222717) [log in to unmask] http://www.fmrib.ox.ac.uk/~steve ------------------------------------------------------------------------ ---