On Oct 5, 2006, at 12:46 PM, Ramon Bastiaans wrote:
> Well we already have a routing queue with a default very low
> walltime (1 minute), to force local users to specify a walltime.
> Thus improving scheduling and efficiency.
>
> Since grid jobs do not specify walltime but simply use the default
> walltime of queues they submit to, they would all end up being
> killed after 1 minute. Because of this right now we set all the
> jobs that come from the grid statically to 24 hours. This is not
> really a solution and causing problems with middleware upgrades too.
>
> What I would like is to 'offer/advertise' multiple longer and
> shorter fake/virtual queues so that VOs can choose which (length)
> to use, without actually making them in the PBS server.
>
Ramon,
You may be able to make use of the torque submit filter method:
http://www.clusterresources.com/wiki/doku.php?
id=torque:appendix:j_submit_filter
to edit the submitted job script before it is passed of to torque.
There is an example
on the page and one here that you could probably work from:
http://quattor.web.lal.in2p3.fr/packages/mpi/submit_filter.pl
However I think (I don't have a jobmanager script to hand to inspect
right now) that the job
manager does a
cat script << EOF > qsub -q <queuename>
EOF
so it may well fail if the queue does not exist. You may have to
change the job manager
instead provides the queue name via a
#PBS -q <queuename>
<JobManager created Script>
directive
Good luck. There will obviously be quite a bit of work to do in
providing sensible information
for your virtual queues.
Steve
> And then the idea would be that the queuename gets removed so that
> the default routing queue is used, while retaining the (default)
> walltime from that fake/virtual queue. Thus indirectly having the
> ability to submit gridjobs with various lengths and improving
> scheduling.
>
> I hope this explanation isn't too confusing. ;)
>
> Kind regards,
> - Ramon.
>
> Valentin Vidic wrote:
>> Maybe you can make use of routing queues in Torque:
>> create queue route
>> set queue route queue_type = Route
>> set queue route route_destinations = batch
>> set queue route resources_default.nodes=1
>> set queue route resources_default.walltime=3600
>> set queue route enabled = True
>> set queue route started = True
>> More info on:
>> http://www.clusterresources.com/torquedocs21/4.1queueconfig.shtml
>
> --
> ing. R. Bastiaans
>
> Systems Programmer / High Performance Computing & Visualisation /
> SARA Computing and Networking Services
> Kruislaan 415 PO Box 194613
> 1098 SJ Amsterdam 1090 GP Amsterdam
> ---
> There are really only three types of people:
>
> Those who make things happen, those who watch things happen
> and those who say, "What happened?"
|