Hi Steve,
Brian responded to Matthew's question privately. The production FTS3 instance at RAL has been overloaded for the past few days, we think due a significantly higher number than normal of submitted transfers from an LHC VO and non-LHC VO. This is the first time this has happened.
There's also another issue that is likely to affect non-LHC VOs who don't use FTS much - it can take some for the auto-tuner to gather statistics before making 'sensible' choices for the numbers of active transfers etc. This morning we set a fixed number of transfers for srm-snoplus.gridpp.rl.ac.uk, and this has significantly improved the throughput. We also did this for another non-LHC VO and it also made a big difference.
Regards,
Andrew.
________________________________________
From: Testbed Support for GridPP member institutes [[log in to unmask]] on behalf of Stephen Jones [[log in to unmask]]
Sent: Thursday, September 17, 2015 1:21 PM
To: [log in to unmask]
Subject: Re: glite file transfers to RAL
Hi Matt,
Our network can theoretically transfer all that data in an hour or two,
and the DPM is nowhere near saturated.
John Bland (Liverpool) suggests FTS may be overloaded.
Does this happen at other sites or has it happened before?
Steve
On 09/16/2015 05:02 PM, Matthew Mottram wrote:
> Hi,
>
> I’m not sure if this is the correct mailing list to send this to,
> please point me in the correct direction if needed. Today I submitted
> a request for around 10k file transfers from Liverpool to RAL to the
> FTS service at https://lcgfts3.gridpp.rl.ac.uk:8443 (the files are on
> the order of 100 MB each). In the past I’ve seen these kinds of
> transfers run very quickly, however that’s not the case today (after
> around 2.5 hours around 300 files have been transferred). SNO+ plan
> to use the FTS to schedule our raw data transfers, however, it’s not
> clear to me whether we should go ahead with this until we better
> understand how the scheduler works. I would have thought 2.5 hours
> would have been sufficient time to transfer much more than the ~30GB
> from Liverpool to RAL, is this really some bandwidth limit or is it
> possible the transfers are too low in priority (our default priority
> is 3, Im not sure how to interpret or change this) and are simply not
> being activated?
>
> If anyone can provide more information and point me to someone to
> discuss what SNO+ will need to do to use the FTS for long term raw and
> production data then I’d be very grateful.
>
> Cheers,
> Matt
> -------------------------------------------------
> Matthew Mottram
> School of Physics and Astronomy
> Queen Mary, University of London
> -------------------------------------------------
>
--
Steve Jones [log in to unmask]
Grid System Administrator office: 220
High Energy Physics Division tel (int): 43396
Oliver Lodge Laboratory tel (ext): +44 (0)151 794 3396
University of Liverpool http://www.liv.ac.uk/physics/hep/
|