Alastair Dewhurst wrote:
> Hi
>
> To re-iterate what Graeme said, we can't throttle it through the FTS as
> that doesn't have the control we require. Technically we could max the
> link with ~6 concurrent file transfers to QMUL but most of the time we
> can run more without problems. Throttling the FTS to prevent these
> peaks in activity would cause a greatly reduced overall performance.
>
> I spoke to Ian about CVMFS and he thinks that the amount of usage that
> should put on the network would be comparable to a normal students use
> so routing that over the college network would probably be sensible. Do
> you have any stats for the network traffic on your CVMFS squid?
>
CVMFS squid is the same as the frontier cache squid -
frontiercache.esc.qmul.ac.uk. I don't directly monitor network traffic
through it, but the usual squid monitoring (which isn't linked on
http://www.gridpp.ac.uk/wiki/Links_Monitoring_pages but should be) may
tell you.
Chris
> Alastair
>
>
> On 16 Nov 2010, at 14:13, Christopher J.Walker wrote:
>
>> John Gordon wrote:
>>> So FTS should control the rate but is the issue that there are other
>>> streams of transfer in addition to FTS?
>>
>> Job submission, lcg-cr (talking to the LFC), lcg-gt (talking to the
>> BDII), pilots talking home, conditions database, CVMFS etc.
>>
>> These are all relatively small bandwidth, but if they fail, jobs start
>> failing.
>>
>> One option we might have is to direct our bulk traffic over our
>> dedicated link, and everything else over the college link - but that
>> requires college to update the router (in their copious free time).
>>
>> Chris
>>
>>
>>
>>>> -----Original Message-----
>>>> From: Testbed Support for GridPP member institutes [mailto:TB-
>>>> [log in to unmask]] On Behalf Of Christopher J.Walker
>>>> Sent: 16 November 2010 12:27
>>>> To: [log in to unmask]
>>>> Subject: Re: GridFTP ToS (and Traffic shaping/policing)
>>>>
>>>> John Gordon wrote:
>>>>> Can we not lobby ATLAS again to accept that the traffic
>>>>> to a site should not just be based on the amount of disk they have
>>>>> but some function of the network bandwidth and disk.
>>>> The problem is not the volume of data ATLAS wish to transfer to us. The
>>>> problem is that traffic from RAL peaks at greater than 1Gbit. You can't
>>>> squeeze more than 1Gbit through a 1Gbit pipe, so traffic will get
>>>> dropped. The question is which packets routers should drop.
>>>>
>>>> We don't really care too much about the bulk data traffic, but job
>>>> submission, lcg-cr etc packets get dropped too - and that seems to
>>>> cause
>>>> these applications a problem.
>>>>
>>>>> One obviously
>>>>> can't just change from disk to bandwidth or a site with a fat pipe
>>>> > and not much disk would suffer a different fate.
>>>>
>>>> Indeed.
>>>>
>>>> Chris
>>>>
>>>>
>>>>>> -----Original Message-----
>>>>>> From: Testbed Support for GridPP member institutes [mailto:TB-
>>>>>> [log in to unmask]] On Behalf Of Christopher J.Walker
>>>>>> Sent: 15 November 2010 19:37
>>>>>> To: [log in to unmask]
>>>>>> Subject: GridFTP ToS (and Traffic shaping/policing)
>>>>>>
>>>>>> After the spring reprocessing, ATLAS maxed out QMUL's link to Janet
>>>> for
>>>>>> a couple of weeks. This seemed to cause job unreliability -
>>>> presumably
>>>>>> they were unable to phone home in the face of large amounts of
>>>> packet loss.
>>>>>> With the recent reprocessing and/or Atlas's move to pd2p, they are
>>>> again
>>>>>> transferring lots of data to QMUL, and again maxing out our link.
>>>>>>
>>>>>> I've ended up implementing traffic policing on our SE. What we do is
>>>>>> drop traffic to gridFTP's data transfer port range when we fill 75%
>>>> of
>>>>>> the link. This causes TCP/IP to backoff - so leaving some space on
>>>> the
>>>>>> link.
>>>>>>
>>>>>> We are currently only dropping incoming packets in the globus
>>>> portrange.
>>>>>> This presumably includes ack packets for outgoing traffic which
>>>> must
>>>>>> be a bad thing. We'll also need to drop outgoing traffic too I
>>>> think[1].
>>>>>> One of the suggestions in the Linux Advanced Routing & Traffic
>>>> Control
>>>>>> howto (http://lartc.org/) is to drop based on the TOS IP headers.
>>>> This
>>>>>> doesn't seem to be set in gridftp packets - (though it does in scp
>>>>>> packets).
>>>>>>
>>>>>> TOS would seem an obvious thing to set - it would enable us and/or
>>>> Janet
>>>>>> to drop the bulk data in preference to the interactive data. It
>>>> would
>>>>>> also mean I can tell our network team to drop bulk packets rather
>>>> than a
>>>>>> complicated host/port range. Does anyone know why globus don't set
>>>> it?
>>>>>> Chris
>>>>>>
>>>>>>
>>>>>> [1] Indeed globus provide a script to do this at
>>>>>> http://www.globus.org/toolkit/data/gridftp/bwlimit.html
|