Hi Brian
Sorry about the tardy reply...
On 8 Jun 2006, at 15:40, brian davies wrote:
> hi all, but really aimed at graeme but someone else may know...
> using graeme's transfer script i get jobs running ( woo hoo!!) but it
> then seems to hang with a couple of files in waiting mode. if i then
> cancel the transfer the filetransfer.py executable then finishes and
> gives me a transfer time. I noticed that the script gives times for
> each file. is rate ( ie the Mbps ):
> 1) (SUM of file transfered)/ (difference in filetransfer.py start
> to end times)
> or
> 2) SUM of ( (file size)/(difference in active and done times) ).
>
> i think it is the first. Is it possible to get the second?
What is actually printed is SUM/(last DONE time - filetransfer.py
start time)
I agree that 2 is better.
In fact Jamie is working on a major modification of the script to
that it will manage multiple transfers (and also transfer for a set
time, rather than just a number of files), and within that we
identified (2) as a better way to calculate the bandwidth. In detail
this will be:
Sum / (last DONE time - first ACTIVE time)
> both would be useful as the second shows actual file transfer times
> and the difference between the two is the overhead for the transfer.
> cheers
> brian
It gets a bit more complex for multiple transfers, because one has to
have a certain number of files always waiting in the wings in a
PENDING state, to ensure the bandwidth gets used properly[1].
Unfortunately there's not really enough information available through
FTS to give a proper notion of overhead. But we could certainly print
both calculations of bandwidth on the job.
Cheers
Graeme
[1] It's even more complex really, because if there are other
transfers which are active on the channel then TCP will expand to
fill the pipe, even while the non-data transfer parts of an
individual transfer (srm / gridftp negotiation), so, at a higher
level, one could claim that there isn't any real overhead at all.
--
Dr Graeme Stewart - http://wiki.gridpp.ac.uk/wiki/User:Graeme_stewart
GridPP DM Wiki - http://wiki.gridpp.ac.uk/wiki/Data_Management
|