hi steve et al,
this is only an action to tak epart on the fts sever?
if not. i do not have a
/opt/glite/etc/config/glite-file-tranfer-agent.cfg.xml
brian
n 31/01/06, Steve Traylen <[log in to unmask]> wrote:
> Hi
>
> I said I would dig out of thread on progress markers with dcache and FTS from
> the service challenge-tech list.
>
> Here it is.
>
> Steve
>
>
> ----- Forwarded message from Gavin Mccance <[log in to unmask]> -----
>
> From: Gavin Mccance <[log in to unmask]>
> Subject: Timeout on progress markers
> Date: Thu, 12 Jan 2006 15:19:19 +0100
> To: [log in to unmask], [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical contacts)" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all LCG Service Challenge Se)" <[log in to unmask]>
> Thread-Topic: Timeout on progress markers
>
> Hi Jane,
>
> I've been playing and got the bottom of the timeout issue. Looks like dCache
> 1.6.6 sends progress markers every 180 seconds. This was exactly what our
> inter-progress-marker timeout defaults to, which is why we aborted a
> significant fraction of the transfers! (the Globus server sends markers much
> more frequently). So the solution is to set the timeout higher than the
> default.
>
> Say 360 seconds:
>
> Add this:
>
> <transfer-agent-ts-urlcopy.TransferMarkersTimeout value="360"/>
>
> into the agent instance declaration for each of your agents, in the conf
> file:
>
> /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml
>
> or set to "0" to disable the timeout completely.
>
> Then rerun the configure script and restart the agent.
>
> cheers,
> gav
>
>
>
> ----- End forwarded message -----
> ----- Forwarded message from Jon Bakken <[log in to unmask]> -----
>
> From: Jon Bakken <[log in to unmask]>
> Subject: Re: Timeout on progress markers
> Date: Thu, 12 Jan 2006 08:58:14 -0600 (CST)
> To: Gavin Mccance <[log in to unmask]>
> Cc: [log in to unmask], [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical contacts)" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all LCG Service
> Challenge Se)" <[log in to unmask]>
> DomainKey-Status: non participant - Generated by CERN IT/IS DomainKeys v1.0
>
> Note that the dCache performance markers period can also be tuned as
> needed via the config batch file.
>
> --
> _|()// 7#3|23S //() |^14[3 11x3 127.0.0.1
> Jon A. Bakken [log in to unmask] (630) 840-4790
>
> On Thu, 12 Jan 2006, Gavin Mccance wrote:
>
> > Hi Jane,
> >
> > I've been playing and got the bottom of the timeout issue. Looks like dCache
> > 1.6.6 sends progress markers every 180 seconds. This was exactly what our
> > inter-progress-marker timeout defaults to, which is why we aborted a
> > significant fraction of the transfers! (the Globus server sends markers much
> > more frequently). So the solution is to set the timeout higher than the
> > default.
> >
> > Say 360 seconds:
> >
> > Add this:
> >
> > <transfer-agent-ts-urlcopy.TransferMarkersTimeout value="360"/>
> >
> > into the agent instance declaration for each of your agents, in the conf
> > file:
> >
> > /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml
> >
> > or set to "0" to disable the timeout completely.
> >
> > Then rerun the configure script and restart the agent.
> >
> > cheers,
> > gav
> >
>
> ----- End forwarded message -----
> ----- Forwarded message from "Zhenping (Jane) Liu" <[log in to unmask]> -----
>
> From: "Zhenping (Jane) Liu" <[log in to unmask]>
> Subject: Re: Timeout on progress markers
> Date: Thu, 12 Jan 2006 10:30:35 -0500
> To: Jon Bakken <[log in to unmask]>
> Cc: Gavin Mccance <[log in to unmask]>, [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical contacts)" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all LCG Service Challenge Se)" <[log in to unmask]>
> DomainKey-Status: non participant - Generated by CERN IT/IS DomainKeys v1.0
>
> Hi Jon and Gavin,
>
> I confirm that in the config of our dCache GridFTP door, the performance
> market period is set as 180 (default value). What new value do you
> suggest us to set? We can change that in config and restart.
>
> Thanks,
> Jane
>
> On Thu, 2006-01-12 at 09:58, Jon Bakken wrote:
> > Note that the dCache performance markers period can also be tuned as
> > needed via the config batch file.
>
> ----- End forwarded message -----
> ----- Forwarded message from Iris Wu <[log in to unmask]> -----
>
> From: Iris Wu <[log in to unmask]>
> Subject: Re: Timeout on progress markers
> Date: Thu, 12 Jan 2006 10:19:46 -0500
> To: Gavin Mccance <[log in to unmask]>
> Cc: "Zhenping (Jane) Liu" <[log in to unmask]>, [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical conta" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all L" <[log in to unmask]>
> DomainKey-Status: non participant - Generated by CERN IT/IS DomainKeys v1.0
>
> Hi Gavin,
>
> Just a few clarifications:
>
> Do you want us to change the content of
> /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml in CERN's FTS?
> I don't think we have right to do that. Or you want us to change it in
> our local FTS server and do the testing ourselves?
>
> At the same time, we may try what Jon Bakken suggested changing
> performance markers period in the dCache's config batch file.
>
>
> Cheers
> Iris
>
>
>
> On Thu, 2006-01-12 at 09:19, Gavin Mccance wrote:
> > Hi Jane,
> >
> > I've been playing and got the bottom of the timeout issue. Looks like dCache
> > 1.6.6 sends progress markers every 180 seconds. This was exactly what our
> > inter-progress-marker timeout defaults to, which is why we aborted a
> > significant fraction of the transfers! (the Globus server sends markers much
> > more frequently). So the solution is to set the timeout higher than the
> > default.
> >
> > Say 360 seconds:
> >
> > Add this:
> >
> > <transfer-agent-ts-urlcopy.TransferMarkersTimeout value="360"/>
> >
> > into the agent instance declaration for each of your agents, in the conf
> > file:
> >
> > /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml
> >
> > or set to "0" to disable the timeout completely.
> >
> > Then rerun the configure script and restart the agent.
> >
> > cheers,
> > gav
>
> ----- End forwarded message -----
> ----- Forwarded message from Gavin Mccance <[log in to unmask]> -----
>
> From: Gavin Mccance <[log in to unmask]>
> Subject: RE: Timeout on progress markers
> Date: Thu, 12 Jan 2006 16:23:23 +0100
> To: [log in to unmask]
> Cc: [log in to unmask], [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical contacts)" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all LCG Service Challenge Se)" <[log in to unmask]>
> Thread-Topic: Timeout on progress markers
>
> Hi Iris,
>
> I will make the changes at CERN. You may wish to make the same changes for
> your agents at BNL for the tier-1 / tier-2 transfers which I think were also
> affected by the same timeout problem.
>
> cheers,
> gav
>
> > -----Original Message-----
> > From: Iris Wu [mailto:[log in to unmask]]
> > Sent: 12 January 2006 16:20
> > To: Gavin Mccance
> > Cc: [log in to unmask]; [log in to unmask];
> > service-challenge-tech (LCG Service Challenges - technical
> > contacts); hep-service-sc-level2 (Contact list for level 2
> > support of all LCG Service Challenge Se)
> > Subject: Re: Timeout on progress markers
> >
> > Hi Gavin,
> >
> > Just a few clarifications:
> >
> > Do you want us to change the content of
> > /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml in
> > CERN's FTS?
> > I don't think we have right to do that. Or you want us to
> > change it in our local FTS server and do the testing ourselves?
> >
> > At the same time, we may try what Jon Bakken suggested
> > changing performance markers period in the dCache's config batch file.
> >
> >
> > Cheers
> > Iris
> >
> >
> >
> > On Thu, 2006-01-12 at 09:19, Gavin Mccance wrote:
> > > Hi Jane,
> > >
> > > I've been playing and got the bottom of the timeout issue.
> > Looks like
> > > dCache
> > > 1.6.6 sends progress markers every 180 seconds. This was
> > exactly what
> > > our inter-progress-marker timeout defaults to, which is why
> > we aborted
> > > a significant fraction of the transfers! (the Globus server sends
> > > markers much more frequently). So the solution is to set
> > the timeout
> > > higher than the default.
> > >
> > > Say 360 seconds:
> > >
> > > Add this:
> > >
> > > <transfer-agent-ts-urlcopy.TransferMarkersTimeout value="360"/>
> > >
> > > into the agent instance declaration for each of your agents, in the
> > > conf
> > > file:
> > >
> > > /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml
> > >
> > > or set to "0" to disable the timeout completely.
> > >
> > > Then rerun the configure script and restart the agent.
> > >
> > > cheers,
> > > gav
> >
> >
>
>
>
> ----- End forwarded message -----
> ----- Forwarded message from Iris Wu <[log in to unmask]> -----
>
> From: Iris Wu <[log in to unmask]>
> Subject: RE: Timeout on progress markers
> Date: Thu, 12 Jan 2006 10:37:55 -0500
> To: Gavin Mccance <[log in to unmask]>
> Cc: "zhenping(Jane) liu" <[log in to unmask]>, [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical contacts)" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all LCG Service Challenge Se)" <[log in to unmask]>
> DomainKey-Status: non participant - Generated by CERN IT/IS DomainKeys v1.0
>
> Hi Gavin,
>
> Thank you. I will make changes in our FTS for tier-1 / tier-2 transfers.
>
> Iris
>
>
>
> On Thu, 2006-01-12 at 10:23, Gavin Mccance wrote:
> > Hi Iris,
> >
> > I will make the changes at CERN. You may wish to make the same changes for
> > your agents at BNL for the tier-1 / tier-2 transfers which I think were also
> > affected by the same timeout problem.
> >
> > cheers,
> > gav
> >
> > > -----Original Message-----
> > > From: Iris Wu [mailto:[log in to unmask]]
> > > Sent: 12 January 2006 16:20
> > > To: Gavin Mccance
> > > Cc: [log in to unmask]; [log in to unmask];
> > > service-challenge-tech (LCG Service Challenges - technical
> > > contacts); hep-service-sc-level2 (Contact list for level 2
> > > support of all LCG Service Challenge Se)
> > > Subject: Re: Timeout on progress markers
> > >
> > > Hi Gavin,
> > >
> > > Just a few clarifications:
> > >
> > > Do you want us to change the content of
> > > /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml in
> > > CERN's FTS?
> > > I don't think we have right to do that. Or you want us to
> > > change it in our local FTS server and do the testing ourselves?
> > >
> > > At the same time, we may try what Jon Bakken suggested
> > > changing performance markers period in the dCache's config batch file.
> > >
> > >
> > > Cheers
> > > Iris
> > >
> > >
> > >
> > > On Thu, 2006-01-12 at 09:19, Gavin Mccance wrote:
> > > > Hi Jane,
> > > >
> > > > I've been playing and got the bottom of the timeout issue.
> > > Looks like
> > > > dCache
> > > > 1.6.6 sends progress markers every 180 seconds. This was
> > > exactly what
> > > > our inter-progress-marker timeout defaults to, which is why
> > > we aborted
> > > > a significant fraction of the transfers! (the Globus server sends
> > > > markers much more frequently). So the solution is to set
> > > the timeout
> > > > higher than the default.
> > > >
> > > > Say 360 seconds:
> > > >
> > > > Add this:
> > > >
> > > > <transfer-agent-ts-urlcopy.TransferMarkersTimeout value="360"/>
> > > >
> > > > into the agent instance declaration for each of your agents, in the
> > > > conf
> > > > file:
> > > >
> > > > /opt/glite/etc/config/glite-file-transfer-agents.cfg.xml
> > > >
> > > > or set to "0" to disable the timeout completely.
> > > >
> > > > Then rerun the configure script and restart the agent.
> > > >
> > > > cheers,
> > > > gav
> > >
> > >
>
> ----- End forwarded message -----
> ----- Forwarded message from Gavin Mccance <[log in to unmask]> -----
>
> From: Gavin Mccance <[log in to unmask]>
> Subject: RE: Timeout on progress markers
> Date: Thu, 12 Jan 2006 16:36:08 +0100
> To: [log in to unmask], [log in to unmask]
> Cc: [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical contacts)" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all LCG Service Challenge Se)" <[log in to unmask]>
> Thread-Topic: Timeout on progress markers
>
> Hi Jane,
>
> Maybe 30 seconds would be good? Just so we get a few markers before the
> transfer completes (with 180 seconds the transfer is sometimes complete
> before we get the first marker!). The hope is to start using this
> performance information in the monitoring of FTS - we already store it but
> don't yet do anything interesting with it.
>
> thanks,
> gav
>
> > -----Original Message-----
> > From: Zhenping (Jane) Liu [mailto:[log in to unmask]]
> > Sent: 12 January 2006 16:31
> > To: [log in to unmask]
> > Cc: Gavin Mccance; [log in to unmask];
> > service-challenge-tech (LCG Service Challenges - technical
> > contacts); hep-service-sc-level2 (Contact list for level 2
> > support of all LCG Service Challenge Se)
> > Subject: Re: Timeout on progress markers
> >
> > Hi Jon and Gavin,
> >
> > I confirm that in the config of our dCache GridFTP door, the
> > performance market period is set as 180 (default value).
> > What new value do you suggest us to set? We can change that
> > in config and restart.
> >
> > Thanks,
> > Jane
> >
> > On Thu, 2006-01-12 at 09:58, Jon Bakken wrote:
> > > Note that the dCache performance markers period can also be
> > tuned as
> > > needed via the config batch file.
> >
> >
>
>
>
> ----- End forwarded message -----
> ----- Forwarded message from "Zhenping (Jane) Liu" <[log in to unmask]> -----
>
> From: "Zhenping (Jane) Liu" <[log in to unmask]>
> Subject: RE: Timeout on progress markers
> Date: Thu, 12 Jan 2006 10:49:36 -0500
> To: Gavin Mccance <[log in to unmask]>
> Cc: [log in to unmask],
> "service-challenge-tech (LCG Service Challenges - technical contacts)" <[log in to unmask]>,
> "hep-service-sc-level2 (Contact list for level 2 support of all LCG Service Challenge Se)" <[log in to unmask]>
> DomainKey-Status: non participant - Generated by CERN IT/IS DomainKeys v1.0
>
> Hi Gavin,
>
> The performance marker period has been set to 30 seconds on our dCache
> GridFTP servers. Let's see whether this will improve things. Thanks.
>
> Jane
>
>
> On Thu, 2006-01-12 at 10:36, Gavin Mccance wrote:
> > Hi Jane,
> >
> > Maybe 30 seconds would be good? Just so we get a few markers before the
> > transfer completes (with 180 seconds the transfer is sometimes complete
> > before we get the first marker!). The hope is to start using this
> > performance information in the monitoring of FTS - we already store it but
> > don't yet do anything interesting with it.
> >
> > thanks,
> > gav
> >
> > > -----Original Message-----
> > > From: Zhenping (Jane) Liu [mailto:[log in to unmask]]
> > > Sent: 12 January 2006 16:31
> > > To: [log in to unmask]
> > > Cc: Gavin Mccance; [log in to unmask];
> > > service-challenge-tech (LCG Service Challenges - technical
> > > contacts); hep-service-sc-level2 (Contact list for level 2
> > > support of all LCG Service Challenge Se)
> > > Subject: Re: Timeout on progress markers
> > >
> > > Hi Jon and Gavin,
> > >
> > > I confirm that in the config of our dCache GridFTP door, the
> > > performance market period is set as 180 (default value).
> > > What new value do you suggest us to set? We can change that
> > > in config and restart.
> > >
> > > Thanks,
> > > Jane
> > >
> > > On Thu, 2006-01-12 at 09:58, Jon Bakken wrote:
> > > > Note that the dCache performance markers period can also be
> > > tuned as
> > > > needed via the config batch file.
> > >
> > >
>
> ----- End forwarded message -----
>
> --
> Steve Traylen
> [log in to unmask]
> http://www.gridpp.ac.uk/
>
|