Looks like the action we took on Friday has made a difference. 900k more LHC jobs in UK for June than on Friday night. 5M more for the whole Grid, cf average 1Mjobs/day. UK Total is 29% higher while the sites who were furthest behind like Manchester, Lancaster ECDF now have 3-5 times as many jobs seen in June. Unfortunately the gantt chart in the portal doesn't seem to have been updated since Friday so I can't tell when each site last published. Some sites njobs hasn't increased at all since Friday. Can people take a look at their logs and let me know what is happening. Don't send to this list please. Thanks, John From: Testbed Support for GridPP member institutes [mailto:[log in to unmask]] On Behalf Of Alessandra Forti Sent: 24 June 2011 15:44 To: [log in to unmask] Subject: Re: APEL data not going in PS According to my logs though yesterday we have started to publish again after 15 days. cheers alessandra On 24/06/2011 15:39, Alessandra Forti wrote: Hi John, nope the first ticket was a hardware problem at RAL. Cristina del Cano Novales 2011-05-23 12:10 Public Diary: Hi Alessandra, We've had some hardware issues with the APEL consumer. The service has now been moved to a different machine while we fix the problems. Can you try to publish and let me know if it still fails? Cheers, Cristina The last part concerns minor corrections we applied since I had to republish a big chunk of data anyway. cheers alessandra On 24/06/2011 15:27, John Gordon wrote: a) we have problems with peaks around 0300 for a while. Spreading out when sites published seemed to help that until this happened. b) my reading of your older ticket is that the problem was at your end. John -----Original Message----- From: Alessandra Forti [mailto:[log in to unmask]] Sent: 24 June 2011 15:22 To: Testbed Support for GridPP member institutes Cc: Gordon, John (STFC,RAL,ESC) Subject: Re: APEL data not going in Last week I opened this ticket. https://ggus.org/ws/ticket_info.php?ticket=71647 I'm not sure the problem is only due to PIC republishing everything as this is the second ticket I open in less than a month. The other being this https://ggus.eu/ws/ticket_info.php?ticket=70801 cheers alessandra On 24/06/2011 15:12, John Gordon wrote: We just discovered that PIC have been republishing all their historical work, millions of records and blocked everyone else. We are taking remedial action but don't want to make things worse on a Friday. John -----Original Message----- From: Testbed Support for GridPP member institutes [mailto:[log in to unmask]] On Behalf Of J Coles Sent: 24 June 2011 15:07 To: [log in to unmask]<mailto:[log in to unmask]> Subject: Re: APEL data not going in Hi Stephen On Tuesday several site admins reported problems with timeouts. This could be an extension/worsening of whatever the underlying problem was then. Have you raised a support ticket against APEL? Thanks, Jeremy On 24 Jun 2011, at 14:57, Stephen Jones wrote: Our APEL data's not going in: http://goc-accounting.grid-support.ac.uk/rss/UKI-NORTHGRID-LIV-HEP_Sync.html But neither is anyone else's, for several days. Does anybody know what's going on? Steve -- Steve Jones [log in to unmask]<mailto:[log in to unmask]> System Administrator office: 220 High Energy Physics Division tel (int): 42334 Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334 University of Liverpool http://www.liv.ac.uk/physics/hep/