Appear to be getting through now.
Alison
-----Original Message-----
From: Testbed Support for GridPP member institutes [mailto:[log in to unmask]] On Behalf Of Stephen Jones
Sent: 08 April 2013 14:26
To: [log in to unmask]
Subject: Re: APEL Not Working
Oops - spoke too soon. Connection Timeout now.
Busyness?
Steve
On 04/08/2013 02:17 PM, Stephen Jones wrote:
> Confirmed - I presently see records rolling in.
>
> Steve
>
>
>
> On 04/08/2013 01:52 PM, Alison Packer wrote:
>> Hi all,
>> Sorry for late reply - I checked the system this morning and was
>> fooled into thinking it was OK as the log file for receiving data had
>> recently updated. Bad timing as things had frozen soon after when
>> the daily processing finished and the table is optimized. I've
>> restarted the service and data is now being loaded again. Please
>> don't do any "gap" publishing for the moment as we still have a large
>> backlog of data. But records are now being loaded.
>>
>> Alison
>>
>> -----Original Message-----
>> From: Testbed Support for GridPP member institutes
>> [mailto:[log in to unmask]] On Behalf Of Stephen Jones
>> Sent: 08 April 2013 12:04
>> To: [log in to unmask]
>> Subject: Re: APEL Not Working
>>
>> Yes - he have no luck either.
>>
>> The untransferred data grows each day.
>>
>> Soon it will hard to get back on top of it.
>>
>> Steve
>>
>>
>>
>> On 04/08/2013 11:50 AM, Alessandra Forti wrote:
>>> PS the whole UK is actually 10 days behind.
>>>
>>>
>>> On 08/04/2013 11:49, Alessandra Forti wrote:
>>>> Hi Alison,
>>>>
>>>> we still have problems. I had to reduce the number of records we
>>>> publish at the time because of memory problems we never had and I'm
>>>> republishing using gap method but it still is not transfering any
>>>> data since this morning at 7 am.
>>>>
>>>> cheers
>>>> alessandra
>>>>
>>>> On 06/04/2013 20:44, Alison Packer wrote:
>>>>> Hi All,
>>>>>
>>>>> There has been a big backlog of publishing since the break at the
>>>>> Easter weekend, really sorry for the inconvenience. I think
>>>>> processing is back to normal now, if a little tardy, but I am
>>>>> monitoring it. Please raise a GGUS ticket if you still have
>>>>> issues publishing next week.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Alison
>>>>>
>>>>> *From:*Testbed Support for GridPP member institutes
>>>>> [mailto:[log in to unmask]] *On Behalf Of *Govind Songara
>>>>> *Sent:* 05 April 2013 10:59
>>>>> *To:* [log in to unmask]
>>>>> *Subject:* Re: APEL Not Working
>>>>>
>>>>> It works now when publishing the Gap.
>>>>> I had some error when Cron run yesterday.
>>>>>
>>>>> On Fri, Apr 5, 2013 at 10:50 AM, John Hill <[log in to unmask]
>>>>> <mailto:[log in to unmask]>> wrote:
>>>>>
>>>>> I've seen the same for the last couple of days. It was also broken
>>>>> over the Easter weekend - which was reported in a couple of
>>>>> broadcasts on Tuesday. I haven't seen an acknowledgement of this
>>>>> particular problem, though it does appear to be a problem at the
>>>>> APEL server, as most (all?) sites are reporting a problem with the
>>>>> org.apel.APEL-Sync Nagios probe.
>>>>>
>>>>> John
>>>>>
>>>>>
>>>>>
>>>>> On 05/04/2013 10:41, emyr.james wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> The APEL publishing cronjob is failing. It's timing out trying to
>>>>> connect to the server. Is this is a problem at my end or the APEL
>>>>> server end ?
>>>>> Note that I have this set in the config...
>>>>>
>>>>> <ConsumerTimeout>18000000</ConsumerTimeout>
>>>>>
>>>>> I.e. a timeout of 18,000 seconds (5 hrs)
>>>>>
>>>>> Anyone got any ideas ?
>>>>>
>>>>> The error snippet is below.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Emyr
>>>>>
>>>>>
>>>>> Thu Apr 4 11:02:24 UTC 2013: apel-publisher -
>>>>> ====================================
>>>>> Thu Apr 4 11:02:24 UTC 2013: apel-publisher - Synchronisation
>>>>> data check Thu Apr 4 11:02:24 UTC 2013: apel-publisher -
>>>>> ====================================
>>>>> Thu Apr 4 11:02:24 UTC 2013: apel-publisher - Finding all records
>>>>> in local database since the last successful publish timestamp :
>>>>> 2013-04-03
>>>>> 06:53:12
>>>>> Thu Apr 4 11:02:25 UTC 2013: apel-publisher - Record/s found in
>>>>> local
>>>>> database: 4555
>>>>> Caught: javax.jms.JMSException: Connection reset
>>>>> javax.jms.JMSException: Connection reset
>>>>> at
>>>>> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSu
>>>>> pp
>>>>> ort.java:62)
>>>>>
>>>>> at
>>>>> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessag
>>>>> eC
>>>>> onsumer.java:458)
>>>>>
>>>>> at
>>>>> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessag
>>>>> eC
>>>>> onsumer.java:577)
>>>>>
>>>>> at org.glite.apel.publisher.AccountChecker.execute(Unknown
>>>>> Source)
>>>>> at
>>>>> org.glite.apel.publisher.AccountChecker.countArchivedRecords(Unkno
>>>>> wn
>>>>> Source)
>>>>> at
>>>>> org.glite.apel.publisher.AccountManager.chkArchivedTuples(Unknown
>>>>> Source)
>>>>> at org.glite.apel.publisher.AccountManager.run(Unknown Source)
>>>>> at
>>>>> org.glite.apel.publisher.ApelPublisher.runJoinProcessor(Unknown
>>>>> Source)
>>>>> at org.glite.apel.publisher.ApelPublisher.run(Unknown Source)
>>>>> at org.glite.apel.publisher.ApelPublisher.main(Unknown
>>>>> Source) Caused by: java.net.SocketException: Connection reset
>>>>> at java.net.SocketInputStream.read(SocketInputStream.java:185)
>>>>> at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
>>>>> at sun.security.ssl.InputRecord.read(InputRecord.java:480)
>>>>> at
>>>>> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:850)
>>>>> at
>>>>> sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:807)
>>>>> at sun.security.ssl.AppInputStream.read(AppInputStream.java:94)
>>>>> at
>>>>> org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpB
>>>>> uf
>>>>> feredInputStream.java:50)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport
>>>>> .j
>>>>> ava:576)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpB
>>>>> uf
>>>>> feredInputStream.java:58)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport
>>>>> .j
>>>>> ava:561)
>>>>>
>>>>> at java.io.DataInputStream.readInt(DataInputStream.java:387)
>>>>> at
>>>>> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireForm
>>>>> at
>>>>> .java:269)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTran
>>>>> sp
>>>>> ort.java:227)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.
>>>>> java:219)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.ja
>>>>> va:202)
>>>>>
>>>>> at java.lang.Thread.run(Thread.java:679)
>>>>> Thu Apr 4 14:12:42 UTC 2013: apel-publisher - program aborted
>>>>> org.glite.apel.core.ApelException: javax.jms.JMSException: Channel
>>>>> was inactive for too long:
>>>>> apel-broker.esc.rl.ac.uk/130.246.140.163:61617
>>>>> <http://apel-broker.esc.rl.ac.uk/130.246.140.163:61617>
>>>>> at org.glite.apel.publisher.AccountChecker.execute(Unknown
>>>>> Source)
>>>>> at
>>>>> org.glite.apel.publisher.AccountChecker.countArchivedRecords(Unkno
>>>>> wn
>>>>> Source)
>>>>> at
>>>>> org.glite.apel.publisher.AccountManager.chkArchivedTuples(Unknown
>>>>> Source)
>>>>> at org.glite.apel.publisher.AccountManager.run(Unknown Source)
>>>>> at
>>>>> org.glite.apel.publisher.ApelPublisher.runJoinProcessor(Unknown
>>>>> Source)
>>>>> at org.glite.apel.publisher.ApelPublisher.run(Unknown Source)
>>>>> at org.glite.apel.publisher.ApelPublisher.main(Unknown
>>>>> Source) Caused by: javax.jms.JMSException: Channel was inactive for too long:
>>>>> apel-broker.esc.rl.ac.uk/130.246.140.163:61617
>>>>> <http://apel-broker.esc.rl.ac.uk/130.246.140.163:61617>
>>>>> at
>>>>> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSu
>>>>> pp
>>>>> ort.java:62)
>>>>>
>>>>> at
>>>>> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQC
>>>>> on
>>>>> nection.java:1259)
>>>>>
>>>>> at
>>>>> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQCon
>>>>> ne
>>>>> ction.java:1251)
>>>>>
>>>>> at
>>>>> org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.
>>>>> java:1863)
>>>>>
>>>>> at
>>>>> org.apache.activemq.ActiveMQMessageProducer.close(ActiveMQMessageP
>>>>> ro
>>>>> ducer.java:147)
>>>>>
>>>>> ... 7 more
>>>>> Caused by: org.apache.activemq.transport.InactivityIOException:
>>>>> Channel was inactive for too long:
>>>>> apel-broker.esc.rl.ac.uk/130.246.140.163:61617
>>>>> <http://apel-broker.esc.rl.ac.uk/130.246.140.163:61617>
>>>>> at
>>>>> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityM
>>>>> on
>>>>> itor.java:247)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.TransportFilter.oneway(TransportFilt
>>>>> er
>>>>> .java:85)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireForm
>>>>> at
>>>>> Negotiator.java:104)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport
>>>>> .java:40)
>>>>>
>>>>> at
>>>>> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCo
>>>>> rr
>>>>> elator.java:60)
>>>>>
>>>>> at
>>>>> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQC
>>>>> on
>>>>> nection.java:1257)
>>>>>
>>>>> ... 10 more
>>>>>
>>>>>
>>>>> --
>>>>> Scanned by iCritical.
>>>>>
>>>>>
>>>>
>>>> --
>>>> Facts aren't facts if they come from the wrong people. (Paul
>>>> Krugman)
>>>
>>> --
>>> Facts aren't facts if they come from the wrong people. (Paul
>>> Krugman)
>>
>
>
--
Steve Jones [log in to unmask]
System Administrator office: 220
High Energy Physics Division tel (int): 42334
Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334
University of Liverpool http://www.liv.ac.uk/physics/hep/
--
Scanned by iCritical.
|