Hi Maarten
Here is logging-info report for this user:
$ glite-wms-job-logging-info -v 2
https://wms01.egee.cesga.es:9000/-1zpfW_Ujyw22Ci8nxIwkw
**********************************************************************
LOGGING INFORMATION:
Printing info for the Job :
https://wms01.egee.cesga.es:9000/-1zpfW_Ujyw22Ci8nxIwkw
---
Event: RegJob
- Arrived = Mon Oct 13 13:34:30 2008 CEST
- Host = wms01.egee.cesga.es
- Ns =
https://84.21.169.16:7443/glite_wms_wmproxy_server
- Nsubjobs = 0
- Parent =
https://wms01.egee.cesga.es:9000/InK6YPogBU0MYErU-oEPUw
- Source = NetworkServer
- Src instance =
https://84.21.169.16:7443/glite_wms_wmproxy_server
- Timestamp = Mon Oct 13 13:34:30 2008 CEST
- User = /DC=es/DC=irisgrid/O=cesga/CN=raul-valin
---
Event: EnQueued
- Arrived = Mon Oct 13 13:43:18 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/workload_manager/input.fl
- Result = OK
- Source = NetworkServer
- Src instance = 19043
- Timestamp = Mon Oct 13 13:43:12 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Match
- Arrived = Mon Oct 13 13:43:18 2008 CEST
- Dest id =
ce2.egee.cesga.es:2119/jobmanager-lcgsge-cesga
- Host = wms01.egee.cesga.es
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 13:43:12 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 13:43:18 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/jobcontrol/queue.fl
- Result = START
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 13:43:12 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 13:43:18 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/jobcontrol/queue.fl
- Result = OK
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 13:43:12 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: DeQueued
- Arrived = Mon Oct 13 13:48:15 2008 CEST
- Host = wms01.egee.cesga.es
- Local jobid = unavailable
- Queue = /var/glite/jobcontrol/queue.fl
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 13:47:53 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 13:48:16 2008 CEST
- Dest host = localhost
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = unavailable
- Destination = LogMonitor
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = START
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 13:47:57 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 13:48:16 2008 CEST
- Dest host = localhost
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = 6410
- Destination = LogMonitor
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = OK
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 13:48:05 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Accepted
- Arrived = Mon Oct 13 13:51:02 2008 CEST
- From = JobController
- From host = localhost
- From instance = unavailable
- Host = wms01.egee.cesga.es
- Local jobid = 6410
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 13:51:01 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 14:08:13 2008 CEST
- Dest host = unavailable
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = unavailable
- Destination = LRMS
- Host = wms01.egee.cesga.es
- Reason = 10 data transfer to the server failed
- Result = FAIL
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:13 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Done
- Arrived = Mon Oct 13 14:08:21 2008 CEST
- Exit code = 1
- Host = wms01.egee.cesga.es
- Reason = Job got an error while in the CondorG
queue.
- Source = LogMonitor
- Src instance = unique
- Status code = FAILED
- Timestamp = Mon Oct 13 14:08:20 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Resubmission
- Arrived = Mon Oct 13 14:08:21 2008 CEST
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = WILLRESUB
- Source = LogMonitor
- Src instance = unique
- Tag = unavailable
- Timestamp = Mon Oct 13 14:08:21 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:08:24 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/workload_manager/input.fl
- Reason = unavailable
- Result = START
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:21 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:08:25 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/workload_manager/input.fl
- Reason = unavailable
- Result = OK
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:21 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: DeQueued
- Arrived = Mon Oct 13 14:08:26 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/workload_manager/input.fl
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:08:25 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Resubmission
- Arrived = Mon Oct 13 14:08:27 2008 CEST
- Host = wms01.egee.cesga.es
- Reason = token still exists
- Result = SHALLOW
- Source = WorkloadManager
- Src instance = 19043
- Tag =
/var/glite/SandboxDir/-1/https_3a_2f_2fwms01.egee.cesga.es_3a9000_2f-1zpfW_5fUjyw22Ci8nxIwkw/token.txt
- Timestamp = Mon Oct 13 14:08:26 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Match
- Arrived = Mon Oct 13 14:08:33 2008 CEST
- Dest id =
ce.egee.cesga.es:2119/jobmanager-lcgsge-cesga
- Host = wms01.egee.cesga.es
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:08:33 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:08:33 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/jobcontrol/queue.fl
- Result = START
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:08:33 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:08:33 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/jobcontrol/queue.fl
- Result = OK
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:08:33 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: DeQueued
- Arrived = Mon Oct 13 14:08:34 2008 CEST
- Host = wms01.egee.cesga.es
- Local jobid = unavailable
- Queue = /var/glite/jobcontrol/queue.fl
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:34 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 14:08:34 2008 CEST
- Dest host = localhost
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = unavailable
- Destination = LogMonitor
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = START
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:34 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 14:08:34 2008 CEST
- Dest host = localhost
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = 6519
- Destination = LogMonitor
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = OK
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:34 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Accepted
- Arrived = Mon Oct 13 14:08:36 2008 CEST
- From = JobController
- From host = localhost
- From instance = unavailable
- Host = wms01.egee.cesga.es
- Local jobid = 6519
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:35 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 14:08:48 2008 CEST
- Dest host = ce.egee.cesga.es:2119/jobmanager-lcgsge
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = unavailable
- Destination = LRMS
- Host = wms01.egee.cesga.es
- Reason = Job successfully submitted to Globus
- Result = OK
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:08:48 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Running
- Arrived = Mon Oct 13 14:23:32 2008 CEST
- Host = wms01.egee.cesga.es
- Node = gt2
ce.egee.cesga.es:2119/jobmanager-lcgsge
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:22:51 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Done
- Arrived = Mon Oct 13 14:24:35 2008 CEST
- Exit code = 1
- Host = wms01.egee.cesga.es
- Reason = File not available.Cannot read
JobWrapper output, both from Condor and from Maradona.
- Source = LogMonitor
- Src instance = unique
- Status code = FAILED
- Timestamp = Mon Oct 13 14:24:29 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Resubmission
- Arrived = Mon Oct 13 14:24:35 2008 CEST
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = WILLRESUB
- Source = LogMonitor
- Src instance = unique
- Tag = unavailable
- Timestamp = Mon Oct 13 14:24:30 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:24:35 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/workload_manager/input.fl
- Reason = unavailable
- Result = START
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:24:30 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:24:35 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/workload_manager/input.fl
- Reason = unavailable
- Result = OK
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:24:30 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: DeQueued
- Arrived = Mon Oct 13 14:24:46 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/workload_manager/input.fl
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:24:31 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Resubmission
- Arrived = Mon Oct 13 14:24:46 2008 CEST
- Host = wms01.egee.cesga.es
- Reason = token still exists
- Result = SHALLOW
- Source = WorkloadManager
- Src instance = 19043
- Tag =
/var/glite/SandboxDir/-1/https_3a_2f_2fwms01.egee.cesga.es_3a9000_2f-1zpfW_5fUjyw22Ci8nxIwkw/token.txt
- Timestamp = Mon Oct 13 14:24:41 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Match
- Arrived = Mon Oct 13 14:24:54 2008 CEST
- Dest id =
ce2.egee.cesga.es:2119/jobmanager-lcgSVGD-GRID
- Host = wms01.egee.cesga.es
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:24:50 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:24:55 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/jobcontrol/queue.fl
- Result = START
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:24:54 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: EnQueued
- Arrived = Mon Oct 13 14:24:55 2008 CEST
- Host = wms01.egee.cesga.es
- Queue = /var/glite/jobcontrol/queue.fl
- Result = OK
- Source = WorkloadManager
- Src instance = 19043
- Timestamp = Mon Oct 13 14:24:55 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: DeQueued
- Arrived = Mon Oct 13 14:24:55 2008 CEST
- Host = wms01.egee.cesga.es
- Local jobid = unavailable
- Queue = /var/glite/jobcontrol/queue.fl
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 14:24:55 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 14:24:56 2008 CEST
- Dest host = localhost
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = unavailable
- Destination = LogMonitor
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = START
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 14:24:55 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 14:24:56 2008 CEST
- Dest host = localhost
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = 6615
- Destination = LogMonitor
- Host = wms01.egee.cesga.es
- Reason = unavailable
- Result = OK
- Source = JobController
- Src instance = unique
- Timestamp = Mon Oct 13 14:24:56 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Accepted
- Arrived = Mon Oct 13 14:25:00 2008 CEST
- From = JobController
- From host = localhost
- From instance = unavailable
- Host = wms01.egee.cesga.es
- Local jobid = 6615
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 14:25:00 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Transfer
- Arrived = Mon Oct 13 15:42:34 2008 CEST
- Dest host = ce2.egee.cesga.es:2119/jobmanager-lcgSVGD
- Dest instance =
/var/glite/logmonitor/CondorG.log/CondorG.1223828170.log
- Dest jobid = unavailable
- Destination = LRMS
- Host = wms01.egee.cesga.es
- Reason = Job successfully submitted to Globus
- Result = OK
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 15:42:33 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: Running
- Arrived = Mon Oct 13 16:30:46 2008 CEST
- Host = wms01.egee.cesga.es
- Node = gt2
ce2.egee.cesga.es:2119/jobmanager-lcgSVGD
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 16:30:46 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
---
Event: ReallyRunning
- Arrived = Mon Oct 13 18:08:45 2008 CEST
- Host = wms01.egee.cesga.es
- Source = LogMonitor
- Src instance = unique
- Timestamp = Mon Oct 13 18:08:45 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
- Wn seq =
UI=000000:NS=0000000004:WM=000015:BH=0000000000:JSS=000009:LM=000010:LRMS=000002:APP=000000:LBS=000000
---
Event: Done
- Arrived = Mon Oct 13 18:08:46 2008 CEST
- Exit code = 0
- Host = wms01.egee.cesga.es
- Reason = Job terminated successfully
- Source = LogMonitor
- Src instance = unique
- Status code = OK
- Timestamp = Mon Oct 13 18:08:45 2008 CEST
- User =
/DC=es/DC=irisgrid/O=cesga/CN=raul-valin/CN=proxy/CN=proxy/CN=proxy/CN=proxy/CN=proxy
**********************************************************************
It seems that job was resubmitted due data transfer failure, We will
investigate why he got this error.
Cheers and thanks
Alvaro
> Hola Alvaro,
>
>
>> Today an user reports to us a problem regarding glite-wms-job-info
>> command. He got this error:
>>
>>
>> #glite-wms-job-info --jdl
>> https://wms01.egee.cesga.es:9000/-1zpfW_Ujyw22Ci8nxIwkw
>>
>> Connecting to the service
>> https://wms01.egee.cesga.es:7443/glite_wms_wmproxy_server
>>
>> Error - WMProxy Server Error
>> Unable to read file:
>> /var/glite/SandboxDir/-1/https_3a_2f_2fwms01.egee.cesga.es_3a9000_2f-1zpfW_5fUjyw22Ci8nxIwkw/JDLToStart
>>
>
> What happened to the job, i.e. what does this report:
>
> glite-wms-job-logging-info -v 2 \
> https://wms01.egee.cesga.es:9000/-1zpfW_Ujyw22Ci8nxIwkw
>
>
>
>
|