I found also there are many error messages at LB like:
File exists (Duplicate entry 'TJBp8nD3zLWALANhYtQc9g' for key 1)
/UtHf8K6S8vuMvDL9nx1P1Q: File exists (Duplicate entry
'UtHf8K6S8vuMvDL9nx1P1Q' for key 1)
/var/log/messages.1:Jun 14 11:37:13 lb-1-kit syslog[20535]: ERROR
CONTROL - https://lb-1-kit.gridka.de:9000/iIfODhGma-dG93Y1zbGcxg: File
exists (Duplicate entry 'iIfODhGma-dG93Y1zbGcxg' for key 1)
/var/log/messages.1:Jun 14 11:37:13 lb-1-kit syslog[20535]: ERROR
CONTROL - https://lb-1-kit.gridka.de:9000/TJBp8nD3zLWALANhYtQc9g: File
exists (Duplicate entry 'TJBp8nD3zLWALANhYtQc9g' for key 1)
[root@lb-1-kit ~]#
[root@lb-1-kit ~]#
[root@lb-1-kit ~]# grep -r "Duplicate entry" /var/log | wc -l
23710
Regards
Dimitri
On 06/17/2013 05:40 PM, Dimitri Nilsen wrote:
> Hi Ronald,
>
> we had not explizit configure LBs to allow or deny WMSs
> however many jobs end with ok status.. but a huge number of them have
> this "not found in sandbox" issue.
> The config file of LB is this bellow.. I think there is just ".*" to
> accept all hosts...
> Or should I enter DNs of WMS hosts?
>
> Regards
> Dimitri
>
> cat /etc/glite-lb/glite-lb-authz.conf
>
> resource "LB" {
>
> action "ADMIN_ACCESS" {
> }
>
> action "STATUS_FOR_MONITORING" {
> }
>
> action "GET_STATISTICS" {
> }
>
> action "REGISTER_JOBS" {
> rule permit {
> subject = ".*"
> }
> }
>
> action "READ_ALL" {
> }
>
> action "PURGE" {
> }
>
> action "GRANT_OWNERSHIP" {
> }
>
> action "LOG_WMS_EVENTS" {
> rule permit {
> subject = ".*"
> }
> }
>
> action "LOG_CE_EVENTS" {
> rule permit {
> subject = ".*"
> }
> }
>
> action "LOG_GENERAL_EVENTS" {
> rule permit {
> subject = ".*"
> }
> }
>
> }
>
>
>
> On 06/17/2013 02:50 PM, Ronald Starink wrote:
>> Hi Dimitri,
>>
>> Are you sure that the LB allows your WMS to query the job status (assuming
>> they're not on the same host)? If the LB does not accept this request from
>> the WMS, the WMS interprets this as "job not found on LB" and purges the job
>> locally.
>>
>> If you use Yaim to configure the LB server, the variable GLITE_LB_WMS_DN
>> should contain the certificate DN of your WMS (or a comma-separated list of
>> DNs). Yaim adds the DN to the file /etc/glite-lb/glite-lb-authz.conf.
>>
>> Cheers,
>> Ronald
>>
>>
>>
>>
>>
>> On 17-06-13 14:23, Dimitri Nilsen wrote:
>>> Hi,
>>>
>>> we have tickets from experiments with an error:
>>> Cannot move ISB ....
>>> error: globus_ftp_client: the server responded with an error500 500-Command
>>> failed. : globus_l_gfs_file_open failed.500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/SOME_FILE_NAME
>>>
>>> it seems that files disappear from sandbox for some reason.
>>> I found some related error messages in log files. It looks like jobPurger
>>> remove jobs.. Could anyone take a look and help us to understand why it
>>> happens?
>>>
>>> Hier some log outputs for one particular job:
>>>
>>> ## grep Jncl2Kvfsp9FOtM8kDlXhA /var/log/wms/glite-wms-purgeStorage.log*
>>>
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -E: [Error]
>>> query_job_status(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:134):
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: edg_wll_JobStat [2]
>>> No such file or directory(no matching jobs found)
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -I: [Info]
>>> operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:306):
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: forced removal,
>>> unknown/removed L&B job
>>>
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]# grep -r Jncl2Kvfsp9FOtM8kDlXhA /var/log
>>> /var/log/messages.1:Jun 13 14:04:21 wms-2-kit syslog[17864]: ERROR
>>> CONTROL - https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: File
>>> exists (Duplicate entry 'Jncl2Kvfsp9FOtM8kDlXhA' for key 1)
>>> /var/log/messages.1:Jun 13 14:04:43 wms-2-kit glite-wms-workload_manager:
>>> jobid https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA was matched to
>>> destination grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,567 DEBUG -
>>> utilities::exists_subsequent_token - Checking subsequent token files of
>>> [/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0]
>>>
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,568 ERROR -
>>> utilities::exists_subsequent_token - boost::filesystem::directory_iterator
>>> constructor:
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA":
>>> No such file or directory
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,568 DEBUG -
>>> iceCommandEventQuery::processEventsForJob() - TID=[29885200] Processing [2]
>>> event(s) for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> userdn
>>> [/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo-/auger/Role=Production/Capability=NULL]
>>> and ce url
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2].
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,569 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000004:LRMS=000000:APP=000000:LBS=000000](port
>>> is not used, actually...)
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,569 INFO -
>>> iceLBLogger::logEvent() - Job Running Event,
>>> worker_node=[grid-wn092.physik.rwth-aachen.de] -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,899 DEBUG -
>>> iceCommandEventQuery::processSingleEvent() - TID=[29885200] Updating ICE's
>>> database for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> status = [DONE-FAILED] exit_code = [N/A] failure_reason = [Cannot move ISB
>>> (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>> error: globus_ftp_client: the server responded with an error500 500-Command
>>> failed. : globus_l_gfs_file_open failed.500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh500-globus_xio:
>>> System error in open: No such file or directory500-globus_xio: A system call
>>> failed: No such file or directory500 End.; /usr/bin/glite-lb-logevent:
>>> edg_wll_LogEvent*(): LB server (bkserver,lbproxy) store protocol error
>>> (edg_wll_LogEvent(): LB server (bkserver,lbproxy) store protocol error;;
>>> Logging library ERROR: LB server (bkserver,lbproxy) store protocol error;;
>>> edg_wll_DoLogEvent(): edg_wll_log_connect error Transport endpoint is not
>>> connected;; edg_wll_gss_connect();; System Error: Connection refused)
>>> /usr/bin/glite-lb-logevent: edg_wll_LogEvent*(): LB server
>>> (bkserver,lbproxy) store protocol error (edg_wll_LogEvent(): LB server
>>> (bkserver,lbproxy) store protocol error;; Logging library ERROR: LB server
>>> (bkserver,lbproxy) store protocol error;; edg_wll_DoLogEvent():
>>> edg_wll_log_connect error Transport endpoint is not connected;;
>>> edg_wll_gss_connect();; System Error: Connection refused) Cannot move ISB
>>> (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>> error: globus_ftp_client: the server responded with an error 500 500-Command
>>> failed. : globus_l_gfs_file_open failed. 500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> 500-globus_xio: System error in open: No such file or directory
>>> 500-globus_xio: A system call failed: No such file or directory 500 End.]
>>> description = [N/A]
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,901 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000006:LRMS=000000:APP=000000:LBS=000000](port
>>> is not used, actually...)
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,901 INFO -
>>> iceLBLogger::logEvent() - Job Done Failed Event, ExitCode=[0],
>>> FailureReason=[Cannot move ISB (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>> error: globus_ftp_client: the server responded with an error500 500-Command
>>> failed. : globus_l_gfs_file_open failed.500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh500-globus_xio:
>>> System error in open: No such file or directory500-globus_xio: A system call
>>> failed: No such file or directory500 End.; /usr/bin/glite-lb-logevent:
>>> edg_wll_LogEvent*(): LB server (bkserver,lbproxy) store protocol error
>>> (edg_wll_LogEvent(): LB server (bkserver,lbproxy) store protocol error;;
>>> Logging library ERROR: LB server (bkserver,lbproxy) store protocol error;;
>>> edg_wll_DoLogEvent(): edg_wll_log_connect error Transport endpoint is not
>>> connected;; edg_wll_gss_connect();; System Error: Connection refused)
>>> /usr/bin/glite-lb-logevent: edg_wll_LogEvent*(): LB server
>>> (bkserver,lbproxy) store protocol error (edg_wll_LogEvent(): LB server
>>> (bkserver,lbproxy) store protocol error;; Logging library ERROR: LB server
>>> (bkserver,lbproxy) store protocol error;; edg_wll_DoLogEvent():
>>> edg_wll_log_connect error Transport endpoint is not connected;;
>>> edg_wll_gss_connect();; System Error: Connection refused) Cannot move ISB
>>> (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>> error: globus_ftp_client: the server responded with an error 500 500-Command
>>> failed. : globus_l_gfs_file_open failed. 500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> 500-globus_xio: System error in open: No such file or directory
>>> 500-globus_xio: A system call failed: No such file or directory 500 End.] -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,942 DEBUG -
>>> IceCore::resubmit_or_purge_job() - Removing purged job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> from ICE's database
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:49,947 ERROR -
>>> IceCore::resubmit_job() - Will NOT resubmit job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> because it's Input Sandbox proxy file is expired: Unable to open the file
>>> [/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy]:
>>> No such file or directory
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -E: [Error]
>>> query_job_status(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:134):
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: edg_wll_JobStat [2]
>>> No such file or directory(no matching jobs found)
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -I: [Info]
>>> operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:306):
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: forced removal,
>>> unknown/removed L&B job
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,284 DEBUG - IceCore::main_loop -
>>> *** Unparsing request <[ Source = 2; Protocol = "1.0.0"; Arguments = [ JobAd
>>> = [ NodeName = "INIT_MPDProtonEpos_gr267_en19_750_th55_65_664303";
>>> CEInfoHostName = "grid-ce2.physik.rwth-aachen.de"; OutputSandboxPath =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output";
>>> StdOutput = "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out";
>>> GlobusResourceContactString =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs"; QueueName = "auger";
>>> ShallowRetryCount = 0; RetryCount = 1; Executable = "/bin/sh"; Type = "job";
>>> LB_sequence_code =
>>> "UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000";
>>> Arguments =
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303"; VirtualOrganisation
>>> = "auger"; JobType = "normal"; DefaultRank =
>>> -other.GlueCEStateEstimatedResponseTime; ce_id =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger"; OutputSandboxDestURI
>>> = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>> }; OutputSandbox = {
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>> }; InputSandbox = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/Modules.tar.bz2","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo"
>>> }; edg_jobid = "https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA";
>>> VOMS_FQAN = "/auger/Role=Production/Capability=NULL"; CertificateSubject =
>>> "/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo"; StdError =
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err"; InputSandboxPath
>>> =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input";
>>> ReallyRunningToken =
>>> "gsiftp://wms-2-kit.gridka.de/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0";
>>> rank = -other.GlueCEStateEstimatedResponseTime; lrms_type = "pbs";
>>> MyProxyServer = "myproxy1.egee.cesnet.cz"; X509UserProxy =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy";
>>> InputSandboxDestFileName = {
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","Modules.tar.bz2"
>>> }; requirements = ( ( ( ( other.GlueCEPolicyMaxCPUTime > 300 ) && (
>>> other.CEId != "cream1.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream1.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream2.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream2.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream-ce.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId
>>> != "cream-ce02.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "ce02.lip.pt:8443/cream-sge-gridq_x86_64" ) && ( other.CEId !=
>>> "ce03.ncg.ingrid.pt:8443/cream-sge-gridq" ) && ( other.CEId !=
>>> "tochtli64.nucleares.unam.mx:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "cccreamceli11.in2p3.fr:8443/cream-sge-long" ) && ( other.CEId !=
>>> "ce0.m3pec.u-bordeaux1.fr:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "creamce.ijs.si:8443/cream-slurm-batch" ) && ( other.CEId !=
>>> "ce01-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "ce04-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "grid012.ct.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_opportunistic" ) && (
>>> other.CEId != "grid-ce.physik.rwth-aachen.de:8443/cream-pbs-auger" ) && (
>>> other.CEId != "cream.egi.cesga.es:8443/cream-sge-GRIDEGI_large" ) ) ) && (
>>> other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is
>>> true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :
>>> !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs
>>> == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && (
>>> EnableWmsFeedback is true ?
>>> RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
>>> WMPInputSandboxBaseURI =
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA"
>>> ] ]; Command = "Submit" ]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,286 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de//var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> iceCommandSubmit::updateIsbList() -
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,308 INFO -
>>> iceCommandSubmit::execute() - TID=[29883760] This request is a Submission
>>> for job [https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,313 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000](port
>>> is not used, actually...)
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,313 INFO -
>>> iceLBLogger::logEvent() - WMS Dequeue Event, qname=[/var/ice/jobdir] -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,689 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000](port
>>> is not used, actually...)
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,689 INFO -
>>> iceLBLogger::logEvent() - Cream Transfer Start Event -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,760 DEBUG -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] Submitting JDL [
>>> NodeName = "INIT_MPDProtonEpos_gr267_en19_750_th55_65_664303";
>>> CEInfoHostName = "grid-ce2.physik.rwth-aachen.de"; OutputSandboxPath =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output";
>>> StdOutput = "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out";
>>> BatchSystem = "pbs"; GlobusResourceContactString =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs"; QueueName = "auger";
>>> ShallowRetryCount = 0; RetryCount = 1; Executable = "/bin/sh"; Type = "job";
>>> LB_sequence_code =
>>> "UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000";
>>> Arguments =
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303";
>>> maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation =
>>> "auger"; JobType = "normal"; DefaultRank =
>>> -other.GlueCEStateEstimatedResponseTime; ce_id =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger"; OutputSandboxDestURI
>>> = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>> }; OutputSandbox = {
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>> }; InputSandbox = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/Modules.tar.bz2","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo"
>>> }; edg_jobid = "https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA";
>>> VOMS_FQAN = "/auger/Role=Production/Capability=NULL"; CertificateSubject =
>>> "/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo"; StdError =
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err"; InputSandboxPath
>>> =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input";
>>> ReallyRunningToken =
>>> "gsiftp://wms-2-kit.gridka.de/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0";
>>> rank = -other.GlueCEStateEstimatedResponseTime; WMSHostname =
>>> "wms-2-kit.gridka.de"; lrms_type = "pbs"; MyProxyServer =
>>> "myproxy1.egee.cesnet.cz"; X509UserProxy =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy";
>>> InputSandboxDestFileName = {
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","Modules.tar.bz2"
>>> }; requirements = ( ( ( ( other.GlueCEPolicyMaxCPUTime > 300 ) && (
>>> other.CEId != "cream1.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream1.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream2.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream2.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream-ce.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId
>>> != "cream-ce02.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "ce02.lip.pt:8443/cream-sge-gridq_x86_64" ) && ( other.CEId !=
>>> "ce03.ncg.ingrid.pt:8443/cream-sge-gridq" ) && ( other.CEId !=
>>> "tochtli64.nucleares.unam.mx:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "cccreamceli11.in2p3.fr:8443/cream-sge-long" ) && ( other.CEId !=
>>> "ce0.m3pec.u-bordeaux1.fr:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "creamce.ijs.si:8443/cream-slurm-batch" ) && ( other.CEId !=
>>> "ce01-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "ce04-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "grid012.ct.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_opportunistic" ) && (
>>> other.CEId != "grid-ce.physik.rwth-aachen.de:8443/cream-pbs-auger" ) && (
>>> other.CEId != "cream.egi.cesga.es:8443/cream-sge-GRIDEGI_large" ) ) ) && (
>>> other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is
>>> true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :
>>> !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs
>>> == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && (
>>> EnableWmsFeedback is true ?
>>> RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
>>> WMPInputSandboxBaseURI =
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA"
>>> ] to [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2]
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/gridsite-delegation]
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,760 DEBUG -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] Sequence code for job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""] is
>>> UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,774 DEBUG -
>>> iceCommandSubmit::register_job() - TID=[29883760] Going to REGISTER Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""] with delegation ID [13690290782E322wms2D22Dkit2Egridka2Ede]
>>> to CREAM
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2]...
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,466 INFO -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] For GridJobID
>>> [https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] CREAM Returned
>>> CREAM-JOBID [https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997]
>>> DB_ID [1353999667000]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,469 DEBUG -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] Going to START
>>> CreamJobID [https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997]
>>> related to GridJobID
>>> [https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA]...
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,610 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000](port
>>> is not used, actually...)
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,610 INFO -
>>> iceLBLogger::logEvent() - Cream Transfer OK Event -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,914 DEBUG -
>>> filelist_request_purger - removing request [ Source = 2; Protocol = "1.0.0";
>>> Arguments = [ JobAd = [ NodeName =
>>> "INIT_MPDProtonEpos_gr267_en19_750_th55_65_664303"; CEInfoHostName =
>>> "grid-ce2.physik.rwth-aachen.de"; OutputSandboxPath =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output";
>>> StdOutput = "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out";
>>> GlobusResourceContactString =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs"; QueueName = "auger";
>>> ShallowRetryCount = 0; RetryCount = 1; Executable = "/bin/sh"; Type = "job";
>>> LB_sequence_code =
>>> "UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000";
>>> Arguments =
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303"; VirtualOrganisation
>>> = "auger"; JobType = "normal"; DefaultRank =
>>> -other.GlueCEStateEstimatedResponseTime; ce_id =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger"; OutputSandboxDestURI
>>> = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>> }; OutputSandbox = {
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>> }; InputSandbox = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/Modules.tar.bz2","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo"
>>> }; edg_jobid = "https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA";
>>> VOMS_FQAN = "/auger/Role=Production/Capability=NULL"; CertificateSubject =
>>> "/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo"; StdError =
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err"; InputSandboxPath
>>> =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input";
>>> ReallyRunningToken =
>>> "gsiftp://wms-2-kit.gridka.de/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0";
>>> rank = -other.GlueCEStateEstimatedResponseTime; lrms_type = "pbs";
>>> MyProxyServer = "myproxy1.egee.cesnet.cz"; X509UserProxy =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy";
>>> InputSandboxDestFileName = {
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","Modules.tar.bz2"
>>> }; requirements = ( ( ( ( other.GlueCEPolicyMaxCPUTime > 300 ) && (
>>> other.CEId != "cream1.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream1.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream2.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream2.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream-ce.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId
>>> != "cream-ce02.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "ce02.lip.pt:8443/cream-sge-gridq_x86_64" ) && ( other.CEId !=
>>> "ce03.ncg.ingrid.pt:8443/cream-sge-gridq" ) && ( other.CEId !=
>>> "tochtli64.nucleares.unam.mx:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "cccreamceli11.in2p3.fr:8443/cream-sge-long" ) && ( other.CEId !=
>>> "ce0.m3pec.u-bordeaux1.fr:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "creamce.ijs.si:8443/cream-slurm-batch" ) && ( other.CEId !=
>>> "ce01-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "ce04-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "grid012.ct.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_opportunistic" ) && (
>>> other.CEId != "grid-ce.physik.rwth-aachen.de:8443/cream-pbs-auger" ) && (
>>> other.CEId != "cream.egi.cesga.es:8443/cream-sge-GRIDEGI_large" ) ) ) && (
>>> other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is
>>> true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :
>>> !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs
>>> == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && (
>>> EnableWmsFeedback is true ?
>>> RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
>>> WMPInputSandboxBaseURI =
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA"
>>> ] ]; Command = "Submit" ]
>>> /var/log/wms/ice.log.2:2013-06-13 14:06:39,345 DEBUG -
>>> iceCommandEventQuery::processEventsForJob() - TID=[29880192] Processing [3]
>>> event(s) for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> userdn
>>> [/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo-/auger/Role=Production/Capability=NULL]
>>> and ce url
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2].
>>> /var/log/wms/ice.log.2:2013-06-13 14:06:39,346 DEBUG -
>>> iceCommandEventQuery::processSingleEvent() - TID=[29880192] Updating ICE's
>>> database for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> status = [IDLE] exit_code = [N/A] failure_reason = [N/A] description = [N/A]
>
> --
> Dimitri Nilsen, Dipl.-Ing(FH)
>
> Karlsruhe Institute of Technology (KIT)
> Steinbuch Centre for Computing
> Kaiserstr. 12
> 76131 Karlsruhe, Germany
>
> Tel.: +49 721 608 28607
> Email: [log in to unmask]
--
Dimitri Nilsen, Dipl.-Ing(FH)
Karlsruhe Institute of Technology (KIT)
Steinbuch Centre for Computing
Kaiserstr. 12
76131 Karlsruhe, Germany
Tel.: +49 721 608 28607
Email: [log in to unmask]
|