Hi Dimitri,
On 17-06-13 17:40, Dimitri Nilsen wrote:
> Hi Ronald,
>
> we had not explizit configure LBs to allow or deny WMSs
> however many jobs end with ok status.. but a huge number of them have this
> "not found in sandbox" issue.
> The config file of LB is this bellow.. I think there is just ".*" to accept
> all hosts...
> Or should I enter DNs of WMS hosts?
I don't know the exact behaviour of the LB. However, I do know that putting
a wrong certificate DN results in the purging of running jobs on the WMS,
similar to what you observed.
The configuration that you sent indeed contains a few wildcards to accept
all hosts. But various entries are completely empty: GET_STATISTICS,
READ_ALL, PURGE. On our LB (configured by Yaim) they do contain our WMS
hosts. You could easily add the DNs there (one "rule permit" block per WMS
DN) and see if helps.
Cheers,
Ronald
>
> Regards
> Dimitri
>
> cat /etc/glite-lb/glite-lb-authz.conf
>
> resource "LB" {
>
> action "ADMIN_ACCESS" {
> }
>
> action "STATUS_FOR_MONITORING" {
> }
>
> action "GET_STATISTICS" {
> }
>
> action "REGISTER_JOBS" {
> rule permit {
> subject = ".*"
> }
> }
>
> action "READ_ALL" {
> }
>
> action "PURGE" {
> }
>
> action "GRANT_OWNERSHIP" {
> }
>
> action "LOG_WMS_EVENTS" {
> rule permit {
> subject = ".*"
> }
> }
>
> action "LOG_CE_EVENTS" {
> rule permit {
> subject = ".*"
> }
> }
>
> action "LOG_GENERAL_EVENTS" {
> rule permit {
> subject = ".*"
> }
> }
>
> }
>
>
>
> On 06/17/2013 02:50 PM, Ronald Starink wrote:
>> Hi Dimitri,
>>
>> Are you sure that the LB allows your WMS to query the job status (assuming
>> they're not on the same host)? If the LB does not accept this request from
>> the WMS, the WMS interprets this as "job not found on LB" and purges the job
>> locally.
>>
>> If you use Yaim to configure the LB server, the variable GLITE_LB_WMS_DN
>> should contain the certificate DN of your WMS (or a comma-separated list of
>> DNs). Yaim adds the DN to the file /etc/glite-lb/glite-lb-authz.conf.
>>
>> Cheers,
>> Ronald
>>
>>
>>
>>
>>
>> On 17-06-13 14:23, Dimitri Nilsen wrote:
>>> Hi,
>>>
>>> we have tickets from experiments with an error:
>>> Cannot move ISB ....
>>> error: globus_ftp_client: the server responded with an error500 500-Command
>>> failed. : globus_l_gfs_file_open failed.500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/SOME_FILE_NAME
>>>
>>> it seems that files disappear from sandbox for some reason.
>>> I found some related error messages in log files. It looks like jobPurger
>>> remove jobs.. Could anyone take a look and help us to understand why it
>>> happens?
>>>
>>> Hier some log outputs for one particular job:
>>>
>>> ## grep Jncl2Kvfsp9FOtM8kDlXhA /var/log/wms/glite-wms-purgeStorage.log*
>>>
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -E: [Error]
>>> query_job_status(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:134):
>>>
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: edg_wll_JobStat [2]
>>> No such file or directory(no matching jobs found)
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -I: [Info]
>>> operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:306):
>>>
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: forced removal,
>>> unknown/removed L&B job
>>>
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]#
>>> [root@wms-2-kit ~]# grep -r Jncl2Kvfsp9FOtM8kDlXhA /var/log
>>> /var/log/messages.1:Jun 13 14:04:21 wms-2-kit syslog[17864]: ERROR
>>> CONTROL - https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: File
>>> exists (Duplicate entry 'Jncl2Kvfsp9FOtM8kDlXhA' for key 1)
>>> /var/log/messages.1:Jun 13 14:04:43 wms-2-kit glite-wms-workload_manager:
>>> jobid https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA was matched to
>>> destination grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,567 DEBUG -
>>> utilities::exists_subsequent_token - Checking subsequent token files of
>>> [/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0]
>>>
>>>
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,568 ERROR -
>>> utilities::exists_subsequent_token - boost::filesystem::directory_iterator
>>> constructor:
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA":
>>>
>>> No such file or directory
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,568 DEBUG -
>>> iceCommandEventQuery::processEventsForJob() - TID=[29885200] Processing [2]
>>> event(s) for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> userdn
>>> [/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo-/auger/Role=Production/Capability=NULL]
>>>
>>> and ce url
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2].
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,569 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000004:LRMS=000000:APP=000000:LBS=000000](port
>>>
>>> is not used, actually...)
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,569 INFO -
>>> iceLBLogger::logEvent() - Job Running Event,
>>> worker_node=[grid-wn092.physik.rwth-aachen.de] -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,899 DEBUG -
>>> iceCommandEventQuery::processSingleEvent() - TID=[29885200] Updating ICE's
>>> database for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> status = [DONE-FAILED] exit_code = [N/A] failure_reason = [Cannot move ISB
>>> (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>>
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>>
>>> error: globus_ftp_client: the server responded with an error500 500-Command
>>> failed. : globus_l_gfs_file_open failed.500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh500-globus_xio:
>>>
>>> System error in open: No such file or directory500-globus_xio: A system call
>>> failed: No such file or directory500 End.; /usr/bin/glite-lb-logevent:
>>> edg_wll_LogEvent*(): LB server (bkserver,lbproxy) store protocol error
>>> (edg_wll_LogEvent(): LB server (bkserver,lbproxy) store protocol error;;
>>> Logging library ERROR: LB server (bkserver,lbproxy) store protocol error;;
>>> edg_wll_DoLogEvent(): edg_wll_log_connect error Transport endpoint is not
>>> connected;; edg_wll_gss_connect();; System Error: Connection refused)
>>> /usr/bin/glite-lb-logevent: edg_wll_LogEvent*(): LB server
>>> (bkserver,lbproxy) store protocol error (edg_wll_LogEvent(): LB server
>>> (bkserver,lbproxy) store protocol error;; Logging library ERROR: LB server
>>> (bkserver,lbproxy) store protocol error;; edg_wll_DoLogEvent():
>>> edg_wll_log_connect error Transport endpoint is not connected;;
>>> edg_wll_gss_connect();; System Error: Connection refused) Cannot move ISB
>>> (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>>
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>>
>>> error: globus_ftp_client: the server responded with an error 500 500-Command
>>> failed. : globus_l_gfs_file_open failed. 500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>>
>>> 500-globus_xio: System error in open: No such file or directory
>>> 500-globus_xio: A system call failed: No such file or directory 500 End.]
>>> description = [N/A]
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,901 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000006:LRMS=000000:APP=000000:LBS=000000](port
>>>
>>> is not used, actually...)
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,901 INFO -
>>> iceLBLogger::logEvent() - Job Done Failed Event, ExitCode=[0],
>>> FailureReason=[Cannot move ISB (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>>
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>>
>>> error: globus_ftp_client: the server responded with an error500 500-Command
>>> failed. : globus_l_gfs_file_open failed.500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh500-globus_xio:
>>>
>>> System error in open: No such file or directory500-globus_xio: A system call
>>> failed: No such file or directory500 End.; /usr/bin/glite-lb-logevent:
>>> edg_wll_LogEvent*(): LB server (bkserver,lbproxy) store protocol error
>>> (edg_wll_LogEvent(): LB server (bkserver,lbproxy) store protocol error;;
>>> Logging library ERROR: LB server (bkserver,lbproxy) store protocol error;;
>>> edg_wll_DoLogEvent(): edg_wll_log_connect error Transport endpoint is not
>>> connected;; edg_wll_gss_connect();; System Error: Connection refused)
>>> /usr/bin/glite-lb-logevent: edg_wll_LogEvent*(): LB server
>>> (bkserver,lbproxy) store protocol error (edg_wll_LogEvent(): LB server
>>> (bkserver,lbproxy) store protocol error;; Logging library ERROR: LB server
>>> (bkserver,lbproxy) store protocol error;; edg_wll_DoLogEvent():
>>> edg_wll_log_connect error Transport endpoint is not connected;;
>>> edg_wll_gss_connect();; System Error: Connection refused) Cannot move ISB
>>> (retry_copy ${globus_transfer_cmd}
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>>
>>> file:///tmp/14965305.grid-ce.physik.rwth-aachen.de/CREAM656117997/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh):
>>>
>>> error: globus_ftp_client: the server responded with an error 500 500-Command
>>> failed. : globus_l_gfs_file_open failed. 500-globus_xio: Unable to open file
>>> /var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>>
>>> 500-globus_xio: System error in open: No such file or directory
>>> 500-globus_xio: A system call failed: No such file or directory 500 End.] -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:46,942 DEBUG -
>>> IceCore::resubmit_or_purge_job() - Removing purged job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> from ICE's database
>>> /var/log/wms/ice.log.1:2013-06-14 07:52:49,947 ERROR -
>>> IceCore::resubmit_job() - Will NOT resubmit job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> because it's Input Sandbox proxy file is expired: Unable to open the file
>>> [/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy]:
>>>
>>> No such file or directory
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -E: [Error]
>>> query_job_status(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:134):
>>>
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: edg_wll_JobStat [2]
>>> No such file or directory(no matching jobs found)
>>> /var/log/wms/glite-wms-purgeStorage.log.1:13 Jun, 18:09:32 -I: [Info]
>>> operator()(/home/condor/execute/dir_26089/userdir/emi.wms.wms-purger/build/../src/purger.cpp:306):
>>>
>>> https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA: forced removal,
>>> unknown/removed L&B job
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,284 DEBUG - IceCore::main_loop -
>>> *** Unparsing request <[ Source = 2; Protocol = "1.0.0"; Arguments = [ JobAd
>>> = [ NodeName = "INIT_MPDProtonEpos_gr267_en19_750_th55_65_664303";
>>> CEInfoHostName = "grid-ce2.physik.rwth-aachen.de"; OutputSandboxPath =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output";
>>>
>>> StdOutput = "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out";
>>> GlobusResourceContactString =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs"; QueueName = "auger";
>>> ShallowRetryCount = 0; RetryCount = 1; Executable = "/bin/sh"; Type = "job";
>>> LB_sequence_code =
>>> "UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000";
>>>
>>> Arguments =
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303"; VirtualOrganisation
>>> = "auger"; JobType = "normal"; DefaultRank =
>>> -other.GlueCEStateEstimatedResponseTime; ce_id =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger"; OutputSandboxDestURI
>>> = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>>
>>> }; OutputSandbox = {
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>>
>>> }; InputSandbox = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/Modules.tar.bz2","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo"
>>>
>>> }; edg_jobid = "https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA";
>>> VOMS_FQAN = "/auger/Role=Production/Capability=NULL"; CertificateSubject =
>>> "/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo"; StdError =
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err"; InputSandboxPath
>>> =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input";
>>>
>>> ReallyRunningToken =
>>> "gsiftp://wms-2-kit.gridka.de/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0";
>>>
>>> rank = -other.GlueCEStateEstimatedResponseTime; lrms_type = "pbs";
>>> MyProxyServer = "myproxy1.egee.cesnet.cz"; X509UserProxy =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy";
>>>
>>> InputSandboxDestFileName = {
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","Modules.tar.bz2"
>>>
>>> }; requirements = ( ( ( ( other.GlueCEPolicyMaxCPUTime > 300 ) && (
>>> other.CEId != "cream1.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream1.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream2.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream2.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream-ce.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId
>>> != "cream-ce02.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "ce02.lip.pt:8443/cream-sge-gridq_x86_64" ) && ( other.CEId !=
>>> "ce03.ncg.ingrid.pt:8443/cream-sge-gridq" ) && ( other.CEId !=
>>> "tochtli64.nucleares.unam.mx:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "cccreamceli11.in2p3.fr:8443/cream-sge-long" ) && ( other.CEId !=
>>> "ce0.m3pec.u-bordeaux1.fr:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "creamce.ijs.si:8443/cream-slurm-batch" ) && ( other.CEId !=
>>> "ce01-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "ce04-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "grid012.ct.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_opportunistic" ) && (
>>> other.CEId != "grid-ce.physik.rwth-aachen.de:8443/cream-pbs-auger" ) && (
>>> other.CEId != "cream.egi.cesga.es:8443/cream-sge-GRIDEGI_large" ) ) ) && (
>>> other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is
>>> true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :
>>> !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs
>>> == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && (
>>> EnableWmsFeedback is true ?
>>> RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
>>> WMPInputSandboxBaseURI =
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA"
>>>
>>> ] ]; Command = "Submit" ]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,286 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de//var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> iceCommandSubmit::updateIsbList() -
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo
>>>
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303
>>>
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out
>>>
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err
>>>
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG -
>>> util::pathName::CTOR() - Trying to unparse
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,287 DEBUG - util::updateOsbList()
>>> - After input sandbox manipulation,
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2
>>>
>>> became
>>> gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,308 INFO -
>>> iceCommandSubmit::execute() - TID=[29883760] This request is a Submission
>>> for job [https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,313 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000](port
>>>
>>> is not used, actually...)
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,313 INFO -
>>> iceLBLogger::logEvent() - WMS Dequeue Event, qname=[/var/ice/jobdir] -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,689 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000000:LRMS=000000:APP=000000:LBS=000000](port
>>>
>>> is not used, actually...)
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,689 INFO -
>>> iceLBLogger::logEvent() - Cream Transfer Start Event -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,760 DEBUG -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] Submitting JDL [
>>> NodeName = "INIT_MPDProtonEpos_gr267_en19_750_th55_65_664303";
>>> CEInfoHostName = "grid-ce2.physik.rwth-aachen.de"; OutputSandboxPath =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output";
>>>
>>> StdOutput = "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out";
>>> BatchSystem = "pbs"; GlobusResourceContactString =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs"; QueueName = "auger";
>>> ShallowRetryCount = 0; RetryCount = 1; Executable = "/bin/sh"; Type = "job";
>>> LB_sequence_code =
>>> "UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000";
>>>
>>> Arguments =
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303";
>>> maxOutputSandboxSize = -1.000000000000000E+00; VirtualOrganisation =
>>> "auger"; JobType = "normal"; DefaultRank =
>>> -other.GlueCEStateEstimatedResponseTime; ce_id =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger"; OutputSandboxDestURI
>>> = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>>
>>> }; OutputSandbox = {
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>>
>>> }; InputSandbox = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/Modules.tar.bz2","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo"
>>>
>>> }; edg_jobid = "https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA";
>>> VOMS_FQAN = "/auger/Role=Production/Capability=NULL"; CertificateSubject =
>>> "/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo"; StdError =
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err"; InputSandboxPath
>>> =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input";
>>>
>>> ReallyRunningToken =
>>> "gsiftp://wms-2-kit.gridka.de/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0";
>>>
>>> rank = -other.GlueCEStateEstimatedResponseTime; WMSHostname =
>>> "wms-2-kit.gridka.de"; lrms_type = "pbs"; MyProxyServer =
>>> "myproxy1.egee.cesnet.cz"; X509UserProxy =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy";
>>>
>>> InputSandboxDestFileName = {
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","Modules.tar.bz2"
>>>
>>> }; requirements = ( ( ( ( other.GlueCEPolicyMaxCPUTime > 300 ) && (
>>> other.CEId != "cream1.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream1.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream2.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream2.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream-ce.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId
>>> != "cream-ce02.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "ce02.lip.pt:8443/cream-sge-gridq_x86_64" ) && ( other.CEId !=
>>> "ce03.ncg.ingrid.pt:8443/cream-sge-gridq" ) && ( other.CEId !=
>>> "tochtli64.nucleares.unam.mx:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "cccreamceli11.in2p3.fr:8443/cream-sge-long" ) && ( other.CEId !=
>>> "ce0.m3pec.u-bordeaux1.fr:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "creamce.ijs.si:8443/cream-slurm-batch" ) && ( other.CEId !=
>>> "ce01-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "ce04-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "grid012.ct.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_opportunistic" ) && (
>>> other.CEId != "grid-ce.physik.rwth-aachen.de:8443/cream-pbs-auger" ) && (
>>> other.CEId != "cream.egi.cesga.es:8443/cream-sge-GRIDEGI_large" ) ) ) && (
>>> other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is
>>> true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :
>>> !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs
>>> == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && (
>>> EnableWmsFeedback is true ?
>>> RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
>>> WMPInputSandboxBaseURI =
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA"
>>>
>>> ] to [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2]
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/gridsite-delegation]
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,760 DEBUG -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] Sequence code for job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""] is
>>> UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000
>>>
>>>
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:44,774 DEBUG -
>>> iceCommandSubmit::register_job() - TID=[29883760] Going to REGISTER Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID=""] with delegation ID [13690290782E322wms2D22Dkit2Egridka2Ede]
>>> to CREAM
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2]...
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,466 INFO -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] For GridJobID
>>> [https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] CREAM Returned
>>> CREAM-JOBID [https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997]
>>> DB_ID [1353999667000]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,469 DEBUG -
>>> iceCommandSubmit::try_to_submit() - TID=[29883760] Going to START
>>> CreamJobID [https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997]
>>> related to GridJobID
>>> [https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA]...
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,610 INFO -
>>> IceLBContext::setLoggingJob - Setting log job to
>>> jobid=[https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA] LB
>>> server=[lb-1-kit.gridka.de:9000] SEQUENCE CODE
>>> [UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000002:LM=000002:LRMS=000000:APP=000000:LBS=000000](port
>>>
>>> is not used, actually...)
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,610 INFO -
>>> iceLBLogger::logEvent() - Cream Transfer OK Event -
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> /var/log/wms/ice.log.2:2013-06-13 14:04:45,914 DEBUG -
>>> filelist_request_purger - removing request [ Source = 2; Protocol = "1.0.0";
>>> Arguments = [ JobAd = [ NodeName =
>>> "INIT_MPDProtonEpos_gr267_en19_750_th55_65_664303"; CEInfoHostName =
>>> "grid-ce2.physik.rwth-aachen.de"; OutputSandboxPath =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output";
>>>
>>> StdOutput = "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out";
>>> GlobusResourceContactString =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs"; QueueName = "auger";
>>> ShallowRetryCount = 0; RetryCount = 1; Executable = "/bin/sh"; Type = "job";
>>> LB_sequence_code =
>>> "UI=000000:NS=0000000004:WM=000006:BH=0000000000:JSS=000000:LM=000000:LRMS=000000:APP=000000:LBS=000000";
>>>
>>> Arguments =
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh
>>> SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303"; VirtualOrganisation
>>> = "auger"; JobType = "normal"; DefaultRank =
>>> -other.GlueCEStateEstimatedResponseTime; ce_id =
>>> "grid-ce2.physik.rwth-aachen.de:8443/cream-pbs-auger"; OutputSandboxDestURI
>>> = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/output/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>>
>>> }; OutputSandbox = {
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.out","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.logs.tar.bz2"
>>>
>>> }; InputSandbox = {
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/k8/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fk8d4srnXAKIbxnBwyHGMCw/input/Modules.tar.bz2","gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input/.BrokerInfo"
>>>
>>> }; edg_jobid = "https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA";
>>> VOMS_FQAN = "/auger/Role=Production/Capability=NULL"; CertificateSubject =
>>> "/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo"; StdError =
>>> "INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.job.err"; InputSandboxPath
>>> =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/input";
>>>
>>> ReallyRunningToken =
>>> "gsiftp://wms-2-kit.gridka.de/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/token.txt_0";
>>>
>>> rank = -other.GlueCEStateEstimatedResponseTime; lrms_type = "pbs";
>>> MyProxyServer = "myproxy1.egee.cesnet.cz"; X509UserProxy =
>>> "/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA/user.proxy";
>>>
>>> InputSandboxDestFileName = {
>>> "input.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303.CORSIKA-7370.sh","INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","SEEDS.INIT.MPDProtonEpos_gr267_en19.750_th55.65_664303","Modules.tar.bz2"
>>>
>>> }; requirements = ( ( ( ( other.GlueCEPolicyMaxCPUTime > 300 ) && (
>>> other.CEId != "cream1.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream1.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream2.farm.particle.cz:8443/cream-pbs-gridauger" ) && (
>>> other.CEId != "cream2.grid.cesnet.cz:8443/cream-pbs-egee_auger" ) && (
>>> other.CEId != "cream-ce.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId
>>> != "cream-ce02.cat.cbpf.br:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "ce02.lip.pt:8443/cream-sge-gridq_x86_64" ) && ( other.CEId !=
>>> "ce03.ncg.ingrid.pt:8443/cream-sge-gridq" ) && ( other.CEId !=
>>> "tochtli64.nucleares.unam.mx:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "cccreamceli11.in2p3.fr:8443/cream-sge-long" ) && ( other.CEId !=
>>> "ce0.m3pec.u-bordeaux1.fr:8443/cream-pbs-auger" ) && ( other.CEId !=
>>> "creamce.ijs.si:8443/cream-slurm-batch" ) && ( other.CEId !=
>>> "ce01-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "ce04-lcg.cr.cnaf.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "grid012.ct.infn.it:8443/cream-lsf-auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_auger" ) && ( other.CEId !=
>>> "cmsosgce3.fnal.gov:2119/jobmanager-condor-group_opportunistic" ) && (
>>> other.CEId != "grid-ce.physik.rwth-aachen.de:8443/cream-pbs-auger" ) && (
>>> other.CEId != "cream.egi.cesga.es:8443/cream-sge-GRIDEGI_large" ) ) ) && (
>>> other.GlueCEStateStatus == "Production" ) ) && ( ( ( ShortDeadlineJob is
>>> true ) ? RegExp(".*sdj$",other.GlueCEUniqueID) :
>>> !RegExp(".*sdj$",other.GlueCEUniqueID) ) && ( other.GlueCEPolicyMaxTotalJobs
>>> == 0 || other.GlueCEStateTotalJobs < other.GlueCEPolicyMaxTotalJobs ) && (
>>> EnableWmsFeedback is true ?
>>> RegExp("cream",other.GlueCEImplementationName,"i") : true ) );
>>> WMPInputSandboxBaseURI =
>>> "gsiftp://wms-2-kit.gridka.de:2811/var/SandboxDir/Jn/https_3a_2f_2flb-1-kit.gridka.de_3a9000_2fJncl2Kvfsp9FOtM8kDlXhA"
>>>
>>> ] ]; Command = "Submit" ]
>>> /var/log/wms/ice.log.2:2013-06-13 14:06:39,345 DEBUG -
>>> iceCommandEventQuery::processEventsForJob() - TID=[29880192] Processing [3]
>>> event(s) for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> userdn
>>> [/DC=es/DC=irisgrid/O=ugr/CN=Julio.Lozano.Bahilo-/auger/Role=Production/Capability=NULL]
>>>
>>> and ce url
>>> [https://grid-ce2.physik.rwth-aachen.de:8443/ce-cream/services/CREAM2].
>>> /var/log/wms/ice.log.2:2013-06-13 14:06:39,346 DEBUG -
>>> iceCommandEventQuery::processSingleEvent() - TID=[29880192] Updating ICE's
>>> database for Job
>>> [GRIDJobID="https://lb-1-kit.gridka.de:9000/Jncl2Kvfsp9FOtM8kDlXhA"
>>> CREAMJobID="https://grid-ce2.physik.rwth-aachen.de:8443/CREAM656117997"]
>>> status = [IDLE] exit_code = [N/A] failure_reason = [N/A] description = [N/A]
>
>
|