Hi Eric,
Thank you. At least we don't need to worry about. But this line is
anyway a bit misleading firstly with number 0 and secondly that it is
there at all. For me this line makes sense only when the state of job
submission changes. Thanks again for help and explanations.
Cheers,
Pavel
On 12/19/2012 04:14 PM, Eric Frizziero wrote:
> On 12/19/2012 03:45 PM, Pavel Weber wrote:
>> Hi Eric,
>>
>> Yes, thank you. For example
>> cream-ge-2-kit.gridka.de:8443/cream-sge-core1. I also thought that 0
>> is disabled, but anyway why it is enables the submission every time?
>> To enable something one has to disable it first, otherwise it makes no
>> sense. But I don't see any logs about disabling the submissions.
> The log message "org.glite.ce.creamapi.jobmanagement.db.DBInfoManager -
> set submissionEnabled to 0 in the database" doesn't mean that the job
> submissions are enabled again but it means that the job submissions
> continue to be enabled also after the execution of the script
> "glite_cream_load_monitor".
>
> glite-ce-allowed-submission cream-ge-2-kit.gridka.de:8443
> Job Submission to this CREAM CE is enabled
>
> Cheers,
> E r i c.
>
>
>> Current load_monitor output is:
>> /usr/bin/glite_cream_load_monitor
>> /etc/glite-ce-cream-utils/glite_cream_load_monitor.conf --show
>> Threshold for Load Average(1 min): 40 => Detected value for Load
>> Average(1 min): 1.25
>> Threshold for Load Average(5 min): 40 => Detected value for Load
>> Average(5 min): 1.06
>> Threshold for Load Average(15 min): 20 => Detected value for Load
>> Average(15 min): 0.96
>> Threshold for Memory Usage: 95 => Detected value for Memory Usage: 19.73%
>> Threshold for Swap Usage: 95 => Detected value for Swap Usage: 0.00%
>> Threshold for Free FD: 500 => Detected value for Free FD: 1915873
>> Threshold for tomcat FD: 800 => Detected value for Tomcat FD: 341
>> Threshold for FTP Connection: 100 => Detected value for FTP Connection: 1
>> Threshold for Number of active jobs: -1 => Detected value for Number
>> of active jobs: 2120
>> Threshold for Number of pending commands: -1 => Detected value for
>> Number of pending commands: 0
>> Threshold for Disk Usage: 95% => Detected value for Partition /tmp : 4%
>> Threshold for Disk Usage: 95% => Detected value for Partition /var : 4%
>> Threshold for Disk Usage: 95% => Detected value for Partition /opt : 5%
>> Threshold for Disk Usage: 95% => Detected value for Partition / : 20%
>>
>>
>> Cheers,
>>
>> Pavel
>>
>>
>> On 12/19/2012 03:30 PM, Eric Frizziero wrote:
>>> On 12/19/2012 03:20 PM, Eric Frizziero wrote:
>>>> Hi Pavel,
>>>>
>>>>
>>>> On 12/19/2012 02:56 PM, Pavel Weber wrote:
>>>>> Hi all,
>>>>>
>>>>> On all our creams at KIT in logs we see every 10 min. e.g.:
>>>>>
>>>>> 19 Dec 2012 13:27:57,813
>>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>>> submissionEnabled to 0 in the database
>>>>> 19 Dec 2012 13:37:57,912
>>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>>> submissionEnabled to 0 in the database
>>>>> 19 Dec 2012 13:47:58,016
>>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>>> submissionEnabled to 0 in the database
>>>>> 19 Dec 2012 13:57:58,032
>>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>>> submissionEnabled to 0 in the database
>>>>> 19 Dec 2012 14:07:58,125
>>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>>> submissionEnabled to 0 in the database
>>>>> 19 Dec 2012 14:17:58,197
>>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>>> submissionEnabled to 0 in the database
>>>>>
>>>>> What does it mean? Unfortunately I can't find any further
>>>>> information in each case. The line appears e.g. after job info:
>>>> Every 10 minutes the glite_cream_load_monitor is executed. it disables
>>>> the submission for some reason.
>>> Sorry ... It enables the submission.
>>> ---------------
>>> SUBMISSION_ENABLED = 0
>>> SUBMISSION_DISABLED_BY_LIMITER = 1
>>> SUBMISSION_DISABLED_BY_ADMIN = 2
>>> ---------------
>>>
>>> Cheers,
>>> E r i c.
>>>
>>>> Could you provide me a cream uri?
>>>>
>>>> Cheers,
>>>> E r i c.
>>>>
>>>>> 19 Dec 2012 14:36:54,520
>>>>> org.glite.ce.creamapi.jobmanagement.cmdexecutor.AbstractJobExecutor -
>>>>> ID=224153; NAME="JOB_START"; PRIORITY_LEVEL=1; IS_ASYNCHRONOUS=true;
>>>>> STATUS=EXECUTING; CATEGORY="JOB_MANAGEMENT";
>>>>> USER_ID="CN_cmspilotjob_vocms143_cern_ch_OU_computers_DC_cern_DC_ch_cms_Role_production_Capability_NULL";
>>>>>
>>>>> CREATION_TIME="Wed Dec 19 14:36:34 CET 2012";
>>>>> START_PROCESSING_TIME="Wed Dec 19 14:36:35 CET 2012";
>>>>> JOB_ID_LIST="CREAM875562543"; IS_ADMIN="false";
>>>>> REMOTE_REQUEST_ADDRESS="128.142.194.72";
>>>>> USER_DN="CN=cmspilotjob/vocms143.cern.ch,OU=computers,DC=cern,DC=ch";
>>>>> USER_FQAN={ /cms/Role=production/Capability=NULL;
>>>>> /cms/Role=NULL/Capability=NULL }
>>>>> lrmsAbsJobId=sge/20121219143636/893739;
>>>>> 19 Dec 2012 14:36:54,716
>>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>>> submissionEnabled to 0 in the database
>>>>>
>>>>> Although we see also the submissions disabled from time to time but
>>>>> doing glite-ce-allowed-submission test say exactly at 14:17:58 when
>>>>> the submissionEnabled=0 reported doesn't show that the cream is
>>>>> disabled. Or it goes so fast back and forth that 1s check I did is
>>>>> not enough to sport it? Thank you.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Pavel
>>>>>
>>>>>
>>>>>
>>>>> On 12/19/2012 02:34 PM, Edith Knoops wrote:
>>>>>> On 12/17/2012 11:18 PM, Maarten Litmaath wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>>>> Yes, but I currently don't have a "feeling" e.g. what are good
>>>>>>>>> values
>>>>>>>>> for a "standard Tier2 CE" (e.g. maximum number of concurrent
>>>>>>>>> gridftp
>>>>>>>>> connections).
>>>>>>>> As a point of information, at UKI-LT2-QMUL, we were regularly
>>>>>>>> hitting
>>>>>>>> the limit of 30. I've just increased it to 60 (which we still hit)
>>>>>>> Try increasing that further, say to 100.
>>>>>>>
>>>>>>> BTW, there also could be stale GridFTP processes that should be
>>>>>>> killed;
>>>>>>> on EMI-2 there is a cron job for that provided by the
>>>>>>> kill-stale-ftp rpm.
>>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We had also the problem of cream disabled due to number of gridftp
>>>>>> session. We found out that the GLOBUS_PORT_RANGE was not fully
>>>>>> opened on
>>>>>> this cream resulting on gridftp failing on timeout (but lasting
>>>>>> quite long).
>>>>>> After correct port opening, the problem was solved. Probably not the
>>>>>> problem in your case but it is always worth to check.
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Edith
>>>>>>
>>
|