Hi Eric,
Yes, thank you. For example
cream-ge-2-kit.gridka.de:8443/cream-sge-core1. I also thought that 0 is
disabled, but anyway why it is enables the submission every time? To
enable something one has to disable it first, otherwise it makes no
sense. But I don't see any logs about disabling the submissions.
Current load_monitor output is:
/usr/bin/glite_cream_load_monitor
/etc/glite-ce-cream-utils/glite_cream_load_monitor.conf --show
Threshold for Load Average(1 min): 40 => Detected value for Load
Average(1 min): 1.25
Threshold for Load Average(5 min): 40 => Detected value for Load
Average(5 min): 1.06
Threshold for Load Average(15 min): 20 => Detected value for Load
Average(15 min): 0.96
Threshold for Memory Usage: 95 => Detected value for Memory Usage: 19.73%
Threshold for Swap Usage: 95 => Detected value for Swap Usage: 0.00%
Threshold for Free FD: 500 => Detected value for Free FD: 1915873
Threshold for tomcat FD: 800 => Detected value for Tomcat FD: 341
Threshold for FTP Connection: 100 => Detected value for FTP Connection: 1
Threshold for Number of active jobs: -1 => Detected value for Number of
active jobs: 2120
Threshold for Number of pending commands: -1 => Detected value for
Number of pending commands: 0
Threshold for Disk Usage: 95% => Detected value for Partition /tmp : 4%
Threshold for Disk Usage: 95% => Detected value for Partition /var : 4%
Threshold for Disk Usage: 95% => Detected value for Partition /opt : 5%
Threshold for Disk Usage: 95% => Detected value for Partition / : 20%
Cheers,
Pavel
On 12/19/2012 03:30 PM, Eric Frizziero wrote:
> On 12/19/2012 03:20 PM, Eric Frizziero wrote:
>> Hi Pavel,
>>
>>
>> On 12/19/2012 02:56 PM, Pavel Weber wrote:
>>> Hi all,
>>>
>>> On all our creams at KIT in logs we see every 10 min. e.g.:
>>>
>>> 19 Dec 2012 13:27:57,813
>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>> submissionEnabled to 0 in the database
>>> 19 Dec 2012 13:37:57,912
>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>> submissionEnabled to 0 in the database
>>> 19 Dec 2012 13:47:58,016
>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>> submissionEnabled to 0 in the database
>>> 19 Dec 2012 13:57:58,032
>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>> submissionEnabled to 0 in the database
>>> 19 Dec 2012 14:07:58,125
>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>> submissionEnabled to 0 in the database
>>> 19 Dec 2012 14:17:58,197
>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>> submissionEnabled to 0 in the database
>>>
>>> What does it mean? Unfortunately I can't find any further
>>> information in each case. The line appears e.g. after job info:
>> Every 10 minutes the glite_cream_load_monitor is executed. it disables
>> the submission for some reason.
> Sorry ... It enables the submission.
> ---------------
> SUBMISSION_ENABLED = 0
> SUBMISSION_DISABLED_BY_LIMITER = 1
> SUBMISSION_DISABLED_BY_ADMIN = 2
> ---------------
>
> Cheers,
> E r i c.
>
>> Could you provide me a cream uri?
>>
>> Cheers,
>> E r i c.
>>
>>> 19 Dec 2012 14:36:54,520
>>> org.glite.ce.creamapi.jobmanagement.cmdexecutor.AbstractJobExecutor -
>>> ID=224153; NAME="JOB_START"; PRIORITY_LEVEL=1; IS_ASYNCHRONOUS=true;
>>> STATUS=EXECUTING; CATEGORY="JOB_MANAGEMENT";
>>> USER_ID="CN_cmspilotjob_vocms143_cern_ch_OU_computers_DC_cern_DC_ch_cms_Role_production_Capability_NULL";
>>> CREATION_TIME="Wed Dec 19 14:36:34 CET 2012";
>>> START_PROCESSING_TIME="Wed Dec 19 14:36:35 CET 2012";
>>> JOB_ID_LIST="CREAM875562543"; IS_ADMIN="false";
>>> REMOTE_REQUEST_ADDRESS="128.142.194.72";
>>> USER_DN="CN=cmspilotjob/vocms143.cern.ch,OU=computers,DC=cern,DC=ch";
>>> USER_FQAN={ /cms/Role=production/Capability=NULL;
>>> /cms/Role=NULL/Capability=NULL } lrmsAbsJobId=sge/20121219143636/893739;
>>> 19 Dec 2012 14:36:54,716
>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>> submissionEnabled to 0 in the database
>>>
>>> Although we see also the submissions disabled from time to time but
>>> doing glite-ce-allowed-submission test say exactly at 14:17:58 when
>>> the submissionEnabled=0 reported doesn't show that the cream is
>>> disabled. Or it goes so fast back and forth that 1s check I did is
>>> not enough to sport it? Thank you.
>>>
>>> Cheers,
>>>
>>> Pavel
>>>
>>>
>>>
>>> On 12/19/2012 02:34 PM, Edith Knoops wrote:
>>>> On 12/17/2012 11:18 PM, Maarten Litmaath wrote:
>>>>> Hi all,
>>>>>
>>>>>>> Yes, but I currently don't have a "feeling" e.g. what are good
>>>>>>> values
>>>>>>> for a "standard Tier2 CE" (e.g. maximum number of concurrent gridftp
>>>>>>> connections).
>>>>>> As a point of information, at UKI-LT2-QMUL, we were regularly hitting
>>>>>> the limit of 30. I've just increased it to 60 (which we still hit)
>>>>> Try increasing that further, say to 100.
>>>>>
>>>>> BTW, there also could be stale GridFTP processes that should be
>>>>> killed;
>>>>> on EMI-2 there is a cron job for that provided by the
>>>>> kill-stale-ftp rpm.
>>>>>
>>>> Hi,
>>>>
>>>> We had also the problem of cream disabled due to number of gridftp
>>>> session. We found out that the GLOBUS_PORT_RANGE was not fully
>>>> opened on
>>>> this cream resulting on gridftp failing on timeout (but lasting
>>>> quite long).
>>>> After correct port opening, the problem was solved. Probably not the
>>>> problem in your case but it is always worth to check.
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Edith
>>>>
>>>
>>
|