On 12/19/2012 03:45 PM, Pavel Weber wrote:
> Hi Eric,
>
> Yes, thank you. For example
> cream-ge-2-kit.gridka.de:8443/cream-sge-core1. I also thought that 0
> is disabled, but anyway why it is enables the submission every time?
> To enable something one has to disable it first, otherwise it makes no
> sense. But I don't see any logs about disabling the submissions.
The log message "org.glite.ce.creamapi.jobmanagement.db.DBInfoManager -
set submissionEnabled to 0 in the database" doesn't mean that the job
submissions are enabled again but it means that the job submissions
continue to be enabled also after the execution of the script
"glite_cream_load_monitor".
glite-ce-allowed-submission cream-ge-2-kit.gridka.de:8443
Job Submission to this CREAM CE is enabled
Cheers,
E r i c.
>
> Current load_monitor output is:
> /usr/bin/glite_cream_load_monitor
> /etc/glite-ce-cream-utils/glite_cream_load_monitor.conf --show
> Threshold for Load Average(1 min): 40 => Detected value for Load
> Average(1 min): 1.25
> Threshold for Load Average(5 min): 40 => Detected value for Load
> Average(5 min): 1.06
> Threshold for Load Average(15 min): 20 => Detected value for Load
> Average(15 min): 0.96
> Threshold for Memory Usage: 95 => Detected value for Memory Usage: 19.73%
> Threshold for Swap Usage: 95 => Detected value for Swap Usage: 0.00%
> Threshold for Free FD: 500 => Detected value for Free FD: 1915873
> Threshold for tomcat FD: 800 => Detected value for Tomcat FD: 341
> Threshold for FTP Connection: 100 => Detected value for FTP Connection: 1
> Threshold for Number of active jobs: -1 => Detected value for Number
> of active jobs: 2120
> Threshold for Number of pending commands: -1 => Detected value for
> Number of pending commands: 0
> Threshold for Disk Usage: 95% => Detected value for Partition /tmp : 4%
> Threshold for Disk Usage: 95% => Detected value for Partition /var : 4%
> Threshold for Disk Usage: 95% => Detected value for Partition /opt : 5%
> Threshold for Disk Usage: 95% => Detected value for Partition / : 20%
>
>
> Cheers,
>
> Pavel
>
>
> On 12/19/2012 03:30 PM, Eric Frizziero wrote:
>> On 12/19/2012 03:20 PM, Eric Frizziero wrote:
>>> Hi Pavel,
>>>
>>>
>>> On 12/19/2012 02:56 PM, Pavel Weber wrote:
>>>> Hi all,
>>>>
>>>> On all our creams at KIT in logs we see every 10 min. e.g.:
>>>>
>>>> 19 Dec 2012 13:27:57,813
>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>> submissionEnabled to 0 in the database
>>>> 19 Dec 2012 13:37:57,912
>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>> submissionEnabled to 0 in the database
>>>> 19 Dec 2012 13:47:58,016
>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>> submissionEnabled to 0 in the database
>>>> 19 Dec 2012 13:57:58,032
>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>> submissionEnabled to 0 in the database
>>>> 19 Dec 2012 14:07:58,125
>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>> submissionEnabled to 0 in the database
>>>> 19 Dec 2012 14:17:58,197
>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>> submissionEnabled to 0 in the database
>>>>
>>>> What does it mean? Unfortunately I can't find any further
>>>> information in each case. The line appears e.g. after job info:
>>> Every 10 minutes the glite_cream_load_monitor is executed. it disables
>>> the submission for some reason.
>> Sorry ... It enables the submission.
>> ---------------
>> SUBMISSION_ENABLED = 0
>> SUBMISSION_DISABLED_BY_LIMITER = 1
>> SUBMISSION_DISABLED_BY_ADMIN = 2
>> ---------------
>>
>> Cheers,
>> E r i c.
>>
>>> Could you provide me a cream uri?
>>>
>>> Cheers,
>>> E r i c.
>>>
>>>> 19 Dec 2012 14:36:54,520
>>>> org.glite.ce.creamapi.jobmanagement.cmdexecutor.AbstractJobExecutor -
>>>> ID=224153; NAME="JOB_START"; PRIORITY_LEVEL=1; IS_ASYNCHRONOUS=true;
>>>> STATUS=EXECUTING; CATEGORY="JOB_MANAGEMENT";
>>>> USER_ID="CN_cmspilotjob_vocms143_cern_ch_OU_computers_DC_cern_DC_ch_cms_Role_production_Capability_NULL";
>>>>
>>>> CREATION_TIME="Wed Dec 19 14:36:34 CET 2012";
>>>> START_PROCESSING_TIME="Wed Dec 19 14:36:35 CET 2012";
>>>> JOB_ID_LIST="CREAM875562543"; IS_ADMIN="false";
>>>> REMOTE_REQUEST_ADDRESS="128.142.194.72";
>>>> USER_DN="CN=cmspilotjob/vocms143.cern.ch,OU=computers,DC=cern,DC=ch";
>>>> USER_FQAN={ /cms/Role=production/Capability=NULL;
>>>> /cms/Role=NULL/Capability=NULL }
>>>> lrmsAbsJobId=sge/20121219143636/893739;
>>>> 19 Dec 2012 14:36:54,716
>>>> org.glite.ce.creamapi.jobmanagement.db.DBInfoManager - set
>>>> submissionEnabled to 0 in the database
>>>>
>>>> Although we see also the submissions disabled from time to time but
>>>> doing glite-ce-allowed-submission test say exactly at 14:17:58 when
>>>> the submissionEnabled=0 reported doesn't show that the cream is
>>>> disabled. Or it goes so fast back and forth that 1s check I did is
>>>> not enough to sport it? Thank you.
>>>>
>>>> Cheers,
>>>>
>>>> Pavel
>>>>
>>>>
>>>>
>>>> On 12/19/2012 02:34 PM, Edith Knoops wrote:
>>>>> On 12/17/2012 11:18 PM, Maarten Litmaath wrote:
>>>>>> Hi all,
>>>>>>
>>>>>>>> Yes, but I currently don't have a "feeling" e.g. what are good
>>>>>>>> values
>>>>>>>> for a "standard Tier2 CE" (e.g. maximum number of concurrent
>>>>>>>> gridftp
>>>>>>>> connections).
>>>>>>> As a point of information, at UKI-LT2-QMUL, we were regularly
>>>>>>> hitting
>>>>>>> the limit of 30. I've just increased it to 60 (which we still hit)
>>>>>> Try increasing that further, say to 100.
>>>>>>
>>>>>> BTW, there also could be stale GridFTP processes that should be
>>>>>> killed;
>>>>>> on EMI-2 there is a cron job for that provided by the
>>>>>> kill-stale-ftp rpm.
>>>>>>
>>>>> Hi,
>>>>>
>>>>> We had also the problem of cream disabled due to number of gridftp
>>>>> session. We found out that the GLOBUS_PORT_RANGE was not fully
>>>>> opened on
>>>>> this cream resulting on gridftp failing on timeout (but lasting
>>>>> quite long).
>>>>> After correct port opening, the problem was solved. Probably not the
>>>>> problem in your case but it is always worth to check.
>>>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Edith
>>>>>
>>>>
>>>
>
>
|