Hi,

yesterday I attended the pre-GDB and I the developers were there. They 
claim nobody ever told them the requirements. I insisted on having an 
accounting TF that covers a number of "loose threads" among which 
getting together a list of requirements for the new portal and interact 
with the EGI developers. So this is what it is going to be proposed 
today. Whether closer interaction will speed up the process or not I 
cannot say. It will also be aimed at WLCCG requirements so if gridpp has 
others either we try to get them integrated or we do our thing. The fact 
that the developers were present at the meeting was positive and I did 
say twice in the meeting and even wrote it in the ticket that this 
situation has been going on for two years.

cheers
alessandra

On 13/04/2016 08:03, Andrew McNab wrote:
>> On 8 Apr 2016, at 12:01, Alessandra Forti <[log in to unmask]> wrote:
>>
>> Hi Steve,
>>
>> On 07/04/2016 23:01, Steve Lloyd wrote:
>>> Hi Alessandra,
>>>    Not sure if you are referring to the production or dev version
>> I started my feedback from the production version and the old development version and then the new development version link was distributed a day before the meeting. EGI might have felt the temperature rising in WLCG about waiting 2 years for a new version while having any development blocked on the old one. This new version is not even close to be of production quality TBH.
> I’ve just been looking through it all after Robin’s comments yesterday about the difficulty of seeing what has actually been put in the APEL database in a fine grained way (“I can’t see if Vac is working!”). In February Robin and I did verify that accounting records were getting through from Vac at Lancaster to the APEL database and visible in the accounting portal. But the way to do this in the portal has to be rather contrived: you have to pick out a “User” with a DN like /DC=uk/DC=ac/DC=lancs/DC=hec/DC=pygrid-vac by hovering over the list in the Site Admin page one by one. This works for the records from Vac because Vac forces the “User” DNs to be something you can select in the portal, but for other finer grained searches (“Show me the efficiency of all multiprocessor jobs on cream-ce01”) you can’t access what the database has.
>
> Given that the portal hasn’t improved for years, I would be suggesting we do something ourselves at this point as (Ewan mode) it’s not rocket science to just stick additional copies of the records into a central GridPP ElasticSearch instance and make our own web pages with tables. Or ask for APEL database dumps for GridPP sites that we can present ourselves. Except that more promises will be made at the GDB today, and we should see what happens with that. How long do we wait though?
>
> Cheers
>
>   Andrew
>
>
>>>   but in the production version the Tier2 view returns nothing for April 2016 whereas the Countries View does. Does the new version still use the stupid optval=1.63 type parameters to get the UK which suddenly gives you Ukraine because they changed the UK to 64 or something? In the Tier2 view the UK is 1.36 whereas in Country view it's 1.63!
>> I think the prod version hasn't be changed in a while and the numbers rather than being fixed maybe a DB index which changes if the number of items in the list changes or the order of insertion is different. That's why if you change view the UK has different numbers. Other site complained about the API being really manual creation of the URL. I can add this to the list of things they shouldn't be doing in the new portal.
>>
>> cheers
>> alessandra
>>>      Cheers
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Prof Steve Lloyd
>>> Head of School of Physics and Astronomy
>>> Queen Mary University of London, Mile End Road, London E1 4NS, UK
>>> E-mail: [log in to unmask] Phone: +44-(0)207-882-6967
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>> On 6 Apr 2016, at 15:44, Alessandra Forti <[log in to unmask]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I mentioned this at the ops meeting yesterday but tomorrow there will be a discussion about accounting at the WLCG Ops Coord in preparation for the pre-GDB dedicated to the same topic. There are 6 points feedback was requested on, I list them below with my comments
>>>>
>>>> 1) Accounting portal
>>>>
>>>> * Discrepancies among different views EMI3(WLCG) and T1 and T2 view has been added with multicore accounting, some other views like T1 and T2 have it but not others.
>>>>    * Efficiencies seem displaced between EGI and EMI3(WLCG) other views may have the same problem.
>>>>    * Difficult to related the numbers to site views and users less selection possibilities
>>>> * API (csv_export.php) not well documented, difficult to check number programmatically
>>>>    * Need to create the view first and manipulate the URL after.
>>>>    * REST API mentioned in the past but not there
>>>> * Probably a lot of this would be solved by the development portal but also the deveopment portal shows discrepancies
>>>>
>>>> 2) Accounting Reports
>>>>
>>>> * Already discussed at length after the WLCG workshop. A presentation was given at the February MB [1] requesting
>>>>    * To use wallclock and compare walltime with walltime. [at the moment cputime is still used and cpu efficiency factors were removed 2 years ago
>>>>    * In addition T1 and T2 monthly reports show the CPU accountng values (both CPUtme and WALLtme) using the same units:     HEPSPEC06-days
>>>>    * cpu efficiency (cputime/walltime) and cpu occupancy (cpu_usage/cpu_pledged) would help to understand how effectively a site is used
>>>> * In addition it was suggested to setup an accounting TF to deal with other
>>>> [1] https://indico.cern.ch/event/459359/contribution/4/attachments/1229392/1801384/20160216_Accounting_Tier1s.pdf
>>>>
>>>> 3) Multicore accounting
>>>>
>>>> * All sites - bar a handful - are accounting for multicore walltime. The only problem now is displaying it correctly on all the views in the accounting portal.
>>>>
>>>> 4) Opportunistic resources
>>>>
>>>> * Virtualised WN can report in a similar way as the normal WNs
>>>> * Non EGI cloud resources and HPC resources support is not there and needs agreement on how to benchmark them.
>>>>    * Need a TF (benchmark or accounting)
>>>>
>>>> 5) Space accounting
>>>>
>>>> * John G.? Sam S.? I know there is something there (even tried to test it last year) but you are currently more involved
>>>>
>>>> 6) REBUS
>>>>
>>>> * Reports pledges correctly as they are inserted manually
>>>> * Reported capacity depends very much on what sites publish in the BDII and it is not always correct nor validated by sites.
>>>>     * Inserting available capacity manually every month was proposed in some places but while this maybe ok for T1s, for T2s needs to be thought out.
>>>>
>>>> Let me know if you have any comment. In particular about storage.
>>>>
>>>> cheers
>>>> alessandra
>>>>
>>>> -- 
>>>> Respect is a rational process. \\//
>>>> Fatti non foste a viver come bruti (Dante)
>>>>
>> -- 
>> Respect is a rational process. \\//
>> Fatti non foste a viver come bruti (Dante)
> Cheers
>
>   Andrew
>
> --
> Dr Andrew McNab
> University of Manchester High Energy Physics,
> LHCb@CERN (Distributed Computing Coordinator),
> and GridPP (LHCb + Tier-2 Evolution)
> www.hep.manchester.ac.uk/u/mcnab
> Skype: andrew.mcnab.uk

-- 
Respect is a rational process. \\//
Fatti non foste a viver come bruti (Dante)