-------- Original Message --------
Subject: Re: Fwd: Dark Data identification tools
Date: Tue, 06 Aug 2013 10:33:31 +0200
From: Franck Michel <[log in to unmask]>
To: Christopher J. Walker <[log in to unmask]>
Dear Chris,
Long time since we talked about those questions (dates back to Munich!).
At last we tackle those issues, in the VAPOR mini-project that I'm leading.
From the link you gave me below dedicated to Atlas, and that other one
<https://twiki.cern.ch/twiki/bin/view/LCG/ConsistencyChecksSEsDumps>
that summarises info about other LHC VOs, and from discussions I had
with different people, I came to know about 2 scripts to dump the
content of a DPM SE to an xml file :
- that one for ATLAS:
https://svnweb.cern.ch/cern/wsvn/adcops/ddmscripts/dark_data_cleanup/dpm_dump.py
- that one for CMS:
http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/UserCode/StorageDumps/DPM/dpmdump.py?view=markup
Both are quite close in logic but with somehow different options. Do you
know them? Did you ever have to use them?
My concern is this: for biomed (for now, but other VOs later) we'd like
to experiment how we can setup the procedure with sites, but I guess it
is better to come with a script ready to work. Then we'll how this can
be managed: in particular how to do this without overloading site admins
with requests from lots of VOs.
Your inputs and ideas are welcome. And btw, could you run one of those
scripts on your SE for biomed, so that I can validate the procedure from
A to Z?
Thx in advance for your help. Have a nice vacation if this is not done yet!
Franck.
Le 30/03/2012 21:07, Christopher J. Walker a écrit :
>
>
> -------- Original Message --------
> Subject: Dark Data identification tools
> Resent-From: <[log in to unmask]>
> Date: Fri, 30 Mar 2012 11:29:24 +0200
> From: Cedric Serfon <[log in to unmask]>
> To: <[log in to unmask]>
>
> Dear cloud squads,
>
> We recently moved many sites to LFC central registration on the Panda
> server. For these sites, we use now the long SURL (i.e. with
> :port/srm/managerv2?SFN=). This seems to generate some problems with the
> tools to detect Dark Data described in this Wiki :
> https://twiki.cern.ch/twiki/bin/view/Atlas/DDMOperationsScripts#Tools_to_detect_Dark_data
>
> , in particular it identifies as Dark Data some files that are properly
> registered in the LFC. Please don't use these tools anymore till a fix
> is provided (hopefully soon).
>
> Cheers,
> Cedric
>
|