Hi Wahid,

it doesn't replicate when --del is used (is it supposed to work?). I rerun it without --del and with --verbose in this way it is making the copies. Still I don't think it'll be enough, yesterday I had 2 problems

1) the same file was reuested 5000 times 2 copies were not enough
2) reco jobs wreaked avoc with proddisk files

Problem 2) means we need to integrate your tool with something that also looks at the hotfiles list of the day and makes copies of those too.

It be nice to know which files have more than one replica independently from the space token. Do you know what mysql query or API one is supposed to use?

thanks

cheers
alessandra


On 12/04/2011 09:52, Wahid Bhimji wrote:
[log in to unmask]" type="cite"> Hello 


I have version gridpp-dpm-tools-2.6.7.DPM174-1

That's good.

According to the output it only deleted 3 files because there were 3 replicas instead of 2. When I run it again it did nothing even though most files have a single replica. I'm running without --del now.

I am surprised it isn't working without the --del option. 
If you use the --list option does it say that there are only one replica for most of the files. 

There is a --verbose option that gives slightly more output which may shed some light. 
I presume you have DPM_HOST, DPNS_HOST set? 

Sorry you having troubles with it. 

Wahid


cheers
alessandra

On 11/04/2011 17:28, Wahid Bhimji wrote:
WAIT!
This isn't really an answer to your question though and so likely to cause confusion (!) For your answer you can skip down but I would read the first bit as a warning.

But I was just about to send a warning to the list about using this tool with the --del option (interesting timing of your email).

Basically (I have mentioned this before but maybe not stressed its importance) there was a bug with the first version of my script that the replicas it created were not permanent.
This means that if:
1) You used an old version of the tool to make replicas AND
2) Your run either old or new version with --del to reduce the number (e.g. from 3 to 2) AND
3) You run dpm-drain on a filesystem containing these replicas
there is a risk of removing all the replicas.
I want to write a tool to list if people have non-permanent replicas and to mark them all permanent - so will send another post when I have done that. But I just wanted to flag it here in case people are in this position.
(If you do suspect you may have done this then let me know (and for atlashotdisk) I can run an integrity check and if there are missing files I can also mark them lost and they will be restored quite easily). To be clear though - you need to have done all 3 above otherwise there is no problem.

So to answer you actual question !
If you have never used the tool before and use the latest version (which is I think your actual question) then then:
A) You do not need --del that just deletes from (e.g 3 to 2) so in your case it will see there is not more than 2 and do nothing.
B) The first time it will take a long time and so I would run it first outside cron.
C) Then once that is done it will only run on new files (those that do not have 2 replicas in ATLASHOTDISK) so you can put it in a daily cron.

Let me know if this is not clear
(and apologies to anyone caught out by the above bug. It is actually easily put right so do let me know if you are concerned and I will send out another post with more details).

Cheers

Wahid


On 11 Apr 2011, at 17:04, Alessandra Forti wrote:

Hi,

I'm going to try this command

dpm-sql-spacetoken-replicate-hotfiles --nreps=2 --st=ATLASHOTDISK --del

I'm wondering how do I catch the new files though and how often can I run it. Is it really something I can put in a cron job? Otherwise I'll go with the normal list-hotfiles and replicate only those files that have>NNN requests unless they have been already replicated.

cheers
alessandra





The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.