 |
 |
Hi Wahid, it doesn't replicate when --del is used (is it supposed to work?). I rerun it without --del and with --verbose in this way it is making the copies. Still I don't think it'll be enough, yesterday I had 2 problems 1) the same file was reuested 5000 times 2 copies were not enough 2) reco jobs wreaked avoc with proddisk files Problem 2) means we need to integrate your tool with something that also looks at the hotfiles list of the day and makes copies of those too. It be nice to know which files have more than one replica independently from the space token. Do you know what mysql query or API one is supposed to use? thanks cheers alessandra On 12/04/2011 09:52, Wahid Bhimji wrote: > Hello > >> >> I have version gridpp-dpm-tools-2.6.7.DPM174-1 >> > That's good. > >> According to the output it only deleted 3 files because there were 3 >> replicas instead of 2. When I run it again it did nothing even though >> most files have a single replica. I'm running without --del now. > > I am surprised it isn't working without the --del option. > If you use the --list option does it say that there are only one > replica for most of the files. > > There is a --verbose option that gives slightly more output which may > shed some light. > I presume you have DPM_HOST, DPNS_HOST set? > > Sorry you having troubles with it. > > Wahid > >> >> cheers >> alessandra >> >> On 11/04/2011 17:28, Wahid Bhimji wrote: >>> WAIT! >>> This isn't really an answer to your question though and so likely to >>> cause confusion (!) For your answer you can skip down but I would >>> read the first bit as a warning. >>> >>> But I was just about to send a warning to the list about using this >>> tool with the --del option (interesting timing of your email). >>> >>> Basically (I have mentioned this before but maybe not stressed its >>> importance) there was a bug with the first version of my script that >>> the replicas it created were not permanent. >>> This means that if: >>> 1) You used an old version of the tool to make replicas AND >>> 2) Your run either old or new version with --del to reduce the >>> number (e.g. from 3 to 2) AND >>> 3) You run dpm-drain on a filesystem containing these replicas >>> there is a risk of removing all the replicas. >>> I want to write a tool to list if people have non-permanent replicas >>> and to mark them all permanent - so will send another post when I >>> have done that. But I just wanted to flag it here in case people are >>> in this position. >>> (If you do suspect you may have done this then let me know (and for >>> atlashotdisk) I can run an integrity check and if there are missing >>> files I can also mark them lost and they will be restored quite >>> easily). To be clear though - you need to have done all 3 above >>> otherwise there is no problem. >>> >>> So to answer you actual question ! >>> If you have never used the tool before and use the latest version >>> (which is I think your actual question) then then: >>> A) You do not need --del that just deletes from (e.g 3 to 2) so in >>> your case it will see there is not more than 2 and do nothing. >>> B) The first time it will take a long time and so I would run it >>> first outside cron. >>> C) Then once that is done it will only run on new files (those that >>> do not have 2 replicas in ATLASHOTDISK) so you can put it in a daily >>> cron. >>> >>> Let me know if this is not clear >>> (and apologies to anyone caught out by the above bug. It is actually >>> easily put right so do let me know if you are concerned and I will >>> send out another post with more details). >>> >>> Cheers >>> >>> Wahid >>> >>> >>> On 11 Apr 2011, at 17:04, Alessandra Forti wrote: >>> >>>> Hi, >>>> >>>> I'm going to try this command >>>> >>>> dpm-sql-spacetoken-replicate-hotfiles --nreps=2 --st=ATLASHOTDISK --del >>>> >>>> I'm wondering how do I catch the new files though and how often can >>>> I run it. Is it really something I can put in a cron job? Otherwise >>>> I'll go with the normal list-hotfiles and replicate only those >>>> files that have>NNN requests unless they have been already replicated. >>>> >>>> cheers >>>> alessandra >>>> >>> >> > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336.