Hello again chaps!
We've got (another) oddity with our dcache, spooted by brian this
morning. A pnfs directory, belonging to the dzero VO, has disappeared
without a trace. The database entries are still there, but the
directory has dissappeared and when we do an ls:
[root@fal-pygrid-20 root]# ls /pnfs/lancs.ac.uk/data/
ls: /pnfs/lancs.ac.uk/data/dzero: No such file or directory
alice babar cdf dteam hone lhcb ops sixt testf zeus
atlas biomed cms fusion ilc magic planck t2k zerolengthfiles
The fact that ls tries to tell us about dzero not existing is
interesting. A mdb show shows the dzero database
[root@fal-pygrid-20 root]# mdb show
*snip*
8 dzero r enabled (r) /opt/pnfsdb/pnfs/databases/dzero
I trawled every billing and log file I know of and could not find any
reference to an action that would lead to the directories
disappearance. I also grepped the history in case I logged on in my
sleep last night and went on a rm-ing spree, but luckily that wasn't
the case. When I try to recreate the directory I get an error:
[root@fal-pygrid-20 root]# mkdir -p "/pnfs/lancs.ac.uk/data/.(8)(dzero)"
mkdir: cannot create directory `/pnfs/lancs.ac.uk/data/.(8)(dzero)': File exists
Luckily the dzero directory was empty and we're not being used yet by
dzero. This could have been a lot worse. Any ideas how/why it
happened? It's got me worried.
cheers,
Matt
|