Hi Ewan,
These are all good questions. Lets see if I can answer them...
On 19/08/08 14:40, Ewan MacMahon wrote:
> OK so far from a user's point of view; but from the DPM's perspective,
> where is the information that a particular file is 'in' a space token
> actually stored?
If you look in the cns_db.Cns_file_replica table, you will see a column
called setname. This is what relates the file *replica* (not just the
namespace entry) with the space token. If the file is deleted, this
entry allows the space in the token to be reclaimed.
> So, if I were (say) an atlas production person I should be able to write
> a
> file to any atlas accessible part of an SE and have that file count
> against
> a space token like ATLASDATADISK?
Yes, if you specify the ATLASDATADISK token when you perform the write
operation.
> This is where I really start losing track. If I'm dpm-drain and I've got
> a
> file that's in the ATLASDATADISK space token, and I want to migrate it
> somewhere, then why don't I use the space that's reserved for the space
> token
When you dpm-drain, you are making a new file replica (of the same
namespace entry). It appears that while DPM drain makes the replica, it
is not replicating the space token information. See below...
> - it seems that dpm-drain is considering that space off-limits, when it
> shouldn't be. Secondly, if being 'in' a space token doesn't require a
> file to
> be in a particular path in the dpm namespace, then why can't dpm-drain
> use
> the general pool space, and /still/ have the file count against the
> space token?
...I think this will explain things. If I manually dpm-replicate a file
(which is basically all that dpm-drain is doing, in addition to deleting
the original replica), you get a second entry in the Cns_file_replica
table. It's clear that there is no space token (setname) information for
the second entry:
mysql> select r_type, f_type, setname, poolname, sfn from
Cns_file_replica where fileid=207275;
+--------+--------+--------------------------------------+----------+-------------------------------------------------------------------------------------------------------------------+
| r_type | f_type | setname | poolname |
sfn
|
+--------+--------+--------------------------------------+----------+-------------------------------------------------------------------------------------------------------------------+
| P | P | 074115be-7665-456c-bf36-61429fbdfd02 | atlas |
pool2.glite.ecdf.ed.ac.uk:/gridstorage007/atlas/2008-08-16/RDO.024291._00929.pool.root.1__DQ2-1218906533.490865.0
|
| S | V | | atlas |
pool2.glite.ecdf.ed.ac.uk:/gridstorage008/atlas/2008-08-20/RDO.024291._00929.pool.root.1__DQ2-1218906533.493261.0
|
+--------+--------+--------------------------------------+----------+-------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)
This is the problem that Glasgow are seeing.
> As far as I can see there's nothing in the space token creation that
> ties it
> to a particular path or pool; e.g:
>
> /opt/lcg/bin/dpm-reservespace --gspace 2T --lifetime Inf --group
> atlas/Role=production --token_desc ATLASPRODDISK
>
> seems to just create the reservation on the system as a whole, not tie
> it to
> any particular backing store.
This is incorrect. You have the --poolname option that you can use to
tie the reservation to a particular pool. Use the -h option to see the help.
> Presumably dpm-drain isn't changing the dpm namespace path (and hence
> SURL) of
> the file during this process, and if whatever metadata says that a
> particular
> file is 'in' a particular token isn't dependent on path or pool I don't
> see
> why this operation would cause an association that says that SURL 'x' is
> in
> token 'y' to suddenly disappear.
See the above explanation.
I think this clarifies everything that was discussed this morning.
Apologies if my verbal explanation wasn't the best.
Cheers,
Greig
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
|