Hi
i won't even attempt to address data manglement, but let me try workload:
at the CE level the relevant number of accounts is not the WNs but as
someone said earlier (Tim??) the total number of jobs running + queued
on your site. If we can have the account mapper assign an existing unix
group to each pool account, I see how we can do the fair-share
scheduling. Without unix group/VO maps, I don't see how but I haven't
thought about it very hard either.
for the RB the situation is quite different. i am not sure how many
accounts you'd need there. suppose you had the policy of keeping all
data on the RB for two weeks to allow people to come back from vacation
and still get their job output ... given the current throughput of the
RB I get an upper limit of about a hundred thousand accounts if it
really is a single account per job.
One could get around this if the sandbox storage area on the RB was
moved from the current system (that essentially looks like a classic SE)
to something that looks more like SRM. In that case I am not sure that
pool accounts, or accounts at all for that matter, are needed on the RB.
JT
Alessandra Forti wrote:
> Hi,
>
> will the job wrapper cleanup after the job *automatically*? If yes, will
> it be able to know what the job has left behind also outside the account
> home dir? Will it work on CE and WNs?
>
> What happens to the account after that? will it be reassigned to the
> same DN? will it be completely released? Oscar reply wasn't that clear.
>
> Will this scheme simplify the log files? How?
>
> Do you think it is a scalable solution?
>
> Shouldn't all of the above work anyway with a smaller number of accounts?
>
> With fairshare none of the VOs should be able to use the whole number of
> cpus, isn't the estimate a bit excessive?
>
> There is a difference between RBs/CEs/SEs/WNs needs.
>
> Assuming everything gets cleaned up and the scheme works perfectly and
> what becomes important the association between DN and job, and not
> anymore the unix account. We can potentially create only a number of
> accounts equal to the number of job slots on each WNs which means in the
> case of dual cpus without hyperthreads 2 per VOs: atlas01, atlas02;
> lhcb01, lhcb02;.....
>
> If the system could hadle it wouldn't it be a much better solution?
>
> cheers
> alessandra
>
> --
> ********************************************
> * Dr Alessandra Forti *
> * Technical Coordinator - NorthGrid Tier2 *
> * http://www.hep.man.ac.uk/u/aforti *
> ********************************************
|