Hi Teng,
Is your cache hit rate on files or blocks? Atlas and CMS data access patterns might be very different, but when I looked at this for CMS the reads were very sparse, I think I worked out that if the jobs had been reading 100% of the files mentioned in the cache logs we would have been more than saturating a 20Gb/s link whereas the actual rate was more like 200Mb/s, so, at least for CMS a 40% cache hit rate would slip to as low as 0.4%. But of course CMS AAA reads are not local data reads. If the Atlas jobs are reading most or all of the file, the xrootd cache setup where it sucks in the rest of the file once you start reading any of it (while prioritising blocks you request) could do very well.
Yours,
Chris.
On 05/07/2018, 15:23, "Teng Li" <[log in to unmask]> wrote:
Hi Sam,
Thanks that's much appreciated.
I'm trying to avoid that we present opposite opinions in back to back
talks :) as I'm going to say that XCache is an efficient network
amplificator for ATLAS (cache hit rate on data files could reach over
40% with optimised config).
Cheers,
Teng
Quoting Sam Skipsey <[log in to unmask]> on Thu, 5 Jul 2018
15:11:44 +0100:
> Hi Teng,
>
> Sure, but I don't want to give your talk before you give it ;)
>
> The reason I'm mentioning that your cache is a local one, in network terms,
> is to contrast it with Chris' results (which are explicitly a cache of (CMS
> AAA) data from remote sites). Hence, the thing your results show are hit
> rates, not Chris' transmission rates etc [because you're not concerned with
> latency comparisons, yet, as these would be v different remote versus
> local, I assume].
>
> In any case, I agree with you that the main point is the cache hit rate,
> for my slides, which substantially agrees with Chris' experience at RALPP
> for remote CMS data.
>
> I've added a sentence to the slide to note that this is intended as a
> general test of the feasibility of a cache for SEs in general, if that
> helps?
>
> Sam
>
> On Thu, Jul 5, 2018 at 2:43 PM Teng Li <[log in to unmask]> wrote:
>
>> Hi Sam,
>>
>> Yes the current implementation is local. But I would address in my
>> talk that it's a test simulating a transparent cache between WNs and a
>> remote SE. As the performance evaluation focuses on the cached
>> content, so wether the backend SE is remote or not is not vital in the
>> study. And XCache should be more useful to cache remote data for
>> future diskless sites.
>>
>> Best Regards,
>> Teng
>>
>>
>> Quoting Sam Skipsey <[log in to unmask]> on Thu, 5 Jul 2018
>> 14:20:00 +0100:
>>
>> > Hi Teng,
>> >
>> > Yes, it's opportunistic because you're just caching "stuff which is being
>> > transferred right now, in case it is useful later".
>> > In the case of the testing you're presenting, the cache is also local -
>> > because it is between your local SE and your local compute. (I know the
>> > plan is to make it more remote, but IIRC, your actual implementation
>> > currently doesn't proxy outside ECDF?)
>> >
>> >
>> > Thanks for the better slide.
>> > Sam
>> >
>> > On Thu, Jul 5, 2018 at 2:16 PM Teng Li <[log in to unmask]> wrote:
>> >
>> >> Hi Sam,
>> >>
>> >> Thanks. Just one comment on slide 11.
>> >>
>> >> Maybe I've understood the term "local opportunistic cache" wrong but I
>> >> think XRootD proxy cache is mostly useful between local WNs and remote
>> >> SEs, which is what we are simulating and planning to address in my
>> >> following talk. So I would like to say "XrootD proxy cache between
>> >> remote SE and worker nodes".
>> >>
>> >> Also the plot is little blurry. I've attached a clearer one.
>> >>
>> >> Cheers,
>> >> Teng
>> >>
>> >>
>> >>
>> >> Quoting Sam Skipsey <[log in to unmask]> on
>> >> Thu, 5 Jul 2018 13:33:23 +0100:
>> >>
>> >> > As promised, here's a draft of the CHEP Tier 2 caching talk.
>> >> >
>> >> > All comments gratefully received (the frontspiece needs me to add
>> credits
>> >> > anyway, so if you comment you get a credit as well)...
>> >> >
>> >> > Sam
>> >> >
>> >> >
>> ########################################################################
>> >> >
>> >> > To unsubscribe from the GRIDPP-STORAGE list, click the following link:
>> >> > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=GRIDPP-STORAGE&A=1
>> >> >
>> >>
>> >>
>> >> --
>> >> The University of Edinburgh is a charitable body, registered in
>> >> Scotland, with registration number SC005336.
>> >>
>> >>
>> >
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
########################################################################
To unsubscribe from the GRIDPP-STORAGE list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=GRIDPP-STORAGE&A=1
########################################################################
To unsubscribe from the GRIDPP-STORAGE list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=GRIDPP-STORAGE&A=1
