Wahid Bhimji wrote:
> Hi,
> 
> I agree that  a hammercloud test with "official" files from ATLAS is 
> something that will be needed and, in some senses, this is speculation 
> of what that might show.

Indeed.

> 
> But, I am not entirely sure it would be that transparent - for one thing 
> it masks effects by testing several things at the same time.

Yes, this is the problem with hammercloud tests. They are great in that 
they stress a site and provide an overall picture of how well the site 
performs. What they don't do is allow you to quickly and reliably test 
small changes on their own to see if they have an effect.

There are various things that we might do on our disk servers (adding 
RAM for example) that might make a difference at the 10-20% level. 
Several of these improvements would be worth having - but we'd want a 
more repeatable test to be able to be sure we'd improved things. One of 
our vendors has suggested some improvements we could make - and asked if 
they improved things. It is rather embarrassing to admit that it's 
really important to us, but we aren't actually able to measure the 
performance increase with a realistic workload.

> For a second thing while hammerclouds do use real user analysis code - 
> your site may get visited by an user/ analysis group that does something 
> much stupider.
> 

Having a range of tests is indeed useful - in fact I asked about a local 
hammercloud test - but there doesn't seem to be effort available to 
provide one.

> In some ways there are benefits in having a range of tests.
> - "raw" file access like in chris W's tests
> - root file access like my jobs
> - local athena jobs
> - hammercloud

Completely agreed.

By measuring the performance of the individual stages, we can work out 
where the problems are, and what the lowest hanging fruit are.

The impression I get, by the way, is that the UK is ahead of the game in 
actually doing real measurements on this.

Chris

> One Q - was the athena job you are submitting local - or through ganga etc?
> If local - can you send me the joboptions file so I can just compare.
> 
> Cheers
> 
> Wahid
> 
> John Bland wrote:
>> Hi,
>>
>> My tests are performed with a sample athena job provided by the Atlas 
>> researchers. Maybe this is why I see (possibly) different results for 
>> RFIO.
>>
>> I concur with others that a widespread hammercloud test is really 
>> needed to test these things fairly (and without duplicating effort too 
>> much).
>>
>> John
>>
>> On 22/02/2010 13:15, Wahid Bhimji wrote:
>>> Hello
>>>
>>> On TTreeCache: I also tried this out but independently to the reordering
>>> (so the improvements I saw were from reordering, though TTreeCache can
>>> also provide an improvement.)
>>> This only seemed to work for reading a local file, with RFIO it seemed
>>> to cause a segfault after reading 2GB.
>>> The plot of the access pattern is on the web page with the rest and how
>>> to use it in Root are in the example on the wiki page. I'm not sure
>>> whether it is yet possible to use it in Athena jobs though.
>>>
>>> On reordered merged files: If you want you can copy my reordered file
>>> (it is a biggy)
>>> lcg-cp -v
>>> srm://srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/AODClone.root 
>>>
>>> scratch0/AODClone.root
>>> (If you are not in the atlas VO, I can copy it to dteam )
>>>
>>> Wahid
>>>
>>> John Bland wrote:
>>>> On 22/02/2010 11:53, Christopher J.Walker wrote:
>>>>> John Bland wrote:
>>>>>
>>>>>> This is assuming that what I've been provided with by the local ATLAS
>>>>>> guys *is* actually ordered ;0). I've already had one set that wasn't.
>>>>>> I'm about to try the local disk tests to see if they show the same
>>>>>> effect as Wahid's.
>>>>>>
>>>>>
>>>>> At the same time as the effect of ordered data, something called Ttree
>>>>> cache was talked about. Are you using this too? I can see that it 
>>>>> might
>>>>> make a difference, though this is clutching at straws a bit.
>>>>
>>>> They're only ordered but I haven't had time to do any more tests,
>>>> particularly as the files I've got are unmerged so it's hard to give a
>>>> fair comparison when cached locally.
>>>>
>>>> John
>>>>
>>>
>>>
>>
>>
> 
>