> -----Original Message-----
> From: GRIDPP2: Deployment and support of SRM and local storage management
> [mailto:[log in to unmask]] On Behalf Of Wahid Bhimji
>
> Though I don't think there's anything all that surprising in what I am
> proposing doing - unless you dispute the whole definition of cloud
> storage of course.... (!)
>
I dispute the whole definition of cloud storage (of course) :-)
The plan actually sounds fine, but if it were me I'd rather cast it
more in terms of evaluating the usefulness of all the storage systems
from an equal starting point.
Essentially, I'd just lose the definition of cloud storage as having
interfaces like the commercial providers; if the question is how we
do storage under a cloud computing system we shouldn't bake the answer
in from the beginning as being 'we do it like AWS'.
Ewan
>
> Cloud storage testing
>
>
> The project will work with both the GridPP cloud group and the WLCG
> Storage Interfaces Group and will be guided by the goals of those groups.
> "Cloud storage" is taken to mean resources which have an interface like
> those provided by current commercial cloud resource providers with a goal
> being to allow use of resources that might be provided in this way in the
> future either by wider university / academic infrastructures or by
> commercial providers.
>
> All tests will either use a very minimum set of functionality on top of
> this storage interface, or use existing (and/or well supported) storage
> solutions to provide access to the interface. It will also use real LHC
> experiment workloads and data flows in testing.
>
> Initial testing will contain two parts:
>
> 1. Application direct reading (or local copying) of data.
> 2. Wide-area data transfers.
>
> Ideally all tests will be compared to using existing access or transfer
> protocols on the same (or identical) hardware.
>
> Direct reading
>
>
> ROOT recently has an S3:// plugin, this will be tested alongside existing
> http:// access for at least two workloads:
> 1) A generic (not VO specific) ROOT application that reads a large data
> file
> 2) An ATLAS D3PDMaker file using the Athena framework.
>
> Data transfer
>
>
> The first level of testing would compare "bare" transfers using cloud
> protocols with a gridFTP server setup on the same hardware.
> The next level of comparison would be to use real experiment workflows.
> Here the current WLCG authentication / security requirements would require
> use of an additional layer. This may be most practically provided by
> storage solutions such as dCache or DPM and so these will be tested
> alongside "bare" transfers.
>
> Resources
>
> * RAL S3 on Ceph
> * Imperial Open Stack
> * Use may also be made of CERN resources (Huawei and Open Stack) as
> well as (at small scale) Amazon.
|