Hans, In terms of "what to measure", I'd suggest looking at the alt-metrics movement, as they are building up a great deal of experience in this area.

It may be worth thinking about the "impact of measuring" issue, but I suspect it's small. The issue mainly comes into effect at extremely small scales (if I understand it correctly). If I measure the rooms in my house, the measurement has no discernible effect. I think you're right though, to be concerned about "gaming the system" (a somewhat different effect, I suspect). Given how smart humans can be (and how dumb), we'll never be free of this, but that doesn't mean that publishing information about takeup and impact is a bad thing. Rather, we have to try to ensure our community norms evolve to make this gaming unacceptable, as gaming the JIF clearly is (though it sill allegedly happens).

When IRs publish information about aspects of takeup, eg downloads, citations etc, I find that very useful and interesting, both as author and consumer.

--
Chris Rusbridge
Mobile: +44 791 7423828
Email: [log in to unmask]
Adopt the email charter! http://emailcharter.org/




On 24 May 2013, at 09:09, Hans Pfeiffenberger wrote:

> Am 22.05.13 09:24, schrieb Angus Whyte:
>> The criterion could be better expressed as ' repositories must publish information to enable journals and depositors to assess its take-up in the community it aims to serve, and the level of access to deposited items, e.g. how frequently these are accessed by repository users"
>> 
>> In either that or the original wording, is this relevant to support a decision on repository recommendation in this context? If so, should it be 'important' or 'mandatory' for data repositories to publish this information?
>> 
>> Angus
> Dear Angus,
> 
> there are two points:
> 
> - should we *already* ask for a measure of take-up?
> - how to measure it (counting downloads? metadata views? likes?) and how to normalize and compare that (per community?)
> 
> As a former physicist I like to say that you cannot observe a system without modifying it, perhaps even destroying it.
> And I guess that in many communities the repository landscape is still so fragile that the destroying clause may apply.
> => so: no *mandatory* criterion on measure of take-up *yet*
> 
> As to the counting of activity at a repository site, this can even more easily be manipulated than the notorious Impact Factor. You would set into motion a race for something ... probably without evidence that it actually has a strong correlation with community take-up (whatever *that* means) or actually: value to a community.
> 
> best,
> 
> Hans