There is 1 message totaling 151 lines in this issue.
Topics of the day:
1. research evaluation and open scholarship
----------------------------------------------------------------------
Date: Thu, 14 Sep 2017 18:41:22 +0000
From: Elizabeth Gadd <
[log in to unmask]>
Subject: Re: research evaluation and open scholarship
Hi Katie
It's great that you can go to this and represent the views of the UK bibliometric community! I've inserted some comments below on the questions they raise.
· How does evaluation affect the behaviour of individual researchers and the culture of research communities?
In my experience I think there is a narrowing of focus, particularly among probationers and early-career staff, from a broad "how can I make a contribution to my discipline" to an almost undergraduate-like fixation on "what do I need to do to pass (probation,
annual review, promotion boards, etc)". I think metrics have played a part in this and highlight again the need for a responsible, holistic approach to evaluation.
· How does research evaluation relate to goals for open scholarship?
I think they are currently at odds - see my 'Double Life' blog post! Evaluation that relies too heavily to citation metrics which are in turn mainly tied to journals do not generally reward academics for publishing in 'open' outlets (other than Hybrid Gold
OA journals which are expensive). Other forms of indicator are needed, but they in turn need to be valued by those doing the evaluating. You could argue that we already have other forms of indicator, but they are not mature enough yet to be used in any formal
evaluations.
· How may “responsible metrics” capture the progress towards full open access to research publications?
I believe that UCL now reward scholars for openness as part of their annual review process. ImpactStory offer 'Open Access Hero' status to individuals who have a high proportion (100%??) of papers available on OA. The OA Spectrum tool grades journals as to
the level of openness they adopt. There was also a paper recently on 'Openness Metrics'
http://onlinelibrary.wiley.com/doi/10.1002/asi.23741/abstract. Not sure how 'responsible' these all are, but they are beginnings!
· How may responsible evaluation capture open scholarship contributions (e.g. Software, datasets) that are currently ignored?
Altmetrics are the obvious tool as they can apply to a larger range of outputs. The challenge with Altmetrics is their 'responsible-ness': many exist to provide rich contextual data rather than numbers for comparative purposes. Certainly very few are normalised
and transparent.
· How may current indicators (e.g. citations, co-authorship, usage, altmetrics) become responsible and open scholarship aware?
Well, there's plenty written on responsible metrics and you are one of the UK's experts Katie! The Metric Tide's 'robust, humble, transparent, diverse and reflexive' are good rules of thumb. How they might become more open scholarship aware is a harder question
as there are so many ways of being open.
· How may responsible and open scholarship aware evaluation systems be used to generate incentives and rewards?
As above, I think moves are being made in this direction with OA outputs, however I'm aware that open scholarship is broader than OA. I think the former needs teasing out before this question can be answered.
Looks like you are in for an interesting couple of days Katie. Really looking forward to hearing about it when you get back.
All best
Lizzie
------------------------------
End of LIS-BIBLIOMETRICS Digest - 13 Sep 2017 to 14 Sep 2017 (#2017-99)
***********************************************************************