I think they are currently at odds - see my 'Double Life' blog post! Evaluation that relies too heavily to citation metrics which are in turn mainly tied to journals do not generally reward academics for publishing in 'open' outlets (other than Hybrid Gold OA journals which are expensive). Other forms of indicator are needed, but they in turn need to be valued by those doing the evaluating. You could argue that we already have other forms of indicator, but they are not mature enough yet to be used in any formal evaluations.· How does research evaluation relate to goals for open scholarship?
I believe that UCL now reward scholars for openness as part of their annual review process. ImpactStory offer 'Open Access Hero' status to individuals who have a high proportion (100%??) of papers available on OA. The OA Spectrum tool grades journals as to the level of openness they adopt. There was also a paper recently on 'Openness Metrics' http://onlinelibrary.wiley.com/doi/10.1002/asi.23741/abstract. Not sure how 'responsible' these all are, but they are beginnings!· How may “responsible metrics” capture the progress towards full open access to research publications?
Altmetrics are the obvious tool as they can apply to a larger range of outputs. The challenge with Altmetrics is their 'responsible-ness': many exist to provide rich contextual data rather than numbers for comparative purposes. Certainly very few are normalised and transparent.· How may responsible evaluation capture open scholarship contributions (e.g. Software, datasets) that are currently ignored?
Well, there's plenty written on responsible metrics and you are one of the UK's experts Katie! The Metric Tide's 'robust, humble, transparent, diverse and reflexive' are good rules of thumb. How they might become more open scholarship aware is a harder question as there are so many ways of being open.· How may current indicators (e.g. citations, co-authorship, usage, altmetrics) become responsible and open scholarship aware?
As above, I think moves are being made in this direction with OA outputs, however I'm aware that open scholarship is broader than OA. I think the former needs teasing out before this question can be answered.· How may responsible and open scholarship aware evaluation systems be used to generate incentives and rewards?