 |
 |
I'm in broad agreement with the previous comments. The area where I've given more depth in my suggested contribution to our institutional response is around sources of citation data, see the following: "In building on the approach taken in REF2014, we would particularly recommend broadening the sources of citation data that are available for use. Citation measures used then were predominantly based on Elsevier data. However, HEFCE's own analysis of coverage in the Metric Tide Supplementary Report II (HEFCE, 2015, http://www.hefce.ac.uk/pubs/rereports/Year/2015/metrictide/Title,104463,en.html) clearly demonstrates significant variation in Elsevier's coverage for different Units of Assessment. In addition, our institutional analyses have identified at least one unit (Computer Science) where Elsevier's coverage of outputs was good, but coverage of the citations to those outputs was poor. Access to additional sources of data, e.g. from Google Scholar*, would be of benefit in such cases, notwithstanding that in some disciplines in the Arts and Social Sciences, coverage is likely to be deemed insufficient from any source." No particular slight to Elsevier intended, I'm just a strong believer in different sources, as well as different metrics, in the basket. I'd also argue that broadly endorsing the principle of using metrics which acknowledge and / or compensate for disciplinary differences and norms, partnered with the recommendation to give panels access to bibliometric expertise, could be of more value than getting into the specifics of which metric to choose at this early stage. The answer might be different for different panels. Discounting journal-level metrics is my obvious exception to this rule! Best wishes, Katie *HEFCE have something on their website saying it proved tricky to obtain Google Scholar data in REF2014, but I don't know the full details... -- Katie Fraser, PhD, MCLIP Senior Research Librarian (Engineering) Libraries, Research and Learning Resources George Green Library University of Nottingham University Park Nottingham NG7 2RD T: 0115 95 13247 E: [log in to unmask]<mailto:[log in to unmask]> http://orcid.org/0000-0003-0844-8734 W: http://blogs.nottingham.ac.uk/librarymatters Twitter: @UoNLibraries<http://twitter.com/UoNLibraries> From: A bibliometrics discussion list for the Library and Research Community [mailto:[log in to unmask]] On Behalf Of Katie Evans Sent: 25 January 2017 16:09 To: [log in to unmask] Subject: Re: Metrics in the next REF Hello All, Thanks for the contributions so far towards a Lis-Bibliometrics response to the consultation about the next REF. This is a gentle reminder to please send contributions to me and/or the list by 6th Feb. The consultation document is available at: http://www.hefce.ac.uk/pubs/year/2016/201636/ Questions where bibliometrics could be relevant are: 18. Do you agree with the proposal for using quantitative data to inform the assessment of outputs, where considered appropriate for the discipline? If you agree, have you any suggestions for data that could be provided to the panels at output and aggregate level? 32. Evaluation of REF 2014 found that provision of impact evidence was challenging for HEIs and panels. Do you have any comments on the following: 32a. The suggestion to provide audit evidence to the panels? 32b. The development of guidelines for the use and standard of quantitative data as evidence for impact? 32c. Do you have any other comments on evidencing impacts in REF 2021? 34a. Do you agree with the proposal to change the structure of the environment template by introducing more quantitative data into this aspect of the assessment? 34b. Do you have suggestions of data already held by institutions that would provide panels with a valuable insight into the research environment? 38. What are your views on the introduction of institutional-level assessment of impact and environment? 39. Do you have any comments on the factors that should be considered when piloting an institutional-level assessment? Kind regards, Katie From: A bibliometrics discussion list for the Library and Research Community [mailto:[log in to unmask]] On Behalf Of Katie Evans Sent: 04 January 2017 11:46 To: [log in to unmask]<mailto:[log in to unmask]> Subject: Re: Metrics in the next REF Hello All, Further to my email this morning about the REF consultation, metrics related thoughts on consultation question 32 on assessing impact (see para 108, p24 of the consultation document) and question 34 on assessing environment (see paras 112-115, p25) would also be welcome. The link for the consultation is: http://www.hefce.ac.uk/pubs/year/2016/201636/ With thanks, Katie From: Katie Evans Sent: 04 January 2017 09:02 To: [log in to unmask] <[log in to unmask]<mailto:[log in to unmask]>> Subject: Metrics in the next REF Dear All, The four UK higher education funding bodies are seeking views on their proposals for the next Research Excellence Framework, see: http://www.hefce.ac.uk/pubs/year/2016/201636/ The Lis-Bibliometric Committee would like to submit a response on behalf of the List, specifically to question 18: "Do you agree with the proposal for using quantitative data to inform the assessment of outputs, where considered appropriate for the discipline? If you agree, have you any suggestions for data that could be provided to the panels at output and aggregate level?" See paragraphs 73 & 74 of the consultation document (p17). Please send your thoughts on this to the List and/or me ([log in to unmask]<mailto:[log in to unmask]>) by 6th Feb. This is most relevant to those of you in the UK, but contributions from colleagues outside the UK are also welcome. Kind regards, Katie (on behalf of the Lis-Bibliometrics Committee) -- Katie Evans, MMath, MSc Econ, MCLIP Research Analytics Librarian University of Bath Tel: 01225 384488 Email: [log in to unmask]<mailto:[log in to unmask]> Working pattern: Mon - Thurs This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. This message has been checked for viruses but the contents of an attachment may still contain software viruses which could damage your computer system, you are advised to perform your own checks. Email communications with the University of Nottingham may be monitored as permitted by UK legislation.