 |
 |
Hi Emily
Interesting question – thanks for posing it. We also currently provide journal percentiles at department level. However, this might change as we roll out the
findings of our responsible metrics working group. I think the way to approach this would be:
·
Always provide a long list of caveats (A&H don’t publish much in journals; a citation means something different to A&H than
to STEM; the citation benchmarking tools don’t cover as many of their journals; the publication sets are therefore going to be small leading to less meaningful results)
·
Take Mirza’s excellent point and stress that the numbers should not be used to compare one department with another, only for
each department’s interest and trend tracking.
·
Be as transparent as possible about the data in the set – don’t just provide indicators (“0% of journals in the Top 25% by
SNIP” doesn’t help anyone)
·
Always use subject normalised journal metrics – SNIP is your best bet for A&H – but remember they are not normalised equally.
(See Katie Evans last BiU presentation on the lis-Bibliometrics files area)
·
Try and work with the departments to see whether you develop a better metric for them. We’re currently attempting this at
Loughborough…
There might be others that colleagues want to chip in!
Good luck
J
Elizabeth Gadd MSc, MCLIP, FHEA
Research Policy Manager (Publications)
Research Office
Loughborough University
Loughborough, Leicestershire, LE11 3TU
T: 01509 228594
S: lizziegadd
Working Hours:
Mon 8.30-5
Tues 8.30-3
Wed 8.30-3
Google Scholar Citation Profile
http://orcid.org/0000-0003-4509-7785
http://about.me/elizabeth.gadd
From: A bibliometrics discussion list for the Library and Research
Community [mailto:[log in to unmask]] On Behalf Of Dr. Mirza Muhammad Naseer
Sent: 27 April 2016 04:57
To: [log in to unmask]
Subject: Re: Bibliometrics and KPIs
Hi Emily,
We, at Institute of Space Technology (a small Degree Awarding Institute), have our indigenous system to measure quality of journal publications as we cannot afford to subscribe SciVal or similar tools. We use
Gold, Silver, Bronze, Honorable, and Unranked categories of Journal Quality Ranking System (JQRS) for evaluation of journal papers. System uses a number of measures to analyze the quality of a journal. Details of the system are available at:
http://jqrs.ist.edu.pk/about.aspx
Once data are compiled at university level using JQRS, we can go to department or even individual level for analysis. However, I also share your reservation about using metrics across disciplines, especially Arts, Social Sciences, and Linguistics.
In my opinion, we can only compare publications / people within a discipline not with other disciplines but management may think otherwise.
With best regards,
Dr. Mirza Muhammad Naseer
Assistant Professor (LIS)
HEC Approved PhD Supervisor
Institute of Space Technology (IST)
1, Islamabad Highway,
Near CDA Toll Plaza,
Islamabad. Pakistan.
Ph. +92-51-9075436
On Tue, Apr 26, 2016 at 8:19 PM, Katie Evans <[log in to unmask]> wrote:
Hello Emily,
We have a very similar situation. The University of Bath’s research strategy says that “we will
monitor the number of research papers published in leading journals as a measure of both the vibrancy and quality of our research” (http://www.bath.ac.uk/about/organisation/strategy/research/index.html
) We do use SciVal’s publication in top journal percentiles metrics for this. I think in this context we look at the University as a whole rather than broken down by department. I share your reservations about whether this is fair across different disciplines,
but I try to address that in other ways (e.g. working with the Departments) if not in the KPI monitoring itself.
Kind regards,
Katie.
From:
A bibliometrics discussion list for the Library and Research Community [mailto:[log in to unmask]]
On Behalf Of Emily Bennett
Sent: 26 April 2016 12:12
To: [log in to unmask]
Subject: Bibliometrics and KPIs
Hi all,
I haven't posted a question on this list before, but I've had a request from our central Planning / Strategy department for some metrics, and I'm a little concerned about what they're
asking for. So I thought I'd see what people on here thought .....
The library has been asked to produce a report detailing the number of publications per department. This report also needs to include the number of citations and the percentage
of articles in 'top journals'. The latter request about 'top journals' is because the university has set a KPI on this.
I am interested to know how other people have handled similar requests?
For example, if we were to use SciVal then we could use the 'Percentage of publications in top 10 journal percentile' metric as an indicator for 'top journals' (ie. for the KPI).
While this may be fine for science and tech subjects, the coverage of SciVal is not ideal for Arts and Hums subject areas, and from doing a quick 'test-run' the data doesn't seem very meaningful in these areas.
Also, I'm not sure that trying to apply the KPI of 'increasing publications in top journals' across all (e.g. arts and hums) departments is a good idea anyway!! So I'd be interested
to hear if any other unis have had similar requests and how to handle them.
Thanks,
Emily
--
--
Dr Emily Bennett
Research Outputs Manager
University Library and Research and Innovation Services
University of Portsmouth
Tel. 02392 843220
or
Tel. 02392 846191
Open Access information page