Thanks Ian, Lizzie,
The Waltman and Traag article mentions the Italian equivalent of REF. I've
done some reviewing for that and liked the way they combine IF, citations
and expert evaluations.
Between them they really pick out the issues of journal metric (e.g. IF)
as predictor of document level metric (e.g. long-term citations) and either
as proxy for some abstract notion of research quality.
In REF language one could argue that long-term citation is a possible proxy
for significance whereas the venue's impact factor might be a *better*
indicator of rigour (or maybe accuracy/truth).
Alan
On 31 January 2018 at 14:11, Elizabeth Gadd <[log in to unmask]> wrote:
> Hi Alan, all,
>
>
>
> There is another defence of the impact factor here:
>
>
>
> https://arxiv.org/abs/1703.02334
>
>
>
> You make a good point about the h-index. In my view this is a far more
> problematic metric than the IF but it doesn’t get nearly as much bad press!
>
>
>
> Lizzie
>
>
>
> *From:* A bibliometrics discussion list for the Library and Research
> Community [mailto:[log in to unmask]] *On Behalf Of *Rowlands,
> Ian
> *Sent:* 31 January 2018 10:55
>
> *To:* [log in to unmask]
> *Subject:* Re: FW: Announcing the Metrics Toolkit
>
>
>
> Hi Alan
>
>
>
> You should read this for a nuanced defence of impact factors in research
> evaluation:
>
>
>
> https://arxiv.org/abs/1612.04075
>
>
>
> There are glaring dangers in terms of conflating journal and article level
> impact, but I don’t think the objections to impact factors are entirely to
> do with the construction of the metric!
>
>
>
> Ian
>
>
>
> *From:* A bibliometrics discussion list for the Library and Research
> Community [mailto:[log in to unmask]
> <[log in to unmask]>] *On Behalf Of *Alan Dix
> *Sent:* 31 January 2018 10:17
> *To:* [log in to unmask]
> *Subject:* Re: FW: Announcing the Metrics Toolkit
>
>
>
>
>
> ... at the risk of getting myself lined up in front of a firing squad ...
>
>
>
> Although everyone hates impact factor, actually (certainly in my field)
> and despite the odd anomaly, the IF ranking is not so far from my
> subjective ranking of venue quality.
>
> Furthermore, 'better' venues typically have more effective review
> mechanisms and higher standards.
>
>
>
> That is quality of venue is an indicator of quality of publication (there
> I said it :-/). It is not a particularly great one, and certainly not the
> preferred one if other more output-specific information is available
> instead. However, for assessment where no other information is available
> (e.g. roughly assessing candidates with recent publications), quality of
> venue will continue be used as a proxy for output quality, and not
> completely without merit.
>
>
>
> So, having good advice on appropriate venue-level metrics does seem really
> important.
>
>
>
> I've been particularly alarmed recently to see journal metrics using
> variants of H-index - IF based on arithmetic averages is bad enough, but
> H-index is far far worse. Not a bad metric to compare people in the same
> field at similar stages of their career, but for venues it is so
> volume-dependent to be totally misleading, if two equal quality journals
> that merge their combined H-index suddenly goes up - not sensible!
>
>
>
> Alan
>
>
>
>
>
> On 31 January 2018 at 09:32, Elizabeth Gadd <[log in to unmask]> wrote:
>
> Dear colleagues
>
>
>
> Looks like the Metrics Toolkit has now been launched. I’ve only had a
> quick look. I really like the concept. Currently there is no category for
> journal metrics so some are listed as measures of journal articles imapct
> instead which I can see as being problematic from a responsible metrics
> standpoint. There are some key indicators missing (SNIP, SJR) but I’m
> guessing it will develop over time.
>
>
>
> One to watch!
>
> Lizzie
>
>
>
>
>
> *From:* [log in to unmask] [mailto:resmetig-request@
> lists.ala.org] *On Behalf Of *Heather Coates
> *Sent:* 30 January 2018 18:56
> *To:* [log in to unmask]
> *Subject:* Announcing the Metrics Toolkit
>
>
>
> [image: MT_screenshot.png]
>
> Colleagues,
>
>
>
> Today, we’ve launched the Metrics Toolkit
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.metrics-toolkit.org%2F&data=01%7C01%7Cian.rowlands%40KCL.AC.UK%7C0a41da96b75d4143455608d56894f66b%7C8370cf1416f34c16b83c724071654356%7C0&sdata=Sm1IqOSFs1XnwXpaGTgLpFjyzJ3Lf%2Bn4LTgP9mWxNmU%3D&reserved=0>,
> a resource built to help researchers and evaluators navigate the
> ever-changing research metrics landscape.
>
>
>
> The Metrics Toolkit includes 27 expert-written, time-saving summaries for
> the most popular research metrics including the Journal Impact Factor and
> Altmetric Attention Score.
>
>
>
> Even better, the Metrics Toolkit includes an app that can recommend
> discipline-specific metrics to satisfy* your specific use cases.*
>
>
>
> Best of all, the Metrics Toolkit carries a CC-BY license, so you can reuse
> its content as you wish!
>
>
>
> VISIT THE TOOLKIT
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.metrics-toolkit.org%2F&data=01%7C01%7Cian.rowlands%40KCL.AC.UK%7C0a41da96b75d4143455608d56894f66b%7C8370cf1416f34c16b83c724071654356%7C0&sdata=Sm1IqOSFs1XnwXpaGTgLpFjyzJ3Lf%2Bn4LTgP9mWxNmU%3D&reserved=0>
>
>
>
> Best wishes,
>
> Heather, Robin & Stacy
>
> Creators, Metrics Toolkit
>
> [log in to unmask]
>
>
>
> --
>
> Heather Coates
>
> https://coateshl.wordpress.com
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcoateshl.wordpress.com&data=01%7C01%7Cian.rowlands%40KCL.AC.UK%7C0a41da96b75d4143455608d56894f66b%7C8370cf1416f34c16b83c724071654356%7C0&sdata=DKs%2BW23ZjJtsAGwu7d%2FsQ9rCDW%2BrX6SZ8DQWsed6IjU%3D&reserved=0>
>
>
>
>
>
> --
>
> Professor Alan Dix
>
>
>
> University of Birmingham, UK
>
>
>
> web: http://www.alandix.com/
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.alandix.com%2F&data=01%7C01%7Cian.rowlands%40KCL.AC.UK%7C0a41da96b75d4143455608d56894f66b%7C8370cf1416f34c16b83c724071654356%7C0&sdata=gyrKVY%2BXAAS44O3ZrWTOUpenmihrnt8oc71GEKYnO4Y%3D&reserved=0>
>
> blog: http://www.alandix.com/blog/
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.alandix.com%2Fblog%2F&data=01%7C01%7Cian.rowlands%40KCL.AC.UK%7C0a41da96b75d4143455608d56894f66b%7C8370cf1416f34c16b83c724071654356%7C0&sdata=tGLpe1J3fvpbJqIJskrwSZwEfgb33IXLKB7yvIRw2co%3D&reserved=0>
>
> Alan walks Wales: http://www.alandix.com/alanwalkswales
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.alandix.com%2Falanwalkswales&data=01%7C01%7Cian.rowlands%40KCL.AC.UK%7C0a41da96b75d4143455608d56894f66b%7C8370cf1416f34c16b83c724071654356%7C0&sdata=rf9R%2Bd5ls8D6Xme8l06uQ%2FHFQF2Z%2FhQ9FUE3nBrQtbk%3D&reserved=0>
>
> Tiree Tech Wave: http://tireetechwave.org/
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftireetechwave.org%2F&data=01%7C01%7Cian.rowlands%40KCL.AC.UK%7C0a41da96b75d4143455608d56894f66b%7C8370cf1416f34c16b83c724071654356%7C0&sdata=SqSPADTd8dfAnb4CY0YmSvqY5Pfsn9WXvtPL5H%2BxiaE%3D&reserved=0>
>
--
Professor Alan Dix
University of Birmingham, UK
web: http://www.alandix.com/
blog: http://www.alandix.com/blog/
Alan walks Wales: http://www.alandix.com/alanwalkswales
Tiree Tech Wave: http://tireetechwave.org/
