Three years ago I commented on the extent to which Web citation was growing as a
means of assessing the impact of research outputs
(http://listserv.utk.edu/cgi-bin/wa?A2=ind0205&L=jesse&D=0&I=-3&P=720). There
was an interesting discussion and, eventually, prompted by the debate, a paper
on the subject in JASIST by Vaughan and Shaw (Volume 54, Issue 14 , Pages 1313
- 1322).
I'm returning to the topic because of the emergence of Scholar Google as an
interesting new venture in the field of bibliographic control, and by my
testing of frequently hit papers in Information Research
(http://InformationR.net/ir/). The results suggest that for university
administrations to rely upon the Web of Science citation counts and the
associated journal impact measures as a means of assessing faculty for
promotion is rather flawed. The same might be set of the decisions by the
Research Assessment panels of the Higher Education Funding Councils in the UK
to rely upon ranked lists of journals in assessing research outputs.
I'm aware, of course, of Peter Jasco's very interesting paper on Scholar and Web
of Science in Current Science, v.89, no. 9, 1537-1547 and no doubt his
criticisms of Scholar will provoke some changes. However, my strategy was not
affected by the system's shortcomings, since I was looking for citations of
known items and I weeded out the occasional 'false drops'.
I looked at the papers in Information Research that had received 10 or more
citations, according to Scholar, and the results can be seen at my Weblog -
http://www.free-conversant.com/irweblog/657.
To take one example: The effect of query complexity on Web searching results, by
BJ Jansen (Information Research, Volume 6 No. 1 October 2000) was said by
Scholar to have 24 citations (actually, 26 were recorded) - WoS gave it zero
citations - and yet the citations found by Scholar were as follows:
Peer reviewed journals - 9 citations (4 journals, all of which are covered by
WoS)
Conference papers - 8 citations
University department paper archive - 7 citations
Research group report - 1 citation
Thesis - 1 citation
The paper also had about 200 Web citations, as measured by searching on
Google.com
What then, are we to regard as "impact"? It would be wise for any candidate for
promotion to press the case for a wider definition than measurement by WoS
citations provides. There are problems with other measures but, for example,
the international impact is likely to be measured better by either Scholar or
Web citation generally. Similarly, in relation to the UK's Research Assessment
exercise, it would be sensible for the Higher Education Funding Councils to
offer better guidance on how the impact of research outputs should be
assessed.
Professor T.D. Wilson, PhD, Hon.PhD
Publisher/Editor in Chief
Information Research
InformationR.net
e-mail: [log in to unmask]
Web site: http://InformationR.net/
___________________________________________________
|