Thank you Fiona, very interesting
Oh the (new) joys of open reviewing!
Takes a bit of getting used to, I think its going to make me a bit over cautious as a reviewer, but of course its got to be a good thing
Frances
Professor Frances Gardner,
Professor of Child and Family Psychology
Fellow of Wolfson College
Centre for Evidence-Based Intervention
Department of Social Policy & Intervention
University of Oxford, 32 Wellington Square, Oxford OX1 2ER, UK
tel:44-1865-270325 / 270334 email: [log in to unmask]
http://www.spsw.ox.ac.uk/staff/academic/profile/details/gardner.html
http://www.neuroscience.ox.ac.uk/directory/frances-gardner
*** please note that this email address has now expired: [log in to unmask] ****
please use: [log in to unmask] or [log in to unmask]
________________________________________
From: Evidence based health (EBH) [[log in to unmask]] on behalf of Fiona Beyer [[log in to unmask]]
Sent: 22 January 2013 12:25
To: [log in to unmask]
Subject: Re: Google Scholar as a single source for systematic reviews has not yet been justified
Interesting that, whether information specialist or not, one of the reviewers agreed that there are methodological difficulties:
"For the reasons describe next, the authors' method is inadequate and their
conclusion is not logically connected to their results. No revision (major, minor, or
discretionary) will save this work and this is why this report is not itemized or
divided into sections."....
(taken from the link given by Shona below)
The other reviewer had no such qualms, but it seems odd that with one such concerned reviewer the paper would be published anyway?
Regards,
Fiona.
Fiona Beyer
Research Associate,
Institute of Health and Society,
Baddiley-Clark Building,
Richardson Road,
Newcastle upon Tyne.
NE2 4AX.
T: 0191 2086368
F: 0191 2226043
E: [log in to unmask]
========================================================================================
I also support these comments.
In terms of peer review, BMC Medical Informatics and Decision Making provide access to the pre-publication history which includes the peer reviewer comments (http://www.biomedcentral.com/1472-6947/13/7/prepub) but no information is provided regarding whether the peer reviewers in this case were information professionals or not.
Shona
Date: Mon, 21 Jan 2013 12:01:13 +0000
From: Shona Kirtley <[log in to unmask]>
Subject: Re: Google Scholar as a single source for systematic reviews has not yet been justified
I also support these comments.
In terms of peer review, BMC Medical Informatics and Decision Making provide access to the pre-publication history which includes the peer reviewer comments (http://www.biomedcentral.com/1472-6947/13/7/prepub) but no information is provided regarding whether the peer reviewers in this case were information professionals or not.
Shona
__________________________________________________
Shona Kirtley
Research Information Specialist
EQUATOR Network
Centre for Statistics in Medicine
University of Oxford
Wolfson College Annexe
Linton Road
Oxford
OX2 6UD
Tel: 01865 284410
Email: [log in to unmask]<mailto:[log in to unmask]>
Web: http://www.equator-network.org/
Spanish website: http://www.espanol.equator-network.org/
Web: http://www.csm-oxford.org.uk/
__________________________________________________
EQUATOR Network - resources for reporting research
Website: http://www.equator-network.org/
__________________________________________________
________________________________
From: Evidence based health (EBH) [[log in to unmask]] on behalf of Harbour Robin (HEALTHCARE IMPROVEMENT SCOTLAND - SD039) [[log in to unmask]]
Sent: 21 January 2013 11:32
To: [log in to unmask]
Subject: Re: Google Scholar as a single source for systematic reviews has not yet been justified
I would just support your comments, and add that I was a bit surprised this got through a peer review process (if it did?) The fundamental point about the difference between finding articles that you know exist, and searching for what may exist on a given topic should surely have raised some questions at that stage. If it was peer reviewed, did the reviewers include information professionals?
Robin
Scottish Intercollegiate Guidelines Network
Delta House | 50 West Nile Street | Glasgow G1 2NP
t: 0141 227 3298
e: [log in to unmask]<https://owa.nexus.ox.ac.uk/owa/UrlBlockedError.aspx>
www.sign.ac.uk<http://www.sign.ac.uk/>
The Scottish Intercollegiate Guidelines Network is part of Healthcare Improvement Scotland.
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Fiona Morgan
Sent: 21 January 2013 10:38
To: [log in to unmask]
Subject: Google Scholar as a single source for systematic reviews has not yet been justified
On Friday 11th January, a group of systematic reviewers and information specialists at Cardiff University sent a comment to BMC Medical Informatics and Decision Making expressing serious concerns about the methodology of a paper, which has been highlighted via this list, entitled 'Is the coverage of Google Scholar enough to be used alone for systematic reviews?'.
Although the comment has been acknowledged by the journal (see below) it has not as yet been posted, ten days after submission, despite a request to the journal for an update on whether it has been accepted or rejected.
Any thoughts or suggestions?
--------------------------------------------------------------------
Thank you for contributing to the discussion of BMC Medical Informatics and Decision Making 2013, 13:7
<http://www.biomedcentral.com/1472-6947/13/7>
Your comment is under moderation and will appear on the site provided it contributes to the topics and complies to our standard terms & conditions.
You will receive an email notification when your comment has been approved or rejected.
Your posting will read:
Alison Weightman, Cardiff University
[log in to unmask]
Google Scholar as a single source for systematic reviews has not been justified
Comment:
This paper forms part of an important debate. Google Scholar (GS) is an increasingly powerful search tool that should rightly be considered by systematic review searchers. Research studies comparing GS to other established search tools are valuable.
However we feel the paper fails to address some really important issues, and believe that the conclusion that GS could be a single search source for systematic reviews has not been justified.
Being able to find a paper once you know about it is not the same as finding the paper in the first instance. Problems identified with this publication, on which we would welcome the authors' and journal editors' responses, are:
1. Google Scholar is constantly updated and it isn't possible to know whether the references included in it at the search date for this paper would have been there at the search date of the candidate systematic reviews. GS trawls many sites and may have picked up these references as a result of their inclusion in the reviews.
2. The lack of advanced search functions is acknowledged but there seems to be an implicit assumption in the paper that review authors would have chosen the right terms to pick up relevant publications, including citations without an abstract. Further, the very low precision rates measured by the authors of this paper from example searches (circa 0.1% precision) severely limit the current value of GS as a single search source for systematic reviews. It would ask a lot of systematic reviewers to trawl/sift 36,000 results in order to find 36 relevant papers.
3. The authors did not require the title to link to an abstract or full text in Google Scholar (GS), to be regarded as a study within GS; a link to a citation was considered sufficient. Each title identified via a citation would need to be searched for in other database(s) to find an abstract, or obtained in full text to assess relevance. This would entail a substantial additional workload for systematic review authors which is not addressed in the paper.
4. The paper has several sections that should have been edited for grammatically correct English. Errors are not surprising given that the authors are not writing in their native language, but it may call the editorial process into question.
Alison Weightman, Fiona Morgan, Mala Mann and Bernadette Coles
University Library Service, Cardiff University, UK
|
