There are actually two issues involved in the question of non-English studies: searching and inclusion.
It would seem wise to always search for studies of all languages. Many non-English studies have English abstracts. With or without such abstracts, it may be possible to get some idea of what types of studies and results will be excluded if these articles are not retrieved and included in the systematic review.
As a practical matter, retrieval and inclusion of non-English studies can add considerable expense to a review. Proper understanding and use of the articles requires a full translation. For some languages this requires paying for a translation. Sometimes it is regrettably necessary to compromise on this issue. Also, some clients who contract for systematic reviews are only interested in domestic studies. With some justification (in some cases) they worry that studies carried out in other settings may not be relevant to their clinical situation.
We do systematic reviews both with and without non-English articles, depending on the topic, the nature of the non-English studies, and the purpose of the review.
David L. Doggett, Ph.D.
Senior Medical Research Analyst
Health Technology Assessment and Information Services
ECRI, a nonprofit health services research organization
5200 Butler Pike
Plymouth Meeting, PA 19462, USA
Phone: (610) 825-6000 x5509
FAX: (610) 834-1275
e-mail: [log in to unmask]
-----Original Message-----
From: Janet Martin [mailto:[log in to unmask]]
Sent: Tuesday, October 29, 2002 5:25 PM
To: [log in to unmask]
Subject: Re: Language bias in meta-analysis
Dear all,
Certainly, language bias is known to occur. It always strikes me as rather vain for English speaking researchers to presume that all good research is published in their native tongue. In fact, to call a review 'systematic', while excluding all non-English research is rather oxymoronic.
On the other hand, to conclude that English-only systematic reviews are always biased is equally presumptuous. The conclusions of a number of systematic reviews would not be changed or further informed by non-English trials. The reason for this may be related to the fact that English is the preferred international language of medicine. Therefore, there is often a lower probability that trials will be found in non-English language literature. Often, but not always. The difficulty is knowing when reviews have been biased by ignoring the non-English literature. Usually this cannot be known without performing an unbiased search for articles in all languages.
Research has been published (at the very least, in abstract form) to suggest that eliminating non-English studies did not change the results of meta-analysis (I will try to track this abstract down).
Personally, I think that the effect of non-English literature will depend on the topic of the review.
Since you cannot always predict the effect on non-English literature, an unbaised and comprehensive search should always be the foundation of a good review.
In a total of seven systematic reviews performed by our research group, one of the reviews was materially affected by non-English studies. But, since we have not published this yet, this remains anectdotal.
Cheers,
Janet E. Martin, BScPhm, PharmD
Project Leader, Evidence-Based Prescribing Initiative
London Heath Sciences Centre
Adjunct Professor, Dept of Physiology & Pharmacology
University of Western Ontario
Associate Scientist, Lawson Health Research Institute
Room S121
375 South Street
London, Ontario
Canada N6A 4G5
tel: 519-685-8500 x77814
pgr: 519-685-8500 x19103
fax: 519-667-6811
[log in to unmask]
>>> "Simon, Steve, PhD" <[log in to unmask]> 10/29/02 09:29 AM >>>
Dr.P.Badrinath writes:
> An interesting meta-analysis has been published in this
> week's BMJ. Weingarten SR, Henning JM, Badamgarav E, Knight
> K, Hasselblad V, Gano A Jr, Ofman JJ. Interventions used in
> disease management programmes for patients with chronic
> illness-which ones work? Meta-analysis of published
> reports. BMJ 2002 Oct 26;325(7370):925
>
> The objective of the authors was to systematically evaluate
> the published evidence regarding the characteristics and
> effectiveness of disease management programmes. This
> meta-analysis was restricted to English language articles
> published between 1987-2001. I thought there is always a
> potential for "language bias" in meta-analysis performed
> only on English papers and sent off my rapid response to
> BMJ.
Good for you! I've only seen one meta-analysis where this wasn't a potential
issue. In that study, the authors were trying to estimate the number of
deaths due to adverse drug reactions or medication errors in the United
States. Other than that, I think that restriction to English language papers
only is a mistake.
Steve Simon, [log in to unmask], Standard Disclaimer.
The STATS web page has moved to
http://www.childrens-mercy.org/stats.
|