First, the distinction between level 1 and level 2 depends in part on the level of evidence system you are using. The CEBM criteria use level 1 for systematic reviews of randomized trials and also use level 1 for well-done randomized trials. They further distinguish them as level 1a and level 1b. The SORT criteria use level 1 for systematic reviews of randomized trials and also for well-done randomized trials. Also keep in mind that a systematic review of lower-quality evidence should not imply that we have better evidence because a systematic search was done. If the evidence we have is limited to lower-quality randomized trials (level 2), then a systematic review of those trials should also be considered level 2. A high-quality systematic review can appropriately conclude level 2 evidence or insufficient evidence. For the explicit criteria we use (http://www.ebscohost.com/dynamed/levels.php): * DynaMed criteria for level 1 (likely reliable) evidence for a randomized trial 1. Full-text report available in English (or language well understood by participating editor) 2. Clinical outcome (also called patient-oriented outcomes) 3. Random allocation method (i.e. not assigned by date of birth, day of presentation, "every other") 4. Allocation concealed 5. Blinding of all persons (patient, treating clinician, outcome assessor) if possible 6. Intention-to-treat analysis comparing groups according to randomization 7. Follow-up (endpoint assessment) of at least 80% of study entrants AND adequate such that losses to follow-up could not materially change results 8. Adequate statistical power 9. No other factors contributing substantial bias, such as a. Differences in management between groups other than the intervention being studied b. Differential loss to follow-up c. Posthoc analysis d. Subgroup analysis e. Baseline differences between groups f. Unclear how missing data is accounted for * DynaMed criteria for level 1 (likely reliable) evidence for a systematic review 1. Full-text report available in English (or language well understood by participating editor) 2. Clinical outcome (also called patient-oriented outcomes) 3. Systematic search 4. Explicit inclusion criteria 5. Systematic selection of included studies 6. Evaluation of study quality 7. Additional criteria if meta-analysis a. Studies are clinically appropriate for pooled analysis (reasonably similar populations, interventions, methodology and outcomes) b. Meta-analysis not limited by statistically significant heterogeneity 8. Conclusion based on primary studies meeting Level 1 evidence criteria 9. Adequate statistical power 10. No other factors contributing substantial bias, such as a. Inclusion criteria that appear to inappropriately exclude important evidence b. Subgroup analysis c. Indirect comparisons -------------------------------------- Brian S. Alper, MD, MSPH Editor-in-Chief, DynaMed ( <http://www.dynamicmedical.com/> www.DynamicMedical.com) Medical Director, EBSCO Publishing 10 Estes St. Ipswich, MA 01938 office (978) 356-6500 extension 2749 cell (978) 804-8719 fax (978) 356-6565 home (978) 356-3266 "It only takes a pebble to start an avalanche." _____ From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Tom Jefferson Sent: Friday, August 03, 2007 5:27 AM To: [log in to unmask] Subject: Re: Level of evidence What Liz says is sensible and I agree. How would classify a systematic review which incorporates a meta-anaylsis with cohort studies and case-control studies in the same stratum (it must have been very difficult to do, but the authors achieved it)? Or one which dodges the main question because the evidence to answer that question does not fit with the researchers' ideas or what the sponsor wants? Or a systematic review which ignores proven (published) evidence of suppressed safety data from trials in very small children? They all exist in published format. Tom. 2007/8/3, Liz Payne <[log in to unmask] <mailto:[log in to unmask]> >: I'd be very wary of designating every systematic review as level 1 evidence and every RCT as level 2 evidence. It would be very important to appraise the quality of research whatever its methodology. A poorly conducted systematic review could be riddled with bias and error, for example, and I wouldn't want to assume that it was better evidence than a well thought out, rigourously conducted RCT, without looking closely at their design and execution. Liz Payne ----Original Message---- From: [log in to unmask] <mailto:[log in to unmask]> Date: 08/02/2007 23:10 To: Subj: Re: Level of evidence Hi Craig, I'd be going with level 1 because for me the inherent value of a systematic review is that you can be confident that you have (at least in theory) all the evidence. For that reason, a systematic review which contains only one RCT is still higher level evidence than that one RCT alone. I'll be interested to hear what others think, Tari Turner Senior Project Officer Monash Institute of Health Services Research Monash Medical Centre Locked Bag 29 Clayton VIC Australia 3168 Ph: +61 3 9594 7568 Fx: +61 3 9594 7554 From: Evidence based health (EBH) [mailto:EVIDENCE-BASED- <mailto:EVIDENCE-BASED-> [log in to unmask] <mailto:[log in to unmask]> ] On Behalf Of Craig Lockwood Sent: Friday, 3 August 2007 4:02 PM To: [log in to unmask] <mailto:[log in to unmask]> Subject: Level of evidence Dear All, Just had a scenario brought to my attention and hoped to benefit from your collective wisdom. If a systematic review with meta analysis is level 1 evidence, and If a well designed RCT with adequate power is level 2 What is a systematic review of 2 or more RCTs that were not amendable to meta synthesis? Is it level 1 because they come from a systematic search and have been narratively summarised Or is it level 2 because although the search was systematic, narrative summary does not increase the point estimate or power? With thanks Craig -- Dr Tom Jefferson tel 0039 3292025051 Ti sei iscritto alla newsletter di Attenti alle Bufale? No? Vai sul sito www.attentiallebufale.it <http://www.attentiallebufale.it> e digita il tuo indirizzo di posta elettronica in alto a sinistra dove dice "Vuoi ricevere in anteprima le migliori dritte di Sun Tzu?"