Print

Print
hi Paul and Cathy, and if I could add, what is interesting is that one could have a level 1 study (RCT) yet poorly conducted with poor methods (so poor quality) yet a weaker cohort observational design (level 2 lets say) that is very well done and strong methods. So I like an explanation where 1.) this is a bot more complex than simply levels and quality for the researcher has some decision making to do and lots of power in their hands and 2.) one has to a priori, stipulate what you are doing, going to do, and thinking and be willing to and able to defend your reasoning. 

just my 2 cents for Cathy's question is a good one and leads to many issues. 
 
 
 
 
 
 
Best,

Paul E. Alexander
 





From: Paul Glasziou <[log in to unmask]>
To: [log in to unmask]
Sent: Thu, January 27, 2011 1:34:22 AM
Subject: Re: question: level of evidence vs quality of study vs cochrane scores

Dear Yuhong,
You are indeed correct that:
1. "Level of evidence" applies to the body of evidence (and GRADE is becoming the standard for this)
2. Quality appraisal is for single studies (eg Jadad or Cochrane Risk of Bias) and is *one* factor in the Level of Evidence

A drawback for getting a level of evidence is that is requires a systematic review (6+ months work).
So the CEBM's new tool is a search heuristic for when you need an assesment today or this week.
(So start by aksing: Is there an exisiting up-t-date systematic review? No? Well, what about ...)
http://www.cebm.net/index.aspx?o=5653

So in answer to your 2nd question, the CEBM table no longer gives a definitive "level". That usually inappropriate based on analysis of a single study.
But a clinician using a trial to help with patient care does not need to assign a "level", but rather ask "Is this a good enough basis for a decision?"

I hope that helps,

Best wishes
Paul Glasziou



On 1/27/2011 9:41 AM, Cathy (Yuhong) Yuan wrote:
Dear Dr Paul and Dr Howick and Dear colleagues,
 
Sorry for asking again... I would like to have your expert opinions for my question about level of evience vs quality of study.
 
I am working on a systematic review that I am giving the level of evidence for some of the recommendations, which I am using the new CEBM-2 tool (http://www.cebm.net/index.aspx?o=5653), it is simpler than the CEBM1 and the GRADE tool, and it is useful.
 
For my understanding, "level of evidence" is different than the quality of individual study, therefore, we have tools like GRADE or CEMB for grading the level of evidence, and we have sales such as Jadad and many others to be used for assessing the quality of a study (although many studies have suggested we should not use a sales to assess the quality of a study, see Juni P, 1999, the harzerds of scoring the quality of RCT for meta-analysis).
 
Therefore, a level of evidence should be given to the overall of the evidence instead of a signle study. I mean, we usually don't call a meta-analysis as a level 1 study, a RCT a level 2 study. Instead, we call the individual study a high quality or low quality study, then give an overall level of evidence to all the evidence available (eg, level 2 of evidence can incude good and low quality of RCTs for a particular statement), just as like when we used GRADE for our Cochrane review, it is for an outcome conclusion but not for an included RCT.
 
However, some references that looked at the correlation/ relationship between "level of evidence" and "quality of study" were brought to my attention, and it is the first time I heard about a tool named "Cochrane scores".
 
Please see below: for example, the Poolman 2006 study gave each RCT an "level 1 evidence or level 2 evidence" then compared it with the "Cochrane quiality of reporing scores" and then gave some interesting conclusions. It stated that this Cochrane scores "was devised by the Cochrane Bone, Joint and Muscle Trauma Group.... This scoring scheme covers aspects of internal and external validity for the assessment of methodological quality.". So I thought it maybe something new, but then when I searched I found this cochrane scores had been used by Brown CH in 2000.
 
The more I read, the more I am confused, Because, as a cochrane reviewer, I have never used or heard about a cochrane scores before.
 
So my questions are:
 
  1. Is the "Cochrane scores" still using by some groups? Should it be used for "quality or reporting"? If so, what is the difference between it and the CONSORT statement? Can anyone give me some experience on its use?
  2. Is it correct we give an individual study a level, by calling it a "level 1 study" or a "level 2 RCT", in stead, we should call it "level 2 evidence" (they may have 10 different quality RCTs)?
 
I really appreciate all your help in your busy schedule.

Poolman RW, Struijs PA, Krips R, Sierevelt IN, Lutz KH, Bhandari M. BMC Med Res Methodol. Does a "Level I Evidence" rating imply high quality of reporting in orthopaedic randomised controlled trials? 2006 http://www.biomedcentral.com.libaccess.lib.mcmaster.ca/1471-2288/6/44

 

Brown CH, Berndt D, Brinales JM, Zong X, Bhagwat D. Evaluating the evidence of effectiveness for preventive interventions: using a registry system to influence policy through science.Addict Behav. 2000 Nov-Dec;25(6):955-64.

 

Best regards

 

Yuhong



-- 
Paul Glasziou
Bond University
Qld, Australia 4229