Here is the abstract:
Are Guidelines Following Guidelines?
The Methodological Quality of Clinical Practice Guidelines in the Peer-Reviewed Medical Literature
Terrence M. Shaneyfelt, MD, MPH; Michael F. Mayo-Smith, MD, MPH; Johann Rothwangl, MD, FACG
JAMA. 1999;281:1900-1905.
Context Practice guidelines play an important role in medicine. Methodological principles have been formulated to guide their development.
Objective To determine whether practice guidelines in peer-reviewed medical literature adhered to established methodological standards for practice guidelines.
Design Structured review of guidelines published from 1985 through June 1997 identified by a MEDLINE search.
Main Outcome Measures Mean number of standards met based on a 25-item instrument and frequency of adherence.
Results We evaluated 279 guidelines, published from 1985 through June 1997, produced by 69 different developers. Mean overall adherence to standards by each guideline was 43.1% (10.77/25). Mean (SD) adherence to methodological standards on guideline development and format was 51.1% (25.3%); on identification and summary of evidence, 33.6% (29.9%); and on the formulation of recommendations, 46% (45%). Mean adherence to standards by each guideline improved from 36.9% (9.2/25) in 1985 to 50.4% (12.6/25) in 1997 (P<.001). However, there was little improvement over time in adherence to standards on identification and summary of evidence from 34.6% prior to 1990 to 36.1% after 1995 (P=.11). There was no difference in the mean number of standards satisfied by guidelines produced by subspecialty medical societies, general medical societies, or government agencies (P=.55). Guideline length was positively correlated with adherence to methodological standards (P=.001).
Conclusion Guidelines published in the peer-reviewed medical literature during the past decade do not adhere well to established methodological standards. While all areas of guideline development need improvement, greatest improvement is needed in the identification, evaluation, and synthesis of the scientific evidence
> -----Original Message-----
> From: Ghosh, Amit K., M.D.
> Sent: Thursday, February 26, 2004 12:55 PM
> To: 'Djulbegovic, Benjamin'; [log in to unmask]
> Subject: RE: evaluating complex interventions...uncertainty...
>
> Dear Prof. Djulbegovic,
>
> Thank you for your interest. I share your views on uncertainty and have enjoyed reading your work.
> Here is the reference.It is available full text online:
>
> Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? JAMA. 1999;281:1900-1905.
>
> I will read your article
>
> Amit K. Ghosh, MD, FACP
> Associate Program Director, General Internal Medicine Research Fellowship
> Consultant
> Division of General Internal Medicine
> Assistant Professor of Medicine
> Mayo Clinic College of Medicine
> 200 1st Street, SW
> Rochester, MN, 55905
> Phone: 507-538-1128
> Fax: 507-284-4959
> [log in to unmask]
>
>
> -----Original Message-----
> From: Djulbegovic, Benjamin [SMTP:[log in to unmask]]
> Sent: Thursday, February 26, 2004 12:42 PM
> To: Ghosh, Amit K., M.D.; [log in to unmask]
> Subject: RE: evaluating complex interventions...uncertainty...
>
> Amish, hi
> I was very intrigued by your statement that "the preparations of guidelines are based on evidence in less than 50% of cases." Is this based on the actual data? If yes, would you mind sending me a reference?
>
> Regarding your other points about the role of uncertainty in medicine, my own view has changed over the years thinking of uncertainty more as a friend (or rather as an opportunity) instead of the enemy. The crucial concept here is recognition or acknowledgement that uncertainty (or disagreement) indeed exists. Once that is achieved, a structure/approach to clinical problem (or research question) can be easier formulated/understood. The same applies not only to physicians/researchers but to the patients as well.>
>
> In the context of clinical research several years ago, I summarized this view in the following article: Djulbegovic B. Acknowledgement of uncertainty-a fundamental means to ensure scientific and ethical validity in clinical research. Cur Oncology Reports 2001; 3:389-95>
>
> thanks
> best
>
> Benjamin Djulbegovic, MD,PhD
> Professor of Oncology and Medicine
> H. Lee Moffitt Cancer Center & Research Institute
> at the University of South Florida
> Department of Interdisciplinary Oncology
> SRB #4, Floor 4, Rm #24031 (Rm# West 31)
> 12902 Magnolia Drive
> Tampa, FL 33612
>
> Editor: Cancer Treatment Reviews (Evidence-based Oncology Section)
> http://www.harcourt-international.com/journals/ctrv/
>
>
> e-mail:[log in to unmask]
> http://www.hsc.usf.edu/~bdjulbeg/
> phone:(813)979-7202
> fax:(813)979-3071
>
>
>
>
>
>
> -----Original Message-----
> From: Ghosh, Amit K., M.D. [mailto:[log in to unmask]]
> Sent: Thursday, February 26, 2004 9:25 AM
> To: [log in to unmask]
> Subject: Re: evaluating complex interventions
>
>
> > Dear Paul,
> >
> > I have struggled with this issue and the reality of the fact is that there is uncertainty in several of our clinical decision making process in General Medicine. The compliance to guideline remains suboptimal and the preparations of guidelines are based on evidence in less than 50% of cases.
> >
> > Physicians have their own tolerance to uncertainty. Martha Gerrity's ( from Oregon, currently editor of JGIM) Physicians response to uncertainty scale has identified that orthopedic's and Urologist have the lowest tolerance to uncertainty, while Psychiatrist and family physicians have the highest tolerance to uncertainty.
> >
> > Ralph Stacey's( UK) classical certainty/ agreement diagram ( high agreement, high certainty- simple case versus low agreement, low certainty resulting in chaos) demonstrates why in a complex adaptive case physicians and patients differ in their perception of the problem.
> > Alternatively varying degree of adherence to the 6 steps of informed decision making process( Debra Roter's work) with minimal time being spent on discussing the uncertainty around a decision make most decision making process far from adequate.
> >
> > The question of patients and physicians risk aversion has also been considered in recent literature and makes interesting reading. In fact they have over 8 scales to measure uncertainty in medical students and physician ( Budner''s scale, Gerrity scale etc.) and medical students can be identified quite early as to their future selection of specialty. Though in US student debt load (> $ 100,000 on average ) has to be factored into the process of career selection ( economics over everything).
> >
> > I routinely give a talk on how to measure and deal with physician uncertainty, in an effort to teach resident and fellows how to develop techniques to deal with complex., and uncertain diagnosis where EBM falls short for e.g., by using tacit reasoning, improving communication, using formal decision analysis (here they fall asleep or leave). I however get the feeling that the disease oriented approach that we preach to our students ( and often believe ourselves) and the method of selection of students on the basis of objective testing and not an overall perusal of their abilities for rationalist reasoning and consideration of medicine as truly being a patient focussed discipline, makes General Medicine currently a threatened specialty. Many students find this complex reasoning process , 1) not uniformly stressed in medical school, 2) sub-specialize and be very comfortable with a series of high agreement, high certainty procedures.
> >
> > I would be happy to provide references on uncertainty on any of our members . A medline search using terms such risk aversion, physicians response to uncertainty scale, articles by Trish Greenhalgh in BMJ on Complexity Science would provide you will more than ample material in this field
> >
> > Amit K. Ghosh, MD, FACP
> > Associate Program Director, General Internal Medicine Research Fellowship
> > Consultant
> > Division of General Internal Medicine
> > Assistant Professor of Medicine
> > Mayo Clinic College of Medicine
> > 200 1st Street, SW
> > Rochester, MN, 55905
> > Phone: 507-538-1128
> > Fax: 507-284-4959
> > [log in to unmask]
> >
> >
> > -----Original Message-----
> > From: Evidence based health (EBH) [SMTP:[log in to unmask]] On Behalf Of Paul Glasziou
> > Sent: Thursday, February 26, 2004 3:15 AM
> > To: [log in to unmask]
> > Subject: evaluating complex interventions
> >
> > Dear All,
> > One of the tricky issues in EBM is the evaluation of complex interventions,
> > such as quality circles or critical appraisal training.
> > Do you think RCTs should always be the gold standard for evaluating highly
> > complex social interventions?
> > I'd encourage you to look at:
> > http://bmj.bmjjournals.com/cgi/content/full/328/7434/282>
> > and have your say in the rapid response,
> >
> > Paul Glasziou
> > Department of Primary Health Care &
> > Director, Centre for Evidence-Based Practice, Oxford
> > ph: 44-1865-227055 www.cebm.net
> ######################################################################
> This transmission may be confidential or protected from disclosure and
> is only for review and use by the intended recipient. Access by
> anyone else is unauthorized. Any unauthorized reader is hereby
> notified that any review, use, dissemination, disclosure or copying of
> this information, or any act or omission taken in reliance on it, is
> prohibited and may be unlawful. If you received this transmission in
> error, please notify the sender immediately. Thank you.
>
> ######################################################################
|