Dear Sheri,
I agree with your analysis, what I too very commonly come across in my daily dealings with clinicians of all hues (including, I have to sadly admit, Professors as well) is an appalling lack of critical appraisal skills and a very shaky understanding of the principles of clinical epidemiology.
Coupled with this, is something very distasteful that many of us have skirted around but have not mentioned openly so far in our discourses on this discussion thread ie Conflicts of Interest (COI). Unless we can divorce COI from EBM and banish it completely (a very difficult task indeed), we will never be able to achieve our ultimate dream ie the intricate weaving of EBM into the tapestry of all clinical guidelines, especially from medical professional bodies.
The Institute of Medicine (IOM) has very recently published a very comprehensive book on 'COI in Medical Education, Research and Practice'. Chapter 7 deals with COI in relation to development of clinical practice guidelines:
Table of Contents
Front Matter i-xxii
Summary 1-22 (skim)
1 Introduction 23-43 (skim)
2 Principles for Identifying and Assessing Conflicts of Interest 44-61 (skim)
3 Policies on Conflict of Interest: Overview and Evidence 62-96 (skim)
4 Conflicts of Interest in Biomedical Research 97-121 (skim)
5 Conflicts of Interest in Medical Education 122-165 (skim)
6 Conflicts of Interest and Medical Practice 166-188 (skim)
7 Conflicts of Interest and Development of Clinical Practice Guidelines 189-215 (skim)
8 Institutional Conflicts of Interest 216-229 (skim)
9 Role of Supporting Organizations 230-239 (skim)
References 240-284 (skim)
Appendix A: Study Activities 285-294 (skim)
Appendix B: U.S. Public Health Service Regulations: Objectivity in Research (42 CFR 50) 295-301 (skim)
Appendix C: Conflict of Interest in Four Professions: A Comparative Analysis 302-357 (skim)
Appendix D: How Psychological Research Can Inform Policies for Dealing with Conflicts of Interest in Medicine 358-374 (skim)
Appendix E: The Pathway from Idea to Regulatory Approval: Examples for Drug Development 375-383 (skim)
Appendix F: Model for Broader Disclosure 384-391 (skim)
Appendix G: Committee Biographies 392-400 (skim)
Index 401-414 (skim)
PDF Summary
Report In Brief
Regards,
Ash
Dr Ash Paul
Medical Director
NHS Bedfordshire
21 Kimbolton Road
Bedford
MK40 2AW
Tel no: 01234795705
Email: [log in to unmask]
________________________________
From: Sheri Strite <[log in to unmask]>
To: [log in to unmask]
Sent: Saturday, 3 October, 2009 20:27:55
Subject: Re: Reliable, Relevant Information About VTE Prophylaxis in THR AND TKR Surgery and More
I fear that with some of the latest comments in this thread that Mike Stuart
and I run the risk of being misunderstood. I want to avoid us being
mischaracterized and clarify some points about the VTE guideline, so I provide
a little more information about us here. I also want to address some important
points raised in this thread. However, my full response is very long, so in the
interest of keeping this spare, I present some information and suggestions here
and have created a link on our webpage with a fuller response.
About the KPHI VTE Guideline and About Delfini
- The guideline is not an industry-funded guideline. It is the Kaiser Permanente
Hawaii VTE Guideline. We trained them, facilitated the process and did much
work on the guideline. We were provided some funding by them which covered
some of the training, and we donated much time.
- The guideline was not made available as a free sample. (We do not sell
clinical guidelines.) We are public-service oriented, are personally trying to
improve a huge problem in health care. We make almost all of our work
available for free on our website in an effort to help improve healthcare.
- Transparency is vital. We make it clear on the website that guideline
supporting documents are available upon request to me at [log in to unmask]
(The files are huge and not easy for me to put up on the website, but I will
happily email specific items upon request.)
- Mike and I are two individuals with a lot of EBM expertise who make a living
providing evidence-based quality improvement services. If we have a bias it is
toward the patient and toward science.
- We are not “playing in EBM space” — Mike, in particular, has incredibly deep
experience putting evidence-based medicine into practice. We explain more at
the longer response.
We announced the information about the guideline as a service to provide
another option to health care providers and patients. I, for one, know that I
would feel more comfortable having a compression device along with drug
therapy. Even though the evidence for compression is weak, weak evidence
does not mean something doesn’t work, it makes sense and harms appear to be
low. I believe I would be better protected. This is a recommendation I would
make to my mother. That is how I view all my work.
Addressing Ben’s last comments and Peter’s Question:
David is right. It is all about critical appraisal. Ben states, “It does not bode
well on EBM if the ACCP guidelines are not considered credible and useful. For
this is ‘as good as it gets’…” I would never accept a statement like this to rely
upon a guideline. Medical science-based content should be scrutinized
especially given the wide variation in critical appraisal skills and frequent need
for judgments to be made (not to mention other possible factors such as group
process, politics, influence, bias, etc.) which I state generally and not specific
to ACCP. Rather I do what Mike and I did. We do a critical appraisal audit
(and after seeing so many problems in quality, more frequently as of late we
favor critically appraising all the included studies if we agree with the search
methods and exclusions).
Regarding Peter’s Question: “How do we factor in the purely human tendency
to accept / value more a product for which we have paid than one which we
have "free" access?” We don’t think one should. We think that it is important
to have skills in critical appraisal, to look at the processes used, to audit the
information used, etc., to evaluate the work of any and all regardless of their
affiliation and relationships, etc.
At this point in time, we should all be very cautious about relying on any group
on the basis of reputation. Important problems can be found in even the most
respected sources such as Cochrane and others. We’ve seen this frequently in
our work. Lundh and Gotzsche found great variation in the quality of the work
of the 50 Cochrane groups in terms of assessing bias in studies. [Lundh A,
Gotzsche PC. Recommendations by Cochrane Review Groups for assessment of
the risk of bias in studies. BMC Med Res Methodol. 2008 Apr 21;8(1):22 [Epub
ahead of print]. PMID: 18426565.]
Considering objectivity is important, but in doing so it is also important to
realize that everyone and every group has some kind of bias. The biggest
problem we see is an overall lack of skill in critical appraisal concepts,
processes and critical thinking. The biggest need we see is a change in this
area which we think is key to solving many problems in health care. Critical
appraisal should be applied to every piece of health care information you
encounter.
Again, David is right. I write more about the problems Mike and I see and what
I believe to be important solutions at this link:
http://www.delfini.org/delfini_EBM_ListservResponse.htm.
Sheri Strite
[log in to unmask]
