I agree Craig but was not necessarily thinking only within the qualitative domain. Why should other 'good' but maybe not 'excellent' studies be excluded on the basis that they are not RCTs (eg. cohort studies, case-control etc. etc.)in line with Muir Gray's 'best available' rather than 'best possible' evidence. So not a case of 'less severe' criteria but more like - 'tell it like it is' - this is currently the best we have to make the decision.
Clinicians have to take some responsibility. Not just be told what to think and do! Guidelines are for guidance - they are not rules. I suppose this links to a recent discussion thread on 'real world EBM'. I am concerned (as Mark suggested below) that we are 'whittling away' potentially useful evidence. That some may continue to choose 'the way we've always done it' over less than perfect but sometimes otherwise useful research. Polarising method and polarising EBM is not helpful and while I do not agree with Holmes et al., in the global context of 'deconstructing discourse', you can (kind of) see where they are coming from.
Donna Waters
-----Original Message-----
From: Craig Lockwood [mailto:[log in to unmask]]
Sent: Friday, 25 August 2006 10:10 AM
To: Donna Waters; [log in to unmask]
Subject: Re: Deconstructing the evidence-based discourse in healthsciences)
Hi Donna
One of the issues that comes to mind is that the terminology and criteria
for measurement of "bias" and "rigour" as used in quantitative studies
really don¹t translate across other paradigms. Forms of qualitative
research answer important questions, but "bias" is almost a redundant
concept in qualitative evidence.
So should criteria be "less severe" or are separate, and appropriate
criteria for appraisal that carry equal weight the answer? Certainly there
are organisations - such as the Joanna Briggs Institute that accepts
research evidence regardless of paradigm as being more rigorous than expert
opinion and has methods of attributing levels of evidence across paradigms -
I would argue that its not just nursing that needs a more pluralistic,
inclusive approach to evidence - allied health, and medicine also have
questions that are not solely focused on effectiveness. Cochrane has a
qualitative methods group, the Campbell collaboration has similarly eclectic
information needs, and the Joanna Briggs Institute has long been a leader in
this area.
Regards
Craig
> RE: "Is this a justifiable state of affairs, or should there be less 'severe'
> exclusion criteria so that valuable evidence can be gleaned from 'less
> rigorous' studies?"
>
> What happened to grading the evidence? I have been around EBP long enough to
> remember that initially, guideline development included the use of grading
> systems to denote the 'level of evidence' used in making the recommendations.
> Have we forgotten that a clinician and patient/client can actually use their
> own judgement and experience to make up their mind about whether the
> recommendation is appropriate in a given context (as long as they are fully
> aware that guidance may come from 'less rigourous' studies)?
>
> The idea of formulating guidance (only) from RCTs has always been a problem
> for nursing. Nursing questions are sometimes best answered by other methods
> (as has been pointed out in this discussion thread already). Matching the
> 'kind' of clinical question to the most appropriate study design has been a
> sensible way for nurses to proceed towards evidence-based nursing practice,
> but doesn't always cut it with EBM devotees. I also think the way EBM has been
> 'sold' in nursing (mostly by nurses themselves) has not made this clear to a
> profession which is dedicated to patient/client 'care' (which does not always
> translate to an intervention or treatment) and is still maturing as an
> academic discipline (in Australia anyway).
>
> I joined this list in 1998 and I don't think I have ever posted anything - but
> Ronán Conroy's posting made me laugh so hard...it has made my day and thank
> you all for a most interesting discussion. It all seems very healthy to me!
>
>
> Donna Waters
> The College of Nursing
> Sydney, Australia
>
> -----Original Message-----
> From: Evidence based health (EBH)
> [mailto:[log in to unmask]] On Behalf Of Mark Pearson
> Sent: Friday, 25 August 2006 1:01 AM
> To: [log in to unmask]
> Subject: Re: Deconstructing the evidence-based discourse in health sciences)
>
> I am in agreement with much of what has been said about the Holmes et al paper
> - it has a cavalier attitude towards making sweeping statements based on
> tenuous evidence, and (problematic though such research may be) the apparent
> unwillingness to conduct methodologically rigorous research on (e.g.) the
> Cochrane Collaboration does not help their case. Despite all this, I still
> think that Michael has a point when he says that "the ideas aren't that
> outrageous... any group defines ways of doing things that excludes others" -
> maybe this is at the root of the authors' use of the word 'fascist' - they had
> gotten fed up not being listened to, and thought that courting controversy
> might be the best option for stimulating debate about the issues concerned
> (and perhaps it has worked, going by the postings on this and the Evidence
> Network mailing lists!)
>
>
>
> I would like to ask some of the other members to elaborate a little more on
> their postings if possible, in the main because it seems that their
> experiences of EBM are so polarized that I have trouble reconciling them. Roy
> states that he has found it "hard to get EBM related concepts into medical
> education [and] funding for EBM work", whilst Owen states that his involvement
> with the education of primary care Doctors and Nurses has been notable for its
> production of practitioners who "just want to be told who to prescribe what
> to". I think what Owen is getting at here is that despite EBM's ideal of
> integrating diverse forms of knowledge (research evidence, Doctor's
> experience, patient's values), the effect has been to produce practitioners
> unable to think. Yet Roy is concerned that EBM concepts have not affected the
> practice of medical education enough!
>
>
>
> In response to Michael (Power), who criticises the rationale in the paper for
> stating that "98% of literature is deemed scientifically imperfect". Yes,
> there are some research questions that can only be rigorously answered using a
> particular method, but there are also many that require the melding of
> knowledge from a number of different methods in order to be adequately
> answered. In my own research in Public Health at NICE, broadly stated it may
> be said that there is concern from the expert committees about (and a somewhat
> lethargic institutional recognition of) the 'whittling away' of research on
> the basis of a hierarchy of evidence, only to be left with 1 or 2 RCTs (out of
> an initial 1000+ citations) upon which to formulate guidance. Is this a
> justifiable state of affairs, or should there be less 'severe' exclusion
> criteria so that valuable evidence can be gleaned from 'less rigorous'
> studies?
>
>
>
> Mark Pearson
>
> School of Law and Social Science
> University of Plymouth
> 20 Portland Villas
> Drake Circus
> Plymouth
> Devon
> PL4 8AA
>
>
> ________________________________
>
> From: Evidence based health (EBH) on behalf of James Woodcock
> Sent: Thu 24/08/2006 11:26
> To: [log in to unmask]
> Subject: Re: Deconstructing the evidence-based discourse in health sciences)
>
>
> Michael,
>
> I think you are too harsh on deconstructionism. Any method can be used badly.
> I think it is noteworthy that their article was not published in a sociology
> journal, my guess is that it would not have been accepted.
>
> James
>
>
> Roy, thanks for bring this paper to our attention.
>
> I wonder how the authors (and their supporters) would feel sitting in a court
> where the only "evidence" allowed was accusations and testimony by their
> enemies. This is how fascism works, and this is how Dave Holmes RN PhD, Stuart
> J Murray PhD, Amélie Perron RN, PhD(cand) and Geneviève Rail PhD judge
> evidence-based healthcare.
>
> To justify this assertion I shall deconstruct one of their deconstructions.
> They say
>
> "For example, one of the requirements of the Cochrane database is that
> acceptable research must be based on the RCT design; all other research, which
> constitutes 98% of the literature, is deemed scientifically imperfect."
>
> The supporting citation is not to the Cochrane Library or Manual of systematic
> reviews, but to "Traynor M. The oil crisis, risk and evidence-based practice.
> Nurs Inq 2002; 9: 162-9." By relying on bigoted and uninformed testimony
> rather than evidence they demonstrate a fascist disdain for the truth. They
> also demonstrate a common psychological failing: seeing your own faults
> magnified in other people.
>
> Similarly, to talk about "the Cochrane database" in this way demonstrates
> their ignorance of what the Cochrane Collaboration does. The Cochrane Library
> has about 6 databases or registers (counting is problematic because it is
> difficult to tell from the website whether a "product" is a separate database
> or a filtered view of a larger database). If they wanted to refer to the
> register of controlled trials they should have said so. But, if they had done
> this it would have exposed the wooliness of thinking in the accusation that
> 98% of the scientific literature is imperfect.
>
> Putting in the number "98%" demonstrates another common psychological failing:
> when you are insecure, throw in a few pseudofacts to cover your ignorance.
> Have they (or anyone else) tried to count "all other research"?
>
> My final deconstruction (I could go on, but it would be pointless) is to note
> the scope of the accusation that "98% of the literature is deemed
> scientifically imperfect". Holmes et al conveniently do not answer the
> question "Imperfect for what?". If it had occurred to them that this is an
> important detail they would have soon realized that science has a large
> toolbox of study methods, and good scientists use the appropriate tool for the
> job. Only a deconstructionist could imagine that an RCT would be used to
> assess the management of avian flu in a particular country. An observational
> study is needed.
>
> Deconstructionists avoid details and facts because they impair their ability
> to insinuate, slur and slander, and because this might highlight the absence
> of constructive criticism. This is a pity, because there is much room for
> improvement in the aspirations, rhetoric and practice of evidence-based
> healthcare. It is a double pity because people who are aware of the
> shortcomings of evidence-based healthcare too easily accept fascist-style
> judgements about evidence-based healthcare.
>
> Deconstructionism is fascist surrealism.
>
> Michael
> ========================
>
> Michael Power MD
> Clinical Knowledge Author, Guideline Developer and Informatician
> Prodigy Knowledge http://www.prodigy.nhs.uk/
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>
>
> Hi Roy et al,
> I've read the paper today and found it readable and pertinent. I've been a
> UK GP for a large number of years and work in a deprived inner city, have an
> MSc in health sciences and clinical evaluation and have taught EBM. I agree
> with the paper, in the UK we have edicts from NICE telling us what to do
> with our patients, we are even financially rewarded for measuring all sorts
> of things, I am told that I 'should' now be screening diabetics for
> depression now (never mind the opportunity cost for what else I could do
> with that time), we 'should' be getting blood pressures below this and that,
> research papers still distort the truth a) by quoting relative risk
> reductions and b) by asking research questions that assume longevity is the
> gold standard and b) giving undue weight to confidence intervals bolstered
> by the internal consistency of the paper's stats, and c) assuming that just
> because a risk can be calculated that it is somehow accurate when we know
> that's a myth.
> Maybe most doctors just like to be told what to do and to be given a cookery
> book of recipes to follow. In my experience of teaching primary care doctors
> and nurse how to calculate risk, this isn't something they will do in the
> course of routine work, they just want to be told who to prescribe what to.
> They have often in fact stopped thinking about what they are doing.
> The public health specialists I've spoken to don't see individual risk
> communication as important, "Thats your problem" they say to me, a GP.
> Eminent epidemiologists have hailed the concept of the 'polypill' ignoring
> the implication that the majority of the over 40s will be pathologised and
> labelled as 'ill', with false expectations of what medicine can deliver
> fostering dependence, the EBM movement has embraced and imposed colonic
> cancer screening but the papers have inadequately measured changing health
> beliefs and the emotional adverse effects .
> I accept that EBM and the controlled trial has achieved a huge amount
> especially in therapeutics, but I suspect the current mania for
> 'preventitive' treatment and screening is doing more harm than good in
> preventitive medicine terms, and I am keen to avoid doing harm.
> I agree with the paper that the discourse of EBM is 'dominating' with its
> moral overtones, and that this does actively discourage and exclude
> alternative views, and the term fascism may feel nasty because of its
> historical connotations but I feel it is in fact justified.
> It feels especially healthy to have these sort of ideas aired, I feel they
> should be taken seriously. Just read a few of the editorials in the BMJ,
> look closely at the language and the way its coercive intent on clinicians.
> This paper in fact feels overdue.
> As a way forward, I think we should take a step back, research findings
> should be presented more neutrally, the adverse effects of screening for
> example should be explored in more depth, family practitioners should have
> more freedom to decide for themselves what treatments feel worthwhile
> without being incentivised to follow the rule of EBM edicts like sheep, we
> should enable and introduce more variability and innovation not seek to
> remove it.
> Owen
> Owen Dempsey
> GP
>
|