 |
 |
Thanks Michael,I was looking at the individual rather than the system values and decisions as I find it is clearer to observe change through individuals than systems. I learned this about myself on my first trip to assist others in a developing nation. When I looked at them all as a group I became overwhelmed with my small presence in the face of an overwhelming cultural breakdown but when I looked at one person I knew I could help that one. Sometimes the ones that change through individual communication are themselves systems changers…and I will cheer for them and you but for the present I shared the view from the corner I can visualize. When we add in personal risk thresholds and time constrained values the options become enough to make one dizzy. Interesting article on decisions in Nature http://www.nature.com/news/why-irrational-choices-can-be-rational-1.14517 I do understand it is needful to consider both and how relationship affects the system. I liked your illustration of the rowers and the anger derailing the group/system direction too.I would like to expand your idea of teaching critical appraisal not only to scientists but also the public especially since 50% of papers are now Open Access and it would be better for all if they can be shown how to apply what they read. For example I just read on Hilda Bastion's SciAm blog where a commentator shared they were a secondary school instructor of statistics and did not know what a meta analysis was….It looks like we all have work to do changing minds and bringing light one person at a time or through systems change or both depending on our unique areas of communication, expertise and expression…BestAmyDate: Monday, January 13, 2014 5:00 PMAmy: I like your idea of a visual metaphor, but it needs to be a little more complicated than a bicycle, because decision-making depends on evidences and values – both plural. However English grammar forbids us to say “evidences” because evidence is a mass noun, and mass nouns are so fuzzy that they defy accurate visualization.
The best metaphor I can imagine is a multi-player computer game in which a boat is propelled by two parallel banks of rowers, one on the starboard side and one on the port side — BTW the ancient Greeks took this technology to the limit with their triremes (http://en.wikipedia.org/wiki/Trireme), which could have several hundred rowers.
Players (representing values) control the rowers (representing evidences).
Each player can adjust the force and direction of each rower’s effort. But, players (values) are not all equal. And rowers (evidences) are not all equal. One or two players and one or two rowers will dominate — this represents our tendency to oversimplify.
The sum of all the rowing efforts determines the boat’s direction and speed, and represents the decision; the final destination represents the outcome.
You can see how a Greek tragedy arises when two powerful values oppose each other, or a Shakespearian tragedy when a dominant evidence and a dominant value are not critically appraised – think of Desdemona’s handkerchief and Othello’s jealousy — the boat spins out of control and is wrecked on the rocks.
Ben: your brief history of EBM is useful. But I would suggest that we do not ignore the parallel movements in improvement (and quality, and safety) and implementation. EBM can learn from them, and vice versa.
Kev: You are right to wave a red flag over hyperbole that could be counterproductive if taken outside our debating arena.
Rakesh, Neal, Zbys, Richard: thanks for your comments.
A final suggestion: the way forward could include a step back (in time) to re-emphasize critical appraisal of evidences, a step forward (in time) to introduce critical appraisal of values, and another step forward (in syncopated time) towards the Improvement, Quality, Safety, Implementation movements.
Michael
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Amy Price
Sent: 12 January 2014 02:25
To: [log in to unmask]
Subject: Re: Real .v. Rubbish EBM
Hi Ben and all,
What if we looked at evidence and decision making like two wheels on a bike? They both need to be full of substance, well connected, lubricated and working in balance with a competent driver with good vision on the seat to get the vehicle where we want it to go? By the same rationale that evidence is necessary but not sufficient for decision making, values are necessary and default to feelings based on social pressures and peer influence and lack focus without evidence and how to apply it. Maybe the bike needs a check up and a little maintenance to run safely and at optimum performance…
I am hopeful that those who are time and location privileged so they can attend this open event, will navigate well and come up with insights together they would not have considered alone. It seems to me that what you and others have pointed out for some time is starting to take root and people are exploring ways to grow it out. I know I have considered this carefully from the beginning after being privileged to enjoy the view from yours and others experience and expression and I have thought yes I see this but how can I best take it forward in the trenches. I hope there are more conversations and collaborations so each one of us that cares can have a part in the relationship as EBM continues to change health care and history…Thank you for all you have done, reaching out to those new to the evidence based way of thinking and your thoughtful papers, much appreciated…
Yours for evidence with informed decision making!
Best
Amy
From: "Djulbegovic, Benjamin" <[log in to unmask]>
Reply-To: "Djulbegovic, Benjamin" <[log in to unmask]>
Date: Saturday, January 11, 2014 7:19 PM
To: <[log in to unmask]>
Subject: Re: Real .v. Rubbish EBM
In trying to distinguish “real vs. rubbish EBM”, have we come a full circle? Some of us have pointed out for some time now that evidence is necessary but not sufficient for decision-making. It appears to me that the conference may end up confusing evidence for decision-making (and vice versa).
Ben Djulbegovic
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Michael Power
Sent: Saturday, January 11, 2014 4:31 PM
To: [log in to unmask]
Subject: Re: Real .v. Rubbish EBM
Hi Ruth
Thank you for the open invitation to your meeting to define real and rubbish EBM.
If I had been able to attend, I would have offered the points appended below.
Best wishes
Michael Power
Decisions, evidence, values, going off the rails
· Decision-making is at the heart of EBM, whether real or rubbish. So, it would be useful to understand what types of decisions we make, and how we do this.
· EBM decisions are about the health and healthcare of individuals or of groups. This classification is exhaustive and exclusive, although guidelines try to span the boundary between groups and individuals.
· All decisions, EBM or otherwise, are made on the basis of both information (evidence) and emotions. The use of evidence is conscious; the influence of emotion is usually subconscious. Emotions can outweigh what rationally seems to be completely convincing evidence.
· Emotions and values are intimately linked. When a decision is in line with (or contradicts) a value, a corresponding emotion is felt and guides the response. Values generally are learnt unconsciously and involuntarily, but can be consciously taught and conscientiously learned.
· Each of us has many different values, some of which are contradictory, incompatible, or incommensurable. Think of “pro-choice” and “pro-life”. I (hopefully like most people) am for both.
· Most people want to do a good job, but organisations and “the system” can turn them into robots or petty box-ticking bureaucrats, more concerned with process than outcomes. A few people are in it for themselves.
Decision-making in Personal and in Public health/care
There are notable differences in the use of evidence and the role of values between EBM decision making in personal health and in public health.
· Evidence. In public healthcare, statistical evidence from groups can be confidently used to infer statistical outcomes for groups. But for in personal healthcare we really want the individual’s outcomes, not those for a population. Bounded rationality (incomplete evidence, imperfect reasoning) is a particular challenge when practicing personal EBM, because our natural tendency is to oversimplify. For example, the media tend to report just the direction of effect: a drug is beneficial (or harmful).
· Values. Values are important in both personal and public EBM, but we need to be aware of the risk of not taking them fully into account. This is particularly true for public health (and guidelines), because decisions are tend to be made on economic values (or non-economic values that have been monetized without considering what has been lost).
· Guidelines. Guidelines should support real EBM, but can be misused in rubbish EBM when box-ticking adherence to processes precludes taking into account relevant personal values and evidence not mentioned in the guidance.
Real vs Rubbish EBM
· Real EBM conscientiously critically appraises and takes account of all the relevant evidence and all the relevant values in order to support the patient or policy-maker to optimise their decision-making. Rubbish EBM does not.
· Clinicians practicing real EBM need to understand, and be able to communicate to patients, not just the direction, but also the size and importance of the average outcome (e.g. mean, or median, or mode); the range of outcomes and shape of the distribution of outcomes (e.g. for cancer prognosis; the typical long tail after the median); the risks of mistakes, bias, blindness, and bullshit; and the risk of unknown unknowns (much of the history of medicine is a series of cautionary tales about unknown unknowns that should have been anticipated). Rubbish EBM does not do this.
· The values of real EBM are the reflected in a key aspect of EBM leadership: to recognise an opportunity for improvement, and then to have the gumption to (responsibly) do something about it. Rubbish EBM does not see, or ignores opportunities for improvement.
· Clinicians practicing real EBM need to be able to understand, identify, elicit the relevant values in themselves, their patients, and their culture. Rubbish EBM does not do this.
· Rubbish EBM prioritises process over outcomes, means over ends, the practitioner or organisation over the patient. Real EBM has its priorities right.
The next step: prevention, diagnosis, and treatment of rubbish EBM
Once we have distinguished between real and rubbish EBM, we need to make the next step, which is to prevent or to diagnose and treat rubbish EBM ‑-- even if this is not on the agenda for the meeting on the 14th.
Most people want to do a good job, but organisations and “the system” can turn them into robots or petty bureaucrats, more concerned with process than outcomes. And, some people when given the opportunity, prioritise their own advancement (prestige, income, comfort) over the quality of service they render.
On both sides of the Atlantic, and on either side of the equator, there are major political and important professional initiatives to change behaviour in healthcare systems by changing people’s values. Generally, the intervention seems to be a mantra-like repetition of a slogan such as real EBM, Quality, Improvement, Patients first, Shared decision-making, Affordable Healthcare, Payment by results, … Theodore Marmor exposed the gap between this type of rhetoric and the reality: the slogan is presented an effective force for improvement, but serial failures to meet expectations, far from dampening wishful thinking, simply inspire senior bureaucrats and policy makers to invent another slogan to promote the next fad.
Why has so little research on values been done (or at least published in the types of journals Richard Lehman reviews). Why aren’t the following sorts of question being asked?
· What values do people who wish to practice real EBM need to be taught, to know, to apply, and to teach?
· What is the optimal way of doing this? What is the role of role models?
· When someone who originally wanted to practice real EBM develops into a practitioner of rubbish EBM, what unwanted values have been gained, and what desirable values have been lost or subverted?
· How can rubbish EBM be prevented, or promptly detected and treated?
What factors create petty bureaucrats, inhibit responsible innovation, foster organizational stupidity, promote self-seeking, and allow wilful blindness and normalization of deviance?
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Ruth Davis
Sent: 09 January 2014 15:45
To: [log in to unmask]
Subject: Real .v. Rubbish EBM
Tuesday 14th January 2014
10:00 -12:30 (Coffee available from 09:30)
Kellogg College, 62 Banbury Road, Oxford
In September 2013 Trish Greenhalgh challenged Carl Heneghan to promote 'real versus rubbish EBM' on Twitter. An example of rubbish EBM might be (for example) putting a 75 year old on statins because the guidelines say you should, with no account of their social situation, comorbidity, life expectancy etc. Real EBM from a clinicians perspective would include taking patient values and circumstances into account alongside evidence from guidelines and from an organizational perspective it would include measures to avoid a managerialist, technocratic approach to promoting guideline adherence.
A meeting has been set up in Oxford with the aim is to define real and rubbish EBM more rigorously. As part of this meeting there will be an open session for all who would like to hear views on real v rubbish EBM from the experts.
Spaces a very limited, to register your interest please email [log in to unmask].
Titles include:
Carl Heneghan - Why most research should not reach the appraisal step
Paul Glasziou - Using healthy skepticism in the patient's interests
Jeremy Howick - A new generation of bias in EBM
Des Spence - EBM as a marketing tool for Big Pharma
Neal Maskrey - It is not real versus rubbish, but EBM versus EBM2
Jon Brassey - An alternative system for systematic review production
Richard Lehman - Patient Centred Evidence - the Unicorn that must be found
Margaret McCartney - Too much treatment for the well, and not enough for the sick
Mike Kelly - Philosophical reflections on rubbish EBM
Iona Heath - EBM is a means – but to what end?
Ruth Davis
CEBM Centre Manager
Department of Primary Care Health Sciences
University of Oxford
Tel: 01865 289322
Tw: @CebmOxford