G’day!

Yes, that list is really out of date. There was a new one on O&G published this year, too:

http://www.biomedcentral.com/1472-6874/6/5

And its references 2 to 6 are similar, recent attempts to measure this.

The problem is, it also depends then in which health service you mean it. Some hospitals are no doubt better than others, for example. It differs from country to country – and some payment systems or regulatory systems make particular forms of evidence-based practice impossible. I live in a country for example where the consultation times are so short, the evidence about the best ways to communicate with patients must be applied only a small proportion of the time, because the structure of the service does not allow for it.

One thing I think is pretty certain though: the 90% claim is one of those types of statements that probably caused a lot of antibodies against EBM. Life’s just not that simple.

Ruth’s question is a really interesting one though: how do we know if we are making progress?

Hilda

Dear Michael,
I think you are right that the statement is pseudoprecise. For starters is the denominator patients (those presenting to a clinic or hospital) or diseases? The latter is problematic as there are maybe 10,000 diseases including 5,000 single gene disorders - for which I assume there is not specific treatment? So the former denominator makes more sense. Then there is the question of treatments for these ....
That's a complex issue but Andrew Booth has compiled a database of attempts to answer the question with patients as the denominator:
www.shef.ac.uk/scharr/ir/percent.html
But remember some of these studies are a decade or more old, and RCTs have almost doubled over that time.
Iain Chalmers is building the Database of Uncertainties about the Effects of Treatments (DUETs), which will compile treatment questions that need answering with trials or systematic reviews. It will be a great source for guiding researchers and funders, but it won't help clarify the "90%" statement.
Season's Greetings
Paul Glasziou


M Power wrote:

Ruth

I think that you are asking an unanswerable question.

Brodie's statement that for "90% of medical conditions there is either no

specific remedy or effectiveness of treatment is unknown" hides the

complexity of medical decision making. The use of "90%" is not even

pseudoprecise: it is subjective and qualitative. Using a number

qualitatively in this way is a rhetorical technique to make an assertion

look objective and quantitative.

Let me explain a bit further.

Firstly, there is no way of counting medical conditions. So, the

denominator of your proportion is uncountable. For example, when does

blood pressure become a medical condition? The way that "hypertension" is

used might give the impression that raised blood pressure is one

condition. But, it might be several conditions, including pre-

hypertension, mild hypertension, moderate hypertension, severe

hypertension, hypertension resistant to treatment, hypertensive crisis,

white coat hypertension, ... All of these might require different

approaches to management, and thus qualify as different conditions.

Secondly, with the exception of a few extreme examples such as lethal

doses, there is always uncertainty about the effectiveness of treatment in

a particular individual and in particular subgroups (defined by age,

race/ethnicity, sex, co-morbidity, environment/risk factors,

severity, ...). Thus the numerator of your proportion is, strictly

speaking, zero. You might object that I am being too strict in requiring

perfect information about the efficacy of treatments in individuals. But,

there are insuperable difficulties in devising a metric for the quantity

of knowledge of a specified quality. Brodie implies that double-blind

randomised trials could be used to inform most medical decisions, so the

temptation is to try to count RCTs. But, for many conditions RCTs are

impractical, impossible, or unethical. Just one example: hormonal

treatments used for emergency contraception have never been compared with

placebo in a clinical trial, and there are uncertainties about optimum

doses in the usual situation, and more uncertainties about optimum doses

in special circumstances such as women who are obese, taking drugs that

induce the liver enzymes that metabolism levonorgestrel, etc. So, would

levonorgestrel be counted in or counted out of your numerator? There are

good arguments for "in" and for "out" decisions, but I doubt that you

could generalise the criteria for your decision so it would apply to all

other treatments. Further, how would you weight the counts: by how common

the condition/treatment is? Or, by how important it is? For example

compare the decision to use a particular brand of toothpaste with the

decision to use a potentially life-saving treatment in a potentially

lethal condition.

So, to conclude, (i) I think that, when Brodie said that 90% of medical

decisions were not evidence based, what he really meant is that "most

medical decisions could (and ought to) be based on better evidence". And,

(ii) I do not think that you can quantify ignorance in any meaningful way.

That every (good) research paper ends with suggestions for further

research suggests that the evidence could always be improved, i.e. our

ignorance is infinite. (Mathmaticians have discovered that some infinities

are more infinite than others. I suspect that our ignorance is high in the

hierarchy of infinities.)

Paul Glasziou and Iain Chalmers had a project to build a database of

treatments for which there is inadequate evidence. If Paul could tell us

what has happened to this project, his practical experience contrasted

with my theoretical analysis might shed some light on your problem.

Michael

Michael Power MD

Clinical Knowledge Author, Guideline Developer and Informatician

Clinical Knowledge Summaries Service www.cks.library.nhs.uk

Sowerby Centre for Health Informatics at Newcastle Ltd www.schin.co.uk

On Wed, 27 Dec 2006 14:50:16 -0600, Ruth Cronje <[log in to unmask]> wrote:

Hello everyone --

In 1980 in the Annals of Internal Medicine, DS

Brody claimed that "If medical practice were

based on truly objective scientific data, it

would be very easy to make defendable decisions

without any fear of self-reproach.

Unfortunately, medicine is not an accomplished

scienceSthere are tremendous gaps in scientific

knowledge.  Most of a physician's daily decisions

do not involve situations that have been tested

in double-blind, randomized trialsSfor

[estimated] 90% of medical conditions there is

either no specific remedy or effectiveness of

treatment is unknown" (p. 720)

Does anyone happen to know of a more current

reference (since 2000) that gives an updated

estimate of the percentage of medical conditions

for which there is no specific remedy or for

which the effectiveness of treatments is unknown?

Is it still 90%, or have we made some progress

since 1980?  A published estimate from the

peer-reviewed literature would be most helpful,

but I'd also be interested in people's own

estimates, based on their familiarity with the

literature.

Thanks for your help.

--

Ruth J. Cronje, PhD

Assistant Professor

Scientific and Technical Rhetoric

University of Wisconsin--Eau Claire

-- 

Paul Glasziou

Director, Centre for Evidence-Based Medicine,

Department of Primary Health Care,

University of Oxford www.cebm.net

ph +44-1865-227055 fax +44-1865-227036