Dear Ben and Richard

 

The purpose of this example was to illustrate the general point that when making decisions, the ‘value’ judgments are about outcomes that are yet to happen and it is also important to assess the probability of the outcome happening in a sensible way.  I agree with your point that the ‘value’ judgments that we attached to surrogate end points are very questionable.  I happened to have this data available that was suitable to illustrate a method of calculating probabilities and making a point.  

 

I agree that there is a desperate need to conduct trials with symptomatic end points that give us analogous information about outcome probabilities if we are to minimise over-treatment and under-treatment.  It would be naïve to think that over-treatment and under-treatment can be eliminated completely.  For that we would need perfect tests that perfectly predict treatment outcomes.  However, we can improve things dramatically with more thoughtfulness.

 

In any case, the reality is that if a diabetic patient is labeled with ‘diabetic nephropathy’ because of the surrogate marker of albuminuria > 200mcg/24hours, they are going to be subjected to the hassle of attending even more clinics, diets etc in addition to symptoms, so it may be worth taking a tablet to reduce the probability of such extra hassle (as well as the more remote possibility of dying with the symptoms of renal failure).  The problem is of course, that the patient will then probably die in some other way, which might be more unpleasant.  Ultimately we need unattainable endpoints that consist of optimal lives and deaths.  So, all outcomes have to be ‘surrogate’ to some extent.

 

I also don’t think that we should apply alarming diagnostic labels e.g. ‘diabetes mellitus’ (which means wasting away as a siphon) to anyone with no symptoms but for example two fasting sugars of 7.1mmol/l in the same way as we do to those with the misery of a full house of complications after many years of severe illness. However, the current thoughtlessness of diagnostic labels and the wooly thinking behind it will only be overcome if we estimate the probability of outcomes in a sensible and as accurate a way as possible, whether they are imperfect symptomatic or imperfect asymptomatic surrogate outcomes.

 

With best wishes

 

Huw

 

---

From: Djulbegovic, Benjamin [[log in to unmask]]
Sent: 11 October 2012 13:04
To: Lehman, Richard; [log in to unmask]; Huw Llewelyn [hul2]
Subject: RE: The causes and remedies of overdiagnosis and overtreatment

Dear Huw & Richard,

Thanks for starting this extremely interesting and important discussion. In the final analysis, it does come to the issue what is most rational approach to (medical) decision-making. Huw endorsed expected utility approach (EUT), the only existing theory that satisfies all (statistical/mathematical) criteria of rationality. According to EUT, for example, if NNT=10 and NNH=1, you would then treat anyone if probability of developing diabetic nephropathy is greater than 9%. Somehow, however, this would plainly indicate that we will overtreat majority of patients. Iztok Hozo & I wrote a little piece questioning this approach “Will Insistence on Practicing Medicine According to Expected Utility Theory Lead to an Increase in Diagnostic Testing?” (attached), which complements the arguments Ash & I made in the piece quoted below.

I believe that this is an important debate to have- looking to further thoughts from you and others\

Best

ben

 

 

From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Lehman, Richard
Sent: Thursday, October 11, 2012 7:46 AM
To: [log in to unmask]
Subject: Re: The causes and remedies of overdiagnosis and overtreatment

 

Dear Huw,

You have actually given a beautiful illustration of Ben's point.

It I am a patient with diabetes, why should I care whether I excrete albumin in small quantities? There may be an observational, statistical relation between albumin excretion and the development of end-stage renal failure, but the latter remains a very remote risk for any individual patient with so-called "diabetic nephropathy". I am not sure it is possible to derive a NNT based on real evidence of ARBs preventing anything that any patient would notice. We are extrapolating from a surrogate marker only. It is perfectly possible to reduce albumin excretion and increase the risk of adverse events. But because we don't have enough long-term evidence, we invent a bogeyman diagnosis called "early diabetic nephropathy" and treat that as it it were a clinical reality.

So it all comes back to our values - treat the herd and hope to reduce renal failure - or the patient's. The rot began when we measured her albumin excretion.

Richard

---

From: Evidence based health (EBH) [[log in to unmask]] on behalf of Huw Llewelyn [hul2] [[log in to unmask]]
Sent: Thursday, October 11, 2012 6:44 AM
To: [log in to unmask]
Subject: Re: The causes and remedies of overdiagnosis and overtreatment

Dear Ben

 

I agree that ‘values’ attached to outcomes always have to be involved, but these have to be multiplied by the probability of that outcome to give the ‘expected value’.  False positive and false negative results are created if test results are dichotomised e.g. into ‘high’ or ‘not high’ etc.  Doing this results in less accurate probabilities however.  This is illustrated with the example below.

 

The indication for treatment is often non-evidence based but some artificially dichotomised result e.g. in diabetic microalbuminuria, an albumin excretion rate of more than 20 µg/min - two standard deviations above the mean of the healthy population. However, evidence suggests that introducing this cut-off point causes errors because when compared to a non-angiotensin reducing agent as control, treatment with an angiotensin receptor blocker does not reduce nephropathy in normotensive patients with an albumin excretion rate of 20-40 µg/min (about 30% of the currently treated population). 

 

Furthermore, an albumin excretion rate of 41 µg/min is ‘abnormal’, but the number needed to treat with an angiotensin receptor blocker to stop one patient at 41 µg/min getting diabetic nephropathy within two years is about 100 (the probability of benefit is 1%).  When this is used to calculate an ‘expected value’, it would probably lead to a decision not to treat.  However, if the 'abnormal' rate is 60 µg/min, the number needed to treat is about 10 (the probability of benefit = 10%) and the ‘expected value’ from treatment would be higher, leading to a decision to treat.  Thus, even if the test result is in an ‘abnormal’ range, the actual value of the test results should be used when choosing whether to treat when the 'expected values' are calculated and then taken into account.

 

This is why I think that accurate evidence-based diagnostic and treatment indication criteria based on evidence-based probabilities are important.  Also combining System I and System II thinking appears to improve probabilities as discussed previously.

 

With best wishes

 

Huw

 

---

From: Evidence based health (EBH) [[log in to unmask]] on behalf of Djulbegovic, Benjamin [[log in to unmask]]
Sent: 11 October 2012 04:01
To: [log in to unmask]
Subject: Re: The causes and remedies of overdiagnosis and overtreatment

I personally think that overdiagnosis and overtreatment is not solvable on technical ground because it inevitably involves VALUES related to trade-offs of false-positives vs. false-negatives. It is closely linked to indication & prevention creep.

Ash Paul wrote a piece  a year or so ago that discussed some of these issues and may be of interest ( From Efficacy to Effectiveness in the Face of Uncertainty

Indication Creep and Prevention

JAMA. 2011;305(19):2005-2006. doi:10.1001/jama.2011.650.

Ben 

On Oct 10, 2012, at 6:44 PM, "Huw Llewelyn [hul2]" <[log in to unmask]> wrote:

Dear All

 

Do you agree that over-diagnosis and over-treatment could be much reduced if we were to arrive at diagnostic and treatment indication criteria in an evidence-based way (as argued in the following 'rapid response' sent this week to the BMJ: http://www.bmj.com/content/345/bmj.e6684/rr/606818 )?

 

Huw Llewelyn

Aberystwyth University