thanks Paul
good point
agree
I boweth to thy superior intelect, knowlede and wisdom!!
gf

________________________________

From: Evidence based health (EBH) on behalf of Paul Glasziou
Sent: Tue 10/01/2012 05:23
To: [log in to unmask]
Subject: Re: Ten commandments for testing - some problems?


Dear All,
This has been a great idea and discussion to watch :-)
However, before spreading further though, its important to get the maths straight. 
A couple of the items that seem to have problems are:
"3. Thou shalt know that if thou dost order a test with a positive predictive value below 50%, thou mightest as well toss a coin."
That's incorrect for several reasons. A test may be *very* useful even with a low PPV - if the negative predictive value is good, and that is what is needed.
For example, the Ottawa Ankle rule is a very sensitive "test" for fractures*, so negative results rule out (SnNout). But the Rule is not very specific, so the positive predictive value is generally less than 50%.
That still seems a useful test - saving around 25% of ankle Xrays - but with weak PPV.
A "coin toss" is indeed a useless test - but that does not mean a PPV less than 50%, but that the pre-test and posi-test probabilities are the same. That is the predictive value positive would be the same as the pre-test probability - which may be above or below 50% depending on clinical circumstances.
So how can you pick a useless test? Two ways to tell are: if the Youden Index (= 1 - (sensitivity + specificity) ) is zero or (equivalently) the (diagnostics) Odds Ratio is 1 (as hence both LR+ and LR- will also be 1).
So 3 might read something like:
"3. Thou shalt know that if thou dost order a test with a Youden Index near 0, thou mightest as well toss a coin."

There is a related problem with Commandment 4:
"4 ... Thou shalt recognise that in a low-prevalence population, even a very sensitive test hath poor predictive value, and if thou shouldest order such a test, thou mightest as well toss a coin."
Again if we are interested in ruling out, then we might us the sensitive test (SnNout) in low prevalence populations. For example, using highly sensitive d-dimer to rule out PE in someone with a some suspcion, but low probability, of PE.
Incidentally, prevalence matters most when tests are only modestly accurate. For a perfect test (100% sensitive and specific) prevalence becomes almost irrelevant. And for either SpPin (100% specific) or SnNout (100% sensitive), then its of low relevance for the +ve or -ve results respectively.

Happy Commandment making ;-)
Paul
* see BMJ. 2003 Feb 22;326(7386):417.



On 1/10/2012 2:00 AM, Ash Paul wrote: 

	Dear colleagues,
	
	
	Here's the corrected draft, with grateful help received from Ms Catherine Ebenezer:

	
	1. Before thou dost order a test, thou shalt remember that "All tests can do harm. Many tests do good. Some tests do more good than harm". 
	
	2. Thou shalt know the predictive positive value and predictive negative value of the test before thou dost order it. 
	
	3. Thou shalt know that if thou dost order a test with a positive predictive value below 50%, thou mightest as well toss a coin. 
	
	4. Thou shalt know the prevalence of the disease in the population from which the person thou art going to test cameth; for combined with positive predictive value or negative predictive value, thou canst then ensure that thou hast interpreted the result properly. Thou shalt recognise that in a low-prevalence population, even a very sensitive test hath poor predictive value, and if thou shouldest order such a test, thou mightest as well toss a coin. 
	
	5. Thou shalt know exactly what thou mightest do with the result of the test before thou dost order it. "If thou knowest not what thou shouldest do with the result, or if the test be not part of an evidence-based pathway, then shalt thou DESIST from ordering such a test." 
	
	6. Thou shalt know the cost of the test thou intendest to order, before thou dost order it, and also thou shouldest enquire whether there be cheaper ways of finding the same result. 
	
	7. Thou shalt know whosoever else hath in the past ordered, or might in the future order, the same test under the same set of circumstances, and then shalt thou ensure that they know that they do but waste resources, since thou hast performed the same already before. Therefore, thou shalt record the result of the test which thou dost order very clearly that all others might see it. 
	
	8. Thou shalt inform patients that "shinier scanners and swizzier kits" might well give better resolution pictures, but might not actually change a clinical management decision. 
	
	9. Thou shalt simply not ask or order patients to undertake diagnostic tests for which there be no proven evidence of benefit. 
	10. Thou shalt explain to the patient, clearly and without bias, that which the test can and cannot tell thee, and thou shalt ask them if they are content to proceed in such wise. 
	 
	Regards,
	
	Ash
	


		
		
________________________________

		From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Ash Paul
		Sent: 09 January 2012 13:22
		To: [log in to unmask]
		Subject: Re: Ten commandments for testing
		 
		Dear Greg,
		 
		I have biblified yours/Anna's list and circulated it to our NHS Public Health Commissioners e-group. I have proposed calling them the Fell-Sayburn Commandments
		 
		 
		 

		1.	Before thou ordereth a test, thou shalt remembereth that "All tests can doeth harm. Many tests doeth good. Some tests doeth more good than harm" 

		 

		2.	Thou shalt knoweth the predictive positive value and predictive negative value of the test before thou ordereth it 

		 

		3.	Thou shalt knoweth that if thou ordereth a test with a positive predictive value below 50%, thou might as well tosseth a coin 

		 

		4.	Thou shalt knoweth the prevalence of the disease in the population from which the person thou art going to test cometh from; for combined with positive predictive value or negative predictive value, thou can then ensureth that thou hast interpreted the result properly. Thou shalt recogniseth that in a low prevalence population, even a very sensitive test has poor predictive value, and if thou ordereth such a test, thou might as well tosseth a coin. 

		 

		5.	Thou shalt knoweth exactly what thou art going to doeth with the result of the test before thou ordereth it. "If thou doth not know what to doeth with the result, or if the test is not part of an evidenced-based pathway, then thou shalt DESIST from ordering such a test." 

		 

		6.	Thou shalt knoweth the cost of the test thou art going to ordereth, before thou ordereth it, and also enquireth whether there are cheaper ways of finding the same result. 

		 

		7.	Thou shalt knoweth who else might have in the past, or might in the future, ordereth the same test under the same set of circumstances, and then ensureth they know they are wasting resources as thou hast done it already before. Therefore, thou shalt recordeth the result of the test thou ordereth very clearly for all others to see. 

		 

		8.	Thou shalt informeth patients that "shinier scanners and swizzier kits" might well giveth better resolution pictures but they might not actually changeth a clinical management decision 

		 

		9.	Thou shall simply not asketh or ordereth patients to undertake diagnostic tests for which there is well proven evidence of no benefit  

		 
		   10. Thou  shalt explaineth to the patient, clearly and without bias, what the test can and canst tell thou, and thou shalt asketh them if they are happy to goeth  ahead on that basis. 
		 
		 
		Regards,
		
		Ash
		 

			 
			
________________________________

			From: Fell Greg <[log in to unmask]> <mailto:[log in to unmask]> 
			To: [log in to unmask] 
			Sent: Monday, 9 January 2012, 12:19
			Subject: Re: Ten commandments for testing
			accept the need for a "full" version (with refs and all that)....guess it depends on intended audience and endpoint. I feel the slightly humerous biblified version might have more "reach" (on account of the humour factor)....but accept that "reach" and "influence" might be two slightly different things.
			I am not going to die in a ditch about either.
			The important thing is that the list sees the light of day outside this e group - published / blogged / other. I am intending on using it extensively locally.
			Do colleagues feel this is publishable. 
			 
			gf



-- 
Paul Glasziou
Bond University
Qld, Australia 4229

--------------------------------------------------------------------
This e-mail is intended for the use of the addressee only and may contain confidential information, copyright material or views/opinions that do not necessarily reflect those of Bradford and Airedale teaching Primary Care Trust. If you receive this email by mistake please advise the sender immediately. All should be aware that this email may be subject to public disclosure under the Freedom of Information Act 2000 and that emails may be monitored.....