Dear Ben,
Accuracy and completeness of search results is key in evidence informed clinical decision making.
Your question is about the willingness to pay for search results that prevent you from being wrongly informed due to inaccurate or incomplete search results, or both.
The consequences of a wrong decision in patient care are loss of health and well being plus the costs this may generate. But I am uncertain how the time and effort for obtaining accuracy and completeness of search results (or the lack thereof) may equate with costs. Moreover, I am uncertain what any value would mean, and whether it would be uniform across different situations and circumstances. This for the 2 following reasons:
First, my willingness to pay for time and effort to improve on the accuracy and completeness of the evidence that informs my decision will capitalise on the a-priori availability of information (including the prior knowledge of me and my colleagues). That is: what is the balance between certainty and consequences?
Second, if the consequences of a decision (in terms of morbidity or mortality risk) are serious or very serious this will increase my willingness to pay for time and effort to improve on the accuracy and completeness of the evidence that informs my decision.
Searching for evidence to inform clinical decision making may appear costly and time consuming. To preserve costly time and efforts on searching evidence, pubmed high specificity clinical query filters have been advocated as a pragmatic approach. They however clearly provide unwarranted comfort in the accuracy and completeness of the search results. This of course is detrimental in evidence informed clinical decision making.
You may be aware that, in view of the high false-positive rate of the Pubmed high-specificity clinical query filters, Lucas Bachmann and his colleagues have coined the term NNR (or number needed to read =1/precision) to describe the number of irrelevant references (titles and/or abstracts from any search result) that one has to screen to find one of relevance. This of course in analogy to the number needed to treat (NNT)
In their discussion Bachmann et al. warn clinicians about the possible consequences of using pubmed high specificity clinical query filters: "Clinical end-users could rely on them and win valuable time by reducing the number needed to read figure from 12.5 to 2.5. However, the price that must be paid is that almost one out of every two relevant articles will be missed (sensitivity of 55%). ................ To our knowledge, no data currently show that the articles that one finds are a random selection of the available ones. This implies that a biased picture based on only half of the evidence cannot be excluded. For this reason, we do not recommend clinicians to rely on the high-specificity filter in PubMed."
You may find this interesting work of Lucas Bachmann and his colleagues as Free PMC Article [PMCID:PMC349381; Bachmann LM, Coray R, Estermann P, Ter Riet G. Identifying diagnostic studies in MEDLINE: reducing the number needed to read. J Am Med Inform Assoc. 2002 Nov-Dec;9(6):653-8. PMID:12386115].
Geert JMG van der Heijden, PhD
Associate Professor of Clinical Epidemiology
Department of Clinical Epidemiology - Division Julius Center for Health Sciences and Primary Care
Department of Otorhinolaryngology - Division Surgical Specialisms
University Medical Center Utrecht
The Netherlands
|