 |
 |
Dear colleagues, In April I asked for your thoughts about "cost of searching for evidence" (see below). As promised, I am sharing the answers I obtained. I thank to those colleagues who shared their important insights (I have not included their names as I forgot to ask for their permission to do so- they can do it in a reply to this e-mail, if so desired). I should mention here that one answer to my question -a rule of thumb proposed by one well known decision scientist is- 1%: apparently it is not worth spending more than 1% of effort (cost) in searching for additional information relative to the total stake of a decision. As you can see it, the answers below considerably deviate from this recommendation. Hope you will find this contribution of our members as invaluable as I have. Best Ben -- "30-40% for bigger issues (fewer good choices for treatment, higher stakes, greater controversy), 5% for problems where the treatment is well-accepted and outcomes are more predictable. Better to spend 2 wks in the library than 6 months in the lab reproducing a study uneccessarily." "...we typically return an 'answer' within 4 hours. In my work in public health we've been doing something similar with public health consultants, but taking 2 days (so around 15 hours) on a Q&A. The issue for me is what are the problems with this? If we assume a systematic review (and meta-analysis) is the gold-standard, as far as I can tell there's little information as to the effect of various short-cuts. You can infer that if you don't do critical appraisal you may experience problems, if you only search English language databases you'll get different problems. But the totality of all these short-cuts - we're clueless and it's dependent on the circumstance. If you did a search of Medline and found a recent Cochrane SR you may well be happy to stop there. If you find 10 RCTs all saying the same thing, again you may stop there. " "It depends on how much work to do the selection and critical appraisal for new technology. For this kind of situation i will pay 90. For the more common others, i will pay 30 " ..."i spent nearly 10 years in a team developing evidence-based guidance for primary care - an information specialist would spend 1 to 3 weeks searching, the lead author would spend 4 to 8 weeks (12 weeks for a large or controversial topic) with the reviewing and writing, and a pharmacist would spend about 2 to 4 weeks writing the prescribing issues section." "Clinical end-users could rely on them and win valuable time by reducing the number needed to read figure from 12.5 to 2.5. However, the price that must be paid is that almost one out of every two relevant articles will be missed (sensitivity of 55%)." "As a searcher (I've worked on over 100 SRs), I can't tell you how frustrating it is when I encounter poorly written abstracts and titles. I'm working on a search right now dealing with breast cancer survivors. I can't be sure that the term "survivor" will always appear so my search strategy is nearly 50 lines long right now as I try to determine all of the terms I need to use to describe someone who had breast cancer and lived. I've already spent over 2 hours working on the search strategy and I'm anticipating that I will put another hour or two into refining it. I must be both comprehensive and directed. I don't want my researcher to sift through 8,000 citations-- that's a waste of time but I still have to make certain I'm inclusive." "...searching set at 75.00 an hour. Evaluation: http://nnlm.gov/mcr/evaluation/calculator.html ROI: http://nnlm.gov/mcr/evaluation/roi.html" From: Djulbegovic, Benjamin Sent: Tuesday, April 10, 2012 9:12 AM To: [log in to unmask] Subject: cost of searching for evidence.... Dear all, At this forum, we have repeatedly discussed the importance of evidence for decision-making. We have also discussed the difficulties related to information search and effort involved in it. However, we have not discussed the cost (in terms of time and effort) of gathering additional information to aid decision-making. If we express the stakes of decision on some arbitrary scale of 0 to 100, how much gathering more information would be valuable to you (as a fraction of the total value of your decision). For example, if you say 10, then you would be willing to "pay" 10% in terms of time and effort to improve your decision. (Note that I am not interested in the value of perfect information, but in the value/cost of SEARCHING for evidence.) If you send your reply to me, I will collate all responses and share with the group. Thanks ben Benjamin Djulbegovic, MD, PhD Distinguished Professor University of South Florida & H. Lee Moffitt Cancer Center & Research Institute Associate Dean for Clinical Research, USF Director of USF Clinical Translation Science Institute, USF Mailing Address: USF Health Clinical Research 12901 Bruce B. Downs Boulevard, MDC02 Tampa, FL 33612 Phone # 813-396-9178 Fax # 813-974-5411 e-mail: [log in to unmask]<mailto:[log in to unmask]> ______________________ Campus Address: MDC02 Office Address : 13101 Bruce B. Downs Boulevard, CMS3057 Tampa, FL 33612