 |
 |
Hi Susan, Thanks for that. I absolutely agree that a balance needs to be taken - an evidence-based one! I would never advocate doing a poor search. But part of the problem lies with not really knowing what is a bad and what is good in a given scenario. So, I'm very keen to explore the cost-benefit of various options. In my journey around systematic reviews the biggest thing that hit me was the opportunity cost. It's fine to aspire to do a SR for everything. But, while you're doing a SR you could be doing other things - say 10-20 rapid reviews. That 1 SR could do a large amount of good. But one could easily make the case that 10-20 rapid reviews could do significantly more good. It's like with interventions in medicine. We often, due to circumstance, have to settle for something other than the gold standard (more so in the UK than the USA I think). We're comfortable doing that as we realise that the cost-benefit doesn't stack up. I just think we need a similar approach to reviews. But, the danger is, as Tom has found - using a standard Cochrane methodology (using them as an example) is often not enough! Thanks again. BW jon On Wed, Mar 20, 2013 at 5:18 PM, Susan Fowler <[log in to unmask] > wrote: > I think the time it takes to do a review depends on many factors and that > there is no one right answer. Not all questions are appropriate for a > systematic review and not all circumstances allow for the time it takes to > do a thorough job of it - that does not mean that therefore taking longer > then x amount of time for any systematic review is wasteful. Systematic > reviews serve a useful purpose and have added to our knowledge base > however, there are plenty of ill produced systematic reviews out there. I > agree that the knee-jerk response of "we must do a SR for everything" is > flawed but lets not jerk our knees in the opposite direction and throw away > perfectly good methods. > > Quick and relatively unskilled searches for research are not appropriate > and have gotten people killed. (ie Hopkins (http://tinyurl.com/dxz8g3dwhere a good lit search would have revealed the drug they were testing had > been discontinued by the fed due to having a toxic effect on lungs). If > quick is all you have time for in a clinical situation then at least the > literature has been consulted which is a big and positive change. However, > critical appraisal skills must be taught and used as well as an awareness > of the difference between a skilled and unskilled search. > > -- > Susan Fowler, MLIS > Medical Librarian > > Evidence at Becker: > http://beckerguides.wustl.edu/ebm > > Systematic Reviews Guide: > http://beckerguides.wustl.edu/SystematicReviews > > Becker Medical Library, Washington University in St. Louis > 314-362-8092 > [log in to unmask] > > On Wed, Mar 20, 2013 at 9:14 AM, Jon Brassey <[log in to unmask] > > wrote: > >> Me again! >> >> With regard to Amy's response. I would question the notion that 'quick >> and clean' (or 'quick and dirty' - there we go with ambiguous language) is >> great for a general overview. I think we're paralysed by the notion that >> getting accurate results take months/years. The paper Tom mentioned >> (Sagliocca) has showed you don't need all the results to get an accurate >> effect size. Sagliocca's paper (alongside a recent paper by Hemens and >> Haynes http://www.ncbi.nlm.nih.gov/pubmed/21856121) are highlighting an >> incorrect assumption of the EBM world that 'systematic reviews' are the >> ultimate source on which to practice healthcare. They are probably the >> most costly (in terms of finance and opportunity costs) and we have no way >> of working out if the cost benefit is worth it. I seriously doubt it. >> >> Being someone who wants quick responses my priority is to see 'how low we >> can go' in terms of timescales - can we get an accurate result in two >> weeks, two days or two hours? >> >> Another related issue is the danger of saying stuff (and I hear it all >> the time and am guilty of this myself) of saying *'if there's a recent >> Cochrane SR then that'll be fine'*. Tom's Tamiflu work has certainly >> made me sit up and be more sceptical. >> >> I do think this needs wider debate, we're too easily swept away with the >> 'knee-jerk' - we must do a systematic review for everything. >> >> BW >> >> jon >> >> >> >> >> >> >> >> On Wed, Mar 20, 2013 at 12:31 PM, Amy Price <[log in to unmask]> wrote: >> >>> Jon and Tom, >>> >>> Thank you for this discussion.I would like to add a qualifier. Quick and >>> dirty is like an oxymoron, I know it is a figure of speech only but lest >>> break that down. Quick and clean is great for a general overview IF the >>> searcher knows hallmarks for sifting through commercial rhetoric and has a >>> general understanding of the research theory and mechanics the authors are >>> using. Perhaps the searches through regulatory materials could be >>> synthesized to add another level to review for example something like the >>> wonderful guidelines we use from CASP and Equator could be created for this >>> purpose by experts in this arena so the patterns of commercial bias in the >>> sea of regulatory paperwork could be sliced through and less expert >>> searchers could cut to the chase? >>> >>> Best, >>> Amy >>> >>> From: Tom Jefferson <[log in to unmask]> >>> Reply-To: Tom Jefferson <[log in to unmask]> >>> Date: Wednesday, March 20, 2013 8:08 AM >>> To: <[log in to unmask]> >>> Subject: Re: Reviews >>> >>> Jon, thank you for your usual carefully thought-through email and your >>> intelligent comments. >>> >>> I am getting ready for next week's presentation at the Evidence Live >>> conference in Oxford. >>> >>> My title is *Bias in the evidence base for neuraminidase inhibitors – >>> are traditional systematic review methods defunct?* >>> >>> The topic of neuraminidase inhibitors is an incidental, a *casus belli*if you want. >>> >>> I think methods of SR are defunct, or should I say *search methods* of >>> SRs are defunct for two reasons: >>> >>> 1. There is mounting empirical evidence that doing a quick and dirty vsa long and clean search of the usual databases >>> and grey literature (as the vast majority of SRs do including Cochrane >>> reviews) does not essentially change the conclusions. >>> >>> For example see the recently published review. Sagliocca et al. A >>> pragmatic strategy for the review of clinical evidence. >>> J.Eval.Clin.Pract. Epub 2013 Jan 15 >>> >>> For the quantitative fiends amongst us, this type of outstanding work >>> shows that ORs are insensitive if you do a quick and dirty vs a long and >>> clean search and consequent review. >>> >>> So for the broader minded: conclusions are unchanged. >>> >>> >>> 2. If the search is carried out only on regulatory material (regulatory >>> comments plus clinical study reports) the conclusions are changed (see >>> Vioxx, Avandia, Tamiflu, Paroxetine, Reboxetine, Zoloft, EPO etc etc) >>> for all the reasons which are well known and for some which are less known >>> or unknown. >>> >>> 1 and 2 probably (not certainly because we neeed confirmatory work to >>> test the relevant null hypotheses) are linked. >>> >>> If the corpus of published clinical trials (especially pharma >>> sponsored) trials is nothing more than marketing material (as in the >>> case of Tamiflu), looking at 2 trials in a quick and dirty or 10 in a long >>> and slow is not going to change the core message which is commercial. >>> >>> We need a debate about all this and its implications and especially we >>> need more work. >>> >>> But that is the message that the evidence is sending me, loud and clear. >>> >>> Best wishes, >>> >>> Tom. >>> >> >> >> >> -- >> Jon Brassey >> Trip Database >> http://www.tripdatabase.com >> Find evidence fast >> >> >> >> >> >> >> -- Jon Brassey Trip Database http://www.tripdatabase.com Find evidence fast