 |
 |
I think the time it takes to do a review depends on many factors and that there is no one right answer. Not all questions are appropriate for a systematic review and not all circumstances allow for the time it takes to do a thorough job of it - that does not mean that therefore taking longer then x amount of time for any systematic review is wasteful. Systematic reviews serve a useful purpose and have added to our knowledge base however, there are plenty of ill produced systematic reviews out there. I agree that the knee-jerk response of "we must do a SR for everything" is flawed but lets not jerk our knees in the opposite direction and throw away perfectly good methods. Quick and relatively unskilled searches for research are not appropriate and have gotten people killed. (ie Hopkins (http://tinyurl.com/dxz8g3dwhere a good lit search would have revealed the drug they were testing had been discontinued by the fed due to having a toxic effect on lungs). If quick is all you have time for in a clinical situation then at least the literature has been consulted which is a big and positive change. However, critical appraisal skills must be taught and used as well as an awareness of the difference between a skilled and unskilled search. -- Susan Fowler, MLIS Medical Librarian Evidence at Becker: http://beckerguides.wustl.edu/ebm Systematic Reviews Guide: http://beckerguides.wustl.edu/SystematicReviews Becker Medical Library, Washington University in St. Louis 314-362-8092 [log in to unmask] On Wed, Mar 20, 2013 at 9:14 AM, Jon Brassey <[log in to unmask]>wrote: > Me again! > > With regard to Amy's response. I would question the notion that 'quick > and clean' (or 'quick and dirty' - there we go with ambiguous language) is > great for a general overview. I think we're paralysed by the notion that > getting accurate results take months/years. The paper Tom mentioned > (Sagliocca) has showed you don't need all the results to get an accurate > effect size. Sagliocca's paper (alongside a recent paper by Hemens and > Haynes http://www.ncbi.nlm.nih.gov/pubmed/21856121) are highlighting an > incorrect assumption of the EBM world that 'systematic reviews' are the > ultimate source on which to practice healthcare. They are probably the > most costly (in terms of finance and opportunity costs) and we have no way > of working out if the cost benefit is worth it. I seriously doubt it. > > Being someone who wants quick responses my priority is to see 'how low we > can go' in terms of timescales - can we get an accurate result in two > weeks, two days or two hours? > > Another related issue is the danger of saying stuff (and I hear it all the > time and am guilty of this myself) of saying *'if there's a recent > Cochrane SR then that'll be fine'*. Tom's Tamiflu work has certainly > made me sit up and be more sceptical. > > I do think this needs wider debate, we're too easily swept away with the > 'knee-jerk' - we must do a systematic review for everything. > > BW > > jon > > > > > > > > On Wed, Mar 20, 2013 at 12:31 PM, Amy Price <[log in to unmask]> wrote: > >> Jon and Tom, >> >> Thank you for this discussion.I would like to add a qualifier. Quick and >> dirty is like an oxymoron, I know it is a figure of speech only but lest >> break that down. Quick and clean is great for a general overview IF the >> searcher knows hallmarks for sifting through commercial rhetoric and has a >> general understanding of the research theory and mechanics the authors are >> using. Perhaps the searches through regulatory materials could be >> synthesized to add another level to review for example something like the >> wonderful guidelines we use from CASP and Equator could be created for this >> purpose by experts in this arena so the patterns of commercial bias in the >> sea of regulatory paperwork could be sliced through and less expert >> searchers could cut to the chase? >> >> Best, >> Amy >> >> From: Tom Jefferson <[log in to unmask]> >> Reply-To: Tom Jefferson <[log in to unmask]> >> Date: Wednesday, March 20, 2013 8:08 AM >> To: <[log in to unmask]> >> Subject: Re: Reviews >> >> Jon, thank you for your usual carefully thought-through email and your >> intelligent comments. >> >> I am getting ready for next week's presentation at the Evidence Live >> conference in Oxford. >> >> My title is *Bias in the evidence base for neuraminidase inhibitors – >> are traditional systematic review methods defunct?* >> >> The topic of neuraminidase inhibitors is an incidental, a *casus belli*if you want. >> >> I think methods of SR are defunct, or should I say *search methods* of >> SRs are defunct for two reasons: >> >> 1. There is mounting empirical evidence that doing a quick and dirty vsa long and clean search of the usual databases >> and grey literature (as the vast majority of SRs do including Cochrane >> reviews) does not essentially change the conclusions. >> >> For example see the recently published review. Sagliocca et al. A >> pragmatic strategy for the review of clinical evidence. J.Eval.Clin.Pract. >> Epub 2013 Jan 15 >> >> For the quantitative fiends amongst us, this type of outstanding work >> shows that ORs are insensitive if you do a quick and dirty vs a long and >> clean search and consequent review. >> >> So for the broader minded: conclusions are unchanged. >> >> >> 2. If the search is carried out only on regulatory material (regulatory >> comments plus clinical study reports) the conclusions are changed (see >> Vioxx, Avandia, Tamiflu, Paroxetine, Reboxetine, Zoloft, EPO etc etc) >> for all the reasons which are well known and for some which are less known >> or unknown. >> >> 1 and 2 probably (not certainly because we neeed confirmatory work to >> test the relevant null hypotheses) are linked. >> >> If the corpus of published clinical trials (especially pharma sponsored) >> trials is nothing more than marketing material (as in the case of >> Tamiflu), looking at 2 trials in a quick and dirty or 10 in a long and slow >> is not going to change the core message which is commercial. >> >> We need a debate about all this and its implications and especially we >> need more work. >> >> But that is the message that the evidence is sending me, loud and clear. >> >> Best wishes, >> >> Tom. >> > > > > -- > Jon Brassey > Trip Database > http://www.tripdatabase.com > Find evidence fast > > > > > > >