 |
 |
As I recall , the access to the protocols allowed avoiding "not-clear" basket category ... Sent from my iPhone (Please excuse typos & brevity) On Sep 26, 2016, at 09:47, Anoop B <[log in to unmask]<mailto:[log in to unmask]>> wrote: Hi Benjamin, Just from taking a quick look at the paper, I just felt like you could have had one more category as 'unclear' than just two categories:Adequate or Inadequate. I think the problem with just 'inadequate' is that you can lump studies with clearly no blinding with studies that could have had blinding/not clear from the paper. So two ES's (small & large) probably gets blended. Do you think it could have made a difference in ES comparisons? Thank you Anoop On Sun, Sep 25, 2016 at 8:28 AM, Djulbegovic, Benjamin <[log in to unmask]<mailto:[log in to unmask]>> wrote: The problem with critical appraisal is that relies on the quality of reporting, but as we showed in these two papers, the quality of reporting is not necessary equal to the quality of actual conduct: http://www.ncbi.nlm.nih.gov/pubmed/22424985 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC313900/ Fair, or not fair, the underlying premise behind critical appraisal is indeed "if it was not reported, it was not done". The insistence on the quality of reporting is of obvious societal benefit ( i.e. increase trust in clinical research, enables/improves conduct of meta-analyses, etc) but it can certainly unfairly characterized some, otherwise, well-done trials. Ben Ps While reporting clinical research is distinctly different from assessing the practice of medicine, it should be noted that the premise that if "it was not reported, it was not done", has also been widely accepted in the quality of improvement arena, which directly impacts the way how health care is financed. Not surprising, many physicians consider this unfair way to characterize their performance. On Sep 25, 2016, at 3:56 AM, Jon Brassey <[log in to unmask]<mailto:[log in to unmask]>> wrote: Hi Helen, The point I'm making is that journal articles are a summary of the activity undertaken in a trial. So, when we critically appraise a journal article, the starting point appears to be that this is a fair reflection of the trial. As I suggested, and Brian reinforced, this is a problematic assumption. As he said, critical appraisal "can only identify reasons for uncertainty" - indicating that critical appraisal can't assert an article is fair. BW jon On 24 September 2016 at 19:09, Helen Pullen <[log in to unmask]<mailto:[log in to unmask]>> wrote: Does your critical appraisal include looking at the statistics in a paper? It appears that statistical literacy (I'm quoting from the book 'Better doctors, better patients, better decisions') is a required skill of all those appraising the evidence. It's a very interesting topic area. I'm a medical librarian teaching critical appraisal and basic medical stats to our staff (large teaching hospital). Kind regards Helen Pullen BSc MCLIP On Sat, 24 Sep 2016 at 14:08, Brian Alper MD <[log in to unmask]<mailto:[log in to unmask]>> wrote: A while back we published “Evolution of evidence-based medicine to detect evidence mutations.” http://www.ncbi.nlm.nih.gov/pubmed/25572989 or http://jrs.sagepub.com/content/108/1/8.long This reported on a handful of high-profile or high-impact studies where critical appraisal as commonly applied was insufficient to capture the most concerning criticisms. The reasons varied – fraud in what was published, failure to cover what was not published, patterns that technically met “critical appraisal criteria” yet managed to require critical thinking to see the loopholes – but critical appraisal cannot provide absolute certainty – it can only identify reasons for uncertainty. Brian S. Alper, MD, MSPH, FAAFP Founder of DynaMed Vice President of Innovations and EBM Development, EBSCO Health www.dynamed.com<http://www.dynamed.com>, health.ebsco.com<http://health.ebsco.com> Twitter: @BrianAlperMD Be a TRANSFORMER (RECYCLE your STRIVING and THRIVING) … https://www.linkedin.com/pulse/transformer-brian-alper From: Evidence based health (EBH) [mailto:[log in to unmask]<mailto:[log in to unmask]>] On Behalf Of Jon Brassey Sent: Saturday, September 24, 2016 7:09 AM To: [log in to unmask]<mailto:[log in to unmask]> Subject: Limitations of critical appraisal Hi All, I've just been reading about the controversy around a potentially 'dodgy' trial and it got me thinking about the limitations of critical appraisal. This trial was deemed to have been fair and influenced numerous guidelines around the globe. Critical appraisal of articles relies on the notion that the article is a fair reflection of the trial. This appears to be an act of faith. While I'm not saying critical appraisal is not a worthwhile activity my concern is that people see a paper, appraise it and therefore assume that it means the study is fair/good. The reality is that after an appraisal what you can say is that the trial, as reported/represented in the article, appears to be fair/good. Does critical appraisal need a health warning? Best wishes jon Jon Brassey Director, Trip Database<http://www.tripdatabase.com> Honorary Fellow at CEBM<http://www.cebm.net>, University of Oxford Creator, Rapid-Reviews.info<http://Rapid-Reviews.info> -- Jon Brassey Director, Trip Database<http://www.tripdatabase.com> Honorary Fellow at CEBM<http://www.cebm.net>, University of Oxford Creator, Rapid-Reviews.info<http://Rapid-Reviews.info>