One additional quick comment, the sentence which reads:
"An expert panel achieved only fair interrater agreement on the strength of recommendations based on a body of evidence (5)” is misinterpretation of the findings reported in our paper; in fact, we say opposite,
even people with minimal training ( these were not "experts") and exposed to GRADE for the first time, achieve rather good agreement on key GRADE domains. This should have been obvious even from the title “GRADE guidelines system is reproducible when instructions
are clearly operationalized even among the guidelines panel members with limited experience with GRADE.”
https://www.ncbi.nlm.nih.gov/pubmed/26845745.
The key message of the paper is that when instructions are clear, people do tend to agree with each other.
Ben djulbegovic
From: Evidence based health (EBH) [mailto:[log in to unmask]]
On Behalf Of Philipp Dahm
Sent: Wednesday, December 07, 2016 8:54 AM
To: [log in to unmask]
Subject: Re: GRADE: time to change
I would encourage interested individuals to also review the 2 commentaries that have been posted in reference to this article, one of which I helped write.
http://annals.org/aim/article/2552985/grade-methods-guideline-development-time-evolve
David – I don’t understand why the issue you raise isn’t EXACTLY the type of issue a systematic approach to rating the quality of evidence such as GRADE provides allows you
to address.
Ph*
Philipp Dahm, MD, MHSc
Professor of Urology, University of Minnesota
Director for Surgery/Specialty Care Service Line Research Activities
Coordinating Editor, Cochrane Urology Group
Minneapolis VA Health Care System, Urology Section 112D
One Veterans Drive
Minneapolis, MN 55417
Phone: 612 467 3532
Fax: 612 467 2232
Email: pdahm@umn.edu
Twitter: EBMUrology
From:
"Evidence based health (EBH)" <[log in to unmask]> on behalf of David Richard Leslie Cawthorpe <[log in to unmask]>
Reply-To: David Richard Leslie Cawthorpe <[log in to unmask]>
Date: Wednesday, December 7, 2016 at 6:48 AM
To: <[log in to unmask]>
Subject: Re: GRADE: time to change
I concur.
In an online evidence-based medicine evaluate course (contact me personally if you want to link to it), one of the feature studies illustrated how clinical outcomes are influenced by selection bias in randomized controlled trial's, and
in the case of the featured paper-published! While the authors had the temerity to publish the selection bias in the comparison of groups, they failed to comment on the fact that the selection bias completely confounded the results.
When these types of results make it into meta-analytic review designed to influence policy, even though (in the case study) the selection bias contributed to a significant finding, it becomes no longer possible to trust meta-analytic findings at face value without
precise examination of each sample.
This type of examination is necessarily qualitative even though the consequences are quantitative.
Hence, approaches such as GRADE will remain limited, and may more reflect the human condition than anything else, given the progress (or lack of it) in evidence-based medicine. Evidence remains something that will require context and interpretation.
David
Sent from my iPhone
On Dec 7, 2016, at 5:08 AM, Juan Gérvas <[log in to unmask]> wrote:
GRADE Methods for Guideline Development: Time to Evolve
Although GRADE methods are evolving (7), they are not currently applicable to many questions that guideline developers face, including those about assessing risk and causality, establishing risk thresholds, or assessing animal studies. Further, GRADE does not provide explicit guidance for complex interventions or when the evidence is linked across a causal pathway, and conceptual frameworks are generally absent. There is only limited GRADE guidance on how to assess the quality of a body of evidence addressing resource use.
The GRADE approach is also challenging to apply to different types of data because it was developed for quantitative data with a pooled estimate and CI for each outcome. When data are qualitative or the outcomes cannot be pooled due to heterogeneity, GRADE must be adapted, although the same framework and elements can still be applied. No GRADE guidance is available on how to assess the quality of data from mathematical models or how to incorporate the results of modeling into the development of recommendations.
http://annals.org/aim/article/2552985/grade-methods-guideline-development-time-evolve-un saludo juan gérvas