Hello Steve, (un)-fortunately there is NO mathematical solution of the problem you described. It is judgment-driven process (which arguably should be different from "feelings")...transparency and explicitness (that hopefully can lead to reproducibility and conclusions that are CONSISTENT with evidence) is currently being advocated...which all explains why this group has existed for so long (if there were mathematical solutions to finding the "truth", our debate would have been boring and certainly as not passionate as it has been on occasions...)
Hope this answer your question to some extent...
Best
ben
Ben Djulbegovic
Professor of Medicine and Oncology
-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Steve Mathieu
Sent: Tuesday, June 01, 2010 2:49 PM
To: [log in to unmask]
Subject: Re: Grading level of evidence
Thank you kindly for your response. May I ask one more question which illustrates well the problems that Dr Cuello mentions and that I have experienced with systematic reviews (primarily due to my own lack of experience).
I have completed a systematic review for the purpose of a viva as a component of the intensive care diploma. This is my first attempt (so please bare with me!) and as a single author, there is clearly a risk of bias although I have tried to be meticulous and explicit about my inclusion and exclusion criteria. The problem I have faced is that I have provided individual gradings of evidence for each paper analysed and then an overall grading which has been more difficult to determine and would be grateful for suggestions as to how I may have improved this process.
Some more details regarding the dissertation:
- based on levosimedan (drug used to improve cardiac performance particuarly in acute heart failure)
- the question i have posed is : - does this drug improve haemodynamics and/or mortality?
- I have asked this question for 5 separate conditions (acute heart failure, sepsis, cardiogenic shock, post MI, and perioperative cardiac surgery
- I decided to include a wide selection of trials from RCT's to case series. I accept that in retrospect this is perhaps too heterogenous a study mix. However, was very keen to analyse all of the data and there are not that many RCT's which address each of the above (in particular sepsis)
Where I have now struggled is that whilst the SIGN system is quite explicit about grading each paper, I have attempted to provide an overall grade of evidence for each question posed e.g levosimendan improves haemodynamics in cardiogenic shock..level of evidence = 1++. I have not attempted to provide level of recommendation.
The way I apporached this was to give an overall impression based on the evidence I have read. There has not been any mathematical process in this decision but simply an overall feeling from a single author. I did not feel that a meta-analysis was appropriate on the basis of heterogeneity in heart failure definitions, concomitant drug therapy and study designs. Is there a better or more widely accepted method for providing an overall grade which perhas I should have considered or is it a case that when systematic reviews are conducted that a group will sit down and discuss an overall level of recommendation. I suspect I have made the process more difficult by not sticking to either RCT's or obs studies but including the full breadth of studies.
Any advice greatly appreciated. I cannot unfortunatley change how i have conducted this review and as mentioned its main purpose is for a viva where it is accpeted that this will be most peoples first effort! I am really trying to prepare and justify my reasoning beyond 'an overall impression was used...' which seems inadequate! If there are suggestions for a better approach (or reassurance!), all welcomed.
Thanks in advance and apologies for long post
Steve
|