I so want that T-shirt!! I’m already entitled…
Good points all. Now of course I think: how much of any of this can we build around the RAMESES project?
Cheers
Gill
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Trisha Greenhalgh
Sent: Tuesday, 26 July 2011 4:04 AM
To: [log in to unmask]
Subject: How to persuade funders...
“how to get funders of reviews to understand the implications of the different methods - in part for how to structure calls, but also for findings (cf 'all arriving at the same findings, useful to all policy bods...' etc”
GOOD THREAD! Mark said:
“In the short-term: personal communication, publication of short methodological papers/ reflections on the use of different SR methods and how policy makers were (or not) informed by them, refer to influential critics
In the medium-term: workshops, secondments of researchers into commissioning institutions, working with service users (can they make sense of the reviews??)
In the long-term: integration into undergrad/postgrad degrees (not just in social research methods courses...), development of centres of expertise (that genuinely have in-depth methodological engagement with a wide range of 'SR' methods)
All of the above could be aimed not simply at SR commissioners, but at the whole caboodle of people involved in an area of practice…”
To which I would add:
[1] recognising that a Kuhnian paradigm shift is often a political and ideological one, not merely a matter of science bursting out of its chrysalis, hence playing the power games (getting yourself on the committees that make the rules, etc etc – why am I on the REF panel?)
[2] NOT bottling out and publishing all your stuff in social science journals where it’s well received by your fans. We all have to keep sending stuff to the Lancet even though we know it will get rejected with a letter saying it’s not reached GCSE standard
[3] ditto for finding bids. EVEN THOUGH the MRC is going to turn it down for being non-randomised and not having enough control groups, we should all keep submitting bids to these powerful, field-defining sponsors
[4] inviting people onto this list and the Delphi panel who are ‘waverers’ from the main camp
[5] but when writing this project up, use rationalist-technicist language as far as possible so our RAMESES guidelines are seen as “evidence based”
[6] perhaps get T-shirts made saying “I submitted a bid for a realist review and all I got was this lousy T-shirt”
J
________________________________________
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [[log in to unmask]] On Behalf Of Gill Westhorp [[log in to unmask]]
Sent: 23 July 2011 01:51
Subject: Re: Realist compared to other systematic reviews
Ah yes, I so agree that it's positive. I just think that a positive step doesn't equal unproblematic!
I should mention my vested interest too. I'm an investigator on one of the realist reviews that got funded through this call (assuming 'this call' to be the DFID/AusAID/IIIE call), and an external support person for others. However - there are some dilemmas with trying to squeeze the realist methodology into the way the call has been structured - about which I expect to be able to say much more later, when we're through the process.
The other thoughts that were triggered though related to:
a) how to get funders of reviews to understand the implications of the different methods - in part for how to structure calls, but also for findings (cf 'all arriving at the same findings, useful to all policy bods...' etc)
b) what the implications of staffing turnover in bureaucracies might be for their understanding of the differences...
Cheers
Gill
-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Mark Pearson
Sent: Friday, 22 July 2011 10:49 PM
Subject: Re: Realist compared to other systematic reviews
I must be wearing my upbeat hat today - despite the contradictions and somewhat dubious claims in parts of the text Gill extracted, I think it s very positive that proposals using M-E, NS, RS, etc. were invited in an area (international development) where the use of systematic reviews is relatively novel. Nevertheless, the text Gill extracted is certainly worth discussing from a realist perspective!
I should mention my vested interest I m PI on one of the (conventional systematic) reviews that got funded through this call. Our review will use some of the tools and techniques of narrative synthesis (Popay, Britten et al) to help us get a handle on how context impacts on effectiveness. One point to note when our protocol was reviewed, we were asked why we weren t doing a theory-driven review, to which my rather blunt reply (beyond the fact that this wasn t what funding had been awarded for) was:
1) The resources aren t on the table
2) The review team don t have the skills (or the time to develop them within the commissioned project)
Of course, neither of these are insurmountable but these rather substantial humps can strangle good intentions (to put other review methods such as RS into practice) at birth
Mark
Mark Pearson PhD
Research Fellow
Peninsula Technology Assessment Group (PenTAG) Peninsula College of Medicine & Dentistry (University of Exeter)
T: 0044 (0) 1392 726079
http://sites.pcmd.ac.uk/pentag/staff.php?selstaff=mpearson
-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Gill Westhorp
Sent: 18 July 2011 04:35
Subject: Realist compared to other systematic reviews
And on another topic entirely: here's a set of little extracts from DFID's overview of systematic reviews (http://www.dfid.gov.uk/r4d/PDF/Publications/OverviewofSystematicReviews.pdf). I almost wish I had a post-grad research class so I could set the following with an essay topic - 'Discuss from a realist synthesis perspective'...
"There are many review methods, including meta-ethnography, narrative synthesis, realist synthesis and qualitative meta-summary for quantitative data as well as the meta-analysis approach to statistical data. ... The key element of a systematic review is the process, rather than the specific method used to aggregate and interpret data. ... A systematic review is also more rigorous than a literature review as anyone could follow the review protocol and arrive at similar conclusions. ... ... Systematic reviews produce authoritative assessments of the evidence base that should be relevant to all decision makers."
Comments from a realist perspective welcomed!
Gill