Print

Print
In response to the concern for "how to get funders of reviews to understand the implications of the different methods" from a Canadian perspective.

At the moment we are having good luck with our Canadian Institutes for Health Research funding a variety of systematic review approaches. The Knowledge Translation branch of CIHR has opened some nice 100CAD (about 50,000 pound) grants for knowledge syntheses (our metanarrative is funded under one of these) and has funded postdoctoral fellowships that involve knowledge syntheses (my own was for a comparative exercise looking at different review methodologies, "camps", approaches including realist review). I spent a bunch of time in the UK 4 years ago visiting lots of different folk and getting a bit of a lay of the land in this regard.

As an aside, there is a ton of very interesting (at least to me) material to think and write about in relation to the lineages and evolution of the methodologies, how that relates to funders, publishers, researchers, topic areas...(audience?) I think my main comment at this stage is that here across the pond, funders don't seem to be very wary of the "newer" styles. I think they are probably not overly educated about them in general and probably very minimally aware of their differences BUT this is translating (thankfully) into some funding being thrown our way because the methods do make a lot of intuitive sense I think. Thankfully someone at CIHR sees the benefit of KT and KS in general and philosophically, they seems pretty open minded, probably somewhat less entrenched than funders elsewhere? Perhaps it is less about how to get funders to understand the different methods than it is to be continue to articulate well the benefits and then show many examples of the work in action. Perhaps some good cross-ocean partnerships might also be beneficial?

Not sure, but from the sounds of your posts it definitely feels like a different kettle of fish over here.

Colleen

On 7/25/2011 3:33 PM, Trisha Greenhalgh wrote:

“how to get funders of reviews to understand the implications of the different methods - in part for how to structure calls, but also for findings (cf 'all arriving at the same findings, useful to all policy bods...' etc”

 

GOOD THREAD!  Mark said:

 

“In the short-term: personal communication, publication of short methodological papers/ reflections on the use of different SR methods and how policy makers were (or not) informed by them, refer to influential critics

 

In the medium-term: workshops, secondments of researchers into commissioning institutions, working with service users (can they make sense of the reviews??) 

 

In the long-term: integration into undergrad/postgrad degrees (not just in social research methods courses...), development of centres of expertise (that genuinely have in-depth methodological engagement with a wide range of 'SR' methods)

 

All of the above could be aimed not simply at SR commissioners, but at the whole caboodle of people involved in an area of practice…”

 

To which I would add:

 

[1] recognising that a Kuhnian paradigm shift is often a political and ideological one, not merely a matter of science bursting out of its chrysalis, hence playing the power games (getting yourself on the committees that make the rules, etc etc – why am I on the REF panel?)

 

[2] NOT bottling out and publishing all your stuff in social science journals where it’s well received by your fans.  We all have to keep sending stuff to the Lancet even though we know it will get rejected with a letter saying it’s not reached GCSE standard

 

[3] ditto for finding bids. EVEN THOUGH the MRC is going to turn it down for being non-randomised and not having enough control groups, we should all keep submitting bids to these powerful, field-defining sponsors

 

[4] inviting people onto this list and the Delphi panel who are ‘waverers’ from the main camp

 

[5] but when writing this project up, use rationalist-technicist language as far as possible so our RAMESES guidelines are seen as “evidence based”

 

[6] perhaps get T-shirts made saying “I submitted a bid for a realist review and all I got was this lousy T-shirt”

 

J

 

 

 

 

 

________________________________________

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [[log in to unmask]] On Behalf Of Gill Westhorp [[log in to unmask]]

Sent: 23 July 2011 01:51

To: [log in to unmask]

Subject: Re: Realist compared to other systematic reviews

 

Ah yes, I so agree that it's positive. I just think that a positive step doesn't equal unproblematic!

 

I should mention my vested interest too.  I'm an investigator on one of the realist reviews that got funded through this call (assuming 'this call' to be the DFID/AusAID/IIIE call), and an external support person for others.  However - there are some dilemmas with trying to squeeze the realist methodology into the way the call has been structured - about which I expect to be able to say much more later, when we're through the process.

 

The other thoughts that were triggered though related to:

a) how to get funders of reviews to understand the implications of the different methods - in part for how to structure calls, but also for findings (cf 'all arriving at the same findings, useful to all policy bods...' etc)

b) what the implications of staffing turnover in bureaucracies might be for their understanding of the differences...

 

Cheers

Gill

 

-----Original Message-----

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Mark Pearson

Sent: Friday, 22 July 2011 10:49 PM

To: [log in to unmask]

Subject: Re: Realist compared to other systematic reviews

 

I must be wearing my  upbeat  hat today - despite the contradictions and somewhat dubious claims in parts of the text Gill extracted, I think it s very positive that proposals using M-E, NS, RS, etc. were invited in an area (international development) where the use of  systematic  reviews  is relatively novel. Nevertheless, the text Gill extracted is certainly worth discussing from a realist perspective!

 

I should mention my vested interest   I m PI on one of the (conventional systematic) reviews that got funded through this call. Our review will use some of the tools and techniques of narrative synthesis (Popay, Britten et al) to help us get a handle on how context impacts on effectiveness. One point to note   when our protocol was reviewed, we were asked why we weren t doing a theory-driven review, to which my rather blunt reply (beyond the fact that this wasn t what funding had been awarded for) was:

 

1) The resources aren t on the table

2) The review team don t have the skills (or the time to develop them within the commissioned project)

 

Of course, neither of these are insurmountable   but these rather substantial humps can strangle good intentions (to put other review methods such as RS into practice) at birth

 

Mark

 

Mark Pearson PhD

Research Fellow

Peninsula Technology Assessment Group (PenTAG) Peninsula College of Medicine & Dentistry (University of Exeter)

E: [log in to unmask]

T: 0044 (0) 1392 726079

http://sites.pcmd.ac.uk/pentag/staff.php?selstaff=mpearson

 

 

-----Original Message-----

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Gill Westhorp

Sent: 18 July 2011 04:35

To: [log in to unmask]

Subject: Realist compared to other systematic reviews

 

And on another topic entirely:  here's a set of little extracts from DFID's overview of systematic reviews (http://www.dfid.gov.uk/r4d/PDF/Publications/OverviewofSystematicReviews.pdf).  I almost wish I had a post-grad research class so I could set the following with an essay topic - 'Discuss from a realist synthesis perspective'...

 

"There are many review methods, including meta-ethnography, narrative synthesis, realist synthesis and qualitative meta-summary for quantitative data as well as the meta-analysis approach to statistical data. ... The key element of a systematic review is the process, rather than the specific method used to aggregate and interpret data. ...  A systematic review is also more rigorous than a literature review as anyone could follow the review protocol and arrive at similar conclusions. ...  ... Systematic reviews produce authoritative assessments of the evidence base that should be relevant to all decision makers."

 

Comments from a realist perspective welcomed!

Gill