Print

Print


I too attempted to combine QCA and realist methods but it became overly complex.

QCA is in some ways quite rigid in determining pathways for solutions – much more quantitative in its approach to calculating pathways. Ultimately QCA proved useful but the task of describing the two different methodologies and finding reviewers willing to review papers using both has

Jo Spangaro
School of Social Sciences




From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Hugh Waddington
Sent: Thursday, 12 April 2018 8:52 PM
To: [log in to unmask]
Subject: Re: Justifying the use of realist approaches

Gill - thanks v much for the advice on QCA. Also interested in the response to Crispin's message. (Crispin thanks will read the following with great interest -  Sager, F., & Andereggen, C. (2012). Dealing with complex causality in realist synthesis: the promise of Qualitative Comparative Analysis. American Journal of Evaluation, 33(1), 60-78.)
A good meta-analysis should focus on explaining heterogeneity in findings not just calculating the average effect. We hope to do this by adding mechanisms to the usual context-outcome synthesis meta-analysisusing CMO synthesis, but let's see if it can work...
Best wishes
Hugh

On Tue, Apr 10, 2018 at 11:09 PM, Gill Westhorp <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Hi Hugh
I have worked on a project which tried to combine CMO and QCA (but not statistical meta-analysis). It was a struggle, not least because QCA does not identify relationships between context and mechanism, as realist analysis does, but between context and outcome.  I could not see how QCA added substantially to the findings for the realist analysis.

I suspect the obvious struggle for adding meta-analysis is that it works by calculating average effects, while realism works by identifying and explaining the differences in effects. The examples I have seen for meta-analysis examining sub-group differences have been on demographic bases, but demographic differences rarely account for differences in realist analysis (and where they do, they are only markers for whatever underlying differences need to be explained).

Cheers
Gill

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards <[log in to unmask]<mailto:[log in to unmask]>> On Behalf Of Hugh Waddington
Sent: Tuesday, 10 April 2018 11:11 PM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: Justifying the use of realist approaches

Thanks Geoff. We will do. Nice and timely as we're working on a SR at 3ie on governance interventions in L&MICs where we want to combine statistical meta analysis with CMO synthesis possibly using QCA. Do you or others know of synthesis studies using this or similar (post positivist) approaches?
Best wishes
Hugh

On Tue, 10 Apr 2018, 08:15 Geoff Wong, <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Dear All,

A common issue that comes up when you plan to or have used a realist approach (i.e. realist review or realist evaluation) is the need to justify your choice of approach(es). This may happen, for example, in your thesis write-up or when making the case in a grant/funding application. Or you may need to convince colleagues that to address a research question, realist review and/or realist evaluation might provide more useful knowledge.

I am sure you all have your 'favourite' resources that you cite and/or refer others to, to make the case. For example, any of the seminal texts from Pawson and Tilley. But sometimes resources that make the case with in as short a space as possible can be helpful - especially if you are going to ask someone to read it :-)

A good starting point for open access resource will be the RAMESES Project website (www.ramesesproject.org<http://www.ramesesproject.org>).

Apologies in advance for the shameless plugging, but below are two sources you might want to share with others whom you have to convince about your choice of using realist review and/or realist evaluation (both are open access):

Making theory from knowledge syntheses useful for public health. Wong G. Int J Public Health 2018. https://doi.org/10.1007/s00038-018-1098-2
https://link.springer.com/article/10.1007%2Fs00038-018-1098-2

Getting to grips with context and complexity − the case for realist approaches. Wong G. Gaceta Sanitaria 2017 doi.org/10.1016/j.gaceta.2017.05.010<http://doi.org/10.1016/j.gaceta.2017.05.010>
https://www.sciencedirect.com/science/article/pii/S0213911117301474

Do please add to this thread the resources you have used to help others understand why using a realist review and/or realist evaluation approach was the rational choice :-D

Geoff


Delhi Evidence Week 2018, 19-20 April<http://www.3ieimpact.org/en/events/3ie-conferences-and-workshops/3ie-delhi-evidence-week-2018/>: Pushing the frontiers of evaluation effectiveness and excellence: 3ie’s first decade

New evidence synthesis report<http://www.3ieimpact.org/en/publications/working-papers/3ie-working-paper-30/>: Does community-driven development build social cohesion or infrastructure?



--
Hugh Waddington
Senior Evaluation Specialist
Managing Editor, Journal of Development Effectiveness
Co-Chair, Campbell Collaboration International Development Coordinating Group
London International Development Centre, UK


Delhi Evidence Week 2018, 19-20 April<http://www.3ieimpact.org/en/events/3ie-conferences-and-workshops/3ie-delhi-evidence-week-2018/>: Pushing the frontiers of evaluation effectiveness and excellence: 3ie’s first decade

3ie-IFAD event on 23 April in Rome<http://www.3ieimpact.org/en/events/external-events/verifying-evidence-policymaking-assessing-financial-services-poo/>: Financial services programmes for the poor: verifying evidence for policymaking