This discussion highlights a critical issue, I think, which relates to the nature and scope of realist reviews. I suspect I take a slightly different view on this than some others. (This will come as a great surprise to those who know me - not.) I don't think it's necessary to narrow down to a particular kind of intervention in order to undertake a realist review. I just think it's necessary to have refined and defined the question.
I offer two examples from my own experience as a case in point. My first attempt at realist review was - like yours - undertaken as part of a PhD. It didn't start with a particular type of intervention, it started with an outcome pattern in a category of types of intervention ('early years early intervention programs' - which I suspect is much broader than 'configurations of managed care networks for dental care') and sought to explain the outcome pattern. It ended up including child care programs, home visiting programs, two generation programs, parenting education programs, parent support programs for particular groups of 'at risk' parents. The reason that it could do so was that it dealt with mechanisms and the contexts affecting them - and __mechanisms are not necessarily specific to the particular design of the program__. I emphasised in my PhD that it was not, and did not pretend to be, a systematic review of these types of programs: it was a methodological test, with prescribed boundaries (i.e. could I turn the realist method around so that I started with an outcome pattern rather than an intervention type, and combine that with small scale evaluation to refine program theory).
The second example is a funded realist review in the international domain that a team of us (Patricia Rogers, Bill Walker and I) have just submitted. The question for this review was: "In what circumstances do community accountability and empowerment interventions improve educational outcomes, particularly for the poor, in low and middle income countries." There isn't a 'class' of community accountability and empowerment interventions per se. It turned out that there were four categories of interventions - decentralisation, school based management, community schools, and specific accountability interventions - of relevance to the review. Only some interventions in each category fitted the review, because they had to have specific elements relating to or outcomes in accountability, specific elements relating to or outcomes in empowerment, and they had to operate at the community level. So the method for delimiting the review was not about having a particular type of intervention, but about selecting interventions from within categories because they had elements that related to two particular 'macro mechanisms' (a term I've just this minute coined for broad, not yet properly defined mechanisms at a social scale!!) - accountability and empowerment.
So - you're absolutely right that you'll have to design the review carefully, and to work out it's parameters to keep it manageable. But in my book - 'understanding exactly how a specific kind of intervention works and in what contexts' is only one kind of realist review - not the only one.
Justin - I think this question (the boundaries of realist review) would make for a great conversation at the upcoming realist conference in Liverpool!
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Caoimhin Mac Giolla Phadraig
Sent: Tuesday, 28 January 2014 1:36 AM
To: [log in to unmask]
Subject: Re: Rapid Realist Review - thoughts
Thanks for all the support and direction.
I have a lot to consider but will try and answer comprehensively. I paraphrase a bit, so please excuse if I misrepresent your ideas.
Regarding the format of review:
Gill makes the point that Realist Synthesis can refer to the analytic paradigm used in analysing reviewed data, as much as the specific format of review used, as long as it does not claim to be something that it is not. Justin goes on to say that the RR format best suits the study of specific programme implementation or groups thereof. Therefore he suggests clarifying the exact resource being offered, thereby facilitating the realist approach. Further, Justin acknowledges the limitations and acceptances of RRRs as a valid option.
In response to this, I am now clearer (and I think somewhat more free) about the utilization of this approach to answer the questions I have, yet at the same time, it highlights the need to have clarity of purpose in designing the review. This presents an issue, in my instance, as I operate within a system in constant flux. I, along with the policy makers / service providers do not know what type of intervention will come out of the current process. The lack of clarity in “resources” as Justin puts it, or of a clear intervention (as I see it) complicate the planning of the Realist Review that will ultimately inform the very same intervention.
A Chicken and Egg argument!
We are currently engaged in a formal process called an Oral Health Needs Assessment to try and measure needs among the population and capacity among services. Once completed, this will highlight the priority areas for policy makers and present the basic building blocks ( from the capacity assessment element ) for configuring a new model of dental care delivery.
Looking down the line, with a realist perspective, I ask how do commissioners / planners put these building blocks together ( in relation to the context and likely mechanisms) to come up with an intervention that will trigger the best outcomes… and in a format, Gill makes the point, that aims to give policy makers a product that they will find useful for the specific tasks at hand.
This touches on Justin’s point that participation of the decision makers (end users) is both essential and costly (in time), and could potentially weaken and strengthen the process and output. I agree completely with Simon’s points on engaging policy makers. I spend and have spent the greater part of my research time over the last two years managing the involvement of policy makers and service users with this in mind. A difficulty I have is that policy makers and service managers may look on me and my work as loosely connected with the “REAL” world of service delivery, change management etc. Indeed, what I see as integral planning for reliable outcomes – such as realist review of literature – they may see as “an academic exercise”.
As such, Simon, getting the policy makers, who are genuinely under a lot of pressure, to engage is difficult. For note – I do see the potential for the research process to change the mechanisms that underlie their responses, which may then change the context of their participation and improve the likelihood of deeper engagement. I apply my minor interventions with them in the hope that this mechanism will kick in.
Following the above points, I believe that RRR is valid option in my instance. I will need to pointedly define my criteria for defining the question, despite the circular argument that it relates to. To do this effectively, your comments reaffirm my need to foster contacts, partnerships and “create buy in”.
The joys of research.
Many thanks and I welcome further advice / comments