My interest in realist evaluation is tangential to what has been my main research area - using Mathematical modelling and computer simulation to evaluate health programmes.

I’m seeing a lot of overlap here - one of the perennial questions in modelling is whether a system needs to be represented by an “agent based model”, where each individual is represented discretely in the model, by a population level model, or by something in between.

Ultimately, my choice of approach is pragmatic — I base it on where the most important drivers of the phenomenon being modelled are occurring. Are they top level population drivers (e.g. policy such as closing night clubs earlier to reduce violence) or are they individual level drivers (e.g. why do people have unprotected sex when they see campaigns educating them about the risks?) 

In this perhaps I am approaching Justin’s pragmatic view.

With the chat about journal collaborations, I thought it might be worth posting the above. I’ve often thought realist evaluations are underused as sources of data by modellers - to me there are obvious similarities between CMO evaluations and our predictive modelling. Would be very interested on collaborating on an introduction to realist evaluation pitched at a modelling audience if anyone else would be.

Cheers

Edward Waters PhD

PLEASE NOTE: When you click 'Reply' to any message it will be sent to all RAMESES List members. If you only want to reply to the sender please remove [log in to unmask] from the 'To:' section of your email.

Gee, Ray, you think those buckled into the back seat don’t have a power base???

 

You’ve never heard of the power of the scream? Or as they get verbal, the nag??

 

😊

 

Cheers

Gill

 

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards <[log in to unmask]> On Behalf Of Raymond Pawson
Sent: Monday, 25 March 2019 10:58 PM
To: [log in to unmask]
Subject: Re: ...in defence of so-called "methodological individualism"

 

PLEASE NOTE: When you click 'Reply' to any message it will be sent to all RAMESES List members. If you only want to reply to the sender please remove [log in to unmask] from the 'To:' section of your email.

Hi All

The RAMESES set-to on whether realist evaluation is inclined to methodological individualism was a little treasure. I’ll not deliver an opinion here (guess which?) but rather make another suggestion inspired by Justin’s comment about the ‘disputatious community of truth seekers’. Discussion lists are dreadfully transient. As we old farts say – ‘nothing significant has been written by thumbs’. The debate is worth some journal space and it would be a good idea to reproduce and extend the fisticuffs in some journal. Why don’t a couple of protagonists get together and think through a format?

The debate was fired by a rather different question about the need to identify actors/agents in realist if-then proposition. Here’s my tuppence worth on that. Answer – most definitely yes. Example – in the RS that Geoff Wong and I conducted on the potential effectiveness of legislation banning smoking in cars carrying children we took into account the reasoning and resources of a whole bunch of stakeholders. Quite typically, any policy change draws in a bunch of collective, conflicted actors. So in this case we have: i) state actors – the legislature, law enforcement, ii) quangos such as national and local health authorities, iii) pressure groups both pro and anti-smoking, iv) economic interests – the tobacco industry, v) science represented by toxicologists and evaluators, vi) smokers and non-smokers, vii) parents and children. All (with exception of those buckled into the back-seat) have a say but act from a different power base. The research, at its most basic, is an assessment of the ensuing arm-wrestle. In short - agents galore!

There are several publications on this – see a summary in The Science of Evaluation, Ch 8.

BW

RAY

 

---

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards <[log in to unmask]> on behalf of Jemma Basham <[log in to unmask]>
Sent: 21 March 2019 16:42:05
To: [log in to unmask]
Subject: Re: ...in defence of so-called "methodological individualism"

 

PLEASE NOTE: When you click 'Reply' to any message it will be sent to all RAMESES List members. If you only want to reply to the sender please remove [log in to unmask] from the 'To:' section of your email. I’m interested too and would like to keep hearing your views😀

Kind regards

Dr J Basham

 

 

Sent from my iPhone

On 20 Mar 2019, at 17:28, Peter A <[log in to unmask]> wrote:

PLEASE NOTE: When you click 'Reply' to any message it will be sent to all RAMESES List members. If you only want to reply to the sender please remove [log in to unmask] from the 'To:' section of your email.

Three other points.

 

1) Take the event: "The army shot live ammunition at the protesters". Knowing what goes on in the hearts and minds of either the soldiers or protesters will tell us little about the event. Far more important as preceding causes and mechanisms in this event is what is going on at a higher social level (e.g. between institutions).

 

2) The original question was whether or not an agent needs to be identified in every CMO configuration. Gill Westhorp's response makes me think that the answer might be yes but that the agent wouldn't necessarily be an individual. I've not read her chapter yet, though - it's on its way to me.

 

3) I'm a bit worried that this debate could end up being too esoteric - should it move to a sub-section of RAMESES rather than clogging the email of busy researchers and students? I'm interested but as many would point out, I'm not normal!

 

All the best

Peter

 

On Wed, Mar 20, 2019 at 2:39 PM Sarah Rybczynska-Bunt <[log in to unmask]> wrote:

PLEASE NOTE: When you click 'Reply' to any message it will be sent to all RAMESES List members. If you only want to reply to the sender please remove [log in to unmask] from the 'To:' section of your email.

 

Justin makes a very robust argument, however, the problem as I can see is it, is that actual events aren’t always caused by mechanisms in hearts and minds, and if they were there would need to be a much more in-depth analysis about hearts and minds – unconsciousness, biological drives, social conditioning etc [which also relies on theorising on the complex interplay of agency/structure]. I realise this is something that independent realistic evaluators perhaps bring in to their analysis, but could the level of depth of how emergence works be more developed in realistic evaluation theory itself? Dave Elder-Vass contends that emergence requires more than just looking at different component parts. Rather researchers need to consider the relationship between component parts which activate a mechanism. Realistic evaluation places importance on these component parts with its emphasis on context, but could the relational aspects of different entities and structures be further developed?

 

Realistic evaluation heralds the work of Margaret Archer who addresses more of these complex issues but there isn’t much critical appraisal of her work from realist evaluators, and what about her critics – (i.e. an overinflated concept of reflexivity and internal processing). Arguably, Archer has predominantly developed these ideas from research with students in higher education where there is a degree of privilege in the way they might articulate internal processes which doesn’t always work well with marginalised populations. Fractured reflexivity is of course a wonderful concept and something I have no doubt many evaluators come across in the implementation of programmes, but perhaps there could be more work to develop this concept further? Perhaps there is something I am missing, it would be great if someone might point out where in realistic evaluation there is a more comprehensive critical appraisal on these more abstract concepts which challenge the researcher to think more in-depth about emergent powers and properties and internal processes? 

 

Being pragmatic is not something that critical realists have always done well at, but doesn’t methodological individualism in realistic evaluation run the risk of being reductionist.

 

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards <[log in to unmask]> On Behalf Of Jagosh, Justin
Sent: 20 March 2019 02:52
To: [log in to unmask]
Subject: ...in defence of so-called "methodological individualism"

 

PLEASE NOTE: When you click 'Reply' to any message it will be sent to all RAMESES List members. If you only want to reply to the sender please remove [log in to unmask] from the 'To:' section of your email.

Hi all,

 

I think there is a solid case to be made for what has been coined “methodological individualism” in Realist Evaluation -- specifically in terms of the definition of mechanism as ‘how people respond or react to resources’. We may ask:  ‘should Realist Evaluation focus mechanisms on the individual-level agency of actors involved in programmes? Or should we be also looking at the emergent properties they create and mechanisms at different levels of a system?’ The debate, in my opinion, should be pinned around questions of pragmatism. What kind of analysis will lead to ideas for the resolution of complex entrenched problems? Perhaps “methodological individualism” as it has been coined, is adequate in many instances for these pragmatic goals – with some exceptions.

 

The case to be made for “methodological individualism” is not in contradiction to an appreciation of emergent properties. Acknowledging the emergence of properties within complex interventions is to appreciate the complexity of reality itself. But if we see mechanisms as what happens in the hearts and minds of people who design, manage, implement and receive interventions – the argument here is that this provides the kinds of insights needed for assessing and innovating the design of programmes. Pragmatism is absolutely key in Realist Evaluation.

 

Realist Evaluation also proposes alternatives to ‘causation via counter-factual’ by illuminating the inner workings of the human element of programmes. Humanizing our understanding of intervention functioning has been Pawson and Tilley’s incredibly valuable contribution to evaluation science. And although that appears to suggest that Realist Evaluation places an over-emphasis on agency of actors, this is not the case. The activity of Realist evaluation is not about isolating mechanisms but rather understanding context-mechanism interactions. There is freedom in using the CMO heuristic to account for the laminated layers of a complex system. Practically speaking however, baby steps into complexity theorizing is needed for many of us - with the first steps being around how individuals buy-in/or not  - this is a very good path forward for many people and may be adequate.

 

Take the following example: Say an evaluator is studying interventions to address gender pay inequity across organizations. Say, hypothetically speaking, that it has been found that women on average are paid less than their male counterparts for jobs that have equal, equitable or even greater roles and responsibilities. Just as with the example of how hydrogen and oxygen when mixed create an emergent property of ‘wetness’, it could be theorized that the mix of differential salary amounts across genders for same work in the workplace creates an emergent property of ‘toxic work environment’. This ‘toxicity’ may be defined as friction amongst staff, feelings of resentment, feelings of being under-valued, feeling entitled, feeling guilty, feeling trapped in gender roles and expectations, – and so on. This may lead to outcomes such as reduced communication between staff, increased inefficiencies and errors, social anxiety and reduced morale and productivity. Now the point here is that even with the emergent property of a ‘toxic work environment’, in order to make pragmatic sense of this and develop solutions, the analysis cannot remain at the level of ‘toxic environment’ even if the concept is at some level useful. The analysis has to go back in to how people think and feel – how employees feel about negotiating their salaries, how managers feel in regard to negotiating across genders, how department leads feel about gender equity sensitivity training, how employees feel when they receive such training etc. etc. So while still retaining an appreciation of the complexity inherent in realist accounts of ontology, “methodological individualism” as it has been coined can be a very valuable emphasis that Realist Evaluation can push forward for understanding where pragmatic innovations can occur in the system to improve conditions and resolve entrenched problems.

 

Justin   

 

Justin Jagosh, Ph.D

Director, Centre for Advancement in Realist Evaluation and Synthesis (CARES)

www.realistmethodology-cares.org

&

Honorary Research Associate

Institute for Psychology, Health and Society

University of Liverpool, UK

 

 

To UNSUBSCRIBE please see: https://www.jiscmail.ac.uk/help/subscribers/faq.html#join

---

This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.

To UNSUBSCRIBE please see: https://www.jiscmail.ac.uk/help/subscribers/faq.html#join

To UNSUBSCRIBE please see: https://www.jiscmail.ac.uk/help/subscribers/faq.html#join

To UNSUBSCRIBE please see: https://www.jiscmail.ac.uk/help/subscribers/faq.html#join

To UNSUBSCRIBE please see: https://www.jiscmail.ac.uk/help/subscribers/faq.html#join

To UNSUBSCRIBE please see: https://www.jiscmail.ac.uk/help/subscribers/faq.html#join