 |
 |
Hi Vanessa
Welcome to the 'disputatious community of truth seekers'.
There is a common root to all of your realisms (generative causation, social morphogenesis). But by my reckoning 'realist evaluation' is closer to 'subtle realism' than it is to 'critical realism'.
Sam Porter has provided you with a couple of references on his critical realist take on realist evaluation. Needless to say, I disagree.
I've already responded to his:
'The Uncritical Realism of Realist Evaluation'
with paper entitled:
'The Ersatz Realism of Critical Realism' .
It will be published in the same journal (Evaluation) but alas there is a bit of a backlog - I think it will be out Issue 1, 2016.
It is a splendidly spirited debate so I hope you enjoy reading it. But my advice, for what it is worth, is that it is not necessary for PhD students to perfect their understanding of ontology and epistemology prior to undertaking empirical work. If you work out some sensible ways to answer your research questions you'll find yourself well on the way to a broader understanding of how the world is put together.
Ray
Dear All,
I’ve been following the discussions from PhD students but was reticent to join in and thus demonstrate my lack of understanding. However I am really struggling to differentiate between realism, subtle realism (Hammersley), critical realism (Sayer, Bhaskar) and realist evaluation (Pawson & Tilley). I’m leaning towards CR but still not sure of the exact differences in philosophy this involves or how it influences my method & data analysis. For example, in one email (2nd Jul) Gill made the point that interviews will only be realist if they are building, testing &/or refining realist theory and that needs to cover context, mechanism and outcome. I don’t think this suits my research question but the basic realist philosophy appeals. I’m using a case study design, 2 areas, looking at a particular policy recommendation that states people who have had a stroke should be reviewed at 6 weeks, 6 months and yearly thereafter. Most people are still under the acute sector for the 6 month review, provision of the 6 month review is patchy, and few fund a yearly review. I’m mainly using interviews with some observation (where ethics allows) and document analysis. My research question:
‘How does the review process support adults with long term need post-stroke?’ & objectives:
By mechanisms, I was using it in the lay sense….but still not sure how critical realism would define versus Pawson & Tilley?
With different perspectives (patient, carer, provider, commissioner) and an over-reliance on interviews I can’t imagine that I can achieve clarity with the mechanisms and outcomes at this stage – it is more exploratory (but I’m not keen on using interpretivism either). So can I frame as a critical realist approach without tying myself in knots with CMO configurations that I won’t have the time/resources to test and refine? And how do I ensure/demonstrate that this informs my method & data analysis.
I’d be very grateful for your suggestions,
Vanessa.
V. Abrahamson
ESRC PhD Candidate in Social Policy
Centre for Health Services Studies
University of Kent
George Allen Wing
Canterbury
Kent CT2 7NF
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]]
On Behalf Of Sonia Dalkin
Sent: 11 August 2015 10:48
To: [log in to unmask]
Subject: Re: Defining mechanisms as processes? What Realist findings look like? PhD newbie questions
Hi Mandy and others,
Thank you for posting your questions Mandy and for all the responses so far, they are all so interesting and can only help us to question our own understanding of realist methods and learn more.
I would echo Trisha and Justin’s comments – picking an examiner with realist experience is essential if you’re using a realist framework and often methodological innovation is best left post PhD. A lot of PhD candidates can trip themselves up when trying to use hybrids as there can be clashes of philosophies and associated methodologies. This isn’t to say that this can’t be done though!
Q1 - I guess the question is why would you want to reinvent the wheel – Pawson & Tilley provide a definition of mechanism and if you’re using their methodology it’s probably best to stick to their definition, unless you have a rationale for changing it. In your PhD it’s always important to be able to justify any deviations from the traditional methodology. Why did you do it - how has it enhanced your research project? What would have been lacking had you not have done this?
Q2 – I think you’re using the term ‘mechanism statements’ as the equivalent of Context-Mechanism-Outcome configurations (CMOc) in Pawson & Tilley. Ana Manzano and Ray Pawson pick up on this further in their diagnostic workshop paper (attached). They state that C, M and O should not be listed (a common mistake) but should be in configurations – in this context, this mechanism is activated which leads to this outcome. A recent paper I wrote (attached) with colleagues from University of Leeds (Dr. Joanne Greenhalgh) and Northumbria University (Dr. Monique Lhussier, Dr. Anna Jones, Dr. Bill Cunningham) highlighted the need to outline resource and reasoning (another common mistake in RE) within CMOc and provided a rejig of the C+M=O formula to aid operationalisation. The paper still conforms to Pawson & Tilley’s original ideas but just provides an alternative way of configuring the formula. In this paper we provide examples of ‘mechanism statements’ or what are usually known as ‘CMOc’ with resource and reasoning. You may find this helpful to see whether your definition of mechanism statement is the same as CMOc.
I hope I’ve interpreted your questions correctly Mandy and hope this in some way helps!
Best of luck,
Sonia
__________________________________________________________________________
Dr Sonia Dalkin
Lecturer in Public Health, Department of Public Health, Faculty of Health & Life Sciences
Twitter: @SoniaDalkin
Room H006, Coach Lane Campus East, Northumbria University, Newcastle upon Tyne, NE7 7XA, United Kingdom
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]]
On Behalf Of Trish Greenhalgh
Sent: 11 August 2015 07:33
To:
[log in to unmask]
Subject: Re: Defining mechanisms as processes? What Realist findings look like? PhD newbie questions
I don’t like the way ‘motivation’ is creeping in here. Can we say ‘agency-dependent’? Motivation seems wrong paradigm.
Trisha Greenhalgh
Professor of Primary Care Health Sciences
T: +44 (0)1865 289363 E: [log in to unmask]
www.phc.ox.ac.uk | @OxPrimaryCare | @trishgreenhalgh
Nuffield Department of Primary Care Health Sciences, University of Oxford
New Radcliffe House, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG
From:
<Jagosh>, Justin <[log in to unmask]>
Reply-To: "Realist and Meta-narrative Evidence Synthesis: Evolving Standards" <[log in to unmask]>, "Jagosh, Justin" <[log in to unmask]>
Date: Tuesday, 11 August 2015 00:34
To: "[log in to unmask]" <[log in to unmask]>
Subject: Re: Defining mechanisms as processes? What Realist findings look like? PhD newbie questions
Dear Mandy,
Your rigorous questioning on the concept of mechanism is excellent!
I agree with Trish that it is vital to provide a definition of terms if you diverge from Pawson and Tilley. But Realist Evaluation is gaining recognition and credibility so if your reason to change the definition is because people in your field are not familiar with it, then as Trish says, you need to instead find the right supervisors/examiners and stick with it. Innovation is better done after you graduate J.
If, on the other hand, you want to diverge from the definition because you don’t feel it works for what you are studying, that’s a different story and it’s totally justified – but that would also need to be clearly explained. Is the definition of mechanism, as defined by P&T as reactions to resources offered – not adequate?
Are you examining youth employment interventions? If yes, do they create resources via implementation of the program theory? How do people respond to these resources, given who they are, their backgrounds, and their contextual constrainers and enablers?
If your research question is : what makes people ‘employable?’, and you are not studying a specific intervention (or class of interventions), then you do have to innovate P&T’s work, because they have built realist methodology pinned around the evaluation/synthesis of interventions, services, policies etc. The question: “what creates youth employability?” is a sociological question that, in and of itself needs to be unpacked.
So your definition of mechanism as:
‘a generalisable, context-dependent and motivation-dependent interaction process, through which outcomes of interest are partly generated (or strongly influenced?)’ to provide conceptual scaffolding for non-realist readers?
is a bit muddy to my mind. Are the concepts ‘generalizable’ and ‘context-dependent’ not antithetical when put together in this statement? Also ‘motivation-dependent interaction process’ is also a bit muddy. Is there such a thing as a motivation-independent interaction process when it comes to people interacting with each other and with policies? Also do people interact with policies? Or just react to them? We are all reacting to our environment all the time, and even when we think we’re not – (because it’s also happening subconsciously). Now if you want to say that mechanism has to do with how people react/respond within an interaction process - in a context - perhaps that might be clearer, I’m not sure.
You might have a look the figure in our paper, linking CMO configurations in a ripple effect - (attached). In that paper, we too looked at a process over time, and suggest that you can link CMOs to explain this kind of emergent causal chain – one in which reactions lead to outcomes, which impact on the context over time or across intervention phases. It’s still rather crude, but it has helped us create realist middle-range theory in our subject area that is more closely aligned with “what’s really going” …(which can be considered one of the possible overarching goals of RE)….
Please feel free to clarify if I haven’t grasped adequately your question! And hopefully others will stir the pot too…
Justin
For example, my research question could ask:
What can be clarified about key processes that shape individual ‘employability’ and employment/unemployment outcomes; particularly for youth populations and with a focus on vocational and ‘at risk’ youth subgroup outcome patterns? How is the variation in youth outcomes, as subgroup patterns, explained by certain types of intervention and/or context conditions affecting those processes?
By the way - improvement in individual ‘employability status’ may be defined and studied as a harder to measure, long run outcome pattern of interest - for which improvements in an individual’s employment outcomes over time are treated as one of a few key employability status/outcome indicators. A Rough Theory section proposes that employment outcomes, and improvements in individual states of employability (based on theorised indicators), could simultaneously be evaluated as intertwined outcome objectives.
Question 2) When you describe mechanisms as research findings, don’t they need to comprise mechanism statements? I.e. they must explain a mechanism’s relevance to a set of outcomes - then also describe something about why, for which subgroup/s, or how at least one other context condition is relevant to understanding an outcome pattern that emerges from that mechanism (interaction process)?
So a mechanism could be summarised as say a 1-3 word label - whereas a mechanism finding must comprise a full sentence statement distinguishing something about context conditions affecting the outcome patterns emerging from that mechanism, and/or affecting an intervention's workability in turn.
Said another way - isn’t it true that a complete mechanism statement/finding must distinguish and relate at least one Context variable to how a Mechanism works, and to an Outcome pattern in turn? Whereas a mechanism on its own can be summarised as say a 1-3 word label (e.g. Recruitment and Selection Mechanisms as being relevant to employment).
I realise the above may be nit-picking but it helps to visualise what comprehensive Realist type ‘findings’ look like before trying to produce them.
Many thanks in advance for anyone who wants to add their two cents to any of the above screeds!
Regards
Mandy McGirr
PhD researcher
School of Government, Victoria University of Wellington
New Zealand