I’m in the third year of my PhD and right now I’m conducting my realist interviews as part of an evaluation case study. I’ve done about 35 so far. I was requested by my supervisory team not to use the realist terminology but to find other way of explaining it. 

Before I explain some of the practical tips I’ll give some background for context. 

I’m not evaluating a typical pre post intervention. My research is evaluating everyday routine practice of rurally based community stroke teams. These teams have been identified from a national audit as being high performing. The reason for this is to find out how they manage to fulfil the audit criteria when other rural teams struggle. 

So when I’m presenting the concept of theories. I tell them I’ve done background reading and stakeholder engagement with community stroke teams and I have ideas about why I think some rural teams are able to fulfil the audit whereas others do struggle. However, I also tell them that my ideas are tentative and if my ideas don’t match their experience to please let me know as it’s important that I accurately capture their experiences. 

When trying to test the theories I use phrases such as 

“ my background reading tells me that X is really important for Y and I’d like to talk to you about it. 

Or 

“I observed the other day that.... and that seems to impact on Y can we chat about it please?”

Or 

“One of ideas I’m currently developing is X and I don’t think I’ve quite got my head round it yet. Can you tell me a bit more about this and help me understand please?”

Sometimes I find they tell me things that are essentially resources - tangible things which the service have implemented to make things run smoother and I’ve found with some people realist mechanisms are hard to elicit. Recently I explored the idea of asking people to imagine that we were on a panel advising the commissioning of a brand new rural community stroke service and essentially we wanted to give the commissioners some tips so that they could learn from the experiences of other rural teams. I start these conversations off by saying. 

“So... we know that having X is really important but from your perspective can you tell me why it is important. What difference does having X make therapists working within the team?”

Sometimes this allows people to think deeper. Some people don’t want to talk about their own experiences in the realist way but by making them think a little bit more abstractly and ask them to think about a brand new service they appear more at ease with discussing the underlying mechanisms. 

I’ve found this an effective way of probing and I plan on continuing to use this strategy. However I’d love to hear other people’s tips and ideas for phrasing questions in these scenarios as I think we can all help each other along this path. 

Happy to converse more and continue this debate with others. 

Thanks 

Jo Howe

Postgraduate Researcher 

University of Nottingham 

Thank you so much Gill and Andrew- I really appreciate the guidance and support. Nice to know I have realist companions on my learning journey! 

Sent from my iPad

On Jan 26, 2018, at 2:31 PM, Gill Westhorp <[log in to unmask]> wrote:

Hi Sharon, Andrew and all

Sharon, I love the idea of the visual approach and would be really interested to hear how it goes.  As a general principle, I think it’s really useful for the discussion list if people discuss their methods!

 

I’d like to pick up Andrew’s point about terminology.  I agree that there are people for whom the CMO language is unhelpful, but I don’t believe it’s that hard to explain to most people, or to build into questions.  More importantly I find that explaining the approach helps respondents to understand what it is you’re trying to get at, and helps them to participate in the interview.  So I’ll often provide a couple of sentences of introduction.

 

When evaluating an intervention I might say something along the lines of: “We assume that nothing works for everyone or everywhere, so we’re trying to work out why it’s different in different circumstances.” (That’s context, of course, but it also starts to get at mechanisms.) And “Interventions themselves don’t cause change. They give people new resources or opportunities, but it’s how people respond to that that causes the change.” (Mechanism). And “And the results are different for different people too.”

 

I find that this gives people ‘permission’ to talk about how it was for them (participants) and about variety in outcomes (workers). 

 

To go deeper into mechanism  I ask people to talk to me about ‘their thinking in relation to…’ to get at reasoning and ‘how they feel about …’ to get at values (they often respond in terms of emotions first, but a bit of probing usually gets below that). 

 

My problem with “If-then-leading to” is that “then” can elicit just ‘the next visible thing’, rather than getting ‘under the surface’ for mechanisms.  I’d be really keen to hear how those who’ve used it get past that? 

 

You can also use “If” (context), “Then” (immediate or intermediate outcome), “Because” (mechanism).

 

Of course – we all mix and match the ways we work within and between interviews and I don’t think there’s one right way to do it…

 

Would love to hear others’ techniques for this.

 

Cheers

Gill

 

From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Andrew Booth
Sent: Saturday, 27 January 2018 8:49 AM
To: [log in to unmask]
Subject: Re: Evaluation Topic and idea for interviewing

 

Hi Sharon

 

Some really interesting ideas in your email. As someone supervising a realist PhD I would encourage you not to impose the unfriendly terminology of CMOs on your informants. In other words you need them to engage with realist logic for your evaluation without having to give them all a Realist 101 course! 

 

So I would suggest you ask them to articulate the CMOs in a more readily accessible way. I have been very influenced by both Mark Pearson and Jo Rycroft Malone and so use the IF (Context) THEN  (Mechanism) LEADING TO (Outcome) construction.  By all means use the novel shaped post its but make the terminology more accessible.

 

Best wishes

 

Andrew 

 

On 26 Jan 2018 21:29, "Provost, Sharon" <[log in to unmask]> wrote:

 

I first want to thank everyone who contributes questions and answers via the RAMESES mail group- every entry is unbelievably helpful!

I am a PhD candidate and beginning realist researcher so thank you in advance for your patience. 

 

1) As I am just beginning my realist review and realist evaluation I wanted to check if anyone else is looking at the same topic with a realist approach: I am evaluating a violence prevention education program to decrease the violence and violence related injuries that healthcare workers experience from patients (those they provide care for). It is an international problem with a large volume of literature but I have not found any research at this point that has taken a realist approach. Is anyone out there doing a similar thing? 😀

 

2) I will be conducting about 45 interviews and 9-12 focus groups across 3 different organizations and 9 sites  (I have funding, stakeholder buy-in, and some research assistant support).  As my own learning/thinking style is visual and interactive an idea keeps surfacing in my head to use props in my interviews (and possibly in the focus groups) to visually explain the initial program theory and elicit ideas about the CMO configurations.

 

I thought about using:

•  little cutout or 3D “people”
•  a large piece of paper with a separate “classroom” and  “workplace” 
•  small shaped post it notes

• circular “contexts”
• thought bubble “mechanisms”
• square “outcomes”. 

 

I would populate some of the c,m,o stickies based on the initial program theory and leave some blank that could be filled in during the interview/focus group. In addition to taping the sessions I thought I could take photos of the configurations that participants create, validate or refute with a small piece of paper in the photo with the coded participant ID, date and time). I wondered if this would help participants quickly understand the realist focus on theory as opposed to more general approach they may have experienced in other interviews while keeping me focused on theory building. 

 

Any thoughts or comments would be greatly appreciated! 

 

 

Thank you 

Sharon Provost 

Interdisciplinary PhD Candidate

University of British Columbia 

Vancouver, BC, Canada

This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please contact the sender and delete the email and attachment. 

Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham. Email communications with the University of Nottingham may be monitored where permitted by law.