Mark

Mark Pearson PhD

Senior Research Fellow

Peninsula Technology Assessment Group (PenTAG)

University of Exeter Medical School

E: [log in to unmask]

T: 0044 (0) 1392 726079

http://medicine.exeter.ac.uk/about/profiles/index.php?web_id=Mark_Pearson

Hi all,

I was wondering if anyone has examples of research that combines realist evaluation with comparative effectiveness research. Comparative effectiveness seems to be a broad catch-all term for research that compares two different interventions or service approaches, using an array of methodologies. It would seem quite logical to use realist concepts in a comparative study in which you collect qualitative data, construct program theory and do a CMO analysis for two separate interventions or models.

I'm also looking for examples of realist evaluations that incorporate cost comparisons as well.

thank you
Justin   

________________________________
Justin Jagosh, Ph.D
Canadian Institutes of Health Research Post-Doctoral Fellow
Centre for Participatory Research at McGill (PRAM)
Department of Family Medicine
McGill University
[log in to unmask]
________________________________

Office and Mailing Address:
Rm 426 - School of Population and Public Health
Universtity of British Columbia
Vancouver, BC, V6T 1Z3
604-822-3814



________________________________________
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [[log in to unmask]] on behalf of Marina Buswell [[log in to unmask]]
Sent: July 23, 2013 8:55 AM
To: [log in to unmask]
Subject: Re: Realist thinking is getting me stuck not started - help

Dear Gill and Geoff,

Geoff - No I hadn't seen the training material http://www.ramesesproject.org/index.php?pr=Project_outputs#train only the papers. These are useful as I can apply the learning exercises to my work -so 'crack on with it' while reading up more.

Gill thanks for your questions. I have an in-house 'quick and dirty' evaluation which actually is a really useful insight into the underlying theory of the program by those who developed it on the ground.

Thank you both.
Marina
-----Original Message-----
From: Gill Westhorp [mailto:[log in to unmask]]
Sent: 19 July 2013 23:52
To: 'Realist and Meta-narrative Evidence Synthesis: Evolving Standards'; Marina Buswell
Subject: RE: Realist thinking is getting me stuck not started - help

Hi Marina
You're certainly at least a closet realist!! :-)   You're thinking in all the right directions, anyway.

Geoff's absolutely right about starting to develop a bit of tentative program theory.  A couple of suggestions about ways to get started:

a) Write a realist version of one of the questions you want to answer.  For example:  Rather than "Is it safe/appropriate/comparable outcomes to in hospital assessment"  perhaps something like  "In what circumstances and for whom is it safe and appropriate? In what ways are outcomes similar to and different from outcomes in hospital assessment? When it is safe, how come? When it's not comparable, how come not?"

b) Work with a handful of people who know their way around the issue (your paramedics might be good for this) and take some rough stabs about the 'realist phrases' in the questions (e.g. 'in what circumstances', 'for whom').

c) Once you've got a few rough stabs in place you can do two things... work out what evidence you'd look for in your evaluation to test this out, and take another look at the evidence you've found in your lit review to refine your rough stabs into slightly less rough (um, looking for a word to continue the analogy -  cuts???)

On another front - you can do realist analysis of qualitative data from interviews to test hypotheses, or to understand mechanisms, or to refine hypotheses before testing.  Joseph Maxwell has published on that.  I've got some unpublished material on it too which I can share if you email me directly ([log in to unmask]) if that's of interest.

Cheers
Gill

-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Marina Buswell
Sent: Friday, 19 July 2013 7:55 PM
To: [log in to unmask]
Subject: Realist thinking is getting me stuck not started - help

Dear RAMESES people,

I’d like to ask for some help in applying a realist approach to my current work, pragmatically yet robustly. I again find that thinking about methodology (just ordered Ray's new book) is stopping me 'cracking on' (which is what I used to do when I had PCT jobs rather than a research position).

I'll introduce myself quickly. I’m Marina currently working on a project to evaluate a new acute geriatric intervention service (AGEIS) delivered by paramedics working with OT/Physios and responding to non-emergency 999 calls for older people (who have fallen or are otherwise unwell), also doing a review to ask ‘what role do ambulance services play in urgent and emergency care for people with dementia?’ But previously I’ve worked in PCTs in ‘commissioning information’ i.e. churning round SUS data into pretty charts, and health promotion (getting people to stop smoking) I’ve done my MPH and learned all the hierarchy of evidence dogma BUT before that I was an organic chemist (PhD) and this is really what shapes how I come to a problem. Observe some stuff, use the best methods out there to figure out what you’ve got, propose some mechanisms to explain how you got from your starting materials to your products. Do some more experiments to test those mechanisms and so on. I think that is the realist angle – or am I a pragmatist?

THE EVALUATION: I’ve had the conversation about what our key questions should be (with the lead paramedic and community geriatrician). They have come out like this so far, unrefined.

1.     Is it safe/appropriate/comparable outcomes to in hospital assessment
2.     Test the model – Paramedic and Therapist (compared to other falls service models)
3.     What are long-term benefits e.g. on preventing falls, hip fracture, hospital admissions, quality of life
4.     Cost-effectiveness
5.     Admission avoidance
6.     Sustainability
7.     What impact does dementia have
8.     How does the service run? What makes it ‘work’ or not?

The service started April this year and has funding until Mar 2014. So it needs to be a timely and pragmatic evaluation and almost certainly can’t show anything statistically significant around admission avoidance. My inclination is to use their in-house activity and outcomes data and do some semi-structured interviews with staff, and patients and carers if I can get appropriate ethics and access. The service sees approx. 70 people per month, with a second vehicle to start Sep 2013. I think 1,2,7&8 can be addressed in someway within the time frame and resource.

THE REVIEW: What role do ambulance services play in urgent and emergency care for people with dementia? I am stuck trying to write up our review (2 paramedics as co-reviewers) because it does not fall neatly into any particular school, it started as a traditional review conducted at the start of a project, took a systematic turn as we filtered through 5000+ titles, went unsystematic as the literature was so sparse we had half from the grey literature and were looking for ‘nuggets’, all the time reading about realist synthesis, critical interpretative synthesis… and now I’m stuck I’d love to say it’s a realist synthesis but it’s not. Calling it a 'critical scoping review' and just writing what we did and what we found.

I get the ideas of context and mechanism but still get a bit stuck round 'mid-range theory' do I have one but don't know it? In my only non-chemistry paper, I wrote this in the intro "In the ‘quit rate’ and targets-obsessed culture of stop smoking services it can be difficult to take time to reflect on why or why not programmes may be working. The aim of this research was to get a fuller understanding of the young people’s experience of smoking and quitting and how useful and appropriate the support offered was to them." http://hej.sagepub.com/content/72/4/408 I think that's early realist leanings?

Advice from those of you more experienced gratefully received. I will now try to just 'crack on' (I'm not used to having all this 'space' to think!) Thanks Marina

Marina Buswell
Research Associate (PT usually Tue, Thu, Fri) Cambridge Institute of Public Health CLAHRC - Old Age Theme

E: [log in to unmask]
T: +44 (0)1223 330341 / 07780 452214
www.clahrc-cp.nihr.ac.uk
Collaborations for Leadership in Applied Health Research and Care