Hi Mairi
To add to Justin's beautiful answer:

So what does it mean when we say we will 'test' a realist theory?  Firstly,
it means our theory will take a certain structure - it will identify
hypothesised (or previously researched, if we're lucky) mechanisms, the
outcomes that they generate, and the contexts in which they do and don't
work.

It will then consider 'what might constitute evidence for these elements of
the theory?'  Because mechanisms operate a) as a function of 'the powers and
liabilities' of the system, and at a different level of the system (either
higher or lower) than the outcomes of interest, this means investigating
different things than the 'outcomes level'.  Most reviews will not be able
to investigate all aspects of the theory and will select a particular
aspect. 

To return to your example: I agree with Justin that goal-setting is a
program strategy or activity rather than a mechanism.  (Simple test:  can
you see it happening?  Is it something that the program (the people in the
program) actually 'do'? - if yes, it's an activity/strategy, not a
mechanism).  If you were investigating goal setting, you'd be asking
questions like:  So how and why does goal setting change the outcomes that
programs generate? What does it provide for whom along the program
implementation pathway?  How does it change the 'reasoning' (catch all term
for whatever happens inside the heads of decision-makers along that route)
of whom?  What different decisions are generated as a result of that
different reasoning, and how do those different decisions generate different
outcomes? When does and doesn't that happen? Having hypothesised those
elements - yes, you would then toddle off to investigate the literature
about those elements, then bring it back and apply it to your particular
topic.

The other way in which realist theory testing differs from other forms of
theory testing is in the notion of 'cases as nuggets of evidence along the
theory chain'.  I.E. - having laid out the THEORY (not just the program
theory per se, but the realist theory as well), the 'nuggets' of evidence
are aligned against the specific aspect of the theory to which they relate.
This helps to overcome a common problem for new realist analysts, who often
want to argue that evidence of outcomes equals evidence of the hypothesised
mechanism.  It doesn't - because several different mechanisms could have
generated the same outcome.   

I've answered here primarily in terms of mechanism but of course the same
sorts of processes also apply to outcomes (NB outcomes at different levels
at different points in time in different levels of the system are all
'outcomes' in realist terms - even though they can also become new contexts
or operate as 'resources' for new mechanisms in the next stage of the
process) and to context.  The trick for context is to work out which
ELEMENTS of context relate to / affect which mechanisms (and/or the ability
of decision-makers to put their decisions into practice).
 

Cheers
Gill 

-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[mailto:[log in to unmask]] On Behalf Of Mairi Anne Young
Sent: Wednesday, 29 January 2014 4:13 AM
To: [log in to unmask]
Subject: Advice on testing theories - Realist Review

Hi, 

I'm a 2nd year Dental Public Health PhD student working on a protocol for a
Realist Review. My research question is: Which mechanisms within child
health interventions, positively influence child health promoting care giver
behaviour? 

The review will form part of my PhD which is an evaluation of a Lay Health
Worker role within the Childsmile dental programme. 

While researching Realist Review methodology and developing my protocol the
only stumbling block I'm coming up against is the concept of 'theory
testing'. I understand the searching is an iterative process, one which
involves identifying and testing theories. However I'm failing to understand
what exactly is involved within this testing phase. 

For example, if it arose that Goal Setting was a theory, would I then be
re-entering the literature with the intention of locating evidence related
to Goal Setting? Would this evidence have to confirm why Goal Setting is an
effective tool? 

I would be grateful for any advice or information anyone could provide on
this matter. 

Kind regards, 
Mairi 




Mairi Young BA (hons) MSc MBPsS 
PhD Student
The University of Glasgow, Glasgow Dental Hospital & School 
Level 8 COH Office 378 Sauchiehall Street
Glasgow G2 3JZ
 
Email: [log in to unmask]
Tel:    0794 7744 517