Thanks Bejoy. This is very useful and I agree with much of what you say -
but (just because I can't help myself) - I have to challenge the idea that
theory based evaluation is usually qualitative. I've never actually done a
study to be sure - but certainly realist methods are designed to be mixed
methods. In my 14 years running realist evaluations, I think I've done
three that were purely qualitative...
Cheers
Gill
-----Original Message-----
From: Nambiar, Bejoy [mailto:[log in to unmask]]
Sent: Thursday, 4 February 2016 2:28 AM
To: [log in to unmask]; Gill Westhorp
Subject: Re: New approaches to evaluating complex health and care systems
Dear all,
An important point Sandeep and a great response Gil!
Just to add-I think there is a distinction to be made between evaluation OF
QI methods (PDSAs, TQM , Lean, Six sigma etc) and evaluation USING QI
methods. The latter, as Gil rightly points out is a very important sub-set
of any evaluation. In fact, the use of annotated run charts, used commonly
in QI projects, can complement the (usually) qualitative data from
theory-based evaluation.
For evaluation of QI interventions, a great place to start would be to look
at theories around Diffusion of Innovation.
Cheers,
Bejoy
Bejoy P.Nambiar
UCL Institute for Global Health
Malawi (+265 992 711 198)
Twitter: @bejoynambiar
www.ucl.ac.uk/igh (Twitter @UCLGlobalHealth) Global Leaders in Quality
Improvement Research bit.ly/1TCbLZW Learn about evaluation module in
MSc.GlobHlth http://bit.ly/1NP9fIp
________________________________________
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
<[log in to unmask]> on behalf of Gill Westhorp
<[log in to unmask]>
Sent: 02 February 2016 22:15
To: [log in to unmask]
Subject: Re: New approaches to evaluating complex health and care systems
Hi Sandeep
Hmm, I think there's an implicit assumption about the purpose, the nature
and the context of evaluation here. Of course program improvement is ONE
purpose for evaluation - is it the only one? (I'd argue not.) Is QI an
appropriate substitute if program improvement is not the purpose (I'd argue
not.)
Of course inbuilt QI cycles are useful in many contexts - do they work in
all contexts? (I'd argue not.) Are they always more cost efficient or more
intrinsically useful? (Nothing works everywhere or for everyone, said the
realist, so probably not.)
Of course evaluation can be 'separate from' QI - but it can also be in
house, cyclical, participatory - in fact, QI (using PDCA or any other
method) can look an awful lot like a sub-set of evaluation. Are they always
different? (I'd argue not.)
And finally - should QI be evaluated? (I'd argue yes.) If so, can it be
evaluated purely using QI methods? (I'd argue that depends on the purpose,
nature and context of the evaluation....).
Cheers
Gil
-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[mailto:[log in to unmask]] On Behalf Of Sandeep Reddy
Sent: Wednesday, 3 February 2016 8:34 AM
To: [log in to unmask]
Subject: Re: New approaches to evaluating complex health and care systems
Thanks Geoff.
From an alternative point of view, there is an emerging school of thought
questioning the need for external program evaluation (whether it be
method-oriented or theory-oriented evaluation). The view is that inbuilt
quality improvement cycles like Deming's PDSA (or as it has been modified in
Japan: PDCA) are more cost-efficient and intrinsically valuable to the
organisation/stakeholders.
-Sandeep
> On 3 Feb 2016, at 7:20 AM, Geoff Wong <[log in to unmask]> wrote:
>
> A common challenge that realist evaluators come across is in making a case
for why a realist evaluation is the right approach to take.
> Help comes in the form of this Analysis article in The BMJ (attached):
>
> New approaches to evaluating complex health and care systems Tara
> Lamont, Nicholas Barber, John de Pury, Naomi Fulop, Stephanie
> Garfield-Birkbeck, Richard Lilford, Liz Mear, Rosalind Raine, Ray
> Fitzpatrick BMJ 2016;352:i154
>
> They argue the case that there are different valid approaches to
> evaluating complex interventions, amongst these approaches - realist
> evaluation :-)
>
> Geoff
>
> <New_approaches_to_evaluating_complex_interventions_BMJ_Feb_2016.pdf>=
|