Dear all,
An important point Sandeep and a great response Gil!
Just to add-I think there is a distinction to be made between evaluation OF QI methods (PDSAs, TQM , Lean, Six sigma etc) and evaluation USING QI methods. The latter, as Gil rightly points out is a very important sub-set of any evaluation. In fact, the use of annotated run charts, used commonly in QI projects, can complement the (usually) qualitative data from theory-based evaluation.
For evaluation of  QI interventions, a great place to start would be to look at theories around Diffusion of Innovation.
Cheers,
Bejoy
Bejoy P.Nambiar
UCL Institute for Global Health
Malawi (+265 992 711 198)
Twitter: @bejoynambiar
www.ucl.ac.uk/igh (Twitter @UCLGlobalHealth)
Global Leaders in Quality Improvement Research bit.ly/1TCbLZW
Learn about evaluation module in MSc.GlobHlth http://bit.ly/1NP9fIp

________________________________________
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards <[log in to unmask]> on behalf of Gill Westhorp <[log in to unmask]>
Sent: 02 February 2016 22:15
To: [log in to unmask]
Subject: Re: New approaches to evaluating complex health and care systems

Hi Sandeep
Hmm, I think there's an implicit assumption about the purpose, the nature and the context of evaluation here.  Of course program improvement is ONE purpose for evaluation - is it the only one?  (I'd argue not.)   Is QI an appropriate substitute if program improvement is not the purpose (I'd argue not.)

Of course inbuilt QI cycles are useful in many contexts - do they work in all contexts? (I'd argue not.)  Are they always more cost efficient or more intrinsically useful? (Nothing works everywhere or for everyone, said the realist, so probably not.)

Of course evaluation can be 'separate from' QI - but it can also be in house, cyclical, participatory - in fact, QI (using PDCA or any other method) can look an awful lot like a sub-set of evaluation.  Are they always different? (I'd argue not.)

And finally - should QI be evaluated?  (I'd argue yes.)  If so, can it be evaluated purely using QI methods? (I'd argue that depends on the purpose, nature and context of the evaluation....).

Cheers
Gil

-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Sandeep Reddy
Sent: Wednesday, 3 February 2016 8:34 AM
To: [log in to unmask]
Subject: Re: New approaches to evaluating complex health and care systems

Thanks Geoff.

From an alternative point of view, there is an emerging school of thought questioning the need for external program evaluation (whether it be method-oriented or theory-oriented evaluation). The view is that inbuilt quality improvement cycles like Deming’s PDSA (or as it has been modified in Japan: PDCA) are more cost-efficient and intrinsically valuable to the organisation/stakeholders.

-Sandeep



> On 3 Feb 2016, at 7:20 AM, Geoff Wong <[log in to unmask]> wrote:
>
> A common challenge that realist evaluators come across is in making a case for why a realist evaluation is the right approach to take.
> Help comes in the form of this Analysis article in The BMJ (attached):
>
> New approaches to evaluating complex health and care systems Tara
> Lamont, Nicholas Barber, John de Pury, Naomi Fulop, Stephanie
> Garfield-Birkbeck, Richard Lilford, Liz Mear, Rosalind Raine, Ray
> Fitzpatrick BMJ 2016;352:i154
>
> They argue the case that there are different valid approaches to
> evaluating complex interventions, amongst these approaches - realist
> evaluation :-)
>
> Geoff
>
> <New_approaches_to_evaluating_complex_interventions_BMJ_Feb_2016.pdf>