Hi all

"... knowledge for change must be created and used through a participatory frame which values all types of knowledge as in principle equally valid, though any particular knowledge claim is open to critical questioning".  This last sentence of the Hills and Carroll (2004) article which Simon circulated bring to my mind some ideas from critical systems thinking.  These focus on "boundary" judgements regarding, for example, what knowledge is relevant to an issue being investigated - such judgements have to be made, and are important in determining what data are gathered and what the analysis of that data produces.

A practical method that has been developed for (potentially) highlighting and opening up such judgements to inspection and discussion is called Critical Systems Heuristics (CSH).  See the chapter by Ulrich and Reynolds (2010) at http://oro.open.ac.uk/21299/1/systems-approaches_ch6.pdf .  The method promotes discussion among stakeholders about both what is and what ought to be the situation (I think this responds to the issue highlighted in the Porter and O'Halloran article, but not with a fixed form of utopian vision, which is what I take them to be advocating).

I think CSH has potential with regard to conducting the participative aspects of realist evaluations and realist reviews.


Alan Boyd
Research Associate in Healthcare and Public Sector Management
The Herbert Simon Institute for Public Policy & Management
Manchester Business School

Email: [log in to unmask]
Tel: 0161 275 2923
Fax: 0161 275 0557
Web: www.mbs.ac.uk/research/people/profiles/alan.boyd

The University of Manchester
Harold Hankins building
University Precinct Centre
Oxford Road
Manchester M13 9PL

The latest Herbert Simon Institute Update magazine is at http://research.mbs.ac.uk/hsi



-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Simon Carroll
Sent: 17 August 2012 00:55
To: [log in to unmask]
Subject: Re: Concrete utopianism

Hi Geoff and Justin,

Interesting beginning of an important discussion. Not sure I would buy exactly how Porter and O'Halloran frame the problem, but I definitely have felt for a long time that there is something about how realistic evaluation has developed that tends to put aside the 'critical' part of 'critical realism', even though it certainly is grounded in Bhaskar's ontological and epistemological argumentation, as opposed to alternative forms of contemporary realism, such as that of Harré.
There is a tension generated by this elision that always opens up a tendency to interpret realistic evaluation as a kind of post-positivism that retains a fairly standard 'scientism' and sometimes seems to be used as little more than a thinly reasoned justification for so-called 'mixed methods'.....I welcome this initial discussion and attach a brief piece I think is relevant that a colleague and I wrote 'way back' in 2004.

Cheers, Simon.

On Thu, Aug 16, 2012 at 2:26 PM, Justin Jagosh, Mr <[log in to unmask]> wrote:
> Geoff,
>
> Thanks for this interesting article. I'm always impressed at the calibre of thinking coming from U.K. health professions research.
>
> The authors state what I also believe to be true, which is that realist evaluation (and synthesis) needs to stay true to the logic of realism in light of its growing association/integration with other methods.
>
> Through the popularization of realism in the health sciences, there will inevitably be a watering down of the principles and methods. Even a limited application of realism may demonstrate utility, but may not actualize the real potential that the realist approach carries in wrangling complex evidence. This is why I think that realist review practitioners, especially those new to the method, who get stuck somewhere along the way in their review or evaluation process, need to go back and continue the process of absorbing the philosophy of realism to push past methodological barriers. This I believe is more important than simple adherence to guidelines (which has its merits), in order to avoid "routinization" leading to dogmatic application of the method.
>
> Justin
>
> ________________________________
> Justin Jagosh, Ph.D
> Canadian Institutes for Health Research Post-Doctoral Fellow Centre
> for Participatory Research at McGill (PRAM) Department of Family
> Medicine McGill University ________________________________ Office and
> Mailing Address:
> Women's Health Research Institute
> BC Women's Hospital and Health Centre
> AB319 - 4500 Oak St.
> Vancouver, BC, V6H 3N1
> 604-875-2424 ext. 4877
>
> ________________________________________
> From: Realist and Meta-narrative Evidence Synthesis: Evolving
> Standards [[log in to unmask]] on behalf of Geoff Wong
> [[log in to unmask]]
> Sent: August 16, 2012 1:52 PM
> To: [log in to unmask]
> Subject: Concrete utopianism
>
> Hi all,
>
> I came across a paper on realistic evaluation that I thought might be of interest to you all as many of the ideas seemed to me to be also relevant to realist synthesis/review.
>
> The use and limitation of realistic evaluation as a tool for evidence based practice: a critical realist perspective by Porter and O'Halloran (attached).
>
> My summary for busy folk is that their central arguments are:
> 1) Randomised controlled trials (RCTs) can tell us whether mechanisms
> can work
> 2) Realistic evaluation (RE) will tell us more about these mechanisms
> (when the work, for whom etc...)
>
> Now this is the interesting bit for me ....
> 3) RE is too pragmatic, which means RE practitioners run the risk of being, "...vulnerab[le] to what Habermas (1971) terms ‘technocratic consciousness’, the sleight of hand which transmutes personal and ethical problems into scientific and technological categories, amenable to purposive-rationality."
> So their suggestion is that we need concrete utopianism. I may have got this wrong, but my understanding is that we need to have a 'goal' / purpose / ideology to drive our research.
> My initial worry was that this sounded like 'playing god' .. so who is to say my purpose is better than yours??? However, the authors did seem to me more balanced, in that they argue we just need to be aware that we can be seduced into being technocrats if we do not realise that research has a purpose and we need to be aware of what our own purpose is.
>
> To me their suggestion that we need more utopianism brought out more questions than it answered?
> Do their arguments make sense? Are we at risk of ending up as technocrats? Is utopianism the only answer? How would we decide if our 'view' was the right one to follow? ... I am sure you are getting the idea.
>
> Geoff