 |
 |
Thanks for the responses so far. The question I suppose could be better framed, as several have sent me or pointed out projects which have been 'realist' process evaluations. This does make sense to me, seeing as you can have a realist [insert method] anything can't you?! However, process evaluations do have guidance, from the MRC, developed by people who wanted to set out the rubric for PE, and I wonder to what extent they, and 'we' are talking about the same thing (generative causation and explanation), but just giving it different names? So in a sense the question 'when would you choose to do a process evaluation over a realist evaluation' depends on whether you are investigation the process of how some intervention reaches its objectives, or interested in finding out more about what works, how it works for whom, in what circumstances and why.... but are these not really the same thing?! Or is it more a matter of emphasis - any kind of process evaluation is (necessarily) only focussed on the "how", at the exclusion of all else (unless they offer explanatory traction to the "how")... ? Is it just a matter of taste? Do we need to employ the realist CMOCs when we can get to the same conclusions using theory of change, or logic models.... ? I guess my question runs a bit deeper then - how do we delineate realist from other kinds of evaluation....? Or is it like trying to say how is realist interviewing different from interviewing? In earlier RAMESES posts, I know that it always seems to come back to the programme theory, looking for then testing out generative mechanisms.... but is this not what a process evaluation does too? Are we saying then that all process evaluation are realist in nature because they do these things? So many more questions and I still feel like I'm missing something obvious....! (I am so glad I'm not involved in running the seminar :)) Thanks again for comments, Best wishes Becky. Rebecca Hardwick PhD Student 01392 727408 Email: [log in to unmask] -----Original Message----- From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Mikkel Møldrup-Lakjer Sent: 25 January 2018 12:10 To: [log in to unmask] Subject: SV: In what circumstances would you do a realist evaluation over a process evaluation? Hello everyone, I believe the answer to the question from Rebecca Hardwick does indeed depend on the meaning of the question itself, because "process evaluation" is not a well-defined evaluation model upon which there is agreement between institutions and evaluators. Process evaluation may mean an "implementation evaluation" as Craig Chambers suggests, and may focus either on barriers and helpers for an implementation with high fidelity, on assessing the fidelity itself, or both. Or we may take the definition from Patton's /Utilization-Focused Evaluation/ (4th edition: 324): "...looking at /how/ a product or outcome is produced rather than looking at the product itself". (I would add: But of course in order to look at the process, you have to take into account the products/outputs/outcomes, because they are the fulfillment of the process). This means asking process questions rather than variance questions according to the typology of Joseph A. Maxwell. But is a mechanism not also a process? Maxwell explicitly states so: “For critical realists, in contrast, the concept of “mechanism” (in the social sciences, “process” is the usual term) is central to explanation, and these mechanisms and processes are seen as real phenomena, rather than simply as abstract models“ (Maxwell 2012:8-9). Summing it all up, I would suggest that realist evaluation is a member of the family of process evaluations which comes with specific foundations, principles and perspectives for looking at "processes". So the choice between realist evaluation and process evaluation would be a choice between a process evaluation with a focus on CMO's or a process evaluation with a focus on other questions. Best regards, Mikkel Møldrup-Lakjer Evaluation Consultant Denmark -----Oprindelig meddelelse----- Fra: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] På vegne af Craig Chambers Sendt: 24. januar 2018 13:49 Til: [log in to unmask] Emne: Re: In what circumstances would you do a realist evaluation over a process evaluation? Hi Rebecca, my very naive response, having never conducted a process evaluation and being only just over a year's worth of part-time PhD into realist evaluation, would be the answer is partly in the question. Is a process evaluation not an evaluation of whether a process has been implemented correctly/successfully or not? This would not have to consider outcomes as they may be contingent upon some other parameter. The focus instead is along the lines of "have the constituent components of this programme been implemented as the programme developers stipulated/intended". Thanks for the interesting question and look forward to being shot down in flames for my ineptitude and naivety ;-) best regards, Craig Chambers Part-time PhD, full time physio