Dear Rasmus and all,

I think it is worth thinking through the language and terminology around the "counterfactual realist" debate to explore articulating the complexity.

First, the contention and objection has to do with the conflation of realist methodology with counterfactual logic and I think the objection is quite valid. Bear in mind the term 'counter-factual' is used differently in philosophy of science, psychology and other disciplines.  If we take a step back and start with a very simple definition, we could say that counterfactual reasoning is about gathering "facts" of one situation, setting or time point and comparing this information or data with "facts" from another situation, setting or time point. The goals of this is to generate explanation about the impact of an intervention or experimental manipulation. The quotations around the word 'fact' are meant to remind us that from a realist inquiry perspective, fact is not a concept quite embraced (in the way theory is). While realism promotes the ontology of a singular reality, it does not assume a taken-for-granted position on easy access to that reality through our knowledge processes (even the "gold standard" processes).

The issue is that the concept of a 'fact' obscures contextual variations and leaves us with the rather 'positivist' impression that we can unearth regularities. We do indeed unearth things that look like 'regularities' but there will always be exceptions - contexts that would change the outcome of that regularity.  Tony Lawson has advanced this using the term 'demi-regularity.'  This means that regular patterns can be detected but are always conditional and there will always be situations (context) that can modify the pattern. This gives us reasons to examine context on an on-going basis. This same principle holds true about 'facts'. And the realist critique of a "fact" is that it assumes a kind of taken-for-granted knowledge about the truth of things. 

Having said that, it would be safe to say that humans and likely other living beings use counter-factual logic everyday for our advancement and survival. For example, if I wanted to purchase a new computer, I may gather the facts for one device and gather similar facts about another device and make a comparison. This comparison may lead to new knowledge which would be great. But that is counterfactual observation which is quite different from counterfactual experimentation that uses aggregation of quantitative data - and this has not really been clarified. While it could be argued that all experiments involve counterfactual processes, not all counterfactual processes involve experimentation. In counterfactual experimentation the process is to compare the 'facts' of a control group with 'facts' of the experimental, or some other variation - whether it be natural experiment or through randomization. In order to undertake the comparison, the facts must have symmetry and be converted to a numerical dimension because the calculation processes can not compare numbers with other kinds of data a statistical formula.  

The problem with counterfactual experimentation has been described well in Ray's body of work. I would add is that it is not the comparison that is the problem per se - but rather it is the reduction of contextual elements into a numerical dimension which creates a 'flat' ontological view of these elements - what I would call an 'artificially prescribed stability of concept.' This means that these elements have to be shaped up along certain dimensions for the quantification and comparison to take place. Maybe that works, and maybe it doesn't. But consider concepts like: "Well-being", "Resiliency" or "Empowerment".  Perhaps the end product of a counter-factual experiment for such outcomes of interest has value, but in principle this still does not sound so reasonable for complex interventions involving core mechanisms of a socially contingent nature. This is because through quantification we loose the opportunity to theorize resources, responses, and outcomes in their contexts - with all their complex contradictions intact, using the idea of ontological depth to guide our understanding.

Qualitative cross-case comparisons can look like comparing the facts of one case with facts of another, and this is not objectionable from a realist perspective because for that, the comparison happens in our interpretive process and as Ray has suggested, the realist paradigm is aligned with a constructivist epistemology.  It's not comparison that is the contention.

Rather, in counter-factual experimentation the comparison requires homogenization and aggregation of data bits, which are compared not through interpretation, but a statistical process. The extent to which complex concepts and elements are reduced to variables, such reduction likely precludes a kind of theorizing that more accurately (deeply?) aligns these concepts to the reality we are seeking to understand.  - Ultimately to support insight and solutions building. Perhaps RCT-style counter-factual experimentation should be called 'numerical aggregate comparison' and the kind of counter-factual observation that would be used in a realist project be called "counter-theoretical comparison".

Thanks for the opportunity to stir the pot. It will be great to see other ideas emerge.

sincerely
Justin