Dear Birger,

The concepts of evidence-based practice and evidence-based design are especially seductive terms for those of us interested in advancing research-informed design. So I consider the points you raise in your post as serious and important. I am sympathetic to your point that “Social sciences like most others are concerned with what is, (the social system) while design is concerned with what ought to be”, and I think this point also speaks to the significant research in design that shows the importance of problem framing in design practice, not only solutioning (f.ex, Dorst, 2015). I am also cautiously sympathetic to the broad view of evidence-based design, however, I share the concern that current models of evidence-based design are unclear and critical examination of the underlying meaning and assumptions of evidence-based design is needed. 

A couple of points:

1. Evidence Quality in EBD
Evidence simply means “grounds for belief”, in this sense all decisions in design practice, which are not intentionally deceptive, are evidence-based. Even those designers whose designs have been dismissed as failures, had grounds to believe that their design would work. What is new in the EBD view is the specific model of what counts as good evidence. The current EBD view draws on the hierarchical model of evidence quality developed by the evidence-based medicine (EBM) movement. The simplified version is as follows:

1. Randomized Controlled Trials (RCT) 
2. Observational Studies
3. Expertise/Mechanistic reasoning 

Systematic reviews of many RCTs or observational studies being better than single studies, and comparative studies being better than individual case studies. 

The levels of evidence quality are based on their freedom from bias; hence expertise is at the very bottom – in many EBM models expertise is not considered evidence at all. It is important to keep this model of evidence quality in mind when considering the definition of EBM: “Evidence-based medicine (EDM). The conscientious, explicit, and judicious use of current best evidence in making decisions about the case of individual patients. The practice of evidence-based medicine requires the integration of individual clinical expertise with the best available external clinical evidence from systematic research and our patient’s unique values and circumstances” (Straus et. al, 2005, p. 280-281). In EBM “best research evidence” really means RCTs and systematic reviews of RCTs. However, conducting RCTs might be more suited to the large pharmaceutical companies and universities in the medical research discipline than the design research discipline, because most changes that designers recommend alter several factors simultaneously which creates confounding variables and makes it difficult to measure the intended effect. While there are some successful RCTs and quasi-experimental studies in design, most systematic design research is observational. The point here is that the gold standard of evidence for EBM decision making just doesn’t often happen in design research and perhaps is not be appropriate for professional design practice. I think that rather than design research investing more and more funding in expensive RCTs, what is needed is a reasonable and practicable theory of evidence that can be put to use in design practice.

2. Evidence relevance in EBD
Evidence is relative to an argument. Grounds/Data/Information, whether from an experiment or a survey or expertise, can only be indirect evidence for a design decision. This is because a reliable design decision (a predictiveness claim) should be based on strong evidence and a sound warrant. The warrant of an argument is a premise that bridges the claim and its supporting evidence connecting them into a logically related pair. A warrant does not answer questions about whether the evidence is accurate but about whether the evidence is relevant to the decision. In other words, an EBD decision not only needs to ask whether X worked somewhere, but also whether it will work for us? This may not seem to be a particularly controversial point. But I think it needs to be taken seriously since the cogency of an argument is only as strong as its weakest premise. The practice of EBD requires more than the skill in appraising the quality of evidence, it also requires the critical thinking and judgment needed to form a cogent argument that shows that the evidence is relevant to the claim. The development of EBD requires stocks of strong evidence, professional designers who can appraise evidence quality and also who have training in critical thinking and strategic judgment (and modesty) not to overestimate what the evidence can deliver.


Luke

Luke Feast, PhD

Postdoctoral Researcher in Design


Aalto yliopisto / Aalto University

Taiteiden ja suunnittelun korkeakoulu / School of Arts, Design and Architecture
Muotoilun laitos / Department of design
Helsinki, Finland



On 14 July 2015 at 13:07, Birger Sevaldson <[log in to unmask]> wrote:

It would be a good idea to review the debates and critique on evidence-based practices and evidence-based design especially and move ahead with this in a thoughtful and critical manner to avoid a similar unproductive split in the design research community. There is a special risk connected to the evidence-based approach because of its great selling power and good fit with current public management ideas where responsibility is removed from individual practitioners to rule-based management, and big data in a massive scale. The potential for serious damage should be obvious.


Dorst, K. (2015). Frame innovation: Create new thinking by design. MIT Press.

Straus, S. E., Richardson, W. S., Glasziou, P., & Haynes, R. B. (2005). Evidence-based medicine: how to practice and teach EBM. Edinburgh: Elsevier Churchill Livingstone.


-----------------------------------------------------------------
PhD-Design mailing list  <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------