Terence,

I've been increasingly staying out of these conversations, but I'd like to clime in here, albeit on only a facet of this conversation.

I like your questions, or list, on what is not being researched — or at least is being widely neglected vis-a-vis the potential value of answering those questions. They imply a research agenda. How about you establish one? A small event or some other mechanism that brings people together around shared questions, and then establishes benchmarks for making progress towards having them answered? Then apply for a grant to advance the agenda? Life is short. Let's make progress! 

I only want to comment on E - "The best strategy so far …"

I think you've jumped in the right direction and I'm very interested in your thoughts, but it feels you've soared over a lilly pad or two. To abstract from this just a bit, you're asking what might anchor a design in valid claims about the world, so that the design — should it be implemented — has the greatest possibility or likelihood of success.

I agree wholeheartedly that some aspects of modeling, simulation, and prototyping from evidence (not inspiration) is critical. And indeed, math(s) provide one such means. But when the issue centers on meaning — and the comparative study of meanings within or across social systems, for example,— then there is less to measure, and more to interpret, which directs us towards interpretive sciences. And yes, there are empirical approaches to the analysis of social phenomena.

This is only to say, lets not jump over other forms of rigorous analysis to land on only one — math. Lets instead reach for the analytical tool appropriate to answering the questions we need to ask to solve the design challenge we face. One of them absolutely will be math. But it is a toolbox, and the art is in mating the tool to the challenge. No such thing as an all-purpose tool …

On this subject, Lisa Rudnick and I will be posting our new paper soon on "evidence-based program design", which we wrote for an Inter-agency working group at the UN. People are trying to figure out what it means to move from opinion-based decision making to evidence-based design for programming. We've provided a conceptual framework for this, and this year, UNIDIR, The Policy Lab and livework (Norway) will be working with UN field teams to help build the user-centered tools necessary to advance that conceptual framework in an operational space.

We have not found "design thinking" to be very relevant. We have, however, built on our previously published work on Strategic Design, by which we mean "a design process anchored in evidence and directed towards a strategic goal." The general design discussions have been very helpful in thinking about this, especially such themes as user-centered design; prototyping; user testing; modeling, and some other discussions that orient us away from the typical approaches used by policymakers.

I'll be following this thread. All the best,

Derek.



_________________
Dr. Derek B. Miller
Director

The Policy Lab
321 Columbus Ave.
Seventh Floor of the Electric Carriage House
Boston, MA 02116
United States of America

Phone
+1 617 440 4409
Twitter
@Policylabtweets
Web
www.thepolicylab.org 

This e-mail includes proprietary and confidential information belonging to The Policy Lab, Ltd. All rights reserved.

On Jun 5, 2012, at 3:38 AM, stefanie di russo wrote:

> Hi Terence,
> 
> Au contraire! It seems you (and now I) are not alone. The issues and
> questions you have raised are precisely what my research investigations
> into design thinking are about. To date, I have only found literature
> highlighting and discussing these issues and the need for more empirical
> research, but none publishing so.
> 
> To share my thoughts on your points...
> 
> *1. The practical limitations of design thinking as a human activity ( i.e.
> what it cannot do)*
> 
> -- Currently investigating this issue through observational case study
> research. The main problem i find is limiting the scope/defining what
> design thinking is or isnt within practice. This is a really tricky point
> to justify in itself (within academia) without enough evidence as you may
> know.
> 
> *2. The practical limits of applicability  of design thinking  (i.e. where
> it
> does not work, and where you should not apply it)*
> 
> -- This will also be answered through investigations above (insights of
> observational research into design thinking)
> 
> *3. The theoretical/ epistemological limitations of the concept of design
> thinking in terms of other more established theories of human cognition.*
> 
> -- This might require another phd in itself, but theoretical limitations
> are proving difficult to justify without leaving oneself open for critique
> by examiners (i have been contemplating the use of grounded theory but was
> warned against doing this)
> 
> *4. What needs to be done instead of 'design thinking' in for design tasks
> in
> which design thinking does not apply.*
> *
> *
> -- This might also depend on how you define design thinking and what it
> consists of. There are problems and circumstances where design thinking
> certainly would not be applicable
> 
> 5. The implications of better understanding in this area for the validity of
> design theory in other areas of design practice
> 
> -- I believe this to be quite important. What prompted myself to
> investigate this issue was one part skepticism (despite being a designer
> and secretly enjoying its success ;) i was also dissatisfied with a) what
> little information there was to justify DT and b) worry that with DT being
> so broadly lathered over every industry, failings were surely due to arise
> and without investigation into how/why DT works, it may cause more harm
> than good. My opinion is that i will find there are certain parameters
> where DT will flourish, and others where it needs to be avoided. And of
> course it goes without saying that it is imperative to design theory. How
> can we promote and use something we cannot justify or explain?
> 
> 
>> * My findings so far:
>> 
>> A) Design thinking does not work for design situations whose outcome
>> behaviour is shaped by 2 or more feedback loops. It typically produces
>> design solutions that fail after a short time. This failure of design
>> thinking unfortunately includes most of the interesting design situations
>> in
>> complex problems and strategy that design thinking is claimed to apply. The
>> proof is easy  to do in a practical way and anyone can prove it for
>> themselves with half an hour or so of effort.*
>> 
>> --  Do you have evidence/case studies of the failures you are referring
> to? As mentioned, it depends upon what context/industry. As i am focusing
> on the application of DT in social innovation and policy design, i can say
> that within social and community projects multiple feedback loops appear to
> be of benefit for outcomes. Within policy design, it appears that
> regardless of design thinking and its iterative cycle- outcomes fail due to
> higher levels of management.
> 
> Furthermore, questions such as: was it DT that failed, or the individual
> that failed to apply it correctly? To me, that seems to be one of the other
> reasons why DT may not be as successful as it should be. DT is less of a
> method and a toolkit for anyone to slap into a project, and more about
> knowing (intuitively as well) when/what/how to apply it within a certain
> context.
> *
> *
> *B) The reason is a biological limitation in human thinking.*
> *
> *
> *--* This is has long been understood. Scholars such as Herbert Simon and
> Horst Rittel both commented that designers can do much more than
> 'satisfying'
> *
> *
> *C) The same reasoning shows that using collaborative design thinking*
> 
>> * approaches also fail for design situations whose outcome behaviour is
>> shaped
>> by 2 or more feedback loops.*
> 
> 
> -- Example? And in what context? Many human centered designers would argue
> with you against that statement.
> 
> 
> 
>> *D) It also shows that drawings, regardless of how insightful, illustrated
>> or complicated, do not explain or enable people to understand design
>> situations whose outcome behaviour is shaped by 2 or more feedback loops.
>> *
> 
> 
> -- Im not sure if feedback loops is relevant to the basic understanding of
> a design problem? Can you elaborate on this point?
> *
> *
> *E) The best strategy so far that produces design outcomes that are*
> 
>> * successful, as expected,  and produce outcomes that are  intended,  is
>> to
>> use mathematically-based simulation methods  with the input information
>> collated from collaborative multiple constituents/stakeholders. The
>> simulation is then used to 'test' proposed designs by using the simulation
>> to reveal the design outcomes over time. System dynamics in its various
>> forms  at the moment appears to be the best design tool/simulation method.
>> An example of this approach  (although conceptually problematic in its
>> reasoning) is *:
>> 
>> -- Simulation methods are also described by H. Simon in The Sciences of
> the Artificial. Correct me if i am wrong, but a mathematically based
> simulation would be of little use in socially sustainable projects, where
> communities and cultural groups of individuals are involved that require
> sensitivity to culture and tradition.
> 
> Furthermore, however wonderful this simulation may be, one cannot predict
> the longevity of outcomes in the future. Even Simon acknowledged this,
> which is why he and others settled on 'satisfying' than 'predicting'
> perfect outcomes no matter how intelligent the computations and algorithms
> are.
> 
>> 
>> F) A new direction I've developed to provide a more holistic design method
>> is through 'dynamic variety space axioms'.   This involves looking at  the
>> dynamics of the distribution of variety across multi-feedback loop  design
>> situations and using axioms I've identified  that characterise the change
>> behaviour and indicate points of leverage to change the future behaviour of
>> the system in particular ways.  This is early days for this approach yet.
>> 
> 
> -- Dynamic Variety Space Axioms. This (and much of your argument/points in
> this post) seem best applied within technological design developments. But
> no matter how much data you receive and how great the computational
> analysis is to predict outcomes- you may be missing the simple point that
> underlies design thinking: that it is a human centered endeavour. No amount
> of mathematics can computate the complexity of human emotion
> 
> I hope to learn more of your insight,
> 
> Regards,
> 
> -- 
> *Stefanie Di Russo*
> 
> PhD Student
> Faculty of Design
> Swinburne University
> *twitter:* @stefdirusso <https://twitter.com/#!/stefdirusso>
> *linkedin: public
> *profile<http://www.linkedin.com/pub/stefanie-di-russo/35/16/a84>