Dear Terry,

Thanks for the explanation, and thank you for providing these literature review documents to the list. I’ve read them before. While I accept that this was a sufficient literature review for your 1998 PhD thesis, these are not sufficient as a critical, analytical review of the design literature.  

At a certain point, I grow uncomfortable in repeating the observation that you read things superficially, but this is a broad overview of the design research literature with little deep analysis. 

The analysis of most items in this literature review doesn't analyse the specific ideas or issues in the source texts you claim to read. Rather, it is a broad bibliographic essay citing the source documents by topic. Rather than analysing or testing the ideas in these sources, you simply aggregate them by author and year. In most cases, you simply claim that all documents in the same category belong in the category to which you assign them. You do not show that the author’s own views warrant your categorical placements by demonstrating the author’s views in his or her own words. It’s as though you simply stacked up piles of books and articles in a rough sort, saying “this stack represents X,” “this stack Y,” “this stack Z,” and so on. This is what I meant by my earlier comment on superficial reading. You do not read carefully what the authors themselves represent in their thinking. 

So, no. I do not agree that you have actually done a careful analysis or review of the design literature. At least not in your PhD thesis. You did a kind of superficial review that would pass muster with an examiner who did not himself or herself know the literature well enough it question your assertions. I’m not saying that everything in the literature review was mistaken: I am saying it was superficial and it does not support your claim that you "personally know the theories of the design research literature are a mess [because you] have tested most of them over the last 40 years.”

You may have tested them, but you do not show it in these documents.

Based on the literature review, you took a superficial tour of books and articles, skimming the pages to categorise them. You now state that this literature review shows these authors and their work to be deficient with respect to design theory. This is not evident to me,

Neither in your thesis nor anywhere else do you systematically demonstrate that any specific group of theories is wrong, false, or incorrect in the way that a mathematician, an engineer, or a scientist would demonstrate that a theorem, hypothesis, or finding is wrong. 

I’m not holding you to your thesis based simply on the work you did then. You are now making the specific claim that you reviewed a thousand or so source documents of design theory in the 1990s, and you present these three document as evidence for this claim. What I’m saying is that this broad but superficial review of the literature does not support your claim of massive flaws through the entire literature.

My view on this may be mistaken. As with the taxonomy tool published in Design Studies, any member of the list is free to read the document. Each reader may draw his own conclusion. Some may agree that you have demonstrated the “past mess of design theory.” I don’t, not even for engineering design. The analytical descriptions are too superficial to draw such a conclusion. 

There are problems in the literature, but not in the way that you describe it. And I can explain why no one has tried for the bottle of wine. Any serious challenge of the kind you propose requires well defined criteria for judgment and an impartial panel of judges. You offer neither well defined criteria nor a reasonable way to choose judges. Whether this offer has been on the table for 15 years or 15 minutes, no one wants to play when the dealer holds all the cards in a marked deck.

Sorry if this seem blunt, but you put these documents forward with the claim that they support your assessment of the design research literature. I disagree. Those who read these documents and agree with you are free to speak. 

Yours,

Ken

Ken Friedman, PhD, DSc (hc), FDRS | Editor-in-Chief | 设计 She Ji. The Journal of Design, Economics, and Innovation | Published by Tongji University in Cooperation with Elsevier | URL: http://www.journals.elsevier.com/she-ji-the-journal-of-design-economics-and-innovation/

Chair Professor of Design Innovation Studies | College of Design and Innovation | Tongji University | Shanghai, China ||| University Distinguished Professor | Centre for Design Innovation | Swinburne University of Technology | Melbourne, Australia

--


> On 2015Sep21, at 16:24, Terence Love <[log in to unmask]> wrote:

—snip—

> Five, you are correct on the problem of reviewing the design literature and the amount of writing necessary.  I attempted to do a review of the whole design theory literature in the course of a PhD that took 8 years. In its draft form the thesis was over 400,000 words. After realising it would take many more years I wrote a thesis on just part of the research focusing on engineering design rather than design as a whole. That was in the 90s and the body of literature was smaller then. That literature review comprised three sections, one of 82 pages, one of 53 pages and one of 29 pages: a total of 164 pages of literature review. I haven't had the enthusiasm to write up the rest.  It is likely many other PhD students have done better or more comprehensive reviews since that time. The three I did are at:
> http://www.love.com.au/docs/Pre2000/Design-theory-lit-review-SEEDT-Tlove-w-refs.pdf 
> http://www.love.com.au/docs/Pre2000/1998%20SEED&DT_WP_Appendix%201.htm
> http://www.love.com.au/docs/Pre2000/1998%20SEED&DT_WP_Appendix%202.htm
> 
> Six, from your comment on the Meta-Theoretical Analysis Tool for analysing design theory, it seems you have not read (or perhaps not understood) the paper. The paper explains how the tool is used. This occurs in the explanation of its development and why it is needed,  and, practically, how to use the tool  in the sections: 'Use of the Meta-Theoretical Structure' and 'Using the results of the Meta-Theoretical Decomposition'   Incidentally, the paper also reviews comments by many others who felt the design theory literature is a mess.
> 
> Seven, You ask me to prove design theory literature is in a mess. I offer a simple test - I like simple accurate tests. If the field is healthy and not a mess, then it should be effortless to choose any paper to critically review and test for the usual theory problems, fallacies, errors of reasoning etc. As I said, I like simple tests. If it is easy to find a proportion of papers that will stand up to critical analysis of the reasoning underpinning their theory, then we can do some statistical analysis on the situation and get some idea how messy (or not) the design theory literature is. The simplest test though is to ask a large number of people to nominate any one paper to review for epistemological problems. If the design theory field is not a mess, providing a paper should be effortless and I provide a prize for it if its analyses don't have problems. This offer has been open for 15 years now and still no valid paper has been submitted.

—snip—


-----------------------------------------------------------------
PhD-Design mailing list  <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------