Dear All,
Got a couple of interesting off-list notes in which people asked me whether I have changed my mind about generalizing from limited data. I have not.
David Sless explained carefully why limited data can make sense for specific cases. This is especially the case when experts use expert judgment given appropriate experience and background knowledge. Jakob Nielsen explains this is the specific context of tests to improve, say, a web site or a product. You’re testing to find out what does not work; then you fix it and test again. Many small tests gives you more information than a single major test:
https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
One of the other partners in the Nielsen Norman Group explains why you cannot generalize from small tests — and in fact, why so much of what comes forward as research in design is not research at all, given the lack of substantial warrants for grand claims:
http://www.jnd.org/dn.mss/why_design_education.html
So I continue to stand on everything that I wrote. And I agree with the specific nuances that David brought forward. You cannot generalize from small numbers. Neither can you learn about appropriate variance. The kind of work that W. Edward Demings called for in industry is impossible with small numbers.
Nevertheless, small numbers — even a single problem! — can allow you to identify a problem. This warning can bring a problem solving team to the right place on the assembly line where the appropriate experts can meet around the problem to start solving it.
We’ve got plenty of small numbers in design. It would be useful in addition to build a knowledge base comparable to the larger knowledge base we see in other fields. In the meantime, it also helps to see expert analysis of specific cases. Despite our fervent bias in favor of hands-on research for individual problems in their situated context, we see too few genuine case studies in the design literature.
Yours,
Ken
Ken Friedman, PhD, DSc (hc), FDRS | Editor-in-Chief | 设计 She Ji. The Journal of Design, Economics, and Innovation | Published by Tongji University in Cooperation with Elsevier | URL: http://www.journals.elsevier.com/she-ji-the-journal-of-design-economics-and-innovation/
Chair Professor of Design Innovation Studies | College of Design and Innovation | Tongji University | Shanghai, China ||| University Distinguished Professor | Centre for Design Innovation | Swinburne University of Technology | Melbourne, Australia
Email [log in to unmask] | Academia http://swinburne.academia.edu/KenFriedman | D&I http://tjdi.tongji.edu.cn
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|