Dear Hernan,
Nigel Cross and Ali Ilhan both make valid points. You are doing social science research *about* design expertise. This requires using social research methods — or at least understanding and explaining why you’ve chosen not to make use of social research methods. In this case, a proper survey of the field requires a far larger sample. Or, you could have made use of a small sample with detailed case studies.
Karl Weick (1979: 35-42) “Research Clock” represents the trade-offs between simplicity, accuracy, and generality. The “research clock” is an imaginary device. It looks like a 12-hour clock face with numbers from 1-12. At 12, place the word “general.” At 4, place the word “accurate.” At 8, place the word “simple.” The clock has two hands of equal length.
Weick proposes that any given research project in the social sciences can only meet two of these three criteria. It does so at the necessary cost of neglecting the third criterion. Case study research, for example, can be simple and accurate, but it cannot be general. Medical trials require massive data leading to generalizable conclusions from thousands of cases. Medicine also involves meta-research that accumulates the data of dozens of clinical trials involving hundreds of thousands of human beings.
If I were to conduct a survey such as this, I’d have asked for some help from an expert in survey methods. An architect and a civil engineer have other kinds of expertise — I wouldn’t ask a psychologist to calculate the structural load of a bridge and I wouldn’t ask a sociologist to plan the heating needs of a building.
At least, as Ali suggested, it would have been helpful to do some study on surveys. Perhaps a literature review on professional expertise in general, and design expertise in specific would also reveal what people have learned about these issues over the past five or six decades.
William Trochim maintains an excellent free web site on social research methods. Trochim is Professor of Policy Analysis and Management at Cornell University — his background is in psychology and he did his PhD in research methodology and evaluation. You’ll find the site here.
https://socialresearchmethods.net/kb/
There is a massive literature on professional expertise and on such topics as situated knowledge that involve expertise. For example, the The Cambridge Handbook of Expertise and Expert Performance (Ericsson, K. Anders, Robert R. Hoffman, Aaron Kozbelt, and A. Mark Williams 2018). This book from Cambridge University Press is now in its 2nd edition. This book offers a solid overview of how to assess, understand, and explain expert performance. The first edition was good — the new edition looks to be even better, both up-to-date and thorough.
http://www.cambridge.org/gb/academic/subjects/psychology/cognition/cambridge-handbook-expertise-and-expert-performance-2nd-edition#RrbD8SboK2SlLkrU.99
Within the larger field of expertise, situated knowledge, and expert performance, there is a smaller, focused literature on design expertise. One chapter in the new edition of The Cambridge Handbook involves design expertise. Nigel Cross is the author. From the 1980s on, there has been a steady stream of books, studies, and articles on the topic.
It seems to me that you are going to face problems with any results you derive from the current version of the survey. If I were still in the relatively early phases of a projects such as this, I'd consider saving the data — and the respondent names — to start again. You may be able to use some of what you’ve gathered for heuristic or exploratory purposes, but a solid survey requires methodological clarity.
You are investing a great deal of work in this project. You’ll be more likely to get useful results by rebuilding the survey before you invest still more work.
Yours,
Ken
References
Ericsson, K. Anders, Robert R. Hoffman, Aaron Kozbelt, and A. Mark Williams. 2018. The Cambridge Handbook of Expertise and Expert Performance. Cambridge: Cambridge University Press.
Weick, Karl. 1979. The Social Psychology of Organizing. Second Edition. New York: McGraw-Hill.
--
Ken Friedman, Ph.D., D.Sc. (hc), FDRS | Editor-in-Chief | 设计 She Ji. The Journal of Design, Economics, and Innovation | Published by Tongji University in Cooperation with Elsevier | URL: http://www.journals.elsevier.com/she-ji-the-journal-of-design-economics-and-innovation/
Chair Professor of Design Innovation Studies | College of Design and Innovation | Tongji University | Shanghai, China ||| Email [log in to unmask] | Academia http://swinburne.academia.edu/KenFriedman | D&I http://tjdi.tongji.edu.cn
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|