Thanks Withanage, Zender, Sless, Love, Popov, for your kind reply about my search for inferential researchers. (Sorry if I missed a name).

Approximating design and multivariate statistics (and avoiding speculative statistics)

I am interested in approaching statistics and design for some reasons:

I had already seen some statistical initiatives that tried to make some of the designers' decisions "more objective”. Among these tools already used we have:
quantitative analysis of insights,
counting words in qualitative interviews,
analysis of numerical patterns in comparative studies,
calculation sheets to compare understanding performance between different documents (something done with excellence by the Communication Research Institute, from David Sless / Australia).
But nothing came close to the tools that are used by Economics and Marketing, like inferential statistics. It is quite different to say that 2 things are "related," such as the presence of serifs and readability, and to say that one thing "causes" the other, that is, the serif is definitely a factor that caused the improvement in reading.

Obviously, statistics have their limitations and pitfalls, such as false positive or false negative results, inadequate generalizations, to name a few. But it has the ability to actually demonstrate whether a choice occurred by chance or due to a deliberately proposed change by a designer.

Statistical capabilities offer very useful tools for designers, such as:
Cluster Analysis - People can be grouped by preferences, beliefs and attitudes in an approach that does more than just group by age, gender, or place where they live.

Logistic Regression - It is possible to predict a binary behavior like "buy or not buy", "open or not open an application", "choose or not choose an option". Unlike a linear regression, which quantifies the effects of some independent variables on a continuously dependent variable, the logistic regression is concerned with verifying the probability of one event or another.

Multinomial Logit - Similar to the logistic regression, but in this case it is possible to predict a behavior, like preference of choice for a type of layout, product, arrangement of design attributes, but without having binary characteristics. You can infer the probability of choosing from different design combinations, that is, which one have the highest preference. An example would be to evaluate the choice of an interface by varying colors, type, illustration, formats and sizes, without using a simplistic A / B test approach. It would be an A / B / C / D / E / F test running simultaneously testing the preference for different arrangements. This approach looks like another tool called Conjoint Analysis.

Factorial Analysis - Using this tool you can group variables that are related to each other and analyze them together. Thus, instead of analyzing 20 different variables that describe the behavior of a group of people, it is possible to reduce the analysis to 4 or 5 factors only, in a similar approach to Cluster Analysis.

Matrix Factorization - Using five fold cross validation you can predict user preferences for design options that they have not yet evaluated, using only other people's available ratings.
So, the possibilities of integration between design and statistics are great. The goal is not to further eliminate the subjectivity of design, but to explore other sources of knowledge that are not just qualitative. They can be “predictive".

It is worth remembering that the use of inadequate statistical tools invalidates the information obtained, generating "speculative" knowledge, as in the case of Miles Tinker with his typography researches that used univariate methods, freezing some variables while varying only one.

Chandler, in his dissertation, presents several points of view concerning "research based on imagination" and the use of inadequate statistics:

"One should be careful to note the methodologies employed in previous research. Isaacs states that much of the research in this area is “speculative” (Isaacs, 1987). In fact, little of the literature regarding type design onscreen consulted for this review of literature was experimental in nature. Perhaps the current thinking about type design is based on historical preference, out of date information or conjecture. The lack of experimentally supported findings regarding multiple variables should underscore the need for this type of research. As early as 1931, Buckingham argued that much research regarding type in print was based on “imagination.” He noted that there are significant methodological flaws in the research and that investigators need a background in typography before undertaking this type of research. He also commented that the univariate model is particularly suspect and argues that multiple variables be examined simultaneously to more accurately understand the relationships between them (Buckingham, 1931). Another more recent critique of typographic research suggests that research needs to be as close to real-world conditions as possible and as similar to tasks completed by practitioners (Hartley & Burnhill, 1977).” – Chandler, S. (2001). Comparing the Legibility and Comprehension of Type Size, Font Selection and Rendering Technology of Onscreen Type. Dissertation. Virginia Polytechnic. Available on http://theses.lib.vt.edu/theses/available/etd-11172001-152449/ <http://theses.lib.vt.edu/theses/available/etd-11172001-152449/> 

My interest in statistics came from my recent studies on marketing and behavioral economics, areas that use inferential research very well to study social facts, something that designers also can do.

This has motivated my search for other researchers who already use a quantitative approach to investigate design issues.

Just my 2 cents. I would love to hear your perspective about this subject.

Regards,

Ricardo Martins



-----------------------------------------------------------------
PhD-Design mailing list  <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------