Dear Rosan,

Thanks for your reply. Sorry it has taken me so long to respond … I
have been thinking about your questions for the past few days, and
trying to write something that properly addresses the nuances of
significant questions.

Perhaps I should have explained the reason I was laughing. You asked
one of the great questions in research on human behavior in groups and
organization. Your impression that the authors of case studies often
explain success using the same factors that explain failure is often
correct. I’d have to see how authors describe and deploy the factors
to see if they are really the same in any given case, but as a general
situation, this often seems to be a problem for case studies in
business, management, innovation, design, and other fields. The problem
is somewhat different in engineering, medicine, and those fields of
professional practice that involve testable propositions in which we can
isolate variables through repeated testing and meta-analysis.

In my earlier response, I attempted to say that cases in many of these
fields generally represent a complex web of contextual variables and
process issues. This is the case for companies such as Microsoft, Apple,
Google, or Toyota; products such as iPad, Blackberry, the Dyson hand
dryer, or the Kindle; and broader innovations such as telephones, light
urban railways (trams), railroads, or automobiles. All of these involve
companies, products, or innovations that emerge against an historical
background. Once they emerge, they change the world around them in some
way, great or small. This makes the case a one-time-only, historically
contingent process. It is impossible to run the case again to isolate
one factor, run it yet again to isolate another factor, then run it
through further iterations in an effort to find out what factors matter
most. We can learn a great deal from history and from cases, and – and
Kathleen Eisenhardt argues – we can form robust theories.
Nevertheless, this is different to the kinds of theory construction
possible in the natural sciences or mathematics and the relatively
greater certainty theories represent in such fields.

Engineering and medicine differ slightly to other case-based fields. In
engineering and medicine, experiment, replication, and meta-analysis
permit us to learn a great deal more from cases than we can learn in
world history or design. While we can understand the scientific and
technological factors of an engineering problem, that still won’t tell
us how human beings will react to the technology. We can define criteria
that allow us to determine which technology functions best with respect
to our criteria, but these criteria won’t tell us which technology
will sell best. In most cases of management, innovation, or design,
methodological triangulation is the best we can do.

Scholars in management studies have been seeking the answer to your
questions for a long time. So have scholars in other fields. These
questions are a general problem in many fields and in those areas of
philosophy of science that apply to the social sciences, political
science, management, economics, and other fields in which the factors
that explain success can also explain failure.

Several challenges render this problem more difficult. These kinds of
cases involve complex ranges of variables operating on different levels
of analysis. Many variables are contextual, exogenous to the company and
to the case. Given the difficulty of accounting for complexity and
external issues, it may be quite reasonable that the same factors appear
in success and failure both. In some cases, we may think we are looking
at a situation in which

(x + y + z) [explain] success 

in one analysis while

(x + y + z) [explain] failure

in another. 

When we account for complex variables and context, we may instead be
looking at a situation in which additional variables come into play.
These additional variables may not be immediately visible, they may be
hidden, or they may bear some relation to the case that simply does not
emerge either in the research or in the analysis. When this happens, 

(x + y + z) + (variables d, g) [explain] success

in one analysis while

(x + y + z) + (variables t, q) [explain] failure

in another account. In these cases, the same factors don’t account
for success and failure both. Rather, a web of interaction within a
complex adaptive system involves the same factors occurring in different
contexts.

In such cases, the problem may arise from the nature of the field
rather than arising from flawed research or poor conceptualization. For
this reason, the same factors in relatively similar circumstances can be
deployed in very different explanatory patterns. 

But there is more to it, and using the same factors to explain success
or failure may indicate flawed research or poor conceptualization.
That’s what leads to scholarly and scientific debates.

This leads to the second point. Researchers do sometimes seek to read
phenomena in a way that supports their viewpoint. This may be purposeful
and cynical. It may involve honestly held but misinformed belief. In
some cases, it’s a matter of struggling to understand what things
mean, making reasonable mistakes while moving toward increasingly better
explanations. In some cases, research struggle honestly and responsibly
with facts but simply gets things wrong. That’s what makes reasoned
argument from evidence so important. That is also why it is important
for any research field to develop a rich peer-reviewed literature in
which people examine the issues, working arguments out through
reflective, considered inquiry. Fields with a rich array of robust
research methods do better than fields that simply go on intuition or on
practice.

Asking deep questions and finding ways to test these questions can
produce important new understandings. The field of behavioral economics
is a case in point. Economics has long been classified as a behavioral
science. Even so, many twentieth century economists treated economics as
a branch of mathematics describing behavior among purely rational actors
who compete in perfect markets where all information is equally visible
to all actors. If these assumptions were correct, there would be
difficult to account for bubbles, booms, or busts, and such phenomena as
arbitrage would make little sense. Imperfect markets or missing
information become visible even in cooperative markets without effective
coordination. The “beer game” demonstrates this problem in supply
chain management and logistics. To see a great example of the
coordination problem, go to the Beer Game Portal

  http://www.beergame.org/

Economics is made of hundreds of such problems, and mathematical
analysis has not solved some of these problems.

At one point, psychologists began to ask deep questions about economic
behavior. Next, they found ways to test the deep questions they were
asking. Some of these tests are both simple and rigorous. Over the past
few decades, behavioral economics overturned a great deal of what
economists in the past treated as established wisdom.

Of course, economists have been at this a long time, nearly three
millennia all told. The influence of psychology and rigorous testing on
economics is quite recent, though some of the great early economists –
Guan Zhong, Xenophon, Aristotle – understood that economics involved
the behavior human beings, as did Adam Smith and David Hume.

All analysis is ex post facto. Some research, however, is oriented
toward future states, and design research – like engineering research
or medicine – involves analysis to design improvements that we can
test. 

At this point, I suspect I’m beginning to shift from a response to
your questions, moving into a response to Derek Miller’s post. I’ll
come back another time.

For now, I’ll simply say that the issues you address are serious,
genuine, and deep. There are ways to develop robust research. Some are
quantitative, some are qualitative, and fields such as design,
management, or innovation seem to do best with methodological
triangulation.

Yours,

Ken

Professor Ken Friedman, PhD, DSc (hc), FDRS | University Distinguished
Professor | Dean, Faculty of Design | Swinburne University of Technology
| Melbourne, Australia | [log in to unmask] | Ph: +61 3 9214 6078 |
Faculty www.swinburne.edu.au/design

--

Rosan Chow wrote:

—snip—

I am glad that you were amused and thanks again for replying. Although
it will take away the amusement, it is useful to contain the general
question in the context which it arises. To repeat: the context was my
reading of popular and professional accounts of the success of Apple
under Jobs. I wanted to know whether the impression I got from the
reading was correct and whether this impression represented a more
general problem among researchers who study successful business /
management / innovation / engineering / design cases. So my question is
more specific than you have taken it to be. 
 
My impression was that there was a tendency for the journalists,
bloggers, or even researchers to read or use the success of Apple to
support their theory or point of view.

I am aware of the values and difficulties of case study research and I
know Nonaka’s work a little bit. And precisely because of this
background knowledge, I was even more struck to find that despite the
theoretical discussion and careful analysis (which of course sets apart
his paper from other more casual commentaries), this particular paper of
his left the same kind of impression mentioned above.

He probably has done more work since to substantiate his theory of
innovation management and I am not at all questioning his theory (which
I actually like but this is not the point). I am curious: in this
particular paper he used the cases studies to support his theories
without, in my judgment, the kind of robust argument that you said
needed for an ex-post facto analysis. For a positivist account of good
theory building from Case Study research, I have located this paper: 

 
 
However, my focus is not on evaluating Nonaka’s paper, but rather on
my impression stated above. I would be happy to hear that my impression
is not correct and there is no problem at all in the research on
successful cases and Nonaka’s paper was written this way because it
was at the beginning of a theory building or whatever …. I am completely
open ... but I would appreciate some pointers.

—snip—