|
Dear Colleagues,
Have received some interesting off-list notes. Was thinking about
them overnight.
This post will shed light on some of the issues directed to my
attention in emails and in a useful telephone dialogue with Chris
Rust.
(1) Exhibitions as research reports
Exhibitions have many purposes, most of the valid.
I want to distinguish among kinds of exhibitions.
Alec's post proposed the exhibition of research outcomes. My post
discussed only ONE category of these.
I support exhibitions for most purposes. Education and learning are
two of these. Beauty, pleasure, and entertainment are three more, all
valid.
I ALSO support exhibitions as a form of research demonstration.
When exhibitions are research reports, they must fulfill all the
criteria that research reports are required to fulfill. They must:
1. State the research problem,
2. Discuss knowledge in the field to date,
3. Discuss past attempts to examine or solve the problem,
4. Discuss methods and approach,
5. Compare possible alterative methods,
6. Discuss problems encountered in the research,
7. Explain how the researcher addresses those problems,
8. Explicitly contribute to the body of knowledge within the field,
9. State implications for future research.
Any exhibition that does this constitutes a research report.
Some forms of exhibition may not be full research reports, but rather
research-related demonstrations. This occurs when a surgeon
demonstrates a new technique. Demonstrating a surgical technique has
one purpose for medical students, another for practicing surgeons,
and yet a third when it is being conducted as part of a research
program.
A design artifact may be exhibited in a limited context that is not
intended as a research report. This is perfectly reasonable.
My argument ONLY concerned those cases in which an artifact or
exhibition is presented AS A RESEARCH REPORT.
Let us clarify this further. In some cases, I do more than ACCEPT
exhibitions as research reports. In some cases, I would REQUIRE them.
If an exhibition is the best way to demonstrate data in a research
report, the report requires it.
In what cases would a research report REQUIRE an exhibition to meet
all criteria of the research report?
This can be seen in an expanded statement of criterion 8.
In some cases, an explicit contribution to the body of knowledge
within the field requires demonstrating a wide range of data, placing
them in practical context, and showing how they function within the
practical context. When this is the case, an exhibition is often an
effective and economical way of presenting the research process, and
it may be an effective way to present the findings.
The difficulty is the aspect of a research report for which
demonstration or exhibition is appropriate is of a different nature
than the other eight criteria.
Core research findings are usually found in the explicit
contribution, including some of the direct aspects of process
description.
The CONTEXTUAL description of process and problems are found in
criteria 1 through 7. The description enables the field to
problematize and reflect on the core research issues. It is in these
activities that researchers are able to understand what the
researcher did and how the research approached the specific program
described in criterion 8.
After all the rest is done, criterion 9 distinguishes useful research
from mere study.
The difference between useful research and mere study is simple.
Every day, millions of students around the world perform experiments
in hundreds of fields and disciplines. These experiments help
students to understand and master the principles of a field. They
range from chemistry experiments and geometry experiments, to writing
experiments and design experiments.
These are experiments in the sense that students do not yet know or
understand the full range of outcomes and consequences. They are NOT
experiments in the sense of scientific research. They do not provide
answers. They do not even provide well-understood problems. These
experiments involve problems, issues, and answers that the field has
solved long ago. These have no implications for future research.
In contrast, any serious research report has some implication for
future research, no matter how small.
Consider the case of replication studies, for example.
A replication study repeats an earlier study or experiment. In many
cases, replication studies tell us with similar results exactly what
the earlier study told us. Even so, every good replication study adds
to the body of knowledge.
Why?
First, few studies are as comprehensive, complete, or broad as they
might be in an ideal world. In natural and social science both,
replication studies may involve studying the same objects of inquiry
in the same way to establish that earlier results were reliable or
credible, or to question them. They may also involve variations,
extensions, or differences, some major, others minor. A scholar or
scientist who is willing to further develop or expand the work of
another scholar contributes to the knowledge of a field by extending
studies. In social science, these may involve extending studies to
wider areas, different populations, or larger samples of the same
populations. There are many good reasons for replication studies,
though these are relatively rare in world where scholars and
scientists seek an original contribution.
The point is that replication studies, even plain and unremarkable
replication studies with close similarity to earlier studies, have
implications for future research.
Learning experiments do not. In contrast, their outcomes are so
predictable that the teacher - and in some cases, the student - can
use predictable results as a measure against which to test the
outcome of the experiment. This is why student experiments do not
count as research.
The same criterion applies to all research. A research result that
has NO IMPLICATIONS has nothing to offer the field. This is another
way of saying that the field has nothing to learn from the research.
Put in still a third way, this means that there has been no original
contribution to the knowledge of the field.
This is related to the confusions often seen on the PhD degree.
All successful research makes an original contribution to knowledge,
great or small. One of the criteria that make the PhD a research
degree is the concept of an original contribution to knowledge.
Without an original contribution to knowledge, there is no research.
There are many kinds of doctorate. The REEARCH doctorate makes a
research contribution.
This is a contrast with the practical doctorate or the professional
doctorate. For these, the candidate demonstrates the highest level of
professional application in applying current knowledge. This is why
these are degrees in professional practice rather than research
degrees.
One distinction between exhibitions in general and exhibitions in
research involves BOTH the original contribution to knowledge and the
explicit description of implications for future research.
Unless the artifact or exhibition explicitly states the implications
for future research, it is not a research result. It may be raw data
for research, but it is not in itself a research result.
Exhibitions are valid as research reports when they meet the criteria
of research reports.
Exhibitions are valid as research results when they meet the criteria
of research results.
My post did not argue against exhibitions or artifacts in research
reports. It argued FOR their valid use.
2) Including exhibitions and artifacts in proceedings
In much the same way, I support the VALID inclusion of exhibitions
and artifacts in proceedings.
It is important to remember what is refereed in refereed proceedings.
What the referees vet is the research.
Referees review papers primarily for central research issues. They
may ALSO offer suggestions on formatting, presentation, literary
style, referencing technique, and a dozen other qualities, but these
are irrelevant if the research is flawed.
Artifacts and exhibitions are ALREADY included in the proceedings of
many fields.
This is so because no one in those fields is confused about the
distinction between the artifact and the research.
The artifact or the exhibition is included in one of several forms.
This includes pictures, images or simulations on a CD-ROM that
accompanies the proceedings, images or simulations on a CD-ROM when
the proceedings IS a CD-ROM, advanced simulations on computer systems
or Web sites linked to the Proceedings CD-ROM, and soon, DVD. For
special conferences and seminars, I have also seen videos and even
artifacts packaged with a paper proceedings volume.
Once we begin to speak of recording or documenting the artifact, we
have already begun loving in the direction of common sense. Common
sense tells us that the artifact or the exhibition is NOT a
self-explanatory research result.
In past debates, the complaint about using books or journal articles
or thesis volumes to document design research that yields artifacts
has been that these documents are a step away from the immediacy of
the supposedly "self-explanatory" artifact.
Now we are told that IF the artifact is elf-explanatory, it is
permissible to document it.
This both steps toward common sense AND it reveals the inadequacy of
the notion of the self-explanatory artifact.
If the documented is external to the artifact itself, then the
document means that the artifact is no longer self-explanatory. Of
course it never was.
Let's just ask that the document include what every other research
report includes:
1. Statement of the research problem,
2. Discussion of knowledge in the field to date,
3. Discussion of past attempts to examine or solve the problem,
4. Discussion of methods and approach,
5. Comparison of possible alterative methods,
6. Discussion of problems encountered in the research,
7. Explanation of how the researcher addresses those problems,
8. Explicit contribution to the body of knowledge within the field,
9. Implications for future research.
Once we have done this, we are talking about the same thing.
If an exhibition does this, we are in complete agreement on the value
of an exhibition as a research report.
3) The difference between documentation and a research report
One of the unfortunate confusions in some design research circles is
the notion that documenting something makes it a research report. As
Nigel Cross noted, these kinds of confusion are even greater in fine
art circles and in the craft world.
Documenting something does not make it research.
A research report is characterized by the nine criteria stated here.
A document may simply record raw data. When a document records a
specific artifact without setting it in the research context, it is a
record of raw data.
Documenting a painting or a design artifact does not constitute
reporting a research outcome. An image of a painting is nothing more
than an image of a painting - it has the same characteristic as a
document whether the image is recorded by the artist who made it or
by another recorder five centuries after the painting is complete.
Since neither image addresses or clarifies the research problems
involved in the creating the artifact, neither document is a research
report. The painting itself is not a research outcome. While the
painting may be the practical outcome of a research process, it is
not a self-explanatory research result.
The same holds true of a knife or a Japanese sword.
At some point in recent years, the notion began that documenting a
process renders it viable as research. If you consider the criteria
that define research, you will see this cannot be so.
There MAY be other criteria. If there are criteria under which the
simple act of documenting or recording an artifact renders that
artifact research, I would be happy to consider them.
I would like to see them stated in explicit terms so I can do so.
A refereed conference proceeding involves research reports. Any
document that fits appropriate research criteria would obviously be
considered research.
A refereed conference proceeding normally documents research. The
reason that the document constitutes a research document is that it
meets criteria. The research output is qualified as research because
it is a research result, not because it is documented.
4) Portraying a process and outcomes
Different forms of narrative media have a role in portraying process
and outcomes. This is why computer simulations are making such a
difference in many fields. The ability to simulate or model the
processes determined through advanced equations is an important aid
to research and to reporting research.
Simulations are becoming common in anthropology, sociology,
epidemiology, medicine, mathematics, physics, chemistry,
biochemistry, cosmology, astrophysics, engineering, pharmacology,
nanotechnology, biology, and dozens of other fields and disciplines.
These are used to report research results. Just as figures and
illustrations are sometimes called "exhibits," these simulations are
exhibitions in the context of a research report.
They demonstrate moving parts, evolving systems, changes of physical
state, ongoing processes, transformation, and other kinds of complex
process.
This is part of the description or narration of the subject field
that constitute the content of research inquiry.
Every research narrative has at least two levels of narration. One
involves the object of inquiry that forms the content of the
research. The other involves inquiry into the process of research
itself. This is a metanarrative.
The research metanarrative involves narrating research process issues
that lie outside the inquiry itself. This includes
1. Stating the research problem,
2. Discussing the knowledge in the field to date,
3. Discussing past attempts to examine or solve the problem,
4. Discussing methods and approach,
5. Comparing possible alterative methods,
6. Discussing problems encountered in the research, and
7. Explaining how the researcher addresses those problems.
The research narrative involves all those issues that
8. Explicitly contribute to the body of knowledge within the field.
This is where researchers demonstrate and exhibit aspects of the
process under study.
A full expansion of this narrative and a description of the forms it
can take would be an article in its own right.
All of the specific internal aspects of the research inquiry would be
found here, and here is where evidence, cases, illustrations,
examples, process demonstrations, and artifacts would be put forward.
After the substantive portion of the research inquiry and the
statement of results, we return to the metanarrative.
We engage in metanarrative when we state implications for future research.
The narrative describes and portrays activities, processes, and
objects in the external world. This is why different forms of
communication can reveal and explain the research inquiry. In some
cases, these explanations are better than words or numbers.
In contrast, the metanarrative is a thinking process that takes place
in the mind of the researcher. Some aspects of the metanarrative are
seen in the larger literature of the field. In some cases, the
metanarrative occurs in the minds of other researchers to be reported
by the author of the research report.
Metanarrative is a process of individual thought and social
communication. While we do not think exclusively in words, we
describe thought in words and symbols. Some metanarrative issues
allow pictorial or numerical modeling. The metanarrative as a whole
requires description. Descriptive narration generally involves words.
The reason that there can be no self-explanatory artifact is that no
artifact can narrate the metanarrative.
I described this issue implicitly in some aspects of my earlier post.
My conversation with Chris Rust on the issue of narration helped me
to frame it more explicitly, distinguishing between narrative and
metanarrative.
There may be more than two levels of narration in some forms of
research. Some forms of research would involve narration of research
content, metanarration of the research process, and a second level of
reflective metanarration on the researcher's engagement with the
process.
Still further levels of metanarration may be possible.
It is likely that all or nearly all levels of metanarration require
words. It is conceivable that some forms of metanarration may involve
images or exhibitions in addition to words.
The attention given to captioning, model making, charting, and
presentation techniques in such books as Anholt's (1994) book on the
art of oral scientific presentation or Todoroff's (1997) book on
scientific presentation skills involve questions of narrative and
metanarrative.
There is no question that visual media other than words help to
communicate the content of science. This is the core theme of Edward
Tufte's series of books (Tufte 1974, 1983, 1990, 1997). Thee can also
support the metanarrative description of research process in some
cases.
At the same time, narrating the narrative and narrating the
metanarrative require words. This becomes particularly clear in
presenting seminars and talks (Anholt 1994: 37-85, 151-184; Feibelman
1993: 27-38; Todoroff 1997: 89-113).
Images, simulations, and demonstrations may communicate the research
content at many points. The overall research report requires
metanarration, and metanarration requires words. Without words, it is
impossible to communicate research.
Neither a single object nor an exhibition can manage the
metanarrative without the support of clear, written text.
Since an exhibition can contain many kinds of communication,
including text, an exhibition MAY include enough material to enable
the exhibition as a whole to be self-explanatory in every sense.
This may also be true of a CD-ROM, a DVD, or a videocassette if the
text is incorporated into the video.
A document of a process alone probably would not work.
Best regards,
Ken
References
Anholt, Robert R. H. 1994. Dazzle 'em with Style. The Art of Oral
Scientific Presentation. New York: W. H. Freeman and Company.
Feibelman, Peter J. 1993. A Ph.D. Is Not Enough. A Guide to Survival
in Science. Reading, Massachusetts: Addison-Wesley Publishing Company.
Todoroff, Cindy. 1997. Presenting Science with Impact. Presentation
Skills for Scientists, Medical Researchers, and Health Care
Professionals. Toronto: Trifolium Books.
Tufte, Edward R. 1974. Data Analysis for Politics and Policy.
Englewood Cliffs, New Jersey: Prentice Hall.
Tufte, Edward R. 1983. The Visual Display of Quantitative
Information. Cheshire, Connecticut: Graphics Press.
Tufte, Edward R. 1990. Envisioning Information. Cheshire,
Connecticut: Graphics Press.
Tufte, Edward R. 1997. Visual Explanations: Images and Quantities
Evidence and Narrative. Cheshire, Connecticut: Graphics Press.
--
Ken Friedman, Ph.D.
Associate Professor of Leadership and Strategic Design
Department of Leadership and Organization
Norwegian School of Management
Visiting Professor
Advanced Research Institute
School of Art and Design
Staffordshire University
|
