Hi,
I'm one of the lurkers of this list;-), but additionally there is also a lot of litterature within the field of usability. In our courses we have made use of "Usability Testing" by Rubin. There is of course many, many other books on this subject......
 
/Charlotte

________________________________

Från: PhD-Design - This list is for discussion of PhD studies and related research in Design genom Terence
Skickat: fr 2008-03-07 01:28
Till: [log in to unmask]
Ämne: Re: rating designs



Dear Peter,

Sounds an interesting project.

 Rather than the apparently easy way out of trying to find 'a method of
evaluation approved in design research', I feel you would find it much
cleaner to tackle it as an ordinary research  problem.

This would entail you asking and answering:

1.      What you are assessing? The design; the design as actualized;
particular aspects of the design/actualized design; the design in use; the
designer's abilities; the user's ability in specific areas of design; the
designers' ability to work in a team; design management, quality of design
process; specific qualities of the functioning of the design (including
emotional/semantic effects); qualities of the presentation..
2.      Who you are assessing? The designer; design team; the lecturer's
abilities to teach; the users' abilities to use; the skills at being able to
assess.
3.      Why you are assessing? Undergraduate; honours; professional skills;
to demonstrate your skills at research..
4.      Against what framework you are assessing? Academic skills,
professional skills; design field skills; guild standards.
5.      From what perspective are you assessing? Aesthetic, constructivist;
efficacy; functionalist; technical skills; execution skills.
6.      The purpose of the experiment from the experimenter's point of view.
7.      Which field(s) the design work is undertaken? Graphics; mass
communication; interaction; mechanical engineering; civil engineering;
advertising..

Assessment of design 'quality' is not straightforward and can easily miss
the target.

For example, I was recently given an undergraduate student PowerPoint
submission as a proposed standard for assessing a Masters of Design
submission. The student PowerPoint was excellently done. It contained really
first class computerised renderings of a product and had achieved high
marks.
In terms of the Masters program, however, it would be inappropriate and have
low marks or even fail. Masters has a different target of learning compared
to undergraduate. It is not ' a better standard of more of the same'.
Student designers learn the craft of design as an undergraduate and
undertake professional formation and skill enhancement in the workplace in
years following graduation. When they return for Masters, the focus is
primarily on gaining the advanced thinking skills and mental models that
enables advanced design production and management. It is assumed the Masters
students are competent in basic design skills. Hence, the irrelevance of the
first class undergraduate artwork. At Masters level, a submission would be
expected to indicate the student had a suite of strong professional
reasoning and analysis skills relevant to design and design management. The
initial assumption as to the standard and type of assessment missed the
target.

Once you know the exact answers to the above six questions, it is then
straightforward to choose appropriate research methods. Any sound university
text on research methods would provide the information. You could try
'Research Design and Methods: A Process Approach (7th edn)'

Terry
===
Dr. Terence Love
[log in to unmask]
Tel: +61 (0)8 9305 7629
Fax: +61 (0)8 9305 7629
Mob: +61 (0)4 3497 5848
===


-----Original Message-----
From: PhD-Design - This list is for discussion of PhD studies and related
research in Design [mailto:[log in to unmask]] On Behalf Of Peter
Scupelli
Sent: Friday, 7 March 2008 1:38 AM
To: [log in to unmask]
Subject: rating designs

Hello there,

I am thinking of running an experiment in which I will have participants
make a design. Some participants would have design guidelines, others a case
study, and others yet nothing. The design task is simple in nature and
involves placing an information artifact in a public location where
different groups of users can access it.

I was interested in measuring the quality of the design solutions. Can
someone on the list please indicate papers, books, journals, magazines,
conferences, or methods that deal with the evaluation of designs? Are there
some agreed upon methods in the design research community?

Thanks,
peter

--------------------------------
Peter Scupelli
PhD Student in Human-Computer Interaction Carnegie Mellon University