Hi Don,
Many thanks again. Makes sense.
I think the assessment gets more interesting if the focus is whether
students have higher professional skills, rather than assessing the objects
they create.
Over the last couple of decades I've been working on portfolio-based
assessment of professional skills and have come to the conclusion (for the
moment at least) that a Commentary section is the key to making evidence
easier, faster and more reliable. It moves the assessment towards direct
assessment of professional *skills* and *knowledge* rather than indirect
assessment through interpretation of design outputs (things).
Most of the ideas are in Love, T. and Cooper, T. (2010). The Central Role
of Commentary on Evidence in E-Portfolios. In N. Buzzetto-More (Ed) The
E-Portfolio Paradigm: Informing, Educating, Assessing and Managing with
E-Portfolios. Santa-Rosa, California: Informing Science Press (pp. 267-288).
A pre-print is at
http://www.love.com.au/PublicationsTLminisite/2010/Commentary%20on%20Evidenc
e%20in%20ePortfolios.htm
If this was combined with computerised automated essay assessment using an
appropriate ontology, it might offer a cheap, easy and reliable assessment
method underpinning peer review of design outputs?
Are you envisaging using Rasch correction in the peer assessment?
Best wishes,
Terry
-----Original Message-----
From: [log in to unmask]
[mailto:[log in to unmask]] On Behalf Of Don Norman
Sent: Tuesday, 24 December 2013 11:20 AM
To: Terry Love
Cc: PhD-Design - This list is for discussion of PhD studies and related
research in Design
Subject: Re: MOOC on DOET: Wonderful or horrid?
On Mon, Dec 23, 2013 at 6:07 PM, Terence Love <[log in to unmask]> wrote:
> Would you please say more about assessment problems you found and how
> you feel they might be solved?
>
We dealt with the assessment problem in a really simple manner: there is
none.
Well, we do have what we call "quizzes," but I call "reflections," where we
ask simple questions and grade them as correct or incorrect. But these are a
pedagogical device to help the student pay attention. The answers are not
used for a grade.
We have projects, which in a real course, would be critiqued and graded.
here, we rely upon the Forum. Stduents post their projects and other
students respond. This is very active. The two instructors (Kristian and
me) provide comments, but with tens of thousands of students, we can not do
very much.
--
There are three solutions to assessment.
1. The peer assessment method that Scott Klemmer developed seems good, if we
could implement it. Students are trained to do assessments (by having to
match the assessments of sample problems against those of instructors), and
when they are certified, they assess other students.
2. Hire design faculty to do the assessment. I am pushing for this, but we
will have to charge for the course and set up new infrastructure to handle
this.
3. Do no assessment. People self-assess. They take the course for their own
benefit.
Note that we now do not offer any sort of certificate.
--
I suspect that 2 is the proper answer and am pushing for this. The course
cost could be low compared to a traditional course (we are aiming at around
US $100). This is under much discussion. We would offer a certificate of
accomplishment. (We also would need to do a full financial analysis to
figure out what the cost would have to be.)
Don
Don Norman
Nielsen Norman Group, IDEO Fellow
[log in to unmask] www.jnd.org http://www.core77.com/blog/columns/
Book: "Design of Everyday Things: Revised and
Expanded<http://amzn.to/ZOMyys>"
(DOET2).
Course: Udacity On-Line course based on
DOET2<https://www.udacity.com/course/design101>
(free).
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]> Discussion of PhD
studies and related research in Design Subscribe or Unsubscribe at
https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|