Rosan:
> My attempt today is to find out about experiences on the streets. I
> would like to hear how RAE or other external research evaluation (in
> different countries) has promoted or stifled design research on
> individual or departmental level. Or to ask differently, what
> this kind
> of 'open' review has to offer or damage to the growth of design
> research.
I don't know whether the idea of open or closed is relevant in this case as both RAE and refereed journals are closed processes
with visible results. You know who is getting the recognition but you don't know how. With a Journal Paper you have the benefit
of being able to read it and form your own opinion and therefore an opinion of the journal's rigour. You can do that with RAE but
it takes a lot more detective work and it allows individuals to hide behind their department's record.
The really radical innovation is true open review as practiced by some of the newer open-access journals on the web.
Biomedcentral.com is probably the best example and I would recommend anybody who is interested in this subject to go there and
look at those journals that have an open review policy (each paper has a link called "history" I think) For each paper you can
see all early drafts, all referees' comments (referees are named) and all authors' responses.
To my mind this encourages responsible behaviour and a real debate that supports the development of a publication. It also allows
the audience to make a much more informed judgement of the paper since they can see how the issues unfolded and relate the
referees judgements to their own background and track record. For example I looked at an example where two referees disagreed.
One was a statistician and the other a surgeon. The difficulty was over a statistical question and one might trust the
statistician to make a good judgement about that, but it was also clear that the surgeon was taking account of issues of practice
that made the statistics less relevant than they might be (eg even though the statistical evidence for benefits was relatively
weak, you might still adopt the recommendation if you were satisfied it was cheap, easy and safe to do so.)
The main problem of the RAE is the long term implication of its decisions - you get several years to bask in the sunshine of a
success or suffer depression from failure. Plenty of opportunity for hubris and one holder of a 5* rating held a seminar called
something like "How to get a 5* rating in the RAE" In the next round they dropped two places in the ratings.
Best regards
Chris
****************************
Professor Chris Rust
Art and Design Research Centre
Sheffield Hallam University
Sheffield S11 8UZ, UK
+44 (0)114 225 2706
[log in to unmask]
|