On 18 Feb 2008, at 12:00, John Smith wrote:
> This attempt/plan to tune and weight the outcome of metrics to mimic
> the previous "panel rankings in each discipline" begs the simple
> question: Were the panel rankings the best way to judge research and
> distribute these funds anyway?
Who knows. They certainly weren't "the only way", they might not have
been "the best way", but they certainly were "the way". An important
property of the RAE was that it had sufficient support to be
politically achievable. Whatever faults it had, at least it (judgement
by peers) was ultimately acceptable to all parties. Whatever replaces
it has to be acceptable to all parties, and that would seem to imply
that it has to be demonstrably similar to "judgement by peers".
> This is making the same mistake that was made when we moved from
> paper-based publishing to net-based publishing and carried across
> all the conventions and limitations of the old system.
I'd argue that we haven't yet moved :-)
> Would it not be better, now we have new forms of measurement, to ask
> what these measurements themselves are telling us about research
> quality or value or importance or whatever dimensions we choose to
> measure it on, and build a better (fairer?) form of assessment?
Or perhaps, before we change horses, we ought to work out what it is
that constitutes fair assessment in the current system, try to target
it through our new set of metrics, and check that we got it right.
> Just a thought :-) .
Just my tuppence.
--
Les Carr
|