Dear Colleagues,

below are a list of questions and answers that were raised at the SCUDD 
Conference and could be of interest to you.

best wishes

Paul

Dear SCUDD list,

Tom Maguire sent me a list of questions as we agreed at the SCUDD 
conference. Below are some answers. These have been considered by all 
members of Sub-Panel UoA65 and the Chair of Panel O.

RAE 2008 UoA 65 Frequently Asked Question

As you can see below, most questions are answered by a careful and 
detailed reading of the appropriate Criteria. The Panel and the Sub-Panel 
worked extremely hard to try to ensure that these statements were clear, 
accurate and above all transparent. Please read the Criteria.


·        Is quantity (e.g. word length or duration) regarded as a measure 
of the quality of any individual output submitted?

No - this exercise is an assessment of the quality of research.

·        Is the place (e.g. journal or venue) in which an output is 
disseminated a factor in rating its quality?

No. Panel O Criteria (4: 18) says 'Research will be assessed where it: has 
been published, exhibited, performed, recorded, screened or broadcast 
during the publication period; and can be judged against the assessment 
criteria...with these conditions in mind the sub-panel will assess the 
intrinsic quality of research wherever and however it is undertaken, and 
whatever form its output.'

·        How are different elements rated against each other, e.g. books, 
journal articles, book chapters, performances?

The Criteria are very clear about this - there is no rating given for 
kinds of publication. However 'significance' and (perhaps) 'rigour' may 
well be enhanced in a more substantial output.

·        Are book reviews submissable as outputs or as measures of esteem?

It is unlikely that a book review would constitute 'research' as outlined 
in the HEFCE/RAE definition of research as detailed in Appendix 3 of the 
Criteria. Nevertheless, a substantial review article may well incorporate 
a significant research element. This must be a matter for the researcher 
to judge (and the Sub-Panel to assess)

·        How will the Panel resolve the tension between the hierarchical 
rating of national and international and the difficulties of work within 
what  might otherwise be categorised as 'area studies' where the work is 
specific to a given geographical region?

Published in an international context or in another language are not 
necessarily criteria for research being considered 'international'. 
Research may well have its focus within a very particular region (for 
example a study of Welsh popular performance 1880-1920). However, its 
international significance may lie in its attempt to articulate a 
relationship between popular/populist cultural activity and the 
development of economic power in the entertainment industry. In addition, 
it may well propose new theoretical and methodological models. In that way 
it may be a major point of reference for work in that field (3*) If it is 
or likely to be an essential point of reference than it may achieve 4*. 
Area or regional studies of performer training in the Asian sub-continent 
may similarly achieve such status.

·        How will future funding decisions by the AHRC and other funding 
councils relate to achievement in RAE 2008?

The Panel and Sub-Panel have no better judgement than anyone else. There 
is no published information on this. For obvious reasons, it is unlikely 
that a funding algorithm will be made public until after the exercise is 
complete.

·        What is the view of the Panel on decisions by institutions to 
predict the scores of individuals or Units of Assessment in advance of 
submission?

It is the right of any HEI to make such guesses and try to predict 
results. The final report of the RAE2001 celebrated the rich diversity of 
research in our subject and remarked upon the range of research 
methodologies and outputs that the subject submitted. However talking with 
colleagues in a wide range of departments, it seems that many HEIs are 
taking submission decision out of the hands of the department and making 
their own judgments. Where this is the case conservatism appears to 
dominate. This means that, for example, many HEIs still look with 
considerable suspicion upon submissions other than printed, published 
outputs. Our subject association has worked extremely hard since the 
RAE1992 on behalf of research in our subject, especially in achieving as 
clear an articulation of Practice as Research as possible. Nonetheless, it 
seems as though this kind of action by HEIs may well overturn one of the 
most positive features of RAE2001. Please note the repeated welcome 
offered by both Panel O and the Sub-Panel for the submission of all types 
of research '...whether produced through writing, making, composing, or 
performing' (Panel O Criteria, 24: p. 21). Panels and Sub-Panels will 
undergo training and undergo a lot of work and discussion over the next 
months in order to work out precisely how they will convert the prose of 
the descriptors into the graded profile.

*       Do all staff submitting need to have four outputs?

The Criteria are very clear on this - especially concerning those new to 
research and those to whom special circumstances apply. (Panel O Criteria, 
33-6: 22-3)

·        Is it true that staff employed by an HE for the first time in 
2006 or 2007 are able to submit only one output without being penalized 
with a 0 rating for the second?

Please read the criteria carefully (Panel O Criteria, 34: 23).

·        How will the Panel judge interdisciplinary outputs?

The criteria articulate our methodology (Panel O Criteria, 30-2: 22). The 
design of the panel structure (the grouping of similar disciplines as sub-
panels with a main panel) was undertaken in order to facilitate 
interdisciplinary submissions.

·        In the last RAE the information box was used to provide these 
things
among others: place and dates of performance, collaborators on the project,
list of further outputs directly related to the main output, types of
evidence of the output available to assessors. The 300 words was a separate
text containing a statement on the research aims and achievements of the
output. In the coming RAE, should the factual information box and the 300
word statement be used in the same way as last time?

The Sub-Panel Criteria (13b: 46) give a very clear account of the use of 
the 300 word statement.

*       What criteria will be applied to judging evidence for the 
dissemination of practice-based outputs?

The evidence will inevitably depend upon the nature of the practice: 
programmes, web-sites, interviews and broadcasts, conferences and 
symposia, supplementary publications, articles and contributions to 
publications. The list may be long, but the researcher should read the 
HEFCE/RAE definition of research (Criteria Annex 3) and assure themselves 
that their findings and research have been appropriately located within 
the public domain.  As indicated above, Panel O Criteria (4: 18) 
says 'Research will be assessed where it: has been published, exhibited, 
performed, recorded, screened or broadcast during the publication period; 
and can be judged against the assessment criteria...with these conditions 
in mind the sub-panel will assess the intrinsic quality of research 
wherever and however it is undertaken, and whatever form its output.'

·        Has specific documentation been requested equivalent to ISSN or 
ISBN numbers that shows independent evidence of the 
performance/screening/exhibition - i.e. its place, date, etc?

No, this is, in part, the purpose of the portfolio or evidence box of 
material (Sub-Panel Criteria, 13c: 47)

·        How will Practice-as-Research be treated in relation to point 29b 
of the Generic Statement, p 10, in particular, in those cases where the 
researcher has not previously submitted to the RAE? May PaR works be 
deemed to require a different judgement in relation to the matter 
of 'entry into the public domain' to books and articles, since 
multiple 'issues' - in the RAE terminology - are intrinsic to the nature 
of performance-based and related practices, and may be important as 
indicators of the significance and impact for the work? Moreover, for 
example, PaR works whose achievement in the public domain is outstandingly 
in the RAE2008 period may have had preliminary 'issue' just prior, in 
2000, but their eligibility under this point seems unclear.

The Generic Criteria in para. 29b seems to be quite clear. Members of the 
Sub-Panel will use their judgment to identify the research significance of 
the outcomes of the research that may properly be dated within the RAE2008 
submission period. Clearly, they cannot grade work that falls outside the 
assessment period. Again, Panel O Criteria (4: 18) says 'Research will be 
assessed where it: has been published, exhibited, performed, recorded, 
screened or broadcast during the publication period; and can be judged 
against the assessment criteria...with these conditions in mind the sub-
panel will assess the intrinsic quality of research wherever and however 
it is undertaken, and whatever form its output.'

·        Regarding the application of Generic Statement point 28 to PaR 
work - the case seems clear for a written work, but less clear for some 
PaR. Will the same view be held to apply where the work is both scripted 
and produced as an artifact by the same person, and where the script is 
separately issued as a publication? The two processes are normally seen as 
independent works.

You may write a play and submit it as a published output. You may submit a 
directed production of that work as a separate output. Members of the Sub-
Panel would use their judgment to assess and grade each individual 
submission. In this case the artist submitting the research would need to 
find ways (through the 300 words and the portfolio) to clearly articulate 
the distinctive (albeit connected) research imperatives of the two 
submissions.

·        Can you confirm that the RA5 narrative is not supposed to be a 
self-assessment, but that it is, rather, a combination of description and 
evidence?

The RA5 documents are not a press release for the department's research! 
The Sub-Panel Criteria describe the potential contents of this document 
clearly. (Criteria, 15-27: 47-50). Your research may well be 'innovative, 
ground-breaking, cutting edge, internationally renowned ...etc', but it is 
the job of the Sub-Panel to make those judgment! Watch the adjectives and 
save words in the RA5!

·        In the case of a performance or series of performances that have 
generated, for example, a journal article or other published form of 
documentation, what code is recommended (i.e. that for 'performance' 
or 'article', etc.)?

In RAE2001, much of the most exciting PaR work represented an iceberg in 
which the actual submission chosen for inclusion in the RA2 represented 
its tip. The 300 words, whilst articulating the nature of the research, 
also drew attention to the body of the iceberg that may well be evidenced 
in the portfolio/ evidence box.

·        In the case of edited collections in which the author has also 
contributed a chapter, will the panel welcome submissions in which the 
volume as a whole is entered as an output, or will they expect only the 
contributed chapter to be entered?

There is no especial expectation. This is a research assessment exercise 
and grading will be done accordingly. Many edited works involve 
considerable research, for example the editing of classic and/or texts of 
an insecure nature involving working with primary sources such as  
manuscripts and marginalia, uncovering the archaeological traces of 
textual transmission, comparing and contrasting different editions, 
establishing a clear evaluative methodology to account for variant 
readings, writing a detailed account of the history of the text and its 
reception, arguing a particular position regarding choices etc.; whilst 
others may involve more scholarly administration than research. The 
researcher should make the judgement. But do remember that judgements will 
not be made solely upon the framing of the RA2.  Note 'The detailed 
examination of research outputs will be based entirely upon the members' 
direct engagement with those outputs cited in the submission...' (Sub-
Panel Criteria, 45: 52)

·        Should all outputs be entirely within the discipline of Drama (or
media). I'm thinking of work on Beckett, for example, that might cross
poetry and fiction, Brecht (poetry) or Friel (short stories) as well as
drama.

The domain of drama, dance and performing arts is, by definition 
interdisciplinary and it is a quality that is frequently celebrated within 
our community. The Sub-Panel tries to reflect this in its UOA descriptor 
and boundaries (Sub-Panel Criteria, 4: 45).

·        Is the 'Other Details' section in RA2 only for practice as 
research
outputs? Can it be used to contextualise published outputs?

Sub-Panel Criteria, 13b: 46 clearly say that research outputs '...may be 
submitted alone where it is deemed to constitute sufficient evidence of 
the research in itself'.  A published output would normally expect to 
fulfil this. Nevertheless, the function of the 300 word statement is 
clearly articulated in 13b: 46 of the Sub-Panel Criteria - in 
particular, '...where the research imperatives and the research 
significance of an output ... might further be made evident by a 
descriptive complement.' Avoid a self-assessment.

·        How will the panel conduct its assessment of practice-based and 
practice as research so that it achieves the Chair's aim of 'allowing the 
practice to speak for itself' and 'respecting the integrity of the 
artistic product'?

Whilst recording is not the same as the performed event, this nonetheless 
places a significant responsibility upon the researcher to find ways 
(through carefully edited, well indexed, good quality DVD etc) to enable 
the Sub-Panel members to understand and appreciate the nature of the 
practice.

·        How will the panel go about assessing work published in foreign 
languages, or which cites extensively in foreign languages.

The 300 word statement may be used to provide a short abstract of the 
work - sufficient to help the Sub-Panel consult with appropriate 
specialist advice. The HEI intentions to submit have identified the range 
and nature of such specialist advice.

Christopher Baugh
Chair RAE2008, Sub-Panel UoA65