Please find below a CFP on scientific evaluation by the online social science
journal 'EspacesTemps'. This critical francophone publication, originally
started by geographers & historians in the 1970s, now also publishes articles
in English. Various formats accepted, not only standard peer-reviews articles
but also shorter opinion pieces. (For the British on this list, this might be
an opportunity to take RAE-related musings to a wider audience...)

-----------------
Questioning scientific evaluation.
Call for paper

http://www.espacestemps.net/document1460.html

You are invited to submit texts for publication, on from September 2005, in a
“Traverse” of EspacesTemps.net. These texts can be personal accounts or
descriptive analyses, as well as theoretical statements and recommendations. To
begin with, you can from now on send your proposition to
[log in to unmask], in the form of an abstract of 2000 signs. You can
also suggest an evaluation mode of the text that you are addressing.


QUESTIONING SCIENTIFIC EVALUATION
Evaluation has taken on a major role in scientific production. By default, what
one calls ‘evaluation’ mostly refers to the quality assessment of just-produced
works — an assessment carried out within an evaluative structure considered as
independent of logics exterior to the interests of knowledge. In physics,
biology, in technical disciplines and, to a lesser extent, in mathematics as
well as in these of the social sciences which strive to follow the ‘hard
sciences model’, a ‘standard’ system has been set up during the last decades.
Although very simple in its principle, this system appears extremely powerful
in its consequences. It appears as a perfectly legible procedure: every
article, submitted to a journal is examined by referees designated by the
journal’s editorial board among the specialists of its field of research. It is
upon the judgment of these very referees that depends, in a mechanical way, the
decision to publish or not to publish.
One can formulate the hypothesis that the progressive success of this method has
had, as an effect, a hierarchical classification of journals according to their
legitimacy. As a result, emerged an effect of positive retroaction integrating
all of the evaluation devices within a coherent system. One can interpret its
logics in the following way. Firstly, an evaluation device, recognized for its
competency and equity, defines ipso facto the quality of a journal that
includes this device in its operating structure. The journals are then
themselves evaluated in the same manner by other evaluators. The occurrence of
bibliographical references to an article within other articles, measured by the
‘science citation index’, provides thereafter a quantitative means of
classification, according to the text’s impact factor, while taking into
account the reputation of each of the relevant media. The evaluation of
researchers and research teams — and, by extension, of departments, faculties,
and universities — can then be immediately deduced from their publication or
citation success within the best classified journals. This can result in an
automatic public or private financing of these very actors and reinforce their
capacity to produce results destined to be awarded by the evaluation device.
Lastly and above all, by its very success, this device has negative effects on
all excluded elements: on refused articles, on journals that do not subscribe
to the evaluation norms, on books, on unclassifiable publications and, even
beyond, on research activities that do not lead to productions publishable
according to these norms… Evaluation, starting with the evaluation of
‘scientific’ journals, appears thus as the organizing force of scientific
legitimacy itself of the entire world of scientific research.
This situation is doubly fascinating for researchers in social sciences.
Firstly, because they are increasingly concerned on a personal level. Secondly,
because it constitutes a fascinating object for their own investigations.
It is in this context that EspacesTemps.net launches its call for contribution
concerning the following theme:
A SCIENTIFIC EVALUATION?
CAN DYNAMICS OF KNOWLEDGE BE NORMED?
This general interrogation is articulated in a series of questions.
1.What and how is being evaluated and who evaluates? Beyond the judging of
submitted texts, one can question the evaluation of any production: PhD thesis,
science books, large public essays, research projects, research teams and
institutions, products not belonging directly to the scientific world
(university and scholar manuals, philosophy, architectural or urban projects),
extra-research activities of scientists… In this way, one can question the
scientific component of any kind of social production.
2. What are the strengths and weaknesses of the ‘standard’ system? Does it
contribute, in a positive way, to the scientific independence of scientists and
does it take the innovative aspect of their work into account? Do norms and set
up hierarchies represent a risk of ignoring the most original among
contributions? Do they represent a hindrance for interdisciplinary research?
Are there malfunctions within this system that alienate its objectives? Does
today’s journal market guarantee for plurality or, on the contrary, does it
unduly tend to impose doxas, due to its oligopolistic structure? Is there any
compatibility between the standard system and a dynamic editorial policy?
3. Can one fix evaluation norms independently from scientific criteria and their
dynamics? Or, on the contrary, shout one first define the latter in order to
deduce from them different modes of evaluation? Could one — and should one ––
evaluate questions rather than answers, conjectures rather then results, global
theories rather than punctual advances?
4. Is there a specificity of human and social sciences in regard of questions of
evaluation and evaluability? If so, what is this specificity and what is its
base? Are we moving towards a convergence of the evaluation modalities in all
of the research domains and disciplines?
5. Can a system of scientific evaluation be evaluated, by analysing its device
or the effects induced on scientific results? 
6. Are there alternatives to the standard system? Are there other modes of paper
production, (see, for example the ‘Public Library of Science’ project)? Are
there other types of evaluators outside of the academic world… deserving
attention? Does the emergence of new scientific media, such as the internet,
modify the relation between the evaluators and the evaluated? Can the
evaluation take into account research components appearing as peripheral but in
fact fundamental, such as epistemology, technological consequences, scientific
culture or citizenship? Can one imagine other actors than researchers
participating in the scientific evaluation process?
7. Must one evaluate science in order to make it innovative? By what criteria
can we tell that science is in progress? Are there means, other than
evaluation, by which one can make sure that scientific organisations respect
the ‘ethical contract of knowledge’ with society and that they make good use of
the means with which this society entrusts them? Does the intervention of
exterior actors, such as governments or of citizens, necessary lead to a
dangerous social control on research?
8. How can one integrate critical reflexivity on evaluation in evaluation
itself? What recommendations can one make so that the evaluation of research
work and that of the scientific life does, in fact, maximize innovation and
production of knowledge?
9. … And, by the way, are there suggestions to ameliorate the evaluation device
of EspacesTemps.net?



-- 
Dr Juliet J. Fall
Department of Geography
University of British Columbia
1984 West Mall
Vancouver, BC V6T 1Z2, Canada
Phone: (604) 822-9105 Fax: (604) 822-6150