---------- Forwarded message ----------
Date: Sat, 30 Jun 2001 18:59:52 +0100
From: Stevan Harnad <[log in to unmask]>
Reply-To: September 1998 American Scientist Forum
<[log in to unmask]>
To: [log in to unmask]
On Sat, 30 Jun 2001, Bernard Lang wrote:
> > bl> why should the quality-control service be provided by publishers?
>
> sh> Because they are providing it now. And there is nothing wrong
> sh> with it (except the extras being forcibly wrapped in with it).
>
> "Because they are providing it now" is not an answer. My point is that
> we have to reanalyze the system from scratch, to see what would be the
> basic rules in the Internet world.
The basic rules for what?
Peer review is medium-independent. It is merely experts (peers)
giving feedback on the work of their fellow-experts (peers) to
a meta-expert (editor) in order to determine what needs to be done to
make it suitable for PUBLICATION (sic), which means,
(medium-independently) CERTIFICATION as having met the quality standards
of that particular journal.
Harnad, S. (1998/2000) The invisible hand of peer review. Nature
[online] (5 Nov. 1998)
http://helix.nature.com/webmatters/invisible/invisible.html
Longer version in Exploit Interactive 5 (2000):
http://www.exploit-lib.org/issue5/peer-review/
http://www.cogsci.soton.ac.uk/~harnad/nature2.html
http://documents.cern.ch/cgi-bin/txt2pre.sh?file=/archive/electronic/other/agenda/a01193/a01193s4t7/URLsonPeerReview.htm&style=cds
Peer review could benefit from some empirical study and the application
of the resulting findings (if any), but that has nothing at all to do
with the pressing issue at hand, which is the need to free the current
refereed literature, such as it is, from the obsolete access barriers
of subscription/license/pay-per-view (S/L/P), online.
The benefits of freeing access to this literature online are
self-evident and immediate.
http://www.cogsci.soton.ac.uk/~harnad/Tp/science.htm
The benefits (if any) of applying the results (if any) of empirical
tests of ways of improving peer review await the performance of those
tests, and whatever their outcomes turn out to be.
The freeing of the current refereed literature should neither wait for
nor depend on the eventual outcome of those tests. Indeed, it has
nothing whatsoever to do with them, and in my opinion it is a big
mistake to link them in any way.
Let peer review experiments proceed at their own pace. But, meanwhile,
let us free this literature (through eprint self-archiving) immediately.
http://www.cogsci.soton.ac.uk/~harnad/Tp/nature4.htm
> Refereeing is a function in itself, which is actually playing an
> increasing role in many areas, largely because of the internet and its
> dynamic interactive character, and in many guises. Typically, any site
> that lists and criticizes products, web pages (for example to tell
> whether it is obscene or violent, and how much), people, books
> (amazon), or software components, is performing a refereeing task.
All fine, and welcome, but completely irrelevant to the subject matter
of this Forum, which is not products, web pages, people, books or
software, but the REFEREED JOURNAL LITERATURE (20,000+ journals
annually). And that literature is ALREADY refereed. It does not need
re-refereeing, or a new form of refereeing. It needs freeing.
> A whole refereeing technology is currently being developed (using
> mathematical tools) because, of course, one has to assess the
> credibility of the refereeing services, either from an absolute (as
> much as that make sense) or from a personal viewpoint.
Splendid. And let that development and assessment proceed at its own
pace. But let it not be coupled IN ANY WAY with the much more urgent
task at hand, which is freeing the current refereed corpus online, now.
> So quality-control, or relevance to a given profile, will be more and
> more a general form of service on the internet. Among other things it
> will be applicable to scientific or litterary resources (paper, data,
> ...). It will be open to competition ... and publishing houses are
> welcome to compete.
Bravo. I look forward to these developments. But there is no reason
whatsoever to make the freeing of the current refereed literature
contingent on them in any way at all.
> But I doubt it will be the source of very high income, especially for
> those who are not willing to pay the referees, since most of the
> infrastructure can be mechanized.
Paid refereeing is another untested variant on peer review. Let it be
tested (for bias, for quality, etc.) and everyone will be interested to
hear the results. But for now, the proposition is that the true costs
of implementing peer review today (when referees referee for free, just
as researchers write their papers for free, not for royalty) need to be
paid. They are the essentials. The rest (paper, PDF) are options, and
must be sold as such, rather than being used to continue to hold the
former hostage to the latter:
http://www.cogsci.soton.ac.uk/~harnad/Tp/resolution.htm#4.2
Will the income from this downsized form of publication be enough to
keep today's refereed journal publishers interested in remaining the
implementers of peer review? The answer is that for some (especially
certain Learned-Society Publishers) it will be enough, and for others
it may not be -- in which case, when the time comes, those titles,
together with their editorial boards and referees and authors, can
migrate to other publishers for whom it will be enough. That's all
there is to it.
> > bl> My view is that they can provide it if they wish and manage to
> > bl> sell it. But it can actually be provided by any individual, any
> > bl> organization, who cares, for a fee or for free, with or without
> > bl> competence.
>
> sh> Of course. But those who wish to free the refereed corpus would
> sh> like to have it done with competence.
>
> And the best way to ensure competence is to have open competition.
We already have competence. It resides in the peer review of the
current, established journals. I have no idea what role "competition"
is supposed to play here, but the established form of competition
between journals is to establish their quality levels through rigorous
peer review, and thereby continue to attract papers of commensurate
quality.
Hypothetical variants on peer review (perhaps involving some sort of
competition among reviewers?) need to be implemented and tested
experimentally before there is any point even speaking about them.
At the moment, we have 20,000 refereed journals (refereed in the
classical way) to free online. Meanwhile, let the refereeing reform
experiments be conducted on the sidelines, until they have reliable
results to report and recommend.
> sh> What's wrong with the quality control we have now? And wouldn't new
> sh> quality-control methods have to be tested first? And what about the
> sh> freeing of the 20,000 while we are waiting for the outcome of the
> sh> test?
>
> It does the job, more or less, with cliques, schools of thought, and
> other human weaknesses.
Can we please separate whatever dissatisfaction we may have with the
current quality-level of the refereed corpus, from the dissatisfaction
we have with the fact that it is not freely accessible? They are not
the same thing, and one does not depend on the other in any way.
(I might add that in theorizing about hypothetical new forms of
multiple, competing refereeing, it would be advisable to bear in mind
that referees are a scarce, overused resource. This topic has been
discussed many times before in this Forum.)
> But we all know that the current system is far from perfect. A
> colleague of mine was barred from publication in a journal for nearly
> ten years, because he had made public a scientific fraud by a member of
> the editorial board. I once had to cover up for attempted unethical
> publishing so as to protect the victim (who had agreed) from the risk
> of further harassment by a powerful scientist.
>
> Let's not kid ourselves, the system is adequate, but far from perfect.
Far from perfect (just like all forms of human judgment). But until
there is a tried and tested alternative that has been shown to be AT
LEAST as good at maintaining the current quality level of the refereed
literature, there is simply no point alluding to these imperfections.
It is this "imperfect" corpus (of 20,000 journals) that we want to free
online, now. Further perfecting it is another matter, another agenda,
and there should be no contingency between the two.
> sh> Why should it be the publishers who implement the
> sh> quality control? Because they are the ones doing it already. And
> sh> whoever does it is the "publisher".
>
> bl> this definition does not leave much room for discussion, does it ?
Why should it? We are talking here about freeing this literature, not
reforming it.
> For me the publisher is the person who makes the work public. Not
> necessarily the person who assess the value. The link between the two
> is a Gutenberg era concept, due to economic constraints.
In the PostGutenberg Era, the "making public" becomes trivial: Everyone
with a website is a "publisher" in this sense. What matters is the
publication of literature of a reliable, established quality level, as
certified by the publisher's imprimatur.
Self-publication is Vanity Press. Caveat Emptor. Refereed Publication
is what we want to preserve (and liberate).
> and still, I do not agree that the quality-control should be
> centralized. Hence even your concept of publisher becomes somewhat
> diluted.
What is "centralization" here? We have the established journal, with
its reliable, known, quality-control "tag," its name, associated until
now with articles of a known kind and quality. The quality is assessed
by peers, who are not centralized. In the hands of a competent editor,
they will be the relevant experts from anywhere on the planet.
I think "centralization" is a red herring -- or at the very least,
needs to be empirically tested as a variable in the peer review
process.
> I never said the service should not be paid ... I only meant that, as
> far as I can tell, those who provide it, even now, are not what we call
> publishers. And I doubt it is (or will be) that expensive
> ... but I may be wrong.
No, you are right. It will not be expensive to implement peer review.
And the peers themselves review for free. That is why the estimate is
that peer-review implementation, per article, costs only 10% (about
$200) of what the planet currently pays for the average article in one
of the 20,000 refereed journals ($2000) (in the form of the collective
institutional S/L/P tolls paid by those institutions that can afford
access at all, to that article).
> > I don't think journal publishers are villains,
>
> no they are not ... well not any more than the silk workers in Lyon who
> were being replaced by automatic weaving machines 2 centuries ago (I
> guess the same happened with wool in the UK).
I think the analogy is unhelpful.
> The difference is that they (some of them, at least) are powerful, and
> they fight back more strongly, for example by lobbying at international
> level to get anti-human laws (I mean laws intended to privilege private
> companies economic interests against the interest of society and human
> freedom).
>
> And that makes them villains.
Perhaps some attempt to do that. But scholars/scientists are the ones
to blame. For we can already take the whole matter, completely legally,
into our own hands, and liberate this literature online overnight:
http://www.cogsci.soton.ac.uk/~harnad/Tp/resolution.htm#Harnad/Oppenheim
We should stop railing at publishers (and waiting for them to "reform")
and just do what is already within our reach. (That is why the proposal
is called "subversive"):
http://www.arl.org/scomm/subversive/toc.html
This vilification of publishers is simply distracting and polarizing;
it is not the direction in which any remedy lies.
> sh> and I don't think peer review needs to be changed (urgently).
>
> no ... but it is useful to assess where we are going, and what is
> meaningful in the long term.
Yes, but not in this context! The incorrect coupling of these two
independent matters is simply another needless retardant on our taking
the simple steps that will immediately usher in the optimal and the
inevitable outcome.
> > What is urgent is freeing the peer reviewed literature, such as it
> > is. Peer review reform is another matter, and empirical one, and the
> > two should not be coupled in any way.
>
> I agree to this, and certainly support this priority. It is politically
> wise to stick to the most urgent goals.
>
> But I will not let supporters of the past take argument of their past
> role in (simply organizing) quality assurance as a reason for
> maintaining the old system. We do not need them for quality assurance,
> though they are welcome to offer it, if they can do it competitively
> (in price and effectiveness). Like any other willing organization or
> individual.
At the moment, they are the ONLY ones doing it. No one wishes to simply
terminate quality control, and no one knows how to wrest it from the
hands of its current purveyors. Nor do we need to. This is all
irrelevant distraction. We should simply go ahead and self-archive all
of our papers (both pre-refereeing preprints and post-refereeing
postprints) in our institutions' distributed, interoperable,
OAI-compliant Eprint Archives (http://www.eprints.org).
http://www.cogsci.soton.ac.uk/~harnad/Tp/resolution.htm#4.1
The rest will take care of itself.
--------------------------------------------------------------------
Stevan Harnad [log in to unmask]
Professor of Cognitive Science [log in to unmask]
Department of Electronics and phone: +44 23-80 592-582
Computer Science fax: +44 23-80 592-865
University of Southampton http://www.cogsci.soton.ac.uk/~harnad/
Highfield, Southampton http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM
NOTE: A complete archive of the ongoing discussion of providing free
access to the refereed journal literature online is available at the
American Scientist September Forum (98 & 99 & 00 & 01):
http://amsci-forum.amsci.org/archives/september98-forum.html
You may join the list at the site above.
Discussion can be posted to:
[log in to unmask]
|
