On Sat, 27 Nov 1999, Prof. Tom Wilson wrote:

> sh> it is not the referees who cost money, but the
> sh> implementation of the refereeing.
> 
tw> I totally agree with the last point - but I wonder if high 
tw> submission, high cost journals are the norm? I referee regularly for 
tw> five or six journals and in all cases the papers for review come 
tw> directly from the editor rather than from the publisher, so I suspect 
tw> that for many journals (and, given a probable Bradford/Zipf 
tw> distribution for submissions to journals, those with thousands of 
tw> submissions must be a very small minority) it is the editor's 
tw> institution that is bearing the cost rather than the publisher - so, 
tw> once again, academia is subsidising the publisher and perhaps this, 
tw> rather than the $300 a paper for the JHEP is the norm. 

It is correct that academia subsidises all phases of refereed research
publication -- from conducting the research, to writing it up, to
refereeing it, to subsidizing journal editorial offices and functions.

But even so, there are some residual costs. In some cases I agree that
those costs are small enough to be borne completely by the editor's
institution, but those are not the problem journals, nor the expensive
ones, nor the high submission-volume ones, nor the high-impact ones --
in short, those are not the journals that are holding back the freeing
of the refereed journal literature at the moment. It is the ones with
the nonzero quality-control implementational costs that are the
problem -- but the problem is small: $300 per paper is small enough to
make people realize how absurd it is to hold the whole literature
hostage to it at the reader-institution end, when it makes so much more
sense to pay it at the author-institution end, out of 20% or less of
the very savings that doing so would generate!

tw> The case of scientific societies is rather different, since they often
tw> make the journals available to their members at rates well below the
tw> commercial and the whole activity takes the form of scientific
tw> collaboration.

But at the cost of keeping the papers behind a financial firewall for
everyone else (and even the cost to members is non-zero)...

tw> It seems that the scholarly community is still not completely persuaded
tw> of the virtues of freely accessible self-archiving (although I am
tw> persuaded - and have been since the idea was first mooted) - so there
tw> are at least some quarters of the community that need to be persuaded.

I agree, of course! The point was not that there isn't still a lot of
persuading to do regarding the virtues of self-archiving; it was about
the fact that the VALUE of self-archiving is already demonstrated,
whereas the value of modifying peer review (and how) is most definitely
not demonstrated, hence whether there is anything to persuade anyone
about in that regard is moot.

tw> Perhaps more difficulty is involved in persuading the universities that
tw> action of this kind is necessary - in spite of the economics of the
tw> situation there appears, at least in the UK, to be a kind of
tw> institutional blindness to the possibilities of reform, of which
tw> self-archiving is one.

I agree (and no one can say I'm not doing my bit to try to lead the
academic cavalry to the waters of self-archiving as well as to get them
to drink!)...

tw> And the institutions are likely to have something to say in the matter,
tw> given the emerging awareness of their stake in intellectual property -
tw> some Universities may decree that, in certain areas, work is of such
tw> commercial significance that their stake must be protected.

Tom, this too has come up repeatedly in this forum (mostly from Joseph
Ransdell in connection with Steve Koonin and the Provosts' initiative).
I recommend a little reflection on this. Just as it was highly
instructive, indeed essential, to make the critical distinction between
the give-away and the non-give-away literature (roughly, journals vs.
books) in order to see the light about self-archiving, so it is
essential not to confound the case of patents, software piracy, etc., in
which universities indeed have a stake, with the case of refereed
research publication.

Here is a simple algorithm: If the paper's NOT one that would have been
published in a refereed journal in the Gutenberg Era, than that is NOT
the kind of paper we are talking about here! Things that would have
been held back from publication for the sake of patents or commercial
exploitation, etc. are on the other side of the give-away line, and
always have been. Even books (of all kinds, from specialist monographs
to wide-spectrum popular books to textbooks) are on that side of the
line; some universities might decide they want a cut in the royalties
because the research was done on their time.

I plead nolo contendere to all of that because it has NOTHING to do with
the kind of literature I am talking about: The papers that appear in the
refereed journals have one value, and one value only: their research
impact. That is what promotes their authors' careers, and their
institutions prestige and research subsidies. That is why
publish-or-perish and impact-factors are the daily bread of the
Academy.

And that is why it is absurd, simply absurd, to conflate the
proprietary intellectual-property (and intellectual-property-sharing)
concerns relevant to the non-give-away (potential royalty, patent,
commercial sale) sector with the free-for-all logic and ethos that
prevails in pure research, and research publication.

The universities want to stem the drain on their resources to journal
publishers: they don't want to take over their business!

sh> But what advocates of "peer review reform" have mostly tended to do is to
sh> promote untested, notional alternatives (such as open commentary, or no
sh> review at all) to the scholarly community. I think that is not very useful
sh> at all.
> 
tw> But, of course, as you say, we have no empirical evidence as to 
tw> whether it is useful or not.

That is methodologically incorrect. We have currently a
quality-controlled literature that has a demonstrated value (otherwise
we wouldn't be paying for it, and wouldn't be trying now to free it).
If you have the hypothesis that the quality-control has been
superfluous, the burden of proof is on you (just is it is on the
Creationists)...

sh> Besides, peer review reform has absolutely nothing to do with the
sh> movement to free the refereed journal literature...
> 
> tw I entirely agree - the issues are completely separate

Then why are you raising them in this context at all?

tw> Well of course the CURRENT citation record is based on a peer-
tw> reviewed literature, because there is nothing else, and until there 
tw> is something else no testing is possible.

You proposed citations as an alternative to peer review. Apart from the
fact that they come rather too late to help me to decide how to navigate
in unfiltered sewage, growing daily (and immediacy is rather important
in many areas of scholarship and science), the extrapolation of the
evaluative power of citations is based on citations of refereed work. It
is not only an empirical question what the quality of the literature
would be without refereeing, but it is an empirical question whether
citations would still have any evaluative power in such a literature.

Testing is always possible; but I think we agreed that it has nothing to
do with the issue at hand, viz., the freeing of the current literature,
such as it is, namely, a peer-reviewed literature, not something else,
some hypothetical unfiltered variant of it, retroactively validated by
some hypothetical variant of refereed citation. (Even today, the
citation of unrefereed papers is done sparingly, and treated with some
scepticism.)

sh> If, as I think is likely,
sh> quality plummeted with the elimination of formal quality control in
sh> favor of opinion polls, no one would have any idea what to make of those
sh> opinions, whether they came in the form of comments or citations.
> 
tw> Well that, i.e., plummeting quality, is as much a guess as my guess 
tw> that it would make very little difference - neither of us have any 
tw> data and the data are not going to be available without experiment.

But the burden of proof, given that the current literature is a
quality-controlled literature, is on you; the default assumption is that
the quality-control is not just a placebo effect, any more than
water-filtration is.

> sh> neither expert nor novice would know how to sort the serious
> sh> from the sewage in the vast unfiltered flow that would confront us all
> sh> daily.
> 
tw> We still have that problem with the filtered product, since the 
tw> tendency is for more and more titles to emerge as publishers find 
tw> gaps in the market - the market either of papers to be published or 
tw> potential subscribers to be satisfied. The difficult thing is that no 
tw> matter how far down the 'food chain' of journals one goes, one still 
tw> cannot rule out the possibility of something of value being found - 
tw> and the literature is full of cases of early discovery being 
tw> overlooked and research being expensively repeated because the search 
tw> (if indeed any was conducted) did not go far enough or deep enough. 

All true, but not an argument against filtering. And freeing the
filtered literature online would not only make it immeasurably more
accessible, searchable and navigable, but it might even help to find the
wheat among the chaff and to root out duplication.

tw> I'm not advocating nor promoting anything - I am raising questions - 
tw> questions that, as you say, could be answered through empirical 
tw> testing. But as long as the focus is on ensuring the survival of the 
tw> commercial trade in scholarly communication rather on freeing that 
tw> communication *completely* from the profit motive the less likely it 
tw> is that any testing can take place.   

I do not see the connection there at all (and thought you yourself had
agreed earlier that the two issues -- freeing the literature from access
tolls and "freeing" it from quality controls -- were separate ones).

> sh> But if you have empirical data to bear on this, it is certainly
> sh> welcome; or even reasons why you think it is NOT an empirical matter.
> 
> As I note above, empirical data demands experiment and experiment in 
> this area requires, in some test field, at least a consensus in that 
> field that the experiment would be valid. And we can only establish 
> that commentary (or citation) could be a substitute for peer review 
> by experiment. 

I agree, but I still do not see the connection with the freeing of the
journal literature. Cannot experimental research on peer review reform
proceed apace, entirely independently of this -- indeed SHOULD it not do
so, rather than effecting any sort of contingency between a KNOWN good
(free access to the quality-controlled literature) and an entirely
UNKNOWN one (freeing the literature from quality-control)?

tw> However, there is a further point about filtering:  
tw> the acknowledged success of the Los Alamos archive raises the 
tw> question of how those thousands of users manage to cope with the 
tw> unfiltered dross it holds - before the cream (to mix a metaphor) 
tw> reaches JHEP? Perhaps they do it in the same way that referees judge 
tw> suitability for publication?

I tried to answer this in my last posting. Virtually all the preprints
self-archived in Los Alamos are simultaneously submitted to the refereed
journals -- of which there is a quality hierarchy, from the best at the
top, to something that approaches an unrefereed vanity press at the
bottom. Within 9 months, on average (Les Carr will soon report the data)
most of the papers in Los Alamos have been accepted in refereed journals,
and the published version is either swapped, added, or noted.

I am told that the amount of revision called for by the referees, and
the proportion of papers rejected at a given level of the Physics
journal hierarchy, is lower than in many other fields. This may or may
not be a relevant factor in why self-archiving caught on first in
Physics. That too is an empirical question.

But the answer to your question is that the "unfiltered" phase only
averages nine months -- and even that is constrained by, because
answerable to, the "Invisible Hand" of peer review for which all those
preprints are destined (and written knowing they are so destined). Hence
they are NOT the requisite empirical indicator of what would happen to
quality if the Invisible Hand of peer review were withdrawn altogether.

    Harnad, S. (1998) The invisible hand of peer review. Nature
    [online] (5 Nov. 1998)
    http://helix.nature.com/webmatters/invisible/invisible.html

sh> the colossal success of Los Alamos (of which
sh> the existence of JHEP is one of the consequences):
> 
tw> With that there can be no argument - but it seems to me to be at 
tw> least debatable whether the creation of a new print journal from an 
tw> archive is either necessary or desirable. 

JHEP is not a print journal. It is a completely free online journal,
with an astronomically high Impact Factor of 11, only in its third
year of publication. Arguably this would not have been possible without
the prior revolution induced by Los Alamos.

http://jhep.cern.ch/

tw> Let me make it clear that I fully support the idea of self-archiving; 
tw> nor am I opposed to refereeing - I simply ask whether it is *always* 
tw> economically in the best interest of the institutions that bear the 
tw> cost, which, in the majority of cases, I suggest, will be academic 
tw> institutions. However, given self-archiving, other strategies for 
tw> subsequent 'publication' will emerge. 

It is not the subsidizing of editorial offices that is breaking the
budget for universities, it is the subsidizing of publishers'
proprietary ADD-ONs to their own authors' give-away product. The
add-ons, no longer necessary, are print-on-paper and their distribution,
as well as their proprietary online counterparts. It is paying for
those add-ons that accounts for at least 80% of universities'
Subscription/Site-License/Pay-Per-View (S/L/P) budgets.

Yes, new strategies will emerge as a consequence of freeing the
literature through self-archiving. One would be to phase out the
obsolete add-ons that account for 80% of the price of journals, and
continue keeping them behind a financial firewall, but at 20% of their
prior price. But a better alternative is to do away with the financial
firewalls altogether, and pay for the only essential service still being
rendered, quality-control/certification QC/C up-front, as a publication
charge to the author-institution, paid for out of 20% of the 100% S/L/P
savings, and leave the distribution and preservation to the Open
Archives (another minuscule subsidy to research from the author's
institution).

http://vole.lanl.gov/ups/ups.htm
http://cogprints.soton.ac.uk/

tw> In many small fields of scholarly endeavour it is possible that the 
tw> archive itself may be all that that community needs (for example, I 
tw> think that there are only about six research centres world-wide that 
tw> carry out work on bees, honey and apiculture); but the community may 
tw> decide that some form of journal presentation (most probably 
tw> electronic) is also needed - refereeing may be employed in some 
tw> cases, in other cases editorial selection may be thought sufficient, 
tw> and in other cases, perhaps in emergent disciplines or research front 
tw> areas, peer commentary will be preferred. In other words, given the 
tw> archive of papers, the solutions to the problems of scholarly 
tw> communication may vary according to the influence of a number of 
tw> factors. In areas like high energy physics, the JHEP solution may be 
tw> the norm, but we cannot advocate that solution as a general solution, 
tw> since it seems clear that not every field of investigation has the 
tw> same characteristics as HEP.

I completely agree. Which is yet another reason for decoupling the
issue of archiving from the issue of peer review reform: Yes, the Open
Archives will include not only the papers from refereed journals high
and low in the hierarchy of rigor and impact, but they will also
contain papers from unrefereed journals, experimental journals, and
papers submitted to no journal at all. They will, in fact, provide a
platform for trying out the experiments you advocate.

All that's needed to get there from here is for authors to go ahead and
self-archive all their papers; and for them to be able to do that, Open
Archives need to be available to put the papers into -- both central
archives, on the Los Alamos model, and distributed, University-based
archives, all of them interoperable and hence integrable into one
seamless, global virtual archive, free for one and all, forever.

I will soon be meeting with Rick Johnson, the director of ARL's SPARC
initiative, which is dedicated to lowering the S/L/P prices of journals
-- but would perhaps not be averse to doing away with S/L/P altogether
-- about ways to facilitate the adoption of (free) Open Archive
software and hence the installation of interoperable Open Archives by
all member universities.

http://www.arl.org/sparc/index.html

tw> Self-archiving will have a major impact on the mores of scholarly 
tw> communication but, just as the pattern is varied at present, so it is 
tw> likely to be varied in the future and no one solution will apply 
tw> everywhere.

I agree that the OUTCOME of self-archiving may differ in different
fields, but I am fairly confident that self-archiving itself is the
optimal strategy for all fields; hence, eo ipso, it will have the effect
of freeing the literature. How the economic model and the
quality-control model evolves as a consequence, is, as they say, beyond
my immediate remit...


--------------------------------------------------------------------
Stevan Harnad                     [log in to unmask]
Professor of Cognitive Science    [log in to unmask]
Department of Electronics and     phone: +44 23-80 592-582
Computer Science                  fax:   +44 23-80 592-865
University of Southampton         http://www.cogsci.soton.ac.uk/~harnad/
Highfield, Southampton            http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM           

NOTE: A complete archive of this ongoing discussion of "Freeing the
Refereed Journal Literature Through Online Self-Archiving" is available
at the American Scientist September Forum (98 & 99):

http://amsci-forum.amsci.org/archives/september98-forum.html




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%