I don't wish to vear away from Victor's thrust with starting this
thread and I would happily sign the petition you suggest.
But I feel I should respond to the assertions about 'problems of peer
review' at least with respect to Journals of my experience.
Some 'Editor handling of submissions' statistics should help quantify
such matters. These are a matter of public record re my IUCr Journals
submission handling statistics ie therefore not confidential and which
basically are:-
approx 1000 article submissions;
my rejection rate 20%;
appeals against my rejections 0.5%;
As Editor in Chief of Acta Cryst between 1996 to 2005 I received three
appeals (out of approx tens of thousands of submissions through all
Coeditors); I rejected these three. [My judgements were confidential
re the details.]
I can add that for the 2000 referees' reports or so for my article
handling of submissions, that colleagues have kindly supplied to my
Editor requests, problems involve:-
about 1% where the report is 'publish as is' AND without any
commendation given; these are in effect not terribly useful reports to
me as an Editor. Another problem, which is growing, is the number of
declines to my invites to referee (around 10%). Even worse are the no
replies at all from invited referees as time is lost to the authors
who rightly expect as prompt as possible handling.
Re your points I offer replies as follows:-
"Let me outline what I think are problems of peer review:
1. 'review by last author name'. Very often the last author is well
known, or a friend, and the reviewers' critical judgement takes a
temporary leave of abesnse.
JRH reply:- Such reports would be easy to spot and are not a problem
in my experience and so resort to double blind review is not necessary
in my experience.
2. 'preferred reviewers'. a double edged sword .. think about it.
JRH reply; these are not so commonly offered suggestions by authors in
fact and where they are one can follow or decide against (see point
1).
3. too much power of decision on editors (professional or academic)
being able to reject papers without peer-review in many journals.
JRH reply;This approach, 'insufficent general interest' is for the
magazines we know and yet still love.
4. Bad refereeing - sometimes I wonder if people read the paper.
JRH reply;Such reports are very few and obvious. The other categories
above are more common (ie 'publish as is' category).
5. Lack of referee expertise: you get papers these days with: a
structure, some biochemistry, some SAXS, some biophysics, and a cell
based assay. Two or three people being
able to pick up all the mistakes is very unlikely.
JRH reply; Papers can be challenging re content and your example here
is a good one. Other chalenging cases are where they include a lot of
maths. That said peer review does its best but can occasionally fail;
this level of failure can be measured by the number of criticism
articles or formal retractions. These are also very few, but it is
true, not zero.
Yours sincerely,
John
On Thu, Nov 18, 2010 at 10:47 AM, Anastassis Perrakis <[log in to unmask]> wrote:
>
> On Nov 18, 2010, at 11:18, James Stroud wrote:
>
> The future of publishing will be
>
> (1) Publish your own work
> (2) Peer review by the entire community
>
> Although I have been remarkably bad at predicting the future, I still like
> attempting to do so ...!
> This will not happen ...! ;-)
> To be honest, I am not even sure its a great idea ...
> Let me outline what I think are problems of peer review:
> 1. 'review by last author name'. Very often the last author is well
> known, or a friend, and the reviewers' critical judgement takes a temporary
> leave of abesnse.
> 2. 'preferred reviewers'. a double edged sword .. think about it.
> 3. too much power of decision on editors (professional or academic) being
> able to reject papers without peer-review in many journals.
> 4. Bad refereeing - sometimes I wonder if people read the paper.
> 5. Lack of referee expertise: you get papers these days with: a structure,
> some biochemistry, some SAXS, some biophysics, and a cell based assay. Two
> or three people being
> able to pick up all the mistakes is very unlikely.
> Having outlines these, I can see ways that all can be amplified if you just
> publish all your work, and anybody can comment on it:
> Pairing to the above problems, you just amplify them:
> 1. Even more tempting to earn brownie points online!
> 2. you can ask your friends or I can ask your enemies to review
> 3. the other way around: far too many things out... how to filter ?
> 4. Lack of 'obligation', or even fear to make yourself look like a fool to
> the editors, will make commenting even more sloppy
> 5. People that think they are experts dwell on meaningless technicalities.
> Peer review is like democracy, its the worst publication system we can have,
> except the ones that have been tried or suggested ...
> A.
>
> (3) Citation = Link
>
> #3 makes it work.
>
> Give it 25 years. The journals won't be in the position to lobby lawmakers
> to prevent this trend if we make sure the journals die so slowly that they
> don't realize it.
>
> James
>
>
> On Nov 18, 2010, at 1:14 AM, John R Helliwell wrote:
>
> Dear Jacob,
>
> Your posting reminds me of a Research Information Network Conference I
>
> went to in 2006 in London.
>
> Your views coincide with a presenter there, Peter Mika.
>
> His talk can be found at:-
>
> http://www.rin.ac.uk/news/events/data-webs-new-visions-research-data-web
>
> In his talk he referred to:- openacademia.org
>
> Peter Mika and I were on the Closing Panel; he advocated that
>
> refereeing is an imposition on a researcher's
>
> individual freedom and thus he/she should 'publish' their work on
>
> their own website. By contrast, I argued in favour of
>
> Journals and peer review, both with respect to my articles and my
>
> experiences as an Editor of more than one Journal.
>
> I would be happy to continue corresponding on this not least as
>
> publication should be a varied spectrum of options.
>
> Also I feel obliged to say that one cannot apply simply, by rote,
>
> 'Learned Society publisher is good', 'commercial publisher is bad';
>
> there are exceptions in both camps. [in effect this was the tone of my
>
> last posting.]
>
> Greetings,
>
> John
>
> On Wed, Nov 17, 2010 at 8:13 PM, Jacob Keller
>
> <[log in to unmask]> wrote:
>
> I guess the practice of being "on your best behavior" is good in terms
>
> of getting the research trimmed into shape, but there is a huge
>
> temptation to fudge things to get published, and to hide unpleasant
>
> artifacts, as can be seen by the many recent (and not so recent)
>
> scandals. Maybe as a lab website things would be more open. Also,
>
> having a comments section always seemed like an excellent idea to me,
>
> even for journals as they are, but would be really easy to implement
>
> in a website. I would love to read comments from others in the field
>
> about the papers I read, as sometimes people can help to point out
>
> gaping holes where one might not see them otherwise. It would be like
>
> "journal club" for the whole scientific community.
>
> Jacob
>
> On Wed, Nov 17, 2010 at 2:08 PM, Jrh <[log in to unmask]> wrote:
>
> Dear Jacob
>
> Re journals out of the window:-
>
> Well, like democracy, journals may not be ideal but I believe other
> alternatives such as free for all personal website publishing, are worse.
> So, journals that are community driven offer an optimal approach, critically
> based on specialist peer review. That is why our community effort IUCr
> Journals I believe are so important. Open access, where we can sustain it
> financially, also can convey access to the widest readership ie that the
> high impact magazines currently, mainly, command.
>
> All best wishes,
>
> John
>
> Prof John R Helliwell DSc
>
>
> On 17 Nov 2010, at 18:28, Jacob Keller <[log in to unmask]>
> wrote:
>
> Supplementary info seems to me to be a double-edged sword--I just read
>
> a Nature article that had 45 pages of supplementary info. This means
>
> that you get a lot more for your money, but all of the methods and
>
> Why not have papers be as long as the authors want, now that almost
>
> everything is internet-based? It would make the papers much more
>
> organized overall, and would obviate the reference issue mentioned in
>
> this thread. To avoid them being too too long, reviewers could object
>
> to long-windedness etc. But, it would definitely make for a more
>
> complete "lab notebook of the scientific community," assuming that
>
> that is what we are after.
>
> Incidentally, I have been curious in the past why journals are not
>
> going out the window themselves--why not have individual labs just
>
> post their most recent data and interpretations on their own websites,
>
> with a comments section perhaps? (I know there are about a thousand
>
> cynical reasons why not...) One could even have a place for
>
> "reliability rating" or "impact rating" on each new chunk of data.
>
> Anyway, it would be much more like a real-time, public lab notebook,
>
> and would make interaction much faster, and cut out the publishing
>
> middlemen.
>
> JPK
>
> On Wed, Nov 17, 2010 at 11:48 AM, Phoebe Rice <[log in to unmask]> wrote:
>
> Another unfortunate aspect of this sort of editorial policy is that many of
> these papers contain almost no technical information at all, except for the
> supplement. I've started to avoid using Nature papers for class discussions
> becuase they leave the students so puzzled, and with a
> glossiness-is-all-that-matters idea of science.
>
>
> =====================================
>
> Phoebe A. Rice
>
> Dept. of Biochemistry & Molecular Biology
>
> The University of Chicago
>
> phone 773 834 1723
>
> http://bmb.bsd.uchicago.edu/Faculty_and_Research/01_Faculty/01_Faculty_Alphabetically.php?faculty_id=123
>
> http://www.rsc.org/shop/books/2008/9780854042722.asp
>
>
> ---- Original message ----
>
> Date: Wed, 17 Nov 2010 17:12:26 +0000
>
> From: CCP4 bulletin board <[log in to unmask]> (on behalf of John R
> Helliwell <[log in to unmask]>)
>
> Subject: Re: [ccp4bb] Citations in supplementary material
>
> To: [log in to unmask]
>
> Dear Victor,
>
> I strongly support the stance that is in the Acta D Editorial.
>
> Manfred Weiss worked very hard assembling those details and over quite
>
> some time; he deserves our thanks.
>
> Greetings,
>
> John
>
>
> On Wed, Nov 17, 2010 at 4:06 PM, Victor Lamzin <[log in to unmask]>
> wrote:
>
> Dear All,
>
> I would like to bring to your attention the recent Editorial in Acta Cryst D
>
> (http://journals.iucr.org/d/issues/2010/12/00/issconts.html), which
>
> highlights the long-standing issue of under-citation of papers published in
>
> the IUCr journals. The Editorial, having looked at the papers published in
>
> 2009 in Nature, Science, Cell and PNAS, concluded:
>
> 'almost half of all references to publications in IUCr journals end up being
>
> published in the supplementary material only... Not only does this mean that
>
> the impact factor of IUCr journals should be higher, but also that the real
>
> overall numbers of citations of methods papers are much higher than what is
>
> reported, for instance, by the Web of Science'
>
> Although this topic may seem to concern mostly methods developers, I think
>
> the whole research community will only benefit from more fair credit that we
>
> all give to our colleagues via referencing their publications. What do you
>
> think?
>
> Victor
>
>
>
>
> --
>
> Professor John R Helliwell DSc
>
>
>
>
>
>
> --
>
> Professor John R Helliwell DSc
>
> P please don't print this e-mail unless you really need to
> Anastassis (Tassos) Perrakis, Principal Investigator / Staff Member
> Department of Biochemistry (B8)
> Netherlands Cancer Institute,
> Dept. B8, 1066 CX Amsterdam, The Netherlands
> Tel: +31 20 512 1951 Fax: +31 20 512 1954 Mobile / SMS: +31 6 28 597791
>
>
>
>
--
Professor John R Helliwell DSc
|
