Thank you Gerard for this comment!
Ylva
Prof. em. Ylva Lindqvist
Dept. of Medical Biochemistry & Biophysics
Div. of Molecular Structural Biology
KAROLINSKA INSTITUTET
Tomtebodavägen 6
S-171 77 Stockholm, Sweden
Phone +46 8 52487695
Mobile +46 708 264356
Fax +46 8 327626
Email: [log in to unmask]
URL: http://www.msb.mbb.ki.se/
________________________________________
From: CCP4 bulletin board [[log in to unmask]] on behalf of Gerard Bricogne [[log in to unmask]]
Sent: Monday, September 12, 2016 12:31 PM
To: [log in to unmask]
Subject: Re: [ccp4bb] Another puzzle: 5gnn: SOLVED
Dear Gert, Randy and colleagues,
After a weekend of 5gnn detox, I wonder if I should take any more
of everyone's time sharing some degree of disquiet about the direction
this thread has taken us into. I think we may be getting grossly ahead
of ourselves by starting to discuss the list of authors of a putative
redeposition.
Firstly, I didn't choose to raise the issue of this structure in
a message to the BB not in order to take the lead in a remediation
effort or fire a pistol to start a sprint towards it. This remediation
will correct one entry in 100,000+ and is in fact a micro-issue. The
main issue, and the one that made me decide to devote some time and
attention to this case, is that it brought into even sharper focus the
matters raised in the discussion that followed the correction of 4NL6
and 4NL7, namely: how could such a deeply flawed structure be (1)
obtained, (2) taken seriously by the authors of the paper to the
degree necessary to write it up, with a full display to the entire
world of its abysmal quality statistics and nonsensical geometry, (3)
approved by the reviewers, who clearly (like the Editor who appointed
them) cannot tell a protein structure from a hole in the wall, and (4)
accepted for deposition by the PDB in spite of the abundance of warm
colours in its validation report. These seemed to me to be the real,
substantial and community-wide issues raised by this structure, issues
that are no less than a systemic failure of the whole system we trust
to oversee the healthy growth of the structural knowledge embodied in
the PDB. Some partial answers to these questions are of course already
available, e.g., regarding (4), that it isn't the role of the PDB to
act as a policeman; but perhaps some fresh thought could be given to
finding a mechanism by which a "You can't be serious" warning could be
fed back to authors (and if disregarded, to the editors) in such
extreme cases.
Secondly, many responses to the 5gnn alert went in the direction
of suggesting a community representation to the authors, which would
have been the collegial way a treating them. It seems to me that this
should have taken the form of a notification that 5gnn had shown up on
the BB and was eliciting much surprise and discussion, giving them "an
opportunity to react to a surprising turn of events" by trying to fix
the problem themselves, perhaps in the light of the suggestions made
by members of the BB (hexagonal rather then trigonal symmetry, wrong
declaration of twinning, etc.). I am not sure that writing to the
corresponding author after having posted a first draft of a corrected
structure to the entire BB was the most considerate way of treating
colleagues in the absence of any indication of an unwillingness on
their part to correct it themselves. As for deliberating over their
heads about who should co-author their corrected result, that seems an
even more serious breach of collegiality: shouldn't they retain the
right to decide, or propose, something themselves? Depositing a rotten
structure should not necessarily lead to an immediate loss of respect.
In any case, this question of defining a code of conduct for
whistleblowers and correctors enlarges the discussion even further in
a very interesting way. Perhaps I should myself have written to the
corresponding author a week ago - but again the correction issue
seemed to me to pale into insignificance compared to the loud
diagnostic of dysfunction of "the system" it carried with it - and
that, I felt, justifed a post to the BB.
Apologies for yet another long e-mail: I hope it might redirect
the discussion towards the issues that really matter and had been
touched upon in the previous thread: what can we, and should we, do to
increase the level of crystallographic literacy among scientists who
use our methods and among editorial boards and refereeing panels of
journals that publish crystal structures, and to make the validation
reports (to the production of which so much effort has been, and is
still being, devoted) have more bite and more filtering power, at the
very least at the deposition stage, but hopefully as far upstream as
possible.
With best wishes,
Gerard.
--
On Fri, Sep 09, 2016 at 06:11:08PM +0100, Randy Read wrote:
> Dear Gert,
>
> It's good to raise those issues. I should say that I sent the coordinates to the corresponding author at the same time as the BB, and I have written to ask what steps he will be taking to resolve this. I think the authors should have an opportunity to react to a surprising turn of events before we as a community contact the editors, for instance.
>
> Best wishes,
> Randy
>
> ----
> Randy J. Read
>
> > On 9 Sep 2016, at 17:36, Gert Vriend <[log in to unmask]> wrote:
> >
> > Dear 5gnn fans,
> >
> > I guess it is a bit too early to really do this, but the whole 5gnn story calls for a discussion about how to publish the new coordinates. Who should now get the credit for them?
> > Further, is anybody telling the 5gnn article editor to discuss this with the (failing) referees? And has anybody checked that the depositors of 5gnn are members of this list, and if not has anybody made them aware of their novel star status in this forum?
> >
> > Gert
> >
> >> On 09/09/2016 06:30 PM, Tristan Croll wrote:
> >> Damn - beaten to the punch (my refinement's currently at 0.209/0.235) ;). There appears to be a rather beautifully resolved citrate in there (see attached)... (un)surprisingly, it's not listed in the crystallisation conditions.
> >>
> >> T
> >>
> >>
> >>
> >>> On 2016-09-10 01:31, Robbie Joosten wrote:
> >>> Hi Randy,
> >>>
> >>> That was brilliant (as was Gerard's decision to make the problem
> >>> public). I agree with Manfred that this should be relayed to the
> >>> authors and the editor of the journal. The model you posted is
> >>> obviously something that should replace the current PDB entry 5gnn.
> >>> I couldn't resist to have a play with your model. When you throw it
> >>> into PDB_REDO as-is you end up with R-free already at 23.9%. If you
> >>> add the missing 4 terminal residues before, that R-free goes down to
> >>> 22.1%. The output from that job is at
> >>> www.cmbi.ru.nl/pdb_redo/upload/5gnn_randy_redo.tar.gz. There is still
> >>> much more to do for the authors to make the model good. I wanted to
> >>> say publishable, but apparently it already was when it was still
> >>> grossly incorrect :(
> >>> For the validators: there is a very nice non-Pro cis-peptide if you
> >>> fix the model manually a bit more.
> >>>
> >>> Cheers,
> >>> Robbie
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: CCP4 bulletin board [mailto:[log in to unmask]] On Behalf Of
> >>>> Manfred S. Weiss
> >>>> Sent: Friday, September 09, 2016 13:40
> >>>> To: [log in to unmask]
> >>>> Subject: Re: [ccp4bb] Another puzzle: 5gnn: SOLVED
> >>>>
> >>>>
> >>>> Good job Randy!
> >>>>
> >>>> Now, in order to finish this off properly somebody should refine the
> >>>> structure, alert the editor of the journal as well as the corresponding author
> >>>> of the paper.
> >>>>
> >>>> If we leave it at that, the only thing we have is the satisfaction that we
> >>>> managed to "peer review" this structure, but nothing else is going to happen.
> >>>>
> >>>> Cheers, Manfred
> >>>>
> >>>>
> >>>> On 09.09.2016 13:34, Gerard Bricogne wrote:
> >>>> > Dear Randy,
> >>>> >
> >>>> > Congratulations for this terrific piece of work!
> >>>> >
> >>>> > As James Fraser was writing two days ago, this case establishes a
> >>>> > new paradigm for the "peer review" of structural papers :-) .
> >>>> >
> >>>> >
> >>>> > With best wishes,
> >>>> >
> >>>> > Gerard.
> >>>> >
> >>>> > --
> >>>> > On Fri, Sep 09, 2016 at 12:20:53PM +0100, Randy Read wrote:
> >>>> >> Dear all,
> >>>> >>
> >>>> >> Following on from Paul’s suggestion that the space group is wrong, I
> >>>> thought it might be interesting to try to solve this structure from scratch,
> >>>> using what I’m currently teaching students at a CCP4 workshop. What
> >>>> worked was to find some distant homologues with HHpred (top 3 hits after
> >>>> ignoring 5gnn), make a trimmed ensemble, and search with that. This gave a
> >>>> very clear solution for one molecule in P65. Model completion from that
> >>>> point works really well, given the resolution, using either phenix.autobuild or
> >>>> ARP/wARP.
> >>>> >>
> >>>> >> For the edification of the members of the BB, the ARP/wARP model is
> >>>> attached! (Hopefully no flames for an attachment of 135kB…) This model
> >>>> hasn’t had some really obvious rotamer fixes or extensions of the termini
> >>>> applied, but it has good stereochemistry, and R/Rfree are 0.241/0.285.
> >>>> >>
> >>>> >> Given the vastly improved statistics for a different space group and the
> >>>> huge differences in the model, it is to be hoped that the authors retract the
> >>>> original publication and PDB entry. Thanks to Gerard for pointing out the
> >>>> issues with this!
> >>>> >>
> >>>> >> Best wishes,
> >>>> >>
> >>>> >> Randy Read
> >>>> >>
> >>>> >
> >>>> >
> >>>> >>
> >>>> >>
> >>>> >> -----
> >>>> >> Randy J. Read
> >>>> >> Department of Haematology, University of Cambridge
> >>>> >> Cambridge Institute for Medical Research Tel: +44 1223 336500
> >>>> >> Wellcome Trust/MRC Building Fax: +44 1223 336827
> >>>> >> Hills Road E-mail: [log in to unmask]
> >>>> >> Cambridge CB2 0XY, U.K. www-structmed.cimr.cam.ac.uk
> >>>> >>
> >>>> >>> On 7 Sep 2016, at 16:51, Paul Adams <[log in to unmask]> wrote:
> >>>> >>>
> >>>> >>> Dear Gerard,
> >>>> >>>
> >>>> >>> thanks for pointing this structure out. This is indeed very startling. The
> >>>> paper indicates that the structure was refined with Phenix. However, if I
> >>>> download the model/data and refine (with Phenix) the starting R-factors are
> >>>> 0.36/0.38 and these get worse during refinement. Clearly the deposited
> >>>> model is not consistent with the R-factors reported in the paper, or in the
> >>>> wwPDB. In addition, analysis of the data suggests strongly that the true
> >>>> symmetry is P6 (possibly with a screw axis) - I suspect that this is the genesis
> >>>> of statements about twinning, but it isn’t clear if that was used in the
> >>>> refinement. Being charitable I can imagine that the wrong model was
> >>>> deposited in the wwPDB. However, this doesn’t explain the R-factors
> >>>> reported by the wwPDB, or the less than convincing images of the structure
> >>>> shown in the paper. I very much agree with you that there must have been
> >>>> ample alarms sounded along the way. It is cautionary that this wasn’t caught
> >>>> at some point. For me this highlights that the issues go beyond the naivety or
> >>>> impatience of a single student.
> >>>> >>>
> >>>> >>> Cheers,
> >>>> >>> Paul
> >>>> >>>
> >>>> >>>> On Sep 7, 2016, at 7:20 AM, Gerard Bricogne
> >>>> <[log in to unmask]> wrote:
> >>>> >>>>
> >>>> >>>> Dear all,
> >>>> >>>>
> >>>> >>>> While the thread on "Another MR pi(t)fall" is still lukewarm,
> >>>> >>>> and the discussion it triggered hopefully still present in readers'
> >>>> >>>> minds, I would like to bring another puzzling entry to the BB's attention.
> >>>> >>>>
> >>>> >>>> When reviewing on Monday the weekend's BUSTER runs on the last
> >>>> >>>> batch of PDB depositions, Andrew Sharff (here) noticed that entry
> >>>> >>>> 5gnn had been flagged as giving much larger R-values when
> >>>> >>>> re-refined with BUSTER (0.3590/0.3880) than the deposited ones
> >>>> >>>> (0.2210/0.2500). This led us to carry out some investigation of that
> >>>> entry.
> >>>> >>>>
> >>>> >>>> The deposited coordinates were flagged by BUSTER as having 4602
> >>>> >>>> bond-length violations, the worst being 205.8 sigmas, and other
> >>>> >>>> wild outliers. The initial Molprobity analysis gave a clash score
> >>>> >>>> of near 100, placing it in the 0-th percentile. The PDB validation
> >>>> >>>> report is dominantly red and ochre, with only a few wisps of green.
> >>>> >>>>
> >>>> >>>> Examining the model and map with Coot showed "waters, waters
> >>>> >>>> everywhere", disconnected density, and molecules separated by large
> >>>> >>>> layers of water. The PDB header lists hundreds of water molecules
> >>>> >>>> in REMARK 525 records that are further than 5.0 Angs from the
> >>>> >>>> nearest chain, some of them up to 15 Angs away.
> >>>> >>>>
> >>>> >>>> The cartoons on the NCBI server at
> >>>> >>>>
> >>>> >>>>
> >>>> http://www.ncbi.nlm.nih.gov/Structure/mmdb/mmdbsrv.cgi?uid=142582&d
> >>>> >>>> ps=1
> >>>> >>>>
> >>>> >>>> show random coils threaded up and down through beta-strands, and
> >>>> >>>> the one on the RCSB PDB site at
> >>>> >>>>
> >>>> >>>> http://www.rcsb.org/pdb/explore.do?structureId=5GNN
> >>>> >>>>
> >>>> >>>> also shows mostly random coil, with only very few and very short
> >>>> >>>> segments of secondary structure.
> >>>> >>>>
> >>>> >>>> In reciprocal space, an oddness of a different kind is that if
> >>>> >>>> one looks at the mtz file, the amplitudes and their sigmas are on a
> >>>> >>>> very small scale. However the STARANISO display shows a smooth and
> >>>> >>>> plausible distribution of I/sig(I) to the full nominal resolution
> >>>> >>>> limit of 1.6A.
> >>>> >>>>
> >>>> >>>> Looking at the publication associated with this entry
> >>>> >>>>
> >>>> >>>> http://www.ncbi.nlm.nih.gov/pubmed/27492925
> >>>> >>>>
> >>>> >>>> indicates that the structure was solved by MR from a model obtained
> >>>> >>>> from a structure prediction server (I-TASSER). No further details
> >>>> >>>> are given, even in the Supplemental Material. Table 1 does report a
> >>>> >>>> MolProbity clash score of 103.59, as well as 10% Ramachandran
> >>>> >>>> outliers and 25.51% rotamer outliers. It also contains a mention of
> >>>> >>>> a twinning operator -h, -k, l with a twinning fraction of 0.5,
> >>>> >>>> although there is no mention of it in the text nor in the PDB file.
> >>>> >>>>
> >>>> >>>> I will follow my own advice and resist the temptation of calling
> >>>> >>>> this "the end of civilisation as we know it", but this is startling.
> >>>> >>>> Perhaps we have over-advertised to the non-experts the few
> >>>> >>>> successes of structure prediction programs as reliable sources of
> >>>> >>>> MR models and thus created unwarranted optimism, besides the usual
> >>>> >>>> exaggeration of the degree to which X-ray crystallography has
> >>>> >>>> become a push-button commodity that can deliver results to
> >>>> >>>> untrained users. What is also disconcerting is that the abundant
> >>>> >>>> alarm bells that rang along the way (the MolProbity clash score and
> >>>> >>>> geometry reports, the contents of the PDB validation report, and
> >>>> >>>> simple common sense when examining electron density and model)
> >>>> >>>> failed to make anyone involved along the way take notice that there
> >>>> was something seriously wrong.
> >>>> >>>>
> >>>> >>>> This case seems to bring to the forefront even more vividly than
> >>>> >>>> 4nl6 and 4nl7 some collective issues that we face. Here the problem
> >>>> >>>> is not one of contamination of a protein prep resulting in crystals
> >>>> >>>> of "the wrong protein": there is also a more diffuse contamination
> >>>> >>>> by deficiencies of judgement, expertise and vigilance at several
> >>>> >>>> consecutive stages, including refereeing and publication.
> >>>> >>>>
> >>>> >>>> Validation is a hot topic at the moment, and this may serve as a
> >>>> >>>> concrete example that some joined-up thinking and action is indeed
> >>>> >>>> a matter of urgency, and that extreme scenarios of things going
> >>>> >>>> wrong do not exist solely in the imaginations of
> >>>> >>>> obsessive-compulsive/paranoid validators.
> >>>> >>>>
> >>>> >>>> I am grateful to several colleagues for correspondance and
> >>>> >>>> discussions on the matters touched upon on this message.
> >>>> >>>>
> >>>> >>>>
> >>>> >>>> With best wishes,
> >>>> >>>>
> >>>> >>>> Gerard
> >>>> >>>>
> >>>> >>>> --
> >>>> >>>>
> >>>> >>>>
> >>>> ==========================================================
> >>>> =====
> >>>> >>>> * *
> >>>> >>>> * Gerard Bricogne [log in to unmask] *
> >>>> >>>> * *
> >>>> >>>> * Global Phasing Ltd. *
> >>>> >>>> * Sheraton House, Castle Park Tel: +44-(0)1223-353033 *
> >>>> >>>> * Cambridge CB3 0AX, UK Fax: +44-(0)1223-366889 *
> >>>> >>>> * *
> >>>> >>>>
> >>>> ==========================================================
> >>>> =====
> >>>> >>>
> >>>> >>> --
> >>>> >>> Paul Adams
> >>>> >>> Division Director, Molecular Biophysics & Integrated Bioimaging,
> >>>> >>> Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced
> >>>> >>> Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of
> >>>> >>> Bioengineering, U.C. Berkeley Vice President for Technology, the
> >>>> >>> Joint BioEnergy Institute Laboratory Research Manager, ENIGMA
> >>>> >>> Science Focus Area
> >>>> >>>
> >>>> >>> Building 33, Room 347
> >>>> >>> Building 80, Room 247
> >>>> >>> Building 978, Room 4126
> >>>> >>> Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul
> >>>> >>>
> >>>> >>> Lawrence Berkeley Laboratory
> >>>> >>> 1 Cyclotron Road
> >>>> >>> BLDG 33R0345
> >>>> >>> Berkeley, CA 94720, USA.
> >>>> >>>
> >>>> >>> Executive Assistant: Louise Benvenue [ [log in to unmask] ][
> >>>> >>> 1-510-495-2506 ]
> >>>> >>> --
> >>>>
> >>>> --
> >>>> Dr. Manfred. S. Weiss
> >>>> Helmholtz-Zentrum Berlin für Materialien und Energie Macromolecular
> >>>> Crystallography (HZB-MX) Albert-Einstein-Str. 15
> >>>> D-12489 Berlin
> >>>> GERMANY
> >>>> Fon: +49-30-806213149
> >>>> Fax: +49-30-806214975
> >>>> Web: http://www.helmholtz-berlin.de/bessy-mx
> >>>> Email: [log in to unmask]
> >>>>
> >>>> ________________________________
> >>>>
> >>>> Helmholtz-Zentrum Berlin für Materialien und Energie GmbH
> >>>>
> >>>> Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher
> >>>> Forschungszentren e.V.
> >>>>
> >>>> Aufsichtsrat: Vorsitzender Dr. Karl Eugen Huthmacher, stv. Vorsitzende Dr.
> >>>> Jutta Koch-Unterseher
> >>>> Geschäftsführung: Prof. Dr. Anke Rita Kaysser-Pyzalla, Thomas Frederking
> >>>>
> >>>> Sitz Berlin, AG Charlottenburg, 89 HRB 5583
> >>>>
> >>>> Postadresse:
> >>>> Hahn-Meitner-Platz 1
> >>>> D-14109 Berlin
> >>>>
> >>>> http://www.helmholtz-berlin.de
--
===============================================================
* *
* Gerard Bricogne [log in to unmask] *
* *
* Global Phasing Ltd. *
* Sheraton House, Castle Park Tel: +44-(0)1223-353033 *
* Cambridge CB3 0AX, UK Fax: +44-(0)1223-366889 *
* *
===============================================================
|