I have tried to summarise the many excellent contributions to this
discussion. There is no proper acknowledgements; this is done in a hurry
since I am about to leave for the ECM meeting
Eleanor
I will try to summarise this thread, in fact several different useful threads, which range much more widely and are more important than any commentary on the suspect structure,2hro. I apologise in advance for any misrepresentation, and for sometimes pushing my own point of view. The contributions have been extremely interesting and far ranging and this does scant justice to the discussion.
The discussion seemed to me to fall under several headings, although many of these are closely linked.
1) What validation tools to use, and what should be validated.
2) How to communicate validation results to referees. Should validation summaries be made available to the public from the PDB?
3) Public access to experimental information - should all images be available, would they be useful, and how can this be done?
4) Should ccp4BB provide a forum for public comment on a particular structure, or does this amount to defamation..
5) The need for crystallographic education for readers and students
Other comments which were interesting but probably less generally useful..
Criticism of Nature editorial policies, and implying that one suspect structure disgraces MX crystallography
1.
*/What validation tools to use and what should be validated./*
This includes the experiment, the structure and the biochemical
deductions based on it.
*(These tools should be used throughout the process of structure
solution to guide the scientist, and can also be used at deposition.)*
In the 1990s there was a EU grouping called CRITQUAL who addressed
this question in great detail. I am glad to say our
recommendations still hold good.
a) Examination of structure.
A bad Ramachandran plot is still a cause for great concern
Bad crystal contacts
Bad rotamers, etc.
Unusual B factor distributions. Surface loops and side chains are
expected to be more mobile than the buried core.
Unusual folds. These may well be quite correct but they need to be
examined carefully.
MOLPROBITY is a new tool which examines the geometric features and
provides a report. COOT has excellent and very “visual” validation
tools.
b) Cell content analysis. Crystals with solvent content above 75%
are rare, and usually fragile. They would be expected to show some
unusual crystal contacts. MSDpisa analyses these.
c) Experimental data analysis. Moments, Cumulative intensity,
standard deviations of amplitudes, etc
d) Agreement between observed and calculated amplitudes. Low
resolution data will not agree well, Free R and R factor
differences should reflect the physical character of
macro-molecular diffraction.
e) Map analysis. This is my favourite, but is harder to quantify.
However the density histograms used for DM can be also used here
as an assessment tool. Structural papers SHOULD show relevant
electron density.
f) The most urgent need and the hardest to satisfy is for new
validation tools to assess the biochemical deductions based on
structure, their substrates and ligands. This is harder to
quantify – ligands do not have the restraints of the peptide
chain, and I see no way to do this except through inspecting the
maps. This could be done better if any experimental phasing
information was also deposited. It also requires that the PDB
includes a description of the ligand used, as a SMILES string or
in some other chemically sensible format.
1.
*/How to communicate validation reports./*
a) There is general agreement that referees at least should have
access to the* *deposited coordinates and experimental data before
publication. This is indeed Nature's policy, and as a community we
should push for it to be a universal policy. Probably the IUCr
could be recruited to help.
However there are two main reasons why this is not at present as
useful as it might be. First not all referees are
crystallographers, and may not know how to use the information
provided. ( It would not be much use asking me to comment on
gels..) Secondly referees have other jobs as well, and usually do
not have the time to do a detailed validation themselves. And
again there is plenty of anecdotal evidence that journals such as
Nature do not necessarily enforce referee's queries. The most
effective and practical solution seems to be for the deposition
site ( RSCB, MSD) to allow their validation reports (always sent
back to the depositor) to be seen by referees. The reports would
have to be carefully written to make their meaning clear to
non-crystallographers.
A related service which I for one would appreciate: it would be very
good if when you download a structure from the PDB you could also
access, or generate for yourself, such a report. As one correspondent
wrote; she is only really interested in safeguarding the quality of the
deposited structures, and knowing that they can be trusted.
3.
*/Freedom of information. Should diffraction images be made
available?/*
There is a legal requirement for many publicly funded laboratories
to keep experimental data for public use. However no-one is quite
sure how best this can be done; one person's lab book is useful,
others are a dog's breakfast..
Kim Henrick gave a very useful summary of the problems and expense
this would entail. It is lab practice in York to store images and
sometimes this has proved very useful. But it requires EXCELLENT
book keeping; we are lucky in having a dictatorial X-ray manager,
but without proper headers recorded in the images, and without
other “meta-data” (eg – what the protein sequence was!) the images
are no use at all. The JSCG data base provides a valuable resource
for software developers, and a wonderful model for what can be
done; however this is the work of a small disciplined group. Some
synchroton facilities archive all images but again for them to be
useful even to the users who collected them other information
about crystal quality etc etc has to be added. And James Holden
points out that only about 1 in 50 data sets is actually used to
refine a deposited structure.
I agree with those who requested that integrated but unmerged data
be deposited along with the coordinates and merged data. This
seems manageable and would be useful as a validation tool, eg for
finding errors in space group.
4.
*/Public criticism of 2hro, and other structures./*
Some contributors were uneasy about this trial by BB. People
agreed it was inappropriate to speculate about how errors in this
structure were generated. But I think it is important that we
discuss such errors; I know that I often get things wrong, and
while embarrassing it helps to have others point out one's
mistakes. (I even sometimes learn how to avoid that mistake in the
future..) Crystallographers would be exceptionally arrogant if
they felt they never made mistakes.
Fraud is another matter – I think it is rare, and we ought to
avoid the Richard Reid shoe syndrome if possible..
One thread which seems very worrying to me is the reluctance of
journals, especially the more newsy ones such as Nature, to
publish any correspondence which criticises their articles. Surely
one of the highlights of a newspaper is the letter page – some are
unfair, some opinionated, some completely off the wall, but they
allow a public debate. It would be excellent if these journals
would allow us to sound off in a publication rather than having to
depend on an electronic bulletin board.
5.
*/The need for better crystallographic education to help users of
crystallography assess their own results./*
This is a very important need, and I guess all of us who
contribute to the bulletin board are trying to some extent to
satisfy it. I know that the CCP4 and PHENIX and TNT and SHARP
developers invest a lot of time and thought into providing
diagnostics. It is difficult to get across the right level of
information but we will all keep on trying!
|
