Maybe indeed this is partly a semantic issue.
But it seems unlikely that what the referee means by “over-refined” is too many refinement cycles. Is the referee’s suggestion then to stop the refinement before convergence ? That makes no sense to me.
Overfitting is a real potential issue. “Over-refined” (per your definition below, running 50 cycles when 20 are enough) is not. That’s because as you’ve described very well all that happens is that nothing moves or changes in any significant way during the extra 30 cycles. In other words the model does not get better but it does not get worse either.
Thierry
From: CCP4 bulletin board [mailto:[log in to unmask]] On Behalf Of Oganesyan, Vaheh
Sent: Wednesday, April 27, 2016 9:53 AM
To: [log in to unmask]
Subject: Re: [ccp4bb] Over refinement
Is the term “over-refined” synonymous to “over-fitted” in this context? At least in my hands if I use 50 cycles of refinement when convergence is reached at 20 no statistically significant changes occur with either R or Rfree or the model. If one will keep adding solvent molecules into small pieces of electron density maps and/or fitting unstructured protein parts into badly defined places then over-fitting term is in place and decreased R and increased Rfree will show that. IMHO in no context these two terms should be used interchangeably.
Regards,
Vaheh Oganesyan
From: CCP4 bulletin board [mailto:[log in to unmask]] On Behalf Of James Phillips
Sent: Tuesday, April 26, 2016 7:33 PM
To: [log in to unmask]
Subject: Re: [ccp4bb] Over refinement
It is not where the R/Rfree "end up" it is the trajectory through the refinement process. As you refine then adjust your model the two should drop. If R drops Rfree stays the same or rises you have over refined. I suggest giving the journal the results of each stage of your refinement.
On Tuesday, April 26, 2016, Professor James Henderson Naismith <[log in to unmask]> wrote:
Dear Colleagues.
We are having difficulty persuading a reviewer that our structure is not over refined.
The structure is a molecular replacement of complex with a published relatively non-isomorphous native structure from another lab.
The same Rfree set was used as the published data.
Our complex is at 2.6A and R/Rfree end up at 18/22
PDB redo gets the same result, so does phenix.refine (with a trivial %). All B-factors were reset and TLS used.
The data are 2.61A and average B is 80A, there are 4500 residues, 68 waters. Unfortunately Mol probity gives us 100th centile and the Rama is also good, bond rms is 0.012 and we used NCS local restraints.
There is no rotational NCS but there is a weak translation symmetry (does not show up in data but when refined the monomers have a bead on a curved string appearance).
The referee has refused accept the paper until we make the R and Rfree higher by some undefined target, since it is 'over refined'
Does anyone have a useful program to make structures worse to some threshold that is considered normal at 2.6 A or does anyone know a good paper that points out Rfree is not susceptible to over refinement since by definition it is not refined.
best
Jim
Jim Naismith
St Andrews
The University of St Andrews is a charity registered in Scotland : No SC013532
To the extent this electronic communication or any of its attachments contain information that is not in the public domain, such information is considered by MedImmune to be confidential and proprietary. This communication is expected to be read and/or used only by the individual(s) for whom it is intended. If you have received this electronic communication in error, please reply to the sender advising of the error in transmission and delete the original message and any accompanying documents from your system immediately, without copying, reviewing or otherwise using them for any purpose. Thank you for your cooperation.
Notice: This e-mail message, together with any attachments, contains
information of Merck & Co., Inc. (2000 Galloping Hill Road, Kenilworth,
New Jersey, USA 07033), and/or its affiliates Direct contact information
for affiliates is available at
http://www.merck.com/contact/contacts.html) that may be confidential,
proprietary copyrighted and/or legally privileged. It is intended solely
for the use of the individual or entity named on this message. If you are
not the intended recipient, and have received this message in error,
please notify us immediately by reply e-mail and then delete it from
your system.