Dear Wolfram,

     I have a perhaps optimistic view of the effect of high-order NCS
on Rfree, in the sense that I don't view it as a "problem". People
have agonised to extreme degrees over the "difficulty" of choosing a
free set of reflections that would produce the expected gap between
Rwork and Rfree, and some of the conclusions were that you would need
to hide almost half of your data in some cases!

     I think it is best to remember that the idea of cross-validation
by Rfree is to prevent overfitting, i.e. ending up with a model that
fits the amplitudes too well compared to how well it determines the
phases. In the case of high-order NCS (in your case, the U/V ratio
that the old papers on NCS identified as the key quantity to measure
the phasing power of NCS would be less than 0.1!) the phases and the
amplitudes are so tightly coupled that it is simply impossible to fit
the amplitudes without delivering phases of an equally good quality.
In other words there is no overfitting problem (provided you do have
good and complete data) and the difference between Rfree and Rwork is
simply within the bounds of the statistical spread of Rfree depending
on the free set chosen.

     You are lucky to have 6-fold NCS, so don't let any reviewer
convince you that it is a curse, and make you suffer for it :-) .


     With best wishes,
     
          Gerard.

--
On Tue, Jun 30, 2015 at 12:58:44PM -0400, wtempel wrote:
> Hello,
> my question concerns refinement of a structure with 6-fold NCS (local
> automatic restraints in REFMAC) against 2.8 A data. The size of my free set
> is 1172 selected in thin resolution shells (SFTOOLS) and corresponding to
> 4.3 % of reflections.
> A refmac run of 10 cycles of TLS and 10 cycles of CGMAT starts out at
> Rfree/Rcryst 0.271/0.272. After the 10th TLS cycle I have 0.227/0.224. Yes,
> Rfree < Rcryst. At the end of CGMAT I have 0.2072/0.2071.
> I understand that NCS stresses the independence assumption of the free set.
> Am I correct in believing that Rfree *may* be smaller than Rcryst even in
> the absence of a major mistake? My hope is that the combined wisdom of
> ccp4bb followers can point out my possible mistake,  suggest tests that I
> may perform to avoid them and, possibly, arguments in defense of a
> crystallographic model with Rfree < Rcryst.
> Many thanks,
> Wolfram Tempel

-- 

     ===============================================================
     *                                                             *
     * Gerard Bricogne                     [log in to unmask]  *
     *                                                             *
     * Global Phasing Ltd.                                         *
     * Sheraton House, Castle Park         Tel: +44-(0)1223-353033 *
     * Cambridge CB3 0AX, UK               Fax: +44-(0)1223-366889 *
     *                                                             *
     ===============================================================