Hi all,

I have read recent SERCA paper on IUCrJ and found their discussion
interesting.
"Structural studies of P-type ATPase–ligand complexes using an X-ray 
free-electron laser" by Maike Bublitz et al.
http://journals.iucr.org/m/issues/2015/04/00/jt5009/index.html
In addition to CC1/2 and paired refinement, they scrambled indices
of reflections and showed Rfree indeed went up.

If we repeat permutations many times and calculate CC1/2 every time,
we can obtain empirical distribution of CC1/2 under null hypothesis
and thus calculate  p-value. This is called permutation test in
statistics. I am wondering how this p-value compares with p-values 
calculated by Student's t-test. Since t-test does not take the effect
of multiplicity into the account, I think the permutation test
gives more accurate value. (That is, CC1/2 calculated from a dataset
with multiplicity 30 should be more accurate and "reliable" than
that calculated from a dataset with multiplicity 2, but this difference
is ignored in t-test.)

Did anyone try this?

Best regards,

Takanori Nakane

On 2015年07月03日 04:41, Tim Gruene wrote:
> Hi Robbie,
>
> I have been wondering how much information would be present in a
> weighted CC1/2 with weights from the ML refinement program.
>
> As I understand the concept behind paired refinement, one can use much
> higher resolution data in refinement than you would expect from the
> (classical) scaling statistics, because refinement programs down-weight
> the outliers.
>
> Maybe a weighted CC1/2 would be a convenient short cut the more time
> consuming paired refinemement? How easily could the calculation of a
> weighted CC1/2 be implemented in refmac? Refmac would have to merge the
> data internally, but that's just a simple formula to implement.
>
> Best,
> Tim
>
> On 07/02/2015 08:28 PM, Robbie Joosten wrote:
>> Hi Jacob,
>>
>> You need unmerged data to calculate cc1/2. That's not the sort of data you get from the PDB.  But anyway, we have a fairly simple automated test that we can use on a case-by-case basis. I would argue that that is nicer than a empirical cut-off that may or may not be correct for you case.
>>
>> Cheers,
>> Robbie
>>
>> Sent with my Windows Phone
>> ________________________________
>> Van: Keller, Jacob<mailto:[log in to unmask]>
>> Verzonden: ‎2-‎7-‎2015 20:12
>> Aan: [log in to unmask]<mailto:[log in to unmask]>
>> Onderwerp: Re: [ccp4bb] paired refinement
>>
>> Well, in that case, one could simply look at the plot of CC1/2 versus resolution and see the step up to one, conclude something was off.
>>
>> I wonder whether PDB REDO was able to get some empirically-determined values for CC1/2 cutoffs by comparing paired refinement versus CC1/2 versus other parameters?
>>
>> JPK
>>
>> -----Original Message-----
>> From: CCP4 bulletin board [mailto:[log in to unmask]] On Behalf Of Dale Tronrud
>> Sent: Thursday, July 02, 2015 1:46 PM
>> To: [log in to unmask]
>> Subject: Re: [ccp4bb] paired refinement
>>
>>     While I was puzzling over an entry in the PDB some years ago (since
>> obsoleted) I noticed that all the high resolution amplitudes were equal to 11.0!  This was before CC1/2 but for this structure it would have been equal to one, and yet the outer data were useless.  A practical test like paired refinement can't be fooled in this way.
>>
>> Dale Tronrud
>>
>> On 7/2/2015 10:25 AM, Edward A. Berry wrote:
>>> My take on this-
>>> No one has been willing to specify a cutoff (and probably there is no
>>> rigorous way to mathematically define the cutoff) and say "If CC* (or
>>> CCfree or
>>> whatever) is below X
>>> then it will not improve your structure, if above X then it will".
>>> Probably depends
>>> among other things on how strong the lower resolution data is, how
>>> good the structure is without the added data.
>>> On the other hand in paired refinement, if adding the data improves
>>> the structure as measured by Rfree in a zone excluding the added data,
>>> then it is hard to deny that that data are worth including.
>>>
>>> eab
>>>
>>> On 07/02/2015 12:52 PM, Keller, Jacob wrote:
>>>> Wasn’t all of this put to bed through the implementation of CC measures?
>>>>
>>>> JPK
>>>>
>>>> *From:*CCP4 bulletin board [mailto:[log in to unmask]] *On Behalf
>>>> Of *Robbie Joosten
>>>> *Sent:* Thursday, July 02, 2015 12:46 PM
>>>> *To:* [log in to unmask]
>>>> *Subject:* Re: [ccp4bb] paired refinement
>>>>
>>>> But it is not the R-free of the shell here. In paired refinement you
>>>> take the R-free of the reflections outside the shell.
>>>>
>>>> Cheers,
>>>> Robbie
>>>>
>>>> Sent with my Windows Phone
>>>>
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------------------------------------------------------
>>>> ---------------------
>>   ---
>>>>
>>>>
>>>> *Van: *Edward A. Berry <mailto:[log in to unmask]>
>>>> *Verzonden: *‎2-‎7-‎2015 18:43
>>>> *Aan: *[log in to unmask] <mailto:[log in to unmask]>
>>>> *Onderwerp: *Re: [ccp4bb] paired refinement
>>>>
>>>> Another criterion for cutoff, also requiring the structure to be
>>>> solved, is the agreement between data and structure, e.g. Rfree or CCfree.
>>>> I think it is very unlikely that you could get Rfree =.2493 in a
>>>> shell which contains only noise. So I would suggest doing paired
>>>> refinement to 2.2 and 2.1 A (if the data is available).
>>>>
>>>> On 07/01/2015 07:15 PM, Eric Karg wrote:
>>>>   > Hi all,
>>>>   >
>>>>   > I have a dataset processed in XDS to 2.3 A (based on CC1/2). I'm
>>>> trying to do "paired refinement" to determine the optimal resolution
>>>> cutoff. Here is what I get at different resolutions set in Phenix:
>>>>   >
>>>>   > Final Rfree/Rwork:
>>>>   > 2.7—> 0.2498/0.2027
>>>>   > 2.6—> 0.2519/0.2009
>>>>   > 2.5—> 0.2567/0.2025
>>>>   > 2.4 —> 0.2481/0.2042
>>>>   > 2.3 —> 0.2493/0.2075
>>>>   >
>>>>   > The geometry of all output structures are similar.
>>>>   >
>>>>   > 1. What is the high resolution cutoff based on these data? I know
>>>> that Rfree/Rwork at different resolution should not be compared, but
>>>> is there a simple way to do the test as described in the K&D 2012
>>>> Science paper using Phenix GUI?
>>>>   >
>>>>   > 2. For refining a structure at a lower resolution (lower than the
>>>> initial dataset), do I simply set the resolution limit in the
>>>> refinement or I need to reprocess the data starting from the images?
>>>> Do I need to do anything with Rfree flags? Based on the discussions
>>>> on this forum I know I should deposit the highest resolution dataset
>>>> but my question is about the mtz file which will be used for refinement.
>>>>   >
>>>>   > Thank you very much for your help!
>>>>   >
>>>>
>>
>