This goes back to my previous idea about using the Bayesian estimates
(<J> & sig(<J>)) of I & sig(I) in the refinement instead of the measured
ones.  This would remove any objection to using negative observed
intensities, though it's hard to see what exactly the objection is.
Basically you're just moving the threshold for the now deprecated
practice of applying a sigma cutoff down from 3 (or whatever) to zero,
and the same objections to this practice still apply, as you point out.

The difference between using Imeas and <J> would in practice only be
that you would get different R factors for the weak data (i.e.
definitely lower and maybe even more realistic, so you might be in a
better position to judge the true worth of the weak data).  As I said
before it's debatable that it would give you a better refined structure,
since you are not putting in any new information, as compared with
simply using all the Imeas data, including the negative ones.

I suggested before that there would only be an effect on the outcome as
a result of using the Bayesian estimate of I if the intensities
themselves were the focus of the experiment, and I said that this is not
usually the case.  However 'usually' is the key word here: there are
situations where the intensities are the focus of interest, namely in
testing for twinning.  My understanding is that currently what happens
is that to obtain the intensity values needed for the twinning tests,
the Bayesian estimates <F> are simply squared.  Of course you could also
use Imeas but the Bayesian estimate ought to work much better,
particularly for the Britton and related tests which rely on detecting
negative intensities on detwinning.  The problem with using <F>^2 is of
course that it's not equal to <J> (particularly for weak data) so
there's a strong argument that that procedure is bad statistics.  In
practice of course the weak data is often omitted for the test because
it's unreliable, but then maybe the reason it's unreliable is that it's
incorrectly calculated!  I haven't tried testing this yet, mainly
because I don't yet have a prog that will compute the <J> values in MTZ
format.

Cheers

-- Ian

> -----Original Message-----
> From: [log in to unmask] [mailto:[log in to unmask]]
On
> Behalf Of Dunten, Pete W.
> Sent: 26 September 2008 00:01
> To: [log in to unmask]
> Subject: Reading the old literature / truncate / refinement programs
> 
> 
> 
> I mentioned previously phenix.refine tosses your weak data if IMEAN,
> SIGIMEAN are chosen during refinement.
> 
> 
> 
> I'm wondering if this omission of weak Fobs from the Fobs-Fcalc
difference
> map explains why the difference maps out of refmac seem to be more
helpful
> in showing where to move atoms.
> 
> 
> 
> D. Crowfoot et al. in The Chemistry of Penicillin (1949) explain why
this
> might be so, and Stout & Jenson elaborate the argument.  Briefly, the
> calculated phase will be closest to the phase of the vector difference
> Fcalc - Fobs when |Fcalc| > |Fobs|.
> 
> 
> 
> I leave it to the reader to try calculating some maps with and without
the
> weak Fobs in phenix.refine or refmac, and perhaps making some
deliberate
> rotamer errors, to see if using the complete data with weak Fobs
helps.



Disclaimer
This communication is confidential and may contain privileged information intended solely for the named addressee(s). It may not be used or disclosed except for the purpose for which it has been sent. If you are not the intended recipient you must not review, use, disclose, copy, distribute or take any action in reliance upon it. If you have received this communication in error, please notify Astex Therapeutics Ltd by emailing [log in to unmask] and destroy all copies of the message and any attached documents. 
Astex Therapeutics Ltd monitors, controls and protects all its messaging traffic in compliance with its corporate email policy. The Company accepts no liability or responsibility for any onward transmission or use of emails and attachments having left the Astex Therapeutics domain.  Unless expressly stated, opinions in this message are those of the individual sender and not of Astex Therapeutics Ltd. The recipient should check this email and any attachments for the presence of computer viruses. Astex Therapeutics Ltd accepts no liability for damage caused by any virus transmitted by this email. E-mail is susceptible to data corruption, interception, unauthorized amendment, and tampering, Astex Therapeutics Ltd only send and receive e-mails on the basis that the Company is not liable for any such alteration or any consequences thereof.
Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science Park, Cambridge CB4 0QA under number 3751674