Thanks for all the interesting answers so far!
The anisotropy issue is one that got me worrying about
truncate for data from DNA-containing crystals in
particular - and the fact that since its a default in ccp4i,
new people have stopped worrying about whether or not they
should use it.
The DNAs usually stack end-to-end, and thus are very often
aligned with a particular axis. Since all those nice flat
bases are ~3.4A apart, there are often whomping spots in
only one direction at ~3.4A (even if the DNA isn't even half
the total scattering mass). So even if the overall
diffraction limits are roughly isotropic, in certain
resolution shells isotropy is still a bad assumption.
Phoebe
---- Original message ----
>Date: Tue, 9 Sep 2008 09:59:58 +0100
>From: Eleanor Dodson <[log in to unmask]>
>Subject: [SPAM:#] Re: [ccp4bb] truncate ignorance
>To: [log in to unmask]
>
>This is a very educational thread but I should remind you
that the
>assumed distributions are NOT reliable when either a) the
data is very
>anisotropic, or b) the data is very incomplete or c) there
is a
>non-crystallographic translation vector in the structure or
d) the data
>is twinned.
>
> I for one dont really know what to do about this, but
remember the Is
>are as measured and are in these cases "safer" reflections
of the
>experiment..
>
>Eleanor
>
>
>Ian Tickle wrote:
>> Having read the remainder of the paper more carefully I
note that the
>> authors do go into an extensive discussion about Jeffreys
(which they
>> don't recommend) and Wilson priors, which indeed overcome
my objection
>> to the use of the improper prior. They conclude that the
simpler
>> expression is adequate for their purposes. George
Sheldrick's objection
>> would be valid for their simple prior since the effect on
intensities in
>> a shell where the true average intensity was zero would
be to give a
>> non-zero positive and hence biased average intensity.
However I don't
>> think it's valid to conclude without more careful
analysis that their
>> simple prior is also adequate in the single crystal case,
since the
>> kinds of errors encountered (namely from deconvoluting
overlapping
>> reflections) are quite different.
>>
>> -- Ian
>>
>>
>>> -----Original Message-----
>>> From: [log in to unmask]
>>> [mailto:[log in to unmask]] On Behalf Of
[log in to unmask]
>>> Sent: 08 September 2008 22:20
>>> To: Jacob Keller
>>> Cc: [log in to unmask]
>>> Subject: Re: [ccp4bb] truncate ignorance
>>>
>>> I would also recommend reading of the following paper:
>>>
>>> D.S. Sivia & W.I.F. David (1994), Acta Cryst. A50, 703-
714. A
>>> Bayesian
>>> Approach to Extracting Structure-Factor Amplitudes from
Powder
>>> Diffraction Data.
>>>
>>> Despite of the title, most of the analysis presented in
this paper
>>> applies equally well to single-crystal data (see
especially
>>> sections 3
>>> and 5). If you are not interested in the specific powder-
diffraction
>>> problems (i.e. overlapping peaks), you can simply skip
>>> sections 4 and 6.
>>>
>>> A few interesting points from this paper :
>>>
>>> (1) The conversion from I's to F's can be done (in a
Bayesian
>>> way) by
>>> applying two simple formula (equations 11 and 12 in the
>>> paper), which,
>>> for all practical purposes, are as valid as the more
complicated
>>> French & Wilson procedure (see discussion in section 5).
>>>
>>> (2) Re. the use of I's rather than F's : this is
discussed on
>>> page 710
>>> (final part of section 5). The authors seem to be more
in favor of
>>> using F's.
>>>
>>>
>>>
>>> Marc Schiltz
>>>
>>>
>>>
>>>
>>>
>>> Quoting Jacob Keller <[log in to unmask]>:
>>>
>>>
>>>> Does somebody have a .pdf of that French and Wilson
paper?
>>>>
>>>> Thanks in advance,
>>>>
>>>> Jacob
>>>>
>>>> *******************************************
>>>> Jacob Pearson Keller
>>>> Northwestern University
>>>> Medical Scientist Training Program
>>>> Dallos Laboratory
>>>> F. Searle 1-240
>>>> 2240 Campus Drive
>>>> Evanston IL 60208
>>>> lab: 847.491.2438
>>>> cel: 773.608.9185
>>>> email: [log in to unmask]
>>>> *******************************************
>>>>
>>>> ----- Original Message -----
>>>> From: "Ethan Merritt" <[log in to unmask]>
>>>> To: <[log in to unmask]>
>>>> Sent: Monday, September 08, 2008 3:03 PM
>>>> Subject: Re: [ccp4bb] truncate ignorance
>>>>
>>>>
>>>>
>>>>> On Monday 08 September 2008 12:30:29 Phoebe Rice wrote:
>>>>>
>>>>>> Dear Experts,
>>>>>>
>>>>>> At the risk of exposing excess ignorance, truncate
makes me
>>>>>> very nervous because I don't quite get exactly what
it is
>>>>>> doing with my data and what its assumptions are.
>>>>>>
>>>>>> From the documentation:
>>>>>>
========================================================
>>>>>> ... the "truncate" procedure (keyword TRUNCATE YES,
the
>>>>>> default) calculates a best estimate of F from I, sd
(I), and
>>>>>> the distribution of intensities in resolution shells
(see
>>>>>> below). This has the effect of forcing all negative
>>>>>> observations to be positive, and inflating the weakest
>>>>>> reflections (less than about 3 sd), because an
observation
>>>>>> significantly smaller than the average intensity is
likely
>>>>>> to be underestimated.
>>>>>>
=========================================================
>>>>>>
>>>>>> But is it really true, with data from nice modern
detectors,
>>>>>> that the weaklings are underestimated?
>>>>>>
>>>>> It isn't really an issue of the detector per se,
although in
>>>>> principle you could worry about non-linear response to
the
>>>>> input rate of arriving photons.
>>>>>
>>>>> In practice the issue, now as it was in 1977
(French&Wilson),
>>>>> arises from the background estimation, profile
fitting, and
>>>>> rescaling that are applied to the individual pixel
contents
>>>>> before they are bundled up into a nice "Iobs".
>>>>>
>>>>> I will try to restate the original French & Wilson
argument,
>>>>> avoiding the terminology of maximum likelihood and
>>>>>
>>> Bayesian statistics.
>>>
>>>>> 1) We know the true intensity cannot be negative.
>>>>> 2) The existence of Iobs<0 reflections in the data set
means
>>>>> that whatever we are doing is producing some values
of
>>>>> Iobs that are too low.
>>>>> 3) Assuming that all weak-ish reflections are being
processed
>>>>> equivalently, then whatever we doing wrong for
reflections with
>>>>> Iobs near zero on the negative side surely is also
going wrong
>>>>> for their neighbors that happen to be near Iobs=0 on
the positive
>>>>> side.
>>>>> 4) So if we "correct" the values of Iobs that went
negative, for
>>>>> consistency we should also correct the values that
are nearly
>>>>> the same but didn't quite tip over into the negative
range.
>>>>>
>>>>>
>>>>>> Do I really want to inflate them?
>>>>>>
>>>>> Yes.
>>>>>
>>>>>
>>>>>> Exactly what assumptions is it making about the
expected
>>>>>> distributions?
>>>>>>
>>>>> Primarily that
>>>>> 1) The histogram of true Iobs is smooth
>>>>> 2) No true Iobs are negative
>>>>>
>>>>>
>>>>>> How compatible are those assumptions with serious
anisotropy
>>>>>> and the wierd Wilson plots that nucleic acids give?
>>>>>>
>>>>> Not relevant
>>>>>
>>>>>
>>>>>> Note the original 1978 French and Wilson paper says:
>>>>>> "It is nevertheless important to validate this
agreement for
>>>>>> each set of data independently, as the presence of
atoms in
>>>>>> special positions or the existence of
noncrystallographic
>>>>>> elements of symmetry (or pseudosymmetry) may abrogate
the
>>>>>> application of these prior beliefs for some crystal
>>>>>> structures."
>>>>>>
>>>>> It is true that such things matter when you get down
to the
>>>>> nitty-gritty details of what to use as the "expected
distribution".
>>>>> But *all* plausible expected distributions will be non-
negative
>>>>> and smooth.
>>>>>
>>>>>
>>>>>
>>>>>> Please help truncate my ignorance ...
>>>>>>
>>>>>> Phoebe
>>>>>>
>>>>>>
==========================================================
>>>>>> Phoebe A. Rice
>>>>>> Assoc. Prof., Dept. of Biochemistry & Molecular
Biology
>>>>>> The University of Chicago
>>>>>> phone 773 834 1723
>>>>>>
>>>>>>
>>>
http://bmb.bsd.uchicago.edu/Faculty_and_Research/01_Faculty/0
1
>>> _Faculty_Alphabetically.php?faculty_id=123
>>>
>>>>>> RNA is really nifty
>>>>>> DNA is over fifty
>>>>>> We have put them
>>>>>> both in one book
>>>>>> Please do take a
>>>>>> really good look
>>>>>> http://www.rsc.org/shop/books/2008/9780854042722.asp
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Ethan A Merritt
>>>>> Biomolecular Structure Center
>>>>> University of Washington, Seattle 98195-7742
>>>>>
>>>>>
>>>
>>
>>
>> Disclaimer
>> This communication is confidential and may contain
privileged information intended solely for the named
addressee(s). It may not be used or disclosed except for the
purpose for which it has been sent. If you are not the
intended recipient you must not review, use, disclose, copy,
distribute or take any action in reliance upon it. If you
have received this communication in error, please notify
Astex Therapeutics Ltd by emailing I.Tickle@astex-
therapeutics.com and destroy all copies of the message and
any attached documents.
>> Astex Therapeutics Ltd monitors, controls and protects
all its messaging traffic in compliance with its corporate
email policy. The Company accepts no liability or
responsibility for any onward transmission or use of emails
and attachments having left the Astex Therapeutics domain.
Unless expressly stated, opinions in this message are those
of the individual sender and not of Astex Therapeutics Ltd.
The recipient should check this email and any attachments
for the presence of computer viruses. Astex Therapeutics Ltd
accepts no liability for damage caused by any virus
transmitted by this email. E-mail is susceptible to data
corruption, interception, unauthorized amendment, and
tampering, Astex Therapeutics Ltd only send and receive e-
mails on the basis that the Company is not liable for any
such alteration or any consequences thereof.
>> Astex Therapeutics Ltd., Registered in England at 436
Cambridge Science Park, Cambridge CB4 0QA under number
3751674
>>
>>
>>
>>
Phoebe A. Rice
Assoc. Prof., Dept. of Biochemistry & Molecular Biology
The University of Chicago
phone 773 834 1723
http://bmb.bsd.uchicago.edu/Faculty_and_Research/01_Faculty/01_Faculty_Alphabetically.php?faculty_id=123
RNA is really nifty
DNA is over fifty
We have put them
both in one book
Please do take a
really good look
http://www.rsc.org/shop/books/2008/9780854042722.asp
|
