> -----Original Message-----
> From: [log in to unmask] 
> [mailto:[log in to unmask]] On Behalf Of James Holton
> Sent: 21 January 2010 08:39
> To: [log in to unmask]
> Subject: Re: [ccp4bb] Refining against images instead of only 
> reflections
> 
> It is interesting and relevant here I think that if you measure 
> background-subtracted spot intensities you actually are measuring the 
> AVERAGE electron density.  Yes, the arithmetic average of all 
> the unit 
> cells in the crystal.  It does not matter how any of the 
> vibrations are 
> "correlated", it is still just the average (as long as you 
> subtract the 
> background).  The diffuse scatter does NOT tell you about the 
> deviations 
> from this average; it tells you how the deviations are 
> correlated from 
> unit cell to unit cell. 

James, as I've pointed out before this is completely inconsistent with
both established DS theory and many experiments performed over the
years.  If you're simulation is producing this result, then the obvious
conclusion is that you're not simulating what you claim to be.  I don't
know of a single experimental result that supports your claim.  In order
for it to be true the total background (i.e. the sum of the detector
noise, air scatter, scattering from the cryobuffer, Compton scattering
from the crystal and of course the diffuse scattering itself) would have
to be a linear function (or more precisely planar since the detector
co-ords are obviously 2-D) of the detector co-ordinates in the region of
the Bragg spots, since that is the background model that is used for
background subtraction.  Whilst it may be true that detector noise and
non-crystalline scattering can be accurately modeled by a linear
background model (at least in the local region of each Bragg spot), this
cannot possibly be generally true of the DS component, and since getting
at the DS component is the whole purpose of the experiment, it is
crucial that this be modeled accurately.  Of course your claim may well
be true if there's no DS, but we're talking specifically about cases
where there is observable DS (otherwise what's the point of your
simulation?).  The reason it can't be true that the DS is a linear
function is that there's a wealth of simulation work and experimental
data that demonstrate that it's not true (not to mention simple manual
observation of the images!).  The simulations cannot easily be dismissed
as unrealistic because in many cases they give an accurate fit to the
experimental data.

As an example see here:
http://journals.iucr.org/a/issues/2008/01/00/sc5007/sc5007.pdf .

Looking at the various simulations here (Figs 3 & 5) it's obvious that
the DS is very non-linear at the Bragg positions (and more importantly
it's also non-linear between the Bragg positions).  Note that the
simulated calculated patterns here contain no Bragg peaks since as noted
in the Figure legends, the average structure (or the average density)
has been subtracted in the calculation, i.e. the simulations are showing
only the DS component.  I fail to see how any kind of background
subtraction model could cope with the DS and give the right answer for
the Bragg intensity in these kind of cases.  Even from the observed
patterns it's plain that the DS is non-linear, and therefore a linear
background correction couldn't possibly correct the raw integrated
intensity for the DS component.

Well-established theory says that the total coherent scattered intensity
is proportional to (~=) the time-average of the squared modulus of the
structure factor of the crystal:

	I(coherent) ~= <|Fc|^2>

If we make the assumption that the deviations of the contributions to
the structure factor from different unit cells are uncorrelated, we can
show that the Bragg intensity is the squared modulus of the time and
lattice-averaged SF sampled at the reciprocal lattice points:

	I(Bragg) 	~= |<F>|^2

The time/lattice-averaged SF is the FT of the average density, and
therefore I(Bragg) indeed corresponds to the average density.

The diffuse intensity is the difference between these:

	I(diffuse) 	= I(coherent) - I(Bragg)

			~= <|F|^2> - |<F>|^2

The assumption above implies that we're assuming that there's no
'acoustic' component of the DS, since this arises from correlations
between different unit cells.  However this doesn't mean that there *is*
no acoustic component, it simply means that we are ignoring it: for one
thing we have no alternative since the acoustic and Bragg scattering are
practically inseparable; for another, correlations between different
unit cells are purely an artifact of the crystallisation process, so
have no biological significance, hence we're usually not interested in
them anyway.

> The diffuse scatter does NOT tell you about the deviations 
> from this average; it tells you how the deviations are 
> correlated from unit cell to unit cell.

This is completely wrong, the previous equation can be rewritten as:

	I(diffuse)	= <|F - <F>|^2>

clearly demonstrating that the DS does indeed tell you about the
mean-squared deviation of the SF from the average (i.e. the variance of
the SF), and therefore the density from its time/lattice average.  Note
that I(diffuse) must necessarily be positive implying that the measured
intensity always overestimates the Bragg intensity; it cannot average
out to zero.

If we further assume the usual harmonic model for the atomic
displacements, we can show that the DS intensity is related to the
covariance (or less correctly the correlation) of the displacements: I
suspect this is what you meant.  This is all nicely explained in Michael
Wall's doctorate thesis which is available online:

http://lunus.sourceforge.net/Wall-Princeton-1996.pdf .

This also has a nice historical survey of all PX DS results obtained up
until 1996.

Cheers

-- Ian


Disclaimer
This communication is confidential and may contain privileged information intended solely for the named addressee(s). It may not be used or disclosed except for the purpose for which it has been sent. If you are not the intended recipient you must not review, use, disclose, copy, distribute or take any action in reliance upon it. If you have received this communication in error, please notify Astex Therapeutics Ltd by emailing [log in to unmask] and destroy all copies of the message and any attached documents. 
Astex Therapeutics Ltd monitors, controls and protects all its messaging traffic in compliance with its corporate email policy. The Company accepts no liability or responsibility for any onward transmission or use of emails and attachments having left the Astex Therapeutics domain.  Unless expressly stated, opinions in this message are those of the individual sender and not of Astex Therapeutics Ltd. The recipient should check this email and any attachments for the presence of computer viruses. Astex Therapeutics Ltd accepts no liability for damage caused by any virus transmitted by this email. E-mail is susceptible to data corruption, interception, unauthorized amendment, and tampering, Astex Therapeutics Ltd only send and receive e-mails on the basis that the Company is not liable for any such alteration or any consequences thereof.
Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science Park, Cambridge CB4 0QA under number 3751674