I have re-visited these calculations over the weekend.  As far as I can 
tell, there is just no way to change background-subtracted spot 
intensities with "diffuse" scattering unless the motions are somehow 
"synchronized" across different unit cells.  Call it "optical" or 
"acoustic" or whatever you like.

  I have now repeated the previously-posted nearBragg simulations with 
10x more atoms, but still 10% of the scattering matter involved in a 
two-headed displacement.  I have also done a "disordered solvent" 
simulation where more than half of the unit cell volume is filled with 
completely random atoms.  In both cases the result was the same as 
before: subtracting the "average-electron-density" diffraction image 
from the "average diffraction over many configurations" image is a 
smooth and "locally uniform" image with no signs of spot intensities.  
This implies that subtracting a smooth "local background" from each spot 
will yield the Fourier coefficients of the average electron density (as 
long as the crystal has no "synchronized disorder").

Examples of "synchronized disorder" would be something like a sound wave 
moving through the crystal.  This would cause the motion of atoms in one 
unit cell to be related in some way to those in another.  Another way to 
do this is if the lattice is distorted by defects.  As this is a pet 
model of mine, I have now added an example of it on my little web page:
http://bl831.als.lbl.gov/~jamesh/diffuse_scatter/index.html#dilatation

This effect does produce little "bumps" under Bragg peaks that can 
become quite significant.  Perhaps this is the "acoustic DS" that Ian is 
talking about?  Or perhaps it should be called "defect DS"?

The really interesting bit I think is that no matter what the 
lattice-distortion model, the fractional changes in spot intensities are 
the same.  If this is true in general, then such a "synchronized 
disorder correction" would be fairly easy to incorporate into a 
refinement program (very few new parameters).  The shape of the DS 
between spots could guide this correction, but might be unnecessary if 
the disorder is apparent in the average electron density.  So, I can 
still claim to be relevant to the original post!

-James Holton
MAD Scientist


Ian Tickle wrote:
> James, I think the problem is that your simulation just doesn't contain
> enough atoms in the unit cell with correlated displacements to exhibit
> significant optic DS, i.e. with only 1 or 2 atoms it will be dominated
> by Einstein-model DS which as I explained before is locally uniform and
> therefore can be fitted by a planar background function.
>
> Cheers
>
> -- Ian
>
>   
>> -----Original Message-----
>> From: [log in to unmask] [mailto:[log in to unmask]]
>>     
> On
>   
>> Behalf Of James Holton
>> Sent: 29 January 2010 09:43
>> To: [log in to unmask]
>> Subject: Re: [ccp4bb] Refining against images instead of only
>>     
> reflections
>   
>> All I'm saying is that when I calculate the average general scattering
>> from 8192 random configurations of one disordered atom per unit cell:
>> http://bl831.als.lbl.gov/~jamesh/diffuse_scatter/xtal_diffuse.gif
>> and then subtract from that the general scattering from an
>> "occupancy-weighted model" with the two possible atom positions are at
>> half occupancy:
>> http://bl831.als.lbl.gov/~jamesh/diffuse_scatter/xtalAB_Fsum.gif
>> I get an difference image that shows only the smooth diffuse-scatter
>> background, with no spots to speak of:
>>
>>     
> http://bl831.als.lbl.gov/~jamesh/diffuse_scatter/xtals_diffuse_minus_Fsu
> m.
>   
>> gif
>>
>> But, if I calculate the average general scattering from an "all A" and
>> an "all B" crystal:
>> http://bl831.als.lbl.gov/~jamesh/diffuse_scatter/xtalAB_Isum.gif
>> and subtract from it the same partial-occupancy model image as above:
>> http://bl831.als.lbl.gov/~jamesh/diffuse_scatter/xtalAB_Fsum.gif
>> I get an image where some of the spots have been subtracted out, but
>> others are still quite pronounced:
>>
>>     
> http://bl831.als.lbl.gov/~jamesh/diffuse_scatter/xtals_Isum_minus_Fsum.g
> if
>   
>> So, in the first case, the partial-occupancy model produced exactly
>>     
> the
>   
>> same background-subtracted spot intensities as the "unsynchronized
>> disorder" case, but this was not so when the disorder was
>>     
> synchronized.
>   
>> What did I do wrong?
>>
>> As far as my "operational" definition of a "Bragg peak" (a term which
>> already has two definitions), I am merely suggesting that the
>> nearly-universal practice of subtracting the "local background" is a
>> very pragmatic definition of a "spot intensity".  Nearly all available
>> data were collected in this way, and it actually is a reasonable thing
>> to do if the disorder from cell to cell is uncorrelated (as evidenced
>> above).
>>
>> However, I totally agree with you that the disorder in protein
>>     
> crystals
>   
>> may well be correlated across large patches of unit cells.  If that is
>> the case, then the "average occupancy model" that is all but
>>     
> universally
>   
>> implemented by refinement programs will never be able to explain the
>> background-subtracted spot intensities.
>>
>> -James Holton
>> MAD Scientist
>>
>>
>> Ian Tickle wrote:
>>     
>>>> If all cells are completely unsynchronized, then the
>>>>
>>>>         
>>> occupancy-weighted
>>>
>>>       
>>>> average electron density map of all the conformers will fully
>>>>         
> explain
>   
>>>> the background-subtracted spot intensities, but if there is
>>>> cell-to-cell synchronization: it won't!
>>>>
>>>>         
>>> This is not correct: as I tried to explain in a previous posting,
>>>       
> the
>   
>>> 'optic' mode DS component which arises from what I would call 'short
>>>       
> to
>   
>>> medium range' correlated displacements (that is correlations due to
>>> rigid side-chain motions, or of secondary-structure units,
>>>       
> individual
>   
>>> helices say, or of whole domains within the same molecule, or of
>>> different molecules within the same unit cell), give rise to a
>>> non-uniform DS distribution over the *whole* diffraction pattern.
>>>       
> You
>   
>>> can't assume that the contributions of the optic DS at the Bragg
>>> positions are zero just because they can't be measured!  From the DS
>>> equation there's absolutely no reason why the DS should be anything
>>> other than non-uniform at the Bragg position as anywhere else.
>>>       
> Since
>   
>>> it's equally non-uniform over the whole pattern, including at and
>>>       
> around
>   
>>> the Bragg positions, a planar background correction can't possibly
>>> remove it from the integrated Bragg intensities.  So it's simply not
>>> correct to say that the mean electron density explains all the
>>>       
> intensity
>   
>>> at the Bragg positions.  There will be a residual I(diffuse) =
>>> I(coherent) - I(Bragg) which is everywhere positive, as I
>>>       
> demonstrated.
>   
>>> I agree with you that what I would call 'long-range' correlations
>>> between different unit cells contribute largely to the 'acoustic'
>>>       
> mode
>   
>>> DS which is centred largely *at* the Bragg peaks.  You say 'if' the
>>> cells are completely unsynchronised, but that's a big 'if' -
>>>       
> certainly
>   
>>> you can't simply assume that it's true.
>>>
>>> On another point you said you wanted an 'operational' definition of
>>> I(Bragg).  I'm not entirely clear what you mean by that.  Are you
>>>       
> saying
>   
>>> that you want I(Bragg) to be the total background-subtracted
>>>       
> integrated
>   
>>> intensity under the peak at the Bragg position, i.e. what I'm
>>>       
> calling
>   
>>> I(coherent).  If so then it can't be the contribution from the mean
>>> density at the same time! - seems to me that's what everyone means
>>>       
> by
>   
>>> I(Bragg) (including you I thought!) so changing the definition will
>>> cause total confusion!
>>>
>>> Cheers
>>>
>>> -- Ian
>>>
>>>
>>> Disclaimer
>>> This communication is confidential and may contain privileged
>>>       
>> information intended solely for the named addressee(s). It may not be
>>     
> used
>   
>> or disclosed except for the purpose for which it has been sent. If you
>>     
> are
>   
>> not the intended recipient you must not review, use, disclose, copy,
>> distribute or take any action in reliance upon it. If you have
>>     
> received
>   
>> this communication in error, please notify Astex Therapeutics Ltd by
>> emailing [log in to unmask] and destroy all copies of the
>> message and any attached documents.
>>     
>>> Astex Therapeutics Ltd monitors, controls and protects all its
>>>       
> messaging
>   
>> traffic in compliance with its corporate email policy. The Company
>>     
> accepts
>   
>> no liability or responsibility for any onward transmission or use of
>> emails and attachments having left the Astex Therapeutics domain.
>>     
> Unless
>   
>> expressly stated, opinions in this message are those of the individual
>> sender and not of Astex Therapeutics Ltd. The recipient should check
>>     
> this
>   
>> email and any attachments for the presence of computer viruses. Astex
>> Therapeutics Ltd accepts no liability for damage caused by any virus
>> transmitted by this email. E-mail is susceptible to data corruption,
>> interception, unauthorized amendment, and tampering, Astex
>>     
> Therapeutics
>   
>> Ltd only send and receive e-mails on the basis that the Company is not
>> liable for any such alteration or any consequences thereof.
>>     
>>> Astex Therapeutics Ltd., Registered in England at 436 Cambridge
>>>       
> Science
>   
>> Park, Cambridge CB4 0QA under number 3751674
>>     
>>>
>>>       
>
>
>
> Disclaimer
> This communication is confidential and may contain privileged information intended solely for the named addressee(s). It may not be used or disclosed except for the purpose for which it has been sent. If you are not the intended recipient you must not review, use, disclose, copy, distribute or take any action in reliance upon it. If you have received this communication in error, please notify Astex Therapeutics Ltd by emailing [log in to unmask] and destroy all copies of the message and any attached documents. 
> Astex Therapeutics Ltd monitors, controls and protects all its messaging traffic in compliance with its corporate email policy. The Company accepts no liability or responsibility for any onward transmission or use of emails and attachments having left the Astex Therapeutics domain.  Unless expressly stated, opinions in this message are those of the individual sender and not of Astex Therapeutics Ltd. The recipient should check this email and any attachments for the presence of computer viruses. Astex Therapeutics Ltd accepts no liability for damage caused by any virus transmitted by this email. E-mail is susceptible to data corruption, interception, unauthorized amendment, and tampering, Astex Therapeutics Ltd only send and receive e-mails on the basis that the Company is not liable for any such alteration or any consequences thereof.
> Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science Park, Cambridge CB4 0QA under number 3751674
>
>
>