Dear Gerard,
Many thanks for these useful clarifications.I see your points clearly.
Just to mention that one remark in James's posting regarding photon counting versus read noise caught my attention. I will follow up on this ASAP , which like fine phi slicing gets to the heart of the measurement physics.
Greetings,
John
Prof John R Helliwell DSc FInstP CPhys FRSC CChem F Soc Biol.
Chair School of Chemistry, University of Manchester, Athena Swan Team.
http://www.chemistry.manchester.ac.uk/aboutus/athena/index.html
On 19 May 2013, at 21:06, Gerard Bricogne <[log in to unmask]> wrote:
> Dear John,
>
> Thank you for your message. I do realise that what I wrote may
> have sounded like a "categoric blanket endorsement of XDS". Perhaps I
> should have slept on my draft message a little longer, but James's
> long e-mail made me wake up and it seemed appropriate to say what I
> wanted to say, in the form I had written it, withouth further delay.
>
> My main point was to try and shake people out of habits that
> are, in part at least, linked to the use of integration programs still
> based on a 2D analysis of diffraction images, and to getting misled
> into thinking that fine slicing doesn't help because it doesn't help
> these programs. What I most wanted to put across is that the merits of
> fine slicing and low-exposure, high-multiplicity collection protocols
> will emerge faster if people are encouraged to evaluate the data they
> yield through processing with XDS - hence my strong endorsement of it
> in this context.
>
> I do not consider XDS to be beyond perfectibility by any means,
> but I was reluctant to make my e-mail longer by analysing its possible
> improvements. Once its use has spread enough to give users of Pilatus
> detectors the best data they can hope for in the current state of the
> art, then its limitations will make concrete sense to enough people
> for their discussion to move spontaneously to the top of the agenda.
> It is indeed because I think there is so much scope for improvement on
> the current diffraction image processing software, including XDS, that
> I have been strongly (some would perhaps say: stridently) advocating
> the deposition of raw diffraction image data, as both an incentive and
> a testing ground for such developments in the future.
>
>
> With best wishes,
>
> Gerard.
>
>
>
> On Sun, May 19, 2013 at 08:01:34PM +0100, John R Helliwell wrote:
>> Dear Gerard,
>> Thank you for sharing these extensive details which I feel sure everyone
>> will appreciate.
>> Just one aspect I wondered about namely your categorical blanket
>> endorsement of XDS. Indeed a very fine program and eg most recently
>> evaluated and discussed at CCP4 2011 I think it was, where it emerged 'the
>> winner'. You probably guess though that I am thinking of our mutually
>> emphasized point that one key reason for raw diffraction data images
>> availability is to see such software improve. Is XDS already perfection? Is
>> its use by users already guaranteed to yield processed data as good as it
>> can get?
>> Greetings,
>> John
>> Prof John R Helliwell DSc
>>
>>
>> On Thu, May 16, 2013 at 6:03 PM, Gerard Bricogne <[log in to unmask]>wrote:
>>
>>> Dear James,
>>>
>>> A week ago I wrote what I thought was a perhaps excessively long and
>>> overly dense message in reply to Theresa's initial query, then I thought I
>>> should sleep on it before sending it, and got distracted by other things.
>>>
>>> I guess you may well have used that whole week composing yours ;-) and
>>> reading it just now makes the temptation of sending mine irresistible. I am
>>> largely in agreement with you about the need to change mental habits in
>>> this
>>> field, and hope that the emphasis on various matters in my message below is
>>> sufficiently different from yours to make a distinct contribution to this
>>> very important discussion. Your analysis of pile-up effects goes well
>>> beyond
>>> anything I have ever looked at. However, in line with Theresa's initial
>>> question, I would say that, while I agree with you that the best strategy
>>> for collecting "native data" is no strategy at all, this isn't the case
>>> when
>>> collecting data for phasing. In that case one needs to go back and consider
>>> how to measure accurate differences of intensities, not just accurate
>>> intensities on their own. That is another subject, on which I was going to
>>> follow up so as to fully answer Theresa's message - but perhaps that should
>>> come in another installment!
>>>
>>>
>>> With best wishes,
>>>
>>> Gerard.
>>>
>>> --
>>> On Tue, May 07, 2013 at 12:04:33AM +0100, Theresa Hsu wrote:
>>>> Dear crystallographers
>>>>
>>>> Is there a good source/review/software to obtain tips for good data
>>> collection strategy using PILATUS detectors at synchrotron? Do we need to
>>> collect sweeps of high and low resolution data separately? For anomalous
>>> phasing (MAD), does the order of wavelengths used affect structure solution
>>> or limit radiation damage?
>>>>
>>>> Thank you.
>>>>
>>>> Theresa
>>> --
>>>
>>> Dear Theresa,
>>>
>>> You have had several excellent replies to your question. Perhaps I
>>> could venture to add a few more comments, remarks and suggestions, which
>>> can
>>> be summarised as follows: with a Pilatus, (1) use fine slicing, (2) use
>>> strategies combining low exposure with high multiplicity, and (3) use XDS!
>>>
>>> As the use of Pilatus detectors has spread widely, it has been rather
>>> puzzling to come across so many instances when these detectors are misused,
>>> sometimes on the basis of explicit expert advice that is simply misguided.
>>> A
>>> typical example will be to see images collected on a Pilatus 6M with an
>>> image width of 1 degree and an exposure time of 1 second. When you see
>>> this,
>>> you know that there is some erroneous thinking (or habit) behind it.
>>>
>>> When talking to various users who have ended up with such datasets,
>>> and
>>> with people who advocate this kind of strategy, it seems clear that a
>>> number
>>> of irrational concerns about fine-slicing and
>>> low-exposure+high-multiplicity
>>> strategies have tended to override published rational arguments in favour
>>> of
>>> those strategies: there is a fear that if the images being collected do not
>>> show spots discernible by the naked eye to the resolution limit that is
>>> being aimed for, the integration software will then somehow not be able to
>>> find those spots in order to integrate them, and the final data resolution
>>> will be lower than expected. Perhaps this may be of concern in relation
>>> with
>>> the use of some integration programs, but if you use XDS, which implements
>>> a
>>> full 3D approach to image integration, this is simply not the case: XDS
>>> will
>>> collect all the counts belonging to a given reflection, whether those
>>> counts
>>> are all from a spot on a single 1-degree image exposed for 1 second, or
>>> from
>>> 10 consecutive images of 0.1 degree width exposed for 0.1 second each, or
>>> from 100 images obtained by grouping together the same 10 images as
>>> previously collected in 10 successive passes with a 10-fold attenuated
>>> beam.
>>> The hallmark of the Pilatus detector is to lead to equivalent signal/noise
>>> ratios for the last two ways of measuring that reflection, because it is a
>>> photon counter and has zero readout noise: therefore the combination
>>> Pilatus+XDS is a powerful one.
>>>
>>> What is different between these three strategies, however, is the
>>> quality of the overall dataset they will produce. There is nothing new in
>>> what I am describing below: it is all in the references that Bob Sweet gave
>>> you in his reply, or is an obvious consequence of what is found in these
>>> references.
>>>
>>> In case 1 (1-degree, 1 second - "coarse slicing") you would presumably
>>> also be (mis-)advised to use a strategy aiming at collecting a complete
>>> dataset in the minimum number of images. These strategies used to make
>>> sense
>>> in the days of films, of image plates, and even of CCDs because of the
>>> image
>>> readout noise, but they have no place any longer in the context of Pilatus
>>> detectors. First of all, using 1-degree image widths can only degrade the
>>> precision with which 2D spots on images are lifted to 3D reciprocal space
>>> for indexing, and hence worsen the quality of that indexing and therefore
>>> the accuracy with which the spot locations will be predicted (unless you
>>> carefully "post-refine") - then the integration step perhaps does need to
>>> "hunt" for those spots locally, and needs them to be somewhat visible.
>>> Secondly, 1 degree is usually greater than the angular width of a typical
>>> reflection: the integration process will therefore pick up more background
>>> noise (variance) than it would have done with a smaller image width.
>>> Thirdly, by collecting only enough images to reach completeness you will
>>> have substantial radiation damage in your late images compared to the early
>>> ones (if you don't, it means you have under-exposed your crystal) and will
>>> therefore end up with internal inconsistencies in your dataset, as well as
>>> perhaps some extra, spurious anisotropy of diffraction limits as a result
>>> of
>>> having to impose increasingly stringent resolution cut-offs in the later
>>> images. This will affect the internal scaling of that dataset and the final
>>> quality of the merged data.
>>>
>>> In case 2 (0.1 degree, 0.1 second - "fine slicing") you will have a
>>> more precise sampling of the 3D shape of each spot, hence more accurate
>>> indexing and prediction of spot positions if you use a genuinely 3D
>>> integration program like XDS. Thanks to that increased precision, spots can
>>> be integrated "blind", even if they are not terribly visible in the images,
>>> and the same number of photons will be collected with no penalty in terms
>>> of
>>> noise level, thanks to the photon-counting noiseless-readout nature of the
>>> Pilatus detector. An improvement will be that the finely sampled 3D shape
>>> of
>>> the spots will be used by XDS to minimise the impact of background variance
>>> on the integrated intensities. On the other hand, the differential
>>> radiation
>>> damage between early and late images will still be the same as in case 1 if
>>> you have chosen one of those old-style strategies (and associated beam
>>> intensity setting) that aim at just about exhausting the useful lifetime of
>>> the crystal by the time you reach completeness.
>>>
>>> In case 3 (like case 2, but collecting n times more images with an
>>> n-fold attenuated beam once you have collected a few "characterisation
>>> images" without that attenuation to carry out the initial indexing) you
>>> still have the two advantages of case 2 (the same total number of photons
>>> will be picked up by XDS, even if the individual images are now so weak
>>> that
>>> you can't see anything) but you are spreading the radiation damage so
>>> thinly
>>> over multiple successive complete datasets that you can choose to later
>>> apply a cut-off on image number at the processing stage, when the
>>> statistics
>>> tell you that diffraction quality has become degraded beyond some critical
>>> level. This is much preferable to having to apply different resolution
>>> cut-offs to different images towards the end of a barely complete dataset,
>>> as in cases 1 and 2. The impact of radiation damage will be quite smoothly
>>> and uniformly distributed across the final unique reflections, and your
>>> scaling problems (as well as any spurious anisotropy in your diffraction
>>> limits) will be minimised.
>>>
>>>
>>> This is becoming quite a long message: you can see why I included a
>>> summary of it at the beginning! Returning to it for a conclusion: Pilatus
>>> detectors, fine-slicing with low-exposure and high-multiplicity strategies,
>>> and XDS are a unique winning combination. If fears that another integration
>>> program may not perform as well as XDS on fine-sliced data make you feel
>>> tempted to revert to old-fashioned strategies (case 1) because it
>>> supposedly
>>> makes no difference: resist the temptation! Switch to those Pilatus-adapted
>>> strategies and to XDS, and enjoy the very real difference in the results!
>>>
>>>
>>> With best wishes,
>>>
>>> Gerard
>>>
>>> and colleagues at Global Phasing.
>>>
>>> --
>>> On Tue, May 07, 2013 at 12:04:33AM +0100, Theresa Hsu wrote:
>>>> Dear crystallographers
>>>>
>>>> Is there a good source/review/software to obtain tips for good data
>>> collection strategy using PILATUS detectors at synchrotron? Do we need to
>>> collect sweeps of high and low resolution data separately? For anomalous
>>> phasing (MAD), does the order of wavelengths used affect structure solution
>>> or limit radiation damage?
>>>>
>>>> Thank you.
>>>>
>>>> Theresa
>>>
>>> --
>>>
>>> ===============================================================
>>> * *
>>> * Gerard Bricogne [log in to unmask] *
>>> * *
>>> * Global Phasing Ltd. *
>>> * Sheraton House, Castle Park Tel: +44-(0)1223-353033 *
>>> * Cambridge CB3 0AX, UK Fax: +44-(0)1223-366889 *
>>> * *
>>> ===============================================================
>>>
>>
>>
>>
>> --
>> Professor John R Helliwell DSc
>
> --
>
> ===============================================================
> * *
> * Gerard Bricogne [log in to unmask] *
> * *
> * Global Phasing Ltd. *
> * Sheraton House, Castle Park Tel: +44-(0)1223-353033 *
> * Cambridge CB3 0AX, UK Fax: +44-(0)1223-366889 *
> * *
> ===============================================================
|
