Dear Mohamed (and BB),



As Kay implicitly mentioned, in case of problems, you should look at the data (the frames) and not at the statistics alone! The human eye is unsurpassed in image analysis and by looking at the frames you will immediately see if there is a problem or not. If the frames look good, keep them, if they look bad, you can consider excluding them. E.g. the ADXV program has an option to play a movie of your data set, so you can quickly scan your complete data set. 



For fine-sliced Pilatus data, you may have to merge every 10 frames or so, to get 1° images which you can analyze by eye, but there are programs to do this.



Best,

Herman



-----Ursprüngliche Nachricht-----

Von: CCP4 bulletin board [mailto:[log in to unmask]] Im Auftrag von Kay Diederichs

Gesendet: Donnerstag, 21. Januar 2016 21:55

An: [log in to unmask]

Betreff: Re: [ccp4bb] Excluding frames in XDS



Dear Mohamed,



such a plot is close to meaningless without knowing what the numbers of reflections are from which it was calculated. Furthermore, it would be more interesting to scale to a maximum of 100%; a "smiley" shape would at least tell you something about possible radiation damage.



I firmly believe that in >90% of cases where people want to reject specific frames or frame range they do so due to a lack of understanding of statistics, and this actually harms the accuracy of the merged data. Throwing away data should never be done in order to beautify Rsym/Rmeas, rather you must have _another_ really good reason for discarding them:

- radiation damage at the end of data collection

- crystal rotating out of the beam 

- cryo nozzle casting shadow on detector

- detector problems, beam problems, ...

In other words, you should be able to say _why exactly_ you want to throw away these data.



just my 2 cents,



Kay



On Thu, 21 Jan 2016 00:47:20 +0000, Mohamed Noor <[log in to unmask]> wrote:



>Dear all

>

>I have a 180-degree dataset (1800 frames) collected at peak wavelength for Fe (for an MR-SAD structure solution). While analyzing the XDS_ASCII.HKL file with Aimless, I noticed something went wrong with a few frames around 1400-1600 (the Rmerge vs. batch plot is attached). When I processed only the first 1200 frames, Aimless indicated a significant anomalous signal up to 3.5 A whereas with the whole dataset (1800 frames), it was only about 7 A. I am trying to process the first 1200 or so images and the last 200 with XDS. How can I achieve this when XDS accepts only one DATA_RANGE line?

>

>As this was the only crystal diffracted, I am trying to squeeze as much as I can out of what I have. The point group is either P 6 or P 6 2 2, not entirely sure yet.

>

>Thanks.

>Mohamed

>