Thank you Kay!
Speaking as a beamline scientist I agree that the instrument should be
set up right and things like the distance, beam center, and various
angles should be known.
However, also speaking as a beamline scientist I feel I should point out
that EVERY instrument goes through at least one period where this is NOT
the case! Those of us who are responsible for putting all those
properly-calibrated numbers into the image headers before your beam time
starts really do appreciate the "feature" of broad radius of convergence
on camera parameters. But if you can get better data by turning it off
once the parameters are known, that is really interesting too!
-James Holton
MAD Scientist
On 1/31/2018 1:07 AM, Kay Diederichs wrote:
> Dear all,
>
> a new BUILT of XDS is available for academic users at http://xds.mpimf-heidelberg.mpg.de . This reverts to the old distance refinement behavior with its larger radius of convergence, and is relevant for processing data from beamlines where the header distance is not accurate.
>
> Thanks again to Oliver and Keitaro!
>
> Kay
>
> On Fri, 26 Jan 2018 16:56:30 +0900, Keitaro Yamashita <[log in to unmask]> wrote:
>
>> Dear Kay,
>>
>> I also tested this using a publicly available data of thaumatin
>> https://zenodo.org/record/10271
>> (resolution ~1.4 Å, wavelength= 0.97625 Å).
>> The camera distance from header is 265.27 mm. I tested this original
>> distance and some shifts (+1, +2, +4, ..., +32 mm) with different
>> versions of XDS.
>>
>> As a result, IDXREF of the old version (170615) successfully refined
>> camera distance to ~265.0 mm if shift was less than or equal to +16
>> mm. The statistics in CORRECT.LP was as good as that without shift.
>>
>> If the camera distance was fixed in IDXREF, statistics was
>> compromised, but the refinement in CORRECT worked well and recycling
>> of GXPARM.XDS improved the data quality.
>>
>> However, the new version (170720 or later) changed camera distance
>> very little. Even with +1 mm shift the statistics was deteriorated,
>> and CORRECT (GXPARM.XDS) also didn't change the distance a lot, which
>> means the recycling of optimized parameters do not help. The new
>> version seems to stick to the given camera distance too much as
>> pointed out by many people.
>>
>> I agree that camera distance in the header should be accurate.
>> However, I also wish to revert to the former behavior of XDS that
>> expires very soon.
>>
>> Actually in previous versions (170615 or former) fixing POSITION in
>> REFINE(IDXREF) was recommended, but the recycling could give an
>> accurate distance. The current version seems to virtually have no
>> option to refine the distance.
>>
>> Best regards,
>> Keitaro
>>
>> On 24 January 2018 at 05:16, Kay Diederichs
>> <[log in to unmask]> wrote:
>>> Dear Oliver,
>>>
>>> sorry for the trouble!
>>> A default should be correct in the majority of situations, but it's impossible to have it work for _all_ situations. The XDS default for REFINE(IDXREF) was changed (i.e. POSITION was removed) to improve the indexing for weak and lousy data, _and_ because the distance values from the header are nowadays almost always accurate. For data with very high resolution such as yours, and significantly wrong distance, this means that at high resolution in particular this default will lead to worse data. generate_XDS.INP (in XDSwiki) and (I think) autoPROC and xia2 explicitly include POSITION in the REFINE(IDXREF), i.e. override the default. Where did you get XDS.INP from?
>>> Some comments:
>>> a) this is why I recommend, for data sets that may be important, to always do one cycle of optimization, i.e. after CORRECT, "mv GXPARM.XDS XPARM.XDS", change XDS.INP to have the correct high-resolution cutoff (cutting after the last shell which still has a "*" in the CC1/2 column), adjust the FRIEDEL'S_LAW= setting, and run JOB=INTEGRATE CORRECT. In your case this would make the refined distance available to INTEGRATE, and should result in very good data.
>>> b) at which beamline did you collect the data? Such a high difference between refined distance and distance from header is unusual, and should be reported to (and fixed by) the beamline staff.
>>> c) for this beamline at least, one should use REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS POSITION . I understand that you tried this and it didn't work? Maybe there is something else wrong then, e.g. another line later in XDS.INP resetting REFINE(IDXREF) to something else.
>>>
>>> If you cannot find the reason why including POSITiON into REFINE(IDXREF) does not help, pls send me enough data to reproduce the problem.
>>>
>>> best wishes,
>>>
>>> Kay
>>>
>>> On Tue, 23 Jan 2018 14:31:09 +0000, "Weiergräber, Oliver H." <[log in to unmask]> wrote:
>>>
>>>> Dear all,
>>>>
>>>> After upgrading XDS from build date 20170601 to 20171218, I am experiencing severe degradation of apparent data quality reported by CORRECT for certain data sets. Following first indications of issues with a slightly problematic candidate, I went back to a previously very well-behaved test case with diffraction extending beyond 1.1 A. Using the same input with both program versions, statistics are not too different out to approx. 1.6 A, but become more and more divergent in outer shells. These are the values for the highest-resolution shell (1.10--1.16 A):
>>>>
>>>> 20170601: I/sigI = 1.80; Rmeas = 59.9 %; CC(1/2) = 82.0 %
>>>> 20171218: I/sigI = 0.14; Rmeas = 506.4 %; CC(1/2) = 5.8 %
>>>>
>>>> The refined cell constants are unreasonably different as well. Obviously, something is going awfully wrong here, presumably related to errors in geometry parameters (which are expected to become increasingly detrimental with decreasing dmin). As it turns out, geometry refinement behaves differently in the latest version of XDS: most notably, IDXREF no longer refines the detector distance by default. This is significant in this case, as in version 20170601 the distance would shift by as much as 1.3 mm, resulting in successful integration and scaling. Unfortunately, re-adding POSITION refinement into REFINE(IDXREF) does not help, and even having it in all refinement scopes (IDXREF, INTEGRATE, CORRECT) only yields a limited improvement of CORRECT statistics.
>>>> It is worth noting that examination of a dataset from an unrelated crystal (dmin = 1.4 A) did not reveal such enormous differences -- in this case, however, the shift in detector distance applied in the old version of XDS was quite small (0.2 mm), which, together with the lower resolution, explains the absence of large discrepancies.
>>>>
>>>> To sum up, I suspect that, due to recent changes to the XDS code, the refinement of geometry parameters is now overly restrained, resulting in drastic problems in cases where the detector distance is not as precise as desirable (and even more so at very high resolution). For such datasets, the new version appears to be essentially unusable (at least within the parameter space I have tested), and even in modest cases may often be inferior to the previous one. Is there an option to revert to the former behaviour?
>>>>
>>>> Best regards
>>>> Oliver
>>>>
>>>> ================================================
>>>> PD Dr. Oliver H. Weiergr�ber
>>>> Institute of Complex Systems
>>>> ICS-6: Structural Biochemistry
>>>> Tel.: +49 2461 61-2028
>>>> Fax: +49 2461 61-9540
>>>> ================================================
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------------------------
>>>> ------------------------------------------------------------------------------------------------
>>>> Forschungszentrum Juelich GmbH
>>>> 52425 Juelich
>>>> Sitz der Gesellschaft: Juelich
>>>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
>>>> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
>>>> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
>>>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
>>>> Prof. Dr. Sebastian M. Schmidt
>>>> ------------------------------------------------------------------------------------------------
>>>> ------------------------------------------------------------------------------------------------
|