Hi Folks,

The idea of comparing different data reduction packages is an
interesting one. It is not however without challenges as alluded to by
Kay. As far as I can see there are two main challenges - the first is
that different users when given the same tools will do different things
- in challenging cases this will be very significant [Kay's point]. The
second is to be able to express why a given package did better than
others for a specific case, then bootstrap your way back to some general
rules (i.e. "expertise") about the packages for future reference. For a
single case this was looked at in (ahem):

Acta Cryst. (2007). F63, 168-172 Structure of 5-formyltetrahydrofolate
cyclo-ligase from Bacillus anthracis (BA4489)
C. Meier , L. G. Carter, G. Winter, R. J. Owens, D. I. Stuart and R. M.
Esnouf 

Where we concluded that XDS did a better job than Mosflm or Denzo
because the mosaic spread was rather high and the oscillations a little
narrow. We did not try d*TREK but I would anticipate it would work well
as the method of integration is similar to XDS. The "better job" in
question though was to give a data set which would give a refinable
molecular replacement solution, where the others did not.

Now something which I have been interested in doing but which is almost
certainly prohibitively expensive in terms of time is to take a number
of data sets of varying levels of challenge and get the program authors
(or delegated experts) to do their absolute best with the data reduction
with that program, then compare the results. This will help to reduce
the impact of lesser known but very helpful options (i.e. outlier
recycling in XDS CORRECT, say) and perhaps generate a more fair
comparison. In the example above there may have been some "tweaks" which
could have been made to improve the results substantially for the 2D
integration packages.

For most (easy) cases however the best tool for the job is the one you
know best. If you have the spare CPU horsepower and time, you can always
reduce the data with all available packages and see which one gives the
best results for your case - something which would be perfectly
reasonable for substructure determination but much more unusual for data
reduction. From my testing of xia2 against structural genomics data I
can say that, for easy data, there is not much to call between Mosflm /
Scala and XDS / XSCALE.

Just my 2c.

Cheers,

Graeme