I should also add to Kay's list the following:

The tests we performed assumed that the default values of the programs (d*Trek, Mosflm 6.13?, XDS) in ~2006 were reasonable, so with tweaking you might get a better milage, but that was not the point when we had to deal with lots of data and no time to fool around with each individual data set. So the question we asked was more like this: with minimal user input which program allows you to progress further (and faster) in structure solution ?

I think (and correct me if I'm wrong) but the folks at JCSG also used XDS as their primary processing program, what was the reason for that ?

Jürgen

P.S. Thanks for the great script you are linking to on the website, very useful and much more sophisticated than the one I wrote.
-
Jürgen Bosch
Johns Hopkins Bloomberg School of Public Health
Department of Biochemistry & Molecular Biology
Johns Hopkins Malaria Research Institute
615 North Wolfe Street, W8708
Baltimore, MD 21205
Phone: +1-410-614-4742
Lab:      +1-410-614-4894
Fax:      +1-410-955-3655
http://web.mac.com/bosch_lab/

On Jan 30, 2011, at 1:25 PM, Kay Diederichs wrote:

Am 20:59, schrieb Van Den Berg, Bert:
I have heard this before. I’m wondering though, does anybody know of a
systematic study where different data processing programs are compared
with real-life, non-lysozyme data?

Bert

Bert,

some time ago I tried to start something to this effect - take a look at
the "Quality Control" article in XDSwiki.
(<http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Quality_Control>).
But it hasn't worked out, i.e. nobody participated (so far).
Possible reasons include:
a) it is considered politically incorrect (many years ago I wrote about
a comparison that I did ... the reactions from a few people were rather
harsh)
b) for reasons un-intelligible to me, people do not like to make their
raw data public (even if I ask directly)
c) it does take time to do and document
d) it's difficult to agree on the right methodology
e) it's a question that seems to interest only specialists
f) there's probably not a single answer
g) the programs are being constantly improved

Concerning the last point, a wiki seems to be a good place to collect
the results (a table can be used to follow progress in a program, but
also to see the differences between programs). But that brings me to my
last point - a wiki article does not count as a paper.

best,

Kay