I think the best way to deal with issues like this can be found in
Strunk & White "The Elements of Style" (1918). Among other things,
these authors put forward a rather simple yet often overlooked rule to
writing in general, which I think applies equally well to computer programs:
"Be clear."
The sentence itself is an example of how brevity need not sacrifice
clarity. Yes, you need labels in the table itself to be short, but
there is space immediately below (and above) every table that (IMHO)
ought to contain the definitions of each and every variable/abbreviation
used in the table, spelled out no matter how obvious it may seem to the
author. I can tell you many long and painful stories about me trying to
figure out what some variable in some equation in some paper actually
meant! Context is everything. If you are tight for space, cite a
reference (such as the manual).
That, and scientists talking about such quantities in email, papers,
etc. (such as myself) should also heed Strunk and White and also not
just assume that everyone knows exactly what "structure factor" means as
opposed to a "structure amplitude", let alone I/sigma. Indeed, the word
"intensity" is an incredibly ill-defined unit all by itself, to the
point of being useless. It can have units of photons, photons/s,
photons/area/s, photons/area, energy/volume, and many many more. Often
even in the same equation!
I would strongly advise against changing the "variable names" printed
out in log files by SCALA and other programs, especially when a given
name has persisted for a decade or more. Adding an "inline definition"
is fine, but changing names not only breaks programs that were written
to read these logs (and sometimes even humans reading the log), but it
also confines the meaning of "I/SIGMA from SCALA" to a particular period
in history.
So, what statistic do we want to look at? That depends on what you are
trying to do with the data. There is no way for Phil to know this, so
it is good that he prints out lots of different statistics. That said,
when talking about the data quality requirements for structure solution
by MAD/SAD, I suggest looking at I/sigma(I) where:
I - merged intensity (proportional to photons) assigned to a
reciprocal lattice point (hkl index)
sigma(I) - the error assigned to I
Exactly what I/sigma(I) is required to solve a structure, or make some
conclusion about a solved structure is a topic for another day.
-James Holton
MAD Scientist
Phil Evans wrote:
> “I/sigma” statistics seem to be contentious & confusing (see recent
> discussions on CCP4BB), particularly in what the various measures
> should be called (and how they should be labelled in a table, where
> there is only room for a very short name). I thought it worth
> commenting on this issue at a little more length.
>
> There are several interacting issues:
>
> 1) Statistics can be calculated either for individual observations Ihl
> or for intensities averaged over multiple (symmetry-related or
> replicate) measurements Ih(avg): both are useful, but they need to be
> distinguished
>
> 2) The statistic can be (a) the ratio of means <I>/<sigma> or (b) the
> mean of ratios <I/sigma> . These are not the same.
>
> 3)The “sigma” used in 2(a) can be either (a) the estimated corrected
> SD or (b) the RMS scatter of observations ie the RMS deviation (which
> is itself generally used to estimate a “correction” to the SD). The
> RMS scatter cannot be used for 2(b) of course, since that needs
> individual sigmas for each reflection.
>
> 4) Values will depend on how many outliers have been rejected.
>
> For what it’s worth, Scala outputs two such statistics:-
>
> (i) “I/sigma”: this is calculated for individual observations Ihl and
> is the (mean intensity <Ihl>)/(RMS scatter of Ihl). RMS scatter = RMS
> [Ihl – Ih(avg)]. This is some measure of the average significance of
> individual observations, but does not take into account multiplicity.
> In my new program under development (a Scala replacement) I have
> relabelled this column “I/RMS” but I don’t really know what best to
> call it. This value is a ratio of means (see 2(a) above).
>
> (ii) “Mn(I/sd)”: this is the mean value of (Ih(avg)/sd(Ih(avg))),
> where Ih(avg) is the (weighted) average over all observations for
> reflection h, and sd(Ih(avg)) is the estimated SD of this average,
> after any “corrections” have been applied. This is, I think, the best
> estimate of “signal-to-noise ratio”, but does depend on realistic
> estimates of sd(Ih(avg)), which is not entirely straightforward (and
> certainly doesn’t allow for systematic errors!). This value is a mean
> of ratios (see 2(b) above).
>
>
>
> The “corrected” sd(Ihl) is calculated in Scala for each observation as
> sd(Ihl)corrected = SdFac * sqrt{sd(I)**2 + SdB*Ihl*LP +
> (SdAdd*Ihl)**2}
> with the parameters SdFac, SdB & SdAdd determined by trying to make
> the RMS normalised deviation Delta(hl) = (Ihl -
> Ih(avg))/sd(Ihl)corrected = 1.0 for all intensity ranges (different
> parameters for each run). If the sd estimates are correct, then the
> distribution of Delta(hl) should have SD = 1.0, and this “correction”
> tries to enforce this. This is more or less equivalent to making the
> RMS scatter == average SD. However the uncertainties in how best to
> estimate the real error do then influence the reliability of the
> Mn(I/sd) statistic (see (ii) above)
>
> So what statistics do we want to look at? Probably the main reason for
> looking at signal/noise statistics is to choose a “real resolution”
> cutoff, from some sort of signal/noise ratio. It isn’t clear (to me)
> what is the best way of doing this, and it is particularly difficult
> if the data are significantly anisotropic. The multiplicity needs to
> be taken into account, so the individual “I/sigma” (see (i) above)
> isn’t the best guide. Personally, I generally cut data at around the
> point where Mn(I/sd) =~ 2, but I would cut off at <2 for anisotropic
> data. I also find a useful guide from the correlation coefficient
> between Ih(avg) (Imean) pairs in half-datasets (plotted by Scala): the
> CC should be >0.5 at least, I think.
>
> Note that the overall value of any of these statistics over all
> resolution ranges is not very useful and can be confusing, depending
> on the distribution of intensities, since it mixes up strong low
> resolution data (high signal/noise) with weak high resolution data
> (low signal/noise).
>
> That leaves the question of how to label these statistics in a
> consistent, clear and concise way: suggestions?
>
> Phil Evans
|