Douglas,
>> Observed intensities are the best estimates that we can come up with in an experiment.
> I also agree with this, and this is the clincher. You are arguing that Ispot-Iback=Iobs is the best estimate we can come up with. I claim that is absurd. How are you quantifying "best"? Usually we have some sort of discrepancy measure between true and estimate, like RMSD, mean absolute distance, log distance, or somesuch. Here is the important point --- by any measure of discrepancy you care to use, the person who estimates Iobs as 0 when Iback>Ispot will *always*, in *every case*, beat the person who estimates Iobs with a negative value. This is an indisputable fact.
First off, you may find it useful to avoid such words as absurd and
indisputable fact. I know political correctness may be sometimes
overrated, but if you actually plan to have meaningful discussion, let's
assume that everyone responding to your posts is just trying to help
figure this out.
To address your point, you are right that J=0 is closer to "true
intensity" then a negative value. The problem is that we are not after
a single intensity, but rather all of them, as they all contribute to
electron density reconstruction. If you replace negative Iobs with
E(J), you would systematically inflate the averages, which may turn
problematic in some cases. It is probably better to stick with "raw
intensities" and construct theoretical predictions properly to account
for their properties.
What I was trying to tell you is that observed intensities is what we
get from experiment. They may be negative, and there is nothing
unphysical about it. Then you build a theoretical estimate of observed
intensities, and if you do it right (i.e. by including experimental
errors), they will actually have some probability of being negative.
>> This background has to be subtracted and what is perhaps the most useful form of observation is Ispot-Iback=Iobs.
> How can that be the most useful form, when 0 is always a better estimate than a negative value, by any criterion?
Given your propensity to refer to what others might say as absurd, I am
tempted to encourage *you* to come up with a better estimate.
Nevertheless, let me try to clarify my point.
What is measured in the experiment is Ispot. It contains Iback which
our theoretical models cannot possibly account for (because we have no
information at the refinement stage about crystal shape and other
parameters that define background). Strategy that has been in use for
decades is to obtain estimates of Iback from pixels surrounding the
integration spot. I hope you find that reasonable.
Once we have Iback estimated, Ispot-Iback becomes Iobs - observed
intensity. There is no need to convert that value simply to avoid bad
feeling brought by negative values. Correctly formulated theoretical
model predicts Iobs and accounts for error in it.
Let me state this again - Iobs are not true intensities and not
estimates of true intensities. They are experimental values sampling
Ispot-Iback. These can be negative. If a theoretical model that
approximates Iobs does not allow for negative Iobs, the model is flawed.
>> These observed intensities can be negative because while their true underlying value is positive, random errorsmay result in Iback>Ispot. There is absolutely nothing unphysical here.
> Yes there is. The only way you can get a negative estimate is to make unphysical assumptions. Namely, the estimate Ispot-Iback=Iobs assumes that both the true value of I and the background noise come from a Gaussian distribution that is allowed to have negative values. Both of those assumptions are unphysical.
See, I have a problem with this. Both common sense and laws of physics
dictate that number of photons hitting spot on a detector is a positive
number. There is no law of physics that dictates that under no
circumstances there could be Ispot<Iback. Yes, E(Ispot)>=E(Iback).
Yes, E(Ispot-Iback)>=0. But P(Ispot-Iback=0)>0, and therefore
experimental sampling of Ispot-Iback is bound to occasionally produce
negative values. What law of physics is broken when for a given
reflection total number of photons in spot pixels is less that total
number of photons in equal number of pixels in the surrounding
background mask?
Cheers,
Ed.
--
Oh, suddenly throwing a giraffe into a volcano to make water is crazy?
Julian, King of Lemurs
