Douglas,
>> Observed intensities are the best estimates that we can come up with in an experiment.
> I also agree with this, and this is the clincher.  You are arguing that Ispot-Iback=Iobs is the best estimate we can come up with.  I claim that is absurd.  How are you quantifying "best"?  Usually we have some sort of discrepancy measure between true and estimate, like RMSD, mean absolute distance, log distance, or somesuch.  Here is the important point --- by any measure of discrepancy you care to use, the person who estimates Iobs as 0 when Iback>Ispot will *always*, in *every case*, beat the person who estimates Iobs with a negative value.   This is an indisputable fact.

First off, you may find it useful to avoid such words as absurd and 
indisputable fact.  I know political correctness may be sometimes 
overrated, but if you actually plan to have meaningful discussion, let's 
assume that everyone responding to your posts is just trying to help 
figure this out.

To address your point, you are right that J=0 is closer to "true 
intensity" then a negative value.  The problem is that we are not after 
a single intensity, but rather all of them, as they all contribute to 
electron density reconstruction.  If you replace negative Iobs with 
E(J), you would systematically inflate the averages, which may turn 
problematic in some cases.  It is probably better to stick with "raw 
intensities" and construct theoretical predictions properly to account 
for their properties.

What I was trying to tell you is that observed intensities is what we 
get from experiment.  They may be negative, and there is nothing 
unphysical about it.  Then you build a theoretical estimate of observed 
intensities, and if you do it right (i.e. by including experimental 
errors), they will actually have some probability of being negative.
>> This background has to be subtracted and what is perhaps the most useful form of observation is Ispot-Iback=Iobs.
> How can that be the most useful form, when 0 is always a better estimate than a negative value, by any criterion?

Given your propensity to refer to what others might say as absurd, I am 
tempted to encourage *you* to come up with a better estimate. 
Nevertheless, let me try to clarify my point.

What is measured in the experiment is Ispot.  It contains Iback which 
our theoretical models cannot possibly account for (because we have no 
information at the refinement stage about crystal shape and other 
parameters that define background).  Strategy that has been in use for 
decades is to obtain estimates of Iback from pixels surrounding the 
integration spot.  I hope you find that reasonable.

Once we have Iback estimated, Ispot-Iback becomes Iobs - observed 
intensity.  There is no need to convert that value simply to avoid bad 
feeling brought by negative values.  Correctly formulated theoretical 
model predicts Iobs and accounts for error in it.

Let me state this again - Iobs are not true intensities and not 
estimates of true intensities.  They are experimental values sampling 
Ispot-Iback.  These can be negative.  If a theoretical model that 
approximates Iobs does not allow for negative Iobs, the model is flawed.
>> These observed intensities can be negative because while their true underlying value is positive, random errorsmay result in Iback>Ispot.  There is absolutely nothing unphysical here.
> Yes there is.  The only way you can get a negative estimate is to make unphysical assumptions.  Namely, the estimate Ispot-Iback=Iobs assumes that both the true value of I and the background noise come from a Gaussian distribution that is allowed to have negative values.  Both of those assumptions are unphysical.

See, I have a problem with this.  Both common sense and laws of physics 
dictate that number of photons hitting spot on a detector is a positive 
number.  There is no law of physics that dictates that under no 
circumstances there could be Ispot<Iback.  Yes, E(Ispot)>=E(Iback).  
Yes, E(Ispot-Iback)>=0.  But P(Ispot-Iback=0)>0, and therefore 
experimental sampling of Ispot-Iback is bound to occasionally produce 
negative values.  What law of physics is broken when for a given 
reflection total number of photons in spot pixels is less that total 
number of photons in equal number of pixels in the surrounding 
background mask?

Cheers,

Ed.

-- 
Oh, suddenly throwing a giraffe into a volcano to make water is crazy?
                                                 Julian, King of Lemurs