Print

Print
On 8 July 2013 18:29, Douglas Theobald <[log in to unmask]> wrote:
That's all very interesting --- do you have a good ref for TDS where I
can read up on the theory/practice?  My protein xtallography books say
even less than S&J about TDS.  Anyway, this appears to be a problem
beyond the scope of this present discussion --- in an ideal world we'd
be modeling all the forms of TDS, and Bragg diffraction, and comparing
those predictions to the intensity pattern over the entire detector ---
not just integrating near the reciprocal lattice points.  Going on what
you said above, it seems the acoustic component can't really be measured
independently of the Bragg peak, while the optic and Einstein components
can, or least can be estimated pretty well from the intensity around the
Bragg peak (which means we can treat it as "background").  In any case,
I'm going to ignore the TDS complications for now. :).

James gave one good reference (Welberry).  Also there's some info here: http://people.cryst.bbk.ac.uk/~tickle/iucr99/s60.html and ditto with s61, s62, s63, s64 & s70 (last one is a reference list: ref nos. 31-36 are for TDS).  I agree with everything you said, as others have also said, about the need to compare the model of the total coherent scattering (Bragg + TDS), also including the incoherent contribution, with the actual data (i.e. the image from the detector).

That's all true, but you can detect peaks independently of one another
on a detector, so obviously there is some minimal distance away from a
crystal where you could completely block any given reflection and
nothing else. Clearly the "reflection stop" would have to be the size of
the crystal (or at least the beam).

As James pointed out most of the background comes from the same place as the Bragg diffraction (i.e. the crystal) so your reflection stop would inevitably block both.  There is no distance at which you could block the Bragg but not at the same time block the TDS.  In fact the theory shows that Bragg & TDS are simply different terms in the total coherent scattering (see 's61' page above for details) that are really separated completely arbitrarily.  This separation of the terms is an artifact of our attempt to model the Bragg term alone.  However the only viable models of MX structures are Bragg ones so we have no option but to work with the Bragg component of the data alone.  In reality there's no distinction between Bragg and TDS: they are both parts of the same coherent scattering and it's meaningless to ask whether a particular photon 'belongs' to one rather than the other, just as with the slit experiment.  The best you can say is that it belongs to both.  F & W is really just a work-around of our initial false assumption that the data consist of Bragg diffraction alone.

If Iback' and Iback" come from the same process, then one informs the
other. Of course you'd have to account for statistical fluctuations.
This is exactly the same principle behind using Iback to give us
information about Iback' in French and Wilson's method.

Aw, come on --- QM is a theory, it says no such thing.  The claim that
"it's meaningless to talk about something you can't observe" is a
philosophical principle, not science.  There are many interpretations of
QM, some involving hidden variables, which are precisely things that
exist that you can't observe.  Heck, I'd say all of science is *exactly*
about the existence of things that we only infer and cannot observe
directly.  Remember, when you get the readout from a detector, you are
not directly observing photons even then --- you are formally inferring
things that you can't observe.  There's a whole chain of theory and
interpretation that gets you from the electronic readout to the claim
that the readout actually represents some number of photons.

In science we make a clear separation of what is 'data' from what is 'model' (or 'inference').  Of course one can be pedantic and argue that everything is really inference since our brains interpret everything that goes on 'out there' by means of inference from its sensory inputs.  Obviously I don't take seriously the premise of the 'Matrix' movies (excellent though they may be!) that such inputs are just a simulation!  At some point you have to believe what your eyes are telling you, as long as there is a clear chain of believable cause-and-effect between the observation and the inference of that observation.  However, we are rightly suspicious of any model that is not supported by data (in fact inference in science requires data).  Note that I always mean 'observation' to be synonymous with 'measurement' (as in 'Fobs'), not 'observation' in the weaker sense of 'seeing'.
 
Again, this is your own personal philosophical interpretation of QM ---
QM itself says nothing of the sort.  For instance, Bohm's pilot wave
interpretation of QM, which is completely consistent with observation
and QM theory and calculation, states that individual photons *do* go
through one slit or the other.  But this is really off point here, I
think --- as I said, I don't want to get into a QM debate.

Neither do I, I would just observe that under no QM interpretation can you determine which slit any individual photon went through and I would argue that it's therefore not even a meaningful question to ask which slit it went through.
 
I disagree.  Following that logic, we could not talk about the error in
our estiamte of the gravitational force on the Earth from the Moon
(because, if our theories of gravity are correct, the force exerted on
the Earth is the sum of the gravitational pull of all massive objects in
the universe, and it is physically impossible for us to, say, remove the
Sun and then independently measure the force exerted by the Moon).

The estimate of the gravitational force on the Earth from the Moon is an inference, not an observation: it relies on Newton's theory of gravitation being correct.  Of course I'm not denying that you can talk about the error estimate for an inference, but I was talking about errors in observations (= measurements).  It's physically impossible to move the Sun, but not logically impossible (i.e. we can imagine it happening).  In contrast separating Ispot into its component Itrue & Iback is a logical impossibility; we can't even imagine it without allowing a logical contradiction: since photons are indistinguishable the same photons in Ispot have to be blocked and not blocked at the same time.  I contend that if a 'measurement' is a logical impossibility (but not necessarily a physical one) then it's meaningless to talk about an error distribution for such a 'measurement'.

I agree completley with that.  My point is, however, that F&W implicitly
assumes that the background we measure (in Ib) comes from the same
process as the background under the spot (Ib').  In other words, the
underlying model is:

Is = Ib' + Ij

Ib

where we experimentally measure Is (the spot) and Ib (the background
around the spot), and we assume that both Ib' and Ib come from the same
(Poisson) distribution, and that Ij (a sample from the true spot
intensity J) comes from a Poisson as well.

If that's not the F&W model, then what is?
This is precisely the point I was trying to make, so we agree.
Ib and Is are not independent.
 
I agree that Is-Ib is approximately Gaussian for reasonably large Is and
Ib. The problem is P(Is-Ib | J,sdJ).  Why is the mean of this Gaussian
J?  Why not J^2, or J/4?  How do you derive the Gaussian
P(Is-Ib |J,sdJ)?  The claim that the mean of Is-Ib is J requires some
sort of justification --- otherwise I might as well claim that
the mean of Is-Ib is the number of quarters in my pocket.

We both agreed that Is and Ib are dependent, and so that necessarily
means that P(Is-Ib | J,sdJ) cannot be the whole story.  Dependence means
that the distribution of Is-Ib will also be a function of B and sdB,
unless you've integrated them out somehow (and even then the form
of the marginalized distribution depends on B and sdB).

F&W assumes that P(Ij|J,sdJ) is Poisson.  If you think F&W doesn't, then
explain how you get the Gaussian P(Is-Ib | J,sdJ).  Again, why is the
mean of Is-Ib equal to J?  I gave one derivation, based on the model
that P(Ij|J), P(Ib|B), and P(Ib'|B) are all Poisson.  If you disagree,
then what is your derivation of the Gaussian P(Is-Ib | J,sdJ)?

OK let's go through it step-by-step and you tell me where I went wrong.  Note that I do not assume a prior P(Ij|J,sdJ), in particular I do not assume that it's Poisson, or even that Ij or P(Ij|J,sdJ) exist: I contend that Ij is certainly not observable.

1. The true values of the Bragg & background components of a spot are J and B, so the true total coherent scattering is S = J + B.

[Note: true values are unknown constants, which obviously do not have uncertainties, i.e. if we were doing a simulation we might assume say J = 10, B = 20; there's no assumption of P(Ij) here.  Of course estimates of true values do have uncertainties, but the Wilson prior P(J), the posterior P(J|Is,Ib) and hence <J> and var(<J>) only come in later after we invoke Bayes' theorem in F & W.]

2. We make a measurement of S and get Is.  If we made multiple measurements we would expect to see (say) a Poisson distribution, in which case for a single measurement we can estimate var(S) = S ~ Is.

3. We also estimate B from the adjacent background Ib and similarly we estimate var(B) = B ~ Ib.

4. So now we have estimates of S (~ Is) and B (~ Ib), so therefore we have an estimate of J = S - B (~ Is - Ib).

5. We have established that the difference of 2 Poissons with means S and B is well-approximated by a Gaussian with mean (S-B) = J and variance (S+B).

6. So P(Is-Ib,J) is Gaussian with mean J, and var(J) = (S+B) ~ (Is+Ib).

I don't see what's wrong with that.

To me it would be strange if F & W, being so keen on the Bayesian method, didn't use any available prior information about P(Ij) since it seems rather an obvious (but wrong!) assumption to make.  They must have had good reason not to.  They're still around (see this dated October 2012: http://www2.warwick.ac.uk/fac/sci/statistics/staff/academic-research/french/publications/bayes_impact_french_wilson.pdf), so we could always ask them!

Cheers

-- Ian