A nice story, nicely told, Ronan. But ...
On 07-Sep-09 09:30:04, Ronan Conroy wrote:
> On 4 MFómh 2009, at 18:09, Paul Elias wrote:
>> I wanted to share this:
>>
>> If you subject your data and analysis to torture and relentless
>> pressure, it will confess anything to you or tell you essentially,
>> what you want to hear.
>
> I was an expert witness in a medico-legal case. The other side had a
> statistician too. The legal teams were disgusted: our reports agreed
> down to the last detail. We both explained that given the same data,
> the calculations yielded the same results.
It is certainly true that applying exactly the same algorithms to
exactly the same data will (or should) yield exactly the same results.
So, if you and the other side's statistician applied the same
algorithms, then no surprise there. In statistics, the algorithm is
usually the result of adopting a particular model for the process
which generated the data. So presumably you both adopted the same
model.
But, come on, get creative, man!
If, for instance, the other side adopt a logistic model for the
incidence of a certain adverse event, you can challenge that model.
Why not a probit model? With the right data, this can predict a
higher incidence than the logistic model would. The main reasons
that a logistic model is generally adopted are that (a) "everybody"
does it -- it is standard; (b) the resulting coefficients can be
interpreted as odds ratios, which are easy [??] to understand --
in order to avoid confusing Judge or Jury with abstruseness, one
can (or Counsel can) use parables based on betting on horses.
But the probit model is (fundamentally) based on the concept of
a latent variable which is Normally distributed, and which can be
interpreted as a "degree of tolerance" to the corresponding risk
factor: if a subject has tolerance greater than the level of the
risk factor, then no adverse event; if less, then adverse event.
The corresponding latent variable for a logistic model is not
Normally distributed (having much heavier tails).
Parable: a basket of hen's eggs. Each egg has a maximum height
from which it can be dropped onto a table without cracking.
This height ("tolerance to dropping") is Normally distributed
across eggs. Pick a random egg, and choose (from a range of heights)
a height from which to drop it. The result (cracked or not cracked)
then tells you whether the max height for that egg was less than,
or greater than, the height from which you dropped it.
Because you are basing your view of "tolerance to dropping" on a
Normal distribution thereof, the appropriate analysis is probit
(which yields maximum-likelihood estimates of the mean and SD of
this Normal distribution, based on the censored data), and not
logistic (which does not). You would then be able to provide an
expert opinion on whether eggs supplied from one chicken farm had
a more favourable distribution of "tolerance to dropping" than
those from another, and hence (for instance) better suited to
being transported by lorry to supermarkets (if that was the issue in
the case for which you were called as expert witness), by comparing
the two estimated Normal distributions.
If the other side used logistic regression, this could be challenged
since the underlying variable is not Normally distributed (and,
"of course", naturally occurring quantities which are the result
of several simultaneously acting influence are Normally distributed,
aren't they?). So away you go.
Historical note: A related problem was what originally converted me
from applied mathematician (when working for the Ministry of Defence)
to statistician, since one day I was looking at the results from
trials of firing armour-piercing shot at sheets of armour plate.
The objective, at the time,was to compare different compositions
of armour plate in terms of their resistance to being shot at.
> One barrister asked me "but can't you statisticians make the data fit
> whatever story you want to?"
> "I'm afraid you are confusing my profession with yours" I replied.
I've seen statisticians at Court/Public Inquiries doing that!
Professionally speaking, of course, the expert witness is called
by the Court (even though engaged by one side or the other), and
their duty is to "assist the Court", so should not attempt (as
a Counsel is entitled to) to present the evidence in the best
light for their client.
However, "making the data fit whetever story you want" can be realised
in terms of deciding what model you are going to fit to the data.
See above. So there is scope for contention about that, and it is
not a foregone conclusion that the two sides' statisticians (along
with opinion, from experts in other scientific areas, which is
relevant to choosing the model) will necessarily agree about the
alogorithm.
> The whole aspect of the case centred around the data was dropped, as
> there was essentially no disagreement between the expert witnesses for
> each side.
>
> So here's a challenge: I will send the results of a negative trial to
> anyone who believes that they can make the trial positive by torturing
> them. And if they can, I will concede that Paul's comment is more than
> a widespread bias based on a deep rooted fear of mathematics instilled
> by generations of incompetent primary school teachers.
>
> Ronan Conroy
> =================================
I dare say that if you supplied enough ancillary data (such as how
far away, for different subjects in the trial, they lived from nuclear
power stations, or from overhead power cables, or from farm-land to
which pesticides are applied, and so on), one would have enough thumbs
to apply the screws to, eventually eliciting a squeal.
Not that I'm advocating torture! I'm a firm believer in attempting
to honestly elicit information from data. But there may be relevant
information there, of a kind not foreseen when the investigation was
designed, which could throw useful light if detected. And this does
imply being ready to embark on "ad hoc" analysis, in a "forensic"
spirit, provided this is used more for "hypothesis generation"
than for "hypothesis testing".
Ted.
--------------------------------------------------------------------
E-Mail: (Ted Harding) <[log in to unmask]>
Fax-to-email: +44 (0)870 094 0861
Date: 07-Sep-09 Time: 11:51:13
------------------------------ XFMail ------------------------------
|