Thank you all for the responses.
I like the suggestion by "bgbg.bg" the most as it seems to me to be the simplest
one. I will also check what Stan Alekman has suggested.
bgbg.bg was right: the distribution of differences between B and A is higly
skewed. However, I'm still not sure it is ligitimate to apply the central limit
theorem to this case. Any help with this question?
________________________________
From: bgbg.bg <[log in to unmask]>
To: [log in to unmask]
Sent: Tue, July 13, 2010 11:36:30 PM
Subject: Re: Assessing accuracy of a measurement method
Jonathan,
here is my suggestion. I use Latex notation extensively, you may paste this
text into in order to obtain the rendered text.
From your question I understand that there is only one measurement per case
with method A. This means that assessing the agreement between A and B
reduces to showing that the mean differences between A and B
(accuracy)($\overline{\Delta_{A,B}} \approx 0$) is as close to 0 as possible
and the standard deviation of these differences (precision) is as low as
possible.
I also learn that the measurements you take can take values between two
numbers. This might lead to the situation where the distribution of
$\Delta_{A,B}$ is far from being normal. On the other hand, you have
multiple measurements of several cases. Now, here comes the tricky part.
Assume that the real mean difference between A and B readings is $\mu$ with
standard deviation of $\sigma$. We may treat those multiple measurements as
different samplings from the overall distribution. Each sampling $i$ has its
own mean difference $\overline{\Delta_i}$. According to the central limit
theorem, the mean of means ($\mu_{\overline{\Delta}}$) is a good
approximation of real $\mu$ and the standard deviation of deltas is
connected to the real standard deviation $sigma$ as follows:
$\sigma_{\overline{\Delta}} = \frac{\sigma}{\sqrt{n}}$. You will be also
able to calculate the 95\% confidence interval of the difference estimate
using either Z or t distribution (depending on the number of cases you have
measured)
Having all this information you will be able to conclude that B agrees with
A within $\mu_{\overline{\Delta}}$ with standard deviation of
$\sigma_{\overline{\Delta}} \times \sqrt{n}$ or that B agrees with A within
$\pm CI_{95\%}$
On Tue, Jul 13, 2010 at 6:51 PM, Stan Alekman <[log in to unmask]> wrote:
> Jonathan,
>
> Youden at the National Bureau of Standards (before it became NIST)
> addressed this question and published.
>
> I don't know if it applies well to your problem but the literature may be
> Google accessible.
>
> I remember something about a Youden plot but I am not home and cannot check
> my files.
>
> Good luck.
> Regards,
> Stan Alekman
>
>
> -----Original Message-----
> From: Jonathan James <[log in to unmask]>
> To: [log in to unmask]
> Sent: Tue, Jul 13, 2010 11:42 am
> Subject: Re: Assessing accuracy of a measurement method
>
> Thank you. I should have mentioned that I have read the1986 Bland and
> Altman
> paper. However, here the situation is a little bit different that what is
> discussed in the paper.
>
> In the case of B&A, each sample is analyzed twice by the two methods under
> the
> comparison. In my case, each sample is analyzed (measured) only once using
> one
> reference method and many times using the another (novel) one. That is why
> I
> find it difficult adopting the B&A methodology.
>
> P.S It is interesting to know that the 1986 B&A paper will be re-published
> this
> August in Int J Nurs Study http://www.ncbi.nlm.nih.gov/pubmed/20430389
>
>
>
>
> ________________________________
> From: John Sorkin <[log in to unmask]>
> To: Jonathan James <[log in to unmask]>
> Sent: Tue, July 13, 2010 6:26:23 PM
> Subject: Re: Assessing accuracy of a measurement method
>
> You might want to start by reading the papers by Bland and Altman
>
> Altman DG, Bland JM (1983). "Measurement in medicine: the analysis of
> method comparison studies". Statistician 32: 307–317.
> doi:10.2307/2987937.
> Bland JM, Altman DG (1986). "Statistical methods for assessing
> agreement between two methods of clinical measurement". Lancet 1 (8476):
> 307–10. PMID 2868172.
> Of the two, I would start with the second as it is a bit easier to
> read.
> John
>
>
>
>
>
>
> John David Sorkin M.D., Ph.D.
> Chief, Biostatistics and Informatics
> University of Maryland School of Medicine Division of Gerontology
> Baltimore VA Medical Center
> 10 North Greene Street
> GRECC (BT/18/GR)
> Baltimore, MD 21201-1524
> (Phone) 410-605-7119
> (Fax) 410-605-7913 (Please call phone number above prior to faxing)>>>
> Jonathan James <[log in to unmask]> 7/13/2010 10:02 AM >>>
> Hello,
> this is my first mail to allstat (although I have been reading the
> archives for
> a while)
>
> I have been asked to assess the accuracy and precision of a new
> measurement
> method (Let's call it method B). This new method is compared to an
> existing
> one (A) that is considered to be "very accurage" and has its own
> specifications
> in terms of stdev of a single measurement. What we do is to measure
> several
> samples with method A and then with method B. Since A is very
> expensive, only
> one A measurement per case is available. Method B is cheaper, so we
> measure
> each sample with method B between 10 and 30 times. Another problem is
> that we
> are unable to find samples that would span across the entire legal
> measurement
> range, resulting in several samples in the first quartile of the
> range, several
> in the last range quartile and almost no in between.
> How can I assess the accuracy and precision of method B? Any help or
> link will
> be appreciated.
> Thank you very much
>
> Jonathan James
>
>
> P.S. This is not a homework.
> P.P.S I admit, I don't know statistics well
>
>
>
>
> You may leave the list at any time by sending the command
>
> SIGNOFF allstat
>
> to [log in to unmask], leaving the subject line blank.
>
>
> Confidentiality Statement:
> This email message, including any attachments, is for the sole use of
> the intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message.
>
>
>
>
>
> You may leave the list at any time by sending the command
>
> SIGNOFF allstat
>
> to [log in to unmask], leaving the subject line blank.
>
> You may leave the list at any time by sending the command
>
> SIGNOFF allstat
>
> to [log in to unmask], leaving the subject line blank.
>
You may leave the list at any time by sending the command
SIGNOFF allstat
to [log in to unmask], leaving the subject line blank.
You may leave the list at any time by sending the command
SIGNOFF allstat
to [log in to unmask], leaving the subject line blank.
