 |
 |
Part of the variability is 'technical error' and this will be reduced by averaging of repeated measures. Thus any screening programme is expected to do better if based on repeat measurements but eventually not technical error will dominate. It might be worth looking at 1. Gardner M, Heady J. Some effects of within-person variability in epidemiological studies. Journal of chronic diseases. 1973;26(12):781-95. and references to it. Chapter 7 of my book Statistical Issues in Drug Development covers (amongst other matters) an analogous issue with repeated baseline in analysis of covariance. Stephen Stephen Senn Competence Center for Methodology and Statistics Luxembourge Institute of Health 1a rue Thomas Edison 1445 Strassen Luxembourg [log in to unmask] -----Original Message----- From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of EVIDENCE-BASED-HEALTH automatic digest system Sent: 22 March 2016 01:05 To: [log in to unmask] Subject: EVIDENCE-BASED-HEALTH Digest - 19 Mar 2016 to 21 Mar 2016 (#2016-65) There are 7 messages totaling 1512 lines in this issue. Topics of the day: 1. will false negatives go down with the repeated same test? (3) 2. will false negatives go down with the repeated same test? (3) 3. ANSWER TO will false negatives go down with the repeated same test? ---------------------------------------------------------------------- Date: Mon, 21 Mar 2016 16:31:29 +0000 From: "Yao, Xiaomei" <[log in to unmask]> Subject: Re: will false negatives go down with the repeated same test? Hi Nick, If the patient population and the test are exactly the same, no matter when you repeat the test, the sensitivity and specificity should be the same theoretically. Xiaomei From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Myles, Nickolas [PH] Sent: March-18-16 1:01 PM To: [log in to unmask] Subject: will false negatives go down with the repeated same test? Hello everyone, I am wondering if this is true to assume that if the screening test is repeated for the same patient over the period of time it will diminish the false negative rate? Assuming, the FOBT test for colorectal cancer screening is 80% sensitive, then FN rate will be 20% (p=0.2). Assuming the tests are independent and the error is completely random, will repeated screening over one year with FOBT make it 0,2x0.2=0.04 (4%) and over the three years under 1% (0.2x0.2x0.2=0.008)? This many sound very bold, but does the repeat measures work that way? of will it stay the same 20% irrespective of the repeat measures? I am not talking about false positives, but the question can certainly be extended for FP as well. thank you, will appreciate your opinion or reference. Nick ------------------------------ Date: Mon, 21 Mar 2016 10:13:00 -0700 From: "Myles, Nickolas [PH]" <[log in to unmask]> Subject: Re: will false negatives go down with the repeated same test? Thank Xiaomei and Brian, yes, the sens/spec are the same but the technical error due to test variation will most likely affect different patients each run (if we take test as a lab value, assuming we repeatedly test the same patient population, for which standard reference test result is known by the "supreme powers" ) . My question is harder that it seems: if we repeat the same test for the same patient often and on the same sample, in case of a negative first test how many times one should repeat the test (which is 80% sensitive and 99% specific) in order to get true positive not false negative result ? Here in the lab we are testing cancer samples for targeted therapies, so the question is practical and the repeat tests are requested often, so it's not just mental gymnastics. Thanks a lot for posting, Nick From: Yao, Xiaomei [mailto:[log in to unmask]] Sent: Monday, March 21, 2016 9:31 AM To: Myles, Nickolas [PH]; [log in to unmask] Subject: RE: will false negatives go down with the repeated same test? Hi Nick, If the patient population and the test are exactly the same, no matter when you repeat the test, the sensitivity and specificity should be the same theoretically. Xiaomei From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Myles, Nickolas [PH] Sent: March-18-16 1:01 PM To: [log in to unmask]<mailto:[log in to unmask]> Subject: will false negatives go down with the repeated same test? Hello everyone, I am wondering if this is true to assume that if the screening test is repeated for the same patient over the period of time it will diminish the false negative rate? Assuming, the FOBT test for colorectal cancer screening is 80% sensitive, then FN rate will be 20% (p=0.2). Assuming the tests are independent and the error is completely random, will repeated screening over one year with FOBT make it 0,2x0.2=0.04 (4%) and over the three years under 1% (0.2x0.2x0.2=0.008)? This many sound very bold, but does the repeat measures work that way? of will it stay the same 20% irrespective of the repeat measures? I am not talking about false positives, but the question can certainly be extended for FP as well. thank you, will appreciate your opinion or reference. Nick ------------------------------ Date: Mon, 21 Mar 2016 21:04:18 +0300 From: vv vlassov <[log in to unmask]> Subject: Re: will false negatives go down with the repeated same test? Hi, Nick, you have to decide first, do you speak about the real world, or in ideal one. In ideal world results will be stable in every patient, and repeating the test have no sense. In more or less real world repeating the test you will get more and more positive results, both true and false. Just enough repetition of the test, and there will be almost no negative results. Oh, this is in the ideal world, pardon. In real world because test are not independent, small number of subjects will collect more and more "positives" and majority of subjects will have no o small number of positive tests. Of course, "small" and 'majority" relates to the situation when true prevalence is small. By repeating the test you will never distill true negatives as well nor true positives vvv On 21.03.2016 20:13, Myles, Nickolas [PH] wrote: > Thank Xiaomei and Brian, > > yes, the sens/spec are the same but the technical error due to test > variation will most likely affect different patients each run (if we > take test as a lab value, assuming we repeatedly test the same patient > population, for which standard reference test result is known by the > "supreme powers" ) . > > My question is harder that it seems: if we repeat the same test for > the same patient often and on the same sample, in case of a negative > first test how many times one should repeat the test (which is 80% > sensitive and 99% specific) in order to get true positive not false negative result ? > > Here in the lab we are testing cancer samples for targeted therapies, > so the question is practical and the repeat tests are requested often, > so it's not just mental gymnastics. > > Thanks a lot for posting, > > Nick > > *From:*Yao, Xiaomei [mailto:[log in to unmask]] > *Sent:* Monday, March 21, 2016 9:31 AM > *To:* Myles, Nickolas [PH]; [log in to unmask] > *Subject:* RE: will false negatives go down with the repeated same test? > > Hi Nick, > > If the patient population and the test are exactly the same, no matter > when you repeat the test, the sensitivity and specificity should be > the same theoretically. > > Xiaomei > > *From:*Evidence based health (EBH) > [mailto:[log in to unmask]] *On Behalf Of *Myles, > Nickolas [PH] > *Sent:* March-18-16 1:01 PM > *To:* [log in to unmask] > <mailto:[log in to unmask]> > *Subject:* will false negatives go down with the repeated same test? > > Hello everyone, > > I am wondering if this is true to assume that if the screening test is > repeated for the same patient over the period of time it will diminish > the false negative rate? > > Assuming, the FOBT test for colorectal cancer screening is 80% > sensitive, then FN rate will be 20% (p=0.2). Assuming the tests are > independent and the error is completely random, will repeated > screening over one year with FOBT make it 0,2x0.2=0.04 (4%) and over > the three years under 1% (0.2x0.2x0.2=0.008)? This many sound very > bold, but does the repeat measures work that way? of will it stay the > same 20% irrespective of the repeat measures? > > I am not talking about false positives, but the question can certainly > be extended for FP as well. > > thank you, will appreciate your opinion or reference. > > Nick > Vasiliy V. Vlassov, MD President, Society for Evidence Based Medicine (osdm.org) e-mail: vlassov[a t]cochrane.ru snail mail: P.O.Box 13 Moscow 109451 Russia Phone Russia +7(965)2511021 Подпишись на новости на osdm.org -- THANK YOU for deleting my e-mail address , any other addresses, and any personal information, from this e-mail, if you plan to forward it. Also, thank you for using “Bcc” instead of “To” and “Cc“ when initiating both individual and group e-mails. These extra actions on your part help to prevent spammers and hackers from obtaining addresses and thus help prevent the proliferation of spam. ------------------------------ Date: Mon, 21 Mar 2016 14:11:30 -0400 From: "Mohammed T. Ansari" <[log in to unmask]> Subject: Re: will false negatives go down with the repeated same test? Hi Nick, In my view test test–retest variability and test sens/spec are sort of unrelated characteristics If test–retest variability is close to zero, with a test which is 80% sensitive and 99% specific, false positive/negative rates will remain the same for the same patient even with repeated tests. If test–retest variability is high, true positive and false negative rates will remain the same except that the patient may be categorized with poor reliability (wide 95% CIs) Repeated testing just gives us a sense of the degree of test–retest variability (or lack thereof) which is random error. On Mon, Mar 21, 2016 at 1:13 PM, Myles, Nickolas [PH] < [log in to unmask]> wrote: > Thank Xiaomei and Brian, > > > > yes, the sens/spec are the same but the technical error due to test > variation will most likely affect different patients each run (if we > take test as a lab value, assuming we repeatedly test the same patient > population, for which standard reference test result is known by the > "supreme powers" ) . > > > > My question is harder that it seems: if we repeat the same test for > the same patient often and on the same sample, in case of a negative > first test how many times one should repeat the test (which is 80% > sensitive and 99% > specific) in order to get true positive not false negative result ? > > > > Here in the lab we are testing cancer samples for targeted therapies, > so the question is practical and the repeat tests are requested often, > so it's not just mental gymnastics. > > > > Thanks a lot for posting, > > Nick > > > > > > *From:* Yao, Xiaomei [mailto:[log in to unmask]] > *Sent:* Monday, March 21, 2016 9:31 AM > *To:* Myles, Nickolas [PH]; [log in to unmask] > *Subject:* RE: will false negatives go down with the repeated same test? > > > > Hi Nick, > > > > If the patient population and the test are exactly the same, no matter > when you repeat the test, the sensitivity and specificity should be > the same theoretically. > > > > Xiaomei > > > > *From:* Evidence based health (EBH) [ > mailto:[log in to unmask] > <[log in to unmask]>] *On Behalf Of *Myles, Nickolas > [PH] > *Sent:* March-18-16 1:01 PM > *To:* [log in to unmask] > *Subject:* will false negatives go down with the repeated same test? > > > > Hello everyone, > > > > I am wondering if this is true to assume that if the screening test is > repeated for the same patient over the period of time it will diminish > the false negative rate? > > > > Assuming, the FOBT test for colorectal cancer screening is 80% > sensitive, then FN rate will be 20% (p=0.2). Assuming the tests are > independent and the error is completely random, will repeated > screening over one year with FOBT make it 0,2x0.2=0.04 (4%) and over > the three years under 1% (0.2x0.2x0.2=0.008)? This many sound very > bold, but does the repeat measures work that way? of will it stay the > same 20% irrespective of the repeat measures? > > > > I am not talking about false positives, but the question can certainly > be extended for FP as well. > > > > thank you, will appreciate your opinion or reference. > > Nick > > > -- [image: --] Mohammed T. Ansari [image: https://]about.me/MTAnsari <https://about.me/MTAnsari?promo=email_sig> ------------------------------ Date: Mon, 21 Mar 2016 18:37:32 +0000 From: "Yao, Xiaomei" <[log in to unmask]> Subject: Re: will false negatives go down with the repeated same test? Hi Nick, Now I understand your question:" If we repeat the same test for the same patient often and on the same sample, in case of a negative first test how many times one should repeat the test (which is 80% sensitive and 99% specific) in order to get true positive not false negative result?" My answer is "We are unable to know.". When you rerun the test on the same sample or different sample from the same patient within one year, it likes a brand new procedure each time, and you should always get the same post-test probability if we ignore the random errors etc., and you always just get a PROBABILITY with 95% CI (not a YES or NO answer to have the disease or not) because FOBT test is not a reference standard. Xiaomei From: Myles, Nickolas [PH] [mailto:[log in to unmask]] Sent: March-21-16 1:13 PM To: Yao, Xiaomei; [log in to unmask] Subject: RE: will false negatives go down with the repeated same test? Thank Xiaomei and Brian, yes, the sens/spec are the same but the technical error due to test variation will most likely affect different patients each run (if we take test as a lab value, assuming we repeatedly test the same patient population, for which standard reference test result is known by the "supreme powers" ) . My question is harder that it seems: if we repeat the same test for the same patient often and on the same sample, in case of a negative first test how many times one should repeat the test (which is 80% sensitive and 99% specific) in order to get true positive not false negative result ? Here in the lab we are testing cancer samples for targeted therapies, so the question is practical and the repeat tests are requested often, so it's not just mental gymnastics. Thanks a lot for posting, Nick From: Yao, Xiaomei [mailto:[log in to unmask]] Sent: Monday, March 21, 2016 9:31 AM To: Myles, Nickolas [PH]; [log in to unmask]<mailto:[log in to unmask]> Subject: RE: will false negatives go down with the repeated same test? Hi Nick, If the patient population and the test are exactly the same, no matter when you repeat the test, the sensitivity and specificity should be the same theoretically. Xiaomei From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Myles, Nickolas [PH] Sent: March-18-16 1:01 PM To: [log in to unmask]<mailto:[log in to unmask]> Subject: will false negatives go down with the repeated same test? Hello everyone, I am wondering if this is true to assume that if the screening test is repeated for the same patient over the period of time it will diminish the false negative rate? Assuming, the FOBT test for colorectal cancer screening is 80% sensitive, then FN rate will be 20% (p=0.2). Assuming the tests are independent and the error is completely random, will repeated screening over one year with FOBT make it 0,2x0.2=0.04 (4%) and over the three years under 1% (0.2x0.2x0.2=0.008)? This many sound very bold, but does the repeat measures work that way? of will it stay the same 20% irrespective of the repeat measures? I am not talking about false positives, but the question can certainly be extended for FP as well. thank you, will appreciate your opinion or reference. Nick ------------------------------ Date: Mon, 21 Mar 2016 11:58:05 -0700 From: "Myles, Nickolas [PH]" <[log in to unmask]> Subject: Re: will false negatives go down with the repeated same test? So, what is the correct way of calculating PPV/NPV for repeat tests? thanks for reliable references if any N Nickolas Myles, MD, PhD, MSc, FRCPC Anatomical pathologist, St.Paul’s Hospital, Clinical Associate Professor, University of British Columbia Department of Pathology and Laboratory Medicine 1081 Burrard St, Vancouver, BC, V6Z1Y6 Phone (604) 682-2344 x 66038 Email: [log in to unmask] -----Original Message----- From: vv vlassov [mailto:[log in to unmask]] Sent: Friday, March 18, 2016 11:04 AM To: Myles, Nickolas [PH]; [log in to unmask] Subject: Re: will false negatives go down with the repeated same test? Hi, Nick you may not look at the repeated test as an independend. By definition. So, the calculations this way have no sense. VVV On 18.03.2016 20:00, Myles, Nickolas [PH] wrote: > Hello everyone, > > I am wondering if this is true to assume that if the screening test is > repeated for the same patient over the period of time it will diminish > the false negative rate? > > Assuming, the FOBT test for colorectal cancer screening is 80% > sensitive, then FN rate will be 20% (p=0.2). Assuming the tests are > independent and the error is completely random, will repeated > screening over one year with FOBT make it 0,2x0.2=0.04 (4%) and over > the three years under 1% (0.2x0.2x0.2=0.008)? This many sound very > bold, but does the repeat measures work that way? of will it stay the > same 20% irrespective of the repeat measures? > > I am not talking about false positives, but the question can certainly > be extended for FP as well. > > thank you, will appreciate your opinion or reference. > > Nick > Vasiliy V. Vlassov, MD President, Society for Evidence Based Medicine (osdm.org) e-mail: vlassov[a t]cochrane.ru snail mail: P.O.Box 13 Moscow 109451 Russia Phone Russia +7(965)2511021 Подпишись на новости на osdm.org -- THANK YOU for deleting my e-mail address , any other addresses, and any personal information, from this e-mail, if you plan to forward it. Also, thank you for using “Bcc” instead of “To” and “Cc“ when initiating both individual and group e-mails. These extra actions on your part help to prevent spammers and hackers from obtaining addresses and thus help prevent the proliferation of spam. ------------------------------ Date: Mon, 21 Mar 2016 13:54:05 -0700 From: "Myles, Nickolas [PH]" <[log in to unmask]> Subject: Re: ANSWER TO will false negatives go down with the repeated same test? Thank you for the discussion. I have found the more direct answer to my question: Quotes from "Repeating tests: different roles in research studies and clinical medicine" Paul A Monach http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3651832/pdf/nihms434214.pdf Repeated measurements on the same person ..averaging repeated measurements on multiple specimens taken from the same person over time mitigates the effects of both within-subject biological variation and analytical imprecision. averaging of test results taken over time is a more efficient way to reduce total variation than replicate testing of individual samples, although only replicate testing can allow calculation of analytical imprecision (CVA) and estimation of within-subject biological variation ... .?.I bet for binary/ordinal variable test, the median will be the answer.?? Nick From: Yao, Xiaomei [mailto:[log in to unmask]] Sent: Monday, March 21, 2016 11:17 AM To: Myles, Nickolas [PH] Subject: RE: will false negatives go down with the repeated same test? Hi Nick, Now I understand your question:" If we repeat the same test for the same patient often and on the same sample, in case of a negative first test how many times one should repeat the test (which is 80% sensitive and 99% specific) in order to get true positive not false negative result?" My answer is "We don't know". Because when you run the test on the same sample or different sample from the same patient within one year, it likes a brand new procedure each time, and you should always get the same post-test probability, and it is just a PROBABILITY (not a YES or NO to have the disease or not) because FOBT test is not a reference standard. Xiaomei From: Myles, Nickolas [PH] [mailto:[log in to unmask]] Sent: March-21-16 1:13 PM To: Yao, Xiaomei; [log in to unmask]<mailto:[log in to unmask]> Subject: RE: will false negatives go down with the repeated same test? Thank Xiaomei and Brian, yes, the sens/spec are the same but the technical error due to test variation will most likely affect different patients each run (if we take test as a lab value, assuming we repeatedly test the same patient population, for which standard reference test result is known by the "supreme powers" ) . My question is harder that it seems: if we repeat the same test for the same patient often and on the same sample, in case of a negative first test how many times one should repeat the test (which is 80% sensitive and 99% specific) in order to get true positive not false negative result ? Here in the lab we are testing cancer samples for targeted therapies, so the question is practical and the repeat tests are requested often, so it's not just mental gymnastics. Thanks a lot for posting, Nick From: Yao, Xiaomei [mailto:[log in to unmask]] Sent: Monday, March 21, 2016 9:31 AM To: Myles, Nickolas [PH]; [log in to unmask]<mailto:[log in to unmask]> Subject: RE: will false negatives go down with the repeated same test? Hi Nick, If the patient population and the test are exactly the same, no matter when you repeat the test, the sensitivity and specificity should be the same theoretically. Xiaomei From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Myles, Nickolas [PH] Sent: March-18-16 1:01 PM To: [log in to unmask]<mailto:[log in to unmask]> Subject: will false negatives go down with the repeated same test? Hello everyone, I am wondering if this is true to assume that if the screening test is repeated for the same patient over the period of time it will diminish the false negative rate? Assuming, the FOBT test for colorectal cancer screening is 80% sensitive, then FN rate will be 20% (p=0.2). Assuming the tests are independent and the error is completely random, will repeated screening over one year with FOBT make it 0,2x0.2=0.04 (4%) and over the three years under 1% (0.2x0.2x0.2=0.008)? This many sound very bold, but does the repeat measures work that way? of will it stay the same 20% irrespective of the repeat measures? I am not talking about false positives, but the question can certainly be extended for FP as well. thank you, will appreciate your opinion or reference. Nick ------------------------------ End of EVIDENCE-BASED-HEALTH Digest - 19 Mar 2016 to 21 Mar 2016 (#2016-65) ***************************************************************************