Thank you for the replies.
We are using a similar protocol as the lifespan HCP (3T Prisma with 2mm isotropic voxel size, 800ms TR, 37ms TE, 8x multiband accel. factor, 208mm FOV, 72 slices, 488 volumes) with both AP and PA phase encoding directions for resting state. This example was run on both directions included in the same classifier, but even when we run them separately, still getting 0% for TNR.
When we use the HCP-generated training-weights files on our data, we get lower TPR (~30%) but still zero for TNR. (Attaching below). I am also sharing an example .ica folder from our data, along with the hand-labeled text file of noise components.
https://drive.google.com/open?id=11XKawhqbg_YJqISmiUm0n8X3P_kH2LgQ
We tested the HCP-generated classifer on the HCP-provided data it was trained on, and got ~100% accuracy for both TPR and TNR, so FIX itself seems to be working correctly.
I have double checked the hand labels and they seem to be in the correct format with square brackets and commas.
Also doing another check to look at components that FIX is classifying as "noise" and comparing them to our hand labels for a few scans. Will report back on the results of that..
Thanks again, we're having difficulty pinning down the problem.
Antoni Kubicki
Results of HCP-trained classifier tested on our data (26 rs runs with AP/PA phase encoding)
34.5 0.0 32.8 0.0 27.6 0.0 25.9 0.0 24.1 0.0 24.1 0.0 24.1 0.0 22.4 0.0
43.4 0.0 39.6 0.0 37.7 0.0 35.8 0.0 32.1 0.0 28.3 0.0 28.3 0.0 28.3 0.0
18.2 0.0 18.2 0.0 15.9 0.0 13.6 0.0 12.5 0.0 12.5 0.0 12.5 0.0 10.2 0.0
27.7 0.0 27.7 0.0 20.6 0.0 12.8 0.0 11.3 0.0 11.3 0.0 11.3 0.0 11.3 0.0
21.3 0.0 20.2 0.0 18.0 0.0 16.9 0.0 16.9 0.0 16.9 0.0 16.9 0.0 15.7 0.0
23.9 0.0 23.9 0.0 18.5 0.0 14.1 0.0 13.0 0.0 13.0 0.0 13.0 0.0 12.0 0.0
34.3 0.0 34.3 0.0 31.4 0.0 31.4 0.0 31.4 0.0 28.6 0.0 27.1 0.0 27.1 0.0
47.4 0.0 44.7 0.0 35.5 0.0 31.6 0.0 28.9 0.0 27.6 0.0 26.3 0.0 26.3 0.0
30.7 0.0 30.7 0.0 29.3 0.0 28.0 0.0 28.0 0.0 25.3 0.0 25.3 0.0 22.7 0.0
29.1 0.0 29.1 0.0 26.7 0.0 24.4 0.0 23.3 0.0 23.3 0.0 20.9 0.0 18.6 0.0
41.2 0.0 41.2 0.0 32.4 0.0 32.4 0.0 30.9 0.0 29.4 0.0 27.9 0.0 26.5 0.0
47.1 0.0 45.6 0.0 42.6 0.0 38.2 0.0 35.3 0.0 32.4 0.0 32.4 0.0 30.9 0.0
13.5 0.0 13.5 0.0 10.1 0.0 6.7 0.0 5.6 0.0 4.5 0.0 2.2 0.0 1.1 0.0
11.5 0.0 11.5 0.0 10.3 0.0 9.0 0.0 6.4 0.0 6.4 0.0 2.6 0.0 1.3 0.0
34.5 0.0 32.7 0.0 27.3 0.0 27.3 0.0 27.3 0.0 25.5 0.0 23.6 0.0 23.6 0.0
29.4 0.0 26.5 0.0 20.6 0.0 19.1 0.0 19.1 0.0 16.2 0.0 13.2 0.0 13.2 0.0
16.9 0.0 16.9 0.0 15.5 0.0 15.5 0.0 15.5 0.0 14.1 0.0 14.1 0.0 14.1 0.0
21.4 0.0 19.0 0.0 16.7 0.0 16.7 0.0 16.7 0.0 15.5 0.0 15.5 0.0 14.3 0.0
60.3 0.0 37.9 0.0 17.2 0.0 17.2 0.0 17.2 0.0 17.2 0.0 15.5 0.0 13.8 0.0
27.7 0.0 24.6 0.0 23.1 0.0 20.0 0.0 16.9 0.0 16.9 0.0 15.4 0.0 15.4 0.0
38.2 0.0 38.2 0.0 38.2 0.0 36.4 0.0 36.4 0.0 36.4 0.0 32.7 0.0 32.7 0.0
44.4 0.0 42.6 0.0 38.9 0.0 38.9 0.0 38.9 0.0 37.0 0.0 33.3 0.0 33.3 0.0
42.7 0.0 39.0 0.0 37.8 0.0 37.8 0.0 35.4 0.0 35.4 0.0 34.1 0.0 34.1 0.0
46.8 0.0 45.5 0.0 42.9 0.0 41.6 0.0 40.3 0.0 39.0 0.0 39.0 0.0 36.4 0.0
17.5 0.0 15.5 0.0 15.5 0.0 15.5 0.0 15.5 0.0 15.5 0.0 15.5 0.0 15.5 0.0
29.6 0.0 28.4 0.0 23.5 0.0 19.8 0.0 17.3 0.0 14.8 0.0 14.8 0.0 13.6 0.0
33.3 0.0 28.2 0.0 24.4 0.0 21.8 0.0 17.9 0.0 16.7 0.0 15.4 0.0 12.8 0.0
28.9 0.0 27.6 0.0 25.0 0.0 22.4 0.0 19.7 0.0 19.7 0.0 18.4 0.0 17.1 0.0
29.7 0.0 29.7 0.0 29.7 0.0 27.0 0.0 24.3 0.0 21.6 0.0 18.9 0.0 18.9 0.0
33.3 0.0 31.7 0.0 30.0 0.0 30.0 0.0 26.7 0.0 25.0 0.0 23.3 0.0 20.0 0.0
16.9 0.0 14.5 0.0 9.6 0.0 7.2 0.0 6.0 0.0 6.0 0.0 4.8 0.0 4.8 0.0
36.6 0.0 30.5 0.0 19.5 0.0 12.2 0.0 8.5 0.0 8.5 0.0 8.5 0.0 7.3 0.0
28.8 0.0 28.8 0.0 25.8 0.0 25.8 0.0 22.7 0.0 22.7 0.0 21.2 0.0 18.2 0.0
33.3 0.0 33.3 0.0 31.9 0.0 27.5 0.0 24.6 0.0 23.2 0.0 23.2 0.0 23.2 0.0
31.8 0.0 29.5 0.0 26.1 0.0 25.0 0.0 25.0 0.0 25.0 0.0 22.7 0.0 22.7 0.0
36.7 0.0 32.9 0.0 26.6 0.0 25.3 0.0 24.1 0.0 22.8 0.0 21.5 0.0 21.5 0.0
37.9 0.0 37.9 0.0 33.3 0.0 30.3 0.0 30.3 0.0 30.3 0.0 28.8 0.0 27.3 0.0
34.1 0.0 30.7 0.0 26.1 0.0 25.0 0.0 20.5 0.0 19.3 0.0 18.2 0.0 18.2 0.0
40.4 0.0 38.6 0.0 36.8 0.0 33.3 0.0 29.8 0.0 29.8 0.0 29.8 0.0 28.1 0.0
44.4 0.0 42.6 0.0 38.9 0.0 37.0 0.0 33.3 0.0 31.5 0.0 29.6 0.0 27.8 0.0
38.3 0.0 36.2 0.0 29.8 0.0 27.7 0.0 27.7 0.0 25.5 0.0 23.4 0.0 21.3 0.0
35.9 0.0 35.9 0.0 34.4 0.0 29.7 0.0 28.1 0.0 26.6 0.0 25.0 0.0 23.4 0.0
31.8 0.0 30.6 0.0 28.2 0.0 28.2 0.0 28.2 0.0 28.2 0.0 24.7 0.0 23.5 0.0
39.7 0.0 38.5 0.0 30.8 0.0 30.8 0.0 26.9 0.0 26.9 0.0 26.9 0.0 24.4 0.0
26.9 0.0 23.9 0.0 23.9 0.0 23.9 0.0 23.9 0.0 23.9 0.0 23.9 0.0 23.9 0.0
28.3 0.0 28.3 0.0 26.7 0.0 25.0 0.0 21.7 0.0 20.0 0.0 16.7 0.0 16.7 0.0
38.8 0.0 37.3 0.0 32.8 0.0 28.4 0.0 26.9 0.0 25.4 0.0 23.9 0.0 23.9 0.0
40.3 0.0 38.7 0.0 31.1 0.0 26.1 0.0 22.7 0.0 17.6 0.0 16.0 0.0 15.1 0.0
25.3 0.0 22.9 0.0 20.5 0.0 18.1 0.0 14.5 0.0 13.3 0.0 13.3 0.0 13.3 0.0
25.0 0.0 23.9 0.0 18.5 0.0 16.3 0.0 15.2 0.0 14.1 0.0 12.0 0.0 12.0 0.0
26.5 0.0 22.1 0.0 20.6 0.0 20.6 0.0 19.1 0.0 19.1 0.0 19.1 0.0 19.1 0.0
29.4 0.0 27.9 0.0 23.5 0.0 22.1 0.0 22.1 0.0 20.6 0.0 19.1 0.0 16.2 0.0
set of thresholds is: 1 2 5 10 20 30 40 50
[TPR,TNR,(3*TPR+TNR)/4] pairs of results (averaged over datasets, one pair per threshold):
mean
32.4 30.4 26.5 24.5 22.8 21.7 20.5 19.5
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
24.3 22.8 19.9 18.4 17.1 16.3 15.4 14.6
median
32.6 30.5 26.6 25.2 23.6 22.8 21.1 19.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
24.4 22.9 20.0 18.9 17.7 17.1 15.8 14.3
|