Dear FSL Experts,

We are a Depression-HCP site and are trying to run FIX to clean up rsfMRI and task fMRI data. We are having some issues with the accuracy of our study-specific trained classifier.

The training, classifying and cleaning stages are all working without error, but we are getting 0% for TNR in all cases for all thresholds.

Using our own manually labelled and trained classifier (n=26 scans but adding 5 more eventually), we achieve ~60% accuracy for TPR, but still zero for TNR (see below).

Using the HCP’s hp2000 classifier, we achieve ~30% accuracy for TPR, but still zero for TNR.

Could this be a problem with how FIX is calling each type of signal in fix4melview_trainingdataset_thr*.txt? We noticed when classification is run on the HCP data provided with FIX, components are classified as “signal, false” and “unclassified noise, true”. When we are running it on our training dataset, all of the signal components are listed as “unknown, false” not “signal, false”. Could this be the issue?

Attaching the results of LOO testing done using our own classifier, on the same 26 subjects that were used to train it.

63.8  0.0 58.6  0.0 55.2  0.0 46.6  0.0 34.5  0.0 31.0  0.0 29.3  0.0 25.9  0.0
69.8  0.0 67.9  0.0 62.3  0.0 54.7  0.0 39.6  0.0 39.6  0.0 35.8  0.0 35.8  0.0
51.1  0.0 45.5  0.0 39.8  0.0 29.5  0.0 22.7  0.0 13.6  0.0 12.5  0.0 12.5  0.0
65.2  0.0 61.7  0.0 47.5  0.0 37.6  0.0 27.0  0.0 23.4  0.0 17.7  0.0 12.1  0.0
50.6  0.0 49.4  0.0 38.2  0.0 30.3  0.0 22.5  0.0 21.3  0.0 18.0  0.0 18.0  0.0
41.3  0.0 33.7  0.0 29.3  0.0 25.0  0.0 19.6  0.0 17.4  0.0 16.3  0.0 15.2  0.0
52.9  0.0 52.9  0.0 44.3  0.0 40.0  0.0 38.6  0.0 34.3  0.0 32.9  0.0 30.0  0.0
57.9  0.0 55.3  0.0 47.4  0.0 43.4  0.0 40.8  0.0 36.8  0.0 36.8  0.0 35.5  0.0
53.3  0.0 50.7  0.0 46.7  0.0 41.3  0.0 36.0  0.0 30.7  0.0 30.7  0.0 24.0  0.0
46.5  0.0 43.0  0.0 38.4  0.0 30.2  0.0 25.6  0.0 22.1  0.0 22.1  0.0 22.1  0.0
55.9  0.0 52.9  0.0 51.5  0.0 42.6  0.0 38.2  0.0 35.3  0.0 32.4  0.0 30.9  0.0
51.5  0.0 47.1  0.0 44.1  0.0 38.2  0.0 35.3  0.0 33.8  0.0 32.4  0.0 32.4  0.0
51.7  0.0 48.3  0.0 32.6  0.0 23.6  0.0 14.6  0.0 12.4  0.0  7.9  0.0  4.5  0.0
41.0  0.0 32.1  0.0 21.8  0.0 14.1  0.0 10.3  0.0 10.3  0.0  9.0  0.0  9.0  0.0
63.6  0.0 61.8  0.0 50.9  0.0 47.3  0.0 38.2  0.0 34.5  0.0 32.7  0.0 27.3  0.0
70.6  0.0 58.8  0.0 41.2  0.0 32.4  0.0 29.4  0.0 22.1  0.0 16.2  0.0 13.2  0.0
80.3  0.0 73.2  0.0 62.0  0.0 53.5  0.0 45.1  0.0 26.8  0.0 21.1  0.0 15.5  0.0
51.2  0.0 48.8  0.0 42.9  0.0 36.9  0.0 32.1  0.0 17.9  0.0 15.5  0.0 14.3  0.0
96.6  0.0 96.6  0.0 96.6  0.0 96.6  0.0 96.6  0.0 96.6  0.0 93.1  0.0 58.6  0.0
36.9  0.0 35.4  0.0 30.8  0.0 27.7  0.0 20.0  0.0 16.9  0.0 16.9  0.0 15.4  0.0
60.0  0.0 60.0  0.0 58.2  0.0 52.7  0.0 49.1  0.0 47.3  0.0 41.8  0.0 36.4  0.0
100.0   0.0 100.0   0.0  94.4   0.0  85.2   0.0  79.6   0.0  74.1   0.0  68.5   0.0  63.0   0.0
68.3  0.0 65.9  0.0 63.4  0.0 53.7  0.0 42.7  0.0 40.2  0.0 39.0  0.0 34.1  0.0
66.2  0.0 63.6  0.0 57.1  0.0 51.9  0.0 46.8  0.0 42.9  0.0 35.1  0.0 35.1  0.0
42.7  0.0 35.9  0.0 30.1  0.0 25.2  0.0 19.4  0.0 18.4  0.0 18.4  0.0 13.6  0.0
49.4  0.0 45.7  0.0 42.0  0.0 35.8  0.0 27.2  0.0 23.5  0.0 22.2  0.0 19.8  0.0
41.0  0.0 34.6  0.0 33.3  0.0 29.5  0.0 26.9  0.0 23.1  0.0 17.9  0.0 16.7  0.0
52.6  0.0 48.7  0.0 39.5  0.0 27.6  0.0 23.7  0.0 21.1  0.0 21.1  0.0 19.7  0.0
62.2  0.0 54.1  0.0 48.6  0.0 40.5  0.0 35.1  0.0 27.0  0.0 27.0  0.0 27.0  0.0
60.0  0.0 56.7  0.0 51.7  0.0 40.0  0.0 31.7  0.0 26.7  0.0 23.3  0.0 21.7  0.0
42.2  0.0 30.1  0.0 18.1  0.0 16.9  0.0 15.7  0.0 13.3  0.0 10.8  0.0  6.0  0.0
40.2  0.0 34.1  0.0 28.0  0.0 25.6  0.0 19.5  0.0 14.6  0.0 13.4  0.0 12.2  0.0
63.6  0.0 62.1  0.0 56.1  0.0 54.5  0.0 45.5  0.0 33.3  0.0 28.8  0.0 22.7  0.0
56.5  0.0 52.2  0.0 46.4  0.0 44.9  0.0 33.3  0.0 30.4  0.0 30.4  0.0 26.1  0.0
54.5  0.0 47.7  0.0 44.3  0.0 40.9  0.0 31.8  0.0 29.5  0.0 23.9  0.0 22.7  0.0
58.2  0.0 49.4  0.0 41.8  0.0 39.2  0.0 30.4  0.0 26.6  0.0 25.3  0.0 25.3  0.0
42.4  0.0 39.4  0.0 37.9  0.0 36.4  0.0 34.8  0.0 34.8  0.0 31.8  0.0 27.3  0.0
56.8  0.0 50.0  0.0 39.8  0.0 34.1  0.0 30.7  0.0 28.4  0.0 26.1  0.0 22.7  0.0
75.4  0.0 71.9  0.0 64.9  0.0 57.9  0.0 49.1  0.0 42.1  0.0 36.8  0.0 33.3  0.0
72.2  0.0 66.7  0.0 51.9  0.0 51.9  0.0 48.1  0.0 42.6  0.0 40.7  0.0 40.7  0.0
72.3  0.0 68.1  0.0 48.9  0.0 40.4  0.0 36.2  0.0 27.7  0.0 25.5  0.0 25.5  0.0
43.8  0.0 42.2  0.0 40.6  0.0 37.5  0.0 31.2  0.0 29.7  0.0 26.6  0.0 25.0  0.0
65.9  0.0 57.6  0.0 52.9  0.0 49.4  0.0 41.2  0.0 34.1  0.0 29.4  0.0 27.1  0.0
61.5  0.0 57.7  0.0 53.8  0.0 46.2  0.0 35.9  0.0 29.5  0.0 28.2  0.0 25.6  0.0
62.7  0.0 61.2  0.0 49.3  0.0 44.8  0.0 34.3  0.0 34.3  0.0 32.8  0.0 31.3  0.0
51.7  0.0 46.7  0.0 43.3  0.0 43.3  0.0 28.3  0.0 23.3  0.0 21.7  0.0 20.0  0.0
59.7  0.0 56.7  0.0 46.3  0.0 44.8  0.0 40.3  0.0 37.3  0.0 35.8  0.0 26.9  0.0
68.9  0.0 65.5  0.0 58.0  0.0 53.8  0.0 42.0  0.0 33.6  0.0 26.1  0.0 19.3  0.0
53.0  0.0 50.6  0.0 42.2  0.0 36.1  0.0 25.3  0.0 18.1  0.0 14.5  0.0 13.3  0.0
44.6  0.0 37.0  0.0 33.7  0.0 29.3  0.0 25.0  0.0 20.7  0.0 16.3  0.0 14.1  0.0
51.5  0.0 44.1  0.0 32.4  0.0 30.9  0.0 23.5  0.0 23.5  0.0 22.1  0.0 20.6  0.0
45.6  0.0 44.1  0.0 38.2  0.0 29.4  0.0 22.1  0.0 22.1  0.0 22.1  0.0 19.1  0.0



set of thresholds is: 1   2   5  10  20  30  40  50
[TPR,TNR,(3*TPR+TNR)/4] pairs of results (averaged over datasets, one pair per threshold):

mean
57.6 53.3 46.4 40.8 34.1 29.8 27.2 24.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
43.2 40.0 34.8 30.6 25.6 22.4 20.4 18.0

median
56.2 51.4 44.3 40.0 32.7 28.0 25.8 22.7
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
42.2 38.6 33.2 30.0 24.6 21.0 19.3 17.0


I appreciate any input, and please let me know if I can provide any more information.

Thanks in advance!
Antoni Kubicki