I think another viable, supplementary approach is to use a reliability analysis. This shows how stable your findings are, and gives some indication if they are likely to replicate. You can use bootstrapping, or the jackknife which Marko proposed (http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0035578), which I used and found very informative, as well as it provided a post-hoc power analysis for all my clusters.

From: SPM (Statistical Parametric Mapping) [mailto:[log in to unmask]] On Behalf Of Tibor Auer
Sent: January-12-16 8:30 AM
To: [log in to unmask]
Subject: Re: [SPM] Reviewer's comment: "Liberal" thresholds in VBM

Dear Lasse

A threshold is always a trade-of between specificity and sensitivity: a more stringent threshold is more specific but less sensitive. A correction (FWE, FDR) does not affect this behavioural, because they are on the same ROC curve. Applying cluster extent threshold may help a bit, but only if it is based on statistics (probability of cluster extents). Long story short, a threshold only tells you how much you can trust in the result.

You can also run a power analysis, to figure out whether you need more sensitivity at the expense of specificity (i.e. lowering threshold) or at the expense of measurement (i.e. more subjects).

Another option is to use non-parametric (SnMP) rather than parametric test (http://arxiv.org/abs/1511.01863).

Vale,

Tibor

Auer, Tibor M.D. Ph.D.

MRC Cognition and Brain Sciences Unit
15 Chaucer Road
Cambridge
CB2 7EF

United Kingdom

Phone/Work: +44-(0)1223-273613

Mail: [log in to unmask]

From: SPM (Statistical Parametric Mapping) [mailto:[log in to unmask]] On Behalf Of Lasse Bang
Sent: Tuesday, January 12, 2016 10:19 AM
To: [log in to unmask]
Subject: [SPM] Reviewer's comment: "Liberal" thresholds in VBM

Dear experts,

I was hoping someone could comment on the following:

We have performed a VBM study (using the VBM8 toolbox, voxel dimensions are 1x1x1), where we used a two-sample t-test (whole-brain) to compare GM volumes of patients (n = 22) vs. controls (n = 22). We initially used a voxel-wise FWE threshold of p < .05, and then a more liberal threshold of p < .001, uncorrected for multiple comparisons. We were unable to detect any significant group differences using these thresholds.

A reviewer is suggesting we use an even more liberal threshold (p < .005, with a minimum cluster extent of 50 voxels). Using this threshold, some group differences emerged in the supplementary motor area (439 voxels), superior frontal gyrus (78 voxels), middle frontal gyrus (83 voxels), and supramarginal gyrus (125 voxels). At least of two of these areas are of particular interest, as previous research have indicated that patients may have reduced GM volume in these regions.

Initially, I was a bit reluctant to use this threshold, as I have rarely seen it in the VBM literature.

So far, I have referred to these results as «trend-level» group differences, and treat them as such.

What are your opinions on using such a threshold: Is it acceptable to present these results as trend-level differences?

I was thinking that one way to elucidate these group differences in greater detail would be to extract the mean values from each cluster

using the get_totals script; so that I could present an effect size measure of these results (and perhaps correlate regional volumes with other interesting variables). Does this approach seem like a reasonable effort to accommodate the reviewer's comment?

Any comments appreciated!

Best,

-Lasse

Lasse Bang,
Ph.D. candidate
Regional Department for Eating Disorders
Oslo University Hospital

Lasse Bang

Researcher / PhD Candidate

Regional Department for Eating Disorders (RASP)

Oslo University Hospital, Ullevål HF

Oslo, Norway

E-mail: [log in to unmask]" target="_blank">[log in to unmask]

Phone: +47 23 02 73 71 /+47 41 42 97 04

IKKE SENSITIVT INNHOLD