Dear Darren,
>Previously I had asked if taking the DCM coupling parameters to a second
>level for either 1 or 2 sample t-tests required a change from Bayesian to
>Frequentist stats. Although random effects Bayesian estimates are available
>for 1-sample tests in spm_dcm_sessions there is no equivalent code listed
>for 2-sample stats.
>
>We submitted a paper recently using DCM and got back the following comment
>from a reviewer (I suppose it's possible that the reviewer is one of you).
>
>>6) On page 8 you say that the Bayesian framework precludes the need for
>>multiple comparisons. Unfortunately, this is not true in your case
>>because you have adopted a two-stage procedure and have reverted to
>>classical inference at the second stage. I would suggest something like:
>>
>>"Our analysis adopted a two-stage procedure that is formally identical to
>>the summary statistic approach used in random effects analysis of
>>neuroimaging data. However, in our case the first level
>>(subject-specific) models were DCMs, whose parameters were the coupling
>>and changes in coupling induced by task. The conditional expectations or
>>modes of these parameters were taken to a second (between-subject) level
>>for classical inference using conventional T-tests and ANOVAs. Because
>>the separate T-test of each connection entailed multiple comparisons we
>>report our results at two levels. First descriptively at 0.05
>>(uncorrected) and second at P<0.001 which corresponds to a conservative
>>correction for multiple comparisons."
>
>Our experiment has subjects performing 2 tasks and then examining the group
>DCM parameters using 1-sample t-tests within task and 2-sample tests
>between tasks.
>
>So- is this reviewer correct, do we need to revert to classical stats or is
>there a Bayesian formulation available for this type of comparison?
The reviewer is absolutely right. You have used classical inference on
summary
statistics from a first-level model. In you case these were MAP
estimators. You
therefore need to protect yourself against false positives at the second
level.
There are ways of making Bayesian inferences at the second level (Will may
want
to comment here). These include Bayesian Model Averaging that, essentially
invokes
a further (between-subject) level to model. However, if classical inferences
suffice to make your point, then there is little motivation for pursuing
Bayesian inference at the between-subject level. Classical inference is
perfectly appropriate for testing hypotheses and is consistent with the
scientific process in neuroscience. Furthermore, most readers will find it
more accessible.
I hope this helps - Karl
|