Dear Karl,

 

Many thanks for your email and my apologies for the delay in replying. 

 

I thought this was a very helpful summary of changes in DCM over the past few years and their underlying motivation.  I do not have much to add, but will only briefly comment on two issues related to the general issue that a generative model - such as DCM - is not just defined by its likelihood function, but also by its priors.  As you say, this means that changes in priors that occurred over the development of DCM (e.g., when unifying different DCM variants) sometimes influenced the results one obtained with different DCM versions.  For clarity and to facilitate replication, it thus helps if the priors used for a given analysis are reported explicitly.  Papers could, for example, list all prior densities in a table (in fact, this has often been done by papers introducing new DCM variants)  A more efficient way of doing this in application papers, however, is to simply report the code version (SPM update revision number) used for inverting a given DCM, e.g., “SPM update r4010”.  This enables everyone to look up the likelihood function and priors (and also inversion scheme) that were used to produce a particular result.  This may be a useful procedure for future publications with DCM (particularly for those that use older versions of SPM) ?

 

The other issue in this context is Bayesian model selection.  In practice, BMS has largely been concerned with the structural layout of the graph so far.  That is, the competing models typically vary with regard to which connections (or inputs) are assumed to be present; these connections (or inputs) then usually have an identical prior with non-zero variance (e.g., identical shrinkage priors for all connections).  In principle, however, one could extend model selection to address the nature of these priors; for example, this has been done previously when introducing the idea of anatomically informed priors (http://www.ncbi.nlm.nih.gov/pubmed/19523523).  An interesting question is whether there is some mileage in following a similar approach for “standard” DCM applications (that do not incorporate external information from anatomy or physiology).  For example, it might be useful to define sets of “canonical” priors which encode categorically different a priori beliefs about connections and inputs, respectively, and which could be used for family-level inference and Bayesian model averaging.  This is something I am starting to look into at the moment.

 

All the very best,

Klaas

 

 

From: Friston, Karl [mailto:[log in to unmask]] 
Sent: Sonntag, 11. März 2012 15:10
To: Hanneke den Ouden; Stephan, Klaas
Cc: [log in to unmask]
Subject:

 

Dear Hanneke,

 

I am copying this to the SPM helpline in case other people using DCM10 or DCM12 are encountering similar issues. I spoke to Klaas last week and he told me that several people were concerned about the differences between the results of model inversion using DCM8 and DCM10. It is probably useful to comment on the issue generally and then focus on the example you sent me.

 

The changes from DCM8 two DCM10 were driven by advances in two-state and stochastic dynamic causal modelling. These extensions and generalisations called for a revision and simplification of the prior densities (and parameterisation) of the models for fMRI used in DCM8. The idea was to make all DCMs for fMRI as internally compatible (and simple) as possible. I think it is important to emphasise that DCM is actually getting simpler with every revision and, in principle, easier to understand and tailor for specific applications. Having said this, the simplifications meant that with relatively noisy data and difficult inversions, DCM10 occasionally ‘flat-lined’. In other words, it supposed that the most likely explanation for the data was that they were caused entirely by observation noise. This led to the revision that Guillaume e-mailed about shortly after Christmas. This revision changed the priors – not on the parameters – but on the precision of observation noise – essentially telling DCM that there was signal to the explained. I will refer to this as DCM12.

 

In short, the continued development of DCM has entailed changes to the priors of both parameters and precisions and has, consequently, led to changes in posterior estimates and model selection. In one sense, this is a natural part of DCM development (and can be seen as informal model optimisation). On the other hand, I appreciate that this can be frustrating for people trying to publish reproducible results. As a general comment, the philosophy behind DCM is that it forces people to think mechanistically about how their data were caused – in terms of distributed neuronal processing. One could argue that the same issue is now emerging in terms of the prior beliefs about models in general and that people should be encouraged to try different priors – noting that every time you change the priors you implicitly change the model. Having said this, this is probably not terribly useful for practitioners of DCM, who would like to use canonical priors that work in most situations. Klaas and others are currently thinking about how to deal with this. Our internal experience here is that the current priors used in DCM12 provide more sensitive model (and parameter) inference than any previous version (the same appears to be true for the examples you sent me).

 

One thing that might help is to have some way of reassuring oneself that model inversion gives sensible results. Below, I enclose a short Matlab routine (that is compatible with early DCM.mat files), which provides some simple diagnostics and rules of thumb to ensure everything is okay (please see the help below for a fuller description and the attached PDF for an exemplar output).

 

 

In relation to the specific example you sent; I'll briefly describe history for people who are interested: you inverted the same dataset using DCM8, two versions of DCM10 and DCM12. As noted above, the two versions of DCM10 provided null results, with parameter estimates that were close their prior expectations, while DCM8 and DCM12 gave more interesting (if different) results. First, it may not be surprising that DCM10 flat-lined because this particular DCM is quite ambitious – the regions are very close together, which means that the data are highly correlated (the smallest correlation between regional responses was 0.84). Furthermore, you are allowed for full connectivity in both the A and B matrices. Finally, you used event related modulatory (bilinear) effects that were a subset of the driving events. Having said this, there are clearly interesting things going on as evidenced by the DCM8 and 12 inversions.

 

Looking at the results, they actually appear fairly consistent (although the bilinear effects focus on afferents to the first region in DCM12 and the second in DCM8). The posterior confidence and number of significant effects is greater with DCM12 than DCM8. The slight caveat here is the interesting self inhibition on the first region that is estimated to be positive (self excitation). It may well be the case that this region’s sensitivity becomes self excitatory during the transient appearance of the stimulus and you should believe the posterior estimates. Alternatively, your prior beliefs that regions should have self inhibition may be sufficiently strong to ignore this model. I increased the prior shrinkage variance on self connections (to 1/256) and fit the model – giving the results in the attached PDF. This had a free energy that was significantly better than either the original DCM8 or DCM12 inversions.

 

In summary, the fact that you were surprised by the results tells you immediately that they have violated your prior beliefs and that the specification of these beliefs in spm_dcm_fmri_priors was not sufficiently precise. When these prior beliefs were implemented the results seemed sensible and better than previous results. The outstanding problem is now how to operationalise this sort of model search and make it easy for people who are less experienced with DCM. I would value your thoughts and I hope that Klaas will comment upon this?

 

With very best wishes – Karl