Dear Bethany

It's fine to include 12 parameters from the DCM in the leave-one-out cross-validation (LOO) analysis - you're asking if the multivariate pattern of connection strengths is predictive of your covariate. However, I would be cautious about doing this repeatedly with different mixtures of connections - classical statistics are used for LOO, which means you have to consider multiple comparisons.



All the best

Peter



-----Original Message-----

From: SPM (Statistical Parametric Mapping) <[log in to unmask]> On Behalf Of Bethany Sussman

Sent: 06 July 2022 21:58

To: [log in to unmask]

Subject: [SPM] DCM LOO with several parameters



⚠ Caution: External sender





Hello DCM/PEB experts,



I've run a resting state DCM PEB with a couple of covariates (correlational). Each covariate correlates with several connections in the PEB.



From previous posts, I know that LOO-CV strategies with several possible parameters can include, for example, selecting the strongest effects to enter into the LOO test, as entering too many (or the entire A matrix) might not be meaningful/significant.



To explore the data a bit, I tried running a LOO for one of the covariates and including all of the A matrix connections that had > .95 Pp correlated (12 of 36). To my surprise, the LOO was significant, even with that many. The r value is 0.37, but the p is 0.005.



I tried running each of these connections in their own LOO (e.g. run 12 tests) and not all of the tests were signficant (even when not making multiple comparisons adjustments). In general, there were a few tests that were highly significant, a couple where p was between .01 and .05,  and then some that p was either around .2 or very large (> .5). I know this is a newer area, but I was wondering how meaningful the LOO with all 12 parameters was? Are the parameters where there was no predictive utility indvidually sort of just being 'pulled' by the ones with highly significant individual LOO tests?



I appreciate any and all guidance.



Thank you,



Bethany