Print

Print


Dear Ged,

I am not sure there are that many 'MVBers' at the present time :)

Although we have not solved the problem yet; it has a simple explanation:
Your target vector and the two-way split used in cross-validation overlap
completely. This means there is no variance in the target vector for either the
test or training sets.  I had not foreseen this because we validated MVB with
fMRI time-series, where first vs, second-half contrasts are very uncommon.

The fix is relatively simple and I will work on it as son as possible and ask
Guillaume to check it into the updates. We will let know know when it is working.

With very best wishes,

Karl




Dear MVBers (!)

I've just started to explore the new MVB decoding software, and have
got some slightly peculiar results, which I'd appreciate your comments
on. If you'd rather reply via the SPM mailing list, please do so. I'm
sorry this email is very long -- if you are in a rush, perhaps just
look at atrophy_smooth.ps, atrophy_sparse_xval.ps, and
atrophy_sparse_xval.txt on:
  http://www.cs.ucl.ac.uk/staff/g.ridgway/mvb

As far as I could tell from the paper, there should be no theoretical
or practical problems in applying the technique over a large mask, for
example the whole-brain analysis mask of a VBM study, or -- as I have
done in the results described below -- to a mask based on the regions
that survive statistical thresholding in a VBM study, in this case, I
chose pFDR<0.05, defining a set of approx. 40,000 voxels. Am I
mistaken in assuming this is theoretically reasonable? (The running
times are impressively short in practice.)

The particular data I'm playing with at the moment is from a very
simple VBM comparison of 30 Alzheimer's disease patients to 30 healthy
controls, with an intracranial volume covariate. The contrast I am
trying to decode is the one-tailed test for greater volume in the
controls [-1 1 0]'. The data were processed in SPM5 as usual, except
that I've used only 5mm smoothing (I get similar MVB results with
13mm, except, somewhat paradoxically perhaps, the sparse model becomes
better than the smooth in this case). I realise 8-12mm would be
better, and that DARTEL followed by lower smoothing would be better
still, but this was just an initial MVB experiment for me, not a
serious study.

With support vectors, the evidence appears not to increase as expected
and the search seems to reach a stopping condition before the 16 steps
I requested, but that might be fine. The histogram of weights and the
prediction accuracy seem okay. For comparison purposes, a
leave-one-out cross-validation with libsvm (default C) gives 70%
accuracy.

With the sparse basis, the evidence has a worrying dive with two
partitions, but recovers, then seems roughly constant for the
remainder of the 16 partitions. The histogram shows nice weight
sparsity, with the same bimodal distribution as one of the paper's
figures, and the predictions are very good.

The smooth basis appears to show very severe over-fitting, visible in
both the bottom panels, unless I am misunderstanding these results.
The evidence actually seems to have a more reasonable progression for
this model. The model comparison favours the smooth model, apparently
not penalising the apparent over-fitting.

I seem to be having problems with the cross-validation. The .txt files
are logs of the MATLAB command window output, showing problems with
singularity. Ignore the slightly non-standard warnings:
  Warning: Returning 0.5 for non-positive v
  Warning: Returning 0.5 for NaN x
I modified my spm_Tcdf because I didn't want it returning NaNs (for
some other purpose) but these modifications have not changed the
functionality at all, and in any case, the problems appear to be much
earlier in the cross-validation procedure. The ps files show that the
smooth and support vector bases have failed to produce any results,
while the sparse basis has results which make no sense to me. Please
forgive me if I am doing something stupid here...

I am using SPM5 with updates 1111 -- the latest, I believe. My matlab version:
>> ver
-------------------------------------------------------------------------------------
MATLAB Version 7.1.0.183 (R14) Service Pack 3
MATLAB License Number: xxxx
Operating System: Linux 2.4.21-53.ELsmp #1 SMP Wed Nov 14 03:54:12 EST 2007 i686
Java VM Version: Java 1.5.0 with Sun Microsystems Inc. Java
HotSpot(TM) Client VM mixed mode
-------------------------------------------------------------------------------------

I have included my SPM.mat and the MVB.mat for the smooth model on the
above website. Please let me know if you need any further information,
or would like access to other MVB.mat files or other image data.

Many thanks and best wishes,
Ged

P.S. I've also noticed the following minor problem: when MVB is
started from the results GUI as usual, the original Graphics window is
left open, which is fine, except that when the print button is used on
the new MVB window, the call-back apparently finds the Graphics window
rather than the MVB one. I originally attempted to modify line 38 of
spm_print (version 902) adding
  || strcmp(get(gcf,'Tag'),'MVB')
to the end of the line, but this hasn't fixed the problem, which I
assume results from spm_figure line 334
  if nargin<2, F='Graphics'; else F=varargin{2}; end
being called-back with only one argument. I can't for the moment see
how to fix this, as I have little experience with GUI programming.
After closing the Graphics window, I've been able to print the MVB one
successfully.

P.P.S. Someone on the mailing list asked about using the MVB weights
for further "unseen" images. I'm also interested in this, and was also
curious just to see the weights more closely than I could in the glass
brain. The function mvb2img on the above website seems to work for me
-- please feel free to include something derived from it in future
updates if you feel it might be useful, in any case, I will probably
offer it directly to the sender of the mailing list post. Please let
me know if you spot any obvious limitations of it, thank you.