Dear Ged
Don't worry, you're not alone in the MVB world. :-)
I also have been playing with MVB on PET and fMRI and have encountered a
few of the "features" you have noticed...
Re number of voxels.
A large number of voxels (wrt. the number of scans) is not a problem.
The algorithm includes some regularization (or constraints) that
provides a stable solution even when the problem is under-determined.
Re stability of the estimate (evidence decreasing/increasing).
Have you checked the number of steps used by the ReML routine ? I found
that sometimes, it wouldn't converge but "oscillate".Then it would stop
after the maximum number of iterations (32). (At least for your
crossvalidation, it doesn't converge but this was to be expected. Ssee
below).
I found a little work around by decreasing the size of hyper-parameter
update step when this happens.
Karl, I can show you the "trick" but was afraid to commit it as it may
break things for other problems...
Re cross-validation.
Yep, it won't work as is with a "binary" model as yours, as it is
splitting the data into 2 parts, corresponding exactly to your groups.
There are a few possible solutions to this but, as Karl mentions it, it
requires changing the code.
BTW Karl, I would use a different crossvalidation depending on the data
used fMRI or PET/MVB.
Re applying MVB weights on unseen images (i.e. doing prediction)
This is also of great interest to me. Anyone with experience on this topic ?
BTW I also devised a little routine to save the weights into an image
file... I quite like the possibility of thresholding the weights that
are close to zero, and saving them as NaN. Much easier to display with
CheckReg on a structural image.
Best,
Chris
Karl Friston a écrit :
> Dear Ged,
>
> I am not sure there are that many 'MVBers' at the present time :)
>
> Although we have not solved the problem yet; it has a simple explanation:
> Your target vector and the two-way split used in cross-validation overlap
> completely. This means there is no variance in the target vector for
> either the
> test or training sets. I had not foreseen this because we validated
> MVB with
> fMRI time-series, where first vs, second-half contrasts are very uncommon.
>
> The fix is relatively simple and I will work on it as son as possible
> and ask
> Guillaume to check it into the updates. We will let know know when it
> is working.
>
> With very best wishes,
>
> Karl
>
>
>
>
>> Dear MVBers (!)
>>
>> I've just started to explore the new MVB decoding software, and have
>> got some slightly peculiar results, which I'd appreciate your comments
>> on. If you'd rather reply via the SPM mailing list, please do so. I'm
>> sorry this email is very long -- if you are in a rush, perhaps just
>> look at atrophy_smooth.ps, atrophy_sparse_xval.ps, and
>> atrophy_sparse_xval.txt on:
>> http://www.cs.ucl.ac.uk/staff/g.ridgway/mvb
>>
>> As far as I could tell from the paper, there should be no theoretical
>> or practical problems in applying the technique over a large mask, for
>> example the whole-brain analysis mask of a VBM study, or -- as I have
>> done in the results described below -- to a mask based on the regions
>> that survive statistical thresholding in a VBM study, in this case, I
>> chose pFDR<0.05, defining a set of approx. 40,000 voxels. Am I
>> mistaken in assuming this is theoretically reasonable? (The running
>> times are impressively short in practice.)
>>
>> The particular data I'm playing with at the moment is from a very
>> simple VBM comparison of 30 Alzheimer's disease patients to 30 healthy
>> controls, with an intracranial volume covariate. The contrast I am
>> trying to decode is the one-tailed test for greater volume in the
>> controls [-1 1 0]'. The data were processed in SPM5 as usual, except
>> that I've used only 5mm smoothing (I get similar MVB results with
>> 13mm, except, somewhat paradoxically perhaps, the sparse model becomes
>> better than the smooth in this case). I realise 8-12mm would be
>> better, and that DARTEL followed by lower smoothing would be better
>> still, but this was just an initial MVB experiment for me, not a
>> serious study.
>>
>> With support vectors, the evidence appears not to increase as expected
>> and the search seems to reach a stopping condition before the 16 steps
>> I requested, but that might be fine. The histogram of weights and the
>> prediction accuracy seem okay. For comparison purposes, a
>> leave-one-out cross-validation with libsvm (default C) gives 70%
>> accuracy.
>>
>> With the sparse basis, the evidence has a worrying dive with two
>> partitions, but recovers, then seems roughly constant for the
>> remainder of the 16 partitions. The histogram shows nice weight
>> sparsity, with the same bimodal distribution as one of the paper's
>> figures, and the predictions are very good.
>>
>> The smooth basis appears to show very severe over-fitting, visible in
>> both the bottom panels, unless I am misunderstanding these results.
>> The evidence actually seems to have a more reasonable progression for
>> this model. The model comparison favours the smooth model, apparently
>> not penalising the apparent over-fitting.
>>
>> I seem to be having problems with the cross-validation. The .txt files
>> are logs of the MATLAB command window output, showing problems with
>> singularity. Ignore the slightly non-standard warnings:
>> Warning: Returning 0.5 for non-positive v
>> Warning: Returning 0.5 for NaN x
>> I modified my spm_Tcdf because I didn't want it returning NaNs (for
>> some other purpose) but these modifications have not changed the
>> functionality at all, and in any case, the problems appear to be much
>> earlier in the cross-validation procedure. The ps files show that the
>> smooth and support vector bases have failed to produce any results,
>> while the sparse basis has results which make no sense to me. Please
>> forgive me if I am doing something stupid here...
>>
>> I am using SPM5 with updates 1111 -- the latest, I believe. My matlab
>> version:
>> >> ver
>> -------------------------------------------------------------------------------------
>> MATLAB Version 7.1.0.183 (R14) Service Pack 3
>> MATLAB License Number: xxxx
>> Operating System: Linux 2.4.21-53.ELsmp #1 SMP Wed Nov 14 03:54:12
>> EST 2007 i686
>> Java VM Version: Java 1.5.0 with Sun Microsystems Inc. Java
>> HotSpot(TM) Client VM mixed mode
>> -------------------------------------------------------------------------------------
>>
>> I have included my SPM.mat and the MVB.mat for the smooth model on the
>> above website. Please let me know if you need any further information,
>> or would like access to other MVB.mat files or other image data.
>>
>> Many thanks and best wishes,
>> Ged
>>
>> P.S. I've also noticed the following minor problem: when MVB is
>> started from the results GUI as usual, the original Graphics window is
>> left open, which is fine, except that when the print button is used on
>> the new MVB window, the call-back apparently finds the Graphics window
>> rather than the MVB one. I originally attempted to modify line 38 of
>> spm_print (version 902) adding
>> || strcmp(get(gcf,'Tag'),'MVB')
>> to the end of the line, but this hasn't fixed the problem, which I
>> assume results from spm_figure line 334
>> if nargin<2, F='Graphics'; else F=varargin{2}; end
>> being called-back with only one argument. I can't for the moment see
>> how to fix this, as I have little experience with GUI programming.
>> After closing the Graphics window, I've been able to print the MVB one
>> successfully.
>>
>> P.P.S. Someone on the mailing list asked about using the MVB weights
>> for further "unseen" images. I'm also interested in this, and was also
>> curious just to see the weights more closely than I could in the glass
>> brain. The function mvb2img on the above website seems to work for me
>> -- please feel free to include something derived from it in future
>> updates if you feel it might be useful, in any case, I will probably
>> offer it directly to the sender of the mailing list post. Please let
>> me know if you spot any obvious limitations of it, thank you.
>
--
===================================================
Christophe Phillips, Ir, PhD
F.N.R.S. Research Associate
Cyclotron Research Centre, B30
University of Liege, Sart Tilman
4000 Liege, Belgium
Tel: +32 4 366 2316 (secr.)
+32 4 366 2366
Fax: +32 4 366 2946
email: [log in to unmask]
web : http://www.ulg.ac.be/crc
===================================================
