Hi Bill,
Hmmm, sounds bad. Having a look at the
experiments you did it sounds to me like there
is something in image A that FLIRT doesn't
like, and that interpolation might be modifying.
If you cannot get a good registration with
FLIRT(A,C) or FLIRT(A,C) -init TAC
but you get something with FLIRT(TAC(A),C)
or with the composite transform TAC, without
re-registration then I am wondering about this
interpolation issue, as that would seem to be
a consistent factor. Thick slices may be part
of the issue.
Does this happen for other cost functions too?
I also saw that your original FLIRT call used
a -refweight. Is that "interesting" in any way?
I hope I do get to see the images, as that is normally
much more helpful for sorting these things out! :)
All the best,
Mark
On 19 May 2010, at 14:22, Crum, Bill wrote:
> Hi Mark,
>
> Thanks very much for this - it all helps get a better picture of
> what is happening.
> I am just confirming from my collaborators that it is OK to send you
> the data - will come back to you on that.
>
> Couple of additional things.
>
> (1) Just wanted to emphasise that this isn't a FLIRT versus OTHER
> scenario but rather FLIRT vs FLIRT. I found I was having problems
> registering scan A to scan C but found that I got reasonable results
> for A to B and B to C. So I generate a composite transformation TAC
> from TAB and TBC and then use this as an init in FLIRT to refine the
> registration from A to C directly. This is where it all started
> going wrong.
>
> (2) Undoubtedly cost function calculation is part of the story. My
> computation of NMI outside FLIRT is very simple and just involves a
> joint histogram of intensities with each voxel pair contributing to
> a single bin. So this is an obvious difference in how FLIRT computes
> NMI for registrations and how I do it outside FLIRT for comparison
> purposes. I would just accept this if there were only marginal
> differences in the measured and visually assessed similarities but
> it seems weird they are so far in disagreement.
>
> (3) I repeated my previous experiment, measuring NMI for (i) the
> unregistered pair A and C, (ii) the pair TAC(A) and C, and (iii) the
> pair FLIRT(A,C) -init TAC, and added a new case (iv) FLIRT(TAC(A), C).
>
> The image similarities I get are as follows:
> (i) UNreg A and C
> (cnscompare.py) cnscompare: ssd 665.736004758 ma 3.7240 mb
> 3.7854 je 7.4490 mi 0.0604 nmi 1.0081
>
> (ii) TAC(A) and C
> (cnscompare.py) cnscompare: ssd 444.313641584 ma 3.7240 mb
> 3.9368 je 7.1270 mi 0.5339 nmi 1.0749
>
> (iii) FLIRT(A, C) -init TAC
> (cnscompare.py) cnscompare: ssd 678.717882956 ma 3.7240 mb
> 3.6110 je 7.2752 mi 0.0598 nmi 1.0082
>
> (iv) FLIRT(TAC(A), C)
> (cnscompare.py) cnscompare: ssd 433.625924558 ma 3.7240 mb
> 3.9486 je 7.1318 mi 0.5408 nmi 1.0758
>
> Case (iv) works exactly how I might expect i.e. it slightly improves
> upon the original TAC (although this may just be an interpolation
> effect). Case (iii) fails as before. So the problem seems to arise
> with -init in (iii), but not when the transformed image is supplied
> to FLIRT directly (iv).
>
> Images A and C are high quality, with very similar intensity
> distributions and ranges and voxel dimensions and sizes. The only
> slightly unusual thing about them is relatively thick slices (about
> 5x in-plane voxel-size, image size ~ 256x256x50).
>
> Don't know if any of this helps or not - hopefully I'll be able to
> get you the data to play with.
>
> Thanks for your patience.
>
> -Bill
>
> ---
> Dr Bill Crum, Senior Lecturer,
> King's College London,
> Centre for NeuroImaging Sciences (PO89)
> Institute of Psychiatry, De Crespigny Park, London, SE5 8AF
> tel: +44 (0)20 3228 3043, fax: +44 (0)20 3228 2116
>
>
>> -----Original Message-----
>> From: FSL - FMRIB's Software Library [mailto:[log in to unmask]] On
>> Behalf Of Mark Jenkinson
>> Sent: 17 May 2010 09:52
>> To: [log in to unmask]
>> Subject: Re: [FSL] flirt schedule file problems
>>
>> Hi Bill,
>>
>> I'll describe my calculation of NMI below, but I am
>> puzzled by this behaviour. It sounds like you are
>> doing everything sensibly but it is ranking the
>> registrations differently in the different packages.
>> There are some slightly different things that
>> FLIRT does when calculating the cost functions,
>> but these would affect things in a big way I didn't
>> think. Would you be able to upload the data for
>> me to have a look at?
>> We have an upload site at:
>> http://www.fmrib.ox.ac.uk/cgi-bin/upload.cgi
>> and you need to send me the reference number
>> for the upload.
>>
>> As for the calculation of NMI - what FLIRT does
>> is to create the joint histogram by going through
>> every reference voxel and firstly looking up the
>> intensity and weight value in the corresponding
>> input and inweight volumes (using interpolation).
>> It then weights the contribution by the product of
>> the interpolated inweight, the refweight (at this
>> voxel) and a geometric weighting which
>> progressively downweights points near the edge
>> (normally only point *within* the outer voxel edge
>> are affected by this, with the default parameters).
>> It also only does this if the corresponding input
>> voxel coordinate is inside the input volume's FOV.
>> Finally, it assigns the weighted contribution (which
>> is 1.0 if all weights are unity) to the joint histogram
>> but in a "fuzzy" way - by assigning partial contributions
>> to a 2x2 set of bins where their centres bound
>> the continuous intensity-pair-coordinate. This is
>> the same as using a trilinear interpolation kernel
>> and doing Parzen windowing. So it doesn't just
>> assign one pair to the "nearest" bin (like a
>> nearest neighbour interpolation kernel would do)
>> but spreads it slightly over bins.
>>
>> So I don't know if any of this has on obvious impact
>> on your problem. If it does please let me know.
>> The interpolation of the input weighting volume
>> can cause some problems if you've got a mixture
>> of very thin and thicker areas, as the thin ones
>> often get downweighted.
>>
>> Anyway, hope this helps.
>> All the best,
>> Mark
>>
>>
>> On 14 May 2010, at 10:51, Crum, Bill wrote:
>>
>>> Hi Mark,
>>>
>>> Sorry this is a little lengthy again - I'm trying to be as clear as
>>> possible.
>>>
>>> Everything you say makes sense and squares with what I thought but
>>> my experience differs a little. With your help, I think my problem
>>> boils down to how the cost function is evaluated. Couple of
>>> clarifications first.
>>>
>>> Just to be clear. I know the init matrix is added to everything - I
>>> want to see whether FLIRT improves matters starting from the init.
>>> So when I talked about doing "nothing" or an identity transform I
>>> meant this in addition to the init.
>>>
>>> My optimise command with 0 max iterations was just an attempt to
>>> measure the cost associated with the init matrix (by not allowing
>>> optimise to perform any iterations). I now do this directly by using
>>> setrow and measurecost and get the same results.
>>>
>>> I concede some confusion on my part over cost versus similarity in
>>> FLIRT. I agree that sort functions as advertised and that from a
>>> cost-function perspective it does the right thing. I think my
>>> problem actually stems from *how* the cost function is computed.
>>>
>>> If you have any further advice much appreciated.
>>>
>>> Here's my FLIRT command:
>>> -----------------------
>>> flirt -v -dof 9 -bins 64 -cost normmi -schedule ../test.sch -init
>>> test_proc_TYPHOON_213213-to-1.mat -refweight proc_NIMROD_1616-mask-
>>> norm.nii.gz -ref proc_NIMROD_1616.nii.gz -in
>>> proc_TYPHOON_213213.nii.gz -omat test-reg-213213.mat -out test-
>>> reg-213213.nii.gz
>>>
>>> Here's part of the FLIRT output - the result of optimise is in the
>>> first row of UF and the identity is in the second row:
>>> ----------------------------------
>>>>> measurecost MAXDOF UF:1-2 0.0 0.0 0.0 0.0 0.0 0.0
>>>>> 0.0 0.0 0.0 rel
>>>>> print U
>>> -0.008305 0.945402 0.305760 0.100657 0.000000 -0.306213 0.854226
>>> 0.281214 0.000000 0.000000 -0.312695 0.949854 0.000000 55.184867
>>> 18.229680 -22.503618 1.000000
>>> -0.004685 1.000000 0.000000 0.000000 0.000000 0.000000 1.000000
>>> 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000
>>> 0.000000 0.000000 1.000000
>>>
>>> Which from what you say means that the first row (result from
>>> optimise) has the lowest cost and is therefore output. The confusion
>>> is that if I compute NMI outside FLIRT using the same number of
>>> bins, same mask and same images I get the following:
>>>
>>> UNreg
>>> (cnscompare.py) cnscompare: ssd 665.736004758 ma 3.0332 mb
>>> 3.1016 je 6.0847 mi 0.0501 nmi 1.0082
>>> With init only
>>> (cnscompare.py) cnscompare: ssd 444.313641584 ma 3.0332 mb
>>> 3.2428 je 5.7635 mi 0.5126 nmi 1.0889
>>> FLIRT
>>> (cnscompare.py) cnscompare: ssd 683.167758922 ma 3.0332 mb
>>> 3.0764 je 6.0440 mi 0.0656 nmi 1.0109
>>>
>>> The exact values of NMI aren't important - I guess there's some
>>> transformation to the cost-function. However the ranking is
>>> different between the different transformation cases inside and
>>> outside FLIRT - hence the confusion. Visually the init-only case is
>>> an excellent result and has the highest NMI outside FLIRT. But the
>>> FLIRT result is catastrophically wrong and has a much lower NMI
>>> computed outside FLIRT but also a lower cost function computed
>>> inside FLIRT. I also tried running with different numbers of bins
>>> and allowing optimise to run more iterations but get the same
>>> results. Any ideas?
>>>
>>> Thanks for ploughing through this.
>>>
>>> Best Wishes
>>>
>>> -Bill
>>>
>>> ---
>>> Dr Bill Crum, Senior Lecturer,
>>> King's College London,
>>> Centre for NeuroImaging Sciences (PO89)
>>> Institute of Psychiatry, De Crespigny Park, London, SE5 8AF
>>> tel: +44 (0)20 3228 3043, fax: +44 (0)20 3228 2116
>>>
>>>> -----Original Message-----
>>>> From: FSL - FMRIB's Software Library [mailto:[log in to unmask]] On
>>>> Behalf Of Mark Jenkinson
>>>> Sent: 14 May 2010 00:41
>>>> To: [log in to unmask]
>>>> Subject: Re: [FSL] flirt schedule file problems
>>>>
>>>> Hi Bill,
>>>>
>>>> The -init matrix is added to everything, so that this becomes
>>>> the new point of reference. Hence an "identity" matrix in
>>>> the schedule file would actually be the same transformation
>>>> as the -init matrix. I hope that makes sense. Effectively,
>>>> any matrix inside the schedule file will have the init matrix
>>>> post-multiplied with it at the beginning of the optimisation.
>>>>
>>>> I'm a bit puzzled about your problem with sort. All values
>>>> used by FLIRT (and this is also true of the documentation
>>>> for schedule files and the terminology in general) is *cost*
>>>> function value - not similarity values. So ascending order
>>>> means that the lowest values ends up in row 1. However,
>>>> lowest cost is the best value, so this is what should be
>>>> used by FLIRT later on.
>>>>
>>>> As for your optimisation problem - is it related to the
>>>> *cost* vs *similiarity* function issue? It is certainly
>>>> possible that starting with a different transformation
>>>> can lead to a better solution (especially when bypassing
>>>> the search/perturbation phases) but your identity
>>>> matrix will be the same as just using the -init matrix,
>>>> so I don't see how you currently have two different
>>>> starting points anyway. I'm also not sure what the code
>>>> does when you specify 0 as the last value in the optimise
>>>> call, as this is the maximum number of iterations, and
>>>> zero doesn't make much sense. So maybe this is a
>>>> problem?
>>>>
>>>> Anyway, I hope this is enough info to help you solve
>>>> the problem.
>>>>
>>>> All the best,
>>>> Mark
>>>>
>>>>
>>>>
>>>> On 13 May 2010, at 10:29, Crum, Bill wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm having some problems with a custom FLIRT schedule file and
>>>>> need
>>>>> a sanity check.
>>>>>
>>>>> Essentially I want to run with a very good initialisation matrix
>> and
>>>>> just do a fine-scale (1mm) adjustment. My initialisation matrix
>>>>> seems to load OK.
>>>>>
>>>>> Problem 1:
>>>>> The optimise command can result in a worse result than doing
>>>>> nothing. It doesn't seem to start from the identity transform even
>>>>> with zero perturbation. Not sure if this is a consequence of using
>>>>> an init matrix, a bug, a feature or a misunderstanding on my part?
>>>>>
>>>>> Fudge 1:
>>>>> So I explicitly measure the cost for the identity transform and
>>>>> use
>>>>> sort to see whether this gives a better result than optimise.
>>>>>
>>>>> Problem 2:
>>>>> The sort command behaves as advertised by sorting in ascending
>> order
>>>>> of similarity. However the first row (least similar) of the matrix
>>>>> gets passed back to flirt rather than the last row (most similar).
>>>>> So in my case it discards the best result.
>>>>>
>>>>> Unfortunately I can't figure out a Fudge for 2. I need to either
>>>>> change the sort order or selectively delete matrix rows. Of course
>>>>> even better if optimise behaved better in the first place ...
>>>>>
>>>>> Below is my schedule file and flirt command. I've tried it a
>>>>> number
>>>>> of different ways but always get the same result.
>>>>> I tested this by commenting/uncommenting out the sort command
>>>>> below
>>>>> and varying the order of the optimise commands.
>>>>>
>>>>> Are these real problems or have I just broken the schedule handler
>>>>> by doing something non-standard further up?
>>>>>
>>>>> Any help greatfully received - this is driving me nuts!
>>>>>
>>>>> Many Thanks
>>>>>
>>>>> -Bill Crum
>>>>>
>>>>> test.sch
>>>>> -------------------------------------------------------------------
>> --
>>>>> # 8mm scale
>>>>> setscale 8
>>>>> clear S
>>>>> clear P
>>>>>
>>>>> # 4mm scale
>>>>> setscale 4
>>>>>
>>>>> # 2mm scale
>>>>> setscale 2
>>>>>
>>>>> # 1mm scale
>>>>> setscale 1
>>>>> setoption smoothing 1
>>>>> setoption boundguess 1
>>>>>
>>>>> # Optimise from this position
>>>>> clear UF
>>>>> setrow UF 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1
>>>>>
>>>>> clear U
>>>>> # Dummy optimise with 0 iterations
>>>>> # I also tried just adding an identity row to UF directly but get
>>>>> the same result
>>>>> optimise MAXDOF UF:1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
>>>>> 0.0 rel 0
>>>>> # Optimise from this position
>>>>> optimise MAXDOF UF:1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
>>>>> 0.0 rel 1
>>>>>
>>>>> clear UF
>>>>> copy U UF
>>>>> clear U
>>>>> measurecost MAXDOF UF:1-2 0.0 0.0 0.0 0.0 0.0 0.0
>>>>> 0.0 0.0 0.0 rel
>>>>> print U
>>>>> sort U
>>>>> print U
>>>>> -------------------------------------------------------------------
>> --
>>>>>
>>>>> flirt -cost normcorr -schedule ../test.sch -nosearch -v -dof 9 -
>>>>> refweight proc_NIMROD_1616-mask-norm-d2.nii.gz -ref
>>>>> proc_NIMROD_1616.nii.gz -in proc_TYPHOON_212212.nii.gz -init
>>>>> test_proc_TYPHOON_213213-to-1.mat -omat test-reg-213213.mat -out
>>>>> test-reg-213213.nii.gz
>>>>>
>>>>> ---
>>>>> Dr Bill Crum, Senior Lecturer,
>>>>> King's College London,
>>>>> Centre for NeuroImaging Sciences (PO89)
>>>>> Institute of Psychiatry, De Crespigny Park, London, SE5 8AF
>>>>>
>>>
>
|