On Thu, 24 Sep 2009, Bradley Warren wrote:

> Hi Jamie,
>
> Thanks for bringing this to our attention, I've never noticed this issue with 
> NGLS data. At the moment I haven't been able to reproduce the problem you 
> describe with another data set, but I'm not entirely sure that the data I'm 
> trying it on was reduced with lehuakona and not humu (reduction was right 
> around the time I made that upgrade). Both my computers here already have 
> nanahope on them, and I'd rather not downgrade one of them to run the test. 
> At the moment I'm having trouble accessing my old computer at McMaster to 
> check which version was used there. Chris, could you please check if orchid2 
> is running at the moment? It's been a few weeks since I last tried to access 
> it, and at the moment the system is telling me it doesn't recognise the name 
> (I can still log into other computers).
>

  Hi Brad,

     I'm just forwarding your email to stardev proper, as I accidently cc'd 
it to "starlink" rather than "stardev" previously.

  Malcolm Currie is looking into it, (see an email he sent below), so I 
guess it is best to wait to see what the explanation was.

  One critical question for NGLS is how much does the occurrence of the bad 
fitting bug depend on the input .sdf and the command line parameters of 
mfittrend. This is probably easier to answer by looking at the code 
differences in the KAPPA versions rather than repeated testing on lots of 
cubes.

> One thing you might want to check is if you get the same error when you're 
> not using method=global or auto.
>

  I will try that.

  Jamie.

Date:    Tue, 22 Sep 2009 20:07:16 +0100
From:    "Malcolm J. Currie" <[log in to unmask]>
Subject: Re: Bug in Kappa's mfittrend.

> I think I've found a bug in mfittrend in the version of kappa
> (1.9-11) which was part of the lehuakona starlink release. It seems
> that straight line fitting can give very wrong fits for certain map
> regions in this release but works in the previous humu and, 
interestingly
> the latest nanahope release.

> This bug could potentially have serious repercussions for any NGLS
> data reduced in the 9 month time period where lehuakona was the main
> release.

Whatever it was has been fixed.  Are you asking that the offending
bug be relocated and a patched lehuakona be made?  Off the top of my
head I don't know what bug was introduced and subsequently fixed.

I'd seen something like this with higher-order polynomials behaving
badly at the spectrum ends, but not as extreme, when trying to automate
the masking and fitting for some of Remo's galaxies.  Thus I recommended
visual checking and why there was a warning message.  You could also
test for anomalies in the residuals.

> Details for reproduction are below.

I actually found it easier to output to B_smooth_bsl and B_smooth_msk
for the baseline and mask NDFs, and use DATACUBE/trendview, for example.

    % trendview -i B_smooth"(-5:3,7:12,)"

to check that the fits are suitable.  The fit lines should approximately
cover the original spectra apart from noise and features.  If you can
see the yellow spectrum and separate green fit curve, something has gone
wrong.

> I do not know if there is any platform dependence - the test machine
> for the buggy lehuakona results was an Ubuntu Linux Hardy Heron machine.

lehuakona reproduced the problem on 64-bit OpenSuSE 11.1.  It appears
mostly frequently in the largest and smallest but it's not a rule.  I
can see well-fit low- and high-signal spectra.

Malcolm

------------------------------