On Wed, 16 Apr 2008, Malcolm J. Currie wrote:

> Tim,
>
>> I've got starbench working inside autoconf. I'm half inclined to promote
>> it out of obsolete. IRAF tests won't run.
>
> Good stuff.  It's worthwhile to preserve the Starlink legacy.
>
> Moving from obsolete sounds fine to me, although the documentation would
> have to reflect the non-working IRAF benchmarks.  You might want to
> consult the site managers, who may still have working IRAF versions.

Well, that depends on whether any one is intending to run it and wants the 
iraf tests.... I can't spend any time on that but anyone can patch it.

There are some autoconf tweaks required to fix the perl path and still 
some errors reporting the clock speed and amount of memory.

>
> I see it also uses private copies of KAPPA KAPSUB routines.
>

Yep. Can't do anything about it because KAPSUB is not a library we can 
link against outside of kappa. If we thought that was an issue we would 
have to move the two routines to kaplibs.

>> STARBENCH is an amazing mismatch of perl + csh + awk. I can't help but
>> think a simple little perl script would have been easier.
>
> Likely historical with Perl coming on to the scene after much of
> STARBENCH had been coded in csh and awk to handle the R or Perl.

Yes. It is amazing.

>
>> I haven't worked out how to get the starMark rating from "scan". I think
>> it's failing because the PISA test fails.
>
> You might consider switching from PISA to EXTRACTOR, although the former
> is likely to be more stable, the current o/p mismatch issue
> notwithstanding.

I'm not going to do much work on the individual benchmarks at the moment.

How do I recalibrate the STARmark number?

There is an issue that the ccdpack benchmark doesn't do what it used to do 
because the ARD masking file has been renamed. ccdexercise has been 
updated to use WCS as well so the starbench version should be synchronized 
with that (I have half a thought that the ccdpack benchmark should call 
ccdexercise and simply modify ccdexercise to support a -nodisplay option 
(which is the only local patch apart from the bit added at start and end 
to support benchmarks).

This all leads to a recalibrated STARmark number.

>
>> The one obvious problem is that on this 3GHz machine with 16GB of RAM
>> the benchmark does not last any where near long enough. 2.3 seconds for
>> the KAPPA test is the longest. The FFT test is significantly faster than
>> the example in SSN23.
>
> It's hardly surprsinig; we have come several Moore intervals since
> STARBENCH originated.  Since then the typical dataset size has increased
> significantly too.  If you were to extend STARBENCH, particularly with
> JAC in mind, you might include some processing of substantial cubes in
> the KAPPA tests, and ensure that it has AST well represented.

Yes. The SMURF MAKEMAP command was a real AST benchmark torture test. In 
the end David made massive speed improvements to AST to get SMURF working 
in "real time" (ie process the data faster than it took to observe it).

>
> Also STARBENCH is intended to be run many times to estimate the errors.
> On some of the first machines it ran on, longer tests might have
> deterred some SMs from running them.  Results were posted in the System
> Management Forum.

Yes.

-- 
Tim Jenness
JAC software
http://www.jach.hawaii.edu/~timj