The NDF data format has in the past been limited in size to no more
than 2,147,483,647 pixels (this is because the software used to access
NDFs uses 32 bit signed integers to store pixel counts, etc). Recent
work at the EAO has removed this limitation by changing the relevant
libraries to use 64 bit integers instead of 32 bit integers. This is a
necessary first step towards supporting future instruments that will
produce much larger data arrays.
One consequence of this work is a small change to the way NDFs are
stored on disk – the integer values that store the pixel origin of
each array are now stored as 64 bit integers (type “_INT64”) rather
than 32 bit integers (type “_INTEGER”). New NDFs created with these
_INT64 values will be readable by software in the “2018A” Starlink
release (July 2018) and subsequent nightly builds, but will not be
readable by older versions of Starlink. However, NDFs in the new
format can be converted to the old format by doing:
% setenv NDF_SHORTORIGIN 1
% kappa
% ndfcopy newndf.sdf oldndf.sdf
It should be noted that the changes described above do not mean that
existing Starlink commands in packages such as Kappa or SMURF can now
be used to create or process huge NDFs. It just means that the NDF
format itself, together with the infrastructure libraries needed to
access NDFs, are ready for the next stage in the process, which will
be the modification of Kappa, SMURF, etc, to use the new facilities.
The one exception is that the CUPID package has now been updated to
use the new facilities and so should now be capable of handling huge
NDFs.
The new features described above are present in the Starlink nightly
build on or after 27th November 2019.
As a final note, this is all new stuff so please be on the look-out
for any strange behaviour in the new version of Starlink and report it
to EAO.
David
########################################################################
To unsubscribe from the STARDEV list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=STARDEV&A=1
|