On Mon, 6 Mar 2006, Peter W. Draper wrote:
> HDS will make temporary copies of mapped data when it's not in the right
> format, not just the right type. So in this case are we talking about a
> bigendian PPC mapping data created on a littlendian PC?
>
> If that's the case KAPPA:NATIVE should make the problem go away (or an
> NDFCOPY)./
>
These were files created on the same computer (my ACSIS spectrum writer).
It's running on an AMD64 machine. I just set the HDS_64BIT to 1 and
retried it and I do indeed get a 5GB temp file on disk. (the input file
for stats is 7GB but a lot of that is resize overhead so 5GB is about
right for the data array).
It's strange, but the files created on the AMD64 machine are smaller than
on the Mac. I'm essentially doing:
do while
if (too small)
unmap
ndfSbnd + NGROW spectra
ndfMap
}
write spectrum
}
and on my mac this no longer works after adding a 1GB of data - the file
itself is nearly 2.5GB by this point. On 64bit the same amount of data
results in a much smaller overhead for the resize. I think datAlter is
doing something really strange but I can't look into it at the moment.
--
Tim Jenness
JAC software
http://www.jach.hawaii.edu/~timj
|