2009/9/21 Tim Jenness <[log in to unmask]>:
>
>
> On Sun, Sep 20, 2009 at 9:10 PM, David Berry <[log in to unmask]>
> wrote:
>>
>> 2009/9/19 Tim Jenness <[log in to unmask]>:
>> >
>> >
>> > On Fri, Sep 18, 2009 at 9:44 PM, David Berry <[log in to unmask]>
>> > wrote:
>> >>
>> >> 2009/9/19 tim.jenness <[log in to unmask]>:
>> >> > On Sep 18, 2009, at 9:28 AM, David Berry wrote:
>> >> >
>> >> >> 2009/9/18 Tim Jenness <[log in to unmask]>:
>> >> >>>
>> >> >>> This is not in a release yet so it's worth experimenting to see how
>> >> >>> things
>> >> >>> go. My only concern is for NDFs in NDF extensions. I have a feeling
>> >> >>> that
>> >> >>> by
>> >> >>> default those should not have automatic history added. SCUBA-2
>> >> >>> files
>> >> >>> have
>> >> >>> lots of NDF extensions and adding history to all of them will not
>> >> >>> be
>> >> >>> efficient. Does NDF know if the NDF it is creating is inside a
>> >> >>> .MORE
>> >> >>> extension?
>> >> >>
>> >> >> No. But since it's a tuning parameter, it is under programatic
>> >> >> control. That is, the program can over-ride the setting of the
>> >> >> environment variable by calling ndfTune prior to creating the new
>> >> >> NDF.
>> >> >
>> >> > I think some more thought is required then. Forcing us to go back and
>> >> > add 3
>> >> > lines of code wherever we create an NDF that we do not want to
>> >> > include
>> >> > history is too much. (ie read current value, force value to false,
>> >> > reset
>> >> > value at end).
>> >>
>> >> The other option is to stick a single call to ndfTune in at the start
>> >> of the application (or even in the monolitjh) to switch auto-history
>> >> off throughout the whole application. Then we are back to the current
>> >> situation where no output NDFs have history by default. At the end of
>> >> the day, if we want some NDFs to have auto-history and others not,
>> >> then we will need to add code somehow to indicate which are which.
>> >>
>> >
>> > globally disabling it is no good. We *want* it for normal NDFs. It's the
>> > 10,000 NDFs in extensions that are going to be the main problem.
>> >
>> >>
>> >> > At the very least something like a single call of
>> >> >
>> >> > ndfHsmod( 'SKIP', indf, status );
>> >> >
>> >
>> > Adding one line to disable per-NDF is fine. Adding 3 is too much. So the
>> > above has to work.
>>
>> With this approach (calling ndfHsmod), the 10000 history components
>> will still be created - but will be left empty. That means each of the
>> 10000 NDFs will be bigger and will take longer to create. Why not just
>> do something like:
>>
>> Save old AUTO_HISTORY value
>> Set AUTO_HISTORY false
>> Create 10000 NDFs
>> Restore old AUTO_HISTORY value.
>>
>> assuming that the 10000 NDFs are created in a block of code that can
>> be delimitied in this way.
>
> That's a big assumption. Look at the number of NDFs inside a SCUBA-2 data
> file. They aren't all made at the same time.
OK - so maybe it couldn't be a single block, but could it be two? or
three? or four? There must surely be a loop somewhere that is creating
at least *most* of these 10000 NDFs?
> Why can't the HISTORY block be
> created at NDF close time, just as the history record itself is written?
> Then a SKIP instruction would not result in anything being created.
I've not looked into it yet, but my first thought is that by the time
the NDF is closed, there will be no way of knowing whether it was
created by NDF_NEW/NDF_CREAT (in which case you want to honour the
setting of AUTO_HISTORY) or by NDF_PROP (in which case you want to
ignore AUTO_HISTORY).
David
|
