Tim,

On Wed, 3 Dec 2003, Tim Jenness wrote:

> To minimize issues such as this we are proposing to develop an abstraction
> layer that will remove the algorithm writer from the DRAMA/NDF issues. The
> obvious model for the NDF part is HDX/NDX (since that can provide an
> identical view for both FITS and NDF) and I've recommended that the FTS
> team adopt it for their pipeline integration. Unfortunately HDX/NDX is
> only in Java which makes it hard to use from C.
>
>   + How much effort would it be to write a C version?
>   + Is there any detailed documentation on HDX (including an XML schema)?

1.  Quite a lot of effort, I think.  The Java HDX system doesn't have
a huge volume of code, but what is there is rather tricky.  Having said
that, quite a lot of the complication is to support functionality which
no-one else but me seems to be terribly excited about, namely the ability
to take a general XML file with HDX-namespace elements and attributes
embedded within it, and read and _modify_ this as if it were the simpler
DOM corresponding to the HDX elements alone.  As a result, no matter how
complicated the input XML, you still see just the HDX parts of it in
a non-namespaced DOM which has the real underlying DOM shadowning it.
This is valuable because it means that you can both make an XML file
an HDX file by simply adding a few attributes to it, or by creating
a separate file which points into it (this latter hasn't been tested
much, and though it should work, it isn't in the regression tests,
I don't think).  However, as I say, no-one else seems to feel this is
a valuable use case; I don't know if they're right.

The support for that does rather stress the compliance of the underlying
SAX/DOM libraries, as my message here on Tuesday? suggests (does anyone
have any comments about that, by the way, and whether we should in fact
revert to JDK1.4.1?).

If, however, it were enough to have a version of HDX which could read
but not modify in place the underlying DOM, then that would be easier to
reimplement.  And a version which omitted the whole namespace extraction
stuff altogether would be easier still.

However, another important part of the HDX layer is the support for
different base layers, such as FITS, exposing their structure through
a DOM.  That'd be pure bloody murder to do in C, though it'd probably
be relatively easily translatable into C++ (yuck!).

Quite separately, do you really need to write architectural parts of
the new applications in C?  I can see the need for something like that
for the computationally intensive parts, but isn't that instead a case for
just using JNI more?


2.  I probably agree that there is a lack of detailed documentation
for HDX.  I expected to do something with that over the summer,
because I expected that that was what I'd be talking about
at ADASS, but this wasn't how it worked out.  The overview at
<http://www.astro.gla.ac.uk/users/norman/note/2003/hdx-overview/>
has expanded from the first version that I produced immediately
before the Cambridge IVOA meeting, but that is still, basically,
it.  That illustrates at the bottom the namespacing stuff I
mentioned above.  There's nothing between that and the javadocs at
<http://www.starlink.ac.uk/starjava/javadocs/>.

I'm not _completely_ sure what's missing in that gap, but part of that
is because I/we need to clarify a little bit just what HDX does and
does not include.  Nothing major, but things like the question of what
exactly counts as valid.

As far as a schema goes, the DTD for HDX is

   <!ELEMENT hdx ANY>

(and no, I don't plan to use XSchema, abomination that it is).
Implementors possibly need a little more guidance than this.

Mark -- do you have any comments on this?

See you,

Norman


--
---------------------------------------------------------------------------
Norman Gray                        http://www.astro.gla.ac.uk/users/norman/
Physics and Astronomy, University of Glasgow, UK     [log in to unmask]