On Fri, 20 Jun 2008, Peter W. Draper wrote:
> On Fri, 20 Jun 2008, David Berry wrote:
>
>> 2008/6/20 Tim Jenness <[log in to unmask]>:
>> > On Fri, 20 Jun 2008, Peter W. Draper wrote:
>> >
>> > > On Fri, 20 Jun 2008, Tim Jenness wrote:
>> > >
>> > > > 27322 (dberry):
>> > > > hds: Speed up the packing and unpacking of Structure Record
>> > > > Vector
>> > > > elements.
>> > > >
>> > > > libraries/hds/dat1_pack_srv.c
>> > > > libraries/hds/dat1_unpack_srv.c
>> > >
>> > > Begs the question, by how much?
>> > >
>> >
>> > From an email David wrote to me:
>> >
>> > "But even having made this change, vtune was still saying that 25% of
>> > the time is spent in hds_unpack_srv, so I had a look at the
>> > hds_unpack_srv.c file (and the corresponding hds_pack_srv.c file).
>> > These files convert between little endian integers stored in the disk
>> > file and native endian integers needed by the processor. I've
>> > re-worked the algorithm written by Brian to do this, and got a further
>> > decrease in run time from 118 seconds to 87 seconds. I have committed
>> > these changes."
>> >
>> > so pretty impressive. I'm a bit confused as to why it needed to
>> > convert at all though because I thought the files were made on an
>> > intel linux box so reading on an intel linux box should not require
>> > endian conversion.
>>
>
> Hmm not sure about this interpretation, looking at that I'd say that the
> disk-order of the bytes is fixed (as it must for the record-oriented
> meta-data, I'd only expect the data itself to be native), and all that this
> code does is pack and unpack that as written.
So on further reflection and reading David's message again, that's what he
is saying.
> (so there's no actual swapping, that's just the unpack doing the reverse
> of the pack).
However, that's slightly misleading. There is swapping, it's just hidden
by the bit shifting (which naturally operates in the endian order of the
machine, so the pack/unpack always works).
|