here is a minimal Fortran program that writes out the contents as H,K,L,FOBS,SDFOBS - could be fed into contemporary F2MTZ to produce a current MTZ file :
REAL h,k,l,fobs,sfobs
OPEN(1,file='hklin',convert='BIG_ENDIAN',ACCESS='STREAM')
DO i=1,20 !skip first 20 4-byte units
READ(1)h
END DO
DO i=1,17218 !print the rest
READ(1)h,k,l,fobs,sfobs
print*,h,k,l,fobs,sfobs
END DO
END
You must symlink semioxyhb_tstate_BetaCobalt.mtz to hklin before running the program
The knowledge about the names of columns, and their number, comes from
strings semioxyhb_tstate_BetaCobalt.mtz
HTH,
Kay
On Wed, 14 Nov 2018 10:25:30 +0000, Eleanor Dodson <[log in to unmask]> wrote:
>Here is the file I was trying to read - please feel free to play with it!!
>Eleanor
>
>On Wed, 14 Nov 2018 at 10:17, Harry Powell <
>[log in to unmask]> wrote:
>
>> Hi
>>
>> Weren't the CCP4 base-level routines re-written from FORTRAN to C sometime
>> in the late 1990's? Very occasionally I used to find bugs that had been
>> introduced in this process (or possibly not corrected...) so it's possible
>> that Eleanor's file might be readable with a really old code base.
>>
>> BTW, I have recently had cause to search out a VMS system and have found
>> someone with a whole farm of running VAXes (or VAXen if you're geeky).
>>
>> Harry
>>
>> On 14 Nov 2018, at 09:46, Ian Tickle wrote:
>>
>> The CCP4 routines for MTZ and map files are written in C and thus do not
>> use a Fortran unformatted OPEN statement, they use C-style block read &
>> write.
>>
>> Cheers
>>
>> -- Ian
>>
>>
>> On Wed, 14 Nov 2018 at 08:59, Kay Diederichs <
>> [log in to unmask]> wrote:
>>
>>> It is not necessary to do error-prone conversions manually: the ifort
>>> Compiler understands the convert='VAXD' Option in its OPEN statement - see
>>>
>>> https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier
>>>
>>> Thus one could just write a tiny read-write loop.
>>>
>>> HTH
>>> Kay
>>>
>>> On Wed, 14 Nov 2018 00:51:02 +0000, Zhijie Li <[log in to unmask]>
>>> wrote:
>>>
>>> >It's also said here, at the end of file :
>>> >
>>> >https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf
>>> >
>>> >"add 1 to the left, with the binary point"
>>> >
>>> >0.10000.....
>>> >
>>> >
>>> >
>>> >________________________________
>>> >From: CCP4 bulletin board <[log in to unmask]> on behalf of Zhijie
>>> Li <[log in to unmask]>
>>> >Sent: Tuesday, November 13, 2018 7:43 PM
>>> >To: [log in to unmask]
>>> >Subject: Re: [ccp4bb] VERY old mtz file..
>>> >
>>> >
>>> >Hi all,
>>> >
>>> >
>>> >I think I know why it is a division of 4 instead of 2 is involved in
>>> conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits
>>> (bias of 128 instead of 127, visible), another 2 is hidden in the
>>> scientific notation.
>>> >
>>> >
>>> >I found this explanation+example on VAX F-float:
>>> >
>>> >http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html
>>> >
>>> >
>>> >So for IEEE754 float32, if we want to represent a same 12.75 (1100.11)
>>> in the above example, we would first conceptually write it in scientific
>>> notation as 1.10011 x 1000 in binary. Then the mantissa part is the part
>>> after the dot filled with zero to 23 bits: '10011000000000000000000', the
>>> exponent part is 3+127=130 (dec)=10000010(bin). Then the binary IEEE754
>>> float32 number is 0[10000010][10011000000000000000000]. (You can check it
>>> here: https://www.h-schmidt.net/FloatConverter/IEEE754.html)
>>> >
>>> >
>>> >Now compare this with the VAX 12.75 in the linked example, we can find
>>> that besides the bias becoming 128, the conceptual binary scientific
>>> notation is actually 0.110011 x 10000, instead of 1.10011 x 1000. So the
>>> exponent needed is 4 instead of 3. Then the exponent bits are
>>> 4+128=132=10000100 and the VAX float32 becomes
>>> >
>>> >0[10000100][10011000000000000000000] ---if we write in a IEEE-style
>>> order. Note that the mantissa appears to be same as the ieee mantissa, and
>>> the exponent to be applied is 132-128=4. If this number is interpreted as
>>> IEEE754, then it will be 1.10011 x 2exp(132-127)=1.10011 x 100000, four
>>> times of what it should be.
>>> >
>>> >
>>> >So, for normalised values, rearranging the VAX F-float bytes, reading as
>>> IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment in
>>> the ccp4 lib is neat.)
>>> >
>>> >
>>> >In this link describing VAX floats, it is unfortunate that it only
>>> states that the bias for F-float is 128, but not that the mantissa starts
>>> from 0.01 instead of 0.1. Therefore the confusion.
>>> >
>>> >https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm
>>> >
>>> >
>>> >
>>> >Thanks to all responded!
>>> >
>>> >
>>> >Zhijie
>>> >
>>> >
>>> >________________________________
>>> >From: Ian Tickle <[log in to unmask]>
>>> >Sent: Tuesday, November 13, 2018 4:54 PM
>>> >To: Zhijie Li
>>> >Cc: [log in to unmask]
>>> >Subject: Re: [ccp4bb] VERY old mtz file..
>>> >
>>> >
>>> >Hi Zhijie
>>> >
>>> >It's definitely a factor 4. The code is in subroutine QTIEEE in the
>>> Fortran source I mentioned previously at this line:
>>> >
>>> >See line:
>>> >
>>> > A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2
>>> >
>>> >If you prefer it in C code it's in function vaxF2ieeeF in:
>>> >
>>> >ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c
>>> >
>>> >See line:
>>> >
>>> >out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */
>>> >
>>> >i.e. subtract 2 from exponent -> division by 4.
>>> >
>>> >Cheers
>>> >
>>> >-- Ian
>>> >
>>> >
>>> >On Tue, 13 Nov 2018 at 19:52, Zhijie Li <[log in to unmask]<mailto:
>>> [log in to unmask]>> wrote:
>>> >If somebody is going to send these files by email, please send one to me
>>> too. Thanks in advance. I actually prefer to get a MTZ file because the
>>> miller indices would serve as good clues for understanding the encodings.
>>> Even the first 1024 bytes of an MTZ would do (data array starts at byte 80
>>> in MTZ).
>>> >
>>> >In my life I had only seen ieee754. According to what I can find, VAX
>>> has an exponent bias of 128 (ieee754 uses 127). Then it seems to me that
>>> when converting from vax to ieee a division of 2 is involved. However all
>>> procedures I have seen use a division of 4, which is quite puzzling to me.
>>> A real data file containing meaningful numbers (eg., HKL indices) would be
>>> very helpful. Thanks in advance.
>>> >
>>> >Zhijie
>>> >
>>> >> On Nov 13, 2018, at 2:21 PM, Johan Hattne <[log in to unmask]<mailto:
>>> [log in to unmask]>> wrote:
>>> >>
>>> >> Related by not exactly on topic: would anybody on the list be able to
>>> share old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX
>>> reals/strings? I�d be interested to see what those files actually look(ed)
>>> like.
>>> >>
>>> >> // Best wishes; Johan
>>> >>
>>> >>> On Nov 9, 2018, at 18:38, Zhijie Li <[log in to unmask]<mailto:
>>> [log in to unmask]>> wrote:
>>> >>>
>>> >>> Hi all,
>>> >>>
>>> >>> On linux there are a few good GUI HEX editors. Here I�d like to
>>> recommend BLESS, which conveniently displays all possible numerical
>>> interpretations of the four bytes under cursor. It also allows the user to
>>> switch between big endian or little endian through a checkbox.
>>> Unfortunately all floats are assumed to be IEEE754, therefore VAX floats
>>> won�t be interpreted correctly. ( The simplest way to convert vax to ieee
>>> float would be to write a little program to do some bit operations. I�d be
>>> happy to take that as my weekend project)
>>> >>>
>>> >>>
>>> >>> BTW, along the line of space efficiency, I can�t help noticing that
>>> the miller indices are saved as float32 in mtz, as all other numbers in
>>> mtz. This certainly have made mtz format a beautiful homogeneous data
>>> format ;). In this particular case, if we have doubts about the
>>> reliability of the machine stamp, trying to restore the miller indices
>>> would be a good way to test hypotheses.
>>> >>>
>>> >>> Zhijie
>>> >>>
>>> >>>> On Nov 9, 2018, at 9:04 PM, James Holton <
>>> [log in to unmask]<mailto:
>>> [log in to unmask]>> wrote:
>>> >>>>
>>> >>>> As a beamline scientist I must say I am glad that diffraction image
>>> data is not usually stored as ASCII text. In fact, I am slowly warming to
>>> the idea of storing it as not just binary, but compressed formats.
>>> Problem, I'm sure will be that it won't be long before we forget how to
>>> decompress them, as most of the algorithms we are using aren't all that
>>> widespread. Probably around the same time future generations will curse us
>>> for using ASCII instead of unicode, which is a 16-bit standard. I'm sure we
>>> will be reviled for limiting ourselves so, just to save a factor of two in
>>> disk space.
>>> >>>> In situations like this I always use the unix "od" command. It
>>> makes everything "human readable" by converting the bytes into strings you
>>> can read. Then it is just a matter of figuring out what the bytes are.
>>> >>>> Unfortunately, "od" only decodes floats on the native platform, so
>>> if the mtz is from another platform (Windows vs Linux, for example), then
>>> you might need to do some swapping. Thus far, I have encountered files
>>> that require one of a few swapping strategies in order to make them work:
>>> >>>>
>>> >>>> 1 2 3 4 - no swapping
>>> >>>>
>>> >>>> 4 3 2 1 - reverse all bytes
>>> >>>>
>>> >>>> 3 4 1 2 - swap words and swap bytes within the words
>>> >>>> 2 1 4 3 - reverse of previous
>>> >>>>
>>> >>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2
>>> before swapping
>>> >>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2
>>> before swapping
>>> >>>> I'm sure there are other combinations, but the oldest MTZ I have is
>>> only from 1996.
>>> >>>>
>>> >>>> -James Holton
>>> >>>> MAD Scientist
>>> >>>>
>>> >>>>
>>> >>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote:
>>> >>>>> Anyone any idea what to do about this?? Created in 1992!!
>>> >>>>> Seems unreadable..
>>> >>>>>
>>> >>>>> No CTYP lines input for file: 1
>>> >>>>> Indices output even if all data items flagged "missing"
>>> >>>>> Warning, NOT all LABOUT data lines given
>>> >>>>> Warning: Machine stamp corrupted? Assuming native format.
>>> >>>>>>>>>>> CCP4 library signal library_file:End of File (Error)
>>> >>>>>
>>> >>>>>
>>> >>>>> To unsubscribe from the CCP4BB list, click the following link:
>>> >>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>> To unsubscribe from the CCP4BB list, click the following link:
>>> >>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>> >>>>
>>> >>>
>>> >>> To unsubscribe from the CCP4BB list, click the following link:
>>> >>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>> >>>
>>> >>
>>> >> Research Specialist @ Gonen Lab
>>> >> ____________________________________________________
>>> >> UCLA * 615 Charles E. Young Drive South
>>> >> BSRB #347 * Los Angeles, CA 90095
>>> >>
>>> >>
>>> ########################################################################
>>> >>
>>> >> To unsubscribe from the CCP4BB list, click the following link:
>>> >> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>> >
>>> >########################################################################
>>> >
>>> >To unsubscribe from the CCP4BB list, click the following link:
>>> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>> >
>>> >________________________________
>>> >
>>> >To unsubscribe from the CCP4BB list, click the following link:
>>> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>> >
>>> >########################################################################
>>> >
>>> >To unsubscribe from the CCP4BB list, click the following link:
>>> >https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>> >
>>>
>>> ########################################################################
>>>
>>> To unsubscribe from the CCP4BB list, click the following link:
>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>>
>>
>> ------------------------------
>>
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>
>>
>>
>> ------------------------------
>>
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>
>
>########################################################################
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>
########################################################################
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
|