I would like to second Ian's bug report, and suggest one minor
improvement. Rather than multiplying I and sigI by 10, one could find
the largest intensity value I(max) and multiply all the I and sigI
values by (say) 9999.99/I(max) to avoid any possibility of overflowing
the format. An additional refinement, useful for datasets extending to
very high resolution that can have a high dynamic range, would be to
change the format from F8.2 to F8.3 or F8.4 when writing out small
intensities. XPREP and other programs do all this.
To avoid any confusion, the SHELX HKLF4 format (last changed when Axel
introduced the free R in 1993) is:
h k l I sigI nf '(3I4,2F8.2,I4)' (but Fortran can read any F8.X instead
of F8.2)
where the free R flag nf is 1 for the working set and -1 for the free R
set. The file may be terminated either by a reflection with indices 0 0
0 or by the end of file. The data may be in any order, merged or
unmerged, and systematic absences if present are removed as required by
all the SHELX programs. The scale of the data is arbitrary.
George
Ian J. Tickle wrote:
> <[log in to unmask]>
>
> <[log in to unmask]>
> <[log in to unmask]>
> <[log in to unmask]>
> <[log in to unmask]>
> <[log in to unmask]>
> <1173096996.22390.46.camel@localhost>
>From: "Ian Tickle" <[log in to unmask]>
>To: <[log in to unmask]>, <[log in to unmask]>
>Cc: <[log in to unmask]>
>Return-Path: [log in to unmask]
>X-OriginalArrivalTime: 05 Mar 2007 13:15:51.0648 (UTC)
> FILETIME=[65E97E00:01C75F28]
>
>
>Martyn, sorry yes you're quite right, I should have submitted a proper bug report, consider this to be it.
>
>What's needed is a way to read an MTZ file with h, k, l, I, SIGI, FREE and to write same in _strict_ Shel-X format:
>
>- There must be NO header info written to the file (i.e. first line is first refln), and NO additional text (e.g. 'FREE') on the lines (same applies to all options that write Shel-X format output).
>
>- I would suggest multiplying I & SIGI by 10 (or user-supplied scale factor) and writing out as nearest integers; this will reduce the chance of overflowing the format (leaving out the dec. pt. gives you one more column to play with which might make all the difference!).
>
>All other problems with mtz2various I think lie with the documentation:
>
>- The program uses labels I & SIGI not IP & SIGIP as stated (standard output also needs to be changed).
>
>- The program states that if I SIGI is input then the same is output; this is what is desired but not what currently happens (F's are written). It's possible of course that the option to write F's is still needed, in which case there needs to be a way of specifying this.
>
>- The user should be very strongly discouraged from re-squaring the F's from Truncate (possibly even by removing the FSQUARED option completely: but at present this is the only way of getting the desired output).
>
>- The bit about having the FREE text there to allow extraction of test set reflns, should be changed to "use 'grep -e -1$ file' to extract ...".
>
>- Possibly additional suggestions from others.
>
>Cheers
>
>-- Ian
>
>
>>-----Original Message-----
>>From: CCP4 bulletin board [mailto:[log in to unmask]] On
>>Behalf Of Martyn Winn
>>Sent: 05 March 2007 12:17
>>To: [log in to unmask]
>>Subject: Re: [ccp4bb] MTZ to Shel-X?
>>
>>A glance through the CVS history of mtz2various and f2mtz shows that
>>there has been a lot of work keeping these up-to-date for various
>>formats, work that is largely unrewarded and unacknowledged. But they
>>are indeed still deficient in places.
>>
>>The required code writing is relatively trivial. The hard-bit
>>is sorting
>>through the available snippets of information, separating fact from
>>fiction, dealing with different versions of software (backwards
>>compatibility), dealing with different scenarios (I present or not,
>>anomalous present or not, Rfree present or not, etc), and dealing with
>>certain working practices (see e.g. Eleanor's email). When I get
>>something working for myself, I'm usually less than halfway
>>to something
>>that is robust enough for general use.
>>
>>The way forward is to submit proper bug reports to
>>[log in to unmask] saying
>>I want to convert from/to version X of software Y. I expect
>>this script
>>to use/produce this reflection file. If column Z is missing, then this
>>happens. This header is optional, this one isn't. Etc, etc.
>>
>>I suggest this as a practical realistic way forward. Of
>>course, one can
>>use ccp4bb to debate what would happen in an ideal world. We (the ones
>>that actually have the power to do so) don't throw people off the BB.
>>
>>Of course, patching mtz2various and f2mtz is not the "best"
>>thing to do.
>>"Someone" should write a modern import/export utility. Yes, we should
>>discuss this at the March meeting.
>>
>>Regards
>>Martyn
>>
>>On Mon, 2007-03-05 at 10:53 +0000, Kevin Cowtan wrote:
>>
>>>You are absolutely right! The difficulty in getting from MTZ to any
>>>other format or back is unacceptable. Expecting working
>>>crystallographers to write Fortran format statements is
>>>
>>ridiculous. I've
>>
>>>been trying to address this by adding support for other formats to
>>>clipper, but my pace has been glacial, owing to the limited
>>>
>>academic
>>
>>>rewards for such work. Even then, it needs someone to put
>>>
>>together an
>>
>>>application and GUI to use it with.
>>>
>>>I will try and remember to raise this issue at the CCP4 developers
>>>meeting this year. If you prod me in April and I'll report
>>>
>>back to the BB.
>>
>>>Kevin
>>>
>>>George M. Sheldrick wrote:
>>>
>>>>Dear Ian,
>>>>
>>>>It is all part of a diabolical CCP4 plot to make it as
>>>>
>>inconvenient as
>>
>>>>possible to move from a REFMAC refinement to SHELXL! I
>>>>
>>hope that i do
>>
>>>>not get excommunicated like DVD for this comment.
>>>>
>>>>To summarize your and Martin's suggestions, I know of
>>>>
>>only two ways to
>>
>>>>move from mtz to hkl:
>>>>
>>>>
>>>>
>>=====================================================================
>>
>>>>1] run mtz2various with the keywords:
>>>>
>>>>LABIN FP=FP SIGFP=SIGFP FREE=FreeR_flag
>>>>OUTPUT SHELX
>>>>FSQUAR
>>>>
>>>>then edit the resulting .hkl file with any text editor to
>>>>
>>remove the
>>
>>>>header (which SHELXL cannot read) and all occurences of
>>>>
>>the word 'FREE'.
>>
>>>>I have from time to time suggested that mtz2various be
>>>>
>>changed so that
>>
>>>>it no longer outputs the extra junk, but it is still
>>>>
>>there (at least in
>>
>>>>CCP4-6.0.2).
>>>>
>>>>This produces an 'inferior' SHELXL HKLF4 format file because the
>>>>intensities and their standard deviations have been
>>>>
>>converted to F and
>>
>>>>back to I, which requires the assumption of an intensity
>>>>
>>distribution
>>
>>>>function that may not be completely valid (e.g. if NCS is
>>>>
>>present) and
>>
>>>>so produces inferior sigma(I) values. In practice one can
>>>>
>>live with
>>
>>>>this, but it is not very scientific.
>>>>
>>>>
>>>>
>>=====================================================================
>>
>>>>2] Use Tim Gruene's mtz2sca to convert from .mtz to .sca.
>>>>
>>>>mtz2sca name
>>>>
>>>>reads name.mtz and writes name.sca. This program is
>>>>
>>avaliable from Tim
>>
>>>>or from the SHELX download area (Linux only). This has
>>>>
>>the advantage
>>
>>>>that it uses intensities if it can find them in the .mtz
>>>>
>>(e.g. if they
>>
>>>>are still there from SCALA) and otherwise uses F's.
>>>>
>>mtz2sca is fine if
>>
>>>>you want to do exprimental phasing (e.g. read the data
>>>>
>>into SHELXC
>>
>>>>either directly or via Thomas Schneider's hkl2map) but
>>>>
>>unfortunately
>>
>>>>.sca format does not know about Free-R flags. So, as
>>>>
>>Martin suggested,
>>
>>>>one can read the .sca file into xprep (start xprep
>>>>
>>without a file name
>>
>>>>and give the full name of the .sca file when prompted)
>>>>
>>and then use the
>>
>>>>xprep option to transfer the Free-R flags from the
>>>>
>>'inferior' .hkl file
>>
>>>>from mtz2various and write it out in SHELXL HKLF4 format.
>>>>
>>This produces
>>
>>>>a slightly better file for refinement when the .mtz file contains
>>>>intensities, but requires xprep.
>>>>
>>>>
>>>>
>>====================================================================
>>
>>>>George
>>>>
>>>>
>>>>
>>>>Martin Hallberg wrote:
>>>>
>>>>
>>>>>I would probably prioritize keeping the same R-free set
>>>>>
>>(thus using
>>
>>>>>the F^2 output by mtz2various) over
>>>>>going through the scalepack format and loosing track of it.
>>>>> You can however use XPREP to transfer the R-free set
>>>>>
>>from the
>>
>>>>>"inferior" F^2 HKLF4 file output by mtz2various to the proper
>>>>>intensities read from the scalepack file generated by
>>>>>
>>mtz2sca. Then
>>
>>>>>you can write out an HKLF4 file for refinement in
>>>>>
>>SHELXL-97. That
>>
>>>>>HKLF4 file will have proper intensities and the same
>>>>>
>>Rfree set that
>>
>>>>>you have used in Refmac5 (or perhaps Restrain?). But
>>>>>
>>then one would
>>
>>>>>wish that mtz2various did this from the beginning...
>>>>>
>>>>>Best regards,
>>>>>
>>>>>Martin
>>>>>
>>>>>On Mar 1, 2007, at 7:58 PM, Ethan Merritt wrote:
>>>>>
>>>>>
>>>>>>On Thursday 01 March 2007 10:17, Ian Tickle wrote:
>>>>>>
>>>>>>
>>>>>>>All, I thought this would be a simple task, but for
>>>>>>>
>>the life of me I
>>
>>>>>>>can't see how to do it! All I want to do is convert
>>>>>>>
>>an MTZ file to
>>
>>>>>>>Shel-X format for refinement. I thought it would take
>>>>>>>
>>me 2 secs, but
>>
>>>>>>>it's taken me at least 5 attempts, and it's still not right!
>>>>>>>
>>>>>>Do the conversion on the shelx side, rather than the CCP4 side.
>>>>>>
>>>>>>http://shelx.uni-ac.gwdg.de/~tg/mtz2sca/mtz2sca.html
>>>>>>
>>>>>>
>>>>>.
>>>>> B. Martin Hallberg, PhD
>>>>> Molecular Cell Biology Program
>>>>> Department of Cell and Molecular Biology
>>>>> Karolinska Institute
>>>>> Nobels väg 3
>>>>> SE-171 77 Stockholm
>>>>>
>>>>
>>>>
>>>>
>>
>
>Disclaimer
>
>This communication is confidential and may contain privileged information intended solely for the named addressee(s). It may not be used or disclosed except for the purpose for which it has been sent. If you are not the intended recipient you must not review, use, disclose, copy, distribute or take any action in reliance upon it. If you have received this communication in error, please notify Astex Therapeutics Ltd by emailing [log in to unmask] and destroy all copies of the message and any attached documents.
>
>
>
>Astex Therapeutics Ltd monitors, controls and protects all its messaging traffic in compliance with its corporate email policy. The Company accepts no liability or responsibility for any onward transmission or use of emails and attachments having left the Astex Therapeutics domain. Unless expressly stated, opinions in this message are those of the individual sender and not of Astex Therapeutics Ltd. The recipient should check this email and any attachments for the presence of computer viruses. Astex Therapeutics Ltd accepts no liability for damage caused by any virus transmitted by this email. E-mail is susceptible to data corruption, interception, unauthorized amendment, and tampering, Astex Therapeutics Ltd only send and receive e-mails on the basis that the Company is not liable for any such alteration or any consequences thereof.
>
|
