JiscMail Logo
Email discussion lists for the UK Education and Research communities

Help for CCP4BB Archives


CCP4BB Archives

CCP4BB Archives


CCP4BB@JISCMAIL.AC.UK


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

CCP4BB Home

CCP4BB Home

CCP4BB  March 2009

CCP4BB March 2009

Options

Subscribe or Unsubscribe

Subscribe or Unsubscribe

Log In

Log In

Get Password

Get Password

Subject:

Re: images

From:

"Herbert J. Bernstein" <[log in to unmask]>

Reply-To:

Herbert J. Bernstein

Date:

Mon, 16 Mar 2009 07:38:29 -0400

Content-Type:

TEXT/PLAIN

Parts/Attachments:

Parts/Attachments

TEXT/PLAIN (179 lines)

Dear Colleagues,

   The issue Harry is describing, of people writing multiple variations of
"image formats" even though all of them are imgCIF is not really a
problems with the images themselves.  Rather it is a lack of agreement on
the metadata to go with the images.  This is similar to the problem of
lack of consistency in REMARKS for early PDB data sets, which eventually
required the adoption of standardized REMARKS and reprocessing of almost
all data sets.  I don't think it would have been easier to reprocess those
data sets if the original data sets had also had their coordinates and
sequences recorded with wide variations in formats.

   The advantage of using imgCIF for an archive is not that it would force
everybody to to their experiments using precisely the same format, but
that, because it is capable of faithfully representing all the wide
variations in current formats, it would allow what we now have to be
captured and preserved and, when someone needed a dataset back, to be
recast in an format appropriate to the use.

   Think of it as that little figure-8 plug and socket we are able to use
to adapt our power cords for travel around the world.  There are other
possible hub format (NeXus, DICOM, etc.), but the sensisble thing for an
archive is to choose one of them for internal use, just as the PDB uses a
variation on mmCIF for its internal use to allow it to easily deliver
valid PDB, CIF and XML versions of sets of coordinates.  For an archive,
the advantages of using imgCIF internally, no matter which of the more
than 200 current formats were to be used at beam lines and in labs, is
that it would not be necessary to discard any of the metadata people
provided and it could be made to interoperate easily with the systems used
internally by the PDB for coordinate data sets.

   For many of the formats in current use, there is no place to store some
of the information people provide and translation to other formats can
sometimes be much more difficult than one might expect unless additional
metadata is provided.  Even such obvious things as image orientations are
sometimes carried separately from the images themselves and can easily get
lost.

   Don't let the perfect be the enemy of the good.  Archiving images in a
common format, such as imgCIF, or, if you prefer, say, in the NeXus
transliteration of imgCIF, would help to make some very useful information
accessible for future use.  It may not be a perfect solution, but it is a
good one.

   This is a good time to start a major crystallogrpahic image archiving
effort.  Money may well be available now that will not be avialable six
month from now, and we have good, if not perfect, solutions available for
many, if not all, of the technical issues involved.  Is it really wise to
let this opportunity pass us by?

   Regards,
     Herbert
=====================================================
  Herbert J. Bernstein, Professor of Computer Science
    Dowling College, Kramer Science Center, KSC 121
         Idle Hour Blvd, Oakdale, NY, 11769

                  +1-631-244-3035
                  [log in to unmask]
=====================================================

On Mon, 16 Mar 2009, Harry Powell wrote:

> Hi
>
> I'm afraid the adoption of imgCIF (or CBF, its useful binary equivalent)
> doesn't help a lot - I know of three different manufacturers of detectors
> who, between them, write out four different image formats, all of which
> apparently conform to the agreed IUCr imgCIF standard. Each manufacturer has
> its own good and valid reasons for doing this. It's actually less work for me
> as a developer of integration software to write new code to incorporate a new
> format than to make sure I can read all the different imgCIFs properly.
>
>
> On 16 Mar 2009, at 09:32, Eleanor Dodson wrote:
>
>> The deposition of images would be possible providing some consistent
>> imagecif format was agreed.
>> This would of course be of great use to developers for certain pathological
>> cases, but not I suspect much value to the user community - I down load
>> structure factors all the time for test purposes but I probably would not
>> bother to go through the data processing, and unless there were extensive
>> notes associated with each set of images I suspect it would be hard to
>> reproduce sensible results.
>>
>> The research council policy in the UK is that original data is meant to be
>> archived for publicly funded projects. Maybe someone should test the
>> reality of this by asking the PI for the data sets?
>> Eleanor
>>
>>
>> Garib Murshudov wrote:
>>> Dear Gerard and all MX crystallographers
>>>
>>> As I see there are two problems.
>>> 1) Minor problem: Sanity, semantic and other checks for currently
>>> available data. It should not be difficult to do. Things like I/sigma,
>>> some statistical analysis expected vs "observed" statistical behaviour
>>> should sort out many of these problems (Eleanor mentioned some and they
>>> can be used). I do not think that depositors should be blamed for
>>> mistakes. They are doing their best to produce and deposit. There should
>>> be a proper mechanism to reduce the number of mistakes.
>>> You should agree that situation is now much better than few years.
>>>
>>> 2) A fundamental problem: What are observed data? I agree with you
>>> (Gerard) that images are only true observations. All others (intensities,
>>> amplitudes etc) have undergone some processing using some assumptions and
>>> they cannot be considered as true observations. The dataprocessing is
>>> irreversible process. I hope your effort will be supported by community. I
>>> personally get excited with the idea that images may be available. There
>>> are exciting possibilities. For example modular crystals, OD, twin in
>>> general, space group uncertaintly cannot be truly modeled without images
>>> (it does not mean refinement against images). Radiation damage is another
>>> example where after processing and merging information is lost and cannot
>>> be recovered fully. You can extend the list where images would be very
>>> helpful.
>>>
>>> I do not know any reason (apart from technical one - size of files) why
>>> images should not be deposited and archived. I think this problem is very
>>> important.
>>>
>>> regards
>>> Garib
>>>
>>>
>>> On 12 Mar 2009, at 14:03, Gerard Bricogne wrote:
>>>
>>>> Dear Eleanor,
>>>>
>>>>   That is a useful suggestion, but in the case of 3ftt it would not have
>>>> helped: the amplitudes would have looked as healthy as can be (they were
>>>> calculated!), and it was the associated Sigmas that had absurd values,
>>>> being
>>>> in fact phases in degrees. A sanity check on some (recalculated) I/sig(I)
>>>> statistics could have detected that something was fishy.
>>>>
>>>>   Looking forward to the archiving of the REAL data ... i.e. the images.
>>>> Using any other form of "data" is like having to eat out of someone
>>>> else's
>>>> dirty plate!
>>>>
>>>>
>>>>   With best wishes,
>>>>
>>>>        Gerard.
>>>>
>>>> --
>>>> On Thu, Mar 12, 2009 at 09:22:26AM +0000, Eleanor Dodson wrote:
>>>>> It would be possible for the deposition sites to run a few simple tests
>>>>> to
>>>>> at least find cases where intensities are labelled as amplitudes or vice
>>>>> versa - the truncate plots of moments and cumulative intensities at
>>>>> least
>>>>> would show something was wrong.
>>>>>
>>>>> Eleanor
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>>   ===============================================================
>>>>   *                                                             *
>>>>   * Gerard Bricogne                     [log in to unmask]  *
>>>>   *                                                             *
>>>>   * Global Phasing Ltd.                                         *
>>>>   * Sheraton House, Castle Park         Tel: +44-(0)1223-353033 *
>>>>   * Cambridge CB3 0AX, UK               Fax: +44-(0)1223-366889 *
>>>>   *                                                             *
>>>>   ===============================================================
>>>>
>>>
>>>
>
> Harry
> --
> Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills Road,
> Cambridge, CB2 0QH

Top of Message | Previous Page | Permalink

JiscMail Tools


RSS Feeds and Sharing


Advanced Options


Archives

March 2024
February 2024
January 2024
December 2023
November 2023
October 2023
September 2023
August 2023
July 2023
June 2023
May 2023
April 2023
March 2023
February 2023
January 2023
December 2022
November 2022
October 2022
September 2022
August 2022
July 2022
June 2022
May 2022
April 2022
March 2022
February 2022
January 2022
December 2021
November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007


JiscMail is a Jisc service.

View our service policies at https://www.jiscmail.ac.uk/policyandsecurity/ and Jisc's privacy policy at https://www.jisc.ac.uk/website/privacy-notice

For help and support help@jisc.ac.uk

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager