Dear Leo,
Thank you for your message. I would hurry to say that there is no
notion of blame in this whole situation - if there were any, then I
would blame myself for being perhaps too abrupt and severe-sounding in
my message of yesterday :-) .
Returning to the main issue, it would certainly be our preferred
option if any anomalies spotted in the results of STARANISO, or in the
behaviour of programs using these results, could be reported to us
first. It isn't a matter of wanting to avoid open discussions and
scrutiny, but this is a program under very active development and most
problems will be bugs, so that reporting them publicly would flood
CCP4BBers' mailboxes with spam and the BB itself with a lot of
ephemeral noise. We have a specific e-mail address for such reports,
given on the server.
Being in charge of a top-notch beamline associated with beamline
scientists like Pierre who have a keen interest in data analysis, you
are in a perfect position to help us improve the program. As you have
noted, it not only does some number crunching, but it offers a 3D
interactive overview of the quality of datasets that is quite novel
and is meant to contribute (we are not the only ones!) to encouraging
users to think in terms of quality again and not just of getting a few
numbers that won't raise reviewers' eyebrows when placed into some
sensitive cells of Table 1.
Let us look forward to some fruitful exchanges of data and
results, then!
With best wishes,
Gerard.
--
On Fri, Nov 24, 2017 at 12:07:26AM +0000, CHAVAS Leonard wrote:
> Dear Gerard
>
> I should certainly be the one to blame, as I was the one spotting this behaviour to Stefan. Telling this, it could indeed be a problem with the data set, yet submitting the original mtz to ContaMiner provided with the strangely hoped and expected ContaMiner negative result. I made few tests afterwards, quick ones, with other mtz files treated or not with Staraniso, and although the direct results were not exactly the same as the extreme 'all positive' results noted earlier, it behaved strange (all 'possible positive', while standard non-contaminant data give you all 'negative' results).
>
> I then ran MorDa standalone and found the same behaviour: for the first trial, it did found a solution... while there were none. Interestingly, Rfac/Rfree in the resulting MorDa solved structure was something like 50/20... I strongly believe that MorDa, and hence ContaMiner, scores its results on the Rfree value. Errors could have been spotted by looking at the ratio, which clearly was showing that something wrong is happening, operation either on MorDa or on ContaMiner side (or even both).
>
> Additional investigations are obviously welcome, however and for now, I guess the message from Stefan was more in the idea to warn people that having too many positive hits for one single data probably means something is wrong with the ContaMiner/MorDa processing, more than something is wrong with the dataset itself. That goes without saying that something could go wrong if the dataset is wrong, obviously...
>
> To Gerard and good willing developers: would you need the badly behaving data, please do let me know and I shall send it to you offline.
>
> Cheers, leo
>
> -
> Leonard Chavas
> -
> Synchrotron SOLEIL
> Proxima-I
> L'Orme des Merisiers
> Saint-Aubin - BP 48
> 91192 Gif-sur-Yvette Cedex
> France
> -
> Phone: +33 169 359 746
> Mobile: +33 644 321 614
> E-mail: [log in to unmask]
> -
>
> > On 23 Nov 2017, at 23:53, Gerard Bricogne <[log in to unmask]> wrote:
> >
> > Dear Pierre,
> >
> > Thank you for throwing some light on the circumstances under
> > which the "suspicious" results cropped up :-) . Was the "Government
> > Heatlh Warning" based on this one and only mtz file, then?
> >
> > We would certainly be interested in examining this dataset for
> > any anomalies in the anisotropy analysis and the mtz file it produced
> > (perhaps you can simply give us the job ID on the server). However the
> > "results" consist of the spurious recognition of molecules that are
> > known not to be in the crystal, so they are the outcome of numerous
> > unspecified steps downstream of the anisotropy analysis itself, that
> > in the end produce a "score" that is misleading. It would be useful to
> > have at least some idea of what those steps are in order to identify
> > the possible causes of the erroneous detections.
> >
> >
> > With best wishes,
> >
> > Gerard.
> >
> > --
> > On Thu, Nov 23, 2017 at 10:05:09PM +0000, LEGRAND Pierre wrote:
> >> Dear Gérard,
> >>
> >> As being part of the group how has initially raised the issue to Stefan, I stand out to try clarifying a misinterpretation.
> >> In brief, because we are happy users of StarAniso, it happened that we have submitted an mtz that it had produced to the ContaMiner server. We were very surprised to find that almost all contaminants evaluated gave high scores according to MoRDa. On the contrary, using an "isotropicaly" treated mtz, no hit could be detected in by ContaMiner.
> >>
> >> Being collected on PROXIMA-1 beamline, it coun't be a "badly collected datasets" ;-)
> >>
> >> So, most probably, as you suggested, the issue is linked to some bad assumptions made by MoRDa or inadequate criteria choices _in_the_cases_ of "corrected" anisotropic data.
> >> We can probably provide some examples of datasets to the developers willing to pursue investigations on this.
> >>
> >> I completely agree with Tristan! I have to admit having lost several weeks in my career (if not months) with "contaminated" crystals. And working on an MX beamline, I can testify that this is unfortunately happening regularly.
> >>
> >> I will finish with a big thanks to all the (StarAniso/ContaMiner/MoRDA) developers how are helping us to untwist the unavoidable experimental mess/reality.
> >>
> >> Kind regards,
> >> Pierre
> >> ________________________________________
> >> De : CCP4 bulletin board [[log in to unmask]] de la part de Gerard Bricogne [[log in to unmask]]
> >> Envoyé : jeudi 23 novembre 2017 19:34
> >> À : [log in to unmask]
> >> Objet : Re: [ccp4bb] new ContaMiner features
> >>
> >> Dear Stefan,
> >>
> >> Regarding your final paragraph: your server carries a warning
> >> with the exact wording:
> >>
> >> "Submitting StarAniso files can give you suspicious results. Use
> >> with care!"
> >>
> >> It seems rather regrettable that you are posting such a public
> >> warning without ever having contacted the STARANISO developers about
> >> your observations, nor giving any information about what you call
> >> "suspicious" or what the "care" you recommend would consist of.
> >>
> >> We have taken a great deal of care ourselves in developing the
> >> program and offering it to the community through a server, and the
> >> least we would have expected is that any pattern of "suspicious"
> >> results would be referred to us so that we could investigate them.
> >> There may be some assumptions made in MoRDa that we are not aware of,
> >> that might be incompatible with assumptions made in STARANISO - who
> >> knows? Or it could be that some particularly badly collected datasets
> >> are made to look worse after their anisotropy analysis.
> >>
> >> Could we discuss your observations, and what it is exactly that
> >> you call "suspicious", before they end up being referred to in such an
> >> uninformative manner as some sort of "Government Health Warning"?
> >>
> >> I think that would be nice :-) and we would be only too keen to
> >> take whatever extra "care" is needed ourselves. We would all learn
> >> something.
> >>
> >>
> >> With best wishes,
> >>
> >> Gerard.
> >>
> >> (on behalf of the STARANISO developers)
> >>
> >> --
> >> On Thu, Nov 23, 2017 at 05:02:39PM +0100, Stefan Arold wrote:
> >>> Dear Community,
> >>>
> >>> A quick message to announce the following two new features on our
> >>> ContaMiner web server for the automated detection of unwantedly
> >>> crystallised contaminants (
> >>> https://strube.cbrc.kaust.edu.sa/contaminer/submit)
> >>>
> >>> 1) online visualisation of 2FoFc and FoFc maps. In cases of positive
> >>> results, the ‘UglyMol’ tab allows to inspect 2FoFc and FoFc maps directly
> >>> in the web browser. Thi
> >>>
> >>> 2) life-update. Previously, results were sent to you once all ~2000 MR jobs
> >>> were finished. Now, the individual results for each potential contaminant
> >>> will appear as soon as they are finished. This feature should substantially
> >>> shorten the time for identifying positive results (i.e. contaminant
> >>> detected), which are terminated faster than negative ones.
> >>>
> >>> 3) custom contaminants. In the ‘Advanced’ tab, users can upload own PDB
> >>> files (more than one is possible) to be included as search models. This
> >>> feature can be used to include PDB files from your lab bench neighbour’s
> >>> project to test for potential lab internal contaminations (through
> >>> bacterial contamination or through mix-up of plasmids or glycerol stocks).
> >>> This feature could also be ‘abused’ as a means to use the MoRDa pipeline to
> >>> run molecular replacements with template structures that are not yet
> >>> deposited in the PDB; for example to run molecular replacement and initial
> >>> refinement for liganded or complexed versions of an unpublished structure.
> >>> This might be particularly interesting for crystallographers away from
> >>> their usual home software environment (e.g. at the beamline).
> >>>
> >>> Finally, a word of warning – Staraniso files might give false positives if
> >>> they have large anisotropic cuts.
> >>>
> >>> Keep your crystals clean!
> >>>
> >>> With best wishes
> >>>
> >>> The ContaMiner Team
