On Monday 01 October 2007 08:49, Peter Adrian Meyer wrote:
> This raises a slightly tangential question though - how do we know how
> what obs/param ratio is good enough?  My understanding was that obs/param
> of 1 was sufficient for linear systems

This is wrong, unless by "sufficient" you only mean "some solution is possible".
More observations are always better.  
A higher obs/param ratio is always better.
Linear/non-linear doesn't really have anything to do with it.

> With my level of math background (aka low), I'm not even sure if I'm
> asking the right question...it seems like it might make more sense to ask
> how many observations are needed to define a unique optima for the
> refinement function(s) that's convex in all dimensions of the model.

We don't want a "unique" solution, we want a robust solution.
That is, we want some assurance that changing the value of any single
observation, or small set of observations, will not significantly
change the solution.  That is exactly what you *don't* get if the
obs/param ratio is 1, because every single observation contributes
critically to the solution.  The more observations you have, the
less your solution is critically dependent on any single one of them.

> if this made sense, that's probably what we'd be talking about.

Sorry, it doesn't make sense.


> 
> Any suggestions for reading material?
> 
> Thanks,
> 
> Pete
> 
> 
> > The question is how significant is this bias, and is the cure (i.e.
> leaving out more reflections from the working set) worse than the
> disease?
> >  For refinements at 'medium' typical resolution (around 2.5 to 2 Ang) we
> > are working with an observation/parameter count ratio of say < 3
> (naturally I'm counting the geometric restraints with the X-ray
> > observations).  The amount of bias in Rwork and other statistics derived
> from the working set depends critically on how close the obs/param ratio
> is to 1.  The Rfree optimisation is used only to determine weighting
> parameters (including sigma-A) and it's unlikely there will be more than
> say 20 of these.  Typically there are at least 1000 refls in the test set,
> > so for the Rfree optimisation the obs/param ratio will be around 50.
> This
> > is much larger than the obs/param ratio for Rwork and may well mean that
> the biasing effect on Rfree is negligible.  It should be easy to do some
> tests comparing Rfree with Rsleep to check the bias (taking into account
> errors to limited sample sizes of course), and also to see what are the
> effects of leaving out the sleeping set on the refinement and the maps.  I
> > don't think it would be wise to rush into this until we have done proper
> evaluations.
> >
> > -- Ian
> >
> >> -----Original Message-----
> >> From: [log in to unmask]
> >> [mailto:[log in to unmask]] On Behalf Of Mark J. van Raaij
> Sent: 01 October 2007 14:58
> >> To: [log in to unmask]
> >> Subject: R-sleep
> >> Dear All,
> >> the short paper by Gerard Kleywegt (ActaD 63, 939-940) treats
> >> an interesting subject (at least I think so...). I agree that
> >> what we are now doing in many cases is effectively refining
> >> against Rfree. For example, the standard CNS torsion angle
> >> refinement does n refinement trials with randomised starting
> >> points. If you then take the one with lowest Rfree (or let a
> >> script do this for you), you are biasing Rfree!
> >> Therefore, his proposal to put an extra set of reflections in
> >> a dormant "vault" (R-sleep) sounds like a good idea to me.
> >> However, how would the "vault" be implemented to be
> >> effective? If left to the experimenter, it would be very
> >> tempting to check R-sleep once in a while (or often) during
> >> refinement, rendering it useless as an unbiased validator.
> >> or am I being paranoid and too pessimistic?
> >> Mark J. van Raaij
> >> Unidad de Bioquímica Estructural
> >> Dpto de Bioquímica, Facultad de Farmacia
> >> and
> >> Unidad de Rayos X, Edificio CACTUS
> >> Universidad de Santiago
> >> 15782 Santiago de Compostela
> >> Spain
> >> http://web.usc.es/~vanraaij/
> >
> >
> > Disclaimer
> > This communication is confidential and may contain privileged
> information
> > intended solely for the named addressee(s). It may not be used or
> disclosed except for the purpose for which it has been sent. If you are
> not the intended recipient you must not review, use, disclose, copy,
> distribute or take any action in reliance upon it. If you have received
> this communication in error, please notify Astex Therapeutics Ltd by
> emailing [log in to unmask] and destroy all copies of the
> message and any attached documents.
> > Astex Therapeutics Ltd monitors, controls and protects all its messaging
> traffic in compliance with its corporate email policy. The Company accepts
> > no liability or responsibility for any onward transmission or use of
> emails and attachments having left the Astex Therapeutics domain.  Unless
> > expressly stated, opinions in this message are those of the individual
> sender and not of Astex Therapeutics Ltd. The recipient should check this
> > email and any attachments for the presence of computer viruses. Astex
> Therapeutics Ltd accepts no liability for damage caused by any virus
> transmitted by this email. E-mail is susceptible to data corruption,
> interception, unauthorized amendment, and tampering, Astex Therapeutics
> Ltd only send and receive e-mails on the basis that the Company is not
> liable for any such alteration or any consequences thereof.
> > Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science
> Park, Cambridge CB4 0QA under number 3751674
> >
> 
> 
> Pete Meyer
> Fu Lab
> BMCB grad student
> Cornell University
> 

-- 
Ethan A Merritt            Courier Deliveries: 1959 NE Pacific
Dept of Biochemistry
Health Sciences Building
University of Washington - Seattle WA 98195-7742