Hi Gunnar,
>> I would have thought that changing the value and gradient of the
>> target function had the potential to alter the minima?
>
> Indeed, the target function is changed during the search, but
> once a stable minimum is found, the DEN potential needs to
> be zero by definition and the coordinates have to sit in a minimum
> of the original target function.
Yes, I believe both statements are correct - I was referring to the nature of the function during the procedure, and you refer to the nature after asymptotic convergence. The nature of the target function and location of the minima are changed during DEN refinement - at any given time step (before asymptotic convergence) the minima of the target function may lie in locations different to the original target function. However, upon convergence, dij ~= Dij regardless of particular parameter values. With DEN, the target function changes shape throughout the procedure, but results in the target function being asymptotically equal to the original, thus refinement converges to a minima of the original target function.
This behaviour is interesting and notable because it is different to other terms used in refinement. Generally, prior information (e.g. geometry terms, external structural information) in the form of restraints is determined externally and remains static during refinement. These determine the nature of the target function, but do not change it during refinement - this contrasts with the DEN approach. Of course, the structure factors are also updated and thus also alter the nature of the target function during refinement. Just some interesting observations!
>> Here are my thoughts: since the DEN update formula is recursive, the
>> equilibrium distance can also be written in > terms of the Dij alone (still assuming gamma=1):
>> dij(t+1) = Dij(0)*(1-kappa)^(t+1) + kappa*sum_n=0^t{Dij(t+1-n)*(1-kappa)^n}
>> This means that the equilibrium distance is indeed dependent on the initial
>> distance Dij(0) for all times t. …
>
> I hope I do not get you wrong, but with this argument
> aren't you just saying that the path/trajectory (of both the atomic coordinates
> and the DEN potential) depends on the starting point?
No, this wasn't quite the point I was trying to make. For sure, we all trivially know that any path during refinement depends on the starting point of the parameter values.
However, it is interesting that the DEN restraint target (or DEN potential, or equilibrium distance) depends on the starting point. Moreover, the DEN restraint target can be expressed in a form that makes this dependency explicit. This observation is not trivial, as it differs from other approaches. As above, it is most interesting to acknowledge that this contrasts with other terms used in refinement. For example, simple geometry/external restraints representing prior information always remain static during refinement. At time t, other restraints do not depend on their value at time t-1, and thus do not depend on their value at time 0. Rather, they are pre-determined before refinement begins.
> The important point is that the decision on how to move the DEN
> minimum from one iteration (at time t) to the next (at time t+1)
> depends only on where the atoms are at t+1 and where the DEN minimum was
> at time t.
Or equivalently, the decision on how to move the DEN minimum from one iteration (at time t) to the next (at time t+1) depends on where the atoms are at t+1, where they were at time t, where they were at time t-1, … , where they were at time 0. Of course, the degree of dependency on distant history is controlled by kappa. Very low values of kappa will result in DEN "remembering" more distant historical values of the interatomic distance, and thus refinement will take longer to converge. Very high values of kappa will result in DEN being dependent only on the immediate history, and thus will have little effect on refinement.
> If we assume that there is a second starting point which results
> in a minimization path that happens to cross exactly the path from the first
> starting point (same atomic coordinates and same position of DEN minimum)
> at some time t'. Then the new position of the DEN minimum at time t'+1 would be
> exactly at the same position that you get from the first path at time t+1.
Of course, if a second minimisation path happens to cross exactly the first minimisation path, then they would both end up with the same final result. There would be something wrong if they didn't! The property that two paths within some neighbourhood of each other both converge to the same final positions is a simple requirement for refinement robustness. Just to clarify, I certainly did not make any incorrect/unsupported claims that DEN is not robust. I was merely investigating the exact nature of the technique.
Interestingly, note that DEN requires both the atomic coordinates to be at the same position AND the DEN minimum to be at the same position in order for this argument to hold. This is a much stronger condition than the analogous requirement without DEN, which would require only that the atomic coordinates be near the same positions in order to achieve in the same final result.
If two paths from different starting values cross such that their atomic coordinates are within some neighbourhood of each other, then one would hope that the DEN potentials would adjust themselves (without taking too many refinement cycles, which would depend on kappa) so that they too become within some neighbourhood of each other, hopefully resulting in them both converging to the same local minima.
> This shows that the DEN update does not depend on the starting point.
No, I believe that the DEN update does depend on the starting point. However, kappa can be set sufficiently large so that the dependency on the starting point is very low. The higher dependency on more recent historical distances does indeed allow the entire conformational space to be accessible, but there is still a dependency on the starting point, technically speaking. True, upon convergence, this dependency on the starting point should be negligible. Logically, this will result in a reduced rate of convergence during refinement, but hopefully with the implicit benefit of increased robustness from regularisation.
Thanks for the interesting discussion! It is good to dissect these techniques, and hopefully it will be useful for some users out there who want to know exactly what is going on!
Cheers,
Rob
On 2 Sep 2012, at 20:15, Gunnar Schroeder wrote:
> Hi Rob,
>
>>> This also means that the position of the minima of the target function
>>> are not changed by the DEN (gamma=1) restraints.
>
>> I would have thought that changing the value and gradient of the
>> target function had the potential to alter the minima?
>
> Indeed, the target function is changed during the search, but
> once a stable minimum is found, the DEN potential needs to
> be zero by definition and the coordinates have to sit in a minimum
> of the original target function.
>
> First we note that if the DEN potential minimum is at the same position
> as the atomic coordinates, the potential and the first derivative are zero.
>
> Assume the atoms are at a stable minimum of the combined energy function
> (original target function + DEN potential with gamma=1) AND the DEN potential
> minimum is different from the atomic positions. Then the DEN potential
> minimum would move towards the atomic positions, which would change the
> combined energy function and its derivative. The atoms would not be in a
> stable minimum anymore, which contradicts the assumption and proofs that
> the DEN potential is always zero if the atoms are in a stable minimium of the
> combined energy function.
>
>
>> Here are my thoughts: since the DEN update formula is recursive, the
>> equilibrium distance can also be written in > terms of the Dij alone (still assuming gamma=1):
>> dij(t+1) = Dij(0)*(1-kappa)^(t+1) + kappa*sum_n=0^t{Dij(t+1-n)*(1-kappa)^n}
>> This means that the equilibrium distance is indeed dependent on the initial
>> distance Dij(0) for all times t. …
>
> I hope I do not get you wrong, but with this argument
> aren't you just saying that the path/trajectory (of both the atomic coordinates
> and the DEN potential) depends on the starting point?
> Every simulation/minimization depends on the starting point.
> In a steepest descent minimization the step size determines
> how long it takes to move away from the starting point, just like
> the parameter kappa determines how long it takes for the DEN potential
> and the atomic coordinates to move away from the starting
> model. I do not see the difference? Am I missing something here?
>
> The important point is that the decision on how to move the DEN
> minimum from one iteration (at time t) to the next (at time t+1)
> depends only on where the atoms are at t+1 and where the DEN minimum was
> at time t. If we assume that there is a second starting point which results
> in a minimization path that happens to cross exactly the path from the first
> starting point (same atomic coordinates and same position of DEN minimum)
> at some time t'. Then the new position of the DEN minimum at time t'+1 would be
> exactly at the same position that you get from the first path at time t+1. This
> shows that the DEN update does not depend on the starting point.
>
> Cheers,
> Gunnar
>
>
>
> PS: Just for the record, here we only discuss DEN refinement for gamma=1.
>
>
> On Aug 31, 2012, at 11:30 AM, Robert Nicholls wrote:
>
>> Hi Gunnar,
>>
>> I generally agree with your comments. However, I'd like to clarify a couple of points:
>>
>>> For gamma=1 the DEN potential can follow anywhere, the entire conformational
>>> space is accessible and dij(t+1) depends only on Dij(t) and dij(t).
>> ...
>>> But, again, the starting (or reference)
>>> model is completely forgotten and never used after the first iteration.
>>
>>
>> Certainly, the entire conformational space is accessible. However, I'm not so sure about the starting model being completely forgotten and never used after the first iteration. Here are my thoughts: since the DEN update formula is recursive, the equilibrium distance can also be written in terms of the Dij alone (still assuming gamma=1):
>> dij(t+1) = Dij(0)*(1-kappa)^(t+1) + kappa*sum_n=0^t{Dij(t+1-n)*(1-kappa)^n}
>> This means that the equilibrium distance is indeed dependent on the initial distance Dij(0) for all times t. For values of kappa in (0,1), this dependency will diminish with time t, but will always exist. In fact, the equilibrium distance dij(t) is dependent on the whole history of the distance throughout the procedure, i.e. Dij(n) for n=0…t. Of course, the degree of influence of the historical information is controlled by kappa. Values of kappa~=0 would mean that the initial distance has very high weight (equilibrium distance dij(t) = Dij(0) in the limit kappa=0), and kappa~=1 would mean that the most recent distances have very high weight (equilibrium distance dij(t) = Dij(t) in the limit kappa=1, as you have already stated). Intermediate values of kappa will give various non-zero weights to the historical values of Dij.
>>
>>> This also means that the position of the minima of the target function
>>> are not changed by the DEN (gamma=1) restraints.
>>
>>
>> I would have thought that changing the value and gradient of the target function had the potential to alter the minima?
>>
>>> It is therefore usually useful to run a final minimization without
>>> restraints to test whether the refinement reached a stable minimum of the
>>> target function.
>>
>> I agree. In the context of REFMAC5, my current favourite strategy at low resolution is to first use external restraints in order to aid the structure to adopt a more sensible conformation, but then subsequently release the external restraints and replace them with jelly-body restraints towards the final refinement stages.
>>
>>> From the user perspective, I think the main difference is that DEN is designed
>>> to be used in simulated annealing MD refinement, whereas jelly-body is designed
>>> to be used in minimization (and cannot be used for MD refinement as there are
>>> no second derivatives).
>>
>> I agree. Since the second derivative is utilised in ML refinement, it is possible to design a regulariser that has the desirable properties X=0 and X'=0 (e.g. jelly-body refinement) in the absence of any externally-derived prior information. Since this is not possible in simulated annealing MD refinement, the analogous solution will undoubtedly have to alter X and/or X'. Either way, all of these 'tricks' are just designed to aid robustness and combat overfitting! Certainly, both approaches can give positive results when refining at low resolution.
>>
>> Cheers
>> Rob
>>
>
> --
> _____________________________________________________________
> Gunnar F. Schröder
> Computational Structural Biology Group
> Institute of Complex Systems (ICS-6)
> Forschungszentrum Jülich
>
> tel +49-2461-61-3259
> http://www.schroderlab.org
|
