I always run a round of scan-static refinement before scan-varying refinement. Complexity does not come in the number of operations but in not quite getting the boundaries right between which operations should be "atomic".
In my opinion, of course.
From: DIALS project general discussion [[log in to unmask]] on behalf of Richard Gildea [[log in to unmask]]
Sent: 01 October 2015 15:13
To: [log in to unmask]
Subject: Re: Reindex
The problem with your suggestion is that although it would reduce the number of files output by DIALS it would add an extra step to the DIALS processing, as you would have to use dials.reindex to reindex both the experiment.json and indexed.pickle, and also assign the space group, then a round of dials.refine scan-static refinement, before eventually running scan-varying refinement. This would make the process more complex, not less.
From: Graeme Winter [[log in to unmask]]
Sent: 01 October 2015 15:02
To: Gildea, Richard (DLSLtd,RAL,LSCI); [log in to unmask]
Subject: Re: Reindex
Actually David that would help with the clutter o files problem that users sometimes complain about. Should be easy enough to fix up in xia2 anyhow - though the bravais_setting n files are re-refined with the lattice constraints applied which could be "a good thing"
On Thu, Oct 1, 2015 at 3:01 PM Graeme Winter <[log in to unmask]<mailto:[log in to unmask]>> wrote:
I thought the sanity check would be useful; also for the odd corner case where reindexing could pick up wrong reflections somehow (had conversation with NKS about this) so deriving a reindex op and then applying this could be more robust than just straight repeat indexing of the list
I also like sanity checks, I certainly need them from time to time :o)
I thought it sounded like something we did back in the day...
@Phil no intention to stick with original indices; more to derive the right reindexing operation by magic!
On Thu, Oct 1, 2015 at 2:58 PM <[log in to unmask]<mailto:[log in to unmask]>> wrote:
In fact if you always re-index the reflection list before refinement (or before integration since you can go straight from dials.refine_bravais_settings to dials.integrate) using the input experiments.json then no need to compare with original indices even (other than perhaps a sanity check). There is a function in indexing module to do assign indices given a reflection list and an experiment list, although might need modifying to cope with scan-varying crystal models if is to be used by dials.refine and dials.integrate.
From: Graeme Winter [[log in to unmask]<mailto:[log in to unmask]>]
Sent: 01 October 2015 14:46
To: Gildea, Richard (DLSLtd,RAL,LSCI); [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: Reindex
Discussed this with David yesterday in train; I think we can discover the reindex operation from inspection of the data - should be simple enough yes by essentially "reindexing" with the matrix file from expts and comparing with the reference indices
Think this was what Phil was getting at...
On Thu, Oct 1, 2015 at 2:43 PM Richard Gildea <[log in to unmask]<mailto:[log in to unmask]><mailto:[log in to unmask]<mailto:[log in to unmask]>>> wrote:
Thanks for the suggestions. I've never been particularly happy with the way the dials.reindex step fits in the process, but haven't been able to come up with a suitable alternative (other than to go back and re-run dials.index in the correct space group). Your suggestion of tracking the change of basis operator alongside the experiments.json and reflections does seem to have merit, although we would need to discuss internally exactly how to go about it.
Another alternative could be to output the bravais_setting_NN.json files in the same basis as the input, i.e. primitive setting, then they would be compatible with the original indexed.pickle without reindexing the reflection list. Then downstream programs like dials.refine could choose to optionally convert to the reference setting on input. Any thoughts on this workflow? When we eventually get around to the one-hdf5-file-to-rule-them-all version of DIALS that James is keen on, then hopefully these kinds of problems will go away.
From: DIALS project general discussion [[log in to unmask]<mailto:[log in to unmask]><mailto:[log in to unmask]<mailto:[log in to unmask]>>] on behalf of Phil Evans [[log in to unmask]<mailto:[log in to unmask]><mailto:[log in to unmask]<mailto:[log in to unmask]>>]
Sent: 01 October 2015 10:50
To: [log in to unmask]<mailto:[log in to unmask]><mailto:[log in to unmask]<mailto:[log in to unmask]>>
Following up on the Dials demo yesterday
The process is
1) dials.index -> indexed.pickle
2) dials.refine_bravais_settings -> bravais_setting_NN.json
choose a setting, then if necessary reindex
3) dials.reindex indexed.pickle change_of_basis_op=a,b,c -> reindexed_reflections.pickle
followed by something like
4) dials.refine bravais_setting_9.json indexed.pickle scan_varying=true [ or reindexed_reflections.pickle]
however it seems to me that step 3 (a complication for pipelines, and a hazard for manual working) could be incorporated into step 4. You know that indexed.pickle is in the “original” frame (and could usefully be internally stamped as such), and bravais_setting_NN.json does or should include the change of basis operator, so it could be applied automatically. This would be safer
Just a suggestion
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom