Hi,
I am bit confused about the purpose of the Q-score validation.
Is it a metric of resolvability or a metric of model fitting to a map?
My understanding is the former. The original paper (Pintilie, G., et
al., 2020, Nat. Methods) repeatedly says "resolvability".
This announcement also says "the first quantitative parameter of residue
and chain resolvability", suggesting the former. However,
the CIF category names contain phrases like "fit_to_map" and
"map_fitting", suggesting the latter.
These two are not the same because:
1. Q-score uses a single Gaussian as a reference for all elements but
it is not the right form to represent Coulomb potential of an atom
2. Local maxima in a map do not coincide with atomic coordinates
(although the error is small except for hydrogens)
3. Q score is sensitive to flexibilities (positional variations) of an
atom
If the purpose is to test resolvability, this is probably fine
(and point 3 is actually beneficial). If the purpose is map-model
fit analysis, I doubt if Q-score is a good metric.
When there is a flexible atom, its Q score will be low. If the refined
atomic model has correspondingly high ADP, the model is correct
(the depositor and the refinement program did a good job), even though
the Q score may be low. This point is important.
If inexperienced people think low Q scores mean a bad model,
they might be tempted to delete atoms with high ADP to
make the average Q score higher. Wrongly positioned atoms
give bad Q scores but bad Q scores do not necessarily mean a bad model.
In my opinion, local real-space correlation would be better for
map-model fit analysis.
In summary, I think one has to distinguish these two purposes and
use the right metric for the purpose.
Best regards,
Takanori Nakane
P.S.
I also wonder why Q-score is calculated for electron crystallography
entries but not for X-ray crystallography entries.
On 2022/07/26 19:55, Sanja Abbott wrote:
> Starting September 23, wwPDB validation of 3DEM structures for which
> there is both a model and an EM volume will include the Q-score metric
> (Pintilie, G., et al., 2020, Nat. Methods
> <https://www.nature.com/articles/s41592-020-0731-1)>). This follows
> recommendations from the wwPDB/EMDB workshop on cryo-EM data management,
> deposition and validation in 2020 (white paper in preparation), as well
> as EM Validation Challenge events (Lawson C., et al., 2020, Struct. Dyn.
> <https://pubmed.ncbi.nlm.nih.gov/32002441/>; Lawson, et al., 2021, Nat.
> Methods <https://www.nature.com/articles/s41592-020-01051-w>). This will
> be the first quantitative parameter of residue and chain resolvability
> for EM maps in wwPDB validation reports and will provide an additional
> map-model assessment criterion.
>
> The Q-score calculates the resolvability of atoms by measuring
> similarity of the map values around each atom relative to a
> Gaussian-like function for a well resolved atom. Q-score of 1 indicates
> that the similarity is perfect whilst closer to 0 indicates the
> similarity is low. If the atom is not well placed in the map then a
> negative Q-score value may be reported. Therefore, Q-score values in the
> reports will be in a range of -1 to +1.
>
> The wwPDB EM validation reports will provide Q-scores for single
> particle, helical reconstruction, electron crystallography and
> subtomogram averaging entries for which both an EM map and coordinate
> model have been deposited.
>
> Validation reports (PDF files) will contain images of the average
> per-residue Q-scores colour-mapped onto ribbon models with views from
> three orthogonal directions. Similar images will also be introduced to
> visualize the per-residue atom-inclusion scores. Comparison of these two
> sets of images will assist in visual assessment of the model-to-map fit
> and quality.
>
> The images below show the model with each residue coloured according to
> its Q-score.
>
> Example showing mostly cyan colors indicating Q-score closer to 1 and a
> good resolvability of atomsExample showing mostly cyan colours
> indicating Q-score closer to 1 and a good resolvability of atoms
> Example showing mostly red colors indicating Q-score closer to 0 and not
> a good resolvability of atomsExample showing mostly red colours
> indicating Q-score closer to 0 and not a good resolvability of atoms
>
> The validation reports will also contain a table of average per-chain
> values of both metrics (Q-score and atom inclusion) as well as their
> overall average values for the entire model.
>
> The per-residue and the per-chain average atom-inclusion and Q-score
> values will also be provided in the mmCIF and XML formatted validation
> files. The mmCIF categories _pdbx_vrpt_summary_entity_fit_to_map and
> _pdbx_vrpt_model_instance_map_fitting will be introduced to include both
> the Q-scores and atom-inclusion values. The existing items,
> _pdbx_vrpt_summary_entity_geometry.average_residue_inclusion
> <https://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_vrpt.dic/Items/_pdbx_vrpt_summary_entity_geometry.average_residue_inclusion.html>
> and _pdbx_vrpt_model_instance_geometry.residue_inclusion
> <https://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_vrpt.dic/Items/_pdbx_vrpt_model_instance_geometry.residue_inclusion.html>
> for atom inclusion will no longer be used.
>
> The PDB Core Archive holds validation reports that assess each 3DEM
> model in the PDB along with the associated experimental 3D volume in
> EMDB. Validation reports of 3DEM structures (map and model) can be
> downloaded at the following wwPDB mirrors:
>
> * wwPDB: https://ftp.wwpdb.org/pub/pdb/validation_reports/
> * RCSB PDB: https://ftp.rcsb.org/pub/pdb/validation_reports/
> * PDBe: https://ftp.ebi.ac.uk/pub/databases/pdb/validation_reports/
> * PDBj: https://ftp.pdbj.org/pub/pdb/validation_reports/
>
> The EMDB Core Archive holds validation reports that assess each EMDB
> map/tomogram entry. Validation reports for all EMDB volumes can be
> downloaded at the following wwPDB mirrors:
>
> * EMDB: https://ftp.ebi.ac.uk/pub/databases/emdb/validation_reports/
> * wwPDB: https://ftp.wwpdb.org/pub/emdb/validation_reports/
> * RCSB PDB: https://ftp.rcsb.org/pub/emdb/validation_reports/
> * PDBj: https://ftp.pdbj.org/pub/emdb/validation_reports/
>
> Additional information about validation reports is available for EM
> map+model <http://www.wwpdb.org/validation/2017/EMValidationReportHelp>,
> EM map-only
> <http://www.wwpdb.org/validation/2017/EMMapValidationReportHelp>, and EM
> tomograms
> <http://www.wwpdb.org/validation/2017/EMTomogramValidationReportHelp>.
>
> If you have any questions or queries about wwPDB validation, please
> contact us at [log in to unmask]
>
> <http://www.wwpdb.org/news/news?year=2022#top>
> <http://www.wwpdb.org/news/news?year=2022#top>
>
>
> ------------------------------------------------------------------------
>
> To unsubscribe from the CCPEM list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1
> <https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1>
>
########################################################################
To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCPEM&A=1
This message was issued to members of www.jiscmail.ac.uk/CCPEM, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
|
