Tables for
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

International Tables for Crystallography (2006). Vol. F, ch. 11.5, p. 238   | 1 | 2 |

Section 11.5.6. Estimating the quality of data scaling and averaging

C. G. van Beek,a R. Bolotovskya§ and M. G. Rossmanna*

aDepartment of Biological Sciences, Purdue University, West Lafayette, IN 47907-1392, USA
Correspondence e-mail:

11.5.6. Estimating the quality of data scaling and averaging

| top | pdf |

A commonly used estimate of the quality of scaled and averaged Bragg reflection intensities is [R_{\rm merge}]. Useful definitions of R factors are: [\eqalignno{R_{\rm merge} = &\quad R_{1} = \left[\left({\textstyle\sum\limits_{h}} {\textstyle\sum\limits_{i}} \left|I_{hi} - \langle I_{h} \rangle \right|\right)\Big/{\textstyle\sum\limits_{h}} {\textstyle\sum\limits_{i}} \left| I_{hi} \right|\right] \times 100\%,\cr&&(\cr &\quad R_{2} = \left\{\left[{\textstyle\sum\limits_{h}} {\textstyle\sum\limits_{i}} \left(I_{hi} - \langle I_{h} \rangle \right)^{2}\right]\Big/{\textstyle\sum\limits_{h}} {\textstyle\sum\limits_{i}} I_{hi}^{2}\right\} \times 100\% \cr&&(\cr \hbox{and } &\quad R_{w} = \left\{\left[{\textstyle\sum\limits_{h}} {\textstyle\sum\limits_{i}} W_{hi} \left(I_{hi} - \langle I_{h} \rangle \right)^2\right]\Big/{\textstyle\sum\limits_{h}} {\textstyle\sum\limits_{i}} W_{hi} I_{hi}^{2}\right\} \times 100\%. \cr&&(\cr}] The linear (R1), square (R2) and weighted ([R_{w}]) R factors can be subdivided into resolution ranges, intensity ranges, reflection classes, frame number and regions of the detector surface. When method 1[link] is used, reflections [h_{i}] can be grouped in terms of the sums of partialities of contributing partial reflections [h_{im}].

The R-factor variation depends on the properties of the detector with respect to intensities. Generally the R factor decreases as intensity increases. Thus, the R factor generally increases with resolution. Any deviation from this behaviour might indicate a problem in the data collection due to nonlinearity of the detector response, ice diffuse diffraction, or any other stray effects superimposed on the crystal diffraction.

A useful indicator of the quality of the intensity estimates of partial reflections is the mean ratio of calculated partiality to observed partiality: [r_{p} = \langle p_{him}^{\rm calc}\big/p_{him}^{\rm obs}\rangle = \langle p_{him}^{\rm calc} \langle I_{h} \rangle\big/I_{him} \rangle . \eqno(] The deviation of this ratio from unity can be examined as a function of the reflection intensity, resolution and calculated partiality.

The comparison of R factors for centric and noncentric reflections can be used to determine the significance of an anomalous-scattering effect. The quality of the anomalous-dispersion signal can be assessed by calculation of the scatter, [\sigma_{Ih}], where [\sigma _{Ih} = \left\{\left[1\big/(n - 1)\right] {\textstyle\sum\limits_{n}} \left(\langle I_{h} \rangle - I_{hn}\right)^{2}\right\}^{1/2} \eqno(] and [\langle I_{h} \rangle] is the average of the n measurements of the full reflection intensities [I_{hn}]. The [\sigma_{Ih}] values for noncentric reflections can be compared to the scatter, [\sigma_{Ih}^{+}] or [\sigma_{Ih}^{-}], of reflections differing only in absorption while excluding Bijvoet opposites. The mean scatter is calculated from all [\sigma_{Ih}] values, [\langle \sigma_{Ih} \rangle = (1/h) {\textstyle\sum\limits_{h}} \left\{\left[1\big/ (n - 1)\right] {\textstyle\sum\limits_{n}} \left(\langle I_{h} \rangle - I_{hn} \right)^{2}\right\}^{1/2}. \eqno(] The ratios [\langle \sigma_{Ih} \rangle\big/\langle \sigma_{Ih}^{+} \rangle] and [\langle \sigma_{Ih} \rangle\big/\langle \sigma_{Ih}^{-} \rangle] should be larger than unity for significant anomalous-dispersion data.

to end of page
to top of page