International
Tables for
Crystallography
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

International Tables for Crystallography (2006). Vol. F, ch. 11.5, p. 238   | 1 | 2 |

## Section 11.5.5. Generalization of the procedure for averaging reflection intensities

C. G. van Beek,a R. Bolotovskya§ and M. G. Rossmanna*

aDepartment of Biological Sciences, Purdue University, West Lafayette, IN 47907-1392, USA
Correspondence e-mail:  mgr@indiana.bio.purdue.edu

### 11.5.5. Generalization of the procedure for averaging reflection intensities

| top | pdf |

Once the scale factors of all frames are determined, they need to be applied to the reflection intensities and error estimates. The reflection intensities with the same reduced Miller indices can then be averaged.

When method 2 is used for averaging, the determination of is more complicated as there are as many estimates of the full intensity as there are partial reflections . Therefore, intensity averaging of reflection h has to be done in two steps. First, for every reflection , the intensity estimates from all partial observations will be the weighted mean, where the weights are based on the estimated standard deviations of each intensity measurement. In the second step, the average is taken over the i different scaled intensities for the observed reflections.

The selection of reflections useful for averaging is the same as for scaling (Table 11.5.3.1), except that it is no longer necessary to reject reflections that have insignificant intensities. Applying a σ cutoff while averaging the scaled intensities will lead to a statistical bias of the weaker reflection intensities.

For samples of three or more equivalent reflections, it is necessary to consider the absolute values of the differences between individual intensities and the median of the sample: . The outliers can be detected by several statistical tests and, once detected, can be either down-weighted or rejected. When the sample consists of only two reflections, they can be considered a discordant pair' if the difference between their intensities is not warranted by the estimated errors and, hence, both reflections can be rejected (Blessing, 1997).

Averaging intensities estimated according to method 2 has an advantage over method 1 as outliers and discordant pairs can be screened' at two levels: firstly, when the estimates of the full reflection intensity , calculated by expression (11.5.2.2) from different parts of the same reflection, are considered, and secondly when the mean intensities from different reflections are considered.

### References

Blessing, R. H. (1997). Outlier treatment in data merging. J. Appl. Cryst. 30, 421–426.