Tables for
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

International Tables for Crystallography (2006). Vol. F, ch. 11.4, pp. 231-232   | 1 | 2 |

Section 11.4.6. Prediction of the diffraction pattern

Z. Otwinowskia* and W. Minorb

aUT Southwestern Medical Center at Dallas, 5323 Harry Hines Boulevard, Dallas, TX 75390-9038, USA, and bDepartment of Molecular Physiology and Biological Physics, University of Virginia, 1300 Jefferson Park Avenue, Charlottesville, VA 22908, USA
Correspondence e-mail:

11.4.6. Prediction of the diffraction pattern

| top | pdf |

The autoindexing procedure assigns Miller indices only to strong spots, ones that can be found through a peak search. The target of the experiment is to estimate structure factors for all reflections captured by the detector. Therefore, positions of all spots need to be predicted by applying the following equations to all possible triplets h. Using [{\bf S} = [A]^{-1} {\bf h}, \eqno(] we have to find the matrix [A] that generates the vector S, which satisfies the diffraction condition [equation ([link])], knowing that the matrix [A] is a function of the crystal orientation [equation ([link])]. The rotation of the crystal during the experiment creates a straightforward algebraic problem that results in a complex equation defining the angle at which the reflection occurs. This angle also defines the image at which the reflection appears. Knowing this angle, the vector S can be calculated, and, from equation ([link]), the direction of the vector X can be found: [{\bf X}/|{\bf X}| = \lambda ({\bf S} - {\bf S}_{0}). \eqno(] Calculation of the length of vector X requires knowledge of the detector orientation, which, for flat detectors, is described here by vector G, perpendicular to the detector and with length equal to the crystal-to-detector distance: [\eqalignno{{\bf Y} &= [R_{2\theta}]^{-1} \lambda ({\bf S} - {\bf S}_{0}), &(\cr {\bf H} &= [R_{z}]^{-1} [R_{y}]^{-1} [R_{x}]^{-1} {\bf G}, &(\cr {\bf X} &= [R_{2\theta}] {{\bf H} \cdot {\bf H} \over {\bf Y} \cdot {\bf H}} {\bf Y}. &(}%(] Then, by inverting equation ([link]), the position in pixels, [\{p, q\}], of the reflection can be calculated: [\eqalign{\{p, q\} &= [D]^{-1} [K]^{-1} ([L]^{-1} ([R_{2\theta}]^{-1} ([R_{z}]^{-1} [R_{y}]^{-1} [R_{x}]^{-1}({\bf X} \cr &\quad- T_{D}) + T_{D}) - T_{D}) + B).} \eqno(] Refinement of crystal and detector parameters

| top | pdf |

The precision of the integration step depends on precise knowledge of the peak position. The autoindexing step provides only the approximate orientation of the crystal, and the result of that step is imprecise if the initial values of the detector parameters are poorly known. A nonlinear least-squares refinement process is used to improve the prediction (EEC Cooperative Workshop on Position-Sensitive Detector Software, 1986[link]). Depending on the particulars of the experiment, the same parameters (e.g. crystal-to-detector distance) are more precisely known a priori, or are better estimated from the diffraction data. DENZO allows for the choice of fixing or refining each of the parameters separately. This flexibility is important to characterize a detector, but when detector parameters are already known, the fit all option and detector-specific default values are quite reliable.

DENZO can refine the position and orientation of the detector (six parameters). It can also refine internal parameters of the detector including:

  • (1) curvature radius and rotation of active surface – film rotation, for cylindrical detectors;

  • (2) x to y scale and x to y skew;

  • (3) radial/angular distortion for spiral scanners; and

  • (4) polynomial distortion, separate for each module of multimodule CCD detectors (Naday et al., 1998[link]).

Detector- and crystal-parameter refinement in DENZO is achieved by minimizing the sum of the three functions of the type in equation ([link]). The contribution resulting from the measurement of position p of the reflection is [\chi_{p}^{2} = {\textstyle\sum\limits_{\rm hkl}} (p_{\rm pred} - p_{\rm cent})^{2}/\sigma_{p}^{2}. \eqno(] The measurement of position q contributes a similar term. Bragg's law for non-ideal conditions: mosaicity

| top | pdf |

The Bragg condition [equation ([link])] assumes ideal crystals and a parallel X-ray beam. In reality, crystals are mosaic and the beam has some angular spread. The value of mosaicity describes the range of orientations of the crystal lattice within a sample. As the impacts of mosaicity and the beam's angular spread on the angular width of reflections are equivalent, the keyword mosaicity describes the sum of both effects.

DENZO assumes the following model of angular shape of diffraction peaks: [M = {\hbox{mos} \over \pi} \left[1 + \cos {\pi (\varphi - \varphi_{c}) \over \hbox{mos}}\right] \eqno(] for [\varphi_{c}] in the range ([\varphi_{c} - \hbox{mos}/2; \varphi_{c} + \hbox{mos}/2), \hbox{ otherwise } M = 0], [P_{\varphi_{1} \varphi_{2}} = {\textstyle\int\limits_{\varphi_{1}}^{\varphi_{2}}} M (\varphi)\ \hbox{d} \varphi, \eqno(] where mos is mosaicity, [\varphi_{c}] is the predicted angle and P is the predicted partiality of data collected by oscillating from [\varphi_{1}] to [\varphi_{2}].

Partiality is a number that represents what fraction of the reflection intensity is present in one image. If partiality is 1, such reflections are called fully recorded; otherwise they are called partials. For partials, predictions of partiality can be compared with the observed fraction [P_{0}] of the reflection intensity present in one image. The partiality model contributes the following term to the refinement: [\chi_{P}^{2} = (P_{\varphi_1 \varphi_{2}} - P_{0})^{2}/{\sigma{_{P_{0}}^2}}. \eqno(] The combined positional and partiality refinement used in DENZO is both stable and very accurate. The power of this method is in proper weighting (by estimated error) of two very different terms – one describing positional differences and the other describing intensity differences. Both detector and crystal variables are uniformly treated in the refinement process. Detector distortions

| top | pdf |

The design of detectors results in pixels not being positioned on an exact square or rectangular grid. A correct understanding of the detector distortions is essential to accurate positional refinement. The types of distortions are detector-specific. The primary sources of error include misalignment of the detector position sensors and optical or magnetic distortion in CCD-based detectors. If the detector distortion can be parameterized, then these parameters should be added to the refinement. For example, in the case of spiral scanners, there are two parameters describing the end position of the scanning head. In a perfectly adjusted scanner, these parameters would be zero. In practice, however, they may deviate from zero by as much as 1 mm. Such misalignment parameters can correlate very strongly with other detector and crystal parameters, particularly for low-symmetry lattices or in the case of low-resolution data. If the distortions are stable, it is better to determine them in a separate experiment optimized for that task.

Fibre-optic tapers used in many CCD detectors have distortion that has to be individually determined for each instrument. The distortion is stable over time and its spatial characteristics are dominated by a smooth component and a small local shear. In high-quality tapers used in X-ray instruments, the small local shear can be ignored. The smooth component can be parameterized in a number of ways, for example by splines (Hammersley, 1998[link]) or polynomials (Messerschmidt & Pflugrath, 1987[link]). DENZO uses two-dimensional Chebyschev polynomials (Press et al., 1989[link]) in {x, y} or {p, q} coordinates, normalized to the range [\{[-1, +1]], [[-1, +1]\}]. Typically, fifth- or seventh-order polynomials result in a positional error (r.m.s.) lower than 7 µm (about one tenth of the detector pixel). DENZO can use either a grid mask pattern or the X-ray diffraction pattern to refine the coefficients of the Chebyschev polynomials. If a grid mask is used, it has to be precisely made and positioned. The use of crystallographic data requires precise knowledge of detector and crystal parameters that are not known a priori with the required precision. The crystal and detector parameters can be determined in the same experiment as detector distortion. However, this experiment needs to be designed to minimize the impact of correlations between the parameters involved. The data analysis requires the description of the distortion function and its inverse. In DENZO, both are approximated in terms of Chebyschev polynomials. The magnitude of the approximation error is the same for the distortion function and its inverse.


EEC Cooperative Workshop on Position-Sensitive Detector Software (1986). Phase I and II, LURE, Paris, 16 May–7 June; Phase III, LURE, Paris, 12–19 November.
Hammersley, A. P. (1998). The FIT2D home page. .
Messerschmidt, A. & Pflugrath, J. W. (1987). Crystal orientation and X-ray pattern prediction routines for area-detector diffraction systems in macromolecular crystallography. J. Appl. Cryst. 20, 306–315.
Naday, I., Ross, S., Westbrook, E. M. & Zentai, G. (1998). Charge-coupled device/fiber optic taper array X-ray detector for protein crystallography. Opt. Eng. 37, 1235–1244.
Press, W. H., Flannery, B. P., Teukolsky, S. A. & Vetterling, W. T. (1989). Numerical recipes – the art of scientific computing. Cambridge University Press.

to end of page
to top of page