International
Tables for Crystallography Volume F Crystallography of biological macromolecules Edited by E. Arnold, D. M. Himmel and M. G. Rossmann © International Union of Crystallography 2012 
International Tables for Crystallography (2012). Vol. F, ch. 13.2, pp. 340346
https://doi.org/10.1107/97809553602060000840 Chapter 13.2. Rotation functions ^{a}Laboratoire de Génétique des Virus, CNRSGIF, 1. Avenue de la Terrasse, 91198 GifsurYvette, France The mathematical procedures required to find the relative orientations of the independent but homologous molecules within a crystal, or their absolute orientations if the structure of a similar molecule is known, are discussed in detail. They correspond to the rotational search employed in the molecularreplacement method. 
We will discuss a technique to find either the relative orientations of homologous but independent subunits connected by noncrystallographic symmetry (NCS) elements or the absolute orientations of these subunits if the structure of a similar molecule or fragment is available. The procedure makes intensive use of properties of the rotation group, so we will start by recalling some properties of rotations. More advanced results are included in Appendix 13.2.1.
A rotation R is specified by an oriented axis, characterized by the unit vector u, and the spin, χ, about it. Positive spins are defined by the righthand screw sense and values are given in degrees. An almost onetoone correspondence between rotations and parameters (χ, u) can be established. If we restrict the spin values to the positive interval , then for each rotation there is a unique vector χu within the sphere of radius 180. However, vectors situated at opposite points on the surface correspond to the same rotation, e.g. (180, u) and (180, −u).
When the unit vector u is specified by the colatitude ω and the longitude with respect to an orthonormal reference frame (see Fig. 13.2.2.1a), we have the spherical polar parameterization of rotations (χ, ω, ϕ). The range of variation of the parameters is Rotations may also be parameterized with the Euler angles (α, β, γ) associated with an orthonormal frame (x, y, z). Several conventions exist for the names of angles and definitions of the axes involved in this parameterization. We will follow the convention by which (α, β, γ) denotes a rotation of α about the z axis, followed by a rotation of β about the nodal line n, the rotated y axis, and finally a rotation of γ about p, the rotated z axis (see Fig. 13.2.2.1b): The same rotation may be written in terms of rotations around the fixed orthonormal axes. By using the group property which is valid for any rotation T, we obtain (see Appendix 13.2.1) The parameters (α, β, γ) take values within the parallelepiped Here again, different values of the parameters may correspond to the same rotation, e.g. (α, 180, γ) and .

Illustration of rotations defined by (a) the spherical polar angles (χ, ω, ϕ); (b) the Euler angles (α, β, γ). 
Although rotations are abstract objects, there is a onetoone correspondence with the orthogonal matrices in threedimensional space. In the following sections, R will denote a orthogonal matrix. An explicit expression for the matrix which corresponds to the rotation (χ, u) isor, in condensed form, where is the Kronecker tensor, are the components of u, and is the Levi–Civita tensor. The rotation matrix in the Euler parameterization is obtained by substituting the matrices in the righthand side of equation (13.2.2.3) by the corresponding expressions given by equation (13.2.2.4).
The idea of distance between rotations is necessary for a correct formulation of the problem of sampling and for plotting functions of rotations (Burdina, 1971; Lattman, 1972). It can be demonstrated that the quantity defines a metric on the rotation group, unique up to a multiplicative constant, which cannot be reduced to a Cartesian metric. This is a topological property of the group, independent of its parameterization. ds is interpreted as the distance between the rotations R and . With the Euler parameterization, equation (13.2.2.6) becomes The volume element of integration, , corresponding to this length element guarantees the invariance of integrals over the rotation angles with respect to initial reference orientations.
The distance defined by equation (13.2.2.6) has a simple physical interpretation. Let us consider a molecule with initial atomic coordinates , referred to an orthonormal frame parallel to the molecule's principal moments of inertia, . Then, the coordinates satisfy the conditions where means `average over atoms'. If we move the molecule from a rotated position characterized by the rotation R to a close one characterized by , the meansquare shift of the atomic coordinates is When all the are equal, σ becomes proportional to ds.
The absolute and relative orientations of the subunits that constitute a crystal may in principle be found by exploiting some properties of the Patterson function. We first express the crystal structure factors in terms of the Fourier transform of the electron density of isolated molecules, , calculated with their centres of gravity placed at the origin. If denotes the position within the crystal of the centre of gravity of the mth molecule, we have where M is the number of molecules within the unit cell. This is a general expression for , although the spacegroup and noncrystallographic rotational symmetries are not explicitly exhibited. The presence of symmetry implies that some of the 's are in fact samples of the same function at rotated arguments. The Fourier coefficients of the Patterson function can then be written as where is the complex conjugate of . The first term in equation (13.2.3.2) involves only intramolecular contributions or selfPatterson terms; they are centred at the origin of the Patterson function. The second term involves intermolecular contributions or crossPatterson terms; they are centred at the intermolecular vectors . Therefore, if we restrict the Patterson function to a region Ω, of volume v, centred at the origin and having a dimension of the order of the dimensions of the isolated molecules, selfPatterson terms will dominate the crossed ones.
Thus it becomes possible to determine the rotation R that superimposes one subunit upon an independent, homologous one by calculating the overlap within the region Ω of the observed Patterson function (the target function ) and a rotated version of either itself or the Patterson function of an isolated known molecule (the search function ): (Rossmann & Blow, 1962). should display a local maximum for the sought rotation. Note that when we rotate the search function by R, its argument contains . When the search and the target functions are the same, is called the selfrotation function; otherwise, it is called the crossrotation function. When a model is available, the target data are calculated by placing the model within a P1 cell whose dimensions guarantee that Ω only contains selfPatterson contributions.
The reciprocalspace formulation of the above integral is obtained by substituting the Patterson functions by their Fourier summations: Taking into account that , we obtain , the interference function, is the Fourier transform of the characteristic function of Ω, i.e., a function that takes the value 1 within Ω and 0 outside. In principle, the domain of integration could have any shape. However, in order to take full advantage of the properties of the rotation group, Ω is usually chosen as a spherical domain of radius b. Calling for short, we have In the case of a spherical shell with inner and outer radii a and b, respectively, the interference function is obtained by subtraction:
Although simple, the resulting expression for the rotation function has the disadvantage of containing entangled h, k and R contributions, which renders its computation timeconsuming if the whole domain of rotations has to be explored. The difficulty may be overcome by expanding the exponentials entering equation (13.2.3.5) in spherical harmonics, . Taking advantage of their transformation under rotations, and using recurrence relationships between spherical Bessel functions, , we obtain (see Appendix 13.2.1)where are the matrices of the irreducible representations of the rotation group and stands for `angular part of vector h'. The awkwardness of equation (13.2.3.8) is apparent rather than real. Indeed:
This formulation is referred to as the fast rotation function (Crowther, 1972).
It may be useful to compare rotation functions obtained under different conditions. For this, some kind of normalization is needed. One possibility is to cast into the form of a correlation coefficient by dividing equation (13.2.3.3) by the norms of the truncated Patterson functions,
A direct evaluation of equation (13.2.3.3) is possible. However, since the values of the Patterson functions are only available at discrete sampling points, an interpolation is needed after each rotation. This method is known as the directspace formulation of the rotation function (Nordman, 1966; Steigemann, 1974).
In the reciprocalspace formulation [equations (13.2.3.5) and (13.2.3.6)], some precautions are to be taken in order to save computing time:
In both formulations, it is worth using a distance between rotations to sample efficiently, as explained in Section 13.2.3.2.
For the fast rotation function [equations (13.2.3.5) and (13.2.3.8)], the calculations are organized as follows (Dodson, 1985; Navaza, 1993):
An odd feature of the rotation domain is that quite different values of the parameters may correspond to very close rotations. This causes the graphic representation of the rotation function to be distorted. For example, when the Euler angle β is small or close to 180, a peak plotted on a Cartesian grid in α and γ is strongly elongated in one direction. By redefining the reference orientation, the same peak near the section will have a quite different – natural – shape.
This problem may be partially overcome by using the distance between rotations introduced in Section 13.2.2.1. Although the metric cannot be reduced to a Cartesian one in three dimensions, it is possible to do so for twodimensional sections. Indeed, for fixed β, the transformation reduces equation (13.2.2.7) to . Therefore, distortionfree β sections may be plotted in terms of the variables (Burdina, 1971; Lattman, 1972). Accordingly, a correct sampling of should consist of points regularly spaced in these variables. The number of points with respect to that of a regular spacing in α, β and γ is approximately Notice that for β equal to 0 or 180, the sections reduce to lines.
In the case of the selfrotation function, when the order of the NCS may be anticipated, it is worth working with equation (13.2.2.6) in the polar parameterization, For constant χ, we have the topology of the spherical surface. It is then natural to plot the function as a stereographic projection.
In the case of the crossrotation function, i.e., when the atomic distribution of the search model is known, we may use equation (13.2.2.9) instead of equation (13.2.2.6) to define the sampling. For this, we choose σ as the smallest rootmeansquare shift in atomic coordinates that produces an appreciable change in the value of the crossrotation function. An acceptable sampling set of should then satisfy the following conditions:
Since the first attempts to detect subunits within the crystal asymmetric unit, a great deal of experience has been gained and the results collected in several treatises on molecular replacement (Rossmann, 1972; Machin, 1985; Dodson et al., 1992; Carter & Sweet, 1997).
The success of the rotation function relies in part on the choice of the domain of integration, i.e., inner and outer radii of the spherical domain, and limits on data resolution. The outer radius is chosen so as to maximize the ratio between the number of intramolecular and intermolecular vectors. A typical value of this radius is 50–75% of the subunit's diameter for spherical molecules. The choice of the inner radius is less crucial, provided that the origin peaks are subtracted from the Patterson functions; it is often set to zero.
The resolution range of the selected diffraction data depends on whether we are computing self or crossrotation functions. For self rotations, data up to the highest available resolution may be used. For cross rotations, the highresolution limit is dictated by the degree of similarity between the probe and the actual molecules that constitute the crystal. The lowresolution limit is usually chosen so as to skip the solvent contribution.
Another parameter of interest concerns the angular resolution, which is rather specific to the fast rotation function. Although it was derived by expanding the interference function to follow the original reciprocalspace formulation as closely as possible, it may also be obtained by expanding the Patterson functions in spherical harmonics and then performing the integration (Crowther, 1972). In this way, the concept of angular resolution is more evident. In fact, the spherical harmonics play the same role in the angular domain as the imaginary exponentials do in Cartesian coordinate space. The analogue of the Miller indexes here is the label ℓ, directly related to the angular resolution. The term with is invariant under rotations and is thus eliminated from the summation in equation (13.2.3.14). It represents the contribution of the best radial functions, in a leastsquares sense, that approximate the Patterson functions within Ω. The origin peak is thus properly removed by omitting the term, as well as a substantial component of the original function. Also, by omitting low ℓ's, the angular resolution of peaks is enhanced. These eliminations may also be done in other formulations, but never as efficiently.
The parameters discussed above are not all independent. The relationship between them stems from the behaviour of the spherical Bessel functions for small values of their argument (Watson, 1958), [see equation (13.2.3.8)]. So, when omitting low ℓ terms, we are also weighting down lowresolution data.
The overlap integral that defines may be calculated by rotating instead of , but with the inverse rotation, This property enables the analysis of the consequence of the symmetries of the Patterson functions upon the rotation function (Tollin et al., 1966; Moss, 1985). For example, when the target and search functions are the same, a trivial symmetry of the selfrotation results: its values at R and are the same or, in Euler angles, at (α, β, γ) and . More generally, if is invariant under the rotation T, i.e., [and similarly for the search function, ], then However, the actual symmetry displayed by depends on the parameterization of the rotations and on the orientation of the orthonormal axes with respect to the crystal ones. Within the Euler parameterization, if the target function has an nfold rotation axis parallel to the orthonormal z axis, then, according to equation (13.2.2.3), will have a periodicity of along α. Similarly, a rotation axis of order n along z of gives rise to a periodicity of along γ. Therefore, the amount of calculation is reduced by choosing z along the Patterson functions' highest rotational symmetry axes [see equation (13.2.3.11)].
The rotational NCS, determined with the help of the selfrotation function, may be used to enhance the signaltonoise ratio of crossrotation functions (Rossmann et al., 1972; Tong & Rossmann, 1990). If denotes the set of NCS rotations, including the identity, and R is a correct orientation of the cross rotation, then must also correspond to a correct orientation. Here we are assuming that the rotational NCS forms a group. Otherwise, either or , but not both, corresponds to another correct orientation. Therefore, a function may be defined, the locked cross rotation, whose values are the average of the crossrotation values at orientations related by the NCS: By redefining the target function, it can be computed as an ordinary cross rotation. Indeed, may be written in a form similar to equation (13.2.3.3), with the target Patterson function substituted by the average over the NCS of the rotated target functions. The computation of equation (13.2.4.2) is particularly simple in the case of the fast rotation function. The substitution where we replaced the sum over by a sum over , because of the rearrangement theorem of group theory, gives the required target coefficients.
The same ideas may be applied to the selfrotation function. Here the NCS is assumed beforehand, with elements in a given reference orientation. They are related to the actual NCS elements by [see equation (13.2.2.2)]. Since each should correspond to a local maximum of the self rotation, the function should also display a maximum for each , but with a noise level reduced by . Equation (13.2.4.5) defines the locked selfrotation function. Its computation by fast rotation techniques is less straightforward than in the locked crossrotation case.
The rotation function was hitherto described in terms of self and crossPatterson vectors. This is perhaps inevitable in the selfrotation case, but the problem of determining the absolute orientation of the subunits when a model structure is available may be formulated in a different way. We may try to compare directly the observed and calculated intensities or structure factors by using any criterion analogous to those employed in refinement procedures, e.g., the crystallographic R factor or correlation coefficients.
When the spacegroup symmetry is explicitly exhibited, the structure factor corresponding to a crystal with M independent molecules in the unit cell takes the form where and denote, respectively, the transformation matrix and the translation associated with the gth symmetry operation of the crystal space group. The corresponding intensity is
For criteria based on amplitudes, the calculated structure factor will contain only the contribution of the rotated model, i.e., the Fourier transform of a single molecule in the crystal cell, assuming P1 symmetry. For criteria based on intensities, some symmetry information may be introduced, A criterion often considered is the correlation coefficient on intensities, where means `average over reflections'. It may be calculated within reasonable computing time provided that
is referred to as the directrotation function (DeLano & Brünger, 1995). A major advantage of this formulation is that the information stemming from alreadypositioned subunits may be taken into account, just by adding their contribution to the calculated intensities.
Each formulation of the rotation function described above has its advantages and disadvantages. The directspace formulation [equation (13.2.3.3)] offers the possibility of modifying the Patterson function or selecting the strongest peaks to be used in the overlap integral. Also, the domain of integration may have any shape, as in the reciprocalspace formulation [equation (13.2.3.5)]. However, in both formulations, the numerical results can be somewhat imprecise because of the approximations introduced to save computing time.
When the domain of integration is spherical, as is usually the case, then the fast rotation function [equation (13.2.3.15)] is faster, more accurate and allows for angularresolution enhancement. Moreover, since most of the computing time is spent in the calculation of the coefficients , a library of these coefficients may be compiled to assess the models in a given molecularreplacement problem more rapidly.
The direct rotation [equation (13.2.5.5)] seems preferable to the crossrotation function, as it includes all selfPatterson terms of the search model. However, when the order of the space group is high, the calculated intensity represents only a small fraction of the observed one, and the discriminative power of the function drops, as compared with the crossrotation function.
Patterson searches now benefit from new supercomputers. The real problem of molecular replacement, split for convenience into rotation and translation searches, is beginning to be tackled by genuine sixdimensional searches where all orientations are tested. This may represent the end of crossrotation and directrotation functions.
Appendix A13.2.1
This appendix aims to present a complete set of formulae which allow the derivation and computation of the fast rotation function. They involve a particular convention for the definition of the irreducible representations of the rotation group suitable for crystallographic computations.
By applying the group property of rotations [equation (13.2.2.2)], the Euler parameterization may be expressed as rotations around fixed axis (see Fig. 13.2.2.1b):
A linear representation of dimension n of the rotation group is a correspondence between rotations and matrices of order n. The matrices , with , are associated with the irreducible representation of dimension (). They have the following properties (Brink & Satchler, 1968):
The 's, with and , constitute a complete set of functions of the unit vector u, having the following properties (Brink & Satchler, 1968):
The 's, with , constitute a complete set of functions having the following properties (Watson, 1958):
References
Brink, D. M. & Satchler, G. R. (1968). Angular Momentum, 2nd ed. Oxford University Press.Burdina, V. I. (1971). Symmetry of rotation function. Sov. Phys. Crystallogr. 15, 545–550.
Carter, C. W. & Sweet, R. M. (1997). Molecular replacement. Methods Enzymol. 276, 558–619.
Crowther, R. A. (1972). In The Molecular Replacement Method, edited by M. G. Rossmann, pp. 173–178. New York: Gordon and Breach.
DeLano, W. L. & Brünger, A. T. (1995). The direct rotation function: rotational Patterson correlation search applied to molecular replacement. Acta Cryst. D51, 740–748.
Dodson, E. J. (1985). In Proceedings of the Daresbury Study Weekend. Molecular Replacement, edited by P. A. Machin, pp. 33–45. Warrington: Daresbury Laboratory.
Dodson, E. J., Gover, S. & Wolf, W. (1992). Editors. Proceedings of the Daresbury Study Weekend. Molecular Replacement. Warrington: Daresbury Laboratory.
Landau, L. D. & Lifschitz, E. M. (1972). Théorie Quantique Relativiste, pp. 109–196. Moscow: Editions MIR.
Lattman, E. E. (1972). Optimal sampling of the rotation function. Acta Cryst. B28, 1065–1068.
Machin, P. A. (1985). Editor. Proceedings of the Daresbury study weekend. Molecular replacement. Warrington: Daresbury Laboratory.
Moss, D. S. (1985). The symmetry of the rotation function. Acta Cryst. A41, 470–475.
Navaza, J. (1993). On the computation of the fast rotation function. Acta Cryst. D49, 588–591.
Nordman, C. E. (1966). Vector space search and refinement procedures. Trans. Am. Crystallogr. Assoc. 2, 29–38.
Rossmann, M. G. (1972). Editor. The Molecular Replacement Method. New York: Gordon and Breach.
Rossmann, M. G. & Blow, D. M. (1962). The detection of subunits within the crystallographic asymmetric unit. Acta Cryst. 15, 24–31.
Rossmann, M. G., Ford, G. C., Watson, H. C. & Banaszak, L. J. (1972). Molecular symmetry of glyceraldehyde3phosphate dehydrogenase. J. Mol. Biol. 64, 237–245.
Steigemann, W. (1974). Die Entwicklung und Anwendung von Rechenverfahren und Regenprogrammen zur Strukturanalyse von Proteinen am beispiel des Trypsin–Trypsininhibitor Komplexes, des Freien und der LAsparaginase. PhD thesis, Technische Universität, Munich, Germany.
Tollin, P., Main, P. & Rossmann, M. G. (1966). The symmetry of the rotation function. Acta Cryst. 20, 404–417.
Tong, L. & Rossmann, M. G. (1990). The locked rotation function. Acta Cryst. A46, 783–792.
Watson, G. N. (1958). A Treatise on the Theory of Bessel Functions, 2nd ed. Cambridge University Press.