Tables for
Volume F
Crystallography of biological macromolecules
Edited by M. G. Rossmann and E. Arnold

International Tables for Crystallography (2006). Vol. F, ch. 11.5, pp. 241-242   | 1 | 2 |

Appendix A11.5.1. Partiality model (Rossmann, 1979[link]; Rossmann et al., 1979[link])

C. G. van Beek,a R. Bolotovskya§ and M. G. Rossmanna*

aDepartment of Biological Sciences, Purdue University, West Lafayette, IN 47907-1392, USA
Correspondence e-mail:

Small differences in the orientation of domains within the crystal, as well as the cross fire of the incident X-ray beam, will give rise to a series of possible Ewald spheres. Their extreme positions will subtend an angle 2m at the origin of the reciprocal space, and their centres lie on a cusp of limiting radius [\delta = m/\lambda], where m is the half-angle effective mosaic spread. As the reciprocal lattice is rotated around the axis (Oy) perpendicular to the mean direction of the incident radiation (Oz), a point P will gradually penetrate the effective thickness of the reflection sphere (Fig. A11.5.1.1[link]). Initially, only a few domain blocks will satisfy Bragg's law, but upon further rotation the number of blocks that are in a reflecting condition will increase. The maximum will be reached when the point P has penetrated halfway through the sphere's effective thickness, after which there will be a decline of the crystal volume able to diffract.

[Figure A11.5.1.1]

Figure A11.5.1.1 | top | pdf |

Penetration of a reciprocal-lattice point P into the sphere of reflection by rotation around Oy. The extremes of reflecting conditions at [P_{A}] and [P_{B}] are equivalent to X-rays passing along the lines [S_{1}O] and [S_{2}O] with centres of the Ewald spheres at [S_{1}] and [S_{2}] and subtending an angle of 2m at O. Hence, in three dimensions, the extreme reflecting spheres will lie with their centres on a circle of radius [\delta = m/\lambda] at [z = -1/\lambda].

Let q be a measure of the fraction of the path travelled by P between the extreme reflecting positions [P_{A}] and [P_{B}], and let p be the fraction of the energy already diffracted. Then the relation between p and q must have the general form shown in Fig. A11.5.1.2[link]. It is physically reasonable to assume that the curve for p is tangential to [q = 0] at [p = 0] and to [q = 1] at [p = 1].

[Figure A11.5.1.2]

Figure A11.5.1.2 | top | pdf |

Relationship between the fraction of the path travelled, q, by a reciprocal-lattice point across an Ewald sphere of finite thickness and the fraction of the total scattered intensity, p. The curve shown is for [p = 3q^{2} - 2q^{3}]. As an extreme case, the line [p = q] is also shown.

A reasonable approximation to the above conditions can be obtained by considering the fraction of the volume of a sphere removed by a plane a distance q from its surface (Fig. A11.5.1.2[link]). It is easily shown that if p is the volume, then [p = 3q^{2} - 2q^{3}. \eqno(\hbox{A}] This curve is shown in Fig. A11.5.1.2[link] and corresponds to assuming that the reciprocal-lattice point is a sphere of finite volume cutting an infinitely thin Ewald sphere. Also shown in Fig. A11.5.1.2[link] is the line [p = q] which would result if the reciprocal-lattice point were a rectangular block whose surfaces were parallel and perpendicular to the Ewald sphere at the point of penetration.

Assuming a right-handed coordinate system (x, y, z) in reciprocal space fixed to the camera, it is easily shown (Wonacott, 1977[link]) that the condition for reflection is [d^{*2} + (2z/\lambda) = 0, \eqno(\hbox{A}] where [d^{*}] is the distance of a reciprocal-lattice point P(x, y, z) from the origin, O, of reciprocal space. Similarly, it can be shown that at the ends of the path of the reciprocal-lattice point through the finite thickness of the sphere, [\eqalign{d^{*2} + \delta^{2} + (2z/\lambda) - 2\delta \left(x_{A}^{2} + y_{A}^{2}\right)^{1/2} &= 0 \quad\hbox{and}\cr d^{*2} + \delta^{2} + (2z/\lambda) - 2\delta \left(x_{B}^{2} + y_{B}^{2}\right)^{1/2} &= 0.} \eqno(\hbox{A}] Therefore, [\eqalign{z_{A} &= (\lambda/2) \left[-d^{*2} - \delta^{2} + 2\delta \left(x_{A}^{2} + y_{A}^{2}\right)^{1/2}\right],\cr z_{B} &= (\lambda/2) \left[-d^{*2} - \delta^{2} + 2\delta \left(x_{B}^{2} + y_{B}^{2}\right)^{1/2}\right].} \eqno(\hbox{A}] Since δ is small, it can be assumed that [2\delta (x^{2} + y^{2})^{1/2}] is independent of the position of the reciprocal-lattice point P between the extreme positions [P_{A}] and [P_{B}] (Fig. A11.5.1.1[link]). Hence, the length of the path through the finite thickness of the sphere is proportional to [z_{A} - z_{B} = 2\lambda \delta \left(x_{P}^{2} - y_{P}^{2}\right)^{1/2}. \eqno(\hbox{A}] Now, if a reflection is only just penetrating the sphere at the end of the oscillation range, then the fraction of penetration is given by [q = PP_{A}/P_{A} P_{B} = (z_{P} - z_{A})/(z_{B} - z_{A}). \eqno(\hbox{A}] Substituting this expression into equation (A11.5.1.4)[link], it follows that [q = {\textstyle{1 \over 2}} \left[1 + (D_{1}/\eta_{1})\right], \eqno(\hbox{A}] where [D = d^{*2} + \delta^{2} + (2z/\lambda) \eqno(\hbox{A}] and [\eta = 2\delta (x^{2} + y^{2})^{1/2}. \eqno(\hbox{A}] The subscripts A and B refer to the beginning and end of the oscillation range for the partial reflection P, respectively.

Similarly, if a reflection is almost completely within the sphere, [{q = PP_{B}/P_{A} P_{B} = (z_{B} - z_{P})/(z_{B} - z_{A}) = {\textstyle{1 \over 2}} \left[1 - (D_{2}/\eta_{2})\right].} \eqno(\hbox{A}] There are indeed four such conditions: two while a reflection is entering the Ewald sphere, and two while it is exiting. As such, it is readily seen that [- 1 \lt (D_{i}/\eta_{i}) \lt 1\;(i = 1 \hbox{ or } 2)] is the range for a partial reflection. The full range of conditions is given in Table A11.5.1.1[link], as are the conditions for a full reflection.

Table A11.5.1.1| top | pdf |
Calculation of the degree of penetration of the Ewald sphere, q

The subscripts refer to the angles [\varphi_{1}] and [\varphi_{2}], designating the beginning and end of the oscillation range, respectively. See Fig. A11.5.1.3[link] for graphical representations of conditions 1 to 4.

 Almost completely within sphereAlmost completely outside sphereFull reflection
Entering Condition 1: Condition 2: [D_{1} \big/\eta_{1} \geq + 1 \hbox{ and } D_{2} \big/ \eta_{2} \leq - 1]
  [ - 1 \lt D_{1} \big/ \eta_{1} \lt + 1 \hbox{ and } D_{2} \big/ \eta_{2} \leq - 1] [-1 \lt D_{2} \big/ \eta_{2} \lt + 1 \hbox{ and } D_{1} \big/ \eta_{1} \geq + 1]
Exiting Condition 3: Condition 4: [D_{1} \big/\eta_{1} \leq - 1 \hbox{ and } D_{2} \big/ \eta_{2} \geq + 1]
  [-1 \lt D_{2} \big/ \eta_{2} \lt + 1 \hbox{ and } D_{1} \big/ \eta_{1} \leq - 1] [-1 \lt D_{1} \big/ \eta_{1} \lt + 1 \hbox{ and } D_{2} \big/ \eta_{2} \geq + 1]

[Figure A11.5.1.3]

Figure A11.5.1.3 | top | pdf |

The four conditions 1, 2, 3 and 4 for partial reflections corresponding to Table A11.5.1.1[link]. The arrow ends and heads correspond to the start and end positions of a reciprocal-lattice point, respectively.


Rossmann, M. G. (1979). Processing oscillation diffraction data for very large unit cells with an automatic convolution technique and profile fitting. J. Appl. Cryst. 12, 225–238.
Rossmann, M. G., Leslie, A. G. W., Abdel-Meguid, S. S. & Tsukihara, T. (1979). Processing and post-refinement of oscillation camera data. J. Appl. Cryst. 12, 570–581.
Wonacott, A. J. (1977). Geometry of the oscillation method. In The rotation method in crystallography, edited by U. W. Arndt & A. J. Wonacott, pp. 75–103. Amsterdam: North Holland.

to end of page
to top of page