International
Tables for
Crystallography
Volume B
Reciprocal space
Edited by U. Shmueli

International Tables for Crystallography (2006). Vol. B, ch. 3.2, pp. 358-359   | 1 | 2 |
doi: 10.1107/97809553602060000560

## Appendix A3.2.1.

R. E. Marsha* and V. Schomakerb

aThe Beckman Institute–139–74, California Institute of Technology, 1201 East California Blvd, Pasadena, California 91125, USA, and  bDepartment of Chemistry, University of Washington, Seattle, Washington 98195, USA
Correspondence e-mail:  rem@xray.caltech.edu

Consider n atoms at observed vector positions (expressed in Cartesians), n constraints (each adjusted position for adjusted' – must be on the plane) and adjustable parameters ( components and the 3 components of the vector of reciprocal intercepts of the plane), so that the problem has degrees of freedom. The weight matrices may be differently anisotropic for each atom, but there are no interatomic correlations. As before, square brackets, ', represent the Gaussian sum over all atoms, usually suppressing the atom indices. We also write , not the of Section 3.2.2, for the Lagrange multipliers (one for each atom); for the direction cosines of the plane normal; and d for the perpendicular origin-to-plane distance.

As before, is the reciprocal of the atomic error matrix: (correspondingly, but ' is no longer the ' of Section 3.2.2. The appropriate least-squares sum is and the augmented sum for applying the method of Lagrange multipliers is is to be minimized with respect to variations of the adjusted atom positions and plane reciprocal intercepts , leading to the equations subject to the plane conditions , each atom, with , . These equations are nonlinear.

A convenient solution runs as follows: first multiply the first equation by and solve for the adjusted atom positions in terms of the Lagrange multipliers and the reciprocal intercepts of the plane; then multiply that result by applying the plane conditions, and solve for the 's Next insert these values for the 's and 's into the second equation:

This last equation, , is to be solved for . It is highly nonlinear: . One can proceed to a first approximation by writing ; i.e., , in dyadic notation. , all atoms; in the multiplier of A linear equation in , this approximation usually works well.6 We have also used the iterative Frazer, Duncan & Collar eigenvalue solution as described by SWMB (1959), which works even when the plane passes exactly through the origin. To continue the solution of the nonlinear equations, by linearizing and iterating, write in the form solve for , reset to , etc., until the desired degree of convergence of toward zero has been attained. By is meant the partial derivative of the above expression for with respect to , as detailed in the next paragraph.

In the Fortran program DDLELSP (double precision Deming Lagrange, with error estimates, least-squares plane, written to explore this solution) the preceding equation is recast as with

The usual goodness of fit, GOF2 in DDLELSP, evaluates to This is only an approximation, because the residuals are not the differences between the observations and appropriate linear functions of the parameters, nor are their variances (the 's) independent of the parameters (or, in turn, the errors in the observations).

We ask also about the perpendicular distances, e, of atoms to plane and the mean-square deviation to be expected in e. Here and are the errors in and . Neglecting `' then leads to We have , but and perhaps still need to be evaluated.

### References

Schomaker, V., Waser, J., Marsh, R. E. & Bergman, G. (1959). To fit a plane or a line to a set of points by least squares. Acta Cryst. 12, 600–604.