Tables for
Volume H
Powder diffraction
Edited by C. J. Gilmore, J. A. Kaduk and H. Schenk

International Tables for Crystallography (2018). Vol. H, ch. 1.1, pp. 19-22

Section 1.1.6. Local and global optimization of crystal structures from powder diffraction data

R. E. Dinnebiera* and S. J. L. Billingeb,c

aMax-Planck-Institute for Solid State Research, Heisenbergstrasse 1, D-70569 Stuttgart, Germany,bDepartment of Applied Physics and Applied Mathematics, Columbia University, 500 West 120th Street, Room 200 Mudd, MC 4701, New York, NY 10027, USA, and cCondensed Matter Physics and Materials Science Department, Brookhaven National Laboratory, PO Box 5000, Upton, NY 11973–5000, USA
Correspondence e-mail:

1.1.6. Local and global optimization of crystal structures from powder diffraction data

| top | pdf | Rietveld refinement

| top | pdf |

More than 40 years have passed since the publication of the pioneering papers by Hugo Rietveld (Rietveld, 1967[link], 1969[link]), in which he described a method for the refinement of crystal structures from neutron powder diffraction data. Neutron data sets from reactor sources were more amenable than X-ray data sets to this method because the line profiles are quite Gaussian. However, it was not long before the method was extended to X-ray powder diffraction. The quality of the data and the computation power available these days have allowed the technique to develop enormously, to the point that even the (successful) Rietveld refinement of small protein structures from synchrotron powder diffraction data is now possible (see Chapter 7.1[link] ). Another development is the extension of the Rietveld method towards parametric refinement on large numbers of complimentary data sets with various as-yet unexplored new applications. Rietveld refinement is so important it is described in detail in Chapter 4.7[link] , but we describe a number of important fundamentals of the method here by way of introduction.

The basic idea behind the Rietveld method is simple: Instead of extracting the integrated intensities of Bragg peaks and fitting models to these, as would be done in single-crystal and early powder diffraction studies, the full powder pattern, for example available as step-scanned intensity data, is fitted using a model whose parameters are refined using a least-squares procedure. The model parameters are varied in such a way as to minimize the sum of the squares of the difference between the n observed [Y{\rm obs}_i] and n calculated [Y{\rm calc}_i(\{p\})] step-scan intensities in the powder pattern, where the latter are calculated from a model containing a set of parameters {p}. The function that is minimized is usually the profile-weighted residual function, or R factor, given by[{R_w} = \textstyle\sum\limits_{i = 1}^{n} {{w_i}{{\big({Y{\rm obs}_i(2\theta) - Y{\rm calc}_i\left(2\theta\semi \{ p\}\right)} \big)}^2}} . \eqno(1.1.91)]The weight [{w_i}] is derived from the variance of the values of [Y{\rm obs}_i], while all covariances between different [Y{\rm obs}_i] values are assumed to be zero.

The calculated intensity [Y{\rm calc}_i] is expressed by combinations of mostly nonlinear and analytic or non-analytic functions as[\eqalignno{Y{\rm calc}_i &= \textstyle\sum\limits_{{ph} = 1}^{\rm phases} {\left({{S_{ph}}\textstyle\sum\limits_{hkl\left({ph} \right)} {\left({{K_{hkl\left({ph} \right)}}{{\left| {{F_{hkl\left({ph} \right)}}} \right|}^2}{\Phi _{hkl\left({ph} \right)}}\left({2{\theta _i} - 2{\theta _{hkl\left({ph} \right)}}} \right)} \right)} } \right)} &\cr &\quad+ {b_i}\left({\rm obs} \right). &(1.1.92)}]The outer sum runs over all phases ph present in the powder pattern, while the inner sum runs over all reflections hkl of a phase ph that contribute to the intensity at the position i in the powder pattern. A scaling factor [{S_{ph}}] is assigned to the reflection intensities for each phase; the scaling factor is proportional to the weight fraction of the phase. [{K_{hkl\left({ph} \right)}}] represents the product of various correction factors to the square of the structure-factor amplitudes, [{\left| {{F_{hkl\left({ph} \right)}}} \right|^2}], which may depend on the diffraction geometry and/or individual reflections. The value of the profile function [{\Phi _{hkl}}\left({2{\theta _i} - 2{\theta _{hkl}}} \right)] is given for the profile point [\left({2{\theta _i} - 2{\theta _{hkl}}} \right)] relative to the position of the Bragg reflection hkl. The observed background at position i in the powder pattern is denoted as [{b_i}\left({\rm obs} \right)]. Parameters in the model such as atomic positions, lattice parameters and experimental factors that affect peak shape and background are varied, using a least-squares approach, until the agreement between the calculated and measured diffraction profiles is optimized. In a least-squares approach, optimization consists of minimizing a cost function that is the weighted sum of the squared differences. This is a refinement method: a good initial guess at, or knowledge of, the structure is required and this model is refined by small adjustments.

This approach requires the modelling of the entire powder pattern. To simplify this complex task, the information content of the powder pattern can be divided into several parts (Fig. 1.1.1[link]), allowing the separation of groups of parameters with respect to their origin:

  • (a) the peak intensity [{\left| {{F_{hkl\left({ph} \right)}}} \right|^2}] – the time- and space-averaged crystal structure and geometrical contributions;

  • (b) the peak position – crystallographic lattice and symmetry, and instrumental contributions;

  • (c) the peak shape [{\Phi _{hkl}}\left({2{\theta _i} - 2{\theta _{hkl}}} \right)] – microstructural parameters and instrument profile;

  • (d) the background [{b_i}\left({\rm obs} \right)] – local structure and instrumental conditions.

Each part contains contributions from the sample and the instrument.

Rietveld refinement is a nonlinear least-squares process and requires starting values for all parameters. It is generally implemented with a local, rather than a global, optimizer and it is important for the starting parameters to be close to those of the actual solution to ensure that it is in the valley in parameter space that contains the global minimum. It is usual to guide the refinement into the (relatively narrow) range of convergence by hand by adding the parameters to the refinement sequentially. In this sense, Rietveld refinement takes some time to learn, but with care it can provide robust quantitative structures and a wealth of information can be extracted from the data.

Of course, there is no reason (other than computational efficiency) why the minimization algorithm could not be a more robust global optimizer, and this is now starting to be implemented in modern Rietveld codes. The most common and most easily implemented global optimizer, though one of the least efficient, is the Metropolis or simulated-annealing (SA) algorithm. The most usual implementation is actually as a `regional' optimizer where the updates to parameters such as atomic position are constrained to be not too far from the previous values in such a way that the algorithm makes a random walk through the parameter space. This algorithm can avoid being trapped in a local minimum by `walking uphill', since changes to the parameters that produce a worse agreement may be accepted with a probability based on the Boltzmann criterion, [\exp \left({ - {{\Delta R} /{kT}}} \right)]. The temperature in this expression is fictitious (i.e., it does not refer to any real temperature) and [\Delta R] is the change in the agreement produced by the trial update. The temperature plays the role of tuning the probability of accepting a bad move. It is initially chosen to have a high value, giving a high probability of escaping a minimum and allowing the algorithm to explore more of the parameter space. Later in the run the temperature is lowered, trapping the solution into successively finer valleys in the parameter space until it settles into (hopefully) the global minimum (Fig. 1.1.26[link]). The calculation of R can be based on the entire profile, or on integrated intensities. For the latter, the correlation between partially or fully overlapping reflections must be taken into account (as shown schematically in Fig. 1.1.25[link]).

[Figure 1.1.25]

Figure 1.1.25 | top | pdf |

Flow diagram of a simulated-annealing procedure used for structure determination from powder diffraction data (from Mittemeijer & Welzel, 2012[link]). F in the double sum is the structure factor from the structural model at each step of the optimization. Each sum runs over all reflections. h and k are summation indices representing hkl and hkl′, respectively.

A flow diagram of a typical SA algorithm as used for structure determination from powder diffraction data is shown in Fig. 1.1.25[link]. Parameters that can be varied during the SA runs include internal and external degrees of freedom like translations (fractional coordinates or rigid-body locations), rotations (Cartesian angles, Eulerian angles or quaternions, describing the orientation of molecular entities), torsion angles, fractional occupancies, displacement parameters etc. Fig. 1.1.26[link] shows the results of a typical simulated-annealing run in which the cost function, χ2, falls dramatically in the first few thousand moves, indicating that the scattering is dominated by the positioning of heavier atoms or globular molecules. Several million trial structures are usually generated before a minimum can be reached. At the end of the simulated-annealing run, Rietveld refinement is used to find the bottom of the global minimum valley.

[Figure 1.1.26]

Figure 1.1.26 | top | pdf |

χ2 (cost function) and `temperature' dependence of the number of moves during a simulated-annealing run. [From Mittemeijer & Welzel (2012[link]). Copyright Wiley-VCH Verlag GmbH & Co. KGaA. Reproduced with permission.]

Special algorithms are not usually used to prevent close contact of atoms or molecules during the global-optimization procedure, as in general these have not been found to be necessary, as the fit to the intensities alone quickly moves the molecules to regions of the unit cell where they do not grossly overlap with neighbouring molecules. A subsequent Rietveld refinement in which only the scale and overall displacement parameters are refined will immediately show whether further refinement of bond lengths and bond angles is necessary. Since unconstrained refinement often results in severe distortions from the ideal molecular geometry, either rigid bodies or soft constraints on bond lengths, the planarity of flat groups and bond angles can be used to stabilize the refinement. Another advantage of the simulated-annealing technique is that hydrogen atoms can often be included at calculated positions from the beginning if their relative position with respect to other atoms can be anticipated, which is often the case for molecular structures.

For inorganic crystal structures in particular, the identification of special positions or the merging of defined rigid bodies is useful during the final stages of structure solution. This can be accomplished by a so-called `occupancy-merge' procedure as proposed by Favre-Nicolin & Černý (2004[link]; see also Chapter 4.5[link] ). Here, the occupancies of the sites are modified as a function of the fractional coordinates, i.e. they are changed when the atoms get `too close' to a special position. The sites are thought of as spheres with a radius r. In this way any number of sites can be merged when their distances are less than 2r. As an example, the crystal structure solution of minium (Pb3O4) is shown in Fig. 1.1.27[link]. In this example, special positions are identified when two oxygen or lead atoms approach within a distance less than the sum of their respective merging radii, which is estimated as 0.7 Å. The occupancies of the sites then become: 1/(1 + intersection fractional volumes).

[Figure 1.1.27]

Figure 1.1.27 | top | pdf |

Screen shot (TOPAS 4.1; Bruker-AXS, 2007[link]) of a simulated-annealing run on Pb3O4 measured with a D8 advance diffractometer in Bragg–Brentano geometry. [From Mittemeijer & Welzel (2012[link]). Copyright Wiley-VCH Verlag GmbH & Co. KGaA. Reproduced with permission.]

The power of the Rietveld approach lies in its ability to extract the maximum information from the region of the data where peaks overlap. Since peak overlap is a significant problem even at moderate d-spacings, this method revolutionized powder diffraction to the point where the quantitative results are often trusted more than those coming from refinements of single-crystal data, since they are less sensitive to factors such as extinction that can affect single-crystal structure refinements. Single-crystal data are still preferred for structure solution, but Rietveld refinement is often the method of choice for obtaining the fine quantitative details of the structure after a solution has been found. However, the Rietveld method has also opened the door to using powder data for structure solution. In structure-solution methods, the structure factors are calculated from the intensities of all the available peaks, and algorithms are used to find the missing phases for each of these peaks and therefore the positions of the atoms in the unit cell. As mentioned above, full profile fitting following the Rietveld method can be carried out without a model, where the `parameters' are the Bragg-peak intensities themselves; this is known as Pawley or Le Bail refinement, depending on details of the approach used (see Chapter 3.5[link] ). This allows more accurate determination of the structure factors from Bragg peaks in regions where there is significant peak overlap.

These days, with high-quality data from synchrotron X-ray sources and excellent algorithms (either direct methods or global-optimization methods in direct space), determination of even quite complex crystal structures from powder diffraction data is becoming a routine method in almost all branches of natural sciences and engineering. The success rate mainly depends on three parameters: the choice of measurement device, how well the pattern profile is described and how good the structure-solving algorithm is. It is becoming increasingly evident that the use of highly monochromatic parallel-beam synchrotron radiation is a huge advantage for obtaining accuracy in the atomic parameters, which allows for the interpretation of bonding and reaction mechanisms. In some cases, even details like rotational disorder can be extracted from powder diffraction data if maximum-entropy methods are combined with high-resolution synchrotron data. Local structure refinement

| top | pdf |

As described in Section[link], similar full-profile-fitting strategies are now also carried out on total-scattering data that include diffuse-scattering intensity residing in what used to be considered as the `background'. This is either done by taking a structural model, which may be similar to the crystal model used in the Rietveld method (but the crystallographic symmetry of the model could also be reduced) or be a discrete cluster or molecule. As with the Rietveld method, structural parameters are varied in such as way as to obtain a good fit of the calculated function to the measured one. These methods go beyond the average structure and yield information about the local structure in the material, which may be different from the long-range ordered (LRO) crystal structure (or indeed there may be no LRO structure, as is the case in liquids and glasses). They are becoming more popular as data quality and computational power increase.

Solving the structures of nanoparticles from PDF data is less well developed, although it has been demonstrated for some simple structures such as C60 and simple inorganic crystalline compounds. We expect that this will grow in importance in the coming years, following the trend of the Rietveld method and structure solution from powders. Parametric Rietveld refinement

| top | pdf |

The conventional approach to analysing a set of powder patterns is to treat each powder pattern independently, thus refining the entire set of all relevant parameters for each pattern separately. Further analysis of the values of these parameters, for example fitting with empirical or physics-based functions such as fitting the temperature dependence of the ADPs with a Debye model, is then performed after the Rietveld refinements. Alternatively, all powder patterns can be subjected to refinement simultaneously, which allows the refinement of the functional dependence of external variables instead of deriving the parameters of the function from the individual Rietveld refinements afterwards. This so-called parametric or surface Rietveld refinement was first introduced by Stinton & Evans (2007[link]). Parametric refinement offers several advantages over the traditional sequential refinement approach because the correlation between parameters and the final standard uncertainty can be reduced by introducing simple and physically meaningful constraints and restraints. Furthermore, it is possible to refine noncrystallographic parameters such as rate constants or temperatures directly from Rietveld refinement (Stinton & Evans, 2007[link]). Of course, introducing external constraints in this way may introduce bias into the refinement if the constraint is not valid. For example, if there is anharmonicity in the motion and the temperature dependence of the ADPs does not follow the Debye law, carrying out a parametric refinement where the Debye law is presumed will result in biased refinements. However, with careful application, this is a potentially powerful approach to maximizing the quantitative information available from powder data in complex systems. In the following, the basic concept of parametric refinement is illustrated with several examples.

If we assume a set of pmax powder patterns from a single sample that have been measured as a function of the value of an external variable, e.g. time, temperature or pressure, equation (1.1.92[link]) can be formally written for each powder pattern separately:[\eqalignno{&Y{\rm calc}_{i,{\rm pattern(1)}} = {\rm function}(p_{1,{\rm pattern(1)}},p_{2,{\rm pattern(1)}},\ldots,{p_{m,{\rm pattern(1)}}}) \cr &Y{\rm calc}_{i,{\rm pattern(2)}} = {\rm function}(p_{1,{\rm pattern(2)}},p_{2,{\rm pattern(2)}},\ldots,p_{m,{\rm pattern(2)}})\cr &Y{\rm calc}_{i,{\rm pattern}(p_{\max})}&\cr &\quad= {\rm function}(p_{1,{\rm pattern}(p_{\max})},p_{2,{\rm pattern}(p_{\max })},\ldots,p_{m,{\rm pattern}(p_{\max})}).\cr &&(1.1.93)}]If a functional dependency of some of the parameters p on external variables T exists, these parameters may be expressed as functions of these variables, for example T. This functional relationship can be used to constrain together the p parameters for individual patterns measured at different temperatures, drastically reducing the number of global parameters. Equation (1.1.93[link]) can thus be written as[\eqalignno{&Y{\rm calc}_{i,{\rm pattern}(1)}&\cr& = {\rm function}\big(p_{1,{\rm pattern}(1)},p_{2,{\rm pattern}(1)} = f(T_1,T_2,\ldots,T_t), \ldots,p_{m,{\rm pattern}(1)}\big)\cr &Y{\rm calc}_{i,{\rm pattern}(2)} &\cr&= {\rm function}\big(p_{1,{\rm pattern}(2)},p_{2,{\rm pattern}(2)} = f(T_1,T_2,\ldots,T_t),\ldots,p_{m,{\rm pattern}(2)}\big)\cr &\vdots&\cr& Y{\rm calc}_{i,{\rm pattern}(p_{\max})} &\cr&= {\rm function}\big(p_{1,{\rm pattern}(p_{\max})},p_{2,{\rm pattern}(p_{\max})} = f(T_1,T_2,\ldots,T_t),&\cr&\quad\quad\ldots,p_{m,{\rm pattern}(p_{\max})}\big). &(1.1.94)}]The cost function (1.1.91[link]) to be minimized changes accordingly:[{R_w} = \textstyle\sum\limits_{{\rm pattern} = 1}^{{p_{\max }}} {\left({\textstyle\sum\limits_{i = 0}^{n - 1} {\left({{w_{i,{\rm pattern}}}{{\left({Y{\rm obs}_{i,{\rm pattern}} - Y{\rm calc}_{i,{\rm pattern}}} \right)}^2}} \right)} } \right)}. \eqno(1.1.95)]


Favre-Nicolin, V. & Černý, R. (2004). Fox: Modular approach to crystal structure determination from powder diffraction. Mater. Sci. Forum, 443–444, 35–38.Google Scholar
Rietveld, H. M. (1967). Line profiles of neutron powder-diffraction peaks for structure refinement. Acta Cryst. 22, 151–152.Google Scholar
Rietveld, H. M. (1969). A profile refinement method for nuclear and magnetic structures. J. Appl. Cryst. 2, 65–71.Google Scholar
Stinton, G. W. & Evans, J. S. O. (2007). Parametric Rietveld refinement. J. Appl. Cryst. 40, 87–95.Google Scholar

to end of page
to top of page