Embedded Cluster Expansion

The embedded cluster expansion (eCE) method is a modification to the classical cluster expansion method. It uses machine-learning tools and dimensionality-reduction techniques to enable an efficient modeling of multi-component systems.

Motivation

Let’s take a ternary system on a square lattice with species A, B, and C. An example of a configuration in this system is shown below.

_images/configuration_example_ABC.png — Illustrative arrangement of a ternary alloy on a square lattice with periodic boundaries.

In order to construct a configurational cluster expansion of this system, let’s use the occupational site basis functions.

_images/sitebasis_example_ABC.png — Occupational site basis functions for a ternary system composed of species A, B, and C.

This basis corresponds to counting B atoms for the site basis function \(\varphi_1\) and C atoms for the site basis function \(\varphi_2\). As a reminder of the documentation Cluster Expansion, the site basis function \(\varphi_0\) yields 1 for all species. Considering only nearest neighbor bonds, the cluster expansion reads as:

\[\begin{split}E(\sigma) &= J_0 + J_1 \langle \varphi_1 \rangle (\sigma) + J_2 \langle \varphi_2 \rangle (\sigma) + J_{11} \langle \varphi_1 \varphi_1 \rangle_{NN} (\sigma) + J_{12} \langle \varphi_1 \varphi_2 \rangle_{NN} (\sigma) + J_{22} \langle \varphi_2 \varphi_2 \rangle_{NN} (\sigma) \\ &= J_0 + J_1 x_B (\sigma) + J_2 x_C (\sigma) + J_{11} x_{BB} (\sigma) + J_{12} x_{BC} (\sigma) + J_{22} x_{CC} (\sigma)\end{split}\]

Here, \(x_{B}\), \(x_{C}\), \(x_{BB}\), \(x_{BC}\), and \(x_{CC}\) represent the content of B atoms, C atoms, B-B bonds, B-C bonds, and C-C bonds, respectively. Now, let’s take the situation where the species B and C are identical, e.g., they correspond isotopes atoms. There are at least two distinct ways to look at this situation:

Learn the similarity between the species through the ECI: Swapping a B atom with a C atom does not change the energy. The interaction strength (i.e., the ECI) associated with a B atom (\(J_1\)) or a C atom (\(J_2\)) must be identical. Similarly, the interaction strength (i.e., the ECI) associated with a B-B bond (\(J_{11}\)), a B-C atom (\(J_{12}\)), or a C-C bond (\(J_{22}\)) must be the same as well. The cluster expansion becomes:

\[E(\sigma) = J_0 + J_1 [x_B (\sigma) + x_C (\sigma)] + J_{11} [x_{BB} (\sigma) + x_{BC} (\sigma) + x_{CC} (\sigma)]\]
Learn the similarity between the species through the site basis functions: Another way to tackle this situation is to learn the similarity prior to the parameterization of the ECI. The idea is to find a set of site basis functions that translates the similarity relation that the atoms B and C behaves the same. A set of site basis functions satisfying the similarity relation is shown below:

Modified occupational site basis functions for a ternary system composed of species A, B, and C, with the similarity relation that the species B and C are the same.

This modified site basis function \(\tilde{\varphi}_1\) now corresponds to counting the occupation of both B and C atoms. Notice that the number of site basis functions is reduced. Only 2 functions are necessary instead of the 3 in the case above. This reduced number of site basis functions automatically leads to a reduced number of fitting parameters (ECI). The modified cluster expansion reads as:

\[E(\sigma) = \tilde{J}_0 + \tilde{J}_1 \tilde{x}_{(B,C)} (\sigma) + \tilde{J}_{11} \tilde{x}_{(B,C)(B,C)} (\sigma)\]

The second picture can be seen as projecting the 3 species in 3 dimensions onto 2 dimensions, or treating a ternary system as an pseudo-binary system. The fundamental idea behind the embedded cluster expansion is to treat a system with \(c\) species as a system with only \(k\) effective species, where \(k<c\), as depicted in the figure below.

_images/embedding_sitebasis_functions.png — Modified site basis functions in the general case where \(c\) species are embedded into \(k\) effective species

In reality, similarity relations are more complex and it is difficult to come up with the modified site basis functions by hand. The idea is rather to let the model learn these similarity relations by learning the coefficients of the site basis function matrix.

Formalism

Even if complete, the cluster expansion method faces difficulties in parameterizing the models as the number of fitting parameters polynomially grows with respect to the number of species present in the system. In order to reduce the explosion of correlation function, the embedded cluster expansion method consists in projecting the site-basis functions to a lower dimension. As a results, the number of effective species is decreased. Additionally, instead of fitting a linear model associated to the correlation function (ECI), a non-linear model replaces the ECI parameters.

Chemical embedding

The projection of the site-basis functions to a lower dimension relies on the chemical similarities between the different species present in the system. These chemical similarities are learnt during the training stage through exposure to data. It consists in finding the weights of a projection matrix. The dimension of the projected space (i.e., the number of embedded site-basis functions) is specified by the user.

Neural Network Energy Model

As mentioned above, a non-linear model is used to predict the the property (e.g., the energy) given a set of (embedded) correlation functions, rather than a linear model as in the classical cluster expansion method. The reason is to give more freedom to the model to capture the complex chemical interactions. Neural networks are typical such non-linear models. The non-linear model needs to take as input the correlation functions computed with the reduced set of embedded site-basis functions and return the property of interest (e.g., the energy).

_images/FNN.png — Illustration of a feed-forward neural network with 3 hidden layers used as energy model (effective cluster interaction model).

eCE Workflow

Given a configuration of an alloy containing \(N\) chemical species and a chemical embedding to \(\tilde{N}\) effective species, the eCE workflow reads as follows:

Transform each specie into \(N\)-dimensional vectors \(\varphi\) with entries corresponding to evaluated site-basis functions (e.g., using Chebychev basis).
Project the \(N\)-dimensional vectors \(\varphi\) into \(\tilde{N}\)-dimensional vectors \(\tilde{\varphi}\) by matrix multiplication with the embedding transformation matrix \(\mathcal{T}\).
Construct embedded correlation functions \(\tilde{\Theta}\) in the same way as in the classical cluster expansion method (as explained in Cluster Expansion), the only difference being the use of the \(\tilde{N}\) embedded site-basis functions as input instead of the \(N\) site-basis functions.
Evaluate the property of interest (e.g., the energy) using the neural network model taking the embedded correlation functions as input.

The training of the eCE model requires a set of data. It is performed by minimizing a loss function using gradient descent methods. Both the chemical embedding matrix and the ECI neural network model are simultaneously trained.

_images/eCE.png — Illustration of a embedded cluster expansion model.

A complete description of the eCE formalism can be found here.

Implementation

pyeCE is built on top of the PyTorch library. The implementation is focused on tensor operations as implemented in that library.

Input tensor

In order to compute the energy of a configuration, an input tensor needs to be constructed. This tensor records all neighbor atoms within its neighborborhood. The neighborhood specifies the range of interactions and is prebuilt within the the eCE object. The input tensor is constructed in the Config object.

It fully specifies the hamiltonian of the system. For each unitcell contained in any supercell, the model computes the embedded correlation functions \(\tilde{\Theta}\) for clusters that are contained within a prebuilt neighborhood.