%Chapter 3
Predicting disorder from XANES spectra requires a sophisticated model or algorithm capable of extracting non-linear features from the data. Increasing the disorder in the structure does not simply shift or scale the spectrum by a scalar; instead, disorder alters the spectrum in a complex and unknown way, for which we rely on machine learning  (ML) to discern. The general goal of ML is to recognize patterns in data, iteratively learning to solve complex, non-linear problems and make powerful predictions on new, unseen data \cite{ML-and-the-physical-sci}. Due to the integral role ML plays in this thesis, the following chapter serves to establish how neural networks fundamentally operate and define all the terms necessary for understanding the neural network implementations discussed in chapter \ref{ch:results}.

We begin by clarifying and distinguishing the following commonly misused terms: machine learning (ML), deep learning, and artificial intelligence (AI). ML is the most general term out of the three, referring to the computational technique of fitting a model on a dataset via an iterative training process. The models created in ML can either be regressors or classifiers: the former predicts a continuous range of values, whereas the latter is a discrete predictor. Artificial neural networks (ANNs), or neural networks (NNs) for short, are one example of a machine learning model that tends to be complex and computationally intensive to train. As a result, ANNs are often highly non-linear models capable of solving complex tasks such as object detection \cite{szegedy2013deep} or speech recognition \cite{ms-speech-recognition-paper} \cite{speech-recognition}. Neural networks were originlly inspired by biological nervous systems, and fittingly, their graphical representations include terms such as ``nodes'' and ``connections.'' The field of ML involving ANNs with many layers is referred to as deep learning \cite{schmidhuber2015deep}. AI is a subfield of deep learning where a neural network is trained to generate human-like responses. Common examples of AI are generative chatbots \cite{chatbots} and a virtual assistants \cite{virtual-assistants} \cite{virtual-assistants2}.

Using the above terminology, we can reframe the goal of this thesis: to predict disorder from XANES, we utilize deep learning to train a regression-based artificial neural network. The following sections walk through the mathematical process of training a simple neural network. In practice, sophisticated APIs such as Google's TensorFlow \cite{tensorflow2015-whitepaper} or Facebook's PyTorch \cite{pytorch-paper} handle the mathematical backend and optimization; however, one must first build a fundamental understanding of the methods these frameworks are executing before attempting to implement them.

\section{Feedforward and Backpropagation in ANNs}

% Understanding how a neural network makes a prediction requires a solid grasp of linear algebra.
The process where an ANN passes information from the input to the subsequent layers to make a prediction is called the ``feedforward process.'' The process of updating the parameters of a neural network is called backpropagation. Whereas feedforward is essentially a chain of linear algebra operations, backpropagation relies principally on vector calculus. Neither action is particularly mathematically complicated; however, there are many parts, and it is easy to get lost in the sea of similar-looking partial derivatives. In this next section (\ref{sec:feedfoward}), we explicitly walk through the mathematics of the feedforward process for a fully connected (affine) neural network.

\subsection{Feedforward} \label{sec:feedfoward}

Consider the neural network in Figure \ref{fig:simpleNN}: it contains an input layer with three nodes, a single hidden layer with five nodes, and an output layer with two nodes. The input layer (zeroth layer) has a cardinality of $ \mu=3 $ and is represented in Einstein notation\footnote{Recall that in Einstein notation repeated indices are implicitly summed over. For example, ${u^i = A^i_j x_j = \sum_{j=1}^5 A_{ij} x_j}$ } as the covector (row vector) $ x_\mu $. Each edge in the graph represents a weight that will be multiplied by the connecting node on its left in the feedforward process. First, each node in the input layer is multiplied by the weight of the connecting edge and added together. Applying this operation for all input nodes and weights can thus be represented as the inner (dot) product of the input layer row vector and a weights matrix. The hidden layer (first layer) has a cardinality of $ \nu=5 $. Thus, the resulting inner product is $ x_\mu W_\nu^{\mu(1)} $, where $ W_\nu^{\mu(1)} $ represents the matrix of weights connecting the zeroth and first layer. While this result has the correct dimensionality for the hidden layer, there are still two operations required to produce the actual values for the nodes $ h_\nu^{(1)} $. First, a small trainable parameter, $ b_\nu $  is added to every element in the resulting vector from the previous calculation. The values in this row vector are called biases and introduced to prevent overfitting---i.e. the phenomenon where a model predicts the training data well but does not generalize to reliably predict unseen data. Biases are a regularization parameter. Regularization techniques are discussed below, and an example is provided in Figure \ref{fig:overfitting}. The final operation applied to produce the first hidden layer's values is known as an activation function. These functions are applied element-wise to the layer, and without them, neural networks would be unable to learn non-linear features. There are several types of activation functions, the three most common being sigmoid, $\tanh$, and ReLU.

\subsubsection{Sigmoid and Tanh}
The sigmoid and tanh activation functions are defined as the following:
\begin{equation}
    \label{sigmoid}
    \sigma (x) = \frac{1}{1 + e^{-x}}
\end{equation}

\begin{equation}
    \label{eqn:tanh}
    \tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}
\end{equation}

\noindent Note, sometimes the term ``sigmoid'' is used to refer to the shape an s-shaped curve, so both equations (\ref{sigmoid}) and (\ref{eqn:tanh}) are considered sigmoids. In machine learning, however, the sigmoid function always refers to equation (\ref{sigmoid}). Note that the sigmoid function maps the input between zero and one. Hence, the it is often used in the final layer of ANNs to output a probability. Note that sigmoid asymptoically approach their minimum and maximum values (for sigmoid, 0 to 1; for tanh -1 to 1) around $ {x=-4} $ and $ {x=4} $ respectively, meaning that the sigmoid activation function is only useful within that limited range of input values. One issue with both the sigmoid and tanh activation functions is the potential for creating a vanishing gradient, the phenomenon where a small gradient from deeper network layers causes the gradients of earlier layers to also be near-zero due to the chain rule in backpropogation\footnote{An in-depth explanation of chaining gradients during backpropgation will be discussed in section \ref{sec:backprop}}. The gradient of either of these functions approaches zero for values above four. This asymptotic approach hurts the ability of the NN to meaningfully update its trainable parameters  \cite{nn-regularization}. The importance of calculating the gradients of these activation functions will be discussed in section \ref{sec:backprop} within the context of backpropagation.

\begin{figure}[h!]
    \centering
    \includegraphics[width=\linewidth]{Chapters/Figures/sigmoids2.pdf}
    \caption[Common Activation Functions]{Plotted above are three common activation functions: sigmoid, $\tanh$, and ReLU. Sigmoid and tanh are particularly useful for scaling the output of a neural network layer to be within a given range. ReLU and its variations are useful for deep ANNs, where vanishing gradients are problematic.}
    \label{fig:ActivationFunctions}
\end{figure}

\subsubsection{ReLU}
The \textbf{Re}ctified \textbf{L}inear \textbf{U}nit activation function (ReLU) has become an important staple of machine learning. Conventially, it is written as $ f(x) $ and defined as:
\begin{equation}
f(x)=
\begin{cases}
    0 & \text{if } x <= 0 \\
    x & \text{if } x > 0
\end{cases}
\end{equation}

\noindent ReLU is important because it provides a much greater range in values as outputs. Whereas sigmoid and $\tanh$ saturate around ${x=4}$, ReLU never saturates for linear values. Additionally, ReLU is simple to calculate and tends to help neural networks converge quickly. Further, because ReLU returns 0 for any negative value fed forward into the node, many ReLU activation functions in a given model help lead to sparser layers, reducing the overall complexity of the model and helping to prevent overfitting. Arguably its greatest benefit is the reduced likelihood of creating a vanishing gradient \cite{orig-relu}.

\begin{figure}[h]
    \centering
    \includegraphics[width=\linewidth]{Chapters/Figures/overfittingOrig2.pdf}
    \caption[Overfitting]{The green curve represents a model overfitting a binary classification problem. Although it makes near-perfect predictions for the training data in this figure, the model will not generalize as well as the simpler black curve when it tries to predict new, unseen data. Introducing biases and dropout layers in neural networks are strategies to prevent overfitting to reduce the model variance and fit the data like the black curve.}
    \label{fig:overfitting}
\end{figure}

With the inner product of the input nodes weight matrix calculated, the baises added, and then the activation function applied to each node, we arrive at the final final vector for the first hidden layer: $ h_\nu^{(1)} $. Mathematically, $ h_\nu^{(1)} = \sigma\left( x_\mu W_\nu ^\mu + b_\nu \right) $. To calculate the next layer, the process is now repeated---only $ h_\nu^{(1)} $ is used instead of the input layer, and the output
$ \hat{y} = \sigma \left( h_\nu^{(1)} W_\kappa ^\nu + b_\kappa \right)$ is the final output of the neural network. The equations for each step as well as the dimensionality of each layer can be found in Figure \ref{fig:simpleNN}.

\begin{figure}[h!]
    \centering
    \includegraphics[width=\linewidth]{Chapters/Figures/einstein_NN_2.pdf}
    \caption[Neural Network Example]{This diagram of a fully connected (affine) neural network has a single hidden layer and two output nodes. The tensors for each hidden layer are written in Einstein notation. The implicitly summed over greek letters and dimensionality are written below the tensors for clarification.}
    \label{fig:simpleNN}
\end{figure}

\subsection{Loss Metrics and Regularization}
In order to update the network parameters, it is necessary to evaluate the quality of every prediction the neural network makes. The measure of error for prediction is referred to as the \textit{loss}, whereas the summed total of all the losses is called the \textit{cost}. For regression problems, the two most common cost functions are the mean squared error and the mean absolute error \cite{regularization-2017survey}. Without regularization, they are defined as:

\begin{align}
    \label{eqn:lossFunctions-no-reg}
    \text{MSE} &= \frac{1}{n} \sum_i \left( \hat{y_i} - y_i \right)^2\\
    \text{MAE} &= \frac{1}{n} \sum_i \abs*{\hat{y_i} - y_i}
\end{align}

\noindent where $ n $ is the number of training samples. Either cost metric can be regulated. The two most common regularizations are L1 (LASSO) and L2 (Ridge). Applied to the MSE, equation (\ref{eqn:lossFunctions-no-reg}) with regularization becomes:

\begin{align}
    \label{eqn:lossFunctions}
    \text{L1 MSE: } J &= \frac{1}{n} \sum_i (\hat{y_i} - y_i)^2 + \lambda \sum_j \abs*{W_j}\\
    \text{L2 MSE: } J &= \frac{1}{n} \sum_i \left( \hat{y_i} - y_i \right)^2 + \lambda \sum_j (W_j)^2
\end{align}

\noindent where $ W_j $ is the $ j^{th} $ weight for training sample \textit{i}, and $\lambda$ is the regularization hyper-parameter.

One simple equation\footnote{This is a simple version of stochastic gradient descent, which will be discussed in depth in section \ref{sec:optimizers} } for updating the weights for loss L is as follows:
\begin{equation}
    % W := (1-\alpha \lambda)W - \frac{\partial L}{\partial W}
    W_j := W_j - \alpha \frac{\partial L}{\partial W_j}
\end{equation}

\noindent where $\alpha$ is the learning rate. L2 regularization is often referred to as weight decay. Every iteration, the weights are pushed closer to zero due to the multiplication of the weights by a value $<1$. L1 is known as LASSO (least absolute shrinkage and selection operator) because it shrinks the less important features' coefficients to zero. This is because for small values, $\abs*{W_i}$ is a much stiffer penalty than $(W_i)^2$. Thus, L1 is a good choice when there are dozens of features \cite{nn-regularization}.

% \noindent Explicitly, for one component of the hidden layer:

% \begin{align}
%     h_j &= \sigma(w_{1j}x_1 + w_{2j}x_2 + w_{3j}x_3 + w_{4j}x_4 + w_{5j}x_5 + b_j) \\
%         &= \sigma\Big(\sum_{i=1}^{i=5}w_{ij}x_{i} + b_j\Big)
% \end{align}

Including biases in the loss function is not the only way to regularize a model. Dropout layers are an entirely different method of regularization used exclusively for neural networks \cite{dropout-srivastava2014}. The idea is to introduce a hidden layer with a probability ``dropping out,'' i.e. ignored. Large weights in a neural network are indicative of a high-variance network, likely to be overfitting the data. By introducing layers with a probability of dropping out, the inward connections to the next layer change stochastically from batch to batch. The result of this behavior has the effect of adding noise to the network, similar to the inclusion of biases \cite{conv-dropout-layers} \cite{conv-dropout-layers2}.

\subsection{Backpropagation} \label{sec:backprop}
Backpropagation is the process of updating all the trainable parameters of the machine learning model, including weights, biases, and any other trainable parameters. The partial derivative requires repeated use of the chain rule. The output of the neural network in Figure \ref{fig:simpleNN} can be written as a functional:

% \begin{equation}
%     \hat{y} = \sigma\left( g \left( f() \right) \right)
% \end{equation}
% \noindent This would be a network with inputs $(x, y)$, two hidden layers ($ f $  and $ g $ ), and output with activation function sigma. Let's say the loss is the RMS loss. Thus,

\begin{equation}
    \label{eqn:functional}
    \hat{y} = \sigma \bigl( h_\nu ^{(1)} \left( x_\nu \right) \bigr)
\end{equation}

\noindent Here (\ref{eqn:functional}), the output layer $ \hat{y} $ is a function of the hidden layer $ h_\nu ^{(1)} $, which in turn is a function of the input layer $ x_\mu $. Consider the MSE cost without regularization:

\begin{align}
    \label{eqn:mse}
    J = \text{MSE} &= \frac{1}{n} \sum_i \left( \hat{y_i} - y_i \right)^2
\end{align}

\noindent Note that $ J $ is really $ J(\hat{y}) $, meaning that it is a function of the output functional (\ref{eqn:functional}). For well-behaved functions, such as (\ref{eqn:mse}), the derivative of a summation is equal to the summation of the derivatives of each term. To see how much to shift the weights, calculate the gradients for each layer. The first partial derivative is trivial:

\begin{equation}
    \frac{\partial J}{\partial J} = 1
\end{equation}

% \noindent For the next layer going backwards (the output layer):

% \begin{equation}
% \frac{\partial \hat{y}}{\partial J} = \frac{\partial J}{\partial J}\frac{\partial \hat{y}}{\partial J}
% \end{equation}

% \noindent For the next nested-function is the sigmoid, $ \sigma $. Taking its derivative:

% \begin{equation}
% \frac{\partial J}{\partial \sigma} = \frac{\partial J}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial \sigma}
% \end{equation}

% \noindent The next layer is the hidden layer before the activation function. For simplicity it will be referred to as $ g $ where $ g = x_\mu W_\nu ^\mu + b_\nu $.

% \begin{equation}
%     \frac{\partial J}{\partial g} = \frac{\partial J}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial \sigma} \frac{\partial \sigma}{\partial g}
% \end{equation}

% \noindent The next layer is the input layer, $ X_\mu $ has no trainable parameters, so the process for this network architecture is complete. If there were more hidden layers, the chaining rule would continue by multiplying the gradient calculated in the previous step by the gradient of the next layer with respect to the previous layer (towards the input layer).

% \begin{equation}
%     \frac{\partial L}{\partial f} = \frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial \sigma} \frac{\partial \sigma}{\partial g} \frac{\partial g}{\partial f}
% \end{equation}

% \subsection{Concrete Example}

% Finally, we will repeat the backpropogation process of updating the weights for the previous example as explicitly as possible. Consider again the functional $ \hat{y} $ in equation (\ref{eqn:functional}) and the L2 loss in equation (\ref{eqn:lossFunctions}), rewritten here as $ L $ . To determine how do shift the weights in the function, want to know $\frac{\partial J}{\partial W}$, where $ W  $ is short for $ W_\kappa ^{\nu (2)} $, the weight matrix connected to the second layer (the output layer).

% \begin{align}
% L = \frac{1}{m}\sum(\hat{y} - y)^2 \\
% \end{align}

% \noindent As before, the first partial derivative is trivial

% \begin{equation}
%     \frac{\partial J}{\partial J} = 1
% \end{equation}

\noindent and the next partial derivative is also straightforward\footnote{The astute may notice that if we instead chose MAE, we encounter a problem taking the derivative when $ {\hat{y}=y} $. Usually zero is returned instead or the MAE is approximated with a differentiable function. Otherwise using MAE is straightforward:
$
\dfrac{\partial J_\text{MAE}}{\partial \hat{y}} =
\begin{cases}
  +1,\quad \hat{y} > y\\
  -1,\quad \hat{y} < y
\end{cases}
$
}:

\begin{equation}
\frac{\partial J}{\partial \hat{y}} = \frac{2}{n}\sum_i (\hat{y}_i - y_i )
\end{equation}

\noindent Applying the chain rule yields:

\begin{equation}
\label{eqn:dldy}
\frac{\partial J}{\partial \hat{y}} = \frac{\partial J}{\partial J}\frac{\partial J}{\partial \hat{y}} \\
= 1 \cdot \frac{2}{n} \sum_i (\hat{y}_i - y_i )
\end{equation}

\noindent The next required term in the chain is the derivative of the loss with respect to the sigmoid:

\begin{equation}
\frac{\partial J}{\partial \sigma} = \frac{\partial J}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial \sigma}
\end{equation}

\noindent We already found the first term (\ref{eqn:dldy}), and the second term is trivial.

\begin{equation}
    \frac{\partial \hat{y}}{\partial \sigma} = 1 \\
    \implies \frac{\partial J}{\partial \sigma} =  \frac    {\partial J}{\partial \hat{y}} \frac{\partial \hat{y}}  {\partial \sigma} \\
    = \frac{2}{n} \sum_i (\hat{y_i} - y_i ) \cdot 1
\end{equation}

\noindent At this point, we have the derivative $ (\partial J / \partial \sigma) $ for the $ \sigma $ in the final layer $ \hat{y} = \sigma \left( h_\nu W_\kappa ^\nu + b_\kappa \right) $. The next derivative in the chain will be $( \partial J / \partial g )$ where $ g = h_\nu W_\kappa ^\nu + b_\kappa $. Continuing the chain,

\begin{equation}
\frac{\partial J}{\partial g} = \frac{\partial J}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial \sigma} \frac{\partial \sigma}{\partial g}
\end{equation}

\noindent where

\begin{align}
\sigma(g) &= \dfrac{1}{1 + e^{-g}} \\
\implies \frac{\partial \sigma}{\partial g} &= \sigma(g)(1 - \sigma(g))
\end{align}

\noindent Combining these previously calculated terms yields:

\begin{equation}
\frac{\partial J}{\partial g} = \frac{2}{n} \left( \sum_i (\hat{y}_i - y_i ) \right) \cdot 1 \cdot  \sigma(z)(1 - \sigma(z))
\end{equation}

Now comes the good part. Recall that the trainable parameters in the network are the weights $ W $ and biases $ b $. The next step is to calculate the gradients with respect to each of these parameters. This, in turn, will be used to update the parameter.

\begin{equation}
\frac{\partial J}{\partial W} = \frac{\partial J}{\partial g}\frac{\partial g}{\partial W} = \frac{\partial J}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial \sigma} \frac{\partial \sigma}{\partial g} \frac{\partial g}{\partial W}
\end{equation}

\noindent The last partial derivative in the chain is

\begin{equation}
\frac{\partial g}{\partial W} = W
\end{equation}

\noindent So,

\begin{equation}
\frac{\partial J}{\partial W} = \frac{2}{n} \left( \sum_i (\hat{y}_i - y_i ) \right) \cdot 1 \cdot  \sigma(z)(1 - \sigma(g)) \cdot W
\end{equation}

\noindent For the baises,

\begin{align}
\label{eqn:dLdW}
\frac{\partial J}{\partial b} &= \frac{\partial J}{\partial g} = \frac{\partial J}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial \sigma} \frac{\partial \sigma}{\partial g} \frac{\partial g}{\partial b} \\
\frac{\partial g}{\partial b} &= 1 \\
\implies \frac{\partial J}{\partial b} &= \frac{2}{n} \left( \sum_i (\hat{y}_i - y_i ) \right) \cdot 1 \cdot  \sigma(g)(1 - \sigma(g)) \cdot 1
\end{align}

Note that $ W $ is actually $ W_\kappa ^{\nu (2)} $, a matrix of weights, and b is actually $ b_\kappa $, a row vector of biases. Thus, the above equation (\ref{eqn:dLdW}) is just the partial derivative for one term in the weight matrix or bias covector. Repeating the process for each term in the matrix $ W $  and vector $ b $  yields the gradients $ \grad_w L $  and $ \grad_B L $, which represent the gradient of the loss function with respect to the weights and biases, respectively. This is the origin of the term, ``gradient descent,'' an optimization algorithm discussed in the next section. This was just the process to calculate the gradients need to update weights for the final layer, but one can see how continuing the process of chaining partial derivatives will yield the gradients for earlier layers in the network.

\section{Optimizers} \label{sec:optimizers}
Having calculated all the gradients via backpropagation, the weights and biases of the network can now be adjusted. The general idea of gradient descent relies on the fact that the gradient of any function points in the direction of the steepest increase. Thus, to optimize the network---which is equivalent to finding the parameters that minimize the value of the loss function---the weights and biases are updated by shifting their values in the opposite direction of the gradient of the loss function with respect to the weights, $ \grad_w L $. Gradient descent is the core principle of machine learning; this efficient algorithm for systematically updating model parameters made it possible to develop deep neural networks and train them with large datasets. Numerous improvements have been made to gradient descent since its inception \cite{cauchy-orig-grad-descent}, AdamW \cite{AdamW-orig} being the current state-of-the-art. In this section, we introduce several optimization algorithms to provide context for Adam, the optimizer used for training our model. In the previous section (\ref{sec:backprop}), the gradients of the weights $ W $ and biases $ b $ were written explicitly. For simplicity, the variable $ \theta $ is introduced to refer to either parameter. As before, $ J(\theta) $  is the cost function; it could be the MAE, MSE, or any other differentiable measurement of fit quality.

\subsubsection{Gradient Descent}
Vanilla gradient descent \cite{gradient-descent-rev-article} updates the parameters in the following way:

\begin{equation}
    \label{batch-grad-descent}
    \theta := \theta - \eta \cdot \grad_\theta J(\theta)
\end{equation}

\noindent Here (\ref{batch-grad-descent}), $ \eta $ is a \textit{hyperparameter} known as the learning rate. A hyperparameter is a user-defined parameter that must be chosen before the training process begins; it is not a trainable parameter. Hyperparameters can be ``tuned'' by repeating the entire training process with various hyperparameters set. Typically, one trains the model with a variety of hyperparameters over a short number of epochs. Once satisfied, the number of epochs is increased, and the training is repeated with the best-found hyperparameters. One limitation to gradient descent is the need for the entire cost $ J $ to be calculated. For large datasets, this can become impractical. Two common variants are batch gradient descent and stochastic gradient descent (SGD). In the former, the training set is divided into batches, and the gradient is updated after each batch. One iteration through all the batches is called an \textit{epoch}. In the latter, the gradient is calculated using the loss function instead of the cost function---i.e. the gradient is calculated, and the parameters are updated after each training sample. Both methods greatly reduced training time with the help of optimized, parallel computing \cite{stoch-grad-desc-parallel}. By updating the parameters after every training sample, SGD will move in the direction of the true gradient. The major limit to these methods, however, is the fixed learning rate \cite{grad-desc-limits}. If the learning rate $ \eta $ is too large, the algorithm will be unstable and ``bounce'' around the global minimum of the cost function. If $ \eta $ is too small, the algorithm will, at best, take a long time to train, and at worst, end up stuck in a local minimum.

\subsubsection{Stochastic Gradient Descent with Momentum}
Compared to regular SGD, stochastic gradient descent with momentum \cite{grad-desc-with-mom-orig} can greatly reduce the time to convergence. The general idea is to add a fraction of the previous parameter update to the current update. The exponential moving average (EMA) is an averaging of points within a period that puts greater weight on more recent points\footnote{In contrast a simple moving average treats each point as equally significant}. Here, $ S_t $ is the $ t^{th} $  value in the sequence S, and $ V_t $ is the $ t^{th} $  value in the new exponential moving averaged sequence, $ V $ .

% \begin{align}
%     % \label{eq:SGD}
%     & V_1 = \beta V_0 + \left(1 - \beta \right) S_1 \\
%     & V_2 = \beta V_1 + \left(1 - \beta \right) S_2 \\
%     & ...
% \end{align}
\begin{equation}
    \label{eq:SGD-w-momentum}
    V_t = \beta V_{t-1} + (1 - \beta)S_t
\end{equation}

\noindent where $ \beta \epsilon [0,1]$, a hyperparameter which partly defines how much weight the previous $ 1/(1-\beta) $ terms of S contribute\footnote{Typically 0.90 is a good starting point}. EMA's are common in market forecasting, so often $ S_t $  is the price at time $ t $. The continuous update for SGD with momentum is as follows:

\begin{align}
    V_t &= \beta V_{t-1}  + \left( 1 - \beta \right) \grad_\theta J \left( \theta \right)\\
    w &= W - \alpha V_t
\end{align}

Here, $ \alpha $  is the learning rate, as always. To be clear, $\grad_w L$ is the gradient of the loss function with respect to the weights. Note that the cost function $ J(\theta) $ may instead be the loss function $ L(\theta) $ if updates are performed after each training sample (i.e.~a batch size of one). SGD with momentum tends to perform better than SGD because it gives a closer estimate of the full gradient from the batch than SGD. Additionally, the momentum helps push the update through ravine-shaped local minima in the correct direction, whereas SGD tends to oscillate back and forth along the ravine's steeper dimension \cite{qian1999momentum}.

\subsubsection{Root Mean Squared Propagation}
Root Mean Squared Propagation (RMSprop)\footnote{RMSprop has an interesting history. It is an unpublished algorithm, first introduced by Geoff Hinton in an online series of lectures. Nevertheless, it is an incredibly popular algorithm and included in most ML platforms.} is another variant of SGD designed to improve convergence speed and remedy Adagrad's \cite{adagrad} tendency to rapidly diminishing gradients \cite{improving-rprop}\cite{2017marginal-adagrad}. The idea is to dampen oscillations in directions when the predictions are close to the cost function's minimum and accelerate movement when far away. In RMSprop, we keep a moving average of the squared gradients for each weight and use these to divide the learning rate by an exponentially decaying average. As before, $\grad_\theta J$ is the gradient of the cost with respect to weights.

\begin{align}
    \label{eqn:RMS_Prop}
    & S_{k+1} = \beta S_k + (1 - \beta)(\grad_\theta J \cdot \grad_\theta J) \\
    & \theta_{k+1} = \theta_k - \alpha \frac{\grad_\theta J}{\sqrt{S_{k+1}} + \epsilon}
\end{align}

\noindent The hyperparameter $ \epsilon $ is included in the denominator to prevent a possible division by zero as well as provide more stability.\footnote{Typical values for $ \alpha $  and $ \beta $  are 0.001 and 0.9, respectively.}

\subsubsection{Adaptive Moment Estimator}
Adaptive Moment Estimator (Adam) is a combination of RMSprop and SGD with momentum. Adam uses the squared gradients to scale the learning rate for each parameter (similar to RMS prop), and it uses a moving average of the gradient (similar to SGD with momentum).

\begin{align}
    \begin{split}
    m_t &= \beta_1 m_{t-1} + (1 - \beta_1) (\grad_\theta J) \\
    v_t &= \beta_2 v_{t-1} + (1 - \beta_2) (\grad_\theta J \cdot \grad_\theta J)
    \end{split}
\end{align}

\noindent The new parameters in this algorithm, $ m_{t-1} $ and $ v_{t-1} $ are the first and second moments of the gradient (the mean and variance), respectively. Adam's 80,000 citations in the six years since its publication gives some indication of the importance and power of this algorithm \cite{orig-ADAM-paper}. This was the chosen algorithm for training our neural network for predicting disorder in XANES.

\section{Normalization} \label{sec:normalization}
Normalization is an important step for any machine learning algorithm. There are three types of normalization utilized in our training process: feature normalization, label normalization, and batch normalization. Without feature normalization, a model will put greater weight on features with larger values. For example, if a model is trained to predict surface stress of a silica bead in silicone gel given the diameter of the bead in microns and the adhesion energy in $ \text{mNm}^{-1} $, the model would learn to ignore the bead's size in its predictions. This is because the particle's size is $ 1000\times $ smaller than the adhesion energy \cite{williamsThesis}. To correct this scaling issue each feature is ``normalized'' on the training data to be on the same scale. Often a z-score is used to center the features around a mean of zero with a standard deviation of one. This is known as standardizing or applying a standard-scalar \cite{statsTextbook}.

\begin{equation}
    \label{z-score}
    Z_{norm}^{(i)} = \dfrac{x_i-\mu}{\sqrt{\sigma^2-\epsilon}}
\end{equation}

\noindent In our neural network, the training features are normalized in this way.

For the same reasons feature normalization is important, the training labels must also be normalized. Instead of using the standard scalar, the labels are normalized using min-max normalization, which normalizes the values between zero and one. This is useful when the labels are known to be evenly distributed over a range. To scale the $ i^{th} $ label (y) via min-max scaling:

\begin{equation}
    Z_{norm}^{(i)} = \dfrac{x_i - \text{Min}(y)}{\text{Max}(y) - \text{Min}(y)}
    \label{eqn:min-max-scaler}
\end{equation}

Scaling the training features means the neural network will predict the scaled values. The prediction can be ``un-scaled'' to retrieve the real, predicted values via:

\begin{equation}
    y^{(i)} = - \dfrac{Z_{norm}^{(i)}\left(\text{Max}(y) - \text{Min}(y)\right)}{\text{Min(y)}}
\end{equation}

\noindent It is important that the scaling parameters $ \mu,~\sigma,~\text{Min}(y)$ and $\text{Max}(y) $ come from the training set---not the testing or validation set. Using the values from the entirety of data constitutes a form of \textit{data leakage}, where the training process is given a hint of the validation or test data. It is important that the model never sees the testing or validation until testing or validation time; otherwise, the model is unlikely to generalize as well to unseen data as one might expect given a loss curve.

Batch normalization is a technique designed to make NN's more robust to internal covariate shift \cite{batch-norm-orig}. Covariate shift refers to a systematic difference between the training and validation data, or in the context of a training batch, a systematic shift in the distribution of data from batch to another \cite{batch-norm-conference}. For example, if a NN is trained for binary classification to predict whether or not an image includes a cat---and the network is trained on images of only black cats---the network is unlikely to make a correct prediction when it encounters an image of an orange cat.

The idea of batch normalization is to normalize each hidden layer similar to how training data or labels are normalized; however, whereas normalizing the training data centers the dataset or labels using fixed parameters such as the mean and variance, in batch normalization the mean and variance of the batch normalization are learnable parameters. In batch normalization, the values for a hidden layer are scaled via:

\begin{equation}
\widetilde{Z}_i = \gamma Z_{norm}^{(i)} - \beta
\end{equation}

\noindent Notice that if $\sqrt{\sigma^2 + \epsilon}$ and $\gamma = \mu$, we get equation (\ref{z-score}), i.e. the hidden layer is normalized in the same way as the input layer. This is generally not useful, however, because normalizing all parameters to be centered around zero causes the sigmoid-like activation functions to be mostly focused on the linear regime.

% The general structure of implementing batch normalization looks something like this:
% \begin{align}
% \text{first pass: }& x \cdot \theta^T \rightarrow z \\
% \text{batch normalize: }& z \rightarrow \widetilde{z} \\
% \text{apply activation function: }& g(\widetilde{z}) = a \\
% \text{second pass: }& a \cdot \theta^T
% \end{align}

% Note, this is for one batch, so $x$ is the $i^{th}$ batch.

The output of the batch normalization is passed forward to the next hidden layer of the network, while the normalized input is retained in the current layer. Normalizing the hidden layers for each batch means that later hidden layers do not have to adapt as much to the earlier hidden layers. Consequently, this allows the deeper layers to do a better job tuning themselves a little more independently of the other layers, improving performance and speeding up the learning process. Note, because each batch is scaled $(z \rightarrow \widetilde{z})$, a small amount of noise is added, which acts as a slight form of regularization\footnote{Note, while batch-norm adds regularization, this is not intended to be used as a form of regularization. L1, L2 regularization or dropout layers should be used instead.}. Recently, several papers  \cite{batch-norm-conference} \cite{whybatchnorm2} \cite{whybatchnorm3} have been published disputing the reason batch normalization improves the model performance; none, however, dispute its efficacy. Data augmentation may be vital in building the neural network trained on simulation data to predict experimental data, for which there is a sparsity of data for training.

\section{Data Sparsity}
Unfortunately, not all project goals include a plethora of diverse training data. This section introduces two techniques for dealing with training data: data augmentation and transfer learning. Data augmentation can be a valuable technique even with an abundance of training data. Transfer learning, on the other hand, aims to solve a complex problem with little data by first training the model on an easier problem with ample data.

\subsection{Data Augmentation}
Data augmentation is a technique for expanding the size and variance of the training data for machine learning purposes. It has been critically important for developing powerful deep neural networks, particularly in the domain of image processing \cite{data-augmentation1}\cite{data-augmentation2} \cite{data-augmentation3} \cite{data-augmentation4}. Consider the example in section \ref{sec:normalization} of a cat-vs-not-cat binary classifier. The idea of data augmentation is to take the dataset containing only images of the black cats and add new images created from the original dataset. Common methods of data augmentation are image cropping, image rotation, and introducing image filters that alter the color, sharpness, or contrast. In the case of the black-cats-only dataset, color filtering may train the network to become color agnostic and correctly classify an image of an orange cat without ever having seen one. For signal processing, including absorption spectroscopy, common methods for expanding the training data size include the introduction of Gaussian noise and shifting the spectra horizontally \cite{data-augmentation5}. By artificially expanding the size and complexity of training data, data augmentation helps prevent models from overfitting.

\subsection{Transfer Learning}
Transfer learning was first introduced in 1976 to \cite{transferlearning-reminder} \cite{transferlearning2} \cite{transferlearning3}. The idea is to alter a model trained on one task to be able to solve a new but similar task. One famous example involves a neural network originally trained to classify pastries, which, utilizing transfer learning, was re-purposed for detecting cancer cells \cite{ny-pastry-article}. Consider a model first trained on dataset $ A $ with the final goal of predicting dataset $ B $. Note that $ A $ and $ B $ in this example are not a train-test split but, instead, inherently different problems. By pre-training the model on $ A $ to solve a similar task with $ B $, the model learns inductive biases which encourage the model's parameters $ \theta_B $ to be similar to $ \theta_A $. This may have the effect of training the model to learn low-level features that may not have been learned from $ B $ alone  \cite{transferBook}. Transfer learning using simulations as training data is a cutting-edge topic of research in ML and particularly in AI. Modern autonomous driving systems rely on testing and training their models using driving simulations to bolster their practice time beyond what would be possible from real-world driving tests alone  \cite{mit-self-driving-car-simulations}. A significant time has been invested in creating platforms specifically for developing autonomous driving systems \cite{carla-dosovitskiy2017}.

Applying transfer learning to incredibly sparse datasets---attempting to teach the model from just a few examples---is called few-shot learning\footnote{Other common terms are ``one-shot''  \cite{one-shot-learning} and ``zero-shot'' \cite{zero-shot-learning} learning. These both refer to the same concept but with only one or even zero training examples, respectively} \cite{few-shot-learning}. This is a very hot area of modern research, as there are many instances in which real-world data is incredibly expensive to acquire, but simulating or obtaining similar data is possible.
In the context of this thesis, transfer learning will be an important tool for creating a neural network capable of predicting disorder from an experimental XANES spectrum. Because of the sparsity of experimental data, it is unfeasible to train the neural network on experimental data alone. Instead, we rely on simulated XANES spectra created with FEFF. The systematic differences between the simulated XANES spectra and the experimental counterparts, however, suggest a neural network trained purely off simulation spectra will not perform well when it encounters an experimental spectrum for the first time. Applying the principles of transfer learning: the neural network can first be trained with the simulated XANES spectra, for which there are ample examples. Then, using the limited number of experimental data---included extra examples created via data augmentation---the network can be trained again via transfer learning.

\section{Covolutional Neural Networks}
Convolution is a mathematical operation for combining two functions, the result of which is a third function revealing the effect of the second function on the first \cite{Boas-mathmethods}. The convolution of two continuous functions, $ f $ and $ g $, is a special type of integral transformation. The result is the integral of the product of $ f $ and the shifted inverse of $ g $, where $ f $ can be thought of as the input function and $ g $ is often referred to as the kernel.

\begin{equation}
    f \otimes g = \int_{-\infty}^{\infty} f(j)g(i-j) \,dj
\end{equation}

\noindent The variable $ i $ (no relation to the imaginary number) is represents the weighted-shift in the function $ g(\tau) $. Different values of $ i $ emphasize different parts of the other function, $ f(\tau) $.

In computer science, convolutions are an important and powerful tool for signal and image processing \cite{1dconv-NN-survey} \cite{deepCNNforImages}. Because images and signals---which can be thought of as a 1D image---are comprised of a discrete number of points (e.g.~pixels), a modified formula is required to perform the convolution. The convolution of the signal $ f $ with the kernel $ g $ can be written as \cite{cornell-convs}:

\begin{equation}
    \label{eqn:cs-1dconv}
    f \otimes g = \sum_{j=1}^m g(j) \cdot  f(i-j+m/2)
\end{equation}

\noindent Here (\ref{eqn:cs-1dconv}), $ m $ is the length of the kernel $ g $, and $ i $ and $ j $ are hyperparameters. To demonstrate visually, consider a simple, example absorption spectrum with only 10 data points (\ref{fig:conv-ex-spectrum}).

\begin{figure}[h!]
    \centering
    \includegraphics[width=.75\linewidth]{Chapters/Figures/conv-example.pdf}
    \caption[Toy Absorption Spectrum]{A simple abosrption spectrum for demonstration purposes.}
    \label{fig:conv-ex-spectrum}
\end{figure}

\noindent Each data point, $ (E, \mu) $ in the spectrum is described as a feature vector, where the feature is the energy value for a given point $ (E, \mu) $. The vector, $ f $,  is depicted below with boxes to represent each element in the vector. Zeros are padded on both sides for reasons that will become clear soon.

\begin{table}[h!]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c|c|}
    \cline{2-13}
    \textit{f} = & 0 & .09 & .10 & .09 & .70 & .80 & .68 & .60 & .62 & .63 & .60 & 0 \\ \cline{2-13}
    \end{tabular}
\end{table}

\noindent Additionally, consider the kernel, $ g $

\begin{table}[h!]
\centering
    \begin{tabular}{lccc}
    \cline{2-4}
    \multicolumn{1}{l|}{\textit{g} =} & \multicolumn{1}{c|}{.1} & \multicolumn{1}{c|}{.1} & \multicolumn{1}{c|}{.1} \\ \cline{2-4}
    \end{tabular}
\end{table}

\noindent The convolution works by multiplying each element in the input vector $ f $ by the corresponding element in the kernel $ g $ and summing the results. The kernel then moves to be centered around the next element in $ f $. One way to think about this process is a kernel or filter sliding over an input signal. Applying the kernel $ g $ onto the first index of $ f $ yields:

\begin{table}[h!]
    \centering
    \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|}
    \hline
    0  & .09 & .10 & .09 & .70 & \multicolumn{1}{c|}{.80} & \multicolumn{1}{c|}{.68} & \multicolumn{1}{c|}{.60} & .62 & .63 & .60 & 0 \\ \hline
    .1 & .1  & .1  &     &     &                          &                          &                          &     &     &     &   \\ \hline
    \end{tabular}
\end{table}
$$
h(1) = (0)(.1) + (.09)(.1) + (.10)(.1) = 0.019
$$

\noindent where $ h(1) $ is 1st index of the resulting vector. For the next point, the kernel shifts to be centered around it.

\begin{table}[h!]
    \centering
    \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|}
    \hline
    0 & .09 & .10 & .09 & .70 & \multicolumn{1}{c|}{.80} & \multicolumn{1}{c|}{.68} & \multicolumn{1}{c|}{.60} & .62 & .63 & .60 & 0 \\ \hline
      & .1  & .1  & .1  &     &                          &                          &                          &     &     &     &   \\ \hline
    \end{tabular}
\end{table}

$$
h(2) = (.09)(.1) + (.10)(.1) + (.09)(.1) = 0.028
$$

\noindent The final resulting vector is:

\begin{table}[h!]
    \centering
    \begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|}
    \cline{2-11}
    \textit{h} = & .019 & .028 & .089 & .159 & .218 & .208 & .190 & .185 & .185 & .123 \\ \cline{2-11}
    \end{tabular}
\end{table}

\begin{figure}[h!]
    \label{fig:conv-res-spectrum}
    \centering
    \includegraphics[width=.75\linewidth]{Chapters/Figures/conv-example-res.pdf}
    \caption[1D Convolution Result]{The result of the 1D convolution of kernel $ g = (.1, .1, .1) $  on the spectrum in Figure \ref{fig:conv-ex-spectrum} is plotted above.}
\end{figure}

The toy example was chosen to demonstrate the basics of a 1D convolution. In this example, the input spectrum was a vector of length ten and the kernel of length three. In the context of applying a 1D convolution to a neural network, the size of the input vector is the cardinality of the hidden layer directly preceding the convolutional layer. Often the hidden layer's output, represented as a vector, is reshaped into an n-dimensional tensor before applying the convolution. To apply a 1D-convolution to a Tensor of rank $ n $, simply apply the convolution to each of the n-vectors separately, ensuring to pad the ends of each vector with zeros. Layers with dimensionality greater than one can be ``un-raveled''. Alternatively, the layers of each dimension can be truncated or ``pooled'' by averaging the layers that are stacked on one another (or taking the max of each layer) until the desired dimensionality is achieved. A common way to achieve this is with a max-pooling layer or an average pooling layer. Max pooling and average pooling layers also have the additional benefit of downsampling the feature space, tending to make the network more robust to slight variations in the position of features in the input image or signal. This is referred to as ``local translation invariance'' \cite{local-translation-invariance}.

Another important possible change in the above example is the stride length. In the above example, we ``slid'' the kernel across the input spectrum one point at a time. This is referred to as a stride size or stride length of one. The stride could be any integer less than the length of the input vector $ f $, though in practice, typically stride lengths tend to be one or close to one. Lastly, in the above example, we only applied one convolutional kernel, or ``filter.'' In practice, many filters are applied sequentially. In the above example, the filter $ g = (.1, .1, .1) $ was simply decided upon \textit{a priori}. In practice, the values for each filter in the convolutional layer are initialized randomly or according to the specified initialization function. The default in Keras is Glorot Uniform. The values of each filter are trainable parameters that evolve to produce the best final prediction given the subsequent layers in the neural network.

\section{How to Train a Neural Network}
Building a successful neural network requires a combination of intuition and procedural know-how. The first key is to start with a simple model, perhaps a single hidden. \textit{If you start with a complex model including data augmentation and regularization, you will never be able to tune the hyperparameters and find a good solution}. A good strategy is to pick a simple architecture and reasonable hyperparameters and train the model on a small subset of training samples, say 1--10 samples. Then, train the model over ten or so epochs and see if the training cost decreases and whether you can overfit it. If you can overfit the small sample size, it means the code is working, and the network architecture makes sense. Now is the time to increase the number of samples in the training data, either to the full train-test split or a subset of if working with ``big data.''\footnote{Big data has become a nebulous term, but a reasonable example is when there is so much data that the dataset cannot be loaded into the computer's memory (RAM) at the same time.} Next, you can run a broad hyperparameter search. Afterward, run maybe 20 epochs and see how the training and validation loss is moving. If they both are going down, the architecture looks good. If not, start over. Ideally, the training and validation loss curves will be decreasing together like in figure \ref{fig:Example-Training-Loss-Curve}. If the model still does not predict well after hyperparameter tuning, the model is underfitting, and a more complex architecture is required (add more layers).

\begin{figure}
    \includegraphics[width=\linewidth]{Chapters/Figures/epoch_loss1.pdf}
    \caption[Example ANN Training Curve]{In this example loss curve, the x-axis is the epoch, and the y-axis is the mean squared error of the predictions. The red curve is the training data, and the blue curve is the validation set. Notice how both curves decrease together, but the training curve has a smaller error than the validation curve. This is the expected behavior of a model that is not overfitting and training properly.}
    \label{fig:Example-Training-Loss-Curve}
\end{figure}

Ideally, the training and validation loss will decrease together over many epochs. It is more likely, however, that after many epochs, the training loss will continue to decrease while the validation loss plateaus. This is the point to start introducing regularization such as dropout layers as well and considering data augmentation. These techniques will allow the model to continue decreasing the validation loss and prevent overfitting. The main goal of training is to minimize the validation loss. The placement of dropout layers and the type of data augmentation are subject to trial and error. Generally, it is best to include dropout layers after a ReLU activation and before an affine layer. There has been some research suggesting the inclusion of low-probability dropout layers after convolutional layers tends to improve model performance \cite{conv-dropout-layers} \cite{conv-dropout-layers2}, but these rules do not work for every network, and it is still worth trying many options while training.

% \section{Autoencoders if they become useful}
% Talk about how autoencoders work. Give a nice broad explanation and really go into the math. Include some nice diagrams

% Here's \cite{ng2011sparse} a good source to read and model off of. Here \cite{Bhowick2019} is another paper that might be interesting to read. It's about getting noise-free data from the original data using an autoencoder. Neat idea, and could actually be very relevant because they're using geophysical data.


\section{Gradients}\label{derivs}
\index{Gradients|ff}\index{Derivatives|(}
By ``gradients'' we generally mean ``the derivative of the energy with respect
to coordinates''.  The two most commonly used gradients are with respect to
Cartesian coordinates, in which case the units are kcal/mol/\AA ngstrom, or
with respect to internal coordinates, in which case the units are either
kcal/mol/\AA ngstrom or kcal/mol/radian, depending on whether the coordinate is
a distance (in which case it would be kcal/mol/\AA ngstrom) or an angle or
dihedral (in which case it would be kcal/mol/radian). The particular gradient
actually being used at any given point should be clear from the context. In all
cases, the gradient can be regarded as the following derivative
$$
g_i = \frac{d(\Delta H_f)}{dx_i}
$$
In discussion ``gradient'' will be reserved for the derivative with respect to
coordinates flagged for optimization (internal or Cartesian), and
``derivative'' will be used for both gradients and terms which are used to
calculate gradients, such as Cartesian derivatives which are used to calculate
internal coordinate gradients.

There are four very different ways to calculate gradients, although all four
result in the same type of derivative.  The four ways are:
\begin{description}
\item[Frozen density matrix finite difference derivatives]~\\
\index{Wavefunctions!variational}
In these procedures, once an SCF has been achieved, the derivatives can be
calculated using the density matrix from the SCF calculation.  These methods
can only be used with variationally optimized wavefunctions.

By default, the derivatives are worked out by calculating the energy of each
pair of atoms, then re-calculating the energy after a small displacement has
been made, and then calculating the derivative from the differences in the
energies and the step.  This is the default, and is the fastest.  If this
method is {\em not} wanted, specify \comp{ANALYT}.

\item[Analytical derivatives, using frozen density matrix approximation]~\\
Not as fast as the first method, but more accurate.  Useful when finite
difference derivatives are suspected to be of insufficient accuracy.  When
analytical derivatives are wanted, specify \comp{ANALYT}.  Analytical
derivatives cannot be used with non-variational finite difference derivatives.

\item[Non-variational finite difference derivatives]~\\
For non-variational \index{Wavefunctions!non-variational}wavefunctions (systems
for which the electronic energy is modified after the SCF calculation is done,
e.g.\ C.I.\ calculations), a  sophisticated derivative routine in \comp{DERNVO}
calculates the effect on the  derivative of the post-SCF energy terms.  This
method is used automatically in RHF C.I.\ calculations.  If this method is {\em
not} wanted, specify \comp{NOANCI}.

\item[Brute force gradients]~\\
These should be avoided whenever possible.  To calculate the gradient, a small
change is made in the desired coordinate, then a full SCF is done, and the
gradient calculated from
$$
g_i = \frac{\Delta H_f-\Delta H_f'}{x-x^{'}}.
$$
These gradients are very slow, and are of poor accuracy, but sometimes they
are the only way to obtain  gradients.  These derivatives cannot be used
with variationally optimized wavefunctions, but can be used with
non-variational wavefunctions by specifying \comp{NOANCI}.
\end{description}

Note that \comp{ANALYT} and \comp{NOANCI} apply to two very different things:
\comp{ANALYT} applies to the derivatives using a frozen density matrix
approximation, and uses true analytical methods.  \comp{NOANCI} prevents
Liotard's C.I.\ derivative method being used.  Of course \comp{NOANCI} has no
meaning for variationally optimized wavefunctions.

\subsection{Frozen density matrix finite difference derivatives}

The first step in calculating the gradients is to calculate the derivatives
with respect to Cartesian coordinates.  This is done in subroutine DCART.
\index{DCART}

DCART calculates the energy of each pair of atoms, then moves one atom a small
distance ($10^{-4}$\AA ) in each of the three Cartesian directions.  The
density matrix for the atom-pair is not changed during this calculation, but is
set equal to the SCF density matrix. The derivative for each atom is then
calculated  from:
$$\left (\frac{dE}{dx}\right )_A = \sum_{B\neq A}\frac{E_{AB}-E_{AB}^{'}}{\delta_x}, $$
$$\left (\frac{dE}{dy}\right )_A = \sum_{B\neq A}\frac{E_{AB}-E_{AB}^{'}}{\delta_y}, $$
$$\left (\frac{dE}{dz}\right )_A = \sum_{B\neq A}\frac{E_{AB}-E_{AB}^{'}}{\delta_z}. $$
where $E_{AB}^{'}$ is the energy of the pair of atoms after displacement in the
appropriate direction.  For a stationary point, these derivatives are zero.

To convert from Cartesian coordinate (c.c.) derivatives into gradients (i.c.),
the sum
$$
g_i = \sum_j\frac{dE}{d({\rm c.c.}_j)}\frac{d({\rm c.c.}_i)}{d({\rm i.c.}_j)}
$$
must be evaluated.  Evaluation of
$\frac{d({\rm c.c.}_i)}{d({\rm i.c.}_j)}$
is quite simple, and in done in routine JCARIN.

\subsection{Hessian matrix in \comp{FORCE} calculations}\index{Hessian|(}\label{ssd}
\index{Single-sided derivatives}
The Hessian matrix is the matrix of second derivatives of the energy with
respect to geometry. The most important Hessian is that used in the
\comp{FORCE} calculation.  Normal modes are expressed as Cartesian
displacements, consequently the Hessian is based on Cartesian rather than
internal coordinates.

\index{Derivatives!``single-sided''}
Although first derivatives are relatively easy to calculate, second derivatives
are not.  The simplest, although not an elegant, way to calculate~\cite{pulayf}
second derivatives is to calculate first derivatives for a given geometry, then
perturb the geometry, do an SCF calculation on the new geometry, and
re-calculate the derivatives. The second derivatives can then be calculated
from the difference of the two first derivatives divided by the step size.
This method, which is used in the EigenFollowing routine, is called
`single-sided' derivatives.

The Hessian is quite sensitive to geometry, and should only be evaluated at
stationary points.  Because of this sensitivity, ``double-sided'' derivatives
\index{Derivatives!``double-sided''} are used:
$$
H_{i,j} = \frac{g_i^{+\delta_j}-g_i^{-\delta_j}}{2\delta}.
$$
Note the asymmetry in the treatment of the Cartesian coordinates $i$ and $j$.
It can be shown that
$$
\frac{g_j^{+\delta_i}-g_j^{-\delta_i}}{2\delta} = \frac{g_i^{+\delta_j}-g_i^{-\delta_j}}{2\delta}.
$$
To help improve precision, the Hessian is calculated from
$$
H_{i,j} = \frac{1}{2}\left ( \frac{g_j^{+\delta_i}-g_j^{-\delta_i}}{2\delta} + \frac{g_i^{+\delta_j}-g_i^{-\delta_j}}{2\delta} \right ).
$$
\index{Hessian|)}
\index{Derivatives|)}


\section{Introduction}\label{sec-intro}

% , further referred to simply as \emph{effects}

\emph{Monads}, introduced to functional programming
by~\citet{1995_wadler_monads}, are a powerful and general approach for
describing effectful (or impure) computations using pure functions. The key
ingredient of the monad abstraction is the \emph{bind} operator, denoted by
\hs{>>=} in Haskell\footnote{We use Haskell throughout this paper, but the
presented results are not specific to Haskell and do not require any advanced
features of the Glasgow Haskell Compiler (any Haskell98~\citep{haskell98}
compliant compiler will do). Furthermore, we release two libraries for selective
applicative functors along with this paper: for Haskell and OCaml.}:

\vspace{1mm}
\begin{minted}[xleftmargin=10pt]{haskell}
(>>=) :: Monad f => f a -> (a -> f b) -> f b
\end{minted}
\vspace{1mm}

\noindent
The operator takes two arguments: an effectful computation \hs{f}~\hs{a}, which
yields a value of type~\hs{a} when executed, and a recipe, i.e. a pure function
of type \hs{a}~\hs{->}~\hs{f}~\hs{b}, for turning~\hs{a} into a subsequent
computation of type \hs{f}~\hs{b}. This approach to composing effectful
computations is inherently sequential: until we execute the effects in
\hs{f}~\hs{a}, there is no way of obtaining the computation \hs{f}~\hs{b},
i.e. these computations can only be performed in sequence.

Consider a simple example, where we use the monad \hs{f}~\hs{=}~\hs{IO} to
describe an effectful program that prints \hs{"pong"} if the user enters
\hs{"ping"}:

\vspace{1mm}
\begin{minted}[xleftmargin=10pt]{haskell}
pingPongM :: IO ()
pingPongM = getLine >>= \s -> if s@\,@==@\,@"ping" then putStrLn "pong" else pure ()
\end{minted}
\vspace{1mm}

\noindent
The first argument of the bind operator reads a string using
\hs{getLine}~\hs{::}~\hs{IO}~\hs{String}, and the second argument is the
function of type \hs{String}~\hs{->}~\hs{IO}~\hs{()}, which prints \hs{"pong"}
when~\hs{s}~\hs{==}~\hs{"ping"}.

This way of composing effects is sometimes unsatisfactory. To \emph{inspect} the
function \hs{\s}~\hs{->}~\hs{...}, we need an~\hs{s}, which becomes available
only during execution; we are therefore unable to predict the effects that
\hs{pingPongM} might perform: instead of conditionally executing \hs{putStrLn},
as intended, it might delete a file from disk, or launch proverbial missiles. As
we will see in sections~\S\ref{sec-static} and~\S\ref{sec-haxl}, in some
applications it is desirable to know all possible effects \emph{statically},
i.e. before the execution.

\emph{Applicative functors}, introduced by~\citet{mcbride2008applicative}, can
be used for composing statically known collections of effectful computations, as
long as these computations are \emph{independent} from each other. The key
ingredient of applicative functors is the \emph{apply} operator, denoted
by~\hs{<*>}:

\vspace{1mm}
\begin{minted}[xleftmargin=10pt]{haskell}
(<*>) :: Applicative f => f (a -> b) -> f a -> f b
\end{minted}
\vspace{1mm}
\newpage

\noindent
The operator takes two effectful computations, which --- independently ---
compute values of types \hs{a}~\hs{->}~\hs{b} and \hs{a}, and returns their
composition that performs both computations, and then applies the obtained
function to the obtained value producing the result of type \hs{b}. Crucially,
both arguments and associated effects are known statically, which, for example,
allows us to pre-allocate all necessary computation resources upfront
(\S\ref{sec-static}) and execute all computations in parallel
(\S\ref{sec-haxl}).

Alas, our ping-pong example cannot be expressed using applicative functors.
Since the two computations must be independent, the best we can do is to print
\hs{"pong"} unconditionally:

\vspace{0.5mm}
\begin{minted}[xleftmargin=10pt]{haskell}
pingPongA :: IO ()
pingPongA = fmap (\s -> id) getLine <*> putStrLn "pong"
\end{minted}
\vspace{0.5mm}

\noindent
We use \hs{fmap}~\hs{(\s}~\hs{->}~\hs{id)} to replace the input string \hs{s},
which we now have no need for, with the identity function
\hs{id}~\hs{::}~\hs{()}~\hs{->}~\hs{()}, thus matching the return type of
\hs{putStrLn}~\hs{"pong"}~\hs{::}~\hs{IO}~\hs{()}. We cannot execute the
\hs{putStrLn} effect conditionally but, on the positive side, the effects are no
longer hidden behind opaque effect-generating functions, which makes it possible
for the applicative functor \hs{f}~\hs{=}~\hs{IO} to statically know the two
effects embedded in \hs{pingPongA}.

At this point the reader is hopefully wondering: can we combine the advantages
of applicative functors and monads, i.e. allow for conditional execution of some
effects while retaining the ability to statically know all effects embedded in
a computation? It will hardly be a surprise that the answer is positive, but it
is far from obvious what the right abstraction should be. For example, one might
consider adding a new primitive called \hs{whenS} to \hs{IO}:

\vspace{0.5mm}
\begin{minted}[xleftmargin=10pt]{haskell}
whenS :: IO Bool -> IO () -> IO ()
\end{minted}
\vspace{0.5mm}

\noindent
This primitive executes the first computation, and then uses the obtained
\hs{Bool} to decide whether to execute the second computation or not. Let us
rewrite the ping-pong example using \hs{whenS}:

\vspace{0.5mm}
\begin{minted}[xleftmargin=10pt]{haskell}
pingPongS :: IO ()
pingPongS = whenS (fmap ("ping"==) getLine) (putStrLn "pong")
\end{minted}
\vspace{0.5mm}

\noindent
We replace the input string~\hs{s} with \hs{True} if it is equal to \hs{"ping"},
and \hs{False} otherwise, thereby appropriately \emph{selecting} the subsequent
effectful computation. This approach gives us both conditional execution of
\hs{putStrLn}~\hs{"pong"}, and static visibility of both effects. Crucially,
\hs{whenS} must be an \hs{IO} primitive instead of being implemented in terms of
the monadic bind (\hs{>>=}), because the latter would result in wrapping
\hs{putStrLn} into an opaque function, as in \hs{pingPongM}.

The main idea of this paper is that \hs{whenS}, as well as many other similar
combinators, can be seen as special cases of a new intermediate abstraction,
called \emph{selective applicative functors}, whose main operator for composing
effectful computations is \emph{select}:

\vspace{0.5mm}
\begin{minted}[xleftmargin=10pt]{haskell}
select :: Selective f => f (Either a b) -> f (a -> b) -> f b
\end{minted}
\vspace{0.5mm}

\noindent
Intuitively, the first effectful computation is used to select what happens
next: if it yields a \hs{Left}~\hs{a} you \emph{must execute} the second
computation in order to produce a \hs{b} in the end; otherwise, if it yields a
\hs{Right}~\hs{b}, you \emph{may skip} the subsequent effect, because you have
no use for the resulting function.

The contributions of this paper are as follows:

\vspace{-1mm}
\begin{itemize}
    \item We introduce \emph{selective applicative functors} as a general
    abstraction situated between applicative functors and monads, characterising
    the relationships between all three abstractions with a set of laws, and
    defining a few important instances (\S\ref{sec-selective}).
    \item We discuss applications of the abstraction on two real-life case
    studies: the OCaml build system \Dune~\citep{dune} (\S\ref{sec-static}) and
    Facebook's \Haxl library~\cite{marlow2014haxl} (\S\ref{sec-haxl}).
    \item We present \emph{free selective applicative functors} and show how to
    use them to implement embedded domain-specific languages with both
    conditional effects and static analysis (\S\ref{sec-free}).
\end{itemize}

We discuss alternatives to selective applicative functors and related work in
sections~\S\ref{sec-alternatives} and \S\ref{sec-related}.


%!TEX root=../Selex.tex
\section*{Discussion} % (fold)
\label{sec:discussion}

% Summary of the study
The over-arching objective of this simulation study was to compare the performance of assuming more structural complexity in selectivity when in fact the real data come from a simple stationary process, and assuming simple structural complexity when real data come from a fishery with dynamic changes in selectivity.  To address this objective, a simulation model conditioned on length-based selectivity and variable length-at-age by year was used to generate simulated data for four alternative assessment models that assumed: (a) selectivity was length-based and stationary, (b) selectivity was length-based and changed discretely in four time periods, and (c) selectivity was age-based and allowed to change each year, and (d) selectivity was age-based and interpolated over age and year using a bicubic spline and 60 equally spaced knots.  From the perspective of a na\"ive analyst who is unfamiliar with the history of the fishery and the source of the catch-age data, adopting a penalized time-varying selectivity may be more appropriate than assuming constant selectivity.  This general result is also consistent with a similar simulation study that examined time-varying changes in catchability \citep{wilberg2006performance}.

The addition of age-composition information into stock assessments greatly enhances the estimability of the underlying production function and related reference points \citep{magnusson2007mfd}.  In addition, age-composition information can also contribute to the estimation of over-all population scale via catch-curves and fixed assumptions about natural mortality and fisheries selectivity.  However, as assumptions about natural mortality and selectivity are relaxed (and freely estimated), information about population scaling degrades \citep{hilborn1992quantitative}. If the relative abundance index available for fitting lacks contrasting information to resolve confounding between overall productivity and population scale, the move towards more flexible selectivity models will lead to greater uncertainty.  In such cases where it is known that selectivity has changed over time, the addition of prior information on population scaling (i.e., priors for $B_o$ or survey $q$) will particularly valuable.

Earlier versions of the multivariate logistic likelihood for the age-composition data added a small fixed constant (1.e-30) to the observed and predicted age-proportions to ensure that the function remained defined when observed proportions-at-age were equal to 0 (i.e., this was done to avoid taking the natural logarithm of 0).  It turns out that the value of the fixed constant would have slight influences on the results and in some cases prevent the non-linear search routine from converging to a solution.  Adding a small constant to 0-observations that are likely to have high measurement error is akin to imputing data and is probably not a safe practice in the long-run.  As an alternative approach, we adopted a method used by \cite{richards1997visualizing} where observed 0 proportions-at-age (or some minimum proportion, e.g., 2\% in their paper) were pooled with the adjacent year class for that year only.  For example, if the observed proportion of age-4 fish in 1985 was equal to 0, the estimation model would compute the likelihood for the number of age 4--5 fish in 1986;  there is no likelihood component for age-4 fish in 1985.  This pooling of year classes eliminates the need for adding small, potentially influential, constants to the likelihood. There is also a small caveat on this pooling approach: if a given cohort never appears in catch-age data (i.e., a complete year-class recruitment failure), the estimation model will equally split the year class into the adjacent cohort.

Changes in selectivity over time are also a special case of time-varying catchability.  It has already been demonstrated that additional sources of information, such as tagging data \citep{martell2002implementing}, and/or area swept information \citep{winters1985interaction}, would reduce the confounding between stock size and stock productivity.  Statistical catch-at-age models rely on a separability assumption where year and age effects in the observed catch-at-age data can be partitioned into fishing mortality and selectivity, respectively.  Having auxiliary information on either one of these effects from area-swept estimates of relative fishing mortality or size-based selectivity based on tag return data would reduce potential confounding and improve the estimability of time-varying parameters \citep[e.g.,][]{sinclair2002disentangling}.

This simulation study examined the specific case where the true underlying selectivity is length-based and the corresponding age-based selectivity changes over time due to changes in growth rates.  Selectivity is a product of vulnerability and availability.  Vulnerability is the probability of catching a fish at a given time/location assuming the fish is available to harvest.  Availability is the probability of fish being present in the time/location where fishing activity is occurring.  These two processes are completely confounded and cannot be separated without additional information that directly measures either vulnerability or availability.  As a result of these two processes, the definition of fisheries selectivity may have many subtle differences among fisheries, or even among years in a given fishery.  Moreover, changes in harvest policy, or changes in allocation among regulatory areas, or fishing fleets, can result in dramatic changes in age-based selectivity due to time/area interactions between various fishing fleets and stock distribution.  Given such complexities, it might be preferable to always adopt an age-based time-varying selectivity sub model in statistical catch-age assessments. Initial assessments could start with very high penalty weights to constrain how much selectivity is allowed to vary (e.g., $\lambda^{(3)}$), then begin to relax the penalty and examine the sensitivity of both model fit and policy performance to the assumed penalty weight.  Simply assuming a fixed selectivity model, or even block selectivities, is akin to extremely large penalty weights on time-varying selectivity.  In this regard, this simulation presented a 'worst-case' scenario for time-varying selectivity.  Future work could simulate the application of time-varying selectivity in conjunction with a set of rules governing adjustment of the $\lambda^{(3)}$  and directly explore its performance (i.e., Table \ref{tab:rankorder}).

One potential concern with the addition of more and more selectivity coefficients is that the assessment model begins to over-fit the age-composition data and explain the observed data with additional recruitment variation and more complex selectivity coefficients.  This result was observed in the Monte Carlo simulations in this study where the underlying data were generated with extremely complex selectivity patterns and the estimation model had a very flexible selectivity model with many estimated parameters.  Such an over-parameterized model would be of less utility in forecasting due to large uncertainties and confounding in selectivity and recruitment deviations in the terminal year. However, the true uncertainty would be better represented, which might lead to more realistic expectations in a decision table framework.  Penalized likelihoods can ameliorate this to some extent, but we've also shown that the use of interpolation methods (e.g., bicubic splines) for computing age-specific selectivity coefficients each year can perform well.  The subjective issue of importance in the case of using a bicubic spline is the number of spline knots that should be estimated for the year effect.  Presumably model selection could proceed in similar fashion as a stepwise-selection that was proposed by  \cite{thorson2012stepwise}.  Rules for this procedure could also be tested in similar simulation studies such as this one.

 The vast majority of assessment models are age-based requiring age-specific estimates of fishing mortality rates, and hence age-based selectivity \citep{gavaris2002sif}.  Adopting a fixed age-based selectivity model would certainly lead to erroneous errors if the true underlying model is length-based and substantial changes in growth rates have occurred over time.  Two options for dealing with this problem are: (1) model length-based selectivity and using empirical length-age data, or (2) model age-based selectivity but allow selectivity to change over time. The first option requires unbiased estimates of length-at-age.  These cannot be obtained if composition data are sampled using length-selective gear and there is any appreciable variance in length for a given age. The second option is being explored by the International Pacific Halibut Commission for dealing with changes in selectivity associated with changes in size-at-age and stock distribution \citep{stewart2012assessment}.  Selectivity in the directed Pacific halibut fishery is length-based given evidence from hooking success studies. There are minimum size-limits in place, so age-specific retention rates vary with changes in size-at-age over time.  The transition to time-varying age-based selectivity was adopted primarily because it solved a retrospective bias that has been of major concern for this stock in recent years.

% MOve to discussion
% It is fairly typical to see such lags in estimates of abundance, even in age-structured models \citep{walters2004simple,cox2008practical}

A better alternative to specific case studies would be to develop a closed-loop feedback control system and an appropriate loss function to better elucidate which selectivity parameterization is more appropriate for achieving intended management objectives.  This is also known in the fisheries realm as management strategy evaluation \citep{de1986simulation,Cooke1999,smith1999implementing}.  Having an appropriate loss function to judge the performance of each alternative model would greatly improve model selection criterion from a policy performance perspective.

% section discussion (end)

\section*{Acknowledgments} % (fold)
\label{sec:acknowledgments}
	The authors would like to thank the organizers of the CAPAM workshop held on March 12-14, 2013 in La Jolla California.  The first author would also like to thank James Ianelli and Dave Fournier for the assistance in developing the bicubic spline model for this application.  Thanks to Bruce Leaman, Andre Punt, Allen Hicks, Robyn Forrest, Ray Hilborn, James Thorson, and many other for feedback on earlier presentations of this work and the provision of the Pacific hake data.
% section acknowledgments (end)


\section{Adjusting the queuing model}\label{sec:scenarios}

This section is comprised of several `what-if' scenarios --- a classic component
of healthcare operational research --- under the novel parameterisation of the
queue established in Section~\ref{sec:model}. The outcomes of interest in this
work are server (resource) utilisation and system times as these metrics capture
the driving forces of cost and flow as well as the overall state of the system,
its staff and its patients. Specifically, the objective of these experiments is
to address the following questions:
\begin{itemize}
    \item How would the system be affected by a change in overall patient
        arrivals?
    \item How is the system affected by a change in resource availability (i.e.\
        a change in \(c\))?
    \item How is the system affected by patients moving between clusters?
\end{itemize}

Owing to the nature of the observed data, the queuing model parameterisation
and its assumptions, the effects on the chosen metrics in each scenario are
given in relative terms with respect to the base case. The base case being those
results generated from the best parameter set recorded in
Table~\ref{tab:comparison}. In particular, the data from each scenario is scaled
by the corresponding median value in the base case meaning that a metric having
a value of 1 is `normal'.

As mentioned in Section~\ref{sec:intro}, the source code used throughout this
work is available online and has been archived online~\cite{Wilde2020github}. In
addition to this, the datasets generated from the simulations in this section
have been archived along with those generated from the parameter
sweep~\cite{Wilde2020results}.

\subsection{Changes to overall patient arrivals}\label{subsec:arrivals}

Changes in overall patient arrivals to a queue reflect real-world scenarios
where some stimulus is improving (or worsening) the condition of the patient
population. Examples of stimuli could include an aging population or independent
life events that lead to a change in deprivation such as an accident or job
loss. Within this model, overall patient arrivals are altered using a scaling
factor denoted by \(\sigma\in\mathbb{R}\). This scaling factor is applied to the
model by multiplying each cluster's arrival rate by \(\sigma\). That is, for
cluster \(i\), its new arrival rate, \(\hat\lambda_i\), is given by:
\begin{equation}\label{eq:lambda}
    \hat\lambda_{i} = \sigma\lambda_i
\end{equation}

\begin{figure}
    \centering
    \begin{subfigure}{.5\imgwidth}
        \includegraphics[width=\linewidth]{lambda_time}
        \caption{}\label{fig:lambda_time}
    \end{subfigure}\hfill%
    \begin{subfigure}{.5\imgwidth}
        \includegraphics[width=\linewidth]{lambda_util}
        \caption{}\label{fig:lambda_util}
    \end{subfigure}
    \caption{%
        Plots of \(\sigma\) against relative (\subref{fig:lambda_time})~system
        time and (\subref{fig:lambda_util})~server utilisation.
    }\label{fig:lambda}
\end{figure}

Figure~\ref{fig:lambda} shows the effects of changing patient arrivals on
(\subref{fig:lambda_time})~relative system times and
(\subref{fig:lambda_util})~relative server utilisation over values of \(\sigma\)
from \input{tex/lambda_scaling_min} to \input{tex/lambda_scaling_max} at a
precision of \input{tex/lambda_scaling_step}. Specifically, each plot in the
figure (and the subsequent figures in this section) shows the median and
interquartile range (IQR) of each relative attribute. These metrics provide an
insight into the experience of the average user (or server) in the system, and
in the stability or variation of the body of users (servers).

What is evident from these plots is that things are happening as one might
expect: as arrivals increase, the strain on the system increases. However, it
should be noted that it also appears that the model has some amount of slack
relative to the base case. Looking at Figure~\ref{fig:lambda_time}, for
instance, the relative system times (i.e.\ the relative length of stay for
patients) remains unchanged up to \(\sigma \approx 1.2\), or an approximate 20\%
increase in arrivals of COPD patients. Beyond that, relative system times rise
to an untenable point where the median time becomes orders of magnitude above
the norm.

However, Figure~\ref{fig:lambda_util} shows that the situation for the system's
resources reaches its worst case near to the start of that spike in relative
system times (at \(\sigma \approx 1.4\)). That is, the median server utilisation
reaches a maximum (this corresponds to constant utilisation) at this point and
the variation in server utilisation disappears entirely.

\subsection{Changes to resource availability}\label{subsec:resources}

As is discussed in Section~\ref{sec:model}, the resource availability of the
system is captured by the number of parallel servers in the system, \(c\).
Therefore, to modify the overall resource availability, only the number of
servers need be changed. This kind of sensitivity analysis is usually done to
determine the opportunity cost of adding service capacity to a system, e.g.\
would adding \(n\) servers sufficiently increase efficiency without exceeding
a budget?

To reiterate the beginning of this section, all suitable parameters are given in
relative terms. This includes the number of servers here. By doing this, the
changes in resource availability are more easily seen, and do away with any
concerns as to what a particular number of servers exactly reflects in the real
world.

\begin{figure}
    \centering
    \begin{subfigure}{.5\imgwidth}
        \includegraphics[width=\linewidth]{servers_time}
        \caption{}\label{fig:servers_time}
    \end{subfigure}\hfill%
    \begin{subfigure}{.5\imgwidth}
        \includegraphics[width=\linewidth]{servers_util}
        \caption{}\label{fig:servers_util}
    \end{subfigure}
    \caption{%
        Plots of the relative number of servers against relative
        (\subref{fig:servers_time})~system time and
        (\subref{fig:servers_util})~server utilisation.
    }\label{fig:servers}
\end{figure}

Figure~\ref{fig:servers} shows how the relative resource availability affects
relative system times and server utilisation. In this scenario, the relative
number of servers took values from \input{tex/num_servers_change_min} to
\input{tex/num_servers_change_max} at steps of
\input{tex/num_servers_change_step} --- this is equivalent to a step size of 1
in the actual number of servers. Overall, these figures fortify the
claim from the previous scenario that there is some room to manoeuvre so that
the system runs `as normal' but pressing on those boundaries results in massive
changes to both resource requirements and system times.

In Figure~\ref{fig:servers_time} this amounts to a maximum of 20\% slack in
resources before relative system times are affected; further reductions quickly
result in a potentially tenfold increase in the median system time, and up to 50
times once resource availability falls by 50\%. Moreover, the variation in the
body of the relative times (i.e.\ the IQR) decreases as resource availability
decreases. The reality of this is that patients arriving at a hospital are
forced to consume larger amounts of resources (simply by being in a hospital)
regardless of their condition, putting added strains on the system.

Meanwhile, it appears that there is no tangible change in relative system times
given an increase in the number of servers. This indicates that the model
carries sufficient resources to cater to the population under normal
circumstances, and that adding service capacity will not necessarily improve
system times.

Again, Figure~\ref{fig:servers_util} shows that there is a substantial change in
the variation in the relative utilisation of the servers. In this case, the
variation dissipates as resource levels fall and increases as they increase.
While the relationship between real hospital resources and the number of servers
is not exact, having variation in server utilisation would suggest that parts of
the system may be configured or partitioned away in the case of some significant
public health event (such as a global pandemic) without overloading the system.

\subsection{Moving arrivals between clusters}\label{subsec:moving}

This scenario is perhaps the most relevant to actionable public health research
of those presented here. The clusters identified in this work could be
characterised by their clinical complexities and resource requirements, as done
in Section~\ref{subsec:overview}. Therefore, being able to model the movement of
some proportion of patient spells from one cluster to another will reveal how
those complexities and requirements affect the system itself. The reality is
then that if some public health policy could be implemented to enact that
movement informed by a model such as this then real change would be seen in the
real system.

In order to model the effects of spells moving between two clusters, the
assumption is that services remain the same (and so does each cluster's \(p_i\))
but their arrival rates are altered according to some transfer proportion.
Consider two clusters indexed at \(i, j\), and their respective arrival rates,
\(\lambda_i, \lambda_j\), and let \(\delta \in [0, 1]\) denote the proportion of
arrivals to be moved from cluster \(i\) to cluster \(j\). Then the new arrival
rates for each cluster, denoted by \(\hat\lambda_i, \hat\lambda_j\)
respectively, are:
\begin{equation}\label{eq:moving}
    \hat\lambda_i = \left(1 - \delta\right) \lambda_i
    \quad \text{and} \quad
    \hat\lambda_j = \delta\lambda_i + \lambda_j
\end{equation}

By moving patient arrivals between clusters in this way, the overall arrivals
are left the same since the sum of the arrival rates is the same. Hence, the
(relative) effect on server utilisation and system time can be measured
independently.

Figures~\ref{fig:moving_time}~and~\ref{fig:moving_util} show the effect of
moving patient arrivals between clusters on relative system time and relative
server utilisation respectively. In each figure, the median and IQR for the
corresponding attribute is shown, as in the previous scenarios. Each scenario
was simulated using values of \(\delta\) from \input{tex/moving_clusters_min} to
\input{tex/moving_clusters_max} at steps of \input{tex/moving_clusters_step}.

Considering Figure~\ref{fig:moving_time}, it is clear that there are some cases
where reducing particular types of spells (by making them like another type of
spell) has no effect on overall system times. Namely, moving the high
resource requirement spells that make up Cluster 0 and Cluster 3 to any other
cluster. These clusters make up only 10\% of all arrivals and this figure shows
that in terms of system times the model is able to handle them without concern
under normal conditions. The concern comes when either of the other clusters
moves to Cluster 0 or Cluster 3. Even as few as one in five of the low
complexity, low resource needs arrivals in Cluster 2 moving to either cluster
results in large jumps in the median system time for all arrivals, and soon
after, as in the previous scenario, any variation in the system times
disappears indicating an overborne system.

With relative server utilisation, the story is much the same. The normal levels
of high complexity, high resource arrivals from Cluster 3 are absorbed by the
system and moving these arrivals to another cluster bears no effect on resource
consumption levels. Likewise, either of the low resource need clusters moving
even slightly toward high resource requirements completely overruns the system's
resources. However, the relative utilisation levels of the system resources can
be reduced by moving arrivals from Cluster 0 to either Cluster 1 or Cluster 2,
i.e.\ by reducing the overall resource requirements of such spells.

In essence, this entire analysis offers two messages: that there are several
ways in which the system can get worse and even overwhelmed but, more
importantly, that any meaningful impact on the system must come from a stimulus
outside of the system that results in more healthy patients arriving to the
hospital. This is non-trivial; the first two scenarios in this analysis show
that there are no quick solutions to reduce the effect of COPD patients on
hospital capacity or length of stay. The only effective intervention is found
through inter-cluster transfers.

\begin{figure}
    \centering
    \includegraphics[width=\imgwidth]{moving_time}
    \caption{%
        Plots of proportions of each cluster moving to another against relative
        system time.
    }\label{fig:moving_time}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=\imgwidth]{moving_util}
    \caption{%
        Plots of proportions of each cluster moving to another on relative
        server utilisation.
    }\label{fig:moving_util}
\end{figure}