The general idea behind this approach is simple: There exists one network which takes the futures' term structure as input and outputs a prediction of spread prices. If not specified otherwise the prices of the next business day are predicted. This is quite similar to figure~\ref{fig:longspread} where the green data points are the input and the blue data points -- but now for the next day! -- the output. Ideally one just has to find optimal hyperparameters.

The first part of this section will be about getting familiar with the data itself. This includes a general description as well as the handling of problematic data points.
Thereafter the network architecture as well as the tunable hyperparameters will be introduced more formally.
At last the the results of this approach are presented in a detailed manner.

\section{Data description and data handling}
\label{sec:aao-data-handling}

Primarily, the available data are the prices of VIX futures since they were first issued in 2004. Supplementary there is data about the VIX itself as well as additional information about the futures: their symbol and their precise date of expiration. All of these are sampled on a daily interval. There is also minutely data available which will not be used unless explicitly mentioned.

\subsection{Input data (term structure)}
\label{sec:aao-input-data}

The set of term structure data includes 3,304 samples in total from March 26th, 2004 to May 5th, 2017. Each term structure consists of three up to eight legs. The legs symbolize the months when the respective future expires. The specific date of expiration is always a workday in the middle of this month. For an example term structure see table~\ref{tab:single-termstructure} and for a graphical overview over the whole data set see figure~\ref{fig:termstructures}. Notice how similar the legs look to each other and to the VIX from figure~\ref{fig:vix}. To make the curvature between the term structures' legs more explicit their difference is calculated and used as network input.

\begin{table}
	\centering
	\caption[Single term structure with expiration dates (February 29th, 2012)]{A single term structure with expiration dates (February 29th, 2012) and corresponding long prices. The eight values are the futures' current prices for the next eight months. The expiration row shows the date the respective future expires. After expiration one can not invest in this future anymore.}
	\begin{tabular}{lllllllll}
	\toprule
	{} &      M1 &      M2 &      M3 &      M4 &      M5 &      M6 &      M7 &      M8 \\
	\midrule
	Value      &   21.00 &   23.99 &   25.60 &   26.50 &   27.75 &   28.40 &   29.05 &   29.25 \\
	Expiration &  Mar 21 &  Apr 18 &  May 16 &  Jun 20 &  Jul 18 &  Aug 22 &  Sep 19 &  Oct 17 \\
    \midrule
    Long Price &         &    1.38 &    0.71 &   -0.35 &    0.60 &    0.00 &    0.45 &         \\
	\bottomrule
	\end{tabular}
 	\label{tab:single-termstructure}
\end{table}

\begin{figure}
	\centering
	\includegraphics[width=0.9\linewidth]{termstructures}
	\caption[Complete data set of term structures]{%
		Complete data set of term structures. Each term structure consists of at most eight legs (M1 to M8) where M1 is closest to expiration (blue). For most of the data -- except in times of crisis -- the legs with later expiration (green) have a higher price. Furthermore the general shape of the data is very similar between legs as well as similar to the VIX itself (see figure~\ref{fig:vix}).}
	\label{fig:termstructures}
\end{figure}

One problem for this task is missing data. If there are values for all eight months of a term structure like in table~\ref{tab:single-termstructure} this is a \emph{complete} sample. Consequently, when there are values missing it is an \emph{incomplete} sample. As seen in figure~\ref{fig:some-nice-graphics} there are especially many incomplete samples in earlier years. Because trading of futures just started it was not as established as of now. Futures are issued per month. But even though this happens regulary for each consecutive month since 2012 this was not always the case before. However, this regularity is necessary for a complete term structure. Removing all incomplete samples would shrink the dataset by $35\%$.
By removing all samples before October 23th, 2006 only samples with up to two missing legs are left. This shrinks the data set by just $20\%$ resulting in 2,656 remaining samples. The remaining incomplete samples need to be handled by the network itself.

\begin{figure}
		\centering
	\begin{subfigure}{0.45\linewidth}
		\includegraphics[width=\linewidth]{number_of_samples}
		\caption{Number of samples by year}
		\label{fig:missing-values}
	\end{subfigure}
	\begin{subfigure}{0.45\linewidth}
		\includegraphics[width=0.95\linewidth]{missing_values}
		\caption{Distribution of missing values}
		\label{fig:weekdays}
	\end{subfigure}
	\caption[Number of samples by year and distribution of missing values]{%
		While a whole term structure consists of eight legs there is often less data available, especially in earlier years. Even though there are less samples from 2004 to 2006 there is a larger number of \emph{incomplete} term structures, often missing up to five legs.}
	\label{fig:some-nice-graphics}
\end{figure}

\subsection{Target data (long spread prices)}
\label{sec:aao-target-data}

For training a network one needs inputs as well as corresponding targets. While the term structure data set holds the necessary inputs, it is easy to calculate the target by applying \eqref{eq:long-spread} thus getting the long spread prices (see table~\ref{tab:single-termstructure}).\footnote{%
	Because of \eqref{eq:spread-correlation} it is sufficient to either calculate long spreads \emph{or} short spreads.}
For a complete term structure we get a complete set of spread prices consisting of six values.
Obviously the targets are the spread prices of the next term. Otherwise the network will not learn to make a prediction but an approximation of the applied formula. Because the term structure is the calculatory basis of the spread prices, they suffer from the same problem of missing data. Furthermore one has to exercise caution when expiration is close, so prediction is not made for already expired values.

\subsection{Seasonality}
\label{sec:seasonality}

As the original problem is finding inefficiencies in the term structure the network needs to be able to predict some recurring patterns. Therefore one might need some additional prior introducing seasonality. Some options are:

\newpage

\begin{itemize}
	\item The day of the month (1st to 31st)
	\item The month itself (Jan to Dec)
	\item The days until expiration of the term structure's first leg (0 to 34; see figure~\ref{fig:daystoexpiration})
\end{itemize}

The latter one -- the days until expiration -- has a nice property that is hopefully\footnote{%
Understanding what exactly a network actually learns is very difficult. One can not simply assume the same behavior as for humans. Most of the time one can only \emph{hope} the own choices were correct.}
advantageous in this context:
There exist a total order. The former two are categorical by nature, e.g. without knowing about the year, January is \emph{not smaller} than February. But it is certainly possible to count down the days until expiration. Therefore it is appropriate to model these as a simple natural number at the network's input. For the alternatives is might have been necessary to use more verbose representations like a one-hot vector.\footnote{%
	A \emph{one-hot vector} is a binary vector with the same length as there are classes. All entries are $0$, except one which holds the value $1$. The position of this $1$ denotes the corresponding class.}

\begin{figure}
	\centering
	\includegraphics[width=0.9\linewidth]{days_to_expiration}
	\caption[Distribution of days to expiration for the termstructures' first leg]{%
		Distribution of days to expiration for the term structures' first leg.}
	\label{fig:daystoexpiration}
\end{figure}

\begin{table}
	\centering
	\caption[Descriptive statistics for futures and spread prices]{Common descriptive statistics for futures, their legs' difference and spread prices.}
	\begin{tabular}{lrrrrrrr}
		\toprule
		{} &  Mean &  Std. &    Min. &   25\% &   50\% &   75\% &  Max. \\
		\midrule
		Futures & 22.38 & 6.91 &  10.24 & 17.55 & 20.70 & 25.70 & 67.90 \\
		Difference & 0.38 & 1.00 & 21.10 & 0.07 & 0.40 & 0.75 & 5.45 \\
		Spread prices  &  0.11 & 0.74 & -13.68 & -0.10 &  0.13 &  0.36 &  4.90 \\
		\bottomrule
	\end{tabular}
	\label{tab:descriptive-stats}
\end{table}

\subsection{Normalization}

Looking at the most common descriptive statistics for input and target data in table~\ref{tab:descriptive-stats} -- especially the quantiles -- the spread prices are usually smaller than the futures by two orders of magnitude. And even for the differences of the futures' legs there are some outliers. This will not necessarily pose a problem but nevertheless \emph{optional} normalization is introduced by:

\begin{equation}
	X' = \frac{X - \bar{X}}{X_{max} - X_{min}} \quad \text{where $\bar{X}$ is the mean.}
\end{equation}

Hence, the resulting values of $X'$ will be in range $[0,1]$.

When explicitly mentioned am alternative formula for normalization is used, leading to zero mean and unit variance:

\begin{equation}
	\label{eq:normalization2}
	X'' = \frac{X - \bar{X}}{Std(X)} = \frac{X - \bar{X}}{\sqrt{Var(X)}}
\end{equation}

\subsection{Validation and test set}
\label{sec:aao-validation-and-test-set}

For validating the performance of deep networks the most widespread approach is to use one part of the data for tuning the hyperparameters introduced in section~\ref{sec:hyperparameters} (the \emph{validation set}) and one part for evaluating the final model (the \emph{test set}). In this work $15\%$ of the data will be used for validation and test set each.

Looking at figure~\ref{fig:termstructures} it becomes evident that there is a temporal connection between areas of low and high price: If you look at some sample with low priced futures there is a high probability that samples next to this one also have a low price and vice versa. Therefore choosing completely random samples for validation and testing will result in datasets highly similar to the training set. But choosing one large connected chunk will result in hyperparameters fitted only to this timeframe. Choosing a tradeoff, test and validation set will be chosen by taking two chunks of data from different locations of the original data each, as shown in figure~\ref{fig:validation-and-test-set}.

\begin{figure}
	\centering
	\includegraphics[width=0.9\linewidth]{images/validation-and-test-set}
	\caption[Splitting whole dataset into training, validation and test set]{Splitting the whole dataset into training, validation and test set. The top figure shows the futures' prices for the next eight months (M1 to M8) of expiration forming the term structure. The bottom figure shows the corresponding spread prices. While the green area shows the data used for validation the red area is used for testing. The remaining data is used for training.}
	\label{fig:validation-and-test-set}
\end{figure}

\section{Network architecture}
\label{sec:aao-network-architecture}

A feedforward neural network is one oft the simplest architectures used in deep learning. First, there will be some definitions describing the network as well as some intuitions about training of neural networks. Thereafter the hyperparameters for tailoring the network to the problem are introduced. To prevent clueless searching over all of these some defaults are adopted. Finally, as already mentioned, the network needs to handle missing data. Some simple approach introducing an additional layer at the networks' output is introduced.

\subsection{Definitions}

The feedforward neural network can formally be described as a sequence of matrix-vector multiplications. Let $L$ be the number of hidden layers where $l \in \{1,\dots,L\}$ is the hidden layer's index. For a layer $l$ there is an input vector $\mathbf{z}^{(l)}$, an output vector $\mathbf{y}^{(l)}$ as well as a weight matrix $W^{(l)}$ and a bias vector $\mathbf{b}^{(l)}$. The input layer be $\mathbf{x} = \mathbf{y}^{(0)}$ and the output $\mathbf{t} = \mathbf{y}^{(L+1)}$. Furthermore there is an activation function $f^{(l)}$. At the output $f^{(L+1)}$ is typically the identity function.

Inference is done by the following operations for $l \in \{1,\dots,L+1\}$:

\begin{align}
	\label{eq:layer-input}
	\mathbf{z}^{(l)} &= W^{(l)\top} \mathbf{y}^{(l-1)} + \mathbf{b}^{(l)} \\
	\label{eq:layer-output}
	\mathbf{y}^{(l)} &= f^{(l)}(\mathbf{z}^{(l)})
\end{align}

Each hidden layer can have an arbitrary width greater zero. A layer's width is the length of the vector $\mathbf{y}^{(l)}$. In this work each hidden layer has the same width. Let this be the network's width $K$. Accordingly, the number of hidden layers $L$ is the network's depth.

For training -- this corresponds to optimizing $W$ and $\mathbf{b}$ -- the \emph{back-propagation} algorithm along with an optimization method like \emph{stochastic gradient descent} is used. Therefore a scalar loss function $J(\mathbf{t}, \mathbf{\hat{t}})$ is introduced, comparing the output of the network $\mathbf{t}$ to the target $\mathbf{\hat{t}}$. By iteratively computing the gradient with respect to the parameters $W$ and $\mathbf{b}$ for $J$ as well as for the activation functions $f^{(l)}$ backwards through the network one can optimize each layer's parameters. For further details see \cite{Rumelhart-1986a} and \cite[p.\,204ff.]{Goodfellow-et-al-2016}.

\subsection{Hyperparameters}
\label{sec:hyperparameters}

These definitions result in a number of tunable hyperparameters, potentially affecting the network's performance:
\begin{itemize}
	\item Activation functions $f^{(l)}$
	\item Loss function $J$
	\item Optimization method (and corresponding hyperparameters)
	\item Weight initializations (choosing initial values for $W^{(l)}$)
	\item Network depth $L$
	\item Network width $K$
\end{itemize}

While for some of these it is relatively easy to find some good values or there is some reasonable default, others are highly dependent on the problem and the data. For these, one has to extensively search for optimal values while constantly validating the performance.

\paragraph{Activation functions.}
(also called hidden units when used together with their input from \eqref{eq:layer-input}) \\
One can read in \cite[p.\,191]{Goodfellow-et-al-2016}:
\begin{quote}
	``The design of hidden units is an extremely active area of research and does not yet have many definitive guiding theoretical principles.''
\end{quote}

\begin{wrapfigure}[8]{r}{0.3\textwidth}
	\vspace{-1em}
	\fbox{%
		\begin{minipage}{0.3\textwidth}\centering
			\includegraphics[width=\textwidth]{images/rectifier}
			\vspace{-20pt}
			\caption[Rectifier function]{%
				Rectifier \\ function $f(z)=\max(0,z)$}
			\label{fig:rectifier}
	\end{minipage}}
\end{wrapfigure}

Although directly followed by:
\begin{quote}
	``Rectified linear units are an excellent default choice of hidden unit.''
\end{quote}

A \emph{rectified linear unit} (ReLU) uses the activation function $f(z)=\max(0,z)$ (also called \emph{rectifier}, see figure~\ref{fig:rectifier}) meaning for $z \leq 0$ it always outputs zero. This offers some nice and simple properties especially regarding its derivative.\footnote{%
	The derivative is 0 across half its domain and 1 in the other. Furthermore the second derivative is 0 almost everywhere eliminating second-order effects.}
Using this function should ensure reasonable performance.

An alternative might be using \emph{scaled exponential linear units} (SELUs) as activations thus building a \emph{self-normalizing neural network}.\cite{DBLP:journals/corr/KlambauerUMH17} This is a very recent approach aiming for mapping the mean and variance from each layer to the next one. They explicitly try to solve some shortcoming of feedforward neural networks, often showing effective regularization properties. The SELU activation function is given by:

\begin{equation}
	\label{eq:selu}
	f(z) = \lambda
	\begin{cases}
	z & \text{if } z > 0 \\
	\alpha e^z - \alpha & \text{if } z \leq 0
	\end{cases}
\end{equation}

The corresponding paper suggests using $\alpha = 1.6733$ and $\lambda = 1.0507$ implying zero mean and unit variance. These are most effective if the training data has zero mean and unit variance, too, which can be accomplished by normalization using equation~\eqref{eq:normalization2}.

\paragraph{Loss function.}
This can be chosen quite conservatively, too. To penalize larger derivations from the target value proportionally more the \emph{mean squared error} (MSE) is used:
\begin{equation}
	\label{eq:mse}
	J(\mathbf{t}, \mathbf{\hat{t}}) =
	\frac{1}{n} \sum_{i=1}^{n}(t_i - \hat{t}_i)^2 \quad
	\text{where } n \text{ is the length of vector } \mathbf{t}.
\end{equation}

\paragraph{Optimization method.}
Most of the time \emph{stochastic gradient descent} (SGD) or some extension is chosen. The general idea is to use the gradient $\nabla_{\theta}J(\theta)$ of a function $J$ with parameters $\theta$ to find the direction (in parameter space) with the steepest descent. By updating the parameters iteratively with small steps\footnote{%
	Typically some value $\tau < 1$ is chosen like $0.01$ or $0.001$.}
of size $\alpha$ one is guaranteed to find a local minimum:

\begin{equation}
	\theta^{(i+1)} = \theta^{(i)} - \alpha \nabla_{\theta}J(\theta^{(i)})
\end{equation}

This formula describes the general gradient descent algorithm. As for the stochastic part, one takes a specific number of samples from the dataset -- called the \emph{batch size} -- and updates the parameters after evaluating just this sample. In deep learning SGD is applied with respect to weights $W$ and bias $\mathbf{b}$ for loss and activation functions. Because of the non-convex structure of neural networks finding a local minimum (instead of a global one) is acceptable. In practice this works reasonably well.

In this work primary the \emph{Adam}\footnote{``\dots the name Adam is derived from adaptive moment estimation.''} algorithm is used.\cite{DBLP:journals/corr/KingmaB14} It shows faster convergence, in general as well as for some specific tests on the current problem. New hyperparameters $\beta_1$ and $\beta_2$ are introduced for estimating the first and second moments of the gradients respectively. \\ The following hyperparameter settings are used: $\alpha=0.001$, $\beta_1 = 0.9$, $\beta_2=0.999$ (the latter two following the original paper's recommendations) and batch size of 32.

\paragraph{Weight initializations.}
For the presented iterative optimization methods the weights $W^{(l)}$ need to be initialized in some way. As mentioned above the optimization of a neural network is a non-convex problem. Therefore the local minimum found depends on these initializations. There are two properties which are especially important when choosing an initialization strategy.

First and foremost the initialization needs to break symmetry between units:

\begin{quote}
	``If two hidden units with the same activation function are connected to the same inputs, then these units must have different initial parameters. If they have the same initial
	parameters, then a deterministic learning algorithm applied to a deterministic cost
	and model will constantly update both of these units in the same way.''\cite[p.\,301]{Goodfellow-et-al-2016}
\end{quote}

Since one generally wants for each unit to learn some different aspects of the given task this is the most important role of initialization.

Secondly, since SGD is used for optimization, the weights are updated with a certain step size. A small step size indicates the assumption that the values after optimization are close to the initial values. Taking both of these properties into account, using a common probability distribution seems to be an obvious solution. This work uses the following uniform distribution suggested by \cite{pmlr-v9-glorot10a}:

\begin{equation}
	\label{eq:glorot-initialization}
	W_{ij} \sim U\left[ -\frac{1}{\sqrt{k}}, \frac{1}{\sqrt{k}} \right]
	\quad \text{where $k$ is the width of the previous layer.}
\end{equation}

\paragraph{Network depth and network width.}
Intuitively the network width increases the amount of information the network can ``remember''. The weight matrices are larger and therefore hold more values. Whereas the network depth increases the network's ability to learn underlying abstractions. A naive solution might be to chose width \emph{and} depth as large as possible. Unfortunately, this approach has serious drawbacks:

\begin{enumerate}
	\item One gets a powerful model which easily leads to overfitting\footnote{%
		This is a general problem in machine learning not limited to neural networks.}.
	The network will not learn to solve the problem but just echo its training. For samples outside the training set the performance will degrade drastically. This can distort the results in such a dramatic manner that some practitioners like \cite[p.\,23: When not to Backtest a Strategy]{Chan-2013} refuse to look at such models completely. One can fight overfitting by carefully choosing an out-of-sample validation set and using different kinds of regularization.

	\item As shortly explained above, the backpropagation algorithm uses the gradients for updating the weights backwards throughout the network. This may lead to the \emph{vanishing gradient problem} where the layers near the outputs are updated much faster than the layers near the inputs. Even though (very) deep networks were quite successful one some tasks during the recent years\footnote{%
		The ResNet architecture is very popular in image recognition. With a depth of 152 layers it even won 1st place on the Large Scale Visual Recognition Challenge (ILSVRC) 2015 classification task.\cite{DBLP:journals/corr/HeZRS15}}
	there is still no general solution for this problem, especially for feedforward networks.

	\item One may wish to force the network to learn abstractions and therefore require a certain depth. But a large depth in combination with a large width will not lead to the desired results because the network will likely use its many parameters to just ``remember'', ignoring the possibilities for abstraction coming with the additional hidden layers and therefore rendering them useless.
\end{enumerate}

There are some recent insights that imply using a powerful model with many parameters generally works well in deep learning.\cite{DBLP:journals/corr/ZhangBHRV16} But ultimately it comes down to the actual performance on the problem at hand. Therefore depth and width of the network will be considered tunable hyperparameters, too, that need to be evaluated empirically.

\subsection{Handling missing values at network level}

Up to now was a description of a standard feedforward architecture. In section~\ref{sec:aao-data-handling} the problem of missing data for inputs as well as outputs was described. These need to be handled partly by the network. At the inputs one can just set the missing values to zero. By looking at \emph{dropout regularization} one can see why this works. Here, the general idea is to randomly remove units during training to prevent them from co-adapting too much making each individual unit more robust eventually. This is implemented by adjusting the layers' input \ref{eq:layer-input} the following way: \cite{JMLR:v15:srivastava14a}

\begin{align}
	\mathbf{z}^{(l)} &= W^{(l)\top} \mathbf{\tilde{y}}^{(l-1)} + \mathbf{b}^{(l)} \\
	\text{where} \quad
	\mathbf{\tilde{y}}^{(l)} &= \underset{\text{entrywise product}}{\mathbf{r}^{(l)} \circ \mathbf{y}^{(l)}}
\end{align}

The values of vector $\mathbf{r}^{(l)}$ are from a Bernoulli distribution meaning $\mathbf{r}^{(l)} \in \{0, 1\}^k$.

Therefore by setting the inputs $\mathbf{y}^{(0)}$ not randomly but deterministically to zero -- in case they were missing in the first place -- the respective input nodes are effectively removed. It is important to note that this will not work for SELU activations because these do not settle at zero. Setting the inputs to $\lim_{z \to -\infty} f(z) = -\lambda \alpha$ might be a slightly better fit but is not really removing the corresponding input nodes.

For partly handling the outputs, missing values were set to zero and a conditional mask layer was added to the end of the network. The mask layer uses days to expiration as additional input: If this value is zero, the output for the spread price is set to zero, too. Both output and target would be zero resulting in $J(0, 0) = 0$. Because one does not know beforehand about the availability of the next term's data there is no way to handle the output in general.

The resulting architecture is illustrated in figure~\ref{fig:architecture1}.

\begin{figure}
	\centering
	\includegraphics[width=0.9\linewidth]{architecture1}
	\caption[Network architecture with hidden layer width $K=5$]{%
		Network architecture with hidden layer width $K=5$. The mask layer just masks the first output in case there are zero days to expiration.}
	\label{fig:architecture1}
\end{figure}

\section{Evaluation}

Training a neural network to estimate a real valued vector comes down to a regression problem. This section is mainly about evaluating this chapter's approach for different configurations of hyperparameters.

\subsection{Network width, depth and normalization}

\begin{figure}
	\centering
	\begin{subfigure}{0.49\linewidth}
		\includegraphics[width=\linewidth]{approach1-ex1-loss-basic}
		\caption{}
	\end{subfigure}
	\begin{subfigure}{0.49\linewidth}
		\includegraphics[width=\linewidth]{approach1-ex1-val-basic}
		\caption{}
	\end{subfigure}
	\\
	\begin{subfigure}{0.49\linewidth}
		\includegraphics[width=\linewidth]{approach1-ex1-loss-normal}
		\caption{}
	\end{subfigure}
	\begin{subfigure}{0.49\linewidth}
		\includegraphics[width=\linewidth]{approach1-ex1-val-normal}
		\caption{}
	\end{subfigure}
	\caption[Resulting loss depending on width, depth and normalization]{The resulting loss depending on width, depth and normalization of training data. (a) Training loss without normalization; (b) Validation loss without normalization; (c) Training loss with normalization; (d) Validation loss with normalization.}
	\label{fig:approach1-ex1}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[width=0.9\linewidth]{approach1-ex1-progression}
	\caption[Progression of training loss for networks with $L=1$ and $K=24$]{Progression of training loss for networks with $L=1$ and $K=24$. Mean and standard derivation (std.) over ten identical networks trained separately.}
	\label{fig:approach1-ex1-progression}
\end{figure}

For evaluating depth $L$ and width $K$ of the network as well as the influence of normalization a number of models different in these hyperparameters were trained. While the choice for normalization is a binary one, $L \in \{1,3,\dots,9\}$ and $K \in \{9,12,\dots,24\}$ were used. The training happened for 1,000 epochs\footnote{%
	One epoch specifies the number of iterations during SGD until the network as seen each sample exactly one time. For 1,000 epochs the network has seen each sample 1,000 times, subsequently.}
and for each combination ten models are trained to get information about mean and variance. A total of $2 \cdot 5 \cdot 6 \cdot 10 = 600$ models were trained.

For evaluating the results, for each model the epoch with the minimum loss was selected. Afterwards the mean over all models with identical hyperparameters was taken forming the figures in~\ref{fig:approach1-ex1}. There are three observations:

\begin{enumerate}
	\item As expected, the training loss decreases for more powerful models with larger depth and width. In contrast, the structure of the validation loss looks different. This is an example of overfitting where the network is not fitted to the problem but the training data only.
	\item Just looking at the validation loss, the performance obviously worsens with increasing depth. Only increasing the width seems to have an influence on decreasing the loss. These are similar results to \cite{Niaki-2013}.
	\item There is no obvious advantage to using normalized data. Looking at figure~\ref{fig:approach1-ex1-progression}, the network in general converges faster using normalized data but also shows higher variance. The differences are neglectable, therefore normalization is not pursued further.
\end{enumerate}

\subsection{Regularization techniques}

\begin{quote}
	``Many strategies	used in machine learning are explicitly designed to reduce the test error, possibly at the expense of increased training error. These strategies are known collectively as regularization.''\cite[p.\,228]{Goodfellow-et-al-2016}
\end{quote}

Regularization is especially important in deep learning, where it is hard to visualize and understand what exactly was learned by the network. There is a large number of regularization techniques available, some of them exclusive for neural networks. Even a simple strategy like ``stopping the training when the validation error increases'' (\emph{early stopping}) is already some kind of regularization. This section mainly focuses on comparing three approaches: The widespread \emph{dropout regularization}, \emph{self-normalizing neural networks} (using SELUs as activation functions) and an attempt on \emph{data augmentation}, using minutely data for training.\footnote{%
	Data augmentation: Creating similar -- but not identical -- data from existing one, increasing the variance and hence making the network more robust. Even though the data was not newly generated the general idea is similar. By using some minute's futures as input, targeting the spread prices of the same minute at the next term, the network would learn a similar mapping. The number of samples increases by factor 1,440.}

\begin{figure}
	\centering
	\begin{subfigure}{0.49\linewidth}
		\includegraphics[width=\linewidth]{approach1-ex3-val-minutely}
		\caption{}
	\end{subfigure}
	\begin{subfigure}{0.49\linewidth}
		\includegraphics[width=\linewidth]{approach1-ex3-val-selu}
		\caption{}
	\end{subfigure}
	\caption[Validation loss for models trained with regularization techniques]{Validation loss for models trained with (a) minutely data and (b) SELU activations.}
	\label{fig:approach1-ex3}
\end{figure}

For evaluating the effectiveness of regularization the range of width $K$ and especially depth $L$ to search over was widened, except for minutely data. Using $L \in \{1,4,\dots,91\}$ and $K \in \{3,6,\dots,33\}$ a total of $30 \cdot 30 = 900$ models were trained. The large raise of the maximum depth was done in hope of finding a network with better abstraction capabilities, since the former experiments favored shallow networks. There was only one training run per model configuration, therefore neglecting statistical soundness.

For minutely data $L \in \{1, 4, \dots, 28\}$ and $K \in \{3,6,\dots,30\}$ were chosen. Further increasing the depth would have been computationally expensive. Because of similar reasons the batch size was set to 1024, also reducing the noise at each weight update. For each such configurations ten models were trained.

Looking at the results in figure~\ref{fig:approach1-ex3}, their implications do not seem much different from the former experiments. Again, for each model the epoch with minimum loss was selected:

\begin{enumerate}
	\item For dropout, the smallest loss was around $0.2$, much higher than all other models. Hence using dropout seems out of question. It neither allows for deeper networks nor does it result in an acceptable loss, regardless of configuration.
	\item Building self-normalizing neural networks with SELUs seems promising. But comparison with figure~\ref{fig:approach1-ex1} shows not much difference, concluding that in this case simple width is more important than some additional regularizations.
	\item Again, a small depth in connection with a large width seems to be the preferred configuration.
\end{enumerate}

\subsection{Additional inputs}

Remembering section~\ref{sec:seasonality}, there were some additional inputs which might hold important information. Adding these inputs to the existing models might decrease the loss under the conditions that these features indeed are useful and that the model is able to learn their underlying pattern.

Therefore two inputs are added: The corresponding day of month and the month itself, represented as simple natural numbers where January $\mapsto 1$, \dots, December $\mapsto 12$. Increasing the depth hurt performance in previous experiments, therefore depth $L = 1$ is chosen while width is set to $K \in \{20,23,\dots,50\}$. For each configuration ten models were trained, resulting in a total of $11 \cdot 10 = 110$ models. Furthermore, there were some training runs with normalized values but since the performance was always much worse these will not be evaluated further.

\begin{figure}[h!]
	\centering
	\includegraphics[width=0.9\linewidth]{approach1-ex4-val-basic}
	\caption[Validation loss using additional inputs with respect to width]{Validation loss for models with additional input. The x-axis shows the networks' width. Ten models were trained, shown by the thin lines. The thick blue line is the mean and the light blue area the standard derivation in either direction.}
	\label{fig:approach1-ex4}
\end{figure}

\newpage

The results are shown in figure~\ref{fig:approach1-ex4}. While mean and standard derivation seem reasonable, the loss itself only gains significance by comparing it to the other experiments. This is done in the following section.

\subsection{Comparison}
\label{sec:aao-comparison}

There are four models used for comparison, corresponding to the experiments described above:

\begin{enumerate}
	\item Basic network as presented in section~\ref{sec:aao-network-architecture} (Basic)
	\item Network using dropout regularization (Dropout)
	\item Self-normalizing network using SELUs (SNN)
	\item Basic network trained with minutely data (Minutely)
	\item Network with additional inputs for days and month (AddIn)
\end{enumerate}

All these networks have a depth $L=1$ and width $K=30$. Using a greater width might lead to marginally better performance. The training data is not normalized. Furthermore the learning rate is reduced by $\sqrt{0.1}$ up to two times if there is no improvement on the validation loss for 20 epochs. If there is still no improvement afterwards the training is stopped (\emph{early stopping}). As a baseline a naive prediction is used: It just assumes the futures' prices did not change for the next day, calculating spread prices and error from these.

By looking at table~\ref{tab:aao-comparison} -- especially the last column using the test dataset not used for network optimization -- the results are quite devastating. The error on most models is worse than for naive prediction. Only the self-normalizing network and the network trained with minutely data beat this by a negligible difference. Furthermore the training error is unusually high, often above the validation MSE. Even though the training data seems to have higher fluctuations -- likely because of the financial crisis in 2008 -- one could expect the networks to fit more to this dataset. Therefore this chapter's approach failed to deliver useful predictions.

\begin{table}
	\centering
	\caption[Comparison of the results obtained by the first approach]{Comparison of the results obtained by the first approach. Smaller error (MSE) indicated better performance. Because training and validation set were used for optimization the test MSE is most meaningful.}
	\begin{tabular}{llrrr}
		\toprule
		{} & Epoch &  Training MSE &  Validation MSE &  \textbf{Test MSE} \\
		\midrule
		Naive    &       &        0.2099 &          0.0985 &    0.1318 \\
		Basic    &   307 &        0.1386 &          0.1013 &    0.1563 \\
		Dropout  &   174 &        0.1895 &          0.2426 &    0.2984 \\
		SNN      &   242 &        0.1425 &          0.1077 &    \textbf{0.1259} \\
		Minutely &    70 &        0.2252 &          \textbf{0.0896} &    0.1294 \\
		AddIn    &   268 &        \textbf{0.1297} &          0.0997 &    0.1321 \\
		\bottomrule
	\end{tabular}
	\label{tab:aao-comparison}
\end{table}


\documentclass{scrartcl}

\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{siunitx}

\title{Higher-order finite elements for embedded simulation}
\subtitle{Errata}
\date{October 2021}
\author{Andreas Longva, RWTH Aachen University}

\begin{document}
\maketitle

This document details some errors discovered in our source code after the publication of our paper.

The code used to produce most of the results for our paper is available at the following URL:

\begin{center}
\url{https://github.com/InteractiveComputerGraphics/higher_order_embedded_fem}
\end{center}

\section*{Material parameters}
While rewriting and transfering some code to a new project, we discovered that our utility function for converting material parameters from Young's modulus and Poisson's ratio to corresponding Lamé parameters was off by a factor 4.

In short, when converting to the Lamé parameters $\lambda$ and $\mu$, our code used an incorrect formula for computing $\lambda$, whereas the formula used for $\mu$ was correct. The formulas we had implemented were:
\begin{align*}
\mu = \frac{E}{2 (1 + \nu)}, \qquad
\lambda^\text{bad} = \frac{1}{2} \cdot \frac{\mu \nu}{1 - 2 \nu}. \\
\end{align*}
The correct formula for $\lambda$ is
\begin{align*}
\lambda = 2 \cdot \frac{\mu \nu}{1 - 2 \nu} = 4 \lambda^\text{bad}.
\end{align*}

As a result, the parameters we give in the paper are not the \emph{true} parameters that were used for the simulation. With the incorrect formulas we can compute the Lamé parameters that were used for the simulation, and use the \emph{correct} conversion formulas the other way to compute corresponding \emph{effective} values for the Young's modulus and Poisson's ratio that would lead to the same Lamé parameters, and therefore the same simulation results. We obtain:
\begin{align*}
\nu^\text{eff} = \frac{\nu^\text{paper}}{4 - 6 \, \nu^\text{paper}},
\qquad
E^\text{eff} = E^\text{paper} \frac{1 + \nu^\text{eff}}{1 + \nu^\text{paper}}.
\end{align*}

The following table gives the original material parameters (as presented in the paper) and the corresponding effective values (up to 4 significant digits) for the experiments in which these material parameters were provided.

\begin{center}
\begin{tabular}{l | c | c}
\textbf{Experiment} & \textbf{Paper parameters} & \textbf{Effective parameters} \\
\hline
Quadrature verification (Section 5.4)
&
\begin{tabular}{l}
$E = \SI{3e6}{\pascal}$ \\ $\nu = 0.4$
\end{tabular}
& \begin{tabular}{l}
$E = \SI{2.679e6}{\pascal}$ \\ $\nu = 0.25$
\end{tabular}
\\
\hline
Twisting cylinder (Section 6.2)
&
\begin{tabular}{l}
$E = \SI{5e6}{\pascal}$ \\ $\nu = 0.48$
\end{tabular}
& \begin{tabular}{l}
$E = \SI{4.826e6}{\pascal}$ \\ $\nu = 0.4286$
\end{tabular}
\\
\hline
Armadillo slingshot (Section 6.5)
&
\begin{tabular}{l}
$E = \SI{5e5}{\pascal}$ \\ $\nu = 0.4$
\end{tabular}
& \begin{tabular}{l}
$E = \SI{4.464e5}{\pascal}$ \\ $\nu = 0.25$
\end{tabular}
\\
\hline
\end{tabular}
\end{center}

We see that the resulting effective parameters are (unfortunately) noticably different from the intended parameters. However, these parameter choices were more or less arbitrary to begin with, so it seems fair to say that it does not significantly change any of the conclusions made in the paper.

\section*{Twisting cylinder boundary conditions}

The twisting cylinder (Section 6.2) is reported in the paper to be 16 meters long. Recently, upon reviewing the code, we realized that the Dirichlet boundary conditions at the end were enforced for nodes with $y$-coordinate $|y| \geq 6.99$ instead of $|y| \geq 7.99$. Therefore the motion of the last meter on each side of the cylinder is prescribed. The experiment therefore effectively simulates a 14 meter long cylinder as opposed to the 16 meters described.

\section*{}
\end{document}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\begmath 16.2  Character-based Graphics --- Single Print Line

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

This subroutine constructs a character string that may be printed as part of a
single line image by the user's program to give a graphical representation of
data.  It is intended primarily for use as a supplement to ordinary tabular
output of data as an aid in spotting trends, wild points, etc.

\subsection{Usage}

\subsubsection{Program Prototype, Single precision}

\begin{description}

\item[INTEGER] \ {\bf NCHAR}

\item[REAL] \  {\bf Y, Y1, Y2}

\item[LOGICAL] \ {\bf RESET}

\item[CHARACTER*{\rm 1}] \ {\bf SYMBOL}

\item[CHARACTER*$n$] \ {\bf IMAGE} \ $[n \geq $ NCHAR]

\end{description}

Assign values to Y, SYMBOL, NCHAR, Y1, Y2, and RESET.

\begin{center}
\fbox{\begin{tabular}{@{\bf }c}
CALL SPRPL (Y, SYMBOL, IMAGE,\\
NCHAR, Y1, Y2, RESET)\\
\end{tabular}}
\end{center}

On return the string, IMAGE, contains the character SYMBOL positioned as a
function of the value of Y.  The user's program may then print IMAGE as part
of a line containing Y and possibly other information.

\subsubsection{Argument Definitions}

\begin{description}
\item[Y] \ [in]  Data value to be plotted.  Y should be between Y1 and Y2;
otherwise see Section E, Error Procedures.
\item[SYMBOL] \ [in]  A single character to be used as a plot symbol.  The character can
be specified literally in the call statement, as for example:
CALL SPRPL (Y, $^\prime $*$^\prime $, ...)
\item[IMAGE] \ [inout]  Character string in which a plot image is constructed.
\item[NCHAR] \ [in]  Number of character positions in IMAGE to be used in constructing
the plot image.  Require NCHAR $\geq  2.$
\item[Y1,Y2] \ [in]  Numbers that bracket the range of values of Y to be plotted in
IMAGE.  Either Y1 $\leq $ Y2 or Y1 $\geq $ Y2 is acceptable.
\item[RESET] \ [in]  Flag to reset the line image.  If RESET\ = .TRUE., the subroutine
will:

\begin{itemize}
\item[1.] Store NCHAR blank characters into IMAGE, and then store the character '0' in
the zero value position if zero is contained in the interval [$ymin$, $ymax$].  See
Section D.

\vspace{3pt}% Need to get decent column break

\item[2.] Store the character specified by SYMBOL in the Y value position.

\end{itemize}
If RESET = .FALSE., the subroutine will only execute Step~2 above.

\end{description}

\subsubsection{Modifications for Double Precision}

For double-precision usage change the REAL type statement to DOUBLE PRECISION,
and change the subroutine name from SPRPL to DPRPL.

\subsection{Examples and Remarks}

Print a set of ($x$, $y$)-data.  On the right side of the page print a
``strip chart" plot of the data with $x$ increasing downward and $y$
increasing to the right.  See DRSPRPL and ODSPRPL for code and output
illustrating this example.

\subsection{Functional Description}

\begin{itemize}
\item[1.] The subroutine will compute $ymin = \min$(Y1, Y2) and $ymax
= \max$(Y1, Y2).  Then, if $ymin = ymax$ these numbers will be
replaced by $ymin = 0.9 \times ymin$ and $ymax = 1.1 \times ymax$
if $ymin \neq 0.$, or by $ymin = -1$, and $ymax = +1$, if $ymin = 0$.

\item[2.] If zero is not in the interval [$ymin$, $ymax$], then the
scaling will be such that $ymin$ corresponds to the center of the
leftmost character position and $ymax$ corresponds to the center of
the rightmost character position.  If zero is in the interval
[$ymin$, $ymax$], and does not correspond to the first or last
character position in IMAGE(), then scaling will be adjusted so the
value zero corresponds to the center of a character position.  This
adjustment guarantees that values located symmetrically with respect
to zero will be plotted in character positions symmetrically located
with respect to the zero character position.
\end{itemize}

\subsection{Error Procedures and Restrictions}

A Y value outside the stated data range, [Y1, Y2], (plus a small tolerance)
will not be plotted, but the message 'OUT' is placed in either the left or
right end of IMAGE() as appropriate.  This message will be suppressed if
NCHAR $< 6.$
\enlargethispage*{40pt}
\subsection{Supporting Information}

The source language is ANSI Fortran~77.

Based on 1969 code by C.L. Lawson, JPL.  Adapted for MATH77 by C.L. Lawson and
S. Chiu, JPL, 1983.  At MATH77 Release~2.2, Nov.~1988, introduced DPRPL, and changed the
name of the previous PRPL to SPRPL.  Programs that were using PRPL should be
changed to use the name SPRPL.

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.2in} {\bf Required Files}\vspace{2pt} \\
DPRPL & \hspace{.35in} DPRPL\rule[-5pt]{0pt}{8pt}\\
SPRPL & \hspace{.35in} SPRPL\\\end{tabular}

\begcode

\medskip\
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRSPRPL}\vspace{10pt}
\lstinputlisting{\codeloc{sprpl}}

\vspace{20pt}\centerline{\bf \large ODSPRPL}\vspace{10pt}
\lstset{language={}}
\lstinputlisting{\outputloc{sprpl}}
\end{document}


%% Dexter Barrows, 2016
%% dbarrows.github.io

\section{SIRS R Function Code}

	R code to simulate the outlined SIRS function.

	\lstinputlisting[style=Rsty]{../../code/sir-functions/StocSIRS.r}

\section{SIRS HMC R Function Code}

	R code to simulate the outlined SIRS function with HMC state reconstruction.

	\lstinputlisting[style=Rsty]{../../code/sir-functions/StocSIRSstan.r}

\section{SMAP Code}

	This code implements an SMAP function on a user-provided time series.

	\lstinputlisting[style=Rsty]{../../code/smap/smap.r}

\section{SMAP Parameter Optimization Code}

	This code determines the optimal parameter values to be used by the S-map algorithm.

	\lstinputlisting[style=Rsty]{../../code/smap/smap_opt.r}

\section{RStan SIRS Code}

    This code implements a periodic SIRS model in Rstan.

    \lstinputlisting[style=Rsty]{../../code/hmc/hmc-sirs/sirs-euler.stan}

\section{IF2 SIRS Code}

    This code implements a periodic SIRS model using IF2 in C++.

    \lstinputlisting[style=Cppsty]{../../code/if2/if2-sirs/if2-sirs.cpp}


%!TEX root =  ../main.tex

\subsection{First Discontinuities}

\objective{Find informal limits at holes and non-holes.}

In this section, we will explore the informal definition of a \textbf{limit}.  Have you ever had anyone
sneer at you over a lost opportunity and say, ``Should of, would of, could of?''  (It probably
sounded like ``shoulda, woulda, coulda.'')  The idea is that everyone can see a hypothetical
in hindsight, though that does not avail you anything now.  Hypotheticals are situations describing
what was likely, or intended, or desired.  In mathematics, we often encounter functions
which appear \textit{as though} there is an expected value, only to have that spot not
even be part of the domain.

\personfeature[0in]{\chapdir/pics/Dedekind.jpg}{Richard Dedekind}{1831-1916}{was one of the greatest mathematicians of the nineteenth-century, as well as one of the most important contributors to algebra and number theory of all time. Any comprehensive history of mathematics will mention him for his investigation of the notions of algebraic number, field, ring, group, module, lattice, etc., and especially, for the invention of his theory of ideals.  He contributed greatly to the idea of continuity, which makes limits possible. \href{https://en.wikipedia.org/wiki/Richard_Dedekind}{Wikipedia}}

Limits are like a hypothetical situation in math.  Suppose your friend bought a ticket for a vacation,
and said they wanted to get away.  Then you didn't see this friend for a long time, and when you
finally saw them you asked, ``Did you ever go on vacation?  Where were you going to go?''
You know they  were going somewhere, but from your perspective, it has yet to be determined if
he or she actual went and where your friend was even headed.  Notice how your second question
doesn't even depend on whether they went anywhere or not: it is about a hypothetical.

Many functions give an indeterminate answer at certain gaps in their domain.  It is obvious
where they were ``headed'', but direct evaluation at that value of the independent variable is
not helpful.

\begin{derivation}{Indeterminate Form}
\index{indeterminate form}
For a given function $f(x)$, if $f(c)=\frac{0}{0}$, then the point $c$ represent a \textbf{hole}
in the domain of $f$.
\end{derivation}

\subsection{Algebra Techniques}
From the perspective of algebra, there are four techniques for constructing
precise replacements for many functions, replacements which are everywhere else the same,
but lack the particular hole.

\subsubsection{Cancelling}
Consider the function $f(x) = \frac{x^2-4}{x-2}$.    Enter it in your calculator and try \Touche[style=function,principal={ZOOM},]
\Touche[style=number, principal=6].  How can it be just a line?  Why isn't it more complicated than that?
Well, it is.  Try plugging in 2.  That is, what is $f(2)$?  $\frac{2^2-4}{2-2} = \frac{0}{0}$.

\begin{derivation}{Factor Removal}
When $f(c)=\frac{0}{0}$ and the removal of a common factor in the numerator and denominator yields a real number $d$,
then $(c,d)$ represents the location of a hole in $f(x)$.
\end{derivation}

Returning to our equation, $x^2-4$ in the numerator factors by the
Difference of Squares to $(x-2)(x+2)$.  This is means
we can write $f(x)=\frac{(x-2)(x+2)}{x-2}$.  It is \emph{not} true that
$\frac{x-2}{x+2} = x+2$: they differ in their domains.
However, $x+2$ is an \textbf{analytic continuation} of $\frac{x^2-4}{x-2}$,
meaning is is everywhere the same as the
original but has an \textit{even larger} domain.  We may use $x+2$ to answer the
question what $f(x)$ would output at $x=2$, were it to exist there.

\subsubsection{Expanding}
Some functions obscure the factor that could be cancelled with further arithmetic.  For example, it is not obvious
what $g(x)=\frac{(x+3)^3-27}{x}$ will be at $x=0$.\footnote{If you are especially keen,
you might notice that this is factorable as the difference of cubes in the numerator,
but we will pretend no one saw that for just a minute!}  Sometimes a small
piece of arithmetic allows us to proceed as in the previous section.  In this case $(x+3)^3=x^3+9x^2+27x+27$.

\begin{align*}
\frac{(x^3+9x^2+27x+27)-27}{x} &=\\
\frac{x(x^2+9x+27)}{x} & \approx x^2+9x+27\\
& \rightarrow (0)^2+9(0)+27 \\
&\rightarrow 27\\
\end{align*}

\subsubsection{Complex Fractions}
\emph{Not to be confused with fractions involving complex numbers!}

Besides simple arithmetic, complex fractions can obfuscate the cancelling
needed to simplify the presence
of a hole.  $\cfrac{2-x}{\frac{1}{x}-\frac{1}{2}}$ is indeterminate at $x=2$.
However, if we simplify this fraction until it has a simple (non-fraction) numerator and
denominator, the indeterminate form will evaporate.

\begin{align*}
\cfrac{2-x}{\frac{2}{2}\cdot\frac{1}{x}-\frac{1}{2}\cdot\frac{x}{x}} &= \cfrac{2-x}{\frac{2-x}{2x}} \\
&\approx \cfrac{2x(2-x)}{2-x} \\
&\approx 2x\\
&\rightarrow 2(2) = 4
\end{align*}

\begin{derivation}{Additional Factor}
When $f(c)=\frac{0}{0}$ and the multiplication by a common factor in the numerator and denominator yields a real number $d$,
then $(c,d)$ represents the location of a hole in $f(x)$.
\end{derivation}

\subsubsection{Conjugates}
What to multiply by can be difficult to decipher.  It often appears as though
we might wish to square individual terms  in the numerator or denominator.
For instance, the function $h(x) = \cfrac{\sqrt{x+1}-1}{x}$ is $\frac{0}{0}$ at $x=0$.
Clearly, it would be ideal to square only the upper-left corner of the fraction,
in order to remove the square root.  The
secret is to recognize the numerator is of the form $a-b$, where
$a=\sqrt{x+1}$ and $b=1$.  Any binomial multiplied
by its conjugate yields the difference of squares (i.e. $(a+b)(a-b)=a^2-b^2$).
In this case, we must multiply top and bottom by $\sqrt{x+1}+1$.

\begin{align*}
\cfrac{\sqrt{x+1}-1}{x} \cdot \cfrac{\sqrt{x+1}+1}{\sqrt{x+1}+1} &\approx \cfrac{(x+1)-1}{x(\sqrt{x+1}+1)}\\
&\approx \cfrac{1}{\sqrt{x+1}+1}\\
&\rightarrow \cfrac{1}{\sqrt{0+1}+1} = \frac{1}{2}
\end{align*}

\subsubsection{Direct Substitution}
Some time, there might seem to be a hole, but none exists.  In that case, we can simply plug
the input into the equation and get a result.

\inlinefig{\chapdir/pics/Continuidad_de_funciones_02.png}{Removable discontinuities are typically hole in the graph, places where there is no value to the function, but the expected value is straightforward to calculate.}


\section{Quantum Mechanics}

This section goes well beyond what is necessary for the PGRE.
Focus mainly on energy levels and probability as these are common themes on all the practice tests.
Perturbation theory is usually just one question but is typically straightforward.
Singlet and triplet spin states come up from time to time as well.
Again, any of this (and more) could be on the exam, but it would be unusual for some of the more advanced material to be there.\\*

\subsection{The Schr\"odinger Equation}

\subsubsection{Time-Dependent}
\(i\hbar\dot{\Psi}=H\Psi\)\\*
Solution: \(\Psi(\mathbf{r},t)=\Psi(\mathbf{r},0)e^{-iHt/\hbar}\)

\subsubsection{Time-Independent}
\(H\psi_n=E_n\psi_n\)\\*
Solution: \(\displaystyle\Psi(\mathbf{r},t)=\sum_{n=1}^{m}c_n\psi_n(\mathbf{r},0)e^{-iE_nt/\hbar}\) (\(m\) can go to \(\infty\))\\\\*
These eigenfunctions (\(\psi_n\)) are called stationary states.
Every expectation value is constant in time. (i.e. \(\langle\hat{p}\rangle=0\) because \(\langle \hat{x}\rangle=\mathrm{const.}\))

\subsubsection{Boundary Conditions}
\(\Psi\) and \(\nabla\Psi\) are both continuous.\\*
If \(V(\mathbf{r}_0)\to\pm\infty\) then only \(\Psi\) is continuous at \(\mathbf{r}_0\).

\subsubsection{Normalization}
\(\int_{-\infty}^{\infty}|\Psi(\mathbf{r},t)|^2\,\mathrm{d}\mathbf{r}=1\)

\subsection{General Information}

\subsubsection{de Broglie Wavelength}
For any particle: \(\displaystyle\lambda=\frac{h}{p}\to p=\hbar k\)

\subsubsection{Energy of a Photon}
\(E=h\nu=\frac{hc}{\lambda}\)

\subsubsection{Operators}
Any operator can be decomposed into Hermitian and anti-Hermitian parts:\\*
\(\displaystyle \Omega=\frac{\Omega+\Omega^{\dag}}{2}+\frac{\Omega-\Omega^{\dag}}{2}\)\\\\*
In the \(x\) basis:
\begin{itemize}
\item \(\hat{x}=x\)
\item \(\hat{p}=-i\hbar\frac{\partial}{\partial x}\)
\end{itemize}
\(\hat{L}_z=-i\hbar\frac{\partial}{\partial\phi}\)\\\\*
In the \(p\) basis:
\begin{itemize}
\item \(\hat{x}=i\hbar\frac{\partial}{\partial p}\)
\item \(\hat{p}=p\)
\end{itemize}

\subsubsection{Change of Basis}
\(x\) basis: \(\displaystyle\Psi(x,t)=\frac{1}{\sqrt{2\pi\hbar}}\int_{-\infty}^{\infty}e^{ipx/\hbar}\Phi(p,t)\,\mathrm{d}p\)\\\\*
\(p\) basis: \(\displaystyle\Phi(p,t)=\frac{1}{\sqrt{2\pi\hbar}}\int_{-\infty}^{\infty}e^{-ipx/\hbar}\Psi(x,t)\,\mathrm{d}x\)

\subsubsection{Commutation Relations}
\([A,B]=AB-BA\)\\*
\([A,B]_+=AB+BA\)\\*
\([AB,C]=A[B,C]+[A,C]B\)\\\\*
\([\hat{x},\hat{p}]=i\hbar\)\\*
\(\displaystyle [f(\hat{x}),\hat{p}]=i\hbar\frac{\mathrm{d}f}{\mathrm{d}x}\)\\*
\([\hat{L}_i,\hat{L}_j]=\epsilon_{ijk}i\hbar\hat{L}_k\)\\*
\([\hat{L}^2,\hat{L}_i]=0\)\\*
\([H,\hat{L}_i]=[H,\hat{L}^2]=0\)

\subsubsection{Uncertainty Principle}
Standard Deviation: \(\sigma_A=\sqrt{\langle A^2\rangle-\langle A\rangle^2}\)\\\\*
\(\sigma_A\sigma_B \geq \frac{1}{2}|\langle[A,B]\rangle|\)\\\\*
Common uncertainties:\\*
\(\sigma_x\sigma_p\geq\frac{\hbar}{2}\)\\*
\(\sigma_E\sigma_t\geq\frac{\hbar}{2}\)\\*
\(\sigma_{L_x}\sigma_{L_y}\geq\frac{\hbar}{2}|\langle L_z\rangle|\)

\subsubsection{Ehrenfest's Theorem}
\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}\langle\hat{Q}\rangle=\frac{i}{\hbar}\langle[H,\hat{Q}]\rangle+\left<\frac{\partial \hat{Q}}{\partial t}\right>\)\\\\*
Uses:\\*
\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}\langle\mathbf{p}\rangle=\langle-\nabla V\rangle\)\\*
\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}\langle\mathbf{L}\rangle=\langle\mathbf{r}\times(-\nabla V)\rangle\)
\newpage
\subsubsection{Probability}
Probability Density: \(\int_{-\infty}^{\infty} P(\mathbf{r})\,\mathrm{d}\mathbf{r}=\int_{-\infty}^{\infty} |\Psi(\mathbf{r})|^2\,\mathrm{d}\mathbf{r}\)\\*
Most Probable Value of \(r\): set \(\frac{\mathrm{d}}{\mathrm{d}r}|\psi(r)|^2r^2=0\), then solve for \(r\) (the \(r^2\) comes from \(\mathrm{d}\mathbf{r} = r^2\sin(\theta)\mathrm{d}r\mathrm{d}\theta\mathrm{d}\phi\))\\\\*
Probability Current: \(\displaystyle J(x,t)=\frac{i\hbar}{2m}\left(\psi\frac{\partial\psi^*}{\partial x}-\psi^*\frac{\partial\psi}{\partial x}\right)\)\\*
Probability of finding a particle in the range \(a<x<b\) at time \(t\): \(\displaystyle\frac{\mathrm{d}P_{ab}}{\mathrm{d}t}=J(a,t)-J(b,t)\)\\\\*
\(\displaystyle\langle H\rangle=\sum_{n=1}^{m}|c_n|^2E_n\)\\*
(same \(c_n\) as in \(\displaystyle\sum_{n=1}^{m}c_n\psi_n(\mathbf{r},0)e^{-iE_nt/\hbar}\))\\*
\(|c_n|^2\) tells you the probability that a measurement of the energy would yield the value \(E_n\).\\*
\(\displaystyle\sum_{n=1}^{m}|c_n|^2=1\)

\subsection{Common Solved Problems}
Be sure to study how each of these solutions look like when they are plotted (especially the first two).
Specifically, focus on how many ``nodes'' each eigenfunction has and where they are located.
When an infinite barrier is introduced to a potential only eigenfunctions with a ``node'' at that barrier survive (think ``wave on a string'').

\subsubsection{Infinite Square Well}
Potential: \[V(x) = \left\{
\begin{array}{l l}
  0 & \quad \mbox{\(0<x<a\)}\\
  \infty & \quad \mbox{otherwise}\\ \end{array} \right. \]
Eigenfunctions: \(\psi_n(x)=\sqrt{\frac{2}{a}}\sin(k_nx)\) where \(k_n=\frac{n\pi}{a}\), \(n=1,2,3,\ldots\)\\*
Energy Levels: \(\displaystyle E_n=\frac{\hbar^2k_n^2}{2m}=\frac{\hbar^2\pi^2}{2ma^2}n^2\)

\subsubsection{Harmonic Oscillator}
Potential: \(V(x)=\frac{1}{2}m\omega^2x^2\)\\*
Eigenfunctions: \(\psi_n(x)=\frac{1}{\sqrt{n!}}(a_+)^n\psi_0\) where \(a_+\) is the raising operator and\\*
\(\displaystyle\psi_0(x)=\left(\frac{m\omega}{\pi\hbar}\right)^{1/4}e^{-\frac{m\omega}{2\hbar}x^2}\)\\*
Energy Levels: \(\displaystyle\hbar\omega\left(n+\frac{1}{2}\right)\),  \(n=0,1,2,\ldots\)\\\\*
Raising and Lowering Operators: \(a_{\pm}=\frac{1}{\sqrt{2\hbar m\omega}}(\pm ip+m\omega x)\)\\*
\([a_-,a_+]=1\)\\*
\(H=\hbar\omega\left(a_-a_+-\frac{1}{2}\right)=\hbar\omega\left(a_+a_-+\frac{1}{2}\right)\)\\*
\(a_+\psi_n=\sqrt{n+1}\psi_{n+1}\)\\*
\(a_-\psi_n=\sqrt{n}\psi_{n-1}\)\\*
\(a_-a_+\psi_n=(n+1)\psi_{n}\)\\*
\(a_+a_-\psi_n=n\psi_{n}\)\\*
of course, \(a_-\psi_0=0\) and \(a_+\psi_{n_{highest}}=0\)

\subsubsection{Free Particle}
Potential: \(V(x)=0\)\\*
\(v_{classical}=v_{group}=2v_{phase}\)\\\\*
\(\displaystyle\Psi(x,t)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\phi(k)e^{i(kx-\frac{\hbar k^2}{2m}t)}\,\mathrm{d}k\)\\*
where \(\displaystyle\phi(k)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\Psi(x,0)e^{-ikx}\,\mathrm{d}x\)

\subsubsection{Delta-Function Potential}
Potential: \(V(x)=-\alpha\delta(x)\)\\*
Eigenfunction: \(\displaystyle\psi(x)=\frac{\sqrt{m\alpha}}{\hbar}e^{-m\alpha|x|/\hbar^2}\)\\*
Only One Bound State Energy: \(\displaystyle E_0=-\frac{m\alpha^2}{2\hbar^2}\)\\\\*
Reflection \& Transmission Coefficients:\\\\*
\(R+T=1\)\\\\*
\(\displaystyle R=\frac{1}{1+(E/|E_0|)}\)\\*
\(\displaystyle T=\frac{1}{1+(|E_0|/E)}\)

\subsubsection{Finite Square Well}
Potential: \[V(x) = \left\{
\begin{array}{l l}
  -V_0 & \quad \mbox{\(-a<x<a\)}\\
  0 & \quad \mbox{otherwise}\\ \end{array} \right. \]
With a wide, deep well the energies approach those of an infinite square well.\\*
\(\displaystyle E_n+V_0=\frac{\hbar^2k_n^2}{2m}\)\\\\*
With a shallow, narrow well there will always be at least one bound state no matter how weak the well is.
\newpage
\subsubsection{Hydrogen Atom}
Potential: \(\displaystyle V(r)=-\frac{e^2}{4\pi\epsilon_0}\frac{1}{r}\)\\*
The eigenfunctions (\(\psi_{nlm_l}(r,\theta,\phi)\)) are complicated and involve Laguerre polynomials and the spherical harmonics.
However, the ground state of the hydrogen atom is easy to remember.\\*
\(\displaystyle\psi_{100}(r)=\frac{1}{\sqrt{\pi a^3}}e^{-r/a}\) where \(a\) is the Bohr radius (\(a\approx .53\mathrm{\text{\AA}}\))\\*
Energy Levels: \(\displaystyle E_n=-\frac{E_1}{n^2}\) where \(E_1\approx 13.6\mathrm{eV}\)\\*
It is important to know that \(E_1\propto m_eZ_1^2Z_2^2\) where \(m_e\) is the mass of the orbiting body (electron), \(Z_1\) is the charge of the orbiting body (in units of electron charge), and \(Z_2\) is the charge of the central body (nucleus).\\\\*
ETS frequently makes you alter the energy level formula for positronium and helium.
Just replace \(m_e\) in \(E_1\) with the reduced mass \(\mu=\frac{m_e}{2}\) for positronium.
For helium, just remember \(Z_2\to 2\).

\subsection{Angular Momentum}
Orbital: \(\mathbf{L}\times\mathbf{L}=i\hbar\mathbf{L}\) (or \([\hat{L}_i,\hat{L}_j]=\epsilon_{ijk}i\hbar\hat{L}_k\))\\*
This means that one cannot have a completely determined angular momentum \emph{vector} just as one cannot completely determine both position and momentum.\\\\*
Spin: \(\mathbf{S}=\frac{\hbar}{2}\vec{\sigma}\)\\\\*
Pauli matrices: \(\sigma_x= \left[\!
  \begin{array}{ c c }
     0 & 1 \\
     1 & 0
  \end{array} \!\right]
\),
\(\sigma_y= \left[\!
  \begin{array}{ c c }
     0 & -i \\
     i & 0
  \end{array} \!\right]
\),
\(\sigma_z= \left[\!
  \begin{array}{ c c }
     1 & 0 \\
     0 & -1
  \end{array} \!\right]
\)\\\\*
It is convenient to express spin in terms of up/down vectors:\\*
Up: \(|\!\uparrow\rangle=\left[\!\begin{array}{c}1 \\ 0 \end{array}\!\right]\)\\*
Down: \(|\!\downarrow\rangle=\left[\!\begin{array}{c}0 \\ 1 \end{array}\!\right]\)\\*
\(\mathbf{S}\times\mathbf{S}=i\hbar\mathbf{S}\)\\\\*
Total: \(\mathbf{J}=\mathbf{L}+\mathbf{S}\)\\*
\(\mathbf{J}\times\mathbf{J}=i\hbar\mathbf{J}\)

\subsubsection{Raising and Lowering Operators}
\(\hat{L}_\pm=\hat{L}_x\pm i\hat{L}_y\)\\*
\([\hat{L}_z,\hat{L}_\pm]=\pm\hbar\hat{L}_\pm\)

\subsubsection{Eigenvalues}
\(\hat{L}^2|lm_l\rangle=l(l+1)\hbar^2|lm_l\rangle\), \(l=0,1,2,\ldots,n\)\\*
\(\hat{L}_z|lm_l\rangle=m_l\hbar|lm_l\rangle\), \(m_l=-l,-l+1,\ldots,0,\ldots,l-1,l\)\\*
\(\hat{L}_\pm|lm_l\rangle=A_l^{m_l}\hbar^2|l(m_l\pm 1)\rangle\)\\\\*
\(\hat{S}^2|sm_s\rangle=s(s+1)\hbar^2|sm_s\rangle\), \(s=0,1,2,\ldots\)\\*
\(\hat{S}_z|sm_s\rangle=m_s\hbar|sm_s\rangle\), \(m_s=-s,-s+1,\ldots,0,\ldots,s-1,s\)

\subsubsection{Addition of Angular Momentum}
\(s=1\) (triplet states):\\*
\(|11\rangle =\) \( \uparrow\uparrow\)\\*
\(|10\rangle =\) \(\frac{1}{\sqrt{2}}(\uparrow\downarrow+\downarrow\uparrow)\)\\*
\(|1(-1)\rangle =\) \(\downarrow\downarrow\)\\\\*
\(s=0\), \(m_s=0\) (singlet state):\\*
\(|00\rangle =\) \(\frac{1}{\sqrt{2}}(\uparrow\downarrow-\downarrow\uparrow)\)

\subsection{Time-Independent Perturbation Theory}
\(H=H_0+\lambda\Delta H\) where \(H_0\) is a solvable Hamiltonian with basis functions \(|n^{(0)}\rangle\)\\*
\(E_n=E_n^{(0)}+\lambda E_n^{(1)}+\ldots\)\\*
\(|n\rangle=|n^{(0)}\rangle+\lambda |n^{(1)}\rangle+\ldots\)

\subsubsection{First-Order Energy Correction}
\(E_n^{(1)}=\langle n^{(0)}|\Delta H|n^{(0)}\rangle\)

\subsubsection{First-Order Eigenfunction Correction}
\(\displaystyle |n^{(1)}\rangle=\sum_{k\neq n}\frac{\langle k^{(0)}|\Delta H|n^{(0)}\rangle}{E_n^{(0)}-E_k^{(0)}}|k^{(0)}\rangle\)\\*
The key point of this equation is \(\langle k^{(0)}|\Delta H|n^{(0)}\rangle\), which determines what new eigenfunctions will be zero (typically using even/odd symmetry arguments).


\documentclass[t,usenames,dvipsnames]{beamer}
\usetheme{Copenhagen}
\setbeamertemplate{headline}{} % remove toc from headers
\beamertemplatenavigationsymbolsempty

\usepackage{amsmath, sfmath, tikz, xcolor, pgfplots, array}
\pgfplotsset{compat = 1.16}
\usetikzlibrary{arrows.meta, calc, decorations.pathreplacing}

\title{Parabolas}
\author{}
\date{}

\AtBeginSection[]
{
  \begin{frame}
    \frametitle{Table of Contents}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
    \maketitle
\end{frame}

\section{Find the vertex, focus, and directrix for a parabola in standard form.}

\begin{frame}{Intro}
If we look at the graph of the quadratic function $f(x) = ax^2 + bx + c$, we obtain what is known as a \emph{parabola}.    \newline\\    \pause

A \alert{parabola} is the set of all points in the plane that are the same distance from the focus and the directrix line.
\end{frame}

\begin{frame}{}
\begin{center}
\begin{tikzpicture}
    \begin{axis}[
    axis lines = middle,
    xmin = -4,
    xmax = 12,
    ymin = 0,
    ymax = 10,
    xtick = \empty,
    ytick = \empty]
    \addplot [color=blue, <->, samples=200, domain=-0.5:6.5, line width = 1.5] {2 + 0.5*(x-3)^2} node[right]{\color{blue}{\textbf{Parabola}}};
    \addplot [mark = *] (3, 3) node[above]{Focus};
    \addplot [dashed, <->, samples=200, domain = -3:8] {1} node[right]{Directrix};
    \addplot [mark = *] (3, 2);
    \draw [<-] (3.25,2) -- (6,2.5) node[right]{Vertex};
    \draw [color=red] (3,1) rectangle (2.5,1.5);
    \draw [color=red, dashed] (3,3) -- (3,2) -- (3,1);
    \end{axis}
\end{tikzpicture}
\end{center}
\pause

The \alert{focal length} is the distance between the focus and vertex (or directrix and vertex) and is $|p|$.
\end{frame}

\begin{frame}{Equations}
    \begin{center}
    \setlength{\extrarowheight}{5pt}
    \begin{tabular}{c|c|c}
    &   \textbf{Opens Up or Down}   &   \onslide<2->{\textbf{Opens Left or Right}}    \\  \hline
    &   $(x-h)^2 = 4p(y-k)$  &   \onslide<2->{$(y-k)^2 = 4p(x-h)$}  \\[5pt]  \hline
    Vertex  &   $(h,k)$ &   \onslide<2->{$(h,k)$} \\[5pt]  \hline
    Focus Point   &   $(h, k+p)$    &   \onslide<2->{$(h+p, k)$}  \\[5pt] \hline
    Directrix   &   $y = k-p$   &   \onslide<2->{$x = h-p$}   \\
    \end{tabular}
\end{center}
\vspace{11pt}
\onslide<3->{
\emph{Note}: Sometimes the equations are written as
\[y = \frac{1}{4p}(x-h)^2 + k \text{ and } x = \frac{1}{4p}(y-k)^2 + h\].}
\end{frame}

\begin{frame}{Finding Vertex Without Technology}
    For $y = ax^2 + bx + c$  \\[8pt] \pause
    \begin{enumerate}
        \item $x$-coordinate: $-\frac{b}{2a}$   \\[8pt]   \pause
        \item $y$-coordinate: Evaluate at $x$-coordinate \\[18pt] \pause
    \end{enumerate}

    For $x = ay^2 + by + c$ \\[8pt] \pause
    \begin{enumerate}
        \item $y$-coordinate: $-\frac{b}{2a}$   \\[8pt] \pause
        \item $x$-coordinate: Evaluate at $y$-coordinate
    \end{enumerate}
\end{frame}

\begin{frame}{Example 1}
Find the vertex, focus, and directrix line for $(x+1)^2 = -8(y-3)$. \pause
Graph the parabola: \newline\\
\begin{minipage}{0.5\textwidth}
\begin{tikzpicture}[scale=0.4]
    \draw [color=gray, dashed] (-8,-5) grid (6,6);
    \draw [<->, >=stealth] (-8.5,0) -- (6.5,0);
    \draw [<->, >=stealth] (0,-5.5) -- (0,6.5);
    \foreach \x in {-8,-7,...,6}
    \draw (\x, 0.2) -- (\x,-0.2);
    \foreach \y in {-5,-4,...,6}
    \draw (0.2,\y) -- (-0.2,\y);
    \draw [<->, >=stealth, domain=-8:6, color=red, line width = 1.25] plot (\x, {-0.125*(\x+1)*(\x+1)+3});
    \onslide<3->{\draw[color=red, fill=red] (-1,3) circle (4pt);}
    \onslide<8->{\draw[color=blue, fill=blue] (-1,1) circle (4pt);}
    \onslide<10->{\draw[color=violet, <->, >=stealth, line width=1.25, dashed] (-8,5) -- (6,5);}
\end{tikzpicture}
\end{minipage}
\hspace{0.5cm}
\begin{minipage}{0.4\textwidth}
\onslide<4->{Vertex: $(-1,3)$} \\[8pt]
\onslide<5->{$4p = |-8|$} \\[8pt]
\onslide<6->{$p = 2$} \\[8pt]
\onslide<7->{Focus: $(-1, 3-2)$} \\[8pt]
\onslide<8->{Focus: $(-1,1)$} \\[8pt]
\onslide<9->{Directrix: $y = 3 + 2$} \\[8pt]
\onslide<10->{Directrix: $y = 5$}
\end{minipage}
\end{frame}

\begin{frame}{Example 2}
Find the vertex, focus, and directrix for $(y-2)^2 = 12(x+1)$.  \newline\\
\pause
\begin{minipage}{0.4\textwidth}
\begin{tikzpicture}[scale=0.5]
    \draw [dashed, gray] (-4,-3) grid (4.5,7.5);
      \draw[<->, >=stealth] (-4.5,0) -- (4.5,0) node[right] {$x$};
      \draw[<->, >=stealth] (0,-3) -- (0,7.5) node[above] {$y$};
      \foreach \x in {-4,-3,-2,...,4}
      \draw (\x, 0.1) -- (\x, -0.1);
      \foreach \y in {-3,-2,...,7}
      \draw (0.1,\y) -- (-0.1,\y);
      \draw[<->, >=stealth, line width = 1.25, domain=-3:7,smooth,variable=\y,red]  plot ({0.0833*\y*\y - 0.333*\y - 0.667},\y);
      \draw [color=red, fill=red] (-1,2) circle (4pt);
      \onslide<6->{\draw[color=blue, fill=blue] (2,2) circle (4pt);}
      \onslide<7->{\draw[<->,>=stealth, line width = 1.25, dashed, color=violet] (-4,-3.5) -- (-4,7.5);}
    \end{tikzpicture}
\end{minipage}
\hspace{0.6cm}
\begin{minipage}{0.48\textwidth}
\onslide<3->{Vertex: $(-2,1)$} \\[10pt]
\onslide<4->{$4p = 12$} \\[10pt]
\onslide<5->{$p = 3$} \\[10pt]
\onslide<6->{Focus: $(-1+3, 2) \rightarrow (2,2)$} \\[10pt]
\onslide<7->{Directrix: $x = -1-3$} \\[10pt]
\onslide<8->{$x = -4$} \\
\end{minipage}
\end{frame}

\section{Find the equation of the standard form of a parabola.}

\begin{frame}{Latus Rectum and Focal Diameter}
The \alert{latus rectum} of a parabola is a line segment through the focus point that is parallel to the directrix line.   \newline\\
\pause
The \alert{focal diameter} is the length of the latus rectum, and is $\mid 4p \mid$.  \newline\\
\pause
\begin{center}
    \begin{tikzpicture}[scale=0.8]
    \begin{axis}[
    axis lines = middle,
    xmin = -4,
    xmax = 12,
    ymin = 0,
    ymax = 10,
    xtick = \empty,
    ytick = \empty]
    \addplot [color=blue, <->, samples=200, domain=-0.5:6.5, line width = 1.5] {1 + 0.5*(x-3)^2};
    \addplot [mark = *] (3, 3);
    \addplot [dashed, domain=1:5, line width = 1.25] {3} node [midway, above, yshift=0.2cm] {L.R.};
    \draw [decoration = {brace}, decorate] (1,3.2) -- (5,3.2);
    \end{axis}
    \end{tikzpicture}
\end{center}
\end{frame}

\begin{frame}{Example 3}
Find the standard form of the parabola with focus $(2, 1)$ and directrix $y = -4$.   \newline\\
\begin{minipage}{0.6\textwidth}
\begin{tikzpicture}[scale=0.4]
\draw [gray, dashed] (-5,-5) grid (7,5);
\draw [<->, >=stealth] (-5.5,0) -- (7.5,0) node [right] {$x$};
\draw [<->, >=stealth] (0,-5.5) -- (0,5.5) node [right] {$y$};
\foreach \x in {-5,-4,...,7}
\draw (\x, 0.1) -- (\x, -0.1);
\foreach \y in {-5,-4,...,5}
\draw (0.1,\y) -- (-0.1,\y);
\draw [<->, >=stealth, color=blue, line width = 1.25] (-5,-4) -- (7,-4);
\draw [color=red, fill=red] (2,1) circle (4pt);
\onslide<2->{\draw [color=blue, fill=blue] (2,-4) circle (4pt);}
\onslide<8->{\draw [<->, >=stealth, domain = -3:7, line width = 1.25] plot (\x, {0.1*(\x-2)*(\x-2)-1.5});}
\onslide<3->{\draw [fill=black] (2,-1.5) circle (4pt);}
\end{tikzpicture}
\end{minipage}
\begin{minipage}{0.35\textwidth}
\onslide<4->{Vertex: $(2,-1.5)$}    \\[11pt]
\onslide<5->{$p = 2.5$} \\[11pt]
\onslide<6->{$4p = 10$} \\[11pt]
\onslide<7->{$(x-2)^2 = 10(y+1.5)$} \\
\end{minipage}
\end{frame}

\begin{frame}{Converting Equations}
To convert parabolas in $y = ax^2 + bx + c$ form to standard form, do the following:  \newline\\  \pause
\begin{enumerate}
    \item Find the coordinates of the vertex. This will give you $h$ and $k$.    \pause  \\[11pt]
    \item Use the relationship that $4p = \frac{1}{a}$.
\end{enumerate}
\end{frame}

\begin{frame}{Example 4a}
Find the vertex, focus, and directrix of the following. \newline\\
(a) \quad $y^2 + 4y + 8x = 4$   \newline\\  \pause
\begin{minipage}{0.4\textwidth}
\begin{tikzpicture}[scale=0.4]
    \draw [dashed, gray] (-2,-7) grid (3,3);
      \draw[<->, >=stealth] (-2.5,0) -- (3.5,0) node[right] {$x$};
      \draw[<->, >=stealth] (0,-7.5) -- (0,3.5) node[above] {$y$};
      \foreach \x in {-2,-1,...,3}
      \draw (\x, 0.2) -- (\x, -0.2);
      \foreach \y in {-7,-6,...,3}
      \draw (0.2,\y) -- (-0.2,\y);
      \draw[<->, >=stealth, line width = 1.25, domain=-7:3,smooth,variable=\y,red]  plot ({-0.125*(\y*\y + 4*\y - 4)},\y);
      \onslide<4->{\draw[color=red,fill=red] (1,-2) circle (4pt);}
      \onslide<8->{\draw[color=blue,fill=blue] (-1,-2) circle (4pt);}
      \onslide<10->{\draw[color=violet, line width=1.25, dashed, <->, >=stealth] (3,-7) -- (3,3);}
\end{tikzpicture}
\end{minipage}
\hspace{0.25cm}
\begin{minipage}{0.4\textwidth}
\onslide<3->{Vertex: $(1,-2)$} \\[10pt]
\onslide<5->{$4p = |8|$} \\[10pt]
\onslide<6->{$p = 2$} \\[10pt]
\onslide<7->{Focus: $(1-2, -2)$} \\[10pt]
\onslide<8->{Focus: $(-1,-2)$} \\[10pt]
\onslide<9->{Directrix: $x = 1 + 2$} \\[10pt]
\onslide<10->{Directrix: $x = 3$}
\end{minipage}
\end{frame}

\begin{frame}{Example 4b}
(b) \quad $x^2 - 2x + 5y = 1$   \newline\\  \pause
\begin{minipage}{0.4\textwidth}
\begin{tikzpicture}[scale=0.6]
    \draw[dashed, gray] (-3,-4) grid (5,2);
    \draw[<->,>=stealth] (-3.5,0)--(5.5,0) node [right] {$x$};
    \draw[<->,>=stealth] (0,-4.5)--(0,2.5) node [above] {$y$};
    \draw[<->,>=stealth,color=red,line width=1.25,domain=-3:5] plot (\x, {-0.2*(\x-1)*(\x-1)+0.4});
    \onslide<4->{\draw[color=red,fill=red] (1,0.4) circle (4pt);}
    \onslide<8->{\draw[color=blue,fill=blue] (1,-0.85) circle (4pt);}
    \onslide<10->{\draw[color=violet,<->,>=stealth,dashed,<->,>=stealth,line width = 1.25] (-3,1.65) -- (5,1.65);}
\end{tikzpicture}
\end{minipage}
\hspace{1.5cm}
\begin{minipage}{0.4\textwidth}
\onslide<3->{Vertex: $\left(1,\frac{2}{5}\right)$} \\[8pt]
\onslide<5->{$4p = |5|$} \\[8pt]
\onslide<6->{$p = 5/4$} \\[8pt]
\onslide<7->{Focus: $\left(1, \frac{2}{5}-\frac{5}{4}\right)$} \\[8pt]
\onslide<8->{Focus: $\left(1, -\frac{17}{20}\right)$} \\[8pt]
\onslide<9->{Directrix: $y = \frac{2}{5} + \frac{5}{4}$} \\[8pt]
\onslide<10->{Directrix: $y = \frac{33}{20}$} \\
\end{minipage}
\end{frame}

\section{Applications of Parabolas}

\begin{frame}{Paraboloid}
    If we rotate a parabola around its axis of symmetry, we obtain a 3-D model of a parabola called a \alert{paraboloid}, or a \alert{paraboloid of revolution}.    \newline\\  \pause

    The nature of paraboloids allows signals to be sent to or from the focus in a directed manner.
\end{frame}

\begin{frame}{Example 5}
    A satellite dish is to be constructed in the shape of a paraboloid of revolution. If the receiver placed at the focus is located 2 ft above the vertex of the dish, and the dish is to be 12 feet wide, how deep will the dish be?
\end{frame}

\begin{frame}{Example 5}
    If we place the vertex at the origin and open the graph upward, \onslide<2->{we get the equation $x^2 = 8y$}    \newline
    \begin{minipage}{0.5\textwidth}
        \begin{tikzpicture}[scale=0.45]
            \draw[dashed, gray] (-7,0) grid (7,6);
            \draw[<->,>=stealth] (-7.5,0) -- (7.5,0) node [right] {$x$};
            \draw[->,>=stealth] (0,0) -- (0,6) node [above] {$y$};
            \draw[<->,>=stealth,color=red,domain=-7:7,line width=1.25] plot ({\x, 0.125*\x*\x});
            \draw[color=blue,fill=blue] (0,2) circle (4pt);
            \onslide<3->{\draw[|<->|,>=stealth,color=violet] (0,5) -- (6,5) node [midway, above] {6 ft};}
            \onslide<5->{\draw[color=red,fill=red] (6,4.5) circle (4pt) node [right] {$(6,y)$};}
        \end{tikzpicture}
    \end{minipage}
    \hspace{1cm}
    \begin{minipage}{0.3\textwidth}
    \begin{align*}
        \onslide<4->{x^2 &= 8y} \\[10pt]
        \onslide<6->{6^2 &= 8y} \\[10pt]
        \onslide<7->{36 &= 8y} \\[10pt]
        \onslide<8->{y &= \frac{36}{8}} \\[10pt]
        \onslide<9->{y &= \frac{9}{2}} \\
    \end{align*}
    \end{minipage}
\end{frame}

\end{document}


% Copyright 2021 Google LLC
%
% Use of this source code is governed by an MIT-style
% license that can be found in the LICENSE file or at
% https://opensource.org/licenses/MIT.

%!BIB program = biber
%!TeX program = lualatex
%!TeX spellcheck = en-US

\documentclass[hctr2.tex]{subfiles}
\begin{document}
\section{HCTR issues}
Our presentation of HCTR2 uses
names that differ from those used to present HCTR.

\begin{itemize}
    \item HCTR and HCTR2 use an unusual mode of stream encryption.
    \cite{hctr,hctrquad} refer to this mode as ``CTR mode'',
    but note the differences between this mode and
    standard CTR mode\cite{ctr}.
    For the avoidance of ambiguity
    we name this mode \emph{XCTR mode}.
    \item What \cite{hctr,hctrquad} refer to as \(C, \mathit{CC}, D\)
    we refer to as \(U, \UU, V\)
    so that we can use \(P, C\) to refer to plaintext and ciphertext.
    We will use our names in what follows.
    \item Because of our use of POLYVAL, HCTR2 draws a distinction
    between the raw hash key \(\hgen\)
    and the value at which the polynomial is evaluated \(\hpoly\).
    HCTR has no such distinction
    and we use \(h\) in our discussion of HCTR.
\end{itemize}

Two errors in previous work on HCTR are addressed in HCTR2.

\subsection{Hash function}\label{badhash}
HCTR uses a hash function based on the polynomial
\begin{displaymath}
    H(X) = \poly(\pad(X) \Concat \fromint(\abs{X}) \Concat 0^n)
\end{displaymath}
Because it assumes a fixed-length tweak it simply sets \(X = M \Concat T\).
However, HCTR requires that the resulting polynomial be nonzero
even when \(X = \lambda\),
so as a special case \(H(\lambda) = h\).

Unfortunately, as \cite{kumarhctr} observes this is no longer
an injective map from \(X\)---we also have \(H(0) = h\). This
breaks the almost-XOR-universal property relied on
in the security bound and
straightforwardly leads to an attack in which two encryption queries
are presented, one of a block width, and the second extending
the first with a single zero (and assuming a zero length tweak).

\subsection{Security bound}\label{badproof}
HCTR was initially presented in \cite{hctr} with a security bound cubic
in the total size of all queries combined.  This is a little low
for comfort; if a 128-bit block cipher is used, it suggests a
distinguisher can be effective given tens of terabytes of queries,
which can quickly be reached over a fast link.
\cite{hctrquad} presents a much more satisfactory quadratic bound,
but the proof has an error that invalidates the claimed bound.

While presented in a different way,
the proof of~\cite{hctrquad} is very similar
to that of this paper: in the game RAND2, all queries get random
responses, the block cipher inputs are inferred, and a collision
in either the inferred plaintext or inferred ciphertext of the block cipher
sets the ``bad'' flag.
Where HCTR2 has \(S = \MM \xor \UU \xor L\),
HCTR simply has \(S = \MM \xor \UU\).
HCTR uses a fixed-length tweak;
for simplicity we assume a zero-length tweak in what follows.

For equation 17, the paper observes that the collision
\(Y_i^r \qeq \UU^s\) occurs iff \(h\) is one of the zeroes
of the polynomial \(Y_i^r \xor U^s \xor H(T^s, V^s)\).
This polynomial has degree at most
\(m^s\), and so can have
at most that many solutions. From this the paper infers
a quadratic bound on the probability of any such collision
given \(\sigma\) input blocks.

In equation 21, the paper considers collisions of the form
\(S_i^r \qeq \MM^s\) and asserts that they are
quadratically bounded for the same reason. However this equation
is crucially different: unlike with \(Y_i^r\), the value we
infer for \(S_i^r\) depends on \(h\). The values of \(h\)
for which this collision occurs are
the zeroes of the polynomial
\begin{displaymath}
    M^r \xor H(T^r, N^r)
    \xor U^r \xor H(T^r, V^r) \xor \fromint(i)
    \xor M^s \xor H(T^s, N^s)
\end{displaymath}
This polynomial can have degree up to
\(\max(m^r, m^s)\), and so the bound of
equation 17 does not apply.
If queries are permitted to be of any length,
this leads to a cubic security bound.
Consider an adversary who
sends a single query with \(x+1\) blocks, followed by
\(x\) queries of one block. For each \(i \in \{1\ldots x\}\) and
for each \(s \in \{2 \ldots x + 1\}\), we have at best
\(\prob{S_i^r = \MM^s} \leq (x + 1)/2^n\). Summing
all these bounds for each such pair, we find that
for queries with \(\sigma = 2x + 1\)
this technique yields an upper bound on
the total probability of such a collision of
\((x^3 + x^2)/2^n\).

\cite{nandimail} observes that a quadratic bound can be recovered
if a bound \(l_\mathrm{max}\) is set on the maximum size of
a single query; in this case we can prove a bound
which is some small multiple of \(l_\mathrm{max}q\sigma/2^n\).
\end{document}


%!TEX root = ./egpaper_for_review.tex
%
\section{Introduction}
Correlation clustering~\cite{Bansal-2002}, also known as the multicut problem~\cite{chopra_1993_mp}
is a basic primitive in computer vision~\cite{andres_2011_iccv,kroeger_2012_eccv,yarkony_2012_eccv,alush_2013_simbad} and data mining~\cite{Arasu-2009,Sadikov-2010,Chen-2012,Chierichetti-2014}.
See Sec.~\ref{sec:problem_formulation} for its formal definition of clustering the nodes of a graph.

Its merit is, firstly, that it accommodates both positive (attractive) \emph{and} negative (repulsive) edge weights.
This allows doing justice to evidence in the data that two nodes or pixels do not wish or do wish to end up in the same cluster or segment, respectively.
Secondly, it does not require a specification of the number of clusters beforehand.

In signed social networks, where positive and negative edges encode friend and foe relationships, respectively,
correlation clustering is a natural way to detect communities~\cite{Chen-2012,Chierichetti-2014}.
Correlation clustering can also be used to cluster query refinements in web search~\cite{Sadikov-2010}.
Because social and web-related networks are often huge, heuristic methods, \eg the PIVOT-algorithm~\cite{Ailon-2008},
are popular~\cite{Chierichetti-2014}.

In computer vision applications, unsupervised image segmentation algorithms often start with an over-segmentation
into superpixels (superregions), which are then clustered into ``perceptually meaningful''
regions by correlation clustering.
Such an approach has been shown to yield
state-of-the-art results on the Berkeley Segmentation Database
\cite{andres_2011_iccv,Kim-2011,yarkony_2012_eccv,alush_2013_simbad}.

While it has a clear mathematical formulation and nice properties,
correlation clustering suffers from NP-hardness.
%
Consequently, partition problems on large scale data, \eg
huge volume images in computational neuroscience~\cite{kroeger_2012_eccv}
or social networks~\cite{Leskovec-2010},
are not tractable because reasonable solutions cannot be computed in acceptable time.

% Importantly, this allows doing justice to evidence in the data that two nodes or pixels do \emph{not} wish to end up in the same cluster or segment. This is in contrast to the submodular potentials so popularized by the graph cut algorithm, which only allow two nodes to be attracted to each other, or at most be agnostic about membership in the same cluster. Secondly, the algorithm does not require a specification of the number of clusters. This is in contrast to methods such as normalized cut \cite{} that can only accommodate attractive interactions and hence need the number of clusters to be specified.

% \paragraph{Contribution.} The evident usefulness of correlation clustering and its clean and compact formulation in terms of an optimization problem, together with its unfortunate NP-hardness, are an invitation to develop fast approximate solvers. The basic idea of the move making algorithm proposed here is to maintain, at all times, a best current partitioning; and to iteratively improve it (as we show, monotonously) by considering diverse and cheaply generated proposal partitionings. Any two nodes that are in one cluster in both the current and the proposal partitioning are contracted. Edges to contracted nodes are equally combined and reweighted, and the correlation clustering problem is then solved on this reduced graph. We offer a polyhedral characterization of this strategy, and evaluate it in conjunction with two versatile proposal generators.   We conduct experiments on a broad range of data sets ranging from 2D and 3D segmentation problems to clustering in signed social networks. The results suggest that this simple move making algorithm is the fastest technique known today, reaching close to globally optimal solutions in a tenth or a hundredth of the time required by the most efficient exact solvers.
\vspace{0.1cm}
\noindent \textbf{Contribution.}
In this work we present novel approaches that are designed for large scale correlation clustering problems.
First, we define a novel energy based agglomerative clustering algorithm that monotonically increases the energy.
With this at hand we show how to improve the anytime performance of Cut, Clue \& Cut~\cite{beier_2014_cvpr}.
%
Second, we improve the anytime performance of polyhedral multicut methods~\cite{kappes_2013_arxiv} by more efficient separation procedures.
%
Third, we introduce cluster-fusion moves, which extend the original fusion moves~\cite{Lempitsky-2010}
used in supervised segmentation to the unsupervised case and give a polyhedral interpretation of this algorithm.
Finally, we propose two versatile proposal generators, and evaluate the proposed methods on existing and new benchmark problems.
Experiments show that we can improve the computation time by one to two magnitudes without worsening the segmentation
quality significantly.

\vspace{0.1cm}
\noindent \textbf{Related Work.}
A natural approach is to solve the integer linear program (ILP) directly~\ref{eq:edgeproblem}.
To this end, efficient separation procedures have been found~\cite{kappes_2011_emmcvpr,kappes_2013_arxiv} that allow to iteratively augment the set of constraints until a valid partitioning is found.
Alternatively, it is possible to relax the integrality constraints of the ILP formulation~\cite{kappes_2013_arxiv}.
Such an outer relaxation can be iteratively tightened. However, intermediate solutions are fractional and therefore rounding is required to obtain a valid partitioning.
For the latter approach column generating methods exist, which work best on planar graphs~\cite{yarkony_2012_eccv}. %or almost planar \cite{yarkony-andres-2013} graphs.

Another line of work uses move making algorithms
to optimize correlation clustering~\cite{Kernighan-1970,bagon_2011_arxiv,beier_2014_cvpr}.
Starting with an initial segmentation, auxiliary max-cut problems are (approximately) solved,
such that the segmentation is strictly improved.
As shown in~\cite{beier_2014_cvpr} only Cut, Glue \& Cut (CGC)
can deal with large scale problems, but can also suffer from very large auxiliary problems.

Outside computer vision, greedy methods~\cite{Soon-2001,Ng-2002,Gionis-2007,Elsner-2008,Ailon-2008} have been suggested for correlation clustering problems, see \cite{Elsner-2009} for an overview.
The PIVOT Algorithm~\cite{Ailon-2008} iterates over all nodes in random order.
If the node is not assigned it constructs a cluster containing the node and all its
unassigned positively linked neighbors.
%
A widely used post-processing method is  Best One Element Move (BOEM)~\cite{Gionis-2007}, which iteratively reassigns nodes to clusters.

For energy minimization problems fusion moves have become increasingly popular~\cite{Lempitsky-2010,kappes_2014_ws}.
For many large scale computer vision applications fusion moves lead to good approximations
with state of the art anytime performance~\cite{kappes_2014_ws}.
Due to the ambiguity of a node-labeling, classical fusion moves~\cite{Lempitsky-2010} cannot be applied directly for correlation clustering.
We will show how to overcome this problem in Sec.~\ref{sec:cc_fm}.

\vspace{0.1cm}
\noindent \textbf{Outline:}
In Sec.~\ref{sec:problem_formulation} we give a
detailed problem definition and introduce
the correlation clustering objective.
Next we give a description of energy based hierarchical clustering in Sec.~\ref{sec:ehc} and
our proposed correlation clustering fusion moves in Sec.~\ref{sec:cc_fm}.
We evaluate the proposed methods in Sec.~\ref{sec:exp} and conclude in Sec.~\ref{sec:future} and \ref{sec:conclusion}.

%-------------------------------------------------------------------------

\input{inputs/fig_notation.tex}
\section{Notation and Problem Formulation}\label{sec:problem_formulation}
Let $G=(V,E, w)$ be a weighted graph of nodes $V$ and edges $E$.
%
The function $w : E \rightarrow \mathbb{R}$ assigns a weight to each edge.
We will use $w_e$ as a shorthand for $w(e)$.
A positive weight expresses the desire that two adjacent nodes should
be merged, whereas a negative weight indicates
that these nodes should be separated into two distinct regions.
%
%A \emph{subgraph} $G_A = \{A, E_A, w\}$ consists
%of nodes $A \subseteq V$ and edges $E_A := E\cap (A\times A)$.
%
%\input{inputs/fig_notation.tex}
A segmentation of the graph $G$ can be either given by a
node labeling $l \in \mathbb{N}^{|V|}$
or an edge labeling $y \in\{0,1\}^{|E|}$, \cf Fig.~\ref{fig:notation}.
An edge labeling is only consistent if it does not violate any cycle constraint~\cite{chopra_1993_mp}.
We denote the set of all consistent edge labelings by $P(G)\subset\{0,1\}^{|E|}$.
The convex hull of this set is known as the \emph{multicut polytope} $MC(G) = \textrm{conv}(P(G))$.
%
By $l(y)$ we denote some node labeling for a segmentation given by $y$.

Given a weighted graph $G=(V,E,w)$ we consider the problem of segmenting $G$ such that the costs
of the edges between distinct segments is minimized. This can be formulated in the node domain
by assigning each node $i$ a label $l_i \in \mathbb{N}$
\begin{align}
  l^* &= \argmin_{l \in \mathbb{N}^{|V|}} \sum_{ (i,j) \in E } w_{ij} \cdot [l_{i} \neq l_{j}], \label{eq:nodeproblem}
\end{align}
or in the edge domain, by labeling each edge $e$ as cut $y_e=1$ or uncut $y_e=0$
\begin{align}
  y^* &= \argmin_{y \in P(G)} \sum_{ (i,j) \in E } w_{ij} \cdot y_{ij} \label{eq:edgeproblem}.%\\
\end{align}
As shown in~\cite{kappes_2013_arxiv}  both problems are equivalent, but formulation \ref{eq:nodeproblem}
suffers from ambiguities in the representation, \cf Fig.~\ref{fig:notation}.

%\input{inputs/fig_notation.tex}


\documentclass[12pt,letterpaper]{article}
\author{Brian B. Maranville}
\title{Polarization correction for neutron reflectometry with transmission spin filters}
\usepackage[utf8]{inputenc}
\usepackage[margin=1.0in]{geometry}
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{mathtools}
\begin{document}
\maketitle

\section{Adaptation from source}
Adaptation of polarization efficiency corrections from
``Neutron scattering studies of magnetic thin films and multilayers''

C.F. Majkrzak (1996). Physica B 221, 342-356

\url{http://dx.doi.org/10.1016/0921-4526(95)00948-5}
\subsection{flipper efficiency}
For any neutron-polarizing optical system, a spin-filter is chosen which selects a particular spin
state which we will call the dominant state.

The transmitted intensity of the dominant spin state through the polarizing optics (at the sample)
is proportional to $(1 + F)$ when the front flipper is off and $(1 - F(1 - 2f))$ when the flipper is on;
the transmitted intensity of the other (leakage) spin state is $(1 - F)$ for flipper off and $(1 + F(1 - 2f))$
for flipper on, where $F, f$ are the efficiencies of the polarizer device and the flipper, respectively.
$F$ and $f$ are defined such that $0 \leq F \leq 1$ and $0 \leq f \leq 1$, so that in the above
equations, for a perfect polarized and flipper the transmission would be 1 for the dominant spin state
with the flipper off, with 0 leakage, and vice-versa with the flipper off.  The same set of equations holds
for the transmission through the spin-analyzer optics after the beam interacts with the sample: the analyzer
efficiency and rear flipper efficiency will be written as $R, r$.

In the reference above, the math is defined for a supermirror polarizer and analyzer in the reflection geometry.
This corresponds to a dominant transmitted neutron spin state of $(+)$ when a flipper is
off and $(-)$ when a flipper is on.

\subsection{spin-filter geometry: transmit or reflect}
To apply the equations in Table 2 of that reference to a system in which the supermirrors are
operated in a transmission geometry, we have to reverse the sign of all the spin-dependent cross-sections described,
(all the subscripts on the $\sigma$) while keeping the relationship of transmission efficiencies to the flipper state
(off vs. on) the same, e.g.
\begin{equation}
\begin{array}{rll}
	I^{\textrm{off off}}_\textrm{reflect} / \beta =& {} & \sigma_{++} (1 + F)(1 + R) \\
	 {} & + & \sigma_{-+} (1 - F)(1 + R) \\
	 {} & + & \sigma_{--} (1 - F)(1 - R) \\
	 {} & + & \sigma_{+-} (1 + F)(1 - R) \\
	 {} & {} & {} \\
	 I^{\textrm{off on}}_\textrm{reflect} / \beta =& {} & \sigma_{++} (1 + F)[1 + R(1-2r)] \\
	 {} & + & \sigma_{-+} (1 - F)[1 + R(1-2r)] \\
	 {} & + & \sigma_{--} (1 - F)[1 - R(1-2r)] \\
	 {} & + & \sigma_{+-} (1 + F)[1 - R(1-2r)] \\
\end{array}
\end{equation}
for a reflection polarizer/analyzer in the reference becomes
\begin{equation}
\begin{array}{rll}
	I^{\textrm{off off}}_\textrm{transmit} / \beta =& {} & \sigma_{--} (1 + F)(1 + R) \\
	 {} & + & \sigma_{+-} (1 - F)(1 + R) \\
	 {} & + & \sigma_{++} (1 - F)(1 - R) \\
	 {} & + & \sigma_{-+} (1 + F)(1 - R) \\
	 {} & {} & {} \\
	 I^{\textrm{off on}}_\textrm{transmit} / \beta =& {} & \sigma_{--} (1 + F)[1 + R(1-2r)] \\
	 {} & + & \sigma_{+-} (1 - F)[1 + R(1-2r)] \\
	 {} & + & \sigma_{++} (1 - F)[1 - R(1-2r)] \\
	 {} & + & \sigma_{-+} (1 + F)[1 - R(1-2r)] \\
\end{array}
\end{equation}
and similarly for the other two flipper states...
\begin{equation}
\begin{array}{rll}
	I^{\textrm{on off}}_\textrm{transmit} / \beta =& {} & \sigma_{--} [1 + F(1-2f)](1 + R) \\
	 {} & + & \sigma_{+-} [1 - F(1-2f)](1 + R) \\
	 {} & + & \sigma_{++} [1 - F(1-2f)](1 - R) \\
	 {} & + & \sigma_{-+} [1 + F(1-2f)](1 - R) \\
	 {} & {} & {} \\
	 I^{\textrm{on on}}_\textrm{transmit} / \beta =& {} & \sigma_{--} [1 + F(1-2f)][1 + R(1-2r)] \\
	 {} & + & \sigma_{+-} [1 - F(1-2f)][1 + R(1-2r)] \\
	 {} & + & \sigma_{++} [1 - F(1-2f)][1 - R(1-2r)] \\
	 {} & + & \sigma_{-+} [1 + F(1-2f)][1 - R(1-2r)] \\
\end{array}
\end{equation}
For a transmitting polarizer/analyzer, we will identify the measured cross-sections as
\begin{equation}
	\mathbf{I} =
	\begin{pmatrix}
		I^{++} \\
		I^{-+} \\
		I^{+-} \\
		I^{--}
	\end{pmatrix}
	=
	\begin{pmatrix}
		I^\textrm{on on} \\
		I^\textrm{off on} \\
		I^\textrm{on off} \\
		I^\textrm{off off}
	\end{pmatrix}_\textrm{transmit}
\end{equation}

\subsection{matrix for transmission spin-filters}
Then re-arranging the cross-sections from the reference into
$\{(++), (+-), (-+), (--)\}$ (our preferred order)
we can write the equations as a matrix:
\begin{equation}
\begin{array}{c}
	 A \cdot \boldsymbol{\sigma} = \mathbf{I} \\[1em]

  A =
  \begin{pmatrix*}[l]
	(1-Fx)(1-Ry) &\!\!\! (1-Fx)(1+Ry) &\!\!\! (1+Fx)(1-Ry) &\!\!\! (1+Fx)(1+Ry) \\
	(1-Fx)(1-R) &\!\!\! (1-Fx)(1+R) &\!\!\! (1+Fx)(1-R) &\!\!\! (1+Fx)(1+R) \\
	(1-F)(1-Ry) &\!\!\! (1-F)(1+Ry) &\!\!\! (1+F)(1-Ry) &\!\!\! (1+F)(1+Ry) \\
	(1-F)(1-R) &\!\!\! (1-F)(1+R) &\!\!\! (1+F)(1-R) &\!\!\! (1+F)(1+R)
  \end{pmatrix*}
\end{array}
\end{equation}
where $x = (1 - 2f), y = (1-2r)$.  From here, we can invert $A$ to get back the intrinsic
scattering cross-sections $\boldsymbol{\sigma}$ by
\begin{equation}
	\boldsymbol{\sigma} = A^{-1} \cdot \mathbf{I}
\end{equation}

\subsection{reduced matrices}
At times, when measuring some assumptions are made about the samples in question which can
simplify the equations and reduce the measurement time
\subsubsection{spin-flip cross-sections equal, nonzero}

In this case, $\sigma_\textrm{SF} \equiv \sigma_{+-} = \sigma_{-+}$ and there are only three
unknowns to solve for.  We can then measure just one spin-flip (SF) cross-section and solve for
the reflectivity $\boldsymbol{\sigma}$
\begin{equation}
\begin{array}{c}
	A^{+-} \cdot
	\begin{pmatrix}
	 	\sigma_{++} \\
	 	\sigma_\textrm{SF} \\
	 	\sigma_{--}
	 \end{pmatrix}
	 =
	 \begin{pmatrix}
		I^{++} \\
		I^{+-} \\
		I^{--}
	\end{pmatrix}  \\[2em]

  A^{+-} =
  \begin{pmatrix*}[l]
	(1-Fx)(1-Ry) &\!\!\! [(1-Fx)(1+Ry) + (1+Fx)(1-Ry)] &\!\!\! (1+Fx)(1+Ry) \\
	(1-Fx)(1-R) &\!\!\! [(1-Fx)(1+R) + (1+Fx)(1-R)] &\!\!\! (1+Fx)(1+R) \\
	(1-F)(1-R) &\!\!\! [(1-F)(1+R) + (1+F)(1-R)] &\!\!\! (1+F)(1+R)
  \end{pmatrix*}
\end{array}
\end{equation}
or similarly
\begin{equation}
\begin{array}{c}
	A^{-+} \cdot
	\begin{pmatrix}
	 	\sigma_{++} \\
	 	\sigma_\textrm{SF} \\
	 	\sigma_{--}
	 \end{pmatrix}
	 =
	 \begin{pmatrix}
		I^{++} \\
		I^{-+} \\
		I^{--}
	\end{pmatrix}  \\[2em]

  A^{-+} =
  \begin{pmatrix*}[l]
	(1-Fx)(1-Ry) &\!\!\! [(1-Fx)(1+Ry) + (1+Fx)(1-Ry)] &\!\!\! (1+Fx)(1+Ry) \\
	(1-F)(1-Ry) &\!\!\! [(1-F)(1+Ry) + (1+F)(1-Ry)] &\!\!\! (1+F)(1+Ry) \\
	(1-F)(1-R) &\!\!\! [(1-F)(1+R) + (1+F)(1-R)] &\!\!\! (1+F)(1+R)
  \end{pmatrix*}
\end{array}
\end{equation}

\subsubsection{spin-flip cross-sections zero}

If the spin-flip cross-sections are expected to be vanishingly small
(which occurs when the in-plane magnetization is strictly parallel or antiparallel to
the field quantization axis) then a further simplification is possible:
\begin{equation}
\begin{array}{c}
	A^\textrm{NSF} \cdot
	\begin{pmatrix}
	 	\sigma_{++} \\
	 	\sigma_{--}
	 \end{pmatrix}
	 =
	 \begin{pmatrix}
		I^{++} \\
		I^{--}
	\end{pmatrix}  \\[2em]

  A^\textrm{NSF} =
  \begin{pmatrix*}[l]
	(1-Fx)(1-Ry) &\!\!\!  (1+Fx)(1+Ry) \\
	(1-F)(1-R) &\!\!\! (1+F)(1+R)
  \end{pmatrix*}
\end{array}
\end{equation}

\subsection{Calibration}
In order to do the polarization correction as described, one has to first determine the parameters
$F,R,f,r$.  These are done exactly as described in the reference; the only thing to note is the
identification of the measured cross-sections for transmission spin-filters:
\begin{equation}
I^{--} \equiv I^\textrm{off off}_\textrm{NS,transmit} = \alpha[FR + 1]
\end{equation}
\begin{equation}
I^{+-} \equiv I^\textrm{on off}_\textrm{NS,transmit} = \alpha[FR(1-2f) + 1]
\end{equation}
\begin{equation}
I^{-+} \equiv I^\textrm{off on}_\textrm{NS,transmit} = \alpha[FR(1-2r) + 1]
\end{equation}

\end{document}