\documentclass[12pt,a4paper]{article}
\usepackage{algorithm, algpseudocode, amsmath, amssymb, amsthm, csquotes, empheq, geometry, graphicx, hyperref, listings, multirow, siunitx, subcaption, upgreek}
\usepackage[italicdiff]{physics}
\usepackage[section]{placeins}
\usepackage[justification=centering]{caption}

\title{Computational Physics\\Problem Set 7}
\author{Saleh Shamloo Ahmadi\\Student Number: 98100872}
\date{November 8, 2021}

\hypersetup{colorlinks=true, urlcolor=cyan}
\newcommand{\fig}{../fig}
\newcommand{\ddfrac}[2]{{\displaystyle\frac{\displaystyle #1}{\displaystyle #2}}}
\newcommand{\multlinecell}[1]{\begin{tabular}[c]{@{}c@{}}#1\end{tabular}}

\begin{document}
	\maketitle
    \section{Monte Carlo Integration}
    \subsection[pi/2 * erf(2): Uniform Sampling vs. Importance Sampling]
    {Uniform Sampling vs. Importance Sampling:\\$\frac{\sqrt{\pi}}{2}\erf(2)=\int_0^2 e^{-x^2} \dd{x}$}
    \begin{table}[hbt!]
        \centering
        \caption{The exact value of the integral $\frac{\sqrt{\pi}}{2}\erf(2)=\int_0^2 e^{-x^2} \dd{x}$
            up to the 6th decimal is $0.882081$. The runtimes are from an AMD Ryzen 7 5800H @3.2GHz (up to 4.4GHz)}
        \begin{tabular}{|c|c|c|c|c|}
            \hline
            \multirow{2}{*}{\multlinecell{Number\\of\\Samples}}
            & \multicolumn{2}{|c|}{Calculated Integral} & \multicolumn{2}{|c|}{Runtime} \\
            \cline{2-5}
            & \multlinecell{Uniform\\Sampling} & \multlinecell{Importance\\Sampling}
            & \multlinecell{Uniform\\Sampling} & \multlinecell{Importance\\Sampling} \\
            \hline
            $10$ & $0.460596$ & $0.93583$ & $\SI{0.891513}{\micro\second}$ & $\SI{3.74894}{\micro\second}$ \\
            $100$ & $0.921345$ & $0.903689$ & $\SI{1.6772}{\micro\second}$ & $\SI{21.2343}{\micro\second}$ \\
            $1000$ & $0.899169$ & $0.882798$ & $\SI{8.6187}{\micro\second}$ & $\SI{183.041}{\micro\second}$ \\
            $10000$ & $0.889229$ & $0.880103$ & $\SI{79.3509}{\micro\second}$ & $\SI{1.82276}{\milli\second}$ \\
            $100000$ & $0.880586$ & $0.881593$ & $\SI{844.887}{\micro\second}$ & $\SI{18.7801}{\milli\second}$ \\
            $1000000$ & $0.880835$ & $0.882562$ & $\SI{9.04039}{\milli\second}$ & $\SI{178.5040}{\milli\second}$ \\
            $10^7$ & $0.882187$ & $0.88204$ & $\SI{93.6652}{\milli\second}$ & $\SI{2.51925}{\second}$ \\
            $10^8$ & $0.882014$ & $0.882084$ & $\SI{0.919785}{\second}$ & $\SI{23.3132}{\second}$ \\
            \hline
        \end{tabular}
    \end{table}
    \begin{table}[hbt!]
        \centering
        \caption{Errors of each method. The \enquote{actual} error is the deviation of the calculation from
            $\frac{\pi}{2}\erf(2)$.}
        \begin{tabular}{|c|c|c|c|c|}
            \hline
            \multirow{2}{*}{\multlinecell{Number\\of\\Samples}}
            & \multicolumn{2}{|c|}{Calculated Standard Error} & \multicolumn{2}{|c|}{Actual Error} \\
            \cline{2-5}
            & \multlinecell{Uniform\\Sampling} & \multlinecell{Importance\\Sampling}
            & \multlinecell{Uniform\\Sampling} & \multlinecell{Importance\\Sampling} \\
            \hline
            $10$ & $0.211308$ & $0.094789$ & $0.421485$ & $0.0537491$ \\
            $100$ & $0.0699633$ & $0.0242373$ & $0.0392635$ & $0.021608$ \\
            $1000$ & $0.0220645$ & $0.00838951$ & $0.0170874$ & $0.000716757$ \\
            $10000$ & $0.00685873$ & $0.0026753$ & $0.00714762$ & $0.00197877$ \\
            $100000$ & $0.00217824$ & $0.000843176$ & $0.00149538$ & $0.000488134$ \\
            $1000000$ & $0.000688905$ & $0.000265692$ & $0.00124646$ & $0.000480197$ \\
            $10^7$ & $0.000217995$ & $\num{8.41355e-5}$ & $0.000106063$ & $\num{4.187e-5}$ \\
            $10^8$ & $\num{6.89327e-5}$ & $\num{2.66006e-5}$ & $\num{6.72089e-5}$ & $\num{2.30387e-6}$ \\
            \hline
        \end{tabular}
    \end{table}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=\linewidth]{\fig/montecarlo-erf2-value}
    \end{figure}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=\linewidth]{\fig/montecarlo-erf2-error}
    \end{figure}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=\linewidth]{\fig/montecarlo-erf2-time}
    \end{figure}
    \FloatBarrier
    \subsection{Center of Mass of Sphere with Linearly Increasing Density in One Direction}
    A sphere with radius $R$ has a density that linearly increases in the $z$ direction, such that the density at $z=2R$
    is twice the density at $z=0$. To calculate the position of the center of mass, we must evaluate the integral
    \begin{equation}
        \vb{r}_{cm} = \frac{\int_{\text{sphere}}\rho(\vb{r})\vb{r}\dd{V}}{\int_{\text{sphere}}\rho(\vb{r})\dd{V}}.
    \end{equation}
    Since the sphere is symmetric in the $x$ and $y$ directions, the center of mass is at $x=0$ and $y=0$. It remains to
    find the $z$ position of the center of mass. In polar coordinates
    \begin{gather}
        \rho(z=2R) = 2\rho(z=0) \implies \rho(r=R, \theta=0) = 2\rho(r=R, \theta=\pi) \\
        \xRightarrow{\rho(z) is linear} \rho(r, \theta) = \rho_0\qty(3+\frac{r}{R}\cos{\theta}) \implies \\
        z_{cm} = \ddfrac{\int_{\text{sphere}}z\rho\dd{V}}{\int_{\text{sphere}}\rho\dd{V}} =
        \ddfrac{\int_0^R\int_0^\pi\int_0^{2\pi} \rho_0\qty(3+\frac{r}{R}\cos{\theta})r^3\sin{\theta}\cos{\theta}
        \dd{\varphi}\dd{\theta}\dd{r}}{\int_0^R\int_0^\pi\int_0^{2\pi} \rho_0\qty(3+\frac{r}{R}\cos{\theta})r^2
        \sin{\theta}\dd{\varphi}\dd{\theta}\dd{r}} \\
        z_{cm} = \ddfrac{\int_0^R\int_0^\pi \qty(3+\frac{r}{R}\cos{\theta})r^3\sin{\theta}\cos{\theta}
        \dd{\theta}\dd{r}}{\int_0^R\int_0^\pi \qty(3+\frac{r}{R}\cos{\theta})r^2\sin{\theta}
        \dd{\theta}\dd{r}} \label{eq:multi}
    \end{gather}
    We can use the Monte Carlo integration method to calculate \eqref{eq:multi} and find the center of mass.
    These integrals can also be calculated analytically to check the accuracy of the numerical results:
    \begin{align}
        z_{cm} &= \ddfrac{R^4 \int_0^\pi \qty(\frac{3}{4}+\frac{\cos{\theta}}{5})\sin{\theta}\cos{\theta}\dd{\theta}}
        {R^3 \int_0^\pi \qty(1+\frac{\cos{\theta}}{4})\sin{\theta}\dd{\theta}} \\
        &= \ddfrac{\frac{3}{8}\int_0^\pi\sin(2\theta)\dd{\theta} + \frac{1}{5}\int_{-1}^{+1}\cos^2{\theta}
        \dd{(\cos{\theta})}} {\int_0^\pi\qty(\sin{\theta} + \frac{\sin(2\theta)}{2})\dd{\theta}}R \\
        &= \frac{-\frac{3}{16}\eval{\cos(2\theta)}_0^\pi + \eval{\frac{x^3}{15}}_{-1}^{+1}}
        {-\eval{\cos{\theta}}_0^\pi - \eval{\frac{\cos(2\theta)}{4}}_0^\pi}
    \end{align}
    \begin{empheq}[box=\fbox]{equation}
        z_{cm} = \frac{R}{15}
    \end{empheq}
    Numerically evaluating the two integrals in \eqref{eq:multi} by the Monte Carlo Integration method,
    uniformly sampling 40 million points, we get
    \begin{equation}
        z_{cm} = (0.06667 \pm 0.00009) R,
    \end{equation}
    which is exactly equal to $R/15$ up to the 5th decimal place.
    \section{Metropolis Algorithm}
    \begin{table}[hbt!]
        \centering
        \caption{For $10^8$ samples}
        \begin{tabular}{|c|c|c|}
            \hline
            Step Size $Delta$ & Acceptance Rate $a_r$ & Correlation Length $\xi$ \\
            \hline
            15.8846 & 0.1004 & $7.23 \pm 0.03$ \\
            7.969 & 0.19996 & $3.53 \pm 0.03$ \\
            5.3048 & 0.300005 & $2.3 \pm 0.03$ \\
            3.888 & 0.4003 & $1.84 \pm 0.01$ \\
            2.9486 & 0.4991 & $2.06 \pm 0.02$ \\
            2.2094 & 0.5992 & $2.72 \pm 0.02$ \\
            1.578 & 0.7007 & $4.11 \pm 0.02$ \\
            1.0236 & 0.8002 & $7.89 \pm 0.02$ \\
            0.5 & 0.9008 & $27.76 \pm 0.01$ \\
            \hline
        \end{tabular}
    \end{table}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=0.65\linewidth]{\fig/metropolis-hist}
        \caption{For $10^8$ samples, step size 3 and acceptance rate 0.49}
    \end{figure}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=\linewidth]{\fig/metropolis-acceptance}
    \end{figure}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=\linewidth]{\fig/metropolis-autocor}
    \end{figure}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=\linewidth]{\fig/metropolis-autocor-log}
    \end{figure}
    \begin{figure}[htb!]
        \centering
        \includegraphics[width=\linewidth]{\fig/metropolis-corlen}
    \end{figure}
\end{document}


\documentclass[a4paper]{article}

\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage[colorinlistoftodos]{todonotes}
\usepackage{natbib}
\usepackage{float}
\usepackage{blindtext}
\usepackage{authblk}
\title{\textbf{Dimension Reduction}\\ \large Linear Algebra Project Report, Dr.Ramin Javadi}
\author[1]{MohammadSadegh Akhoundzadeh\thanks{ms.akhondzadeh@gmail.com}}
\author[1]{Maryam Meghdadi\thanks{marmegh@gmail.com}}
\affil[1]{Department of Electrical and Computer Engineering, Isfahan University of Technology
}

%\subt {Linear Algebra Project Report, Dr.Ramin Djavadi}

\begin{document}
\maketitle

\begin{abstract}
In this report we give a review on some methods of dimension reduction. Given some input high-dimensional data, the goal of dimension reduction is to map them to a low-dimensional space such that certain properties of the initial data are preserved. Finally, we will compare discussed methods in some experiments.
\end{abstract}

\section{Introduction}
\label{sec:introduction}

In machine learning classification problems, there are often too many factors on the basis of which the final classification is done. These factors are basically variables called features. The higher the number of features, the harder it gets to visualize the training set and then work on it. Sometimes, most of these features are correlated, and hence redundant. This is where dimensionality reduction algorithms come into play. Dimensionality reduction is the process of reducing the number of random variables under consideration, by obtaining a set of principal variables. It can be divided into feature selection and feature extraction.
Some of the dimension reduction methods are based on linear transformation like PCA (Principal Component Analysis) and some of them are based on nonlinear techniques such as Isomap and Spectral Embedding.
\section{Theory}
\label{sec:theory}

\subsection{Linear Methods}

\subsubsection{PCA}
Principal component analysis(PCA) is a very popular technique for dimensionality reduction. Given a set of n-dimensional data, PCA algorithm aims to find d-dimensional data (d $<$ n) on a linear subspace. Such a reduced subspace
attempts to maintain most of the variability of the data. The linear subspace can be specified by d orthogonal vectors that form a new coordinate
system, called the 'principal components'. The goal is to find some vectors on a subspace which the variation of the data is maximal on it.\\
\textbf{Example:} In this example we projected a 3D data in a 2D subspace. Actually, we project the data on the plane of two eigenvectors U1 and U2.(Figure 1)\\

\begin{figure}
\centering
\includegraphics[width=1.1\textwidth]{pca.png}
\caption{\label{fig:pca}PCA applied to a 3D data set to reduce it to two dimensions}
\end{figure}

In order to capture as much of the variability as possible, let us choose the first principal component, denoted by $U_1$, to have maximum variance. Our data is n $\times$ t matrix X and our first eigenvector is a combination of X with $\omega$ coefficients such that $\omega$ = [$\omega_1 \ldots \omega_n$]. So we have:
\begin{equation}
U_1 = \omega^T X
\end{equation}
\begin{equation}
\text{var}(U_1) = \text{var}(\omega^T X) = \omega^T S \omega
\end{equation}
where S is the n $\times$ n sample covariance matrix of X.
To reach our goal we should maximize the variance we obtained in (2).
\begin{center}
max  $\omega^T$S $\omega$ \space \space s.t. $\omega^T \omega$ = 1
\end{center}
To do this, we use lagrange method and introduce a lagrange multiplier $\lambda$:
\begin{equation}
L(\omega, \lambda) = \omega^T S \omega - \lambda(\omega^T \omega - 1)
\end{equation}
By differentiating L with respect to $\omega$ we have:
\begin{equation}
S\omega = \lambda_1\omega
\end{equation}
This shows that the first principal component is given by the normalized eigenvector with the largest associated eigenvalue of the sample covariance matrix S. A similar argument can show that the d dominant eigenvectors of covariance matrix S determine the first d principal components. To obtain these eigenvectors we find singular value decomposition (SVD):
\begin{equation}
X = U \Sigma V^T
\end{equation}
where columns of U are eigenvectors of $XX^T$ (covariance matrix).
We define Y as our projected d-dimensional data (d $<$ n):
\begin{equation}
Y = U(:, 1:d)^T X \qquad or \qquad Y = U_d^T X
\end{equation}
Next step is to reconstruct the training data which is:
\begin{equation}
\hat{X} = U_d Y
\end{equation}
For a new test example x we compute $y = U^T x$ and then reconstruct it $\hat{x} = U y$ .
%In general, from (2) and (4) we have:
%\begin{equation}
%\omega^T S \omega = \omega^T \lambda \omega = \lambda
%\end{equation}
%and final variance is:
%\begin{equation}
%\sum_{i = 1}^{t} \text{var}(
%\end{equation}
% check n & t
[2]

\subsubsection{Dual PCA}
Dual PCA is similar to PCA but it's using a trick that make this method faster in the case that the number of features is bigger than the samples. The goal is to reduce dependence of our algorithm on n (the number of features). So instead of decomposition of $XX^T$, we decompose $X^TX$. Note that in the SVD factorization $U\Sigma V^T$, so $XV = U\Sigma$, where the eigenvectors in U corresponds to nonzero singular values in $\Sigma$. Thus, the following conversion between the top d eigenvectors can be derived:
\begin{equation}
U = XV\Sigma^{-1}
\end{equation}
Replacing all uses of U in PCA algorithm gives us Dual PCA algorithm. First we compute Y:
\begin{equation}
Y = U^TX = \Sigma V^T
\end{equation}
Then, reconstruct data:
\begin{equation}
\hat{X} = UY = U\Sigma V^T = XV\Sigma^{-1}\Sigma V^T = XVV^T
\end{equation}
Next, project out of sample point x:
\begin{equation}
y = U^T x = \Sigma^{-1} V^T X^T x
\end{equation}
And finally reconstruct this sample point:
\begin{equation}
\hat{x} = U y = UU^T x = XV\Sigma^{-2} V^T X^T x
\end{equation}

\subsubsection{Random Projection}
Again, if the number of samples is large, we can use a random method which guarantees that after projecting the points from the n-dimensional
space to K dimensional space using the randomly drawn matrix W, the distances between the high dimensional points is preserved in the lower dimensional projections up to some approximation factor. This matrix W is:

\[
W[i,j] = \left\{
				\begin{array}{ll}
				+\frac{1}{\sqrt{K}} \qquad \text{with probability 1/2}\\
                -\frac{1}{\sqrt{K}} \qquad \text{with probability 1/2}
				\end{array}
         \right \}
  \]

\subsection{Nonlinear Methods}

\subsubsection{Isomap}
The euclidean distance of points on a linear space remains unchanged but on a submanifold (nonlinear space) this distance will change. This method is similar to MDS but we change euclidean distance to geodesic distance. The
geodesic distances represent the shortest paths along the curved surface of the manifold measured as if the surface were flat. This can be approximated by a sequence of short steps between neighboring sample points. Isomap then applies MDS to the geodesic rather than straight line distances to find a low-dimensional mapping that preserves these pairwise distances.
Steps of Isomap are:
\begin{enumerate}
\item Construct a k-nearest neighbor graph on n data points (connect each point to it's k-nearest neighbor.

\item Compute shortest path between all the points ($D^G$) as an estimation of geodesic distance, using Dijkstra's algorithm and Floyd-Warshall's algorithm.

\item Compute $k = -\frac{1}{2} H(D^G)^2 H$. find eigenvectors of k and call it V. then find the top p eigenvalues of k and form $\hat{\Lambda}$. The final solution is $Y = {\hat{\Lambda}}^\frac{1}{2} V^T$. Note that matrix H is a centering matrix and is equal to $H = I - \frac{1}{n}e e^T$, where e is a n $\times$ 1 vector of ones and $\hat{\Lambda}$ is a diagonal matrix which diagonal values are the top p eigenvalues of $X^T X$.

\end{enumerate}

\subsubsection{Laplacian Eigenmaps (Spectral Embedding)}
Given t points in n-dimensional space, the Laplacian eigenmaps Method (LEM)  starts by constructing a weighted graph with t nodes and a set of
edges connecting neighbouring points. The neighbourhood graph can be
constructed by finding the k nearest neighbours. For weighting the edges, we can use adjacency matrix, which elements are one for the nodes that are connected and zero for disconnected nodes. Another way is to use gaussian distance:
\begin{equation}
W_{ij} = e{^{-\frac{||x_i - x_j|| ^ 2}{\gamma}}}
\end{equation}
The embedding map is then provided by the following objective:
\begin{equation}
\min_{Y} \sum_{i = 1}^{t} \sum_{j = 1}^{t} (y_i - y_j)^2 W_{ij}
\end{equation}
subject to appropriate constraints. From spectral clustering we know that:
\begin{equation}
\sum_{i,j} (y_i - y_j)^2 W_{ij} = Y^T L Y
\end{equation}
So From (14) and (15) we reach to:
\begin{equation}
\min_{Y} Tr(YLY^T)
\end{equation}
\begin{center}
 \text{s.t.} $YY^T = I$\\
\end{center}
Where L is laplacian matrix which is $L = D - W$ and D is diagonal and equals to $D_{ii} = \sum_{j=1}^{t} W_{ij}$. From (13) and (15) we conclude that if $x_i$ and $x_j$ are far apart then $W_ij$ becomes small, so we don't care about $y_i$ and $y_j$. But if $x_i$ and $x_j$ are close, $W_ij$ becomes large, so $y_i$ and $y_j$ should be small and we reach to our goal.

\section{Experiment}
We start by generating three different data to compare the algorithms.
\subsection{Part1}

At first we create data in the shape of "hello"(figure 2).
\begin{figure}[H]
\centering
\includegraphics[scale = 0.33]{simple_hello.png}
\caption{\label{fig:simple_hello}hello-shaped data}
\end{figure}
Then for the first part of our experiments we use a linear function with a gaussian noise to add a new dimension to the data and make it 3D. (figure 3)
\begin{figure}[H]
\centering
\includegraphics[scale = 0.33]{3_dhello_linear_gussian.png}
\caption{\label{fig:3_dhello_linear_gussian}3D hello-shaped data}
\end{figure}
\newpage
See the result of the the algorithms below:
\begin{figure}[H]
\centering
\includegraphics[scale = 0.32]{pca_part1.png}
\caption{\label{fig:pca_part1}PCA applied to 3D hello(figure 3) set to reduce it to two dimensions}
\includegraphics[scale = 0.32]{random_projection_part1.png}
\caption{\label{fig:random_projection_part1}Random Projection applied to 3D hello(figure 3) set to reduce it to two dimensions}

\includegraphics[scale = 0.32]{isomap_part1.png}
\caption{\label{fig:isomap_part1}Isomap applied to 3D hello(figure 3) set to reduce it to two dimensions}

\end{figure}

\begin{figure}[H]
\centering
\includegraphics[scale = 0.32]{laplacian_part1.png}
\caption{\label{fig:laplacian_part1}Laplacian applied to 3D hello (figure 3) set to reduce it to two dimensions}
\end{figure}

%\begin{figure}
%\centering
%\includegraphics[scale = 0.4]{3_dhello_linear_gussian.png}
%\caption{\label{fig:3_dhello_linear_gussian}...}
%\end{figure}

\subsection{Part2}
In the next part instead of adding a new dimension to the data, we maked the “hello” data curved (figure 8)
\begin{figure}[H]
\centering
\includegraphics[scale = 0.32]{curved_hello.png}
\caption{\label{fig:curved_hello}curved hello}
\end{figure}
so now we have a manifold in the shape of “hello” in three dimensions and our goal is to reduce the dimension while preserving the certain properties of the data.
\newpage
See the result of the the algorithms below:
\begin{figure}[H]
\centering
\includegraphics[scale = 0.32]{pca_part2.png}
\caption{\label{fig:pca_part2}PCA applied to curved hello (figure 8) set to reduce it to two dimensions}
\includegraphics[scale = 0.32]{random_projection_part2.png}
\caption{\label{fig:random_projection_part2}Random Projection applied to curved hello (figure 8) set to reduce it to two dimensions}
\includegraphics[scale = 0.32]{isomap_part2.png}
\caption{\label{fig:isomap_part2}Isomap applied to curved hello (figure 8) set to reduce it to two dimensions}

\end{figure}

% \begin{figure}[H]
% \centering
% \end{figure}

\begin{figure}[H]
\centering
\includegraphics[scale = 0.32]{laplacian_part2.png}
\caption{\label{fig:laplacian_part2}Laplacian applied to curved hello(figure 8) set to reduce it to two dimensions}
\end{figure}

\subsection{Part3}
In the last part, we use a S-shaped data which is curved again. we repeat previous section with this new data.
\begin{figure}[H]
\centering
\includegraphics[scale = 0.32]{s_3d.png}
\caption{\label{fig:s_3d}S-shaped curve}
\end{figure}
\newpage
See the result of the the algorithms below:
\begin{figure}[H]
\centering
\includegraphics[scale = 0.32]{pca_part3.png}
\caption{\label{fig:pca_part3}PCA applied to S-shaped (figure 13) set to reduce it to two dimensions}
\includegraphics[scale = 0.32]{random_projection_part3.png}
\caption{\label{fig:random_projection_part3}Random Projection applied to S-shaped (figure 13) set to reduce it to two dimensions}

\includegraphics[scale = 0.32]{isomap_part3.png}
\caption{\label{fig:isomap_part3}Isomap applied to S-shaped (figure 13) set to reduce it to two dimensions}
\end{figure}

\begin{figure}[H]

\centering
\includegraphics[scale = 0.32]{laplacian_part3.png}
\caption{\label{fig:laplacian_part3}Laplacian applied to S-shaped  (figure 13) set to reduce it to two dimensions}
\end{figure}

% \section{Results and interpretation 2-3 pages}

% \section{Discussion 1/2-1 page}
% Discuss your results. Compare the two values of $n_{s}$ that you've found in the previous section. Compare your results with literature and comment on the difference. If you didn't know the value of the resistance quantum, would you be able to deduce it from your measurements? If yes/no, why?

\newpage
\cite{Kokiopoulou2011}
\cite{Ghodsi2006}
\cite{Projections}
\bibliographystyle{plain}
\bibliography{Mendeley}
\end{document}


\label{ch:sdc}

\section{SDC Overview}

Spectral deferred corrections (SDC) is an iterative scheme used to integrate
the thermodynamic variables, $(\rho X_k,\rho h)$, over a time step.  It has
been shown to be more accurate and efficient than Strang splitting in a
terrestrial, non-stratified, low Mach number reacting flow solver \cite{Non11},
so we would like to develop an SDC version of \maestro.

\maestro\ integrates the following system of equations:
\begin{eqnarray}
\frac{\partial\Ub}{\partial t} &=&
    -\Ub\cdot\nabla\Ub  - \frac{1}{\rho}\nabla\pi
    - \frac{\rho-\rho_0}{\rho} g\eb_r,\label{eq:momentum}\\
\frac{\partial(\rho X_k)}{\partial t} &=&
    -\nabla\cdot(\rho X_k\Ub) + \rho\omegadot_k,\label{eq:species}\\
\frac{\partial(\rho h)}{\partial t} &=&
    -\nabla\cdot(\rho h\Ub) + \frac{Dp_0}{Dt}
    + \rho\Hnuc + \nabla\cdot k_{\rm th}\nabla T,\label{eq:enthalpy}
\end{eqnarray}
together with base state evolution equations and a constraint equation.
By default, \maestro\ advances the thermodynamic variables by coupling
the different physical processes together (advection, diffusion, reactions) using
Strang splitting.  Specifically, we integrate the reaction terms
over half a time step ignoring contributions from advection and diffusion,
then from this intermediate state we integrate the advection and diffusion terms over
a full time step (while ignoring reactions), and finally
integrate the reactions over a half time step (ignoring advection and diffusion).
For problems where
the reactions and/or diffusion greatly alter the energy balance or composition
as compared to advection, this operator splitting approach can lead to large
splitting errors and highly inaccurate solutions.
This issue is can be particularly exasperating
for low Mach number methods that can take large advection-based time steps.

An alternate approach to advancing the thermodynamic variables is SDC.
SDC is an iterative approach to couple the various processes
together, with each process seeing an
approximation of the other processes as a source term.  The SDC
algorithm converges to an integral representation of the solution in
time that couples all of the processes together in a self-consistent
fashion, see \cite{Non11}.

As a first attempt, we will work on coupling advection and reactions only
via SDC, with a base state that is fixed in time, but not space.

\section{Strang-Splitting Without Thermal Diffusion or Base State Evolution}

In the Strang splitting version of \maestro, the reaction and advection
processes operate independent of one-another.  The species and
enthalpy equations are integrated over $\Delta t$ using the following
sequence:

\begin{itemize}

\item React for $\Delta t/2$:

In the Strang splitting version of \maestro, the reaction network solves just
the reaction portion of the species evolution equations along with a
temperature evolution equation.  This is done using a standard stiff ODE solver
package (like {\tt VODE}) on the system:
\begin{eqnarray}
\frac{dX_k}{dt} &=& \omegadot_k(\rho,X_k,T), \\
\frac{dT}{dt}   &=&
    \frac{1}{c_p} \left ( -\sum_k \xi_k  \omegadot_k  + \Hnuc \right ).
\end{eqnarray}
Here, $T$ is evolved solely to evaluate the reaction rates,
$\omegadot_k(\rho,X_k,T)$.  Furthermore, we simplify the problem
``freezing'' the thermodynamics---i.e., $c_p$ and $\xi_k$ are evaluated at the
start of the integration and held constant thereafter.
The density remains constant during this step, i.e.,
$\rho^\mathrm{new} = \rho^\mathrm{old}$, and we
update the enthalpy at the end of the integration as:
\begin{equation}
h^\mathrm{new} = h^\mathrm{old} + \frac{\Delta t}{2} \Hnuc.
\end{equation}

\item Advect for $\Delta t$:

Beginning with the results from the previous reaction half time step, we integrate
that state using the equations
\begin{eqnarray}
\frac{\partial(\rho X_k)}{\partial t} &=&
    -\nabla\cdot(\rho X_k\Ub), \\
\frac{\partial(\rho h)}{\partial t} &=&
    -\nabla\cdot(\rho h\Ub) + \frac{Dp_0}{Dt}.
\end{eqnarray}
Note that no reaction terms appear here.  Since the advection
takes place using the state updated from the reaction step, the effect
of the reactions is implicitly contained in the advective update.

\item React for $\Delta t/2$:

Finally, we react again, starting with the state left by the advection
step.

\end{itemize}

Note that \maestro\ uses a predictor-corrector approach.  After integrating $(\rho X_k,\rho h)$ over
the time step, we use this time-advanced state to get a better estimate of a time-centered $\beta_0$
and $S$.  We re-compute the advective velocities using an updated divergence constraint and repeat
the thermodynamic variable advance.

\section{SDC Without Thermal Diffusion or Base State Evolution}
In the SDC version, the {\tt VODE} integration at the end of an SDC
iteration is responsible for updating all the thermodynamic quantities
including both the advection (incorporated via a piecewise constant advective
flux divergence source term) and the reactions.  This provides a much stronger coupling between
the physical processes.  In particular, our system now looks like:
\begin{eqnarray}
\frac{d(\rho X_k)}{dt} &=& \rho \omegadot_k(\rho,X_k,T) - \underbrace{\nabla\cdot(\rho X_k\Ub)}_{A_{\rho X_k}}\label{eq:sdc:rhoX} \\
\frac{d(\rho h)}{dt}   &=& \rho \Hnuc - \underbrace{\nabla\cdot(\rho h\Ub) + \frac{Dp_0}{Dt}}_{A_{\rho h}} \label{eq:sdc:rhoh}
\end{eqnarray}
Here, $A_{\rho X_k}$ and $A_{\rho h}$ are piecewise-constant (in time)
approximations to the change in ${\rho X_k}$ and ${\rho h}$ (respectively)
due to the advection.  These are constructed by calling {\tt density\_advance}
and {\tt enthalpy\_advance} in \maestro\ and passed into the network solver
during the reaction step.  A flowchart of the \maestro\ SDC algorithm is
shown in figure~\ref{fig:sdc:flowchart}.

\begin{figure}[tb]
\centering
\includegraphics[scale=0.65]{\sdcfigpath/flowchart_SDC}
\caption[Graphical flowchart of \maestro\ SDC]
  {\label{fig:sdc:flowchart} A flowchart of the \maestro\ SDC algorithm.  The
  thermodynamic state variables and local velocity are
  indicated in each step.  The base state is not shown as it is time-independent.
  Red text indicates that quantity was
  updated during that step.  The predictor is
  outlined by the dotted box.  The blue text indicates state
  variables that are the same in {\bf Step 3} as they are in
  {\bf Step 1}, i.e., they are unchanged by the predictor steps.
  The SDC loop is shown in the gray dotted box.}
\end{figure}

\subsection{Advective Update}

In the advective update, our goal is to compute $A_{\rho X_k}$ and
$A_{\rho h}$.  These terms approximate the following:
\begin{eqnarray}
A_{\rho X_k} &=&  \left [- \nabla \cdot (\rho X_k \Ub) \right ]^{n+1/2} \\
A_{\rho h}   &=&  \left [- \nabla \cdot (\rho h \Ub) + \frac{Dp_0}{Dt} \right ]^{n+1/2}
\end{eqnarray}
The construction of the interface states used in the advective terms
uses either a time-lagged or iteratively-lagged approximation to the reaction
terms ($I_{\rho X_k}$ and $I_{\rho h}$, see below) as a source term in the interface
prediction.  This explicitly couples the reaction process to the
advection process.

\subsection{Final Update}

The RHS routine that the ODE solver operates on will first construct
the density as:
\begin{equation}
\rho = \sum_k (\rho X_k)
\end{equation}
It will then derive the temperature from the equation of state.  If we
are running with {\tt use\_tfromp = T}, then we do
\begin{equation}
T = T(\rho, p_0, X_k)
\end{equation}
otherwise, we do
\begin{equation}
T = T(\rho, h, X_k)
\end{equation}
Note that in contrast to the Strang splitting version, here we call the EOS
every time we enter the RHS routine, but here we call the EOS to compute temperature
rather than thermodynamic coefficients.

Finally we integrate the ODE system (Eqs.~\ref{eq:sdc:rhoX} and \ref{eq:sdc:rhoh}).
At the end of the integration, we define $I_{\rho X_k}$ and $I_{\rho h}$.  The actual
form of these depends on what quantities we predict to edges during
the construction of the advective fluxes.
Note that we only need $I_{\rho X_k}$ and $I_{\rho h}$ for the
prediction of the interface states, and not the {\tt VODE} integration.
This is because all we need from the advection solver is the
approximation to $A_{\rho X_k}$ and $A_{\rho h}$ and not the final
updated state.

\subsection{Species Source Terms.}
For the species prediction, the form of $I$ depends on
{\tt species\_pred\_type} (see \S \ref{sec:pred:density}).
We note that there is no $I$ term for $\rho$ or $\rho'$ prediction, since
the density evolution equation does not have a reaction source term.
\begin{itemize}
\item {\tt species\_pred\_type} = 1 ({\tt predict\_rhoprime\_and\_X})
or 3 ({\tt predict\_rho\_and\_X})

\begin{equation}
I_{X_k} = \frac{1}{\rho^{n+\myhalf}} \left [
      \frac{(\rho X_k)^\mathrm{new} -
            (\rho X_k)^\mathrm{old}}{\Delta t} - A_{\rho X_k}  \right ].
\end{equation}
(Andy's Idea) Define $I_{X_k}$ using
\begin{equation}
I_{X_k} = \frac{X_k^\mathrm{new} - X_k^\mathrm{old}}{\Delta t} - A_{X_k},
\end{equation}
where we first define a state that has only been updated with advection:
\begin{equation}
\frac{(\rho X_k)^{(1)} - (\rho X_k)^\mathrm{old}}{\Delta t} = A_{\rho X_k},
\end{equation}
and then define the species mass fractions,
\begin{equation}
X_k^{(1)} = (\rho X_k)^{(1)} / \sum_k (\rho X_k)^{(1)}, \quad
X_k^\mathrm{old} = (\rho X_k)^\mathrm{old} / \sum_k (\rho X_k)^\mathrm{old}, \quad
X_k^\mathrm{new} = (\rho X_k)^\mathrm{new} / \sum_k (\rho X_k)^\mathrm{new},
\end{equation}
and finally define $A_{X_k}$ using
\begin{equation}
\frac{X^{(1)} - X^\mathrm{old}}{\Delta t}= A_{X_k}.
\end{equation}
\item {\tt species\_pred\_type} = 2 ({\tt predict\_rhoX})
\begin{equation}
I_{\rho X_k} = \frac{(\rho X_k)^\mathrm{new} - (\rho X_k)^\mathrm{old}}{\Delta t} - A_{\rho X_k}.
\label{eq:sdc:Irhoo}
\end{equation}
\end{itemize}

\subsection{Enthalpy Source Terms.}
The appropriate constructions are:

\begin{itemize}
\item {\tt enthalpy\_pred\_type} = 0 ({\tt predict\_rhoh})
\begin{equation}
I_{\rho h} = \frac{(\rho h)^{\rm new} - (\rho h)^{\rm old}}{\Delta t} - A_{\rho h}.
\end{equation}

\item {\tt enthalpy\_pred\_type} = 1 ({\tt predict\_rhohprime}, not implemented yet)

(Andy's Idea) Here we need an $I_{\rho h}$ term for the $(\rho h)'$ evolution
equation (see Eq.~\ref{rhohprime equation}).  In this case we will use
$I_{(\rho h)'} = I_{\rho h}$.  Since we are not evolving the base state, the PDE
for $(\rho h)_0$ is simply $\partial(\rho h)_0/\partial t = 0$, and thus the
evolution equation for $(\rho h)'$ is the same as the evolution equation
for $\rho h$.

In the future, when we enable base state evolution, the base state enthalpy
evolution equation may need to know about the $I_{\rho h}$ source term.
In particular, should $(\rho h)_0$ see a $\overline{(\rho \Hnuc)}$ term?
what about an average thermal diffusion?

\item {\tt enthalpy\_pred\_type} = 2 ({\tt predict\_h} )

This is the most straightforward prediction type.  The SDC solver
integrates the equation for $(\rho h)$:
\begin{equation}
\frac{\partial(\rho h)}{\partial t} = -\nabla\cdot(\rho h \Ub) + \frac{Dp_0}{Dt}  + \rho H_{\rm nuc}
\end{equation}
(shown here without diffusion or external heat sources).  Expanding
the time derivative and divergence, and using the continuity equation
we see:
\begin{equation}
\frac{\partial h}{\partial t} = -\Ub \cdot \nabla h + \frac{1}{\rho} \frac{Dp_0}{Dt}  + \frac{1}{\rho} (\rho H_{\rm nuc}) \label{eq:sdc:h}
\end{equation}
Comparing these equations, we see that
\begin{equation}
I_{h}  = \frac{1}{\rho^{n+\myhalf}} \left [
    \frac{(\rho h)^\mathrm{new} - (\rho h)^\mathrm{old}}{\Delta t} - A_{\rho h} \right ]
\end{equation}
(Andy's Idea) Form $I_h$ in the same way we would form $I_{X_k}$ from above:
\begin{equation}
I_h = \frac{h^\mathrm{new} - h^\mathrm{old}}{\Delta t} - A_h,
\end{equation}
where we first define
\begin{equation}
\frac{(\rho h)^{(1)} - (\rho h)^\mathrm{old}}{\Delta t} = A_{\rho h},
\end{equation}
and then define $h$,
\begin{equation}
h^{(1)} = (\rho h)^{(1)} / \sum_k(\rho X_k)^{(1)}, \quad h^\mathrm{old} = (\rho h)^\mathrm{old} / \sum_k(\rho X_k)^\mathrm{old}, \quad h^\mathrm{new} = (\rho h)^\mathrm{new} / \sum_k(\rho X_k)^\mathrm{new},
\end{equation}
and finally define $A_h$ using
\begin{equation}
I_h = \frac{h^{(1)} - h^\mathrm{old}}{\Delta t} = A_h.
\end{equation}

\item {\tt enthalpy\_pred\_type} = 3 ({\tt predict\_T\_then\_rhoprime}) or
      {\tt enthalpy\_pred\_type} = 4 ({\tt predict\_T\_then\_h} )

Both of these {\tt enthalpy\_pred\_type}s predict temperature.  Expressing
$h = h(p_0,T,X_k)$ and differentiating along particle paths:
\begin{eqnarray}
\frac{Dh}{Dt} &=& \left . \frac{\partial h}{\partial T} \right |_{p,X_k} \frac{DT}{Dt} +
                  \left . \frac{\partial h}{\partial p} \right |_{T,X_k} \frac{Dp_0}{Dt} +
           \sum_k \left . \frac{\partial h}{\partial X_k} \right |_{p,T} \frac{DX_k}{Dt} \\
              &=& c_p \frac{DT}{Dt} + h_p  \frac{Dp_0}{Dt} + \sum_k \xi_k \omegadot_k
\end{eqnarray}
where $c_p$, $h_p$, and $\xi_k$ are as defined in the table of symbols
(Table~\ref{table:sym}), and we substitute $DX_k/Dt = \omegadot_k$ (from the species
continuity equation, Eq.~\ref{species equation}).  Using Eq.~\ref{eq:sdc:h}, we have
the familiar temperature evolution equation:
\begin{equation}
\rho c_p \frac{DT}{Dt} = \underbrace{(1 - \rho h_p) \frac{Dp_0}{Dt}}_{\begin{smallmatrix}\text{already~accounted~for} \\ \text{in~T~prediction}\end{smallmatrix}} - \sum_k \xi_k \rho \omegadot_k + \rho \Hnuc
\end{equation}
where the underbraced term is already present in {\tt mktempforce}.  Recognizing that
Eq.~\ref{eq:sdc:Irhoh} is the SDC approximation to $(\rho \Hnuc)$ and Eq.~\ref{eq:sdc:Irhoo} is the
SDC approximation to $(\rho \omegadot_k)$, we can define
\begin{equation}
I_T = \frac{1}{\rho^{n+\myhalf} c_p^{n+\myhalf}} \left \{
  \left [ \frac{(\rho h)^\mathrm{new} - (\rho h)^\mathrm{old}}{\Delta t} - A_{\rho h} \right ]
  - \sum_k \xi_k^{n+\myhalf} \left [      \frac{(\rho X_k)^\mathrm{new} -
            (\rho X_k)^\mathrm{old}}{\Delta t} - A_{\rho X_k}  \right ] \right \}
\end{equation}

(Andy's Idea) The idea is to advance the species and enthalpy with advection
terms only, and compute the resulting temperature, $T^{(1)}$.  Compare that temperature
with the final temperature computed by the SDC VODE call.  The difference
between these values is $I_T$.
\begin{equation}
I_T = \frac{T^\mathrm{new} - T^\mathrm{old}}{\Delta t} - A_T,
\end{equation}
with $A_T$ given by
\begin{equation}
\frac{T^{(1)} - T^\mathrm{old}}{\Delta t} = A_T,
\end{equation}
and $T^{(1)}$ computed using the equation of state from $\rho^{(1)}, X_k^{(1)}$,
and $h^{(1)}$ (or $p_0$, if {\tt use\_tfromp = T}).
\end{itemize}

\subsection{Implementation}

This is done in {\tt advance.f90} just after the call to {\tt react\_state},
stored in the \multifab\ called {\tt intra}.
These terms are used as the source terms for the
advection step in the next SDC iteration.

\subsection{Summary of Changes}
The major changes from the non-SDC-enabled burners is the addition of
the advective terms to the system of ODEs, the fact that we integrate
$(\rho X_k)$ instead of just $X_k$, integrate $(\rho h)$ instead
of $T$, and the need to derive the
temperature from the input state for each RHS evaluation by {\tt VODE}.

Note also that the SDC integration by {\tt VODE} does not operate on
the velocities at all.  That update is handled in the same fashion
as the Strang splitting version of the code.

The {\tt ignition\_simple\_SDC} burner shows how to setup the system
for {\tt use\_tfromp = T} or {\tt F}.  Presently, this implementation
does not support {\tt evolve\_base\_state = T} (in particular, we
need to evolve $p_0$ in the RHS routine).

\section{Algorithm Flowchart - ADR with Fixed Base State}
We now include thermal diffusion and assume the base state is constant in time but not space:
\begin{eqnarray}
\frac{\partial\Ub}{\partial t} &=&
    -\Ub\cdot\nabla\Ub  - \frac{1}{\rho}\nabla\pi
    - \frac{\rho-\rho_0}{\rho} g\eb_r,\\
\frac{\partial(\rho X_k)}{\partial t} &=&
    -\nabla\cdot(\rho X_k\Ub) + \rho\omegadot_k,\label{eq:sdc species}\\
\frac{\partial(\rho h)}{\partial t} &=&
    -\nabla\cdot(\rho h\Ub) + \underbrace{\Ub\cdot\nabla p_0}_{Dp_0/Dt}
    + \rho\Hnuc
    + \underbrace{\nabla\cdot\frac{\kth}{c_p}\nabla h - \sum_k\nabla\cdot\frac{\xi_k k_{\rm th}}{c_p}\nabla X_k - \nabla\cdot\frac{h_p k_{\rm th}}{c_p}\nabla p_0}_{\nabla\cdot k_{\rm th}\nabla T}.\nonumber\\
\label{eq:sdc enthalpy}
\end{eqnarray}

The time-advancement is divided into three major steps.  The first step is the predictor, where we integrate the thermodynamic variables, $(\rho,\rho X_k,\rho h)$, over the full time step.  The second step is corrector, where we use the results from the predictor to perform a more accurate temporal integration of the thermodynamic variables.  The third step is the velocity and dynamic pressure update.\\

{\bf Step 1:} ({\it Compute advection velocities})\\ \\
Use $\Ub^n$ and a second-order Godunov method to compute time-centered edge velocities, $\uadvsdcstar$, with time-lagged dynamic pressure and explicit buoyancy as forcing terms.  The $\star$ superscript indicates that this field does not satisfy the divergence constraint.  Compute $S^{n+\myhalf,{\rm pred}}$ by extrapolating in time,
\begin{equation}
S^{n+\myhalf,{\rm pred}} = S^n + \frac{\Delta t^n}{2}\frac{S^n - S^{n-1}}{\Delta t^{n-1}},
\end{equation}
and project $\uadvsdcstar$ to obtain $\uadvsdcpred$, which satisfies
\begin{equation}
\nabla\cdot\left(\beta_0^n\uadvsdcpred\right) = S^{n+\myhalf,\rm{pred}}.
\end{equation}

{\bf Step 2:} ({\it Predictor})\\ \\
 In this step, we integrate $(\rho, \rho X_k, \rho h)$ over the full time step.  The quantities $(S, \beta_0, k_{\rm th}, c_p, \xi_k, h_p)^n$ are computed from the the thermodynamic variables at $t^n$.  This step is divided into several sub-steps:\\

{\bf Step 2A:} ({\it Compute advective flux divergences})\\ \\
Use $\uadvsdcpred$ and a second-order Godunov integrator to compute time-centered edge states, $(\rho X_k, \rho h)^{n+\myhalf,(0)}$, with time-lagged reactions ($I^{\rm lagged} = I^{(j_{\rm max})}$ from the previous time step), explicit diffusion, and time-centered thermodynamic pressure as source terms.  Define the advective flux divergences as
\begin{eqnarray}
A_{\rho X_k}^{(0)} &=& -\nabla\cdot\left[\left(\rho X_k\right)^{n+\myhalf,{(0)}}\uadvsdcpred\right],\\
A_{\rho h}^{(0)} &=& -\nabla\cdot\left[\left(\rho h\right)^{n+\myhalf,(0)}\uadvsdcpred\right] + \uadvsdcpred\cdot\nabla p_0.
\end{eqnarray}
Next, use these fluxes to compute the time-advanced density,
\begin{equation}
\frac{\rho^{n+1} - \rho^n}{\Delta t} = \sum_k A_{\rho X_k}^{(0)}.
\end{equation}
Then, compute preliminary, time-advanced species using
\begin{equation}
\frac{\rho^{n+1}\widehat{X}_k^{n+1,(0)} - (\rho X_k)^n}{\Delta t} = A_{\rho X_k}^{(0)} + I_{\rho X_k}^{\rm lagged}.\label{eq:sdc species 2}
\end{equation}

{\bf Step 2B:} ({\it Compute diffusive flux divergence})\\ \\
Solve a Crank-Nicolson-type diffusion equation for $\widehat{h}^{n+1,(0)}$, using transport coefficients evaluated at $t^n$ everywhere,
\begin{eqnarray}
\frac{\rho^{n+1}\widehat{h}^{n+1,(0)} - (\rho h)^n}{\Delta t} &=& A_{\rho h}^{(0)} + I_{\rho h}^{\rm lagged}\nonumber\\
&& + \half\left(\nabla\cdot\frac{\kth^n}{c_p^n}\nabla h^n + \nabla\cdot\frac{\kth^n}{c_p^n}\nabla \widehat{h}^{n+1,(0)}\right)\nonumber\\
&& - \half\left(\sum_k\nabla\cdot\frac{\xi_k^n k_{\rm th}^n}{c_p^n}\nabla X_k^n + \sum_k\nabla\cdot\frac{\xi_k^n k_{\rm th}^n}{c_p^n}\nabla\widehat{X}_k^{n+1,(0)}\right)\nonumber\\
&& - \half\left(\nabla\cdot\frac{h_p^n k_{\rm th}^n}{c_p^n}\nabla p_0 + \nabla\cdot\frac{h_p^n k_{\rm th}^n}{c_p^n}\nabla p_0\right),\label{eq:sdc enthalpy 2}
\end{eqnarray}
which is equivalent to
\begin{eqnarray}
\left(\rho^{n+1} - \frac{\Delta t}{2}\nabla\cdot\frac{k_{\rm th}^n}{c_p^n}\nabla\right)\widehat{h}^{n+1,(0)} &=& (\rho h)^n + \Delta t\Bigg[A_{\rho h}^{(0)} + I_{\rho h}^{\rm lagged} + \left(\half\nabla\cdot\frac{\kth^n}{c_p^n}\nabla h^n\right)\nonumber\\
&&\hspace{0.85in} - \half\left(\sum_k\nabla\cdot\frac{\xi_k^n k_{\rm th}^n}{c_p^n}\nabla X_k^n + \sum_k\nabla\cdot\frac{\xi_k^n k_{\rm th}^n}{c_p^n}\nabla\widehat{X}_k^{n+1,(0)}\right)\nonumber\\
&&\hspace{0.85in} - \half\left(\nabla\cdot\frac{h_p^n k_{\rm th}^n}{c_p^n}\nabla p_0 + \nabla\cdot\frac{h_p^n k_{\rm th}^n}{c_p^n}\nabla p_0\right)\Bigg].
\end{eqnarray}

{\bf Step 2C:} ({\it Advance thermodynamic variables})\\ \\
Define $Q_{\rho X_k}^{(0)}$ as the right hand side of (\ref{eq:sdc species 2}) without the $I_{\rho X_k}^{\rm lagged}$ term, and define $Q_{\rho h}^{(0)}$ as the right hand side of (\ref{eq:sdc enthalpy 2}) without the $I_{\rho h}^{\rm lagged}$ term.  Use {\tt VODE} to integerate (\ref{eq:sdc species}) and (\ref{eq:sdc enthalpy}) over $\Delta t$ to advance $(\rho X_k, \rho h)^n$ to $(\rho X_k, \rho h)^{n+1,(0)}$ using the piecewise-constant advection and diffusion source terms:
\begin{eqnarray}
\frac{\partial(\rho X_k)}{\partial t} &=& Q_{\rho X_k}^{(0)} + \rho\dot\omega_k\\
\frac{\partial(\rho h)}{\partial t} &=& Q_{\rho h}^{(0)} + \rho\Hnuc.
\end{eqnarray}
At this point we can define $I_{\rho X_k}^{(0)}$ and $I_{\rho h}^{(0)}$, or whatever term we need depending on our species and enthalpy edge state prediction types, for use in the corrector step.  In our first implementation, we are predicting $\rho X_k$ and $\rho h$, in which case we define:
\begin{eqnarray}
I_{\rho X_k}^{(0)} &=& \frac{(\rho X_k)^{n+1,(0)} - (\rho X_k)^n}{\Delta t} - Q_{\rho X_k}^{(0)}\\
I_{\rho h}^{(0)} &=& \frac{(\rho h)^{n+1,(0)} - (\rho h)^n}{\Delta t} - Q_{\rho h}^{(0)}.
\end{eqnarray}

{\bf Step 3:} ({\it Update advection velocities})\\ \\
First, compute $S^{n+\myhalf}$ and $\beta_0^{n+\myhalf}$ using
\begin{equation}
S^{n+\myhalf} = \frac{S^n + S^{n+1,(0)}}{2}, \qquad \beta_0^{n+\myhalf} = \frac{\beta_0^n + \beta_0^{n+1,(0)}}{2}.
\end{equation}
Then, project $\uadvsdcstar$ to obtain $\uadvsdc$, which satisfies
\begin{equation}
\nabla\cdot\left(\beta_0^{n+\myhalf}\uadvsdc\right) = S^{n+\myhalf}.
\end{equation}

{\bf Step 4:} ({\it Corrector Loop})\\ \\
We loop over this step from $j=1,j_{\rm max}$ times.  In the corrector, we use the time-advanced state from the predictor to perform a more accurate integration of the thermodynamic variables.  The quantities $(S, \beta_0, k_{\rm th}, c_p, \xi_k, h_p)^{n+1,(j-1)}$ are computed from $(\rho,\rho X_k,\rho h)^{n+1,(j-1)}$.  This step is divided into several sub-steps:\\

{\bf Step 4A:} ({\it Compute advective flux divergences})\\ \\
Use $\uadvsdc$ and a second-order Godunov integrator to compute time-centered edge states, $(\rho X_k, \rho h)^{n+\myhalf}$, with iteratively-lagged reactions ($I^{(j-1)}$), explicit diffusion, and time-centered thermodynamic pressure as source terms.  Define the advective flux divergences as
\begin{eqnarray}
A_{\rho X_k}^{(j)} &=& -\nabla\cdot\left[\left(\rho X_k\right)^{n+\myhalf,(j)}\uadvsdc\right],\\
A_{\rho h}^{(j)} &=& -\nabla\cdot\left[\left(\rho h\right)^{n+\myhalf,(j)}\uadvsdc\right] + \uadvsdc\cdot\nabla p_0.
\end{eqnarray}
Then, compute preliminary, time-advanced species using
\begin{equation}
\frac{\rho^{n+1}\widehat{X}_k^{n+1,(j)} - (\rho X_k)^n}{\Delta t} = A_{\rho X_k}^{(j)} + I_{\rho X_k}^{(j-1)}.\label{eq:sdc species 3}
\end{equation}

{\bf Step 4B:} ({\it Compute diffusive flux divergence})\\ \\
Solve a backward-Euler-type correction equation for $\widehat{h}^{n+1,(j)}$,
\begin{eqnarray}
\frac{\rho^{n+1}\widehat{h}^{n+1,(j)} - (\rho h)^n}{\Delta t} &=& A_{\rho h}^{(j)} + I_{\rho h}^{(j-1)}\nonumber\\
&& + \nabla\cdot\frac{\kth^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla\widehat{h}^{n+1,(j)} + \half\left(\nabla\cdot\frac{\kth^n}{c_p^n}\nabla h^n - \nabla\cdot\frac{\kth^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla h^{n+1,(j-1)}\right)\nonumber\\
&& - \half\left(\sum_k\nabla\cdot\frac{\xi_k^n k_{\rm th}^n}{c_p^n}\nabla X_k^n + \sum_k\nabla\cdot\frac{\xi_k^{n+1,(j-1)} k_{\rm th}^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla\widehat{X}_k^{n+1,(j)}\right)\nonumber\\
&& - \half\left(\nabla\cdot\frac{h_p^n k_{\rm th}^n}{c_p^n}\nabla p_0 + \nabla\cdot\frac{h_p^{n+1,(j-1)}k_{\rm th}^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla p_0\right),\label{eq:sdc enthalpy 3}
\end{eqnarray}
which is equivalent to
\begin{eqnarray}
\left(\rho^{n+1} - \Delta t\nabla\cdot\frac{\kth^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla\right)\widehat{h}^{n+1,(j)} &=& (\rho h)^n + \Delta t\Bigg[A_{\rho h}^{(j)} + I_{\rho h}^{(j-1)} \nonumber\\
&& + \half\left(\nabla\cdot\frac{\kth^n}{c_p^n}\nabla h^n - \nabla\cdot\frac{\kth^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla h^{n+1,(j-1)}\right)\nonumber\\
&& - \half\left(\sum_k\nabla\cdot\frac{\xi_k^n k_{\rm th}^n}{c_p^n}\nabla X_k^n + \sum_k\nabla\cdot\frac{\xi_k^{n+1,(j-1)} k_{\rm th}^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla\widehat{X}_k^{n+1,(j)}\right)\nonumber\\
&& - \half\left(\nabla\cdot\frac{h_p^n k_{\rm th}^n}{c_p^n}\nabla p_0 + \nabla\cdot\frac{h_p^{n+1,(j-1)}k_{\rm th}^{n+1,(j-1)}}{c_p^{n+1,(j-1)}}\nabla p_0\right)\Bigg].
\end{eqnarray}

{\bf Step 4C:} ({\it Advance thermodynamic variables})\\ \\
Define $Q_{\rho X_k}^{(j)}$ as the right hand side of (\ref{eq:sdc species 3}) without the $I_{\rho X_k}^{(j-1)}$ term, and define $Q_{\rho h}^{(j)}$ as the right hand side of (\ref{eq:sdc enthalpy 3}) without the $I_{\rho h}^{(j-1)}$ term.  Use {\tt VODE} to integerate (\ref{eq:sdc species}) and (\ref{eq:sdc enthalpy}) over $\Delta t$ to advance $(\rho X_k, \rho h)^n$ to $(\rho X_k, \rho h)^{n+1,(j)}$ using the piecewise-constant advection and diffusion source terms:
\begin{eqnarray}
\frac{\partial(\rho X_k)}{\partial t} &=& Q_{\rho X_k}^{(j)} + \rho\dot\omega_k\\
\frac{\partial(\rho h)}{\partial t} &=& Q_{\rho h}^{(j)} + \rho\Hnuc.
\end{eqnarray}
At this point we can define $I_{\rho X_k}^{(j)}$, $I_{\rho h}^{(j)}$, and any other $I$ terms we need depending on
our species and enthalpy edge state prediction types, for use in the predictor in the next time step.  In our first implementation, we are predicting $\rho X_k$ and $\rho h$, in which case we define:
\begin{eqnarray}
I_{\rho X_k}^{(j)} &=& \frac{(\rho X_k)^{n+1,(j)} - (\rho X_k)^n}{\Delta t} - Q_{\rho X_k}^{(j)}\\
I_{\rho h}^{(j)} &=& \frac{(\rho h)^{n+1,(j)} - (\rho h)^n}{\Delta t} - Q_{\rho h}^{(j)}.
\end{eqnarray}

{\bf Step 5:} ({\it Advance velocity and dynamic pressure})\\ \\
Similar to the original \maestro\ algorithm, more to come.


\section{Design}

\subsection{Decision - Red-Black Tree}

After examining the time complexity for various dictionary implementations, I decided a red-black tree was the most effective implementation for a general-purpose dictionary.

\begin{table}[hp]
\centering
\begin{tabular}{| l | l | l | l |}
\hline
\textbf{Implementation} & \textbf{Search} & \textbf{Insert} & \textbf{Delete} \\ \hline
AVL tree & O(log n) & O(log n) & O(log n) \\ \hline
\color{red} Red-black tree & \color{red} O(log n) & \color{red} O(log n) & \color{red} O(log n) \\ \hline
B-Tree & O(log n) & O(log n) & O(log n) \\ \hline
Splay tree & (a) O(log n) & (a) O(log n) & (a) O(log n) \\ \hline
Skip list & O(n) & O(n) & O(n) \\ \hline
Trie & O(m) & O(m) & O(m) \\ \hline
Stratified tree & O(log log N) & O(log log N) & O(log log N) \\ \hline
\end{tabular}
\\{\tiny{(n = number of elements in data structure, m = length of string, N = size of universe, (a) = amortised)}}\\
\caption{Considered data structures along with their worst-case time complexities.}
\end{table}

When deciding on the optimal data structure, I first ruled out splay tree and skip list, since the skip list worst case time is above the required O(log n) and there are other data structures faster than the splay tree where amortisation is not required. I then ruled out Trie, although it is extremely fast (independent of size of dictionary), since I aimed to implement a general-purpose dictionary and Trie only holds strings \cite{trie}.

Initially I implemented basic functionality of a stratified tree \cite{stratified}, also known as a van Emde Boas tree. The implementation had extremely fast lookup (\code{contains}, \code{max}, \code{min} O(1), and \code{successor}, \code{predecessor}, \code{add}, \code{delete} O(log log N)). However, my primary aim for this project was a \textit{general-purpose} and \textit{unbounded} dictionary. van Emde Boas trees are extremely fast, however the domain is bounded and each element requires an associated integer key. In the end I decided that van Emde Boas trees are too restrictive, and also more suited for a dictionary interface defined using keys and values (like \code{java.util.Map.put(K key, V value)}) and thus a more general purpose and unbounded but less efficient O(log n) solution would be more appropriate for this situation.

A red-black tree and a B-Tree seem to be very similar in efficiency, especially since a red-black is simply a type of B-Tree, as implied by the red-black tree's original name: a symmetric binary B-Tree \cite{bayer}. It was challenging to find a significant advantage of red-black trees over B-Trees, however I chose to rule out B-Trees since they may waste space if a node is not full, whereas red-black trees do not have this disadvantage.

Finally, I chose a red-black tree over an AVL tree after considering the usage of the dictionary described in the project: keeping track of users logged in to a web-service. I assume that in most cases, more users log on and off than search for each other, so \textit{insertions and deletions} are more common than \textit{lookups}. Since red-black trees have generally faster inserts and deletions \cite{clrs} - whereas AVL trees are faster at searches with slower modifications - a red-black tree would be the most appropriate choice for this scenario.

\subsection{Implementation}

The majority of my implementation is adapted from the pseudocode in the book `Introduction to Algorithms: Third Edition' by T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. However, due to the specified requirements of the dictionary, my adaptation of the red-black tree has ended up with various modifications that differ significantly from the CLRS description. In this section, I detail the implementation of the \code{RedBlackTree} class.

\subsubsection{Searching}
I considered various options for the \code{contains()} method. My final implementation runs in logarithmic time by using the typical binary search tree search (traversing down the tree). However, I also considered the option of having a separate internal data structure such as a hash table. A hash table would provide \textit{much} faster lookups on average, but unfortunately still has a O(n) worst case. The project specifications require O(log n) worst case, and storing the data in a hash table as well would also essentially double the space requirement. In the end, I decided a basic tree search would suffice for this method.

\subsubsection{Successors/predecessors}
This part of the dictionary took several refactorings and was the most challenging part of the project to implement. At first I developed the methods to first locate the node with the given value, and then find the successor of that node using well-known binary search tree methods. Therefore initially, \code{hasPredecessor(Object)} and \code{hasSuccessor(Object)} would take logarithmic time, as they would first check whether the element was contained in the dictionary (O(log n)) and then check whether it was within the bounds of the dictionary (not $\leq$ min or $\geq$ max respectively). However after implementation I realised that it was not required that the argument be contained in the dictionary. Therefore \code{hasPredecessor(Object)} and \code{hasSuccessor(Object)} could be reduced to constant time - since the argument only has to be less than the maximum element to have a successor, and vice versa for predecessor. My final design for \code{successor(Object)} uses a method \code{above(Node)}, which finds the least node strictly greater than the argument (and \code{below(Node)} is symmetric for \code{predecessor(Object)}).

\subsubsection{Minimum/maximum}
Originally I implemented \code{max()} and \code{min()} to simply call the helper methods \code{minimum(Node)} and \code{maximum(Node)} with the root as the argument. This takes logarithmic time. However, I later modified the class to maintain two extra fields \code{min} and \code{max}. Upon insertion, the current values of \code{min} and \code{max} are compared with the newly inserted element - if the new node is greater than the max or less than the min, the references are updated. This adds a worst-case cost of 2  extra comparisons to the \code{add(Object)} method and reduces \code{min()} and \code{max()} to O(1).

Upon deletion, if the item being deleted is the minimum or the maximum, then the min/max is found by searching the tree again from the root. This is a minor cost when compared to the performance gains from constant time access to the extremum of the dictionary (and doesn't make the time complexity of \code{delete(Object)} any worse).

\subsubsection{Insertion/deletion}

\begin{figure}[b]
\centering
\raisebox{-.5\height}{\includegraphics[width=0.25\textwidth]{resources/before}}
$\implies$
\raisebox{-.5\height}{\includegraphics[width=0.25\textwidth]{resources/after}}
\caption{Inserting characters \textit{a} to \textit{m} into the red-black tree, and then deleting the root \textit{d}. Note that in both cases, the height $h = 5$ and for a dictionary with 12-13 nodes the height bound is $h \leq 2\log_2(12+1) \approx 7$ which is satisfied by both structures.}
\end{figure}

The \code{add(Object)} and \code{delete(Object)} methods are where my implementation has strong similarities to the red-black tree detailed in CLRS.

The \code{add(Object)} method creates a new node and calls the \code{insert(Node)} method, which simply finds the appropriate location for the new node, inserts it there, colours it red, and then restores the red-black tree properties using \code{fixInsert(Node)}.

The \code{delete(Object)} method first locates the node to delete using a basic tree search, and replaces it with one of its children if necessary. Then it calls \code{fixDelete(Node)} to restore any red-black tree properties that may have been violated.

Since detailing how red-black tree insertion and deletion work would take more space than I have room for in this report, I will instead detail some implementation decisions used in regards to \code{add} and \code{delete} (the add- and delete-related code is commented in the \code{TreeIterator} class).

I chose to use a sentinel node \code{nil} for inserting and deleting, rather than null pointers. This is suggested by CLRS, and it has the advantage of not having to consider exceptional cases as frequently - instead of having to avoid dealing with null as a separate case, the \code{nil} node can be treated as any other node and have its \code{parent}, \code{left}, and \code{right} fields take on arbitrary values.

I felt one disadvantage to my insertion and deletion related code is that there is a lot of symmetrical/repeated code, that deals with the same case, but having every reference to a `left' child replaced with a `right' child, and vice versa. In development I was unable to find a clean way that would enable me to abstract this to deal with both left and right cases.

\subsubsection{Iteration}

In implementing the above methods, I tried to `modularise' my code as much as possible - that is, for example, instead of putting the deletion code inside the public \code{delete(Object)} method, I extracted it into a separate \code{delete(Node)} method. This gave me the advantage of being able to easily implement the iterator in a short amount of time, since I could rely on \code{successor(Node)} and \code{delete(Node)} to perform all operations needed by the \code{TreeIterator} class.

The iterator constructor takes in a start node, stores a reference to that start node in an instance variable, and then on each call to \code{next()}, `increments' the node by replacing it with its successor (using \code{successor(Node)}) and returns the previous value.

To implement \code{Iterator\#delete()}, I also stored a reference to the last node returned by \code{next()}. On deletion, I simply call the \code{delete(Node)} method in the enclosing \code{RedBlackTree} class, and then set \code{last} to the \code{nil} sentinel. Setting this field to \code{nil} also works as a check to ensure that the iterator state is valid - if \code{last} is \code{nil}, then we know that either \code{next()} hasn't been called yet or \code{last} has already been deleted. In this case, I throw an \code{IllegalStateException}.

Ensuring the iterator is fail-fast is also implemented in a fairly simple way - in the \code{RedBlackTree} class, a field \code{operations} keeps track of the total number of successful insertions and deletions on the dictionary. In the constructor of \code{TreeIterator}, this number is recorded. Then, each time a method is called on the iterator class, the two numbers are compared - if the number of operations on the tree is not the same as when the iterator was constructed, then a \code{ConcurrentModificationException} is immediately thrown.

\subsubsection{Logging}

Each action made on the dictionary will cause a new line to be appended to the log string. \code{getLogString()} will return this string and clear the log.

Each line of the log string reports the name of the method called, the value of the argument if applicable, and the number of comparisons used for that method.

Note that an assumption I have made in development is that the number of comparisons is the number of calls to \code{compareTo}, plus, in general, the number of significant colour comparisons (e.g. in \code{fixInsert(Node)} the number of times the parent node's colour is checked) and significant reference equality checks (e.g. moving down the tree in the \code{minimum(Node)} method.