\documentclass{article}

\usepackage{url}
\usepackage{natbib}
\usepackage{graphicx}
\usepackage{placeins}
\usepackage{tabularx,ragged2e,booktabs,caption}
\usepackage{amsmath}
\usepackage{lscape}
\usepackage{hyperref}
\usepackage[margin=1in]{geometry}
\usepackage[none]{hyphenat}

\begin{document}

\title{Moist Adiabats: Quest for Analytic Solution v2.0}
\author{Nadya Moisseeva}
\date{November 2015}    % type date between braces

\maketitle
% \tableofcontents

\newpage
\section*{Overview}
This document is intended to summarize the proposed approach to analytical modelling of moist adiabats. The current summary offers a simple (but rather inelegant) answer to: \emph{How can we determine temperature $T$, given pressure $P$ and moist adiabat $\Theta_w$?}

\FloatBarrier
\section*{Problem Setup}
The preliminary steps follow the solution originally suggested by R.Stull, with several modifications introduced to increase the precision of directly calculated moist adiabats. The following changes were made to the original method:
\begin{itemize}
\item{adjusted several constant values, including adjusted Clausius-Clayperon constant (Koutsoyiannis 2011)}
\item{replaced original Clausius-Clayperon relation with updated formula (Koutsoyiannis 2011)}
\item{introduced temperature-dependent latent heat of vaporization constants}
\item{introduced alternative $\Theta_e$ formula}
\item{increased pressure range to the top of the atmosphere (1kPa)}
\item{decreased pressure calculation step to 0.001 kPa}
\item{increased temperature range to -40C - 40C}
\end{itemize}

A sample of the resultant curves can be seen in Fig.~\ref{stuve}. Following Stull's original approach, moist adiabats can be normalized using equivalent potential temperature $\Theta_e$ and dry potential temperature $\Theta_d$, to produce a set of curves shown in Fig.~\ref{norm_theta}. From this point, Stull's solution attempts to normalize the pressure range, such that the curves collapse into a single shape. Based on the current investigation, it appears that the nature of the underlaying nonlinearity in the problem doesn't lend itself to such direct normalization. Hence, the current solution proceeds on a different path described below.

\begin{figure}
\centering
\includegraphics[width=4.5in]{../figs/stuve.pdf}
\caption{Directly calculated curves for moist adiabats, potential and equivalent potential temperatures. Limited temperature range shown. }\label{stuve}
\end{figure}

\begin{figure}
\centering
\includegraphics[width=4.5in]{../figs/norm_theta.pdf}
\caption{Normalized moist adiabats, following R.Stull. Select curves shown. }\label{norm_theta}
\end{figure}

\FloatBarrier
\section*{Modelling the Effects of Pressure on Normalized Curves}
The first step of the current solution attempts to remove some of the inherent nonlinearity by further normalizing the set of curves shown in Fig.\ref{norm_theta}. By choosing one of the normalized adiabats to represent a pressure reference curve $P_{ref}$, Fig.\ref{norm_theta} can be reduced to Fig.\ref{trans_adiabats}. The transformed adiabats shown are produced using a normalization with $P_{ref} = +40C $. This particular choice of $P_{ref}$ does not imply its theoretical importance. It is possible to choose any of the directly calculated normalized adiabats to represent $P_{ref}$. Depending on the choice, the resultant transformed adiabats will shift around the $P_{ref}$ unity line. The single consideration for choosing a particular $P_{ref}$ is the availability of simple analytical function that can represent all of the transformed curves.

\begin{figure}
\centering
\includegraphics[width=4.5in]{../figs/trans_adiabats.pdf}
\caption{Transformed normalized adiabats. Colors correspond to negative (blue) and positive (red) temperatures.}\label{trans_adiabats}
\end{figure}

$P_{ref}$ can easily be modelled analytically using polynomial functions. This is convenient, since polynomials are generally well-behaved functions. The choice of the degree of polynomial depends on the desired precision level. Since we are examining a fixed range of temperatures relevant to atmospheric applications, the potentially chaotic behavior of high degree polynomials outside of the modelled range is not a primary concern. For this example, an extremely high accuracy threshold was chosen: the aim was to ensure that the mean average error (MAE) is on the order of $1x10^{-5}$. The true and modelled $P_{ref} = +40C$ can be seen in Fig.\ref{pref}. The fit was produced using a 28th degree polynomial.

\begin{figure}
\centering
\includegraphics[width=4.5in]{../figs/pref_fit.pdf}
\caption{True and modelled $P_{ref} = +40C$ (difference between curves not apparent at this scale).}\label{pref}
\end{figure}

The next step is to choose a single functional form, which could represent the entire family of curves in Fig.\ref{trans_adiabats}. Each given shape of a particular curve is then controlled by variable parameters of the same function. A number of simple functions exist, which are able to model the relationship. For this work bi-exponential, arctan, rational and polynomial functions were tested. Generally, a reasonable fit can be achieved with both bi-exponential and arctan functions using as little as three variable parameters. While efficient, the results of such fit are unlikely to be sufficiently accurate to be useful. Another concern with these choices, is that the the variable parameters are not well-behaved functions and are hence difficult to model.

Polynomial fitting doesn't appear to suffer from such issues. Moreover, the degree of accuracy can be controlled by increasing the degree of the polynomial and, hence, allowing a higher number of variable parameters. In this example, the curves were modelled using 8th degree polynomials, resulting in 9 variable parameters. Conveniently, and unlike other functional forms mentioned above, these parameters are also well-behaved. They can, again, be modelled using high-degree polynomials to the desired level of accuracy. Fig.\ref{fit_params} shows the results of parameter fitting for this given example. A demanding threshold of $MAE<1x10^{-6}$ was chosen for this example, and achieved using polynomial degrees of 17-24 (depending on the particular parameter).

\begin{figure}
\centering
\includegraphics[width=7in]{../figs/fit_params.pdf}
\caption{True and modelled polynomial parameter fitting (difference between curves not apparent at this scale).}\label{fit_params}
\end{figure}

\FloatBarrier

\section*{Error Testing}

To test the accuracy of the proposed method, the moist adiabats can be modelled using the analytical approach and subsequently compared to those produced by direct iterative computation. For given $\Theta_w$, parameters $k1-k9$ in Fig.\ref{fit_params} are calculated using corresponding polynomial functions. For given $P$, $P_{ref}$ is derived using it's own polynomial relationship (Fig.\ref{pref}). Finally, normalized moist theta is calculated using 8th degree polynomial for given $P_{ref}$ and parameters $k1-k9$. The results are shown in Fig.\ref{final_fit}.

\begin{figure}
\centering
\includegraphics[width=4.5in]{../figs/final_fit.pdf}
\caption{True and modelled normalized moist theta (difference between curves not apparent at this scale). Select curves shown.}\label{final_fit}
\end{figure}

As mentioned earlier, higher accuracy may be achieved given the willingness to use even higher degrees of polynomials for fits. This is as effective as it is inelegant. For the current example the mean error profile for all 80 modelled adiabats is shown in Fig.\ref{error}.

\begin{figure}
\centering
\includegraphics[width=4.5in]{../figs/error_profile.pdf}
\caption{Vertical mean error profile.}\label{error}
\end{figure}

\end{document}


\chapter{Results}
\label{chap:results}

This chapter first explores the behavior of Sum and Product Coupling.
Then it compares to PAIRpred and the other variants of graph convolution.
Some convolution filters are then visualized, along with network output for one trained network.
Finally, it summarizes the effect of protein size on network training time.

\section{Sum and Product Coupling Performance}

\begin{table}
	\begin{center}
		\begin{tabular}{lccccc}
			\toprule
			\multirow{2}{*}{Method} &
			Receptive Field & \multicolumn{4}{c}{Layers Before Merge} \\
			& Size & 1 & {2} & {3} & {4} \\
			\midrule
			No Convolution & N/A & \textbf{0.815} & 0.812 & 0.800 & 0.811  \\\cline{1-6}
			\multirow{2}{*}{Sum Coupling} & 11 & 0.868 & 0.889 & 0.882 & 0.884 \\
			& 21 & 0.875 & \textbf{0.903} & 0.880 & 0.890 \\\cline{1-6}
			\multirow{2}{*}{Product Coupling} & 11 & 0.856 & 0.869 & 0.885 & 0.868 \\
			& 21 & 0.863 & 0.876 & 0.896 & \textbf{0.899} \\
			\bottomrule
		\end{tabular}
		\caption{Median area under the receiver operating characteristic curve (AUC) across all complexes in the test set for two variants of graph convolution, Sum Coupling and Product Coupling, as well as No Convolution. Results are shown for two different sizes of receptive field, 11 and 21, for different numbers of convolutional layers before the pairwise merge operation. Bold faced values indicate best performance for each method.
		\label{tab:med_auc}}
	\end{center}
\end{table}

Table \ref{tab:med_auc} shows the results of experiments involving Sum Coupling and Product Coupling, as well as No Convolution.
Comparing Sum and Product Coupling to No Convolution reveals that convolution is beneficial.
In other words, incorporating information from neighboring residues helps indicate whether a residue is part of an interface, which is consistent with the biological properties of interfaces.
It's also clear that a receptive field of size 21 is generally better than 11.
Interestingly, this value is near the median number of interface residues for a single protein, which is 27.
When using convolution, performance improves with network depth up to a point, then either decreases or remains relatively constant.
In Sum Coupling, this maximum occurs in the second layer, whereas in Product Coupling, this maximum occurs in the fourth layer.
Though not shown in the table, Product Coupling indeed fails to improve when using 5 or 6 layers.
In contrast, networks without convolution are best with only one pre-merge layer.
This suggests that depth alone does not improve performance, but when convolution is performed, a useful hierarchical representation is learned.
Other applications of deep learning have seen this same trend of increasing and decreasing performance.
The commonly given explanations for such behavior include insufficient training data, a vanishing gradient, and a difficulty with optimization~\cite{he2015}.
Overall, both Sum and Product Coupling achieved similar performance in AUC.

\begin{table}
	\begin{center}
		\begin{tabular}{lccccc}
			\toprule
			\multirow{2}{*}{Method} &
			Receptive Field & \multicolumn{4}{c}{Layers Before Merge} \\
			& Size & 1 & {2} & {3} & {4} \\
			\midrule
			No Convolution & N/A & \textbf{48} & 55 & 53 & 66 \\\cline{1-6}
			\multirow{2}{*}{Sum Coupling} & 11 & 32 & 28 & 70 & 86 \\
			& 21 & \textbf{26} & 37 & 56 & 63 \\\cline{1-6}
			\multirow{2}{*}{Product Coupling} & 11 & 30 & 46 & 26 & 51 \\
			& 21 & 26 & \textbf{25} & 36 & 37 \\
			\bottomrule
		\end{tabular}
		\caption{Median rank of the first positive prediction (RFPP) across all complexes in the test set for two variants of graph convolution, Sum Coupling and Product Coupling, as well as No Convolution. Results are shown for two different sizes of receptive field, 11 and 21, for different numbers of convolutional layers before the pairwise merge operation. Bold faced values indicate best performance for each method (lower is better).}
		\label{tab:med_rfpp}
	\end{center}
\end{table}

Table \ref{tab:med_rfpp} parallels Table \ref{tab:med_auc} but shows RFPP instead of AUC.
As before, Sum and Product Coupling perform similarly to each other, and both outperform No Convolution.
In this case, however, the best performance for Sum and Product Coupling is seen for fewer layers than AUC.
This difference is not surprising, considering the cross-entropy loss function leads to optimization of performance on \emph{all} pairs, not just the top scoring ones.
In other words, RFPP is not being explicitly optimized in this problem, whereas AUC is more closely related to the quantity being optimized.

\begin{figure}
	\includegraphics[width=\textwidth]{med_auc.png}
	\caption{Median area under the receiver operating characteristic curve (AUC) across all complexes in the test set, separated by complex class. Sum and Product Coupling are shown for two receptive field sizes each (11 and 21), as well as No Convolution, for 1-4 pre-merge layers. Product Coupling performs better for difficult complexes, but worse overall because ther are far more rigid and medium difficulty complexes.
		\label{fig:med_auc}}
\end{figure}

To understand the behavior of each method in more detail, we can separate performance by the difficulty class of the test proteins.
Figure \ref{fig:med_auc} shows performance for rigid, medium difficulty, and difficult classes, with 33, 16, and 6 complexes respectively.
Here it appears that Sum and Product Coupling are closely matched for rigid and medium difficulty.
A slight difference is seen for the difficult complexes, where Product Coupling appears to be performing better for two and three layers, but with so few complexes it's unclear if this is a true trend.

\begin{figure}
	\includegraphics[width=0.8\textwidth]{sum_20_2_histo1.png}
	\caption{Histogram of area under the receiver operating characteristic curve (AUC) for complexes in the test set, colored by difficulty class. Scores are from Sum Coupling with two layers and receptive field size 21, which had the highest median AUC of all methods.}
	\label{fig:histo1}
\end{figure}

For another picture of performance across difficulty classes, we can examine a histogram of AUCs, as shown in Figure \ref{fig:histo1}.
These AUCs are heavily skewed, justifying the choice of median as a summary measure.
Surprisingly, there is no clear divide between rigid, medium difficulty, and difficult classes.
In fact, the worst AUC is a rigid complex, and one difficult complex achieves AUC above 0.9.
It appears that the distinguishing characteristic between classes is the number of complexes that achieve above 0.95 AUC.
These "trivial" complexes are most frequent in the rigid class, less so in the medium difficulty class, and absent for the difficult class.
This suggests that the networks are effectively coping with some complexes that under conformational change.

\begin{figure}
	\includegraphics[width=\textwidth]{med_rfpp.png}
	\caption{Median rank of the first positive prediction (RFPP) across all complexes in the test set, separated by difficulty class. Vertical axes are log scaled. Sum and Product Coupling are shown for two receptive field sizes each (11 and 21), as well as No Convolution, for 1-4 pre-merge layers. Lower RFPP is better. Best performance on rigid complexes is achieved with just one layer for all networks. As difficulty increases, so does the number of layers needed to achieve best results.
		\label{fig:med_rfpp}}
\end{figure}

RFPP can also be separated by difficulty class.
From figure \ref{fig:med_rfpp} we can observe some heterogeneus behavior across methods, but there are some trends worth noting.
For each class, there appears to be a favored number of layers where a dip (improvement) in RFPP is observed.
This optimal depth appears to increase with difficulty class, suggesting that harder complexes require more layers to achieve best performance.
Therefore an ensemble approach with varying depths may perform well on complexes of any difficulty.

\section{Comparison to Other Methods}

\begin{table}
	\begin{center}
		\begin{tabular}{l c c c c c }
			\toprule

			\multirow{2}{*}{Method} & \multirow{2}{*}{Variant} &  \multicolumn{4}{c}{Layers} \\
			& & 1 & 2 & 3 & 4 \\

			\midrule
			No Convolution & N/A & \textbf{0.815} & 0.812 & 0.800 & 0.811 \\
			\midrule
			\multirow{2}{*}{Sum Coupling} & $\text{RF}=11$ & 0.868 & 0.889 & 0.882 & 0.884 \\
			& $\text{RF}=21$ & 0.875 & \textbf{0.903} & 0.880 & 0.890 \\
			\midrule
			\multirow{2}{*}{Product Coupling} & $\text{RF}=11$ & 0.856 & 0.869 & 0.885 & 0.868 \\
			& $\text{RF}=21$ & 0.863 & 0.876 & 0.896 & \textbf{0.899} \\
			\midrule
			PAIRpred~\cite{minhas2014}   & N/A  & (\textbf{0.863})* & - & - & - \\

			\midrule
			\multirow{2}{*}{PATCHY-SAN~\cite{niepert2016}}  & $\text{RF}=11$ & 0.862 & 0.867 & 0.883 & 0.891 \\
			& $\text{RF}=21$ & 0.850 & 0.875 & \textbf{0.897} & 0.886 \\
			\midrule
			\multirow{2}{*}{Fingerprint~\cite{duvenaud2015}}  & $\text{RF}=11$ & 0.857 & 0.850 & 0.863 & 0.833 \\
			& $\text{RF}=21$ & 0.861 & 0.867 & 0.881 & \textbf{0.891} \\
			\midrule

			\multirow{6}{*}{R-GCN~\cite{schlichtkrull2017}} & No basis fns, $\text{RF}=11$ & 0.862 & 0.871 & 0.886 & 0.893 \\
			& No basis fns, $\text{RF}=21$ & 0.876 & \textbf{0.901} & 0.892 & 0.897 \\
			& 2 basis fns, $\text{RF}=11$ & 0.851 & 0.872 & 0.779 & -	   \\
			& 2 basis fns, $\text{RF}=21$ & 0.873 & 0.804 & 0.539 & -     \\
			& 5 basis fns, $\text{RF}=11$ & 0.870 & 0.747 & 0.748 & -	   \\
			& 5 basis fns, $\text{RF}=21$ & 0.867 & 0.900 & 0.709 & -     \\
			\midrule
			\multirow{2}{*}{DTNN~\cite{schutt2017}}& $\text{RF}=11$ & 0.853 & 0.872 & 0.878 & 0.861 \\
			& $\text{RF}=21$ & 0.862 & 0.880 & 0.873 & \textbf{0.885} \\
			\midrule
			\multirow{4}{*}{DCNN~\cite{atwood2016}} & 2 hops, $\sigma=2$\AA{} & 0.782 & - & - & - \\
			& 2 hops, $\sigma=4$\AA{} & 0.801 & - & - & - \\
			& 5 hops, $\sigma=2$\AA{} & \textbf{0.838} & - & - & - \\
			& 5 hops, $\sigma=4$\AA{} & 0.819 & - & - & - \\

			\bottomrule

		\end{tabular}
		\caption{Comparison of proposed convolutions with existing classification methods. Bold values indicate best performance for each method. For reference, retraining Sum Coupling ten times with a different random seed yields a standard deviation of 0.006, and other methods are believed to behave similarly. *PAIRpred is an SVM-based approach, so the result is not really associated with a layer number.}
		\label{tab:results_compare}
	\end{center}
	%\end{minipage}
\end{table}

Table \ref{tab:results_compare} compares the median AUC of the various existing methods discussed.
PAIRpred, the state of the art pairwise interface predictor, establishes the baseline.
Interestingly, most of the graph convolution based methods exceed this.
DCNN is the exception, showing worse performance than PAIRpred.
This is perhaps unsurprising, since it differs considerably from the other convolution methods, which are very similar to one another.

The order imposing PATCHY-SAN performs quite well, suggesting that the chosen ordering of neighbors by proximity to the central vertex was a good one.
Its best AUC of 0.897 is within one standard deviation (0.006) of the best overall performance, which was 0.903 from Sum Coupling.
Therefore it is difficult to say whether order-free or ordered methods perform better for this problem.
Only one ordered method was examined, leaving the possibility that others may do better.
Because of the unique weights used for each neighbor, ordered methods have significantly more parameters than order-free methods, making them more susceptible to overfitting, though that is not observed in these results.
This method also uses information from all edges in the receptive field, whereas the other methods are only concerned with edges connecting neighbors to the central vertex.

R-GCN did the best without basis matricess.
The original intent for basis matricess was to allow for weight sharing across several relation types.
This is less relevant for these protein graphs since only a single neighborhood is being considered per central vertex, so in a sense there is only one relation type.
When using basis matricess, larger networks (starting at three layers) drop significantly in performance compared to other methods.
At four layers, networks consistently classified all residue pairs with the same score, suggesting all filters had become zero valued (note that AUC cannot be calculated with all scores are identical).
Note that AUC cannot be calculated when all scores are identical.
This is possible when using ReLU nonlinearities, since the output and gradient vanishes for signals less than zero.
It's possible that a different initialization scheme or a different nonlinearity would remedy this problem.
Regardless, the use of basis matricess in this context is somewhat unjustified because these graphs lack the numerous relation types that prompted use of basis matricess in the first place~\cite{schlichtkrull2017}.
Nevertheless, the two layer, five basis matrices version performed similarly to the best results observed.
But testing with 8 basis matrices did not show improvement.
Without basis matricess, R-GCN is equivalent to Sum Coupling if edge information is excluded.
The fact that R-GCN without basis matricess performs similarly to Sum Coupling suggests that the added edge information in Sum Coupling is not very useful.

Fingerprint is arguably the simplest form of graph convolution since it uses the same weight matrix for both central and neighbor vertices.
Its best performance is only 0.01 below that of R-GCN, but it requires twice as many layers.
The number of layers to five or six exhibits no substantive improvement.

Whereas PATCHY-SAN, Fingerprint, and R-GCN are most similar to Sum Coupling, DTNN allows for a multiplicative coupling between neighbors and edges, similar to Product Coupling.
Like Product Coupling, best performance is achieved at a greater depth than the other methods, but does not continue to improve for five or six layers.
This method is notably worse than the three just mentioned.
One potentially limiting aspect of this method is that the number of channels is necessarily unchanged.
This may explain why this method performs worse than Product Coupling.

For DCNN, five hops performs better than two hops, presumably for the same reason that a larger receptive field is better in the other convolution methods: information is able to propagate further across the graph.
The Gaussian standard deviation determines the range of diffusion, where smaller values limit diffusion to a localized neighborhood for each hop, and larger values allow diffusion across longer distances.
In the two hop DCNN, the larger standard deviation allows more diffusion to occur across the graph, compensating for the limited number of hops.
This explains the overall better performance for $\sigma=4\AA{}$.
In contrast, the five hop DCNN performed better for $\sigma=2\AA{}$, suggesting that the larger number of hops allows sufficient information propagation, eliminating the need for diffusion across greater distances for each hop.

\section{Filter Visualization}

It is often difficult to understand the behavior of a model by simply looking at the overall performance.
For convolutional neural networks on images, intuition can be gained through a variety of methods, including visualizing filters directly, tracking a filter's activation as different regions of the image are occluded, and identifying which images maximally activate each filter.
To understand the filters being learned in this pairwise neural network architecture, network scores and filter activations were mapped to a protein complex heatmap.
Figure \ref{fig:filter_vis} presents color maps on the complex 3HI6~\cite{zhang2009} which depict the true interface, maximum predicted scores for each residue, and the activation of two filters from last convolutional layer, for the best performing method (Sum Coupling with two layers and receptive field size of 21).
Scores are partner specific, so they are shown for a single residue after taking the maximum over all potential partners.
This partner independent visualization matches well with the true interface.
In contrast, convolutional filters occur at the residue level and can be visualized directly.
The two filters shown illustrate learned features which are useful for interface prediction.
Specifically, one activates only for buried residues (indicating an \emph{unlikeliness} to participate in an interface), and the other activates only for residues near the true interface.

\begin{figure}
	\includegraphics[width=0.6\textwidth]{3HI6_collage.png}
	\caption{PyMOL~\cite{schrodinger2015} visualizations of the best performing test complex (3HI6~\cite{zhang2009}). Upper left: Ligand (red) and receptor (blue), along with the true interface (yellow). Upper right: Visualization of predicted scores, where brighter colors (cyan and orange) are higher scores and darker colors (blue and red) are lower scores. Since scores are for pairs of residues, we take the max score over all partners in the opposing protein. Bottom row: Activations of two filters in the second convolutional layer, where brighter colors indicate greater activation and black indicates activation of zero. Lower left: Filter which provides higher activations for buried residues, a useful screening criterion for interface detection. Lower right: Filter which gives high activations for residues near the interface of this complex.
		\label{fig:filter_vis}}
\end{figure}

\section{Training Time}

Figure \ref{fig:train_times} shows a log-log plot of training time as a function of the number of residue pairs in a given complex.
The relationship is roughly linear in this space, indicating a power-law relationship.
Fitting lines to the data shows that the power is slightly greater than 1.0 in all cases, showing that training time increases near linearly in the number of pairs as expected.
This is due to the pairwise nature of the problem.
For problems on a single graph, the training time would likely scale near linearly in the number of vertices.
Deeper networks take longer to train, and summing over neighbors (as in Product Coupling), takes longer than just using the central vertex (as in No Convolution).
These trends are as expected.

\begin{figure}
	\includegraphics[width=1.0\textwidth]{training_time_product_coupling_1-4.png}
	\caption{Training time as a function of number of residue pairs, for a single No Convolution layer, as well as four depths of Product Coupling, receptive field size 21. Linearity in this log-log plot indicates a power law relationship. In this case, the relationship is near linear, with powers of 1.02, 1.22, 1.25, 1.25, and 1.24 respectively for No Convolution 1 layer, and Product Coupling for 1, 2, 3, and 4 layers. Other convolution methods have similar power law relationships.
		\label{fig:train_times}}
\end{figure}


\chapter{Loop Optimization}

    This section is specific to loop optimization and show several
tutorial examples using the optimization mechanisms within ROSE.
\fixme{We might want to reference Qing's work explicitly since this is
       really just showing off here work.}

\section{Example Loop Optimizer}
     Simple example translator showing use of pre-defined loop optimizations.
\fixme{We are not running performance tests within this tutorial, but perhaps we could later.}

   Figure~\ref{Tutorial:exampleLoopOptimization} shows the code required to
call some loop optimizations within ROSE.  The translator that we build for this tutorial
is simple and takes the following command line options to control which optimizations are done.

\begin{verbatim}
   -ic1 :loop interchange for more reuses
   -bk1/2/3 <blocksize> :block outer/inner/all loops
   -fs1/2 :single/multi-level loop fusion for more reuses
   -cp <copydim> :copy array
   -fs0 : loop fission
   -splitloop: loop splitting
   -unroll [locond] [nvar] <unrollsize> : loop unrolling
   -bs <stmtsize> : break up statements in loops
   -annot <filename>:
        Read annotation from a file which defines side effects of functions
   -arracc <funcname> :
        Use special function to denote array access (the special function can be replaced
        with macros after transformation). This option is for circumventing complex
        subscript expressions for linearized multi-dimensional arrays.
   -opt <level=0> : The level of loop optimizations to apply (By default, only the outermost
                 level is optimized).
   -ta <int> : Max number of nodes to split for transitive dependence analysis (to limit the
            overhead of transitive dep. analysis)
   -clsize <int> : set cache line size in evaluating spatial locality (affect decisions in
                applying loop optimizations)
   -reuse_dist <int> : set maximum distance of reuse that can exploit cache (used to evaluate
                 temporal locality of loops)
\end{verbatim}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleDirectory/loopOptimization.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleDirectory/loopOptimization.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Example source code showing use of loop optimization mechanisms.}
\label{Tutorial:exampleLoopOptimization}
\end{figure}

\clearpage
\section{Matrix Multiply Example}

   Using the matrix multiply example code shown in
figure~\ref{Tutorial:exampleInputCode_LoopOptimization}, we run the loop optimizer in
figure~\ref{Tutorial:exampleLoopOptimization} and generate the code shown in
figure~\ref{Tutorial:exampleOutput_LoopOptimization}.

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleDirectory/inputCode_LoopOptimization_blocking.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleDirectory/inputCode_LoopOptimization_blocking.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Example source code used as input to loop optimization processor.}
\label{Tutorial:exampleInputCode_LoopOptimization}
\end{figure}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_blocking.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_blocking.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Output of loop optimization processor showing matrix multiply
         optimization (using options: {\tt -bk1 -fs0}).}
\label{Tutorial:exampleOutput_LoopOptimization}
\end{figure}

\clearpage
\section{Loop Fusion Example}

   Using the loop fusion example code shown in
figure~\ref{Tutorial:exampleInputCode_LoopOptimization_fusion}, we run the loop optimizer in
figure~\ref{Tutorial:exampleLoopOptimization} and generate the code shown in
figure~\ref{Tutorial:exampleOutput_LoopOptimization_fusion}.

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleDirectory/inputCode_LoopOptimization_fusion.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleDirectory/inputCode_LoopOptimization_fusion.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Example source code used as input to loop optimization processor.}
\label{Tutorial:exampleInputCode_LoopOptimization_fusion}
\end{figure}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_fusion.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_fusion.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Output of loop optimization processor showing loop fusion (using options: {\tt -fs2}).}
\label{Tutorial:exampleOutput_LoopOptimization_fusion}
\end{figure}

\section{Example Loop Processor (LoopProcessor.C)}
     This section contains a more detail translator which uses the command-line for input
    of specific loop processing options and is more sophisticated than the previous
    translator used to handle the previous two examples.

   Figure~\ref{Tutorial:exampleLoopProcessor} shows the code required to
call the loop optimizations within ROSE.  The translator that we build for this tutorial
is simple and takes command line parameters to control which optimizations are done.

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/loopProcessor.C.aa}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/loopProcessor.C.aa}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Detailed example source code showing use of loop optimization mechanisms
    (loopProcessor.C part 1).}
\label{Tutorial:exampleLoopProcessor}
\end{figure}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/loopProcessor.C.ab}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/loopProcessor.C.ab}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{loopProcessor.C source code (Part 2).}
\label{Tutorial:exampleLoopProcessor2}
\end{figure}

\clearpage
\section{Matrix Multiplication Example (mm.C)}

   Using the matrix multiplication example code shown in
figure~\ref{Tutorial:exampleInputCode_LoopOptimization_mm}, we run the loop optimizer in
figure~\ref{Tutorial:exampleLoopProcessor} and generate the code shown in
figure~\ref{Tutorial:exampleOutput_LoopOptimization_mm}.

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleDirectory/inputCode_LoopOptimization_mm.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleDirectory/inputCode_LoopOptimization_mm.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Example source code used as input to loopProcessor, show in figure~\ref{Tutorial:exampleLoopProcessor}.}
\label{Tutorial:exampleInputCode_LoopOptimization_mm}
\end{figure}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_mm.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_mm.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Output of loopProcessor using input from figure~\ref{Tutorial:exampleInputCode_LoopOptimization_mm}
    (using options: {\tt -bk1 -fs0}).}
\label{Tutorial:exampleOutput_LoopOptimization_mm}
\end{figure}

 \clearpage
\section{Matrix Multiplication Example Using Linearized Matrices (dgemm.C)}

   Using the matrix multiplication example code shown in
figure~\ref{Tutorial:exampleInputCode_LoopOptimization_dgemm}, we run the loop optimizer in
figure~\ref{Tutorial:exampleLoopProcessor} and generate the code shown in
figure~\ref{Tutorial:exampleOutput_LoopOptimization_dgemm}.

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleDirectory/inputCode_LoopOptimization_dgemm.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleDirectory/inputCode_LoopOptimization_dgemm.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Example source code used as input to loopProcessor, show in figure~\ref{Tutorial:exampleLoopProcessor}.}
\label{Tutorial:exampleInputCode_LoopOptimization_dgemm}
\end{figure}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_dgemm.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_dgemm.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Output of loopProcessor using input from figure~\ref{Tutorial:exampleInputCode_LoopOptimization_dgemm}
    (using options: {\tt -bk1 -unroll nvar 16}).}
\label{Tutorial:exampleOutput_LoopOptimization_dgemm}
\end{figure}

 \clearpage
\section{LU Factorization Example (lufac.C)}

   Using the LU factorization example code shown in
figure~\ref{Tutorial:exampleInputCode_LoopOptimization_lufac}, we run the loop optimizer in
figure~\ref{Tutorial:exampleLoopProcessor} and generate the code shown in
figure~\ref{Tutorial:exampleOutput_LoopOptimization_lufac}.

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleDirectory/inputCode_LoopOptimization_lufac.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleDirectory/inputCode_LoopOptimization_lufac.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Example source code used as input to loopProcessor, show in figure~\ref{Tutorial:exampleLoopProcessor}.}
\label{Tutorial:exampleInputCode_LoopOptimization_lufac}
\end{figure}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_lufac.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_lufac.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Output of loopProcessor using input from figure~\ref{Tutorial:exampleInputCode_LoopOptimization_lufac}
    (using options: {\tt -bk1 -fs0  -splitloop -annotation}).}
\label{Tutorial:exampleOutput_LoopOptimization_lufac}
\end{figure}

 \clearpage
\section{Loop Fusion Example (tridvpk.C)}

   Using the loop fusion example code shown in
figure~\ref{Tutorial:exampleInputCode_LoopOptimization_tridvpk}, we run the loop optimizer in
figure~\ref{Tutorial:exampleLoopProcessor} and generate the code shown in
figure~\ref{Tutorial:exampleOutput_LoopOptimization_tridvpk}.

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleDirectory/inputCode_LoopOptimization_tridvpk.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleDirectory/inputCode_LoopOptimization_tridvpk.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Example source code used as input to loopProcessor, show in figure~\ref{Tutorial:exampleLoopProcessor}.}
\label{Tutorial:exampleInputCode_LoopOptimization_tridvpk}
\end{figure}

\begin{figure}[!h]
{\indent
{\mySmallFontSize

% Do this when processing latex to generate non-html (not using latex2html)
\begin{latexonly}
   \lstinputlisting{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_tridvpk.C}
\end{latexonly}

% Do this when processing latex to build html (using latex2html)
\begin{htmlonly}
   \verbatiminput{\TutorialExampleBuildDirectory/rose_inputCode_LoopOptimization_tridvpk.C}
\end{htmlonly}

% end of scope in font size
}
% End of scope in indentation
}
\caption{Output of loopProcessor input from figure~\ref{Tutorial:exampleInputCode_LoopOptimization_tridvpk}
    (using options: {\tt -fs2 -ic1 -opt 1 }).}
\label{Tutorial:exampleOutput_LoopOptimization_tridvpk}
\end{figure}


\documentclass{article}
\usepackage{afterpage}
\usepackage{float}
\usepackage{longtable}
\usepackage{graphicx}
\usepackage{pdflscape}
\usepackage[numbers,sort&compress]{natbib}
\usepackage{psfrag}

\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{graphicx}
%\usepackage{nicefrac}
\usepackage{graphicx}
\usepackage{caption}
% \usepackage{subcaption}
\usepackage{subfigure}
% \usepackage{algorithm}
% \usepackage{paralist}
% % \usepackage[geometry]{ifsym}
\usepackage{rotating}
\usepackage{setspace}
\newcommand{\uu}[1]{\boldsymbol #1}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{subfigure}
\usepackage{fullpage}
\lstset{language=C++,
                keywordstyle=\color{blue},
                stringstyle=\color{red},
                commentstyle=\color{green},
                morecomment=[l][\color{magenta}]{\#}
}
\onehalfspace
\begin{document}
\section{Convergence results}

\begin{table}[h!]
\begin{center}
\begin{tabular}{ccccccccc}
\hline \hline
$\ell$ &    Dofs $\uu{u}_h/p_h$ & $\|\uu{u}-\uu{u}_h\|_{L^2(\Omega)}$ & order & $\|\uu{u}-\uu{u}_h\|_{H^1(\Omega)}$ & order & $\|{p}-{p}_h\|_{L^2(\Omega)}$ & order \\
\hline \hline
 1 &     375/27 &  1.1189e-01 &     0.00 &  5.1107e-01 &     0.00 &  1.3055e-01 &      0.00 \\
 2 &    2187/125 &  1.9008e-02 &     3.02 &  1.7464e-01 &     1.83 &  1.5402e-02 &      4.18 \\
 3 &   14739/729 &  2.7787e-03 &     3.02 &  5.2700e-02 &     1.88 &  2.6362e-03 &      3.00 \\
 4 &  107811/4913 &  5.5815e-04 &     2.42 &  1.6718e-02 &     1.73 &  4.5891e-04 &      2.75 \\
\hline \hline
\end{tabular}
\caption{Fluid convergence}
\label{tab:FluidConvergence}
\end{center}
\end{table}

\begin{table}[h!]
\begin{center}
\begin{tabular}{cccccccccc}
\hline \hline
$\ell$ &    Dofs $\uu{b}_h/r_h$ & $\|\uu{b}-\uu{b}_h\|_{L^2(\Omega)}$ & order & $\|\uu{b}-\uu{b}_h\|_{H({\rm curl},\Omega)}$ & order & $\|\uu{r}_h\|_{L^2(\Omega)}$ \\ \hline \hline
 1 &     98/27 &  1.8167e-05 &     0.00 &  3.9061e-05 &        0.00 & 1.9044e-14 \\
 2 &    604/125 &  1.3020e-05 &     0.55 &  2.3989e-05 &        0.80 & 2.8631e-08 \\
 3 &   4184/729 &  7.6478e-06 &     0.82 &  1.2832e-05 &        0.97 & 7.7598e-10 \\
 4 &  31024/4913 &  4.0170e-06 &     0.96 &  6.4902e-06 &        1.02 & 1.3356e-09 \\
\hline \hline
\end{tabular}
\caption{Magnetic convergence}
\label{tab:MagneticConvergence}
\end{center}
\end{table}

\section{Preconditioning iterations}

\begin{table}[h!]
\begin{center}
\begin{tabular}{cccccccccc}
\hline \hline
$\ell$ &    Dofs & $it_{\rm NL}$& $it_{\rm av}$ \\
\hline \hline
  1 &   527    & 2 & 22.0 \\
  2 &  3041    & 1 & 7.0\\
  3 & 20381    & 1 & 3.0\\
  4 & 148661   & 1 & 2.0\\
\hline \hline
\end{tabular}
\caption{Magnetic convergence}
\label{tab:Its}
\end{center}
\end{table}
\section{Issues}

\begin{enumerate}
    \item From tables \ref{tab:FluidConvergence} and \ref{tab:MagneticConvergence} the convergence results seems to approach the expected rates but due to the fact I am having issues getting the larger problems (see the next items) to run I cannot check them. I have tried using a stricter nonlinear tolerance but I get exactly the same rates.
    \item One of the main problems at the moment is the calculation of the Stokes initial guess. Due to the Neumann boundary conditions, you are not able to use the mass matrix in the Stokes preconditioner in a standard fashion. Apparently (after chatting with my Dad) for Neumann problems you need to modify the mass matrix to become a ``Neumann'' mass matrix but I can't find how I would do this.... Dominik, have you heard of this before?
    \item Due to this I am limited with respect to the dimension of the matrices that I can solve. For some reason, when I use kaldi (the CS server) to try and run the next level it seems to stall. I don't really know why this happens....
    \item From my understanding, the reason why we are looking at the Hartmann flow is that it is a slightly more realistic example? However, due the exact solution the initial guess seems to solve the problem pretty well so that we only need to do a few nonlinear iterations. Dominik, from what you remember is this what you observed with your implementation of this model?
    \item The solution seems fairly simple:
    $$ \uu{u} = (u(y,z),0,0), \ \uu{b} \approx (0,1,0), \ p\approx -Gx \ \mbox{and} \ r = 0,$$
    since the magnitude of the Fourier series $$|b(y,z)| \approx \mbox{1e-6}.$$ From my experiments, table \ref{tab:Its}, we see that the preconditioned iterations are very low and maybe this is due the the solution of the problem?
    \item For all the above reasons I'm not 100\% sure that it is worth pursuing this solution further. At the moment, it seems that the 3D smooth solution is harder for the preconditioner to handle than the 3D Hartmann flow.
\end{enumerate}

\newpage

\section{Hartmann flow solution}

In this example, we consider the 3D unidirectional
flow in the rectangular duct given
by~$\Omega=(0,L)\times(-y_0,y_0)\times(-z_0,z_0)$ with~$y_0,z_0\ll
L$ under the constant transverse magnetic field $\uu{b}_D=(0,1,0)$. We take $\uu{f}=\uu{g}=\uu{0}$ and consider solutions of the
form
\begin{align*}
\uu{u}(x,y,z)& =(u(y,z), 0,0 ),&
p(x,y,z)& =-Gx+p_0(y,z),\\[0.1cm]
\uu{b}(x,y,z)& =(b(y,z),1,0),& r(x,y,z)& \equiv 0.
\end{align*}

We enforce the boundary conditions
\begin{alignat*}2
\uu{u} &=\uu{0} & \qquad &\mbox{for $y=\pm y_0$ and $z=\pm z_0$},\\[0.1cm]
(p\underline{\uu{I}}-\nu \nabla\uu{u}) \uu{n} &= p_N\uu{n} &\qquad
&\mbox{for $x=0$ and $x=L$},\\[0.1cm]
\uu{n} \times \uu{b}   &= \uu{n} \times \uu{b}_D & \qquad &\mbox{on
$\partial \Omega$},\\
r &= 0  & \qquad &\mbox{on $\partial \Omega$},
\end{alignat*}
with $p_N(x,y,z)=-Gx-\frac{\kappa {b(y,z)}^2}{2}+10$. The functions $u(y,z)$ and $b(y,z)$ are given by the Fourier
series
$$u(y,z) =  -\frac{G}{2\nu} (z^2-z_0^2) + \sum_{n=0}^\infty u_n(y)\cos(\lambda_n z) \quad \mbox{and} \quad b(y,z)=\sum_{n=0}^\infty b_n(y)\cos(\lambda_n z),$$
where
\begin{equation*}
\begin{split}
\lambda_n&=\frac{(2n+1)\pi}{2z_0},\\[0.1cm]
u_n(y)&=A_n\cosh(p_1y) + B_n\cosh(p_2y),\\[0.1cm]
b_n(y)&=\frac{\nu}{\kappa}\left(
A_n\frac{\lambda_n^2-p_1^2}{p_1}\sinh(p_1y) +
B_n\frac{\lambda_n^2-p_2^2}{p_2}\sinh(p_2y)\right),\\[0.1cm]
p_{1,2}^2&=\lambda_n^2+{\rm Ha}^2/2 \pm {\rm Ha}\sqrt{\lambda_n^2+{\rm Ha}^2/4},\\[0.1cm]
A_n&=\frac{-p_1(\lambda_n^2-p_2^2)}{\Delta_n}u_n(y_0)\sinh(p_2y_0),\\[0.1cm]
B_n&=\frac{p_2(\lambda_n^2-p_1^2)}{\Delta_n}u_n(y_0)\sinh(p_1y_0),\\[0.1cm]
\Delta_n&=p_2(\lambda_n^2-p_1^2)\sinh(p_1y_0)\cosh(p_2y_0)-p_1(\lambda_n^2-p_2^2)\sinh(p_2y_0)\cosh(p_1y_0),\\[0.1cm]
u_n(y_0)&=\frac{-2G}{\nu \lambda_n^3 z_0}\sin(\lambda_n z_0).
\end{split}
\end{equation*}
We note that $p_0(y,z)$ and~$-\frac{\kappa b(y,z)^2}{2}$ are exact up to a constant and $p(x,y,z) = p_N(x,y,z)$.

In our experiments, we use $L=10$, $y_0=2$, $z_0=1$, $\nu= \kappa =1$, $\nu_m = \mathrm{1e4}$ and $G=0.5$.

\end{document}


\documentclass[main.tex]{subfiles}
\begin{document}

\section*{Thu Oct 17 2019}

\section{The mathematical description of curved spacetime}

The metric describes the spacetime. It depends on the coordinates.

\paragraph{Euclidean metric}

In 2D, it is \(\dd{s^2}  = \dd{x^2} + \dd{y^2} \). We can express it as
%
\begin{equation}
  \dd{s^2} = \dd{x^{\mu }} \dd{x_{\mu }} = \dd{x^{\mu}} \dd{x^{\mu }} g_{\mu \nu }
\,,
\end{equation}
%
computed with \(g_{\mu \nu } = \delta_{\mu \nu }\).

\paragraph{Polar coordinates}

Let us see how this changes when we change coordinates: for example, we can go to polar coordinates:
%
\begin{equation}
    \begin{cases}
        x = r \cos(\theta ) \\
        y = r \sin(\theta )
    \end{cases}
    \qquad \qquad
    \begin{cases}
        r = \sqrt{x^2+y^2} \\
        \theta = \arctan(y/x)
    \end{cases}
\,.
\end{equation}
%

Let us compute the differentials \(\dd{x} \)   and \(\dd{y}\):
%
\begin{equation}
  \dd{x} = \dd{r} \cos(\theta ) - r \sin(\theta ) \dd{\theta }
\,,
\end{equation}
%
and
%
\begin{equation}
  \dd{y} = \dd{r} \sin(\theta ) + r \cos(\theta ) \dd{\theta }
\,.
\end{equation}
%

Plugging these into the metric and simplifying we get
%
\begin{equation}
  \dd{s^2}  = \dd{r^2} + r^2 \dd{\theta^2}
\,.
\end{equation}
%

It is not clear to see that these are equivalent.
We then want to compute scalar quantities to characterize them.

Another issue is the fact that the polar metric is singular at the origin: the metric should always have the same signature, and if it is nondegenerate (no zero eigenvalues) we should always be able to invert it: in this case however we'd also have to invert
%
\begin{equation}
  g_{\mu \nu } = \left[\begin{array}{cc}
  1 & 0 \\
  0 & r^2
  \end{array}\right]
\,
\end{equation}
%
at \(r=0\): we cannot do it! This is a \emph{coordinate singularity}. There is nothing wrong with the space; \(\mathbb{R}^{2} \) is perfectly regular at \((0,0)\), but our coordinate description fails; besides, what value of \(\theta \) should we assign to that point?

An \emph{arbitrarily large} change in \(\theta \) can result in an arbitrarily small distance travelled if we move close enough.

\paragraph{Spherical coordinates}
In \(\mathbb{R}^3 \) we have the same issue. We use:
%
\begin{equation}
  \begin{cases}
      x = \cos(\theta ) \cos(\varphi ) \\
      y = \cos(\theta ) \sin(\varphi)  \\
      z = \sin(\theta )
  \end{cases}
\,,
\end{equation}
%
and it is a simple computation as before to see that
%
\begin{equation}
  g_{\mu \nu }= \left[\begin{array}{ccc}
  1 & 0 & 0 \\
  0 & r^2 & 0 \\
  0 & 0 & r^2 \sin(\theta )^2
  \end{array}\right]
\,,
\end{equation}
%
therefore these coordinates are not defined on the \emph{whole \(z\) axis}!

\paragraph{General coordinate transformations}

Now we consider a general transformation of space, which we denote by \(x^{\prime \mu } (x^{\mu })\): we see how the metric should change in order for the spacetime distance to be invariant: we define \(g_{\mu \nu }'\) by
%
\begin{equation}
    g_{\mu \nu } \dd{x^{\mu }} \dd{x^{\nu }}
    \overset{!}{=}
    g_{\mu \nu }' \dd{x^{\prime \mu }} \dd{x^{\prime \nu }}
\,.
\end{equation}
%

This means that the metric changes as a tensor:
%
\begin{equation}
  g_{\mu \nu } \rightarrow
  g_{\mu \nu }' =
  \pdv{x^{ \alpha }}{x^{\prime \mu }}
  \pdv{x^{ \beta  }}{x^{\prime \nu  }}
  g_{\alpha \beta }
\,,
\end{equation}
%
and we can see that it transforms as a \((0,2)\) tensor since the primes are in the denominator: it transforms with the \emph{inverse} of the Jacobian matrix.

The inverse metric transforms as:
%
\begin{equation}
  g^{\mu \nu } \rightarrow
  g^{\prime \mu \nu } =
  \pdv{x^{\prime \mu  }}{x^{\alpha  }}
  \pdv{x^{\prime \nu   }}{x^{\beta }}
  g^{\alpha \beta }
\,,
\end{equation}
%
which we can check by proving that \(g^{\mu \nu }g_{\nu \rho  } = \delta^{\mu }_{\rho }\) is conserved when we transform both the metric and the inverse metric. We get:
%
\begin{equation}
    g^{\prime \mu \nu } g_{\sigma \tau  }' \delta_{\nu }^{\sigma } =
    \pdv{x^{\prime \mu  }}{x^{\alpha  }}
    \pdv{x^{\prime \nu   }}{x^{\beta }}
    g^{\alpha \beta }
    \delta_{\nu }^{\sigma }
    \pdv{x^{ \pi  }}{x^{\prime \sigma  }}
    \pdv{x^{ \lambda   }}{x^{\prime \tau  }}
    g_{\pi  \lambda  }
\,,
\end{equation}
%
which can be simplified using the relations between the partial derivatives:
%
\begin{equation}
    \pdv{x^{\prime \nu   }}{x^{\beta }}
    \delta_{\nu }^{\sigma }
    \pdv{x^{ \pi  }}{x^{\prime \sigma  }}
    = \pdv{x^{\pi }}{x^{\beta }}
    = \delta^{\pi }_{\beta }
\,,
\end{equation}
%
since we are multiplying the Jacobian with its inverse, thus we get the identity. One can make this reasoning more explicit by seeing it as an application of the chain rule: \(x^{\mu }\) is a function of \(x^{\prime \mu }\) which is a function of \(x^{\mu } \).
Plugging this in we get
%
\begin{equation}
    g^{\prime \mu \nu } g_{\sigma \tau  }' \delta_{\nu }^{\sigma } =
    \pdv{x^{\prime \mu  }}{x^{\alpha  }}
    g^{\alpha \beta }
    \delta^{\pi }_{\beta }
    \pdv{x^{ \lambda   }}{x^{\prime \tau  }}
    g_{\pi  \lambda  }
\,,
\end{equation}
%
and now we can apply our hypothesis that \(g^{\alpha \beta }g_{\beta \lambda } = \delta^{\alpha }_{\lambda }\) to contract some more indices, get one more multiplication of a Jacobian with its inverse, which we can simplify in the same way as before to finally get the identity.
This proves that the inverse metric is a \((2,0)\) tensor.

\subsection{Lengths, areas, volumes and so on}

In 4D space, if we fix \(x^{0}\) and \(x^{3}\) and we seek the shape of the differential area element --- the \emph{volume form}  on  the surface \(\qty{x^{0} = x^{3}=0}\) --- it will look like \(\dd{A} = \sqrt{g_{11}} \dd{x^{1}} \sqrt{g_{22}} \dd{x^{2}} \) if the metric is diagonal.
The square roots come from the fact that the diagonal components tell us how to measure length along a certain direction: if we  are moving along the \(x\) axis from point \(A\) to point \(B\), then
%
\begin{align}
\left. \Delta s^{1} \right\vert_{A}^{B} = \int_{A}^{B} \dd{s} = \int_{A}^{B} \sqrt{g_{11}} \dd{x^{1}} = \sqrt{g_{11} }  \left. \Delta x^{1} \right\vert_{A}^{B}
\,,
\end{align}
%
since we have the relation \(\dd{s^2} = g_{11} (\dd{x^{1}})^2\).
So, a length measured along the \(x^{1}\) direction is given by \(\sqrt{g_{11} }\) times the coordinate distance. Of course, in general we will have to integrate if the metric depends on the point.

Similarly, the 4-volume is just
%
\begin{equation}
  \dd{v}
  = \sqrt{-g_{00}g_{11}g_{22}g_{33}}
  \dd{x^{0}} \dd{x^{1}} \dd{x^{2}} \dd{x^{3}}
  = \sqrt{-g}
  \dd[4]{x}
\,,
\end{equation}
%
where \(g = \det g_{\mu \nu }\). We have shown it for a diagonal metric, but it can be shown that it holds for any general metric. The minus sign comes from the fact that our metric has signature \(-1\) (and, as we will see, this holds in general): the determinant is then negative, and to take the square root we need a positive number.

This is not put in artificially, but it is a consequence of the fact that we measure positive times for curves with positive \emph{proper} time \(\dd{\tau^2} = - \dd{s^2}\).

\begin{claim}[Unproven]
    A metric can be always diagonalized, at least locally.
\end{claim}

When the metric is diagonal, then \(\dd{v} = \sqrt{-g} \dd[4]{x} \); then this always holds, since [claim!] \(\sqrt{-g} \dd[4]{x} \) is a scalar.

Intuitively, the claim follows from the equivalence principle: we put a free-falling observer in our space. They will perceive spacetime as being flat.

Under a diffeomorphism with the determinant of the jacobian equal to \(J\) we have the following transformation law for the determinant:
%
\begin{equation}
  g' = J^{-2} g
  \qquad \implies \qquad
  \sqrt{-g'}  = J^{-1} \sqrt{-g}
\,,
\end{equation}
%
which follows from the following properties of determinants:
%
\begin{align}
  \det(ABC)= \det A \det B \det C
  \qquad \text{and} \qquad
  \det (A^{-1}) = 1 / \det A
\,,
\end{align}
%
applied to the transformation equation for the metric tensor:
%
\begin{align}
g_{\mu \nu }^{\prime } = \pdv{x^{\alpha }}{x^{\prime \mu }} \pdv{x^{\beta }}{x^{\prime \nu }} g_{\alpha \beta }
\,.
\end{align}
%

This can be expressed by saying that \(\sqrt{-g} \) is a \emph{tensor density} of weight \(-1\).

\begin{definition}
A tensor density of weight \(w\) transforms just like a tensor, except we need to multply by a factor \(J^{w}\) in the transformation, where \(J\) is the determinant of the Jacobian of the transformation.

A \((p, q)\) \emph{tensor density} of weight \(w\) is an object \(M_{\mu_1 \dots \mu_p}^{\nu_1 \dots \nu_q}\) with many components indexed by \(p+q\) indices, which transforms as:

\begin{equation}
    M_{\mu_1 \dots \mu_p}^{\nu_1 \dots \nu_q}
    \rightarrow
    J^{w}
    \tensor{\Lambda}{_{\mu_1}^{\mu'_1}}\dots
    \tensor{\Lambda}{_{\mu_p}^{\mu'_p}}
    \tensor{\Lambda}{^{\nu_1}_{\nu'_1}}\dots
    \tensor{\Lambda}{^{\nu_q}_{\nu'_q}}
    M_{\mu_1' \dots \mu_p'}^{\nu_1' \dots \nu_q'}
\end{equation}
%
under diffeomorphisms with Jacobian matrix \(\tensor{\Lambda}{_\mu^\nu}\), with determinant \(J\).
\end{definition}

When we transform the 4-volume element \(\dd[4]{x'} = J \dd[4]{x} \), we get a Jacobian: the coordinate volume element \(\dd[4]{x} \) is a \emph{tensor density of weight 1}.

Then we see that the volume element \(\sqrt{-g} \dd[4]{x} \) is a \emph{scalar}, since when multiplying tensor densities their weights are added (or, more simply put, the \(J\)s cancel out).

\subsection{Vectors in curved spacetime}

They are not objects \emph{in spacetime}: instead they belong to the \emph{tangent space}.

For each point \(x\) in the manifold we define a basis at that point: \(\qty{e^{\mu }_{(\alpha )}}(x)\), where \(\mu \) is a vector index while \((\alpha )\) denotes which vector we are considering.

Any vector \(a^{\mu }(x)\) can be then decomposed as
%
\begin{equation}
  a^{\mu }(x) = a^{(\alpha) }(x) e^{\mu }_{(\alpha )}(x)
\,.
\end{equation}
%

Since spacetime is not flat, the dependence on \(x\) is not trivial.

The scalar product between the vectors can be expressed with respect to the basis:
%
\begin{equation}
  a(x) \cdot b(x) = a^{(\alpha )} b^{(\beta )} e_{(\alpha )}\cdot e_{(\beta )}
\,,
\end{equation}
%
and we can select our basis so that at every point it is orthonormal:
%
\begin{equation}
    e_{(\alpha )}\cdot e_{(\beta )} = \eta_{(\alpha ) (\beta )}
\,.
\end{equation}
%

We \emph{cannot sum vectors in different tangent spaces}.
But we want to: we need to define derivatives!
To solve this issue, we will introduce the notion of \emph{parallel transport}.

\end{document}


\documentclass[11pt,a4paper]{article}
\usepackage{isabelle,isabellesym}
\usepackage{amsmath}
\usepackage{amssymb}

% this should be the last package used
\usepackage{pdfsetup}

% urls in roman style, theory text in math-similar italics
\urlstyle{rm}
\isabellestyle{it}

\begin{document}

\title{The Budan-Fourier Theorem and Counting Real Roots with Multiplicity}
\author{Wenda Li}
\maketitle

\begin{abstract}
	This entry is mainly about counting and approximating real roots (of a polynomial) with multiplicity.
	We have first formalised the Budan-Fourier theorem: given a polynomial with real coefficients, we can calculate sign variations on Fourier sequences to over-approximate the number of real roots (counting multiplicity) within an interval. When all roots are known to be real, the over-approximation becomes tight: we can utilise this theorem to count real roots exactly. It is also worth noting that Descartes' rule of sign is a direct consequence of the Budan-Fourier theorem, and has been included in this entry. In addition, we have extended previous formalised Sturm's theorem to count real roots with multiplicity, while the original Sturm's theorem only counts distinct real roots. Compared to the Budan-Fourier theorem, our extended Sturm's theorem always counts roots exactly but may suffer from greater computational cost.
\end{abstract}

Many problems in real algebraic geometry is about counting or approximating roots of a polynomial. Previous formalised results are mainly about counting distinct real roots (i.e. Sturm's theorem in Isabelle/HOL \cite{Sturm_Tarski-AFP,Sturm_Sequences-AFP}, HOL Light \cite{harrison-poly}, PVS \cite{Narkawicz:2015do} and Coq \cite{Mahboubi:2012gg}) and limited support for multiple real roots (i.e. Descartes' rule of signs in Isabelle/HOL \cite{Descartes_Sign_Rule-AFP}, HOL Light and ProofPower\footnote{According to Freek Wiedijk's "Formalising 100 Theorems" (\url{http://www.cs.ru.nl/~freek/100/index.html})}). In comparison, this entry provides more comprehensive support for reasoning about multiple real roots.

The main motivation of this entry is to cope with the roots-on-the-border issue when counting complex roots \cite{li_evaluate_cauchy,Count_Complex_Roots-AFP}, but the results here should be beneficial to other developments.

Our proof of the Budan-Fourier theorem mainly follows Theorem 2.35 in the book by Basu et al. \cite{Basu:2006bo} and that of the extended Sturm's theorem is inspired by Theorem 10.5.6 in Rahman and Schmeisser's book \cite{Rahman:2016us}.

%\tableofcontents

% include generated text of all theories
\input{session}

\section{Acknowledgements}
The work was supported by the ERC Advanced Grant ALEXANDRIA (Project 742178), funded by the European Research Council
and led by Professor Lawrence Paulson at the University of Cambridge, UK.

\bibliographystyle{abbrv}
\bibliography{root}

\end{document}


\section{Technical Approach}
\label{s:approach}
%
In this section, we describe our technical approach in detail.

\subsection{A syntax-directed reduction}
\label{s:reduction}
%
\sys, given $P \in \rtlc$ in $\tau \in \types$, attempts to determine
$P \hastype \tau$ by generating a CHC system $\mathcal{C}$ such that
each valid typing of the sub-expressions of $P$ that proves $P
\hastype \tau$ corresponds to a solution of $\mathcal{C}$.
%
$\mathcal{C}$ is generated by a syntax-directed translation over $P$.
%
For the rest of this section, $P$ and $\tau$ are fixed.

% define the space of relational predicates:
\subsubsection{Relational Predicates}
\label{s:rel-preds}
The relational predicates of $\mathcal{C}$ are defined as follows.
%
For each $e \in \subexps(P)$, there is a collection of relational
predicates that correspond to required types of all free variables of
$e$ and the guaranteed type of $e$ itself.
%
The relational predicates that model correct typings of $e$ are as
follows.
%
Let $\freevars(e)' = \add{\freevars(e)}{\nu}$.
%
For each $x \in \freevars(e)'$ and $\tau \in \basetypes$,
%
\BH{define}
%
let the relational predicates of $\tau$ in $e$, denoted
$\relpredsof{e}{x}{\tau}$, be defined inductively as the structure of
$\tau$:
% define relational predicates by case splitting on \tau:
\begin{itemize}
\item
  % case: type is Bool:
  If $\tau$ is $\setformer{\nu : \bools}{ \true }$, then
  $\typerelpredsof{ e }{ x }{ \tau } = \elts{ \relpredof{ e }{ x }{
      \bools } }$.
\item
  % case: type is Int:
  If $\tau$ is $\setformer{\nu : \ints}{ \true }$, then
  $\typerelpredsof{ e }{ x }{ \tau } = \elts{ \relpredof{ e }{ x }{
      \ints } }$.
\item
  % case: type is type var:
  If $\tau$ is $X \in \typevars$, then $\typerelpredsof{ e }{ x }{
    \tau } = \elts{ \relpredof{ e }{ x }{ X } }$.
\item
  % case: type is function type:
  If $\tau$ is $\tau_0 \to \tau_1$ for $\tau_0, \tau_1 \in
  \basetypes$, then $\typerelpredsof{ e }{ x }{ \tau } =
  \setformer{ \paramrelcons(R) }{ R \in \typerelpredsof{ e }{ x }{
      \tau_0 } } %
  \union \setformer{ \resrelcons(R) }{ R \in
    \typerelpredsof{e}{x}{\tau_1} }$.
  %
  \BH{add relational symbol for the top-level function}
\item
  % case: type is recursive type:
  If $\tau$ is $\mu X. C_0(y_0^0 : \tau_0^0, \ldots, y_0^{|C_0|} :
  \tau_0^{|C_0|}, \ldots, C_n(y_n^0 : \tau_n^{|C_n|})$, then
  $\relpredsof{ e }{ x }{ \tau } = $
  %
  \BH{add a relational symbol for each component of each
    constructor}
\end{itemize}
%
Then $\symrelpreds{ e }{ x } = \typerelpredsof{ e }{ x }{ \basetype{ e
  }{ x } }$.
%
\BH{define arguments of each relational predicate}

% define the constraint clauses generated:
\subsubsection{Constraint Clauses}
\label{s:clauses}
The clauses of $\mathcal{C}$ are defined as follows.

%
\BH{define subtype constraint over symbol}

%
\subsection{Liquid Type Synthesis}
\label{s:liquid-type-syn}