\section{2K Analysis (90 pts)\label{sec:6}}

    In this section a 2k3 analysis (2k analysis with 3 repetitions) is performed on throughput and response time for
    experimental parameters listed in table \ref{tab:6_setup} using single keys for GET requests. In a 2k analysis
    factors are used as parameters to infer system behaviour. Table \ref{tab:6_2k-factors} lists these factors and their
    interpretation as per sign table (this approach was chosen for the 2k analysis). It is important to note that for a
    $2^k r$ experiment with $k = 3$ (3 factors observed) and $r = 3$ (three repetitions) the data gets appropriately
    normalized for the calculation of the q-values (multiplication by $\tfrac{1}{2^3}$) and the sum of squares (SSQ) is
    multiplied by $2^3 * 3$ to be able to infer the variation explained.

    \begin{table}
        \scriptsize{
            \begin{tabular}{|l|c|}
                \hline Number of servers                & 1 and 3 \\
                \hline Number of client machines        & 3 \\
                \hline Instances of memtier per machine & 1 (1 middleware) or 2 (2 middlewares) \\
                \hline Threads per memtier instance     & 2 (1 middleware) or 1 (2 middlewares) \\
                \hline Virtual clients per thread       & 32 \\
                \hline Workload                         & Write-only and Read-only \\
                \hline Multi-Get behavior               & N/A \\
                \hline Multi-Get size                   & N/A \\
                \hline Number of middlewares            & 1 and 2 \\
                \hline Worker threads per middleware    & 8 and 32 \\
                \hline Repetitions                      & 3 or more (at least 1 minute each) \\
                \hline
            \end{tabular}
        \caption{Experimental parameters for 2k analysis.\label{tab:6_setup}}
        }
    \end{table}

    \begin{table}
        \small{
            \begin{tabular}{r l l l }
                \toprule
                Sign & Number of \srv{}s (\textbf{A}) & Number of \mw{}s (\textbf{B}) & Number of worker threads (\textbf{C})   \\
                \midrule
                -1  & 1                            & 1                           & 8   \\
                1   & 3                            & 2                           & 32  \\
                \bottomrule
            \end{tabular}
            \caption{Sign table interpretation with respective configuration value.\label{tab:6_2k-factors}}
        }
    \end{table}

    For 2k analysis two models exist, the additive and multiplicative one. Considering the factors to analyse, it
    becomes clear that increasing \textbf{A} and either \textbf{B} or \textbf{C}, an additive relationship exists
    whereas increasing \textbf{B} also modifies \textbf{C} (but not necessarily vice-versa\textemdash a
    quasi-multiplicative relationship). A mixed model would therefore be optimal under these considerations. Looking at
    the expected data and including the strength of the multiplicative model for expected values $y$ to exist only for
    a large enough threshold of $\tfrac{y_{max}}{y_{min}}$ it is reasonable to evaluate the system with an additive
    model.

    \begin{table}
          \def\sym#1{\ifmmode^{#1}\else\(^{#1}\)\fi}%
        \footnotesize{
            \centering
            \begin{subfigure}[t!]{0.45\textwidth}
                \centering
                \begin{tabular}{l*{4}{c}}
                    \toprule
                    & \multicolumn{2}{c}{Throughput}  & \multicolumn{2}{c}{Response Time} \\
                    \cmidrule(lr){2-3}\cmidrule(lr){4-5}
                    & \multicolumn{1}{c}{Effect} & \multicolumn{1}{c}{Variation} &
                      \multicolumn{1}{c}{Effect} & \multicolumn{1}{c}{Variation} \\
                    \midrule
                    q0            & 5695 & \textemdash & 44.1  & \textemdash \\
                    \addlinespace
                    qA            & 2760 & 98.24\%     & -21.3 & 99.73\% \\
                    qB            & 173  & 0.38\%      & -0.5  & 0.05\% \\
                    qC            & 138  & 0.25\%      & -0.4  & 0.04\% \\
                    \addlinespace
                    qAB           & 175  & 0.40\%      & -0.5  & 0.06\% \\
                    qAC           & 137  & 0.24\%      & -0.4  & 0.04\% \\
                    qBC           & -136 & 0.24\%      & 0.4  & 0.04\% \\
                    \addlinespace
                    qABC          & -136 & 0.24\%      & 0.4   & 0.03\% \\
                    \addlinespace
                    Error         & \textemdash & 0.01\% & \textemdash & 0.00\% \\
                    \bottomrule
                \end{tabular}
                \caption{2k3 factors with variation for GET requests.\label{tab:6_get-factors}}
            \end{subfigure}
            \hspace{4em}
            \begin{subfigure}[t!]{0.45\textwidth}
                \centering
                \begin{tabular}{l*{4}{c}}
                    \toprule
                    & \multicolumn{2}{c}{Throughput}  & \multicolumn{2}{c}{Response Time} \\
                    \cmidrule(lr){2-3}\cmidrule(lr){4-5}
                    & \multicolumn{1}{c}{Effect} & \multicolumn{1}{c}{Variation} &
                      \multicolumn{1}{c}{Effect} & \multicolumn{1}{c}{Variation} \\
                    \midrule
                    q0            & 7775 & \textemdash & 26.4 & \textemdash \\
                    \addlinespace
                    qA            & -574 & 7.87\%      & 2.1  & 9.55\% \\
                    qB            & 1149 & 31.56\%     & -3.9 & 32.81\%\\
                    qC            & 1530 & 55.93\%     & -5.0 & 54.85\% \\
                    \addlinespace
                    qAB           & 171  & 0.70\%      & -1.0 & 2.41\% \\
                    qAC           & -177 & 0.75\%      & -0.0 & 0.00\% \\
                    qBC           & 341  & 2.77\%      & 0.3  & 0.17\% \\
                    \addlinespace
                    qABC          & 115  & 0.32\%      & -0.2 & 0.11\% \\
                    \addlinespace
                    Error         & \textemdash & 0.09\% & \textemdash & 0.10\% \\
                    \bottomrule
                \end{tabular}
                \caption{2k3 factors with variation for SET requests.\label{tab:6_set-factors}}
            \end{subfigure}
        \caption{2k3 factor analysis summaries for GET and SET requests. Numbers are rounded for throughput to integers,
                 for latency to a single decimal.\label{tab:6_factor-analysis}}
        \vspace*{-0.75\baselineskip}
        }
    \end{table}

    In the case of GET packets the factor \textbf{A} is with nearly 100\% variational effect for throughput and
    response time the only relevant factor in determining the system behaviour. Of interesting note is the switch in
    sign yet including the interpretation that high throughput implies low response times it becomes apparent why a
    change in sign must happen for corresponding factors. As previously observed we see for three \cli{}s and one
    \srv{} immediately a saturation point and the middleware has no measurable effect, either good or bad on the
    system performance for GET. This matches the results observed. Visualizing the residuals (Figure
    \ref{fig:6_r_get_tp}) and doing a QQ plot (figure \ref{fig:6_qq_get_tp}) of the measured data for the
    throughput and response time (the latter omitted for behaving similar to the former) shows very consistent
    behaviour at either ends of the request throughputs on the residual plot. A few outliers are observed but these
    cannot be ruled out in a cloud environment. The QQ plot has a very shallow line in the center where the highest
    likelihood of an observation is for a standard distribution. Both of these plots show an insufficient model
    according to the book referenced in the lecture (Box 18.1) and a multiplicative model is recommended to be observed.
    After evaluating that model no reasonable gains were able to be found and the resulting plots are apart from a change
    of axis labels very similar. Even though the results indicate problems, when including the real system behaviour,
    both numbers make sense and are not only verified by the 2k3 analysis but also empirically to correlate.

    In the case of SET requests a mixture of factors \textbf{C} and \textbf{B} shows to be mostly significant with
    around 55.9\%/54.9\% and 31.6\%/32.8\% for throughput and response time respectively. Of minor significance yet
    measurable are \textbf{A} with around 7.8\%/9.6\% for throughput and response time and \textbf{BC} for
    throughput with 2.8\% (which interprets as increasing the number of \mw{}s and worker threads simultaneously).
    The change in sign occurs again correctly. The analysis aligns with previous observations and conclusions while
    bringing up the hypothesis that worker threads result in a larger gain in performance compared to adding more
    \mw{}s. The hypothesis seems to not hold though as in Experiment 3 we expect for the case of two \mw{}s and 16
    worker threads less performance than for one \mw{} and 32 worker threads. This is clearly not the case and as
    such the belief is the model is unable to correctly differentiate between these two parameters (assuming both
    \mw{}s have the exact same throughput and response time behaviour). The factor \textbf{A}, adding more \srv{}s to
    the system, does align with results from experiments \ref{sec:3} and \ref{sec:4} where the number of \srv{}s changes
    from one to three and a decrease in throughput is observed. The factor proposes an increase in \srv{}s introduces a
    negative effect as the \mw{} is designed to share the SET operation with each \srv{} in the system. This key
    distribution introduces additional latencies into the system. Again residual (figure \ref{fig:6_r_set_tp}) and QQ
    plots (figure \ref{fig:6_qq_set_tp}) were created but only the throughput ones are presented (similar behaviour in
    either case). The residuals are scattered without anything being possible to interpret and the QQ plot shows a
    quasi-linear behaviour. Both great signs according to the book as the residual error must be IID and small (it is
    smaller by a magnitude than the x-labels) and the distribution of errors being uniform at random. As such the model
    should be good enough to describe system performance per the book. Given empirical data the trend is clearly visible
    and overlaps but the strength of individual components is questioned to be correct.

    \begin{figure*}
        \vspace*{-.5\baselineskip}
            \centering
        \begin{subfigure}[t!]{0.45\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../data_analysis/figures/6-0_throughput-set-residual.png}
            \caption{Residual plot for SET Throughput.\label{fig:6_r_set_tp}}
            \includegraphics[width=\textwidth]{../data_analysis/figures/6-0_throughput-get-residual.png}
            \caption{Residual plot for GET Throughput.\label{fig:6_r_get_tp}}
        \end{subfigure}
        \begin{subfigure}[t!]{0.45\textwidth}
            \centering
            \includegraphics[width=\textwidth]{../data_analysis/figures/6-0_throughput-set-qq.png}
            \caption{QQ-Plot for SET Throughput.\label{fig:6_qq_set_tp}}
            \includegraphics[width=\textwidth]{../data_analysis/figures/6-0_throughput-get-qq.png}
            \caption{QQ-Plot for GET Throughput.\label{fig:6_qq_get_tp}}
        \end{subfigure}
        \caption{Residual and QQ plots based on memtier measurements for experiment 6.0. Only throughput plots
                 are depicted.\label{fig:6_tp}}
    \end{figure*}


% !TEX root = ../ms.tex

\defmath\B{\mathbb B}

\vspace{-1em}
\section{Introduction~\label{sec:introduction}}

%The simulation of quantum computing circuits is important for studying noise, synthesizing and optimizing circuits and

Classical simulation of quantum computing is useful for circuit design~\cite{zulehner2017one} and studying noise resilience in the era of Noisy Intermediate-Scale Quantum (NISQ) computers~\cite{preskill2018quantum}.
Moreover, identifying classes of quantum circuits that are classically simulatable, helps in excluding regions where a quantum computational advantage cannot be obtained.
For example, circuits containing only Clifford gates (a non-universal quantum gate set), using an all-zero initial state, only compute the so-called `stabilizer states' and can be simulated in polynomial time
\cite{gottesman1998heisenberg,aaronson2008improved,gottesman1997stabilizer}.
%Stabilizer states arise when computing using only Clifford gates (a non-universal quantum gate set), and
Stabilizer states, and associated formalisms for expressing them, are fundamental to many quantum error correcting codes~\cite{gottesman1997stabilizer} and play a role in measurement-based quantum computation~\cite{raussendorf2001oneway}.
% circuit limited to the Clifford gate set stay in a tractable region of the so-called stabilizer states , which forms the basis for measurement-based quantum computation~\cite{raussendorf2001oneway} and many quantum error correction codes.
In fact, simulation of general quantum circuits is fixed-parameter tractable in the number of non-Clifford gates~\cite{bravyi2016trading}, a principle on which many modern simulators are based~\cite{bravyi2016trading,bravyi2017improved,bravyi2019simulation, huang2019approximate,kocia2018stationary,kocia2020improved}.

\begin{wrapfigure}{R}{.35\textwidth}\vspace{-1.5em}
	\includegraphics[width=.24\textwidth]{pics/venn-diagram.pdf}
    \centering
    \vspace{-1em}
	\caption{
        The set of stabilizer states and states represented as:
        poly-sized \limdds and QMDDs.
        \vspace{-\baselineskip}
	}%\vspace{-.5em}
    \label{fig:venn-diagram}
\end{wrapfigure}

Another method for simulating universal quantum computation is based on (algebraic) decision diagrams (DDs)~\cite{akers1978binary,bryant86,580054,bryant1995verification,sanner,10.1145/157485.164569,fujita1997multi,viamontes2003improving,viamontes2004high,miller2006qmdd,zulehner2018advanced}.
A DD is a directed acyclic graph (DAG) in which each path represents a quantum amplitude, enabling the succinct representation of many quantum states through the combinatorial nature of these paths.
Various manipulation operations for DDs exist which implement any quantum gate operation in polynomial time
in the size of the DD. Together with other DD operations that can be used for measurement,
strong simulation is easily implemented using a DD data structure~\cite{miller2006qmdd,zulehner2018advanced}.
Indeed, DD-based simulation was empirically shown to be competitive with state-of-the-art simulators~\cite{viamontes2004high,zulehner2018advanced} and is used in several simulator implementations~\cite{viamontes2009quantum}.
%\todo{Vedran: at-the time? I mean, simulators have advanced a lot since 2018... As I mentioned they simulated shor on 60 qubits. good luck with that in any other method.}.
DDs and the stabilizer formalism are introduced in \autoref{sec:preliminaries}.
%However, in contrast to the stabilizer formalism, little is known about which quantum circuits are efficiently simulatable with DD-based approaches.

In this paper, we show that certain stabilizer states, called cluster states \cite{briegel2000persistent}, yield exponentially large \qmdds, the currently most succinct version of DDs
(see \autoref{sec:exponential-separations}).
In order to unite the strengths of DDs and the stabilizer formalism,
in \autoref{sec:isomorphism-qmdd}, we propose \limdd: a new DD for quantum computing simulation using local invertible maps (LIMs).
Specifically, \limdds eliminate the need to store multiple states which are equivalent up to LIMs, allowing more succinct DD representations.
%\todo{Tim: please check, it might sound a bit like \limdds only can encode stabilizer states now}
We prove that the set of quantum states that can be \emph{represented} by poly-sized \limdds
is larger than those that can be expressed in either the stabilizer formalism or a poly-sized \qmdd.
\autoref{fig:venn-diagram} shows the resulting separation.
In \autoref{sec:quantum-simulation}, we give procedures for analyzing and simulating quantum
circuits using \limdds and conclude in \autoref{sec:discussion} with evidence that \limdd-based simulation can be powerful than modern techniques using low-rank stabilizer decomposition~\cite{bravyi2019simulation}.

The workhorse behind \limdds is a novel algorithm which merges two DD nodes when they are `isomorphic:'
Two quantum states $\ket{\phi}$ and $\ket{\psi}$ are isomorphic when there is a series of Pauli operators $P_j$ and a complex nonzero number $\lambda$ such that $\ket{\phi}=\lambda P_n\otimes\cdots\otimes P_1\ket \psi$.
There is a plethora of work on investigating the effect of similar local operations in the context of stabilizer states \cite{nest2005local, englbrecht2020symmetries}; we emphasize that here we consider arbitrary quantum states $\ket{\phi}, \ket{\psi}$.%\todo{Tim: should rephrase, now it looks like no-one ever thought of considering locally-equivalent states beyond stabilizer states...}
To find such an isomorphism, we compute (generators of) the stabilizer (sub)group $\Stab(\ket \psi)$ of the state $\ket \psi$ represented by each DD node, and then we exploit the fact that the set of all such isomorphisms can be expressed as the coset $\pi\cdot \Stab(\ket \psi)$ for some isomorphism $\pi$.
To make the diagram canonical ---an important property for realizing efficient manipulation operations~\cite{darwiche2002knowledge}---
our algorithm then chooses a ``lexicographically smallest'' element from a Pauli coset.

%\todo[inline]{Vedran: general comment: i am a bit worried that nothing about efficiency of algorithms for all the manipulations is said for such a long time... I lose the connection to simulation of circuits... The Introduction for me is missing a link and clarification of the relationship between representation and simulation."}

%\todo[inline]{Give another / better example of polytime simulatable QC; introduce stabilizer states before they are mentioned.}

%\todo[inline]{Process Vedran's feedback, so write something about Clifford gates that's actually true.}

%This work is organized as follows.
%After providing the necessary background in \autoref{sec:preliminaries}, we formally introduce \limdds in \autoref{sec:isomorphism-qmdd}.
%In \autoref{sec:exponential-separations}, we prove the exponential separation between QMDDs and \limdds, by showing that there is a sequence of $n$-qubit stabilizer states that can only be represented with exponentially-sized QMDDs.


\documentclass[pdf]{beamer}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{listings}
\usepackage{tcolorbox}
\usepackage[all]{xy}

\mode<presentation>{}

% ------------------------------------------------------------------------------
% Theme

\usetheme[usetitleprogressbar, nosmallcapitals, usetotalslideindicator]{m}

\lstloadlanguages{Haskell}
\lstnewenvironment{code}
    {\lstset{}%
      \csname lst@SetFirstLabel\endcsname}
    {\csname lst@SaveFirstLabel\endcsname}
    \lstset{
      basicstyle=\ttfamily\footnotesize,
      flexiblecolumns=false,
      basewidth={0.5em,0.45em},
      literate={+}{{$+$}}1 {/}{{$/$}}1 {*}{{$*$}}1
               {\\\\}{{\char`\\\char`\\}}1
               {=>}{{$\Rightarrow$}}2
               {forall}{{$\forall$}}2
               {->}{{$\rightarrow$}}2
               {<-}{{$\leftarrow$}}2
               {>>}{{>>}}3 {>>=}{{>>=}}3,
      commentstyle={\ttfamily\color{gray}},
      language=haskell
    }

% ------------------------------------------------------------------------------
% Presentation

\title{Give me freedom!}
\subtitle{Or let me forget}
\date{\today}
\author{Joseph Tel Abrahamson}

\renewcommand{\to}{\ensuremath{\rightarrow}}
\DeclareMathOperator{\Free}{\texttt{Free}}
\DeclareMathOperator{\Forget}{\texttt{Forget}}
\DeclareMathOperator{\Monad}{\texttt{Monad}}
\DeclareMathOperator{\Functor}{\texttt{Functor}}
\DeclareMathOperator{\ty}{\texttt{ :: }}

\begin{document}

\maketitle

\begin{frame}
  \frametitle{Synopsis}
  \begin{itemize}
  \item Freedom: a \textit{noun}, no an \textit{adjective}, no a \textit{verb}!
  \item Knowing about Freedom by Forgetting what we know
  \item Putting Forgetfulness to work
  \end{itemize}
\end{frame}

\section{\texttt{Free} is a noun}

\begin{frame}[fragile]
  \frametitle{The \texttt{Free} monad}
\begin{lstlisting}
data Free f a
  = Return a
  | Free (f (Free f a))
\end{lstlisting}
\end{frame}

\begin{frame}
  \frametitle{Construction and destruction}

  \begin{itemize}
  \item<1> When \texttt{f} is a \texttt{Functor}, \texttt{Free f} is a
    \texttt{Monad}
  \item<2> For any value \texttt{x :: f a} we have \texttt{lift x :: Free f a}
  \item<3> For any
    \begin{enumerate}
    \item \texttt{Monad} \texttt{m} and
    \item \textit{interpretation} of \texttt{f} into \texttt{m}, \texttt{phi ::
        forall x. f x -> m x},
    \end{enumerate}
    we have \texttt{fold phi :: forall a . Free f a -> m a}.
  \end{itemize}
\end{frame}

\begin{frame}[fragile]
  \frametitle{e.g.}
\begin{lstlisting}
lift :: Functor f => f a -> Free f a
fold :: Monad m => (forall x . f x -> m x) -> (forall x . Free f x -> m x)
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Very nice embedded DSLs... for \textit{less}!}
\begin{lstlisting}
data TeletypeF a
  = PutStrLn String a
  | GetLine (String -> a)
    deriving ( Functor )

type Teletype = Free TeletypeF

putStrLnTT :: String -> Teletype ()
putStrLnTT line = lift (PutStrLn line ())

getLineTT :: Teletype String
getLineTT = lift (GetLine id)
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Very nice embedded DSLs... for \textit{less}!}
\begin{lstlisting}
echoTT :: Teletype ()
echoTT = forever $ do
  line <- getLineTT
  putStrLineTT line
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Very nice embedded DSLs... for \textit{less}!}
\begin{lstlisting}
interp :: TeletypeF a -> IO a
interp x = case x of
  PutStrLn line a -> putStrLn line >> return a
  GetLine next -> do
    line <- getLine
    return (next line)

echoIO :: IO ()
echoIO = fold interp echoTT
\end{lstlisting}
\end{frame}

\section{Free is an adjective}

\begin{frame}
  \begin{center}
    \texttt{Free}
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Other Freedoms we have}
  \begin{enumerate}
  \pause\item Free \texttt{Monad}s
  \pause\item Free \texttt{MonadPlus}es
  \pause\item Free \texttt{Applicative}s
  \pause\item Free \texttt{Alternative}s
  \pause\item Free \texttt{Monoid}s (``lists'')
  \pause\item Free \texttt{Category}s
  \pause\item ...
  \end{enumerate}
\end{frame}

\begin{frame}
  \frametitle{What does it mean to be free?}
  \pause

  \textit{This is the wrong question!}
\end{frame}

\section{Free is a verb}

\begin{frame}[fragile]
  \frametitle{What is \texttt{Free}, really?}

  \uncover<3->{But really more like...}

  \begin{align*}
    & \action<+->{\mathtt{>}\ \mathtt{:kind}\ \Free_{\Monad}} \\
    & \action<+->{
        \Free_{\Monad} \ty
          (\star \to \star)_{\uncover<3->{\Functor}} \to
          (\star \to \star)_{\uncover<3->{\Monad}}
      }
  \end{align*}

\pause

\begin{tcolorbox}[boxrule=0pt, arc=0pt, outer arc=0pt]
\begin{lstlisting}
-- remember...
instance Functor f => Monad (Free f)
\end{lstlisting}
\end{tcolorbox}
\end{frame}

\begin{frame}[fragile]
  \frametitle{Freedom is a process}

\begin{lstlisting}
data IsAFunctor f
  = IsAFunctor
    { fmap :: forall a b . (a -> b) -> (f a -> f b)
    }

data IsAMonad f
  = IsAMonad
    { return :: forall a   . a -> f a
    , bind   :: forall a b . (a -> f b) -> (f a -> f b)
    }

free :: IsAFunctor f -> IsAMonad (Free f)
\end{lstlisting}
\end{frame}

\begin{frame}
  \frametitle{Other ``Freedoms'' we have}
  \begin{enumerate}
  \item Free \texttt{Monad}s
  \item Free \texttt{MonadPlus}es
  \item Free \texttt{Applicative}s
  \item Free \texttt{Alternative}s
  \item Free \texttt{Monoid}s
  \item Free \texttt{Category}s
  \end{enumerate}

  \pause

  All \textit{underdefined}. Freedom goes from a source to a target!
\end{frame}

\begin{frame}
  \frametitle{A picture of ``Free monads"}
  \begin{description}
  \item[$\Free_{\Monad}$]
    \begin{displaymath}
      \mathtt{Functor}\xymatrix{\bullet \ar[r]^{\Free} & \bullet}\mathtt{Monad}
    \end{displaymath}
  \pause
  \item[$\mathtt{List}$]
    \begin{displaymath}
      \mathtt{Hask}\xymatrix{\bullet \ar[r]^{\Free} & \bullet}\mathtt{Monoid}
    \end{displaymath}
  \pause
  \item[$\mathtt{Coyoneda}$]
    \begin{displaymath}
      \mathtt{Hask}_{(\star\to\star)} \xymatrix{\bullet \ar[r]^{\Free} & \bullet}\mathtt{Functor}
    \end{displaymath}
  \end{description}

\end{frame}

\section{Gaining Freedom and Forgetting it all}

\begin{frame}
  \begin{center}
    \begin{displaymath}
      \mathtt{Functor}
      \xymatrix{
        \bullet \ar@/^/[r]^{\Free} &
        \bullet
      }
      \mathtt{Monad}
    \end{displaymath}
  \end{center}
\end{frame}

\begin{frame}
  \begin{center}
    \begin{displaymath}
      \mathtt{Functor}
      \xymatrix{
        \bullet \ar@/^/[r]^{\Free} &
        \bullet \ar@/^/[l]^{\Forget}
      }
      \mathtt{Monad}
    \end{displaymath}
  \end{center}
\end{frame}

\begin{frame}[fragile]
  \frametitle{It's easy to forget}
  \begin{align*}
    \Forget_{\Monad} \ty
        (\star \to \star)_{{\Monad}} \to
        (\star \to \star)_{{\Functor}}
  \end{align*}
\end{frame}

\begin{frame}[fragile]
\begin{lstlisting}
type Forget f a = f a

forget :: IsAMonad f -> IsAFunctor (Forget f)
forget (IsAMonad { bind, return }) = IsAFunctor fmap where
  fmap f = bind (return . f)
\end{lstlisting}
\end{frame}

\begin{frame}
  \frametitle{Too much to ask for}
  \begin{align*}
    (\Free \circ \Forget)(M) &\neq M \\
    (\Forget \circ \Free)(F) &\neq F
  \end{align*}
\end{frame}

\begin{frame}
  \frametitle{Just right}
  If
  \begin{flalign*}
    M = \Free(F)
  \end{flalign*}
  for some \texttt{Functor} $F$, then
  \begin{flalign*}
    (\Free \circ \Forget)(M) = M
  \end{flalign*}
\end{frame}

\begin{frame}
  \frametitle{Just right}
  If
  \begin{flalign*}
    F = \Forget(M)
  \end{flalign*}
  for some \texttt{Monad} $M$, then
  \begin{flalign*}
    (\Forget \circ \Free)(F) = F
  \end{flalign*}
\end{frame}

\begin{frame}
  \frametitle{Adjunctions}
  \begin{align*}
    F \circ G \circ F &= Id \\
    G \circ F \circ G &= Id
  \end{align*}
\end{frame}

\begin{frame}
  \frametitle{Adjunctions}

\end{frame}

\end{document}

%%% Local Variables:
%%% coding: utf-8
%%% mode: latex
%%% TeX-engine: xetex
%%% End:


\section{Grand Unification and SUSY}

\begin{frame}{Gauge Couplings in the Standard Model}
\addtocounter{framenumber}{-1}
The Standard Model is described as the most general renormalizable 	field theory with gauge group
\begin{equation*}
	\mathcal{G}_{\mathrm{SM}} = \mathrm{SU}(3) \times \mathrm{SU}(2) \times \mathrm{U}(1),
\end{equation*}
with  associated gauge couplings $\alpha_3$, $\alpha_2$ and $\alpha_1$,
three generations of fermions and a scalar \cite{Hebecker2020}. \\[2em]
\begin{itemize}
\item The couplings are larger for the larger component of the gauge group, i.\,e. \begin{equation*}
	\alpha_3(m_Z) > \alpha_2(m_Z) > \alpha_1(m_Z)
	\end{equation*}
	\item Interesting observation: Values of the running couplings come close together at some high energy scale $\Lambda_{\mathrm{GUT}} \sim 10^{16}\ \mathrm{GeV}$ (cf. next slide).\\[1em]
	\item \alert{\textsc{Georgi-Glashow}} \cite{GeorgiGlashow1974}: Embed $\mathcal{G}_{\mathrm{SM}}$ in larger gauge group, i.\,e. $\operatorname{SU}(5) \implies \mathrm{GUT?}$
 \end{itemize}
 \end{frame}

\begin{frame}{One Loop Running of the SM Gauge Couplings I}
	\begin{figure}
	\centering
	\includegraphics[scale = 0.6]{figures/dgut-1}%\hspace{3em}
	\caption{Running of the (inverse) SM gauge couplings, plot inspired by \cite{Kazakov2000}.}
	\end{figure}
\end{frame}

\begin{frame}{One Loop Running of the SM Gauge Couplings II}
	\begin{itemize}
			\item In the Georgi-Glashow model, we need $-\mu^2 \sim -(100\ \mathrm{GeV})^2$ to reproduce the correct $W$ and $Z$ masses $\implies$ \alert{Gauge hierarchy problem!} \cite{PeskinSchroeder1995}\\[1em]
		\item SUSY provides way out: If SUSY breaking works such that the mass differences between the superpartners are large enough, one can reproduce the correct Higgs mass $\implies$ superpartners influence the running of the (MS)SM gauge couplings (cf. next slide)
	\end{itemize}
\begin{figure}
	\centering
	\includegraphics{figures/running_diagrams.pdf}
	\begin{itemize}
		\item In the end it remains a complicated \alert{fine tuning} task!
	\end{itemize}

	\end{figure}

\end{frame}

\begin{frame}{Comparison: SM vs. MSSM}
	\begin{figure}
	\begin{subfigure}
		\centering
	\includegraphics[scale = 0.55]{figures/dgut-1}
	\end{subfigure}
	\begin{subfigure}
		\centering
	\includegraphics[scale = 0.55]{figures/dgut-2}
	\end{subfigure}

	\caption{Running of the (inverse) gauge couplings in the SM and the MSSM, plots inspired by \cite{Kazakov2000}.}
	\end{figure}
\end{frame}


\chapter{Introduction}
\label{chap:intro}

The major goal of this line of research is to develop
high order, numerically stable fast algorithms for solving elliptic
partial differential equations using the Minimum Sobolev norm (MSN).
This will require much more work than can be completed in one dissertation.
The present work focuses on developing fast algorithms to solve interpolation
and ordinary differential equation problems using the MSN method,
which will be a stepping stone to understand the
structure of the matrices arising in 2D and 3D PDEs.
We introduce these ideas by discussing some of the problems
present in interpolation methods and how the MSN method
attempts to solve them.

\section{Lagrange Interpolation and Known Difficulties}
\label{sec:int_class_interp}

The well-known Weierstrass Approximation Theorem, which we reproduce for
completeness, says continuous functions on compact, connected intervals can
be approximated arbitrarily well by polynomials:

\begin{thm}[Weierstrass Approximation Theorem;
    Theorem 7.26 in \cite{baby_rudin}]
\label{thm:WeierstrassThm}
If $f\in C[a,b]$, then there exists a sequence
of polynomials $\braces{P_{n}}_{n=1}^{\infty}$ such that

\begin{equation}
    \lim_{n\to\infty} \norm{f-P_{n}}_{\infty,[a,b]} = 0.
\end{equation}
\end{thm}

\noindent
Here,

\begin{equation}
    \norm{g}_{\infty,[a,b]} \equiv \sup_{x\in\brackets{a,b}} \abs{g(x)}
\end{equation}

\noindent
is the supremum norm.
When the interval $\brackets{a,b}$ is understood, we may
write $\norm{\cdot}_{\infty}$ in place of $\norm{\cdot}_{\infty,[a,b]}$.
This theorem shows that the set of polynomials $\mathcal{P}$
is dense in $C[a,b]$, the space of continuous functions on $[a,b]$.
We let $\mathcal{P}_{n}$ denote all polynomials of degree at most $n$.
This gives rise to an important concept: degree of
approximation.
We also have the following theorem:

\begin{thm}[Best Uniform Approximation of Continuous Functions;
    Section 1.1 in~\cite{rivlin2003introduction}]
If $f\in C[a,b]$ and $n\in\N_{0}$, then there exists
a unique $q_{n}\in\mathcal{P}_{n}$ so that
%
\begin{equation}
    \norm{f-q_{n}}_{\infty} = \inf_{p\in\mathcal{P}_{n}}\norm{f-p}_{\infty}.
\end{equation}
%
We set
%
\begin{equation}
    E_{n}(f) \equiv \inf_{p\in\mathcal{P}_{n}}\norm{f-p}_{\infty}
\end{equation}
%
and have
%
\begin{equation}
    E_{0}(f)\ge E_{1}(f)\ge E_{2}(f) \ge \cdots \to 0.
\end{equation}
\end{thm}

There are many ways to prove the Weierstrass Approximation theorem.
In~\cite[Chapter 7]{baby_rudin}, Rudin convolves against a polynomial kernel.
This is useful theoretically but in practice, one may only
have function and derivative information at particular points.
In order to reconstruct the underlying function, we want to use
these function and derivative values to build an approximation,
frequently chosen to be a polynomial.
This is interpolation.

\textbf{Interpolation Problem:} Let $f\in C[-1,1]$ be continuous
and specify a sequence of grid points

\begin{equation}
    -1\le x_{1;n} < x_{2;n} < \cdots < x_{n;n}\le1.
\end{equation}

\noindent
for $n\in\N$.
Determine a polynomial $p_{n}$ with $\deg p_{n} = m(n)$
which satisfies the conditions

\begin{equation}
    f(x_{k;n}) = p_{n}(x_{k;n}),
\end{equation}

\noindent
and ascertain under what restrictions on $f$ and $\braces{x_{k;n}}_{k=1}^{n}$
ensure

\begin{equation}
    \norm{f-p_{n}}_{\infty} \to0,\quad n\to\infty.
\end{equation}

A popular choice is to set $m(n) = n-1$, resulting in Lagrange
interpolation.
In Fig.~\ref{fig:intro_runge_plot}, we see an example of Lagrange
interpolation on equally-spaced nodes of the Runge function
$\brackets{1+25x^{2}}^{-1}$ on $[-1,1]$.
This function is analytic; even so, in~\cite{runge1901} Runge
proved that Lagrange interpolation diverges in this case.
In particular, there are large oscillations near the boundary points.

% Plot of Runge phenomenon
\input{plots/runge_figure.tex}

In fact, much more is known about Lagrange interpolation,
and we introduce notation which will make this discussion easier.
Let

\begin{equation}
    X = \braces{x_{k;n} \mid k=1,\cdots,n; n\in\N}
\end{equation}

\noindent
be an interpolatory matrix. Then, given $f\in C[-1,1]$,
we have the following standard
definitions~\cite[Chapter 1]{interpFunctionsBook}:

\begin{samepage}
\begin{align}
    L_{n}(f,X,x) &= \sum_{k=1}^{n}\ell_{k,n}(X,x)f(x_{k;n}) \nonumber\\
    \Omega_{n}(X,x) &= \prod_{i=1}^{n}\parens{x-x_{i;n}}
        \nonumber\\
    \ell_{k,n}(X,x) &= \frac{\Omega_{n}(X,x)}{\Omega_{n}'(X,x_{k;n})(x-x_{k;n})}
        \nonumber\\
    \lambda_{n}(X,x) &= \sum_{k=1}^{n}\abs{\ell_{k,n}(X,x)} \nonumber\\
    \Lambda_{n}(X) &= \norm{\lambda_{n}(X,x)}_{\infty,[-1,1]}.
\end{align}
\end{samepage}

\noindent
Naturally, $L_{n}(f,X,x)$ is the Lagrange interpolating polynomial
of degree at most $n-1$ for the interpolation matrix $X$.
The Lebesgue constants $\Lambda_{n}(X)$ are of
critical importance, as we see

\begin{align}
    \abs{L_{n}(f,X,x) - f(x)} &\le \abs{L_{n}(f,X,x) - q_{n-1}(x)}
            + \abs{f(x) - q_{n-1}(x)} \nonumber\\
    &\le \abs{L_{n}(f-q_{n-1},X,x)} + E_{n-1}(f) \nonumber\\
    &\le \brackets{\Lambda_{n}(X)+1}E_{n-1}(f).
    \label{eq:Ln_upper_bound_err}
\end{align}

\noindent
Here, $q_{n-1}\in\mathcal{P}_{n-1}$ is the minimizer in the supremum norm,
and we note $L_{n}:\mathcal{P}_{n-1}\to\mathcal{P}_{n-1}$ is the identity map.
Because $E_{n}(f)\to0$, Lagrange interpolation converges when
$\Lambda_{n}(X)E_{n-1}(f)\to0$.

If we could find an interpolatory matrix $Y$ so that $\Lambda_{n}(Y)$ is
bounded, then $L_{n}(f,Y)\to f$ uniformly. Unfortunately, this is not
the case. In~\cite{vertesi1990optimal}, V\'{e}rtesi references Faber (1914)
as proving

\begin{equation}
    \Lambda_{n}(X) > \frac{1}{8\sqrt{\pi}}\log n
\end{equation}

\noindent
for every $X$, showing $\Lambda_{n}(X)$ is unbounded.
One popular set of interpolation nodes is the zeros
of the Chebyshev polynomials:

\begin{align}
    T &= \braces{z_{k}^{n} \mid k=1,\cdots,n; n\in\N} \nonumber\\
    z_{k}^{n} &= \cos\brackets{\frac{\pi}{n}\parens{n-k+\frac{1}{2}}}.
\end{align}

\noindent
The Chebyshev polynomials $T_{n}(x)$ are a set of orthogonal polynomials
which will be discussed in detail in Sec.~\ref{ssec:kar_cheby}.
In~\cite{Brutman1978}, it was shown

\begin{equation}
    \Lambda_{n}(T) < 8 + \frac{2}{\pi}\log n,
\end{equation}

\noindent
For this reason, we see that the interpolatory matrix $T$
is close to optimal and,
coupled with fast interpolation methods, gives reason
for its popularity. Better bounds for Lebesgue
constants can be found in~\cite{smith2006lebesgue}.

We also give bounds for equally-spaced points, setting

\begin{equation}
    E = \braces{\left. -1+2\frac{k-1}{n-1} \right| k=1,\cdots,n; n\in\N}.
\end{equation}

\noindent
In~\cite{trefethen1991two}, Trefethen and Weideman give the bounds

\begin{equation}
    \frac{2^{n-2}}{n^{2}} < \Lambda_{n}(E) < \frac{2^{n+3}}{n}
\end{equation}

\noindent
as well as referencing the asymptotic result and some of the history
of equally-spaced interpolation.
Clearly, the exponential growth of
$\Lambda_{n}(E)$ helps quantify how much worse $E$ is when
compared with $T$.

This divergence is not restricted to equally-spaced point
distributions, though.
In fact, we have the following result:

\begin{thm}[Theorem 4.3 in~\cite{interpFunctionsBook}]
For an interpolatory matrix $X\subset[-1,1]$, there exists $h\in C[-1,1]$
so that
%
\begin{equation}
    \limsup_{n\to\infty} \abs{L_{n}(h,X,x)} = \infty
\end{equation}
%
for almost every $x\in[-1,1]$.
\end{thm}

\noindent
So, there is no interpolatory matrix $Y$ so that
$\norm{L_{n}(f,Y)-f}_{\infty}\to0$ for all continuous functions $f$
and the approximation error can be arbitrarily bad.

\section{Possible Solutions to Divergence of Lagrange Interpolation}
\label{sec:poss_lagrange_sol}

Although the previous result paints a bleak picture of Lagrange interpolation,
this is true only in extreme situations.
From~\cite[Chapter 1]{rivlin2003introduction}, we have the following theorem
discussing how the degree of approximation is related to smoothness:

\begin{thm}[Jackson Inequality]
\label{thm:best_uni_err}
If $g\in C^{k}[-1,1]$ and $g^{(k)}$ is $\alpha$-H\"{o}lder
with H\"{o}lder constant $L$, then for $n>k$, we have
%
\begin{equation}
    E_{n}(g) \le \frac{c}{n^{k}}\parens{\frac{1}{n-k}}^{\alpha}
\end{equation}
%
with $c=6^{k+1}e^{k}(1+k)^{-1}L$.
\end{thm}

\noindent This theorem shows that if $g$ is merely $\alpha$-H\"{o}lder
continuous, then $\norm{L_{n}(g,T)-g}_{\infty}\to0$ by
Eq.~\eqref{eq:Ln_upper_bound_err}.
As noted above, we can have $\norm{L_{n}(f,E)-f}_{\infty}\not\to0$
even when $f$ is analytic.

We previously noted $L_{n}(f,E)$ has large oscillations in the Runge example.
Because of this, there has been interest in Hermite-Fej\'{e}r interpolation.
Given an interpolatory matrix $X$, we let $H_{n}(f,X,x)\in\mathcal{P}_{2n-1}$
so that

\begin{align}
    H_{n}(f,X,x_{k;n}) &= f(x_{k;n}) \nonumber\\
    H_{n}'(f,X,x_{k;n}) &= 0.
\end{align}

\noindent
In this case, it can be shown $\norm{H_{n}(f,T)-f}_{\infty}\to0$
as $n\to\infty$ for all $f\in C[-1,1]$~\cite[Chapter 5]{interpFunctionsBook}.
Unfortunately, this does not hold in general; in fact,
for equally-spaced nodes we have the particularly bad
result

\begin{align}
    f(x) &= x \nonumber\\
    \limsup_{n\to\infty} \abs{H_{n}(f,E,x)} &= \infty,\quad 0 <\abs{x}\le1,
\end{align}

\noindent
which is discussed in~\cite[Chapter 6]{interpFunctionsBook}.
Controlling the derivative of the interpolation polynomial
at the Chebyshev nodes appears to give sufficient control of the polynomial
in order to obtain convergence for all continuous functions.
Even so, while this gives convergence in the limit,
it is not useful in practice because we purposefully limit the
accuracy of interpolation near, but not at, interpolation nodes.

In another direction, Bernstein polynomials give up interpolation
to get overall approximation.
In fact, \cite{davis_interpolation,rivlin2003introduction}
use Bernstein polynomials to prove the Weierstrass Approximation Theorem.
The downside is that convergence to the solution is slow:

\begin{thm}[Error Estimate for Berstein polynomials;
    Theorem 1.2 in \cite{rivlin2003introduction}]
\label{thm:berstein_polynomial_error}
Suppose $g\in C[0,1]$ is $\alpha$-H\"{o}lder
with H\"{o}lder constant $L$ and $B_{n}g$ is the Berstein
polynomial of degree $n$ for $g$; then
%
\begin{equation}
    \norm{g - B_{n}g}_{\infty,[0,1]} \le
        \frac{3L}{2}\frac{1}{n^{\alpha/2}},
\end{equation}
%
and this bound in $n$ cannot be improved.
\end{thm}

\noindent
This precludes it from being of much use in practice,
especially when $f$ is smooth.

By relaxing the condition $\deg L_{n}(f,X) \le n-1$,
Erd\H{o}s was able to prove in~\cite{erdos1943some} that,
under some conditions on $X$, one could prove convergence
for all continuous functions by choosing $p_{n}$ so that
$\deg p_{n} = c(X)n$, with $c$ a constant depending only on $X$.
The extension to all matrices $X$ is shown
in~\cite[Theorem 2.7]{interpFunctionsBook}.
This is important in practice, because we can not
always choose the interpolation nodes.
It is beneficial for a method to work well independent of
node location, especially if, because of instrument specifications,
data collection location cannot be modified.
Unfortunately, these results require function values at arbitrary points,
and this is not possible in practice.

\section{Interpolation in Higher Dimensions}
\label{sec:Interp_MD}

Up to this point, we have only talked about methods for approximating
functions on $[a,b]$; even so, many problems in science and engineering
are inherently two- and three-dimensional.
A review of recent methods for multivariable polynomial interpolation
can be found in~\cite{gasca2001history,gasca2000polynomial}.
One challenge of interpolation in higher dimensions is choosing
the correct polynomial space and point distribution.
Now, the fact $\dim \mathcal{P}_{n-1}=n$ makes this easy in 1D
but in higher dimensions there does not appear to be a simple way to
choose a multivariable polynomial space of arbitrary dimension.
Naturally, this is a topic of great interest.
In~\cite{gasca2000polynomial}, some standard methods discussed include
tensor products of univariate polynomials, Gr\"{o}bner bases,
and ideal interpolation schemes.

\section{Hermite and Birkhoff Interpolation}
\label{sec:Birkhoff_MD}

Hermite or Birkhoff interpolation problems involve interpolating
function and derivative values.
Hermite interpolation consists of interpolating function and derivative
values up to a certain degree at interpolation nodes.
Birkhoff interpolation is more general, allowing any combination
of specified function and derivative values at nodes.
Hermite interpolation is well-posed and can easily be solved in 1D.
This is not the case for Birkhoff interpolation, where only
certain combinations ensure a unique
solution~\cite{karlin1972hermite,lorentz1971birkhoff}.
The problem is even more complicated in dimension 2 and larger;
see~\cite{lorentz2000multivariate,rudy} for a review of these topics.
An additional challenge in multidimensional interpolation
comes from proving error bounds and determining sufficient conditions for
convergence.

\section{Characteristics of Good Algorithms}
\label{sec:Good_Alg_Details}

This dissertation focuses on the development, implementation,
and analysis of fast MSN methods. The ideas behind the MSN method
will be discussed in the Sec.~\ref{sec:msn_intro}, but here
we discuss good qualities that numerical algorithms should have,
especially algorithms for approximation.
These are high-order convergence, low computational complexity,
and numerical stability.

Given a low-order method and a high-order method of similar computational cost,
a faster-converging method is more effective and useful.
In practice, there is always a limit to the amount of computational
resources (memory, processor speed, or bandwidth), so a high-order
method would be preferred as it would lead to less work overall.
As mentioned before, Bernstein polynomials converge to all
continuous function but do so at a slow rate. This alone does not
necessarily disqualify the algorithm, but from Thm.~\ref{thm:best_uni_err},
we know smoother functions can have better polynomial
approximations. This incentivizes developing accurate approximations
and algorithms to compute them.

While some methods may be of theoretical importance, algorithms will only
be of practical value if there are efficient methods to compute them.
The total cost should be of reasonable size, so that both the
asymptotic growth ($O(\log n)$ or $O(n^{3})$) and
the explicit cost ($10^{6}\log n$ and $\frac{2}{3}n^{3}$) are important.
Because computational resources are always limited, asymptotics
may not be as important as the prefactor hidden by Big O notation.

Finally, numerical stability is of critical importance.
Almost all algorithms are implemented on computers using
floating-point arithmetic, inevitably leading to small errors.
It is necessary for practical algorithms to be immune to these
changes; namely, small changes in inputs should lead to small changes
in outputs.
The condition number quantifies how much changes in outputs come
from changes in inputs; a standard reference for numerical stability
is~\cite{HighamASNA}.

\section{The Minimum Sobolev Norm Method}
\label{sec:msn_intro}

We previously showed Lagrange interpolation
does not work, for there can be large oscillations in the interpolating
polynomial as seen in the Runge phenomenon, while using a polynomial
of higher degree allows continuous functions to be approximated
arbitrarily well.
By combining these observations, the
Minimum Sobolev norm (MSN) method was developed: a general
method for computing approximate solutions to problems
with linear constraints.

The MSN method has been used to solve problems in
interpolation~\cite{msnInterp},
Birkhoff interpolation~\cite{msnBirkhoff}, and
partial differential equations~\cite{msnPDE},
For simplicity,
we assume we are performing approximations using algebraic polynomials,
even though theoretical work often uses trigonometric polynomials.
The main idea is this: given $N$ linear constraints and polynomials
of up to degree $M(N)$ contained in $V$, unknown coefficients $a$,
correct values $f$, and a diagonal matrix $D_{s}$ with
condition number $O(M^{s})$, the MSN solution solves the equation

\begin{equation}
    \min_{Va=f} \norm{D_{s}a}_{2}.
    \label{eq:msn_def}
\end{equation}

\noindent
We choose $D_{s}$ so that $\norm{D_{s}a}_{2}$ is a Sobolev norm.
This implies that we seek an approximation which satisfies the linear
constraints as well as having the smallest derivative norm.
Here we focus on computing the minimum 2-norm solution
because this dissertation investigates efficient numerical algorithms
for MSN equations and we explicitly compute LQ factorizations;
methods for $p$-norm minimization are discussed in~\cite{msnInterp,msnBirkhoff}.
Additionally, this description is independent of dimension and node location.
The parameter $s$ determines which derivative of the
polynomial approximation we wish to control.
Larger $s$ gives more derivative control on the approximation
but leads Eq.~\eqref{eq:msn_def} to have higher condition
numbers. Great care is required to limit the effects of these
condition numbers in order to ensure convergence to the underlying
solution~\cite{msnBirkhoff}.

The technical challenge of this method is to determine
the explicit form of $M(N)$ to ensure convergence to the solution.
The methods in~\cite{msnInterp,msnBirkhoff} involve the close
approximation of integral kernels by polynomials.
The end result is that it is sufficient to choose $M(N) = C\eta^{-1}$,
where $\eta$ is the minimum separation between between
interpolation nodes. Although this is a theoretically optimal result,
knowing from~\cite{interpFunctionsBook} that this result cannot
be improved except in the constant, it is not useful in practice
because the constants from~\cite{msnInterp,msnBirkhoff}
are difficult to explicitly compute.
In practice, we have found that choosing the $M(N) = 2\pi\eta^{-d}$
is sufficient, where $d$ is the dimension of the space.
These details, along with
implementation issues, will be discussed more in the next section.

One advantage of the MSN method is that we do not insist on forming
a square linear system.
In fact, it is necessary
to take enough columns (more than twice the number of rows)
in order to ensure a good approximation. Choosing the proper
polynomial space was a challenge mentioned in Secs.~\ref{sec:Interp_MD}
and \ref{sec:Birkhoff_MD}.

\section{MSN Interpolation Examples}
\label{sec:MSN_slow_examples}

We present some results of MSN interpolation on equally-spaced
nodes in single and double precision for 1D and 2D.
We do this to show that the difficulty of approximating
functions on equally-spaced points arises from using suboptimal methods
of interpolation rather than node location.
These and similar results were published in~\cite{msnInterp,msnBirkhoff}.

We can rewrite Eq.~\eqref{eq:msn_def} as

\begin{align}
    &\min_{VD_{s}^{-1}x=f} \norm{x}_{2} \nonumber\\
    &\quad a = D_{s}^{-1}x.
    \label{eq:msn_def_rework}
\end{align}

\noindent
In order to compute the MSN solution,
we must compute the minimum norm solution from Eq.~\eqref{eq:msn_def_rework}.
To do this, we must compute an LQ factorization of $VD_{s}^{-1}$,
where $L$ is a lower triangular matrix and $Q$ is orthogonal.
As previously mentioned, large $s$ leads to greater derivative control
but also gives $VD_{s}^{-1}$ high condition number. Because
of this, the standard pivoted LQ factorization based on QR with
Column Pivoting is insufficient. A Rank-Revealing QR factorization
based on~\cite{gu1996efficient} would be better, but an
implementation is not readily available so we use another method presented
here and described in~\cite{msnBirkhoff}; see Alg.~\ref{alg:slow_msn_lq}.

\input{algs/slow_msn_alg.tex}

The unique feature of the algorithm may be Lines 4 and 5.
Clearly, $VD_{s}^{-1}$ is badly column-scaled.
LQ factorizations can deal with poor row-scaling but not poor column-scaling.
We compute the singular value decomposition
$U\Sigma V^{*} = Q_{1}D_{s}^{-1}\Pi$ in Line 4 and
see $U^{*}Q_{1}D_{s}^{-1}\Pi \approx \Sigma V^{*}$ to machine precision.
This ensures we can accurately compute the pivoted LQ
factorization $P_{2}L_{2}Q_{2} = U^{*}Q_{1}D_{s}^{-1}\Pi$ in Line 5.
Thus, $U$ is a preconditioner for numerical stability, showing
that we can safely convert poor column-scaling to poor row-scaling.
Using Alg.~\ref{alg:slow_msn_lq}, the effective condition number of this
problem appears to be that of $V$ and not $VD_{s}^{-1}$.

Looking at the algorithm, we see two pivoted LQ factorizations
and one SVD are required.
Because we have $N$ interpolation requirements
and $cN$ columns, this gives us $O(N^{3})$ floating-point operations
and $O(N^{2})$ units of memory.
At first glance, this does not seem too bad.
If we are in dimension $d$ with $n^{d}$ tensor grid points, then $N = n^{d}$ and
we require $O(n^{3d})$ flops and $O(n^{2d})$ units of memory.
While these costs may be acceptable for $d=1$ and bearable for $d=2$,
when $d=3$ this is too great.
Parallel computation would not be
of much use here because the communication required for pivoted LQ
and the SVD would cause the entire process to be extremely slow,
although there has been recent work in reducing the communication cost
in pivoted QR factorizations~\cite{demmel2015communication}.
In order for these algorithms to be used when solving large, difficult
problems, we need to investigate other methods.
Similar costs arise when solving differential equations and
this necessitates fast, structured algorithms.
When developing fast algorithms, it is critical
that we are able to convert the poor columns scaling to poor row scaling.
The inherent structure of the linear system allows us to do this
using careful factorizations.

We present some examples of MSN interpolation and Birkhoff interpolation.
The functions we approximate are

\begin{samepage}
\begin{align}
    f(x) &= \frac{1}{1+25x^{2}} \nonumber\\
    g(x,y) &= \frac{1}{1+ 25(x^2 + y - 0.3)^{2}}
       + \frac{1}{1+ 25(x + y - 0.4)^{2}} \nonumber\\
       &\quad + \frac{1}{1+ 25(x + y^{2} - 0.5)^{2}}
       + \frac{1}{1+ 25(x^{2} + y^{2} - 0.25)^{2}}
        \nonumber\\
    h(x) &= g(x,-0.96).
    \label{eq:intro_runge_functions}
\end{align}
\end{samepage}

\noindent
Naturally, $f$ is the usual Runge function. Here, $g$ is a 2D
function with Runge functions on one line, one circle, and two
parabolas.

We remember machine precision is $2^{-23}\approx 1.2\times10^{-7}$ in single
precision and $2^{-52}\approx 2.2\times10^{-16}$ in double precision.
This is the smallest relative error that we could expect for
any nonzero result. All of the plots show results for
$\norm{f-p}_{\infty}/\norm{f}_{\infty}$ for true function $f$
and approximation $p$. The sup-norm is approximated
by sampling the function at a large number of locations and taking
the maximum.

The results for interpolating $f$ are shown in Fig.~\ref{fig:intro_msn_interp}.
For single precision, all error curves decay toward $10^{-7}$ as
we increase the number of points. The main exception is for $s=6$,
which starts to increase around 60 points. We believe this occurs
because of rounding error. This would also make sense given for
$s\in\braces{3,4,5}$, the error curves hover close to $10^{-6}$.
We see a similar results for double precision.
In this case, the beginnings of the U-shaped error curve seem
present for $s\in\braces{8,10,12}$.

In Fig.~\ref{fig:intro_msn_birkhoff_1d}, we have the results
of MSN Birkhoff interpolation in 1D for $h$.
Although the error curve for $s=2$ in single and double precision hovers around
$10^{-3}$, the other error curves decay to machine precision.
The beginning of a U-shaped error curve may be seen
for $s\in\braces{4,5}$ in single precision and $s\in\braces{10,12}$
in double precision. The errors are low, although they may be slightly
larger than those we see in regular MSN interpolation.
This could stem from the fact the condition number is inherently larger for
Birkhoff interpolation than for interpolation.

In Fig.~\ref{fig:intro_msn_birkhoff_2d},
we have results for MSN Birkhoff interpolation in 2D
for $g$ from Eq.~\eqref{eq:intro_runge_functions}.
In every case the error decreases with increasing data except
for $s=5$ with single precision. In this case, we may start
to see the beginning of the effects of roundoff error.
The challenge for 2D problems is the long time required to
run Alg.~\ref{alg:slow_msn_lq}.

\input{plots/msn_interp_examples_slow.tex}

\section{Dissertation Outline}
\label{sec:dis_outline}

As we noted above, the slow methods for solving problems using
MSN become difficult in 2D and practically impossible in 3D
due to memory requirements and flop count. With the eventual
desire to use MSN to solve 3D PDEs,
we will need to take advantage of \emph{everything} we can.
Keeping this in mind, the focus of this dissertation
will be developing fast algorithms for solving interpolation
and differential equations using the MSN method on Chebyshev nodes
and express our solution in a Chebyshev polynomial basis.
We review notation conventions and structured
matrices in Chapter~\ref{chap:K_and_R}.
In Chapter~\ref{chap:CV_Prop}, we review some of the properties
of Chebyshev-Vandermonde matrices which arise when developing
these fast algorithms.
Matrix factorizations important for interpolation problems are
discussed in Chapters~\ref{chap:CV_mat_1D_I} and \ref{chap:CV_mat_HD_I}.
Using these factorizations, we present examples of MSN approximation
in Chapter~\ref{chap:func_interp}.
Next, we present new proofs showing that our fast methods
will converge to the solution under minimal smoothness assumptions
of the underlying function in Chapter~\ref{chap:cvip_converge}.
We investigate fast algorithms for Boundary Value Problems for ODEs
in Chapter~\ref{chap:fast_ode}.

In the Chapter~\ref{chap:random}, we discuss results related
to randomized low-rank approximations, unrelated to the previous work.
Some of this was discussed in~\cite{randomHSSLBL}
but more details and examples will be shown here.

\section{Algorithms Similar to the MSN Method}
\label{sec:similar_methods}

The ideas pursued in this dissertation are similar to those
used by Chebfun~\cite{driscoll2014chebfun}, a software package
in \textsc{Matlab}~\cite{guide1998mathworks} which attempts
to have the ``feel'' of symbolic software with the speed
of numerics.
The book Approximation Theory and Approximation Practice~\cite{ATAP}
uses Chebfun to introduce the field of  Approximation Theory.
Here, we focus on investigating fast algorithms based
on values computed on Chebyshev polynomial roots.
This is similar to the fast algorithms present in Chebfun,
which computes values on the Chebyshev polynomial extrema.
The book Exploring ODEs~\cite{ExpODEs} also uses
Chebfun to introduce advanced differential equation topics.
The methods in~\cite{ExpODEs} are built on the work
from~\cite{driscoll2015rectangular,aurentz2017block,xu2015explicit}
and are incorporated into Chebfun.
Although spectral methods are well-known~\cite{boyd2001chebyshev},
\cite{aurentz2017block} \emph{adds} additional rows
to the square linear system to impose boundary or other requirements
instead of replacing rows.
Naturally, this requires increasing the degree of the approximation.
The work presented here does not force a square system, which allows
us to add a finite number of additional requirements which do not
affect the asymptotic complexity of the overall algorithm.
From~\cite[Appendix A]{ExpODEs}, it appears that Chebfun uses
standard dense linear algebra algorithms to solve its ODEs.
This is unfortunate, because the linear systems arising from ODEs
are highly structured when approximated on Chebyshev nodes.
This dissertation will show this structure and construct associated
fast algorithms.
The work here could be used to speedup the Chebfun ODE solver.


\vssub
\subsubsection{~Curvilinear grids} \label{sub:num_space_curv}
\conthead{\ws (NRL Stennis)}{W. E. Rogers, T. J. Campbell}

\noindent
As an extension to traditional ``regular'' grids, computations may be made on ``irregular''
grids within \ws\ . This makes it possible to run the model on alternate grid
projections (e.g. Lambert conformal conic), rotated grids, or
shoreline-following grids with higher resolution near shore, though the
restrictions on time step from the conditionally stable schemes still
apply. The same propagation schemes are utilized for irregular grids as for
regular grids (\para\ref{sub:num_space_trad}).

The implementation is described in detail in \cite{rep:RC09}, and summarized
here: a Jacobian is used to convert the entire domain between the normal,
curving space, and a straightened space. This conversion is performed only
within the propagation routine, rather than integrating the entire model in
straightened space. A simple, three step process is used every time the
propagation subroutine is called (i.e. every time step and every spectral
component): first, the dependent variable (wave action density) is converted
to straightened space using a Jacobian; second, the wave action density is
propagated via subroutine calls for each (of two) grid axes; third, the wave
action density is converted back to normal, curved space. The actual flux
computation is not significantly modified from its original, regular grid
form. The same process occurs, regardless of grid type (regular or irregular);
for regular grids, the Jacobian is unity.

Regarding the user interface: in {\file ww3\_grid.inp}, a string is used to
indicate the grid type. In cases where this grid string is `{\code RECT}', the
model processes input for a regular grid. In case where this grid string is
`{\code CURV}' , the model processes input for an irregular grid. [Note that
with \ws\ version 4.00, the coordinate system (i.e. degrees vs. meters) and
the closure type (e.g. global/wrapping grid) are also specified in {\file
ww3\_grid.inp} ; the switches {\code LLG} and {\code XYG} are deprecated.]

With \ws\ version 5, capability is added to run on a special type of curvilinear grid, the ``tripole grid'' using the first-order propagation scheme. In the northern hemisphere, this grid type uses two poles instead of one, and both are over land to prevent singularities in grid spacing. This type of grid is sometimes used in ocean models, e.g. \citep{art:Murr96} and \citep{art:Metz14}. No special switch is required, and the grid is read in as any other irregular grid would be, but the user must specify a closure type ({\code CSTRG}) of {\code TRPL} in {\file ww3\_grid.inp}. Specific details can be found in the documentation for {\file ww3\_grid.inp} in \para\ref{sub:ww3grid}. Propagation and gradient calculations are modified to deal with the new closure method. The {\code TRPL} closure type is compatible only with the first-order {\code PR1} propagation scheme. An attractive feature of the tripole grid is that it allows the user to run a single grid which extends all the way to the North Pole.  However, though the three poles are over land, there is still a convergence of meridians at the sea points nearest to them, meaning that the grid spacing in terms of real distances (which determines the maximum propagation time step) is still highly variable. More efficient grid spacing (meaning: with less variation of grid spacing in terms of real distances) can be achieved through the use of the multi-grid capability. Though this scheme addresses singularities in grid spacing at the pole, it does not address the singularity associated with definition of wave direction.