\chapter{Applications II:  \alphatap}\label{alphatapchapter}

In this chapter we examine a second application of nominal logic
programming, a declarative theorem prover for first-order classical
logic. We call this prover \alphatap, since it is based on the
\leantapsp\cite{beckert95leantap} prover and written in
\alphakanren. Our prover is a relation, without mode restrictions;
given a logic variable as the theorem to be proved, \alphatapsp
\textit{generates} valid theorems.

\leantapsp is a lean tableau-based theorem prover for first-order
logic due to \citet{beckert95leantap}.  Written in
Prolog, it is extremely concise and is capable of a high rate of
inference. \leantapsp uses Prolog's cut (\texttt{!}) in three of its
five clauses in order to avoid nondeterminism, and uses
\mbox{\texttt{copy\_term/2}} to make copies of universally quantified
formulas. Although Beckert and Posegga take advantage of Prolog's
unification and backtracking features, their use of the impure cut and
\mbox{\texttt{copy\_term/2}} makes \leantapsp non-declarative.

% : reordering goals within the prover may cause divergence.

%% new definition of nondeclarative?

In this chapter we translate \leantapsp from Prolog to impure
miniKanren, using \scheme|match-a| to mimic Prolog's cut, and
\scheme|copy-termo| to mimic \mbox{\texttt{copy\_term/2}}.  We then show how
to eliminate these impure operators from our translation. To eliminate the
use of \scheme|match-a|, we introduce a tagging scheme that makes our
formulas unambiguous.  To eliminate the use of \scheme|copy-termo|, we
use substitution instead of copying terms.  Universally quantified
formulas are used as templates, rather than instantiated directly;
instead of representing universally quantified variables with logic
variables, we use the noms of nominal logic. We then use nominal
unification to write a substitution relation that replaces quantified
variables with logic variables, leaving the original template
untouched.

The resulting declarative theorem prover is interesting for two
reasons. First, because of the technique used to arrive at its
definition: we use declarative substitution rather than
\scheme|copy-termo|.  To our knowledge, there is no method for
copying arbitrary terms declaratively. Our solution is not completely
general but is useful when a term is used as a template for copying,
as in the case of \leantap.  Second, because of the flexibility of the
prover itself: \alphatapsp is capable of instantiating non-ground
theorems during the proof process, and accepts non-ground
\textit{proofs}, as well.  Whereas \leantapsp is fully automated and
either succeeds or fails to prove a given theorem, \alphatapsp can
accept guidance from the user in the form of a partially-instantiated
proof, regardless of whether the theorem is ground.

We present an implementation of \alphatapsp in
section~\ref{implementation} , demonstrating our technique for
eliminating cut and \mbox{\texttt{copy\_term/2}} from \leantap. Our
implementation demonstrates our contributions: first, it illustrates a
method for eliminating common impure operators, and demonstrates the
use of nominal logic for representing formulas in first-order logic;
second, it shows that the tableau process can be represented as a
relation between formulas and their tableaux; and third, it
demonstrates the flexibility of relational provers to mimic the full
spectrum of theorem provers, from fully automated to fully dependent
on the user.

This chapter is organized as follows. In section~\ref{tableau} we
describe the concept of tableau theorem proving. In
section~\ref{alphatap} we motivate our declarative prover by examining
its declarative properties and the proofs it returns. In
section~\ref{implementation} we present the implementation of
\alphatap, and in section~\ref{performance} we briefly examine
\alphatap's performance. Familiarity with tableau theorem proving
would be helpful; for more on this topic, see the references given in
section~\ref{tableau}.  In addition, a reading knowledge of Prolog
would be useful, but is not necessary; for readers unfamiliar with
Prolog, carefully following the miniKanren and \alphakanrensp code
should be sufficient for understanding all the ideas in this chapter.

\section{Tableau Theorem Proving}\label{tableau}

We begin with an introduction to tableau theorem proving and its
implementation in \leantap.

Tableau is a method of proving first-order theorems that works by
refuting the theorem's negation. In our description we assume basic
knowledge of first-order logic; for coverage of this subject and a
more complete description of tableau proving, see
\citet{fitting1996fol}.  For simplicity, we consider only
formulas in Skolemized \textit{negation normal form} (NNF).
Converting a formula to this form requires removing existential
quantifiers through Skolemization, reducing logical connectives so
that only $\wedge$, $\vee$, and $\neg$ remain, and pushing negations
inward until they are applied only to literals---see section~3 of
\citet{beckert95leantap} for details.

To form a tableau, a compound formula is expanded into branches
recursively until no compound formulas remain.  The leaves of this
tree structure are referred to as \textit{literals}. \leantapsp forms
and expands the tableau according to the following rules. When the
prover encounters a conjunction $x \wedge y$, it expands both $x$ and
$y$ on the same branch. When the prover encounters a disjunction $x
\vee y$, it splits the tableau and expands $x$ and $y$ on separate
branches.  Once a formula has been fully expanded into a tableau, it
can be proved unsatisfiable if on each branch of the tableau there
exist two complementary literals $a$ and $\neg a$ (each branch is
\textit{closed}).  In the case of propositional logic, syntactic
comparison is sufficient to find complementary literals; in
first-order logic, sound unification must be used. A closed tableau
represents a proof that the original formula is unsatisfiable.

 The addition of universal quantifiers makes the expansion process more
 complicated. To prove a universally quantified formula \mbox{$\forall x. M$},
 \leantapsp generates a logic variable $v$ and expands $M$,
 replacing all occurrences of $x$ with $v$ (i.e., it expands $M^{\prime}$ where
 $M^{\prime} = M[v/x]$).  If \leantapsp is unable to close the current branch
 after this expansion, it has the option of generating another logic
 variable and expanding the original formula again. When the prover
 expands the universally quantified formula \mbox{$\forall x.  F(x) \wedge ( \neg F({\sf a})
   \vee \neg F({\sf b}) )$}, for example, \mbox{$\forall x.  F(x)$}
 must be expanded twice, since $x$ cannot be instantiated to both
 \textsf{a} and \textsf{b}.

\section{Introducing \alphatap}\label{alphatap}

We begin by presenting some examples of \alphatap's abilities, both in
proving ground theorems and in generating theorems. We also explore
the proofs generated by \alphatap, and show how passing
partially-instantiated proofs to the prover can greatly improve its
performance.

\subsection{Running Forwards}\label{forwards}

Both \leantapsp and \alphatapsp can prove ground theorems; in
addition, \alphatap\ produces a proof.  This proof is a list
representing the steps taken to build a closed tableau for the
theorem; \citet{paulson99generic} has shown that translation to
a more standard format is possible. Since a closed tableau represents
an unsatisfiable formula, such a list of steps proves that the
negation of the formula is valid. If the list of steps is ground, the
proof search becomes deterministic, and \alphatapsp acts as a proof
checker.

\leantapsp encodes first-order formulas using Prolog terms.  For
example, the term \mbox{\texttt{(p(b),all(X,(-p(X);p(s(X)))))}}
represents \mbox{$p($\textsf{b}$) \wedge \forall x . \neg p(x) \vee
  p(s(x))$}. In our prover, we represent formulas using Scheme lists
with extra tags:

%, and in our final version we adopt a more extensive tagging
%scheme. The \schemeresult|forall| binder is represented by
%\alphakanren's \scheme|tie|, and variables are represented by noms.
%Our example formula is represented by the ground list:

\schemedisplayspace
\begin{schemeresponse}
(and-tag (pos (app p (app b))) (forall (tie anom (or-tag (neg (app p (var-tag anom)))
                                                     (pos (app p (app s (var-tag anom))))))))

\end{schemeresponse}

% The Prolog query \mbox{\texttt{prove(Fml,[],[],[],VarLim)}} succeeds
% if the formula \texttt{Fml} is unsatisfiable.  Similarly, the
% \alphakanrensp goal \mbox{\scheme|(proveo fml '() '() '() proof)|}
% succeeds if \scheme|fml| can be shown to be unsatisfiable via the
% proof \scheme|proof|.

Consider Pelletier Problem 18~\cite{pelletier1986sfp}: \mbox{$\exists
  y.  \forall x. F(y) \Rightarrow F(x)$}. To prove this theorem in
\alphatap, we transform it into the following \textit{negation} of the
NNF:

\schemedisplayspace
\begin{schemeresponse}
(forall (tie anom (and-tag (pos (app f (var-tag anom))) (neg (app f (app g1 (var-tag anom)))))))
\end{schemeresponse}

\noindent where \schemeresult|`(app ,g1 (var-tag anom))| represents the
application of a Skolem function to the universally quantified
variable $a$. Passing this formula to the prover, we obtain the proof
\schemeresult|`(univ conj savefml savefml univ conj close)|. This proof
lists the steps the prover (presented in section~\ref{matcha}) follows to close
the tableau. Because both conjuncts of the formula contain the nom
$a$, we must expand the universally quantified formula more than once.

Partially instantiating the proof helps \alphatapsp prove theorems
with similar subparts. We can create a non-ground proof that describes
in general how to prove the subparts and have \alphatapsp fill in the
trivial differences. This can speed up the search for a proof
considerably. By inspecting the negated NNF of Pelletier Problem~21,
for example, we can see that there are at least two portions of the
theorem that will have the same proof. By specifying the structure of
the first part of the proof and constraining the identical portions by
using the same logic variable to represent both, we can give the
prover some guidance without specifying the whole proof. We pass the
following non-ground proof to \alphatap:

\schemedisplayspace
\vspace{-2pt}
\begin{centering}
\begin{schemeresponse}
(conj univ split (conj savefml savefml conj split Xvar Xvar)
      (conj savefml savefml conj split (close) (savefml split Yvar Yvar)))
\end{schemeresponse}
\end{centering}
\vspace{-2pt}

\noindent On our test machine, our prover solves the original problem
with no help in 68 milliseconds (ms); given the knowledge that the
later parts of the proof will be duplicated, the prover takes only 27
ms. This technique also yields improvement when applied to Pelletier
Problem 43: inspecting the negated NNF of the formula, we see two
parts that look nearly identical. The first part of the negated
NNF---the part representing the theorem itself---has the following
form:

\schemedisplayspace
\vspace{-2pt}
\begin{centering}
\begin{schemeresponse}
(and-tag (or-tag (and-tag (neg (app Q (app g4) (app g3)))
              (pos (app Q (app g3) (app g4))))
         (and-tag (pos (app Q (app g4) (app g3)))
              (neg (app Q (app g3) (app g4))))) ...)
\end{schemeresponse}
\end{centering}
\vspace{-2pt}

\noindent Since we suspect that the same proof might suffice for both
branches of the theorem, we give the prover the partially-instantiated
proof \mbox{\schemeresult|`(conj split Xvar Xvar)|}. Given just this
small amount of help, \alphatapsp proves the theorem in 720 ms,
compared to 1.5 seconds when the prover has no help at all.  While
situations in which large parts of a proof are identical are rare,
this technique also allows us to handle situations in which different
parts of a proof are merely similar by instantiating as much or as
little of the proof as necessary.

\subsection{Running Backwards}\label{backwards}

% \begin{figure}[H]
% \begin{centering}
% \begin{tabular}{| r | c | c | c | c |}
%   \hline
%   Problem & \thinspace \leantap \thinspace\footnotemark[4] \thinspace &
%   Translation\footnotemark[3] & \thinspace \alphatap\footnotemark[3]
%   \thinspace & \thinspace \alphatap$\!_G$\footnotemark[4]$^,$\footnotemark[6]  \\
%   \hline
%   1 & ? &

%   \hline
% \end{tabular}
% \caption{\alphatap's Performance on Pelletier's Problems\protect\footnotemark[2]
%   \label{fig:performance}}
% \end{centering}
% \end{figure}

%\vspace{-6pt}

%  Testing our prover on
% several of Pelletier's 75 problems~\cite{pelletier1986sfp} shows that
% \alphatapsp is about three to five times slower than our translation
% of \leantap. The translation solves problem 32, for example, in about
% one second, while \alphatapsp takes about three
% seconds; problem 26 takes our translation of \leantapsp about 13
% seconds, while \alphatapsp needs 36 seconds.

Unlike \leantap, \alphatapsp can generate valid theorems.  Some
interpretation of the results is required since the theorems generated
are negated formulas in NNF.\footnote{The full implementation of
  \alphatapsp includes a simple declarative translator from negated
  NNF to a positive form.}  In the example

\smallskip

\scheme|(run1 (q) (exist (x) (proveo q '() '() '() x)))|

\hspace{0.1cm}$\Rightarrow$
\schemeresult|`((and-tag (pos (app _.0)) (neg (app _.0))))|

\smallskip

\noindent
the reified logic variable \schemeresult|_.0| represents any
first-order formula $p$, and the entire answer represents the formula
$p \wedge \neg p$.  Negating this formula yields the original theorem:
$\neg p \vee p$, or the law of excluded middle.  We can also generate
more complicated theorems; here we use the ``generate and test'' idiom
to find the first theorem matching the negated NNF of the inference
rule {\it modus ponens}:

\schemedisplayspace
\begin{schemedisplay}
(run1 (q)
  (exist (x)
    (proveo x '() '() '() q)
    (== `(and-tag (and-tag (or-tag (neg (app a)) (pos (app b))) (pos (app a))) (neg (app b)))
        x)))
\end{schemedisplay}
 \vspace{-.1cm}
\noindent $\Rightarrow$ \schemeresult|`((conj conj split (savefml close) (savefml savefml close)))|

\smallskip

\noindent This process takes about 5.1 seconds; {\it modus ponens} is the
173rd theorem to be generated, and the prover also generates a proof
of its validity. When this proof is given to \alphatap, {\it modus ponens}
is the sixth theorem generated, and the process takes only 20 ms.

Thus the declarative nature of \alphatapsp is useful both for
generating theorems and for producing proofs. Due to this flexibility,
\alphatapsp could become the core of a larger proof system.  Automated
theorem provers like \leantapsp are limited in the complexity of the
problems they can solve, but given the ability to accept assistance
from the user, more problems become tractable.

%can solve more difficult problems.

%\footnotetext[7]{\alphatap$\!_G$ uses the unique name and preprocessor
%  approach described in section 4.2.}

As an example, consider Pelletier Problem 47: Schubert's Steamroller.
This problem is difficult for tableau-based provers like \leantapsp
and \alphatap, and neither can solve it
automatically~\cite{beckert95leantap}.  Given some help, however,
\alphatapsp can prove the Steamroller. Our approach is to prove a
series of smaller lemmas that act as stepping stones toward the final
theorem; as each lemma is proved, it is added as an assumption in
proving the remaining ones.  The proof process is automated---the user
need only specify which lemmas to prove and in what order. Using this
strategy, \alphatapsp proves the Steamroller in about five seconds;
the proof requires twenty lemmas.

\alphatapsp thus offers an interesting compromise between large proof
assistants and smaller automated provers. It achieves some of the
capabilities of a larger system while maintaining the lean deduction
philosophy introduced by \leantap. Like an automated prover, it is
capable of proving simple theorems without user guidance. Confronted
with a more complex theorem, however, the user can provide a
partially-instantiated proof; \alphatapsp can then check the proof and
fill in the trivial parts the user has left out.  Because \alphatapsp
is declarative, the user may even leave required axioms out of the
theorem to be proved and have the system derive them. This flexibility
comes at no extra cost to the user---the prover remains both concise
and reasonably efficient.

%% New

The flexibility of \alphatapsp means that it could be made interactive
through the addition of a read-eval-print loop and a simple proof
translator between \alphatap's proofs and a more human-readable
format. Since the proof given to \alphatapsp may be partially
instantiated, such an interface would allow the user to conveniently
guide \alphatapsp in proving complex problems. With the addition of
equality and the ability to perform single beta steps, this
flexibility would become more interesting---in addition to reasoning
about programs and proving properties about them, \alphatapsp would
instantiate non-ground programs during the proof process.

\section{Implementation}\label{implementation}

We now present the implementation of \alphatap. We begin with a
translation of \leantapsp from Prolog into \alphakanren. We then show
how to eliminate the translation's impure features through a
combination of substitution and tagging.

\leantapsp implements both expansion and closing of the tableau. When
the prover encounters a conjunction, it uses its argument
\texttt{UnExp} as a stack (Figure~\ref{fig:translation}): \leantapsp
expands the first conjunct, pushing the second onto the stack for
later expansion. If the first conjunct cannot be refuted, the second
is popped off the stack and expansion begins again.  When a
disjunction is encountered, the split in the tableau is reflected by
two recursive calls. When a universal quantifier is encountered, the
quantified variable is replaced by a new logic variable, and the
formula is expanded.  The \texttt{FreeV} argument is used to avoid
replacing the free variables of the formula.  \leantapsp keeps a list
of the literals it has encountered on the current branch of the
tableau in the argument \texttt{Lits}.  When a literal is encountered,
\leantapsp attempts to unify its negation with each literal in
\texttt{Lits}; if any unification succeeds, the branch is closed.
Otherwise, the current literal is added to \texttt{Lits} and expansion
continues with a formula from \texttt{UnExp}.

\subsection{Translation to \alphakanren}\label{translation}

While \alphakanrensp is similar to Prolog with the addition of nominal
unification, \alphakanrensp uses a variant of interleaving
depth-first search~\cite{backtracking}, so the order of
\scheme|conde| or \scheme|match-e| clauses in \alphakanrensp is irrelevant. Because of
Prolog's depth-first search, \leantapsp must use \texttt{VarLim} to
limit its search depth; in \alphakanren, \texttt{VarLim} is not
necessary, and thus we omit it.

In Figure~\ref{fig:translation} we present mK\leantap, our translation
of \leantapsp into \alphakanren; we label two clauses (\onet, \twot),
since we will modify these clauses later. To express Prolog's cuts,
our definition uses \scheme|match-a|.  The final two clauses of
\leantapsp do not contain Prolog cuts; in mK\leantap, they are
combined into a single clause containing a \scheme|conde|.  In place
of \leantap\thinspace's recursive call to \texttt{prove} to check the
membership of \texttt{Lit} in \texttt{Lits}, we call \scheme|membero|,
which performs a membership check using sound unification.\footnote{We define \scheme|membero| in Figure~\ref{fig:ending}; \scheme|membero| \emph{must} use sound unification, and cannot use \scheme|==-no-check|.}  % Prolog's \texttt{copy\_term/2} is
% not built into \alphakanren; this addition is available as part of the
% mK\leantapsp source code.

%\begin{figure}[ht]
\begin{figure}[H]
%\vspace{-.3in}

\begin{tabular}{l l}

 &

\begin{minipage}{2.3in}
\begin{schemedisplay}
 (define proveo
   (lambda (fml unexp lits freev)
     (match-a fml
\end{schemedisplay}
\end{minipage} \\

\begin{minipage}{2.3in}
\begin{verbatim}
prove((E1,E2),UnExp,Lits,
      FreeV,VarLim) :- !,
  prove(E1,[E2|UnExp],Lits,
        FreeV,VarLim).
\end{verbatim}
\end{minipage}
 &
\begin{minipage}{2in}
\begin{schemedisplay}
      (`(and-tag ,e1 ,e2)
        (proveo e1 `(,e2 . ,unexp) lits freev))
\end{schemedisplay}
\end{minipage}
\\

\begin{minipage}{2in}
\begin{verbatim}
prove((E1;E2),UnExp,Lits,
      FreeV,VarLim) :- !,
  prove(E1,UnExp,Lits,FreeV,VarLim),
  prove(E2,UnExp,Lits,FreeV,Varlim).
\end{verbatim}
\end{minipage}
 &
\begin{minipage}{2in}
\vspace{1mm}
\begin{schemedisplay}
      (`(or-tag ,e1 ,e2)
        (proveo e1 unexp lits freev)
        (proveo e2 unexp lits freev))
\end{schemedisplay}
\vspace{1mm}
\end{minipage}
\\

\begin{minipage}{2in}
\begin{verbatim}
prove(all(X,Fml),UnExp,Lits,
      FreeV,VarLim) :- !,
  \+ length(FreeV,VarLim),
  copy_term((X,Fml,FreeV),
            (X1,Fml1,FreeV)),
  append(UnExp,[all(X,Fml)],UnExp1),
  prove(Fml1,UnExp1,Lits,
        [X1|FreeV],VarLim).
\end{verbatim}
\end{minipage}
 &
\begin{minipage}{2in}
\begin{schemedisplay}
     $\onet$(`(forall ,x ,body)
         (exist (x1 body1 unexp1)
           (copy-termo `(,x ,body ,freev)
                       `(,x1 ,body1 ,freev))
           (appendo unexp `(,fml) unexp1)
           (proveo body1 unexp1 lits
                   `(,x1 . ,freev))))
\end{schemedisplay}
\end{minipage}
\\

\begin{minipage}{2in}
\begin{verbatim}
prove(Lit,_,[L|Lits],_,_) :-
  (Lit = -Neg; -Lit = Neg) ->
   (unify(Neg,L);
    prove(Lit,[],Lits,_,_)).
\end{verbatim}

\end{minipage}
 &
\begin{minipage}{2in}
\begin{schemedisplay}
     $\twot$(fml
         (conde
           ((match-a `(,fml ,neg)
              (`((not ,neg) ,neg))
              (`(,fml (not ,fml))))
            (membero neg lits))
           \end{schemedisplay}
           \end{minipage}
           \\

           \begin{minipage}{2in}
           \begin{verbatim}
           prove(Lit,[Next|UnExp],Lits,
                    FreeV,VarLim) :-
           prove(Next,UnExp,[Lit|Lits],
                           FreeV,VarLim).
           \end{verbatim}
           \end{minipage}
           &
           \begin{minipage}{2in}
           \begin{schemedisplay}
        ((exist (next unexp1)
           (== `(,next . ,unexp1) unexp)
           (proveo next unexp1 `(,fml . ,lits)
                   freev))))))))
\end{schemedisplay}
\end{minipage}
\\

\end{tabular}
\caption{\leantapsp and mK\leantap\thinspace: a translation from Prolog to \alphakanren
  \label{fig:translation}}
%\vspace{-.3in}
\end{figure}

\subsection{Eliminating \copytermo}\label{copytermo}
\enlargethispage{1\baselineskip} %

Since \scheme|copy-termo| is an impure operator, its use makes
\scheme|proveo| non-declarative: reordering the goals in the prover
can result in different behavior. For example, moving the call to
\scheme|copy-termo| after the call to \scheme|proveo| causes the
prover to diverge when given any universally quantified formula. To
make our prover declarative, we must eliminate the use of
\scheme|copy-termo|.

Tagging the logic variables that represent universally quantified
variables allows the use of a declarative technique that creates two
pristine copies of the original term: one copy may be expanded and the
other saved for later copying.  Unfortunately, this copying examines
the entire body of each quantified formula and instantiates the
original term to a potentially invalid formula.

Another approach is to represent quantified variables with symbols or
strings. When a new instantiation is needed, a new variable name can
be generated, and the new name can be substituted for the old without
affecting the original formula. This solution does not destroy the
prover's input, but it is difficult to ensure that the provided data
is in the correct form declaratively: if the formula to be proved is
non-ground, then the prover must generate unique names.  If the
formula \textit{does} contain these names, however, the prover must
\textit{not} generate new ones. This problem can be solved with a
declarative preprocessor that expects a logical formula
\textit{without} names and puts them in place. If the preprocessor is
passed a non-ground formula, it instantiates the formula to the
correct form. %We have implemented this strategy in a Prolog prover we
%call \alphatap$\!_G$;
The requirement of a preprocessor, however,
means the prover itself is not declarative.

We use nominal logic to solve the \scheme|copy-termo| problem.
Nominal logic is a good fit for this problem, as it is designed to
handle the complexities of dealing with names and binders
declaratively.
%Using
%noms to represent universally quantified variables and the
%\scheme|tie| operator to represent the $\forall$ binder allows us to
%avoid the use of logic variables to represent quantified variables.
Since noms represent unique names, we achieve the benefits of the
symbol or string approach without the use of a preprocessor. We can
generate unique names each time we encounter a universally quantified
formula, and use nominal unification to perform the renaming of the
quantified variable. If the original formula is uninstantiated, our
newly-generated name is unique and is put in place correctly; we no
longer need a preprocessor to perform this function.

Using the tools of nominal logic, we can modify mK\leantapsp to
represent universally quantified variables using noms and to perform
substitution instead of copying.  When the prover reaches a literal,
however, it must replace each nom with a logic variable, so that
unification may successfully compare literals. To accomplish this, we
associate a logic variable with each unique nom, and replace every nom
with its associated variable before comparing literals. These
variables are generated each time the prover expands a quantified
formula.

To implement this strategy, we change our representation of formulas
slightly. Instead of representing $\forall x. F(x)$ as
\mbox{\schemeresult|`(forall Xvar (f Xvar))|}, we use a nom wrapped in
a \scheme|var-tag| tag to represent a variable reference, and the
term constructor \scheme|tie| to represent the $\forall$ binder:
\mbox{\schemeresult|`(forall (tie anom (f (var-tag anom))))|}, where $a$ is
a nom.  The \scheme|var-tag| tag allows us to distinguish noms
representing variables from other formulas. We now write a relation
\scheme|subst-lito| to perform substitution of logic variables for
tagged noms in a literal, and we modify the literal case of
\scheme|proveo| to use it. We also replace the clause handling
\schemeresult|forall| formulas and define \scheme|lookupo|. The two
clauses of \scheme|lookupo| overlap, but since each mapping in the
environment is from a unique nom to a logic variable, a particular nom
will never appear twice.

We present the changes needed to eliminate \scheme|copy-termo| from
mK\leantapsp in Figure~\ref{fig:changes}. Instead of copying the body
of each universally quantified formula, we generate a logic variable
\scheme|x| and add an association between the nom representing the
quantified variable and \scheme|x| to the current environment. When we
prepare to close a branch of the tableau, we call \scheme|subst-lito|,
replacing the noms in the current literal with their associated logic
variables.

\begin{figure}[H]

\noindent \begin{tabular}{l l}
\begin{minipage}{2.5in}
\small
\begin{schemedisplay}
$\onet$(`(forall (tie-tag ,@a ,body))
   (exist (x unexp1)
     (appendo unexp `(,fml) unexp1)
     (proveo body unexp1 lits
             `((,a . ,x) . ,env))))

$\twot$(fml
  (exist (lit)
    (subst-lito fml env lit)
    (conde
      ((match-a `(,lit ,neg)
         (`((not ,neg) ,neg))
         (`(,lit (not ,lit))))
       (membero neg lits))
      ((exist (next unexp1)
         (== `(,next . ,unexp1) unexp)
         (proveo next unexp1 `(,lit . ,lits)
                 env))))))
\end{schemedisplay}

\vspace{.1cm}
\end{minipage}
&

\begin{minipage}{1.2in}
\small
%\schemeinput{code/lookupo}
\begin{schemedisplay}
(define lookupo
  (lambda (a env out)
    (match-e env
      (`((,a . ,out) . ,rest))
      (`(,first . ,rest)
       (lookupo a rest out)))))
\end{schemedisplay}

\begin{schemedisplay}
(define subst-lito
  (lambda (fml env out)
    (match-a `(,fml ,out)
      (`((var-tag ,a) ,out)
       (lookupo a env out))
      (`((,e1 . ,e2) (,r1 . ,r2))
       (subst-lito e1 env r1)
       (subst-lito e2 env r2))
      (`(,fml ,fml)))))
\end{schemedisplay}

\end{minipage}

\end{tabular}

\caption{Changes to mK\leantapsp to eliminate \protect\scheme|copy-termo|
  \label{fig:changes}}
%\vspace{-.2in}
\end{figure}

The original \mbox{\texttt{copy\_term/2}} approach used by \leantapsp and
mK\leantapsp avoids replacing free variables by copying the list
\scheme|`(,x ,body ,freev)|. The copied version is unified with the list
\scheme|`(x1 body1 ,freev)|, so that \textit{only} the variable
\scheme|x| will be replaced by a new logic variable---the free
variables will be copied, but those copies will be unified with the
original variables afterwards. Since our substitution strategy does
not affect free variables, the \scheme|freev| argument is no longer
needed, and so we have eliminated it.

\subsection{Eliminating \matchasymbol}\label{matcha}

Both \scheme|proveo| and \scheme|subst-lito| use \scheme|match-a|
because the clauses that recognize literals overlap with the other
clauses. To solve this problem, we have designed a tagging scheme that
ensures that the clauses of our substitution and \scheme|proveo|
relations do not overlap.  To this end, we tag both positive and
negative literals, applications, and variables. Constants are
represented by applications of zero arguments. Our prover thus accepts
formulas of the following form:

% \begin{center}
%   \begin{tabular}{lcl}
%     $<$Fml$>$ & $\rightarrow$ & $($\textsf{or} $<$Fml$>$ $<$Fml$>)$
% % \\ & $|$ &
% $|$ $($\textsf{and} $<$Fml$>$ $<$Fml$>)$
%  \\ & $|$ &
% $($\textsf{forall} $<$nom$>$ $<$Fml$>)$
% % \\ & $|$ &
% $|$ $($\textsf{lit} $<$Lit$>)$
% \\
%     $<$Lit$>$ & $\rightarrow$ & $($\textsf{pos} $<$Term$>)$
% % \\ & $|$ &
% $|$ $($\textsf{neg} $<$Term$>)$
%  \\
% $<$Term$>$ & $\rightarrow$ & $($\textsf{sym} $<$symbol$>)$
% % \\ & $|$ &
% $|$ $($\textsf{var} $<$nom$>)$
% % \\ & $|$ &
% $|$ $($\textsf{app} $<$symbol$>$ $<$Term$>$*$)$ \\
%   \end{tabular}
% \end{center}

% \begin{center}
%   \begin{tabular}{lcl}
%     Fml & $\rightarrow$ & $($\textsf{and} Fml Fml$)$
% % \\ & $|$ &
% $|$ $($\textsf{or} Fml Fml$)$
% % \\ & $|$ &
% $|$ $($\textsf{forall} $($\scheme|tie| nom Fml$))$
% % \\ & $|$ &
% $|$ Lit
% \\
%     Lit & $\rightarrow$ & $($\textsf{pos} Term$)$
% % \\ & $|$ &
% $|$ $($\textsf{neg} Term$)$
%  \\
% Term & $\rightarrow$ & %$($\textsf{sym} symbol$)$
% % \\ & $|$ &
% %$|$
% $($\textsf{var} nom$)$
% % \\ & $|$ &
% $|$ $($\textsf{app} symbol Term*$)$ \\
%   \end{tabular}
% \end{center}

%\vspace{-.2cm}

\begin{center}
  \begin{tabular}{lcl}
    \textit{Fml} & $\rightarrow$ & $($\textsf{and}  \textit{Fml}  \textit{Fml}$)$
$|$ $($\textsf{or}  \textit{Fml} \textit{Fml}$)$
$|$ $($\textsf{forall} $($\scheme|tie| \textit{Nom} \textit{Fml}$))$
$|$ \textit{Lit}
\\
    \textit{Lit} & $\rightarrow$ & $($\textsf{pos} \textit{Term}$)$
$|$ $($\textsf{neg} \textit{Term}$)$
 \\
\textit{Term} & $\rightarrow$ &
$($\textsf{var} \textit{Nom}$)$
$|$ $($\textsf{app} \textit{Symbol} \textit{Term}*$)$ \\
  \end{tabular}
\end{center}

%\vspace{-.2cm}

This scheme has been chosen carefully to allow unification to compare
literals. In particular, the tags on variables \textit{must} be
discarded before literals are compared.  Consider the two non-ground
literals \mbox{\schemeresult|`(not (f Xvar))|} and
\mbox{\schemeresult|`(f (p Yvar))|}.  These literals are complementary:
the negation of one unifies with the other, associating $x$ with
\mbox{\schemeresult|`(p Yvar)|}. When we apply our tagging scheme,
however, these literals become \mbox{\schemeresult|`(neg (app f (var-tag Xvar)))|} and \mbox{\schemeresult|`(pos (app f (app p (var-tag Yvar))))|}, respectively, and are no longer complementary: their
subexpressions \mbox{\schemeresult|`(var-tag Xvar)|} and
\mbox{\schemeresult|`(app p (var-tag Yvar))|} do not unify. To avoid this
problem, our substitution relation discards the \textsf{var} tag when
it replaces noms with logic variables.

\begin{figure}[H]
%\vspace{-.2in}
\hspace{-.1in}
\begin{tabular}{l l}
\begin{minipage}{1.8in}
%\schemeinput{code/alphatapleft}
\begin{schemedisplay}
(define proveo
  (lambda (fml unexp lits env proof)
    (match-e `(,fml ,proof)
      (`((and-tag ,e1 ,e2) (conj . ,prf))
       (proveo e1 `(,e2 . ,unexp)
               lits env prf))
      (`((or-tag ,e1 ,e2) (split ,prf1 ,prf2))
       (proveo e1 unexp lits env prf1)
       (proveo e2 unexp lits env prf2))
      (`((forall (tie-tag ,@a ,body)) (univ . ,prf))
       (exist (x unexp1)
         (appendo unexp `(,fml) unexp1)
         (proveo body unexp1 lits
                 `((,a . ,x) . ,env) prf)))
      (`(,fml ,proof)
       (exist (lit)
         (subst-lito fml env lit)
         (conde
           ((== `(close) proof)
            (match-e `(,lit ,neg)
              (`((pos ,tm) (neg ,tm)))
              (`((neg ,tm) (pos ,tm))))
            (membero neg lits))
           ((exist (next unexp1 prf)
              (== `(,next . ,unexp1) unexp)
              (== `(savefml . ,prf) proof)
              (proveo next unexp1 `(,lit . ,lits)
                      env prf)))))))))
\end{schemedisplay}
%\vspace{1.3cm}
\end{minipage}

&

\hspace{-0.3in}
\begin{minipage}{1.8in}
%\schemeinput{code/alphatapright}
\begin{schemedisplay}
(define appendo
  (lambda-e (ls s out)
    (`(() ,s ,s))
    (`((,a . ,d) ,s (,a . ,r))
     (appendo d s r))))

(define subst-lito
  (lambda-e (fml env out)
    (`((pos ,l) ,env (pos ,r))
     (subst-termo l env r))
    (`((neg ,l) ,env (neg ,r))
     (subst-termo l env r))))

(define subst-termo
  (lambda-e (fml env out)
    (`((var-tag ,a) ,env ,out)
     (lookupo a env out))
    (`((app ,f . ,d) ,env (app ,f . ,r))
     (subst-term* d env r))))

(define subst-term*
  (lambda-e (tm* env out)
    (`(() __ ()))
    (`((,e1 . ,e2) ,env (,r1 . ,r2))
     (subst-termo e1 env r1)
     (subst-term* e2 env r2))))

(define membero
  (lambda (x ls)
    (exist (a d)
      (== `(,a . ,d) ls)
      (conde
        ((== a x))
        ((membero x d))))))
\end{schemedisplay}
%\vspace{1.0cm}
\end{minipage}

\end{tabular}
\caption{Final definition of \alphatap
  \label{fig:ending}}
%\vspace{-.3in}
\end{figure}

Given our new tagging scheme, we can easily rewrite our substitution
relation without the use of \scheme|match-a|. We simply follow the
production rules of the grammar, defining a relation to recognize
each.

Finally, we modify \scheme|proveo| to take advantage of the same tags.
We also add a \scheme|proof| argument to \scheme|proveo|.  We call
this version of the prover \alphatap, and present its definition in
Figure~\ref{fig:ending}. It is declarative, since we have eliminated
the use of \scheme|copy-termo| and every use of \scheme|match-a|. In
addition to being a sound and complete theorem prover for first-order
logic, \alphatapsp can now generate valid first-order theorems.

\section{Performance}\label{performance}
\enlargethispage{1\baselineskip} %

Like the original \leantap, \alphatapsp can prove many theorems in
first-order logic. Because it is declarative, \alphatapsp is generally
slower at proving ground theorems than mK\leantap, which is slower
than the original \leantap. Figure~\ref{fig:performance} presents a
summary of \alphatap's performance on the first 46 of Pelletier's 75
problems~\cite{pelletier1986sfp}, showing it to be roughly twice as
slow as mK\leantap.

These performance numbers suggest that while there is a penalty to be
paid for declarativeness, it is not so severe as to cripple the
prover. The advantage mK\leantapsp enjoys over the original \leantapsp
in Problem 34 is due to \alphakanren's interleaving search strategy;
as the result for mK\leantapsp shows, the original \leantapsp is faster
than \alphatapsp for any given search strategy.

Many automated provers now use the TPTP problem
library~\cite{stucliffe1994tpl} to assess performance. Even though it
is faster than \alphatap, \leantapsp solves few of the TPTP
problems. The Pelletier Problems, on the other hand, fall into the
class of theorems \leantapsp was designed to prove, and so we feel
they provide a better set of tests for the comparison between
\leantapsp and \alphatap.

\begin{figure}[h]
%\vspace{-.2in}
\begin{centering}
\begin{tabular}{l l}

\hspace{-.1in}
\begin{minipage}{2.7in}
\begin{tabular}{| r | c | c | c | } %c |
  \hline
  \thinspace \thinspace \# & \thinspace \leantap  \thinspace &
  mK\leantap \thinspace & \thinspace \alphatap
  \thinspace %& \thinspace \alphatap$\!_G$\footnotemark[5]$^,$\footnotemark[7]
  \\
  \hline
1 & 0.1 & 0.7 & 2.0 \\
2 & 0.0 & 0.1 & 0.3 \\
3 & 0.0 & 0.2 & 0.5 \\
4 & 0.0 & 1.0 & 1.7 \\
5 & 0.1 & 1.2 & 2.5 \\
6 & 0.0 & 0.1 & 0.2 \\
7 & 0.0 & 0.1 & 0.2 \\
8 & 0.0 & 0.3 & 0.8 \\
9 & 0.1 & 4.3 & 9.7 \\
10 & 0.3 & 5.5 & 10.2 \\
11 & 0.0 & 0.3 & 0.6 \\
12 & 0.6 & 17.7 & 31.9 \\
13 & 0.1 & 3.7 & 8.2 \\
14 & 0.1 & 4.2 & 9.7 \\
15 & 0.0 & 0.8 & 1.9 \\
16 & 0.0 & 0.2 & 0.6 \\
17 & 1.1 & 9.2 & 18.1 \\
18 & 0.1 & 0.5 & 1.2 \\
19 & 0.3 & 15.1 & 33.5 \\
20 & 0.5 & 8.1 & 12.7 \\
21 & 0.4 & 22.1 & 38.7 \\
22 & 0.1 & 3.4 & 6.4 \\
23 & 0.1 & 2.5 & 5.4 \\

  \hline
\end{tabular}

\end{minipage}

&

\begin{minipage}{2.5in}
\begin{tabular}{| r | c | c | c |} %c |
  \hline
  \# & \thinspace \leantap  \thinspace &
  mK\leantap \thinspace & \thinspace \alphatap
  \thinspace %& \thinspace \alphatap$\!_G$\footnotemark[5]$^,$\footnotemark[7]
\\
  \hline
24 & 1.7 & 31.9 & 60.3 \\
25 & 0.2 & 7.5 & 14.1 \\
26 & 0.8 & 130.9 & 187.5 \\
27 & 2.3 & 40.4 & 79.3 \\
28 & 0.3 & 19.1 & 29.6 \\
29 & 0.1 & 27.9 & 57.0 \\
30 & 0.1 & 4.2 & 9.6 \\
31 & 0.3 & 13.2 & 23.1 \\
32 & 0.2 & 23.9 & 42.4 \\
33 & 0.1 & 15.9 & 39.2 \\
34 & 199129.0  & 7272.9 & 8493.5 \\
35 & 0.1 & 0.5 & 1.1 \\
36 & 0.2 & 6.7 & 12.4 \\
37 & 0.8 & 123.3 & 169.2 \\
38 & 8.9 & 4228.8 & 8363.8 \\
39 & 0.0 & 1.1 & 2.8 \\
40 & 0.2 & 8.1 & 19.2 \\
41 & 0.1 & 6.9 & 17.0 \\
42 & 0.4 & 15.0 & 32.1 \\
43 & 43.2 & 668.4 & 1509.6 \\
44 & 0.3 & 15.1 & 35.7 \\
45 & 3.4 & 145.3 & 239.7 \\
46 & 7.7 & 505.5 & 931.2 \\

  \hline
\end{tabular}
\end{minipage}
\end{tabular}

\caption{Performance of \leantap, mK\leantap, and \alphatapsp on the
  first 46 Pelletier Problems.
  All times are in milliseconds, averaged over 100 trials.
  All tests were run \mbox{under} Debian
  Linux on an IBM Thinkpad
  X40 with a 1.1GHz Intel Pentium-M processor and 768MB RAM.
  \leantapsp tests were run under SWI-Prolog 5.6.55;
  mK\leantapsp and \alphatapsp tests were run under Ikarus Scheme
  0.0.3+.
  \label{fig:performance}}
\end{centering}
%\vspace{-.2in}

\end{figure}

\section{Applicability of These Techniques}

To avoid the use of \scheme|copy-termo|, we have represented
universally quantified variables with noms rather than logic
variables, allowing us to perform substitution instead of copying.  To
eliminate \scheme|match-a|, we have enhanced the tagging scheme for
representing formulas.

Both of these transformations are broadly applicable. When
\scheme|match-a| is used to handle overlapping clauses, a carefully
crafted tagging scheme can often be used to eliminate
overlapping. When terms must be copied, substitution can often be used
instead of \scheme|copy-termo|---in the case of \alphatap, we use a
combination of nominal unification and substitution.


%% LyX 2.1.4 created this file.  For more info, see http://www.lyx.org/.
%% Do not edit unless you really know what you are doing.
\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{textcomp}
\usepackage{amsmath}
\usepackage{esint}

\makeatletter

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% LyX specific LaTeX commands.
%% Because html converters don't know tabularnewline
\providecommand{\tabularnewline}{\\}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% User specified LaTeX commands.
\usepackage{babel}

\makeatother

\usepackage{babel}
\begin{document}

\title{Understanding the Rational Approximation of the Exponential Integrator
(REXI)}

\author{Martin Schreiber <M.Schreiber@exeter.ac.uk>\\
 Pedro S. Peixoto <pedrosp@ime.usp.br>}

\maketitle
This document serves as the basis for implementing the Rational approximation
of the EXponential Integrator (REXI). Here, we purely focus on the
linear part of the shallow-water equations (SWE) and show the different
steps to approximate solving this linear part with an exponential
integrator. This paper mainly summarises previous work on REXI.

\section{Problem formulation}

We use linearised shallow water equations (SWE) with respect to a
rest state with mean water depth of $\eta_{0}$ and defined for perturbations
of height $\eta$ (see \cite{Schreiber:Formulations of the shallow-water equations}).
The linear operator ($L$) may be written as

\[
L(U):=\left(\begin{array}{ccc}
0 & -\eta_{0}\partial_{x} & -\eta_{0}\partial_{y}\\
-g\partial_{x} & 0 & f\\
-g\partial_{y} & -f & 0
\end{array}\right)U
\]
where $U:=(\eta,u,v)^{T}$. Here, we neglect all non-linear terms
and consider $f$ constant (f-plane approximation). The time evolution
of the PDE, with the subscript $t$ denoting the derivative in time,
is given by

\[
U_{t}=L(U).
\]
It is further worth noting, that this system describes an oscillatory
system (2D wave equation), hence the operator $L$ is hyperbolic and
has imaginary eigenvalues. Linear initial value differential problems
are well known to be solvable with exponential integrators for arbitrary
time step sizes via
\[
U(t)=e^{Lt}U(0).
\]
see e.g. \cite{Moler:Nineteen Dubious Ways to Compute the Exponential of a Matrix}.
However, this is typically quite expensive to compute and analytic
solutions only exist for some simplified system of equations, see
e.g. \cite{Schreiber:Formulations of the shallow-water equations}
for f-plane shallow-water equations. These exponential integrators
can be approximated with rational functions and this paper is on giving
insight into this approximation.

\section{1D rational approximation}

Terry et al. \cite{Terry:High-order time-parallel approximation of evolution operators}
developed a rational approximation of the exponential integrator.
First, we like to get more insight into it with a one-dimensional
formulation before applying REXI to a rational approximation of a
linear operator. Our main target is to find an approximation of an
operator with a \emph{complex exponential shape}, in our case $e^{ix}$,
which (in one-dimension) is given as a function $f(x)$. We will end
up in an approximation given by the following rational approximation:

\[
e^{ix}\approx\sum_{n=-N}^{N}Re\left(\frac{\beta_{n}}{ix+\alpha_{n}}\right)
\]
with complex coefficients $\alpha_{n}$ and $\beta_{n}$. We point
out that the coefficients $\alpha_{n}$ will always have non zero
real part, so no singularity occurs with the rational function.

\subsection{Step A: Approximation of solution space}

First, we assume that we can use Gaussian curves as basis functions
for our approximation. So first we find an approximation of one of
our underlying Gaussian basis function

\[
\psi_{h}(x):=(4\pi)^{-\frac{1}{2}}e^{-x^{2}/(4h^{2})}
\]

In this formulation, $h$ can be interpreted as the horizontal ``stretching''
of the basis function. Note the similarities to the Gaussian distribution,
but by dropping certain parts of the vertical scaling as it is required
for probability distributions. We can now approximate our function
$f(x)$ with a superposition of basis functions $\psi_{h}(x)$ by

\[
f(x)\approx\sum_{m=-M}^{M}b_{m}\psi_{h}(x+mh)
\]
with $M$ controlling the interval of approximation (\textasciitilde{}size
of ``domain of interest'') and $h$ will be related to the accuracy
of integration (\textasciitilde{}resolution in ``domain of interest'').
We choose $h$ small enough so that the support of the Fourier transform
of $f$ is mainly localised within $[-1/(2h),1/(2h)]$, i.e. almost
zero outside this interval. $M$ is chosen such that the approximation
will be adequate in the interval $|x|<Mh$.

To compute the coefficients $b_{m}$, we rewrite the previous equation
in Fourier space with
\[
\frac{\hat{f}(\xi)}{\hat{\psi_{h}}(\xi)}=\sum_{m=\infty}^{\infty}b_{m}e^{2\pi imh\xi},
\]
where the $\hat{\cdot}$ symbols indicate the Fourier transforms of
the respective functions. The $b_{m}$ are now the Fourier coefficients
of the series for the function $\frac{\hat{f}(\xi)}{\hat{\psi_{h}}(\xi)}$
and can be calculated as (see \cite{Terry:High-order time-parallel approximation of evolution operators},
page 11),
\[
b_{m}=h\intop_{-\frac{1}{2h}}^{\frac{1}{2h}}e^{-2\pi imh\xi}\frac{\hat{f}(\xi)}{\hat{\psi_{h}}(\xi)}d\xi,
\]
for $m$ integer and $1/h$ defines the periodicity of the trigonometric
basis function.

Since we are interested in approximating $f(x)=e^{ix}$, we can simplify
the equation by using the response in frequency space $\hat{f}(\xi)=\delta(\xi-\frac{1}{2\pi})$,
where here $\delta$ is the Dirac distribution, and

\[
b_{m}=h\,e^{-imh}\hat{\psi_{h}}\left(\frac{1}{2\pi}\right)^{-1}.
\]
The Fourier transform of the Gaussian function is well known and given
by

\begin{eqnarray*}
\hat{\psi_{h}}(\xi) & = & \intop_{-\infty}^{\infty}\frac{1}{\sqrt{4\pi}}e^{-\left(\frac{x}{2h}\right)^{2}}e^{-2\pi ix\xi}dx\\
 & = & \frac{1}{\sqrt{4\pi}}\intop_{-\infty}^{\infty}e^{-\left(\left(\frac{x}{2h}\right)^{2}+2\pi ix\xi+(2h\pi i\xi)^{2}-(2h\pi i\xi)^{2}\right)}dx\\
 & = & \frac{1}{\sqrt{4\pi}}e^{-(2h\pi\xi)^{2}}\intop_{-\infty}^{\infty}e^{-\left(\frac{x}{2h}+2h\pi i\xi\right)^{2}}dx\\
 & = & \frac{1}{\sqrt{4\pi}}e^{-(2h\pi\xi)^{2}}\intop_{-\infty}^{\infty}e^{-\left(\frac{x}{2h}\right)^{2}}dx\\
 & = & he^{-(2h\pi\xi)^{2}}
\end{eqnarray*}
where we used that $\intop_{-\infty}^{\infty}e^{-\left(\frac{x}{2h}\right)^{2}}dx=h\,\sqrt{4\pi}$
and completed squares in the exponential term. For the case $\xi=\frac{1}{2\pi}$,
we get

\[
\hat{\psi}_{h}\left(\frac{1}{2\pi}\right)=h\,e^{-h^{2}}.
\]
Finally, one can obtain the equation

\[
b_{m}=h\,e^{-imh}\frac{1}{h\,e^{-h^{2}}}=e^{-imh}e^{h^{2}}
\]
to compute the coefficients $b_{m}$ for $f(x)=e^{ix}$.

\subsection{Step B: Approximation of basis function}

The second step is the approximation of the basis function $\psi_{h}(x)$
itself with a rational approximation, see \cite{Damle:Near optimal rational approximations of large data sets}.
Our basis function is given by

\[
\psi_{h}(x):=(4\pi)^{-\frac{1}{2}}e^{-x^{2}/(4h^{2})}
\]
and a close-to-optimal approximation of $\psi_{1}(x)$ with a sum
of rational functions is given by

\[
\psi_{1}(x)\approx Re\left(\sum_{l=-K}^{K}\frac{a_{l}}{ix+(\mu+i\,l)}\right)
\]
with the $\mu$ and $a_{l}$ given in \cite{Damle:Near optimal rational approximations of large data sets},
Table 1. We can generalise this approximation to arbitrary chosen
$h$ via
\[
\psi_{h}(x)\approx Re\left(\sum_{l=-K}^{K}\frac{a_{l}}{i\frac{x}{h}+(\mu+i\,l)}\right).
\]
The theory of how these coefficients are calculated are in \cite{Damle:Near optimal rational approximations of large data sets}
and will not be described here. Therefore, we assume that the coefficients
$a_{l}$ are given.

\subsection{Step C: Approximation of the approximation}

We then combine the approximation (B) into the approximation (A),
yielding the approximation $\tilde{f}$ for $f$ given by
\[
\tilde{f}(x)=\sum_{m=-M}^{M}b_{m}\psi_{h}(x+mh)=\sum_{m=-M}^{M}b_{m}Re\left(\sum_{l=-K}^{K}\frac{a_{l}}{i\frac{x+mh}{h}+(\mu+i\,l)}\right)
\]

\[
=\sum_{m=-M}^{M}b_{m}\sum_{k=-K}^{K}Re\left(\frac{ha_{l}}{ix+h(\mu+i(m+l))}\right).
\]

We further like to simplify this equation and we observe, that for
$n:=m+k$, the denominator is equal. We can hence express parts of
the denomiator in terms of $n:=m+k$ by

\begin{equation}
\alpha_{n}:=s(\mu+in).\label{eq:alpha}
\end{equation}
Now, we merge the $b_{m}$ and $a_{l}$ coefficients and first have
a look at the $b_{m}$ which is complex valued. We observe the following
property: Assuming that we want to compute the real value of $f(x)$,
only the real value of $b_{m}$ has to be merged with the sum, since
the imaginary component would be dropped afterwards. This allows us
to move the $Re(b_{m})$ values inside the $\sum_{K}$:

\[
Re(f(x)):=Re\left(\sum_{m=-M}^{M}\,\,\sum_{k=-K}^{K}\frac{Re(b_{m})\,ha_{l}}{ix+h(\mu+i(m+k))}\right).
\]
Now we can collect all nominators with equivalent denominator (if
$n=m+k$ and by using $\delta$ as the Kronecker delta), yielding

\begin{equation}
\beta_{n}^{Re}:=h\sum_{m=-M\,}^{M}\sum_{k=-K}^{K}Re(b_{m})ha_{l}\delta(n,\,m+k)\label{eq:beta_re}
\end{equation}
for real values $f(x)$ and

\begin{equation}
\beta_{n}^{Im}:=h\sum_{m=-M\,}^{M}\sum_{k=-K}^{K}Im(b_{m})a_{l}\delta(n,\,m+k)\label{eq:beta_im}
\end{equation}
for complex values of $f(x)$. Note, that the complexity of this operation
is $O(N\,K)$, which is negligible for small $N$ and $K$. This can
be optimized by using $L_{1}=max(-K,n-M)$ and $L_{2}=min(K,n+M)$
(see \cite{Terry:High-order time-parallel approximation of evolution operators})
and we can compute $\beta_{n}^{Re}$ with

\begin{equation}
\beta_{n}^{Re}:=h\sum_{k=L_{1}}^{L_{2}}Re(b_{n-k})a_{k},\label{eq:beta_re_fast}
\end{equation}
and $\beta_{n}^{Im}$ correspondingly.

Finally, we get the REXI approximation

\[
e^{ix}\approx\sum_{n=-N}^{N}Re\left(\frac{\beta_{n}^{Re}}{ix+\alpha_{n}}\right)+i\,Re\left(\frac{\beta_{n}^{Im}}{ix+\alpha_{n}}\right).
\]

\subsection{Example coefficents}

For a better understanding and discussion of the poles, we provide
some explanatory coefficients for $\alpha_{n}$ and $\beta_{n}^{Re}$
computed with $M:=2$ and $h:=0.2$:

{\scriptsize{}}%
\begin{tabular}{|c|c|c|}
\hline
$n$ & {\scriptsize{}$\alpha_{n}$} & {\scriptsize{}$\beta_{n}$}\tabularnewline
\hline
\hline
{\scriptsize{}-13} & {\scriptsize{}(-0.863064302175, -2.6)} & {\scriptsize{}(-2.0794560075645e-08,5.312368394177e-09)}\tabularnewline
\hline
{\scriptsize{}-12} & {\scriptsize{}(-0.863064302175, -2.4)} & {\scriptsize{}(-1.8562925598646e-08,-1.6892470811809e-07)}\tabularnewline
\hline
{\scriptsize{}-11} & {\scriptsize{}(-0.863064302175, -2.2)} & {\scriptsize{}(6.8570271350932e-07,-4.4377515257134e-08)}\tabularnewline
\hline
{\scriptsize{}-10} & {\scriptsize{}(-0.863064302175, -2)} & {\scriptsize{}(1.9470768200785e-07,2.1186231739561e-06)}\tabularnewline
\hline
{\scriptsize{}-9} & {\scriptsize{}(-0.863064302175, -1.8)} & {\scriptsize{}(3.037169144916e-06,-3.8007524015554e-06)}\tabularnewline
\hline
{\scriptsize{}-8} & {\scriptsize{}(-0.863064302175, -1.6)} & {\scriptsize{}(-0.00020292956274934,-9.4793805592883e-05)}\tabularnewline
\hline
{\scriptsize{}-7} & {\scriptsize{}(-0.863064302175, -1.4)} & {\scriptsize{}(0.00051562027155282,0.0033198141762956)}\tabularnewline
\hline
{\scriptsize{}-6} & {\scriptsize{}(-0.863064302175, -1.2)} & {\scriptsize{}(0.023802856324805,-0.020097812439831)}\tabularnewline
\hline
{\scriptsize{}-5} & {\scriptsize{}(-0.863064302175, -1)} & {\scriptsize{}(-0.16210306892042,-0.057527918763957)}\tabularnewline
\hline
{\scriptsize{}-4} & {\scriptsize{}(-0.863064302175, -0.8)} & {\scriptsize{}(0.083936569694558,0.55379453117192)}\tabularnewline
\hline
{\scriptsize{}-3} & {\scriptsize{}(-0.863064302175, -0.6)} & {\scriptsize{}(0.87683903065806,-0.58136186212318)}\tabularnewline
\hline
{\scriptsize{}-2} & {\scriptsize{}(-0.863064302175, -0.4)} & {\scriptsize{}(-0.87618099667542,-0.6444132979014)}\tabularnewline
\hline
{\scriptsize{}-1} & {\scriptsize{}(-0.863064302175, -0.2)} & {\scriptsize{}(-0.2112750856805,0.51693268636776)}\tabularnewline
\hline
{\scriptsize{}0} & \textbf{\scriptsize{}(-0.863064302175, 0)} & \textbf{\scriptsize{}(0.21113064943379,1.1012434042446e-07)}\tabularnewline
\hline
{\scriptsize{}1} & {\scriptsize{}(-0.863064302175, 0.2)} & {\scriptsize{}(-0.2112752777559,-0.51693263772868)}\tabularnewline
\hline
{\scriptsize{}2} & {\scriptsize{}(-0.863064302175, 0.4)} & {\scriptsize{}(-0.87618105783081,0.6444131761443)}\tabularnewline
\hline
{\scriptsize{}3} & {\scriptsize{}(-0.863064302175, 0.6)} & {\scriptsize{}(0.87683907406497,0.58136183517238)}\tabularnewline
\hline
{\scriptsize{}4} & {\scriptsize{}(-0.863064302175, 0.8)} & {\scriptsize{}(0.083936534477108,-0.55379454106338)}\tabularnewline
\hline
{\scriptsize{}5} & {\scriptsize{}(-0.863064302175, 1)} & {\scriptsize{}(-0.16210304313401,0.057527824955638)}\tabularnewline
\hline
{\scriptsize{}6} & {\scriptsize{}(-0.863064302175, 1.2)} & {\scriptsize{}(0.023802980792584,0.020097827969804)}\tabularnewline
\hline
{\scriptsize{}7} & {\scriptsize{}(-0.863064302175, 1.4)} & {\scriptsize{}(0.00051562077173168,-0.0033196934926057)}\tabularnewline
\hline
{\scriptsize{}8} & {\scriptsize{}(-0.863064302175, 1.6)} & {\scriptsize{}(-0.0002030221163996,9.4802957184526e-05)}\tabularnewline
\hline
{\scriptsize{}9} & {\scriptsize{}(-0.863064302175, 1.8)} & {\scriptsize{}(3.0281700967037e-06,3.7434363774526e-06)}\tabularnewline
\hline
{\scriptsize{}10} & {\scriptsize{}(-0.863064302175, 2)} & {\scriptsize{}(2.2311216999616e-07,-2.1234907990132e-06)}\tabularnewline
\hline
{\scriptsize{}11} & {\scriptsize{}(-0.863064302175, 2.2)} & {\scriptsize{}(6.871098037128e-07,5.5123982746463e-08)}\tabularnewline
\hline
{\scriptsize{}12} & {\scriptsize{}(-0.863064302175, 2.4)} & {\scriptsize{}(-2.1322288893395e-08,1.6899352278552e-07)}\tabularnewline
\hline
{\scriptsize{}13} & {\scriptsize{}(-0.863064302175, 2.6)} & {\scriptsize{}(-2.0738399377275e-08,-5.6642992624128e-09)}\tabularnewline
\hline
\end{tabular}{\scriptsize \par}

\section{REXI on linear operators}

In this section, we investigate the linear operator $L$ with the
rational approximation.

\subsection{Reducing number of computations for $L$}

Note the property (see Sec. 3.3 in \cite{Terry:High-order time-parallel approximation of evolution operators})
for the $\alpha_{n}$ and $\beta_{n}$: There is an anti-symmetry
around the central pole with
\begin{equation}
\alpha_{-n}=\bar{\alpha}_{+n}\label{eq:alpha_conjugate_symmetry}
\end{equation}
and
\begin{equation}
\beta_{-n}=\bar{\beta}_{+n}\label{eq:beta_conjugate_symmetry}
\end{equation}
In particular, with $Im(\alpha_{0})=Im(\beta_{0})=0$, there is a
zero imaginary number for the central pole.

Furthermore, it holds that
\[
\overline{(L+\alpha)^{-1}U(0)}=(L+\overline{\alpha})^{-1}U(0)
\]
with the overbar denoting the complex conjugate. We can then reformulate
the approximation
\begin{eqnarray*}
e^{\tau L} & \approx & \sum_{n=-N}^{N}Re\left(\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}\right)\\
 & = & \sum_{n=-N}^{-1}Re\left(\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}\right)+Re\left(\frac{\beta_{0}^{Re}}{\tau L+\alpha_{0}}\right)+\sum_{n=-N}^{-1}Re\left(\frac{\beta_{N+n+1}^{Re}}{\tau L+\alpha_{N+n+1}}\right)
\end{eqnarray*}
and using the properties \eqref{eq:alpha_conjugate_symmetry} and
\eqref{eq:beta_conjugate_symmetry}, we can write this as
\begin{align*}
\sum_{n=-N}^{-1}Re\left(\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}\right)+Re\left(\frac{\beta_{0}^{Re}}{\tau L+\alpha_{0}}\right)+\sum_{n=-N}^{-1}Re\left(\frac{\overline{\beta_{N}^{Re}}}{\tau L+\overline{\alpha_{N}}}\right)\\
=\sum_{n=-N}^{-1}\left(Re\left(\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}+\overline{\left(\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}\right)}\right)\right)+Re\left(\frac{\beta_{0}^{Re}}{\tau L+\alpha_{0}}\right).
\end{align*}
Since the imaginary parts cancel out for $Re(a+\overline{a})=a+\overline{a}$
and with $\alpha_{0}$ and $\beta_{0}$ being only real-valued, we
can simplify the equation to
\[
e^{\tau L}\approx\sum_{n=-N}^{N}Re\left(\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}\right)=\sum_{n=-N}^{-1}\left(2\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}\right)+\frac{\beta_{0}^{Re}}{\tau L+\alpha_{0}}.
\]
This allows us to reduce the computational amount almost by a factor
of two for solving $(L+\alpha)^{-1}$ giving the real valued solution
\[
U(\tau)=e^{\tau L}U(0)\approx\sum_{n=0}^{N}\gamma_{n}^{Re}\left(\tau L+\alpha_{n}\right)^{-1}U(0)
\]
with
\[
\gamma_{n}:=\begin{cases}
\begin{array}{c}
\beta_{0}\\
2\beta_{n}
\end{array} & \begin{array}{c}
for\,n=0\\
else
\end{array}\end{cases}
\]

\subsection{Matrix exponential}

\label{sec:mat_exp} We would like to apply REXI to a formulation
such as
\[
U(t)=e^{\tau L}U(0).
\]
To see the relationship between the approximation of $e^{ix}$ with
$e^{\tau L}$ we assume that $L$ is skew hermitian and therefore
has only purely imaginary eigenvalues, and maybe decomposed as $\Sigma\Lambda\Sigma^{H}$,
yielding
\begin{equation}
e^{\tau L}=\sum_{k=0}^{\infty}\frac{(\tau L)^{k}}{k!}=\sum_{k=0}^{\infty}\frac{\tau\Sigma\Lambda^{k}\Sigma^{H}}{k!}=\Sigma\left(\sum_{k=0}^{\infty}\frac{(\tau\Lambda)^{k}}{k!}\right)\Sigma^{H}=\Sigma e^{\tau\Lambda}\Sigma^{H},\label{eq:expL}
\end{equation}
where we used the orthonormality of $\Sigma$ to cancel it out from
the summation, and

\[
e^{\tau\Lambda}=\left(\begin{array}{ccc}
...\\
 & e^{i\lambda_{j}\tau}\\
 &  & ...
\end{array}\right)
\]
where we have explicitly detached the imaginary unit from the eigenvalues,
therefore $\lambda_{n}$ are assumed real. Since $e^{\tau\Lambda}$
is diagonal, it can be eigenvalue-wise approximated in the same way
as in $e^{ix}$ with REXI.

Although $L$ has imaginary eigenvalues, we wish to evaluate the $e^{\tau L}U(0)$,
which is real valued. Therefore, we will use the real approximation
of $e^{ix}$
\begin{equation}
exp(\tau L)\approx Re\left(\sum_{n=-N}^{N}\beta_{n}(\tau L+\alpha_{n})^{-1}\right),\label{eq:rexi}
\end{equation}
where $\beta_{n}$ is given by equation \eqref{eq:beta_re} and $\alpha_{n}$
by equation \eqref{eq:alpha}.

Given the linear operator $L$ to be applied on $U(0)$, we continue
to show the relation of the linear operator to the REXI approximation
of the real term
\[
e^{\tau L}\approx\sum_{n=-N}^{N}Re\left(\frac{\beta_{n}^{Re}}{\tau L+\alpha_{n}}\right)
\]
as we use it later on. Next, we rewrite $L$ with our EV decomposition
as
\begin{eqnarray*}
e^{\tau L} & \approx & \sum_{n=-N}^{N}Re\left(\beta_{n}^{Re}(\tau\Sigma\Lambda\Sigma^{-1}+\alpha_{n}I)^{-1}\right)\\
 & = & \sum_{n=-N}^{N}Re\left(\beta_{n}^{Re}(\Sigma(\tau\Lambda\Sigma^{-1}+\alpha_{n}\Sigma^{-1}))^{-1}\right)\\
 & = & \sum_{n=-N}^{N}Re\left(\beta_{n}^{Re}(\Sigma^{-1}(\tau\Lambda+\alpha_{n}\Sigma^{-1}\Sigma)){}^{-1}\Sigma^{-1}\right)\\
 & = & \sum_{n=-N}^{N}Re\left(\beta_{n}^{Re}\Sigma(\tau\Lambda+\alpha_{n})^{-1}\Sigma^{-1}\right)\\
 & = & \approx\Sigma e^{\tau\Lambda}\Sigma^{H}
\end{eqnarray*}
Note, that we use the same REXI approximation for different $\lambda_{j}$.
Hence, the approximation has to be sufficiently accurate over the
entire range of all Eigenvalues$\lambda_{j}$ which is what we use
the approximation for.

\subsection{Choosing $h$ and $M$}

Some important points about the choice of $M$ and $h$ have to be
made at this point. We know that $e^{ix}$ is accurately approximated
with REXI for the interval $|x|<hM$, where $h$ is chosen small enough
to obtain a good approximation in step (A), and $M$ will define the
interval size and number of approximation points. In the matrix case,
$M$ has to be chosen so that $hM>t\bar{\lambda}$, where $\bar{\lambda}=\max_{n}|\lambda_{n}|$,
in order to capture all wavelengths of $L$. In other other words,
$hM$ need to be set to capture the fastest wave. Note that if this
is used as a time stepping method, with time step $t=\tau$, then,
the larger the timestep, the larger $M$ will be. Exact evaluations
of the choices for $h$ and $M$ may be done based on equation (3.6)
of \cite{Terry:High-order time-parallel approximation of evolution operators}.

TODO: We have to investigate this by far in more depth. Can we possibly
use a standard time step restriction of the linear equations?

\subsection{Handling $\tau$ in REXI\label{sub:Handling-tau-in-REXI}}

We reformulate the REXI approximation scheme given by

\[
(\tau L+\alpha)^{-1}U(\tau)=U(0)
\]
and by factoring $\tau$ out, yielding

\[
(L+\frac{\alpha}{\tau})^{-1}U(\tau)\tau^{-1}=U(0)
\]
So instead of solving for $U(\tau)$, we are solving for $U^{\tau}(\tau):=U(\tau)\tau^{-1}$
as well as $\alpha^{\tau}:=\frac{\alpha}{\tau}$.

To summarize, we have to solve the system of equations given by

\begin{equation}
(L+\alpha^{\tau})^{-1}U^{\tau}(\tau)=U(0)\label{eq:unit_rexi_timestep}
\end{equation}
with $U(0)$ the initial conditions. For sake of simplicity, we stick
to the formulation without the $\tau$ notation.

\subsection{Computing inverse of $(L+\alpha)^{-1}$}

For computing the inverse, arbitrary solvers can be used. However
we like to note, that $\alpha$ is a complex number. Hence, requiring
solvers with support for solving in complex space.

We wish to solve the differential problem for each time step
\[
(-L+\alpha)U=U(0)
\]
so that $U=(-L+\alpha)^{-1}U_{0}$. Note, the change in sign before
$L$ for convenience which has to be accounted for afterwards. We
will do this converting the problem into and elliptic equation.\textbf{}\\

First, lets expand the equations with the definition of $L$,
\begin{eqnarray}
fv+g\eta_{x}+\alpha u & = & u_{0},\label{eq:mom_u}\\
fu+g\eta_{y}+\alpha v & = & v_{0},\label{eq:mom_v}\\
\bar{\eta}(u_{x}+v_{y})+\alpha\eta & = & \eta_{0}.\label{eq:mass}
\end{eqnarray}

Let $f$ be constant (f-plane approximation), $\delta:=u_{x}+v_{y}$
be the wind divergence, $\zeta:=v_{x}-u_{y}$ be the wind (relative)
vorticity and $\Delta\eta:=\eta_{xx}+\eta_{yy}$ the Laplacian of
the fluid depth. We will re-write the problem in a divergence-vorticity
formulation by taking 2 steps. First, sum the $\partial_{x}$ of equation
\eqref{eq:mom_u} and the $\partial_{y}$ of equation \eqref{eq:mom_v},
yielding
\begin{equation}
-f\zeta+g\Delta\eta+\alpha\delta=\delta_{0}.\label{eq:zeta}
\end{equation}
Then subtract the $\partial_{y}$ of equation \eqref{eq:mom_u} from
the $\partial_{x}$ of equation \eqref{eq:mom_v}, yielding
\begin{equation}
f\delta+\alpha\zeta=\zeta_{0}.\label{eq:delta}
\end{equation}
Using equation \eqref{eq:zeta} in equation \eqref{eq:delta} gives
us

\[
\delta=-\frac{1}{f^{2}+\alpha^{2}}\left(\alpha g\Delta\eta-\alpha\delta_{0}-f\zeta_{0}\right).
\]
Finally, substituting $\delta$ in equation \eqref{eq:mass}, that
reads $\bar{\eta}\delta+\alpha\eta=\eta_{0}$, results in
\[
-\frac{\bar{\eta}}{f^{2}+\alpha^{2}}\left(\alpha g\Delta\eta-\alpha\delta_{0}-f\zeta_{0}\right)+\alpha\eta=\eta_{0},
\]
which may be simplified into the elliptic equation by multiplying
by $-\frac{f^{2}+\alpha^{2}}{\bar{\eta}\alpha g}$

\begin{equation}
\Delta\eta-\kappa^{2}\eta=r_{0}\label{eq:ellip}
\end{equation}
where
\[
\kappa^{2}=\frac{f^{2}+\alpha^{2}}{\bar{\eta}g}
\]
and
\[
r_{0}=-\frac{\kappa^{2}}{\alpha}\eta_{0}+\frac{1}{g}\delta_{0}+\frac{f}{\alpha g}\zeta_{0}.
\]

Multiplying the equation \eqref{eq:ellip}by $-g\bar{\eta}$ gives

\[
((\alpha^{2}+f^{2})-g\bar{\eta}\Delta)\eta=\frac{f^{2}+\alpha^{2}}{\alpha}\eta_{0}-\bar{\eta}\delta_{0}-\frac{f\bar{\eta}}{\alpha}\zeta_{0}
\]

We can compute$\eta$ with any elliptic solver. Special attention
has to be given to the LHS of the form $(\gamma-\Delta)\eta$ for
a spectral solver and this is the reason for this brief excursion.
We use $\tilde{}$ to annotate an operator or quantity to be given
in spectral space. Then, we factor $\tilde{\eta}$ in and write form
of the LHS in spectral space with the identity matrix $I$ as
\[
(\gamma I\tilde{\eta}-\tilde{\Delta}\tilde{\eta})=(\gamma I-\tilde{\Delta})\tilde{\eta}
\]
Here, the minus operator in spectral space has to be applied on all
elements of $\tilde{\Delta}$ and not only to the $0^{th}$ mode as
it is typically the case for adding a constant in spectral space.

We need to retrieve the velocities by solving the $2\times2$ system
formed by equations \eqref{eq:mom_u} and \eqref{eq:mom_v}, which
reads
\[
A_{\alpha}U=U_{0}-g\nabla\eta
\]
with
\[
A_{\alpha}=\left(\begin{matrix}\alpha & -f\\
f & \alpha
\end{matrix}\right).
\]
The solution is
\[
U=A_{\alpha}^{-1}(U_{0}-g\nabla\eta)
\]
where
\[
A_{\alpha}^{-1}=\frac{1}{f^{2}+\alpha^{2}}\left(\begin{matrix}\alpha & f\\
-f & \alpha
\end{matrix}\right)
\]
Finally, since we computed $(-L+\alpha)^{-1}$, we also have to invert
the sign of the computed solution $U$. Alternatively, we can change
the signs of $\alpha_{n}$ and $\beta_{n}$.

\subsection{Interpretation of $\tau$}

We like to close this section with a brief discussion of $\tau$ by
having a look on the REXI reformulation

\[
\left(L-\frac{\alpha}{\tau}\right)^{-1}U\tau^{-1}=U_{0}
\]
We see, that for an increasing $\tau$, hence an integration in time
over a larger time period, the poles given by $\alpha$ are getting
closer. This can possibly lead to a loss in accuracy for the data
sampled by the outer poles $\alpha_{-N}$ and $\alpha_{N}$. Therefore,
the number $N$ of poles is expected to scale linearly with the size
of the coarse time step,
\[
|N|\propto\tau.
\]

Indeed, we saw in section \ref{sec:mat_exp} that for larger $\tau$,
$M$ needs to be larger.

\subsection{Numerical Dispersion}

We continue to (try to) analyze possible dispersion effects of the
REXI approximation. Here, we assume that we use an accurate solver
to compute $(L-\alpha_{i})^{-1}$. We are interested in answering
the question which error is introduced when not using enough terms
in the approximation.

According to \ref{sec:mat_exp}, we can decompose the linear operator
into EVals and EVects and write the approximation in the following
way:

\[
e^{\tau L}\approx\Sigma\left[\sum_{n=0}^{N}Re\left(\gamma_{n}(\tau\Lambda+\alpha_{n})^{-1}\right)\right]\Sigma^{H}.
\]
Alternatively without solving a linear operator, one can also think
directly about approximating a set of exponents given by $\Lambda$
with

\[
e^{\tau\Lambda}\approx\sum_{n=0}^{N}\left(\gamma_{n}(\tau\Lambda+\alpha_{n})^{-1}\right).
\]
We can then use the spectral representation of the solution
\[
e^{\tau\hat{L}}=\hat{\Sigma}e^{\tau\hat{\Lambda}}\hat{\Sigma}^{H}
\]
with $\hat{L}=\hat{\Sigma}\hat{\Lambda}$the spectral EValue/EVector
decomposition. We can then approximate the exponential $e^{\tau\Lambda}$
by

\[
e^{\tau\hat{L}}\approx\hat{\Sigma}\sum_{n=0}^{N}\left(\gamma_{n}(\tau\hat{\Lambda}+\alpha_{n})^{-1}\right)\hat{\Sigma}^{H}
\]
For selected modes $(k_{1},k_{2})$ at a specific point in time $\tau=\sigma$,
the solution and operators in spectral space are then given by

\begin{align*}
\hat{U}(k_{1},k_{2},\tau)= & e^{\tau\hat{L}_{k_{1},k_{2}}}\hat{U}(k_{1},k_{2},0)\\
\approx & \hat{\Sigma}_{k_{1},k_{2}}\left[\sum_{n=-N}^{N}Re\left(\beta_{n}^{Re}(\tau\hat{\Lambda}_{k_{1},k_{2}}+\alpha_{n})^{-1}\right)\right]\hat{\Sigma}_{k_{1},k_{2}}^{H}\hat{U}(k_{1},k_{2},0).
\end{align*}
Case A) For $k_{1}\neq0$ and $k_{2}\neq0$, we know that $\hat{\Lambda}$contains
eigenvalues

\begin{tabular}{rcl}
Vortical mode: &  & $\omega_{0}=0$\tabularnewline
Gravitational modes: &  & $\omega_{\pm1}=\pm\sqrt{4\pi^{2}(\eta_{0}gk_{1}^{2}+\eta_{0}gk_{2}^{2})+f^{2}}$\tabularnewline
\end{tabular}\\
Case B) For $k_{1}=k_{2}=0$, we get the eigenvalues

\begin{tabular}{rcl}
Vortical mode: &  & $\omega_{0}=0$\tabularnewline
Gravitational modes: &  & $\omega_{\pm1}=\pm f$\tabularnewline
\end{tabular}\\
These eigenvalues describe the frequency in $\hat{\Lambda}$ which
we approximate. As soon as this frequency cannot be approximated anymore
with REXI, this results in errors in the dispersion of this particular
frequency.

With the computational requirements (M) of REXI being related to the
fastest waves, we can identify the requirement $M\sim\sqrt{\eta_{0}g}$
and $M\sim f$.

\emph{Vortical modes}: Since the vortical modes are always zero, we
do not expect any (analytical) errors in these modes. However, due
to discretization, errors can accumulate and show up.

\emph{Gravitational modes: }We can observe that errors in the frequency
$\omega_{\pm1}$ in case A area generated. The dominating frequency
is only dependent on $\eta_{0}$ and $g$ in case of
\[
4\pi^{2}(\eta_{0}gk_{1}^{2}+\eta_{0}gk_{2}^{2})=4\pi^{2}\eta_{0}g|k|^{2}>f^{2}
\]
and $f$ otherwise. The factor $4\pi^{2}$ shows up since our domain
is on $\Omega=[0;1]^{2}$. The Coriolis frequency $f$ is also independent
of the spatial frequency, hence also the resolution.

Furthermore, we can observe a relation to the Rossby radius given
by
\[
L_{R}:=4\pi\text{\texttwosuperior}\frac{\sqrt{g\eta_{0}}}{f}
\]
which describes at which scale at the Coriolis effect also strongly
contributes to the simulation results compared to the gravitational
and height values.

\subsection{Complex solver}

We are interested in solving a system of the form,
\[
(-L+\alpha)U=U_{0},
\]
where $\alpha$ is a complex number, therefore we must allow complex
solutions for $U$. The system can be transformed to have only real
arithmetic in the following way.

First we decompose $U$ into its real and imaginary parts, $U=U^{r}+iU^{i}$,
and allow $U_{0}$ to be decomposed in the same way. Although $\alpha$
can be a general complex number, it real part is always constant in
REXI, given by $\mu$. We will therefore write $\alpha=\mu+i\alpha^{i}$,
and we can absorb $\mu$ into $L$ writing $D=-L+\mu I$, where $I$
is the identity matrix. Now
\[
(D+i\alpha^{i}I)(U^{r}+iU^{i})=U_{0}^{r}+iU_{0}^{i},
\]
\[
DU^{r}-\alpha^{i}U^{i}+i(\alpha^{i}U^{r}+DU^{i})=U_{0}^{r}+iU_{0}^{i},
\]
therefore
\begin{eqnarray}
DU^{r}-\alpha^{i}U^{i} & = & U_{0}^{r}\\
\alpha^{i}U^{r}+DU^{i} & = & U_{0}^{i},
\end{eqnarray}
which in matrix notation gives
\[
\left(\begin{matrix}D & -\alpha^{i}I\\
\alpha^{i}I & D
\end{matrix}\right)\left(\begin{matrix}U^{r}\\
U^{i}
\end{matrix}\right)=\left(\begin{matrix}U_{0}^{r}\\
U_{0}^{i}
\end{matrix}\right),
\]
or,
\[
\left(\begin{matrix}-L+\mu I & -\alpha^{i}I\\
\alpha^{i}I & -L+\mu I
\end{matrix}\right)\left(\begin{matrix}U^{r}\\
U^{i}
\end{matrix}\right)=\left(\begin{matrix}U_{0}^{r}\\
U_{0}^{i}
\end{matrix}\right).
\]

A similar approach may be taken with the elliptic equation
\begin{equation}
\Delta\eta+\theta\eta=r_{0},
\end{equation}
where $\eta$, $\theta$ and $r_{0}$ may be complex. The resulting
system is given by
\begin{eqnarray}
\Delta\eta^{r}+\theta^{r}\eta^{r}-\theta^{i}\eta^{i}=r_{0}^{r},\\
\Delta\eta^{i}+\theta^{i}\eta^{r}+\theta^{r}\eta^{i}=r_{0}^{i},
\end{eqnarray}
which can also be written in matrix notation and solved with arbitrary
elliptic equation solvers.

\section{Filtering}

The method described in the previous section is well defined for skew
hermitian $L$. If $L$ is not skew hermitian, the real eigenvalues
might cause the REXI to have absolute values larger than 1, which
can lead to instabilities if used as time stepping method.

To ensure that the REXI is bounded by unit, a filtering process is
proposed in \cite{Terry:High-order time-parallel approximation of evolution operators}.
REXI is prone to exceed unit in the neighborhood of $|t\lambda|\approx hM$,
therefore in the highest frequencies. The idea is to construct a rational
function $S(ix)$ that is approximately $1$ in a smaller interval
$|t\lambda|<hM_{0}$, with $M_{0}<M$, and decays very fast to zero
outside this interval. Then we multiply this filters function to the
original REXI, which will lead to a unit bounded REXI.

Further details of how $S(ix)$ is computed will be added later.

\section{Bringing everything together}

Using the spectral methods (e.g. in SWEET), we can directly solve
the Helmholtz problem for the height in Eq. (\ref{eq:helmhotz}) and
then solver for the velocity in Eqs. (\ref{BROKEN: Ref: eq:elliptic_velo...},\ref{eq:elliptic_velocity_v}).
Note that the Helmholtz problem is in complex space, as $\alpha_{n}$
are complex. This is straightforward with spectral methods. For finite
difference/element methods, the problem can be split into its real
and imaginary parts.

Then, the problem is reduced to computing the REXI as given in Eq.
\eqref{eq:rexi}.

\section{Notes on HPC}
\begin{itemize}
\item The terms in REXI to solve are all independent. Hence, for latency
avoiding, the communication can be interleaved with computations.
\item The iterative solvers are memory bound. Instead of computing $c:=a*b$
for the stencil operations, we could compute $\vec{c}:=a\vec{b}$
with $a$ one coefficient in the stencil. This allows vectorization
over $c$ and $b$ on accelerator cards with strided memory access.
\item It is unknown which method is more efficient to solve the system of
equations:

\begin{itemize}
\item iterative solvers have low memory access,
\item inverting the system and storing it as a sparse matrix allows fast
direct solving but can yield more memory access operations.
\end{itemize}
\item Splitting the solver into real and complex number would store them
consecutively in memory. This has a potential to avoid non-strided
memory access and using the same SIMD operations (Just a rough idea,
TODO: check if this is really the case).
\end{itemize}

\section{Acknowledgements}

Thanks to Terry for the feedback \& discussions!
\begin{thebibliography}{1}
\bibitem{Schreiber:Formulations of the shallow-water equations}Formulations
of the shallow-water equations, M. Schreiber, P. Peixoto et al.

\bibitem{Terry:High-order time-parallel approximation of evolution operators}High-order
time-parallel approximation of evolution operators, T. Haut et al.

\bibitem{Moler:Nineteen Dubious Ways to Compute the Exponential of a Matrix}Nineteen
Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years
Later, Cleve Moler and Charles Van Loan, SIAM review

\bibitem{Damle:Near optimal rational approximations of large data sets}Near
optimal rational approximations of large data sets, Damle, A., Beylkin,
G., Haut, T. S. \& Monzon\end{thebibliography}

\end{document}


%%% Template originaly created by Karol Kozioł (mail@karol-koziol.net) and modified for ShareLaTeX use

\documentclass[a4paper,11pt]{article}

\usepackage{amsmath}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{xcolor}

\usepackage{sansmath}
\renewcommand\familydefault{\sfdefault}
\usepackage{tgheros}

\usepackage{amsmath,amssymb,amsthm,textcomp}
\usepackage{enumerate}
\usepackage{multicol}
\usepackage{tikz}
\usetikzlibrary{shapes, positioning}

\usepackage{geometry}
\geometry{total={210mm,297mm},
left=25mm,right=25mm,%
bindingoffset=0mm, top=20mm,bottom=20mm}

\linespread{1.3}

\newcommand{\linia}{\rule{\linewidth}{0.5pt}}

% custom theorems if needed
\newtheoremstyle{mytheor}
    {1ex}{1ex}{\normalfont}{0pt}{\scshape}{.}{1ex}
    {{\thmname{#1 }}{\thmnumber{#2}}{\thmnote{ (#3)}}}

\theoremstyle{mytheor}
\newtheorem{defi}{Definition}

% my own titles
\makeatletter
\renewcommand{\maketitle}{
\begin{center}
\vspace{2ex}
{\huge \textsc{\@title}}
\vspace{1ex}
\\
\linia\\
\@author \hfill \@date
\vspace{4ex}
\end{center}
}
\makeatother
%%%

% custom footers and headers
\usepackage{fancyhdr}
\pagestyle{fancy}
\lhead{}
\chead{}
\rhead{}
\lfoot{Automatic~Verification Assignment \#2}
\cfoot{}
\rfoot{Page \thepage}
\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{0pt}
%

% all section titles centered and bolded
\usepackage{sectsty}
\allsectionsfont{\bfseries\large}
%
% add section label
\renewcommand\thesection{Problem~\arabic{section}:}
%

%%%----------%%%----------%%%----------%%%----------%%%

\begin{document}

\title{Homework Assignment~\#2}

\author{R02943142 Hsieh, Chiao}

%\date{01/01/2014}

\maketitle

\section{Lattices and Complete Lattices}
Prove that every finite lattice is a complete lattice. Be clear about the cases
of the supremum and the infimum of an empty subset.
\medskip \\
Answer.
\smallskip \\
To prove that every finite lattice is a complete lattice, we have to prove that
there are $\bigvee S$ and $\bigwedge S$ for all $S \subseteq L$ in any given
finite lattice $L$.

First, we can derive $\bigwedge S$ for all nonempty subsets $S$ by following means:
\smallskip\\
Assume $S=\{x_1,x_2,\dots,x_n\}$ is a nonempty subset of a lattice $L$, we derive
$y_n$ by
\begin{align*}
L \ni y_0 &= x_1 \\
L \ni y_1 &= y_0 \wedge x_1 = x_1 \wedge x_1 = x_1 \\
L \ni y_2 &= y_1 \wedge x_2 = x_1 \wedge x_2 \\
L \ni y_3 &= y_2 \wedge x_3 = x_1 \wedge x_2 \wedge x_3 \\
\dots \\
L \ni y_n &= y_{n-1} \wedge x_n = x_1 \wedge x_2 \wedge x_3 \wedge \dots \wedge x_n
\end{align*}
By definition of $\wedge$, we know $y_i \leq x_i$ and $y_i \leq y_{i-1}$
and, therefore, $y_n \leq x_i$ for all $i$ in $[1 \dots n]$. This means
$y_n$ is a lower bound of $S$. Further, we prove $y_n$ is $\bigwedge S$.
\smallskip\\
Given $z \in L$, any lower bound of $S$,
\begin{align*}
y_n \wedge z &= y_{n-1} \wedge (x_n \wedge z) = y_{n-1} \wedge z \\
y_{n-1} \wedge z &= y_{n-2} \wedge (x_{n-1} \wedge z) = y_{n-2} \wedge z \\
\dots \\
y_0 \wedge z     &= x_1 \wedge z = z
\end{align*}
That is, for any lower bound $z$, $z \wedge y_n = z \implies z \leq y_n$;
hence $y_n$ is the greatest lower bound $\bigwedge S$.
Dually we can derive $\bigvee S$ in a similar manner.

Second, when $S = \emptyset$, we have to prove $L$ has a top element,
$\top = \bigvee L = \bigwedge S$ and a bottom element,
$\bot = \bigwedge L = \bigvee S$. The proof is directly derived from
that, since $L \subseteq L$ and $L$ is finite, $\bigvee L$ and $\bigwedge
L$ must exist by the proof in first paragraph.

Combining both paragraphs, we proved that there are $\bigvee S$ and
$\bigwedge S$ for all $S \subseteq L$ in every finite lattice $L$.
Thus, every finite lattice is a complete lattice. Q.E.D.

\section{Complete Partial Orders}
Prove that every complete lattice is a complete partial order.
\medskip \\
Answer.
\smallskip \\
To prove that every complete lattice $CL$ is a complete partial order, it
suffices to show that:
\begin{enumerate}
\item $CL$ has a bottom element $\bot$ and
\item $\bigsqcup D$ exists for each directed subset $D$ of $CL$
\end{enumerate}
The first condition is mentioned in Problem 1 that every complete lattice
has a bottom element. The second condition is trivially fulfilled by the
definition of complete lattice. Since, by definition, $\bigvee S$ exists
for every subset $S \subseteq CL$, $\bigsqcup D$ must exist for directed
subsets $D$ of $CL$. Hence, we finished the proof. Q.E.D.

\section{Continuous Maps}
For a discrete ordered set of your choice, find a self-map on the set
(i.e., a function mapping from the set to itself) that is monotonic
(order-preserving), but not $\sqcup$-continuous. Please state monotonicity
and $\sqcup$-continuity precisely in terms of the chosen ordered set
before presenting the example self-map and explaining why it meets the
requirements.
\medskip \\
Answer.
\smallskip \\
$\mathbb{N}_{\top} = \mathbb{N} \oplus \{\top\}$ is a discrete infinite
(complete partial order) set. The elements in $\mathbb{N}$ follow the
order in natural number, and all numbers are less than $\top$, a manually
introduced top element. \\
We can then define a monotonic self-map $f: \mathbb{N}_{\top} \mapsto \mathbb{N}_{\top}$ s.t.
\begin{equation*}
f(x) = \left\{
  \begin{array}{lr}
    1  & : x \in \mathbb{N} \\
    2  & : x = \top
  \end{array}
\right.
\end{equation*}
The mapping $f$ preserves order since for all $x, y \in \mathbb{N},
x \leq y \implies 1 = f(x) \leq f(y) = 1$, and for all $x \in \mathbb{N},
x \leq \top \implies 1 = f(x) \leq f(\top) = 2$.

The mapping $f$ is not $\sqcup$-continuous. We can disprove
by examining the subset $\mathbb{N} \subseteq \mathbb{N}_{\top}$.
$\mathbb{N}$ is a directed subset because, for any two elements $x, y
\in \mathbb{N}$, we know either $x \leq y$ or $y \leq x$ and, in either
case, $\exists z = x \vee y$ s.t. $z \in \mathbb{N}$ and $z \in \{x,
y\}^u$. However, $f(\bigsqcup \mathbb{N}) = f(\top) = 2 \neq 1 =
\bigsqcup \{1,\dots,1\} = \bigsqcup f(\mathbb{N})$.
Therefore, $f$ is not a $\sqcup$-continuous mapping.

\end{document}