\documentclass[12pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{makeidx}
\usepackage{graphicx}
\usepackage{lmodern}
\usepackage{url}
\usepackage{bm}
\usepackage{color}
\usepackage{booktabs}
\usepackage{float}
\usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm]{geometry}
\author{Mudathir Mahgoub}
\title{Project report}
\begin{document}

\maketitle

\section{Description of PDL solver}
This PDL solver checks the satisfiability of formulas in propositional dynamic logic (PDL).
The input is a kripke frame, which is optional, followed by a formula in PDL. The output is the satisfiability result of this formula which is either \textit{unsat}, \textit{sat} or \textit{unknown}. If the result is \textit{sat}, the solver outputs a kripke frame that satisfies the formula. Furthermore, it specifies the set of states where the formula is satisfied. If this set is equal to $K$, the set of all states in the kripke frame, then the formula is valid in this kripke frame. Otherwise, this set satisfies the formula and its complement, with respect to $K$, falsifies the formula.

To check the validity of a formula in all kripke frames, its negation should be used as an input and if the result is \textit{unsat}, then it is valid. If the result is \textit{sat}, then the returned kripke frame is a counter example that falsifies the original formula.

\section{PDL Complexity}

The satisfiability problem for PDL formulas is decidable, and it is \textit{EXPTIME-complete} (theorm 8.5 in \cite{dynamic}). Therefore the solver may fail to determine the satisfiability within the time limit (by default it is 30 seconds), and hence it returns \textit{unknown} result.

% THEOREM 6.5 ( SMALL MODEL THEOREM): Let $\varphi$ be a satisfiable formula of
% PDL. Then $\varphi$ is satisfied in a Kripke frame with no more than $2^{\vert \varphi \vert}$ % states.

On the other hand the satisfiability of a PDL formula in a given Kripke frame can be determined in polynomial time (exercise 6.4 in \cite{dynamic}). So one expects this to be easy for CVC4 when the model (Kripke frame) is given and ``check-sat" is executed. Unfortunately, CVC4 doesn't terminate if the formula is \textit{unsat} and uses transitive closure operation. After asking an expert (Andrew Reynolds), apparently CVC4 keeps  ``adding a repeating pattern of lemmas related to transitive closure" which needs to be reviewed. Fortunately I found a work around by just removing the constraint related to the PDL formula and asserting only the frame constraints. After CVC4 finds the model, the PDL formula is evaluated. If the result is true, then it is \textit{sat}, otherwise it is \textit{unsat}.

\subsection{Exercise 6.4}
Exercise 6.4 in the book says:
\begin{quote}
Give a polynomial time algorithm to check whether a given PDL formula
is satisfied at a given state in a given Kripke frame. Describe briefly the data
structures you would use to represent the formula and the Kripke frame. Specify
your algorithm at a high level and give a brief complexity analysis
\end{quote}

The following is my reasoning for this exercise where $K$ is the set of all states in the frame:
\begin{itemize}
\item Formulas $\textbf{0}$, $\textbf{1}$, and atomic propositions are subsets of $K$, therefore they can be computed in $O(\vert K \vert)$.
\item Programs $\textbf{skip}?$, $\textbf{fail}?$, and atomic programs are subsets of $K \times K$, therefore they can be computed in $O(\vert K \vert^2)$.

\item Boolean operators $\neg, \wedge, \vee, \rightarrow, \leftrightarrow$ involve set operations $-, \cap, \cup: \mathbb{P}(K) \times \mathbb{P}(K) \rightarrow \mathbb{P}(K)$ where $\mathbb{P}$ denotes the power set. These set operations be computed in $O(\vert K \vert^2)$ or $O(\vert K \vert)$ if hash tables are used.

\item Program operators $;, \cup, *$ and modal operators $[], \langle \rangle$ involve set operations and relation operations $\circ, *: \mathbb{P}(K \times K) \times \mathbb{P}(K \times K) \rightarrow \mathbb{P}(K \times K)$. Both $\circ$ and $*$ can be computed using graph data structures and DFS algorithm.
For $\circ$ operation, the DFS algorithm stops at depth 2, and for $*$ it continues until the end.
If the graph is dense, both operations $\circ, *$ require at most $O(\vert K \vert^3)$ to be computed for all states (all graph vertices).

\item If the number of operators in a given formula $\varphi$ is $n$, then the formula semantics $m(\varphi)$ can be evaluated in $O(n \vert K \vert ^3)$. Finally it takes constant time to evaluate $m(\varphi) \neq \varphi$ to  determine the satisfiability answer.
\end{itemize}

From the above discussion, the worst case scenario takes $O(n \vert K \vert ^3)$ time which is polynomial.

\section{Installation}

The following commands download the source code from github and compile it to generate the solver file ``pdl.jar".

\begin{verbatim}
git clone https://github.com/mudathirmahgoub/pdl
cd pdl
chmod 777 gradlew
./gradlew build
cd bin
chmod 777 cvc4_linux
java -jar pdl.jar -i test.pdl
java -jar plantuml.jar test.dot
\end{verbatim}

The last command ``java -jar plantuml.jar test.dot" requires graphvis to be in the path. Installation instructions for graphvis are available in \url{https://graphviz.gitlab.io/download/}.

\section{Software components}
\begin{figure}[H]
\center
\includegraphics[scale=0.35]{solver.png}
\caption{Software components}
\end{figure}

As shown in the figure above, the input string is parsed into PDL abstract syntax tree (AST) which is read by the translator. Then it gets translated into SMT AST which is sent to CVC4 as SMT-LIB code. CVC4 checks the satisfiability and returns either \textit{sat}, \textit{unsat} or \textit{unknown}. When \textit{sat} result is returned, CVC4 is asked to return the SMT model and the values of additional SMT expressions. The returned SMT model and values are parsed into SMT ASTs which are converted into a kripke frame and a set of states satisfied by the PDL formula. Finally kripke, and dot files are generated.

The project is written in Java 8 and available in github\footnote{\url{https://github.com/mudathirmahgoub/pdl}}. To build the project, Gradle is used to download and compile the project and generate the jar files. Graphviz software is needed to visualize the generated dot file\footnote{\url{https://www.graphviz.org}}.

\section{PDL formula translation}

What does it mean to have a satisfiable formula in PDL ?

A PDL formula $\varphi$ is satisfiable if there exists a kripke frame $\mathfrak{K}=(K, m_{\mathfrak{K}})$ and a state  $u$ such that $u \in m_{\mathfrak{K}}(\varphi) $, and we write $\mathfrak{K}, u \models \varphi$. In simple words, a formula is satisfiable if its meaning is nonempty in some kripke frame (i.e. $m_{\mathfrak{K}}(\varphi) \neq \phi$).

Since the PDL semantics uses set and relation operations, they are translated directly into their corresponding operations in the relation theory in CVC4. The only exception is the reflexive transitive closure $*$ which is not supported directly in CVC4, but supported through the built-in transitive closure. Table \ref{tab:translation} summarizes this translation. Other PDL operators are defined using the basic operators in table \ref{tab:translation}. The book \cite{dynamic} introduces the converse operator $^{\_}$ in page 177 for \textit{backtracking} programs which was not covered in class. Its semantics is the transpose relation which is defined as:
\begin{align*}
m_{\mathfrak{K}}(a^{\_}) = m_{\mathfrak{K}}(a)^{\_} = \lbrace (v, u) \; \vert \; (u, v) \in m_{\mathfrak{K}}(a) \rbrace
\end{align*}

\begin{table}[H]
\begin{center}
\begin{tabular}{ll}
\toprule
PDL & CVC4 \\
\midrule
$K=\lbrace u_0, u_1, \cdots u_n \rbrace$ &
$\begin{matrix}
\text{\textbf{Atom} : \text{Uninterpreted sort}} \\
u_0, u_1, \cdots, u_n: \textbf{Atom} \\
\textit{atomUniverse}: \text{Set(Tuple (\textbf{Atom}))} = \lbrace \langle u_0 \rangle, \langle u_1 \rangle, \cdots , \langle u_n \rangle \rbrace \\
\textit{atomIdentity}:  = \lbrace \langle u_0, u_0 \rangle, \langle u_1, u_1 \rangle, \cdots , \langle u_n, u_n \rangle \rbrace \\
\end{matrix}$ \\ \midrule
\textbf{0} & \textit{emptyset} : (Set (Tuple \textbf{Atom})) \\
\textbf{1} & \textit{atomUniverse} \\
Atomic propositions $p, q, r, \cdots$ & $p, q, r, \cdots : \text{Set(Tuple (\textbf{Atom}))}$ \\
Atomic programs $a, b, c, \cdots$ & $a, b, c \cdots : \text{Set(Tuple (\textbf{Atom}$,$\textbf{Atom}))}$ \\
$p \vee q$ & $p \cup q$ \\
$p \wedge q$ & $p \cap q$ \\
$\neg p$ & $\textit{atomUniverse} - p$ \\
$p \rightarrow q$ & $(\textit{atomUniverse}- p)\cup q$ \\
$a;b$ & $a \circ b$ where $\circ$ is the join operator\\
$a \cup b$ (choice) & $a \cup b$ (union)\\
$a^*$ (iteration) & $\textit{atomIdentity} \cup a^+$, (transitive closure)\\
$p?$ (test) & $(p \times p) \cap \textit{atomIdentity}$\\
\color{red}
$a^{\_}$ (converse) & \color{red} $a^t (transpose)$\\

\bottomrule
\end{tabular}
\end{center}
\caption{PDL translation to CVC4.} \label{tab:translation}
\end{table}

\subsection{The semantics of the equivalence operator $\leftrightarrow$}

The textbook \cite{dynamic} did not mention explicitly the semantics of the operator $\leftrightarrow$. However in theorem 5.8 (and other theorems), the book proves the validity of the formula
\begin{align*}
\langle \alpha \cup \beta \rangle \varphi \leftrightarrow \langle \alpha \rangle \varphi \vee \langle \beta \rangle \varphi
\end{align*}
by proving this set equality
\begin{align*}
m_\mathfrak{K}( \langle \alpha \cup \beta \rangle \varphi ) = m_\mathfrak{K}(\langle \alpha \rangle\varphi  \vee  \langle \beta \rangle \varphi)
\end{align*}

This set equality $=_{\mathbb{P}(K)} : \mathbb{P}(K) \times \mathbb{P}(K) \rightarrow \lbrace \textit{true, false} \rbrace$ ($\mathbb{P}(K)$ is the power set of $K$) is an operation that returns a boolean. This is different than the semantics of operators $\rightarrow, \wedge, \vee: \mathbb{P}(K) \times \mathbb{P}(K) \rightarrow \mathbb{P}(K)$ which returns a subset of $K$. It is possible to use the set equality in $CVC4$ with an $ite$ (\textit{if-then-else}) expression to return a subset of $K$ as follows:
\begin{align*}
m_\mathfrak{K} (p \leftrightarrow q) = ite(m_\mathfrak{K} (p) =  m_\mathfrak{K}(q), K,  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right))
\end{align*}

In my translation, however,  I used the following semantics:
\begin{align*}
m_\mathfrak{K} (p \leftrightarrow q) = m_\mathfrak{K} ((p \rightarrow q) \wedge (q \rightarrow p))
\end{align*}
The following two lemmas connect between these three approaches in PDL formulas.

\subsection*{Lemma 1}

\begin{align*}
m_\mathfrak{K} ((p \rightarrow q) \wedge (q \rightarrow p)) = ite(m_\mathfrak{K} (p) =  m_\mathfrak{K}(q), K,  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right))
\end{align*}
\subsubsection*{\textit{Proof}}
\begin{enumerate}
\item Case $m_\mathfrak{K} (p) =  m_\mathfrak{K}(q)$:
\begin{align*}
m_\mathfrak{K} ((p \rightarrow q) \wedge (q \rightarrow p)) &=
((K - m_\mathfrak{K}(p)) \cup m_\mathfrak{K} (q)) \cap
((K - m_\mathfrak{K}(p)) \cup m_\mathfrak{K} (q)) \\
&= K \cap K  \\
&=K  = ite(m_\mathfrak{K} (p) =  m_\mathfrak{K}(q), K,  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right))
\end{align*}
\item Case $m_\mathfrak{K} (p) \neq  m_\mathfrak{K}(q)$:
\begin{align*}
m_\mathfrak{K} ((p \rightarrow q) \wedge (q \rightarrow p)) &=
((K - m_\mathfrak{K}(p)) \cup m_\mathfrak{K} (q)) \cap
((K - m_\mathfrak{K}(p)) \cup m_\mathfrak{K} (q)) \\
&= \left( K - (m_\mathfrak{K}(p) - m_\mathfrak{K}(q) \right)  \cap
   \left( K - (m_\mathfrak{K}(q) - m_\mathfrak{K}(p) \right) \\
&=  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right) \\
&= ite(m_\mathfrak{K} (p) =  m_\mathfrak{K}(q), K,  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right))
\end{align*}
\end{enumerate}

\subsection*{Lemma 2}

The PDL formula $p \leftrightarrow q$ is valid using the textbook definition $m_\mathfrak{K}(p)= m_\mathfrak{K}(q)$ if and only if the formula $\neg ((p \rightarrow q) \wedge (q \rightarrow p))$ is \textit{unsatisfiable}.
\subsubsection*{\textit{Proof}}
Assume the formula $p \leftrightarrow q$ is valid. This means $m_\mathfrak{K}(p)= m_\mathfrak{K}(q)$. Then
\begin{align*}
m_\mathfrak{K}(\neg ((p \rightarrow q) \wedge (q \rightarrow p)))
&= K - m_\mathfrak{K}((p \rightarrow q) \wedge (q \rightarrow p)) \\
&= K - ite(m_\mathfrak{K} (p) =  m_\mathfrak{K}(q), K,  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right))  \tag{\text{using lemma 1}} \\
&= \phi
\end{align*}

Assume the formula $\neg ((p \rightarrow q) \wedge (q \rightarrow p))$ is \textit{unsatisfiable}. Then
\begin{align*}
m_\mathfrak{K}(\neg ((p \rightarrow q) \wedge (q \rightarrow p))) &= \phi \Rightarrow \\
K- m_\mathfrak{K}((p \rightarrow q) \wedge (q \rightarrow p)) &= \phi \Rightarrow \\
K - ite(m_\mathfrak{K} (p) =  m_\mathfrak{K}(q), K,  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right)) &= \phi \Rightarrow \\
   ite(m_\mathfrak{K} (p) =  m_\mathfrak{K}(q), K,  K - \left( (m_\mathfrak{K}(p) - m_\mathfrak{K}(q))  \cup
   (m_\mathfrak{K}(q) - m_\mathfrak{K}(p)) \right)) &= K \Rightarrow \\
   m_\mathfrak{K} (p) =  m_\mathfrak{K}(q)
\end{align*}

\section{Examples}

\subsection{A PDL formula without Kripke frame}

\subsubsection*{Input}
\begin{verbatim}
(p and q) and
<do p -> a | q -> b od>
    (
        (<b> (p and not q))  and
        (<a> (q and not p))
    )
\end{verbatim}
The above formula is equivalent to:
\begin{align*}
((p \wedge q) \wedge \langle\textbf{ do }p \rightarrow a \vert q \rightarrow b\textbf{ od }\rangle(\langle b\rangle(p \wedge \neg q) \wedge \langle a \rangle(q \wedge \neg p)))
\end{align*}
\subsection*{Output (.kripke file)}
\begin{verbatim}
K = {0, 2, 1, 3}
m(p) = {0, 3}
m(q) = {1, 3}
m(a) = {(2,1), (3,2)}
m(b) = {(2,0)}
Satisfying states: [3]
\end{verbatim}
\subsection*{Output (.dot file)}

\begin{figure}[H]
\center
\includegraphics[scale=0.4]{example1.png}

\caption{The formula is satisfied in state $3$ (green) and unsatisfied in states $0, 1, 2$ (red). }
\end{figure}

\subsection{A PDL formula with Kripke frame}

\subsubsection*{Input}
\begin{verbatim}
K = {01,2,3,4,5}
m(p) = {01,2,4}
m(q) = {01, 3}
m(r) = {5}
m(a) = {(01,2), (01,4), (2,4), (01,3)}
m(b) = {(01,3), (3,3)}
m(c) = {(5,5)}
[a union b union c] (q and not p)
\end{verbatim}
The above formula is equivalent to:
\begin{align*}
[((a \cup b) \cup c)](q \wedge \neg p)
\end{align*}
\subsection*{Output (.kripke file)}
\begin{verbatim}
K = {s01, s2, s3, s4, s5}
m(p) = {s01, s2, s4}
m(q) = {s01, s3}
m(r) = {s5}
m(a) = {(s01,s2), (s01,s4), (s2,s4), (s01,s3)}
m(b) = {(s01,s3), (s3,s3)}
m(c) = {(s5,s5)}
Satisfying states: [s3, s4]
\end{verbatim}
\subsection*{Output (.dot file)}

\begin{figure}[H]
\center
\includegraphics[scale=0.4]{example2.png}

\caption{The formula is satisfied in states $s3, s4$ (green) and unsatisfied in states $s01, s2, s5$ (red). }
\end{figure}

\subsection{A valid PDL formula}
\subsubsection*{Input}
\begin{verbatim}
not (<a converse> [a] p -> p)
\end{verbatim}
The above formula is equivalent to
\begin{align*}
\neg(\langle a^{\_} \rangle [a] p \rightarrow p)
\end{align*}

\subsubsection*{Output}
unsat

This means the formula $\langle a^{\_} \rangle [a] p \rightarrow p$ is valid in all kripke frames.
\section{Benchmark}
To rigorously test the PDL solver, I used the valid formulas from chapter 5 as a benchmark. The total number of valid formulas in the benchmark is 42. The PDL solver managed to prove 24 formulas in less than 2 seconds, and timed out after 1 minute in 18 formulas.  I guess this benchmark can be used for the relation theory in CVC4. Table \ref{tab:benchmark} summarizes the result.

\begin{table}[H]
\center
\begin{tabular}{lccr}
\toprule
Label & PDL formula & Output & Duration \\
\midrule
Book formula 1 & $\neg ([a](p \wedge q) \leftrightarrow  [a] p \wedge [a]q)$   & unknown   & 60,000 ms \\
Book formula 2 & $\neg ([a;b]p  \leftrightarrow  [a][b]p)$  & unknown   & 60,000 ms \\
Book formula 3 & $\neg ([\textit{if }p\textit{ then }a\textit{ else }b]p  \leftrightarrow  [\textit{if }\neg  p\textit{ then }b\textit{ else }a]p)$  & unknown   & 60,000 ms \\

Deductive system 2 & $\neg ([a](p \rightarrow  q) \rightarrow  ([a]p \rightarrow  [a]q))$  & unsat & 100 ms \\

Deductive system 3 & $\neg ([a](p \wedge q) \rightarrow  ([a]p \wedge [a]q))$  & unknown   & 60,000 ms \\

Deductive system 4 & $\neg ([a \cup b]p  \leftrightarrow  [a]p \wedge [b]p)$  & unknown   & 60,000 ms \\

Deductive system 5 & $\neg ([a;b]p  \leftrightarrow  [a][b]p)$  & unknown   & 60,000 ms \\

Deductive system 6 & $\neg ([q?]p  \leftrightarrow  (q \rightarrow  p))$  & unsat  & 77 ms \\

Deductive system 7 & $\neg (p \wedge [a][a^{*}] p  \leftrightarrow  [a^{*}]p)$  & unknown   & 60,000 ms \\

Deductive system 8 & $\neg (p \wedge [a^{*}](p \rightarrow  [a]p) \rightarrow  [a^{*}]p)$  & unknown   & 60,000 ms \\

Deductive system 9 MP & $\neg ((p \wedge (p \rightarrow  q)) \rightarrow  q)$  & unsat  & 73 ms \\

Theorem 6.1 & $\neg ( \langle  a \rangle  (p \vee q)  \leftrightarrow   \langle  a \rangle  p \vee  \langle  a \rangle  q)$  & unsat & 90 ms \\

Theorem 6.2 & $\neg ([a](p \wedge q)  \leftrightarrow  [a]p \wedge [a]q)$  & unknown   & 60,000 ms \\

Theorem 6.3 & $\neg ( \langle  a \rangle  p \wedge [a] q \rightarrow   \langle  a \rangle  (p \wedge q))$  & unsat & 78 ms \\

Theorem 6.4 & $\neg ([a](p \rightarrow  q) \rightarrow  ([a]p \rightarrow  [a] q))$  & unsat & 99 ms \\

Theorem 6.5 & $\neg ( \langle  a \rangle  (p \wedge q) \rightarrow   \langle  a \rangle   p \wedge  \langle  a \rangle   q)$  & unsat & 67 ms \\

Theorem 6.6 & $\neg ([a]p \vee [a] q \rightarrow  [a] (p \vee q))$  & unknown   & 60,000 ms \\

Theorem 6.7 & $\neg ( \langle  a \rangle   0  \leftrightarrow  0)$  & unsat & 57 ms \\

Theorem 6.8 & $\neg ([a]p  \leftrightarrow  \neg   \langle  a \rangle   \neg  p)$  & unsat & 95 ms \\

Theorem 8.1 & $\neg ( \langle  a \cup b \rangle   p  \leftrightarrow   \langle  a \rangle   p \vee  \langle  b \rangle   p)$  & unsat & 82 ms \\

Theorem 8.2 & $\neg ([a \cup b] p  \leftrightarrow  [a] p \wedge [b] p)$  & unknown   & 60,000 ms \\

Theorem 10.1 & $\neg ( \langle  a ; b \rangle   p  \leftrightarrow   \langle  a \rangle   \langle  b \rangle   p)$  & unsat  & 93 ms  \\

Theorem 10.2 & $\neg ([a ; b] p  \leftrightarrow  [a][b] p)$  & unknown   & 60,000 ms \\

Theorem 11.1 & $\neg ( \langle  p? \rangle   q  \leftrightarrow  p \wedge q)$  & unsat & 82 ms \\

Theorem 11.2 & $\neg ([p?] q  \leftrightarrow  (p \rightarrow  q))$  & unsat & 82 ms \\

Theorem 12.1 & $\neg ([p?] q  \leftrightarrow  (p \rightarrow  q))$  & unsat  & 80 ms \\

Theorem 15.1 & $\neg ([a^{*}]p \rightarrow  p)$  & unsat  & 66 ms \\

Theorem 15.2 & $\neg (p \rightarrow   \langle  a^{*} \rangle   p)$  & unsat & 68 ms \\

Theorem 15.3 & $\neg ([a^{*}]p \rightarrow  [a]p)$  & unsat & 74 ms \\

Theorem 15.4 & $\neg ( \langle  a \rangle  p \rightarrow   \langle  a^{*} \rangle  p)$  & unsat  & 71 ms \\

Theorem 15.5 & $\neg ([a^{*}]p  \leftrightarrow  [a^{*};a^{*}]p)$  & unsat &  1,200 ms\\

Theorem 15.6 & $\neg ( \langle  a^{*} \rangle  p  \leftrightarrow   \langle  a^{*};a^{*} \rangle  p)$  & unsat & 109ms \\

Theorem 15.7 & $\neg ([a^{*}]p  \leftrightarrow  [(a^*)^*]p)$  & unknown   & 60,000 ms \\

Theorem 15.8 & $\neg ( \langle  a^{*} \rangle  p  \leftrightarrow   \langle  (a^*)^* \rangle  p)$  & unknown   & 60,000 ms \\

Theorem 15.9 & $\neg ([a^{*}]p  \leftrightarrow  p \wedge [a][a^{*}]p)$  & unknown   & 60,000 ms \\

Theorem 15.10 & $\neg ( \langle  a^{*} \rangle  p  \leftrightarrow  p \vee  \langle  a \rangle   \langle  a^{*} \rangle  p)$  & unknown   & 60,000 ms \\

Theorem 15.11 & $\neg ([a^{*}]p  \leftrightarrow  p \wedge [a^{*}](p \rightarrow  [a]p))$  & unknown   & 60,000 ms \\

Theorem 15.12 & $\neg ( \langle  a^{*} \rangle  p  \leftrightarrow  p \vee  \langle  a^{*} \rangle  (\neg  p \wedge  \langle  a \rangle  p))$  & unknown   & 60,000 ms \\
theorem 13.1 & $\neg (p \rightarrow  [a] \langle  a^{\_} \rangle  p)$  & unsat & 375 ms \\
theorem 13.2 & $\neg (p \rightarrow  [a^{\_}] \langle  a \rangle  p)$  & unsat & 73 ms \\
theorem 13.3 & $\neg ( \langle  a \rangle  [a^{\_}]p \rightarrow  p)$  & unsat & 79 ms \\
theorem 13.4 & $\neg ( \langle  a^{\_} \rangle  [a]p \rightarrow  p)$  & unsat & 74 ms \\
\bottomrule
\end{tabular}
\caption{PDL solver results for 5 valid formulas in chapter 5 with  1 minute timeout.}
\label{tab:benchmark}
\end{table}

\bibliographystyle{plain}

\bibliography{references}

\end{document}


%%
%% pytorch-neural-doodle/docs/content/chapters/introduction.tex
%%
%% Created by Paul Warkentin <paul@warkentin.email> on 21/08/2018.
%% Updated by Bastian Boll <mail@bbboll.com> on 03/10/2018.
%%

\section{Introduction}
\label{section:introduction}

\textbf{by Bastian Boll} \\

In light of meaningful advances in the field of image classification using convolutional neural networks (CNNs) \cite{vgg2014}, interest has been sparked to exploit the inverse process for image generation. Because the intrinsic dimension of the input image space \(I\) is generally much larger than the intrinsic dimension of the label space \(L\), such a CNN cannot canonically be a bijective mapping. The inverse process can therefore be seen as selecting an element \(x\) of the preimage \(\mathrm{CNN}^{-1}(y)\) for a fixed label \(y\in L\) such that a measure of visual or artistic appeal is maximized.

For the purpose of a more fine-grained analysis of this process, suppose a given network which is discriminatively trained for object classification. Let \(L_i\) denote the feature space of said network at layer \(i\). The network can be characterized as functions \(\text{net}_i\colon I \to L_i\) which encode a given input image into the feature space \(L_i\). Sampling the preimage \(\text{net}_i^{-1}(y_i)\) of a given feature vector \(y_i\in L_i\) can serve to visuallize the way in which such networks extract features from the input image. One can observe that the lower levels of the network capture mostly local texture information while higher levels encode more abstract information about the objects visible in the input image.

% MAYBE: Leveraging encoding into these feature spaces for style transfer requires the combination of two general fields of expertise: style / texture and content.

In texture generation, a related area of research, efforts have been made to characterize texture as image statistics \cite{julesz1962visual,heeger1995pyramid,rosenholtz2012summary}. Given a textured image, one can generate a different image which human observers percept to contain the same texture. This is done by looking for an image that reproduces the respective image statistics. However, careful hand-crafting of such image statistics is inherently limited and the results still fall short of being able to reproduce the full range of natural texture. In 2015, Gatys et al. \cite{gatys2015texture} proposed using the feature encoding in lower levels of a VGG19 network \cite{vgg2014} to provide the image statistics needed to capture texture. This particular architecture is especially well suited for generative tasks, as it does not aggressively optimize feature representation for the purpose of classification \cite{nikulin2016exploring}. Training the network to classify objects also results in functions \(\text{net}_i\) which encode low level image statistics such as texture. This new approach has had a large impact on the still young field of texture synthesis.

% TODO: reference to early visual system as motivation for the following

In the same year, Gatys et al. also introduced a precise formulation of artistic style transfer as reproducing a given content image using texture from a different style image \cite{gatys2015neural}. They achieve very convincing results by constructing an image which reproduces the low level texture statistics of the style image while also reproducing higher level statistics of the content image.
This requires an optimization process during which a target image is constructed to minimize a combined style- and content loss function.
Let \(p\) denote the original (content) image and \(x\) denote the target image to be generated.
Consider the activations \(P^i\in \R^{M_i\times N_i}\) and \(F^i\in \R^{M_i\times N_i}\) at layer \(i\) of the VGG network for the original image \(p\) and the target image \(x\) respectively.
The content loss is defined by Gatys et al. as a simple mean squared error loss between the activations of the content image and the target image.
\[\mathcal{L}_\text{c}(p,x,i) = \frac{1}{2}\sum_{j,k} (F^i_{jk}-P^i_{jk})^2\]
On the other hand the style loss as described in \cite{gatys2015texture} requires further preparation. We define Gram matrices \(G^i\in \R^{M_i\times M_i}\) through
\[G^i_{jk} = \sum_n F^i_{jn}F^i_{kn}\]
and set the style loss to be
\[\mathcal{L}_\text{s} = \sum_i \frac{w_i}{M_i^2N_i^2} \sum_{jk} (G^i_{jk}-\hat G^i_{jk})^2\]
Here, \(G^i\) refers to the Gram matrix computed from the activations of the target image and \(\hat G^i\) to those for the style image. The weights \(w_i\) are adjusted to pronounce activations in lower network layers. In practice, many of these weights will be set to zero in order to save computational effort.

Investigating the motivation behind the Gram matrix representation and loss function construction\footnote{This is detailed in a supplementary blog post to the paper \cite{ulyanov2016texture}: \url{https://dmitryulyanov.github.io/feed-forward-neural-doodle/}}, one can assume the activations \(F^i\) to follow an unknown true distribution \(P\). Choosing a Gaussian distribution with zero mean \(Q\sim \mathcal{N}(0,\Sigma)\) as a model, we can compute the information projection
\[\Sigma^\ast = \arg\min_\Sigma \text{KL}(P||Q)\]
where \(\text{KL}(\cdot)\) denotes the Kullback-Leibler divergence. This in turn yields the solution
\[\Sigma^\ast = \mathbb{E}_{x\sim P} \,x^Tx\]
Because the true distribution \(P\) is not obtainable, we instead fit \(Q\) to the data by replacing the above with the empirical covariance and obtain
\[\Sigma^\ast \approx \frac{1}{MN} \sum_{jk} G_{jk}\]
for the respective layer of the network. In fact, Gatys et al. use a slightly different normalization constant, but we find this derivation to be more convincing.

Choosing a weighting factor \(\alpha\) yields the full loss function for artistic style transfer as
\[\mathcal{L} = \mathcal{L}_\text{c}+\alpha \mathcal{L}_\text{s}\]

Because the Gram matrices describe global correlations between feature activations as detailed above, they do not retain local features of the texture at hand. This presents a problem for artistic style transfer, as local glitches in the generated image can hardly be addressed. As an alternative approach, one can define a patch based style loss function \cite{mrf2016} which respects local pixel statistics. Let \(\Psi(x)\) denote the list of all \(k\times k\) patches of the activations for the input image \(x\) at a given layer of the network. We can define a style loss function as
\[\mathcal{L}_{s,p}(x,p) = \sum_i \|\Psi_i(x)-\Psi_{\text{NN}(i)}(p)\|_2^2\]
where \(\text{NN}(i)\) refers to the index of the nearest neighbour patch of \(\Psi_i(x)\) in the set of patches \(\Psi(p)\) with respect to normalized cross-correlation
\[\text{NN}(i) = \arg\min_{j} \frac{\Psi_i(x)\cdot \Psi_j(p)}{\|\Psi_i(x)\|\|\Psi_j(p)\|}\]
Using a patch-based style loss reduces the overall number of glitches in the generated image, as the algorithm respects local pixel statistics and is not exclusively guided by global correlations.

Having defined and explored the problem domain of artistic style transfer, subsequent work focuses on more practical and artistic aspects. While the number of glitches can and should be reduced by the means described above, one can certainly see the point of wanting to interact with the algorithm at work. Opening up hooks for interactive input from a user can potentially turn this algorithm into an artistic tool which can be led by human intuition. In this vein, Champandard proposes using semantic segmentation maps of both input style and content images \cite{doodles2016}. The following main part of the present paper focuses on how to implement this approach.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
%  problems remaining:
%  #  why the z-vector equation only solved once? we do not need to solve 3N
%      equations?
%
%
%
%
%
%

\chapter{Configuration Interaction Singles Method}
%
%
%
%
In the configuration interaction method, the most easiest way is to just take
the first excitation states into consideration. Such method is called
``Configuration Interaction Singles Method'', so it's abbreviated as
``CIS'' method.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{CIS Formulation}
%
%
%
%
Firstly, let's go to investigate the formulation to calculate the CIS
energy. In the first step, we will only consider the restricted type
of determinants so that to make things easier.

Suggest we have $n$ occupied MOs, and the electrons are $2n$; then the
ground state Slater determinant is:
\begin{equation}
  \label{eq:CISeq:1}
  \Phi_{HF} =  |\varphi_{1}(1)\alpha(1)\varphi_{1}(2)\beta(2)\cdots
\varphi_{n-1}(2n-1)\alpha(2n-1)\varphi_{n}(2n)\beta(2n)|
\end{equation}

It's energy is:
\begin{multline}\label{CISeq:2}
% \nonumber to remove numbering (before each equation)
  E
  =\sum_{i}^{occ}\langle\varphi_{i}(1)|\hat{h}_{1}|\varphi_{i}(1)\rangle
  +  \\
  \sum_{i < j}^{occ} \left\{
    2\langle\varphi_{i}(1)\varphi_{j}(2)|\frac{1}{r_{12}}|\varphi_{i}(1)\varphi_{j}(2)\rangle-
    \langle\varphi_{i}(1)\varphi_{j}(2)|\frac{1}{r_{12}}|\varphi_{j}(1)\varphi_{i}(2)\rangle
  \right\}
\end{multline}

More specifically, the energy for the orbital is (here we suppose that
the orbital is in $\alpha$ spin state):
\begin{eqnarray}\label{CISeq:3}
% \nonumber to remove numbering (before each equation)
  \epsilon_{i} &=&  \langle \varphi_{i}| \hat{F} | \varphi_{i} \rangle \nonumber \\
  &=& \langle i|h|i \rangle + \sum_{j \neq i} \left\{
    2[ii|jj] - [ij|ij]
  \right\}
\end{eqnarray}
This is for the occupied orbital, so we have to specify that $j \neq
i$. For the virtual orbital, it's energy is:
\begin{eqnarray}\label{CISeq:4}
% \nonumber to remove numbering (before each equation)
  \epsilon_{a} &=&  \langle \varphi_{a}| \hat{F} | \varphi_{a} \rangle \nonumber \\
  &=& \langle a|h|a \rangle + \sum_{j}^{occ} \left\{
    2[aa|jj] - [aj|aj]
  \right\}
\end{eqnarray}
Here we note that we use a, b, c etc. to designate the virtual
orbitals, the i, j, k etc. to refer to the occupied orbitals; and p,
q, r etc. to specify the general orbitals.

Now through the HF calculation, we have gotten $n$ occupied orbitals
and another $m$ virtual orbitals ($n+m$ is the number of basis
sets). The next question is, how to form the single excitation states
from HF orbitals?

In this process, an electron from the occupied orbital is ``fired up''
into some virtual orbitals, so that to form some new determinant. Then
the new Slater determinant can be expressed as:
\begin{equation}
    \label{eq:CISeq:5}
  \Phi_{i}^{a} =  |\cdots\varphi_{a}(k)\alpha(k)\varphi_{i}(k+1)\beta(k+1)\cdots
\varphi_{n-1}(2n-1)\alpha(2n-1)\varphi_{n}(2n)\beta(2n)|
\end{equation}
Here the electron $k$ with spin state of $\alpha$ is excited into the
virtual orbital $\varphi_{a}$, so we use the $\varphi_{a}$ to replace
the original $\varphi_{i}$ in the $\alpha$ spin state (see the picture
of \ref{ris_pic}).
\begin{figure}[htbp]
\begin{center}
\includegraphics[scale=0.3]{ris1.eps}\label{ris_pic}
\caption{single excitation state}
\end{center}
\end{figure}

By such configuration method, actually we can form $n\times m$ Slater
determinants. Then the trial wave function can be expressed as their
linear combinations:
\begin{equation}
  \label{CISeq:6}
  \Psi = \sum_{i}^{occ}\sum_{a}^{vir}C_{i}^{a}\Phi_{i}^{a}
\end{equation}

Here there are two points should be clarified. Firstly, we note that
in the expression of (\ref{CISeq:6}) the root determinant (ground
state determinant) is not included. Why?

This reason for this is from Brillouin theorem, which has been
demonstrated in the above content. The Brillouin theorem says that the
HF determinant is unable to mix with the single excitation states, so
there's no coupling between them. If we include the HF determinant
into the expression of (\ref{CISeq:6}), the Hamiltonian matrix will be
just like the form below:
\begin{equation}
  \label{CISeq:7}
  \begin{bmatrix}
    \langle\Phi_{HF}|\hat{H}|\Phi_{HF}\rangle & 0  \cdots & \cdots \\
    0 & \langle \Phi_{i}^{a}|\hat{H}|\Phi_{i}^{a} \rangle & \cdots \\
    \cdots & \langle \Phi_{i}^{a}|\hat{H}|\Phi_{i}^{a} \rangle & \cdots \\
  \end{bmatrix}
\end{equation}
Here we can see that the first column and first low are all zero
except the $\langle\Phi_{HF}|\hat{H}|\Phi_{HF}\rangle$, hence it's no
use to put the $\Phi_{HF}$ into the (\ref{CISeq:6}).

Secondly, we should pay attention to the spin state for each trial
wave function. Actually there are many potential spin states for the
trial wave function, take $H_{2}$ molecule (close shell molecule) as
an example, it's excitation states may be singlet ($S = 0$), or
triplet (S = 1). As what we have said in section \ref{SIC4}, only the
same spin state determinant can mix together so that to form some
trial wave function of $\Psi$:
\begin{align}
  \label{CISeq:8}
\Psi_{single} &= \sum_{i}\sum_{a}C_{i}^{a}\Phi_{single} \nonumber \\
\Psi_{triplet} &= \sum_{i}\sum_{a}C_{i}^{a}\Phi_{triplet}
\end{align}
That's what I want to say, however; so far we do not take spin into
account so to avoid adding complicity into our consideration.

On the other hand, we note that the forming method for the Slater
determinant can also varied. It can also be unrestricted method, so
the $\alpha$ electrons and $\beta$ electrons are not arranging into
the same spatial orbitals space. However, just as what we have
demonstrated, there are spin contamination situation in the
unrestricted method, so this method is not reliable.

Finally let's take the trial wave function in (\ref{CISeq:6}) into the
Schroedinger equation:
\begin{equation}
  \label{CISeq:9}
  \sum_{i}^{occ}\sum_{a}^{vir}\hat{H}|\Phi_{i}^{a}\rangle C_{i}^{a} =
E_{CIS} \sum_{i}^{occ}\sum_{a}^{vir}C_{i}^{a}|\Phi_{i}^{a}\rangle
\end{equation}
For short, we abbreviate the $\sum_{i}^{occ}\sum_{a}^{vir}$ as
$\sum_{ia}$. Then we multiply $\Phi_{j}^{b}$ to the above equation, it
becomes:
\begin{equation}
  \label{CISeq:10}
 \sum_{ia}\langle\Phi_{j}^{b}|\hat{H}|\Phi_{i}^{a}\rangle C_{i}^{a}
 = E_{CIS}\sum_{ia}C_{i}^{a}\delta_{ij}\delta_{ab}
\end{equation}
That's the equation we finally get.

Next, we have to know that how to evaluate the matrix element for the
equation of (\ref{CISeq:10}).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Matrix element in CIS equation}
\label{sec:ME_CIS}
%
%
%
%
From the (\ref{CISeq:10}), it's easily to know that there are only
four type of matrix element:
\begin{itemize}
\item $\langle\Phi_{i}^{a}|\hat{H}|\Phi_{i}^{a}\rangle$
\item $\langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{a}\rangle$, where $i
\neq j$
\item $\langle\Phi_{i}^{a}|\hat{H}|\Phi_{i}^{b}\rangle$, where $a
\neq b$
\item $\langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{b}\rangle$, where $i
\neq j$, $a \neq b$
\end{itemize}

firstly, let's calculate the
$\langle\Phi_{i}^{a}|\hat{H}|\Phi_{i}^{a}\rangle$. For simplicity, we
assume that we are in open shell situation, and the electron on the
orbital $i$ is fired up into the orbital $a$; just as demonstrated in
the picture below:

Based on the HF results, we can have:
\begin{align}
\label{CISeq:11}
  \langle\Phi_{i}^{a}|\hat{H}|\Phi_{i}^{a}\rangle &= \sum_{k\neq
    i}^{occ} \bra{\varphi_{k}}\hat{h}_{1}\ket{\varphi_{k}} -
\bra{\varphi_{i}}\hat{h}_{1}\ket{\varphi_{i}} +
\bra{\varphi_{a}}\hat{h}_{1}\ket{\varphi_{a}} \nonumber \\
&+ \sum_{k<l}^{occ}\left\{2[kk|ll] - [kl|kl] \right\} -
\sum_{k \neq i}^{occ}\left\{2[kk|ii] - [ki|ki] \right\} \nonumber \\
&+ \sum_{k}^{occ}\left\{2[kk|aa] - [ka|ka] \right\} +
\left\{2[aa|ii] - [ai|ai] \right\}
\end{align}

Firstly we note that in the (\ref{CISeq:11}) it's correspondent to
triplet, where the electrons on the orbital $i$ and orbital $a$ have
the same spin state. On the other hand, if it's the singlet state,
then the exchange term between orbital $i$ and orbital $a$ is
naturally vanished, and the other terms are kept.

Then according to the expression for orbital energy in (\ref{CISeq:3})
and (\ref{CISeq:4}), we can rewrite the whole expression as:
\begin{equation}
  \label{CISeq:12}
    \langle\Phi_{i}^{a}|\hat{H}|\Phi_{i}^{a}\rangle = \Big\{ E_{HF} +
    \varphi_{a} - \varphi_{i}\Big\} + (ii||aa)
\end{equation}
Where the $(ii||aa)$ is abbreviated as:
\begin{equation}
  (ii||aa) = \int dr d r^{'}
  \frac{\varphi_{i}^{*}(r)\varphi_{i}(r)\varphi_{a}^{*}(r^{'})\varphi_{a}(r^{'})
   - \varphi_{i}^{*}(r)\varphi_{a}(r)\varphi_{a}^{*}(r^{'})\varphi_{i}(r^{'})}
  {|r-r^{'}|}
\end{equation}

Secondly, let's go to see the term of
$\langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{a}\rangle$ and
$\langle\Phi_{i}^{a}|\hat{H}|\Phi_{i}^{b}\rangle$. For the single
electron energy part, it's easily to show that it's zero; because we
can always find that some orbital in the bra can not find its
counterpart in the ket. For example, in the
$\langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{a}\rangle$ if the alpha
electron is fired up from the orbital $i$ to the orbital $a$, then we
still have a beta electron leaving in the orbital $i$. Then it forms
some vacant position in orbital $i$ with alpha spin state. However, we
have the alpha electron in the orbital $i$ in the ket, but its
counterpart in bra is disappeared.

For the second electron energy part, the same situation holds. In most
of case, no matte how we compose the pair of orbitals, such as
$(ai||ka)$, it's turned out that in the rest of orbitals, there's
always has some orbital that can not find its counterpart so that the
whole integral goes zero. There's only one exception, which is the
integral between occupied-virtual orbital pairs; only this integral
survives. Hence we can finally have:
\begin{align}
  \label{CISeq:13}
  \langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{a}\rangle &= (aa||ij)
  \nonumber \\
  \langle\Phi_{i}^{a}|\hat{H}|\Phi_{i}^{b}\rangle &= (ab||ii)
\end{align}

Finally, for the $\langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{b}\rangle$,
similarly we only have the $(ij||ab)$ integral does not go zero. So it
can be expressed as:
\begin{equation}
  \label{CISeq:14}
  \langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{b}\rangle = (ab||ij)
\end{equation}

All in all, combined the results in the (\ref{CISeq:12}),
(\ref{CISeq:13}) and (\ref{CISeq:14}), we can have the matrix element
in the CIS equation expressed as:
\begin{equation}
  \label{CISeq:15}
  \langle\Phi_{i}^{a}|\hat{H}|\Phi_{j}^{b}\rangle = \Big\{ E_{HF} +
    \varphi_{a} - \varphi_{i}\Big\}\delta_{ij}\delta_{ab} + (ab||ij)
\end{equation}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{CIS Equation}
\label{sec:CIS_solution}
%
%
%
%
%
Now let's take the (\ref{CISeq:15}) into the general form of
(\ref{CISeq:10}), we can have:
\begin{align}
 \label{CISeq:16}
 \sum_{ia}\langle\Phi_{j}^{b}
\left\lbrace (\varphi_{a} - \varphi_{i})\delta_{ij}\delta_{ab} +
(ab||ij)\right\rbrace  C_{i}^{a}
 &=  \sum_{ia}(E_{CIS} -E_{HF})C_{i}^{a}\delta_{ij}\delta_{ab}
\nonumber \\
\left\lbrace
(\varphi_{a} - \varphi_{i})\delta_{ij}\delta_{ab} +
(ab||ij)\right\rbrace  C_{i}^{a} &= \sum_{ia}\omega
C_{i}^{a}\delta_{ij}\delta_{ab}
\end{align}
Here, $\omega$ is the $E_{CIS} -E_{HF}$ which just characterizes the
excitation energy.

Now we have to enter into the coding procedure, so how can we code
the CIS equation?

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{The Analytical Gradient for CIS Equation}
%
%
%
%
Now let's step into the gradient of CIS equation. For the expression in
(\ref{CISeq:16}), we have:
\begin{align}
 \label{CIS_gradient:1}
\omega &= \sum_{ai}\sum_{bj}X_{ai}A_{ai,bj}X_{bj}
\end{align}
Where
\begin{equation}
 A_{ai,bj} = (\epsilon_{a} - \epsilon_{i})\delta_{ij}\delta_{ab} +
(ai||bj)
\end{equation}
According to the Winger's theorem, for the gradient expression the variational
parameters does not go into it so that we have:
\begin{align}
\label{CIS_gradient:2}
 \omega^{[x]} &= \sum_{ai}\sum_{bj}X_{ai}A^{[x]}_{ai,bj}X_{bj} \nonumber \\
&=   \sum_{ai}\sum_{bj}X_{ai} \left\lbrace( \epsilon^{[x]}_{a} -
\epsilon^{[x]}_{i})\delta_{ij}\delta_{ab} +
(ai||bj)^{[x]}\right\rbrace X_{bj}
\end{align}
Giving by (\ref{CPHF_hf_derivatives_eq:10}), which is:
\begin{align}
  \label{CIS_gradient:3}
  \epsilon^{[x]} &= F_{pp}^{[x]} = H^{x}_{pp} + \sum_{k}^{occ}\left
\{\Pi^{x}_{ppkk} -
    \Pi^{x}_{pkkp} \right\}
  \nonumber \\
  &- S^{x}_{pp}\epsilon_{p} - \sum_{k}^{occ}\sum_{t}^{occ}S^{x}_{kt}
  \left\{ \Pi_{pptk} - \Pi_{ptkp} \right\} \nonumber \\
  &+ \sum_{k}^{occ}\sum_{t}^{vir}U^{x}_{tk}\left\{ 2\Pi_{pptk} -
    \Pi_{ptkp} - \Pi_{pktp} \right\} \nonumber \\
&= F^{x}_{pp} - S^{x}_{pp}\epsilon_{p}
 - \sum_{k}^{occ}\sum_{t}^{occ}S^{x}_{kt}(pp||tk) \nonumber \\
&+2\sum_{k}^{occ}\sum_{t}^{vir}U^{x}_{tk}(pp||tk)
\end{align}
Where the $F^{x}_{pp} $ is given as:
\begin{equation}
 \label{CIS_gradient:10}
F^{x}_{pp} = H^{x}_{pp} + \sum_{k}^{occ}(pp||kk)^{x}
\end{equation}
$p$ is referred to some general orbital.  For the $\epsilon^{[x]}_{a} -
\epsilon^{[x]}_{i}$, given by the above expression it is:
\begin{equation}
 \begin{split}
\epsilon^{[x]}_{a} - \epsilon^{[x]}_{i} &=  (F^{x}_{aa} -
F^{x}_{ii}) +
(S^{x}_{ii}\epsilon_{i} - S^{x}_{aa}\epsilon_{a})  \\
&+\sum_{k}^{occ}\sum_{l}^{occ}S_{kl}^{x}[(ii||lk) -
(aa||lk)] +  2\sum_{k}^{occ}\sum_{c}^{vir}U_{ck}[(aa||ck) - (ii||ck)]
 \end{split}
\label{CIS_gradient:11}
\end{equation}

As for the gradient expression for $(ai||bj)$ in
(\ref{two_electron_MO_INT_gradient_eq:1}), which is:
\begin{align}
 \label{CIS_gradient:4}
(ai||bj)^{[x]} &= (ai|bj)^{[x]} - (ab|ij)^{[x]} \nonumber \\
&= \sum_{t}\left[
U_{ta}(ti|bj) +
U_{ti}(at|bj) +
U_{tb}(ai|tj) +
U_{tj}(ai|bt)
\right] + (ai|bj)^{x} \nonumber \\
&-
\sum_{t}\left[
U_{ta}(tb|ij) +
U_{tb}(at|ij) +
U_{ti}(ab|tj) +
U_{tj}(ab|it)
\right] - (ab|ij)^{x}
%\nonumber \\
%&= \sum_{t}\left[
%U_{ta}(ti||bj) +
%U_{ti}(at||bj) +
%U_{tb}(ai||tj) +
%U_{tj}(ai||bt)
%\right] + (ai||bj)^{x}
\end{align}
Since in section of \ref{CPHF} we have derived the fact that within HF
framework, for the orbital response matrix of $U$ only $U_{occ, vir}$ and
$U_{vir, occ}$ exist, the $U_{occ,occ}$ and $U_{vir, vir}$ blocks are zero.
Hence we can safely set the $U_{occ,occ}$ and $U_{vir, vir}$ to zero, so the
(\ref{CIS_gradient:4}) can be transformed into:
\begin{equation}
\begin{split}
(ai||bj)^{[x]} &=
\sum_{k}^{occ}\left\{ U_{ka}\left[(ki|bj) - (kb|ij)\right] \right\} +
\sum_{c}^{vir}\left\{ U_{ci}\left[(ac|bj) - (ab|cj)\right] \right\} \nonumber
\\
&+\sum_{k}^{occ}\left\{ U_{kb}\left[(ai|kj)-(ak|ij)\right] \right\} +
\sum_{c}^{vir}\left\{ U_{cj}\left[(ai|bc)  - (ab|ic)\right] \right\}
\nonumber \\
&+  (ai|bj)^{x} - (ab|ij)^{x}
\end{split}
 \label{CIS_gradient:12}
\end{equation}
Now let's make some analysis. Since in the expression, the summation is over
all the $i,j,a,b$ label, so actually we can see that:
\begin{align}
\sum_{abc}^{vir}\sum_{i}^{occ}(ai|bc)  &= \sum_{abc}^{vir}\sum_{i}^{occ}(ab|ic)
\nonumber \\
\sum_{ijk}^{occ}\sum_{b}^{occ}(ki|bj) &= \sum_{ijk}^{occ}\sum_{b}^{occ}(kb|ij)
 \label{CIS_gradient:13}
\end{align}
In the first term, we only need to make $a\leftrightarrow c$, and in the second
term to make $j \leftrightarrow k$; then we can derive the equality relation.

Finally the (\ref{CIS_gradient:12}) becomes:
\begin{equation}
 \begin{split}
 (ai||bj)^{[x]} &=
  \sum_{c}^{vir}\left\{ U_{ci}\left[(ac|bj) - (ab|cj)\right] \right\}
+\sum_{k}^{occ}\left\{ U_{kb}\left[(ai|kj)-(ak|ij)\right] \right\}  \nonumber
\\
&+  (ai|bj)^{x} - (ab|ij)^{x}
 \end{split}
\label{CIS_gradient:14}
\end{equation}

Finally, we can see that the (\ref{CIS_gradient:2}) becomes:
\begin{align}
 \label{CIS_gradient:6}
\omega^{[x]} &= \sum_{ai}\sum_{bj}X_{ai}\Bigg\{\Big[  (F^{x}_{aa} -
F^{x}_{ii}) +
(S^{x}_{ii}\epsilon_{i} - S^{x}_{aa}\epsilon_{a})  \nonumber \\
&+\sum_{k}^{occ}\sum_{l}^{occ}S_{kl}^{x}[(ii||kl) -
(aa||kl)]\Big]\delta_{ij}\delta_{ab} + (ai||bj)^{x} \Bigg\} X_{bj} \nonumber
\\
&+
\sum_{ai}\sum_{bj}X_{ai}\Bigg\{2\delta_{ij}\delta_{ab}\sum_{k}^{occ}\sum_{c}^{
vir}U_{ck}[(aa||ck) - (ii||ck)]\nonumber \\
& +\sum_{c}^{vir}\left\{ U_{ci}\left[(ac|bj) - (ab|cj)\right] \right\}
+\sum_{k}^{occ}\left\{ U_{kb}\left[(ai|kj)-(ak|ij)\right] \right\} \Bigg\}X_{bj}
\end{align}
In the expression of (\ref{CIS_gradient:6}), there both has $U_{occ, vir}$
and $U_{vir, occ}$ blocks, however; we can use the relation defined in
(\ref{overlap_MO_INT_gradient_eq:3}) to eliminate it:
\begin{align}
 \label{CIS_gradient:7}
U_{kb} &= -U_{bk} - S_{bk}^{x}
\end{align}
Then the (\ref{CIS_gradient:6}) becomes:
\begin{align}
 \label{CIS_gradient:7}
\omega^{[x]} &= \sum_{ai}\sum_{bj}X_{ai}\Bigg\{\Big[  (F^{x}_{aa} -
F^{x}_{ii}) +
(S^{x}_{ii}\epsilon_{i} - S^{x}_{aa}\epsilon_{a})  \nonumber \\
&+\sum_{k}^{occ}\sum_{l}^{occ}S_{kl}^{x}[(ii||kl) -
(aa||kl)]\Big]\delta_{ij}\delta_{ab} \nonumber
\\
& -\sum_{k}^{occ}\left\{S^{x}_{bk}\left[(ai|kj)-(ak|ij)\right] \right\} +
(ai||bj)^{x} \Bigg\} X_{bj}
\nonumber \\
&+
\sum_{ai}\sum_{bj}X_{ai}\Bigg\{2\delta_{ij}\delta_{ab}\sum_{k}^{occ}\sum_{c}^{
vir}U_{ck}[(aa||ck) - (ii||ck)]\nonumber \\
&+\sum_{c}^{vir}\left\{ U_{ci}\left[(ac|bj) - (ab|cj)\right] \right\}
-\sum_{k}^{occ}\left\{ U_{bk}\left[(ai|kj)-(ak|ij)\right] \right\} \Bigg\}X_{bj}
\end{align}

Now we can abbreviate the term related to $U$ matrix as:
\begin{align}
\label{CIS_gradient:8}
 Y_{ck} &= 2\sum_{ai}\sum_{bj}X_{ai}X_{bj}
\delta_{ij}\delta_{ab}[(aa||ck) - (ii||ck)]\nonumber \\
&+\sum_{ab}\sum_{j}X_{ak}X_{bj}\left\{\left[(ac|bj)
- (ab|cj)\right] \right\} \nonumber \\
&-\sum_{a}\sum_{ij}X_{ai}X_{cj}\left\{
\left[(ai|kj)-(ak|ij)\right] \right\}
\end{align}

Then (\ref{CIS_gradient:7}) becomes:
\begin{align}
 \label{CIS_gradient:9}
\omega^{[x]} &=  \sum_{ai}\sum_{bj}X_{ai}\Bigg\{\Big[  (F^{x}_{aa} -
F^{x}_{ii}) +
(S^{x}_{ii}\epsilon_{i} - S^{x}_{aa}\epsilon_{a})  \nonumber \\
&+\sum_{k}^{occ}\sum_{l}^{occ}S_{kl}^{x}[(ii||kl) -
(aa||kl)]\Big]\delta_{ij}\delta_{ab} \nonumber
\\
& -\sum_{k}^{occ}\left\{S^{x}_{bk}\left[(ai|kj)-(ak|ij)\right] \right\} +
(ai||bj)^{x} \Bigg\} X_{bj}
\nonumber \\
&+\sum_{c}^{vir}\sum_{k}^{occ}U_{ck}Y_{ck}
\end{align}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Z-Vector Method}
%
%
%
Now the issue inside the (\ref{CIS_gradient:9}) is that we have to solve the
$U$ matrix so that to get the explicit derivative expression. By using the CPHF
equation, which is shown in (\ref{CPHF_hf_derivatives_eq:17}) and
(\ref{CPHF_hf_derivatives_eq:18}), we generally have:
\begin{equation}
 \label{Zvector_CIS_eq:1}
AU^{a} = B^{a}\Leftrightarrow \sum_{jb}A_{ij, ab}U^{a}_{bj} =
B^{a}_{ia} \Rightarrow AU = B
\end{equation}
Where matrix of $A$ is $(occ\times vir)^{2}$ dimension, the U and $B$ are both
$occ\times vir$ dimension.

For the (\ref{Zvector_CIS_eq:1}), it can be further expressed as:
\begin{equation}
 \label{Zvector_CIS_eq:2}
U = A^{-1}B
\end{equation}
Now let's make a direct product between $U$ matrix and the $Y$ matrix defined
in (\ref{CIS_gradient:9}), combined with (\ref{Zvector_CIS_eq:2}), it's (for
matching with the label in \ref{CIS_gradient:8}, we use the label of $c,k$ etc.
rather than $b,j$ etc.):
\begin{align}
  \label{Zvector_CIS_eq:3}
\sum_{c}^{vir}\sum_{k}^{occ}U_{ck}Y_{ck} &=
\sum_{c}^{vir}\sum_{d}^{vir}\sum_{k}^{occ}\sum_{l}^{occ}A_{kl,cd}^{-1}B^{a}_{ld}
Y_{ck} \Rightarrow \nonumber \\
\sum_{c}^{vir}\sum_{k}^{occ}U_{ck}Y_{ck} &=
\sum_{d}^{vir}\sum_{l}^{occ}Z_{dl}B^{a}_{ld}
\end{align}
Where the vector of $Z$ is given by:
\begin{equation}
 \label{Zvector_CIS_eq:4}
Z_{dl} = \sum_{c}^{vir}\sum_{k}^{occ}A_{kl,cd}^{-1}Y_{ck} \Rightarrow
\sum_{d}^{vir}\sum_{l}^{occ}A_{kl,cd}Z_{dl} = Y_{ck}
\end{equation}

The new vector of $Z$, compared with the MO rotation matrix of $U$, only has
the dimension of $occ*vir$ for all the perturbations; that means; the $Z$ vector
can be solved only once and used for all the $3\times N$ atoms gradient
calculation. That's the advantages that why we solve the Z-vector rather than
the rotation matrix of $U$.

Now let's expand the (\ref{Zvector_CIS_eq:4}) into more detailed way:
\begin{equation}
 \begin{split}
&\sum_{l}^{occ}\sum_{d}^{vir}\left\lbrace   \left(\epsilon_{c} -
\epsilon_{k}\right)\delta_{kl}\delta_{cd} + \left[
2(kc|dl) -(kd|lc) - (kl|dc) \right] \right\rbrace Z_{dl} =  \\
&2\sum_{ai}\sum_{bj}X_{ai}X_{bj}
\delta_{ij}\delta_{ab}[(aa||ck) - (ii||ck)]
+\sum_{ab}\sum_{j}X_{ak}X_{bj}\left\{\left[(ac|bj)
- (ab|cj)\right] \right\}  \\
&-\sum_{a}\sum_{ij}X_{ai}X_{cj}\left\{
\left[(ai|kj)-(ak|ij)\right] \right\}
 \end{split}
 \label{Zvector_CIS_eq:5}
\end{equation}

After getting the vector of $Z$, then we can get the gradient expression for CIS equation.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../../main"
%%% End:


\section{Rank}


\documentclass[blue]{beamer}
\usepackage{etoolbox}
\mode<presentation>

%\usetheme{Warsaw}

\usepackage{paralist}
  \let\itemize\compactitem
  \let\enditemize\endcompactitem
  \let\enumerate\compactenum
  \let\endenumerate\endcompactenum
  \let\description\compactdesc
  \let\enddescription\endcompactdesc
  \pltopsep=\medskipamount
  \plitemsep=1pt
  \plparsep=1pt
\usepackage[english]{babel}

%/////////////////////////////////////////////////////////////////////////////
% math pkgs
\usepackage{graphicx} % more modern
\usepackage{bbm, bm, amsmath, amssymb, amsthm, mathrsfs, booktabs,float}
\usepackage{color}
\usepackage{subfigure}
% accent helper function
\usepackage{accents}
\DeclareMathSymbol{\fixwidehatsym}{\mathord}{largesymbols}{"62}
\newcommand\lowerwidehatsym{%
  \text{\smash{\raisebox{-1.3ex}{%
    $\fixwidehatsym$}}}}
\newcommand\fixwidehat[1]{%
  \mathchoice
    {\accentset{\displaystyle\lowerwidehatsym}{#1}}
    {\accentset{\textstyle\lowerwidehatsym}{#1}}
    {\accentset{\scriptstyle\lowerwidehatsym}{#1}}
    {\accentset{\scriptscriptstyle\lowerwidehatsym}{#1}}
}

%/////////////////////////////////////////////////////////////////////////////
% tikz
\usepackage{tikz}
\usetikzlibrary{arrows}

\newenvironment{figure*}%
{\begin{figure}}
{\end{figure}}

%////////////////////////////////////////////////////////////////////
%////////////////////////////////////////////////////////////////////

\title{Understanding Model Predictions via Influence Functions}

\subtitle{Statistical Machine Final Project}

\author{Ze Yang \and Zhengyang Qi \and Yundong Liu \and Yuze Liu}

\institute[Carnegie Mellon University] % (optional, but mostly needed)
{Carnegie Mellon University}
% - Use the \inst command only if there are several affiliations.
% - Keep it simple, no one is interested in your street address.

\date{2018.05.05}

% If you have a file called "university-logo-filename.xxx", where xxx
% is a graphic format that can be processed by latex or pdflatex,
% resp., then you can add a logo as follows:

% \pgfdeclareimage[height=0.5cm]{university-logo}{university-logo-filename}
% \logo{\pgfuseimage{university-logo}}

% Delete this, if you do not want the table of contents to pop up at
% the beginning of each subsection:
\AtBeginSubsection[]
{
  \begin{frame}<beamer>{Outline}
    \tableofcontents[currentsection,currentsubsection]
  \end{frame}
}

% Let's get started
\begin{document}

\begin{frame}
  \titlepage
\end{frame}

\begin{frame}{Outline}
  \tableofcontents
  % You might wish to add the option [pausesections]
\end{frame}

% Section and subsections will appear in the presentation overview
% and table of contents.
\section{Backgrounds}

\subsection{Introduction}

\begin{frame}{Introduction}{}
    \begin{itemize}
        \item  In this project, we reproduced the \emph{Best Paper} awardee of ICML 2017: \textit{Understanding Black-box Predictions via Influence Functions}, by Koh \& Liang.
        \item Our main topic is \textbf{influence function}, a technique to trace a model’s prediction through the learning algorithm and back to its training data.
        \item  We carried out efficient implementations with \texttt{Tensorflow}.
        \item We demonstrated its performance \& applications on real-world datasets.

    \end{itemize}
\end{frame}

\subsection{What is Influence Function?}

\begin{frame}{What is Influence Function: Some Definitions}{}
  \begin{itemize}
  \item \textbf{Statistical Functional:} $T(F_Z)$, a functional maps from the space of population CDFs to a field (such as $\mathbb{R}$).
  \item \textbf{Risk Optimizer:} Idealy, minimizes EPE: A statistical functional.
  $$
  \bm{\theta}(\fixwidehat{F}_Z)=\underset{\bm{\theta} \in \Theta}{\mathrm{argmin}}\int\mathcal{L}(\bm{z},\bm{\theta})d\fixwidehat{F}_Z
  $$
  \item \textbf{Empirical Risk Optimizer:} In reality, minimize empirical risk, a \textit{plug-in} estimator to the risk optimizer.
    $$
  \bm{\theta}(\fixwidehat{F}_Z)=\underset{\bm{\theta} \in \Theta}{\mathrm{argmin}}~\frac{1}{n}\sum_{i=1}^n\mathcal{L}(\bm{z}_i,\bm{\theta})
  $$
  \end{itemize}
\end{frame}

\begin{frame}{What is Influence Function: G\^ateaux Derivative}{}
  \textbf{G\^ateaux Derivative} generalizes the idea of \textbf{directional derivatives} - the inputs are CDFs!
\begin{align}\label{gateaux}
  \mathcal{I}_{\bm{\theta}}(G) &:= \nabla_{G} \bm{\theta}(F_Z)\biggr\rvert_{F_Z=F^*_Z} \notag\\
  &= \underset{h \to 0}{\lim}\frac{\bm{\theta}((1-h)F^*_Z + hG) - \bm{\theta}(F^*_Z)}{h}
\end{align}
Meaning: the sensitivity of the statistical functional $\bm{\theta}(F_Z)$ with respect to ``mixing the CDF with a little bit $G$''.
\end{frame}

\begin{frame}{Empirical Influence Function}{}
Let $\hat{F}_Z$ be the empirical CDF; let the direction $G$ to be "some probability mass" at a training point $\bm{z}_{tr}$.
\begin{align}\label{empinfluence}
  \mathcal{I}_{\fixwidehat{\bm{\theta}}}(\bm{z}_{tr}) = \underset{h \to 0}{\lim}\frac{\bm{\theta}((1-h)\fixwidehat{F}_Z+ h\delta_{\bm{z}_{tr}}) - \bm{\theta}(\fixwidehat{F}_Z)}{h}
\end{align}
\begin{itemize}
  \item $\delta_{\bm{z}_{tr}}$ is the Heaviside step function centered at $\bm{z}_{tr}$
  \item Meaning: the sensitivity of the statistical functional $\bm{\theta}(\fixwidehat{F}_Z)$ with respect to ``adding a little bit probability mess at $\bm{z}_{tr}$ into the empirical CDF''.
\end{itemize}
Similar to other statistical functionals, the prediction loss, for example:
    \begin{align}\label{lossinfluence}
  \mathcal{I}_{\mathcal{L}}(\bm{z}_{tr}, \bm{z}_{te}) = \underset{h \to 0}{\lim}\frac{\mathcal{L}(\bm{z}_{te}, \bm{\theta}((1-h)\fixwidehat{F}_Z + h\delta_{\bm{z}_{tr}})) - \mathcal{L}(\bm{z}_{te}, \bm{\theta}(\fixwidehat{F}_Z))}{h}
    \end{align}
\end{frame}

\subsection{How to Compute?}

% You can reveal the parts of a slide one at a time
% with the \pause command:
\begin{frame}{Naive Approach: LOO Refitting, But...}
 Take $h=-\frac{1}{n}\to0$, the ``contaminated CDF'' is:
 $$
 (1+1/n)\fixwidehat{F}_Z- \delta_{\bm{z}_{i}}/n = (1+1/n)\fixwidehat{F}_{-i} + o(1/n)\approx \widehat{F}_{-i}
 $$
 It's the leave-$i$-out empirical CDF!
\begin{align}\label{I_theta}
\mathcal{I}_{\fixwidehat{\bm{\theta}}}(\bm{z}_{i}) &= \underset{n \to \infty}{\lim}\frac{\bm{\theta}(\widehat{F}_{-i}) - \bm{\theta}(\fixwidehat{F}_Z)}{-1/n} \notag\\
 &= \underset{n \to \infty}{\lim}-n(\fixwidehat{\bm{\theta}}_{-\frac{1}{n}, \bm{z}_i} - \fixwidehat{\bm{\theta}})
\end{align}
\begin{itemize}
  \item $\fixwidehat{\bm{\theta}}_{-\frac{1}{n}, \bm{z}_i}$ is just the LOO estimate.
  \item Influence function is the asymptotic approximation of LOO difference in parameter estimate!
\end{itemize}
\end{frame}

\begin{frame}{But!}
\begin{itemize}
  \item If we want the influence from all training examples, we need to do LOO refit $n$-times!
  \item Super expensive. \textbf{NO}!
\end{itemize}
\end{frame}

\begin{frame}{Let's Solve the Easier Version First}
Suppose the loss is strictly convex, twice differentiable.
\begin{itemize}
  \item Use the FOC for the LOO empirical risk minimization problem $\Rightarrow$ A closed-form solution for LOO estimator $\fixwidehat{\bm{\theta}}_{-\frac{1}{n}, \bm{z}_i}$.
  \item Taylor approximation, drop high order infinitesimals...
  \item The LOO difference is approximately
\begin{align}
(\fixwidehat{\bm{\theta}}_{-\frac{1}{n}, \bm{z}_i} - \fixwidehat{\bm{\theta}}) \approx-\frac{1}{n}\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}})^{-1} \nabla_{\bm{\theta}} \mathcal{L}(\bm{z}_i, \fixwidehat{\bm{\theta}})
\end{align}
\item Plug into the asymptotic formula:
\begin{align}\label{I_theta}
\mathcal{I}_{\fixwidehat{\bm{\theta}}}&= \underset{n \to \infty}{\lim}-n(\fixwidehat{\bm{\theta}}_{-\frac{1}{n}, \bm{z}_i} - \fixwidehat{\bm{\theta}}) \\
&\approx -\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}})^{-1} \nabla_{\bm{\theta}} \mathcal{L}(\bm{z}_i, \fixwidehat{\bm{\theta}})
\end{align}
No retraining, only hessian and gradients.
\end{itemize}
\end{frame}

\begin{frame}{Same for the Influence on Loss}
\begin{align}
\label{I_loss}
\mathcal{I}_{\mathcal{L}}(\bm{z}_{tr}, \bm{z}_{te}) &= \nabla_{\delta_{\bm{z}_{tr}}} \mathcal{L}(\bm{z}_{te}, \bm{\theta}(F_Z))\biggr\rvert_{F_Z=\fixwidehat{F}_Z} \\[-2pt]
&= \nabla_{\bm{\theta}}\mathcal{L}(\bm{z}_{te}, \fixwidehat{\bm{\theta}})^{\top}\underset{h \to 0}{\lim}\frac{\bm{\theta}(\fixwidehat{F}_{-\bm{z}_{tr}}) - \bm{\theta}(\fixwidehat{F}_Z)}{h}\notag\\[0pt]
&\approx-\nabla_{\bm{\theta}}\mathcal{L}(\textcolor{red}{\bm{z}_{te}}, \fixwidehat{\bm{\theta}})^{\top}\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}})^{-1} \nabla_{\bm{\theta}} \mathcal{L}(\textcolor{blue}{\bm{z}_{tr}}, \fixwidehat{\bm{\theta}})\notag\\
&~\quad\qquad(1\times p)~~~~~(p\times p)~~~~~~(p\times 1)\notag
\end{align}
\begin{itemize}
  \item Nothing but a fancy version of \textit{chain rule}!
  \item Meaning: the validation loss of point $\textcolor{red}{\bm{z}_{te}}$'s sensitivity (w.r.t. to adding a little prob. mass at) training point $\textcolor{blue}{\bm{z}_{tr}}$.
\end{itemize}
\end{frame}

\section{Implementation}

\subsection{Exact Computation}
\begin{frame}{How to Compute $\mathcal{I}_{\mathcal{L}}(\bm{z}_{i}, \bm{z}_{te})$?}{}
Recall $n = \text{number of observations}$, $p=\text{number of parameters}$.
\begin{align}
\label{I_loss}
\mathcal{I}_{\mathcal{L}}(\bm{z}_{i}, \bm{z}_{te}) \approx-\underbrace{\nabla_{\bm{\theta}}\mathcal{L}(\textcolor{red}{\bm{z}_{te}}, \fixwidehat{\bm{\theta}})^{\top}\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}})^{-1}}_{\text{Does not depend on $i$}} \nabla_{\bm{\theta}} \mathcal{L}(\textcolor{blue}{\bm{z}_{i}}, \fixwidehat{\bm{\theta}})
\end{align}
\begin{itemize}
  \item We want this for all training points $\bm{z}_1, \bm{z}_2, ..., \bm{z}_n$.
  \item Idea: evaluate the part that does not depend on $i$ only \textbf{once}, then $n$ inner products. $O(np) + O(?)$.
  \item What is $O(?)$?
\end{itemize}
\end{frame}

\begin{frame}{How to Compute the part that does not depend on $i$?}{}
Let $\bm{s}_{te} := \underbrace{\nabla_{\bm{\theta}}\mathcal{L}(\textcolor{red}{\bm{z}_{te}}, \fixwidehat{\bm{\theta}})^{\top}\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}})^{-1}}_{\text{Does not depend on $i$}}$
\begin{itemize}
  \item $\nabla_{\bm{\theta}}\mathcal{L}(\textcolor{red}{\bm{z}_{te}}, \fixwidehat{\bm{\theta}})^{\top}$: just a 1 to $p$ gradient. $O(p)$.
  \item $\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}})^{-1}$: full hessian of the empirical risk.
  \begin{itemize}
    \item Empirical risk: \texttt{reduce\_sum}( $n$ things ). $\times n$
    \item Hessian of 1 ``thing'': 1 to $p$, $O(p^2)$
  \end{itemize}
  Forming the hessian: $O(np^2)$. Inverting the hessian $O(p^3)$.
  \item $O(np^2 + p^3)$ in total.
\end{itemize}

Too expensive! On \texttt{MNIST}, even multi-classes logistic regression has $p=(28^2+1)(10-1)=7065$.
\end{frame}

\subsection{Conjugate Gradients}

\begin{frame}{Conjugate Gradients}{}
$\bm{s}_{te} = \nabla_{\bm{\theta}}\mathcal{L}(\bm{z}_{te}, \fixwidehat{\bm{\theta}})^{\top}\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}})^{-1}$
is actually the solution to the linear system
$$
\nabla^2_{\bm{\theta}}R(\fixwidehat{\bm{\theta}}) \bm{x} = \nabla_{\bm{\theta}}\mathcal{L}(\bm{z}_{te}, \fixwidehat{\bm{\theta}})^{\top}
$$
\begin{itemize}
  \item This is a positive definite linear system, which is good.
  \item Linear conjugate gradients method (CG) is an efficient iterative algorithm to solve PD linear system $\bm{Ax} = \bm{b}$.
  \item Only requires quantities like $\bm{Av}$ for arbitrary vectors $\bm{v}$.
  \item No need for $\bm{A}$ itself!
\end{itemize}
Can avoid explicitly forming and inverting hessian matrix. Only need to implement $\bm{Hv}$ - \textbf{hessain-vector products} (HVPs).
\end{frame}

\begin{frame}{Hessain-Vector Product}{}
\begin{itemize}
  \item Easy to implement in auto-gradients systems like \texttt{Tensorflow}.
  \item Single HVP evaluation Cost is $O(np)$.
  \item If $m$ CG iterations, total cost is $O(nmp)$. (typically, $m=O(p)$)
  \item Should be a big improvement from $O(np^2 + p^3)$.
\end{itemize}
\end{frame}

\subsection{Stochastic Taylor Approximations}

\begin{frame}{Stochastic Taylor Approximations}{}
\begin{itemize}
  \item $\widehat{\bm{H}}$ be the full hessian matrix of empirical risk.
  \item If the Jordan normal form of $(\bm{I}-\fixwidehat{\bm{H}})$ is $\bm{J}$, then $\exists$ an invertible matrix $\bm{P}$ such that $(\bm{I}-\fixwidehat{\bm{H}}) = \bm{P}\bm{J}\bm{P}^{-1}$.
  \item $(\bm{I}-\fixwidehat{\bm{H}})^m = \bm{P}\bm{J}^m\bm{P}^{-1} \to 0$ as $m\to \infty$, if and only if $|\lambda| < 1$ for all the eigenvalues of $(\bm{I}-\fixwidehat{\bm{H}})$.
\end{itemize}
The idea is to use power series (which converges in this case) to approximate hessian inverse. $(\bm{I}-(\bm{I}-\fixwidehat{\bm{H}}))^{-1}=\fixwidehat{\bm{H}}^{-1} = \sum_{i=0}^{\infty} (\bm{I}-\fixwidehat{\bm{H}})^i$. The recursive form:
\begin{align}\label{lissarule}
\fixwidehat{\bm{H}}^{-1}_j \bm{v} &= \left[\bm{I} + (\bm{I} - \fixwidehat{\bm{H}})\fixwidehat{\bm{H}}^{-1}_{j-1}\right]\bm{v}\notag\\
\bm{p}_j &= \bm{v} + \bm{p}_{j-1} - \fixwidehat{\bm{H}} \bm{p}_{j-1}
\end{align}
\textit{LiSSA}: a stochastic (minibatch) version of the above. Look at only one training points in each iteration step. $O(np + dp)$.
\end{frame}

\section{Experiments}
\subsection{Ridge Regression Models}

\begin{frame}{Experiments}
   \begin{itemize}
       \item
        {
            Recall, $\mathcal{I}_{\fixwidehat{\bm{\theta}}}(\bm{z}_{i}) = {\lim}_{n \to \infty}-n(\fixwidehat{\bm{\theta}}_{-\frac{1}{n}, \bm{z}_i} - \fixwidehat{\bm{\theta}})$.
        }
        \item Influence function = Asymptotic approximation of the LOO difference (of the influenced statistical functional)!
        \item Same argument holds for $\mathcal{I}_{\mathcal{L}}(\bm{z}_{tr}, \bm{z}_{te})$.
        \item We test our influence function by plotting it against the LOO retraining difference.
   \end{itemize}
\end{frame}

\begin{frame}{Ridge Regression}
   \begin{itemize}
       \item Ridge regression empirical risk:
        \begin{align}\label{ridge_R}
        \vspace{-0.15in}
        R_{\text{ridge}}(\fixwidehat{\bm{\theta}}) = \frac{1}{n}\sum_{i=1}^n \left( (y_i - \bm{x}_i^{\top} \fixwidehat{\bm{\theta}} )^2 + \frac{\lambda}{n} \left\lVert\fixwidehat{\bm{\theta}}  \right\rVert_2^2\right)
        \vspace{-0.15in}
        \end{align}
        \item The influence has a closed-form:
        \begin{align}\label{ridge_I_loss}
        \vspace{-0.15in}
        \mathcal{I}_{\mathcal{L}_{\text{ridge}}}(\bm{z}_{tr}, \bm{z}_{te}) &= \frac{n}{2}\left[ -2\bm{x}_{tr}^{\top}(y-\bm{x}_{tr}^{\top}\fixwidehat{\bm{\theta}}) + \frac{2 \lambda}{n} \fixwidehat{\bm{\theta}}\right]^{\top}\left(\bm{X}^{\top} \bm{X} + \lambda \bm{I}_p\right)^{-1}\cdot\notag\\
        &\qquad \left[ -2\bm{x}_{te}^{\top}(y-\bm{x}_{te}^{\top}\fixwidehat{\bm{\theta}}) + \frac{2 \lambda}{n} \fixwidehat{\bm{\theta}}\right]
        \vspace{-0.15in}
        \end{align}
   \end{itemize}
\end{frame}

\begin{frame}{Ridge Regression}
   \begin{itemize}
       \item Tested on the \href{https://archive.ics.uci.edu/ml/datasets/forest+fires}{\texttt{ForestFires}} dataset, a regression task with 517 instances and 13 features. \item After training the model, compute the difference between the validation loss after doing actual LOO retaining and the original loss with $\fixwidehat{\bm{\theta}}$, for all training points.
        \item Plot the actual LOO difference against the influence to assess the accuracy of our influence functions with all three methods that we discussed.
   \end{itemize}

\begin{figure*}
\vskip 0.0in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{fig-ridge}}
\vskip -0.1in
\label{ridge_examples}
\end{center}
\vskip -0.25in
\end{figure*}
\end{frame}

\subsection{Hyperplane Classification}

\begin{frame}{Logistic Regression}
   \begin{itemize}
       \item The logistic empirical risk is:
        \begin{align}\label{logit_R}
        \vspace{-0.2in}
        R_{\text{logit}}(\fixwidehat{\bm{\theta}}) = \frac{1}{n}\sum_{i=1}^n \left(-y_i\fixwidehat{\bm{\theta}}^{\top}\bm{x}_i + \log(1+e^{\fixwidehat{\bm{\theta}}^{\top} \bm{x}_i})\right)
        \vspace{-0.2in}
        \end{align}
        \item
        {
        Let $\sigma(t)=1/(1+e^{-t})$, label $y\in \{0,1\}$. The influence also has a closed-form:
        \begin{align}\label{logit_I_loss}
        \vspace{-0.15in}
        \mathcal{I}_{\mathcal{L}_{logit}}(\bm{z}_{tr}, \bm{z}_{te}) &= -\bm{x}_{tr}^{\top}\left(\frac{1}{n}\sum_{i=1}^n \sigma(\fixwidehat{\bm{\theta}}^{\top}\bm{x}_i)\sigma(-\fixwidehat{\bm{\theta}}^{\top}\bm{x}_i)\bm{x}_i\bm{x}_i^{\top}\right)^{-1} \bm{x}_{te}\cdot\notag\\
        &(\sigma(\fixwidehat{\bm{\theta}}^{\top}\bm{x}_{tr})-y_{tr})(\sigma(\fixwidehat{\bm{\theta}}^{\top}\bm{x}_{te})-y_{te})
        \vspace{-0.15in}
        \end{align}
        }
   \end{itemize}
\end{frame}

\begin{frame}{Logistic Regression}
   \begin{itemize}
       \item
        {
        We fit an $L_2$-regularized logistic regression on the \href{http://yann.lecun.com/exdb/mnist/}{\texttt{MNIST}} dataset.
        \item Binary case for simplicity: digits 1 and 7. Let the label $y=1$ stand for digit 1 and $y=0$ for digit 7.
        }
        \item We plot the actual LOO difference against the influence function.
   \end{itemize}
\end{frame}

\begin{frame}{Logistic Regression}

\begin{figure*}
\vskip 0.0in
\begin{center}
\centerline{\includegraphics[width=3in]{fig-logit}\includegraphics[width=1.05in]{fig-logit-img}}
\vskip -0.1in
\label{logit_examples}
\end{center}
\vskip -0.25in
\end{figure*}

\end{frame}

\begin{frame}{Support Vector Machine}

\begin{itemize}
  \item Many loss functions are non-differentiable.
  \item Idea: construct a smooth function that \emph{converges} to the non-differentiable components.
  \item Linear SVC Empirical Risk (Non-differentiable):
  \begin{align}\label{svc_R}
\vspace{-0.2in}
R_{\text{sv}}(\fixwidehat{\bm{\theta}}) = \frac{1}{n}\sum_{i=1}^n \left[h\left(y_i(\bm{x}_i^{\top} \fixwidehat{\bm{\beta}} + \fixwidehat{\beta_0})\right) + \frac{\lambda}{2n} \left\lVert \bm{\beta}\right\rVert_2^2\right]
\vspace{-0.2in}
\end{align}
\item ``Smooth Hinge'': replace $h(x) = \max\{0, 1-x\}$ with $h_t(x) := t\log(1+\text{exp}(\frac{1-x}{t}))$.
\end{itemize}

\begin{figure*}
\vskip 0.0in
\begin{center}
\centerline{\includegraphics[width=1.0\columnwidth]{fig-svm}}
\vskip -0.1in
\label{fig_svm}
\end{center}
\vskip -0.2in
\end{figure*}

\end{frame}

\section{Applications}

\subsection{Understanding the Components of Influence}

\begin{frame}{Understanding the Components of Influence}
\begin{align}\label{logit_I_loss2}
\vspace{-0.15in}
\mathcal{I}'_{\mathcal{L}_{logit}}(\bm{z}_{tr}, \bm{z}_{te}) &= -\bm{x}_{tr}^{\top}\textcolor{red}{\nabla_{\bm{\theta}}^2 R(\fixwidehat{\bm{\theta}})^{-1}} \bm{x}_{te}\cdot\\
&y_{tr}\textcolor{blue}{\sigma(-y_{tr}\fixwidehat{\bm{\theta}}^{\top}\bm{x}_{tr})}\cdot y_{te}\textcolor{orange}{\sigma(-y_{te}\fixwidehat{\bm{\theta}}^{\top}\bm{x}_{te})}\notag
\vspace{-0.15in}
\end{align}

\begin{align}\label{logit_I_loss2}
\vspace{-0.15in}
\mathcal{I}'_{\mathcal{L}_{logit}}(\bm{z}_{tr}, \bm{z}_{te}) &= -[\textcolor{red}{\text{Hessian Inv}} \text{ scaled inner product}]\cdot \\
&(\textcolor{blue}{\text{training leverage}})(\text{sgn}(y_{tr}y_{te}))(\textcolor{orange}{\text{constant}})\notag
\vspace{-0.15in}
\end{align}

\begin{figure}[ht]
\vskip 0.0in
\begin{center}
\centerline{\includegraphics[width=4.5in]{fig-components}}
\vskip -0.1in
\label{logit_components}
\end{center}
\vskip -0.25in
\end{figure}
\end{frame}

\subsection{Understanding Model Behavior}
\begin{frame}{Understanding Model Behavior}{}
\begin{itemize}
  \item The influence function can help us get a better understanding upon how black-box models rely on and extrapolate from the training data.
  \item Logistic regression V.S. linear SVC.
\end{itemize}
\begin{figure}[ht]
\vskip 0.0in
\begin{center}
\centerline{\includegraphics[width=4.5in]{fig-app1}}
\vskip -0.1in
\label{logit_components}
\end{center}
\vskip -0.25in
\end{figure}

\end{frame}

\subsection{Software Package}
\begin{frame}{Software Package}{}
\begin{itemize}
  \item We built a generic empirical risk optimizing framework with \texttt{Tensorflow} to compute influence functions for various models.
  \item With the help of auto-gradients system, the required gradients and hessian-vector products are automatically kept track of.
  \item The user only need to define the empirical risk funtion and its parametrization.
  \item All of our code, data, and reproducible experiments are available in our \href{https://github.com/zedyang/46927-Project}{Github repository}: \textcolor{blue}{https://github.com/zedyang/46927-Project}.
  \item Feel free to download and playaround!
\end{itemize}
\end{frame}

\frame
{

\begin{center}
\LARGE
Thanks for your attention
\end{center}

}

\end{document}


\chapter{The State of the Art}

\section{Rogue Wave Research} \label{sec:sota-rogue}

\sidefigure{AI art generated by VQGAN + CLIP \citep{esser_taming_2021,radford_learning_2021}. Prompt: \emph{\enquote{offshore oil platform in a storm | rogue wave | Kodak}}.}[fig:vqgan-2]{
    \includegraphics[width=.9\linewidth]{vqgan-images/vqgan-44}
}
%
What people think of when they hear the term \enquote{rogue wave} or \enquote{freak wave} heavily depends on their personal context, even among experts.

In popular science, the term rogue wave is often used to describe any large ocean wave, and they are often credited to be responsible for the loss of ships and lives at sea \citep{didenkulova_catalogue_2019}. This is contrary to the scientific definition, which is a relative criterion based on the observed wave height $H$ and the height of the surrounding waves, characterized by the significant wave height $H_s$:

\begin{align}
    \text{\footnotesize\spacedlowsmallcaps{Rogue wave criterion}} && \frac{H}{H_s} > \kappa && \label{eq:rogue}
\end{align}

Usually, the rogue threshold $\kappa$ is taken to be $2.0$ or $2.2$ for crest-to-trough wave heights and $1.2$ for crest heights. The significant wave height $H_s$ is defined as 4 times the standard deviation of the surface elevation (see \figref{fig:anatomy} for an illustration of these quantities). This is roughly equivalent to the mean of the highest third of waves, which aligns with the average wave height reported by a trained observer \citep{holthuijsen_waves_2010}.

\begin{figure}
    \begin{sidecaption}{Anatomy of an Eulerian wave observation, in which the observer is fixed in space (like a seaward facing laser, or --- approximately --- a tightly moored wave buoy).}[fig:anatomy]
		\antimpjustification
		\includegraphics{sota/timeseries.pdf}
	\end{sidecaption}
\end{figure}

The definition \eqref{eq:rogue} immediately reveals the first fundamental issue in rogue wave research: most rogue waves are neither dangerous nor interesting. It is not noteworthy when a \SI{50}{\centi\metre} wave occurs in a \SI{20}{\centi\metre} sea state, but it is as much a rogue wave as a \SI{20}{\metre} wave in an \SI{8}{\metre} sea. For this definition to make sense, it implicitly encodes 2 fundamental assumptions:

\begin{enumerate}
    \item Small rogue waves are caused by the same generation mechanisms as big rogue waves.
    \item Waves above the rogue wave threshold are somehow fundamentally different from those below it.
\end{enumerate}

Both assumptions are non-trivial. In fact, the articles in \chapref{chap:main} present evidence that the second assumption does not hold throughout most sea states in the real ocean, and \chapref{chap:outro} discusses some of the implications.

Research interest in rogue waves was originally triggered by the indisputable measurement of a rogue wave at the Draupner oil rig in the North Sea in 1995, at a wave height of \SI{25.6}{\metre} and crest height of \SI{18.5}{\metre} during a storm with significant wave height of \SI{12}{\metre} \citep{sunde1995kjempebolger,haver2004possible}. With a relative crest height of \num{1.55}, this event would be extremely rare under the existing theory for linear, narrow-bandwidth waves \citep{longuet1952statisticaldistribution}. This disconnect sent the research community searching for a theory that attaches a higher probability to this and similar events.

This ultimately resulted in a debate on the fundamental nature of these waves: are they themselves extremely rare, or the conditions under which they are generated? Or in the words of \citeauthor{hayer2000freak}: \citebook{hayer2000freak}\sidenote[-4]{This is in fact an excellent question to address with machine learning. All we need to do is to see how well a model can reliably predict rogue waves given the sea state --- and hope that we have collected enough and the right kind of data.}. The following sections outline the ideas behind both hypotheses, and present the state of the art in rogue wave research.

\subsection{Linear Waves} \label{sec:waves-linear}

To lowest order, the properties of a 1-dimensional wave measurement (like a time series observation at a fixed location) are fully described by its spectral density $\mathcal{S}(f)$, often just called a \enquote{wave spectrum} (see \figref{fig:spectrum} for an example).

\begin{figure}
    \strictpagechecktrue
    \begin{sidecaption}{A typical bi-modal wave spectrum representing the overlap of swell and wind sea. Idealized Ochi-Hubble six-parameter wave spectrum with spectral peaks at periods \SI{6}{\second} and \SI{14}{\second} \citep{ochi_michel_k_six-parameter_1976}.}[fig:spectrum]
		\antimpjustification
		\includegraphics{sota/spectrum.pdf}
	\end{sidecaption}
\end{figure}

To see this, we adopt a simple model called the random phase-amplitude model \citep[see \eg][]{holthuijsen_waves_2010}: We view the wave train as a superposition of independent harmonics with frequency $f$, where each harmonic has an amplitude depending on the corresponding value of the wave spectrum $\mathcal{S}(f)$ and an independent, uniformly random phase $\phi \in (0, 2\pi)$. After all, many processes acting on waves in the real ocean are highly stochastic --- like wave generation from winds or scattering and refraction at a fractal coastline geometry --- so a random phase without a preferred value makes intuitive sense. In this case, the surface elevation $\eta$ is just the sum of each harmonic:

\begin{equation}
    \eta(t) = \sum_i \sqrt{2 \Delta f \mathcal{S}(f_i)} \sin(2\pi f_i t + \phi_i)
\end{equation}

\sidefigure{An ensemble of sea surface elevations drawn from the same wave spectrum (as shown in \figref{fig:spectrum}).}[fig:ensemble]{\includegraphics[width=\linewidth]{sota/ensemble.pdf}}[-1]
%
with time $t$, frequency $f$, frequency resolution $\Delta f$, and random phase $\phi$. In the case of a spectrum with many independent harmonics $f_i$ (as in the real ocean), this represents the sum of a large number of random variables with finite mean and variance. So per the central limit theorem, $\eta$ is a Gaussian random variable with zero mean and a variance that is fully determined by the significant wave height. This inherent stochasticity of random phases also implies that even under an identical spectrum no two wave fields will look exactly the same (\figref{fig:ensemble}).

In the limit of a narrow-band spectrum, the sea surface elevation only has one maximum / minimum per wave, and the wave heights and crest heights are Rayleigh distributed \citep{longuet1952statisticaldistribution,holthuijsen_waves_2010}:

\sidedef{Rayleigh wave distribution}{}{
\begin{align}
    \text{\footnotesize\spacedlowsmallcaps{Wave heights}} & & P(H / H_s > \kappa) &= \exp( -2 \kappa^2 ) \label{eq:rayleigh} \\
    \text{\footnotesize\spacedlowsmallcaps{Crest heights}} & & P(h / H_s > \kappa) &= \exp( -8 \kappa^2 )
\end{align}
}

These probability distributions will serve as the baseline for all further comparisons. They also tells us that, under these assumptions\sidenote[-1]{Assumptions behind Rayleigh-distributed wave heights:
\begin{renum}
    \item independent, non-interacting harmonics (linear waves);
    \item narrow spectral bandwidth.
\end{renum}
}, we would expect about \num{1} in \num{10000} waves to be a rogue wave (with a threshold $\kappa=2.0$ for waves and $1.2$ for crests) through mere random linear superposition.

Unfortunately, the real ocean is not so simple. One commonly violated assumption is that of \emph{narrow bandwidth}, which is used to derive the Rayleigh wave height distribution above. In fact, most seas do \emph{not} have Rayleigh distributed wave heights, as we will see in \chapref{chap:main} (not even seas that are approximately Gaussian). In particular, to create a rogue wave, both crest and trough have to be large, which makes them sensitive to the group structure of the wave train.

\sidefigure{Wave height probability density (top) and survival function for large wave heights (bottom). Curves are Tayfun distributions as in \eqref{eq:tayfun} with different values of $r$. The case $r=1$ is identical to the Rayleigh distribution \eqref{eq:rayleigh}.}[fig:wavedist]{\includegraphics[width=\linewidth]{sota/wavedist.pdf}}
%
When taking finite bandwidths into account, things are more difficult, and there are several competing wave height distributions in bandwidth-limited seas \citep[\eg the Boccotti, Naess, and Tayfun distributions:][]{naess_distribution_1985,boccotti_mechanics_1989,tayfun_m._aziz_distribution_1990}. As an example, the Tayfun distribution is based on a parameter $r$ (which we call \emph{crest-trough correlation}) that is the value of the wave envelope at half the zero-crossing period (\ie at the expected location of the trough following a crest). For large wave heights $\gtrsim H_s$ it can be approximated as \citep{tayfun_wave-height_2007}:

\begin{equation}
    P(H / H_s > \kappa) = \sqrt{\frac{1 + r}{2 r}} \bigg( 1 + \frac{1-r^2}{4r\kappa^2} \bigg) \exp\bigg( -\frac{1}{4(1+r)} \kappa^2 \bigg) \label{eq:tayfun}
\end{equation}

with $r \in [0, 1]$. In the limit $r \to 1$ this reduces to the Rayleigh distribution for wave heights (\figref{fig:wavedist}).

\subsection{The Stokes Wave}

So far we have only considered waves and crests with independently random phases. This assumption is not fulfilled anymore as soon as waves are allowed to interact with each other, which couples the phases of different harmonics. Stokes theory extends this to weakly nonlinear waves with low characteristic steepness $\varepsilon = k H$ (with wave number $k$).

As in virtually all problems in fluid dynamics, an appropriate starting point is with the incompressible Navier-Stokes equations and the continuity equation (encoding momentum balance and mass conservation, respectively):

\sidedef{Navier-Stokes equations}{}{
\begin{gather}
\frac{\partial \vec{u}}{\partial t} + (\vec{u} \cdot \nabla) \vec{u} - \nu \nabla^2 \vec{u} = -\frac{1}{\rho} \nabla p - g \cdot \hat{k} \\
\nabla \cdot \vec{u} = 0
\end{gather}
}

with velocity vector $\vec{u}$, viscosity $\nu$, pressure $p$, density $\rho$, gravitational acceleration $g$, and unity vector in $z$ direction $\hat{k}$.

Assuming inviscid ($\nu=0$) and irrotational ($\nabla \times \vec{u} = 0$) fluid flow, we can introduce a velocity potential $\phi$:

\sidedef{Velocity potential}{}{
\begin{equation}
    \nabla \phi = \vec{u}
\end{equation}
}

This reduces the Navier-Stokes equations to the Bernoulli equation, and the continuity equation to a Laplace equation \citep[see \eg][]{holthuijsen_waves_2010}:

\sidedef{Bernoulli equation}{}{
\begin{gather}
\frac{\partial \phi}{\partial t} + \frac{1}{2} \lvert \nabla \phi
\rvert^2 \frac{p}{\rho} + gz = 0  \label{eq:bernoulli} \\
\nabla^2 \phi = 0
\end{gather}
}

A central missing ingredient is a set of boundary conditions at the top and bottom of the sea that give rise to finite surface elevations --- waves --- and shallow water effects. At each boundary we impose a kinematic boundary condition that ensures that water particles only move parallel to the respective surface $\eta(x, y, t)$:

\sidedef{Kinematic boundary condition}{}{
\begin{align}
    & u_z = \frac{\partial \eta}{\partial t} + u_x \frac{\partial \eta}{\partial x} + u_y \frac{\partial \eta}{\partial y} & \text{at $z=\eta$}\\
    \Leftrightarrow\quad & \frac{\partial\phi}{\partial z} = \frac{\partial \eta}{\partial t} + \frac{\partial \phi}{\partial x} \frac{\partial \eta}{\partial x} + \frac{\partial \phi}{\partial y} \frac{\partial \eta}{\partial y} & \text{at $z=\eta$} \label{eq:bc-kin}
\end{align}
}

For a flat bottom this just reduces to $\partial \phi / \partial z = 0$ at the ocean floor, but at the surface all terms are generally non-zero. At the surface we also find a dynamic boundary condition for the pressure $p$ that we plug into the Bernoulli equation:

\sidedef{Dynamic boundary condition}{}{
\begin{equation}
    p = 0 \quad\Rightarrow\quad \frac{\partial \phi}{\partial t} + \frac{1}{2} \lvert \nabla \phi \rvert^2 + g \eta = 0 \qquad \text{at $z=\eta$} \label{eq:bc-dyn}
\end{equation}
}

This assumes that the pressure at the water surface equals a constant atmospheric pressure.

The set of equations \eqref{eq:bernoulli}--\eqref{eq:bc-dyn} gives rise to a whole zoo of surface gravity waves in the ocean\sidenote[-1]{Excluding planetary-scale waves like Rossby and Kelvin waves, and neglecting interactions with bottom topograpy and breaking waves.}. The equations are nonlinear (containing terms $\propto \lvert \phi \rvert^2$ and $\nabla \phi \cdot \eta$) and cannot be solved analytically without further assumptions. The linear wave solution with non-interacting harmonics (as in \secref{sec:waves-linear}) is recovered by dropping all nonlinear terms and using the plane wave ansatz $\eta(x, t) = a \cos(\omega t - k x)$ with amplitude $a$, frequency $\omega$, and wave number $k$.

In the Stokes wave expansion, all nonlinear terms and unknown quantities (such as $\eta$ and $\omega$) are expanded in orders of the (assumed) small parameter $\varepsilon=ak$, the characteristic wave steepness \citep[see \eg][]{dean_water_1991}. By keeping only terms up to a certain order $n$ in $\varepsilon$, this leads to weakly nonlinear corrections of $n$-th order that generate wave trains with higher crests and flatter troughs than purely linear waves.

Weakly nonlinear corrections also cause a modification of the wave height distribution and enhance rogue wave probabilities, especially for rogue crests \citep{gemmrich_dynamical_2011,fedele_real_2016,fedele_large_2019}. This leads to conditions that have slightly elevated rogue wave probabilities, which supports the \enquote{rare realizations of a typical population} theory of rogue waves.

\subsection{Cnoidal Waves}

In shallow water the Stokes expansion converges very slowly, which makes Stokes theory inapplicable in this case (see \figref{fig:wave-regimes} for an overview). A characteristic parameter in this context is the Ursell number \citep{ursell_long-wave_1953}:

\sidedef{Ursell Number}{}{
\begin{equation}
    \mathrm{Ur} = \frac{\lambda^2 H}{D^3}
\end{equation}
}

with wavelength $\lambda = 2\pi/k$, wave height $H$, and water depth $D$. For high values of $\mathrm{Ur}$, an expansion in the relative depth $\widetilde{D}=kD$ is more fruitful than the Stokes expansion \citep{dean_water_1991}, which leads to the Korteweg-de Vries (KdV) equation and cnoidal theory \citep{korteweg1895xli}. Notably, cnoidal theory is the simplest theory that allows for solitary waves (solitons) --- waves that travel entirely above the water level and preserve their shape. Solitons have been studied intensely as a possible mechanism for rogue wave generation \citep{clamond_interaction_2002,kharif_physical_2003,chabchoub_rogue_2011}.

\begin{figure}
    \centering
    \includegraphics[width=.7\textwidth]{sota/wave-regimes.png}
    \caption{The range of applicability for different weakly nonlinear theories. From \citet{holthuijsen_waves_2010}, originally \citet{mehaute_introduction_2013}. Here, $T$ is the wave period, $H$ wave height, $d$ water depth, $L$ wavelength.} \label{fig:wave-regimes}
\end{figure}

\subsection{Highly Nonlinear Theory}

A large body of rogue wave research does not consider linear and weakly nonlinear solutions, as it is implicitly assumed that these mechanisms cannot be responsible for observed extreme rogue waves like the Draupner wave. Instead, these studies focus on highly nonlinear phenomena\sidenote[-3]{Highly nonlinear in the sense that these waves are not just small perturbations to the linear wave profile, but entirely new solutions with unique properties.} such as breathers, solitons, or the modulational instability as possible creation mechanisms \citep[\eg][]{kharif_physical_2003,onorato_extreme_2006,onorato_approximate_2012,toffoli_evolution_2010,shukla_instability_2006,kharif_focusing_2001,dematteis_experimental_2019}.

A prototypical framework for these solutions is the nonlinear Schrödinger equation (NLS), which is also based on an expansion in orders of characteristic steepness $\varepsilon$ and an expansion of the dispersion relation around a dominant wave number $k_0$ / frequency $\omega_0$ \citep[see \eg][for a derivation]{johnson_modern_1997}. In contrast to the Stokes wave solution, the (now complex) wave amplitude $A(x, t)$ is allowed to evolve in time and space and satisfies the nonlinear Schrödinger equation \citep{slunyaev_rogue_2011}:

\sidedef{Nonlinear Schrödinger equation}{}{
\begin{equation}
    -2i \bigg(\frac{\partial A}{\partial t} + c_g \frac{\partial A}{\partial x} \bigg) + \frac{\omega_0}{8 k_0^2} \frac{\partial^2 A}{\partial x^2} + \frac{\omega_0 k_0^2}{2} A \lvert A \rvert^2 = 0
\end{equation}
}

with group speed $c_g$. This equation has solutions that grow exponentially due to energy transfer between the carrier wave and its sidebands, an effect called modulational instability or Benjamin-Feir instablity \citep{benjamin_disintegration_1967}. These solutions are referred to as breathers, one of which is the Peregrine soliton \citep[\figref{fig:peregrine}]{peregrine_water_1983}. The strength of the modulational instability is governed by the Benjamin-Feir index \citep{alber_effects_1978}:
%
\sidefigure{The Peregrine solution. Shown is the evolution of the wave height envelope $A$ in space and time, with a clear localized maximum.}[fig:peregrine]{
    \includegraphics{sota/peregrine.pdf}
}[3.3]

\sidedef{Benjamin-Feir index}{}{
    \begin{equation}
        \mathrm{BFI} = \frac{k_0 A}{\Delta\omega / \omega_0}
    \end{equation}
}

with spectral bandwidth $\Delta \omega$. The original derivation of the nonlinear Schrödinger equation assumes deep water, unidirectional propagation, and narrow-banded spectra. Modifications that relax these assumptions exist \citep{davey_three-dimensional_1974,dysthe_note_1979}, and modified versions of the BFI that take shallow water and directional spreading into account have been suggested \citep{serio_computation_2005,fedele_kurtosis_2015}. There is good evidence demonstrating the modulational instability in wave tanks \citep{onorato_extreme_2006}, but studies considering real ocean conditions have so far not confirmed an enhancement of extreme waves \citep{gramstad_influence_2007,xiao_rogue_2013}.

The nonlinear Schrödinger equation is not the only nonlinear wave equation with unstable solutions. In general, waves transfer energy via nonlinear four-wave interactions \citep{hasselmann_feynman_1966}. This is accounted for explicitly in the Zakharov equation \citep{zakharov_stability_1968}, which can be studied to derive higher-order corrections to the wave height distribution \citep[\eg as in][]{janssen_nonlinear_2003}.

\subsection{Other Causes of Rogue Waves}

There are several other hypothesized causes for rogue waves that we have not considered so far \citep[see][for reviews]{adcock_physics_2014,slunyaev_rogue_2011,dudley_rogue_2019}. While the Bernoulli equations \eqref{eq:bernoulli} and associated boundary conditions are very general in terms of the permitted dynamics \emph{within} the fluid, most of the real-world complexities \emph{outside} the fluid are neglected. Examples for this include:

\begin{items}
    \item Interactions with non-uniform topography such as abrupt transitions in water depth or waves on top of a slope \citep{trulsen_laboratory_2012};
    \item The non-stationarity of the sea state, \ie its evolution in time \citep{trulsen_rogue_2018};
    \item The interaction between waves and currents \citep{mallory_abnormal_1974,didenkulova_rogue_2021,onorato_triggering_2011};
    \item Direct wind-wave interactions \citep{adcock_energy_2011};
    \item Wave breaking, \eg the influence of crossing seas on the onset and shape of breaking waves \citep{mcallister_laboratory_2019}.
\end{items}

All of these effects impact the formation of large waves, but they are also inherently \emph{local}, which causes them to be averaged out of bulk statistics (such as buoy measurements from many different locations).

\clearpage

\section{Physics and Machine Learning} \label{sec:sota-ml}

\sidefigure{AI art generated by VQGAN + CLIP \citep{esser_taming_2021,radford_learning_2021}. Prompt: \emph{\enquote{a cartoon robot surfing on a big wave}}.}{
    \includegraphics[width=.9\linewidth]{vqgan-images/vqgan-24.png}
}
%
The are many examples of studies that apply machine learning to physical problems, most of which aim for improvements in computational efficiency or predictive performance of simulations \citep[\eg][]{pestourie_physics-enhanced_2021,bar-sinai_data-driven_2018,kochkov_machine_2021,li_kohn-sham_2020,cranmer_bayesian_2021,cranmer_lagrangian_2020}.
These efforts undoubtedly contribute tremendous value. Yet, better \emph{predictions} are not the same as improved \emph{understanding}, the foundation of all science. Ideally, machine learning would lead to advances on both fronts, but unfortunately, process understanding seems much harder to come by, in part also due to the immense complexity of real-world data and governing processes \citep[\figref{fig:ml-challenges};][]{reichstein_deep_2019}.

\begin{figure}
    \begin{sidecaption}{Challenges when applying machine learning to earth system data. Figure from \citet{reichstein_deep_2019}.}[fig:ml-challenges]
        \antimpjustification
        \includegraphics[width=\textwidth]{sota/ml-challenges.png}
    \end{sidecaption}
\end{figure}

One necessary ingredient for true understanding is the robust identification of causal connections over mere association. The emerging field of causality has formalized the identifiability of causal connections from data and provides tools for both causal inference and causal discovery \citep[see \eg][]{peters_elements_2017}. There are first promising applications of these methods \citep[for example in climate:][]{hannart_causal_2016,kretschmer_using_2016,runge_inferring_2019}, but it is still a long way to go before we will be able to identify arbitrary causal connections in real-world spatiotemporal systems. Nevertheless, explicitly encoding or enforcing causal relationships in models is a promising way to make machine learning a more dependable tool for scientific discovery.

Causal connections in physical systems are typically representable by simple mathematical relationships\sidenote[-1]{An observation dubbed \citebook{wigner_unreasonable_1960}.}. Machine learning can exploit this through symbolic regression, a method that aims to fit sparse mathematical expressions to data. While the idea itself is not new \citep[traditionally based on genetic programming,][]{schmidt_distilling_2009}, it is now elevated through increasingly sophisticated machine learning algorithms, with some first successes \citep{udrescu_ai_2020,zanna_data-driven_2020,lemos_rediscovering_2022,cranmer_discovering_2020}.

An in physics ubiquitous special case is systems of differential equations, which can be replaced or augmented with neural networks \citep[neural ODEs / UDEs,][]{chen_neural_2019,rackauckas_universal_2021,kidger_neural_2022}. In combination with symbolic regression, this leads to methods for the automated discovery of differential equations (and thus system dynamics) from data, an approach that shows huge potential \citep{brunton_discovering_2016,long_pde-net_2019,champion_data-driven_2019,bakarji_discovering_2022,reinbold_robust_2021} but is still very much a matter of active research, and still struggles with observational noise.
%
\sidefigure{The increasing resolution of climate models over time leads to exponentially increasing data volumes. Horizontal resolution over Europe from IPCC (Intergovernmental Panel on Climate Change) reports FAR (1990), SAR (1996), TAR (2001a), and AR4 (2007). Figure from \citet{change2007climate}.}[fig:climate-res]{
    \includegraphics[width=.9\linewidth,trim=5 5 5 5,clip]{sota/climate-res.jpeg}
}[-12]

A methodologically much simpler approach is \enquote{data-mining inspired induction} \citep{voit_perspective_2019}, where interpretable machine learning \citep{molnar_interpretable_nodate} and data mining guide the scientist towards the formation of hypotheses that can then be independently verified --- as opposed to setting out with a specific hypothesis to test.

Data-mining inspired induction addresses a common challenge in modern science: data volumes have increased exponentially in the last decades (\figref{fig:climate-res}), while human resources are approximately fixed. The central idea is to combine the strengths of machine learning models (large-scale data analysis taking into account orders of magnitude more data than the human mind could) and humans (causal reasoning and interpretation) into a modern scientific workflow. The remainder of this thesis is a concrete application of this approach and serves as a case study on its feasibility, challenges, and opportunities on a real physical problem.


% This LaTeX was auto-generated from an M-file by MATLAB.
% To make changes, update the M-file and republish this document.

      \subsection{value.m}

\begin{par}
\textbf{Summary:} Compute expected (discounted) cumulative cost for a given (set of) initial state distributions
\end{par} \vspace{1em}

\begin{verbatim}   function [J, dJdp] = value(p, m0, S0, dynmodel, policy, plant, cost, H)\end{verbatim}
    \begin{par}
\textbf{Input arguments:}
\end{par} \vspace{1em}
\begin{verbatim}p            policy parameters chosen by minimize
policy       policy structure
  .fcn       function which implements the policy
  .p         parameters passed to the policy
m0           matrix (D by k) of initial state means
S0           covariance matrix (D by D) for initial state
dynmodel     dynamics model structure
plant        plant structure
cost         cost function structure
  .fcn       function handle to the cost
  .gamma     discount factor
H             length of prediction horizon\end{verbatim}
\begin{par}
\textbf{Output arguments:}
\end{par} \vspace{1em}

\begin{verbatim}J             expected cumulative (discounted) cost
dJdp          (optional) derivative of J wrt the policy parameters\end{verbatim}
    \begin{par}
Copyright (C) 2008-2013 by Marc Deisenroth, Andrew McHutchon, Joe Hall, and Carl Edward Rasmussen.
\end{par} \vspace{1em}
\begin{par}
Last modification: 2013-03-21
\end{par} \vspace{1em}

\subsection*{High-Level Steps}

\begin{enumerate}
\setlength{\itemsep}{-1ex}
   \item Compute distribution of next state
   \item Compute corresponding expected immediate cost (discounted)
   \item At end of prediction horizon: sum all immediate costs up
\end{enumerate}

\begin{lstlisting}
function [J, dJdp] = value(p, m0, S0, dynmodel, policy, plant, cost, H)
\end{lstlisting}

\subsection*{Code}

\begin{lstlisting}
policy.p = p;            % overwrite policy.p with new parameters from minimize
p = unwrap(policy.p); dp = 0*p;
m = m0; S = S0; L = zeros(1,H);

if nargout <= 1                                       % no derivatives required

  for t = 1:H                                  % for all time steps in horizon
    [m, S] = plant.prop(m, S, plant, dynmodel, policy);      % get next state
    L(t) = cost.gamma^t.*cost.fcn(cost, m, S);     % expected discounted cost
  end

else                                               % otherwise, get derivatives

  dmOdp = zeros([size(m0,1), length(p)]);
  dSOdp = zeros([size(m0,1)*size(m0,1), length(p)]);

  for t = 1:H                                  % for all time steps in horizon
    [m, S, dmdmO, dSdmO, dmdSO, dSdSO, dmdp, dSdp] = ...
      plant.prop(m, S, plant, dynmodel, policy); % get next state

    dmdp = dmdmO*dmOdp + dmdSO*dSOdp + dmdp;
    dSdp = dSdmO*dmOdp + dSdSO*dSOdp + dSdp;

    [L(t), dLdm, dLdS] = cost.fcn(cost, m, S);              % predictive cost
    L(t) = cost.gamma^t*L(t);                                      % discount
    dp = dp + cost.gamma^t*( dLdm(:)'*dmdp + dLdS(:)'*dSdp )';

    dmOdp = dmdp; dSOdp = dSdp;                                 % bookkeeping
  end

end

J = sum(L); dJdp = rewrap(policy.p, dp);
\end{lstlisting}