% Experimental evaluation
\section{ Experimental Evaluation}
\label{sec:results}

We ran experiments on real world datasets to
evaluate the performance of the algorithm. All the experiments were run
on an 4GB Intel Core i7 machine with a clock speed of $2.67$ GHz running
Ubuntu Linux 10.04. The code was written in C++ and
compiled using g++ version $4.4$ with -O3
optimization flag. The default number of random walks is $K=500$.

\begin{comment}
\begin{table}[!h]
  \centering
    \begin{tabular}{|c|c|c|c|c|}
      \hline
      Dataset & $|V|$ & $|E|$ & $|\Sigma|$ & \small{Preprocessing time(sec)} \\
      \hline
      CMDB & $10466$ & $15122$ & $84$ & $329.31$ \\
      SCOP & $39256$ & $154328$ & $20$ & $17.377$ \\
      PPI & $4950$ & $16515$ & $4950$ & $339.45 $\\
      %Synthetic & $2755$ & $5002$ & $60$ \\
	  \hline
    \end{tabular}
    \caption{Dataset Properties}
	\label{tab:db}
\end{table}
\end{comment}

\begin{figure}[!h]
\centering
\subfloat[Dataset Statistics] {
	\label{subfig:dataset}
	\begin{tabular}{|c|c|c|c|c|}
	\hline
		Dataset & $|V|$ & $|E|$ & $|\Sigma|$ & \small{Preprocessing time(sec)} \\
		\hline
		CMDB & $10466$ & $15122$ & $84$ & $329.31$ \\
		SCOP & $39256$ & $154328$ & $20$ & $17.377$ \\
		PPI & $4950$ & $16515$ & $4950$ & $339.45 $\\
		%Synthetic & $2755$ & $5002$ & $60$ \\
		\hline
		\end{tabular}
} \\
\subfloat[Maximal Pattern Statistics] {
	\label{subfig:maxpat_stats}
	\begin{tabular}{|c|c|c|c|}
	\hline
		Dataset & $|V|$ & $|E|$ & Degree\\
		\hline
		CMDB & $11$ & $12.24$ & $2.223$\\
		SCOP & $5.965$ & $6.725$ & $2.225$\\
		PPI & $6.453$ & $5.956$ & $1.655$\\
		%Synthetic & $2755$ & $5002$ & $60$ \\
		\hline
		\end{tabular}
}
  \caption{ \protect\subref{subfig:dataset}: Input graph statistics,
    \protect\subref{subfig:maxpat_stats}: Maximal pattern statistics (the numbers
    shown are average values).
  }
  \label{fig:stats}
\end{figure}

\subsection{Configuration Management Database\\ (CMDB)}

A CMDB is used to manage and query the IT infrastructure of an
organization. It stores information about the so-called configuration
items (CIs) -- servers, software, running processes, storage systems,
printers, routers, etc. As such it can be considered to be a single
large multi-attributed graph, where the vertices represent the various CIs
and the edges represent the connections between the CIs (e.g., the
processes on a particular server, along with starting and ending times).
Mining such graphs is challenging because they are large, complex,
multi-attributed, and have many repeated labels.  We used a real-world
CMDB graph for a large multi-national corporation (name not revealed due
to non-disclosure issues) from HP's Universal Configuration Management
Database (UCMDB).  Table~\ref{tab:db} shows the size of the CMDB graph.

\smallskip\noindent{\textit{Cost Matrix}:}
The set of labels in a CMDB form a
hierarchy which can be obtained from HP's UCMDB. In the absence of
domain knowledge, one way to obtain a cost matrix is by assigning low
costs for pairs of labels that share many ancestors in the hierarchy and
high costs otherwise. The algorithm is general in that it doesn't depend
on how the label matching costs are assigned, the range of these
values or whether the cost matrix is symmetric
.  Consider any two labels $l_1$, $l_2$ and their corresponding
paths $p_1$, $p_2$ to the root vertex in the hierarchy.  We first define
the similarity between the labels to be proportional to the number of
common labels in $p_1 \cap p_2$, as follows
\begin{equation*}
  sim(l_1,l_2) =  \frac{|p_1 \cap p_2|}{2} \times
  \left(\frac{1}{|p_1|} + \frac{1}{|p_2|}\right)
\end{equation*}
The cost of matching the labels is then
$\matij{C}{l_1}{l_2} = 1 - sim(l_1,l_2)$.

\begin{figure}[!ht]
  \centerline{
    \includegraphics[width=2.5in]{ge.eps}
	}
    \caption{CMDB: Time for different values of
	$minsup$}
    \label{fig:ge}
\end{figure}

\smallskip\noindent{\textit{Results}:} Figure \ref{fig:ge} shows the
time for random walks for different values of $minsup$ and $\alpha =
0.5$. Interesting, and somewhat counter-intuitively, the time increases
for higher minimum support values.  The reason is that with higher
minimum support, the random walk in the search space goes through the
vertices that have a large number of embeddings and it takes more time to
enumerate a single pattern.

%%%%%%%% Orbits idea explained as an optimization in the mining section
\begin{comment}
The running time results above include a particular optimization that we
applied for CMDB graphs, given the large multiplicities of the different
labels. For such graphs, the
support computation procedure can be improved by computing the sets of
equivalent vertices, i.e., vertices that have the same neighborhood and
are indistinguishable. The representative set $R(u)$ of all equivalent
vertices are equal, and thus it has to be computed only once.  In
abstract algebra terms, such vertices belong to an orbit of the
automorphism group of the graph \cite{orbits}.  So, we can prune the
candidate representative sets of all the vertices in an orbit by
matching the labels of any vertex in the orbit with the labels of
vertices in the candidate set. Computing the orbits of an arbitrary
graph is a hard problem. Several heuristics have been proposed to
compute the orbits of a graph \cite{Everett}.  We use a simple heuristic
to find subsets of vertices, common ancestor leaves (CAL) that are
guaranteed to be in the same orbit. These are the subset of leaves that
have the same label and are connected to a common ancestor.  CAL is a
subset $S \subseteq \vg$, such that $\forall u \in S$ and the following
three properties hold true: i) $|N(u)|=1$, ii) $\exists v \in \vg$,
$(v,u) \in \eg$, and  iii) $L(u) = l$  for some $l \in \Sigma$.
An example of CAL is set of all vertices labeled ``9 $\times$ process'' in
figure \ref{fig:gepatsB}; here 9 is the multiplicity of that label.
CAL sets have to be computed once for every
candidate and this step had negligible impact on the overall run time.
\end{comment}

\begin{figure}[!h]
    \centerline{
    \subfloat[Pattern $A$] {
	\label{fig:gepatsA}
      \scalebox{0.6}{
        \begin{pspicture}(-2,-0.5)(3,5)
          \begin{psmatrix}[rowsep=1,colsep=1]
          \Toval[name=n4]{running\_software} & & \\
             \Toval[name=n1]{windows\_service} & \Tcircle[name=n2]{iis} &
            \Toval[name=n3]{iisftpservice}\\
            \Tcircle[name=n5]{nt} & & \\
            \Toval[name=n6]{ip\_address} & \Toval[name=n7]{webvirtualhost} &
            \Toval[name=n8]{iiswebsite}
            %\Toval[name=n1]{business\_application} & \Toval[name=n2]{windows\_service}\\
            %\Tcircle[name=n3]{nt} & \\
            %\Toval[name=n5]{sqlserver} & \Toval[name=n4]{process}\\
            %& \Toval[name=n6]{windows\_service $\times$ 8}
          \end{psmatrix}
          \ncline{n1}{n4}
          \ncline{n1}{n2}
          \ncline{n2}{n3}
          \ncline{n2}{n5}
          \ncline{n2}{n7}
          \ncline{n1}{n5}
          \ncline{n1}{n4}
          \ncline{n6}{n5}
          \ncline{n7}{n6}
          \ncline{n7}{n8}
          \ncline{n3}{n8}
        \end{pspicture}
      }
	  }}
	\centerline{
    \subfloat[Pattern $B$] {
	\label{fig:gepatsB}
      \scalebox{0.5}{
        \begin{pspicture}(-2,-1)(4,7)
          \begin{psmatrix}[rowsep=1,colsep=1]
            & \Toval[name=n1]{9 $\times$ process} & & \Toval[name=n2]{ip\_address} \\
            & \Toval[name=n3]{windows\_service} & \Tcircle[name=n4]{nt} &
            \Toval[name=n5]{ip\_address} \\
            \Toval[name=n6]{iisftpservice} & \Tcircle[name=n7]{iis} & &
            \Toval[name=n8]{webvirtualhost} \\
            & & \Toval[name=n9]{iisappool} & \\
            & \Toval[name=n10]{iisftpservice} & & \Toval[name=n11]{iiswebsite} \\
          \end{psmatrix}
        \ncline{n1}{n4}
        \ncline{n2}{n4}
        \ncline{n3}{n4}
        \ncline{n5}{n4}
        \ncline{n6}{n7}
        \ncline{n3}{n7}
        \ncline{n8}{n7}
        \ncline{n7}{n9}
        \ncline{n11}{n9}
        \ncline{n11}{n8}
        \ncline{n7}{n10}
        \ncline{n10}{n11}
        \ncline{n5}{n8}
        \end{pspicture}
      }
	  }}
    \caption{CMDB: Approximate Patterns}
    \label{fig:gepats}
  \end{figure}

\smallskip\noindent{\textit{Example Patterns}:}
Figure \ref{fig:gepats} shows maximal approximate
patterns from the real world CMDB graph. Both these patterns show
typical ``default''
configurations of the IT infrastructure in this company. They show the
connection between some services running on an NT server, and also the
web/ftp services. The node with label \textit{9 $\times$ process}
in figure \ref{fig:gepatsB} indicates that there are nine nodes
in the maximal pattern with label \textit{process} all of which
are connected to a common node. This is an example where the run time
for computing the representative sets is significantly reduced by
the optmization proposed in section \ref{sec:labelcheck}. All of the nine
nodes belongs to the same orbit and hence their representative sets are
identical.

\begin{figure}
    \centering
    \scalebox{0.6}{
    \begin{pspicture}(0,0)(3,5)
          \begin{psmatrix}[rowsep=1,colsep=1]
          & \Toval[name=n1]{running\_software} & \\
          \Toval[name=n2]{EnrichActImpl} & & \Toval[name=n3]{process}\\
          \Toval[name=n4]{process} & \Toval[name=n5]{nt} &
          \Toval[name=n6]{ip\_address}\\
          & \Toval[name=n7]{$8 \times process$} & \\
          \end{psmatrix}
          \ncline{n1}{n2}
          \ncline{n2}{n3}
          \ncline{n2}{n4}
          \ncline{n2}{n5}
          \ncline{n4}{n5}
          \ncline{n5}{n6}
          \ncline{n5}{n3}
          \ncline{n5}{n1}
          \ncline{n5}{n7}
    \end{pspicture}
    }
    \caption{Complete Enumeration Expensive}
    \label{fig:geex}
\end{figure}

To show the effectiveness of the pruning based on labels, we compared
the time taken to enumerate a single maximal pattern in CMDB database.
We compared the time with and without label-based pruning.
Both the methods terminated the random walk with the maximal pattern
shown in Figure~\ref{fig:geex}.
However, the total time taken to enumerate the pattern
without using any derived label is $18306$ secs whereas by using the
\ncl label the total time reduced to only $15.5776$ secs. The huge
difference between the times arises due to the multiplicity effect in
CMDB graphs.

%%% compare the performance of gapprox, label pruning and no pruning

\subsection{Protein Structure Dataset (SCOP)}
SCOP (\url{scop.mrc-lmb.cam.ac.uk/scop/})
is a hierarchical classification of proteins based on structure
and sequence similarity. The four levels of hierarchy in this
classification are: class, fold, superfamily and family.  The $3D$
structure of a protein can be represented as an undirected graph with
the vertex labels being the amino acids, with an edge connecting two
nodes if the distance between the
3D coordinates of the two amino acids (their
$\alpha$-Carbon atoms) is within a threshold (we use 7 Angstroms).
We constructed a database of
$100$ protein structures belonging to $5$ different families with $20$
proteins from each family.
We chose the proteins from different levels in the SCOP hierarchy, and
we also focused on large proteins (those with more than 200 amino
acids).
The 3D protein structures were downloaded from the
protein data bank (\url{http://www.rcsb.org/pdb}).  The database
can be considered as a single large graph with $100$ connected
components.
For the SCOP dataset, the support is redefined as
the number of proteins containing the pattern, i.e.,
even if a protein contains multiple embeddings we count them only once for the support.

\smallskip\noindent{\textit{Cost Matrix}:}
Since there are 20 different amino acids, we need a $20 \times 20$ cost
matrix. BLOSUM62~\cite{HH92} is a commonly used substitution matrix for aligning protein
sequences.  The $i,j$ entry in BLOSUM denotes the log-odd score
of substituting the amino acids $a_i$ and $a_j$, defined as
\begin{equation*}
    \label{eq:blosum}
    \matij{B}{i}{j} = \frac{1}{\lambda}
	\log\frac{p_{ij}}{f_i \cdot f_j}
\end{equation*}
where $p_{ij}$ denotes the probability that  $a_i$ can be
substituted by $a_j$;
$f_i$, $f_j$ denote the prior probabilities for observing the
amino acids; and $\lambda$ is a constant. We compute $f_i$ and $f_j$
from the database, and then reconstruct $p_{ij}=f_if_j e^{\lambda
B_{i}}$. Next, we define the pair-wise amino acid cost matrix as
$\matij{C}{i}{j} = 1-\frac{p_{ij}}{p_{ii}}$, which ensures that
the diagonal entries are $\matij{C}{i}{i} =0$.

\begin{figure}[!ht]
	\centerline{
    \includegraphics[width=3in]{5F20P.eps}
	}
	\caption{SCOP: Effect of $\alpha$}
    \label{fig:5F20P}
\end{figure}

\begin{figure}[!ht]
  \centerline{
    \includegraphics[width=2.5in]{runtime_compare.eps}
	}
    \caption{SCOP: Runtime comparison}
    \label{fig:runtime}
\end{figure}

\begin{figure}[!ht]
  \centerline{
    \includegraphics[width=2.5in]{scop_minsup.eps}
	}
	\caption{SCOP: Effect of $minsup$}
    \label{fig:5F20P_ft}
\end{figure}

\smallskip\noindent{\textit{Results}:}
Figure \ref{fig:5F20P} shows the time taken for enumerating approximate
maximal patterns for different values of $\alpha$ (with fixed $minsup =
20$). The plots show the time for random walks with and without the
label pruning. It can be seen that by using the label-based pruning the
time for random walks reduces significantly (by over 100\%).  As
expected, the time increases as the values of $\alpha$ increases, since
the number of isomorphisms clearly increases for a more relaxed (larger)
cost threshold.  When $\alpha = 0.01$, the patterns are exact as
$\matij{C}{i}{j} > \alpha$ $,\forall i \neq j$.

Figure \ref{fig:runtime} compares the time taken to mine $500$
maximal patterns from the SCOP dataset using the \ncl label algorithm
and two naive isomorphism enumerating algorithms.
The value of $minsup$ is $15$ and
the threshold $\alpha$ is chosen as $0.7$.
The \textit{no pruning}
algorithm computes the representative sets from the candidate representative
sets directly using the verification procedure described in
section \ref{sec:verification}. The \textit{gApprox} algorithm is based on
\cite{gapprox} and stores all isomorphisms during the course of enumerating
a maximal pattern. For each candidate pattern, it computes the isomorphisms
from the isomorphisms of the frequent pattern from which the candidate
pattern is generated. It can be see that, the run time for the \ncl based
algorithm is significantly less compared to the naive enumeration algorithms
as it prunes invalid candidates without performing an expensive verification
procedure or storing potentially an expensive number of isomorphims.

Figure \ref{fig:5F20P_ft} shows the time taken for $K=500$ random walks
for various values of $minsup$, but with a fixed $\alpha = 0.7$. The bar
plot shows time spent in \khop matching (Hops), \ncl matching
(Neighbors) and pattern verification (Enumeration). In general, the time
increases as the $minsup$ increases because the representative sets
$R(u)$ become larger. However, there is no fixed trend as the total time
depends on the regions of pattern space that the random walk explores.

\begin{figure}[!ht]
  \centerline{
    \includegraphics[width=2.5in]{scop_effectiveness.eps}
	}
	\caption{SCOP: Effectiveness of labels}
    \label{fig:D5F20P_eff}
\end{figure}

Figure~\ref{fig:D5F20P_eff} compares the effectiveness of the \ncl
label and \khop label for different values of the
threshold $\alpha$ on the SCOP dataset.  For each value of $\alpha$, the
left bar shows the time with \ncl label, whereas the right bar
shows the time using only the \khop label.  The \ncl label clearly
reduces the time taken. In fact, it reduces the time for both the \khop
matching and the pattern verification steps, since \ncl is very
effective in pruning the representative set.  This effect is best seen
for $\alpha = 0.75$, where the total time for the enumeration reduces
even though matching the neighbors takes more time compared to \khop
matching.  This shows the effectiveness of the \ncl label versus
\khop label in isolation.

\begin{figure}[!ht]
  \centerline{
  \subfloat[Pattern $A$]{
    \label{fig:scopA}
    \includegraphics[width=1.5in,height=1.25in]{1ysw.pat.eps}
	}
  \subfloat[Pattern $B$]{
    \label{fig:scopB}
    \includegraphics[width=1.5in,height=1.0in]{1r2e.pat.eps}
	}
	}
	\centerline{
	\subfloat[Structural Motif $A$]{
    \label{fig:scopAS}
    \includegraphics[width=1.5in,height=1.5in]{1ysw.eps}
	}
	\subfloat[Structural Motif $B$]{
    \label{fig:scopBS}
    \includegraphics[width=1.5in,height=1.5in]{1r2e.eps}
	}
	}
	\caption{SCOP: Approximate Graph Patterns and their Structures}
	\label{fig:scoppats}
\end{figure}

\smallskip\noindent{\textit{Example Patterns}:}
Figure~\ref{fig:scoppats} shows examples of approximate protein graph
patterns and their corresponding 3D structure extracted from the SCOP
dataset.  For example, the graph in \ref{fig:scopA} appears in only one of
the families. This pattern occurs in 18 of the 20 members, and the
structure of one its occurrences, in protein PDB:1YSW, is shown in
\ref{fig:scopAS}. The common motif corresponds to the black colored
amino acids.  Another approximate pattern is shown in \ref{fig:scopB},
and its structure in PDB:1R2E is shown in \ref{fig:scopBS}; it has
support 19.  It is important to note that the cost of this isomorphism
is $C(\phi) = 0.4541$, indicating that exact isomorphism cannot find the
motif.

\subsection{Protein-Protein Interaction Network (PPI)} We ran
experiments on a yeast (Saccharomyces cerevisiae) PPI network. The list of
interacting proteins for yeast was downloaded from the DIP database
(\url{http://dip.doe-mbi.ucla.edu}). As seen in Table~\ref{tab:db}, the
PPI network has 4950 proteins and 16,515 interactions.  Unlike the other
datasets, each node in the PPI network essentially has a unique label,
which is the protein name.

\smallskip\noindent{\textit{Cost Matrix}:}
To construct the cost matrix for the protein network we consider the
similarity between the protein sequences for any two adjacent nodes.
Sequence similarity is obtained via the BLAST alignment
score~\cite{altschul90}, that returns the expected value (E-value) of
the match. A low E-value implies high similarity, thus we create a
binary cost matrix between the proteins by setting
$\matij{C}{p_i}{p_j} =0$ iff the proteins $p_i$ and $p_j$ have high
similarity, i.e., iff $E-value(p_i, p_j) \le \epsilon$. We empirically
set $\epsilon = 0.003$.

\begin{figure}[!h]
	\centerline{
	\includegraphics[width=2.5in]{ppi.eps}
	}
    \caption{PPI: Time for different values of $minsup$}
    \label{fig:ppiwalks}
\end{figure}

\smallskip\noindent{\textit{Results}:} Figure \ref{fig:ppiwalks} shows
the time for random walks in the yeast $PPI$ network for different values of
$minsup$.  It can be seen that the time for random walks decreases as
the support value increases.  One of the differences for the PPI graph
is that we do not utilize the \khop labels.  The complexity of matching
the \khop labels depends on the number of literals in the \khop label.
As each label(protein) is unique in the PPI graph, the number of
literals in the \khop label of a vertex $v$ in a PPI network is equal to
the number of vertices reachable in \khops. This increases the run time
for the \khop label matching.  Therefore, for mining PPI networks we use
only the \ncl labels.

\begin{figure}[!ht]
  \subfloat[Pattern $A$]{
    \label{fig:ppipatsA}
    \includegraphics[width=2in]{ppipat2.eps}
	}\\
\subfloat[GO Terms for $A$]{
  % Table for the search space pruning
    \label{fig:ppipatsAT}
  \begin{tabular}{c|p{2in}}
GO Terms & Description\\
\hline
BP:0051603&      proteolysis involved in cellular protein catabolic
process\\
\hline
MF:0004298&      threonine-type endopeptidase activity\\
\hline
CC:0034515&      proteasome storage granule\\
  \end{tabular}
  %\label{subfig:match}
  } \\
  \subfloat[Pattern $B$]{
    \label{fig:ppipatsB}
    \includegraphics[width=2in]{ppipat1.eps}
	}\\
  \subfloat[GO Terms for $B$]{
    \label{fig:ppipatsBT}
  \begin{tabular}{c|p{2in}}
GO Terms & Description\\
\hline
BP:0004674  & protein serine/threonine kinase activity\\
\hline
MF:0016301   &   kinase activity\\
MF:0005524   &   ATP binding\\
  \end{tabular}
  }
    \caption{Approximate PPI Patterns and GO Enrichment}
    \label{fig:ppipats}
\end{figure}

\begin{comment}
\begin{figure}[!ht]
  \centerline{
  \subfloat[Pattern $A$]{
    \label{fig:ppipatsA}
    \includegraphics[width=2in]{ppipat2.eps}
	}}
	\centerline{
  \subfloat[GO Terms for $A$]{
    \label{fig:ppipatsAT}
  \small
  \begin{tabular}{c|p{2in}}
GO Terms & Description\\
\hline
BP:0051603&      proteolysis involved in cellular protein catabolic
process\\
\hline
MF:0004298&      threonine-type endopeptidase activity\\
\hline
CC:0034515&      proteasome storage granule\\
  \end{tabular}
  }}
  \centerline{
  \subfloat[Pattern $B$]{
    \label{fig:ppipatsB}
    \includegraphics[width=2in]{ppipat1.eps}
	}}
	\centerline{
  \subfloat[GO Terms for $B$]{
    \label{fig:ppipatsBT}
  \small
  \begin{tabular}{c|p{2in}}
GO Terms & Description\\
\hline
BP:0004674  & protein serine/threonine kinase activity\\
\hline
MF:0016301   &   kinase activity\\
MF:0005524   &   ATP binding\\
  \end{tabular}
  }}
    \caption{Approximate PPI Patterns and GO Enrichment}
    \label{fig:ppipats}
\end{figure}
\end{comment}

\smallskip\noindent{\textit{Example Patterns}:}
Figures~\ref{fig:ppipatsA} and \ref{fig:ppipatsB} show two of the mined
maximal frequent approximate patterns (using $minsup=5$). The proteins
are labeled with their DIP identifiers (e.g., DIP-2818N); the last
number in the label is just a sequential node id.  It is worth
emphasizing that exact subgraph isomorphism would not yield any patterns
in this dataset, since each label is unique. However, since we allow a
protein to be replaced by a similar protein via the cost matrix \Cs,
we obtain interesting approximate patterns. To judge the quality of the
mined patterns we use the gene ontology (GO;
\url{www.geneontology.org}), which comprises three structured,
controlled vocabularies (ontologies) that describe gene products in
terms of their associated biological processes (BP), molecular functions
(MF), and cellular components (CC).  For each of the mined approximate
patterns we obtain the set of all the GO terms common to all proteins in
the pattern. This serves as an external validation of the mined results,
since common terms imply meaningful biological relationships among the
proteins.  Figure~\ref{fig:ppipatsAT} shows the common GO terms for
pattern $A$.  This subgraph comprises proteins involved in proteolysis
as the biological process, i.e., they act as enzymes that lead to the
breakdown of other proteins into amino acids. Their molecular function
is endopeptidase activity, i.e., breakdown of peptide bonds of
non-terminal amino acids, in particular the amino acid Threonine.  These
proteins are located in the proteasome storage granule, and most likely
comprise a protein complex (proteasome) -- a molecular machine --
that digests proteins into amino acids.  The common GO terms for pattern
$B$ in Figure~\ref{fig:ppipatsBT} indicate that the proteins function as
Kinases, proteins that are responsible for adding a phosphate to an
amino acid. The biological process is Phosphorylation, the
post-translational modification of proteins corresponding to adding a
phosphate, in particular modifying amino acids Serine and Threonine.


\documentclass{article}
\usepackage[utf8]{inputenc}

\title{MAT257 Notes}
\author{Jad Elkhaleq Ghalayini}
\date{November 2 2018}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{mathtools}
\usepackage{enumitem}
\usepackage{graphicx}
\usepackage{cancel}

\usepackage[margin=1in]{geometry}

\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{definition}{Definition}
\newtheorem*{corollary}{Corollary}
\newtheorem{exercise}{Exercise}

\DeclareMathOperator{\Int}{Int}
\DeclareMathOperator{\grad}{grad}

\newcommand{\reals}[0]{\mathbb{R}}
\newcommand{\nats}[0]{\mathbb{N}}
\newcommand{\ints}[0]{\mathbb{Z}}
\newcommand{\rationals}[0]{\mathbb{Q}}
\newcommand{\brac}[1]{\left(#1\right)}
\newcommand{\sbrac}[1]{\left[#1\right]}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\eval}[3]{\left.#3\right|_{#1}^{#2}}
\newcommand{\ip}[2]{\left\langle#1,#2\right\rangle}
\newcommand{\prt}[2]{\frac{\partial #1}{\partial #2}}
\newcommand{\mb}[1]{\mathbf{#1}}

\begin{document}

\maketitle

\section{Multivariate Taylor Series}

\subsection{Review of Single Variable Taylor Series}
Recall the definition of the \(n^{th}\) degree Taylor polynomial of \(f\) centered at \(a\)
\[P_{n, a}(f) = \sum_{k = 0}^n\frac{f^{(k)}(a)}{k!}(x - a)^k\]
We define \(R_{n, a}\) to be the ``remainder'', the difference between \(P_{n, a}\) and \(f\). We can write this remainder in ``Lagrange form'' as
\[\frac{f^{(n + 1)}(t)}{(n + 1)!}(x - a)^{n + 1}\]
Clearly,
\[\lim_{x \to a}\frac{R_{n, a}(x)}{(x - a)^n} = 0\]
We can now define the \underline{Taylor series} of \(f\) at \(a\) to be
\[T_af = \sum_{k = 0}^\infty\frac{f^{(k)}(a)}{k!}(x - a)^k\]
\subsubsection{Why these coefficients?}
Suppose
\[f(x) = \sum_{k = 0}^\infty a_k(x - a)^k\]
Note
\[\frac{d^n}{dx^n}(x - a)^k = \left\{\begin{array}{cc}
  0 & \text{if } k < n \\
  k! & \text{if } n = k \\
  k(k - 1)...(k - n + 1)(x - a)^{k - n} & \text{if } k > n
\end{array}\right.\]

\subsection{Multivariable Case}
For example: What's your favorite polynomial? Zero? I'm sure you meant something more like
\[f(x, y, z) = a_{12,8,2}x^{12}y^8z^2 + a_{0,22,0}y^{22} + a_{0,0,1}z + a_{20,20,20}x^{20}y^{20}z^{20}\]
How to do spot \(a_{12, 8, 2}\)? Apply:
\[\left.\frac{\partial^{22}f}{\partial x^2 \partial y^8 \partial z^2}\right|_{(x, y, z) = (0, 0, 0)} = a_{12, 8, 2}12!8!2!\]
In general, if \(f\) is a sum of terms like
\[a_{\alpha_1,...,\alpha_n}x^{\alpha_1}....x^{\alpha_n}\]
Then
\[a_{\alpha_1,...,\alpha_n} = \left.\frac{\partial^{\alpha_1 + ... + \alpha_n}f}{\partial x_1^{\alpha_1} ... \partial x_n^{\alpha_n}}\right|_{(x_1,...,x_n) = (0,...,0)}\frac{1}{\alpha_1!...\alpha_n!}\]

\subsection{Multi-index notation}
We're mathematicians here, and what do mathematicians do, we generalize. But we generalize in such a way such that the general case looks like the specialized case so we don't have to remember different notation when teaching first-years and when doing research. Multi-index notation is a prime example. Compare:
\begin{itemize}

  \item Let \(f: \reals \to \reals\). Then
  \[f = \sum_{k \in \nats}\frac{1}{k!}f^{(k)}(a)(x - a)^k\]

  \item Let \(f: \reals^n \to \reals\). Then
  \[f = \sum_{\alpha \in \nats^n}\frac{1}{\alpha!}\frac{\partial^{|\alpha|}}{\partial x^\alpha}(a)(x - a)^\alpha\]

\end{itemize}
To make this to work, all we have to do is define, for
\[\alpha = (\alpha_1,...,\alpha_n) \in \nats^n\]
the notation
\[\alpha! = \alpha_1!...\alpha_n!\]
\[|\alpha| = \alpha_1 + ... + \alpha_n\]
\[\partial x^\alpha = \partial x_1^{\alpha_1} ... \partial x_n^{\alpha_n}\]
\[(x - a)^\alpha = (x_1 - a_1)^{\alpha_1}...(x_n - a_n)^{\alpha_n}\]

\subsubsection{Examples}
\begin{enumerate}

  \item
  \[\frac{1}{6!y!}12x^6y^7 = \frac{12}{(6, 7)!}(x, y)^{6, 7}\]

  \item
  \[x^5 + x^4y + x^3y^2 + ... + y^5 = \sum_{\beta + \gamma = 5}(x, y)^{(\beta, \gamma)} = \sum_{|\alpha| = 5}(x, y)^\alpha\]
  Note that we're assuming here \(\beta, \gamma\) are non-negative. On good days you can just do what we did above, but if it worries you, write it down.

  \item
  \[a_{0, 0} + a_{1, 0}x + a_{0, 1}y + a_{1, 1}xy + a_{2, 0}x^2 + a_{0, 2}y^2 = \sum_{|\alpha| \leq 2}a_{\alpha}(x, y)^{\alpha}\]

\end{enumerate}

\section{Multivariable Taylor Series}

\begin{theorem}
  If \(f: \reals^n \to \reals\) is \(\mc{C}^{k + 1}\) then
  \[f(a + h) = \sum_{|\alpha| \leq k}\frac{1}{\alpha!}\frac{\partial^{|\alpha|}f}{x^\alpha}(a)h^\alpha + \sum\_{|\alpha| = k + 1}\frac{1}{\alpha!}\frac{\partial^{|\alpha|}f}{\partial x^\alpha}(a + \theta h)h^\alpha\]
  where \(\theta \in (0, 1)\)
\end{theorem}
\begin{proof}
  The key idea is
  \[F(t) = f(a + th)(\text{Old Taylor } + \text{ Chain Rule})\]
  More rigorously, we know from single variable calculus that
  \[F(t) = F(0) + \frac{F'(0)}{1!}t + ... + \frac{F^{(k)}(0)}{k!}t^k + \frac{F^{k + 1}(\theta)}{(k + 1)!}t^{k + 1}\]
  Set \(t = 1\) to get \(F(1) = f(a + h)\).
  We know \(F(0) = f(a)\). We have
  \[\left.\frac{dF}{dt}\right|_{t = 0} = \left.\frac{d}{dt}f(a + th)\right|_{t = 0} = \sum_{i = 1}^n\left.\prt{f}{x}(a + th)\right|_{t = 0}\prt{x_i}{t} = \sum_{i = 1}^n\left[\left.\prt{f}{x_i}(a + th)\right|_{t = 0}\right]h_i
  = \sum_{i = 1}^n\prt{f}{x_i}(a)h_i\]
  We have
  \[\left.\frac{d^2F}{dt}\right|_{t = 0} = \sum_{i = 1}^n\left[
    \sum_{j = 1}^n\frac{\partial^2f}{\partial x_j \partial x_i}(a)h_j
  \right]h_i\]
  By the magical process of ...,
  \[\left.\frac{d^kF}{dt^k}\right|_{t = 0} = \sum_{i_1,...,i_k = 1}^n\frac{\partial^kf}{\partial x_{i_1} ... \partial x_{i_k}}(a)h_{i_1}...h_{i_k}\]
  Now all we need is some combinatorics, combined with the equality of mixed partials (for \(\mc{C}^{k + 1}\) functions), to complete the proof. Recall the \textit{multinomial coefficient}
  \[{{|\alpha|} \choose {\alpha_1,...,\alpha_n}} = \frac{|\alpha|!}{\alpha_1!...\alpha_n!}\]
  is the number of ways of taking \(|\alpha|\) things and making \(k\) groups of size \(\alpha_1,...,\alpha_n\).
  We can use this to rewrite the above as
  \[F^{(S)}(0) = \sum_{|\alpha| = S}\frac{S!}{\alpha_1!...\alpha_S!}\frac{\partial^{|\alpha|}f}{\partial x^\alpha}(a)h^\alpha\]
  So plugging this into our original expression for the Taylor series of \(F\), where we divide the \(S^{th}\) term by \(\frac{1}{S!}\), we have
  \[F(t) = F(0) + \sum_{i = 1}^k\sum_{|\alpha| = i}\frac{1}{\alpha_1!...\alpha_k!}\frac{\partial^{|\alpha|}f}{\partial x^\alpha}(a)h^\alpha
  = f(a) + \sum_{|\alpha| \leq k}\frac{1}{\alpha!}\frac{\partial^{|\alpha|}f}{\partial x^\alpha}(a)h^\alpha\]
  as desired.

\end{proof}

\end{document}


\section{Catalan array, precisely}
\label{sec:C:precisely}

% big comment: skip it, maybe the content is interesting, but stated very ugly.
\iffalse
Let $\mathcal{C}_{h}$ a principal $h$-cluster of
the Catalan array, in its traditional definition: We attempt to point out
some properties about $\mathcal{C}_{h+1}$:
\begin{itemize}
    \item a more general pattern, for any $h\in\mathcal{N}$,
        a coefficient $d_{p^{h}-1,k}$, for $k\in\lbrace0,\ldots,p^{h}-1 \rbrace$,
        satisfies:
        \begin{displaymath}
            d_{p^{h}-1,k} \equiv_{p} 0
        \end{displaymath}
    \item consider the subcluster $\mathcal{C}_{h}^{(0,p^{h}-1)}$ of
        $\mathcal{C}_{h}$ obtained by removing antidiagonal $0$ (namely,
        the boundary one, composed by $1$s only) and by removing row $p^{h}-1$.
        By construction, $\mathcal{C}_{h}^{(0,p^{h}-1)}$ is a triangle too,
        with $p^{h}-2$ coefficients per side. Therefore, $\mathcal{C}_{h+1}$
        is composed on the very top, starting at the root, by a copy of $\mathcal{C}_{h}$,
        while on the bottom there are three triangles:
        \begin{itemize}
            \item an equilateral triangle $T_{\equiv_{p} 0}^{(h)}$,
                with $p^{h}-1$ coefficients on each side, such that:
                \begin{displaymath}
                    d_{nk} \in T_{\equiv_{p} 0}^{(h)} \rightarrow d_{nk} \equiv_{p} 0
                \end{displaymath}
                where $n\in\lbrace p^{h},\ldots,p^{h+1}-2\rbrace$ and
                $k\in\lbrace 0,\ldots, n-p^{h}\rbrace$, in other words
                $k\in\lbrace 0,\ldots, p^{h}(p-1)-2\rbrace$;
            \item a mirror copy of $\mathcal{C}_{h}^{(0,p^{h}-1)}$ respect to column orientation,
                defining coefficients $d_{nk}$
                where $n\in\lbrace p^{h},\ldots,p^{h+1}-2\rbrace$ and
                $k\in\lbrace 1,\ldots, p^{h}-2\rbrace$;
            \item a segment of coefficients $d_{n, p^{h}-1} \equiv_{p} 0$
                where $n\in\lbrace p^{h},\ldots,p^{h+1}-2\rbrace$;
            \item a copy of $\mathcal{C}_{h}^{(0,p^{h}-1)}$,
                defining coefficients $d_{nk}$
                where $n\in\lbrace p^{h}+1,\ldots,p^{h+1}-2\rbrace$ and
                $k\in\lbrace p^{h},\ldots, n-1\rbrace$;
        \end{itemize}
\end{itemize}
\fi

We are now in the position to formally define the Catalan
array $\mathcal{C}$ and to introduce some mathematical objects
that will be used later; formally, $\mathcal{C}$ is defined as
\begin{displaymath}
    \mathcal{C}=\left(\frac{1-\sqrt{1-4\,t}}{2\,t},
        \frac{1-\sqrt{1-4\,t}}{2}\right)
\end{displaymath}
and it is a \emph{renewal} array \cite{rogers:1977} because
$h(t)=t\,d(t)$. Application of the substitution $k=0$ in the definition of the
coefficient $\displaystyle d_{n,k} = [t^{n}]d(t)h(t)^{k}$ entails that the
$n$-th Catalan number $C_{n}$ satisfies
\begin{displaymath}
    C_{n} = [t^{n}]d(t)= [t^{n}]\frac{1-\sqrt{1-4\,t}}{2\,t}
\end{displaymath}
where the operator $[t^{n}]$ extracts the coefficient of $t^{n}$ in the series expansion
of the function $g(t)$ to which it is applied.

An equivalent characterization of a each matrix $\mathcal{R}(d(t), h(t))$ in
the Riordan group is given by two sequences of coefficients
$\left(a_{n}\right)_{n\in\mathbb{N}}$  and
$\left(z_{n}\right)_{n\in\mathbb{N}}$ called $A$-sequence and $Z$-sequence,
respectively. The former one can be used to define every
coefficient $d_{n,k}$ with $k>0$,
\begin{equation}
    d_{n+1, k+1} = a_{0}d_{n,k} + a_{1}d_{n,k+1} + a_{2}d_{n,k+2} + \ldots + a_{j}d_{n,k+j} + \ldots %+ a_{j+1}d_{n,k+j+1} + \cdots
\end{equation}
where the sum is finite because exists $j\in\mathbb{N}$ such that $n=k+j$. On
the other hand, the latter one can be used to define every coefficient
$d_{n,0}$ lying on the first column,
\begin{equation}
    d_{n+1, 0} = z_{0}d_{n,0} + z_{1}d_{n,1} + z_{2}d_{n,2} + \ldots + z_{n}d_{n,n} + \ldots %+ z_{n+1}d_{n,n+1} + \cdots
\end{equation}
where the sum is finite because $d_{n,k+j}=0$ for $j>n-k$.

Moreover, let $A(t)$ and $Z(t)$ be the generating functions of the $A$-sequence
and $Z$-sequence, respectively, then relations
\begin{equation}
    h(t) = tA(h(t)) \quad\text{and}\quad d(t)=\frac{d_{0,0}}{1-tZ(h(t))}
\end{equation}
connect them with functions $d(t)$ and $h(t)$,
where $d_{0,0}$ is the very first element %in the top left corner
of $\mathcal{R}$, see \cite{merlini:some:alternative:characterizations:1997}.

A last fact to be aware of is a fundamental theorem that allows us to perform
matrix-vector product of an array $\mathcal{R}$ and an infinite column vector
$\vect{\omega}=(\omega_{0},\omega_{1},\omega_{2},\ldots)$
as the convolution
\begin{equation}
    \mathcal{R}\cdot\vect{\omega} = d(t)\Omega(h(t))
\end{equation}
where $\Omega$ is the $\vect{\omega}$'s generating function which admits the
series expansion $\Omega(t)=\sum_{k\in\mathbb{N}}{\omega_{k}t^{k}}$.

In the rest of the paper we use the notation $c_{n,k}\in\mathcal{C}$ to denote
a coefficient belonging to the Catalan array at row $n$ and column $k$.
We instantiate previous properties for the Catalan array $\mathcal{C}$,
starting with its $A$-sequence
\begin{displaymath}
    A_{\mathcal{C}}(t)=\frac{1}{1-t}=1+t+t^{2}+t^{3}+t^{4}+t^{5}+t^{6}+t^{7}+t^{8}+
        \mathcal{O}(t^{9})
\end{displaymath}
and, since $\mathcal{C}$ is a renewal array, it is not difficult to see that
its $Z$-sequence is the same as the $A$-sequence. Consequently, function $Z(t)$
defines each Catalan number $C_{n}$ as the sum $\displaystyle C_{n} = c_{n,0} =
\sum_{k=0}^{n-1}{c_{n-1,k}} $ of coefficients lying on row $n-1$ and
the application of $[t^{n}]$ to the generating function $d(t)$
allows us to write a well known closed formula for the $n$-th Catalan number
\begin{equation}
    C_{n} = \frac{1}{n+1}{{2n}\choose{n}} = {{2n}\choose{n}} - {{2n}\choose{n+1}}.
    \label{eq:catalan:coeff:rewriting}
\end{equation}

On the other hand, an arbitrary element $c_{n,k}\in\mathcal{C}$
admits closed formulae too and, according to \cite{luzon:2012631}
\begin{align}
    & c_{n,k}=\frac{k+1}{n+1}{{2n-k}\choose{n-k}}\quad\text{and}
    \label{eq:catalan:array:first:identity}\\
    & c_{n,k}={{2n-k}\choose{n-k}} - {{2n-k}\choose{n-k-1}}
    \label{eq:catalan:array:second:identity}
\end{align}
are of particular interest for our work.  In order to formally characterize
$\mathcal{C}_{\equiv_{2}}$ we use the definition to write the generic coefficient
\begin{displaymath}
    c_{n,k} = [t^n] \frac{1-\sqrt{1-4\,t}}{2\,t}\,
        \left(\frac{1-\sqrt{1-4\,t}}{2}\right)^{k}
           = [t^{n+1}] \left(\frac{1-\sqrt{1-4\,t}}{2}\right)^{k+1}
\end{displaymath}
where the right most term requires to extract coefficient $n+1$ from the $(k+1)$-fold
convolution of Catalan numbers' generating function with itself, \emph{shifted
by one place}
\begin{equation}
    c_{n,k} = [t^{n+1}] \left(\frac{1-\sqrt{1-4\,t}}{2}\right)^{k+1}
            %= \sum_{i_{1}+ i_{2}+ \ldots+ i_{k+1}=n+1}{
                %\hat{c}_{i_{1}}\,\hat{c}_{i_{2}}\,\ldots\,\hat{c}_{i_{k+1}} }
            = \sum_{i_{1}+ i_{2}+ \ldots+ i_{k+1}=n+1}{
                C_{i_{1}-1}\,C_{i_{2}-1}\,\ldots\,C_{i_{k+1}-1} }.
    \label{eq:convolution:expansion:for:generic:element:in:catalan:array}
\end{equation}


\subsection{Common Setup}
  We propose two different kinds of games, both finite (but possibly generalizable to the infinite setting). The first
  consists of one sole strategy where the players do not initially know whether they will be buyers, sellers or nothing at
  all, this being decided in the last moment. The second game consists of three strategies: buyers, sellers and middlemen.
  Before delving into the details of each game, we first describe their common elements.

  The general approach taken is as follows: After a game is described in detail, each player is assigned a specific strategy and
  a relevant utility function. All players are considered to follow their respective strategy without deviating from it, except
  for one player that is allowed to follow any desired strategy; her utility function however remains unchanged. If that player
  is proven to have an incentive to deviate from her appointed strategy, we can deduce that the given strategies and utility
  functions do not constitute a Nash equilibrium. If on the other hand no player has an incentive to deviate, regardless of
  her appointed strategy, then we will come to the conclusion that the given strategies and utility functions do constitute a
  Nash equilibrium. This approach is common in game theoretic analyses, given that allowing for all players to be rational and
  then searching for a Nash equilibrium constitutes a practically intractable problem \cite{nasheqcomp}. Another common
  approach employed here is that of considering only a generic product that all buyers want and all sellers have, and not a
  variety of different products.

  A description of the structure that is common for both games follows. The game graph has a random initial configuration
  where every player has a random direct trust towards every other player, as well as a random capital. These values may be
  uniformly distributed in an interval or may follow another distribution such as the exponential, or may have a high
  probability of being zero. The exact distribution however is not determined at this point, as it is not yet needed. This
  distribution will be common knowledge to the players. All capitals and direct trusts are publicly viewable. Further
  constraints may be applied to each game separately.  Transaction fees are not considered.

  The game consists of $R$ rounds, the number of which is common knowledge for the players. The players play simultaneously in
  each round and can do any of the known actions. If two actions conflict (e.g. $A$ reduces $DTr_{A \rightarrow B}$ and $B$
  steals from $DTr_{A \rightarrow B}$ as well), then one of the two actions is chosen with equal probability ($50\%$). To
  better model a player's actions and the aforementioned conflict resolution, we demand that each fund reallocation explicitly
  mentions the source and the destination of the funds for each of her actions. Player $A$ decides on the values of all the
  following variables. This constitutes a concrete round for $A$.

  \begin{gather*}
    \forall B, C \in \mathcal{V}, move\left(A, \left(A, B\right), \left(A, C\right) \right) \in \mathbb{R^+} \\
    \forall B, C \in \mathcal{V}, B \neq A, move\left(A, \left(B, A\right), \left(A, C\right) \right) \in \mathbb{R^+}
  \end{gather*}
  The first argument is the player who decides, the second argument is from which direct trust to take the funds and the third
  is to which direct trust to deposit the funds. The first type of moves corresponds to $Add\left(\right)$ and the second to
  the $Steal\left(\right)$ actions.

  To clarify a detail, for any $B \in \mathcal{V}$ (including $A$), $A$ is not allowed to set $move\left(A, \left(A, B\right),
  \left(A, B\right) \right)$ to any value different than 0. This choice is made to facilitate the analysis.

  There are some constraints for player's $A$ move:
  \begin{itemize}
    \item There is no reason to be able to deposit to and withdraw from a specific direct trust in the same round.
    Furthermore, such a possibility would allow for "chain reactions" in the conflict resolution phase that would add
    unnecessary complications.  This constraint applies only to outgoing direct trusts, because incoming direct trusts cannot
    be increased.
    \begin{gather*}
      \forall B, C, D \in \mathcal{V}, move\left(A, \left(A, B\right), \left(A, C\right) \right) \cdot move\left(A, \left(A,
      D\right), \left(A, B\right) \right) = 0 \\
      \mbox{and} \\
      \forall B, C, D \in \mathcal{V}, move\left(A, \left(A, B\right), \left(A, C\right) \right) \cdot move\left(A, \left(D,
      A\right), \left(A, B\right) \right) = 0
    \end{gather*}

    \item One cannot use more funds than are available from a single direct trust.
    \begin{gather*}
      \forall B \in \mathcal{V}, \sum\limits_{C \in \mathcal{V}} move\left(A, \left(A, B\right), \left(A, C\right) \right)
        \leq DTr_{A \rightarrow B} \\
      \forall B \in \mathcal{V}, \sum\limits_{C \in \mathcal{V}} move\left(A, \left(B, A\right), \left(A, C\right) \right)
        \leq DTr_{B \rightarrow A} \\
    \end{gather*}
  \end{itemize}

  If two players try to change the same direct trust, then set the relevant moves of one of the two players (chosen uniformly
  at random) to 0.
  \begin{lstlisting}[label=conflict, style=numbers]
resolveConflict((*@$A$@*), (*@$B$@*)) :
  sum1 = (*@$\sum\limits_{C \in \mathcal{V}}move\left(A, \left(A, B\right), \left(A, C\right) \right)$@*)
  sum2 = (*@$\sum\limits_{C \in \mathcal{V}}move\left(B, \left(A, B\right), \left(B, C\right) \right)$@*)
  if (sum1*sum2 != 0)
    choice (*@$ \overset{\$}{\gets} \{A, B\}$@*)
    if (choice == (*@$A$@*))
      (*@$\forall C \in \mathcal{V}, move\left(A, \left(A, B\right), \left(A, C\right) \right)$@*) = 0
    else # if (choice == (*@$B$@*))
      (*@$\forall C \in \mathcal{V}, move\left(B, \left(A, B\right), \left(B, C\right) \right)$@*) = 0

resolveAllConflicts() :
  (*@$\forall A, B \in \mathcal{V}$@*)
    resolveConflict((*@$A$@*), (*@$B$@*))
    resolveConflict((*@$B$@*), (*@$A$@*))
  \end{lstlisting}

  \noindent \texttt{resolveAllConflicts()} is executed after all players choose their moves for a round. In case of an
  adversary, the random choice is resolved in their favor.

  \begin{figure}[p]
  \label{fig:game}
    \centering
    \makebox[\textwidth][c]{\includegraphics[width=1.55\textwidth,angle=90]{game}}
    \caption{The general form of the game \cite{sgtm}}
  \end{figure}

  Fig.~\ref{fig:game} depicts the evolution of a two-player game. The players' choices for the first round are represented by
  the solitary table at the top of the figure. More concretely, each distinct choice that player $A$ can make is depicted as a
  row and likewise $B$'s choices are represented by columns. At each intersection of choices lies a subgame. The lines
  departing from these intersections lead to the conflict resolution stage, which is decided by Nature. Depending on the
  result of the conflict resolution, a new round begins where players can choose a new course of action. All the possible
  states that players can be found in at the beginning of round 2 are depicted by the series of tables at the 2nd level. The
  game evolves in a similar fashion with alternation between players' choices and Nature's conflict resolution, until players
  can make their $R$-th choice. The keen observer can note that, instead of new departing lines, each cell of the tables of
  the last round ($R$) contains a pair of numbers which represent the players' utilities for that outcome.

  Note that a concrete player's strategy consists of all the choices that she would make in every possible state that she can
  find herself in. Thus, even though only $R$ choices will be made by each player, a strategy is comprised of exponentially
  many choices. Furthermore, if we had to consider a three-player game, we would have to replace all tables with 3-dimensional
  cubes and so on for more players.

  We now move on to describe the individual games.


\chapter{Conclusion and Future Work}\label{sec:conclusion_and_future_work}
To conclude this work, we summarize the presented models, algorithms, implementations, studies and the main findings. Moreover, we give an outlook on future work and additional research questions that can be approached building on our work.

\section{Summary of this Work}

The overarching goal of this work was to enable simulations of the neuromuscular system using detailed, biophysical multi-scale models with high resolutions. The simulations should compute numerically accurate results, run efficiently on various hardware and allow parallel scaling to large problem sizes, which should be solved on supercomputers.

As a result, this work established a computational framework for multi-scale modeling of skeletal muscles, their neural activation, muscle contraction and generation of EMG signals on the skin surface. Our approach combined existing models for various parts of the neuromuscular system into a comprehensive multi-scale model framework. Scalability and parallel efficiency of our software were ensured by efficient algorithms, suitable, parallelized numerical schemes and by our accompanying performance analyses.

% ieser Arbeit war ... biophysikalisch, hoch aufgelöst, schnell, skalierbar, genau ...

We described the following topics in this work: After the introduction in \cref{chap:introduction}, we compared two modeling approaches to describe the movement of the upper arm in \cref{chap:comparative_study}. Based on data of experimental trials we conducted during a graduate school workshop, we developed a first, data-driven model using Gaussian process regression and a second model based on a biophysical simulation with two muscle models. The parameters for the biophysical simulation were fitted to experimental training data using numerical optimization. The comparison of the two approaches revealed a slightly better fit for the biophysical simulation model. This approach had the additional benefit of giving biophysical insights into the functioning of the system and provided estimates for subject-specific muscle parameters. While this study used Hill-type muscle models, which describe muscle forces on a 1D line of action, we considered more accurate multi-scale models in the remainder of this work.

\Cref{sec:generation_of_meshes_for_multiscale} dealt with the generation of structured 3D meshes and embedded 1D meshes for muscle fibers. The approach of only using structured meshes, which allowed for a simple domain decomposition proved to be beneficial for the parallel performance of our simulations.
We described a workflow, how to obtain these meshes from biomedical imaging data. We developed a serial algorithm and a parallel algorithm to construct the required meshes and to ensure a good mesh quality, even for meshes with high resolutions. The algorithms were based on our novel approach of using harmonic maps to transform reference meshes to cross-sectional slices of the muscle mesh.

In \cref{sec:muscle_fibers_and_motor_units}, we described ways to associate muscle fibers with motor units (MUs) in a physiological manner. We developed efficient algorithms for this task for different premises, and employed the algorithms to associate up to \num{270000} muscle fibers to \num{100} MUs for the subsequent use in our simulations.

In \cref{chap:models_and_discretization}, we first described all equations of the state-of-the models that we used, and how they can be combined into a multi-scale description. Then, we described their discretization using the finite element method for the spatial derivative terms and various timestepping and operator splitting schemes for the temporal derivatives. One original contribution is the derivation of the finite element formulation for the multidomain equation. Further, we gave a detailed description of the nonlinear solid mechanics discretization, which we used in our implementation.

Next, we presented details on our simulation software OpenDiHu, which we used to solve various combinations of the described multi-scale model framework to simulate the neuromuscular system. \Cref{chap:usage} gave an introduction to the design and usage of the software and demonstrated its application using various example problems.

\Cref{sec:implementation} described the implementation of OpenDiHu in more detail, motivated various design decisions, introduced the data handling and several algorithms, e.g., to construct a parallel domain decomposition or to map data between meshes, and described the implementation of various solvers for particular parts of the multi-scale model.

\Cref{sec:results} presented numerical results, which were obtained using our simulation software. We simulated the passive mechanical behavior of muscle tissue, subcellular models given in CellML description, electrophysiology on muscle fibers, electric conduction in the muscle and the adipose tissue to obtain surface EMG signals, electrophysiology using the 3D homogenized multidomain description, and coupled scenarios of electrophysiology and muscle contraction. We discussed effects of model and structural parameters and interpreted the obtained simulation results.

In \cref{sec:performance_analysis}, we analyzed the computational performance of our software in general and various solvers in particular. We conducted numerical studies of universal convergence properties with the software OpenCMISS, which also helped to parameterize the numerical solvers in OpenDiHu. Further studies on mesh widths and used linear solvers were carried out directly using OpenDiHu. We evaluated various optimization options in OpenDiHu and compared the most optimized settings in OpenDiHu with the baseline solver OpenCMISS, yielding a high speedup of more than two orders of magnitude. Moreover, we investigated the computational performance of our models on the GPU, and conducted parallel strong scaling and parallel weak scaling tests on small clusters and the supercomputers at the High Performance Computing Center Stuttgart.

\section{Summary of Main Findings}

The present work simulated numerous scenarios with various model combinations, which provided different insights. In the following, we summarize the observed findings. We address the biophysical observations in \cref{sec:observations_biophysics} and results of the performance measurements in \cref{sec:observations_performance}.

\subsection{Observations from the Fields of Biophysics and Biomechanics}\label{sec:observations_biophysics}
The comparison of the linear and nonlinear mechanics models in \cref{sec:solver_solid_mechanics} showed qualitatively different results and demonstrated that the accurate behavior of deforming muscle tissue can only be described by a proper nonlinear anisotropic solid mechanics model.

Initially, an open question was also how to relate the accuracy of the simulated EMG signals to the number of fibers and the mesh resolution. Our numerical studies in \cref{sec:action_potential_velocity}, which compared the resulting action propagation velocity for different mesh widths of the 1D muscle fiber meshes showed that a mesh width of \SI{100}{\micro\meter} or 100 elements per \SI{}{\centi\meter} gives reasonably accurate results.

To evaluate the 3D mesh width and the spacing between the muscle fibers, we conducted simulations with different 3D mesh resolutions and numbers of fibers in \cref{sec:effects_of_the_mesh_width_emg}. The number of fibers was scaled up to the realistic number of \num{270000} fibers in a biceps brachii muscle. We concluded that the most accurate solution is obtained for a mesh width as fine as possible, as the EMG results were qualitatively different for every refinement step. This emphasizes the need for highly resolved simulation scenarios (representing the real number of fibers in a muscle accurately) for realistic EMG computations and, as a result, the need for High Performance Computing techniques.

However, if the EMG is to be sampled by electrodes, i.e., if the EMG recording process should also be part of the simulation, lower mesh widths might be possible, as the EMG is only captured at the locations of the electrodes.

One possible approach to reduce the computational effort for EMG simulations would be to only consider the muscle tissue down to a certain depth below the surface with the EMG electrodes. We observed in \cref{sec:simfiber_mu}, that the EMG signal is highly influenced by MUs, whose territories are located close to the electrodes. However, our numerical experiments with EMG decomposition algorithms in \cref{sec:simfiber_decomposition} showed that large MUs located opposite to the EMG electrodes at the deepest muscle tissue layers are detectable in the surface EMG signals. Thus, neglecting the deeper parts of the muscle would remove relevant information from the system and is, therefore, not a valid approach to reduce the computational load.

Furthermore, the layer of adipose tissue on top of the muscle showed a smoothing effect on EMG recordings in our simulations, both with the fiber based approach in \cref{sec:simfiber_fat} and with the multidomain approach in \cref{sec:multidomain_components,sec:multidomain_simulation_emg}. One advantage of our simulations compared to experimental studies is that the thickness of the fat layer is known exactly and can also be adjusted.

Simulations of muscle contraction with coupled electrophysiology and solid mechanics models showed a spatially inhomogeneous contraction  for the biceps muscle while the muscle activation is ramped up. The simulation in \cref{sec:fiber_based_contraction} of an isolated, contracting muscle belly without tendons showed transverse bending, alternating between the left and right-hand sides, as a result of the subsequently activated MUs at the different sides of the muscle. We also simulated the biceps brachii muscle together with its tendons and observed a ripple in the generated muscle force, which is caused by the same inhomogeneous MU activity.

The simulations of muscle contraction also showed that, if the muscle is initially in a stress-free state, the model can only achieve a maximum contraction of approximately \SI{85}{\percent}. However, the muscles of the musculoskeletal system are known to exhibit prestresses in their relaxed states. Accordingly, we added prestress to our simulations. The amount of prestress is adjustable in the simulation settings, and the required amount can be determined by a comparison with experimental studies.

% linear-nonlin
% accuracy -> high mesh resolution
% fibers: effects of fat mesh, distance between fibers to surface, size of MUs (EMG decomposition)
% multidomain: more smoothed
% coupled solid mechanics: inhomogeneous contraction, prestretch required (without only 85% contraction)

\section{Summary of Performance Results}\label{sec:observations_performance}

A major part of the work was also concerned with improving the performance of the simulation software, and, thus, enabling larger simulation scenarios in shorter runtimes.

Previously, literature on biophysical, multi-scale models of skeletal muscles was mainly focused on modelling and interpretation of the results, rather than targeting efficient computations. The work of Röhrle et al. \cite{Roehrle2012} introduced the multi-scale model, which we based our work on, and simulated the tibialis anterior muscle using a 3D mechanics mesh with 12 elements. The work of Heidlauf et al. \cite{Heidlauf2013} considered the same geometry and simulated 400 muscle fibers. The authors parallelized their OpenCMISS based implementation for a fixed number of four processes. We built upon this work with the goal to push the limits of feasible problem sizes, and, in \cref{sec:effects_of_the_mesh_width_emg}, executed our optimized simulation with \num{26912} processes, \num{273529} muscle fibers and a 3D mesh for the electrophysiology model with approximately \num{1e8} degrees of freedom.

The performance analyzes in \cref{sec:performance_analysis} showed that the subcellular model contributes a large portion to the total runtime and, thus, is the most crucial part to optimize. By using proper memory layouts, vectorization is possible. Our approach of using explicit vector instructions outperformed the auto-vectorization capabilities of the compiler. The approximation of the exponential function and an improved parallelization scheme for the 1D electric conduction problem additionally contributed to a high speedup. The comparison to the baseline solver OpenCMISS Iron in a strong scaling study in \cref{sec:strong_scaling_runtimes_opencmiss_opendihu} revealed a maximum speedup of 363 for the purely implementation-specific improvements and an additional speedup of 2.5, shown in \cref{sec:opencmiss_numeric_improvements}, by using more efficient numerical methods.

In addition, the memory characteristics of the solvers were investigated in \cref{sec:strong_scaling_runtimes_opencmiss_opendihu}.
The linear increase in memory consumption of the baseline solver in a weak scaling setting was improved to a nearly constant scaling. Our analysis using a roofline performance model showed that our solvers are compute bound and achieve a computational performance of approximately \SI{25}{\percent} peak performance, which is a very good value.

Moreover, hybrid shared/distributed memory parallelism and computations on the GPU were investigated, but both approaches were found to be not competitive with our highly optimized distributed memory parallelization. For the GPU, potentially more efficient approaches than our approach using OpenMP exist, such that a performance improvement in the future could be possible.

The modularity of the CellML infrastructure, where computational models can be shared among researchers and are interchangeable in multi-scale simulations was preserved during all optimization endeavors. Our approach was to implement a source-to-source code generator, which transformed the given CellML code into optimized code for the CPU or the GPU.

For the solution of the multidomain model, we evaluated various preconditioners and selected the most performant preconditioner-solver combination for our computations.
One previously unforeseen result is the large discrepancy of required runtime between the fiber based and the multidomain based electrophysiology models, presented in \cref{sec:solver_multidomain_model}. We measured by a factor of 1000 longer computation times for the multidomain model, which result from the structure of the model. Despite the high computational effort, the multidomain model is useful in practice as it can simulate effects that are not captured by the fiber based model. We gave a detailed comparison between both approaches in \cref{sec:multidomain_differences}.

In summary, we provided a computationally efficient and scalable tool for applied biophysics researchers to solve problems in the domains of EMG generation and muscle contraction. For example, the effect of different muscle fiber organizations and MU recruitment strategies can be tested with our software. We demonstrated its use with state-of-the-art EMG decomposition algorithms, which provide the bridge to the experimental domain.
Thus, we hope to contribute one step on the pathway of complementing in vivo with in silico experiments to increase the understanding of the neuromuscular system.

% provide a tool for applied biophysics researchers
% --------------------
% complement in-vivo and in-silico experiments
% test different muscle fiber organizations -> possible
% decomposition of EMG -> tested

% vc better than auto-vec, AVX-512, memory layout
% comparison to OpenCMISS
% preserve CellML modularity -> code generator
% GPU, OpenMP
% proper choice of solvers
% multidomain performance vs fibers

% performance
% --------------
% Röhrle2012: TA 12 elements, MU association
% Heidlauf2013: 400 fibers, TA, OpenCMISS, mechanics
% made new insights possible

%wir hatten ja mal vorgenommen dass
%tatsächlich ist gelungen:
%überraschenderweise, dass:

%bewerten mit den Ergebnissen
%was bewahrheitet,

%unterschiede zwischen grob und hochaufgelöster Sim
%anzahl realitäts anzahl Muskelfaser anzahl

\section{Outlook and Future Work}\label{sec:future_work}

The presented work could be extended in multiple directions, spanning performance improvements and model extensions.

First, some ideas for further performance improvements could be implemented and evaluated.
The monodomain equation could be solved with implicit-explicit (IMEX) schemes, which could potentially achieve higher precision.
The numerically stiff subcellular model is currently solved explicitly. Implicit schemes could be developed, and the implicit iteration equations could be solved symbolically in a preprocessing step using the parsed CellML code.

To improve the performance of the multidomain model, the following algorithmic improvements are promising options.
The 3D problems of action potential propagation in the muscle volume for every compartment could be restricted to the subset of nodes, where the occupancy factors are above a certain threshold, effectively reducing the problem sizes, and reducing the effect of higher MU counts on the runtime. However, this would bring difficulties to ensure a balanced parallel domain decomposition.
Instead of the current parallel partitioning of the domain, the multidomain model could also be parallelized by distributing the MUs to different processes or by a combination of both approaches.

On the numerical side, an extended error analysis could be carried out for all model parts, and the timestep widths, which are currently chosen conservatively, could potentially be increased, while keeping the numerical error below a given threshold. Error estimators could be developed, which would allow an adaptive adjustment of the timestep widths.
The 3D model solvers for the 3D electrophysiology and multidomain problems could be enhanced with geometric or algebraic multigrid preconditioners.

Since all subcellular points in a muscle are usually in similar states at any time, a hybrid approach using analytic descriptions of action potential propagation, as in \cref{sec:sim_rosenfalck}, and a fully numerical treatment could be chosen, and surrogate models could be adaptively added to the computational description.

On the technical side, computations on the GPU could be re-evaluated in the future using the existing OpenMP approach with more mature compiler versions or different accelerator targeting programming technologies.

Second, the range of simulated models could be extended. The simulations could be applied to further muscle geometries such as the triceps brachii or the tibialis anterior muscles. Muscles with more complex geometries and fiber arrangements could be investigated.
A mechanically coupled problem of agonist-antagonist pair could be considered such as a system of biceps and triceps brachii. Apart from the mechanical coupling, a coupling of the neural recruitment involving sensory organs in the muscles could be implemented and used to approach further biomechanical research questions. Such a neuromuscular feedback loop could also be investigated first for a single muscle, e.g., by extending the preliminary implementation in OpenDiHu for the biceps muscle.

Pathological conditions could be simulated to understand muscular diseases and neuromuscular electrical stimulation of the muscle for stroke rehabilitation could be considered.

By using the preCICE adapters in OpenDiHu, more advanced mechanics solvers could be coupled to an electrophysiology simulation in OpenDiHu, allowing to, e.g., study mechanical effects of surrounding tissue.

On a larger scale, the interplay of more organs could be taken into account. Blood perfusion and muscle metabolism could be added, and coupled by models of the lung and general metabolism in the organism. Thus, a digital human model can be envisioned, which allows to study the effects of anomalies and to develop new therapies, effectively utilizing simulation technology for human wellbeing.

%reduced to 1D problems using appropriate transformations.

% performance improvements
% ----------------------------
% more timestepping methods: CVODE (https://computing.llnl.gov/projects/sundials/cvode), imex
% different parallelisation where not all ranks have to be involved (for multidomain) -> this feature already exists for the fibers with multipleInstances
% more numeric tests on exp function? no
% multidomain: compute 3D problem as 1D problem, or adaptive computation of the parts of the fr factors
% GPU

% extensions
% -------------
% other muscles, more muscles,
% couple more models:
% neuromuscular feedback loop with sensors,
%


\chapter{Work and Energy}

In this chapter, we are going to talk about how engineers define work
and energy.  We have already talked about force. Force is measured in
newtons, and one newton is equal to the force necessary to accelerate one
kilogram at a rate of $1 m/s^2$.

When you lean on a wall, you are exerting a force on the wall, but you
aren't doing any work. On the other hand, if you push a car for a mile,
you are clearly doing work. Work, to an engineer, is the force you
apply to something, as well as the distance that it moves, in the direction
of the applied force. We measure work in \textit{joules}. A joule is one
newton of force over one meter.

\includegraphics[width=0.8\textwidth]{Work_vs.png}

For example, if you push a car uphill with a force of 10 newtons for 12
meters, you have done 120 joules of work.\index{work}
% ADD: We can represent this with the equations, Work Energy Therom

Work is how energy is transferred from one thing to another. When you
push the car, you also burn sugars(energy of the body) in your blood. That energy is then
transferred to the car: after it has been pushed uphill.

Thus, we measure the energy something consumes or generates in
units of work: joules, kilowatt-hours, horsepower-hours, foot-pounds,
BTUs( British Thermal Unit), and calories.

Let's go over a few different forms that energy can take.
% KA: https://www.khanacademy.org/science/ms-physics/x1baed5db7c1bb50b:energy/x1baed5db7c1bb50b:changes-in-energy/a/changes-in-energy
\section{Heat}\index{heat}

When you heat something, you are transferring energy to it. The BTU
 is a common unit for heat: One BTU is the
amount of heat required to raise the temperature of one pound of water,
by one degree. One BTU is about 1,055 joules. In fact, when you buy and sell
natural gas as fuel, it is priced by the BTU.\index{heat} \index{BTU}

\section{Electricity}\index{electricity}

Electricity is the movement of electrons. When you push electrons
through a space that resists their passage (like a light bulb),
energy is transferred from the power source ( a battery)
 into the source of the resistance.

Let's say your lightbulb consumes 60 watts of electricity, and you leave it on for 24 hours.
We would say that you have consumed 1.44 kilowatt hours or 3,600,000 joules.

% KA: https://www.khanacademy.org/science/in-in-class10th-physics/in-in-electricity/in-in-electric-current-circuit/v/intro-to-charge

\section{Chemical Energy}\index{chemical energy}

As mentioned early, some chemical reactions consume energy and some
produce energy. Thus, energy can be stored in the structure of a
molecule. When a plant uses photosynthesis to rearrange water and
carbon dioxide into a sugar molecule, it converts the energy in
the sunlight( solar energy) into chemical energy. Remember photosythesis is a process that releases energy.
Therefore, the sugar molecule has more chemical energy than the carbon dioxide and water molecules that were
used in its creation.
% ADD: photosythesis equation
% KA: https://www.khanacademy.org/science/ap-biology/cellular-energetics/photosynthesis/a/intro-to-photosynthesis

In our diet, we measure this energy in \textit{kilocalories}. A
calorie is the energy necessary to raise one gram of water one degree
Celsius: it is about 4.19 joules. This is a very small unit: an apple
has about 100,000 calories( 100 kilocalories), so people working with food started
measuring everything in kilocalories.\index{calories}
% ADD: Conversion chapter should come before this chapter

Here is where things get confusing: People who work with food got tired of
saying ``kilocalories'', so they just started using ``Calorie'' to
mean 1,000 calories.  This has created terrible confusion over the
years. So if the C is capitalized, ``Calorie'' probably means kilocalorie.

\section{Kinetic Energy}\index{kinetic energy}

A mass in motion has energy. For example, if you are in a moving car
and you slam on the breaks, the energy from the motion of the
car will be converted into heat in the breaks and under the tires.

How much energy does the car have?
% ADD: section specifically about KE AND U, use roller coaster diagram

\begin{mdframed}[style=important, frametitle={Formula for Kinetic Energy}]

$$E = \frac{1}{2} m v^2$$

where $E$ is the energy in joules, $m$ is the mass in kilograms, and
$v$ is the speed in meters per second.

\end{mdframed}

\section{Gravitational Potential Energy}\index{potential energy!gravitational}
% KA: https://youtu.be/oGzwVYPxKjg

When you lift something heavy onto a shelf, you are giving it
\textit{potential energy}. The amount of energy that you transferred
to it is proportional to its weight and the height that you lifted it.

On the surface of the earth, gravity will accelerate a heavy object downward at
a rate of $9.8 m/s^2$.

\begin{mdframed}[style=important, frametitle={Formula for Gravitational Potential Energy}]
On earth, then, gravitational potential energy is given by

$$E = (9.8)mh$$

where $E$ is the energy in joules, $m$ is the mass of the object you
lifted, and $h$ is the height that you lifted it.

\end{mdframed}

There are other kinds of potential energy. For example, when you draw
a bow, you have given that bow potential energy. When you release it,
the potential energy is transferred to the arrow, which expresses it
as kinetic energy.
% ADD: section about KE and U

\section{Conservation of Energy}

The first law of thermodynamics says ``Energy is neither created nor
destroyed.''\index{energy!conservation of}

Energy can change forms: Your cells consume chemical energy to give
gravitational potential energy to a car you push up a hill. However, the total amount of
energy in a closed system stays constant.
% ADD: Create Systems chapter before introducing concept here

\begin{Exercise}[title={The Energy of Falling}, label=energy_falling]

A 5 kg cannonball falls off the top of a 3 meter ladder. Just before
it hits the floor, all of its gravitational potential energy has been
converted into kinetic energy.  How fast is the cannonball going when
it hits the floor?

\end{Exercise}
\begin{Answer}[ref=energy_falling]

  At the top of the ladder, the cannonball has $(9.8)(5)(3) = 147$ joules of potential energy.

  At the bottom, the kinetic energy $\frac{1}{2}(5)v^2$ must be equal
  to 147 joules. So $v^2 = \frac{294}{5}$.  Thus it is going about
  $7.7$ meters per second.

  (Yes, a tiny amount of energy is lost to air resistance. For a dense
  object moving at these relatively slow speeds, this energy is
  neglible.)

\end{Answer}

\section{Efficiency}
% KA: https://www.khanacademy.org/science/ap-biology/cellular-energetics/cellular-energy/a/the-laws-of-thermodynamics

Although energy is always conserved as it moves through different
forms, scientists aren't always that good at controlling it.\index{efficiency}

For example, a car engine consumes the chemical energy in gasoline. Only
about 20\% of the energy consumed is used to turn the wheels.  Most of
the energy is actually lost as heat. If you run a car for a while, the engine
gets very hot and the exhaust going out the tailpipe turns hot.

A human is about 25\% efficient. Most of the loss is in the heat produced
during the chemical reactions that turns food into motion.
% ADD: Cellular Respiration

In general, if you are trying to increase efficiency in any system,
the solution is usually easy to identify because heat is produced. Reduce heat, Increase efficiency.

Light bulbs are an interesting case. To get the light of a 60 watt
incandescent bulb, you can use an 8 watt LED or a 16 watt fluorescent
light. Thus, we say that the LED light is much more efficient: If you
run both, the incandescent bulb will consume 1.44 kilowatt-hours. The
LED will consume only 0.192 kilowatt-hours.

Besides light, the incandescent bulb is producing a lot of heat. If it
is inside your house, what happens to the heat? It warms your house.

In the winter, when you want light and heat, the incandescent bulb is
100\% efficient!

In the summer, if you are running the air conditioner, the
incandescent bulb is worse than just ``inefficient at making light'' --
it is actually counteracting the air conditioner!