\subsection{Decomposition-based solution algorithm}
In this section, we will present a solution algorithm for the integrated locomotive scheduling and driver rostering problem. Its schematic representation can be seen in Figure~\ref{flow:flowchart}. Its exact definition will be introduced in a future update to this deliverable. Here, we will limit ourselves to a description of its major components.

The most important part of the algorithm is to decompose the large model introduced above into two smaller ones. This is done by relaxing constraints \ref{eq:joint-model-locoDriverCompatibilityConstraint} and \eqref{eq:joint-model-driverLocoCompatibilityConstraint}. A number of computational experiments has shown that one of the decomposed subproblems is far easier to solve than the other. Hence, we develop a number of valid inequality classes for the easier of the subproblems, which will result in its solution being feasible in terms of the other one. These inequalities enabled us to come up with feasible solutions for the instances supplied by our industrial partner. We also developed a long-break heuristic, whose aim is to come up with a partial solution to one of the decomposed subproblems, and hence reduce its computation time.

Additionally, to ensure the global feasibility of the algorithm's solution, we will take advantage of the Combinatorial Benders' Cuts (as introduced by \cite{codato_combinatorial_2006}). These cuts will remove any infeasible solution that was not cut off by the valid inequalities we derived.

\input{04_solution_approach/flowchart}


%%% exercise 07 %%%
\documentclass{article}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath, amssymb, amsfonts, enumitem, fancyhdr, tikz}
% get rid of paragraph indent
\setlength{\parindent}{0 pt}
% allow section.equation numbering
\numberwithin{equation}{section}
% allows you to copy-paste code segments. requires pygments to be installed
% natively, i.e. not just in a Python venv! for example, on WSL Ubuntu you
% need to sudo apt-get install python3-pygments (unfortunately, the version
% will be a bit dated) and cannot rely on your venv Pygments version.
\usepackage{minted}
% makes clickable links to sections
\usepackage{hyperref}
% make the link colors blue, as well as cite colors. urls are magenta
\hypersetup{
    colorlinks, linkcolor = blue, citecolor = blue, urlcolor = magenta
}
% fancy pagestyle so we can use fancyhdr for fancy headers/footers
\pagestyle{fancy}
% add logo in right of header. note that you will have to adjust logo path!
\fancyhead[R]{\includegraphics[scale = 0.15]{../bac_logo1.png}}
% don't show anything in the left and center header
\fancyhead[L, C]{}
% give enough space for logo by reducing top margin height, head separator,
% increasing headerheight. see Figure 1 in the fancyhdr documentation. if
% \topmargin + \headheight + \headsep = 0, original text margins unchanged.
\setlength{\topmargin}{-60 pt}
\setlength{\headheight}{50 pt}
\setlength{\headsep}{10 pt}
% remove decorative line in the fancy header
\renewcommand{\headrulewidth}{0 pt}

% title, author + thanks, date
\title{Exercise 7}
\author{Derek Huang\thanks{NYU Stern 2021, BAC Advanced Team.}}
\date{June 6, 2021}

% shortcut links. the % characters strip extra spacing.
\newcommand{\pytest}{\href{https://docs.pytest.org/en/stable/}{pytest}}
\newcommand{\minimize}{%
    \href{%
        https://docs.scipy.org/doc/scipy/reference/generated/%
        scipy.optimize.minimize.html%
    }{\texttt{scipy.optimize.minimize}}%
}
% don't wanna keep typing this out
\newcommand{\tr}{\operatorname{tr}}
\newcommand{\npinv}{%
    \href{%
        https://numpy.org/doc/stable/reference/generated/numpy.linalg.inv.html%
    }{\texttt{numpy.linalg.inv}}%
}
\newcommand{\pdb}{%
    \href{https://docs.python.org/3/library/pdb.html}{\texttt{pdb}}%
}
\newcommand{\armijobacktrack}{\texttt{\_armijo\_backtrack}}
\newcommand{\nagsolver}{\texttt{nag\_solver}}

\begin{document}

\maketitle
% need to include this after making title to undo the automatic
% \thispagestyle{plain} command that is issued.
\thispagestyle{fancy}

\section{Introduction}

The goal of this exercise is to implement Nesterov's accelerated gradient
method, where step sizes can be either fixed or chosen by the Armijo rule
using backtracking line search.

\section{Instructions}

\subsection{General}

The \texttt{exercise\_07.py} file contains the \texttt{FastGradResult} class,
skeletons for the functions \armijobacktrack{} and \nagsolver, unit tests,
and \pytest{} fixtures. Complete the \armijobacktrack{} and \nagsolver{}
functions by replacing the \texttt{\#\#\# your code goes here \#\#\#} comment
blocks with appropriate Python code.

\medskip

Your code \textbf{must} be written in the areas marked by these blocks. Do
\textbf{not} change any of the pre-written code. The exercise is complete
$ \Leftrightarrow $ \texttt{pytest /path/to/exercise\_07.py} executes with
zero test failures.

\subsection{Implementing \armijobacktrack}

The \armijobacktrack{} method should compute a step size using the
backtracking line search algorithm discussed in the lecture, where the search terminates once the Armijo rule is satisfied. In this implementation, given a
previous parameter estimate $ \mathbf{x}_{t - 1} $, the descent direction
used should be $ -\nabla f(\mathbf{x}_{t - 1}) $, $ f $ the objective, so
$ \eta_t \leftarrow \gamma\eta_t $ until we satisfy the Armijo rule, that is
we have
\begin{equation*}
    f(\mathbf{x}_{t - 1} - \eta_t\nabla f(\mathbf{x}_{t - 1})) \le
    f(\mathbf{x}_{t - 1}) - \alpha\eta_t
    \Vert\nabla f(\mathbf{x}_{t - 1})\Vert_2^2
\end{equation*}

Here $ \alpha, \gamma $ correspond to the \texttt{arm\_alpha},
\texttt{arm\_gamma} parameters respectively. Since the negative gradient is
the descent direction, no descent direction is required as input to
\armijobacktrack. For convenience, use the variables \texttt{eta},
\texttt{fobj}, \texttt{fgrad} defined by the function skeleton in your code.

\subsection{Implementing \nagsolver}

The \nagsolver{} method should implement Nesterov's accelerated gradient
descent for differentiable objectives, i.e. there is no proximal step after
computing the gradient step using the extrapolation points
$ \{\tilde{\mathbf{x}}_t\}_{t \in \mathbb{N}} $. The comments in the
\nagsolver{} function skeleton given additional hints as to what you should
do. For convenience, use the variables \texttt{x}, \texttt{nest\_x}, and
\texttt{grad} defined in the \nagsolver{} skeleton, noting that
$ \forall t \in \mathbb{N} $, \texttt{x} should store the actual parameter
estimates $ \mathbf{x}_{t - 1} $, $ \mathbf{x}_t $ while \texttt{nest\_x}
should store the extrapolation points $ \tilde{\mathbf{x}}_{t - 1} $,
$ \tilde{\mathbf{x}}_t $. Remember that gradient steps should only be
performed on the extrapolation points
$ \{\tilde{\mathbf{x}}_t\}_{t \in \mathbb{N}} $, i.e. on \texttt{nest\_x}.
Also note that the learning rate $ \eta_t $ is controlled by the
\texttt{learning\_rate}, \texttt{arm\_alpha}, \texttt{arm\_gamma} parameters,
where \texttt{learning\_rate="constant"} $ \Rightarrow \forall t \in
\mathbb{N} $, $ \eta_t \triangleq \eta_0 $, while
\texttt{learning\_rate="backtrack"} $ \Rightarrow \eta_t $ is computed using
\armijobacktrack{} with \texttt{arm\_alpha=arm\_alpha} and
\texttt{arm\_gamma=arm\_gamma}.

\subsection{Tips}

\begin{enumerate}
    \item
    Carefully read the docstrings for \armijobacktrack{} and \nagsolver.
    Review slides 7 and 15.

    \item
    Remember that Nesterov's method keeps track of both the actual parameter
    estimates $ \{\mathbf{x}_t\}_{t \in \mathbb{N}} $ and the extrapolation
    points $ \{\tilde{\mathbf{x}}_t\}_{t \in \mathbb{N}} $, but that gradients
    are evaluated only at the extrapolation points.

    \item
    Use NumPy functions whenever possible for efficiency, brevity, and ease
    of debugging.

    \item
    Invoke \pytest{} with the \texttt{--pdb} flag to start \pdb, the Python
    debugger. Doing so allows you to inspect the variables in the current call
    frame and look more closely at what went wrong.
\end{enumerate}

\end{document}


\documentclass[a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{graphicx}
\usepackage{multicol}
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{amsthm}
\usepackage{geometry}
\geometry{a4paper}
\usepackage{fancyhdr}
\usepackage{xcolor}
\usepackage{amssymb}
\usepackage{multicol}
\begin{document}
\author{\textbf{Elshimaa Ahmed}}
\title{\textbf{Math 501 \\
\large Lecture 1\\}}
\date {\today}
\maketitle
\noindent
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\section{Preposition}
\paragraph{}
  a preposition is a claim or a declerative statement which has a truth value , can be proven to be either $True$ or $False$
\paragraph{For Example: }
\begin{itemize}
    \item "Shimaa studies discrete mathimatics" is considered as $Preposition$
    \item "How was your day?" is considered as $ not-Preposition$
    \item $x$ + 1 = 2 is considered as $non$ $declerative$ $statement$ because it is truth value depends on a variable so cannot be proven to be $True$ or $False$ without knowing the value of that variable
\end{itemize}
\section{Logical Operators}
     \begin{definition}[\textbf{Negation}]
       let $p$ a preposition the negation of $p$ , denoted by $\neg$$p$ .The truth value of the negation of $p$ $\neg$ $p$ is the opposite value of $p$
       , expressed in English as "It's not the case that. p"
     \end{definition}
    \textbf{For Example:}
    \begin{itemize}
      \item the negation of "I have more than 5 friends" will become "I have at most 5 friends"
    \end{itemize}

    \begin{definition}[\textbf{Conjunction}]
      let $p$ and $q$ be prepositions , the conjunction of $p$ and $q$ denoted by $p$ $\wedge$ $q$ is a preposition "$p$ and $q$" that become true only if both $p$ and $q$ are both $True$
    \end{definition}

    \begin{definition}[\textbf{Disjunction}]
      let $p$ and $q$ be prepositions .The Disjunction of $p$ and $q$ denoted by $p$ $\vee$ $q$ is a preposition "$p$ or $q$" which is $False$ only if both of $p$ and $q$ are $False$

    \end{definition}
    \begin{definition}[\textbf{Exclusive Disjunction}]
      let $p$ and $q$ be prepositions .The esclusive or denoted by $p$ $\oplus$ $q$ is a preposition that is $True$ if exactly one of $p$ or $q$ are $True$ , and $False$ otherwise
    \end{definition}
\section{Conditional Statements}
\begin{definition}
  let $p$ and $q$ be prepositions . The Conditional Statements $p$ $\rightarrow$ $q$ , "if $p$ then $q$" is false whenever p is $False$ or q is $True$
\end{definition}
The meaning of $p$ $\rightarrow$ $q$ assert that $q$ is true whenever $p$ holds but not vise versa ,when $p$ is $False$ it does not matter what the value of $q$ for implication to be $True$ ,$p$ is called (hypothesis or antecedent or premise ) while $q$ is called conclusion or consequence .
\newline
\newline
\textbf{English Phrases to express conditional statements:}
\begin{multicols}{2}
\begin{itemize}
  \item "if $p$, then $q$"
  \item "$p$ implies $q$"
  \item "$p$ is sufficient of $q$"
  \item "$p$ only if $q$"
  \item "$q$ is necessary for $p$"
  \item "$q$ unless $\neg$ $p$"  \textbf{important}
  \item "$p$ only if $q$"
  \item "$q$ whenever $p$ "
\end{itemize}

\end{multicols}
$\newline$
\textbf{Converse, Contrapositive, and Inverse:}
for conditional statement $p$ $\rightarrow$ $q$
\begin{itemize}
  \item $q$ $\rightarrow$ $p$ called the converse .
  \item $\neg q$ $\rightarrow$ $\neg p$ called the Contrapositive and has the same truth value as the original statement
  \item $\neg p$ $\rightarrow$ $\neg q$ called the inverse
\end{itemize}
\begin{definition}
  Let $p$ and $q$ be prepositions , the biconditional statement $p$ $\leftrightarrow$ $q$ is a preposition "$p$ if and only if $q$" which is $True$ when $p$ and $q$ have the same truth values and $False$ otherwise
\end{definition}
\section{Precedence of Logical Operators}
\begin{tabular}{|c|c|c|}
 \hline
 Logical Operator & Precedence \\
 \hline
 $\neg$ & 1\\
 \hline
 $\wedge$ & 2\\
 $\vee$ & 3\\
 \hline
 $\rightarrow$ & 4\\
 $\leftrightarrow $& 5\\
 \hline
\end{tabular}
\section{Logical Equavilance}
two preposition $p$, $q$ are said to be equavilant if $p\leftrightarrow q$ is a $tautology$
\begin{itemize}
  \item $p\rightarrow q \equiv \neg p \vee q$
  \item $p\leftrightarrow q \equiv (p\rightarrow q)\wedge (q\rightarrow p)$
  \item $p \wedge T_{0} \equiv p$ , $p \vee F_{0} \equiv p$ (identity law)
  \item $p \wedge F_{0} \equiv F_{0}$ , $p \vee T_{0} \equiv T_{0}$ (domination law)
  \item $p \wedge p\equiv p$ , $p \vee p \equiv p$ (idempotent (okay to apply many times))
  \item $\neg (p \wedge q) \equiv \neg p \vee \neg q$ , $\neg (p \vee q) \equiv \neg p \wedge \neg q$ (De morgans law)
  \item $ p \wedge (p \vee q) \equiv p$ , $p \vee (p \wedge q)\equiv p$ (apsorption law)
\end{itemize}
\end{document}


\subsection{Bayesian causal forests}


\documentclass[10pt]{article}

% Manage page layout
\usepackage[margin=2.5cm, includefoot, footskip=30pt]{geometry}
\pagestyle{plain}
\setlength{\parindent}{0em}
\setlength{\parskip}{1em}
\renewcommand{\baselinestretch}{1}

%%%%%%%PACKAGES HERE%%%%%%%
\usepackage{tikz}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{mathtools}
\usepackage{amsthm}
\usepackage{graphicx}
\usepackage{subcaption}
\usepackage{standalone}
\usepackage{booktabs}
\usepackage{setspace}
\usepackage[algoruled,lined]{algorithm2e}
\usepackage[noend]{algpseudocode}
\usepackage{wrapfig}
\usepackage{hyperref}
\usepackage{authblk}
\usepackage[toc,page]{appendix}
\usetikzlibrary{calc, shapes, patterns, decorations.pathreplacing}

\makeatletter
\def\BState{\State\hskip-\ALG@thistlm}
\makeatother

\newcommand{\R}{\mathbb{R}}
\newtheorem{theorem}{Theorem}
\usetikzlibrary{decorations.pathmorphing, decorations.pathreplacing, angles,
                quotes, calc, er, positioning}

\newtheorem{lemma}[theorem]{Lemma}
\def\arraystretch{1.5}

\title{Stability of defection, optimisation of strategies and the limits of
       memory in the Prisoner's Dilemma.}
\author[1]{Nikoleta E. Glynatsi}
\author[1]{Vincent A. Knight}

\affil[1]{Cardiff University, School of Mathematics, Cardiff, United Kingdom}
\date{}
\setcounter{Maxaffil}{0}
\renewcommand\Affilfont{\itshape\small}

\begin{document}

\maketitle

\begin{abstract}
    Memory-one strategies are a set of Iterated Prisoner's Dilemma strategies
    that have been praised for their mathematical tractability and performance
    against single opponents. This manuscript investigates \textit{best
    response} memory-one strategies as a multidimensional
    optimisation problem. Though extortionate memory-one strategies have gained
    much attention, we demonstrate that best response memory-one strategies do not
    behave in an extortionate way, and moreover, for memory one strategies to be
    evolutionary robust they need to be able to behave in a forgiving way. We
    also provide evidence that memory-one strategies suffer from their limited
    memory in multi agent interactions and can be out performed by
    longer memory strategies.
\end{abstract}

The Prisoner's Dilemma (PD) is a two player game used in understanding the
evolution of cooperative behaviour, formally introduced in~\cite{Flood1958}.
Each player has two options, to cooperate (C) or to defect (D). The decisions
are made simultaneously and independently. The normal form representation of the
game is given by:

\begin{equation}\label{equ:pd_definition}
    S_p =
    \begin{pmatrix}
        R & S  \\
        T & P
    \end{pmatrix}
    \quad
    S_q =
    \begin{pmatrix}
        R & T  \\
        S & P
    \end{pmatrix}
\end{equation}

where \(S_p\) represents the utilities of the row player and \(S_q\) the
utilities of the column player. The payoffs, \((R, P, S, T)\), are constrained
by \(T > R > P > S\) and \(2R > T + S\), and the most common values used in the
literature are \((R, P, S, T) = (3, 1, 0, 5)\)~\cite{Axelrod1981}.
The PD is a one shot game, however, it is commonly studied in a manner where the
history of the interactions matters. The repeated form of the game is called the
Iterated Prisoner's Dilemma (IPD).

Memory-one strategies are a set of IPD strategies that have been
studied thoroughly in the literature~\cite{Nowak1990, Nowak1993}, however, they have gained
most of their attention when a certain subset of memory-one strategies was
introduced in~\cite{Press2012}, the zero-determinant strategies (ZDs). In~\cite{Stewart2012} it
was stated that ``Press and Dyson have fundamentally changed the viewpoint on
the Prisoner's Dilemma''.
A special case of ZDs are extortionate strategies that choose their actions so that a linear relationship is forced
between the players' score ensuring that they will always
receive at least as much as their opponents. ZDs are
indeed mathematically unique and are proven to be robust in pairwise
interactions, however, their true effectiveness in tournaments and
evolutionary dynamics has been questioned~\cite{adami2013, Hilbe2013b,
Hilbe2013, Hilbe2015, Knight2018, Harper2015}.

In a similar fashion to~\cite{Press2012} the purpose of this work is to consider
a given memory-one strategy; however, whilst~\cite{Press2012} found a way for a
player to manipulate a given opponent, this work will consider a
multidimensional optimisation approach to identify the best response to a given
group of opponents. The outcomes of our work reinforce known results, namely
that memory-one strategies must be forgiving to be evolutionarily stable~\cite{Stewart2013, Stewart2016}
and that longer-memory strategies have a certain form of advantage over short memory
strategies~\cite{Hilbe2017, Pan2015}.

In particular, this work presents a compact method of
identifying the best response memory-one strategy against a given set of
opponents, and evaluates whether it behaves in a
zero-determinant way which in turn indicates whether it can be extortionate. This is also done in evolutionary settings. Moreover, we
introduce a well designed framework that allows the comparison of an optimal
memory one strategy and a more complex strategy which has a larger memory. This
is used to identify conditions for which defection is stable; thus identifying
environments where cooperation will not occur.

\section{Methods and Results}
\subsection{Utility}

One specific advantage of memory-one strategies is their mathematical
tractability. They can be represented completely as an element of \(\R^{4}_{[0, 1]}\). This
originates from~\cite{Nowak1989} where it is stated that if a strategy is
concerned with only the outcome of a single turn then there are four possible
`states' the strategy could be in; both players cooperated (\(CC\)),
the first player cooperated whilst the second player defected (\(CD\)),
the first player defected whilst the second player cooperated (\(DC\)) and
both players defected (\(DD\)).
Therefore, a memory-one strategy can be denoted by the probability vector of
cooperating after each of these states; \(p=(p_1, p_2, p_3, p_4) \in \R_{[0,1]}
^ 4\).

In~\cite{Nowak1989} it was shown that it is not necessary to simulate the play
of a strategy $p$ against a memory-one opponent $q$. Rather this exact behaviour
can be modeled as a stochastic process, and more specifically as a Markov chain
whose corresponding transition matrix \(M\) is
given by Eq.~\ref{eq:transition_matrix}. The long run steady state probability
vector \(v\), which is the solution to \(v M = v\), can be
combined with the payoff matrices of Eq.~\ref{equ:pd_definition} to give the expected
payoffs for each player. More specifically, the utility for a memory-one
strategy \(p\) against an opponent \(q\), denoted as \(u_q(p)\), is given by
Eq.~\ref{eq:press_dyson_utility}.

\begin{equation}\label{eq:transition_matrix}
    \resizebox{.5\hsize}{!}{$\input{tex/m_matrix.tex}$}
\end{equation}

\begin{equation}\label{eq:press_dyson_utility}
    u_q(p) = v \cdot (R, S, T, P).
\end{equation}

This manuscript has explored the form of \(u_q(p)\), to the authors knowledge no
previous work has done this, and it proves that \(u_q(p)\) is given by a ratio
of two quadratic forms~\cite{kepner2011},
(Theorem~\ref{theorem_one}):

\begin{equation}\label{eq:optimisation_quadratic}
       u_q(p) = \frac{\frac{1}{2}pQp^T + cp + a}
                   {\frac{1}{2}p\bar{Q}p^T + \bar{c}p + \bar{a}},
\end{equation}

where \(Q=Q(q), \bar{Q}=\bar{Q}(q)\) \(\in \R^{4\times4}\), \(c=c(q) \text{ and } \bar{c}=\bar{c}(q)\)
\(\in \R^{4 \times 1}\), \(a=a(q) \text{ and } \bar{a}=\bar{a}(q) \in \R.\)

This can be extended to consider multiple
opponents. The IPD is commonly studied in tournaments and/or Moran Processes
where a strategy interacts with a number of opponents. The payoff of a player in
such interactions is given by the average payoff the player received against
each opponent. More specifically the expected utility of a memory-one strategy
against a \(N\) number of opponents is given by:

\begin{align}\label{eq:tournament_utility}
       & \frac{1}{N} \sum\limits_{i=1} ^ {N} {u_q}^{(i)} (p) =
       \frac{\frac{1}{N} \sum\limits_{i=1} ^ {N} (\frac{1}{2} pQ^{(i)} p^T + c^{(i)} p + a^ {(i)})
       \prod\limits_{\tiny\begin{array}{l} j=1 \\ j \neq i \end{array}} ^
       N (\frac{1}{2} p\bar{Q}^{(j)} p^T + \bar{c}^{(j)} p + \bar{a}^ {(j)})}
       {\prod\limits_{i=1} ^ N (\frac{1}{2} p\bar{Q}^{(i)} p^T + \bar{c}^{(i)} p + \bar{a}^ {(i)})}.
\end{align}

Estimating the utility of a memory-one strategy against any number of opponents
without simulating the interactions is the main result used in the rest of this manuscript.
It will be used to obtain best response memory-one strategies, in tournaments
and evolutionary dynamics, and to explore the conditions under which defection
dominates cooperation.

\subsection{Stability of defection}\label{subsection:stability_defection}

An immediate result from our formulation can be
obtained by evaluating the sign of Eq. \ref{eq:tournament_utility}'s derivative
at \(p=(0, 0, 0, 0)\). If at that point the
derivative is negative, then the utility of a player only decreases if they were
to change their behaviour, and thus \textbf{defection at that point is stable}.

\begin{lemma}\label{lemma:stability_of_defection}
    In a tournament of \(N\) players \(\{q^{(1)}, q^{(2)}, \dots, q^{(N)} \}\)
    for \(q^{(i)} \in \R_{[0, 1]} ^ 4\)
    defection is stable if the transition probabilities of the
    opponents satisfy conditions Eq. \ref{eq:defection_condition_one} and Eq. \ref{eq:defection_condition_two}.

    \begin{equation}\label{eq:defection_condition_one}
        \sum_{i=1} ^ N (c^{(i)T} \bar{a}^{(i)} - \bar{c}^{(i)T} a^{(i)}) \leq 0
    \end{equation}

    while,

    \begin{equation}\label{eq:defection_condition_two}
        \sum_{i=1} ^ N \bar{a}^{(i)} \neq 0
    \end{equation}
\end{lemma}

\begin{proof}
    For defection to be stable the derivative of the utility
    at the point \(p = (0, 0, 0, 0)\) must be negative.

    Substituting \(p = (0, 0, 0, 0)\) in
    Eq. \ref{eq:mo_tournament_derivative} gives:

    \begin{equation}
        \left.\frac{d\sum\limits_{i=1} ^ {N} {u_q}^{(i)} (p)}{dp} \right\rvert_{p=(0,0,0,0)} =
    \sum_{i=1} ^ N \frac{(c^{(i)T} \bar{a}^{(i)} - \bar{c}^{(i)T} a^{(i)})}
    {(\bar{a}^{(i)})^2}
    \end{equation}

    The sign of the numerator \( \displaystyle\sum_{i=1} ^ N (c^{(i)T} \bar{a}^{(i)} - \bar{c}^{(i)T} a^{(i)})\)
    can vary based on the transition probabilities of the opponents.
    The denominator can not be negative, and otherwise is always positive.
    Thus the sign of the derivative is negative if and only if
    \( \displaystyle\sum_{i=1} ^ N (c^{(i)T} \bar{a}^{(i)} - \bar{c}^{(i)T} a^{(i)}) \leq 0\).
\end{proof}

Consider a population for which defection is known to be stable. In that
population all the members will over time adopt the same behaviour; thus in such
population cooperation will never take over. This is demonstrated in
Fig.~\ref{fig:stability_of_defection}.
These have been simulated using~\cite{axelrodproject} an open
source research framework for the study of the IPD.

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=.4\linewidth]{img/stability_of_defection_plots.pdf}
    \caption{A. For \(q_{1}=(0.22199, 0.87073, 0.20672, 0.91861)\),
    $q_{2}=(0.48841, 0.61174, 0.76591, 0.51842)$ and
    $q_{3}=(0.2968, 0.18772, 0.08074, 0.73844)$, Eq.~\ref{eq:defection_condition_one} and
    Eq.~\ref{eq:defection_condition_two} hold and Defector takes over the
    population. B. For $q_{1}=(0.96703, 0.54723, 0.97268, 0.71482)$,
    $q_{2}=(0.69773, 0.21609, 0.97627, 0.0062)$ and
    $q_{3}=(0.25298, 0.43479, 0.77938, 0.19769)$, Eq.~\ref{eq:defection_condition_one} fails
    and Defector does not take over the population.}\label{fig:stability_of_defection}
\end{figure}

\subsection{Best response memory-one strategies}\label{section:best_response_memory_one}

As discussed ZDs have been acclaimed for their robustness
against a single opponent. ZDs are evidence that extortion works
in pairwise interactions, their behaviour ensures that the strategies will never
lose a game. However, this paper argues that in multi opponent interactions,
where the payoffs matter, strategies trying to exploit their opponents will
suffer.
Compared to ZDs, best response memory-one strategies, which have a
theory of mind of their opponents, utilise their behaviour in order to gain the
most from their interactions. The question that arises then is whether best
response strategies are optimal because they behave in an extortionate way.

To answer this question, we initially define \textit{memory-one best response}
strategies as a multi dimensional optimisation problem given by:

\begin{equation}\label{eq:mo_tournament_optimisation}
    \begin{aligned}
    \max_p: & \ \sum_{i=1} ^ {N} {u_q}^{(i)} (p)
    \\
    \text{such that}: & \ p \in \R_{[0, 1]}
    \end{aligned}
\end{equation}

Optimising this particular ratio of quadratic forms is not trivial. It can be
verified empirically for the case of a single opponent that there exists at
least one point for which the definition of concavity does not hold.
The non concavity of \(u(p)\) indicates multiple local
optimal points. This is also intuitive. The best response against a cooperator,
\(q=(1, 1, 1, 1)\), is a defector \(p^*=(0, 0, 0, 0)\). The strategies
\(p=(\frac{1}{2}, 0, 0, 0)\) and \(p=(\frac{1}{2}, 0, 0, \frac{1}{2})\) are also
best responses. The approach taken here is to introduce a compact way of
constructing the discrete candidate set of all local optimal points, and evaluating
the objective function Eq.~\ref{eq:tournament_utility}. This gives the best
response memory-one strategy. The approach is given in
Theorem~\ref{memone_group_best_response}.

Finding best response memory-one strategies is analytically feasible using the
formulation of Theorem~\ref{memone_group_best_response} and resultant
theory~\cite{Jonsson2005}. However, for large systems building the resultant
becomes intractable. As a result, best responses will be estimated
heuristically using a numerical method, suitable for problems with local optima,
called Bayesian optimisation~\cite{Mokus1978}.

This is extended to evolutionary settings. In these settings
self interactions are key. Self interactions can be incorporated in the
formulation that has been used so far. More specifically, the optimisation
problem of Eq.~\ref{eq:mo_tournament_optimisation} is extended to include self
interactions:

\begin{equation}\label{eq:mo_evolutionary_optimisation}
\begin{aligned}
\max_p: & \ \frac{1}{N} \sum\limits_{i=1} ^ {N} {u_q}^{(i)} (p) + u_p(p)
\\
\text{such that}: & \ p \in \R_{[0, 1]}
\end{aligned}
\end{equation}

For determining the memory-one best response in an evolutionary setting, an
algorithmic approach is considered, called \textit{best response dynamics}. The
best response dynamics approach used in this manuscript is given by
Algorithm~\ref{algo:best_response_dynamics}.

\begin{center}
\begin{minipage}{.55\textwidth}
\begin{algorithm}[H]
       $p^{(t)}\leftarrow (1, 1, 1, 1)$\;
       \While{$p^{(t)} \neq p ^{(t -1)}$}{
       $p^{(t + 1)} =  \text{argmax} \frac{1}{N} \sum\limits_{i=1} ^ {N} {u_q}^{(i)}
       (p^{(t + 1)}) + u_p^{(t)}(p^{(t + 1)})$\;
       }
       \caption{Best response dynamics Algorithm}
       \label{algo:best_response_dynamics}
\end{algorithm}
\end{minipage}
\end{center}

The results of this section use Bayesian optimisation to generate a data set of best response
memory-one strategies, in tournaments and evolutionary dynamics whilst \(N=2\).
The data set is available at~\cite{glynatsi2019}. It contains a total of 1000 trials
corresponding to 1000 different instances of a best response strategy in
tournaments and evolutionary dynamics. For each trial a set of 2 opponents is
randomly generated and the memory-one best responses against them is found.

The source code for the experiments presented in this manuscript has been written in a sustainable manner~\cite{Benureau2018}.
It is open source (\url{https://github.com/Nikoleta-v3/Memory-size-in-the-prisoners-dilemma})
and tested which ensures the validity of the results. It has also been archived
and can be found at~\cite{nikoleta_glynatsi_2019}.

In order to investigate whether best responses
behave in an extortionate matter the SSE method~\cite{Knight2019} is used.
In~\cite{Knight2019} it is proven that
all extortionate ZDs reside on a triangular plane. For a given \(p\), a strategy
\(x^*\) is defined as the nearest ZDs. The distance between
the two strategies is explicitly calculated and referred to as the sum of squared
errors of prediction (SSE); which corresponds to how far \(p\) is from behaving as a ZDs.
Thus, a high SSE implies non ZD, which in turn implies a non extortionate behaviour.
The SSE method has been applied to the data set.
A statistics summary
of the SSE distribution for the best response in tournaments and evolutionary dynamics is
given in Table~\ref{table:sserror_stats}.

For the best response in tournaments the distribution of SSE is skewed to the
left, indicating that the best response does exhibit ZDs behaviour and so could
be extortionate,
however, the best response is not uniformly a ZDs. A positive measure of
skewness and kurtosis indicates a heavy tail to the right. Therefore, in several
cases the strategy is not trying to extort its opponents. Similarly the
evolutionary best response strategy does not behave uniformly extortionately. A
larger value of both the kurtosis and the skewness of the SSE distribution
indicates that in evolutionary settings a memory-one best response is even more
adaptable.

The difference between best responses in tournaments and in evolutionary
settings is further explored by Fig.~\ref{fig:behaviour_violin_plots}.
Though, no statistically significant differences have been found, from
Fig.~\ref{fig:behaviour_violin_plots}, it seems that evolutionary best
response has a higher median $p_2$; which corresponds to the probability of cooperating
after receiving a defection. Thus, they are more likely to forgive after
being tricked. This is due to the fact that they could be playing against
themselves, and they need to be able to forgive so that future cooperation can
occur.

\begin{table}
\begin{center}
\resizebox{.6\columnwidth}{!}{%
\begin{tabular}{lrrrrrrrrrrr}
    \toprule
    & mean & std  & 5\% & 50\% &  95\% & max & median & skew & kurt\\
    \midrule
\textbf{Tournament} & 0.34  & 0.40  & 0.028  & 0.17  &
1.05  & 2.47  & 0.17  & 1.87 & 3.60 \\
\textbf{Evolutionary Setting} & 0.17 & 0.23 & 0.01 &
0.12 & 0.67 & 1.53 & 0.12 & 3.42 & 1.92 \\
    \bottomrule
\end{tabular}}
\end{center}
\caption{SSE of best response memory-one when \(N=2\)}\label{table:sserror_stats}
\end{table}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=.55\textwidth]{img/behaviour_violin_plots.pdf}
    \caption{Distributions of \(p^*\) for best responses in tournaments and
    evolutionary settings. The medians, denoted as \(\bar{p}^*\), for tournaments
    are \(\bar{p}^* = (0, 0, 0, 0)\), and for evolutionary settings
    \(\bar{p}^* = (0, 0.19, 0, 0)\).}
    \label{fig:behaviour_violin_plots}
\end{figure}

\subsection{Longer memory best responses}

This section focuses on the memory size of strategies. The effectiveness of
memory in the IPD has been previously explored in the literature, however, no one
has compared the performance of longer-memory
strategies to memory-one best responses.

In~\cite{Harper2017}, a strategy called \textit{Gambler} which makes
probabilistic decisions based on the opponent's \(n_1\) first moves, the
opponent's \(m_1\) last moves and the player's \(m_2\) last moves was
introduced. In this manuscript Gambler with parameters: $n_1 = 2, m_1 = 1$ and $m_2 = 1$ is used
as a longer-memory strategy.
By considering the opponent's first two moves, the opponents last move and the
player's last move, there are only 16 $(4 \times 2 \times 2)$ possible outcomes
that can occur, furthermore, Gambler also makes a probabilistic decision of
cooperating in the opening move. Thus, Gambler is a function \(f: \{\text{C,
D}\} \rightarrow [0, 1]_{\R}\). This can be hard coded as an element
of \([0, 1]_{\R} ^ {16 + 1}\), one probability for each outcome plus the opening
move. Hence, compared to Eq.~\ref{eq:mo_tournament_optimisation}, finding an
optimal Gambler is a 17 dimensional problem given by:

\begin{equation}\label{eq:gambler_optimisation}
    \begin{aligned}
    \max_p: & \ \sum_{i=1} ^ {N} {U_q}^{(i)} (f)
    \\
    \text{such that}: & \ f \in \R_{[0, 1]}^{17}
    \end{aligned}
\end{equation}

Note that Eq. \ref{eq:tournament_utility} can not be used here for the utility
of Gambler, and actual simulated players are used. This is done using~\cite{axelrodproject}
with 500 turns and 200 repetitions, moreover, Eq. \ref{eq:gambler_optimisation}
is solved numerically using Bayesian optimisation.

Similarly to previous sections, a large data set has been generated with
instances of an optimal Gambler and a memory-one best response, available
at~\cite{glynatsi2019}. Estimating a best response Gambler (17 dimensions) is
computational more expensive compared to a best response memory-one (4
dimensions). As a result, the analysis of this section is based on a total of
152 trials. For each trial two random opponents have been selected. The 152 pair
of opponents are a sub set of the opponents used in section~\ref{section:best_response_memory_one}.

The ratio between Gambler's utility and the best response memory-one strategy's utility has been calculated and its distribution in
given in Fig.~\ref{fig:utilities_gambler_mem_one}.
It is evident from Fig.~\ref{fig:utilities_gambler_mem_one} that
Gambler always performs as well as the best response memory-one strategy and often performs better. There are
no points where the ratio value is less than 1, thus Gambler never performed less
than the best response memory-one strategy and in places outperforms it. This seems to be at odd with the
result of~\cite{Press2012} that against a memory-one opponent having a longer memory
will not give a strategy any
advantage. However, against two memory-one opponents Gambler's performance is better than
the optimal memory-one strategy. This is evidence that in the case of two opponents having a
shorter memory is limiting.

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=.5\textwidth]{img/gambler_performance_against_mem_one.pdf}
    \caption{The ratio between the utilities of Gambler and best response memory-one
    strategy for 152 different pair of opponents.}\label{fig:utilities_gambler_mem_one}
\end{figure}

\section{Discussion}
This manuscript has considered \textit{best response} strategies in the IPD game, and
more specifically, \textit{memory-one best responses}. It has proven that there is
a compact way of identifying a memory-one best response to a group of opponents,
and moreover it obtained a condition for which in an
environment of memory-one opponents defection is the stable choice, based only
on the coefficients of the opponents.
The later parts of this paper focused on a series of empirical results, where it
was shown that the performance and the evolutionary stability of memory-one
strategies rely on adaptability and not on extortion. Finally, it was shown that
memory-one strategies' performance is limited by their memory in cases where
they interact with multiple opponents.

Following the work described in~\cite{Nowak1989}, where it was shown that the
utility between two memory-one strategies can be estimated by a Markov
stationary state, we proved that the utilities can be written as a ration of two
quadratic forms in $R^4$, Theorem~\ref{theorem_one}. This was extended to
include multiple opponents, as the IPD is commonly studied in such situations.
This formulation allowed us to introduce an approach for identifying memory-one
best responses to any number of opponents;
Theorem~\ref{memone_group_best_response}. This does not only have game theoretic
novelty, but also a mathematical novelty of solving quadratic ratio optimisation
problems where the quadratics are non concave. The results were used to
define a condition for which defection is known to be stable.

This manuscript presented several experimental results. All data for the results
is archived in~\cite{glynatsi2019}. These results were mainly to investigate the
behaviour of memory-one strategies and their limitations. A large data set which
contained best responses in tournaments and in evolutionary settings for $N=2$
was generated. This allowed us to investigate their respective behaviours, and
whether it was extortionate acts that made them the most favorable strategies.
However, it was shown that it was not extortion but adaptability that allowed
the strategies to gain the most from their interactions. In evolutionary settings
it was shown that the best response strategy was even more adaptable, and there
is some evidence that it is more likely to forgive after being tricked.
Moreover, the performance of
memory-one strategies was put against the performance of a longer memory
strategy called Gambler. There were several cases where Gambler would outperform
the memory-one strategy, however, a memory-one strategy never managed to
outperform a Gambler. This result occurred whilst considering a Gambler with a
sufficiently larger memory but not a sufficiently larger amount of information
regarding the game.

All the empirical results presented in this manuscript have been for the case of
$N=2$. In future work we would consider larger values of $N$, however, we
believe that for larger values of $N$ the results that have been presented here
would only be more evident. In addition, we would investigate potential
theoretical results for the evolutionary best responses dynamics algorithm
discussed.

By specifically exploring the entire memory space-one strategies to identify
the optimal strategy for a variety of situations, this work casts doubt
on the effectiveness of ZDs, highlights the importance of adaptability and provides
a framework for the continued understanding of these important questions.

\section{Acknowledgements}

A variety of software libraries have been used in this work:

\begin{itemize}
    \item The Axelrod library for IPD simulations~\cite{axelrodproject}.
    \item The Scikit-optimize library for an implementation of Bayesian optimisation~\cite{tim_head_2018_1207017}.
    \item The Matplotlib library for visualisation~\cite{hunter2007matplotlib}.
    \item The SymPy library for symbolic mathematics~\cite{sympy}.
    \item The Numpy library for data manipulation~\cite{walt2011numpy}.
\end{itemize}

% Bibliography
\bibliographystyle{plain}
\bibliography{bibliography.bib}

\section{Appendix}

\subsection{Theorem~\ref{theorem_one}}
\begin{theorem}\label{theorem_one}

    The expected utility of a memory-one strategy \(p\in\mathbb{R}_{[0,1]}^4\)
    against a memory-one opponent \(q\in\mathbb{R}_{[0,1]}^4\), denoted
    as \(u_q(p)\), can be written as a ratio of two quadratic forms:

    \begin{equation}\label{eq:optimisation_quadratic}
    u_q(p) = \frac{\frac{1}{2}pQp^T + cp + a}
                {\frac{1}{2}p\bar{Q}p^T + \bar{c}p + \bar{a}},
    \end{equation}
    where \(Q, \bar{Q}\) \(\in \R^{4\times4}\) are square matrices defined by the
    transition probabilities of the opponent \(q_1, q_2, q_3, q_4\) as follows:

    \begin{center}
    \begin{equation}
    \resizebox{.65\linewidth}{!}{\arraycolsep=2.5pt%
    \boldmath\(
    Q = \input{tex/q_numerator}\)},
    \end{equation}
    \begin{equation}\label{eq:q_bar_matrix}
    \resizebox{.65\linewidth}{!}{\arraycolsep=2.5pt%
    \boldmath\(
    \bar{Q} =  \input{tex/q_denominator}\)}.
    \end{equation}
    \end{center}

    \(c \text{ and } \bar{c}\) \(\in \R^{4 \times 1}\) are similarly defined by:

    \begin{equation}\label{eq:q_matrix_numerator}
    \resizebox{0.25\linewidth}{!}{\arraycolsep=2.5pt%
    \boldmath\(c = \input{tex/c_numerator}\),}
    \end{equation}
    \begin{equation}\label{eq:q_matrix_denominator}
    \resizebox{0.25\linewidth}{!}{\arraycolsep=2.5pt%
    \boldmath\(\bar{c} = \input{tex/c_denominator}\),
    }
    \end{equation}
    and the constant terms \(a, \bar{a}\) are defined as \(a = \input{tex/numerator_constant}\) and
    \(\bar{a} = \input{tex/denominator_constant}\).
\end{theorem}

\begin{proof}

    It was discussed that \(u_q(p)\) it is the product of the steady states \(v\) and
    the PD payoffs,

    \[u_q(p) = v \cdot (R, S, T, P).\]

    More specifically, with \((R, P, S, T) = (3, 1, 0, 5)\)

    \begingroup
    \footnotesize
    \begin{equation}
        u_q(p) =
        \left(
          \frac
            {\parbox{6in}{$
                p_{1} p_{2} (q_{1} q_{2} - 5 q_{1} q_{4} - q_{1} - q_{2} q_{3} + 5 q_{3} q_{4} + q_{3}) + p_{1} p_{3} (- q_{1} q_{3} + q_{2} q_{3}) + p_{1} p_{4} (5 q_{1} q_{3} - 5 q_{3} q_{4}) + p_{3} p_{4} (- 3 q_{2} q_{3} + 3 q_{3} q_{4}) +$ \\
                \hspace*{1cm} $ p_{2} p_{3} (- q_{1} q_{2} + q_{1} q_{3} + 3 q_{2} q_{4} + q_{2} - 3 q_{3} q_{4} - q_{3}) + p_{2} p_{4} (- 5 q_{1} q_{3} + 5 q_{1} q_{4} + 3 q_{2} q_{3} - 3 q_{2} q_{4} + 2 q_{3} - 2 q_{4}) + $ \\
                \hspace*{1cm} $ p_{1} (- q_{1} q_{2} + 5 q_{1} q_{4} + q_{1}) + p_{2} (q_{2} q_{3} - q_{2} - 5 q_{3} q_{4} - q_{3} + 5 q_{4} + 1) + p_{3} (q_{1} q_{2} - q_{2} q_{3} - 3 q_{2} q_{4} - q_{2} + q_{3}) +$ \\
                \hspace*{4cm} $ p_{4} (- 5 q_{1} q_{4} + 3 q_{2} q_{4} + 5 q_{3} q_{4} - 5 q_{3} + 2 q_{4}) + q_{2} - 5 q_{4} - 1$
            }}
            {\parbox{6in}{$
            p_{1} p_{2} (q_{1} q_{2} - q_{1} q_{4} - q_{1} - q_{2} q_{3} + q_{3} q_{4} + q_{3}) + p_{1} p_{3} (- q_{1} q_{3} + q_{1} q_{4} + q_{2} q_{3} - q_{2} q_{4}) + p_{1} p_{4} (- q_{1} q_{2} + q_{1} q_{3} + q_{1} + q_{2} q_{4} - q_{3} q_{4} - q_{4}) +$ \\
            $ p_{2} p_{3} (- q_{1} q_{2} + q_{1} q_{3} + q_{2} q_{4} + q_{2} - q_{3} q_{4} - q_{3}) + p_{2} p_{4} (- q_{1} q_{3} + q_{1} q_{4} + q_{2} q_{3} - q_{2} q_{4}) + p_{3} p_{4} (q_{1} q_{2} - q_{1} q_{4} - q_{2} q_{3} - q_{2} + q_{3} q_{4} + q_{4}) + $ \\
            $ p_{1} (- q_{1} q_{2} + q_{1} q_{4} + q_{1}) + p_{2} (q_{2} q_{3} - q_{2} - q_{3} q_{4} - q_{3} + q_{4} + 1) + p_{3} (q_{1} q_{2} - q_{2} q_{3} - q_{2} + q_{3} - q_{4}) + p_{4} (- q_{1} q_{4} + q_{2} + q_{3} q_{4} - q_{3} + q_{4} - 1) + $ \\
            \hspace*{7cm} $q_{2} - q_{4} - 1$
          }}
        \right).
    \end{equation}
    \endgroup

    Let us consider the numerator of \(u_q(p)\). The cross product terms \(p_ip_j\)
    are given by,

    \begingroup
    \footnotesize
    \begin{align*}
    p_{1} p_{2} (q_{1} q_{2} - 5 q_{1} q_{4} - q_{1} - q_{2} q_{3} + 5 q_{3} q_{4}
    + q_{3}) + p_{1} p_{3} (- q_{1} q_{3} + q_{2} q_{3}) + p_{1} p_{4} (5 q_{1} q_{3} -
    5 q_{3} q_{4}) + p_{3} p_{4} (- 3 q_{2} q_{3} + 3 q_{3} q_{4}) +  \\
    p_{2} p_{3} (- q_{1} q_{2} + q_{1} q_{3} + 3 q_{2} q_{4} + q_{2} - 3 q_{3} q_{4} - q_{3}) +
    p_{2} p_{4} (- 5 q_{1} q_{3} + 5 q_{1} q_{4} + 3 q_{2} q_{3} - 3 q_{2} q_{4} +
    2 q_{3} - 2 q_{4}).
    \end{align*}
    \endgroup

    This can be re written in a matrix format given by Eq.~\ref{eq:cross_product_coeffs}.

    \begin{equation}\label{eq:cross_product_coeffs}
        \resizebox{0.8\linewidth}{!}{\arraycolsep=2.5pt%
        \boldmath\(
        (p_1, p_2, p_3, p_4) \frac{1}{2} \input{tex/q_numerator} \begin{pmatrix}
        p_1 \\
        p_2 \\
        p_3 \\
        p_4 \end{pmatrix}
        \) }
    \end{equation}

    Similarly, the linear terms are given by,

    \begingroup
    \footnotesize
    \begin{align*}
    p_{1} (- q_{1} q_{2} + 5 q_{1} q_{4} + q_{1}) + p_{2} (q_{2} q_{3} - q_{2} - 5 q_{3} q_{4} - q_{3} + 5 q_{4} + 1) + p_{3} (q_{1} q_{2} - q_{2} q_{3} - 3 q_{2} q_{4} - q_{2} + q_{3}) + \\
    p_{4} (- 5 q_{1} q_{4} + 3 q_{2} q_{4} + 5 q_{3} q_{4} - 5 q_{3} + 2 q_{4}).
    \end{align*}
    \endgroup

    and the expression can be written using a matrix format as Eq.~\ref{eq:linear_coeffs}.

    \begin{equation}\label{eq:linear_coeffs}
        \resizebox{0.38\linewidth}{!}{\arraycolsep=2.5pt%
        \boldmath\(
        (p_1, p_2, p_3, p_4) \input{tex/c_numerator}\)}
    \end{equation}

    Finally, the constant term of the numerator, which is obtained by substituting
    $p=(0, 0, 0, 0)$, is given by Eq.~\ref{eq:constant}.

    \begin{equation}\label{eq:constant}
    q_{2} - 5 q_{4} - 1
    \end{equation}

    Combining Eq.~\ref{eq:cross_product_coeffs}, Eq.~\ref{eq:linear_coeffs} and Eq.~\ref{eq:constant}
    gives that the numerator of \(u_q(p)\) can be written as,

    \begingroup
    \tiny\boldmath
    \begin{align*}
        \frac{1}{2}p & \input{tex/q_numerator} p^T +  \\
        & \input{tex/q_numerator} p + q_{2} - 5 q_{4} - 1
    \end{align*}
    \endgroup

    and equivalently as,

    \[\frac{1}{2}pQp^T + cp + a\]

    where \(Q\) \(\in \R^{4\times4}\) is a square matrix defined by the
    transition probabilities of the opponent \(q_1, q_2, q_3, q_4\) as follows:

    \begin{equation*}
        \resizebox{0.7\linewidth}{!}{\arraycolsep=2.5pt%
        \boldmath\(
        Q = \input{tex/q_numerator}\)},
    \end{equation*}

    \(c\) \(\in \R^{4 \times 1}\) is similarly defined by:

    \begin{equation*}
        \resizebox{0.3\linewidth}{!}{\arraycolsep=2.5pt%
        \boldmath\(c = \input{tex/c_numerator}\),}
    \end{equation*}

    and \(a = \input{tex/numerator_constant}\).

    The same process is done for the denominator.
\end{proof}

\subsection{Theorem~\ref{memone_group_best_response}}
\begin{theorem}\label{memone_group_best_response}

The optimal behaviour of a memory-one strategy player
\(p^* \in \R_{[0, 1]} ^ 4\)
against a set of \(N\) opponents \(\{q^{(1)}, q^{(2)}, \dots, q^{(N)} \}\)
for \(q^{(i)} \in \R_{[0, 1]} ^ 4\) is given by:

\[p^* = \textnormal{argmax}\sum\limits_{i=1} ^ N  u_q(p), \ p \in S_q.\]

The set \(S_q\) is defined as all the possible combinations of:

{\scriptsize
\begin{equation}\label{eq:s_q_set}
    S_q =
    \left\{p \in \mathbb{R} ^ 4 \left|
        \begin{aligned}
            \bullet\quad p_j \in \{0, 1\} & \quad \text{and} \quad \frac{d}{dp_k}
            \sum\limits_{i=1} ^ N  u_q^{(i)}(p) = 0 \\
            & \quad \text{for all} \quad j \in J \quad \&  \quad k \in K  \quad \text{for all} \quad J, K \\
            & \quad \text{where} \quad J \cap K = \O \quad
            \text{and} \quad J \cup K = \{1, 2, 3, 4\}.\\
            \bullet\quad  p \in \{0, 1\} ^ 4
        \end{aligned}\right.
    \right\}.
\end{equation}
}

Note that there is no immediate way to find the zeros of
\(\frac{d}{dp} \sum\limits_{i=1} ^ N  u_q(p)\) where,

{\scriptsize
\begin{align}\label{eq:mo_tournament_derivative}
    \frac{d}{dp} \sum\limits_{i=1} ^ {N} {u_q}^{(i)} (p) & = \displaystyle\sum\limits_{i=1} ^ {N}
    \frac{\left(pQ^{(i)} + c^{(i)}\right) \left(\frac{1}{2} p\bar{Q}^{(i)} p^T + \bar{c}^{(i)} p + \bar{a}^ {(i)}\right)}
    {\left(\frac{1}{2} p\bar{Q}^{(i)} p^T + \bar{c}^{(i)} p + \bar{a}^ {(i)}\right)^ 2}
    - \frac{\left(p\bar{Q}^{(i)} + \bar{c}^{(i)}\right) \left(\frac{1}{2} pQ^{(i)} p^T + c^{(i)} p + a^ {(i)}\right)}
    {\left(\frac{1}{2} p\bar{Q}^{(i)} p^T + \bar{c}^{(i)} p + \bar{a}^ {(i)}\right)^ 2}
\end{align}
}

For \(\frac{d}{dp} \sum\limits_{i=1} ^ N  u_q(p)\) to equal zero then:

{\scriptsize
\begin{align}\label{eq:polynomials_roots}
    \displaystyle\sum\limits_{i=1} ^ {N}
    \left(pQ^{(i)} + c^{(i)}\right) \left(\frac{1}{2} p\bar{Q}^{(i)} p^T + \bar{c}^{(i)} p + \bar{a}^ {(i)}\right)
    - \left(p\bar{Q}^{(i)} + \bar{c}^{(i)}\right) \left(\frac{1}{2} pQ^{(i)} p^T + c^{(i)} p + a^ {(i)}\right)
    & = 0, \quad {while} \\
    \displaystyle\sum\limits_{i=1} ^ {N} \frac{1}{2} p\bar{Q}^{(i)} p^T + \bar{c}^{(i)} p + \bar{a}^ {(i)} & \neq 0.
\end{align}}

\end{theorem}

\begin{proof}
    The optimal behaviour of a memory-one strategy player
    \(p^* \in \R_{[0, 1]} ^ 4\)
    against a set of \(N\) opponents \(\{q^{(1)}, q^{(2)}, \dots, q^{(N)} \}\)
    for \(q^{(i)} \in \R_{[0, 1]} ^ 4\) is established by:

    \[p^* = \textnormal{argmax}\left(\sum\limits_{i=1} ^ N  u_q(p)\right), \ p \in S_q,\]

    where \(S_q\) is given by:
    {\scriptsize
    \begin{equation}\label{eq:s_q_set}
        S_q =
        \left\{p \in \mathbb{R} ^ 4 \left|
            \begin{aligned}
                \bullet\quad p_j \in \{0, 1\} & \quad \text{and} \quad \frac{d}{dp_k}
                \sum\limits_{i=1} ^ N  u_q^{(i)}(p) = 0 \\
                & \quad \text{for all} \quad j \in J \quad \&  \quad k \in K  \quad \text{for all} \quad J, K \\
                & \quad \text{where} \quad J \cap K = \O \quad
                \text{and} \quad J \cup K = \{1, 2, 3, 4\}.\\
                \bullet\quad  p \in \{0, 1\} ^ 4
            \end{aligned}\right.
        \right\}.
    \end{equation}}

    The optimisation problem of Eq.~\ref{eq:mo_tournament_optimisation}

    \begin{equation}\label{eq:mo_tournament_optimisation}
        \begin{aligned}
        \max_p: & \ \sum_{i=1} ^ {N} {u_q}^{(i)} (p)
        \\
        \text{such that}: & \ p \in \R_{[0, 1]}
        \end{aligned}
    \end{equation}

    can be written as:

    \begin{equation}\label{eq:mo_tournament_optimisation_standard}
        \begin{aligned}
        \max_p: & \ \sum_{i=1} ^ {N} {u_q}^{(i)} (p)
        \\
        \text{such that}: p_i & \leq 1 \text{ for } \in \{1, 2, 3, 4\} \\
        - p_i & \leq 0 \text{ for } \in \{1, 2, 3, 4\} \\
        \end{aligned}
    \end{equation}

    The optimisation problem has two inequality constraints and regarding the optimality
    this means that:

    \begin{itemize}
        \item either the optimum is away from the boundary of the optimization domain, and so the constraints plays no role;
        \item or the optimum is on the constraint boundary.
    \end{itemize}

    Thus, the following three cases must be considered:

    \textbf{Case 1:} The solution is on the boundary and any of the possible
    combinations for $p_i \in \{0, 1\}$ for $i \in \{1, 2, 3, 4\}$ are candidate
    optimal solutions.

    \textbf{Case 2:} The optimum is away from the boundary of the optimization domain
    and the interior solution $p^*$ necessarily satisfies the condition
    \(\frac{d}{dp} \sum\limits_{i=1} ^ N  u_q(p^*) = 0\).

    \textbf{Case 3:} The optimum is away from the boundary of the optimization domain
    but some constraints are equalities. The candidate solutions in this case
    are any combinations of $p_j \in \{0, 1\} \quad \text{and} \quad \frac{d}{dp_k}
    \sum\limits_{i=1} ^ N  u_q^{(i)}(p) = 0$
    forall $ j \in J \text{ \& } k \in K \text{ forall } J, K
    \text{ where } J \cap K = \O \text{ and } J \cup K = \{1, 2, 3, 4\}.$

    Combining cases 1-3 a set of candidate solution is constructed as:
    {\scriptsize
    \begin{equation*}
        S_q =
        \left\{p \in \mathbb{R} ^ 4 \left|
            \begin{aligned}
                \bullet\quad p_j \in \{0, 1\} & \quad \text{and} \quad \frac{d}{dp_k}
                \sum\limits_{i=1} ^ N  u_q^{(i)}(p) = 0
                \quad \text{for all} \quad j \in J \quad \&  \quad k \in K  \quad \text{for all} \quad J, K \\
                & \quad \text{where} \quad J \cap K = \O \quad
                \text{and} \quad J \cup K = \{1, 2, 3, 4\}.\\
                \bullet\quad  p \in \{0, 1\} ^ 4
            \end{aligned}\right.
        \right\}.
    \end{equation*}}

    This set is denoted as $S_q$ and the optimal solution to
    Eq.~\ref{eq:mo_tournament_optimisation} is the point from $S_q$ for which the
    utility is maximised.
\end{proof}

\end{document}


%
% CMPT 433: Embedded Systems - A Course Overview
% Section: Electronics
%
% Author: Jeffrey Leung
%

\section{Electronics}
	\label{sec:electronics}
\begin{easylist}

& \textbf{RS232:} Recommended Standard which is a specific bitrate with voltages -12V meaning 1 and 12V meaning 0

& Circuits are often damaged through water, short-circuiting, and ESD
	&& \textbf{Electrostatic Discharge (ESD):} Static shock given to a system

& \textbf{Voltage:} Pressure which causes electricity flow; the difference in pressure between two points in a circuit
	&& Symbol: $V$
	&& SI Unit: Volt (V)

& \textbf{Current:} Flow of electrons caused by a difference in pressure between two points in a circuit
	&& Symbol: $I$
	&& SI Unit: Amp (A)
		&&& \textbf{Microamp:} $1/1,000,000$ of an amp
		&&& \textbf{Kiloamp:} $1,000$ amps
		&&& \textbf{Megaamp:} $1,000,000$ amps
	&& Flows from higher voltage (+) to lower voltage (-)
	&& Electrons flow in the opposite direction

& \textbf{Resistance:} Resistance experienced by the current when flowing between two points
	&& Symbol: $R$
	&& SI Unit: Ohm ($\Omega$)
	&& Ohm's Law: $R = \frac{V}{I}$

\end{easylist}
\subsection{Circuits}
\begin{easylist}

& \textbf{Open circuit:} Uncompleted circuit which has infinite resistance and no current
& \textbf{Closed circuit:} Completed circuit
& \textbf{Short circuit:} Circuit which has no resistance and high current

& Components:
	&& Current is constant across all components
	&& Voltage is lost across each component to equal the total voltage
	&& \textbf{Sourcing:} Components which outputs voltage
	&& \textbf{Sinking:} Component which inputs voltage
	&& \textbf{Shorted:} Component which is directly connected to a power source or ground
	&& \textbf{Floating:} Component which is neither connected to a power source nor a ground

& Serial circuits: Calculate the total resistance, calculate current (constant across all components), and use it to find voltage drop across each component

& \textbf{Diode:} Component which only allows current in one direction
	&& \textbf{Light Emitting Diode:} Diode which creates illumination

& \textbf{Resistor:} Component which generates resistance and creates voltage drop
	&& \textbf{Current-limiting resistor:} Resistor with the purpose of reducing current
	&& \textbf{Pull-down resistor:} Large resistance between a component and ground to create a weak pull-down effect when a circuit is open
	&& \textbf{Pull-up resistor:} Large resistance between a component and the power source to create a weak pull-up effect when a circuit is open
	&& \textbf{Resistor divider:} Equal pull-up and pull-down resistors which divides the voltage in two

& \textbf{Pulse Code Modulation (PCM):} Format of an audio file which stores the amplitude at each sample point
	&& \textbf{Quantization:} Accuracy of a measurement
	&& \textbf{Bit depth:} Number of bits per quantization
	&& WAV file has a bit depth of 16
	&& \textbf{Superposition:} Combining multiple signals to create a single signal

\end{easylist}
\subsection{Software Interactions}
\begin{easylist}

& \textbf{Debouncing:} Process of removing fluctuation when switching states to read a stable value

& \textbf{General Purpose Input-Output (GPIO):} Pins which can be set to read a value, or output a a value (0 or 1)
& \textbf{Inter-Integrated Circuit (I\textsuperscript{2}C):} Direct communication value setting/reading protocol for hardware chips
& \textbf{Analog-to-Digital (A2D):} Method to read analog voltages to a digital value
& \textbf{Pulse-Width Modulation (PWM):} Method to generate analog voltage by toggling a digital value over time

\end{easylist}
\clearpage


\subsection{Partially linear models}


% Created 2021-09-21 Tue 11:20
% Intended LaTeX compiler: pdflatex
\documentclass[presentation,aspectratio=169, usenames, dvipsnames]{beamer}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage{khpreamble}
\usepackage{amssymb}
\usepgfplotslibrary{groupplots}
\newcommand*{\shift}{\operatorname{q}}
\definecolor{ppc}{rgb}{0.1,0.1,0.6}
\definecolor{iic}{rgb}{0.6,0.1,0.1}
\definecolor{ddc}{rgb}{0.1,0.6,0.1}
\usetheme{default}
\author{Kjartan Halvorsen}
\date{\today}
\title{Design of control systems}
\hypersetup{
 pdfauthor={Kjartan Halvorsen},
 pdftitle={Design of control systems},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.3 (Org mode 9.4.6)},
 pdflang={English}}
\begin{document}

\maketitle

\section{Course content}
\label{sec:orgd1eaf3e}

\begin{frame}[label={sec:org7b42b0b}]{Feedback control systems are ubiquitous}
\begin{center}
  \includegraphics[width=.6\linewidth]{../../figures/PnID-ex.png}
\end{center}
\end{frame}

\begin{frame}[label={sec:org2eee146}]{Feedback control systems}
The problem situation

\begin{center}
  \includegraphics[width=.34\linewidth]{../../figures/mars-rover-curiosity-vehicle-cosmos.jpg}
\end{center}
\end{frame}

\begin{frame}[label={sec:orga50fd22}]{Feedback control system}
\begin{columns}
\begin{column}{0.4\columnwidth}
\begin{center}
 \includegraphics[width=1.0\linewidth]{../../figures/curiosity-wheel.jpg}
\end{center}
\end{column}

\begin{column}{0.6\columnwidth}
\pause

\begin{center}
\includegraphics[width=\linewidth]{../../figures/electric-drive-block.png}
\end{center}
\end{column}
\end{columns}
\end{frame}

\section{Control systems specifications}
\label{sec:org381a6df}

\begin{frame}[label={sec:org3355b21}]{Performance requirements - time domain}
\begin{center}
  \includegraphics[width=.8\linewidth]{../../figures/step-response-specifications}
\end{center}
\end{frame}

\begin{frame}[label={sec:org4402dc5}]{Performance requirements - time domain}
\alert{Activity} Does the system satisfy the requirements?

\begin{columns}
\begin{column}{0.3\columnwidth}
\begin{center}
\begin{tabular}{ll}
Rise time & < 1.5s\\
Overshoot & < 18\%\\
\end{tabular}
\end{center}
\end{column}

\begin{column}{0.7\columnwidth}
\begin{center}
 \includegraphics[width=1.0\linewidth]{../../figures/second-order-response-example}
\end{center}
\end{column}
\end{columns}
\end{frame}
\begin{frame}[label={sec:org72943a6}]{Performance requirements - frequency domain}
\begin{center}
  \includegraphics[width=.8\linewidth]{../../figures/spec-bode-closed-loop-new}
\end{center}
\end{frame}

\begin{frame}[label={sec:orgdebb4b0}]{Performance requirements - frequency domain}
\begin{center}
  \includegraphics[width=1.0\linewidth]{../../figures/bode-closed-loop-example-responses}
\end{center}

\pause

\alert{Activity} What is the gain and phase shift at \(\omega = 2\) rad/s?
\end{frame}

\begin{frame}[label={sec:orgfefd3ae}]{Performance requirements - frequency domain}
\begin{center}
  \includegraphics[width=.8\linewidth]{../../figures/spec-bode-closed-loop-new}
\end{center}
\end{frame}

\begin{frame}[label={sec:org10c4d2c}]{Performance requirements - frequency domain}
\alert{Activity} Does the system satisfy the requirements?

\begin{columns}
\begin{column}{0.7\columnwidth}
\begin{center}
 \includegraphics[width=1.0\linewidth]{../../figures/bode-closed-loop-example}
\end{center}
\end{column}
\begin{column}{0.3\columnwidth}
\begin{center}
\begin{tabular}{ll}
Bandwidth & >3 rad/s\\
Resonance peak & <9dB\\
\end{tabular}
\end{center}
\end{column}
\end{columns}
\end{frame}

\section{Feedback, sensitivity and complementary sensitivity}
\label{sec:org2cd98ba}

\begin{frame}[label={sec:org205a6a1}]{Block diagram algebra}
\begin{center}
  \includegraphics[width=.6\linewidth]{../../figures/block-simple-feedback}
\end{center}

Transfer function from \(r(t)\) to \(y(t)\):
\[ \frac{Y(s)}{R(s)} = \frac{G(s)}{ 1+ G(s)}\]
\end{frame}

\begin{frame}[label={sec:org99fd74b}]{Block diagram algebra}
\alert{Activity} Pair the block-diagram with the correct closed-loop transfer function!

\begin{longtable}{cccc}
\textcolor{red}{A} & \textcolor{red}{B} & \textcolor{red}{C} & \textcolor{red}{D}\\
\includegraphics[width=3cm]{../../figures/block-simple-control-feedback} & \includegraphics[width=3cm]{../../figures/block-simple-control-feedback2} & \includegraphics[width=3cm]{../../figures/block-simple-control-feedback3} & \includegraphics[width=3cm]{../../figures/block-simple-control-feedback4}\\
\end{longtable}

\begin{longtable}{cccc}
\textcolor{blue!80!black}{I} & \textcolor{blue!80!black}{II} & \textcolor{blue!80!black}{III} & \textcolor{blue!80!black}{IV}\\
\(\frac{Y(s)}{R(s)}=\frac{G(s)F(s)}{1 + G(s)}\) & \(\quad \frac{Y(s)}{R(s)}=\frac{G(s)}{1 + G(s)F(s)}\quad\) & \(\frac{Y(s)}{R(s)}=\frac{1}{1 + G(s)F(s)}\) & \(\frac{Y(s)}{R(s)}=\frac{G(s)F(s)}{1 + G(s)F(s)}\)\\
\end{longtable}
\end{frame}
\end{document}


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[margin=1in]{geometry} % handle page geometry

% import packages and commands
\input{../header2.tex}

\newcommand{\Gr}{\opname{Gr}}

\title{Math 217 -- Geometry and Physics -- Lec11}
\author{UCLA, Fall 2014}
\date{\formatdate{27}{10}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\maketitle

\section{More on $L$-classes, $\hat{A}$-classes}

Last time we started talking about the Hirzebruch $L$-classes. For a manifold $M^{2n}$ we the $L$-class is given by
\begin{eqn}
L(TM) = \prod_{j=1}^n \frac{x_j}{\tanh x_j},
\end{eqn}
and we have
\begin{eqn}
\int_{M^{2n}} L(TM) = \opname{sgn} M.
\end{eqn}
For $M^4$ we have
\begin{align}
L(TM) &= 1 + \frac{1}{3} P_1 (TM), \\
\hat{A} &= 1 - \frac{1}{24} P_1(TM).
\end{align}
where the $\hat{A}$-class is given by
\begin{align}
\hat{A}(TM) &= \prod_{j=1}^n \frac{x_j / 2}{\sinh(x_j/2)}, \\
\opname{Ind} D &= \int_{M^{2n}} \hat{A}(TM) = \text{$\hat{A}$-genus},
\end{align}
where $D$ is the Dirac operator.

Given $M^{4k}$ a compact closed manifold,
\begin{eqn}
\begin{matrix}
H^{2k}(M, \R) & \times & H^{2k}(M, \R) & \rightarrow & \R \\
\alpha & & \beta & \mapsto & \int_M \alpha \wedge \beta
\end{matrix}
\end{eqn}
gives us a quadratic form. We have
\begin{eqn}
\opname{sgn} M = (\text{\# positive eigenvalues}) - (\text{\# negative eigenvalues}),
\end{eqn}
and
\begin{eqn}
\opname{sgn} M = \int_M L(TM) = \frac{1}{3} \int_M P_1(TM) = -8 \int_M \hat{A}(TM) = -8 \opname{Ind} D.
\end{eqn}
[$M$ a spin-manifold [?]]
\begin{definition}
A spin-manifold is an oriented manifold $M$ with structure group of $TM$ given by $SO(n)$ with $n \geq 2$. It turns out that $\pi_1 (SO(n)) = \Z_2$, so we have
\begin{eqn}
1 \rightarrow \Z_2 \rightarrow \opname{Spin}(n) \rightarrow SO(n) \rightarrow 1,
\end{eqn}
where $\opname{Spin}(n)$ is the universal cover of $SO(n)$. %break defn here?
Then the principle bundle $P_{SO(n)}$ of $SO(n)$ is a frame bundle of $TM$, and we know there exists a principle $\opname{Spin}(n)$ bundle
\begin{eqn}
P_{\opname{Spin}(n)} \overset{2:1}{\longrightarrow} P_{SO(n)} \rightarrow M.
\end{eqn}
which is where the name comes from.
\end{definition}
[missed some stuff here on what this means in terms of Stiefel-Whitney classes]

So if $M^4$ is a spin-manifold, we find that
\begin{eqn}
\opname{sgn} M = o(8) \text{ [divisible by 8?]} = -8 \underbrace{\opname{Ind} D}_\text{even}.
\end{eqn}

\begin{theorem}[Rochlin Theorem]
Given $M^4$ a compact smooth spin manifold, we have $\opname{sgn} M^4 = o(16)$.
\end{theorem}
\begin{proof}
The index or $\hat{A}$-genus is even for $8k+4$ dimensional manifold.
\end{proof}

\section{K-theory, Chern character}

Given manifold $M$ with vector bundles $E, F$, denote $[E], [F]$ as their equivalence classes. Then
\begin{eqn}
[E] = [F] \quad \iff \quad E \oplus \zeta_1 \cong F \oplus \zeta_2
\end{eqn}
where $\zeta_1, \zeta_2$ are trivial bundles. Note that it is not true that
\begin{eqn}
0 \rightarrow F \rightarrow G \rightarrow E \rightarrow 0 \quad \implies \quad G \cong E \oplus F.
\end{eqn}
Rather if we have
\begin{eqn}
[E] + [F] = [E \oplus F], \quad [E] [F] = [E \otimes F],
\end{eqn}
then we find
\begin{eqn}
[E] - [F] = [G] \quad \implies \quad [E] = [G] + [F].
\end{eqn}

\begin{definition}
We define $K(M)$ as the set of equivalence classes of vector bundles on $M$ with the operations.
\end{definition}
\begin{remark}
$K(M)$ is a \textit{generalized} Cohomology theory.
\end{remark}

Now we have a map
\begin{eqn}
\begin{matrix}
\opname{ch} & : & K(M) & \rightarrow & H^*(M, \Q) \\
& & [E] & \mapsto & \opname{ch} E = \sum_{j=1}^r e^{x_i}.
\end{matrix}
\end{eqn}
Let $\set{x_i}$ be Chern roots of $E$, i.e. $\tr \exp \frac{i}{2\pi} \Omega$. If $E$ is a real vector bundle then
\begin{eqn}
\opname{ch} E = \opname{ch} (E \otimes_\R \C).
\end{eqn}
Given two vector bundles $E, E'$, we have
\begin{enumerate}
\item $\opname{ch} (E \oplus E') = \opname{ch} E + \opname{ch} E'$,
\item $\opname{ch} (E \otimes E') = \opname{ch} E \, \opname{ch} E'$,
\item $\opname{ch} (E - E') = \opname{ch} E - \opname{ch} E'$.
\end{enumerate}

We have
\begin{eqn}
K(M) \overset{f_!}{\longrightarrow} K(N) \overset{\opname{ch}}{\longrightarrow} H^*(N)
\end{eqn}
and
\begin{eqn}
K(M) \overset{\opname{ch}}{\longrightarrow} H^*(M) \overset{f_*}{\longrightarrow} H^*(N).
\end{eqn}
In general they do not commute. But if we multiply the Chern characters $\opname{ch}$ by $\hat{A}(TM)$ then they do commute.

For $M$ a complex manifold we have $T$-odd class
\begin{eqn}
T \dif(TM) = \prod_{j=1}^n \frac{x_j}{1 - e^{-x_j}}.
\end{eqn}
This gives us [Top transfer, Riemann of Cohomology theory, Roch, something for holomorphic map between projective manifolds [???]]. Then
\begin{eqn}
K(M \times S^{2N}) \cong K(M).
\end{eqn}

\end{document}


\chapter{Evaluation} \label{ch:evaluation} We have used a machine learning toolkit named as GRT to recognize hand gestures using skeletal points tracking algorithm and Adaptive Naive Bayes classifier (ANBC) for classification and prediction.

The classifier is based on a statistical model of x,y,z coordinate positions of static hand gestures and provides a likelihood measure for recognized gesture. Furthermore, the gesture recognition pipeline uses two post processing modules such as Class Label Filter and Class Label Change Filter to exclude lower frequent spikes in the prediction results and trigger an output only when there is a change in prediction.

In this chapter, we present the experiments carried out to evaluate and validate our system to recognize hand gestures using skeletal points. The goal is to demonstrate the effectiveness of the classifier and to evaluate its potential for real time input at 30 fps. In the classification phase, input samples are normalized using Min-Max Scaling and Null Rejection is enabled to detect non-gestures. Therefore, the evaluation consists of computing the prediction accuracy for various null rejection coefficient and compare it with other supervised learning classifiers such as Support Vector Machine (SVM) and Minimum Distance (MinDist).

\section{Mean and Standard Deviation} ANBC is a supervised learning algorithm that can be used to classify any type of N-dimensional signal. It fundamentally works by fitting an N-dimensional Gaussian distribution to each class when it is trained.

During the training phase, first all the input samples are normalized using Min-Max Scaling with the range from 0 to 1 and then GRT computes mean $\mu$ and standard deviation $\sigma$ to create a model for each class. During the prediction phase, it basically computes the maximum a posterior probability of an input vector belonging to any of the trained class. Figure \ref{fg:ev:mean} shows the mean positions of left and right hand for every gesture. Table \ref{tb:ev:mean} and \ref{tb:ev:sd} show mean and standard deviations of the labeled training data of all the five classes.

\section{Classification and Prediction} Our gesture recognition pipeline is trained with 11918 input samples of 6 dimensional vector for 5 classes. Class 1,2,3,4,5 are mapped to Walk, Turn Right, Turn Left, Move Right, Move Left gestures respectively. We have carried out experiments to evaluate the classification, prediction and post processing efficiency of our system. Graph \ref{ev:test:prediction} show change is positions of left and hand in Cartesian coordinates while test data is recorded and corresponding prediction for every input sample.

Test data consists of 1400 samples which are recorded under supervised arrangement. Input vectors that is not containing left and right hand are removed from the test data. Furthermore, input vectors which are recorded when the hand is at the field of view of the camera are also excluded. A program is implemented with GRT to read all the test data and execute prediction on every input sample and then results are stored to CSV file. Finally results are plotted using MATLAB.

Prediction results shown in the graph \ref{ev:test:prediction} frequently falls down to Class Label 0 that is reserved for non-gestures. Non-gestures are detected with the help of Null Rejection thresholds. Therefore, this prediction is based on normalized classification data for Adaptive Naive Bayes classifier with Null rejection coefficient of 1.0.

\section{Prediction Accuracy Vs Null Rejection Accuracy} \label{sec:ev:accuracy} Classifiers of GRT offers various customization that could produce different results for the same test data. Accuracy of a gesture recognition system does not depend only on th precise predictions of trained gestures, but also differentiating them from unintended hand gestures. Graph \ref{ev:accuracy:anbc} shows the prediction and null rejection accuracies of Adaptive Naive Bayes Classifier (ANBC). Graph shows the accuracies of 5 trained gestures and a non-gesture. Non-Gesture data is recorded with both the hands are pointing downwards. GRT allows us to set null rejection coefficient for the classifier and therefore, graph contains the dataset of accuracies with null rejection coefficients range from 0 to 10.

Graph \ref{ev:accuracy:anbc} shows that increase in null rejection coefficient causes an increase in the accuracy of trained gestures, however, causes a decrease in the accuracy of non-gesture. Therefore, it is optimal to use a null rejection coefficient of 2.0 with ANBC.

Graph \ref{ev:accuracy:svm} shows prediction results of Support Vector Machine classifier with RBF Kernel. It is apparent that accuracy of trained gestures are consistently above 95 percent. However, null rejection accuracy is constantly lesser than 10 percent, thus, denoting that SVM is not useful in our case, because the robot should not execute any unintended commands.

Graph \ref{ev:accuracy:mindist} shows some interesting results of Minimum Distance classifier with 4 clusters. Graph shows that prediction results are unpredictable as there is increase in null rejection coefficient. However, with coefficient of around 6.3, MinDist shows compromising accuracy above 90 percent for trained gesture and non-gesture. This shows that MinDist is a better alternative to ANBC.

\input{chapter/tables/ev-mean-sd}

\clearpage
\input{chapter/figures/ev-all-ges-mean}
\input{chapter/figures/ev-test-all}
\input{chapter/figures/ev-accuracy}


\documentclass[13pt,onlymath]{beamer}
\usefonttheme{serif}
\usepackage{graphicx,amsmath,amssymb,tikz,psfrag,epstopdf,fancyvrb}
\usepackage[lighttt]{lmodern}
%\usepackage{graphicx,psfrag}

\input defs.tex

%% formatting

\mode<presentation>
{
\usetheme{default}
}
\setbeamertemplate{navigation symbols}{}
\usecolortheme[rgb={0.13,0.28,0.59}]{structure}
\setbeamertemplate{itemize subitem}{--}
\setbeamertemplate{frametitle} {
    \begin{center}
      {\large\bf \insertframetitle}
    \end{center}
}

\newcommand\footlineon{
  \setbeamertemplate{footline} {
    \begin{beamercolorbox}[ht=2.5ex,dp=1.125ex,leftskip=.8cm,rightskip=.6cm]{structure}
      \footnotesize \insertsection
      \hfill
      {\insertframenumber}
    \end{beamercolorbox}
    \vskip 0.45cm
  }
}
\footlineon

\AtBeginSection[]
{
    \begin{frame}<beamer>
        \frametitle{Outline}
        \tableofcontents[currentsection,currentsubsection]
    \end{frame}
}

%% begin presentation

\title{\large \bfseries Mathematics}

\author{Jaehyun Park\\[3ex]
CS 97SI\\
Stanford University}

\date{\today}

\begin{document}

\frame{
\thispagestyle{empty}
\titlepage
}

\section{Algebra}

\begin{frame}{Sum of Powers}
\BEAS
\sum_{k=1}^n k^2 &=& \frac{1}{6}n(n+1)(2n+1) \\
\sum k^3 &=& \left(\sum k\right)^2 = \left(\frac{1}{2}n(n+1)\right)^2
\EEAS
\BIT
\item Pretty useful in many random situations
\item Memorize above!
\EIT
\end{frame}

\begin{frame}{Fast Exponentiation}
\BIT
\item Recursive computation of $a^n$:
\[
a^n = \begin{cases}
1 & n = 0\\
a & n = 1\\
(a^{n/2})^2 & n \mbox{ is even}\\
a (a^{(n-1)/2})^2 & n \mbox{ is odd}
\end{cases}
\]
\EIT
\end{frame}

\begin{frame}[fragile]{Implementation (recursive)}
\begin{Verbatim}[xleftmargin=25pt]
double pow(double a, int n) {
    if(n == 0) return 1;
    if(n == 1) return a;
    double t = pow(a, n/2);
    return t * t * pow(a, n%2);
}
\end{Verbatim}
\BIT
\item Running time: $O(\log n)$
\EIT
\end{frame}

\begin{frame}[fragile]{Implementation (non-recursive)}
\begin{Verbatim}[xleftmargin=25pt]
double pow(double a, int n) {
    double ret = 1;
    while(n) {
        if(n%2 == 1) ret *= a;
        a *= a; n /= 2;
    }
    return ret;
}
\end{Verbatim}
\BIT
\item You should understand how it works
\EIT
\end{frame}

\begin{frame}{Linear Algebra}
\BIT
\item Solve a system of linear equations
\item Invert a matrix
\item Find the rank of a matrix
\item Compute the determinant of a matrix
\item All of the above can be done with Gaussian elimination
\EIT
\end{frame}

\section{Number Theory}

\begin{frame}{Greatest Common Divisor (GCD)}
\BIT
\item $\gcd(a, b)$: greatest integer divides both $a$ and $b$
\item Used very frequently in number theoretical problems

\item Some facts:
\BIT
\item $\gcd(a, b) = \gcd(a, b-a)$
\item $\gcd(a, 0) = a$
\item $\gcd(a, b)$ is the smallest positive number in $\{ax+by \,|\, x, y \in \integers\}$
\EIT
\EIT
\end{frame}

\begin{frame}{Euclidean Algorithm}
\BIT
\item Repeated use of $\gcd(a, b) = \gcd(a, b-a)$
\item Example:
\BEAS
\gcd(1989, 867) &=& \gcd(1989-2\times 867, 867) \\
&=& \gcd(255, 867) \\
&=& \gcd(255, 867-3\times 255) \\
&=& \gcd(255, 102) \\
&=& \gcd(255-2\times 102, 102) \\
&=& \gcd(51, 102) \\
&=& \gcd(51, 102-2\times 51) \\
&=& \gcd(51, 0) \\
&=& 51
\EEAS
\EIT
\end{frame}

\begin{frame}[fragile]{Implementation}
\begin{Verbatim}[xleftmargin=25pt]
int gcd(int a, int b) {
    while(b){int r = a % b; a = b; b = r;}
    return a;
}
\end{Verbatim}
\BIT
\item Running time: $O(\log(a+b))$
\item Be careful: \verb,a % b, follows the sign of \verb,a,
\BIT
\item \verb,5 % 3 == 2,
\item \verb,-5 % 3 == -2,
\EIT
\EIT
\end{frame}

\begin{frame}{Congruence \& Modulo Operation}
\BIT
\item $x \equiv y \pmod{n}$ means $x$ and $y$ have the same remainder when divided by $n$
\item Multiplicative inverse
\BIT
\item $x^{-1}$ is the inverse of $x$ modulo $n$ if $x x^{-1} \equiv 1 \pmod{n}$
\item $5^{-1} \equiv 3 \pmod{7}$ because $5 \cdot 3 \equiv 15 \equiv 1 \pmod{7}$
\item May not exist (\eg, inverse of $2$ mod $4$)
\item Exists if and only if $\gcd(x, n) = 1$
\EIT
\EIT
\end{frame}

\begin{frame}{Multiplicative Inverse}
\BIT
\item All intermediate numbers computed by Euclidean algorithm are integer combinations of $a$ and $b$
\BIT
\item Therefore, $\gcd(a, b) = ax+by$ for some integers $x, y$
\item If $\gcd(a, n) = 1$, then $ax + ny = 1$ for some $x, y$
\item Taking modulo $n$ gives $ax \equiv 1 \pmod{n}$
\EIT
\item We will be done if we can find such $x$ and $y$
\EIT
\end{frame}

\begin{frame}{Extended Euclidean Algorithm}
\BIT
\item Main idea: keep the original algorithm, but write all intermediate numbers as integer combinations of $a$ and $b$
\item Exercise: implementation!
\EIT
\end{frame}

\begin{frame}{Chinese Remainder Theorem}
\BIT
\item Given $a,b,m,n$ with $\gcd(m, n) = 1$
\item Find $x$ with $x\equiv a \pmod{m}$ and $x \equiv b \pmod{n}$

\item Solution:
\BIT
\item Let $n^{-1}$ be the inverse of $n$ modulo $m$
\item Let $m^{-1}$ be the inverse of $m$ modulo $n$
\item Set $x = a n n^{-1} + b m m^{-1}$ (check this yourself)
\EIT
\item Extension: solving for more simultaneous equations
\EIT
\end{frame}

\section{Combinatorics}

\begin{frame}{Binomial Coefficients}
\BIT
\item $\dbinom{n}{k}$ is the number of ways to choose $k$ objects out of $n$ distinguishable objects
\item same as the coefficient of $x^k y^{n-k}$ in the expansion of $(x+y)^n$
\BIT
\item Hence the name ``binomial coefficients''
\EIT
\item Appears everywhere in combinatorics
\EIT
\end{frame}

\begin{frame}{Computing Binomial Coefficients}
\BIT
\item Solution 1: Compute using the following formula:
\[
\binom{n}{k} = \frac{n(n-1) \cdots (n-k+1)}{k!}
\]
\item Solution 2: Use Pascal's triangle

\item Can use either if both $n$ and $k$ are small
\item Use Solution 1 carefully if $n$ is big, but $k$ or $n-k$ is small
\EIT
\end{frame}

\begin{frame}{Fibonacci Sequence}
\BIT
\item Definition:
\BIT
\item $F_0 = 0$, $F_1 = 1$
\item $F_n = F_{n-1} + F_{n-2}$, where $n \ge 2$
\EIT
\item Appears in many different contexts
\EIT
\end{frame}

\begin{frame}{Closed Form}
\BIT
\item $F_n = (1/\sqrt{5})(\varphi^n - \overline{\varphi}^n)$
\BIT
\item $\varphi = (1+\sqrt{5})/2$
\item $\overline{\varphi} = (1-\sqrt{5})/2$
\EIT
\item Bad because $\varphi$ and $\sqrt{5}$ are irrational
\item Cannot compute the exact value of $F_n$ for large $n$

\item There is a more stable way to compute $F_n$
\BIT
\item ... and any other recurrence of a similar form
\EIT
\EIT
\end{frame}

\begin{frame}{Better ``Closed'' Form}
\[
\left[\begin{array}{c}F_{n+1} \\ F_n \end{array} \right] =
\left[\begin{array}{cc}1&1\\1&0 \end{array} \right] \left[\begin{array}{c}F_n \\ F_{n-1} \end{array} \right] =
\left[\begin{array}{cc}1&1\\1&0 \end{array} \right]^n \left[\begin{array}{c}F_1 \\ F_0 \end{array} \right]
\]
\BIT
\item Use fast exponentiation to compute the matrix power
\item Can be extended to support any linear recurrence with constant coefficients
\EIT
\end{frame}

\section{Geometry}

\begin{frame}{Geometry}
\BIT
\item In theory: not that hard
\item In programming contests: more difficult than it looks
\item Will cover basic stuff today
\BIT
\item Computational geometry in week 9
\EIT
\EIT
\end{frame}

\begin{frame}[fragile]{When Solving Geometry Problems}
\BIT
\item Precision, precision, precision!
\BIT
\item If possible, don't use floating-point numbers
\item If you have to, always use \verb,double, and never use \verb,float,
\item Avoid division whenever possible
\item Introduce small constant $\epsilon$ in (in)equality tests
\BIT
\item e.g., Instead of \verb,if(x == 0),, write \verb,if(abs(x) < EPS),
\EIT
\EIT
\item No hacks!
\BIT
\item In most cases, randomization, probabilistic methods, small perturbations won't help
\EIT
\EIT
\end{frame}

\begin{frame}{2D Vector Operations}
\BIT
\item Have a vector $(x, y)$
\item Norm (distance from the origin): $\sqrt{x^2+y^2}$
\item Counterclockwise rotation by $\theta$:
\[
\left[ \begin{array}{cc} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right]
\left[ \begin{array}{c} x \\ y \end{array}\right]
\]
\BIT
\item Make sure to use correct units (degrees, radians)
\EIT
\item Normal vectors: $(y, -x)$ and $(-y, x)$

\item Memorize all of them!
\EIT
\end{frame}

\begin{frame}{Line-Line Intersection}
\BIT
\item Have two lines: $ax+by+c=0$ and $dx+ey+f=0$
\item Write in matrix form:
\[
\left[ \begin{array}{cc} a & b \\ d & e \end{array}\right]
\left[ \begin{array}{c} x \\ y \end{array}\right]
=
-\left[ \begin{array}{c} c \\ f \end{array}\right]
\]
\item Left-multiply by matrix inverse
\[
\left[ \begin{array}{cc} a & b \\ d & e \end{array}\right]^{-1} =
\frac{1}{ae-bd}\left[ \begin{array}{cc} e & -b \\ -d & a \end{array}\right]
\]
\BIT
\item Memorize this!
\EIT
\item Edge case: $ae=bd$
\BIT
\item The lines coincide or are parallel
\EIT
\EIT
\end{frame}

\begin{frame}{Circumcircle of a Triangle}
\BIT
\item Have three points $A, B, C$
\item Want to compute $P$ that is equidistance from $A, B, C$

\item Don't try to solve the system of quadratic equations!
\item Instead, do the following:
\BIT
\item Find the (equations of the) bisectors of $AB$ and $BC$
\item Compute their intersection
\EIT
\EIT
\end{frame}

\begin{frame}{Area of a Triangle}
\BIT
\item Have three points $A, B, C$
\item Want to compute the area $S$ of triangle $ABC$
\item Use cross product: $2S = |(B-A) \times (C-A)|$
\item Cross product:
\[
(x_1, y_1) \times (x_2, y_2) = \left| \begin{array}{cc} x_1 & x_2 \\ y_1 & y_2 \end{array}\right| = x_1 y_2 - x_2 y_1
\]
\BIT
\item Very important in computational geometry. Memorize!
\EIT
\EIT
\end{frame}

\begin{frame}{Area of a Simple Polygon}
\BIT
\item Given vertices $P_1, P_2, \ldots, P_n$ of polygon $P$
\item Want to compute the area $S$ of $P$
\item If $P$ is convex, we can decompose $P$ into triangles:
\[
2S = \left| \sum_{i=2}^{n-1}(P_{i+1} - P_1) \times (P_i - P_1) \right|
\]
\item It turns out that the formula above works for non-convex polygons too
\BIT
\item Area is the absolute value of the sum of ``signed area''
\EIT
\item Alternative formula (with $x_{n+1} = x_1, y_{n+1} = y_1$):
\[
2S = \left| \sum_{i=1}^n (x_i y_{i+1} - x_{i+1} y_i) \right|
\]
\EIT
\end{frame}

\begin{frame}{Conclusion}
\BIT
\item No need to look for one-line closed form solutions
\item Knowing ``how to compute'' (algorithms) is good enough

\item Have fun with the exercise problems
\BIT
\item ... and come to the practice contest if you can!
\EIT
\EIT
\end{frame}

\end{document}


\subsection{Spectral theorem for finite-dimensional vector spaces}


%
% CMPT 379: Principles of Compiler Design - A Course Overview
% Section: Lexical Analysis (Scanning)
%
% Author: Jeffrey Leung
%

\section{Lexical Analysis (Scanning)}
	\label{sec:lexical-analysis}
\begin{easylist}

& \textbf{Lexical analysis (scanning):} Transforming an input program string into tokens
	&& Does not validate, simply transforms
	&& Challenges:
		&&& Prove that the implementation captures all tokens specified by the language definition
		&&& Prove correctness of transformations

& \textbf{Token:} Symbol which represents a specific composable section of code
	&& E.g. Open bracket, number
	&& Can be denoted as \lstinline{T_IDENT}, \lstinline{T_LPAREN}, etc.
	&& \textbf{Lexeme/token attribute:} Value of a token
		&&& Not all tokens have values
		&&& E.g. For \lstinline{T_INTCONSTANT}, a possible value is 1
	&& The same character should only represent a single token (e.g. \lstinline{-} should be represented by the same token whether it is a unary or binary operator)

& \textbf{Loop and switch scanner:} Lexical analyzer which loops over each character sequentially and categorizes it based on context

\end{easylist}
\subsection{Regular Expressions}
	\label{subsec:reg-ex}
\begin{easylist}

& \textbf{Formal language:} A valid set of strings from a given alphabet of symbols
	&& \textbf{Symbol:} Single distinct character
		&&& \textbf{Alphabet:} The finite set of symbols
			&&&& Denoted by $\sum$ or $\{\textrm{symbol}\}$
	&& \textbf{String:} Sequence of symbols
		&&& \textbf{Empty string:} $\varepsilon$
		&&& Set of all strings: $\sum *$

& \textbf{Formal grammar:} Concise description of a formal language

& \textbf{Regular language:} A formal language which can be expressed through specific operations (i.e. any regular expression)
	&& For each regular language, there is an equivalent finite-state automaton
	&& The set of all regular languages includes:
		&&& The empty set
		&&& $\{a\}$ for all $a$ in $\sum \varepsilon$
		&&& For $L_1, L_2, L$ which are regular languages:
			&&&& Concatenation of $L_1 \cdot L_2$ where $\{xy | x \in L_1 \textrm{ and } y \in L_2\}$
			&&&& Union of $L_1 \cup L_2$
			&&&& Kleene closure as $L^* = \bigcup_{i=0}^{\infty} L^i$
		&&& No other regular languages

& \textbf{Regular expression:} Concise description of a regular language
	&& E.g. The set of all strings over the alphabet $\{a, b\}$ which end in $abb$ is expressed by $(a|b)*abb$
	&& Used to define tokens
	&& Core operators:
		&&& \textbf{Alternation:} One of several options
			&&&& E.g. $a|b = a \textrm{ or } b$ is a regular expression where $a, b$ are regular expressions
		&&& \textbf{Concatenation:} Combination of multiple regular expressions
			&&&& E.g. $ab$, a concatenation of $a$ and $b$, is a regular expression where $a, b$ are regular expressions
		&&& \textbf{Repetition:} An arbitrary amount of a regular expression
			&&&& E.g. $a*$, any number of $a$ regular expressions, is a regular expression
	&& Used in lexical analysis to match the largest valid string
	&& Can be expressed as a recursive tree structure or as a finite state automaton

\end{easylist}
\subsection{Finite State Automata}
\label{subsec:finite-state-automata}
\begin{easylist}

& \textbf{Finite State Automaton:} System consisting of a finite set of states and a transition function between states
	&& Notations:
		&&& Alphabet of symbols: $\Sigma$
		&&& Finite set of states: $S$
		&&& Start state: Outlined
		&&& Final/accepting state: Double-outlined
		&&& Transition function: $\sigma: S \times \Sigma = S$
	&& Equivalent to a unique regular language
  	&& Transition of $\epsilon$ refers to no input from the string corresponding to a transition in state
		&&&& Non-deterministic

& \textbf{Deterministic Finite Automata (DFA):} Finite State Automata where no state has more than one transition per input, and no $\epsilon$ moves exist
	&& For any given string, only one path exists from the start state to the final state
	&& Runtime complexity: $O(r^s)$
& \textbf{Nondeterministic Finite Automata (NFA):} Finite State Automata where at least one state has more than one transition per input, or an $\epsilon$ move exists
	&& For any given string, multiple paths may exist from the state state to the final state
	&& Usually larger than DFA
	&& Runtime complexity: $O(2^r)$ where $r$ is the initial cost of creating the automaton
	&& \textbf{Subset Construction:} Process to convert an NFA into a DFA
		&&& From the set of start states, given any input in the alphabet, output the set of potential resulting states. Repeat until no new sets of states are generated. Use each of the unique potential sets of states as a single DFA state.
		&&& Runtime complexity: Can convert regex of size $r$ to a $2^r$ state DFA

\end{easylist}
\clearpage


\chapter{Mathematical Documentation}
\label{chapter:mdoc}

The \faust compiler provides a mechanism to produce a self-describing documentation of the mathematical semantic of a \faust program, essentially as a pdf file. The corresponding options are \lstinline!-mdoc! (short) or \lstinline!--mathdoc! (long).

\section{Goals of the mathdoc}
\label{sec:goals-of-mdoc}

There are three main goals, or uses, of this mathematical documentation:
\begin{enumerate}
\item to preserve signal processors, independently from any computer language but only under a mathematical form;
\item to bring some help for debugging tasks, by showing the formulas as they are really computed after the compilation stage;
\item to give a new teaching support, as a bridge between code and formulas for signal processing.
\end{enumerate}

\section{Installation requirements}
\label{sec:inst-requ}

\begin{itemize}
\item \lstinline!faust!, of course!
\item \lstinline!svg2pdf! (from the Cairo 2D graphics library), to convert block-diagrams, as \latex doesn't eat \svg directly yet...
\item \lstinline!breqn!, a \latex package to handle automatic breaking of long equations,
\item \lstinline!pdflatex!, to compile the \latex output file.
\end{itemize}

\section{Generating the mathdoc}
\label{sec:generating-mdoc}

The easiest way to generate the complete mathematical documentation is to call the \lstinline!faust2mathdoc! script on a \faust file, as the \lstinline!-mdoc! option leave the documentation production unfinished. For example:
\begin{lstlisting}
faust2mathdoc noise.dsp
\end{lstlisting}

\subsection{Invoking the -mdoc option}
\label{sec:invoking-mdoc}

Calling directly \lstinline!faust -mdoc! does only the first part of the work, generating:
\begin{itemize}
\item a top-level directory, suffixed with "\texttt{-mdoc}",
\item 5 subdirectories (\lstinline!cpp/!, \lstinline!pdf/!, \lstinline!src/!, \lstinline!svg/!, \lstinline!tex/!),
\item a \latex file containing the formulas,
\item \svg files for block-diagrams.
\end{itemize}

At this stage:
\begin{itemize}
\item \lstinline!cpp/! remains empty,
\item \lstinline!pdf/! remains empty,
\item \lstinline!src/! contains all \faust sources used (even libraries),
\item \lstinline!svg/! contains \svg block-diagram files,
\item \lstinline!tex/! contains the generated \latex file.
\end{itemize}

\subsection{Invoking faust2mathdoc}
\label{sec:invok-faust2m}

The \lstinline!faust2mathdoc! script calls \lstinline!faust --mathdoc! first, then it finishes the work:
\begin{itemize}
\item moving the output C++ file into \lstinline!cpp/!,
\item converting all \svg files into pdf files (you must have \lstinline!svg2pdf! installed, from the Cairo 2D graphics library),
\item launching \lstinline!pdflatex! on the \latex file (you must have both \lstinline!pdflatex! and the \lstinline!breqn! package installed),
\item moving the resulting pdf file into \lstinline!pdf/!.
\end{itemize}

\subsection{Online examples}
\label{sec:mdoc-examples}

To get an idea of the results of this mathematical documentation, which captures the mathematical semantic of \faust programs, you can look at two pdf files online:
\begin{itemize}
\item \myurl{http://faust.grame.fr/pdf/karplus.pdf} (automatic documentation),
\item \myurl{http://faust.grame.fr/pdf/noise.pdf} (manual documentation).
\end{itemize}

You can also generate all \emph{mdoc} pdfs at once, simply invoking the \lstinline!make mathdoc! command inside the \lstinline!examples/! directory:
\begin{itemize}
\item for each \lstinline!%.dsp! file, a complete \lstinline!%-mdoc! directory will be generated,
\item a single \lstinline!allmathpdfs/! directory will gather all the generated pdf files.
\end{itemize}

\section{Automatic documentation}
\label{sec:auto-docum}

By default, when no \lstinline!<mdoc>! tag can be found in the input \faust file, the \lstinline!-mdoc! option automatically generates a \latex file with four sections:
\begin{enumerate}
\item ''\textbf{Equations of process}'', gathering all formulas needed for \lstinline!process!,
\item ''\textbf{Block-diagram schema of process}'', showing the top-level block-diagram of \lstinline!process!,
\item ''\textbf{Notice of this documentation}'', summing up generation and conventions information,
\item ''\textbf{Complete listing of the input code}'', listing all needed input files (including libraries).
\end{enumerate}

\section{Manual documentation}
\label{sec:manual-mdoc}

You can specify yourself the documentation instead of using the automatic mode, with five xml-like tags. That permits you to modify the presentation and to add your own comments, not only on \lstinline!process!, but also about any expression you'd like to. Note that as soon as you declare an \lstinline!<mdoc>! tag inside your \faust file, the default structure of the automatic mode is ignored, and all the \latex stuff becomes up to you!

\subsection{Six tags}
\label{sec:doc-tags}

Here are the six specific tags:
\begin{itemize}
\item \lstinline!<mdoc></mdoc>!, to open a documentation field in the \faust code,
  \begin{itemize}
  \item \lstinline!<equation></equation>!, to get equations of a \faust expression,
  \item \lstinline!<diagram></diagram>!, to get the top-level block-diagram of a \faust expression,
  \item \lstinline!<metadata></metadata>!, to reference \faust metadatas (cf. declarations), calling the corresponding keyword,
  \item \lstinline!<notice />!, to insert the "adaptive'' notice all formulas actually printed,
  \item \lstinline!<listing [attributes] />!, to insert the listing of \faust files called.
  \end{itemize}
\end{itemize}

The \lstinline!<listing />! tag can have up to three boolean attributes (set to \lstinline!"true"! by default):
\begin{itemize}
\item \lstinline'mdoctags' for \lstinline'<mdoc>' tags;
\item \lstinline'dependencies' for other files dependencies;
\item \lstinline'distributed' for the distribution of interleaved \faust code between \lstinline'<mdoc>' sections.
\end{itemize}

\subsection{The mdoc top-level tags}
\label{sec:mdoc-tag}

The \lstinline!<mdoc></mdoc>! tags are the top-level delimiters for \faust mathematical documentation sections. This means that the four other documentation tags can't be used outside these pairs (see section \ref{sec:documentation}).

In addition of the four inner tags, \lstinline!<mdoc></mdoc>! tags accept free \latex text, including its standard macros (like \lstinline!\section!, \lstinline!\emph!, etc.). This allows to manage the presentation of resulting tex file directly from within the input \faust file.

The complete list of the \latex packages included by \faust can be found in the file \lstinline!architecture/latexheader.tex!.

\subsection{An example of manual mathdoc}
\label{sec:ex-mathdoc}

\footnotesize
\begin{lstlisting}
<mdoc>
\title{<metadata>name</metadata>}
\author{<metadata>author</metadata>}
\date{\today}
\maketitle

\begin{tabular}{ll}
	\hline
	\textbf{name}		& <metadata>name</metadata> \\
	\textbf{version} 	& <metadata>version</metadata> \\
	\textbf{author} 	& <metadata>author</metadata> \\
	\textbf{license} 	& <metadata>license</metadata> \\
	\textbf{copyright} 	& <metadata>copyright</metadata> \\
	\hline
\end{tabular}
\bigskip
</mdoc>
//-----------------------------------------------------------------
// Noise generator and demo file for the Faust math documentation
//-----------------------------------------------------------------

declare name 		"Noise";
declare version 	"1.1";
declare author 		"Grame";
declare author 		"Yghe";
declare license 	"BSD";
declare copyright 	"(c)GRAME 2009";

<mdoc>
\section{Presentation of the "noise.dsp" Faust program}
This program describes a white noise generator with an interactive volume, using a random function.

\subsection{The random function}
</mdoc>

random  = +(12345)~*(1103515245);

<mdoc>
The \texttt{random} function describes a generator of random numbers, which equation follows. You should notice hereby the use of an integer arithmetic on 32 bits, relying on integer wrapping for big numbers.
<equation>random</equation>

\subsection{The noise function}
</mdoc>

noise   = random/2147483647.0;

<mdoc>
The white noise then corresponds to:
<equation>noise</equation>

\subsection{Just add a user interface element to play volume!}
</mdoc>

process = noise * vslider("Volume[style:knob]", 0, 0, 1, 0.1);

<mdoc>
Endly, the sound level of this program is controlled by a user slider, which gives the following equation:
<equation>process</equation>

\section{Block-diagram schema of process}
This process is illustrated on figure 1.
<diagram>process</diagram>

\section{Notice of this documentation}
You might be careful of certain information and naming conventions used in this documentation:
<notice />

\section{Listing of the input code}
The following listing shows the input Faust code, parsed to compile this mathematical documentation.
<listing mdoctags="false" dependencies="false" distributed="true" />
</mdoc>
\end{lstlisting}
\normalsize

The following page which gathers the four resulting pages of \lstinline!noise.pdf! in small size. might give you an idea of the produced documentation.

\subsection{The -stripmdoc option}
\label{sec:striping-option}

As you can see on the resulting file \lstinline!noisemetadata.pdf! on its pages 3 and 4, the listing of the input code (section\,4) contains all the mathdoc text (here colored in grey). As it may be useless in certain cases (see Goals, section \ref{sec:goals-of-mdoc}), we provide an option to strip mathdoc contents directly at compilation stage: \lstinline!-stripmdoc! (short) or \lstinline!--strip-mdoc-tags! (long).

\section{Localization of mathdoc files}
\label{sec:localization-mdoc}

By default, texts used by the documentator are in English, but you can specify another language (French, German and Italian for the moment), using the \lstinline!-mdlang! (or \lstinline!--mathdoc-lang!) option with a two-letters argument (\lstinline!en!, \lstinline!fr!, \lstinline!it!, etc.).

The \lstinline!faust2mathdoc! script also supports this option, plus a third short form with \lstinline!-l!:
\begin{lstlisting}
faust2mathdoc -l fr myfaustfile.dsp
\end{lstlisting}

If you would like to contribute to the localization effort, feel free to translate the mathdoc texts from any of the \lstinline!mathdoctexts-*.txt! files, that are in the \lstinline!architecture! directory (\lstinline!mathdoctexts-fr.txt!, \lstinline!mathdoctexts-it.txt!, etc.). As these files are dynamically loaded, just adding a new file with an appropriate name should work.

\includepdf[pages=-, frame=true, angle=-90, scale=0.75, nup=1x2]{images/noisemetadata}

\section{Summary of the mathdoc generation steps}
\label{sec:mdoc-summary}

\begin{enumerate}
\item First, to get the full mathematical documentation done on your faust file, call \lstinline!faust2mathdoc myfaustfile.dsp!.
\item Then, open the pdf file \lstinline!myfaustfile-mdoc/pdf/myfaustfile.pdf!.
\item That's all !
\end{enumerate}