\documentclass[12pt]{article}

\title{Math Foundations}
\author{Abel Doñate}
\date{}

\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{graphicx}
\graphicspath{ {./images/} }
\usepackage{wrapfig}

\setcounter{page}{0}

%Geometry
\usepackage{geometry}
\geometry{a4paper, margin=1in}

\begin{document}

\maketitle
\tableofcontents
\newpage

\section{Useful techniques to solve problems}
    \subsection{The Well ordered principle and Descent to infinity}
    The well ordering principle tells us that we cannot have a sequence of non-negative integers such that $a_1>a_2>...$.
    This is useful to apply in descent to infinite proofs. \\ \\
    Suppose you have a problem in which you are given a condition and you have to check what numbers satisfy the condition. In this case you can apply descent to infinite:
    \begin{itemize}
        \item[1)] Find a condition that guarantees you that a set of numbers satisfy the condition if and only if another set of numbers that are strictly less then the original set satisfy the condition.
        \item[2)] By the Well ordering principle, reason that it is impossible to keep descending infinitely many times , so the sequence must stop somewhere.
        \item[3)] If we can apply it many times, that means that there is no set of numbers that satisfy the condition (or only does the trivial ones like $(0,0,0)$).
    \end{itemize}
    A good example is to find the triples $(a, b, c)$ that satisfy $a^3 + 2b^3 = 4c^3$.\\
    Because the right hand side (RHS) is even, then $a$ must be even $\implies a=2x$. \\
    Dividing by $2$ the equation is $4x^3 + b^3 = 2c^3$ \\
    Because RHS is even, then $b$ must be even $\implies b=2y$. \\
    Dividing by $2$ the equation is $2x^3 + 4y^3 = c^3$ \\
    Because LHS is even, then $c$ must be even $\implies c=2z$. \\
    Dividing by $2$ the equation is $x^3 + 2y^3 = 4z^3$ \\

    We observe that we are left with the original case. That tells us that $(a, b, c)$ only holds if $(a/2, b/2, c/2)$ also holds. Applying descent to infinity, we assure that there is no solution except the trivial one $(0,0,0)$

    \subsection{Mathematical induction}
    We can prove lots of types of problem by this simple principle. \\

    Suppose we have to prove that a given condition holds for every natural number. Then one way to prove it is to check if holds for $k=1$, and then prove that if it holds for $k$, then, it must hold for $k+1$. \\

    The sketch of the steps is the following:
    \begin{itemize}
        \item[1)] \textbf{Base case.} First we prove that the condition holds for $k=1$ (or the first element of the set you are given).
        \item[2)] \textbf{Induction statement.} Now we prove that if $k$ holds, then $k+1$ also holds.
        \item[3)] \textbf{Proof.} Finally you reason that the problem has been solved by the principle of mathematical induction
    \end{itemize}

    \subsection{Double counting}
    Sometimes it is useful to exploit different perspectives of a problem. If your task is to prove an equivalence, perhaps the easiest way to do it is double counting, or compute the expression in two different ways, but showing that both are equal.

\section{Inequalities}
    \subsection{Means inequalities}
    We have the chain of inequalities $HM\leq GM\leq AM\leq QM$:
    \[\frac{1}{\frac{1}{a}+\frac{1}{b}+\frac{1}{c}}\leq \sqrt[3]{abc}\leq \frac{a+b+c}{3}\leq \sqrt{\frac{a^2+b^2+c^2}{3}} \]
    The equality holds when $a=b=c$.

    \subsection{Cauchy's inequality}
    \[\vec{a}\cdot \vec{b}\leq |\vec{a}||\vec{b}| \implies \left(\sum{a_ib_i}\right)^2 \leq \left(\sum{a_i^2}\right)\left(\sum{b_i^2}\right) \]

    \subsection{Chebyshev's inequality}
    If we have two ordered vectors (both increasing or decreasing):
    \[n\sum{a_ib_i}\geq \left(\sum{a_i}\right)\left(\sum{b_i}\right)\geq n\left(\sum{a_ib_{n+1-i}}\right)\]

    \subsection{Jensen's inequality}
    Let $f$ be a convex function (i.e. $f''\geq 0$). Then:
    \[f\left(\sum{\alpha_ix_i}\right) \leq \sum{\alpha_i f(x_i)} \ \ \ \ \ where \ \ \sum{\alpha_i}=1\]

\section{Combinatorial numbers}
    The number of ways to choose k elements in a set of n elements is \textit{n choose k}:
    \[\binom{x}{k}=\frac{n!}{k!(n-k)!}\]
    Some of the properties of combinatorial numbers are:
    \begin{align}
        (x+y)^n &= \sum{\binom{n}{k}x^ny^{n-k}} \\
        \binom{n+m}{k} &= \sum{\binom{n}{i}\binom{m}{k-i}} \ \ \ \text{(Vandermonde's identity)}
    \end{align}

    \begin{wrapfigure}{r}{0.23\textwidth}
        \centering
        \includegraphics[width=0.23\textwidth]{pascal}
    \end{wrapfigure}

    These numbers appear also in Pascal's triangle. The ith element of the nth row (begining both with 0) is $\binom{n}{i}$.\\ \\
    An extension of \textit{n choose k} is the following:
    \[\binom{n}{k_1 \cdots k_m}=\frac{n!}{k_1!\cdots k_m!}\]
    and it appears in the multinomial theorem:
    \[(x_1+\cdots +x_m)^n=\sum_{k_1+ \cdots +k_m=n}{\binom{n}{k_1 \cdots k_m}x_1^{k_1}\cdots x_m^{k_m}}\]

\section{Sets}
    \subsection{Operations and De Morgan's Law}
    Main operations
    \begin{itemize}
    	\item Union. $A \cup B$
    	\item Intersection $A \cap B$
    	\item Difference $A \setminus B$ or $A-B$
    	\item Symmetric difference $A \bigtriangleup B = (A\cup B)-(A\cap B)$
    	\item Complement. $\bar{A}$
    \end{itemize}
    \textbf{De Morgan's Law:}
    \begin{itemize}
    	\item $\overline{A \cup B}=\bar{A}\cap \bar{B}$
    	\item $\overline{A\cap B}=\bar{A}\cup \bar{B}$
    \end{itemize}

	\subsection{Canonical decomposition}
	Every function $f$ can be decomposed in a surjective, bijective and injective functions. 		For instance: $f=i \circ b \circ \pi$.
	\begin{itemize}
		\item $\pi$ maps to its equivalence class
		\item $b$ maps to the function of every equivalence class
		\item $i$ maps to itself
    \end{itemize}
    \section{Algebraic structures}
	\subsection{Group}
	A group is a set with one binary operation $(S,\star)$ that must satisfy:
	\begin{itemize}
		\item Closure under $\star$
		\item Associative $(a\star b)\star c=a\star (b\star c)$
		\item Identity element $e$ such that $e\star a=a\star e=a$
		\item Inverse element. Every $a$ has an inverse $a^{-1}$
	\end{itemize}

	If the group is commutative $a\star b= b\star a$, then is called \textbf{Abelian}. \\

	Examples of abelian group are $\mathbf{Z}_n$ (group whose elements are $0, 1, \cdots, n-1$, and the operation is the sum modulo n), or the group or rotations of a regular polygon. \\

	An example of non-abelian group is the group of rotations of a cube. The proof is left to the reader.

	\subsection{Ring}
	A Ring is a set with two binary operations $(S,\star, \top)$ that must satisfy:
	\begin{itemize}
		\item $(S, \star)$ is an abelian group
		\item Distributive $a\top (b\star c)= (a\top b) \star (a\top c)$
		\item Identity elements $e_\star, e_\top$
	\end{itemize}

	If $e_\star \neq e_\top$ and $a\top b\neq e_\star$, then is called an \textbf{Integral domain}.\\

	One example of a Ring is $\mathbf{Z}$ with usual sum and product.\\

	$\mathbf{Z}_p$ with $p$ prime is an example of an integral domain. $\mathbf{Z}_6$, because 6 is not prime, is not an integral domain, as $2\times 3=0=e_+$.

	\subsection{Field}
	A field is a ring with no inverse for $e_\star$ under $\top$.\\

	The most common example is $\mathbf{R}$. Observe that the element $0=e_+$ has no inverse under multiplication.

\section{Stereographic projection}
    \begin{wrapfigure}{r}{0.30\textwidth}
        \centering
        \includegraphics[width=0.30\textwidth]{stereo1}
    \end{wrapfigure}

    The stereographic projection maps every point in a sphere except the upper pole to the $XY$ plane. On the other hand we can take the inverse stereographic projection, that maps every point in the plane to the sphere (except the upper pole). \\

    This map is achieved by drawing a line from the upper pole to the point in the sphere and taking the intersection with the $XY$ plane. \\

    If we define the coordinates of the plane as $x, y$ as usual and the coordinates of the sphere as $\alpha, \beta, \gamma$, we can derive the following equations:
    \[F(\alpha, \beta, \gamma)=\left(\frac{\alpha}{1-\gamma}, \frac{\beta}{1-\gamma}\right) \ \ \ \
    F^{-1}(x, y)=\left(\frac{2x}{x^2+y^2+1}, \frac{2y}{x^2+y^2+1}, \frac{x^2+y^2-1}{x^2+y^2+1}\right)\]

\end{document}


\section{Background}

\subsection{The original multi-head attention
mechanism}

The scaled dot-product attention proposed by
\citet{vaswani2017attention} transforms the vectorial embedding $\vec{Y}_i$ of a token as a
 function of a sequence of other embedding $\vec{X}_j$. Where
 $\vec{Y}_i$ and $\vec{X}_j$ are all vectors of size $D$. A key
$\vec{K}_j$ and value $\vec{V}_j$ are attributed to each vector
$\vec{X}_j$, and query $\vec{Q}_i$ is attributed to $\vec{Y}_i$.
 The vectors are obtained by linear projection from dimension $D$ to $d$ using three matrices of learnable parameters. The new vector
 $\vec{y}_i'$ is a weighted sum of the $\vec{V}_j$. The weights are
 scores of matching between the query $\vec{Q}_i$ and the keys
$\vec{K}_j$, calculated as the dot product between the two vectors.
They are also \emph{softmaxed} to sum up to 1. The transformation of $L_Q$
vectors $\vec{Y}_i$ as a function of $L_K$ vectors $\vec{X}_j$ can
 be efficiently computed with matrix multiplications:

\begin{equation}
A = \mathrm{softmax}\left(\frac{Q \times K^T}{\sqrt{d}}\right) \times V
\end{equation}

With $Q$ a matrix of shape $(L_Q, d)$, $K$ a matrix of shape
 $(L_K, d)$ and $V$ a matrix of shape $(L_K, d)$. The $\sqrt{d}$
at the denominator is a scaling factor used to avoid saturation in the
exponential terms of the softmax function.

The multi-head attention performs $h$ different projections into spaces
 of dimension $d = D/h$. The resulting vector $\vec{Y}_i'$ is the
concatenation of the $h$ vectors $\vec{y}_i'$ obtained. Thus the
embedding dimension is preserved. Using multiple heads was found
beneficial by the authors over using a single head of dimension
$d = D$.

During training, the cross entropy of the $n^{th}$ predicted token is
 calculated assuming all previous tokens have been generated correctly.
 This enables to parallelize training completely without need for
recurrence. However as the $n^{th}$ token should not depend of the
following tokens, the cells in the upper right corner of the score
matrix are set to $-\infty$ such that after the softmax they are equal
to 0, and the rows still sums up to 1.

\subsection{Improving Transformer scalability with sequence
length}

The original attention mechanism requires the computation of a score
matrix $Q \times K^T$ of shape $(L_Q, L_K)$, with complexity
$O(L_QdL_K)$. If the query and key sequence lengths are multiplied by
two, then the memory used and computation time are multiplied by 4. To
improve the scalability of the transformer with sequence length, several axis of research have been explored.

\citet{kitaev2020reformer} proposed
the Reformer's architecture, which uses an hash-bucketting algorithm to
reduce the complexity of the original multi head attention operation
from $O(L^2)$ to $O(L\log(L))$.

\citet{dai2019transformerxl} proposed the
Transformer-XL's architecture, which cuts the sequence in segments of
length L. The model predicts each stage of the current segment as a
function of the previous and current segment. All the segments are
computed sequentially with a recurrence mechanism. The complexity is linear
with sequence length, but the computation cannot be completely
 parallelized due to the recurrence mechanism, although more than a RNN,
as segments can be computed in one go.

Other publications explored using a sparse attention matrix, such as the
 Longformer by \citet{beltagy2020longformer} and the Big Bird model by
 \citet{zaheer2021big}. As each
token attends to a fixed number of all other tokens, the scalability is
improved. These sparse attention models however require custom
operations implemented in CUDA.

Some other works propose to modify the attention mechanism to be
 compatible with linear complexity. The Linformer by
\citet{wang2020linformer} projects the
 key and values onto a smaller sequence length dimension with matrix
 multiplication. It cannot however generalize to sequences longer than
 during training, as the weights of the projection for such tokens would
 be undefined.

\citet{shen2020efficient} proposed to
 replace the softmax attention score.
 $A = \mathrm{softmax}\left(\frac{Q \times K^T}{\sqrt{d}}\right) \times V$ is
changed into $A = \rho(Q) \times \rho(K)^T \times V$. With $\rho$
the softmax function along the embedding dimension. Thanks to matrix
multiplication commutativity, the order of the operations can be chosen.
If $Q$, $K$ and $V$ are of shape $(L_Q, d)$, $(L_K, d)$ and
$(L_K, d)$ respectively, the complexity of
$(\phi(Q) \times \phi(K)^T) \times V$ is $O(L_Q \times d \times L_K)$ whereas the complexity of
$\phi(Q) \times (\phi(K)^T \times V)$ is
$O\left(max(L_Q, L_k) \times d^2 \right)$. The right-side-first
operation is linear in complexity with sequence length. The shape of the
intermediate result matrix is also changed, allowing to scale the better
in memory requirements as well. The original \emph{softmaxed} attention score
matrix was giving rows of positive scores that sum to 1. With this
change the elements of the score matrix remain positive as $\phi(Q)$
and $\phi(K)^T$ are matrices of positive values, but the rows of the
score matrix does not sum up to 1. This work also does not give a linear complexity formulation for masked attention. If the right-side-first
scheme is adopted, the attention score matrix
$\phi(Q) \times \phi(K)^T$ is never explicitly computed, and can't be
masked.

Building on this idea of commutative attention function proposed by
\citep{shen2020efficient}, \citet{katharopoulos2020transformers} introduced their kernerlized attention function as:

\begin{equation}
A = \frac{\phi(Q) \times \phi(K)^T}{\sum_j \left( \phi(Q) \times \phi(K)^T \right)} \times V
\end{equation}

The function $\phi$ is applied element-wise and can be any positive
function, for example $\phi(x) = elu(x) + 1$. This attention is
row-wise normalized so that all rows of the score matrix are sets of
positive weights adding up to one. This preserves the objective of the
original softmaxed attention scores, while allowing to perform
operations in an optimal order.

The Performer by \citet{choromanski2021rethinking} exploits the same idea of a kernelized attention introduced
by \citet{katharopoulos2020transformers}, with an algorithm that betters approximate softmaxed attention. Most
importantly they also give in annex a prefix sum algorithm to perform
operations in the right-side-first order while giving the same result as
masked left-side-first operation.

Although the author did not specify how to implement it, the only
 implementations we found of this operation requires custom CUDA code. In
 this work we will give an implementation of the right-side-first masked
operation, with usual functions from neural network frameworks, that
remains linear in complexity.

\subsection{Alternatives to absolute positional
encoding}

The original multi-head attention operation introduced by
\citet{vaswani2017attention} was
intrinsically invariant by token order permutation. As token position was
an important information for sequence to sequence models, they encoded
the global position of each token in their embedding. Since then, some
modified attention mechanisms, that depend on relative tokens position,
have been proposed.

\citet{shaw2018selfattention} explored
modifying the attention mechanism so that it depends on the relative
distance between tokens. A second score matrix that is function of the
query and the query/key relative distance is added to the original score
matrix that only depends on query/key vector representation. $A = softmax\left(\frac{Q \times K^T}{\sqrt{d}}\right) \times V$ becomes
$A = \left(\frac{Q \times K^T + S_{rel}}{\sqrt{d}}\right) \times V$ with $S_{rel}$ of shape $(L_Q, L_K)$ defined as
${S_{rel}}_{ij} = \vec{Q_i} \cdotp \vec{RP}_{clip(i-j, -k, k)}$. Where $k$ is the attention horizon length and $\vec{RP}_n$ is one of
$2k+1$ relative positional embedding, vectors of size $d$.
\citet{shaw2018selfattention} and
\citet{huang2018music} observed that introducing this attention scheme improved performances.
The naive calculation of this term however has a complexity of
$O(L_QL_Kd)$. No algorithm was provided to linearize the complexity.

More recently \citet{liutkus2021relative} gives a stochastic positional encoding that is linear in
complexity with regards to sequence length. However the implementation
is complex and its stochastic nature requires that the operations be
repeated several times in parallel.

\citet{horn2021translational} noted that
the term $S^{rel} \times V$ can be computed with linear complexity for
the case where $RP_{-k} = RP_{k}$. However this is restraining as the
model can't make the difference between tokens before the attention
horizon or after.

In this work we will show that the computation of $S^{rel} \times V$
can also be done with linear complexity, without concession.

\endinput


\mychapter{6}{Lesson 6} %181012

\section{Computationally secure encryption}

Having a better idea of what can and can't be accomplished in the cryptographic world, by means of theorems and proofs, we can focus now on the goal of defining a cryptographic system that meets our requirements. In this lesson, we focus specifically on the secrecy-oriented schemes, thus dealing with encryption and decryption of messages.

The requirements of a ``good'' encryption scheme are collectively called those of \emph{computationally secure encryption}: the characterizing requirement is to design a task, or routine, that is \emph{computationally hard} for an attacker to revert.
In detail: this task usually involves a secret key\footnotemark, and is accomplished in polynomial time, and any attacker who wishes to revert it has no efficient means of doing it without knowing such key. Other properties include:

\footnotetext{This is the case for symmetric-key schemes, though many other kinds exist: some involving ``public'' keys, some others not having any key at all}

\begin{enumerate}
    \item \label{prop:owk} \emph{one-way}ness with respect to the encryption key: given $c = \Enc(k, m)$, it should be hard to recover $k$
    \item \label{prop:owm} \emph{one-way}ness with respect to the original message: given $c = \Enc(k, m)$, it should be hard to recover $m$
    \item \label{prop:nol} In a stricter sense: no information whatsoever must ``leak'' from the message
\end{enumerate}

To start visualizing these concepts, let $\Pi = (\Enc, \Dec)$ be a secrecy scheme, and consider the game depicted in figure \ref{cryptogame:otindist}  where the adversary ``wins'' the game when the challenger outputs 1.

\begin{cryptogame}
    {otindist}
    {$\cryptog{ind}(\lambda, b)$}
    {ind}

    \send{}{$m_0, m_1 \in \M$}{}

    \receive{\shortstack[l]{
        $k \pickUAR \K$ \\
        $b \pickUAR \binary$ \\
        $c \pickUAR \Enc(k, m_b)$ }}
    {$c$}{}

    \cseqdelay

    \send{}{$b'$}{\textsc{Output 1 iff} $b = b'$}

\end{cryptogame}

\begin{definition}
    The scheme $\Pi$ is said to be \emph{computationally one-time secure} iff:
    \[
        \forall \adversary \in \ppt \implies \cryptog{ind}(\lambda, 0) \compindist \cryptog{ind}(\lambda, 1)\footnotemark
    \]
    or, rephrased in probability terms:
    \[
        \forall \adversary \in \ppt \implies \lvert\Pr[\cryptog{ind}(\lambda, 0) = 1] - \Pr[\cryptog{ind}(\lambda, 1) = 1]\rvert \in \negl(\lambda) \qedhere
    \]
\end{definition}
\footnotetext{$\cryptog{ind}$ refers to the indistinguishability of the messages sent by \adversary{} during the game}

This last definition shows how such a scheme is compliant with the three properties exposed beforehand. In particular:

\begin{enumerate}
    \item \emph{It is hard to recover the key}. If not, then an adversary \adversary{} can efficiently recover the key and use it to decrypt the ciphertext, which in turn enables him to perfectly distinguish $m_{0}$ from $m_{1}$ on any instance;
    \item \emph{It is hard to recover the message}. This is analogous, and even more obvious than the preceding point. Nevertheless, this is a necessary condition for a secrecy scheme to be ``good'', and it mustn't be forgotten;
    \item \emph{No information about the message whatsoever may leak from the ciphertext}. This may seem subtler than the previous point, but it warrants caution. Observe how an adversary \adversary{}, if it has the ability to extract even a tiny bit of information of the original message from the ciphertext, then it is actually able to make an educated guess on which message was encrypted in the first place, putting him at an advantage. This leads the probabilities described in the definition to be sensibly more unbalanced than negligible, forfeiting the desired secrecy.
\end{enumerate}

By extension, we may ask ourselves what scheme may or may not be \emph{computationally two-time secure}. For instance, let $\Pi_{\oplus} = (\Enc, \Dec)$ be a secrecy scheme using a \prg{} $G : \binary^\lambda \to \binary^n$, structured as follows:

\begin{itemize}
    \item $\K = \binary^\lambda$, $\M = \C= \binary^n$
    \item $\Enc(k, m) = G(k) \oplus m$
    \item $\Dec(k, c) = c \oplus G(k) = m$
\end{itemize}

To be two-time secure means that, even if an adversary \adversary{} gets hold of a valid plaintext-ciphertext couple $(\overline{m}, \overline{c})$, he is unable to decrypt any future ciphertexts\footnotemark, apart from the obvious $\overline{c}$. However, observe that \adversary{} is now able to extract valuable information for decrypting future ciphertexts:

\footnotetext{This example models a technique called \emph{Chosen Plaintext Attack}, which will be discussed in depth later}

\[
    \overline{c} = \Enc(k, \overline{m}) = G(k) \oplus \overline{m} \implies \overline{c} \oplus \overline{m} = G(k)
\]
so now, for any second ciphertext \adversary{} receives, he can \emph{mimic} the decryption routine, and efficiently uncover the underlying plaintext. This proves that $\Pi_\oplus$ is not two time-secure; nevertheless, it is still one-time secure:

\begin{theorem}
    If $G$ is a \prg, then $\Pi_\oplus$ is computationally one-time secure
\end{theorem}

\begin{proof}
    This proof is another example that showcases the use of hybrid games. Recalling the one-time security definition, we need to show that:
    \[
        \forall \adversary \in \ppt \implies \cryptog{ind}[\Pi_\oplus](\lambda, 0) \compindist \cryptog{ind}[\Pi_\oplus](\lambda, 1)
    \]
    Consider the hybrid game in figure \ref{cryptogame:xorothybrid}, where the original encryption routine is changed to use a completely random value, instead of using $G(k)$\footnotemark. As an exercise, compare it with the original one-time secure definition in figure \ref{cryptogame:otindist}, to check that it perfectly matches.

    \footnotetext{The observant student may recognize that this modification yields exactly the ``one-time pad'' secrecy scheme discussed in lesson 1}

    \begin{cryptogame}
        {xorothybrid}
        {$\hybridg{}[\Pi_\oplus](\lambda, b)$}
        {r-ind}

        \send{$m_0 \neq m_1 \in \binary^l$}{$m_0, m_1$}{}

        \receive{\shortstack[l]{
            $r \pickUAR \binary^l$ \\   % R \sim \unifdist{\binary^l}
            $b \pickUAR \binary$ \\     % B \sim \unifdist{\binary}
            $c = r \oplus m_b$            % C = R \oplus m_B
        }}
        {$c$}{}

        \cseqdelay

        \send{}{$b'$}{\textsc{Output 1 iff} $b' = b$}

    \end{cryptogame}

    The proof begins by affirming that:
    \begin{claim}
        \[
            \forall \adversary \in \ppt \implies \hybridg{}[\Pi_\oplus](\lambda, 0) \equiv \hybridg{}[\Pi_\oplus](\lambda, 1)
        \]
    \end{claim}

    To prove it, notice that $r$ is chosen uniformly at random, and independently of $b$. Thus, no matter how the messages $m_0$ and $m_1$ are structured, $r$ will effectively make the chosen message completely unrecognizable. In formal terms, let $B$ and $R$ be the random variables for \challenger{}'s picks of $b$ and $r$ respectively; then:
    \begin{align*} % AP190906: Totally original, need a peer review here
        & \left| \Pr(B = 0 \knowing C = c) - \Pr(B = 1 \knowing C \evaluatesto c) \right| & \\
        =& \left| \Pr(B = 0 \knowing R \oplus m_B \evaluatesto c) - \Pr(B = 1 \knowing R \oplus m_B \evaluatesto c) \right| & \text{(C definition)} \\
        =& \mathrlap{\frac{\left| \Pr(R \oplus m_B = c \knowing B \evaluatesto 0) \Pr(B = 0) - \Pr(R \oplus m_B = c \knowing B \evaluatesto 1) \Pr(B = 1) \right|}{\Pr(R \oplus m_B = c)}} & \\
        & & \text{(Bayes' theorem)} \\
        =& \frac{\left| \Pr(R \oplus m_0 = c) \Pr(B = 0) - \Pr(R \oplus m_1 = c) \Pr(B = 1) \right|}{\Pr(R \oplus m_B = c)} & \text{(Cond. collapse)} \\
        =& \frac{\left| \oneover{2^l} \half - \oneover{2^l} \half \right|}{\Pr(R \oplus m_B = c)} = 0 & \\
    \end{align*}

    Having proven that \adversary's success is equivalent to straight guessing in $\hybridg{}[\Pi_\oplus]$, we now relate the hybrid game to the original one, affirming that:

    \begin{claim}
        \[
            \forall \adversary \in \ppt,\, \forall b \in \binary \implies \hybridg{}[\Pi_\oplus](\lambda, b) \compindist \cryptog{ind}[\Pi_\oplus](\lambda, b) \qedhere
        \]
    \end{claim}

    The proof proceeds by reduction as depicted in figure \ref{cryptoredux:xorotprg}, by assuming the existence of a distinguisher $\textsf{D}^{\textsc{ind}}$ for $c = G(k) \oplus m_{b}$ and $c = r \oplus m_{b}$, and using it to break $G$'s pseudo-random generation property.

    \begin{cryptoredux}
        {xorotprg}
        {Reducing to breaking a \prg}
        {prg}
        {G-ind}[2]

        \receive{\shortstack[l]{
            $k \pickUAR \binary^\lambda$ \\
            $x_0 \pickUAR G(k)$ \\
            $x_1 \pickUAR \binary^l$ \\
            $b \pickUAR \binary$
        }}{$x_b$}{}

        \return{}{$m_0, m_1$}{}

        \invoke{\shortstack[l]{
            $\beta \pickUAR \binary$ \\
            $c = x_b \oplus m_\beta$
        }}{$c$}{}

        \return{}{$\beta'$}{}

        \cseqdelay
        \send{$b' = \begin{cases}
            0 &\textsc{iff } \beta' = \beta \\
            1 &\textsc{else}
        \end{cases}$
        }{$b'$}{\textsc{Output 1 iff} $b' = b$}

    \end{cryptoredux}

    Again, notice how the games of indistinguishability and pseudo-random generation are reliably reproduced on their respective sides. This construct's value centers on how $\distinguisher^\textsc{ind}$ will perform in its own challenge:
    \begin{itemize}
        \item if $x_b$ is a random value, then $\distinguisher^\textsc{ind}$ will perform as depicted in the hybrid game, thus giving right or wrong answers at random;
        \item if b is the result of $G$, then $\distinguisher^\textsc{ind}$ has a better chance in finding the right answer by its own design;
    \end{itemize}
    From these observations, especially the second point, the adversary \adversary{} has a better chance of winning the \prg{} game by asserting that $x_b$ comes from $G$ whenever $\distinguisher^\textsc{ind}$ makes a correct guess; conversely, \adversary{} will preferably declare that $x_b$ is truly random whenever $\distinguisher^\textsc{ind}$ fails a guess, as the probability of the latter getting fooled by a random value is sensibly greater than by a pseudo-random one.

    Either way, by the existence of $\distinguisher{}^\textsc{ind}$, \adversary{} gains an edge in efficiently recognizing $G$, which cannot happen by $G$'s definition; the claim is proven. The theorem's proof can be completed by putting the pieces together, as is usual in the hybrid argument:
    \[
        \cryptog{ind}[\Pi_\oplus](\lambda, 0) \compindist
        \hybridg{}[\Pi_\oplus](\lambda, 0) \equiv
        \hybridg{}[\Pi_\oplus](\lambda, 1) \compindist
        \cryptog{ind}[\Pi_\oplus](\lambda, 1)
    \]

    which finally states that $\cryptog{ind}[\Pi_\oplus](\lambda, 0) \compindist \cryptog{ind}[\Pi_\oplus](\lambda, 1)$.
\end{proof}

\section{Pseudorandom functions}

\textsc{Prg}s are used in practice as a stepping stone for building \emph{pseudo-random functions}, \prf{} henceforth, which are the principal construct in several cryptographic schemes. Before formally introducing what a \prf{} is, we begin instead by defining what a \emph{truly random function} is:

\begin{definition}
    A random function $R : \binary^n \to \binary^l$ is a function that, depending on what is known about its previous applications:

    \begin{itemize}
        \item if $x$ is ``fresh'' (in formal terms, $R$ has never been applied to $x$ beforehand), then a value $y$ is chosen \uar{} from $R$'s codomain, and it is permanently associated as the image of $x$ in $R$\footnote{This property is also called \emph{lazy sampling}.};

        \item if $x$ is not fresh, then $R(x)$ is directly returned instead. \qedhere
    \end{itemize}
\end{definition}

It should be noted that, if such functions are to be implemented in computers, they would occupy too much space in memory. Suppose all the possible outputs of $R$ have been generated and stored as an array in memory; then its total size in bits will be $l \cdot 2^n$:

\[
    \color{black!50}
    \overbracket[0.5pt][5pt]{\;\color{black}\framebox[4em][c]{0010...}\;}^{l \text{ bits}}_1\;
    \overbracket[0.5pt][5pt]{\;\color{black}\framebox[4em][c]{1110...}\;}^{l \text{ bits}}_2\;
    \overbracket[0.5pt][5pt]{\;\color{black}\framebox[4em][c]{0011...}\;}^{l \text{ bits}}_3\;\;
    \color{black!25}
    \framebox[2em][c]{\rule{0pt}{1.5ex}...}\;
    \framebox[2em][c]{\rule{0pt}{1.5ex}...}\;
    \framebox[2em][c]{\rule{0pt}{1.5ex}...}\;
    \framebox[2em][c]{\rule{0pt}{1.5ex}...}\;\;
    \color{black!50}
    \overbracket[0.5pt][5pt]{\;\color{black}\framebox[4em][c]{1110...}\;}^{l \text{ bits}}_{2^n}
\]

Such a function becomes cumbersome and difficult to maintain in practice; therefore, it is desirable to find a kind of function which looks as a random function possible, but does not require to be wholly memorized, while not forgetting to maintain poly-time complexity. Pseudo-randomness comes to the rescue here:

\begin{definition}
    Let $f$ be a function, then it is deemed pseudo-random (therefore, $f$ is a \prf{}) iff it is computationally indistinguishable from a true random function.
\end{definition}

In detail, \prf{}s are actually designed as function families $f_k$\footnotemark, where $k$ is a parameter that indexes the functions inside the family. To model the \prf{}s' indistinguishability from random functions, let $F \in \binary^\lambda \to (\binary^{n(\lambda)} \to \binary^{l(\lambda)} )$, usually denoted simply by $f_k$, be a \prf{}, and define $\mathfrak{R}(n, l)$ to be the domain that collects the random functions from $\binary^{n(\lambda)}$ to $\binary^{l(\lambda)}$.

\footnotetext{Does this remind you of something else? If not, look back in lesson 2 and 3.}

Consider the indistinguishability game drawn in figure \ref{cryptogame:prf}; although one may thing that \prg{}s and \prf{}s aren't much different, the game tells a different story, which is best put by an introductory paragraph about \prf{}s in their Wikipedia page:

\begin{quotation}
    ``Pseudorandom functions are not to be confused with pseudorandom generators (\prg{}s). The guarantee of a \prg{} is that a single output appears random if the input was chosen at random. On the other hand, the guarantee of a \prf{} is that all its outputs appear random, regardless of how the corresponding inputs were chosen, as long as the function was drawn at random from the \prf{} family.''\footnote{\linkicon \href
        {https://en.wikipedia.org/wiki/Pseudorandom_function_family}
        {\textsf{Pseudorandom function family --- Wikipedia}}}
\end{quotation}

\begin{cryptogame}
    {prf}
    {The \prf{} indistinguishability game}
    {prf}

    \cseqchallenger{\shortstack[l]{
        $k \pickUAR \binary^\lambda$ \\
        $R \pickUAR \mathfrak{R}(n, l)$ \\
        $b \pickUAR \binary$
    }}

    \cseqdelay
    \cseqbeginloop

    \send{}{$x$}{}

    \receive{\shortstack[l]{
        $y_0 = f_k(x)$ \\
        $y_1 = R(x)$
    }}{$y_b$}{}

    \cseqendloop
    \cseqdelay

    \send{}{$b'$}{\textsc{Output 1 iff} $b = b'$}

\end{cryptogame}

In this game, \adversary{} is allowed to make multiple queries to $\challenger^\prf$, as opposed to the \prg{} indistinguishability game where it can perform just one query before making its guess. To reiterate the \prf{} definition in terms of this game:

\begin{definition}
    A function family $F = f_k$ is a \prf{} iff:
    \[
        \cryptog{prf}[f_k](\lambda, 0) \compindist \cryptog{prf}[f_k](\lambda, 1) \qedhere
    \]
\end{definition}

\begin{exercise}
    Prove the following statements:
    \begin{itemize}
        \item No \prg{} is secure against unbounded attackers;
        \item No \prf{} is secure against unbounded attackers.
    \end{itemize}

\end{exercise}

\subsection{\textsc{Ggm}-tree}

This section is dedicated to a concrete example of a \prf{} which is built from the ground up using a \prg. This construct has been designed from Oded Goldreich, Shafi Goldwasser and Silvio Micali, and its structure is akin to a binary tree, hence its name: \emph{\ggm-tree}.

\begin{construction}
    Let $G \in \binary^\lambda \to \binary^{2\lambda}$ be a \prg{} such that it doubles the length of its argument, and denote the images' first and second halves as $G_0(k)$ and $G_1(k)$ respectively, so that:
    \[
        k \mapsto (G_0(k), G_1(k))
    \]

    Since the principal mechanism makes use of the halves being the same length of the argument, in the same spirit, we will denote the action of using one half of an image of $G$ as argument of $G$ itself in a shorter fashion, as demonstrated in the following example:
    \[
        G_a(G_b(G_c(k))) =: G_{abc}(k)
    \]

    This leads to the final step: let $f_k$ be a function family, where $k \in \binary^\lambda$, such that:
    \[
        f_k(r) = G_r(k)
    \]

    This is our candidate \prf. To visualize it, consider the tree structure depicted in figure \ref{fig:ggmtree}: at each level of the tree, a single bit of $r$ is used to decide which half of $G$'s image will be used in the next level. For example, $f_k(01 \dots 10)$ would evaluate as $G_0(G_1( \dots G_1(G_0(k))))$.

    \begin{figure}
        \centering
        \begin{tikzpicture}[
            level 1/.style={sibling distance=16em},
            level 2/.style={sibling distance=8em},
            level 3/.style={sibling distance=4em}]

            \node{$k$}
                child{ node {$G_0(k)$}
                    child{ node {$G_{00}(k)$}
                        child{ node{$G_{000}(k)$} node[below]{$\vdots$}}
                        child{ node{$G_{001}(k)$} node[below]{$\vdots$}}
                    }
                    child{ node {$G_{01}(k)$}
                        child{ node{$G_{010}(k)$} node[below]{$\vdots$}}
                        child{ node{$G_{011}(k)$} node[below]{$\vdots$}}
                    }
                }
                child{ node {$G_1(k)$}
                    child{ node {$G_{10}(k)$}
                        child{ node{$G_{100}(k)$} node[below]{$\vdots$}}
                        child{ node{$G_{101}(k)$} node[below]{$\vdots$}}
                    }
                    child{ node {$G_{11}(k)$}
                        child{ node{$G_{110}(k)$} node[below]{$\vdots$}}
                        child{ node{$G_{111}(k)$} node[below]{$\vdots$}}
                    }
                };
        \end{tikzpicture}
        \caption{The \textsc{ggm}-tree for $G$}
        \label{fig:ggmtree}
    \end{figure}

\end{construction}

The proof of \prf-ness will be discussed in the next lesson.


%
% CMPT 310: Artificial Intelligence - A Course Overview
% Section: Game Playing and Adversarial Search
%
% Author: Jeffrey Leung
%

\section{Game Playing and Adversarial Search}
	\label{sec:game-playing-adversarial-search}
\begin{easylist}

& Game playing: Scenario where a game with discrete states is played with two players, each seeking to maximize their position which minimizes the opponent's position
	&& E.g. Chess, checkers
	&& \textbf{Heuristic/evaluation function:} Estimate of the optimality of a game state
	&& \textbf{Terminal state:} Game state which has no further states
	&& Example diagram of game state evaluations in a tree: See figure~\ref{fig:game-tree}

\begin{figure}[!htb]
	\caption{Game Tree (Minimizer first)}
	\label{fig:game-tree}
	\centering

	\begin{forest}
		for tree={
			draw,
			circle,
			minimum size=1cm,
			line width=0,
			align=center
		},
		[0
			[2
				[2]
				[-3]
				[-14]
			]
			[14
				[14]
				[17]
				[20]
			]
			[0
				[-12]
				[-18]
				[0]
			]
		]
	\end{forest}
\end{figure}

& \textbf{Minimax algorithm:} Decision-making rule which minimizes the potential loss for a worst-case scenario, given two entities who are alternatively trying to maximize and minimize a score
	&& Performs the equivalent of a depth-first search
	&& Each alternating level of the decision tree is minimized or maximized from the values in the levels below it
	&& Complete: Yes, as long as the tree is finite

& \textbf{$\alpha / \beta$ pruning:} Keeping the maximum/minimum searched values at a given level so far to eliminate nodes which will not exceed a minimum/maximum
	&& $\alpha$: Lower bound on the potential value of a maximum node
	&& $\beta$: Upper bound on the potential value of a minimum node
	&& \href{http://inst.eecs.berkeley.edu/~cs61b/fa14/ta-materials/apps/ab_tree_practice/}{Resource for practice}
	&& See figure~\ref{fig:game-tree-ab} for an example of pruned nodes (notated by dashed edges)

\begin{figure}[!htb]
	\caption{Game Tree with $\alpha / \beta$ Pruning (Minimizer first)}
	\label{fig:game-tree-ab}
	\centering

	\begin{forest}
		for tree={
			draw,
			circle,
			minimum size=1cm,
			line width=0,
			align=center
		},
		[0
			[2
				[2]
				[-3]
				[-14]
			]
			[14
				[14]
				[17, edge={densely dashed}]
				[20, edge={densely dashed}]
			]
			[0
				[-12]
				[-18]
				[0]
			]
		]
	\end{forest}
\end{figure}

& Time, space, and optimal complexity of algorithms: See figure~\ref{fig:algos-complexity}

\end{easylist}
\clearpage


\chapter{Algorithms}
In this chapter, we will list some important algorithms which may be used in the solution of each subproblem.

\section{Shortest Path Between Two Nodes}

Algorithms for the calculation of the Shortest Path between two points are very important for the solution of this problem. They will be used to calculate the shortest path between multiple points when calculating a vehicle's path, such as the shortest path between the garage and a pick up point. To do this, there are many algorithms at our disposal. We will evaluate some of them and make a decision on which is(are) best for the situation at hand.

\subsection{Dijkstra's Algorithm}

\subsubsection{Description}
This algorithm was conceived by Edsger W. Dijkstra's in 1956 and is used to calculate shortest paths in directed or undirected graphs, so long there are no edges with negative weight. Upon utilization of this algorithm, a tree holding the shortest paths from the origin node to all others is formed.

\subsubsection{Extra Data, Data Structures and Algorithms Required}
\begin{itemize}
    \item Minimum distance to the origin node from each of the graph's nodes (dist)
    \item Node that comes before it in its path (path)
    \item Priority queue to hold the nodes to be processed next
    \item Decrease-Key function, to maintain the nodes with shortest distance in the top of the queue
\end{itemize}

\input{Algorithms/Dijkstra.tex}

\subsubsection{Analysis}
The first part of the algorithm's goal is to prepare the data (lines 1-6): paths are set to null, distances to infinity, the priority queue is initialized and the origin node is inserted. Then, a breadth-first search is performed; a check is made to every node found in order to understand if its path can be shortened by the use of the edge in analysis. After that, if the node is not already in the queue, it must be inserted, to be eventually processed. If it is, its key must be reduced. The items of the priority queue are ordered by their key. This system is used so that the nodes which are closest to the origin are processed first. This procedure ensures that the distance of already processed nodes remains intact, boosting the algorithm's efficiency and making this a greedy algorithm. \\
%\begin{itemize}
%    \item \textbf{Loop invariant} - $ priorityQueue \neq \emptyset $, as each time a node is processed, its shortest path is defined and all nodes will be processed when the queue is empty
%    \item \textbf{Loop variant} - the number of nodes to be processed, since the algorithm would stop when the queue is empty
%\end{itemize}

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O((|N|+|E|) \times \log(|N|) $, where E is the set of edges and N the set of nodes of the graph. Decrease-Key's time efficiency is $ O(|E| \times log(|N|)$. It can be $ O(1) $ if Fibonacci Heaps are used instead of regular Priority Queues, making the whole algorithm $ O(|N| \times log(|N|) $
    \item \textbf{Spatial Complexity} - Depends on the implementation, but usually $ O(|N|) $
\end{itemize}

\subsubsection{Usability}
This algorithm is one to consider using in the problem, but there may be slightly more efficient ones.

\subsection{Bellman-Ford Algorithm}

\subsubsection{Description}
The Bellman-Ford Algorithm is another algorithm used to calculate the shortest path between nodes in a graph. It was first proposed by Alfonso Shimbel in 1955 but ended up being named after Richard Bellman and Lester Ford, who later officially published it independently. This algorithm is useful to calculate shortest paths in graphs with negative weight edges and to detect negative edge cycles in them.

\subsubsection{Extra Data, Data Structures and Algorithms Required}
\begin{itemize}
    \item Minimum distance to the origin node from each of the graph's nodes (dist)
    \item Node that comes before it in its path (path)
\end{itemize}

\input{Algorithms/Bellman-Ford.tex}

\subsubsection{Analysis}
In this algorithm, the first step is to prepare the graph (lines 1-4). Then, all edges are analyzed $|N| - 1 $ times(lines 6-7), checking if they can be used to shorten the path from s to the destination node of the edge(line 8). This process is repeated this amount of times because it is the maximum length in edges for a path. In the end, a check is made, to see if the algorithm worked. If it did not, the cycle as negative weight loops (lines 14-18). This algorithm is considered to implement dynamic programming.

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O(|N| \times |E|) $, where E is the set of edges and N the set of nodes of the graph
    \item \textbf{Spatial Complexity} - Depends on the implementation, but in this case $ O(1) $
\end{itemize}

\subsubsection{Usability}
This algorithm poses little interest to this problem given that our edges' weights represent the length of a road, which would never be negative.

\subsection{Bidirectional Dijkstra}

\subsubsection{Description}
Ira Pohl was the first one to design and implement a bidirectional heuristic search algorithm in 1971.
Bidirectional Dijkstra's that implements the idea of a bidirectional search into Dijkstra's algorithm. It finds a shortest path from an initial node to a goal node in a directed or undirected graph. It runs two simultaneous searches: one forward from the initial state, and one backward from the goal, alternating between the two. The stopping criteria is one that must be very well implemented to guarantee the shortest path. Compared to normal Dijkstra's there's a speedup of 2x.

\subsubsection{Extra Data, Data Structures and Algorithms Required}
\begin{itemize}
    \item Minimum distances to the origin and goal nodes from each of the graph's nodes (dist)
    \item Nodes that come before each node in both forward and backward paths (not implemented in pseudocode)
    \item Two priority queues to hold the nodes to be processed next in each direction
    \item Way of search (way) (1 if it was encountered by forward search first, 2 if it was encountered by backward search first, 0 default)
    \item Decrease-Key function, to maintain the nodes with shortest distance in the top of the queue
\end{itemize}

\input{Algorithms/Bidirectional.tex}
\uline{Note:} The pseudocode is not complete, missing the parts to calculate the actual path and to calculate the transposed graph (needed for the backward search because this is a directed graph). These parts were excluded in order to not make the algorithm's pseudocode even more complex and spacious.

\subsubsection{Analysis}
This algorithm basically performs two Dijkstra's algorithms: one starting from the initial node and going forward(lines 16-23); another one starting from the ending node and going backward (lines 24-31). Each time they cross, the distance is registered (lines 25-26 for example). The stopping criteria is a tricky problem in this case. In many situations, the algorithm ends when the two sets collide (forward search and backward search). We found that this criteria did not guarantee the shortest path. Instead, we chose to end the algorithm when there can be no longer any better distances because the nodes being processed are already too far apart.
%\begin{itemize}
%    \item \textbf{Loop invariant} - $ dist(n) + distR(v) > bestDist $, where best distance is the best Distance for a complete path obtained yet, dist(n) is the distance of the node being processed in forward way to the starting node and $ distR(v) $ the distance of the node being processed in backward way to the ending node
%    \item \textbf{Loop variant} - the number of nodes to be processed in each way, since the algorithm would stop when each search had got no more nodes to process on the queue
%\end{itemize}

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O((|N|+|E|) \times \log(|N|) $, where E is the set of edges and N the set of nodes of the graph.
    \item \textbf{Spatial Complexity} - Depends on the implementation, but usually $ O(|N|) $
\end{itemize}
Even though there's a speed up, the complexities will be the same as in Dijkstra's, since the difference is multiplication and division by constants, which is ignored in Big O order.

\subsubsection{Usability}
Bidirectional Dijkstra's algorithm is one of the fastest algorithms when it comes shortest paths. It has the down-side of requiring the previous knowledge of the ending/destination node, but in this scenario that is not a problem.

\subsection{A* Algorithms}
\subsubsection{Description and Analysis}
The first time a A* Algorithm came up was in \textit{Shakey Project}, project of the first general purpose mobile robot capable of reasoning about is own actions. In an A* (pronounced A-star) Search Algorithm, the 'level' of a node in a graph becomes a sum of two different characteristics. For instance, applied to this problem, the distance a node is from the origin in the search algorithm is substituted for the sum of:
\begin{itemize}
    \item $d_{sv}$ - actual distance to the origin
    \item $\pi_{nt}$ - estimated guess of the distance to the goal node (heuristic)
\end{itemize}
The reliability of this technique depends on the efficacy of the heuristic. In this case, because the weight of edges is distance in meters, the efficacy is great if we assume $\pi$ to be the euclidean distance. This algorithm can be implemented into both Bidirectional and normal Dijkstra's.
\begin{itemize}
    \item \textbf{Upside} - Speeds up the process immensely
    \item \textbf{Downside} - The algorithm does not absolutely guarantee the best solution.
\end{itemize}

\subsubsection{Usability}
This is a technique that shows to be very promising and a good idea to implement in the solution of this problem.

\newpage
\section{Shortest Path Between All Nodes}
This section is dedicated to algorithms that calculate the shortest path between all pairs of points. Such algorithms may be used in the solution, specially in the pre-processing part.

\subsection{Floyd-Warshall}

\subsubsection{Description}
This algorithm was published by Robert Floyd in 1962, yet it was virtually the same as some algorithms published by Stephen Warshall and Bernard Roy, 1962 and 1959 respectively. It takes an adjacency matrix that represents the graph as an input (empty in the beginning) and calculates the shortest paths between all nodes. The value of a path between to vertices is the sum of all edge's weights through that path. It is registered in the matrix as such: distance of shortest path between 1 and 4 would be matrix[1][4]. Edges may have negative values but the graph can't have a negative cycle.
% For each pair of vertices, the shortest path is calculated. Time complexity is O \begin{math}(|V| ^3).\end{math}, and space complexity is O \begin{math}(|V| ^2).\end{math}

\subsubsection{Extra Data, Data Structures and Algorithms Required}
\begin{itemize}
    \item $ |N| \times |N| $ matrix (another one if paths are needed), where $ |N| $ is the number of vertices
\end{itemize}

\input{Algorithms/Floyd-Warshall.tex}
\uline{Note:} The pseudocode is not complete, missing the matrix to calculate the paths and respective instructions. This was done in order to simplify the algorithm and minimize it.

\subsubsection{Analysis}
First part is basically the preparation of the matrix: fill it with $\infty$ distances; for the edges existant (lines 1-3), change the values in the matrix to their weight (lines 4-6). Then, all pairs of nodes will be checked on if there is any node the path can pass through that would minimize its cost (lines 7-14).

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O(|N|^3) $, where N is the set of nodes of the graph.
    \item \textbf{Spatial Complexity} - Depends on the implementation, but usually $ O(|N|^2) $, for the matrix of size $|N| \times |N|$
\end{itemize}

\subsubsection{Usability}
This algorithm is an interesting algorithm to be used in the beginning of the program, to leave all shortest paths processed. However, this might take a long time, and it might be a better idea to simply calculate the shortest path each time it is needed.

\subsection{Dijkstra's Algorithm}

\subsubsection{Description and Analysis}
Dijkstra's can also be used to calculate all shortest paths. This is done by simply executing the algorithm for every node in the graph as origin.

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O(|N| \times (|N|+|E|) \times \log(|N|) $, where E is the set of edges and N the set of nodes of the graph. Same as Dijkstra's but multiplied by $|N|$
    \item \textbf{Spatial Complexity} - Depends on the implementation, but usually $ O(|N|) $, for the priority queue, same as Dijkstra's
\end{itemize}

\subsubsection{Usability}
This is a better option for less dense graphs compared to Floyd-Warshall. As this is not the case for a road network, this algorithm is relatively irrelevant.

\newpage
\section{Connectivity}
The algorithms in this section check the connectivity of the graph. By other words, this section is dedicated to algorithms that detect if any nodes are inaccessible to others, dividing the graph in strongly connected components. A graph will be strongly connected if there is a path between all pairs of points, or if the graph is a strongly connected component itself. Algorithms to test connectivity may be useful to solve the problem of unpredictability.

\subsection{Brute Force with Floyd-Warshall}

\subsubsection{Description and Analysis}
One way to test the connectivity of a graph is by analyzing the results of Floyd-Warhsall in a certain graph. If there are any pairs of nodes which the distance remains $\infty$, they do not belong to the same strongly connected component.

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O(|N|^3) $, where N is the set of nodes of the graph, same as Floyd-Warshall
    \item \textbf{Spatial Complexity} - Depends on the implementation, but usually $ O(|N|^2) $, for the matrix of size $|N| \times |N|$
\end{itemize}

\subsubsection{Usability}
This algorithm works yet is not a good option. Even if the Floyd-Warshall is going to be utilized anyway, the complexity of traversing the whole matrix is $O(|N|^2)$. Therefore, there are much better algorithms for the job.

\subsection{Kosaraju's algorithm}

\subsubsection{Description}
The algorithm was first published in 1981 by Micha Sharir but gets is name from whom first mencioned it in 1971, Sambasiva Rao Kosaraju. This algorithm allows to identify strongly connected components in a directed graph, testing its connectivity.

\subsubsection{Extra Data, Data Structures and Algorithms Required}
\begin{itemize}
    \item boolean identifying if a node has been visited or not (visited)
    \item identifier of the strong component the node belongs to (strongComponent)
    \item function to invert the edges of the graph
    \item a queue to hold the order of the visits
\end{itemize}

\input{Algorithms/Kosaraju.tex}

\subsubsection{Analysis}
The algorithm performs two depth-first searches in the graph. In the first one, it marks the nodes by visit order (lines 5-9). The second one starts on the nodes with lowest order and is performed on the graph with the edges inverted (lines 10-14). This second one will generate multiple expansion trees (or just one if the graph is strongly connected), each being a strongly connected component. If two nodes are not in the same strongly connected component, one of them cannot be reached from the other.

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O(|N|+|E|) $, where N is the set of nodes of the graph and E the set of edges. Each dfs is $O(|N|+|E|)$
    \item \textbf{Spatial Complexity} - Depends on the implementation, but in this one it is $ O(|N|) $, being the size of the queue $|N|$
\end{itemize}

\subsubsection{Usability}
This algorithm is a serious candidate to check the connectivity of the graph.

\subsection{Tarjan's Algorithm}

\subsubsection{Description}
The Tarjan's Algorithm was named after its inventor, Robert Tarjan, and is another algorithm capable of identifying the strongly connected components in a given graph. It does so, similarly to Kosaraju's, with the help of dfs, only this time the graph is only traversed once.

\subsubsection{Extra Data, Data Structures and Algorithms Required}
\begin{itemize}
    \item boolean identifying if a node has been visited or not (visited)
    \item integer identifying the counter when a node is first found (num)
    \item integer identifying the lowest num reachable by a node (low)
\end{itemize}

\input{Algorithms/Tarjan.tex}

\subsubsection{Analysis}
The algorithm is fairly simple: the graph is traversed by a depth first search (lines 5-6); the $num$ and $low$ of a node are set to the $numCounter$(lines 12-13); each time an already visited node is found, all nodes with $low > that nodes low$ will be changed (lines 16-18). This way, all vertices with the same $low$ belong to the same strongly connected component.

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O(|N|+|E|) $, where N is the set of nodes of the graph and E the set of edges. The dfs is $O(|N|+|E|)$. This is the same complexity as Kosaraju's, but the algorithm is twice as fast since it only performs one dfs.
    \item \textbf{Spatial Complexity} - Depends on the implementation, but in this one it is $ O(1) $.
\end{itemize}

\subsubsection{Usability}
This algorithm is an even better candidate than the last one, since it is supposedly twice as fast.

\newpage
\section{Clustering Algorithms}

A Clustering is a machine learning task which involves the recognition of natural grouping in data. Such algorithms can be used in the distribution of orders per vehicle.

\subsection{Agglomerative Hierarchical Clustering}

\subsubsection{Description}

This algorithm is one of the most common Hierarchical Clustering algorithms. It is used to form groups of objects of data based on their similarity. In this case, we will use it to form N groups of data nodes with great similarity.

\input{Algorithms/Clustering.tex}
\uline{Note:} the implementation presented above is just a rough sketch, the one to be implemented will be different.

\subsubsection{Analysis}
The algorithm begins to find the pair of points which are most similar. In this case, similarity is measured by distance (lines 6-14). After he has found one, he merges the two sets (. In between sets, there are many criteria to calculate similarity. The one we will implement is taking the similarity between two sets as the similarity of the most similar pair of points where one belong to the first set and other to the second set. This algorithm adopts a Bottom-Up strategy (as do all agglomerative hierarchical clustering algorithms), as it start by processing each node and joining them to obtain the final result.

\subsubsection{Efficiency}
\begin{itemize}
    \item \textbf{Temporal Complexity} - $ O(|S|^2*(|S| - n)) $, where S is the set of points. This is only an estimated guess.
    \item \textbf{Spatial Complexity} - Depends on the implementation, but in this one it is $ O(|S|) $.
\end{itemize}

\subsubsection{Usability}
This is the only algorithm analysed in this cathegory, thus it is the chosen one to perform the task.

\section{Extras}
There are other algorithms, simpler or less relevant, that may (or may not) be used in our solution:
\begin{itemize}
    \item Breadth-First Search (bfs) - Graph searching algorithm
    \item Depth-First Search (dfs) - Graph searching algorithm
    \item Decrease-Key Function - Priority changing function on a priority queue
    \item Graph Transposing Function - Inversion of the edges in a directed graph
    \item Fibonacci Heaps (Data structure) - Improved priority queue
    \item Euclidian Distance function - To calculate the euclidian (straight-line) distance between two points
\end{itemize}


\subsection{Node-tree manipulation and the decoder time complexity}
In the following we detail the routines to grow and merge node-trees. These completes the description of the decoder as it is analogous to the original Union-Find algorithm with the addition of these two steps.

The original Union-Find algorithm had a complexity driven by the time complexity of merging the union-find data structures representing the cluster-trees. This is known to be $\mathcal{O}(n\alpha(n)$, where $\alpha(n)$ is the inverse Ackermann function, a sublogarithmic growing function.

The worst-case time complexity of the Union-Find Partitioned-Growth decoder of \Cref{algo:ufbb} follows directly from the complexity of the growth and merge of the node trees. They are still quasilinear. In particular, in sections \ref{} and \ref{}, we describe respectively the routines for the growth and the merge of the node-trees and show that they have an amortized time-complexity $\mathcal{O}(n\log n)$. In consequence, the complexity of the total algorithm is driven by these two steps and the amortized time-complexity is $\mathcal{O}(n\log n)$.

\subsubsection{Growing node-trees}\label{sec:grownodetrees}

The Union-Find decoder grows clusters by adding half-edges to all boundary vertices of the cluster. Instead, here we propose to grow only the nodes with minimal $PMW$. For this, it is necessary to save the list of boundary vertices to each node (\Cref{fig:nodetypes}c). In this algorithm, to \emph{grow a node} means to grow all edges connected to its boundary vertices a half-edge per the Union-Find decoder. The proposed growth algorithm consists of 3 depth-first-searches starting from a reference node $R$. Since the minimal PMW can be found without the PWM of the reference node $R$ itself, any node $N\in\nset$ may be chosen as the reference node $R$.

\begin{algorithm}[h]
  \BlankLine
  \KwData{Node tree $\nset$}
  \KwResult{$\nset$}
  \BlankLine

  Choose a reference node $R \in \nset$\;
  Calculate node parities via tail-recursive DFS in $\nset$ from $R$ via \Cref{eq:snodeparity,eq:jnodeparity}\;
  Calculate PMW with respect to $R$ via head-recursive DFS in $\nset$ from $R$ via \Cref{eq:pmwdif}\;
  Grow all nodes with minimal PMW in a DFS in $\nset$ from $R$\;

  \caption{Grow node-tree (naive)}\label{algo:grow}
\end{algorithm}

When two clusters merge on vertices $a$ and $b$, their respective node-trees must also be merged. This is performed by creating an edge between nodes to which vertices $a$ and $b$ belong to.

\textcolor{cyan}{
  It is possible to reduce the cost of \Cref{algo:grow} by two implementations. First, the DFS's need not be performed during every growth iteration. For this, we introduce the \emph{node delay} $\delta_\nset$, the difference in PMW with the minimal PMW in the node-tree.
}

\begin{equation}\label{eq:nodedelay}
  \delta_N = \text{PWM}(N) - \min_{N\in\nset}{\text{PWM}(N) }
\end{equation}

\textcolor{cyan}{
  With the node delay, the first two DFS's for the calculation of the parity and PMW need only to be performed during the first growth iteration after a merger, during which we now additionally sort the nodes with boundary vertices in the node-tree by the node delay. In the first growth iteration after a merger, we iterate over the sorted nodes and grow all nodes with delay 0, and add 1 to the delay, until a node with delay 1 is encountered. In the second iteration, we grow all nodes with delay 1, and so on. The number of operations for iterating through the sorted nodes is bounded by the number of growth operations of boundary edges, and its contribution to the complexity is thus defined within the original Union-Find decoder. The additional cost of growing a node-tree $\nset$ while not merging is thus $\abs{\nset}$.
}

\textcolor{cyan}{
  A second reduction in the cost is possible by a selection procedure for the reference node $R_{\text{merged}}$ in the merged node-tree $\nset_{\text{merged}}$. Since $\nset_{\text{merged}}$ must be odd to grow, the node-trees involved its merger must consist of at least one odd node-tree $\nset_O$ and a number of other node-trees. If the reference node $R_O$ of $\nset_O$ is chosen for $R_{\text{merged}}$ after the merger, the parity and delays within the sub-tree of nodes in $R_O$ do not change during the merger. Thus, the DFS's can be limited all other node-trees but $\nset_O$. The greatest reduction in cost is achieved if the reference node of the largest odd node-tree in the merger is chosen as the new reference node.
}

After calculating the node delay, each subsequent growth iteration is performed by a single DFS. In this growth DFS, each node with zero delay is grown, and the delay of other nodes are subtracted by $\nicefrac{1}{2}$. This procedure is effectively equivalent to growing the nodes with minimal PMW. If a merge with another node-tree occurs, the PMW's within the merged node-tree change, and a recalculation of the node parities and reference PMW's is required. The reduced cost algorithm is described in \Cref{algo:growdelay}.

\begin{algorithm}[h]
  \BlankLine
  \KwData{Node tree $\nset$, growth iteration $i$ after merge}
  \KwResult{$\nset$}
  \BlankLine

  \If(){$i=0$}{
    Choose a reference node $R \in \nset$.\;
    Calculate node parities via tail-recursive DFS in $\nset$ from $R$ via \Cref{eq:snodeparity,eq:jnodeparity}.\;
    Calculate PMW with respect to $R$ via head-recursive DFS in $\nset$ from $R$ via \Cref{eq:pmwdif}.\;
    Calculate node delay $\delta_N$ in $\nset$ via \Cref{eq:nodedelay}.\;
    Sort nodes with boundary edges by their node delay in a list $\mathcal{L}$.\;
  }
  \For(){node $N$ in $\mathcal{L}$}{
    \eIf(){$\delta_N =i$}{
      Grow boundary edges of $N$. Add 1 to $\delta_N$.
    }(){
      Break loop.
    }
  }
  \caption{Grow node-tree (low-cost)}\label{algo:growdelay}
\end{algorithm}

Note that while the cost is reduced in \Cref{algo:growdelay}, the complexity of the growth procedure is identical to \Cref{algo:grow}, which is proportional to the size of the node-tree due to the growth DFS. However, \Cref{algo:growdelay} allows us to solve a problem described in the next section.

% An addition reduction in cost is possible by retaining the same root node through several mergers, such that the delay and PMW DFS's need only to be performed on a partition of the node-tree. However, these optimizations through implementation do not reduce the overall complexity of the algorithm, and will thus not be covered in this article.

% Equations \Cref{eq:pmwdif,eq:pseudodelay,eq:snodeparity,eq:jnodeparity} show that the differences in PMW with respect to the root node can be calculated through two depth-first searches (DFS) of $\nset$. In the first DFS, the \emph{parity DFS}, we calculate the node parity $\pi_N$ a via tail-recursive function using \Cref{eq:snodeparity,eq:jnodeparity}. In the second DFS, the \emph{PMW DFS}, we calculate the PMW differences $\delta_N$ via a head-recursive function using \Cref{eq:pmwdif,eq:pseudodelay}.

% \todo[inline]{MOVE? or expand, growth DFS is not yet introduced}During this second DFS, the minimal PMW difference within $\nset$ can be kept track of by comparison for calculating the node delay $\delta_N$ at a later instance during the \emph{growth DFS}.

% The Union-Find decoder grows clusters by adding half-edges to all boundary vertices of the cluster. Instead, here we propose to grow only the nodes with minimal $PMW$.
%A single growth iteration, which is applied in the Union-Find decoder by adding half-edges to all boundary vertices of the cluster, is now replaced by another DFS of $\nset$ from the same root $R$ as before.
%During this DFS, $\delta_N$ of every node $N$ is calculated via \Cref{eq:delaypseudo} and $N$ is conditionally grown --- adding half-edges to the boundary vertices in the current node and adding $\nicefrac{1}{2}$ to its radius $\rho_N$ --- if $\delta_N = 0$.
% For this, it is necessary to save the list of boundary vertices to each node (\Cref{fig:nodetypes}c). %When the delays $\delta_N$ for all nodes in $\nset$ are zero, all nodes are grown simultaneously within the same iteration.

% As the nodes grow, the values for $\delta_N$ across $\nset$ change. %This would mean that the DFS's of node parity and delay need to be done after each growth iteration.
% Fortunately, if no new nodes are added to $\nset$ after a growth iteration, the parities %$\pi_N$ on iteration $t+\theta$
% can be calculated via previous values of the PMW differences. %pseudo-delays in iteration $t$ by the introduction of the node \textbf{wait} parameter $\omega_N$.
% During the growth DFS, if $\delta_N \neq 0$, add 1 to $\omega_N$. If no nodes are added to $\nset$, the node delay in the next iteration are
% \begin{multline}\label{eq:delay}
%     \delta_N(t+\theta) = \tilde{\delta}_N(t) - \min_{X \in \nset(t)}{\tilde{\delta}_X} - \omega_N(t+\theta-1),  \\
%     \hspace{.3cm} N \in \nset, \hspace{.3cm} \nset(t+\theta) = \nset(t).
% \end{multline}

% Note that we have not stated which node in $\nset$ should be the root node. In fact, any node in $\nset$ could have been picked as the root of the node-tree. The only requirement is that the DFS of cluster growth must be performed along the same direction as the DFS's of the parity and delay calculations. If no cluster mergers occur, the node delays can be reused in the node suspension calculation prior to node growth. The node-tree is constructed by storing all neighbors of a node to a list. This way, the DFS's' direction can be determined by simply saving the root node, the starting point of the DFS's, to the cluster. All node variables are depicted in \Cref{fig:nodetypes}c.
% In the next section, we expand upon this idea of "reusing" some intermediate parameters to calculate the node suspensions after a cluster merger.

% \subsection{Joining node-trees}\label{sec:nodejoin}

% \Figure[b](topskip=0pt, botskip=0pt, midskip=0pt){figures/tikz/build/main-figure5.pdf}{
%     \emph{(a)} An odd cluster $\nset_o=\{A, B, O\}$ with root $A$ joins with an even cluster $\nset_e=\{C, E\}$ with root $C$ on nodes $O, E$, respectively, to a joined node-tree. If we choose to \emph{(b)}, make $E$ a child of $O$, the parities and delays the subtree of $\nset_o$ can unchanged, and we only have to perform partial parity and delay calculations over the subtree of $\nset_e$. If we choose to \emph{(c)}, make $O$ a child of $E$, parities and delays have to be recalculated in the entire joined node-tree. \label{fig:inversion}}

\Figure[tb](topskip=0pt, botskip=0pt, midskip=0pt){figures/tikz/build/main-figure4.pdf}{
    The relevant data structures. \emph{(a)} The cluster-tree of the Union-Find data structure, whose elements are vertices $v$. The path from a vertex to the root of the cluster-tree is traversed to find the root element, which differentiates clusters. Next to the cluster size and parity, the root node $R$ of the node-tree $\nset$ is stored at the root of the cluster-tree. Each vertex $v$ additionally stores to which node $N$ it belongs. \emph{(b)} A spanning-tree $\vset$ with 3 non-trivial vertices. As $\vset$ is strictly acyclic, the cluster's edges must be maintained such that no cycles are created. This is done during growth by removing edges (red dotted lines) if a cycle is detected. \emph{(c)} The node-tree $\nset$, which consists of the syndrome-nodes $\dot{A}, \dot{B}, \dot{C}$ with primer vertices $a, b, c$, and junction-nodes $\bar{D}, \bar{I}$ with roots $d,i$. At each node $N$ and the root $R=\dot{B}$ various variables are stored.\label{fig:nodetypes}}

% As clusters grow in size they merge with other clusters.
% %In the Union-Find (UF) algorithm, odd clusters
% %parity clusters of an odd number of non-trivial vertices, which are elements of $\sigma$,
% %grow %in size
% %repeatedly and merge with other clusters until all clusters are even.
% %The data structures of the merged cluster can be obtained by merging the data structures of the original clusters: %During these mergers, the node-trees and the cluster-trees are merged. %of the Partitioned-Growth data structure are combined.
% %Let us first make a  distinction between the merging protocols of the underlying data structures;
% %the cluster-trees are merged with the $\Union$ function, whereas
% In the following we describe how to obtain the node-tree of the merged cluster from the original node-trees. % are merged with the $\Nodejoin$ function which we describe in the following. %After a join of multiple node-trees, the node delays within the combined node-tree change. Therefore, the $\Nodejoin$ protocol's focus is to minimize the DFS's of the recalculation of the node parity and delays in the combined node-tree.

% %Let the parity $\nset$ be the number of syndrome-nodes in $\nset$, which is equivalent to the parity of the cluster of $\nset$.

% Since only odd clusters grow, only two types of clusters can merge: even with odd and odd with odd.

% For even node-trees, the PMW is undefined since a perfect matching would not be possible when merging with a non-trivial vertex. Consequently, PMW differences are undefined for even node-trees and do not need to be calculated.

% The second type of merger is between an even and an odd cluster. %The combined cluster is odd, and its growth is continued. Thus, its node delays must be computed.
% In order to define the new node-tree, we need to choose a new root. We consider two different options, choosing the root of the odd cluster and choosing the root of the even cluster:

% Let $\nset_o,\nset_e$ be the odd and even node-trees that are to be joined, and let $O\in \nset_o,E \in \nset_e$ be the nodes connected (\Cref{fig:inversion}\emph{a}).
% If $\nset_o$'s root becomes the root of the joined node-tree (\Cref{fig:inversion}\emph{b}), $E$ becomes a child node of $O$.
% Since $\nset_e$ contains an even number of syndrome-nodes, the node parities in $\nset_o$ do not change.
% Hence, the node parity DFS is only necessary in the subtree $\nset_e$, which now has $E$ as subroot potentially different than the original root.
% Furthermore, as the node PMW difference depends only on its own properties and its parent's, the node PMW DFS is also only required from node $E$ and within the subtree of $\nset_e$.
% Hence, only partial DFS's within the even cluster are required. % of the node-tree are precisely what was required, as the node parity and delays in $\nset_e$ were undefined.

% Alternatively, if $\nset_e$'s root becomes the root of the combined tree (\Cref{fig:inversion}\emph{c}), an odd number of syndrome-nodes are attached to $E$. In consequence the parities of the nodes on the path from $E$ to the root are changed. This choice of root requires the DFS's on the entire combined node-tree to calculate for node parities and PMW differences.

% For this reason, when merging node-trees we keep the root of the odd node-tree. We dub this strategy \textbf{Odd-Rooted Join}.

% In addition, a cluster can be subjected to multiple mergers within the same growth iteration, during which the parity of the merged cluster changes depending on the number of mergers and the parities of the clusters involved. The DFS's related to the parities and PMW differences should not be calculated directly after the joining of node-trees. %After all, it may be possible for the cluster to merge again such that the parities and delays become invalid.
% \textcolor{red}{To prevent redundant calculations, subroots of the even subtrees are stored in a list at the root of the node-tree (\Cref{fig:nodetypes}\emph{c}). When multiple mergers occur, the root node that stores the now redundant subroots is replaced by a new root with a new list of even subtrees subroots. If a cluster is selected for growth, we loop over the list of the subroots in the list at the new root node and initiate the DFS's from these subroots.} We call this the \textbf{Root List Replacement}.


%!TEX root=report.tex
\subsection{Time Series Analysis}

In time series analysis one attempts to find a model for predicting future observations
(this is different form the OLS analysis which tries to estimate the trends in the data, such as velocity and acceleration).
 Once such a model is found, it should be validated by analyzing the residuals. These residuals must be approximately white noise distributed \cite[p.~130]{time-series-analysis}, otherwise the model is invalid.

\subsubsection{ARIMA}

ARIMA is the name of the stochastic model used. The theory is very complex so it will not be discussed in detail here, instead we refer to \cite[p.~130]{time-series-analysis}.

In short ARIMA uses the previous values with some weight to predict future values. Which values are used is denoted by the following notation:
\begin{equation}
(p, d, q) \times (P, D, Q)_s
\end{equation}

$p$ is the highest lags of actual measurements used and $q$ is the maximum amount of lags in residuals used. The $d$ part indicates how many difference operators which should be used to transform the data. $(P, D, Q)$ is completely similar, but steps not by one lag but by $s$ lags ($s=$season length). This allows for seasons trends such as an yearly pattern.

\subsubsection{AutoCorrelation function (ACF) and Partial AutoCorrelation Function (PACF)}

ACF and PACF are measures of correlation between different time lags and are used to make a qualified guess about how the prediction model should look like.
For more detail on how these are estimatied, please see \cite[p.~146]{time-series-analysis}.

When ACF and PACF have been estimated, rules \cite[table~6.1]{time-series-analysis} for how the stochastic model should look can be applied.
Typically when dealing with complex models, this becomes an iterative and subjective process - different analysts might reach different models.
The iterative step consist of estemating parameters, then calculate ACF and PACF, interpet them, estimate parameters, ... , and so it continues until the ACF and PACF suggest that no more lag terms(weights) are needed.

\subsection{Ljung-Box test}

The null hypothesis for the Ljung-Box is ``The data is independently distributed''.
In the ARIMA case it is applied to model residuals; so in other words it tests whether residuals at different lags have any correlation with each other.
 Thus a low p-value means that the residuals aren't white noise.

The Ljung-Box is a $\chi^2$ test with the statistical value:
\begin{equation}
L = n \cdot (n + 2) \sum_{k=1}^h \frac{\hat{\rho}_k^2}{n - k}
\end{equation}

$n$ is the number of observations. The parameter $h$ is the highest lag in the ACF which should be considered. The $\chi^2$ statistics, have $h - (p + q + P + Q)$ degrees of freedom.


\documentclass[10pt,english]{article}

%\usepackage[T1]{fontenc}
%\usepackage[utf8]{inputenc}
\usepackage[a4paper,left=0.4in, right=0.4in,top=0.5in,bottom=0.6in]{geometry}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{subcaption}
\usepackage{epstopdf}
\usepackage{graphicx}
\usepackage{mathrsfs}
\usepackage{epstopdf}
\usepackage{mathtools}
\usepackage{textcomp}
\usepackage{graphicx}
\usepackage{xcolor}
\usepackage{epstopdf}
%\usepackage{multicol}
%\usepackage{tikz}
\usepackage{hyperref}

% code listing settings
\usepackage{listings}
\lstset{
    language=Python,
    basicstyle=\ttfamily\small,
    aboveskip={1.0\baselineskip},
    belowskip={1.0\baselineskip},
    columns=fixed,
    extendedchars=true,
    breaklines=true,
    tabsize=4,
    prebreak=\raisebox{0ex}[0ex][0ex]{\ensuremath{\hookleftarrow}},
    frame=lines,
    showtabs=false,
    showspaces=false,
    showstringspaces=false,
    keywordstyle=\color[rgb]{0.627,0.126,0.941},
    commentstyle=\color[rgb]{0.133,0.545,0.133},
    stringstyle=\color[rgb]{01,0,0},
    numbers=left,
    numberstyle=\small,
    stepnumber=1,
    numbersep=10pt,
    captionpos=t,
    escapeinside={\%*}{*)}
}
\author{Steven Porretta \\ Student \# 100756494}
\title{Assignment \#4}
\date{}
\begin{document}
    \maketitle
    Code can be found at \url{https://github.com/0xSteve/learning_automata_simulator}
    \section{Question 1}
        In this section we will examine some code snippets from the first question.
    \begin{lstlisting}[label={list:first},caption=Testbench code for the Tsetlin.]
    c2 = 0.7
    c1 = 0.05
    for i in range(0, 7):
        print("c1 = " + str(c1) + ", c2 = " + str(c2) + ", N = 13.")
        a = la.Tsetlin(13, 2, [c1, c2])
        a.simulate(50, 30001)
        b = ala.Tsetlin.stationary_probability_analytic([c1, c2], 13)
        c = ala.Tsetlin.number_of_states_estimate([c1, c2])
        print("Tsetlin P1(infinity) = " + str(b) + "(Analytic)")
        print("Tsetlin P1(infinity) = " + str(a.action_average[0]) + "(Simulated)")
        print("Tsetlin # of states required = " + str(c) + "(Estimate)")
        c1 += 0.1
        c1 = round(c1, 2)
\end{lstlisting}
This excerpt of code generates the entire quantity of required code for this question.  As can be seen in the following code-snippet.

\begin{lstlisting}[label={list:first},caption=Testbench output.]
c1 = 0.05, c2 = 0.7, N = 13.
Tsetlin P1(infinity) = 0.9999999890725503(Analytic)
Tsetlin P1(infinity) = 1.0(Simulated)
Tsetlin # of states required = 3(Estimate)
c1 = 0.15, c2 = 0.7, N = 13.
Tsetlin P1(infinity) = 0.9999778874501014(Analytic)
Tsetlin P1(infinity) = 1.0(Simulated)
Tsetlin # of states required = 4(Estimate)
c1 = 0.25, c2 = 0.7, N = 13.
Tsetlin P1(infinity) = 0.9990142374252533(Analytic)
Tsetlin P1(infinity) = 0.999998000067(Simulated)
Tsetlin # of states required = 6(Estimate)
c1 = 0.35, c2 = 0.7, N = 13.
Tsetlin P1(infinity) = 0.9865794150962881(Analytic)
Tsetlin P1(infinity) = 0.999797340089(Simulated)
Tsetlin # of states required = 9(Estimate)
c1 = 0.45, c2 = 0.7, N = 13.
Tsetlin P1(infinity) = 0.9151468144874294(Analytic)
Tsetlin P1(infinity) = 0.980783973868(Simulated)
Tsetlin # of states required = 18(Estimate)
c1 = 0.55, c2 = 0.7, N = 13.
Tsetlin P1(infinity) = 0.7453193640776348(Analytic)
Tsetlin P1(infinity) = 0.786514449518(Simulated)
Tsetlin # of states required = 0(Estimate)
c1 = 0.65, c2 = 0.7, N = 13.
Tsetlin P1(infinity) = 0.5680008401435711(Analytic)
Tsetlin P1(infinity) = 0.570874970834(Simulated)
Tsetlin # of states required = 0(Estimate)
\end{lstlisting}
Now that it is seen working as one would expect, considering rounding errors from python 3.6, it is time to take a look at the useful snippets of code governing the functionality of the Tsetlin, and Krylov automata.

\begin{lstlisting}[label={list:first},caption=Tsetlin core code.]
     def next_state_on_reward(self):
        '''Find the next state of the learner, given that the teacher
           rewarded.'''
        if (self.current_state mod (self.N / self.R) != 1):
            self.current_state -= 1

    def next_state_on_penalty(self):
        '''Find the next state of the learner, given that the teacher
           penalized.'''
        if(self.current_state mod (self.N / self.R) != 0):
            self.current_state += 1
        elif(self.current_state mod (self.N / self.R) == 0):
            # Don't really add states, just cycle through N, 2N, 4N, etc.
            if(self.current_state != self.N):
                a = (self.N / self.R) mod self.N
                self.current_state = a + self.current_state
            else:
                self.current_state = self.N / self.R

    # Determine the next state as the teacher.
    def environment_response(self):
        '''Determine the next state of the learner from the perspective
        of the teacher.'''
        response = uniform(0, 1)
        penalty_index = 1
        if(self.current_state <= self.n):
            self.actions[0] += 1
            penalty_index = 0
        else:
            self.actions[1] += 1

        if(response > self.c[penalty_index]):
            # Reward.
            self.next_state_on_reward()
        else:
            # Penalty.
            self.next_state_on_penalty()
\end{lstlisting}
The above is the core code of the Tsetlin machine, governing state translations and action choices.  Essentially, whenever it is in the range 1 to N, it chooses action $\alpha_1$ and $\alpha_2$ otherwise.  This code is essentially the same for the Krylov machine, which we will see in the next section.

\section{Question 2}
\begin{lstlisting}[label={list:first},caption=testbench for the Krylov 2-action.]
    for i in range(0, 7):
    print("c1 = " + str(c1) + ", c2 = " + str(c2) + ", N = 13.")
    a = la.Tsetlin(13, 2, [c1/2, c2/2])
    a.simulate(50, 30001)
    b = ala.Tsetlin.stationary_probability_analytic([c1, c2], 13)
    c = ala.Tsetlin.number_of_states_estimate([c1, c2])
    d = la.Krylov(13, 2, [c1, c2])
    d.simulate(10, 50000)
    e = ala.Tsetlin.stationary_probability_analytic([c1, c2], 13)
    f = ala.Tsetlin.number_of_states_estimate([c1, c2])
    print("Tsetlin P1(infinity) = " + str(b) + "(Analytic)")
    print("Tsetlin P1(infinity) = " + str(a.action_average[0]) + "(Simulated)")
    print("Tsetlin # of states required = " + str(c) + "(Estimate)")
    print("Krylov P1(infinity) = " + str(e) + "(Analytic)")
    print("Krylov P1(infinity) = " + str(d.action_average[0]) + "(Simulated)")
    print("Krylov # of states required = " + str(f) + "(Estimate)")
    c1 += 0.1
    c1 = round(c1, 2)
\end{lstlisting}
As can be seen from the code, this test bench looks very similar to the test bench of question 1, however, note the $c$ vector for the Tsetlin automaton is now $c_1 / 2 $, $c_2/2$.  As is expected, both automata behave in the same manner. as can be seen from the output code snippet.
\begin{lstlisting}[label={list:first},caption=testbench output for the Krylov 2-action.]
    c1 = 0.05, c2 = 0.7, N = 13.
    Tsetlin P1(infinity) = 0.9999999890725503(Analytic)
    Tsetlin P1(infinity) = 1.0(Simulated)
    Tsetlin # of states required = 3(Estimate)
    Krylov P1(infinity) = 0.9999999890725503(Analytic)
    Krylov P1(infinity) = 1.0(Simulated)
    Krylov # of states required = 3(Estimate)
    c1 = 0.15, c2 = 0.7, N = 13.
    Tsetlin P1(infinity) = 0.9999778874501014(Analytic)
    Tsetlin P1(infinity) = 0.999999333356(Simulated)
    Tsetlin # of states required = 4(Estimate)
    Krylov P1(infinity) = 0.9999778874501014(Analytic)
    Krylov P1(infinity) = 1.0(Simulated)
    Krylov # of states required = 4(Estimate)
    c1 = 0.25, c2 = 0.7, N = 13.
    Tsetlin P1(infinity) = 0.9990142374252533(Analytic)
    Tsetlin P1(infinity) = 1.0(Simulated)
    Tsetlin # of states required = 6(Estimate)
    Krylov P1(infinity) = 0.9990142374252533(Analytic)
    Krylov P1(infinity) = 1.0(Simulated)
    Krylov # of states required = 6(Estimate)
    c1 = 0.35, c2 = 0.7, N = 13.
    Tsetlin P1(infinity) = 0.9865794150962881(Analytic)
    Tsetlin P1(infinity) = 1.0(Simulated)
    Tsetlin # of states required = 9(Estimate)
    Krylov P1(infinity) = 0.9865794150962881(Analytic)
    Krylov P1(infinity) = 1.0(Simulated)
    Krylov # of states required = 9(Estimate)
    c1 = 0.45, c2 = 0.7, N = 13.
    Tsetlin P1(infinity) = 0.9151468144874294(Analytic)
    Tsetlin P1(infinity) = 1.0(Simulated)
    Tsetlin # of states required = 18(Estimate)
    Krylov P1(infinity) = 0.9151468144874294(Analytic)
    Krylov P1(infinity) = 1.0(Simulated)
    Krylov # of states required = 18(Estimate)
    c1 = 0.55, c2 = 0.7, N = 13.
    Tsetlin P1(infinity) = 0.7453193640776348(Analytic)
    Tsetlin P1(infinity) = 0.983473884204(Simulated)
    Tsetlin # of states required = 0(Estimate)
    Krylov P1(infinity) = 0.7453193640776348(Analytic)
    Krylov P1(infinity) = 0.999998(Simulated)
    Krylov # of states required = 0(Estimate)
    c1 = 0.65, c2 = 0.7, N = 13.
    Tsetlin P1(infinity) = 0.5680008401435711(Analytic)
    Tsetlin P1(infinity) = 0.670734975501(Simulated)
    Tsetlin # of states required = 0(Estimate)
    Krylov P1(infinity) = 0.5680008401435711(Analytic)
    Krylov P1(infinity) = 0.870114(Simulated)
    Krylov # of states required = 0(Estimate)
\end{lstlisting}
Observing the code for the Krylov machine, one notices that most of the code is inherited from the Tsetlin machine, except the state translations. It is incredible, that the only major distinction is that a penalty is treated as a penalty with 50\% probability and a success otherwise. Literally, all other code for the Krylov machine is inherited from the Tsetlin.
\clearpage
\begin{lstlisting}[label={list:first},caption=Krylov core code.]
def next_state_on_penalty(self):
        '''Find the next state of the learner, given that the teacher
           penalized.'''

        # If this number is greater than 0.5, then penalize the learner.
        is_penalty = uniform(0, 1)

        if(is_penalty >= 0.5):
            Tsetlin.next_state_on_penalty(self)
        else:
            Tsetlin.next_state_on_reward(self)

\end{lstlisting}

\section{Question 3}

First let us consider state changes in the $L_{R-I}$, since there really are no states, but instead just an interval, $\lbrace0, 1\rbrace$, of possibilities.  Consider the following code-snippet.

\begin{lstlisting}[label={list:first},caption=State Translation in the $L_{R-I}$  automaton.]
 def do_reward(self, action):
        if(action == 2):
            self.p1 = self.k_r * self.p1
        else:
            self.p1 = 1 - (self.k_r * self.p2)
        self.p2 = 1 - self.p1

    def do_penalty(self):
        pass
\end{lstlisting}
It can be seen from the above code-snippet, that when the environment rewards, an action is updated based on the action selected.  When the environment issues a penalty, nothing happens.  The python command \textit{pass} is command that simply does nothing, and is usually used for prototyping.  In this case, pass is included to explicitly show that a penalty does nothing.

From this automaton some interesting things can be observed.  First, let us consider the output of the testbench.py file.  Time complexity has been measured in terms of discrete steps instead of actual time spent in the processor. The physical time, does not represent the number of actions being computed, as many operations can be processed simulatneously, in the background of a system, the time is not accurate.  A corolary to this event is that the number of discrete calls to the program, accurately represent the time complexity, when each action is considered a time unit.  To better understand this, observe the following code snippet.
\begin{lstlisting}[label={list:first},caption=State Translation in the $L_{R-I}$  automaton.]
=============================================================
The optimal K_r value is: 0.74995
The optimal lambda_r value is: 0.25005
The accuracy for k_r = 0.74995 is: 0.963
The computation time in iterations is: 18
=============================================================
=============================================================
The optimal K_r value is: 0.7811937499999999
The optimal lambda_r value is: 0.2188062500000001
The accuracy for k_r = 0.7811937499999999 is: 0.958
The computation time in iterations is: 26
=============================================================
=============================================================
The optimal K_r value is: 0.7811937499999999
The optimal lambda_r value is: 0.2188062500000001
The accuracy for k_r = 0.7811937499999999 is: 0.965
The computation time in iterations is: 16
=============================================================
=============================================================
The optimal K_r value is: 0.8124374999999999
The optimal lambda_r value is: 0.18756250000000008
The accuracy for k_r = 0.8124374999999999 is: 0.953
The computation time in iterations is: 45
=============================================================
=============================================================
The optimal K_r value is: 0.874925
The optimal lambda_r value is: 0.12507500000000005
The accuracy for k_r = 0.874925 is: 0.976
The computation time in iterations is: 57
=============================================================
=============================================================
The optimal K_r value is: 0.90616875
The optimal lambda_r value is: 0.09383125000000003
The accuracy for k_r = 0.90616875 is: 0.969
The computation time in iterations is: 160
=============================================================
=============================================================
The optimal K_r value is: 0.96865625
The optimal lambda_r value is: 0.031343750000000004
The accuracy for k_r = 0.96865625 is: 0.977
The computation time in iterations is: 1298
=============================================================
\end{lstlisting}
It is observable that as $c1$ approaches $c2$ the number of operations increase exponentially.  It is also observable that as $n \to \infty$, then $\lambda_R \to 0$. Corolary: $c_1 \to c_2$, then $\lambda_R \to 0$ or $k_R \to \infty$.  To get a better understanding of this effect, observe the following graph, Figure \ref{blah}. It can be seen that $\lambda_R$ approaches $0$ as the time complexity goes to infinity.  This is representative of the difficulty of the system.  When the percent difference between penalties of choosing action 1 and action 2 approaches 0, then the time complexity approaches infinity, and the $\lambda_R$ required becomes an undesirably low value.
\begin{figure}[h!]
    \centering
    \includegraphics{plot.png}
    \caption{$\lambda_R$ Vs. $n$} \label{blah}
\end{figure}
\end{document}


As well as in logistic regression as in neural networks, minimizing(or maximizing depending on your setup) the cost function is a central problem, and can to some extent be stated as \textit{the} problem. Having a method which is both efficient and converges towards the minimum is, luckily, not a new problem in computer science. In the logistic regression optimization, three main methods will be utilized, while in the neural network we will focus on the stochastic conjugate gradient descent. For logistic regression we will use a learning rate optimized gradient descent method, and

Gradient descent measures the local gradient of the cost function, with regards to the weights $\bm{w}$(or $\beta$. Since the gradient goes in the direction of fastest increase, we will go in the opposite direction, i.e. negative gradient. We start by choosing random values for $\bm{w}$ (since our cost function is convex any choice should give correct results), calculate the gradient, update the $\bm{w}$ values, and do this iteratively until the algorithm converges to a minimum. The size of the steps is important, and is determined by the learning rate. If the learning rate is too small, we will need many iterations which is time consuming. However, if the learning rate is too high, we might overshoot and miss the minimum. One way to choose the learning rate is too let it depend on the size of the gradient. If the gradient is large, i.e a steep slope, the learning rate can be relatively high. When the gradient is small, the learning rate is also small.

\subsubsection{Gradient descent}
Gradient descent is set up in a general fashion as, utilizing an optimized learning rate $\eta_k$ find and a scaling parameter $\gamma$. The scaling parameter $\gamma$ is introduced in order to prevent results from blowing up.
\begin{algorithm}[H]
    \caption{Gradient descent.}
    \label{alg:gd}
    \begin{algorithmic}[1]
        \State Input: $\bm{X}$, $\bm{y}$, $\bm{w}_0$, $\eta$
        \State Set initial weights, $\bm{w} = \bm{w}_0$
        \State Set previous gradient with current, $\nabla_0 = \nabla_1$
        \While{$i < N_\mathrm{max}$ and $||\nabla\mathcal{C}(\bm{w})||<\varepsilon$}
            \State $z = \bm{X}\cdot \bm{w}$
            \State $p = \sigma(z)$
            \State $\nabla_0 = \nabla_1$, set previous gradient to current.
            \State $\nabla_1 = - \bm{X}^T \cdot (\bm{y} - \bm{p}) / \gamma + \lambda f_L(\bm{w}) / \gamma$
            \State $\bm{w}_0 = \bm{w}$
            \State Update learning parameter, $\eta_k$
            \State $\bm{w} = \bm{w}_0 - \eta_k \nabla_1$, update weights.
        \EndWhile
        \State Return $\bm{w}$
    \end{algorithmic}
\end{algorithm}
The full implementation with the optimized learning rate $\eta_k$ can be seen in the article \citet{optimal-learning-rate}.

\subsubsection{Stochastic gradient descent}
Stochastic gradient descent is similar to gradient descent, expect that we first randomly shuffle our data, then divide our data into mini batches, $N_\mathrm{mb}$. Then we run gradient descent on each of the mini batches, and take the average of the outputted gradients as our gradient descent step. We repeat this $N_\mathrm{epochs}$.
\begin{algorithm}[H]
    \caption{Stochastic gradient descent(SGD).}
    \label{alg:sgd}
    \begin{algorithmic}[1]
        \For $i_e$ in $N_\mathrm{epochs}$ epochs
            \State Shuffle data $\bm{X}_\mathrm{train}$, $\bm{y}_\mathrm{train}$
            \State Split into mini batches, $\bm{X}_\mathrm{train}$, $\bm{y}_\mathrm{train}$
            \For $i_\mathrm{mb}$ in mini batches,
                \State Perform gradient descent step, and retrieve $\nabla\bm{w}_{i_\mathrm{mb}}$
            \EndFor
            \State Take the average of the $\nabla\bm{w}_{i_\mathrm{mb}}$ and update the weight matrix $\bm{w}$.
        \EndFor
    \end{algorithmic}
\end{algorithm}

% To minimize this cost function we use gradient descent. \husk{Gradient descent measures the local gradient of the cost function, with regards to $\beta$ in our case. Since the gradient goes in the direction of fastest increase, we will go in the opposite direction, i.e. negative gradient. We start by choosing random values for $\beta$ (since our cost function is convex any choice should give correct results), calculate the gradient, update the $\beta$ values, and do this iteratively until the algorithm converges to a minimum. The size of the steps is important, and is determined by the learning rate. If the learning rate is too small, we will need many iterations which is time consuming. However, if the learning rate is too high, we might overshoot and miss the minimum. One way to choose the learning rate is too let it depend on the size of the gradient. If the gradient is large, i.e a steep slope, the learning rate can be relatively high. When the gradient is small, the learning rate is also small.}

% \sjekk{Returning to the logistic regression problem, the derivative of the cost function is
% \begin{equation}\label{eqT:diff_cost}
% \begin{split}
% \frac{\partial J(\beta)}{\partial \beta} &=-\frac{1}{N}\bm{X}^T(\bm{y}-\bm{p})\\
%  &= \frac{1}{N}\bm{X}^T(\bm{p}-\bm{y}),
% \end{split}
% \end{equation}
% where $\bm{X}$ is the $N\times(p+1)$ matrix of $x_i$ values, $\bm{p}$ is the vector of fitted probabilities with $i$th element $p(x_i;\beta)$ and $\bm{y}$ is the vector of $y_i$ values. The new $\beta$ using gradient descent is then\\
% $\beta_{new} = \beta_{old}\ -\ \frac{\partial J(\beta)}{\partial \beta}lr$, where $lr$ is the learning rate (step size).} \husk{This is done iteratively until we reach the set max iterations or $\frac{\partial J(\beta)}{\partial \beta}$ is within a given tolerance of zero.}

% \sjekk{Like we introduced Lasso and Ridge regression to avoid overfitting in Project 1, we can add a penalty term to the cost function in Equation \ref{eq:Geron_cost}. In our project we used two different penalties: $L1 = \lambda|\beta|$ and $L2 = \lambda||\beta||^2$. When fitting the model we need to include the derivatives of the penalty term in Equation \ref{eqT:diff_cost}.} \husk{The gradient with the penalty term is,
% \begin{equation}
% \begin{split}
% \frac{\partial J(\beta)}{\partial \beta} &=\frac{1}{N}\bm{X}^T(\bm{p}-\bm{y})\ + \lambda\cdot{sign(\beta)},\ for\ L1\ regularization\\
%  &\mathrm{or}\\
% \frac{\partial J(\beta)}{\partial \beta} &=\frac{1}{N}\bm{X}^T(\bm{p}-\bm{y})\ + \lambda\cdot{2\beta},\ for\ L2\ regularization.
% \end{split}
% \end{equation}
% }