\section{Model-based Analysis}

As described in the introduction, prefetching has the potential to enable visualization systems to respond with interactive latencies.  However, current approaches are designed for interfaces with limited options for user interaction~\cite{}; many have suggested that increasing the expressivity of the interface (e.g., the number of possible user actions) makes the prediction task significantly more challenging.  In this section, we seek to understand the minimum accuracy for model needs in order to support low user perceived latencies, as well as the conditions that affect this accuracy.

\subsection{The Model}

Our goal is to estimate the minimum prediction accuracy $\alpha$ needed in order to ensure that the user perceived latency $l_{user}$ is below a fixed threshold (e.g. 100ms~\cite{} or 500ms).
Let $T=0$ be the time.
We assume that the user will perform her request at $T=t$ where $t \ge 0$, and that the cost of answering a request consists of fetching and rendering the results.
$l_{user}$ is defined as the time between $t$ and the change reflected in the visualization.
Let $l_{net}$ be the latency to execute a request, transfer the results across the network, and render it on screen.   In a typical system, $l_{user} = l_{net}$, which can be undesirable when query processing or the network latency are high.
Prefetching allows the client to proactively answer a request at $T=0$ and store the results in a client cache.
If a future request accessed data in the cache, then it can take much less time $l_{cache} < l_{net}$ and result in a more responsive user perceived experience.

We now model the expected user perceived latency as $l_{cache} + max(0, l_{net} - t)$ if model accurately predicted the request, and $l_{net}$ if it mispredicted.  The $max()$ operation accounts for cases when the cost of a cache miss is larger than $t$:
$$l_{user} = (l_{cache} + max(0, l_{net}-t))\times \alpha + l_{net}\times(1-\alpha) $$
Rearranging the terms, we can derive the minimum prediction accuracy in order to maintain $l_{user}$.
%$$\alpha = \frac{l_{net} - l_{user}}{l_{net} - l_{cache}}$$
$$\alpha = \frac{l_{user} - l_{net}}{l_{cache} + max(0, l_{net}-t) - l_{net}}$$

Prior perception and interactive visualization literature have used both $100$ and $500$ millisecond response times as thresholds for what is considered interactive~\cite{liu2014effects}.   Similarly, we assume request latencies can vary from $0$ to $2$ seconds~\cite{}, which embodies latencies that stem from query processing, serialization, network latencies and client overheads.  Finally, it is typically challenging to make accurate predictions more than several hundred milliseconds into the future~\cite{pasqual2014mouse}; we vary the ratio $\frac{t}{l_{net}}$ between $0.5$ and $1$.   We use the above constants in our analysis below.

Figure~\ref{fig:model_base} plots the model accuracy for two commonly cited $l_{user}$ thresholds (100 and 500ms).  The x-axis varies $l_{net}$ and each line represents the percentage of the prefetching costs that the user will experience.  For instance, when $\frac{t}{l_{net}}=1$, the user initiates the request at $T=t=l_{net}$ and does not experience any of the prefetching costs, whereas when $\frac{t}{l_{net}}=0.5$, the prefetching request is only half complete by the time the user initiates her request at $T=0.5\times l_{net}$.  We use a vertical line to mark the network latency at $1$ second.

The first observation is that when $l_{net}$ is smaller than the threshold, the user perceived accuracy is independent of the model accuracy.  When the threshold is $100$, the minimum model accuracy quickly exceeds $80\%$ by even a request latency of $500$ms; although the minimum accuracy is lower under the $500$ms threshold, the accuracy converges to $70\%$.
It is challenging to guarantee such high user prediction accuracy for a single request---the accuracy in existing literature is slightly above $50\%$ for a constrained user interface with 9 interaction options~\cite{battle2016dynamic}.

\ewu{Argue that assuming $t < l_{net}$ is a reasonable assumption.}

\begin{figure}[ht]
	\centering
	\includegraphics[width=1\columnwidth]{figures/model_base}
 	\caption{Minimum $\alpha$ vs network latency (x-axis), $\frac{t}{l_{net}}$ ratios (lines), and two thresholds (facets).}
    \label{fig:model_base}
\end{figure}

\stitle{Vary Prefetch Concurrency}
Most modern data processing systems are able to execute multiple concurrent requests~\cite{ebenstein2016fluxquery,giannikis2012shareddb}, thus we now modify the model to vary the number of concurrent prefetch requests $N$.
The $(1-\alpha)^N$ term is the probability that none of the prefetch requests match the user's actual request at $T=t$:
%
$$l_{user} = (l_{cache} + max(0, l_{net} - t)\times (1-(1-\alpha)^N) + l_{net}\times(1-\alpha)^N $$
%
Rearranging the terms results in the following minimum prediction accuracy:
%
$$\alpha = 1 - \left(\frac{l_{cache}+max(0,l_{net}-t)-l_{user}}{max(0,l_{net}-t)-l_{net}}\right)^{1/N}$$

\begin{figure}[h]
	\centering
	\includegraphics[width=1\columnwidth]{figures/model_concurrency}
 	\caption{Minimum $\alpha$ vs network latency (x-axis), concurrency (lines), and two latency thresholds (facets).}
  \label{fig:model_concurrency}
\end{figure}

Figure~\ref{fig:model_concurrency} shows that increasing the number of concurrent requests has an immediate effect on $\alpha$ ($t=l_{net}$ in these plots).    With $N=20$, a prediction model need only be $12\%$ accurate to ensure an interactive latency of $l_{user}=100$, while ensuring  $l_{user}=500$ only requires an accuracy of $3.5\%$.  At the limit, sending concurrent requests for all possible queries (assuming the cost is $l_{net}$) means the user latency is independent of prediction accuracy.  These results suggest that increasing the concurrency is an effective counter balance to model accuracy.

\stitle{Network Latency Variance}
Figure~\ref{fig:model_std} simulates $l_{net}$ drawn from a gaussian distribution with standard deviation of $std \in \{0, 100, 500\}$ milliseconds.  We set $N=20$, $t=l_{net}$, and the y-axis is from 0 to $0.4$.  Although the request variance affects the required model accuracy when $l_{net}$ is low, the curves for each threshold converge to the same levels independent of $std$; this is simply because $l_{net}$, rather than $std$, is the dominant factor as it increases.

\begin{figure}[h]
	\centering
	\includegraphics[width=1\columnwidth]{figures/model_std}
 	\caption{Minimum $\alpha$ vs network latency (x-axis), concurrency (lines), and two latency thresholds (facets).}
  \label{fig:model_std}
\end{figure}

\begin{figure}[h]
	\centering
	\includegraphics[width=1\columnwidth]{figures/model_partial}
 	\caption{Required prediction accuracy as a function of network latency, under progressive conditions where partial responses are sufficient.}
    \label{fig:model_partial}
\end{figure}

\stitle{Progressive Results}
The visualization and database communities have argued for approximate visualizations that are slightly inaccurate but potentially much faster to compute.  Techniques such as Online Aggergation~\cite{control,wanderjoin} or streaming wavelet compression~\cite{} are considered {\it progressive} because they are able to immediately return (and render) approximate results that improve over time; thus providing a natural trade-off between latency and result quality.  Our final analysis studies progressive results that require a fraction of the full request latency to be ``sufficiently good''.
\ewu{Use sampling and immens arguments as basis for 25\% partial results.} We find that $5\%$ accuracy is enough to maintain user perceived latency of $100$ when $25\%$ of the results is sufficient; at threshold of $500$, the perceived latency is independent of the model accuracy when $l_{net}\le 2000$.

\subsection{Discussion}
We find that a careful combination of existing techniques (concurrency and progressive results) can dramatically reduce the required model accuracy to enable low user perceived latency---in some extreme cases when the threshold is $500$, the model accuracy can even be $0\%$.
\ewu{Argue that given  fixed network throughput between client and server, increasing concurrency beyond a certain point naturally requires partial results.  Thus the two go hand in hand.   We also see that the prediction model simply needs to be ``good enough'' with an accuracy of $5-10\%$.  When the number of possible interactions are limited to $10-20$ possible interactions, then even a random model is sufficient.  However in rich interfaces with potentially dozens or a hundred possible interactions, there may simply be too many choices to learn a good model.}


\chapter{Introduction}
Epic introduction goes here.\\

Then you can do things like say as shown in Fig.\ \ref{fig:sample}
or add citiations to epic papers like \cite{eliassen1951}. If we also decide
to add a table, we can add a reference to Table \ref{tab:lorenz}.

\begin{table}[t]
\caption[This caption appears in the LoT]{This is a sample table caption and
  table layout from the AMS LaTeX template file.Table from Lorenz (1963).}
\label{tab:lorenz}
\begin{center}
\begin{tabular}{ccccrrcrc}
\hline\hline
$N$ & $X$ & $Y$ & $Z$\\
\hline
 0000 & 0000 & 0010 & 0000 \\
 0005 & 0004 & 0012 & 0000 \\
 0010 & 0009 & 0020 & 0000 \\
 0015 & 0016 & 0036 & 0002 \\
 0020 & 0030 & 0066 & 0007 \\
 0025 & 0054 & 0115 & 0024 \\
\hline
\end{tabular}
\end{center}
\end{table}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.75\textwidth]{./chap1/fig/sample_fig}
    \caption[This caption is only in the LoF]{This is a sample figure with a
    sample caption.}
    \label{fig:sample}
\end{figure}


\subsection{Cardinality of the integers}

\subsubsection{Cardinality of integers}


\documentclass[abstract=true]{scrartcl}
%set letterpaper, 10pt for american
\usepackage{amsmath, amssymb, amsthm} %necessary math packages
\usepackage{verbatim} %if you need not to interpret latex
\usepackage{graphicx} %insert figures
\usepackage{booktabs} %nice tables
\usepackage[colorlinks]{hyperref} %links
\usepackage{xcolor} %definecolors
\usepackage{enumerate} %enumerate
\usepackage{natbib} %bib organization
\usepackage[affil-it]{authblk} %better maketitle
\usepackage{mdframed} %frame example
\usepackage{microtype} %small improvement
\usepackage{caption} %caption
\usepackage{style/matheusfarias} %my style
%\usepackage[body={4.8in,7.5in},
%    top=1.2in, left=1.8in]{geometry} %american page layout

\begin{document}
\title{Template}
\date{Fall, 2021}

\author{Matheus S. Farias%
  \thanks{E-mail address: \href{mailto:matheusfarias@g.harvard.edu}{matheusfarias@g.harvard.edu}}}
\affil{School of Engineering and Applied Sciences, Harvard University}

\maketitle

\begin{abstract}
    Here is the abstract with a citation \cite{shannon48}.
\end{abstract}

\tableofcontents

\section{First Section}
See Example \ref{ex:ref}.
\begin{example}
    First example.
    \label{ex:ref}
\end{example}

This is an \emph{example} of emphasize and here a \textbf{bold text}.

\subsection{First Subsection}

Table \ref{tab:ref} is the first one.
\begin{table}[h]
    \centering
    \caption{\label{tab:ref}First table.}
    \vspace{5pt}
    \begin{tabular}{@{}lll@{}}
    \toprule
    \textbf{Quadrant}    & $\mathbf{5}$ \textbf{m}    & $\mathbf{10}$ \textbf{m}    \\
    \midrule
    Quadrant I  & $8.63\%$ & $9.11\%$ \\
    Quadrant II & $5.63\%$ & $7.77\%$   \\
    \bottomrule
\end{tabular}
\end{table}

Figure \ref{fig:ref} is the first one.
\begin{figure}
    \centering
    \includegraphics[width = \textwidth]{logo.pdf}
    \caption{First figure.}
    \label{fig:ref}
\end{figure}

\bibliographystyle{apalike}
\bibliography{bib/mybib.bib} %my bib
\end{document}


\subsection{Plain objects typed by a subclass}
\label{subsec:library_of_transformations:instance_level_transformations:objects_of_subtype}

\begin{figure}
    \centering
    \begin{subfigure}{0.45\textwidth}
        \centering
        \includegraphics{images/05_library_of_transformations/03_instance_level_transformations/03_objects_of_subtype/subclass_instance.pdf}
        \caption{$Im_{Subclass}$ with one object identified as $x$}
        \label{fig:library_of_transformations:instance_level_transformations:objects_of_subtype:visualisation:ecore}
    \end{subfigure}
    \begin{subfigure}{0.45\textwidth}
        \centering
        \input{images/05_library_of_transformations/03_instance_level_transformations/03_objects_of_subtype/subclass_as_node_type_instance.tikz}
        \caption{$IG_{Subclass}$ with one node identified as $x$}
        \label{fig:library_of_transformations:instance_level_transformations:objects_of_subtype:visualisation:groove}
    \end{subfigure}
    \caption{Visualisation of the transformation of objects typed by a subclass}
    \label{fig:library_of_transformations:instance_level_transformations:objects_of_subtype:visualisation}
\end{figure}

In this section, the transformation of plain objects typed by a regular subclass is discussed. The corresponding type level transformation can be found in \cref{subsec:library_of_transformations:type_level_transformations:regular_subclasses}. This transformation is very similar to the the transformation of plain objects discussed in \cref{subsec:library_of_transformations:instance_level_transformations:plain_objects}. Like that transformation, it is possible to introduce an arbitrary amount of instances of the subclass introduced on the type level. First, the definition of the corresponding instance model is given.

\begin{defin}[Instance model $Im_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass}
Let $Im_{Subclass}$ be the instance model containing a set of objects $objects$ which are all typed by subclass $name$, which extends class $supertype$. Furthermore, an injective function $fid$ is defined which maps every object in the set to its corresponding identifier. $Im_{Subclass}$ is typed by $Tm_{Subclass}$ (\cref{defin:library_of_transformations:type_level_transformations:regular_subclasses:tmod_subclass}) and is defined as:
\begin{align*}
Object =\ &objects \\
\mathrm{ObjectClass} =\ & \begin{cases}
    (ob, name) & \mathrm{if }\ ob \in objects
\end{cases}\\
\mathrm{ObjectId} =\ & \begin{cases}
    (ob, fid(ob)) & \mathrm{if }\ ob \in objects
\end{cases}\\
\mathrm{FieldValue} =\ & \{\} \\
\mathrm{DefaultValue} =\ & \{\}
\end{align*}
\isabellelref{imod_subclass}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{defin}

\begin{thm}[Correctness of $Im_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass_correct}
$Im_{Subclass}$ (\cref{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass}) is a valid instance model in the sense of \cref{defin:formalisations:ecore_formalisation:instance_models:model_validity}.
\isabellelref{imod_subclass_correct}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{thm}

A visual representation of $Im_{Subclass}$ with $objects = \{ob\}$ and $fid(ob) = x$ can be seen in \cref{fig:library_of_transformations:instance_level_transformations:objects_of_subtype:visualisation:ecore}. Although this visualisation only shows one object, it is possible to have an arbitrary amount of objects in $Im_{Subclass}$, as long as they are all typed by the corresponding class introduced on the type level. In the visualisation, the identifier $.\type{NewType}$ is used for the class, in correspondence with \cref{fig:library_of_transformations:type_level_transformations:regular_subclasses:visualisation:ecore} The correctness proof of $Im_{Subclass}$ is trivial, and therefore not included here. The proof can be found as part of the Isabelle validated proofs.

In order to make composing transformation functions possible, $Im_{Subclass}$ should be compatible with the instance model it is combined with.

\begin{thm}[Correctness of $\mathrm{combine}(Im, Im_{Subclass})$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass_combine_correct}
Assume an instance model $Im$ that is valid in the sense of \cref{defin:formalisations:ecore_formalisation:instance_models:model_validity}. Then $Im$ is compatible with $Im_{Subclass}$ (in the sense of \cref{defin:transformation_framework:instance_models_and_instance_graphs:combining_instance_models:compatibility}) if:
\begin{itemize}
    \item All requirements of \cref{defin:library_of_transformations:type_level_transformations:regular_subclasses:tmod_subclass_combine_correct} are met, to ensure the combination of the corresponding type models is valid;
    \item All the objects in $Im_{Subclass}$ have an (internal and explicit) identity that is not yet used in $Im$;
    \item $Im$ is not typed by a type model that defines any fields for the $supertype$ class.
\end{itemize}
\isabellelref{imod_subclass_combine_correct}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{thm}

\begin{proof}
Use \cref{defin:transformation_framework:instance_models_and_instance_graphs:combining_instance_models:imod_combine_merge_correct}. It is possible to show that all assumptions hold. Now we have shown that $\mathrm{combine}(Im, Im_{Subclass})$ is consistent in the sense of \cref{defin:formalisations:ecore_formalisation:instance_models:model_validity}.
\end{proof}

Please note that in this case, it has been made explicit that the new objects introduced do not have any fields defined. This is by ensuring the supertype does not define any fields. The new subclass does not have fields itself, as it cannot have existed in the combined type model.

The definitions and theorems for introducing plain objects of regular subclasses within Ecore are now complete.

\subsubsection{Encoding as nodes}

As was the case with plain objects of regular classes, a possible encoding for plain objects of subclasses in Ecore is using nodes in GROOVE. Each node is typed by the node type that was introduced in $TG_{Subclass}$, and copies the identifiers set of the objects to the corresponding nodes. The encoding corresponding to $Im_{Subclass}$ can then be represented as $IG_{Subclass}$, defined in the following definition:

\begin{defin}[Instance graph $IG_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:ig_subclass_as_node_type}
Let $IG_{Subclass}$ be the instance graph with as nodes the converted $objects$ of $Im_{Subclass}$ (\cref{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass}). Furthermore, reuse the injective function $fid$ that maps every object to its identifier. Finally, use the node type $name$ introduced in $TG_{Subclass}$, that extends the $supertype$ node type. (\cref{defin:library_of_transformations:type_level_transformations:regular_classes:tg_class_as_node_type}). $IG_{Subclass}$ is defined typed by $TG_{Subclass}$ and is defined as:
\begin{align*}
N =\ & objects \\
E =\ & \{\} \\
\mathrm{ident} =\ & \begin{cases}
    (fid(ob), ob) & \mathrm{if }\ ob \in objects
\end{cases}
\end{align*}
with
\begin{align*}
\mathrm{type}_n =\ & \begin{cases}
    (ob, \mathrm{ns\_\!to\_\!list}(name)) & \mathrm{if }\ ob \in objects
\end{cases}
\end{align*}
\isabellelref{ig_subclass_as_node_type}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{defin}

\begin{thm}[Correctness of $IG_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:ig_subclass_as_node_type_correct}
$IG_{Subclass}$ (\cref{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:ig_subclass_as_node_type}) is a valid instance graph in the sense of \cref{defin:formalisations:groove_formalisation:instance_graphs:instance_graph_validity}.
\isabellelref{ig_subclass_as_node_type_correct}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{thm}

A visual representation of $IG_{Subclass}$ with $objects = \{ob\}$ and $fid(ob) = x$ can be seen in \cref{fig:library_of_transformations:instance_level_transformations:objects_of_subtype:visualisation:groove}. Like the previous example for the Ecore instance model, only one node is shown here, but multiple nodes can be introduced at once if there are more objects in the $objects$ set. As shown in the definition, the node type identified by $\type{NewType}$ is used to type all the nodes, in correspondence with \cref{fig:library_of_transformations:type_level_transformations:regular_subclasses:visualisation:groove}. The correctness proof of $IG_{Subclass}$ is trivial, and therefore not included here. The proof can be found as part of the Isabelle validated proofs.

In order to make composing transformation functions possible, $IG_{Subclass}$ should be compatible with the instance graph it is combined with.

\begin{thm}[Correctness of $\mathrm{combine}(IG, IG_{Subclass})$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:ig_subclass_as_node_type_combine_correct}
Assume an instance graph $IG$ that is valid in the sense of \cref{defin:formalisations:groove_formalisation:instance_graphs:instance_graph_validity}. Then $IG$ is compatible with $IG_{Subclass}$ (in the sense of \cref{defin:transformation_framework:instance_models_and_instance_graphs:combining_instance_graphs:compatibility}) if:
\begin{itemize}
    \item All requirements of \cref{defin:library_of_transformations:type_level_transformations:regular_subclasses:tg_subclass_as_node_type_combine_correct} are met, to ensure the combination of the corresponding type graphs is valid;
    \item All the nodes in $IG_{Subclass}$ have an (internal and explicit) identity that is not yet used in $IG$;
    \item There are no edge types from or to the $supertype$ node type, this includes edges from and to types that $supertype$ inherits from.
\end{itemize}
\isabellelref{ig_subclass_as_node_type_combine_correct}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{thm}

\begin{proof}
Use \cref{defin:transformation_framework:instance_models_and_instance_graphs:combining_instance_graphs:ig_combine_merge_correct}. It is possible to show that all assumptions hold. Now we have shown that $\mathrm{combine}(IG, IG_{Subclass})$ is valid in the sense of \cref{defin:formalisations:groove_formalisation:instance_graphs:instance_graph_validity}.
\end{proof}

Like the correctness for the Ecore instance model, validity is guaranteed here by assuming there exist no edge types from and to the $supertype$ node type.

The next definitions define the transformation function from $Im_{Subclass}$ to $IG_{Subclass}$:

\begin{defin}[Transformation function $f_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass_to_ig_subclass_as_node_type}
The transformation function $f_{Subclass}(Im)$ is defined as:
\begin{align*}
N =\ & Object_{Im} \\
E =\ & \{\} \\
\mathrm{ident} =\ & \begin{cases}
    (fid(ob), ob) & \mathrm{if }\ ob \in Object_{Im}
\end{cases}
\end{align*}
with
\begin{align*}
\mathrm{type}_n =\ & \begin{cases}
    (ob, \mathrm{ns\_\!to\_\!list}(name)) & \mathrm{if }\ ob \in Object_{Im}
\end{cases}
\end{align*}
\isabellelref{imod_subclass_to_ig_subclass_as_node_type}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{defin}

\begin{thm}[Correctness of $f_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass_to_ig_subclass_as_node_type_func}
$f_{Subclass}(Im)$ (\cref{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:imod_subclass_to_ig_subclass_as_node_type}) is a valid transformation function in the sense of \cref{defin:transformation_framework:instance_models_and_instance_graphs:combining_transformation_functions:transformation_function_instance_model_instance_graph} transforming $Im_{Subclass}$ into $IG_{Subclass}$.
\isabellelref{imod_subclass_to_ig_subclass_as_node_type_func}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{thm}

The proof of the correctness of $f_{Subclass}$ will not be included here. Instead, it can be found in the validated Isabelle theories. The proof is quite trivial, as extending $Im$ can only add extra objects, but not remove the existing ones.

Finally, to complete the transformation, the transformation function that transforms $IG_{Subclass}$ into $Im_{Subclass}$ is defined:

\begin{defin}[Transformation function $f'_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:ig_subclass_as_node_type_to_imod_subclass}
The transformation function $f'_{Subclass}(IG)$ is defined as:
\begin{align*}
Object =\ &N_{IG} \\
\mathrm{ObjectClass} =\ & \begin{cases}
    (ob, name) & \mathrm{if }\ ob \in N_{IG}
\end{cases}\\
\mathrm{ObjectId} =\ & \begin{cases}
    (ob, fid(ob)) & \mathrm{if }\ ob \in N_{IG}
\end{cases}\\
\mathrm{FieldValue} =\ & \{\} \\
\mathrm{DefaultValue} =\ & \{\}
\end{align*}
\isabellelref{ig_subclass_as_node_type_to_imod_subclass}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{defin}

\begin{thm}[Correctness of $f'_{Subclass}$]
\label{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:ig_subclass_as_node_type_to_imod_subclass_func}
$f'_{Subclass}(IG)$ (\cref{defin:library_of_transformations:instance_level_transformations:objects_of_subtype:ig_subclass_as_node_type_to_imod_subclass}) is a valid transformation function in the sense of \cref{defin:transformation_framework:instance_models_and_instance_graphs:combining_transformation_functions:transformation_function_instance_graph_instance_model} transforming $IG_{Subclass}$ into $Im_{Subclass}$.
\isabellelref{ig_subclass_as_node_type_to_imod_subclass_func}{Ecore-GROOVE-Mapping-Library.SubclassInstance}
\end{thm}

Once more, the correctness proof is not included here but can be found in the validated Isabelle proofs of this thesis.


\input{permve-ntnu-latex-assignment.tex}

\usepackage{float}

\title{
\normalfont \normalsize
\textsc{Norwegian University of Science and Technology\\IT3105 -- Artificial Intelligence Programming}
\horrule{0.5pt} \\[0.4cm]
\huge Module 4:\\ Using Minimax with\\ Alpha-Beta Pruning to play 2048\\
\horrule{2pt} \\[0.5cm]
}

\author{Per Magnus Veierland\\permve@stud.ntnu.no}

\date{\normalsize\today}

\begin{document}

\fancyfoot[C]{}
\maketitle

\newpage
\fancyfoot[C]{\thepage~of~\pageref{LastPage}} % Page numbering for right footer
\setcounter{page}{1}

\section*{Branching factor}

The aim of module~4 is to achieve a winning score in the game \textsc{2048}. The game consists of a two-dimensional $4\times 4$ grid where each tile in the grid is either empty or has a number between $2^1$ and $2^{17}$ (inclusive). An upper bound of the game's branching factor can be found by multiplying the number of possible player moves~(4) by the maximum number of empty tiles~(16) by the number of possible tile spawns~(2):

\begin{displaymath}
b_{\textit{upper}} = 4 \cdot 16 \cdot 2 = 128
\end{displaymath}

\section*{State representation}

Given such a large branching factor for each player move it is evident that a large number of states must be evaluated to perform even a shallow search involving a low number of moves. To enable a large number of states to be considered an efficient state representation scheme is beneficial.

Even if the theoretical maximum tile is $2^{17}$, the largest tile required to win the game, 2048, is only $2^{11}$. Given that the number 1 is not present in the game, it is possible to represent the empty tile and tiles for powers of two between $2^1$ and $2^{15}$ by using 4~bits to represent each tile. With 4~bits reserved for each tile, all 16~tiles can be represented by a single 64-bit integer. Using a commonly available 64-bit processor architecture, each board state can be held in a single CPU register.

The resulting representation with example values is shown in Figure~\ref{figure:represent} showing tile values and the corresponding bit values.

\begin{figure}[H]
\centering
\includegraphics[scale=1.0]{images/2048_state}
\caption{64-bit integer state representation}
\label{figure:represent}
\end{figure}

\section*{Lookup tables}

Another benefit from the compact state representation is that it allows for efficient use of lookup tables. The logic necessary to compute the successor state when given a move requires loops and branches. The effect of performing a move is also local to each row or column, depending on the direction of the move. Each row or column can then be considered in isolation and depend on the exact same logic to produce their successor. The number of possible rows or columns is $2^{16}=65536$, so representing a lookup table for a single direction will take $2^{16}*8=524288$~bytes, or $0.52$~MB. By using four lookup tables, one for each direction, successor states can be evaluated by performing four lookups, one for each row or column.

Lookup tables can also be used to evaluate both board heuristics and board scores. The board heuristic may be a costly function to evaluate, making it very beneficial to pre-compute. However, unlike the state transitions which can be considered as functions considering each row and column individually, the heuristic function evaluates the entire board state. Since a lookup table containing entries for all states would be $2^{64}*8=18.45$~exabytes, this is currently infeasible. However by imposing the limitation that the heuristic function can only consider a single row or column at a time, only a single lookup table with $2^{16}$ entries is needed. This limits the information available to the heuristic function, but the implemented heuristic function is still able to provide sufficient guidance to the search. When evaluating the heuristic value for a state; 8~lookups are performed, one for each row and column, and the result is summed. The advantage of this approach is that the heuristic evaluation can use very costly operations in the pre-computation, since its effective use consists only of lookups.

Evaluating the score of a given state is also done using lookup tables. This is a simple convenience such that the scoring of a state can be performed using four row lookups.

\section*{Expectimax}

The algorithm used to solve \textsc{2048} is \textit{Expectimax} with pruning based on a probability threshold and a transposition table. \textit{Expectimax} was chosen over \textit{Minimax} since the ``adversary'' is stochastic and not actively minimizing the expected outcome. The code centers around the two functions \texttt{score\_player\_node} and \texttt{score\_chance\_node} which calls each other recursively with successive board evaluations, until either the search is pruned due to the node having a probability beneath the probability threshold, or because the depth exceeds the depth threshold, or if the current board state is found in the transposition table.

When traversing downwards into the search, the current probability is tracked such that nodes falling beneath a probability threshold is pruned and heuristically evaluated instead of diving further down their state space. This is done to avoid spending time evaluating unlikely cases, such as when a $4$-tile spawns three times in a row. Using pruning based on probability is very effective, and the pruning rate can typically be 60-80\% without harming the effectiveness of the algorithm.

In addition to pruning based on probability, a configurable depth limit is also imposed to control the algorithm.

When evaluating the game state space, there will be many overlaps where different moves and/or different chance scenarios will end up resulting in the same game state. To improve the algorithm, a transposition table is used where evaluated states in chance nodes are added with their score and depth to a transposition table. If later on, a node is evaluated with a board state which exists in the transposition table, and where the stored depth is greater or equal to the current depth of the new search node, meaning that the current node's state has been evaluated at least as deep as what what the current search would do; the existing score stored in the transposition table is used instead of evaluating the node's state space again. The transposition table typically has a hit rate of 60-70\%.

\section*{Heuristic}

The \textit{Expectimax} algorithm requires a heuristic to evaluate the quality of a given state as it is unfeasible to fully evaluate the full state space for a given node due to a high branching ratio and a typical ``win'' state after approximately 2000~plies.

The chosen heuristic depends on four metrics and six constants. Since a lookup approach is used with the heuristic, the heuristic function can only evaluate a single row or column in isolation. For a given row the number of direct merges, $M_1$, secondary merges, $M_2$, and ternary merges, $M_3$, are recorded. E.g. for a row ``2 2 4 8'' the number of direct merges possible is one (``2'' with ``2'' resulting in ``4''), and the number of secondary merges is one (the resulting ``4'' with the existing ``4'', resulting in ``8''). The number of ternary merges is one (the resulting ``8'' with the existing ``8''). The last metric tracked is the number of ``bad tiles''; $B$. This is simply all non-zero tiles in a row which are not part of a merge.

The first constant, $k_L$, is a base used to distinguish losing states from non-losing states. Since losing states, i.e. states where there are no possible moves, are evaluated to zero -- the heuristic should result in a positive number for any non-losing state.

The three next constants is used to value the different merges, while the constants $k_{B1}$ and $k_{B2}$ is used to adjust the value of ``bad tiles''. A base constant is raised to the power of the number of bad tiles; penalizing each additional ``bad tile'' increasingly.

\begin{displaymath}
H(\textit{row}) = k_L + k_1 \cdot M_1 + k_2 \cdot M_2 + k_3 \cdot M_3 - \frac{{k_{B1}}^{B}}{k_{B2}}
\end{displaymath}

More sophisticated heuristics can be constructed which takes into account the monotonicity of the board state, the values of the tiles involved, and the number of empty tiles. However, the implemented heuristic performs well for shallow depths and is able to achieve a ``2048'' tile on ~80\% of runs with a maximum depth of 4 in under 3 minutes. It also regularly accomplishes the ``4096'' tile; but has not been able to score ``8196'', despite attempts with a maximum depth of 10.

The following constants have been found by thought, trial and error. Achieving good constants for the heuristic will likely involve a meta-search using e.g. a gradient descent approach to tune the algorithm.

\begin{displaymath}
k_L = 100, k_1 = 10, k_2 = 5, k_3 = 2.5, k_{B1} = 5, k_{B2} = 0.08
\end{displaymath}

\end{document}


\section{Tests}
	Tests have been taken from the ones used in laboratories and from TSPLIB, a library of sample instances for the TSP from various sources and of various types. A total of 13 instances, ranging in size from 12 to 100, have been tested 10 times for each method. Higher size tests should have been included but ILP solver already starts to slow at 100 nodes and the time to test was not so much.

	\subsection{ILP}
		Results obtained testing the ILP solver gives expected results, as it always finds optimal solution, and execution times grow with problem size. Average solution times on 10 runs are displayed in Table 1.\\

		\begin{center}
			\begin{tabular}{ | l | c | r | }
				\hline
				\multicolumn{1}{|c|}{Test} & Size & \multicolumn{1}{|c|}{Avg Time (s)}\\
				\hline
				tsp12.dat		& 12		& 0,133\\
				tsp15.dat		& 15		& 0,062\\
				tsp17.dat		& 17		& 0,162\\
				tsp26.dat		& 26		& 0,298\\
				tsp29a.dat		& 29		& 0,633\\
				tsp29b.dat		& 29		& 0,668\\
				tsp42.dat		& 42		& 2,462\\
				tsp48a.dat		& 48		& 8,371\\
				tsp48b.dat		& 48		& 10,357\\
				tsp48c.dat		& 48		& 2,275\\
				tsp60.dat		& 60		& 27,066\\
				tsp100a.dat	& 100	& 101,527\\
				tsp100b.dat	& 100	& 128,637\\
				\hline
			\end{tabular}
		\end{center}
		\captionof{table}{ILP tests results}
		\vspace{1em}
		We can see that while solving small instances is very efficient, bigger ones require more time and going from size 60 to 100 (5/3 ratio) it gets way slower (more than 5 times), and the trend continue growing in size. This is due to exponential growth of the solution space the solver has to explore to find and prove optimality.

	\subsection{Genetic Algorithm}
		Genetic algorithms, due to their random nature, give different results at each execution, so an average of 10 runs is considered. Obtained solutions and computation time are also greatly related to set parameters, which are reported for each instance. Parameters don't differ so much, but more generations and bigger populations are used for bigger instances, and to keep computational times low populations evaluated are reduced. Results obtained with specified tests are reported in Table 2.

		\begin{center}
			\hspace*{-1.2cm}
			\begin{tabular}{ | l | c | r | r | c | c | c | c |}
				\hline
				\multicolumn{1}{|c}{\multirow{2}{*}{Test}} & \multicolumn{1}{|c|}{\multirow{2}{*}{Size}} & Average & Avg. opt. & Opt. & \multicolumn{3}{c|}{Parameters} \\
				\cline{6-8}
				& & \multicolumn{1}{c}{time} & \multicolumn{1}{|c|}{distance} & found & Populations & Pop. size & Generations\\
				\hline
				tsp12.dat		& 12		& 0,405	& 0,00\%	& 10/10	& 5	& 500	& 200\\
				tsp15.dat		& 15		& 0,401	& 0,00\%	& 10/10	& 5	& 500	& 200\\
				tsp17.dat		& 17		& 0,502	& 0,02\%	& 9/10	& 5	& 500	& 250\\
				tsp26.dat		& 26		& 1,037	& 0,36\%	& 9/10	& 10	& 500	& 250\\
				tsp29a.dat		& 29		& 1,038	& 0,68\%	& 2/10	& 10	& 500	& 250\\
				tsp29b.dat		& 29		& 1,049	& 0,40\%	& 3/10	& 10	& 500	& 250\\
				tsp42.dat		& 42		& 1,246	& 0,75\%	& 2/10	& 10	& 500	& 300\\
				tsp48a.dat		& 48		& 1,779	& 2,30\%	& 0/10	& 7	& 1000	& 300\\
				tsp48b.dat		& 48		& 1,777	& 1,65\%	& 0/10	& 7	& 1000	& 300\\
				tsp48c.dat		& 48		& 1,859	& 2,91\%	& 0/10	& 7	& 1000	& 300\\
				tsp60.dat		& 60		& 1,802	& 3,19\%	& 0/10	& 7	& 1000	& 300\\
				tsp100a.dat	& 100	& 3,053	& 7,46\%	& 0/10	& 7	& 1000	& 500\\
				tsp100b.dat	& 100	& 3,083	& 6,33\%	& 0/10	& 7	& 1000	& 500\\
				\hline
			\end{tabular}
		\end{center}
		\captionof{table}{Genetic algorithm tests results}
		\vspace{1em}
		Note that computation time is highly dependent on set parameters, so if time is less important parameters could be increased to get better solutions. A direct consequence is that with genetic algorithms it's possible to get the desired precision/efficiency ratio. However as GAs don't provide an optimality proof, if an optimal solution is found, all remaining computation is executed anyway.

		Another thing to consider is that, because of GAs random nature, solution effectiveness might vary a lot from run to run. This is however not the case as the solutions found, even in bigger instances, are not more than 2\% over the average. This means that implemented genetic algorithm is reliable even if random.

	\subsection{Tests details}
		Here have been shown the most important results, all recorded data can be found in this spreadsheet:

		\href{https://docs.google.com/spreadsheets/d/1wJ98av-soxw_okteJ7X8zMJUfesh_16ECjA9uRbP8Hg/edit?usp=sharing}{\color{blue}{https://docs.google.com/spreadsheets/d/1wJ98av-soxw\_okteJ7X8zMJUfesh\_16ECjA9uRbP8Hg/edit?usp=sharing}}


\documentclass[parskip=half]{scrartcl}

\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage[textsize=tiny]{todonotes}
\graphicspath {{images/}}
\hypersetup{
    colorlinks=true,
    citecolor=blue
}

\begin{document}

\title{CS698 (Winter 2017) Notes}
\author{Vineet John (v2john@uwaterloo.ca)}
\date{\today}
\maketitle

\section{Supervised Learning}\label{supervised-learning}

    Classification: domain of output is categorical (discrete)\\
    Regression: domain of output is numerical (continuous)

    Input domain can be either discrete or continuous for both.

    Examples:\\
    Appln - Domain - Range - Type of Problem\\
    Spam Identification - Words/Characters - spam/ham - classification\\
    Stock Prices - Prev price/news - predicted value - regression\\
    Speech recognition - Audio signal - words - classification\\
    Digit recognition - Image - Digits - classification\\
    Housing value - Area/bedrooms/baths - market value - regression\\
    Weather prediction - Local weather/Prev weather - Temp/Rain - Regression/Classificn

    Consider space of probably hypotheses\\
    Approximation of a best solution is done within this space\\
    A good hypothesis generalizes well

    When is it not possible to find a consistent hypothesis

    \begin{itemize}
        \item
        Insufficient Hypothesis space
        \item
        Noise in the training data
    \end{itemize}

    Tradeoff: Expressiveness vs complexity in finding a hypothesis - in the
    hypothesis space

\section{Statistical Learning}\label{statistical-learning}

    \begin{itemize}
        \item
        Joint probabilities
        \item
        Marginalization - sum up a join probability over a variable to express
        it in terms of another variable
        \item
        Conditional probabilities i.e A, given B
        \item
        Conditional probabilies need not sum up to 1, since the idea of a
        context is introduced.
    \end{itemize}

    \subsection{Bayes Rule}\label{bayes-rule}

    \begin{itemize}
        \item
        Posterior probabiliy = prediction, Prior = hypothesis, evidence =
        input variables, likelihood = probability of evidence given the prior
        hypothesis
        \item
        Weigh each hypothesis and use all hypothesis to get an avg prediction
        \item
        No overfitting, all hypotheses considered and weighed according to
        their probabilities
    \end{itemize}

\section{Logistic Regression}
\label{logistic-regression}

    \subsection{Overview - Logistic Regression} % (fold)
    \label{sub:overview_logistic_regression}
    \begin{itemize}
        \item
        In Mixture of Gaussians, we learn the prior and the likelihood and used Bayesian learning to compute the posterior.
        \item
        Unlike Mixture of Gaussians, we learn the posterior directly by maximum likelihood
    \end{itemize}

    % subsection overview_logistic_regression (end)

    \subsection{Problem - Singular Hessian}\label{problem---singular-hessian}

    \begin{itemize}
        \item
        Attempt is to make the ridge of the sigmoid function as shape as
        possible
        \item
        When weights tend to negative infinity, the sigmoid function tends to
        zero
        \item
        As the sigmoid is a component of the hessian, the hessian diagonal
        then is comprised of zeoes.
        \item
        This could result in the Hessian becoming a singular matrix
    \end{itemize}

    \subsection{Solutions}\label{solutions}

    \begin{itemize}
        \item
        Adding a penalty term to the loss function
        \item
        Boils down to adding a magnified identity matrix to the Hessian
    \end{itemize}

\section{Generalized Linear models}\label{generalized-linear-models}

    \begin{itemize}
        \item
        Mapping the input space into a lower dimensional space
        \item
        Perform linear regression of the dataset
        \item
        Polynomial class of functions used as basis functions.
        \item
        Basis functions are used for the mapping
        \item
        The input space x is mapped as $\Phi(x)$
    \end{itemize}

\newpage

\section{Perceptron \& Neural Nets}\label{perceptron-neural-nets}

    \begin{itemize}
        \item
        Output in the form of a non-linear function
        \item
        A neuron produces a single output, which may, in turn, be fed to
        multiple other neurons.
        \item
        Types of activation function:

        \begin{itemize}
            \item
            Threshold: Step function, non-linear
            \item
            Sigmoid: Smoother version of the step function
        \end{itemize}
        \item
        Nets can be trained by tuning the weights.
    \end{itemize}

    \section{Threshold Perceptron Learning}\label{threshold-perceptron-learning}

    \begin{itemize}
        \item
        Add/subtract the input vector to/from the weights if the predicted
        value doesn't match the target.
        \item
        Eta determines the magnitude of the step-size
        \item
        Gradient descent: Assuming threshold perceptron, will converge if data
        is linearly separable.
        \item
        If the data isn't linearly separable

        \begin{itemize}
            \item
            The algorithm will never converge
            \item
            The step vector keeps being reset back \& forth
        \end{itemize}
    \end{itemize}

    \section{Sigmoid Perceptron}\label{sigmoid-perceptron}

    \begin{itemize}
        \item
        Similar to logistic regression in that it computes a sigmoid
        separator.
        \item
        Multiple sigmoid hyperplanes could be used to simulate a ridge-like
        hyperplane. Further compounding can lead to simulating a bump-like
        hyperplane.
    \end{itemize}

    \section{Multi-layer neural networks}\label{multi-layer-neural-networks}

    \begin{itemize}
        \item
        Mapping inputs into a linear feature space, and then applying linear
        learning algorithms like linear regression, threshold functions,
        mixtures of gaussians and logistic regression.
        \item
        A neural net could be used to approximately just about any function
        using a combination of neurons.
    \end{itemize}

    \subsection{Feedforward neural
    network}\label{feedforward-neural-network}

    \begin{itemize}
        \item
        Adaptive non-linear basis function = function chaining
    \end{itemize}

    \section{Multi-Layer Neural Nets}\label{multi-layer-neural-nets}

    \begin{itemize}
        \item
        Sequential gradient descent used, similar to a gradient descent
        approach for linear regression.
        \item
        Backpropagation permits simultaneous calculation of weights for the
        perceptrons in a multi-layered neural network.
        \item
        Involves a forward and a backward phase.
        \item
        The forward phase calculates the output $z_j$ for each unit $j$
        \item
        The backward phase propagates errors to the input units, error denoted
        by $\delta_j$ at each unit $j$
    \end{itemize}

    Neural net = composition of functions
    \begin{itemize}
        \item
        Decompose the derivative of the error function, using the chain rule
        \item
        $w_{ij}$ is the weight metric from unit i to unit j
        \item
        $\delta_{ij}$ is the error metric from unit j to unit i
        \item
        Example using tanh as a hypothesis function to calculate the
        activation weights
    \end{itemize}

\newpage

\section{Kernel methods}\label{kernel-methods}

    \begin{itemize}
        \item
        Avoids the complexity of convex optimization
        \item
        Kernel Trick - Related to duality
        \item
        Derived from the dot product of the input and it's embedding into a
        new feature space.
    \end{itemize}

    \subsection{Generalized linear
    functions}\label{generalized-linear-functions}

    \begin{itemize}
        \item
        Fixed non-linear basis functions
        \item
        Limited hypothesis space
        \item
        Easier to optimize
    \end{itemize}

    \subsection{Neural networks}\label{neural-networks}

    \begin{itemize}
        \item
        Adaptive non-linear basis functions
        \item
        Rich hypothesis space
        \item
        Difficult to opitimize (by virtue of being non-convex)
    \end{itemize}

    \section{Kernel methods (contd.)}\label{kernel-methods-contd.}

    \begin{itemize}
        \item
        Rather than trying to compute the function $\Phi$, the
        objective is to be able to start with an arbitrary Gram matrix.
        \item
        Slide 10 - dimensionality of each vector is 2. The square function is
        the new space than the gram matrix can emulate by computing the dot
        product of the x vector and the z vector.
        \item
        Gaussian kernel is unrelated to the PDF of a gaussian distribution.
        \item
        Primal solution - scales with the number of basis functions
        \item
        Dual solution - scales with data
        \item
        A valid kernel must has non-zero eigenvalues, if it's being guessed rather than computed
    \end{itemize}

\newpage

\section{Gaussian Processes} % (fold)
\label{sec:gaussian_processes}
\begin{itemize}
    \item
    Uncertainty in the weights for a linear regressor can be translated into a corresponding uncertainty for the output space. Both are modeled as Gaussian distributions.
    \item
    Continuous functions implies that there are infinite Gaussian distributions that describe the output space.
    \item
    Kernel function is used as a covariance matrix to indicate that the Gaussian distribution of the output space is very similar for points that are closely-correlated.
    \item
    Describes a distribution over each point in the target function.
    \item
    Bayesian Learning used to learn a posterior distribution of what the Gausian processes look like.
    \item
    Function space view: Scales with the amount of data rather than the number of dimensions. Useful for problems where the dimensions might be infinitely many. The weight space view applies to the opposite scenario.
    \item
    Example of Gait Optimization shown in Figure \ref{fig:gait_optimization}.
    \begin{itemize}
        \item
        Experimentation done using new feature vectors (denoted by $X_{new}$).
        \item
        Minimizes the magnitude of the Gaussian deviation.
        \item
        Iteratively run new experiments with different values of $X_{new}$ to converge to a result.
    \end{itemize}
    \begin{figure}[th]
    \centering
    \includegraphics[width=0.6\textwidth]{gait_optimization}
    \caption{Gait Optimization}
    \label{fig:gait_optimization}
    \end{figure}
\end{itemize}

% section gaussian_processes (end)

\newpage

\section{Support Vector Machines} % (fold)
\label{sec:support_vector_machines}

    \subsection{Overview - SVM} % (fold)
    \label{sub:overview_svm}

    \begin{itemize}
        \item
        Also uses kernel trick, but the kernel uses a subset of the data.
        \item
        SVM attempt to maximize the margin between the support vectors and the hyperplane. Ref Figure \ref{fig:svm_general}
        \begin{figure}[th]
        \centering
        \includegraphics[width=0.4\textwidth]{svm_general}
        \caption{SVM Diagram}
        \label{fig:svm_general}
        \end{figure}
        \item
        SVMs are also called max-margin classifiers. $a > b$ in Figure \ref{fig:max_margin}
        \begin{figure}[th]
        \centering
        \includegraphics[width=0.4\textwidth]{max_margin}
        \caption{Max margin}
        \label{fig:max_margin}
        \end{figure}
        \item
        $w^T \Phi(x) = 0$ represents the linear separator
        \item
        Dividing by $\Vert w\Vert$ ensures that the distance to the linear separator is normalized in terms of unit length.
        \item
        The problem is made tractable in terms of the data by relying only on the support vectors while performing the quadratic optimization.
        \item
        The quadrative objective is reformulated as a dual representation in $\Phi(x)$
        \item
        The result is an optimization objective in the auxilliary variable $a$, which is the weight for the penalty term. $a_n = 0$ for any data-points apart from the support vectors.
    \end{itemize}
    % subsection overview_svm (end)

    \subsection{Overlapping classes and slack variables} % (fold)
    \label{sub:overlapping_classes_and_slack_variables}

    \begin{itemize}
        \item $\epsilon$ parameterized the soft-margin which is an allowance for some data point to lie beyond the margin.
        \item If misclassfied, $\epsilon > 1$.\\ If within the soft margin, $1 > \epsilon > 0$.\\ If on the margin, $\epsilon = 0$\\
        Refer to figure \ref{fig:soft_margins}
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.4\textwidth]{soft_margins.png}
            \caption{Soft Margins - SVM}
            \label{fig:soft_margins}
        \end{figure}
        \item $C$ controls how much to weigh vanilla SVMs over the new optimization problem for soft margins. Hard margin classifier implies that we're assuming the data is linearly separable.
        \item For the new optimization problem, earlier only support vectors were considered, but now, we consider points on the margin, points within the margin as well as misclassified points.
        \item Computation complexity increases as the number of vectors increase.
    \end{itemize}

    % subsection overlapping_classes_and_slack_variables (end)

    \subsection{Muticlass SVMs} % (fold)
    \label{sub:muticlass_svms}
    \begin{itemize}
        \item One vs Rest, for all classes, then combine.
            \begin{figure}[ht]
                \centering
                \includegraphics[width=0.4\textwidth]{one_vs_all_and_pairwise.png}
                \caption{One vs. All / Pairwise}
                \label{fig:one_vs_all_and_pairwise}
            \end{figure}
            Shaded regions are uncertain
        \item Pairwise comparison - exhaustive
            Similar to the prev. SVM, there are areas of uncertainty between classes.
        \item Continuous ranking - Best ranked
            \begin{figure}[ht]
                \centering
                \includegraphics[width=0.4\textwidth]{continuous_rank_svm.png}
                \caption{Continuous Ranking}
                \label{fig:continuous_rank_svm}
            \end{figure}
            No uncertainty as we pick the highest ranked class.
            This needs a vector $w$ to define the hyperplane for each class. where $w$ represents the distance of a point to the separating hyperplane.
    \end{itemize}
    % subsection muticlass_svms (end)

    \subsection{Multi-class Classification} % (fold)
    \label{sub:multi_class_classification}
    \begin{itemize}
        \item Multi-class expression is a generalization of the binary SVM expression.
        \item The concept of soft-margins can also be applied here.
    \end{itemize}
    % subsection multi_class_classification (end)

    \subsection{Comparison to Perceptrons} % (fold)
    \label{sub:comparison_to_perceptrons}

    Perceptrons:
    \begin{itemize}
        \item
        Linear separator - depends on starting values
        \item
        Simple update rule
        \item
        Prone to over-fitting
    \end{itemize}

    Support vector machines
    \begin{itemize}
        \item
        Unique max-margin linear separator
        \item
        Quadratic optimization
        \item
        Robust to over-fitting
    \end{itemize}

    % subsection comparison_to_perceptrons (end)

% section support_vector_machines (end)

\newpage

\section{Hidden Markov Models} % (fold)
\label{sec:hidden_markov_models}

    \subsection{Overview - Hidden Markov Models} % (fold)
    \label{sub:overview_hidden_markov_models}

        \begin{itemize}
            \item Data-points are not always IID. Context taken into account while predicting current value.
            \item Correlation with the prior data-points are leveraged.
            \item Stationary assumption: Assumes that the correlation between 2 output variables remains constant over the output space.
            \item Emission - $P(x|y)$, Transition - $P(y_t, y_{t-1})$
            \item Both the emission and transition are defined in terms of multinomial (discrete) distributions. e.g. values of a dice
            \item Joint Distribution - $P(y_{1..t}, x_{1..t}) = P(y_i) \prod_i P(x_i|y_i) \prod_i P(y_i|y_{i-1}) $
            \item Example - cascading uncertainty for a robot's movements over time.
            \item HMM Problem: $P(y_t|x_t .. x_1) $
        \end{itemize}

    % subsection overview_hidden_markov_models (end)

    \subsection{Applications of HMMs} % (fold)
    \label{sub:applications_of_hmms}

        \textbf{Monitoring}
        \begin{itemize}
            \item Predicting the current output state given a historical set of inputs.
            \item Can be expressed in terms of a recursive algorithm, that will required the output classes of the previous epochs.
            \item Forward algorithm - computes from the 0th epoch. Linear complexity in the number of epochs.
        \end{itemize}

        \textbf{Prediction/Forecasting}
        \begin{itemize}
            \item Similar approach to monitoring. Can be expressed recursively.
        \end{itemize}

        \textbf{Hindsight}
        \begin{itemize}
            \item Computing the output class at a given epoch, given input observations in the past and future. $1 < k < t$
            \item Forward-backward algorithm - uses epochs $1..k$ in the forward phase and $k..t$ in the backward phase.
        \end{itemize}

        \textbf{Most likely explanation}
        \begin{itemize}
            \item Predict entire sequence of output class $y_1..y_t$. Can be used in the context of speech recognition.
        \end{itemize}

        Viterbi algorithm - dynamic programming formulation. Discrete distribution of the output class y depends on the Initial State ($\pi$), Transition ($\theta$) and Emission ($\phi$)

        \textbf{Example}
        \begin{itemize}
            \item 2 classes - walking($c_1$) and sitting ($c_2$)
            \item input measurements (accelerometer) - high ($v_1$) and low ($v_2$)
            \item 100 training experiments
            \item
                Statistics
                \begin{itemize}
                    \item $\#c_1^{start} = 90$, $\#c_2^{start} = 10$
                    \item $\#(c_1, c_1) = 80000$, $\#(c_2, c_1) = 20000$
                    \item $\#(c_1, c_2) = 10000$, $\#(c_2, c_2) = 90000$
                    \item $\#(v_1, c_1) = 75000$, $\#(v_2, c_1) = 25000$
                    \item $\#(v_1, c_2) = 30000$, $\#(v_2, c_2) = 70000$
                    \item $P(y_{start} = c_1) = 0.9$, $P(y_{start} = c_2) = 0.1$
                    \item Frequency of starting with a particular class
                        \begin{itemize}
                            \item Transition
                            \begin{table}[ht]
                                \centering
                                \begin{tabular}{| l | c c |}
                                    \hline
                                    \textbf{} & \textbf{$c_1$} & \textbf{$c_2$} \\
                                    \hline\hline
                                        $c_1$ & 0.8 & 0.2 \\
                                    \hline
                                        $c_2$ & 0.1 & 0.9 \\
                                    \hline
                                \end{tabular}
                                \caption{Transition Probabilities}
                                \label{tab:transition_probabilities}
                            \end{table}
                            \item Emission
                            \begin{table}[ht]
                                \centering
                                \begin{tabular}{| l | c c |}
                                    \hline
                                    \textbf{} & \textbf{$v_1$} & \textbf{$v_2$} \\
                                    \hline\hline
                                        $c_1$ & 0.75 & 0.25 \\
                                    \hline
                                        $c_2$ & 0.3 & 0.7 \\
                                    \hline
                                \end{tabular}
                                \caption{Emission Probabilities}
                                \label{tab:emission_probabilities}
                            \end{table}
                        \end{itemize}
                \end{itemize}
        \end{itemize}

        \textbf{Forward algorithm - Example of Monitoring}
        \begin{itemize}
            \item Input set is $[v_1, v_1, v_2, v_2]$. Could be accelometer readings.
            \item $P(y_1|v_1) \propto P(v_1|y_1)P(y_1) = [0.675, 0.03] \propto [0.9574,  0.0426] (normalized) $
            \item $P(y_2|v_1, v_1) \propto P(v_1|y_2) \sum_{y_1} P(y_2|y_1) P(y_1|v_1) = [0.5809, 0.0402] \propto [0.9553, 0.0647] (c_1, c_2) $
            \item $P(y_3|v_1, v_1, v_2) \propto P(v_3|y_3) \sum_{y_2} P(y_3|y_2) P(y_2|v_1, v_1) = [0.1903, 0.1063] \propto [0.6417, 0.3583] $
            \item $P(y_4|v_1, v_1, v_2, v_2) \propto P(v_4|y_4) \sum_{y_3} P(y_4|y_3) P(y_3|v_1, v_1, v_2) = [0.1463, 0.2707] \propto [0.3308, 0.6492] $
        \end{itemize}

        \textbf{Most likely explanation - Example}
        \begin{itemize}
            \item
            $$f(y_1, y_2) = P(y_2|y_1) P(v_1|y_1) P(y_1)$$
            $$[y_1 = c_1, y_1 = c_2] [y_2 = c_1, y_2 = c_2] = [[0.54, 0.135], [0.003, 0.027]] $$
            $$\max_{y_1} P(y_1y_2|v_1) = \max_{y_1} f(y_1, y_2) = [0.54, 0.135] $$
            \item
            $$f(y_2, y_3) = P(y_3|y_2) P(v_1|y_2) \max_{y_1} P(y_1y_2|v_1)$$
            $$[y_2 = c_1, y_1 = c_2] [y_3 = c_1, y_2 = c_2] = [[0.324, 0.081], [0.0041, 0.0345]] $$
            $$\max_{y_1y_2} P(y_1y_2y_3|v_1v_1) = \max_{y_2} f(y_2, y_3) = [0.324, 0.081] $$
            \item
            $$f(y_3, y_4) = P(y_4|y_3) P(v_2|y_3) \max_{y_1y_2} P(y_1y_2y_3|v_1v_1)$$
            $$[y_3 = c_1, y_1 = c_2] [y_4 = c_1, y_2 = c_2] = [[0.0648, 0.0162], [0.0057, 0.0310]] $$
            $$\max_{y_1y_2y_3} P(y_1y_2y_3y_4|v_1v_1v_2) = \max_{y_3} f(y_3, y_4) = [0.0648, 0.051] $$
            \item
            $$f(y_4) = P(v_2|y_4) \max_{y_1y_2y_3} P(y_1y_2y_3y_4|v_1v_1v_2)$$
            $$[y_4 = c_1, y_4 = c_2] = [0.0162, 0.357] $$
            $$\max_{y_1y_2y_3y_4} P(y_1y_2y_3y_4|v_1v_1v_2v_2) \propto \max_{y_4} f(y_4) = 0.0357 $$
        \end{itemize}

    % subsection applications_of_hmms (end)

% section hidden_markov_models (end)

\newpage

\section{Deep Neural Networks} % (fold)
\label{sec:deep_neural_networks}

    \begin{itemize}
        \item
        Example of scaling w.r.t. depth of the network as opposed to a single hidden layer (Parity function.)
        \item
        Most popular activation functions for units - sigmoid (0 to 1) and tanh (-1 to 1)
        \item
        \textbf{Vanishing gradient}
        \begin{itemize}
            \item
            Vanishing gradient problem: Appearing to converge too soon. Gradient might not result in any change in the initial layers after backpropagation.
            \item
            Sequential squashing of the gradient results in gradually vanishing over time. Chain rule results in multiplying multiple numbers between 0 and 1, which could lead to computing a gradient below the required threshold to take a step in the direction of the minima of the convex function.
            \item
            Compounded product of the output value ranges over multiple layers resultings in the gradient converging to 0. This is because both the output and the gradient for both logistic sigmoid units and hyperbolic tan units are always between 0 and 1 in terms of magnitude.
            \item
            Possible solutions include
            \begin{itemize}
                \item
                Using a larger gradient for the layers close to the output layer.
                \item
                Using pre-trained units.
                \item
                Using recitified linear units.
            \end{itemize}
        \end{itemize}
    \end{itemize}

    \subsection{Rectified Linear Units} % (fold)
    \label{sub:rectified_linear_units}

    \begin{itemize}
        \item
        $h(a) = max(0, a) $
        \item
        Softplus: $h(a) = log(1 + e^a) $, where $a = \sum_i w_i x_i $
        \item
        If gradient is negative, don't back-propagate, if not, then set gradient to 1. This ensures that the gradient doesn't vanish for pathways in the network with $a > 0$.
        \item
        Softplus might also result in a vanishing gradient if the network has several layers.
        \item
        The disadvantage of using the rectified linear function is that it only approximately a linear function at most. Piecewise linear function (Figure \ref{fig:piecewise-linear-function}).
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.4\textwidth]{piecewise-linear-function}
            \caption{Piecewise Linear Function}
            \label{fig:piecewise-linear-function}
        \end{figure}
    \end{itemize}

    % subsection rectified_linear_units (end)

    \subsection{Maxout Units} % (fold)
    \label{sub:maxout_units}

        \begin{itemize}
            \item Maxout Unit (Figure \ref{fig:maxout-unit})
            \begin{figure}[ht]
                \centering
                \includegraphics[width=0.4\textwidth]{maxout-unit}
                \caption{Maxout Unit}
                \label{fig:maxout-unit}
            \end{figure}
            \item
            Choose $a$ such that $a = max(a_1, a_2, a_3)$, where $a_{ij} = \sum_j w_{ij} z_j $
            \item
            Produces a piecewise linear function, which is also convex.
        \end{itemize}

    % subsection maxout_units (end)

    \subsection{Overfitting} % (fold)
    \label{sub:overfitting}

    \begin{itemize}
        \item
        Due to high expressivity, the data could be very easily overfit.
        \item
        Solutions:
        \begin{itemize}
            \item
            Regularization: Penalizing large weights
            \item
            Data augmentation: Minor variations in the input data to allow the network to train a model that is invariant to small changes in the input data.
            \item
            Dropout: Randomly drop some units from the network during the training phase. Intended to increase network robustness against overfitting to specific datapoints.\\
            Dropout can be viewed as an approximation ensemble learning. Units need to stay in memory for this to work.
        \end{itemize}
    \end{itemize}

    % subsection overfitting (end)

% section deep_neural_networks (end)

\section{Convolutional Neural Networks} % (fold)
\label{sec:convolutional_neural_networks}

    \begin{itemize}
        \item
        Convolution: A mathematical operation on two function that produced a third, which can be viewed as a modified version of one of the initial two functions.
        \item
        Example of convolution: smoothing a function. (Figure \ref{fig:cnn-smoothing})
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.4\textwidth]{cnn-smoothing}
            \caption{Smoothing function - convolution}
            \label{fig:cnn-smoothing}
        \end{figure}
        \item
        $x$, the original input, is multiplied by a weight factor $w$ to obtain an output (either discrete or continous)
        \item
        Image example: white-to-black produces a black edge and vice-versa. Simple convolution.
        \item
        Could also be used for feature extraction when combined with an activation function.
        \item
        A convolution is a linear subset of units based on a specific weight vector.
        \item
        Activation functions confirm the existence of a patter.
        \item
        Gabor filters: Base weights used for image recognition. Comparisons against them could be use to detect patterns via convolutions.
        \item
        Architecture: Comprised of alternating convolutional and pooling layers. (Figure \ref{fig:cnn-arch})
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.4\textwidth]{cnn-arch}
            \caption{CNN Architecture}
            \label{fig:cnn-arch}
        \end{figure}
        \item
        Pooling: Aggregating the output of the result of the convolutional layer. It could be using a max/sum/avg function.
        \item
        A max pooling function could simulate an invariant i.e. doesn't matter which section of a given patch a pattern is found in.
        \item
        Pooling function is typically a commutative math operator.
    \end{itemize}

    \subsection{CNN Properties} % (fold)
    \label{sub:cnn_properties}

        \begin{itemize}
            \item
            CNNs won't usea fully connected network. Connections are sparse because only contiguous patches of the image are considered as input features to the convolutional layers.
            \item
            Weights are the same across a single layer, so the fact that they are shared simplifies the network.
        \end{itemize}

    % subsection cnn_properties (end)

    \subsection{MNIST CNN Example} % (fold)
    \label{sub:mnist_cnn_example}

        \begin{itemize}
            \item
            32x32 images, 5x5 patches as features for the convolutional layer, resulting in an abstracted 28x28 input space. Pooling layer of 2x2 dimensions, which means that every 4 units is pooled, reducing the feature space to a 14x14 space.
            \item
            This 14x14 is now treated as the original 32x32 image, and recursively worked on by alternating convolutional and pooling layers.
        \end{itemize}

    % subsection mnist_cnn_example (end)

% section convolutional_neural_networks (end)

\newpage

\section{Recurrent/Recursive Neural Networks} % (fold)
\label{sec:recurrent_recursive_neural_networks}

    \begin{itemize}
        \item
        Typically used when the input data is comprised of sequences, time series, textual data.
        \item
        Weights are shared across time steps.
        \item
        Unit representation: (Figure \ref{fig:rnn-units})
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.4\textwidth]{rnn-units}
            \caption{RNN Units}
            \label{fig:rnn-units}
        \end{figure}
        \item
        Errors tend to cascade in RNNs, as they are based on sequential data. The errors might also be compounded.
        \item
        HMMs can be a specific implementation of an RNN.
        Belief monitoring can be done as shown below. Figure \ref{fig:rnn}
        $P(Y_1|X_1)$ could be computed as either the sigmoid or softmax (binary/categorical outputs), or a Gaussian distribution (continuous outputs)
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.7\textwidth]{rnn}
            \caption{RNN - Belief Monitoring}
            \label{fig:rnn}
        \end{figure}
        \item
        Prediction cannot be done since the output $y$ at a given time-step depends on it's input at the same step.
        \item
        HMM: more flexible queries\\
        RNN: more expressive distributions
    \end{itemize}

    \subsection{Bi-directional RNN} % (fold)
    \label{sub:bi_directional_rnn}
        \begin{itemize}
            \item Hindsight queries : Figure \ref{fig:bi-directional_rnn}
            \begin{figure}[ht]
            \centering
            \includegraphics[width=0.7\textwidth]{bi-directional_rnn}
            \caption{Bi-directional RNN - Hindsight queries}
            \label{fig:bi-directional_rnn}
        \end{figure}
            \item
            $P(y_2| x_1, x_2, x_3, x_4)$ is an example of a hindsight prediction that can be made using bi-directional RNNs.
        \end{itemize}
    % subsection bi_directional_rnn (end)

    \subsection{Encoder - Decoder Model} % (fold)
    \label{sub:encoder_decoder_model}

        \begin{itemize}
            \item
            \textbf{Usage in Machine translation:}
            \begin{itemize}
                \item
                Encoding sentences into a vector representation.
                \item
                The encoded segment is the context.
                \item
                While decoding, each step in the model stores what the network has translated thus far, the output for this is the word in the new language.
            \end{itemize}
        \end{itemize}

    % subsection encoder_decoder_model (end)

    \subsection{Recursive Neural Networks} % (fold)
    \label{sub:recursive_neural_networks}

        \begin{itemize}
            \item
            Generalize recurrent neural networks from chains to trees.
            \item
            Useful to identify vector representation of variable length data.
            \item
            Applications for Semantic Parsing. Figure \ref{fig:dependency-parser}
            \begin{figure}[ht]
                \centering
                \includegraphics[width=0.7\textwidth]{dependency-parser}
                \caption{Dependency Parsing}
                \label{fig:dependency-parser}
            \end{figure}
        \end{itemize}

    % subsection recursive_neural_networks (end)

    \subsection{LSTM} % (fold)
    \label{sub:lstm}

        \begin{itemize}
            \item
            Gate units determine if the input is to be kept in memory or discarded
            \item
            Recurrent gate units decide whether or not to preserve the memory contents of the previous time-step
            \item
            Similarly there exists a gated output which decides whether or not the LSTM unit produces an output at the given time-step
        \end{itemize}

    % subsection lstm (end)

% section recurrent_recursive_neural_networks (end)

\section{Autoencoders} % (fold)
\label{sec:autoencoders}

    \begin{itemize}
        \item
        $x \Rightarrow f(x) \Rightarrow g(f(x)) \Rightarrow x$
        \item
        Architecture: Figure \ref{fig:autoencoders}.
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.7\textwidth]{autoencoders}
            \caption{Autoencoders}
            \label{fig:autoencoders}
        \end{figure}
    \end{itemize}

    \textbf{Linear Autoencoder}
    \begin{itemize}
        \item
        No activation function involved. The hidden layer is infered from the weights.
        \item
        $W_g W_f x = x$
        \item
        $W_g W_f$ would need to be an identity matrix. This derivation is not trivial due to the reduced dimensionality. $W_g$ and $W_f$ are pseudo-inverses of each other.
        \item
        Deriving the weights using the Euclidean norm objective is equivalent to PCA.
    \end{itemize}

    \textbf{Non-linear Autoencoder}
    \begin{itemize}
        \item
        Non-linear manifold
        \item
        $f$ is a non-linear function that can map data into a lower-dimensional representation. $g$ is another non-linear function that does the exact opposite.
    \end{itemize}

    \textbf{Deep Autoencoder:}
    $f$ and $g$ both consist of multiple layers

    \textbf{Sparse Representation:}
    The hidden layer can be just as large as the input layer, but the loss function penalizes non-zero entries, and tries to create the intermediate representation of the data which is sparse.

    \textbf{Denoising Data:}
    Loss function would use the noisy version as input and the clean version as output.

    \textbf{Probabilistic Autoencoder:}
    Illustration: Figure \ref{fig:autoencoders}.
    \begin{figure}[ht]
        \centering
        \includegraphics[width=0.7\textwidth]{probabilistic-autoencoder}
        \caption{Probabilistic Autoencoder}
        \label{fig:probabilistic-autoencoder}
    \end{figure}

    \textbf{Generative Model:}
    Sample the hidden `h' and use it as a generative base by which to generate values of `x'.

% section autoencoders (end)

\section{Generative Adversarial Networks} % (fold)
\label{sec:generative_adversarial_networks}

    \begin{itemize}
        \item
        Similar to Probabilistic Autoencoders, we can use the distribution $P(h)$ to sample new values for $x$.
        \item
        Problem: how to come up with a $P(h)$?
    \end{itemize}

    \textbf{Variational Autoencoder:}
    \begin{itemize}
        \item
        The encoded representation $h$ should be a Gaussian distribution with mean 0 and variance 1.
        \item
        Encoder part of network should approximate the mean and the variance.
        \item
        Illustration: Figure \ref{fig:variation-autoencoder}.
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.7\textwidth]{variation-autoencoder}
            \caption{Variational autoencoder}
            \label{fig:variation-autoencoder}
        \end{figure}
        \item
        Sampling hinders backpropagation.
    \end{itemize}

    \textbf{Generative Adversarial Network:}
    \begin{itemize}
        \item
        Game theoretic basis
        \item
        Generator vs Discriminator (compete; trained simultaneously)
        \item
        $W_d$ is minimized by a gradient descent algorithm during which $W_g$ remains static.
        Conversely, $W_g$ is maximized by a gradient ascent algorithm during which $W_d$ remains static.
        \item
        Illustration: Figure \ref{fig:gan-architecture}.
        \begin{figure}[ht]
            \centering
            \includegraphics[width=0.5\textwidth]{gan-architecture}
            \caption{GAN Architecture}
            \label{fig:gan-architecture}
        \end{figure}
    \end{itemize}

% section generative_adversarial_networks (end)

\newpage

\section{Ensemble Learning} % (fold)
\label{sec:ensemble_learning}

    \begin{itemize}
        \item
        Method to combine several hypotheses into a single stronger hypotheses.
        \item
        Bagging: Majority voting for a hypothesis, in classification problems
        \item
        Boosting: Computes a weighted majority
    \end{itemize}

    \textbf{Bagging vs. Boosting}
    \begin{itemize}
        \item
        Bagging
        \begin{itemize}
            \item
            Majority vote of a set of hypotheses
            \item
            Assumptions: Hypotheses are independent (error probability independent and roughly the same)
        \end{itemize}
        \item
        Boosting
        \begin{itemize}
            \item
            Weighted majority vote
            \item
            Weighted based on respective error rates
            \item
            Different hypotheses can be obtained by varying the weight of each datapoint in the training set.
        \end{itemize}
    \end{itemize}

    \begin{figure}[ht]
        \centering
        \includegraphics[width=.4\textwidth]{bagging_independent_classifiers}
        \caption{Bagging - Independent Classifiers}
        \label{fig:bagging_independent_classifiers}
    \end{figure}

    \textbf{Random forests:} Bagging, with decision trees (only classification)

    \textbf{Combining classifier parameters: } Might not always provide the benefits of ensemble learning. Combining the predictions usually proves to be more useful. Refer to Figure \ref{fig:parameter_averaging}. The 2 neural networks are independently accurate at computing the XOR, but averaging the weights yields an incorrect computation.
    \begin{figure}[ht]
        \centering
        \includegraphics[width=.4\textwidth]{parameter_averaging}
        \caption{Parameter Averaging}
        \label{fig:parameter_averaging}
    \end{figure}

% section ensemble_learning (end)

\newpage

\section{Stream Learning} % (fold)
\label{sec:stream_learning}

    \begin{itemize}
        \item
        Model continuously refreshed as new training data arrives.
        \item
        Bayesian learning being inherently suited to stream/online learning.
        \item
        Posterior hypothesis at each point is used as the prior belief for the next point.
        \item
        To avoid iterating multiple times over the training data, stochastic gradient descent is used.
    \end{itemize}

    \textbf{Robbins-Monro sufficient conditions}
    \begin{itemize}
        \item
        Choose learning rate $a_n$ such that $\sum_{n=1}^{\infty} a_n = \infty$, but $\sum_{n=1}^{\infty} (a_n)^2 < \infty$
        \item
        Example of $a_n$ is $\frac{1}{n} $
    \end{itemize}

% section stream_learning (end)

\newpage

\section{Formulae} % (fold)
\label{sec:formulae}

    \begin{itemize}

        \item
        KNN:
        $$y_x = mode({y_{x'}| x' \in knn(x)})$$

        \item
        Weighted KNN (used for regression):
        $$y_x = \sum_{x' \in knn(x)} w(x, x') y_{x'}$$

        \item
        Euclidean Loss:
        $$L(w) = \frac{1}{2} \sum_{n=1}^{N} (y(x_n, w) - t_n)^2 $$

        \item
        Linear regression Convex Optimization Objective:
        $$w = A^{-1}b$$
        where
        $$A = \sum_{n=1}^{N} \bar{x_n} \bar{x_n}^T$$ and $$b = \sum_{n=1}^{N} t_n \bar{x_n}$$

        \item
        Regularized Ridge Regression:
        $$w = argmin_w \frac{1}{2} \sum_{n=1}^{N} (y(x_n, w) - t_n)^2 + \lambda \lVert w \rVert $$
        equivalent to
        $$w = (\lambda I + A)^{-1}b$$

        \item
        Joint distribution marginalization
        $$Pr(A = a) = \sum_b Pr(A = a \wedge B = b) $$

        \item
        Conditional Probability
        $$Pr(A|B) = \frac{Pr(A \wedge B)}{Pr(B)} $$
        implying the chain rule,
        $$Pr(A \wedge B) = Pr(A|B) Pr(B)$$

        \item
        Bayes Rule:
        $$Pr(B|A) = \frac{Pr(A|B) Pr(B))}{Pr(A)} $$

        \item
        Bayesian Learning: \\
        Prior: $Pr(H)$ \\
        Likelihood: $Pr(e|H)$ \\
        Evidence: $e = <e_1, e_2, ... e_n>$
        $$Pr(H|e) = k Pr(e|H) Pr(H)$$
        where $k$ is a normalizing constant equivalent to $Pr(e)$ in Bayes Rule

        For a given hypothesis,
        $$Pr(e|h) = \prod_n P(e_n|h)$$

        \item
        Maximum a posteriori (MAP):
        $$h_{map} = argmax_{h_i} P(h_i|e)$$
        $$h_{map} = argmax_{h_i} P(h) P(e|h_i)$$

        \item
        Maximum likelihood estimation:
        Same as MAP, except that all the hypotheses have an equal prior probability.
        $$h_{mle} = argmax_{h_i} P(e|h_i)$$
        $$h_{mle} = argmax_{h_i} \sum_n \log P(e_n|h)$$

        \item
        Gaussian PDF:
        $$\frac{1}{\sqrt{2\pi\sigma}} e^{- \frac{(x - \mu)^2}{2 \sigma^2}}$$

        \item
        Bias - Variance tradeoff
        $$E[loss] = bias^2 + variance + noise$$

        \item
        Bayesian Linear Regression:
        $$\bar{w} = \sigma^{-2} A^{-1} \bar{X}^T y $$
        where
        $$A = \sigma^{-2} \bar{X}^T \bar{X} + \Sigma^{-1}$$

        \item
        Mixture of Gaussians Posterior Distribution:
        $$Pr(C_k|x) = \frac{1}{1 + e^{-(w^Tx + w_0)}}$$
        where
        $$w = \Sigma^{-1} (\mu_k - \mu_j)$$
        $$w_0 = -\frac{1}{2} \mu_k^T \Sigma^{-1} \mu_k + \frac{1}{2} \mu_j^T \Sigma^{-1} \mu_j + \ln\frac{\pi_k}{\pi_j}$$

        \item
        Mixture of Gaussians MLE Solution:
        $$\pi = \frac{\sum_{n \in c_1} 1}{N}$$
        $$\mu_1 = \frac{\sum_{n \in c_1} x_n}{N}; \mu_2 = \frac{\sum_{n \in c_2} x_n}{N}$$
        $$\Sigma = \frac{S_1 + S_2}{N}$$
        where
        $$S_1 = \sum_{n \in c_1} (x_n - \mu_1) (x_n - \mu_1)^T; S_2 = \sum_{n \in c_2} (x_n - \mu_2) (x_n - \mu_2)^T$$

        \item
        Logistic Regression Loss function:
        $$L(w) = - \sum_n y_n \ln \sigma(w^T \bar{x}) + (1 - y_n) \ln (1 - \sigma(w^T \bar{x})) $$

        \item
        Logistic Regression Gradient descent:
        $$w = w - H^{-1} \nabla L(w)$$
        where H is the Hessian matrix
        $$H = \bar{X} R \bar{X}^T$$
        R is a diagonal matrix with $R_{ii} = \sigma_i(1 - \sigma_i)$ and $\sigma_i = \sigma(w^T \bar{x}_n)$

        \item
        Perceptron Learning Gradient Descent:
        $$w = w - \eta \nabla E$$
        where $$E = - \sum_{(x_n, y_n) \in M} y_n w^T x_n$$
        and M is a set of misclassified examples and $y \in (1, -1) \forall y$

        \item
        Sigmoid perceptron learning min squared error:
        $$E(w) = \frac{1}{2} \sum_n (y_n - \sigma(w^T x_n))^2 $$
        $Also, \nabla(\sigma) = \sigma(1 - \sigma)$

        \item
        Multi-layer neural network error function:
        $$E(w) = \frac{1}{2} \lVert f(x_n, W) - y_n \rVert_2^2 $$

        \item
        Multi-layer neural network Gradient descent
        $$w_{ji} = w_{ji} - \eta \frac{\partial E_n}{\partial w_{ji}} $$

        \item
        Backpropagation algorithm
        \begin{equation*}
        \delta_j = \left\{
        \begin{array}{@{}ll@{}}
        h'(a_j) (z_j - y_j), & \text{base case, if j is an output unit} \\
        h'(a_j) \sum_k w_{kj}\delta_k, & \text{recursion, if j is a hidden unit}
        \end{array}\right.
        \end{equation*}

        \item
        Kernel function:
        $$k(x, x') = \phi(x)^T \phi(x') $$

        \item
        Weights:
        $$w = - \frac{1}{\lambda} \sum_n (w^T \phi(x_n) - y_n) \phi(x_n) $$
        $$\Phi = [\phi(x_1), \phi(x_2) ... \phi(x_N)] $$
        $$w = \Phi a $$
        $$a_n = - \frac{1}{\lambda} (w^T \phi(x_n) - y_n) $$

        \item
        Gram matrix
        $$K = \Phi^T \Phi$$
        Optimal $a$
        $$a = (K + \lambda I)^{-1} y $$

        \item
        Kernel prediction:
        $$y_* = \phi(x_*)^T w $$
        $$y_* = \phi(x_*)^T \Phi a $$
        $$y_* = k(x_*, X) (K + \lambda I)^{-1} y $$
        where $(X, y)$ is the training set and $(x_*, y_*)$ is the test instance

        \item
        Given that $k_1(x, x')$ and $k_2(x, x')$ are valid kernels, the below kernels are valid too
        \begin{itemize}
            \item
            $ck_1(x, x') \forall c>0 $
            \item
            $f(x)k_1(x, x')f(x') $
            \item
            $q(k_1(x, x')) $ where q is a polnomial with co-efficients $\geq 0$
            \item
            $exp(k_1(x, x')) $
            \item
            $k_1(x, x') + k_2(x, x')$
            \item
            $k_1(x, x')k_2(x, x')$
            \item
            $k_3(\phi(x), \phi(x'))$
            \item
            $x^T A x'$ where A is symmetric positive semi-definite
            \item
            $k_a(x_a, x'_a) + k_b(x_b, x'_b)$
            \item
            $k_a(x_a, x'_a)k_b(x_b, x'_b)$ where $x = \left(\begin{array}{c} x_a\\ x_b\\ \end{array} \right)$
        \end{itemize}

    \end{itemize}

    % section formulae (end)

\end{document}