\subsection{Background subtraction}
\label{background}

Raman spectra of liquid samples usually consist of many components, including
the signal of the analyte under investigation and the spectrum of the solvent.
In the case of absorbing analyte, the spectrum
is acquired from the near proximity of the sample cell wall, and so it
usually contains a signal of quartz.
The resulting spectrum can also contain unspecific background from different
sources like fluorescence from residuals from chemical synthesis and elastic
scattering of light from different light sources in the laboratory.

Therefore the background subtraction procedure took several steps.
Each spectrum was acquired as a series of frames, and all frames were stored
for further analysis.
The spectrum of solvent, buffer, and sample cell together with the intensity
normalization to the subtracted buffer signal was performed on each frame in
the first step using the \emph{Bgcor} program
\parencite{Bgcor2017}
(written in \cite{Matlab}),
developed as part of this thesis. The unspecific background was then removed
from each spectrum using background subtraction in frames decomposed to
spectral components by PCA
\parencite{Palacky2011}.
The thirds step consisted of extrapolation to zero time, where PCA decomposed
the frames to spectral components, and the spectrum at zero time
was reconstructed from significant ones.
The last optional step performed only when a series of spectra was
taken, for example, temperature-dependent measurement was to subtract the
background once more using PCA the same way as in step two but currently
applied to the whole series of the extrapolated spectra from the third step.
This section focuses mainly on the first step -- the subtraction of the
solvent, buffer, and sample cell, together with the intensity normalization.

Bgcor program allows specifying at most three background components that can
be used for subtraction.
Moreover, the unspecific polynomial background can be
simultaneously subtracted.
The background subtraction process is semiautomatic, where an automated
procedure finds the initial background removal draft, and manual adjustment
is then possible if needed.
The background subtraction mechanism is based on mean square fit, but it allows
to favor selected regions of the spectra by specifying different weights and
also can also highly penalize (by customizable factor) regions where the
subtracted background spectrum has a higher signal than the measured spectrum,
see
\figref{background:bgcor_main}.

\begin{figure}
	\centering
	\ig{1}{results_and_discussion/assets/background/bgcor_main}
	\caption[%
		Bgcor -- program for background correction.
	]{%
		\captiontitle{%
			Bgcor -- program for background correction.
		}
		The UVRR spectrum opened in the Bgcor program
		\parencite{Bgcor2017}.
		The automated subtraction draft was performed on the spectral region below
		1140\,\icm{}, negative bands in difference spectra were penalized by the
		factor of 100, the wavenumber scale was automatically adjusted at the
		position of the cacodylate band at 2932\,\icm{}, and the resulting spectrum
		intensity was normalized to the subtraction factor of the cacodylate
		spectrum.
		A cacodylate spectrum was used as \emph{bg1}, a spectrum of water as
		\emph{bg2}, and a spectrum of quartz sample cell as \emph{bg3}.
		The spectrum was acquired with 5\,mW of 257\,nm excitation laser at the
		sample from 500\,\g{m}M (in phosphates) poly(dAdT) at 10\,\textdegree{}C as
		the first frame with the 60\,s accumulation time.
	}
	\label{\figlabel{background:bgcor_main}}
\end{figure}

If requested, one of the backgrounds can also be used for intensity
normalization and fine-tuning of the wavenumber scale.
The correction of the wavenumber scale is performed by a fit (see
\cref{minimization})
of the gaussian curve with a polynomial background to a selected band and shift
of the wavenumber axis so that the band position from the background spectrum
matches the one from the measured spectrum.
The wavenumber scale shifts can also be further smoothed by polynomial fit, see
\figref{background:bgcor_xshift}.

\begin{figure}
	\centering
	\ig{0.7}{results_and_discussion/assets/background/bgcor_xshifts}
	\caption[%
		Bgcor -- wavenumber scale correction.
	]{%
		\captiontitle{%
			Bgcor -- wavenumber scale correction.
		}
		Overview of wavenumber scale correction of the spectrum from
		\figref{background:bgcor_main}.
		A linear trend was used for shifts approximation.
	}
	\label{\figlabel{background:bgcor_xshift}}
\end{figure}

The program also has a handy function for temperature measurements where it
allows to match a number from the file name containing the measured spectrum
with the number from a file name containing the background.
Intensities of background spectra from temperature measurements were corrected
for deviations of conditions between measurements.
The major band of each background spectrum used for intensity normalization
(cacodylate band at 606\,\icm{} in most of the measurements) was modeled as
described in
\cref{band_intensities}.
The resulting intensities were then fitted by a polynomial, and the values from
fit were used to normalize the intensities of the background spectra.

All the Bgcor settings can be seen in
\figref{background:bgcor_settings}.

\begin{figure}
	\centering
	\ig{1}{results_and_discussion/assets/background/bgcor_settings}
	\caption[%
		Bgcor -- settings.
	]{%
		\captiontitle{%
			Bgcor -- settings.
		}
		Overview of all the settings for the background correction for the spectrum
		from
		\figref{background:bgcor_main}.
	}
	\label{\figlabel{background:bgcor_settings}}
\end{figure}

\Figref{background:bgcor_main} shows that the spectrum also contains unspecific
fluorescence background, which was subtracted by the method described in
\textcite{Palacky2011}.
The resulting spectrum can be seen in
\figref{background:corrected_frame}.

\begin{figure}
	\centering
	\input{results_and_discussion/assets/background/corrected_frame}
	\vspace{3mm}
	\caption[%
		Single frame with a corrected background.
	]{%
		\captiontitle{%
			Single frame with a corrected background.
		}
		The spectrum from
		\figref{background:bgcor_main}
		with additionally corrected unspecific background by the method described
		in
		\textcite{Palacky2011}.
	}
	\label{\figlabel{background:corrected_frame}}
\end{figure}

Finally, the spectrum was extrapolated to zero time $t$ (to the middle of the
first spectrum accumulation) by using the \emph{Textrapol} program
\parencite{Textrapol2017} (developped in \cite{Matlab}).
The output from the program can be seen in
\figref{background:textrapol}.

\begin{figure}
	\centering
	\ig{1}{results_and_discussion/assets/background/textrapol}
	\caption[%
		Textrapol -- program for extrapolation and interpolation of series of
		spectra using PCA.
	]{%
		\captiontitle{%
			Textrapol -- program for extrapolation and interpolation of series of
			spectra using PCA.
		}
		The UVRR spectrum opened in the Textrapol program
		\parencite{Textrapol2017}.
		The spectral series of 12 consecutive frames with 5\,min acquisition time
		acquired with 5\,mW of 257\,nm excitation laser at the sample
		from 500\,\g{m}M (in phosphates) polyC at 20\,\textdegree{}C.
		The resulting spectrum was extrapolated to half of the accumulation of
		the first frame. Only the first principal component is shown, but the first
		two principal components were used.
	}
	\label{\figlabel{background:textrapol}}
\end{figure}


\section{Reader}
\subsection{Motivation}
\begin{frame}[fragile]{Reader Monad}{Motivation}
  \begin{columns}[t]
    \begin{column}{.4\textwidth}
      {\tiny\inputminted[escapeinside=\\`\\`]{haskell}{reader-motivation.hs}}
    \end{column}
    \begin{column}{.4\textwidth}
      \begin{itemize}
        \item Notice how we must pass this \texttt{\mylib{wc}} around
        \item We must pass it through functions that may never use it
      \end{itemize}
    \end{column}
  \end{columns}
\end{frame}
\subsection{Definition}
\begin{frame}[fragile]{Reader Monad}{Type}
  \inputminted[escapeinside=||,fontsize=\tiny]{haskell}{reader-types.hs}
  {
    \vspace*{-4cm}
    \hspace{7cm}
    \begin{minipage}{.35\textwidth}
      \begin{tcolorbox}[colframe=gray,colback=white,boxrule=1pt,arc=3.4pt,boxsep=0mm]
        \begin{minted}{csharp}
          class Reader<TC, TA> {
            Reader(T runReader) {
              this.runReader = runReader;
            }
          }
        \end{minted}
      \end{tcolorbox}
    \end{minipage}%
  }
  \vspace*{2.5cm}
  \begin{block}{\textbf{Exercise: }Reader Type Definition}
    \begin{align*}
      \mathsf{newtype}\ \ReaderH{a} &=
        \mathsf{Reader} \left\{ \fn{runReader} ::\ \uncover<2->{\mylib{c}\ \rightarrow\ \mylibo{a}} \right\}\\
    \end{align*}
  \end{block}
\end{frame}
\begin{frame}[fragile]{Reader Monad}{Type}
  Recall: $\mathsf{newtype}\ \Reader{a} = \mathsf{Reader} \left\{ \fn{runReader} ::\ c\ \rightarrow\ a \right\}$
  \begin{block}{\textbf{Exercise: }Reader Type Definition}
    \begin{align*}
      \return &:: a\ \rightarrow\ \Reader{a}\\
      \return\ a\ &= \uncover<2->{\mathsf{Reader}\ (\lambda\,\underline{\phantom{c}}\ \rightarrow\ a)}\\[1.5em]
      (\bind) &:: \Reader{a}\ \rightarrow\ (a\ \rightarrow\ \Reader{b})\ \rightarrow\ \Reader{b}\\
      x\ \bind\ fn &=\uncover<3->{\mathsf{Reader}\ (\lambda\,c\ \rightarrow
      \fn{runReader}\ (fn\ (\fn{runReader}\ x\ c))\ c)}\\
    \end{align*}
  \end{block}
\end{frame}
\begin{frame}[fragile]{Reader Monad}{Functions}
    \begin{block}{\textbf{Homework: }Reader Functions}
      \begin{align*}
        \fn{ask} &:: \Reader{c}\\
        \fn{asks} &:: (c\ \rightarrow\ a)\ \rightarrow\ \Reader{a}\\
        \fn{local} &:: (c\ \rightarrow\ c\,')\ \rightarrow\ \mathbf{Reader}\,c\,'\,a\ \rightarrow\ \Reader{a}\\
      \end{align*}
    \end{block}
\end{frame}
\subsection{Example}
\begin{frame}
  \centering
  \begin{tcolorbox}[enhanced, size=minimal,auto outer arc,
    width=2.8cm,octogon arc, colback=red,colframe=white,colupper=white, fontupper=\fontsize{7mm}{7mm}\selectfont\bfseries\sffamily, halign=center,valign=center,
    square,arc is angular, borderline={1mm}{-3mm}{red} ]
    CODE
  \end{tcolorbox}
\end{frame}


\section{Introduction and peak Finding}
Introduction
\begin{enumerate}
    \item Algorithmic Thinking: Peak Finding
    \item Sorting \& Trees: Event Simulation
    \item Hashing: Genome Comparison
    \item Numerics: RSA Encryption
    \item Graphs: Rubik’s Cube
    \item Shortest Paths: Caltech \textrightarrow MIT
    \item Dynamic Programming: Image Compression
\end{enumerate}


\subsection{Quasi-likelihood function}


\chapter{Multi-Modal Interaction-Aware Trajectory Optimization}
\label{text:approach}

To the best of the author's knowledge, exploiting the gradient of a deep neural network for trajectory optimization has not been done before, likely due to the tight runtime requirements of an online optimization as well as the lack of guarantees and insights that can be retrieved from such an approach without further ado. Several approaches have dealt with data-driven prediction models outputs before. Accessing the model's internal structure enables computing gradients that are more informed about the actual interactions happening in a multi-agent scene beyond the trajectory predictions. However, computing a gradient by backward passing through the whole network is widely associated with sizeable computational effort. While this may be true in general, trajectory optimization networks and input data are comparably small enough to perform these computations efficiently.

\begin{figure}[!ht]
\begin{center}
\begin{tikzpicture}

	\pie[radius=2.2, explode={0, 0, 0.2, 0}] {14.6/data-processing , 17.7/forward-pass , 19.0/backward-pass , 40.2/loss-evaluation, 1.5/others,  7.0/evaluation}

\end{tikzpicture}
\end{center}
\caption{Runtime evaluation of training the Trajectron model \cite{Salzmann2020} on pre-processed data of the ETH Hotel 2 pedestrian dataset \cite{Pellegrini2009}}
\label{img:training_runtime}
\end{figure}

Figure \ref{img:training_runtime} gives an inside into the runtime distribution of the Trajectron model, during training. Here, merely roughly one-fifth of the overall runtime is spent performing the backward pass, while most time is spent evaluating the loss function. In particular, forward- and backward-pass do not differ much regarding their overall runtime. While this is only one out of many examples, it illustrates the capability of performing a backward pass online.
\newline\newline
In this chapter, an overview of the approach is provided. In Section \ref{text:approach/formulation}, the problem of socially aware motion planning is formulated formally, including the assumptions made within this project.
\newline
Section \ref{text:approach/overview} presents an overview of the full trajectory optimization pipeline. It is shown how the algorithm is built to allow us to deal with general graph-like pedestrian behavior prediction models, with multimodal  probabilistic outputs. Furthermore, the advantages of formulating socially aware motion planning as an optimization problem while utilizing a graph-based prediction model, e.g., deep learning-based models such as the Trajectron \cite{Ivanovic2018}, over purely learned \cite{Chen2017} or purely optimization-based \cite{vandenBerg2011} algorithms are illustrated. Finally, it is explained how the usage of a general-purpose, computation-graph-based framework such as PyTorch \cite{pytorch} or TensorFlow \cite{tensorflow} can be harnessed for a highly modular, efficient, and versatile implementation.
\newline
The optimization problem's exact formulation is further explained in Section \ref{text:approach/objective} and \ref{text:approach/constraint}. Both the goal-driven and the interaction-driven parts are illustrated, beginning with the objective function and continuing with a derivation of their gradients. Furthermore, several designs of the specific objective function are discussed. The subsequent section addresses the design of the constraints used. Likewise, several constraint function designs are compared.
\newline
To use the system in real-world applications, it must comply with several requirements. While the optimization design has tackled some of these, e.g., the feasibility of the robot's control limitations or safety regarding robot-human interaction, the system underlies runtime limits. Several runtime improvements have been implemented to achieve this goal, which are described in Section \ref{text:approach/runtime}.

\input{problem_formulation}
\input{system_overview}

\section{Objective Function Design}
\label{text:approach/objective}
\input{objective_goal}
\input{objective_interaction}

\section{Constraint Function Design}
\label{text:approach/constraint}
\input{constraint_dynamics}
\input{constraint_safety}

\section{Runtime optimization}
\label{text:approach/runtime}
\input{runtime_unrolling}
\input{runtime_warmstart}
\input{runtime_attention}


\section{Method}\label{sec:method}
In order to process the accelerometer and TAC value timeseries data, an approach dealing with the continuous data stream over time is required.
Since almost all machine learning approaches require an input of constant length, a technique such as the sliding window time series analysis, combined with sequence padding, can be used.
This method enables the analysis on fractions of the time series data with a constant length.
The sliding window therefore partitions the data into a number of finite-length segments and tries to relate $ws$ past data points to a prediction $y$ by grouping \cite{MOZAFFARI2015150}.
The sliding window is defined by the parameters windows size ($ws$) and the stride size ($ss$).
So at any given point $t_n$ in time $t$, the subset tuple

\begin{equation}
	s_{t_n} = (\{ x_{t_{n - i}}, x \in X, ws > i \geq 0 \}, y_{t_n})
\end{equation}

is calculated.
As all data points are evenly spaced out due to the resampling, we do not have to take the elapsed time between observations into account.
The stride indicates the increment of $n$ for $t_n$.
For this regression, a window size of $10\si{\second}$ and a stride of $2\si{\second}$ was used.
The window size was chosen as in \cite{DBLP:conf/ijcai/KillianPNMC19} and the stride values were determined experimentally in order to create a feasible amount of data to work with that could still represent motion and does lack specific scientific justification.

\subsection{Feature Extraction}\label{ssec:feature}
By applying the sliding window with the parameters as stated above on the data resampled at $40 \si{\hertz}$, this results in a feature matrix of $400 \times 3$ features. In order to reduce the amount of data and increase the information at the same time, a manual feature extraction phase was implemented.
The extracted features are shown in table \ref{tab:features}. Principal component analysis (PCA) is used for dimensionality reduction to avoid issues like overfitting. The three-dimensional accelerometer data is reduced into one dimension by using the PCA class from  \texttt{scikit-learn} \cite{scikit-learn}.

\begin{table}[h]
	\resizebox{\columnwidth}{!}{%
		\centering
		\begin{tabular}{|l|c|l|}
			\hline
			Features & Generated data & Definition\\
			\hline

			First & 3 & The first elements per axis\\
			Last & 3 & The last elements per axis\\
			Mean & 3 & The mean per axis\\
			Median & 3 & The median per axis\\
			Var & 3 & The variation per axis\\
			Std & 3 & The standard deviation per axis\\
			Min & 3 & The minimum per axis\\
			Max & 3 & The maximum per axis\\
			Argmin & 3 & The index of the minimum per axis\\
			Argmax & 3 & The index of the maximum per axis\\
			Sum & 3 & The sum per axis\\
			Q50 & 3 & The 50\% quantile per axis\\
			Q75 & 3 & The 75\% quantile per axis\\
			Q25 & 3 & The 25\% quantile per axis\\
			PCA & 400 & The principal component value of all 3 axis\\
			MinPCA & 1 & The minimum of PCA\\
			MaxPCA & 1 & The maximum of PCA\\
			MeanPCA & 1 & The mean of PCA\\

			\hline
		\end{tabular}
	}
	\caption{Features extracted per sliding window dataframe.}
	\label{tab:features}
\end{table}

Additional parameters have been extracted using \texttt{tsfresh} \cite{DBLP:journals/ijon/ChristBNK18}, but not all algorithms have been trained with the additional features due to runtime constraints.

\begin{table}[h]
	\resizebox{\columnwidth}{!}{%
		\centering
		\begin{tabular}{|l|c|l|}
			\hline
			Features & Generated data & Description \\
			\hline

			abs\_energy & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.abs_energy}{absenergy per axis}\\
			absolute sum of changes & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.abs_energy}{absolute sum of changes per axis}\\
			autocorrelation & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.autocorrelation}{autocorrelation per axis}\\
			binned\_entropy & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.binned_entropy}{binned entropy per axis}\\
			c3 & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.c3}{c3 per axis}\\
			cid\_ce & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.cid_ce}{cid ce per axis}\\
			count\_above & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.count_above}{count above per axis}\\
			count\_above\_mean & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.count_above_mean}{count above mean per axis}\\
			count\_below & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.count_below}{count below per axis}\\
			count\_below\_mean & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.count_below_mean}{count below mean per axis}\\
			first\_location\_of\_maximum & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.first_location_of_maximum}{first location of maximum per axis}\\
			first\_location\_of\_minimum & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.first_location_of_minimum}{first location of minimum per axis}\\
			kurtosis & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.kurtosis}{kurtosis per axis}\\
			last\_location\_of\_maximum & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.last_location_of_maximum}{last location of maximum per axis}\\
			last\_location\_of\_minimum & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.last_location_of_minimum}{last location of minimum per axis}\\
			longest\_strike\_above\_mean & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.longest_strike_above_mean}{longest strike above mean per axis}\\
			longest\_strike\_below\_mean & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.longest_strike_below_mean}{longest strike below mean per axis}\\
			mean\_abs\_change & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.mean_abs_change}{mean abs change per axis}\\
			mean\_change & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.mean_change}{mean change per axis}\\
			mean\_second\_derivative\_central & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.mean_second_derivative_central}{mean second derivative central per axis}\\
			number\_crossing m & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.number_crossing_m}{number crossing m per axis}\\
			number\_peaks & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.number_peaks}{number peaks per axis}\\
			skewness & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.skewness}{skewness per axis}\\
			time\_reversal\_asymmetry\_statistic & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.time_reversal_asymmetry_statistic}{time reversal asymmetry statistic per axis}\\
			variation\_coefficient & 3 & \href{https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html\#tsfresh.feature_extraction.feature_calculators.variation_coefficient}{variation coefficient per axis}\\
			\hline
		\end{tabular}
	}

	\caption{Additional features extracted using \texttt{tsfresh} \cite{DBLP:journals/ijon/ChristBNK18}.}
	\label{tab:features-additional}
\end{table}

In comparison to \cite{DBLP:conf/ijcai/KillianPNMC19}, a few things are handled differently already: first, all of the above features are calculated on the whole window, while \citeauthor{DBLP:conf/ijcai/KillianPNMC19} used a two-tiered window approach, a short-term and a long-term window, and some additional features, that were calculated separately \cite{DBLP:conf/ijcai/KillianPNMC19}.
Secondly, \citeauthor{DBLP:conf/ijcai/KillianPNMC19} extracted features more specific to audio analysis.
Their features sum up to $1215$ data points, while we currently extract $445$ with $72$ additional \texttt{tsfresh} features, creating a total of $517$.
This already shows a significant difference between the methods used and introduces additional obstacles for comparison.

\subsection{Hyperparameter Optimization}\label{ssec:hpo}
In order to create the best possible regressor, another important element besides feature extraction is the tuning of said model.
While parameters are learned during training – for example the slope of linear regression or weights of neural network, hyperparameters can be arbitrarily set by a data scientist beforehand.

\begin{figure}[h]
	\centering
	\includegraphics[width=\columnwidth]{hpo-parallel}
	\caption{Exemplary HPO value plot for \texttt{LightGBM} with sample \texttt{JB3156} chunk 0}
	\label{fig:hpo-parallel}
\end{figure}

Therefore, the selection of correct values for these hyperparameters is crucial and can significantly improve the performance of a model.
This tuning process can be viewed as a type of optimization problem itself cause we aim to find the right combination of a set of hyperparameters to e.g. minimize the loss or maximize the performance metric.
Apart from a manual search, random search and grid search are viable options.
However, such an approach is more or less equal to a brute force attempt.
Hence, so called automated hyperparameter optimization (HPO) algorithms have evolved, using techniques such as bayesian optimization, gradient descent and evolutionary algorithms.
This enables efficiently searching large spaces and pruning of unpromising trials for faster results.

Multiple libraries can be used; most famous contenders \texttt{Hyperopt} \cite{DBLP:conf/icml/BergstraYC13} and \texttt{Optuna} \cite{DBLP:conf/kdd/AkibaSYOK19} are subject of countless articles comparing them (e.g. \cite{neptune_2019}). \texttt{Optuna} was chosen based on the better documentation and build-in visualization.

A hyperparameter optimization study was performed separately for each of the algorithms presented in \ref{ssec:algorithms}.
Figure \ref{fig:hpo-parallel} shows a value plot for different hyperparameters for the \texttt{LightGBM} \cite{DBLP:conf/nips/KeMFWCMYL17} algorithm, validated the chunk number $0$ of sample \texttt{JB3156}.
This dataset was chosen because it is the largest file after splitting.

Depending on the runtime of the algorithm and the respective features, each study was optimized for $100 - 1000$ trials.

\begin{figure*}[bt]
	\centering
	\includegraphics[width=\textwidth]{hpo-sliceBIG}
	\caption{Exemplary HPO slice plot for \texttt{LightGBM} with sample \texttt{JB3156} chunk 0}
	\label{fig:hpo-slice}
\end{figure*}

\begin{figure}[h]
	\centering
	\includegraphics[width=\columnwidth]{hpo-hist}
	\caption{Exemplary HPO history plot for \texttt{LightGBM} with sample \texttt{JB3156} chunk 0}
	\label{fig:hpo-hist}
\end{figure}

Figure \ref{fig:hpo-slice} shows the variation of each hyperparameter over time.
Whereas the final optimization history is shown in figure \ref{fig:hpo-hist}.
This graph already clearly indicates, that the optimization reached a plateau quite fast and was unable to significantly improve the performance further on.
In case that this model can not satisfy the desired performance requirements, further work in \ref{ssec:feature} is needed.

\subsection{Algorithms}\label{ssec:algorithms}
In ensemble learning theory, we call weak learners (or base models) models that can be used as building blocks for designing more complex models by combining several of them.
Most of the time, these basics models perform not so well by themselves either because they have a high bias often due to low degree of freedom models or because they have too much variance to be robust in case of high degree of freedom models.
But a low bias and a low variance, although they most often vary in opposite directions, are the two most fundamental features expected for a model.
This is the well known bias-variance tradeoff.

So the idea of ensemble methods is to try reducing bias and/or variance of such weak learners by combining several of them together in order to create a strong learner (or ensemble model) that achieves better performances than any algorithm alone.
One can combine these models in different ways; three major kinds are known in literature: bagging to decrease the model’s variance, boosting decreasing the model’s bias, and stacking to increasing the predictive force of the classifier.

Thus, in consideration of our research question, the algorithms used all implement the concept of ensemble learning.
We evaluated \texttt{ExtraTrees}, \texttt{RandomForest}, \texttt{AdaBoost}, \texttt{GradientBoosting}, and \texttt{HistGradientBoosting} from the popular \texttt{scikit-learn} library \cite{scikit-learn} to establish a baseline for better comparison.

Nonetheless, one of the most popular shallow learning techniques in recent years has been gradient boosting, dominating many Kaggle (subsidiary of Google LLC, online community of data scientists and machine learning practitioners) competitions with heterogeneous tabular data.
Similar to random forest, gradient boosting works by ensembling many decision trees in order to perform regression or classification.
However, unlike random forest, gradient boosting grows trees sequentially, iteratively growing trees based on the residuals of the previous tree.
Doing so allows gradient boosting to focus on particularly tricky observations and yields an extraordinarily powerful ensemble of trees.
Among the most common algorithms in recent years are \texttt{XGBoost} \cite{DBLP:conf/kdd/ChenG16}, \texttt{LightGBM} \cite{DBLP:conf/nips/KeMFWCMYL17} and \texttt{CatBoost} \cite{DBLP:journals/corr/abs-1810-11363}.

All used algorithms share an implementation of the \texttt{scikit-learn} API \cite{sklearn_api}, which made it easy to use a shared infrastructure.


\startreport{Gambling on Decision Trees}
\reportauthor{Ramesh Subramonian}
\newcommand{\extra}{0}

\section{Introduction}

Decision trees are a well understood statistical technique \cite{Hastie2009}.
This paper makes the following contributions:
\be
\item Well known metrics used to determine the best decision to be made
at a given node are (i) gini impurity (ii) information gain and (iii)
variance reduction. In contrast, we propose a metric that is inspired by looking
at the decision making as a gambling problem, the aim being to maximize the
expected gain at each step.
\item
Well known metrics used to measure the quality of the decision tree
induced by the splitting metric chosen above are
precision, recall, accuracy, f1-score, \ldots
In contrast, continuing the gambling metaphor, we propose a ``payout'' metric which
measures the quality of the decision tree.
\item Lastly, we show that the quality of the decision tree is affected by the
choice of splitting metric. What this means is that one should choose the
splitting metric {\bf after} one has decided which quality metric is most
appropriate for the problem at hand.
\ee

We believe that the proposed changes to the way decision trees are created and
evaluated is particularly relevant to the case where relative
confidences are involved. This is often the case when multiple candidates are
competing for a limited resource, like user attention.
For example, supposed we have 2 decision trees, \(C_A, C_B\), one for product A and one for
product B.  We need \(C_A, C_B\) to return \(p_A, p_B\), where \(p_A\) is
\(C_A\)'s estimate of how likely the user is to select product A. In order to
make the best business decision, we would like \(p_A, p_B\) to be as asccurate
as possible.

Random forests are a common way is which an ensemble of decision tres can be
used to produce better results than any single tree. This paper specifically
does not address how random forests are created. For example, different decision
trees can be created by withholding some of the features, or some of the
instances. What we focus on is creating the ``best'' possible decision tree
given the inputs.

%% \subsection{Conventions}
%% Throughout the paper, we use the data sets listed in Table~\ref{tbl_data_sets}
%% for our experiments.
%% \begin{table}
%% \centering
%% \begin{tabular}{|l|l|l|l|} \hline \hline
%% {\bf Data Set Name } & {\bf Rows} & {\bf Columns} & {\bf Location} \\ \hline
%% Breast Cancer & X & X & \\ \hline
%% Titanic & X & X & \\ \hline
%% \ldots & X & X & \\ \hline
%% \end{tabular}
%% \label{tbl_data_sets}
%% \caption{Data Sets used for experimentation}
%% \end{table}
%%
\subsection{A brief introduction to \cal{Q}}

\cal{Q} is a column oriented database that is used to implement the decision
tree algorithm. In this section, we present the essential aspects of \cal{Q}.

\cal{Q} allows efficient manipulation of vectors. All elements of vectors are
the same type. The 2 types we use in this paper are F4 and I4, representing
single precision floating point and 4-byte signed integers. A vector has a fixed
length \(n\) and can be thought of as a map from \(i\) to \(f(i)\),
where \(i \in [0, n-1]\).

\subsection{Notations}

\bi
\item Let \(T = \{t_i\}\) be a table of vectors of type F4, representing the features.
\item Let \(g\) be a vector of type I4, representing the {\bf goal} or
outcome which we wish to
predict. We assume the goal can be either Heads, represented by the integer value 1 or Tails, represented as 0.
\item boolean features are
modeled as having type F4 and the comparison is always \(\leq 0\), so that the
left branch has instances where the feature value is false and the
right branch has instances where the feature value is true.
\ei

A decision tree is a Lua table where each element identifies
\be
\item a feature e.g., age
\item the comparison to be made e.g., age \(\leq 40\). Currently, the comparison
is {\bf always} \(\leq\)
\item a left decision tree, which may be null
\item a right decision tree, which may be null
\ee

\begin{figure}
\centering
\fbox{
\begin{minipage}{35cm}
\begin{tabbing} \hspace*{0.25in} \=  \hspace*{0.25in} \=
                \hspace*{0.25in} \=  \hspace*{0.25in} \= \kill
Let \(\alpha\) be minimum benefit required to continue branching \\
Let \(n_G = 2\) be the number of values of the goal attribute \\
Let \(n_T =\) number of instances classified as tails \\
Let \(n_H =\) number of instances classified as heads \\
Let \(T\) be table of vectors, each vector being an attribute \\
Initialize, decision tree \(D = \{\}\) \\
\(F, g\) as described above \\
{\bf function } MakeDecisionTree(D, T, g) \+ \ \\
  \(C = Q.\mathrm{numby}(g, n_G);~n_T = C[0];~n_H = C[1]\) \\
  {\bf forall} \(f \in T:~ b(f), s(f) = \mathrm{Benefit}(f, g, n_T, n_H)\) \\
  Let \(f'\) be feature with maximum benefit. \\
  {\bf if} \(b(f') < \alpha\) {\bf then } \+  \\
    {\bf return} \- \\
  {\bf endif} \\
    \(x = \mathrm{Q.vsgt}(f', s(f'))\) \\
    \(T_L = T_R = \{\}\) \\
    {\bf forall} \(f \in F\) {\bf do} \+ \\
      \(f_L = Q.\mathrm{where}(f, x)\) \\
      \(f_R = Q.\mathrm{where}(f, \mathrm{Q.not}(x))\) \\
      {\bf if } \(Q.max(f_L) > Q.min(f_L)\) {\bf then} \+ \\
        \(T_L = T_L \cup f_L \) \- \\
      {\bf endif} \\
      {\bf if } \(Q.max(f_R) > Q.min(f_R)\) {\bf then} \+ \\
        \(T_R = T_R \cup f_R \) \- \\
      {\bf endif} \- \\
    {\bf endfor} \\
    D.feature = \(f'\) \\
    D.threshold = \(b(f')\) \\
    D.left = \(\{\}\) \\
    D.right = \(\{\}\) \\
    \(DT(D.left, g_L, T_L)\) \\
    \(DT(D.right, F_R, g_R, T_R)\) \- \\
{\bf end}
\end{tabbing}
\end{minipage}
}
\label{dt_pseudo_code}
\caption{Decision Tree algorithm}
\end{figure}

\begin{figure}
\centering
\fbox{
\begin{minipage}{15cm}
\begin{tabbing} \hspace*{0.25in} \=  \hspace*{0.25in} \=
                \hspace*{0.25in} \=  \hspace*{0.25in} \= \kill
{\bf function } \(\mathrm{Benefit}(f, g, n_T, n_H)\) \+  \\
  \(f', h^0, h^1 = CountIntervals(f, g)\) \\
  \(b = \mathrm{Q.wtbnft}(h^0, h^1, n_T, n_H)\) (Table~\ref{algo_weighted_benefit}) \\
  \(b', \_, i = \mathrm{Q.max}(b)\) \\
  {\bf return} \(b', f'[i]\)  \- \\
{\bf end}
\end{tabbing}
\end{minipage}
}
\label{compute_benefit_numeric}
\caption{Benefit Computation (numeric attributes)}
\end{figure}
%%-------------------------------------------
\begin{table}
\centering
\begin{tabular}{|l|l|l|} \hline \hline
{\bf Variable} & {\bf Formula} & {\bf Explanation} \\ \hline \hline
\(n_T^L\)  & & number of tails on left \\ \hline
\(n_H^L\)  & & number of heads on left \\ \hline
\(n_T^R\)  & & number of tails on right \\ \hline
\(n_H^R\) & & number of heads on right \\ \hline

\(n_L\)    & \(n_T^L + n_H^L \) & number on left \\ \hline
\(n_R\)    & \(n_T^R + n_H^R \) & number on right \\ \hline

\(n\)      & \(n_T + n_H \) & total number \\ \hline

\(w_L \) & \( n^L/n\)  & weight on left \\ \hline
\(w_R \) & \( n^R/n\)  & weight on left \\ \hline
\hline
\(o_H \) & \( n_T/n_H\) & odds for heads \\ \hline
\(o_T \) & \( n_H/n_T\) & odds for tails \\ \hline
\hline

\(p_H^L \) & \( n_H^L/n_L\) & probability of heads on left \\ \hline
\(p_H^R \) & \( n_H^R/n_R\) & probability of heads on right \\ \hline
\(p_T^L \) & \( n_T^L/n_L\) & probability of tails on left \\ \hline
\(p_T^R \) & \( n_T^R/n_R\) & probability of tails on right \\ \hline
\hline

\(b_H^L\) &  \(o_H^L \times p_H^L + (-1)  \times p_T^L\) &
            benefit of betting heads on left \\ \hline
\(b_T^L\) & \(o_T^L \times p_T^L + (-1)  \times p^H_L \) &
            benefit of betting tails on left \\ \hline

\(b_H^R\) & \(o_H^R \times p_H^R + (-1)  \times p_T^R \) &
            benefit of betting heads on right \\ \hline
\(b_T^R\) & \(o_T^R \times p_T^R + (-1)  \times p_H^R \) &
            benefit of betting tails on right \\ \hline
\hline

\(b_L\) & \( \mathrm{max}(b_H^L, b_T^L)\) & benefit on left \\ \hline
\(b_R\) & \( \mathrm{max}(b_H^R, b_T^R)\) & benefit on right \\ \hline
\hline

\(b\) &  \(b_L \times w_L + b_R \times w_R\) & weighted benefit \\ \hline
\end{tabular}
\label{algo_weighted_benefit}
\caption{Weighted Benefit Computation}
\end{table}
%%-------------------------------------------
\begin{figure}
\centering
\fbox{
\begin{minipage}{15cm}
\begin{tabbing} \hspace*{0.25in} \=  \hspace*{0.25in} \= \kill
{\bf function } \(\mathrm{CountIntervals}(f, g)\) \+  \\
{\bf Inputs} \+ \\
Vector \(f\) of length \(n\) type F4 \\
Vector \(g\) of length \(n\) type I4 with values 0, 1 \- \\
{\bf Outputs} \+ \\
Vector \(f'\) of length \(n'\) type F4 \\
Vector \(h^0\) of length \(n'\) type I4  \\
Vector \(h^1\) of length \(n'\) type I4  \- \\
\(f'\) is the unique values of \(f\), sorted ascending \\
Let \(\delta(x)\) = 1 if \(x\) is true and 0 otherwise \\
For \(k = 0, 1\), compute \(h^k_j = \sum_{i=1}^{i=n} \delta(g_i = k \wedge f_i \leq f_j)\) \- \\

{\bf end}
\end{tabbing}
\end{minipage}
}
\label{count_intervals}
\caption{Count Intervals}
\end{figure}
%%-------------------------------------------

\section{Algorithm}

Since decision trees have been well documented, we focus our attention on the
salient characteristics of our approach which are
\be
\item how does one decide on the comparison to be performed at a node?
\item how does one decide when to not split a node further, assuming that asplit
is possible at all.
\ee

Given the simple recursive nature of decision trees, we focus our
decision on what is done at a single node. Without loss of generality,
let this be the root node. At this point, we know the number of elements
classified as Heads and Tails, \(n_H, n_T\) respectively. This allows us to set
the odds of betting on heads or tails. At each node, we are asked to a single
question.
\bi
\item For a numeric attribute, it is of the form \(x \leq y\)
\item For a boolean attribute, it is of the form \(x == y\), where
\(y\) can be true or false
%% TODO LATER
%% \item For an unordered categorical attribute that can take on values in \(X\),
%% it is of the form \(x \in y \subset X\)
\ei
We iterate over all possible values of \(y\) for each attribute and find the
attribute and decision that produces the maximum benefit. If this benefit
exceeds a pre-defined threshold, we split the data into two, a left branch and a
right branch and proceed recursively. Else, the expansion of the decision tree
stops at this point.
Figure~\ref{dt_pseudo_code} describes the algorithm at this level. For clarity of
exposition, we only describe the treatment of numeric attributes.

\subsection{Q Operators Used}
The Q operators used in Table~\ref{tbl_q_op}.
\begin{table}
\centering
\begin{tabular}{|l|l|l|l|} \hline \hline
{\bf Operator} & {\bf Input} & {\bf Output} & {\bf Explanation} \\ \hline \hline
y = not(x) & x B1 Vector & y B1 Vector & \(y_i = ~x_i\) \\ \hline
y = vsgt(x, s) & x Vector, s Scalar & \(y\) B1 Vector &
\(x_i \geq s \Rightarrow y_i = \) true; else, \(y_i = \)  false \\ \hline
y = sum(x) & x Vector & y Scalar & \(y = \sum_i x_i\)  \\ \hline
numby(x, n) & x Vector, n number,
% \(x_i \in [0, n-1]\)
&
y Vector & \(y_i = \) number of elements of \(x\) that have value \(i\) \\ \hline
% . Note that \(y:\mathrm{length} == n\)
\hline
\end{tabular}
\caption{Q Operators Used}
\label{tbl_q_op}
\end{table}

\subsection{Benefit Computation}
It is useful to model this problem as a gambling problem where we are
sampling with replacement.
\bi
\item Let there be 100 balls, with 60  being marked Heads and 40
being marked Tails.
\item Assume that we have to gamble on whether a ball picked at random is Heads
ot Tails. If the ``house'' sets odds to ensure a fair game, then it would offer
a payout of 4/6 if you bet heads and 6/4 if you bet tails.
\item If you bet heads, then the expected benefit is
\(\frac{60}{100} \times \frac{4}{6} +
\frac{40}{100} \times (-1)  = 0\)
\item If you bet tails, then the expected benefit is
\(\frac{60}{100} \times (-1) +
\frac{40}{100} \times \frac{6}{4}  = 0\)
\ei

Now, assume that before we place out bet, we are allowed to ask one question
which has an answer of left or right. If left, we are told that the ball was
selected from a population of 40 heads and 15 tails. If right, we are told that
the ball was selected from a population of 20 heads and 25 tails. Clearly, we
would bet heads if the answer is left and tails if the answer is right. The
question is: What is the expected benefit for 100 trials?
The answer is \(29 \frac{1}{6}\) and not 0 as before, calculated as follows.

\(100 ( \times ( \frac{40+15}{100} \times (
  \frac{40}{40+15} \times \frac{4}{6}  +
  \frac{15}{40+15} \times -1 ) ) +
  ( \frac{20+25}{100} \times (
  \frac{20}{20+25} \times -1 +
  \frac{25}{20+25} \times \frac{6}{4}  ) ) )
  \)
which is \(
 = (40 \times  \frac{4}{6}) -15 - 20 + (25 \times \frac{6}{4} )
 = 29 \frac{1}{6}\)

\ifthenelse{\equal{extra}{1}}{
\subsubsection{Categorical Attributes}
We deal with categorical attributes as follows. Let \(X\) be a categorical
attribute that takes on values \(\{x_1, x_2, \ldots x_M\}\), where \(M\) is
typically small. We can assume less than 100 values.

Let \(Y = \{(x_i, g_i)\}\) be the data set, where \(i = 1, \ldots N\).
By this we ean
that the \(i^{th}\) point has value \(x_i \in X\) and its goal attribute has
value \(g_i \in \{0, 1\}\).
\(Y\) may have other features but for now we are concentrating on \(X\).

Create a table, \(Z\), of vectors \(N, P, \rho\) of length \(M\) as follows.
\begin{enumerate}
\item
\(n_j = \sum_{i=1}^{i=N} z(i, j)\) where
\(z_j = 1 \) if \(x_i = x_j \) and \(g_i = 0\)
\item
\(p_j = \sum_{i=1}^{i=N} z(i, j)\) where
\(z_j = 1 \) if \(x_i = x_j \) and \(g_i = 1\)
\item
\(rho_j = \frac{p_j}{p_j+n_j}\)
\end{enumerate}
Sort \(Z\) on \(rho\). Create vectors, \(N', P'\) of length \(M\) as follows
\be
\item
\ee
}{}

\newpage
\section{Evaluating a decision tree}
\label{payout}
In addition to the standard metrics, such as classification accuracy, we
evaluate decision trees using the ``payout'' metric.
Table~\ref{dt_eval_code} describes what happens at a particular leaf of the decision tree. Let's explain the intuition with an example. Consider a leaf which had
\bi
\item 20 Heads and 30 Tails in the training set and
\item 10 Heads and 40 Tails in the testing set
\ei
When a head is presented to this leaf, the prediction is Tails. We award the
algorithm a credit of 2/5 and a debit of 3/5.
Similarly, when a tail is presented to this leaf, the prediction is Tails. We award the
algorithm a credit of 3/5 and a debit of 2/5. We average the payout for every
testing instance and that is construed to be the ``benefit'' of this decision tree.

Let \(b_x, w_x\) be the values for leaf \(x\). Then, the total payout of the tree is \(\frac{\sum_x b_x}{\sum_x w_x}\)

\begin{table}[hbtp]
\centering
\begin{tabular}{|l|l|l|} \hline \hline
\(n^H_1\) & number of heads in test data & \\ \hline
\(n^T_1\) & number of tails in test data & \\ \hline
\(n^H_0\) & number of heads in train data&  \\ \hline
\(n^T_0\) & number of tails in train data&  \\ \hline
%%
\(p^H\) & probability of heads at leaf & \(n^H_0 / (n^H_0 + n^T_0)\) \\ \hline
\(p^T\) & probability of tails at leaf & \(n^T_0 / (n^H_0 + n^T_0)\) \\ \hline
%%
\(b^H\) & payout for heads in test data & \(n^H_1 \times ( p^H - p^T )\)  \\ \hline
\(b^T\) & payout for tails in test data & \(n^T_1 \times ( p^T - p^H )\)  \\ \hline
%%
\(b\) & payout for test data & \(b_H + b_T\) \\ \hline

\(w\) & number of test instances & \(n^H_1 + n^T_1\) \\ \hline
\(\bar{b}\) & payout per instance & \(b/w\) \\ \hline
\hline
\end{tabular}
\label{dt_eval_code}
\caption{How a Leaf of a Decision Tree is evaluated}
\end{table}

\section{When to stop splitting?}

In Figure~\ref{dt_pseudo_code}, we stated that we stop splitting when the weighted benefit
is less than a threshold of \(\alpha\). We are now in a position to describe how
\(\alpha\) is determined. We search the space of possible values of \(\alpha =
(0, 1)\), evaluating the ``payout'' (Section~\ref{payout}) for each \(\alpha\).
We pick the \(\alpha\) which produces the maximum payout.
%% A sample on the breast cancer data set (reference TODO) is in
%% Figure~\ref{fig_breast_cancer_cost_versus_alpha}.
%
\section{Evalaution}

For evaluation, we chose the following data sets
\be
\item 2 publicly available data sets, the breast cancer data set and the
Titanic data set from the UCI ML Repository.
\item 3 proprietary data sets --- ds1, ds2, ds3
\item 2 synthetic data sets generated as follows
\be
\item \TBC
\item \TBC
\ee
\ee

We compare Q against sklearn.
In case of Q, the hyper-parameter to be optimized is \(\alpha\).
In case of sklearn, we use grid search to optimize for the hyper-parameters.We
pick the model that optimizes f1 score.

See \verb+../python/grid*txt+ \TBC

\begin{table}
\centering
\begin{tabular}{|l||l|l|l|l|l|l|l|} \hline \hline
{\bf Data Set} & {\bf Algorithm} & {\bf accuracy} & {\bf precision} & {\bf recall} & {\bf f1 score} & {\bf payout} \\ \hline \hline
cancer & Q &      92.0962 & 0.9385 &  0.9333 &  0.9359 &  \textcolor{red}{0.8385} \\ \hline
cancer & sklearn & 94.1581 & 0.9605 &  0.9444 &  0.9524 &  0.8196 \\ \hline
\hline
titanic & Q &      78.7946 & 0.7453 &  0.6897 &  0.7164 &  \textcolor{red}{0.5609} \\ \hline
titanic & sklearn &       81.0268 & 0.7871 &  0.7011 &  0.7416 &  0.4937 \\ \hline
\hline
ds1 & Q &    76.7681 &0.2441  &0.1511  & 0.1866  & \textcolor{red}{0.4723} \\ \hline
ds1 &  sklearn &     75.6626 & 0.2659 &  0.2155 &  0.238 &   0.4687 \\ \hline
\hline
ds2 & Q &     62.0271 & 0.5597 &  0.7833 &  0.6529 &  \textcolor{red}{0.1505} \\ \hline
ds2 & sklearn &     60.8696 & 0.5424 &  0.9077 &  0.679 &   0.1102 \\ \hline
\hline
ds3 & Q &   60.1256 & 0.5714 &  0.6323 &  0.6003 &  \textcolor{red}{0.1722} \\ \hline
ds3 & sklearn &   59.056 &  0.5386 &  0.944 &   0.6859 &  0.0787 \\ \hline
\hline
\end{tabular}
\label{tbl_results}
\caption{Results comparing Q versus sklearn}
\end{table}

\subsection{Discussion}

The key takeaway from the results is that you get what you look for. In case of
Q, we are building the decision tree with optimizing payout in mind. In the case
of sk-learn, we are building the decision tree with optimizing f1 score in mind.
It should not come as a surprise that Q out performs sklearn when payout is the
quality metric and that sklearn out performs Q when f1 score is the quality
metric.

This leads us to the conclusion that the choice of splitting decision
should not be made independent of the metric that will be used to judge the
resultant decision tree. Of course, the quality metric should be chosen
base on how well it models the business problem at hand.

\subsection{Grid Search specifications for sklearn}

See \verb+f1_score_sklearn.txt+ \TBC

\bibliographystyle{alpha}
\bibliography{../../../DOC/Q_PAPER/ref.bib}


\documentclass[12pt]{article}

\usepackage[noend]{algpseudocode}
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{float}
\usepackage{graphicx}
\usepackage[margin=.75in]{geometry}
\usepackage{amsmath,amsthm,amssymb}
\usepackage{dsfont}
\usepackage{subcaption}
\usepackage{amsthm}
\usepackage{mathtools,amssymb}
\usepackage{wrapfig}
\usepackage[font=small,skip=-20pt]{caption}
\allowdisplaybreaks

\newtheorem*{definition}{Definition}
\newtheorem*{question}{Question}
\newtheorem{theorem}{Theorem}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{conjecture}[theorem]{Conjecture}

\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\Rgz}{\mathbb{R}_{\ge 0}}

\newcommand{\ip}[2]{\left\langle{#1},{#2}\right\rangle}
\newcommand{\norm}[1]{\left\lVert{#1}\right\rVert} % Yufei
\newcommand{\sizeof}[1]{\left\lvert{#1}\right\rvert} % Yufei

\newcommand{\woloss}{without loss of generality }

\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\cone}{cone}
\DeclareMathOperator*{\hull}{hull}

\newcommand{\1}[1]{\mathds{1}[{#1}]}
\renewcommand{\P}[1]{\mathds{P}\left[{#1}\right]}
\newcommand{\E}[1]{\mathds{E}\left[{#1}\right]}
\newcommand{\Var}[1]{\mathrm{Var}[{#1}]}

\newcommand{\unit}{\mathds{1}}
\newcommand{\lo}{\succ}

% \renewcommand{\thesubsection}{\thesection.\alph{subsection}}
% \renewcommand\thesubsection{\ \ (\alph{subsection})}

\begin{document}

% \renewcommand{\qedsymbol}{\filledbox}

\title{Voting with $2$-dimensional preferences}
\author{
  Clay Thomas \\
  claytont@princeton.edu
\and
  Yufei Zheng\\
  yufei@cs.princeton.edu
}

\maketitle
\begin{abstract}
  The study of voting rules often restricts attention to
  well-behaved classes of possible preferences of voters.
  We define a new class: $2$-dimensional preferences,
  for which very good voting rules are possible.
  We argue that $2$-dimensional preferences are in some ways
  more natural and expressive than more traditional classes
  such as single-peaked preferences.
  Furthermore, we give an almost-complete combinatorial classification
  of $2$-dimensional preferences, and provide some additional
  results about the natural extension of $d$-dimensional preferences.
\end{abstract}

\section{Introduction}

  We believe that $d$-dimensional preferences are a very natural setting,
  and that they give a beautiful geometric explanation for psychological
  concepts such as opinion and agreement.

\clearpage
%============================================================
%============================================================
\part{Combinatorics}

\section{Definition of $d$-dimensional preferences}
  Let $\{x_1,\ldots,x_m\} = X\subseteq \Rgz^d$ be any (ordered) set of
  $m$ distinct points with nonnegative coordinates.
  We call $X$ the set of \emph{outcomes}.
  Given any $a\in \Rgz^d$, define a order $>_a$ on $X$ as follows:
  $x_i >_a x_j$ if and only if $\ip{a}{x_i} > \ip{a}{x_j}$.
  Define
  \begin{align*}
    P_d(X) = \{ \succ \in R(X) | \exists a\in\Rgz^d: x \succ y \iff x >_a y\}
  \end{align*}
  Note that we consider only total orders, in particular,
  ties are not allowed.
  % This maybe could use a footnote:
  % \footnote{
  %   This is largely without loss of generality.
  %   For a given $x,y$ pair, the set of $a$ for which
  %   $\ip{a}{x} = \ip{a}{y}$ has measure zero.
  % }

  Some first observations:
  \begin{itemize}
    \item If $d=1$, then $|P(X)| = 1$, i.e. preferences are completely
      determined by the underlying set $X$.
    \item If $d=n$, then every linear preference on $X$ can occur in $P(X)$.
      \begin{proof}
        Let $X = \{e_i\}$ simply be the standard basis vectors.
        To induce an ordering $i_1, i_2, \ldots, i_n$, just create a preference
        vector $a$ which gives weight $1/k$ to coordinate $i_k$.
      \end{proof}
  \end{itemize}
  The above hints that there should be some sort of continuum between $d=1$
  and $d=n$ of how complex the set $P(X)$ can be.

\section{Lemmas}
  \subsection{Lemmas based on the structure of $X$}
    These first few lemmas relate geometric properties of $X$ to
    limitations on the structure of $>_a$ for a specific, fixed $a$.

    \begin{definition}
      Let $x,y\in X\subseteq \Rgz^d$.
      We say $x$ \emph{dominates} $y$, denoted $x\gg y$,
      if $x[k] > y[k]$ for each $k=1,\ldots,d$.
    \end{definition}
    \begin{proposition}
      If $x \gg y$, then $x >_a y$ for any nonzero $a\in \Rgz^d$.
    \end{proposition}
    \begin{proof}
      Simply observe $a_i x_i \ge a_i y_i$ for each $i=1,\ldots, d$.
      Because $a\ne 0$, there is also some $i$ where $a_i x_i > a_i y_i$.
    \end{proof}

    We'll need to use the familiar geometric notion of a convex hull:
    ((ACTUALLY, I'M NOT SURE IF WE DO))
    \begin{definition}
      For points $x_1,\ldots,x_n \in \R^d$, let $\hull(x_1,\ldots,x_n)
      = \{u_1x_1 + \ldots + u_nx_n | 0\le u_i\le 1, \sum_{i=1}^n u_i = 1\}$
      denote the convex hull of $x_1,\ldots,x_n$.
    \end{definition}
    \begin{lemma}
      Let $z,x_1,\ldots,x_k \in \Rgz^d$ and $a\in \Rgz^d \setminus \{0\}$.
      If $z >_a x_i$ for $i=1,\ldots,k$, then $z >_a w$
      for any $w\in \hull(x_1,\ldots,x_k)$.
    \end{lemma}
    \begin{proof}
      We have $\ip{a}{z} > \ip{a}{x_i}$ for each $i=1,\ldots,k$.
      If $w = u_1x_1+ \ldots + u_nx_n$ and $\sum_i u_i =1$, then
      $\ip{a}{w} = u_1\ip{a}{x_1}+\ldots+u_n\ip{a}{x_n}
      < u_1\ip{a}{z} + \ldots + u_n\ip{a}{z} = \ip{a}{z}$
    \end{proof}
    \begin{proposition}
      Let $z,x_1,\ldots,x_k \in \Rgz^d$.
      Suppose that there exists $w\in \hull(x_1,\ldots,x_k)$
      such that $w \gg z$.
      Then no $a$ satisfies $z >_a x_i$ for each $i=1,\ldots, k$.
    \end{proposition}
    \begin{proof}
      For contradiction, suppose such an $a$ exists.
      Then $z >_a w$ as well. However, because $w \gg z$,
      this is a contradiction.
    \end{proof}

  \subsection{Lemmas relating different preference vectors}
    This next group of lemmas relates different

    \begin{definition}
      For any vectors $a_1,\ldots, a_k \in \Rgz^d$, define
      $\cone(a_1,\ldots,a_k) = \{ u_1a_1 + \ldots + u_ka_k | u_i\ge 0 \forall j\}$.
      That is, the convex cone of vectors defined by $a_1,\ldots,a_k$.
    \end{definition}
    \begin{proposition}
      Suppose that for preference weights $a_1,\ldots, a_k$,
      we have $x >_{a_1} y, \ldots, x >_{a_k} y$.
      Then for any nonzero $b\in \cone(a_1,\ldots, a_k)$,
      $x >_b y$ as well.
    \end{proposition}
    \begin{proof}
      Let $b = u_1a_1+ \ldots + u_ka_k$.
      We get $\ip{b}{x} = u_1\ip{a_1}{x} + \ldots + u_k\ip{a_k}{x}
      > u_1\ip{a_1}{y} + \ldots + u_k\ip{a_k}{y} = \ip{b}{y}$,
      as desired.
    \end{proof}

    ((INSERT LEMMAS ABOUT SLOPES) ---- INSERTED, FIGURES MISSING)
    \begin{claim} \label{clm:angle}
      For any pair of points $i = (i_1, i_2), j = (j_1, j_2) \in X$,
      where $i_1 < j_1$ and $i_2 > j_2$,
      let $a = (a_1, a_2) \in \R_{\geq 0}^2$
      be the preference weight vector that gives $i =_a j$,
      then for any preference weight vector $b = (b_1, b_2) \in \R_{\geq 0}^2$,\begin{enumerate}
        \vspace{-5pt}
        \item \label{clm:angle1}
        $\theta_a < \theta_b$ if and only if $i >_b j$.
	\vspace{-5pt}
	\item \label{clm:angle2}
	$\theta_a > \theta_b$ if and only if $i <_b j$.
      \end{enumerate}
    \end{claim}
    % ADD FIGURE clmAngle
    \begin{proof}
      \ref{clm:angle1} and \ref{clm:angle2} are completely analogous,
      here we only prove \ref{clm:angle1}.
      \begin{itemize}
        \item ``$\implies$'':
        On the one hand, from $\theta_a < \theta_b$
        we have $\frac{a_2}{a_1} = \tan{\theta_a} < \tan{\theta_b} = \frac{b_2}{b_1}$.
	On the other hand, by $i =_a j$,
	$a_1 i_1 + a_2 i_2 = a_1 j_1 + a_2 j_2$,
	so we get $\frac{a_2}{a_1} = \frac{i_1 - j_1}{j_2 - i_2} < \frac{b_2}{b_1}$.
	Notice that $j_2 - i_2 < 0$, then $b_1 (i_1 - j_1) > b_2 (j_2 - i_2)$,
	which simplifies to $b_1 i_1 + b_2 i_2 > b_1 j_1 + b_2 j_2$, that is $i >_b j$.

	\item ``$\impliedby$'':
	By $i >_b j$, $b_1 i_1 + b_2 i_2 > b_1 j_1 + b_2 j_2$,
	and $\frac{b_2}{b_1} > \frac{j_1 - i_1}{i_2 - j_2}$.
	From $i =_a j$ we get that $\frac{j_1 - i_1}{i_2 - j_2} = \frac{a_2}{a_1}$,
	then $\frac{b_2}{b_1} > \frac{j_1 - i_1}{i_2 - j_2} = \frac{a_2}{a_1}$,
	$\theta_b > \theta_a$.
      \end{itemize}
    \end{proof}

    \begin{lemma} \label{lem:slopes}
      Let $i, j, i', j'$ be two pairs of points in $X$ satisfying
      \begin{itemize}
        \vspace{-5pt}
        \item neither of $i, j$ dominates the other: $i_1 < j_1$ and $i_2 > j_2$,
        \vspace{-5pt}
        \item neither of $i', j'$ dominates the other: $i_1' < j_1'$ and $i_2' > j_2'$,
        \vspace{-5pt}
        \item the slope of line $ij$ is less than the slope of line $i'j'$:
        $\frac{j_2 - i_2}{j_1 - i_1} < \frac{j_2' - i_2'}{j_1' - i_1'}$.
      \end{itemize}
      \vspace{-5pt}
      Then for any preference weight vector $b \in \mathbb{R}_{\geq 0}^2$,
      \begin{enumerate}
        \vspace{-5pt}
        \item \label{lem:slopes1}
	  If $i >_b j$, $i' >_b j'$.
	\vspace{-5pt}
	\item \label{lem:slopes2}
	  If $j' >_b i'$, $j >_b i$.
      \end{enumerate}
    \end{lemma}
    \begin{proof}
      Again \ref{lem:slopes1} and \ref{lem:slopes2} are symmetric,
      we only prove \ref{lem:slopes1}.
      Let $a, a'$ be the preference weight vectors that gives
      $i =_a j$ and $i' =_{a'} j'$ respectively.
      By Claim \ref{clm:angles}, if $i >_b j$, $\theta_b > \theta_a$,
      and to show $i' >_b j'$, we only need to show $\theta_b > \theta_{a'}$.
      To this end, we prove that $\theta_a > \theta_{a'}$:

      By definitions of $a$ and $a'$,
      we have $\frac{a_2}{a_1} = \frac{j_1 - i_1}{i_2 - j_2}$
      and $\frac{a_2'}{a_1'} = \frac{j_1' - i_1'}{i_2' - j_2'}$.
      From the relation of the $2$ slopes,
      $\frac{j_1 - i_1}{i_2 - j_2} > \frac{j_1' - i_1'}{i_2' - j_2'}$,
      therefore $\frac{a_2}{a_1} > \frac{a_2'}{a_1'}$, $\theta_a > \theta_{a'}$.
    \end{proof}

\section{Bounding the number of $2$-dimensional preferences}

\section{Impossibility Results}

  ((MOST AND LEAST FAVORITE IS IMPOSSIBLE FOR D=2,
  mention that later we'll apply this for Sandwich))

  The following preference set may be important in high dimension.
  We call it $d$-{\sc Cycle}
  \begin{align*}
    \begin{array}{ccccccccccccccccccccccccccccccccccccccccc}
    1 &>& 2 &>& 3 &>&\ldots &>& d-2 &>& d-1 &>& d \\
    2 &>& 3 &>& 4 &>& \ldots &>& d-1 &>& d &>& 1 \\
    3 &>& 4 &>& 5 &>& \ldots &>& d &>& 1 &>& 2 \\
      &&    &&    &&   \vdots \\
    d-1&>&d&>& 1 &>&\ldots &>& d-4 &>& d-3 &>& d-2\\
    d &>& 1 &>& 2 &>& \ldots &>& d-3 &>& d-2 &>& d-1 \\
    \end{array}
  \end{align*}
  \begin{theorem}
    $d$-{\sc Cycle} is not $(d-1)$-dimensional for any $d$.
  \end{theorem}
  \begin{proof}
    Suppose for contradiction there existed
    $a_1,\ldots,a_d\in\Rgz^{d-1}$ such that $a_i$ yielded the $i$th line of
    $d$-{\sc Cycle} (in particular, the line whose favorite point is $i$).
    The vectors $\{a_i\}$ cannot be linearly independent.
    Thus, there exists a linear combination of vectors
    $u_1 a_1 + \ldots + u_n a_n = 0$.
    Let $S = \{a_i | u_i > 0\}$ and let $T = \{a_i | u_i < 0\}$.
    Note that, because every coordinate of each $a_i$ is nonnegative
    (and no $a_i$ is zero) neither $S$ nor $T$ are empty.
    We have
    \[ \sum_{a_i \in S} u_i a_i = \sum_{a_i \in T} -u_i a_i \]
    That is, there exists a nonzero $b\in \cone(S) \cap \cone(T)$.

    Now, I claim that $b$ satisfies the following:
    \[ 1 >_b 2 >_b 3 >_b \ldots >_b d-1 >_b d >_b 1 \]
    Proof: For each pair $i > i+1$,
    observe that the inequality is satisfied for all the vectors $a_i$ except
    for one of them (namely, $a_{i+1}$ which ranks $i+1$ highest).
    Thus, for either $S$ or $T$, every vector $a_i$ in the set has the opinion
    $i >_{a_i} i+1$. Thus, every vector in the cone of that set
    has the opinion $i > i+1$ as well, in particular, $i >_b i+1$.
    Of course, the above argument works for the pair $d>1$ as well.

    Thus, transitivity gives us $1>_b 1$, a contradiction.
  \end{proof}

  ((SET THE STAGE TO PROVE FLIPFLOP IS IMPOSSIBLE FOR D=2...
  Actually I think the listy upper bound should suffice))

\clearpage
%============================================================
%============================================================
\part{Voting}

\section{Definitions}
  \subsection{Voting}
    Consider a set of preferences $P$ (i.e. linear orders) on a
    set of outcomes $M$.
    We follow the convention that $m$ is the number of outcomes and $n$
    is the number of voters.
    We have the following definitions:
    \begin{itemize}
      \item A \emph{social welfare function on $P$}
        is a function $F : P^n \to P$
      \item A \emph{social choice function on $P$}
        is a function $f : P^n \to M$
      \item A welfare function $F$ is \emph{unanimous} if,
        for any $\succ\ \in P$, we have $F(\succ,\ldots,\succ) =\ \succ$
      % \item A choice function $f$ is \emph{unanimous} if,
      %   whenever there exists a fixed $a$ with $a \succ_i b$ for all
      %   $i=1,\ldots, n$ and $b\in M\setminus \{a\}$,
      %   we have $f(\succ_1,\ldots,\succ_n) = a$.
      %   I'm not sure if this is a standard notion.
      \item A welfare function $F$ is a \emph{dictatorship} if
        there exists an $i$ such that $F(\succ_1,\ldots,\succ_n) = \succ_i$
      \item A choice function $f$ is a \emph{dictatorship} if
        there exists an $i$ such that $f(\succ_1,\ldots,\succ_n) = a_i$,
        where $a_i$ is the favorite outcome of $\succ_i$
      \item A welfare function $F$ satisfies \emph{in dependence of
        irrelevant alternatives} if, whenever $a\succ_i b \iff a\succ_i' b$
        and $\succ = F(\succ_1,\ldots,\succ_n),
        \succ' = F(\succ'_1,\ldots,\succ'_n)$,
        we get $a\succ b \iff a\succ' b$
      \item A choice function $f$ is \emph{incentive compatible} if,
        for any $\succ_1,\ldots,\succ_n, i, \succ_i'$, we have \\
        $f(\succ_1,\ldots,\succ_n) \succ_i f(\succ_1,\ldots,\succ_i'\ldots,\succ_n)$
      \item A collection of preferences $\succ_1,\ldots,\succ_n$,
        has a Condorcet winner $x \in M$ if, for any other $y\in M$,
        we have $x \succ_i y$ for more than half of the indices
        $i=1,\ldots,n$
    \end{itemize}

    Let $R(M)$ denote the set of all linear orders on $M$.
    Recall that when $P = R(M)$, there are many known impossibility results,
    the two most famous of which are:

    \begin{theorem}[Arrow's Impossibility Theorem]
      If $|M| \ge 3$, then
      every unanimous social welfare function on $M$ which satisfies independence of
      irrelevant alternatives is a dictatorship.
    \end{theorem}

    \begin{theorem}[Gibbard?Satterthwaite]
      If $|M| \ge 3$, then
      every surjective, incentive compatible social choice function on $M$
      is a dictatorship.
    \end{theorem}

  \subsection{Single-peaked preferences}
    Our main point of contrast will be the well-understood class of
    \emph{single peaked} preferences.

    Let $S\subseteq [0,1]$ be a finite set of $m$ points in the unit interval.
    We call $S$ the set of \emph{outcomes}.
    Let $R(S)$ denote the set of linear orders on $X$.
    A preference $\succ \in R(S)$ is called \emph{single peaked} if
    there exists an outcome $p\in S$ (called the ``peak'' of $\succ$)
    such that $x < y < p \implies x \prec y$ and $p < y < x \implies x \prec y$.
    In other words, the preference has a favorite outcome,
    and its opinion strictly decreases as you move farther away from the favorite.
    Note that no assumption is made about outcomes on different sides of the peak.
    Define
    \begin{align*}
      P_{sp}(S) = \{ \succ \in R(S) | \succ \text{is single peaked }\}
    \end{align*}

\section{Single-peaked verses $2$-dimensional preferences}

  % TODO: at end, put this at start of whichever page makes the most sense
  \begin{wrapfigure}{r}{0.2\textwidth}
    \begin{center}
      \vspace{-0.5in}
      \begin{align*}
        1 > 2 > 3 \\
        2 > 3 > 1 \\
        3 > 1 > 2 \\
      \end{align*}
      \caption*{\textsc{Cycle}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 \\
        1 > 3 > 2 \\
        3 > 2 > 1 \\
        2 > 3 > 1 \\
      \end{align*}
      \caption*{\textsc{Sandwich}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 \\
        2 > 1 > 3 \\
        2 > 3 > 1 \\
        3 > 2 > 1 \\
      \end{align*}
      \caption*{\textsc{GoodCompromise}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 \\
        1 > 3 > 2 \\
        3 > 1 > 2 \\
        3 > 2 > 1 \\
      \end{align*}
      \caption*{\textsc{BadCompromise}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 > 4 \\
        1 > 2 > 4 > 3 \\
        2 > 1 > 3 > 4 \\
        2 > 1 > 4 > 3 \\
      \end{align*}
      \caption*{\textsc{FlipFlop}}
      \vspace{-0.25in}
      \begin{align*}
        1 > 2 > 3 > 4 \\
        4 > 3 > 2 > 1 \\
      \end{align*}
      \caption*{\textsc{Reverse}}
    \end{center}
  \end{wrapfigure}

  When there are exactly three candidates, $2$-dimensional
  preferences have strictly more expressive power than
  single-peaked preferences.
  \begin{proposition}
    Every single-peaked preference set on $m=3$ outcomes
    is a $2$-dimensional preference set. In particular,
    up to relabeling it is a subset of $P_2(X)$, for
    $X = \{ (3,0), (2,2), (0,3) \}$.
  \end{proposition}
  \begin{proof}
    Without loss of generality, relabel the outcomes
    of the single peaked preferences as $S = \{1,2,3\}$.
    The resulting preferences are given by {\sc GoodCompromise}.
    ((INSERT PICTURE OF GOOD COMPROMISE AS A 2D SET))
  \end{proof}
  Furthermore, {\sc BadCompromise} gives an example of a preference set
  which is $2$-dimensional, but not single-peaked:
  \begin{proposition}
    In a single peaked set of preferences, it is not possible
    for every outcome to be the lowest ranked outcome of some preference.
  \end{proposition}
  \begin{proof}
    The median outcome (via the standard order on $[0,1]$)
    cannot be the lowest ranked.
  \end{proof}

  When $m>3$, neither single-peaked nor $2$-dimensional preferences
  are a subset of the other. One was to see this to to simply
  count the number of single-peaked preferences, and see that it
  grows much faster with $m$ than $2$-dimensional preferences do
  (we'll see later, the maximal size of a $2$-dimensional
  preference set is $O(m^2)$).
  \begin{proposition}
    The number of single-peaked preferences for any set $S$ of $m$ outcomes
    is at least $2^{\Omega(m)}$
  \end{proposition}
  \begin{proof}
    Choose the median outcome of $S$ to be the peak.
    Given a subset $T\subseteq [m-1]$ of size $|T| = \lfloor m/2\rfloor$,
    we can associate a
    unique single-peaked preference as follows:
    treat the preference as an array of outcomes, ranked highest to lowest.
    Put the median of $S$ at index $0$.
    Let the outcomes to the left of the median occupy the indices corresponding
    to set $T$, and let those to the right occupy the other indices.
    There are ${m-1 \choose \lfloor m/2\rfloor } \ge 2^{\Omega(m)}$ such
    subsets $T$.
  \end{proof}

  A cleaner, more constructive way to see this is the following proposition
  \begin{proposition}
    The preference set {\sc FlipFlop} is single-peaked, but not
    $2$-dimensional.
  \end{proposition}
  \begin{proof}
    To obtain a single-peaked representation, order the outcomes
    from left to right as follows: $3,1,2,4$.
    All preferences will then be possible with peak $1$ or $2$.

    Now we show that {\sc FlipFlop} is not $2$-dimensional.
    Note that neither $1$ nor $2$ can dominate the other,
    and the same holds for $3$ and $4$.
    Let $a,b$ satisfy $1=_a2$ and $3=_b4$.
    Suppose $a$ has angle smaller than $b$, the other case being symmetric.

    Consider the pairs $(1,2)$ and $(3,4)$ as they appear in the list in the
    upper bound section. Because all combinations of $1>2$ vs. $2>1$ and
    $3>4$ vs. $4>3$ occur, neither pair can be before or after the other in the
    list. This is a contradiction.

    ((REVISE IMPOSSIBILITY HALF))
  \end{proof}

  ((DISCUSS SETS WHICH ARE NEITHER SINGLE-PEAKED NOR 2-D))

  ((ARGUE THAT 2-D IS MORE NATURAL THAN SINGLE-PEAKED,
  EVEN THOUGH THERE ARE (ASYMPTOTICALLY) MORE SINGLE-PEAKED))
\section{Voting for $2$-dimensional preferences}

  \begin{theorem}
    The median-angle voting rule for $2$-dimensional preferences
    is incentive compatible.
  \end{theorem}
  \begin{proof}
    Let $u$ denote the median-angled preference vector under when all
    agents report their true preference.
    Suppose the manipulative voter $i$'s true preference $a$
    has angle less than $u$ (the other case is symmetric).
    The only way in which $i$ can change the chosen preference vector
    is to increase its angle, say to some $v$.
    Note that $u \in \cone(v,a)$.

    Suppose $y$ is the outcome chosen by the non-manipulated median
    angle preference $u$.
    Now, in order for $i$ to profit by his manipulation,
    there must be some $x$ such that
    $x >_a y$, $y >_u x$, and $x >_v y$.
    However, this is a contradiction, via the lemma about cones
    preserving agreed upon opinions.

    ((PROOF NEEDS REVISION))
  \end{proof}

  \begin{theorem}
    The median-angle social welfare rule satisfies independence of irrelevant
    alternatives
  \end{theorem}
  \begin{proof}
  \end{proof}

\section{$2$-dimensional entails a Condorcet winner}

  \begin{theorem}
    Among an odd number of $2$-dimensional preferences,
    there is always a Condorcet winner, which is selected
    by the median angle voting scheme
  \end{theorem}
  \begin{proof}
    Let $u$ be the median-angled preference vector, and let
    $x$ be the favorite outcome of $u$.
    Given some outcome $y\ne x$, if $y \ll x$, then
    $y$ is never preferred to $x$.
    So suppose neither $x$ nor $y$ dominate each other, and let
    $w$ be the vector such that $x =_w y$.
    On one side of $w$, preferences prefer $x$ to $y$,
    and on the other side, they prefer $y$ to $x$.
    Because $x$ is preferred by the median-angled voter,
    more than half of the preference vectors lie on the side of $w$
    which prefers $x$.

    ((PROOF NEEDS REVISION))
  \end{proof}

  We note the following important consequence of the previous theorem:
  if preferences are known to lie in $P_2(X)$ for some $X$,
  but no details about $X$ are known, then a voting protocol can simply
  ask voters for their list of preferences and look for the Condorcet
  winner (note that there is at most one Condorcet winner).
  Moreover, it hints that when agents act approximately via a weighted
  sum of two attributes, it is more likely that there is a Condorcet winner.
  ((NOTE: CONTEMPLATE THE FOLLOWING HOLE IN THE ABOVE LOGIC:
  WHAT IF AN AGENT STRATEGICALLY REPORTS A PREFERENCE THAT IS NOT TWO
  DIMENSIONAL FOR THE UNDERLYING SET X. FOR ONE, THE AGENT MY FORCE THE
  PROCEDURE TO FAIL BECAUSE IT WON'T HAVE A CONDORCET WINNER. BUT AD HOC
  WE DON'T KNOW HE CAN'T MANIPULATE THE CONDORCET WINNER))

\end{document}


\subsection{Papers read by Wei Chen}
The first paper was "Mining Large Graphs: Algorithms, Inference, and Discoveries".
\cite{DBLP:conf/icde/KangCF11}
\begin{itemize*}
\item {\em Main idea}:
This paper discuss how to do inference in graphical model under distributed setting, what if graph can not fit into main memory. They propose a variant of Belief Propagation called \emph{Line Graph Fixed Point(LFP)} to address this issue. The key step is to induce a new graph called \emph{Directed Line Graph} L(G) from original graph, it flips the definition of node and edge. Each node of L(G) represents an edge of G and edge exists between node $n_{i}$ and $n_j$ if and only if their related edges are incident. L(G) has a nice property that it's easy to derive exact recursive equation for it, actually the message updating equation of original Belief Propagation \cite{bp} can be adopted without modification. Then the author generalize mesage updating procedure as general Matrix-Vector operations, namely combine2(), combineAll(), sumVector() and assign() from \cite{Kang09}. While directly apply LFP without careful design will lead to drastic storage requirement to store L(G), the idea to tackle this is \emph{Lazy Multiplication}, all operations on L(G) is is actually run on G. They propose an algorithm employing this idea, and run on Hadoop, its name is \emph{Hadoop Line Graph Fixed Point}. Ha-LFP solves the biggest issue of the most graph mining algorithm --- scalability. Since Ha-LFP is based on Hadoop, it inherits fault tolerance, data replication natively.
\item {\em Use for our project}:
      LFP seems like a promising method to do Belief Propagation, combine2, combineAll, assign is natural in SQL. Matrix operation is tractable in SQL, and can be optimized by RBDMS. The performance could be comparable to Hadoop version. Also we don't need to construct L(G) explicitly, this avoids expensive cost in storage and time to access and update L(G).
\item {\em Shortcomings}:
      It's kind of tricky to say this algorithm solve scalability perfectly. I think this is more contributed to Hadoop instead of LFP. There may exist better formulation of Belief Propagation on Hadoop. The experiments are conducted on M45, which is one of the most advanced supercomputer.
\end{itemize*}

The second paper was "Understanding Belief Propagation and its Generalizations".
\cite{bp}
\begin{itemize*}
\item {\em Main idea}: Marginal Probability is important in graphical model inferencing, while most of the time it is expensive to calculate, the running time is usually exponential to the number of nodes. \emph{Belief Propagation} is a neat technique to bring the running time down to linear to the number of edges in graphical model, and it has been widely used beyong machine learning, like computer vision\cite{felzenszwalb2006efficient}, turbo code
\cite{mceliece1998turbo}, etc. The success of Belief Propagation results from a property called \emph{Conidtional Independence}. Most graphical models have the property that there exist many conditional independence between nodes. The core idea of Belief Propagation is to exploit this property to decouple the computation of global probability into several local computations. The marginal probability of node $n_{i}$ is called \emph{belief}. Each node $n_{i}$ sends messages to other nodes $n_{j}$. \emph{Message} $m_{ij}(n_{j})$ represents how node $n_{i}$ thinks node $n_{j}$'s state should be. A node can calculate its belief once it has received messages from all its neighbouts. So each edge is associate with two messages. The paper shows that each message can be calculated only once, thus we can use dynamic programming to compute every node's belief in linear time. The paper also introduce Generalized Belief Propagation which extends BP to group of nodes.
\item {\em Use for our project}:
      This paper discusses how to compute belief of each node in one round(for trees). We can first implement the algorithms mentioned in this paper as a baseline. It is also helpful for understanding other optimizing algorithms for Belief Propagation.
\item {\em Shortcomings}:
       There is not much discussion about the implementation in SQL. But user defined functions provided by PostgreSQL has limited expressiveness, so the real implementation will be quite different.
\end{itemize*}

The third paper was "A Comparison of Approaches to Large-Scale Data Analysis".
\cite{pavlo2009comparison}
\begin{itemize*}
\item {\em Main idea}:
    This paper compares the tradeoff between Parallel DBMS and MapReduce framework in large data analysis. The authors want to advocate the capability of Parallel DBMS in implementing large scale analytic tasks. The paper analyze the fundamental difference of Parallel DBMS and MapReduce, their support in schema, data distribution, fault tolerance, index, and utility tools. Then they conduct experiments about their performance in different tasks, like pattern searching, aggregation, join, etc. They conclude that Parallel DBMS outperforms MapReduce in all kinds of tasks. But Parallel DBMS requires a lot of effort to configure and profile to reach good performance, MapReduce is relatively much easier to use. In the end, the paper admit MapReduce's advantage over complex Parallel DBMS, while still suggest Parallel DBMS as an option for specific tasks.
\item {\em Use for our project}:
    We can borrow some ideas from theirs experiments about investigate SQL function. Their analysis method is also a good model to follow.
\item {\em Shortcomings}:
    The tasks they experimented are kind of trivial. We don't know the real difference in when the systems are used for more complex graph mining algorithms.
\end{itemize*}

The fourth paper was "Pig Latin: A Not-So-Foreign Language for Data Processing".
\cite{olston2008pig}
\begin{itemize*}
\item {\em Main idea}:
    Either RDBMS or MapReduce framework represents some extreme in large scale data analysis, either unnatural for programmers' mind(SQL) or too low level to express many algorithms(MapReduce). Different solutions are proposed. Pig is a platform of Yahoo that try to reach a balance between these two styles to support more flexible(ad-hoc) data analysis tasks. It takes a hybrid strategy, provides friendly user interface(programming, debugging), while actual tasks are running on powerful Hadoop platform without loss of performance. And it propose a new high level programming language called \emph{Pig Latin} that gives user more control of data flow, which programmer can adopt an imperative style, while retain the expressiveness of SQL. It also support several key operations that can be automatically parallelized, like filter, cogroup, group, join, etc. Its underlying implementation is compile Pig Latin programs into Hadoop jobs, so that it is also equipped with the benefits of MapReduce.
\item {\em Use for our project}:
    Pig Latin's SQL-like syntax provides finer grained control over dataflow, which is more suitable for implementing graph mining algorithms. We can compare with SQL user defined function implementation of these algorithms, and analyze their intrinsic data accessing characteristic.
\item {\em Shortcomings}:
    There are few graph mining algorithms implementation in Pig Latin, only a few open source analysis package like Linkedin's Datafu. And Pig only supports read-only data analysis workloads, which may lead to many unnecessary efforts if we want to do extensive write during computation.
\end{itemize*}

The fifth paper was "Fast counting of triangles in large real networks without counting: Algorithms and laws" \cite{tsourakakis2008fast}.
\begin{itemize*}
\item {\em Main idea}:
    The paper proposed a analytical method to count the number of triangles in social networks. It is based upon two theorems regarding the fundamental property of adjencency matrix. The first theorem is that the number of global triangles is proportional to the sum of cubes of its eigenvalues, which is more formally defined as follow:
      \begin{equation}
      \Delta(G) = \frac{1}{6}\sum_{i=1}^{n} \lambda_{i}^{3}
      \end{equation}
    Another one is about local triangle, the formula for its count is as follows:
      \begin{equation}
      \Delta_{i} = \frac{\sum_{j} u_{ij}^{2} \lambda_{j}^{3}}{2}
      \end{equation}
    The author conducted several experiments on different datasets. and discovered several patterns using visualization, such as power law.
\item {\em Use for our project}:
    For task 7, we adopt the algorithms described in the paper to count the number of global triangles and local triangles. The algorithm to calculate eigenvalues and eigenvectors is Lanczos, which is well suited for solving approximate top-k eigenvalues and eigenvectors.
\item {\em Shortcomings}:
    The paper didn't discuss the selection of k, which is the number of top eigenvalues use for calculation. So most implementations have to randomly pick a small number and verify its effect. And how the magnitude of k will affect the running time of the algorithm.
\end{itemize*}


\documentclass[11pt]{beamer}
\let\Tiny=\tiny
\let\epsilon=\varepsilon
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{array}
\usepackage{tikz}
\usepackage{framed}

\definecolor{cherryred}{RGB}{214,31,74}
\definecolor{sandiasilver}{RGB}{204,204,204}
\definecolor{lightblue}{RGB}{240,245,255}

\usecolortheme[named=cherryred]{structure}
\usefonttheme[onlymath]{serif}
\setbeamertemplate{navigation symbols}{}
\setbeamertemplate{blocks}[shadow=true]
\setbeamersize{sidebar width left=0.2cm}
\setbeamercolor{sidebar}{bg=sandiasilver}
\setbeamercolor{frametitle}{fg=white, bg=cherryred}
\setbeamertemplate{theorems}[numbered]

\author{David E. Weirich}
\title{Neural Networks: \\ The Hard Way}
\date{\today}

\newcommand{\R}{\mathbb{R}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\D}{\mathcal{D}}
\newcommand{\A}{\mathcal{A}}
\newcommand{\B}{\mathfrak{B}}
\newcommand{\Dh}{\tilde{\mathcal{D}}}

\newcommand{\supp}{\operatorname{Support}}
\newcommand{\ch}{\operatorname{ch}}
\newcommand{\proj}{\operatorname{Proj}}
\newcommand{\spn}{\operatorname{span}}
\newcommand{\quadr}{\operatorname{Quad}}
\newcommand{\qak}{Q_\alpha^k}
\newcommand{\one}{\textbf{1}} % Get the better looking one working!

\newcommand{\mean}[2]{\langle #2 \rangle_{#1}}

\newtheorem{conjecture}[theorem]{Conjecture}

\newcommand{\zak}{z_\alpha^k}

\newcommand{\bfred}[1]{{\color{cherryred} \textbf{#1}}}
\begin{document}

\begin{frame}
\titlepage
\end{frame}

\begin{frame}{Talk Objectives}

\begin{itemize}
\item Understand the mathematical basis for artificial neural networks.

\item Use that knowledge to design and build a neural network without relying on preexisting ML frameworks.
\end{itemize}

\bigskip

\pause

Or in other words...

\bigskip

\pause

\hspace{10mm}\textbf{Learn Neural Networks THE HARD WAY}

\end{frame}

\begin{frame}{Disclaimer}
\begin{center}
\includegraphics[scale=0.18]{ThereWillBeMath}
\end{center}
\end{frame}

\section{What are these Neural Networks Things?}

\begin{frame}{Why the ``hard way''??? I prefer easy!}

\emph{What do I mean by "the hard way"?}

\bigskip

\bigskip

Doing things the hard way means teaching you how to teach yourself.

\end{frame}

\begin{frame}{The Problem}

We wish to approximate a function given a finite collection of known inputs and outputs.

\bigskip

$$||F - \widehat{F}|| < \epsilon$$

\bigskip

Here $F$ is the true function, and $\widehat{F}$ is our approximation.
\end{frame}

\begin{frame}{What are Neural Networks?}
\textbf{Q:} What is a neural network?

\bigskip

\textbf{A:} \emph{A neural network is this super confusing thing that is kind of like an artificial brain.}

\bigskip

\textbf{Q:} Why do we care?

\bigskip

\textbf{A:} \emph{It can learn anything, do anything, and no one really understands them.}
\end{frame}

\begin{frame}{Neurons}

\begin{center}
\begin{figure}
\includegraphics[scale=0.2]{Blausen_0657_MultipolarNeuron}
\caption{A neuron. These things are involved somehow?}
\end{figure}
\end{center}

\end{frame}

\begin{frame}{Pictures that look like this}

\begin{center}
\begin{figure}
\includegraphics[scale=0.2]{0_IlHu39jf2c7QC4kn}
\hspace{5mm}
\includegraphics[scale=0.2]{topdown}

\vspace{5mm}

\includegraphics[scale=0.25]{296px-Colored_neural_network}
\hspace{5mm}
\includegraphics[scale=0.15]{Feed-forward-neural-network-with-two-hidden-layers}
\caption{Some diagrams with circles and arrows.}
\end{figure}
\end{center}

\end{frame}

\begin{frame}{A Motivating Example: Classifying Digits}
Suppose you were the kind of person who wanted to identify some pictures of numerical digits.

\begin{center}

\includegraphics[scale=0.4]{digits}

\pause

\includegraphics[scale=0.4]{digits2}

\pause

\includegraphics[scale=0.4]{digits3}

\end{center}
\end{frame}

\begin{frame}{The $\Sigma$exy Part of the Talk}
Let's take a look at the math under the hood:
\begin{align*}
x &= \left[\begin{array}{c}
x_1 \\
x_2 \\
\vdots \\
x_N
\end{array}\right] \\
\widehat{y} &= \widehat{F}(x) = W_2\sigma(W_1\sigma(W_0x + b_0) + b_1) + b_2
\end{align*}

where $W_i$ is a weight matrix, $b_i $ is a bias vector, and $\sigma$ is an activation function.
\end{frame}

\begin{frame}{Breaking that down}

Let's look at that again, step by step:

\begin{align*}
z_0 &:= W_0x + b_0 \\
a_0 &:= \sigma(z_0) \\
z_1 &:= W_1a_0 + b_1 \\
a_1 &:= \sigma(z_1) \\
&\vdots \\
\widehat{y} &:= W_n a_{n-1} + b_n
\end{align*}

\end{frame}

\begin{frame}{Universal Approximation Theorem}
\begin{theorem}[Cybenko (1989), Hornik (1991)]
Let $\sigma$ be a nonconstant, bounded, and monotonically increasing function.
Given any $\epsilon > 0$ and any continuous function $F$ defined on a compact subdomain $\Omega$ of $\R^n$ there exists a constant $N$, real constants $v_i$ and $b_i$, and real vectors $w_i$ such that we may define
$$\widehat{F}(x) := \sum_{i = 1}^N v_i \sigma(w_i^T \cdot x + b_i)$$
with
$$\left|F(x) - \widehat{F}(x)\right| < \epsilon$$
for all $x$ in $\Omega$.
\end{theorem}
\end{frame}

\section{Training}
\begin{frame}{Okay, fine, I get it. But how do I set those weights?}

For $M$ observations $(x_i, y_i)$, define a loss function

\begin{align*}
J(y_i, \widehat{y}) &:= \frac{1}{2M}\sum_{i =1}^N (y - \widehat{y})^2 \\
&= \frac{1}{2M}\sum_{i =1}^N \left(y - \widehat{F}(x_i)\right)^2 \\
&= \frac{1}{2M}\sum_{i =1}^N \left(y - \widehat{F}(x_i; W, b)\right)^2
\end{align*}
Now we just have to minimize the loss!
\end{frame}

\begin{frame}{The Gradient}
Recall from multivariable calculus that the \emph{gradient} is a vector which points in the direction of steepest assent on a surface.
\begin{center}
\begin{figure}
\includegraphics[scale=0.25]{720px-Gradient_Visual}
\caption{A function of two variables, with its gradient.}
\end{figure}
\end{center}
\end{frame}

\begin{frame}{Calculating the Gradient}

Normally we would use an algorithm called ``Back Propogation\footnote{Or as I like to call it, the chain rule with clever bookkeeping.}'' to calculate the gradients.

\bigskip

But I just did it this way:

\begin{center}
\includegraphics[scale=0.04]{20180305_112255}
\end{center}

\end{frame}

\begin{frame}{The Game Plan}
\begin{itemize}
\item We start by setting the weights and biases of the NN randomly.
\item Calculate the gradient of the loss function $J$ with respect to these parameters.
\item Update the parameters a tiny bit by taking a step in the direction of the negative gradient.
\item Repeat until the error is acceptable.
\end{itemize}
\end{frame}

\section{The Code}

\begin{frame}{WE'LL DO IT LIVE}
\begin{center}
\includegraphics[scale=0.3]{114486a48d800d14f972cb0e74f7b0d9}

WE'LL DO IT LIVE!
\end{center}
\end{frame}

\section{End}

\begin{frame}{Concluding Remarks}

\begin{itemize}
\item Neural networks are powerful because they combine the ability to approximate any function with a systematic method for improving that approximation.
\item The loss function measures how well a given NN matches the desired function.
\item Gradient descent is the process by which we slowly nudge the parameters of the network in the proper direction in an attempt to minimize the loss function.
\end{itemize}

\end{frame}

\begin{frame}
\begin{center}
Thank you! :)
\end{center}
\end{frame}

\end{document}


\XtoCBlock{LookupTable}
\label{block:LookupTable}
\begin{figure}[H]\includegraphics{LookupTable}\end{figure}

\begin{XtoCtabular}{Inports}
In & Table index\tabularnewline
\hline
\end{XtoCtabular}

\begin{XtoCtabular}{Outports}
Out & Table output\tabularnewline
\hline
\end{XtoCtabular}

\begin{XtoCtabular}{Mask Parameters}
Lookup & Look-up Table\tabularnewline
\hline
\end{XtoCtabular}

\subsubsection*{Description:}
Look-up Table with 256+1 values.

  Note: 257th value is used for preventing index overflow during interpolation.

      -> for periodic signals the 257th value should be set equal to 1st value

      -> for non-periodic signals the 257th value should be set equal to 256th value

\subsubsection*{Implementations:}
\begin{tabular}{l l}
\textbf{FiP8} & 8 Bit Fixed Point Implementation\tabularnewline
\textbf{FiP16} & 16 Bit Fixed Point Implementation\tabularnewline
\textbf{FiP32} & 32 Bit Fixed Point Implementation\tabularnewline
\end{tabular}

\XtoCImplementation{FiP8}
\index{Block ID!160}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP8 \tabularnewline
\textbf{ID} & 160 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & LookupTable\_FiP8.c \tabularnewline
\textbf{H filename} & LookupTable\_FiP8.h \tabularnewline
\end{tabular}
\vspace{1ex}

8 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
Table & Lookup table content\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int8          *In;
     int8          Out;
     const int8    *Table;
} LOOKUPTABLE_FIP8;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/General/Doc/Test_LookupTable_FiP8.tex}{}{}
\fi
\XtoCImplementation{FiP16}
\index{Block ID!161}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP16 \tabularnewline
\textbf{ID} & 161 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & LookupTable\_FiP16.c \tabularnewline
\textbf{H filename} & LookupTable\_FiP16.h \tabularnewline
\end{tabular}
\vspace{1ex}

16 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
Table & Lookup table content\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int16         *In;
     int16         Out;
     const int16   *Table;
} LOOKUPTABLE_FIP16;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/General/Doc/Test_LookupTable_FiP16.tex}{}{}
\fi
\XtoCImplementation{FiP32}
\index{Block ID!162}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP32 \tabularnewline
\textbf{ID} & 162 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & LookupTable\_FiP32.c \tabularnewline
\textbf{H filename} & LookupTable\_FiP32.h \tabularnewline
\end{tabular}
\vspace{1ex}

32 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
Table & Lookup table content\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int32         *In;
     int32         Out;
     const int32   *Table;
} LOOKUPTABLE_FIP32;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/General/Doc/Test_LookupTable_FiP32.tex}{}{}
\fi


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{movcw4at}
\section*{\hspace*{-1.6cm} movcw4at}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Four atoms rotating, analyzed by the Choi-Williams distribution.
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
M = movcw4at(N)
M = movcw4at(N,Np)
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty movcw4at} generates the movie frames illustrating the
influence of an overlapping in time and/or frequency of different
components of a signal on the interferences of the Choi-Williams
distribution between these components.\\

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
        {\ty N}  & number of points of the analyzed signal\\
        {\ty Np} & number of snapshots & {\ty 7}\\
\hline  {\ty M}  & matrix of movie frames\\
\hline
\end{tabular*}

\end{minipage}
\vspace*{1cm}

{\bf \large \sf Example}
\begin{verbatim}
          M=movcw4at(128,15);
          movie(M,10);
\end{verbatim}
\vspace*{.5cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
movpwjph, movpwdph, movsc2wv, movsp2wv, movwv2at.
\end{verbatim}
\end{minipage}


\documentclass{article}
\usepackage{amsmath, amssymb}

\title{Towards a Neural Game Theory}

\author{Eric Purdy}

\begin{document}

\maketitle

\section{Some puzzles}

\begin{itemize}
\item Why is it that approximately 50\% of people vote?
\item Why is Bitcoin worth money?
\end{itemize}

\section{Toy model of voting}

Imagine that we have $n$ voters and $k$ agent fragments, with $n \gg k$.

Let us denote the agent fragment values by $x \in \mathbb{R}^k$.

We posit the following model, which for reasons that will become obvious we
call the Brightside Model. Let $\mathrm{eager}_i$ be the eagerness of agent $i$
to vote, expressed as a probability that the agent will vote. Let
$\mathrm{opinion}_i$ be the opinion of the agent, expressed on an axis from $1$
(reasonable) to $-1$ (shitty).
\begin{align*}
\mathrm{eager}_i &= \sigma(A x + a) \\
\mathrm{opinion}_i &= \tanh(B x + b) \\
\intertext{We then have that the expected net vote is}
\mathbb{E}\left[\mathrm{vote}_\mathrm{net}\right] &= \sum_i \mathrm{eager}_i \cdot \mathrm{sign} (\mathrm{opinion}_i) \\
\intertext{and the expected outcome of the election is}
\mathrm{outcome} &= \mathrm{sign} \left(\mathrm{vote}_\mathrm{net} \right) \\
\intertext{and the expected reward of agent $i$ is then}
\mathbb{E} \left[ r_i \right] &= -\mathrm{eager}_i + 10 \cdot \mathrm{opinion}_i \cdot \mathrm{outcome}  \\
\intertext{We then consider the following problem:}
\max_{A_i, a_i} \min_{x} \mathbb{E} \left[ r_i \right]
\intertext{where $A_i, a_i$ refer to the portions of $A$ and $a$ that are under the control of agent $i$}
\end{align*}
We find that optimizing this objective via block gradient descent produces
populations of agents that vote with probability that often settles near 50 \%.

Note that we use a differentiable variant of the sign function:
\begin{align*}
\mathrm{sign}(\theta) &= \tanh \left(\theta / \epsilon \right)\\
\end{align*}
where $\epsilon$ is chosen to be $1e-6$.

\section{Toy model of the stock market}

TODO. Note that simulations are currently producing prices that sometimes
spiral off to huge values or tiny values; how do we incorporate real-world
limits on prices?

\section{Simulation results}

\begin{itemize}
\item If we assign random values to $x$ rather than minimizing the expected
  reward with respect to $x$, almost nobody votes.
\item If we maximize the expected reward with respect to $x$, very few people
  vote.
\item If we minimize the expected reward with respect to $x$, an average of
  $39.6\%$ people vote.
\end{itemize}

\section{Theoretical considerations}

\subsection{What does $x$ represent?}

$x$ seems to represent ``what everyone knows'' or ``social facts''. That is,
things that have no existence independent of the minds of the agents, but which
nevertheless have an existence that cannot be willed away by any single agent.

\subsection{Why minimize over $x$?}

It sort of makes sense if you think about it. We are maximizing our expected
reward given the contents of everyone else's heads; we are pessimistic about
the contents of everyone else's heads. It is thus a fairly standard application
of the minimax rule: act so that you will do well given the worst-case value of
every option.

\subsection{How do people know the values they would need to know?}

We suspect that there are some simple sufficient statistics that people could
plausibly know. Need to calculate the gradients by hand to check this out
though.

\begin{align*}
\frac{\partial \mathbb{E}\left[ r_i \right]}{\partial x} &=
-\frac{\partial \mathrm{eager}_i}{\partial x} + 10 \cdot \frac{\partial \mathrm{opinion}_i}{\partial x} \cdot \mathrm{outcome} + 10 \cdot \mathrm{opinion}_i \cdot \frac{\partial \mathrm{outcome}}{\partial x} \\
&= -\mathrm{eager}_i \left(1 - \mathrm{eager}_i \right) A_i +
10 \cdot \left( 1 - \mathrm{opinion}_i ^ 2 \right) \cdot \mathrm{outcome} \\
& \qquad + 10 \cdot \mathrm{opinion}_i \cdot \left(1 - \frac{\mathrm{outcome}^2}{\epsilon^2} \right)
\frac{\partial \mathrm{vote}_\mathrm{net}}{\partial x}\\
\intertext{and}
\frac{\partial \mathrm{vote}_\mathrm{net}}{\partial x} &=
\sum_j \left[ \frac{\partial \mathrm{eager}_j}{\partial x} \cdot \mathrm{sign} \left(\mathrm{opinion}_j\right) +
\mathrm{eager}_j \cdot \frac{\partial \mathrm{sign} \left(\mathrm{opinion}_j\right)}{\partial x} \right] \\
&= \sum_j \left[ \mathrm{eager}_j (1 - \mathrm{eager}_j) A_j \cdot \mathrm{sign} \left(\mathrm{opinion}_j\right) +
\mathrm{eager}_j \cdot \left(1 - \frac{\mathrm{opinion}_j^2}{\epsilon^2}\right) B_j \right] \\
\end{align*}

\subsection{But people have free will!}

The voting model is consistent with free will, since there is a free bias
parameter for every agent. Every agent is thus free to adjust that parameter so
that they almost always vote or almost always don't vote. Nevertheless, having
to pick a behavior other than the default behavior is more difficult and
therefore less likely in our simulations.

\end{document}