\documentclass[10pt,conference,compsocconf]{IEEEtran}

\usepackage{hyperref}
\usepackage{graphicx}	% For figure environment

% for text in math mode
\usepackage{amsmath}

% for table
\usepackage{tabu}
\usepackage{tabularx}
\usepackage{multirow}

\begin{document}
\title{Higgs Boson Machine Learning Challenge}

\author{
  Cheng Chun Lee, I Made Sanadhi Sutandi, Haziq Razali\\
  \textit{School of Computer and Communication Science, EPFL}
}

\maketitle

\begin{abstract}
  The discovery of the Higgs-Boson in 2012 was a major breakthrough in particle physics and it was the combination of effort by inter-discipline join of physicists and data scientists, as an attempt to better understand this particle. In this paper, we tackle the problem of identifying the Higgs-Boson through the implementation of machine learning techniques. We investigate the usage of linear models to discriminate between the Higgs-Boson (signal) and background. Coupled with data analysis, we show the efficacy of our pipeline on the Higgs-Boson dataset with promising results.
\end{abstract}

\section{Introduction}

The Higgs-Boson is the first elementary particle discovered in nature through experiments at the Large Hadron Collider. Our aim in this project is to develop a learning system capable of identifying the Higgs-Boson given its decay signature. To achieve this, we use the CERN's public dataset consists of 250,000 observations, each with a vector of 30 features representing  the decay signature of a collision event. The observations are labeled 1 for an actual event and -1 for background noise.

\section{Pre-processing}

\subsection{Data partitioning}

Initial exploratory data analysis reveals an interesting structure within the dataset: that there are several features that tightly couped by the value of PRI\_JET\_NUM (feature 22). The number of jet emissions, ranging from 0 to 3, dictate the availability of specific readings with outliers value of being designated the integer -999. It would thus make sense to posit that the number of jets emitted will result in different readings of features. Consequently, we partitioned the dataset into 4 disjoint subsets according to the emission count, removing all missing entries including the jet number. In addition to this partitioning, we also figure out that the predicitons most probably are background noises if outliers are present in DER\_MASS\_MMC (feature 1). Therefore we split each subset again into two subsets based on the presence of outliers in this feature, giving us 8 subsets in total accordingly.

\subsection{Features Removal}
We plot of a histogram of all variables and split according to the observation (signal/background). Then we find that the variables 'phi' to have no distinct pattern (Figure 1). We thus suspect that these variables do not provide useful information and decide to remove all phi related variables from the dataset.

\begin{figure}[t]
\begin{center}
  \includegraphics[width=\columnwidth]{feature.jpg}
  \caption{Histogram of the variable PRI\_lep\_phi for true observations (top) and background (bottom).}
\end{center}
   \label{fig:phi}
\end{figure}

\subsection{Feature Expansion}
Next, we augment all 8 feature matrices with non-linear transformations of the raw data to increase the expressive power of our models. More specifically, for each matrix ${\textbf X_i}$ we concatenate as follows:
\vspace*{-1mm}
\begin{equation}
{\textbf X} \leftarrow \big[ \ {\textbf X} \ \vert \ {\textbf A} \ \vert \ {\textbf B} \ \vert \ {\textbf C} \ \vert \ {\textbf d} \ \vert \ {\textbf E} \ \vert \ {\textbf F} \ \big]
\end{equation}
\vspace*{-5mm}
\begin{equation}
a_{ij} = x_{ij}^2, \ \ b_{ij} = \log (1 + x_{ij}), \ \ c_{ij} = b_{ij}^{-1}, \ \ d_{i} = 1
\end{equation}
\vspace*{-5mm}
\begin{equation}
E = (X)*(X), \ \ f_{ij} = \sqrt{x_{ij}}
\end{equation}

bringing the size of each matrix from $ (N \times m_i) $ to $ (N \times (6m_i + 1))$. A drawback of feature augmentation is the increased tendency for models to overfit which we can reduce via regularization. Note that we have also append a column vector of ones to the final matrix to act as the bias term present in linear models.

\begin{figure*}[t]
\begin{center}
  \includegraphics[width=\textwidth]{box1.jpg}
  \caption{Box plot distributions of our models.}
\end{center}
   \label{fig:boxplot}
\end{figure*}

\subsection{Standardization}
Learning algorithms often benefit from the many standardization techniques to prevent any particular feature from dominating the objective function. In our implementation, we standardize our data using standard score (z-score) normalization in which by scaling each feature to have zero-mean and unit-variance.

\section{Models}

We experiment with the following linear models: Least Squares and Logistic Regression with $L_2$ regularization. All weights are initialized by sampling from a normal distribution with zero-mean and unit variance. Recall from section II that the data was split into 8 subsets; our algorithm thus builds 8 separate classifiers trained on each subset. Since we define 5000 iterations for each subset, we use small value of step-size of 0.000002 in order to best compromise between stability and time. Smaller step size ensures we oscillate closely to the global minimum. We then train all models using gradient descent with a total of 5000x8 iterations. We list all parameters in Table 1. At test time, we simply process the input data as described in section II and select the appropriate classifier for discrimination.

\begin{table}[t]
 	\small
	\begin{tabu} to \columnwidth { | X[l] | X[c] | X[0.9c] |}
	    \hline
		\multirow{2}{*}{\textbf{Dataset}} & \multicolumn{2}{c|}{\textbf{Accuracy (\%) }} \\
		\cline{2-3}
		& Logistic Regression & Least Squares \\
		\hline
		Jet 0 with mass & 80.836 & 80.458 \\
		\hline
		Jet 0 without mass & 95.050 & 95.035 \\
		\hline
        Jet 1 with mass & 79.499 & 78.312 \\
		\hline
        Jet 1 without mass & 92.290 & 92.542 \\
		\hline
        Jet 2 with mass & 83.526 & 82.187 \\
		\hline
        Jet 2 without mass & 92.107 & 92.988 \\
		\hline
        Jet 3 with mass & 82.675 & 81.476 \\
		\hline
        Jet 3 without mass & 94.177 & 96.208 \\
		\hline
        Ensemble & 83.167 & 82.399 \\
		\hline
		\end{tabu}
	\medskip
	\caption{The accuracies of each component in the ensemble along with their aggregated scores}
\end{table}

\section{Experiments}

We evaluate the above-mentioned models on the Higgs-Boson dataset using 10-fold cross validation. Since we perform several pre-processing steps, it is reasonable to evaluate how much the overall result depends on each component. In our first experiment, we study the improvements obtained when we gradually append additional steps in our pre-processing pipeline. Summarized in Figure 2 are the box-plot distributions of our models, each with additional components gradually added. Note that the figures on the left and middle display the results when using only a single classifier i.e. we do not partition the data as described in 2A. Also note that their features have been standardized taking only non missing entries into account with these entries then set to 0. The figure on the right display the results of the complete system as discussed in previous sections.

These results show fascinating studies. Firstly, we see that both models trained on the standardized raw-data are quite close in their prediction accuracy. The removal of all phi-related features provides a small boost in performance. The figure also clearly illustrates the improvements obtained when training the models on the augmented feature matrix with an approximate increase of 4\% in the accuracy of both classifiers, suggesting the presence of a non-linear decision boundary. Lastly, we can observe that training an ensemble of classifiers on the 8 subsets has a positive influence on the overall performance.

We then measure the performance of each component in the ensemble and their overall accuracy, weighted by the size of the subsets. It is interesting to see that the classifiers for datasets without mass have higher accuracy, although we have already found that these datasets have a majority of their observations labeled as background.

From all the experiments, we note that the least squares classifier lags behind the logistic in terms of accuracy. This is most likely due to the cost function in least squares that renders it less resilient against outliers. Lastly, the deviations in the accuracy also implies that the classifiers did not over-fit the training data.

\section{Conclusion}

We have presented a technique for identifying the Higgs-Boson. Our system employs an ensemble of 8 classifiers that heavily relies on feature processing. We saw in the previous section that the largest improvement is obtained when we map the features into a higher dimensional space. Possible extensions can thus incorporate the use of highly non-linear classifiers such as neural networks. The inclusion of domain specific feature engineering could also prove worthwhile.

\bibliographystyle{IEEEtran}
\bibliography{literature}

\end{document}


% ----------------------------------------------------------------------
% $Id: cgb.tex 84 2009-02-07 07:53:22Z thomas-sturm $
% ----------------------------------------------------------------------
% Copyright (c) 1999-2009 Andreas Dolzmann and Thomas Sturm
% ----------------------------------------------------------------------
% Redistribution and use in source and binary forms, with or without
% modification, are permitted provided that the following conditions
% are met:
%
%    * Redistributions of source code must retain the relevant
%      copyright notice, this list of conditions and the following
%      disclaimer.
%    * Redistributions in binary form must reproduce the above
%      copyright notice, this list of conditions and the following
%      disclaimer in the documentation and/or other materials provided
%      with the distribution.
%
% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
% "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
% LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
% A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
% OWNERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
% SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
% LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
% DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
% THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
% (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
% OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
%

\documentstyle{article}
\overfullrule2mm
\newcommand{\C}{{\bf C}}
\newcommand{\R}{{\bf R}}
\newcommand{\Id}{{\rm Id}}
\begin{document}
\title{CGB: Computing Comprehensive Gr\"obner Bases}
\author{Andreas Dolzmann \& Thomas Sturm\\
Department of Mathematics and Computer Science\\ University of Passau\\
D-94030 Passau, Germany\\[1ex]
e-mail: {\tt dolzmann@uni-passau.de}, {\tt sturm@uni-passau.de}\\[1ex]
and\\[1ex]
Winfried Neun\\
Konrad-Zuse-Zentrum f\"ur Informationstechnik Berlin\\
Takustra\ss{}e 7\\
D-14195 Berlin-Dahlem, Germany\\[1ex]
e-mail: {\tt neun@zib.de}}
\date{15 April 1999}
\maketitle

\section{Introduction}
Consider the ideal basis $F=\{ax,x+y\}$. Treating $a$ as a parameter,
the calling sequence
\begin{verbatim}
torder({x,y},lex)$
groebner{a*x,x+y};

  {x,y}
\end{verbatim}
yields $\{x,y\}$ as reduced Gr\"obner basis. This is, however, not
correct under the specialization $a=0$. The reduced Gr\"obner basis
would then be $\{x+y\}$. Taking these results together, we obtain
$C=\{x+y,ax,ay\}$, which is correct wrt.~{\em all} specializations for
$a$ including zero specializations. We call this set $C$ a {\em
comprehensive Gr\"obner basis} ({\sc cgb}).

The notion of a {\sc cgb} and a corresponding algorithm has been
introduced bei Weispfenning \cite{Weispfenning:92}. This algorithm
works by performing case distinctions wrt.~parametric coefficient
polynomials in order to find out what the head monomials are under all
possible specializations. It does thus not only determine a {\sc cgb},
but even classifies the contained polynomials wrt.~the specializations
they are relevant for. If we keep the Gr\"obner bases for all cases
separate and associate information on the respective specializations
with them, we obtain a {\em Gr\"obner system}. For our example, the
Gr\"obner system is the following;
$$
\left[
\begin{array}{c|c}
a\neq0 & \{x+y,ax,ay\}\\
a=0 & \{x+y\}
\end{array}
\right].
$$
A {\sc cgb} is obtained as the union of the single Gr\"obner bases in
a Gr\"obner system. It has also been shown that, on the other hand, a
Gr\"obner system can easily be reconstructed from a given {\sc cgb}
\cite{Weispfenning:92}.

The CGB package provides functions for computing both {\sc cgb}'s and
Gr\"obner systems, and for turning Gr\"obner systems into {\sc cgb}'s.
%
\section{Using the REDLOG Package}
For managing the conditions occurring with the {\sc cgb} computations,
the CGB package uses the package REDLOG implementing first-order
formulas, \cite{DolzmannSturm:97a,DolzmannSturm:99}, which is also
part of the \textsc{reduce} distribution.
%
\section{Term Ordering Mode}
The CGB package uses the settings made with the function {\tt torder}
of the GROEBNER package. This includes in particular the choice of the
main variables. All variables not mentioned in the variable list
argument of {\tt torder} are parameters. The only term ordering modes
recognized by \textsc{cgb} are {\tt lex} and {\tt revgradlex}.
%
\section{CGB: Comprehensive Gr\"ob\-ner Basis}
The function {\tt cgb} expects a list $F$ of expressions. It returns a
{\sc cgb} of $F$ wrt.~the current {\tt torder} setting.
%
\subsection*{Example}
\begin{verbatim}
torder({x,y},lex)$
cgb{a*x+y,x+b*y};

{x + b*y,a*x + y,(a*b - 1)*y}

ws;

{b*y + x,

 a*x + y,

 y*(a*b - 1)}
\end{verbatim}
Note that the basis returned by the {\tt cgb} call has not undergone
the standard evaluation process: The returned polynomials are ordered
wrt.~the chosen term order. Reevaluation changes this as can be seen
with the output of {\tt ws}.
%
\section{GSYS: Gr\"obner System}
The function {\tt gsys} follows the same calling conventions as {\tt
cgb}. It returns the complete Gr\"obner system represented as a nested
list
\begin{center}
\begin{tt}
$\bigl\{\bigl\{c_1,\{g_{11},\ldots,g_{1n_1}\}\bigr\},\dots,
\bigl\{c_m,\{g_{m1},\dots,g_{1n_m}\}\bigr\}\bigr\}$.
\end{tt}
\end{center}
The {\tt $c_i$} are conditions in the parameters represented as
quantifier-free REDLOG formulas. Each choice of parameters will obey
at least one of the {\tt $c_i$}. Whenever a choice of parameters obeys
some {\tt $c_i$}, the corresponding {\tt $\{g_{i1},\ldots,g_{in_i}\}$}
is a Gr\"obner basis for this choice.
%
\subsection*{Example}
\begin{verbatim}
torder({x,y},lex)$
gsys {a*x+y,x+b*y};

{{a*b - 1 <> 0 and a <> 0,

  {a*x + y,x + b*y,(a*b - 1)*y}},

 {a <> 0 and a*b - 1 = 0,

  {a*x + y,x + b*y}},

 {a = 0,{a*x + y,x + b*y}}}
\end{verbatim}
As with the function {\tt cgb}, the contained polynomials remain
unevaluated.

Computing a Gr\"obner system is not harder than computing a {\sc cgb}.
In fact, {\tt cgb} also computes a Gr\"obner system and then turns it
into a {\sc cgb}.

\subsection{Switch CGBGEN: Only the Generic Case}
If the switch {\tt cgbgen} is turned on, both {\tt gsys} and {\tt
cgb} will assume all parametric coefficients to be non-zero ignoring
the other cases. For {\tt cgb} this means that the result equals---up
to auto-reduction---that of {\tt groebner}. A call to {\tt gsys} will
return this result as a single case including the assumptions made
during the computation:
%
\subsection*{Example}
\begin{verbatim}
torder({x,y},lex)$
on cgbgen;
gsys{a*x+y,x+b*y};

{{a*b - 1 <> 0 and a <> 0,

  {a*x + y,x + b*y,(a*b - 1)*y}}}

off cgbgen;
\end{verbatim}
%
\section{GSYS2CGB: Gr\"obner System to CGB}
The call {\tt gsys2cgb} turns a given Gr\"obner system into a {\sc
cgb} by constructing the union of the Gr\"obner bases of the single
cases.
%
\subsection*{Example}
\begin{verbatim}
torder({x,y},lex)$
gsys{a*x+y,x+b*y}$
gsys2cgb ws;

{x + b*y,a*x + y,(a*b - 1)*y}
\end{verbatim}
%
\section{Switch CGBREAL: Computing over the Real Numbers}\label{cgbreal}
All computations considered so far have taken place over the complex
numbers, more precisely, over algebraically closed fields. Over the
real numbers, certain branches of the {\sc cgb} computation can become
inconsitent though they are not inconsistent over the complex numbers.
Consider, e.g., a condition $a^2+1=0$.

When turning on the switch {\tt cgbreal}, all simplifications of conditions
are performed over the real numbers. The methods used for this are
described in \cite{DolzmannSturm:97c}.
%
\subsection*{Example}
\begin{verbatim}
torder({x,y},lex)$
off cgbreal;
gsys {a*x+y,x-a*y};

   2
{{a  + 1 <> 0 and a <> 0,

                     2
  {a*x + y,x - a*y,(a  + 1)*y}},

              2
 {a <> 0 and a  + 1 = 0,{a*x + y,x - a*y}},

 {a = 0,{a*x + y,x - a*y}}}

on cgbreal;
gsys({a*x+y,x-a*y});

{{a <> 0,

                     2
  {a*x + y,x - a*y,(a  + 1)*y}},

 {a = 0,{a*x + y,x - a*y}}}
\end{verbatim}

\section{Switches}
\begin{description}
\item[cgbreal] Compute over the real numbers. See
Section~\ref{cgbreal} for details.

\item[cgbgs] Gr\"obner simplification of the condition. The switch
{\tt cgbgs} can be turned on for applying advanced algebraic
simplification techniques to the conditions. This will, in general,
slow down the computation, but lead to a simpler Gr\"obner system.

\item[cgbstat] Statistics of the CGB run. The switch {\tt cgbstat}
toggles the creation and output of statistical information on the CGB
run. The statistical information is printed at the end of the run.

\item[cgbfullred] Full reduction. By default, the CGB functions
perform full reductions in contrast to pure top reductions. By turning
off the switch {\tt cgbfullred}, reduction can be restricted to top
reductions.
\end{description}

\section{Updates}
Information on and updates of the CGB package will be provided on

\centerline{{\tt http://www.fmi.uni-passau.de/\char126 reduce/}.}

\bibliographystyle{alpha}
\bibliography{cgb}
\end{document}


\subsection*{Future Work}

There are many potential improvements and future steps that could be taken. The spread in signal trigger time will be investigated, as it may have an effect of the $\Delta t50$ values calculated. Additionally, many channels which were ignored in this work could be calibrated with more data and more careful fitting of each individual channel.

More sophisticated pulse shape analysis methods could be applied to achieve better position determination. Utilizing a more complex function (such as the inverse sigmoid function shown in \cite{cci21}, would likely give better results.
??????
A typical pulse shape analysis technique in experiments like GRETINA is to model fields in the detector, generate a library of expected signals, and by comparing to these signals and interpolating, determine interaction positions. This technique will be compared to the simple linear method used here, to evaluate its advantages.
?????????
 A model of the detector and a signal library (for depth information only) has been built with  ADL3 \cite{adl3}. Using the GEANT4 data discussed earlier, interactions could be simulated at realistic positions. The resultant signals could be treated as experimental data to compare against the reference signals and check how useful this library approach is at determining interaction positions.

Here we have focused only on the depth of interaction. The lateral position can be determined through simple methods like taking the ratio of image charges on neighboring strips to the collecting electrode, or through more complex methods like the signal library method described above. Here, the lateral position determination proved difficult due to the worse SNR of image charge signals, the high number of uncalibrated channels, and suspected cross-talk. However, through more careful event selection, lateral position determination methods could be studied.

Lastly, using collimation to acquire events are specific locations would provide a verification and comparison of these methods.

MACHINE LEANRING

MULTI SITE EVENT FILTERING (RISE TIME OF SHAPE)


\chapter{Image Simulation}

This chapter deals with image simulation algorithm. Using objects transmittance and reflectance and sensor characteristics, it can be possible to generate realistic hyperspectral synthetic set of data. This chapter includes PROSPECT (leaf optical properties) and SAIL (canopy bidirectional reflectance) model. Vegetation optical properties are modeled using PROSPECT model \cite{Jacquemoud2009}.

\section{PROSAIL model}

PROSAIL \cite{Jacquemoud2009} model is the combinaison of PROSPECT leaf optical properties model and SAIL canopy bidirectional reflectance model. PROSAIL has also been used to develop new methods for retrieval of vegetation biophysical properties. It links the spectral variation of canopy reflectance, which is mainly related to leaf biochemical contents, with its directional variation, which is primarily related to canopy architecture and soil/vegetation contrast. This link is key to simultaneous estimation of canopy biophysical/structural variables for applications in agriculture, plant physiology, or ecology, at different scales. PROSAIL has become one of the most popular radiative transfer tools due to its ease of use, general robustness, and consistent validation by lab/field/space experiments over the years. Here we present a first example, which returns Hemispheric and Viewing reflectance for wavelength sampled  from $400$ to $2500 nm$. Inputs are leaf and Sensor (intrinsic and extrinsic) characteristics.

\label{sec:Prosail}
\input{ProsailModel.tex}

\section{Image Simulation}

Here we present a complete pipeline to simulate image using sensor characteristics and objects reflectance and transmittance properties. This example use :

\begin{itemize}
\item input image
\item label image : describes image object properties.
\item label properties : describes each label characteristics.
\item mask : vegetation image mask.
\item cloud mask (optional).
\item acquisition rarameter file : file containing the parameters for the acquisition.
\item RSR File : File name for the relative spectral response to be used.
\item sensor FTM file : File name for sensor spatial interpolation.
\end{itemize}

Algorithm is divided in following step :

\begin{enumerate}
\item LAI (Leaf Area Index) image estimation using NDVI formula.
\item Sensor Reduce Spectral Response (RSR) using PROSAIL reflectance output interpolated at sensor spectral bands.
\item Simulated image using Sensor RSR and Sensor FTM.
\end{enumerate}

\subsection{LAI image estimation}

\label{sec:LAIFromNDVI}
\input{LAIFromNDVIImageTransform.tex}

\subsection{Sensor RSR Image Simulation}

\label{sec:LAIAndPROSAILToSensorResponse}
\input{LAIAndPROSAILToSensorResponse.tex}

% the clearpage command helps to avoid orphans in the title of the next
% section.
\clearpage


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%        LINEAR ALGEBRA APPENDIX        %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Linear Algebra}
\index{Linear Algebra}
\label{app:lin-alg}
Here we review various techniques within linear algebra.

\section{Tensors}

\section{Diagonalization}
\label{sec:app-lin-alg-legendre}


\section{Between service mixing}
The preceding section describes the creation of heterogeneous mixing matrices by age for each of the nine health service clusters individually. These mixing matrices are then combined to create a single time-varying heterogeneous mixing matrix by service and age resulting in a 144 by 144 (\(9\times16=144\)) square mixing matrix. The force of infection for an index service is calculated from the mixing matrices of the age-assortative matrix for each of the services modelled. The spatial mixing matrix is based on the adjacency of health service clusters as indicated in Table \ref{tab:intercluster_mixing}.

\begin{table}[ht]
\renewcommand{\baselinestretch}{1}
	\begin{tabular}[ht]{| p{2cm} | p{0.9cm} | p{0.9cm} | p{0.9cm} | p{0.9cm} | p{0.9cm} | p{0.9cm} | p{0.9cm} | p{0.9cm} | p{0.9cm} |}
	\hline
	 & \rotatebox{90}{Barwon South West} & \rotatebox{90}{Gippsland} & \rotatebox{90}{Hume} & \rotatebox{90}{Loddon-Mallee} & \rotatebox{90}{Grampians} & \rotatebox{90}{North Metro} & \rotatebox{90}{South East Metro} & \rotatebox{90}{South Metro} & \rotatebox{90}{West Metro} \\
	\hline
	Barwon South West & R & 0 & 0 & 0 & M & M & 0 & 0 & M \\[4ex]
	\hline
	Gippsland & 0 & R & M & 0 & 0 & 0	 & M & M & 0 \\[4ex]
	\hline
	Hume & 0 & M & R & M & 0 & M & M & 0 & 0 \\[4ex]
	\hline
	Loddon-Mallee & 0 & 0 & M & R & M & M & 0 & 0 & M \\[4ex]
	\hline
	Grampians & M & 0 & 0 & M & R & M & 0 & 0 & M \\[4ex]
	\hline
	North Metro & M & 0 & M & M & M & R & M & 0 & M \\[4ex]
	\hline
	South East Metro & 0 & M & M & 0 & 0 & M & R & M & 0 \\[4ex]
	\hline
	South Metro & 0 & M & 0 & 0 & 0 & 0 & M & R & 0 \\[4ex]
	\hline
	West Metro & M & 0 & 0 & M & M & M & 0 & 0 & R \\[4ex]
	\hline
    \end{tabular}
    \title{Adjacency-based spatial mixing matrix.}
    \caption{\textbf{Adjacency-based spatial mixing matrix.} 0, no mixing between spatial patches; M, calibrated inter-service mixing parameter for adjacent services; R, the diagonal matrix elements are populated with the complement of the other values for each row/column (and so may take a different value in each cell in which it appears)}
    \label{tab:intercluster_mixing}
\end{table}

\section{Model initialisation}
The model is commenced from late March to allow dynamic vaccination to occur prior to seeding of the epidemic. The epidemic is then seeded predominantly through the Metropolitan clusters from early July to initiate transmission.


\newpage{}

\subsection{A001 - Duration per Word}

Let us dive in with a very very simple algorithm. In fact it is the
first algorithm I tought of.

Let's say we have a task. May it read ``Buy a big book and put it in the
shelf''. And this takes a certain amount of time.

\begin{verbatim}
AmountOfTime = "Buy a big book and put it in the shelf"
\end{verbatim}

Ok, I know, language doesn't really work like that. But lets us play
stupid right now. We just split the thing into words and distribute the
duration equally to each word. The sentence above is 10 words long, so
each word gets a tenth of AmountOfTime associated with it.

Now we have a minimal time table.

We can now take another input, like ``Buy a small book''.

We know the durations associated with ``Buy'', ``a'' and ``book''. We
will have to do something about the unknown word. Ignore it, or add 10\%
to the sum of the known durations. Something like that. And voilà, we
have a time estimate.

Funny thing:

It is actually quite reasonable for the time being, that, in the absence
of further information, assigning half the duration to each subtask is
not the most stupid thing one could do.

And it is actually quite reasonable to think that buying a small book is
about the effort of buying a big one. But that is just the example.

Anyways, we now have a time estimate and noone got hurt.

\hypertarget{using-the-algorithm-from-powershell}{%
\subsection{Using the algorithm from powershell}}

The algorithm is added to EstimatePS, see
Source/experiments/duration-per-word/experiment1.ps1 for a complete
example with learning, caching and usage.

Finally it is as easy as:

\begin{verbatim}
$inSeconds = Get-DPWEstimate -Model $model -DurationInSecondsFor "add a new bookkeeping api" -ProbabilityInPercent 95
\end{verbatim}


\chapter{Figures} \label{chap:fig}

\lipsum

\begin{figure}
    \centering
    \subfloat[$f_w$ is over-fitting.]{\input{figures/appendix/over_fitting_yes.tex}
    \label{of:no}}
    \hfil
    \subfloat[$f_w$ is generalising.]{\input{figures/appendix/over_fitting_no.tex}
    \label{of:yes}}
  \caption{A beautiful illustration of over-fitting done with mathcha \url{https://www.mathcha.io/}.}
  \label{fitting}
\end{figure}


\chapter{Numerical models and simulations}

\section{Theoretical framework}

This section explains details of the numerical model considered and the criteria used for its election. We consider an agent-based model of spherical active particles with overdamped dynamics in a 2-dimensional representation.

\subsection{Agent-based models}

In biophysics, models can be sorted into two classes, agent-based, and continuum models. Agent-based models simulate the dynamics of cells individually, offering accurate system descriptions. Depending on the complexity of the model, they can be costly in a computational sense. On the other hand, continuum models consider a coarse-grained description with density and velocity fields. We work in a low-density regime, so a continuum description is meaningless. Therefore, we will use agent-based models. For simplicity, we consider cells with spherical shapes. In reality, the cell shape depends on the growth phase of the bacteria culture, but \textit{E.coli}'s body is a spherocylinder with typical dimensions of \SI{0.5}{\micro\meter} in radius and \SI{2}{\micro\meter} in length.

\subsection{Overdamped dynamics}

To describe the dynamics of single cells, we invoke Newton's second law:

\begin{equation}
	m\ddot{\textbf{r}} = \sum \textbf{F} = \textbf{F}_{\text{hydro}} + \textbf{F}_{\text{flag}} + \textbf{F}_{\text{brown}} + \textbf{F}_{\text{coll}} ,
\end{equation}

where we define four different force sources, $\textbf{F}_{\text{hydro}}$ the hydrodynamic friction produced by the fluid, $\textbf{F}_{\text{flag}}$ the force produced by the flagella, $\textbf{F}_{\text{brown}}$ the Brownian force due to collisions with the liquid molecules, and $\textbf{F}_{\text{coll}}$ produced due to cell-cell and cell-surface collisions. We can deduce an important fact widely accepted in microscopic systems with the following analysis. To analyze the role of inertia, let's imagine we have isolated \textit{E.coli} bacterium that suddenly stops swimming, so $\textbf{F}_{\text{flag}} = \textbf{F}_{\text{coll}} =0$. Although the influence of $\textbf{F}_{\text{brown}}$ is essential, its effect is stochastic with zero mean. If we repeat the situation many times and average the ensemble, its contribution is expected to be zero. Therefore, for the moment, we will neglect it. Then we are left with the equation:

\begin{equation}
	m\ddot{\textbf{r}} = \textbf{F}_{\text{hydro}} = -\gamma \dot{\textbf{r}} = - 6 \pi R \eta \dot{\textbf{r}},
\end{equation}

where we used Stokes' law to calculate the drag force coefficient $\gamma$ with $R=$ \SI{0.5}{\micro\meter} being the particle radius and $\eta=$ \SI{d-3}{\pascal\cdot\second} the viscosity. Since the mass of an \textit{E.coli} is $m=$ \SI{d-12}{\gram}, the characteristic time of slowing down is given by $ m/(6 \pi R \eta) \approx $ \SI{d-7}{\second}. This means that in the timescale of \SI{1}{\micro\second} the bacteria should have stopped. We record at 10 fps in our experiments, so the detention is immediate for our time resolution. This means that we are working in the overdamped limit, where inertia is negligible, and so it is correct to simplify the dynamics as:

\begin{equation} \label{eq:overdamped_model}
	\gamma \dot{\textbf{r}} = \textbf{F}_{\text{flag}} + \textbf{F}_{\text{brown}} + \textbf{F}_{\text{coll}}.
\end{equation}

\subsection{Rotational diffusion}

We considered two different effects of the fluid in the bacterial dynamics. First, $\textbf{F}_{\text{hydro}}$ is a phenomenological dynamical drag force that represents the average effect of the liquid on an object moving through the fluid, and second, $\textbf{F}_{\text{brown}}$ of stochastic kind accounting for the thermal fluctuations. The latter produces a mean square displacement of $\langle r^2 \rangle (t) = 4 D t $ where $D$ is the diffusion constant given by the fluctuation-dissipation theorem \cite{Soto2016KineticPhenomena}:

\begin{equation}
	D = \frac{k_bT}{\gamma} \approx \SI[per-mode = symbol]{1.5d-1}{\square\micro\meter \per \second},
\end{equation}

where $k_b$ is the Boltzmann constant, $T$ the temperature of the fluid. The same value for $\gamma$ was used. This diffusion constant is purely translational. Nevertheless, the collisions between cells and water molecules also exert torques on the bacteria, meaning that the orientation of swimming is subject to an equivalent stochastic process. This process has a rotational diffusion constant $D_r$ independent of $D$. For smooth-swimming \textit{E.coli} it has been measured at $D_r=$ \SI[per-mode = symbol]{0.057}{\square\radian \per \second} \cite{Drescher2011FluidScattering}. We can perceive the importance of rotational diffusion by neglecting collisions in equation \eqref{eq:overdamped_model} and calculating the mean square displacement as described in \cite{Lauga2020TheMotility}.

\begin{equation} \label{eq:flagellar force}
	 \dot{\textbf{r}} =\frac{\textbf{F}_{\text{flag}} + \textbf{F}_{\text{brown}} }{\gamma} = u \textbf{p} + \textbf{F}^*_{\text{brown}},
\end{equation}

where $u=$ \SI[per-mode = symbol]{20}{\micro\meter \per \second} is the mean speed of bacteria, $\textbf{p}$ the vector of orientation, and $\textbf{F}^*_{\text{brown}}$ is just the force with the coefficient $\gamma$ absorbed. Integrating equation \eqref{eq:flagellar force} in time we obtain:

\begin{equation} \label{eq:positon}
	\textbf{r}(t) - \cancelto{0}{\textbf{r}(0)}  = \int_0^t [u \textbf{p}(t^\prime) +  \textbf{F}^*_{\text{brown}}(t^\prime)  ]dt^\prime ,
\end{equation}

Taking the dot product of \eqref{eq:flagellar force} and \eqref{eq:positon} we obtain an equation for the time derivative of the square displacement.

\begin{equation} \label{eq:derivative of msd}
	\dot{\textbf{r}} \cdot \textbf{r} = \frac{1}{2}\frac{d}{dt} (r^2) = \int_0^t  [u^2 \textbf{p}(t^\prime) \cdot \textbf{p}(t) + \textbf{F}^*_{\text{brown}}(t) \cdot \textbf{F}^*_{\text{brown}}(t^\prime)]dt^\prime + \text{irrelevant terms}.
\end{equation}

All the terms in the right side of equation \eqref{eq:derivative of msd} are stochastic and vary from cell to cell. Nevertheless, their average for all particles has properties that allow progress in the calculation. For example, the terms cataloged as irrelevant consider dot products of two quantities completely uncorrelated, the director vector $\textbf{p}$ and the force $\textbf{F}^*_{\text{brown}}$. Therefore after averaging on the ensemble of cells, the average dot product is zero, and we are left with:

\begin{equation}
	\frac{d}{dt} \langle r^2 \rangle &= 2 \int_0^t  \langle u^2 \textbf{p}(t^\prime) \cdot \textbf{p}(t) + \textbf{F}^*_{\text{brown}}(t) \cdot \textbf{F}^*_{\text{brown}}(t^\prime) \rangle dt^\prime.
\end{equation}

The quantity $\langle\textbf{p}(t^\prime) \cdot \textbf{p}(t)\rangle$ is the mean time correlation of the vector director \textbf{p} and depends on the rotational diffusion coefficient as $e^{-2D_r|t-t^\prime|}$ \cite{Lauga2020TheMotility}. On the other side $\textbf{F}^*_{\text{brown}}$ is assumed to be an uncorrelated noise, that satisfies $\langle \textbf{F}^*_{\text{brown}}(t) \cdot \textbf{F}^*_{\text{brown}}(t^\prime) \rangle = 2D\delta(t^\prime -t)$. Therefore, integrating over both $t$ and $t^\prime$ we obtain:

\begin{equation}
    \langle r^2 \rangle (t) = \frac{u^2}{D_r}\left( t + \frac{e^{-2D_rt}}{2D_r} - \frac{1}{2D_r} \right) + 4Dt
\end{equation}

At short times,  $t \ll D_r^{-1} =$ \SI{18}{\second} the exponential can be expanded in a Taylor series $e^{-2D_rt}\approx 1 - 2D_rt + 2(D_r t^2) + \mathcal{O}(t^3)$ giving $\langle r^2 \rangle (t)  \approx (ut)^2$, which means that at short times particles swim is straight lines. Bacteria swimming is more important than diffusion even at our lowest time scale of $t=$ \SI{0.1}{\second}, as in that case $4Dt$ is two order of magnitude lower than $(ut)^2$. In the other case $t \gg D_r^{-1}$ the mean square displacement is $(u^2/D_r+4D)t$. Adding the translational diffusion discussed previously, we deduce an effective diffusion constant for long times $D_{\text{eff}}$ given by:

\begin{equation}
    D_{\text{eff}} = D + \frac{u^2}{4D_r}
\end{equation}

For the typical values mentioned, $D_{\text{eff}}\approx$ \SI[per-mode = symbol]{2d3}{\square\micro\meter \per \second} which is four orders of magnitude larger than $D$. This means that bacteria swimming makes translational Brownian motion irrelevant compared to the effects of rotational diffusion. We conclude that $\textbf{F}_{\text{brown}}$ can be set to zero for simplicity without losing relevant dynamics. We are left with the equation:

\begin{equation} \label{eq:final_model}
    \dot{\textbf{r}} = u\textbf{p} + \frac{1}{\gamma}\textbf{F}_{\text{coll}} = u\textbf{p} + \textbf{F}^*_{\text{coll}} ,
\end{equation}

where $\textbf{F}^*_{\text{coll}}$ absorbs the drag coefficient $\gamma$ and hence has units of speed.

\subsection{Alignment with the wall}
\label{section:steric alignment}

We observe that bacteria interacting with the curved and flat walls, suffer a torque that aligns them with the wall \cite{Bianchi2017HolographicBacteria}. This alignment has its origin on steric forces. If we consider equation \eqref{eq:final_model} and take the dot product with the vector perpendicular to the surface  $\hat{\textbf{n}}_w$ at the point of contact, we obtain:

\begin{equation} \label{eq:steric_force}
   \cancelto{0}{\dot{\textbf{r}} \cdot \hat{\textbf{n}}_w}  = u\textbf{p} \cdot \hat{\textbf{n}}_w + \textbf{F}^*_{\text{coll}} \cdot \hat{\textbf{n}}_w ,
\end{equation}

where $\dot{\textbf{r}} \cdot \hat{\textbf{n}}_w = 0$ is imposed as bacteria do not cross the surface. Since the collision force with the wall $\textbf{F}^*_{\text{coll}}$ is exclusively perpendicular to the wall, equation \eqref{eq:steric_force} gives the magnitude of the collision force as $\gamma u \textbf{p} \cdot \hat{\textbf{n}}_w$, where $\gamma$ reappeared because we are calculating the force. Then we can write the equation for the angle of swimming $\theta$ considering that the inertia of the cell is negligible and so that the sum of torques must be zero. Simplifying the system, we consider the torques respect to the center of mass as the rotational drag and the torque exerted by the wall. Other torques can be considered for more complete models to explain observations such as circular trajectories and longer residence times in the wall \cite{Lauga2006SwimmingBoundaries, Sipos2015HydrodynamicWalls}. For this thesis we decided to work with a simple model because these effects are not required to explain the abandonment of the curved wall by bacteria. Using equation \eqref{eq:steric_force},

\begin{equation} \label{eq:deduction_of_K}
    0 = - \gamma_r \dot{\theta} \hat{\textbf{z}} - u \gamma (\textbf{p} \cdot \hat{\textbf{n}}_w) (\textbf{r}_{cm} \times \hat{\textbf{n}}_w),
\end{equation}

where $\gamma_r$ is the rotational drag coefficient and $\textbf{r}_{cm}$ the vector from the center of mass to the point of contact, as we are calculating the torque with respect to the center of mass. Therefore, $\textbf{r}_{cm} \times \hat{\textbf{n}}_w = L_{cm} (\textbf{p} \cdot \hat{\textbf{t}}_w) \hat{\textbf{z}}$ where $\hat{\textbf{t}}_w$ is the tangential vector to the wall in the point of contact that satisfies $\hat{\textbf{z}} = \hat{\textbf{t}}_w \times \hat{\textbf{n}}_w$  and we used $\textbf{r}_{cm} = L_{cm} \textbf{p} $ with $L_{cm}$ is the distance between the center of mass and the point of contact taking into consideration the contribution of the flagella to the center of mass. The elements involve in these equations are shown in figure \ref{cp_wall_diagram}. We are left with the equations:

\begin{align}
    \textbf{p} &= \cos{\theta}\hat{\textbf{x}}+\sin{\theta}\hat{\textbf{y}}, \\
    \dot{\theta} &= -K (\textbf{p} \cdot \hat{\textbf{t}}_w)  (\textbf{p} \cdot \hat{\textbf{n}}_w) \Gamma(\textbf{r}, \textbf{p}),
    \label{eq:wall alignment}
\end{align}

where $K\equiv u L_{cm}\gamma / \gamma_r$. The scalar $K$ controls the intensity of the alignment and in ref. \cite{Bianchi20193DInterface} a fit of experimental data yields $K=$ \SI[per-mode = symbol]{4.9}{\radian \per \second}. In our study we consider $K$ as a free parameter with values between \SIlist[per-mode = symbol, list-units=single]{0;7}{\radian\per\second}. This is because we are considering the more complicated case of a curved wall. Also, $\Gamma(\textbf{r}, \textbf{p})$ is a step function that indicates if the bacteria is in contact with the wall or not. It is equal to $1$ when the distance from the center of the particle $\textbf{r}$ to the closest point of the wall is smaller than one radius, and its swimming direction points towards the wall i.e. $\textbf{p} \cdot \hat{\textbf{n}}_w < 0$ indicating the cell is going into the wall and not away from it. Otherwise $\Gamma$ equals zero.  Equation \eqref{eq:wall alignment} will align the vector $\textbf{p}$ with the $\pm\hat{\textbf{t}}_w$ depending on the sign of $\textbf{p} \cdot \hat{\textbf{t}}_w$. This effect is only considered for the flat and curved wall as the frontal wall is already taken into account because simulations are two-dimensional.

\begin{wrapfigure}{r}{0.5\linewidth}
\centering
\includesvg[width=\linewidth,height=4cm]{imagenes/cp_wall_diagram}
\caption[Diagram of the components involve in the wall allignment]{Diagram of the components involve in the wall alignment. $O$ represents the origin of coordinates.}
\label{cp_wall_diagram}
\end{wrapfigure}

We calculate the point of contact as the closest point in the walls to the cell by dividing the interval $[x-R, x+R]$ in 300 points where $x = \textbf{r} \cdot \hat{x}$. Then, we calculate the distance between the cell and the position obtained with the parametric definition of the wall for each point. The point $\textbf{r}_w$ with the lowest distance is the closest with a precision of \SI{3d-3}{\micro\meter}. This ``brute force" algorithm works because a point outside of the interval $[x-R, x+R]$ is not in contact with the cell, and it is convenient because the distance to the wall as a function of the $x$-axis has many local minima.

% \vspace{1cm}

\section{Simulations}

Equations \eqref{eq:final_model}--\eqref{eq:wall alignment} summarize the physics involved in the description of the system. We are only missing two aspects; rotational diffusion and the details of $\textbf{F}_{\text{coll}}^*$. This section first explains how we implement these phenomena, summarize the model, and finally explain how the simulations are performed.

\subsection{The collision force}

$\textbf{F}^*_{\text{coll}}$ is the force produced by the collision of cells with other cells or the wall, but divided by the drag coefficient $\gamma$. We considered it as an elastic force for both cell-cell and cell-wall collisions. We parameterize the force $\textbf{F}^{*,i}_{\text{coll}}$ that the $i$-th particle experiments through the following:

\begin{equation}
   \textbf{F}^{*,i}_{\text{coll}} &= \sum_{w} k_{\text{wall}} (R-d_{iw}) \hat{\textbf{d}}_{iw}\Theta(R-d_{iw}) + \sum_{j \in \mathbb{NN}_i} k_{\text{cell}} (2R-d_{ij}) \hat{\textbf{d}}_{ij}\Theta(2R-d_{ij}),
\end{equation}

where $d_{iw}$, $\hat{\textbf{d}}_{iw}$ represent the distance and the unit vector between the particle $i$ and the closest point of the wall $w$, and $d_{ij}$, $\hat{\textbf{d}}_{ij}$ are the same but between the particles $i,j$ where $j$ is part of $\mathbb{NN}_i$ which is the set of nearest neighbors of the particle $i$. $\Theta$ is the usual Heaviside function. Finally the parameters $k_{\text{wall}}$, $k_{\text{cell}}$ are the intensities of these forces, and have units of \SI[]{}{\per\second}. We consider $k_{\text{wall}}=$ \SI[]{1d3}{\per\second}, which means that for $R-d_{iw}=$ \SI{2d-2}{\micro\meter} the magnitude of the interaction with the wall is equal to $u$, so it is impossible for bacteria to go through the surface. The parameter $k_{\text{wall}}$ is not related to $K$ it only defines the distance at which the elastic force is enough to repel bacteria. Meanwhile $k_{\text{cell}}$ will be considered as zero, meaning there are no cell-cell interactions. The explanation for such consideration is on section \ref{section: clustering}.

\subsection{Rotational diffusion}

 Rotational diffusion is naturally added to \eqref{eq:wall alignment} as a Gaussian white noise $\eta_i(t)$ \cite{Digregorio2018FullSeparation,Caporusso2020Motility-InducedSystem} meaning it has zero mean and its completly uncorrelated in time and between particles, $\langle \eta_i(t)\eta_j(t^\prime)  \rangle \propto \delta_{ij}\delta (t-t^\prime)$. This white noise will change randomly the direction of swimming $\textbf{p}$ of the cell. We will go into more detail in the following sections.

\subsection{Final model}

The final set of equations that are used in the model are:

\begin{align}
    \label{eq:dynamics of position}
    \dot{\textbf{r}}_i &= u\textbf{p}_i + \textbf{F}^{*,i}_{\text{coll}}, \\
    \textbf{p}_i &= \cos{\theta_i}\hat{\textbf{x}}+\sin{\theta_i}\hat{\textbf{y}}, \\
    \label{eq:dynamics of angle}
    \dot{\theta}_i &= -K (\textbf{p}_i \cdot \hat{\textbf{t}}_w)  (\textbf{p}_i \cdot \hat{\textbf{n}}_w) \Gamma(\textbf{r}_i, \textbf{p}_i) + \eta_i(t), \\
    \label{eq:elastic force}
    \textbf{F}^*_{\text{coll}} &=   k_{\text{wall}} (R-d_{iw}) \hat{\textbf{d}}_{iw}\Theta(R-d_{iw}) + \sum_{j \in \mathbb{NN}} k_{\text{cell}} (2R-d_{ij}) \hat{\textbf{d}}_{ij}\Theta(2R-d_{ij}).
\end{align}

All of the quantities used in these equations and on the model are described in table \ref{table:model parameters}. These equations are written for the dynamics of a circular particle $i$, involving other entities such as the wall and the nearest neighbors. Equations \eqref{eq:dynamics of position}--\eqref{eq:elastic force} belong to the class of Langevin equations due to their stochastic nature.

% Tabla del modelo
\begin{table}[!h]
   \centering
    \small
    \caption[Summary of the quantities used in the simulations]{Quantities used for the model, with their description and values if adequate. For parameters whose value changes, that column will have a $-$ symbol and in chapter 4, results will have that value specified. }
    \begin{tabularx}{\textwidth}{lXl}
    \hline\noalign{\smallskip}
         Variable  & Description & \quad   \\
    \noalign{\smallskip}\hline\noalign{\smallskip}
         \textbf{r} & Position of the particle $i$. & \quad \\
         \textbf{p} & Direction of swimming of the particle $i$, defined by the angle $\theta_i$. & \quad \\
         $\eta$ & White noise associated with rotational diffusion. & \quad \\
         $\hat{\textbf{t}}_w$ & Unitary vector tangent to the wall in the closest point to the cell $i$. & \quad \\
         $\hat{\textbf{n}}_{w}$ & Unitary vector normal to the wall in the closest point to the cell $i$. The normal vectors points away from the wall. & \quad \\
         $\hat{\textbf{d}}_{iw}$ & Unitary vector pointing from the closest point in the wall to the cell $i$. & \quad \\
         $d_{iw}$ & Distance between the cell $i$ and the closest point of the wall. & \quad \\
         $\mathbb{NN}$ & Set of nearest-neighbors of the particle $i$. & \quad \\
         $\hat{\textbf{d}}_{ij}$ & Unitary vector pointing from the neighbor $j$ to the cell $i$. & \quad \\
         $d_{ij}$ & Distance between the cell $i$ and the neighbor $j$. & \quad \\
    \hline\noalign{\smallskip}
        Parameter  & Description & Value   \\
    \noalign{\smallskip}\hline\noalign{\smallskip}
        $\rho$ & Density of cells. This density relates to the number of particles seen in the focal plane. & \SI[per-mode = symbol]{3d-3}{\cells \per \square\micro\meter} \\
        $u$ & Swimming speed. & \SI[per-mode = symbol]{20}{\micro\meter\per\second} \\
        $R$ & Radius of the cells. & \SI{0.5}{\micro\meter} \\
         $K$ & Magnitude of the aligment with the wall. & - \\
         $D_r$ & Rotational diffusion coefficient. & - \\
         $k_{\text{wall}}$ & Elastic constant for cell-wall collisions. & \SI{1d3}{\per\second}\\
         $k_{\text{cell}}$ & Elastic constant for cell-cell collisions. & 0 \\
         $\Delta t$ & Time step for the integration of the equations. & \SI{d-3}{\second} \\
         $\Delta t_r$ & Time step for the recording of data. & \SI{d-1}{\second} \\
         $T$ & Time duration of the simulations. & \SI{1200}{\second} \\
    \hline\noalign{\smallskip}
    \end{tabularx}
    \label{table:model parameters}
\end{table}

Quantities with the lower index $w$ depend on the wall. Simulations have a flat and a curved wall with amplitude $A$ and wavelength $\lambda$. The parametric definition of the walls positions are $y=y_f$ for the flat and $y=y_c+A\sin{\frac{2\pi x}{\lambda}}$ for the curved wall. The values of $y_f, y_c$ are so that the mean distance between walls is \SI{100}{\micro\meter}. Also, periodic boundary conditions will be applied. To avoid problems with discontinuities in the wall, the length of the channel is adjusted to have an integer number of wavelengths and be greater than \SI{300}{\micro\meter}. The total particle number is adjusted so that all simulations have the same particle density $\rho$. These values are similar to the experiments.

\newpage

\subsection{Numerical integration}

Typically, there are many options to solve a differential equation numerically. In this case, the equations are somewhat simple, so it is tempting to integrate \eqref{eq:dynamics of position} and \eqref{eq:dynamics of angle} with Euler's method. That integration method is a first-order method which means that during a step $\Delta t$ of integration, the integrands are taken as constants. Therefore we would obtain:

\begin{align}
    \textbf{r}_i(t+\Delta t) &=  \textbf{r}_i(t) + \Delta t[u \textbf{p}_i(t) + \textbf{F}^*_{\text{coll}}(t)], \\
    \label{eq:wrong integration}
    \theta_i(t+\Delta t) &=  \theta_i(t) + \Delta t[- K (\textbf{p}_i(t) \cdot \hat{\textbf{t}}_w)  (\textbf{p}_i(t) \cdot \hat{\textbf{n}}_w) \Gamma(\textbf{r}_i(t), \textbf{p}_i(t)) + \eta_i(t)].
\end{align}

However, there is a problem with the last term of equation \eqref{eq:wrong integration}, which relies on the timescale in which $\eta$ acts. Remember that $\eta$ represents the torque produced by collisions with the liquid molecules. These thermal fluctuations have a characteristic time between collisions of $\sim$\SI{1.9d-13}{\second} \cite{Soto2016KineticPhenomena}. That means we have two options, either we consider a minimal time step $\Delta t$ so the hypothesis that $\eta$ is constant in the interval is true, or we treat $\eta$ separately. The latter is the best option, as $\eta$'s timescale is much smaller than all the others present in the system. The correct equations are:

\begin{align}
    \textbf{r}_i(t+\Delta t) &=  \textbf{r}_i(t) + \Delta t[u \textbf{p}_i(t) + \textbf{F}^*_{\text{coll}}(t)], \\
    \theta_i(t+\Delta t) &=  \theta_i(t) - K \Delta t  (\textbf{p}_i(t) \cdot \hat{\textbf{t}}_w)  (\textbf{p}_i(t) \cdot \hat{\textbf{n}}_w) \Gamma(\textbf{r}_i(t), \textbf{p}_i(t)) \nonumber \\
     &\ \ \ + \int_t^{t+\Delta t}\eta_i(t^\prime)dt^\prime .
\end{align}

Then a new problem arises, how do we calculate the integral of $\eta$. We can find the answer on stochastic calculus, and here we describe one of the possible demonstrations. First, we treat $\eta$ as what it is, a discrete function representing all the collisions with liquid molecules.

\begin{equation}
    \int_t^{t+\Delta t}\eta(t^\prime)dt^\prime = \sum_{n=1}^N \phi\eta_n,
\end{equation}

where $\eta_n$ is proportional to the angle displacement produced in a certain collision $n$. The magnitude of these rotations is contained in $\phi$ so that we can treat them as standard normal Gaussians. As we stated, these collision displacements are also uncorrelated. Therefore, we can use that the distribution of the sum of two gaussian uncorrelated variables, $x\sim\mathcal{N}(\mu_x,\sigma_x^2)$ and $y\sim\mathcal{N}(\mu_y,\sigma_y^2)$, is given by $\mathcal{N}(\mu_x+\mu_y,\sigma_x^2+\sigma_y^2)$ and by induction obtain:

\begin{equation}
    \sum_{n=1}^N \eta_n \sim \mathcal{N}(0,N) = \sqrt{N}\mathcal{N}(0,1),
\end{equation}

were the last equality is in the sense of probability density distribution. Then, the total number of collisions $N$ is proportional to $\Delta t$, for instance $N=\alpha \Delta t$, where $\alpha$ is a rate of collisions. Then we can write:

\begin{align}
    \textbf{r}_i(t+\Delta t) &=  \textbf{r}_i(t) + \Delta t[u \textbf{p}_i(t) + \textbf{F}^*_{\text{coll}}(t)], \\
    \label{eq:correct integration}
    \theta_i(t+\Delta t) &=  \theta_i(t) - K\Delta t (\textbf{p}_i(t) \cdot \hat{\textbf{t}}_w)  (\textbf{p}_i(t) \cdot \hat{\textbf{n}}_w) \Gamma(\textbf{r}_i(t), \textbf{p}_i(t)) \nonumber \\
     &\ \ \ + \sqrt{\phi^2\alpha\Delta t} \eta(t).
\end{align}

Here $\phi$ is in \SI{}{\radian} and $\alpha$ in \SI{}{\per\second} so $\phi^2\alpha$ has units of \SI{}{\square\radian\per\second}, the same as the rotational diffusion coefficient. Therefore, we naturally obtained the diffusion coefficient, normally defined as $2D_r\equiv\phi^2 \alpha$. As already mentioned, for smooth swimmer \textit{E.coli} in liquids it has been measured that $D_r=$ \SI[per-mode = symbol]{0.057}{\square\radian \per \second}. Nevertheless we consider $D_r$ as a free parameter with values between $D_r=$ \SIrange[per-mode = symbol, range-units=single
]{0.001}{0.05}{\square\radian \per \second} as is expected to be lower in our case, due to the constraints of the walls. Therefore $K$ and $D_r$ are the free parameters of the model.

The result of equation \eqref{eq:correct integration} is really important for the consistency of the numerical integration, since the equation $\langle \eta(t)\eta(t^\prime)  \rangle &= \delta (t-t^\prime)$ implies that $\eta$ has units of \SI{}{\second^{-1/2}} so multiplying by $\Delta t$ instead of $\sqrt{\Delta t}$ is dimensionally wrong.

\subsection{Intensity profiles in simulations}

In experiments we construct intensity profiles using the intensity of bacteria fluorescence. In simulations there is no such thing, we can only measure bacteria positions $\textbf{r}_i$. Therefore, we have to measure the mean bacteria density. For each particle in the band of a wall $w$ we can assign a interval in the $x$-axis defined as $[x-\Delta x/2,x+\Delta x/2]$ where $x_i = \textbf{r}_i \cdot \hat{x} - x_{\text{valley}}$ is contained. Here $x_{\text{valley}}$ is the position of the nearest valley to the particle, meaning $x$ is on a interval of one wavelength $\lambda$ and $x=0$ is the position of the valley. This defines the mean bacteria density over a period $n^w(x)$ as the count of bacteria that were in contact with the wall $w$ in the interval defined by $x$.

Obviously $n^w(x)$ is not the same as intensity profiles. To create a comparable quantity, we convolve $n^w(x)$ with a Gaussian function $G(x)=\exp(-x^2/(2R^2))$. We call the result the intensity near a wall $w$ for the simulation:

\begin{equation}
    i^w_{\text{sim}}(x) = n^w(x) * G(x).
\end{equation}

This treatment means we consider particles having a Gaussian intensity in space. This is not equal to the intensities measured in experiments because bactearia are not spherical. Nevertheless, treating intensity not spherically in simulations would lead to inconsistencies. The amplitude of the Gaussian does not matter as we will normalize by the mean accumulation in the flat wall. If the mean intensity on the flat surface is $\bar{i}^f_{\text{sim}} = \langle  i_{\text{sim}}^f(x)\rangle_x $. We define the normalized intensity profile in the curved wall as:

\begin{equation} \label{eq:Intensity profiles in simulations}
    I_{\text{sim}}(x) = \frac{i_{\text{sim}}^c(x) }{\bar{i}^f_{\text{sim}}}
\end{equation}

The normalized intensity profiles of simulations $I_{\text{sim}}(x) $ will be compared to the experimental result $I(x)$ in the chapter 4.

\newpage

\subsection{Algorithm}

The algorithm used in the simulations is described in the following steps.

\begin{itemize}
    \item[1.] Create a random initial condition for all the particles positions and swimming directions. If a particle is out of boundaries or in contact with a wall, its initial conditions are generated again. Then start iterating the time steps.
    \item[2.] Update the closest point of the wall for all particles close to the walls.
    \item[3.] Integrate the equation for $\theta_i$.
    \item[4.] Determine the $\mathbb{NN}$ set for all particles.
    \item[5.] Calculate the force $\textbf{F}^*_{\text{coll}}$.
    \item[6.] Integrate the equation for $\textbf{r}_i$.
    \item[7.] Every time interval $\Delta t_r$ record the relevant data. This avoids high time correlations between measurements.
    \item[8.] Stop when a time $T=$ \SI{1200}{\second} has passed.
\end{itemize}

The algorithm was implemented in a C++ program with object oriented programming. In figure \ref{sim trajectories} we display trajectories for the parameters $K=$ \SI[per-mode = symbol]{5}{\radian\per\second}, $D_r=$ \SI[per-mode = symbol]{0.05}{\radian\per\second} and $k_{\text{cell}}=0$.

\begin{figure}[H]
	\centering
	\includesvg[scale=1]{imagenes/sim_trajectories}
	\caption[Example of trajectories for the simulations]{Example of trajectories  obtained with the simulations for the parameters $K=$ \SI[per-mode = symbol]{5}{\radian\per\second}, $D_r=$ \SI[per-mode = symbol]{0.05}{\square\radian\per\second} and $k_{\text{cell}}=0$. The star represents the start of each track and the triangles the end of them. The values of $A$ and $\lambda$ are included on the top of the plot. Trajectories last for \SI{10}{\second}.}
	\label{sim trajectories}
\end{figure}


%!TEX root = ../thesis.tex
%*******************************************************************************
%****************************** Second Chapter *********************************
%*******************************************************************************

\chapter{Background}

\ifpdf
    \graphicspath{{Chapter2/Figs/Raster/}{Chapter2/Figs/PDF/}{Chapter2/Figs/}}
\else
    \graphicspath{{Chapter2/Figs/Vector/}{Chapter2/Figs/}}
\fi

\section{Neural networks}

\textit{The history of neural networks is convoluted and winding, and it is difficult to give a concise overview of their history. What follows is a synchronic description of how they are used -- a brief historical overview follows in the next section.}

\subsection{A brief practitioner's guide to neural networks for NLP}

Neural networks are nonlinear function approximators of a certain shape: they perform an affine transformation of the input, followed by a non-linear function, to obtain an output. In this view, logistic regression corresponds to the simplest class of neural networks, with the choice of non-linearity being the logistic function. To exclude cases like this, typically one adds the condition that one composes multiple functions of the above form to obtain the input-output function. This creates \textit{hidden layers}: the intermediate states in the chain of non-linear affine transformations.

The simplest example of a neural network is when the affine transformation is a simple matrix multiplication followed by a translation. This is referred to as a \textit{feedforward layer}. An alternative is to convolve the input with a small kernel: a matrix much smaller than the input size. This gives rise to a \textit{convolutional layer}, the building block of a \textit{convolutional neural network}. These have been tremendously successful in image processing and speech recognition recently @@@cite, and have also been applied for

One notable fact about neural networks is that, with one hidden layer of infinite width, they are able to universally approximate any function \citep{Cybenko:89,Hornik:91} under mild assumptions on the non-linearity. This provides one justification for their use: given enough hidden units, they can learn any input-output mapping in a supervised task.

One recent

A neural network consists of two parts: an architecture, and a learning algorithm. Typically, the structure of a neural network can be represented by a directed acyclic graph. Nodes without incoming edges are input nodes, which is how information is fed into the neural network. Nodes without outgoing edges are output nodes, which represent the network's computation on the input. Each node, or \textit{neuron}, takes all input values from incoming connections, does some computation on these input values, and stores the result as its output value. Commonly, one multiplies each incoming value by a \textit{weight}, and then sums them up to form an intermediate value. Classically (e.g. the perception), this intermediate result would be compared to a threshold: if the intermediate was greater than the threshold, the neuron would output 1, otherwise the neuron would output 0. If we denote by $\mathbf{x}$ the vector of input values for a neuron, this corresponds to the neuron computing the following function:
\begin{equation}
    f(\mathbf{x}) = H(\mathbf{w} \cdot \mathbf{x} + b)
\end{equation}
where $H$ denotes the Heaviside step function, $\mathbf{w}$ the vector of weights for the incoming connections, and $b$ the (negative) threshold (also called the \textit{bias}). For optimisation reasons (see later), it is often preferable to have a differentiable non-linearity. Thus, sigmoid, tanh or ReLU non-linearities are often used in contemporary neural networks.

\subsection{Common neural network architectures in NLP}

Different graph structures give rise to different neural network architectures. At present, it is most common to organise the nodes into successive \textit{layers} $l_1, \dots, l_n$ such that neurons in each layer only have incoming connections from nodes in the previous layer and outgoing connections to nodes in the subsequent layer. This also makes it easier to implement neural networks, as the computations for each layer can be calculated as affine transformations, which take advantage of optimized matrix-vector and matrix-matrix multiplication routines.

The most basic instantiation of the above architectures has (possibly many) layers of nodes, with each node in each layer connected to all nodes in the preceding and subsequent layers. This is known as a \textit{fully-connected network}. Another variant which reignited interest in neural network models for vision is called a \textit{convolutional neural network}. Here, a small window of neurons (called a \textit{filter}), with the same incoming weights and biases, are scanned over an input incrementally, based on the intuition that visual features tend to be local and translation-invariant within an image. These networks appear to learn low-level visual features in early layers, such as edges and basic shapes, while higher layers seem to capture more global semantic information. @@@cnns for text: n-gram features (whether over words or characters).

Neural networks can also contain self connections, where the state of a layer is computed based on its state in the previous timestep. This introduces recurrence into the network, and networks with these connections are called \textit{recurrent neural networks} (RNNs). These models are particularly useful for models with sequential structure, such as text, as the hidden state at a particular timestep depends on all previous timesteps, and in effect acts as a kind of memory. The most basic instantiation of RNNs are Elman recurrent networks. Here, the hidden state at time $t$ depends on both the hidden state at the previous timestep and the current input:
\begin{equation}
    h_t = f(W h_{h-1} + U x_t + b)
    \label{eqn:rnn}
\end{equation}
where $f$ is a non-linearity. This update rule rewrites the memory of the network at each timestep, and hence makes long-term temporal credit assignment difficult (cf. the vanishing/exploding gradients problem).

Hochreiter and Schmidthuber @@@cite proposed a solution by including another memory state in the recurrence: the \textit{cell state}. This is only written to additively, and this helps prevent the vanishing/exploding gradient problem. Concretely, this changes the RNN update rule (Equation \ref{eqn:rnn}) to the following:
\begin{align}
    f_t &= \sigma(W_f h_{t-1} + U_f x_t + b_f) \\
    i_t &= \sigma(W_t h_{t-1} + U_t x_t + b_i) \\
    o_t &= \sigma(W_o h_{t-1} + U_o x_t + b_o) \\
    c_t &= f_t c_{t-1} + i_t \tanh(W_c h_{t-1} + U_c x_t + b_c) \\
    h_t &= o_t \tanh(c_t)
\end{align}
where $\sigma$ is the sigmoid function, and all vector-vector products are to be intepreted pointwise. The intuition is that $c_t$ represents the long-term memory of the network, and reading and writing to this is performed additively. $i_t$ and $f_t$ are the input and forget gates respectively, which guard how much of the previous timestep's cell state is kept and overwritten respectively. $o_t$ is the output gate, which controls how much of the cell state is exposed as the new hidden state. Note that the previous timestep hidden state is only indirectly used to calculate the new hidden state.

Due to the sequential nature of RNN processing, the hidden state at time $t$ can only consider inputs at time less than $t$. However, for some tasks, knowledge of future inputs can be useful, such as disambiguating noun/verb word senses for POS tagging. A simple way to incorporate future context is to process the input in reverse order with another RNN to obtain a sequence of hidden states $h_t^{rev}$, where each $h_t^{rev}$ captures the forward context of the word to the end of the sentence. Then, the forward and reverse hidden states are concatenated to obtain the final hidden state $h_t$ at time $t$. This is known as a bidirectional RNN, and the reverse RNN effectively acts as a powerful lookahead feature extractor @@@cite.

One of the major uses of neural networks in NLP is representation learning, especially for sentences. As neural networks compute using vectors of real numbers, the final hidden state of the neural network after processing an entire sentence can be used as a feature vector to a linear model. This can be either the final hidden state of an RNN or LSTM scanning over the sentence, or the final hidden layer of a stack of convolutions. The neural network acts as an encoder, compressing the variable length input into a fixed-width representation.

A particularly important application of sentence representation learning is in the sequence-to-sequence framework. In many conditional language generation tasks, such as machine translation, we often need to generate language from structured input data. The sequence-to-sequence approach encodes the input data using a neural network to a vector representation, which is then fed into a neural language model decoder which generates the output text one word at a time. This approach demonstrates the modularity and flexibility of neural networks, as one can use a wide range of network architectures in both the encoder and decoder.

One issue with the above approach is that the same dimension vector is used to represent every sentence, no matter what length the sentence is. For long sentences, therefore, there is information loss in the compression process, and empirically the performance of a simple sequence-to-sequence model on translating long sentences is poor. To bypass this bottleneck, @@@Bahdanau propose the \textit{attention} mechanism, where the encoding of the input is dynamically computed during decoding. In detail, rather than encoding the input as a single vector, an attentional model encodes the input as a sequence of vectors $c_1, \dots, c_n$, called annotations. Then, at timestep $t$ during decoding, the decoder uses its current hidden state, $h_{t-1}$, to query the annotations, and then uses the (normalised) query scores as weights to calculate the context vector for the current timestep. This context vector is then fed into the LSTM recurrence:
\begin{align}
    a_i &= \text{sim}(h_{t-1}, c_i) \\
    w_i &= \frac{a_i}{\sum a_i} \\
    s_t &= \sum w_i c_i
    h_t = LSTM(h_{t-1}, x_t, s_t)
\end{align}

Many choices of similarity function have been proposed. The original paper used a multi-layer perceptron to calculate the similarity score, while @@@luong showed that a bilinear scoring function performed better in machine translation empirically.

\subsection{Feature representation in neural networks}

Neural networks manipulate vectors of real numbers. Therefore, when processing language with neural networks, one fundamental consideration is how to represent the symbolic language input in a format amenable to neural computation. There are issues to do with the granularity of the input (what scale do we discretise the language) and coding (how do we represent the discretised units of language numerically).

For many languages, words appear to be the basic unit of language, and indeed many languages are written word-segmented, such as English. This suggests processing language on the word-level, and using words as input features. This is the standard approach for contemporary NLP. However, for many languages, such as Chinese, words are not segmented when written. This means that an additional tokenisation step has to be run, which may introduce errors into the pipeline. Further, due to the long tail of language, particularly for languages with rich morphology, storing a separate feature for each word results in a prohibitive number of features, and many words will not be seen more than once in the training data\footnote{In the standard training split of the Wall Street Journal corpus, xx\% of words are hapax legomena}. To deal with this, typically we limit the word features to the most frequent $k$ words in the training corpus, or take only words which appear more than $n$ times. Even if we use all words seen in the training data as features, we can expect to find many words in test data which have not been seen in the training data, and representing these unseen words poses a challenge.

One approach is to use a special cover token (typically denoted UNK, for UNKnown word) to represent all unseen words, and train the model using this feature by replacing certain words in the training data with this token. However, this approach completely disregards any information contained in the unseen token. Another approach is to incorporate additional features into our input apart from just word identity. For example, many parsing models @@@ make use of additional capitalisation and part-of-speech features, which are represented by a binary feature and an index into a closed vocabulary respectively.

To help deal with the word sparsity issue, sub-word level features can also be used. Morphemes, being the minimal unit of meaning, make appealing features, as they contain both semantic information in the lemma and any derivational affixes, as well as syntactic information in inflectional affixes. This helps smooth over the rare-word issue, as many words in the long tail tend to be morphologically related to high frequency words. However, breaking down words into morphemes is a tricky process: hand-written morphological analysers are brittle and require lots of engineering effort, while (minimally) supervised approaches are still fairly error-prone. The most extreme approach is to use the raw characters of the word as input features. This bypasses the issue of having to provide a morphological analysis, but requires a powerful model to capture the (largely arbitrary) form-meaning mapping. Some models elect to use a combination of features, and indeed including character level features and a powerful character composition model (such as an RNN or a CNN) can often increase performance across a wide range of NLP tasks @@@cite.

Whichever features we choose, we must turn them into a numerical representation to feed into a neural network. The typical way of doing this is using \textit{one-hot coding}. This represents features as a sparse vector of dimension the total number of features, with a 1 at the dimension corresponding to the index of the feature and 0's elsewhere. If there are multiple disjoint sets of features, such as word and POS features, they are typically one-hot coded separately, and then the resulting one-hot vectors are concatenated before being fed into the model. This means that the first weight layer of the neural network is effectively a lookup table which associates each feature with a vector called an \textit{embedding}.

One special class of embeddings are the representations a neural network learns for word features. These are called \textit{word embeddings}, and they can be initialised randomly and trained with the rest of the network. However, many datasets are small, and so words which appear few times in the training corpus may have their parameters badly estimated during the training procedure. An alternative is to pretrain word embeddings on an unsupervised task using copious amounts of unlabelled data and use these to initialise the model word embedding parameters. This simple technique can be viewed as a cheap way of doing semi-supervised learning, and has been shown to improve performance at a wide range of NLP tasks (@@@cite).

Word embedding pretraining typically involves some form of language modelling objective. The earliest approaches took the word embedding parameters from a neural language model; however, with a large vocabulary, language modelling can be computationally intensive. Consequently, a plethora of methods for efficient neural langauge model training have been published (@@@cite hierarchical softmax, importance sampling, NCE). @@@Mikolov introduced a new unsupervised paradigm: instead of predicting the next word given the context, decide whether two words appear in the same context or not. This avoids computing the softamx entirely, and resulted in orders of magnitude speedups for word embedding estimation.

An alternative approach is to derive word embeddings directly from word cooccurrence counts. Latent semantic analysis performs an SVD factorisation of a word-word cooccurrence matrix, and uses the resulting low-dimensional vectors as word representations. Another popular word embedding model, GLoVE @@@cite, also can be viewed as an enhanced matrix factorisation model which factorises the log-probability of word coccurence rather than the raw cooccurence count.

One interesting phenomenon of all of these models is that geometric similarity of words in the embedding space seems to reflect the underlying conceptual similarity of those words. This is perhaps explained by the \textit{distributional hypothesis}, which states that words which occur in similar contexts tend to have similar meanings. Indeed, the embedding space can be viewed as a form of \textit{conceptual space} @@do more here.

\subsection{Training neural networks}

The second component of a neural network is the learning algorithm. This specifies how the neural network should adapt to the training data in order to minimise some error criterion. While many different learning rules have been proposed in the literature, the \textit{de facto} standard currently is gradient descent on the parameters (i.e. the weights) of the neural network to minimise some loss function. Common loss functions include the cross-entropy between the model prediction and the true label, or margin-based losses between the model prediction and the ground truth.

A naive approach towards calculating the gradients of the loss function with respect to the model parameters is quadratic in the number of layers, as we would have to effectively traverse the computational graph for each layer. However, if we memoise the intermediate steps, we can calculate all of the derivatives in one backwards pass through the network. This is the basic idea behind the backpropagation algorithm, which in practice is how derivatives are computed for neural networks.

Once gradients are calculated, the parameters have to be updated. One common way of doing this is to use (minibatched) stochastic gradient descent (SGD): for a parameter $\theta$, with gradient $\nabla\theta$, the update is
\begin{equation}
    \theta \leftarrow \theta + \alpha \nabla \theta
    \label{eqn:sgd}
\end{equation}
where $\alpha$ is the \textit{step size}, which is a hyperparameter of the SGD algorithm. Minibatch here refers to the fact that typically, purely online SGD (calculating gradients based on one training set example at a time) suffers too much from variance in the gradient, while fully batched SGD is too memory-intensive. Thus, the gradient estimate $\nabla\theta$ in Equation \ref{eqn:sgd} is calculated using a small batch of data points\footnote{in the order of 32-100 examples} sampled from the training data.

Pure SGD can have problems converging in certain pathological conditions, such as very flat minima. A commonly used variant introduces a momentum term to accelerate convergence, which simulates the behaviour of a heavy ball under gravity. This changes the update rule to:
\begin{align}
    z &\leftarrow \beta z + \nabla \theta \\
    \theta &\leftarrow \theta + \alpha z
\end{align}
where now we introduce another hyperparameter $\beta$, which controls how strongly we remember previous gradients. In practice, momentum appears to speed up the convergence of gradient descent, and makes SGD more resilient to choices of the step size hyperparamater.

Typical convergence proofs of SGD require the step size hyperparameter to go to zero over the course of optimisation. There are many ways of scheduling the learning rate, such as epoch-based, or by monitoring performance on a held-out development set; these all add to the number of hyperparameters which need to be tuned. Adaptive learning rate algorithms (@@@cite adagrad, adadelta, adam) attempt to automatically schedule the learning rate using convergence information. They also claim to be more resilient to hyperparameter choices. Although well-tuned SGD with momentum typically performs better than using an adaptive optimizer with off-the-shelf hyperparameter settings, the latter perform well enough to justify the convenience.

As neural networks are low-bias function approximators, they overfit very easily. Regularisation is therefore crucial to good model performance. There are many common regularisation techniques for neural networks: one common one is $\ell_2$-regularisation, which penalises weights with high $\ell_2$ norm. The most widely used technique is dropout @@@cite. Here, a proportion of the output values in each layer are randomly zeroed out. This was originally presented to be an empirically effective technique, but the theoretical motivation was lacking. @@@gal derived connections between dropout and variational inference, and showed how dropout can also give uncertainty estimates out-of-the-box, and also how to apply dropout to self-connections in neural networks. @@@gabor showed that, when properly regularised, LSTM models outperform more complicated architectures at language modelling, which highlights the power of dropout.

\section{Generative models}

In machine learning, generative modelling is the name given to a class of approaches which try to learn the distribution of the data. For a classification task, this involves learning the joint distribution of inputs $X$ and the labels $Y$; stretching the definition, generative modelling can also mean capturing the distribution of given data $X$ in unsupervised learning with a view towards drawing samples from the learnt model which resemble actual data points. One of the most common ways of specifying a generative model is as a graphical model, where nodes represent random variables and edges represent dependence relations.

Another application of generative models in NLP is parsing. PCFG-based parsers

@@@generative parsing models?

\section{Deep generative models}

Deep generative models are a class of approaches towards generative modelling which make heavy use of neural networks to parametrise certain components of the model.

The earliest generative models which use neural networks were Hopfield networks, and their stochastic counterparts Boltzmann machines. These define the energy of a configuration, which corresponds to an unnormalised log-probability. Training a Boltzmann machine corresponds to increasing the probability of the training data. @@@computing normalising constant is hard, have to use gibbs sampling or contrastive divergence

Restricted Boltzmann machines (RMBs) aim to alleviate the difficulty of training general Boltzmann machines by making the node structure bipartite. One set of nodes corresponds to the input data, and the other are the `hidden' or latent layer. This simplifies sampling from the model, as now we can use blocked Gibbs sampling to sample either the input nodes or the latent nodes in one go. As an added bonus, inferring the most likely latent variable configuration for a given input gives us a low-dimensional `representation' of the input.

However, inference in RBMs is still difficult, as calculating the normalising constant requires summing over exponentially many configurations in the number of total nodes. While sampling-based Monte Carlo approaches can be used, it is often difficult to tell whether the samples are of good quality. An alternative approach is to use variational approaches to perform approximate inference. @@@

@@@variational autoencoder, generative adversarial network


\section{Explicit Nameing Of Type Variables}
\label{sec:expl-name-type}

The type constructor it introduces can be used in places where a type
variable is \verb|not allowed|.

\begin{ocamlcode}
let f (type t) () =
    let module M = struct exception E of t end in
    (fun x -> M.E x ), (function M.E x -> Some x | _ -> None);;
val f : unit -> ('a -> exn) * (exn -> 'a option) = <fun>
\end{ocamlcode}

The exception defined in local module can not be captured by other
exception handler except wild catch.

Another example:
\begin{ocamlcode}
let sort_uniq (type s) (cmp : s -> s -> int) =
    let module S = Set.Make(struct type t = s let compare = cmp end) in
    fun l -> S.elements (List.fold_right S.add l S.empty);;
val sort_uniq : ('a -> 'a -> int) -> 'a list -> 'a list = <fun>
\end{ocamlcode}
The functor needs a type constructor(type variable is not allowed)