\input{permve-ntnu-latex-assignment.tex}

\usepackage{float}

\title{
\normalfont \normalsize
\textsc{Norwegian University of Science and Technology\\IT3105 -- Artificial Intelligence Programming}
\horrule{0.5pt} \\[0.4cm]
\huge Module 2:\\Combining Constraint-Satisfaction Problem-Solving with Best-First
Search\\
\horrule{2pt} \\[0.5cm]
}

\author{Per Magnus Veierland\\permve@stud.ntnu.no}

\date{\normalsize\today}

\newacro{CSP}{Constraint Satisfaction Problem}
\newacro{GAC}{General Arc Consistency}

\begin{document}

\maketitle

\section*{Generality of A* implementation}

The \ac{CSP} and \ac{GAC} related code is implemented in the \texttt{vi.csp} namespace. A \texttt{Network} object holds all \texttt{Variable} and \texttt{Constraint} objects, as well as a \texttt{domains} mapping from \texttt{Variable} objects to the domains of the variables. The values within a domain can be of any type. Also within the \texttt{vi.csp} namespace are the functions implementing \textsc{REVISE*} and \ac{GAC}. None of this code is tied to search code; and it can be used in full isolation where needed.

A \texttt{Constraint} holds the list of variables it is linked to, as well as a condition which can be evaluated. A \texttt{Variable} has an identity which can be of any type suiting the problem, as well as the list of contraints it is involved in.

Fusing the \ac{GAC} algorithm with A* is done by the \texttt{vi.search.gac.Problem} class, which is constructed from a \texttt{vi.csp.Network} object -- see Figure~\ref{figure:vi_astar_gac}. The \texttt{Network} object serves as the state of each search node. The \texttt{Problem} class provides the methods \texttt{goal\_test}, \texttt{heuristic}, \texttt{initial\_node}, and \texttt{successors} -- which forms the interface to the A* search class.

No problem specific code exists in the \texttt{vi.search.gac.Problem} class. It has a goal test which checks that all domains in the current state has a size of one. It has a heuristic which returns the sum of the length of all domains minus one. The initial search node returned from the \texttt{Problem} class is simply the initial network passed through one iteration of the \texttt{general\_arc\_consistency} function.

The crucial part of the \texttt{Problem} class implementation lies in the \texttt{successors} method. All successors generated from a search node is based on assumptions made about a single variable. The implementation selects the variable which has the smallest domain which contains more than one value. A successor state is generated for each value in the assumed variable's domain. The \ac{GAC} domain filtering loop is then warm rebooted to revise the domains of all variables which share a constraint with the assumed variable.

\begin{figure}[H]
\centering
\includegraphics[scale=0.7]{images/vi_astar_gac}
\caption{VI CSP and A*-GAC classes}
\label{figure:vi_astar_gac}
\end{figure}

\section*{Generality of A*-GAC implementation}

The input to the vertex coloring problem is a pure graph description; with no direct relation to the vertex coloring problem. The file input is parsed and used to construct a graph using the \texttt{vi.graph.Vertex}, \texttt{vi.graph.Edge} and \texttt{vi.graph.Graph} classes; which are fully problem independent.

A single function in the \texttt{vi.app.vertex\_coloring} namespace takes the \texttt{vi.graph.Graph} object constructed from the file input, together with a $K$-value describing the number of colors -- and builds a \texttt{vi.csp.Network} which contains the variables, constraints and domains which describes the vertex coloring problem. It simply generates one \texttt{Variable} for each \texttt{Vertex} object; with the \texttt{Vertex} object as the variable's identity. For each \texttt{Edge} object a \texttt{Constraint} object is built which binds the two \texttt{Variable} objects associated with each \texttt{Vertex} in the edge together. The condition assigned to each constraint simply verifies that the values assigned to each of the two variables are different.

\section*{Constraint network}

A constraint network consists of constraints, variables, and domains. Much of this information is identical between search states. The assignment specifies that the variables and constraints will be the same in all states. The difference between the states is the sizes of the domains belonging to the different variables.

The constraint network is represented with the \texttt{vi.csp.Network} class. It holds a all variables involved in the network; a list of all constraints involved in the network; and it holds the mapping from variables to their respective domains. It is important to note that there is no direct connection from a \texttt{Variable} instance to its domain; this connection is only made by the mapping which exists in each \texttt{Network} instance. A \texttt{Network} instance is the state for each search node in the A*-GAC implementation. However importance is place on how the \texttt{Network} object is copied and which data is shared between states.

All \texttt{vi.csp.Network} instances which belong to the same problem share the list of variables and the list of constraints. There is only a single list of variables and a single list of constraints; and each instance of the \texttt{Network} class points to the same two lists. It is still important that each instance contains these two pointers since \texttt{Network} objects belonging to different problems will necessarily maintain different lists of variables and constraints.

The \texttt{Network.copy()} member function returns a new instance of the class which points to the same lists of variables and constraints; but which takes a shallow copy of the domains mapping. This means that no actual domains are copied, only the mapping from variables to domains. When applying the GAC algorithm to a \texttt{Network} instance, the relevant mappings of revised domains is updated to point at the new revised domain for the involved variables. This ensures high reuse in representation and avoids redundant information, without increasing the complexity of the implementation.

\section*{Code chunks}

In the given implementation a \texttt{Constraint} class instance holds a single condition and a list of variables involved in the condition. The condition must be callable and must accept one parameter. The parameter passed to a condition is a mapping of variables to specific values for all variables involved in the constraint. This makes the \texttt{Constraint} interface very general and allows any hashable type to be used as the \texttt{Variable} identity and in variable domains.

Since all variables involved in a \texttt{Constraint} is explicitly specified it is easy to generate the required permutations of values from variable domains in the \textsc{REVISE*} algorithm.

The interface was chosen for its semantical purity and generality. It can be used with both interpreters parsing user input as well as direct \texttt{eval} approaches simply by adding code to build the necessary \texttt{Constraint} objects and callable conditions. Since no parsing of user input was necessary for the given problem; only a simple lambda verifying the inequality of two variable values in the given value set was used.

\end{document}


\chapter{Background information}
\label{chap:Background}
HydroDyn is currently undergoing a conversion to the FAST modularization framework. During this conversion, some additional capabilities are being added to HydroDyn including jacket platforms. At present, HydroDyn does not account for second order wave forces: those forces that arise from the sum and difference of the frequencies of incident waves. HydroDyn also only includes uni-directional waves where all waves are incident from the same direction.

At the start of a FAST simulation, HydroDyn is initialized and calculates the wave spectrum\footnote{
      The wave spectrum is complex-valued and therefore includes all the phase information, so the time history of the wave can be found easily through an FFT.}
and time history using the \emph{waves} sub-module.  This information is then used by the \emph{WAMIT} sub-module and a few other sub-modules to calculate the first order wave foreces on the platform.  Because the frequency, direction, and phase of each of the wave components is known at initialization, a time series of the second order forces can be calculated using the \emph{WAMIT2} sub-module before the simulations begins. This requires that the following assumptions are made:
\begin{itemize}
   \item{the wave information in frequency space is known before the start of the simulation (the time series can be found through an FFT),}
   \item{the time series of second order forces will act on a single point at the origin (as the first order forces do),}
   \item{wave forces and moments are fixed to a reference frame that translates (but does not rotate) with the platform,}
   \item{Wavelengths are large compared to the motion of the platform.}
\end{itemize}

\section{Platform Displacement and Rotation}
The complete timeseries for the forces and moments comprising the loads from the first and second order waves are calculated during the initialization of a FAST simulation.  These calculated loads are given at the platform origin in the platform rest reference frame and applied during the simulation.  During the simulation, the platform will translate and rotate as a result of these loads, aerodynamic loads, and other real time loads (mooring lines etc.).
The calculation of the first and second order wave loads do not take into account this displacement.

During the simulation, the FAST glue code calls \HD to calculate the platform loads at each timestep.  Since the first- and second-order wave load timeseries were calculated during the initialization, these values are looked up at each timestep and reported back to the glue code on a point mesh (includes translation and rotation).

\endinput

The implication of this is that as the platform translates and rotates, the wave field and resulting loads are in effect translated and rotated with it.  For translation, this is not problematic since the translation is slow in comparison with the wave velocity.  However, it is not desirable to have the wave directionality also changing as the platform yaws, so an additional step is taken within \HD to compensate for the platform rotation before returning the forces and moments to the glue code.  This keeps the wave directionality and resulting moments in the inertial (non-rotated) reference frame of the platform.  This could potentially result in inaccuracies in response if the pitch and roll responses were dramatically different for waves incident along $x$ or $y$ respectively.

\endinput
FIXME: add figure from page 141 of notebook

Effectively means that the wave field and resulting loads move with the platform.  This is not likely to cause problems for surge and sway translation since the platform surge response tends to be slow in comparison with the wave velocity. There may be potential for issues with large heave displacements since WAMIT assumes the translational motions are \emph{small} compared to the wavelength during response force calculations.


\documentclass[main.tex]{subfiles}
\begin{document}

\section*{Fri Dec 20 2019}

\section{Gravitational waves}

They are solutions of the linearized EFE on a given fixed background. Today we only treat GW on a Minkowski background, but we could do it for any number of other (cosmological) background.

So, we say:
%
\begin{align}
g_{\mu \nu } = \eta_{\mu \nu } + h_{\mu \nu }
\,,
\end{align}
%
with \(h_{\mu \nu } \ll 1\), so we neglect anything which is \(\mathcal{O}(h^2)\).

GW are very topical today: they could allow us to see beyond the last scattering surface, when the universe stopped being opaque to EM radiation: then, it had been transparent to gravitational radiation from a long time before.

We can do statistics to the stochastic GW background, for now however we have only detected GW from localized event.

We must start to write down the EFE from the Christoffel symbols: in general they are
%
\begin{align}
\Gamma^{\alpha }_{\mu \nu } = \frac{1}{2} g^{\alpha \lambda } \qty(g_{\lambda \nu , \mu } + g_{\mu \lambda , \nu } - g_{\mu \nu , \lambda })
\,,
\end{align}
%
but in the derivatives of the metric we only have \(\mathcal{O}(h)\) terms, so in the inverse metric we neglect all the \(\mathcal{O}(h)\) terms since their global contribution would be \(\mathcal{O}(h^2)\): so we get
%
\begin{align}
    \Gamma^{\alpha }_{\mu \nu } = \frac{1}{2} \eta ^{\alpha \lambda } \qty(h_{\lambda \nu , \mu } + h_{\mu \lambda , \nu } - h_{\mu \nu , \lambda })
\,,
\end{align}
%
and when we compute the Ricci tensor we will only keep the \(\partial \Gamma \) terms, since the \(\Gamma \Gamma \) terms are \(\mathcal{O}(h^2)\). So our expression becomes
%
\begin{subequations}
\begin{align}
R_{\mu \nu } &= \partial_{\alpha } \Gamma^{\alpha }_{\mu \nu } - \partial_{\nu } \Gamma^{\alpha }_{\mu \alpha }  \\
&= \frac{1}{2} \eta^{\alpha \lambda } \partial_{\alpha } \qty(h_{\lambda \nu , \mu } + \cancelto{}{h_{\mu \lambda , \nu }} - h_{\mu \nu , \lambda }) - \frac{1}{2} \eta^{\alpha \lambda } \partial_{\nu  }\qty(h_{\lambda \alpha , \mu } + \cancelto{}{h_{\mu \lambda , \alpha }} - h_{\mu \alpha , \lambda })  \\
&= \frac{1}{2} \eta^{\alpha \lambda } \qty(h_{\lambda \nu , \mu \alpha } - h_{\mu \nu , \alpha \lambda } - \frac{1}{2} h_{\lambda \alpha , \mu \nu } - \frac{1}{2} h_{\lambda \alpha , \mu \nu } + h_{\mu \alpha , \lambda \nu } )
\,,
\end{align}
\end{subequations}
%
where we split a term in two in order to collect, so we get
%
\begin{align}
R_{\mu \nu } = \frac{1}{2} \eta^{\alpha \lambda } \partial_{\mu } \qty(h_{\lambda \nu , \alpha } - \frac{1}{2} h_{\lambda \alpha , \nu })
+ \frac{1}{2} \eta^{\alpha \lambda } \partial_{\nu }
\qty(h_{\mu \alpha , \lambda } - \frac{1}{2} h_{\lambda \alpha , \mu })- \frac{1}{2} \eta^{\alpha \lambda } \partial_{\alpha } \partial_{\lambda } h_{\mu \nu }
\,,
\end{align}
%
where we recognize the Dalambertian \(\square = \eta^{\alpha \lambda  }  \partial_{\alpha } \partial_{\lambda }\) and the trace of \(h_{\mu \nu }\): \(h \equiv h^{\mu }_{\mu }\). So we get
%
\begin{align}
R_{\mu \nu } = - \frac{1}{2} \square h_{\mu \nu }
+ \frac{1}{2} \partial_{\mu } \qty(\partial_{\lambda } h^{\lambda }_{\nu } - \frac{1}{2} \partial_{\nu }h)
+ \frac{1}{2} \partial_{\nu }\qty(\partial_{\lambda}  h^{\lambda }_{\mu } - \frac{1}{2} \partial_{\mu} h)
\,,
\end{align}
%
so we need to solve \(R_{\mu \nu }  =0 \) with some initial conditions, since we need to solve the EFE \(G_{\mu \nu } = M_P^{-2} T_{\mu \nu } = 0\), which implies \(R_{\mu \nu }= 0 \).
The problem is that the solution is not unique: we have the freedom of choosing a gauge, since different metrics connected to each other through changes of coordinates represent the same physical scenario: \(h_{\mu \nu } \) \emph{has no direct physical meaning}.

In general we have a change of coordinates \(x \to \widetilde{x}\), and if we do this
%
\begin{align}
g_{\mu \nu } \to \widetilde{g}_{\mu \nu } = \pdv{x^{\alpha }}{\widetilde{x}^{\mu  }} \pdv{x^{\beta }}{\widetilde{x}^{\nu }} g_{\alpha \beta }
\,,
\end{align}
%
but we can only do changes of coordinates which leave the Minkowski part of the metric invariant, that is \(\widetilde{g}_{\mu \nu } = \eta_{\mu \nu } + \widetilde{h}_{\mu \nu }\). We can actually restrict ourselves to infinitesimal changes of coordinates: \(x \to \widetilde{x} = x + \epsilon \). So we do
%
\begin{align}
\pdv{\widetilde{x}^{\mu }}{x^{\alpha }} = \partial_{\alpha } \qty( x^{ \mu } + \epsilon^{\mu })
= \delta_{\alpha }^{\mu } + \pdv{\epsilon^{\mu }}{x^{\alpha }} = \delta_{\mu }^{\alpha } + \partial_{\alpha } \epsilon^{\mu }
\,.
\end{align}

Inserting this inside the inverse change of coordinates formula for the metric (the one for \(g_{\mu \nu }\) in terms of \(\widetilde{g}_{\mu \nu }\), the inverse of the one we wrote down before) we get
%
\begin{subequations}
\begin{align}
\qty(\delta^{\mu}_{\alpha } + \partial_{\alpha } \epsilon^{\mu })
\qty(\delta^{\nu }_{\beta } + \partial_{\beta } \epsilon^{\nu })
\qty(\eta_{\mu \nu } + \widetilde{h}_{\mu \nu })
&= \eta_{\alpha \beta }
+h_{\alpha \beta }  \\
\qty(\delta^{\mu }_{\alpha } \delta^{\nu }_{\beta } + \delta^{\mu }_{\alpha } \partial_{\beta } \epsilon^{\nu }
+ \delta^{\nu  }_{\beta  } \partial_{\alpha  } \epsilon^{\mu  } + \mathcal{O} (\epsilon^2))  \qty(\eta_{\mu \nu } + \widetilde{h}_{\mu \nu }) &= \eta_{\alpha \beta }  + h_{\alpha \beta }  \\
\widetilde{h}_{\alpha \beta }+
\partial_{\alpha } \epsilon_{\beta } + \partial_{\beta } \epsilon_{\alpha } &= h_{\alpha \beta }
\,.
\end{align}
\end{subequations}

This freedom of changing \(h_{\mu \nu }\) is called gauge freedom: if \(h_{\alpha \beta }\) solves the linearized EFE, then \(\widetilde{h}_{\alpha \beta } = h_{\alpha \beta } + \partial_{(\alpha } \epsilon_{\beta )}\) also does (there is a factor 2 missing, but it does not matter since \(\epsilon \) is generic and still infinitesimal).

We can solve \(R_{\mu \nu } \) together with some extra conditions: this means \emph{choosing a gauge}.
We can do this as long as, starting from a generic \(h_{\mu \nu }\), we can find a transformation \(\epsilon_{\mu }\) which leaves \(\eta_{\mu \nu } \) invariant and for which \(\widetilde{h}_{\mu \nu } = h_{\mu \nu } - \partial_{(\mu } \epsilon_{\nu )}\).

For example, a condition which cannot be imposed is \(h_{\mu \nu } =0\): if we could impose it, then it would be equivalent to have a wave or not to have it.
This can be shown from degrees of freedom: we only have 4 degrees of gauge freedom.

So when we set a condition we must prove that we \emph{can actually} do so.

Then, any gauge we are actually allowed to impose will give the same results for any physical experiment.

An electromagnetic analogy: since \(\vec{\nabla} \cdot \vec{B} = 0\), we know that we can locally choose a potential \(\vec{A} \) such that \(\vec{B} = \vec{\nabla} \times \vec{A}\). This has no intrinsic physical meaning, as any \(\vec{A}' = \vec{A} + \vec{\nabla} \lambda \) for a scalar function \(\lambda \) also gives the exact same magnetic field.
Some gauge fixing choices are \(\vec{\nabla} \cdot \vec{A} = 0\) (Coulomb), \(A_{3} = 0\) (axial).

We do, however, use potentials since they are convenient.
Analogously, \(h_{\mu \nu }\) is convenient to use but it is not measurable nor unique.

\todo[inline]{Is there a gauge-independent tensorial quantity we can define starting from \(h_{\mu \nu }\), a gravitational ``field strength''?}

The last parentheses in the expression for \(R_{\mu \nu } \) is annoying: so we choose the gauge
%
\begin{align}
\partial_{\lambda } h^{\lambda  }_{\nu } - \frac{1}{2} \partial_{\nu } h = 0
\,,
\end{align}
%
and then in this gauge we have \(R_{\mu \nu } = - \frac{1}{2}  \square h_{\mu \nu }\).

\begin{bluebox}
This gauge choice can also be written in terms of the Christoffel symbols:
%
\begin{align}
g^{\mu \nu } \Gamma^{\rho }_{\mu \nu } = 0
\,.
\end{align}
%
\end{bluebox}

There are two questions we must ask: can we actually impose this condition, and why is it called harmonic?
% The answers are yes, and because it gives a harmonic equation \(\square h_{\mu \nu } = 0\).

Let us prove the first one:
%
\begin{subequations}
\begin{align}
0 \overset{?}{=} \partial_{\lambda } \widetilde{h}^{\lambda }_{\nu } - \frac{1}{2} \partial_{\nu }\widetilde{h}
&= \partial_{\lambda } \qty( h^{\lambda }_{\nu } - \partial^{\lambda } \epsilon_{\nu } - \cancelto{}{\partial^{\nu } \epsilon_{\lambda }} ) - \frac{1}{2} \partial_{\nu } \qty(h - 2 \cancelto{}{\partial_{\lambda } \epsilon^{\lambda }})  \\
&= \partial_{\lambda } h^{\lambda }_{\nu } - \frac{1}{2} \partial_{\nu } h  - \square \epsilon_{\nu }
\,,
\end{align}
\end{subequations}
%
which can be solved by setting an \(\epsilon_{\nu }\) such that:
%
\begin{align}
  \square  \epsilon_{\nu } = \partial_{\lambda } h^{\lambda }_{\nu } - \frac{1}{2} \partial_{\nu } h
\,.
\end{align}

Notice that in this gauge \(\square \widetilde{h}_{\mu \nu } = 0 \): the wave equation, for a wave which propagates with velocity 1.

We have residual gauge freedom! As long as \(\square \widetilde{\epsilon}_{\mu} = 0\), we can still change \(h_{\mu \nu }\).
So the solution is not unique. The gauge is called harmonic because we can do harmonic residual gauge fixing.
We have 4 more degrees of freedom: we choose \(h_{0 \mu } \equiv 0\). Now we prove that this can be done.

We do not prove this in general (although it can be done), because we want to move towards a more specific example.

We then consider plane wave solutions: waves that only depend on one spatial coordinate, and are constant with respect to both other spatial coordinates.

If the direction of motion is the \(z\) axis, then the solution looks like
%
\begin{align}
h_{\mu \nu } (t, x, y, z) = C_{\mu \nu } \exp(-i (k t - k z ))
\,.
\end{align}

For a wave propagating in a generic direction we will have
%
\begin{align}
h_{\mu \nu } (t, \vec{x}) = C_{\mu \nu } \exp(i k_{\alpha } x^{\alpha })
\,,
\end{align}
%
where \(k^{\alpha } k_{\alpha } = 0\) (which comes from the Fourier expression of the Dalambertian: \(\square = -k^{\alpha } k_{\alpha }\)) is a constant vector. The scalar product in the exponent must be computed with the Minkowski metric since we are working at  first order in \(h\) and \(C_{\mu \nu }\) is already first order.
%
\begin{subequations}
\begin{align}
\square h_{\mu \nu } &= \eta^{\alpha \beta } \partial_{\alpha } \partial_{\beta } \qty(C_{\mu \nu } \exp(i k_{\alpha }x^{\alpha }))  \\
&= \eta^{\alpha  \beta } C_{\mu \nu } \partial_{\alpha } \qty(\exp(i k_{\alpha } x^{\alpha }) \partial_{\beta } \qty(i k_{\alpha } x^{\alpha }))  \\
&= \eta^{\alpha  \beta } C_{\mu \nu } i k_{ \beta } \partial_{\alpha } \qty(\exp(i k \cdot x )) \\
&= \eta^{\alpha \beta } (i k_{ \beta }) (ik_{\alpha }) C_{\mu \nu } \exp(i k \cdot x) = - k^2 C_{\mu \nu } \exp(i k \cdot x)
\,,
\end{align}
\end{subequations}
%
then we see that this is always zero as long as \(k^2=0\), which is what we impsed for \(k^{\alpha }\).

What is the particle analog of a gravitational wave? it is a so-called \emph{graviton}.
For it we will have a momentum operator \(\vec{p} = - i \hbar \vec{\nabla}\) and an energy operator \(E = i \hbar \partial_{t}\). We can apply both of these to our wave to find that it is an eigenvector of both:
%
\begin{align}
E h_{\mu \nu } = i \hbar \partial_{t} \qty(C_{\mu \nu } \exp(i k \cdot x))= \hbar k^{t} h_{\mu \nu }
\,,
\end{align}
%
and we know that \(k^{t} = \abs{\vec{k}}\).

If we compute the momentum, analogously we find \(\vec{p} h_{\mu \nu } = \hbar \vec{k}\).

This means that \(p \cdot p  = 0\), which means that the mass of the graviton is equal to zero.

\end{document}


\section{The Lorentz metric}


\chapter{PCG Techniques}
The most important thing to keep in mind when working with PCG is that different environments require different PCG methods. Before we start looking specifically into dungeon generation we must first identify the major PCG techniques to decide which one fits best.

\section{Genetic algorithms}
Togelius et al. distinguishes two types of content generation. The first is with the use of a {\em constructive algorithm} that consists of generating the content in a single run with little or no backtracking \citep{springerlink:10.1007/togelius1}. The other method, referred to as {\em generate-and-test algorithms}, carries out a number of attempts and select only the candidates that satisfy a set of tests \citep{springerlink:10.1007/togelius1}. Typically, such algorithms are more complicated and more time-consuming, however if handled with care they can provide results in only a fraction of a second \citep{DBLP:conf/cig/TogeliusPBWHY10}. Due to the nature of genetic algorithms, they tend to provide very good results for generating outdoor terrain or, in other words, generating the height-map of a level \citep{DBLP:conf/gecco/RaffeZL11}.

Genetic algorithms are essentially generate-and-test algorithms \citep{springerlink:10.1007/togelius1}. A genetic algorithm can metaphorically be described as:
\begin{quote}
``Genetic Algorithms mimic real life evolution, in particular based on Darwin’s theory of Natural Selection" ... ``Features change over time due to natural mutations and mutual heritance." \citep{DBLP:conf/ACMace/MouratoSB11}.
\end{quote}

The idea of a genetic algorithm is to create a series of candidates, selectively pick out the best ones and then repeat this process until a satisfactory solution emerges \citep{DBLP:conf/ACMace/MouratoSB11}.

\begin{figure}[h!]
  \centering
    \includegraphics[width=0.4\textwidth]{images/genetic-flowchart.png}
\caption{Flowchart representation of a genetic algorithm. Image sourced from \citep{DBLP:conf/cec/WalshG10}}
\end{figure}

\subsection{Fitness function}
The key element in any genetic algorithm is the {\em fitness function}, which is the function that defines `how good' a solution is. This function is used to test the validity of a candidate \citep{DBLP:conf/evoW/SorensonP10}.

Based on the usage of this fitness function, we can obtain entirely different results. Sorenson and Pasquier demonstrated that it is possible to conceive a generic framework for PCG by considering the fitness function as the amount of fun a level can convey. Naturally, the concept of fun is a rather abstract concept from a mathematical point of view. Their fitness function is dependant on a challenge measurement function, which is specific to a certain genre of game. This abstraction on the fitness function allows the algorithm to generate content both for a 2D side-scrolling platform game (such as {\em Super Mario Brothers}) as well for a dungeon-like top-down view game (such as {\em The Legend of Zelda}) \citep{DBLP:conf/evoW/SorensonP10}.

The fitness function can measure anything based on what is required. For instance, using a fitness function that measures fun is probably more viable for creating the environment of a video game \citep{DBLP:conf/evoW/SorensonP10}, whereas a function that measures the aesthetics of a terrain is more suited if the algorithm is intended to generate beautiful terrain \citep{DBLP:conf/cec/WalshG11}.

\subsection{Genetic operators}
When candidates are selected they will evolve in some way. Raffe et al. distinguish two kinds of selections: {\em parent selection} and {\em gene selection} \citep{DBLP:conf/gecco/RaffeZL11}. Distinguishing between the two is not necessary, but in doing so one gets an increased amount of control on how the algorithm behaves \citep{DBLP:conf/gecco/RaffeZL11}.

\paragraph{Mutation} This is where a single candidate evolves on its own, meaning certain genes will change into different values. The mutation can be random or selective \citep{DBLP:conf/ACMace/MouratoSB11}.
\paragraph{Cross-over} This is where two candidates are mixed together. How these two candidates are mixed depends on the type of problem being solved and the desired result \citep{DBLP:conf/ACMace/MouratoSB11}.

\section{Search-Based}
Search-based content generators are based around viewing the generation process as a search problem with a starting state, a goal state and several randomised intermediate states. They can be used to generate different kinds of content depending on how the algorithm is used. According to Togelius et al., Search-based procedural content generation (SBPCG) algorithms are generally based upon some sort of evolutionary algorithm \citep{DBLP:conf/cig/TogeliusPBWHY10}.

Togelius et al. define a search-based algorithm as such:
\begin{quote}
``Search-based procedural content generation (SBPCG) is a particular type of generate-and-test PCG, where the generated candidate content is not simply rejected or accepted by the test but graded on one or several numeric dimensions, and where a search algorithm is used to find better content based on the evaluations of previously generated content."\citep{DBLP:conf/cig/TogeliusPBWHY10}
\end{quote}

A SBPCG algorithm was used by Togelius et al. to procedurally generate maps for the Real-time strategy game StarCraft. These maps are generated by using in-game objectives such as mineral fields and geysers as the goals of the search-problem \citep{DBLP:conf/cig/TogeliusPBWHY10}.

However, search-based procedural generation techniques can also be used for generating random maze-like structures using a method suggested by Ashlock et al. This method used an algorithm that consists of visiting parts of the maze at random using a fitness function. An interesting feature about this method is that it's independent from the representation of the maze. The authors demonstrate the execution on three kinds of representations \citep{DBLP:journals/tciaig/AshlockLM11} :
\begin{itemize}
\item {\bf Direct:} Represented by a grid where each element is a bit that represents either a wall, by the value 1, or a walk-able area, by the value 0.
\item {\bf Indirect positive:} Values are stored in an array and represent the walls of the maze.
\item {\bf Indirect negative:} Values are stored in an array and represent the empty space (rooms and corridors) of the maze.
\end{itemize}
Naturally all these representation can be expanded to include more possible values and, in doing so, allow different kinds of terrain / walls across the maze \citep{DBLP:journals/cim/AshlockLM11}.

Ashlock et al. also clearly illustrate the importance of the fitness function. Here are some examples of different results obtain by merely using a different function \citep{DBLP:journals/tciaig/AshlockLM11}:
\begin{itemize}
\item {\bf Exit Path Length Fitness ($F_1$):} The function causes the maze to be composed of a single long path.
\item {\bf Primitive Re-convergence Sum Fitness ($F_2$):} This can produces a maze with dead-ends, also referred to as cul-de-sacs.
\item {\bf Isolated Primary Re-convergence Sum Fitness ($F_3$):} This function produces a maze with paths that split off and meet up again at the end.
\end{itemize}

\begin{figure}[h!]
  \centering
    \includegraphics[width=0.50\textwidth]{images/ashlockMazes.png}
  \caption{An illustration of the various types of mazes generated \citep{DBLP:journals/tciaig/AshlockLM11}.}
\end{figure}

\section{Rhythm-Based}
In platform-genre video games such as {\em Super Mario} and so forth, the aspect of timing and rhythm is particularly important \citep{DBLP:journals/tciaig/SmithWMTMC11}. Such games revolve around the fact that the player must hit the right keys at the right time. One example of timing is the act of running and jumping to get across a gap in the level. If the player jumps too early, they will not reach the next platform and fall. If the player presses the jump button too late then their character would fall before starting the jump \citep{Smith:2008:FAP:1401843.1401858}.

\begin{figure}[h!]
  \centering
    \includegraphics[width=0.28\textwidth]{images/rhythm.png}
  \caption{Platform game level generation using a rhythm-based technique \citep{DBLP:journals/tciaig/SmithWMTMC11}}
\end{figure}

A level generator called Launchpad has been designed based on this key concept of rhythm \citep{DBLP:journals/tciaig/SmithWMTMC11}.
The generator creates sections that are either `jump' or `walk'. The frequency of these jumps is defined by the rhythm of the algorithm. Additionally the length and height of the terrain will vary at random \citep{DBLP:journals/tciaig/SmithWMTMC11}.

N. Shaker et al. then conceived an optimisation to such an algorithm using Artificial Intelligence (AI) \citep{DBLP:conf/aiide/ShakerYT10}. The algorithm consists of first generating a level at random. Afterwards, an AI player plays through the level and the action's of that player are recorded. Based on these actions the algorithm will then optimise the level to make it more enjoyable.

Needless to say, such algorithms are directed specifically to platform games and are not directly applicable to other genres. Although these algorithms are aimed at 2D games a 3D implementation could still be possible but would involve a large increase of constraints.

\section{Occupancy-regulated}
The main disadvantage with purely randomised levels is that things can look a little bit unnatural or lack the sense of creativity. Human-designed levels present the advantage of being more unique and more suited to the type of game developed \citep{DBLP:conf/cig/MawhorterM10}. Mawhorter and Mateas have suggested an occupancy-regulated algorithm that allows random levels to be used whilst maintaining liberty and control for human level designer.

The authors summarise the algorithm as such:
\begin{quote}
``Occupancy-regulated extension works by assembling a level using chunks from a library." \citep{DBLP:conf/cig/MawhorterM10}.
\end{quote}

\begin{figure}[h!]
  \centering
    \includegraphics[width=0.35\textwidth]{images/mario.png}
  \caption{An illustration of Mawhorter and Mateas' algorithm. The two top panels are individual chunks. The lower left panel presents the two chunks mixed. The lower right panel is the result after post-processing. Image source: \citep{DBLP:conf/cig/MawhorterM10}}
\end{figure}

The idea is that level designers need only to design small key sections of the level that are referred to as `chunks'. Each chunk consists primarily of empty space along with certain aspects of the game, such as a jump, an NPC spawn point and so forth. Along with this, each chunk has a set of pre-defined locations where the players can legally situate themselves. These locations are called anchors \citep{DBLP:conf/cig/MawhorterM10}.

The algorithm then uses these anchors to blend several chunks together. It will take one anchor from each chunk and maps those anchors to the same location in the world. Once the level is complete the post-processor will fill the empty areas with additional components such as decorations and un-passable terrain. \citep{DBLP:conf/cig/MawhorterM10}.

This method can produce some very interesting and enjoyable levels but can cause a lot of repetition if the chunk-library is too small \citep{DBLP:conf/cig/MawhorterM10}. For example, players of the game title Diablo 3 will notice a lot of identical rooms being repeated. This suggests that this title uses some kind of occupancy-regulated algorithm, although there is no real evidence to support this.

\section{Procedural Modelling}
Procedural modelling is special kind of PCG in which the algorithm generates a 3D mesh. Algorithms that we have explored until now focused on designing the structure of a maze, or the layout of platforms. However, none of these were particularly focused on the modelling aspect of an environment.

In an article published in 2003 the authors present an interesting way of procedurally modelling skyscrapers in real-time \citep{DBLP:conf/graphite/GreuterPSL03}. The method consists of building the floor-plan by piling primitive shapes on one another and obtaining the union of that region \citep{DBLP:conf/graphite/GreuterPSL03}. The border of this floor-plan is then extruded to form a 3D object and this process is repeated for the succeeding stories \citep{DBLP:conf/graphite/GreuterPSL03}. This method is a simple, elegant and a cost-efficient way of procedurally modelling a random room. A method like this could be used to generate the rooms of a dungeon.

Procedural modelling can used also be done using L-systems. Parish and M\"uller suggested a method of using extended L-systems to generate the road map in the form of a graph. They then use L-systems again to procedurally construct the geometry of the buildings. The authors also propose a method of procedurally generating the textures of the meshes, which effectively causes a wider range of building styles to be generated \citep{Parish:2001:PMC:383259.383292}.

\begin{figure}[h!]
  \centering
    \includegraphics[width=0.75\textwidth]{images/floorplan.png}
  \caption{The various steps of the floor-plan generation. Image source: \citep{DBLP:conf/graphite/GreuterPSL03}}
\end{figure}

Another interesting topic is how to interconnect structures together through bridges, staircases and so forth. One simple approach to link two structures together is by simply interconnecting them linearly using a shape such as a box for instance \citep{DBLP:journals/cgf/KrecklauK11}. However this may not always look convincing and is not sufficient to generate interconnections with curves and deformed pieces \citep{DBLP:journals/cgf/KrecklauK11}.

Various methods exist to generate non-linear interconnections. One example is to use a rigid chain, a structure which is best described as a:
\begin{quote}
``sequence of revolute (rotational) and prismatic (translational) joints." \citep{DBLP:journals/cgf/KrecklauK11}
\end{quote}
The angles and positions of these joints can be calculated through the use of Inverse Kinematics (IK) \citep{DBLP:journals/cgf/KrecklauK11}. However when solving an IK problem a solution may not always exist, such as if the target point is out of arm's reach \citep{Tolani2000353}. A good IK solving algorithm must not only be able to find a solution if one exists, but it should also detect and report if a result is unreliable or no solution could be found \citep{Tolani2000353}.

Hahn et al. have developed a method to generate the interior content of a building with multiple floors in real-time. It uses lazy execution to insure that content is only generated once the player approaches a portal near an un-explored region. They use a number of strict generation rules to follow when generating content in order to do so efficiently. The algorithm uses a {\em generation tree} to divide the large space into floor divisions, hallway divisions, and room cluster division \citep{Hahn:2006:PRB:1183316.1183342}.


\section{Related Work}

Quantum algorithms have drawn much attention and there are many problems for which specialized algorithms have been designed, such as searching \cite{Grover96} \cite{BoyerBHT98}, Element Distinctness \cite{Santha08}, Minimum-Finding \cite{DurrH96} and many others (e.g., \cite{DurrHHM04} \cite{BerzinaDFLS04}, \cite{Zhangthesis06}).

Evolutionary Algorithms (EA) aim not to look specifically into a certain problem, but rather provide some wide-applicable algorithms which may be used if it is not possible to analyze the problem thoroughly - either because the problem complexity is too high, or because the algorithm designer can not spend much time and resources into investigating the problem.

Quantum Evolutionary Algorithms (QEA) have the same objective, but are algorithms that run on a quantum computer. They are not to be confused with Quantum-Inspired Evolutionary Algorithms (QIEA), which use ideas from quantum computing, but are algorithms that run on a classical machine.

It is difficult to test QEAs in practice, because quantum computers do not exist apart from proof-of-concept machines. It is possible to run QEAs by simulating a quantum computer on a classical machine (e.g., \cite{SpectorBBS99}), but this approach is cumbersome since such simulations have exponential running time.

Santha and Szegedy have shown \cite{SanthaS09} that quantum and classical query complexities of local search are polynomially related. Moreover, Magniez et al. have shown \cite{MagniezNRS09} that the speed-up in terms of hitting times is exactly quadratic for a large class of algorithms (``quantum random walks'' in the sense of \cite{Szegedy04} - note that there are different notions of quantum random walks, cf. \cite{MagniezNRS07} for an overview).

For some classes of local search problems, more specific and superior algorithms are available: In particular for search spaces with transition probabilities that are symmetric \cite{Szegedy04}, or ergodic \cite{MagniezNRS07}.

In our paper, we do not prove lower bounds. Conjectures of Aaronson \cite{Aaronson06} imply that the EA-algorithms considered in our paper are asymptotically optimal.  He has shown his conjecture for hypercubes, and Zhang \cite{Zhang06} has generalized the result to product graphs.

%\cite{MagniezNRS09}, \cite{Aaronson06}, \cite{DurrHHM04},
%\cite{DurrH96}, \cite{DurrHHM06}, \cite{HoyerNS02}, \cite{SanthaS04},
%\cite{MagniezNRS07}, \cite{Santha08}, \cite{SanthaS09},
%\cite{SpectorBBS99}, \cite{Zhang06}, \cite{LiR07}, \cite{ZhouZHW05},
%\cite{NarayananM96}, \cite{HanK02}, \cite{LiLR07}, \cite{XiaoYLYZ08},
%\cite{ZhangZPW08}, \cite{AraujoNM08}, \cite{MahdabiJA08},
%\cite{ZhangGW08}, \cite{LiSGS09}, \cite{WangWF09}, \cite{XingJBLQW09},
%\cite{ZhangS06}, \cite{Ambainis06}, \cite{Ambainis08},
%\cite{AmbainisSW09}, \cite{AmbainisCGT09}, \cite{HoyerNS01}


\input{header.tex}
\begin{document}

\frame{\maketitle}

%\begin{frame}
%\tableofcontents
%\end{frame}

\begin{frame}{ $IV$ curve: $0$ to $\SI{130}{\volt}$}

  \begin{columns}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./iv_curve_reverse_bias_pixel_detectors.png}
       \caption{ Theoretical $IV$ curve \cite{pixel_detectors}. }
       \label{ fig: iv_curve_theoretical}
     \end{figure}
   \end{column}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./iv_curve_0_130_V.pdf}
       \caption{Measured $IV$ curve. }
       \label{ fig: iv_curve_measured}
     \end{figure}
   \end{column}

  \end{columns}

\end{frame}

\section{Laserwidth}

\begin{frame}{ Measurment - $\SI{5}{\milli\meter}$ }

  \begin{figure}
    \centering
    \includegraphics[width=\textwidth]{./5_mm_measurment_plot.pdf}
    \caption{Measured Laserbeam }
    \label{ fig: 5_mm }
  \end{figure}

\end{frame}

\begin{frame}{ Fit - $\SI{5}{\milli\meter}$ }
  \begin{columns}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./5_mm_erorbar_plot_col.pdf}
       \caption{ Fit Columns }
       \label{ fig: iv_curve_theoretical}
     \end{figure}
   \end{column}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./5_mm_erorbar_plot_row.pdf}
       \caption{ Fit Rows }
       \label{ fig: iv_curve_measured}
     \end{figure}
   \end{column}

  \end{columns}

\end{frame}

\begin{frame}{ Measurment - $\SI{9}{\milli\meter}$ }

  \begin{figure}
    \centering
    \includegraphics[width=\textwidth]{./9_mm_measurment_plot.pdf}
    \caption{Measured Laserbeam }
    \label{ fig: 5_mm }
  \end{figure}

\end{frame}

\begin{frame}{ Fit - $\SI{9}{\milli\meter}$ }
  \begin{columns}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./9_mm_erorbar_plot_col.pdf}
       \caption{ Fit Columns }
       \label{ fig: iv_curve_theoretical}
     \end{figure}
   \end{column}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./9_mm_erorbar_plot_row.pdf}
       \caption{ Fit Rows }
       \label{ fig: iv_curve_measured}
     \end{figure}
   \end{column}

  \end{columns}

\end{frame}

\begin{frame}{ Measurment - $\SI{15}{\milli\meter}$ }

  \begin{figure}
    \centering
    \includegraphics[width=\textwidth]{./15_mm_measurment_plot.pdf}
    \caption{Measured Laserbeam }
    \label{ fig: 5_mm }
  \end{figure}

\end{frame}

\begin{frame}{ Fit - $\SI{15}{\milli\meter}$ }

  \begin{columns}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./15_mm_erorbar_plot_col.pdf}
       \caption{ Fit Columns }
       \label{ fig: iv_curve_theoretical}
     \end{figure}
   \end{column}

   \begin{column}{0.48\textwidth}
     \begin{figure}
       \centering
       \includegraphics[width=1.05\textwidth]{./15_mm_erorbar_plot_row.pdf}
       \caption{ Fit Rows }
       \label{ fig: iv_curve_measured}
     \end{figure}
   \end{column}

  \end{columns}

\end{frame}

\begin{frame}{ First Result }

     \begin{figure}
       \centering
       \includegraphics[width=0.8\textwidth]{./perfect_height.pdf}
       \caption{ Sigma against height }
       \label{ fig: iv_curve_theoretical}
     \end{figure}

\end{frame}

\end{document}


\chapter{Introduction}
\section{Introduction}
The \textit{algorithm selection} (AS) problem presents the question, "Given a set of algorithms, how to choose one that best solves a given problem?" AS is best applied in solving intractable problems, whose solutions often require brute-force or heuristic approaches. An effective method of performing AS has broad implications on solving real-world problems which often are intractable in nature.

\section{Problem Background}
A survey of AS techniques \citep{kotthoff2016algorithm} reported a majority of approaches that used supervised learning in training an AS model. Can reinforcement learning (RL) techniques be used? Recently, RL emerged as a superior approach to learning as demonstrated in applications such as automatic design of neural network architecture \citep{zoph2016neural} and board game AI exhibiting superhuman performance \citep{silver2016mastering}. This study explores the potential of RL in training AS models with superior performance. The AS models in this study focus on solving a set of hard computational tasks called subgraph isomorphism problems. This is an NP-complete problem under graph theory where the objective is to determine if a graph contains a pattern of a much smaller graph. A recent study tackled this problem by building and training an AS model using pairwise random forest regression \citep{kotthoff2016portfolios}. This study tries to improve upon the results of the previous study by proposing RL as an alternative to training AS models.

\section{Problem Statement}
This study addresses how reinforcement learning can be applied to algorithm selection for subgraph isomorphism problems. Specifically, it attempts to answer the following questions:
\begin{itemize}
	\item How can the algorithm selection problem be viewed from the perspective of reinforcement learning?
	\item Can reinforcement learning be used to train an algorithm selection model in solving subgraph isomorphism problems?
	\item Does an algorithm selection model perform better when trained using reinforcement learning as compared to supervised learning?
\end{itemize}

\section{Research Objectives}
This study aims to use reinforcement learning in training an algorithm selection model for subgraph isomorphism problems. Specifically, it attempts to meet the following objectives:
\begin{itemize}
	\item To prepare the subgraph isomorphism dataset for training the algorithm selection model.
	\item To implement a reinforcement learning algorithm for training an algorithm selection model.
	\item To evaluate whether reinforcement learning can improve the performance of the algorithm selection model as compared to supervised learning.
\end{itemize}

\section{Scope of the Study}
The constraints observed in this study are outlined below:
\begin{itemize}
	\item The AS models are trained and tested using the subgraph isomorphism dataset generated from a previous related study by Kotthoff \citep{kotthoff2016portfolios}
	\item The characteristics of the dataset impose restrictions on how training and testing can proceed.
	\begin{itemize}
		\item The problems focus only on subgraph isomorphism problems.
		\item The set of algorithms are predefined and fixed.
		\item The model can only be trained to match one algorithm to one problem at a time.
	\end{itemize}
	\item Only one RL algorithm is considered in this study.
\end{itemize}

\section{Significance of the Study}
This study contributes to the following:

\begin{itemize}
	\item Investigates the feasibility of training algorithm selection models using reinforcement learning.
	\item Provides insights on how reinforcement learning algorithms can be applied to algorithm selection.
	\item Introduces the potential of reinforcement learning in training effective algorithm selection models.
\end{itemize}


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 21, Section 4.3: 3, 4, 7, 10, 15, 25}
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{3.}
This set does not form a basis for $R^3$. The set is linearly dependent and does not span $R^3$
\subsection*{4.}
The determinant equals 1, so these vectors are linearly independent. These vectors form a basis for $R^3$.
\subsection*{7. }
This set does not form a basis for $R^3$ because it is linearly independent and  $m \geq n$.
\subsection*{10.}
The RREF is $ \begin{bmatrix}  1 & 0 & 0 & 2 & 3 & 0 \\ 0 & 1 & 0 & -1 & -2 & 0 \\ 0 & 0 & 1 & 0 & -2 & 0  \end{bmatrix} $. The span is then $ \begin{bmatrix}  -2 \\ 1 \\ 0 \\ 1 \\ 0  \end{bmatrix} $, $ \begin{bmatrix}  -3 \\ 2 \\ 2 \\ 0 \\ 1  \end{bmatrix} $
\subsection*{15.}
The RREF is $ \begin{bmatrix}  1 & 0 & 2 & 2 & 3 \\ 0 & 1 & -2 & -1 & -1 \\ -2 & 2 & -8 & 10 & -6 \\ 3 & 3 & 0 & 3 & 9   \end{bmatrix} $. This reduces to \\
$ \begin{bmatrix}  1 & 0 & 2 & 2 & 3 \\ 0 & 1 & -2 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{bmatrix}    $
\subsection*{25.}
Yes it is a basis a typical element in H is $sv1 + (t-s)v2 + sv3$.

we can re-write this as $s(v1+v3) + (t-s)(v2), so \{v1+v3,v2\}$ is a basis for H.

the problem is that the span of \{v1,v2,v3\} is much bigger than H, it is all of $R^3$.
This means that just because it is linearly independent, it is not the basis for H. Since the basis of the subspace is not isomorphic to R or $R^2$, it's too big for the span.

\end{document}


\subsection{Clinical stratification} \label{clin}
The age-stratified late exposed/incubation and both the early and late active disease compartments were further stratified into five ``clinical" categories: \textit{1)} asymptomatic, \textit{2)} symptomatic ambulatory, never detected, \textit{3)} symptomatic ambulatory, ever detected, \textit{4)} ever hospitalised, never critical, and \textit{5)} ever critically unwell (Figure \ref{fig:clinical_strat}).
The proportion of new infectious persons entering stratum 1 (asymptomatic) is age-dependent and constant over time. The proportion of symptomatic patients (strata 2 to 5) ever detected (strata 3 to 5) is set through a parameter that represents the time-varying proportion of all symptomatic patients who are ever detected (the case detection rate). Of those ever symptomatic (strata 2 to 5), an age-specific proportion is considered to be hospitalised (entering strata 4 or 5, constant over time). Of those hospitalised (entering strata 4 or 5), a fixed proportion was considered to be critically unwell (entering stratum 5, Figure \ref{fig:clinical_rationale}, not age-specific, constant over time).

\begin{figure}[ht]
    \includegraphics[width=\textwidth]{../covid_19/stratifications/covid_19_clinical_strat.pdf}
    \title{Illustration of the implementation of the clinical stratification.}
    \caption{\textbf{Illustration of the implementation of the clinical stratification.} Depth of pink/red shading indicates the infectiousness of the compartment. Typical parameter values represented, although the infectiousness of asymptomatic persons is varied in calibration.}
    \label{fig:clinical_strat}
\end{figure}

\begin{figure}[ht]
    \includegraphics[width=\textwidth]{../covid_19/stratifications/covid_19_clinical_rationale.pdf}
    \title{Illustration of the rationale for the clinical stratification.}
	\caption{\textbf{Illustration of the rationale for the clinical stratification.}}
    \label{fig:clinical_rationale}
\end{figure}

\subsection{Hospitalisation} \label{hosp}
For COVID-19 patients who are admitted to hospital, the sojourn time in the early and late active compartments is modified, superseding the default values of the sojourn times for these compartments. The point of admission to hospital is considered to be the transition from early to late active disease, such that the sojourn time in the late disease represents the period of time admitted to hospital. For patients admitted to ICU, admission to ICU occurs at this same transition point. For this group, the period of time hospitalised prior to ICU admission is estimated as a proportion of the early active period, such that the early active period represents both the period ambulatory in the community and the period in hospital prior to ICU admission.

\subsection{Infectiousness} \label{infect}
Asymptomatic persons are assumed to be less infectious per unit time active than symptomatic persons not undergoing case isolation (typically by around 50\%, although this is varied in calibration/uncertainty analysis). Infectiousness is also decreased for persons who have been detected to reflect case isolation, and for those admitted to hospital or ICU to reflect infection control procedures (by 80\% for all groups, parameter value consistent with typically used modelled values but not informed by empiric estimates). Presymptomatic individuals are assumed to have equivalent infectiousness to those with early active COVID-19.

\subsection{Application of COVID-19-related death}
Age-specific infection fatality rates (IFRs) were applied and distributed across strata 4 and 5, with no deaths typically applied to the first three strata. A ceiling of 50\% is set on the proportion of those admitted to ICU (entering stratum 5) who die. If the infection fatality rate is greater than this ceiling, the proportion of critically unwell persons dying was set to 50\%, with the remainder of the infection fatality rate then applied to the hospitalised proportion. Otherwise, if the infection fatality rate is less than half of the absolute proportion of persons critically unwell, the infection fatality rate is applied entirely through stratum 5 (such that the proportion of critically unwell persons dying in that age group becomes \textless 50\% and the proportion of stratum 4 dying is set to zero). In the event that the infection fatality rate for an age group is greater than the total proportion hospitalised (which is unusual, but could occur for the oldest age group under certain parameter configurations), the remaining deaths are assigned to the asymptomatic stratum. This approach was adopted for computational ease and is valid because the duration active for persons entering this stratum is the same as for the other non-hospitalised strata, such that the dynamics are identical to assigning the deaths to any of the first three strata. We used the age-specific IFRs previously estimated from age-specific death data from 45 countries and results from national-level seroprevalence surveys \cite{RN6}.

\begin{table}[ht]
\renewcommand{\baselinestretch}{1}
    \begin{tabular}{| p{2cm} | p{6.6cm} | p{2.5cm} | l | l |}
    	\hline
        \textbf{Clinical stratum} & \textbf{Stratum name} & \textbf{Pre-symptomatic} &\textbf{Early} & \textbf{Late}\\
        \hline
        1 & Asymptomatic & \cellcolor[HTML]{DC6464}\textcolor[HTML]{FFFFFF}{0.5} &
        \cellcolor[HTML]{DC6464}\textcolor[HTML]{FFFFFF}{0.5} & \cellcolor[HTML]{DC6464}\textcolor[HTML]{FFFFFF}{0.5} \\
        2 & Symptomatic ambulatory never detected & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} \\
        3 & Symptomatic ambulatory ever detected & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{F0BEBE}\textcolor[HTML]{FFFFFF}{0.2} \\
        4 & Hospitalised never critical & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{F0BEBE}\textcolor[HTML]{FFFFFF}{0.2} \\
        5 & Ever critically unwell &
        \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{C90000}\textcolor[HTML]{FFFFFF}{1} & \cellcolor[HTML]{F0BEBE}\textcolor[HTML]{FFFFFF}{0.2} \\
        \hline
    \end{tabular}
    	\title{Illustration of the relative infectiousness of disease compartments by clinical stratification and stage of infection.}
    \caption{\textbf{Illustration of the relative infectiousness of disease compartments by clinical stratification and stage of infection.} Typical parameter values displayed.}
    \label{tab:clinical}
\end{table}


\section{Introduction}

We consider the problem of performing SQL-style join operations on secret shared tables with three parties and an honest majority. In particular, the proposed protocol takes two or more arbitrarily secret shared database tables and constructs another secret shared table containing a join of the two tables, without revealing \emph{any} information beyond the secret shares themselves. Our protocol is constant round and  has $O(n)$ computation and communication  overhead to join two tables with $n$ records. Simulation-based security is achieved in the semi-honest setting with an honest-majority. Our protocol can perform inner, left and full joins along with union and arbitrary circuit computation on the resulting table. A central requirement in achieving high efficiency and no leakage is that the join-keys must be unique.

New techniques \cite{usenix:PSZ14,USENIX:PSSZ15,PSZ16,CCS:KKRT16,PSWW18,CLR17,CHLR18,cryptoeprint:2017:738,RA17,KLSAP17,OOS17,KMPRT17} for performing set intersection, inner join and related functionalities have shown great promise for practical deployment. To name a few, Ion et al. at Google recently deployed a private set intersection sum protocol\cite{cryptoeprint:2017:738} to allow customers of Google Adwords to correlate the impact online advertising has on generating offline sales while preserving user privacy. Pinkas et al. \cite{PSWW18} also introduced a practical protocol that can compute any (symmetric) function of the intersection and associated data. In both cases these protocols can be framed in terms of SQL queries where an inner join is computed followed by an aggregation on the resulting table, e.g. summing a column.

The majority of these protocols consider the two party setting and are based on various cryptographic primitives, e.g. exponentiation~\cite{cryptoeprint:2017:738}, oblivious transfer\cite{PSWW18}, or fully homomorphic encryption\cite{CLR17}. However, in this work we alter the security model to consider three parties with an honest majority. The motivation is that typical protocols in this setting (e.g.\cite{highthroughput}) require less computation and communication than similar two party protocols, by a factor of at least the security parameter $\kappa=128$. Moreover, we will see that the honest majority enables various algorithms which are orders of magnitude more efficient, e.g. oblivious permutations.

Given this observation we investigate how to leverage the efficiency gains in the three party setting to construct practical protocols for performing set intersection and other SQL like operations where both the inputs and outputs are secret shared. One critical aspect of this input/output requirement is that join operations can then be \emph{composed} together, where the output of a join can be the input to another. Allowing this composability greatly increases the ability to perform highly complex queries and enables external parties to contribute data simply by secret sharing it between the primary parties which participate in the protocol.

\subsection{Functionality}

Our protocol offers a wide variety of functionality including set intersection, set union, set difference and a variety of SQL-like joins with complex boolean queries. Generally speaking, our protocol works on tables of secret shared data which are functionally similar to SQL tables. This is contrasted by traditional PSI and PSU protocols\cite{usenix:PSZ14,USENIX:PSSZ15,PSZ16,CCS:KKRT16} in that each record is now a tuple of values as opposed to a single key.

We define our database tables in the natural way. Each table can be viewed as a collection of rows or as a vector of columns. For a table $X$, we denote the $i$th row as $X[i]$ and the $j$th column as $X_j$.
\iffullversion
Note that each column of a table has the same length but can contain different data types, e.g. $X_1$ is a column of 32 bit fixed point decimal values and $X_2$ is a column of 1024 bit strings.

\fi
Our core protocol requires each table to contain unique values in the column defining the join  (i.e., we can only join on ``unique primary keys''). For example, if we consider the following SQL styled join/intersection query
$$
\texttt{select } X_2 \texttt{ from } X \texttt{ inner join } Y \texttt{ on } X_1 = Y_1
$$
then the join-keys are $X_1$ and $Y_1$. This uniqueness condition can be extended to the setting where multiple columns are being compared for equality.
\iffullversion
In this case the tuple of these columns must be unique.
\fi
Later on we will discuss the case when such a uniqueness property does not hold. Our protocols also support a \texttt{where} clause that filters the selection using an arbitrary predicate of the $X$ and $Y$ rows. Furthermore, the \texttt{select} clause can also return a function of the two rows. For example,
\begin{align*}
\texttt{select }&  X_1,max(X_2, Y_2)  \texttt{ from } X \texttt{ inner join } Y \\
\texttt{on }& X_1 = Y_1 \texttt{ where } Y_2 > 23.3
\end{align*}
In general, the supported join operations can be characterized in three parts: 1) The select function $S(\cdot)$ that defines how the rows of $X,Y$ are used to construct each output row, e.g. $S(X,Y)=(X_1, max(X_2,Y_2))$
\iffullversion
in the example above
\fi. 2) The predicate $P(\cdot)$ that defines the \texttt{where} clause, and 3) which columns are being joined on.
\iffullversion
 We require that both $S$ and $P$ must be expressible in the framework of \cite{aby3} which we provide a custom implementation of.
\fi

\iffullversion
So far we have discussed inner join (intersection) between two tables.
\fi
Several other types of joins are also supported including left and right joins, set union and set minus (difference) and full joins. A left join takes the inner join and includes all of the missing records from the left table. For the records solely from the left table, the resulting table contains \texttt{NULL} for the columns from the right table. Right join is defined symmetrically. A full join is a natural extension where all the missing rows from $X$ and $Y$ are added to the output table.

We define the union of two tables to contain all records from the left table, along with all the records from the right table which are not in the intersection with respect to the join-keys. Note that this definition is not strictly symmetric with respect to the left and right tables due to rows in the intersection coming from the left table. %\footnote{We define union in this way to resolve the ambiguity around which table the records in the intersection should come from. For example, consider two tables each with one row $X[0]=(k, x_2,x_3,...), Y[0]=(k, y_2,y_3,...)$. The union of $X,Y$ with $X_1,Y_1$ being the join-keys should only contain one row $(k,...)$ but which values $x_2,x_3,...$ or $y_2,y_3,...$ that should be in the output table is ambiguous. We therefore define it to be $(k,x_2,x_3,...)$. A full join allows both sets of values to be in the output.}.
Table minus is similarly defined as all of the left table whose join-column value is not present in the right table.

Beyond these various join operations, our framework supports two broad classes of operations which are a function of a single table. The first is a general SQL select statement which can perform computation on each row (e.g. compute the max of two columns) and filter the results using a \texttt{where} clause predicate. The second class is referred as an aggregation which perform an operation across all of the rows of a table. For example, computing the sum, counts or the max of a given column.
\subsection{Our Results}

We present the first practical secure multi-party computation protocol for performing SQL styled database joins with linear overhead and constant round. Our protocol is fully composable in that the input and output tables are generically secret shared between the parties.
\iffullversion
	No partial information is revealed at any point.
\fi
	We achieve this result by combining various techniques from private set intersection and secure computation more
\iffullversion
	broadly while requiring the joined on keys be unique.
\else
	broadly.
\fi
 We build on the the binary secret sharing technique of \cite{highthroughput} with enhancements described by \cite{aby3}. We then combine this secret sharing scheme with cuckoo hashing\cite{usenix:PSZ14}, an MPC friendly PRF\cite{lowmc} and a custom protocol for evaluating an oblivious switching network\cite{MS13}. Using these building blocks our protocol is capable of computing the intersection of two tables of $n=2^{20}$ rows in 4.9 seconds.
\iffullversion
	Alternatively, the cardinality of the intersection can be computed in just 3.1 seconds.
\fi
  Beyond these two specific functionalities, our protocol allows arbitrary computation applied to a shared table. Compared to existing three party protocols with similar functionality (composable), our implementation is roughly $1000\times$ faster. When compared with \emph{non-composable} two party protocol, we observe a larger difference ranging from our protocol being $1.25\times$ slower to $4000\times$ faster depending on what functionality is being computed.

Building on our proposed protocol we demonstrate it's utility by showcasing two potential applications. The first prototype would involve running our protocol between and within the states of the United States to validate the accuracy of the voter registration data in a privacy preserving way. The Pew Charitable Trust\cite{pew} reported 1 in 8 voter registration records in the United States contains a serious error while 1 in 4 eligible citizens remain unregistered. Our privacy preserving protocol identifies when an individual's address is out of date or more seriously if someone is registered to vote in more than one state which could allow them to cast two votes.
\iffullversion
Additionally, our protocol can help register eligible citizens. Our protocol ensures that only the minimum amount of information is revealed, namely the identities of individuals with serious registration errors.
\fi
Due to how the data is distributed between different governmental agencies, it will be critical that our protocol allows for composable operations. We implement this application and demonstrate that it is practical to run at a national scale (quarter billion records) and low cost.

The second application that we consider allows multiple organizations to compare computer security incidents and logs to more accurately identify unwanted activities, e.g. a bot net. Several companies already offer this service including Facebook's ThreatExchange\cite{threat} and an open source alternative\cite{alt_threat}. One of the primary limitations of these existing solutions is the requirement that each organization send their security logs to a central party, e.g. Facebook. We propose using our protocol to distribute the trust of this central party between three parties such that privacy is guaranteed so long as there is an honest majority.

\input{related}