% To be compiled with pdf LaTeX
% This file is to be included into master file via \input command
% Note that there is no \begin{document} \end{document} brackets!

\newpage
\section{Laser guide star modeling}
\label{sec:lgs}

\mbox{}

Laser guide star modeling and propagation to and through telescope is viewed
from algorithmic standpoint. The formulas derived herein can be
used directly for coding. No approximations are used unless necessary to
reduce excessive computational complexity.

The goals of this section are
\begin{enumerate}
	\item Find wavefront phase map in the telescope entrance pupil given the
	following set of parameters:
	\begin{itemize}
		\item laser launch telescope location, pointing and focus position;
		\item telescope pointing;
		\item position in the telescope entrance pupil.
	\end{itemize}
	\item Find spot elongation and orientation in the detector focal plane.
	\item Find photon fluxes through the wavefront sensor subapertures.
\end{enumerate}

\subsection{Nomenclature}

\mbox{}

$g$-system - ``global'' or ``laboratory'' coordinate system with respect to
which all
other coordinates and coordinate systems are defined. $Z$-axis is along the
telescope optical axis at zenith position pointing towards the sky (Fig.
\ref{fig:lgs-geom}).
\\

$t$-system - ``telescope'' coordinate system such that its $z$-axis is always
along the telescope optical axis for any zenith or azimuth angle.
\\

$l$-system - ``laser launch telescope'' coordinate system such that its $z$-axis
is always along the laser launch telescope optical axis.
\\

$(\bm{o},\mathcal{R})_{b}^{a}$ - the ``orientation pair'' consisting of the
origin coordinate vector $\bm{o}$ and rotation matrix $\mathcal{R}$ to specify
coordinate transformation from coordinate system $a$ to coordinate system $b$
or, in other words, origin and ort coordinates of $b$-system written in
$a$-system.
\\

$h_{0},h_{+},h_{-}$ - median, upper and lower altitudes of the \texttt{Na}
layer, [m].
\\

$\texttt{Eu}(\alpha,\beta,\gamma)$ - coordinate rotation by Euler angles
$\alpha,\beta,\gamma$, [rad].
\\

$\beta_{t}^{g}$ - zenith angle of telescope with respect to g-system, [rad].
\\

$\beta_{l}^{t}$ - zenith angle of the $l^{th}$ LGS with respect to $t$-system,
[rad].
\\

$\bm{r}_{l}^{t}$ - coordinates of $l^{th}$ Laser Launch Telescope (LLT)
projected to the telescope Entrance Pupil (EnP), [m].
\\

$\bm{r}_{p}^{t}$ - coordinates of $p^{th}$ point in the EnP grid, [m].
\\

$\bm{r}_{li}^{l}$ - coordinates of $i^{th}$ point source in the $l^{th}$ Laser
Guide Star (LGS), in $l$-system, [m].
\\

$r_{lip}$ - distance from $i^{th}$ point source in the $l^{th}$ LGS to
$p^{th}$ point in EnP, [m].
\\

$\Phi_{li}$ - photon flux from $i^{th}$ point source belonging to $l^{th}$ LGS,
[photons].
\\

$s^{\texttt{Na}}$ - sodium coupling efficiency, [(photons m$^{2}$ )/(s W
atom)].
\\

$C^{\texttt{Na}}$ - sodium abundance, [atoms/m$^2$].
\\

$\lambda$ - wavelength, [m].
\\

$k = \frac{2 \pi}{\lambda}$ - wave number, [rad/m].

\subsection{LGS propagation geometry}
\label{sec:lgs-geometry}

Geometry of the LGS propagation problem is presented on Fig.
\ref{fig:lgs-geom}. The parameters describing the problem are chosen to 1) be
close to the ones directly measurable on real telescope, 2) be able to
describe quite general situation.

\begin{figure}[htp]
\begin{center}
 \includegraphics[width = 0.7\textwidth]{LGS.png}
\end{center}
\caption{Geometry of laser guide star propagation to the telescope entrance
pupil.}
\label{fig:lgs-geom}
\end{figure}

Three coordinate systems are involved:
\begin{enumerate}
	\item Global or $g$-system is the Cartesian coordinate system with respect
	to which all other coordinate systems are defined. The orientation pair
	$(\bm{o},\mathcal{R})_{g}^{0}$ for this system is just ($\bm{0},\mathcal{I}$).
	\item Telescope or $t$-system is the local Cartesian coordinate system
	rotating with respect to the $g$-system such that the $t$-system $z$-axis is
	always along the telescope optical axis. The orientation pair
	$(\bm{o},\mathcal{R})_{t}^{g}$ for this system describes the telescope
	pointing. If we assume that at zenith pointing the $t$-system coincides with
	the $g$-system, then \\
	\begin{equation} \label{eq:t-origin}
		\bm{o}^{g}_{t} = \bm{o}^{0}_{g} = \bm{0},
	\end{equation}
	\begin{equation} \label{eq:t-rotation}
		\mathcal{R}^{g}_{t} = \texttt{Eu}(\alpha^{g}_{t},\beta^{g}_{t},0),
	\end{equation}
	where Euler angles $\alpha^{g}_{t},\beta^{g}_{t}$ have meaning of the
	telescope azimuth and zenith angles measured with respect to the g-system
	(note the $g$ superscript). The standard Euler rotation is
	\begin{equation} \label{Euler}
		\texttt{Eu}(\alpha,\beta,\gamma) = \mathcal{A}_{z}(\alpha)
		                                   \mathcal{A}_{y}(\beta)
		                                   \mathcal{A}_{z}(\gamma),
	\end{equation}
	$$ \mathcal{A}_{z}(\alpha) = \left[
     \begin{array}{crc}
	     \cos{\alpha} & -\sin{\alpha} & 0 \\
	     \sin{\alpha} & \cos{\alpha} & 0 \\
	     0 & 0 & 1 \\
		 \end{array} \right],
	$$
	$$ \mathcal{A}_{y}(\beta) = \left[
     \begin{array}{rcc}
	     \cos{\beta} & 0 & \sin{\beta} \\
	     0 & 1 & 0 \\
	     -\sin{\beta} & 0 & \cos{\beta} \\
		 \end{array} \right].
	$$
	Coordinates of point $p$ in the entrance pupil given in $t$-system are
	$[x,y,0]_{p}^{t} = \bm{r}_{p}^{t}$.
	\item Laser launch telescope or $l$-system is a local Cartesian coordinate
	system chosen such that its $z$-axis is always along the optical axis of the
	LLT. This system is naturally defined with respect to the $t$-system because
	the LLT is mounted on the moving main telescope mount:
	\begin{equation} \label{eq:l-system}
		(\bm{o},\mathcal{R})_{l}^{t} = ([x,y,0]_{l}^{t},
                                 \texttt{Eu}(\alpha_{l}^{t},\beta_{l}^{t},0),
	\end{equation}
	where $[x,y,0]_{l}^{t} = \bm{r}_{l}^{t}$ are the $l^{th}$ LLT location in
	$t$-system (pupil
	coordinates), $\alpha_{l}^{t},\beta_{l}^{t}$ are the azimuth and zenith
	angles of the LLT orientation with respect to the main telescope optical
	axis. Note that, for simplicity, we count one LGS per one LLT. In reality
	one LLT of GMT will generate a pair of LGSs. To account for this we simply
	assume that there are two differently oriented virtual LLTs at the location
	of one real LLT.

	The LGS is modeled as a combination of the point sources (PSR) distributed
	within the \texttt{Na} layer. Coordinates
	$\bm{r}_{li}^{l}, \,\, l =
	1,...,\#LLT, \,\, i=1,...,\#PSR$ of the sources are given in the $l$-system.
\end{enumerate}

Given orientation pair $(\bm{o},\mathcal{R})_{b}^{a}$ defining $b$-system
coordinates with respect to $a$-system coordinates, the transformation of
coordinates written in $b$-system into the same coordinates written in
$a$-system is the \emph{direct affine transform}: \index{affine transform}
\begin{equation} \label{eq:direct-affine}
	\bm{r}^{a} = \mathcal{R}_{b}^{a} \bm{r}^{b} + \bm{o}_{b}^{a}.
\end{equation}
Correspondingly, the \emph{inverse affine transform} gives the coordinates
written in $b$-system from the ones written in $a$-system:
\begin{equation} \label{eq:inverse-affine}
	\bm{r}^{b} = (\mathcal{R}_{b}^{a})^{T} (\bm{r}^{a} - \bm{o}_{b}^{a}).
\end{equation}

\subsection{Finding LGS spot elongation and orientation on the detector}
\label{sec:elongation-orientation}

An important geometrical calculation needed for the theoretical evaluation of
the detector noise covariance matrix is to find the parameters of the
elongated LGS image in the detector focal plane.

Let a LGS is produced as a light-emitting column of length $L$ inside the
\texttt{Na} layer. We need to find the length of its image in the
Shack-Hartmann WFS focal plane behind a lenslet with location $\bm{r}^{t}_{l}$
(in the $t$-system)
projected on the telescope entrance pupil and the image orientation with
respect to the detector pixel grid. Let the beginning and end of the LGS light
column have $t$-system coordinates $\bm{r}^{t}_{1}$ and $\bm{r}^{T}_{2}$,
respectively. Consider vectors $\bm{r}^{t}_{1t} =
\bm{r}^{t}_{1}-\bm{r}^{t}_{l}$ and $\bm{r}^{t}_{2t} =
\bm{r}^{t}_{2}-\bm{r}^{t}_{l}$. Then, up to a scaling factor and a possible
mirror flip, the LGS
image on the detector is determined by a $(\varepsilon,\theta)$-pair, where
$\varepsilon$ is the angular size of the LGS column as seen from the lenslet
center $\bm{r}_{l}^{t}$, that is, the angle between vectors $\bm{r}^{t}_{1l}$
and $\bm{r}^{t}_{2l}$, and $\theta$ is the angle
between the $t$-system x-axis and the line of intersection of the $t$-system
xy-plane and the plane made by vectors $\bm{r}^{t}_{1l}$ and $\bm{r}^{t}_{2l}$.
Vectors $\bm{r}_{1,2}$ are most conveniently definable in the $l$-system,
where
\begin{equation} \label{eq:lgs-ends-l}
	\bm{r}^{l}_{1,2} = [0 \, 0 \, z^{l}_{1,2}]^{T}.
\end{equation}
The transformation to $t$-system is
\begin{equation} \label{eq:lgs-ends-t}
	\bm{r}^{t}_{1,2} = \mathcal{R}^{t}_{l} \bm{r}^{l}_{1,2} + \bm{o}^{t}_{l}.
\end{equation}
Using scalar and vector products we get
\begin{eqnarray} \label{eq:epsilon-theta}
	\hat{\bm{r}}^{t}_{1,2} = \frac{\bm{r}^{t}_{1,2}}{|\bm{r}^{t}_{1,2}|}, \\
	\cos \varepsilon = \hat{\bm{r}}^{t}_{1} \cdot \hat{\bm{r}}^{t}_{2}, \\
	\bm{p}^{t}_{12} = \hat{\bm{r}}^{t}_{1} \times \hat{\bm{r}}^{t}_{2}, \\
	\sin \varepsilon = |\bm{p}^{t}_{12}|, \\
	\hat{\bm{p}}^{t}_{12} = \frac{\bm{p}^{t}_{12}}{\sin \varepsilon}, \\
	\hat{\bm{x}}^{t} = [1 \, 0 \, 0]^{T}, \\
	\cos \theta = \hat{\bm{x}}^{t} \cdot \hat{\bm{p}}^{t}_{12}, \\
  \sin \theta = | \hat{\bm{x}}^{t} \times \hat{\bm{p}}^{t}_{12} |.
\end{eqnarray}
Note that, since $\varepsilon$ is very small, in order to preserve accuracy,
all calculations need to be done in double precision.

\subsection{Free-space propagation from LGS to telescope entrance pupil}
\label{sec:free-space-propagation}

The electric field $E$ from the LGS on the telescope entrance pupil is the
superposition of spherical waves emitted from each point source (PS) that
makes the elongated laser guide star spot in the \texttt{Na} layer:
\begin{equation} \label{eq:spherical-superposition}
	E_{lp} = \sum_{i=1}^{\#PSR} w_{li} \frac{\exp(i k r_{lip})}{r_{lip}},
\end{equation}
where $w_{li}$ is the weight describing relative intensity of the $i^{th}$ PS
of the $l^{th}$ LGS, $r_{lip}$ is the distance from the $i^{th}$ PS
of the $l^{th}$ LGS to $p^{th}$ point in the telescope entrance pupil. This
distance is easily found through the affine transform:
\begin{equation} \label{eq:source-to-pupil}
	\bm{r}_{li}^{t} = \mathcal{R}_{l}^{t} \bm{r}_{li}^{l} + \bm{o}_{l}^{t},
\end{equation}
$$ r_{lip} = |\bm{r}_{li}^{t} - \bm{r}_{p}^{t}|. $$

\subsection{Geometrical optics propagation through atmosphere}
\label{sec:geom-propagation}

The turbulent atmosphere on the way between an LGS and a telescope is modeled
as a set of infinitely thin random \emph{phase screens} (PS)
\index{phase screen}. Because of small phase perturbations caused by each
layer and because the typical scale of turbulence is much larger than a
wavelength the geometrical optics model for propagation through the layers is
assumed. The model is based on the following postulates:
\begin{itemize}
	\item The propagation of the electromagnetic waves is treated as propagation
	of \emph{rays} that are normals to the constant phase surfaces of the waves,
	the \emph{wavefronts}. \index{ray} \index{wavefront}
	\item Rays are always straight: the phase screens are weak enough for not
	to change direction of rays, they only add a path difference
	$\delta r_{lip,j}$:
	\begin{equation} \label{eq:path-diff}
		\delta r_{lip,j} = \phi_{lip,j}/k, \,\, j = 1,...,\#PS,
	\end{equation}
	$$ r_{lip}^{turb} = r_{lip}^{free} + \sum_{j=1}^{\#PS} \delta r_{lip,j}. $$
	where $\phi_{lip,j}$ is the phase on the $j^{th}$ turbulent layer
	intersected by a ray emitting from $i^{th}$ point source of $l^{th}$ LGS
	towards $p^{th}$ point in the entrance pupil. So, the electric field
	on the entrance pupil will be
	\begin{equation} \label{eq:spherical-superposition-turb}
		E^{turb}_{lp} =
		\sum_{i=1}^{\#PSR} w_{li}
		\frac{\exp(i k r_{lip}^{turb})}{r_{lip}^{free}}.
	\end{equation}
\end{itemize}

Thus, it is necessary to find the intersection of a ray with a turbulent layer.

\subsubsection{Turbulence layers are perpendicular to the telescope optical
axis.}

Assume that the turbulence layers are chosen such that they are perpendicular
to the telescope optical axis regardless of the pointing.
In this case the turbulence strength depends on the azimuth angle
$\beta_{t}^{g}$, namely,
\begin{equation} \label{eq:turb-vs-beta}
	C_{n}^{2} (\beta_{t}^{g}) = \frac{C_{n}^{2}(0)} {\cos \beta_{t}^{g}}.
\end{equation}

Simple geometrical analysis gives for the
relationship between the pupil and turbulence layer coordinates:
\begin{equation} \label{eq:pupil-to-layer}
  \bm{r}^{t}_{lip,j} =
  \frac{h_{j}}{z^{t}_{li}} \bm{r}_{p}^{t} +
  (1-\frac{h_{j}}{z^{t}_{li}})
  \left[
  \begin{array}{c}
	  x^{t}_{li} \\
	  y^{t}_{li} \\
	  0
	\end{array}
  \right], \,\,
  \bm{r}^{t}_{li} =
  \left[
  \begin{array}{c}
	  x^{t}_{li} \\
	  y^{t}_{li} \\
	  z^{t}_{li}
	\end{array}
  \right],
\end{equation}
where $h_{j}$ is distance between the telescope and $j^{th}$ phase screen
along the telescope optical axis,
$\bm{r}^{t}_{lip,j}$ are xyz-coordinates in $t$-system of an
intersection point on phase screen $j$ for a ray emitting from $i^{th}$ point
source of $l^{th}$ LGS with $t$-coordinates $\bm{r}^{t}_{li}$ and
passing through $p^{th}$ point in telescope entrance pupil with $t$-coordinates
$\bm{r}^{t}_{p}$. For the source at infinity (a natural guide star or a
scientific target) Eq. (\ref{eq:pupil-to-layer}) simplifies to
\begin{equation} \label{eq:pupil-to-layer-inf}
	\bm{r}^{t}_{lip,j} =
	\left[
	\begin{array}{c}
		\cos \alpha^{t}_{li} \\
		\sin \alpha^{t}_{li} \\
		0 \\
	\end{array}
	\right] h_{j} \tan \beta^{t}_{li} + \bm{r}^{t}_{p},
\end{equation}
where $(\alpha,\beta)^{t}_{li}$ are the angular coordinates (first and second
Euler angles) of the light source. Note that in both cases the position of the
ray intersection with a layer can be written as $ (a \bm{r}_{p}^{t} +
\bm{b}^{t}_{li}) $.

\subsubsection{Turbulence layers are parallel to the ground.}

Assume that the layers are parallel to the ground, i.e. perpendicular to the
$g$-system's $z$-axis, and have altitudes $h_{j}, \,\, j = 1,...,\#PS$. In
this case the turbulence $C_{n}^{2}$ profile is not altered with the zenith
angle. Direct ray tracing technique is used to find intersection with a
turbulence layer.

Coordinates of a ray can be described through the parametric equation:
\begin{equation} \label{eq:ray-equation}
	\bm{r}(t) = \bm{i} t + \bm{r}_{0},
\end{equation}
where $\bm{r}_{0}$ is the ray origin (position of the light-emitting source),
$\bm{i}$, $|\bm{i}| = 1$ is the ray
direction vector, $t$ is the ray path length. In the case of propagation path
shown of Fig. \ref{fig:lgs-geom}
$$ \bm{r}^{l}_{0} = \bm{r}^{l}_{li}, $$
where $\bm{r}_{0}$ is given in $l$-system,
$$ \bm{i}^{t} = \frac{\bm{r}^{t}_{p} - \bm{r}^{t}_{li}}
                 {|\bm{r}^{t}_{p} - \bm{r}^{t}_{li}|}, $$
where $\bm{i}$ is given in $t$-system. Since the turbulent layers are most
conveniently defined in the $g$-system, the ray coordinates need to be
transformed into $g$-system:
\begin{equation} \label{eq:ray-transform}
	\bm{r}^{t}_{0} = \mathcal{R}_{l}^{t} \bm{r}^{l}_{0} + \bm{o}_{l}^{t},
\end{equation}
$$ \bm{i}^{t} = \frac{\bm{r}^{t}_{p} - \bm{r}^{t}_{0}}
                 {|\bm{r}^{t}_{p} - \bm{r}^{t}_{0}|}, $$
$$ \bm{r}^{g}_{0} = \mathcal{R}_{t}^{g} \bm{r}^{t}_{0} + \bm{o}_{t}^{g}, $$
$$ \bm{i}^{g} = \mathcal{R}_{t}^{g} \bm{i}^{t}. $$
The coordinates of ray intersection with turbulent layer are found by equating
$z$-coordinate of a ray to the layer altitude:
\begin{equation} \label{eq:layer-intersection}
	t = \frac{h_{j}-r^{g}_{0z}}{i^{g}_{z}},
\end{equation}
$$ \bm{r}^{g}_{lip,j} = \bm{i}^{g} t + \bm{r}_{0}^{g}, $$
where $x$- and $y$-coordinates of the $\bm{r}^{g}_{lip,j}$-vector are used to
find the phase $\phi^{turb}(\bm{r}^{g}_{lip,j}) = k \delta r^{turb}_{lip,j}
(\bm{r}^{g}_{lip,j})$ on the phase
screen corresponding to the ray intersection and substitute it to Eq.
(\ref{eq:spherical-superposition-turb}).

\subsection{Point source distribution in the LGS}
\label{sec:point-distribution}
TBC

The positions $\bm{r}^{l}_{li}$ and weights $w_{li}$ of point sources making
each LGS are found from superposition of the intensity distribution of the laser
radiation forward-propagated through atmosphere to the \texttt{Na} layer and
the vertical distribution of the \texttt{Na} density. A possible operation
flow for defining the LGS distribution is the following:
\begin{enumerate}
	\item Define a 3D mesh covering a part of the \texttt{Na} layer penetrated
	by the laser radiation. Each cell of this mesh is an elementary volume for
	which amount of laser flux is assigned.
	\item Center of each 3D mesh cell is the candidate location of the LGS point
	source. If flux through the cell exceeds a threshold, a point source is
	assigned for this cell.
	\item For each mesh cell for which a point source is assigned multiply the
	cell flux by \texttt{Na} density at the cell center taken from the
	\texttt{Na} vertical profile. Find relative distribution of the return flux,
	which is the source weights $\{ w_{li} \}_{i=1}^{\#PSR}$:
	\begin{equation} \label{eq:source-weights}
		w_{li} = \frac{ \Phi_{li} C^{\texttt{Na}}_{li} }
		               { \sum_{i=1}^{\#PSR} \Phi_{li} C^{\texttt{Na}}_{li} },
		           \,\, i = 1, ... , \#PSR
	\end{equation}
	where $\{ \Phi_{li} \}_{i=1}^{\#PSR}$ are the cell fluxes,
	$\{ C^{\texttt{Na}}_{li} \}_{i=1}^{\#PSR}$ are the \texttt{Na} abundances at
	the cell locations.
\end{enumerate}

\subsection{Return flux in the entrance pupil}
\label{sec:return-flux}

The photon flux returning from each LGS point source is considered uniformly
distributed over the spherical surface of the wavefront. Since the solid angle
$\Omega$ at which the telescope is seen from the source is typically very
small, the variation of $r^{free}_{lip}$ in the denominator of Eq.
(\ref{eq:spherical-superposition-turb}) over the pupil can be neglected. Thus,
the fraction of energy emitting from a point source that passes through the
telescope entrance pupil is
\begin{equation} \label{eq:energy-fraction}
	\frac{\Omega}{4 \pi} = \frac{1}{4 \pi}
	\frac{ A_{p} }
	     { |\bm{r}^{t}_{li}|^{2} \cos^{2} \beta^{t}_{li} },
\end{equation}
where $A_{p}$ is the pupil area, $|\bm{r}^{t}_{li}|$ is the distance from
pupil center to the point source, $\beta^{t}_{li}$ is the angle between the
telescope optical axis and the direction to the point source as seen from the
pupil center,
\begin{equation} \label{eq:angle-to-source}
	\cos \beta^{t}_{li} = \left( \frac{\bm{r}^{t}_{li}}
	                                  {|\bm{r}^{t}_{li}|} \right)_{z}.
\end{equation}
The photon flux into the entrance pupil from a point source is
\begin{equation} \label{eq:return-flux}
	\Phi_{li} = \tau T_{AOS} \frac{T_{ATM}}{\cos{\beta^{g}_{li}}}
	            s^{\texttt{Na}} C^{\texttt{Na}}
	            P_{l} w_{li}
	            \frac{\Omega}{4 \pi}, \,\, [\texttt{photons}],
\end{equation}
where \\
\begin{tabular}{lcll}
	&&& \\
  Exposure time & : & $\tau$ & = 2 ms, \\
  Atmosphere transmittance & : & $T_{ATM}$ & = 0.89, \\
  AO system transmittance & : & $T_{AOS}$ & = 0.448, \\
  Sodium coupling efficiency & : & $s^{\texttt{Na}}$ &
  = 130 (photons m$^{2}$ )/(s W atom), \\
  Sodium abundance & : & $C^{\texttt{Na}}$ & = $2.1\times10^{13}$ atoms/m$^2$,\\
	Laser power per LGS & : & $P_{l}$ & = 20 W, \\
	&&& \\
\end{tabular} \\
$\beta^{g}_{li}$ is the angle between zenith direction and the point source
direction as viewed from the entrance pupil center.

The fluxes from all point sources are later summed on the detector.


\chapter{Summary}
\label{summary}

\section{Project Summary}
In this thesis, the computational fluid dynamics toolbox OpenFOAM was used to simulate detonation waves within a square tube. Different solvers were tested until the hybrid solver \verb|rhoReactingCentralFoam| was decided as best due to the use of the Kurganov and Tadmor central schemes which reproduce compressible, high speed flow with good accuracy. Different ignition methods were tested, and gradient ignition was found to produce results with the least noise. Next, the chemistry was tested for sensitivity and then matched to Chapman-Jouguet targets and the automatic time stepping algorithm was tested to determine how large the CFL number could be allowed to increase before numerical ringing and noise reduced solution accuracy. A value was found that agreed with similar detonation modeling published works. Static computational meshes were then compared to determine the required mesh resolution to resolve shock and detonation structures. The threshold for resolving finer flow field structures as well as the von Neumann spike was determined, as well as the threshold for overall convergence and detonation wave shape integrity. Detonation cells were shown to be able to form with the solver. Adaptive meshing parameters were then explored, namely refinement levels, buffer layers, and refinement/unrefinement bounds. It was shown that refinement levels and the lower refinement bound are exponential in expense and the buffer layers are linear in expense. No more than 4 buffer layers led to a significant increase in solution accuracy, and a normalized pressure gradient lower bound smaller than 0.05 would significantly increase computational cost. Refinement levels started to converge and reproduce the von Neumann spike for 4 and more levels of refinement for the 250-40-1 mesh, which corresponds to a mesh resolution of 4000-640-1 mesh, where the von Neumann spike began to resolve in the static mesh. Lastly, some AMR simulations were compared to static mesh simulations. It was found that using AMR one could potentially reduce the computational expense by 85\% matching a static profile nearly exactly, and reduce computational expense by over 96\% by allowing the detonation spacial position to vary while still targeting peak solution values.

\section{Next Steps}
\subsection{Areas to Improve}
Increased knowledge as to the effects of AMR in three-dimensional simulations versus the use in two-dimensional simulations may be valuable. Two-dimensional simulations were the primary target of this thesis work due to a balance of feasibility in computational time as well as sufficient expense to justify and show the benefits of AMR.

An exploration into different tracking parameters other than the normalized gradient of pressure, pressure, temperature, and velocity is useful as a more efficient parameter (i.e., determining a parameter that better describes where refinement is absolutely necessary) may further reduce computational cost. While pressure, temperature, and velocity were all tested during the thesis work, tuning them was found to be more difficult than the normalized pressure gradient. Additionally, combining parameters together may produce smarter AMR ``active'' periods as the refinement is not free and can be potentially more expensive than static mesh cases if not carefully set up.

Better load balancing in parallel computing in OpenFOAM could use improvement. Currently, intelligent setup and awareness of the detonation itself is required to ensure that the AMR does not offset considerable computational load onto one processor or another as the detonation moves through the domain. For detonation tubes, this is not a problem as long decompositions can be used to balance the processor load. However, for simulations of RDEs or rocket engine combustion chambers that can be inherently unpredictable, smart domain partitioning is difficult. Like adaptive meshing, an adaptive domain decomposition may assist with improving load balancing in these scenarios.

Further characterization as to how the base mesh affects the solution should be done. While this is seemingly obvious as increased base resolutions will help give a better solution, the AMR can unrefine back to the base resolution, so if large turbulent structures or other larger structures want to be resolved still after adaptive unrefinement, the base resolution needs to be considered.

The AMR routines in parallel can be unstable. This does not seem to affect the solution, but it is not feasible to ``babysit'' a simulation and restart it if it crashes. The crashes seem to be due to AMR cells not getting communicated correctly with MPI, leading to a disproportionate number of cell faces shared at domain-decomposed boundaries where the MPI communication is occurring. Further work here is needed, and it will likely improve as development continues on the AMR routines.

\subsection{Future Work}
Further work in detonation modeling in the context of RDEs is the logical next step. This technology is still emerging and having better characterization of the highly chaotic flow field inside the engine will guide better engineering design and analysis towards improved propulsion technology. AMR applied to RDE simulation in OpenFOAM will reduce computational cost, especially for three-dimensional RDE simulations.

Deflagration to detonation transition modeling is another area to be explored with this solver. Characterizing this will allow the solver to be used in a wider set of scenarios, where sudden detonation of reactants is a potential concern.

Personally, I think application of AMR simulating the high-speed and chaotic environments of rocket engine combustion chambers in OpenFOAM would be really interesting. Combustion instability within these environments is still very hard to characterize and an ongoing topic of research and development within industry.

\section{Impact}
The targeted impact area for this research is the field of detonation modeling, with consideration for future use in RDE modeling and other propulsion systems that are expensive computationally. By utilizing the computational tools developed, tested, and validated in this project, existing research in detonation modeling using the computational fluid dynamics toolbox OpenFOAM can be performed more quickly or to a greater extent. Specifically, I showed:
\begin{itemize}
\item The solver \verb|rhoReactingFoam| and \verb|rhoCentralFoam| themselves are unable to model detonations accurately, or at all;
\item The solver \verb|rhoReactingCentralFoam| is able to simulate detonations;
\item How to simulate detonations in OpenFOAM and what the parameters will do to the solution;
\item Detonation cells can be modeled with \texttt{rhoReactingCentralFoam};
\item Results are sensitive to Arrhenius pre-exponential factor order, and care must be taken such that the shock does not decouple from the flame;
\item CFL number needs to be lower than typical high-speed flows due to the reactive nature of detonations;
\item How fine PDE/detonation tube meshes must be to resolve fine detonation and shock structure;
\item Certain AMR parameters have different effects computationally, and tuning certain parameters over others will optimize the overall cell count by placing refinement cells where they are truly needed;
\item Adaptive mesh refinement can decrease computation cost for detonation modeling up to 96\% while remaining true to the static mesh resolution results.
\end{itemize}


\section{Summary}
\label{sec:summary}
To summarize the thesis, we first explained how we trained and evaluated our word2vec model in \cref{sec:training-and-eval-our-word2vec-impl}. We showed that our data preprocessing steps and hyperparameter choices led to the training of a relatively high-quality word embedding model that, among other things, understood syntactic and semantic relationships. We also showed the ability of the model to identify underlying concepts of the English language.

Following, we performed a cluster analysis of word embeddings in \cref{sec:analysis-of-word-embeddings-word-clustering}. We showed that by performing clustering of cluster word embeddings and evaluating the results using internal validation methods, we end up with clusters that intuitively make sense. Additionally, we showed that by clustering word embeddings of distinct word groups, we deepen our understanding of the word embedding manifold. We also used dimensionality reduction methods to visualize results of clustering and showed that the clustering results from the clustering algorithms matched the clusters appearing in the plots.

Next, we investigated the idea of topological polysemy in \cref{sec:analysis-of-embeddings-topological-polysemy}, where we computed the topological polysemy of our word2vec models as well some pre-trained word embedding models. We saw that we were unable to reproduce the results presented in the original paper of topological polysemy, \cite{jakubowski2020topology}. In particular, between the topological polysemy of word embeddings and the true number of word meanings, we got lower correlations and relationships in the opposite direction of the results from \cite{jakubowski2020topology}. Due to the inconsistency in these results, we created a correlation matrix for comparing topological polysemy between our word embedding models and the pre-trained word embeddings. From this, we saw that the fastText model used in the experiments of \cite{jakubowski2020topology} did not correlate well with the rest of the models, signalizing that the word embeddings used in \cite{jakubowski2020topology} can not be compared to other word embeddings. In other words, by applying topological polysemy to other word embeddings than the ones used in \cite{jakubowski2020topology}, we can get unexpected and incorrect results. To deepen our understanding of how the topological polysemy was computed, we conducted two experiments by computing topological polysemy of two custom data sets. The two custom data sets consisted of two spheres intersecting in a single data point, in which topological polysemy should, in theory, be able to identify. The first data set was generated without noise, while the second data set had some noise added to it. The two data sets were created using varying dimensions of the spheres, ranging from 2 to 300. We used the sphere data set with no noise in the first experiment and the sphere data set with noise in the second experiment. From the first experiment, we showed that the measure of topological polysemy was unable to identify the intersection point, but rather a spherical area around the intersection point, for all dimensions of the spheres. In the second experiment, we showed that the measure of topological polysemy was unable to identify both the intersection point and the area around it for small sphere dimensions, $d \in \enclc{2, 3}$, but setting the sphere dimension to 300, we showed that it was possible to identify the intersection point, as it was a significant outlier.

Next, we showed the results of applying the Geometric Anomaly Detection (GAD) algorithm to our word embeddings in \cref{sec:analysis-of-embeddings-geometric-anomaly-detection}, where we compared the results to the number of word meanings, as perceived by topological polysemy and WordNet. Using the results from GAD, we were unable to effectively identify polysemous words. We note, however, that this might be due to a misconfiguration of the hyperparameters.

Following, we used ID estimation methods to compute the estimated local ID of word embeddings in \cref{sec:analysis-of-embeddings-intrinsic-dimension-estimation}, where we showed its relation to the number of word meanings perceived by WordNet, which indicated low correlations.

Finally, we created two supervised models estimating the number of word meanings and predicting whether or not a word is polysemous in \cref{sec:analysis-of-embeddings-supervised-polysemy-prediction}. We used the results from applying topological polysemy, GAD and ID estimation methods to word embeddings to create data used in the supervised models. Furthermore, we used this data to train and evaluate two supervised models for predicting the number of word meanings. We showed a weak correlation between the results from the first supervised model and the number of word meanings and visualized the feature importances in the model. For the second supervised model, we showed the results using confusion matrices, displaying the performance of the model when evaluated to words it has not seen yet, and we showed the feature importances in the model as well. Following, in the next section, we will conclude the thesis.


\chapter{Analysis of Results} % Chapter title
%
\label{ch:analysis}

\begin{figure}[h]
	\centering
	\includegraphics[width=\textwidth]{gfx/phew.pdf}
	\caption{Effect of Dirichlet prior against MAP with BREAK-2 ($\mu_1$) and with MAKE ($\mu_2$)}
	\label{phew}
\end{figure}

Before indexing the documents, the dataset must be split according to the desired BREAK variant.
For the usage of BREAK-n, note that one can store the positional vectors while indexing the corresponding term, or generate the vector directly during evaluation.
This choice highly depends on the time-space tradeoff.
However, in our experiments we have stored the vectors during indexing, prioritizing faster evaluation.

\section{Analysis of Dirichlet Prior}

The common trend between the experimental setup is that Dirichlet tends to perform better than Okapi BM25.
This is due to the fact that Dirichlet prior smoothing factor, takes account the language modelling, which in turn, makes out to be more successful than vanilla BM25.
We investigate the effect of Dirichlet prior with BREAK-2 $\mu_1$ and with MAKE in $\mu_2$.
Figure \ref{phew} shows that both reach the maximum MAP equally, however $\mu_1$ reaches earlier.

\section{Analysis of Hyperparameters}

Now we analyze the effect of weighted average equation of MAKE, the hyperparameter tuning of the factor $\lambda$.
We investigate the effect of hyperparameter sensitivity in isolated spelling correction $\lambda_1$ and with cognate detection in $\lambda_2$.
\begin{figure}[h]
	\centering
	\includegraphics[width=\textwidth]{gfx/phew1.pdf}
	\caption{The effect of hyperparameter sensitivity in isolated spelling correction $\lambda_1$ and with cognate detection in $\lambda_2$.}
	\label{phew1}
\end{figure}

Figure \ref{phew1} describes the optimal conditions of the suitable hyperparameters.
The late reach of $\lambda_2$ suggests that cognate detection experiments depend more on graphical error modelling, MAKE algorithm than the BREAK heuristic.


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 25, Section 4.7: 1, 4, 7, 11, 14}
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{1.}
$\begin{bmatrix}  6 & 9 \\ -2 & -4  \end{bmatrix} $
\subsection*{4.}
(i)
\subsection*{7.}
Yes and no. Column A is a four dimensional subspace of $R^4$, it coincides with $R^4$. Since the null space cannot be $R^3$ (Null A has 7 entries) Nul A must be a 3 dimensional subspace of $R^7$.
\subsection*{11.}
2
\subsection*{14. A)}
$A = \begin{bmatrix}  1 & 2 & 1 \\ 0 & 1 & 2 \\ -3 & -5 & 0  \end{bmatrix} $
\subsection*{14. B)}
Solving $ \begin{bmatrix}  x_1 \\ x_2 \\ x_3  \end{bmatrix} $ gets us $ \begin{bmatrix}  0 \\ 0 \\ 1  \end{bmatrix} $ and from that we can obtain $t^2 = 3(1 - 3t^2) - 2(2 + t - 5t^2) + ( 1 + 2t)$

\end{document}


In games of imperfect information, the players are not always sure of the current state of the game. That is, a state may be indistinguishable from other states for some players. This is a classic application of a possible worlds model. In this section based on \cite{benthem2001a}, we are going to combine game trees and possible worlds models in order to model games of imperfect information.

% { \color{red} I don't think bisimulations are important in this article. This is a introductory article on the connections between game theory and epistemic logic. Bisimulations are a more advanced tool for reducing the size of your games so that they are easier to handle. Hence, they are mostly a tool for gaining efficiency and efficiency is not in introductory topic. I would rather describe how to compute the Nash equilibrium of the Liar's dice example. }

\subsection{Perfect information game trees} \label{seq:perfect-information}

%{ \color{red} These is possibly too much theory in this subsection but I wasn't quite sure where this subsection was headed when I started. }

First, we consider two-player games of perfect information: Given a set $ \atomicprops $ of atomic propositions, a game tree $ M = (S, \{ R_{a} : a \in A \}, V) $ for such a game is a set $ S $ of states, a set containing one binary relation $ R_{a} \subseteq (S \times S) $ for each action $ a $ in the set $ A $ of all actions, and a valuation function $ V : S \rightarrow 2^{\atomicprops} $ \cite{benthem2001a}. The actions are also called moves. The states are partitioned among the players, so that each state is controlled by exactly one player. We let $ \atomicprops $ contain the propositions $ \turn_{i} $ and $ \win_{i} $ for each player $ i \in I $ in the set of players $ I $. We define $ V $ such that $ \turn_{i} $ holds of a state $ s $ if and only if player $ i $ controls $ s $. Also, we define $ V $ such that $ \win_{i} $ holds of a state $ s $ if and only if player $ i $ wins at $ s $. Finally, we let $ \leaf $ be true of exactly the leaves of $ M $.

In order to express properties of a state in a game $ M = (S, \{ R_{a} : a \in A \}, V) $, we introduce a modal language with the following syntax, where $ a \in A $ is any action and $ p \in \atomicprops $ is any proposition:
\begin{align*}
\alpha &::= a \barspace \alpha_{1} \cup \alpha_{2} \barspace \alpha_{1} \alpha_{2} \barspace \alpha^{\ast} \\
\beta &::= \langle \alpha \rangle \barspace [\alpha] \barspace \beta_{1} \beta_{2} \\
\gamma &::= \beta p
\end{align*}
Here, $ \alpha $ generates formulas describing sets of sequences of actions. The Kleene star denotes arbitrary finite iteration. The symbol $ \beta $ generates modalities and $ \gamma $ is the starting symbol. The expression $ \langle \alpha \rangle \phi $ denotes that $ \phi $ holds in some state resulting from performing one of the action sequences of $ \alpha $. The expression $ [\alpha] \phi $ denotes that $ \phi $ holds in all states resulting from performing all action sequences of $ \alpha $.

A \emph{strategy} $ \sigma : S \rightarrow A $ for player $ i $ is a partial function from a state $ s $ with $ \turn_{i} $ to an action available from $ s $. If every action occurs at most once in the game tree, we can view a strategy as a set of actions. With this view, a \emph{winning strategy} $ \sigma $ for player $ i $ is one satisfying the following \cite{benthem2001a}:
\begin{gather*}
\WIN_{i} \leftrightarrow (\leaf \wedge \win_{i}) \vee (\turn_{i} \wedge \langle A \rangle \WIN_{i}) \vee (\neg \turn_{i} \wedge [A] \WIN_{i}) \\
[A^{\ast}](\turn_{i} \rightarrow \langle \sigma \rangle \WIN_{i})
\end{gather*}
Here, we use the shorthand notation $ A $ for the $ \alpha $-formula $ a_{1} \cup a_{2} \cup \dots $ describing the choice between all possible actions $ a_{1}, a_{2}, \dots $. The first formula expresses that a state $ s $ is winning for player $ i $, if 1) the game ends at $ s $ with player $ i $ as the winner, or if 2) player $ i $ has a move from $ s $ that leads him to a winning state, or if 3) the opponent has the turn but all her moves lead to states that are winning for player $ i $. The second formula expresses that from every state $ s $, if player $ i $ has the turn from $ s $, then the move described by the strategy leads to a winning state.

\subsubsection*{The powers of the players}

What can we say about the outcomes that the players can force? Define $ \forcing{i} X $ to be the proposition that player $ i $ has a strategy from state $ s $ in $ M $ whose resulting states are always in the set $ X $ of leaves. It is obvious that $ \forcing{i} $ is closed under superset \cite{benthem2001a}. Let this property be C1. It is also clear that if $ \forcing{1} X $ and $ \forcing{2} Y $, then $ X $ and $ Y $ must overlap \cite{benthem2001a}. Otherwise, the players could force the game to end with inconsistent outcomes. Let this property be C2. Finally, games of perfect information are \emph{determined} meaning that for any set of leaves, one of the players must have a forcing strategy \cite{benthem2001a}. That is, if $ \neg \forcing{1} X $, then $ \forcing{2} (S - X) $ and vice versa, if we swap player $ 1 $ and player $ 2 $. We use C3 to denote determinacy.

%{ \color{red} We need this subsection so that we can contrast with games of imperfect information in later subsections. We only need to describe the \emph{the dynamic modal language}, if we want to write down concrete strategies for our example. }

%\begin{itemize} \color{red}
%\item Formal definition of perfect-information game trees
%\item Formal definition of strategies in perfect-information games.
%\item Short introduction to outcomes and powers (C1, C2, and C3).
%\end{itemize}

\subsection{Imperfect information game trees}

For games of imperfect information, we extend the definition of game trees from \secref{seq:perfect-information} with possibility relations describing the states that the players cannot tell apart. Hence, a game tree $ M = (S, \{ R_{a} : a \in A \}, \{ \sim_{i} : i \in I \}, V) $ of a game of imperfect information contains a possibility relation $ \sim_{i} $ for each of the players $ i $ in the set $ I $ of players.

In order to express properties of such games, we need to add the knowledge operators of epistemic logic to the modal logic defined in \secref{seq:perfect-information}. This allows us to write expressions like $ K_{2} \neg K_{1} (\langle a \cup b \rangle \win_{1}) $, which states that player $ 2 $ does not know that player $ 1 $ knows that at least one of the moves $ a $ and $ b $ leads to victory of player $ 1 $.

The introduction of uncertainty renders the above definition of a strategy troublesome. This definition allows player $ i $ to play move $ a_{1} $ from the state $ s_{1} $ and move $ a_{2} $ from the state $ s_{2} $, even in situations where player $ i $ cannot distinguish $ s_{1} $ and $ s_{2} $. To avoid this counterintuitive situation, we define \emph{uniform strategies} to be strategies where player $ i $ plays the same move from two states, if these are indistinguishable to player $ i $.

The concept of a \emph{winning strategy} also becomes less clear with the introduction of player uncertainty. Player $ i $ may play according to a strategy $ \sigma $ that guarantees a win but player $ i $ may not be aware of this. Is this a winning strategy? In other words, should we define winning strategies in terms of the actual outcomes or in terms of the knowledge of the outcomes? \cite{benthem2001a} suggests that the latter seems more natural and defines the notion of \emph{predictive strategies} which are defined as follows: A uniform strategy $ \sigma $ for player $ i $ is predictive with respect to $ \phi $, if during all possible plays played according to $ \sigma $, player $ i $ always knows that the outcome satisfies $ \phi $.

\subsubsection*{The powers of the players}

Since non-uniform strategies do not model the capabilities of the players in games of imperfect information, we need to update the definition of $ \forcing{i} $: We replace \emph{strategy} by \emph{uniform strategy} and leave the remaining parts of the definition unchanged. After this change, C1 and C2 still hold \cite{benthem2001a}. However, determinacy (C3) is no longer guaranteed as shown by the example in \figref{fig:non-determinancy}: Player $ 1 $ can force the game to end in $ \{ s_{1}, s_{2} \} $ or $ \{ s_{3}, s_{4} \} $ by playing $ c $ or $ d $, respectively. Player $ 2 $ can force the game to end in $ \{ s_{1}, s_{3} \} $ or $ \{ s_{2}, s_{4} \} $ by playing $ a $ or $ b $, respectively. Hence, we have $ \neg \forcing{1}\{ s_{2}, s_{3} \} $ but not $ \forcing{2}\{ s_{1}, s_{4} \} $, which disproves C3.

\begin{figure}[htb]
\centering
\begin{tikzpicture}[->,level/.style={sibling distance = 2.5cm/#1, level distance = 1.25cm}] ]
\node {1}
    child {
        node (e1) {2}
        child {
            node {$ s_{1} $} edge from parent node [left] {$ a $}
        }
        child {
            node {$ s_{2} $} edge from parent node [right] {$ b $}
        }
        edge from parent node [above left] {$ c $}
    }
    child {
        node (e2) {2}
        child {
            node {$ s_{3} $} edge from parent node [left] {$ a $}
        }
        child {
            node {$ s_{4} $} edge from parent node [right] {$ b $}
        }
        edge from parent node [above right] {$ d $}
    };
\path (e1) edge [-, dashed] node [above] {2} (e2);
\end{tikzpicture}
\caption{Game tree of an example game with imperfect information. First,  player $ 1 $ chooses between the moves $ c $ and $ d $. Player 2 cannot distinguish these moves, so she has to blindly choose between her moves $ a $ and $ b $. This figure is inspired by \cite[fig.~2]{benthem2001a}.}
\label{fig:non-determinancy}
\end{figure}

\subsubsection*{Limitations on the possibility relations}

The presented definition of games of imperfect information is rather broad. It allows for many different kinds of uncertainties. It is possible, that player $ i $ does not know, which move the opponent has played. This is the case of \figref{fig:non-determinancy}. However, more exotic kinds of uncertainties are also possible: It is possible that player $ i $ does not know the outcome of a move he just played himself. It is also possible that he does not know, whose turn it is, what his available moves are, or whether the game has ended. It seem natural to impose limitations on the game trees, such that these kinds of uncertainties are impossible.

First, it is natural to assume that the players know the outcomes of the moves they just played. If we assume that every state $ s $ is labeled with a unique atomic proposition $ s' $, then we can enforce this by requiring the following for all states $ s \in S $ and all actions $ a \in A $: $ \turn_{i} \wedge \langle a \rangle s' \rightarrow \langle a \rangle \know_{i} s' $. Note that this does not guarantee that the players know the outcomes of their moves, \emph{before} they play them. This property is captured by $ \turn_{i} \wedge \langle a \rangle s' \rightarrow \know_{i} \langle a \rangle s' $.

Second, it is natural to assume that the players know, who is next to make a move. This can be enforced by demanding $ \turn_{i} \rightarrow \cknow_{I} \turn_{i} $, where $ \cknow $ is the common knowledge operator and $ I $ is the set of players \cite{benthem2001a}.

Next, it seem natural that the players know what moves are currently possible. That is, the same set of moves should be possible from two states, if the player to move cannot tell apart these two states. Formally, we demand $ \turn_{i} \wedge \langle a \rangle \true \rightarrow \know_{i} \langle a \rangle \true $ for all moves $ a \in A $ \cite{benthem2001a}. If we instead want the currently available moves to be common knowledge, we can demand $ \langle a \rangle \true \rightarrow \cknow_{I} \langle a \rangle \true $.

Finally, if all players should be aware that the game has ended and if this should be common knowledge, we can impose the following limitation: $ \leaf \rightarrow \cknow_{I} \leaf $.

% { \color{red} The interchange principle (the beer example?)}

%\begin{itemize} \color{red}
%\item Formal definition of imperfect-information game trees
%\item Strategies in imperfect games (the discussion from V.3). Maybe it is better to describe this after perfect recall.
%\item Short introduction to outcomes and powers in imperfect games (C3 does not hold anymore).
%\item Remarks on the looseness of this definition: "Players need not know what the opponent has played, or what they played themselves, they need not know if it is their turn, or whether the game has ended, etc. One can think up plausible scenarios with all of the pictures shown in Figure 6."
%\item How to restrict the imperfect-information-game-trees model to be more "realistic": Describe the examples from "III.3. Constraints for special axioms".
%\end{itemize}

\subsection{Liar's Dice} \label{sec:liars-dice}

As an example of a game of imperfect information, we now consider the game Liar's Dice. In this game\footnote{A variety of dice games are called Liar's Dice. They all have elements of concealed die rolls and deception. The variant we consider is probably the simplest.}, two players take turns rolling a single die under a cup, privately looking at the result, and then call out an outcome of the die roll. The player may lie about the outcome of the roll. Now, the opponent can accept the call and start a new round by re-rolling the die or she can challenge the call by lifting the cup. If the call was correct, she looses the game and if the call was a lie, she wins the game. In each round, a player must call out a greater outcome than the one called in the previous round.

Given a die with $ k $ faces, this can be formalised as follows \cite{ferguson1991}. In round $n$ player $i$ has the turn and rolls the die. She observes the outcome, which is a random integer $X(n)$ taking the values from $ 1 $ to $ k $ with equal probabilities. Then she announces an integer $y(n)$ between $y(n-1)+1$ and $ k $, both inclusive, with $y(0) = 0$. The next player $j$ then announces whether she doubts or believes the claim. If $y(n)=k$ then she always doubts. If she doubts, then $j$ wins if $X(n) < y(n)$ and $i$ wins otherwise. If she believes, then round $n+1$ begins and $j$ has the turn. A game tree for this game is shown in \figref{fig:liars-dice-tree}.

\begin{figure}[htbp]
\centering
\input{pic/liars-dice-tree.tex}
\caption{Game tree of the Liar's Dice game described in \secref{sec:liars-dice}. In this instance of the game, the "die" has two faces. We let N, 1, and 2 denote nature, player 1, and player 2, respectively. In order to keep the figure simple, we only depict when an uncertainty first arises. These uncertainties propagate down the tree under the rule that the two players can distinguish all actions except from the ones performed by nature.}
\label{fig:liars-dice-tree}
\end{figure}

What can we say about the uncertainties in this game? The player not rolling is uncertain about the move of nature but this is the only type of uncertainty. All players know the outcomes of their own actions. Also, because we have no vertical uncertainty lines in the game tree, all players always know, whose turn it is. Finally, all players always know the currently available moves and all players know when the game has ended.

Using backward induction (see \secref{sec:rationality}) can we predict the outcome of the game shown in \figref{fig:liars-dice-tree}? First, assume that each player has one util at stake and that the winner takes it all. Then, it is obvious that the two states labeled "roll${}_{2}$" have expected value zero, as the die alone determines the winner from these states and each outcome is equally likely. If player $ 1 $ call out "1" after the first roll, then player $ 2 $ always looses by lifting, so see always rolls. Hence, calling "1" yields player $ 1 $ zero utils in expectancy.

If player $ 1 $ rolls 1 but calls "2", she always looses. If player $ 2 $ rolls 2 and calls "2", she always wins. Hence, the expected payoff from rolling $ 1 $ is zero and the expected value from rolling 2 is one. Therefore, in expectancy, player $ 1 $ earns 0.5 utils, which causes player $ 2 $ to loose 0.5 utils, since this is a zero-sum game. Thus, having the first roll is an advantage. Also, the optimal strategy for player $ 1 $ is to be honest and the optimal strategy for player $ 2 $ is to trust the call of player $ 1 $ and roll again. Notice that these are uniform strategies.

\subsection{Perfect recall}
Liar's Dice satisfies the property \emph{Perfect Recall}~\cite{benthem2001a}: The players remember their own previous moves, as well as the uncertainties they had at each stage. This can be shown, by looking at the game tree and verifying that, for any player $i$ and any two states related by the uncertainty relation $\sim_i$, the sequence of moves (made by $i$) leading to each of the two states are the same. Also, the uncertainties along these sequences are the same at all stages.

Note that for this game in particular, Perfect Recall by itself has little significance under the assumption that all agents are rational, since the rounds that came before the previous one shouldn't impact the decision for the current round.

%{ \color{red} Predictive strategies? Ask Nina\dots }

\subsection{Indistinguishable actions and product update}
We already introduced the notion of epistemic indistinguishability between states, modelled by the possibility relations $\sim_i$. Now, we will see that a natural next step is to add possibility relations between actions to our game models as well. With this expansion of the possibility relations, we obtain a more general model, as different actions can lead to the same outcomes. An example of indistinguishable actions in Liar's Dice are those performed by Nature, from the perspective of a player that doesn't have the turn.

The \emph{Product Update} mechanism is described as follows \cite{benthem2001a}:
$$
(x,a)\sim_i(y,b) \longleftrightarrow x\sim_i y \mbox{ and } a\sim_i b
$$
where $(x,a)$ and $(y,b)$ are ordered pairs consisting of 1) the previous state, and 2) the last performed action. It can be used iteratively to compute the next levels of game trees. \cite{benthem2001a} proves that the following three properties follows from Product Update:
\begin{enumerate}
\item[(a)] Perfect Recall.
\item[(b)] Propagation of uncertainty: If $x\sim_i y$ then after action $a$ in both cases $(x,a)\sim_i (y,a)$.
\item[(c)] Actions are either always or never distinguishable: If $(x,a)\sim_i (y,b)$ then whenever $u\sim_i v$ also $(u,a)\sim_i (v,b)$.
\end{enumerate}


\section{Introduction}

Weather forecast is important for people to make their schedule for the next day. Among those technical terms in the forecasting, temperature and precipitation are the most important for the public. Nowadays weather forecasts, especially the precipitation forecast is conducted with the help of satellite images of clouds. This requires knowledge in meteorology. So we wonder if we can predict whether it will rain tomorrow based on the basic weather record (like temperature, precipitation, wind), with the classification method we learned in class.

We choose Seattle for our study as it usually has half of time in a year raining, so the data are balanced. In order to make better forecasting, feature engineering is needed. Also the data comes in form of daily report by each station. Combining the weather report each day from different stations is also a challenge for our study.  The location of weather stations and relationship between variables are discussed in the section of data exploration. And we are going to use Random Forest, $k$-NN and SVM for classification. At last part we are going to use the models trained with Seattle data to predict precipitation in another city to see whether the models are generally usable.


\chapter{Developer Documentation}
\label{ch:impl}

Creation of the program for histopathologic cancer detection is divided into four major parts:
\begin{enumerate}
	\itemsep0em
	\item Assembling dataset structure (required for input to a convolutional neural network), image preprocessing (loading, removing noise, normalization, whitening) and data augmentation (expanding the size of the dataset by applying a series of random transformations to each image)
	\item Building the convolutional neural networks (class of deep neural networks applied to analyze images), and training them on data
	\item Improving prediction accuracy of networks (solving underfitting and overfitting problems) with hyperparameter tuning (choosing optimal parameters for learning algorithm) and changes to network architecture
	\item Creating a graphical user interface for the program, which allows user to load histopathologic slide and select network which has to be applied on it, and to get as output category to which that slide belongs, with the additional possibility to visualize network representations (heatmaps of class activations, filters of convolutional layers and intermediate activations)
\end{enumerate}
In the remainder of this chapter each part will be thoroughly analyzed, and directory/file structure of source the code will be illustrated, as well as use-case and class diagrams.

System requirements discussed in user documentation (\textcolor{red}{\hyperref[sysreq]{Section 2.1}}) apply to developer documentation as well.
\clearpage

\section{Program Structure}

Histopathologic Cancer Detection program is divided into six major modules (\textcolor{red}{\autoref{fig:dirdiag}}, \textcolor{red}{\autoref{fig:dirdiag2}}):
\begin{enumerate}
	\itemsep 0em
	\item Data - includes dataset creation (\textcolor{red}{\hyperref[createdata]{Section 3.4}})
	\item Models - includes creation, training and testing of CNNs (\textcolor{red}{\hyperref[cnn]{Section 3.5}\textcolor{black}{,} \\ \hyperref[vgg19]{Section 3.6}})
	\item Experiments - includes hyperparameter tuning (\textcolor{red}{\hyperref[exp]{Section 3.7}})
	\item Graphical User Interface - includes creation of all application windows and their interconnection (\textcolor{red}{\hyperref[gui]{Section 3.8}})
	\item Utilities - includes dataset analysis and visualization and network performance assessment and visualization (\textcolor{red}{\hyperref[utils]{Section 3.9}})
	\item Tests - includes unit testing of Data, Models, Experiments and Utilities (\textcolor{red}{\hyperref[tests]{Section 3.11}})
\end{enumerate}

\begin{figure}[h]
	\centering
	\includegraphics[scale=1.3]{code_structure_1.jpg}
	\caption{Diagram of directories and Python scripts of Data, Models, Experiments and Tests modules of Histopathologic Cancer Detection}
	\label{fig:dirdiag}
\end{figure}

\clearpage

\begin{figure}[h]
	\centering
	\includegraphics[scale=1.3]{code_structure_2.jpg}
	\caption{Diagram of directories and Python scripts of Tests, Graphical User Interface and Utilities modules of Histopathologic Cancer Detection}
	\label{fig:dirdiag2}
\end{figure}

\section{Use-Case Diagram}

One of the main goals of the Histopathologic Cancer Detection program was the ease of use, i.e. straightforward graphical user interface which makes complex operations look quite simple and effortless. Even though there are extremely advanced algorithms with millions of parameters behind the program, GUI was made in such a way that everyone can use it. The first step is loading the image and selecting tissue type (breast or colorectal tissue), after which classification is being done. At every step of the way, current work can be saved, and a new image can be loaded to start the process from scratch. After the classification, it is possible to visualize network representations and perform further analysis of the results by visualizing layer activations, network filters, and heatmaps (\textcolor{red}{\autoref{fig:usecase}}).

\begin{figure}[h]
	\centering
	\includegraphics[scale=1.35]{use_case.jpg}
	\caption{Use-Case Diagram of Histopathologic Cancer Detection}
	\label{fig:usecase}
\end{figure}

\section{Class Diagrams}

Classes of Histopathologic Cancer Detection can be divided into two main segments: window classes and neural network classes.

BaseCNN class is the common class of all neural network classes, and it contains common attributes, such as dataset name, network name, compile parameters, and common methods, such as the creation of data generators, compilation, and training of the network. Neural network classes will be discussed in more detail in \textcolor{red}{\hyperref[cnn]{Section 3.5}, \hyperref[vgg19]{Section 3.6}}.

\begin{figure}[h]
	\centering
	\includegraphics[scale=0.6]{nets_class_diagram.png}
	\caption{Class diagram of neural network classes of Histopathologic Cancer Detection}
	\label{fig:class1}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[scale=0.38]{main_class_diagram.png}
	\caption{Class diagram of window classes of Histopathologic Cancer Detection}
	\label{fig:class2}
\end{figure}

Window class is the base class of all window classes, and it implements common methods, such as setting up the window size and central widget. On the other side, MainWindow class is the central point of GUI, as it defines the window which appears when the program is run, and every other window is invoked from it (\textcolor{red}{\autoref{fig:class2}}). Window classes, along with their attributes and methods, will be discussed in more detail in \textcolor{red}{\hyperref[gui]{Section 3.8}}.

\section{Creation of Datasets}
\label{createdata}

Performance and accuracy of convolutional neural networks rely largely on datasets, i.e. on quality of available data, dataset size, class balance, etc. But before feeding data to the network, if using \texttt{Keras} API, certain dataset structures must be satisfied. More precisely, datasets must have the following structure: train, validation, and test directories, each with a subdirectory for each class. Scripts responsible for the creation of required directory structure and distribution of data are:
\begin{itemize}
	\itemsep 0em
	\item \texttt{break\_his\_dataset\_creation.py},
	\item \texttt{nct\_crc\_he\_100k\_dataset\_creation.py}.
\end{itemize}
They work by extracting datasets downloaded from \cite{breakhis_bib}, \cite{nctcrche100k_bib}, creating necessary directory tree, and distributing images between created subdirectories. After executing scripts, datasets are ready to be fed into convolutional neural networks (in order to train them), but before that, neural network architecture has to be built.
\clearpage

\section{CNNSimple Implementation}
\label{cnn}

CNNSimple convolutional neural network (\textcolor{red}{\hyperref[src:py1]{Code 3.1}}) was created using \texttt{Keras} Sequential model in order to classify images from NCT-CRC-HE-100K dataset, which contains 100.000 images divided into 9 tissue/cancer categories.

The convolutional base of CNNSimple is composed of four blocks of convolutional and max-pooling layers, where the first two blocks have two convolutional and one max-pooling layer, and the last two blocks have three convolutional and one max-pooling layer. Each convolutional layer has a 3$\times$3 convolution window, uses the ReLU activation function, and the number of feature maps (filters) increases exponentially from 32 to 256. Each max-pooling layer has pool size of 2$\times$2.

Classification top of CNNSimple is composed of two fully-connected layers, each followed by a dropout layer, and an output (also fully-connected) layer. Fully-connected layers use ReLU activation function, and number of neurons grows from 512 to 1024. Dropout layers use 50\% dropout rate (fraction of neurons which will be ignored in each passing). Output layer uses softmax activation function, and has nine neurons (one neuron per tissue/cancer subtype output).
\vspace{1mm}
\lstset{caption={CNNSimple network architecture (defined in \texttt{cnn\_simple.py})},label=src:py1}
\begin{lstlisting}[language={Python}, basicstyle=\scriptsize]
	model = Sequential()
	model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3),
	                 name='block1_conv1'))
	model.add(Conv2D(32, (3, 3), activation='relu', name='block1_conv2'))
	model.add(MaxPooling2D((2, 2), name='block1_pool'))
	model.add(Conv2D(64, (3, 3), activation='relu', name='block2_conv1'))
	model.add(Conv2D(64, (3, 3), activation='relu', name='block2_conv2'))
	model.add(MaxPooling2D((2, 2), name='block2_pool'))
	model.add(Conv2D(128, (3, 3), activation='relu', name='block3_conv1'))
	model.add(Conv2D(128, (3, 3), activation='relu', name='block3_conv2'))
	model.add(Conv2D(128, (3, 3), activation='relu', name='block3_conv3'))
	model.add(MaxPooling2D((2, 2), name='block3_pool'))
	model.add(Conv2D(256, (3, 3), activation='relu', name='block4_conv1'))
	model.add(Conv2D(256, (3, 3), activation='relu', name='block4_conv2'))
	model.add(Conv2D(256, (3, 3), activation='relu', name='block4_conv3'))
	model.add(MaxPooling2D((2, 2), name='block4_pool'))
	model.add(Flatten(name='flatten'))
	model.add(Dense(512, activation='relu', name='dense1'))
	model.add(Dropout(0.5, name='dropout1'))
	model.add(Dense(1024, activation='relu', name='dense2'))
	model.add(Dropout(0.5, name='dropout2'))
	model.add(Dense(9, activation='softmax', name='prediction'))
\end{lstlisting}

\section{Transfer Learning}
\label{vgg19}

Although CNNs are a powerful tool for image classification, in order to achieve high accuracy, a large amount of data is required. The problem occurs when only a small dataset is available (as is often in healthcare, ex. BreakHis dataset). In such cases transfer learning can be used: take a model trained on a large dataset and transfer knowledge to a small dataset, i.e. freeze convolutional base, and only train classification top of the network. The main idea is that early layers of convolutional base learn low-level features applicable across all images, such as edges and patterns.

\subsection{VGG19Simple Implementation}

VGG19Simple convolutional neural network (\textcolor{red}{\hyperref[src:py2]{Code 3.2}}) was created using \texttt{Keras} Graphical API in order to classify images from BreakHis dataset, which contains 2.081 images divided into 8 tissue/cancer categories.

Convolutional base of VGG19Simple is VGG19 \cite{simonyan2014very} pre-built network pre-trained on ImageNet \cite{deng2009imagenet} dataset (without top classification part), using imagenet weights.

Classification top of VGG19Simple is composed of two fully-connected layers, each followed by a dropout layer, and an output (also fully-connected) layer. Fully-connected layers use the ReLU activation function, and the number of neurons grows from 512 to 1024. Dropout layers use a 50\% dropout rate (fraction of neurons which will be ignored in each passing). The output layer uses softmax activation function, and has eight neurons (one neuron per tissue/cancer subtype output).

\vspace{3mm}
\lstset{caption={VGG19Simple network architecture (defined in \texttt{vgg19\_simple.py})},label=src:py2}
\begin{lstlisting}[language={Python}, basicstyle=\scriptsize]
	input = Input((150, 150, 3))
	convolutional_base = VGG19(weights='imagenet', include_top=False,
	                           input_tensor=input)
	for layer in convolutional_base.layers:
		layer.trainable = False
	x = Flatten(name='flatten')(convolutional_base.output)
	x = Dense(512, activation='relu', name='dense_1')(x)
	x = Dropout(0.5, name='dropout_1')(x)
	x = Dense(1024, activation='relu', name='dense_2')(x)
	x = Dropout(0.5, name='dropout_2')(x)
	x = Dense(8, activation='softmax', name='predictions')(x)

	model = Model(input, x)
\end{lstlisting}

\section{Experiments and Results}
\label{exp}

Performance of neural networks is determined by how well will it generalize, i.e. how high accuracy will it achieve on previously unseen data (if it performs well on training data, but underachieves on test data, it is said that CNN overfits). In order to prevent overfitting, a number of techniques can be used: increase the size of the dataset, change network architecture or apply hyperparameter tuning techniques, which consist of selecting a set of optimal hyperparameters for the learning algorithms. The selection of such parameters for networks is done in \texttt{hyperparameter\_tuning.py}. The first step consists of defining the hyperparameter dictionary with parameters and values to be tested, such as the number of epochs for which the network is to be trained, optimization techniques, etc. The next step consists of training a network with all combinations of parameters and values defined, after which network performances are compared in order to determine the optimal hyperparameter set.

CNNSimple network trained on the NCT-CRC-HE-100K dataset was trained for 35 epochs, using RMSProp optimizer with a learning rate of 0.00004, using a categorical cross-entropy loss function. Before feeding data to the network, data augmentation (applying random transformations in order to produce more images, such as translation, rotation, sheer) has been used. In order to assess the performance of the network, the accuracy metrics function was employed. CNNSimple achieved 93.89\% validation and 94.21\% test accuracy, and 0.24 validation and 0.06 test loss (\textcolor{red}{\autoref{fig:netsperf}}).

VGG19Simple network trained on the BreakHis dataset was trained for 90 epochs, using RMSProp optimizer with a learning rate of 0.0001, using a categorical cross-entropy loss function. Before feeding data to the network, data augmentation (applying random transformations in order to produce more images, such as translation, rotation, sheer) has been used. In order to assess the performance of the network, the accuracy metrics function was employed. After training a network with a frozen convolutional base, the last two convolutional blocks were unfrozen, and network was trained again using RMSProp optimizer with a learning rate of 0.00004. VGG19Simple achieved 85.3\% validation and 83.59\% test accuracy, and 0.72 validation and 1.02 test loss (\textcolor{red}{\autoref{fig:netsperf}}).

\begin{figure}[h]
	\centering
	\subfigure[\scriptsize CNNSimple Accuracy]{\label{fig:a}\includegraphics[width=72mm]{cnn_acc.png}}
	\subfigure[\scriptsize CNNSimple Loss]{\label{fig:b}\includegraphics[width=72mm]{cnn_loss.png}}
	\subfigure[\scriptsize VGG19Simple Accuracy]{\label{fig:vgg19perf}\includegraphics[width=72mm]{vgg19_acc.png}}
	\subfigure[\scriptsize VGG19Simple Loss]{\label{fig:b}\includegraphics[width=72mm]{vgg19_loss.png}}
	\caption{CNNSimple and VGG19Simple performance on BreakHis and NCT-CRC-HE-100K train and validation datasets (respectively)}
	\label{fig:netsperf}
\end{figure}

\begin{figure}[h]
	\centering
	\begin{minipage}{.5\textwidth}
		\centering
		\hspace*{-0.7cm}
		\vspace*{-0.45cm}
		\includegraphics[scale=0.37]{cnn_cf.png}
		\captionof{figure}{Confusion Matrix (a summary table of correct and incorrect predictions broken down by each class) of CNNSimple on NCT-CRC-HE-1OOK test dataset}
		\label{fig:cnncf}
	\end{minipage}%
	\begin{minipage}{.5\textwidth}
		\centering
		\hspace*{-0.7cm}
		\includegraphics[scale=0.37]{vgg19_cf.png}
		\captionof{figure}{Confusion Matrix of VGG19Simple on BreakHis test dataset}
		\label{fig:vgg19cf}
	\end{minipage}
\end{figure}

\clearpage

\section{Graphical User Interface}
\label{gui}

In order to make using program simple and straightforward, graphical user interface using PyQt5 has been created. Following window classes have been constructed:
\begin{itemize}
	\itemsep 0em
	\item Window class in \texttt{window.py}, used as base class for every other window, which contains primitive functions for setting up window, creating central widget, retranslating text, etc.
	\item AboutAuthorWindow class in \texttt{about\_author\_window.py}, which contains two labels (containing image and CV respectively)
	\item AboutDatasetsWindow class in \texttt{about\_datasets\_window.py}, which contains two labels (containing dataset sample images and dataset overview numbers respectively)
	\item AboutModelsWindow class in \texttt{about\_models\_window.py}, which contains five labels (containing network name, network architecture, accuracy, loss and confusion matrix plots respectively)
	\item SimpleWindow class in \texttt{simple\_window.py}, which contains one image label
	\item InspectConvWindow class in \texttt{inspect\_conv\_window.py}, which contains three labels (containing convolutional layer text, number text and image respectively), button (with associated action), combo box (containing network layer names)  and line edit (for filter/channel selection)
	\item MainWindow class in \texttt{main\_window.py}, which connects every other window and provides high-level program functionality
\end{itemize}
In addition, every window-specific information is located in \texttt{config.py}, such as window sizes, positions and names of labels, paths to images, etc. GUI component definitions are located in \texttt{gui\_components.py}.

\subsection{Main Window}

MainWindow class is the central part of the GUI, as it defines the window which appears when the program is started. It has a menu bar from which every other window can be reached, and a central widget which contains input image, tissue-type radio buttons, classify button and output class label, and probabilities plot. Function \emph{classifyButtonEvent}, associated with \emph{classifyButton} is an integral part of the class, as it loads network based on the tissue type of the input image, classifies it, and writes output to labels.

\section{Utilities}
\label{utils}

For program to work seamlessly, certain utilities needed to be implemented:
\begin{itemize}
	\itemsep 0em
	\item \texttt{dataset\_overview.py}, used for obtaining basic information about dataset, such as sample images and image distribution per class
	\item \texttt{misc.py}, containing functions for reading file contents, loading images, etc.
	\item \texttt{predict\_image.py}, used for predicting class to which an image belongs
	\item \texttt{save\_model.py}, used for saving neural network information and performance after being trained, such as arguments, architecture, accuracy, loss and confusion matrix plots, filters, etc.
	\item \texttt{visualize\_filters.py}, used for visualizing network filter patterns
	\item \texttt{visualize\_intermediate\_activations\_and\_heatmaps.py}, used for visualizing intermediate activations of the network, and heatmaps of class activation
\end{itemize}

\section{Implementing Additional Features}

In addition to the ease of use of the Histopathologic Cancer Detection program, source code was written in such a way to make expanding scope (problem space) of the program by including additional features quite simple and fast. If new tissue type (ex. lung tissue), along with tissue/cancer subtype classification was to be added to the program, it would be accomplished in the following four steps:
\begin{enumerate}
	\itemsep 0em
	\item \textbf{Creating dataset} \\
	Obtaining dataset of new tissue type, and preparing dataset to be fed to \texttt{Keras}-built CNN, which includes creating appropriate directory structure and distributing data
	\item \textbf{Building CNN} \\
	Crating new convolutional neural network class inherited from BaseCNN by defining data generator transformations, network architecture, etc.
	\item \textbf{Fine-tuning CNN} \\
	Defining hyperparameter dictionary and training neural networks in order to increase classification accuracy and prevent overfitting
	\item \textbf{Extending GUI} \\
	Adding an additional radio button to MainWindow class, and extending action associated with classify button, as well as further analysis actions, to use newly created network
\end{enumerate}

\section{Testing}
\label{tests}

Source code for Histopathologic Cancer Detection can be divided into two equal chunks: code responsible for CNN-related functionality and code responsible for GUI-related functionality.

The code responsible for CNN-related functionality has been unit-tested using \texttt{PyTest} testing framework, and has a code coverage over 93\%. Unit tests for data module can be found in \texttt{test\_data.py}, unit tests for models and experiments modules can be found in \texttt{test\_models\_experiments.py}, and unit tests for utilities can be found in \texttt{test\_utils.py}.

Code responsible for GUI-related functionality has been tested manually, where numerous histopathologic slides have been loaded and classified, as well as further analyzed, in order to inspect program functionality.

\section{External Code}

Main ideas for advanced use of Histopathologic Cancer Detection, i.e. for further analysis of CNN representations and visualization of heatmaps of class activations, intermediate activations and filters of convolutional layers, as well as some parts of source code, have been taken from \cite{chollet2018deep}. The code is licensed under MIT license.


\chapter{Temporal resolution of pulse detection}
\label{ch:timeres}

In this chapter we measure the temporal localization precision once the
presence of a signal pulse is established.

To this end, we simulate events each containing only one signal at a known
position. In principle we could use the LNGS data (\autoref{sec:lngsdata}), but
we do not know the jitter of the trigger pulse and we may reach a temporal
resolution below the sampling period, while in the simulation we know the exact
actual temporal location of signals.

\section{Event simulation}

Every event is the sum of a \SI1{PE} signal and a noise waveform. We do not add
a baseline, so the noise has mean zero and the signals taper down to zero. The
signals are negative. We use the same scale of the LNGS data; the scale does
not affect the results of this study since we are not simulating digitalization.
The following paragraphs describe in detail the event simulation procedure.

\subsection{Signal generation}
\label{sec:toysignal}

We generate the signal pulse shape according to the trigger-aligned template
from \SI1{PE} laser pulses in Tile~57, see \autoref{sec:cctemplate} and
\autoref{fig:template}.

The template is sampled at \SI1{GSa/s}, but the simulated events are sampled at
\SI{125}{MSa/s}, which is the sampling frequency planned for the DarkSide20k
digitizers. The randomly generated signal time is not required to be aligned
with either clock. Given the generated temporal position, we round it by excess
\emph{and} defect to the \SI{1}{ns} clock tick. Then we downsample the template
by averaging samples in groups of~8. This is done twice: once with the groups
aligned to the floor-rounded temporal position, once with the ceiling-rounded
one. Finally we interpolate linearly between the two downsampled templates.
\autoref{fig:interptempl} shows a series of waveforms generated following this
procedure. Here averaging before downsampling has the role of an antialias
filter. While more refined antialiasing filters exist, a simple average is
sufficient for our application.

In each simulation event we vary the amplitude of the signal by an additive
Gaussian random variable, which has the standard deviation observed in the LNGS
data, \SI{2.9}\% of the average \SI1{PE} amplitude. This value is obtained by
computing the difference in quadrature between the ``quantile standard
deviations'' (see \autoref{sec:fingerplot}) of the \SI{1}{PE} and \SI{0}{PE}
peaks in the fingerplot done with a \SI{1.5}{\micro s} average, the same used
for the template in \autoref{sec:cctemplate}, and dividing it by the median of
the \SI{1}{PE} peak.

\begin{figure}

    \widecenter{\includempl{figinterptempl}}

    \figcaption{interptempl}{The signal template downsampled from \SI1{GSa/s}
    to \SI{125}{MSa/s} and translated continuously instead of by discrete steps
    with linear interpolation.}

\end{figure}

\subsection{Noise simulation}

To study the dependence of the algorithms on noise, we simulate three different
noise distributions: Gaussian white noise; noise sampled from the LNGS data;
noise sampled from the Proto0 data.

\stracka{Eliminare ``A persistence... in Figure 4.3''}

The white noise is generated in the simulation. The LNGS noise is sampled from
the pre-trigger region of Tile~57 data, the same data used for the signal
template, ignoring any event with any sample less than 700 as in
\autoref{sec:snrdata}. The Proto0 noise is copied from data collected operating
Tile~57 below the breakdown voltage, keeping the whole events without
selection. A persistence plot of the Proto0 data is shown in
\autoref{fig:hist2dtile155759}. The spectra are shown in
\autoref{sec:spectrum}, the autocorrelations in \autoref{fig:autocorrlngs}.

The preprocessing applied on the pre-trigger window of LNGS data does not fully
reject \SI1{PE} pulses, and spurious \SI1{PE} pulses may appear in the
simulation. In the analysis we will use robust statistics, i.e., quantiles, to
deal with outliers caused by this or any other unanticipated feature of the
data.

When the noise waveforms for multiple simulated events are extracted from the
same data event, we skip \SI{1}{\micro s} between each waveform segment in the
data, to avoid correlations between the simulated events.

We downsample the noise in the same way as the signal, by averaging nearby
samples. The noise spectra from both sources (LNGS and Proto0) decrease with
frequency, so an antialiasing with an average should suffice. The Proto0 data,
as is available to us, is pre-downsampled without antialiasing from
\SI{250}{MSa/s} to \SI{125}{MSa/s}.

\emph{After} downsampling, the noise obtained from data is normalized to the
desired variance. The order matters because downsampling with averaging reduces
the variance of the noise, see \autoref{fig:noise}. We normalize the variance
separately for each \emph{data} event, also fixing the mean to zero, such that
in the simulated events the variance has a realistic variation.

\stracka{Eliminare ``we normalize... data event''}

\begin{figure}

    \widecenter{\includempl{fignoise}}

    \figcaption{noise}{The LNGS and Proto0 noise at the original sampling
    frequency (normalized to zero mean and unit variance) and downsampled.}

\end{figure}

\subsection{Event layout}

Each event is the sum of a noise waveform and a shorter signal waveform. Before
the beginning of the signal there is a noise-only region which is chosen long
enough for the filters to be in a stationary state when the signal occurs. The
length of this region is set to be \SI{2304}{ns}, i.e., longer than the largest
filter-length parameter considered in our study, \SI{2048}{ns}.

The simulation is repeated for various raw signal to noise ratios (SNR),
calculated as the peak height relative to the baseline of the original
\SI1{GSa/s} signal template over the noise standard deviation.

The reason we consider the signal amplitude at \SI1{GSa/s} and not at the
actual sampling frequency of the simulation is because downsampling reduces the
peak height. For convenience, we want to keep the definition of signal height
comparable at different sampling frequencies.

\marginpar{I should use the peak height, averaged over continuous positioning,
at the simulation sampling frequency, instead of at \SI{1}{GSa/s}. Then the
comparisons at different sampling frequencies would need to adjust both for the
reduction of the noise variance and of the peak height, but the definition of
SNR would be the standard one.}

Simulations with different raw SNR differ only in the multiplicative constant
of the noise, so we use exactly the same noise and signal arrays for every SNR
to speed up the code. This means that there is no random variation between
results obtained at different SNR (or with different filters), keep this in
mind if the smoothness of some curves would seem to suggest that the Monte
Carlo error is negligible.

\stracka{Eliminare ``keep this... is negligible''}

\autoref{fig:toyevent} shows a complete example event.

\begin{figure}

    \widecenter{\includempl{figtoyevent}}

    \figcaption{toyevent}{A simulation event. The dots are the minima of the
    filters output. The minima are searched in the shaded region only; this
    makes no difference with high enough SNR like in this example, but in the
    limit SNR = 0 the minimum fluctuates around uniformly: the search range
    sets the endpoints of this distribution.}

\end{figure}

\section{Temporal localization}
\label{sec:temploc}

To reconstruct the time position of the signals, We run the three filters
described in \autoref{sec:filters} (moving average, exponential moving average,
cross correlation), and take the minimum (the signals are negative) of the
filtered waveform as the location of the signal. We also take the minimum of
the unfiltered waveform as a baseline comparison.

The minimum of the filter output occurs at a shifted position relative to the
signal location, but this is not a problem since the choice of the point of the
signal to be taken as reference is arbitrary, and, for each filter, the shift
is constant from one event to another.

To build the template for the cross correlation filter, we first truncate it as
described in \autoref{sec:cctemplate}, and then downsample it in the same way
we downsample the signal template and the noise.

\begin{figure}

    \widecenter{\includempl{figtoyfilttempl}}

    \figcaption{toyfilttempl}{Some cross correlation filter templates for
    different lengths. It may appear strange that the endpoint on the left has
    a different height than the endpoint on the right for a given template,
    since we choose the truncation to maximize the norm; it happens because we
    downsample \emph{after} truncation.}

\end{figure}

To allow for a localization more precise than the sampling clock bin, we
interpolate the minimum sample and its first neighbors with a parabola. We also
try upsampling the waveform to \SI{1}{GSa/s} (with sample repetition) prior to
filtering to check if it improves performance.

\subsection{Time resolution results}

Assuming Proto0 noise spectrum, we simulate 1000 events and repeat the time
position reconstruction varying the filter, filter length parameter, and raw
SNR. The signal template position is generated uniformly within one clock bin.
\autoref{fig:lochist} shows the histograms of the temporal localization for all
filters for a choice of SNR and filter length.

\begin{figure}

    \widecenter{\includempl{figlochist}}

    \figcaption{lochist}{Histograms of the temporal localization error, i.e.\
    the difference between the filter output minimum and the signal template
    start, translated to have zero median, for a choice of SNR and filters
    length. The error bars mark the \SI{16}\% and the \SI{84}\% quantiles. As
    definition of temporal resolution we take half the distance between those
    quantiles. The sampling step is \SI{8}{ns}.}

\end{figure}

We see that the distribution of reconstructed signal time positions can be
non-Gaussian, so to quantify the resolution we use, instead of the standard
deviation, half the distance between the \SI{16}\% and \SI{84}\% quantiles,
which is equivalent to a standard deviation for a Gaussian, but gives a
meaningful measure for the width of the distribution even when it is highly
skewed or with heavy tails.

\stracka{Non a capo}

\autoref{fig:rescurve} shows the temporal resolution thus defined for each
filter, filter length, and raw SNR. The exponential moving average has a
consistently poor performance compared to the other filters. The cross
correlation filter is the best one, with performance improving with
filter-length, and at a length of 96 samples (\SI{768}{ns}) is already
practically optimal. The moving average can get close to the cross correlation
filter by choosing appropriately the number of samples.

\begin{figure}

    \widecenter{\includempl{figrescurve}}

    \figcaption{rescurve}{Pulse detection temporal resolution for a range of
    raw SNR and filter lengths. The shaded region marks the sampling
    step~\SI{8}{ns}. The right endpoint of the cross correlation filter curves
    is at~\SI{2.6}{ns}.}

\end{figure}

In the experiment The online processing of the PDM output will happen in two
steps: the digitizers must find the signals, then send them to the front end
processors (FEPs) for further analysis. The computational resources of the
digitizers are limited compared to those available in the FEPs.

The exponential moving average can be implemented on the digitizers with few
logic resources. The cross correlation with 64 samples could probably be
performed with the resources available on the digitizers since a computation
with similar complexity was implemented in firmware and run on an evaluation
card featuring the same FPGA installed on the DarkSide digitizer boards.

The FEPs can and should probably use the best filter, so they would run a long
cross correlation filter, since achieving a good temporal resolution may be
beneficial for offline analysis.

To summarize, out of all the temporal resolution curves the most relevant are:
%
\begin{itemize}
    %
    \item the best time resolution we can achieve with the exponential moving
    average and moving average;
    %
    \item the long cross correlation filters;
    %
    \item the 64 samples cross correlation filter.
    %
\end{itemize}
%
We plot these curves together in \autoref{fig:rescomp}, adding the resolution
plots obtained in the best configuration when changing the noise spectrum in
the simulation, to show the impact of the noise spectrum on the performances.
We note in particular that a different noise spectrum makes a large difference
at low SNR. Finally, we plot a curve computed with and without upsampling,
which shows that upsampling does not improve significantly the performance.

\begin{figure}

    \widecenter{\includempl{figrescomp}}

    \figcaption{rescomp}{Pulse detection temporal resolution vs.\ SNR for
    various filters. The shaded region marks the sampling step~\SI{8}{ns}. The
    hatched band is the interval of SNR observed in Proto0; the vertical line
    is the SNR in the LNGS data, after downsampling to~\SI{125}{MSa/s}. Where
    not specified, the noise is from Proto0.}

\end{figure}

\section{Data reduction}

In this section we study the effect of some data reduction strategies at the
digitizer level on the time resolution that can be achieved in subsequent data
processing stages. We said that the digitizers must find signals in the
waveform stream and send them to the FEPs for further processing. Depending on
the background rate, the bandwidth of the connection between the digitizers and
the FEPs can be a bottleneck. Two possible ways of reducing the amount of
transmitted data are keeping only the minimum number of samples around each
signal, and reducing the sampling frequency. Both have an effect on the
temporal resolution, which we assess in the next paragraphs.

\subsection{Waveform truncation}

We repeat the simulation, as in \autoref{sec:temploc}, but this time we use
only a fixed smaller number of samples in each event to compute the filter
output. We call this selection of samples a ``window''. On the window we run
only a long cross correlation filter since that is what would be done on the
FEPs. As past and future boundary condition we use zero. We evaluate the filter
even after the sample window end because the window can be shorter than the
filter.

In this study we did not attempt any optimization of the left/right balance of
the window. The number of samples to be stored \emph{before} the onset of the
signal is driven by the requirement to allow a proper baseline subtraction
procedure. However, by applying the zero padding just described, we are
implicitly assuming that a proper baseline subtraction procedure has been
applied prior to running the filter. Thus, we only focus on determining the
number of samples that should be saved \emph{after} the onset of the signal, by
considering windows that very skewed to the right. Keep into account that the
measure we are looking at, the temporal resolution, does not depend critically
on getting the baseline right.

While the length of the window is fixed, its placement is not fixed relative to
the true signal location. Instead we use the temporal localization with another
filter feasible on the digitizers, calibrated to have the median aligned to the
beginning of the signal template. The window then extends for a given number of
samples to the left and to the right of this localization.

\stracka{Non andare a capo}

\autoref{fig:windowevent} shows this procedure graphically for a single
event. \autoref{fig:windowtempres} shows the temporal resolution versus
unfiltered SNR curves for various choices of window length, noise, and filter
used to align the window, where for reasons of computation time the latter was
computed at a fixed SNR that does not follow the value on the x-axis.

\stracka{Eliminare ``where for reasons... x-axis''}

\begin{figure}

    \widecenter{\includempl{figwindowevent}}

    \figcaption{windowevent}{Left panel: a simulation event filtered with the
    exponential moving average. Right panel: the same event filtered with a
    long cross correlation filter, both using the whole waveform and using only
    the samples in the shaded window, which is centered using the localization
    from the filter in the left panel.}

\end{figure}

\begin{figure}
    \widecenter{\includempl{figwindowtempres}}

    \figcaption{windowtempres}{Pulse detection temporal resolution with a long
    cross correlation filter applied only on a short window of samples centered
    using a shorter cross correlation filter (left panels) or an exponential
    moving average (right panels). The various curves correspond to different
    window lengths, while the black dots are the resolution without windowing.}

\end{figure}

From \autoref{fig:windowtempres} we conclude that it would be necessary to save
at least \SI1{\micro s} of waveform after the onset of the signal to avoid
degrading the temporal resolution. On the other hand, we also observe that the
performances improve quickly to almost optimal ones when increasing the window
length. In the top right panel, i.e., with Proto0 noise and centering with an
exponential moving average, the resolution does not converge to the value
without windowing as the window length increases. This is due to the standard
deviation of the distribution of the window center, \SI{17}{Sa}, being not
small enough compared to the left window margin, \SI{32}{Sa}. This means that
in a non-negligible fraction of cases, the window does not include the leading
edge of the signal. We show this problem intentionally to underline the
importance of the left/right balance.

\subsection{Downsampling}

Another way of reducing the data throughput is downsampling. In
\autoref{fig:tempresdowns} we show the temporal resolution achieved with a long
cross correlation filter at different sampling frequencies. The downsampling is
computed averaging nearby samples. So, in other words, we are comparing
applying the cross correlation at the full sampling frequency to first applying
an antialiasing filter, downsampling and then computing the cross correlation
with a downsampled template. We observe that downsampling by a factor of 2 from
\SI{125}{MSa/s} to \SI{62}{MSa/s} maintains almost the same temporal
resolution, while going to \SI{31}{MSa/s} lowers it visibly.

When downsampling a waveform, the variance of the noise is reduced. At each
sampling frequency the simulation sets the SNR looking at the standard
deviation of the already downsampled noise, so the SNR scales are off by the
factor of the noise amplitude reduction. To make the simulations comparable, we
should start from a common ``master simulation'' at \SI{1}{GSa/s}, then
downsample it various times. Our code does not implement this and repeats the
simulation from scratch at each sampling frequency, renormalizing the
downsampled noise to unitary variance. To account for this, in
\autoref{fig:tempresdowns} we apply a correction factor on the raw SNR before
plotting the time resolution results.

\begin{figure}

    \widecenter{\includempl{figtempresdowns}}

    \figcaption{tempresdowns}{Pulse detection temporal resolution at different
    sampling frequencies with a cross correlation filter with template length
    \SI{2048}{ns}. The SNR scale is at \SI{125}{MSa/s}; curves for different
    sampling frequencies are rescaled horizontally by the factor written in the
    legend to account for the noise variance reduction with downsampling, as
    described in the text.}

\end{figure}

We also check if downsampling is associated to signal to noise ratio
degradation in the cross correlation filter output. In
\autoref{tab:filtsnrdowns} we report the ratio between SNR after and before
filtering. It does not appear to change significantly.

\marginpar{Add digitalization to the simulation and make a plot like the one
for the sampling frequency but varying the number of bits at \SI{125}{MSa/s}.
Don't use the number of bits, use the ratio signal peak over digit which is
well defined.}

\begin{table}
    \centering

    \begin{tabular}{c*4S[table-format=1.1]}
        \toprule
               & \multicolumn4c{SNR after over before filtering} \\
        \cmidrule(l){2-5}
         Noise & {\SI{1}{GSa/s}} & {\SI{125}{MSa/s}} & {\SI{62.5}{MSa/s}} & {\SI{31.2}{MSa/s}} \\
        \midrule
        Proto0 &               &             3.3 &              3.3 &              3.3 \\
          LNGS &           5.6 &             5.5 &              5.7 &              6.0 \\
         White &           4.3 &             4.3 &              4.2 &              4.2 \\
        \bottomrule
    \end{tabular}

    \caption{\label{tab:filtsnrdowns} Ratio of SNR after over before filtering
    with a cross correlation filter with template length \SI{2048}{ns}. The
    \SI{125}{MSa/s} column contains the actual SNR ratios of the simulations,
    while the values for the other sampling frequencies are divided by the
    noise standard deviation reduction with downsampling relative to
    \SI{125}{MSa/s} to make them comparable.}

\end{table}