%\subsection{Search strategy for the resonance analysis: the BumpHunter algorithm}
%\label{sec:searchstrategy}

%The main statistical test employed in the dijet resonance search is based
%on the \BumpHunter\ algorithm~\cite{Aaltonen:2008vt,Choudalakis:2011bh} and
%is used to establish the presence or absence of a resonance in the dijet
%mass spectrum, as described in greater detail in previous
%publications~\cite{EXOT-2010-07,EXOT-2011-07}.
%The algorithm operates on the binned \mjj data, comparing the background estimate with the data in mass intervals of
%varying widths formed by combining neighboring bins. Starting with a two-bin signal window,
%the algorithm scans across the entire distribution,
%then steps through successively larger signal windows up to half of the whole fit range.
%For each point in the scan, it computes the significance of the difference between the data and the background.
%The most significant departure from the smooth spectrum
%(``bump'') is defined by the set of bins that have the smallest probability
%of arising from a Poisson background fluctuation.
%During this procedure, the background model is not changed or refit to the data
%outside of the excluded region.

%The \BumpHunter\ algorithm accounts for the so-called ``look-elsewhere effect''
%~\cite{lyons2008, Gross2010}, by performing a series of pseudo-experiments
%drawn from the background estimate to determine the probability that random
%fluctuations in the background-only hypothesis would create an excess
%anywhere in the spectrum at least as significant as the one observed.
%
%To make practical use of this algorithm, one must ensure the background
%estimate is not biased by the signal.
%Therefore, \BumpHunter\ is run in two steps.  In the first, the full
%distribution is fit and passed to \BumpHunter.
%If the most significant local excess from the background fit has a $p$-value
%smaller than 0.01, this region is excluded and a new background fit is
%performed. The exclusion is then progressively widened bin by bin until
%the $p$-value of the remaining fitted region is acceptable.
%During the 2015 analysis it was found that simply excluding one additional bin on the low mass
%side of the signal window removed most of the residual bias (if any
%did exist).  Then the result of this fit is used for the second
%stage where an unbiased estimate of the global significance of any
%excess is obtained.

Once the background is derived, the BumpHunter algorithm ~\cite{Aaltonen:2008vt,Choudalakis:2011bh} is employed to test the consistency or discrepancy between background and the observed data.
It can locate the local excess above the background and quantifies the degree of discrepancy between the observed data and background, based on the frequentist $p$-value from one of three test statistics: $\chi^{2}$, Log Likelihood and BumpHunter(described below).

%\subsection{Frequentist $p$-value}
%
%In the comparison between observed data and background, if the background comes only from the SM, this is called the background-only hypothesis or null hypothesis, denoted by $H_{0}$. If the null hypothesis is correct, the observed data, denoted $D$, will be the statistical fluctuation of the background. The validity of $H_{0}$ can be tested by determining the probability of obtaining the data spectrum as a fluctuation of the background-only hypothesis.
%
%The frequentist probability is commonly used in high energy physics, which is a statement on the frequency of a certain outcome given a large number $N$ of repeated experiments. The frequentist hypothesis test determines the consistency between $H_{0}$ and the observed experimental outcome $x$ by fixing in advance a value of probability $\alpha$ below which the hypothesis will be rejected as too discrepant. Specifically, if the observation falls in a space of possible outcomes $\omega$ such that
%\begin{equation}
%P(x \in \omega|H_{0}) \leq \alpha
%\end{equation}
%then the null hypothesis $H_{0}$ will be rejected, otherwise the data is consider to be consistent with background.

%Define a test statistic $t$ to be any numerical quantity which describes the compatibility
%between data and background, it usually increases monotonically with decreasing compatibility. The $p$-value of this test statistic is the probability of obtaining a value at least as extreme as the observed $t = t_{0}$ given $H_{0}$:
%\begin{equation}
%p = P(t>t_{0}|H_{0})
%\end{equation}
%where small $p$-value means small consistency between data and background.

\subsection{Test statistic}
Three test statistics are employed in the BumpHunter algorithm to quantify the discrepancy between observed data and background: $\chi^{2}$, Log Likelihood and the BumpHunter. They are represented by a single value which characterises the degree of agreement between the observed data and background and are used in defining $p$-values.

%The $\chi^{2}$ test statistic is defined as the sum in quadrature of the differences
%between observations and expectations, normalised to the variance. For a comparison
%where the observation and prediction are both binned histograms with contents $d_{i}$ and $b_{i}$ in bin $i$:
%\begin{equation}
%\chi^{2} = \sum_{i}\frac{(d_{i}-b_{i})^{2}}{b_{i}}\,.
%\end{equation}
%The "reduced $\chi^{2}$" value is defined as $\chi^{2}/\mathrm{NDF}$, which is often used to test goodness-of-fit, where NDF is the number of degrees of freedom in the fit.

%The Likelihood, $\mathcal{L}$, test statistic is an effective one in comparison of two binned histograms. In comparison of a mass spectrum, each bin content follows the Poisson distribution, so the Log Likelihood is defined as the product of the Poisson probability in each bin
%over all bins:
%\begin{eqnarray}
%\mathcal{L} = \prod_{i}\frac{b_{i}^{d_{i}}e^{-b_{i}}}{d_{i}!}.
%\end{eqnarray}
%The  Negative Log Likelihood (NLL), $-2\ln\mathcal{L}$ is defined as:
%\begin{equation}
%-2\ln\mathcal{L} = -2\ln\prod_{i}\frac{b_{i}^{d_{i}}e^{-b_{i}}}{d_{i}!}.
%\end{equation}

Both $\chi^{2}$ and log likelihood can quantify the discrepancy between the observed data and background in individual bin.
However, in the comparison of two binned spectra, the discrepancy in the window of neighbouring bins is more meaningful.
Several adjacent bins with large excess in each bin indicate new physics, however three bins with a large excess, a large deficit and a large excess may produce the large $\chi^{2}$ and NNL but would be of much less physical interest.

The third test statistic has therefore been defined to quantify the ``bump'' above the background, the ``BumpHunter statistic'', which is the default test statistic in the BumpHunter algorithm.
For a set of adjacent bins, a value $t$ is calculated as the Poisson probability of obtaining a result at least as significant as the one observed, define $d$ as the sum of the data and $b$ as the background in these neighbouring bins:
\begin{equation}
t =
\begin{cases}\displaystyle
\sum_{n=0}^{d}\frac{b^{n}}{n!}e^{-b} \quad \mathrm{for} \quad d<b\,, \\\displaystyle
\sum_{n=d}^{\infty}\frac{b^{n}}{n!}e^{-b} \quad \mathrm{for} \quad d \geq b\,.
\end{cases}
\end{equation}
The above expression can be represented in terms of gamma functions:
\begin{equation}
 t =
\begin{cases}\displaystyle
\Gamma(d+1, b) = 1 - \Gamma(d+1, b) \quad \mathrm{for} \quad d<b\,, \\
\Gamma(d, b) \quad \mathrm{for} \quad d \geq b\,.
\end{cases}
\end{equation}
This value accounts for the direction of neighbouring fluctuations by looking at the
overall excess or deficit in the region.

For every possible window along the mass spectrum, $t$ is calculated. The possible windows are found by looping over all widths between a minimum and a maximum number of bins. The BumpHunter statistic describing the overall spectrum is defined as the negative log of the smallest probability obtained for any window, defined as:
\begin{equation}
t_{0} = -\log t_\mathrm{min}\,.
\end{equation}

Once the test statistic is determined, the $p$-value can be calculated through generating many pseudo-data which can be derived by a randomly draw in each bin from a Poisson distribution with parameter equal to the expected bin content from the background spectrum.
The selected test statistic is then calculated for each pseudo-data.
The fraction of these cases for which the test statistic is more than that in observed data can be easily computed, $p$-value.
The full procedure to calculate the $p$-value obtained from a selected test statistic can be summarised as:
\begin{itemize}
        \item Calculate the value of the test statistic $t_{0}$ which compares background-only hypothesis to observed data, $t_{0}$,
        \item Generate a collection of pseudo-data from background to represent a range of possible experimental outcomes in the case the background-only hypothesis is correct,
        \item Compare each pseudo-data with background to calculate the test statistic value for each pseudo data, $t_{i}$,
        \item Calculate the fraction of $t_{i}$ for which $T > t_{0}$, this fraction is the $p$-value obtained from the test statistic $t$.
\end{itemize}

\subsection{The BumpHunter algorithm}
The BumpHunter algorithm compares the background with the observed data in intervals of varying widths formed by combining neighboring bins.
It scans across the entire distribution with the window width varying from 2 up to half of the number of the bins.
For each window in the scan, it computes the significance of the difference between the observed data and the background.
The most significant departure from the background spectrum is defined by the set of bins that has the smallest probability of arising from a Poisson background fluctuation.
If the measured $p$-value obtained from the BumpHunter statistic is less than 0.01, it may mean the existence of new physics.

As the pseudo-experiments are drawn from the background, the random fluctuations in the background-only hypothesis would create an excess anywhere in the spectrum at least as significant as the one observed, so the BumpHunter algorithm also accounts for the look-elsewhere effect~\cite{lyons2008, Gross2010}.

The $p$-value got in BumpHunter is used to quantify the discrepancy between observed data and background.
The residual in each bin can also been quantified by a $p$-value and described in detail in \cite{Choudalakis:2011bh}.
In each bin, a measured $p$-value is defined as the probability of measuring a discrepancy between data and background at least as large as the one observed.
The $p$-value is then translated into a $z$-value defined as the number of standard deviations to the right of the mean of the normal distribution:
\begin{equation}
p\mathrm{-value} = \int_{z\mathrm{-value}}^{\infty}\frac{}{\sqrt{2\pi}}e^{-\frac{x^{2}}{2}}dx\,.
\end{equation}
Bins with a $z$-value less than zero show no difference of any interest, while those with a $z$-value of more than two or three indicate a significant discrepancy.
For clarity of interpretation, one would like the sign of the $z$-value drawn in residual plots to depend on whether the data falls above or below the hypothesis.
Therefore, in the plots, any bins with a negative $z$-value are set to zero, while those with a positive $z$-value are drawn positive or negative depending on whether an excess or a deficit is observed.


\documentclass[14pt,letterpaper]{report}
\usepackage[utf8]{inputenc}
\usepackage[left=1.00in, right=1.00in, top=0.50in, bottom=1.00in]{geometry}

\usepackage{enumitem}

\title{BucketVision “Angry Eyes” NetworkTables \protect\\ Interface Control Document v1.0}
\makeatletter

\begin{document}
	\begin{center}
		{\LARGE \@title}

		{\textit \@date}
	\end{center}

	\section*{Overview}

	The “Angry Eyes” computer vision pipeline detects and determines the position of the dual-slanted retroreflective targets used in the 2019 FRC game. As this software runs on a separate system from the main robot controls, it publishes the position of detected targets through the NetworkTables interface so other systems may act on that data. This document described the format of that data.

	\section*{Configurable Items}

	The top-level table under which all items are published (1 level below root) is passed in as an argument to all of the new pipeline components. In the current test code, this is named BucketVision but a table with any name may be passed in here.

	\noindent Each component of the new pipeline which publishes data in the NetworkTable accepts a parameter for the name, with which it will get or create a table by that name under the top-level table above. In the current test code, this is FrontCamera. While there is no logic to detect differing names, care should be taken to use the same camera name for all relevant components, otherwise data will be published under different tables.

	\section*{Published Data}

	as noted above, these items are all published in a camera table which is in a top-level table

	\begin{itemize}[label={--}]
		\item \textbf{NumTargets} - Number

		An integer number representing the number of targets detected. All the following arrays will be this length, and each index will represent a single target.

		\item \textbf{distance} - Number Array

		The estimated distance (in meters) between the camera and the target. Do not rely on its accuracy, it is calculated crudely.

		$$ \textrm{Average Height}_{px} = \frac{\textrm{Height}_L + \textrm{Height}_R}{2} $$

		$$ \textrm{Height}_{deg} = \frac{\textrm{Average Height}_{px}}{\textrm{Camera px per degree}} $$

		$$ dist = \frac{\textrm{Rectangle Height}}{\textrm{tan}^{-1}(\textrm{Height}_{deg})} $$

		\item \textbf{pos\_x} - Number Array

		The horizontal location of the target in normalized image-space coordinates, from 0 (left) to 1 (right).

		\item \textbf{pos\_y} - Number Array

		The vertical location of the target in normalized image-space coordinates, from 0 (top) to 1 (bottom).

		\item \textbf{size} - Number Array

		The distance between the left and right portions of the target as a fraction of the width of the image.

		\item \textbf{parallax} - Number Array

		Scaled left right difference as a fraction of total height: (negative if camera is right of target)

		$$ \textbf{parallax} = (1000) \frac{Height\_left-Height\_right}{Height\_left+Height\_right} $$

		\item \textbf{angle} - Number Array

		The angle (in radians) between the left and right portions of the target.

	\end{itemize}

\end{document}


    \filetitle{dat2ttrend}{Construct linear time trend from date range}{dates/dat2ttrend}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
[TTrend,BaseDate] = dat2ttrend(Range)
[TTrend,BaseDate] = dat2ttrend(Range,BaseYear)
[TTrend,BaseDate] = dat2ttrend(Range,Obj)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{Range} {[} numeric {]} - Date range from which an integer
  linear time trend will be constructed.
\item
  \texttt{BaseYear} {[} model \textbar{} VAR {]} - Base year that will
  be used to construct the time trend.
\item
  \texttt{Obj} {[} model \textbar{} VAR {]} - Model or VAR object whose
  base year will be used to construct the time trend; if both
  \texttt{BaseYear} and \texttt{Obj} are omitted, the base year from
  \texttt{irisget('baseYear')} will be used.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\item
  \texttt{TTrend} {[} numeric {]} - Integer linear time trend, unique to
  the input date range \texttt{Range} and the base year.
\item
  \texttt{BaseDate} {[} numeric {]} - Base date used to normalize the
  input date range; see Description.
\end{itemize}

\paragraph{Description}\label{description}

For regular date frequencies, the time trend is constructed the
following way. First, a base date is created first period in the base
year of a given frequency. For instance, for a quarterly input range,
\texttt{BaseDate = qq(baseYear,1)}, for a monthly input range,
\texttt{BaseDate == mm(baseYear,1)}, etc. Then, the output trend is an
integer vector normalized to the base date,

\begin{verbatim}
TTrend = floor(Range - BaseDate);
\end{verbatim}

For indeterminate date frequencies, \texttt{BaseDate = 0}, and the
output time trend is simply the input date range.

\paragraph{Example}\label{example}


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{comment}
\usepackage{listings}
\usepackage{mathtools}
\usepackage{hyperref}
\usepackage{amsthm}

\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]

\newtheorem{theorem}{Theorem}[section]

\DeclarePairedDelimiter \abs{\lvert}{\rvert} % short cut for absolute value

\setlength{\parindent}{0em}
\setlength{\parskip}{0.5em}

\title{Chapter 2: Algorithm Analysis}
\author{Yangtao Ge}
\date{\today}

\begin{document}
\maketitle
\begin{abstract}
This section discusses about:
\begin{itemize}
    \item how to estimate the time required for a program
    \item how to reduce running time of a program
    \item The result of careless use of recursion
    \item very efficient algorithms to raise a number to a power
    \item compute GCD
\end{itemize}
\end{abstract}
\section{Mathematical Background}
Four definitions of the framework:
\begin{definition}[Upper bound]
    $T(N) = O(f(N))$ if there are positive \emph{constants} $c$ and $n_0$ such that $T(N) \leq cf(N)$
    when $N \geq n_0$
\end{definition}

\begin{definition}[Lower bound]
    $T(N) = \Omega(g(N))$ if there are positive \emph{constants} $c$ and $n_0$ such that $T(N) \geq cg(N)$
    when $N \geq n_0$
\end{definition}

\begin{definition}[Envelope]
    $T(N) = \Theta(h(N))$ iff $T(N) - O(h(N))$ and $T(N) = \Omega(h(N))$
\end{definition}

\begin{definition}
    $T(N) = o(p(N))$ if for \textbf{\textit{all}} positive constants $c$ there exists an $n_0$ such that
    $T(N) < cp(N)$ when $N > n_0$.
\end{definition}
\emph{Ref: p.30 for detail theorem}

Typical growth rates:\newline
\begin{tabular}{|p{3cm}|p{5cm}|}
    \hline
    Function & Name\\
    \hline
    \hline
    $c$ & Constant \\
    $\log_{}N$ & Logarithmic \\
    $\log^2_{}N$ & Log-Squared \\
    $N$ & Linear \\
    $N\log_{}N$ & \ \\
    $N^2$ & Quadratic \\
    $N^3$ & Cubic \\
    $2^N$ & Exponential \\
    \hline
\end{tabular}

Some Theorem from the definition:
\begin{theorem}
    If $T_1(N) = O(f(N))$ and $T_2(N) = O(g(N))$, then:
    \begin{enumerate}
        \item $T_1(N) + T_2(N) = O(f(N) + g(N))$
        \item $T_1(N) * T_2(N) = O(f(N) * g(N))$
    \end{enumerate}
\end{theorem}

\begin{theorem}
    If $T(N$) is a polynomial of degree $k$, then $T(N) = \Theta(N^k)$
\end{theorem}

\begin{theorem}
    $\log^k_{}N = O(N)$ for any constant $k$, which tell us `Logarithms grow very slowly
\end{theorem}

For big-O notation answers: \underline{Lower-order terms can generally be ignored}
e.g. $f(N) = 2N^2 + N$ then its big-O notation is $T(N) = O(N^2)$

For \emph{relative growth rates} of two Function, we using `\emph{L'Hopitals's rule}' to determine it:
\begin{theorem}[L'Hopitals's rule]
    If $\lim_{N\to\infty}f(N) = \infty$ and $\lim_{N\to\infty}g(N) = \infty$,
    then $\lim_{N\to\infty} \frac{f(N)}{g(N)} = \lim_{N\to\infty} \frac{f'(N)}{g'(N)}$
\end{theorem}

Four possible results:
\begin{itemize}
    \item Limit is 0: $f(N) = 0(g(N))$
    \item Limit is $c\neq 0$: $f(N) = \Theta(g(N))$
    \item Limit is $\infty$: $g(N) = o(f(N))$
    \item Limit does not exist: No relations
\end{itemize}

\section{Model}
Basically a normal computer which execute instructions sequentially. And it has inifinite memory

\section{What to Analyze}
What we focus:
\begin{itemize}
    \item Input size
    \item Running time
\end{itemize}

What we use:
\begin{itemize}
    \item $T_{avg}(N)$ : average running time -- reflect typical behaviour.
    \item $T_{worst}(N)$: worst running time -- a guarantee for performance on any possible input
\end{itemize}
\emph{Ref: pp. 33 - 35} (Maximum subsequence Sum Problem)

\section{Running Time Calculations}
We only focus on \emph{big-O notation} answers.

\subsection{A simple Example}
Calculations of $\sum_{i=1}^N i^3$. The code is as follows:
\begin{lstlisting}[language=Java]
    public static int sum(int n){
        int partialSum = 0;           // Line 1
        for(int i = 1; i <= n; i++){  // Line 2
            partialSum += i * i * i;  // Line 3
        }
        return partialSum;            // Line 4
    }
\end{lstlisting}

\underline{Analysis:}
\begin{itemize}
    \item Line 1: 1 time for assignment
    \item Line 2: 2N + 2 times in total:
    \begin{itemize}
        \item 1 time for initialization
        \item N + 1 times for comparison
        \item N times for increment
    \end{itemize}
    \item Line 3: 4N times in total(2 multiplication, 1 addition, 1 assignment)
    \item Line 4: 1 time for return
\end{itemize}
In total $6N + 4$, so we say that it is a $O(N)$ method

\subsection{General Rules}
\begin{enumerate}
    \item `For' loops: $T = T_{statements} * iterations$ (at most)
    \item `Nested' loops: $T = T_{statements} * \prod_{}SizeOfLoop$ (Analysis them inside-out)
    \item Consecutive statements: Use `big-O' notation add \newline
    e.g. $O(N)$ followed by $O(N^2)$ is still $O(N^2)$
    \item `if-else': $T = \max(T_{Stat1}, T_{Stat2})$ (sometimes overestimate, but never underestimate)
\end{enumerate}

\subsection{Solution for the Problem of Maximum subsequence Sum}
Four algorithms are provided here (\emph{Ref: pp.39 - 49})

\subsection{Logarithms in the Running Time}
Some Logarithms algorithms:
\begin{itemize}
    \item \emph{divide-and-conquer} algorithms: $O(N\log_{}N)$ time
    \item General rules:
    \begin{itemize}
        \item $O(\log_{}N)$: takes constant $O(1)$ time to cut the problem size by a fraction (usually $1/2$)
        \item $O(N)$: constant time requred to merely reduce the problem by a constant amount
    \end{itemize}
\end{itemize}

Following subsection are some common Logarithmic algorithms:

\subsubsection{Binary Search}

\subsubsection{Euclid's Algorithm}

\subsubsection{Exponentiation}

\subsection{A Grain of Salt}
Sometimes, Worst case is better than average case:
\begin{itemize}
    \item average case is very complex
    \item Analysis needs to be tightened
    \item average running time is less significant than worst case running time
\end{itemize}

\end{document}


\section{Introduction}
\label{sec:introduction}

The trading pit of a stock exchange is often imagined by outsiders as a frenzy place, with telephones constantly ringing and traders shouting orders across the room at a frenetic rhythm. This was probably the reality thirty years ago, when open outcry was still the main communication system between pit traders. Since then the floors have become more and more quiet as the majority of the orders moved to electronic trading systems. Notwithstanding, investment decisions were still made by humans who could now execute their orders without passing through the pit traders. In the last decade, the markets have witnessed the widespread adoption of \emph{Automated Trading Systems} (ATS), that can make investment decisions in a fully automatized way at speeds with orders of magnitude greater than any human equivalent. In 2014, more than $75\%$ of the stock shares traded on United States exchanges were originated from ATS orders and this amount kept growing since then. Quantitative hedge funds, such as Renaissance Technologies, D.E. Shaw, Citadel and many others, are employing mathematicians, physicists and other scientists to develop algorithms able to extract trading signals from large amount of data and automatically trade. These algorithms are typically based on advanced statistics, signal processing, machine learning and other fields of mathematics. However, few of these hedge funds publish their profit-generating ``secret sauce'' and not much can be found in the literature. In this project we develop an automated trading algorithm based on \emph{Reinforcement Learning} (RL), a branch of \emph{Machine Learning} (ML) which has recently been in the spotlight for being at the core of the system who beat the Go world champion in a 5-match series \cite{silver2016mastering}.\\
This document is organized as follows. In Section \ref{sec:basics_reinforcement_learning} we introduce the basic concepts of RL and present two learning algorithms that allow two determine an approximation for the optimal policy of a sequential decision problem. In section \ref{sec:application_to_systematic_trading} we discuss the asset allocation problem from a mathematical point of view and show how these learning algorithms can be applied in this setting. In Section \ref{sec:python_prototype} we start discussing the implementation of the model in Python, which has been used during the prototyping phase. In Section \ref{sec:c++_implementation} we discuss a more efficient C++ implementation. In Section \ref{sec:execution_pipeline} we describe the execution pipeline used to run the learning experiment. In Section \ref{sec:numerical_results} we present the numerical results for a synthetic asset, whose price follows a particular stochastic process. In Section \ref{sec:conclusion} we conclude with some final remarks and we discuss some future research directions.


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{enumerate}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{March 26, 2014}
\maketitle
\section*{lesson 17}
this stuff is like the prototype solution setting up method of characteristic lines, we'll see more of this later.

Today we look at Alembert's solution
\begin{align*}
  &\text{PDE}&&\qquad u_{tt}=c^2u_{xx}&c&>0&\infty<&x<+\infty&0<&t<\infty\\
  &\text{IC}&&\left.
  \begin{aligned}
    u(x,0)&=f(x)\\
    u_t(x,0)&=g(x)
  \end{aligned}
  \right\}&&&\infty<&x<+\infty
\end{align*}
change independent variables
\begin{align*}
  \xi&=x+ct\\
  \eta&=x-ct
\end{align*}
point will get $u_{\xi\eta}$

See page 130(146) for more explanation
\end{document}


\chapter{Discussion}
\label{ch:7}
In this chapter, we seek to contextualize the results from the preceding chapter.
We apply theory from \Cref{ch:2} and compare the result to the expected behavior
% expect from what we know about their theoretical construction/base.

\section{Similarity Scores}

We have established that the measures define what makes trajectories similar.
Some of the measures have a similar base idea, and thus we expect to see this reflected in the results.
From the rankings of most similar pairs, there are a few things that stand out and we will go over them here.

%We reiterate this is not the case for all of them, and consequently, not all columns of TABLE can be used in this manner.

First of all, EDR with $\epsilon$ set to 0.203 was the column in \Cref{tab:top-10-sim-pairs} which had the least number of frequent “most similar pairs”.
When adjusting the parameter down to 0.101, our approximation of the recommend parameter value, all of its eight most pairs were found elsewhere in the table.
Recall that EDR’s parameter is the threshold distance that determines if points are “matching”.
The similarity ranking with the lower parameter value aligns more with the other measures, and this could indicate that the other threshold was set too high.
We would presume that EDR would give a similar ranking as the other noise-tolerant edit-distance inspired method, MSM.
From what can be seen in the table, this appears to hold, but it varies with its parameter value too.

%   that would have similar values to the

Next, we note that the Hausdorff distance generated similar pairs which were quite different from the other measures.
As the Hausdorff metric is parameter-free, we reckon that the reason its results stood out was that it defined trajectory similarity in a different manner than the rest.
We established that the Hausdorff distance does not take into account the direction of a trajectory and the only other measure that is invariant to direction is SSPD.
Indeed they share many of the top 10 ranked scores, however, SSPD distinguishes itself Hd as it reduces the similarity score to a single element-element pair making altering their internal rankings again.

% Next we observe that the Hausdorff distance contains the second least “frequent pairs” and as it parameter free, we reason that there must be something with how it defines trajectory similarity that makes it stand out. Recall that the Hausdorff distance stood out as the only measure that did not take into account the direction of a trajectory. Furthermore, it stands out as the only measure that reduces the similarity distance to an element-element pair.

The final observation from \Cref{tab:top-10-sim-pairs} we noted was that Euclidean distance, ERP, and one of the MSM measures gave the exact same top 10 pairs.
Upon closer examination, it turned out that their ranking remained the same until the 44th pair, and the similitude between the measures continued beyond that.
MSM with cost value 1 and the Euclidean distance's ranking match for 9456 of the 10 thousand most similar trajectory pairs.
We reason that this occurred as a consequence of the cost of Splits and Merges being set too high, making MSM  default to the Move operation whose cost is the $L_2$ norm between trajectory elements.
As for ERP, we reason that setting the reference point, $g$, to the origin, increased the cost for trajectory elements that were far away.
Again, making the method depend on the $L_2$ norm between trajectory elements as the cost of an edit.

\section{Davies-Bouldin Results}

The essence of this thesis is to examine how the definition of similarity varies with different measures.
Clustering is a manner of grouping the trajectories such that traits which are seen as the most defining characteristics under various definitions get highlighted.

As stated in \Cref{ch:2}, there are a number of cluster evaluation techniques that aim to numerically rank the quality of clustering results.
One of these is the DB-index however we will not an assumption regarding which type of type similarity was the most “correct” one.
The ranking created by this criterion is not invalid, but the insight it adds to is limited.
Nevertheless, we have chosen to leave in this stage of the experiment and in the report on the account of the efforts that were put into computing it as well as its natural role as a starting point for analyzing the clusters.

Our version of Davis-Bouldin favors clusterings where each trajectory element differs as little as possible from the mean of all trajectory elements when computed in a. A consequence of this is that the cluster can appear fuzzy since the placement of the surrounding points does not affect the index.
From \Cref{tab:cluster-dbidx} it appears as if the lock-step method and EDR with threshold parameter $\epsilon=0.203$ were the best performing methods, and this is as expected.

Moreover, methods that do not take into account order observations, or methods that would reorder the trajectory elements received low DB-index rankings. In \Cref{fig:cluster-best-worst-db-ap,fig:cluster-best-worst-db-h}  the clusters that were generated by the measures which received highest and lowest DB-indexes are drawn. These figures illustrate the bias of our DB-index implementation.

\begin{figure}[h!]
  \centering
  \hspace{1em}
   \begin{subfigure}[c]{0.37\linewidth}
     \includegraphics[width=\linewidth]{figs/clusters/CLU_AP_ALL[EDR;e=.203].png}
    \caption{EDR;$\epsilon=0.203$, highest rank}
  \end{subfigure}
  \hfill
  \begin{subfigure}[c]{0.37\linewidth}
    \includegraphics[width=\linewidth]{figs/clusters/CLU_AP_ALL[SSPD].png}
    \caption{SSPD, lowest rank}
  \end{subfigure}
  \caption{The AP-clusters of the measures the best and wort Davies-Bouldin indexes.}
  \label{fig:cluster-best-worst-db-ap}
\end{figure}

\begin{figure}[h!]
  \centering
  \hspace{1em}
   \begin{subfigure}[c]{0.4\linewidth}
     \includegraphics[width=\linewidth]{figs/clusters/CLU_H_ALL[ERP;g=0,0].png}
    \caption{ERP, $\epsilon=0.203$, highest rank}
  \end{subfigure}
  \hfill
  \begin{subfigure}[c]{0.4\linewidth}
    \includegraphics[width=\linewidth]{figs/clusters/CLU_H_ALL[SSPD].png}
    \caption{SSPD, lowest rank}
  \end{subfigure}
  \caption{The HCA-clusters of the measures the best and wort Davies-Bouldin indexes.}
  \label{fig:cluster-best-worst-db-h}
\end{figure}

As a consequence of the unreliability of the DB-index, the main tool for examining the clusters will be visual inspection.
This will allow us to account for the different aspects of trajectory similarity and formulate conclusions accordingly.
Re-examining \Cref{fig:cluster-best-worst-db-ap,fig:cluster-best-worst-db-h} with visual inspection it would appear that SSPD produced cleaner and fewer clusters than both EDR and ERP.
This is in stark contrast to the ranking created by the Davies-Bouldin criteria under both HCA and AP analysis.

This evaluation alternative comes with its own drawbacks, one of which is the matter of subjectivity.
At the same time, it has been stated that human assessment is an important component for determining the quality of clusters\cite{79-SilhouetteAnalysis}.
We will not be declaring any measure better than the others as our results do not provide support for an accurate ranking.

% We cannot declare one measure just better than all others from the evaluation that we executed here as it makes no sense. This is in large part due to the inherent bias of using a strict lockstep method to evaluate the final clusters.
% By visual inspection, SSPD produced cleaner and fewer clusters than EDR, $\epsilon=0.203$ under AP, despite having a much worse criterion. This pattern repeats itself under HCA when predetermined even with  the amount of final clusters. EDR  as well \Cref{fig:cluster-best-worst-h-db}.

% Based on those arguments, the main tool for examining the clusters will be visual inspection. This will allow us to account for the different definitions/aspects of similarity and formulate conclusions in an aggregate manner.
% This method comes with its own drawbacks, one of which is the matter of subjectivity. Yet as brought up in supported previous works[CITATIONS], human assessment key for determining the quality of clusters.

\section{Cluster Behavior}

First, we describe what we expected the clusters to look like based on the properties of the specific similarly distance function.
For the methods that operate with a parameter value, we discuss how changing that value would affect the cluster results.
Then we compare our expectations to the clusters themselves.
Affinity propagation is a was the parameter-free method, thus its results are weighted more.
On the other hand, the number of hierarchical clusters was arbitrarily set, leading us to use its results as a supplementary evaluation.

% https://www.wordhippo.com/what-is/another-word-for/foreseeable.html
\subsection{Expectations}
In general, there are three factors that will affect how we expect the final clusters to appear.
Measures that are context-aware should calculate fewer fuzzy clusters.
Next, measures that account for local time shifts should group trajectories that have similar sub-sections but with an offset creating a wider band of similarity.
Lastly, we expect the noise-sensitive trajectories to create fuzzier trajectories.

\textbf{Euclidean Distance} is the only lock-step measure, and trajectories that are in the same region might receive an artificially high similarly score by nature of being near each other.
A trajectory pair will be rewarded if they have elements at the same index which are near each.
This would create fuzzy clusters as each point evaluated isolation and taking the average of the pairwise element-distances will artificially smooth out the distances.
An extra amount of fuzziness is expected to come from the fact that this measure is sensitive to noise.

% WE COULD observe that there are trajectories that oscillate around the same region at a similar pace- giving them a lock-step similar score, but the cluster ends up looking fuzzy.

\clearpage
\textbf{Dynamic Time Warping}  optimizes for local shape similarity and warps trajectories to reflect similarity after shape-preserving transformations.
We expect the final clusters to appear fuzzy at first glance as the local similarities would be out of sync.
Yet upon closer examination, we should be able to spot bands of cohesive trajectory sections.
DTW is a noise-sensitive method, so we would expect some level of fuzziness to be present, however, the clusters should be crisper than that of the Ed.

From \textbf{Hausdorff Distance}, we would expect crisper clusters than both Ed and DTW if we could guarantee the data free of noise.
The maximizing over a minimum of all element-element pairings means that the overall shape similarity is weighted in a way that the other measures do not account for.
In other words, the global resemblance matters more under this definition.
However, the data set we used is not noise-free so while we expect some crispness, there will be fuzziness of the clusters due to its noise sensitivity.

% All of the trajectories' points have to be near each other for if the pair is to receive a high similarity score.
% stemming from the reduction of similarity distance to distance between two trajectory elements.

One of the key features of \textbf{Symmetrized Segment-Path Distance} is that it accounts for whole trajectory shape similarity.
This means that we expect it to generate the crispest looking clusters.
The expectation of crisp clusters is further substantiated by its tolerance to noise
This method aimed to be invariant to the physical locations, meaning that trajectories of similar shapes with different origins could be grouped together. This could result in some broader bands of similarity.

% All of the parametric measures adapt to local time shifts like in the same manner as DTW, and  therefore we expect the clusters they generate exhibit some of the same type of fuzziness. This, of course, is not the only factor that affects how fuzzy or crisp the clusters become. The choice of parameter, as well as the (internal workings) of the algorithms matter greatly.
%SOMETHING ABOUT HOW DTW ERP etc have re-s sim to dtw as noted in \cite{26}

Recall that \textbf{Edit Distance on Real Sequences} uses the parameter $\epsilon$ to as the matching threshold for how close how two trajectory elements are.
Decreasing the parameter value leads to an increase in strictness in how close elements have to be, and in turn, would lead to crisper clusters.
It goes without saying that a too restrictive $\epsilon$ would no longer be accurate; if no trajectory elements are matching, the only basis for similarity would be the trajectory lengths.
Clusters are expected to have some level of fuzziness as EDR considers the trajectory element in isolation.

\textbf{Edit Distance with Real Penalty} was computed with one parameter value.
We have established that this metric is sensitive to noise and that it accommodates local time shifts.
As EDR, this measure does isolates the trajectory elements when computing the similarity distance.
This means that there are three factors that contribute to fuzzy clusters, thus we expect the fuzziest clusters to be generated from this metric.
% As it is a measure that handles local time shifts and does not compare a trajectory as a whole, we expect the clusters to appear quite fuzzy. This expectation is further supported by its low tolerance to noise.

\textbf{Move-Split-Merge} handles local time shifts in the same manner as DTW and the other Edit Distance-based measures.
We expect to observe trajectory sections that match and create wider bands.
MSM distinguishes itself by accounting for the values that surround a given element when computing the similarity score.
This means that we expect the clusters are crisper than those of generated by EDR and ERP.
The parameter of MSM determines to which degree Splits and Merges are favored over Moves.
As the cost increases, they will increasingly be evaded in favor of directly substituting the element.
The cost of a Move is pairwise element-element distance, and in turn, the clusters would resemble those created by a Euclidean distance based measure.
On the other hand, if the parameter is set too low we expect more trajectories that do not resemble each other to get clustered together.

%I thin,, it is hard to tell

% Finally we study how we expect the change of the cost of MSM transformations to impact the final clusters. The parameter determines how much splits and merges should be favored over moves. At the extreme end, MSM will resemble DTW as element-element distance would dominate the cost matrix. That is, the higher the value of c, the more the cluster should resemble DTW. For lower values of c, we should see a preference for actively splitting and merging, bringing its cluster results closer to that of ERP.

% Setting the cost to 1, results in similarity scores that are

\subsection{Visual Inspection}

The visual inspection inspects the apparent fuzziness of the clusters as well as how scatted the trajectories within a cluster are.
We will refer to the latter of these characteristics as the \textit{band} of similarity which describes the general \textit{trend} of the cluster.
Where is it possible, we remark how larger trajectory sections were processed.
Unfortunately, spotting tendencies like that is not something visual inspection excels at.

In the case of affinity propagation, we take note of how many clusters it generated.
The corresponding evaluation for the hierarchical clusters is taking note of how balanced the final clusters are.
With how hierarchical clusters are created, some imbalance is expected as the most distinct trajectories will be connected last.

% contour
%  silhouette
%  hierarchical affinity

\textbf{Euclidean Distance:}
\begin{itemize}
\item AP:  In terms of overall shape, the clusters appear very fuzzy.
There are some clusters where that have a clearer contour and some where it is possible to spot a trend for the trajectories.
Nevertheless the general impression remains fuzzy; we observe trajectories that seem to oscillate freely around the apparent trajectory band.
While oscillations make the clusters appear fuzzier, the clusters do not come across as randomly grouped observations.

\medskip
\item HCA: There appears to be a bias and clusters are unbalanced.
Some of the clusters contained a few trajectories while some of them encompassed a large number of them.
In those clusters the bands were obvious, but the oscillations were even greater than those observed under AP.
This makes sense as the new clusters are created by merging similar sub-clusters.
We would expect a clearer band, but a fuzzier contour.
\end{itemize}

\textbf{Dynamic Time Warping:}
\begin{itemize}
\item AP: We observed two clusters that were very crisp and even more clusters that exhibited clear trends.
The trajectories do not seem to oscillate around the band in the clusters as much as they did under Ed.
We can spot some trajectory sections that appear as if they have been re-aligned– leading to a less messy expression.
Still, there are some oscillations and noisy clusters that make it hard to tell exactly why some trajectories were clustered together.
\medskip
\clearpage
\item HCA: The clusters are unbalanced, however less so than the ones created by Ed.
They illustrate how shape similarity is persevered under this measure by displaying a clear trend in the clusters.
Yet, the clusters are fuzzy in that their trajectories still deviate from the central band.
From these clusters it appears as if some of the trajectories are outliers;
they appear to be distinct from the rest of the trajectories and are placed in their own clusters.
\end{itemize}

\textbf{Hausdorff Distance}
\begin{itemize}

\item AP: Compared to DTW and Ed, the final number of clusters increased and the clusters created were both crisper and fuzzier.
The clusters display both the most advantageous and most disadvantageous aspects of the Hausdorff metric.
% Reducing the similarity score to one element-element distance did indeed  amount
However, the number of crisp clusters outweighs the fuzzy ones, and the contours of the clusters are crisp enough to highlight more intricate trajectory details.
% This contrasts the other clusters clustered together, rather than a general direction across the grid

\medskip
\item HCA: Again, we see both crisp and intricate clusters as well as very fuzzy ones.
The observations from affinity propagation hold for these clusters as well;
both the advantages and disadvantages of setting the similarity score as the distance between two trajectory elements are highlighted.
There is one cluster that is so fussy that we were quite puzzled by how it was formed.
% It may be that trajectories with outliers and noise were put together in such a way that they were at some point considered so similar to each other that they were gathered.
\end{itemize}

\textbf{Symmetrized Segment-Path Distance:}
\begin{itemize}

\item AP: This measure created fewer clusters than Hd, yet the clusters it created appear to be at least equally as crisp.
We observed that the clusters had such tight bands that curves of the trajectories were accentuated.
In contrast, DTW clustered the trajectories by their general shape and direction.
Whereas Hd was sensitive to noise, it becomes clear that SSPD addressed that weakness.
Two clusters stand out as more fuzzy than the others, but this is likely due to noisy data.
\medskip
\item HCA: The clusters are about as balanced as those created by Hd.
As was demonstrated in the AP-clustering, there is a trend towards showing the intricacies of shape similarity at a larger scale.
The bands of similarity are thinner than both Ed end DTW and more detailed than Hd.
\end{itemize}

\textbf{Edit Distance with Real Penalty:}
\begin{itemize}

\item AP:  The clusters created by ERP are significantly more fuzzy than that of Hd and SSPD.
This is expected as it gives similarity scores based on edits before and then the distance between trajectory elements.
However, it is unexpected that the resulting clusters were as fuzzy as those created by DTW and Ed, especially as it created far more clusters than either of them did.
The increased number of clusters should indicate that we would have more distinguished clusters, but this does not appear to be the case.
Even still, we observe that clusters were not randomly put together.
Looking closer at specific clusters, it is possible to spot turns and segments that are just shifted from each other.
\medskip
\item HCA: These clusters are unbalanced, there is a cluster that only has one trajectory.
Yet, that trajectory does not appear significantly distinct from the ones existing clusters.
From visual inspection it challenging to get more insight.
The bands across the clusters are quite general and there is not a consistent show of shifted segments.

\end{itemize}

\textbf{Edit Distance on Real Sequence: }
In discussing this method, we examine the clusters obtained from both parameter values in parallel.
% $\epsilon = \{0.101, 0.201\}$

% contour
%  silhouette
%  hierarchical affinity propagation

\begin{itemize}
\item AP: This measure generated the two largest number of AP-clusters.
The most restrictive parameter led to 48 clusters while the other value resulted in 39 clusters.
In our subjective opinion, both of the parameters resulted in too many clusters for a data set of 300 observations.
However, with fewer trajectories in each cluster analysis through visual inspection becomes is easier.
We observed that the clusters contained trajectories whose segments were shifted from each other.
Both parameter values resulted in crisp clusters, and our approximation of the recommended value gave seemingly more defined contours.
While increasing the threshold value led to fuzzier clusters, these clusters were crisper than those of ERP.
This is an expected result as EDR is noise-tolerant whereas ERP is not.
\medskip
\item HCA: For both parameter values, there were bands that obscured the intricacies of shape similarity.
Once more we observed that increasing the parameter value resulted in fuzzier clusters.
However, the increase in fuzziness was less than expected.
The largest parameter value led to a more unbalanced distribution of trajectories, further signifying that the smaller one remains the most suited value for $\epsilon$.
\end{itemize}

\textbf{Move-Split-Merge:}

As we did for EDR, the clusters created by MSM and the effect of varying the parameter are discussed in parallel.
We first note that the clusters MSM created with the cost was set to 1 and were indistinguishable from those of Ed.
This was not an entirely unexpected result given what we know about MSM, and that the two methods had identical rankings of the most similar trajectory pairs.
By reason of that cost parameter being set too high and the accompanying clusters already being described, we will focus on the two remaining parameter values.

  % cost$= \{1, 0.1, 0.01\}$
\begin{itemize}

\item AP: We observed that the lowest parameter value resulted in fewer clusters, but both of the parameter values led to a larger number of clusters than expected.
As with EDR, the large number of clusters makes it easier to visually analyze them.
Given that this measure is both moderately noise-tolerant and context-aware we expect the clusters to be crisp.
Indeed, the majority of the clusters have crisp contours wherein the intricacies of the trajectories are preserved.
Some clusters contain what appear to be outlier trajectories.
This would suggest that the discrimination degree of this measure was high enough to isolate them in a way that no measures did.
The smaller parameter value resulted in both fuzzier clusters and an increase in clusters with few trajectories.
This may be an indication of that value being too low, suggesting that $0.1$ was the most appropriate cost for this data set.

\medskip
\item HCA: The clusters created by the lowest cost parameter were more unbalanced and fuzzier than those generated by the middle one.
This further supports the argument that the middle of the cost value is the most fitting one.
Interestingly, there were still trajectories that were clustered by themselves as if they were outliers.
It looks as if these trajectories are consistent across both parameter values and clustering techniques.

\end{itemize}

\section{Reflections}
% Other Applications write here, but i i think it might belong to "further work"

The measures created clusters whose qualities coincide with the anticipated results.
We observed fuzzy clusters where the underlying measure was less tolerant to noise and crisper clusters where the measure accounted for whole trajectory similarity.
Almost all of the measures were elastic, and we observed several clusters with trajectories that were time-shifted.

By visual inspection, it was hard to tell if metricity affected the clusters although this may have been a result of picking clustering techniques that did not require a metric distance function.
% metricity.

The clusters' appearance remained consistent between AP and HCA for a given measure, even as the number of clusters changed.
This observation was to be expected as the models that created the clusters used the same similarity distances scores.

Lastly, we comment on a few algorithm-specific observations.
The fuzziness of the clusters created by the Euclidean distance align with its lock-step design just as the crispness of SSPD’s clusters aligns with its design purposes.
It was initially unexpected that ERP would have the fuzziest looking clusters. However, upon closer examination, we reasoned that it made sense that a non-context aware, noise sensitive method that adapts to time shifts would lead to quite fuzzy clusters.

The way MSM could consistently filter out trajectories that did not behave like the others would imply that it would be well suited for outlier detection.

% For \textbf{MSM} the effects of its parameter, setting $c$ to $0.1$ resulted in AP generating a cluster consisting of just one trajectory and decreasing it $0.01$  lead to 5 of these single trajectory clusters. This could be an indication of poorly chosen parameter values. Looking at the HCA- generated clusters support this assumption. We observed less balanced clusters as the cost parameter decreased. Based on this, we have conclude that MSM with cost parameter $c=1$ best is the most representative for this measure and further analysis will be based on it.

\section{Inconsistencies from Simplifications}
As noted in \Cref{ch:5}, several simplifications were made for this experiment.
We close this chapter by reflecting on how these simplifications have affected the results.

% \clearpage

\subsection{Data format}
%  For real life data, it is not reasonable to assume that the trajectories will be of the same length and furthermore, if the trajectories are of the same length this still does not mean the time spent in each location matches up.

The first simplification that was made was truncating the trajectories to equal length.
In doing this we removed the option to study how measures would have handled cases like this.
There would be insight to be gained from examining measures at different trajectory lengths.
In particular, it is not reasonable to assume that real data would have this property.
The clusters could have ended up looking quite different, possibly highlighting the difference between trajectory section and whole trajectory similarity.

Of the algorithms in this thesis, only the Euclidean distance lacked a definition for trajectories of unequal length.
It is possible to design a comparative study where lock-step measures can give scores for these trajectories.
An option would be to artificially add more trajectories elements by interpolation.
However, we decided against it as we felt confident there would be enough properties to examine after the simplification.

The next data-format simplification we did was the re-scaling.
The manner in which it was done meant that the data would lose its connection to the real world.
To exemplify this we refer to \Cref{fig:sspd_loc_real}.
The clusters were created based on the scaled data, thus those clustered were quite crisp.
However, when displaying those same clusters with their raw coordinates it becomes clear how scattered the trajectories are.
It becomes clearer that trajectories were clustered together based on shape similarity.
We reiterate that the intent was to study measures themselves, thus the data set selection— and thereby the re-scaling was inconsequential.

\begin{figure}[h]
  \centering
  \includegraphics[width=.9\linewidth,height=.9\textheight,keepaspectratio]{figs/clusters/CLU_AP_ALL[SSPD]_REAL.png}
  \caption{Affinity propagation clusters created with SSPD, but showing the raw trajectory locations}
  \label{fig:sspd_loc_real}
\end{figure}

\subsection{Distance Computation and Evaluation}
% Tweaking the format and scale of the data is one thing, but the next line of simplifications mattered more for the evaluations of the measures.

We tested three measures that required a parameter and we did not do any proper parameter tuning.
ERP would likely have had more agreement with the other measures in \Cref{tab:top-10-sim-pairs} if different values for the reference point had been tested out.

% Next, while we did not  the measues

% The last simplification we would like to bring attention to is also something that potentially affected the final evaluation.

The application we chose for the similarity distance measures, clustering, is itself an area of active research.
The interconnectedness of clustering techniques and distance algorithms could have been studied in more detail before settling on affinity propagation and hierarchical clustering analysis as the evaluation basis.
We acknowledge that there may be traits of the selected measures that have been obscured or misrepresented.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                 %
%  GEANT manual in LaTeX form                              %
%                                                                 %
%  Michel Goossens (for translation into LaTeX)                   %
%  Version 1.00                                                   %
%  Last Mod. Jan 24 1991  1300   MG + IB                          %
%                                                                 %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Origin{G.Tromba, P.Bregant}
\Submitted{10.10.89}\Revised{16.12.93}
\Version{Geant 3.16}\Routid{PHYS251}
\Makehead{Simulation of Rayleigh scattering}
\Shubr{GRAYL}{}

\Rind{GRAYL} generates Rayleigh scattering of a photon using
the random-number
composition and rejection technique to sample the momentum
of the scattered photon and the scattering angle, according to the
form-factor distribution. In this reaction no new particles are
generated and the kinematical quantities of the scattered photon
replace the original ones in the \FCind{/GCTRAK/} common block.

Activation of the Rayleigh scattering is done via the {\tt FFREAD} data
record {\tt RAYL}. If this process is activated, \Rind{GRAYL} is called
by \Rind{GTGAMA} when a Rayleigh scattering occurs.
\section{ Method }

The Rayleigh differential cross section as a function of $ q^2 $ is given
by \cite{bib-NELS}:
\begin{equation}
\frac{d \sigma_R \left( q^2 \right) }{ d \Omega } =
 \frac{ \pi r_0^2}{ k^2 } \left( \frac{1 + \mu^2 }{ 2 } \right)
 \left| F_T \left( q \right) \right| ^2
\end{equation}
where:\\

\[\begin{array}{LL}
r_0       & \mbox{electron radius} \\
k         & \mbox{incident wave vector} \\
q=2k sin \frac{ \theta }{2}  & \mbox{momentum of scattered
              photon ($\theta$ is the scattering angle)} \\
\mu = cos \theta  & = 1 - \frac{q^2 }{ 2 k^2} \\
F_T \left( q \right)     & \mbox{molecular form factor}
\end{array} \]

Under the assumption that the atoms of a molecule are completely independent,
$ \left| F_T \left( q \right) \right| ^2 $ is given by:
\begin{equation}
 \left| F_T \left( q \right) \right| ^2 = \sum_{i=1}^{N}
 \frac{W_i}{A_i}  \left| F_i \left( q_i , Z_i  \right) \right| ^2
 \sigma_{c_i} \left( Z_i , E \right)
\end{equation}
where the index $i$ runs on the $N$ atoms in the molecule and:

\[\begin{array}{LL}
 W_i        & \mbox{proportion by weight} \\
 Z_i , A_i  & \mbox{atomic number and weight} \\
 F_i        & \mbox{form factor} \\
\sigma_{c_i}& \mbox{total atomic cross section for coherent scattering}\\
\end{array} \]

Using the combined composition and rejection sampling method described in
\Rind{GPAIRG} ({\tt [PHYS211]}) we may set:
\begin{equation}
f \left( q \right) = \sum_{i=1}^{N} \alpha_i f_i \left( q \right)
g_i \left( q \right) = \sum_{i=1}^{N} A \left( q_i^2 \right)
\frac{ \left| F_T \left( q \right) \right| ^2 }{ A \left( q_n^2 \right) }
\left( \frac{1+\mu^2}{2} \right)
\end{equation}

where:

\[\begin{array}{LL}
 n          & \mbox{number of energy bins} \\
 q_i        & \mbox{momentum of the photon with energy $ E_i $ of the
                 $i^{th}$  bin} \\
 q_n        & \mbox{upper limit for the momentum of the scattered photon} \\
 \alpha_i   & A \left( q_i^2 \right)  \\
 f_i \left( q \right )  &
 \frac{\left| F_T \left( q \right) \right| ^2}{A \left( q_n^2 \right)}  \\
 g_i \left( q \right )  & = \frac{1+\mu^2}{2} \mbox{\hspace{0.5cm}
rejection function.}
\end{array} \]

Therefore, for given values of the random numbers $r_1$ and $r_2$, \Rind{GRAYL}
samples the momentum of the scattered photon and the scattering angle
$\theta$ via the following steps:

\begin{enumerate}
\item sample $ A \left( q^2 \right) = r_1 A \left( q_n^2 \right) $
\item find the $ \left (  q_{i-1} , q_i  \right] $ interval
which gives $ A \left( q_{i-1}^2 \right) \leq A \left( q^2 \right)
\leq  A \left( q_i^2 \right) $
\item calculate the linear extrapolation:
\begin{equation}
q = q_{i-1} + \left( A \left( q^2 \right) -  A \left( q_{i-1}^2 \right) \right)
\frac{q_i - q_{i-1} }
{ A \left( q_i^2 \right) -  A \left( q_{i-1}^2 \right)} \nonumber
\end{equation}
\item calculate $\mu = cos \theta  = 1 - q^2/(2 k^2) $
\item calculate $g_i \left( q \right) = (1 + \mu^2 )/2 $
\item if $g_i \left( q \right) > r_2 $ the event is accepted, otherwise
go back to 1.
\end{enumerate}


\section{Optical Tracking}
\subsection{Overview}
The Optical Tracking System is implemented with three subsystems:
\begin{enumerate}
  \item Video Extraction
  \item Pinhole Camera Model
  \item Epipolar Geometry
\end{enumerate}
\subsection{Video Extraction}
The Video Extraction system's function is to extract the location of a patricular object within the video footage. The objects, in this case, are differently colored small balls. The small balls represent sever key points of the Quadcopter. By determining the location of the balls, we will be able to calculate the position and attitude of the Quadcopter.
The procedure of video extraction is described as following:
\begin{enumerate}
  \item Shotting: Get images from camera,
  \item Transforming: transform the image to HSV colorspace,
  \item Extracting: use tresholding to extract pixels in patricular color range, then use histogram and backprojection to determin the probablity that the pixels belong to the model,
  \item Denoising: use opening and closing to denoise,
  \item Determining: use \emph{Lucas-Kanade} and \emph{CamShift} algorithm to determin the centroid of the extracted pixels.
\end{enumerate}
\subsubsection{Shooting}
In this project, v4l\footnote{a.k.a. Video For Linux} is the middleware of OpenCV and camera. With \emph{v4l} and \emph{OpenCV}, reading an image from camera is as easy as following:
\lstset{language=python}
\begin{python}
cap = cv2.VideoCapture(source)
img = cap.read()
\end{python}
\subsubsection{Transforming}
Different from the RGB colorspace, HSV colorspace has a unique character: its V channel represents the brightness. If we remove V channel from tresholding, the same color profile will be able to work in different lighting conditions.
The transforming process is described below\cite{cite4}:
\begin{align}
  M &= \operatorname{max}(R, G, B) \\
  m &= \operatorname{min}(R, G, B) \\
  C &= M - m\\
  H^\prime &=
    \begin{cases}
      \mathrm{undefined},        &\mbox{if } C = 0 \\
      \frac{G - B}{C} \;\bmod 6, &\mbox{if } M = R \\
      \frac{B - R}{C} + 2,       &\mbox{if } M = G \\
      \frac{R - G}{C} + 4,       &\mbox{if } M = B
    \end{cases} \\
  H        &= 60^\circ \times H^\prime\\
  V &= M\\
   S &=
    \begin{cases}
      0,           &\mbox{if } C = 0 \\
      \frac{C}{V}, &\mbox{otherwise}
    \end{cases}
\end{align}
The code used to transform the image is:
\begin{python}
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
\end{python}
Note that by default, OpenCV uses BGR color space.
\subsubsection{Extracting}
OpenCV's built in treshold function is used to utilize multithreading.
The process of tresholding is printed below:
\begin{python}
mask = cv2.inRange( hsv,
                    np.array((0., 60., 32.)),
                    np.array((180., 255., 255.))
                    )
\end{python}
However, simply tresholding is often not enough. The object tracked can have a complex color feature. In order to track the entire object, it is necessary to consider all of the features. Histogram is used to model the color distribution of an object, and me can then map the probability of pixels using backprojection.\\
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.2\textwidth]{../Pictures/histogram.JPG}
    \caption{A sample histogram\cite{cite5}}\\
\end{figure}
Backprojection uses the modeled histogram to find the probability that certain pixel belongs to the model. Consider an image matrix $M$, each pixel can be described as pair $(H_{i,j},S_{i,j})$. We can use the two values to create a 2D histogram that represents the distribution of certain color range in $H$ and $S$. Below is a sample 2D histogram generated by \emph{OpenCV Python Sample}.\\
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.3\textwidth]{../Pictures/2dhist.png}
    \caption{A sample 2D histogram}\\
\end{figure}
After generating the histogram, the probability that a pixel belongs to the model can be expressed by the following steps:
\begin{enumerate}
  \item Find the pair $(H_{i,j},S_{i,j})$ of a pixel,
  \item find the correspondent bin in the histogram,
  \item get the relative frequency of the bin.
\end{enumerate}
Then we can use the frequency to map a binary image. In this image, each pixel represents the probability that the correspondent pixel in the original image belongs to the model. Below is an example backproject image:\\
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.5\textwidth]{../Pictures/backproject.png}
    \caption{A sample backprojrction}\\
\end{figure}
\subsubsection{Denoising}
After calculating the backproject image, we will be able to use morphology transformation to remove noises. In this projcet, we used Opening and Closing to remove small noise pixels and fill black pixel holes in the backprojection.\\
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.3\textwidth]{../Pictures/Opening.png}
    \caption{Sample Opening operation\cite{cite7}}\\
\end{figure}
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.3\textwidth]{../Pictures/Closing.png}
    \caption{Sample Closing operation\cite{cite6}}\\
\end{figure}

The result is significant. A great deal of noises is removed in this process:\\
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.3\textwidth]{../Pictures/before.png}
    \caption{Befor morphology transformation}\\
\end{figure}
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.3\textwidth]{../Pictures/after.png}
    \caption{After morphology transformation}\\
\end{figure}
\newpage

\subsubsection{Extracting}
There are two stages of extracting. They are:
\begin{enumerate}
  \item Detecting
  \item Tracking
\end{enumerate}
The difference between them is that \emph{Tracking} requires the original location of the object to be known. That information should be fed with \emph{Detecting} in the \emph{first} frame of video source.

By \emph{Detecting}, we find locations where the image of \emph{source} is similar to \emph{template}.
The algotighm we used in \emph{Detecting} is rather simple. It "shifts" the \emph{template} on the \emph{source}, and find the similarities bewteen the \emph{template} and the image below.

The method of template matching is printed below:
\begin{minted}[mathescape,
               linenos,
               numbersep=5pt,
               gobble=2,
               frame=lines,
               framesep=2mm]{python}
  import numpy as np
  import cv2
  import math

  def match(template, image):
    '''
    Match template on image and return the centroid's coord.
    '''

    image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)

    result = cv2.matchTemplate(gray, patch, cv2.TM_CCOEFF_NORMED)
    result = np.abs(result) ** 3
    val, result = cv2.threshold(result, 0.01, 0, cv2.THRESH_TOZERO)

    minVal, maxVal, minLoc, maxLoc = cv2.minMaxLoc(result)

    height, width, depth = template.shape
    x, y = maxLoc
    x += width/2
    y += height/2

    return (x,y)
\end{minted}

In order to make things clearer, we used a marker from the TV series \emph{Person of Interest} to mark the point of interest.
Here's the code we used to mark the object:
\begin{minted}[mathescape,
               linenos,
               numbersep=5pt,
               gobble=2,
               frame=lines,
               framesep=2mm]{python}
  import numpy as np
  import cv2
  import math

  def draw_machine_mark(size, location, image):
      '''
      This function draws a machine mark on the image
      '''
      original = cv2.imread("assets/machine.png", -1)

      osize = math.sqrt(original.size)/2
      ratio = size / osize
      height, width, depth = image.shape
      timg = cv2.resize(original, (0,0), fx=ratio, fy=ratio)
      x, y = location

      image[:] *= 0.8

      x_start = max(0, x - size/2)
      x_end = min(x_start + size, width)
      x_start = min(x_end - size, x_start)
      y_start = max(0, y - size/2)
      y_end = min(y_start + size, height)
      y_start = min(y_end - size, y_start)

      roi = image[y_start : y_end, x_start : x_end]
      for c in range(0,3):
          roi[:,:,c] = timg[:,:,c] * (timg[:,:,3] / 255.0) + roi[:,:,c] * (1 - timg[:,:,3] / 255.0)
      image[y_start : y_end, x_start : x_end] = roi
\end{minted}
Below is an image of the effect:\\
\begin{figure}[h!]

  \centering
    \includegraphics[width=0.5\textwidth]{../Pictures/after.png}
    \caption{Example of tracking effect}\\
\end{figure}