\input{../../style/preamble}
\input{../../latex-math/basic-math}
\input{../../latex-math/basic-ml}
\input{../../latex-math/ml-bagging.tex}
\input{../../latex-math/ml-boosting.tex}
\input{../../latex-math/ml-trees.tex}

\newcommand{\titlefigure}{figure/fig-gb-concept-2.png}
\newcommand{\learninggoals}{
  \item Understand idea of forward stagewise modelling
  \item Understand fitting process of gradient boosting for regression problems
}

\title{Introduction to Machine Learning}
\date{}

\begin{document}

\lecturechapter{Gradient Boosting}
\lecture{Introduction to Machine Learning}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{Gradient Boosting}

% \begin{vbframe}{Gradient descent}
% % \begin{vbframe}{Forward stagewise additive modeling}
%
% % Let's recall gradient descent from numerical optimization.
% Let $\risk(\theta)$ be (just for the next few slides) an arbitrary, differentiable, unconstrained objective function, which we want to minimize. The gradient $\nabla \risk(\theta)$ is the direction of the steepest ascent, $-\nabla \risk(\theta)$ of \textbf{steepest descent}.
%
% \lz
%
% For an intermediate solution $\theta^{[k]}$ during minimization, we can iteratively improve by updating
% $$
% \theta^{[k+1]} = \theta^{[k]} - \beta \nabla \risk(\theta^{[k]}) \qquad  \text{for } 0 < \beta \leq c\left(\theta^{[k]}\right)
% $$
% % \enquote{Walking down the hill, towards the valley.}
%
% % $f(x_1, x_2) = -\sin(0.8 x_1) \cdot \frac{1}{2\pi} \exp\left( (x_2 x_1 + \pi / 2)^2 \right)$
% <<sd-plot>>=
% #modified from ../../../cim1/2017/11-Optimierung/functions.
%
% sd_plot = function(col = terrain_hcl, theta = 40, phi = 40, xlab = "x", ylab = "y") {
%   if (is.function(col)) col = col(nrow(z) * ncol(z))
%
%   par(mfrow = c(1, 2))
%
%   par(mar = rep(0.5, 4))
%   require("colorspace")
%   pmat = persp2(x, y, z, theta = theta, phi = phi, ticktype = "detailed",
%       xlab = xlab, ylab = ylab, zlab = "", col = col, lwd = 0.3, border = NA)
%
%   for (j in seq_along(p)) {
%     t3d = trans3d(p[[j]][[1]], p[[j]][[2]], do.call(foo, p[[j]]), pmat)
%     if (j > 1) {
%       t3d2 = trans3d(p[[j - 1]][[1]], p[[j - 1]][[2]], do.call(foo, p[[j- 1]]), pmat)
%       lines(c(t3d$x, t3d2$x), c(t3d$y, t3d2$y))
%       points(x = t3d2$x, y = t3d2$y, pch = 16, col = heat_hcl(1))
%     }
%     points(x = t3d$x, y = t3d$y, pch = 16, col = heat_hcl(1))
%   }
%   par(mar = c(4.1, 4.1, 1.1, 1.1))
%   image(x, y, z, col = col, xlab = xlab, ylab = ylab, useRaster = TRUE)
%   contour(x, y, z, add = TRUE, nlevels = 15)
%   for (j in seq_along(p)) {
%     if (j > 1) {
%       lines(c(p[[j]][1], p[[j - 1]][1]), c(p[[j]][2], p[[j - 1]][2]))
%       points(p[[j - 1]][1], p[[j - 1]][2], pch = 16, col = heat_hcl(1))
%     }
%     points(p[[j]][1], p[[j]][2], pch = 16, col = heat_hcl(1))
%   }
%   invisible(NULL)
% }
%
% @
% <<gradient-descent, fig.align="center", echo=FALSE, fig.width=8, fig.height=4, out.height="3cm", out.width="6cm">>=
% foo = function(x, y) {
%   -1 * sin(.8 * pi*x) * dnorm(-y * x, mean = pi / 2, sd = 0.8)
% }
%
% x = seq(0, 2.5, length = 50)
% y = seq(-3, 1, length = 50)
% z = outer(x, y, foo)
% p = c(list(list(1.8, -.5)), optim0(1.8, -.5, FUN = foo, maximum = FALSE, maxit = 19))
%
% sd_plot(phi = 35, theta = -20, xlab = "x_1", ylab = "x_2", col = viridis::viridis)
% @
%
% % {\scriptsize step size  $\beta = 1$}
%
% % \framebreak
%
% % $\beta$ is called \textbf{step size}, and can be set by
% % \begin{itemize}
% % \item fixing it to a (smallish) constant
% % \item adapting it according to previous gradient values, the local Hessian, etc.
% % \item line search methods, which solve $\beta^{[k]} = \argmin_{\beta} f\left(x^{[k]} - \beta \nabla f\left(x^{[k]}\right)\right)$. Only one real parameter $\beta$, i.e, \enquote{easy} to solve\dots
% % \end{itemize}
%
%
%
%
% \end{vbframe}

\begin{vbframe}{Forward stagewise additive modeling}

Assume a regression problem for now (as this is simpler to explain);
and assume a space of base learners $\mathcal{B}$.

\lz

% A weak learner should have the property that it delivers better predictions than by random chance (e.g. for a balanced training set a misclassification error less than 1/2).

% \lz

We want to learn an additive model:

$$
\fx = \sum_{m=1}^M \betam \bmmxth.
$$

Hence, we minimize the empirical risk:

$$
\riskef = \sum_{i=1}^n L\left(\yi,\fxi \right) =
\sum_{i=1}^n L\left(\yi, \sum_{m=1}^M \betam \bmmxth\right)
$$

% \framebreak

% A common \textbf{loss} for \textbf{regression} is the \textbf{squared error} with
% $\Lxy = (y-\fx)^2$.

% \lz

% Apparently, $\risk$ depends on the \textbf{base learners} $b(x, \thetam)$,
% or rather their parameters $\thetam$ and weights $\betam$. Hence, we have to optimize these.

% \lz

\framebreak
\textbf{Why is gradient boosting a good choice for this problem?}
\begin{itemize}
\item Because of the additive structure it is difficult to jointly minimize $\riskef$ w.r.t. $\left(\left(\beta^{[1]}, \bm{\theta}^{[1]}\right), \ldots, \left(\beta^{[M]}, \bm{\theta}^{[M]}\right)\right)$, which is a very high-dimensional parameter space (though this is less of a problem nowadays, especially in the
case of numeric parameter spaces).
% - however, this is nowadays especially in the case of numeric parameter spaces not a real problem anymore.
\item Considering trees as base learners is worse as we would have to grow $M$ trees in parallel so they
  work optimally together as an ensemble.
\item Stagewise additive modeling has nice properties, which we want to make use of, e.g. for regularization, early stopping, \dots
\end{itemize}

\framebreak

Hence, we add additive components in a greedy fashion by sequentially minimizing the risk only w.r.t. the next additive component:

$$ \min \limits_{\beta, \bm{\theta}} \sum_{i=1}^n L\left(\yi, \fmdh\left(\xi\right) + \beta b\left(\xi, \bm{\theta}\right)\right) $$

\lz

Doing this iteratively is called \textbf{forward stagewise additive modeling}.

\input{algorithms/forward_stagewise_additive_modeling.tex}

\end{vbframe}

% \section{Gradient Boosting}

\begin{vbframe}{Gradient boosting}

% \begin{footnotesize}
This is not really an algorithm, but an abstract principle.
To find $b\left(\xv, \thetam\right)$ and $\betam$, we use gradient descent, but in function space!

\lz
\begin{columns}
\column{5cm}
% \textbf{Thought experiment:}
Consider a model $f$ whose predictions we can arbitrarily define for each training $\xi$, i.e.,
$f$ is a finite vector
  $$\left(f\left(\xv^{(1)}\right), \ldots,  f\left(\xv^{(n)}\right)\right)^\top $$

This implies $n$ parameters $\fxi$ (and the model would provide no generalization...).

Also, we let's assume $L$ to be differentiable.

\column{5cm}
\begin{center}
  \vspace{-1cm}
  \includegraphics[width=\textwidth]{figure/fig-gb-concept-2.png}
\end{center}

\end{columns}
% \end{footnotesize}
\end{vbframe}

\begin{vbframe}{Gradient boosting}

\textbf{Aim:} Define a movement in function space so we can push our current function towards the data points.

\vspace*{0.1cm}
\textbf{Given:} Regression problem with one feature $x$ and target variable $y$.

\vspace*{0.1cm}
\textbf{Initialization:} Set all parameters to the optimal constant value (e.g., the mean of $y$ for $L2$).

\begin{figure}
  \includegraphics[width=\textwidth]{figure/fig-gb-concept-1.png}
\end{figure}

\end{vbframe}

\begin{frame}{Pseudo Residuals}
% \begin{footnotesize}
How do we distort our $f$ to move it towards the labels and reduce risk?
\vspace*{0.2cm}
Let's minimize risk with GD.

So, we calculate the (negative) gradient of the risk for each parameter, which (weirdly) here are the outputs $\fxi$
(0 if $i \neq j$):
% \footnote{The gradient of all terms with $i \neq j$ are 0.}

$$
\tilde{r}^{(i)} = - \fp{\riske}{\fxi} = - \fp{\sum_j L(y^{(j)}, f(\xv^{(j)}))}{\fxi} = - \fp{\Lxyi}{\fxi}.
$$

\vspace*{0.1cm}

At each point, we would like to change the output via:
$ \tilde{r}(f) = - \fp{L(y, f)}{f} .$

\begin{columns}
\begin{column}{0.45\textwidth}
\begin{center}
  \textbf{L2 Example:} The PRs
  match the usual residuals:
  $$    \tilde{r}(f) = - \fp{0.5(y - f)^2}{f} = y - f $$
\end{center}
\end{column}
\begin{column}{0.55\textwidth}
\begin{center}
  \includegraphics[width=0.7\textwidth]{figure/pseudo_residual_1.png}
\end{center}
\end{column}
\end{columns}

% \end{footnotesize}

\end{frame}

\begin{vbframe}{Boosting as Gradient Descent}

%\begin{footnotesize}
\vspace*{0.2cm}
Combining this with \enquote{forward stagewise modeling}, we are at $\fmd$ during minimization.
Here, we calculate the direction of the negative gradient or vector of PRs:

$$ \rmi = -\left[\fp{\Lxyi}{f(\xi)}\right]_{f=\fmd} $$

\lz

The gradient descent update for each vector component of $f$ is:

$$
  \fm (\xi) =  \fmd (\xi) + \beta \rmi.
$$

Like this, we should \enquote{nudge} $f$ in the direction best risk reduction.

%\end{footnotesize}
\end{vbframe}

\begin{vbframe}{Gradient boosting}

\textbf{Iteration 1:}

Let's move our function $\fxi$ a fraction towards the pseudo-residuals with a learning rate of $\beta = 0.6$.

\begin{figure}
  \includegraphics[width=\textwidth]{figure/fig-gb-concept-pseudo-resi-1.png}
\end{figure}

\framebreak

\textbf{Iteration 2:}

Let's move our function $\fxi$ a fraction towards the pseudo-residuals with a learning rate of $\beta = 0.6$.

\begin{figure}
  \includegraphics[width=\textwidth]{figure/fig-gb-concept-pseudo-resi-2.png}
\end{figure}

\framebreak
% \begin{footnotesize}
% We find our $\betam$ by minimizing with line search:

% $$
  % \betam = \argmin_{\beta} \sumin L(\yi, \fmdh(x) + \beta b(x, \thetamh)),
% $$

% where $h(x, \thetam) = \rmm$.

% \lz

%What is the point of doing all this?
As said, such a model parameterization is pointless.

\vspace*{0.3cm}

So, we restrict our additive components to $b\left(\xv, \thetam\right) \in \mathcal{B}$.

% \framebreak

The pseudo-residuals are calculated exactly as stated above,
then we fit a simple model $b(\xv, \thetam)$ to them:
$$ \thetamh = \argmin_{\bm{\theta}} \sum_{i=1}^n \left(\rmi - b(\xi, \bm{\theta})\right)^2. $$

\lz

\begin{columns}
\column{5cm}
So, evaluated on the training data,
$b(\xv, \thetam)$ corresponds as closely as possible to the negative risk gradient and generalizes over $\Xspace$.

\column{5cm}
\vspace*{-1cm}
\begin{figure}[th]
  \includegraphics[width=\textwidth]{figure/fig-gb-concept-idea.png}
\end{figure}

\end{columns}
% \end{footnotesize}

\framebreak
% \begin{footnotesize}
\textbf{In a nutshell}: One boosting iteration is exactly one approximated gradient descent step in function space,
which minimizes the empirical risk as much as possible.

\vspace*{0.1cm}
\textbf{Iteration 1:}
\begin{figure}
  \includegraphics[width=0.9\textwidth]{figure/fig-gb-concept-idea-1.png}
\end{figure}
% \end{footnotesize}
\framebreak
% \begin{footnotesize}
Instead of moving the function values for each observation by a fraction closer to the observed data, we fit a regression base learner to the pseudo-residuals (right plot).

\vspace*{0.1cm}
\textbf{Iteration 2:}
\begin{figure}
  \includegraphics[width=0.9\textwidth]{figure/fig-gb-concept-idea-2.png}
\end{figure}
% \end{footnotesize}
\framebreak
% \begin{footnotesize}
This BL is added to the of the ensemble, weighted by a learning rate (here: $\beta = 0.4$).
Then we iterate.

\vspace*{0.1cm}
\textbf{Iteration 3:}
\begin{figure}
  \includegraphics[width=\textwidth]{figure/fig-gb-concept-idea-3.png}
\end{figure}

% \begin{footnotesize}
% This procedure is continued stepwise until the boosting algorithm terminates.
% \end{footnotesize}
% \end{footnotesize}
\end{vbframe}
%% Combining this with the iterative additive procedure
%% of \enquote{forward stagewise modelling}, we are at the spot $\fmd$ during minimization.
%% At this point, we now calculate the direction of the negative gradient:
%%
%% $$ \rmi = -\left[\fp{\Lxyi}{f(\xi)}\right]_{f=\fmd} $$
%%
%% We will call these $\rmi$ \textbf{pseudo residuals}. For squared loss they match the usual residuals
%%
%%
%% $$
%% - \fp{\Lxy}{\fx} = - \fp{0.5(y - \fx)^2}{\fx} = y - \fx
%% $$
%%
%%
%% \framebreak
%%
%% % We find our $\betam$ by minimizing with line search:
%%
%% % $$
%%   % \betam = \argmin_{\beta} \sumin L(\yi, \fmdh(x) + \beta b(x, \thetamh)),
%% % $$
%%
%% % where $h(x, \thetam) = \rmm$.
%%
%% % \lz
%%
%% What is the point in doing all this? A model parameterized in this way is senseless,
%% as it is just memorizing the instances of the training data...?
%%
%% \lz
%%
%% So, we restrict our additive components to $b\left(x, \thetam\right) \in \mathcal{B}$.
%%
%% % \framebreak
%%
%% The pseudo-residuals are calculated exactly as stated above,
%% then we fit a regression model $b(\bm{x}, \thetam)$ to them:
%% $$ \thetamh = \argmin_{\thetab} \sum_{i=1}^n (\rmi - b(\xi, \thetab))^2 $$
%% So, evaluated on the training data,
%% our $b(x, \thetam)$ corresponds as closely as possible to the negative
%% loss function gradient and generalizes to the whole space.
%%
%% \lz
%%
%% \textbf{In a nutshell}: One boosting iteration is exactly one approximated gradient step in function space,
%% which minimizes the empirical risk as much as possible.
%%
%% \end{vbframe}

\begin{vbframe}{Gradient boosting algorithm}

\input{algorithms/gradient_boosting_general.tex}

Note that we also initialize the model in a loss-optimal manner. %Also, the constant learning rate can be replaced by a line-search, however a small constant learning rate is commonly used.

\end{vbframe}

\begin{vbframe}{Line Search}
The learning rate, as always, influences how we converge.
Although a small constant LR is commonly used, we can also run line search,
% to solve this one-dimensional optimization problem:
$$\betamh = \argmin_{\beta} \sumin L(\yi, \fmd(\xv) + \beta b(\xv, \thetamh))$$

Alternatively, an (inexact) backtracking line search can be used to find the
$\betam$ that minimizes the above equation.

\end{vbframe}

\endlecture
\end{document}


\subsection{Causal forests}


\section{Introduction}
	In plasticity model, plastic deformation depends on the internal state variables which are measurable variables like strain rate, temperature, stress and material variables. The relation between all the internal state variables is expressed using flow rule.

In general, flow rule is the resistance to the plasticity and expressed as the ratio of the plastic driving force to the flow resistance. During plastic deformation, the driving forces direct the internal slip mechanisms which causes shearing. A general power law flow rule representation equ.(\ref{power_law_equ})$^{\cite{chowdhury2017}}$ \par

\begin{equation}\label{power_law_equ}
\dot{\gamma}^{(\alpha)}=\dot{\gamma}_{0}\left|\frac{\tau^{\alpha}}{g^{\alpha}}\right|^{n} \operatorname{sgn}\left(\tau^{\alpha}\right)
\end{equation}

In equ.(\ref{power_law_equ}) $\dot{\gamma}^{(\alpha)}$ is slip rate of the slip system $\alpha$, $\tau^{\alpha}$ is the shear stress, $g^{\alpha}$ is the internal state variable which depends on temperature.

Plasticity is mediated through dislocation. The basic types of dislocations are edge, screw and mixed dislocation. During plastic deformation, dislocation motion will occur in multiple slip system simultaneously. This lead to the interaction of dislocation with dislocations of other slip systems. This collective dislocation dynamics plays an important role in understanding the material response like fatigue, creep, yield.$^{\cite{Messerschmidt2010}}$

The entanglement of different dislocation generate a complex stress field. These complex stress field creates a low energy barriers which act as obstacles for dislocation movement having long range interactions. This complex system of interaction leads to the formation of dislocation avalanches. Dislocation avalanches are the basic mechanism of plastic flow in solids at the nano-scale. Dislocation avalanches are also associated with strain bursts which is a sudden increase in strain rate.
They follow power law distribution.
$^{\cite{Papanikolaou_2017}}$

A comprehensive overview on dislocation avalanche physics and the phenomena leading to strain bursts in crystal plasticity is discussed further.


%% \listfiles
\documentclass[apj]{emulateapj}
%\documentclass[preprint2,12pt]{emulateapj}
%% \usepackage{natbib}
\usepackage{graphicx}
\usepackage{epsfig}
\usepackage{amssymb,amsmath}
\usepackage{array}
\usepackage{threeparttable}
\usepackage{hyperref,graphicx}

\doublespace

%definitions
\newcommand{\Msol}{${\rm M_{\sun}}$}

%% Editing markup...
\usepackage{color}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% WARNING: This LaTeX block was generated automatically by authors.py
% Do not change by hand: your changes will be lost.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% --------------------- Ancillary information ---------------------
\shortauthors{SURP et al.}
\shorttitle{CITA Final Project}

\begin{document}

\title{CITA Final Project: Introduction to Galpy}
 %% ---------

\author{Yumna Arshad}
%\altaffiltext{1}{CITA, University of Toronto}

\section{Introduction}
My final project aims to introduce the methods of galpy, a module in python used for modelling the orbital dynamics of objects in the galaxy.
The project consists of 2 parts.
The first part is an introductory approach to galpy and focuses on specifically learning how to use both potential and orbit objects from the module as well as the various methods associated with them.
The second part consists of modelling the accretion of a globular cluster, that is initially bound to a satellite galaxy of the Milky Way, onto the Milky Way galaxy itself.

\section{Part 1: Intro to Galpy Methods}

For this part of the project there are 3 separate tasks: 1) plotting rotation curve of MW galaxy, 2) integrating orbit of Sun and 3) integrating orbit of a specific tidal stream named GD -1.

\subsection{Rotation Curve of MW Galaxy}
To plot the rotation curve of the MW galaxy I created 3 different potential instances corresponding to the potential of the disk, bulge and halo separately and plotted them on one plot using the plotRotcurve method. Then I summed up all the potentials to create the overall galactic rotation curve which peaks at around 225 km/s and slowly declines afterward, this is the sum of the bulge, disk and halo potentials.
A plot of these rotation curves can be seen in Fig.\ref{fig:Q1_rot_curve}.

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q1a.pdf}
    \caption{Plot of the rotation curve of the Milky Way Galaxy including its labelled components: bulge, halo and disk.}
    \label{fig:Q1_rot_curve}
\end{figure}

\subsection{Orbit of Sun in MW Galaxy}
To plot the orbit of the Sun in the MW Galaxy I initialized an Orbit instance and integrated it for 10 Gyr using the MW potential in part 1 to properly model the orbit.
I used the plot method of the sun orbit object to display the plot.
A figure of this plot is shown in Fig.\ref{fig:Q1_Sun}

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q1b.pdf}
    \caption{Plot of the orbit of the Sun in Milky Way potential for 10 Gyr (integrated forward). The orbit is plotted as: vertical height above plane of galaxy in kpc as a function of galactocentric radius in kpc.}
    \label{fig:Q1_Sun}
\end{figure}

\subsection{GD-1 Tidal Stream}
The path of GD-1 tidal stream in the sky is plotted by: initializing an orbit object with the given coordinates for the stream as specified in paper by Bovy and Webb \cite{GD-1}.
Figures showing the GD-1 stream in declination vs right ascension are shown in Fig.\ref{fig:Q1_radec_f} (integrated forwards) and in Fig.\ref{fig:Q1_radec_b} (integrated backwards).
Figures showing the GD-1 stream in distance (from the Sun) vs right ascension are shown in Fig.\ref{fig:Q1_radist_f} (integrated forwards) and in Fig.\ref{fig:Q1_radist_b} (integrated backwards).
For all 4 orbits, the integration period is 50 Myr either forwards or backwards in time.

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q1c_1.pdf}
    \caption{Tidal stream GD-1's declination vs right ascension integrated forwards in time.}
    \label{fig:Q1_radec_f}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q1c_3.pdf}
    \caption{Tidal stream GD-1's declination vs right ascension integrated backwards in time.}
    \label{fig:Q1_radec_b}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q1c_2.pdf}
    \caption{Tidal stream GD-1's distance (from Sun) vs right ascension integrated forwards in time.}
    \label{fig:Q1_radist_f}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q1c_4.pdf}
    \caption{Tidal stream GD-1's distance (from Sun) vs right ascension integrated backwards in time.}
    \label{fig:Q1_radist_b}
\end{figure}

\section{Part 2: Stimulating Accretion of Globular Cluster onto Milky Way}

\subsection{Method}
To stimulate the accretion of a globular cluster onto the Milky Way Galaxy: the first thing I did was import and integrate the orbits of all the satellite galaxies of the Milky Way and plotted their apocenters and pericenters vs their current galactocentric radii.
Using these plots, the satellite with the smallest pericenter is identified to be TucanaIII and the time at which it 1st reaches its smallest galactocentric radius is -1.77 Gyr (i.e 1.77 Gyr in the past). This satellite will be used to model the accretion of the globular cluster.
To begin with, the orbit of TucanaIII integrated backwards for 10 Gyr is plotted with the effects of dynamical friction included. Dynamical friction is implemented using an instance of the ChandrasekharDynamicalFrictionForce with the density of the MW potential (imported using MWPotential2014 which is a list of potential objects making up the Milky Way) and given satellite parameters such as mass and size. Then, the orbit of the satellite is integrated back in time using the combined potential of the MW and the dynamical friction. This orbit is shown in Fig.\ref{fig:Q2_cdf}. The radius of the satellite at 10 Gyr is determined to be 240.6 kpc.
\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q2c.pdf}
    \caption{Plot showing the orbit of TucanaIII satellite galaxy, the satellite with the smallest pericenter radius, integrated backwards in time for 10 Gyr. The orbit is shown in radius vs time.}
    \label{fig:Q2_cdf}
\end{figure}

A star-cluster is then initialized to be on a circular orbit of radius 4 kpc within the satellite galaxy, the orbital velocity can be determined using Eq.\ref{equation: v_circ} and this gives a velocity of $\approx$ 328 km/s.

\begin{equation}
     v = \sqrt{\frac{GM}{r}}
    \label{equation: v_circ}
\end{equation}

As well, the satellite + cluster system are moved to the radius determined above (i.e. 240.6 kpc). To model the orbit of the satellite characerized by a Hernquist potential, a MovingObjectPotential object instance is created with the associated orbit being that of the satellite as previously determined, i.e. orbit of TucanaIII.
Finally, the orbits of both the satellite and the star-cluster are integrated and plotted. The results are shown in Fig.\ref{fig:Q2_satellite} and Fig.\ref{fig:Q2_star_cluster}, respectively.

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q2_sat.pdf}
    \caption{Plot showing the orbit of the satellite galaxy integrated forwards in time for 12 Gyr. The orbit is shown in radius vs time and includes the effects of dynamical friction.}
    \label{fig:Q2_satellite}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=1.0\columnwidth]{Q2_sc.pdf}
    \caption{Plot showing the orbit of the star-cluster system within the satellite galaxy for 12 Gyr. Note that r is the galactocentric distance, plotted as a function of time.}
    \label{fig:Q2_star_cluster}
\end{figure}

\subsection{Comment on Orbits of Star Cluster and Satellite Galaxy}
The orbit of the satellite galaxy as shown in Fig.\ref{fig:Q2_satellite} is the same as the orbit in Fig.\ref{fig:Q2_cdf} since it initially starts at its position at -10 Gyr and follows the same path forward in time as it did to go back.
The star cluster's orbit within the satellite follows a bound orbit such that it becomes accreted onto the Milky Way galaxy as the satellite spirals in and remains bound to the MW galaxy.

\begin{thebibliography}{99}

\bibitem{GD-1}
J. J. Webb and J. Bovy, “Searching for the GD-1 stream progenitor inGaiaDR2 with directN-body simulations,” Monthly Notices of the Royal Astronomical Society, vol. 485, no. 4, pp. 5929–5938, 2019.

\end{thebibliography}

\end{document}


\section{Results}
\label{sec:results}
Present a series of limits on the potential gains from optimization.

\begin{itemize}
	\item The loosest limit we can impose is the fact that static power will be around. If somehow we reduce the activity factor of some workload, we will get at best x\% improvement.
	\item that's not so meaningful, and a large amount of the activity factor comes simply from clock circuits and whatnot. So we extrapolate this base from our power data, and basically power(0)..power(4) is our feasible range. See diagramx
	\item Ignoring blocking IO the critical path of a program takes x cycles. Assume fixed, non-fixed times. Power per cycle @ various frequencies.
\end{itemize}

\todo{Diagramx: Plot showing how clock circuitry dominates functional circuitry. We perform the same amount of work per cycle, but we either do [1..4] instructions on 1 core or 1 instruction on [1..4] cores. Expect two lines diverging. As each core comes on stream we see the clock circuitry and other supporting stuff cause steps.}


\documentclass[12pt,a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
%\usepackage[english]{isodate}
%\usepackage[parfill]{parskip}

\usepackage{graphicx}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{newtxtext,newtxmath}
\usepackage{bm}
\usepackage{listings}
\usepackage{color}
\usepackage{url}
\usepackage{intmacros}
\usepackage{syntax}
\usepackage{manual}

\lstset{%
%    backgroundcolor=\color{yellow!20},%
    basicstyle=\ttfamily,%
    %numbers=left, numberstyle=\tiny, stepnumber=2, numbersep=5pt,%
    numbers=left, numberstyle=\tiny, numbersep=5pt,%
	frame=single,%
    }%

\lstset{emph={%
    let, var, init, at, wait, once, when, goto, then, end, prop, param%
    },emphstyle={\color{blue}\bfseries}%
}

\newcommand{\LNCS}[1]{\textit{LNCS}~#1}
\newcommand{\ENTCS}[1]{\textit{ENTCS}~#1}

\begin{document}

\title{HySIA Manual (version~0.1.1)}
\author{Daisuke Ishii \quad \texttt{dsksh@acm.org}}

\maketitle

\setlength{\grammarparsep}{8pt plus 1pt minus 1pt} % increase separation between rules
\setlength{\grammarindent}{8em} % increase separation between LHS/RHS

\emph{HySIA} is a reliable simulator and verifier for hybrid systems.
%
HySIA supports nonlinear \emph{hybrid automata} (HAs) whose ODEs, guards, and reset functions are specified with nonlinear expressions. It assumes a deterministic class of HA; a transition to another location happens whenever a guard condition holds.
%
Main functionalities of HySIA are as follows:

\textbf{Simulation}.
%
HySIA simulates an HA based on interval analysis;
it computes an overapproximation of a bounded trajectory that is composed of \emph{boxes} (i.e., closed interval vectors) and \emph{parallelotopes} (linear transformation of boxes).
%
Intensive use of interval analysis techniques distinguishes HySIA from other reachability analysis tools.
%
First, the simulation process carefully reduces \emph{wrapping effect} that can expand an enclosure interval. As a result, HySIA is able to simulate HA for more number of steps than other overapproximation-based tools; e.g., it can simulate a periodic bouncing ball for more than a thousand steps.
%
Second, HySIA assures the soundness of each interval computation, so that the resulting overapproximation is verified to contain a theoretical trajectory.
%This verification often fails, e.g., when an ODE is \emph{stiff} or when a trajectory and a guard are close to tangent.
%Thanks to this \emph{quasi-complete} manner, the simulation process of HySIA is decidable and performs efficiently.
%
See Reference~\cite{GI2016} for details of the underlying method.

\textbf{Monitoring}.
%
HySIA takes a temporal property as an input and monitors whether a simulated trajectory satisfies the property; otherwise, HySIA computes (an interval overapproximation of) a robustness signal for the property.
The verification process is based on a monitoring procedure of the \emph{signal temporal logic} (STL) formulas~\cite{Maler2004}, which is extended to handle overapproximation of trajectories.
%The soundness of interval computation is again utilized here to evaluate the logical negation against an approximated trajectory.
%
See Reference~\cite{Ishii2016IEICE} for more details.

\section{Short Tutorial}

HySIA provides a specification language for hybrid automata and temporal logic properties.
An HA is simulated and analyzed about a property using the \verb|hysia| command.
This section exemplifies the modeling, simulation, and verification of an HA using HySIA.

\subsection{Modeling Hybrid Automata}

Below is an example specification that describes a simple bouncing ball model (\texttt{bb.ha}):
\begin{lstlisting}
(* first part *)
let   g = 1
let   c = 0.9
let   pertb = [-1e-5, 1e-5]

(* second part *)
var   y, vy

init  Loc, 1+pertb, 0+pertb

at Loc wait vy, -g
      once (y, -vy) goto Loc then y, -c*vy
end

(* third part *)
param order = 20
param t_max = 100
param dump_interval = 0.1
\end{lstlisting}
%
The specification consists of three parts.
The first part (Lines~1--4) defines three constants \verb|g|, \verb|c|, and \verb|pertb| to be used in the second part.
The values of constants can be either real values or intervals.

The second part (Lines~6--13) describes an HA.
Line~7 declared the state variables \verb|y| and \verb|vy| of the HA, which are evaluated over timeline.
Line~9 describes the initial state as a comma-separated list of a location and values for each state variable.
Lines~11--13 defines a location named \verb|Loc|.
After the keyward \verb|wait|, the derivatives of the state variable are specified as $\frac{d}{dt}\mathtt{y} = \mathtt{vy}$ and $\frac{d}{dt}\mathtt{vy} = -\mathtt{g}$.
At Line~12, an inter-location transition is specified.
The tuple after \verb|once| tells that the guard condition for this transition is $\mathtt{y} = 0 ~\land~ \mathtt{vy} > 0$. (Only the left-hand side of the implicit form is described.)
Whenever the guard is satisfied, an execution will transit to the same location \verb|Loc| with a reset of the state variables as $\mathtt{y} := \mathtt{y}$ and $\mathtt{vy} := -\mathtt{c}\,\mathtt{vy}$.

The last part (Lines~15--18) configures some parameters of the simulator implementation, i.e., the order \verb|order| of Taylor coefficient expansion, the time horizon \verb|t_max| of a continuous state evolution, and \verb|dump_interval| that bounds the step size when computing a dumped data.

In general, an HA consists of multiple locations. The bouncing ball system can be modeled with two locations by modelling the second part as follows: %(\texttt{bb1.ha}):
\begin{lstlisting}
init  Fall, 1+pertb, -0+pertb

at Fall wait vy, -g
      once (y, vy) goto Rise then y, -c*vy
end

at Rise wait vy, -g
      once (vy, -y) goto Fall then y, vy
end
\end{lstlisting}

\subsection{Simulation of a Model}

Once a specification is prepared, a user can simulate the model for 10 transitions with the following command:
\begin{verbatim}
$ hysia bb.ha -n 10 -dump
\end{verbatim}
The option \verb|-dump| let HySIA to output the result of the simulation to the file \verb|pped.dat| in a JSON format.
The output data consists of a set of \emph{boxes} (interval vectors) that encloses the trajectories of the HA.
(In this example, the interval value \verb|pertb| allows perturbation of trajectories; these trajectories are enclosed within the boxes.)
The result can be visualized as shown in Figure~\ref{f:bb}.

\begin{figure}[t]
\centering
\includegraphics[width=0.8\linewidth]{bb.eps}
\caption{Dumped trajectory enclosure of the bouncing ball example}
\label{f:bb}
\end{figure}

Simulation and verification of HySIA are all computed with validated interval analysis.
Thus, a resulting interval enclosure of a system's state expands as longer the simulation length and more the \emph{wrapping effect} occurs.
However, thanks to the underlying \emph{parallelotope}-based simulation method for wrapping effect reduction, HySIA is able to simulate a large number of jumps for various HA.
Indeed, for the bouncing ball model, when we modify the parameter values as \verb|c = 1| and \verb|pertb = 0|, HySIA can simulate for more than a thousand steps.

As we have seen so far, HySIA allows models to involve some uncertainties derived by interval values.
HySIA computes (the overapproximation of) the reachable region with respect to the uncertainties.
However, here we may encounter a drawback of HySIA; a simulation is not always successful but may result in an error.
%
When we modify the value of \verb|pertb| to a slightly large interval \verb|[-1e-4, 1e-4]| and run a simulation, \verb|hysia| will output as follows:
\begin{verbatim}
$ hysia bb.ha -n 10 -dump
step 0 (0.000000 < inf) at Loc
step 1 (1.414043 < inf) at Loc
step 2 (3.959155 < inf) at Loc
step 3 (6.249714 < inf) at Loc
step 4 (8.310768 < inf) at Loc
step 5 (10.163552 < inf) at Loc
step 6 (11.809965 < inf) at Loc
libc++abi.dylib: terminating with uncaught exception of type std::
runtime_error: zero in the derivative
Abort trap: 6
\end{verbatim}
The simulation fails after the sixth jump. The output implies that HySIA fails in the detection of a discrete change because a state enclosure becomes too large.
In fact, in each detection, HySIA checks whether the orientation between the trajectory and the guard is regular enough so that it is sure that each trajectory within an enclosure satisfies the guard.

\subsection{Verification of STL Properties}

HySIA provides a function for verifying properties described in the \emph{signal temporal logic} (STL).
For example, we can add an STL property
\begin{lstlisting}
prop G[0,10] F[0,1] y-0.3
\end{lstlisting}
in the above mentioned model.
(It should be added in between the second and third parts.)
The property intuitively claims that, for the duration of $10$ time units, the height of the ball (\verb|y|) goes beyond $0.3$ (\verb|y-0.3| is interpreted as $\texttt{y} - 0.3 > 0$) within every duration of $1$ time unit.

The verification can be done with the following command:
\begin{verbatim}
$ hysia bb.ha -a
\end{verbatim}
The \verb|hysia| command calculates the \emph{necessary} simulation length for the verification (in this case, 11 time units), performs a simulation, and evaluates the satisfiability of this property.
Accordingly, the verification succeeds with the following output:
\begin{verbatim}
$ hysia bb.ha -a
step 0 (0.000000 < 11.000000) at Loc
step 1 (1.414196 < 11.000000) at Loc
step 2 (3.959734 < 11.000000) at Loc
step 3 (6.250722 < 11.000000) at Loc
step 4 (8.312608 < 11.000000) at Loc
step 5 (10.168295 < 11.000000) at Loc
true, 0.036239
\end{verbatim}

\section{Getting the Tool}

The HySIA tool can be obtained and used in three ways.

\subsection{Web Demonstration}

A web demonstration site is available at:
\begin{quote}
\url{http://bit.ly/hysia}
\end{quote}
%
Through a web browser, you can access a GUI, load the basic examples, modify the HA/STL specification, simulate, and verify the specification.

\subsection{Docker Image}

For those who are familier with \emph{Docker},\footnote{\url{https://www.docker.com/}} Docker containers are available at:
\begin{quote}
\url{https://hub.docker.com/r/dsksh/}
\end{quote}

The image \verb|dsksh/hysia-web| contains the server program for the web demonstration. A container can be launched by:
\begin{verbatim}
$ docker run -p 8080:80 dsksh/hysia-web
\end{verbatim}
Then, the server will be available at:
\begin{quote}
\url{http://localhost:8080}
\end{quote}

\subsection{Source Distribution}

The HySIA source code is distributed via GitHub:
\begin{quote}
\url{https://github.com/dsksh/hysia}.
\end{quote}

\subsubsection*{Requirements}

In addition to a standard UNIX-like environment, the following softwares are required to compile HySIA:
\begin{itemize}
    \item \emph{C/C++ compiler}.
        We have compiled with both \verb|gcc| (versions~4.7.4–4.8.4) and \verb|clang| (Apple LLVM version~8).
        %
    \item \emph{OCaml compiler}.
        We have tested with versions~3.12–4.02.
        %
    \item \emph{CAPD library}.\footnote{http://capd.ii.uj.edu.pl/}
        HySIA is built on an old release of the CAPD-DynSys 3.0 library, which was distributed around 2014, and is not available on the official site.
        A (slightly modified) source package is available at:
        \begin{quote}\small
        \hspace{-4em}
        \url{https://www.dropbox.com/s/3uf7t2nsizfebno/capdDynSys-201406.zip}
        \end{quote}
        %
    \item \emph{Boost library}.\footnote{http://www.boost.org/}
        HySIA uses \verb|shared_ptr|.
        %
    \item \emph{OUnit}.\footnote{http://ounit.forge.ocamlcore.org/} Optional for test cases compilation.
    \item \emph{Eliom}.\footnote{http://ocsigen.org/eliom/} Optional for building the web application.
\end{itemize}

\subsubsection*{Build}

In the root directory of the source code, HySIA can be compiled with:
\begin{verbatim}
$ ./configure; make
\end{verbatim}

When compilation succeeds, the program file
\begin{verbatim}
src_ocaml/hss.opt
\end{verbatim}
is generated.

\section{Examples}

\subsection{Simple Rotation System}

TBD.

\subsection{Bouncing Planet}

TBD.

\section{Reference Manual}

\subsection{Command-Line Tool}

The basic syntax for the command-line execution is as follows:
\begin{grammar}
<command> ::= `hysia' <options> <filename>
\end{grammar}
%
The \texttt{hysia} command accepts the following options:
\begin{description}
\item[\texttt{-h}, \texttt{-help}, or \texttt{--help}] Displays a summary of the options accepted by the command.
\item[\texttt{-n}] Specifies the number steps to simulate (default is $\infty$).
\item[\texttt{-t}] Specifies the max simulation time (default is $\infty$).
\item[\texttt{-a}] Decides the simulation length automatically from the STL property.
\item[\texttt{-g}] Sets the debug flag.
\item[\texttt{-dump}] Activates dumping plot to the file ``pped.dat.''
\item[\texttt{-cm_thres}] Sets the threshold for character matrix selection.
\end{description}

The option \texttt{-cm_thres} specifies the character matrix $B$ to be used in the parallelotope method (see the corresponding publication for the detail). It is selected as follows:
\begin{itemize}
	\item $-1$: $B := (\mathrm{mid}\J) A$.
	\item $0$: $B := I$ (i.e., identity matrix).
	\item $1$: $B := \mathrm{orthogonalize}((\mathrm{mid}\J) A)$.
	\item $n > 1$ (default):
		\[
			B := \begin{cases}
				(\mathrm{mid}\J) A & \text{if $\kappa((\mathrm{mid}\J) A) < n$} \\
				\mathrm{orthogonalize}((\mathrm{mid}\J) A) & \text{otherwise.}
			\end{cases}
		\]
\end{itemize}

\subsection{Solving Parameters}

\begin{description}
\item[\texttt{order}] Order of Taylor expansion.
\item[\texttt{t_max}] Max time horizon assumed in the simulation of each step.
\item[\texttt{h_min}] Min time CAPD integration can take.
\item[\texttt{epsilon}] Specifies the precision of the event detection.
\item[\texttt{dump_interval}] Sets the precision of the dumped flowpipe data.
\item[\texttt{delta}] Parameter for the box inflation process.
\item[\texttt{tau}] Parameter for the box inflation process.
\item[\texttt{cm_thres}] Parameter for the character matrix selection.
\end{description}

\subsection{Specification Language}

This section describes the grammar of the specification language of HySIA.
A specification consists of the definition of a hybrid automaton, an STL formula, and solving parameter configurations.

\subsubsection{Lexical conventions}

The lexical class of digits, letters, and identifiers is the following:
\begin{grammar}
<digit> ::= [`0'--`9']

<letter> ::= [`a'--`z' `A'--`Z']

<id> ::= <letter> (<digit> | <letter> | `_')*
\end{grammar}

The syntax of various numeral expressions is as follows:
\begin{grammar}
<integer> ::= <digit>+

<float> ::=
%<digit>+ ~~|~~ <digit>+ `.' <digit>*
<digit>+ ~ (`.' <digit>*)? ~ ( (`e'|`E') (`+'|`-')? <digit>+ )?
%\alt <digit>+ `.' <digit>* (`e'|`E')  <digit>+
%\alt <digit>+ (`e'|`E') <digit>+
%\alt <digit>+ (`e'|`E') (`+'|`-') <digit>+

<float-pn> ::= <float> ~~|~~ `-' <float>

<interval> ::= <interval-noun> ~|~ <float-pn>

<interval-noun> ::= `(' <float-pn> `,' <float-pn> `)'

<interval-list> ::= <interval> <interval-list-rest>
\alt `(' <interval> <interval-list-rest> `)'

<interval-list-rest> ::= `,' <interval> <interval-list-rest> ~~|~~ <empty>
\end{grammar}

\noindent
\textbf{Comments.}
Comments are either enclosed between \texttt{(*} and \texttt{*)} (can be nested) or prefixed with \texttt{\#}.

\subsubsection{Toplevel syntax}

The syntax for the toplevel of specifications is the following:
\begin{grammar}
<specification> ::= <statement>+ <property>? <solver-param>*

<statement> ::=
`let' <id> `=' <interval>
\alt `let' <id> `=' `R' <float>
\alt `var' <var-list>
\alt `init' <expr-list>
\alt `at' <id> <flow> <invariant>? <edge>* `end'

<property> ::= `prop' <stl-formula>

<solver-param> ::= `param' <id> `=' <float-pn>

<flow> ::= `wait' <expr-list>

<invariant> ::= `inv' <expr-list>

<edge> ::= (`when' | `once') `(' <expr> `,' <expr-list> `)' `goto' <id> `then' <expr-list>
\end{grammar}

\subsubsection{Expressions}

The operators and function application in expressions have the priorities and associativities as shown in the table below (from lowest to greatest priority):
\begin{table}[ht]
	\centering
	%\caption{\label{t:operators} Terms}
	%\small
    \begin{tabular}{|l|l|} \hline
		construct & associativity \\
		\hline
		`\texttt{+}', `\texttt{-}' & left \\
		`\texttt{*}', `\texttt{/}' & left \\
		function application & left \\
		`\texttt{\^}', `\texttt{-}' (unary) & --- \\
		\hline
	\end{tabular}
\end{table}

The syntax for expressions is the following:
\begin{grammar}
<expr> ::= <expr> `+' <expr> ~~|~~ <expr> `-' <expr>
\alt <expr> `*' <expr> ~~|~~ <expr> `/' <expr>
\alt `-' <expr> ~~|~~ <expr> `^' <integer>
\alt <function> <expr>
\alt <id> ~~|~~ <interval>
\alt `(' <expr> `)'

<function> ::= `sqrt' | `exp' | `log' | `sin' | `cos' | `atan' | `asin' | `acos'

<expr-list> ::= <expr> <expr-list-rest> ~~|~~ `(' <expr> <expr-list-rest> `)'

<expr-list-rest> ::= `,' <expr> <expr-list-rest> ~|~ <empty>
\end{grammar}

\subsubsection{STL formulae}

The operators in formulae have the following priorities and associativities:
\begin{table}[ht]
	\centering
	%\caption{\label{t:operators} Terms}
	%\small
    \begin{tabular}{|l|l|} \hline
		construct & associativity \\
		\hline
		`\texttt{->}' & right \\
		`\texttt{|}' & right \\
		`\texttt{\&}' & right \\
		`\texttt{U} [$\cdot$,$\cdot$]' & right \\
		`\texttt{G} [$\cdot$,$\cdot$]', `\texttt{F} [$\cdot$,$\cdot$]' & --- \\
		`\texttt{!}' & --- \\
		\hline
	\end{tabular}
\end{table}

The syntax for formulae is the following:
\begin{grammar}
<stl-formula> ::= `true' ~~|~~ `false'
\alt <id> ~~|~~ <expr>
\alt `!' <stl-formula>
\alt <stl-formula> (`&'|`|'|`->') <stl-formula>
\alt (`F'|`G') <noun-interval> <stl-formula>
\alt <stl-formula> `U' <noun-interval> <stl-formula>
\alt `(' <stl-formula> `)'
\end{grammar}

\begin{thebibliography}{99}
\bibitem{Donze2013}
    A. Donz\'{e}, T. Ferr\`{e}re, O. Maler:
    {Efficient Robust Monitoring for STL},
    Proc. of CAV, pp.~264--279, \LNCS{8044}, 2013.

\bibitem{GI2016}
    A. Goldsztejn, D. Ishii:
    {A Parallelotope Method for Hybrid System Simulation},
    Reliable Computing, 23:163--185, 2016.

\bibitem{Ishii2016IEICE}
    D. Ishii, N. Yonezaki, A. Goldsztejn:
    {Monitoring Temporal Properties using Interval Analysis}.
    IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences,
    E99-A, 2016.

\bibitem{Ishii2015-NSV}
    D. Ishii, N. Yonezaki, A. Goldsztejn:
    {Monitoring Bounded LTL Properties Using Interval Analysis}.
    Proc. of 8th International Workshop on Numerical Software Verification (NSV),
    \ENTCS{317}, pp.~85--100, 2015.

\bibitem{Maler2004}
    O. Maler and D. Nickovic:
    Monitoring Temporal Properties of Continuous Signals,
    Proc. of FORMATS, pp.~152--166, \LNCS{3253}, 2004.
\end{thebibliography}

\end{document}


\documentclass{article}

\usepackage{verbatim}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{array}
\usepackage{fullpage}

\author{B.~Barras}
\title{An introduction to syntax of Coq V8}

%% Le _ est un caractère normal
\catcode`\_=13
\let\subscr=_
\def_{\ifmmode\sb\else\subscr\fi}

\def\NT#1{\langle\textit{#1}\rangle}
\def\NTL#1#2{\langle\textit{#1}\rangle_{#2}}
\def\TERM#1{\textsf{\bf #1}}

\newenvironment{transbox}
  {\begin{center}\tt\begin{tabular}{l|ll} \hfil\textrm{V7} & \hfil\textrm{V8} \\ \hline}
  {\end{tabular}\end{center}}
\def\TRANS#1#2
  {\begin{tabular}[t]{@{}l@{}}#1\end{tabular} &
   \begin{tabular}[t]{@{}l@{}}#2\end{tabular} \\}
\def\TRANSCOM#1#2#3
  {\begin{tabular}[t]{@{}l@{}}#1\end{tabular} &
   \begin{tabular}[t]{@{}l@{}}#2\end{tabular} & #3 \\}

\begin{document}

\maketitle

The goal of this document is to introduce by example to the new syntax of
Coq. It is strongly recommended to read first the definition of the new
syntax, but this document should also be useful for the eager user who wants
to start with the new syntax quickly.

\section{Changes in lexical conventions w.r.t. V7}

\subsection{Identifiers}

The lexical conventions changed: \TERM{_} is not a regular identifier
anymore. It is used in terms as a placeholder for subterms to be inferred
at type-checking, and in patterns as a non-binding variable.

Furthermore, only letters (unicode letters), digits, single quotes and
_ are allowed after the first character.

\subsection{Quoted string}

Quoted strings are used typically to give a filename (which may not
be a regular identifier). As before they are written between double
quotes ("). Unlike for V7, there is no escape character: characters
are written normaly but the double quote which is doubled.

\section{Main changes in terms w.r.t. V7}

\subsection{Precedence of application}

In the new syntax, parentheses are not really part of the syntax of
application. The precedence of application (10) is tighter than all
prefix and infix notations. It makes it possible to remove parentheses
in many contexts.

\begin{transbox}
\TRANS{(A x)->(f x)=(g y)}{A x -> f x = g y}
\TRANS{(f [x]x)}{f (fun x => x)}
\end{transbox}

\subsection{Arithmetics and scopes}

The specialized notation for \TERM{Z} and \TERM{R} (introduced by
symbols \TERM{`} and \TERM{``}) have disappeared. They have been
replaced by the general notion of scope.

\begin{center}
\begin{tabular}{l|l|l}
type & scope name & delimiter \\
\hline
types & type_scope & \TERM{T} \\
\TERM{bool} & bool_scope & \\
\TERM{nat} & nat_scope & \TERM{nat} \\
\TERM{Z} & Z_scope & \TERM{Z} \\
\TERM{R} & R_scope & \TERM{R} \\
\TERM{positive} & positive_scope & \TERM{P}
\end{tabular}
\end{center}

In order to use notations of arithmetics on \TERM{Z}, its scope must be opened with command \verb+Open Scope Z_scope.+ Another possibility is using the scope change notation (\TERM{\%}). The latter notation is to be used when notations of several scopes appear in the same expression.

In examples below, scope changes are not needed if the appropriate scope
has been opened. Scope nat_scope is opened in the initial state of Coq.
\begin{transbox}
\TRANSCOM{`0+x=x+0`}{0+x=x+0}{\textrm{Z_scope}}
\TRANSCOM{``0 + [if b then ``1`` else ``2``]``}{0 + if b then 1 else 2}{\textrm{R_scope}}
\TRANSCOM{(0)}{0}{\textrm{nat_scope}}
\end{transbox}

Below is a table that tells which notation is available in which
scope. The relative precedences and associativity of operators is the
same as in usual mathematics. See the reference manual for more
details. However, it is important to remember that unlike V7, the type
operators for product and sum are left associative, in order not to
clash with arithmetic operators.

\begin{center}
\begin{tabular}{l|l}
scope & notations \\
\hline
nat_scope & $+ ~- ~* ~< ~\leq ~> ~\geq$ \\
Z_scope & $+ ~- ~* ~/ ~\TERM{mod} ~< ~\leq ~> ~\geq ~?=$ \\
R_scope & $+ ~- ~* ~/ ~< ~\leq ~> ~\geq$ \\
type_scope & $* ~+$ \\
bool_scope & $\TERM{\&\&} ~\TERM{$||$} ~\TERM{-}$ \\
list_scope & $\TERM{::} ~\TERM{++}$
\end{tabular}
\end{center}
(Note: $\leq$ is written \TERM{$<=$})

\subsection{Notation for implicit arguments}

The explicitation of arguments is closer to the \emph{bindings} notation in
tactics. Argument positions follow the argument names of the head constant.

\begin{transbox}
\TRANS{f 1!t1 2!t2}{f (x:=t1) (y:=t2)}
\TRANS{!f t1 t2}{@f t1 t2}
\end{transbox}

\subsection{Universal quantification}

The universal quantification and dependent product types are now
materialized with the \TERM{forall} keyword before the binders and a
comma after the binders.

The syntax of binders also changed significantly. A binder can simply be
a name when its type can be inferred. In other cases, the name and the type
of the variable are put between parentheses. When several consecutive
variables have the same type, they can be grouped. Finally, if all variables
have the same type parentheses can be omitted.

\begin{transbox}
\TRANS{(x:A)B}{forall (x:~A), B ~~\textrm{or}~~ forall x:~A, B}
\TRANS{(x,y:nat)P}{forall (x y :~nat), P ~~\textrm{or}~~ forall x y :~nat, P}
\TRANS{(x,y:nat;z:A)P}{forall (x y :~nat) (z:A), P}
\TRANS{(x,y,z,t:?)P}{forall x y z t, P}
\TRANS{(x,y:nat;z:?)P}{forall (x y :~nat) z, P}
\end{transbox}

\subsection{Abstraction}

The notation for $\lambda$-abstraction follows that of universal
quantification. The binders are surrounded by keyword \TERM{fun}
and $\Rightarrow$ (\verb+=>+ in ascii).

\begin{transbox}
\TRANS{[x,y:nat; z](f a b c)}{fun (x y:nat) z => f a b c}
\end{transbox}

\subsection{Pattern-matching}

Beside the usage of the keyword pair \TERM{match}/\TERM{with} instead of
\TERM{Cases}/\TERM{of}, the main change is the notation for the type of
branches and return type. It is no longer written between \TERM{$<$ $>$} before
the \TERM{Cases} keyword, but interleaved with the destructured objects.

The idea is that for each destructured object, one may specify a variable
name to tell how the branches types depend on this destructured objects (case
of a dependent elimination), and also how they depend on the value of the
arguments of the inductive type of the destructured objects. The type of
branches is then given after the keyword \TERM{return}, unless it can be
inferred.

Moreover, when the destructured object is a variable, one may use this
variable in the return type.

\begin{transbox}
\TRANS{Cases n of\\~~ O => O \\| (S k) => (1) end}{match n with\\~~ 0 => 0 \\| (S k) => 1 end}
\TRANS{Cases m n of \\~~0 0 => t \\| ... end}{match m, n with \\~~0, 0 => t \\| .. end}
\TRANS{<[n:nat](P n)>Cases T of ... end}{match T as n return P n with ... end}
\TRANS{<[n:nat][p:(even n)]\~{}(odd n)>Cases p of\\~~ ... \\end}{match p in even n return \~{} odd n with\\~~ ...\\end}
\end{transbox}

\subsection{Fixpoints and cofixpoints}

An easier syntax for non-mutual fixpoints is provided, making it very close
to the usual notation for non-recursive functions. The decreasing argument
is now indicated by an annotation between curly braces, regardless of the
binders grouping. The annotation can be omitted if the binders introduce only
one variable. The type of the result can be omitted if inferable.

\begin{transbox}
\TRANS{Fix plus\{plus [n:nat] : nat -> nat :=\\~~ [m]...\}}{fix plus (n m:nat) \{struct n\}: nat := ...}
\TRANS{Fix fact\{fact [n:nat]: nat :=\\
~~Cases n of\\~~~~ O => (1) \\~~| (S k) => (mult n (fact k)) end\}}{fix fact
  (n:nat) :=\\
~~match n with \\~~~~0 => 1 \\~~| (S k) => n * fact k end}
\end{transbox}

There is a syntactic sugar for mutual fixpoints associated to a local
definition:

\begin{transbox}
\TRANS{let f := Fix f \{f [x:A] : T := M\} in\\(g (f y))}{let fix f (x:A) : T := M in\\g (f x)}
\end{transbox}

The same applies to cofixpoints, annotations are not allowed in that case.

\subsection{Notation for type cast}

\begin{transbox}
\TRANS{O :: nat}{0 : nat}
\end{transbox}

\section{Main changes in tactics w.r.t. V7}

The main change is that all tactic names are lowercase. This also holds for
Ltac keywords.

\subsection{Ltac}

Definitions of macros are introduced by \TERM{Ltac} instead of
\TERM{Tactic Definition}, \TERM{Meta Definition} or \TERM{Recursive
Definition}.

Rules of a match command are not between square brackets anymore.

Context (understand a term with a placeholder) instantiation \TERM{inst}
became \TERM{context}. Syntax is unified with subterm matching.

\begin{transbox}
\TRANS{match t with [C[x=y]] => inst C[y=x]}{match t with context C[x=y] => context C[y=x]}
\end{transbox}

\subsection{Named arguments of theorems}

\begin{transbox}
\TRANS{Apply thm with x:=t 1:=u}{apply thm with (x:=t) (1:=u)}
\end{transbox}

\subsection{Occurrences}

To avoid ambiguity between a numeric literal and the optionnal
occurrence numbers of this term, the occurrence numbers are put after
the term itself. This applies to tactic \TERM{pattern} and also
\TERM{unfold}
\begin{transbox}
\TRANS{Pattern 1 2 (f x) 3 4 d y z}{pattern (f x at 1 2) (d at 3 4) y z}
\end{transbox}

\section{Main changes in vernacular commands w.r.t. V7}

\subsection{Binders}

The binders of vernacular commands changed in the same way as those of
fixpoints. This also holds for parameters of inductive definitions.

\begin{transbox}
\TRANS{Definition x [a:A] : T := M}{Definition x (a:A) : T := M}
\TRANS{Inductive and [A,B:Prop]: Prop := \\~~conj : A->B->(and A B)}%
      {Inductive and (A B:Prop): Prop := \\~~conj : A -> B -> and A B}
\end{transbox}

\subsection{Hints}

The syntax of \emph{extern} hints changed: the pattern and the tactic
to be applied are separated by a \TERM{$\Rightarrow$}.
\begin{transbox}
\TRANS{Hint Extern 4 (toto ?) Apply lemma}{Hint Extern 4 (toto _) => apply lemma}
\end{transbox}

\end{document}


\section{Results}
In table \ref{tab:mse} the mean squared error (MSE) of all trained models is shown. Clearly these errors are quite low, but in the context of our regression model the error is naturally quite low, because the mean squared error is not a relative value but an absolute value and depends on the value range which, in our case, is small.

\begin{table}[h]
	\resizebox{.65\columnwidth}{!}{%
		\centering
		\begin{tabular}{|l|c|}
			\hline
			Model & MSE \\
			\hline
			ETS & 0.005806121997238007 \\
			HGB & 0.006537057262386852 \\
			LGB & 0.006659725937704261 \\
			XGB & 0.009520274940766137 \\
			\hline
		\end{tabular}
	}
	\caption{Mean squared error of trained models}
	\label{tab:mse}
\end{table}

In the figure \ref{fig:predictions} the orange lines represent our prediction and the blue lines represent the ground truth. With these figures it is quite clear why the mean squared error value needs to be even lower.

\begin{figure*}[bt]
	\begin{subfigure}{.24\textwidth}
		\centering
		\includegraphics[width=\textwidth]{XGB_results}
		\caption{XGBoost}
	\end{subfigure}
	\begin{subfigure}{.24\textwidth}
		\centering
		\includegraphics[width=\textwidth]{HGB_results}
		\caption{HistoryGradientBoosting}
	\end{subfigure}
	\begin{subfigure}{.24\textwidth}
		\centering
		\includegraphics[width=\textwidth]{LGB_results}
		\caption{LightGBM}
	\end{subfigure}
	\begin{subfigure}{.24\textwidth}
		\centering
		\includegraphics[width=\textwidth]{ETS_results}
		\caption{ExtraTrees}
	\end{subfigure}
	\caption{Comparison of concatenated predictions (orange) to ground truth (blue).}
	\label{fig:predictions}
\end{figure*}

Our results show that the regression is not able to differentiate between below and above the legal limit with the features extracted.

\subsection{Discussion}
We had a deadline to submit our results and in the mean time, the project continued to scale up as we progressed through different stages.
Thus, we had to compromise a bit of quality to gain speed and finish our project on time. These difficulties and compromises include the feature extraction where we could use more promising signal processing approaches to obtain better results.
More detailed description of this matter is further explained in the section \ref{sec:futurework}.

In the case of an application like this to become publicly accessible to users, there comes great responsibility with the data collection of the owners.
For one, this is extremely sensible data that could be trained to recognize even further information about an individual.
Another point is that in case the TAC level data gets leaked or sold to a third party, it could be abused to target the user with relevant ads or give drunk users higher transportation fares.
And finally, the app itself could produce defective detections, like a false negative resulting in irresponsible behavior while depending on a feedback from the application as an excuse to drive, for example.

A direct comparison of the results of our trained regression model to the model proposed in \cite{DBLP:conf/ijcai/KillianPNMC19} is not possible, because not only did they not specify, on which chunks of the dataset they tested their trained models, they also performed a classification. And our regression has a more precise output and that results in worse metrics for our model, but does not necessarily represent the performance, because a threshold could be undershot by a tiny amount and would not necessarily mean a worse result, just a more precise one.

In addition to the aforementioned points, because this paper was created during a university course, many of our expectations couldn't be met because of strict time constraints.
If more time was available we could have increased our model performance by using details we discussed in the beginning of this section and chapter \ref{sec:futurework}.

\subsection{Conclusion}
Overall this project was a huge learning experience for all of us, and we learned a lot about data analysis. Additionally, we learned how fast a project can grow in dimension by adding more complexity and data. Furthermore we can conclude that in shallow machine learning algorithms the feature extraction has a huge impact on the model performance.

Our results were not meeting our expectations, but were of course limited by the time constraints of a university project and lacking the computational power to train more  models with the proposed changes. Otherwise, we see our whole pipeline of data analysis, cleaning and model tuning as a huge success.


\documentclass[11pt, oneside]{article}   	% use "amsart" instead of "article" for AMSLaTeX format
\usepackage{geometry}                		% See geometry.pdf to learn the layout options. There are lots.
\geometry{letterpaper}                   		% ... or a4paper or a5paper or ...
%\geometry{landscape}                		% Activate for rotated page geometry
%\usepackage[parfill]{parskip}    		% Activate to begin paragraphs with an empty line rather than an indent
\usepackage{graphicx}				% Use pdf, png, jpg, or eps§ with pdflatex; use eps in DVI mode
								% TeX will automatically convert eps --> pdf in pdflatex
\usepackage{amssymb}
\usepackage{amsmath}
\PassOptionsToPackage{hyphens}{url} % url is loaded by hyperref
\usepackage[unicode=true]{hyperref}
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}
\usepackage{lineno}
\linenumbers
% math typesetting
\let\vec\mathbf
\newcommand{\matrx}[1]{{\left[ \stackrel{}{#1}\right]}}

%SetFonts

%SetFonts

\pagestyle{headings}
\markright{Yoav Ram, \today \; v2\hfill}

\begin{document}
Here I describe how I analyzed the stability of a modifier allele $m$ with vertical transmission rate $\rho$ to invasion by a modifier $M$ with a vertical transmission rate $P$, as described in the main text, eqs. 34-35, in environmental regime $AkBl$.
\\
The analysis is similar to that used in the main text to analyze stability in $A1B1$, but it uses computation because the cases where $k>1$ or $l>1$ cannot be analyzed using closed form expressions.
\\
The analysis includes the following steps for a fixed $W, w, k,$ and $l$:
\begin{enumerate}

\item Find the stable frequency vector $\vec{x^*}$ without invader $M$, i.e. $x^*_3=x^*_4=0$. This is done by minimizing the expression $|x_{k+l} - x_{0}|$ where $x_{t}$ is defined in eq. 9 of the main text. The minimization is done by iterating the recurrence until it converges, i.e. until the difference is smaller than machine precision (roughly $10^-8$ when subtracting similar small numbers).

\item Define $F_A(x)$ by eq. 35 of main text with $w_A=W$ and $w_B=w$ ($W>w$), and similarly $F_B(x)$ with $w_B=W$ and $w_A=w$.

\item Define, similar to eq. 37 of main text,
\begin{equation}
\vec{F} = \underbrace{F_A \circ \ldots \circ F_A}_{k \text{ times}} \circ
	\underbrace{F_B \circ \ldots \circ F_B}_{l \text{ times}}.
\label{eq:F}\end{equation}

\item Calculate the linear approximation $\vec{J}$ of $\vec{F}$  near $\vec{x^*}$, that is, the Jacobian matrix of $\vec{F}$ at $\vec{x^*}$:
\begin{equation}
\vec{J}_{ij} = \frac{\partial F_i}{\partial x_j}.
\label{eq:jacobian}\end{equation}

\item Define $\vec{L}=\vec{L_{ex}}$ as in eqs. 40-45 in the main text such that
\begin{equation}
\vec{J} = \begin{pmatrix}
\vec{L_{in}} & * \\
* & \vec{L_{ex}}
\end{pmatrix}.
\end{equation}

\item Calculate the eigenvalues $\lambda_1 > \lambda_2$ of $\vec{L}$; by the Perron-Frobenius theorem, the leading eigenvalue $\lambda_1$ is real and positive. Denote $\lambda_1(\rho, P)$ the resulting leading eigenvalue with resident rate $\rho$ and invader rate $P$. Note that
\begin{equation}
\lambda_1(\rho, \rho) = 1,
\end{equation}
for any $\rho \in (0,1)$.

\item The evolutionary stable rate $\rho^*$ is defined as stable to invasion, that is,
\begin{equation}
\lambda_1(\rho^*,\rho^* \pm dP) < 1 = \lambda_1(\rho^*,\rho^*)
\end{equation}
for a small enough $dP>0$.
Therefore,
\begin{equation}
\frac{\partial \lambda_1}{\partial P}\big(\rho^*,\rho^*\big) = 0.
\label{eq:dλdP=0} \end{equation}

\item We use Brent's root-finding method (Brent, 1971) to find $\rho^*$ that satisfies \autoref{eq:dλdP=0}. If, due to numerical instability of the process, we have
\begin{equation}
\frac{\partial \lambda_1}{\partial P}\big(0,0\big) \cdot \frac{\partial \lambda_1}{\partial P}\big(1,1) > 0,
\end{equation}
i.e., the partial derivative sign is identical at $\rho=P=0$ and $\rho=P=1$, then we cannot use Brent's method. In these cases we assume that the partial derivative doesn't have a root in $(0,1)$ and we determine the stable rate $\rho^*$ by the rule
\begin{equation}
\rho^* = \begin{cases}
0, \quad \text{if } \frac{\partial \lambda_1}{\partial P}\big(0,0\big) \le 0, \\
1, \quad \text{if } \frac{\partial \lambda_1}{\partial P}\big(0,0\big) > 0.
\end{cases}
\end{equation}

\end{enumerate}

\paragraph{Results}
\autoref{fig:AkBk_stable_modifier_w_0.1}, \autoref{fig:AkBk_stable_modifier_w_0.5} and \autoref{fig:AkBk_stable_modifier_w_0.9}, show the leading eigenvalue $\lambda_1$ of the external stability matrix $\vec{L}$ for different choices of environmental cycles $AkBk$, resident rate $\rho$ and invader rate $P$. The arrows show a series of invading modifiers generated using Algorithm~\autoref{algorithm} (this algorithm cannot be efficiently used to determine the stable rate as it requires fine tuning of the meta-parameter $\eta$).

The leading eigenvalue calculation is good for small $k$ but for large $k$ and especially for $w=0.1$ the calculation is unstable.
This is probably because when the environment is constant for a long period of time, the frequencies are close to the boundaries (i.e. 0 and 1) and the gradient just after the environment changes is very sharp.
Therefore, the calculation of $\vec{J}$ is unstable.
Also, because most of the time the frequencies don't change by much, the elements of $\vec{L}$ are small, leading to unstable calculation of the leading eigenvalue.
\\

\begin{algorithm}
% https://en.wikibooks.org/wiki/LaTeX/Algorithms
\caption{Modifier gradient ascent algorithm}\label{algorithm}
\begin{algorithmic}[1]
\State $\eta > 0$ is a step size
\State $\rho_{0} = 0.5$ is a starting condition
\State $t = 0$ is an iteration counter
\While {$\frac{|\rho_{t} - \rho_{t-1}|}{\rho_{t-1}} < \epsilon \;\text{and}\; \eta > \eta_0$}
	\State $P_{t} \gets \rho_{t} + \eta \frac{\partial \lambda_1(\rho_{t}, \rho_{t})}{\partial P}$
	\If {$\lambda_1(\rho_{t}, P_{t})$}
	\State {$\rho_{t+1} \gets P_{t}$}
	\Else
	\State {$\eta \gets \eta / 2$}
	\EndIf
	\State $t \gets t+1$
\EndWhile
\State $\rho^* \gets \rho_t$
\end{algorithmic}
\end{algorithm}

\begin{figure}[htbp]
  \centering
    \includegraphics[width=\linewidth]{../figures/AkBk_stable_optimal_rate.pdf}
  \caption{Fitness "optimal" and evolutionary stable vertical transmission rate in $AkBk$ selection regime. \textbf{(A)} The vertical transmission rate $\hat{rho}$ that maximized the geometric average of the population mean fitness is zero, i.e. complete oblique transmission, when selection cycles quickly between favoring phenotype $A$ and $B$, and then abruptly transitions to ~0.2, followed by a slow decrease (see Fig. SX for details on the abrupt transition).
  \textbf{(B)} The evolutionary stable rate $\rho^*$ which cannot be invaded by modifiers with either higher or lower vertical transmission rate $P$, rapidly increases from zero when selection cycles are short ($k=1$ or $2$) to roughly 1 when the environment cycles are longer. The stable rate was found by finding the root of $\frac{\partial \lambda_1}{\partial P}$, the leading eigenvalue of the external stability matrix w.r.t to the rate of the invading modifier (see Supplementary Information for details). The dashed lines shows $1-\frac{1}{k-1}$, which seems to fit the values for $w=0.5$ (Carja et al. 2011). The values for $w=0.1$ (blue) could not be calculated for $k>21$ due to numerical instability when selection is strong and the duration between selection fluctuations is long.
  In all cases, $W=1$}
  \label{fig:AkBk_stable_modifier_w_0.1}
\end{figure}

\begin{figure}[htbp]
  \centering
    \includegraphics[width=\linewidth]{{../figures/AkBk_stable_modifier_w_0.1}.pdf}
  \caption{The leading eigenvalue of the external stability matrix in $AkBk$ with selection. The colors show $\lambda_1(W, w, \rho, P, k, l)$, the leading eigenvalue of the external stability matrix for combinations of $\rho$ (y-axis) and $P$ (x-axis), the vertical transmission rates of resident and invader modifiers, respectively. Purple shows values lower than 1, leading to stability of $\rho$; white equals to 1; green larger than 1, leading to invasion of $P$.
  The arrows show an "evolutionary" path $\rho_{0} \to \rho_{1} \to \ldots \to \rho^*$: at each step, a modifier with rate $P_t$ invades the resident modifier with rate $\rho_t$ and becomes the resident modifier $\rho_{t+1}$, until consecutive invaders have similar rates (see Algorithm~\autoref{algorithm}).
  Here, $W=1$ and $w=0.1$.}
  \label{fig:AkBk_stable_modifier_w_0.1}
\end{figure}

\begin{figure}[htbp]
  \centering
    \includegraphics[width=\linewidth]{{../figures/AkBk_stable_modifier_w_0.5}.pdf}
  \caption{The leading eigenvalue of the external stability matrix in $AkBk$ with intermediate selection.
  Same as~\autoref{fig:AkBk_stable_modifier_w_0.1} except here, $W=1$ and $w=0.5$.}
  \label{fig:AkBk_stable_modifier_w_0.5}
\end{figure}

\begin{figure}[htbp]
  \centering
    \includegraphics[width=\linewidth]{{../figures/AkBk_stable_modifier_w_0.9}.pdf}
  \caption{The leading eigenvalue of the external stability matrix in $AkBk$ with intermediate selection.
  Same as~\autoref{fig:AkBk_stable_modifier_w_0.1} except here, $W=1$ and $w=0.9$.}
  \label{fig:AkBk_stable_modifier_w_0.9}
\end{figure}

\paragraph{Note}
Crucially, the Jacobian $\vec{J}$ in~\autoref{eq:jacobian} can be calculated using \textit{automatic differentiation} with good precision from a function that iteratively calculates $\vec{F}$ according to~\autoref{eq:F}. Similarly, the partial derivative $\frac{\partial \lambda_1}{\partial P}$ in Algorithm~\autoref{algorithm} can be calculated from a function that calculates $\lambda_1$ using simple arithmetic operations, as the eigenvalues can be found from a quadratic formula.
\\
Note that \textit{automatic differentiation} does not mean \textit{symbolic} or \textit{numerical differentiation}, which can lead to inefficient or inaccurate estimation of $\vec{J}$ when $k$ is not very small. Rather, from Bartholomew-Biggs et al. (2000):
\begin{quotation}\textit{
Automatic differentiation is a set of techniques for transforming a program that calculates numerical values of a function, into a program which calculates numerical values for derivatives of that function with about the same accuracy and efficiency as the function values themselves.
}\end{quotation}

\paragraph{References} ~\\
\begin{itemize}

\item Bartholomew-Biggs, M., S. Brown, B. Christianson, and L. Dixon. 2000. Automatic differentiation of algorithms. J. Comput. Appl. Math. 124:171–190.

\item Brent, R. P. 1971. An algorithm with guaranteed convergence for finding a zero of a function. Comput. J. 14:422–425.

\end{itemize}

\end{document}


\section{Method}
\label{sec:method}
Bug bounty programs work on the premise that humans as a crowd are efficient at searching and finding bugs. Their mere existence is a {\it de facto} recognition that market approaches for bug discovery bring efficiency, beyond in-house security. Bug bounty programs signal that organizations are ready to complement their vertical cost-effective security operations with market approaches, which are traditionally perceived as less cost-effective, yet more adaptive \cite{coase1937}. Early on, Brady et al. \cite{brady1999murphy} have offered a hint for the existence of such markets for bugs: according to their proposed theory, each researcher has slightly different skills and mindset. When a security researcher tests a software piece by choosing the inputs, she offers a unique operational environment. This environment is prone to the discovery of new bugs, which may not have been seen by other researchers. The proposed theory by Brady et al. \cite{brady1999murphy} intrinsically justifies the existence of bug bounty program structures as markets, which provide the necessary diversity to account for the highly uncertain risk horizon of bug discovery. Here, we develop a quantitative method to formalize a mechanism and to test the theory proposed in \cite{brady1999murphy}. This validation step shall help provide organizational design insights for bug bounty programs.\\

For that, we investigate the interplay between the vulnerability discovery process and the cumulative rewards distributed to security researchers within and across 35 public bounty programs hosted on HackerOne. When a bug bounty program starts, it attracts a number of security researchers, who in turn submit bug reports. Subsequent bug discoveries get increasingly difficult for each individual researcher, and to some extent for all researchers together. The difficulties faced by security researchers can be technical. They can also be the result of insufficient or conflicting incentives. Here, we develop and test a model, which accounts for both technical difficulties and insufficient incentives. We further address conflicting incentives by measuring the effect of newly launched bug bounty programs on incumbent programs.\\

Starting from an initial probability of discovering the first vulnerability $P(k=0) = 1$, the probability to find a second bug is a fraction of the former probability: $P_{k+1} = \beta \cdot P_k$ with $\beta$ a constant strictly smaller than one. The probability that no more discoveries will be made after $k$ steps is given by $P_k = \beta^{k} (1-\beta)$. Conversely, starting from the initial reward $R_0 = R(k=0)$, the subsequent reward $R_1 = \Lambda_1 \cdot R_0$, and further additional reward $ R_2 = \Lambda_2 \Lambda_1 \cdot R_{0}$. After $n$ steps, the total reward is the sum of all past rewards:

\begin{equation}
R_{n} = R_{0} \sum_{k=1}^{n} \Lambda_1 ... \Lambda_k.
\end{equation}

Thus, $R_{n}$ is the recurrence solution of the
Kesten map ($R_{n} = \Lambda_n R_{n-1} +R_0$)
\cite{kesten1973random,sornette1997convergent}:  as soon as amplification occurs (technically,
some of the factors $\Lambda_k$ are larger than $1$), the distribution
of rewards is a power law, whose exponent $\mu$ is a function of $\beta$
and of the distribution of the factors $\Lambda_k$. In the case where
all factors are equal to $\Lambda$, this model predicts three possible regimes for the distribution of rewards (for a given program): thinner than exponential for $\Lambda < 1$, exponential for $\Lambda = 1$, and power law for $\Lambda > 1$ with exponent $\mu = |\ln \beta|/ \ln \Lambda$ \cite{sornette2013exploring}. The expected payoff of vulnerability discovery is given by,

\begin{equation}
\label{ }
U_k = P_k \times R_k,
\end{equation}

\noindent with both $P_k$ and $R_k$ random variables respectively determined by $\beta$ and $\Lambda$. Because $U_k$ is a multiplication of diverging probability and reward components, its nature is reminiscent of the St. Petersburg paradox (or St. Petersburg lottery), proposed first by the Swiss Mathematician Nicolas Bernoulli in 1713, and later formalized by his brother Daniel in 1738 \cite{bernoulli1954exposition}. The St. Petersburg paradox states the problem of decision-making when both the probability and the reward are diverging for $k \rightarrow \infty$: a player has a chance to toss a fair coin at each stage of the game. The pot starts at two and is doubled every time a head appears. The first time a tail appears, the game ends and the player wins whatever is in the pot. Thus, the player wins two if a tail appears on the first toss, four if a head appears on the first toss and a tail on the second, eight if a head appears on the first two tosses and a tail on the third, and so on. The main interest of Bernoulli was to determine how much a player would be ready to pay for this game, and he found that very few people would like to play this game even though the expected utility increases (in the simplest case proposed by Bernoulli, $U_n = \sum_{k=0}^{n} U_k = n$) \cite{bernoulli1954exposition}. The situation of a security researcher differs from the St. Petersburg lottery as bug search costs are incurred at every step. Since these costs influence the probability to find an additional bug, they can be at least partially integrated in $P_k$. We could assume equivalently that costs are integrated into a net utility as $U^{*}_k = U_k - c_k$. Here, we do not factor these costs in because their exact nature is largely undetermined and our data do not offer a reliable proxy. The security researcher may also decide to stop searching for bugs in a program, at any time $k$: this is equivalent to setting $P_{k+1} = 0$.\\

The expected payoff $U_k$ therefore determines the incentive structure for security researchers, given that the bounty program manager can tune $R_0$ and to some extent $\Lambda$. The utility function may also incorporate non-monetary incentives, such as reputation: finding a long series of bugs may signal some fitness for a bug bounty program and thus create a permanent job opportunity \cite{moussouris2016}. Similarly, discovery of a rare (resp. critical) bug that no other researcher has found before has a strong signaling effect, which can help make a career. However, these strategies are high-risk high-return. Therefore, they result in additional fame. In the next section, we will calibrate our model to the bug discovery process associated with 35 bounty programs publicly documented on the HackerOne platform.

 %However, rules are generally set upfront and shall not be changed in the course of the bounty program. Changing game rules is risky as it may undermine trust in the program. Here, we assume that bounty program managers don't tune their reward policy after the bug bounty program has started.

%\sout{In principle, the manager could set $R_0$ to influence $P_0$ and indirectly $P_{k}$. Mapping the discovery rank $k$ into the rate of discovery may also help considering discounting aspects in presence of competing opportunities and inter-temporal choices under uncertainty \cite{loewenstein1992anomalies}. A new public bounty program is launched at a Poisson rate, approximately every 2 months, and each launch brings its windfall effect, leaving the researcher with the choice to either keep digging increasingly harder vulnerabilities (rare but with higher reward), or turning to the low hanging fruit (frequent but with low reward) of a new program. We shall therefore verify whether newly launched programs actually influence security researchers.}

%The propensity for security researchers to keep searching for vulnerabilities is conditioned by the trend of $U(k)$: if $U \rightarrow 0$ when $k \rightarrow \infty$, then it's likely that they will leave the program (it's a bit sketchy here). If do not consider the costs (i.e., effort spent on finding the $k^{th}$ bug, which we cannot measure here (see next subsection on timing effects), but which is indirectly captured by $P(k)$), then in the presence of a variety of programs at different stages of their life (i.e., at different values of $k$), the security researcher is left with comparing utility between programs.

%\begin{figure}
%\begin{center}
%%\includegraphics[width=10cm]{figures/decay.eps}
%\caption{Distribution of discovery waiting time between ranks $\rightarrow$ the hope is to find that time increases as $k \rightarrow \infty$.}
%\label{fig:decay}
%\end{center}
%\end{figure}

%The incentive mechanism is then completely driven by the probability of discovery of the $k+1$ vulnerability given that $k$ bugs have already been discovered on the one hand, and by the expected reward for the $k+1$ discovery on the other hand. When the probability of finding a bug decreases fast while the reward for a new bug increases fast, in such a way that the product $f_k \times R_k$ remains constant, then the expected value of the game increases linearly as $k \rightarrow \infty$. This problem is reminiscent

%When a bug bounty program (by an organization) is launched immediate vulnerabilities are found by security researchers and rewarded by organizations. The cost of finding a new vulnerability increases as more discoveries occur, and thus, incentives should be set accordingly to keep onboard security researchers, or at least, the best researchers who have the capabilities to find increasingly subtle problems.

%Searching for bugs in software is an attrition process: When the search starts, obvious bugs are found first. Subsequent bug discoveries get increasingly difficult. When a bug bounty program (by an organization) is launched immediate vulnerabilities are found by security researchers and rewarded by organizations. The cost of finding a new vulnerability increases as more discoveries occur, and thus, incentives should be set accordingly to keep onboard security researchers, or at least, the best researchers who have the capabilities to find increasingly subtle problems.
%
%{\bf aim 1:} understand the evolution of expected utility of security researchers as an increasing amount of vulnerabilities get discovered for a given program. And how the incentive mechanism varies from one program to another\\
%
%{\bf aim 2:} understand how bug bounties compete with each others? Each time, a new bug bounty program is launched, a new set of opportunities arises, and security researchers must make a choice (though not a binary choice) between digging further in an existing program (with lower probability to find a vulnerability, but presumably with higher utility) or turning to a new program (with higher probability to find a vulnerability, but presumably with lower utility).
%
%{\bf aim3: } What can we say about the whole eco-system?


\section{Model Functions}

This module is intended to be used an approximation of fuel slosh attached to the spacecraft. It is intended to have multiple fuel slosh particles attached to the spacecraft. Also it is recommended to attach at least 3 fuel slosh particles with different sloshing directions to get motion across all axes. Below is a list of functions that this model performs:

\begin{itemize}
	\item Compute it's contributions to the mass properties of the spacecraft
	\item Provides matrix contributions for the back substitution method
	\item Compute it's derivatives for $\rho$, $\dot{\rho}$, and $\dot{m}$
	\item Adds energy and momentum contributions to the spacecraft
\end{itemize}

\section{Model Assumptions and Limitations}
Below is a summary of the assumptions/limitations:

\begin{itemize}
	\item Is an approximation to sloshing fuel
	\item Is derived in such a manner that does not require constraints to be met
	\item A single fuel slosh particle can only move along one direction, $\hat{\bm p}_j$, as seen in Figure~\ref{fig:Flex_Slosh_Figure}
	\item Multiple fuel slosh particles are not interconnected, each fuel slosh particle is attached the rigid body hub independently
	\item Only constant linear spring and damping terms
	\item The mass, spring and damping coefficients can be attenuated to approximate frequencies expected by the fuel slosh
	\item There are no travel limits for the fuel slosh therefore the particles could travel past the limits of the fuel tank boundary (this can be avoided by varying the mass and/or spring constant while considering the expected accelerations of the s/c to stay within the tank)
	\item The mass of the fuel slosh particles add to the total mass of the fuel tank
	\item This model could be used with other fuel slosh models like a pendulum based fuel slosh model
\end{itemize}