\section{Numerical Modelling Theory}
\label{sec:NumericsTheory}

\subsection{FEMDEM Model}
The combined finite-discrete element method, \texttt{FEMDEM} or \texttt{FDEM}\cite{Wan18, Mun95, Mun99, Mun04, Mun12, Mun13, Guo16, Gao14, Xu14, Che18, Mun13}, contains discrete element that interact with neighbouring discrete elements. In addition, each discrete element is discretised into finite elements. Each finite element mesh captures the deformability of a discrete element.

\bigbreak
Fracture and fragmentation is modelled as a transition from continua to discontinua. It is also in principle possible to imagine an inverse process of particles merging together.\cite{Mun04}

\bigbreak
Based on the \texttt{3D} \texttt{FDEM} code \texttt{Y3D} by Munjiza et al \cite{Mun95, Mun99, Mun04}, Xiang et al \cite{Xia09} added many detailed improvements and features to the code. The new model, \texttt{Solidity}, is capable of creating realistic coupled multi-physics simulations. In contrast to conventional models, Solidity is not reliant on element deformability restriction constraints (due to the locking problem) and uses simpler triangular, quadratic and tetrahedral elements \cite{Lat15}.

\subsection{Governing Equations}

The governing equations for the finite element calculations in the \texttt{FEMDEM} method are the equations of motion. The equations are given by

\begin{equation}
    M\ddot{x}+\mu\dot{x}+f_{\rm{int}}=f_{\rm{ext}}=f_{\rm{l}}+f_{\rm{b}}+f_{\rm{c}}\,,
\end{equation}

with lumped nodal mass matrix $M$, nodal displacements $x$, viscosity $\mu$, internal nodal forces $f_{\rm{int}}$ and external nodal forces $f_{\rm{ext}}$. External forces contain consist of external loads $f_{\rm{l}}$, bonding forces $f_{\rm{b}}$ and contact forces $f_{\rm{c}}$. Internal forces $f_{\rm{int}}$ are generated by element deformation. \texttt{FEMDEM} systems solve these equations via explicit time integration using the forward \texttt{Euler} method \cite{Lei16}.

\subsection{Contact Detection}

The combination of discrete and finite elements is established via an interaction algorithm \cite{Lei16}. This algorithm consists of contact detection \cite{Che15} and contact interaction \cite{Mun13}. Contact detection is carried out using search algorithms.

\bigbreak
The contact detection algorithm detects couples of discrete elements close to each other by eliminating couples of discrete elements that are too far from each other to be in contact. In other words, contact detection avoids processing contact interaction when there is no contact. This reduces \texttt{CPU} requirements and processing run time \cite{Mun04}.

\bigbreak
Contact interaction is only considered for discrete elements which are within a buffer size $b$ of each other, given by

\begin{equation}
    \label{eq:buff}
    b = 0.1\,\Delta_{\rm{min}}\,,
\end{equation}

where $\Delta_{\rm{min}}$ is the minimum edge\footnote{minimum element side length, minimum size}.

\subsection{Contact Interaction}

Penetration of a contractor\footnote{index c} body into a target\footnote{index t} body is implemented in the \texttt{Y} code by the \texttt{potential contact force method}, see Fig. \ref{fig:contact} \cite{Mun04}. Assumption of a conservative force field enables a description of the contact forces using potentials,

\begin{align}
    f_{\rm{c}}^{\rm{2D}}&=-f_{\rm{t}}^{\rm{2D}}=\oint_{\Gamma_{\beta{\rm{t}}\cap\beta{\rm{c}}}}n_{\Gamma}\left(\varphi_{\rm{c}}-\varphi_{\rm{t}}\right)\,{\rm{d}}\Gamma\,,\\
    f_{\rm{c}}^{\rm{3D}}&=-f_{\rm{t}}^{\rm{3D}}=\int_{S_{\beta{\rm{t}}\cap\beta{\rm{c}}}}n_{S}\left(\varphi_{\rm{c}}-\varphi_{\rm{t}}\right)\,{\rm{d}}S\,,
\end{align}

with force potentials $\varphi$ and outward unit normal $n$ to the penetration boundary $\Gamma$ or surface area $S$.

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\columnwidth]{Contact.png}
    \caption{Infinitesimal overlap of contractor and target body. \cite{Mun04}}
    \label{fig:contact}
\end{figure}

\bigbreak
The penetration distance $d$ for each case is given by \cite{Mun04}

\begin{equation}
    \label{eq:d}
    d=\frac{\sigma\,h}{p}\,,
\end{equation}

with applied pressure $\sigma$, element height $h$ and penalty factor $p$.
\bigbreak
The penalty factor $p$ is given by

\begin{equation}
    \label{eq:p}
    p=\alpha\,E\,,
\end{equation}

with \rm{Young's Modulus} $E$ and a constant $\alpha$ (in this paper: $\alpha=10$) to regulate penetration.

\bigbreak
Contact damping (Rayleigh damping, viscous damping) due to plastic deformation, breakage of surface asperities, etc. is given by \cite{Mun04}

\begin{equation}
    \label{eq:sigmac}
    \sigma_{\rm{C}}=4\,\xi\frac{\sqrt{p/\rho}}{h}\dot{d}\,,
\end{equation}

\addtocounter{footnote}{-1}
\stepcounter{footnote}
with damping ratio $0\leq\xi\footnotemark\leq1$ and finite element density $\rho$.

\footnotetext{describes energy dissipation}

\bigbreak
The mass damping coefficient is given by

\begin{equation}
    \label{eq:massdamp}
    \zeta = \sqrt{E\,\rho}
\end{equation}


\documentclass[11pt,letterpaper]{article}

\usepackage[pdftex]{graphicx}
\usepackage{natbib}
\usepackage{fullpage}
\usepackage{lineno}
\usepackage{multirow}
\usepackage{wrapfig}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{sidecap}
\usepackage{hyperref}

\begin{document}

\setlength{\parindent}{0mm}
\setlength{\parskip}{0.4cm}

\bibliographystyle{apalike}

%\modulolinenumbers[5]
%\linenumbers

\title{A Summary of Some of the Elliptic Equations Used in GEMINI}
\author{Matthew D. Zettergren, PhD\\ Associate Professor of Engineering Physics\\ Center for Space and Atmospheric Physics\\ Physical Sciences Department \\Embry-Riddle Aeronautical University\\mattzett@gmail.com\\zettergm@erau.edu}
\maketitle

\tableofcontents

\pagebreak

This document briefly outlines the electrodynamics equations solved in GEMINI - This information is a summary for conveniences and is not intended to replace the full description in the main documentation.

\section{General ortho, static, EFL} \label{sec:main}

The most common description of electrodynamics used in GEMINI is an equipotential field line (EFL) quasistatic equation:
\begin{eqnarray}
&~& \frac{\partial}{\partial x_2} \left( \Sigma_P^{(2)} \frac{\partial \Phi}{\partial x_2} \right) + \frac{\partial}{\partial x_3} \left( \Sigma_P^{(3)} \frac{\partial \Phi}{\partial x_3} \right) -  \frac{\partial \Sigma_H'}{\partial x_2} \frac{\partial \Phi}{\partial x_3} + \frac{\partial \Sigma_H'}{\partial x_3} \frac{\partial \Phi}{\partial x_2} \nonumber \\ &=& \int h_1 h_2 h_3 \left\{ \nabla_\perp \cdot \left[ \boldsymbol{\sigma}_\perp \cdot \left( \mathbf{v}_{n\perp} \times \mathbf{B} \right) \right] + + \nabla_\perp \cdot \left( \boldsymbol{\sigma}_\perp \cdot \mathbf{E}_{0\perp} \right) \right\} d x_1 + \left[ h_2 h_3 J_1 \right] \left|^{x_{1,max}}_{x_{1,min}} \right. . \label{eqn:curvpotEFL}
\end{eqnarray}
The coefficients of this equation have the functional dependencies:  $\Sigma_P^{(2,3)}(x_2,x_3)$, $\Sigma_H'(x_2,x_3)$, and the right-hand side of Equation \ref{eqn:curvpotEFL} is an inhomogeneous term that can be computed from an atmosphere with a known state (in our case from MAGIC).

The quasistatic equation must be evaluated multiple times (by default every time step - but this is likely a bit overkill) as the simulation progresses since atmospheric forcing will cause variations of the conductances ($\Sigma$ terms) with time.  These variations are slow compared to other electrodynamic processes such that they can be considered effectively fixed (wrt time) during a single time step.

Equation \ref{eqn:curvpotEFL} can be used even for many 3D systems due to strong mapping of electric fields along highly conducting geomagnetic field lines.  Nonetheless it does begin to break down at very small scales as discussed by Farley et al.

\section{General ortho, static, non-EFL}

A fully resolved equation (i.e. along geomagnetic field lines) may be needed in cases where the scale sizes are small.  This situation is currently only solved in GEMINI in 2D as it has been found to be extraordinarily computational expensive in 3D (to the point of being totally infeasible with MUMPS, umfpack, and other direct solvers).  This equation reads:
\begin{eqnarray}
\frac{1}{h_1 h_2 h_3} \left[ \frac{\partial}{\partial x_1} \left( \frac{h_2 h_3}{h_1} \sigma_0 \frac{\partial \Phi}{\partial x_1} \right) + \frac{\partial}{\partial x_2} \left( \frac{h_1 h_3}{h_2} \sigma_P \frac{\partial \Phi}{\partial x_2} \right) -  \right. \frac{\partial}{\partial x_2} \left( h_1 \sigma_H \right) \frac{\partial \Phi}{\partial x_3} &+& \nonumber \\ \left. \frac{\partial}{\partial x_3} \left( h_1 \sigma_H \right) \frac{\partial \Phi}{\partial x_2} + \frac{\partial}{\partial x_3} \left( \frac{h_1 h_2}{h_3} \sigma_P \frac{\partial \Phi}{\partial x_3} \right)
\right]  = \nabla_\perp \cdot \left( \sum_s n_s m_s \nu_s \boldsymbol{\mu}_{s\perp} \cdot \mathbf{v}_{n\perp} \right) + \nabla_\perp \cdot \left( \boldsymbol{\sigma}_\perp \cdot \mathbf{E}_{0\perp} \right) \label{eqn:curvpot}
\end{eqnarray}
For 2D solves (the only type found feasible so far) all $x_3$ derivatives are taken to be zero.  All coefficients in the general 3D case depend on all three coordinates $x_1,x_2,x_3$ and the equation is treated as quasistatic as described in Section \ref{sec:main}.  The right-hand side of this equation is effectively known - i.e. specified from user or other model input.

\section{Cartesian, EFL, dynamic}

In cases where extremely high resolution is needed, higher-order quasistatic corrections are required as described in Kintner and Seyler, (1985).  In Cartesian coordinates $(x_1,x_2,x_3)=(z,x,y)$ this equation is:
\begin{eqnarray}
\frac{\partial}{\partial x_2} \left( \Sigma_P \frac{\partial \Phi}{\partial x_2} \right) + \frac{\partial}{\partial x_3} \left( \Sigma_P \frac{\partial \Phi}{\partial x_3} \right) -  \frac{\partial \Sigma_H}{\partial x_2} \frac{\partial \Phi}{\partial x_3} + \frac{\partial \Sigma_H}{\partial x_3} \frac{\partial \Phi}{\partial x_2} &+& \nonumber \\
\left\{ \frac{\partial}{\partial x_2} \left[ C_M \frac{\partial}{\partial t} \left( \frac{\partial \Phi}{\partial x_2} \right) \right] + \frac{\partial}{\partial x_3} \left[ C_M \frac{\partial}{\partial t} \left( \frac{\partial \Phi}{\partial x_3} \right) \right] \right\} &+& \nonumber \\
\frac{\partial}{\partial x_2} \left( C_M v_2 \frac{\partial^2 \Phi}{\partial x_2^2} + C_M v_3 \frac{\partial^2 \Phi}{\partial x_3 \partial x_2} \right) &+& \nonumber \\
\frac{\partial}{\partial x_3} \left( C_M v_2 \frac{\partial^2 \Phi}{\partial x_2 \partial x_3} + C_M v_3 \frac{\partial^2 \Phi}{\partial x_3^2} \right) &=& \nonumber \\
= \int \left\{ \nabla_\perp \cdot \left[ \boldsymbol{\sigma}_\perp \cdot \left( \mathbf{v}_{n\perp} \times \mathbf{B} \right) \right] + + \nabla_\perp \cdot \left( \boldsymbol{\Sigma}_\perp \cdot \mathbf{E}_{0\perp} \right) \right\} d x_1 + \left[J_1 \right] \left|^{x_{1,max}}_{x_{1,min}} \right. , \label{eqn:electrodynamic}
\end{eqnarray}
where the coefficients only depend on $x_2,x_3$ and the right-hand side terms are computed from given user or model input.

Equation \ref{eqn:electrodynamic} is mixed-type and is currently just solved with Backward Euler time stepping (!).  Higher order in time solutions are very expensive and it is not clear that they are necessary for many problems where the capacitance (coefficient for the non-elliptic terms) is fairly small, but this really should be examined more systematically.  Truthfully, few people bother with this level of physical detail in the ionosphere so it is a bit of an underexplored aspect.

\end{document}


\section{Discussion}

The forecast accuracy of our \gls{esn} for the Lorenz model does not persist
for quite as long as in other work \cite{pathak_using_2017}. However, our
model successfully replicates the environment that produces the Lorenz
Attractor. Further, each randomly instantiated reservoir has a unique set of
optimal hyper-parameters. It is impossible to replicate the exact conditions of
other works without information about a seed for the random state. We have
included this information in Table \ref{tab:lorenzparam} for future work to compare with our results.

For each target variable (demand, wind, and solar) we found that sun
elevation angle, while not always the best, improved the forecast error in
every case. We hypothesize that additional weather features effect model
performance due to their temporal
complexity relative to the target variable. Electricity demand,
for example, is quite ``predictable," and therefore has relatively low complexity.
Meanwhile, air
temperature and other weather related variables are less predictable. Thus,
adding air temperature as a model input for electricity demand increases the total system complexity
and weakens performance. Further, solar elevation angle is completely
deterministic, perfectly predictable, and either improved,
or neutrally effected, model performance in every case. Like electricity
demand, solar elevation angle has both diurnal and annual periodicity but has
lower complexity than air temperature and electricity demand itself.
Conversely, solar and wind energy are both nonlinear functions of many weather
variables and consequently have greater complexity than air temperature. Thus,
adding temperature as a model input improved the forecast error because
the total system complexity decreased relative to solar and wind energy alone.
Including wind speed only improved the wind energy forecasts, likely because it
has greater complexity than solar energy but less than wind energy. Relative
humidity has an inconsistent and poorly understood effect on model performance.
It improved the forecast for 48-hour ahead electricity demand but worsened it
for the 4-hour ahead forecast, as shown in Table \ref{tab:demand48} and Table
\ref{tab:demand04}. The opposite trend occurred for solar energy.
Quantifying the predictability and complexity of these systems is in progress.
% \textit{Weighted permutation entropy} is a good measure for this type of
% complexity \cite{fadlallah_weighted-permutation_2013,garland_model-free_2014, pennekamp_intrinsic_2019}.

These results point to an important disadvantage of using \glspl{esn} to
forecast renewable energy: Although simple and fast, \glspl{esn} remain a black
box. We assume that there exists some underlying dynamics in the energy system
that can be ``learned,'' but we cannot observe the learning process nor extract
important features from \glspl{esn}.

We decided the forecast lengths based on the requirements for improved
economics and planning mentioned in the literature
 \cite{wang_quantifying_2016,mc_garrigle_quantifying_2015,brancucci_martinez-anido_value_2016}. The \gls{esn} model performed reasonably well at predicting
 4-hours ahead but did not improve on the state-of-the-art
 \cite{wang_quantifying_2016,powers_weather_2017}. The model did not perform
 well at the 48-hour ahead forecasts, potentially due to the lack of higher
 resolution data. \glspl{esn} are able to predict highly non-linear
  systems \cite{jaeger_harnessing_2004,lukosevicius_reservoir_2009}, yet
 using hourly data could add spurious complexity that confounds the model
 \cite{garland_model-free_2014}

\subsection{Future Work}
One appealing avenue of continued work is to leverage \glspl{esn} to generate
synthetic data that respects real dynamics. Sythetic data are often useful for
other machine learning or optimization algorithms. Typically, these data are
produced by sampling from an Auto-Regressive Moving Average (ARMA) model \cite{baker_optimal_2018, garcia_dynamic_2016}, which tacitly assumes the
training data can be made stationary. \glspl{esn} can replicate
the environment of a dynamical system, although it remains unclear how far
in the future this behavior persists \cite{pathak_using_2017,pathak_model-free_2018}.
Future work will also explore the effect of data resolution on model
performance, as well as evaluate improvements to the \gls{esn}
algorithm.


\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[osf]{libertine}
\usepackage[scaled=0.8]{beramono}
\usepackage[margin=1.5in]{geometry}
\usepackage{url}
\usepackage{booktabs}
\usepackage{amsmath}
\usepackage{nicefrac}
\usepackage{microtype}

\usepackage{sectsty}
\sectionfont{\large}
\subsectionfont{\normalsize}

\usepackage{titlesec}
\titlespacing{\section}{0pt}{10pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}
\titlespacing{\subsection}{0pt}{5pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}

\usepackage{pgfplots}
\pgfplotsset{
  compat=newest,
  plot coordinates/math parser=false,
  tick label style={font=\footnotesize, /pgf/number format/fixed},
  label style={font=\small},
  legend style={font=\small},
  every axis/.append style={
    tick align=outside,
    clip mode=individual,
    scaled ticks=false,
    thick,
    tick style={semithick, black}
  }
}

\pgfkeys{/pgf/number format/.cd, set thousands separator={\,}}

\usepgfplotslibrary{external}
\tikzexternalize[prefix=tikz/]

\newlength\figurewidth
\newlength\figureheight

\setlength{\figurewidth}{8cm}
\setlength{\figureheight}{6cm}

\setlength{\parindent}{0pt}
\setlength{\parskip}{1ex}

\newcommand{\acro}[1]{\textsc{\MakeLowercase{#1}}}
\newcommand{\given}{\mid}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\data}{\mc{D}}
\newcommand{\intd}[1]{\,\mathrm{d}{#1}}
\newcommand{\inv}{^{-1}}

\begin{document}

\section*{Coin flipping}

Suppose there is a coin that may be biased -- this coin has unknown
probability $\theta$ of giving a ``heads.''  If we repeatedly flip
this coin and observe the outcomes, how can we maintain our belief
about $\theta$?

Note that the coin-flipping problem can be seen as a simplification of
the survey problem we discussed last time, where we assume that people
always tell the truth, are sampled uniformly at random, and whose
opinions are generated independently (by flipping a coin!).

Before we select a prior for $\theta$, we write down the likelihood.
For a particular problem, it is almost always easier to derive an
appropriate likelihood than it is to identify an appropriate prior
distribution.

Suppose we flip the coin $n$ times and observe $x$ ``heads.''  Every
statistician, regardless of philosophy, would agree that the
probability of this observation, given the value of $\theta$, comes
from a binomial distribution:
\begin{equation*}
  \Pr(x \given n, \theta)
  =
  \binom{n}{x} \theta^x (1 - \theta)^{n - x}.
\end{equation*}

\subsection*{Classical method}

Before we continue with the Bayesian approach, we pause to discuss how
a classical statistician would proceed with this problem.  Recall that
in the frequentist approach, the value $\theta$ can only be considered
in terms of the frequency of success (``heads'') seen during an
infinite number of trials.  It is not valid in this framework to represent
a ``belief'' about $\theta$ in terms of probability.

Rather, the frequentist approach to reasoning about $\theta$ is to
construct an \emph{estimator} for $\theta$, which in theory can be any
function of the observed data: $\hat{\theta}(x, n)$.  Estimators are
then analyzed in terms of their behavior as the number of observations
goes to infinity (for example, we might prove that $\hat{\theta} \to
\theta$ as $n \to \infty$).  The classical estimator in this case is
the empirical frequency $\hat{\theta} = \nicefrac{x}{n}$.

\subsection*{Bayesian method}

An interesting thing to note about the frequentist approach is that it
ignores all prior information, opting instead to only look at the
observed data.  To a Bayesian, every such problem is different and
should be analyzed contextually given the known information.

With the likelihood decided, we must now choose a prior distribution
$p(\theta)$.  A convenient prior in this case is the \emph{beta
  distribution,} which has two parameters $\alpha$ and $\beta$:
\begin{equation*}
  p(\theta \given \alpha, \beta)
  =
  \mc{B}(\theta; \alpha, \beta)
  =
  \frac{1}{B(\alpha, \beta)}
  \theta^{\alpha - 1}(1 - \theta)^{\beta - 1}.
\end{equation*}
Here the normalizing constant $B(\alpha, \beta)$ is the \emph{beta
  function:}
\begin{equation*}
  B(\alpha, \beta)
  =
  \int_{0}^{1} \theta^{\alpha - 1}(1 - \theta)^{\beta - 1} \intd{\theta}.
\end{equation*}

The support of the beta distribution is $\theta \in (0, 1)$, and by
selecting various values of $\alpha$ and $\beta$, we can control its
shape to represent a variety of different prior beliefs.

Given our observations $\data = (x, n)$, we can now compute the
posterior distribution of $\theta$:
\begin{equation*}
  p(\theta \given x, n, \alpha, \beta)
  =
  \frac
      {     \Pr(x \given n, \theta) p(\theta \given \alpha, \beta)}
      {\int \Pr(x \given n, \theta) p(\theta \given \alpha, \beta) \intd{\theta}}.
\end{equation*}

First we handle the normalization constant $\Pr(x \given n, \alpha, \beta)$:
\begin{align*}
  \int \Pr(x \given n, \theta) p(\theta \given \alpha, \beta) \intd{\theta}
  &=
  \binom{n}{x}
  \frac{1}{B(\alpha, \beta)}
  \int_{0}^{1}
  \theta^{\alpha + x - 1}(1 - \theta)^{\beta + n - x - 1} \intd{\theta}
  \\
  &=
  \binom{n}{x}
  \frac{B(\alpha + x, \beta + n - x)}{B(\alpha, \beta)}.
\end{align*}

Now we apply Bayes theorem:
\begin{align*}
  p(\theta \given x, n, \alpha, \beta)
  &=
  \frac
      {     \Pr(x \given n, \theta) p(\theta \given \alpha, \beta)}
      {\int \Pr(x \given n, \theta) p(\theta \given \alpha, \beta) \intd{\theta}}
  \\
  &=
  \biggl[
    \binom{n}{x}
    \frac{B(\alpha + x, \beta + n - x)}{B(\alpha, \beta)}
  \biggr]\inv
  \biggl[
    \binom{n}{x} \theta^x (1 - \theta)^{n - x}
  \biggr]
  \biggl[
    \frac{\theta^{\alpha - 1}(1 - \theta)^{\beta - 1}}{B(\alpha, \beta)}
  \biggr]
  \\
  &=
  \frac{1}{B(\alpha + x, \beta + n - x)}
  \theta^{\alpha + x - 1}(1 - \theta)^{\beta + n - x - 1}
  \\
  &=
  \mc{B}(\alpha + x, \beta + n - x).
\end{align*}
The posterior is therefore another beta distribution with parameters
$(\alpha + x, \beta + n - x)$; we have added the number of successes
to the first parameter and the number of failures to the second.

The rather convenient fact that the posterior remains a beta
distribution is because the beta distribution satisfies a property
known as \emph{conjugacy} with the binomial likelihood.  This fact
also leads to a common interpretation of the parameters $\alpha$ and
$\beta$: they serve as ``pseudocounts,'' or fake observations we
pretend to have seen before seeing the data.

Figure \ref{coin_flipping} shows the relevant functions for the coin
flipping example for $(\alpha, \beta) = (3, 5)$ and $(x, n) = (5, 6)$.
Notice that the likelihood favors higher values of $\theta$, whereas
the prior had favored lower values of $\theta$.  The posterior, taking
into account both sources of information, lies in between these
extremes.  Notice also that the posterior has support over a narrower
range of plausible $\theta$ values than the prior; this is because we
can draw more confident conclusions from having access to more
information.

\begin{figure}
  \centering
  \input{figures/beta_example.tex}
  \caption{An example of Bayesian updating for coin flipping.  Figure
    produced by \texttt{plot\_beta\_example.m}.}
  \label{coin_flipping}
\end{figure}

\section*{Hypothesis testing}

We often wish to use our observed data to draw conclusions about the
plausibility of various hypotheses.  For example, we might wish to
know whether the parameter $\theta$ is less than $\nicefrac{1}{2}$.
The Bayesian method allows us to compute this value directly from the
posterior distribution:
\begin{equation*}
  \Pr(\theta < \nicefrac{1}{2} \given x, n, \alpha, \beta)
  =
  \int_{0}^{\nicefrac{1}{2}} p(\theta \given x, n, \alpha, \beta) \intd{\theta}.
\end{equation*}
For the example in Figure \ref{coin_flipping}, this probability is
approximately 15\%.

There is a sharp contrast between the simplicity of this approach and
the frequentist method.  The classical approach to hypothesis testing
uses the likelihood as a way of generating fake datasets of the same
size as the observations.  The likelihood then serves as a so-called
``null hypothesis'' that allows us to generate hypothetical datasets
under some condition.

From these, we compute \emph{statistics,} which, like estimators, can
be any function of the hypothesized data.  We then identify some
critical set $C$ for this statistic which contains some large portion
$(1 - \alpha)$ of the values corresponding to the datasets generated
by our null hypothesis.  If the statistic computed from the observed
data falls outside this set, we reject the null hypothesis with
``confidence'' $\alpha$.  Note that the ``rejection'' of the null
hypothesis in classical hypothesis testing is purely a statement about
the observed data (that it looks ``unusual''), and not about the
plausibility of alternative hypotheses!

\end{document}


% !TEX root =  manual.tex

\section{Settings File}\label{sec:settings}

The file \path{settings.dat} controls the operation of most of the Distribution Sampling programs.

\begin{description}
    \item[MODEL\_OUTPUT\_DIRECTORY] (default: \path{model_output}) The directory (relative to the statistical analysis directory) where the model inputs and outputs can be found. See Section \ref{sec:DirectoryStructure} for the structure of this directory. If specified as a relative path, the path is considered to be relative to the statistical analysis directory.

    \item[EXPERIMENTAL\_RESULTS\_FILE] (default: \path{experimental_results.dat}) This file contains the observed values from the actual system being modeled. See Section \ref{sec:ObservationalData} for the format of this file. If specified as a relative path, the path is considered to be relative to the statistical analysis directory.

    \item[VERBOSE] (default: 0) Print warnings and other potentially useful information. When this option is set to false, the programs will exit silently when everything has succeeded.

    \item[READER\_VERBOSE] (default: 0) Print copious amounts of information while reading the data directory structure.

    \item[GENERATE\_TRAINING\_POINTS\_NUMBER\_OF\_POINTS] (default: 100) The number of training points generated by \path{madai_generate_training_points}

    \item[GENERATE\_TRAINING\_POINTS\_PARTITION\_BY\_PERCENTILE] (default: 1) If on, this will divide the parameter space up according to an even sampling of the percentile space. For example, of there are five samples to be taken in one dimension of the parameter space, then the values of the parameter will be determined by calculating the values of the parameter at percentiles 0, 0.25, 0.5, 0.75, and 1.0. For a uniformly-distributed parameter, this translates into an even sampling of the parameter in the range specified by the uniform distribution's minimum and maximum. For a Gaussian-distributed parameter, this translates into the sample parameter values corresponding to the percentile of the Gaussian distribution assigned to the parameter.

    \item[GENERATE\_TRAINING\_POINTS\_STANDARD\_DEVIATIONS] (default: 3) If the prior distribution for a parameter is Gaussian, this specifies how far out should the uniformly-spaced sampled points go. It is defined as a multiple of the sigma parameter for the Gaussian distribution.

    \item[GENERATE\_TRAINING\_POINTS\_USE\_MAXIMIN] (default: false) Controls whether \path{madai_generate_training_points} uses the maximin algorithm to generate the training points. This algorithm attempts to find a Latin hypercube sampling that has a large minimimum distance between points in parameter space.

    \item[GENERATE\_TRAINING\_POINTS\_MAXIMIN\_TRIES] (default: 20) When \newline GENERATE\_TRAINING\_POINTS\_USE\_MAXIMIN is on, this option determines the number of Latin hypercubes that are generated when searching for one with the largest minimum distance between points in parameter space.

    \item[GENERATE\_POSTERIOR\_POINTS\_NUMBER\_OF\_POINTS] (default 20) After performing the MCMC trace, one may run the program \path{madai_generate_posterior_points} to generate a sample of points consistent with the posterior distribution.

    \item[PCA\_FRACTION\_RESOLVING\_POWER] (default: 0.95)  When \path{madai_train_emulator} is deciding how many principal components to retain, it considers the total resolving power of the model. The resolving power of a single principal component is $\sqrt{(1 + \lambda_i)}$, where $\lambda_i$ is the eigenvalue associated with that variable. The total resolving power is the product the resolving powers of all of the components. \path{madai_train_emulator} will retain enough components to keep this fraction of the resolving power.

    \item[EMULATOR\_COVARIANCE\_FUNCTION] (default: SQUARE\_EXPONENTIAL\_FUNCTION) See section \ref{sec:CovarianceFunctions}. Allowable values are SQUARE\_EXPONENTIAL\_FUNCTION, \newline POWER\_EXPONENTIAL\_FUNCTION, MATERN\_32\_FUNCTION, and \newline MATERN\_52\_FUNCTION.

    \item[EMULATOR\_REGRESSION\_ORDER] (default: 1) The assumed functional form of the model before Gaussian Process emulation. This is the order of the polynomial. 0, 1, 2, or 3.

    \item[EMULATOR\_NUGGET] (default: 0.001) $\theta_1$. See section \ref{sec:CovarianceFunctions}.

    \item[EMULATOR\_AMPLITUDE] (default: 1) $\theta_0$. See section \ref{sec:CovarianceFunctions}.

    \item[EMULATOR\_SCALE] (default: 0.01) See section \ref{sec:CovarianceFunctions}. The final value for each parameter's scale $\theta_{k+i}$ where $k$ depends on which covariance function has been chosen and $i$ is the parameter index will be EMULATOR\_SCALE times the width of the middle two quartiles of that parameter's prior distribution.

    \item[EMULATOR\_TRAINING\_ALGORITHM] (default: basic) Determines the rigor with which the emulator is trained. Currently, two algorithms are supported.

    \begin{itemize}

      \item The ``basic'' algorithm sets the emulator parameter $\theta_i$ to the interquartile range of the parameter prior distribution.

      \item The ``exhaustive\_geometric\_kfold\_common'' performs an exhaustive search over a range of potential $\theta_i$ hyperparameters using a geometric series to increment between a minimum and maximum possible $\theta_i$ hyperparameter. The minimum value is $4 / N$ where $N$ is the number of training points and the maximum value is 1.0. The common setting of all $\theta_i$ hyperparameters is chosen to maximize a $k$-fold cross validation of the training data.

    \end{itemize}

    \item[SAMPLER] (default: ``MetropolisHastings'') Determines which sampler to use to generate sample points. Available samplers are ``MetropolisHastings'' and ``PercentileGrid''.

    \item[SAMPLER\_NUMBER\_OF\_SAMPLES] (default: 100) How many samples should be generated in a trace. The default is small for testing purposes. Values around $10^6$ are recommended.

    \item[MCMC\_USE\_MODEL\_ERROR] (default: 0) Specifies whether error reported by the model should be used in the log likelihood calculation. Turning this off may make computation of the log likelihood faster at the cost of assuming that the model error is zero for each output.

    \item[MCMC\_NUMBER\_OF\_BURN\_IN\_SAMPLES] (default: 0) The number of samples to be discarded at the beginning of the MCMC run.

    \item[MCMC\_STEP\_SIZE] (default: 0.1) Specifies how big each step should be in the Metropolis-Hastings algorithm. (This will be scaled by the characteristic length of each parameter's prior distribution)

    \item[EXTERNAL\_MODEL\_EXECUTABLE] (default: none) Path to an external executable that follows the protocol for writing to stdout and reading from stdin assumed by the \path{madai_generate_trace} program. If this option is not set, \path{madai_generate_trace} uses a trained emulator to generate the trace.

    \item[EXTERNAL\_MODEL\_ARGUMENTS] (default: none) Arguments to pass to the executable pointed to by EXTERNAL\_MODEL\_EXECUTABLE. All arguments must be specified on a single line.

    \item[EMULATE\_WRITE\_HEADER] (default: 1) Indicates whether \path{madai_emulate} should write out header information when it first starts up. If enabled, emits information to stdout about the emulated model in the format required of executables that talk to \path{madai_generate_trace} when the EXTERNAL\_MODEL\_EXECUTABLE is set. If disabled, this information is not written to stdout and \path{madai_emulate} immediately enters query-response mode.

    \item[POSTERIOR\_ANALYSIS\_DIRECTORY] (default: \path{posterior_model_output}) Specifies the directory where \path{madai_generate_posterior_samples} generates its results. If specified as a relative path, the path is considered to be relative to the statistical analysis directory.
\end{description}


\section{Methods}
\frame{\tableofcontents[currentsection, hideothersubsections]}

\begin{frame}
\frametitle{Methods: Deep DPG}
\begin{figure}
    \centering
    \includegraphics[scale=0.25]{ddpg_algo}
\end{figure}

{\footnotesize
Deep DPG (=DPG+DQN) components:
\begin{itemize}
  \item Deterministic Policy-Gradient (DPG) \cite{Silver2014}
  \item Deep Q-Network (DQN= QLearning + deepLearning) {\scriptsize\cite{Mnih2013}}
  \item Actor-Critic Methods \cite{Sutton1998}
\end{itemize}
}
\end{frame}

\begin{frame}
\frametitle{Methods: DPG \cite{Silver2014}}

DPG: Deterministic Policy Gradient, an actor-critic approach:
\begin{itemize}
  \item actor: $\mu (s|\theta^{\mu})$,
  specifies the current policy by deterministically mapping states to a specific action
  \item critic: $Q(s, a)$,
  learned using the Bellman equation as in Q-learning.
\end{itemize}
\vspace{3mm}

Update $\theta^{\mu}$ by applying the chain rule to the expected return from
the start distribution, $J$, with respect to the actor parameters, $\theta^{\mu}$:
\begin{equation} \label{equ:policy_grad}
\begin{split}
\nabla_{\theta^{\mu}} J &  \approx \mathbb{E}_{s_t \sim \rho^{\beta}} \Big[ \nabla_{\theta^{\mu}} Q(s,a|\theta^Q) |_{s = s_t, a = \mu(s_t|\theta^{\mu})} \Big] \\
  & = \mathbb{E}_{s_t \sim \rho^{\beta}} \Big[ \nabla_{\theta^{\mu}} Q(s,a|\theta^Q) |_{s = s_t, a = \mu(s_t)} \nabla_{\theta^{\mu}} \mu(s|\theta^{\mu})|_{s = s_t} \Big]
\end{split}
\end{equation}

Equ.~\ref{equ:policy_grad} is the policy gradient (the gradient of the policy's performance),
in practice the discount in $\rho^{\beta}$ is ignored.
\end{frame}

\begin{frame}
\frametitle{Methods: Replay-buffer, as in DQN~\cite{Mnih2013}}
Why:
\begin{itemize}
\item NeuralNets assume iid samples;\\
the samples from exploring sequentially in an environment are \textbf{not} iid
\item to make efficient use of hardware optimizations:\\
to learn in minibatches rather than (fully) online
\end{itemize}
\vspace{5mm}

How:
\begin{itemize}
  \item replay buffer, $R$, contains $(s_t, a_t, r_t, s_{t+1})$ sampled from $E$ using an exploration policy
  \item At each timestep the actor and critic are updated by sampling a minibatch uniformly from the buffer
% Because DDPG is an off-policy algorithm, the replay buffer can be large, allowing the algorithm to benefit from learning across a set of uncorrelated transitions.
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Methods: Fix target network, {\footnotesize as in DQN~\cite{Mnih2013}}}
Why:
\begin{itemize}
\item Q-learning (eq.\ref{equ:qloss}) with NeuralNets is unstable, \\
since the network $Q(s, a|\theta^Q)$ being updated is also used in
calculating the target value, $y$
\end{itemize}
\vspace{5mm}

How:
\begin{itemize}
\item {\footnotesize create a copy of the actor and critic networks, i.e.}
  $Q'(s, a|\theta^{Q'})$ and $\mu'(s|\theta^{\mu'})$
\item use $Q'(s, a|\theta^{Q'})$ and $\mu'(s|\theta^{\mu'})$ for calculating the target values
\item target networks update: \\
    $\theta' \leftarrow \tau \theta + (1 - \tau) \theta'$ with $\tau \ll 1$\\
    (warn: may slow learning, since the target network delays the propagation of value estimations, but
    in practice, this was greatly outweighed by the stability of learning)
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Methods: Feature scaling: batch normalization}
Why:
\begin{itemize}
\item different components of the observation may have different physical units (for example, positions versus velocities)
\item their ranges may vary across environments
% \item make it difficult
%   a) for the network to learn effectively
%   b) to find hyper-parameters which generalise across environments with different scales of state values.
% \item to minimize covariance shift during training, by ensuring that each layer receives whitened input
\end{itemize}
\vspace{5mm}

How:
\begin{itemize}
\item normalizes each dimension across the samples in a minibatch to have unit mean and variance.
\item maintains a running average of the mean and variance to use for normalization during testing
(during xploration or evaluation)
\end{itemize}

\end{frame}

\begin{frame}
\frametitle{Methods: Exploration}
Why:\\
Exploration-exploitation dillema
\vspace{5mm}

How:\\
\begin{itemize}
\item Construct an exploration policy $\mu^{xplor}$ by adding noise sampled from
a noise process $\mathcal{N}$ to our actor policy:
$\mu^{xplor} = \mu(s_t | \theta^{\mu}_t) + \mathcal{N}$.\\
\item Here, $\mathcal{N}$ is from Ornstein-Uhlenbeck process,\\
to generate temporally correlated exploration for exploration efficiency in physical control problems with inertia.
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Methods: Deep DPG Algo}
\begin{figure}
    \centering
    \includegraphics[scale=0.35]{ddpg_algo}
\end{figure}
\end{frame}


\section{PGFplotstable}

The figure data i nthe figure was

\begin{figure}
    \includegraphics[width=0.5\textwidth]{example_3}
\end{figure}

\begin{table}[htbp]
    \caption{Generated with PGFplotstable}
    \centering
    \pgfplotstabletypeset[
        col sep=comma,
        every column/.style={fixed zerofill,precision=2},
        columns/0/.style={
            column name={No},
            precision=0},
        columns/alpha/.style={
            column name={$\alpha$},
            precision=2},
        columns/beta/.style={
            column name={$\beta$},
            precision=1},
        every head row/.style={before row=\toprule,after row=\midrule},	% style the first row
        every last row/.style={after row=\bottomrule},	% style the bottom row
        ]{./tabs/example_3.csv}
\end{table}


\section{Copilot Language}
\label{sec:language}

Copilot is embedded into the functional programming language Haskell
\cite{PeytonJones02}, and a working knowledge of Haskell is necessary to use
Copilot effectively. Copilot is a pure declarative language; i.e., expressions
are free of side-effects and satisfies referential transparency.  A program
written in Copilot, which from now on will be referred to as a
\emph{specification}, has a cyclic behavior, where each cycle consists of a
fixed series of steps:

\begin{itemize}
\item Sample external variables, arrays, and functions.
\item Update internal variables.
\item Fire external triggers. (In case the specification is violated.)
\end{itemize}

\noindent We refer to a single cycle as an \emph{iteration}.

All transformation of data in Copilot is propagated through streams.
A stream is an infinite, ordered sequence of values which must conform to the same type.
E.g., we have the stream of Fibonacci numbers:

\begin{center}
$s_{fib} = \{0, 1, 1, 2, 3, 5, 8, 13, 21, \dots \}$
\end{center}

\noindent We denote the $n$th value of the stream $s$ as $s(n)$, and the first
value in a sequence $s$ as $s(0)$. For example, for $s_{fib}$ we have that $s_{fib}(0) = 0$,
$s_{fib}(1) = 1$, $s_{fib}(2) = 1$, and so forth.

Copilot declarations provide an optional type and a definition for an identifier.  For
example,
%
\begin{code}
x :: Stream Int32
x = 5
\end{code}
%
\noindent
gives the identifier {\tt x} the type {\tt Stream Int32} and defines {\tt x} to
be 5.  All Copilot expressions have the type {\tt Stream t}, where {\tt t} is
some base-type for the stream (we discuss Copilot types in
Section~\ref{sec:types}).

Constants as well as arithmetic, boolean, and relational operators are
lifted to work pointwise on streams:

%\begin{minipage}{0.3\textwidth}
\begin{code}
x :: Stream Int32
x = 5 + 5

y :: Stream Int32
y = x * x

z :: Stream Bool
z = x == 10 && y < 200
\end{code}
%\end{minipage}

\noindent Here the streams {\tt x}, {\tt y}, and {\tt z} are simply
\emph{constant streams} such that
$$
\begin{array}{rcl}
\mathtt{x} &\textnormal{ yields } &\{10, 10, 10, \dots \}\\
\mathtt{y} &\textnormal{ yields } &\{100, 100, 100,  \dots \}\\
\mathtt{z} &\textnormal{ yields } &\{\mbox{T},\; \mbox{T},\; \mbox{T},\; \dots \}
\end{array}
$$
Two types of \emph{temporal} operators are provided, one for delaying streams and one for
looking into the future of streams:
%
\begin{code}
(++) :: [a] -> Stream a -> Stream a
drop :: Int -> Stream a -> Stream a
\end{code}
%
Here {\tt xs ++ s} prepends the list {\tt xs} at the front of the stream {\tt
  s}.  For example the stream {\tt w} is defined as follows, given our previous
definition of {\tt x}:
%
\begin{code}
w = [5,6,7] ++ x
\end{code}
%
evaluates to the sequence
$\{5, 6, 7, 10, 10, 10, \dots\}$.
The expression {\tt drop k s} skips the first {\tt k} values of the stream {\tt
  s}, returning the remainder of the stream.
For example we can skip the first two values of {\tt w}:
%
\begin{code}
u = drop 2 w
\end{code}
%
which yields the sequence
$\{7, 10, 10, 10, \dots\}$.

\subsection{Streams as Lazy-Lists} \label{sec:stream}

A key design choice in Copilot is that streams should mimic \emph{lazy lists}.
In Haskell, the lazy-list of natural numbers can be programmed like this:
%
\begin{code}
nats_ll :: [Int32]
nats_ll = [0] ++ zipWith (+) (repeat 1) nats_ll
\end{code}
%
where $\texttt{zipWith}$ is a higher-order function that takes a
function and two lists as parameters and produces a new list formed by
applying the given function to the elements of the two input lists at
the same position in each list. The Haskell function \texttt{repeat}
creates an infinite list with each element having the value of the
first parameter.   As both constants and arithmetic
operators are lifted to work pointwise on streams in Copilot, there is
no need for {\tt zipWith} and {\tt repeat} when specifying the stream
of natural numbers:
%
\begin{code}
nats :: Stream Int32
nats = [0] ++ (1 + nats)
\end{code}
%
In the same manner, the lazy-list of Fibonacci numbers can be specified as follows:
%
\begin{code}
fib_ll :: [Int32]
fib_ll = [1, 1] ++ zipWith (+) fib_ll (drop 1 fib_ll)
\end{code}
%
In Copilot we simply throw away {\tt zipWith}:
\begin{code}
fib :: Stream Int32
fib = [1, 1] ++ (fib + drop 1 fib)
\end{code}

Copilot specifications must be \emph{causal}, informally meaning that
stream values cannot depend on future values.  For example, the following stream
definition is allowed:
%
\begin{code}
f :: Stream Word64
f = [0,1,2] ++ f

g :: Stream Word64
g = drop 2 f
\end{code}
%

But if instead {\tt g} is defined as {\tt g = drop 4 f}, then the definition is
disallowed.  While an analogous stream is definable in a lazy language, we bar
it in Copilot, since it requires future values of {\tt f} to be
generated before producing values for {\tt g}.  This is not possible since
Copilot programs may take inputs in real-time from the environment (see
Section~\ref{sec:interacting}).

\subsection{Functions on Streams} \label{sec:FnOnStreams}

Given that constants and operators work pointwise on streams, we can use Haskell
as a macro-language for defining functions on streams.  The idea of using
Haskell as a macro language is powerful since Haskell is a
general-purpose higher-order functional language.

\begin{example}
We define the function, {\tt even}, which given a stream of
integers returns a boolean stream which is true whenever the input stream
contains an even number, as follows:
%
\begin{code}
even :: Stream Int32 -> Stream Bool
even x = x `mod` 2 == 0
\end{code}
%
Applying {\tt even} on {\tt nats} (defined above) yields the sequence
$\{T, F, T, F, T, F, \dots\}$.
\end{example}

If a function is required to return multiple results, we simply use plain
Haskell tuples:

\begin{example}
We define complex multiplication as follows:
%
\begin{code}
mul_comp
  :: (Stream Double, Stream Double)
  -> (Stream Double, Stream Double)
  -> (Stream Double, Stream Double)
(a, b) `mul_comp` (c, d) = (a * c - b * d, a * d + b * c)
\end{code}
%
Here {\tt a} and {\tt b} represent the real and imaginary part of the left
operand, and {\tt c} and {\tt d} represent the real and imaginary part
of the right operand.
\end{example}

\subsection{Stateful Functions} \label{sec:stateful}

In addition to pure functions, such as {\tt even} and {\tt mul\_comp},
Copilot also facilitates \emph{stateful} functions. A \emph{stateful} function
is a function which has an internal state, e.g. as a latch (as in electronic
circuits) or a low/high-pass filter (as in a DSP).

\begin{figure*}
\begin{minipage}{0.4\linewidth}
\begin{tabular}{c|c||c}
$\mathtt{x}_i$: & $\mathtt{y}_{i-1}$: & $\mathtt{y}_i$:\\
\hline
$F$ & $F$ & $F$ \\
\hline
$F$ & $T$ & $T$ \\
\hline
$T$ & $F$ & $T$ \\
\hline
$T$ & $T$ & $F$
\end{tabular}
\end{minipage}
\hspace{1cm}
\begin{minipage}{0.6\linewidth}
\begin{code}
latch :: Stream Bool -> Stream Bool
latch x = y
  where
  y = if x then not z else z
  z = [False] ++ y
\end{code}
\end{minipage}
\caption{A latch. The specification is provided at the left and the
implementation is provided at the right.}
\label{fig:jk_latch}
\end{figure*}

\begin{example}
We consider a simple latch, as described in \cite{Farhat2004}, with a single
input and a boolean state. Whenever the input is true the internal state is reversed.
The operational behavior and the implementation of the latch is shown in Figure
\ref{fig:jk_latch}.\footnote
{In order
to use conditionals (if-then-else's) in Copilot specifications,
as in Figures~\ref{fig:jk_latch} and~\ref{fig:counter}, the GHC
language extension {\tt RebindableSyntax} must be set on.}
\end{example}

\begin{figure*}
\begin{minipage}{0.4\linewidth}
\begin{tabular}{c|c||c}
$\mathtt{inc}_i$: & $\mathtt{reset}_i$: & $\mathtt{cnt}_i$: \\
\hline
$F$ & $F$ & $\mathtt{cnt}_{i-1}$ \\
\hline
* & $T$ & $0$ \\
\hline
$T$ & $F$ & $\mathtt{cnt}_{i-1} + 1$ \\
\hline
\end{tabular}
\end{minipage}
\hspace{1cm}
\begin{minipage}{0.6\linewidth}
\begin{code}
counter :: Stream Bool -> Stream Bool
        -> Stream Int32
counter inc reset = cnt
  where
  cnt = if reset then 0
          else if inc then z + 1
                 else z
  z = [0] ++ cnt
\end{code}
\end{minipage}
\caption{A resettable counter. The specification is provided at the left and the
implementation is provided at the right.
}
\label{fig:counter}
\end{figure*}

\begin{example}
We consider a resettable counter with two inputs, {\tt inc} and {\tt reset}.
The input {\tt inc} increments the counter and the input {\tt reset} resets the
counter. The internal state of the counter, {\tt cnt}, represents the value of the
counter and is initially set to zero. At each cycle, $i$, the value of
$\mathtt{cnt}_i$ is determined as shown in the left table in Figure
\ref{fig:counter}.
\end{example}

%\begin{figure}
%\begin{code}
%fir2pole :: Double -> Double -> Double -> Double
%  -> Double -> Sig Double -> Sig Double
%fir2pole a1 a2 b0 b1 b2 x0 = y0
%  where
%    y0 = - (constant a1)*y1 - (constant a2)*y2
%         + (constant b0)*x0 + (constant b1)*x1 + (constant b2)*x2
%    x2 = [0, 0] ++ x0 ; x1 = drop 1 x2
%    y2 = [0, 0] ++ y0 ; y1 = drop 1 y2
%\end{code}
%\caption{A $2$-pole IIR filter.}
%\label{fig:2_pole_iir_filter}
%\end{figure}

\subsection{Types} \label{sec:types}

Copilot is a typed language, where types are enforced by the Haskell
type system to ensure generated C programs are well-typed.  Copilot is
\emph{strongly typed} (i.e., type-incorrect function application is
not possible) and \emph{statically typed} (i.e., type-checking is done
at compile-time).  The base types are Booleans, unsigned and signed
words of width 8, 16, 32, and 64, floats, and doubles.  All elements
of a stream must belong to the same base type.  These types have
instances for the class {\tt Typed a}, used to constrain Copilot
programs. The constraining type classes are listed to the left of the
$\texttt{=>}$ in a function definition.

We provide a {\tt cast} operator
%
\begin{code}
cast :: (Typed a, Typed b) => Stream a -> Stream b
\end{code}
%
that casts from one type to another.  The cast operator is only defined for
casts that do not lose information, so an unsigned word-type {\tt a} can only be
cast to another unsigned type at least as large as {\tt a} or to a signed word
type strictly larger than {\tt a}.  Signed types cannot be cast to unsigned
types but can be cast to signed types at least as large.

\subsection{Interacting With the Target Program}
\label{sec:interacting}

All interaction with the outside world is done by sampling
\emph{external symbols} and by evoking \emph{triggers}.  External
symbols are symbols that are defined outside Copilot and which reflect
the visible state of the target program that we are monitoring.  They
include variables, arrays, and functions (with a non-void return
type).  Analogously, triggers are functions that are defined outside
Copilot and which are evoked when Copilot needs to report that the
target program has violated a specification constraint.

\paragraph{Sampling.}
A Copilot specification is \emph{open} if defined with external
symbols in the sense that values must be provided externally at
runtime.  To simplify writing Copilot specifications that can be
interpreted and tested, constructs for external symbols take an
optional environment for interpretation.

External variables are defined by using the {\tt extern} construct:
%
\begin{code}
extern :: Typed a => String -> Maybe [a] -> Stream a
\end{code}
%
\noindent
where a value of type $\texttt{Maybe a }$ is either empty ($\texttt{Nothing}$)
or a value of type $\texttt{a}$ ($\texttt{Just a}$).  The function
\texttt{extern} takes the name of an external variable and a possible (Haskell)
list of values to serve as the ``environment'' for the interpreter, since in a
compiled monitor, these variable values come from the sampled program.  The
interpreter generates a stream by sampling the variable at each clock cycle,
modeled as the elements in the list.  For example,
%
\begin{code}
sumExterns :: Stream Word64
sumExterns = let ex1 = extern "e1" (Just [0..])
                 ex2 = extern "e2" Nothing
             in  ex1 + ex2
\end{code}
%
is a stream that takes two external variables {\tt e1} and {\tt e2} and adds
them.  The first external variable contains the infinite list {\tt [0,1,2,...]}
of values for use when interpreting a Copilot specification containing the
stream.  The other variable contains no environment.  ({\tt sumExterns} must have
an environment for both of its variables to be interpreted).  If {\tt ex1} and
{\tt ex2} both were defined with the lists {\tt [0,1,2,...]}, then the output of
interpreting {\tt sumExterns} would be {\tt [0,2,4,...]}.

Sometimes, type inference cannot infer the type of an external variable.  For
example, in the stream definition
%
\begin{code}
extEven :: Stream Bool
extEven = e0 `mod` 2 == 0
  where e0 = extern "x" Nothing
\end{code}
%
\noindent
the type of {\tt extern "x"} is ambiguous, since it cannot be inferred from a
Boolean stream and we have not given an explicit type signature.  For
convenience, typed {\tt extern} functions are provided, e.g., {\tt externW8} or
{\tt externI64} denoting an external unsigned 8-bit word or signed 64-bit word,
respectively.  In general it is best practice to define external symbols with
top-level definitions, e.g.,
%
\begin{code}
e0 :: Stream Word8
e0 = extern "e0" (Just [2,4..])
\end{code}

\noindent
so that the symbol name and its environment can be shared between streams.

Besides variables, external arrays and arbitrary functions can be sampled.  The
external array construct has the type
%
\begin{code}
externArray :: (Typed a, Typed b, Integral a)
            => String -> Stream a -> Int
            -> Maybe [[a]] -> Stream b
\end{code}
%
The construct takes (1) the name of an array, (2) a stream that generates indexes for
the array (of integral type), (3) the fixed size of the array, and (4) possibly a list of lists that is the
environment for the external array, representing the sequence of array values.  For example,
%
\begin{code}

extArr :: Stream Word32
extArr = externArray "arr1" arrIdx size
           (Just $ repeat (permutations [0,1,2]))
  where
  arrIdx :: Stream Word8
  arrIdx = [0] ++ (arrIdx + 1) `mod` size

  size = 3
\end{code}
% $
{\tt extArr} is a stream of values drawn from an external array containing
32-bit unsigned words.  The array is indexed by an 8-bit variable.  The index
is ensured to be less than three by using modulo arithmetic.  The environment
provided produces an infinite list of all the permutations of the list {\tt
  [0,1,2]}.\footnote{The function {\tt permutations} comes from the Haskell
    standard library {\tt Data.list}.}

%% %
%% \begin{code}
%% externArray :: (Typed a, Typed b, Integral a) => String -> Stream a -> Stream b
%% externFun :: Typed a => String -> [FunArg] -> Stream a
%% arg :: Typed a => Stream a -> FunArg
%% \end{code}
%% %
%% The constructor \texttt{externArray} takes two arguments: the name of array and
%% an index into the array.  The index is given by a Copilot stream (of integral
%% type) that is used as an index into the table.

%
\begin{example}
\label{exm:e}
Say we have defined a lookup-table (in C99) of a discretized continuous function that we want to use
within Copilot:
%
\begin{code}
double someTable[42] = { 3.5, 3.7, 4.5, ... };
\end{code}
%
We can use the table in a Copilot specification as follows:
%
\begin{code}
lookupSomeTable :: Stream Word16 -> Stream Double
lookupSomeTable idx =
  externArray "someTable" idx 42 Nothing
\end{code}
%
Given the following values for \texttt{idx}, $\{1, 0, 2, 2, 1, \dots \}$, the output of
\texttt{lookupSomeTable idx} would be $$\{3.7, 3.5, 4.5, 4.5, 3.7, \dots \}$$
\end{example}

Finally, the constructor \texttt{externFun} takes (1) a function name, (2) a
list of arguments, and (3) a possible list of values to provide its environment.
%
\begin{code}
externFun :: Typed a => String -> [FunArg]
          -> Maybe [a] -> Stream a
\end{code}
%
Each argument to an external function is given by a Copilot stream.  For
example,
%
\begin{code}
func :: Stream Word16
func = externFun "f" [arg e0, arg nats] Nothing
  where
  e0 = externW8 "x" Nothing
  nats :: Stream Word8
  nats = [0] ++ nats + 1
\end{code}
% $
samples a function in C that has the prototype
%
\begin{code}
uint16_t f(uint8_t x, uint8_t nats);
\end{code}
%

Both external arrays and functions must, like external variables, be defined
in the target program that is monitored. Additionally, external functions must be
without side effects, so that the monitor does not cause undesired side-effects when
sampling functions.  Finally, to ensure Copilot sampling is not
order-dependent, external functions cannot contain streams containing other
external functions or external arrays in their arguments, and external arrays
cannot contain streams containing external functions or external arrays in their
indexes.  They can both take external variables, however.

\paragraph{Triggers.}
Triggers, the only mechanism for Copilot streams to effect the outside world,
are defined by using the {\tt trigger construct}:
%
\begin{code}
trigger :: String -> Stream Bool -> [TriggerArg] -> Spec
\end{code}
%
The first parameter is the name of the external function, the second parameter is the
guard which determines when the trigger should be evoked, and the third parameter
is a list of arguments which is passed to the trigger when evoked.
Triggers can be combined into a specification by using the \emph{do}-notation:
%
\begin{code}
spec :: Spec
spec = do
  trigger "f" (even nats) [arg fib, arg (nats * nats)]
  trigger "g" (fib > 10) []
  let x = externW32 "x" Nothing
  trigger "h" (x < 10) [arg x]
\end{code}
%
The order in which the triggers are defined is irrelevant.

\begin{example}
\label{exm:engine}
We consider an engine controller with the following property: If the temperature
rises more than 2.3 degrees within 0.2 seconds, then the fuel injector should
not be running.  Assuming that the global sample rate is 0.1 seconds, we can
define a monitor that surveys the above property:
%
\begin{code}
propTempRiseShutOff :: Spec
propTempRiseShutOff =
  trigger "over_temp_rise"
    (overTempRise && running) []

  where
  max = 500 -- maximum engine temperature

  temps :: Stream Float
  temps = [max, max, max] ++ temp

  temp = extern "temp" Nothing

  overTempRise :: Stream Bool
  overTempRise = drop 2 temps > (2.3 + temps)

  running :: Stream Bool
  running = extern "running" Nothing
\end{code}
%

Here, we assume that the external variable {\tt temp} denotes the temperature of
the engine and the external variable {\tt running} indicates whether the fuel
injector is running.  The external function {\tt over\_temp\_rise} is called
without any arguments if the temperature rises more than 2.3 degrees within 0.2
seconds and the engine has not been shut off.  Notice there is a latency of one
tick between when the property is violated and when the guard becomes true.
\end{example}

\subsection{Explicit Sharing}
\label{sec:explicit_sharing}

\begin{figure}[ht]
\begin{minipage}{0.25\textwidth}
\begin{code}
s1 = let x = nats + nats
     in x * x
\end{code}
\end{minipage}%
\begin{minipage}{0.25\textwidth}
\begin{code}
s2 = local (nats + nats)
      (\x -> x * x)
\end{code}
\end{minipage}
\caption{Implicit sharing (s1) versus explicit sharing (s2).}
\label{fig:explicit_sharing}
\end{figure}

\noindent Copilot facilitates sharing in expressions by the \emph{local}-construct:
%
\begin{code}
local
  :: (Typed a, Typed b)
  => Stream a
  -> (Stream a -> Stream b)
  -> Stream b
\end{code}
%
The local construct works similar to \emph{let}-bindings in Haskell.  For
example, from a semantic point of view, the streams {\tt s1} and {\tt s2} in
Figure~\ref{fig:explicit_sharing} are equivalent.  However, by sharing
expression definitions, the size of the Copilot program can be dramatically
reduced.  Printing the expressions results in something like the following:
%
\begin{code}
s1 = (nats + nats) * (nats + nats)
s2 = let x = nats + nats in
     x * x
\end{code}
%
\noindent
Note that in {\tt s1}, because the sharing is at the \emph{Haskell} level, the
expressions are inlined, resulting in replication in the resulting expression.

\subsection{Extended Example: The Boyer-Moore Majority-Vote Algorithm}
\label{subsec:boyer_moore}

In this section we demonstrate how to use Haskell as an advanced macro language
on top of Copilot by implementing an algorithm for solving the voting problem
in Copilot.

\begin{figure*}
\begin{code}
majorityPure :: Eq a => [a] -> a
majorityPure []     = error "majorityPure: empty list!"
majorityPure (x:xs) = majorityPure' xs x 1

majorityPure' []     can _   = can
majorityPure' (x:xs) can cnt =
  let
    can' = if cnt == 0 then x else can
    cnt' = if cnt == 0 || x == can then succ cnt else pred cnt
  in
    majorityPure' xs can' cnt'
\end{code}
\caption{The first pass of the majority vote algorithm in Haskell.}
\label{fig:majority_pure}
\end{figure*}

\begin{figure*}
\begin{code}
aMajorityPure :: Eq a => [a] -> a -> Bool
aMajorityPure xs can = aMajorityPure' 0 xs can > length xs `div` 2

aMajorityPure' cnt []     _   = cnt
aMajorityPure' cnt (x:xs) can =
  let
    cnt' = if x == can then cnt+1 else cnt
  in
    aMajorityPure' cnt' xs can
\end{code}
\caption{The second pass of the majority vote algorithm in Haskell.}
\label{fig:amajority_pure}
\end{figure*}

Reliability in mission critical software is often improved by replicating
the same computations on separate hardware and by doing a vote in the end
based on the output of each system. The majority vote problem consists of
determining if in a given list of votes there is a candidate that has more
than half of the votes, and if so, of finding this candidate.

The Boyer-Moore Majority Vote Algorithm \cite{MooreBoyer82,Hesselink2005} solves
the problem in linear time and constant memory. It does so in two passes: The
first pass chooses a candidate; and the second pass asserts that the
found candidate indeed holds a majority.

Without going into details of the algorithm, the first pass can be implemented
in Haskell as shown in Figure \ref{fig:majority_pure}. The second pass, which
simply checks that a candidate has more than half of the votes, is
straightforward to implement and is shown in Figure \ref{fig:amajority_pure}.
E.g. applying {\tt majorityPure} on the string {\tt AAACCBBCCCBCC} yields {\tt
  C}, which {\tt aMajorityPure} can confirm is in fact a majority.

\begin{figure*}[ht]
\begin{code}
majority :: (Eq a, Typed a) => [Stream a] -> Stream a
majority []     = error "majority: empty list!"
majority (x:xs) = majority' xs x 1

majority' []     can _   = can
majority' (x:xs) can cnt =
  local
    (if cnt == 0 then x else can) $
      \ can' ->
        local (if cnt == 0 || x == can then cnt+1 else cnt-1) $
          \ cnt' ->
            majority' xs can' cnt'
\end{code}
\caption{The first pass of the majority vote algorithm in Copilot.}
\label{fig:majority}
\end{figure*}

\begin{figure*}[ht]
\begin{code}
aMajority :: (Eq a, Typed a) => [Stream a] -> Stream a -> Stream Bool
aMajority xs can = aMajority' 0 xs can > (fromIntegral (length xs) `div` 2)

aMajority' cnt []     _   = cnt
aMajority' cnt (x:xs) can =
  local
    (if x == can then cnt+1 else cnt) $
      \ cnt' ->
        aMajority' cnt' xs can
\end{code}
\caption{The second pass of the majority vote algorithm in Copilot.}
\label{fig:amajority}
\end{figure*}
% $

When implementing the majority vote algorithm for Copilot, we can use reuse
almost all of the code from the Haskell implementation. However, as functions in
Copilot are macros that are expanded at compile time, care must be taken in
order to avoid an explosion in the expression size; thus, note the use of the
{\tt local} construct.  The Copilot implementations of the first and the second
pass are given in Figure \ref{fig:majority} and Figure \ref{fig:amajority}
respectively. Comparing the Haskell implementation with the Copilot
implementation, we see that the code is almost identical, except for the type
signatures and the explicit sharing annotations.

\section{Tools} \label{sec:tools}

\begin{figure*}[ht]
\begin{center}
  \begin{tikzpicture}[->, node distance=3cm, auto, shorten >=1pt, bend angle=45,
      thick]
    \tikzstyle{every state}=[rectangle, rounded corners]

    \node[state]         (Int)            {Interpreter};
    \node[state]         (Lang) [above right of=Int]
         {
           \begin{tabular}[b]{l}
           Copilot Libraries\\ \hline Copilot Language
           \end{tabular}};
    \node[state]         (Core) [below right of=Lang]          {Copilot Core};
    \node[state]         (PP) [right of=Core]          {Pretty Printer};

    \node[state]         (Atom) [below left of=Core]     {Atom Back-End};
    \node[state]         (SBV) [below right of=Core]     {SBV Back-End};
    \node[state]         (C99A) [below of=Atom]     {C99};
    \node[state]         (C99S) [below of=SBV]     {C99};

    \tikzstyle{every node}=[]

    \path %% (Libs) edge              node {0,1,L} (Lang);
    %% edge                 node {1,1,R} (C)
    (Lang) edge   [bend left, anchor=west, text width=2.5cm] node {Reification and DSL-specific type-checking} (Core)
    %% edge                 node {0,1,L} (C)
    (Core) edge              node {Translation} (Atom)
    edge              node {} (SBV)
    edge              node {} (PP)
    edge              node [swap] {Evaluation} (Int)
    (Int) edge              [<->, bend right] node {QuickCheck} (Atom)
    (Atom) edge             node {Compilation} (C99A)
    (SBV) edge             node {Compilation} (C99S)
    (C99A) edge              [<->] node  {Model checking} (C99S);

    %% edge [bend left]     node {Translation} (SBV)
    %% (Atom) edge [loop below] node {1,1,R} (D)
    %%     edge                 node {0,1,R} (Libs)
    %% (SBV) edge [bend left]   node {1,0,R} ();
  \end{tikzpicture}
\end{center}
  \caption{The Copilot toolchain.}
  \label{fig:toolchain}
\end{figure*}

As a framework for runtime verification, Copilot comes with a variety of tools
to support the generation of runtime monitor code as well as to assist in
assurance of the monitors. The Copilot toolchain is depicted in
Figure~\ref{fig:toolchain}.  A Copilot program is transformed into the Copliot
Core language via process called reification. An interpreter providing an
executable semantics for the Core language is provided as an integral part of
the toolchain. The interpreter can be used to prototype Copilot programs as well
as for verification and validation purposes. Reified programs can be printed
using the custom pretty-printing tool.  Currently, two back-ends are included in
the toolchain, which we describe in Section~\ref{sec:compiling}.Both back-ends
translate Copilot Core-language programs into a Haskell-hosted embedded
domain-specific language (eDSL) for C code
generation.  In addition, a custom Quickcheck engine and a test harness are
provided along with support for model checkers to aide in verifying the
generated C monitors.  We will demonstrate these tools and their usage in the
remainder of this section.  A more thorough discussion about the verification
approach can be found in~\cite{PikeWNG2012}.

%%Copilot comes with a variety of tools, including a pretty-printer, an interpreter,
%%two compilers targeting C, and a verifier front-end. In the following section, we will
%%demonstrate some of these tools and their usage.

\subsection{Pretty-Printing} \label{sec:pretty-printing}
Pretty-printing is straightforward.  For some specification {\tt spec},
%
\begin{code}
prettyPrint spec
\end{code}
%
\noindent
returns the specification after static macro expansion.  Pretty-printing can
provide some indication about the complexity of the specification to be
evaluated.  Specifications that are built by recursive Haskell programs (e.g.,
the majority voting example in Section~\ref{subsec:boyer_moore}) can generate
expressions that are large.  Large expressions can take significant
time to interpret or compile.

\subsection{Interpreting Copilot}

The copilot interpreter is invoked as follows (e.g. within GHCI, the GHC compiler's
interpreter for Haskell):
%
\begin{code}
GHCI> interpret 10 propTempRiseShutOff
\end{code}
%
The first argument to the function \emph{interpret} is the number of iterations that we want to evaluate.
The third argument is the specification (of type {\tt Spec}) that we wish to interpret.

The interpreter outputs the values of the arguments passed to the trigger, if
its guard is true, and {\tt --} otherwise.  For example, consider the following
Copilot program:
%
\begin{code}
spec = do
  trigger "trigger1" (even nats) [ arg nats
                                 , arg (odd nats)]
  trigger "trigger2" (odd nats) [arg nats]
\end{code}
% $
where {\tt nats} is the stream of natural numbers, and {\tt even} and {\tt odd}
are functions that take a stream and return whether the point-wise values are
even or odd, respectively.  The output of
%
\begin{code}
interpret 10 spec
\end{code}
%
is as follows:
%
\begin{code}
trigger1:   trigger2:
(0,false)  --
--         (1)
(2,false)  --
--         (3)
(4,false)  --
--         (5)
(6,false)  --
--         (7)
(8,false)  --
--         (9)
\end{code}
%

Sometimes it is convenient to observe the behavior of a stream without defining
a trigger.  We can do so declaring an \emph{observer}.  For example:
%
\begin{code}
spec :: Spec
spec = observer "obs" nats
\end{code}
%
can be interpreted using
%
\begin{code}
interpret 5 spec
\end{code}
%
as usual.  Observers can be combined in larger Copilot programs.  For example,
consider the following:
%
\begin{code}
spec :: Spec
spec = do
  let x = externW8 "x" (Just [0..])
  trigger "trigger" true [arg $ x < 3]
  observer "debug_x" x
\end{code}
% $
Interpreting {\tt spec} as follows
%
\begin{code}
interpret 10 spec
\end{code}
%
yields
%
\begin{code}
trigger:  debug_x:
(true)    0
(true)    1
(true)    2
(false)   3
(false)   4
(false)   5
(false)   6
(false)   7
(false)   8
(false)   9
\end{code}

\subsection{Compiling Copilot}  \label{sec:compiling}

Compiling the engine controller from Example \ref{exm:engine} is
straightforward.  First, we pick a back-end to compile to.  Currently, two
back-ends are implemented, both of which generate constant-time and
constant-space C code.  One back-end is called \emph{copilot-c99} and targets
the Atom language,\footnote{\url{http://hackage.haskell.org/package/atom}}
originally developed by Tom Hawkins at Eaton Corp. for developing control
systems.  The second back-end is called \emph{copilot-sbv} and targets the SBV
language\footnote{\url{http://hackage.haskell.org/package/sbv}}, originally
developed by Levent Erk\"{o}k.  SBV is primarily used as an interface to SMT
solvers~\cite{smt} and also contains a C-code generator.  Both
languages are open-source.

The two back-ends are installed with Copilot, and they can be imported,
respectively, as

\begin{code}
import Copilot.Compile.C99
\end{code}
\noindent
and
\begin{code}
import Copilot.Compile.SBV
\end{code}

After importing a back-end, the interface for compiling is as
follows:\footnote{Two explanations are in order: (1) {\tt reify} allows sharing in the expressions to be compiled~\cite{DSLExtract}, and {\tt >>=} is a higher-order
  operator that takes the result of reification and ``feeds'' it to the compile
  function.}
%
\begin{code}
reify spec >>= compile defaultParams
\end{code}
%
\noindent
(The compile function takes a parameter to rename the generated C files; {\tt
  defaultParams} is the default, in which there is no renaming.)

The compiler now generates two files:

\begin{itemize}
\item ``copilot.c'' ---
\item ``copilot.h'' ---
\end{itemize}

The file named ``copilot.h'' contains prototypes for all external variables, functions, and arrays,
and contains a prototype for the ``step''-functions which evaluates a single iteration.

\begin{code}
/* Generated by Copilot Core v. 0.1 */

#include <stdint.h>
#include <stdbool.h>

/* Triggers (must be defined by user): */

void over_temp_rise();

/* External variables (must be defined by user): */

extern float temp;
extern bool running;

/* Step function: */

void step();
\end{code}

Using the prototypes in ``copilot.h'' we can build a driver as follows:

\begin{code}
/* driver.c */
#include <stdio.h>
#include "copilot.h"

bool running = true;
float temp = 1.1;

void over_temp_rise()
{
  printf("The trigger has been evoked!\n");
}

int main (int argc, char const *argv[])
{
  int i;

  for (i = 0; i < 10; i++)
  {
    printf("iteration: %d\n", i);
    temp = temp * 1.3;
    step();
  }

  return 0;
}
\end{code}

Running ``gcc copilot.c driver.c -o prop'' gives a program ``prop'', which when executed
yields the following output:
%
\begin{code}
iteration: 0
iteration: 1
iteration: 2
iteration: 3
iteration: 4
iteration: 5
iteration: 6
iteration: 7
The trigger has been evoked!
iteration: 8
The trigger has been evoked!
iteration: 9
The trigger has been evoked!
\end{code}
%

\subsection{QuickCheck}
QuickCheck~\cite{qc} is a library originally developed for Haskell such that
given a property, it generates random inputs to test the property.  We provide a
similar tool for checking Copilot specifications.  Currently, the tool is
implemented to check the copilot-c99 back-end against the interpreter.  The tool
generates a random Copilot specification, and for some user-defined number of
iterations, the output of the interpreter is compared against the output of the
compiled C program.  The user can specify weights to influence the probability
at which expressions are generated.

The copilot QuickCheck tool is installed with Copilot and assuming the binary is
in the path, it is executed as

\begin{code}
copilot-c99-qc
\end{code}

%%%
%If you have installed Copilot, you can execute the quickCheck tests by executing
%the program {\tt CopilotC99Test}.  The default installation for the executable
%is in {\tt \$HOME/.cabal/bin}. Assuming the executable is in your path, simply
%execute it.  It will direct the user to enter the number of specifications to
%test.  The program will then generate that many random specifications, testing
%the output of the interpreter against the executed C program.  By default, it
%tests the outputs for ten iterations.

\subsection{Verification} \label{sec:verification}
``Who watches the watchmen?''  Nobody.  For this reason, monitors in
ultra-critical systems are the last line of defense and cannot fail.  Here, we
outline our approach to generate high-assurance monitors.  First, as mentioned,
the compiler is statically and strongly typed, and by implementing an eDSL, much
of the infrastructure of a well-tested Haskell implementation is reused.  We
have described our custom QuickCheck engine.  We have tested millions of
randomly-generated programs between the compiler and interpreter with this approach.

Additionally, Copilot includes a tool to generate a driver to prove the
equivalence between the copilot-c99 and copilot-sbv back-ends that each generate
C~code (similar drivers are planned for future back-ends).  To use the driver,
first import the following module:
%
\begin{code}
import qualified Copilot.Tools.CBMC as C
\end{code}
%
\noindent
(We import it using the {\tt qualified} keyword to ensure no name space
collisions.)  Then in GHCI, just like with compilation, we execute
%
\begin{code}
reify spec >>= C.genCBMC C.defaultParams
\end{code}
%
\noindent
This generates two sets of C sources, one compiled through the copilot-c99
back-end and one through the copilot-sbv back-end.  In addition, a driver (that is, a
{\tt main} function) is generated that executes the code from each back-end.  The
driver has the following form:
%
\begin{code}
int main (int argc, char const *argv[])
{
  int i;

  for (i = 0; i < 10; i++)
  {
    sampleExterns();
    atm_step();
    sbv_step();
    assert(atm_i == sbv_i);
  }

  return 0;
}
\end{code}
%
This driver executes the two generated programs for ten iterations, which is the
default value.  That default can be changed; for example:
%
\begin{code}
reify spec >>=
  C.genCBMC C.defaultParams {C.numIterations = 20}
\end{code}
%
\noindent
The above executes the generated programs for 20 executions.

The verification depends on an open-source model-checker for C source-code
originally developed at Carnegie Mellon University~\cite{cbmc}.  A license for
the tool is available.\footnote{\url{http://www.cprover.org/cbmc/LICENSE}.  It
  is the user's responsibility to ensure their use conforms to the license.}
CBMC must be downloaded and installed separately; CBMC is actively maintained
at the time of writing, and is available for Windows, Linux, and Mac OS.

CBMC symbolically executes a program.  With different options, CBMC can be used
to check for arithmetic overflow, buffer overflow/underflow, floating-point NaN
results, and division by zero.  Additionally, CBMC can attempt to verify
arbitrary {\tt assert()} statements placed in the code.  In our case, we wish to
verify that on each iteration, for the same input variables, the two back-ends
have the same state.

CBMC proves that for all possible inputs, the two programs have the same outputs
for the number of iterations specified.  The time-complexity of CBMC is
exponential with respect to the number of iterations.  Furthermore, CBMC cannot
guarantee equivalence beyond the fixed number of iterations.

After generating the two sets of C source files, CBMC can be executed on the
file containing the driver; for example,
%
\begin{code}
cbmc cbmc_driver.c
\end{code}
%


% !TEX root = ../main.tex

\chapter{Data}
\thispagestyle{myheadings}

 This paper utilises tick-by-tick data from cryptocurrency exchange Bitfinex\footnote{https://www.bitfinex.com/} collected by Bitcoinchart \footnote{https://bitcoincharts.com/}. Bitfinex is one of the cryptocurrency exchange with the greatest liquidity founded since 2012. It is famous for providing numerous altcoins\footnote{Cryptocurrencies other than Bitcoin}. Given that the history and market share of Bitfinex, it is considered as a proper benchmark for deriving Bitcoin volatility.\\

The origin data includes each tick in the form of Unix timestamp, price and quantity for each order. Raw data is segmented into 10 min, 30 min, 60 min and 120 min intervals. Firstly, all tick-by-tick data is segmented according to Unix timestamp. It is possible that some segments contain thousands of trades information, while some have none. Price for calculating return in Figure \ref{calReturn}, is to derive the first trade\footnote{In this case is Trade 1 from Figure \ref{calReturn}} and last trade\footnote{Correspondingly Trade 2 from Figure \ref{calReturn}} that happened during this specific segment. A rare case, none trade happened at that time range, the return of this specific segment is defined as zero for presenting no-fluctuation. \\

\begin{figure}[H]
\includegraphics[width=15.8cm]{figures/calReturn}
\caption{Calculation of Return}
\label{calReturn}
\end{figure}

\section{Realized Volatility}

Consider Bitcoin price at time t as $p_t$, derivative in logarithmic $P_t$ is consists of instantaneous volatility and drift.
$$\log(p_t)=\mu_t df+\delta_t d\omega_t$$

Assuming no jump in price, Where $\delta_t$ and $\mu_t$ stands for instantaneous volatility and drift respectively, $\omega_t$ is Brownian motion. In forecasting latent Integrated Variance (IV), the daily Integrated Variance is defined as below:

$$IV_t=\int_{t-1}^t\delta_s^2 \, ds$$

Given that the integrated variance is not directly observable. Hereby this essay defines Realized Volatility as:
$$RV_t=\sum_{i=1}^Mr_{t,i}^2$$\\
Where $r_{t,i}=\log(p_{t-1+i\Delta})-\log(p_{t-1+(i-1)\Delta})$, and $M=\frac{1}{\Delta}$, in that $\Delta$ is the amount of fragments of each day. With the approximation of $\Delta$ to 0. RV is a consistent estimation of IV according to proposition of \cite{andersen_answering_1998}.\\

\begin{figure}[h]
\centering
\begin{minipage}[h]{0.48\textwidth}
\centering
\includegraphics[width=6cm]{figures/RVdist}
\caption{Distribution of Realized Volatility}
\label{Dist_RV}
\end{minipage}
\begin{minipage}[h]{0.48\textwidth}
\centering
\includegraphics[width=6cm]{figures/logRVdist}
\caption{Logarithmized RV}
\label{log_RV}
\end{minipage}
\end{figure}

Figure \ref{Dist_RV} is Distribution of Realized Volatility. It clearly shows that the left-skewness. Left-skewness Realized Volatility is solved is by logarithm in Figure \ref{log_RV}.\\

\newpage

\begin{wrapfigure}{l}[-20pt]{0.48\textwidth}
\vspace{-20pt}
\centering
    \includegraphics[width=0.48\textwidth]{figures/QQplot}
\vspace{-10pt}
\caption{QQ plot for log RV}
\end{wrapfigure}

For further examination of the normal distribution, quantile-quantile plot is drawn. Log (RV) generally fits normal distribution with extra extremum on both ends, which meets the expectation of existing structure breaks in the Bitcoin market.\\

~\\

To determine the Autoregressive process, this paper uses Autocorrelation Function, and Partial Autocorrelation Function developed by \cite{box_time_2016} and \cite{davis_extremogram_2009}, from where it is for the Autoregressive process is concentrated on the second day and first week. This confirms the long-memory behaviour of RV of Bitcoins. Therefore the HAR model is suitable to use.\\

\begin{figure}[h]
\centering
\begin{minipage}[h]{0.48\textwidth}
\centering
\includegraphics[width=6cm]{figures/ACF}
\caption{Autocorrelation Function}\label{ACF}
\end{minipage}
\begin{minipage}[h]{0.48\textwidth}
\centering
\includegraphics[width=6cm]{figures/PACF}
\caption{Partial Autocorrelation Function}\label{PACF}
\end{minipage}
\end{figure}

\section{Realized Quarticity}

In reality, there is a limitation of fragmentation in intraday data. Realized Volatility is asymptotically evaluated by:
$$RV_t = IV_t + \eta_t, \eta_t\sim{MN(0, 2\Delta IQ_t)}$$
Where MN stands for Mixed Normal distribution and $IQ_t=\int_{t-1}^t\delta_s^4\, ds$ based on what stated by \cite{barndorffnielsen_econometric_2002}. $\eta$ is measurement error of Intergraded Volatility from Realized Volatility. Also, when $\Delta$ approximates to 0, Realized Quarticity is a consistent estimation of Intergraded Quarticity. Same in Realized Volatility, $M=\frac{1}{\Delta}$, and $\Delta$ is the amount of fragments of each day.

$$RQ_t=\frac{M}{3}\sum_{i=1}^{M}r_{t,i}^4$$

In Table \ref{rvrq_descriptive_data} presents descriptive Data of Realized Volatility and Realized Quarticity. It is worth mentioning the markedness of the standard deviation and in-sample range of both Realized Volatility and Realized Quarticity.

\begin{table}[H]
\centering
\setlength{\tabcolsep}{10 mm}{
\begin{tabular}{|l|l|l|}
\hline
         & Realized Volatility & Realize Quarticity  \\
\hline
Count    & 1298.000000         & 1298.000000         \\
\hline
Mean     & 0.003220            & 9.005061E-08        \\
\hline
std      & 0.014603            & 1.746010E-06        \\
\hline
min      & 0.000018            & 8.709702E-14        \\
\hline
25\%     & 0.000267            & 9.855636E-12        \\
\hline
50\%     & 0.000735            & 7.302184E-11        \\
\hline
75\%     & 0.002028            & 5.553551E-10        \\
\hline
max      & 0.368688            & 4.580565E-05        \\
\hline
Skewness & 16.170590           & 24.779420           \\
\hline
Kurtosis & 343.737437          & 623.184983          \\
\hline
\end{tabular}}
\caption{Realized Volatility and Realized Quarticity Descriptive Data}
\label{rvrq_descriptive_data}
\end{table}

\section{Metrics of Cryptocurrency}

Furthermore, metrics of cryptocurrency data are collected from coinmetrics\footnote{https://coinmetrics.io/, an open-source cryptoasset analytics website.}. In this research, all metrics available from this site is examined for sorting out indices that may help in predicting Realized Volatility in one-day latency. Definitions of metrics are described as \cite{thecoinmetricsteam_introducing_2018} :

\paragraph*{txVolume(USD) and adjustedTxVolume(USD)}

adjustedTxVolume(USD) equals to txVolume (transaction volume in USD) that removed mixers\footnote{Mix a batch of cryptocurrencies to reduce traceability}, self-churn\footnote{Outputs spent within 4 blocks of their first expenditure are subtracted to avoid double-counting according to the early spent output heuristic from \cite{arvind_bitcoin_nodate}}, privacy enhancements, spam, and change outputs (in UTXO\footnote{Unspent transaction output} chains)

\paragraph*{txCount and paymentCount}
Transaction Count refers to the number of transactions happening on the public blockchain daily. Additionally, txCount underestimates UTXO network, for example, Bitcoin can batch a bunch of transactions into one. Payment counting for UTXO coins is defined as the sum of outputs’ count minus one for each transaction.

\paragraph*{marketCap(USD) and realizedCap(USD)}
Market cap is a calculation of price multiples theoretical total Bitcoin amount. While realizedCap refers to the existing Bitcoins' market cap.

\paragraph*{exchangeVolume(USD) and medianTxValue(USD)}
The dollar value of the volume at exchanges like GDAX\footnote{https://pro.coinbase.com/} and Bitfinex. medianTxValue(USD) is the median value of Bitcoin transaction.

\paragraph*{generatedCoins and averageDifficulty}
generatedCoins refers to the number of new coins that have been brought into existence on that day. averageDifficulty is daily average mining difficulty for Bitcoin.

\paragraph*{fees and medianFees}
In Bitcoin network, broadcasting information onto blockchain requires a part of sending Bitcoin as a "fee" to other people who are involved in this network and help pack onto the blockchain. Fees are total transaction fee in the form of BTC while the median fee is the median transaction fee happened at a specific date.

\paragraph*{activeAddresses}
Active addresses are the number of unique sending and receiving addresses participating in transactions on the given day.

\paragraph*{blockSize and blockCount}
The block size is the maximum limit a block can be filled up with transactions and blockCount is the number of the lasted block on the blockchain.\\

Summed up descriptive data of metrics in Table \ref{Logarithmic_Metrics_Descriptive_Data}, data are in the raw unit except specified in brackets. Compared to historical Realized Volatility and Realized Quarticity, kurtosis of metrics in Bitcoins is prevalently less. The concentration of distribution constructs a solid parametric basis in time-series prediction functions.

\begin{table}[H]
\centering
\begin{sideways}
\setlength{\tabcolsep}{3mm}{
\begin{tabular}{|l|l|l|l|l|l|l|l|l|}
\hline
logarithmic           & count       & mean      & std       & min        & 25\%      & 50\%      & 75\%      & max        \\
\hline
txVolume(USD)         & 1298.000000 & 19.711722 & 0.959412  & 17.308142  & 19.099714 & 19.582663 & 20.473876 & 23.985118  \\
\hline
adjustedTxVolume(USD) & 1298.000000 & 18.758655 & 0.777156  & 16.837822  & 18.185984 & 18.711123 & 19.435734 & 20.970551  \\
\hline
txCount               & 1298.000000 & 11.561145 & 0.574996  & 10.314603  & 11.080272 & 11.494436 & 12.148367 & 12.701915  \\
\hline
paymentCount          & 1298.000000 & 12.156837 & 0.596746  & 10.725402  & 11.772605 & 12.197687 & 12.679312 & 14.448771  \\
\hline
marketCap(USD)        & 1298.000000 & 22.333939 & 0.628417  & 20.474146  & 21.970554 & 22.467239 & 22.796025 & 23.357107  \\
\hline
realizedCap(USD)      & 1298.000000 & 21.971073 & 0.682383  & 20.122969  & 22.134220 & 22.174637 & 22.277882 & 22.619077  \\
\hline
exchangeVolume(USD)   & 1298.000000 & 3.901e+07 & 4.181e+07 & 0.000e+00  & 1.322e+07 & 2.531e+07 & 5.621e+07 & 3.633e+08  \\
\hline
medianTxValue(USD)    & 1298.000000 & 4.081724  & 0.592777  & -3.723677  & 3.805915  & 4.084623  & 4.349774  & 6.366660   \\
\hline
generatedCoins        & 1298.000000 & 8.190681  & 0.278538  & 7.279319   & 8.153350  & 8.249314  & 8.348775  & 8.779557   \\
\hline
averageDifficulty     & 1298.000000 & 23.574159 & 2.655655  & 16.313120  & 22.534783 & 24.566675 & 25.453782 & 26.460335  \\
\hline
fees                  & 1298.000000 & 3.228438  & 0.675149  & 2.086189   & 2.605480  & 3.221733  & 3.768822  & 5.822325   \\
\hline
medianFees            & 1298.000000 & -8.944887 & 0.522635  & -10.049670 & -9.210340 & -9.210340 & -8.914690 & -7.600902  \\
\hline
activeAddresses       & 1298.000000 & 12.390833 & 0.642612  & 10.787296  & 12.029162 & 12.430889 & 12.995259 & 13.551645  \\
\hline
blockSize             & 1298.000000 & 17.854582 & 0.582302  & 16.466191  & 17.420028 & 17.823433 & 18.453732 & 18.880478  \\
\hline
blockCount            & 1298.000000 & 5.055781  & 0.125687  & 4.718499   & 4.969813  & 5.043425  & 5.135798  & 5.560682   \\
\hline
\end{tabular}}
\end{sideways}
\caption{logarithmic Metrics Descriptive Data}\label{Logarithmic_Metrics_Descriptive_Data}
\end{table}

\begin{comment}
\begin{table}[H]
\setlength{\tabcolsep}{5mm}{
\begin{tabular}{|c|l|c|c|c|c|}
\hline
\multicolumn{6}{|c|}{Metrics Model Result}                                                                        \\ \hline
\multicolumn{2}{|c|}{Dep. Variable:}        & logRV        & \multicolumn{2}{c|}{R-squared:}          & 0.364     \\
\multicolumn{2}{|c|}{Model:}                & OLS          & \multicolumn{2}{c|}{Adj. R-squared:}     & 0.361     \\
\multicolumn{2}{|c|}{Method:}               & LeastSquares & \multicolumn{2}{c|}{F-statistic:}        & 147.7     \\
\multicolumn{2}{|c|}{Log-Likelihood:}       & -2073.8      & \multicolumn{2}{c|}{Prob (F-statistic):} & 4.05e-124 \\
\multicolumn{2}{|c|}{No. Observations:}     & 1298         & \multicolumn{2}{c|}{AIC:}                & 4160.     \\
\multicolumn{2}{|c|}{Df Residuals:}         & 1292         & \multicolumn{2}{c|}{BIC:}                & 4191.     \\
\multicolumn{2}{|c|}{Df Model:}             & 5            & \multicolumn{2}{c|}{Covariance Type:}    & nonrobust \\ \hline
\multicolumn{2}{|c|}{}                      & coef         & std err             & t                  & P>|t|     \\
\multicolumn{2}{|c|}{const}                 & 11.3472      & 2.037               & 5.571              & 0.000     \\
\multicolumn{2}{|c|}{adjustedTxVolume(USD)} & 1.4322       & 0.133               & 10.769             & 0.000     \\
\multicolumn{2}{|c|}{txCount}               & -2.2426      & 0.160               & -14.003            & 0.000     \\
\multicolumn{2}{|c|}{marketcap(USD)}        & -1.0135      & 0.133               & -7.600             & 0.000     \\
\multicolumn{2}{|c|}{averageDifficulty}     & 0.0717       & 0.031               & 2.316              & 0.021     \\
\multicolumn{2}{|c|}{medianTxValue(USD)}    & 0.3696       & 0.085               & 4.338              & 0.000     \\ \hline
\multicolumn{2}{|c|}{Omnibus:}              & 16.198           & \multicolumn{2}{c|}{Durbin-Watson:}      & 0.786     \\
\multicolumn{2}{|c|}{Prob(Omnibus):}        & 0.000        & \multicolumn{2}{c|}{Jarque-Bera (JB):}   & 16.447    \\
\multicolumn{2}{|c|}{Skew:}                 & 0.267        & \multicolumn{2}{c|}{Prob(JB):}           & 0.000268  \\
\multicolumn{2}{|c|}{Kurtosis:}             & 3.137        & \multicolumn{2}{c|}{Cond. No.}           & 2.43e+03  \\ \hline
\end{tabular}}
\caption{Selected Metrics Model Result (10min interval)} \label{selected_metrics_model_result}
\end{table}
\end{comment}