\chapter*{Overview of Scheme}

This paper gives an overview of
the small language of \rsevenrs.
The purpose of this overview is to explain
enough about the basic concepts of the language to facilitate
understanding of the \rsevenrs{} report, which is
organized as a reference manual.  Consequently, this overview is
not a complete introduction to the language, nor is it precise
in all respects or normative in any way.

\vest Following Algol, Scheme is a statically scoped programming
language.  Each use of a variable is associated with a lexically
apparent binding of that variable.

\vest Scheme has latent as opposed to manifest types.  Types
are associated with objects (also called values) rather than
with variables.  (Some authors refer to languages with latent types as
untyped, weakly typed or dynamically typed languages.)  Other languages with
latent types are Python, Ruby, Smalltalk, and other dialects of Lisp.  Languages
with manifest types (sometimes referred to as strongly typed or
statically typed languages) include Algol 60, C, C\#, Java, Haskell, and ML.

\vest All objects created in the course of a Scheme computation, including
procedures and continuations, have unlimited extent.
No Scheme object is ever destroyed.  The reason that
implementations of Scheme do not (usually!) run out of storage is that
they are permitted to reclaim the storage occupied by an object if
they can prove that the object cannot possibly matter to any future
computation.  Other languages in which most objects have unlimited
extent include C\#, Java, Haskell, most Lisp dialects, ML, Python,
Ruby, and Smalltalk.

Implementations of Scheme must be properly tail-recursive.
This allows the execution of an iterative computation in constant space,
even if the iterative computation is described by a syntactically
recursive procedure.  Thus with a properly tail-recursive implementation,
iteration can be expressed using the ordinary procedure-call
mechanics, so that special iteration constructs are useful only as
syntactic sugar.

\vest Scheme was one of the first languages to support procedures as
objects in their own right.  Procedures can be created dynamically,
stored in data structures, returned as results of procedures, and so
on.  Other languages with these properties include Common Lisp,
Haskell, ML, Ruby, and Smalltalk.

\vest One distinguishing feature of Scheme is that continuations, which
in most other languages only operate behind the scenes, also have
``first-class'' status.  First-class continuations are useful for implementing a
wide variety of advanced control constructs, including non-local exits,
backtracking, and coroutines.

In Scheme, the argument expressions of a procedure call are evaluated
before the procedure gains control, whether the procedure needs the
result of the evaluation or not.  C, C\#, Common Lisp, Python,
Ruby, and Smalltalk are other languages that always evaluate argument
expressions before invoking a procedure.  This is distinct from the
lazy-evaluation semantics of Haskell, or the call-by-name semantics of
Algol 60, where an argument expression is not evaluated unless its
value is needed by the procedure.

Scheme's model of arithmetic provides a rich set of numerical types
and operations on them.  Furthermore, it distinguishes \textit{exact}
and \textit{inexact} numbers: Essentially, an exact number
object corresponds to a number exactly, and an inexact number
is the result of a computation that involved rounding or other approximations.

\chapter{Basic types}

Scheme programs manipulate \textit{objects}, which are also referred
to as \textit{values}.
Scheme objects are organized into sets of values called \textit{types}.
This chapter gives an overview of the fundamentally important types of the
Scheme language.

\begin{note}
  As Scheme is latently typed, the use of the term \textit{type} in
  the Scheme context differs from the use of the term in the context of other
  languages, particularly those with manifest typing.
\end{note}

\paragraph{Numbers}

Scheme supports a rich variety of numerical data types, including
integers of arbitrary precision, rational numbers, complex numbers, and
inexact numbers of various kinds.

\paragraph{Booleans}

A boolean is a truth value, and can be either
true or false.  In Scheme, the object for ``false'' is written
\schfalse{}.  The object for ``true'' is written \schtrue{}.  In
most places where a truth value is expected, however, any object different from
\schfalse{} counts as true.

\paragraph{Pairs and lists}

A pair is a data structure with two components.  The most common use
of pairs is to represent (singly linked) lists, where the first
component (the ``car'') represents the first element of the list, and
the second component (the ``cdr'') the rest of the list.  Scheme also
has a distinguished empty list, which is the last cdr in a chain of
pairs that form a list.

\paragraph{Symbols}

A symbol is an object representing a string,
the symbol's \textit{name}.
Unlike strings, two symbols whose names are spelled the same
way are never distinguishable.  Symbols are useful for many applications;
for instance, they can be used the way enumerated values are used in
other languages.

In \rsevenrs, unlike \rfivers, symbols and identifiers are case-sensitive.

\paragraph{Characters}

Scheme characters mostly correspond to textual characters.
More precisely, they are isomorphic to a subset of the \textit{scalar values} of
the Unicode standard, with possible implementation-dependent extensions.

\paragraph{Strings}

Strings are finite sequences of characters with fixed length and thus
represent arbitrary Unicode texts.

\paragraph{Vectors}

Vectors, like lists, are linear data structures
representing finite sequences of arbitrary objects.
Whereas the elements of a list are accessed
sequentially through the chain of pairs representing it,
the elements of a vector are addressed by integer indices.
Thus, vectors are more appropriate than
lists for random access to elements.

\paragraph{Bytevectors}

Bytevectors are similar to vectors, except that their contents
are \textit{bytes}, exact integers in the range 0 to 255 inclusive.

\paragraph{Procedures}

Procedures are values in Scheme.

\paragraph{Records}

Records are structured values, and are
aggregations of zero or more \textit{fields}, each of which holds a single location.
Records are organized into \textit{record types}.
A predicate, a constructor, and field accessors and
mutators can be defined for each record type.

\paragraph{Ports}

Ports represent input and output devices.  To Scheme, an input port is
a Scheme object that can deliver data upon command, while an output
port is a Scheme object that can accept data.

\chapter{Expressions}

The most important elements of Scheme code are
\textit{expressions}.  Expressions can be
\textit{evaluated}, producing a \textit{value} (actually, any number
of values.)  The most
fundamental expressions are literal expressions:

\begin{scheme}
\schtrue{} \ev \schtrue
23 \ev 23%
\end{scheme}

This notation means that the expression \schtrue{} evaluates to
\schtrue{}, that is, the value for ``true'',  and that the expression
{\cf 23} evaluates to a number representing the number 23.

Compound expressions are formed by placing parentheses around their
subexpressions.  The first subexpression identifies an operation; the
remaining subexpressions are operands to the operation:
%
\begin{scheme}
(+ 23 42) \ev 65
(+ 14 (* 23 42)) \ev 980%
\end{scheme}
%
In the first of these examples, {\cf +} is the name of
the built-in operation for addition, and {\cf 23} and {\cf 42} are the
operands.  The expression {\cf (+ 23 42)} reads as ``the sum of 23 and
42''.  Compound expressions can be nested---the second example reads
as ``the sum of 14 and the product of 23 and 42''.

As these examples indicate, compound expressions in Scheme are always
written using the same prefix notation.  As
a consequence, the parentheses are needed to indicate structure.
Consequently, ``superfluous'' parentheses, which are often permissible in
mathematical notation and also in many programming languages, are not
allowed in Scheme.

As in many other languages, whitespace (including line endings) is not
significant when it separates subexpressions of an expression, and
can be used to indicate structure.

\chapter{Variables and binding}

Scheme
allows identifiers to stand for locations containing values.
These identifiers are called variables.  In many cases, specifically
when the location's value is never modified after its creation, it is
useful to think of the variable as standing for the value directly.

\begin{scheme}
(let ((x 23)
      (y 42))
  (+ x y)) \ev 65%
\end{scheme}

In this case, the expression starting with {\cf let} is a binding
construct.  The parenthesized structure following the {\cf let} lists
variables alongside expressions: the variable {\cf x} alongside {\cf
  23}, and the variable {\cf y} alongside {\cf 42}.  The {\cf let}
expression binds {\cf x} to 23, and {\cf y} to 42.  These bindings are
available in the \textit{body} of the {\cf let} expression, {\cf (+ x
  y)}, and only there.

\chapter{Definitions}

The variables bound by a {\cf let} expression
are \textit{local}, because their bindings are visible only in the
{\cf let}'s body.  Scheme also allows creating top-level bindings for
identifiers as follows:

\begin{scheme}
(define x 23)
(define y 42)
(+ x y) \ev 65%
\end{scheme}

(These are actually ``top-level'' in the body of a top-level program or library.)

The first two parenthesized structures are \textit{definitions}; they
create top-level bindings, binding {\cf x} to 23 and {\cf y} to 42.
Definitions are not expressions, and cannot appear in all places
where an expression can occur.  Moreover, a definition has no value.

Bindings follow the lexical structure of the program:  When several
bindings with the same name exist, a variable refers to the binding
that is closest to it, starting with its occurrence in the program
and going from inside to outside, and referring to an outermost
binding if no
local binding can be found along the way:

\begin{scheme}
(define x 23)
(define y 42)
(let ((y 43))
  (+ x y)) \ev 66

(let ((y 43))
  (let ((y 44))
    (+ x y))) \ev 67%
\end{scheme}

\chapter{Procedures}

Definitions can also be used to define
procedures:

\begin{scheme}
(define (f x)
  (+ x 42))

(f 23) \ev 65%
\end{scheme}

A procedure is, slightly simplified, an abstraction of an
expression over objects.  In the example, the first definition defines a procedure
called {\cf f}.  (Note the parentheses around {\cf f x}, which
indicate that this is a procedure definition.)  The expression {\cf (f
  23)} is a procedure call meaning,
roughly, ``evaluate {\cf (+ x 42)} (the body of the procedure) with
{\cf x} bound to 23''.

As procedures are objects, they can be passed to other
procedures:
%
\begin{scheme}
(define (f x)
  (+ x 42))

(define (g p x)
  (p x))

(g f 23) \ev 65%
\end{scheme}

In this example, the body of {\cf g} is evaluated with {\cf p}
bound to {\cf f} and {\cf x} bound to 23, which is equivalent
to {\cf (f 23)}, which evaluates to 65.

In fact, many predefined operations of Scheme are provided not by
syntax, but by variables whose values are procedures.
The {\cf +} operation, for example, which receives
special syntactic treatment in many other languages, is just a regular
identifier in Scheme, bound to a procedure that adds numbers.  The
same holds for {\cf *} and many others:

\begin{scheme}
(define (h op x y)
  (op x y))

(h + 23 42) \ev 65
(h * 23 42) \ev 966%
\end{scheme}

Procedure definitions are not the only way to create procedures.  A
{\cf lambda} expression creates a new procedure as an object, with no
need to specify a name:

\begin{scheme}
((lambda (x) (+ x 42)) 23) \ev 65%
\end{scheme}

The entire expression in this example is a procedure call; {\cf
  (lambda (x) (+ x 42))}, evaluates to a procedure that takes a single
number and adds 42 to it.

\chapter{Procedure calls and syntactic keywords}

Whereas {\cf (+ 23 42)}, {\cf (f 23)}, and {\cf ((lambda (x) (+ x 42))
  23)} are all examples of procedure calls, {\cf lambda} and {\cf
  let} expressions are not.  This is because {\cf let}, even though
it is an identifier, is not a variable, but is instead a \textit{syntactic
  keyword}.  A list that has a
syntactic keyword as its first subexpression obeys special rules determined by
the keyword.  The {\cf define} identifier in a definition is also a
syntactic keyword.  Hence, definitions are also not procedure calls.

The rules for the {\cf lambda} keyword specify that the first
sublist is a list of parameters, and the remaining sublists are the body of
the procedure.  In {\cf let} expressions, the first sublist is a list
of binding specifications, and the remaining sublists constitute a body of
expressions.

Procedure calls can be distinguished from these
\textit{expression types} by
looking for a syntactic keyword in the first position of a list:
if the first position does not contain a syntactic keyword, the expression
is a procedure call.
The set of syntactic keywords of Scheme is
fairly small, which usually makes this task fairly simple.
It is possible, however, to create new bindings for syntactic keywords.

\chapter{Assignment}

Scheme variables bound by definitions or {\cf let} or {\cf lambda}
expressions are not actually bound directly to the objects specified in the
respective bindings, but to locations containing these objects.  The
contents of these locations can subsequently be modified destructively
via \textit{assignment}:
%
\begin{scheme}
(let ((x 23))
  (set! x 42)
  x) \ev 42%
\end{scheme}

In this case, the body of the {\cf let} expression consists of two
expressions which are evaluated sequentially, with the value of the
final expression becoming the value of the entire {\cf let}
expression.  The expression {\cf (set! x 42)} is an assignment, saying
``replace the object in the location referenced by {\cf x} with 42''.
Thus, the previous value of {\cf x}, 23, is replaced by 42.

\chapter{Derived syntax and macros}

Many of the expression types specified
as part of the \rsevenrs{} small language
can be translated into more basic expression types.
For example, a {\cf let} expression can be translated
into a procedure call and a {\cf lambda} expression.  The following two
expressions are equivalent:
%
\begin{scheme}
(let ((x 23)
      (y 42))
  (+ x y)) \ev 65

((lambda (x y) (+ x y)) 23 42) \lev 65%
\end{scheme}

Syntax expressions like {\cf let} expressions are called \textit{derived}
because their semantics can be
derived from that of other kinds of expressions by a syntactic
transformation.  Some procedure definitions are also derived expressions.  The
following two definitions are equivalent:

\begin{scheme}
(define (f x)
  (+ x 42))

(define f
  (lambda (x)
    (+ x 42)))%
\end{scheme}

In Scheme, it is possible for a program to create its own derived
expressions by binding syntactic keywords to macros:

\begin{scheme}
(define-syntax def
  (syntax-rules ()
    ((def f (p ...) body)
     (define (f p ...)
       body))))

(def f (x)
  (+ x 42))%
\end{scheme}

The {\cf define-syntax} construct specifies that a parenthesized
structure matching the pattern {\cf (def f (p ...) body)}, where {\cf
  f}, {\cf p}, and {\cf body} are pattern variables, is translated to
{\cf (define (f p ...) body)}.  Thus, the {\cf def} expression appearing in
the example gets translated to:

\begin{scheme}
(define (f x)
  (+ x 42))%
\end{scheme}

The ability to create new syntactic keywords makes Scheme extremely
flexible and expressive, allowing many of the features
built into other languages to be implemented directly in Scheme:
any Scheme programmer can add new expression types.

\chapter{Syntactic data and datum values}

\textit{Datum values} constitute a subset of Scheme objects.
These include booleans, numbers, characters, symbols,
and strings as well as lists, vectors, and bytevectors whose elements are datum values.  Each
datum value can be represented textually as a
\textit{syntactic datum}, which can be written out
and read back in without loss of information.
There is in general more than one syntactic datum corresponding to each datum value.
Moreover, each datum value
can be trivially translated to a literal expression in a program by
prepending a {\cf\singlequote} to a corresponding syntactic datum:

\begin{scheme}
'23 \ev 23
'\schtrue{} \ev \schtrue{}
'foo \ev foo
'(1 2 3) \ev (1 2 3)
'\#(1 2 3) \ev \#(1 2 3)%
\end{scheme}

The {\cf\singlequote} shown in the previous examples
is not needed for representations of literal constants other than
symbols and lists.
The syntactic datum {\cf foo} represents a
symbol with name ``foo'', and {\cf 'foo} is a literal expression with
that symbol as its value.  {\cf (1 2 3)} is a syntactic datum that
represents a list with elements 1, 2, and 3, and {\cf '(1 2 3)} is a literal
expression with this list as its value.  Likewise, {\cf \#(1 2 3)}
is a syntactic datum that represents a vector with elements 1, 2 and 3, and
{\cf '\#(1 2 3)} is the corresponding literal.

Syntactic datums are a superset of Scheme expressions.  Thus, data
can be used to represent Scheme expressions as data objects.  In
particular, symbols can be used to represent identifiers.

\begin{scheme}
'(+ 23 42) \ev (+ 23 42)
'(define (f x) (+ x 42)) \lev (define (f x) (+ x 42))%
\end{scheme}

This facilitates writing programs that operate on Scheme source code,
in particular interpreters and program transformers.

\chapter{Continuations}

Whenever a Scheme expression is evaluated there is a
\textit{continuation} wanting the result of the
expression.  The continuation represents an entire (default) future
for the computation.  For example, informally the continuation of {\cf 3}
in the expression
%
\begin{scheme}
(+ 1 3)%
\end{scheme}
%
adds 1 to it.  Normally these ubiquitous continuations are hidden
behind the scenes and programmers do not think much about them.  On
rare occasions, however, a programmer needs to deal with
continuations explicitly.  The {\cf call-with-current-continuation}
procedure allows
Scheme programmers to do that by creating a procedure that reinstates
the current continuation.  The {\cf call-with-current-continuation}
procedure accepts a procedure, calls it immediately with an argument
that is an \textit{escape procedure}.  This
escape procedure can then be called with an argument that becomes the
result of the call to {\cf call-with-current-continuation}.  That is,
the escape procedure abandons its own continuation, and reinstates the
continuation of the call to {\cf call-with-current-continuation}.

In the following example, an escape procedure representing the
continuation that adds 1 to its argument is bound to {\cf escape}, and
then called with 3 as an argument.  The continuation of the call to
{\cf escape} is abandoned, and instead the 3 is passed to the
continuation that adds 1:
%
\begin{scheme}
(+ 1 (call-with-current-continuation
       (lambda (escape)
         (+ 2 (escape 3))))) \lev 4%
\end{scheme}
%
An escape procedure has unlimited extent: It can be called after the
continuation it captured has been invoked, and it can be called
multiple times.  This makes {\cf call-with-current-continuation}
significantly more powerful than typical non-local control constructs
such as exceptions in other languages.

\chapter{Libraries}

Scheme code can be organized in components called
\textit{libraries}.  Each library contains
definitions and expressions.  It can import definitions
from other libraries and export definitions to other libraries.

The following library called {\cf (hello)} exports a definition called
{\cf hello-world},  and imports the base library
and the display library.
The {\cf hello-world} export is a procedure that displays {\cf Hello World}
on a separate line:
%
\begin{scheme}
(define-library (hello)
  (export hello-world)
  (import (scheme base)
          (scheme display))
  (begin
    (define (hello-world)
      (display "Hello World")
      (newline))))%
\end{scheme}

\chapter{Programs}

Libraries are invoked by other libraries, but ultimately by a Scheme
\textit{program}.  Like a library, a
program contains imports, definitions and expressions, and specifies
an entry point for execution.  Thus a program defines, via
the transitive closure of the libraries it imports, a Scheme program.

The following program obtains the first argument from the command line
via the {\cf command-line} procedure from the process-context
library.
It then opens the file using {\cf with-input-from-file},
which causes the file to be the current input port, and
arranges for it to be closed at the end.
Next, it calls the {\cf read-line} procedure to read a line of text
from the file, and then {\cf write-string} and {\cf newline} to
output the line, then looping until the end of file:
%
\begin{scheme}
(import (scheme base)
        (scheme file)
        (scheme process-context))
(with-input-from-file
  (cadr (command-line))
  (lambda ()
    (let loop ((line (read-line)))
      (unless (eof-object? line)
        (write-string line)
        (newline)
        (loop (read-line))))))%
\end{scheme}

\chapter{The REPL}

Implementations may provide an interactive session called a
\defining{REPL} (Read-Eval-Print Loop), where import declarations,
expressions and definitions can be
entered and evaluated one at a time.  The REPL starts out
with the base library imported, and possibly other libraries.
An implementation may provide a mode of operation in which the REPL
reads its input from a file.  Such a file is not, in general, the same
as a program, because it can contain import declarations in places other than
the beginning.

Here is a short REPL session.  The {\cf >} character represents the REPL's
prompt for input:

\begin{scheme}
> ; A few simple things
> (+ 2 2)
4
> (sin 4)
Undefined variable: sin
> (import (scheme inexact))
> (sin 4)
-0.756802495307928
> (define sine sin)
> (sine 4)
-0.756802495307928
> ; Guy Steele's three-part test
> ; True is true ...
> \#t
\#t
> ; 100!/99! = 100 ...
> (define (fact n)
    (if (= n 0) 1 (* n (fact (- n 1)))))
> (/ (fact 100) (fact 99))
100
> ; If it returns the *right* complex number,
> ; so much the better ...
> (define (atanh x)
    (/ (- (log (+ 1 x))
          (log (- 1 x)))
       2))
> (atanh -2)
-0.549306144334055+1.5707963267949i%
\end{scheme}

%%% Local Variables:
%%% mode: latex
%%% End:


%!TEX root = ../thesis.tex
%*******************************************************************************
%****************************** Second Chapter *********************************
%*******************************************************************************

\chapter{Preliminaries}

\ifpdf
    \graphicspath{{Chapter2/Figs/Raster/}{Chapter2/Figs/PDF/}{Chapter2/Figs/}}
\else
    \graphicspath{{Chapter2/Figs/Vector/}{Chapter2/Figs/}}
\fi

\section{Machine Learning}
\label{preliminaries:machine_learning}

Machine learning is a field in computer science that we try teaching the computers to do tasks without explicitly programmed \citep{samuel2000some, koza1996automated}. Machine learning is strongly associated with computational statistics, which explores data and builds algorithms that could find patterns in data and make predictions. More formal definition of the algorithms used in the machine learning is provided by Mitchell ``A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E`` \citep{mitchell1997machine}.

Typically, tasks in machine learning can be grouped into three wide categories \citep{russell1995modern}:

\begin{itemize}
\item Supervised learning: the given data contains input values and their desired output values, the aim of algorithms in supervised learning is to find a general rule that maps inputs to outputs. In this thesis we are focus on the supervised learning approach for our fraud detection problem,
\item Unsupervised learning: there are no desired outputs in the dataset, it means that we give the learning algorithms freely to explore a hidden structure of the data,
\item Reinforcement learning: this kind of learning is slightly different with the above, reinforcement learning builds a dynamic environment then let a computer interacts with it to perform a certain goal, after each action a feedback is provided to the computer as a reward or a punishment.
\end{itemize}

Supervised learning is a most common task in the world and can be seen as a function from labeled training data \citep{mohri2012foundations}. Given a set of $N$ data points of a form ${(x_1, y_1), \dots, (x_N, y_N)}$ such that $x_i$ is a attribute vector of the i-th data point and $y_i$ is its desired output, a supervised learning algorithm is a function $g: X \rightarrow Y$, where $X$ is an input space (matrix of attribute vectors) and $Y$ is a output space (matrix of desired outputs). Typically the function $g$ is in a function space $G$, or we could represent $g$ as a scoring function $f: X \times Y \rightarrow \mathbb{R}$ such that function $f$ will return the output value $y$ which has the highest score $g(x) = argmax_y f(x, y)$. For example, $g$ could be a conditional probability model $g(x) = P(y \mid x)$, e.g. logistic regression \citep{walker1967estimation, cox1958regression}, or $f$ could takes a form of a join t probability model $f(x, y) = P(x, y)$, e.g. naive Bayes \citep{russell1995modern}.

In order to measure a performance of learning function, we could define a loss function $L: Y \times Y \rightarrow \mathbb{R}$ and a risk $R(g)$ of learning function $g$ is defined as an expected loss of function $g$ which could be estimated by:

\begin{equation}
\widehat{R} (g) = \dfrac{1}{N} \sum_i L(y_i, g(x_i))
\end{equation}

\subsection*{Decision tree learning}

A decision tree learning is one common approach in data mining \citep{rokach2014data}, it aims to create a model that predicts values like making a decision from a tree-based analysis. An example is shown in the figure \ref{img:titanic_survival_decision_tree} \citep{decision_tree_learning}.

\begin{figure}
\includegraphics[scale=0.4]{Images/Titanic_Survival_Decison_Tree_SVG.png}
\centering
\caption{A tree showing survival of passengers on the Titanic}
\label{img:titanic_survival_decision_tree}
\end{figure}

A tree learner could classify the data points by splitting the input dataset into subsets based on a given criterion. The splitting process at each node is repeated in a recursive procedure until all data points are classified. There are two kinds of decision trees:

\begin{itemize}
\item Regression tree: create a tree to predict the desired output as a real number,
\item Classification tree: an analysis aims to predict a categorical output.
\end{itemize}

A term Classification and Regression Tree (CART) analysis is used to refer to both the above analysis \citep{breiman1984classification}. There are many decision tree algorithms, including:

\begin{itemize}
\item ID3 (Iterative Dichotomiser 3) \citep{quinlan1986induction},
\item C4.5: an extension of ID3 algorithm \citep{quinlan2014c4}.
\end{itemize}

Algorithms for constructing decision trees use different metrics for measuring which is the best feature to split the dataset. These measures compute the homogeneity of the output values within the subsets, some common measures in decision tree learning is Entropy, Information Gain, and Gini.

\subsection*{Ensemble learning}

In machine learning, ensemble learning is a method combines multiple learning algorithms to obtain better accuracy than from any individual algorithms \citep{opitz1999popular, rokach2010ensemble}. The term ensemble is usually reserved for methods that generate multiple hypotheses using the same base learner. There are three common types of ensemble learning, such as:

\subparagraph{Bootstrap aggregating (bagging)}

An ensemble technique is bootstrap aggregating which is usually called bagging (see figure \ref{img:bagging_ensemble}), it combines multiple learnings via their equal weight votes. In order to support the model variance, bagging trains each model from a randomly drawn subset. And a famous example of bagging is the Random Forest algorithm which uses random decision trees as base learners.

\begin{figure}
\includegraphics[scale=0.4]{Images/bagging.jpg}
\centering
\caption{Bagging ensemble \citep{raschka2015python}}
\label{img:bagging_ensemble}
\end{figure}

\subparagraph{Boosting}

Similar with the bagging ensemble, boosting is an incremental algorithm that will train each new model to underline the data points that previous models mis-classified (see figure \ref{img:boosting_ensemble}). Boosting could have a better accuracy than bagging in some cases, however it also tends to overfit to the training data. A most common boosting algorithm is Adaboost \citep{freund1999short}.

\begin{figure}
\includegraphics[scale=0.4]{Images/boosting.png}
\centering
\caption{Boosting ensemble \citep{raschka2015python}}
\label{img:boosting_ensemble}
\end{figure}

\subparagraph{Stacking}

Slightly different with these above techniques, stacking is a supervised learning algorithm that uses the predictions of several other learning algorithms as an input dataset to makes the final prediction (see figure \ref{img:stacking_ensemble}). All of the other algorithms are trained using the given dataset independently, then a combiner algorithm is trained on that which can be any algorithm. Therefore, stacking can theoretically represent any of the ensemble techniques, although in practice we often use the Logistic Regression or Linear Regression as the combiner.

\begin{figure}
\includegraphics[scale=0.4]{Images/stacking.png}
\centering
\caption{Stacking ensemble \citep{raschka2015python}}
\label{img:stacking_ensemble}
\end{figure}

\subsection*{Random Forest}

Random Forest \citep{breiman2001random} or random decision forests \citep{ho1995random, ho1998random} are an ensemble learning method for classification or regression that perform by constructing many trees, e.g. decision trees, then combine these results to makes final output values as a mode of the classes for classification tasks or a mean prediction for regression tasks. The Random Forest model based on two elements:

\begin{enumerate}
\item Base learner: each tree in the forest is the base learner, typically it is a weak learner with high variance,
\item Ensemble learning: combine the results of several weak learners to make a final prediction.
\end{enumerate}

The Decision Tree above that are grown very deep tend to overfit their training sets, i.e. have a low bias, but very high variance. We use the tree-based method as the base learner by its strength, and to handle its weakness we combine all of the predictions via ensemble learning. This method is called Random Forest, it can formulate the algorithm as the following statement: given a dataset $X = x_1, \dots, x_n$ with $n$ data points and respectively desired output values $Y = y_1, \dots, y_n$, bagging repeatedly $B$ times selects a random sample from the dataset $X$ then training trees on these samples.

\begin{algorithm}
   \caption{Tree bagging}
    \begin{algorithmic}
      \Function{bagging}{$X, Y, B$}

        \For{$b = 1$ to ${B}$}
            \State $(X_b, Y_b)$ is samples from $(X, Y)$ which have $n$ data points
            \State Train a classification or regression tree $f_b$ on $X_b, Y_b$
        \EndFor

       \EndFunction

\end{algorithmic}
\end{algorithm}

The authors of Random Forest have proposed using an average function to makes the final prediction in regression tasks or takes the majority vote in classification tasks \citep{breiman2001random}. However, there is no standard function as a combiner, thus we are free to choose the combination function. This bootstrapping procedure could decrease the variance of the model without increasing the bias. In order to estimate the uncertainty of the prediction, it could be computed as a standard deviation of the predictions from all the individual trees:

\begin{equation}
\sigma = \sqrt{ \dfrac{ \sum_{b=1}^B ( f_b(x) - \hat{f} ) }{ B - 1 } }
\end{equation}

Random Forest uses the above bagging algorithm with only one difference, it uses a modified tree that selects a random subset of the features to split a current subset data. Typically, the dataset has $p$ features then for a classification problem $\sqrt{p}$ features will be used in each split and for regression problems the inventor recommends that we should use $p/3$ as the number of features \citep{friedman2001elements}.

\section{Measurement}
\label{measurement_premilinaries}

The fraud detection problem is the classification task and in order to measure how well of the classifiers, there are many metrics could be used to evaluate the algorithms. In this section, we will review some common metrics which are typically used in classification tasks.

\subsection*{Accuracy}

Accuracy is a measure of statistical variablity which represent a percentage of correct predictions on the total number of cases examined. In the fields of science and engineering, the accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's true value \citep{bipm2008international}. Consider 2 sets with $n$ data points: the true labels from the given dataset $Y = {y_1, \dots, y_n}$ and our predicted values $\hat{Y} = {\hat{y_1}, \dots, \hat{y_n}}$, the accuracy is:

\begin{equation}
accuracy = \dfrac{ 1 }{ n } \sum_{i = 1}^n \mbox{1\hspace{-4.25pt}\fontsize{12}{14.4}\selectfont\textrm{1}}_{ y_i = \hat{y_i} }
\end{equation}

\subsection*{Precision and recall}

In pattern recognition, information retrieval and binary classification, precision and recall are two common metrics. Precision (also called positive predictive value) is the fraction of relevant data points among the given data points, and recall (also known as sensitivity) is the fraction of relevant data points that have been retrieved over the total amount of relevant data points (see figure \ref{img:precision_and_recall}).

\begin{figure}
\includegraphics[scale=0.35]{Images/Precisionrecall.png}
\centering
\caption{Precision and recall \citep{wiki2018precision_recall}}
\label{img:precision_and_recall}
\end{figure}

In the information retrieval contexts, precision and recall were defined by Perry, Kent \& Berry (1955) \citep{perry1955machine} as a set of retrieved elements and a set of relevant elements. Precision is the fraction of retrieved documents that are relevant to the query:

\begin{equation}
\text{precision} = \dfrac{ \mid {\text{relevant elements}} \cap {\text{retrieved elements}} \mid }{ \mid {\text{retrieved elements}} \mid }
\end{equation}

And the recall is the fraction of the relevant elements that are successfully retrieved:

\begin{equation}
\text{precision} = \dfrac{ \mid {\text{relevant elements}} \cap {\text{retrieved elements}} \mid }{ \mid {\text{relevant elements}} \mid }
\end{equation}

In the classification tasks, considers four terms true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). Precision and recall are defined as \citep{olson2008advanced}:

\begin{equation}
\text{precision} = \dfrac{ TP }{ TP + FP }
\end{equation}

\begin{equation}
\text{recall} = \dfrac{ TP }{ TP + FN }
\end{equation}

\subsection*{Receiver operating characteristic (ROC)}

A receiver operating characteristic curve is a graphical plot which is created by plotting the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. ROC analysis provides tools to select a possible optimal threshold for models, an example ROC curve plot of three predictors of peptide cleaving in the proteasome is shown in figure \ref{img:roc_curve}.

\begin{figure}
\includegraphics[scale=2.5]{Images/Roccurves.png}
\centering
\caption{ROC curve of three predictors of peptide cleaving in the proteasome \citep{wiki2018roc}}
\label{img:roc_curve}
\end{figure}

\subsection*{Area under the ROC curve (AUC)}

We cannot compare classifiers base on the ROC because the ROC curve is just a graphical plot. In order to represent ROC performance as a single scala, a common method is to calculate the area under the ROC curve (AUC for short). Because the AUC is a size of the area of the unit square, therefore the AUC value will always be between 0 and 1. However, random guessing produces an area of 0.5, no realistic classifier should have an AUC less than 0.5 then it makes the baseline performance for all classifiers is 0.5. In summary, the AUC metric is a measure of how much the ROC curve is close to the point of perfect classification.

The AUC metric usually used in machine learning community for model comparison \citep{hanley1983method}. However, in practice researchers recently noticed that AUC was quite noisy as a classification measure \citep{hanczar2010small} and some other studies have other significant problems in model comparison \citep{lobo2008auc, hand2009measuring}.

\subsection*{F-score}

$F_1$ score (also F-score or F-measure) is a measure that considers both precision and recall by taking a harmonic average of these metrics.

\begin{equation}
F_1 = 2 \cdot \dfrac{ \text{precision} \cdot \text{recall} }{ \text{precision} + \text{recall} }
\end{equation}

A general formula of F-score for any positive real $\beta$ is:

\begin{equation}
F_\beta = (1 + \beta^2) \cdot \dfrac{ \text{precision} \cdot \text{recall} }{ (\beta^2 \cdot \text{precision}) + \text{recall} }
\end{equation}

Van Rijsbergen et al. \citep{van1979information} interpret $F_\beta$ that "measures the effectiveness of retrieval with respect to a user who attaches $\beta$ times as much importance to recall as precision". In practice, the F-score metric is often used in machine learning when the dataset is imbalanced because it isn't affected by the imbalance problem.


\section{Confusion matrices}
        \label{sec:confusion_matrices}

        We present some confusion matrices that help analyzing models performance on test data.

        \begin{figure}[H]
                \centering
                \includegraphics[width=0.8\linewidth]{lr.png}
                \caption{Logistic regression test confusion matrix.}
                \label{fig:lr}
        \end{figure}

        \begin{figure}[H]
                \centering
                \includegraphics[width=0.8\linewidth]{lda.png}
                \caption{Linear discriminant analysis test confusion matrix.}
                \label{fig:lda}
        \end{figure}

        \begin{figure}[H]
                \centering
                \includegraphics[width=0.8\linewidth]{k_nn.png}
                \caption{K-nearest neighbors test confusion matrix.}
                \label{fig:k_nn}
        \end{figure}

        \begin{figure}[H]
                \centering
                \includegraphics[width=0.8\linewidth]{rf.png}
                \caption{Random forest test confusion matrix.}
                \label{fig:rf}
        \end{figure}

        \begin{figure}[H]
                \centering
                \includegraphics[width=0.8\linewidth]{svm.png}
                \caption{Support vector machine test confusion matrix.}
                \label{fig:svm}
        \end{figure}


\documentclass[main.tex]{subfiles}
\begin{document}

% \marginpar{Friday\\ 2019-11-15, \\ compiled \\ \today}
% \section*{Fri Nov 15 2019}

% We start again from where we left off, with hydrogen recombination.

% We want to estimate the moment at which hydrogen first formed, which marks the point at which electrons and photons interact efficently: before, they interacted with Compton scattering which is very efficient; after they interact with hydrogen atoms in a way that is very inefficient.

% After this, then, we say that photons and matter are \emph{decoupled}.

% The scattering cross section (for Compton?) goes like the inverse square of the mass.

% This means that the universe is not only \emph{globally},  but also \emph{locally} neutral.

% This decoupling is what allows for star formation.
% Also, this is when the CMB starts.
% It is made of microwaves now, but it was higher earlier.

% The phase space distribution of photons is scale-invariant since they have zero mass: so we can say that the photons' distribution \emph{looks} thermal, but it is actually not technically since there are no interactions anymore.

% However the photons travel freely and are perceived as thermal, and they give an almost perfect blackbody! The errorbars in a plot for it must be magnified by \num{e4} in order to be seen.

% Neutrality implies \(n_e = n_p\).
Let us now introduce the total baryon density, \(n_b\).
In principle, we should account for Helium: as we will see, a couple of minutes after the Big Bang He-4 nuclei started to form, but they made up only something like \(25\%\) of the mass, which means \(6\%\) of the number density: so we ignore them and say that the number density of baryons is
%
\begin{align}
  n_b = n_p + n_H
\,.
\end{align}

% Our ansatz for the Boltzmann equation is \(\mu_e + \mu _p = \mu _H\) since photons have no chemical potential.

% \(i\) denotes a generic one in \(e\), \(p\) and \(H\).
% Then
% %
% \begin{align}
%   n_i = g_i\qty(\frac{m_i T}{2 \pi })^{3/2} \exp(\frac{\mu _i - m_i}{T})
% \,,
% \end{align}
% %
% we need to account for the chemical potential since it is the driver of this process.
The quantity we want to describe is how much hydrogen is still ionized: this is given by the ratio of the free electrons to the total baryons, \(n_e / n_b\). This is called the \textbf{ionization fraction} \(X_e\), and is also equal to \(n_p / n_b\) since the universe must be locally neutral.

We expect to have \(X_e = 1\) in the early universe, and \(X_e = 0\) after the end of reionization.

This is a good model to keep in mind although it is not precisely correct, since simulations show that there will be some \emph{residual ionization}: \(X_e\) only goes to around \(\num{e-4} \divisionsymbol \num{e-5}\) at the end of reionization.
Also, in the modern universe a large fraction of the hydrogen has become ionized again, especially in the intergalactic medium; this is likely due to the energy injected into it by structure formation, which definitely breaks the approximation of global thermal and chemical equilibrium.
This is not a concern for us: here, we only wish to describe the processes in the interval \(1000 \lesssim z \lesssim 2000\).

%  naively we'd expect it to get to 0, but actually there remains some residual ionization, some free protons and electrons.
% How much is \(n_e / n_b\)? the same as \(n_p / n_b\). We call this quantity \(X_e\), the ionization number.
% A proper calculation would account for the non-equilibrium contributions.
% However, we estimate the process as being in equilibrium: this will underestimate the number of electrons.

% Today, most of the hydrogen is ionized (there is a ***-Peterson effect which shows this): this means that matter and radiation interact again.

% This is the second important time in the history of the universe.

% Can we see the early stars? Not really, we see galaxies only up to \(z \sim 10\), these stars would be at something like \(z \sim 30\)\dots
% There might be more to this.

% Inserting this and the Saha equation we get
% So the number density of hydrogen atoms is given by:
%
% \begin{align}
%   n_H = g_H \qty(\frac{m_H T}{2 \pi })^{3/2} \exp(\frac{\mu_p - \mu_e - m_p - m_e + B}{T})
% \,,
% \end{align}
%
% and we can simplify things since \(m_p, m_H \gg m_e, B\), and substitute in the number densities for electrons and protons.
The binding energy of the hydrogen is \(B=m_p+m_e-m_H=\SI{13.6}{eV}\), so instead of \(m_H\) we can write \(m_H = m_p+m_e-B\).
Let us insert this expression and the Saha equation in the expression for the hydrogen number density; then we will recognize part of the expressions for the proton and electron number densities, which we can
%
\begin{subequations}
\begin{align}
  n_H &=  g_H \qty(\frac{m_H T}{2 \pi })^{3/2} \exp(\frac{\mu _H - m_H}{T})  \\
  &= g_H \qty(\frac{m_H T}{2 \pi })^{3/2} \exp(\frac{\mu_e + \mu _p - m_e - m_p + B}{T}) \\
  &= \underbrace{\frac{g_H}{g_e g_p}}_{= 1} \cancelto{}{\qty(\frac{m_H T}{2 \pi })^{3/2}}
  \qty(\frac{m_e T}{2 \pi })^{-3/2}
  \cancelto{}{\qty(\frac{m_p T}{2 \pi })^{-3/2}}
  n_e n_p
  \exp(\frac{B}{T}) \\
  \frac{n_H}{n_e n_p} &=  \qty(\frac{m_e T}{2 \pi })^{-3/2} \exp(\frac{B}{T})  \\
  \frac{n_b - n_p}{n_p^2} &= \qty(\frac{m_e T}{2 \pi })^{-3/2} \exp(\frac{B}{T})
\,,
\end{align}
\end{subequations}
%
which we can manipulate, using the following identity:
%
\begin{align}
  \frac{n_b - n_p}{n_p^2} = \frac{n_b \qty(1 - n_p/ n_b)}{n_b^2 X_e^2} = \frac{1}{n_b} \frac{1 - X_e}{X_e^2}
\,,
\end{align}
%
where we use \(n_e = n_p\) and the definition of \(X_e = n_p / n_b\). Then, we bring the \(n_b\) to the other side of the equation and multiply and divide by the photon number density, which is given by
%
\begin{align} \label{eq:n-gamma}
  n_{\gamma } = \frac{2 \zeta (3) T^3}{\pi^2}
\,.
\end{align}

With this, we can insert the baryon fraction \(\eta_b\) (which is conserved, so we can use its current value):
%
\begin{subequations}
\begin{align}
  \frac{1- X_e}{X_e^2} &= \underbrace{\frac{n_b}{n_\gamma }}_{\eta_p}
  n_\gamma
  \qty(\frac{m_e T}{2 \pi })^{-3/2} \exp(\frac{B}{T})   \\
  &= \eta_0 \frac{2 \zeta (3)T^3}{\pi^2} \qty(\frac{m_e T}{2 \pi })^{-3/2} \exp(\frac{B}{T})   \\
  &= \eta_0  \frac{4 \sqrt{2} \zeta (3)}{\sqrt{\pi }} \qty(\frac{T}{m_e})^{3/2} \exp(\frac{B}{T}) \label{eq:saha-equation-ionization}
\,,
\end{align}
\end{subequations}
%
which we can solve numerically to find \(X_e\) as a function of temperature, or of redshift. The results are shown in figure \ref{fig:ionization}.

\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{figures/ionization}
\caption{Ionization fraction as a function of redshift: a numerical solution of equation \eqref{eq:saha-equation-ionization}.}
\label{fig:ionization}
\end{figure}

Let us try to understand what is going on. Roughly speaking, we get intermediate values for \(X_e\), like \(\num{.5}\), when the right-hand side is of order 1.
If the right-hand side only contained the terms \((T/m_e)^{3/2}\exp(B/T)\) without any multiplicative factor in front (meaning, roughly, that we had \(\eta_0 \sim 1\)), then we would see recombination starting to occur already at \(z \sim 4000\) (\(T \sim \SI{1}{eV}\)), and by \(z \sim 2000\) if would almost be over.
The reason this does not happen is that \(\eta_0 \) is \emph{very small}: there are a lot of photons, many more than the baryons, which are ready to dissociate any atoms which start to form.

So, hydrogen is truly formed only when \(T \sim \SI{0.3}{eV}\), much lower than its ionization energy.

\todo[inline]{But even with \(\eta_0 \sim 1\) we get formation at \(T \sim \SI{1}{eV}\), an order of magnitude less than \(B\)! is there an intuitive argument as to why this is the case?}

The estimates we gave do depend on the value we assign to \(\Omega_{0b}\) and \(h\), however within the currently accepted experimental ranges the main predictions do not vary substantially.

% Approximately, it occurred somewhere around \(z \sim 1100\) (we conventionally say that recombination happened when \(X_e = 0.1\)).
The process of recombination is gradual, but we can choose a conventional redshift as, for example, the time when we reach an arbitrary threshold like \(X_e = \num{.1}\).

The interaction rate between photons and free electrons is \(\Gamma_{\gamma } = n_e \sigma_T c\), where \(\sigma _T\) is the Thompson cross section while \(n_e\) is the number density of free electrons, which can be estimated as \(n_e = n_b X_e \approx \rho_C \Omega_{b} X_e / m_p\).
% After recombination, we have the \emph{last scattering}: the moment at which the CMB was formed.
Then, we can estimate the moment of the last scattering by checking the decoupling condition: \(\Gamma _\gamma < H\).
With the numbers given earlier, we find \(z \sim 1120\): a very good approximation to the currently accepted value \(z \sim 1089\)!

% One Nobel prize this year was awarded to Jim Peebles, a friend of Sabino's: together with his PhD supervisor Dicke, he was the first to calculate this stuff.

% Peebles in 1964 (?) did this calculation both in GR and in Brahms-Dicke theory, a modified gravity theory.

\todo[inline]{Then Pacciani has an argument as to why \(T \sim a^{-2}\) for matter, should this be included here? It basically comes from entropy conservation \(\dd{E} = - P \dd{V}\), the ideal gas law \(P = n_b k_B T\) and the equipartition theorem \(E = \frac{3}{2} n_b a^3 k_B T\).}

\section{Primordial nucleosynthesis}

Already in the 1940s it was noticed by Alpher, Bethe as well as Gamow that the abundances of certain nuclides could not be explained if they were formed in stellar interiors alone.
Specifically, the issue is with the abundances of light elements: deuterium \(\ce{^{2}H}\), as well as \(\ce{^{3}He}\) and \(\ce{^{4}He}\), while heavier nuclides (with mass number \(A \geq 7\)) could not form in the early universe \cite[sec.\ 8.6.1]{colespCosmology2002}.\footnote{Environments which allow their production (stellar interiors) have lower temperature but a higher density, since the main obstruction to their production is the absence of stable nuclides at \(A = 5\) and \(A= 8\) --- the environment needs to raise the odds of an unstable \ce{^{8}Be} nuclide colliding with an \(\alpha \) particle to form carbon before it decays.}

The most abundant of these light nuclides by far is \ce{^{4}He}, whose mass fraction is denoted as \(Y \approx \SI{25}{\percent}\), while its number fraction is approximately \SI{6}{\percent}. Our model will need to yield this many helium-4 nuclides after primordial nucleosynthesis.

A mechanism for the synthesis of these light nuclides in the early universe is needed.
% Let us describe the early universe, before the first nucleosynthesis.
% Important papers in this topic are by G. Gamow, and by Alpher, Bethe and Gamow.

We shall model this mechanism, under the following assumptions (which are, as far as we know, roughly verified):
%
\begin{enumerate}
    \item the universe passed through a very high temperature phase, with \(T > \SI{e12}{K}\), during which thermal equilibrium held;
    \item the universe at this stage is described by General Relativity and the Standard Model of particle physics, and it is homogeneous and isotropic;
    \item the chemical potentials for the neutrinos \(\mu_{\nu }\) have certain upper bounds, such that the number of neutrino types is approximately 3;
    \item there is no matter-antimatter separation (as in, antimatter ``bubbles'');
    \item there are no strong magnetic fields;
    \item the number of exotic particles has a certain upper bound (it is small compared to the number of photons).
\end{enumerate}

% There are magnetic fields in the universe, but they are not homogeneous and relatively weak.
% Exotic particles are predicted by certain unification theories, they are generically defined as ones which we have not observed yet.

% We have to explain the fact that we observe an excess of He-4 in the early universe: we define the yield
%
% \begin{align}
%   y \equiv \frac{m_{\ce{He-4}}}{m_b} > \num{.25}
% \,.
% \end{align}

% In terms of particle number, the ratio is more like \num{.06}.
% We do not produce Carbon or anything higher than it: the process which forms it is inefficient at high temperature, low density like the early universe.
% Higher \(Z\) elements are only produced in stars.

% Helium-4 is produced but also destroyed by stars.

The main formation channels in the early universe are:
%
\begin{enumerate}
    \item \(n+p \leftrightarrow d + \gamma \) (\(d= \ce{^{2}H}\) denotes deuterium);
    \item \(d + d \leftrightarrow \ce{^{3}He} + n\);
    \item \(\ce{^{3}He} + d \leftrightarrow \ce{^{4}He} + p\).
\end{enumerate}

Note that these processes, unlike the stellar ones, do not involve the weak interaction: neutrons and protons do not turn into each other.
In stars, there are no free neutrons so this process is not possible.

The slowest process of the three is the first, since it is heavily affected by photons, which destroy deuterium.
After we have produced deuterium, Helium-4 is readily produced.
% We cannot treat this properly, we only give a story.

% The binding energy of deuterium is around \SI{2.2}{MeV}.

In order to find out how much deuterium we have, we need the proton-to-neutron ratio.
We are working at energies of around \SI{1}{MeV}, so protons and neutrons are not relativistic anymore; \emph{as long as they are in equilibrium} through weak processes both will obey Boltzmann statistics, so for \(i = n, p\):
% his process takes place around three minutes after the beginning.
%
\begin{align}
  n_i = g_{i} \qty(\frac{m_i T}{2 \pi })^{3/2} \exp(\frac{\mu_i - m_i}{T})
\,,
\end{align}
%
meaning that their number ratio is given by\footnote{We are neglecting the chemical potentials since, as explained by \textcite[sec.\ 8.6.2]{colespCosmology2002}, as long as both weak and electromagnetic interactions are in chemical equilibrium the chemical potentials are forced to be zero by all the balance equations.}
\todo[inline]{The explanation in \cite[]{colespCosmology2002} is not the same as the one given by \textcite[]{paccianiAppuntiCorsoPhysical2018}\dots I'm inclined to trust the former.}
%
\begin{align}
  \frac{n_n}{n_p} \sim \exp(-\frac{m_n - m_p}{T})
\,,
\end{align}
%
where \(m_n - m_p \approx \SI{1.3}{MeV} \approx \SI{1.5e10}{K}\).

The proton is the lightest baryon and is therefore stable; while the neutron is unstable: it can decay through the weak-interaction processes
\begin{enumerate}
    \item \(n + \nu _e \leftrightarrow p + e^{-}\);
    \item \(n + e^{+} \leftrightarrow p + \overline{\nu _e} \);
    \item \(n \rightarrow p + e^{-} + \overline{\nu _e} \).
\end{enumerate}

The neutron fraction keeps decreasing as \(T\) decreases and these processes keep happening, however as we have previously discussed at around \(T_{d \nu } \sim \SI{1}{MeV}\) neutrinos decouple, at which point the first two back-and-forth reactions stop, and we are left with \(n_n / n_p \approx \exp(- \Delta m / T_{d \nu }) \approx \num{.27}\).
% We can replace the temperature in the exponential by \(T \rightarrow T_{d_\nu }\), the decoupling temperature of the neutrinos, since that is the moment around which this happpens.

We define the number fraction of neutrons, which is approximately
%
\begin{align}
  X_n (t) \equiv \frac{n_n}{n_n+n_p} \approx \num{.21}
\,.
\end{align}

The third reaction, which is \(\beta^{-} \) decay, keeps occurring since it does not require the presence of neutrinos, so after the decoupling of neutrinos the number fraction decays exponentially as:
%
\begin{align}
  X_n(t) = X_n(t_{d_\nu }) \exp( - \frac{t - t_{d_\nu }}{\tau _n})
\,,
\end{align}
%
where \(\tau _n = \log 2 \tau_{1/2}\), and the half-life of neutrons is given by \(\tau_{1/2} \approx \SI{10.5+-0.2}{min}\).
So, each minute neutrons stay unbound some of them are decaying; the process of deuterium formation however is rather fast as we shall see, so not many of them are lost.

Let us then move to deuterium formation: its binding energy is around \(B_d = m_p + m_n - m_d \approx \SI{2.2}{MeV}\).
We proceed exactly like we did with hydrogen: since \(\mu _p + \mu _n = \mu _d\), the deuterium number density can be expressed as
%
\begin{align}
n_d &= g_d \qty(\frac{m_d T}{2 \pi })^{3/2}  \exp(\frac{\mu _d - m_d}{T})  \\
&= \frac{g_d}{g_p g_n} n_n n_p \qty(\frac{m_d}{m_n m_p})^{3/2} \qty(\frac{T}{2 \pi })^{-3/2} \exp( \frac{B_d}{T})
\,,
\end{align}
%
which, dividing through by \(n_b\) and using \(g_d = 3\) (since deuterium has spin 1) and \(g_p = g_n = 2\), can be expressed as:
%
\begin{subequations}
\begin{align}
  X_d &= \frac{3}{4} n_b X_n X_p \qty(\frac{m_d}{m_nm_p})^{3/2} \qty(\frac{T}{2 \pi })^{-3/2} \exp(\frac{B_d}{T}) \\
  &= \frac{3}{4} \eta_0 X_n X_p \qty(\frac{m_d}{m_nm_p})^{3/2}
  \frac{2 \zeta (3)}{\pi^2} (2 \pi T)^{3/2}  \exp(\frac{B_d}{T})
  \marginnote{Substituted \(n_b = \eta_0 n_\gamma \).}\\
  &\approx \frac{3}{4} \eta_0 X_n (1-X_n) \qty(\frac{m_d}{m_nm_p})^{3/2} \frac{2}{\pi^2} (2 \pi T)^{3/2} \zeta (3)  \exp(\frac{B_d}{T})
  \marginnote{Approximated \(X_p + X_n \approx 1\), ignoring heavier nuclides.}
\,,
\end{align}
\end{subequations}
% \todo[inline]{Check calculation.}
\todo[inline]{Does approximating \(X_p + X_n \approx 1\) not ignore deuterium as well? Or rather: by the definition given before \(X_n + X_p = 1\) is exact, and if we do normalize by \(n_n + n_p + n_d +\dots\) we should specify\dots}
%
which describes the \emph{deuterium bottleneck}: similarly to hydrogen recombination, the presence of many photons for each nuclide keeps compound particles from forming for quite a long time, and this precludes the formation of Helium.

\end{document}


%!TEX ROOT=sample-thesis.tex
%!TEX PROGRAM=xelatex

\chapter{List of Symbols}

{\renewcommand{\arraystretch}{2}
\begin{tabular}{@{} >{\bfseries\boldmath }l @{\hspace{1em}--\hspace{1em}} l@{}}

$\theta$ & temperature\\

$G$ & Gravitational acceleration $= \SI{9.81}{\meter\per\second}$

\end{tabular}
}


\documentclass[12pt]{article}

\usepackage{fullpage}
\usepackage{/home/nick/Documents/analysis}
\usepackage{listings}
\usepackage{verbatim}

\begin{document}
\title{FrameLab\\\small{Software for framelet-based numerical analysis}}
\author{Nick Henscheid\\\small{ University of Arizona Program in Applied Mathematics}}
\date{\today}
\maketitle

\paragraph{Goals}
The idea of this project is to develop a complete software suite for computational imaging via framelets.  The code will be sorted into the following general categories:

\begin{enumerate}
\item Framelet expansion code
\begin{itemize}
\item Fast single and multilevel decomposition and reconstruction for 1D through 3D
\item Signal sparsity analysis, best basis selection?
\item Computing framelet coefficients
\end{itemize}
\item Numerical optimization code
\begin{itemize}
\item $l_1$ compressed sensing via ADMM/AL/DAL
\item Comparison of different algorithms?
\item $l_0$ code
\end{itemize}
\item Utilities
\begin{itemize}
\item Graphics and figure creation
\item Shrinkage designer
\item PDEs for image processing
\item Forward/adjoint operators for imaging
\item Classical signal processing
\end{itemize}
\item Applications
\begin{itemize}
\item CT
\item MRI
\item SPECT
\item Segmentation
\item Registration?
\item Deblurring
\item Deconvolution
\item Integral equations?
\item Matrix factorization?
\item Matrix completion
\item BM3D
\end{itemize}
\end{enumerate}

\section{Implementation Details}

\paragraph{Programming languages and interfaces}
To do:
\begin{itemize}
\item GPU interface to python?  pycuda?  (e.g. python alternative to mex)
\end{itemize}

\paragraph{Data Types}

\section{Framelet Expansions}
The first task to create/modify functions to perform framelet decomposition and reconstruction

\section{Optimization Methods}
%\lstinputlisting[numbers=left,language=Matlab,frame=single,caption={}Matlab function to solve the analysis-based sparse recovery problem via Augmented Lagrangian.]{../optimization/ALsolve.m}

\section{Imaging Systems}
\subsection{2D Computed Tomography}

\subsection{3D Computed Tomography}

\section{Test Scripts}
\begin{enumerate}
\item testAdjoint.m
\item testBall.m
\item testCBCTCGLS.m
\item testCBCTInterp.m
\item testCBCTInterpAugmented.m
\item testCGLSClass.m
\item testConeBeamScanner.m
\item
\end{enumerate}

\end{document}


\chapter{Inference of Guarded Recursion}
\todo{Should we mention somewhere that we don't plan on running these? And can
  we even do that? What about evaluation in types?}
\label{cha:infer-guard-recurs}
As mentioned in Section~\ref{sec:less-restr-prod}, we wish to extend Idris with
a less restrictive productivity checker using guarded recursion. However, as
we saw in Section~\ref{sec:guarded-recursion}, defining guarded recursive
programs can be quite tedious and requires the user to thoroughly understand guarded
recursion in order to write provably productive programs. In this chapter, we propose a system
where we can check the productivity of programs using guarded recursion with
only a minimal amount of user involvement. The main focus will be
implementation, showing how we have put theory into practice.
\section{Intuition}
%##########
% Type-directed (undersøg præcis hvad dette betyder)
% Overgang fra teori til Idris - Hvilke praktiske hensyn tages der?
% Fix-punktet - hvordan forstås det?
% Kun eet ur!
% Kausale og ikke-kausale funktioner
%% Problemet bag: hvorfor er vi nødt til at skelne?
% Eksempel på inferens
%##########
Our system is divided into two parts: (1) An inference system that builds a
guarded recursive term from a user written term, and (2) an algorithm that
checks the structure of a term and its type according to the rules from
Figure~\ref{fig:guarded_recursion_rules_clocks}. While the general idea is that
these two parts work together, such that the inference system first builds a term
for the checking algorithm to check, it is our goal that these two should work
as disjoint modules. This is important as we want our checking algorithm to be a
general-purpose guarded recursion checker, not just an algorithm for checking
the output of the inference system.

Furthermore, it is important to note that our system is an extension of
Idris. This means that we do not add or change any core elements of the language, and
therefore do not jeopardise or tamper with the correctness of the Idris type system.

\subsection{Practical Considerations}
When realising a theoretical model such as guarded recursion in an existing
system, there are practical considerations to be taken. Because of the way Idris
is implemented, how we construe the theory might not be immediately obvious. In
the following we will lay out these practical considerations, and how they
relate to the theory.

\paragraph{Target}
As discussed in Chapter~\ref{cha:idris}, an Idris program is represented in
multiple different ways throughout compilation. The time and representation on
which to perform our analysis has great impact on how a solution
unfolds.

Discarding the representations obviously not useful for this analysis (namely
concrete Idris, IBC, and binaries), we are left with two options: \IdrisM and
TT. As guarded recursion is a typing discipline our system needs access to
the types of any term, and since \IdrisM does not contain all type information
our analysis is performed on the core language, TT.

\paragraph{Parameters}
In Section~\ref{sec:guarded-recursion} we discussed and used a guarded fixpoint
operator. Due to the type of this operator, ($(\later A \rightarrow A)
\rightarrow A$), the recursive reference will always have a $\later$-type. This
means that any application of a recursive reference to an argument must be done
using $\tensor ^{\kappa}$\todo{This operator must be given a human-readable name}.

Idris, like many other languages, have named parameters which influence the type
of a function. Looking at the rule for $\tensor ^{\kappa}$ in
Figure~\ref{fig:guarded_recursion_rules} we see that it does not take any type
substitution into account unlike the rules for regular application in Idris from
Figure~\ref{fig:TT_typing_rules}. This means that we cannot use $\tensor ^{\kappa}$ for
parameter application. To better understand this, consider the following example.

\begin{figure}[h]
  \begin{lstlisting}[mathescape]
repeat : $\forall \kappa$ ((a : Type) $\rightarrow$ a $\rightarrow$ Stream$^{\kappa}$ a)
repeat = $\Lambda\kappa.$ fix$^{\kappa}$($\lambda{}$rec$.\lambda{}$a$.\lambda{}$n$.$
             StreamCons a n ((rec $\tensor ^{\kappa}$ (Next$^{\kappa}$ a)) $\tensor ^{\kappa}$ (Next$^{\kappa}$ n)))
\end{lstlisting}
  \caption{An infinite stream of the same element.}
  \label{fig:repeat_guarded_example}
\end{figure}

In Figure~\ref{fig:repeat_guarded_example} an infinite stream \texttt{repeat} is
defined. It simply just repeats the same element infinitely. Note that even
though this problem also applies to implicit arguments, all arguments in the
example are explicit for simplicity. This means that \texttt{StreamCons} has
type \texttt{(a : Type) $\rightarrow$ a $\rightarrow$ $\later^{\kappa}$ Stream$^{\kappa}$ a $\rightarrow$ Stream$^{\kappa}$ a}.

When checking if this implementation conforms to the guarded rules we have to
check that \texttt{(rec $\tensor ^{\kappa}$ (Next$^{\kappa}$ a)) $\tensor ^{\kappa}$ (Next$^{\kappa}$ n)} has type
\texttt{$\later ^{\kappa}$Stream$^{\kappa}$a} we run into trouble. In order for this to hold,
according to the $\tensor ^{\kappa}$ rule, \texttt{rec $\tensor ^{\kappa}$ (Next$^{\kappa}$ a)} must have
type \texttt{$\later^{\kappa}$(a $\rightarrow$ Stream$^{\kappa}$ a)}. As shown in
Figure~\ref{fig:repeat_typing} we cannot show this as \texttt{rec} has a
different type in our environment to what we are trying to show.

\begin{figure}[h]
\[
\frac { \begin{matrix} \inference { \frac { ? }{ \Gamma '\, \vdash \, rec\, :\, \later
        ^{\kappa}(Type\, \rightarrow \, a\, \rightarrow \, Stream^{\kappa}\, a) } \,
      \frac {
        \frac {  }{ \Gamma '\, \vdash \, a\, :\, Type }
      }
      { \Gamma '\, \vdash \,
        Next^{\kappa}\, a\, :\, \later ^{\kappa}\, Type }
    }{ \, \Gamma '\, \vdash \, rec\,
      \tensor ^{\kappa} \, (Next^{\kappa}\, a)\, :\, \later ^{\kappa}(a\, \rightarrow \, Stream^{\kappa}\,
      a)\ }  & \inference { \inference {  }{ \Gamma '\, \vdash \, n\, :\, a }  }{
      \Gamma '\, \vdash \, Next^{\kappa}\, n\, :\, \later ^{\kappa} \, a }  \end{matrix} }{
  \Gamma '\, \vdash \, (rec\, \tensor ^{\kappa} \, (Next^{\kappa}\, a))\, \tensor ^{\kappa} \,
  (Next^{\kappa}\, n))\, :\, \later ^{\kappa} Stream^{\kappa}\, a }
\]
\[
\Gamma '\, =\, \Gamma ,\, rec\, :\, \later ^{\kappa}((a\, :\, Type)\, \rightarrow \,
a\, \rightarrow \, Stream^{\kappa}\, a),\, a\, :\, Type,\, n\, :\, a
\]
  \caption{Part of an attempt to type repeat.}
  \label{fig:repeat_typing}
\end{figure}

One way of solving this could be to have the $\tensor ^{\kappa}$-rule take substitution in
types into account as seen in Figure~\ref{fig:tensor_with_subst}. This, however,
is not yet backed by the theory, and is therefore not an option.

\begin{figure}[h]
\[
\inference { \Gamma \, \vdash \, t\, :\, \later^{\kappa} ((a\, :\, A)\,
  \rightarrow \, B)\quad \Gamma \, \vdash \, u\, :\, \later^{\kappa} A }{
  \Gamma \, \vdash \, t\, \tensor ^{\kappa} \, u\, :\, \later^{\kappa} B[{ u
  }/{ a }] }
\]
  \caption{Tensor with substitution in types.}
  \label{fig:tensor_with_subst}
\end{figure}

Instead we change what is fixed by the fixpoint. We pull any parameter out,
changing the fixed type, such that \texttt{rec} now has type \texttt{$\later
  ^{\kappa}$(a $\rightarrow$ Stream$^{\kappa}$ a)}. This gives us the definition seen in
Figure~\ref{fig:repeat_guarded_example_new}. Here the \texttt{a} is fixed and
not part of the recursive reference. This allows us to type part of
\texttt{repeat} as exemplified in Figure~\ref{fig:repeat_typing_new}.

\begin{figure}[h]
  \begin{lstlisting}[mathescape]
repeat : (a : Type) $\rightarrow$ $\forall \kappa$ (a $\rightarrow$ Stream$^{\kappa}$ a)
repeat a = $\Lambda \kappa$ fix$^{\kappa}$($\lambda{}$rec$.\lambda{}$n$.$
             StreamCons a n (rec $\tensor ^{\kappa}$ (Next$^{\kappa}$$^{\kappa}$ n)))
\end{lstlisting}
  \caption{Repeat with the type parameter fixed.}
  \label{fig:repeat_guarded_example_new}
\end{figure}

\begin{figure}[h]
\[
\frac { \frac {  }{ \Gamma '\, \vdash \, rec\, :\, \later ^{\kappa}(a\, \rightarrow
    \, Stream^{\kappa}\, a) } \, \frac { \Gamma '\, \vdash \, n\, :\, a }{ \Gamma '\,
    \vdash \, Next^{\kappa}\, n\, :\, \later ^{\kappa}\, a }  }{ \, \Gamma '\, \vdash \, rec\,
  \tensor ^{\kappa} \, (Next^{\kappa}\, n)\, :\, \later ^{\kappa}Stream^{\kappa}\, a }
\]
\[
 \Gamma '\, =\, \Gamma ,\, rec\, :\, \later ^{\kappa}(a\, \rightarrow \, Stream^{\kappa}\,
 a),\, a\, :\, Type,\, n\, :\, a
\]

  \caption{Part of typing repeat with fixed type parameter.}
  \label{fig:repeat_typing_new}
\end{figure}

\paragraph{Multiple clauses}
Another problem occurs when we consider functions with multiple clauses,
e.g. functions defined with pattern matching. To understand the problem we must
first step out of the guarded recursive world for a moment and consider the
function \texttt{cycle} in Figure~\ref{fig:cycle_non_guarded}. Note that it is
not a guarded recursive definition, but a standard Idris definition.

\begin{figure}[h]
  \begin{lstlisting}[mathescape]
cycle : Nat $\rightarrow$ Nat $\rightarrow$ Stream Nat
cycle Z m = Z :: (cycle m m)
cycle (S n) m = (S n) :: (cycle n m)
\end{lstlisting}
  \caption{A function cycle in standard Idris}
  \label{fig:cycle_non_guarded}
\end{figure}

When transforming this to a guarded recursive form we want to make the fixed
point over both clauses. An intuitive way to get around this could be to
convert the pattern matching match into case-expressions. This is not possible
as TT does not have case-expressions, only top-level pattern
matching. Furthermore, a case-expression based solution would make even less
sense when we consider dependent pattern matching, since we by removing the
dependent pattern match change the type of the left hand side.

Instead, we can define cycle as two functions: One with the fixed point, and one
with the pattern match, as seen in Figure~\ref{fig:cycle_guarded}. This way, all
clauses of \texttt{cycle} are part of the same fixpoint. The recursive reference
\texttt{ref} is handed to \texttt{cycle} as a parameter. Note that this way of
handling the fixed point also works for functions with only one clause.

\begin{figure}[h]
\begin{lstlisting}[mathescape]
cycle : $\later ^{\kappa}$ (Nat $\rightarrow$ Nat $\rightarrow$ Stream$^{\kappa}$ Nat) $\rightarrow$
             Nat $\rightarrow$ Nat $\rightarrow$ Stream$^{\kappa}$ Nat
cycle rec    Z  m =
         StreamCons Z     (rec $\tensor ^{\kappa}$ (Next$^{\kappa}$ m)) $\tensor ^{\kappa}$ (Next$^{\kappa}$ m)
cycle rec (S n) m =
         StreamCons (S n) (rec $\tensor ^{\kappa}$ (Next$^{\kappa}$ n)) $\tensor ^{\kappa}$ (Next$^{\kappa}$ m)

cycle$'$ : $\forall \kappa$ (Nat $\rightarrow$ Nat $\rightarrow$ Stream$^{\kappa}$ Nat)
cycle$'$ = $\Lambda \kappa$ fix$^{\kappa}$($\lambda$rec.$\lambda$n.$\lambda$m. cycle rec n m)
\end{lstlisting}
  \caption{A guarded definition of cycle}
  \label{fig:cycle_guarded}
\end{figure}

Again, this solution has short comings. We fix the recursive
reference as \texttt{ref} and thereby also fix its type. This works out fine in
the simply typed case, but once we consider higher-order and especially
dependent types we run into problems. If we consider the function prepend from
Figure~\ref{fig:guarded_prepend}, the \texttt{n} in the type of \texttt{rec} is
fixed, e.g. in the second clause to \texttt{S n}. Since \texttt{xs} has type
\texttt{Vect n a}, this term is not well-typed.

\begin{figure}[h]
\begin{lstlisting}[mathescape]
prepend : (Later' (Vect n a -> Stream a -> Stream a)) -> Vect n a -> Stream a -> Stream a
prepend rec [] s = s
prepend rec (x :: xs) s = StreamCons x (compose {n=Now} (compose {n=Now} rec (Next xs)) (Next s))

prepend' : Forall (Vect n a -> Stream a -> Stream a)
prepend' = LambdaKappa (fix(\rec, xs, s => prepend rec xs s))
\end{lstlisting}
  \caption{A function prepending a vector on a stream.}
  \label{fig:guarded_prepend}
\end{figure}\todo{Prettify and verify figure}

To circumvent this we eliminate the \texttt{fix} operator all together according
to the elimination rules in Figure~\ref{fig:fix_elim_rules}. By eliminating the
fixpoint, we no longer fix the type of the recursive reference. Because of the
type of the guarded fixed point operator, the type of \texttt{rec} is
$\later^{\kappa}$A, which is also the type of \texttt{Next f}.

\begin{figure}[h]
  \[
\frac { \Delta ,\, \kappa ; \, \Gamma \, \vdash \, f\, =\, { fix }^{ \kappa  }(\lambda
  rec.\, e)\, :\, A }{ \Delta ,\, \kappa \, ; \Gamma \vdash \, f\, =\, e[{ (Next^{ \kappa
    }\, f) }/{ rec }]\, :\, A } fix_{E_1}
\]

\[
\frac { \Delta \, ; \Gamma \vdash \, f\, =\, \Lambda \kappa .{ fix }^{ \kappa  }(\lambda
  rec.\, e)\, :\, \forall \kappa .A }{ \Delta \, ; \Gamma \vdash \, f\, =\, \Lambda
  \kappa .e[{ (Next^{ \kappa  }\, f[\kappa ]) }/{ rec }]\, :\, \forall \kappa .A
} fix_{E_2}
\]
  \caption{Rules for fix elimination}
  \label{fig:fix_elim_rules}
\end{figure}

\paragraph{Singleton Clock}
To simplify the implementation we have limited our system to only have one
clock, rather than a clock environment. Either the clock is open, or the clock
is closed. This changes the rules we discussed in
Section~\ref{sec:guarded-recursion}, as seen in
Figure~\ref{fig:gr_rules_sin_clock}\todo{insert one clock rules}. Here
$\sqcup$ denotes the open clock, and $\sqcap$ the closed. \todo{Explain how we
  came to these rules}

\begin{figure}[h]

  \caption{Guarded Recursion rules with singleton clock.}
  \label{fig:gr_rules_sin_clock}
\end{figure}

While deriving this rule set from the original one has not followed a specific
methodology, a general trend is:

\begin{itemize}
\item If a rule required a specific clock in the environment, it now requires an
  open clock.
\item Side conditions talking about a specific clock being not being free in the
  environment are transformed to that no free clocks are allowed in the
  environment at all.
\end{itemize}

Note that the $\kappa$ has gone from a lot of the types and terms. This is
because we no longer talk about specific clocks, but about the
clock. I.e. $\later$ is always on the same clock and as such there is no need to
specify which. Also note that the $\kappa$ remains on some $\forall \kappa$ and
$\Lambda \kappa$. While these still only mention the singleton clock, the
$\kappa$ is kept to disambiguate these from regular quantification and
abstraction.

This change of course limits the set of programs for which we can infer guarded
recursion, and thereby ensure productivity. In Section~\ref{cha:evaluation} we
discuss how this limitation affects our system.

\paragraph{Modality of Functions}
In general, when defining a guarded recursive function, one has to consider how
to quantify over clocks. For example, in the definition of \texttt{repeat} from
Figure~\ref{fig:repeat_guarded_example_new} the clock quantification is on the
entire type, where as in the definition of \texttt{evens} from
Figure~\ref{fig:guarded_recursion_evens}, the clock quantification is on each
individual \texttt{Stream}. As there we increase the number of arguments we also
increase the number of different way we could quantify the function, making it
increasingly difficult to infer how to place them.

Because of this we have simplified to the problem. A function is either
\emph{causal} or \emph{not causal}. We call this the \emph{modality} of a
function. If a function is causal it means that the quantification is on the
entire type. In this sense \texttt{repeat} is said to be causal. If a function
is not causal the quantification is on each individual guarded type, meaning
that \texttt{evens} is \emph{not} causal.

While inferring the modality of a function is indeed be possible, simply by
trial and error, there might exist not guarded function with both a causal and
not causal guarded implementation. In such a situation the system would have no
way of knowing which modality to assign the function.

Instead of attempting to infer the modality, we let the user assign a modality
to a function. By doing this we ensure that the system does not infer a function
with the wrong modality.\todo{Explain what this means for recursive ref}

\section{Implementation}
In this section we go over how we have realized the above intuition into an
actual Idris implementation. We start by presenting the overall additions to the
Idris compiler environment, and then how the inference and checker systems have
been implemented.

\paragraph{Guarded Recursion Library}
The first step to adding guarded recursion to Idris is to add the guarded
recursive types and functions. We have added these to the Idris built-in
library, and not as a part of the prelude. This is because the user should be
able to check functions for productivity using our guarded recursion checker
without having to rely on the standard library.  While most of the
implementations are straight forward, and can be found in Appendix\todo{Add ref
  to appendix. And add appendix!}, there are interesting parts to discuss. We
have added the notion of \emph{how much later} something is, through the idea of
\texttt{Availability}, seen in Figure~\ref{fig:availability}. This is, in
conjunction with \texttt{Later} done so that we can many $\later$ applications
as a single \texttt{Later} applications.

\begin{figure}[h]
  \begin{lstlisting}[mathescape]
data Availability = Now | Tomorrow Availability

data Later' : Type -> Type where
  Next : {a : Type} -> a -> Later' a

Later : Availability -> Type -> Type
Later Now a = a
Later (Tomorrow n) a = Later' (Later n a)
\end{lstlisting}
  \caption{Availability.}
  \label{fig:availability}
\end{figure}

\paragraph{Guarded Names}
As we want to infer new types and terms, and not override the existing ones, we
need a system for creating a guarded name from an existing user written
name. While this is fairly trivial, it is an important part of our system. We
will hence forth refer to these as the \emph{guarded names} as oppose to their
original \emph{Idris names}. We will mark these guarded names with a subscript
$g$, such that a name $Name$ becomes $_gName$.

During compilation we keep a map from Idris names to their guarded names. This
means that we can for any Idris name find their guarded name, and vice
versa. This comes in handy when we infer the guarded terms.

\paragraph{Clocked Types}
These are made by inferring a new data declaration for any codata or corecord
declaration. This new data declaration is simply constructed during the
elaboration of the original and then elaborated by itself. This means that for
any user written codata or corecord declaration a guarded version is an
intrinsic part of the program.

A such guarded version of a declaration is fairly straight forward. Guarded
names are given to the type and to all constructors. Any recursive reference in
the type of the constructor is placed under a $\later$-type, in a similar
fashion to how Idris already handles lazy evaluation. The type constructor
remains unchanged. An example of this inferences for a codata declaration can be
seen in Figure~\ref{fig:guarded_stream_inf}. Note that the inferred declaration
is a data declaration, not codata. This is because we do not need both Idris's
built in laziness (\texttt{Inf}), and the guarded recursion laziness ($\later$)
\todo{Explain why these are not the same}.

\begin{figure}[h]
\begin{lstlisting}[mathescape]
codata Stream : Type -> Type where
  MkStream : a -> Stream a -> Stream a

data $_g$Stream : Type -> Type where
  $_g$MkStream : a -> $\later_g$Stream a -> $_g$Stream a
\end{lstlisting}
  \caption{Inference of the guarded stream declaration.}
  \label{fig:guarded_stream_inf}
\end{figure}

In the corecord case, we take a similar approach. However, instead of
constructors we have to guard projections. Again, any recursive projection is
guarded such that $A \rightarrow A$ becomes $A \rightarrow \later A$, and all
projections are given a guarded name. Figure~\ref{fig:guarded_tree_inf} shows an
example of such an inference.

\begin{figure}[h]
\begin{lstlisting}[mathescape]
corecord Tree a where
  left : Tree a -> Tree a
  value : Tree a -> a
  right : Tree a -> Tree a

record $_g$Tree a where
  $_g$left : $_g$Tree a -> $\later_g$Tree a
  $_g$value : $_g$Tree a -> a
  $_g$right : $_g$Tree a -> $\later_g$Tree a
\end{lstlisting}
  \caption{Inference of the guarded tree declaration.}
  \label{fig:guarded_tree_inf}
\end{figure}

Furthermore, for each projection we also generate what we call a \emph{forall
  projection}, henceforth denoted as $_\forall Name$. We use this for
projections on quantified types. Consider a coinductive type $A$ with a
projection $p$ of type $A \rightarrow A$, and a variable $x$ of type $\forall
\kappa.A$. In order to perform $p$ on $x$ and maintain the quantification in the
type, we must first apply the clock to $x$, apply $_gp$, and then abstract over the
clock again using $\Lambda \kappa$. This gives us:

\[
\Lambda \kappa . _gp (apply\,x) : \forall \kappa . \later A
\]

The type of the above is isomorphic with $\forall \kappa . A$ according to the
isomorphism described by Rasmus M\o gelberg\,\cite{Mogelberg:2014} seen in
Figure~\ref{fig:quantified_later_iso}. As a short for all of this we simply use
$_\forall p$ which has type $\forall A \rightarrow \forall A$, giving us the
same result. As such these \emph{forall projections} do not add anything new,
they are just a simpler way for us to perform projections on terms of quantified
type. The intuition is that because of Figure~\ref{fig:quantified_later_iso} we
can perform projections on quantified types without having to worrying about
their lateness. Because they are quantified over clocks, they are always available.

\begin{figure}[h]
\[
\forall \kappa .A\cong \forall \kappa .\rhd ^\kappa A
\]
  \caption{A type isomorphism describe by Rasmus M\o gelberg\,\cite{Mogelberg:2014}.}
  \label{fig:quantified_later_iso}
\end{figure}

\subsection{Inference}
% Regler - forklar (også notation)
% Rekursive referencer
% Eksempel - zeros?
% Preprocessing
% Renaming
% Forskel på causal og non causal

%(1) Gennemgang af interessante regler
%(2) Eksempel
%(3) Implementation details, e.g. when Next and tensor rules clash
%%(3.1) Preprocessing, herunder renaming
%%(3.2) Forskel på causal og non causal

Inference of guarded recursive TT terms from elaborated TT terms happens by a
bottom-up, type-directed derivation according to the rules given in Figure~\ref{fig:epsilon_rules}\todo{Behøver vi tjekke :Type præmisserne her?}. Let
$c$ be the singleton clock, $\Gamma$ be a typing environment, $\rho$ be a tuple matching an Idris name $n$ to a
guarded name $_gn$ (and for projections, a name for its forall-projection, $_{\forall}n$) and $\phi$ be a (possibly empty) collection of such tuples:
\begin{align*}
   c,\,c^\prime &::=\,\sqcup\,|\,\sqcap \\
   \rho &::= (n,\, _gn)\,|\,(n,\, _gn,\, _{\forall}n)  \\
   \phi &::=\,\cdot\,|\,\rho\,\phi \\
   IE,\,IE^\prime &::= \phi;\,c;\,\Gamma
\end{align*}
The singleton clock can appear either as open ($\sqcup$), closed ($\sqcap$), or
irrelevant ($c$). Then guarded recursive terms are derived in an inference environment $IE$, such that
${IE\vdash e : A \overset{B}{\Longrightarrow} IE^\prime \vdash e^\prime : B}$ describes the
transformation ($\overset{B}{\Longrightarrow}$) of an elaborated TT term $e$ of type
$A$ in $IE$ to a guarded recursive TT term $e^\prime$ of type $B$ in
$IE^\prime$. The $B$ in $\overset{B}{\Longrightarrow}$ implies that the transformation
is type-directed, since it denotes that the desired output type $B$ is input to
the transformation function.

In Figure~\ref{fig:epsilon_rules}, the $\Longrightarrow_{Next^\kappa}$ rule follows directly from the $Next$ rule in Figure~\ref{fig:guarded_recursion_rules_clocks}. The
$\Longrightarrow_{\Lambda\kappa}$ and $\Longrightarrow_{apply^\kappa}$ rules have a
similar structure, but here the clock changes state from premise to conclusion
(e.g. from open to closed). For handling guarded names, the
$\Longrightarrow_{\phi}$ rule says that if some Idris name has type $A$, and its
guarded version has type $A^\prime$, then we can insert the guarded name in its
place. Note that the type $A^\prime$ of the guarded name is not arbitrary, but follows the
rules for causal and non-causal function types (see Section~\todo{insert ref
  here}). Renaming is important for the $\Longrightarrow_{App}$ rule, since it
is often used for the transformations required in the premises. This reasoning
also applies to the $\Longrightarrow_{\tensor^{\kappa}_{n}}$ rule. Here,
the $n$ is a natural number denoting the required ``lateness'' of both input and
output, where $n \ge 1$. Specifically, ${\laterkappa_{\!\!1} B \cong \laterkappa B}, {\laterkappa_{\!\!2}
\cong \laterkappa\laterkappa B}$ and so on. For trivial cases, the
$\Longrightarrow_{Refl}$ rule is used. \todo{Say something about abstraction and
let-rules here, when they have been formulated.}

\begin{figure}
\textbf{The transformation to guarded recursive terms}$\hfill \boxed{IE\vdash e : A \overset{B}{\Longrightarrow} IE^\prime \vdash e^\prime : B}$
\input{figures/epsilon_next_rule.tex}
\input{figures/epsilon_lambdakappa_rule.tex}
\input{figures/epsilon_apply_rule.tex}
\input{figures/epsilon_renaming_rule.tex}
\input{figures/epsilon_app_rule.tex}
\input{figures/epsilon_tensor_rule.tex}
\input{figures/epsilon_refl_rule.tex}
\input{figures/epsilon_lambda_rule.tex}
  \caption{Rules for inferring guarded recursive terms.}
  \label{fig:epsilon_rules}
\end{figure}

\paragraph{An Example}

Figure~\ref{fig:epsilon_zeros} shows how a guarded recursive version of
\texttt{zeros} can be inferred using the rules presented in Figure~\ref{fig:epsilon_rules}. The
derivation proceeds by continually pushing type requirements down into the
syntax tree, until these can be resolved either trivially (using the
$\Longrightarrow_{Refl}$ rule), or by renaming (using the
$\Longrightarrow_{\phi}$ rule). Note that all premises ensuring type consistency
have been omitted for brevity. A full example, along with a more advanced one,
can be found in Appendix~\ref{app:infer-guard-recurs}.\todo{Make this appendix}
\begin{figure}
\begin{lstlisting}[mathescape]
causal zeros : Stream Nat
zeros = Z :: zeros

$_g$zeros : $\forall\kappa.$ $_g$Stream Nat
$_g$zeros = $\Lambda\kappa.$ Z $_g$:: (Next (apply $_g$zeros))
\end{lstlisting}
  % \[
  % \frac{x}{\eps{IE}{Z :: zeros}{Stream Nat}{IE}{Z :: gzeros}{gStream Nat}}
  % \]
\input{figures/epsilon_zeros}
  \caption{Above: The user-provided program \texttt{zeros}, along with the
    inferred program $_g$\texttt{zeros}. Below: Inferring $_g$\texttt{zeros} from \texttt{zeros} using the rules from
  Figure~\ref{fig:epsilon_rules}.}
  \label{fig:epsilon_zeros}
\end{figure}

\subsubsection{Implementing the Inference System}

\paragraph{Preprocessing} Two aspects of the inference system, renaming and
delaying the recursive reference, are mandatory, and these
are therefore handled in a separate preprocessing step. Both must take modality
into account.

Renaming, i.e. all applications of the $\Longrightarrow_{\phi}$ rule, is
performed by substituting all Idris names in $\phi$ with their guarded
versions. Special care must be taken when substituting names of projection
functions, since the modality of the function in question, $f$, influences the choice
of guarded name. If $f$ is causal,

Delaying the recursive reference
%Preprocessing
%Modality and Recursive reference
%When rules clash

\subsection{Checking}
To check that what we infer actually conforms to the guarded recursion rules, we
have implemented a checker. This algorithm simply checks a term to have a given
type within the rule from Figure~\ref{fig:gr_rules_sin_clock}. To the algorithm
less error prone it is implemented in the simplest fashion possible and as
attempts to stick strictly to the typing rules. This means that the checking
algorithm is very simple, conservative, and rigid. It expect types and terms to
be of a specific structure, and it discards anything it can guarantee the
correctness off.

The checker traverses a term, and unfolds them according to the rules. Any rules
that requires a type equality is done by asking the Idris type system for
conversion equality, and if the type of a term is needed it is checked using the
Idris type checker.

The structure of the code itself aims to take care of as much as the check using
pattern matching. While not always possible, ideally the conclusion of a rule
should be represented by a pattern match, and the premise(s) by a right-hand
side. By doing this we can, through different patterns, be very selective about
what is accepted by the checker and discard anything that does not match the
conclusion of a rule.

Getting more into the actual implementation the checking algorithm is a function
\texttt{check}, with the type signature seen in Figure~\ref{fig:check_type}. The
parameters are as follows:\todo{Is this too low level?}

\begin{itemize}
\item \texttt{Clock} is the clock under which we are checking the term. A clock
  is simply either open or closed.
\item \texttt{Env} is the local environment, usually generated from the
  left-hand side of the clause being checked.
\item \texttt{(Name, Type)} is name and the type of the recursive
  reference.
\item \texttt{Term} is the term being checked.
\item \texttt{Type} is the type the term is being checked to have.
\item \texttt{Idris Totality} is the result of the algorithm wrapped in the
  Idris monad\todo{Is the Idris monad explained at this point?}.
\end{itemize}

\begin{figure}[h]
\begin{lstlisting}
check :: Clock ->
         Env ->
         (Name, Type) ->
         Term ->
         Type ->
         Idris Totality
\end{lstlisting}
  \caption{Checking algorithm type.}
  \label{fig:check_type}
\end{figure}

The majority of the conclusions of the rules can be reflected as patterns. If we
wish to e.g. delay a term, the $Next_i$-rule requires that, (1) the clock is
open, (2) the term is on the form $Next t$, and (3) the type is on the form
$\later A$. These requirements are expressed as a pattern in
Figure~\ref{fig:next_pattern}. The view-patterns can be read as regular pattern
match, such that \texttt{next -> Extracted t} is \texttt{Next t}. The premise
from the rule is reflected on the right-hand side. simply check that given the
same environments \texttt{t} has type \texttt{a}.

\begin{figure}[h]
\begin{lstlisting}
check Open g n (next -> Extracted t) (later -> Extracted a)
     =
     check Open g n t a
\end{lstlisting}
  \caption{$Next_{i}$ pattern. Note the use of view-patterns (the -> syntax).}
  \label{fig:next_pattern}
\end{figure}

\paragraph{No Free Clocks in Environment}
% No free clocks in finished definitions.
% Check everything in local environment with closed clock.
Some rules have a side condition dictating that there must no free clocks in the
variable environment. These are checked by checking that every term in the
variable can be checked under a closed clock. This is the same as no clocks
being free, because any term or type with a free clock would require the clock
to be open in order to be well-formed.
\paragraph{Identifying the Recursive Reference}
\todo{This paragraph needs more implementation to be written}
%##########
% To faser: Inferens og check - bør kunne beskrives separat
% Inferens - regler
% Check - regler
% Correctness claim - Hvordan kan vi være sikre på at vi inferer guarded
% recursion?
% Preservation of semantics - Har den infererede term samme semantik som
% input-termen?
% Tensor-regel og parametre
%##########

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../copatterns-thesis"
%%% End:


\chapter[Bremstrahlung]{Bremstrahlung}

Bremstrahlung dominates other muon interaction processes in the region of
catastrophic collisions ($v \geq 0.1$ ); at "moderate" muon
energies - above knock onelectron production kinematic limit.

\section{Differential cross section }

The differential cross section for muon bremstrahlung (in units $cm^{2}/(g~GeV)$)
can be written as :

\begin{eqnarray}
    \frac{d \sigma (E, \epsilon, Z, A )} {d \epsilon} &=&
  \frac{16}{3} \alpha N_{A} (\frac{m}{\mu} r_{e} )^2 \frac{1}{ \epsilon A} Z
  (Z \Phi_{n}+ \Phi_{e})(1-v+\frac{3}{4}v^{2})   \\
    \frac{d \sigma (E, \epsilon, Z, A )} {d \epsilon} &=& 0 \quad
    if \quad \epsilon \geq \epsilon_{max} = E- \mu  \nonumber
\end{eqnarray}
%
where $\mu$ and $m$ are the muon and  electron masses,
$Z$ and $A$ are the atomic number and atomic weight of the material
and $N_{A}$ is Avogadro's number.
 If we indicate with $E$ and $T$ the initial total and kinetic energy of muon,
$\epsilon$--emitted photon energy we have: $\epsilon = E - E'$ and
relative energy transfer $v = \epsilon /E$.

Functions $\Phi_{n}$ and $\Phi_{e}$ represent contribution
of nuclear and electrons and can be expressed as :

\begin{eqnarray*}
 \Phi_{n} &=& \ln \frac {BZ^{-1/3}(\mu + \delta (D_{n}' \sqrt{e} -2))}
    {D_{n}'(m+ \delta \sqrt{e}BZ^{-1/3})} ; \\
   &=&  0 \quad if \quad negative;
\end{eqnarray*}

\begin{eqnarray*}
  \Phi_{e} &=& \ln \frac {B'Z^{-2/3} \mu }
   {(1+ \frac{\delta \mu}{m^{2} \sqrt{e}})(m+ \delta \sqrt{e} B'Z^{-2/3}) }  \\
   &=&  0 \quad if \quad \epsilon \geq \epsilon'_{max} = E/(1+ \mu^{2}/2mE) \\
   &=&  0 \quad if \quad negative
\end{eqnarray*}
%
\begin{eqnarray*}
\delta &=& \mu^{2} \epsilon /2EE' = \mu^{2} v/2(E- \epsilon) \\
D'_{n} &=& D_{n}^{(1-1/Z)}, \quad D_{n}= 1.54A^{0.27}  \\
B &=& 183, \quad B'=1429, \quad \sqrt{e}=1.648(721271).
\end{eqnarray*}
%
Special case for hydrogen ($Z$=1): $B = 202.4,\: B' = 446, \: D_{n}' = D_{n}$.

Formulae are mostly taken from (\cite{brem.kel95}) and (\cite{brem.kel97}).
They include improved (in the region $v \sim 1$ and low $Z$,
 in comparison with
(\cite{brem.petr68}) correction for nuclear size. Bremstrahlung on atomic
electrons (taking into account target recoil and atomic binding) is
introduced instead of a rough substitution $Z(Z+1) $. Also  (\cite{brem.andr94})
correction for processes with nuclear excitation is included here.

\subsection{Aplicability and restrictions of the method}

1. $E \gg \mu $, ultrarelativistic approximation used in this method; \\
2. $E \leq 10^{20}$ eV, above LPM cross section suppression can be expected;\\
3. $v \geq 10^{-6}$ , below , Ter-Mikaelyan suppression can be expected.
However, in this region cross section of muon bremstrahlung is many
orders of magnitude less then for other processes.\\
4. Coulomb corrections is not included. However Bugaev's calculations
show that it is small for muon bremstrahlung.

\section{Continuous energy loss}

The restricted energy loss for muon bremsrahlung $(d E/ dx)_{rest}$
 with relative transfers $v = \epsilon / (T+ \mu) \leq v_{cut}$
can be calculated as follows :
$$
\frac{d E}{d x}
= \int_{0}^{\epsilon_{cut}}  \epsilon \sigma (E,\epsilon )  \: d \epsilon  =
(T+ \mu ) \int_{0}^{v_{cut}}  v \sigma (E, v )  \: d v
$$
%
If user's cut $v_{cut} \geq v_{max}=T/(T+ \mu)$, total average
energy loss is calculated. Integration is done on the basis of Gauss quadratures,
binnig provides  the accuracy better
than about 0.03\% for $T = 1$ GeV, $Z=1$ (rapidly improving with increasing $T$
and $Z$).

\section{Total cross-section}

The integration of differential cross section  (1) over
$d\epsilon$ gives the total cross-section for muon bremstrahlung:

\begin{equation}
\sigma_{tot} (E, \epsilon_{cut})
= \int_{\epsilon_{cut}}^{\epsilon_{max}}  \sigma (E,\epsilon )  \: d \epsilon  =
 \int_{\ln v_{cut}}^{\ln v_{max}}  v \sigma (E, v )  \: d (\ln v)\:,
\end{equation}
where $v_{max}=T/(T+ \mu)$. If $v_{cut} \geq v_{max}$ , $\sigma_{tot}$ = 0.

\section{Sampling}

Random photon energy $\epsilon_{p}$ is found from the numerical solution
of the equation :
$$ P \:= \int_{\epsilon_{p}}^{\epsilon_{max}}  \sigma (E,\epsilon,Z,A )  \: d \epsilon \:
 / \: \int_{\epsilon_{cut.}}^{\epsilon_{max}}  \sigma (E,\epsilon,Z,A )  \: d \epsilon .
$$
%
Here $P$ is random uniform probability, $\epsilon_{max}=T$,
$\epsilon_{cut}=(T+\mu) \cdot v_{min.cut}$;
$v_{min.cut}=10^{-5}$ being a relative transfer cut adopted
in the algorithm.

For fast sampling, solution of (2) is tabulated at initialization
for a set of $Z,T,P$. In simulation, sampling subroutine returns random
$\epsilon_{p}$ corresponding to probability $P$,
interpolation in the table being used.

Tabulation subroutine uses function (1) for differential
cross section. The table contains the values :
\begin{equation}
 x_{p} = \ln (v_{p} / v_{max})/\ln (v_{max}/v_{min.cut}),
\end{equation}
where $v_{p}= \epsilon_{p}/(T+ \mu),\: v_{max} = T/(T+ \mu).$
The table parameters are : $Z_{min}=1, \: Z_{max}=128; \:
T_{min}=1$ GeV, $T_{max}=10^{6}$ GeV ;
$P_{min}=10^{-5}, \: P_{max}=1.$

Atomic weight (which is the necessary parameters in (1)) is estimated
here with iterative solution of the known relation (approximate) :
$$ A = Z (2+0.015 A^{2/3}),$$
for $Z=1$ it is used $A=1$ .
%
To  find $x_{p}$ (and, respectively, $\epsilon_{p}$) corresponding to
a given probability $P$, in sampling subroutine the linear
interpolation in $\ln Z, \: \ln T$, and cubic (4 points Lagrange) interpolation
in $\ln P$ is realised. For $P \leq P_{min}$ , linear interpolation
in $(P,x)$ - coordinates is used, noticeably, $x = 0$ at $P = 0$.
Then, random energy $\epsilon_{p}$ is obtained from the inverse
transformation of (3) :
%
$$\epsilon_{p} = (T+ \mu ) v_{max} (v_{max}/v_{min})^{x_{p}} $$
%
The algorithm with the parameters described above has been
tested for various $Z$ and $T$. It provides the reproduction
of the differential cross section with about 0.2 - 0.7 \%
accuracy for $T \geq 10$ GeV, the average total energy loss
being kept within 0.3\%. Accuracy improves with the increase
of $T$, satisfactory results are obtained also for
$1 \leq T \leq 10$ GeV.

It is important to note that this sampling scheme allows
to generate $\epsilon_{p}$ for different user's cuts on $v$
grater then $v_{min.cut}$.
To realize such simulation, it is sufficient to
re-define probability variable :
%
$$P' = P \: \sigma_{tot} \: (v_{user.cut}) / \sigma_{tot} (v_{min.cut})$$
%
and just use $P'$ in sampling subroutine.
Thus, time consuming re-calculation of 3-dimensional table
is not required (only calculation
of $\sigma_{tot}(v_{user.cut})$ is needed).

After the successful sampling of $\epsilon$,  the direction
 of the emitted photon is generated with respect to the direction of the
incident particle. The azimuthal angle $\phi$ is generated isotropically;
the polar angle
$\theta$ is calculated from the energy momentum conservation.
This information
is used to calculate the energy and momentum of both scattered
particles and to transform them into the {\em global} coordinate system.

\section{Status of this document}
 9.10.98 created by R.Kokoulin and A.Rybin .

\begin{thebibliography}{99}

\bibitem[Keln95]{brem.kel95}
  S.R.Kelner, R.P.Kokoulin, A.A.Petrukhin. Preprint MEPhI 024-95, Moscow, 1995.
\bibitem[Keln97]{brem.kel97}
  S.R.Kelner, R.P.Kokoulin, A.A.Petrukhin. Phys. Atomic Nuclei, 60(1997)576.
\bibitem[Petr68]{brem.petr68}
  A.A.Petrukhin, V.V.Shestakov. Canad.J.Phys., 46(1968)S377.
\bibitem[Andr94]{brem.andr94}
  Yu.M.Andreyev, L.B.Bezrukov, E.V.Bugaev. Phys. Atomic Nuclei, 57(1994)2066.
\end{thebibliography}