\section{RemotePeer Meter Metrics}
\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
lastSeen = 0
for connection of allConnections
  lastSeen = max(lastSeen, connection.lastSeen)
\end{lstlisting}
\caption{Youngest LastSeen of all Connections}
\label{lst:mit-last-seen}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
lastSeen = getYoungestLastSeen()
isCrashed = currentTick - lastSeen > GLOBAL_CRASH_THRESHOLD
\end{lstlisting}
\caption{RemotePeer has crashed}
\label{lst:mit-crashed}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
qualitySum = 0
for connection of allConnections
  qualitySum += connection.quality
averageQuality = qualitySum / allConnections.size
\end{lstlisting}
\caption{Average connection quality}
\label{lst:mit-average-connection-quality}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
bestQuality = 0
for connection of allConnections
  bestQuality = max(bestQuality, connection.quality)
\end{lstlisting}
\caption{Get best connection quality}
\label{lst:mit-best-connection-quality}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
hasProtectedConnection = false
for connection of allConnections
  if connection is protected
    hasProtectedConnection = true

hasNotSatisfiedConnectionGoal =
  reportedDirectConnections < GLOBAL_CONNECTION_GOAL

isProtected =
  hasNotSatisfiedConnectionGoal && hasProtectedConnection
\end{lstlisting}
\caption{Is a protected RemotePeer}
\label{lst:mit-welpenschutz}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
allViaCons = peer.getAllViaConnectionsTo(remotePeer)
allDirectCons = peer.getAllDirectConnections(remotePeer)

totalCons = allViaCons  allDirectCons
saturation =
  (GLOBAL_DIRECT_CON_MAXIMUM - totalCons) / GLOBAL_CONNECTION_GOAL

saturation = limit(0, saturation, 1)
\end{lstlisting}
\label{lst:mit-connection-saturation}
\caption{Connection saturation of a RemotePeer}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
punishedConnectionAvg =
  punishedConnections / totalDirectConnections
\end{lstlisting}
\label{lst:mit-punished-conenctions}
\caption{Punished connections average}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
reportQuality = bestConnectionQuality * connectionSaturation
\end{lstlisting}
\label{lst:mit-remote-peer-quality-report}
\caption{Quality of a RemotePeer for PeerUpdate}
\end{Listing}

\begin{Listing}[H]
\begin{lstlisting}[xleftmargin=3em]
quality = avgConnectionQuality * (1 - avgConnectionPunishment)
\end{lstlisting}
\label{lst:mit-remote-peer-quality}
\caption{Quality of a RemotePeer}
\end{Listing}


%!TEX root = thesis-kdyoung.tex

\chapter{Theoretical Background} \label{chap:theory}
In this chapter we provide the theoretical background needed
for the work presented in the following chapters.
As such, we give summaries of the relevant literature from
four fields spanning mathematics and computer science.
These include background knowledge of the problem of assembly
line balancing (Sect. \ref{sec:lit:ALB}); modelling and solving technology from the areas
of linear programming (Sect. \ref{sec:lit:mip}) and constraint programming (Sect. \ref{sec:lit:cp});
and lastly we delve into some of the recent
advances into the theory of Benders decomposition (Sect. \ref{sec:lit:bend}).

\section{Assembly Line Balancing}
\label{sec:lit:ALB}
Optimizing the construction of an assembly line to maximize or minimize
some performance measure has been a problem of great interest to the
research community since its introduction by Henry Ford
in 1915 \cite{Ford1922}.
Simply, it can be stated as a decision problem
where we seek optimal partition of work to assemble
a product.
The work must be divided
among discrete stations along a line (\eg conveyor belt),
while respecting a series of restrictions imposed by physical
limitations.
All variants of the \albp{} begin from this common starting point
and add further restrictions or considerations which
results in a range of generalizations.
We refer the interested reader to one of the surveys
of \authciteb{Baybars1986},
\authciteb{Becker2006} or
\authciteb{Boysen2007} for a detailed snapshot of
the problem's various extensions.

In this section we detail the basic version of the
problem and the particular variant which
is relevant to our work.

\subsection{Simple Assembly Line Balancing Problem}
\label{sec:lit:SALBP}
The Simple Assembly Line Balancing Problem is the basic
type of the \albp{} \cite{Baybars1986,Scholl1999}.
It consists of a serial production line which produces a
single product.
The tasks which need to be executed to achieve a product's completion
all have fixed processing times and all stations
along the line are uniform, \ie each is equivalently equipped
and manned by homogeneous workers or machines.

There are two primary parameters of the assembly line
which dictate the aim when solving the problem; these parameters
are the \emph{cycle time} and the number of stations, denoted
by $c$ and $m$ respectively.
We denote the \emph{load} of station $k$ by $l_k$, which is
defined as the sum of all processing times of tasks assigned to station
$k$.
The cycle time of an assembly line is defined as follows,
\[
	c=\max\{ \: l_k \:|\: k \in K \:\},
\]
\ie the largest load among all stations.
Physically, the cycle time of an assembly
line can be interpreted as how much time each station is given to work on a product,
before the product moves to the next station.
A lower cycle time means the conveyor belt can move faster and
products are output at a higher rate.

From the parameters $c$ and $m$, the \albp{} can be formulated as three
different but highly related problems: type-1, type-2 and
type-E.
We now summarize each variety and its particular qualities in turn,
\begin{itemize}
	\item Type-1: Fixed $c$, variable $m$.\\[1pt]
	The type-1 variety, abbreviated as \sab{1} in the simple case, considers the
	cycle time to be fixed and the number of stations to be decided.
	This most commonly occurs when constructing a new assembly line.
	As an example, the \sab{1} is characterized using the classification
	scheme of  \authciteb{Boysen2007} by $[-|-|m]$.
	\item Type-2: Variable $c$, fixed $m$.\\[1pt]
	For the type-2 variety, the aim is to find the optimal cycle time given
	that the number of stations along the line is fixed.
	In practice, this variety commonly corresponds to adapting an existing assembly line
	for a new purpose.
	\item Type-E: Variable $c$, variable $m$.\\[1pt]
	This variety deals with the case when both $c$ and $m$ are considered
	to be decision variables of the problem. Here, the objective
	is to optimize some measurement of line efficiency.
\end{itemize}

\subsection{Setup Assembly Line Balancing and Scheduling Problem}
\label{sec:lit:SUALBSP}
In this document, we are concerned with the SetUp Assembly Line Balancing and Scheduling Problem.
This variant differs from the basic \sab{} by introducing
sequence-dependent setup times between pairs of tasks
within a station's workload.
This type of consideration was first made in the literature by \authciteb{Andres2008}
in an attempt to reduce the gap between the existing academic theory
and the realities of assembly line problems posed by the industry.
In the problem tackled by \citeauthor{Andres2008},
they considered a setup time to exist between
a pair of consecutive tasks inside a station's sequence.
As such, it needed to be added to that station's
global processing time
and so these setup times directly affected
the cycle time of the line.
The authors considered heuristic rules and employed a Greedy
Randomized Adaptive Search Procedure (\grasp) to tackle the
type-1 version of the problem.
They also developed a lower-bound which was used
to estimate the quality of their solutions found.

Later, some of the same authors continued this work in
\authciteb{Martino2010}, where over 50 priority-rules were considered
for a heuristic procedure designed for the type-1 version.
All results found for the problem at this point in the
literature indicated that adding these sequencing considerations
for the tasks made the problem much more difficult to solve.
This is demonstrated by only very small instances of the problem
being able to be optimally solved.

More recently, the work of \authciteb{Scholl2013} extended the previous
work on this problem and formally named it the \sua{}.
The \sua{} differs slightly from the problem tackled previously
by introducing a distinguishment between \emph{forward} and \emph{backward}
setup times.
The motivation for this differentiation is so the theoretical
problem can more accurately capture the physical aspects
of sequencing tasks.
A forward setup time occurs between two consecutive tasks assigned the same station,
if each task is processing the same item.
A backward setup time occurs between two consecutive tasks assigned the same station,
if each task is operating on different items on the assembly line.
Physically, a backward setup time captures the setup cost between the last task on the current
product and the first task on the next product.
We demonstrate this idea using
a possible task sequence within two stations in Figure \ref{fig:lit:forwBackSetupExAbstractB}.
Forward setup times are denoted by $\phi$ and backward setup times
by $\beta$.

% \begin{figure}[H]
% 	\centering
% 	\caption{Setup times of a station's sequence (test A)}
% 	\vspace{2mm}
% 	\includegraphics[width=0.9\textwidth]{images/forwBackSetupExA.eps}
% 	\label{fig:lit:forwBackSetupExAbstractA}
% \end{figure}

\begin{figure}[tpb]
	\centering
	\caption{Setup times of a station's sequence}
	\vspace{2mm}
	\includegraphics[width=0.9\textwidth]{images/forwBackSetupExB.eps}
	\label{fig:lit:forwBackSetupExAbstractB}
\end{figure}

% \begin{figure}[H]
% 	\centering
% 	\caption{Setup times of a station's sequence (test C)}
% 	\vspace{2mm}
% 	\includegraphics[width=0.9\textwidth]{images/forwBackSetupExC.eps}
% 	\label{fig:lit:forwBackSetupExAbstractC}
% \end{figure}

\authciteb{Scholl2013} proposed an exact solution approach by a
mathematical formulation of the type-1 problem as a mixed integer program.
Using the scheme of \authciteb{Boysen2007},
the \sua{1} can be formally classified as,
\[	\text{\sua{1}: }[\Delta t_{dir}|-|m].	\]
Further to their mathematical model, a range of tools were proposed which could be composed
in a number of ways to create new heuristic approaches for the problem.

\authciteb{Esmaeilbeigi2016} give the most recent contribution of the literature
to the study of the \sua{}.
In their paper, three new mixed-integer formulations are presented of the \sua{1}
which are based on different ways of representing the decisions.
Their first two formulations are station-based and the last is a purely
scheduling-based formulation which has abstracted the
assignment decisions away from the problem.
Each of the three models presented are also modified to be suited
to the type-2 version of the \sua{}.
Although these models were presented, the type-2 problem
was not the authors primary concern and as such the models were not tested.

In the literature there have been few attempts to tackle the type-2 variety of the \sua{}.
The first was by \authciteb{Yolmeh2012} who use a Genetic Algorithm
which is applicable to either the type-1 case or type-2 case.
The only other approaches to the type-2 problem
which have been explored are the three mixed integer programs
mentioned earlier from \authciteb{Esmaeilbeigi2016}.

\section{Mixed Integer Programming}
\label{sec:lit:mip}
A Mixed Integer Program is a common extension of the fundamental
Linear Program (LP) of the Operations Research discipline.
MIPs present the additional constraint on some of their decision
variables that restrict their domain to exclusively integer values.
Problems which require integer valued solutions occur in many
real-world scenarios.
Example \ref{ex:lit:simpleMIP} presents a simple but quite
useful application of a problem with integer variables.

\begin{example}\label{ex:lit:simpleMIP}
	Consider assigning staff members to complete
	a given set of tasks.
	For a solution of this problem to be feasible we require that
	no fraction of any staff member is split across multiple tasks.
	Using continuous variables to model
	this problem will lead to invalid solutions and thus integer-valued
	variables are required. \qed
\end{example}

The flexibility of the mixed integer programming approach to modelling
problems has allowed MIPs to model a wide-range of optimization
problems.
A typical solution methodology when approaching a MIP is to
consider its linear relaxation.
This presents a much simpler problem to solve, but
recovering a feasible integer solution from this relaxed problem
can prove challenging.

\subsection{Branch and Bound}
\label{sec:lit:mipBB}
The Branch and Bound (\bab) algorithm is a general tool which can
be applied to solve optimization problems which was first proposed by \authciteb{land1960}.
The structure of the method is general enough that it can be used
as a design paradigm for more involved ways of exploring the
solution space. As a result, it has spawned
a variety of adaptations in the field of operations research since its
inception. Some of these include branch-and-cut (\authcite{Padberg1991}),
branch-and-price (\authcite{Savelsbergh1997})
and branch-and-infer (\authcite{Bockmayr1998}).
A procedure based on the \bab algorithm is the typical way of
approaching MIPs and so we now give an overview of
how this method operates.

The \bab method constructs a rooted tree used to explore the set of
all feasible solutions.
At a node of the tree, a decision variable is chosen as
the branching variable and the domain
of possible values for that variable is divided
between two child nodes.
The solution space of the current node is split
into two smaller disjoint subsets in these child nodes.
At the root of the tree, the solution space corresponds to all
possible feasible solutions to the original LP relaxation of the problem.
For each new node created by branching, the corresponding
relaxed LP is solved to determine the optimal
value at each node.
When each node is processed, a procedure called \emph{fathoming},
its optimal value is checked for both
integrality and feasibility.
If the node's solution is optimal and integer then it is compared to
the current best integer solution.
In a minimization (maximization) problem, if the new solution
is less (more) than the current best, then
the current best is replaced with the new optimal integer solution.
This new solution is called the incumbent.
Branching on the feasible solution space and solving each
relaxed sub-problems continues until all branches have been either
explored or are deemed unnecessary to explore, \ie \emph{pruned}.
At termination, the current incumbent solution is the optimal solution
to the original non-relaxed MIP, and thus the original problem
is solved.
We provide some pseudo-code of the \bab algorithm
for a minimization MIP in Algorithm \ref{alg:lit:bab}
in the Appendices.

\subsection{MIP Solver: \gurobi}
\label{sec:lit:mipGurobi}
\gurobi is a commercial solver designed for mathematical optimization problems including
LPs, quadratic programs (QPs) and MIPs.
Significant advances have been made to Gurobi's capabilities and other MIP solvers
during the last few decades due to a number of improvements
in the theory and computational hardware available \cite{Bixby2002}.
Some of the most important improvements in this area
are detailed in \authciteb{Lodi2010}.
These factors have given MIP solvers the ability to
solve highly complex optimization problems while providing
a relatively easy to use interface for doing so.

To solve a MIP, Gurobi uses the branch-and-cut version
of the \bab algorithm.
Preprocessing is a common procedure implemented by solvers
where the problem's input to the solver is altered
to remove obviously sub-optimal solutions from
the solution space without removing any possibly optimal solution.
Further to this pre-solving step, there are numerous
improvements made to the procedure of the \bab also.
Additional constraints called \emph{cutting planes}, or
simply \emph{cuts}, are added to the model
which do not remove any feasible solution but result in the linear relaxation
being closer to the non-relaxed problem's feasible solution space.
The feasible solution space of the original problem is
called the \emph{convex hull}.

Advances in the hardware over the past few decades
has been the other main source of improvement to solvers such as Gurobi.
Utilizing an algorithm which depends on branching the solution space
into disjoint subsets, has naturally favoured solvers such as Gurobi
by allowing them to take full advantage of parallel computing.
Currently, \gurobi can concurrently solve 1024 branches of
a \bab tree.
Taking advantage of this parallelization can provide
more than just a 1024-fold increase is computational power
due to the communication of solutions between branches.
For example, if one branch finds a high-quality integer feasible solution
then all other branches can immediately reap the benefits
by getting access to more effective pruning.

\section{Constraint Programming}
\label{sec:lit:cp}
Constraint Programming (CP) is a framework for modelling and solving optimization problems.
This framework has its roots in the field of Artificial Intelligence and the initial
conception of the idea is due to \authciteb{Jaffar1987},
who pointed out that the language PROLOG II is a special
case of a more general scheme: Constraint Logic Programming.
CP uses a declarative descriptive of the problem by defining a set of decision variables
each with a set of possible values, \ie domains;
and a set of constraints which restrict the possible combinations of values for the
decision variables.

Over the past two decades, CP has been used extensively
to tackle a variety of combinatorial optimization problems.
The success of constraint programming can generally be attributed
to two of its unique features
\begin{enumerate}
	\item \emph{heterogeneous global constraints} which allow the user to write a high level
	model using the global constraints to capture important combinatorial structures.
	In most cases these constraints are formulated in such a way to be abstracted
	away from the specific properties of any one problem.
	\item \emph{user-defined search} which allows the user to specify a structured
	search procedure for exploring the solution space, using their familiarity
	with the problem.
\end{enumerate}

CP has proven itself to be an effective solving technology
for many scheduling problems such as project \cite{Berthold2010},
train \cite{Rodriguez2007} and
employee scheduling \cite{Demassey2005}
as well as other types of combinatorial problems such as bin packing \cite{Pisinger2007}.

We now provide short explanations of the features of CP mentioned above
and a description of the CP solver we chose to use, \chuffed.
To communicate with the solver, we chose to write our constraint
programs using the solver-independent CP language
MiniZinc \cite{Nethercote2007}.
MiniZinc provides a vast library of global constraints
as well as an adequate search language.

\subsection{Global Constraints}
\label{sec:lit:cpGlob}
Global constraints are one of the powerful tools made available to us
when formulating constraint programs. They allow us to succinctly encode
complicated combinatorial structures that commonly occur among
discrete problems. One of the simpler,
but highly applicable, global constraints is given in
Example \ref{ex:globCons}.

\begin{example}\label{ex:globCons}
	\aldif is a global constraint that takes an array of integer
	variables, each with a possibly different domain of possible values. This constraint
	simply requires all given variables to take different values in the solution. \qed
\end{example}

One drawback of global constraints is that each one must be individually
implemented in the solver. When posed by a problem with inherent combinatorial
structures, your choice of solver can be restricted as some solvers may
only partially support, or not support at all, the needed global constraints.
However the benefits of this type of constraint are not to be understated.
The fact that global constraints are formulated to be problem-agnostic can
provide users with much easier access to advances in the state-of-the-art solving technology.
To illustrate this point we consider the global constraint \cumu.

The \cumu global constraint was first introduced by \authciteb{Aggoun1993}
to efficiently solve and succinctly model a complex scheduling
problem.
Since then, the \cumu constraint has been widely used in a variety of problems
due to the fact that resources which are cumulatively restricted occur in many real-world problems.
A cumulatively restricted resource could represent a limited work force needed to execute a set
of tasks or a machine with a restriction on the number
of tasks it can run in parallel \cite{Schutt2011a}.

The \cumu constraint has commonly been applied to the Resource Constrained
Project Scheduling Problem (\RCPSP), which has many variants that
are well-disposed to utilize the benefits of this global constraint.
Since its introduction, the \cumu constraint has had a number of improvements
proposed, some of these were by \authciteb{Caseau1994}, \authciteb{Nuijten1994},
\authciteb{Baptiste2000} and \authciteb{Carlier2004}. Each time an improvement
is made to \cumu~--- either by strengthening its propagation or by allowing it
to capture more varied and complex structures --- the results of all
problems where \cumu has previously been applied are improved as well.

% \begin{figure}[tbp]
% 	\centering
% 	\caption{Example of \cumu applied to a small cumulative resource problem {\color{red} (not complete)}}
% 	\includegraphics[width=0.95\textwidth]{images/cumu.png}
% 	\label{fig:examplePrecGraph}
% \end{figure}

By encoding the combinatorial nature of a problem using the global
constraints made available to us by CP, we are able to not
only take advantage of all the previous theoretical improvements to a global
constraint, but all future advances as well.

\subsection{User-Defined Search Procedures}
\label{sec:lit:cpSearch}
MiniZinc is a powerful modelling language for constraint programming
which is capable of concisely specifying complex models.
One of the reasons it was the language we chose to write
our constraint programs with was its ability to define relatively
structured search procedures with ease \cite{Schrijvers2013a,Schrijvers2013b,VanHentenryck2000}.

The current search facilities provided by MiniZinc can be
separated into two types: basic searches and compositional searches.
To demonstrate the features of a basic search we present
a simplification of MiniZinc code here in
Listing \ref{lst:intsearch}. This basic search specifies a
search procedure over an integer variable, namely \intser,
\begin{lstlisting}[caption={Basic integer search procedure},label={lst:intsearch},language=minizinc]
int_search(array[int] of var int, ann, ann);
\end{lstlisting}\vspace{5mm}

The declaration of the basic integer search, requires 3 inputs to fully
define how it divides the set of feasible solutions
and chooses which variables to branch on. Let the declaration be defined by input
$(x,varselect,varsplit)$. The decisions variables which need to have
their values fixed are given by $x$. The annotation $varselect$ specifies
the order in which to select the variables depending on some selection criteria.
Lastly, $varsplit$ defines how to branch on the chosen variable's
possible values. During this branching process, variables which have their
domains reduced to a singleton value are fixed and removed from
later consideration.

There are many variable selection strategies which MiniZinc's search language
offers. The simplest selection criterion is {\tt input\_order} which selects
the variables in the order given.
Some of the possible selection criteria which
are offered by MiniZinc are presented in Table \ref{tab:selectCrit}.

MiniZinc also offers a multitude of value splitting methods to allow
the user to specify application tailored branching strategies.
The branching of any search procedure is characterized by
creating one branch satisfying the specified splitting
property and another branch satisfying its negation.
For example, using the splitting method {\tt indomain\_min}
will create one branch in the search tree where the chosen variable's
domain is reduced to its minimum value --- \ie this variable is fixed
--- and another branch where the minimum value of the variable
is removed from the domain.
Some of the possible branching methods MiniZinc
has implemented are presented in Table \ref{tab:valueSplit}.
Example \ref{ex:lit:basicSearch} presents a simple
use of the \intser procedure on an array of integer
variables.

\begin{table}[tpb]
	\def\arraystretch{1.2}
	\centering
	\caption{Examples of MiniZinc's variable selection criteria}
	\vspace{2mm}
	\begin{tabular}{p{0.25\textwidth}p{0.65\textwidth}}
		\toprule
		Selection Criteria & Definition \\
		\midrule\midrule
		{\tt input\_order} & Choose variables in the order given \\
		{\tt smallest} & Choose the variables with the smallest possible value in their domain \\
		{\tt smallest\_largest} & Choose the variables with the smallest largest possible value in their domain \\
		{\tt first\_fail} & Choose the variables with the smallest domain \\
		\bottomrule
	\end{tabular}
	\label{tab:selectCrit}

	\caption{Examples of MiniZinc's variable splitting methods}
	\vspace{2mm}
	\begin{tabular}{p{0.25\textwidth}p{0.65\textwidth}}
		\toprule
		Variable Splitting & Definition \\
		\midrule\midrule
		{\tt indomain\_min} & Branch on the minimum value of the chosen variable \\
		{\tt indomain\_max} & Branch on the maximum value of the chosen variable \\
		{\tt indomain\_random} & Branch on a random value in the  chosen variable's domain \\
		{\tt indomain\_split} & Branch on the chosen variable such that the domain is split in half \\
		\bottomrule
	\end{tabular}
	\label{tab:valueSplit}
\end{table}

\begin{example}\label{ex:lit:basicSearch}
	For example, consider a search annotation
	$${\tt int\_search([x,y,z],~smallest,~indomain\_min)},$$
	with current domains of the variables $x\in\{5\},\:y\in\{7,\ldots,12\},\:z\in\{2,\ldots,9\}$.
	As we have the {\tt smallest} variable selection strategy, the variable $z$ will be chosen
	as the branching variable. Using {\tt indomain\_min} will result in one branch were $z$'s value
	is fixed to 2 and another branch where its domain is reduced to $\{3,\ldots,9\}$. \qed
\end{example}

Further to the search strategy for arrays of integer variables, MiniZinc also has
similar search structures available for arrays of boolean variables ({\tt bool\_search}) and
arrays of float variables ({\tt float\_search}).

The most important feature in MiniZinc's search language is its ability to compose
primitive searches to create complex application-tailored search procedures.
This is done through the use of search combinators developed in \cite{Schrijvers2013b}
and \cite{Schrijvers2013a}.
We briefly detail here how MiniZinc's sequential search
combinator, {\tt seq\_search}, can be used. MiniZinc's sequential search definition
is given in Listing \ref{lst:seqsearch} and we show how it
can be used in Example \ref{ex:lit:seqSearch}.

\begin{lstlisting}[caption={Sequential search procedure},label={lst:seqsearch},language=minizinc]
seq_search(array[int] of ann);
\end{lstlisting}\vspace{5mm}

\begin{example}\label{ex:lit:seqSearch}
	Consider the sequential search annotation in Listing \ref{lst:seqsearchEx},
	which first will decide all boolean variables $b1,\:\ldots,\:b3$, by branching on
	a random variable in each one's domain.
	After all boolean variables are fixed, the procedure will search over
	possible values for the $x$ variables choosing to branch on the one with the smallest value in its domain. \qed
\end{example}
\begin{lstlisting}[caption={Example of a sequential search annotation},label={lst:seqsearchEx},language=minizinc]
seq_search( [bool_search([b1,b2,b3], input_order, indomain_random),
             int_search( [x1,x2,x3], smallest,    indomain_min)    ] );
\end{lstlisting}\vspace{5mm}

Later in Section \ref{sec:bend:cpSearch}, we formulate specific
search strategies to effectively tackle our problem.
To achieve this, we will provide
a more in-depth look into some of the recent developments in
MiniZinc's search language and how we have employed them.

\subsection{CP Solver: \chuffed}
\label{sec:lit:cpChuffed}
To solve the constraint programs that arise in our solution
methodology, we chose to use the CP solver \chuffed which
was initially developed in the Ph.D thesis of \authciteb{Chu2011}.
In this section we will note the advantages that this solver
provides and also the solver's drawbacks.

What sets \chuffed apart from other competing CP solvers
are its two main features:
lazy clause generation (\LCG) and boolean satisfiability solving (\SAT).
Since the introduction of \LCG to solving technology, CP solvers
which have integrated it into their procedures
have become very well suited to scheduling problems \cite{Ohrimenko2009}.
Their ability to learn nogood clauses
has allowed this type of solver to effectively prune the search space,
using conflict-driven search to guide the branching strategy \cite{Szeredi2016,Schutt2015,Schutt2011b,Schutt2013b}.

An important feature of CP solvers is their Finite Domain (FD) propagation.
A propagator can be thought of as an `explanation' of a truth. Generally
this can be represented by $a_1\wedge a_2 \wedge \ldots\wedge a_n \rightarrow d$,
where each $a_i$ represents an existing domain restriction and $d$ represents
the implied domain restriction. This is demonstrated by Example \ref{ex:lit:domProp}.
CP solvers which have access to \SAT solving technology extend their ability
to use FD propagation on clauses of literals, rather
than being restricted to propagation of variable domains.
\begin{example}\label{ex:lit:domProp}
	Suppose our problem has three decision variables with domains
	$x\in\{3,10\}$, $y\in\{2,10\}$ and $z\in\{0,10\}$. Given the constraint
	$z\geq x+y$, we can use finite domain propagation to add the following
	propagator to our model $(x\geq3)\wedge(y\geq2) \rightarrow (z\geq5)$.\qed
\end{example}

Later in Section \ref{sec:bend:cpGlob}, we make use of
the global constraint \cumu to encode some of the combinatorial
nature of our problem.
Recent additions in the propagation qualities of \cumu, such as
time-table filtering \cite{Schutt2011a} and
time-table-edge-finding filtering \cite{Schutt2013c},
are all readily available in \chuffed.

The final advantage of \chuffed is that all
search capabilities offered by the language MiniZinc
have been fully implemented.
Perhaps most importantly, \chuffed
is the only available solver that supports a recent
addition to MiniZinc's search language,
which will be detailed further in Section \ref{sec:bend:cpPriSearch}.
This together with our familiarity with \chuffed and
it being a state-of-the-art solver for numerous scheduling
problems, which are closely related to the constraint programs we encountered,
made it the obvious choice of solver.

One significant disadvantage of \chuffed which must be mentioned,
is its inability to be run in parallel.
The most common way to
parallelize CP solvers is by splitting the search space into disjoint
subspaces and running independently on each branch.
Although \chuffed being a \LCG solver with nogood learning has presented some unique
difficulties in achieving an effective parallelization.
Communicating learnt clauses  between parallel solvers is advantageous to reduce
the amount of repeated work, but the ability to communicate complex clauses is quite limited.
Recent work by \authciteb{Ehlers2016} has shown that it is
possible to achieve an effective parallelization of \chuffed,
but unfortunately this technology is still in development and
we were not able to make use of it for our computational tests.

\section{Benders Decomposition}
\label{sec:lit:bend}
Benders decomposition is a method for solving optimization problems
which was original proposed in the seminal paper
by J. F. Benders in \citeyear{Benders1962}.
The method is loosely based on the notion of
``learning from one's mistakes'';
which is achieved through intelligently
partitioning the problem and delayed constraint generation.
This approach is well-suited to problems where the
decision variables can be divided into a set of primary
and secondary decisions.
When the secondary variables, also called \emph{complicating} variables,
are fixed the original problem becomes substantially
simpler to solve.
Fixing the complicating variables either partitions the problem
into two easier problems, namely the \emph{master}
and the \emph{sub-problem}, or results in such a structure
where the solution is straightforward.
This idea is demonstrated in the following linear program,
\begin{IEEEeqnarray}{RcRcRcRCl}
	\IEEEeqnarraymulticol{9}{c}{\text{Min }~ x_1+x_2+x_3 ~{\color{red} +~y}}  \label{eq:lit:compVars1}\nonumber\\[\eqnv]
	\text{subject to }\quad 2 x_1 &\hspace{1mm}& +~x_2 &\hspace{1mm}& &\hspace{1mm}& {\color{red} +~3y} &\geq& 8  \label{eq:lit:compVars2}\nonumber\\[\eqnv]
	 &\hspace{1mm}& x_2 &\hspace{1mm}& -~2x_3 &\hspace{1mm}& {\color{red} -~2 y} &\geq& 3  \label{eq:lit:compVars3}\nonumber\\[\eqnv]
	 &\hspace{1mm}& &\hspace{1mm}& x_3 &\hspace{1mm}& {\color{red} +~7 y}  &\geq& 2.  \label{eq:lit:compVars4}\nonumber
\end{IEEEeqnarray}
We have a set of complicating variables, $y$,
occurring in all constraints
and a set of primary variables, $x$, whose solution would be straightforward
if $y$ were fixed.

% \begin{figure}[tbp]
% 	\centering
% 	\caption{Simple example of complicating variables in a LP {\color{red} (not complete)}}
% 	\label{fig:lit:compVarsEx}
% 	\includegraphics[width=0.5\textwidth]{images/compVarsExample.png}
% \end{figure}

Once the problem is divided into the master and sub-problems, the
Benders algorithm iterates between them; solving each one in turn and uses
these solutions to gradually learn where the most
important infeasibilities of the problem exist.
A top-level view of how the Benders decomposition procedure
iterates is given in Figure \ref{fig:lit:topLevelBenders}.
In the case of minimization, the solution to the relaxed master
problem, or RMP, provides a lower bound on the optimal value.
This trial solution of the RMP is then passed to the sub-problem
and, if possible, the sub-problem is solved to optimality.
If the solution to the sub-problem is infeasible or sub-optimal,
we find out why and then use this information to design
a constraint to pass back to the master, which will rule out
the current trial solution.
This constraint is called a \emph{Benders cut}.
It is preferable
to formulate these cuts such that not only the current trial solution
is removed from the relaxed master's feasible region, but also
a large class of other solutions which are infeasible/sub-optimal for
similar reasons.

\begin{figure}[tbp]
	\centering
	\caption{High level structure of the Benders decomposition method}
	\vspace{2mm}
	\label{fig:lit:topLevelBenders}
	\includegraphics[width=0.6\textwidth]{images/bendersAbstract.eps}
\end{figure}

Since its introduction by \citeauthor{Benders1962},
this approach has been generalized to be applicable to a wider range of problem
structures. We now provide a review of the classical version of the method
as well as some of its recent adaptations.

\subsection{Classical Benders}
\label{sec:lit:clasBend}
The initial decomposition method proposed by \citeauthor{Benders1962}
was used to find the solutions of MIPs via partitioning
the problem into two simpler linear programs.
We refer the interested reader to the surveys by
\authciteb{Costa2005} and \authciteb{Rahmaniani2017}.
We now present how this decomposition could be applied
to a general minimization problem, based on the work
of \authciteb{Saharidis2010}.

Consider a problem where the set of decision variables
can be divided into two sets, $x$ and $y$, where
$y$ is a set of complicating variables.
The problem can be modelled by the following program
\begin{IEEEeqnarray}{rCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Min }~ c^T x+d^T y}  \label{eq:lit:bendSimple1}\\[\eqnv]
	\text{s.t. }\quad Ax+By &\leq& b  \label{eq:lit:bendSimple2}\\[\eqnv]
	Fy &\leq& p    \label{eq:lit:bendSimple3}
\end{IEEEeqnarray}
By applying Benders decomposition here we receive
two simpler problems: a master deciding the $y$ variables with
all constraints involving $y$,
and a sub-problem deciding the $x$ variables with all
constraints involving $x$.
So by considering a trial solution, $y=\bar{y}$, of the complicating variables,
we can pass this information to the sub-problem
which is formulated as follow,
\begin{IEEEeqnarray}{rCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Min }~ c^T x+d^T \bar{y}}  \label{eq:lit:bendSP1}\\[\eqnv]
	\text{s.t. }\quad Ax &\leq& b - B\bar{y}  \label{eq:lit:bendSP2}
\end{IEEEeqnarray}
Using the duality properties of LPs we can instead consider the
dual variables $u$ and represent this sub-problem by its dual as follows,
\begin{IEEEeqnarray}{rCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Max }~ (b-B\bar{y})u } \label{eq:lit:bendSPdual1}\\[\eqnv]
	\text{s.t. }\quad A^T u &\leq& c  \label{eq:lit:bendSPdual2}
\end{IEEEeqnarray}
This is the sub-problem which is solved to optimality or
proven infeasible.
The objective function of the sub-problem varies with
$y$ but the feasible set of solutions is independent
of these complicating variables.

If the solution of the master problem is feasible
then the dual of the sub-problem will find optimality at
an extreme point, while if the master's solution was
infeasible, the sub-problem is unbounded indicating the
existence of an extreme ray in its solution space.
In either case, we can formulate a Benders cut to add
back into the master problem to remove the current solution
from the feasible set.

When the sub-problem's solution is bounded and optimal
we can add what are called \emph{optimality} cuts to the master problem
\begin{IEEEeqnarray}{rCl}
	z&\geq& (b-By)\bar{u}_i^T + d^T y,\quad i=1,2,\ldots \label{eq:lit:bendExtPt}
\end{IEEEeqnarray}
where each $u_i$ corresponds to an extreme point. When the solution is unbounded
we can create \emph{feasibility} cuts for each extreme ray
\begin{IEEEeqnarray}{rCl}
	(b-By)\bar{v}_j^T &\leq& 0,\quad j=1,2,\ldots \label{eq:lit:bendExtRay}
\end{IEEEeqnarray}
where each $v_j$ corresponds to an extreme ray. After these cuts are created
we add them into the master problem and re-solve to find a new trial solution
for $y$. The new RMP is formulated as follows
\begin{IEEEeqnarray}{rClCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Min }~ z} &\hspace{4mm}& \label{eq:lit:bendSimpleMast1}\\[\eqnv]
	\text{s.t. }\quad Fy &\leq& p & & \label{eq:lit:bendSimpleMast2}\\[\eqnv]
	z &\geq& (b-By)\bar{u}_i^T + d^T y & & i=1,2,\ldots   \label{eq:lit:bendSimpleMast3}\\[\eqnv]
	(b-By)\bar{v}_c^T &\leq& 0 & & j=1,2,\ldots   \label{eq:lit:bendSimpleMast4}
\end{IEEEeqnarray}

\authciteb{Geoffrion1972} generalized the classical approach to a broader class
of optimization problems which meant the sub-problems no longer needed
to be formulated as a linear program but could also be nonlinear.
Further to this improvement, \authciteb{Magnanti1981} investigated
ways to generate Pareto-optimal Benders cuts to produce a tighter
relaxation in the master problem.
Other possible enhancement methods for Benders decomposition
exist including two- and three-phase heuristic approaches devised by \authciteb{Cordeau2001};
trust regions to significantly reduce the runtime considered by \authciteb{Santoso2005}
and \authciteb{Maher2014};
and parallelized solving of the decomposed problem studied by \authciteb{Dempster1998}
and \authciteb{Nielsen1997}.

\subsection{Logic-Based Benders}
\label{sec:lit:logicBend}
\citeauthor{Hooker2003}'s seminal paper from \citeyear{Hooker2003}
propose another generalization of Benders decomposition
to a much wider context while maintaining the strategy of
``learning from ones mistakes''.
One of the central ingredients in Benders decomposition
is the Benders cuts which allow information to be
communicated between the partitions of the problem.
In the classical case, the formulation of the sub-problems
is restricted to linear and non-linear programs.
However, this requirement hinders the ability of
Benders decomposition to be applied to problems
which are highly combinatorial in nature, such as scheduling,
as it can be impractical to formulate combinatorial problems
using a linear or non-linear model.

\citeauthor{Hooker2003} note that ``the key to generalizing Benders decomposition
is to extend the class of problems for which a suitable dual can be formulated''.
To this end, they introduced the concept of an \emph{inference dual} of any
optimization problem to the literature.
We note that this dual is different to the traditional LP dual
that the reader my be familiar with.
The inference dual is a proof of optimality which allows us to formulate
Benders cuts purely through logical reasoning.
This adaptation of the classical method proposed in \cite{Benders1962}
is named \emph{logic-based} Benders decomposition.

A formal description of logic-based Benders decomposition
applied to a minimization problem, based on the work
of \authciteb{Hooker2007}, is presented as follows.

Logic-based Benders decomposition applies to problems of the form
\begin{IEEEeqnarray}{rCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Min }~ f(x,y)}  \label{eq:lit:bendLogicFormal1}\\[\eqnv]
	\IEEEeqnarraymulticol{3}{c}{\text{s.t. }\quad C(x,y)}   \label{eq:lit:bendLogicFormal2}\\[\eqnv]
	x &\in& D_x \label{eq:lit:bendLogicFormal3}\\[\eqnv]
	y &\in& D_y \label{eq:lit:bendLogicFormal4}
\end{IEEEeqnarray}
where $C(x,y)$ is a set of constraints depending on both the $x$ and $y$ variables.
General domains of $x$ and $y$ are denoted by $D_x$ and $D_y$ respectively.
If we consider fixing the value of $y$ to $\bar{y}\in D_y$, then
the following sub-problem arises
\begin{IEEEeqnarray}{rCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Min }~ f(x,\bar{y})}  \label{eq:lit:bendLogicFormalSP1}\\[\eqnv]
	\IEEEeqnarraymulticol{3}{c}{\text{s.t. }\quad C(x,\bar{y})}   \label{eq:lit:bendLogicFormalSP2}\\[\eqnv]
	x &\in& D_x \label{eq:lit:bendLogicFormalSP3}
\end{IEEEeqnarray}
where $C(x,\bar{y})$ is the set of constraints that result by fixing $y=\bar{y}$.

The inference dual of the program (\ref{eq:lit:bendLogicFormalSP1}--\ref{eq:lit:bendLogicFormalSP3})
is the problem of inferring the tightest possible lower bound on $f(x,\bar{y})$ from $C(x,\bar{y})$.
Using the symbol ``$\Rightarrow$'' to denote logical implication, the
inference dual can be represented as follows
\begin{IEEEeqnarray}{rCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Max }~ v} \label{eq:lit:bendLogicFormalSPdual1}\\[\eqnv]
	\text{s.t. }\quad C(x,\bar{y}) \overset{P}{\Rightarrow} f(x,\bar{y})&\geq& v  \label{eq:lit:bendLogicFormalSPdual2}\\[\eqnv]
	v &\in& \mathbb{R} \label{eq:lit:bendLogicFormalSPdual3}\\[\eqnv]
	P &\in& \mathcal{P} \label{eq:lit:bendLogicFormalSPdual4}
\end{IEEEeqnarray}
where $A\overset{P}{\Rightarrow} B$ means that $B$ can be deduced from $A$ via proof $P$
where $\mathcal{P}$ is a family of proofs.

The solution found by optimizing the inference dual can be interpreted
as a proof of the tightest possible bound, $\hat{v}$, on
$f(x,y)$ when $y$ is fixed to $\bar{y}$.
From this bound we derive a bounding function $B_{\bar{y}}(y)$
for other values of $y$.
The bounding function must satisfy two properties:
\begin{enumerate}
	\item[B1:] $B_{\bar{y}}(y)$ provides a valid lower bound on $f(x,y)$
	for any given $y\in D_y$. That is, $f(x,y)\geq B_{\bar{y}}(y)$ for
	any feasible $(x,y)$ in the program (\ref{eq:lit:bendLogicFormal1}--\ref{eq:lit:bendLogicFormal4}).
\item[B2:] In particular, $B_{\bar{y}}(\bar{y})=\hat{v}$.
\end{enumerate}
If we denote the objective function value of the program (\ref{eq:lit:bendLogicFormal1}--\ref{eq:lit:bendLogicFormal4})
by $z$ then the valid inequality $z\geq B_{\bar{y}}(y)$ is a Benders cut whose validity is proved
by the inference dual.

At the $\mu^{th}$ iteration of the Benders algorithm,
we solve the master problem with constraints defined
as the cuts generated so far
\begin{IEEEeqnarray}{rClCl}
	\IEEEeqnarraymulticol{3}{c}{\text{Min }~ z} &\hspace{4mm}& \label{eq:lit:bendLogicFormalRMP1}\\[\eqnv]
	\text{s.t. }\quad z &\geq& B_{y^h}(y) & & h=1,2,\ldots,\mu-1  \label{eq:lit:bendLogicFormalRMP2}\\[\eqnv]
	z &\in& \mathbb{R} & & \label{eq:lit:bendLogicFormalRMP3}\\[\eqnv]
	y &\in& D_y & &\label{eq:lit:bendLogicFormalRMP4}
\end{IEEEeqnarray}
where $y^1,\ldots,y^{\mu-1}$ are the previous solutions of the master problem.
The solution $\bar{y}$ of the $\mu^{th}$ master %, \ie program (\ref{eq:lit:bendLogicFormalRMP1}--\ref{eq:lit:bendLogicFormalRMP4}),
defines the next sub-problem to be solved.

We denote $v_1^*,\ldots,v_{\mu-1}^*$ as the optimal values of the previous $\mu-1$ sub-problems.
The algorithm continues to iterate between
the master and sub-problem until the optimal value $z_\mu^*$
of the master equals $v^*=\min\{v_1^*,\ldots,v_{\mu-1}^*\}$.

% In Figure \ref{fig:lit:topLevelBendersFlowchart}, we provide
% a flowchart detailing the execution of logic-based Benders decomposition.
% We advise the reader to take this as only a framework
% of the process rather than an exact step-by-step method,
% as it's generality makes it well-suited
% to a number of modifications.
% \begin{figure}[tbp]
% 	\centering
% 	\caption{High level flowchart of logic-based Benders decomposition}
% 	\label{fig:lit:topLevelBendersFlowchart}
% 	\includegraphics[width=0.5\textwidth]{images/noimg.png}
% \end{figure}

This new decomposition method allows the general strategy
of Benders decomposition to be applied to a much wider class
of problems, however using the logic-based version
means that for each problem a new method of generating Benders
cuts needs to be devised.
Further to this, each new Benders cut must be rigorously
proven to be valid, \ie not remove any possibly optimal
solution from the solution space.
In the classical case, this formal proof was not
expressly required as the validity of any cut was
confirmed by the duality theory of linear programming.

\subsection{Hybrid-Benders}
\label{sec:lit:hybridBend}
To illustrate the benefits that employing logic-based Benders decomposition
can provide, we detail one of its a special cases which has demonstrated success in
the literature: hybrid Benders decomposition.
What distinguishes the hybrid variant
is the use of multiple kinds of solving technology
to tackle a single problem
within a logic-based Benders decomposition framework.

Here we will examine examples from the literature
where MIP solvers were effectively combined with CP solvers
to take advantage of their complimentary strengths.
The benefits of employing such an approach are clear when the
structure of a problem allows a natural decomposition,
and the resulting partitions can each be efficiently
solved by different kinds of solvers.
MIP solvers have been successfully applied to solve a wide
range of problems including network synthesis,
crew scheduling, planning and capital budgeting.
While the FD propagation of CP solvers has been effectively
applied to solve combinatorial discrete optimization problems
and feasibility problems for resource allocation and scheduling.

\authciteb{Jain2001} considered a parallel machine sequencing problem
and devised a Benders decomposition of the original problem into a set of assignment decisions
and a set of sequencing decisions.
A MIP model was devised to tackle the assignment problem,
which could take advantage of the LP relaxation and
\bab search offered by MIP solvers.
Testing the feasibility of the solutions of the relaxed problem
could be achieved efficiently by making use of the propagation
engines offered by CP solvers.
The feasibility sub-problems were linked to the assignment master problem
via a set of feasibility Benders cuts.
In their case, multiple nogood cuts were generated
at each iteration of the Benders algorithm.

The complimentary strengths between MIP and CP solvers
have been noted by \authciteb{Timpe2002}, \authciteb{Benoist2002}
and \authciteb{Hooker2005}.
The problems tackled by these papers
demonstrate the wide applicability of this framework.
Specifically these authors approached the
following problems: polypropylene batch scheduling,
call centre scheduling and multi-machine scheduling respectively.

\authciteb{Li2009a} use the hybrid Benders decomposition
framework to model a variant of the well-studied
Resource Constrained Project Scheduling Problem (\RCPSP).
The \RCPSP is a highly combinatorial optimization problem
with a number of extensions in the literature \cite{Szeredi2016,Tran2012}.
This scheduling problem is naturally suited to be modelled
by the global constraints \cumu and \disj offered by CP.
Due to this, a number of advances have been made in theory
of CP solvers to improve their ability to solve the \RCPSP.
Some of these advances were detailed previously
in Section \ref{sec:lit:cp}.

The variant considered by \citeauthor{Li2009a}
introduces an additional layer of complexity to the problem
by allowing each of the resources to have more than just
a singular capability, as is the case in the basic \RCPSP.
This addition results in a set of assignment decisions
between the resources and the tasks of the project.
As the problem now had two distinct sets of decisions ---
assignment and scheduling --- they chose to employ
a Benders decomposition.
The consequence was a relaxed master problem, modelled as a MIP
to handle the assignment decisions and a feasibility sub-problem
encoded as a constraint program.
As a result of this decomposition, \citeauthor{Li2009a} were still able to utilize the
global constraints of CP to effectively encode the scheduling aspect of the problem.
The authors' hybrid Benders decomposition is notable
due to the variety of both optimality and feasibility
Benders cuts they were able to formulate.
In their further work \cite{Li2009b,Li2012,Li2015} a number of
problems involving assignment and scheduling decisions
are tackled again using this hybrid approach to achieve
an effective decomposition.

\authciteb{Tran2012} consider another variant of the \RCPSP,
closely related to the assembly line balancing
problem we are concerned with in this thesis.
Specifically, their focus was on the resource constrained scheduling problem
with unrelated machines and sequence-dependent setup times between tasks.
They applied a logic-based Benders decomposition and formulated
a hybridized solution methodology.
Their decomposition of the problem gave them the
ability to formulate each sub-problem as an asymmetric TSP.
The TSP is a combinatorial optimization problem which is of
particular interest to the academic community and as such,
dedicated solvers which are tailored to this problem
are available \cite{Applegate2017}.
Having the ability to take advantage of the wealth of
research into the inherent combinatorial sub-structure of their problem,
the TSP, allowed their method to solve problems
six orders of magnitude faster than what was previously possible
by MIP solvers, while also allowing the optimal solution
to be found for instances twice as large as previously able.

\subsection{Summary}
We hope these applications of logic-based Benders decomposition
to a range of difficult problems has provided
some insight into this framework's applicability.
When problems can naturally be decomposed into their more basic
combinatorial components, it is possible to achieve superior
results by combining solution approaches which can more readily
exploit the inherent structures of the problem.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% This is the LaTeX file for the chapter:
% Blockchain Security
% for the manual:
% COMP716 - Highly Secure Systems
% Written by Jeff Nijsse
% September & October, 2018.
% Last updated Dec 10, 2018.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
\chapter{Blockchain \& Cryptocurrency}\label{Ch:BC}
The \defn{crypto} in cryptocurrency comes from cryptography; thus, a cryptocurrency is a currency that is cryptographically secured. In 2017 the term \defn{cryptocurrency} was added to the Oxford English Dictionary along with \defn{bitcoin} and \defn{blockchain}. This chapter will introduce the technology that enables bitcoin and some of the security implications of a blockchain based application. Blockchain and bitcoin have become synonymous but there are important differences. First we will look at the blockchain as a data structure.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Blockchain Data Structure                                                     %%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Blockchain Data Structure}\label{Se:BCDataStructure}
A blockchain is a data structure whereby a single block of data contains a reference to a previous block. Usually this has occurred in the past, and so a chain of blocks can represent a chronological ordering of data. When a new block is created it must include a reference pointer to the previous block in the chain. This is done by including a hash of the previous block.
\begin{center}
\includegraphics[width=0.85\textwidth]{blockchain}
\end{center}
The blockchain must be created one block at a time and mass deletion or appending of new blocks is not possible to maintain the correct hash linking. If there are multiple new blocks to be added to the data structure, they must queue and be processed according to some predetermined rules. In this sense, a blockchain is considered to be immutable, and append-only. The first block, labelled ID:1 above, is often called the genesis block, and this is the only block that has been created by design at the inception of the program. All other blocks are added in sequence according to the protocol of the system.

Recall that a hash function takes a variable length input and produces a fixed length output with no discernible pattern. A good hash function is one-way which means that it is easy to calculate (verify) but difficult to reverse. Given a hash it should be computationally infeasible to both produce the data that created the hash, and to find a message that produces the same hash. This scenario is called a collision and for the blockchain to be secure it should be composed using a collision-resistant hash function.

If an adversary wishes to modify data in a block -- such as financial transaction data -- the resulting hash pointer will need to be updated. (Even a variation of a single bit yields a different hash.) So rather than having to store the entire data set for verification, we can store the hash pointers and easily see if they are correct. Our adversary that altered some data would have to change every subsequent hash pointer to the tip of the blockchain. Simply storing the most recent hash in a place that can't be modified is enough to verify the whole chain. This can be done by keeping multiple copies in different locations.

\begin{center}
	\includegraphics[width=0.5\textwidth]{merkle}
\end{center}

The pages inside the books (data) can also be represented using hash pointers through a \defn{Merkle tree}, named after computer scientist and cryptographer Ralph Merkle. The leaves of the binary tree structure are the data and a hash pointer is created for every pair of leaves. When there are two pairs of leaves, the two hash pointers are combined and hashed into a third hash pointer. This continues until the root of the tree is a single hash representing all the data. The Merkle tree is secure in the same way as the blockchain -- if Eve tries to modify some data, then the root hash pointer will change.

A blockchain data structure can be \defn{permissioned} and require some authentication to browse elements and append blocks, or \defn{permissionless} allowing anyone to view the data and perhaps add blocks depending on the rules of the protocol. There are applications for both styles of blockchain and, as many believe, currency works best as a public and transparent data structure.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Cryptocurrencies                                                              %%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Cryptocurrencies}\label{Se:Cryptocurrencies}
Financial transaction data\footnote{Blockchains are sometimes called open ledgers in reference to an accounting ledger that tracks income and disbursements; similarly, distributed ledger technology refers to a blockchain combined with distributed computing architecture.} is an ideal use case for a blockchain data structure as it is generally chronological, and does not contain any cyclic elements (such as changing your name, which was appended to the database at birth). Hash pointers only work in a system like this because if a previous element changes, the subsequent hash pointers would need to be updated. Transactions themselves may be reversed, but from an  accounting perspective this is simply a debit back to the customer represented as a second transaction. Both transactions are stored and should be retrievable in the future.

A monetary system is an evolution of social trust. In small tribes or individual families, money is not necessary because the members trust each other to keep a tally\footnote{The term \emph{tally} comes from an old British system of keeping debts with notched wooden sticks called tallies.} of various debts and credits. In the case of a cryptocurrency, hash functions and public key cryptography allow a user to trust a stranger with their money. A monetary system can be constructed with a number of axioms.

Any monetary system must be:
	\begin{description}
	\item[Fungible] No single token is differentiable in value from another. Artwork exchange would be non-fungible because of the inherent personal differentiation of value.
	\item [Durable] The tokens must last at least long enough to be used in two transactions: one to receive them and one to spend them. This is solved easily with digital tokens.
	\item [Scarce] If everyone had easy access to it, it would not be valuable. This is a central problem with digital currency because copying digital entities results in a perfect duplicate indistinguishable from the original.
	\item [Divisible] Bartering with whole goods is difficult because value might not be aligned. This is a benefit of a digital system; coins can have a very high resolution, much greater than \$0.01.
	\end{description}

A cryptographically secure financial transaction system should embody the following additional elements:
	\begin{description}
	\item [Decentralization] Under control of the users; as opposed to the administrators. This helps ensure that value cannot be stolen, diluted, or written off by powerful actors in the system.
	\item [Uniqueness] Coins should only be allowed to be spent once. This is a key barrier to overcome when transacting in a digital medium.
	\item [Predictability of Supply] Creation and distribution of new tokens according to rules the users agree with.
	\item [Distributed Network] The network should be invulnerable to attack or regulatory control from a single entity.
	\item [Security] Ownership of addresses (coins) and the ledger should be computationally secure and  hackers or bad actors should not be able to modify the historical record.
\end{description}

\subsection*{A Brief History: Before Bitcoin}
In 1983 David Chaum came up with a scheme for electronic cash involving distribution, tracking, and spending of coins that relied on a bank entity for clearing. Chaum's idea was to use what are called \defn{blind signatures} to keep the identity of the user hidden from the central clearing house, while still being able to trust that the hidden user had viable funds to spend. If the bank issues a new digital note to you and lets \textit{you} pick the serial number and then the bank signs it, there is now a new note in existence guaranteed by the bank but they don't know who they gave it to. If you tried to spend your new digital note twice the bank could step in and compare the transactions and call you out on the double-spend attempt.

Chaum started DigiCash in 1989 to allow users to conduct anonymous online transactions. DigiCash and others in the intervening years relied on banks for the issuance of new currency, and although had unique privacy features, banks largely did not need the new technology and did not adopt it. Much of the work since this time has involved trying to remove the bank from the equation to create a decentralized currency.

Another issue to consider is the minting of the digital coins. A good solution requires a cost structure similar to the printing of modern notes where capital expenditure is large to acquire printing presses and develop the security features. This cost goes down over time as subsequent print runs cost less and less making it difficult for single users to print their own money. For digital minting, the computational power required to copy a digital serial number is negligent and so a more intensive process is required.

In 1997 original cypherpunk Adam Back came up with \defn{Hashcash}. Hashcash was developed to solve the problem of email spam. DDoS attacks on email servers can be mitigated by hashing a proof of computational effort and including it in the email header. Spammers and phishers must send thousands of emails to get a response and so a small delay in each mail sent can be very frustrating. The delay is negligible for the average user as their own processor could compute the proof-of-work during composing their message and not notice the cpu time required. The work that the cpu is doing is finding a solution to a hash puzzle. A variant of Hashcash is used in bitcoin and offers a solution to the coin minting problem. Section \ref{Se:POW} discusses proof-of-work.

%Hashcash is included in Linux distributions To invoke hashcash:
%\begin{code} terminal code to get some hashcash output
%\end{code}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Bitcoin                                                                       %%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Bitcoin}\label{Se:bitcoin}
In 2008 a whitepaper emerged by Satoshi Nakamoto that combined many of the elements we have studied thus far and solved the double-spend problem in a distributed system. The new feature in bitcoin is called \defn{emergent consensus}\footnote{The term Nakamoto consensus is gaining popularity in the literature.} and allows a network of independent users and operators to monitor transactions against a global ledger. Every node independently verifies transactions and the network globally agrees on new blocks to be appended to the blockchain. This agreement of the blockchain state by its users is the last piece in the digital cash puzzle thereby eliminating the need for a trusted intermediary. Bitcoin tied together the following modules to create a distributed, cryptographically secure financial system:

\begin{description}
	\item [Decentralization] Bitcoin software is open-source so anyone can run the software and set up a node in the network. There has been considerable growth since 2009; estimates indicate there are around 10,000 full nodes online. Each node contains a copy of the blockchain and can independently validate transactions. This eliminates single-point-of-failure problems common in centralized systems.
	\item[Consensus] All participants must agree on the state of the ledger. It is this trust in the ledger that allows one party to interact with another that they did not previously know or trust personally. Sometimes this is referred to as \defn{trustless}.
	\item [Uniqueness] Every transaction has a hash pointer within a block. In this sense all the transactions are unique. Individual coins do not actually exist, rather hash pointers can indicate unspent transaction outputs that can show \begin{code}9.87654321 bitcoin\end{code}\footnote{A bitcoin is divisible into 8 decimal places; 0.000 000 01 is called a satoshi.}, for example. Whoever has the private key to these unspent outputs is considered the owner.
	\item [Supply Stability] Every block that is added to the chain results in the minting of new bitcoins. This amount is halved about every four years so the total supply is a decreasing geometric series. The maximum number of bitcoins that will be minted is just under 21 million.
	\item [Security] Bitcoin uses the NIST specification \begin{code}secp256k1\end{code} elliptic curve digital signature algorithm. This keeps users private keys secure and authenticates transactions in the network.

\end{description}
How do you keep the state of the whole blockchain safe from sticky fingers? What if one party decides to spend their coins to buy some gold, and then after receiving the gold, they spend their coins \textit{again}, this time sending them to another address they control. We need a way to distinguish between the two transactions without mediation. The next section will discuss this issue: how can everyone agree on what transaction is valid?

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Blockchain Consensus                                                          %%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Consensus}\label{Se:Consensus}
Solving the situation described above, known as the \defn{double spend problem}, is the final piece of the puzzle that was put in place for trustless decentralized digital cash to succeed. First we will describe the \defn{Byzantine Generals Problem}, a famous computer science communication problem.

Imagine three generals of the same army, all wanting to attack an enemy fortress at dawn the next morning. Each general knows their individual divisions cannot win the enemy fortress. Each general also knows that with the help of the other's forces they can win the position. So General \textit{Alpha} sends a messenger to General \textit{Beta} with the message ``We attack at dawn.'' This is where the problems start. There are many scenarios that could play out. The messenger could be captured by the enemy; the messenger could be a double-agent; the messenger could get lost and not deliver the message; General Beta could double-cross Alpha, etc. To win the battle, the three generals must come to consensus and agree to attack at dawn. Practically speaking, each general must receive confirmation from each of the other generals that they received the message \emph{and} are in agreement. This scenario maps nicely onto cryptography and message passing.
%distributed consensus state diagram
\begin{center}
	\begin{tikzpicture}[scale=0.2]
	\tikzstyle{every node}+=[inner sep=0pt]
	\draw [black] (35,-20.2) ellipse (15 and 2);
	\draw (35,-20.2) node {all nodes must agree};
	\draw [black] (35,-31.4) ellipse (15 and 2);
	\draw (35,-31.4) node {some nodes must propose updates};
	\draw [black] (45,-13.6) circle (3);
	\draw (45,-13.6) node {How?};
	\draw [black] (26,-37.9) circle (3);
	\draw (26,-37.9) node {Who?};
	\draw [black] (22.205,-30.413) arc (-124.75549:-235.24451:5.615);
	\fill [black] (22.2,-21.19) -- (21.26,-21.23) -- (21.83,-22.05);
	\draw [black] (47.747,-21.315) arc (52.66893:-52.66893:5.64);
	\fill [black] (47.75,-30.28) -- (48.69,-30.2) -- (48.08,-29.4);
	\draw [black] (37.4,-18.2) -- (42.43,-15.15);
	\fill [black] (42.43,-15.15) -- (41.49,-15.14) -- (42.01,-16);
	\draw [black] (32,-33.44) -- (28.43,-36.14);
	\fill [black] (28.43,-36.14) -- (29.37,-36.08) -- (28.79,-35.27);
	\end{tikzpicture}
\end{center}
A decentralized blockchain application has many nodes in the network and each of them has a copy of the blockchain. If Bob wants to send Alice his tokens, the nodes in the network must first agree that Bob has available tokens to spend, and then update the state before Bob can double-spend his tokens. The network needs to be in a state of \defn{consensus} for people to trust it. Consensus in the Byzantine Generals problem has been proven to be impossible if more than a third of the nodes are malicious.\footnote{The term Byzantine in computer science indicates an unpredictable actor--this includes malicious activity.}

Bitcoin does not exactly mirror the model of the Byzantine Generals and so is not impossible to reach consensus. In other words, bitcoin does achieve consensus despite Byzantine behaviour. More practically, bitcoin achieves a state of \textit{emergent} consensus. This means that over time as blocks get added to the chain, a general state emerges that everyone agrees on. So how does this occur without clear rules? A closer look at the double-spend attack will illustrate how nodes come to agreement.

\subsubsection*{The Double-Spend Attack}
A dodgy actor, Eve, could manipulate the blockchain by sending some coins to Jeff and receiving a good or service. Then she could send those same coins to Bob's address whom she also controls. If the transaction involving Bob's address is validated on the blockchain, then Eve would have successfully double-spent the same coins. She now has the coins in Bob's address and the goods from the transaction with Jeff.\\
\begin{center}
\includegraphics[width=0.8\textwidth]{doubleSpendSuccess}
\end{center}
How can Jeff avoid this scenario, especially when he does not know the other party he is transacting with? The solution is to wait. Over time as more blocks are added, say every ten minutes, the older blocks are more likely to be valid.\footnote{Standard confirmation time for bitcoin is six blocks, or one hour. The number of blocks to wait until considering a transaction to be confirmed is a personal preference.} Stated another way, as more blocks are added to the chain there is an exponential decrease in the probability of a block being rejected by other nodes. This is the method that secures the blockchain against malicious nodes attempting to double spend.

\begin{center}
	\includegraphics[width=0.8\textwidth]{doubleSpendSolution}
\end{center}

And what about Eve changing history? She would have to alter the block where the transaction is stored, as well as any subsequent blocks. The older the block is, the more computational effort is required because the hash pointers are linked. Consider that Eve is also competing against other nodes in the network that are honest. (We will return to the case of dishonest nodes when discussing a 51\% attack.) The next question is how to keep nodes honest, especially when dealing through the medium of currency?

\subsection*{Proof-of-Work}\label{Se:POW}
Here is an important distinction where bitcoin varies from other methods of reaching consensus. There are incentives for nodes to act honestly that are built into the protocol. The first is called the \defn{coinbase} transaction and awards freshly minted bitcoins to whoever added the block to the chain. This is how new bitcoins come into circulation. The second is from transactions fees. By listening to the network, validating transactions, and including them in a block, whoever is operating the node can choose to include transactions that also offer an extra fee. Because bitcoin itself is designed to be digital money, this makes perfect sense and is why cryptocurrency is considered the killer app for blockchain.

Referring back to the state diagram, who gets to propose these blocks and earn the block reward? How do we ensure the distribution is random? How are Sybil attacks prevented? Mining is the answer and provides the incentive structure to maintain this delicate balance.

\subsubsection*{Mining}
A miner is a network participant that contributes their computing power in a demonstrable way. A fair way to allocate the incentives would be by some resource that can not be gamed or monopolized. One such way is by computing power, and is called \defn{proof-of-work}. Another method is by value held in reserve and is called proof-of-stake. Proofing methods is a very active research topic as it has many game theoretic variables.\footnote{Other methods include DPoS: delegated proof-of-stake, PoET: proof of elapsed time, and PoA: proof of authority.} Bitcoin miners participate by using their computing power to validate transactions and suggest new blocks. For this effort they receive rewards in proportion of their contribution to the network as a whole. Bitcoin uses a \begin{code}SHA256\end{code} hashing algorithm as the hash puzzle that miners have to find a solution for.
\begin{center}
\begin{code}
	Block = Hash(nonce||previousHash||data||data||...||data) < target
\end{code}
\end{center}

 Hash function output has a random distribution and so to find a block, your hash must be below a certain target level. The target level will be the hash size with a certain number of leading zero bits, for example:\begin{center}\begin{code}
	000000000000000000117c467ab5336077cb04f7f70ea6ebcd68e0b3ef6cf909
\end{code}\end{center}
was the successful hash of block \begin{code}529283\end{code}. The only way to find a hash with a smaller value than the target is to change a nonce value and re-hash the bundle of transactions over and over. When a target is hit, the block is broadcast to the network as a proof of computational work done in solving the hash puzzle.
The bits here should be randomly distributed, like a lottery, to prevent gaming the system and earning more rewards than your proportion\footnote{As the bitcoin network has matured, dedicated hardware called ASICs (application specific integrated circuits) to solve the \begin{code}SHA256\end{code} algorithm have dominated. It is no longer feasible for a single participant to mine bitcoin without dedicated hardware.}. Computing cycles in the bitcoin network is called \defn{hashpower} in reference to \begin{code}SHA256\end{code}. As more miners come online, the total hashpower increases leading to greater overall probability of successfully hashing a value below the target. To keep the temporal distribution of blocks even, this target difficulty automatically adjusts according to the protocol every 2016 blocks, or approximately two weeks.

Below is a log plot of the mining difficulty from 2009-2018 showing that the difficulty has increased exponentially. The slope roughly correlates to the growth of the network.\footnote{Source: \url{https://www.blockchain.com/charts/difficulty}}

\begin{center}\vspace{0.3cm}
	\includegraphics[width=1.0\textwidth]{difficulty}
	% source https://www.blockchain.com/charts/difficulty
\end{center}

\subsubsection*{Block Time}
The difficulty adjustment aims to keep the time between blocks (successful hashes) at around ten minutes. In the early days Satoshi and a few others could use their PC processors to find blocks every ten minutes. The hashpower has steadily increased and so has the difficulty target to keep the block time constant. Finding a hash of a block that is below the target size is a discrete event; it is either below or it is not. As with a lottery, it is only a matter of time before a hash is found, and the previous hash is independent of the current attempt. Statistically, this is a \defn{Poisson} distribution. %The probability of a block being found is $P(N)=1-e^{-1}$

\begin{center}
	\includegraphics[width=0.85\textwidth]{blockTimePoisson}
	% source https://en.bitcoin.it/wiki/Confirmation
\end{center}

From the graph\footnote{Source: \url{https://en.bitcoin.it/wiki/Confirmation}} you can see its possible to find a block immediately after the last one, but unlikely. Similarly, its possible that no block could be found for an hour or two, but unlikely. In practice, the average block time is just under ten minutes because as more miners join the network it is easier overall to find blocks, until the difficultly is reset.

\subsubsection*{Consensus Summary}
In summary, consensus of nodes in the network emerges from an implicit agreement on the longest chain of blocks. The nodes are all agreeing that this blockchain represents the most proof of computational work. Miners will abandon shorter chains to compete to build on the longer one to win the block reward. The transactions in this chain will have an increasing probability of being accepted over time as new blocks are mined.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Blockchain Security Considerations                                            %%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Security}\label{Se:BCSecurity}
The first security consideration was the double-spend attack as discussed above. The next weakness comes at the network level. To this point we have said that most nodes in the network will act honestly and the mining block reward incentivises this behaviour. If a collective forms with a considerable portion of nodes this could lead to a 51\% attack.

\subsubsection*{51\% Attack}\label{Se:51}
Should a single entity gain control of more than half of the hashpower in the network, this could lead to a 51\% attack. The attacker can't steal coins directly as this involves subverting elliptic curve cryptography. The attacker can however user their majority status in some interesting ways. At $>50\%$ you have the ability to find more blocks and direct consensus of the blockchain. The attacker can censor transactions by refusing to add blocks containing someone's address. This would be akin to blacklisting certain addresses, but does not completely exclude these from being included by honest nodes (the 49\%). Next, you may say okay, lets just increase the block reward and award myself 1 million coins for the next block found. This will be rejected by the network because the block reward is hard-coded and so the malicious ``plus-one-million'' transaction can never be spent.

An adversary doesn't necessarily need 51\% of the hashpower, but with a large number of nodes in the network they could facilitate a double spend attack by the following steps.
\begin{enumerate}
	\item Broadcast a transaction they intend to double spend that the receiver believes is legitimate.
	\item Mine a different branch of the blockchain that doesn't include their transaction from (1), but does include a transaction to themselves.
	\item Continue mining the forked branch until the illegitimate transaction is accepted by the receiver and then broadcast their alternate blockchain.
	\item As consensus relies on the longest chain representing the most proof of work, this new (and falsified) chain will now be considered valid.
\end{enumerate}

The main practical threat to a single entity controlling the majority of the hashpower is that it will lead to centralization and cause users to abandon the system altogether. Once hashpower gets close to this threshold it is also possible that developers will step in and update the software to limit this behaviour. It is unknown how this will play out in the future. Thus far the bitcoin network has been relatively decentralized, but a few mega mining companies have emerged. There have been some cases of 51\% attacks on alternative cryptocurrencies such as Verge, Horizen, and Vertcoin. Smaller cryptocurrencies are more vulnerable to a large mining pool shifting their resources for this purpose.

\subsubsection*{Cryptography}\label{Se:bitcoinCryptography}
Contrary to popular belief there is no standardized encryption in the bitcoin protocol. As a decentralized system of exchange, there is no need for encryption. All the transactions are stored in the blockchain and accessible to everyone. Access to the private keys controlling addresses is maintained solely through personal security of the user.

So can someone steal my coins? Not by hacking. Elliptical curve cryptography keeps keys safe from cryptanalysis via the difficulty in solving the discrete logarithm problem. The standard curve \begin{code}secp256k1\end{code} was chosen by the designer of the bitcoin protocol. This is a choice unique to bitcoin, as the more common \begin{code}secp256r1\end{code} is used in Transport Layer Security (TLS) for web browsing, email, etc., and discussed in Section 2.4. ECC was chosen because it provides the same level of relative security with smaller keys compared to RSA.

It is imperative that users keep their private keys secret as in any other cryptosystem. There are some added enhancements for encrypting passwords and wallets but these are third party additions and not built in to the protocol.

\subsubsection*{Key Storage}
Storage of cryptocurrency is similar to storage of your online banking details. There are no actual bitcoins to keep safe; there is no \begin{code}<bitcoin object>\end{code} to keep password protected. Remember the blockchain is an open ledger of every transaction in the network. Just like your online banking system keeps track of all account balances and doesn't actually store any fiat\footnote{Fiat is government issued money. New Zealand dollars and Japanese yen are both considered fiat currency.}. Bitcoin is cryptographically secured, and if you have the private keys to an address, then you control the funds that address references in the blockchain.

A bitcoin address is the public key of a public-private key pair generated by a 256 bit elliptic curve function. The public key is encoded in base-58 to reduce errors in transcription by removing similar characters: \begin{code}0O1l\end{code}.
\begin{center}
	\includegraphics[width=0.65\textwidth]{keysQR}\end{center}
% source bitcoin presentation
Cryptocurrencies are more risky than traditional online banking because if you lose your private keys they are not recoverable. There is no password reset, or government issue ID verification to recover your account.

There are a number of way to store your keys:
\begin{description}
	\item[Hot Wallet] This is the most common. A software application that stores your private keys so you can sign transactions, keeps track of your tokens, and maybe offers some additional functionality like multiple addresses, or multiple token support. The risk here is that if you lose your device, your private keys are gone with it. Day-to-day transactions using a smartphone would be done through a hot wallet.
	\item[Cold Storage] For larger amounts, savings, investments, and custodial services, a cold wallet is recommended. This is a device that is not powered and has no connectivity and stores your private keys internally. When tokens are required, it can be plugged into a USB and connected to the network.
	\item[Paper Wallet] Once a key pair has been generated and used to receive tokens, that private key will always have ownership of the tokens. The private key can be printed out or written down, sometimes as a QR code for easy scanning. After the memory has been cleared there is no more digital record of the private keys.
	\item[Brain Wallet] The most hard-core of storage systems is to remember your private keys and destroy all physical and digital evidence of them. Certain memory mnemonics can help with this.
\end{description}
Note that for all these methods including memorization, its possible to \textit{send} tokens to your address without connecting the keys to the network. This transaction will be recorded by all the nodes running the blockchain and propagated accordingly. Private keys are only required to sign transactions (spend tokens). All of these methods are vulnerable to crisis such as fire or theft, and represent a single point of failure. Backing up your keys can solve the crisis issues, but someone could still steal the backup.

\subsubsection*{Secret Sharing}
Lets assume that eventually someone \textit{will} steal our private keys. In this scenario they need the entire key, in the correct order to use it. So why not split the key in two or several parts? This is known as \defn{secret sharing} and works like this: split the key into $N$ shares such that any $K$ of those shares can reconstruct the key, but with $<K$ shares nothing can be learned. Lets split the secret $S$  into 2 shares so that $N=2$ and $K=2$. If $S$ is a 128 bit number, a random 128 bit number $R$ is generated and the 2 shares can be $R$ and $S\oplus R$ (bitwise XOR). Either share alone does not help because $R$ is random, and $S\oplus R$ depends on knowing $R$. In essence the secret has been encrypted with a one-time pad cipher where $R$ is the key. This works for $N=K$. What about for $N>K$?

Say you split the secret in to four shares, give one to each of your trustworthy relatives, and at any time two of them are required to reconstruct the secret. This can be accomplished with some linear algebra. Any two points can construct a line, but given a third random point it is unlikely to be collinear with the first two. To generate $N$ shares, prepare a line with $N$ points. Now any two of these points can reconstruct the line, and determine the $x-$intercept ($S$), but any single point is useless because the slope is unknown.

\begin{center}
	\begin{tikzpicture}
	\pgfplotsset{ticks=none}
	\begin{axis}
	[grid=none,
	scale=1,
	xmin=0, xmax=5,
	ymin=0, ymax=2.5,
	axis lines=middle,
	axis line style={->},
	xlabel=$x$,
	ylabel=$y$,
	]
	\addplot[->,domain=0:5,samples=50,smooth,thick,black] {0.25*x+1};
	\addplot[mark=*, nodes near coords={$(0,S)$},every node near coord/.style={anchor=160}] coordinates {(0,1)};
	\addplot[mark=*, nodes near coords={$(1,S+R)$},every node near coord/.style={anchor=160}] coordinates {(1,1.25)};
	\addplot[mark=*, nodes near coords={$(2,S+2R)$},every node near coord/.style={anchor=160}] coordinates {(2,1.5)};
	\addplot[mark=*, nodes near coords={$(3,S+3R)$},every node near coord/.style={anchor=160}] coordinates {(3,1.75)};
	\end{axis}
	\end{tikzpicture}
\end{center}

The point $(0,S)$ represents the secret, a large random number less than a large prime, $P$. The shares are linear combinations modulo $P$, up to $N$, where any 2 of them will recover the secret.
\begin{align*}
x=0,\quad y&=S\\
x=1,\quad y&=(S+R)\mod P\\
x=2,\quad y&=(S+2R)\mod P\\
x=3,\quad y&=(S+3R)\mod P\\
\vdots\\
x=N,\quad y&=(S+NR)\mod P\\
\end{align*}
What if you require more than 2 shares necessary to reconstruct the key? If we increase the degree of our share-reconstruction function from linear to parabolic, then we have $K=3$ is necessary to find $S$ because 3 points can uniquely define a parabola. This can be continued up to $K=N-1$ shares.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Privacy                                                                       %%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Privacy}\label{Se:Privacy}
Identity in a decentralized system is based on private keys. Whoever has access to a private key can operate within the network and this activity is directly linked to the key. Just like how digital signatures provide non-repudiation, the activity of an address within an open ledger can be thought of as someone's identity. Keeping with this idea of keys as identities, multiple keys could be multiple identities. You could generate a new identity with a new random key pair and never use the old identity again.

There is a problem here if you want to remain anonymous. Anonymity means that you can still participate in the network but it is difficult or impossible to link your activity with your real identity. Even if one user has multiple addresses, the activities of those addresses is stored in the open -- every node in the network has a copy. For this reason, bitcoin is considered to be \defn{pseudonymous}. The activity of one address can be tracked, and this can be mapped to behaviour, possibly providing compelling evidence to link the two. Privacy in cryptocurrencies is a contentious issue and often cited as a founding principle. It goes right back to the core tenet of cryptography -- communicating in secret. For a user of cryptocurrency this may be as innocent as purchasing a VPN to use the internet where content and websites may be censored, or for much more nefarious purposes.\footnote{For many years bitcoin was synonymous with the \defn{Silk Road}, an online marketplace that remained out of reach of authorities and sold illegal goods. The possibility to purchase illegal material is still a main point of debate against cryptocurrencies. Many people have studied anonymity in markets and cash is still the best way to transact without revealing your identity.}

Coin mixing or laundry services aggregate transactions together and wash by sending a large transaction to one or many addresses controlled by the service. Funds are then returned less a service fee. This will help to obfuscate origin and destination of funds from forensic analysis but requires a large amount of trust in the mixing service. This is a weak and impractical solution to transaction anonymity.

\subsubsection*{Zero-Knowledge Proofs}
Privacy can be built in cryptographically at the protocol level using a technique called \defn{zero knowledge}. In cryptography, zero-knowledge proofs are a way to mathematically prove that you know a result without having to reveal the details of how you found it. An example comes from mining. You can prove to someone that you have the hash of a block by revealing the nonce and therefore give away the formula for how you found the hash. A zero-knowledge proof lets you do this without revealing the nonce. The details are non-trivial and out of the scope of the present course. Refer to Foundations of Cryptography by Oded Goldreich, 2001, for an overview.

Several alternative cryptocurrencies have implemented zero-knowledge proofs such as Z-Cash, Horizen, and Ethereum's Byzantium upgrade. These are not compatible with bitcoin as a different set of cryptographic principles had to be coded from the beginning. In Z-Cash, which was derived from zerocoin, the user has the option to transact privately by invoking zero knowledge verification. This takes extra computational resources and reduces efficiency representing a trade-off between privacy and speed.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Code-Breaking                                                                 %%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Code-Breaking}\label{Se:code-breaking}
In October, 2018, an image was posted online claiming to contain clues to unlock a wallet holding 310 bitcoin (at the time of writing worth \$2.9 million NZD). Just days later the coins were moved from the wallet indicating the private keys had been found in the image. The image below contains a total of four keys, and one with 0.31 bitcoin is still unclaimed. A high resolution image can be downloaded at \url{https://bitcoinchallenge.codes/}.
\begin{center}
	\includegraphics[width=1.0\textwidth]{BTCchallenge}
	% source https://bitcoinchallenge.codes/
\end{center}
%
%
%
%
% \\END Blockchain


\begin{comment}

 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.

\end{comment}

\subsection{Stepwise Generalized Linear Regression}

\noindent{\bf Description}
\smallskip

Our stepwise generalized linear regression script selects a model based on the Akaike information criterion (AIC): the model that gives rise to the lowest AIC is provided. Note that currently only the Bernoulli distribution family is supported (see below for details). \\

\smallskip
\noindent{\bf Usage}
\smallskip

{\hangindent=\parindent\noindent\it%
{\tt{}-f }path/\/{\tt{}StepGLM.dml}
{\tt{} -nvargs}
{\tt{} X=}path/file
{\tt{} Y=}path/file
{\tt{} B=}path/file
{\tt{} S=}path/file
{\tt{} O=}path/file
{\tt{} link=}int
{\tt{} yneg=}double
{\tt{} icpt=}int
{\tt{} tol=}double
{\tt{} disp=}double
{\tt{} moi=}int
{\tt{} mii=}int
{\tt{} thr=}double
{\tt{} fmt=}format

}

\smallskip
\noindent{\bf Arguments}
\begin{Description}
	\item[{\tt X}:]
	Location (on HDFS) to read the matrix of feature vectors; each row is
	an example.
	\item[{\tt Y}:]
	Location (on HDFS) to read the response matrix, which may have 1 or 2 columns
	\item[{\tt B}:]
	Location (on HDFS) to store the estimated regression parameters (the $\beta_j$'s), with the
	intercept parameter~$\beta_0$ at position {\tt B[}$m\,{+}\,1$, {\tt 1]} if available
	\item[{\tt S}:] (default:\mbox{ }{\tt " "})
	Location (on HDFS) to store the selected feature-ids in the order as computed by the algorithm,
	by default it is standard output.
	\item[{\tt O}:] (default:\mbox{ }{\tt " "})
	Location (on HDFS) to write certain summary statistics described in Table~\ref{table:GLM:stats},
	by default it is standard output.
	\item[{\tt link}:] (default:\mbox{ }{\tt 2})
	Link function code to determine the link function~$\eta = g(\mu)$, see Table~\ref{table:commonGLMs}; currently the following link functions are supported: \\
	{\tt 1} = log,
	{\tt 2} = logit,
	{\tt 3} = probit,
	{\tt 4} = cloglog.
	\item[{\tt yneg}:] (default:\mbox{ }{\tt 0.0})
	Response value for Bernoulli ``No'' label, usually 0.0 or -1.0
	\item[{\tt icpt}:] (default:\mbox{ }{\tt 0})
	Intercept and shifting/rescaling of the features in~$X$:\\
	{\tt 0} = no intercept (hence no~$\beta_0$), no shifting/rescaling of the features;\\
	{\tt 1} = add intercept, but do not shift/rescale the features in~$X$;\\
	{\tt 2} = add intercept, shift/rescale the features in~$X$ to mean~0, variance~1
	\item[{\tt tol}:] (default:\mbox{ }{\tt 0.000001})
	Tolerance (epsilon) used in the convergence criterion: we terminate the outer iterations
	when the deviance changes by less than this factor; see below for details.
	\item[{\tt disp}:] (default:\mbox{ }{\tt 0.0})
	Dispersion parameter, or {\tt 0.0} to estimate it from data
	\item[{\tt moi}:] (default:\mbox{ }{\tt 200})
	Maximum number of outer (Fisher scoring) iterations
	\item[{\tt mii}:] (default:\mbox{ }{\tt 0})
	Maximum number of inner (conjugate gradient) iterations, or~0 if no maximum
	limit provided
	\item[{\tt thr}:] (default:\mbox{ }{\tt 0.01})
	Threshold to stop the algorithm: if the decrease in the value of the AIC falls below {\tt thr}
	no further features are being checked and the algorithm stops.
	\item[{\tt fmt}:] (default:\mbox{ }{\tt "text"})
	Matrix file output format, such as {\tt text}, {\tt mm}, or {\tt csv};
	see read/write functions in SystemDS Language Reference for details.
\end{Description}

\noindent{\bf Details}
\smallskip

Similar to {\tt StepLinearRegDS.dml} our stepwise GLM script builds a model by iteratively selecting predictive variables
using a forward selection strategy based on the AIC (\ref{eq:AIC}).
Note that currently only the Bernoulli distribution family ({\tt fam=2} in Table~\ref{table:commonGLMs}) together with the following link functions are supported: log, logit, probit, and cloglog ({\tt link $\in\{1,2,3,4\}$ } in Table~\ref{table:commonGLMs}).

\smallskip
\noindent{\bf Returns}
\smallskip

Similar to the outputs from {\tt GLM.dml} the stepwise GLM script computes the estimated regression coefficients and stores them in matrix $B$ on HDFS; matrix $B$ follows the same format as the one produced by {\tt GLM.dml} (see Section~\ref{sec:GLM}).
Additionally, {\tt StepGLM.dml} outputs the variable indices (stored in the 1-column matrix $S$) in the order they have been selected by the algorithm, i.e., $i$th entry in matrix $S$ stores the variable which improves the AIC the most in $i$th iteration.
If the model with the lowest AIC includes no variables matrix $S$ will be empty.
Moreover, the estimated summary statistics as defined in Table~\ref{table:GLM:stats}
are printed out or stored in a file on HDFS (if requested);
these statistics will be provided only if the selected model is nonempty, i.e., contains at least one variable.

\smallskip
\noindent{\bf Examples}
\smallskip

{\hangindent=\parindent\noindent\tt
	\hml -f StepGLM.dml -nvargs X=/user/biadmin/X.mtx Y=/user/biadmin/Y.mtx	B=/user/biadmin/B.mtx S=/user/biadmin/selected.csv O=/user/biadmin/stats.csv link=2 yneg=-1.0 icpt=2 tol=0.000001  moi=100 mii=10 thr=0.05 fmt=csv

}