% $Id$

Many applications in Earth system modeling require
transforming fields between different grids.  Such
cases occur when coupling two components that utilize
different grids (eg coupling atmosphere and ocean
components of a climate model) or when a single component
utilizes more than one grid (eg nested grids, multi-grid
or spectral transforms).
Fields must then be interpolated, averaged or remapped
from one grid to another.  We will refer to this process
as {\em regridding}.

Many different methods
for performing regriddings are required.
Regridding heat or water fluxes between models requires
fluxes to be regridded in a way that conserves
the total energy or water in the model.  In other
cases, it is more important that higher-order
regridding methods be used to prevent discontinuities
in the interpolated fields or their gradients.

In addition to moving fields between grids,
components require moving ungridded data (eg observations)
to and from component grids.  Transforming fields
from physical space to spectral space is also
a frequent operation.

The ESMF Regridding facility (Regrid) will
supply necessary functions and data structures for
implementing regridding operations.  Appropriate
functions will be supplied for all supported ESMF grids,
including grids in spherical, Cartesian and cylindrical
coordinates and non-gridded lists of points
in those coordinates (see physical grid requirements for
complete list of supported grids).  A variety of methods will
be supplied, including conservative, spectral transform,
linear and higher-order methods.  The
remainder of this document outlines specific requirements
for all of the supported options.

\subsection{Location}

Regridding and interpolation is part of the ESMF
Infrastructure.  It will use information from physical
grids or distributed grids to compute
regridding information and use distributed grids
for performing data motion related to regridding.  It will
(presumably) be used by couplers and fields as well as
within component models when necessary.

\subsection{Scope}

ESMF Regrid is meant to be used for interpolating or
remapping data between supported ESMF grids.  Support
for moving fields between points on staggered grids
will be included in the physical grid.


\documentclass{article}
\usepackage{amsmath, array, graphicx}
\title{Bespoke splines}
\author{Raph Levien}
\date{\today}

\newcommand{\kcurve}{$\kappa$-Curve}

\begin{document}
\maketitle

\begin{abstract}
The quest for better splines for interactive curve design continues. The Euler spiral spline has a long tradition and is very fine when bending angles in the control polygon are small, but does not have a unique stable solution when angles increase. A new analysis shows that the family of ``two parameter splines'' can encompass a wide range of desirable behavior based on properties of a 2-dimensional curvature map. In particular, a periodic curvature map can yield robust splines, so a perturbation of the control points changes the resulting curve only slightly. This paper proposes a new spline that is robust over all inputs and preserves much of the quality of the Euler spiral spline. This spline is but one instance of a powerful design methodology that can produce splines optimizing for specific desired properties, based on the two-parameter paradigm.
\end{abstract}

\section{Introduction}

Tools for interactive curve design still feel primitive, in spite of much work on interpolating splines. Most professional curve design work, including font design, is done using B{\'e}zier curves. Such curves are difficult to learn and require expert operation to produce smooth results. Yet, they are expressive, and feel very robust in interactive use, so they remain.

One of the oldest and most promising splines for interactive use is the Euler spiral spline. With gentle inputs (no sharp bending angles), it produces very fine results. But in interactive exploration, for example when sketching new ideas, it's not hard to put the spline into a state where loops suddenly flip direction. Some implementations, fail to converge entirely, as Newton-based solvers are not always well conditioned. The author's own Spiro implementation, which includes Euler spiral splines, occasionally produces curves resembling particle accelerator tracks when the solver fails.

Recently Adobe has proposed a new spline \cite{Yan:2017:KCI:3072959.3073692} which solves the robustness problem, but at the cost of lumpier results than the Euler spiral spline. The main theme of this paper is to explore the obvious question of whether it is somehow possible to combine the best features of both.

\subsection{What is the best spline?}

It would be nice to design a perfect spline, but such a thing is not possible. Instead, there are a number of properties of splines, and it is possible to optimize for a subset. A fairly thorough listing of such properties is given in Chapter 2 of my thesis \cite{Lev09}. In some cases, these properties are in direct conflict with each other.

Take \emph{roundness} for example, the property that three points in a cyclic configuration will form the unique circle through those three points (and, more generally, the tendency of the spline to generate circular arcs). This seems like one of the most basic and desirable properties of a spline, but a round spline sacrifices \emph{boundedness} and \emph{robustness,} as can be seen when the points pass through a co-linear configuration; the circle goes through an infinite radius.

Even so, listing desirable properties provides scant help in actually designing a spline. The space of all possible splines is vast -- for any sequence of control points, it produces a curve (an infinite-dimensional class of objects) constrained only by the requirement to interpolate the control points. Is there some discipline that can reduce this space to something tractable to reason about, preserving the ``good'' splines? I argue below that the \emph{two parameter paradigm} is exactly such a discipline.

\section{Two parameter splines}

Chapter 4 of my thesis \cite{Lev09} proposed a framework of \emph{two parameter splines.} In this framework a family of curves to be instantiated at each segment between to control points, parametrized by the tangent angles at each endpoint relative to the chord. The segment is rotated and scaled in place so the endpoints of the segment match the control points.

In this framework, the tangents at each control point are free parameters (there is only one tangent for each control point, as G1 continuity is by construction). The spline then solves these parameters for G2 continuity at each control point.

Note that in this framework, the assignment of tangents depends only on the curvature at the endpoint of each segment, which in turn is a function of the two parameters. In this way, it \emph{decouples} the shape of the segments from the global behavior of the spline. Two curve families with the same \emph{curvature map} will be assigned the same tangents even if the interior shapes differ.

The loose coupling is emphasized even more if there is a four-parameter curve family that can fit arbitrary tangent and curvature constraints at both endpoints. With such a family, any curvature map can be accommodated.

The most familiar example of such a four-parameter family is a cubic B{\'e}zier. Other examples include 4th order polynomial spirals (Spiro curves), or quintic B{\'e}zier curves constrained to have nearly constant velocity.

\subsection{Existing 2-parameter splines}

A number of existing splines into the literature fit into the 2-parameter framework. In many cases, such splines are optimized for some property, providing evidence for the proposition that by choosing a 2-parameter curve family based on some property, the spline is globally optimized for this property.

Perhaps the most striking example is the Minimum Energy Curve (MEC). Defining a 2-parameter curve family minimizing the bending energy of a thin strip of flexible material passing through the endpoints with the given tangent constraints, the resulting spline globally minimizes bending energy for all curves passing through the control points.

The MEC has a number of drawbacks; most notably it tends to produce curves of lower tension (longer length) than intuitively desirable, as bending energy scales inversely with curve length. Of the approaches to increase tension, perhaps the most effective is the Euler spiral spline, which fits segments of the Euler spiral, or, equivalently, defines curve segments as having curvature linear in arclength. The resulting splines are quite pleasing and have a number of desirable properties, but also a significant drawback: at large bending angles, there is not a unique solution, and a solver tends to ``flip'' suddenly from one configuration to another, even for small perturbations of the control points. This is likely a major reason Euler spliral splines have not been adopted more widely (certainly as argued by Yan et al \cite{Yan:2017:KCI:3072959.3073692}).

The curvature map of the Euler spiral is shown in Figure \ref{euler_kmap}. For this region of the parameter space (angles between $-\pi/2$ and $\pi/2$), it is very smooth, almost linear, and curvature is bounded.

\begin{figure}
\centering
\includegraphics[scale=0.5]{euler_kmap}
\caption{Curvature map of Euler spiral}
\label{euler_kmap}
\end{figure}

Both MEC and Euler spiral splines are \emph{extensional,} meaning that refining a spline by placing an additional control point on the curve does not change the spline. As my thesis explains, extensional 2-parameter splines are quite restricted; all curves must be sections of a fixed generating curve.

Another spline that fits into the two-parameter framework is the \emph{circle spline} of S{\'e}quin et al \cite{DBLP:journals/cad/SequinLY05}. In this spline, the curvature at an endpoint is fixed as that of the unique circle passing through the tangent constraint at that endpoint and intersecting the other endpoint. Circle splines have excellent \emph{locality} but fails extensionality quite dramatically (producing particularly lumpy results when placing a new point near an inflection).

\subsection{Analysis of \kcurve{}s in 2-parameter framework}

With a bit of insight, it can be seen that the 2-parameter framework is general enough to include the \kcurve{} spline. In this analysis, the curve family consists of two parabola segments, with the vertex of each parabola at the endpoint, and the (absolute) curvature matched at the join point. This can be called a ``bi-parabola'' in analogy to the ``biarc'' as used in the Ikarus system \cite{Karow87}. Solving for G2 continuity at control points produces a single parabola segment that spans both sides of each control point (there is a unique parabola of given tangent and curvature at the vertex). This is true even if the interiors of segments in the curve family are not G2 continuous, as is the case for in the pseudo-inflectional case where the endpoints have curvature of opposite sign.

The construction of a bi-parabola from tangent constraints is shown in Figure \ref{biparabola}. For any pair of tangent constraints, there is a unique pair of parabolas with vertex at the endpoints, going through the tangents, and meeting at a point of equal curvature magnitude.

\begin{figure}
\centering
\includegraphics[scale=0.5]{biparabola}
\includegraphics[scale=0.5]{biparabola2}
\caption{Construction of the bi-parabola from tangent constraints}
\label{biparabola}
\end{figure}

This spline has strikingly different properties than the others listed above. It is bounded and is excellently robust. However, it deviates considerably from roundness, and cannot form smooth inflections.

Looking at the curvature map of the bi-parabola curve family reveals a number of fascinating properties. Most notably, the curvature map is periodic in both endpoint tangents, both with period $\pi$. Near a tangent angle of $\pi/2$, the curve forms a \emph{cusp} of infinite curvature, and a reversal of direction. One way or another, the curve has to reverse direction as the tangent goes through its full rotation; with bounded curvature it must flip, so going smoothly through a cusp seems the only other option.

\begin{figure}
\centering
\includegraphics[scale=0.5]{biparab_kmap}
\caption{Curvature map of bi-parabola}
\label{biparab_kmap}
\end{figure}

The \kcurve{} is not extensible, and deviation from roundness is high. It is not monotone curvature, but preserves half of that property - no curvature \emph{maxima} are allowed, while curvature minima are not only allowed but mandatory in the non-inflectional case.

\subsection{Variations on the 2-parameter framework}

Not all interesting splines fall squarely into the 2-parameter paradigm. This section touches on a number of variations.

Perhaps one of the most relevant is the Hobby spline \cite{Hobby85}, which can be seen as an approximation of the Euler spiral spline, particularly at small angles. It is based on a 2-parameter family of curves (at least, ignorning the optional tension parameter) but does not use G2 continuity to determine the endpoints. Instead, it enforces continuity of ``mock curvature,'' a linear approximation of curvature. It has the advantages of robustness and boundedness, at the cost of roundness and extensibility. It is used as the basis for curve design in Apple's iWork products.

Likewise, Ikarus \cite{Karow87} uses a 2-parameter family of curves (biarcs) but uses another scheme to determine the tangent parameters.

A 4-parameter scheme produces splines with G4 continuity. Existing 4-parameter splines include Minimum Variation Curvature \cite{Moreton92} and a 4th order polynomial spiral spline (Chapter 7 of my thesis).

\section{A new spline}

This paper proposes a new spline, fairly simple and easy to compute, that combines the robustness of the \kcurve{} spline with the fine qualities of the Euler spiral spline when the control polygon has low bending angles. In fact, it approximates Euler spiral fairly closely as angles decrease.

First, the formula for the spline is given. As in the Hobby spline, the two-parameter curve family is a single cubic B{\'ezier} segment between each pair of control points. The tangents are given by the parameters.

A cubic B{\'e}zier has two internal control points. Both must lie on the line tangent to the corresponding endpoint, in order to enforce the given tangent constraint. That leaves merely the length the control arm (the signed distance from the endpoint to the control point along the tangent vector) as a free parameter. The new spline uses this simple function for the left control arm length, as a function of $\theta_0$ and $\theta_1$; the right arm is symmetrical:

\[
\begin{aligned}
\mbox{arm length} & = \tfrac{5}{12}(\sin \theta_a - 0.2\sin 3\theta_a), \quad \mbox{\emph{where}} \\
\theta_a & = \theta_0 - 0.3 \sin (2\theta_1 - 0.4\sin 2\theta_1) \\
\end{aligned}
\]

Then, given this two-parameter curve, use a global solver to choose tangents so that the curvature is matched on both sides of each control point. A Newton-style solver as in my thesis will work, but requires a bit more care to make sure that both direction and curvature match, given the possibility of reversals. It also has to cope with curvature being unbounded, as opposed to the bounded curvature of previously published two parameter splines.

For small angles, this spline is quite similar to the Hobby spline; in fact, both can be seen as small-angle approximations to the Euler spiral. But as angles increase, instead of less accurate and more ``globby'' approximations to the Euler spiral shape, the length of the curve creases, and ultimately forms an infinite curvature cusp; beyond that angle, the direction reverses. Its curvature map is periodic (with period $\pi$ in both tangent angles), and grossly similar in this way to the bi-parabola, and so we expect the spline to have similar robustness as the \kcurve spline.

Because it approximates the Euler spiral, the new curve easily handles inflection points, with smooth variation of curvature on either side of the inflection. The new spline has three important elements, which together make it have the desired properties:

\begin{itemize}

\item It is a two-parameter spline with G2 continuity at the control points.

\item The curvature map is periodic, with cusps formed at high angles.

\item The curve family smoothly transitions through zero-curvature inflection points.

\end{itemize}

The curvature map is shown in Figure \ref{newcurve_kmap}.

\begin{figure}
\centering
\includegraphics[scale=0.5]{newcurve_kmap}
\caption{Curvature map of new spline curve family}
\label{newcurve_kmap}
\end{figure}

Three families of curves are shown in the following figure, the Euler spiral in red, the bi-parabola in green, and the new curve in blue. The horizontal axis is the left tangent angle, and vertical is the right tangent angle, both in 15 degree increments.

\begin{figure}
\includegraphics[scale=0.7]{biparab_euler_mycurve}
\caption{Bi-parabola is green, Euler spiral is red, new curve is blue}
\end{figure}

\subsection{Endpoint tangents}

In a cyclic configuration, enforcing G2 continuity across control points discharges all parameters. However, in an open curve, the tangents at the endpoints are unconstrained. These could be explicitly set (for example, in a graphical user interface), but failing that, the spline can set values that make the resulting curve look natural.

It is easiest to set the tangent at the endpoint as a function of the tangent (relative to chord) of the next control point. For example, setting it equal (as in the Euler spiral) sets the terminal segments to circular arcs (as the Euler spiral spline is round and thus generates circular arcs for symmetric tangent constraints). The Minimum Energy Curve sets the endpoint tangents to solve for zero curvature, as this minimizes total bending energy.

A $\pi$-periodic spline has an additional consideration, if it is to be robust -- when the next-to-terminal tangent is $\pi/2$, the terminal tangent should be $0$. This allows the curve to pass smoothly through $\pi/2$ to the other side without a visible ``jump'' to the curve.

The \kcurve{} spline sets the endpoint tangent to $\arctan{2\tan \theta} - \theta$, which solves for a single parabola. We find that $0.5\sin 2\theta$ produces nice results, a bit more curvature than a parabola but less than a circular arc, approximating a circular arc at low angles, and going to $0$ when the opposing tangent is $\pi/2$ for smoothness. Others may prefer a different formula, however.

\subsection{Comparison with \kcurve{}}

Figures \ref{spiral} and \ref{cshape} show a comparison of the new spline with \kcurve{} splines. The first is a simple series of points arranged in a spiral, and shows that the new spline is a much smoother interpolation. The \kcurve{} shows the underlying control polygon too strongly, with prominent curvature maxima.

Perhaps more worrying, Figure \ref{cshape} shows additional pseudo-inflection points added by \kcurve{} for a straightforward ``C'' shape as might be commonly drawn in a font. These curvature reversals are caused by the fact the bi-parabola is not only incapable of forming a true inflection, it also does not have in its repertoire a curve that approaches inflection, with moderate curvature on one end and much lower curvature on the other. The Euler spiral very easily attains such a curve, and the new spline has very similar behavior.

\begin{figure}
\centering
\includegraphics[scale=0.5]{biparab_mine_spiral}
\caption{New spline on the left{}, \kcurve{} on the right, spiral figure}
\label{spiral}
\end{figure}

\begin{figure}
\centering
\includegraphics[scale=0.5]{biparab_mine_c}
\caption{New spline on the left{}, \kcurve{} on the right, C shape}
\label{cshape}
\end{figure}

The new spline is not curvature monotonic in the general case. Neither does it exactly respect the property (held by \kcurve) of no internal curvature maxima. However, deviation from this principle is small, and it is easy to imagine a refined version, probably based on more than a single cubic B{\'e}zier per segment, that does.

For inputs containing no large bending angles, the new spline produces results that approximate the Euler spiral, and for this limited class of inputs, the spline approximately has its properties, including roundness, extensionality, and monotone curvature. A refined version of the spline might exactly match the Euler spiral within a certain parameter envelope.

\section{Acknowledgments}

Thanks to Jacob Rus and Carlo S{\'e}quin for discussions providing much insight into the analysis and design of splines.

\section{Bibliography}

\bibliography{paper}{}
\bibliographystyle{plain}

\end{document}


%%!TEX TS-program = latex
\documentclass[11pt]{article} %DIF >
\usepackage{etex}
\usepackage[utf8]{inputenc}
\input ../AuxFiles/PreambleNotes.tex

\begin{document}
\onehalfspace

\vspace*{\fill}
\begingroup
\centering

\Large {\scshape Introduction to Statistics}\\

(Lectures 5-6)

\endgroup
\vspace*{\fill}

\newpage

\section{A Decision-Theoretic Approach to Statistics}
The following two lectures (Lecture 5-6) provide an introduction to Statistics from a \emph{decision-theoretic} perspective, as in the foundational work of \cite{Wald50}. The decision-theoretic approach to statistics presents problems such as prediction, estimation, testing, and inference, as decision problems under uncertainty. There is emphasis on how an agent evaluates a set of decisions that can be taken. Statistical models play a central role in the analysis, because they provide the link that explains how the observed data can be useful for the decision maker. You can find interesting introductions to  statistical inference from a decision-theoretic perspective in Chapter 1 of \cite{Lehman05} and Chapter 1 of \cite{Ferguson67}, p. 1-11. Section 2.5 of \cite{leCam2000} presents very interesting, but brief, historical remarks.

{\scshape Overview of Lectures 5-6:} The objective of Lecture 5 and 6 is to present a framework to think about statistics from a decision-theoretic perspective.  We will define the following objects:

\begin{enumerate}[a)]
\item $x$: Data
\item $\{P_{\theta}\}_{\theta \in \Theta}$: Statistical Model
\item $\Theta$: Parameter space
\item $\mathcal{L}(a,\theta)$: Payoff or loss  of an action.
\item $d(x)$: Decision rule
\item $\expec_{\theta}[L(d(x),\theta)]$: Expected payoff /loss of an strategy conditional on $\theta$.
\end{enumerate}

\subsection{Data and Statistical model}

Let $X$ be a random variable taking values in some set $\mathcal{X}$. A realization of $X$ will be refered to as \underline{data}. Thus, data, are nothing else than realizations of some random variable. \\

To conduct statistical analysis we will require some structure on the process that generated the data. Such structure is described by a statistical model:\\

\noindent \textbf{Definition 1:}A \underline{statistical model} for the data is a collection of probability distributions over $\mathcal{X}$:
\[ \{\mathbb{P}_{\theta}\}_{\theta \in \Theta} \]
 indexed by an element $\theta$ in some space $\Theta$. The index $\theta$ is referred to as \underline{parameter} and the set $\Theta$ as the \underline{parameter space}.

 A statistical model is also often called a \emph{statistical experiment} or simply an experiment (see \cite{leCam2000} Chapter 2).\footnote{\cite{leCam2000} seems to credit \cite{Blackwell1953} for the definition of experiment.} The notion of statistical model plays a key role in econometrics. Concepts such as ``identification’’ and ``sufficient statistics’’ are formally defined in terms of a statistical model. You will have a chance to appreciate this point in the problem set (Problems 1 and 2).

\subsection{Statistical (Decision) Problems}

It is often helpful to define a \underline{statistical decision problem} as a game involving two players (``nature'' and the statistician) and two stages (data generation and decision making).  The extensive form description of the game is as follows.

In the first stage, nature selects a parameter $\theta \in \Theta$ and uses it to generate data according to the distribution $\mathbb{P}_{\theta}$. In the second stage, the econometrician/statistician/decision maker observes the data, but does not observe the parameter selected by nature. The decision maker is not under a full veil of ignorance as he/she observes the data.

 Based on the realized data, the econometrician would like to take an \emph{action} ``$a$'' whose payoff depends on the parameter selected by nature. This means that some actions are reasonable in some states but not in others. This is modeled by endowing the decision maker with a state-contingent utility or loss:
$$u(a,\theta) \quad \text{or} \quad \mathcal{L}(a,\theta).$$
The action is assumed to live in an action space $\mathcal{A}$. The \emph{decision problem} faced by the econometrician is the selection of an action depending on the realization of the data.

This leads to the following definition:\\

\noindent {\textbf{Definiton 2}: } A \underline{statistical decision problem} is a tuple:
\[(\Theta, A, u, \{P_{\theta}\})\]
containing a parameter space, an action space, a utility (or loss) function, and a statistical model. See Section 1.3 of Chapter 1 in \cite{Ferguson67}.\\

\subsection{Decision Rules}

In a statistical game (as in any other game) the objective of the players is to think about reasonable strategies that can be played. In the (extensive form) statistical game the information sets for the statistician are the data realizations. Thus,\\

\noindent \textbf{Definition 3:} A strategy of \underline{decision rule} $d$ for the statistician in a statistical game is a function $d: \mathcal{X} \rightarrow A$.  \\

Here are some examples of statistical problems and decision rules:

\begin{enumerate}
\item \underline{Estimation Problem}: The action space for the econometrician is $\Theta$. This means that after observing a data realization, the econometrician needs to decide what is the parameter $\theta$ that generated the data.

The decision rules for this problem are usually called \textbf{estimators}.

A typical loss for this problem is \textbf{quadratic loss}: $\mathcal{L}(a,\theta)=(a-\theta)^2$.

\item \underline{Testing Problem}: The parameter space is partitioned into two sets: $\Theta_0$ is called the null hypothesis and $\Theta_1$ is the alternative hypothesis. The action space of the econometrician contains only two elements $\{a_0, a_1\}$ ($a_0$ is interpreted as expressing support for the null and $a_1$ as expressing support for the alternative).

Decision rules for this problem are usually called \textbf{tests}.

A typical loss for this problem is the so-called \textbf{0-1 loss}: $\mathcal{L}(a_1,\theta_0)=1$ and $\mathcal{L}(a_0,\theta_1)=1$ (with the loss being 0 otherwise).

\item \underline{Inference problem:} The action for the econometrician consists of subsets of the parameter space. The interpretation is that each subset contains the best candidate values for $\theta$.

Decision rules for this problem are usually called \textbf{confidence sets}.

A typical loss for this problem is:

\[ \mathcal{L}(C;\theta)\equiv\delta \mathbf{1}\{ \theta \notin C \} + (1-\delta) \textrm{Vol}(C).  \]

\end{enumerate}

\subsection{``Optimal’’ Decision Rules}

\cite{Ferguson67} Chapter 1, p. 1:

\begin{quote}
\emph{``The fundamental problem of decision theory can be stated quite simply. Given a game $(\Theta,A,u,\{P_{\theta}\})$ and a random observable $X$ whose distribution depends on $\theta \in \Theta$, what decision rule $\delta$ should the statistician use?’’}
\end{quote}

For each action taken, the utility or loss function is deterministic. However, when the statistician reports a decision rule the loss associated to a particular decision, the payoff is a random variable: each action will lead to a different value of the loss function, depending on the data realization.

One way to summarize the performance of a decision rule in different points of the parameter space is by reporting the \emph{risk function}: \\

\noindent \textbf{Definition 4:} (Ferguson p. 7 ) The \underline{risk function} of a decision rule $d$ is defined as:
\[ R(\theta; d) = \mathbb{E}_{P_{\theta}}[\mathcal{L}(d(X),\theta)]. \]

\noindent Another cite from \cite{Ferguson67} p. 7:

\begin{quote}
\emph{``It is a natural reaction to search for a best decision rule, a rule that has the smallest risk no matter what the true state of nature is. Unfortunately, situations in which the best decision rule exists are rare and uninteresting.’’}
\end{quote}

\noindent While a rule that has the smallest risk need not exist, it is simple to use a dominance criterion to discard bad decision rules. This is analogous to the idea of discarding dominated strategies in the analysis of games. \\

\noindent \textbf{Definition 5:} A decision rule $d$ is \underline{dominated} if there is a decision rule $d’$ such that:
\[R(\theta;d’) \leq R(\theta;d)\]
for every $\theta \in \Theta$, with strict inequality for some $\theta$. Decision rules that are not dominated are called \emph{admissible}.  See Chapter 2 p. 54 in \cite{Ferguson67} for an equivalent definition.

Not all decision rules can be compared using the dominance criterion (dominance is a partial order). There are two general methods for creating a complete ordering over decision rules: ``average’’ and ``worst-case’’ risk . These criteria lead to decision rules that can be defined generally for any statistical problem.

\subsubsection{Bayes Rules}

Given a probability distribution $\pi$ on $\Theta$ (a \underline{prior}) we define the the \underline{Bayes risk} of a decision rule $d$ as:
\[ r(\pi, d) \equiv \int_{\Theta} R(\theta, d) d \pi (\theta) \equiv \mathbb{E}_{\pi} [ R(\theta; d) ]. \]

\noindent \textbf{Definition 6:} A decision rule $d^*$ is said to be a \underline{Bayes Rule} with respect to the prior distribution $\pi$ (and relative to a class of decision rules $D$) if:

\[ r(\pi, d^*) = \inf_{d} r(\pi, d). \]

\noindent That is, if it minimizes Bayes risk. See Chapter 2 p. 31 in \cite{Ferguson67}.

By construction, Bayes rules require the specification of prior. Consequently, the properties of a Bayes decision rule will depend on the choice of such prior. However, the are some limits on how bad a Bayes rule can behave. The following result shows that under some minimal assumptions, Bayes rules are admissible; hence, it is not possible to improve upon them uniformly over $\Theta$. \\

\noindent \textbf{Result}: Suppose that the risk function $R(d; \cdot)$ is continuous in $\theta$ for any decision rule $d$. Let $\pi$ be any prior with full support on $\Theta$.\footnote{A point $\theta$ is said to be on the support of a distribution $\pi$ is for every neighborhood $\mathcal{N}_{\theta}$ of $\theta$ have strictly positive probability. The distribution $\pi$ has full support if every $\theta \in \Theta$ is in the support of the distribution.} The Bayes rule $d^*$ corresponding to $\pi$ is admissible. \\

\noindent {\scshape Proof:} Suppose $d^*$ is not admissible. Then there exists $d^{\prime}$ such that:
\[ R(d^{\prime}; \theta) \leq R(d^*; \theta),  \]
with strict inequality for some $\theta^* \in \Theta$. Since $R(d^*,\theta)$ is continuous in $\theta$, there exists a neighborhood $N_\theta^*$ such that
\[  R(d^{\prime}, \theta) < R(d^*, \theta) \quad \forall \quad \theta \in N_{\theta^*}. \]

\noindent Since $\pi$ has full support
\[ \int_{N_{\theta^*}} R(d’;\theta) d \pi(\theta) < \int_{N_{\theta^*}} R(d’;\theta^*) d \pi(\theta).   \]
Consequently:
\[ r(\pi, d^{\prime}) < r(\pi, d^*).   \]

\noindent A contradiction. \\

\begin{flushright}
$\square$
\end{flushright}

\noindent Thus, Bayes Rules are admissible under mild regularity conditions. Interestingly, a converse of this result is also true: any admissible decision rule is, with some qualifications,	 Bayes for some prior. This result is known as the \emph{Complete Class Theorem}; see Chapter 2 (in particular 2.10) in \cite{Ferguson67}, if interested.

Bayes rules are not a statistical panacea. A poorly selected prior (one that ignores certain parts of the parameter space) leads to a Bayes rule that can be strictly improved.

Bayes Rules are defined very generally, but minimizing Bayes risk is a conceptually complicated problem: we are optimizing over a space of functions (the decision rules). We now show that it is possible to simplify the optimization problem, by distinguishing between prior and posterior information.

Let $f(x;\theta)$ denote the p.d.f corresponding to $\mathbb{P}_{\theta}$ and assume that $\pi(\theta)$ is also a p.d.f. Note that:
\begin{eqnarray*}
r(\pi, d) &\equiv& \int_{\Theta} R(d(x); \theta ) \pi(\theta) d{\theta}, \\
&=&  \int_{\Theta}\Big( \int_{\mathcal{X}} \mathcal{L}(d(x); \theta) f(x| \theta) d x  \Big)  \pi(\theta) d{\theta}, \\
&& (\textrm{by definition of Risk}) \\
&=& \int_{\mathcal{X}}   \Big( \int_{\Theta} \mathcal{L}(d(x); \theta) f(x| \theta) \pi(\theta) d \theta  \Big)   dx, \\
&=& \int_{\mathcal{X}}   \Big( \int_{\Theta} \mathcal{L}(d(x); \theta)\pi (\theta | x) d \theta  \Big) f^*(x)  dx.
\end{eqnarray*}

\noindent where $f^*(x)= \int_{\theta} f(x|\theta) \pi(\theta) d \theta$ is the marginal density of the data in a model with joint p.d.f. given by $f(x|\theta) \pi(\theta)$. So, minimizing (ex-ante) Bayes Risk is the same as choosing the action $d(x) \in \mathcal{A}$ that minimizes:

\[ \int_{\Theta} \mathcal{L}(d(x); \theta)\pi (\theta | x) d \theta.  \]

\noindent The latter quantity is referred to as \underline{posterior loss}. Minimizing posterior loss is an easier problem, as action spaces are typically subsets of $\mathbb{R}^{n}$ not spaces of functions. In the next lectures, we will work out a few examples to develop a better understanding of this result.

\subsubsection{Minimax rules}

An essentially different type of ordering of the decision rules may be obtained by comparing the rules according to the worst that could happen to the statistician. In other words, $d_1$ is weakly preferred to $d_2$ if

\[ \sup_{\theta} R(\theta, d_1) \leq \sup_{\theta} R(\theta, d_2). \]

\noindent \textbf{Definition 7:} A decision rule $d_0$ is said to be minimax (relative to a class $D$ of decisions) if:

\[ \sup_{\theta \in \Theta} R(\theta, d_0) = \inf_{d \in D} \sup_{\theta \in \Theta} R(\theta, d). \]

See Definition 3, Chapter 1 in \cite{Ferguson67}. The minimax rule is designed to protect the statistician against worst-case situations: the rule is chosen to minimize the worst-case risk (or to obtain the best performance in the worst possible situation).

One of the reasons minimax decision rules are appealing is that their construction does not require the user to specify any prior belief. This is an attractive property, but it comes at the cost of making the reasonability of minimax decision rules harder to justify. In general, there is no guarantee that minimax rules are admissible. We will not show it, but under some conditions \emph{minimax rules are Bayes}; that is, there exists a prior distribution $\pi_0$ that makes the minimax rule $d_0$ a Bayes rule.

A further connection between Bayes and minimax rules is as follows. \\

\noindent \textbf{Result:} Suppose that $d^*$ is a Bayes rule with constant risk; that is $R(\theta, d^*)= C$ for some $C \in \mathbb{R}$. Then $d^*$ is minimax. \\

\noindent \emph{Proof:} If $\pi$ has constant risk for any prior $\pi$
\[ \sup_{\theta \in \Theta} R(\theta, d^*) = \int_{\Theta} R(\theta, d^*) d \pi(\theta)   \]
 In particular, if $d^*$ is Bayes for $\pi^*$, for any $d$
 \[  \int_{\Theta} R(\theta, d^*) d \pi^*(\theta) \leq \int_{\Theta} R(\theta, d) d \pi^*(\theta),    \]
by definition. However, for any decision rule $d$
 \[  \int_{\Theta} R(\theta, d) d \pi^*(\theta) \leq \int_{\Theta} \sup_{\theta \in \Theta} R(\theta, d) d \pi^*(\theta) = \sup_{\theta \in \Theta} R(\theta,d).    \]
 We conclude that for any $d$,
 \[\sup_{\theta \in \Theta} R(\theta, d^*) \leq  \sup_{\theta \in \Theta} R(\theta,d).  \]
Implying $d^*$ is minimax.
%See exercise 1-2, p. 59

%A bit more formally, a distribution $\tau_0$ is said to be least favorable if:

%\[ \inf_{\delta} r(\tau_0,\delta) = \sup_{\tau} \inf_{\delta} r(\tau,\delta) \]

\subsection{Least Favorable Distribution}
In the previous section we defined a minimax rule as the decision rule that protects the statistician against worst-case situations. Thinking back about the game-based definition of a statistical decision problem, it is possible to also define a worst-case strategy for nature.

Let $\Theta^*$ denote the space of all probability distributions over $\Theta$. \\

\noindent \textbf{Definition 7:} A distribution $\pi_0 \in \Theta^*$ is said to be least favorable if
\[ \inf_{d \in \mathcal{D}} r(\pi_0, d) = \sup_{\pi \in \Theta^*} \inf_{d \in \mathcal{D}} r(\pi, d).  \]

As explained by \cite{Ferguson67}, \emph{``the name `least favorable' derives from the fact that if the statistician were told which prior distribution nature was using he would like least to be told a distribution $\pi_0$'' } as defined above.

In some decision problems it is possible to show that the \emph{minimax theorem} holds:
\begin{equation}
\sup_{\pi \in \Theta^*} \inf_{d \in \mathcal{D}} r(\pi, d)  = \inf_{d \in D} \sup_{\theta \in \Theta} R(\theta, d).
\end{equation}

If the minimax theorem holds, and a least favorable distribution $\pi_0$ exists, then any minimax rule $d_0$ is Bayes with respect to $\pi_0$.

\newpage

\bibliographystyle{../AuxFiles/ecta}
\bibliography{../AuxFiles/BibMaster}

\end{document}


%!TEX TS-program = lualatex
%!TEX encoding = UTF-8 Unicode

\documentclass[12pt]{exam}

%\printanswers

\usepackage{fontspec}
\setmainfont[Ligatures={TeX}, BoldFont={* Bold}, ItalicFont={* Italic}, BoldItalicFont={* BoldItalic}, Numbers={Monospaced, Lining}]{Linux Libertine O}
\setsansfont[Scale=MatchLowercase,Ligatures=TeX]{Linux Biolinum O}
\setmonofont[Scale=MatchLowercase]{Inconsolatazi4}
\newfontfamily{\tablenumbers}[Numbers={Monospaced,Lining}]{Linux Libertine O}
\usepackage{microtype}

\usepackage{amsmath}

\usepackage{unicode-math}
\setmathfont[Scale=MatchLowercase]{TeX Gyre Termes Math}

\usepackage{geometry}
\geometry{letterpaper, left=1.5in, bottom=1in}
%\geometry{landscape}                % Activate for for rotated page geometry
\usepackage[parfill]{parskip}    % Activate to begin paragraphs with an empty line rather than an indent

\usepackage{longtable}

%\usepackage{siunitx}
\usepackage{booktabs}
\usepackage{array}
\newcolumntype{L}[1]{>{\raggedright\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{C}[1]{>{\centering\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{R}[1]{>{\raggedleft\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}

\usepackage{enumitem}
\setlist{leftmargin=*}
\setlist[1]{labelindent=\parindent}
\setlist[enumerate]{label=\textsc{\alph*}.}
\setlist[itemize]{label=\color{gray}\textbullet}
%\usepackage{hyperref}
%\usepackage{placeins} %PRovides \FloatBarrier to flush all floats before a certain point.
%\usepackage{hanging}

\usepackage[sc]{titlesec}

%% Commands for Exam class
\renewcommand{\solutiontitle}{\noindent}
\unframedsolutions
\SolutionEmphasis{\bfseries}

\pagestyle{headandfoot}
\firstpageheader{\textsc{bi}\,063 Evolution and Ecology}{}{\ifprintanswers\textbf{KEY}\else Name: \enspace \makebox[2.5in]{\hrulefill}\fi}
\runningheader{}{}{\footnotesize{pg. \thepage}}
\footer{}{}{}
\runningheadrule

\begin{document}

\subsection*{Population growth practice problems}

Here are ten practice problems for population growth, five for exponential growth and five for logistic growth.  Fill in the blanks as provided.  Answers are on the second page.

$N_1$ means the new population size, after you have calculated $\Delta N$. Assume $\Delta t$ to be one generation. I did not round any other values for the calculations.

I rounded all values of $N_1$ to whole individuals.

\begin{tabular}[l]{@{}R{1in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}@{}}
\toprule
Exponential	& 1	& 2	& 3	& 4	& 5 \\
\midrule
$b$	& 0.06	& 0.016	& 0.16	& 0.68	& 0.38 \\
$d$	& 0.09	& 0.009	& 0.69	& 0.66	& 0.52 \\
$N$	& 983	& 1058	& 526	& 2068	& 195 \\
& 	& 	& 	& 	&  \\
$r$	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt} \\
& 	& 	& 	& 	&  \\
$\Delta N$	&\rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt} \\
& 	& 	& 	& 	&  \\
$N_1$	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt} \\
\bottomrule
& 	& 	& 	& 	&  \\
& 	& 	& 	& 	&  \\
\toprule
Logistic	& 1	& 2	& 3	& 4	& 5 \\
\midrule
$b$	& 0.06	& 0.016	& 0.16	& 0.68	& 0.38 \\
$d$	& 0.09	& 0.009	& 0.69	& 0.66	& 0.52 \\
$N$	& 983	& 1058	& 526	& 2068	& 195 \\
$K$	& 1000	& 1000	& 750	& 3000	& 250 \\
& 	& 	& 	& 	&  \\
$\Delta N$	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt} \\
& 	& 	& 	& 	&  \\
$N_1$	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt}	& \rule{0.5in}{0.4pt} \\
\bottomrule
\end{tabular}

\newpage

\subsubsection*{Answer Key}

\begin{longtable}[l]{@{}R{1in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}R{0.75in}@{}}
	\toprule
	Exponential	& 1	& 2	& 3	& 4	& 5 \tabularnewline
	\midrule
	$b$	& 0.06	& 0.016	& 0.16	& 0.68	& 0.38 \tabularnewline
	$d$	& 0.09	& 0.009	& 0.69	& 0.66	& 0.52 \tabularnewline
	$N$	& 983	& 1058	& 526	& 2068	& 195 \tabularnewline
	& 	& 	& 	& 	&  \tabularnewline
	$r$	& -0.03	& 0.007	& -0.53	& 0.02	& -0.14 \tabularnewline
	$\Delta N$	& -29.49	& 7.41	& -278.78	& 41.36	& -27.30 \tabularnewline
	$N_1$	& 954	& 1065	& 247	& 2109	& 168 \tabularnewline
	\bottomrule
	& 	& 	& 	& 	&  \tabularnewline
	& 	& 	& 	& 	&  \tabularnewline
	\toprule
	Logistic	& 1	& 2	& 3	& 4	& 5 \tabularnewline
	\midrule
	$b$	& 0.06	& 0.016	& 0.16	& 0.68	& 0.38 \tabularnewline
	$d$	& 0.09	& 0.009	& 0.69	& 0.66	& 0.52 \tabularnewline
	$N$	& 983	& 1058	& 526	& 2068	& 195 \tabularnewline
	$K$	& 1000	& 1000	& 750	& 3000	& 250 \tabularnewline
	& 	& 	& 	& 	&  \tabularnewline
	$\Delta N$	& -0.50	& -0.43	& -83.26	& 12.85	& -6.01 \tabularnewline
	$N_1$	& 983	& 1058	& 443	& 2081	& 189 \tabularnewline
	\bottomrule

\end{longtable}

	\bigskip

	$r$ is the same for both tables.

	\bigskip

	Exponential:
	\begin{equation*}
	\dfrac{\Delta N}{\Delta T} = rN
	\end{equation*}

	\bigskip

	Logistic:
	\begin{equation*}
	\dfrac{\Delta N}{\Delta T} = rN\left(\dfrac{K-N}{K}\right)
	\end{equation*}

\end{document}


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[english]{babel} % for hyphenation dictionary
%\setdefaultlanguage{english} % polyglossia command for use with XeTeX / LuaTeX
\usepackage[margin=1in]{geometry} % handle page geometry

% import packages and commands
\input{../header2.tex}

% title information
\title{Phys 221A -- Quantum Mechanics -- Lec20}
\author{UCLA, Fall 2014}
\date{\formatdate{10}{12}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed
         % format: formatdate{dd}{mm}{yyyy}

\begin{document}
\maketitle

\section{Even mooooore on angular momentum}

Recall that for the total angular momentum $\v J$ we have the commutation relation
\begin{eqn}
[J_i, J_j] = i \hbar \epsilon_{ijk} J_k,
\end{eqn}
which also applies individually to the orbital angular momentum $\v L$ and spin angular momentum $\v S$ (just take $\v J$ to be either one). Generally, it is useful to work in the descending $m$ basis, i.e.
\begin{eqn}
\sum_m c_m \ket{j,m} \rightarrow \pmat{c_j \\ c_{j-1} \\ \vdots \\ c_{-j+1} \\ c_{-j}}.
\end{eqn}
So using the eigenvalues of $\v J^2, J_z$ along with the ladder operators, we can construct the various matrices of matrix elements in this basis. We are familiar already with the typical case of $j = 1/2$ where
\begin{eqn}
\v{J} = \frac{\hbar}{2} \v \sigma = \set{ \frac{\hbar}{2} \pmat{0 & 1 \\ 1 & 0}, \frac{\hbar}{2} \pmat{0 & -i \\ i & 0}, \frac{\hbar}{2} \pmat{1 & 0 \\ 0 & -1} }.
\end{eqn}
More generally we have
\begin{eqn}
J_z = \hbar
\begin{pmatrix}
j & & & & \\
& j-1 & & & \\
& & \ddots & & \\
& & & -j+1 & \\
& & & & -j
\end{pmatrix}
\end{eqn}

Of course the matrix elements of $\v J^2$ are just the eigenvalues down the diagonal, but they're all the same so we have

[will finish writing this part later]

In general, we can make a choice of complete set of commuting operators $\v J^2, J_z, H$. This works because generally, physical Hamiltonians are isotropic in the full Hilbert space,
\begin{eqn}
[H, J_k] = 0, \qquad \text{all $k$}.
\end{eqn}
However, we could also choose individually $\v L^2, L_z, \v S^2, S_z$, but \emph{generally} we can only include $H$ in this set if we neglect relativistic effects. It does work, for example, in the Zeeman effect where we have Hamiltonian
\begin{eqn}
H = \frac{p^2}{2m} + U(r) - \gamma \v B \cdot \v S
\end{eqn}
which commutes with $\v S^2, S_z$.

\subsection{Orbital angular momentum and spherical coordinates}

Consider a single spinless particle which has Hilbert space $L_2$, so orbital angular momentum only. We are interested in central potentials
\begin{eqn}
H = \frac{p^2}{2m} + U(r).
\end{eqn}
We will use the commutation relations for $\v x, \v p$ and $\v L = \v x \times \v p$,
\begin{eqn}
[x_i, p_j] = i \hbar \delta_{ij}, \qquad
[L_i, L_j] = i \hbar \epsilon_{ijk} L_k
\end{eqn}
to derive properties of central potential systems. We are interested in symmetries under rotations $R(\phi \uv n) \in SO(3)$. Under such a rotation the wavefunction changes by
\begin{eqn}
\psi (\v r) \rightarrow \psi'(\v r) = \psi(R\inv \v r) = e^{-i\phi \uv n \cdot \v L / \hbar} \psi(\v r).
\end{eqn}
So, rotating about the $z$-axis by some small angle $\epsilon$, we have
\begin{eqn}
R\inv_z(\epsilon) = R_z(-\epsilon) = \pmat{1 & \epsilon & 0 \\ -\epsilon & 1 & 0 \\ 0 & 0 & 1}
\qquad \implies \qquad
R\inv_z \v r = \pmat{x + \epsilon y \\ y - \epsilon x \\ z}
\end{eqn}
so that
\begin{align}
\psi'(\v r) &\approx (1 - \frac{i \epsilon}{\hbar} L_z) \psi(\v r) \\
	&= (1 - \epsilon (x \partial_y - y \partial_x)) \psi(\v r) \\
	&= (1 - \epsilon \partial_\phi) \psi(r, \theta, \phi),
\end{align}
so we identify $L_z = -i\hbar \partial_\phi$. Furthermore, we have
\begin{eqn}
L_\pm = L_x \pm i L_y = -i \hbar e^{\pm i \phi} (\pm i \partial_\theta - \cot \theta \partial_\phi).
\end{eqn}
So working out the full angular momentum we have
\begin{align}
\v L^2 &= L_z^2 + \frac{1}{2} (L_+ L_- + L_- L_+) \\
	&= -\hbar^2 \left[ \frac{1}{s\in\theta} \partial_\theta (\sin\theta \partial_\theta) + \frac{1}{\sin^2 \theta} \partial_\phi^2 \right].
\end{align}
Notice that this is just the angular part of the Laplacian,
\begin{eqn}
\nabla^2 = \frac{1}{r^2} \left[ \partial_r (r^2 \partial_r) - \v L^2 / \hbar^2 \right].
\end{eqn}

\subsection{Spherical harmonics}

Now, since $\v L^2, L_z$ form a complete set of (linearly independent) commuting operators, we can factorize the wavefunction
\begin{eqn}
\psi(r,\theta,\phi) = f(r) Y_\ell^m (\theta,\phi).
\end{eqn}
This factorization is unique up to constants, and we call the functions $Y_\ell^m(\theta,\phi)$ with a particular normalization \emph{spherical harmonics}. Furthermore since $Y_\ell^m$ must be an eigenfunction of $L_z$, we have
\begin{eqn}
(\partial_\phi - im) Y_\ell^m = 0
\qquad \implies \qquad
Y_\ell^m(\theta,\phi) = e^{im\phi} y_\ell^m(\theta).
\end{eqn}
Finally, we can show that $L_+ Y_\ell^\ell = 0$ which gives us a solution for $Y_\ell^\ell$, and we can generate the rest using the ladder operators. [See Yaroslav's notes for more on spherical harmonics.]

\end{document}


\documentclass{memoir}
\usepackage{notestemplate}

% \begin{figure}[ht]
%     \centering
%     \incfig{riemmans-theorem}
%     \caption{Riemmans theorem}
%     \label{fig:riemmans-theorem}
% \end{figure}

\begin{document}
\chapter{Groups}
\label{cha:groups}

\section{Fundamentals of Groups}
\label{sec:fundamentals_of_group_theory}

\begin{defn}[Group]
	A \textbf{group} is a set \(G\) that has an associative operation with an identity \(e\) and inverses for all elements. If the operation is commutative, then we say \(G\) is \textbf{abelian}.
\end{defn}

\begin{defn}[Order]
	Let \(g \in G\). The \textbf{order of \(g\)}, denoted by \(o(g)\), is the smallest positive integer \(k\) satisfying \(g^{k}=e\). If there is no such \(k\), then we say that \(o(g) = \infty\).
\end{defn}
In the finite case, the powers of \(g\) are periodic, and so the order is the smallest period. \\

\end{document}

Abstract Algebra - Introductory/Algebra I/Notes/source/2020-03-24-IntroGroups.tex


\documentclass[11pt,a4paper,oneside,openright]{report}

\usepackage[bindingoffset=5mm,left=20mm,right=20mm,top=20mm,bottom=20mm,footskip=10mm]{geometry}
\usepackage[utf8x]{inputenc}
\usepackage{hyperref}
\usepackage[english]{babel}
\usepackage{listings}
\usepackage{subfiles}
\usepackage{longtable}
\usepackage{multirow}
\usepackage{ragged2e}
\usepackage{cmap} % avoid fi ligatures in pdf file
\usepackage{amsthm} % example numbering
\usepackage{color}
%\usepackage{bold-extra} % for bold tt font. Remember to include bold-extra.sty file
\usepackage{graphicx}
\usepackage[yyyymmdd]{datetime}
\usepackage{float}

% style for code listing
\renewcommand{\familydefault}{\sfdefault}
\newtheorem{example}{Example}[chapter]  % example numbering
\lstset{language=C}                     % formatting for code listing
\lstset{basicstyle=\ttfamily,breaklines=true}
\definecolor{darkGreen}{rgb}{0,0.4,0}
\definecolor{mybrown}{rgb}{0.40,0.10,0.05}
\lstset{commentstyle=\color{darkGreen}}  % comments color
\lstset{keywordstyle=\color{blue}}       % keyword color
\lstset{stringstyle=\color{mybrown}}     % string color
\lstset{showstringspaces=false}          % don't mark spaces in strings

\renewcommand{\dateseparator}{-}

% command for turning indent back on after \flushleft
\newcommand{\indenton}{\RaggedRight\parindent=15pt}

% command for vertical space
\newcommand{\vspacesmall}{\vspace{3mm}}
\newcommand{\vspacebig}{\vspace{6mm}}

% style for code inlined in text:
\newcommand{\codei}[1]{\bfseries \ttfamily{#1}\normalfont}

\begin{document}

\begin{titlepage}
    \centering

    \null %empty box needed for vfill to work
    \vfill

   {\bfseries\Huge
    Vector3d.h
    \vspacesmall

    3-dimensional vector extension for
    \vspacesmall

    C++ vector class library
    \vspacebig

   }
    \vspacebig

   {\Large
    Agner Fog
    \vspacebig

    \copyright\ \today. Apache license 2.0
   }

    \vfill

    \includegraphics[width=306pt]{freesoftwarelogo.jpg}
    \vfill

\end{titlepage}

\RaggedRight

\chapter{Introduction}\label{chap:Introduction}
3-dimensional vectors are useful in geometry and physics.
The file vector3d.h provides vector classes, operators, and functions for
calculations with 3-D vectors. This is an extension to the Vector Class Library.
\vspacesmall

The classes listed below are defined. Common operators and functions are defined for these classes:

\begin {table}[H]
\caption{3-D vector classes}
\label{table:Vector3DClasses}
\begin{tabular}{|p{24mm}|p{20mm}|p{20mm}|p{22mm}|p{20mm}|p{28mm}|}
\hline
\bfseries vector class & \bfseries Precision &  \bfseries 3-D vectors per instance & \bfseries Correspon-ding real vector class & \bfseries Total bits & \bfseries Recommended minimum \newline instruction set \\ \hline
Vec3Df  & \centering single & \centering  1 & \centering Vec4f & \centering 128 & SSE2 \\ \hline
Vec3Dd  & \centering double & \centering 1 & \centering Vec4d & \centering 256 & AVX \\ \hline
\end{tabular}
\end{table}
\vspacebig

\section{Compiling} \label{Compiling}
The 3-D vector class extension to the Vector Class Library is compiled in the same way as the Vector Class Library itself. All x86 and x86-64 platforms are supported, including Windows, Linux, and Mac OS.
The following C++ compilers can be used: Gnu, Clang, Microsoft, and Intel.
See the Vector Class Library manual for further details.
\vspacesmall

This example shows how to use the 3-D vector classes:

\begin{example}
\label{example1}
\end{example} % frame disappears if I put this after end lstlisting
\begin{lstlisting}[frame=single]
// Example for 3-D vectors
#include <stdio.h>
#include "vectorclass.h"  // vector class library
#include "vector3d.h"     // extension for 3-D vectors

// function to print 3-D vector:
template <typename V>
void printv3 (const char * text, V a) {
    auto aa = a.to_vector();  // get elements as real vector
    printf("\n%s ", text);    // print text
    printf("(%.3G,%.3G,%.3G)", aa[0], aa[1], aa[2]);
}

int main() {
    // define 3-D vectors
    Vec3Dd a(1,2,3);                // x = 1, y = 2, z = 3
    Vec3Dd b(4,5,6);                // x = 4, y = 5, z = 6
    Vec3Dd c = a + b;               // add vectors
    Vec3Dd d = cross_product(a, b); // x-product
    double e = dot_product(a, b);   // dot-product
    // print results
    printv3("a = ", a);             // a = (1,2,3)
    printv3("b = ", b);             // b = (4,5,6)
    printv3("c = ", c);             // c = (5,7,9)
    printv3("d = ", d);             // d = (-3,6,-3)
    printf ("\ne = %f", e);         // e = 32
}
\end{lstlisting}
\vspacesmall

\chapter{Constructing 3-D vectors and loading data}
\label{Constructing3Dvectors}

There are several ways to create 3-D vectors and put data into them. These methods are listed here.
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & default constructor \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & the 3-D vector is created but not initialized.\newline
The value is unpredictable \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a;    // creates a 3-D vector
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & Construct from x,y,z coordinates \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & The parameters define the x, y, and z coordinates \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);  // a = (1,2,3) (x = 1, y = 2, z = 3)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & member function load(p) \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & Load data from array of same precision. \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
float a[3] = {2,5,-1};
Vec3Df b;
b.load(a);  // b = (2,5,-1)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & member function store(p) \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & Save data into array of same precision \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
double a[3];
Vec3Dd b(4,0,3);
b.store(a);  // a = {4,0,3}
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & member function get\_x() \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & Get the x-coordinate \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
double b = a.get_x();  // b = 1
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & member function get\_y() \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & Get the y-coordinate \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
double b = a.get_y();  // b = 2
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & member function get\_z() \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & Get the z-coordinate \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
double b = a.get_z();  // b = 3
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Method & member function extract(i) \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & i = 0, 1, 2 give the x, y, or z-coordinate, respectively \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
double b = a.extract(2);  // b = 3
double c = a[2];          // b = 3 (the same)
\end{lstlisting}
\vspacebig

\chapter{Operators}\label{chap:Operators}

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Operator & + \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Add two vectors \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
Vec3Dd b(5,6,7);
Vec3Dd c = a + b;    // c = (6,8,10)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Operator & - \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Subtract two vectors \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(11,10,9);
Vec3Dd b(5,6,7);
Vec3Dd c = a - b;    // c = (6,4,2)
Vec3Dd d = - b;      // d = (-5,-6,-7)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Operator & * \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Multiply two vectors element by element, or one vector and one scalar of the same precision \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
Vec3Dd b(4,5,6);
Vec3Dd c = a * b;    // c = (4,10,18)
Vec3Dd d = a * 10.0; // d = (10,20,30)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Operator & / \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Divide a vector by a scalar of the same precision \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(10,20,30);
Vec3Dd b = a / 5.0; // b = (2,4,6)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Operator & == \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Compare for equality.\newline
The result is a boolean scalar. \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1, 2,3);
Vec3Dd b(1,-2,3);
bool   c = (a == b);  // c = false
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Operator & != \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Compare for not equal.\newline
The result is a boolean scalar. \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1, 2,3);
Vec3Dd b(1,-2,3);
bool   c = (a != b);  // c = true
\end{lstlisting}
\vspacebig

\chapter{Mathematical functions}\label{chap:MathematicalFunctions}

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & cross\_product \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Gives the X-product of two vectors \\ \hline
\bfseries Efficiency & medium \\ \hline
\bfseries Accuracy & Calculation of the X-product involves the calculation of sums of products. Loss of precision may occur if the result is close to zero. \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
Vec3Dd b(4,5,6);
Vec3Dd c = cross_product(a,b); // c = (-3,6,-3)
Vec3Dd d = cross_product(b,a); // d = (3,-6,3)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & dot\_product \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Gives the dot-product of two vectors. The result is a scalar \\ \hline
\bfseries Efficiency & medium \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
Vec3Dd b(4,5,6);
double c = dot_product(a,b); // c = 32
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & vector\_length \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Gives the length of the vector (Euclidian norm) \\ \hline
\bfseries Efficiency & medium \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(3,0,4);
double b = vector_length(a); // b = 5
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & normalize\_vector \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Divides the vector by its length to give a vector with the same direction and length one. \\ \hline
\bfseries Efficiency & medium \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(3,0,4);
Vec3Dd b = normalize_vector(a); // b = (0.6, 0.0, 0.8)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & rotate \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Rotates a vector by multiplying a 3x3 rotation matrix by the column vector. The first three parameters define the columns of the rotation matrix. The last parameter is the vector to rotate. \\ \hline
\bfseries Efficiency & medium \\ \hline
\bfseries Accuracy & Calculation of the rotated vector involves the calculation of sums of products. Loss of precision may occur if the result is close to zero. \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);    // vector to rotate
Vec3Dd c0(1,0,0);   // first column of matrix
Vec3Dd c1(0,0,-1);  // second column of matrix
Vec3Dd c2(0,1,0);   // third column of matrix
Vec3Dd d = rotate(c0,c1,c2,a); // d = (1,3,-2)
\end{lstlisting}
\vspacebig

\chapter{Other functions}\label{chap:OtherFunctions}

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & to\_vector \\ \hline
\bfseries Defined for & all 3-D vectors classes \\ \hline
\bfseries Description & Convert to a vector of class Vec4f or Vec4d. \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Df a(1,2,3);
Vec4f  b = a.to_vector(); // b = (1,2,3,0)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & select \\ \hline
\bfseries Defined for & all 3-D vectors classes  \\ \hline
\bfseries Description & Choose between two vectors. \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Df a(1,2,3);
Vec3Df b(4,5,6);
Vec3Df c = select(true,a,b);  // c = (1,2,3)
Vec3Df d = select(false,a,b); // d = (4,5,6)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & to\_float \\ \hline
\bfseries Defined for & Vec3Dd  \\ \hline
\bfseries Description & Convert to lower precision. The result is a Vec3Df \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Dd a(1,2,3);
Vec3Df b = to_float(a);// b = (1,2,3)
\end{lstlisting}
\vspacebig

\begin{tabular}{|p{25mm}|p{100mm}|}
\hline
\bfseries Function & to\_double \\ \hline
\bfseries Defined for & Vec3Df  \\ \hline
\bfseries Description & Convert to higher precision. The result is a Vec3Dd \\ \hline
\bfseries Efficiency & good \\ \hline
\end{tabular}
\vspacesmall

\begin{lstlisting}[frame=none]
// Example:
Vec3Df a(1,2,3);
Vec3Dd b = to_double(a);// b = (1,2,3)
\end{lstlisting}
\vspacebig

\end{document}


\documentclass[10pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[french]{babel}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{stmaryrd}

\usepackage{tikz}
\usetikzlibrary{arrows,automata}

\title{Distributed systems : Building~a~distributed~data-manager}
\author{Polet Pierre-Etienne}
\date{2017, second semester}

{\large

\begin{document}
	\maketitle

	\section*{Introduction:}
		For this project we had to build a distributed data-manager.

	\section{Topology choice:}
		I choose to use an hypercube topology for this project, because of the main advantages of this topology:
		\begin{itemize}
			\item Each nodes are at distance at most $\log_2(N)$ from the others, so we can expect a low number of messages to transmit informations through the network.
			\item Each node have $\log_2(N)$ neighbors, so we hope to prevent to get a non-connected network in case of crash of few of them.
		\end{itemize}

		Intuitively this topology may allow to have a low numbers of messages and a "safe" network without getting an highly connected graph. On the other hand there is no order between the nodes and solutions to communicate efficiently seems not trivial.

		\subsection{Node sketch:}
			In the topology a node will be an Erlang thread, once assigned in the network each node will know its unique identifier related to this network.

			The identifier ID of a node is an integer, each node will be connected to another node if and only if their ID differ from 1 digit in their binary representation.

			Each node will then had variable to memorize his neighborhood and data saved in the network.

			\begin{figure}
							\begin{tikzpicture}[-,>=stealth',shorten >=1pt,auto,node distance=2cm,
							semithick]
							\tikzstyle{every state}=[fill=white,draw=blue,text=black]

							\node[state] (0)                 {$0$};
							\node[state] (1) [right of=0] 	{$1$};
							\node[state] (2) [right of=1] 	{$2$};
							\node[state] (3) [right of=2] 	{$3$};
							\node[state] (4) [right of=3] 	{$4$};
							\node[state] (5) [right of=4] 	{$5$};
							\node[state] (6) [right of=5] 	{$6$};
							\node[state] (7) [right of=6] 	{$7$};

							\path
							(0) edge[bend left] 	node {} (1)
							(0)	edge[bend left]		node {} (2)
							(0)	edge[bend left]		node {}	(4)

							%(1) edge[->] 			node {} (2)
							(1) edge[bend right]	node {} (3)
							(1) edge[bend right]	node {} (5)

							(2) edge[bend left]		node {} (3)
							(2) edge[bend left]		node {} (6)

							%(3) edge[->] 	 		node {} (4)
							(3) edge[bend right]	node {} (7)

							(4) edge[bend left]		node {} (5)
							(4) edge[bend left]		node {} (6)

							%(5) edge[->]  	 		node {} (6)
							(5) edge[bend right]	node {} (7)

							(6) edge[bend left]		node {} (7);

							\end{tikzpicture}
				\caption{Links in an hypercube topology. (communication)}
			\end{figure}

		\newpage
		\subsection{Communication algorithm:}
			I decide to mainly use waves algorithms, I call them broadcast in the source code.

			\paragraph{Node to node:}
				This protocol is not implemented in the Erlang project.

				The idea is to contact one node by sending a message through the topology. When a node have the message, he know the destination and the message kept in memory its visited nodes.

				A node search the first different bit between its Id and the receiver's Id, he will send the message to its neighbor which differ from this bit if the message had not yet pass through this node.

				An example of this protocol is given (see Fig 2).

							\begin{figure}
								\begin{tikzpicture}[->,>=stealth',shorten >=1pt,auto,node distance=2cm,
								semithick]
								\tikzstyle{every state}=[fill=white,draw=blue,text=black]

								\node[state] (0)                {$abc$};
								\node[state] (1) [right of=0] 	{$ab\overline{c}$};
								\node[state] (2) [right of=1] 	{$a\overline{b}c$};
								\node[fill=white,draw=red,text=black] (3) [right of=2] 	{$a\overline{bc}$};
								\node[state] (4) [right of=3] 	{$\overline{a}bc$};
								\node[state] (5) [right of=4] 	{$\overline{a}b\overline{c}$};
								\node[state] (6) [right of=5] 	{$\overline{ab}c$};
								\node[state] (7) [right of=6] 	{$\overline{abc}$};

								\path
								(0) edge[bend left]  	node {1} (1)

								(1) edge[bend right]	node {2} (3)
								(3) edge[bend right]	node {3} (1)
								(1) edge[bend right]	node {4} (5)

								(5) edge[bend right]	node {6} (7);

								\end{tikzpicture}
								\caption{Example of node to node.}
							\end{figure}

			\paragraph{Broadcast:}
				I mainly use broadcast for this project, I implemented two type of broadcast:

				\begin{itemize}
					\item Direct broadcast, nodes know if they had to send the message to their neighbors via their data.
					\item Waves algorithms, node need to save a specific identifier to know if they already receive a message related to this broadcast. With waves protocols nodes also send a message back to their father. At the end their is a Direct broadcast used to remove the identifier from the memory.
				\end{itemize}

	\section{Implemented parts:}
		\subsection{Basic topology (1):}
			We can spawn to kind of agent,
			\begin{itemize}
				\item Waiting agents they have no information and they wait to be contacted by a node from an existing topology.
				\item The node 0 of a topology, he can receive all the commands (But some as storing data will not work with less than 2 nodes in the topology).
			\end{itemize}
				\bigbreak
			Once you've got a node with an Identifier it can receive all kind of messages. All the node in the hypercube had the same role and their is no "father" or central node.

		\subsection{Adding new agents (1):}
			Non waiting agent can receive a message with the PID of a waiting node, they will add him to the topology.

			To do so, they will search for the node with the lower number of connexion (with a wave protocol), then this node will attribute an identifier to the waiting node and with an other wave algorithm they will prevent all the topology.

			If a node is at an hamming distance 1 of the new node it will ask to be linked.

		\subsection{Kill an agent (1):}
			\paragraph{With command: (implemented)}
				A node can prevent its neighborhood when it will shutdown. Others will disconnect from it and replace it following the rules of Adding new agents when a new node will join the topology.

				\medbreak
				Once the topology is an hypercube of dimension $\log(n)$ the network can handle $\log(n) - 1$ shutdown before being disconnected.

				\medbreak
				The node which leave the topology do nothing more that saying "I leave".

			\paragraph{On crash: (not implemented)}
				As the node that leave the topology had nothing to do, in this case we just had to detect when a node crash and execute the previous algorithm.

				\medbreak
				The idea is to ping the neighbors one by one from time to time and if they do not respond after few seconds we assume their dead.

		\subsection{Broadcast (2):}
			Almost all the protocols works with broadcasts.
				\paragraph{Direct}

				\paragraph{Wave algorithm}

		\subsection{Receive data (3):}
			Nodes from the topology can receive data, from another Erlang thread. The will send an "unique" Identifier back to retrieve the data later.

		\subsection{Store data (3):}
			Each nodes can store data, the had a maps that link "unique" identifier to data. They can send the data if we give them the identifier.

			\medbreak
			When an external node ask for a data, the node that handle its request will start a broadcast to ask which one had the data, then one of them will send the data to the external node.

		\subsection{Spread data through the topology (3):}
			I decide to double the data through the topology, as we said on  "Kill an agent" section when a node disconnect it did not help others before leaving.
			\medbreak
			So each data is stored at least on two nodes when the systems is stable. When a node have to add data, he search for the two nodes that had the less data in memory (with a broadcast). After that it will send them data.
			\medbreak
			Each nodes know where data are stored and which data contain each node of the network, all those informations are stored on a map and updated via broadcast.

		\subsection{Save data when someone left (3):}
			When a node left its neighborhood will quickly know it, they will prevent all the other node that the node had left (with a broadcast). If a node had common data with it he will check if it is the last one with those data, if it is the case he will found 2 nodes in the topology and attribute them the data.

			\medbreak
			If this node is shutdown before re-attributing they will be lost.

		\section{Code:}

		\begin{itemize}
			\item There is no deployment script provided.
			\item Code work on Erlang/OTP 18 [erts-7.3].
		\end{itemize}
		The code did not work with Erlang 5 provided on the ENS network, because I use map that are not yet implemented in Erlang 5.

		\subsection{Useful functions:}
		Building the structure:
		\begin{itemize}
			\item algorithm:s\_(h) 	: spawn a node 0 in a new topology.
			\item algorithm:s\_(w) 	: spawn a waiting node.
			\item PID!\{put, PID2\} : add the node with PID2 in the topology of PID.
			\item PID!\{stop\}		: shutdown the node.
		\end{itemize}

		For data:
		\begin{itemize}
			\item PID!\{saveObject, Object, PID1\} : save Object in the network, the identifier will be sent back to PID1.
			\item PID!\{getObject, Id, PID1\} : send the object with this Id to PID1 if it exist.
		\end{itemize}

\end{document}
}


\label{section:results}

We use the \corn dataset with the frequency model and the canonical strategy as a baseline for all comparisons.
For a select logical variables various values are examined if applicable.
First we see whether the use of prior knowledge affects the resulting performance.
Secondly we verify whether our chosen poset consistency strategy works as intended.
Thirdly we investigate whether the use of the flat, frequency or depth model works best.
Fourth we iterate over all machine learning methods and how they most optimally can be configured.
Finally we inspect all corpora and see how various configurations of machine learners perform.

\subsection{Prior knowledge}
\label{section:prior}
First we examine in which manner prior knowledge affects the performance of the various machine learning methods.
Prior knowledge in this case means that we also train the models with the prerequisite datasets.
We only examine the \corn dataset in this section.
Thus for these results we've also trained the \emph{prior} models with the \coq standard library and \mathclasses.
We expect that having this prior knowledge provides more context to the datasets, and will thus result in a better performance.

\begin{figure}[H]
  \centering
  \begin{tabular}{r|cll}
    Method & mode & \oocover & \auc \\\hline
    \input{data/prior-vs-nonprior-corn-frequency-canonical}
  \end{tabular}
  \caption{Results for the \corn (frequency) dataset with canonical strategy, with prior knowledge compared to without}
\end{figure}

This result however is not consistent with regards to our expectations.
The models that have not been trained on prior corpora (being \coq and \mathclasses) outperform
those that have been.
The only exception to this is the \adarank method, but only by a small margin.
Normally we would expect more context to provide more suitable material to make an educated guess.
Instead we think more noise was introduced into the models.
It shows that the machine learning methods have a problem capturing the proper context in this case.
For \knn the dataset without prior knowledge performs better for a lower number of neighbors $k$.
For \nb the dataset without prior knowledge performs better with a lower rewarding constant $s$ for known features.
Finally \adarank performs better with a lower number of rounds $T$.
These facts are at least indicative of a smaller trainingset, which indeed the trainingset is.

\begin{figure}[H]
  \centerline{\includegraphics{prior-knn-corn-frequency-canonical-oocover.pdf}}
  \caption{\oocover of the \knn method for both the prior and non-prior versions of the \corn dataset for different values of $k$}
\end{figure}

When looking into the performance of the \knn method we see that it has the same behaviour regardless which dataset you have, except for
the slightly lower performance for the prior dataset.

\subsection{Poset consistency}
\label{section:poset-consistency}
Previously we have formulated various strategies to filter out illegal knowledge from our trainingset.
While these strategies may be better than the other in a performance sense, they should not be viewed in that light.
Rather, some strategies should be regarded as being better suited in an academic sense.
Whilst the canonical method is more consistent with regards to the research by Kaliszyk \cite{kaliszyk2014machine},
the pessimistic method eliminates the factor of the order of the trainingset, which makes comparing results more fair.
We now compare the performance between these strategies in order to validate our understanding of their relative effect on performance.

\begin{figure}[H]
  \centerline{\includegraphics{posetcons-knn-corn-frequency-oocover.pdf}}
  \caption{\oocover of the \knn method for the \corn dataset for different values of $k$ and various strategies}
\end{figure}

As expected the optimistic strategy does not perform well at all.
We conjecture that a lack of context results in this bad performance, which is somewhat contrary to the previous results.
As expected the pessimistic strategy performs better than the canonical method, as it removes the bare minimum of illegal knowledge
from the test set.

\subsection{Frequency, depth and flat}
The original research by Kaliszyk only uses a frequency-based model of the dataset.
It is hard to say whether a depth-based model or a flat model will be better.
We expect that a depth-based model is more indicative of the type of solutions that exist within a definition,
and will thus perform better.

\begin{figure}[H]
  \centerline{\includegraphics{field-knn-corn-canonical-oocover.pdf}}
  \caption{\oocover of the \knn method for the \corn dataset for different values of $k$}
\end{figure}

Interestingly the depth model outperforms the frequency model by quite some margin.
It might be the case that this model is too descriptive and invites overfitting.
If not so, than it might prove to be even better with the frequency model combined in some kind of ensemble.

\subsection{Methods}
% we iterate over all machine learning methods and how they most optimally can be configured.
For this thesis we implemented various machine learning methods.
These methods have varying parameters that can be configured.
In this section we will explore how various configurations of these methods affect performance.

\subsubsection{\knn}
\begin{figure}[H]
  \centerline{\includegraphics{dataset-knn-frequency-canonical-oocover.pdf}}
  \caption{\oocover of the \knn method for various datasets for different values of $k$}
\end{figure}

For the \knn method peak \oocover performance for each corpus occurs after $K\geq20$.
\corn and \mathclasses perform notably worse than the rest.
After a sharp ascent, a slow stagnation or decrease in performance occurs after peak performance.
For low values of $K$ not enough suggestions are made.
After the peak too many suggestions are made, and the relevant suggestions are more frequently made after the first one hundred.

\subsubsection{\nb}
The \nb method has three parameters: a baseline value $\tau$, a rewarding constant $\pi$ and punishing constant $\sigma$.
We have run \nb for a vast range of these parameters in order to get a feeling on how they affect performance.
The results of this have been plotted in graphs, which are rendered with interpolation enabled,
computed with steps of $\Delta_\pi = 8$ and $\Delta_\tau = \Delta_\sigma = 2$.

\begin{figure}[H]
  \centerline{\includegraphics{dataset-nb-frequency-canonical-oocover.pdf}}
  \caption{\oocover of the \nb method for the \corn (frequency) dataset for $\sigma = -15$ or $\tau = 0$}
\end{figure}

As expected, any value for $\tau \neq 0$ does not work properly.
The interpolation depicts fuzzy edges for the $\pi$-$\tau$-graph, whilst actually for any value of $\tau \neq 0$ the performance
is radically worse.
This implies that the implementation differs with the work by Kuhlwein \cite{kuhlwein2013mash} and Kaliszyk \cite{kaliszyk2014machine},
as they recommend to use $\pi = 10$, $\sigma = -15$ and $\tau = 20$.
This is either a bug (in either implementation), or an underspecification in the other papers.
Any value for $\pi \geq 10$ and $\sigma \leq -15$ seems to work fine.

\subsubsection{\adarank}
The \adarank method is parameterized in the number of rounds $T$ the learning algorithm is run for.
This parameter is almost irrelevant for the computational time required to compute the performance on a testset.
Also, in the learning algorithm most of the time is spent in computing values derived from the pairs of documents, and not in computing the rounds themselves.
We plot the \oocover performance of \adarank parameterized in $T$ for several datasets:

\begin{figure}[H]
  \centerline{\includegraphics{dataset-adarank-frequency-canonical-oocover.pdf}}
  \caption{\oocover of the \adarank method for various (frequency) datasets for various values of $T$}
\end{figure}

A dip can be seen for $T=4$ and another relatively big improvement for $T=6$, especially for \corn and \mathcomp.
Given that a high value of $T$ has almost no impact on computational resources required, the highest value for $T$ seems to be the best choice.
As for each trainingset the derived features are ordered separately, it is not possible to state which derived feature performs best using this approach.
This can be done as future work.

\begin{figure}[H]
  \centering
  \begin{tabular}{r|rrrrrr}
    T & \oocover & \ooprecision & \recall & \rank & \auc & \volume \\\hline
    \input{data/adarank-corn-frequency-canonical}
  \end{tabular}
  \caption{Performance of the \adarank method for the \corn (frequency) dataset}
\end{figure}

Interestingly for $T=5$ the volume of suggestions is reduced by $6\%$, but the \oocover is not affected.
Thus it succesfully discourages invalid suggestions.
The \auc is marginally impacted negatively, so the order in which suggestions are made changes slightly for the worse.

\subsection{Datasets}

Per dataset we've ran a batch of various machine learning methods and various parameter configurations.
For each dataset we list the best performing parameter configuration.
We can easily read which method performs best, but hopefully will also be able to infer aspects of the nature of the datasets.
As a preview we have compiled the \oocover performance of each method and each corpus in Figure \ref{fig:best-oocover}.

\begin{figure}[H]
  \centerline{\includegraphics{datasets-best-oocover.pdf}}
  \caption{\oocover performance of the various methods for all (frequency) datasets}
  \label{fig:best-oocover}
\end{figure}

We have also added the \emph{Omniscient} method, which is a method that cheats by knowing all aspects of the testset beforehand.
It will try to formulate the optimal answer to the premise selection problem.
By measuring the performance of these answers, we know the upper bound performance that a machine learning method can reach for that given dataset.

\subsubsection{\coq}

\begin{figure}[H]
  \centering
  \resizebox{1.0\textwidth}{!}{%
  \begin{tabular}{r|rrrrrr}
    Method & \oocover & \ooprecision & \recall & \rank & \auc & \volume \\\hline
    \input{data/best-coq-frequency-canonical}
  \end{tabular}%
  }
  \caption{Performance of the various methods for the \coq (frequency) dataset}
\end{figure}

Adarank clearly performs best, with an outstanding performance on \auc.
The performance of \nb and \ensemble is respectable,
especially considering it has a similar \auc compared to \adarank,
but a far better \volume.

\subsubsection{\formalin}

\begin{figure}[H]
  \centering
  \resizebox{1.0\textwidth}{!}{%
  \begin{tabular}{r|rrrrrr}
    Method & \oocover & \ooprecision & \recall & \rank & \auc & \volume \\\hline
    \input{data/best-ch2o-frequency-canonical}
  \end{tabular}%
  }
  \caption{Performance of the various methods for the \formalin (frequency) dataset}
\end{figure}

The performance for \formalin is slighty better than the performance for \coq.
The volume is a bit lower across all methods.

\subsubsection{\corn}
\begin{figure}[H]
  \centering
  \resizebox{1.0\textwidth}{!}{%
  \begin{tabular}{r|rrrrrr}
    Method & \oocover & \ooprecision & \recall & \rank & \auc & \volume \\\hline
    \input{data/best-corn-frequency-canonical}
  \end{tabular}%
  }
  \caption{Performance of the various methods for the \corn (frequency) dataset}
\end{figure}

Relative to the \coq corpus we perform a lot worse with the \corn dataset.
Note that \adarank still performs reasonably well for \oocover, but also \nb and \ensemble perform quite well on \auc.
Whilst with previous corpii the \knnadaptive performance could keep up with the \knn performance, this is no longer the
case for the \corn dataset.

\subsubsection{\mathclasses}
\begin{figure}[H]
  \centering
  \resizebox{1.0\textwidth}{!}{%
  \begin{tabular}{r|rrrrrr}
    Method & \oocover & \ooprecision & \recall & \rank & \auc & \volume \\\hline
    \input{data/best-mathclasses-frequency-canonical}
  \end{tabular}%
  }
  \caption{Performance of the various methods for the \mathclasses (frequency) dataset}
\end{figure}

The \oocover performance for the \mathclasses corpus is quite similar to that of the \coq corpus.
This is very interesting, because that means that all complexity of the \corn dataset is included in that dataset, and not in its dependency \mathclasses.
The \auc performance is comparable to that for \corn.

\subsubsection{\mathcomp}
\begin{figure}[H]
  \centering
  \resizebox{1.0\textwidth}{!}{%
  \begin{tabular}{r|rrrrrr}
    Method & \oocover & \ooprecision & \recall & \rank & \auc & \volume \\\hline
    \input{data/best-mathcomp-frequency-canonical}
  \end{tabular}%
  }
  \caption{Performance of the various methods for the \mathcomp (frequency) dataset}
\end{figure}

We perform worst on the \mathcomp corpus on the \oocover metric.
The \auc metric is comparable to our \coq performance.
Notably the \omniscient method does not get a perfect \oocover grading.
This means that there are a few theorems that use more than a hundred definitions.


\subsection{Read-Eval-Print-Loop}

    \subsubsection{Overview and Analysis}
        A simple REPL interface was constructed to interact with the code written in~\ref{ssec:calculus-implementation}.
        This was done using extended regular expressions similar to those found in the Perl programming language.
        These do not meet the normal definitions of regular expressions as they additionally support recursive matches, backreferences and various other syntactic additions that make them context-free grammars rather than regular expressions.
        In fact, given a suitable engine for applying context-free grammars, this would be a preferable way to parse expressions. \\

        For each agent, there was an extended regex matching its string representation, with backreferences to extract the necessary data.
        Each of $(?P\langl name \rangl \textit{expression})$ represents a named backreference to the given expression and $(? \, \& rec)$ is a recursive call to the outer $(?\langl rec \rangl \textit{expression})$ expression.

        \begin{definition}{Solo Matching\\}
            Solos are built using the regex as follows:
            \begin{align*}
                u \, \tilde{x}       & \quad \longleftarrow \quad \backslash s? (?P \langl subject \rangl [a-z \, 0-9]+) \backslash s (?P \langl objects \rangl ([a-z \, 0-9]+ \backslash s?)+) \backslash s? \\
                \bar{u} \, \tilde{x} & \quad \longleftarrow \quad \backslash s? \backslash ^{\wedge} (?P \langl subject \rangl [a-z \, 0-9]+) \backslash s (?P \langl objects \rangl ([a-z \, 0-9]+ \backslash s?)+) \backslash s?
            \end{align*}
            where a valid name is any lowercase alphanumeric word and $\bar{u}$ is inputted as \texttt{$^{\wedge}$u} and each of \texttt{subject} and \texttt{objects} represent the subject and objects of the solo respectively.
        \end{definition}

        \begin{definition}{Replication\\}
            Replicators are built using the regex as follows:
            \begin{align*}
                ! P \quad \longleftarrow \quad \backslash s? \; !(?P \langl agent \rangl.*)\backslash s?
            \end{align*}
            where \texttt{agent} represents $P$ for the expression $!P$.
        \end{definition}

        \begin{definition}{Scope\\}
            Scopes are built using the regex as follows:
            \begin{align*}
                (x)P \quad \longleftarrow \quad \backslash s? \backslash ((?P \langl bindings \rangl([a-z \, 0-9]+ \backslash s?)+)\backslash )(?P\langl agent \rangl[^{\wedge}\backslash s].+)\backslash s?
            \end{align*}
            where each of \texttt{bindings} and \texttt{agent} represent $x$ and $P$ respectively for the expression $(x) P$.
        \end{definition}

        \begin{definition}{Composition\\}
            Compositions are built using the recursive regex as follows:
            \begin{align*}
                P_1 \, | \ldots | \, P_n \quad \longleftarrow \quad & \backslash s? \backslash (?\langl agents\rangl~\texttt{R}+) (\backslash | (?\&agents))?)\backslash )\backslash s? \\
                \text{where} \quad \texttt{R} \quad \defeq \quad & (?\langl agent\rangl([^{\wedge}|()] | (?\langl rec\rangl\backslash ((?:[^{\wedge}()]++|(?\&rec))*\backslash )))
            \end{align*}
            where each \texttt{agent} is collected in \texttt{agents} and represents each of $P_1 \ldots P_n$.
        \end{definition}
        It is accepted that this is difficult to read and would benefit from a better-structured parsing system.
        For this project and as a write-once solution, it is suitable enough.

    \subsubsection{Testing and Correctness}\label{sssec:repl-testing}
        Testing was first done of the structural equivalence and equality of expressions.
        First, for each of the congruences described above, a normalised expression and its known normal form were tested for equality.
        Testing of reductions was done through testing each of the four cases of fusions being performed:
        \begin{itemize}
            \item Normal fusion $(\tilde{x})(a \tilde{x} \,|\, \bar{a} \, \tilde{y})$
            \item Cross-replicator fusion $(\tilde{x})(a \, \tilde{x} \,|\, !(\bar{a} \, \tilde{y}))$
            \item Multi-replicator fusion $(\tilde{x})(!(a \, \tilde{x}) \,|\, !(\bar{a} \, \tilde{y}))$
            \item Inter-replicator fusion $(\tilde{x})(!(a \, \tilde{x} \,|\, \bar{a} \, \tilde{y}))$
        \end{itemize}
        where for each, there exist approximately four cases of which variables are bound and where the matching scope lies.
        %% TODO: Finally, a test of an expression containing all of the above agents was reduced and checked for correctness.

        Due to the open-endedness of the problem, it is difficult to provide a conclusive, complete testing suite.
        In particular, checking for false positives presents a large space of possible tests.
        This was managed by reducing to the above unit tests and combining with the integration test of each case of fusion.\\

        A sample of test outputs can be found in~\ref{ssec:appendix-calculus-tests}

    \subsubsection{Discussion}
        No research was done as to the usability of this interface.
        Future work would vastly improve the description provided here, particularly with regards to error-reporting.
        Another notable flaw is a disconnect between text inputted and outputted, for example \textit{$^{\wedge}$p a b c} $\rightarrow$ \textit{$\bar{p}$ a b c}.\\

        With further work on the project, I would have hoped to embed this REPL interface in the REST server described in~\ref{ssec:rest-server}.
        Through this, the diagram presented by the server could be changed on the server-side.