\chapter{Design of Experiments}
\chaptermark{DOE}
\label{ch:doe}

This chapter is devoted to the matter of designing studies, and follows the lines of \cite{cox_theory_2000} and \cite{cox_principles_2011}.
As we will see, empirical studies come in many forms and shapes.
The most fundamental distinction is between \emph{observational studies} and \emph{designed experiments}.\marginnote{Observational Studies}
This distinction is performed along our ability to control the studied effects, and will have crucial implications on the interpretation of results, and in particular, on causality claims.

The type of designed experiments we will be discussing is also known as a \emph{Randomized Controlled Trial} (RCT).
\marginnote{Randomized Controlled Trial}.

\section{Challenges in Empirical Studies}

The challenges in all empirical studies, be it designed experiments, or observational are:
\begin{description}
	\item [Systematic errors]
	Can be thought of as bias, even thought much more general than the narrow definition in statistical theory.
	Refers to the distortion of conclusions due to confusion sources that do not cancel out in the long run.
	\item [Non-systematics errors]
	Can be thought of as the ``noise''.
	If the non-systematic errors are large, then noise may mask the signal.
	\item [Uncertainty in conclusions]
	Only in the formalism of mathematics and logic things are true or false with certainty.
	In natural sciences we have assumptions, and confidence levels.
	Each conclusions should thus be accompanied by the level of certainty in which it is stated.
	\item [Efficiency]
	The above goals may certainly be achieved given unlimited resources.
	Achieving them quickly and cheaply, is a worthy goal, and a veritable art.
	\item [Range of validity]
	Do our conclusions hold in all countries? Weathers? Altitudes? Races?
	How general are our conclusions?
\end{description}

\subsection{Dealing with Systematic Errors}

Systematic errors arise from two sources: either the study design is such that it does not measure what we think it measures, or leakage of personal judgment.

The first source of systematic errors can be dealt at the design stage, or at the analysis stage.
Much emphasis is given in the literature to unbiased estimation, but reality is that a good design can save a lot of trouble at the analysis stage.
Later in this chapter, we will discuss in detail the means to avoid systematic errors.

\subsection{Dealing with Non-Systematic Errors}

Informally speaking, non-systematic errors, i.e. noise, is reduced by:
\begin{description}
	\item [Compare like with like] Comparing similar units. This is done by the sampling scheme.
	\item [Replications]
	Repeat a measurement enough times, and the non-systematic error will average out.
\end{description}

\section{DOE Preliminaries}
\label{sec:DOE_preliminaries}

We will now discuss \emph{designed experiments}.
Observational studies are briefly discussed in Section~\ref{sec:observational}.

When designing a product (remember DFSS \ref{sec:dfss}), or once a control chart has signaled an alert, we will want to know what \emph{controllable inputs} influenced our process, and how to reduce its variability.
In our SPC terminology, we will want to know what are the \emph{causal effects} of our controllable inputs, on our \emph{CTQ}.
The theory of discovering these effects is the theory of \emph{design of experiments} (DOE), which uses a a slightly different terminology which we will define in the next section.

\subsection{Terminology}
Many, if not most of the following terms, originate in R.A. Fisher's seminal book ``The Design of Experiments'' \citep{fisher_design_1960}.
As such, the DOE literature is rich in agricultural terms, due to its historical origins.
When old ideas get new names, we try to emphasize this in the text.

\begin{tcolorbox}[breakable]
\begin{description}

\item [Design]  Complete specification of experimental test runs, including blocking, randomization, repeat tests, replication, and the assignment of factor–level combinations to experimental units.

\item [Experimental Unit]  Entity on which a measurement or an observation is made.

\item [Homogenous Experimental Unit] Units that are as uniform as possible on all characteristics that could affect the response.

\item [Factors]  A controllable experimental variable that is thought to influence the response.
In the language of SPC: \emph{a controllable input}.

\item [Level] Specific value of a factor.

\item[Treatment] The particular factor-level combination applied to an experimental unit.
\Aka \emph{manipulation}, or \emph{cell}.

\item [Factor Encodings]
The numerical encoding of factor levels.
Of minor importance for designing.
Of major importance for analysis.
Two level factor encodings include:
	\begin{enumerate}
	\item Effect coding: where levels are encoded with $\set{-1,1}$.
	\item Dummy coding: where levels are encoded with $\set{0,1}$.
	\end{enumerate}

\item [Design Matrix] A matrix description of an experiment that is useful for constructing and analyzing experiments.

\item [Response]
The CTQ in the SPC literature.
The $y$ variable in the regression literature.
The \emph{response} in the DOE literature.

\item [Main Effect] Change in the expected response between two factor–levels.
We emphasize that effects, unlike simple population parameters, imply a causal relationship.
Akin to the \emph{assignable causes} in the SPC literature, and $\beta$'s in the regression literature.

\item [Interaction] Existence of joint factor effects in which the effect of each factor depends on the levels of the other factors.
Part of the \emph{assignable causes} in the SPC literature.

\item [Noise] \Aka \emph{error}.
The variability in the response that cannot be attributed to any factor.
This is the \emph{common} variability in the SPC literature, and the $\varepsilon$ in the regression literature.

\item [Replication] A single repetition of an entire experiment, i.e., a single measurement of the response in all experimental conditions.

\item [Repeats] Repeated measurements on the response under the same conditions, i.e., under the same treatment. \Aka \emph{repeat tests}.

\item [Covariate]  An variable that influences the response but is unaffected by any other experimental factors.
We cannot change at will, but we can \emph{control/account for them}.

\item [Non Specific Factor] A variable that we suspect to affect the response, but we can only vaguely define, thus impossible to measure.
As a consequence, we will not care to study its effect, but rather just remove it (e.g. by blocking).
Think of ``life style'' as an example of a non-specific factor.

\item [Blocking]  Blocking, or \emph{grouping}, is an experimental design technique that removes excess variation by grouping experimental units or test runs so that those units or test runs within a block are more homogeneous than those in different blocks. Blocking attributes are also known as \emph{non specific factors}.
The blocking in the design needs to be accounted at the time of the data analysis.

\item [Confounding] When the design is such that several effects cannot be told apart. \Aka \emph{aliasing}.

\end{description}

\end{tcolorbox}

Several matters should be emphasized before we dive in.
\begin{description}

	\item [Randomization] is the random assignment of units to treatments.
	It is fundamental to our purpose because the idea of an \emph{effect} implies \emph{causality}.
	Since we seek to intervene in the production to reduce variability, we only care about causal effects.
	It is the mechanism of \emph{randomization}, that allows us to conclude that the correlations we find are indeed causal, and not merely statistical.
	For a treatment of causal inference in non-randomized experiments, see the Bibliographic Notes section.

	\item [Pre-experiment] In this text we take it for granted that the purpose of the experiment is well known, and the candidate factors defined.
	We are fully aware, as should be the reader, that in application this is a non-trivial luxury.
	Indeed, a lot of meetings, planning, and expertise go into the selection of factors, their candidate levels, etc.

	\item [Power Analysis] Part of the pre-experiment may include a power analysis.
	The pre-experiment power analysis will typically be very approximate, and rely on many assumptions.
	It is still important, as it gives an idea of the feasibility of an experiment, and avoids wasting resources.

	\item [No Textbook Solution] We will present many design ideas and principles, yet in real life problems rarely obey text-books.
	You should thus feel free, and even obliged, to think about your particular problem and adapt the experiment as you best see fit.

	\item[Data Analysis]
	In this text, we only discuss the \textbf{design} of the experiment, and not the \textbf{analysis} of the data.
	This is a non-standard choice as DOE is typically presented alongside the \emph{analysis of variance} (ANOVA) framework. \marginnote{ANOVA}
	We decouple the two for several reasons.
	First, because \cite{cox_theory_2000} do so.
	Second, because these are two different thing.
	The ANOVA framework may be replaced by the framework of \emph{linear models}, \emph{mixed models}, \emph{variance components}, and possibly others analysis frameworks.
	There is a vast literature focusing on the analysis.
	If asked, this author may recommend \cite{hocking_analysis_1985}, which presents both the ANOVA terminology, and the linear models terminology.

\end{description}

\section{Systematic Errors in DOE}

Dealing with systematic errors is possibly the greatest concern in causal inference.
Be it a medical intervention, an industrial production process, an economic policy, or a social welfare plan- all of these critically rely on a fair assessment of the magnitude of the effect of the intervention. Bias is disastrous.

Bias originates from two sources.
The first is some unwelcome property of the data collecting process.
The second is by the leakage of some personal judgment into the measuring or analysis.

\begin{example}[Judgment leaking into measurements]
	Consider cancer patients assigning themselves to the treatment or placebo groups.
	This \emph{self selection} will clearly bias the reported effect of the new drug (who would willingly choose the placebo?!?). \marginnote{Self Selection}
\end{example}

\begin{example}[Judgment leaking into analysis]
	Return to the new-drug experiment.
	This time, imagine the data analysis is done by an employee of the pharmaceutical company, knowing that ``Drug A'' is the new drug his company has been working on, which will grant him a considerable bonus if shown efficient.
	Can he be expected to perform a fair analysis?
\end{example}

Two ways to deal with such bias:
\begin{description}
	\item [Balanced Design]
	In the simplest interpretation of ``balance'' we mean a design where an equal number of units is assigned to each treatment.
	More generally, we will keep ``balance'' by \emph{randomization}, and implying some symmetry in the combinatorial design of the experiment.

	\item [Bliding]
	By blinding we mean that personnel is unaware of the factors being studied.
	In regular blinding, subject are unaware of the treatment they are being administered.
	In \emph{double blinding}, the experimenters and analysis are unaware of the treatments they are studying.

\end{description}

\section{Non Systematic Errors in DOE}
\label{sec:variance_components}
%TODO: relate to imptoving precision in sampling.

The idea that random samples come with (common causes of) variability, i.e. noise, should not be new to the reader.
In this section, we will try to decompose variability into it sources, and learn several techniques to reduce them.
Starting with a motivating example, which you should use to fix ideas as you progress along this section.

\begin{example}[Web Design]
\label{eg:web_design}
Consider the problem of optimizing a web site, where individuals performance is measured by conversion rate (the probability of a new user to signup, purchase, etc).
In the DOE language, the conversion is the \emph{response}.
A user (or ip address) is an \emph{experimental unit}.
The site's attributes are the \emph{factors}.
A particular site's design, i.e. a combination of factors, is the \emph{treatment}.
The users' attributes are \emph{covariates}.
The user's life-style, a \emph{non-specific factor}.
If the site's attributes affect differently users with different attributes, we say there is an \emph{interaction}.
\end{example}

\begin{example}[Two Competing Diets]
\label{eg:diets}
Consider the problem of comparing two nutrition diets.
Clearly, the effect of a diet on life expectancy strongly interacts with both genetics and life style.
It would be thus very nice if we could block experimental units, so that each block has a homogenous genetics and life-style.
That is why \textbf{twins} are so popular when designing experiments!
\end{example}

\subsection{Gage R\&R Studies}
R\&R stands for \emph{repeatability} and \emph{reproducibility}.
These experiments are aimed at assessing the irreducible non-systematic errors in measurement and \emph{range of validity} of conclusions.
As such, in a typical gauge R\&R experiment, measurement will be made under the same conditions in different labs, by different technicians, etc.

\subsection{Completely Randomized Designs}
In the simplest of designs, all experimental units are randomly assigned to treatments.
This is typically easy to implement.
In Example~\ref{eg:web_design} this would imply randomly assigning users to interfaces.
In Example~\ref{eg:diets} this would mean ignoring the fact that the experimental units come in naturally blocked groups of two, and randomly assigning all people to treatments.
If you suspect, as you should, that in both examples we may reduce variability by grouping units together, keep reading.

\subsection{Randomized Block Designs}
The idea of \emph{blocking} is to replace the complete randomization scheme by a restricted randomization scheme so that variability can be reduced without introducing bias.
The restricted randomization is created by \emph{grouping}, or \emph{blocking} groups of experimental units, and randomizing allocation within the group.

In the twins example, nature has provided us with natural blocks of two.
If comparing two treatments, we would naturally randomize treatments within twin-pair.
This is known as a \emph{randomized complete block design}. \marginnote{Complete Block Design}
If we had three treatments, or more, to compare, we cannot completely randomize. This is knows as an \emph{incomplete block design}.
There are several approaches to incomplete block designs, but we refer the reader to \cite[Sec.4.2]{cox_theory_2000} for details.

\begin{think}[Paired samples]
While we try to avoid matter of data-analysis, it should be obvious to the reader that a two-group randomized block design should be analyzed as a \textbf{paired sample}.
If adding a Gaussianity assumption, we have a paired-sample \textbf{t-test}.
\end{think}

In our web-design example nature has not provided natural homogenous blocks, but we can create them ourselves.
We could block individuals along, say, the age covariate.
We would then randomly assign users to layouts, only \textbf{within} age groups, so that all layouts are presented to each age-group.
If each age group has as many users as there are different site designs, this is a randomized complete block design.

If there are more layouts than users per age group, this is an incomplete block design.
Another idea is to use each user as his own block, by introducing a temporal dimension to the experiment.
We could show all layouts to the same person.
This particular type of blocking is known as a \emph{crossover} design, discussed in Section~\ref{sec:crossover}.

\begin{think}
	With twins, you can have a complete block design comparing only two treatments.
	How would you go about to compare $3$ treatments?
	And $k>2$ treatments?
\end{think}

In summary, blocking is done in order to create homogeneous groups.
Put differently: compare like-with-like.
One can block in along factors, covariates, or non-specific factors.
One can then compare treatments only within blocks (e.g. the paired sample t-test), thus gaining accuracy by avoiding the variability between blocks.
A good blocking strategy is such where blocks are very homogeneous within themselves, but very different between themselves, at least with respect to the response being studied.
Is there any downside to blocking? Not statistically, maybe economically.

\subsection{Crossover Design}
\label{sec:crossover}

Return to the diet comparison of Example~\ref{eg:diets}.
Now assume that $5$ diets are being compared, and requiring quintuplets\footnote{Five identical siblings.} will severely limit the available sample size.
If the response of different people to the diets is strongly affected by genetics \andor lifestyle, it may be the case that by (randomly) assigning diets to individuals the diets effect will be completely masked by the variability between subjects.
To solve this, we could give each subject all $5$ diets.
Each subject would \textbf{act as his own block}.
Diet effect would be analyzed within subjects, and hopefully, having removed the variability between subjects, the diet effect would reveal itself.

Enter a new complication.
What if the response to a particular diet is affected by the previous diet?
This is known as a \emph{carryover effect}, or \emph{residual effect}, which may bias the main layout effects of interest. \marginnote{Carryover Effect}
If all subject receive diet B after A, then the estimated effect of B will be biased by the carryover effect of A.
A way to ``average out'' this carryover effect is to balance the treatment sequences.
If each subject is randomly assigned to one of the $5!$ diet arrangements this balance would be achieved.

The idea of administering all treatments to each unit, in a random sequence that balances carryover effects is known as a \emph{fully randomized crossover design}, or as a \emph{counterbalanced design}. \marginnote{Fully Randomized Crossover}
Table~\ref{tab:crossover} demonstrates the $3!=6$ sequences for a $3$ treatment problem.
\begin{table}[ht]
\centering
\begin{tabular}{rlll}
  \hline
 & 1 & 2 & 3 \\
  \hline
1 & 1 & 2 & 3 \\
  2 & 2 & 3 & 1 \\
  3 & 3 & 1 & 2 \\
  4 & 1 & 3 & 2 \\
  5 & 2 & 1 & 3 \\
  6 & 3 & 2 & 1 \\
   \hline
\end{tabular}
\caption[Crossover Design]{Crossover design: a balanced sequence of administration of $3$ treatments, generated with the \rcode{des.MOLS()} function of the \rcode{crossdes} \R package. }
\label{tab:crossover}
\end{table}

\section{Efficiency in Design-- Factorial Designs}
\label{sec:factorial_design}

In the previous section we discussed methods of reducing the non-systematic errors in the response to each treatment, so that we can better reveal differences between treatments.
In the current section we try to break treatments to their building blocks; \emph{factors}.
By doing so we will gain both economical and statistical efficiency.
Each sample will give us information on the effect of not one, but several \emph{factors}.
We will also be able to learn if the effect of factors are additive, or non additive, and we will be able to extrapolate from observed treatments, to unobserved treatments.

\begin{example}[Web design revisited]
\label{eg:web_design_II}
Back to the web-design from Example~\ref{eg:web_design}, assume now that treatments, i.e. layouts, differ in the location of a button, the colour of the heading and the size of the logo.
If each has two possible levels, we deal with $3$ factors with $2$ levels each, totalling in $8$ different layouts.
Instead of estimating the effect of the $8$ layouts, we estimate the effect of each of the $3$ factors.
Given a budget of, say, $n=8$ observations, we would have one observation per treatment.
The magic of a factorial design is that we have $4$ observations for each factor level.
If we treat a web-design as the sum of its factor effects, we have gained accuracy
\end{example}

\begin{think}
Is the magic of the factorial design a free lunch?
No. The lunch has a price.
The price is that we have assumed that a site is the sum of its components, without interactions.
Without this assumption, there is no statistical accuracy gained from a factorial design.
There is however, the benefit that we can actually verify the validity of the additivity assumption.
\end{think}

\subsection{Full Factorial Designs}
A \emph{full factorial}, or \emph{complete factorial} design, is one where all factor-level combinations are replicated the same number of times.
The factors $A,B,C,...$ each with numbers of levels $l_A,l_B,l_C,\dots$, define $l_A\times l_B \times l_C \times \dots$ treatments.
By far, the most common of the full-factorial designs is where all factors have two levels: $l_A=2, l_B=2, \dots$.
These is known as $2^k$-design, where all combinations of the $k$ factors is measured in each replication.

\subsubsection{$2^k$ design}
Consider two factors denoted $A$ and $B$.
Adopt the effect coding so that we encode their levels by $\set{-1,1}$.
The design matrix of a single replication is depicted in Figure~\ref{fig:full_factorial} (top right) along with a visualization of the design (top left).
Allowing $n$ observations per condition, the experiment will include $4n$ observations, which will be randomized between conditions.

\begin{figure}[H]
\centering
\includegraphics[width=0.7\linewidth, height=0.3\textheight]{art/full_factorial}
\caption[Full Factorial Design]{Full factorial designs: $2^2$ and $2^3$. \newline \url{http://chemwiki.ucdavis.edu/Analytical_Chemistry/Analytical_Chemistry_2.0/14_Developing_a_Standard_Method}}
\label{fig:full_factorial}
\end{figure}
With this $2^2$ design, we may recover several effects:
\begin{description}
\item [Main effect of A] The effect of varying $A$ from $(-)$ to $(+)$.
\item [Main effect of B] The effect of varying $B$ from $(-)$ to $(+)$.
\item [Main effects with interaction] The effect of varying both $A$ and $B$ from $AB=(--)$ to $AB=(++)$.
\end{description}

\begin{think}
	How would look the design matrix, $X$, of a $2^2$ experiment with $3$ replications, and effect coding?
\end{think}

We typically denote the (mean) response for each treatment by the $\mu_{\smiley}$ notation, where $\smiley$ encodes the applied treatment by stating the conditions that were at their $+$ state:
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|c|}
\hline Treatment & A & B & Mean \\
\hline 1 & + & - & $\mu_a$ \\
\hline 2 & + & + & $\mu_{ab}$ \\
\hline 3 & - & - & $\mu_{(1)}$ \\
\hline 4 & - & + & $\mu_b$ \\
\hline
\end{tabular}
\end{table}

\marginnote{Interaction}

Slightly intruding into the realm of data analysis, a visualization of interactions is known as the \emph{interaction plot}, depicted in Figure~\ref{fig:interaction_plot}.
The upper left panel demonstrates a lack of interaction (think why), while the upper right panel depicts an interaction.
\begin{figure}[ht]
\centering
\includegraphics[width=0.3\textheight]{art/attribute_doe_interaction_plot}
\caption[Interactions plot]{Interactions plot. \newline \url{http://blog.minitab.com/blog/statistics-in-the-field/optimizing-attribute-responses-using-design-of-experiments-doe-part-2}}
\label{fig:interaction_plot}
\end{figure}

\begin{remark}[Popularity of $2^k$]
The $2^k$ designs are probably the most popular full factorial designs.
This may be attributed to the fact that many factors studied really have two levels, but more plausibly, since these are merely \emph{screening} experiments.
Once non related factors have been screened, the experimenter may proceed from the $2^k$ design to more elaborate ones.
\end{remark}

\begin{extra}[Intermediate Factor Levels]
In a $2^k$ design, a factor may actually be a \textbf{continuous} controllable input which was restricted to two values for convenience.
After estimating the effect of the factor, we may want to know what the effect would have been, had we set it on some intermediate level.
It is customary to assume that a main effect acts linearly in-between experimental conditions, yet you should remember that there is nothing in the data to support this.
For a more rigorous approach, see the Continuous Factors Section~\ref{sec:continouos_factors}.
\end{extra}

\subsubsection{$3^k$ Designs}
The name $3^k$ design is rather self explanatory.
If a continuous variable is discretized to $3$ levels, one may actually estimate and check for non-linear main-effects, which is impossible with $2^k$ designs.
Then again, more than $2$ levels are rarely treated as full factorial experiments.
This is because $3$ level factors typically appear when aiming at optimizing the factor level combination, for which the \emph{response surface} methodology of Section~\ref{sec:response_surface} is more economical.

\subsubsection{Full Factorial vs. One-Factor-at-a-Time}
Importantly, a factorial design is better than $k$ experiments with one factor at a time. This is because:
\begin{enumerate}
\item Factorial experiments make better (statistical) use of each sample unit for estimating main effects.
\item Factorial experiments allow the estimation of interactions between factors.
\end{enumerate}
A classical illustration of the second point, is given in Figure~\ref{fig:one_factor_at_a_time}.
The figure depicts a one-factor-at-a-time optimization sequence (from A to H). Since the factors are not varied simultaneously, the experiments is unable to identify that the response is not a linear surface.
Put differently, he is unable to estimate an interaction.

\begin{figure}[ht]
\centering
\includegraphics[height=0.3\textheight]{art/optimization-without-statistical-doe-2015-03-23-7-638}
\caption[One-at-a-time optimization]{Optimizing factors, one at a time.\newline \url{http://www.slideshare.net/PaulFishbein/optimization-without-statistical-doe-2015-03-23-46817073}}
\label{fig:one_factor_at_a_time}
\end{figure}

\subsection{Fractional Factorial}
Full factorial designs are the simplest designs to setup and interpret.
A major drawback, are the resources required when $k$ is large.
This is where the \emph{fractional factorial}, or \emph{partial factorial} designs kick in.
The fundamental idea is to design a full factorial, but skip a couple experimental conditions.
If conditions to skip are wisely selected, the information we give up is the least interesting.

\begin{example}[From $2^2$ to $2^{2-1}$]
\label{eg:fractional_factorial}
As a first toy example, we will try to save some time and money by eliminating particular conditions of the $2^2$ design in Figure~\ref{fig:full_factorial}.
As the name may suggest, a $2^{2-1}$ design, has $2$ experimental conditions in each run.
There are thus $\binom{4}{2}=6$ possible eliminations, which are enumerated in Table~\ref{tab:partial_factorial} along with the extractable information in each elimination.
\begin{table}[ht]
\begin{tabular}{|p{2.5cm}|p{10cm}|}
\hline Elimination &  Problem \\
\hline
\hline 1,2 &  No information on $a$. \\
\hline 1,3 &  No information on $b$.\\
\hline 1,4 &  $a$ aliased with $b$ aliased with $ab$. \\
\hline 2,3 &  $a$ aliased with $b$ aliased with $ab$. \\
\hline 2,4 &  No information on $b$. \\
\hline 3,4 &  No information on $a$.\\
\hline
\end{tabular}
\caption[Aliasing]{Aliasing in a $2^{2-1}$ design: All possible eliminations from the $2^2$ design that lead to a $2^{2-1}$ design.}
\label{tab:partial_factorial}
\end{table}
\end{example}

The lesson from Example~\ref{eg:fractional_factorial} is that in a fractional factorial our savings in time and money, come at the cost of the information that can be drawn from the experiment.
The idea behind partial factorial experiments, is that by an informed choice of the conditions skipped, we can choose what information to give up. The information lost, is known as the \emph{alias structure}.\marginnote{Alias Structure}

In practice, we will rarely design the experiment by going over all the $\binom{2^k}{2^k-2^{k-p}}$ possible eliminations like in Example~\ref{eg:fractional_factorial}, but rather choose from pre-selected designs.
Table~\ref{tab:partial_factorial_ii}, generated with the \rcode{FrF2()} in the \rcode{FrF2} \R package, is an optimal $2^{5-2}$ design.
Using the \rcode{design.info()} function of that same package, we know that the aliasing structure of this design is
$a=bd=ce, b=ad, c=ae, d=ab, e=ac$.
We will not go into the details of how the aliasing structure is computed, but rather refer the reader to \cite{cox_theory_2000}.
\begin{table}[ht]
\centering
\begin{tabular}{rrrrrr}
  \hline
 & A & B & C & D & E \\
  \hline
1 & -1 & 1 & -1 & -1 & 1 \\
  2 & 1 & -1 & 1 & -1 & 1 \\
  3 & -1 & -1 & -1 & 1 & 1 \\
  4 & -1 & 1 & 1 & -1 & -1 \\
  5 & -1 & -1 & 1 & 1 & -1 \\
  6 & 1 & 1 & 1 & 1 & 1 \\
  7 & 1 & -1 & -1 & -1 & -1 \\
  8 & 1 & 1 & -1 & 1 & -1 \\
   \hline
\end{tabular}
\caption[Fractional Factorial Design]{$2^{5-2}$ design.}
\label{tab:partial_factorial_ii}
\end{table}

\begin{extra}[Coding Theory]
There is a close relationship between design of experiments and coding theory in computer science.
A possible reference on the matter is \cite{hill_first_1986}, or \cite{hedayat_orthogonal_1999}.
\end{extra}

%
%\subsection{Split Plot Design}
%\label{sec:split_plot}
%Revisiting our web site layout problem:
%We want to block along age and country.
%For each age and country combination (full factorial blocking) we can randomly assign users to site layouts.
%This combination of a factorial design for blocking, is known as a \emph{split plot}, or \emph{split unit} design.
%For more on split plots see \cite[Sec.6.4]{cox_theory_2000}.
%

%\section{Specialized blocking techniques}
%
%\subsubsection{Latin Square Design}
%[TODO: put section in context- variance? bias? economy?]
%When homogenous groups are defined by two non-specific factors, we would like to create blocks that are balanced, so that the non-specific factors do not bias effect estiamtes.
%In our running example, non-specific factors may be age and gender.
%We could construct all age and gender combinations, and randomly assign users from each age-gender to each site design.
%This would is an example of a \emph{split plot} design, discussed in Section~\ref{sec:split_plot}.
%If, however, we do not deal with age-gender, but rather with age-country, we may not have enough users in each cell to assign to all site designs.
%Since we do not want to estimate the effect of age, nor country, but merely balance the design so that age and country effects do not alias the site's effect, we do not actually need all age-country combinations.
%
%
%
%If we have $k$ designs, we may group ages and countries into $k$ categories each, and each design is seen only once in each age group and once in each country group.
%This allocation of treatment to blocks is known as a \emph{Latin Square}.
%Table~\ref{tab:latin_square} demonstrates a Latin Square for $7$ web site designs, and compares it to a non-balanced allocation.
%\begin{table}[ht]
%    \begin{minipage}{.5\linewidth}
%        \centering
%		\begin{tabular}{rlllllll}
%		  \hline
%		 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		  \hline
%		1 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		2 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		3 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		4 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		5 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		6 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		7 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		   \hline
%		\end{tabular}
%    \end{minipage}%
%    \begin{minipage}{.5\linewidth}
%      \centering
%		\begin{tabular}{rlllllll}
%		  \hline
%		 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		  \hline
%		1 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
%		  2 & 2 & 3 & 4 & 5 & 6 & 7 & 1 \\
%		  3 & 3 & 4 & 5 & 6 & 7 & 1 & 2 \\
%		  4 & 4 & 5 & 6 & 7 & 1 & 2 & 3 \\
%		  5 & 5 & 6 & 7 & 1 & 2 & 3 & 4 \\
%		  6 & 6 & 7 & 1 & 2 & 3 & 4 & 5 \\
%		  7 & 7 & 1 & 2 & 3 & 4 & 5 & 6 \\
%		   \hline
%		\end{tabular}
%		\end{minipage}
%	\caption{A $7$-treatment, two-factor design.
%	Left pane is not balanced.
%	Right pane is balanced with a Latin Square, generated with the \rcode{MOLS()} function of the \rcode{crossdes} \R package.}
%	\label{tab:latin_square}
%\end{table}
%
%
%The following example makes the same point, with the original agricultural motivation for this design.
%\begin{example}[Agricultural Yield Study]
%\label{eg:latin_square}
%Consider an farmer growing corn.
%He wishes to study the effect of $7$ candidate fertilizers (single factor with $7$ levels).
%He is aware that the location of the plot may affect yield, due to slightly different sunlight, irrigation, altitude, etc.
%He thus assumes he has to deal with two extra variance sources: row and column of the plot.
%He could treat the row and column as two factors, with $7$ levels each, be he does not care to estimate the row/column effect, but merely to avoid bias.
%He will thus try to look for an allocation of fertilizer to rows and columns so that any row/column effect will be averaged out.
%The Latin Square design of Table~\ref{tab:latin_square} does just that.
%\end{example}
%
%
%\begin{extra}\noindent
%\begin{enumerate}
%\item If the Latin Square reminds you of Soduko, it should. Soduko is just that!
%\item The Latin Square is a very economical design, and is not used as often as it should be.
%\item If we had reason to believe that different age groups in different countries respond differently to each site (an interaction), then the Latin Square may actually introduce aliasing.
%\end{enumerate}
%\end{extra}
%
%
%
%\subsubsection{Extensions of the Latin Square}
%\begin{enumerate}
%\item \textbf{Latin Hypercube}: When balancing more than two non-specific factors, we will call upon \emph{latin hypercube designs}, \aka \emph{orthogonal latin squares}. \marginnote{Orthogonal Latin Squares}
%\item \textbf{Greco-Latin Square}: A latin-hypercube with three variability sources.
%\end{enumerate}

\section{Optimal Designs}

Our discussion until now has been informal with respect to the desirable qualities of a design.
We used the idea of ``balance'' and ``orthogonality'' to avoid bias and unwelcome variability.
In this section, we try to formalize the notion of a ``good design''.
We start with some motivating examples.

\begin{example}[Design for linear regression]
	\label{eg:design_linear}
	Figure~\ref{fig:design_linear} demonstrates the effect of the different location of the sampling points ($x$) on the quality of the estimated regression line.
	In a \textbf{linear} model, the figure depict and intuition suggests, that it is preferable to spread the sampling points as far as possible.
	\begin{figure}[h]
		\centering
		\includegraphics[height=0.3\textheight]{art/linear}
		\caption[Design for Linear Models]{
			Design for linear regression. Different panels show different designs. True function as a dashed line. Estimated function as a full line.}
		\label{fig:design_linear}
	\end{figure}
\end{example}

\begin{example}[Design for non linear regression]
	\label{eg:design_non_linear}
	Figure~\ref{fig:design_nonlinear} demonstrates the effect of the different location of the sampling points ($x$) on the quality of the estimated regression line, in a \textbf{nonlinear} model.
	It may seems that unlike the linear case (Example~\ref{eg:design_linear}) optimality is achieved in some intermediate spread of the $x$s.
	\begin{figure}[h]
		\centering
		\includegraphics[height=0.3\textheight]{art/nonlinear}
		\caption[Design for Non Linear Models]{Design for non linear regression. Different panels show different designs. True function as a dashed line. Estimated function as a full line.}
		\label{fig:design_nonlinear}
	\end{figure}
\end{example}

Now for some facts, supported by the previous examples:
\begin{enumerate}
	\item The idea of ``balancing'' as a design criterion is very useful with discrete factors, but limited with continuous factors.
	\item For continuous factors with \textbf{linear} effects, the optimal design implies spreading the sampled factors levels as much as possible.
	This is no longer true for \textbf{non-linear} effects where the optimal design may depend on the unknown true effects.
\end{enumerate}

\subsection{Space Filling Design}
\label{sec:space_filling}
The most natural of designs, which is particularly suitable when we have no a-priori assumption on the functional relation ($f(x)$) between the (continuous) factors and the response, is known as a \emph{space filling design}.
As the name suggests, in a space filling design we aim at filling the factor space. Lacking any a-priori information, the filling will typically be as uniform as possible.
We note however, that once information on $f(x)$ is made available, then a space filling design is typically sub optimal (see Example~\ref{eg:design_linear}).

\begin{extra}[Space Filling and Hashing]
	If you are familiar with the idea of \emph{hashing functions}, then you may see the similarity between space filling and the \emph{uniformity} property of hash functions.
	For a more rigorous discussion, see \cite{hill_first_1986}.
\end{extra}

\subsection{Covariance Optimality}
When estimating the effect of a single continuous factor, we would like a design that gives us the most information per observation on some effect $\beta$.
This is the same as minimizing the variance of the estimator, $\min \{Var[\hat{\beta}]\}$, with respect to the design.
In the case of linear regression with a single coefficient we know that
\begin{align}
Var[\hat{\beta}] &= \frac{\sigma^2_\varepsilon}{\sum_{i=1}^{n}(x_i-\bar{x})^2}.
\end{align}
Minimizing $Var[\hat{\beta}]$ is thus the same as spreading the $x$'s as far as possible, in accordance with the intuition from Example~\ref{eg:design_linear}.
Now recall that in a multivariate linear regression problem with an $n \times p$ design matrix, then
\begin{align}
Var[\hat{\beta}] &= \sigma^2_\varepsilon (X'X)^{-1}.
\end{align}
In this multivariate case, there may be several notions of ``maximal spread''.
Denoting $$M:= (X'X)^{-1},$$ we can define:
\begin{definition}[A-Optimality]
	A design is said to be \emph{A-optimal} if it minimizes the average univariate variance.
	Formally: $\min \set{\Tr(M)}$.
\end{definition}
A-optimality does not account for covariances.
In an extreme scenario, if we have several copies of the same variable, the more copies we have, the more importance that variable will be given by A-optimality.
The most popular optimality criterion is known as \emph{D-optimality}, and does not suffer from this phenomenon.
\begin{definition}[D-Optimality]
	A design is said to be \emph{D-optimal} if it minimizes the volume of the confidence region for $\beta$.
	Formally: $\min \set{\det(M)}$.
\end{definition}

Both A-optimality and D-optimality implicitly target linear models, such as in Example~\ref{eg:design_linear}, because they aim at spreading the $x$'s. From Example~\ref{eg:design_non_linear} we know this to be sub-optimal for non-linear models.
For non-linear designs, one may opt for space filing designs (\ref{sec:space_filling}), or consult \cite{pukelsheim_optimal_1993}.

\begin{extra}[Other Optimality Criteria]
	There are as many optimality criteria as there are matrix norms.
	For instance, recall why not optimize the Spectral-Norm or the Frobenius-Norm of $M$, which we defined for the multivariate $s^2$ chart (Section~\ref{sec:multivarite_s}).
	For a more detailed review, see \cite{wikipedia_optimal_2015}.
	For a mathematical rigorous, and through treatment, see \cite{pukelsheim_optimal_1993}.
\end{extra}

\section{Continuous Factors}
\label{sec:continouos_factors}

If aiming at screening factors, the fact that a factor is continuous is immaterial.
One may sample a continuous factor at two points, and treat it as a $2$ level factor.
This is not the case, however, if aiming at characterizing the full response surface, for which the factorial designs are no longer appropriate.

When dealing with \emph{continuous factors}, or \emph{quantitative factors}, we have many more analysis strategies than when dealing with qualitative factors.

\section{Sequential Designs}
\label{sec:sequantial}
% Mark Vandelmeulenbruke

Unlike ``one-shot'' experiments, in sequential experiments the next step is determined by the outcome of the previous.

\subsection{Sequential Testing}

Consider a clinical trial with a treatment and control group.
Now assume the medicine being tested is a miracle cure with immediate improvement.
Do we really need to keep administering placebos to the control group, just because that was the initial experimental design?
This is where sequential designs come in.
Interestingly, the initial application of a sequential design was not in drug testing, but rather in a military context \citep{wald_sequential_1945}.

The problem with sequential testing, is the \emph{type-I error inflation}, which is simply a \emph{multiplicity problem}.
To see this, assume the null hypothesis is true and tested with the (ridiculous) test, where each observation is tested as it comes.
We will be making $n$ tests, each at level $\alpha$.
If observations independent, the probability of a type $I$ error is the familiar $1-(1-\alpha)^n > \alpha$.

\subsection{Sequential Optimization}

All the following designs aim at finding optimal factor combinations, or recovering the whole response surface.
They do so without an explicit assumption on the shape of the response surface (such as linear).
By being sequential, they are more efficient than one-shot designs such as $3^k$.

\begin{description}
	\item[Central Composite Design]
	Aimed at finding optimal factor combinations.
	At each stage we decide which factors have a non-linear effect and add levels to these factors.
	Chooses the level do be added using a quadratic surface assumption.

	\item[Bayesian Optimization] Like the Central Composite design, but replaces the quadratic surface assumption with a more general functional form\footnote{A Gaussian process prior, i.e., a function which's gradients are assumed to havea Gaussian distribution.}.

	\item[Response Surface Methdology] As the name suggests, aimed at recovering the response surface, and not only optimal combinations.
\end{description}

\subsection{Designs}
\emph{Pooled designs}, \aka as \emph{group testing}, or \emph{hierarchical design}, appears when it is cheaper to measure the response for a group of experimental units simultaneously, rather then for each single one.
It originated in the context of identifying soldiers with syphilis in WWII. It was the Harvard Economist Robert Dorfman that suggested that one may mix blood samples from several soldiers and then repeat the same in the subgroups that show the presence of syphilis \citep{dorfman_detection_1943}.
The method became immensely popular in the DNA analysis age, where technology allows biologists to test for the presence of particular protein in a sample from a group of cells (e.g. a blood sample).

\begin{think}
	Try to use the idea of group testing, and airborne-chemical sensors, to design a luggage inspection process at the airport.
\end{think}

\begin{extra}[Bloom filters]
If you are familiar with data structures and particularly \emph{Bloom Filters}, then you will probably note that a Bloom filter is a data structure that groups objects in a way that facilitates lookup using group testing.
\end{extra}

\section{Taguchi Methods}

\emph{Taguchi methods} is a collective name for the philosophy, design, and analysis methods in industrial production applications promoted by Genichi Taguchi in 1970's Japan.
Focusing on the design principles, we summarize Taguchi's emphasis.
To understand why this emphasis is non trivial, recall that Taguchi was importing DOE from agricultural problems to the newly-born field of industrial-production.

\begin{enumerate}

	\item Achieving low variability is no-less important, and more challenging then achieving a target value.
	Log variability ($\log s$) is thus often used as the response.

	\item Factors which can be controlled only in a lab, but not in production, are deliberately varied in the lab.
	%Typically with split plot designs (\ref{sec:split_plot}).

	\item An ample use of Plackett–Burman designs, and its extensions, to screen large numbers of factors. \marginnote{Plackett Burman}
	A \emph{Plackett-Burman Design} is a factorial design optimized for \textbf{screening}.
	It is a very economical design when studying the \textbf{main effects} of a \textbf{large number} of factors.

\end{enumerate}

\begin{extra}[Definite Screening Designs]
For cutting edge screening designs see \emph{Definite Screening Designs} in \cite{jones_class_2011}.
\end{extra}

\section[AB testing]{$A\backslash B$ Testing}
Our web-site optimization example is interesting not only because it accommodates so many DOE practices, but it is also a practical problem of great interest.
For historical reasons, the experimentation with several web-site layouts is not called a factorial experiment (which it is), but rather an \emph{$A\backslash B$-test}.
One may think of a series of naive A\\B tests as a series of ``one-factor-at-a-time'' experiment.
A less naive approach to A\\B testing would be to see the series of tests as a sequential factorial experiment, and choose the competing layouts using the factorial-design framework.

Some particular characteristics of $A\backslash B$ testing include:
\begin{enumerate}
\item Users come by the millions.
Sample size is rarely an issue, so that avoiding systematic errors is more important than avoiding non-systematic errors which can be averaged out.
\item Browser cookies, or login details define blocks.
Technology permits to optimize the size for each block separately.
If blocking is to very specific subgroups (female users, running Linux, that purchased on Amazon, etc.), then sample size, and thus variability, may become an issue again.
\item Sites have many layout parameters (locations, colours, sizes,...).
Studying all these combinations may result on a formidable task.
Bayesian Optimization and Central Composite designs will be preferred over full factorials.
\end{enumerate}

\section{Computer Experiments}
% no randomness
% space filling parameter choices for simulation
\begin{example}[Designing Wings]
\label{eg:wings}
Consider the problem of designing an air-craft's wing.
We would like to know how the wing's attributes, i.e., factors, govern its lift.
We could obviously conduct real-life experiments by varying the wing's attributes, building the wing, flying the air-plane, and recording results.
Needless to say how expensive this process is.
It is much more reasonable to program the differential equations that govern the lift to a computer, fix several factors values, and solve the equations.
This is what \emph{computer experiments} are all about.
\end{example}

The wind design example (\ref{eg:wings}) demonstrates the following points:
\begin{enumerate}
\item Computer experiments are essentially numerical solutions to complicated systems of equations.
\item Because solutions take a lot of time, only a small finite set of factor levels may be evaluated.
\item The ``response'' to each treatment, is deterministic.
\item The problem of interest is in reconstructing the response at non measured factor levels, so that optimal values may be identified.
\end{enumerate}
It is thus not uncommon to call upon DOE theory for choosing the factor combinations to be experimented with. The analysis of computer experiment is very different than real-life experiment since we have no non-systematic errors.
Space filling designs (Sec.~\ref{sec:space_filling}), and Bayesian Optimization (\ref{sec:continouos_factors}) being a particularly prevalent choices.
See \cite{sacks_design_1989} or \cite{santner_design_2013} for further details.

\section{Observational Studies}
\label{sec:observational}

In this section we abandon designed experiments, in which we were able to randomize and choose the factors levels to be measured.
In \emph{observational studies} we will typically deal with many more covariate than factors.

\begin{example}[Post-sale testing]
	\label{ex:post-sale}
	Consider the notorious Galaxy Note7\footnote{That would spontaneously burst into flames, until Samsung recalled all devices.}, or any car, plane, smartphone, or software.
	Bugs and malfunctions will always be found after launch\footnote{In the pharmaceutical industry, this is known as the \emph{Phase IV} of a clinical trial.}.
	One way to discover what triggers the malfunction is to try and replicate it in a controlled environment, i.e., in a designed experiment.
	A second, possibly more efficient way, is to collect post-sale failure data.
	This is why your software asks you to send crash reports.
	This is why plane and engine manufacturers equip their engines with sensors that constantly usage reports to the engine manufacturer headquarters.
\end{example}

Observational studies classify into:
\begin{description}
	\item[Cross Sectional] Where each sample unit, i.e. individual, is observed in a single point in time.
	\item[Prospective] Where each sample unit is observed on various occasions in the future, until some outcome occurs.
	\item[Retrospective] Where measurements are recovered on sample units from the past, only if some outcome has occurred.
\end{description}

\emph{Cross sectional} studies are the simplest type of observational study, \aka a \emph{simple random sample}.
There really isn't much to say about them, except a vivid warning against causal inference from such studies (see \ref{sec:causal}).\marginnote{[Simple Random Sample]}

When units are followed over time, we call this a \emph{longitudinal study}, or \emph{panel data}, or \emph{repeated measures}.\marginnote{[Longitudinal Study]}
Because measurements are taken within units, social scientists also call this \emph{multilevel analysis}.
If the emphasis is on the variability between units, this is also known as \emph{variance componnets analysis}.
The idea of following units over time has indeed many names. This is because it is such a fundamental practice.
An important distinction we make is whether the data is collected forward in time, or backward.

\emph{Prospective studies}, or \emph{cohort study}, sampling units are selected and then measured over time alongside covariates.
As such, in prospective studies we follow units into the future, i.e., forward in time.

For the study of rare outcomes, prospective studies may be very wasteful.
\emph{Retrospective studies}, \aka \emph{case-control studies} remedy this by collecting units with the desired outcome, and only then matching them with controls and recovering their event history.
We thus follow units back in time, thus the name, retrospective study.
In the words of \cite{cox_principles_2011}: {\em ``\dots a prospective study looks for the effects of causes whereas a retrospective study examines the causes of effects``}.\marginnote{Case-Control Study}
This is clearly not a random sample in the population, so it is unclear that inference in retrospective studies is valid.

To demonstrate the difference in inference between the sampling schemes consider a binary outcome $Y\in\set{0,1}$ and a binary covariate $X\in\set{0,1}$.
In terms of the post-sale problem in Example~\ref{ex:post-sale}, $X$ can stand for a device's version indicator, and $Y$ a failure indicator.
The effect of $X$ on $Y$, can be quantified by the \emph{odds ratio}\footnote{We need to define such a measure because simple measures like the correalation coefficient, or the regression coefficient are, inadequate for discrete outcomes such as fail/no-fail.}.
To define the odds ratio, we first define the \emph{odds}.

\begin{definition}[Odds]
	The odds of two events, $Y=1$, and $Y=0$, is defined as
	\begin{align}
	Odds:= P(Y=1)/P(Y=0).
	\end{align}
\end{definition}
The \emph{odds}, just like \emph{probabilities} is an uncertainty measure.
It quantifies the ratio of ``successes'' per ``fail''.
In some communities, the odds are actually more popular than probabilities (think horse racing).
In our post-sale example, the odds measures the ratio of working devices per faulty device, and not per \emph{produced} device like the probability measure.

The odds ratio is the ratio of odds under two conditions:
\begin{definition}[Odds Ratio]
	\begin{align}
	OR := \frac{P(Y=1|X=1) / P(Y=0|X=1)}{P(Y=1|X=0) / P(Y=0|X=0)}.
	\end{align}
\end{definition}
Just in order to emphasize what the OR is \textbf{not}, we also define the \emph{relative risk}, which is perhaps a more natural effect measure:
\begin{definition}[Relative Risk]
	\begin{align}
	RR := \frac{P(Y=1|X=1)}{P(Y=1|X=0)}.
	\end{align}
\end{definition}
You may verify that if $X$ has not effect on $Y$, i.e. $P(Y|X)=P(Y)$, for all $X$, then both $OR=1$ and $RR=1$.
We now show how the $OR$ can be estimated from the various designs.

\subsection{Prospective Study}
A prospective study means that we decide how many samples of each version to take and wait until some fail, and then compare the failure probabilities.
Formally this means we directly estimate $P(Y=1|X=1)$ and $P(Y=1|X=0)$.
Estimating $OR$ is thus trivial.

\subsection{Retrospective Study}
A retrospective study means that we sample a fixed number of working and broken devices.
We can thus estimate the frequency of each version given the outcome: $P(X|Y)$.
To relate it to the $OR$, we will need to relate the estimable $P(X|Y)$, to the desired $P(Y|X)$ via Bayes' Theorem:
\begin{align}
\label{eq:bayes}
	P(Y|X)=P(X|Y)P(Y)/P(X)
\end{align}

Several applications of Eq.(\ref{eq:bayes}) yields that
\begin{align}
\label{eq:retrospective_OR}
	OR 	&= \frac{P(X=1|Y=1)/P(X=1|Y=0)}{P(X=0|Y=1)/P(X=0|Y=0)}.
\end{align}
Eq.(\ref{eq:retrospective_OR}) is very good news.
It means that even if we sample retrospectively, we can estimate the OR as if we sampled prospectively.

\begin{think}
	Can we estimate other effect measures such as the RR from a retrospective study?
\end{think}

\begin{extra}[Effect parameters as distances]
	The OR can be estimated from both prospective and retrospective studies because it is a \emph{symmetric} effect measure. As such, it can be seen as a \emph{distance} between distributions.
	The RR, while more easily interpretable, is not symmetric, and thus cannot be thought of as a distance.
\end{extra}

\subsection{Cross-Section}
A cross-section study is \textbf{not} a probable sampling scheme for our post-purchase example.
It implies that we sample randomly in a ``pile'' of devices of all versions and states.
We thus get an estimate of the proportion of each $X,Y$ combination, and need to relate it to the difference in failure probabilities.
Formally, we estimate all $P(Y,X)$'s.
Via Bayes' Theorem we then have
\begin{align*}
	OR = \frac{P(Y=1,X=1)P(Y=0,X=0)}{P(Y=1,X=0)P(Y=0,X=1)},
\end{align*}
which shows that the OR can indeed be estimated from a cross-section study.

\begin{think}
	Can you estimate the RR from a cross section study?
\end{think}

\begin{think}
	In regression analysis we typically analyze bias and variance while conditioning on the design matrix, $X$.
	Is such a practice consistent with the designed experiment or the observational view of a study?
	If observational, is it cross sectional? Prospective? Retrospective?
\end{think}

\subsection{Respondent Driven Sampling}

\emph{Survey samples} are a type of empirical studies.
In survey samples, like the ones conducted by the National Bureau of Statistics, the goal of the sample is to use a small sample to estimate some quantity from a larger, but \emph{finite}, population.
Simple random sampling in a finite population can be seen as an instance of an observational study: a cross-sectional one.
Sampling in finite populations, however, is a type of observational study that has its own challenges. In particular, because we would like, if possible, that each and every sample we take is representative of the population (and not only on average, like in infinite populations).

\emph{Respondent driven sampling} is currently, one of the most advanced sampling schemes in finite populations.
The next example gives an industrial application of a respondent-driven-sample, and the challenges it introduces.

\begin{example}[Respondent Driven Sampling]
	\label{ex:RDS}
	Consider the following chain-referral sampling scheme.
	A Facebook questionnaire is passed by rewarding respondents some credit if they name other respondents.
	The probability of being sampled is thus not fixed for everyone, but rather a function of the number of your Facebook friends.
	Now imagine the questionnaire is meant to estimate the public awareness to product ZZZ.
	It seems reasonable to assume that the more friends you have, the more likely you are to be sampled, and also, the more likely to be familiar with ZZZ.
	This sampling scheme will thus lead to ZZZ being more popular in your sample than in the whole population.
\end{example}

If the sampling scheme in Example~\ref{ex:RDS} is not accounted for at the analysis stage, we will clearly have biased popularity estimates.
To deal with the over-representation in our sample we will want to weight each observation by its probability of being sampled. The following estimator does just that, and yields an unbiased estimate of the population's expectation.
\begin{definition}[Horowitz Thompson Estimator]
	\label{def:horowitz-thompson}
	If observation $i$ has probability $\pi_i$ of being sampled and response $y_i$, then an unbiased estimate of the population mean of $y$ is given by
	\begin{align}
	\frac{\sum_{i \in S} y_i/\pi_i}{\sum_{i \in S} 1/\pi_i},
	\end{align}
	where $i \in S$ means that $i$ has actually been sampled.
\end{definition}
You may verify that the Horowitz-Thompson estimator in Definition~\ref{def:horowitz-thompson} return the usual sample mean, if the sampling is simple, thus all $\pi_i$ are equal.

\subsection{Causal Inference}
\label{sec:causal}

Causal inference is typically the Holy-Grail of empirical research.
Causality may be inferred from a designed experiment, or from an observational study.
As already stated on several occasions, inferring causality from observational studies is much riskier than from designed experiments.
This is because in observational studies we have two sources of uncertainty, that are canceled by the randomization mechanism of designed experiments:
\begin{enumerate}
	\item \textbf{Non-controlled variables} that affect both the covariates and the outcome (think of stress in the smoking example in Extra Info~\ref{ex:smoking}).
	\item \textbf{Reverse Causality}: if a correlation is found between A and B, does A cause B, or B cause A?
\end{enumerate}
It is not impossible to infer causality from observational studies, but the price to pay is added assumptions.
The first, and probably harmless assumption is that causality works forward in time.
For this reason a prospective study is a good way to go.
Retrospective studies also allow causal inference, but with some more assumptions.
Even cross-section studies allow causal inference, but many assumptions will be required.
A set of assumptions, is the underlying scientific theory.
In physics this is a helpful guideline, but in the social sciences, this is merely a call for endless debates.
See the Bibliographic notes section for some references on causal inference from observational studies.

\section{Bibliographic Notes}
Our general discussion of observational studies, designed experiments, causal inference, etc, is derived from \cite{cox_principles_2011}.
For an overview of DOE see \cite{cox_theory_2000}, \cite{mason_statistical_2003}, \cite{everitt_cambridge_2010}.
Another nice and freely available resource is a Penn State course on the topic\footnote{\url{https://onlinecourses.science.psu.edu/stat503/node/1}}.
Some seminal references in the field include \cite{fisher_design_1960} and \cite{box_statistics_1978}.
A good textbook on sampling is \cite{thompson_sampling_2012}.
For the respondent driven sampling see \cite{berchenko_modeling_2013}.
For causal inference in see \cite{rosenbaum_observational_2002}.
For optimal designs see \cite{pukelsheim_optimal_1993}.
For analysis of data: linear models, ANOVA, etc.\ there are endlessly many books. This author's recommendations include \cite{hocking_analysis_1985}, \cite{greene_econometric_2003}.
For the relation between factorial designs and coding theory in computer science, see \cite{hill_first_1986}.
For design and analysis of computer experiments, see \cite{sacks_design_1989} or \cite{santner_design_2013}.


\section{Implementation} \label{implementation}

\subsection{Polynomial}

At first, we tried to use \textit{vector} data structure from the \textit{std}
library to implement polynomial class since it can handle coefficients
dynamically. However due to the time consuming dynamic memory allocation process,
it has been changed to static process.

We use template class with an integer template $n$ to represent the maximum
possible degree of polynomial. Which means although a polynomial class with
template $n$, it's highest degree could be $n1$ where $n1 \leq n$. Although in
this way, developers need to handle the possible change over the template when
writing some operators, like $*,+$, programs can avoid dynamic memory
allocation, which may save a lot of running time.

%\subsection{Replace $x$ with $x+a$}

%Changing $P(x)$ to $P(x+a)$ in a polynomial is a very common operation in both
%methods, since both methods need to use the coefficients of $P(x+a)$ to get the
%sign variance.

%At first, we tried a very primitive idea for this process. Initialize a very
%simple polynomial $p = x+ a$. Then multiply it by coefficients and increase its
%order one by one. Add them together at last to get transformed polynomial.

%In order to optimize this process, we apply Taylor expansion. This process is
%also called Taylor shift\cite{Taylor}. It runs as appling Taylor expansion at
%point $a$ with $\Delta x = x$. Then we have:

%$$
%P(x+a) = P(a) + P'(a) *x + ... + \frac{1}{n!}P^{(n)}(a)*x^n
%$$

%This process avoids multiplication of two polynomials. Multiplication needs
%$O(n^2)$ operations in our implementation. In order to make sure Taylor
%expansion can improve the performance of the program, we made some experiments
%to compare the computation time of these two implementations. Results are in
%table~\ref{tb1}.  Coefficients of test polynomials are generated randomly from
%$-1000$ to $1000$. Since this project does not pay attention to polynomials with
%degrees higher than 6, we only test polynomials with degree from 3 to 6.

%\begin{center}
%\label{tb1}
%\begin{tabular}{ |c|c|c| }
 %\hline

 %Degree  & Tylor Expansion   & Original Method\\

 %\hline
 %3   & 1876 $ns$  & 2272 $ns$\\
 %4   & 2640 $ns$  & 3439 $ns$\\
 %5   & 3307 $ns$  & 4910 $ns$\\
 %6   & 4574 $us$  & 7103 $us$\\
 %\hline
%\end{tabular}
%\end{center}

%As shown above, the running time of polynomial shifting with Taylor Expansion is
%about $25\%$  faster than the primitive one in our implementation.

\subsection{Approximate GCD}

Errors in floating point computation are unavoidable. Therefore, it is hard to
check if one polynomial is zero or not. Greatest common divisor and Square-free
decomposition can not perform perfectly all the time.

In order to solve this error problem, we tried tolerance. If a number
smaller than this tolerance, it will be treated as zero. At first we used fixed
tolerance, which cannot perform well as the magnitude of the largest coefficient
grows.

Then we tried the method introduced by Matu-Tarow\cite{Approximate}. It suggests
to turn polynomials \textit{regular} before performing $GCD$. A polynomial is
\textit{regular} means its leading coefficient is $O(1)$ and any other
coefficients either $O(1)$ or $0$. The notation $O(c)$ not same as the Landau’s
symbol and it means a number of approximately the same magnitude as $c$. Any
univariate polynomial can be transformed to \textit{regular} by scaling
$P\rightarrow \xi P $ and $x \rightarrow  \eta x$.

However, it is hard to define the $\eta$ $\xi$ when transforming a
polynomial to \textit{regular}. Besides that, if we transform $x$ to $\eta x$,
roots will be changed accordingly. And the precision of the algorithm will be
changed. Furthermore, this process will introduce extra computation and make the
program less efficient.

\subsection{Interval arithmetic}

Due to the problems mentioned in the last part of the previous subsection, we discard
approximate GCD and use \textit{interval arithmetic}. Interval arithmetic represents each
value as a range of possibilities so that we can keep track of errors from floating
point computation in algorithms. We use interval arithmetic implemented in
\textit{boost} library.

For the numbers that can be represented by a computer perfectly, no error might be
introduced through computation. However, if we get some numbers that not
suitable for a computer to store, errors might appear. For example, $\frac{1}{3}$
might be represented as $0.33333333333 \pm \epsilon$, where $\epsilon$ might as small
as $1e^{-17}$.

%Another approach to handle instability of $GCD$ and Yun's algorithm is interval
%arithmetic, which was first introduced in numerical analysis by Moore in 1960’s.
%The main purpose of introducing interval computation is to increase the
%stability of floating point computation.

%With above problem, I discard the methods introduced by
%Matu-Tarow\cite{Approximate} and applied \textit{interval arithmetic} to the
%project. Every exact number in project has been changed to a interval that
%represents a set of possible values. If zero in such interval, this number will
%be seen as zero.

With interval arithmetic, the computation has been doubled since every
computation need to calculate both ends of a interval. However, since it can
handle the error from floating point computation, our program becomes more stable.

Although interval arithmetic can not make our program runs perfectly on all of
cases, using interval arithmetic allows us to observe how errors propagated
through computation. With this observation, we can figure out in what
condition our program might fail. This part will be discussed in
Section\ref{analysis}.


\chapter{Conclusion and Discussion}
\label{chapterlabel4}

In this chapter, we summarize the work carried out during this dissertation and discuss the results obtained in the experiments. Then, we expose the drawbacks and limitations found during the project. Finally, we describe future directions to improve the proposed approaches.
% that solve the recommender task.

\section{Summary}

By and large, we have demonstrated that recent advances in deep reinforcement learning allow us to define agents that learn to generate good recommendations to users by reducing the complexity of acting over a large set of items. Moreover, the model is also able to generalize and solve the cold-start problem. After defining the recommender task under the reinforcement learning domain, we describe a baseline model using the Deep Deterministic Policy Gradient algorithm with a k-nearest neighbours policy for action space reduction, and trained it using a content-based and a collaborative-filtering representation of items. Additionally, we propose a new policy architecture based on a Factorization Machine model for Top-K ranking recommendation.

To evaluate the performance of each model under the recommender task we monitor the average return obtained by the reinforcement agents over the learning episodes, as well as the average precision along user sessions. Then, a quantitative and qualitative analysis of the results demonstrated that the FM-policy architecture outperforms the baselines and presents an slighty better progression on its performance, while the DRL-kNN-CB model has much less speedup during the first episodes but after 5000 steps it starts to stabilize. As a result, the DDPG algorithm using any of the above policies is able to perform well under the recommender system domain, nevertheless, a factorization machine model helps to build a stronger policy than the k-NN algorithm.f

During the evaluation of the policy architectures, we noticed about certain limitations on the simulated environment that we define to emulate the user behaviour under the recommender task: (1) the transition matrix and ratings predictor where constructed using the basic techniques exposed in Chapter \ref{sec:chapterlabel4} which would tend to fail under scenarios with high sparsity; and (2) there were cases where the same item was recommended more than twice under the same user session. Even if it is not wrong, users typically want to find different items than the ones previously recommended. Therefore, novelty and/or diversity should be taken into account during recommendations either in the simulated environment or in the policy definition.

\section{Future Work}

Possible next steps can attempt to define a more robust definition of the simulated environment under the OpenAI gym framework, specially in terms of modelling the user behaviour, or improving the ratings predictor for scenarios under high sparsity (e.g. using a stronger model like the FM with MCMC inference \cite{rendle2012factorization}). Altertatively, the model could be connected to a live recommender system in order to test its performance under a real scenario.

Further research may focus on testing other deep reinforcement learning algorithms under the recommender task, such as deep neural networks with Monte Carlo tree search \cite{silver2016mastering} or the Normalized Advantage Functions (NAF) model \cite{gu2016continuous} which outperforms the actor-critic based methods such as DDPG. Finally, the FM-policy could also be redefined to consider a diversity parameter when selecting the next item to recommend.


\chapter{Singularities}

\textbf{Very} rough draft.

\section{Singularities funcoids: some special cases}
We attempt to prove that $\up z$ is closed regarding finite intersections.

For consideration of this, let's consider two special cases (first of which is a specialization of the second).

Let $\mu=\nu$ be the natural proximity on real numbers $\mathbb{R}$.

Let $\Delta$ is the entourage filter of zero.

1. $z=\Delta\times^{\mathsf{FCD}}\Delta$.

2. $z=\nu\circ (\uparrow^{\mathsf{FCD}} f)|_{\Delta}$ for an arbitrary function $f:\mathbb{R}\rightarrow\mathbb{R}$.

(1) is [[also formulated in elementary terms|http://math.stackexchange.com/questions/568513/is-a-set-closed-under-finite-intersections-about-filters]] (without using funcoids).

These two above conjectures are shown to be false by a counter-example in [[this blog post|http://portonmath.wordpress.com/2013/12/18/a-negative-result-on-a-conjecture/]]. It is a discouraging result as it seems from it the plain funcoids can't be used for the multilevel theory of singularities.

\section{Using plain funcoids}
This way if we succeed is the best way to create metasingular numbers because, it (if we succeed) involves just funcoids not some fancy generalization of funcoids.

Approximate definition of "singularity level": //Singularity level// is a transitive, $T_2$-separable endofuncoid.

Now define the funcoid $\nu_{i+1}=\operatorname{SLA}(\nu_i)$:

$\operatorname{Ob}(\nu_{i+1})$ is defined as the set of all generalized limits (having fixed $\mu$, $\nu$, and $G$).

$X \mathrel{[ \nu_{i+1}]^{\ast}} Y \Leftrightarrow \exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z \exists x \in \bigcup X, y \in \bigcup Y : x, y\sqsubseteq K$.

The trouble is to prove that the funcoid $\nu_{i+1}$ exists (is really a funcoid).

$\neg(X \mathrel{[ \nu_{i+1}]^{\ast}} \emptyset)$ and $\neg(\emptyset \mathrel{[ \nu_{i+1}]^{\ast}} Y)$ are obvious. We need to prove
$$I\cup J \mathrel{[ \nu_{i+1}]^{\ast}} Y \Leftrightarrow I \mathrel{[ \nu_{i+1}]^{\ast}} Y \vee J \mathrel{[ \nu_{i+1}]^{\ast}} Y$$ and
$$X \mathrel{[ \nu_{i+1}]^{\ast}} I\cup J \Leftrightarrow X \mathrel{[ \nu_{i+1}]^{\ast}} I \vee X \mathrel{[ \nu_{i+1}]^{\ast}} J.$$

Let's attempt to prove the first of the above equations (the second is dual).

$I \cup J \mathrel{[ \operatorname{SLA} ( \nu)]^{\ast}} Y \Leftrightarrow \\ \exists z
\in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z \exists x \in \bigcup I \cup \bigcup J, y \in \bigcup Y :
x, y \sqsubseteq K \Leftrightarrow \\
\exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z : ( \exists x \in \bigcup I \cup
\bigcup J : x \sqsubseteq K \wedge \exists y \in \bigcup Y : y \sqsubseteq K) \Leftrightarrow \\
\exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z \exists x \in \bigcup I \cup \bigcup J :
x \sqsubseteq K \wedge \\ \exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z
\exists y \in \bigcup Y : y \sqsubseteq K \Leftrightarrow \\
?? \\
\exists z \in \bigcup \operatorname{Ob} \nu : ( \forall K \in \operatorname{up} z \exists x \in \bigcup I : x
\sqsubseteq K \vee \\ \forall K \in \operatorname{up} z \exists x \in \bigcup J : x \sqsubseteq
K) \wedge \exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z \exists y \in
\bigcup Y : y \sqsubseteq K \Leftrightarrow \\
( \exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z \exists x \in \bigcup I : x
\sqsubseteq K \vee \\ \exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z
\exists x \in \bigcup J : x \sqsubseteq K) \wedge \exists z \in \bigcup \operatorname{Ob} \nu \forall
K \in \operatorname{up} z \exists y \in \bigcup Y : y \sqsubseteq K \Leftrightarrow \\
( \exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z \exists x \in \bigcup I : x
\sqsubseteq K \wedge \exists z \in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z
\exists y \in \bigcup Y : y \sqsubseteq K) \vee \\ ( \exists z \in \bigcup \operatorname{Ob} \nu \forall
K \in \operatorname{up} z \exists x \in \bigcup J : x \sqsubseteq K \wedge \exists z \in
\bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z \exists y \in \bigcup Y : y \sqsubseteq K)
\Leftrightarrow \\
( \exists z \in \bigcup \operatorname{Ob} \nu : ( \forall K \in \operatorname{up} z \exists x \in \bigcup I :
x \sqsubseteq K \wedge \forall K \in \operatorname{up} z \exists y \in \bigcup Y : y
\sqsubseteq K)) \vee \\ ( \exists z \in \bigcup \operatorname{Ob} \nu : ( \forall K \in \operatorname{up}
z \exists x \in \bigcup J : x \sqsubseteq K \wedge \forall K \in \operatorname{up} z \exists y
\in \bigcup Y : y \sqsubseteq K)) \Leftrightarrow \\
( \exists z \in \bigcup \operatorname{Ob} \nu : ( \forall K \in \operatorname{up} z : ( \exists x \in
\bigcup I : x \sqsubseteq K \wedge \exists y \in \bigcup Y : y \sqsubseteq K))) \vee \\ \exists z
\in \bigcup \operatorname{Ob} \nu \forall K \in \operatorname{up} z : ( \exists x \in \bigcup J : x
\sqsubseteq K \wedge \exists y \in \bigcup Y : y \sqsubseteq K) \Leftrightarrow \\
I \mathrel{[ \operatorname{SLA} ( \nu)]^{\ast}} Y \vee J \mathrel{[ \operatorname{SLA} (
\nu)]^{\ast}} Y$.

To finish the proof we need to fulfill ?? in the above formula. For this it's enough to prove

$\forall K \in \operatorname{up} z \exists x \in \bigcup I\cup \bigcup J : x \sqsubseteq K \Rightarrow \\ \forall K \in \operatorname{up}
z \exists x \in \bigcup I : x \sqsubseteq K \vee \forall K \in \operatorname{up} z \exists x
\in \bigcup J : x \sqsubseteq K$.

If $z=\uparrow Z$ is a principal funcoid, then

$\forall K \in \operatorname{up} z \exists x \in \bigcup I\cup \bigcup J : x \sqsubseteq K \Rightarrow \\
\exists x \in \bigcup I\cup \bigcup J : x \sqsubseteq z \Rightarrow \\
\exists x \in \bigcup I : x \sqsubseteq z \vee \exists x \in \bigcup J : x \sqsubseteq z \Rightarrow \\
\forall K \in \operatorname{up}
z \exists x \in \bigcup I : x \sqsubseteq K \vee \forall K \in \operatorname{up} z \exists x
\in \bigcup J : x \sqsubseteq K$.

Following the idea of [[the proof in this math.stackexchange.com question|http://math.stackexchange.com/questions/562908/an-implication-involving-filters\#562974]] it is easy to show that our implication is true if $\operatorname{up} z$ is closed regarding finite meets. See [[this page|Singularities funcoids: some special cases]] for attempts to set it true.
The question is whether our statement holds for non-principal funcoids. Or is there a counterexampe?

\section{Singularities funcoids: special cases proof attempts}
To prove that $\operatorname{GR} ( \Delta \times^{\mathsf{FCD}} \Delta)$ is closed under finite intersections, it's enough to prove that for every $f \in \operatorname{GR} ( \Delta \times^{\mathsf{FCD}} \Delta)$ there is a positive $\varepsilon$ such that $\forall x \in ] - \varepsilon ; \varepsilon[ : f x \in \Delta$.

Really, under this assumption:

For $g \in \operatorname{GR} ( \Delta \times^{\mathsf{FCD}} \Delta)$ exists $\zeta > 0$ such that $\forall x \in ] - \zeta ; \zeta[ : g x \in \Delta$. Let $\eta = \min \{ \varepsilon, \zeta \}$. So $\forall x \in ] - \eta ; \eta[ : ( \langle f \rangle x \in \Delta \wedge \langle g \rangle x \in \Delta)$ and so $\forall x \in ] - \eta ; \eta[ : \langle f \cap g \rangle x \in \Delta$ that is $\forall x \in ] - \eta ; \eta[ : \langle \uparrow^{\mathsf{FCD}} ( f \cap g) \rangle^{\ast} \{ x \} \sqsupseteq \Delta$ and consequently $f \cap g \in \operatorname{GR} ( \Delta \times^{\mathsf{FCD}} \Delta)$.


% Theory introduction goes here

\section{Theory}
\subsection{The Ising Model} \label{sec:ising-model}
\input{theory/ising_model.tex}

\subsection{Logistic regression}
\input{theory/logistic_regression.tex}

% \subsection{Cost Functions}
% \input{theory/cost_functions.tex}

\subsection{Optimization}
\input{theory/optimization.tex}

\subsection{Regularization}
\input{theory/regularization.tex}

\subsection{Neural Networks}
\input{theory/neuralnetworks.tex}


\chapter{Summary}
\label{chap:conclusions}

The purpose of this thesis was to give an overview of arbitrage-free pricing methodology and affine short-rate processes used in interest rate modeling and credit risk. These kind of models have a long history starting from the late 1970's. Their heyday was pre-financial crisis of 2007-08. After the crisis, many significant changes have occurred in interest rate markets, for example:
\begin{itemize}
	\item IBOR rates are not anymore considered as riskless and relevant spreads have widened.
	\item Credit adjustments are required for unsecured positions.
	\item Multi-curve pricing has been the industry standard at least for linear products such as swaps.
	\item Extraordinary monetary policies of central banks have caused negative interest rates, which were earlier considered impossible. As such, the possibility of negative rates in Va\v{s}\'{i}\v{c}ek--model was considered a flaw earlier.
\end{itemize}
Although short-rate models have been surpassed by market models in practice, it is an interest question consider how short-rate models fare in the current market structures.

Short-rate models have several theoretical short-comings. The first is conceptual, there is no actual instantaneous short-rate. It is purely theoretical concept created to explain how the interest rates are formed. Although having an unobservable process as a main ingredient of a theory is troubling, it can be forgiven if implied theoretical structure is otherwise logically sound and it can produce accurate results. The viability of Black-Scholes option pricing methodology is based on the assumption that the future volatility of the stock process can be accurately inferred, even if it is not actually observable. Same can be true for short-rate modeling of interest rates. If the observed term structure is coherent with the implications of a hypothesized short-rate process, then the model might be useful even if the model might be fundamentally wrong.

The second main theoretical short-coming of short-rate modeling is the fact that it is mainly concerned of a single point, the next infinitesimal future time-step. As such, it is not fat-fetched to hypothesize that short-rate models have hard time to explain complex temporal evolution of interest rate curve. For example, we have demonstrated that for affine short-rate models with one factor, the long-rates are perfectly correlated. This is a severe limitation but it can be mitigated with the introduction of multiple factors, time-varying parameters or  technique of dynamic extension.

However, short-rate models are not without merits. They are conceptually simple to understand. Affine models are analytically tractable with explicit analytical bond pricing formulas. Some models even have explicit analytical bond option pricing formulas which can be converted to the price caps and floors.

The calibration of models to the market data was inconclusive in the sense that we did not achieve consistent results. For simple models without credit risk, although there were some precise calibrations, no model was consistently accurate. Since we employed stochastic optimization algorithms for calibration, there is no certainty that global minimums were found. The curse of dimensionality makes the optimization problem very hard for multi-factor models. Thus a bad fit does not indicate that the model is unsuitable for the observed data. Since single-factor models have a manageable number of parameters, we could expect optimization to be fairly dependable. For $A(0,1)+$--model, the alternative calibration replicated the original parameter values almost perfectly. For $A(0,1)+$--model, the alternative calibration produced significantly different parameter values but the accuracy was very similar. Therefore we can infer with reasonable confidence that $A(0,1)+$ and $A(1,1)+$--models can not necessarily fit post-crisis term-structures. Since some of the considered multi-factor models of family $A(M,N)+$ offered decent accuracy, we believe that two or three factor models could be used to fit the recent observed data. As the alternative calibrations led to significantly different parameters and calibration errors, the inference about model quality of multi-factor models of type $A(M,N)+$ is not reliable.

Since the calibration data included maturities ranging from overnight rate to 30-year rate, the observed rate structure is complex. Calibration to subset of these maturity ranges will likely produce significantly better fits. This reasoning is supported results of the dynamic calibration of Euribor rates ranging from one week to annual maturity, which generally give significantly better accuracy.

For simple models with credit risk, the calibration errors were large. However, calibration was only attempted with recent data and only models with two-factors were considered. As we had seen, single-factor models did not perform well in this environment for interest rate curves and these factors had to explain both the interest rate and spread curves. It is probably that models with more factors could work better.

The calibration methodology employed had severe short-comings. Although differential evolution has the desired ability to explore the optimization space, it probably wasted lots of function calls to explore infeasible regions. On the other, since the alternative optimizations tended not to converge original points, it seems that meta-parameters for the optimization were misspecified. Also it is not clear if differential evolution is the best choice for this kind of calibration. Particle swamp optimization or simulated annealing might have been better alternatives. The large number of solutions near the optimization borders suggest that those borders might have been inappropriate. On the other hand. Since the shifts were stopped at borders, the borders were also likely to be hit during optimization tries.

It could be interesting to test how well these affine models and their dynamic extensions compare against more modern models such as SABR when they are calibrated to recent volatility structures. Since descendants of LIBOR models are based on the assumption of log-normal distribution, negative interest rates causes problems that require unnatural solutions such as shifts or normality assumptions that may lower model quality. Negative rates are possible for affine models with Gaussian factors, but this probably does not compensate for inferior volatility fabrics of these models.


\documentclass[a4paper]{article}
\begin{document}
\title {An Incomplete Scan Matching Tutorial}
\author{Giorgio Grisetti}
\maketitle
In this document we provide an explanation of the basic concepts for developing a
simple maximum likelihood mapper for laser data.
We first introduce the notation, then we describe how to compute an occupancy grid map, given a known set od poses. Subsequantly we present a simple gradient based scan matchgin algorithm.

\section{Notation}
\begin{itemize}
\item $x_t$: robot pose at time $t$.
\item $z_t=\underbrace{(z^1_t,\cdots,z^n_t)}_\mathrm{beams}$:
scan taken at time $t$.
\item $u_t$: odometry movement which brings the robot from $x_{t-1}$ to $x_{t}$, $t$.
\item $m_t=f_t(x,y)\rightarrow[0,1]$ occupancy grid map which can be seen as a function that maps each point in the probability of being occupied.
\end{itemize}

\section{The Scan Matching Problem}
A scan matching algorithm works in two steps:
\begin{itemize}
\item it determines the most likely pose, given the actual measurements and the previously build best map:
\begin{equation}
\hat x_t = \mathop{argmax}_{x_t}(p(x_t|z_t, \hat x_{t-1}, m_{t-1}, u_t))
\nonumber
\end{equation}
\item it computes the next step map given the previously built one, the corrected pose and the range reading:
\begin{equation}
p(m_t|m_{t-1}, \hat x_t, \hat z_t)
\nonumber
\end{equation}
\end{itemize}
In the following we present two simple approaches for performing these two steps.

\section{Frequancy Based Occupancy grids}
An occupancy grid is a discrete world representation. It describes the world as a matrix, whose cells represents the probability of being occupied. The cells are considered independent.
Here we present a simple algorithm for updating an occupancy grid, based on a frequentist approach.

For each cell of the map $m^{x,y}$ we keep two numbers: the number of times that cell has been visited $v^{x,y}$, and the number of times that cell has been found occupied $b^{x,y}$.

Let $m_{t-1}$ be the map at the previous time step, $x_t$ the robot pose at time $t$ and $z_t=(z^1_t,\cdots,z^n_t)$ the reading.
We first consider the range reading translated according to the current robot pose
\begin{equation}
\hat z_t=(\hat z^1_t,\cdots, \hat z^n_t)=\hat x_t \oplus z_t
\end{equation}
Each beam $z^i_t$ can be described with its endpoint $\hat p^i_t$.
The cells in the map spanned by such a beam lies on the segment $(\hat x_t, \hat p^i_t)$. All the cells wich are before th endpoint are detected as free, while the endpoint is occupied.

The we can incrementally update the cells of the map for each beam, in the following way:
\begin{equation}
m^{x,y}_t=(b^{x,y}_t,v^{x,y}_t)=
\left\{
\begin{array}{cc}
(b^{x,y}_t+1,v^{x,y}_t+1) & \mathrm{if\;occupied}\\
(b^{x,y}_t,v^{x,y}_t+1) & \mathrm{if\;free}
\end{array}
\right.
\end{equation}
The probability that a cell is pccupied is
\begin{equation}
p(m^{x,y}_t)=\frac{b^{x,y}_t}{v^{x,y}_t}
\end{equation}

\newpage
\section{A simple gradient descent scan matcher}
Here we describe the simple scamnmatcher you find in this software bundle.
The basic principle of this algorithm is to perform a gradinent descent search,
on a score function $s(x,z,m)$ of the pose, th reading and the map.
The overall algorithm works as follows:
\begin{itemize}
\item $bestPose=x_\mathrm{init}$
\item $bestScore=s(bestPose,z,m)$
\item $searchStep=initialSearchStep$
\item $iterations=0$
\item while (!$iterations<maxIterations$)
	\begin{itemize}
	\item $maxMoveScore=bestScore$
	\item $bestMovePose=bestPose$
	\item for $move$ in ($Backward, Forward, Left, Right, RotateLeft, RotatrRight$)
		\begin{itemize}
		\item  $testPose=computePose(bestPose, move)$
		\item  $score=s(testPose,z,m)$
		\item if ($maxMoveScore<score$)
			\begin{itemize}
			\item $maxMoveScore=score$
			\item $bestMovePose=testPose$
			\end{itemize}
		\end{itemize}
	\item if ($bestScore<maxMoveScore$)
		\begin{itemize}
		\item $bestScore<maxMoveScore$
		\item $bestPose=bestMovePose$
		\end{itemize}
	\item else
		\begin{itemize}
		\item $searchStep=searchStep/2$
		\item $iterations++$
		\end{itemize}
	\end{itemize}
\end{itemize}

Here a critical issue is how to compute the score function.
A simple approach is to consider a score function as  the sum of the score of the single beams score.
\begin{equation}
s(x,z,m)=\sum_i s(x,z^i,m)
\nonumber
\end{equation}
Our choice is to use a simple endpoint based model. Given a pose $x$ and a reading we consider individually the single beams $z^i$.
\begin{itemize}
\item We compute the cell of the map in which the beam endpoint falls. This can be done by translating the endpoint according to $x$ as  $\hat z^i= x \oplus z^i$
\item We compute the cell of the map in which th endpoint falls.
\item We found the busy cell $\hat m^{x,y}$ in the map which is closest to $\hat z^i$.
\item We compute the score as a function which decreases with the increase of the distance among $\hat z^i$ and $(x,y)^T$. The squared distance $d^2$ can be computed as $d^2=(\hat z^i- (x,y)^T))^T\cdot(\hat z^i- (x,y)^T))$
\begin{equation}
s(x,z^i,m) = e^{d^2/\sigma}
\nonumber
\end{equation}
\end{itemize}
As an additional optimization for each map cell $m^{x,y}$ we can consider as its center the center of mass of the points falling in that cell.
\end{document}


\chapter*{List of Symbols} \label{symbols}

\hspace*{-0.6in}
\begin{tabular}{@{}C{1in} L{5.25in}}
$\phi_p$ & Horizontal stress \\
$\beta$ & Angle between the normal and horizontal planes \\
$\pi$ & Pi \\
$i$ & Imaginary unit \\
$P_2$ & Universal parabolic constant \\
$t_i$ & Time at step $i$ \\
$\Delta\mu$ & Change in energy \\
\end{tabular}


\documentclass[a4paper]{article}
%\usepackage{a4wide}
\usepackage{oz}
\usepackage[final]{nips_2017}

\title{Wumpus World Formal Specification}
\author{Michael Papasimeon}
\date{}

\begin{document}
\maketitle

\section{Introduction}
This article describes a very (incomplete) formal specification of the Wumpus world game.
The specification is defined in the Z language. The purpose of the
specification is to formally define different types of interactions
between the the environment in which the Wumpus lives in, and the way it
interacts with an agent that is situated in this environment.

\section{Definitions and Utility Functions}
A function $uniform$ is defined that returns a uniform random number
between $x$ and $y$.
\begin{axdef}
	uniform : \nat \cross \nat \tfun \real
\ST
	\forall x,y : \nat | y \geqslant x
		\dot x \leqslant uniform(x,y) \leqslant y
\end{axdef}

A function $random$ is defined that returns a random natural number in
the range specified by the parameters to the function.
\begin{axdef}
	random : \natone \cross \natone \tfun \natone
\ST
	\forall x,y : \natone | y \geqslant x
		\dot random(x,y) \in x..y
\end{axdef}

\section{Agent}
This section defines the details of the Hunter Agent which explores the
cave world in the hope of finding the hidden treasure. The cave
environment in which the Hunter Agent is situated in has a number of
threats, including a number of bottomless pits and a monster called the
wumpus.

The Hunter Agent can only travel from room to room in the cave in
certain directions. He can travel only up or down and left or right. The
agent cannot travel or move diagonally.
\begin{zed}
	DIRECTION ::= Up | Down | Left | Right
\end{zed}

The agent can receive a predefined number of percepts from the
environment, while in any given room of the cave.
\begin{description}
	\item[Stench] If the agent smells a stench, indicates that the wumpus
	is one of the adjacent rooms (not diagonally).
	\item[Breeze] If the agent feels a breeze, this indicates that there
	is pit in one of the adjacent caves (not diagonally).
	\item[Glitter] If the agent sees glitter, then the agent has found
	the gold in the current room.
	\item[Bump] If the agent perceives a bump that means it has hit the
	wall of the cave and cannot travel further in the direction he was
	travelling in.
	\item[Scream] If the agent hears a scream, this means that the
	wumpus has been killed and any further stench can be ignored.
\end{description}
\begin{zed}
	PERCEPT ::= Stench | Breeze | Glitter | Bump | Scream
\end{zed}

The Hunter Agent only has five different types of actions available to
him. He can either travel one room forward, turn left or right, shoot a
single arrow in the direction the agent is facing, or climb out of the
cave. The agent only has a single arrow available. If he shoots the
arrow it flies in the direction in which the agent was facing until it
hits the wall of the cave or until it hits the wumpus. If it hits the
wumpus, the wumpus is killed. The agent can only climb out of the cave
if there is an opening in the room. In this scenario, the only opening
will be in the room in which the agent entered the cave; that is the
start room.
\begin{zed}
	ACTION ::= Forward | TurnRight | TurnLeft | Shoot | Climb
\end{zed}

The agent schema has a sequence of $PERCEPT$ representing a list of the
percepts the agent is currently perceiving (while in the current
location). Similarly, the schema also has sequence of $ACTION$ to
indicate the actions that it whishes to perform this time step.
The schema indicates that the agent has two basic beliefs about the
world. Firstly, it knows if it is alive, and secondly it knows if it has
shot the arrow it starts out with. Note that the agent doesn't know
anything about it's current position in the cave. In this particular
case the agent isn't storing it's own local coordinates, but it still
doesn't know about the cave's global coordinate system.

\begin{schema}{Agent}
	percepts : \seq PERCEPT \\
	actions : \seq ACTION \\
	alive : \bool \\
	shotArrow : \bool
%\where
\end{schema}

When the agent is initialised it has no percepts and no actions to
perform (and hence the empty sequences). It also knows that it is alive
and that it has yet to shoot the arrow it has.
\begin{schema}{AgentInit}
	\Delta Agent \\
\where
	percepts' = \emptyseq \\
	actions' = \emptyseq \\
	alive' = \true \\
	shotArrow' = \false
\end{schema}

\section{Wumpus}
All the wumpus knows in the wumpus schema is, if it is alive or not.

\begin{schema}{Wumpus}
	alive : \bool
\end{schema}

When the wumpus is initialised it knows it is alive.

\begin{schema}{InitWumpus}
	\Delta Wumpus
\where
	alive' = \true
\end{schema}

\section{Environment}
In this section the environment in which the wumpus and the agent are
situated in is defined. The cave is represented as a two dimensional
grid of rooms. Therefore, and particular room can be defined by its
position in the grid by a tuple representing the grid coordinates.
A $ROOM$ is defined by its position coordinates, which is a relation
between two natural numbers greater than one. Each room can then be
refered to as a row/column tuple pair.
\begin{zed}
	ROOM == \natone \rel \natone
\end{zed}

One particular room is special in the cave. That is the entrance. The
entrance is special because the agent can climb into and climb out of
the cave only when at the entrance. The entrance is defined as the first
row/column pair in the cave $(1,1)$ or $\{1\map1\}$.
\begin{axdef}
	entrance : ROOM
\ST
	entrance = \{ 1 \map 1 \}
\end{axdef}

The cave schema defines the cave firstly by the number of rows and
columns it has. This indirectly defines the number of rooms in the cave.
A finite subset of these rooms will have bottomless pits in them.
It is not possible for there to be a pit in the entrance room.
The gold will be located in a room in the cave. The wumpus will
also be located in a single room. It is entirely possible that wumpus
and the gold are in the same room. The cave also contains the hunter
agent. The cave schema also keeps track of the agent's location and the
direction it is facing.

\begin{schema}{Cave}
	numRows : \natone \\
	numColumns : \natone \\
	pits : \fset ROOM  \\
	gold : \fsetone ROOM \\
	wumpus : \fsetone ROOM \\
	agentLocation : \fsetone ROOM \\
	agentDirection : DIRECTION \\
	agent : Agent
\where
	\#pits \leqslant numRows * numColumns \\
	\dom pits \subs 1..numRows \\
	\ran pits \subs 1..numColumns \\
	entrance \notin pits \\
	\dom gold \subs 1..numRows \\
	\ran gold \subs 1..numColumns \\
	\#gold = 1 \\
	\#wumpus = 1 \\
	\dom wumpus \subs 1..numRows \\
	\dom wumpus \subs 1..numColumns \\
	\dom agentLocation \subs 1..numRows \\
	\ran agentLocation \subs 1..numColumns \\
\end{schema}

The cave is initialised using three parameters. Firstly the number of
rows and the columns in the cave is set, followed by the probability
that there is a pit in any given room. The cave initialisation schema
also places random pits in the cave, and also randomly places the wumpus
and the gold. It is entirely possible that the gold ends up in a
bottomless pit, in which case it is impossible for the agent to succeed
at its mission. It is also possible for the gold and the wumpus to end
up in the same room.

\begin{schema}{InitCave}
	\Delta Cave \\
	n?, m? : \natone \\
	pitChance? : \real
\where
	numRows' = n? \\
	numColumns' = m? \\
	0.0 \leqslant pitChance? \leqslant 1.0 \\
	\forall i,j : \natone | \\
	\t1	i \in 1..numRows \land j \in 1..numColumns
		  \land \{ i \map j \} \neq entrance \dot \\
	\t1 uniform(0,1) \leqslant pitChance? \\
	\t1 \implies pits' = pits \union \{ i \map j\} \\
	\forall gx, gy : \natone | \\
	\t1 gx \in 1..numRows \land gy \in 1..numColumns \dot \\
	\t1 gx = random(1,numRows) \land gy = random(1,numColumns) \\
	\t1 \implies gold' = \{ gx \map gy \} \\
	\forall wx, wy : \natone | \\
	\t1 wx \in 1..numRows \land wy \in 1..numColumns \dot \\
	\t1 wx = random(1,numRows) \land wy = random(1,numColumns) \\
	\t1 \implies wumpus' = \{ wx \map wy \} \\
	agentLocation' = entrance \\
	agentDirection' = Left
\end{schema}

\end{document}