%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% fphw Assignment
% LaTeX Template
% Version 1.0 (27/04/2019)
%
% This template originates from:
% https://www.LaTeXTemplates.com
%
% Authors:
% Class by Felipe Portales-Oliva (f.portales.oliva@gmail.com) with template
% content and modifications by Vel (vel@LaTeXTemplates.com)
%
% Template (this file) License:
% CC BY-NC-SA 3.0 (http://creativecommons.org/licenses/by-nc-sa/3.0/)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------

\documentclass[
	12pt, % Default font size, values between 10pt-12pt are allowed
	%letterpaper, % Uncomment for US letter paper size
	%spanish, % Uncomment for Spanish
]{fphw}

% Template-specific packages
\usepackage[utf8]{inputenc} % Required for inputting international characters
\usepackage[T1]{fontenc} % Output font encoding for international characters
\usepackage{fontspec,unicode-math} % Required for using utf8 characters in math mode
\usepackage{parskip}  % To add extra space between paragraphs
\usepackage{mathpazo} % Use the Palatino font
\usepackage{graphicx} % Required for including images
\usepackage{booktabs} % Required for better horizontal rules in tables
% \usepackage{listings} % Required for insertion of code
\usepackage{enumerate}% To modify the enumerate environment
\setlength{\parindent}{15pt}

%----------------------------------------------------------------------------------------
%	ASSIGNMENT INFORMATION
%----------------------------------------------------------------------------------------

\title{Task 1 \\ The Logarithmic Spiral} % Assignment title

\author{Emilio Domínguez Sánchez} % Student name

\date{October 3rd, 2020} % Due date

\institute{University of Murcia \\ Faculty of Mathematics} % Institute or school name

\class{Geometría de Superficies} % Course or class name

\professor{Dr. Pascual Lucas Saorin} % Professor or teacher in charge of the assignment

%----------------------------------------------------------------------------------------
%	Definitions
%----------------------------------------------------------------------------------------

\usepackage{physics}
\DeclareMathOperator{\len}{len}
\newcommand{\R}{\mathbb{R}}

\begin{document}

\maketitle % Output the assignment title, created automatically using the information in the custom commands above

%----------------------------------------------------------------------------------------
%	ASSIGNMENT CONTENT
%----------------------------------------------------------------------------------------

\section*{Problem}

\begin{problem}
    Let $α : \R → \R^2$ be the parametrized curve given by $α(t) = ae^{bt}(\cos t, \sin t)$,
where $a > 0$ and $b < 1$.
Find the arc length function and a reparametrization with respect to arc length.
\end{problem}

%----------------------------------------------------------------------------------------

\subsection*{Answer}

    The arc length function can be as the integral of the norm of the derivative.
That is,

\begin{multline*}
    \len(t_0, τ) =
    \int_{t_0}^τ \norm{\dv{α}{t}} \dd{t} = \\
    \int_{t_0}^τ \norm{ae^{bt} \qty(b\cos t - \sin t, b\sin t + \cos t)} \dd{t} =
    \int_{t_0}^τ ae^{bt}\sqrt{b^2 + 1} \dd{t} = \\
    \left\{ \begin{split}
        (b = 0) & &
        = a\eval[τ|_{t_0}^τ
        = a(τ-t_0). \\
        (b ≠ 0) & &
        = \frac{a\sqrt{b^2 + 1}}{b} \eval[e^{bt}|_{t_0}^τ =
        k (e^{bτ} - e^{bt_0}). \\
    \end{split} \right.
\end{multline*}

\noindent
where we have defined $k$ as $\frac{a\sqrt{b^2 +1}}{b}$ for convenience.

    When $b = 0$, the spiral becomes a circunference
that is traversed at the constant speed $a$.
This degenerate case can be reparametrized as

\begin{align*}
    \vectorunit*{α}(t) & =
    a\qty(\cos \frac{t}{a}, \sin \frac{t}{a}), \\
    \norm{\dv{\vectorunit*{α}(t)}{t}} & =
    \norm{\qty(-\sin \frac{t}{a}, \cos \frac{t}{a})} = 1.
\end{align*}

    For the case $b \neq 0$, owing to the fact that
the (timed by $t_0$) position can be chosen arbitrarily,
and, because either
$\lim_{t_0 \to -∞} e^{bt_0} = 0$ if $b > 0$ or
$\lim_{t_0 \to ∞} e^{bt_0} = 0$ if $b < 0$,
we can consider the integral when $t_0 = -∞$ or $t_0 = ∞$.
Hence, and for simplicity, we will assume $b > 0$ from here onwards
and write the length as $\len(-∞, τ) = \len(τ) = ke^{bτ}$.

    To reparametrizate the curve with respect to its length,
we guess that we need to find the function

\begin{align*}
    \R & \to \R^2 \\
    t & \mapsto α(p_t | \len(p_t) = t), \\
\end{align*}

which ought to be

\begin{align*}
    \R & \to \R^2 \\
    t & \mapsto α(\len^{-1}(p_t)). \\
\end{align*}

\noindent
We have seen in class a theorem that asserts that the inverse and this reparametrization
always exist when $\dv{α}{t} ≠ 0$.
For our particular problem, it is enough to clear $τ$ from the relation $len(τ) = ke^{bτ}$,
giving
%TODO clear = despejar?

\begin{align*}
    \len(τ) & = ke^{bτ} \\
    τ & = \frac{\ln \len(τ) - \ln k}{b}\\
    \len^{-1}(τ) & = \frac{\ln τ - \ln k}{b}.
\end{align*}

\noindent
Thus, the formula for the reparametrization would be

\begin{multline*}
    \vectorunit*{α}(t) =
    α\qty(\len^{-1}(t)) =
    α\qty(\frac{\ln t - \ln k}{b}) = \\
    ae^{b\frac{\ln t - \ln k}{b}} \qty(\cos \frac{\ln t - \ln k}{b}, \sin \frac{\ln t - \ln k}{b}) =
    \qty( k = \frac{a\sqrt{b^2 +1}}{b} ) = \\
    \frac{bt}{\sqrt{b^2+1}} \qty(\cos \frac{\ln t - \ln k}{b}, \sin \frac{\ln t - \ln k}{b}).
\end{multline*}

%----------------------------------------------------------------------------------------

\subsection*{Extra}

    As you may have noticed, the expression for $\vectorunit*{α}$
(which is in polar form)
shows that the radius of the spiral does not depend on the choice of $a$.
And although the angle does (via $k$),
the derivative of the angle doesn't.
What this means is that the set of curves with the same $b$ are all similar by rotation.
We can check that a rotation of a logarithmic spiral does, indeed,
give another logarithmic spiral with the same $b$ parameter.
That is, a scaled version of the original curve.
Similarly, a scaled version of the original curve can be obtained via a rotation.
This property of the logarithmic spiral is called self-similarity.

%----------------------------------------------------------------------------------------

\end{document}


\section{Triangulation system geometry}
\label{sec:init-modelanalysis}
At first, we analyse the errors due to the geometry of the triangulation system. For simplicity, we will focus on the standard geometry (described in Section \ref{sec:lctt}): the analysis for other geometries is trivial. \\

As illustrated in Figure \ref{fig:laser-triang}, the laser-camera pair form a triangle with angle $\phi$ between the baseline (the plane along the distance between the laser and the camera), and the optical axis of the camera. In \acs{SOL} systems, $\phi$ is
\clearpage
  \begin{figure}[h]
    \centering
    \begin{minipage}[c]{\textwidth}
      \centering
      \includegraphics[width=0.7\textwidth]{./images/model/laser_triang.png}
      \caption{\acs{FOV} and characteristic angles}
      \label{fig:laser-triang}
    \end{minipage}
    \vfill
    \begin{minipage}[c]{\textwidth}
      \centering
      \includegraphics[width=0.7\textwidth]{./images/model/laser_triang_pdv.png}
      \caption{Angles definitions}
      \label{fig:laser-triang-pdv}
    \end{minipage}
  \end{figure}
\clearpage \noindent
called \textit{triangulation angle}. If $\phi$ and the baseline are known, any point in the 3D space belonging to the laser plane, is located estimating the offset $\alpha$ with respect to $\phi$, and the offset $\beta$ with respect to the optical axis of the camera. Accordingly with \cite{th:quattrini}, and using simple mathematical relations, we can locate the $y$ coordinate as function of the angle $\alpha$ as follows:
  \begin{equation}
    \label{eq:model-ya}
    y\left( \alpha \right) = y_f + z_f \tan \left( \phi + \alpha \right)
  \end{equation}
where $\left( x_f, y_f, z_f \right)$ are the principal point projection coordinates in the laser plane (in the world reference system), as shown in Figure \ref{fig:laser-triang-pdv}. Note that using this notation, $z_f$ is the distance from the laser to the camera, previously called baseline. \\

Now let's look at the Figure \ref{fig:laser-triang-pdv}. As mentioned at the end of the Section \ref{sec:lctt}, camera's resolutions varies changing the distance of the target from the camera. In particular this is true for the resolution along the laser line (axis $x$ with respect to out notation). Points having the same $x$ coordinates but different $y$ are estimated with different camera resolutions. This means that the values of $x$ strongly depends on the angle $\beta$ (that identify the value of $x$ with respect the the optical axis of the camera), but also on the angle $\alpha$ (that identifies the value of $y$ with respect to the triangulation angle). For these reasons we can write
  \begin{equation}
  \label{eq:model-xab}
    x(\alpha, \beta) = \frac{y\left( \alpha \right) - y_f}{\sin(\phi + \alpha)}\tan(\beta)
  \end{equation} \\
As we can see, $x$ strongly depends on $y$, accordingly with what we said in Section \ref{sec:lctt}, about the trapezoidal shape of the laser plane. Still for the shape of the laser, a similar consideration can be made for the $y$ coordinate, because of the trapezoidal shape of the laser plane.
% To be precise, a similar consideration can be made for the $y$ coordinate, because of the trapezoidal shape of the laser plane.
However, in this case the differences between the nearest and the farthest \acs{FOV}s along the $x$ axis, are negligible. For theses reasons we will consider Equation \ref{eq:model-ya} as a good approximation of the relation between $y$ and $\alpha$. \\

Accordingly with what we said in Section \ref{sec:teo-calibration}, during calibration phase the choice of the reference system is arbitrary, so we can put $\left( x_f, y_f, z_f \right) = \left( 0, 0, z_f \right)$ without loss of generality. In this way, Equations \ref{eq:model-ya} and \ref{eq:model-xab} can be simplified. \\

At this point it is easy to estimate the error associated with the two newly introduced measures. Applying the simplification, for Equation \ref{eq:model-ya} we can write:
  \begin{equation}
    \label{eq:model-ya-err0}
    \sigma_{y_\alpha} = \sqrt{
      \left( \frac{\partial y}{\partial z_f} \right)^2 \sigma_{z_f}^2 +
      \left( \frac{\partial y}{\partial \phi} \right)^2 \sigma_\phi^2 +
      \left( \frac{\partial y}{\partial \alpha} \right)^2 \sigma_\alpha^2
    }
  \end{equation}
while for Equation \ref{eq:model-xab} we can write
  \begin{equation}
    \label{eq:model-xab-err0}
    \sigma_{x\left( \alpha, \beta \right)} = \sqrt{
      \left( \frac{\partial x}{\partial \phi} \right)^2 \sigma_\phi^2 +
      \left( \frac{\partial x}{\partial \alpha} \right)^2 \sigma_\alpha^2 +
      \left( \frac{\partial x}{\partial \beta} \right)^2 \sigma_\beta^2
    }
  \end{equation}\\
In these last equations, the $\sigma$ are the errors committed evaluating each component in the Equations \ref{eq:model-ya} and \ref{eq:model-xab}. As we can see, in Equations \ref{eq:model-ya-err0} and \ref{eq:model-xab-err0} we are considering also $\phi$ and $z_f$ that are constructive parameters that depend on the accuracy with which the system was built or installed. Typically, these parameters are corrected thanks to the calibration process, that allows to estimate camera intrinsic and extrinsic parameters, as mentioned in Section \ref{sec:teo-calibration}. In addition, the calibration process uses many algorithms that in turn use different heuristics to estimate camera parameters. This means that all parameters are affected by error, but as we can see later, we can consider these errors negligible. So we can simplify Equations \ref{eq:model-ya-err0} and \ref{eq:model-xab-err0} respectively as follows
  \begin{equation}
    \label{eq:model-ya-err1}
    \sigma_{y_\alpha} = \sqrt{
      \left( \frac{\partial y}{\partial \alpha} \right)^2 \sigma_\alpha^2
    }
  \end{equation}

  \begin{equation}
    \label{eq:model-xab-err1}
    \sigma_{x\left( \alpha, \beta \right)} = \sqrt{
      \left( \frac{\partial x}{\partial y_\alpha} \right)^2 \sigma_{y_\alpha}^2 +
      \left( \frac{\partial x}{\partial \alpha} \right)^2 \sigma_\alpha^2 +
      \left( \frac{\partial x}{\partial \beta} \right)^2 \sigma_\beta^2
    }
  \end{equation} \\

Keeping focus on the geometry of the system, the second element to consider is the laser plane. In standard geometry, variations on laser pitch and roll rotations, with respect to the reference system, can affect heavily the final measure. By trigonometry we know that by changing the angle, the point projections also change in the two axes of the reference system. The same effect is present when we rotate the laser plane with respect to the $x$ axis (roll) or with respect to the $y$ axis (pitch). These errors are more apparent in the other triangulation geometries. What we have to do, is to compensate these rotations projecting the laser plane on the ideal one. The compensations are performed as follows:
  \begin{equation}
    \begin{matrix}
      y_w = y(\alpha) \cdot cos(\rho) \\ ~ \\
      x_w = x(\alpha, \beta) \cdot \cos(\gamma)
    \end{matrix}
    \label{eq:radial-compensations}
  \end{equation} \\
where $\rho$ is the laser roll angle, and $\gamma$ is the laser pitch angle. Using the model introduced in Equation \ref{eq:er_prop_2}, we can write, respectively:
  \begin{equation}
    \sigma_{y_w} = \sqrt{
      \left( \frac{\partial y_w}{\partial y(\alpha)} \right)^2 \sigma_{y_\alpha}^2
      + \left( \frac{\partial y_w}{\partial \rho} \right)^2 \sigma_\rho^2
    }
    \label{eq:err-radial-comp-yw}
  \end{equation}
  \begin{equation}
    \sigma_{x_w} = \sqrt{
      \left( \frac{\partial x_w}{\partial x\left( \alpha, \beta \right)} \right)^2 \sigma_{x\left( \alpha, \beta \right)}^2 +
      \left( \frac{\partial x_w}{\partial \gamma} \right)^2 \sigma_\gamma^2
    }
    \label{eq:err-radial-comp-xw}
  \end{equation} \\
where $\sigma_{y_\alpha}$ and $\sigma_{x\left( \alpha, \beta \right)}$ are the ones evaluated in Equations \ref{eq:model-ya-err1} and \ref{eq:model-xab-err1} respectively. \\

A practical example in which these compensations are needed, is the \acs{WPMS}. In autonomous \acs{WPMS}s placed alongside the rails, specular geometry is generally used, and wheels are measured while the train is running. In these cases we are not sure to measure the wheel exactly along its axis, so the acquired profiles have to be compensated. This type of correction is called \textit{radial compensation}. \\
% The errors due to non-ideality of laser plane placement can be seen also as non-ideality of work conditions. In autonomous \acs{WPMS}s placed alongside the rails, specular geometry is generally used, and wheels are measured while the train is running. In these cases we are not sure to measure the wheel exactly along its axis, and typically corrections like the ones for rolls are needed. In these field, these corrections are referred as \textit{radial compensation}. \\
We can consider the same things about pitch. Sometimes the wheel under analysis is not perpendicular with the laser plane, because wheel yaw and camber. Also in these cases $x$ values must be compensated.

%We can see here, the determination of $x$ is fully dependent by the formulation on $y$.


% Line numbering does not work well with diplay math?
% that is why iI use the following macro
% to end a paragraph just before a displaymath
% without getting to much vertical space
\newcommand\premathpar{\vspace{-2\parskip}\par}
% To have linenumbering, add a \par before each displaymath
% The text below was taken from the sample.tex document by
% \author{Matthias K. Gobbert}
% \begin{savequote}[8cm]
%   \sffamily
%   Beautiful math is the purpose of \TeX\ and \LaTeX.
%   \qauthor{Mantra of \TeX-ies}
% \end{savequote}
\chapter{Mathematics to show off}
% first som definitions
% some personal command definitions as examples:
\newcommand{\half}{\frac{1}{2}}
\newcommand{\eps}{\varepsilon}
\newcommand{\rh}{\rho}
\newcommand{\mtheta}{\vartheta}
\newcommand{\ph}{\varphi}
% this command is for partial derivatives and takes 2 input arguments:
\newcommand{\der}[2]{\frac{\partial {#1}}{\partial {#2}}}

Here, you see how a mathematical equation can be generated in line, for
instance $f(x) = \frac{1}{1+25 x^2}$.
The \verb+$+-symbols enclose the formula.
As a so-called displayed formula, it would look like\premathpar
\begin{displaymath}
  f(x) = \frac{1}{1+25 x^2}.
\end{displaymath}
It is customary that mathematical functions are \emph{not} set in math-italics,
so \LaTeX\ has the basic ones pre-defined; you should use the commands
\verb+\cos+, \verb+\exp+, etc.\ to get $f_1(x) = \cos x$,
$f_2(x) = - e^x \sin^2 x$, etc.

Here, I use some of my commands defined above: I like $\eps = \varepsilon$
better than the default $\epsilon$. A partial derivative (with 2 arguments)
would be obtained as follows. If $f(x,y) = x^2 y^3$, then \premathpar
\begin{displaymath}
  \der{f}{x} = 2 x y^3, \quad \der{f}{y} = 3 x^2 y^2.
\end{displaymath}
\section{Sums and Integrals}
When you say ``capital sigma,'' you probably did not really mean $\Sigma$,
but rather a summation symbol. You would get that as in\premathpar
\begin{displaymath}
  \sum_{i=0}^{\infty} r^i = \frac{1}{1 - r} \quad \mbox{for all $|r| < 1$}.
\end{displaymath}
Finally, we have\premathpar
\begin{displaymath}
  \int_0^1 \sin(2 \pi x) \, dx = 0
\end{displaymath}
and\premathpar
\begin{displaymath}
  \int\!\!\int f(x) g(y) \, dx \, dy = \int f(x) \, dx \,\, \int g(y) dy.
\end{displaymath}
Here, \verb+\,+ gives a small space, while \verb+\!+ forces things closer
together; you have to work on the proper spacing for integrals, as \LaTeX\
does not understand, what is going on.
\clearpage % hack to move section to next page
\section{Matrices in \LaTeX}
A matrix $A \in \mathrm{R}^{m \times n}$ could be defined by\premathpar
\begin{displaymath}
  A = \left( \begin{array}{ccccc}
        11     & 12     & 13     & \cdots & 1n     \\
        21     & 22     & 23     & \cdots & 2n     \\
        \vdots & \vdots & \vdots & \ddots & \vdots \\
        m1     & m2     & m3     & \cdots & mn     \\
      \end{array} \right)
\end{displaymath}
Here, the word \verb+dots+ in the commands stands for an ellipsis
(i.e., three dots) placed horizontally in the centre (\verb+\cdots+),
vertically (\verb+\vdots+), or diagonally (\verb+\ddots+); what is
not mentioned is \verb+\ldots+ for horizontal dots at the baseline.
Use the baseline or central version as appropriate, for instance\premathpar
\begin{eqnarray*}
  a_1, a_2, \ldots, a_n & \mbox{and not} & a_1, a_2, \cdots, a_n, \\
  a_1 + a_2 + \cdots + a_n & \mbox{and not} & a_1 + a_2 + \ldots + a_n, \\
\end{eqnarray*}

Some more comments on the matrix are needed, I suppose:
The \verb+\left(+ and \verb+\right)+ create the variable-sized parentheses
around the actual array of terms. You can also use \verb+\left[+ and
\verb+\right]+, or \verb+\left\{+ and \verb+\right\}+ in other situations.
The actual array arrangement is organised by the \verb+array+ environment;
you need the arguments \verb+ccccc+ to indicate that there are five columns
and you want the entries centered (``c''), other options are left (``l'')
and right (``r''). Notice how \verb+&+ separate columns and \verb+\\+
the rows.

Here is another matrix example.
A matrix multiply used with 3D graphics:
\begin{displaymath}
  \left[ \begin{array}{cccc}
        R_{11} & R_{12} & R_{13} & 0 \\
        R_{21} & R_{22} & R_{23} & 0 \\
        R_{31} & R_{32} & R_{33} & 0 \\
        0      & 0      & 0       & 1
    \end{array} \right]
  \cdot
  \left[ \begin{array}{cccc}
      1 & 0 & 0 & X \\
      0 & 1 & 0 & Y \\
      0 & 0 & 1 & Z \\
      0 & 0 & 0 & 1
  \end{array} \right]
=  \left[ \begin{array}{cccc}
        R_{11} & R_{12} & R_{13} & T_x \\
        R_{21} & R_{22} & R_{23} & T_y \\
        R_{31} & R_{32} & R_{33} & T_z \\
        0      & 0      & 0       & 1
  \end{array} \right]
\end{displaymath}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


\section{The Integers and the Rationals}
\subsection{Integers}
The integers $\mathbb Z$ consists of all expressions $n,-n$ where $n$ is a natural number, and $0$.
We can define $+, \times$ etc. in the obvious way.
And it is easy and trivial to check all the necessary rules apply.
And we define $a<b$ as $a+c=b$ for some natural number $c$.
All previous rules appply except we need $c$ to be positive to have $a<b\implies ac<bc$.
Also, for any $a$, $a+0=a$ and that there is a $b$ such that $a+b=0$.
This makes the integers a group.
\subsection{Rationals}
We can define the rationals $\mathbb Q$ as well.
It shall consist of all expressions $a/b$ for some integers $a,b$ with $n\neq 0$.
And we shall have
$$a/b=c/d\iff ad=bc$$
And we define
$$\frac{a}{b}+\frac{c}{d}=\frac{ad+bc}{bd}$$
We have to check that this is well defined, since $\mathbb Q$ is constructed based on equivalence classes.
For example, we cannot assign an operation sending $a/b\to a^2/b^3$ because it will be ill defined, as $1/2$ and $2/4$ go to different places.
\begin{proposition}
    The addition is well-defined on $\mathbb Q$.
\end{proposition}
\begin{proof}
    Trivial.
\end{proof}
\begin{proposition}
    We can define multiplication similarly that satisfies all usual rules and that every nonzero rational number has an inverse under multiplication.
\end{proposition}
\begin{proof}
    Trivial.
\end{proof}
So the rationals excluding $0$ is a group.\\
As for order, $a/b<c/d\iff ad<bc$ for $b,d>0$.
We can also check all the rules we want


\subsection*{Lecture Schedule}

Below is a tentative schedule of the theory and practice topics covered by lecture number.

\begin{enumerate}[(1)]
\item \textbf{Theory:} Review of syllabus, introducing science and modeling, definition of phenomena / the response $y$, reality vs. approximation, measurement vs. prediction and simulation, definition of learning from data, model validation, heuristics, ambiguous models, mathematical models, causal inputs, response spaces $\mathcal{Y}$, definition of features $\x$ and its feature space $\mathcal{X}$.

\textbf{Practice:} Short history of \texttt{R}, introduction to RStudio, arithmetic, assignment of variables, mathematical functions, logical operations, numeric / integer / boolean data types, vectors, sequences, subsetting, sorting, taxonomy of illegal values.

\item \textbf{Theory:} Feature spaces, binary, categorical and continuous data types, definition of metrics, ordinal codings, sample size $n$ vs. number of features $p$, error due to ignorance of information $\delta$, optimal response surface $f$, definition of training data $\mathbb{D}$, definition of candidate function set $\mathcal{H}$, definition of prediction function $g$, algorithms that produce prediction functions $\mathcal{A}$, definition of misspecification error, definition of optimal candidate function $h^*$, irreducible error $\mathcal{E}$, estimation error and residuals $e$.

\textbf{Practice:} Realizations of popular random variables, PDF / PMF / CDF / empirical CDF computations, quantile computations, factor-type variables, matrix data type and its critical functions, if / if-else / else / switch programmatic control, for / while / repeat loops, console printing, errors and warnings, try-catch control.

\item \textbf{Theory:} Visualization of training data $\mathbb{D}$, threshold models for classification, null model for classification $g_0$, concept of a parameter $\theta$ and parameter space $\Theta$, degrees of freedom, definition of objective function / error function, accuracy, sum of squared error (SSE), optimization within $\mathcal{A}$, linear threshold models, perception learning algorithm (PLA), introduction to neural networks.

\textbf{Practice:} Review of hashing and the list data type, the array data type for general tensors, naming for vectors / matrices / tensors, introduction to specifying functions, arguments, argument defaults, creating data matrices, tabling multiple features, the dataframe data type.

\item \textbf{Theory:} Review of lines in multiple dimensions with Hesse Normal Form, derivation of the support vector machine (SVM) using maximum margin objective, hinge error, SVM using the Vapnik objective, defintion of hyperparameters $\lambda$, $K$-nearest neighbors algorithm (KNN).

\textbf{Practice:} Installing and loading libraries from CRAN and githuv, public vs private functions and scoping, loading datasets from libraries / files / URLs, creating threshold models, writing the PLA, matrix operations: arithmetic / transpose / inverse / rank / trace, optimization algorithms e.g. Nelder-Mead, writing the KNN algorithm, using the SVM library and setting the hyperparameter.

\item \textbf{Theory:} Null model for regression $g_0$, linear models for continuous responses $\bbeta$, minimization of SSE for $p=1$ using basic calculus to arrive at the ordinary least squares (OLS) solution $\b$, review of covariance of two random variables $\sigma_{X,Y}$ and its estimate $s_{X,Y}$, review of correlation $\rho_{X,Y}$ and its estimate $r_{X,Y}$.

\textbf{Practice:} Computing sample covariance and correlation in the context of the OLS algorithm for $p=1$, computing OLS error metrics, the formula object, using the \texttt{lm} function, visualizing the OLS line atop a scatterplot, computing predictions in OLS, OLS in the boston housing data

\item \textbf{Theory:} Error metrics for regression: SSE, mean squared error (MSE) and root mean squared error (RMSE), approximate prediction confidence intervals using RMSE, sum of squares total (SST), concept of proportion of variance explained $R^2$, class of models with $R^2 < 0$, perfect fit models with $R^2 = 1$.

\textbf{Practice:} OLS on the Galton height data and the etymology of the word \qu{regression} in statistics, computing OLS in the case of categorical variables (ANOVA), dummifying variables, computing model matrices.

\item \textbf{Theory:} OLS estimates being the group averages with one binary feature, independence, dependence, association, correlation, OLS estimates with $p>1$, design matrix $X$, vector derivative properties: constant scalars, multiples, multiplication, quadratic forms, general OLS solution, review of matrix inverses, transposes, rank and symmetric matrices.

\textbf{Practice:} Visualizing $R^2$ for a model using error density estimation, computing general OLS estimates in multiple dimensions from scratch, making predictions using the \texttt{predict} interface for modeling.

\item \textbf{Theory:} OLS predictions as linear transformations, review of the linear algebra concepts of dimension, length, norm, subspace, linear in/dependence, column space, derivation of the orthogonal projection matrix for one dimension via law of cosines, outer products, idempotency.

\textbf{Practice:} Eigendecomposition, computing error metrics for general OLS, computing the null model.

\item \textbf{Theory:} Derivation of the orthogonal projection in multiple dimensions, equivalence of the OLS algorithm with orthongal projection, definition of the hat matrix $H$, review of the linear algebra concepts of eigenvectors and eigenvalues, computing the eigenvectors and eigenvectors of $H$.

\textbf{Practice:} Computing the hat matrix $H$ for the null model and in general, confirming its eigendecomposition and idempotency and rank, using $H$ to find the OLS predictions and residuals and verifying their orthogonality.

\item \textbf{Theory:} Verification of the symmetry and idempotency of $H$, proving that one multidimensional orthogonal projection is in general not the same as the sum of orthogonal projections in the component dimensions except if the component dimensions themselves are orthogonal, review of orthonormal matrices $Q$, proving the equivalence of $H = QQ^\top$, definition of $X = QR$ decomposition, the Gram-Schmidt algorithm, computing $R$, deriving the least squares estimate using $Q$ and $R$.

\textbf{Practice:} Computing $QR$ decomposition and confirming the orthonormality of $Q$, verifying the OLS predictions are the same with $Q$, writing the Gram-Schmidt algorithm, an overview of the piping / chaining concept in modern programming and in \texttt{R} with demos.

\item \textbf{Theory:} Definition of sum of squares for the regression $SSR$, proving the sum of squares identity $SST = SSR + SSE$, showing that if $p$ increases by one dimension, then SSR is obligated to increase forcing $R^2$ higher, definition of overfitting in modeling, definition of chance capitalization, demonstrating that full overfitting in OLS leads to $H = I$, a superficial introduction to regularization (lasso regression and ridge regression).

\textbf{Practice:} Demo showing the iterative addition of a feature and $R^2$ monotonically increasing and RMSE monotonically decreasing (overfitting), demonstrating that random vectors are never truly orthogonal and thus their projections are non-zero, a lasso fit and a ridge fit of a dataset with $p \geq n$.

\item \textbf{Theory:} Definition of in-sample error metrics vs. out-of-sample (oos) error metrics, splitting $\mathbb{D}$ into $\mathbb{D}_{\text{train}}$ and $\mathbb{D}_{\text{test}}$ via split constant $K$, definition of \qu{honest} validation via oos error metrics, definition of the final model $g_{\text{final}}$ definition of underfitting, tracing the underfitting-overfitting complexity curve, definition of optimal-complexity models.

\textbf{Practice:} A more full demo of overfitting with visualizations, demo of consistency of OLS estimates, code to create train-test splits, demonstration that oos error is larger than in-sample error in the scenario of the model being overfit.

\item \textbf{Theory:} Definition of raw features versus derives features, increasing complexity in $\mathcal{H}$ using polynomial functions of raw features, interpretation of OLS coefficients $\b$, Weierstrauss Approximation Theorem, OLS with polynomial features, definition of the full rank Vandermonde matrix, definitions of interpolation vs. extrapolation.

\textbf{Practice:} Square, cube and higher order polynomial fitting both raw and orthogonal with visualization, overfitting with high-degree polynomials, demonstration of extrapolation in models of many different polynomial degrees, prediction with polynomial models, extrapolation in the Galton height data.

\item \textbf{Theory:} OLS using the log transformation on both features and response and interpretation of $\b$, derivation the log change is approximately percentage change, definition of first-order interactions in OLS, interpretations of coefficients in interaction models.

\textbf{Practice:} Log-linear model fitting and log-log linear model fitting, logging response to reduce the effect of influential observations, the grammar of graphics and the \texttt{ggplot} package to create histograms, scatterplots, box-whisker, violin plots, smoothing plots, overloading plots with many features, faceting, coloring, aesthetics and themes, using color illustrations and faceting to visualize potential first-order interactions in a linear model, fitting interaction models.

\item \textbf{Theory:} oos error metrics as estimates of model generalization error, sources of variance in these estimates, mitigation by adjusting $K$, further mitigation by using cross-validation (CV), $K$-fold CV and its aggregated error estimates, approximation confidence intervals for generalization error, discussion of reasonable values of $K$ in practice.

\textbf{Practice:} Simulating many different train-test splits to underscore that $K$ trades bias vs. variance in the generalization estimate, writing code for $K$-fold CV, using the package \texttt{mlr3} to automate $K$-fold CV.

\item \textbf{Theory:} Introduction of the fundamental problem of \qu{model selection} of candidates $g_1, \ldots, g_M$, model selection with honest validation via splitting $\mathbb{D}$ into $\mathbb{D}_{\text{train}}$, $\mathbb{D}_{\text{select}}$ and $\mathbb{D}_{\text{test}}$ via split constants $K_{\text{select}}$ and $K_{\text{test}}$, procedure to select best model among $M$ candidates and validation of the best model.

\textbf{Practice:} Writing code for the model selection procedure, review of basic \texttt{C++}, optimizing \texttt{R} code via the \texttt{Rcpp} package, benchmarking routines that require heavy looping between \texttt{Rcpp} and base \texttt{R}, benchmarking routines that require heavy recursion between \texttt{Rcpp} and base \texttt{R}.

\item \textbf{Theory:} Double-CV in the model selection procedure using inner folds and outer folds, discussion of reasonable values of $K_{\text{select}}$ and $K_{\text{test}}$ in practice, applying the model selection procedure to grid searching to locate the best value of hyperparameters $\lambda$ in algorithms that require $\lambda$, definition of stepwise modeling using the model selection procedure via the underfitting-overfitting complexity curve concept, stepwise OLS with a large basis of candidate terms.

\textbf{Practice:} Using the package \texttt{mlr3} to automate the double-CV using inner and outer loops, using the package \texttt{mlr3} to automate the locating of optimal hyperparameters, demo of forward stepwise linear modeling and tracing the underfitting-overfitting complexity curve.

\item \textbf{Theory:} Definition of hyperrectangle basis for $\mathcal{X}$ and its OLS solution, unfeasibility of this algorithm in high $p$, introduction of the regression tree algorithm.

\textbf{Practice:} Binning model demo and visualization for varying bin sizes, introduction to data wrangling using the packages \texttt{dplyr} and \texttt{data.table}: filtering, sorting, grouping, summarizing, feature derivation, dataframe joining (left, right, inner, full, between / overlap), benchmarking the two libraries.

\item \textbf{Theory:} Full specification of regression tree algorithm: definition of a binary tree, definition of orthogonal-to-axes splits, nodes vs. leaves, left-right SSE weighting, leaf assignments, overfitting and tree-pruning.

\textbf{Practice:} Using the \texttt{YARF} package to produce regression trees, querying tree stats, visualizing trees and tree model predictions, tree differences by the pruning hyperparameter.

\item \textbf{Theory:} MSE of $g$ decomposition into bias and irreducible error for one $\mathbb{D}$, MSE of $g$ decomposition into bias, irreducible error and variance for multiple $\mathbb{D}$'s, MSE decomposition of $M$ different $g$'s averaged, strategies to eliminate bias and variance, non-parametric bootstrap sampling, Breiman's concept of bootstrap aggregation (bagging), correlation $\rho$ among the bootstrapped models, out-of-bag (oob) observations, validation in bagging via oob samples.

\textbf{Practice:} Visualizing $M$ bagged trees, demonstrating near zero bias, demonstrating variance reduction as $M \rightarrow \infty$, a comparison to OLS and high degree polynomial models, demonstration of generalization error improvement, demonstration of validation in bagging.

\item \textbf{Theory:} Demonstrating the bias for regression trees is near zero, reducing correlation among the bootstrapped models using feature sampling, introduction of random forests (RF) algorithm.

\textbf{Practice:} Demonstration that RF decreases $\rho$ and demonstration that it outperforms bagging in both regression and classification.

\item \textbf{Theory:} A basic discussion of \qu{causality} from a philosophical perspective, directed causal graphs, correlation vs. causation, incidental effects, lurking variables, spurious correlation, causation is defined by manipulation, a quick definition of randomized experimentation, real-world causal diagrams, wrong interpretations of $\b$ in OLS, the highly limited but true / complete paragraph-long interpretation of $\b$ in OLS, a discussion of how OLS regression accomplishes estimation of single features with other features \emph{ceteris paribus}.

\textbf{Practice:} Demos of whimsical spurious correlations, demonstration that spurious correlations are easy to find in simulation, a nice illustration of correlation without causation using an OLS model that reveals the true interpretation of $\b$.

\item \textbf{Theory:} Introduction of classification tree algorithm, the gini metric, leaf assignments, the two errors: false negatives and false positives, the 2$\times$2 confusion matrix and its metrics: precision, recall, accuracy, $F_1$ metric, false discovery rate, false omission rate.

\textbf{Practice:} Using the \texttt{YARF} package to produce classification trees, querying tree stats, visualizing trees and tree model predictions, tree differences by the pruning hyperparameter, measuring the two errors, computing confusion matrices and the other metrics.

\item \textbf{Theory:} Missing data mechanisms (MDMs): missing completely at random, missing at random, not missing at random with examples, strategies to handle missingness: listwise deletion, imputation, multiple imputation, miss forests algorithm and its convergence, the concept of \qu{retaining} missingness even after imputation, introduction of probability estimation using the independent bernoulli random variable model, optimal probability function, likelihood of $\mathbb{D}$, $\mathcal{H}$ for probability functions, generalized linear modeling, link functions: logistic / probit / complementary log-log, numerical approximations to the likelihood optimization.

\textbf{Practice:} An example of a dataset with missingness, assessing the different MDMs, listwise deletion, imputation and the \texttt{missForest} package for the recommended imputation, creating the missingness dummies as derives features in $X$.

\item \textbf{Theory:} Definition of logistic regression (LR), log-odds interpretation of LR $\b$, prediction in LR, error metrics for probability estimation: Brier and Log scoring rules, classification modeling from probability regression, optimal asymmetric cost modeling, response-operator curves (ROC), detection-error tradeoff curves (DET).

\textbf{Practice:} Fitting LR models using the \texttt{glm} package, predicting with LR models, validating creating asymmetric cost classifiers in LR models, constructing ROC and DET plots using LR models, locating optimal models with minimal cost.

\item \textbf{Theory:} Causal diagrams, confounding, correlation does not imply causation, correct interpretation of OLS estimates, asymmetric cost classification in tree models, introduction to boosting as a meta-algorithm, adaboost

\textbf{Practice:} demonstrating confounding and linear models by hiding the confounder and then revealing it, demonstration of asymmetric classification using trees, demonstration of boosting using package \texttt{xgboost}.
\end{enumerate}


%!TEX root = ../main.tex
%-------------------------------------------------------------------------------
\subsection{Mathematical formulation}\label{Mathematics}
%-------------------------------------------------------------------------------
EKW models are set up as a standard Markov decision process (MDP) \citep{Puterman.1994, White.1993}. When making sequential decisions under uncertainty, the task is to determine the optimal policy $\pi^*$ with the largest expected total discounted utilities $v^{\pi^*}_1(s_1)$ as formalized in equation~\eqref{Objective Risk}. In principle, this requires evaluating the performance of all policies based on all possible sequences of utilities, each weighted by the probability with which they occur. Fortunately, however, the multistage problem can be solved by a sequence of simpler inductively defined single-stage problems.\footnote{Optimal decisions in an MDP are a deterministic function of the current state $s$ only, i.e., an optimal decision rule is always deterministic and Markovian. We restrict our notation to this special case right from the beginning.}

The value function $v^\pi_t(s_t)$ captures the expected total discounted utilities under policy $\pi$ from period $t$ onwards for an individual experiencing state $s_t$:
%
\begin{equation*}
  v^\pi_t(s_t) = \E_{s_t}^\pi\left[\left.\sum^{T - t}_{j = 0}  \delta^j\, u_{t + j}(s_{t + j}, a^\pi_{t + j}(s_{t + j})) \,\right]\right..
\end{equation*}
%
Then we can determine $v_1^\pi(s_1)$ for any policy by recursively evaluating equation~\eqref{MDP Policy Equations}:
%
\begin{equation}\label{MDP Policy Equations}
v^\pi_t(s_t) = u_t(s_t,  a^\pi_t(s_t)) + \delta\,\E^\pi_{s_t} \left[\left.v^\pi_{t + 1}(s_{t + 1})  \,\right]\right..
\end{equation}
%
Equation~\eqref{MDP Policy Equations} expresses the total value $v^\pi_t(s_t)$ of adopting policy $\pi$ going forward as the sum of its immediate utility and all expected discounted future utilities.

The principle of optimality \citep{Bellman.1954} allows to construct $\pi^*$ by solving the optimality equations \eqref{MPD Optimality} for all $s$ and $t$ recursively:
%
\begin{equation}\label{MPD Optimality}
	v^{\pi^*}_t(s_t) = \max_{a_t \in A}\bigg\{ u_t(s_t, a_t) + \delta\, \E^{\pi^*}_{s_t} \left[\left.v^{\pi^*}_{t + 1}(s_{t + 1})\,\right]\right. \bigg\}.
\end{equation}

\noindent The optimal value function $v^{\pi^*}_t$ is the sum of the expected discounted utilities in $t$ over the remaining time horizon assuming the optimal policy is implemented going forward. The optimal action is choosing the alternative with the highest total value:
%
\begin{equation*}
	a^{\pi^*}_t(s_t) = \argmax_{a_t\in A} \bigg\{ u_t(s_t, a_t) + \delta\,\E^{\pi^*}_{s_t} \left[\left.v^{\pi^*}_{t + 1}(s_{t + 1})\,\right]\right. \bigg\}.\\
\end{equation*}

\autoref{Backward induction procedure} allows to solve the MDP by a simple backward induction procedure. In the final period $T$, there is no future to take into account, and the optimal action is choosing the alternative with the highest immediate utilities in each state. With the decision rule for the final period at hand, the other optimal decisions can be determined recursively following equation \eqref{MPD Optimality} as the calculation of their expected future utilities is straightforward given the relevant transition probabilities.

\floatname{algorithm}{\sffamily\small Algorithm}
%\vspace{0.5cm}%
\begin{algorithm}[t]
	\caption{\small\!\textbf{.\:\:}\textsf{\strut Backward induction procedure}}\label{Backward induction procedure}
	\begin{algorithmic}\vspace{0.3cm}
		\For{$t = T, \hdots, 1$}
		\If{t = T}
		\State $v^{\pi^*}_T(s_T) =  \underset{a_T\in A}{\max} \bigg\{ u_T(s_T, a_T) \bigg\}\qquad \forall\, s_T\in S$
		\Else
		\State Compute $v^{\pi^*}_t(s_t)$ for each $s_t\in S$ by
		\State $\qquad v^{\pi^*}_t(s_t) = \underset{a_t\in A}{\max} \bigg\{ u_t(s_t, a_t) + \delta\,\E^\pi_{s_t} \left[\left.v^{\pi^*}_{t + 1}(s_{t + 1}) \:\,\right]\right. \bigg\}.$
		\State and set
		\State $\qquad a^{\pi^*}_t(s_t) = \underset{a_t\in A}{\argmax} \bigg\{ u_t(s_t, a_t) + \delta\,\E^\pi_{s_t} \left[\left.v^{\pi^*}_{t + 1}(s_{t + 1}) \:\,\right]\right. \bigg\}.$
		\EndIf
		\EndFor
		\vspace{0.3cm}\end{algorithmic}
\end{algorithm}\FloatBarrier


% section8

\chapter{Diagonalization}

%8.1
\section{Matrix Representations of Linear Transformations}

\begin{exer}Let $T : \mathbb{R}^{5} \rightarrow \mathbb{R}^{3}$ be the linear operator given by the formula
$$T(x_{1}, x_{2}, x_{3}, \hspace{1mm} x_{4}, x_{5}) = (7x_{1}+12x_{2}-5x_{3}, \hspace{1mm} 3x_{1}+10x_{2}+13x_{4}+x_{5}, \hspace{1mm} -9x_{1}-x_{3}-3x_{5})$$
and let $B = \{\mathbf{v}_{1}, \hspace{1mm} \mathbf{v}_{2}, \hspace{1mm} \mathbf{v}_{3}, \hspace{1mm} \mathbf{v}_{4}, \hspace{1mm} \mathbf{v}_{5}\}$ and $B' = \{\mathbf{v}'_{1}, \hspace{1mm} \mathbf{v}'_{2}, \hspace{1mm} \mathbf{v}'_{3}\}$ be the bases for $\mathbb{R}^{5}$ and $\mathbb{R}^{3}$, respectively, in which $\mathbf{v}_{1} = (1, \hspace{1mm} 1, \hspace{1mm} 0, \hspace{1mm} 0, \hspace{1mm} 0)$, $\mathbf{v}_{2} = (0, \hspace{1mm} 1, \hspace{1mm} 1, \hspace{1mm} 0, \hspace{1mm} 0)$, $\mathbf{v}_{3} = (0, \hspace{1mm} 0, \hspace{1mm} 1, \hspace{1mm} 1, \hspace{1mm} 0)$, $\mathbf{v}_{4} = (0, \hspace{1mm} 0, \hspace{1mm} 0, \hspace{1mm} 1, \hspace{1mm} 1)$, $\mathbf{v}_{5} = (1, \hspace{1mm} 0, \hspace{1mm} 0, \hspace{1mm} 0, \hspace{1mm} 1)$, $\mathbf{v}'_{1} = (1, \hspace{1mm} 2, \hspace{1mm} -1)$, $\mathbf{v}'_{2} = (2, \hspace{1mm} 1, \hspace{1mm} 3)$, and $\mathbf{v}'_{3} = (1, \hspace{1mm} 1, \hspace{1mm} 1)$.

\vspace{2mm}
\begin{enumerate}

\item[(a)]
Find the matrix $[T]_{\tiny{B}', \tiny{B}}$.
\vspace{1mm}
\item[(b)]
For the vector $\mathbf{x} = (3, \hspace{1mm} 7, \hspace{1mm} -4, \hspace{1mm} 5, \hspace{1mm} 1)$, find $[\mathbf{x}]_{\tiny{B}}$ and use the matrix obtained in part~(a) to compute $[T(\mathbf{x})]_{\tiny{B}'}$.
\vspace{1mm}
\item[(c)] Find the factorization of [$T$] which is the standard matrix for the linear transformation $T$ using Formula (28) in Section 8.1.
\end{enumerate}

\end{exer}

\begin{sol}

\vspace{1mm}

\begin{enumerate}

\vspace{1mm}
\verb""
\item[(a)]
\begin{verbatim}
v1 = [1 1 0 0 0]'; v2 = [0 1 1 0 0]'; v3 = [0 0 1 1 0]';
v4 = [0 0 0 1 1]'; v5 = [1 0 0 0 1]';
nv1 = [1 2 -1]'; nv2 = [2 1 3]'; nv3 = [1 1 1]';

T = [7 12 -5 0 0; 3 10 0 13 1; -9 0 -1 0 -3];
B1 = [v1 v2 v3 v4 v5]; B2 = [nv1 nv2 nv3];
format short;

% Find the matrix representation with respect to the bases B1 and B2.
TB = T*B1;
TB1B2 = B2\TB;

disp('The matrix representation of T with respect to the basis B1 and B2 is');
disp(TB1B2);
\end{verbatim}

\begin{outputs}
\begin{verbatim}

The matrix representation of T with respect to the basis B1 and B2 is
   34.0000    5.0000  -22.0000  -11.0000   22.0000
   40.0000    2.0000  -40.0000  -25.0000   25.0000
  -95.0000   -2.0000   97.0000   61.0000  -65.0000
\end{verbatim}

\end{outputs}

\item[(b)]
\begin{verbatim}
% Find the coordinate vector of x with respect to the basis B1.
x = [3 7 -4 5 1]';  x_B1 = B1\x;
disp('The coordinate vector of x with respect to the basis B is');
disp(x_B1');

% Find the coordinate vector of T(x) with respect to the basis B2.
Tx_B2 = TB1B2 * x_B1;
disp('The coordinate vector of T(x) with respect to the basis B'' is');
disp(Tx_B2');
\end{verbatim}

\begin{outputs}
\begin{verbatim}

The coordinate vector of x with respect to the basis B is
     9    -2    -2     7    -6

The coordinate vector of T(x) with respect to the basis B' is
  131.0000  111.0000 -228.0000
\end{verbatim}

\end{outputs}

\item[(c)]
\begin{verbatim}
% Transition matrix from B to the standard basis for R^n.
U=B1;

% Transition matrix from B' to the standard basis for R^m.
V=B2;

T=[7 12 -5 0 0 ; 3 10 0 13 1; -9 0 -1 0 -3];

disp('V'); disp(V);
disp('TB1B2'); disp(TB1B2);
disp('inv(U)'); disp(inv(U));
disp('V*TB1B2*inv(U)');disp(V*TB1B2*inv(U));
disp('T'); disp(T);
\end{verbatim}

\begin{outputs}
\begin{verbatim}

V
     1     2     1
     2     1     1
    -1     3     1

TB1B2
   34.0000    5.0000  -22.0000  -11.0000   22.0000
   40.0000    2.0000  -40.0000  -25.0000   25.0000
  -95.0000   -2.0000   97.0000   61.0000  -65.0000

inv(U)
    0.5000    0.5000   -0.5000    0.5000   -0.5000
   -0.5000    0.5000    0.5000   -0.5000    0.5000
    0.5000   -0.5000    0.5000    0.5000   -0.5000
   -0.5000    0.5000   -0.5000    0.5000    0.5000
    0.5000   -0.5000    0.5000   -0.5000    0.5000

V*TB1B2*inv(U)
    7.0000   12.0000   -5.0000         0         0
    3.0000   10.0000         0   13.0000    1.0000
   -9.0000   -0.0000   -1.0000   -0.0000   -3.0000

T
     7    12    -5     0     0
     3    10     0    13     1
    -9     0    -1     0    -3
\end{verbatim}

\end{outputs}

\end{enumerate}
\end{sol}

%8.2

\section{Similarity and Diagonalizability}

\begin{exer}

\begin{enumerate}
\item[(a)] Show that the matrix
$$A = \begin{bmatrix}-13&\hspace{1mm} -60&\hspace{1mm} -60\\ 10 & 42 & 40\\ -5 & -20 & -18 \end{bmatrix}$$
is diagonalizable by finding the nullity of $\lambda I - A$ for each eigenvalue $\lambda$ with the use of Theorem 8.2.11 in the Section 8.2.
\item[(b)] Find a basis for $\mathbb{R}^{3}$ consisting of eigenvectors of $A$.
\end{enumerate}
\end{exer}

\begin{sol}
\verb""
\begin{enumerate}
\item[(a)]
\begin{verbatim}
% For the exact computation of the eigenvalues,
% we use symbolic computation.

% Set A as a symbolic matrix.
A = sym([-13 -60 -60; 10 42 40; -5 -20 -18]);

n = length(A);

% Find the eigenvalues of A by using the command eig.
eigenvalues = eig(A);

for j = 1 : n
    fprintf('The eigenvalue lambda is '); disp(eigenvalues(j));

    % nullity(lambda*I - A) = n - rank(lambda*I - A);
    nullity = n - rank((eigenvalues(j) * eye(n)) - A);

    fprintf('The nullity of (lambda*I - A) is '); disp(nullity);
end

% Since the geometric multiplicity of each eigenvalue of A
% is the same as the algebraic multiplicity,
% by the Theorem 8.2.11, A is diagonalizable.
\end{verbatim}

\begin{outputs}

\begin{verbatim}

The eigenvalue lambda is 2
The nullity of (lambda*I - A) is      2
The eigenvalue lambda is 2
The nullity of (lambda*I - A) is      2
The eigenvalue lambda is 7
The nullity of (lambda*I - A) is      1
\end{verbatim}

\end{outputs}

\item[(b)]
\begin{verbatim}
% Since the eigenvalue = 2 of A has the multiplicity = 2,
% find two linearly independent eigenvectors of A corresponding to lambda = 2.

%Find a basis for the null space of (2*I-A).
eigvec12=null((2 * eye(n)) - A);

% Since the eigenvalue = 7 of A has the multiplicity = 1,
% find an eigenvector of A corresponding to lambda = 7.

%Find a basis for the null space of (7*I-A).
eigvec3=null((7 * eye(n)) - A);

p1 = eigvec12(:, 1); p2 = eigvec12(:, 2); p3 = eigvec3(:, 1);

% By the Theorem 8.2.7, since the eigenvectors corresponding to
% distinct eigenvalues are linearly independent,
% the three obtained eigenvectors {p1, p2, p3} form a basis for R^{3}.

disp('A basis {p1, p2, p3} for R^{3} consisting of the eigenvectors of A is');
fprintf('p1 ='); disp(p1');
fprintf('p2 ='); disp(p2');
fprintf('p3 ='); disp(p3');
\end{verbatim}

\begin{outputs}

\begin{verbatim}

A basis {p1, p2, p3} for R^{3} consisting of the eigenvectors of A is
p1 =[ -4, 1, 0]
p2 =[ -4, 0, 1]
p3 =[ 3, -2, 1]
\end{verbatim}
\end{outputs}

\end{enumerate}
\end{sol}

% 8.3
\section{Orthogonal Diagonalizability; Functions of a Matrix}

\begin{exer}
\begin{enumerate}
Let
$$A=\begin{bmatrix}\frac{1}{2} & 0 & \frac{3}{2} & 0 \\ 0 & \frac{1}{2} & 0 & \frac{3}{2} \\ \frac{3}{2} & 0 & \frac{1}{2} & 0 \\ 0 & \frac{3}{2} & 0 & \frac{1}{2}\end{bmatrix}.$$
\begin{enumerate}
\item[(a)]
Find a matrix $P$ that orthogonally diagonalizes the matrix $A$. You may use the MATLAB command \textit{eig} and perform the Gram-Schmidt process. Use your result to find a diagonal matrix $D$ satisfying $A=PDP^{T}$.
\vspace{1mm}
\item[(b)]
Confirm that the matrix $A$ satisfies its characteristic equation, in accordance with the Cayley-Hamilton theorem. You may use the symbolic object to find the characteristic polynomial and use the MATLAB command \textit{coeffs} to find the coefficient of obtained characteristic polynomial.
\vspace{1mm}
\item[(c)]
%Compute $e^{A}$ as given in Example 5 of Section 8.3. Compare the result with the MATLAB syntax \textit{exp(A)}.
%\item[(d)]
Find the spectral decomposition of $A$.
\end{enumerate}
\end{enumerate}
%\end{exer}

\begin{sol}
\verb""
\begin{enumerate}
\item[(a)]
\begin{verbatim}
A=[1/2 0 3/2 0; 0 1/2 0 3/2; 3/2 0 1/2 0; 0 3/2 0 1/2];

% V: eigen vector, D: eigen value
[V D]=eig(A);

% Gram-Schmidt process
P=GS_process(V);
disp('P is'); disp(P);
disp('D is'); disp(D);
disp('P_transpose is'); disp(P');
disp('P*D*P_transpose is'); disp(P*D*P');
disp('A is'); disp(A);
\end{verbatim}

\begin{outputs}

\begin{verbatim}

P is
   -0.7071         0         0   -0.7071
         0    0.7071    0.7071         0
    0.7071         0         0   -0.7071
         0   -0.7071    0.7071         0

D is
    -1     0     0     0
     0    -1     0     0
     0     0     2     0
     0     0     0     2

P_transpose is
   -0.7071         0    0.7071         0
         0    0.7071         0   -0.7071
         0    0.7071         0    0.7071
   -0.7071         0   -0.7071         0

P*D*P_transpose is
    0.5000         0    1.5000         0
         0    0.5000         0    1.5000
    1.5000         0    0.5000         0
         0    1.5000         0    0.5000

A is
    0.5000         0    1.5000         0
         0    0.5000         0    1.5000
    1.5000         0    0.5000         0
         0    1.5000         0    0.5000
\end{verbatim}

\end{outputs}

\item[(b)]
\begin{verbatim}
% Symbolic variable lambda
syms lambda;

% Characteristic polynomial
char_poly=det(lambda*eye(size(A))-A);

% Expand the characteristic polynomial cf. simplify
polynomial=expand(char_poly);

% Coefficients extraction
coeff=coeffs(polynomial);

% According to the descending order of lambda degree
coefficient=coeff(end:-1:1);

 % Compute the matrix polynomial
poly_A=polyvalm(double(coefficient), A);

disp('Coefficients of the matrix characteristic polynomial is');
disp(double(coefficient));
disp('Matrix characteristic polynomial is'); disp(poly_A);
\end{verbatim}

\begin{outputs}

\begin{verbatim}

Coefficients of the matrix characteristic polynomial is
     1    -2    -3     4     4

Matrix characteristic polynomial is
     0     0     0     0
     0     0     0     0
     0     0     0     0
     0     0     0     0
\end{verbatim}
\end{outputs}

\item[(c)]
\begin{verbatim}
[V D]=eig(A);
sum_A=0;
for i=1:size(A,1);
  % spectral decomposition
  sum_A=sum_A+D(i,i)*V(:,i)*V(:,i)';

  fprintf('lambda_%d is %f \n', i, D(i,i));
  fprintf('corresponding u_%d is \n', i);
  disp(V(:,i));
end

disp('spectral decomposition of A is'); disp(sum_A);
disp('A is'); disp(A);
\end{verbatim}

\begin{outputs}

\begin{verbatim}
lambda_1 is -1.000000
corresponding u_1 is
   -0.7071
         0
    0.7071
         0

lambda_2 is -1.000000
corresponding u_2 is
         0
    0.7071
         0
   -0.7071

lambda_3 is 2.000000
corresponding u_3 is
         0
    0.7071
         0
    0.7071

lambda_4 is 2.000000
corresponding u_4 is
   -0.7071
         0
   -0.7071
         0

spectral decomposition of A is
    0.5000         0    1.5000         0
         0    0.5000         0    1.5000
    1.5000         0    0.5000         0
         0    1.5000         0    0.5000

A is
    0.5000         0    1.5000         0
         0    0.5000         0    1.5000
    1.5000         0    0.5000         0
         0    1.5000         0    0.5000
\end{verbatim}
\end{outputs}
\end{enumerate}
\end{sol}

\end{exer}

% 8.4

\section{Quadratic Forms}

\begin{exer}(\textit{Cholesky Factorization})\\
In this problem, we find a Cholesky factorization of the Hilbert matrix
$$A=\begin{bmatrix}1 & 1/2 & 1/3 & 1/4\\ 1/2 & 1/3 & 1/4 & 1/5 \\ 1/3 & 1/4 & 1/5 & 1/6 \\ 1/4 & 1/5 & 1/6 & 1/7\end{bmatrix}.$$
To generate the Hilbert matrix, you may use the MATLAB command \textit{hilb}.
\begin{enumerate}
\item[(a)] Show that $A$ is positive definite symmetric matrix by finding its eigenvalues and the MATLAB command \textit{issymmetric}.
\vspace{1mm}
\item[(b)] Make a function file \verb"ludecomp.m" to find the $LU$-decomposition of an invertible $n \times n$ matrix $A$ such that $A$ can be reduced to row echelon form by Gaussian elimination without row interchanges. You may refer to the four steps given in Page 157. Check your result by applying this function for the matrix given in the Example 2 of the Section 3.7.
\vspace{1mm}
\item[(c)] Referring to the Section 3.7, find the $LDU$-factorization of $A$ from an $LU$-factorization of $A$ by \verb"ludecomp.m".
\vspace{1mm}
\item[(d)] From the $LDU$-factorization of $A$ obtained in (c), find a Cholesky factorization $A=R^{T}R$, where $R$ is upper triangular.
\vspace{1mm}
\item[(e)] Use the MATLAB command \textit{chol} to find a Cholesky factorization of $A$. Compare it with the result obtained in (d).
\end{enumerate}

\begin{sol}
\verb""
\begin{enumerate}
\item[(a)]
\begin{verbatim}
format rat;
A=hilb(4);

eig_val=eig(A);
if all(eig_val>0) && issymmetric(A)==1
    disp('Given matrix A is'); disp(A);
    disp('A is symmetric and positive definite matrix.');
    fprintf('eigen value of A is '); disp(eig_val');
end
\end{verbatim}

\begin{outputs}
\begin{verbatim}
Given matrix A is
       1              1/2            1/3            1/4
       1/2            1/3            1/4            1/5
       1/3            1/4            1/5            1/6
       1/4            1/5            1/6            1/7

A is symmetric and positive definite matrix.
eigen value of A is    66/682507   101/14989   262/1549   3500/2333
\end{verbatim}
\end{outputs}

\item[(b)]
\begin{verbatim}
%--- This is a function file 'ludecomp.m' ---%
function [L, U] = ludecomp(A)

% The number of rows and columns of the matrix A.
[nrow, ncol] = size(A);

% Initialization of U and L.
U = A; L = eye(ncol);

% Forward Elimination %
for i=1:nrow
    % Find the first nonzero entry of the ith row.
    for k=i:ncol
        if U(i,k) ~= 0
            break % Terminates the execution of the loop.
        end
    end
    temp1 = U(i,k); % Save U(i,k) in temp.
    U(i,:) = (1/temp1) * U(i,:);
    % Normalize the pivot (i,k)-entry by 1 to the ith row.
    L(i,i) = (1/temp1)^(-1);
    % Place the reciprocal of the multiplier in that position in U.
        if i ~= nrow
        for j=(i+1):nrow
            temp2 = U(j,k); % Save U(j,k) in temp2.
            U(j,:) = ((-temp2) * U(i,:)) + U(j,:);
            % Add minus (j,k)-entry times the ith row to the jth row
            L(j,i) = -(-temp2);
            % Place the negative of the multiplier in that position in U.
        end
    end
end
\end{verbatim}

\item[(c)]
\begin{verbatim}
% LU-factorization of A without row interchanges
[L, U] = ludecomp(A);

% From an LU-factorization of A, we can find the LDU-factorization of A,
% by appropriate normalization of L.
D = diag(diag(L));

for i = 1:4
    L(:, i) = L(:, i)./L(i, i);
end

disp('The LDU-factorization of A is');
disp('L = '); disp(L); disp('D = '); disp(D); disp('U = '); disp(U);

\end{verbatim}

\begin{outputs}
\begin{verbatim}

The LDU-factorization of A is
L =
       1              0              0              0
       1/2            1              0              0
       1/3            1              1              0
       1/4            9/10           3/2            1

D =
       1              0              0              0
       0              1/12           0              0
       0              0              1/180          0
       0              0              0              1/2800

U =
       1              1/2            1/3            1/4
       0              1              1              9/10
       0              0              1              3/2
       0              0              0              1
\end{verbatim}
\end{outputs}

\item[(d)]
\begin{verbatim}
% From the LDU-factorization of A, find the Cholesky factor.
R1 = (L*sqrt(D))';
disp('The Cholesky factor R1 from the LDU-factorization of A is');
disp(R1);
\end{verbatim}

\begin{outputs}
\begin{verbatim}

The Cholesky factor R1 from the LDU-factorization of A is
       1              1/2            1/3            1/4
       0            390/1351       390/1351       351/1351
       0              0            317/4253       323/2889
       0              0              0            153/8096
\end{verbatim}
\end{outputs}

\item[(e)]
\begin{verbatim}
% Find a Cholesky factorization of A by using the MATLAB command.
R2 = chol(A);
disp('The Cholesky factor R2 from the MATLAB command chol is');
disp(R2);
\end{verbatim}

\begin{outputs}
\begin{verbatim}

The Cholesky factor R2 from the MATLAB command chol is
       1              1/2            1/3            1/4
       0            390/1351       390/1351       351/1351
       0              0            317/4253       323/2889
       0              0              0            153/8096
\end{verbatim}
\end{outputs}
\end{enumerate}
\end{sol}

\end{exer}

_codes/intro/Learning MATLAB with Linear Algebra (Jeon, Lee)/section8.tex

files/intro/Learning MATLAB with Linear Algebra (Jeon, Lee)/section8.tex


\section{Accessing the data}
Optimization has also a \textbf{hardware} component: accessing the data has to be done efficiently, measuring and modeling physical costs.

\begin{wrapfigure}{L}{0.55\textwidth}
	\vspace{-15pt}
	\includegraphics[width=0.55\textwidth]{disk.png}
	\vspace{-25pt}
\end{wrapfigure}

Older devices store memory in a stack of \textit{rotating disks} on which an arm writes data with its head. Each disk is divided in \textbf{tracks} and \textbf{sectors}, of which the outer are longer than the inner.

Cylinders are organized in zones: each of them contains a fixed number of consecutive cylinders, with the same amount of sectors per track.

Since the disk is rotating, the throughput is higher on outer cylinders, and reading is only possible where the head is, making this operation quite slow since it is impossible to jump. On SSD, this is indeed implemented.

A good approximation of the \textbf{seek time} among $d$ cylinders is:
$$seektime(d) = \begin{cases}
	c_1 + c_2\sqrt{d} & d \leq c_0 \\
	c_3 + c_4d & d> c_0
\end{cases}$$
Constants indicate the maximum number of cylinders where no coast take place, i. e. the point in which the head can be efficient.

The square is introduced because in the low part of hardware the distance is less, hence the head moves faster. On the other hand, the formula is linear if the distance is short.

Since the current position on the disk is unpredictable, it is difficult to produce single cot estimation, however modeling many accesses gives a somewhat accurate measurement.

\begin{wrapfigure}{R}{0.5\textwidth}
	\vspace{-15pt}
	\includegraphics[width=0.5\textwidth]{parameters.png}
	\vspace{-30pt}
\end{wrapfigure}

Parameters are aggregated, introducing average latency time (for positioning) and sustained read/write assuming a sequential scan.

The time a disk needs to read and transfer $n$ bytes is approximately $D_{lat} + \frac{n}{D_{srr}}$. Examples of these values are given in the table.

Database developers, as introduced before, distinguish between \textbf{random} and \textbf{sequential} access. The latter is faster, since it assumes data only needs to be \textit{read in sequence} after finding its position, while a random access implies seeking the location of each page (the smallest portion which can be read, generally 8kb).

Example: to read 100MB, sequential read takes $5ms + 1ms$, while random read has to scan 8k pages and takes $65s$.

This has consequences in practice: for instance, an index may be efficient only if a \textit{small portion of the data is accessed}, else there is a risk for the whole structure to be traversed with a random access behavior. On the other hand, it could be expensive to perform a scan if there are many pages involved.

\subsection{Counting the number of accesses}
A proposed cost function to calculate sequential scan time is:
$$5ms + \frac{\abs{page\_size}}{\abs{bandwidth}} * \#pages$$
Now, the problem becomes estimating the bandwidth, the number of needed pages and their cost. These can vary between different use cases, or be influenced by indexes.

More parameters are introduced for a better overview, assuming a uniform distribution of the tuples:
\begin{itemize}
	\item $N$, number of tuples (items) in relation $R$;
	\item $m$, number of pages (buckets) in which tuples of $R$ are stored ($m = 1$ all pages are accessed);
	\item $B = \frac{N}{m}$, tuples for page (blocking factor);
	\item $k$, number of distinct TID ($k = 1$ implies one page is accessed).
\end{itemize}
The probability to request a set with $k$ items is $\frac{1}{{{N}\choose{k}}}$, because of the uniformity assumption.

\subsubsection{Yao's formula (direct, uniform, distinct)}
Considering $m$ buckets with $n$ items, then there is a total of $N = nm$ items. Randomly selecting $k$ \textbf{distinct} items give a number of qualifying buckets which is:

$$\bar{y}^{N, m}_n(k) = m * y^N_n(k)$$
$$y^N_n(k) = \begin{cases}
	[1 - p] & k \leq N - n \\
	1 & k > N - n
\end{cases}$$
$y^N_n(k)$ is the probability that bucket $n$ contains at least one tuple, and $p$ is the probability that a bucket contains none of the $k$ items.
\begin{wrapfigure}{R}{0.25\textwidth}
	\vspace{-25pt}
	\includegraphics[width=0.25\textwidth]{yao.png}
	\vspace{-30pt}
\end{wrapfigure}
$$p \quad = \quad \frac{{{N-n}\choose{k}}}{{{N}\choose{k}}} \quad = \quad \prod_{i=0}^{k-1} \frac{N-n-i}{N-i} \quad = \quad \prod_{i=0}^{n-1}\frac{N-k-i}{N-i}$$
These formulas are useful to avoid computing $N!$, which is too expensive to do in practice. The choice of formula depends on $N$ and $k$.

However, even with simplifications, calculations are not cheap, and fractions may not be integers: further steps must be taken, such as introduction of a Gamma function and approximations (\textbf{Waters}):
$$p \approx \Big( 1 - \frac{k}{N} \Big)^n$$
This works implying there is a fraction of $\frac{k}{N}$ tuples which are relevant for the query, multiplied for the number of pages. The problem here is, if the first tuple does not qualify, the likelihood that a latter one qualifies increases (there is less space); in general, probability is not uniform.

The estimation is roughly accurate with $N >> n$, and much cheaper to compute. There are other formulas which are simpler, but more complicated to express (\textbf{Bernstein}):
$$y_n^{N,m}(k) \approx \begin{cases}
	k & k < \frac{m}{2} \\
	\frac{k+m}{3} & \frac{m}{2} \leq k \leq 2m \\
	m & 2m \leq k
\end{cases}$$
This evaluation tries to interpolate $k$ depending on its comparison with $m$, and is faster to compute. Each case is plausible, yet not very precise.

Dihr and Saharia computed some upper and lower bounds for $p$, claiming these are accurate with a minimal difference from real-case scenarios.

\subsubsection{Cheung's formula (direct, uniform, non-distinct)}
Index nested loop joins often involve reading the same tuple multiple times, fetching join partners using the index. This strategy uses very little memory, with time linear in the left side.

In this case, Yao's formula is not sufficient, and a \textbf{multiset} (set with duplicates) approach is introduced. The number of multiset with cardinality $k$ containing only elements from a set $S$ with $\abs{S} = N$ is:
$${N+k-1}\choose{k}$$

This works transforming a multiset into a set by summing a crescent value from 1 to $(k-1)$ to each element, making them unique. The other way around applies subtraction.

Cheung's formula is an extension of Yao's formula considering multisets (not necessarily distinct items among $k$):
$$\overline{Cheung}^{N, m}_n(k) = m * Cheung^N_n \qquad Cheung^N_n(k) = [1 - \tilde{p}] \qquad \tilde{p} \approx \Big( 1 - \frac{n}{N}\Big)^k$$
$$\tilde{p} \quad = \quad \frac{{n-n+k-1\choose k}}{{n+k-1\choose k}} \quad = \quad \prod_{i=0}^{k-1} \frac{N-n+i}{N+i} \quad = \quad \prod_{i=0}^{n-1} \frac{N-1-i}{N-1+k-i}$$
This just expands binomials using the duplicates property. The approximation of $p$ (\textbf{Cardenas}) is derived assuming there are $1 - \frac{n}{N}$ tuples on the current page and applying the power.

The number of \textbf{distinct} values in a $k$-multiset of cardinality $N$ with uniform distribution is:
$$D(n, k) = \frac{Nk}{N + k - 1}$$
This follows by previous statements, expanding the binomial coefficient. This allows to do a simplification (\textit{model switching}), i. e. using Yao's formula after removing duplicates.

Non-uniform distributions work similarly, yet modeling each probability to access a group of buckets, using summation instead of product.

\subsection{Sequential accesses on disk}
Accesses on disk are relevant to estimate the cost of \textbf{jumping between two pages or cylinders}: this directly depends on the distance gap. When estimating seek costs, there must be a probability distribution for the distances.

Assuming the situation is a \textbf{bitvector} of length $B$ with $b$ bits set to 1, then $B - b$ bits are zero, and the accesses can be modeled with $B$ corresponding to the number of cylinders while $b$ indicates that a cylinder qualifies.

Then, the probability distribution of the number $j$ of zeros between two consecutive ones, before the first or after the last is:
$$B^B_b(j) = \frac{{B-j-1\choose b-1}}{{B\choose b}} = \frac{b}{B-j}\prod_{i=0}^{j-1} \Big( 1 - \frac{b}{B - i}\Big)$$
The distance between two ones is obtained taking the expected value of the number of zeros, and adding one for the potential first or last position.

Some other values can be deduced from this, such as the total number of bits from the first bit to the last one (extremes included), which is:
$$B_{tot}(B, b) = \frac{Bb + b}{b + 1}$$
All these computations have several real-life applications: the original motivation implies retrieving values from a B-tree and accessing them on disk, sorting the entries to avoid multiple random lookups. A bitvector module gives information about the distance between pages, to understand how expensive a fetch can be.

\section{Selectivity estimations}
Selectivity estimation has always been assumed to be known until now, yet it is a statistic which needs to be computed, since it is essential for query optimization.

Unfortunately, this is a \textit{fundamentally difficult problem}, and can take quite a long time.

Specifically, there are different selectivity problems requiring different approaches. For arbitrarily difficult queries it is impossible to make an estimation, yet the following three are the most common:

\begin{lstlisting}[language=SQL]
SELECT *
FROM relation r
WHERE r.a = 10

SELECT *
FROM relation r
WHERE r.b > 2

SELECT *
FROM relation1 r1, relation2 r2
WHERE r1.a = r2.b
\end{lstlisting}

\subsection{Heuristic estimations}
There are some textbook estimations which are not advanced strategies, depending on the predicate.

\begin{wrapfigure}{R}{0.6\textwidth}
	\vspace{-20pt}
	\includegraphics[width=0.6\textwidth]{selectivity.png}
	\vspace{-30pt}
\end{wrapfigure}

Knowing the domain of $A$, for instance, can happen when the column has an \textbf{index}, yet an uniform distribution must be assumed.

Having a range query is also helped by an index, since this additional data structure stores the minimum and maximum value, hence selection can be obtained by interpolation.

Joins again depend on indices and domain of the involved tables.

\subsection{Building histograms}
Heuristics are quite simple and have the restriction of uniform distribution, so multiple DBMS employ histograms: aggregated data is \textbf{partitioned in buckets} such that $H_A(b) = |\{r\ |\ r \in R \;and R.A \in b\}|$ and thus $\sum_{b \in B} H_A(b) = \abs{R}$.

This methods leads to a much better estimation than previous ones, and computing histograms is also easy since it only involves a sequential scan: the real challenge is selecting the appropriate $B$.

Assuming data is already in buckets, there are formulas giving the appropriate result, applying linear interpolation to the newly obtained histogram:
$$A = c \quad \frac{\sum_{b \in B:c \in b H_A(b)}}{\sum_{b \in B}H_A(b)}$$
$$A > c \quad \frac{\sum_{b \in B:c \in b \frac{max(b) - c}{max(b) - min(b)} H_A(b) + \sum_{b \in B:min(b) > c} H_A(b)}}{\sum_{b \in B}H_A(b)}$$
$$A_1 = A_2 \quad \frac{\sum_{b_1 \in B_1, b_2 \in B_2, b' = b_1 \cap b_2: b' \neq \emptyset} \frac{max(b') - min(b')}{max(b_1) - min(b_1)} H_{A_1}(b_1) \frac{max(b') - min(b')}{max(b_2) - min(b_2)} H_{A_2}(b_2)}{\sum_{b_1 \in B_1}H_{A_1}(b_1) \sum_{b_2 \in B_2}H_{A_2}(b_2)}$$

These computations only give an upper bound, while the lower bound can be up to zero, and it is impossible to predict. Furthermore, the upper bound can be quite an overestimation in the first case, which represents only a rough approximation.

The problem comes from the restriction in the mathematical meaning of $P(A = c) = 0$, which gives a sound result when $A > c$ but fails when the query involves equality. Joining works using some assumptions as well, such as independence.

\subsubsection{Equiwidth}
Building histograms concerns the previously mentioned issue: finding the optimal number of buckets. Typically, this is a fixed value, and it has to work with an unknown distribution.

One particular strategy is partitioning the domain into buckets of equal width, simply constructing them from left to right. This is easy to compute, since it does not require boundaries: it is enough to store minimum, maximum and size.

However, it fails when data is skewed, causing uneven buckets and therefore a greater estimation error.

\subsubsection{Equidepth}
Another method is the equidepth, forcing each bucket to have the same number of elements, adopting to data distribution. This technique is very common and not too time consuming, so it is widely used in practice.

The only disadvantages are having to sort the values beforehand, and store boundaries and ties (obviously not the count).

\subsubsection{Interpolation}
Interpolation is a technique to improve accuracy, however it can be difficult to interpolate a histogram: a potential solution is using the distribution function instead, which is continuous and monotonic. This works particularly well in the case $A > c$.

Further research includes analyzing correlations, multi-dimensional histograms and cardinality estimators.


\section{Introduction to Proofs}

\frame{
{Part 1: Introduction to Proofs.}

\tableofcontents[currentsection,hideallsubsections, firstsection=2, sections={2-4}]
}

\subsection{What is a Proof?}

\begin{frame}{What is a proof?}

  Some concepts are easy to understand, but not easy to show that they are true.\bigskip

  \begin{columns}
    \column{0.3\textwidth}
      \centering
      \includegraphics[width=.5\textwidth]{../img/triangle_rect}
    \column{0.7\textwidth}
      \begin{itemize}
        \item Pythagoras Theorem:
        \begin{equation*}
          a^2+b^2=c^2
        \end{equation*}
        \item It is easy to show this is true for {\bf any one triangle}.
        \item But how do you show it is is true for {\bf all} triangles?
      \end{itemize}
  \end{columns}
  \bigskip

  The proof of the Pythagoras theorem is {not obvious}: there are more than 100 different proofs!
\end{frame}

\begin{frame}{What is a proof?}{One Pythagoras Proof}

  \begin{columns}[T]
    \column{0.7\textwidth}
      \begin{itemize}
        \item {\bf Proof}: by geometric construction
        \item Arrange four identical triangles;
        \item Show that internal angles are right;
        \item Internal square area: $c^2$
        \item External square area: $(a+b)^2$
        \item $(a+b)^2 = c^2 + 4(\text{area triangle})$
        \item $(a+b)^2 = c^2 + 4(\frac{ab}{2})$
        \item $a^2 + 2ab + b^2 = c^2 + 2ab$
        \item $a^2 + b^2 = c^2$ \hfill $\blacksquare$
      \end{itemize}
    \column{0.3\textwidth}
      \includegraphics[width=\textwidth]{../img/triangle_pytagoras}
  \end{columns}
  \bigskip

  {\bf Remember:} There are many other possible proofs.\\(Beautiful proofs, short proofs, wrong proofs, etc.)
\end{frame}

\begin{frame}{False Proofs}{Infinite Chocolate!}
  \begin{center}
    \includegraphics[width=.9\textwidth]{../img/false_proof}
  \end{center}
  \begin{itemize}
    \item What is wrong with the proof above?
    \item \alert{Be careful!} A false proof can have many correct steps, and {\bf only one} impossible step.
  \end{itemize}

  \begin{block}{}
    We can use proofs to show that something is {\bf incorrect} as well!
  \end{block}
\end{frame}

\subsection{Proofs and Computer Science}

\begin{frame}{Proofs and Computer Science}{Why are proofs important for Computer Science?}

  \begin{itemize}
    \item We don't use proofs only for mathematical equations.
    \item We can use proofs to {\bf show that a program is correct}. (or incorrect)
    \vfill

    \item Example cases:
    \begin{itemize}
    \item Use proofs to show that the result of a program is correct for any input;
    \item Use proofs to show that one type of input will cause a bug in the program;
    \item Use proofs to show that a program finishes in $N$ steps;
    \end{itemize}
  \end{itemize}
\end{frame}

\begin{frame}[fragile]{Proofs and Computer Science}{Example:}
  \begin{itemize}
    \item Is the program below correct or incorrect?
    \item Can you show by using a proof?
  \end{itemize}

\begin{verbatim}
int triangle_type(int a, int b, int c)
  // a, b, c are the length of the sides of a triangle;
  if (a == b)
    if (b == c)
      return "all sides are equal";
    else
      return "two sides are equal";
  else if (b == c)
    return "two sides are equal";
  else
    return "all sides are different";
\end{verbatim}
\end{frame}

\subsection{Definitions}

\begin{frame}{Proof Concepts}{Propositions}
  A proposition is a statement that is either \structure{True} or \alert{False}, and nothing else.\bigskip

  \begin{columns}[T]
    \column{0.5\textwidth}
    \structure{Proposition}
    \begin{itemize}
      \item $2 + 3 = 5$
      \item $1 + 1 = 3$
      \item $513 \times 435 = 223165$
      \item There is no human taller than 3 meters.
      \item It rained on October, 3rd, 2020, 10:00 in Tokyo.
      \item Emacs is better than Vim.
    \end{itemize}
    \column{0.5\textwidth}
    \alert{Not proposition}
    \begin{itemize}
      \item What is $2 \times 8$?
      \item Please give me cake.
      \item It is raining now.
    \end{itemize}
  \end{columns}
\end{frame}

\begin{frame}{Proof Concepts}{Predicates}
  A predicate is a kind of proposition where the truth value depends on one or more variables:\bigskip

  \begin{itemize}
    \item $P(n)$: $n$ is a prime number;
    \item $L(N)$: The name $N$ has five letters;
    \item $M(x,y)$: $x$ and $y$ are members of the same group;
  \end{itemize}\vfill

  \begin{alertblock}{Do not confuse predicates and number expressions!}
    Numeric expressions have numeric values, predicates have True or False values.
    \begin{itemize}
      \item $p(x) = x^2 + 3x + 1$.\hfill This is a numeric expression;
      \item $P(X)$: $p(x+1) = p(x) + x + 1$.\hfill This is a predicate;
      \item $P(X)$ is True for any $x \geq 0$.\hfill This is a proposition;
    \end{itemize}
  \end{alertblock}
\end{frame}

\begin{frame}{Proof Concepts}{Implication (IF)}
  An \structure{implication} is a particular type of predicate that we use a lot, so it is important to know it well:
  \begin{center}
    $P \implies Q$
  \end{center}
  There are many ways to describe the implication:
  \begin{itemize}
    \item $I(P,Q)$: If P is true, Q is true;
    \item $I(P,Q)$: When P is true, Q is true;
  \end{itemize}
  We usually don't write the $I(P,Q)$ part, but it is important to remember that the \structure{implication} itself is a predicate.
  \bigskip

  \begin{itemize}
    \item \alert{Be Careful!} When P is false, Q could be anything.
    \item A related predicate is {\bf If and only If (iff)}:
    \begin{itemize}
      \item IFF(P,Q): $P\implies Q$ AND $Q\implies P$.
      \item also written as $P\iff Q$
    \end{itemize}
  \end{itemize}
\end{frame}

\begin{frame}{Proof Concepts}{Proof Methods}
  How do we prove something?
  \begin{block}{Proposition}
    For every nonnegative integer $n$, the value of $p(n) = n^2+n+41$ is prime.
  \end{block}\medskip

  We could try to test values of $n$ one by one:
  \begin{center}
    $p(0) = 41$, prime; $p(1) = 43$, prime; $p(2) = 47$, prime; $\ldots$, $p(20) = 461$, prime...
  \end{center}\medskip

  \begin{itemize}
    \item When do we stop?
    \item ($p(40) = 41\times41$, is not prime...)
  \end{itemize}
  We need better ways to prove propositions!
\end{frame}


\section{Risk estimation}\label{sec:RiskEstimators}

Let~$\func{\dec}{\domainX}{\real}$ be a decision function from feature vector~$\X$ to a real number, and let~$\func{\loss}{\domainX \times \domainY}{\real_{{\geq}0}}$ be the loss function.  \textit{Risk estimator},~$\risk$,\footnote{Although expected risk~$\risk$ is a function of~$\dec$ as shown in Eq.~\eqref{eq:RiskEstimator:Expectation}, we drop~$\dec$ from our notation for brevity going forward.} quantifies $\dec$'s~expected loss formally as

\begin{equation}\label{eq:RiskEstimator:Expectation}
  \risk(\dec) = \mathbb{E}_{(\X,\y) \sim \joint}\sbrack{\floss{\decX}{\y}}\text{.}
\end{equation}

Since $\joint$~is unknown, $\risk$~is also unknown so in practice empirical estimate~$\emprisk$ is used.  This section provides an overview of the PN, PU (specifically nnPU), and PUbN risk estimators as well as their empirical estimates.\footnote{All empirical risk estimates described here support both batch and stochastic gradient descent.}

\subsection{PN --- positive-negative}

PN~classification has access to both positive and negative labeled examples.  Therefore, Eq.~\eqref{eq:RiskEstimator:Expectation} exactly specifies its expected risk.

\paragraph{Empirical Estimation} Estimating the PN~empirical risk is straightforward as shown in Eq.~\eqref{eq:EmpRisk:PN}; it is merely the mean loss for all examples in~$\train$.  This formulation applies irrespective of any covariate shift/bias.

\begin{equation}\label{eq:EmpRisk:PN}
  \emprisk = \frac{1}{\abs{\train}} \sum_{(\X,\y) \in \train} \floss{\decX}{\y}
\end{equation}

\subsection{nnPU --- non-negative positive-unlabeled}

Since positive\-/unlabeled~(PU) learning has no negative labeled examples, traditional supervised learning cannot be used. By Bayes' Rule, the expected risk can be decomposed into the risk associated with each label (positive and negative) as shown in Eq.~\eqref{eq:Risk:Bayes}.  ${\varrisk{D}{\ypred}}$ denotes the expected loss when predicting label~$\ypred$ for samples drawn from distribution~${\pDist_{D}}$ where ${D \in \set{\textnormal{P}, \textnormal{N}, \X}}$.

\begin{align}
  \risk &= \prior \mathbb{E}_{\X \sim \pcond}\sbrack{\floss{\decX}{\pcls}} + (1-\prior) \mathbb{E}_{\X \sim \ncond}\sbrack{\floss{\decX}{\ncls}} \nonumber \\
        &= \prior \prisk{P} + (1-\prior) \nrisk{N} \label{eq:Risk:Bayes}
\end{align}

Since the unlabeled set is drawn from marginal distribution,~$\marginal$, it is clear that:

\begin{align}
  \nrisk{U} &= \mathbb{E}_{\X \sim \marginal} \sbrack{\floss{\X}{\ncls}} \nonumber \\
            &= \prior \mathbb{E}_{\X \sim \pcond} \sbrack{\floss{\X}{\ncls}} + (1 - \prior) \mathbb{E}_{\X \sim \ncond} \sbrack{\floss{\X}{\ncls}}\nonumber \\
            &= \prior \nrisk{P} + (1 - \prior) \nrisk{N} \label{eq:Risk:Unlabeled}
\end{align}

\noindent
Rearranging the above and combining it with Eq.~\eqref{eq:Risk:Bayes} yields the unbiased positive\-/unlabeled~(uPU) risk estimator below.~\cite{duPlessis:2014}

\begin{equation}\label{eq:Risk:uPU}
  \risk = \prior \prisk{P} + \nrisk{U} - \prior \nrisk{P}
\end{equation}

\paragraph{Non\-/negativity} By the definitions of~$\loss$ and~$\risk$, it is clear that $\nrisk{N}$~must be non\-/negative.  However, highly expressive learners (e.g.,~neural networks) often cause ${\nrisk{U} - \prior \nrisk{P}}$ to slip negative --- primarily due to overfitting.  Kiryo\etal~\cite{Kiryo:2017} proposed the non\-/negative positive\-/unlabeled~(nnPU) risk estimator in Eq.~\eqref{eq:Risk:nnPU}; the primary difference versus uPU is the negative risk surrogate is explicitly forced non\-/negative by the~$\max$.

\begin{equation}\label{eq:Risk:nnPU}
  \risk = \prior \prisk{P} + \max\left\{0, \nrisk{U} - \prior \nrisk{P} \right\}
\end{equation}

Whenever the negative risk surrogate is less than~0, nnPU updates the model parameters using special gradient ${\gamma \nabla_{\theta} \prior \nrisk{P} - \nrisk{U}}$ where hyperparameter~${\gamma \in (0,1]}$ attenuates the learning rate.  Observe that the negative risk surrogate is deliberately negated; this is done to ``defit'' the learner so that it no longer underestimates the negative class' expected risk.

Although forcing~$\nrisk{N}$'s surrogate to be non-negative introduces an estimation bias (i.e.,~expected value does not equal the true expectation), nnPU often performs better in practice and guarantees ERM uniform convergence.

\paragraph{Empirical Estimation} Each risk term in nnPU can be empirically estimated from the training set where:

\begin{equation}\label{eq:EmpRisk:Pos}
  \evrisk{P}{\ypred} = \frac{1}{\abs{\ptrain}} \sum_{\X \in \ptrain} \floss{\decX}{\ypred}
\end{equation}

\noindent
and for unlabeled set ${\utrain \sim \marginal}$,

\begin{equation}
  \evrisk{U}{-} = \frac{1}{\abs{\utrain}} \sum_{\X \in \utrain} \floss{\decX}{\ncls} \text{.}
\end{equation}

\noindent
Prior~$\prior$ and attenuator~$\gamma$ are hyperparameters.

\subsection{PUbN --- positive, unlabeled, biased-negative}

Let $\latent$~be a latent random variable representing whether tuple~${(\X,\y)}$ is eligible for labeling.  The joint distribution,~${\trijoint = \pDist(\X, \y, \latent)}$, then becomes trivariate.  By definition, ${\pDist(\latent = \pcls \vert \X, \y = \pcls) = 1}$ (i.e.,~no positive selection bias) or equivalently ${\pDist(\y = \ncls \vert \X, \latent = \ncls) = 1}$.  The biased-negative conditional distribution is therefore~${\bncond = \pDist(\X \vert \y = \ncls, \latent = \pcls)}$.

The marginal distribution can be partitioned as

\begin{equation*}
  \begin{aligned}
    \marginal = &\pDist(\y = \pcls) \pDist(\X \vert \y = \pcls) \\
                &+ \underbrace{\pDist(\y = \ncls, \latent = \pcls)}_{\plabel}\pDist(\X \vert \y = \ncls, \latent = \pcls)
                + \underbrace{\pDist(\y = \ncls, \latent = \ncls)}_{1 - \prior - \plabel} \pDist(\X \vert \y = \ncls, \latent = \ncls)
  \end{aligned}
\end{equation*}

\noindent
where ${\plabel = \pDist(\y = \ncls, \latent = \pcls)}$ is a hyperparameter. The expected risk therefore becomes

\begin{equation}\label{eq:Risk:WithBN}
  \risk = \prior \prisk{P} + \plabel \nrisk{bN} + (1 - \prior - \rho) \smrisk
\end{equation}

Define ${\sigma(\X) = \pDist(\latent = \pcls \vert \X)}$.  While the proof is well beyond the scope of this document, Hsieh\etal~\cite{Hsieh:2018} proved that with guaranteed estimation error bounds $\smrisk$~decomposes as

\begin{equation}\label{eq:ExpectedRisk:PUbN:Latent}
  \begin{aligned}
    \smrisk = &\mathbb{E}_{\X \sim \marginal}\sbrack{\mathbbm{1}_{\sigX \leq \eta} \floss{\decX}{\ncls} \sigdiff} \\
              &+ \prior \mathbb{E}_{\X \sim \pcond} \sbrack{\mathbbm{1}_{\sigX > \eta} \floss{\decX}{\ncls} \frac{\sigdiff}{\sigX}} \\
              &+ \plabel \mathbb{E}_{\X \sim \bncond} \sbrack{\mathbbm{1}_{\sigX > \eta} \floss{\decX}{\ncls} \frac{\sigdiff}{\sigX}}
  \end{aligned}
\end{equation}

\noindent
where $\mathbbm{1}$~is the indicator function and $\eta$~is a hyperparameter that controls the importance of unlabeled data versus $\textnormal{P}$/$\textnormal{bN}$ data.

\paragraph{Empirical Estimation} Similar to nnPU, $\prisk{P}$~and~$\nrisk{bN}$ can be estimated directly from~$\ptrain$ and~$\bntrain$ respectively.  Estimating~$\smrisk$ is more challenging and actually requires the training of two classifiers.

First, $\sigX$~is empirically estimated by training a positive\-/unlabeled probabilistic classifier with labeled set ${\ptrain \sqcup \bntrain}$ and unlabeled set~$\utrain$; refer to this learned approximation as~$\hsigX$.  Probabilistic classifiers must be adequately calibrated to generate probabilities.  Hsieh\etal\ try to achieve this by training with the logistic loss but that provides no calibration guarantees.~\cite{Guo:2017}

Rather than specifying hyperparameter~$\eta$ directly, Hsieh\etal\ instead specified hyperparameter~$\tau$ to calculate~$\eta$ via

\begin{equation}\label{eq:EtaCalculation}
  \abs{\setbuild{\X \in \utrain}{\hsigX \leq \eta}} = \tau (1 - \prior - \plabel)\abs{\utrain} \text{.}
\end{equation}

\noindent
This approach provides a more intuitive view into the balance between~$\utrain$ and $\ptrain$/$\bntrain$.

PUbN's second classifier minimizes Eq.~\eqref{eq:Risk:WithBN} and empirically estimates the risk when ${\latent = \ncls}$ as

\begin{equation}\label{eq:EmpRisk:PUbN:Latent}
  \begin{aligned}
    \smrisk = &\frac{1}{\abs{\utrain}} \sum_{\xvar{U} \in \utrain} \sbrack{\mathbbm{1}_{\hsig(\xvar{U}) \leq \eta} \floss{\dec(\xvar{U})}{\ncls} \big(1 - \hsig(\xvar{U})\big)} \\
              &+\frac{\prior}{\abs{\ptrain}} \sum_{\xvar{P} \in \ptrain} \sbrack{\mathbbm{1}_{\hsig(\xvar{P}) > \eta} \floss{\dec(\xvar{P})}{\ncls} \frac{1 - \hsig(\xvar{P})}{\hsig(\xvar{P})}} \\
              &+\frac{\plabel}{\abs{\bntrain}} \sum_{\xvar{bN} \in \bntrain} \sbrack{\mathbbm{1}_{\hsig(\xvar{bN}) > \eta} \floss{\dec(\xvar{bN})}{\ncls} \frac{1 - \hsig(\xvar{bN})}{\hsig(\xvar{bN})}} \text{.}
  \end{aligned}
\end{equation}


\section{Elementary matrices}
\label{sec:elementary-matrices}

\begin{outcome}
  \begin{enumerate}
  \item Use multiplication by elementary matrices to apply row
    operations.
  \item Find the elementary matrix corresponding to a particular row
    operation.
  \item Write the {\rref} of a matrix $A$ in the form $R=UA$, where
    $U$ is invertible.
  \item Write a matrix as a product of elementary matrices.
  \end{enumerate}
\end{outcome}


\chapter{SLAM}
\label{chp:slam}

This chapter describes the basic concepts behind the robotics problem known as \emph{Simultaneous Localization And Mapping (SLAM)}, which addresses the issues of when a robot doesn't know the map of its environment.
The principle behind SLAM is for a robot to use measurements and control metrics to construct a map, while simultaneously localizing it self relative to this map, as it moves within the given environment.

SLAM is currently a substantial topic of research within robotics, with a lot of advanced methods, because the robot might loose track of where it is by virtue of its own motion uncertainties.
However this report will focus on Graph SLAM, an older version of SLAM that is easily comprehensible and serves as a good introduction to the concepts of SLAM in practice.\\\\

\noindent In Graph SLAM we can reduce the mapping problem to additions into a matrix and a vector, and a simple matrix multiplication.\\
The matrix, $\Omega$, and the vector, $\xi$, is generated by gathering the following constraints of the robot:
\begin{itemize}
    \item The initial location
    \item Relative motion
    \item Relative measurements to landmarks
\end{itemize}
The matrix and the vector can then be used find the best estimate, $\mu$ of both all the robot locations and all the landmark positions, by the formula:
\begin{align*}
    \mu = \Omega^{-1} \xi
\end{align*}
The best way to explain the process is through a 1D example.

\myFigure{slamEx1}{Illustration of Graph SLAM example. The triangles represent the robot at different times, the cylinders represents landmarks, the solid arrows represents the robots movements and the dashed arrow represents measurements.}{fig:slamEx1}{0.7}

The robot has an initial location which we define as $x_0 = 0$, and moves 5 steps forward to a new position, $x_1$.
This movement constraint can be described as:
\begin{align*}
x_1 &= x_0 + 5\\
-5  &= x_0 - x_1
\end{align*}

\noindent These equations, including the initial location equation, $x_0 = 0$,  are what should be added into the matrix $\Omega$ and the vector $\xi$, such that we get:
\begin{table}[!h]
    \centering
\begin{tabular}{cc|rrrrr c crc}
          &       & $x_0$ & $x_1$ & $x_2$ & $L_0$ & $L_1$ &

          \qquad &        &                        & \\

          \cline{2-7}
          & $x_0$ &   2   &  -1   &  0    &  0    &   0   &

                 &        &\multicolumn{1}{r|}{-5} & $x_0$\\

          & $x_1$ &  -1   &   1   &  0    &  0    &   0   &

                 &        & \multicolumn{1}{r|}{5} & $x_1$\\

$\Omega=$ & $x_2$ &   0   &   0   &  0    &  0    &   0   &

                 & $\xi=$ &\multicolumn{1}{r|}{0} & $x_2$\\

          & $L_0$ &   0   &   0   &  0    &  0    &   0   &

                 &        & \multicolumn{1}{r|}{0} & $L_0$\\

          & $L_1$ &   0   &   0   &  0    &  0    &   0   &

                 &        & \multicolumn{1}{r|}{0} & $L_1$\\
\end{tabular}
\end{table}

\noindent Now suppose that the robot moves backwards by 4 steps to $x_2$:
\begin{align*}
x_2 &= x_1 - 4\\
-4  &= x_2 - x_1
\end{align*}
This is now added to the already existing $\Omega$ and $\xi$, such that we get:
\begin{table}[!h]
    \centering
    \begin{tabular}{cc|rrrrr c crc}
        &       & $x_0$ & $x_1$ & $x_2$ & $L_0$ & $L_1$ &

        \qquad &        &                        & \\

        \cline{2-7}
        & $x_0$ &   2   &  -1   &  0    &  0    &   0   &

        &        &\multicolumn{1}{r|}{-5} & $x_0$\\

        & $x_1$ &  -1   &   2   &  -1    &  0    &   0   &

        &        & \multicolumn{1}{r|}{9} & $x_1$\\

        $\Omega=$ & $x_2$ &   0   &   -1   &  1    &  0    &   0   &

        & $\xi=$ &\multicolumn{1}{r|}{-4} & $x_2$\\

        & $L_0$ &   0   &   0   &  0    &  0    &   0   &

        &        & \multicolumn{1}{r|}{0} & $L_0$\\

        & $L_1$ &   0   &   0   &  0    &  0    &   0   &

        &        & \multicolumn{1}{r|}{0} & $L_1$\\
    \end{tabular}
\end{table}

\noindent If the robot at position $x_1$ saw a landmark, $L_0$, at a distance of 9 away and at position $x_2$ saw a landmark, $L_1$, at a distance of -3 away,  this should be added to the matrix and vector as well.
\begin{align*}
x_1 - L_0 &= -9\\
9  &= L_0 - x_1\\
x_2 - L_1 &= 3\\
-3  &= L_1 - x_2
\end{align*}

\newpage
Giving us:
\begin{table}[!h]
    \centering
    \begin{tabular}{cc|rrrrr c crc}
        &       & $x_0$ & $x_1$ & $x_2$ & $L_0$ & $L_1$ &

        \qquad &        &                        & \\

        \cline{2-7}
        & $x_0$ &   2   &  -1   &  0    &  0    &   0   &

        &        &\multicolumn{1}{r|}{-5} & $x_0$\\

        & $x_1$ &  -1   &   3   &  -1    &  -1    &   0   &

        &        & \multicolumn{1}{r|}{0} & $x_1$\\

$\Omega=$&$x_2$ &   0   &   -1   &  2    &  0    &  -1   &

        & $\xi=$ &\multicolumn{1}{r|}{-1} & $x_2$\\

        & $L_0$ &   0   &   -1   &  0    &  1   &   0   &

        &        & \multicolumn{1}{r|}{9} & $L_0$\\

        & $L_1$ &   0   &   0   &  -1    &  0    &   1  &

        &        & \multicolumn{1}{r|}{-3} & $L_1$\\
    \end{tabular}
\end{table}

\noindent If we now calculate the landmark and robot positions we get:
\begin{table}[!h]
    \centering
    \begin{tabular}{cr|c}
                                      &  0 & $x_0$ \\
                                      &  5 & $x_1$ \\
       $\mu = \Omega^-1 \cdot \xi = $ &  1 & $x_2$ \\
                                      & 14 & $L_0$ \\
                                      & -2 & $L_1$ \\
    \end{tabular}
\end{table}

\noindent where we can see we get what we expected, for both the robots position at each time index, and the landmarks location.\\\\

\noindent Normally SLAM problem would be in at least 2 dimensions, which Graph SLAM also is able to handle, either by expanding the $\Omega$ matrix and the $\xi$ vector, or by creating separate matrices for each dimension.