\chapter{Non-dimensionalisation}\label{Non-dimensionalisation}
\begin{aquote}{Wild Thing. J. Bazell 2013.}
\textit{In metric, one milliliter of water occupies one cubic centimeter, weighs one gram, and requires one calorie of energy to heat up by one degree centigrade — which is 1 percent of the difference between its freezing point and its boiling point. An amount of hydrogen weighing the same amount has exactly one mole of atoms in it. Whereas in the [imperial] system, the answer to ``How much energy does it take to boil a room-temperature gallon of water?" is ``Go fuck yourself'', because you can't directly relate any of those quantities.}
\end{aquote}

Thus, far we have been fairly lax about defining the quantities we have actually been measuring. Further, once we have specified what the quantity actually is, what units are we using to measure the quantity. For example, if we are measuring distance are we doing it in mm, miles or light-years? Equally, is time measured in seconds, minutes or hours? Finally, constitutive laws often introduce parameters that are not quantified accurately, or alternatively, we may be interested in understanding how a solution depends on a particular parameter as it is varied.

The Law of Mass Action, in particular, could be thought to be a troublesome law as it introduces a rate parameter for each reaction equation that is considered. For example, if a system of ODEs is defined by a set of non-linear equations it is highly unlikely to be solvable in closed form. Thus, if there are a large number of parameters in the system, it becomes very difficult to predict how varying a single parameter (or group of parameters) will influence the solution.

However, we have already seen cases in which we do not need to consider parameters individually, as specific groups of the parameters are seen to act in the same way. For example, in the case of the spring pendulum (see \sect{Pendulums_section}), we saw that the frequency of oscillation depended on $\sqrt{k/m}$. Thus, stiffening the spring (increasing $k$) has the same effect on the solution as decreasing the mass (decreasing $m$), \ie they both increase the frequency of the oscillations. Equally, in the example of a zombie infection (see example \ref{Zombies}), the parameters of interest were not $a$, $b$ or $c$, but rather $\alpha=b+c$ and $\beta=c-a$.

In this chapter we introduce a technique, called non-dimensionalisation, that will benefit us in two ways. Firstly, it allow us to brush away worries about dealing with units and, secondly, it will allow us to reduce the number of effective parameters in our system. Specifically, we will be able to define parameter groupings that will influence the final result in the same way.

\section{The central idea}
To non-dimensionalise a system of equations, we have the following rules:
\begin{enumerate}
\item Identify all the variables;
\item Replace each variable with a quantity scaled relative to a characteristic unit of measure (to be determined);
\item Choose the definition of the characteristic unit for each variable;\label{Choose}
\item Rewrite the system of equations in terms of the new dimensionless quantities.
\end{enumerate}
We note three particular points about these rules. Firstly, the theory behind non-dimensionalisation is straight forward. Namely, we substitute scaled variables into an equation system and massage the equations until we have rearranged the system to produce the desired outcome. However, in practice the difficulty of the technique lies in the algebraic manipulation; it is very easy for the terms to become lost during the manipulation. Thus, care must be taken during the algebraic manipulation stage.

Secondly, you will notice the word `choose' in point \ref{Choose}. This means that it possible to construct many different non-dimensionalised systems from the same system of equations, \ie non-dimensionalisation is non-unique. We usually choose the characteristic unit of each variable to either emphasise one of the terms in a system or to remove as many parameters as possible.

Finally, this technique is hard to demonstrate in generality. It is much better to consider a number of system and see how the technique works in action. Thus, what follows will be a select number of examples, which along with your problem sheets should give you a good basis in the theory. However, do not think that these are all the examples you could face.

It should be noted that there is little consistency in nomenclature across book when considering the separation of variables into their dimensional and non-dimensional components. Thus, always be clear in your definitions.

\subsection{Examples of non-dimensionalisation through substitution of variables}\label{Examples of non-dimensionalisation through substitution of variables}
\begin{example}[frametitle=Substituting variables]
\begin{itemize}
\item Consider the equation for exponential growth,
\bb
\dot{u}=ru, \quad u(0)=u_0.\label{Non-dim_1}
\ee
\COL{The variables in \eqn{Non-dim_1} are $u$ and $t$. We rewrite them as $u=[u]u'$ and $t=[t]t'$, where $[u]$ and $[t]$ are the dimensional scales and $u'$ and $t'$ are the non-dimensional variables. We are free to define the values of $[u]$ and $[t]$ as we please. It is our job to choose appropriate definitions that simplify \eqn{Non-dim_1}. Critically, although we are free to choose the value of $[u]$ and $[t]$ the values must have consistent units. Namely, $[t]$ must have units of time and $[u]$ must have units of density.

We substitute the expanded variables into \eqn{Non-dim_1} and rearrange to produce
\bb
\frac{\rd u'}{\rd t'}=[t]ru', \quad u(0)=\frac{u_0}{[u]}.\nonumber
\ee
Hence, we see that if we choose $[t]=1/r$ and $[u]=u_0$ then \eqn{Non-dim_1} simplifies to
\bb
\dot{u}=u, \quad u(0)=1,\nonumber
\ee
where we note that we have dropped the prime symbols, $'$, for notational convenience.

For mathematicians dropping primes is often done as the last step because we infrequently care about the actual values of the variables, rather we study the dynamics available in the equation. However, in any specific application we should be careful to remember that the variables we are dealing with are non-dimensional and that the solution is not complete until we `re-dimensionalise' the variables.}

\COL{In this example we see that the values of $r$ and $u_0$ in \eqn{Non-dim_1} do not influence the dynamics of the simulation. Specifically, they only scale the time and initial condition.

Although this was a fairly trivial example, a good way to check consistency of the answer at the end of the manipulation is to check that all of the dimensions agree. As mentioned $[t]$ should have units of time and $[u]$ should have units of density. We return to \eqn{Non-dim_1} and consider the dimensions of each component.

For example, $\dot{u}=\rd u/ \rd t$ has units of density/time. By equality, $ru$ must have units of density/time since $u$ has units of density then $r$ must have units of 1/time. Thus,
\bb
\textrm{dim}([t])=\textrm{dim}(1/r)=\textrm{time}.\nonumber
\ee
Equally, $[u]=u_0$ can trivially be seen to have the correct units of density.}

\item Consider the equation for logistic growth,
\bb
\dot{u}=ru\l 1-\frac{u}{K} \r, \quad u(0)=u_0.\label{Non-dim_2}
\ee
\COL{Again, $u=[u]u'$ and $t= [t]t'$ can be substituted into \eqn{Non-dim_2} to produce
\bb
\frac{\rd u'}{\rd t'}=[t]ru'\l 1-\frac{[u]}{K}u'\r,\quad u'(0)=\frac{u_0}{[u]},\nonumber
\ee
from which we see that it would be wise to once again take $[t]=1/r$. Beyond this we see that we have a choice. Should we take $[u]=K$, or $[u]=u_0$? Both are valid non-dimensionalisations and either maybe be appropriate depending on the context of the problem.

Here, we are going to take $[u]=K$ as we are interested in the dynamics of the system, rather than the initial condition. Thus, after dropping primes we see that we can non-dimensionalise \eqn{Non-dim_2} to
\bb
\frac{\rd u}{\rd t}=u\l 1-u\r,\quad u(0)=U_0,\nonumber
\ee
where $U_0=u_0/[u]=u_0/K$.

In this case the non-dimensionalisation demonstrates that the only parameter that the solution depends on is the initial conditions. Changing $r$ does not change the dynamics of the system, it only changes the time scale, since $r=1/[t]$. Equally, changing $K$ simply scales the size of the solution, as $u=Ku'$.}

\item Consider the following equations (the Schnakenberg kinetics)
\begin{align}
&\dot{u}=k_1-k_2u+k_3u^2v, \quad u(0)=u_0,\label{Non-dim_31}\\
&\dot{v}=k_4-k_3u^2v, \quad v(0)=v_0.\label{Non-dim_32}
\end{align}
\COL{This time we use the scales $u=[u]u'$, $v=[v]v'$, $t=[t]t'$ to derive
\begin{align}
&\frac{\rd u'}{\rd t'}=\frac{[t]k_1}{[u]}-[t]k_2u'+[t]k_3[u][v]u'^2v', \quad u'(0)=\frac{u_0}{[u]},\nonumber\\
&\frac{\rd v'}{\rd t'}=\frac{[t]k_4}{[v]}-[t]k_3[u]^2u'^2v', \quad v'(0)=\frac{v_0}{[v]}.\nonumber
\end{align}}\COL{
Lots of potential choices for scale balances; how do we choose? In an exam you will be given the form of an equation to produce and your task will be to derive the corresponding scales. For example, suppose we wanted to convert \eqns{Non-dim_31}{Non-dim_32} into
\begin{align}
&\frac{\rd u'}{\rd t'}=\alpha-u'+u'^2v', \quad u'(0)=u'_0\nonumber\\
&\frac{\rd v'}{\rd t'}=\beta-u'^2v', \quad v'(0)=v'_0,\nonumber
\end{align}
then we know that we would have to set
\bb
1=[t]k_3[u][v]=[t]k_2=[t]k_3[u]^2.\nonumber
\ee
Thus,
\begin{align}
&[t]=\frac{1}{k_2},\nonumber\\
&[u]=\sqrt{\frac{1}{[t]k_3}}=\sqrt{\frac{k_2}{k_3}},\nonumber\\
&[v]=[u]=\sqrt{\frac{k_2}{k_3}},\nonumber
\end{align}
which means that
\begin{align}
\alpha=\frac{[t]k_1}{[u]}=\frac{k_1}{k_2}\sqrt{\frac{k_3}{k_2}},\nonumber\\
\beta=\frac{[t]k_4}{[v]}=\frac{k_4}{k_2}\sqrt{\frac{k_3}{k_2}},\nonumber\\
u'_0=\frac{u_0}{[u]}=u_0\sqrt{\frac{k_3}{k_2}},\nonumber\\
v'_0=\frac{v_0}{[v]}=v_0\sqrt{\frac{k_3}{k_2}}.\nonumber
\end{align}

Finally, we check the consistency of the scales. From \eqns{Non-dim_31}{Non-dim_32} we infer that
\bb
\textrm{dim}(k_1)=\frac{\textrm{density}}{\textrm{time}}, \quad \textrm{dim}(k_2)=\frac{1}{\textrm{time}}, \quad \textrm{dim}(k_3)=\frac{1}{\textrm{density}^2\textrm{time}}, \quad \textrm{dim}(k_4)=\frac{\textrm{density}}{\textrm{time}}.\nonumber
\ee
Hence,
\bb
\textrm{dim}([u])=\sqrt{\frac{1/\textrm{time}}{1/(\textrm{density}^2\textrm{time})}}=\sqrt{\textrm{density}^2}=\textrm{density}.
\ee
The scales $[v]$ and $[t]$ can be checked similarly. We also need to ensure the the variables $\alpha, \beta, u'_0, v'_0$ are have no dimension. For example
\bb
\textrm{dim}(v'_0)=\textrm{density}\sqrt{\frac{1/(\textrm{density}^2\textrm{time})}{1/\textrm{time}}}=\textrm{density}\sqrt{\frac{1}{\textrm{density}^2}}=1.
\ee
The other variables can be checked similarly.}
\end{itemize}
\end{example}

\subsection{Examples of non-dimensionalisation through the arrow method}
The substitution method shown in \sect{Examples of non-dimensionalisation through substitution of variables} will always work supposing that the algebra is manipulated correctly. However, the method can be cumbersome and slow. Moreover, because it involves lots of algebraic manipulations there are many chances to make a mistake.

An alternative method rests on using arrows to identify the desired balances. This can be much quicker as the initial stages do not require laborious substitution. However, we have to be more careful because not all balances that we can `draw' using the arrows will be valid.

The idea behind the arrow method is that you draw arrows between the quantities that are going to `balance', which simply means they are going to have the same coefficient in the final non-dimensionalised form. The process is generally the same as the substitution method. However, we must remember that in order to specify the problem completely the number of valid arrow balances must equal the number of variables. For example, if a problem depends on $u$ and $t$ we would need two balances. Alternatively, if the problem depended on $u$, $v$, and $t$ we would need three valid balances.  This section is going to depend primarily on examples, again, and we will see an invalid balance at the end of the demonstrations.
\begin{example}[frametitle=Arrow method]\label{Arrow method}
\item Consider the following equation
\bb
  \tikzmark{a}\dot{u}=k_0+k_1\tikzmark{b}u+k_2\tikzmark{c}u^2, \quad u(0)=u_0.\label{Non-dim_4}
\tikz[overlay,remember picture]
{\draw[square arrow1] (a.south) to (b.south);}
\tikz[overlay,remember picture]
{\draw[square arrow1] (b.south) to (c.south);}
\ee
\COL{We have two variables, $u$ and $t$, and so we need two balances. Specifically, the arrows state that we want to balance the derivative, linear and quadratic terms,
\bb
\frac{[u]}{[t]}=k_1[u]=k_2[u]^2,\nonumber
\ee
from which it is simple to discover that
\bb
[t]=\frac{1}{k_1},\quad [u]=\frac{k_1}{k_2}.\nonumber
\ee
We still need to substitute the scales into the equations. Namely, $u=u'k_1/k_2$ and $t=t'/k_1$, but again the arrow method simplifies this task. Specifically, we know that, by design, the coefficient of the derivative, linear and quadratic term are going to be the same. Thus, we can divide through by one of them to speed up the derivation,
\bb
\frac{\rd u'}{\rd t'}=\frac{k_0}{k_1[u]}+u'+u'^2.\nonumber
\ee
Finally, redefining the last parameter as $\alpha=k_0/(k_1[u])=k_0k_2/(k_1^2)$ and the initial condition $u'(0)=k_2u_0/k_1=u'_0$, we can non-dimensionalise \eqn{Non-dim_4} to the final form of
\bb
\dot{u}=\alpha+u+u^2,\quad u(0)=u'_0,\label{Non-dim_5}
\ee
where we have dropped the primes from the variables for simplicity. Once again, we would have to ensure that $\alpha$ and $u'_0$ where non-dimensional and that $[u]$ and $[t]$ had the right dimensions, but this }\COL{is left as an exercise.
}\COL{
In this example we can illustrate the power of the non-dimensionalisation through the parameter groupings
\bb
\alpha=\frac{k_0k_2}{k_1^2},\quad u'_0=\frac{k_2u_0}{k_1}.\nonumber
\ee
Specifically, suppose we double each of the kinetic parameters \ie $k_0\mapsto 2k_0$, $k_1\mapsto 2k_1$ and $k_2\mapsto 2k_2$ then neither $\alpha$, nor $u'_0$ changes. This means that under this transformation the solution of \eqn{Non-dim_5} is exactly the same. But, how does this transformation of the original equations? Well, $[u]=k_1/k_2$ does not change, but $[t]=1/(2k_1)$ will be half its previous value. Hence, the solution to this `doubled-parameter' problem (call it $u_2(t)$) will reach the same solution values as the original problem, but in half the time \see{Non_dim_example},
\bb
u_2(t)=u\l\frac{t}{2}\r.\label{Non-dim_6}
\ee}

% we notice that if we double the value of $k_2$ and simultaneously half $k_0$ and $u_0$ then the values of $\alpha$ and $u'_0$ do not change, this means that the solution of \eqn{Non-dim_5} is the same under this transformation and, thus, so must the solutions of \eqn{Non-dim_4}.
\end{example}
\begin{figure}[!!!h!!!tb]
\centering
\includegraphics[width=\ttp]{../Pictures/Non_dim_example.png}
\caption{\label{Non_dim_example} Two simulations of \eqn{Non-dim_4} with parameter values $k_0=k_1=k_2=1$ (blue line, $u(t)$) and $k_0=k_1=k_2=2$ (red line, $u_2(t)$). Illustrating that the evolution of the red line is the same as the blue line, except that the red line evolution occurs twice as fast, as predicted by \eqn{Non-dim_6}.}
\end{figure}

\begin{example}[frametitle=Non-uniqueness]
 To illustrate the non-uniqueness of non-dimensionalisation we rerun example \ref{Arrow method} but this time we balance the time derivative, the constant term and the initial condition,
\bb
  \tikzmark{a}\dot{u}=k_0\tikzmark{b}+k_1u+k_2u^2, \quad u\tikzmark{c}(0)=u_0\tikzmark{d}.\label{Non-dim_7}
\tikz[overlay,remember picture]
{\draw[square arrow1] (a.south) to (b.south);}
\tikz[overlay,remember picture]
{\draw[square arrow1] (c.south) to (d.south);}
\ee
\COL{We quickly find that
\bb
\frac{[u]}{[t]}=k_0,\quad [u]=u_0,\nonumber
\ee
thus, $[t]=u_0/k_0$. We can divide the equation through by $k_0$, because we know that this is the }\COL{balance of the first two terms in \eqn{Non-dim_7}. Thus, we derive
\bb
\dot{u'}=1+\frac{k_1[u]}{k_0}u'+\frac{k_2[u]^2}{k_0}u'^2, \quad u'(0)=1,\nonumber
\ee
which would be rewritten as
\bb
\dot{u}=1+\beta u+\gamma u^2, \quad u(0)=1,\label{Non-dim_8}
\ee
where
\bb
\beta=\frac{k_1u_0}{k_0},\quad \gamma=\frac{k_2u_0^2}{k_0}.\nonumber
\ee}
\end{example}

Both forms of the non-dimensionalised equation, \eqref{Non-dim_5} and  \eqref{Non-dim_8}, are perfectly valid. The most useful form will depend on what factor dominates the equation. If $k_0$ is small and $k_1$ is big (relative to one another) then \eqn{Non-dim_5} would be more useful as $\alpha\approx0$ and we would be able to manipulate the equation to provide more information. Alternatively, if $k_0$ was big and $k_2$, or $k_1$, was small then, \eqn{Non-dim_8} would be more useful as we would, again, be able to remove one of the constants based on this assumption.

\begin{example}[frametitle=Failure]
As mentioned not all balances are valid, which is what we will be seen in this example. Consider the following ODE system
\begin{align}
  \tikzmark{a}\dot{u}=k_0\tikzmark{b}+k_1\tikzmark{c}u-k_2uv, \quad u(0)=u_0,\\
 \nonumber \\
    \tikzmark{e}\dot{v}=k_3\tikzmark{f}+k_4\tikzmark{g}v-k_2uv, \quad v(0)=v_0.
\tikz[overlay,remember picture]
{\draw[square arrow1] (a.south) to (b.south);}
\tikz[overlay,remember picture]
{\draw[square arrow1] (b.south) to (c.south);}
\tikz[overlay,remember picture]
{\draw[square arrow1] (e.south) to (g.south);}
\end{align}
\COL{There are three variables $u$, $v$ and $t$ and so we need three balances. The chosen balances are illustrated on the equations using arrows. Extracting information from the balances we find that
\bb
\frac{[u]}{[t]}=k_0=k_1[u], \quad \frac{[v]}{[t]}=k_4[v].
\ee
From this point we quickly discover that
\bb
[t]=\frac{1}{k_1} \textrm{ and } [t]=\frac{1}{k_4}.
\ee
Since, generally, $k_1\neq k_4$ we cannot satisfy both balances, thus, we must consider a different non-dimensionalisation.}

One possible valid non-dimensionalisation is
\begin{align}
  \tikzmark{a}\dot{u}=k_0\tikzmark{b}+k_1\tikzmark{c}u-k_2uv, \quad u(0)=u_0,\nonumber\\
 \nonumber \\
    \tikzmark{e}\dot{v}=k_3\tikzmark{f}+k_4\tikzmark{g}v-k_2uv, \quad v(0)=v_0.\nonumber
\tikz[overlay,remember picture]
{\draw[square arrow1] (a.south) to (b.south);}
\tikz[overlay,remember picture]
{\draw[square arrow1] (b.south) to (c.south);}
\tikz[overlay,remember picture]
{\draw[square arrow1] (e.south) to (f.south);}
\end{align}
See the board for details.
\end{example}

Although each case of non-dimensionalisation is different, the algorithm you should follow is the same in each case. The steps are:
\begin{enumerate}
\item write down the variables in the equations, this tells you how many balances you need;
\item specify balances and check that they are valid;
\item define non-dimensional scales that allow you to minimise the number of free parameters;
\item substitute the scales into the equations and collect together the remaining parameters into the smallest possible groups and give them a new variable name (DO NOT FORGET to do the same thing for the initial conditions. Everyone always forgets to do the initial conditions);
\item demonstrate that the scales you have derived have the correct dimension;
\item demonstrate that the new parameter groupings are dimensionless.
\end{enumerate}
Although we have not completed the last two points for every example, you will be expected to do every step in an exam.
%\begin{figure}[!!!h!!!tb]
%\centering
%\subfigure[\label{IC_0.1}]{\includegraphics[width=\ttp]{../Pictures/Comparing_pendulums_IC_1.png}}
%\subfigure[\label{IC_1}]{\includegraphics[width=\ttp]{../Pictures/Comparing_pendulums_IC_10.png}}
%\caption{\label{Different_ICs}Comparing \eqns{Spring_eqn}{Bob_eqn} with initial conditions (a) $y=0=\theta$ and (b) $y=1=\theta$. Parameter values $r=g=k=m=1$.}
%\end{figure}
%
%
%\begin{example}[frametitle=Zombies]\label{Zombies}
%Humans, $H$ and zombies, $Z$ interact through the following three interactions \see{Zombie_picture}]:
%\end{example}
%\begin{figure}[!!!h!!!tb]
%\centering
%\includegraphics[width=\tp]{../Pictures/Zombies.png}
%\caption{\label{Zombie_picture} Possible outcomes of human-zombie interactions.}
%\end{figure}

\section{Check list}
By the end of this chapter you should be able to:
\begin{todolist}
\item non-dimensionalise a system of equations using direct substitution, or the arrow method;
\item demonstrate that the derived scales have the correct dimension;
\item demonstrate that remaining parameter groupings are non-dimensional;
\end{todolist}


\section{Length of a vector}

\begin{outcome}
  \begin{enumerate}
  \item Compute the distance between points in $n$-dimensional space.
  \item Compute the length of a vector algebraically and geometrically.
  \item Find vectors that are a given distance from other vectors.
  \item Use algebraic properties of the length operation to prove
    equalities.
  \item Normalize a vector.
  \end{enumerate}
\end{outcome}

In this section, we explore what is meant by the length of a vector in
$\R^n$.  We develop this concept by first looking at the distance
between two points in $\R^n$. Consider two points $P=(p_1,p_2)$ and
$Q=(q_1,q_2)$ in the plane, as in the following picture.
\begin{equation*}
  \begin{tikzpicture}[scale=1]
    \draw[dashed] (0,0) -- (0,2) -- (3,2);
    \draw[fill=blue!10] (0,0) -- (3,0) -- (3,2) -- cycle;
    \draw[thick] (0,0) -- (3,2);
    \draw[fill](0,0) circle [radius=1.8pt] node[left=1ex]{$Q=(q_1,q_2)$};
    \draw[fill](3,2) circle [radius=1.8pt] node[right=1ex]{$P=(p_1,p_2)$};
  \end{tikzpicture}
\end{equation*}
The distance between $P$ and $Q$ is shown in the picture as a solid
line, which is the hypotenuse of a right triangle.  The lengths of the
two other sides of this triangle are $\abs{p_1-q_1}$ and
$\abs{p_2-q_2}$. Therefore, the Pythagorean Theorem implies the
length of the hypotenuse (and thus the distance between $P$ and $Q$)
equals
\begin{equation*}
  d(P,Q)
  =\sqrt{\abs{p_1-q_1}^2+\abs{p_2-q_2}^2}
  =\sqrt{(p_1-q_1)^2+(p_2-q_2)^2}.
\end{equation*}
Now consider two points $P=(p_1,p_2,p_3)$ and
$Q = (q_1,q_2,q_3)$ in $3$-dimensional space.
\begin{equation*}
  \begin{tikzpicture}[scale=1]
    \draw[dashed] (3,0,-2) -- (0,0,-2) -- (0,0,0);
    \draw[dashed] (0,0,0) -- (0,2,0);
    \draw[dashed] (0,0,-2) -- (0,2,-2);
    \draw[fill=blue!20] (0,0,0) -- (3,0,0) -- (3,0,-2) -- cycle;
    \draw[fill=blue!10] (0,0,0) -- (3,0,-2) -- (3,2,-2) -- cycle;
    \draw[dashed] (0,2,0) -- (3,2,0) -- (3,2,-2) -- (0,2,-2) -- cycle;
    \draw[dashed] (3,0,0) -- (3,2,0);
    \draw[thick] (0,0,0) -- (3,2,-2);
    \draw[fill](0,0,0) circle [radius=1.8pt] node[left=1ex]{$Q=(q_1,q_2,q_3)$};
    \draw[fill](3,2,-2) circle [radius=1.8pt] node[right=1ex]{$P=(p_1,p_2,p_3)$};
    \draw[fill](3,0,-2) circle [radius=1.8pt] node[right=1ex]{$R=(p_1,p_2,q_3)$};
    \draw[fill](3,0,0) circle [radius=1.8pt] node[right=1ex]{$S=(p_1,q_2,q_3)$};
  \end{tikzpicture}
\end{equation*}
We will use the Pythagorean Theorem twice to find the length of the
solid line connecting $P$ and $Q$. First, by the Pythagorean Theorem
applied to the right triangle $QSR$, the length of the line joining
$R$ and $Q$ equals
\begin{equation*}
  d(R,Q) = \sqrt{(p_1-q_1)^2+(p_2-q_2)^2}.
\end{equation*}
Second, by the Pythagorean Theorem applied to the triangle $QRP$, the
length of the line joining $P$ and $Q$ equals
\begin{equation*}
  d(P,Q)=\sqrt{d(R,Q)^2+(p_3-q_3)^2}
  =\sqrt{(p_1-q_1)^2+(p_2-q_2)^2+(p_3-q_3)^2}
\end{equation*}
This discussion motivates the following definition for the distance
between points in $\R^n$.

\begin{definition}{Distance between points}{distance-between-points}
  Let $P=(p_1,\ldots,p_n)$ and $Q=(q_1,\ldots,q_n)$ be two points in
  $\R^n$. Then the \textbf{distance}%
  \index{distance!point to point} between these points is defined as
  \begin{equation*}
    d(P, Q) = \sqrt{(p_1-q_1)^2 + \ldots + (p_n-q_n)^2}.
  \end{equation*}
  This formula is also called the \textbf{distance formula}%
  \index{distance formula}. We may also write $\abs{PQ}$ for the
  distance between $P$ and $Q$.
\end{definition}

In the following example, we use
Definition~\ref{def:distance-between-points} to find the distance
between two points in $\R^4$.

\begin{example}{Distance between points}{distance-between-points}
  Find the distance between the points $P=(1,2,-4,6)$ and
  $Q=(2,3,-1,0)$ in $\R^4$.
\end{example}

\begin{solution}
  Using the distance formula, we have
  \begin{equation*}
    d(P,Q)
    = \sqrt{(1-2)^2+(2-3)^2+(-4-(-1))^2+(6-0)^2}
    = \sqrt{1^2+1^2+3^2+6^2}
    = \sqrt{47}.
  \end{equation*}
\end{solution}

\begin{example}{The plane between two points}{plane-between-two-points}
  Describe the points in $\R^3$ that are equally distant from the two
  points $Q=(1,2,3)$ and $R=(0,1,2)$.
\end{example}

\begin{solution}
  Let $P = (p_1,p_2,p_3)$ be such a point. Then $P$ is the same
  distance from $Q$ and $R$, thus $d(P,Q)=d(P,R)$. By the distance
  formula, we have
  \begin{equation*}
    \sqrt{(p_1-1)^2+(p_2-2)^2+(p_3-3)^2}=
    \sqrt{(p_1-0)^2+(p_2-1)^2+(p_3-2)^2}.
  \end{equation*}
  Squaring both sides, we obtain
  \begin{equation*}
    (p_1 -1)^2+(p_2 -2)^2+(p_3 -3)^2
    = p_1^2+(p_2-1)^2+(p_3 -2)^2,
  \end{equation*}
  and so
  \begin{equation*}
    \allowbreak (p_1^2-2p_1+1)+(p_2^2-4p_2+4)+(p_3^2-6p_3+9)=p_1^2+(p_2^2-2p_2+1)+(p_3^2-4p_3+4).
  \end{equation*}
  Simplifying, this becomes
  \begin{equation*}
    -2p_1-4p_2-6p_3+14=-2p_2-4p_3+5,
  \end{equation*}
  which can finally be written as
  \begin{equation}\label{distance-plane}
    2p_1+2p_2+2p_3=9.
  \end{equation}
  Therefore, the points $P = (p_1,p_2,p_3)$ that are the same
  distance from $Q$ and $R$ form a plane whose equation is given by
  \eqref{distance-plane}.
\end{solution}

We can now use our understanding of the distance between two points to
define what is meant by the length of a vector.

\begin{definition}{Length of a vector}{length-of-vector}
  Let
  \begin{equation*}
    \vect{u} = \begin{mymatrix}{c}u_1\\\vdots\\u_n\end{mymatrix}
  \end{equation*}
  be a vector in $\R^n$. Then the \textbf{length}%
  \index{vector!length}%
  \index{length of a vector} of $\vect{u}$, written $\norm{\vect{u}}$,
  is given by
  \begin{equation*}
    \norm{\vect{u}} = \sqrt{u_1^2 + \ldots + u_n^2}.
  \end{equation*}
  The length of a vector is also sometimes called its
  \textbf{magnitude}%
  \index{magnitude!of a vector}%
  \index{vector!magnitude} or its
  \textbf{norm}%
  \index{norm!in Rn@in $\R^n$}%
  \index{vector!norm}.
\end{definition}

This definition corresponds to
Definition~\ref{def:distance-between-points}, if we consider the
vector $\vect{u}$ to have its tail at the point
$0 = (0,\ldots,0)$ and its tip at the point
$U = (u_1,\ldots, u_n)$.  Then the length of $\vect{u}$ is equal
to the distance between $0$ and $U$. In general,
$\norm{\longvect{PQ}} = d(P,Q)$.

Reconsider Example~\ref{exa:distance-between-points}. We could have also
computed the distance between $P$ and $Q$ as the length of the vector
connecting them. This vector is $\longvect{PQ} = \mat{1,1,3,-6}^T$,
and its length is
\begin{equation*}
  \norm{\longvect{PQ}} = \sqrt{1^2+1^2+3^2+6^2} = \sqrt{47}.
\end{equation*}

The following proposition states a few important properties of the length
of vectors.

\begin{proposition}{Properties of length}{properties-length}
  The following hold for all vectors $\vect{u},\vect{v}$ and scalars $k$.
  \begin{itemize}
  \item $\norm{\vect{u}}\geq 0$
  \item $\norm{\vect{u}}=0$ if and only if $\vect{u}=\vect{0}$.
  \item $\norm{k\vect{u}} = |k|\,\norm{\vect{u}}$.
  \end{itemize}
\end{proposition}

We conclude this section by giving a special name to vectors of length
$1$.

\begin{definition}{Unit vector}{unit-vector}
  A vector\/ $\vect{u}\in\R^n$ is called a \textbf{unit vector}%
  \index{unit vector}%
  \index{vector!unit vector} if it has length $1$, that is, if
  \begin{equation*}
    \norm{\vect{u}} = 1.
  \end{equation*}
\end{definition}

Let $\vect{v}$ be a non-zero vector in $\R^n$. Then there is a unit
vector $\vect{u}$ that points in the same direction as $\vect{v}$, but
has length 1%
\index{vector!corresponding unit vector}. This vector is given by
\begin{equation*}
  \vect{u} = \frac{1}{\norm{\vect{v}}} \vect{v}.
\end{equation*}
We often use the term \textbf{normalize}%
\index{vector!normalizing a vector} to refer to this process. When we
normalize a vector, we find the corresponding unit vector.

\begin{example}{Normalizing a vector}{unit-vector}
  Consider the vector $\vect{v} = \mat{1, -3, 4}^T$. Find the unit
  vector $\vect{u}$ that has the same direction as $\vect{v}$
\end{example}

\begin{solution}
  We have $\norm{\vect{v}} = \sqrt{1^2 + (-3)^2 + 4^2} =
  \sqrt{26}$, and therefore
  \begin{equation*}
    \vect{u}
    = \frac{1}{\norm{\vect{v}}} \vect{v}
    = \frac{1}{\sqrt{26}} \mat{1, -3, 4}^T
    = \mat{\frac{1}{\sqrt{26}}, -\frac{3}{\sqrt{26}}, \frac{4}{\sqrt{26}}}^T.
  \end{equation*}
\end{solution}


\subsection{Within and between transformation}

\subsubsection{Introduction}

We can group the data in two ways, one gets between differences and the other within differences.

In the above example, we could find the effects of schools, or of departments.

\(y_{ij}=\alpha + X_{ij}\theta +\epsilon_{ij}\)

\((y_{ij}-\bar y_{j})=(\alpha -\bar \alpha )+(X_{ij}-\bar X_{j})\theta  +(\epsilon_{ij}-\bar \epsilon_{j})\)

\((y_{ij}-\bar y_{j})=(X_{ij}-\bar X_{j})\theta  +(\epsilon_{ij}-\bar \epsilon_{j})\)

Or alternatively:

\((y_{ij}-\bar y_{i})=(X_{ij}-\bar X_{i})\theta  +(\epsilon_{ij}-\bar \epsilon_{i})\)

Regardless of the form we choose, we can write this as:

\(\ddot y_{ij}=\ddot X_{ij}\theta +\ddot \epsilon_{ij}\)


%================================
%::::::::::::::::::::::::::::::::
\chapter{Some Examples of Topological Spaces}
\label{ch: some examples of topological spaces}
%::::::::::::::::::::::::::::::::
%================================

%================================
\section{Sorgenfrey line}
\label{sec: sorgenfrey line}
%================================

\begin{enumerate}
	\item \href{https://proofwiki.org/wiki/Definition:Sorgenfrey_Line}{Definition}.
	\item \href{https://proofwiki.org/wiki/Sorgenfrey_Line_is_Lindel%C3%B6f}{Sorgenfrey line is Lindel\"of}.
	\item \href{https://proofwiki.org/wiki/Sorgenfrey_Line_is_Separable}{Sorgenfrey line is separable}.
	\item \href{https://proofwiki.org/wiki/Sorgenfrey_Line_is_not_Second-Countable}{Sorgenfrey line is not second-countable}.
\end{enumerate}

%================================
\section{Niemytzki Plane}
\label{sec: niemytzki plane}
%================================


\chapter{Reachability of the Tree's Root Node}

\section{Determining the maximum alpha}
In the previous chapter we have determined how many divisions by two lead to a cycle in the Collatz tree. We will now study the case in which a Collatz sequence reaches the root node $v_{n+1}=1$. Our proof builds on theorem~\ref{theo:1}. As in the last chapter we replace $(1+\frac{1}{3v_i})$ with the variable $\beta_i$:

\[
v_{n+1}=3^nv_1\prod_{i=1}^{n}\beta_i\prod_{i=1}^{n}2^{-\alpha_i}
\]

Setting $v_{n+1}=1$ leads to:

\begin{equation}
\label{eq:reach_1}
\begin{array}{l}
1=3^nv_1\prod_{i=1}^{n}\beta_i\prod_{i=1}^{n}2^{-\alpha_i}
\\[\medskipamount]
\prod_{i=1}^{n}2^{\alpha_i}=3^nv_1\prod_{i=1}^{n}\beta_i
\end{array}
\end{equation}

\par\medskip
Equation~\ref{eq:reach_1} defines the maximum possible value of $\alpha$ for a given Collatz sequence. When a Collatz sequence reaches this alpha value, it finishes at the root node. The number of divisions by two required for this is referred to as $\hat\alpha$ subsequently:

\begin{equation*}
\begin{array}{l}
2^{\hat\alpha}=3^nv_1\prod_{i=1}^{n}\beta_i
\\[\medskipamount]
\hat\alpha=nlog_23+log_2v_1+log_2\prod_{i=1}^{n}\beta_i
\end{array}
\end{equation*}

\par\medskip
In the previous chapter we proved $1<\prod_{i=1}^{n}\beta_i<2$. We use this knowledge to further restrict $\hat\alpha$ in theorem~\ref{theo:3}.

\bigskip
\begin{theorem}
\label{theo:3}
The maximum possible number of divisions by two in a Collatz sequence can be calculated as follows:
\[
\hat\alpha=\lfloor n\cdot log_23+log_2v_1\rfloor+1
\]
If a Collatz sequence reaches $\hat\alpha$, it ends with the result $v_{n+1}=1$.
\end{theorem}

\par\medskip
Since $\hat\alpha$ is a whole number, we truncate the fractional part. Knowing that $1<\prod_{i=1}^{n}\beta_i<2$ we add one to the result.

\bigskip
\begin{example}
Setting $v_{n+1}=13$ and $n=2$ leads to:
\[
v_{2+1}=3^2\cdot13\cdot\left(1+\frac{1}{3\cdot13}\right)\cdot\left(1+\frac{1}{3\cdot5}\right)\cdot2^{\lfloor2\cdot\log_23+log_213\rfloor+1}
\]
\end{example}

Building on $\hat\alpha$ we define the following restrictions on the alpha of a Collatz sequence:

\begin{equation}
\label{eq:reach_3}
n\le\alpha\le\hat\alpha
\end{equation}

Condition~\ref{eq:reach_3} is not only valid for $k=3$, but for all $k$. Similar to $\bar\alpha$, the variable $\hat\alpha$ could form the basis for a proof of the Collatz conjecture. As $\bar\alpha$ teaches us about cycles in the Collatz tree, $\hat\alpha$ leads us the way to its root node. If one shows that each Collatz sequence finally reaches $\hat\alpha$, the problem is solved as a whole. This is not in the scope of the present paper. However, it could become the foundation for a future work.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS630: Database Management Systems
% Copyright 2014 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 2}

Let $a$ and $b$ be integer-valued attributes that may be \texttt{NULL} in some tuples.
For each of the following conditions that may appear in a \texttt{WHERE} clause, describe exactly the set of $(a,b)$ tuples that satisfy the condition, including the case where $a$ and/or $b$ is \texttt{NULL}.

\begin{enumerate}[label=(\alph*)]
\item $a=10$ or $b=20$
\item $a=10$ and $b=20$
\item $a<10$ or $a>=10$
\item $a=b$
\end{enumerate}

\textbf{Solution:}

For ease of representation, sets $I$ and $J$ are defined respectively as set of integers and nullable integers; the latter being set of all possible integer values and the \texttt{NULL} value, $\lambda$.
As well, set of all binary relations that satisfy specified condition is defined as $S$.

\begin{enumerate}[label=(\alph*)]
\item
\begin{equation}\nonumber
S = \{ (a,b) | a = 10, b\in J \} \cup \{ (a,b) | b = 20, a\in J \}
\end{equation}
\item
\begin{equation}\nonumber
S = \{ (a,b) | a = 10, b = 20 \}
\end{equation}
\item
\begin{equation}\nonumber
S = \{ (a,b) | a \in I, b \in J\}= J - \{(a,b)|a = \lambda , b \in J\}
\end{equation}
\item
\begin{equation}\nonumber
S = \{ (a,b) | a = b, a \in I, b \in I\}
\end{equation}

\end{enumerate}


\subsection{Linear and affine functions}

\subsubsection{Linear maps}

Linear maps can be written as:

\(v=Mu\)

These go through the origin. That is, if \(u=0\) then \(v=0\).

\subsubsection{Affine function}

Affine functions are more general than linear maps. They can be written as:

\(v=Mu+c\)

Where \(c\) is a vector in the same space as \(v\).

Affine functions where \(c\ne 0\) are not linear maps. They are not homomorphisms which preserve the structure of the vector space.

If we multiply \(u\) by a scalar \(s\), then \(v\) will not increase by the same proportion.


Note that (a), (c) and (d) are equivalent by the definition of a tree and
\cite{notes}, Theorem 21. We must prove these are additionally equivalent
to (b) and (d).

\paragraph{$(a)\Rightarrow(b)$} Suppose $G$ is a tree. Then it is, by
definition, connected, so every two vertices are joined by at least one
path. Suppose now that there are two vertices $x$ and $y$ that are joined
by two distinct paths $P_1$ and $P_2$. Consider $p$, the vertex before the
first vertex that differs between $P_1$ and $P_2$, and let $q$ be the first
vertex after $p$ in $P_1$ which also occurs after $p$ in $P_2$. Let
$P'_1$ be the path from $p$ to $q$ along $P_1$, and $P'_2$ be the path from
$p$ to $q$ along $P_2$. It is clear that $P'_1$ concatenated with the
reverse of $P'_2$ is a closed walk starting and ending at $p$. To show it is
a cycle, it is sufficient to show that no vertex internal to $P'_1$ appears
in $P'_2$. However, if such a vertex $q'$ were to exist, it would be a
vertex after $p$ on $P_1$, which also occurs after $p$ on $P_2$, and
moreover before $q$ on $P_1$, contradicting the definition of $q$ as the
first such vertex. Therefore the closed walk described is a cycle, which
contradicts that two vertices in $G$ are joined by distinct paths. $(b)$
follows.

\paragraph{$(b)\Rightarrow(e)$}
Suppose $G$ contains a cycle, and let two vertices on this cycle be $x$
and $y$. Then by following the cycle from $x$ to $y$ in each direction,
two internally disjoint paths from $x$ to $y$ can be found, contradicting
that there is a unique $x-y$ path in $G$. So $G$ is acyclic.
Now, since $G$ is acyclic, any
cycle in $G+uv$ contains the edge $uv$. But clearly there is at least one
such cycle, that obtained by following the unique path from $u$ to $v$ in
$G$, then using the edge $uv$ (since $u$ and $v$ are nonadjacent, this edge
is not in $G$, so hasn't been used earlier in the cycle). Now suppose $C$ is
a cycle in $G+uv$. It must also contain $uv$, hence $u$ and $v$. Now
consider the path obtained by following $C$ from $u$ to $v$, not using the
edge $uv$. This is a path connecting $u$ and $v$ in $G$, hence it must be
the unique path connecting $u$ and $v$ in $G$, and hence $C$ is the cycle
obtained from this path as described earlier. So the cycle is unique.

\paragraph{$(e)\Rightarrow(a)$} $G$ is already acyclic, so it suffices to
prove it connected. Let $x$ and $y$ be two vertices in $G$. If they are
adjacent, there is clearly a path between them. If they are not, then
$G+xy$ contains a cycle $C$. Since $G$ is acyclic, $C$ includes
$xy$ and hence $x$ and $y$. Now, following $C$ from $x$ to $y$, not using
the edge $xy$ gives a path from $x$ to $y$ in $G$. So any pair
of vertices in $G$ is connected, so $G$ is. Hence $G$ is a tree.


\section{The ElGamal cryptosystem}
The ElGamal cryptosystem was proposed by Taher ElGamal in 1985 as an extension to the Diffie-Hellman protocol which can be applied in other cyclic groups, such as \textbf{Galois fields}.

It is a public-key encryption algorithm, based on the intractability of the discrete logarithm, and considered over the group $\mathbb{Z}^*_p$ with $p$ prime.

Its key aspect is the \textbf{randomized encryption}, adding a layer of security, and its principal applications are establishing a secure channel for key sharing and message encryption.

The ElGamal algorithm was proposed to remedy one flaw of the Diffie-Hellman key exchange: it requires interaction of both parties to calculate a common private key. This can constitute in a problem in case they are not able to interact, due to delays in transmission or unavailability of receiver.

The random exponent is therefore introduced to replace the private exponent of the receiving entity, so that it does not have to take part in the exchange.

\subsection{From Diffie-Hellman to ElGamal}
ElGamal follows immediately from Diffie-Hellman: to send an encrypted message $x$, Alice must perform a key exchange to derive a shared key $k_M$. However, as previously stated, only the receiver needs to create a key in advance and publish it.

A large prime $p$ and a primitive element $\alpha$ need to be generated, to whom ElGamal adds a random multiplicative mask to encrypt:
$$y \equiv x \cdot k_M \mod p$$

\begin{figure}[h]
	\includegraphics[scale=0.35]{ElGamalPrinciplePaar.png}
	\centering
\end{figure}

This protocol is similar to Diffie-Hellman: the private and public keys of Bob are computed the same way, and do not change over time, however Alice has to generate a new pair for the encryption of every message.

The key $k_E$ is \textbf{ephemeral}, meaning that it only exists temporarily, and $k_M$ is the joint key for masking the plain text. To encrypt a message, Alice multiplies the plaintext message $x$ by $k_M$ and Bob reverses this operation using the inverse mask.

\subsection{Protocol}
ElGamal encryption method works thanks to an important property of cyclic groups: given $k_M \in \mathbb{Z}^*_p$, every message $x$ maps to another cipher text computing $k_M \cdot x$, and random values allow to obtain the \textit{same probability} for every cipher text.

\begin{figure}[h]
	\includegraphics[scale=0.35]{ElGamalProtocolPaar.png}
	\centering
\end{figure}

The actual protocol is composed by three phases: \textbf{setup}, \textbf{encryption} and \textbf{decryption}. It starts by fixing a very large finite field $\mathbb{Z}^*_p$ and an element $\alpha \in \mathbb{Z}^*_p$ (preferably, but not necessarily a generator).

Supposing an use of plain text message units with numerical equivalents in $\mathbb{Z}^*_p$, each user $A$ randomly chooses an integer $i$ in the range $0 < i < p - 1$. This integer $i$ is the secret deciphering key, while the public enciphering key is the element $k_E = \alpha^i \in \mathbb{Z}^*_p$.

To send a message $y$ to the user $A$, an integer $d$ is chosen at random, and $A$ is sent the pair of elements:
$$(\alpha^d, y\alpha^{id})$$
$\alpha^{id} = k_E^d$ can be computing without knowing $i$ simply by raising $\alpha$ by the $d$-th power. Now Alice, who is aware of $i$, can recover $y$ from this pair simply by raising the first element $\alpha^d$ to the $i$-th power and dividing the result by the second element.

Public keys can be obtained in any way, for instance from key servers or unencrypted means. There is no security issues involved in this transmission, as the only secret is the exponent and therefore computationally infeasible.

In other words, the message is being masked but also contains a way to be recovered only by someone knowing the initial value $i$.

The Diffie-Hellman sequence of operations is rearranged, since Bob receives only one message from Alice containing the ephemeral key and the encrypted text. In this case, however, the couple $(k_E, y)$ is generally \textbf{twice as long} as the message, since parameters have a bit length of $\lceil\log_2p\rceil$.

\subsubsection{Example}
Alice wants to send the message $x = 26$.

Bob generates $p = 29$ and $\alpha = 2$, then chooses $k_{pr, B} = d = 12$ to compute $\beta = \alpha^d \equiv 7 \mod 29$.

Alice receives the triple $k_{pub, B} = (p, \alpha, \beta) = (29, 2, 7)$, chooses $i = 5$ and then uses those to obtain the two temporary keys:
\begin{enumerate}
	\item $k_E = \alpha^i \equiv 3 \mod 29$
	\item $k_M = \beta^i \equiv 16 \mod 29$
\end{enumerate}
Alice encrypts $y = x \cdot k_M \equiv 10 \mod 29$, and sends it to Bob with the ephemeral key $k_E$.

Bob computes $k_M = k_E^d \equiv 16 \mod 29$, which is the same as Alice, and uses it to decrypt the message:
$$x = y \cdot k^{-1}_M \equiv 10 \cdot 20 \equiv 26 \mod 29$$

\subsection{Proof}
Assuming the value $d_{k_{pr}}(k_E, y)$ actually returns the original message $x$:
\begin{equation}
\begin{split}
d_{k_{pr}}(k_E, y) &\equiv y \cdot (k_M)^{-1} \mod p \\
&\equiv [x \cdot k_M] \cdot (k_E^d)^{-1} \mod p \\
&\equiv [x \cdot (\alpha^d)^i]^{-1} \mod p \\
&\equiv x \cdot \alpha^{d \cdot i - d \cdot i} \mod p \\
&\equiv x \mod p
\end{split}
\end{equation}

\subsection{Practical applications}
ElGamal encryption is not widely used in practice, since one of the best known practices to break it exploits its malleability: the ciphertext $(k_E, y)$ can be replaced with $(k_e, sy)$ for some integer $s$.

The receiver therefore then computes:
\begin{equation}
\begin{split}
d_{k_{pr}}(k_E, sy) &\equiv sy \cdot (k_M)^{-1} \mod p \\
&\equiv s \cdot (x \cdot k_M) \cdot k_M^{-1} \mod p \\
&\equiv sx \mod p
\end{split}
\end{equation}

The decrypted text is also a multiple of $s$, and although an attacker would not be able to decrypt the message, he is able to manipulate it for instance doubling or tripling the value of the result and compromising communication.

The ElGamal encryption system is used in GNU Privacy Guard System and recent versions of PGP.

GnuPG implementation, however, until late 2003 had an insecure algorithm whose private exponent was too short and thus easy to break. All signatures created before then are considered compromised.

\subsubsection{Computational aspects}
As previously stated, the ciphertext is twice as long as the message: therefore, the message expansion factor of ElGamal is two.

Furthermore, it is a \textbf{probabilistic encryption scheme}: encrypting two identical messages $x_1, x_2 \in \mathbb{Z}^*_p$ using the same public key results with \textit{extremely high likelihood} in two different ciphertexts $y_1 \neq y_2$.

This happens since $i$ is chosen at random from $\{2, 3, \dots, p - 2\}$ every time a message is encrypted, causing the session key $k_M = \beta^i$ to also be random. This is an effective method to prevent brute-force attacks.

Key generation involves finding a prime number $p$, which needs to have a length of at least 1024 bits and can be found through one of the prime-finding algorithms.

The public key can easily be computed using the square-and-multiply algorithm, since it requires exponentiation operations.

This is also applied to the encryption procedure: it involves two modular exponentiations and one modular multiplication, all operands having a bit length of $\lceil\log_2p\rceil$. Since exponentiations are independent from the plain text, they can be calculated in advance and stored to be retrieved when actual encryption is needed, to reduce the total time.

Decryption is performed again through the exponentiation $k_M = k^d \mod p$, using square-and-multiply, followed by an inversion performed with extended Euclidean algorithm.

\textbf{Fermat's Little Theorem }allows to combine steps, using the following property:
$$k_E^{p-1} \equiv 1 \mod p \qquad \forall\ k_E \in \mathbb{Z}^*_p$$

Decryption can be performed as:
\begin{equation}
\begin{split}
k_M^{-1} &\equiv (k_E^d){-1} \mod p \\
&\equiv (k_E^d){-1}\cdot k_E^{p-1} \mod p \\
&\equiv k_E^{p-d-1} \mod p
\end{split}
\end{equation}
This equivalence relation allows to compute the inverse of the masking key using only a single exponentiation with $p - d - 1$, which is essentially one execution of square-and-multiply.

After that, one modular multiplication is required to recover $x \equiv y \cdot k_M^{-1} \mod p$.

\subsubsection{Security}
To analyse the security of ElGamal, firstly an assumption must be introduced, consisting in the \textbf{Decisional Diffie-Hellman}. This assumption states that, considering a group $G$ and a generator $a$, given $x, y \in \mathbb{Z}^*_q$, the value $a^{xy}$ looks like a generic random element.

This can also be stated imposing that the probability distributions $(a^x, a^y, a^{xy})$ and $(a^x, a^y, a^z)$, with $x, y, z \in \mathbb{Z}^*_q$ randomly and independently chosen, are computationally indistinguishable.

Decisional Diffie-Hellman is a stronger assumption than just discrete logarithm, since it forbids computing discrete logarithms of $a^x$ and $a^y$ separately to then combine them and cracking $a^{xy}$.

The security of ElGamal can be further assessed distinguishing between two kinds of attacks:
\begin{itemize}
	\item Passive, which are listen-only;
	\item Active, which allow an attacker to generate and alter messages.
\end{itemize}

Passive attacks consist in retrieving the message $x$ from the information $p, \alpha, \beta = \alpha^d, k_E = \alpha^i$ and $y = \beta^i$ obtained by eavesdropping the channel.

Currently, there is no method to solve Diffie-Hellman protocol other than computing discrete logarithms, therefore the most efficient way to avoid that is choosing strong keys. In fact, the only ways an attacker can break the ElGamal scheme are:
\begin{enumerate}
	\item Finding $d$, i.e.\ computing $d = \log_\alpha \beta \mod p$, which is computationally infeasible;
	\item Trying to guess the random exponent $i = log_\alpha k \mod p$, which still involves solving the discrete logarithm problem.
\end{enumerate}
The index-calculus algorithm can be used in both cases, thus in order to guarantee security of ElGamal, $p$ should be at least of 1024 bits and $\alpha$ has to be a primitive element.

Active attacks compromise the authenticity of public keys, having them belong to an untrusted third party. This can be prevented using certificates or digital signatures.

Another pitfall is that the random exponent $i$ \textbf{should not be reused}: assuming Alice uses the same value for the encryption of two subsequent messages $x_1$ and $x_2$, then the two masking keys would be the same, namely $k_M = \beta^i$, along with the ephemeral keys.

Two identical cyphertexts $(y_1, k_E)$ are therefore sent through the channel. If an attacker can guess the first message, the second one can be computed as well using the masking key $k_M \equiv y_1x_1^{-1} \mod p$ since $x_2 \equiv y_2k_M^{-1} \mod p$.

This consequently works for every text encrypted with the same $i$. Furthermore, an attacker knows that $i$ is being reused, since it leads to the same ephemeral key. As a consequence, the secret exponent must not be repeated, choosing different seeds for picking random numbers.

Seminar - Algebraic Methods and Algorithms in Cryptology  /chapters/elgamal.tex


\section{Network Flow Problems}
\subsection{The Maximal Flow Problem}
\begin{definition}
    A network (or directed graph) is a set $N=\{1,\ldots,n\}$ of nodes (or vertices) together with a collection $A\subset N\times N$ of ordered pairs $(i,j)$ called arcs or directed edges.
    The flow in the network is a collection of numbers $(x_{i,j})$ indexed by elements in $N\times N$.
\end{definition}
The problem we wish to consider is formulated in the following way:
Each arc $(i,j)$ is assigned a capacity $c_{i,j}\ge 0$.
In reality (or for notational convenience), we will pretend all arcs in $N\times N$ exists and $c_{i,j}=0$ for $(i,j)\notin A$.\\
We distinguish two special nodes in the network.
Node $1$ is called the source node while the node $n$ is called the sink.
The maximum flow problem is to pump as much flow through the network, from the source to the sink, subject to the capacity constraints and the constraint that the flow-in and flow-out cancelled out in any notes except the source and the sink.
\footnote{Normally, there is no flow-in at the source and no flow-out at the sink.}
Formally, we want to maximise $v\in\mathbb R$ subject to $0\le x_{i,j}\le c_{i,j}$ and
$$\sum_{j\in N}x_{i,j}-\sum_{j\in N}x_{j,i}=\begin{cases}
    v\text{, if $i=1$}\\
    0\text{, if $1<i<n$}\\
    -v\text{, if $i=n$}
\end{cases}$$
which, as one can discover joyfully, is a linear program.
As an aside, the variable $v$ does not have to appear as it can be expressed in terms of $(x_{i,j})$ in the constraint.
It is simply for notational convenience.\\
Of course, we can apply the barrier method or simplex algorithm and call it a day, but certainly there are more useful structures in this system that allows us to simplify it.
\subsection{The Ford-Fulkerson Algorithm}
A nice algorithm developed by Ford-Fulkerson is as follows:\\
First, assign an initial feasible flow $(x_{i,j})$ with value $v$ (for example we can take all of them to be zero).
Then find an augmented path, defined a a collection of nodes $1=i_1,i_2,\ldots,i_{r-1},i_r=n$ with eiher $c_{i_k,i_{k+1}}-x_{i_k,i_{k+1}}>0$ or $x_{i_{k+1},i_k}>0$ for each $k$.
If an augmented path exists, then increase the flow along the path by the amount
$$\delta=\min_{k\in\{1,\ldots,r-1\}}\max\{c_{i_k,i_{k+1}}-x_{i_k,i_{k+1}},x_{i_{k+1},i_k}\}$$
which is positive by assumption and
$$x_{i,j}'=\begin{cases}
    x_{i,j}+\delta\text{, if $i,j$ are consecutive in the path}\\
    x_{i,j}-\delta\text{, if $j,i$ are consecutive in the path}\\
    x_{i,j}\text{, otherwise}
\end{cases}$$
Then, as one can check, $(x_{i,j}')$ is also feasible with $v'=v+\delta$.
We shall show that the non-existence of an augmented path is equivalent to the maximality of the flow.
\begin{definition}
    Let $S\subset N$ be a collection of nodes.
    The complement of $S$ in $N$ is denoted $\bar{S}=N\setminus S$.
    A cut seperating the source and the sink is the set of arcs $(i,j)\in S\times \bar{S}$ where $\{1,n\}\in S$.
    THe capacity of a cut $(S,\bar{S})$ is defined as
    $$c(S,\bar{S})=\sum_{i\in S,j\in\bar{S}}c_{i,j}$$
\end{definition}
\begin{theorem}[The max-flow min-cut Theorem]\label{max-flow-min-cut}
    For a network, the maximum value of a feasible flow from the source to the sink equals to the minimum of all cut capacities of all cuts seperating the source and the sink.
\end{theorem}
\begin{proof}
    Fix any feasible flow $(x_{i,j})$.
    For any $S,T\subset N$ we write
    $$f(S,T)=\sum_{i\in S,j\in T}x_{i,j}$$
    be the total flow over arcs joining $S$ and $T$.
    Now let $S,\bar{S}$ be a cut seperating the source and the sink, the constraint tells us
    \begin{align*}
        v&=\sum_{i\in S,j\in N}x_{i,j}-\sum_{j\in N,i\in S}x_{ji}\\
        &=f(S,N)-f(N,S)\\
        &=(f(S,S)+f(S,\bar{S}))-(f(S,S)+f(\bar{S},S))\\
        &=f(S,\bar{S})-f(\bar{S},S)\\
        &\le f(S,\bar{S})\\
        &\le c(S,\bar{S})
    \end{align*}
    Hence the value of any flow is less than or equal to the capacity of any cut.
    To show that they are actual equal, notice that the inequality above becomes an equality iff $x_{i,j}=c_{i,j}$ and $x_{j,i}=0$ for all arcs $(i,j)$ in $(S,\bar{S})$.
    SO let $(x_{i,j}^\star)$ be a maximal flow (which exists by compactness).
    We define a set $S$ by induction in the following criteria:\\
    1. The source is in $S$.\\
    2. If $i\in S$ and $x_{i,j}^\star<c_{i,j}$, then $j$ in $S$.\\
    3. If $i\in S$ and $x_{j,i}^\star>0$, then $j\in S$.\\
    Now the sink is in $\bar S$ since otherwise there would be a path (by the inductive process) from the source to the sink pumping strictly more flow, contradicting the maximality of $(x_{i,j}^\star)$.
    The cut $(S,\bar{S})$ has the property we wanted.
\end{proof}
The same constructive idea shows that if no augmented path can be found, then the flow is maximal as one can construct such a cut which has capacity equal to the value of the flow.\\
Also, the algorithm is guaranteed to terminate if the capacities and elements of the initial flow are all integer (hence it also terminates if all are rational by multiplying an integer).
This is because the $\delta$ we obtain from the algorithm is at least $1$, so if the value of $v$ increases indefinitely through the algorithm, it cannot be bounded, which contradicts the compactness of the domain of the problem.
But if something is irrational, the algorithm may not terminate.
\subsection{The Dual Problem}
The Lagrangian of the maximal flow problem is (note that $v$ can be expressed in terms of $(x_{i,j})$):
\begin{align*}
    L&=v+\sum_i\lambda_i\left( \sum_jx_{i,j}-\sum_jx_{j,i} \right)-\lambda_1v+\lambda_nv+\sum_{i,j}\mu_{i,j}(c_{i,j}-x_{i,j}-z_{i,j})\\
    &=\sum_{i,j}c_{i,j}\mu_{i,j}+v(1-\lambda_1+\lambda_n)+\sum_{i,j}x_{i,j}(\lambda_i-\lambda_j-\mu_{i,j})-\sum_{i,j}\mu_{i,j}z_{i,j}
\end{align*}
where $z$ is the slack variable.
For $L$ to be bounded above varying $x,z$, we have
$$\begin{cases}
    1-\lambda_1+\lambda_n=0\\
    \lambda_i-\lambda_j\le\mu_{i,j}\\
    \mu_{i,j}\ge 0
\end{cases}$$
So the dual problem is to minimise $\sum_{i,j}c_{i,j}\mu_{i,j}$ subject to $\lambda_1-\lambda_n=1,\lambda_i-\lambda_j\le\mu_{i,j},\mu_{i,j}\ge 0$.
By complementary slackness, if $x$ is optimal for the primal problem and $(\lambda,\mu)$ is optimal for the dual problem, then $\lambda_i-\lambda_j=\mu_{i,j}$ and $\mu_{i,j}=0$.\\
By Theorem \ref{max-flow-min-cut}, there is a set $S$ with $1\in S,n\notin S$ such that the optimal dual variables are given by
$$\lambda_i=\begin{cases}
    1\text{, if $i\in S$}\\
    0\text{, if $i\in\bar{S}$}
\end{cases},\mu_{i,j}=\begin{cases}
    1\text{, if $i\in S,j\in\bar{S}$}\\
    0\text{, otherwise}
\end{cases}$$
due to our previously discussed dual constraints.
\subsection{Combinatorial Applications}
There are certain combinatorial applications of Theorem \ref{max-flow-min-cut}.
One combinatorial problems is formulated like this:
Suppose that a standard deck of $52$ playing cards dealt into $13$ piles of $4$ cards each, we want to show that it is possible to select exactly one card from each pile such that the selected cards contain one card of each of the $13$ ranks A,2,\dots,10,J,Q,K.\\
To prove this, we build a network with nodes
$$A,a_1,\ldots,a_{13},b_1,\ldots,b_{52},c_1,\ldots,c_{13},B$$
as follows:
The nodes $(a_i)$ represent the $13$ piles, and the nodes $(b_j)$ represent the $52$ cards, and the node $(c_k)$ represent the $13$ ranks.
The node $A$ is the source and $B$ is the sink.\\
We put an arc of capacity $1$ from $A$ to each of $a_i$, and an arc of infinite (or very large) capacity from $a_i$ to $b_j$ if card $j$ is in stack $i$.
There will be exactly four arcs leaving each $a_i$ and exactly one arc entering each $b_j$.\\
We also put an arc of infinite capacity from $b_j$ to $c_k$ if card $j$ has rank $k$, so again there will be exactly one arc leaving each of the nodes $b_j$ and exactly four arcs entering each of the nodes $c_k$.\\
Lastly, put an arc of capacity $1$ from each node $c_k$ to the sink $B$.
\begin{claim}
    The minimum capacity of any cut is $13$.
\end{claim}
\begin{proof}
    The cut $\{(A,a_i):i=1,\ldots,13\}$ has capacity $13$.
    We shall show that any other cut has capacity at least $13$.
    Consider a cut with finite capacity, then it does not contain any arc of the form $(a_i,b_j)$ or $(b_j,c_k)$.
    If there are $r$ arcs of the form $(c_k,B)$ for some $k$, then there are at least $s=13-r$ nodes $c_k$ still connected to $B$.
    Each of these $s$ many $c_k$'s is connected to four of the nodes $b_j$ exclusively, so $4s$ many $b_j$ is connected to $B$.
    But then at least $(4s)/4=s$ many nodes $a_i$ are connected to those $b_j$, hence to $B$, and they are also connected to $A$.
    So there are at least $s$ arcs of the form $(A,a_i)$ to be removed in order to complete the cut.
    This produces a total capacity of at least $r+s=13$.
\end{proof}
So by Theorem \ref{max-flow-min-cut}, there is a flow of value $13$.
But the capacities are either $1$ or infinity (or a very big integer), the Ford-Fulkerson algorithm starting with initial flow $0$ will terminate.
Hence there is an optimal flow (whose flows along every arc must be $0$ or $1$ by assumption).
Let $x$ be the optimal flow, then necessarily $x_{A,a_i}=1$.
As it is a flow, there is some $j$ such that $x_{a_i,b_j}=1$ and some $k$ such that $x_{b_j,c_k}=1$.
So we can match nodes in the way $a_i,c_k$ which provides the solution.
\subsection{The Transportation Problem}
The problem is as follows:
Imagine there are $m$ suppliers of a good labelled $1,\ldots,m$.
For any $i$, the supplier $i$ can supply $S_i$ units of the good.
There are $n$ destinations labelled $1,\ldots,n$ that require the supply of the good.
For any $j$, destination $j$ requires $D_j$ units of the good.
For each pair $(i,j)$, there is an associated unit transport cost $d_{i,j}$.
It is assumed in addition that the supply and demand matches, i.e.
$$\sum_{i=1}^mS_i=\sum_{j=1}^nD_j$$
This is basically the linear program to minimise the total cost $\sum_{i,j}d_{i,j}x_{i,j}$ subject to
$$\begin{cases}
    \forall i,\sum_{j=1}^nx_{i,j}=S_i\\
    \forall j,\sum_{i=1}^mx_{i,j}=D_j\\
    \forall i,j,x_{i,j}\ge 0
\end{cases}$$
The Lagrangian is then
\begin{align*}
    L(x,\lambda,\mu)&=\sum_{i,j}d_{i,j}x_{i,j}+\sum_{i=1}^m\lambda_i\left( S_i-\sum_{j=1}^n x_{i,j} \right)+\sum_{j=1}^n\mu_j\left( D_j-\sum_{i=1}^mx_{i,j} \right)\\
    &=\sum_{i,j}(d_{i,j}-\lambda_i-\mu_j)x_{i,j}+\sum_{i=1}^m\lambda_iS_i+\sum_{j=1}^n\mu_jD_j
\end{align*}
Note that $x_{i,j}\ge 0$, so we have $d_{i,j}\ge \lambda_i+\mu_j$ for the Lagrange multipliers to be feasible, hence
$$\Lambda=\{(\lambda,\mu):\inf_{x\ge 0}L(x,\lambda,\mu)>-\infty\}=\{(\lambda,\mu):\forall i,j,d_{i,j}\ge \lambda_i+\mu_j\}$$
If $(x_{i,j})$ is optimal for the problem, then there are feasible $(\lambda,\mu)$ such that
$$\forall i,j,(d_{i,j}-\lambda_i-\mu_j)x_{i,j}=0$$
by complementary slackness.
We can of course use simplex algorithm as it is a linear program, but the additional structure here allows us to simplify it.
\subsection{The Transportation Algorithm}
We design an algorithm for it as follows:
First, find an initial feasible assignment.
As there are $m+n-1$ linearly independent relations constraining the feasible set, any basic feasible solution will have $m+n-1$ basic variables.
There are two ways to do this: the North-West method and the greedy algorithm.
These will be introduced in a second.\\
After the initial feasible assignment is done, we assign Lagrange multipliers.
We may take $\lambda_1=0$ and enforce complementary slackness by choosing $\lambda_i$ and $\mu_i$ subject to $d_{i,j}=\lambda_i+\mu_j$ for each basic cell.\\
Next, we check for optimality by looking at the dual feasibility of the Lagrange multipliers.
The feasibility constraint is just $d_{i,j}\ge\lambda_i+\mu_j$ for each $i,j$.
If this is valid, then we are done.\\
Otherwise, we do the pivoting operation by picking one of the cells $(i,j)$ such that $\lambda_i+\mu_j>d_{i,j}$.
the rule is to pick the cell with the largest difference $\lambda_i+\mu_j-d_{i,j}$.
Now put an amount $\epsilon>0$ units of flow into the pivot cell.
At the same time add or substract $\epsilon$ from the basic cell to maintain feasibility.
We will try to choose the largest $\epsilon$ possible such that the flow is feasible.
Then do the same thing again and again until we reach optimality.
\begin{example}
    Suppose the table of supply, demand and cost is
    $$\begin{array}{c|c|c|c|c|c}
        d_{i,j}&&&&&D_j\\
        \hline
        &8&6&7&5&12\\ \hline
        &4&3&5&4&8\\ \hline
        &9&8&6&7&11\\ \hline
        S_i&7&6&10&8&
    \end{array}$$
    The North-West method is filling in the number of unit in each transport flow starting from North-West (top-left) of the array and fit in as many as we can, and the precedence of filling is from top-left to bottom-right.
    This way would give the basic feasible solution:
    $$\begin{pmatrix}
        7&5&&\\
        &1&7&\\
        &&3&8
    \end{pmatrix}$$
    Then we assign Lagrange multipliers by complementary slackness:
    $$\begin{array}{c|cccc|c}
        &8&6&8&9&\mu_j\\ \hline
        0&7&5&0&0&\\
        -3&0&1&7&0&\\
        -2&0&0&3&8&\\ \hline
        \lambda_i&&&&&
    \end{array}$$
    Now we test the optimality by writing out the payoffs, i.e. $\lambda_i+\mu_j-d_{i,j}$ in each of the zero cells (denoted by the terms in square brackets), which gives
    $$\begin{array}{c|cccc|c}
        &8&6&8&9&\mu_j\\ \hline
        0&7&5&[1]&[4]&\\
        -3&[1]&1&7&[2]&\\
        -2&[-3]&[-4]&3&8&\\ \hline
        \lambda_i&&&&&
    \end{array}$$
    So we need to pivot the top right corner, i.e. make it basic.
    Suppose it is changed to $\epsilon$, then we need to change our solution to
    $$\begin{pmatrix}
        7&5-\epsilon&&\epsilon\\
        &1+\epsilon&7-\epsilon&\\
        &&3+\epsilon&8-\epsilon
    \end{pmatrix}$$
    which is always feasible.
    For the solution to be basic, we take $\epsilon=5$ to swipe out an entry to get the new solution, which, after calculating Lagrange multipliers and payoffs,
    $$\begin{array}{c|cccc|c}
        &8&2&4&5&\mu_j\\ \hline
        0&7&[-4]&[-3]&5&\\
        1&[5]&6&2&[2]&\\
        2&[1]&[-4]&8&3&\\ \hline
        \lambda_i&&&&&
    \end{array}$$
    So we pivot the second row of the first column with, as we can find, has $\epsilon=2$.
    Doing this again and again and again and again and again and again and again and again and again yields our final table
    $$\begin{array}{c|cccc|c}
        &7&6&4&5&\mu_j\\ \hline
        0&[-1]&5&[-3]&7&\\
        -3&7&1&[-4]&[-2]&\\
        2&[0]&[0]&10&1&\\ \hline
        \lambda_i&&&&&
    \end{array}$$
    where all the entries in the square bracket is nonpositive, hence this is the optimal solution.\\
    The greedy algorithm works as well.
    This time we start with the cell of lowest cost and fill in as many as possible, then the second lowest, etc..
    In our case, this will produce the initial feasible assignment
    $$\begin{pmatrix}
        6&&&6\\
        &6&&2\\
        1&&10&
    \end{pmatrix}$$
    to which we can apply the same method to get to optimality.
\end{example}
\begin{remark}
    If the original posture of the prblem is such that the total supply exceeds the total demand, then we can reformulate the problem by adding another destination with demand
    $$D_{n+1}=\sum_{i=1}^mS_i-\sum_{j=1}^nD_j$$
    and set $d_{i,n+1}=0$ for all $i$ to ensure that all differences are consumed.
\end{remark}
Why does pivoting decrease the total cost?
For any feasible $x$ and any collection $(\lambda_i)$ and $(\mu_j)$ of Lagrange multipliers we have
\begin{align*}
    f(x)&=L(x,\lambda,\mu)\\
    &=\sum_{i,j}(d_{i,j}-\lambda_i-\mu_j)x_{i,j}+\sum_{i=1}^m\lambda_iS_i+\sum_{j=1}^n\mu_jD_j
\end{align*}
So suppose $x_0$ is the initial feasible assignment, then we can choose $\lambda_{i,0}$ and $\mu_{i,0}$ to enforce complementary slackness $(d_{i,j,0}-\lambda_{i,0}-\mu_{j,0})x_{i,j,0}=0$, then
$$f(x_0)=\sum_{i=1}^m\lambda_{i,0}S_i+\sum_{j=1}^n\mu_{j,0}D_j$$
Let $B=\{(i,j):x_{i,j,0}>0\}$ and $N=\{(i,j):x_{i,j,0}=0\}$ be the basis of the basic feasible solution $x_0$.
We choose $\lambda_{i,0}$ and $\mu_{i,0}$ such that $\lambda_{i,0}+\mu_{j,0}=d_{i,j}$ for all $(i,j)\in B$.
Suppose the next feasible assignment $x_1$ is non-degenerate, as one of the initial non-basic cells (the pivot cell) is made basic, there is a cell $(i_N,j_N)\in N$ such that
$$x_{i_N,j_N,0}=0,x_{i_N,j_N,1}=\epsilon>0$$
Then we obtain
\begin{align*}
    f(x_1)&=\sum_{(i,j)\in B}(d_{i,j}-\lambda_{i,0}-\mu_{j,0})x_{i,j,1}+\sum_{(i,j)\in N}(d_{i,j}-\lambda_{i,0}-\mu_{j,0})x_{i,j,1}
    +f(x_0)\\
    &=(d_{i_N,j_N}-\lambda_{i_N,0}-\mu_{j_N,0})\epsilon+f(x_0)\\
    &>f(x_0)
\end{align*}
by assumption.


\lab{Algorithms}{QR Decomposition (Householder)}{Canonical Transformations and the QR Decomposition}
\label{Ch:Canonical Transformations}

\objective{Use orthogonal transformations to perform QR decomposition.}

\section*{Orthogonal transformations}
Recall that a matrix $Q$ is \emph{unitary} if $Q^* Q = I$ or for real matrices, $Q^T Q = I$ (since the conjugate of a real number is itself). We like unitary transformations because they're very numerically stable. The number $\kappa(A) = \norm{A} \norm{A^{-1}}$ is called the \emph{condition number} of $A$. We'll discuss condition number more in Lab \ref{Ch:Norms and Geometry}; for now, all you need to know is that if $\kappa(A)$ is small, then problems involving $A$ are less susceptible to numerical errors. For induced matrix norms (which include most of the matrix norms we would ever care about),  it holds that $\norm{Q}=1$ when $Q$ is unitary. The inequality $\norm{AB} \leq \norm{A} \norm{B}$ also holds for these norms. It follows that $\kappa(A) = \norm{A} \norm{A^{-1}} \geq \norm{A A^{-1}} = \norm{I} = 1$. Note that if $Q$ is unitary, $Q^{-1} = Q^*$ and $Q^*$ is also unitary, so $\kappa(Q) = \norm{Q} \norm{Q^*} = 1$. This means that orthogonal matrices have the smallest possible condition number, which is great!

Any unitary matrix $Q$ can be described as a reflection, a rotation, or some combination of the two. If $det(Q) = 1$, then $Q$ is a rotation; if $det(Q) = -1$, then $Q$  is the composition of a reflection and a rotation.  Let's explore these two types of unitary transformations and some of their applications. We will focus on the real case to simplify matters.

\section*{Householder reflections}
A Householder reflection is a linear transformation $P: \mathbb{R}^n \rightarrow \mathbb{R}^n$ that reflects a vector $x$ about a hyperplane. See figure \ref{fig:Householder reflector}. Recall that a hyperplane can be defined by a unit vector $v$ which is orthogonal to the hyperplane. As shown in the figure, $x - \langle v,x \rangle v$ is the projection of $x$ onto the hyperplane defined by $v$. (You should verify this geometrically.) However, to reflect \emph{across} the hyperplane, we must move twice as far; that is, $Px = x - 2\langle v,x \rangle v$. This can be written $Px = x - 2v(v^\ast x)$, so $P$ has matrix representation $P = I - 2vv^\ast$. Note that $P^\ast P = I$; thus $P$ is orthogonal.

\begin{figure}
	\centering
	\includegraphics[width= \textwidth]{fig1}
	\caption{Householder reflector}
	\label{fig:Householder reflector}
\end{figure}

\subsection*{Householder triangularization}
Consider the problem of computing the $QR$ decomposition of a matrix $A$. You've already learned the Gram-Schmidt and the Modified Gram-Schmidt algorithms for this problem. The $QR$ decomposition can also be computed using Householder triangularization. Gram-Schmidt and Modified Gram-Schmidt \emph{orthogonalize} $A$ by a series of \emph{triangular} transformations. Conversely, the Householder method \emph{triangularizes} $A$ by a series of \emph{orthogonal} transformations.

Let's demonstrate this method on a $4 \times 3$ matrix $A$. First we find a orthogonal transformation $Q_1$ that maps the first column of A into the range of $e_1$ (where $e_1$ is the vector where the first element is one and the remainder of the elements are zeros).

\def\mc#1{\multicolumn{1}{c|}{#1}}
\begin{equation*}
\begin{pmatrix}
\ast & \ast & \ast \\
\ast & \ast & \ast \\
\ast & \ast & \ast \\
\ast & \ast & \ast
\end{pmatrix}
\underrightarrow{Q_1}
\begin{pmatrix}

\ast & \ast & \ast & \\ \cline{2-3}
\mc{0} & \ast & \mc{\ast}& \\
\mc{0} & \ast & \mc{\ast} & \\
\mc{0}& \ast & \mc{\ast} & \\ \cline{2-3}
\end{pmatrix}
\end{equation*}
Let $A_2$ be the boxed submatrix of $A$. Now find an orthogonal transformation $Q_2$ that maps the first column of $A_2$ into the range of $e_2$.

\begin{equation*}
\begin{pmatrix}
\ast & \ast \\
\ast & \ast \\
\ast & \ast
\end{pmatrix}
\underrightarrow{Q_2}
\begin{pmatrix}
\ast & \ast \\
0 & \ast \\
0 & \ast
\end{pmatrix}
\end{equation*}
Similarly, $ \begin{pmatrix} \ast \\ \ast \end{pmatrix} \underrightarrow{Q_3} \begin{pmatrix} \ast \\ 0 \end{pmatrix} $. (Technically $Q_2$ and $Q_3$ act on the whole matrix and not just on the submatrices, so that $Q_i: \mathbb{R}^n \rightarrow \mathbb{R}^n$ for all $i$. $Q_2$ leaves the first row alone, and $Q_3$ leaves the first two rows alone.) Then $Q_3 Q_2 Q_1 A =$

\begin{equation*}
Q_3 Q_2 Q_1
\begin{pmatrix}
\ast & \ast & \ast \\
\ast & \ast & \ast \\
\ast & \ast & \ast \\
\ast & \ast & \ast \\
\end{pmatrix}
= Q_3 Q_2
\begin{pmatrix}
\ast & \ast & \ast \\
0 & \ast & \ast \\
0 & \ast & \ast \\
0 & \ast & \ast \\
\end{pmatrix}
= Q_3
\begin{pmatrix}
\ast & \ast & \ast \\
0 & \ast & \ast \\
0 & 0 & \ast \\
0 & 0 & \ast \\
\end{pmatrix}
=
\begin{pmatrix}
\ast & \ast & \ast \\
0 & \ast & \ast \\
0 & 0 & \ast \\
0 & 0 & 0 \\
\end{pmatrix}
\end{equation*}

We've accomplished our goal, which was to triangularize $A$ using orthogonal transformations. But now, how do we find the $Q_i$ that do what we want? Using Householder reflections. (Surprise!)

For example, to find $Q_1$, we choose the right hyperplane to reflect $x$ into the range of $e_1$. It turns out there are two hyperplanes that will work, as shown in figure \ref{fig:two reflectors}. (In the complex case, there are infinitely many such hyperplanes.) Between the two, the one that reflects $x$ further will be more numerically stable. This is the hyperplane perpendicular to \textbf{$v = sign(x_1)\norm{x}_2 e_1 + x$}. The whole process is summarized in Algorithm \ref{Alg:Householder triangularization}.

\begin{figure}
	\centering
	\includegraphics[width= \textwidth]{fig2}
	\caption{two reflectors}
	\label{fig:two reflectors}
\end{figure}

\begin{pseudo}{Householder triangularization}{A}
\label{Alg:Householder triangularization}
m,n \GETS size(A)\\
\FOR k \GETS 1 \TO n-1 \DO
\BEGIN
   x = A_{k:m,k}\\
   v_k = sign(x_1)\norm{x}_2 e_1 + x\\
   v_k = v_k / \norm{v_k}_2\\
   P_k = eye(m,m)\\
   P_k[k:m,k:m] = P_k[k:m,k:m] - 2 v_k v_k^T\\
   A = sp.dot(P_k,A);
\END
\end{pseudo}

This algorithm returns upper triangular $R$. You can find $Q$ s.t. $QR = A$ by multiplying the $P_k$ together appropriately.

\begin{problem}
\label{prob:HouseholderQR}
Write a script using Householder reflections to find the QR decomposition of a matrix A.
\end{problem}

\subsection*{Stability of the Householder QR algorithm}

Try the following in Python.

\begin{lstlisting}
In [1]:  import scipy as sp
In [2]:  import numpy.linalg as la
In [3]:  import my_householder
In [4]:  Q,X = la.qr(sp.rand(50,50)) #create a random orthogonal matrix:
In [5]:  R = sp.triu(sp.rand(50,50)) # create a random upper triangular matrix
In [6]:   A = sp.dot(Q,R) #Q and R are the exact QR decomposition of A
# use your Householder QR script to estimate Q and R:
In [7]:   Q1,R1 = my_householder.qr(A)
#now check the relative errors of Q1 and R1
In [8]:  la.norm(Q1-Q)/la.norm(Q)
Out [8]:  0.282842955725
In [9]:  la.norm(R1-R)/la.norm(R)
Out[9]:  0.0428922016647
\end{lstlisting}
This is terrible! Python works in $16$ decimal points of precision. But $Q_1$ and $R_1$ are only accurate to $0$ and $1$ decimal points, respectively. We've lost $16$ decimal points of precision!

Don't lose hope. Check how close the product $Q_1 R_1$ is to $A$.
\begin{lstlisting}
In [10]:  A1 = sp.dot(Q1,R1)
In [11]:  la.norm(A1-A)/la.norm(A)
Out[11]:  9.73996046986e-16
\end{lstlisting}
We've now recovered $15$ digits of accuracy. The errors in $Q_1$ and $R_1$ were somehow ``correlated," so that they canceled out in the product. The errors in $Q_1$ and $R_1$ are called \emph{forward errors}. The error in $A_1$ is the \emph{backward error}. The Householder $QR$ algorithm is a backward stable algorithm.

Householder QR factorization is more numerically stable than Gram-Schmidt or even Modified Gram-Schmidt (MGS). However, MGS is still useful for some types of iterative methods, because it finds the orthogonal basis one vector at a time instead of all at once (for example see Lab \ref{Ch:EigSolve}).

\subsection*{Upper Hessenberg Form}

%I'm not sure about this math. Is this true in the real case?
An upper Hessenberg matrix is a square matrix with zeros below the first subdiagonal. Every  $n \times n$ matrix $A$ can be written $A = Q^THQ$ where $Q$ is orthogonal and $H$ is an upper Hessenberg matrix, called the Hessenberg form of $A$. Note the similarity of this decomposition to the Schur decomposition in Lab \ref{Ch:Jordan}.

The Hessenberg decomposition can be computed using Householder reflections, in a process very similar to Householder triangularization. Let's demonstrate this process on a $5 \times 5$ matrix $A$. Note that $A=Q^THQ$ is equivalent to $QAQ^T = H$; thus our strategy is to multiply $A$ on the right and left by a series of orthogonal matrices until it is in Hessenberg form. If we try the same $Q_1$ as in the first step of the Householder algorithm, then with $Q_1 A$ we introduce zeros in the first column of $A$. However, since we now have to multiply $Q_1 A$ on the left by $Q_1^T$, all those zeros are destroyed, as demonstrated below. (Although this process may seem futile now, it actually does tend to decrease the size of the subdiagonal entries. If we repeat it over and over again, the subdiagonal entries will often converge to zero. That's the idea behind the $QR$ algorithm in Lab \ref{Ch:EigSolve}.)
\[
\begin{array}{ccccc}
\begin{pmatrix}
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
\end{pmatrix}
&\underrightarrow{Q_1 \cdot }&
\begin{pmatrix}
* & * & * & * & *\\
0 & * & * & * & *\\
0 & * & * & * & *\\
0 & * & * & * & *\\
0 & * & * & * & *\\
\end{pmatrix}
&\underrightarrow{\cdot Q_1^T }&
\begin{pmatrix}
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
\end{pmatrix}
\\
A & & Q_1A & & Q_1 A Q_1^T
  \end{array}
\]
Instead, let's try starting with a different $Q_1$ that leaves the \emph{first} row alone and reflects the \emph{rest} of the rows into the range of $e_2$. This means that $Q_1^T$ leaves the first column alone.
\[
\begin{array}{ccccc}
\begin{pmatrix}
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
* & * & * & * & *\\
\end{pmatrix}
&\underrightarrow{Q_1 \cdot }&
\begin{pmatrix}
* & * & * & * & *\\
* & * & * & * & *\\
0 & * & * & * & *\\
0 & * & * & * & *\\
0 & * & * & * & *\\
\end{pmatrix}
&\underrightarrow{\cdot Q_1^T }&
\begin{pmatrix}
* & * & * & * & *\\
* & * & * & * & *\\
0 & * & * & * & *\\
0 & * & * & * & *\\
0 & * & * & * & *\\
\end{pmatrix}
\\
A & & Q_1A & & Q_1 A Q_1^T
  \end{array}
\]
We now iterate through the matrix until we obtain
\begin{equation*}
Q_3 Q_2 Q_1 A Q_1^T Q_2 ^T Q_3^T =
\begin{pmatrix}
* & * & * & * & *\\
* & * & * & * & *\\
0 & * & * & * & *\\
0 & 0 & * & * & *\\
0 & 0 & 0 & * & *\\
\end{pmatrix}
\end{equation*}

\begin{problem}
Write a script that transfers an input matrix to upper Hessenberg form. (Hint: You only need to modify your code code from problem \ref{prob:HouseholderQR} slightly.) We will use this technique in the eigenvalue lab later.
\end{problem}

%Sources: http://www.cs.unc.edu/~krishnas/eigen/node5.html
% http://en.wikipedia.org/wiki/Givens_rotation
%http://en.wikipedia.org/wiki/QR_decomposition
%	Note the Operation count: Householder is 2/3 n^3, MGS is 2 n^3
%http://en.wikipedia.org/wiki/QR_algorithm
%Applied Numerical methods using MATLAB by Yang has some code written for this
%http://www.math.kent.edu/~reichel/courses/intr.num.comp.2/lecture21/evmeth.pdf
%	These are eigenvalue algorithms explained carefully
%http://en.wikipedia.org/wiki/Householder_transformation
%Numerical Linear Algebra, by Lloyd N. Trefethen and David Bau III, Chapters 10 and 16


\documentclass{article}

\usepackage[colorlinks=true]{hyperref}
\usepackage[cmex10]{amsmath}
\usepackage{bbm}
\usepackage{graphicx}
\usepackage{subfig}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{comment}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{multirow}

\DeclareMathOperator*{\argmin}{\mathrm{argmin}}
\DeclareMathOperator*{\pro}{\mathcal P_{\Omega}}
\DeclareMathOperator*{\pron}{\mathcal N_{\Omega}}
\DeclareMathOperator*{\proa}{\mathcal P_{\Psi}}
\newcommand{\BigO}[1]{\ensuremath{\operatorname{O}\left(#1\right)}}

\begin{document}

\title{PCPKit Documentation}
\author{Stephen Tierney}
\maketitle

\section{Function Listing}

\begin{table}[!h]
{\small{
\centering

\begin{tabular}{c | c | c}

\hline
Problem & Function & Section\\

\hline
{\bf Matrix Completion - Noise Free}\\

\hline
$\begin{array}{c} \min_{\mathbf A} \; \tau \| \mathbf A \|_* +  \frac{1}{2} \| \mathbf{ A } \|_F^2\\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \end{array}$ & mc\_svt	& \ref{mc_svt}\\

\hline
\multirow{2}{*}{$\begin{array}{c} \min_{\mathbf A} \; \tau \| \mathbf A \|_*\\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \end{array}$} & mc\_ialm	& \ref{mc_ialm}\\
		& solve\_lin	& \ref{mc_lin}\\

\hline
{\bf Matrix Completion - Noisy}\\

\hline
\multirow{3}{*}{$\begin{array}{c} \tau \| \mathbf A \|_*  +  \frac{\lambda}{2} \| \mathbf{ \pro (A) - \pro (M)  } \|^2_F  \end{array}$}
	& mc\_relax\_lin & \ref{mc_relax}\\
	& mc\_relax\_lin\_ext &\\
	& mc\_relax\_lin\_acc &\\

\hline
$\begin{array}{c} \min_{\mathbf{A,E}} \; \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \mathbf E \|_F^2\\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) + \pro (\mathbf E) \end{array}$ & mc\_exact\_fro	& \ref{mc_exact_fro}  \\

\hline
{\bf Principal Component Pursuit}\\

\hline
$\begin{array}{c} \min_{\mathbf{A,E}} \; \| \mathbf A \|_* + \frac{\lambda}{2} \| \mathbf E \|_F^2\\
\text{s.t.} \; \mathbf { M = A + E } \end{array}$ & pcp\_fro & \ref{pcp_fro} \\

\hline
$\begin{array}{c} \min_{\mathbf{A,E}} \; \| \mathbf A \|_* + \lambda \| \mathbf E \|_1\\
\text{s.t.} \; \mathbf { M = A + E } \end{array}$ & pcp\_l1 & \ref{pcp_l1}  \\

\hline
$\begin{array}{c} \min_{\mathbf{A,E}} \; \| \mathbf A \|_* + \lambda \| \mathbf E \|_{1,2}\\
\text{s.t.} \; \mathbf { M = A + E } \end{array}$ & pcp\_l1l2 & \ref{pcp_l1l2} \\

\hline
{\bf Selective PCP}\\

\hline
$\begin{array}{c} \min_{\mathbf A, \mathbf E} \;  \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \mathbf E \|^2_F \\
\text{s.t.} \; \pro(\mathbf{M}) = \pro(\mathbf{A}) \nonumber \\
\proa(\mathbf{M}) = \proa(\mathbf{A}) + \proa(\mathbf E) \nonumber \\  \end{array}$  & sel\_pcp & \ref{sel_pcp}

\end{tabular}
}}
\end{table}

\newpage
\section{Matrix Completion}

\subsection{Noise Free Data}

\subsubsection{SVT}
\label{mc_svt}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_values}, \mathbf{stop\_vals} ] = \text{mc\_svt}( \mathbf M, \Omega, \tau, \mu, iterations, tol )\notag
\end{align}
solves the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* +  \frac{1}{2} \| \mathbf{ A } \|_F^2 \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \nonumber
\end{align}\
as proposed by the authors of \cite{cai2010singular}.

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\mu$ - step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsubsection{Inexact ALM}
\label{mc_ialm}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{mc\_ialm}( \mathbf M, \Omega, \tau, \mu, iterations, tol )\notag
\end{align}
solves the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \nonumber
\end{align}
as proposed by the authors of \cite{lin2010augmented}.

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\mu$ - step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsubsection{Linearised ALM}
\label{mc_lin}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{mc\_lin}( \mathbf M, \Omega, \tau, \mu, \rho, iterations, tol ) \notag
\end{align}
solves the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_* \\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) \nonumber
\end{align}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\mu$ - step size (optional)
\item $\rho$ - linearisation step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsection{Noisey Data Relaxation}
\label{mc_relax}

The functions
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{mc\_relax\_lin}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol ) \notag \\
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{mc\_relax\_lin\_ext}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol ) \notag \\
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{mc\_relax\_lin\_acc}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol )\notag
\end{align}
solve the following
\begin{align}
\min_{\mathbf A} \; \tau \| \mathbf A \|_*  +  \frac{\lambda}{2} \| \mathbf{ \pro (A) - \pro (M)  } \|^2_F
\end{align}
with increasing convergence speed based on \cite{ji2009accelerated}.

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\lambda$ - regularisation (optional)
\item $\rho$ - linearisation step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\subsection{Noisey Data Exact}
\label{mc_exact_fro}

The function
\begin{align}
[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_exact\_fro}( \mathbf M, \Omega, \tau, \lambda, \rho, iterations, tol ) \notag
\end{align}
solve the following
\begin{align}
\min_{\mathbf A, \mathbf E} \; \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \mathbf E \|_F^2\\
\text{s.t.} \; \pro (\mathbf M) = \pro (\mathbf A) + \pro (\mathbf E) \nonumber
\end{align}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\Omega$ - vector of constrained matrix indices
\item $\tau$ - regularisation (optional)
\item $\lambda$ - regularisation (optional)
\item $\rho$ - linearisation step size (optional)
\item $iterations$ - maximum number of iterations (optional)
\item $tol$ - stopping criteria tolerance (optional)
\end{itemize}

\newpage
\section{Principal Component Pursuit}

\subsection{PCP Gaussian}
\label{pcp_fro}

The function
\begin{align}
[ \mathbf A ] = \text{pcp\_fro}( \mathbf M, \lambda ) \notag
\end{align}
solve the following
\begin{align}
\min_{\mathbf{A, E}} \; \| \mathbf A \|_* + \frac{\lambda}{2} \| \mathbf E \|_F^2\\
\text{s.t.} \; \mathbf { M = A + E } \nonumber
\end{align}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\lambda$ - regularisation
\end{itemize}

\subsection{PCP Sparse}
\label{pcp_l1}

The function
\begin{align}
[ \mathbf A ] = \text{pcp\_l1}( \mathbf M, \lambda ) \notag
\end{align}
solve the following
\begin{align}
\min_{\mathbf{A, E}} \; \| \mathbf A \|_* + \lambda \| \mathbf E \|_1\\
\text{s.t.} \; \mathbf { M = A + E } \nonumber
\end{align}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\lambda$ - regularisation
\end{itemize}

\subsection{PCP Column-wise Gaussian}
\label{pcp_l1l2}

The function
\begin{align}
[ \mathbf A ] = \text{pcp\_l1l2}( \mathbf M, \lambda ) \notag
\end{align}
solves the following
\begin{align}
\min_{\mathbf{A, E}} \; \| \mathbf A \|_* + \lambda \| \mathbf E \|_{1,2}\\
\text{s.t.} \; \mathbf { M = A + E } \nonumber
\end{align}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\lambda$ - regularisation
\end{itemize}

\newpage
\section{Selective PCP}
\label{sel_pcp}

The function
\begin{align*}
[ \mathbf A ] = \text{sel\_pcp}( \mathbf M, \tau, \lambda, \Omega, \Psi )
\end{align*}
solves the following
\begin{align*}
\min_{\mathbf A, \mathbf E} \;  \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \mathbf E \|^2_F \\
\text{s.t.} \; \pro(\mathbf{M}) = \pro(\mathbf{A}) \nonumber \\
\proa(\mathbf{M}) = \proa(\mathbf{A}) + \proa(\mathbf E) \nonumber
\end{align*}

\begin{itemize}
\item $\mathbf M$ - matrix with observed entries
\item $\tau$ - regularisation
\item $\lambda$ - regularisation
\item $\Omega$ - uncorrupted indices
\item $\Psi$ - noisy indices
\end{itemize}

%The function
%\begin{align}
%[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_mixed}( \mathbf M, \Omega, \mathbf H, \Omega_H, \tau, \mu, \rho, iterations, tol ) \notag
%\end{align}
%solves the following
%\begin{align}
%\min_{\mathbf A} \; \tau \| \mathbf A \|_* + \frac{\lambda}{2} \| \proe(\mathbf A) -  \proe(\mathbf H) \|^2_F \\
%\text{s.t.} \; \pro(\mathbf{M}) = \pro(\mathbf{A}) \nonumber
%\label{objective}
%\end{align}
%
%\begin{itemize}
%\item $\mathbf M$ - matrix with observed entries
%\item $\Omega$ - vector of constrained matrix indices
%\item $\mathbf H$ - matrix with approximate entries
%\item $\Omega_H$ - vector of approximately penalised indices
%\item $\tau$ - regularisation (optional)
%\item $\mu$ - step size (optional)
%\item $\rho$ - linearisation step size (optional)
%\item $iterations$ - maximum number of iterations (optional)
%\item $tol$ - stopping criteria tolerance (optional)
%\end{itemize}
%
%\subsection{Noisey Data}
%
%The functions
%\begin{align}
%[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_mixed\_e}( \mathbf M, \Omega, \mathbf H, \Omega_H, \tau, \mu, \rho, iterations, tol ) \notag \\
%[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_mixed\_e\_ext}( \mathbf M, \Omega, \mathbf H, \Omega_H, \tau, \mu, \rho, iterations, tol ) \notag  \\
%[ \mathbf A, \mathbf{f\_vals}, \mathbf{stop\_vals} ] = \text{solve\_mixed\_e\_acc}( \mathbf M, \Omega, \mathbf H, \Omega_H, \tau, \mu, \rho, iterations, tol ) \notag
%\end{align}
%solve the following
%\begin{align}
%\min_{\mathbf A} \; \tau \| \mathbf A \|_* + \frac{\lambda}2\|\proe(\mathbf A) - \proe(\mathbf H) \|^2_F +  \frac{\lambda_2}{2} \| \mathbf{ \pro(E)} \|^2_F \\
%\text{s.t.} \; \mathbf{\pro(M) = \pro(A) +  \pro(E)} \nonumber
%\end{align}
%
%\begin{itemize}
%\item $\mathbf M$ - matrix with observed entries
%\item $\Omega$ - vector of constrained matrix indices
%\item $\mathbf H$ - matrix with approximate entries
%\item $\Omega_H$ - vector of approximately penalised indices
%\item $\tau$ - regularisation (optional)
%\item $\mu$ - step size (optional)
%\item $\rho$ - linearisation step size (optional)
%\item $iterations$ - maximum number of iterations (optional)
%\item $tol$ - stopping criteria tolerance (optional)
%\end{itemize}

\newpage
\bibliographystyle{plain}
\bibliography{references}

\end{document}