\section{Tutorial 2: Delilah and Complex Math Formulae}
Delilah is working on a part of a material about linear algebra. As her work proceeds, she will obtain the ability to deal with complex mathematical formulae in \LaTeX{}, especially those methods provided by the ams packages loaded in xjtlumath.

\subsection{Multiple Lined Formulae}
A system of linear equations is a fundamental part of linear algebra. When Delilah tries to type a group of equations, she encounters a problem. In the predefined \verb=\[\]= and environment equation, she finds no option to start a new line. Even the line-break options of \LaTeX{} like \verb=\\= and \verb=\newline= do not work there. Of course the equations should not be put in one line, so what should she do now? Later she learns that the environment \emph{aligned} is designed to allow a system of equations to be aligned in multiple lines:

\begin{miniexammar}{.4\textandmarginlen}{
\[
\begin{aligned}
x+y &= 1\\
x-y &= 2
\end{aligned}
\]
}
\begin{lstlisting}
\[
\begin{aligned}
x+y &= 1\\
x-y &= 2
\end{aligned}
\]
\end{lstlisting}
\end{miniexammar}
Here, the ampersand sign \verb=&= is used before the symbols according to which the equations are to be aligned. The line break sign \verb=\\= starts a new line of equation. Note that other line-break operations cannot be used here.

Delilah likes the result, but she feels that the equations are too lonely. She thinks that adding a large curly bracket for them will comfort them. \LaTeX{} supports a syntax to put things before and after a group of things.

\begin{miniexammar}{.4\textandmarginlen}{
\[
\left\{
\begin{aligned}
x+y &= 1\\
x-y &= 2
\end{aligned}
\right.
\]
}
\begin{lstlisting}
\[
\left\{
\begin{aligned}
x+y &= 1\\
x-y &= 2
\end{aligned}
\right.
\]
\end{lstlisting}
\end{miniexammar}
The \verb=\left= command defines what is to be put on the left, and the \verb=\right= command defines what is to be put on the right. Delilah does not want to put anything on the right, so she writes \verb=.= for nothing.

For a group of equations that requires no alignment, or for a single equation that is too long to fit in one line, the environment \emph{gathered} that does no alignment is a better choice:

\begin{miniexammar}{.45\textandmarginlen}{
\[
\begin{gathered}
\cos {z} = 1 - \frac{z^2}{2!} + \frac{z^4}{4!} - \frac{z^6}{6!} + \cdots \\
= \sum_{n=0}^\infty {\frac{(-1)^n z^{2n}}{(2n!)}}
\end{gathered}
\]
}
\begin{lstlisting}
\[
\begin{gathered}
\cos {z} = 1 - \frac{z^2}{2!} + \frac{z^4}{4!} - \cdots \\
= \sum_{n=0}^\infty {\frac{(-1)^n z^{2n}}{(2n!)}}
\end{gathered}
\]
\end{lstlisting}
\end{miniexammar}

After writing several groups of equations, Delilah wants to reference one of them. She uses the equation environment instead of \verb=\[\]=, but finds out that the equations are numbered as a whole.

\begin{miniexammar}{.4\textandmarginlen}{
\begin{equation}
\begin{aligned}
x+y &= 1\\
x-y &= 2
\end{aligned}
\end{equation}
}
\begin{lstlisting}
\begin{equation}
\begin{aligned}
x+y &= 1\\
x-y &= 2
\end{aligned}
\end{equation}
\end{lstlisting}
\end{miniexammar}
So it is difficult for her to reference a single equation in a group. amsmath provides the environment align for this purpose:

\begin{miniexammar}{.4\textandmarginlen}{
\begin{align}
x+y &= 1\\
x-y &= 2
\end{align}
}
\begin{lstlisting}
\begin{align}
x+y &= 1\\
x-y &= 2
\end{align}
\end{lstlisting}
\end{miniexammar}

If she doesn't want to number a single equation, she needs to append \verb=\nonumber= at the end of that line.
\begin{miniexammar}{.4\textandmarginlen}{
\begin{align}
x+y &= 1\\
z&=10 \nonumber\\
x-y &= 2
\end{align}
}
\begin{lstlisting}
\begin{align}
x+y &= 1\\
z &= 10 \nonumber\\
x-y &= 2
\end{align}
\end{lstlisting}
\end{miniexammar}

Without the ``ed'' suffix, \emph{gather} is also a standalone environment that does what gathered do. But there is a major difference between the normal version and ``ed''ed version. Delilah finds it impossible to put the bracket again before a align or gather, because they don't need to be surrounded by mathematical environments. Also, their width are fixed to be the width of texts, while their ``ed''ed versions can be of any width.

As the same as the equation environment, their starred versions give no number by default.
\begin{miniexammar}{.4\textandmarginlen}{
\begin{align*}
x+y &= 1\\
x-y &= 2
\end{align*}
}
\begin{lstlisting}
\begin{align*}
x+y &= 1\\
x-y &= 2
\end{align*}
\end{lstlisting}
\end{miniexammar}

Delilah is able to put multiple groups of equations in one align, just by adding ampersands between the groups.
\begin{miniexammar}{.4\textandmarginlen}{
\begin{align*}
x+y &= 1  & a+b &= 3\\
x-y &= 2  & a-b &= 4
\end{align*}
}
\begin{lstlisting}
\begin{align*}
x+y &= 1  & a+b &= 3\\
x-y &= 2  & a-b &= 4
\end{align*}
\end{lstlisting}
\end{miniexammar}
The space between the groups is adjusted automatically by align.

\subsection{Matrices}
Matrices are vital to linear algebra, as they represent linear mappings from a vector space to another in specific bases. Also, the coefficient matrix and the augmented matrix are convenient in operating linear equations.

amsmath provides several environments for typing matrices.
\begin{miniexammar}{.5\textandmarginlen}{
\[
\begin{bmatrix}
1&2&3&4\\
5&6&7&8\\
9&10&11&12\\
13&14&15&16
\end{bmatrix}
\]
}
\begin{lstlisting}
\[
\begin{bmatrix}
1&2&3&4\\
5&6&7&8\\
9&10&11&12\\
13&14&15&16
\end{bmatrix}
\]
\end{lstlisting}
\end{miniexammar}
The environments pmatrix, Bmatrix, vmatrix, and Vmatrix produce delimiters of \verb=()=, \verb={}=, \verb=||=, and \verb=|| ||=, respectively.

To use matrices in inline mode, Delilah uses the environment smallmatrix, which has no p,b,B,v,V versions in amsmath, as it is the author's responsibility to decide the delimiters.
\begin{miniexammar}{.4\textandmarginlen}{
The matrix $\left(\begin{smallmatrix} a&b\\c&d \end{smallmatrix}\right)$ is so small and cute!
}
\begin{lstlisting}
The matrix $\left(\begin{smallmatrix} a&b\\c&d \end{smallmatrix}\right)$ is so small and cute!
\end{lstlisting}
\end{miniexammar}

When Delilah tries to put fractions inside a matrix, she finds something annoying.
\begin{miniexammar}{.3\textandmarginlen}{
\[
\begin{bmatrix}
1&\frac{1}{2}&\frac{1}{3}\\
1&\frac{1}{4}&\frac{1}{5}\\
\end{bmatrix}
\]
}
\begin{lstlisting}
\[
\begin{bmatrix}
1&\frac{1}{2}&\frac{1}{3}\\
1&\frac{1}{4}&\frac{1}{5}\\
\end{bmatrix}
\]
\end{lstlisting}
\end{miniexammar}
The fractions above and below are so close that they touch each other! This is not what Delilah wants and she is surprised that \LaTeX{} doesn't detect this and do something. Fortunately, in amsmath environments, an optional argument is allowed to be passed to \verb=\\= to define the actual vertical space between lines. For fractions, 2ex is a good option. Also, the fractions are in inline mode. The \verb=\dfrac= command gives displayed fractions.
\begin{miniexammar}{.3\textandmarginlen}{
\[
\begin{bmatrix}
1&\dfrac{1}{2}&\dfrac{1}{3}\\[2ex]
1&\dfrac{1}{4}&\dfrac{1}{5}
\end{bmatrix}
\]
}
\begin{lstlisting}
\[
\begin{bmatrix}
1&\dfrac{1}{2}&\dfrac{1}{3}\\[2ex]
1&\dfrac{1}{4}&\dfrac{1}{5}
\end{bmatrix}
\]
\end{lstlisting}
\end{miniexammar}

Sometimes a matrix is too large to be displayed fully. At these times, the use of ellipses (plural of ellipsis, not ellipse) is important. When Delilah writes the inverse of a matrix, she uses ellipses.
\begin{miniexammar}{.5\textandmarginlen}{
\[
A^{-1} = \frac{1}{\det A}
\begin{bmatrix}
C_{11} & C_{21} & \cdots & C_{n1} \\
C_{12} & C_{22} & \cdots & C_{n2} \\
\vdots & \vdots & \ddots & \vdots \\
C_{n2} & C_{n2} & \cdots & C_{nn} \\
\end{bmatrix}
\]
}
\begin{lstlisting}
\[
A^{-1} = \frac{1}{\det A}
\begin{bmatrix}
C_{11} & C_{21} & \cdots & C_{n1} \\
C_{12} & C_{22} & \cdots & C_{n2} \\
\vdots & C\vdots & \ddots & \vdots \\
C_{n2} & C_{n2} & \cdots & C_{nn} \\
\end{bmatrix}
\]
\end{lstlisting}
\end{miniexammar}

\subsection{Texts and Operator Names}
To put text inside math environments, Delilah uses the \verb=\text= command provided by amsmath.
\begin{miniexammar}{.57\textandmarginlen}{
\begin{definition}[Null Space]
The null space of an $m \times n$ matrix $A$, written as $\Nul A$, is the set of all solutions
of the homogeneous equation $A\vec{x} = \vec{0}$. In set notation,
\[
\Nul A = \{\vec{x}:\vec{x} \text{ is in } \mathbb{R}^n \text{ and } A\vec{x} = \vec{0} \}
\]
\end{definition}
}
\begin{lstlisting}
\begin{definition}[Null Space]
The null space of an $m \times n$ matrix $A$, written as $\Nul A$, is the set of all solutions
of the homogeneous equation $A\vec{x} = \vec{0}$. In set notation,
\[
\Nul A = \{\vec{x}:\vec{x} \text{ is in } \mathbb{R}^n \text{ and } A\vec{x} = \vec{0} \}
\]
\end{definition}
\end{lstlisting}
\end{miniexammar}

The commands like \verb=\Nul=, \verb=\sin=, ... are math operators. Part of predefined math operators in \LaTeX{} are shown in Table \ref{tab:predefmathop}.
\begin{table}[hbpt]
\begin{center}
\small
\begin{tabular}{cl|cl|cl}
Result & Command & Result & Command & Result & Command \\
\hline
arccos & \verb=\arccos= & arcsin & \verb=\arcsin= & arctan & \verb=\arctan= \\
cos & \verb=\cos= & sin & \verb=\sin= & tan & \verb=\tan= \\
cot & \verb=\cot= & sec & \verb=\sec= & csc & \verb=\csc= \\
cosh & \verb=\cosh= & sinh & \verb=\sinh= & tanh & \verb=\tanh= \\
lim & \verb=\lim= & lim inf & \verb=\liminf= & lim sup & \verb=\limsup= \\
ln & \verb=\ln= & log & \verb=\log= & lg & \verb=\lg= \\
max & \verb=\max= & min & \verb=\min= & sup & \verb=\sup= \\
inf & \verb=\inf= &  &  &  &  \\
ker & \verb=\ker= & det & \verb=\det= & exp & \verb=\exp=
\end{tabular}
\end{center}
\caption{Some Predefined Math Operators}
\label{tab:predefmathop}
\end{table}

In fact, operator \verb=\Nul= and \verb=\Span= are defined by xjtlumath as in the forms in the year 1 linear algebra textbook of XJTLU. Also, xjtlumath changes the default \verb=\vec= command in \LaTeX{} so that vectors appear in bold form rather than with a arrow above them.

Some operators, like \verb=\lim=, are designed to support taking limits. That is, in displayed mode, when one tries to give one of such operators a subscript using \verb=_=, the subscript will appear at the bottom of the operator.
\begin{parexammar}{.4\textandmarginlen}{
\[
\lim_{x\to 0} f(x)
\]
}
\begin{lstlisting}
\[
\lim_{x\to 0} f(x)
\]
\end{lstlisting}
\end{parexammar}

Delilah is able to explicitly control the limit style by using \verb=\limits= and \verb=\nolimits=. Note that these two commands can only be used after a operation that supports taking limits.
\begin{parexammar}{.4\textandmarginlen}{
$\lim\limits_{x \to 0}f(x)$
\[
\lim\nolimits_{x\to 0} f(x)
\]
}
\begin{lstlisting}
$\lim\limits_{x \to 0}f(x)$
\[
\lim\nolimits_{x\to 0} f(x)
\]
\end{lstlisting}
\end{parexammar}

\subsection{Delimiters}
Delilah already knows how to type basic delimiters. For parentheses and square brackets, plain text will do; and since the curly brackets are reserved by \LaTeX{}, Delilah needs to add a backslash before each of them.
\begin{parexammar}{.45\textandmarginlen}{
The range of a function may be expressed explicitly by its domain and itself: the range of $f: X \to Y$ is $f(X)$.

In some context the arguments of a function are enclosed by square brackets: $f[x]$.

Curly brackets are often used to show a set: $S := \{2,4,\cdots\}$.
}
\begin{lstlisting}
The range of a function may be expressed explicitly by its domain and itself: the range of $f: X \to Y$ is $f(X)$.

In some context the arguments of a function are enclosed by square brackets: $f[x]$.

Curly brackets are often used to show a set: $S := \{2,4,\cdots\}$.
\end{lstlisting}
\end{parexammar}

Yet in some conditions where the expression being enclosed has a different height, the result becomes unsatisfactory. \LaTeX{} provides a mechanism to enable the user to automatically or manually adjust the size of delimiters.

Adding \verb=\left= and \verb=\right= around a pair of delimiters automatically adjust their size to match the expression being enclosed. However, sometimes we want the delimiters to be a bit bigger or smaller, and that's when we need to manually adjust the size.
\begin{parexammar}{.5\textandmarginlen}{
\[
\left( \frac{1}{2} \right)
\Bigg( \bigg( \Big( \big( x\big) \Big) \bigg) \Bigg)
\]
}
\begin{lstlisting}
\[
\left(\frac{1}{2}\right) \Bigg( \bigg( \Big( \big( x\big) \Big) \bigg) \Bigg)
\]
\end{lstlisting}
\end{parexammar}

Delilah once mismatched a pair of delimiters, but she found that the height of the two symbols are still the same. She learns in addition that the dot `.' can be used to inform \LaTeX{} to insert nothing.
\begin{parexammar}{.5\textandmarginlen}{
A fraction enclosed left by a parenthesis, and right by a curly bracket.
\[
\left(\frac{a}{b}\right\}
\]
A system of equations
\[
\left\{
\begin{aligned}
x+y&=1\\
x-y&=2
\end{aligned}
\right.
\]
}
\begin{lstlisting}
A fraction enclosed left by a parenthesis, and right by a curly bracket.
\[
\left(\frac{a}{b}\right\}
\]
A system of equations
\[
\left\{
\begin{aligned}
x+y&=1\\
x-y&=2
\end{aligned}
\right.
\]
\end{lstlisting}
\end{parexammar}

Now Delilah knows how to properly handle delimiters, but the repetitive \verb=\left= and \verb=\right= really makes her sick. xjtlumath simplifies the work by providing these predefined groups of delimiters. The following examples show the delimiter groups defined by xjtlumath, as well as the explicit control of the size.
\begin{parexammar}{.5\textandmarginlen}{
These delimiters are defined by xjtlumath.
\[
\rbra{x},\ \sbra{x},\ \cbra{x},\ \abs{x},\ \floor{x},\ \ceiling{x}
\]
One can also change the delimiter resizer.
\[
\abs[\bigg]{x}
\]
}
\begin{lstlisting}
These delimiters are defined by xjtlumath.
\[
\rbra{x},\ \sbra{x},\ \cbra{x},\ \abs{x},\ \floor{x},\ \ceiling{x}
\]
One can also change the delimiter resizer.
\[
\abs[\bigg]{x}
\]
\end{lstlisting}
\end{parexammar}

\subsection{Symbols}
The standard \LaTeX{} font in math environments is neat and clean. Yet in some special occasions Delilah would like to change the font of certain symbols. For example, to represent some conventional sets, she uses the blackboard font.
\begin{parexammar}{.4\textandmarginlen}{
\[
\mathbb{R}\quad \mathbb{N}\quad \mathbb{Q}\quad \mathbb{Z}
\]
}
\begin{lstlisting}
\[
\mathbb{R}\quad \mathbb{N}\quad \mathbb{Q}\quad \mathbb{Z}
\]
\end{lstlisting}
\end{parexammar}

Writing \verb=\mathbb= every time is somehow irritating. For this purpose, xjtlumath defines shorthands for them.
\begin{parexammar}{.45\textandmarginlen}{
\[
\setr \quad \setq \quad \setz \quad \setn \quad \setnp
\]
}
\begin{lstlisting}
\[
\setr \quad \setq \quad \setz \quad \setn \quad \setnp
\]
\end{lstlisting}
\end{parexammar}

Other font controlling methods are like what we have talked about in subsection \ref{subsec:fonts}. For example, \verb=\mathrm= gives font in \textrm{Roman} family, and \verb=\mathbf= gives font in \textbf{bold} series.
\begin{parexammar}{.45\textandmarginlen}{
\[
\mathrm{Like normal text} \quad \mathbf{bold}
\]
}
\begin{lstlisting}
\[
\mathrm{Like normal text} \quad \mathbf{bold}
\]
\end{lstlisting}
\end{parexammar}


\section{Using custom cost function}

\subsection{Implementation}
A cost function that satisfies the specified conditions is shown equation \ref{eq:custom_cost_function}

\begin{equation}
\label{eq:custom_cost_function}
cost(h_{\theta}(x_i), y_i) = 100(1-y_i)h_{\theta}(x_i) + 100y_i(1-h_{\theta}(x_i))
\end{equation}

where,

\begin{equation}
\label{eq:logistic_regression}
h_{\theta}(X) = \frac{1}{e^{-\theta_0 - \theta_1 * X}}
\end{equation}

The minimization function J($\theta$) is given by equation \ref{eq:custom_minimization_function}

\begin{equation}
\label{eq:custom_minimization_function}
J(\theta) = \frac{100}{m} \sum_{i=1}^{m}(1-y_i)h_{\theta}(x_i) + y_i(1-h_{\theta}(x_i))
\end{equation}

The update rules for $\theta$s for the above minimization function are as follows.

\begin{equation}
\label{eq:theta_0_update}
\theta_0 = \theta_0 + \frac{100\alpha}{m}\sum_{i=1}^{m}h_\theta(x_i)(1-h_\theta(x_i))(1-2y)
\end{equation}

\begin{equation}
\label{eq:theta_1_update}
\theta_1 = \theta_1 + \frac{100\alpha}{m}\sum_{i=1}^{m}h_\theta(x_i)(1-h_\theta(x_i))(1-2y)x_{i1}
\end{equation}

\subsection{Observation}
The final values of $\theta$s for a run are as follows

% Theta 0 final value
\begin{equation}
\theta_0 = -13.915013230977546
\end{equation}

% Theta 1 final value
\begin{equation}
\theta_1 = 4.276707007556025
\end{equation}

The plot of decision curve for above $\theta$s is shown in figure \ref{fig:custom_cost_function}

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.4\textheight,keepaspectratio]{custom_cost_function_0_01.png}
  \caption{Sigmoid graph for custom cost function}
  \label{fig:custom_cost_function}
\end{figure}

If we overlap the curve obtained from logistic regression shown in figure \ref{fig:logistic_regression} and curve in figure \ref{fig:custom_cost_function}
the resulting figure is shown in figure in \ref{fig:compare_log_vs_custom}.

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.4\textheight,keepaspectratio]{compare_log_vs_custom.png}
  \caption{Comparison of logistic regression and custom cost function}
  \label{fig:compare_log_vs_custom}
\end{figure}

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.4\textheight,keepaspectratio]{custom_cost_function_0_0001.png}
  \caption{Improved curve for custom cost function}
  \label{fig:custom_cost_function_0_0001}
\end{figure}

\subsection{Conclusion}
As shown by figure \ref{fig:compare_log_vs_custom}, after running for the same numbers of iterations the boundary line obtained
from custom cost function does not clearly divides the data into two seperate regions as some of the data points lie in the boundary line.
The equations \ref{eq:theta_0_update} and \ref{eq:theta_1_update} shows that the learning rate is multiplied by a scalar factor of 100. Due which the
learning rate for $\theta$s becomes high. We can overcome this by further decreasing the learning rate to about 0.0001 as shown in figure \ref{fig:custom_cost_function_0_0001}

\subsection{Source Code}
\lstinputlisting[language=python]{task_4.py}


\section{Background}
\label{SEC:background}

We briefly review the relevant properties of cryptographic $(k, n)$-threshold
schemes.  Although the specific $(k,n)$-threshold scheme that is used
is not fundamental to our
work, we describe Shamir Secret Sharing \cite{shamir1979share}, which we use to develop
explicit examples within the text.

\emph{The $(k, n)$-threshold scheme}

Threshold schemes protect secret information (usually a key) by deriving $n$
different shares from this information.  A threshold scheme describes how any $k$
shares (from a set of n total shares) can be used to recover an original
secret.  The number of needed shares, $k$, is called the threshold.  If fewer than
$k$ shares are known, no information about the secret is provided.

Shamir Secret Sharing is an algorithm that describes how a secret is divided
into a set of $n$ shares.  If a threshold $k$ of shares are input (specified when the
secret is divided), the original secret can be reconstructed. To hide
a secret, Shamir Secret Sharing computes $k - 1$ random coefficients for a $k - 1$
degree polynomial $f(x)$ in a finite field (commonly GF-256 or GF-65536). The $k$th
term (commonly the constant term) contains the secret. To compute a share, a
value between 1 and the order of the field is chosen. The polynomial is
evaluated with $x$ equal to the share value, where the terms $x$ and $f(x)$ are used
as the share. To reconstruct the secret from at least $k$ shares, a party can
interpolate the values in the finite field to find the constant term (i.e., the
secret). In practice, interpolation is often computationally optimized so that
only the constant term is recovered.

Suppose that a secret, 235, is to be hidden so that it can only be
reconstructed if three shares are provided. Because the threshold is three, two
random terms are first generated (24 and 182) to build a GF-256 polynomial,
such as $f(x) = 24x^2 + 182x + 235$. Shares can then be generated by computing $x$
and $f(x)$, such as: (1, 92), (2, 148), (3, 37), (4, 69), etc. A party that has
at least three shares can interpolate to reconstruct the full polynomial of
$f(x)$ and thus, the secret (235).

It is possible to generate additional shares after recovering the secret, (e.g.,
if Lagrange interpolation is performed during reconstruction). This makes share
recovery slightly more computationally complex but also makes it possible (and
efficient) to generate additional shares simply by evaluating f(x) for the
specified share.  This means that all shares do not need to be created
initially --- they may instead be added (or recovered) after the secret has
been reconstructed.

In many cases, the secret will be larger than the size of the finite field.  A
large secret can be stored by breaking it into segments that are the size of
the finite field (often one or two bytes) and applying the above technique
separately to each segment.  The same share number, $x$, is typically used for
each share $f_i(x)$.  This simplifies --- and effectively hides --- the fact that a
secret has only a limited size.  An integrity check can be added to detect whether
an incorrect share has been provided.  As was previously described, when given a set
of any k distinct shares, whether valid or invalid, Shamir Secret Sharing will
produce a polynomial of the appropriate length. This means that if any share is
invalid, its resulting polynomial will be incorrect. To avoid this problem,
implementations of Shamir Secret Sharing typically store an integrity check by
appending the hash of the secret to the secret; this check detects if an
incorrect share has been provided.  When the shares are reconstructed, the
additional integrity check provides verification that the correct shares were
given.


\section{Number Representation}

By default the standard binary system can only represent unsigned integers.
With other number representation systems you can also represent signed numbers and floats.
This chapter describes different number systems.

\subsection{Sign \& Magnitude}
{\bf Sign \& Magnitude} is the (for humans) the simplest method of representing numbers.
You add one bit at the most significant bit (the `sign' bit).
This bit defines if the rest of the bitstring is positive or negative.
A \(0\) indicates a positive number, a \(1\) a negative one.
Applying direct operations to these numbers does not work, for example \(5 + -5\).
When you do this with Sign \& Magnitude you'll end up with \(2\).
This is obviously not correct.
Also you end up with two zero's, \(0000\) and \(1000\).

\begin{table}[h]
	\centering
	\begin{tabular}{c | c | c}
		Decimal & Binary    & S \& M    \\
		\hline
		72      & 0100 1000 & 0100 1000 \\
		-57     & --        & 1011 1001 \\
		8       & 0000 1000 & 0000 1000 \\
		-8      & --        & 1000 1000 \\
		-43.75  & --        & --        \\
	\end{tabular}
	\caption{Sign \& Magnitude}
\end{table}

\subsection{One's Complement}
{\bf One's complement} is the same as Sign \& Magnitude,
but when the sign is positive you need to take the complement of the number.
It's a lot better for computer to work with, but you still cannot directly do operations to it.
If you do attempt to do calculations to it, you will get an offset of \(1\).

\begin{table}[h]
	\centering
	\begin{tabular}{c | c | c}
		Decimal & Binary    & One's Complement \\
		\hline
		72      & 0100 1000 & 0100 1000        \\
		-57     & --        & 1100 0110        \\
		8       & 0000 1000 & 0000 1000        \\
		-8      & --        & 1111 0111        \\
		-43.75  & --        & --               \\
	\end{tabular}
	\caption{One's Complement}
\end{table}

\subsection{Two's Complement}
{\bf Two's complement} is the same as one's complement.
The difference is that you'll add 1 to the result when the number is negative.
This also makes sure there is only one zero.

\begin{table}[h]
	\centering
	\begin{tabular}{c | c | c}
		Decimal & Binary    & Two's Complement \\
		\hline
		72      & 0100 1000 & 0100 1000        \\
		-57     & --        & 1100 0111        \\
		8       & 0000 1000 & 0000 1000        \\
		-8      & --        & 1111 1000        \\
		-43.75  & --        & --               \\
	\end{tabular}
	\caption{Two's Complement}
\end{table}

\subsection{Excess-N}
{\bf Excess-N} is great for really large positive or negative numbers.
The \(N\) in Excess-N stands for the offset used.
For example, if you want to represent \(5\) you can do \(5 + N\).
This gives \(5 + 3 = 8\) in Excess-3.
To get the \(5\) back you can just substract the \(N\) to the number.
For example \(8 - 3 = 5\).

\begin{table}[h]
	\centering
	\begin{tabular}{c | c | c | c}
		Decimal & Binary    & Excess-128 & Excess-57 \\
		\hline
		72      & 0100 1000 & 1100 1000  & 1000 0001 \\
		-57     & --        & 0100 0111  & 0000 0000 \\
		8       & 0000 1000 & 1000 1000  & 0100 0001 \\
		-8      & --        & 0111 1000  & 0011 0001 \\
		-43.75  & --        & --         & --        \\
	\end{tabular}
	\caption{Excess-N}
\end{table}

\subsection{IEEE-754}
{\bf IEEE-754} is an way of representing floats.
It has the following parts:

\begin{table}[h]
	\centering
	\begin{tabular}{l | c | c| l}
		Name     & Bits (32-bit) & Bits (64-bit) & Description                    \\
		\hline
		Sign     & 1             & 1             & 0 for positive, 1 for negative \\
		Exponent & 8             & 11            & Exponent for the Mantissa      \\
		Mantissa & 23            & 52            & The actual number, encodeds    \\
	\end{tabular}
	\caption{Parts of IEEE-754}
\end{table}

To convert a number to IEEE-754 you first have to convert the number to binary.
Let's take the number \(-43.75\).
We can already see that the sign bit should be 1, as the number is negative.
Let's first convert the interger first part, this gives \(0010 1011\).
Then convert the decimal part seperately and add it.
This gives \(0010 1011 . 1100\).
Now move the point to the first \(1\) and remember the places you've moved.
This gives \(001.0 1011 1100\), with 5 places moved.
The 5 is your exponent, but it needs a bias to deal with negative numbers.
With 32-bit the bias is \(127\), in 64-bit \(1023\).
We are using 32-bit, so apply a bias of \(127\).
This gives an exponent of \(exp = 5 + 127 = 132\) or \(1000 0100\).
For the mantissa part you copy all digits after the dot and right-pad with 0's till 23 bits.
This gives you \(010 1111 0000 0000 0000 0000\).
The complete format IEEE-754 gives \(1100 0010 0010 1111 0000 0000 0000 0000\)

\begin{table}[h]
	\centering
	\begin{tabular}{c | c | c | c | c}
		Decimal & Binary    & Sign & Exponent  & Mantissa                     \\
		\hline
		72      & 0100 1000 & 0    & 1000 0101 & 001 0000 0000 0000 0000 0000 \\
		-57     & --        & 1    & 1000 0100 & 110 0100 0000 0000 0000 0000 \\
		8       & 0000 1000 & 0    & 1000 0010 & 000 0000 0000 0000 0000 0000 \\
		-8      & --        & 1    & 1000 0010 & 000 0000 0000 0000 0000 0000 \\
		-43.75  & --        & 1    & 1000 0100 & 010 1111 0000 0000 0000 0000 \\
	\end{tabular}
	\caption{IEEE-754}
\end{table}


\lab{QR Decomposition using Householder reflectors}{QR Decomposition using Householder reflectors}
\objective{Use orthonormal transformations to perform QR decomposition.}
\label{lab:Canonical_Transformations}

\section*{Orthonormal transformations}
Recall that a matrix $Q$ is \emph{unitary} if $Q^\mathsf{H} Q = I$ or for real matrices,
$Q^T Q = I$.
For the real case we say that such a matrix is \emph{orthonormal}.

Unitary transformations have the very desirable property of being numerically stable.
The number $\kappa(A) = \norm{A} \norm{A^{-1}}$ is called the \emph{condition number} of $A$.
We'll discuss condition number more in another lab; for now, all you need to know is that if $\kappa(A)$ is small, then calculations involving $A$ are less susceptible to numerical errors.
For the induced 2-norm, it holds that $\norm{Q}=1$ when $Q$ is unitary.
The Cauchy-Schwarz inequality $\norm{AB} \leq \norm{A} \norm{B}$ also holds for this norm, and
so it follows that $\kappa(A) = \norm{A} \norm{A^{-1}} \geq \norm{A A^{-1}} = \norm{I} = 1$.
Note that if $Q$ is unitary, $Q^{-1} = Q^\mathsf{H}$ and $Q^\mathsf{H}$ is also unitary, so $\kappa(Q) = \norm{Q} \norm{Q^\mathsf{H}} = 1$.
This means that orthonormal matrices have the smallest possible condition number.

Any orthogonal matrix $Q$ can be described as a reflection, a rotation, or some combination of the two.
If $det(Q) = 1$, then $Q$ is a rotation.
If $det(Q) = -1$, then $Q$  is a reflection or a composition of a reflection and a rotation.
Let's explore these two types of unitary transformations and some of their applications.
We will focus on the real case to simplify matters.

\section*{Householder reflections}
A Householder reflection is a linear transformation $P: \mathbb{R}^n \rightarrow \mathbb{R}^n$ that reflects a vector $x$ about a hyperplane.
See figure \ref{fig:Householder_reflector}.
Recall that a hyperplane can be defined by a unit vector $v$ which is orthogonal to the hyperplane.
As shown in figure \ref{fig:Householder_reflector}, $x - \langle v,x \rangle v$ is the projection of $x$ onto the hyperplane orthogonal to $v$.
However, to reflect \emph{across} the hyperplane, we must move twice as far; that is, $Px = x - 2\langle v,x \rangle v$.
This can be written $Px = x - 2v(v^\mathsf{H} x)$, so $P$ has matrix representation $P = I - 2v v^\mathsf{H}$.
Note that $P^\mathsf{H} P = I$; thus $P$ is orthonormal.

\begin{figure}
\includegraphics[width= \textwidth]{fig1}
\caption{Householder reflector}
\label{fig:Householder_reflector}
\end{figure}

\subsection*{Householder triangularization}
Consider the problem of computing the $QR$ decomposition of a matrix $A$.
You've already learned the Gram-Schmidt and the Modified Gram-Schmidt algorithms for this problem.
The $QR$ decomposition can also be computed by applying a series of Householder reflections.
Gram-Schmidt and Modified Gram-Schmidt make $A$ \emph{orthonormal} using a series of transformations stored in an \emph{upper triangular} matrix.
On the other hand, we can use Householder reflections to make $A$ \emph{triangular} by a series of \emph{orthonormal} transformations.

Let's demonstrate this method on a $4 \times 3$ matrix $A$.
First we find an orthonormal transformation $Q_1$ that maps the first column of A into the span of $e_1$
(where $e_1$ is the vector where the first element is one and the remainder of the elements are zeros).

\def\mc#1{\multicolumn{1}{c|}{#1}}
\begin{equation*}
\begin{pmatrix}
* & * & * \\
* & * & * \\
* & * & * \\
* & * & *
\end{pmatrix}
\underrightarrow{Q_1}
\begin{pmatrix}

* & * & * & \\ \cline{2-3}
\mc{0} & * & \mc{*}& \\
\mc{0} & * & \mc{*} & \\
\mc{0}& * & \mc{*} & \\ \cline{2-3}
\end{pmatrix}
\end{equation*}
Let $A_2$ be the boxed submatrix of $A$.
Now find an orthonormal transformation $Q_2$ that maps the first column of $A_2$ into the span of $e_2$.

\begin{equation*}
\begin{pmatrix}
* & * \\
* & * \\
* & *
\end{pmatrix}
\underrightarrow{Q_2}
\begin{pmatrix}
* & * \\
0 & * \\
0 & *
\end{pmatrix}
\end{equation*}
Similarly, $ \begin{pmatrix} * \\ * \end{pmatrix} \underrightarrow{Q_3} \begin{pmatrix} * \\ 0 \end{pmatrix} $.
(Technically $Q_2$ and $Q_3$ act on the whole matrix and not just on the submatrices, so that $Q_i: \mathbb{R}^n \rightarrow \mathbb{R}^n$ for all $i$.
$Q_2$ leaves the first row and the first column alone, and $Q_3$ leaves the first two rows and the first two columns alone.)
Then $Q_3 Q_2 Q_1 A =$

\begin{equation*}
Q_3 Q_2 Q_1
\begin{pmatrix}
* & * & * \\
* & * & * \\
* & * & * \\
* & * & *
\end{pmatrix}
= Q_3 Q_2
\begin{pmatrix}
* & * & * \\
0 & * & * \\
0 & * & * \\
0 & * & *
\end{pmatrix}
= Q_3
\begin{pmatrix}
* & * & * \\
0 & * & * \\
0 & 0 & * \\
0 & 0 & *
\end{pmatrix}
=
\begin{pmatrix}
* & * & * \\
0 & * & * \\
0 & 0 & * \\
0 & 0 & 0
\end{pmatrix}
\end{equation*}

We've accomplished our goal, which was to triangularize $A$ using orthonormal transformations.
But how do we find the $Q_i$ that do what we want? The answer lies in using Householder reflections.

To find $Q_1$, we first identify an appropriate hyperplane to reflect $x$ into the span of $e_1$.
It turns out there are two hyperplanes that will work, as shown in figure \ref{fig:two reflectors}.
(In the complex case, there are infinitely many such hyperplanes.)
Between the two, the one that reflects $x$ further will be more numerically stable.
This is the hyperplane perpendicular to $v = sign(x_1)\norm{x}_2 e_1 + x$.

To see how this works, let $x$ be the first column of the submatrix that we want to project onto the span of $e_1$.
In order for this to be a unitary operation, this will need to preserve the norm of $x$.
This means that $\left( I - 2 v v^\mathsf{H} \right) x = \pm \norm{x} e_1$, or, in other words,

\[ 2 v v^\mathsf{H} x =
\begin{pmatrix}
x_1 \pm \norm{x} \\
x_2 \\
x_3 \\
\vdots \\
x_n
\end{pmatrix}\]

Let $u$ be the vector on the right hand side of this expression.
It can be shown that the vector  $\frac{u}{\norm{u}}$ is the proper choice for $v$.
%We will show that the vector $\frac{u}{\norm{u}}$ is the proper choice for $v$.
%Notice that:
%
%\[\norm{u}^2 = \norm{x}^2 \pm 2 \norm{x} x_1 + x_1^2 + x_2 + \dots + x_n^2 = 2 \norm{x}^2 \pm 2 \norm{x} x_1 \]
%
%and that
%
%\[\norm{x}^2 \pm \norm{x} x_1 = u^\mathsf{H} x \]
%
%So we have
%
%\begin{align*}
%2 v v^\mathsf{H} x &= 2 u \frac{\norm{x}^2 \pm x_1 \norm{x}}{\norm{u}^2} \\
%		&= 2 u \frac{u^\mathsf{H} x}{\norm{u}^2} \\
%		&= 2 \frac{u}{\norm{u}} \left( \frac{u}{\norm{u}} \right)^\mathsf{H} x
%\end{align*}
%
%So $\frac{u}{\norm{u}}$ is a proper choice of $v$ that will project $x$ into the span of $e_1$.

This whole process is summarized in Algorithm \ref{Alg:Householder}.

\begin{figure}
\includegraphics[width= \textwidth]{fig2}
\caption{two reflectors}
\label{fig:two reflectors}
\end{figure}

\begin{algorithm}
\caption{Householder triangularization}
\label{Alg:Householder}
\begin{algorithmic}[1]
\Procedure{Householder}{$A$}
\State $m, n \gets \text{shape} \left( A \right)$
\State $R \gets \text{copy} \left( A \right)$
\State $Q \gets I_m$
\For{$0 \leq k < n-1$}
    \State $v_k \gets \text{copy} \left( R_{k:,k} \right)$
    \State $v_{k_0} \gets v_{k_0} + \text{sign} \left( v_{k_0} \right) \norm{v_k}$
    \State $v_k \gets v_k / \norm{v_k}$
    \State $R_{k:,k:} \gets R_{k:,k:} - 2 v_k \left( v_k^\mathsf{H} R_{k:,k:} \right)$
    \State $Q_{k:} \gets Q_{k:} - 2 v_k \left( v_k^\mathsf{H} Q_{k:} \right)$
\EndFor
\State \pseudoli{return} $Q^\mathsf{H}, R$
\EndProcedure
\end{algorithmic}
\end{algorithm}

To see how we are operating on the matrices $A$ and $Q$, consider the way each orthonormal transformation defined by the $v_k$ operates blockwise on each matrix.
The matrix form of each operation on $A$ and $Q$ can be represented in block form like this:

\[
\begin{pmatrix}
I & 0 \\
0 & I - 2 v_k v_k^\mathsf{H}
\end{pmatrix}
\]

Notice that a block matrix of this form operates only on entries that lie in the rows from $k$ onward.
Consider what happens when we left-multiply a $m \times n$ matrix by a block matrix of this form.
We obtain the following:

\[
\begin{pmatrix}
I & 0 \\
0 & I - 2 v_k v_k^\mathsf{H}
\end{pmatrix}
\cdot
\begin{pmatrix}
A[:k,:k] & A[:k,k:] \\
A[k:,:k] & A[k:,k:]
\end{pmatrix}
=
\begin{pmatrix}
A[:k,:k] & A[:k,k:] \\
A[k:,:k] - 2 v_k v_k^\mathsf{H} A[k:,:k] & A[k:,k:] - 2 v_k v_k^\mathsf{H} A[k:,k:]
\end{pmatrix}
\]

And, when we consider right multiplication by the same block matrix, we see that it fixes the first $k-1$ columns as below.

\[
\begin{pmatrix}
A[:k,:k] & A[:k,k:] \\
A[k:,:k] & A[k:,k:]
\end{pmatrix}
\cdot
\begin{pmatrix}
I & 0 \\
0 & I - 2 v_k v_k^\mathsf{H}
\end{pmatrix}
=
\begin{pmatrix}
A[:k,:k] & A[:k,k:] - 2  A[:k,k:] v_k v_k^\mathsf{H} \\
A[k:,:k] & A[k:,k:] - 2 A[k:,k:] v_k v_k^\mathsf{H}
\end{pmatrix}
\]

When we are iterating through the columns of $R$ and zeroing out the entries below the main diagonal we are able to safely ignore all the entries that lie in columns we have already processed because they are already zero.

This algorithm returns orthonormal $Q$ and upper triangular $R$ satisfying $A = QR$.
Notice that we did not explicitly construct each orthonormal reflector matrix.
We applied the changes we needed to each portion of the array that needed to be changed.
Doing the operations in this way allows us to avoid unnecessarily increasing the computational complexity of the algorithm.
A few other clever optimizations can still be applied, but they will not change the overall complexity of the algorithm.

%It should now be clear how it was that we computed $R$ using this algorithm.
%$Q$ is computed in much the same way.
%Since each of the orthonormal operations is self-inverse (i.e. idempotent), $Q$ can be computed by applying the these operations to the identity in reverse order.
%In other words, you could make an identity matrix and then for $k$ such that $n-2 \geq k > -1$ , do $I[k:,k:] -= 2 v_k v_k^\mathsf{H}$
%In our computation, it may be more convenient to simply apply the operations to an identity matrix as we go, just like we are doing to $R$, then take the transpose at the end to invert $Q$.
%This way we do not have to store the $v_k$ as we go.
%There is one key difference, when applying these operations to $Q$ we cannot ignore columns we have already processed because they are not necessarily zero.
%It is interesting to note that we can use the $v_k$ to behave like $Q$ or $Q^{-1}$ depending on the order in which we apply them.
%Such an approach does not require the computation of $Q$ or $Q^{-1}$ at all.

Another important thing to notice is that an outer product is needed to compute
$v_k \left( v_k^\mathsf{H} A[k:,k:] \right)$, not an inner product.
Make sure that you account for this when you write the code to run this algorithm.
You can either make the vectors $v_k$ column vectors (two dimensional with a single column) instead of just one-dimensional arrays, or you can use the built in function \li{np.outer} in the appropriate place.

\begin{problem}
\label{prob:HouseholderQR}
Write a function \li{householder} that accepts an array $A$ as input, and performs
the algorithm described above to compute the QR decomposition of $A$. Return the
matrices $Q$ and $R$.

It is simple to check that your code works: multiply the two output matrices
of your function, and check that the result matches the original input matrix.
\end{problem}

\subsection*{Stability of the Householder QR algorithm}
We will now examine the stability of the Householder QR algorithm.
We will use SciPy's built in QR factorization which uses Householder reflections internally.

Try the following in Python.

\begin{lstlisting}
>>> import numpy as np
>>> from numpy.random import rand
>>> from scipy import linalg as la
>>> Q, X = la.qr(rand(500,500)) # create a random orthonormal matrix:
>>> R = np.triu(rand(500,500)) # create a random upper triangular matrix
>>> A = np.dot(Q,R) # Q and R are the exact QR decomposition of A
>>> Q1, R1 = la.qr(A) # compute QR decomposition of A
>>> la.norm(Q1-Q)/la.norm(Q) # check error in Q
0.282842955725
>>> la.norm(R1-R)/la.norm(R) # check error in R
0.0428922016647
\end{lstlisting}
This is terrible!
This algorithm works in $16$ decimal points of precision, but $Q_1$ and $R_1$ are only accurate to $0$ and $1$ decimal points, respectively.
We've lost $16$ decimal points of precision!

Don't lose hope.
Check how close the product $Q_1 R_1$ is to $A$.
\begin{lstlisting}
>>> A1 = Q1.dot(R1)
>>> np.absolute(A1 - A).max()
3.9968028886505635e-15
\end{lstlisting}
We've now recovered $15$ digits of accuracy.
Considering the error relative to the norm of $A$ (using the 2-norm for matrices), we see that this relative error is even smaller.
\begin{lstlisting}
>>> la.norm(A1 - A, ord=2) / la.norm(A, ord=2)
8.8655568331889288e-16
\end{lstlisting}
The errors in $Q_1$ and $R_1$ were somehow ``correlated," so that they canceled out in the product.
The errors in $Q_1$ and $R_1$ are called \emph{forward errors}.
The error in $A_1$ is the \emph{backward error}.

In fact, the large errors in \li{Q1} and \li{R1} were not because the algorithm was bad, it was because $A$ was poorly conditioned.
The condition number for randomly generated upper triangular matrices generally very high, and this was the case here.
This has, in turn, made the condition number of $A$ extremely large as well.
Try the following to compute the condition numbers of $A$.
In this case the condition numbers of $A$ and $R$ are computed to be different, though, in theory, they should be exactly the same.
\begin{lstlisting}
>>> from numpy.linalg import cond
>>> cond(A)
4.1426075832870472e+18
>>> cond(R)
3.1767577244363792e+19
\end{lstlisting}

Householder QR factorization is more numerically stable than Gram-Schmidt or even Modified Gram-Schmidt (MGS).
However, MGS is still useful for some types of iterative methods, because it finds the orthonormal basis one vector at a time instead of all at once (for an example see Lab \ref{lab:EigSolve}).

\subsection*{Upper Hessenberg Form}
An upper Hessenberg matrix is a square matrix with zeros below the first subdiagonal.
Every  $n \times n$ matrix $A$ can be written $A = Q^THQ$ where $Q$ is orthonormal and $H$ is an upper Hessenberg matrix, called the Hessenberg form of $A$.

The Hessenberg decomposition can be computed using Householder reflections, in a process very similar to Householder triangularization.
Let's demonstrate this process on a $5 \times 5$ matrix $A$.
Note that $A=Q^THQ$ is equivalent to $QAQ^T = H$; thus our strategy is to multiply $A$ on the right and left by a series of orthonormal matrices until it is in Hessenberg form.
If we try the same $Q_1$ as in the first step of the Householder algorithm, then with $Q_1 A$ we introduce zeros in the first column of $A$.
However, since we now have to multiply $Q_1 A$ on the left by $Q_1^T$, all those zeros are destroyed, as demonstrated below.
In order to zero out the entire first column we chose $Q_1$ so that it does not fix the first row.
When we apply the same operation on the right, this ruins the column that we just zeroed out.
(Although this process may seem futile now, it actually does tend to decrease the size of the subdiagonal entries.
If we repeat it over and over again, the subdiagonal entries will often converge to zero.
That's the idea behind the $QR$ algorithm in Lab \ref{lab:EigSolve}.)
\[
\begin{array}{ccccc}
\begin{pmatrix}
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & *
\end{pmatrix}
&\underrightarrow{Q_1 \cdot }&
\begin{pmatrix}
* & * & * & * & * \\
0 & * & * & * & * \\
0 & * & * & * & * \\
0 & * & * & * & * \\
0 & * & * & * & *
\end{pmatrix}
&\underrightarrow{\cdot Q_1^T }&
\begin{pmatrix}
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & *
\end{pmatrix}
\\
A & & Q_1A & & Q_1 A Q_1^T
  \end{array}
\]
Instead, let's try starting with a different $Q_1$ that leaves the \emph{first} row alone and reflects the \emph{rest} of the rows into the span of $e_2$. This means that $Q_1^T$ leaves the first column alone.
\[
\begin{array}{ccccc}
\begin{pmatrix}
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & * \\
* & * & * & * & *
\end{pmatrix}
&\underrightarrow{Q_1 \cdot }&
\begin{pmatrix}
* & * & * & * & * \\
* & * & * & * & * \\
0 & * & * & * & * \\
0 & * & * & * & * \\
0 & * & * & * & *
\end{pmatrix}
&\underrightarrow{\cdot Q_1^T }&
\begin{pmatrix}
* & * & * & * & * \\
* & * & * & * & * \\
0 & * & * & * & * \\
0 & * & * & * & * \\
0 & * & * & * & *
\end{pmatrix}
\\
A & & Q_1A & & Q_1 A Q_1^T
  \end{array}
\]
We now iterate through the matrix until we obtain
\begin{equation*}
Q_3 Q_2 Q_1 A Q_1^T Q_2 ^T Q_3^T =
\begin{pmatrix}
* & * & * & * & * \\
* & * & * & * & * \\
0 & * & * & * & * \\
0 & 0 & * & * & * \\
0 & 0 & 0 & * & *
\end{pmatrix}
\end{equation*}

This is even more convenient when we are working with Hermitian matrices.
In that case, the matrices applied on the left zero out everything below the first subdiagonal and the matrices applied on the right zero out everything above the first superdiagonal, leaving us with a tridiagonal matrix.
There are remarkably efficient ways to solve systems involving tridiagonal matrices, so this is especially convenient.

The pseudocode for computation of the Hessenberg form of a matrix is shown in Algorithm \ref{Alg:Hessenberg}.
The exact inner workings of this algorithm are similar to the inner workings of Algorithm \ref{Alg:Householder}.

\begin{algorithm}
\caption{Reduction to Hessenberg Form}
\label{Alg:Hessenberg}
\begin{algorithmic}[1]
\Procedure{Hessenberg}{$G,u,l,p$}
\State $m, n \gets \text{shape}(A)$
\State $H \gets \text{copy}(A)$
\State $Q \gets I_m$
\For{$0 \leq k < n-2$}
    \State $v_k \gets H_{k+1:, k}$
    \State $v_{k_0} \gets v_{k_0} + \text{sign}(v_{k_0}) \norm{v_k}$
    \State $v_k \gets v_k/norm{v_k}$
    \State $H_{k+1:,k:} \gets H_{k+1:,k:} - 2v_k(v_k^\mathsf{H} H_{k+1:,k:})$
    \State $H_{:,k+1:} \gets H_{:,k+1:} - 2(H_{:,k+1:} v_k) v_k^\mathsf{H}$
    \State $Q_{k+1:} \gets Q_{k+1:} - 2v_k(v_k^\mathsf{H} Q_{k+1:})$
\EndFor
\State \pseudoli{return} $Q, R$
\EndProcedure
\end{algorithmic}
\end{algorithm}

\begin{problem}
\label{prob:hessenberg}
Write a function \li{hessenberg} that computes the Hessenberg form of a real-valued
input matrix $A$. The function should return $Q$ and $H$ satisfying $A = Q^THQ$,
where $Q$ is orthonormal and $H$ has zeros below the first subdiagonal.

The code for this algorithm will be fairly similar to the code for the QR factorization using Householder reflections.
This factorization technique will be used later on in Lab \ref{lab:EigSolve}.
Notice what happens when you compute the Hessenberg factorization of a Hermitian matrix.
\end{problem}

%Sources: http://www.cs.unc.edu/~krishnas/eigen/node5.html
% http://en.wikipedia.org/wiki/Givens_rotation
%http://en.wikipedia.org/wiki/QR_decomposition
%	Note the Operation count: Householder is 2/3 n^3, MGS is 2 n^3
%http://en.wikipedia.org/wiki/QR_algorithm
%Applied Numerical methods using MATLAB by Yang has some code written for this
%http://www.math.kent.edu/~reichel/courses/intr.num.comp.2/lecture21/evmeth.pdf
%	These are eigenvalue algorithms explained carefully
%http://en.wikipedia.org/wiki/Householder_transformation
%Numerical Linear Algebra, by Lloyd N. Trefethen and David Bau III, Chapters 10 and 16


\section{Gaussians}
\smallskip \hrule height 2pt \smallskip

Properties of Gaussians:
\begin{itemize}
	\item Affine transformation (multiplying by a scalar and adding a constant) are Gaussian.
		If X $\sim$ N($\mu$,$\sigma^2$) and Y = aX + b, then Y $\sim$ N($a\mu+b, a^2\sigma^2$)
  	\item Sum of Gaussians is Gaussian.
			If X $\sim$ N($\mu_X, \sigma^2_X$),
			Y $\sim$ N($\mu_Y, \sigma^2_Y$),
			and Z = X+Y, then
			Z $\sim$ N($\mu_X+\mu_Y, \sigma_X^2 +\sigma_Y^2$)
	\item  Easy to differentiate.
\end{itemize}

Learn a Gaussian: $P(x | \mu, \sigma) = \frac{1}{\sigma \sqrt{2 \pi}}e^\frac{-(x-\mu)^2}{2\sigma^2}$. \hfill \\
MLE for Gaussian: Prob of i.i.d. samples D = $\{x_1, \dots, x_N\}$:  \hfill \\
$\displaystyle  P(D|\mu, \sigma) = ( \frac{1}{\sigma \sqrt{2 \pi}})^N \prod_{i=1}^N e^\frac{-(x_i-\mu)^2}{2\sigma^2}$.   \hfill \\
Note: it is \underline{not} $P(\mu, \sigma | D)$, like I thought in class.  \hfill \\
Find $\mu_{MLE}$, $\sigma_{MLE} = \argmax_{\mu, \sigma} P(D | \mu, \sigma)$.  \hfill \\

Log-likelihood:  $ \displaystyle \ln P(D | \mu, \sigma) = \ln[\mbox{thing above}] = -N \ln \sigma \sqrt{2\pi} - \sum_{i=1}^N \frac{(x_i - \mu)^2}{2\sigma^2}$.  \hfill \\
Differentiate w.r.t. $\mu$ and set = 0.  End up with $ \displaystyle \widehat{\mu} = \frac{1}{N} \sum_{i=1}^N x_i$.  \hfill \\
Differentiate w.r.t. $\sigma$ and set = 0.  End up with $ \displaystyle \widehat{\sigma}^2_{MLE} = \frac{1}{N} \sum_{i=1}^N (x_i-\widehat{\mu})^2$.  \hfill \\
But actually, that leads to a biased estimate, so people actually use  $ \displaystyle \widehat{\sigma}^2_{unbiased} = \frac{1}{N-1} \sum_{i=1}^N (x_i-\widehat{\mu})^2$  \hfill \\

The conjugate priors: mean: use Gaussian prior:  $ \displaystyle  P(\mu | \nu, \lambda) = \frac{1}{\lambda \sqrt{2 \pi}}e^\frac{-(\mu - \nu)^2}{2\sigma^2} $.  (Instead of $\sigma$, use $\lambda$ and replace the $(x-\mu)^2$ with $(\mu - \nu)^2$).  \hfill \\
For variance: use Wishard Distribution:


\section*{Exercises}

\begin{ex}
  For each of the following subspaces of $\R^4$, find a basis and
  determine the dimension.
  \begin{enumerate}
  \item $V_1 = \sspan\set{
      \begin{mymatrix}{r} 2 \\ 1 \\ 1 \\ 1 \end{mymatrix},~
      \begin{mymatrix}{r} -1 \\ 0 \\ -1 \\ -1 \end{mymatrix},~
      \begin{mymatrix}{r} 5 \\ 2 \\ 3 \\ 3 \end{mymatrix},~
      \begin{mymatrix}{r} -1 \\ 1 \\ -2 \\ -2 \end{mymatrix}
    }$.
  \item $V_2 = \sspan\set{
      \begin{mymatrix}{r} 0 \\ 1 \\ 1 \\ -1 \end{mymatrix},~
      \begin{mymatrix}{r} -1 \\ -1 \\ -2 \\ 2 \end{mymatrix},~
      \begin{mymatrix}{r} 2 \\ 3 \\ 5 \\ -5 \end{mymatrix},~
      \begin{mymatrix}{r} 0 \\ 1 \\ 2 \\ -2 \end{mymatrix}
    }$.
  \item $V_3 = \sspan\set{
      \begin{mymatrix}{r} -2 \\ 1 \\ 1 \\ -3 \end{mymatrix},~
      \begin{mymatrix}{r} -9 \\ 4 \\ 3 \\ -9 \end{mymatrix},~
      \begin{mymatrix}{r} -33 \\ 15 \\ 12 \\ -36 \end{mymatrix},~
      \begin{mymatrix}{r} -22 \\ 10 \\ 8 \\ -24 \end{mymatrix}
    }$.
  \item $V_4 = \sspan\set{
      \begin{mymatrix}{r} -1 \\ 1 \\ -1 \\ -2 \end{mymatrix},~
      \begin{mymatrix}{r} -4 \\ 3 \\ -2 \\ -4 \end{mymatrix},~
      \begin{mymatrix}{r} -3 \\ 2 \\ -1 \\ -2 \end{mymatrix},~
      \begin{mymatrix}{r} -1 \\ 1 \\ -2 \\ -4 \end{mymatrix},~
      \begin{mymatrix}{r} -7 \\ 5 \\ -3 \\ -6 \end{mymatrix}
    }$.
  \end{enumerate}
\end{ex}

\begin{ex}
  Find a basis and the dimension of each of the following subspaces of
  $\R^n$.
  \begin{enumerate}
  \item $S_1 =
    \set{\left.\begin{mymatrix}{c}
          4u+v-5w \\
          12u+6v-6w \\
          4u+4v+4w
        \end{mymatrix} ~\right\vert~u,v,w\in \R}.$
  \item $S_2 =
    \set{\left.\begin{mymatrix}{c}
          2u+6v+7w \\
          -3u-9v-12w \\
          2u+6v+6w \\
          u+3v+3w
        \end{mymatrix} ~\right\vert~u,v,w\in \R}.$
  \item $S_3 =
    \set{\left.\begin{mymatrix}{c}
          2u+v \\
          6v-3u+3w \\
          3v-6u+3w
        \end{mymatrix} ~\right\vert~u,v,w\in \R}.$
  \end{enumerate}
  % \begin{sol}
  % \end{sol}
\end{ex}

\begin{ex}
  Find a basis and the dimension of each of the following subspaces of
  $\R^n$.
  \begin{enumerate}
  \item $W_1 =
    \set{\left.\begin{mymatrix}{c} u \\ v \\ w \end{mymatrix}
        ~\right\vert~
      \mbox{$u+v=0$ and $u-2w=0$}}.$
  \item $W_2 =
    \set{\left.\begin{mymatrix}{c} u \\ v \\ w \end{mymatrix}
        ~\right\vert~
      \mbox{$u+v+w=0$}}.$
  \item $S =
    \set{\left.\begin{mymatrix}{c} u \\ v \\ w \\ x \end{mymatrix}
        ~\right\vert~
      \mbox{$u+v=w+x$ and $u+w=v+x$}}.$
  \end{enumerate}
  % \begin{sol}
  % \end{sol}
\end{ex}

\begin{ex}
  Find the vector $\vect{v}$ that has coordinates
  \begin{equation*}
    \coord{\vect{v}}_B = \begin{mymatrix}{r} 2 \\ 1 \\ -3 \end{mymatrix}
  \end{equation*}
  with respect to the basis $B=\set{\vect{u}_1,\vect{u}_2,\vect{u}_3}$
  of\/ $\R^3$, where
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{r} 2 \\ 4 \\ 1 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} 1 \\ -1 \\ 0 \end{mymatrix},\quad
    \mbox{and}\quad
    \vect{u}_3 = \begin{mymatrix}{r} -1 \\ 0 \\ 3\end{mymatrix}.
  \end{equation*}
\end{ex}

\begin{ex}
  Find the coordinates of each of $\vect{v}$, $\vect{w}$ with respect
  to the basis $B=\set{\vect{u}_1,\vect{u}_2,\vect{u}_3}$, where
  \begin{equation*}
    \vect{v} = \begin{mymatrix}{r} 4 \\ 3 \\ 8 \end{mymatrix},\quad
    \vect{w} = \begin{mymatrix}{r} -1 \\ -1 \\ 3 \end{mymatrix},\quad
    \vect{u}_1 = \begin{mymatrix}{r} 1 \\ 0 \\ 3 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} 0 \\ 1 \\ 1 \end{mymatrix},\quad
    \vect{u}_3 = \begin{mymatrix}{r} 2 \\ 2 \\ 1 \end{mymatrix}.
  \end{equation*}
\end{ex}

\begin{ex}
  Extend $\set{\vect{u}_1,\vect{u}_2}$ to a basis of $\R^3$, where
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{r} 3 \\ 3 \\ -6 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} 0 \\ -1 \\ 2 \end{mymatrix}.
  \end{equation*}
\end{ex}

\begin{ex}
  Let
  \begin{equation*}
    V ~=~ \set{\left.
        \begin{mymatrix}{c} x \\ y \\ z \\ w \end{mymatrix}
        ~\right\vert~
      x+y+z+2w = 0
    }.
  \end{equation*}
  Note that $\vect{u}_1,\vect{u}_2\in V$, where
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{c} 1 \\ 1 \\ 0 \\ -1 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{c} -2 \\ -2 \\ -2 \\ 3 \end{mymatrix}.
  \end{equation*}
  Is $\set{\vect{u}_1,\vect{u}_2}$ a basis of $V$? If not, extend it
  to a basis of $V$ by adding additional basis vectors.
\end{ex}

\begin{ex}
  Shrink $\set{\vect{u}_1,\vect{u}_2,\vect{u}_3,\vect{u}_4}$ to a
  basis of $\R^4$ by removing redundant vectors, where
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{r} 1 \\ 2 \\ 3 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} 0 \\ -1 \\ 2 \end{mymatrix}.
    \vect{u}_3 = \begin{mymatrix}{r} 2 \\ 3 \\ 8 \end{mymatrix},\quad
    \vect{u}_4 = \begin{mymatrix}{r} -7 \\ 2 \\ 1 \end{mymatrix}.
  \end{equation*}
\end{ex}

\begin{ex}
  Use one of the basis tests of
  Proposition~\ref{prop:basis-test-k-vectors} to determine whether the
  vectors
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{r} 4 \\ -2 \\ 1 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} -2 \\ 4 \\ 1 \end{mymatrix},\quad
    \mbox{and}\quad
    \vect{u}_3 = \begin{mymatrix}{r} 1 \\ -2 \\ 4 \end{mymatrix}
  \end{equation*}
  form a basis of\/ $\R^3$.
\end{ex}

\begin{ex}
  Use one of the basis tests of
  Proposition~\ref{prop:basis-test-k-vectors} to determine whether the
  vectors
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{r} 2 \\ -1 \\ -1 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} -1 \\ 2 \\ -1 \end{mymatrix},\quad
    \mbox{and}\quad
    \vect{u}_3 = \begin{mymatrix}{r} -1 \\ -1 \\ 2 \end{mymatrix}
  \end{equation*}
  form a basis of\/ $\R^3$.
\end{ex}

\begin{ex}
  In this exercise, we use scalars from the field $\Z_5$ of integers
  modulo $5$ instead of real numbers (see Section~\ref{sec:fields},
  ``Fields'').  Find a basis and the dimension of each of the
  following subspaces of $(\Z_5)^n$.
  \begin{enumerate}
  \item $V_1 = \sspan\set{
      \begin{mymatrix}{r} 2 \\ 3 \\ 1 \\ 4 \end{mymatrix},~
      \begin{mymatrix}{r} 1 \\ 4 \\ 3 \\ 2 \end{mymatrix},~
      \begin{mymatrix}{r} 4 \\ 1 \\ 2 \\ 3 \end{mymatrix},~
      \begin{mymatrix}{r} 2 \\ 2 \\ 1 \\ 0 \end{mymatrix}
    }$.
  \item $V_2 =
    \set{\left.\begin{mymatrix}{c} u \\ v \\ w \end{mymatrix}
        ~\right\vert~
      \mbox{$2u+v=0$ and $u+4w=0$}}.$
  \item $V_3 =
    \set{\left.\begin{mymatrix}{c} u \\ v \\ w \end{mymatrix}
        ~\right\vert~
      \mbox{$u+2v+3w=0$}}.$
  \end{enumerate}
  \begin{sol}
    \begin{enumerate}
    \item Basis $\set{\begin{mymatrix}{r} 2 \\ 3 \\ 1 \\ 4 \end{mymatrix},
      \begin{mymatrix}{r} 2 \\ 2 \\ 1 \\ 0 \end{mymatrix}}$, dimension 2.
    \item Basis $\set{\begin{mymatrix}{r} 1 \\ 3 \\ 1 \end{mymatrix}}$,
      dimension 1.
    \item Basis $\set{\begin{mymatrix}{r} 2 \\ 0 \\ 1 \end{mymatrix},
      \begin{mymatrix}{r} 3 \\ 1 \\ 0 \end{mymatrix}}$,
      dimension 2.
    \end{enumerate}
  \end{sol}
\end{ex}

\begin{ex}
  In this exercise, we use scalars from the field $\Z_7$ of integers
  modulo $7$ instead of real numbers.
  Find the coordinates of $\vect{v}$ with respect
  to the basis $B=\set{\vect{u}_1,\vect{u}_2,\vect{u}_3}$ in $(\Z_7)^3$, where
  \begin{equation*}
    \vect{v} = \begin{mymatrix}{r} 3 \\ 1 \\ 4 \end{mymatrix},\quad
    \vect{u}_1 = \begin{mymatrix}{r} 2 \\ 4 \\ 2 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} 1 \\ 3 \\ 3 \end{mymatrix},\quad
    \vect{u}_3 = \begin{mymatrix}{r} 4 \\ 1 \\ 2 \end{mymatrix}.
  \end{equation*}
\end{ex}

\begin{ex}
  In this exercise, we use scalars from the field $\Z_2$ of integers
  modulo $2$ instead of real numbers.
  Extend $\set{\vect{u}_1,\vect{u}_2}$ to a basis of $(\Z_2)^4$, where
  \begin{equation*}
    \vect{u}_1 = \begin{mymatrix}{r} 1 \\ 0 \\ 1 \\ 1 \end{mymatrix},\quad
    \vect{u}_2 = \begin{mymatrix}{r} 0 \\ 1 \\ 1 \\ 1 \end{mymatrix}.
  \end{equation*}
\end{ex}

\begin{ex}
  True or false? Explain.
  \begin{enumerate}
  \item Every set of $5$ vectors in $\R^5$ is linearly independent.
  \item Every set of $4$ vectors in $\R^5$ is linearly independent.
  \item Every set of $6$ vectors in $\R^5$ is linearly dependent.
  \item No set of $4$ vectors spans $\R^5$.
  \item Every linearly independent set of $5$ vectors in $\R^5$ is a
    basis of $\R^5$.
  \item Every linearly independent set of $4$ vectors in $\R^5$ is a
    basis of $\R^5$.
  \item Some linearly independent set of $4$ vectors in $\R^5$ is a
    basis of $\R^5$.
  \item Every spanning set of $6$ vectors in $\R^5$ is a basis of
    $\R^5$.
  \item Every linearly independent set of $4$ vectors in $\R^5$ spans
    a $4$-dimensional subspace of $\R^5$.
  \end{enumerate}
  \begin{sol}
    \begin{enumerate}
    \item No. For example, the vectors
      $\set{\vect{0},\vect{0},\vect{0},\vect{0},\vect{0}}$ are
      linearly dependent.
    \item No. For example, the vectors
      $\set{\vect{0},\vect{0},\vect{0},\vect{0}}$ are linearly
      dependent.
    \item Yes, by Proposition~\ref{prop:size-linearly-independent-or-spanning}.
    \item Correct, by Proposition~\ref{prop:size-linearly-independent-or-spanning}.
    \item Yes, by Proposition~\ref{prop:basis-test-k-vectors}.
    \item No, in fact no such set is a basis of $\R^5$, since every
      basis of $\R^5$ consists of $5$ vectors by
      Theorem~\ref{thm:bases-same-size}.
    \item No, as noted in the previous answer.
    \item No, no basis of $\R^5$ can have $6$ elements.
    \item Yes, because a linearly independent set of vectors is a
      basis of the subspace it spans.
    \end{enumerate}
  \end{sol}
\end{ex}

\begin{ex}
  If you have $6$ vectors in $\R^5$, is it possible they are
  linearly independent? Explain.
  \begin{sol}
    No. As a $5$-dimensional space, $\R^5$ is spanned by $5$
    vectors. By the Exchange Lemma, any linearly independent set can
    have size at most $5$.
  \end{sol}
\end{ex}

\begin{ex}
  Suppose $V$ and $W$ both have dimension equal to $7$ and they are
  subspaces of $\R^{10}$. What are the possibilities for the dimension
  of $V\cap W$? \textbf{Hint:} Remember that a linear independent set can be
  extended to form a basis.
  % \begin{sol}
  % \end{sol}
\end{ex}

\void{
  %% Something is wrong with this problem. Whoever wrote the solution
  %% did not prove that the $p+q-k$ vectors are in fact linearly
  %% independent. This is required to conclude $p+q-k \neq n$.
  \begin{ex}
    Suppose $V$ has dimension $p$ and $W$ has dimension $q$ and they
    are each contained in a subspace, $U$ which has dimension equal to $n$ where
    $n>\max (p,q)$. What are the possibilities for the dimension of
    $V\cap W$? \textbf{Hint:} Remember that a linearly independent set can be
    extended to form a basis.
    \begin{sol}
      Let $\set{x_1,\ldots,x_k}$ be a
      basis for $V\cap W$. Then there are bases for $V$ and $W$ which are
      respectively
      \begin{equation*}
        \set{x_1,\ldots,x_k,y_{k+1},\ldots,y_p},\ \set{
          x_1,\ldots,x_k,z_{k+1},\ldots,z_q}.
      \end{equation*}
      None of the $y_i$ are in $W$ (or else they would be in
      $V\cap W$, and therefore a linear combination of
      $x_1,\ldots,x_k$). Similarly, none of the $z_j$ are in
      $V$. It follows that $k+p-k+q-k\leq n$, and so
      \begin{equation*}
        p+q-n\leq k
      \end{equation*}
    \end{sol}
  \end{ex}
}


\section{Phase space, Hamiltonian dynamics, and the fundamental postulate of statistical mechanics}

\subsection{Phase space}

At a microscopic level, a thermodynamic system is comprised of $N$ particles, each with some mass $m_i$, which can interact with each other and with the boundaries of the system. (Although we are going to motivate this discussion with a thermodynamic system, everything the follows is also true for other systems where the total energy of the system is conserved --- i.e. anywhere that a Hamiltonian could show up.) If we know the positions and momenta  for each of the particles then we can completely describe the state of the system. In three dimensions, this means we must know the values for each of the $6N$ degrees of freedom (3 for position and 3 for momentum); more generally, we have $2d$ d.o.f. where $d$ is the number of d.o.f. for the positions. That is the state of the system is described by a point in a $6N$ (or $2d$) dimensional phase space.
Once a point in the microscopic phase space is know, the evolution of the system is completely governed by the equations of motion for the system. These in turn are prescribed by the Hamiltonian of the system

$$H = T+U = H({\bf r}, {\bf v}) = H({\bf q}, {\bf p}),$$
$T=$ kinetic energy, $U=$ potential energy. In general, $H$, $T$, and $U$ may all depend on both the positions and the momenta of the particles, however, for many systems we find that, conveniently, $T = \sum_{i=1}^{N} T_i$, where $T_i=\frac{{\bf p}_i^2}{2m_i}$, and the potential $U$ depends only on the positions: $U = U({\bf q}).$

If the Hamiltonian is known for a system, then the equations of motion for that system are given by the \textit{Hamiltonian equations of motion}:
\begin{eqnarray*}
	\frac{d{\bf q}_i}{dt} &=& ~~\frac{\partial H}{\partial {\bf p_i}}, \\
	\frac{d{\bf p}_i}{dt} &=& -\frac{\partial H}{\partial {\bf q_i}}.
\end{eqnarray*}

If we write ${\bf x} = [{\bf q}^\top,{\bf p}^\top]^\top$, then we can compactly express the equations of motion as

\begin{equation}
	\frac{d{\bf x}}{dt} = f({\bf x}) = J^{-1}\nabla H({\bf x}),
\end{equation}
where
$$
	J =
	\begin{bmatrix}
		0 & -Id\\
		Id & 0
	\end{bmatrix}.
$$
($Id$ is the identity matrix of dimension $d$.)

Given a Hamiltonian, H, and a suitable set of initial conditions ${\bf x}_0$, the evolution of the solution ${\bf x}(t)$ can be quite complicated (Hamiltonian systems are one of the archetypal systems for studying chaotic dynamics). However, the fact that the system is Hamiltonian provides us with some additional information that can help us characterise the state of the system. For example, it is not hard to prove that a solution ${\bf x}(t)$ that satisfies $\dot{\bf x} = f({\bf x}) = J^{-1}\nabla H({\bf x}),$ ${\bf x}_0 = {\bf x}(0)$ has $H({\bf x}(t)) = H({\bf x}_0)= const.$ That is, Hamiltonian systems conserve energy along their solutions. We identify the Hamiltonian $H$ with the internal energy of the system.

Knowing that $\frac{dH}{dt}=0$, we can reduce the size of the phase space where ${\bf x}(t)$ can lie, from $2d$ d.o.f. down to $2d-1$. If we know other quantities that remain invariant over time (e.g. the total momentum of the system) then we can further reduce the size of the phase space. Strictly speaking, there are $2d-1$ invariants (AKA first integrals or constants of motion) for a Hamiltonian system. This defines a curve in the remaining 1 d.o.f. along which the solution must lie (that is, a line in the phase space). This can be seen as a consequence of the uniqueness theorem for differential equations (think back to the 2nd year maths course on D.E.s). In reality, we typically can't find a closed form expression for most of these conserved quantities and have no idea what they might be.

It is possible that a (vanishingly) small change in the positions of the particles in phase space can lead to a large change in the state of the system and in the subsequent trajectories of the particles.
This is similar to saying that the dynamics of a Hamiltonian system can switch abruptly from regular to chaotic motion, as some parameter of the system is varied.
Such quantities are not helpful in characterising the thermodynamics of the system.
In statistical mechanics, we are interested in the invariant quantities that only change by a little when there is a small change in the microscopic variables of the system.
E.g. ${\bf x}\rightarrow {\bf x}+\delta {\bf x} \implies H\rightarrow H+\delta H$.

One way to phrase this is to say that we want to consider systems where the region in phase space where the system is evolving can be (almost) completely identified, for thermodynamic purposes, by the values of a (not too large) number of observable invariant quantities.

\subsection{Observables}
In statistical mechanics, we say that an observable is a function defined on the phase space of the system that varies smoothly with changes in the representative point of the system. This definition is different to the definition in quantum mechanics.

The kinetic energy $\sum_i \frac{p_i}{2m_i}$ is an observable, while the function $\Theta = \prod_i \theta(r_0^2-r_i^2)$, where $\theta(x)$ is the Heaviside function, is not, since an arbitrarily small change in a position $r_i$ can lead to a finite change in $\Theta$. This behaviour is not smooth which means $\Theta$ is not an observable. This non-smoothness results from the presence of the Heaviside function (also known as the unit step function), which is equal to zero when its argument is negative and one when it is positive; this function has a jump when the argument is zero and is thus obviously not smooth.

Other examples of (thermodynamic) observables include the internal energy $E$ (i.e. the Hamiltonian) and the number of particles in the system ($N = \int d{\bf r}\sum\delta({\bf r}-{\bf r}_i)$).
It turns out that the entropy $S=S(X_0=E=H,X_1,\ldots,X_r)$ is also an observable. However, we don't yet have a way to express the entropy --- the extensive variable of a thermodynamic system --- as a function of the microscopic state of the system and its equations of motion. To do so, we are going to invoke the fundamental postulate of statistical mechanics.

\subsection{The fundamental postulate of statistical mechanics}
The fundamental postulate expresses the entropy $S$ as a function of the accessible volume of the phase space of a dynamical system. I.e. that part of the phase space where the observables have values that are compatible with a specific thermodynamic state.

Assume that the thermodynamic state of a system is determined by the extensive variables $X_1,\ldots,X_r$ and that each of the $X_i$ can be expressed as a function over the microscopic phase space; I.e. $X_i = X_i({\bf x})$.
The region of phase space that is accessible is defined by the value of $X = (X_1,\ldots,X_r)$. Well, almost; we actually also allow for small discrepancies $\epsilon_i,~~i=1,\ldots,r$ to account for the effect of any non-extensive terms.

We denote the region of phase space in which the observable $X_i$ have the specific values that match the thermodynamic state as $\Gamma$ and we denote the corresponding phase space volume $\text{vol}(\Gamma)=\Omega$. Then the fundamental postulate of thermodynamics states that
\begin{equation}
	S = k_B\ln(\Omega),
	\label{eqS}
\end{equation}
where $S$ is the thermodynamic entropy, $k_B$ is Boltzmann's constant ($k_B = 1.384\times10^{-23}$J/K) and the equality should be interpreted as meaning ``is equal, up to terms of size $\epsilon_i$''. The accessible phase space volume $\Omega$ measures the amount of the phase space available to the microscopic system while it is in this macroscopic state. If the system had only discrete micro-states, then we could simply count the number of micro-states that correspond to the same macro-state.

\subsection{Symplectic transformations and phase space area preservation}
Why is phase space so important in the formulation of entropy? To understand this we are going to look closer at some of the properties that being Hamiltonian impose on a system, including the choice of coordinate system.

We saw earlier that the equations of motion for a time-independent Hamiltonian system can be written as$ \frac{d{\bf x}}{dt} = f({\bf x}) = J^{-1}\nabla H({\bf x})$. This Hamiltonian form is not preserved by arbitrary coordinate transformations; it \emph{is} preserved by \emph{symplectic} transformations.

{\bf Definition:} A linear transformation $A: \mathbb{R}^{2d}\rightarrow\mathbb{R}^{2d}$ is said to be symplectic if it satisfies
$$ A^\top J A = J.$$
For a non-linear transformation $\mathcal{A}$, we replace $A$ above with its jacobian $D\mathcal{A}$.

An alternative, but equivalent, definition of a symplectic transformation --- and one that will turn out to be useful in a bit --- borrows some tools from differential geometry. (No prior experience with differential geometry is assumed.)

{\bf Definitions:} A symplectic transformation is a diffeomorphism (a smooth, invertible mapping between manifolds) $f:\mathbb{R}^{2d}\rightarrow\mathbb{R}^{2d}$, with coordinates $q$ and $p$, which preserves the symplectic two-form $\omega$ (an anti-symmetric, bilinear form) that acts on pairs of vectors and gives the sum of the oriented areas that they define. In two dimensions, this is the area of the parallelogram spanned by the two vectors.

More explicitly,
\begin{equation*}
	\omega(u,v) = u^\top Jv = \sum_{i=1}^d u_{p_i} v_{q_i} - u_{q_i} v_{p_i}.
\end{equation*}

At this point it is necessary to introduce a (potentially) new concept --- the flow map of a vector field. The flow $\varphi_{t,f}({\bf x})$ of a Hamiltonian system is a function that maps a point ${\bf x}^*$ in the phase space of the system with equations of motion $\dot{\bf x} = f({\bf x}) =J^{-1}\nabla H({\bf x})$ forward by time $t$. I.e. for any $f({\bf x})$ and ${\bf x}_0$, $\varphi_{t,f}({\bf x}_0)$ gives the solution to $\dot{\bf x}=f({\bf x})$ at time $t$. That is, it shows the time evolution of {\bf x} and {\bf p} on the phase space.

{\bf Theorem:} If the Hamiltonian $H({\bf x})$ is a twice differentiable function on some neighbourhood of $\mathbb{R}^{2d}$, then the time-$t$ flow map $\varphi_{t,f}({\bf x}_0)$ of the Hamiltonian vector field is a symplectic map for all $t$.
{\bf Proof:} We will show that $\omega(u,v)=u^\top J v$ is constant along solutions of $\dot{\bf x} = J^{-1}\nabla H({\bf x})$.

The tangent vector $u(t) = \frac{\partial \varphi_t}{\partial {\bf x}_0}({\bf x}_0)u(0)$ satisfies $\frac{du}{dt} = J^{-1}\nabla^2H({\bf x}(t))u(t)$ and the tangent vector $v(t)$ satisfies a similar expression. Now we can calculate the time derivative of $\omega(u,v)$.
\begin{eqnarray*}
	\frac{d}{dt}\omega(u,v) &=& \dot{u}^{\top}Jv+u^\top J\dot{v}\\
		&=& (J^{-1}\nabla^2Hu)^\top Jv + u^\top JJ^{-1}\nabla^2Hv\\
		&=& 0
\end{eqnarray*}
since $J^{-\top}=(J^{-1})^\top$, $J^{-\top}J=-I$ and $JJ^{-1} = I$.

I.e. as the system evolves according to its equations of motion, it always preserves the area for $d=1$, or the sum of the oriented areas for $d \geq2$.

\subsection{Liouville's theorem and preservation of phase space volume}
We are now in a position to prove that the flow of a Hamiltonian system preserves phase space volume. That is, for every bounded, open set $A\subset\mathcal{R}^{2d}$ and every $t$ for which $\varphi_{t,f}({\bf x})$ exists
$$\text{Vol}(\varphi_{t,f}(\Gamma))=\text{Vol}(\Gamma)$$
where $f=J^{-1}\nabla H$ and $\text{Vol}(\Gamma) = \int_{\Gamma}d{\bf x}$.

To prove this, we will actually prove a slightly more general case: the flow of a differential equation $\dot{\bf x}=f({\bf x})$ in $\mathbb{R}^n$ is volume preserving if and only if it is divergence-free; i.e. $\nabla\cdot f=0$.

But first, why do we care about preservation of phase space volume? The fundamental postulate of statistical mechanics, $S=k_B\ln(\Omega)$ related the entropy of a system to the volume of the part of the phase space accessible to that system when it is in that particular thermodynamic state. We would like to be sure that this quantity doesn't vary too much over time as the microscopic system evolves, according to the Hamiltonian equations of motion. Although the microscopic system might be in continuous motion, the trajectories of the particles it is comprised of are such that the extensive variables which characterise the macroscopic state of the system are (almost) constant. I.e. there is no change in entropy, no heat flow, no work done while the system is at equilibrium. This is reassuring --- it says that disorder doesn't increase unless something happens.

{\bf Proof: volume preservation:}
The Jacobian of the flow map $\varphi_{t,f}({\bf x}_0)$ defines the matrix function $X(t)=D\varphi_{t,f}({\bf x}_0)$. The volume of the shape spanned by the columns of $X$ is given by $\det(X)$, and $X(t)$ is a solution of the equation
$$
\dot{X}=A(t)X(t),\qquad X(0)=I,
$$
where $A(t):=Df({\bf x}(t))$ denotes the Jacobian of $f({\bf x}(t))$ evaluated at ${\bf x}(t)=\varphi_{t,f}({\bf x}_0)$. To show that the volume defined by $X$ is constant we use the Abel-Liouville-Jacobi-Ostrogradskii identity to find the time derivative of $\det X$:
$$
\frac{d}{dt}\det X(t)=\text{trace} A(t)\cdot\det X(t).
$$
Since $A(t)=Df({\bf x}(t))$ we have $\text{trace} A(t)=\sum_i\frac{\partial f_i}{\partial {\bf x}_i}({\bf x}(t))=\nabla\cdot f({\bf x}(t))$ and therefore $\det X(t)=\det X(0)=1$ if and only if $\nabla\cdot f({\bf x})=0.$

\subsection{Recommended reading}
This section has mostly followed the first half of chapter 3 of \emph{Statistical Mechanics in a Nutshell}. However, some of the significance of Hamiltonian systems is lost in SMiaN, so I've supplemented it with more substantive (mathematically oriented) details from \emph{Geometric Numerical Integration} by E. Hairer, Ch. Lubich and G. Wanner (Springer, 2006) --- one of the best books on (the numerics of) Hamiltonian dynamical systems and a personal favourite.


\subsection{Least Squares Fit}

Given a discrete set of data points which represent a measured step  response of
an  unknown system, the input parameters  of  the  function  $G_n(s,r)$  can  be
tweaked such that the squared error between its step response and the input data
is minimised. The squared error is computed using:

\begin{equation}
    S = \sum_{i} \left(g(t_i)-y_i\right)^2
\end{equation}

where $g(t)$ is the inverse Laplace transform  of $\frac{G_n(s,r)}{s}$ (the time
domain step response) and ($t_i$,$y_i$) are  data  points  of  the measured step
response.

By fitting a system $G_n(s,r)$ to  the  input  data,  it  is possible to further
refine the results  obtained by the two methods mentioned thus far, or otherwise
find  optimal  values  for  $T$  and  $r$ when dealing with  noisy  input  data.

The possibility of finding local minima exists. It is therefore advised to first
use one of the four previous methods to find optimal initial values for $T$, $r$
and $n$ before performing the fit.

It will be shown that the  least  squares  fit  approach  will  yield  the  most
accurate results by orders of magnitude. The downside to this method, of course,
is the large amount of computation time required.


\subsection{Simultaneous games}

\subsubsection{One round simultaneous games}

Economic agents face options from some set. This could be consumption choices, numbers of hours to work, or how much capital to invest in at a factory.

Consider the prisoner’s dilemma game:
	table.table.table-bordered
		thead
			tr
				th
				th Silent
				th Tell
		tbody
			tr
				td Silent
				td (5,5)
				td (10,0)
			tr
				td Tell
				td (0,10)
				td (8,8)

In this game we have two agents who simultaneously choose

Let’s compare the decision to “tell” to the decision to be “silent”.

No matter what the other agent does, you are always better off choosing “tell”. As a result we say the strategy “tell” strictly dominates “silent”.

If under some circumstances the agent is indifferent to the strategy and another, then the strategy only weakly dominates.

So one way to solve a game is to choose dominating strategies. However an agent may not have strictly dominating strategies. Another method it to rule out strategies. If one strategy is strictly dominated for an agent, we can rule out them choosing it. This may reveal strategies which are dominant one actions of another agent can be ruled out.

If after iterations of this process we are left with only one strategy for each agent, we say this is a Von Neumann solution, an analytic solution.

But what if there are still multiple options? Consider

	table.table.table-bordered
		thead
			tr
				th
				th Opera
				th Tell
		tbody
			tr
				td Opera
				td (10,5)
				td (0,0)
			tr
				td Football
				td (0,0)
				td (5,10)

And

	table.table.table-bordered
		thead
			tr
				th
				th Rock
				th Paper
				th Scissors
		tbody
			tr
				td Rock
				td (0,0)
				td (-1,1)
				td (1,-1)
			tr
				td Paper
				td (1,-1)
				td (0,0)
				td (-1,1)
			tr
				td Scissors
				td (-1,1)
				td (1,-1)
				td (0,0)

In both of these there is no strategy which is always better to follow, even weakly. But these games are very different. In the former, if agents agree to both got to football, or both to opera, neither would be better off by defecting. In the second example there is no such “Nash equilibrium”.

This is relevant for considering how to expand the game. In the former example a couple can talk to each other and coordinate actions.  For example one agent could commit to going to the football, and the other agent would rationally join.

In the latter no such coordination is beneficial.

In the context of the game, the player can instead of choosing a pure strategy such as “rock”, which may not always be appropriate, choose a mixed strategy.

For example a player could choose each of the \(3\) moves \(\dfrac{1}{3}\) of the time.

\subsubsection{Nash equilibrium}

\subsubsection{Minimax strategy}

\subsubsection{Pure/mixed strategy}


\documentclass[../../question_3_array_question.tex]{subfiles}
\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%% Subset
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Subset(Combination and Permutation)}
\label{part4_array_subset}
The Subset B of a set A is defined as a set within all elements of this subset are from set A. In other words, the subset B is contained inside the set A, $B \in A$. There are two kinds of subsets: if the order of the subset doesnt matter, it is a combination problem, otherwise, it is a permutation problem. To solve the problems in this section, we need to refer to the backtracking in Sec~\ref{sec_combination}. When the subset has a fixed constant length, then hashmap can be used to lower the complexity by one power of n.

\textbf{Subset VS Subsequence}. In the subsequence, the elements keep the original order from the original sequence. While, in the set concept, there is no ordering, only a set of elements.

In this type of questions, we are asked to return subsets of a list. For this type of questions, backtracking~\ref{sec:backtrack} can be applied.
\subsection{Combination}
\label{part4_array_combine}
The solution of this section is heavily correlated to Section~\ref{sec_combination}.
78. Subsets
\begin{lstlisting}
Given a set of distinct integers, nums, return all possible subsets (the power set).

Note: The solution set must not contain duplicate subsets.

Example:

Input: nums = [1,2,3]
Output:
[
  [3],
  [1],
  [2],
  [1,2,3],
  [1,3],
  [2,3],
  [1,2],
  []
]
\end{lstlisting}
\textbf{Backtracking}. This is a combination problem, which we have explained in backtrack section. We just directly gave the code here.
\begin{lstlisting}[language = Python]
def subsets(self, nums):
    res, n = [], len(nums)
    res = self.combine(nums, n, n)
    return res

def combine(self, nums, n, k):
    """
    :type n: int
    :type k: int
    :rtype: List[List[int]]
    """
    def C_n_k(d, k, s, curr, ans): #d controls the degree (depth), k is controls the return level, curr saves the current result, ans is all the result
        ans.append(curr)
        if d == k: #the length is satisfied

            return
        for i in range(s, n):
            curr.append(nums[i])
            C_n_k(d+1, k, i+1, curr[:], ans) # i+1 because no repeat, make sure use deep copy curr[:]
            curr.pop()

    ans = []
    C_n_k(0, k, 0, [], ans)
    return ans
\end{lstlisting}
\textbf{Incremental}. Backtracking is not the only way for the above problem. There is another way to do it iterative, observe the following process. We can just keep append elements to the end of of previous results.
\begin{lstlisting}
[1, 2, 3, 4]
l = 0, []
l = 1, for 1, []+[1], -> [1],  get powerset of [1]
l = 2, for 2, []+[2], [1]+[2], -> [2], [1, 2], get powerset of [1, 2]
l = 3, for 3, []+[3], [1]+[3], [2]+[3], [1, 2]+[3], -> [3], [1, 3], [2, 3], [1, 2, 3], get powerset of [1, 2, 3]
l = 4, for 4, []+ [4]; [1]+[4]; [2]+[4], [1, 2] +[4]; [3]+[4], [1,3]+[4],[2,3]+[4], [1,2,3]+[4], get powerset of [1, 2, 3, 4]
\end{lstlisting}
\begin{lstlisting}[language=Python]
def subsets(self, nums):
    result = [[]] #use two dimensional, which already have [] one element
    for num in nums:
        new_results = []
        for r in result:
            new_results.append(r + [num])
        result += new_results

    return result
\end{lstlisting}
90. Subsets II
\begin{lstlisting}
Given a collection of integers that might contain duplicates, nums, return all possible subsets (the power set).

Note: The solution set must not contain duplicate subsets.

Example:

Input: [1,2,2]
Output:
[
  [2],
  [1],
  [1,2,2],
  [2,2],
  [1,2],
  []
]
\end{lstlisting}
Analysis: Because of the duplicates, the previous superset algorithm would give repetitive subset. For the above example, we would have [1, 2] twice, and [2] twice.  If we try to modify on the previous code. We first need to sort the nums, which makes the way we check repeat easiler. Then the code goes like this:
\begin{lstlisting}[language = Python]
    def subsetsWithDup(self, nums):
        """
        :type nums: List[int]
        :rtype: List[List[int]]
        """
        nums.sort()
        result = [[]] #use two dimensional, which already have [] one element
        for num in nums:
            new_results = []
            for r in result:
                print(r)
                new_results.append(r + [num])
            for rst in new_results:
                if rst not in result: # check the repetitive
                    result.append(rst)

        return result
\end{lstlisting}
However, the above code is extremely inefficient because of the checking process. A better way to do this:
\begin{lstlisting}
[1, 2, 2]
l = 0, []
l = 1, for 1, []+[1]
l = 2, for 2, []+[2], [1]+[2]; []+[2, 2], [1]+[2, 2]
\end{lstlisting}
So it would be more efficient if we first save all the numbers in the array in a dictionary. For the above case, the dic = {1:1, 2:2}. Each time we try to generate the result, we use 2 up to 2 times. Same way, we can use dictionary on the backtracking too.
\begin{lstlisting}[language=Python]
class Solution(object):
    def subsetsWithDup(self, nums):
        """
        :type nums: List[int]
        :rtype: List[List[int]]
        """
        if not nums:
            return [[]]
        res = [[]]
        dic = collections.Counter(nums)
        for key, val in dic.items():
            tmp = []
            for lst in res:
                for i in range(1, val+1):
                    tmp.append(lst+[key]*i)
            res += tmp
        return res
\end{lstlisting}

77. Combinations
\begin{lstlisting}
Given two integers n and k, return all possible combinations of k numbers out of 1 ... n.

Example:

Input: n = 4, k = 2
Output:
[
  [2,4],
  [3,4],
  [2,3],
  [1,2],
  [1,3],
  [1,4],
]
\end{lstlisting}
Analysis: In this problem, it is difficult for us to generate the results iteratively, the only way we can use the second solution is by filtering and get only the results with the length we want. However, the backtrack can solve the problem easily as we mentioned in Section~\ref{sec_combination}.
\begin{lstlisting}[language=Python]
def combine(self, n, k):
    """
    :type n: int
    :type k: int
    :rtype: List[List[int]]
    """
    ans = []
    def C_n_k(d,k,s,curr):
        if d==k:
            ans.append(curr)
            return
        for i in range(s, n):
            #curr.append(i+1)
            #C_n_k(d+1, k, i+1, curr[:])
            #curr.pop()
            C_n_k(d+1, k, i+1, curr+[i+1])
    C_n_k(0,k,0,[])

    return ans
\end{lstlisting}
%%%%%%%%%%%%%%%%%%%combination sum%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Combination Sum}
39. Combination Sum

Given a set of candidate numbers (candidates) \textbf{(without duplicates)} and a target number (target), find all unique combinations in candidates where the candidate numbers sums to target.

The same repeated number may be chosen from candidates \textbf{unlimited number} of times.
\begin{lstlisting}
Note:

    All numbers (including target) will be positive integers.
    The solution set must not contain duplicate combinations.

Example 1:

Input: candidates = [2,3,6,7], target = 7,
A solution set is:
[
  [7],
  [2,2,3]
]

Example 2:

Input: candidates = [2,3,5], target = 8,
A solution set is:
[
  [2,2,2,2],
  [2,3,3],
  [3,5]
]
\end{lstlisting}
\textbf{DFS Backtracking}. Analysis: This is still a typical combination problem, the only thing is the return level is when the sum of the path we gained is larger than the target, and we only collect the answer when it is equal. And Because a number can be used unlimited times, so that each time after we used one number, we do not increase the next start position.
\begin{lstlisting}[language=Python]
def combinationSum(self, candidates, target):
    """
    :type candidates: List[int]
    :type target: int
    :rtype: List[List[int]]
    """
    ans = []
    candidates.sort()
    self.combine(candidates, target, 0, [], ans)
    return ans

def combine(self, nums, target, s, curr, ans):
    if target < 0:
        return  # backtracking
    if target == 0:
        ans.append(curr)
        return
    for i in range(s, len(nums)):
        # if nums[i] > target:
        #     return
        self.combine(nums, target-nums[i], i, curr+[nums[i]], ans) # use i, instead of i+1 because we can reuse
\end{lstlisting}
40. Combination Sum II

Given a collection of candidate numbers \textbf{(candidates with duplicates)} and a target number (target), find all unique combinations in candidates where the candidate numbers sums to target.

Each number in candidates may only \textbf{be used once} in the combination.
\begin{lstlisting}
Note:

    All numbers (including target) will be positive integers.
    The solution set must not contain duplicate combinations.

Example 1:

Input: candidates = [10,1,2,7,6,1,5], target = 8,
A solution set is:
[
  [1, 7],
  [1, 2, 5],
  [2, 6],
  [1, 1, 6]
]

Example 2:

Input: candidates = [2,5,2,1,2], target = 5,
A solution set is:
[
  [1,2,2],
  [5]
]
\end{lstlisting}
\textbf{Backtracking+Counter}. Because for the first example, if we reuse the code from the previous problem, we will get extra combinations: [7, 1], [2, 1, 5]. To avoid this, we need a dictionary to save all the unique candidates with its corresponding appearing times. For a certain number, it will be used at most its counter times.
\begin{lstlisting}[language=Python]
def combinationSum2(self, candidates, target):
    """
    :type candidates: List[int]
    :type target: int
    :rtype: List[List[int]]
    """

    candidates = collections.Counter(candidates)
    ans = []
    self.combine(list(candidates.items()), target, 0, [], ans) # convert the Counter to a list of (key, item) tuple
    return ans

def combine(self, nums, target, s, curr, ans):
    if target < 0:
        return
    if target == 0:
        ans.append(curr)
        return
    for idx in range(s, len(nums)):
        num, count = nums[idx]
        for c in range(count):
            self.combine(nums, target-num*(c+1), idx+1, curr+[num]*(c+1), ans )
\end{lstlisting}
377. Combination Sum IV (medium)
\begin{lstlisting}
 Given an integer array with all positive numbers and no duplicates, find the number of possible combinations that add up to a positive integer target.

Example:

nums = [1, 2, 3]
target = 4

The possible combination ways are:
(1, 1, 1, 1)
(1, 1, 2)
(1, 2, 1)
(1, 3)
(2, 1, 1)
(2, 2)
(3, 1)

Note that different sequences are counted as different combinations.

Therefore the output is 7.

Follow up:
What if negative numbers are allowed in the given array?
How does it change the problem?
What limitation we need to add to the question to allow negative numbers?
\end{lstlisting}
\textbf{DFS + MEMO}. This problem is similar to 39. Combination Sum. For [2, 3, 5], target = 8,  comparison:
\begin{lstlisting}
[2, 3, 5], target = 8
39. Combination Sum. # there is ordering (each time the start index is same or larger than before)
[
  [2,2,2,2],
  [2,3,3],
  [3,5]
]
377. Combination Sum IV, here we have no ordering( each time the start index is the same as before). Try all element.
[
  [2,2,2,2],
  [2,3,3],
* [3,3,2]
* [3,2,3]
  [3,5],
* [5,3]
]
\end{lstlisting}
\begin{lstlisting}[language=Python]
def combinationSum4(self, nums, target):
    """
    :type nums: List[int]
    :type target: int
    :rtype: int
    """
    nums.sort()
    n = len(nums)
    def DFS(idx, memo, t):
        if t < 0:
            return 0
        if t == 0:
            return 1
        count = 0
        if t not in memo:
            for i in range(idx, n):
                count += DFS(idx, memo, t-nums[i])
            memo[t] = count
        return memo[t]
    return(DFS(0, {}, target))
\end{lstlisting}
Because, here we does not need to numerate all the possible solutions, we can use dynamic programming, which will be shown in Section~\ref{}.

\subsection{K Sum}
In this subsection, we still trying to get subset that sum up to a target. But the length here is fixed. We would have 2, 3, 4 sums normally. Because it is still a combination problem, we can use the \textbf{backtracking} to do. Second, because the fixed length, we can use \textbf{multiple pointers} to build up the potential same lengthed subset.  But in some cases, because the length is fixed, we can use \textbf{hashmap} to simplify the complexity.

1. Two Sum
Given an array of integers, return \textbf{indices} of the two numbers such that they add up to a specific target.

You may assume that each input would have \textbf{exactly} one solution, and you may not use the same element twice.
\begin{lstlisting}
Example:

Given nums = [2, 7, 11, 15], target = 9,

Because nums[0] + nums[1] = 2 + 7 = 9,
return [0, 1].
\end{lstlisting}
\textbf{Hashmap}. Using backtracking or brute force will get us $O(n^2)$ time complexity. We can use hashmap to save the nums in a dictionary. Then we just check target-num in the dictionary. We would get $O(n)$ time complexity. We have two-pass hashmap and one-pass hashmap.
\begin{lstlisting}[language=Python]
# two-pass hashmap
def twoSum(self, nums, target):
    """
    :type nums: List[int]
    :type target: int
    :rtype: List[int]
    """
    dict = collections.defaultdict(int)
    for i, t in enumerate(nums):
        dict[t] = i
    for i, t in enumerate(nums):
        if target - t in dict and i != dict[target-t]:
            return [i, dict[target-t]]
# one-pass hashmap
def twoSum(self, nums, target):
    """
    :type nums: List[int]
    :type target: int
    :rtype: List[int]
    """
    dict = collections.defaultdict(int)
    for i, t in enumerate(nums):
        if target - t in dict:
            return [dict[target-t], i]
        dict[t] = i
\end{lstlisting}

15. 3Sum

Given an array S of n integers, are there elements a, b, c in S such that a + b + c = 0? Find all unique triplets in the array which gives the sum of zero.

Note: The solution set must not contain duplicate triplets.

For example, given array S = [-1, 0, 1, 2, -1, -4],
\begin{lstlisting}
A solution set is:
[
  [-1, 0, 1],
  [-1, -1, 2]
]
\end{lstlisting}

Solution: Should use three pointers, no extra space. i is the start point from [0,len-2], l,r is the other two pointers. l=i+1, r=len-1 at the beignning. The saving of time complexity is totally from the sorting algorithm.
\begin{lstlisting}
[-4,-1,-1,0,1,2]
i, l-> ``````<-r
\end{lstlisting}
How to delete repeat?
\begin{lstlisting}[language = Python]
def threeSum(self, nums):
    res = []
    nums.sort()
    for i in xrange(len(nums)-2):
        if i > 0 and nums[i] == nums[i-1]: #make sure pointer not repeat
            continue
        l, r = i+1, len(nums)-1
        while l < r:
            s = nums[i] + nums[l] + nums[r]
            if s < 0:
                l +=1
            elif s > 0:
                r -= 1
            else:
                res.append((nums[i], nums[l], nums[r]))
                l+=1
                r-=1

                #after the first run, then check duplicate example.
                while l < r and nums[l] == nums[l-1]:
                    l += 1
                while l < r and nums[r] == nums[r+1]:
                    r -= 1
    return res
\end{lstlisting}
Use hashmap:
\begin{lstlisting}[language = Python]
def threeSum(self, nums):
        """
        :type nums: List[int]
        :rtype: List[List[int]]
        """
        res =[]
        nums=sorted(nums)
        if not nums:
            return []
        if nums[-1]<0 or nums[0]>0:
            return []
        end_position = len(nums)-2
        dic_nums={}
        for i in xrange(1,len(nums)):
            dic_nums[nums[i]]=i# same result save the last index

        for i in xrange(end_position):
            target = 0-nums[i]
            if i>0 and nums[i] == nums[i-1]: #this is to avoid repeat
                continue
            if target<nums[i]: #if the target is smaller than this, we can not find them on the right side
                break
            for j in range(i+1,len(nums)): #this is to avoid repeat
                if j>i+1 and nums[j]==nums[j-1]:
                    continue
                complement =target - nums[j]
                if complement<nums[j]: #if the left numbers are bigger than the complement, no need to keep searching
                    break
                if complement in dic_nums and dic_nums[complement]>j: #need to make sure the complement is bigger than nums[j]
                    res.append([nums[i],nums[j],complement])
        return res
\end{lstlisting}
The following code uses more time
\begin{lstlisting}[language = Python]
for i in xrange(len(nums)-2):
            if i > 0 and nums[i] == nums[i-1]:
                continue
            l, r = i+1, len(nums)-1
            while l < r:
                if l-1>=i+1 and nums[l] == nums[l-1]: #check the front
                    l += 1
                    continue
                if r+1<len(nums) and nums[r] == nums[r+1]:
                    r -= 1
                    continue
                s = nums[i] + nums[l] + nums[r]
                if s < 0:
                    l +=1
                elif s > 0:
                    r -= 1
                else:
                    res.append((nums[i], nums[l], nums[r]))
                    l += 1; r -= 1
        return res
\end{lstlisting}
18. 4Sum
\begin{lstlisting}[language = Python]
def fourSum(self, nums, target):
        def findNsum(nums, target, N, result, results):
            if len(nums) < N or N < 2 or target < nums[0]*N or target > nums[-1]*N:  # early termination
                return
            if N == 2: # two pointers solve sorted 2-sum problem
                l,r = 0,len(nums)-1
                while l < r:
                    s = nums[l] + nums[r]
                    if s == target:
                        results.append(result + [nums[l], nums[r]])
                        l += 1
                        r-=1
                        while l < r and nums[l] == nums[l-1]:
                            l += 1
                        while l < r and nums[r] == nums[r+1]:
                            r -= 1
                    elif s < target:
                        l += 1
                    else:
                        r -= 1
            else: # recursively reduce N
                for i in range(len(nums)-N+1):
                    if i == 0 or (i > 0 and nums[i-1] != nums[i]):
                        findNsum(nums[i+1:], target-nums[i], N-1, result+[nums[i]], results) #reduce nums size, reduce target, save result

results = []
        findNsum(sorted(nums), target, 4, [], results)
        return results
\end{lstlisting}

454. 4Sum II

Given four lists A, B, C, D of integer values, compute how many tuples (i, j, k, l) there are such that A[i] + B[j] + C[k] + D[l] is zero.

To make problem a bit easier, all A, B, C, D have same length of N where $0 \leq N \leq 500$. All integers are in the range of -228 to 228–1 and the result is guaranteed to be at most 231–1.

Example:
\begin{lstlisting}
Input:
A = [ 1, 2]
B = [-2,-1]
C = [-1, 2]
D = [ 0, 2]

Output:
2
\end{lstlisting}

Explanation:

\begin{lstlisting}
The two tuples are:
1. (0, 0, 0, 1) -> A[0] + B[0] + C[0] + D[1] = 1 + (-2) + (-1) + 2 = 0
2. (1, 1, 0, 0) -> A[1] + B[1] + C[0] + D[0] = 2 + (-1) + (-1) + 0 = 0
\end{lstlisting}
Solution: if we use brute force, use 4 for loop, then it is $O(N^4)$. If we use divide and conquer, sum the first half, and save a dictionary (counter), time complexity is $O(2N^2)$. What if we have 6 sum, we can reduce it to $O(2N^3)$, what if 8 sum.

\begin{lstlisting}[language = Python]
def fourSumCount(self, A, B, C, D):
    AB = collections.Counter(a+b for a in A for b in B)
    return sum(AB[-c-d] for c in C for d in D)
\end{lstlisting}

\subsubsection{Summary}
As we have seen from the shown examples in this section, to solve the combination problem, backtrack shown in Section~\ref{sec_combination} offers a universal solution. Also, there is another iterative solution which suits the power set purpose. And I would include its code here again:
\begin{lstlisting}[language = Python]
def subsets(self, nums):
    result = [[]] #use two dimensional, which already have [] one element
    for num in nums:
        new_results = []
        for r in result:
            new_results.append(r + [num])
        result += new_results

    return result
\end{lstlisting}
If we have duplicates, how to handle in the backtrack?? In the iterative solution, we can replace the array with a dictionary saves the counts.

\subsection{Permutation}
46. Permutations
\begin{lstlisting}
Given a collection of distinct numbers, return all possible permutations.

For example,
 [1,2,3] have the following permutations:

[
  [1,2,3],
  [1,3,2],
  [2,1,3],
  [2,3,1],
  [3,1,2],
  [3,2,1]
]
\end{lstlisting}

47. Permutations II

Given a collection of numbers that might contain duplicates, return all possible unique permutations.

For example,
\begin{lstlisting}
 [1,1,2] have the following unique permutations:

[
  [1,1,2],
  [1,2,1],
  [2,1,1]
]
\end{lstlisting}

301. Remove Invalid Parentheses

Remove the minimum number of invalid parentheses in order to make the input string valid. Return all possible results.

Note: The input string may contain letters other than the parentheses ( and ).

Examples:
\begin{lstlisting}
"()())()" -> ["()()()", "(())()"]
"(a)())()" -> ["(a)()()", "(a())()"]
")(" -> [""]
\end{lstlisting}
\end{document}