%
% finished at Aug. 5th, 2009
%
%
\chapter{Position representation and momentum representation}
\label{position_momentum representation}
% introduce the coordinates
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
%
% why we have this chapter?
%
%
In the previous sections, we have brought into the concept of
Hilbert space and operator, and have discussed their features in
general. However, all the discussions have been made are abstract.
For example, the wave functions are only considered as some
``vectors'' in the Hilbert space and the discussion are mainly on
the ``vector'' property. Samely, in the discussion about the
operators, no analysis has bee made on the concrete expression of
the operators; for instance; does the Hamiltonian operator commute
with the momentum operator? such subjects has been put down so far.

Well, since this chapter we will go into such discussions. We will
derive the concrete expression for both wave functions and operators
from the above general discussion. Here we will focus on the subject
that how to associate the concrete wave function of $\Psi(\bm{r},
t)$ with the corresponding vector of $\Psi$ in the Hilbert
space?\footnote{Here the analysis within in this chapter is taken
from the Xinlin Ke's book\cite{XingLinKe}.}

The general thread for derivation within this chapter has been given
in section \ref{vector_schalar_in_operator}. Here we only follow the
clues there.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Eigen states and eigen values for position
and momentum operator}
\label{eigen_states_in_position_momentum}
%
% the changing of eigen states are continuous for x and p
%
%
Now let's concentrate on one dimension motion of a single particle for
simplicity. Suggest that $\ket{x}$ and $\ket{p}$ are some normalized
eigen states for position operator $\hat{x}$ and momentum operator
$\hat{p}$, the corresponding eigen values are $x$ and $p$. hence we
have:
\begin{align}
\label{PRAMReq:1}
\hat{x}\ket{x} &= x\ket{x} \nonumber \\
\hat{p}\ket{p} &= p\ket{p}
\end{align}
Here the $\ket{x}$ and $\ket{p}$ are just some generalization for
the wave functions we have shown in the (\ref{sec:PWF_in_Hilbert}).
There the $\nu_{p}(x)$ is just some concrete function to stand for
the $\ket{p}$ in the position representation.

Now let's go to see what kind of value can be achieved by $x$ and
$p$.

From the commutation relation of $[\hat{x}, \hat{p}] = i\hbar$, we can
use $\hat{p}$ to construct some unitary operator of $\hat{q}$:
\begin{align}
  \label{PRAMReq:2}
\hat{q}(\xi) &= e^{\frac{i}{\hbar}\xi\hat{p}} \Rightarrow \nonumber \\
\hat{q}^{+}(\xi) &= e^{-\frac{i}{\hbar}\xi\hat{p}} \Rightarrow
\nonumber \\
\hat{q}(\xi)\hat{q}^{+}(\xi) &= I
\end{align}
Here $\xi$ is some arbitrary real number. From the mathematical
analysis, it's known that we can expand the $\hat{q}(\xi)$ into series
expansion (it holds true for all $\xi$ value):
\begin{equation}
  \label{PRAMReq:3}
  \hat{q}(\xi) = 1 + \left(\frac{i}{\hbar}\xi\hat{p}\right) +
 \frac{\left(\frac{i}{\hbar}\xi\hat{p}\right)^{2}}{2!} +
\frac{\left(\frac{i}{\hbar}\xi\hat{p}\right)^{3}}{3!} + \cdots
\end{equation}

Now we have to resort to an equation which will be introduced in the
following content (\ref{OPERATORMOREeq:11}):
\begin{equation}
  \label{PRAMReq:4}
[\hei{r}, f(\hei{p})] = i\hbar \frac{\partial f(\hei{p})}{\partial
\hei{p}}
\end{equation}

Here the operation is for the vector operator, but it can be directly
applied to the scalar operator; so by the (\ref{PRAMReq:4}) we can
get:
\begin{align}
  \label{PRAMReq:5}
    [\hat{x}, \hat{q}(\xi)] &=
i\hbar \frac{\partial\hat{q}(\xi)}{\partial \hat{p}} \nonumber \\
&=i\hbar \frac{i}{\hbar}\xi\hat{q}(\xi) \nonumber \\
&=-\xi\hat{q}(\xi)
\end{align}

So we have:
\begin{align}
  \label{PRAMReq:6}
\hat{x}\hat{q}(\xi) - \hat{q}(\xi)\hat{x} &= -\xi\hat{q}(\xi)
\Rightarrow \nonumber \\
\hat{x}\hat{q}(\xi) &= \hat{q}(\xi)\hat{x} - \xi\hat{q}(\xi)
\end{align}

Hence for the $\ket{x}$, we have:
\begin{align}
  \label{PRAMReq:7}
\hat{x}\hat{q}(\xi)\ket{x} &= (\hat{q}(\xi)\hat{x} -
\xi\hat{q}(\xi))\ket{x} \nonumber \\
&=\hat{q}(\xi) (x - \xi)\ket{x} \nonumber \\
&=(x - \xi)\hat{q}(\xi)\ket{x}
\end{align}

From (\ref{PRAMReq:7}) it turns out that if $\ket{x}$ is the eigen
state for the $\hat{x}$, then $\hat{q}(\xi)\ket{x}$ is also the
$\hat{x}$ eigen state which gives the eigen value of $x - \xi$. This
holds true for any real value of $\xi$. Hence, this conclusion implies
that the position can adopt any real value for the wave functions.

What's more, the (\ref{PRAMReq:7}) indicates that through
$\hat{q}(\xi)$ the eigen state of $\ket{x}$ is transformed into the
eigen state of $\ket{x - \xi}$, that is:
\begin{equation}
  \label{PRAMReq:8}
  \hat{q}(\xi)\ket{x} = \ket{x - \xi}
\end{equation}
Thus the $\hat{q}(\xi)$ is also called ``down operator'' for the
$\ket{x}$.

Similarly, by the same procedure we can construct the ``up operator''
for the $\ket{x}$, which is $\hat{q}^{+}(\xi) =
e^{-\frac{i}{\hbar}\xi\hat{p}}$. It transforms the $\ket{x}$ into the
$\ket{x + \xi}$:
\begin{equation}
  \label{PRAMReq:9}
    \hat{q}^{+}(\xi)\ket{x} = \ket{x + \xi}
\end{equation}

Hence, from the $\ket{x}$, and the corresponding up and down operators
we can get all the eigen states for the $\ket{x}$.

As for the momentum, we can build the similar up and down operators of
$\hat{t}(\xi)$ and $\hat{t}^{+}(\xi)$:
\begin{align}
  \label{PRAMReq:10}
  \hat{t}(\xi) = e^{\frac{i}{\hbar}\xi\hat{x}} &\quad \hat{t}^{+}(\xi) =
  e^{-\frac{i}{\hbar}\xi\hat{x}} \nonumber \\
 \hat{t}(\xi)\ket{p} = \ket{p + \xi} &\quad \hat{t}^{+}(\xi)\ket{p} =
 \ket{p - \xi}
\end{align}
through same procedure, we can know that $\ket{p}$ also adopts all the
real values.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Position representation and momentum representation}
\label{sec:PRAMR_in_position_representation}
%
% 1 how to express the \ket{x} and \ket{p} in matrix form
% 2 the physical meaning of matrix element
% 3 obtain the expression of \hat{x} for \ket{x}
%
From the above section, we have generally set up some important
feature for the $\ket{x}$ and $\ket{p}$, that both of their eigen
values are continuously changed. Here in this section, we will further
investigate their details by concentrating on one question: how to
associate the abstract Hilbert vector of $\ket{x}$ with the concrete
wave function of $\Psi(x)$?

Now let's consider the complete sets of $\ket{x}$ and $\ket{p}$. Since
the $\ket{x}$ is the eigen states for the $\hat{x}$, then it's the
representation for the $\hat{x}$, we can call it ``position
representation''; on the other hand, the $\ket{p}$ is the
representation for the $\hat{p}$ so that we can call it as ``momentum
representation''.

According to the discussion in the above paragraph, the eigen states
and eigen values for the $\ket{x}$ and $\ket{p}$ are continuously
changed. So we can express the closure relation as:
\begin{align}
\label{PRAMReq:11}
  \int^{+\infty}_{-\infty}\ket{x}\bra{x}dx &= 1 \nonumber \\
  \int^{+\infty}_{-\infty}\ket{p}\bra{p}dp &= 1
\end{align}

Firstly let's consider the position representation. Let's multiply
$\ket{x^{'}}$ to the (\ref{PRAMReq:11}), it leads to:
\begin{equation}
  \label{PRAMReq:12}
    \int^{+\infty}_{-\infty}\ket{x}\bra{x} x^{'}\rangle dx =
    \ket{x^{'}}
\end{equation}

However, since the position is continuously changed, we can use the
delta function to express it:
\begin{equation}
  \label{PRAMReq:13}
  \ket{x^{'}}  = \int^{+\infty}_{-\infty}
  \ket{x}\delta(x - x^{'}) dx
\end{equation}

Hence we have:
\begin{equation}
  \label{PRAMReq:14}
 \langle x|x^{'} \rangle = \delta(x - x^{'})
\end{equation}

From (\ref{PRAMReq:14}) we have gotten the expression for the inner
product. Now let's express some arbitrary quantum state of
$\ket{\Psi}$ via $\ket{x}$:
\begin{align}
  \label{PRAMReq:15}
\ket{\Psi} &= \left\{\int^{+\infty}_{-\infty}  \ket{x}\bra{x}\,
dx\right\}\ket{\Psi}\nonumber \\
&=\int^{+\infty}_{-\infty}  \langle x|\Psi \rangle
\ket{x} dx \nonumber \\
&= \int^{+\infty}_{-\infty} \Psi_{x}\ket{x} dx
\end{align}

According to the discussion in the section of \ref{REPRESENTATION:2},
the $\Psi_{x}$ is just the representation for the $\ket{\Psi}$; so the
$\ket{\Psi}$ can be expressed as:
\begin{equation}
  \label{PRAMReq:16}
  \ket{\Psi} \Leftrightarrow \begin{pmatrix}
                               \vdots \\
                               \Psi_{x} \\
                               \vdots \\
                               \Psi_{x^{'}} \\
                               \vdots \\
                             \end{pmatrix}
\end{equation}
Here an important feature is, because the index of $x$ is continuously
changed so the the matrix element of the $\Psi_{x}$ is also continuously
changed.

The $\bra{\Psi}$ can be similarly expressed as:
\begin{equation}
  \label{PRAMReq:17}
  \bra{\Psi} \Leftrightarrow
  \begin{pmatrix}
  \cdots & \Psi^{*}_{x} & \cdots & \Psi^{*}_{x^{'}} & \cdots \\
  \end{pmatrix}
\end{equation}
The inner product of $\langle\Psi|\Psi\rangle$ is:
\begin{equation}
  \label{PRAMReq:18}
  \langle\Psi|\Psi\rangle \Leftrightarrow
  \begin{pmatrix}
  \cdots & \Psi^{*}_{x} & \cdots & \Psi^{*}_{x^{'}} & \cdots \\
  \end{pmatrix}
  \begin{pmatrix}
  \vdots \\
  \Psi_{x} \\
  \vdots \\
  \Psi_{x^{'}} \\
  \vdots \\
  \end{pmatrix}
\end{equation}
For some arbitrary operator of $\hat{A}$, we have:
\begin{equation}
  \label{PRAMReq:19}
  \begin{split}
    \langle\Psi|\hat{A}|\Psi\rangle &=
\int^{+\infty}_{-\infty}dx \int^{+\infty}_{-\infty}dx^{'}
   \Psi^{*}_{x}\Psi_{x^{'}} \langle x|\hat{A}|x^{'}\rangle
\quad
\underrightarrow{A_{xx^{'}} = \langle x|\hat{A}|x^{'}\rangle} \\
&\Leftrightarrow
\int^{+\infty}_{-\infty}dx \int^{+\infty}_{-\infty}dx^{'}
  \begin{pmatrix}
  \cdots & \Psi^{*}_{x} & \cdots & \Psi^{*}_{x^{'}} & \cdots \\
  \end{pmatrix} \\
&  \begin{pmatrix}
    \cdots & \cdots & \cdots & \cdots & \cdots \\
    \cdots & \cdots & \cdots & \cdots & \cdots \\
    \cdots & \cdots & A_{xx^{'}} & \cdots & \cdots \\
    \cdots & \cdots & \cdots & \cdots & \cdots \\
    \cdots & \cdots & \cdots & \cdots & \cdots \\
  \end{pmatrix}
  \begin{pmatrix}
  \vdots \\
  \Psi_{x} \\
  \vdots \\
  \Psi_{x^{'}} \\
  \vdots \\
  \end{pmatrix}
  \end{split}
\end{equation}

Suggest that the $\hat{A}$ in the (\ref{PRAMReq:19}) is $\hat{x}$,
then we can have:
\begin{align}
  \label{PRAMReq:26}
\langle x^{'}|\hat{x}|x \rangle &= x \langle x^{'}|x \rangle \nonumber
\\
&= x \delta(x - x^{'})
\end{align}
Hence the matrix for $\hat{x}$ is some continuous diagonal matrix with
the its elements all equal to infinity.

Now let's try to associate the $\ket{x}$ with the wave function form
of $\Psi(x)$. Firstly, let's give some physical interpretation for the
$\Psi_{x}$.  Since the $x$ inside the $\Psi_{x}$ are continuously
varied, and $\Psi_{x}$ physically characterizes the weight the
$\ket{x}$ in the $\ket{\Psi}$; so it turns out that we can set up such
one to one correspondence:
\begin{equation}
\label{PRAMReq:20} \Psi_{x} \Leftrightarrow \Psi(x)
\end{equation}
This means, the abstract vector of $\ket{\Psi}$ in Hilbert space can
be considered as some function based on the variable of $x$. The two
expressions are identical to each other. Such one to one
correspondence generally holds true for an arbitrary vector of
$\ket{\Psi}$ in any Hilbert space.

This final conclusion also give an explanation that why in the
(\ref{sec:WPDAOYE_in_basic}) and (\ref{sec:PWC_in_basic}) we express
the wave function in terms of $\bm{r}$. This can be naturally
derived from the the more strict framework of quantum mechanics.

Generally such conclusion can be extended to three dimensional
space, that is:
\begin{equation}
\label{PRAMReq:21} \Psi_{\bm{r}} \Leftrightarrow \Psi(\bm{r})
\end{equation}
For multi-particles system, we can also get:
\begin{equation}\label{PRAMReq:22}
\Psi_{\bm{r_{1}}, \bm{r_{2}}, \cdots, \bm{r_{n}}} \Leftrightarrow
\Psi(\bm{r_{1}}, \bm{r_{2}}, \cdots, \bm{r_{n}})
\end{equation}

From now on we can just write the $\Psi_{x}$ as $\Psi(x)$. However,
here there's one thing still needed to be solved, that is how to
express the operator of $\hat{x}$ and $\hat{p}$ in the $\Psi_{x}$? Now
let's go to see how to express the $\hat{x}$.

Suggest that $\hat{A}\ket{\Psi} = \ket{\Phi}$, hence we have:
\begin{equation}\label{PRAMReq:25}
\begin{split}
  \ket{\Phi} &= \hat{A}\ket{\Psi} \quad
  \underrightarrow{\text{From closure relation}} \\
  \langle x|\Phi \rangle &= \int^{+\infty}_{-\infty} \langle
  x|\hat{A}|x^{'} \rangle \langle x^{'}|\Psi
  \rangle dx^{'}\\
  \Phi_{x} &= \int^{+\infty}_{-\infty} A_{xx^{'}}\Psi_{x^{'}}dx^{'}
\end{split}
\end{equation}
Now suggest the $\hat{A}$ is just the $\hat{x}$, then according to the
(\ref{PRAMReq:26}) we can have:
\begin{equation}\label{PRAMReq:23}
  \begin{split}
      \Phi_{x} &= \int^{+\infty}_{-\infty} x \delta(x -
      x^{'})\Psi_{x^{'}}dx^{'} \\
&= x \Psi_{x}
  \end{split}
\end{equation}
Therefore, if we have $\hat{x}\ket{\Psi} = \ket{\Phi}$, then in the
position representation we can conveniently express the position
operator as $\hat{x}\Psi(x) = x\Phi(x)$. Generally, for the position
operator of $\hei{r}\ket{\Psi} = \ket{\Phi}$, we have
$\hei{r}\Psi(\bm{r}) = \bm{r}\Phi(\bm{r})$ to correspond to it.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{How to express momentum operator in the position
  representation}
\label{sec:momentum_operator_in_position_momentum}
%
%
% how to express the momentum operator in position representation?
%
%
%
Now in this section, we are going to seek the concrete expression for
the $\hat{p}$ in the representation of $\ket{x}$. Firstly, let's
concentrate how to express $\langle x|p \rangle$.

By using the up and down operators in the
(\ref{eigen_states_in_position_momentum}), we can have:
\begin{equation}
  \label{PRAMReq:24}
  \langle x|p \rangle = \langle x|e^{\frac{i}{\hbar}p\hat{x}}|0_{p}
  \rangle
\end{equation}
Here $\ket{0_{p}}$ is just to express the eigen state gives the $0$
eigen value: $\hat{p}\ket{0_{p}} = 0\ket{0_{p}}$.

On the other hand, the operator of $e^{\frac{i}{\hbar}p\hat{x}}$ is
also some operator for the $\bra{x}$, and by using that
$\bra{x}\hat{x} = \bra{x}x$, we can have:
\begin{equation}
  \label{PRAMReq:27}
  \langle x|p \rangle = \langle x|e^{\frac{i}{\hbar}p\hat{x}}|0_{p}
  \rangle  = e^{\frac{i}{\hbar}px}\langle x|0_{p} \rangle
\end{equation}

For the $\langle x|0_{p} \rangle$, by using the up operator for
$\bra{x}$, we can have:
\begin{equation}
  \label{PRAMReq:28}
  \langle x|0_{p} \rangle = \langle 0_{x}|e^{\frac{i}{\hbar}x\hat{p}}|0_{p}
  \rangle
\end{equation}
Similarly the $e^{\frac{i}{\hbar}x\hat{p}}$ is also the operator on
the $\ket{p}$, so we have $e^{\frac{i}{\hbar}x\hat{p}}\ket{0_{p}} =
e^{\frac{i}{\hbar}x\times 0}\ket{0_{p}} = \ket{0_{p}}$; therefore we have:
\begin{equation}
  \label{PRAMReq:29}
  \langle x|p \rangle = e^{\frac{i}{\hbar}px}\langle 0_{x}|0_{p} \rangle
\end{equation}

Now let's go to see how to express the $\langle 0_{x}|0_{p} \rangle$:
\begin{equation}
  \label{PRAMReq:30}
  \begin{split}
    \langle p^{'}|p \rangle = \delta (p^{'} - p) &=
    \int^{+\infty}_{-\infty} \langle p^{'}|x \rangle \langle x|p
    \rangle dx \\
    &= \int^{+\infty}_{-\infty}
    e^{-\frac{i}{\hbar}p^{'}x}e^{\frac{i}{\hbar}px} |\langle
    0_{x}|0_{p}
    \rangle|^{2} dx \\
    &= \int^{+\infty}_{-\infty} e^{\frac{i}{\hbar}(p-p^{'})x}|\langle
    0_{x}|0_{p}
    \rangle|^{2} dx \quad
    \underrightarrow{\text{From delta function}} \\
    &= |\langle 0_{x}|0_{p} \rangle|^{2} 2\pi\hbar\delta(p^{'} - p)
    \quad \Rightarrow \\
    \langle 0_{x}|0_{p} \rangle &= \frac{1}{\sqrt[2]{2\pi\hbar}}
  \end{split}
\end{equation}

Based on the (\ref{PRAMReq:30}), let's go to see how to express the
$\langle x|\hat{p}|x^{'}\rangle$:
\begin{equation}
  \label{PRAMReq:31}
  \begin{split}
    \langle x|\hat{p}|x^{'}\rangle &= \int^{+\infty}_{-\infty}dp
    \int^{+\infty}_{-\infty}dp^{'} \langle x|p\rangle \langle
    p|\hat{p}|p^{'}\rangle \langle p^{'}|x^{'}\rangle \quad
    \text{by closure relation in \ref{PRAMReq:1}}  \\
    &= \frac{1}{2\pi\hbar}\int^{+\infty}_{-\infty}dp
    \int^{+\infty}_{-\infty}dp^{'}
    e^{\frac{i}{\hbar}px}e^{-\frac{i}{\hbar}p^{'}x^{'}}
    p^{'}\delta(p-p^{'}) \\
    &= \frac{1}{2\pi\hbar}\int^{+\infty}_{-\infty}pdp
    e^{\frac{i}{\hbar}p(x-x^{'})}  \\
    &= \frac{1}{2\pi\hbar}\int^{+\infty}_{-\infty}dp
    \frac{\hbar}{i}\frac{\partial e^{\frac{i}{\hbar}p(x-x^{'})} }
    {\partial x} \\
    &= \left(\frac{\hbar}{i}\frac{\partial}{\partial x}\right)
    \frac{1}{2\pi\hbar}\int^{+\infty}_{-\infty}dp
    e^{\frac{i}{\hbar}p(x-x^{'})}  \\
    &=\frac{1}{2\pi\hbar}\left(-i\hbar \frac{\partial}{\partial
        x}\right)
    2\pi\hbar \delta(x - x^{'}) \\
    &=-i\hbar \frac{\partial}{\partial x}\delta(x - x^{'})
  \end{split}
\end{equation}

Now let's use the (\ref{PRAMReq:31}) to get the concrete expression
for the momentum operator. Similar to the (\ref{PRAMReq:25}), we
suggest that we have $\hat{p}\ket{\Psi} = \ket{\Phi}$, so we have:
\begin{equation}\label{PRAMReq:32}
\begin{split}
  \ket{\Phi} &= \hat{p}\ket{\Psi} \quad
  \underrightarrow{\text{From closure relation}} \\
  \langle x|\Phi \rangle &= \int^{+\infty}_{-\infty} \langle
  x|\hat{p}|x^{'} \rangle \langle x^{'}|\Psi
  \rangle dx^{'}\\
  \Phi_{x} &= \int^{+\infty}_{-\infty} p_{xx^{'}}\Psi_{x^{'}}dx^{'}
  \quad
  \underrightarrow{\text{From \ref{PRAMReq:31}}} \\
  &= \int^{+\infty}_{-\infty} \left(-i\hbar \frac{\partial}{\partial
      x}\delta(x - x^{'})\right) \Psi_{x^{'}}dx^{'} \\
  &=  -i\hbar \frac{\partial}{\partial x}\Psi_{x}
\end{split}
\end{equation}
Hence we get the expression of momentum operator for wave function
of $\Psi(x)$:
\begin{equation}\label{PRAMReq:33}
\hat{p}\Psi(x) = -i\hbar \frac{\partial}{\partial x}\Psi(x)
\end{equation}

For three-dimensional space, such conclusion can be generally
extended as:
\begin{equation}
  \label{PRAMReq:34}
  \hei{p} =  -i\hbar \nabla
\end{equation}

In quantum mechanics, since all the other operators are constructed
by the position operator and the momentum operator, then from the
$\hei{r}$ and $\hei{p}$ we are able to construct the expression for
all the other operators in the position representation.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{How to express the wave function and operators
in momentum expression}
\label{sec:momentum_operator_in_position_momentum}
%
%
% How to express the wave function and operators
% in momentum expression
%
%
%
In the above content, we have fully solved the question that how to
associate the $\ket{\Psi}$ with wave function of $\Psi(\bm{r})$, and
how to express the $\hei{r}$ and $\hei{p}$ in terms of the position
representation.

However, according to the axiom \ref{axiom5}, the position and
momentum in quantum mechanics are ``paired'' together. That means,
if we can express the $\hei{r}$ and $\hei{p}$ in terms of $\bm{r}$,
then all the other operators will be some function of $\bm{r}$; and
the corresponding wave functions will be in the form of
$\Psi(\bm{r})$; which is one to one correspondent to the
$\ket{\Psi}$. To be contrary, the judgment holds true for the
momentum $\bm{p}$. We can express the relation as:
\begin{align}\label{PRAMReq:35}
 \Psi(\bm{r}) \Leftrightarrow &\ket{\Psi}
 \Leftrightarrow \Psi(\bm{p}) \nonumber \\
\hei{r} = \hei{r}(\bm{r}), \hei{p} = \hei{p}(\bm{r})
&\Leftrightarrow \hei{r} = \hei{r}(\bm{p}), \hei{p} =
\hei{p}(\bm{p})
\end{align}

The derivation process for getting the expression for the $\hei{r}$
and $\hei{p}$ in terms of the momentum representation is nearly the
same with the above derivation for position representation. By
starting from the (\ref{PRAMReq:11}), and each step we exchange the
position representation $\ket{x}$ with the momentum representation
$\ket{p}$, we are indeed getting the conclusions shown in
(\ref{PRAMReq:35}). More specifically, we point out that in the
momentum representation, the operator of $\hei{r}$ and $\hei{p}$ are
expressed as:
\begin{align}\label{PRAMReq:36}
\hei{p} &= p \nonumber \\
\hei{r} &= -i\hbar \frac{\partial}{\partial \bm{p}}
\end{align}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../../main"
%%% End:


\section{Line Integrals}
\input{./lineSurfaceIntegrals/lineIntegralsScalarFunctions}
\input{./lineSurfaceIntegrals/lineIntegralsVFs}
\input{./lineSurfaceIntegrals/conservativeVFs}
\input{./lineSurfaceIntegrals/ftcLineIntegrals}
\input{./lineSurfaceIntegrals/testconservativeVF}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%% ICML 2013 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Use the following line _only_ if you're still using LaTeX 2.09.
%\documentstyle[icml2013,epsf,natbib]{article}
% If you rely on Latex2e packages, like most moden people use this:
\documentclass{article}

% For figures
\usepackage{graphicx} % more modern
%\usepackage{epsfig} % less modern
\usepackage{subfigure}

% For citations
\usepackage{natbib}

% For algorithms
\usepackage{algorithm}
\usepackage{algorithmic}

% As of 2011, we use the hyperref package to produce hyperlinks in the
% resulting PDF.  If this breaks your system, please commend out the
% following usepackage line and replace \usepackage{icml2013} with
% \usepackage[nohyperref]{icml2013} above.
\usepackage{hyperref}

% Packages hyperref and algorithmic misbehave sometimes.  We can fix
% this with the following command.
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Employ the following version of the ``usepackage'' statement for
% submitting the draft version of the paper for review.  This will set
% the note in the first column to ``Under review.  Do not distribute.''
\usepackage{icml2013}
% Employ this version of the ``usepackage'' statement after the paper has
% been accepted, when creating the final version.  This will set the
% note in the first column to ``Proceedings of the...''
% \usepackage[accepted]{icml2013}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Submission and Formatting Instructions for ICML 2013}

\begin{document}

\twocolumn[
\icmltitle{Analysis of Self Implemented Logistic Regression}

% It is OKAY to include author information, even for blind
% submissions: the style file will automatically remove it for you
% unless you've provided the [accepted] option to the icml2013
% package.
\icmlauthor{Carl Cortright}{carl.cortright@colorado.edu}

% You may provide any keywords that you
% find helpful for describing your paper; these are used to populate
% the "keywords" metadata in the PDF but will not be shown in the document
\icmlkeywords{boring formatting information, machine learning, ICML}

\vskip 0.3in
]

\section{Introduction}
Logistic regression is a common machine learning technique used for both binary and multiclass classification problems. In this paper, I demonstrate how logistic regression can be used to identify the topic of a document. Specifically, I use a technique called stochastic gradient decent to build a fast and accurate model for classification. Analysis of the forulas used will show how large beta values are used heavily in predicting a class.

\section{Formulas}

\subsection{The Sigmoid Fuction}
The sigmoid function is used heavily in logistic regression because it sqeezes any real value into a number between 0 and 1 (like a probability) and is easily differentiable. To calculate the probability of a given feature vector, we feed the dot product of beta values and the training examples into the sigmoid function.

\subsection{The Delta}
On each beta update, we change each beta value by some amount delta. We know that delta needs to move beta in the right direction, so we take the actual label and subtract the calculated probability and then we multiply by the number of times that feature appears in the feature vector. After multiplying this value by the step function, we add it to the beta vector, allowing us to shift the beta vector is the right direection based on the training example.

\subsection{Regularization}
In order to ensure that our model generalizes well to unseen data, we need to prevent overfitting. The most common way to prevent overfitting in logistic regression is by using regularization, or penalizing large beta values. The way we do this is by shrinking large beta values. We define a shrinkage value that gets larger based on the iteration. We then take all of the features that we just updated and multiply them by our shrinkage number raised to the gap power (how long it has been since we last updated). This will effectively scale the beta value down if it is very large.

\section{Questions}

\subsection{What is the learning rate?}
The learning rate determines how quickly our model will converge. In this program, the learning rate is determined in our lambda function step and is set to 0.05.

\subsection{How many passes over the data did we need?}
In this program, we are able to converge to a reasonable answer fairly quickly, using 1 or 2 passes over the data. Heuristically, I would guess the sweetspot is somewhere between 5-10 passes with mu being less than 0.1. Any more and we would risk overfitting.

\subsection{What are the best predictors?}
The best predictors of each class, are the highest weighted beta values for that class. Hockey was denoted by negative beta values and when we sorted them we found that the highest predictors were:

\begin{tabular}{|c|c|}
  Feature & Beta \\
  \hline
  hockey & -2.39701629657 \\
  playoffs & -1.44564218453 \\
  golchowy & -1.17516769744 \\
  ice & -1.03574704859 \\
  next & -1.02160900872 \\
  goals & -1.01665832771 \\
  pick & -1.01647140833 \\
  playoff & -0.976664795926 \\
  points & -0.954740265196 \\
  biggest & -0.934753162921 \\

\end{tabular}

And for baseball, the beta values were positive:

\begin{tabular}{|c|c|}
  Feature & Beta \\
  \hline
  runs & 1.43294142746 \\
  hit & 1.10998401766 \\
  pitching & 1.03061549829 \\
  baseball & 1.02607170122 \\
  catcher & 0.854081642361 \\
  ball & 0.793975224712 \\
  anyone & 0.784188784459 \\
  saves & 0.764849612496 \\
  run & 0.697562386451 \\
  book & 0.684478585989 \\

\end{tabular}

Interestingly, the words that best predicted baseball were verbs associated with the activity, with the actual word baseball coming in at number 4.

\subsection{What are the worst predictors?}
The worst predictors for a given class are the best predictors for the other class. In this case the worst predictors for hockey are the best predictors for baseball and vice versa.

\subsection{What happens if mu is 0?}
If mu is zero, our regularization will have no effect. This means a lot of things including that we might risk overfitting. It also means it takes less passes for our model to converge.

\section{Conclusion}
In this lab, we used a logistic regression model to predict whether a document was talking about hockey or baseball. We used this model to predict which unigram features were most predictive of each class, and created a highly accurate classifier to go along with our testing and training data. We learned about regularization and stochastic gradient decent. I was most supprised at how accurate the model became even on a small dataset.

\end{document}

% This document was modified from the file originally made available by
% Pat Langley and Andrea Danyluk for ICML-2K. This version was
% created by Lise Getoor and Tobias Scheffer, it was slightly modified
% from the 2010 version by Thorsten Joachims & Johannes Fuernkranz,
% slightly modified from the 2009 version by Kiri Wagstaff and
% Sam Roweis's 2008 version, which is slightly modified from
% Prasad Tadepalli's 2007 version which is a lightly
% changed version of the previous year's version by Andrew Moore,
% which was in turn edited from those of Kristian Kersting and
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.


\chapter{Create a Global Grid}
\label{chap:global-grids}
In this section, we show how to create a global grid with \Atlas.
We remind the reader that global grid refers to a grid covering
the entire sphere (e.g. the entire earth for instance).
In addition, with the term \textit{grid} we intend a
list of points without any specific connectivity rule.

As explained in \chap{chap:structure}, \Atlas supports various
global grids. Within the library, these grids are identified
by some convenient keys. In the rest of the chapter we will
divide the various global grids available in the following
macro areas:
%
\begin{itemize}
\item Structured grids,
\item Unstructured grids.
\end{itemize}
%
and we will specify the keys needed to automatically generate
a given grid in both C++ and Fortran. Note that it is also possible
to create a grid manually, however this approach is not currently
explained in this document.
It is also important to remark that for the Structured grids, the memory
footprint is negligible since the information regarding the points are not
stored but are computed at runtime when requested.
The last typology, unstructured, has instead a non-negligible memory
footprint since in this case the points are effectively stored in memory.

\section{Structured Grids}
There exists various subsets of structured grids in \Atlas.
All of them can be constructed using the same method
called \inltc{Structured::create()}. In particular, we need to pass
a predefined key identifying the grid we want to create
to the constructor in order to generate the object grid.
This object, in turn, contains various methods that allows
the user to retrieve the number of points of the generated
grid, the number of longitudes and latitudes, and the
coordinates a given longitude and latitude.

The predefined keys existing in \Atlas for Gaussian grids
are reported in the following:
%
\begin{itemize}
\item Classic reduced Gaussian grid --- \inltc{N\#},
\item Regular Gaussian grid --- \inltc{F\#},
\item Octahedral reduced Gaussian grid --- \inltc{O\#},
\end{itemize}
%
where \inltc{\#} represents the number of latitudes
in one hemisphere.
%
\begin{itemize}
\item RegularLonLat --- \inltc{L\#} or \inltc{L\#$\times$\#}
\item ShiftedLonLat --- \inltc{S\#} or \inltc{S\#$\times$\#}
\end{itemize}
%
where in the alternative key, the first \inltc{\#} represents the total number
of longitudes, while the second \inltc{\#} specifies the total number of
latitudes from pole to pole.

In the next two subsections we will present two simple
examples, the first in C++ and the second in Fortran,
to generate global grids through the predefined keys
just outlined. We will also show how to retrieve some
useful information regarding the grid generated.

\subsection{C++ version}
The \lista{Structured-C} shows how to construct a generic
structured grid.
%
\lstinputlisting[caption=Generating a Gaussian grid with
\Atlas using C++, style=CStyle, label=Structured-C]{global-grids-Structured.cc}
%
We create the global
grid object (see line 14). In particular, we call the
grid constructor passing the command-line string (that
represents the key of the grid we want to create) to it.
The default value is set to \inltc{O32}, that represents
a octahedral reduced Gaussian grid with 32 latitudes in one hemisphere
(i.e. 64 latitudes in total).
The command-line string (i.e. the key) must be contained
in the predefined keys for reduced grid generation as specified
before. If the user uses a non-existent keyword, there will
appear an error message.
The grid object, \inltc{grid}, is
a \inltc{Structured} type and it will allow
us to access some useful information about the grid.

It is now possible to run this simple program by using one
command-line argument representing the keyword that identifies
an \Atlas predefined grid. For instance, we can execute the
following command line
%
\begin{lstlisting}[style=BashStyle]
./atlas_c-global-grids-Structured
\end{lstlisting}
%
This will generate an octahedral reduced Gaussian grid with
32 latitudes on one hemisphere (i.e. 64 latitudes in total).

The output to the screen of the code we just executed
contains some useful information regarding the grid.

In particular, the first three numbers represent the
number of latitudes, longitudes and points of the structured
grid, respectively. Note that we can directly access them
through the \inltc{grid} object (see lines 19, 20, 21).
\begin{tipbox}
For the number of latitudes and points we do not need to provide
any argument to the function retrieving the information. On the
other hand, for the number of longitudes we do need to additionally
provide the index of a specific latitude, since the number of
longitudes may depend on the latitude (see \chap{chap:structure}
for more details on how a structured grid is constructed).
\end{tipbox}

The last three numbers represent the coordinates of a given
latitude, longitude and both longitude and latitude together,
respectively. Note again how these information can be directly
accessed through the object \inltc{grid}.
Also, for retrieving a given latitude coordinate it is only
necessary to specify one index that refers to that particular
latitude (see line 23). On the other hand, for a given longitude
we need to provide both the index of the latitude and the index
of the longitude, since the longitude may depend on the latitude
(see line 21 and 24).

\subsection{Fortran version}
The \lista{Structured-F} shows how to construct a generic
structured grid.

%
\lstinputlisting[caption=Generating a Gaussian grid with \Atlas
using Fortran, style=FStyle, label=Structured-F]{global-grids-Structured.F90}
%

We create the global
grid object (see line 9). In particular, we call
the grid constructor passing the command-line string
(that represents the key of the grid we want to create) to
it. The default value is set to \inltc{O32}, that represents
a octahedral reduced Gaussian grid with 32 latitudes in one hemisphere
(i.e. 64 latitudes in total).
The grid object, \inltc{grid}, is
an \inltc{atlas\_grid\_Structured} type and it will allow
us to access some useful information about the grid.

It is now possible to run this simple program with the following command line
%
\begin{lstlisting}[style=BashStyle]
./atlas_f-global-grids-Structured
\end{lstlisting}
%
This will generate an octahedral reduced Gaussian grid
with 32 latitudes on one emisphere (i.e. 64 latitudes
in total).

The output to the screen of the code we just executed
contains some useful information regarding the grid.

In particular, the first three numbers represent the
number of latitudes, longitudes and points of the grid,
respectively. Note that we can directly access them
through the \inltc{grid} object (see lines 11, 14, 17).
%
\begin{tipbox}
For the number of latitudes and points we do not need to provide
any argument to the function retrieving the information. On the
other hand, for the number of longitudes we do need to additionally
provide the index of a specific latitude, since the number of
longitudes may depend on the latitude (see \chap{chap:structure}
for more details on how a structured grid is constructed).
\end{tipbox}
%
The last two numbers represent the coordinates of a given
latitude and longitude, respectively. Note again how these
information can be directly accessed through the object
\inltc{grid}.
Also, for retrieving a given latitude coordinate it is only
necessary to specify one index that refers to that particular
latitude (see line 20). On the other hand, for a given longitude
we need to provide both the index of the latitude and the index
of the longitude, since the longitude may depend on the latitude
(see line 23).

You can now play with line 9 to generate
different types of global structured grid using the keys introduced
at the beginning of this section!

\section{Unstructured Grids}
Coming soon ...


\documentclass[11pt]{article}

\usepackage{cmap}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[english]{babel}

\usepackage{microtype}
\usepackage{csquotes}
\usepackage{enumitem}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{mathtools}

\setlength{\parindent}{0cm}

\newtheorem{definition}{Definition}
\newtheorem{example}{Example}

\begin{document}

\section{Adverse Selection}

In reality, information is often asymmetrically held by market participants. Examples of such situations are

\begin{itemize}
	\item Firms do not know the abilities of workers to be hired
	\item Investors / creditors cannot observe the abilities / projects of firms' managers
	\item Automobile insurance companies do not know the individual driving skills of their customers
	\item Buyers of used car can rarely check the cars' qualities perfectly
\end{itemize}

Such hidden information can lead to adverse selection, signalling and screening. Following, we will examine the case of adverse selection.

\subsection{Basic Labour Market Model}

Assume that in a market, there are many workers of different types $\theta$, e.g. with different productivity, such that

	$$ \theta \in [\underline{\theta}, \overline{\theta}] \text{ with }0 \leq \underline{\theta} < \overline{\theta} < \infty.  $$

The proportion of workers with productivity of $\theta$ or less is given by the cumulative distribution function $F(\theta)$. Additionally, we assume there to be many identical potential firms that can hire workers seeking to maximise their expected profits. A worker can choose to work either at a firm or at home, and the utility of worker $\theta$ is thereby given by

	$$ u(w, \theta) = \begin{cases} \omega & \text{ if he accepts wage } \omega \\ r(\theta) & \text{ if he does not accept } \omega \end{cases} $$

Firms produce with labour only and have constant production technologies

	$$ \theta = \text{ output (=revenue) of worker } \theta $$

Pareto-optimum: Set of workers employed is $\theta = \left\{ \theta \colon \theta \geq r(\theta) \right\}$.

\subsubsection{Case 1: $\theta$ is publicly observable}

The resulting wage will depend on each type, i.e. $\omega(\theta)$. Because the labour of each different type of worker is a distinct, publicly known good, there is a distinct equilibrium wage. As the profit of a firm ($\pi$) from a type $\theta$ worker:

	$$ \pi = \theta - \omega(\theta) $$

They will only seek employment if $\theta > \omega(\theta)$. The resulting demand for labour ($z$) of type $\theta$ is hence:

	$$ z(\theta) = \begin{cases} 0 & \text{ if } \theta < \omega(\theta) \\ [0, \infty) & \text{ if } \theta = w(\theta) \\ \infty & \text{ if } \theta > \omega(\theta) \end{cases} $$

Therefore, given the competitive, constant returns nature of the firm, in a \textit{competitive} equilibrium the different wage offers (i.e. $\pi = 0$) are

	$$ \omega(\theta) = \theta, $$

and the set of workers accepting employment in a firm is

	$$ \Theta = \left\{ \theta \colon r(\theta) \leq \theta \right\} $$

As expected from the first fundamental welfare theorem, the outcome is, due to the competitive nature of the market, a Pareto optimal equilibrium. Note: For this result firms need not know $r(\theta)$, and thus, the market can successfully aggregate and process information!

\subsubsection{Case 2: $\theta$ is ex ante observable only to the worker}

We have to begin by noting that the resulting equilibrium in this competitive environment is of asymmetric information. A firm will learn about $\theta$ only \textit{ex post}, therefore  this is not useful for \textit{ex ante} contracting. This means the wage cannot depend on $\theta$, and hence, there must be a single wage $\omega$ offered to all workers. Again, a worker of type $\theta$ is willing to work for a firm if and only if $r(\theta) \leq \omega$. Hence, the set of worker types who are willing to accept employment at wage rate $\omega$ is

	$$ \Theta(\omega) = \left\{ \theta \colon r(\theta) \leq \omega \right\} $$

and gives us the labour supply. Let $\mu$ be the expected average productivity of employed workers (to be determined). If a firm assumes $\mu$, its demand for labour is given by

	$$ z(\omega)= \begin{cases} 0 & \text{ if } \mu < \omega \\ [0, \infty) & \text{ if } \mu = \omega \\ \infty & \text{ if } \mu > \omega \end{cases} $$

Now, if worker types in set $\Theta^*$ are accepting employment offers in a competitive equilibrium, and if firms' beliefs about the productivity of potential employees correctly reflect the actual average productivity of the workers hired in this equilibrium, then we must have

	$$ \mu = \mathbb{E} \left[\theta ~|~\theta \in \Theta^* \right] $$

	Hence, the labour demand implies that the demand can equal its supply in an equilibrium with a positiv level of employment if and only if

	$$ w = \mathbb{E} \left[ \theta ~|~ \theta \in \Theta^* \right] $$

	This leads to the notion of a competitive equilibria

	\begin{definition}
		A \textbf{competitive equilibrium} is a wage $\omega^*$, an employment set $\Theta^*$, and a belief $\mu^*$ such that

		\begin{align*}
			\Theta^* & = \left\{ \theta \colon r(\theta) \leq \omega^* \right\} \\
			\omega^* & = \mu^* \\
			\mu^* & = \begin{cases} \mathbb{E} \left[ \theta ~|~\theta \in \Theta^* \right] & \text{ if } \Theta^* \neq \emptyset \\ \mathbb{E} \left[ \theta \right] & \text{ if } \Theta^* = \emptyset \end{cases}
		\end{align*}
	\end{definition}

Note, that the expectation in the definition above seems not to be well defined when no workers are accepting employment in an equilibrium, i.e. when $\Theta^* = \emptyset$. In the following discussions however, we can assume for simplicity that in this circumstance each firm's expectation of potential employees' average productivity is simply the unconditional expectation $\mathbb{E} \left[ \theta \right]$, and we take

	$$ \omega^* = \mathbb{E} \left[ \theta \right] $$

in any such equilibrium. Typically a competitive equilibrium as defined above will fail to be Pareto optimal which is seen in the following example.

\begin{example}
	Assume $r(\theta) = r$ for all $\theta$, with $\underline{\theta} < r < \overline{\theta}$, i.e. every worker is equally productive at home. The Pareto optimal allocation of labour in this setting has workers with $\theta \geq r$ accepting employment at a firm and those with $\theta < r$ not doing so. In the competitive equilibrium, a worker is willing to accept employment if $\omega \geq r$. This means at a given wage, $\Theta(\omega)$, is either:

	$$ \Theta = \left[ \underline{\theta}, \overline{\theta} \right] \text{ if } \omega \geq r ~\text{ or }~ \Theta = \emptyset \text{ if } \omega < r $$

	Thus, in either case, $\mu = \mathbb{E}\left[ \theta ~|~\theta \in \Theta^*(\omega) \right] = \mathbb{E} \left[ \theta \right]$, i.e. for all $\omega$ and unconditional. By the definition of competitive equilibria  the equilibrium wage rate must be

		$$ \omega^* = \mathbb{E} \left[ \theta \right] $$

	If $\mathbb{E}[\theta] \geq r$, then all workers accept employment at a firm; if $\mathbb{E}[\theta] < r$, then none do. Which type of equilibrium arises depends on the relative fraction of good and bad workers. For example, if there is a high fraction of low-productivity workers then, because firms cannot distinguish good workers from bad, they will be unwilling to hire any workers at a wage that is sufficient to have them accept employment, i.e.

		$$ \mathbb{E} \left[ \theta \right] < r $$

	On the other hand, if there are very few low-productivity workers, then the average productivity of the workforce will be avoce $r$, and so the firms will be willing to hire workers at a wage that they are willing to accept, i.e.

	$$ \mathbb{E} \left[ \theta \right] \geq r $$

	In one case, too many workers are employed relative to the Pareto optimal allocation, and in the other too few, and therefore,  neither case is Pareto-optimal since

	$$\Theta^* \neq \left\{ \theta \colon \theta \geq r \right\}. $$

	The cause of this failure of the competitive allocation to be Pareto optimal is simple to see: because firms are unable to distinguish among workers of differing productivities, the market is unable to allocate workers efficiently between firms and home production. ~\bigskip

	Note: If workers could not observe their own type $\theta$ the same allocation would result. Thus, knowledge does not do harm in this example, but cannot be optimally processed by the market.
\end{example}

\section{Signalling}

In the model with adverse selection, good workers were underpaid because they were unable to credibly inform firms about their productivity. Hence, they should have an incentive to \enquote{signal} their productivity, if this is possible. This will now be investigated in a simple setting.

\subsection{Simple Signalling}

\subsubsection{Model}

In this section we assume that there are two firms competing for hiring workers. Furthermore, there are only two types of workers, $L$ and $H$, with $0 < \theta_L < \theta_H$ and $\lambda = \mathbb{P}(\theta = \theta_H)$ where $0 < \lambda < 1$. We assume

	$$ 0 \leq r(\theta_L) \leq r(\theta_H) < \mathbb{E}[\theta]. $$

The important extension of our previous model is that before entering the job market a worker can signal, this can have the form of getting some education and the amount of education that a worker receives is observable. Regardless of the type, signalling does nothing for a worker's productivity. Type $\theta_H$  can produce such a signal at cost $c$, and we assume that type $\theta_L$ cannot produce a signal, where

	$$ 0 \leq c < \theta_H - \mathbb{E}[\theta] $$

\subsubsection{Timing}

In this setting, we can identify four stages developing from setup to possible employment:

\begin{enumerate}
	\item Nature chooses workers' types, observed only by them
	\item Workers may produce a signal
	\item Firms observe signals and simultaneously announce wage offers
	\item Workers decide whether to join a firm or not. If they are indifferent, they choose either option with probability $p = 0.5$.
\end{enumerate}

\textbf{Stage 3}: ~\smallskip

At the fourth stage the decision is merely where a worker maximises his profit, we will therefore start with stage three. Here, we notice that wages cannot depend on the type $\theta$ as those aren't observable. However, the wages can depend on the signal:

\begin{itemize}
	\item Let $\omega_s$ be the wage offered to a worker who has signalled
	\item Let $\omega_{nonS}$ be the wage offered to a worker who has not signalled.
\end{itemize}

The competitive equilibrium that emerges is hence
$$ \omega_s = \theta_H ~\text{ and }~ \omega_{nonS} \in \left[\theta_L, \mathbb{E}[\theta] \right] $$

\textbf{Stage 2}: ~\smallskip

As by assumption $0 \leq c < \theta_H - \mathbb{E}[\theta]$ holds, all workers of type $H$ will now signal.

\subsubsection{Consider two sub-cases}

\begin{itemize}
	\item $r(\theta_L) \leq r(\theta_H) \leq \theta_L$: Signalling is wasteful
	\item $\theta_L < r(\theta_L) < r(\theta_H) < \mathbb{E} \left[ \theta \right]$: Signalling may be useful
\end{itemize}

Hint: In both cases, the equilibrium in the pure adverse selection model (without signalling) is

	$$ \omega^* = \mathbb{E}[\theta] $$

\end{document}


\subsection{Boltzmann Machine}

%Lucas.

\begin{figure}[htbp]
	\begin{center}
		\includegraphics[width=0.5\textwidth]{inc/Restricted_Boltzmann_machine.png}
		\caption{Illustration of a Restricted Boltzmann Machine.\protect\footnotemark}
		\label{fig:restricted_boltzmann_machine}
	\end{center}
\end{figure}
\footnotetext{Original image (CC BY-SA): \url{https://en.wikipedia.org/wiki/File:Restricted_Boltzmann_machine.svg}}

Unfortunately, the Boltzmann Machine is very slow to train and is thus only practical for simpler problem domains. If the BM is scaled beyond any trivial domain it becomes too slow and almost stops learning. This is due to the fact that all units of the BM are fully connected to each other which does not scale well.

In order to use the BM for bigger tasks a restriction has to be made. Namely, connections between units in the same layer can not be allowed. This is called the Restricted Boltzmann Machine (or ``Harmonium'' as the original author referred to it) \cite{smolensky1986information}.

Versions of the Restricted Boltzmann Machine (RBM) have been successfully used in many applications such as deep learning \cite{hinton2012better} and speech recognition \cite{dahl2010phone}.

The general idea when using RBM's in deep learning is to ``stack'' several RBM's on top of each other. The activities of the units in the hidden layer of one RBM can be used as input vector for the next RBM. This way the overall system does not have the problem with scalability that the ordinary Boltzmann Machine suffered as well as the added bonus that the generative model is improved each time a new layer is added on top of the existing ones.

A variant of the Deep Boltzmann Machine \cite{salakhutdinov2009deep} (which is used for deep learning) called the Shape Boltzmann Machine (SBM) has been shown to be able to ``restore'' parts of an image when shown only a part of it \cite{eslami2014shape}. It does not ``restore'' the image to its original form, but fill in the blanks with its interpretation of the data it has been given. This can bee seen as a form of auto-associative memory.


\subsection{Intel Lab Data}
\label{sec:intel-lab-data-evaluation}

We also evaluated our outlier detection framework on sensor data from the publicly available Intel Lab Data set\footnote{\url{http://db.csail.mit.edu/labdata/labdata.html}}. The Intel Lab Data contains data collected from 54 sensors spread throughout the Intel Berkeley Research Lab. Each data entry contains information including temperature, humidity, light and voltage taken from a Mica2Dot sensor and weatherboard. The dataset contains a total of approximately 2.3 million measurements.

The Intel lab dataset has known outliers from faulty sensor readings due to periods of critically low voltage. During these periods, the sensors go haywire and produce faulty measurements. For example, the temperature may be registered as over $120$ degrees Celsius, which is obviously abnormal behavior in a human environment such as where the sensors were deployed.

We analyzed a sample of 1000 data points selected at random from the sensor data; due to the numerical nature of this data, the Simple Gaussian and Mixture models are better-suited to analyzing it than the Histogram model.

We also compare the results of our models to Local Outlier Factors, a common outlier detection methodology, in this section.

\subsubsection{Simple Gaussian Model}

The results from running the sensor data set through the Simple Gaussian model are shown in Figure~\ref{fig:gauss15}. The data is plotted in light green, and the outliers are marked by dark red crosses.

In this experiment we flag the entries with column values that fall outside $1.5$ standard deviations of the mean of that particular column as outliers. This model runs relatively fast, as no correlations are computed (\timing{.03}{0}{.12}).
%As we observe in Figure~\ref{fig:gauss15}, this leads to the extreme values on the temperature spectrum as well as the humidity spectrum to be identified as outliers.
%However, many points in the normal range of operation are flagged as outliers.
%The cause of the noise can be seen in Figure~\ref{fig:sensors_gaus_1-5b}.
%In this plot we see that the values of both voltage and light are relatively evenly distributed within their respective ranges.
%Therefore many values lie outside $1.5$ standard deviations from the mean, despite having reasonable values in other dimensions such as temperature.
%Using information from correlations is one way in which we improve on this, which we describe in the next section.

\subsubsection{Mixture Model}
We set the statistical threshold to $0.7$, which produces two correlations between temperature and humidity and between temperature and voltage.
Figure~\ref{fig:gmm1t1} shows the results when using a single Gaussian component. Points flagged as outliers have a likelihood of less than $7.5\%$ of being produced by the Gaussian generated by the model (\timing {.03}{.34}{.73}).

This model is able to detect values with high temperature and low voltage as outliers.

%We show in Figure~\ref{fig:sensors_nocorr} the benefits of pruning the data via correlations before feeding it into the Gaussian model.
%In this experiment, we allow the statistical analyzer to mark all columns as correlated.
%The model produces a Gaussian with so much noise from the light data that it detects many points even within the normal operating range as outliers, even when the threshold is reduced to $0.5\%$.
%Thus, using some mechanism to find correlations is useful in narrowing the search space for outliers.

Figure~\ref{fig:gmm2t05} shows the results obtained using the Mixture model with two components, using the same 1000 randomly selected data points. Flagged values have a likelihood of less than $7.5\%$ under their dominant Gaussian (\timing {.03}{.35}{0.78}).
When using two Gaussians, the points clustered around the temperature $120$ degrees Celsius are no longer detected as outliers: they are modeled by their own Gaussian (although this Gaussian's weight is smaller than its counterpart). This model highlights the points within normal sensor operation that have outlying results.

\subsubsection{Local Outlier Factors}
\label{sec:lof-evaluation}

In this section we compare the results of our Gaussian and Mixture models to Local Outlier Factors (LOF)~\cite{Breunig2000}, a frequently used method for outlier detection.
LOF measures the degree to which a data point is an outlier by comparing each data point's reachability to those of its $k$ nearest neighbors.
The higher the LOF, the more isolated the data point relative to its local neighborhood and therefore the more likely the point is to be an outlier.

One downside of LOF compared to \dBoost/ is that it can only evaluate two-dimensional data.
The original algorithm also has significant computation complexity in order to calculate the distance to the nearest neighbors of each data point.
One benefit of LOF, however, is that the algorithm returns a continuous value that indicates the degree to which a point is an outlier, as opposed to a binary value.

Figure~\ref{fig:lofk2} shows the outliers detected by LOF when $k=2$.
We observe that contrary to the Gaussian and Mixture models, the outliers detected by LOF are scattered throughout the data.
The outliers are not necessarily the points one would intuitively assume are outliers.
This is because points that are within the normal range of the data will be selected as outliers if they are far enough away from the other points nearest to them.
%When the number of nearest neighbors evaluated is increased to $10$ in Figure~\ref{fig:lof_10}, the outliers detected are further outside the main cluster of points.
We find that LOF is not as useful at pointing out the tagged outliers in the sensor data set.


\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[a4paper, total={6in, 8in}]{geometry}
\usepackage[shortlabels]{enumitem}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{mathtools}
\usepackage{lmodern}
\DeclarePairedDelimiter\ceil{\lceil}{\rceil}
\DeclarePairedDelimiter\floor{\lfloor}{\rfloor}
\setlength{\parskip}{0.5em}

\usepackage{clrscode3e}

\title{Binary Search: Find it Quickly}
\author{Competitive Programming UTEC}

\begin{document}

\maketitle

\section{Introduction}

\subsection{Motivation}

Lets say you are given an integer array $A$ of size $n$, and you have to answer the following query: Given an integer $x$ find, $i$ such that $A_i = x$, or determine that such position doesn't exist.

There is one obvious trivial solution for this problem, we iterate through all elements in the array and find an index that satisfies the condition. It is simple to see that this approach would have a complexity of $O(n)$, so if we have multiple queries this might be too slow.

Another sensible approach would be to keep an index \textit{map}, in which we store for every value in the array it's position. With this apporach we can have a time of $O(1)$ per query, but will also need an auxiliary space of $O(n)$.

For most arrays, this are our only options. However, if the array is \textit{sorted}, we can use binary seach to solve this problem in logarithmic time and $O(1)$ of memory.

\subsection{Binary Search}

Binary search is a \textit{divide and conquer} search algorithm that allows us to look for values in sorted lists in logarithmic time. Thanks to it versatility, binary search is used in thousands of algorithms in order to boost up performance. Most importantly, its uses go \textbf{way beyond looking for values in an array}, as we will learn later.

In order to be able to apply binary search the following conditions must be satisfied:

\begin{itemize}
	\item \textbf{Random Access:} We must be able to jump to any element in the sequence with an $O(1)$ complexity.
	\item \textbf{Order:} Elements in the sequence must follow some kind of order.
\end{itemize}

\subsection{The Algorithm}

The binary search algorithm is very simple. Lets say I want to find element $x$ in array $A$. I could begin by guessing at which position $x$ will be, for example I could guess that $A[i] = x$. There are three possibilities:

\begin{enumerate}
	\item If am very lucky then $A[i] = x$.
	\item $A[i] < x$
	\item $A[i] > x$
\end{enumerate}

It is obvious that in the first case I just need to return $i$ as my answer, but what about cases 2 and 3. Lets say that $A[i] < x$, then as the array is sorted, we know that it is impossible that $x$ is to the left of $A[i]$ as they will be all smaller than $A[i]$, and therefore, smaller than $x$. This means that if I were to guess again, I could ignore all indexes that are smaller or equal to $i$. Symmetrically, if $A[i] > x$, than I now that it is impossible that $x$ is anywhere to the right of $A[i]$.

Now what if instead of guessing, we picked an index systematically, so that after every comparison we can discard a large number of the elements in the array of our candidates. The best way to do this is to pick the middle element of the array, as regardless of how $A[i]$ compares with it, we will get rid of half of the array in one move. Then we can repeatedly apply binary search until we find the value we are looking for.

As after each iteration we are halving the array, it is simple to see than in the worst case we will have to split the array until a single element remains. This is what yields a complexity of $O(lgn)$ for the search.

Just as it is the case with most divide and conquer algorithms, the most intuitive approach for the implementation of binary search is recursive. In the implementation $l$ and $r$ represent that range in which we can still find value $x$. The algorithm stops when we find $x$ of when $l > r$, which means that $x$ wasn't in the array.

\begin{codebox}
\Procname{$\proc{binarySearch}(A, l, r, x)$}
	\li \If $l > r$ \li \Then
		\Return -1
	\End

	\li $m = \floor{\frac{l + r}{2}}$
	\li \If $A[m] < x$ \li \Then
		\Return $\proc{binarySearch}(A, l, m - 1, x)$
	\End
	\li \If $A[m] > x$ \li \Then
		\Return $\proc{binarySearch}(A, m + 1, r, x)$
	\End

	\li \Return $m$
\end{codebox}

This solution occupies $O(lgn)$ memory because of the extra space that needs to be allocated in the stack, unless we use the \textit{tail recursion optimization}, case in which it will just consume $O(1)$ memory. A simple iterative implementation of binary search also exists. The idea for this is the same, but instead of calling the binary search method recursively, we use an while loop to update the values of $l$ and $r$.

\begin{codebox}
\Procname{$\proc{binarySearch}(A, l, r, x)$}
	\li $l \gets 1$
	\li $r \gets \attrib{A}{size}$
	\li \While $l \leq r$ \li \Do
		$m \gets \floor{\frac{l + r}{2}}$
		\li \If $A[m] = x$ \li \Then
			\Return m
		\End
		\li \If $A[m] < x$ \li \Then
			$r \gets m - 1$
		\li \Else
			\li $l \gets m + 1$
		\End
	\End

	\li \Return $-1$
\end{codebox}

Both of this implementation have the same asymptotic complexity of $O(lgn)$, however the iterative implementation is usually faster.

\section{Orderings}

In \textit{order} to better understand binary search we need to first understand what \textit{sorted} means. We say thing are sorted when they follow an \textit{order}, but this inevitably arises the question, what is order?

In mathematics order is a binary relationship that exists in a set of elements. We will usually refer to this relationship with the symbol $\leq$. In order for a relationship to be considered an order it must follow some conditions, the most important being:

\begin{itemize}
	\item \textbf{Antisymmetry:} $a \leq b \land b \leq a \rightarrow a = b$
	\item \textbf{Transitivity:} $a \leq b \land b \leq c \rightarrow a \leq c$
\end{itemize}

Under this definition of $order$ we can define as array as sorted if:

$$\forall i, j \leq n (i < j \rightarrow A_i \leq A_j)$$

The most important thing of this is to remember that $\leq$ can represent any relationship that satisfies the condition described above. For example, $a \leq b$ can mean $a$ is smaller than $b$, $a$ is greater than $b$, $|a|$ is grater than $|b|$, etc...

\subsection{Types of order}

There are two types of order: total order and partial order. The only difference between this is that total order maintains \textit{connexity}, that is to say that the given a pair of elements $a$ and $b$, there must exist a relationship between them (either $a \leq b$ or $b \leq a$). In practice, this means that in a partial order, some pairs of elements don't have a defined order between them.

For example, lets say that we define $a \leq b$ as $a$ is an ancestor of $b$. How would I compare with my cousin? We can't either say that I am my cousins ancestor or that my cousin is my ancestor, so how we should compare is not defined.

When we define an order, it is really important that we make sure this order is total, as partial ordering might lead to undefined behaviour. From here forward when we talk about order we will be referring to total orders, as with partial ordering some complications arise in the algorithms and theory we are going to explore.

\subsection{Order in a Computer}

In the context of computer science, we can think of the relationship $a \leq b$, as a 2-parameter boolean function $\proc{comp}(a, b)$, that returns \texttt{true} if the relation $a \leq b$ exists and \texttt{false} otherwise. This function is called the \textit{comparator} and it fully defines the order of the array. In order to ensure the correctness of our algorithms, we must be sure that our comparator obeys the same restrictions the order relationship $\leq$ did.

\end{document}


\chapter{Language Facilities}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Constants}

In nML, constants are declared using the \textbf{\textit{let}} keyword. For example,
the following line of code declares a constant called {\texttt A} that has the value {\texttt 100}:

\begin{lstlisting}
let A = 100
\end{lstlisting}

Constants are global. They can be defined only once and can be used in any context
their values could stand.

Constants can help extend nML. Any information about a machine that can be given with
a single number or string can easily be defined as a constant.

\subsection{Constant Data Types}

Constants can be represented by integer numbers of inifinite precision and strings.
Integer constants are cast to specific nML data types depending on the context in
which they are used.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Data Types}

A data type specifies the format of values stored in registers or memory.
nML supports the following data types:

\begin{itemize}

\item \textbf{\textit{int(N)}}:
N-bit signed integer data type. Negative numbers are stored in two's complement
form. The range of possible values is [-2$^{n-1}$ ... 2$^{n-1}$ - 1].

\item \textbf{\textit{card(N)}}:
N-bit unsigned integer data type. The range of possible values is [0 ... 2$^n$ - 1].

\item \textbf{\textit{float(N, M)}}: IEEE 754 floating point number, where fraction
size is N and exponent size is M. The resulting type size is N + M +1 bits, where 1
is an implicitly added bit for store the sign. Supported floating-point formats include:

\begin{itemize}
\item 32-bit single-precision. Defined as float(23, 8).
\item 64-bit double-precision. Defined as float(52, 11).
\item 80-bit double-extended-precision. Defined as float(64, 15).
\item 128-bit quadruple-precision. Defined as float(112, 15).
\end{itemize}

\end{itemize}

nML allows declaring aliases for data types using the \textbf{\textit{type}} keyword.
For example, in the code below, WORD is declared as an alias for card(32):

\begin{lstlisting}
type WORD = card(32)
\end{lstlisting}

\subsection{Converting Data Types}

In nML, type conversion is performed explicitly using special functions.
Implicit type conversion is not supported. Type conversion functions:

\begin{itemize}
\item \textbf{\textit{sign{\_}extend(type, value)}}:
Sign-extends the value to the specified type. Applied to any types. Requires
the new type to be larger or equal to the original type.

\item \textbf{\textit{zero{\_}extend(type, value)}}:
Zero-extends the value to the specified type. Applied to any types. Requires
the new type to be larger or equal to the original type.

\item \textbf{\textit{coerce(type, value)}}:
Converts an integer value to another integer type. For smaller types the value
is truncated, for larger types it is extended. If the original type is a signed
integer, sign extension is performed. Otherwise, zero-extension is performed.

\item \textbf{\textit{cast(type, value)}}:
Reinterprets the value as the specified type. Applied to any types. The new type
must be of the same size as the original type.

\item \textbf{\textit{int{\_}to{\_}float(type, value)}}:
Converts 32 and 64-bit integers into 32, 64, 80 and 128-bit floating-point
values (IEEE 754).

\item \textbf{\textit{float{\_}to{\_}int(type, value)}}:
Converts 32, 64, 80 and 128-bit floating-point values (IEEE 754) into 32 and
64-bit integers.

\item \textbf{\textit{float{\_}to{\_}float(type, value)}}:
Converts 32, 64, 80 and 128-bit floating-point values (IEEE 754) into each
other.

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Memory, Registers and Variables}

Memory and registers represent the visible state of a microprocessor. Only this state is carried
across execution of instruction calls. Both entities are modeled as arrays of the specified
data type. Although memory is external to the microprocessor while registers are internal, they
are described in nML in the same way. Differences are related to intergration with the MMU model
and are transparent to users.
In addition to memory and registers, nML supports defining temporary variables, which can be used
to exchange data between primitives that constitute a microprocessor instruction. Such variables
do not represent any externally visible state and do not save their values across instruction
calls.

\subsection{Memory}

Memory is described as a data array that represents the state of physical memory. Virtual memory
issues such as address translation, caching, etc. are not covered by nML specifications.

A memory array is defined using the \textbf{\textit{mem}} keyword. For example, the code below
defines a memory array called {\texttt MEM} that consists of 2$^{30}$ words:

\begin{lstlisting}
mem MEM [2 ** 30, WORD]
\end{lstlisting}

The array lenght must be specified as a contant expression. In the initial state, the array is
filled with zeros.

\subsection{Registers}

Registers are defined using the \textbf{\textit{reg}} keyword. Like memory, they are described
as an array of the specified type. For example, the following code defines a register array that
includes 32 registers storing a word:

\begin{lstlisting}
mem GPR[32, WORD]
\end{lstlisting}

The array length must be specified as a constant expression. Uninitialized registers hold
zeros. When the array contains only a single element, the length expression can be skipped:

\begin{lstlisting}
reg HI[WORD]
\end{lstlisting}

\subsection{Variables}

Variables are defined using the \textbf{\textit{var}} keyword. Their role is to store
temporary values used by instructions. They are similar to registers, but they do not represent
any visible state and do not save their values across instruction calls. Variables are initialized
with zeros and reset their values after the execution of an instruction call. For example, the code
below defines a variable for storing a 33-bit signed integer value:

\begin{lstlisting}
var temp[int(33)]
\end{lstlisting}

\subsection{Aliases}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Operators}

Here is the list of predefined nML operators:

\begin{itemize}

\item \textbf{Binary +, --}

\item \textbf{Unary +, --}

\item \textbf{$\ast$, /, \%}

\item \textbf{$\ast\ast$}

\item \textbf{<, >, <=, >=, ==, !=}

\item \textbf{{<}{<}, {>}{>}, \&, |, \^{}, \~{}}

\item \textbf{{<}{<}{<}, {>}{>}{>}}

\item \textbf{!, \&\&, ||}

\item \textbf{::}

\item \textbf{sqrt}

\item \textbf{is{\_}nan}

\item \textbf{is{\_}signaling{\_}nan}

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Special Functions}

TODO:\\

\textbf{format(format-string, args*)}:
Used to put together the string values of the \textbf{\textit{syntax}} and \textbf{\textit{image}}
attributes. The format string is a variation of the printf format string well known from C.
It may contain alphanumeric characters, blanks, tabs ('\textbackslash t'), newlines
('\textbackslash n'), and format directives of the form \%nC, where n is an optional field size
and C is one of the following characters:

\begin{itemize}

\item  d This takes an Integer or Cardinal argument from the argument list and formats it as a
         decimal number.

\item  x This takes a Cardinal argument and formats it as a hexadecimal number.

\item  b This takes a Cardinal argument and formats it as a binary number. It may also take a
         binary string, i.e. a string containing only 1s, 0s and (ignored) whitespace.

\item  s This takes a string argument and incorporates it as a whole.

\end{itemize}