\section{Modeling with ordinary differential equations}

\begin{example}[Exponential growth]
  Bacteria are living on a substrate with ample nutrients. Each
  bacteria splits into two after a certain time $\Delta t$. The time
  span for splitting is fixed and independent of the individuum. Then,
  given the amount $u_0$ of bacteria at time $t_0$, the amount at
  $t_1 = t_0+\Delta t$ is $u_1 = 2 u_0$. Generalizing, we obtain
  \begin{gather*}
    u_n = u(t_n) = 2^n u_0, \qquad t_n = t_0 + n\Delta t.
  \end{gather*}

  After a short time, the number of bacteria will be huge, such that
  counting is not a good idea anymore. Also, the cell division does
  not run on a very sharp clock, such that after some time, divisions
  will not only take place at the discrete times $t_0+n\Delta t$, but
  at any time between these as well. Therefore, we apply the continuum
  hypothesis, that is, $u$ is not a discrete quantity anymore, but a
  continuous one that can take any real value. In order to accommodate
  for the continuum in time, we make a change of variables:
  \begin{gather*}
    u(t) = 2^{\frac{t-t_0}{\Delta t}} u_0.
  \end{gather*}

  Here, we have already written down the solution of the problem,
  which is hard to generalize. The original description of the problem
  involved the change of $u$ from one point in time to the next. In
  the continuum description, this becomes the derivative, which we can
  now compute from our last formula:
  \begin{gather*}
    \tfrac{d}{dt} u(t) = \frac{\ln 2}{\Delta t} 2^{\frac{t-t_0}{\Delta t}} u_0
    = \frac{\ln 2}{\Delta t} u(t).
  \end{gather*}

  We see that the derivative of $u$ at a certain time depends on $u$
  itself at the same time and a constant factor, which we call the
  growth rate $\alpha$. Thus, we have arrived at our first
  differential equation
  \begin{gather}
    \label{eq:models:1}
    u'(t) = \alpha u(t).
  \end{gather}
  What we have seen as well is, that we had to start with some
  bacteria to get the process going. Indeed, any function of the form
  \begin{gather*}
    u(t) = c e^{\alpha t}
  \end{gather*}
  is a solution to equation~\eqref{eq:models:1}. It is the initial
  value $u_0$, which anchors the solution and makes it unique.
\end{example}

\begin{example}[Predator-prey systems]
  We add a second species to our bacteria example. Let's say, we
  replace the bacteria by sardines living in a nutrient rich sea, and
  we add tuna eating sardines. The amount of sardines eaten depends on
  the likelyhood that a sardine and a tuna are in the same place, and
  on the hunting efficiency $\beta$ of the tuna. Thus,
  equation~\eqref{eq:models:1} is augmented by a negative change in
  population depending on the product of sardines $u$ and tuna $v$:
  \begin{gather*}
    u' = \alpha u - \beta u v.
  \end{gather*}

  In addition, we need an equation for the amount of tuna. In this
  simple model, we will make two assumptions: first, tuna die of
  natural causes at a death rate of $\gamma$. Second, tuna procreate
  if there is enough food (sardines), and the procreation rate is
  proportional to the amount of food. Thus, we obtain
  \begin{gather*}
    v' = \delta u v - \gamma v.
  \end{gather*}

  Again, we will need initial populations at some point in time to
  compute ahead from there.
\end{example}

\begin{remark}
  The Lotka-Volterra-equations have periodic solutions. Even though
  none of these exist in closed form the sulotions can be simulated:
  \begin{figure}[tp]
    \begin{center}
      \includegraphics[width=.6\textwidth]{fig/lotkavolterra}
      \caption{Plot of a solution to the Lotka-Volterra equation with
        parameters $\alpha = \frac 23$, $\beta  = \frac 43$, $\delta = \gamma = 1$
        and initial values $u(0) = 3$, $v(0) = 1$. Solved with a Runge-Kutta
        method of order five and step size $h = 10^{-5}$.}
    \end{center}
    \label{fig:lotkavolterra}
  \end{figure}
  Lotka and Volterra became interested in this system as they had
  found that the amount of predatory fish caught had increased
  during World War I. During the war years there was a strong
  decrease of fishing effort. In conclusion, they thought, there had
  to be more prey fish.

  A (far too rarely) applied consequence is that in order to diminish
  the amount of e.g. foxes one should hunt rabbits as foxes feed
  on rabbits.
\end{remark}

\begin{example}[Graviational two-body systems]
  According to Newton's law of universal gravitation, two bodies of
  masses $m_1$ and $m_2$ attract each other with a force
  \begin{gather*}
    \vec F_1 = G \frac{m_1m_2}{r^3} \vec r_1,
  \end{gather*}
  where $\vec F_1$ is the force vector acting on $m_1$ and $\vec r_1$
  is the vector pointing from $m_1$ to $m_2$ and $r = \lvert\vec r_1\rvert = \lvert\vec r_2\rvert$.

  Newton's second law of motion on the other hand relates forces and
  acceleration:
  \begin{gather*}
    \vec F = m \vec x'',
  \end{gather*}
  where $\vec x$ is the position of a body in space.

  Combining these, we obtain equations for the positions of the two bodies:
  \begin{gather*}
    \vec x''_i = G \frac{m_{3-i}}{r^3} (\vec x_i - \vec x_{3-i}), \qquad i=1,2.
  \end{gather*}
  This is a system of 6 independent variables. Nevertheless, it can be
  reduced to three by using that the center of mass moves
  inertially. Then, the distance vector is the only variable to be
  computed for:
  \begin{gather*}
    \vec r'' = - G \frac{m}{r^3} \vec r.
  \end{gather*}
  Intuitively, that we need an initial position and an initial
  velocity for the two bodies. Later on, we will see that this can
  actually be justified mathematically.
\end{example}

\begin{example}[Celestial mechanics]
  Now we extend the two-body system to a many-body system. Again, we
  subtract the center of mass, such that we obtain $n$ sets of 3
  equations for an $n+1$-body system. Since forces simply add up, this
  system becomes
  \begin{gather}
    \label{eq:celestial}
    \vec x_i = -G \sum_{j\neq i} \frac{m_j}{r_{ij}^3} \vec r_{ij}.
  \end{gather}
  Here, $\vec r_{ij} = \vec r_j - \vec r_i$ and $r_{ij} = \lvert \vec r_{ij}\rvert$.
  Initial data for the solar system can be obtained from
  \begin{center}
    \texttt{https://ssd.jpl.nasa.gov/?horizons}
  \end{center}
\end{example}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "notes"
%%% End:


\section{Discrete Fourier Transform}

The \emph{Discrete Fourier Transform (DFT)} of a series of complex numbers $\left\{x_n\right\} = x_0, \dots, x_{N-1}$
are the $N$ numbers defined as
\begin{equation*}
    X_k = \symcal{F}\left\{x_n\right\}_k = \frac{1}{\sqrt{N}}\sum_{n=0}^{N-1} x_n e^{-i\frac{2\pi}{N}n k}.
\end{equation*}
An example can be found in \autoref{tab:ex}.
% For intuitive understanding one can think of $X_k$ as the intensity of the frequency,
% $\nu =\frac{\omega}{2\pi}= k / N$ in the signal.

\begin{table}
    \centering
    \caption{Example of the DFT of the sequence 4, 8, 15, 16, 23, 42.}
    \begin{tabular}{S[table-format=1.0] S[table-format=2.0] S[table-format=2.1] >{\hspace{-1em}}c<{\hspace{-.8em}} S[table-format=2.1]}
        \toprule
        {$k$} & {$x_k$} & \multicolumn{3}{c}{$X_k$}               \\
        \midrule
        0     & 4       & 44.1                      & $+i$ & 0.0  \\
        1     & 8       & -2.4                      & $+i$ & 14.8 \\
        2     & 15      & -9.8                      & $+i$ & 9.1  \\
        3     & 16      & -9.8                      & $+i$ & 0.0  \\
        4     & 23      & -9.8                      & $-i$ & 9.1  \\
        5     & 42      & -2.4                      & $-i$ & 14.8 \\
        \bottomrule
    \end{tabular}
    \label{tab:ex}
\end{table}

For a deeper, intuitive understanding let's take a step back and remember that Fourier
found that (nearly) any periodic function can be represented as a weighted sum of cosine and
sine functions.
This does not only apply for continuous, but also discontinuous functions,
such as a square wave.
To put it in a more mathematical way: The
$L^2([0, T])$-space\footnote{This means every complex-valued function defined on $[0, T]$, which's absolute square is integrable over $[0, T]$.}
has the orthonormal basis functions
\begin{equation*}
    \phi_k(x)=\exp\left({-2ikx\frac{2\pi}{T}}\right) \quad k\in \mathbb{Z},
\end{equation*}
so any function $f(x)\in L^{2}([0, T])$ can be written as
\begin{equation*}
    f(x) \equiv \sum_{k=-\infty}^{\infty} \underbrace{\left< f, \phi_k \right>}_{a_k} \phi_k(x)
    = \sum_{k=-\infty}^{\infty} {a_k} \exp\left({-2ikx\frac{2\pi}{T}}\right)
\end{equation*}
where $\equiv$ means "equals almost everywhere under the norm induced by the scalar product $\left<\cdot,\cdot\right>$".
This sum is called a Fourier series.
The scalar product (inducing the norm) in this space is
\begin{equation*}
    \left< f, g\right> = \int_0^T f(x)g^*(x) \dif x.
\end{equation*}
Thus, the Fourier coefficients $a_k$ are
\begin{equation*}
    a_k = \int_0^T f(x) \exp\left({ikx\frac{2\pi}{T}}\right) \dif x.
\end{equation*}
This is a very abstract definition, but if we get used to the math,
this only precises the findings of Fourier: A periodic function
(our functions are restricted to $[0, T]$, but they can be thought of as being periodically continued everywhere else)
can be represented as a weighted (infinite) sum of sines and cosines (in our formulation hidden with the Euler identity in $e^{ix}$).

If we now have a function which is neither confined nor periodic, we can handwavely argue, that $T$ goes to infinity and
that we need a continues set of basis functions. This is a way of approaching the continuous \emph{Fourier Transform (FT)}.
The idea of the FT is to transform the original function $f$ to another function $\hat f$ without loosing any information and thus being
able to transform the function back.
The transformation is similar to the definition of the Fourier coefficients\footnote{Watch out for
    different definitions of factors when comparing different sources}
\begin{equation*}
    \hat f(\nu) = \int_{-\infty}^{\infty} f(x) e^{-i2\pi\nu x} \dif x.
\end{equation*}
The inverse Fourier transform is given by\footnote{
    To proof the corectness, expressing the Dirac $\delta$-function as
    $\delta(x)~=~\int_{-\infty}^{\infty} e^{-i2\pi \nu x} \dif \nu$ is helpful.
}
\begin{equation*}
    f(x) = \int_{-\infty}^{\infty} \hat f(\nu) e^{i2\pi\nu x} \dif \nu.
\end{equation*}
For this the function $f: \mathbb{R}\to \mathbb{C}$ only has to be absolutely integrable, so that
$\int_{\mathbb{R}} \abs{f(x)} \dif x$ is defined.
Again $\hat f(\nu)$ can be thought of as the intensity of the frequency $\nu$ in the function. That's
why we name the domain $\nu$ and $\hat f$ is often called $f$ in frequency- or Fourier space.
This definition is easily extended to functions which take multi-dimensional input $f: \mathbb{R}^n \to \mathbb{C}$
\begin{equation*}
    \hat f(\vec \nu) = \int_{\mathbb{R}^n} f(\vec x) e^{-i2\pi\vec \nu \cdot \vec x} \dif{}^n x.
\end{equation*}
And the inverse transform likewise.

Now we can motivate the DFT: If we define a function (to be exact: a distribution) as
\begin{equation*}
    f(x) = \frac{1}{\sqrt N} \sum_{n=0}^{N-1} x_n\ \delta\left(x-{n}\right)
\end{equation*}
the DFT of the points is the FT of this function at the frequencies $k/N$
\begin{equation*}
    \begin{split}
        \hat f\left(\frac{k}{N}\right) = \int_{-\infty}^{\infty}  \frac{1}{\sqrt N} \sum_{n=0}^{N-1}
        x_n\ \delta\left(x-{n}\right) e^{-i2\pi\frac{k}{N} x} \dif x\\
        = \frac{1}{\sqrt N} \sum_{n=0}^{N-1} x_n e^{-i2\pi k \frac{n}{N}}
        = X_k.
    \end{split}
\end{equation*}
And likewise, if we define the function
\begin{equation*}
    F(\nu) = \frac{1}{\sqrt N} \sum_{n=0}^{N-1} X_n\ \delta\left(\nu-{n}\right)
\end{equation*}
it's inverse FT at $k/N$ is the inverse DFT of the points $X_k$
\begin{equation*}
    \begin{split}
        \hat F\left(\frac{k}{N}\right)
        = \int_{-\infty}^{\infty}  \frac{1}{\sqrt N} \sum_{n=0}^{N-1}
        X_n\ \delta\left(\nu-{n}\right) e^{i2\pi\frac{k}{N} \nu} \dif \nu\\
        = \frac{1}{\sqrt N} \sum_{n=0}^{N-1} X_n e^{i2\pi k \frac{n}{N}}
        = \frac{1}{N} \sum_{n,m=0}^{N-1}
        x_m e^{-i2\pi \frac{n}{N}(m-k)}\\
        =  \sum_{m=0}^{N-1}
        x_m\delta_{m,k} = x_k.
    \end{split}
\end{equation*}
Where we have proofed the relation using the geometric series
\begin{equation*}
    \frac{1}{N}
    \sum_{n=0}^{N-1}
    e^{-i2\pi \frac{n}{N}(m-k)}=
    \begin{cases}
        \frac{1-e^{-i2\pi (m-k)}}{1-e^{-i2\pi\frac{m-k}{N}}} = 0 & \forall m\neq k \\
        1                                                        & \forall m=k
    \end{cases}
    = \delta_{m,k}.
\end{equation*}
The definition of the function might seem abstract at first glance, but it is nothing
more than a more mathematical way of expressing, that we only have $N$ samples of
an unknown functions at these points.
The normalization factor of $1/\sqrt N$ is chosen so the inverse transform does not rescale
the points.

As with the FT, the DFT is easily enlarged to more dimensions
\begin{equation}
    X_{k,l} = \frac{1}{\sqrt{N_1N_2}}\sum_{n=0}^{N_1-1}\sum_{m=0}^{N_2-1}x_{n,m} \ e^{-i{2\pi}\ \left(\!\frac{kn}{N_1}+\frac{ml}{N_2}\right)}
    \label{eqn:dft2d}
\end{equation}
and the inverse
\begin{equation}
    x_{k,l} = \frac{1}{\sqrt{N_1N_2}}\sum_{n=0}^{N_1-1}\sum_{m=0}^{N_2-1} X_{n,m} \ e^{i{2\pi}\ \left(\!\frac{kn}{N_1}+\frac{ml}{N_2}\right)}.
    \label{eqn:idft2d}
\end{equation}
I chose the three-dimensional example here, since we are going to look at images with a color channel, which is three-dimensional data.
Other dimensionality are analogous.

\subsection{Real Numbered Input Data}
An image will only contain real numbered input values. This yields the following symmetry in the transformation
\begin{equation}
    X_{l,m,n} = X^*_{N_1-l,N_2-m,N_3-m}.
    \label{eqn:realFSymmetry}
\end{equation}
On the one hand, this is a very useful result: If we transform an image consisting of $N$ real numbers,
we do not want to end up with $N$ complex (or equivalently $2N$ real) numbers and thus having more numbers than we started with.
On the other hand the problem of displaying the transformed image still remains: we have to visualize a two-dimensional array of complex numbers.
Usually we display the magnitude of the transformed image and its phase (also called the argument) separately.

Since it seems quite artificial to introduce complex numbers in a problem only consisting of real numbers,
for image compression usually a very closely related transform is used, the \emph{Discrete Cosine Transfrom (DCT)}.
It transforms $N$ real-valued input numbers to $N$ real-valued output numbers.
The transform is given by
\begin{equation}
    \begin{split}
        X_k = \symcal{C}\left(\vec x\right)_k = \frac{2c_k}{\sqrt N}\sum_{n=0}^{N-1}x_n \cos \left(\frac{\pi\left(2n+1\right)k}{2N} \right)
        \\
        \text{where }
        c_k =
        \begin{cases}
            \frac{1}{\sqrt{2}} \quad & k=0           \\
            1                        & \text{ else }
        \end{cases}
    \end{split}
\end{equation}
and it's inverse by
\begin{equation}
    x_k = \symcal{C}^{-1}\left(\vec X\right)_k
    =\frac{1}{\sqrt N}\sum_{n=1}^{N-1}c_n X_n \cos \left(\frac{\pi(2k+1)}{2N}n\right).
\end{equation}
\cite{DCT}
The DCT can be calculated using the DFT. An efficient algorithm for this from \cite{DCTUFFT} first reorders the points to
\begin{equation*}
    \begin{rcases}
        x'_k       & = x_{2k}  \\
        x'_{N-1-k} & =x_{2k+1}
    \end{rcases}
    k=0,1, \dots, \frac{N}{2}-1
\end{equation*}
to obtain the relation
\begin{equation*}
    \symcal{C}\left(\vec x\right)_k = X_k =2 c_k
    \Re e^{i\pi\frac{k}{2N}}\symcal{F}^{-1}\left(\vec x'\right)_k
\end{equation*}
and it's inverse
\begin{equation*}
    \begin{split}
        \symcal{C}^{-1}\left(\vec X\right)_{2k} =& x_{2k} =
        \Re \symcal{F}^{-1}\left\{c_lX_le^{i\frac{\pi l}{2N}}\right\}_k
        \\
        &x_{2k+1} = x_{2(N-1-k)}.
    \end{split}
\end{equation*}
Note, that we only need to calculate half of the Fourier coefficients, if we make use of
Equation \eqref{eqn:realFSymmetry}.

\section{The Fast Fourier Transform}
Our goal is to find an efficient algorithm for our numerical implementation of
the discrete Fourier transform.
Firstly, we notice that the multidimensional DFT is
separable into multiple one dimensional DFTs, performed one after another
\begin{equation*}
    X_{l,m} = \symcal{F} \left\{\symcal{F}\{(x_i)_j\}_m\right\}_l.
\end{equation*}
Thus, we will concentrate on optimizing the one dimensional DFT.

We can see that the direct implementation of the DFT is nothing more than a matrix-multiplication
\footnote{In fact, it is possible to interpret the DFT as a curve fitting problem with the design matrix $A_{m,n}=e^{-i2\pi\frac{mn}{N}} / \sqrt{N}$.}.
As that, the computational complexity of the bare DFT implementation scales with $\symcal{O}(N^2)$.
But the concrete form of the transform allows us to get down to $\symcal{O}(N\log N)$.
From this scaling law, we can already guess, that we will use a divide-and-conquer algorithm.
The basic idea is to express the Fourier transform of $N$ points in multiple smaller Fourier transforms, and thus dividing and conquering.
Let's assume $N$ is even and observe that
\begin{equation*}
    \begin{split}
        X_k &= \frac{1}{\sqrt{N}}\sum_{n=0}^{N-1} x_n e^{-i\frac{2\pi}{N}n k}\\
        &=\frac{1}{\sqrt N}
        \sum_{n=0}^{N/2-1} x_{2n} e^{-i\frac{2\pi}{N}2n k}+
        \frac{1}{\sqrt N}
        \sum_{n=0}^{N/2-1} x_{2n+1} e^{-i\frac{2\pi}{N}(2n+1) k}\\
        &= \frac{1}{\sqrt 2}
        \symcal{F}\left\{x_{2n}\right\}_k
        +\frac{1}{\sqrt 2}
        e^{-i\frac{2\pi}{N}k}\symcal{F}\left\{x_{2n+1}\right\}_k.
    \end{split}
\end{equation*}
The last expression is only defined up to $k=N/2$, but for the other half we get
\begin{equation*}
    X_{k+\frac{N}{2}}=\frac{1}{\sqrt 2}
    \symcal{F}\left\{x_{2n}\right\}_k
    -\frac{1}{\sqrt 2}
    e^{-i\frac{2\pi}{N}k}\symcal{F}\left\{x_{2n+1}\right\}_k.
\end{equation*}
Here we have expressed the DFT of $N$ points as the DFT of the $N/2$ even points and $N/2$ odd points.
If now count the calculation steps, we have two times the DFT of $N/2$ points, so in total $2\times N^2/2^2=N^2/2$ steps,
reducing the original complexity.
This is called the \emph{Fast Fourier Transform (FFT)}, to be precise the radix-2-decimation-in-time Cooley-Tukey algorithm.
It was originally discovered by Gauss, but due to the lack of computers at his time, it was not a very known result.
Cooley and Tukey rediscovered the algorithm 160 years later in the sixties and popularized it.

We can similarly express the inverse DFT by noticing
\begin{equation*}
    \symcal{F}^{-1}\left\{x_n\right\}
    =\symcal{F}\left\{x_{N-n}\right\}
    =\symcal{F}\left\{x_{n}^*\right\}^*.
\end{equation*}
Thus, we only need the implementation of the DFT to also get the inverse DFT.

The complete divide and conquer in this form only works for $N$, which are powers of two.
But, it is possible to generalize this to any composite number of the form $N=N_1N_2$.
In that case we will combine $N_1$ smaller DFTs with a DFT of size $N_2$\footnote{Note that the combination rule for the even $N$ case can be understood as a DFT of two points.}.
For a full description see \cite{CTAlg}.
Note that this yields problems if our input data has prime-number length. For this special case there
exists an algorithm allowing the DFT of prime-number-sized points in $N\log N$ time, using group-theory
of prime-numbered modulo groups.
The algorithm is called \emph{Rader's algorithm} and a short overview can be found here \cite{radersalg}

\subsection{Code}
The code which produced the results is written in \texttt{C++} and can be found here\cite{githubfft}.
No numerical algorithms outside the standard template library were used, which
means I implemented a version of the FFT and DCT myself.
The code makes full use of every prime factor of $2$ in the number of transformed points,
but not the full powered Tukey-Cooley algorithm, since an efficient implementation of that
requires complicated reordering of the input numbers.
The used algorithm can handle arbitrary length input data, which makes it
more general than most implementations found online\footnote{One implementation I took some inspiration from is for example \cite{fftoreil}.}, since they are usually only implemented
for powers of 2.
Nonetheless, the implementation should be viewed more as a toy-implementation for testing
and can't compete with industry-grade optimized algorithms\cite{fftw}.
The FFT, DCT and their respective inverses where implemented in a generic fashion,
so that they can be used on any data structure supporting the square-bracket indexing
operator. Thus, it can be used on raw pointers as well as on higher level structures
like vectors.

The images in the Fourier space were represented by complex matrices from the \texttt{Eigen}
library.
To handle images, as well as image-input and -output the Boost Generic Image Library \texttt{Boost::GIL}\cite{boostgil} was used.

\section{Applications}
One of the most notable places the Fourier transform appears in physics is the
Fraunhofer-diffraction equation. In the far-field approximation, the diffraction image
of an incoming planar wave is proportional to the 2D-Fourier transform of the aperture
\begin{equation}
    A(x, y, z) \propto \iint_{\symup{Aperture}}\hspace{-3em} e^{-i\frac{2\pi}{\lambda z}(x'x+y'y)} \dif x' \dif y'.
    \label{eqn:fraunhofer}
\end{equation}
Thus, if we use a DFT of an image resembling an aperture, we can numerically approximate the diffraction image.
Every photograph taken is slightly distorted through diffraction effects through the aperture.
In space-telescopes for example the reflection mirror needs to be attached to the rest of the satellite,
and the light from stars is diffracted at the attached rods, which creates the typical 4 or 6-fold diffraction
symmetry of star-pictures.

Another application is the lossy compression of images, sound and other media types.
This is what (among other tricks) the \texttt{.jpg} image format does.

Images can also be enhanced with Fourier transform. Assume for example we take a photograph through an insect-net.
We will have periodic information overlaying the picture. This can be easily removed in Fourier space.
Generally, a lot of interesting edits can be performed in Fourier space, having complex outcome in real space.
And by removing high-frequencies the sharpness can be increased and by doing the inverse the image will be blurred.


\chapter{Math}

\section{Functions}
\rih{Equals.} Requirements for equals
\begin{enumerate}
\item Reflexive
\item Symmetric
\item Transitive
\item Non-null
\end{enumerate}
\rih{Compare.} Requirements for compares (total order):
\begin{enumerate}
\item Antisymmetry
\item Transitivity
\item Totality
\end{enumerate}

\section{Prime Numbers}
\subsection{Sieve of Eratosthenes}
\subsubsection{Basics}
To find all the prime numbers less than or equal to a given integer n by Eratosthenes' method:
\begin{enumerate}
\item Create a list of consecutive integers from 2 through n: (2, 3, 4, ..., n).
\item Initially, let $p$ equal 2, the first prime number.
\item Starting from $p$, enumerate its multiples by counting to n in increments of $p$, and mark them in the list (these will be $2p$, $3p$, $4p$, ... ; the $p$ itself should not be marked).
\item Find the first number greater than $p$ in the list that is not marked. If there was no such number, stop. Otherwise, let $p$ now equal this new number (which is the next prime), and repeat from step 3.
\end{enumerate}

When the algorithm terminates, the numbers remaining not marked in the list are all the primes below $n$.

\subsubsection{Refinements}
The main idea here is that every value for $p$ is prime, because we have already marked all the multiples of the numbers less than $p$. Note that some of the numbers being marked may have already been marked earlier (e.g., 15 will be marked both for 3 and 5).

As a refinement, it is sufficient to mark the numbers in step 3 starting from $p^2$, because all the smaller multiples of $p$ will have already been marked at that point by the previous smaller prime factor other than $p$. From $p^2$, $p$ becomes the smaller prime factor of a composite number. This means that the algorithm is allowed to terminate in step 4 when $p^2$ is greater than n.

Another refinement is to initially list odd numbers only, (3, 5, ..., n), and count in increments of $2p$ in step 3, thus marking only odd multiples of $p$. This actually appears in the original algorithm. This can be generalized with wheel factorization, forming the initial list only from numbers coprime with the first few primes and not just from odds (i.e., numbers coprime with 2), and counting in the correspondingly adjusted increments so that only such multiples of $p$ are generated that are coprime with those small primes, in the first place.

To summarized, the refinements include:
\begin{enumerate}
\item Starting from $p^2$.
\item Preprocessing even numbers and then only process odd numbers; thus the increment becomes $2p$.
\end{enumerate}}

\subsubsection{code}
\begin{python}
def countPrimes(n):
    """
    Find prime using Sieve's algorithm
    :type n: int
    :rtype: int
    """
    if n < 3:
        return 0

    is_prime = [True for _ in xrange(n)]
    is_prime[0], is_prime[1] = False, False
    for i in xrange(2, int(math.sqrt(n))+1):
        if is_prime[i]:
            for j in xrange(i*i, n, i):
                is_prime[j] = False

    return is_prime.count(True)
\end{python}

\subsection{Factorization}
Backtracking: Section-\ref{factorization}.

\section{Median}
\subsection{Basic DualHeap}
DualHeap to keep track the median when a method to find median is called multiple times. \begin{python}
import heapq

class DualHeap(object):
  def __init__(self):
    self.min_h = []
    self.max_h = []  # need to negate the value

  def insert(self, num):
    if not self.min_h or num > self.min_h[0]:
      heapq.heappush(self.min_h, num)
    else:
      heapq.heappush(self.max_h, -num)
    self.balance()

  def balance(self):
    l1 = len(self.min_h)
    l2 = len(self.max_h)
    if l1-l2 > 1:
      heapq.heappush(self.max_h,
                     -heapq.heappop(self.min_h))
      self.balance()
    elif l2-l1 > 1:
      heapq.heappush(self.min_h,
                     -heapq.heappop(self.max_h))
      self.balance()
    return

  def get_median(self):
    """Straightforward"""
\end{python}

\subsection{DualHeap with Lazy Deletion}\label{dh_lazy_del}
Clues:
\begin{enumerate}
\item Wrap the value and wrap the heap
\item When delete a value, mark it with tombstone.
\item When negate the value, only change the value, not the reference.
\item When heap pop, clean the op first.
\end{enumerate}
\begin{python}
import heapq
from collections import defaultdict

class Value(object):
    def __init__(self, val):
        self.val = val
        self.deleted = False

    def __neg__(self):
        """negate without creating new instance"""
        self.val = -self.val
        return self

    def __cmp__(self, other):
        assert isinstance(other, Value)
        return self.val - other.val

    def __repr__(self):
        return repr(self.val)

class Heap(object):
    def __init__(self):
        self.h = []
        self.len = 0

    def push(self, item):
        heapq.heappush(self.h, item)
        self.len += 1

    def pop(self):
        self._clean_top()
        self.len -= 1
        return heapq.heappop(self.h)

    def remove(self, item):
        """lazy delete"""
        item.deleted = True
        self.len -= 1

    def __len__(self):
        return self.len

    def _clean_top(self):
        while self.h and self.h[0].deleted:
            heapq.heappop(self.h)

    def peek(self):
        self._clean_top()
        return self.h[0]

class DualHeap(object):
    def __init__(self):
        self.min_h = Heap()  # represent right side
        self.max_h = Heap()  # represent left side
    # others similar as the previous section's above DualHeap
\end{python}


\section{Generating Functions}

Generating functions are functions that encode sequences of numbers as the coefficients of power series. One example are the moment generating functions in probability theory, though they are generally extremely useful in combinatorics problems and, almost equivalently, discrete probability problems.

\citeasnoun{bogart2004combinatorics} approaches the concept in terms of \textit{Picture Functions}. For each element $s\in S$, there is a picture function $P(s)$, so that, for example, the multiset $\{1,1,2\}$ can be written as $P(1)^1 P(2)$. Collections of combinations can be rewritten in terms of sums and products, which enables factorization and overall easier accounting. Combinations with particular properties can be filtered by looking at exponents.

The picture function enables writing down combinations of elements as an enumerating function $E_P(s)$. For example, the enumerating function for all multisets that include either one or two times some element $a$ and between zero and two times some element $b$ is written:

\begin{equation}
\begin{array}{rl}
E_P(s) &= P(a)+ P(a)^2 + P(a)P(b) + P(a)^2P(b) +  + P(a)P(b)^2 + P(a)^2P(b)^2\\
&\left(P(a) + P(a)^2\right)\left(1 + P(b) + P(b)^2\right)
\end{array}
\end{equation}

Generating functions get much more complicated. \citeasnoun{wilf2013generating} is a good resource.

\subsection{Example: Binomial Coefficients}

Consider the case of a collection of $n$ indistinguishable objects $s$, and write $P(s) = x$. Then the enumerator for selecting any subset of those $n$ objects is given by:

\begin{equation}
E_P(s) = \prod_i^n (x^0+x^1) = (1+x)^n = \sum_i^n {n \choose i}x^i
\end{equation}

Where each term $(x^0 + x^1)$ corresponds to the two options of either excluding a particular element ($x^0$) or including a particular element ($x^1$). The exponent of the expanded product encodes how many objects were included into a particular subset. This is one way of "proving" the binomial coefficients, and one can says $(1_x)^n$ is the generating function for the binomial coefficients ${n \choose i}$.

\subsection{Example: Basket of Goods}

An apple costs $20c$, a pear costs $25c$ and a banana costs $30c$. How many different fruit baskets can be bought for $100c$?

By replacing the picture function $P(s)$ with $x$, it was possible to identify subsets of $n$ objects by looking at the exponent of $x^n$ in the enumerating function. In this case, the exponent is supposed to show the price. This can be done by writing $P(apple) = x^{20}, P(pear) = x^{25}$ and $P(banana) = x^{30}$.

\begin{equation}
E_P(s) = \left( \sum_{i=0}^5 x^{20} \right)\left( \sum_{i=0}^4 x^{25} \right)\left( \sum_{i=0}^3 x^{30} \right)
\end{equation}

Which results in some power series of the form:

\begin{equation}
E_P(s) = 1x^0 + 1x^{20} + 1x^{25} + 1x^{30} + 1x^{40} +... + 2x^{60} + ... + 1x^{290}
\end{equation}

To obtain the number of combinations that correspond to a cost of exactly $100c$, one can apply the operator $\frac{1}{n!}\frac{d^n}{dx^n}$ and set $x=0$ to obtain the desired term. This is what is done with moment generating functions in statistics.

But actually it is easier to think through what the coefficients will be so that:

\begin{equation}
\sum_{l=0}^{n+m+h}d_l x^l = \left(\sum_{i=0}^n a_i x^i\right)\left(\sum_{j=0}^m b_j x^j\right)\left(\sum_{k=0}^h c_k x^k\right)
\end{equation}

\begin{equation}
d_l = \sum_{\begin{array}{c}i,j,k\\i+j+k=l\end{array}} a_i b_j c_k
\end{equation}

If $a_i = b_j = c_k = 1$, then:

\begin{equation}
d_l = \sum_{\begin{array}{c}i,j,k\\i+j+k=l\end{array}} 1 = {l + 3 - 1 \choose l}
\end{equation}

Which is the number of all multisets of size $l$ and 3 classes.

In this case, however, the coefficients are equal to 1 only for $i=a20$, $j=b25$, and $k=c30$ for $a,b,c \in \mathbb{N}_0$, and zero otherwise, so that there are 4 combinations of $a,b,c$ for which $20a+25b+30c=100$. Hence, $d_{100} = 4$ for the basket of goods above.

The sum can also be rewritten:

\begin{equation}
d_l = \sum^l_{i=0}\sum^{l-i}_{j=0} a_i b_j c_{l-i-j}
\end{equation}

Of course, this is the discrete version of a convolution. So, it is no surprise that the addition of random variables winds up being a convolution.

\subsection{Example: Dice}

How many ways are there for $n$ dice with $k$ faces to show $s$ eyes?

\begin{equation}
E_P = \left(\sum_{i=0}^\infty a_i x^i\right)^n = \sum_{i=0}^{\infty}d_i x^i
\end{equation}

Where $a_i = 1\ \forall i\in(1,k)$ and $a_i = 0$ otherwise.

\begin{equation}
E_P = \left(\sum_{i=1}^k x^i\right)^n = \left(x(1-x^k)\sum_{i=0}^\infty x^i\right)^n = x^n\left(\frac{1-x^k}{1-x}\right)^n = x^n \left(\sum_{i=0}^n (-1)^i {n \choose i} x^{ik}\right)\left(\sum_{j=0}^{\infty} (-1)^j {-n \choose j} x^j\right)
\end{equation}

The coefficient for $x^s$ is given the sum:

\begin{equation}
d_s = \sum_{ki+j=s-n} (-1)^{i+j}{n \choose i}{-n \choose j}\\
i \in [0,n] \\
j \in [0,\infty]
\end{equation}

Where the indices $i$ and $j$ satisfy $ki+j = s-n$. For example, for $s=7$, $n=2$ and $k=6$:

\begin{equation}
6i+j = 7-2 = 5s
\end{equation}

Holds for $i=0, j=5$:

\begin{equation}
d_s = (-1)^5 {2 \choose 0}{-2 \choose 5} = (-1)^{10} 1 {2+5-1 \choose 5} = {6 \choose 5} = \frac{6!}{5!1!} = 6
\label{eq:d6}
\end{equation}

Indeed, there are 6 ways for two d6 to add to 7:

\begin{equation}
[6,1],[5,2],[4,3],[3,4],[2,5],[1,6]
\end{equation}

\begin{figure}
\centering
\includegraphics[scale=0.5]{d6throw.png}
\includegraphics[scale=0.5]{d20throw.png}
\caption{Probability of the sum of 6-sided and 20-sided dice, calculated with Eqn. \ref{eq:d6}. Note that these are simply the convolutions of $n=1,2,3,...$ square waves, which rapidly adopts the shape of a Bell curve.}
\end{figure}


\subsection{Dynamic programming}
$$
a = \begin{bmatrix}
	\dot{x_1}\\
	\dot{x_2}
\end{bmatrix} = \begin{bmatrix}
	x_2\\
	-0.4x_1 -0.2x_2^2
\end{bmatrix} + \begin{bmatrix}
	0\\
	1
\end{bmatrix}u
$$
$$
\begin{bmatrix}
	x_1(k+1)\\
	x_2(k+1)
\end{bmatrix} = \begin{bmatrix}
	x_2(k)\\
	-0.4x_1(k) -0.2x_2^2(k) + u(k)
\end{bmatrix} \Delta t + \begin{bmatrix}
	x_1(k)\\
	x_2(k)
\end{bmatrix}
$$
In MATLAB Code, Control law will save in .mat file and we can use for another initial condition very fast without so much processing.
\begin{figure}[H]
	\caption{Dynamic Programming}
	\centering
	\includegraphics[width=11.5cm]{../Figure/Q4/DP.png}
\end{figure}


\lab{Monte Carlo Integration}{Monte Carlo Integration}
\objective{Implement Monte Carlo integration to estimate integrals. Use Monte Carlo Integration to calculate the integral of the joint normal distribution.}

Some multivariable integrals which are critical in applications are impossible to evaluate symbolically.
For example, the integral of the joint normal distribution
\[
\int_{\Omega} \frac{1}{\sqrt{(2 \pi)^k}} e^{- \frac{\bold{x}^T\bold{x}}{2}}
\]
is ubiquitous in statistics.
However, the integrand does not have a symbolic antiderivative.
This means we must use numerical methods to evaluate this integral. The standard technique for numerically evaluating multivariable integrals is \emph{Monte Carlo Integration}. In the next lab, we will approximate this integral using a modified version of Monte Carlo Integration. In this lab, we address the basics of Monte Carlo Integration.

Monte Carlo integration is radically different from techniques like Simpson's rule. Whereas Simpson's rule is purely computational and deterministic, Monte Carlo integration uses randomly chosen points  in the domain to calculate the integral.
Although it converges slowly, Monte Carlo integration is frequently used to evaluate multivariable integrals because the higher-dimensional analogs of methods like Simpson's rule are extremely inefficient.

\section*{A Motivating Example} % =============================================

Suppose we want to numerically compute the area of a circle of radius 1.
From analytic methods, we know the answer is $\pi$.
Empirically, we can estimate the area by randomly choosing points in a $2 \times 2$ square.
The percentage of points that land in the inscribed circle, times the area of the square, should approximately equal the area of the circle (see Figure \ref{fig:MCCircle}).

\begin{figure}
\includegraphics[width=.7\textwidth]{figures/MC.png}
\caption{Finding the area of a circle using random points}
\label{fig:MCCircle}
\end{figure}

We do this in NumPy as follows. First generate 500 random points in the square $[0,1] \times [0,1]$.
\begin{lstlisting}
>>> N = 500 # Number of sample points
>>> points = np.random.rand(2, N)
\end{lstlisting}
We rescale and shift these points to be uniformly distributed in $[-1,1]\times[-1,1]$.
\begin{lstlisting}
>>> points = points*2-1
\end{lstlisting}
Next we determine the number of points in the unit circle.
We compute the Euclidean distance from the origin for each point, then count the points that are within a distance of 1 from the origin.
%The function \li{np.hypot(a, b)} returns the norm of the vector $(a, b)$ where $a$ and $b$ are the $x$- and $y$-components, respectively.
\begin{lstlisting}
>>> # Compute the distance from the origin for each point
>>> pointsDistances = np.linalg.norm(points,axis=0)
>>> # Count how many are less than 1
>>> numInCircle = np.count_nonzero(pointsDistances < 1)
\end{lstlisting}
The fraction of points inside the circle is \li{numInCircle} divided by \li{N}. By multiplying this fraction by the square's area, we can estimate the area of the circle.
\begin{lstlisting}
>>> circleArea = 4.*(numInCircle/N)
>>> circleArea
3.024
\end{lstlisting}
This differs from $\pi$ by about 0.117.

\begin{problem}
\label{prob:sphere}
Write a function that estimates the volume of the unit sphere. Your function should have a keyword argument \li{N} that defaults to $10^5$. Your function should draw \li{N} points uniformly from $[-1,1]\times[-1,1]\times[-1,1]$ to make your estimate. The true volume is $\frac{4}{3}\pi \approx 4.189$.
\end{problem}

%This demonstrates the main idea of Monte Carlo integration: that random points can be used to estimate a volume.
%In the next section we will present the generalized method of Monte Carlo integration, which can be used to integrate any function.

\section*{Monte Carlo Integration} % ==========================================

In the examples above, we drew a bounding box around a volume, then used random points drawn from the box to estimate that volume.
This is easy and intuitive when the volume is a circle or sphere.
But it's hard to generalize this to any arbitrary integral - in order to draw the box, we have to already know something about the volume we are estimating.
Instead, given an arbitrary function $f(x) : \mathbb{R}^n \mapsto \mathbb{R} $ and a region $\Omega \subset \mathbb{R}^n$ in the domain of $f$, we would like to use random points drawn from $\Omega$ to estimate the integral
$\int_\Omega f(x)\:dV$,
\textit{without} having to specify a bounding box around the volume of integration.

We can estimate this integral using the approximation.
\begin{equation}\label{equ:mc_eq}
\int_\Omega f(x)\:dV \approx V(\Omega) \frac{1}{N} \sum_{i=1}^N f(x_i)
\end{equation}
where $x_i$ are uniformly distributed random points in $\Omega$ and $V(\Omega)$ is the volume of $\Omega$.
This is the generalized formula for Monte Carlo integration.

The intuition behind \eqref{equ:mc_eq} is that $\frac{1}{N} \sum_{i=1}^N f(x_i)$ approximates the average value of $f$ on $\Omega$.
We multiply this (approximate) average value by the volume of $\Omega$ to get the (approximate) integral of $f$ on $\Omega$.

For further intuition, compare \eqref{equ:mc_eq} to the Average Value Theorem from single-variable calculus.
By the Average Value Theorem, the average value of $f(x) : \mathbb{R} \mapsto \mathbb{R} $ on ${[a,b]}$ is given by
\begin{equation}\label{equ:avg_value_eq}
f_{avg} = \frac{1}{b-a}\int_a^b f(x)\:dx.
\end{equation}

If we let $\Omega = {[a,b]}$ in \eqref{equ:avg_value_eq} (noting that $V(\Omega) = b - a$) and replace $f_{avg}$ with the approximation $\frac{1}{N} \sum_{i=1}^N f(x_i)$, then we get precisely the Monte Carlo integration formula in Equation \eqref{equ:mc_eq}!

As it turns out, we can refactor the circle-area problem slightly so that it uses Equation \eqref{equ:mc_eq}.
Let $f$ be defined by
$$f(x,y) = \begin{cases} 1 &\mbox{ if $(x$,$y)$ is in the unit circle} \\ 0 &\mbox{ otherwise} \end{cases}$$
and let $\Omega =   {[-1,1]} \times {[-1,1]}$.
The area of the circle is given by
$\int_\Omega f(x)\:dV$,
which we can estimate with the Monte Carlo integration formula:
\[
\mbox{Area of unit circle } \approx V(\Omega) \frac{1}{N} \sum_{i=1}^N f(x_i) = \frac{4}{N} \sum_{i=1}^N f(x_i).
\]

To summarize, we have the following steps to estimate the integral of any function $f$ over a region $\Omega$ in the domain of $f$:
\begin{enumerate}
\item Draw $N$ random points uniformly distributed in $\Omega$.
\item Find the image of each point under $f$, and take the average of these images.
\item Multiply by the volume of $\Omega$.
\end{enumerate}

\begin{comment}
% Part of the previous explanation of Monte Carlo integration, which was reworked above.
You can calculate the area of the unit circle with the following integration problem:
\[
\mbox{Area of unit circle } = \int_{[-1,1]\times[-1,1]} f(x,y) dA
\]
where
\begin{equation}\label{equ:mc_func}
f(x,y) = \begin{cases} 1 &\mbox{ if $(x$,$y)$ is in the unit circle} \\ 0 &\mbox{ otherwise.} \end{cases}
\end{equation}
%$

This method essentially draws a box around the function to estimate the integral. This works fine if we know the bounds of the function. However, if we don't know the bounds of the function we don't know the dimensions of the box much be to encapsulate the function.

Luckily, we can still use a random-points method as above to approximate any integral, even if we don't know the bounds. Suppose we wish to evaluate
\[
\int_\Omega f(x)\:dV.
\]
We can approximate this integral using the formula
\begin{equation}\label{equ:mc_eq}
\int_\Omega f(x)\:dV \approx V(\Omega) \frac{1}{N} \sum_{i=1}^N f(x_i),
\end{equation}
where $x_i$ are uniformly distributed random vectors in $\Omega$ and $V(\Omega)$ is the volume of $\Omega$.
This is the formula for Monte Carlo integration.

In our example, $\Omega$ was the box $[-1,1] \times [-1,1]$ and $f$ was the function defined in \eqref{equ:mc_func}.
Then $\sum_{i=1}^N f(x_i)$ is the number of points in the unit circle, $N$ is the total number of points, and \eqref{equ:mc_eq} is the same as the formula we derived previously.

The intuition behind \eqref{equ:mc_eq} is that $\frac{1}{N} \sum_{i=1}^N f(x_i)$ approximates the average value of $f$ on $\Omega$.
We multiply this (approximate) average value by the volume of $\Omega$ to get the (approximate) integral of $f$ on $\Omega$.

As a 1-dimensional example consider the integral
\[
\int_0^1 x\,dx \approx (1-0)\frac{1}{N} \sum_{i=1}^N x_i=\frac{1}{N} \sum_{i=1}^N x_i.
\]

The integral on the left-hand-side is $1/2$.
In the approximation on the right-hand-side, $x_i$ is drawn from a uniform distribution on $[0,1]$.
The average of $N$ such draws will converge to $1/2$.

\begin{problem}
\label{prob:crazy_sine}
Monte Carlo Integration is particularly useful when trying to approximate integrals that would be difficult to calculate otherwise. Write a function that approximates the following integral:
$$\int_{1}^{5} \left | \text{sin}(10x)\text{cos}(10x) + \sqrt{x}\text{sin}(3x) \right | dx$$
Your function should accept a keyword argument \li{N} that defaults to $10^5$. Your answer should be approximately $4.502$.
\end{problem}

\end{comment}

\begin{problem}
\label{prob:mc_1d}
Write a function that performs 1-dimensional Monte Carlo integration.
Given a function $f: \mathbb{R} \mapsto \mathbb{R}$, an interval ${[a,b]}$, and the number of random points to use, your function should return an approximation of the integral
$\int_a^b f(x)\:dx$.
Let the number of sample points default to $10^5$.
Test your function by estimating integrals that you can calculate by hand.

\end{problem}

\begin{problem}
\label{prob:mc}
Generalize Problem \ref{prob:mc_1d} to multiple dimensions.
Write a function that accepts a function handle $f$ to integrate, the bounds of the interval to integrate over, and the number of points to use.
Let the number of sample points default to $10^5$.
Your implementation should be robust enough to integrate any function $f$ : $\mathbb{R}^n \mapsto \mathbb{R}$ over any interval in $\mathbb{R}^n$.

Hints:
\begin{enumerate}
\item To draw a random array of points from the given interval, first create a random array of points in $[0,1] \times \ldots \times [0,1]$.
Multiply this array by the dimensions of the interval to rescale it, then add the lower bounds of integration to shift it.
Think about using array broadcasting.

\item You can use \li{np.apply_along_axis()} to apply a function to each column of an array.
Here is an example of applying a function to points in $\mathbb{R}^2$:

\begin{lstlisting}
>>> points = np.random.rand(2,4)
>>> points
array([[ 0.33144631,  0.52558001,  0.67766158,  0.45570083],
       [ 0.70935864,  0.20985475,  0.25917177,  0.19431292]])
# Apply the norm function to each point
>>> np.apply_along_axis(np.linalg.norm,0,points)
array([ 0.78297275,  0.565927  ,  0.72553099,  0.49539959])
\end{lstlisting}

This is especially useful for functions that don't work nicely with array inputs.
For example, the code below uses a simple thresholding function \li{f}; calling \li{f(points)} would throw an error, but using \li{np.apply_along_axis} gives the expected result.
(Note that \li{points} must have at least 2 dimensions for this to work.)
Refer to the NumPy docs for more information.

\begin{lstlisting}
# Simple function that returns a 0 if x is less than 0.5
>>> f = lambda x: x if x > 0.5 else 0.
# Get 4 random points. The 1 forces the array to be 2-D.
>>> points = np.random.rand(1,4)
>>> points
array([[ 0.2144746 ,  0.02490517,  0.86593995,  0.86401139]])
# Evaluate f at each element of points
>>> np.apply_along_axis(f,0,points)
array([ 0.        ,  0.        ,  0.86593995,  0.86401139])
\end{lstlisting}

%Experiment with this on your own to make sure the function does what you expect.
%In this example, we chose the axis parameter to be 1 to evaluate the rows of the matrix.
%If you would like a refresher on axes, see Lab \ref{lab:NumPyArrays}.

\end{enumerate}
\end{problem}

\begin{comment}

\begin{problem}
\label{prob:mc_test}
The exact value of the integral of
\[
f(x,y,z,w) = \text{sin}(x) y^5 -y^3 + zw + yz^3
\]
on $[-1,1]\times[-1,1]\times[-1,1]\times[-1,1]$ is 0.
Run the function \li{mc_int()} you wrote in Problem 3 on $f$ with 100, 1000, and 10000 sample points. Use the default value of 100 iterations for your approximations.
Print the errors of your estimates to the terminal.
\end{problem}

\begin{problem}
The standard normal distribution is an important object of study in probability and statistic.
It is defined by the density function $\frac{1}{\sqrt{2 \pi}} e^{- \frac{x^2}{2}}$.
(Here we are assuming a mean of $0$ and a variance of $1$).
This is a function that cannot be integrated symbolically.
We can use monte carlo integration to estimate the probability that a normally distributed random variable will take a value below a given point.
The probability that the random variable we are considering is less than (or equal to) a given value $x$ is
\[\int_{-\infty}^x \frac{1}{\sqrt{2 \pi}} e^{- \frac{t^2}{2}} dt\]
This function is essentially zero for values of $x$ that lie reasonably far from the mean, so we can estimate this probability by integrating from $-5$ to $x$ instead.

Compare your result at $x = 1$ with the output of the code

\begin{lstlisting}
from scipy.stats import norm
N = norm()
N.cdf(1)
\end{lstlisting}

\end{problem}
\end{comment}

One application of Monte Carlo integration is integrating probability density functions that do not have closed form solutions.

\begin{problem}
The joint normal distribution of $N$ independent random variables with mean 0 and variance 1 is
\[
f(\x) = \frac{1}{\sqrt{2 \pi}^N} e^{- \frac{\x^T\x}{2}}.
\]
The integral of $f(\x)$ over a box is the probability that a draw from the distribution will be in the box.
This is an important distribution in statistics.
However, $f(\x)$ does not have a symbolic antiderivative.
\begin{enumerate}
\item Let $\Omega=[-1.5, 0.75]\times[0,1]\times[0, 0.5]\times[0,1] \subset \mathbb{R}^4$.
Use the function you wrote in Problem \ref{prob:mc} to integrate $f(\x)$ on $\Omega$.
Use 50000 sample points.
\item SciPy has a built in function specifically for integrating the joint normal distribution.
The integral of $f(\x)$ on $B = [-1,1]\times [-1,1]\times[-1,1] \subset \mathbb{R}^3$ can be computed in SciPy with the following code.

\begin{lstlisting}
>>> from scipy import stats

# Define the bounds of the box to integrate over
>>> mins = np.array([-1, -1, -1])
>>> maxs = np.array([1, 1, 1])

# Each variable has mean 0
>>> means = np.zeros(3)

# The covariance matrix of N independent random variables
#    is the NxN identity matrix.
>>> covs = np.eye(3)

# Compute the integral
>>> value, inform = stats.mvn.mvnun(mins, maxs, means, covs)
\end{lstlisting}

Then \li{value} is the integral of $f(\x)$ on $B$.

Use SciPy to integrate $f(\x)$ on $\Omega$.

\item Return your Monte Carlo estimate, SciPy's answer, and (assuming SciPy is correct) the relative error of your Monte Carlo estimate.
\end{enumerate}
\end{problem}

\section*{Convergence} % ======================================================

The error of the Monte Carlo method is proportional to $1/\sqrt{N}$, where $N$ is the number of points used in the estimation.
This means that to divide the error by 10, we must sample \emph{100 times} more points.

This is a slow convergence rate, but it is independent of the number of dimensions of the problem.
The error converges at the same rate whether integrating a 2-dimensional or a 20-dimensional function.
This gives Monte Carlo integration an advantage over other methods, and makes it especially useful for estimating integrals in high dimensions.

\begin{figure}
\includegraphics[width=.7\textwidth]{figures/MC_error_2.pdf}
\caption{The Monte Carlo integration method was used to compute the volume of the unit sphere.
The blue line plots the average error in 50 runs of the Monte Carlo method on $N$ sample points.
The red line is a plot of $1/\sqrt{N}$. }
\label{fig:mc_error_2}
\end{figure}

\begin{problem}
In this problem we will visualize how the error in Monte Carlo integration depends on the number of sample points.

Run Problem \ref{prob:sphere} with \li{N} equal to 50, 100 and 500, as well as 1000, 2000, 3000, ..., 50000;
having some additional small values of \li{N} will help make the visualization better.

For each value of \li{N}:
\begin{enumerate}
\item Estimate the volume of the unit sphere using Problem \ref{prob:sphere}, and use the true volume to calculate the relative error of the estimate.
\item Repeat this multiple times to get an average estimate of the relative error. Your function should accept a keyword argument \li{numEstimates} that defaults to 50.
\item Calculate and store the mean of the errors.
\end{enumerate}

Plot the mean relative error as a function of \li{N}.
For comparison, plot the function $1/\sqrt{N}$ on the same graph.
Your plot should resemble Figure \ref{fig:mc_error_2}).

\end{problem}

\section*{A Caution} % ========================================================

You can run into trouble if you try to use Monte Carlo integration on an integral that does not converge.
For example, we may attempt to evaluate
\[
\int_0^1 \frac{1}{x}\:dx
\]
with Monte Carlo integragtion using the following code.
\begin{lstlisting}
>>> k = 5000
>>> np.mean(1/np.random.rand(k,1))
21.237332864358656
\end{lstlisting}

Since this code returns a finite value, we could assume that this integral has a finite value.
In fact, the integral is infinite.
We could discover this empirically by using larger and larger values of $k$, and noting that Monte Carlo integration fails to converge.

\begin{comment}
\begin{problem}
\label{prob:mc_flawed}
Create a new function (based upon the function from Problem \ref{prob:mc}) that uses a ``flawed'' random number generator that doesn't produce numbers between $-.95$ and $-1$. Test your method on the function from Problem \ref{prob:mc_test}. How bad is the error?
\end{problem}
\end{comment}


\subsection{Linearity}
\noindent
We already know from calculus that the integral is a linear operator.
That is, scalar multiplication can be pulled out of the integral, and the integral of the sum of two functions is the same as the sum of the integrals of the functions.
Since the Laplace transform is just an integral, it is also a linear operator.
So we can say
\begin{equation*}
	\Laplace{cf + g} = c\Laplace{f} + \Laplace{g} \text{, } c \in \R.
\end{equation*}


\subsection{Gram-Schmidt process}

The Gram-Schmidt process is an algorithm to transform a set of vectors
into an orthonormal set spanning the same subspace, that is generating
the same collection of linear combinations (see Definition~\ref{def:linear-combination}).

The goal of the Gram-Schmidt process is to take a linearly
independent set of vectors and transform it into an orthonormal set
with the same span.  The first objective is to construct an orthogonal
set of vectors with the same span, since from there an orthonormal set
can be obtained by simply dividing each vector by its length.

\begin{algorithm}{Gram-Schmidt process}{gram-schmidt-algorithm}
Let $\set{\vect{u}_1,\ldots,\vect{u}_n} $ be a set of
linearly independent vectors in $\R^{n}$.

\textbf{I:} Construct a new set of vectors $\set{\vect{v}_1,\ldots,\vect{v}_n} $ as follows:
\[ \begin{array}{ll}
\vect{v}_1 & = \vect{u}_1 \\
\vect{v}_{2} & = \vect{u}_{2} - \paren{\dfrac{ \vect{u}_2 \dotprod \vect{v}_1}{\norm{\vect{v}_1}^2}}  \vect{v}_1\\
\vect{v}_{3} & = \vect{u}_{3} - \paren{\dfrac{\vect{u}_3 \dotprod \vect{v}_1}{\norm{\vect{v}_1}^2}}  \vect{v}_1
	 - \paren{\dfrac{\vect{u}_3 \dotprod \vect{v}_2}{\norm{\vect{v}_2}^2}}  \vect{v}_2\\
\vdots \\
\vect{v}_{n} & = \vect{u}_{n} - \paren{\dfrac{\vect{u}_n \dotprod \vect{v}_1}{\norm{\vect{v}_1}^2}}  \vect{v}_1
	 - \paren{\dfrac{\vect{u}_n \dotprod \vect{v}_2}{\norm{\vect{v}_2}^2}} \vect{v}_2 - \ldots
	 - \paren{\dfrac{\vect{u}_{n} \dotprod \vect{v}_{n-1}}{\norm{\vect{v}_{n-1}}^2}} \vect{v}_{n-1} \\
\end{array} \]

\textbf{II:} Now let $\vect{w}_i = \dfrac{\vect{v}_i}{\norm{\vect{v}_i}}$ for $i=1,\ldots,n$.

Then
\begin{enumerate}
\item $\set{\vect{v}_1,\ldots, \vect{v}_n }$ is an orthogonal set.
\item  $\set{\vect{w}_1,\ldots, \vect{w}_n  } $ is an orthonormal set.
\item $\sspan\set{\vect{u}_1,\ldots,\vect{u}_n } = \sspan \set{\vect{v}_1,\ldots, \vect{v}_n } = \sspan\set{\vect{w}_1,\ldots,\vect{w}_n }$.
\end{enumerate}
\end{algorithm}

\begin{proof}
The full proof of this algorithm is beyond this material, however here is an indication of the arguments.

To show that  $\set{\vect{v}_1,\ldots, \vect{v}_n  } $ is an orthogonal set, let
\[ a_2 =  \dfrac{ \vect{u}_2 \dotprod \vect{v}_1}{\norm{\vect{v}_1}^2} \]
then:
\[
\begin{array}{ll}
 \vect{v}_1 \dotprod \vect{v}_2 &  =  \vect{v}_1 \dotprod (\vect{u}_2 - a_2 \vect{v}_1)  \\
 & = \vect{v}_1 \dotprod \vect{u}_2 - a_2 (\vect{v}_1 \dotprod \vect{v}_1  \\
 & = \vect{v}_1 \dotprod \vect{u}_2 - \dfrac{ \vect{u}_2 \dotprod \vect{v}_1}{\norm{\vect{v}_1}^2} \norm{\vect{v}_1}^2 \\
 & = ( \vect{v}_1 \dotprod \vect{u}_2 ) - ( \vect{u}_2 \dotprod \vect{v}_1 ) =0\\
\end{array}
\]
Now that you have shown that $\set{\vect{v}_1, \vect{v}_2}$ is orthogonal,  use the same method as above to show that
$\set{\vect{v}_1, \vect{v}_2, \vect{v}_3}$ is also orthogonal,  and so on.

Then in a similar fashion you show that $\sspan\set{
\vect{u}_1,\ldots,\vect{u}_n } = \sspan\set{
\vect{v}_1,\ldots,\vect{v}_n }$.

Finally defining $\vect{w}_i =
\dfrac{\vect{v}_i}{\norm{\vect{v}_i}}$ for $i=1,\ldots,n$ does not affect orthogonality and yields vectors of length 1,
hence an orthonormal set. You can also observe that it does not affect
the span either and the proof would be complete.
\end{proof}

Consider the following example.

\begin{example}{Find orthonormal set with same span}{orthonormal-same-span}
Consider the set of vectors $\set{\vect{u}_1, \vect{u}_2}$ given as in Example~\ref{exa:span-vectors}. That is
\[
\vect{u}_1=\begin{mymatrix}{r}
1 \\
1 \\
0
\end{mymatrix}, \vect{u}_2=\begin{mymatrix}{r}
3 \\
2 \\
0
\end{mymatrix} \in \R^{3}
\]

Use the Gram-Schmidt algorithm to find an orthonormal set of vectors $\set{\vect{w}_1, \vect{w}_2}$ having the same span.
\end{example}

\begin{solution}
We already remarked that the set of vectors in $\set{\vect{u}_1,
\vect{u}_2}$ is linearly independent, so we can proceed with the
Gram-Schmidt algorithm:
\begin{eqnarray*}
\vect{v}_1 &=& \vect{u}_1 =  \begin{mymatrix}{r}
1 \\
1 \\
0
\end{mymatrix} \\
&& \\
\vect{v}_{2} &=& \vect{u}_{2} - \paren{\dfrac{\vect{u}_2 \dotprod \vect{v}_1}{\norm{\vect{v}_1}^2}}  \vect{v}_1\\
&& \\
&=& \begin{mymatrix}{r}
3 \\
2 \\
0
\end{mymatrix}
-
\frac{5}{2}
 \begin{mymatrix}{r}
1 \\
1 \\
0
\end{mymatrix} \\
&&\\
&=&  \begin{mymatrix}{r}
\vspace{0.05in}\frac{1}{2} \\
-\vspace{0.05in}\frac{1}{2} \\
0
\end{mymatrix}
\end{eqnarray*}

Now to normalize simply let
\begin{eqnarray*}
\vect{w}_1  = \frac{\vect{v}_1}{\norm{\vect{v}_1}}  = \begin{mymatrix}{r}
\vspace{0.05in}\frac{1}{\sqrt{2}}  \\
\vspace{0.05in}\frac{1}{\sqrt{2}} \\
0
\end{mymatrix} \\
\vect{w}_2  = \frac{\vect{v}_2}{\norm{\vect{v}_2}}  = \begin{mymatrix}{r}
\vspace{0.05in}\frac{1}{\sqrt{2}}  \\
-\vspace{0.05in}\frac{1}{\sqrt{2}} \\
 0
\end{mymatrix}
\end{eqnarray*}

You can verify that $\set{\vect{w}_1, \vect{w}_2}$ is an orthonormal set of vectors  having the same span as
$\set{\vect{u}_1, \vect{u}_2}$,  namely the $XY$-plane.
\end{solution}

In this example, we began with a linearly independent set and found an orthonormal set of vectors which had the same span. It turns out that if we start with a basis of a subspace and apply the Gram-Schmidt algorithm, the result will be an orthogonal basis of the same subspace. We examine this in the following example.

\begin{example}{Find a corresponding orthogonal basis}{orthogonal-basis}
Let
\[ \vect{x}_1=\begin{mymatrix}{c} 1\\ 0\\ 1\\ 0 \end{mymatrix},
\vect{x}_2=\begin{mymatrix}{c} 1\\ 0\\ 1\\ 1 \end{mymatrix},
\mbox{ and }
\vect{x}_3=\begin{mymatrix}{c} 1\\ 1\\ 0\\ 0 \end{mymatrix},\]
and let $U=\sspan\set{\vect{x}_1, \vect{x}_2,\vect{x}_3}$. Use the Gram-Schmidt Process
to construct an orthogonal basis $B$ of $U$.
\end{example}

\begin{solution}
First $\vect{f}_1=\vect{x}_1$.

Next,
\[ \vect{f}_2=\begin{mymatrix}{c} 1\\ 0\\ 1\\ 1 \end{mymatrix}
-\frac{2}{2}\begin{mymatrix}{c} 1\\ 0\\ 1\\ 0 \end{mymatrix}
=\begin{mymatrix}{c} 0\\ 0\\ 0\\ 1 \end{mymatrix}.\]

Finally,
\[ \vect{f}_3=\begin{mymatrix}{c} 1\\ 1\\ 0\\ 0 \end{mymatrix}
-\frac{1}{2}\begin{mymatrix}{c} 1\\ 0\\ 1\\ 0 \end{mymatrix}
-\frac{0}{1}\begin{mymatrix}{c} 0\\ 0\\ 0\\ 1 \end{mymatrix}
=\begin{mymatrix}{c} 1/2\\ 1\\ -1/2\\ 0 \end{mymatrix}.\]

Therefore,
\[ \set{
\begin{mymatrix}{c} 1\\ 0\\ 1\\ 0 \end{mymatrix},
\begin{mymatrix}{c} 0\\ 0\\ 0\\ 1 \end{mymatrix},
\begin{mymatrix}{c} 1/2\\ 1\\ -1/2\\ 0 \end{mymatrix}
}\]
is an orthogonal basis of $U$.
However, it is sometimes more convenient to deal with vectors
having integer entries, in which case we take
\[ B=\set{
\begin{mymatrix}{c} 1\\ 0\\ 1\\ 0 \end{mymatrix},
\begin{mymatrix}{c} 0\\ 0\\ 0\\ 1 \end{mymatrix},
\begin{mymatrix}{r} 1\\ 2\\ -1\\ 0 \end{mymatrix}
}.\]
\end{solution}


\subsection{Resolution rule}

\subsubsection{Proof by resolution}

If we have a string of or statements, \(A\lor B\lor C\), and another which contains the completement of one element \(X\lor \neg B\lor Y\), we can infer:

\(A\lor C\lor X\lor Y\)

If the second statement has only one formula, then we have:

\(A\lor B\lor C\) and \(\neg B\) implying \(A\lor C\)


		\subsubsection*{Max flow}
			We can make nodes $v_{u,v},\{u,v\}\in E$, $v_i,i\in V$, a source $S$ and a sink $T$. Then connect edges from $S$ to $v_{u,v}$ with capacity of $1$, from $v_{u,v}$ to $v_u$ and $v_v$ with capacity of $1$ and from $v_i$ to $T$ with capacity of $c(i)$. If this graph has a maximum flow of $|E|$, then the original graph has a feasible orientation.\par
			\begin{figure}[H]
				\centering\includegraphics[width=3in]{source/qwd/qa.eps}
			\end{figure}
			For a maximum flow $S$, since the flow is full and nodes $v_{u,v}$ has exactly one unit flow input, exactly one of the edges $<v_{u,v}, v_u>$ and $<v_{u,v}, v_v>$ will has a flow. We can simply orientate edge $\{u,v\}$ to $u\to v$ if $<u_{u,v}, v_v>$ has a flow, or $v\to u$ if $<u_{u,v}, v_u>$ has a flow. Thus, the in-dgree of $i$ is just the flow through $v_i$, which is less than or equal to the capacity to sink, $c(i)$. So, we get a feasible orientation.\par
			For a feasible orientation, %since convert ${u,v}$ to both $<u,v>$ and $<v,u>$ is meaningless(otherwise we can delete one of them and make maximum in-dgree of $u$ and $v$ decreas by $1$).
			if $\{u, v\}$ is converted to $u\to v$, we can add the flow on path $S\to v_{u, v}\to v_v\to T$ by one unit, or $S\to v_{u, v}\to v_u\to T$ when converted to $v\to u$. Since the in-degree of a node $i$ doesn't exceed $c(i)$, so the flow from $v_i$ to $T$ is less than or equal to the capacity $c(i)$.
		\subsubsection*{Maximum matching}
			We can make nodes $vl_{u,v},\{u,v\}\in E$ as $X$ and $vr_{i,j},i\in V,1\leq j\leq c(i)$ as $Y$. Then connect $c(u)$ edges between $vl_{u, v}$ and $vr_{u,j},1\leq j\leq c(u)$, $c(v)$ edges between $vl_{u, v}$ and $vr_{v,k},1\leq k\leq c(v)$. If this bipartite graph has a maximum matching of size $|E|$, then the original graph has a feasible orientation.\par
			\begin{figure}[H]
				\centering\includegraphics[width=2in]{source/qwd/qb.eps}
			\end{figure}
			For a maximum matching $S$, since the size of matching is $|E|$, every node $vl_{u, v}$ has a paired node in $Y$, then we convert $\{u, v\}$ to $u\to v$ if the paired node is $vr_{v,i}$, or to $v\to u$ if that node is $vr_{u,i}$. Since nodes $vr_{i, j}$ in $Y$ is no more than $c(i)$, so the in-degree of $i$ is less or equal to $c(i)$. Then we get a feasible orientation.\par
			For a feasible orientaion, we processe edges one by one in an arbitrary order. If $\{u, v\}$ is converted to $u\to v$, we can match $vl_{u, v}$ with $vr_{v, i}$, or $vr_{u, j}$ when converted to $v\to u$, where $i$ and $j$ is the smallest one that $vr_{v, i}$, $vr_{u, j}$ has not matched before. Since the in-degree of a node $i$ doesn't exceed $c(i)$, so there is always an avalible $i$ or $j$.


% !TEX root = main.tex
%=====================================================================
\chapter{Bayesian Inference}\label{chap:randomprocesses}

%----------------------------------------------------------------------
%\section{The Bayesian approach}
%----------------------------------------------------------------------

So far we have looked at \emph{frequentist inference}, which assumes that an unknown parameter $\theta$ has a fixed (but unknown) value.
\bit
\it The PMF/PDF of an observation is written as $f(x;\theta)$.
\it The likelihood function is written as $L(\theta;x)$.
\it Estimators such as the MME and MLE claim to estimate the `true' value of $\theta$.
\eit

For \emph{Bayesian inference}, we instead think of an unknown parameter $\theta$ as a \emph{random variable}.
\bit
\it The PMF/PDF of an observation is written as $f(x|\theta)$.
\it The likelihood function of is written as $L(\theta|x)$.
\it We seek to estimate the distribution of $\theta$.
\eit

%----------------------------------------------------------------------
\section{Bayes' theorem}
%----------------------------------------------------------------------

% events
If the events $A_1,A_2,\ldots$ form a partition of event $B$, Bayes' theorem states that
\[
\prob(A_j|B)
%	= \frac{\prob(B|A_j)\prob(A_j)}{\prob(B)}
	= \frac{\prob(B|A_j)\prob(A_j)}{\sum_k \prob(B|A_k)\prob(A_k)}.
\]

% discrete
Let $X$ and $Y$ be discrete random variables taking values in the sets $\{x_1,x_2,\ldots\}$ and $\{y_1,y_2,\ldots\}$ respectively. Because the events $\{Y=y_1\},\{Y=y_2\},\ldots$ form a partition of the event $\{X=x_i\}$, we have % so applying Bayes' theorem we obtain
\[
\prob(Y=y_j|X=x_i)
%= \frac{\prob(X=x_i|Y=y_j)\prob(Y=y_j)}{\prob(X=x_i)}
= \frac{\prob(X=x_i|Y=y_j)\prob(Y=y_j)}{\sum_k \prob(X=x_i|Y=y_k)\prob(Y=y_k)}.
\]
To avoid cluttering the notation with subscripts, we write this as
\[
\prob(Y=y|X=x)
%= \frac{\prob(X=x|Y=y)\prob(Y=y)}{\prob(X=x)}
= \frac{\prob(X=x|Y=y)\prob(Y=y)}{\sum_y \prob(X=x|Y=y)\prob(Y=y)},
\]
where the sum in the denominator is taken over the range of $Y$.
In terms of PMFs, this becomes
\[
f_{Y|X}(y|x)
%= \frac{f_{X|Y}(x|y)f_Y(y)}{f_X(x)}
= \frac{f_{X|Y}(x|y)f_Y(y)}{\sum_y f_{X|Y}(x|y)f_Y(y)}.
\]

% continuous
This extends directly to the case of continuous random variables, the only difference being that the denominator (which is the marginal PDF of $X$) is expressed by an integral:
\[
f_{Y|X}(y|x)
%= \frac{f_{X|Y}(x|y)f_Y(y)}{f_X(x)}
= \frac{f_{X|Y}(x|y)f_Y(y)}{\int f_{X|Y}(x|y)f_Y(y)\,dy}.
\]

% parameter estimation
For Bayesian inference, the unknown parameter $\theta$ takes the role of $Y$ in the above formulation. To simplify the notation we denote the PMF/PDF of $\theta$ by the symbol $\pi$:
\[
\pi(\theta|\mathbf{x}) = \left\{\begin{array}{ll}
	\displaystyle\frac{f(\mathbf{x}|\theta)\pi(\theta)}{\sum_{\theta}f(\mathbf{x}|\theta)\pi(\theta)} 			& \text{\quad ($\theta$ discrete)}, \\[5ex]
	\displaystyle\frac{f(\mathbf{x}|\theta)\pi(\theta)}{\int f(\mathbf{x}|\theta)\pi(\theta)\,d\theta} 	& \text{\quad ($\theta$ continuous)}.
\end{array}\right.
\]

This can also be expressed in terms of likelihood functions,
\[
\pi(\theta|\mathbf{x}) = \left\{\begin{array}{ll}
	\displaystyle\frac{L(\theta|\mathbf{x})\pi(\theta)}{\sum_{\theta}L(\theta|\mathbf{x})\pi(\theta)} 			& \text{\quad ($\theta$ discrete)}, \\[5ex]
	\displaystyle\frac{L(\theta|\mathbf{x})\pi(\theta)}{\int L(\theta|\mathbf{x})\pi(\theta)\,d\theta} 	& \text{\quad ($\theta$ continuous)}.
\end{array}\right.
\]

%----------------------------------------------------------------------
\section{The prior and posterior distributions}
%----------------------------------------------------------------------

%----------------------------------------------------------------------
\subsection*{The prior distribution}
%----------------------------------------------------------------------
% prior
Suppose we have an initial estimate for distribution of $\theta$, perhaps obtained as a result of some preliminary experiments.
\bit
\it This is called the \emph{prior distribution} of $\theta$, which we denote by $\pi_0(\theta)$.
\eit

An initial point estimate of $\theta$ can be computed from the prior distribution, for example
\bit
\it by the \emph{mean} of the prior distribution: $\hat{\theta} = \expe\big[\pi_0(\theta)\big]$ or
\it by the \emph{mode} of the prior distribution: $\hat{\theta} = \argmax_{\theta}\big[\pi_0(\theta)\big]$.
\eit

If we have no prior knowledge, we should initially consider every value of $\theta$ to be equally likely. For example, if $\theta$ is continuous and all we know is that $\theta$ belongs to some interval $[a,b]$, we should adopt the \emph{uniform} distribution over $[a,b]$ as the prior distribution of $\theta$,
\[
\pi_0(\theta) = \left\{\begin{array}{ll}
	1/(b-a)	& \text{if } a\leq\theta\leq b, \\
	0	& \text{otherwise.}
\end{array}\right.
\]
In this context, the uniform distribution is often called the \emph{na\"{\i}ve} or \emph{non-informative} prior.

%----------------------------------------------------------------------
\subsection*{The posterior distribution}
%----------------------------------------------------------------------
% posterior
Suppose we now obtain some sample data $\mathbf{x}=(x_1,\ldots,x_n)$.
\bit
\it Bayes' theorem can be used to combine the prior distribution with the data.
\it This yields an updated PMF/PDF $\pi_1(\theta)$, called the \emph{posterior distribution} of $\theta$.
\eit

By Bayes' theorem,
%\[
%\pi_1(\theta|\mathbf{x}) = \left\{\begin{array}{ll}
%	\displaystyle\frac{f(\mathbf{x}|\theta)\pi_0(\theta)}{\sum_{\theta}f(\mathbf{x}|\theta)\pi_0(\theta)} 			& \text{\quad ($\theta$ discrete)}, \\[3ex]
%	\displaystyle\frac{f(\mathbf{x}|\theta)\pi_0(\theta)}{\int_{\theta}f(\mathbf{x}|\theta)\pi_0(\theta)\,d\theta} 	& \text{\quad ($\theta$ continuous)}.
%\end{array}\right.
%\]
%
%This can also be expressed in terms of likelihood functions,
%\[
%\pi_1(\theta|\mathbf{x}) = \left\{\begin{array}{ll}
%	\displaystyle\frac{L(\theta|\mathbf{x})\pi_0(\theta)}{\sum_{\theta}L(\theta|\mathbf{x})\pi_0(\theta)} 			& \text{\quad ($\theta$ discrete)}, \\[3ex]
%	\displaystyle\frac{L(\theta|\mathbf{x})\pi_0(\theta)}{\int_{\theta}L(\theta|\mathbf{x})\pi_0(\theta)\,d\theta} 	& \text{\quad ($\theta$ continuous)}.
%\end{array}\right.
%\]

\[
\pi_1(\theta|\mathbf{x})
	= \frac{f(\mathbf{x}|\theta)\pi_0(\theta)}{\sum_{\theta}f(\mathbf{x}|\theta)\pi_0(\theta)}
\qquad\text{or}\qquad
\pi_1(\theta|\mathbf{x})
	= \frac{f(\mathbf{x}|\theta)\pi_0(\theta)}{\int f(\mathbf{x}|\theta)\pi_0(\theta)\,d\theta}.
\]
In terms of likelihood functions,
\[
\pi_1(\theta|\mathbf{x})
	= \frac{L(\theta|\mathbf{x})\pi_0(\theta)}{\sum_{\theta}L(\theta|\mathbf{x})\pi_0(\theta)}
\qquad\text{or}\qquad
\pi_1(\theta|\mathbf{x})
	= \frac{L(\theta|\mathbf{x})\pi_0(\theta)}{\int L(\theta|\mathbf{x})\pi_0(\theta)\,d\theta}.
\]

Having obtained the posterior distribution, we can compute an updated estimate of $\theta$, for example
\bit
\it by the mean of the posterior distribution: $\hat{\theta} = \expe\big[\pi_1(\theta)\big]$, or
\it by the mode of the posterior distribution: $\hat{\theta} = \argmax_{\theta}\big[\pi_1(\theta)\big]$.
\eit

% defn: MAP estimator
\begin{definition}
The mode of the posterior is called the \emph{maximum a-posteriori} or \emph{MAP} estimator of $\theta$.
\end{definition}

% remark
\begin{remark}
%The posterior distribution is
%\[
%\pi_1(\theta|\mathbf{x}) =
%\displaystyle\frac{L(\theta|\mathbf{x})\pi_0(\theta)}{\sum_{\theta}L(\theta|\mathbf{x})\pi_0(\theta)}
%\text{\quad or\quad}
%\pi_1(\theta|\mathbf{x}) =
%\displaystyle\frac{L(\theta|\mathbf{x})\pi_0(\theta)}{\int_{\theta}L(\theta|\mathbf{x})\pi_0(\theta)\,d\theta}
%\]
The denominator of the posterior depends only on $\mathbf{x}$, so the posterior is proportional to the likelihood times the prior:
%\begin{align*}
\[
\pi_1(\theta|\mathbf{x}) 	\propto L(\theta|\mathbf{x})\pi_0(\theta),
%\qquad\text{or}\qquad
%\text{"posterior}				\propto \text{likelihood}\times\text{prior"}.
\]
The MAP estimator is the value of $\theta$ that maximizes the numerator $L(\theta|\mathbf{x})\pi_0(\theta)$ of the posterior.
\bit
\it Note that when $\pi_0$ is the uniform distribution, the MAP estimator is just the MLE.
\eit
To compute the mean of $\pi_1(\theta|\mathbf{x})$ we also need to compute the denominator. This is not always easy, which is why the MAP estimator is more widely used in practical applications.
\end{remark}

%% example
%\begin{example}\label{ex:biscuits}
%We have three tins of biscuits. The first tin contains $30$ chocolate and $10$ plain biscuits, the second tin contains $20$ chocolate and $20$ plain biscuits, and the third tin contains $10$ chocolate and $30$ plain biscuits. A tin is chosen at random, and a biscuit is chosen at random from the tin.
%\ben
%\it If a chocolate biscuit is observed, estimate which tin was chosen.
%\een
%The experiment is repeated, but this time two biscuits are chosen at random from the tin.
%\ben\stepcounter{enumi}
%\it If two chocolate biscuits are observed, estimate which tin was chosen.
%\it If one chocolate biscuit and one plain biscuit are observed, estimate which tin was chosen.
%\een
%\end{example}
%
%\begin{solution}
%\ben
%\it % one chocolate
%Let $\Theta = \{\theta_1,\theta_2,\theta_3\}$ where the value $\theta_k$ indicates that tin $k$ was chosen ($k=1,2,3$). Before observing the biscuit, it is reasonable to suppose that each tin is equally likely to be chosen. Thus we adopt the \emph{uniform} prior distribution for $\theta$:
%\[
%\pi_0(\theta) = \frac{1}{3} \quad\text{for all}\quad \theta\in\Theta.
%\]
%Let $A$ be the event that a chocolate biscuit was observed. Then
%\[
%\prob(A|\theta_1)=3/4,\qquad \prob(A|\theta_2)=1/2,\qquad \prob(A|\theta_3)=1/4,
%\]
%or equivalently
%\[
%L(\theta_1| A)=3/4,\qquad L(\theta_2| A)=1/2,\qquad L(\theta_3| A)=1/4.
%\]
%By Bayes' theorem, the posterior distribution is
%\[
%\pi_1(\theta_k|A) = \frac{L(\theta_k| A)\pi_0(\theta_k)}{\sum_{k=1}^3 L(\theta_k| A)\pi_0(\theta_k)}
%\text{\quad\qquad ($k=1,2,3$).}
%\]
%For the first tin,
%\[
%\prob(\theta_1|A) = \frac{3/4\times 1/3}{(3/4\times 1/2) + (1/2\times 1/3) + (1/4\times 1/3)} = \frac{1}{2}.
%\]
%Similar calculations for the second and third tins yield the following posterior:
%\[
%\pi_1(\theta_1) = 1/2,\qquad \pi_1(\theta_2) = 1/3, \qquad \pi_1(\theta_3) = 1/6.
%\]
%The MAP estimate is $\hat{\theta}_{MAP} = \theta_1$, so our best guess is that the first tin was chosen.
%
% % <<
%
%\it % two chocolate
%Let $B$ be the event that two chocolate biscuits are observed,
%\[
%\prob(B|\theta_1)=87/156,\qquad \prob(B|\theta_2)=38/156,\qquad \prob(B|\theta_3)=9/156.
%\]
%If we assume the uniform prior $\pi_0$, we obtain the posterior distribution
%\[
%\pi_1(\theta_1|B) = 87/134,\qquad \pi_1(\theta_2|B) = 38/134, \qquad \pi_1(\theta_3|B) = 9/134.
%\]
%The MAP estimator again suggests that the first tin was chosen.
%
%\it % one chocolate, one plain
%Let $C$ be the event that one chocolate and one plain biscuit are observed,
%\[
%\prob(C|\theta_1)=60/156,\qquad \prob(C|\theta_2)=80/156,\qquad \prob(C|\theta_3)=60/156.
%\]
%If we assume the uniform prior $\pi_0$,
%\[
%\pi_1(\theta_1|C) = 3/10,\qquad \pi_1(\theta_2|C) = 4/10, \qquad \pi_1(\theta_3|C) = 3/10.
%\]
%This time, the MAP estimator leads us to assert that the second tin was chosen.
%\een
%\end{solution}

%----------------------------------------------------------------------
%\section{Example}
%----------------------------------------------------------------------
% example
\begin{example}\label{ex:biscuits}
Suppose we have three tins of biscuits. The first tin contains $30$ chocolate and $10$ plain biscuits, the second tin contains $20$ chocolate and $20$ plain biscuits, and the third tin contains $10$ chocolate and $30$ plain biscuits. A tin is selected at random, and a biscuit is chosen at random from the tin.
\ben
\it If a chocolate biscuit is chosen, estimate which tin was selected.
\een
The biscuit is replaced, then a biscuit is again chosen from the tin.
\ben\stepcounter{enumi}
\it If a chocolate biscuit is chosen, update your estimate regarding which tin was selected.
\it If a plain biscuit is chosen, update your estimate regarding which tin was selected.
\een
\end{example}

\begin{solution}
Let $\Theta = \{\theta_1,\theta_2,\theta_3\}$ where the value $\theta_k$ indicates that tin $k$ was selected ($k=1,2,3$). Before the biscuit is chosen, it is reasonable to suppose that each tin is equally likely to be selected. Thus we adopt the \emph{uniform} prior distribution for $\theta$:
\[
\pi_0(\theta) = \frac{1}{3} \quad\text{for all}\quad \theta\in\Theta.
\]

\ben
\it % one chocolate
Let $A$ be the event that a chocolate biscuit is chosen. Then
%\[
%\prob(A|\theta_1)=3/4,\qquad \prob(A|\theta_2)=1/2,\qquad \prob(A|\theta_3)=1/4,
%\]
%or equivalently
\[
L(\theta_1| A)=3/4,\qquad L(\theta_2| A)=1/2,\qquad L(\theta_3| A)=1/4.
\]
Using Bayes' theorem, the posterior distribution is
\[
\pi_1(\theta_k|A) = \frac{L(\theta_k| A)\pi_0(\theta_k)}{\sum_{k=1}^3 L(\theta_k| A)\pi_0(\theta_k)}
\text{\quad\qquad ($k=1,2,3$).}
\]
For the first tin,
\[
\pi_1(\theta_1|A) = \frac{3/4\times 1/3}{(3/4\times 1/3) + (1/2\times 1/3) + (1/4\times 1/3)} = \frac{1}{2}.
\]
Similar calculations for the second and third tins yield the following posterior distribution:
\[
\pi_1(\theta_1) = 1/2,\qquad \pi_1(\theta_2) = 1/3, \qquad \pi_1(\theta_3) = 1/6.
\]
The MAP estimate is $\hat{\theta}_{MAP} = \theta_1$, so our best guess is that the first tin was selected.

 % <<

\it % two chocolate
Let $B$ be the event that the a chocolate biscuit was chosen the second time:
\[
L(\theta_1|B)=3/4,\qquad L(\theta_2|B)=1/2,\qquad L(\theta_3|B)=1/4.
\]
Using $\pi_1$ as our prior distribution, we obtain an updated posterior distribution:
\[
\pi_2(\theta_k|B) = \frac{L(\theta_k|B)\pi_1(\theta_k)}{\sum_{k=1}^3 L(\theta_k| B)\pi_1(\theta_k)}
\text{\quad\qquad ($k=1,2,3$).}
\]
For the first tin,
\[
\pi_2(\theta_1|B) = \frac{3/4\times 1/2}{(3/4\times 1/2) + (1/2\times 1/3) + (1/4\times 1/6)} = \frac{9}{14}.
\]
Similar calculations for the second and third tins yield the following posterior:
\[
\pi_2(\theta_1) = 9/14,\qquad \pi_2(\theta_2) = 4/14, \qquad \pi_2(\theta_3) = 1/14.
\]
The MAP estimator again leads us to estimate that the first tin was selected.

\it % one chocolate, one plain
Let $C$ be the event that a plain biscuit was chosen the second time:
\[
L(\theta_1|C)=1/4,\qquad L(\theta_2|C)=1/2,\qquad L(\theta_3|C)=3/4.
\]
Again using $\pi_1$ as our prior distribution, we obtain an updated posterior:
\[
\pi_2(\theta_k|C) = \frac{L(\theta_k|C)\pi_1(\theta_k)}{\sum_{k=1}^3 L(\theta_k|C)\pi_1(\theta_k)}
\text{\quad\qquad ($k=1,2,3$).}
\]
For the first tin,
\[
\pi_2(\theta_1|C) = \frac{1/4\times 1/2}{(1/4\times 1/2) + (1/2\times 1/3) + (3/4\times 1/6)} = \frac{3}{10}.
\]
Similar calculations for the second and third tins yield the following posterior distribution:
\[
\pi_2(\theta_1) = 3/10,\qquad \pi_2(\theta_2) = 4/10, \qquad \pi_2(\theta_3) = 3/10.
\]
This time, the MAP estimator leads us to estimate that the second tin was selected.
\een
\end{solution}

%%----------------------------------------------------------------------
%\section{The scientific method}
%%----------------------------------------------------------------------
%
% remark
\begin{remark}%[The Scientific Method]
We can think of the biscuit tins in Example~\ref{ex:biscuits} as competing scientific hypotheses:
\bit
\it The probability assigned to each hypothesis indicates its \emph{relative plausibility}.
\it We update the relative plausibility of each competing hypothesis based on \emph{observation}.
\eit

%\vspace*{2ex}
In this way, Bayesian inference embodies the \emph{scientific method}.
%\ben
%\it Start with an initial set of beliefs about the relative plausibility of various hypotheses.
%\it Collect new data by conducting experiments.
%\it Refine the relative plausibility of the various hypotheses in the light of the new data.
%\it Repeat (2) and (3).
%\een
\end{remark}

\begin{exercise}
Suppose we have three coins $A$, $B$ and $C$ which have probabilities $1/4$, $1/2$ and $3/4$ respectively of showing heads. A coin is chosen at random, and tossed three times. If exactly two heads are obtained, use the maximum a-posteriori (MAP) estimator to estimate which coin was chosen.
\begin{answer}
First we define the parameter $\theta\in\{1,2,3\}$ such that $\{\theta=1\}$ is the event that coin $A$ is chosen, $\{\theta=2\}$ is the event that coin $B$ is chosen, and $\{\theta=3\}$ is the event that coin $C$ is chosen. We should initially assume that each coin is equally likely to be chosen, so we choose the uniform prior distribution:
\begin{align*}
\pi_0(1) & = \prob(\theta=1) = 1/3 \\
\pi_0(2) & = \prob(\theta=2) = 1/3 \\
\pi_0(3) & = \prob(\theta=3) = 1/3
\end{align*}
Let $T$ be the event that exactly two heads are obtained. Then
\begin{align*}
\prob(T|\theta=1)	& = 3(1/4)^2(3/4) = 9/64 \\
\prob(T|\theta=2)	& = 3(1/2)^2(1/2) = 3/8 \\
\prob(T|\theta=3)	& = 3(1/4)(3/4)^2 = 27/64
\end{align*}
The denominator of the posterior is the overall probability of obtaining exactly two heads:
\begin{align*}
\prob(T)
	& = \prob(T|\theta=1)\prob(\theta=1) + \prob(T|\theta=2)\prob(\theta=2) + \prob(T|\theta=3)\prob(\theta=3) \\
	& = 3(1/4)^2(3/4)(1/3) + 3(1/2)^3(1/3) + 3(3/4)^2(1/4)(1/3) \\
	& = 3/64 + 8/64 + 9/64 \\
	& = 20/64
\end{align*}
Hence the posterior distribution is given by
\[
\pi_1(\theta) = \frac{\prob(T|\theta)\pi_0(\theta)}{\prob(T)} =
\]
from which we obtain
\begin{align*}
\pi_1(1) & = 3/20, \\
\pi_1(2) & = 8/20, \\
\pi_1(3) & = 9/20.
\end{align*}
The MAP estimator (mode of the posterior) is $\theta=3$, which corresponds to coin $C$.
\end{answer}
\end{exercise}

%----------------------------------------------------------------------
\section{The binomial model}
%----------------------------------------------------------------------
A suitable model for estimating the distribution of a parameter in the interval $[0,1]$ is provided by the \emph{beta distribution}.

% definition
\begin{definition}\label{def:beta_distribution}
The beta distribution with parameters $\alpha,\beta>0$ is defined by the PDF
\[
f(x;\alpha,\beta) = \begin{cases}
	\displaystyle\frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)} & \text{if $0\leq x\leq 1$}, \\
	0												& \text{otherwise,}
\end{cases}
\]
where $B(\alpha,\beta)$ is the so-called \emph{beta function},
\[
B(\alpha,\beta) = \int_0^1 t^{\alpha-1}(1-t)^{\beta-1}\,dt,
\]
which is defined for all $\alpha,\beta>0$.
\end{definition}

%\begin{remark}[Special case]
%If $X\sim\text{Beta}(1,1)$, then $X\sim\text{Uniform}(0,1)$.
%\end{remark}

% lemma
\begin{lemma}
Let $X\sim\text{Beta}(\alpha,\beta)$. Then $\expe(X) = \displaystyle\frac{\alpha}{\alpha+\beta}$ and $\mode(X) = \displaystyle\frac{\alpha-1}{\alpha+\beta-2}$ provided that $\alpha,\beta > 1$.
%For $X\sim\text{Beta}(\alpha,\beta)$,
%\[
%\expe(X) = \frac{\alpha}{\alpha+\beta} \text{\quad and\quad} \var(Y) = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}.
%\]
%and if $\alpha,\beta > 1$,
%\[
%\mode(X) = \frac{\alpha-1}{\alpha+\beta-2}.
%\]
\end{lemma}

% proof
\begin{proof}
Exercise.
\end{proof}

% example
\begin{example}
Let $X\sim\text{Binomial}(n,\theta)$ where $n$ is known, but $0<\theta<1$ is unknown.
\ben
\it We conduct a sequence of $n$ independent trials and observe $k$ successes. Find a suitable prior distribution for $\theta$, compute the posterior distribution, and find its mean and mode.
\it We conduct a further sequence of $n$ independent trials, this time observing $k'$ successes. Compute an updated posterior distribution for $\theta$, and find its mean and mode.
\een
\end{example}

% solution
\begin{solution}
\ben
\it % <<< (i)
Let $f(x|\theta)$ be the PMF of the $\text{Binomial}(n,\theta)$ distribution:
\[
f(x|\theta) = \binom{n}{k}\theta^x(1-\theta)^{n-x}
\]

Initially we should consider every value of $\theta$ to be equally likely. Thus we adopt the uniform prior distribution for $\theta$.
\[
\pi_0(\theta) = \left\{\begin{array}{ll}
	1	& \text{if } \theta\in[0,1] \\
	0	& \text{otherwise.}
\end{array}\right.
\]
Given $k$ successes in $n$ trials, the likelihood function is
\[
L(\theta|k) = f(k|\theta) = \binom{n}{k}\theta^k(1-\theta)^{n-k}
\]

The posterior distribution combines the observation with the prior distribution:
\[
\pi_1(\theta) = \pi_1(\theta|k)
	= \frac{L(\theta|k)\pi_0(\theta)}{\int L(\theta|k)\pi_0(\theta)\,d\theta}
	= \frac{\theta^k(1-\theta)^{n-k}}{\int_0^1\theta^k(1-\theta)^{n-k}\,d\theta}.
\]

We recognise $\pi_1(\theta)$ as the PDF of the $\text{Beta}(\alpha,\beta)$ distribution, with parameters
\[
\alpha=k+1 \text{\quad and\quad}\beta=n-k+1.
\]

\bit
\it The mode of $\pi_1(\theta)$ is $k/n$. This is the MAP estimator of $\theta$.
\it Note that this co-incides with the MLE of $\theta$.
\eit
\bit
\it The expected value of $\pi_1(\theta)$ is $(k+1)/(n+2)$.
\it This is approximately equal to the MAP estimator when $k$ and $n$ are both large.
\eit

\it % <<< (ii)
Given $k'$ successes in $n$ trials, the likelihood function is
\[
L(\theta|k') = f(k'|\theta) = \binom{n}{k'}\theta^{k'}(1-\theta)^{n-k'}.
\]

Using $\pi_1$ as the new prior distribution for $\theta$, the new posterior distribution $\pi_2$ is
\[
\pi_2(\theta) =
\pi_2(\theta|k,k')
	= \frac{L(\theta|k')\pi_1(\theta)}{\int_0^1 L(\theta|k')\pi_1(\theta)\,d\theta}
	= \frac{\theta^{k+k'}(1-\theta)^{2n-(k+k')}}{\int_0^1\theta^{k+k'}(1-\theta)^{2n-(k+k')}\,d\theta}.
\]

We recognise $\pi_2(\theta)$ as the PDF of the $\text{Beta}(\alpha,\beta)$ distribution, with parameters
\[
\alpha=k+k'+1 \text{\quad and\quad}\beta=2n-(k+k')+1.
\]

Hence our updated MAP estimate of $\theta$ is
\[
\hat{\theta}_{MAP} = \frac{k+k'}{2n}.
\]

\bit
\it If $k'<k$, the mode shifts to the left (adjusted down).
\it If $k'>k$, the mode shifts to the right (adjusted up).
\eit
\een
\end{solution}

%----------------------------------------------------------------------
\section{The exponential model}
%----------------------------------------------------------------------
A suitable model for estimating the distribution of a non-negative parameter $\theta\geq 0$ is provided by the \emph{gamma distribution}.

% definition
\begin{definition}\label{def:gamma_distribution}
The \emph{gamma distribution} with parameter $\alpha,\beta>0$ is defined by the PDF
\[
f(x;\alpha,\beta) = \begin{cases}
	\displaystyle\frac{\beta^{\alpha}}{\Gamma(\alpha)}\, x^{\alpha-1} e^{-\beta x} & \text{for $x>0$}, \\
	0												& \text{otherwise.}
\end{cases}
\]
where $\Gamma(\alpha)$ is the so-called \emph{gamma function},
\[
\Gamma(\alpha) = \int_0^{\infty} t^{\alpha-1}e^{-t}\,dt,
\quad\text{which is defined for all $\alpha\in\R$.}
\]
\end{definition}

%% remark
%\begin{remark}[Special cases]
%\bit
%\it If $X\sim\text{Exponential}(\lambda)$, where $\lambda$ is a rate parameter, then $X\sim\text{Gamma}(1,\lambda)$.
%\it If $X\sim\text{Chi-squared}(k)$ then $X\sim\text{Gamma}(k/2,2)$
%\eit
%\end{remark}

% lemma
\begin{lemma}
Let $X\sim\text{Gamma}(\alpha,\beta)$. Then $\expe(X) = \displaystyle\frac{\alpha}{\beta}$ and $\mode(X) = \displaystyle\frac{\alpha-1}{\beta}$ provided that $\alpha > 1$.
%\quad\text{and}\quad
%(2)\quad \mode(X) = \frac{\alpha-1}{\beta}
%\text{ provided that $\alpha > 1$}.
%\]
\end{lemma}
% proof
\begin{proof}
Exercise.
\end{proof}

% example: exponential
\begin{example}
Let $X\sim\text{Exponential}(\lambda)$, where $\lambda>0$ is an unknown rate parameter. Let $X_1,X_2,\ldots,X_n$ be a random sample of observations from the distribution of $X$, and suppose that we adopt the $\text{Gamma}(\alpha,\beta)$ distribution as a prior distribution for $\lambda$, where $\alpha,\beta>0$ are fixed values (perhaps estimated in some preliminary experiments).

\ben
\it Find the mean and mode of the prior distribution.
\it Show that the posterior of $\lambda$ is the $\text{Gamma}(\alpha+n,\beta+\sum_{i=1}^n x_i)$ distribution.
\it Find the mean and mode of the posterior distribution.
\een
\end{example}

% solution
\begin{solution}
Let $f(x|\lambda)$ be the PDF of the $\text{Exponential}(\lambda)$ distribution:
\[
f(x|\lambda) = \left\{\begin{array}{ll}
	\lambda\exp(-\lambda x) & \text{for $x>0$}, \\
	0							& \text{otherwise.}
\end{array}\right.
\]

The PDF of the prior distribution is
\[
\pi_0(\lambda) = 	\frac{\beta^{\alpha}}{\Gamma(\alpha)}\,\lambda^{\alpha-1}\exp(-\beta\lambda).
\]
which has mean $\alpha/\beta$ and mode $(\alpha-1)/\beta$.

Let $\boldx$ be a realisation of the sample. The likelihood function is
\[
L(\lambda|\mathbf{x})
	= \prod_{i=1}^n f(x_i|\lambda)
	= \prod_{i=1}^n \lambda\exp(-\lambda x_i)
	= \lambda^n \exp\Big(-\lambda\textstyle\sum_{i=1}^n  x_i\Big).
\]

The PDF of the posterior distribution is
\[
\pi_1(\lambda|\mathbf{x})
	= \frac{L(\lambda|\mathbf{x})\pi_0(\lambda)}{\displaystyle\int_0^{\infty} L(\lambda|\mathbf{x})\pi_0(\lambda)\,d\lambda}.
\]

The numerator is the product of the likelihood $L(\lambda|\mathbf{x})$ and the prior $\pi_0(\lambda)$:
\begin{align*}
L(\lambda|\mathbf{x})\pi_0(\lambda)
	& = \Big[\lambda^n \exp\big(-\lambda\textstyle\sum_{i=1}^n  x_i\big)\Big]\left[\displaystyle\frac{\beta^{\alpha}}{\Gamma(\alpha)}\lambda^{\alpha-1}\exp(-\beta\lambda)\right] \\[1ex]
	& = \frac{\beta^{\alpha}}{\Gamma(\alpha)} \lambda^{\alpha+n-1}\exp\Big[-\big(\beta+\textstyle\sum_{i=1}^n x_i\big)\lambda\Big]
\end{align*}

%Notice that this resembles the PDF of the $\text{Gamma}\Big(\alpha+n,\beta+\sum_{i=1}^n x_i\Big)$ distribution.

The PDF of the posterior distribution becomes
\[
\pi_1(\lambda|\mathbf{x})
	= \frac{\lambda^{\alpha+n-1}\exp\Big[-\big(\beta+\sum_{i=1}^n x_i\big)\lambda\Big]}
		{\displaystyle\int_0^{\infty} \lambda^{\alpha+n-1}\exp\Big[-\big(\beta+\sum_{i=1}^n x_i\big)\lambda\Big]\,d\lambda}
\]

To compute the denominator, change the variable of integration to $t = -\big(\beta+\sum_{i=1}^n x_i\big)\lambda$. This yields
\[
\int_0^{\infty} \lambda^{\alpha+n-1}\exp\Big[-\big(\beta+\textstyle\sum_{i=1}^n x_i\big)\lambda\Big]\,d\lambda
	= \displaystyle\frac{\Gamma(\alpha+n)}{\big(\beta+\sum_{i=1}^n x_i\big)^{\alpha+n}}
\]

Thus the PDF of the posterior distribution is
\[
\pi_1(\lambda|\mathbf{x})
	= \frac{\big(\beta+\sum_{i=1}^n x_i\big)^{\alpha+n}}{\Gamma(\alpha+n)}
	\lambda^{\alpha+n-1}\exp\Big[-\big(\beta+\sum_{i=1}^n x_i\big)\lambda\Big]
\]
which is the PDF of the $\text{Gamma}\Big(\alpha+n,\beta+\sum_{i=1}^n x_i\Big)$ distribution.

 % <<

The mean and mode of $\lambda\sim\text{Gamma}\Big(\alpha+n,\beta+\sum_{i=1}^n x_i\Big)$ are
\[
\expe(\lambda) = \frac{\alpha+n}{\beta+\sum_{i=1}^n x_i}
\text{\quad and\quad}
\text{Mode}(\lambda) = \frac{\alpha+n-1}{\beta+\sum_{i=1}^n x_i}
\text{\quad respectively.}
\]
Hence the MAP estimator of $\lambda$ is
\[
\hat{\lambda}_{MAP} = \frac{\alpha+n-1}{\beta+\sum_{i=1}^n x_i}
\]

\bit
\it When $n=0$, this is simply the mode of the prior distribution, $\text{Gamma}(\alpha,\beta)$.
\it As $n$ increases, the influence of the prior distribution decreases.
\it If we write $\hat{\lambda}_{MAP}$ as
\[
\hat{\lambda}_{MAP}
	= \frac{1 + \left(\frac{\alpha-1}{n}\right)}{\frac{1}{n}\sum_{i=1}^n x_i + \left(\frac{\beta}{n}\right)}.
\]
we see that $\hat{\lambda}_{MAP}=\bar{X}^{-1}$ as $n\to\infty$.
\it This is the method-of-moments estimator (MME) of $\lambda$, which is based entirely on the data and takes no account of the prior distribution.
\eit
\end{solution}

%----------------------------------------
\begin{exercise}
\begin{questions}

\question
Let $X\sim\text{Geometric}(\theta)$ where $0<\theta<1$ is unknown.
\begin{parts}
\part % << (i)
A single experiment yields the observation $k$. Find a suitable prior distribution for $\theta$, compute the corresponding posterior distribution, and find the MAP estimator of $\theta$ for this posterior.
\begin{answer}
Let $f(x|\theta)$ be the PMF of the $\text{Geometric}(\theta)$ distribution:
\[
f(x|\theta) = \theta^x(1-\theta)^{n-x}
\]
Without any information about $\theta$, we should choose the \emph{na\"{\i}ve} prior:
\[
\pi_0(\theta) = \begin{cases}
	1	& \text{if $0\leq\theta\leq 1$,} \\
	0	& \text{otherwise.}
\end{cases}
\]
For the observation $X=k$, the likelihood function is
\[
L(\theta|k) = f(k|\theta) = \theta(1-\theta)^{k-1}
\]
The posterior distribution is:
\[
\pi_1(\theta|k)
	= \frac{L(\theta|k)\pi_0(\theta)}{\int L(\theta|k)\pi_0(\theta)\,d\theta}
	= \frac{\theta(1-\theta)^{k-1}}{\int_0^1\theta(1-\theta)^{k-1}\,d\theta}.
\]
We recognise this as the PDF of the $\text{Beta}(\alpha,\beta)$ distribution, with parameters $\alpha=2$ and $\beta=k$. The mode of the $\text{Beta}(\alpha,\beta)$ distribution is $(\alpha-1)/(\alpha+\beta-2)$, so the MAP estimator is
\[
\hat{\theta}_{MAP} = \frac{1}{k}.
\]
\end{answer}
\part % << (ii)
A second experiment yields the observation $k'$. Compute an updated posterior distribution for $\theta$, and find a new MAP estimator for $\theta$.
\begin{answer}
For the observation $X=k'$, the likelihood function is
\[
L(\theta|k') = f(k'|\theta) = \theta(1-\theta)^{k'-1}
\]
Using $\pi_1$ as the new prior distribution for $\theta$, the new posterior distribution $\pi_2$ is
\[
\pi_2(\theta|k,k')
	= \frac{L(\theta|k')\pi_1(\theta)}{\int_0^1 L(\theta|k')\pi_1(\theta)\,d\theta}
	= \frac{\theta^2(1-\theta)^{k+k'-2}}{\int_0^1\theta^2(1-\theta)^{k+k'-2}\,d\theta}.
\]
which we recognise as the PDF of the $\text{Beta}(\alpha,\beta)$ distribution, with parameters $\alpha=3$ and $\beta=k+k'-1$. Hence the new MAP estimator is
\[
\hat{\theta}_{MAP} = \frac{2}{k+k'}.
\]
\end{answer}
\end{parts}

\question
Let $X\sim\text{Poisson}(\lambda)$, where $\lambda>0$ is unknown, and let $X_1,X_2,\ldots,X_n$ be a random sample of observations from the distribution of $X$. Suppose we adopt the $\text{Gamma}(\alpha,\beta)$ distribution as a prior distribution for $\lambda$, where $\alpha,\beta>0$ are fixed values.
\begin{parts}
\part % << (i)
Show that the MAP estimator of $\lambda$ is given by
\[
\hat{\lambda}_{MAP} = \frac{\alpha-1+\sum_{i=1}^n X_i}{n+\beta}
\]
\begin{answer}
Let $f(x|\lambda)$ be the PMF of the $\text{Poisson}(\lambda)$ distribution:
\[
f(x|\lambda) = \begin{cases}
	\frac{\lambda^{x}e^{-\lambda}}{x!} & \text{for $x=0,1,2,\ldots$}, \\
	0							& \text{otherwise.}
\end{cases}
\]
The PDF of the prior distribution is
\[
\pi_0(\lambda) = \frac{\beta^{\alpha}}{\Gamma(\alpha)}\,\lambda^{\alpha-1}\exp(-\beta\lambda).
\]
which has mean $\alpha/\beta$ and mode $(\alpha-1)/\beta$.

Let $\boldx=(x_1,x_2,\ldots,x_n)$ be a realisation of the sample. The likelihood function is
\[
L(\lambda|\mathbf{x})
	= \prod_{i=1}^n f(x_i|\lambda)
	= \prod_{i=1}^n \frac{\lambda^{x_i}e^{-\lambda}}{x_i!}
	= \lambda^{\sum x_i} e^{-n} \prod_{i=1}^n \frac{1}{x_i!}
\]
The PDF of the posterior distribution is
\[
\pi_1(\lambda|\mathbf{x})
	= \frac{L(\lambda|\mathbf{x})\pi_0(\lambda)}{\int_0^{\infty} L(\lambda|\mathbf{x})\pi_0(\lambda)\,d\lambda}.
\]
To find the MAP estimator, we need to find the value of $\lambda$ that maximises the numerator:
\[
L(\lambda|\mathbf{x})\pi_0(\lambda)
	= c\lambda^{(\alpha-1+\sum x_i)} e^{-(n+\beta)\lambda}
	\quad\text{where}\quad
	c = \left(\prod_{i=1}^n \frac{1}{x_i!}\right) \frac{\beta^\alpha}{\Gamma(\alpha)}
\]
Let $g(\lambda)=\lambda^{\alpha-1+\sum x_i} e^{-(n+\beta)\lambda}$. Then
\[
g'(\lambda) = \lambda^{(\alpha-2+\sum x_i)} e^{-(n+\beta)\lambda}\left[(\alpha-1+\sum _{i=1}^n x_i) - \lambda(n+\beta)\right]
\]
Setting $g'(\lambda)$ to zero and solving for $\lambda$, we obtain the MAP estimator
\[
\hat{\lambda}_{MAP} = \frac{\alpha-1+\sum_{i=1}^n X_i}{n+\beta}
\]
as required.
\end{answer}
\part % << (ii)
Comment on the limiting cases (i) $n=0$ and (ii) $n\to\infty$.
\begin{answer}
\bit
\it When $n=0$, $\hat{\lambda}_{MAP}=(\alpha-1)/\beta$ is the mode of the prior distribution.
\it When $n\to\infty$,
\[
\hat{\lambda}_{MAP}
	= \frac{\frac{\alpha-1}{n}+\frac{1}{n}\sum_{i=1}^n X_i}{1+\frac{\beta}{n}}
	\to \frac{1}{n}\sum_{i=1}^n X_i
\]
As the sample size increases, the effect of the prior decreases, and the MAP estimator approaches the sample mean in the limit as $n\to\infty$.
\eit
\end{answer}
\end{parts}

\question
Let $X_1,\ldots,X_n$ be a random sample from the $N(\mu,\sigma^2)$ distribution, where the mean $\mu$ is unknown but the variance $\sigma^2$ is known. Suppose we adopt the $N(\mu_0,\sigma_0^2)$ distribution as a prior for the unknown mean $\mu$ (where $\mu_0$ and $\sigma_0^2$ are known constants). Compute the maximum a-posteriori (MAP) estimator of $\mu$.

\begin{answer}
Let $\pi_0(\mu)$ denote the prior density function of $\mu$:
\[
\pi_0(\mu) = \frac{1}{\sigma_0\sqrt{2\pi}}\exp\left[-\frac{1}{2}\left(\frac{\mu-\mu_0}{\sigma_0}\right)^2\right].
\]

For the observed sequence $\mathbf{x}=(x_1,x_2,\ldots,x_n)$, the likelihood function is
\begin{align*}
L(\mu\,|\,\mathbf{x})
	& = \prod_{i=1}^n \left(\frac{1}{\sigma\sqrt{2\pi}}\right)\exp\left[-\frac{1}{2}\left(\frac{x_i-\mu}{\sigma}\right)^2\right] \\
	& = \left(\frac{1}{\sigma\sqrt{2\pi}}\right)^{n/2}\exp\left[-\frac{1}{2}\sum_{i=1}^n\left(\frac{x_i-\mu}{\sigma}\right)^2\right].
\end{align*}

The posterior density function of $\mu$ combines the data and the prior:
\begin{align*}
\pi_1(\mu)
	& = \frac{L(\mu\,|\,\mathbf{x})\pi_0(\mu)}{\int L(\mu|\mathbf{x})\pi_0(\mu)\,d\mu}.
\end{align*}

The MAP estimator of $\mu$ is the value that maximises the posterior $\pi_1(\mu)$. Since the denominator in the above expression for $\pi_1$ is a constant, it is sufficient to find the value of $\mu$ that maximises the numerator,
\begin{align*}
L(\mu|\mathbf{x})\pi_0(\mu)
	= \left(\frac{1}{\sigma_0\sqrt{2\pi}}\right)\left(\frac{1}{\sigma\sqrt{2\pi}}\right)^{n/2}
			\exp\left[-\frac{1}{2}\sum_{i=1}^n\left(\frac{x_i-\mu}{\sigma}\right)^2 -\frac{1}{2}\left(\frac{\mu-\mu_0}{\sigma_0}\right)^2\right].
\end{align*}
Let
\[
g(\mu) = \exp\left[-\frac{1}{2}\sum_{i=1}^n\left(\frac{x_i-\mu}{\sigma}\right)^2 -\frac{1}{2}\left(\frac{\mu-\mu_0}{\sigma_0}\right)^2\right].
\]
The value of $\mu$ that maximizes $L(\mu|\mathbf{x})\pi_0(\mu)$ also maximizes $g(\mu)$. The first derivative of $g$ with respect to $\mu$ is
\[
g'(\mu) = \left[\frac{1}{\sigma^2}\sum_{i=1}^n (x_i-\mu) - \frac{1}{\sigma_0^2} (\mu-\mu_0)\right]g(\mu).
\]
Setting this equal to zero,
\[
\frac{1}{\sigma^2}\sum_{i=1}^n (x_i-\mu) = \frac{1}{\sigma_0^2}(\mu-\mu_0),
\]
and solving for $\mu$, we obtain the MAP estimator,
\[
\hat{\mu}_{MAP} = \left(\frac{\sigma^2\sigma_0^2}{\sigma^2+n\sigma_0^2}\right)\left(\frac{1}{\sigma^2}\sum_{i=1}^n x_i + \frac{\mu_0}{\sigma_0^2}\right).
\]
\bit
\it This expression can be rearranged to give
\[
\hat{\mu}_{MAP}\left(1 + \frac{n\sigma_0^2}{\sigma^2}\right) = \frac{\sigma_0^2}{\sigma^2}\sum_{i=1}^n x_i + \mu_0.
\]
This shows that $\hat{\mu}=\mu_0$ when $n=0$, so the $\hat{\mu}_{MAP}$ is equal to the mean of the prior distribution when there is no data.
\it The expression can also be rearranged to give
\[
\hat{\mu}_{MAP}\left(1 + \frac{\sigma^2}{n\sigma_0^2}\right) = \frac{1}{n}\sum_{i=1}^n x_i + \left(\frac{\sigma^2}{n\sigma_0^2}\right)\mu_0.
\]
This shows that $\hat{\mu}_{MAP} \to \bar{X}$ as $n\to\infty$ (which is independent of the prior).
\eit
\end{answer}

%----------------------------------------
\end{questions}
\end{exercise}
%----------------------------------------------------------------------