\section{Generative models}
In unsupervised learning we want to find generative models which allow us to generate new data
from previously made observations. We sometimes assume that observations are generated from a
process which is not immediately observable from the data at hand. A general form of such a
generative model can be written as:
\begin{align}
	\mb{x} = f(\mb{s})
	\label{eq:GM}
\end{align}
where $\mb{x}$ is a $D \times 1$ vector for a given observation, which is assumed to be
generated by an arbitrary function $f(\mb{s})$. $\mb{s}$ is a $M \times 1$ vector of
source signals for the given observation. If $\mb{s}$ is known the problem of inferring
function $f(\mb{s})$ becomes a supervised learning task. However, if the sources are
unknown the problem becomes an unsupervised learning task. UNder certain assumptions
it possible to infer sources from observations, this is called blind source separation.

\subsection{The linear generative model (LGM)}
In the following we will only consider linear generative models for which equation \eqref{eq:GM}
takes the form:
\begin{align}
	\mb{x} = \mb{A s}
	\label{eq:LGM}
\end{align}
where $\mb{x}$ is an $D \times 1$ observation vector, $\mb{s}$ is the $M \times 1$ vector
of sources and $\mb{A}$ is a $D \times M$ transformation matrix.
We distinguish three classes of LGM dependent on the dimensionality of $\mb{A}$:

\begin{enumerate}
	\item Overcomplete LGM: \textbf{A} is a fat matrix with $D<M$. \\
		  The number of source signals is assumed to be bigger than the dimensionality
		  of our observations. The generative model is overcomplete since blow up the number
		  of possible sources.
	\item Undercomplete LGM: \textbf{A} is a flat matrix with $D>M$. \\
	      The undercomplete case is the inverse case an overcomplete LGM where the observations
	      are assumed to be generated from less sources that the dimensionality of our observations.
	\item Complete LGM: \textbf{A} is a square matrix with $D=M$, full rank
	      and $\forall m = 1,\dots,M: \, \mathrm{Var}[\mb{s_m}] > 0.$ \\
	      In the complete case the number of source signals is assumed to equal the dimensionality
	      of our observations. In the following we will focus on complete LGMs.
\end{enumerate}

% Table of UL algorithms used for different types of LGM
\begin{figure}[h]
\centering
\begin{tabular}{|l|l|}
\hline
Generative Model & Unsupervised Learning algorithm \\
\hline
Complete LGM & Principle Component Analysis (PCA), \\
             & Independent Components Analysis (ICA) \\
%\hline
Overcomplete LGM & Factor Analysis, Independent Factor Analysis \\
\hline
\end{tabular}
\caption{Some examples of unsupervised leanring algorithms for the different classes of LGM.
For all those algorithms it is assumed that observations \textbf{X} can be decomposed into
sources $\mb{S}$ and transformation matrix $\mb{A}$.}
\end{figure}

\subsection{The complete LGM}
In this section we assume that our observations and signals have the same dimensionality,
i.e. we are dealing with a complete LGM. We further assume that matrix $\mb{A}$ is orthogonal
which means that:
\begin{align}
	 \mb{A}\TT \mb{A} = \mb{A} \mb{A}\TT = \imat_D
\end{align}
\noindent
We will further focus on normally distributed source signals as this will be relevant for the
orthogonality assumption with the application of principle component analysis. In fact orthogonality
of $\mb{A}$ is only guaranteed if our observations are normally distributed. We can show that given
the sources are normally distributed, our observations will be too:

%% From lecture
%\textbf{Linear mappings of Gaussian RVs:} \\
%$s \sim \	{N}(\mathbf{\mu_s},\mathbf{C_s})$
%\begin{align*}
%	x_k &= a_k s_1 + b_k s_2 \\
%	s_j &\sim \mathcal{N}(0,\sigma_j^2); \qquad \rho(s_1,s_2) = \rho(s_1)\rho(s_2) \\
%	\Rightarrow x_n &\sim \mathcal{N}(0,a_k^2 \sigma_1^2 + b_k^2 \sigma_2^2)
%\end{align*}

\begin{proposition}[Linear mappings of Gaussian random variables]
Given the sources are independent and normally distributed such that
\begin{align*}
	\mb{s} \sim \mathcal{N}(\greekvec{\mu}_s,\mb{C_s}); \qquad
	\rho(\mb{s}) = \prod_{m=1}^M \rho(s_m); \qquad
	s_m \sim \mathcal{N}(0,\sigma_m^2) &
\end{align*}
and the observations are linear mappings of the sources such that
\begin{align*}
	x_d = \sum_{m=1}^M \mb{A}_{dm} s_m
\end{align*}
The observations will be normally distributed according to
\begin{align*}
	x_d \sim \mathcal{N}(0, \sum_{m=1}^M \mb{A}_{dm}^2 \sigma_m^2)
\end{align*}
\end{proposition}

\begin{proof}[Sketch of proof.]
	\begin{align*}
	(i) &\qquad (\mb{A}_{dm} s_m) \sim \mathcal{N}(0,\mb{A}_{dm}^2 \sigma_m^2) \\
	(ii) &\qquad \text{generally for } \\
		 &\qquad z = u + v : \rho(z) = \underbrace{\rho(u) \Asterisk \rho(v)}_{\text{Convolution}}\\
	     &\qquad \Rightarrow \underbrace{\mathcal{F}[\rho(z)]}_{\text{Fourier transform}}
	       = \underbrace{\underbrace{\mathcal{F}[\rho(u)]}_{Gaussian} \underbrace{\mathcal{F}[\rho(v)]}_{Gaussian} }_{Gaussian} \\
	     &\qquad \Rightarrow \rho(z) = \underbrace{\mathcal{F}^{-1}(\cdots)}_{Gaussian}
\end{align*}
\end{proof}
\noindent
The density of the sum of two random variables is obtained by convolving their individual densities.
The convolution of two Gaussians is a multiplication of two Gaussians in the Fourier space. Backtransformation again results
in a Gaussian. Below we will see how the covariance $\mb{C}_x$ of observations $\mb{x}$ can be decomposed given an LGM.

%Sketch of proof:
%\begin{align*}
%	(i) &\qquad (a_k s_1) \sim \mathcal{N}(0,a_k^2 \sigma_1^2) \\
%	    &\qquad (b_k s_2) \sim \mathcal{N}(0,b_k^2 \sigma_2^2)	\\
%	(ii) &\qquad \text{generally for } \\
%		 &\qquad z = u + v : f(z) = f(u) + f(v)\\
%	     &\qquad \Rightarrow \underbrace{\mathcal{F}[f(z)]}_{\text{Fourier transform}}
%	       = \underbrace{\underbrace{\mathcal{F}[f(u)]}_{Gaussian} \underbrace{\mathcal{F}[f(v)]}_{Gaussian} }_{Gaussian} \\
%	     &\qquad \Rightarrow f(z) = \underbrace{\mathcal{F}^{-1}(\cdots)}_{Gaussian}
%\end{align*}
%Aside: The Fourier transform of a density $\hat{=}$ the characteristic function $\E_{\rho(z)}[\E^{ikz}]$

\begin{theorem}[Linear transform theorem]
Given $\mb{x} = \mb{A s}$ is a linear mapping and $\mb{s}$ and $\mb{x}$ are normally distributed such that
\begin{align*}
	\mb{s} \sim \mathcal{N}(\greekvec{\mu}_s,\mb{C}_s) \qquad
	\text{ and } \qquad
	\mb{x} \sim \mathcal{N}(\greekvec{\mu}_x,\mb{C}_x) \qquad
\end{align*}
then the expectation and covariance of $\mb{x}$ are given by
\begin{align*}
	(i)& \qquad \greekvec{\mu}_x = \mb{A} \greekvec{\mu}_s \\
	(ii)&  \qquad \mathbf{C}_x = \mathbf{A} \mathbf{C}_s \mathbf{A}\TT %
\end{align*}
where we refer to $(ii)$ as the "Sandwich theorem".
\end{theorem}

\begin{proof}[Derivation]
\qquad \\
\begin{minipage}{0.45\textwidth}
\begin{align*}
	\greekvec{\mu}_x &= \Ex{\rho(\mb{x})}{\mb{x}} \\
	                 &= \Ex{\rho(\mb{x})}{\mb{A s}} \\
	                 &= \mb{A} \Ex{\rho(\mb{s})}{\mb{s}} \\
	                 &= \mathbf{A} \greekvec{\mu}_s \\
\end{align*}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\begin{align*}
	\mb{C}_x &= \Ex{}{\mb{x x}\TT} - \Ex{}{\mb{x}} \Ex{}{\mb{x}}\TT \\
		     &= \Ex{}{\mathbf{A s s }\TT \mb{A}\TT}
				    - \Ex{}{\mb{A s}} \Ex{}{\mb{s}\TT \mathbf{A}\TT} \\
				 &= \mb{A} \Ex{}{\mathbf{s s}\TT} \mb{A}\TT
				    - \mb{A} \Ex{}{\mathbf{s}} \Ex{}{\mb{s}}\TT \mb{A}\TT \\
				 &= \mathbf{A} \left(\Ex{}{\mathbf{s s}\TT} - \Ex{}{\mathbf{s}} \Ex{}{\mb{s}}\TT \right) \mathbf{A}\TT \\
				 &= \mathbf{A} \mathbf{C}_s \mathbf{A}\TT \\
\end{align*}
\end{minipage}
\newline
\end{proof}

%\begin{align*}
%	\mathbf{u} &= \mathbf{W v} \\
%	\rho(\mathbf{v}) &= \mathcal{N}(\mathbf{v}|\greekvec{\mu_v},\mathbf{C_v})\\
%	\rho(\mathbf{u}) &= \mathcal{N}(\mathbf{u}|\greekvec{\mu_u},\mathbf{C_u})\\
%	\greekvec{\mu_u} &= \Ex{\rho(\mathbf{u})}{\mathbf{u}}
%	                  = \Ex{\rho(\mathbf{u})}{\mathbf{W v}}
%	                  = \mathbf{W} \E_{f(\mathbf{u})}[\mathbf{v}] = \mathbf{W} \greekvec{\mu_v} \\
%	\mathbf{C_u} &= \E[\mathbf{u u}\TT] - \E[\mathbf{u}] \E[\mathbf{u}]\TT \\
%				 &= \E[\mathbf{W v v}\TT \mathbf{W}\TT]
%				    - \E[\mathbf{W v}] \E[\mathbf{v}\TT\mathbf{W}\TT] \\
%				 &= \mathbf{W} \E[\mathbf{v v}\TT] \mathbf{W}\TT
%				    - \mathbf{W} \E[\mathbf{v}] \E[\mathbf{v}]\TT \mathbf{W}\TT \\
%\end{align*}

%\begin{proposition}[Rules for expectations of linear mappings]
%	\begin{align*}
%		(i)& \qquad \Ex{}{\mathbf{W v}} = \mathbf{W} \Ex{}{\mathbf{v}} \\
%		(ii)&  \qquad \Ex{}{\mathbf{A X B}} = \mathbf{A} \Ex{}{\mathbf{X}} \mathbf{B} %
%	\end{align*}
%\end{proposition}
%
%\begin{theorem}[Linear transform theorem]
%	\begin{align*}
%		\mathbf{u}  = \mathbf{W v}
%	\end{align*}
%
%	\begin{align*}
%		(i)& \qquad \greekvec{\mu_u} = \mathbf{W} \greekvec{\mu_v} \\
%		(ii)&  \qquad \mathbf{C_u} = \mathbf{W} \mathbf{C_v} \mathbf{W}\TT %
%	\end{align*}
%	We call this the "Sandwich theorem"
%\end{theorem}

\noindent We can now apply the sandwich theorem to calculate the covariance of a normally distributed random
variable with independent normally distributed sources as sketched above.
\begin{proposition}[Covariance matrix for observations from independent normally distributed sources]
Given
	\begin{align*}
		\mb{s} \sim \mathcal{N}(\greekvec{\mu}_s,\mb{C_s}); \qquad
		\rho(\mb{s}) = \prod_{m=1}^M \rho(s_m); \qquad
		\mb{C}_s = \mathrm{diag}(\sigma_1^2, \dots ,\sigma_M^2) &
	\end{align*}
then
	\begin{align}
		\begin{split}
		\mathbf{C}_x &= \mb{A} \mb{C}_s \mb{A}\TT \\
		             &= \sum_{m=1}^M \sigma_m^2 \underbrace{\mathbf{a_m a_m\TT}}_{\text{rank-1-matrix outer product}}
		             \label{eq:Cx}
		\end{split}
	\end{align}
where $\mathbf{a_m}$ is a $D \times 1$ basis vector of matrix $\mb{A}$.
\end{proposition}

\noindent For visualization of the source covariance before transformation and the covariance of the observations
after applying $\mb{A}$ to the sources we introduce another transformation which yields a one-dimensional
marginal distribution.

\begin{proposition}[Variance contour lines]
Take an arbitrary $D \times 1$ vector $\mb{w}$ with $\|\mathbf{w}\| = 1$ such that
	\begin{align}
		&u = \mathbf{w\TT x}
	\end{align}
\noindent yields a one-dimensional marginal distribution. For a covariance of $\mb{x}$ as shown in equation
\eqref{eq:Cx} the variance of $u$ is given by
\begin{align}
\begin{split}
	\mathrm{Var}[u] &= \mathbf{w C_x w} \\
	                          &= \mathbf{w}\TT \left(\sum_{m=1}^M \sigma_m^2 \mathbf{a_m a_m\TT}\right) \mathbf{w} \\
	                          &= \sum_{m=1}^M \sigma_m^2 (\mathbf{w\TT a_m}) (\mathbf{a_m\TT w}) \\
	                          &= \sum_{m=1}^M \sigma_m^2 (\mathbf{w\TT a_m})^2 \label{eq:Cu}
\end{split}
\end{align}
We notice that the marginalization introduces a projection of the basis vectors of $\mb{A}$ onto the axis that is
spanned by $\mb{w}$ where the variance $\mathrm{Var}[u]$ is the variance along $\mb{w}$. We can now calculate
the variance along any direction in the sample space and visualize it using variance contour lines.
\end{proposition}

\subsubsection{Example for three LGM with different source distributions}
To inspect the influence of the linear transformation on the covariance of the sources we utilize the above
marginalization to calculate variance contour lines. We will distinguish three cases of source distributions,
two of which are Gaussians and one is a uniform distribution. For visualization purposes we will focus on 2-dimensional
examples although the approach can be applied to cases with higher dimensions.

\paragraph{Special case: Isovariance with $\sigma_1^2 = \dots = \sigma_M^2 = \sigma^2$}
\label{par:cov_isovar}
\qquad \newline
Given
	\begin{align*}
		\mb{s} \sim \mathcal{N}(\greekvec{\mu}_s,\mb{C_s}); \qquad
		\mathbf{C_s} &= \sigma^2 \imat_M &
	\end{align*}

\begin{minipage}{0.45\textwidth}
\begin{align*}
	\mathbf{C_x} &= \mathbf{A C_s A\TT} \\
	             &= \sigma^2 \mathbf{A} \imat_M \mathbf{A}\TT \\
	             &= \sigma^2 \underbrace{\mathbf{A}\mathbf{A}\TT}_{\mathbf{A}\text{ is orthogonal}} \\
	             &= \sigma^2 \imat_M = \mathbf{C_s} \\
\end{align*}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\begin{align*}
	\mathrm{Var}[\mb{w\TT x}] &= \mathbf{w\TT C_x w} \\
				    &= \mathbf{w\TT C_s w} \\
	                &= \sigma^2 \mathbf{w}\TT \imat_M \mathbf{w} \\
	                &= \sigma^2 \mathbf{w}\TT \mathbf{w} \\
	                &= \sigma^2 \|\mathbf{w}\|^2 = \sigma^2 \\
\end{align*}
\end{minipage}
\newline
We see that given the source distribution has isovariance and the transformation matrix $\mb{A}$ is orthogonal
the corvariance of observations will have the same covariance as the source distribution. In turn the variance
of the introduced marginalization will be constant for any $\mb{w}$ with $\|\mathbf{w}\|^2 = 1$.
\figref{fig:LGM_isovar} illustrates the variance contour lines for the three cases.
With the next two cases we will focus on visual inspection of the contour lines without making any further
derivations.

\begin{figure}
	\includegraphics[width=\textwidth]{./lecture10/lineartransform_geometricmeaning.pdf}
	\caption{Variance contour lines before and after applying $\mb{A}$.}
	\label{fig:LGM_isovar}
\end{figure}

\paragraph{Non-degenerate case: $\sigma_1^2 > \sigma_1^2 > \dots > \sigma_M^2$}
Given the source distribution is a Gaussian with independent sources and unequal source variances.
We see that applying matrix $\mb{A}$ can introduce a rotation of the variance contour line and in turn
introduces correlation for the dimensions of $\mb{x}$. That is, although the source were perfectly
independent the matrix $\mb{A}$ introduces correlations. It becomes apparent that if we find a matrix
$\mb{A}^{-1}$ which can be applied to the data, we can remove those correlations.

\paragraph{Non-Gaussian case: $\sigma_1^2 > \sigma_1^2 > \dots > \sigma_M^2$}
In this case we assume a uniform source distribution with unequal source variances. Although the
density contour line differs from the variance contour line under the Gaussian assumption we see
that applying $\mb{A}$ introduces the same correlations in the observations as in the Gaussian
case.

\subsection{Blind source separation}
If we make observations which are correlated and we assume that the sources are uncorrelated we
can find a tranform $\mb{A}\TT$ which removes correlations and recovers the uncorrelated signals.
We call this approach whitening. But how can we uniquely identify $\mb{A}$?

\begin{proposition}[Sufficient condition]
	If we assume that $\sigma_1^2 > \sigma_2^2 > \dots > \sigma_k^2$ and if we assume an orthogonal LGM
	then we can uniquely identify $\mathbf{A}$ and the solution is given by the Eigenvectors of the convariance
	matrix $\mathbf{C_x}$ of the data. This is what we call principle component analysis.
\end{proposition}


\section{Cayley-Hamilton Theorem and Multiplicity of Eigenvalues}
\begin{theorem}[Cayley-Hamilton Theorem]
    Let $V$ be a finite dimensional vector space over $F$ and $\alpha\in L(V)$ with characteristic polynomial $\chi_\alpha(t)=\det(\alpha-t\operatorname{id})$, then $\chi_\alpha(\alpha)=0$.
\end{theorem}
Consequently, $m_\alpha|\chi_\alpha$.
\begin{proof}[Proof for $F=\mathbb C$]
    We know that $\alpha$ is triangulable, so there is a basis $B$ such that it has matrix
    $$[\alpha]_B=\begin{pmatrix}
        a_1&&\ast\\
        &\ddots&\\
        0&&a_n
    \end{pmatrix}$$
    Therefore $\chi_\alpha(t)=(t-a_1)\cdots (t-a_n)$ (up to sign).
    But then easily $\chi_\alpha(\alpha)=\chi_\alpha([\alpha]_B)=0$.
\end{proof}
\begin{proof}[Proof for the General Case]
    For $A\in M_n(F)$, we write
    $$(-1)^n\chi_A(t)=\det(t\operatorname{id}-A)=t^n+a_{n-1}t^{n-1}\cdots+a_0$$
    for some $a_i\in F$.
    Now if $\operatorname{adj}(t\operatorname{id}-A)=B_{n-1}t^{n-1}+\cdots B_0$ for matrices $B_i$, then
    $$(t\operatorname{id}-A)(B_{n-1}t^{n-1}+\cdots B_0)=(t^n+a_{n-1}t^{n-1}\cdots+a_0)\operatorname{id}$$
    Equating the coefficients gives
    $$\operatorname{id}=B_{n-1},a_{n-1}\operatorname{id}=B_{n-2}-AB_{n-1},\ldots,a_0\operatorname{id}=-AB_n$$
    Therefore
    \begin{align*}
        (-1)^n\chi_A(A)&=A^n+a_{n-1}A^{n-1}\cdots+a_0\operatorname{id}\\
        &=A^nB_{n-1}+A^{n-1}(B_{n-2}-AB_{n-1})+\cdots+A^0(-AB_0)\\
        &=0
    \end{align*}
    by telescoping.
\end{proof}
\begin{definition}
    Let $\alpha\in L(V)$ and $\lambda$ an eigenvalue of $\alpha$, then the algebraic multiplicity $a_\lambda$ of $\lambda$ is the multiplicity of $\lambda$ as a root of $\chi_\alpha(t)$.\\
    The geometric multiplicity $g_\lambda$ of $\lambda$ is $\dim\ker(\alpha-\lambda\operatorname{id})$.
\end{definition}
\begin{remark}
    Obviously $a_\lambda,g_\lambda\ge 1$.
\end{remark}
\begin{lemma}\label{alg_geom_mult_ineq}
    $g_\lambda\le a_\lambda$.
\end{lemma}
\begin{proof}
    Let $\{v_1,\ldots,v_{g_\lambda}\}$ be a basis of $V_\lambda=\ker(\alpha-\lambda\operatorname{id})$ and extend it to a basis $B=\{v_i\}$ of $V$.
    Then
    $$[\alpha]_B=\begin{pmatrix}
        \lambda\operatorname{id}_{g_\lambda}&\ast\\
        0& A_1
    \end{pmatrix}$$
    for some $A_1$.
    Then
    $$\det(\alpha-\lambda\operatorname{id})=\det\begin{pmatrix}
        (\lambda-t)\operatorname{id}_{g_\lambda}&\ast\\
        0&A_1-t\operatorname{id}
    \end{pmatrix}=(\lambda-t)^{g_\lambda}\chi_{A_1}(t)$$
    which implies the claim.
\end{proof}
\begin{lemma}
    Let $\lambda$ be an eigenvalue of $\alpha$ and write $c_\lambda$ as the multiplicity of $\lambda$ as a root of the minimal polynomial $m_\alpha$.
    Then $1\le c_\lambda\le a_\lambda$.
\end{lemma}
\begin{proof}
    $c_\lambda\le a_\lambda$ is obvious as $m_\alpha|\chi_\alpha$.
    To see $c_\lambda\ge 1$, as $\lambda$ is an eigenvalue, we can find $v\neq 0$ such that $\alpha(v)=\lambda v$, so $\alpha^p(v)=\lambda^pv$.
    Hence $0=m_\alpha(\alpha)v=(m_\alpha(\lambda))v$ which means $m_\alpha(\lambda)=0$, hence $c_\lambda\ge 1$.
\end{proof}
\begin{example}
    1. Consider
    $$A=\begin{pmatrix}
        1&0&-2\\
        0&1&1\\
        0&0&2
    \end{pmatrix}$$
    then $\chi_A(t)=(t-1)^2(t-2)$.
    So $m_A(t)$ is either $(t-1)^2(t-2)$ or $(t-1)(t-2)$ by the preceding lemma.
    Indeed the latter works and has a smaller degree, hence $m_A(t)=(t-1)(t-2)$.\\
    2. Let $A$ be the Jordan block
    $$A=\begin{pmatrix}
        \lambda&1&&\\
        &\ddots&\ddots&\\
        &&\lambda&1\\
        &&&\lambda
    \end{pmatrix}$$
    Then as one can check, $g_\lambda=1,a_\lambda=c_\lambda=n$.\\
    3. Tak $A=\lambda\operatorname{id}$, then $g_\lambda=a_\lambda=n$ and $c_\lambda=1$.
\end{example}
\begin{lemma}
    Take $F=\mathbb C$, $V$ a finite dimensional vector space over $F$ and $\alpha\in L(V)$, then the followings are equivalent:\\
    (i) $\alpha$ is diagonalisable.\\
    (ii) For any eigenvalue $\lambda$ of $\alpha$ we have $a_\lambda=g_\lambda$.\\
    (iii) For any eigenvalue $\lambda$ of $\alpha$ we have $c_\lambda=1$.
\end{lemma}
\begin{proof}
    We already know that (i) is equivalent to (iii) by Theorem \ref{distinct_linear_diag}.
    To see (i) is equivalent to (ii), let $\lambda_1,\ldots,\lambda_k$ be distinct eigenvalues of $\alpha$.
    We have already seen that $\alpha$ is diagonalisable iff $V=\bigoplus_iV_{\lambda_i}$.
    But $\dim V=n=\deg\chi_\alpha=\sum_ia_{\lambda_i}$ by FTA and $\dim\bigoplus_iV_{\lambda_i}=\sum_ig_\lambda$.
    So $\alpha$ is diagonalisable iff $\sum_ig_{\lambda_i}=\sum_ia_{\lambda_i}$ iff $a_{\lambda_i}=g_{\lambda_i}$ for all $i$ by Lemma \ref{alg_geom_mult_ineq}.
\end{proof}


\section{Conclusion}

In this assignment we implemented and trained a Rosenblatt perceptron on a randomly generated datasets.
In our experiments, we estimated the probability of success of the perceptron to find a linear separation in a set of random datapoints.
For a limited number of training iterations $n_{max}$ and features $N$, we obtained that the capacity of the Rosenblatt perceptron is about $\alpha = 1.75$.

The difference from the theoretical results seems to be mainly caused by the limited number of training iterations:
by allowing longer trainings, i.e. greater number of epochs, the plot of the success rate $Q_{l.s.}$ as a function of $alpha$ moves to the right and the capacity tends to the theoretical result $\alpha = 2$.

The parameter $N$ influences the shape of the function:
for small values of $N$, the function is smooth in the regions before and after the step;
for higher values of $N$, the function gets closer to the theoretical step function.

The parameter $c$ in the training algorithm does not influence the performances of the perceptron:
indeed, it only changes the magnitude of the weights $w$ learned during the training.
However, since each training step in the algorithm can change the weights by at most $\frac{1}{N}\xi^{\mu(t)} S^{\mu(t)}$ (where $\mu(t)$ denotes the current training example), the algorithm will take a longer time to converge for higher values of $c$.

Rosenblatt perceptrons only learns separation hyperplanes that goes through the origin.
In general, it may happen that the hyperplane that linearly separates the dataset does not pass through the origin.
It is possible to generalize the Rosenblatt perceptron to learn also inhomogeneous separation hyperplanes by adding an artificial dimension to the dataset and forcing it to a non-zero constant.
The success rate $Q_{l.s.}$ of inhomogeneous perceptrons seems to be slightly higher than homogeneous ones, even for random datasets distributed around the origin of the axes.


\documentclass[inequalities.tex]{subfile}

\begin{document}
	\section{Complex Numbers}
	We may occasionally need complex numbers. So we provide a brief introduction here. A complex number $z$ is defined as
		\begin{align*}
			z
				& = x+iy
		\end{align*}
	where $x,y$ are real numbers and $i$ is the imaginary unit such that $i^{2}=-1$. We call $x$ the real component and $y$ the complex component of $z$. Complex numbers follow the same properties as vectors which we established in \autoref{sec:cs}. That is for two complex numbers $z_{1}=x_{1}+iy_{1}$ and $z_{2}=x_{2}+iy_{2}$,
		\begin{align*}
			z_{1}\pm z_{2}
				& = (x_{1}\pm x_{2})+i(y_{1}\pm y_{2})\\
			\langle z_{1},z_{2}\rangle
				& = x_{1}x_{2}+y_{1}y_{2}
		\end{align*}
	Here $\langle z_{1},z_{2}\rangle$ is the dot product of $z_{1}$ and $z_{2}$. We define an additional operation $\cdot$ as
		\begin{align*}
			z_{1}\cdot z_{2}
				& = x_{1}y_{2}-x_{2}y_{1}
		\end{align*}
	The \textit{modulus} of $z$ is similar to $L_{2}$ norm.
		\begin{align*}
			|z|
				& = \sqrt{x^{2}+y^{2}}
		\end{align*}
	We have the following property.
		\begin{align*}
			|z_{1}z_{2}|
				& = |z_{1}|\cdot|z_{2}|
		\end{align*}
	For a complex number $z=x+iy$, the \textit{conjugate} of $z$ is defined as
		\begin{align*}
			\bar{z}
				& = x-iy
		\end{align*}
	So, we have $z\bar{z}=|z|^{2}=|\bar{z}|^{2}$. The \textit{argument} of $z$ is defined as
		\begin{align*}
			\arg(z)
				& = \tan\left(\dfrac{y}{x}\right)
		\end{align*}
	You can think of it as the angle the point $(x,y)$ creates with the positive $X$ axis and the origin.
		\begin{problem}
			Prove the triangle inequality for complex numbers.
				\begin{align*}
					|z_{1}+z_{2}|
						& \leq |z_{1}|+|z_{2}|
				\end{align*}
			This inequality can be generalized as the following.
				\begin{align*}
					|z_{1}+\ldots+z_{n}|
						& \leq |z_{1}|+\ldots+|z_{n}|
				\end{align*}
			When does equality occur?
		\end{problem}
\end{document}


\subsubsection{Real Distinct Eigenvalues}
\noindent
Real, distinct eigenvalues are the simplest case, similar to real, distinct roots of an auxiliary equation.
\begin{theorem}
	Let $\left\{\lambda_1, \ldots, \lambda_n\right\}$ be the set of unique eigenvalues and $\left\{\vec{v_1}, \ldots, \vec{v_n}\right\}$ be the corresponding set of non-zero, unique, eigenvectors for an $n \times n$ matrix $A$. Then the set of fundamental solutions to the system $\vec{x}' = A\vec{x}$ is $\left\{e^{\lambda_1 t}\vec{v_1}, \ldots, e^{\lambda_n t}\vec{v_n} \right\}$
\end{theorem}

\ifodd\includeLinearSystemsExamples\input{./linearSystems/homogeneousSystems/realDistinctEigenvalues_example.tex}\fi


\subsection{Absolute Extrema}\label{subsec:AbsoluteExtremaSubsection}

Absolute extrema are also commonly referred to as
\dfont{global extrema}. Unlike local extrema, which are
only ``extreme" relative to points ``close to'' them, an
absolute (or global) extrema is ``extreme" relative to \emph{all}
other points in the interval under consideration.

\begin{definition}{Absolute Maxima and Minima}{AbsMaxMinDef}
	A real-valued function $f$ has an \deffont{absolute maximum}
	on an interval $I$ at $x_0$ if $f(x_0)$ is the largest value
	of $f$ on $I$; in other words, $f(x_0)\geq f(x)$ for all $x$
	in the domain of $f$ that are in $I$.

	\medskip
	A real-valued function $f$ has an \deffont{absolute minimum}
	on an interval $I$ at $x_0$ if $f(x_0)$ is the smallest value
	of $f$ on $I$; in other words, $f(x_0)\leq f(x)$ for all $x$
	in the domain of $f$ that are in $I$.
\end{definition}

\begin{example}{Absolute Extrema}{AbsoluteExtremaExamples}
	Consider the function $f(x)=x^2$ on the interval $(-\infty,\infty)$.
	This parabola has an absolute minimum at $x=0$.
	However, it does not have an absolute maximum.

	\medskip
	Consider the function $f(x)=|x|$ on the interval $[-1,2]$.
	This graph looks like a check mark. It has an absolute minimum
	at $x=0$ and an absolute maximum at $x=2$.

	\medskip
	Consider the function $f(x)=\cos x$ on the interval $[0,\pi]$.
	It has an absolute minimum at $x=\pi$ and an absolute maximum at $x=0$.

	\medskip
	Consider the function $f(x)=e^x$ on any interval $[a,b]$,
	where $a<b$. Since this exponential function is increasing,
	it has an absolute minimum at $x=a$ and an absolute maximum at $x=b$.
\end{example}

Like Fermat's Theorem, the following theorem has an intuitive appeal.
However, unlike Fermat's Theorem, the proof relies on a more advanced
concept called \textbf{compactness}, which will only be covered in a course
typically entitled Analysis. So, we will be content with understanding the
statement of the theorem.

\begin{theorem}{Extreme-Value Theorem}{ExtremeValueTheorem}
	If a function $f$ is continuous on a closed interval $[a,b]$,
	then $f$ has both an absolute maximum and an absolute minimum on $[a,b]$.
\end{theorem}

Although this theorem tells us that an absolute extremum exists, it
does not tell us what it is or how to find it.

Note that if an absolute extremum is inside the interval (i.e. not
an endpoint), then it must also be a local extremum. This immediately
tells us that to find the absolute extrema of a function on an interval,
we need only examine the local extrema inside the interval, and the
endpoints of the interval.

We can devise a method for finding absolute extrema for a function $f$ on a closed interval $[a,b]$:

\begin{enumerate}
	\item Verify the function is continuous on $[a,b]$.
	\item Find the derivative and determine all critical values of $f$ that are in $[a,b]$.
	\item Evaluate the function at the critical values found in Step 2 and the end points of the interval.
	\item Identify the absolute extrema.
\end{enumerate}

Why must a function be continuous on a closed interval in order to
use this theorem? Consider the following example.

\begin{example}{Absolute Extrema of a $1/x$}{AbsExtReciprocal}
	Find any absolute extrema for $f(x)=1/x$ on the interval $[-1,1]$.
\end{example}
\begin{solution}
	The function $f$ is not continuous at $x=0$. Since $0\in [-1,1]$,
	$f$ is not continous on the closed interval:
	\begin{align*}
		\lim_{x\to 0^+}f(x)&=+\infty	\\
		\lim_{x\to 0^-}f(x)&=-\infty\, ,
	\end{align*}
	so we are \emph{unable} to apply the Extreme-Value Theorem. Therefore,
	$f(x)=1/x$ does not have an absolute maximum or an absolute minimum on $[-1,1]$.
\end{solution}

However, if we consider the same function on an interval where it is
continuous, the theorem will apply. This is illustrated in the following example.

\begin{example}{Absolute Extrema of a $1/x$}{AbsExtReciprocal}
	Find any absolute extrema for $f(x)=1/x$ on the interval $[1,2]$.
\end{example}
\begin{solution}
	The function $f$ is continous on the interval, so we can apply the
	Extreme-Value Theorem. We begin with taking the derivative to be
	$f'(x)=-1/x^2$ which has a critical value at $x=0$, but since this
	critical value is not in $[1,2]$ we ignore it. The only points where
	an extrema can occur are the endpoints of the interval. To find the
	maximum or minimum we can simply evaluate the function: $f(1)=1$ and
	$f(2)=1/2$, so the absolute maximum is at $x=1$ and the absolute minimum is at $x=2$.
\end{solution}

Why must an interval be closed in order to use the above theorem? Recall
the difference between open and
closed intervals. Consider a function $f$ on the open interval $(0,1)$.
If we choose successive values of $x$ moving closer and closer to $1$,
what happens? Since 1 is not included in the interval we will not attain
exactly the value of 1. Suppose we reach a value of 0.9999 --- is it
possible to get closer to 1? Yes: There are infinitely many real numbers
between 0.9999 and 1. In fact, any conceivable real number close to 1
will have infinitely many real numbers between itself and 1. Now, suppose
$f$ is decreasing on $(0,1)$: As we approach 1, $f$ will continue to
decrease, even if the difference between successive values of $f$ is
slight. Similarly if $f$ is increasing on $(0,1)$.

Consider a few more examples:

\begin{example}{Determining Absolute Extrema}{AbsExtOne}
	Determine the absolute extrema of $f(x)=x^3-x^2+1$ on the interval $[-1,2]$.
\end{example}
\begin{solution}
	First, notice $f$ is continuous on the closed interval $[-1,2]$, so
	we're able to use Theorem~\ref{thm:ExtremeValueTheorem} to determine
	the absolute extrema. The derivative is $f'(x)=3x^2-2x$, and the critical
	values are $x=0,2/3$ which are both in the interval $[-1,2]$. In order
	to find the absolute extrema, we must consider all critical values that
	lie within the interval (that is, in $(-1,2)$) \emph{and} the endpoints of the interval.
	\begin{align*}
		f(-1)&=(-1)^3-(-1)^2+1=-1	\\
		f(0)&=(0)^3-(0)^2+1=1	\\
		f(2/3)&=(2/3)^3-(2/3)^2+1=23/27	\\
		f(2)&=(2)^3-(2)^2+1=5
	\end{align*}

	The absolute maximum is at (2,5) and the absolute minimum is at (-1,-1).
\end{solution}

\begin{example}{Determining Absolute Extrema}{AbsExtOne}
	Determine the absolute extrema of $f(x)=-9/x-x+10$ on the interval $[2,6]$.
\end{example}
\begin{solution}
	First, notice $f$ is continuous on the closed interval $[2,6]$,
	so we're able to use Theorem~\ref{thm:ExtremeValueTheorem} to
	determine the absolute extrema. The function is not continuous
	at $x=0$, but we can ignore this fact since 0 is not in $[2,6]$.
	The derivative is $f'(x)=9/x^2-1$, and the critical values are
	$x=\pm 3$, but only $x=+3$ is in the interval. In order to find
	the absolute extrema, we must consider all critical values that
	lie within the interval \emph{and} the endpoints of the interval.
	\begin{align*}
		f(2)&=-9/(2)-(2)+10=7/2=3.5	\\
		f(3)&=-9/(3)-(3)+10=4	\\
		f(6)&=-9/(6)-(6)+10=5/2=2.5
	\end{align*}

	The absolute maximum is at (3,4) and the absolute minimum is at (6,2.5).
\end{solution}

When we are trying to find the absolute extrema of a function on an
open interval, we cannot use the Extreme Value Theorem. However, if
the function is continuous on the interval, many of the same ideas apply.
In particular, if an absolute extremum exists, it must also be a local extremum.
In addition to checking values at the local extrema, we must check the behaviour
of the function as it approaches the ends of the interval.

Some examples to illustrate this method.

\begin{example}{Extrema of Secant}{SecantExtrema}
	Find the extrema of $\sec(x)$ on $(-\pi/2,\pi/2)$.
\end{example}
\begin{solution}
	Notice $\sec(x)$ is continuous on $(-\pi/2,\pi/2)$ and has one local minimum at 0. Also
	\[\lim_{x\to(-\pi/2)^+}\sec(x)=\lim_{x\to(\pi/2)^-}\sec(x)=+\infty\, ,\]
	so $\sec(x)$ has no absolute maximum, but the point $(0,1)$ is the absolute minimum.
\end{solution}

A similar approach can be used for infinite intervals.

\begin{example}{Extrema of $\frac{x^2}{x^2+1}$}{ExtofRationalFraction}
	Find the extrema of $\ds\frac{x^2}{x^2+1}$ on $(-\infty,\infty)$.
\end{example}
\begin{solution}
	Since $x^2+1\neq 0$ for all $x$ in $(-\infty,\infty)$ the function is continuous on this interval. This function has only one critical value at $x=0$, which is the local minimum and also the absolute minimum. Now, $\lim_{x\to\pm\infty}\frac{x^2}{x^2+1}=1$, so the function does not have an absolute maximum: It continues to increase towards 1, but does not attain this exact value.
\end{solution}

\Opensolutionfile{solutions}[ex]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Exercises for \ref{subsec:AbsoluteExtremaSubsection}}

\begin{enumialphparenastyle}

\begin{ex}
	Find the absolute extrema for $f(x)=-\frac{x+4}{x-4}$ on $[0,3]$.
	\begin{sol}
		Absolute maximum $(3,7)$; Absolute minimum $(0,1)$.
	\end{sol}
\end{ex}

\begin{ex}
	Find the absolute extrema for $f(x)=-\frac{x+4}{x-4}$ on $[0,3]$.
	\begin{sol}
		Absolute maximum $(3,7)$; Absolute minimum $(0,1)$.
	\end{sol}
\end{ex}

\begin{ex}
	Find the absolute extrema for $f(x)=\csc(x)$ on $[0,\pi]$.
	\begin{sol}
		Absolute minimum $(\pi/2,1)$; No absolute maximum.
	\end{sol}
\end{ex}

\begin{ex}
	Find the absolute extrema for $f(x)=\ln(x)/x^2$ on $[1,4]$.
	\begin{sol}
		Absolute minimum $(1,0)$; Absolute maximum $(e^{1/2},\frac{1}{2e})$.
	\end{sol}
\end{ex}

\begin{ex}
	Find the absolute extrema for $f(x)=x\sqrt{1-x^2}$ on $[-1,1]$.
	\begin{sol}
		Absolute minimum $(1,0)$; Absolute maximum $(e^{1/2},\frac{1}{2e})$.
	\end{sol}
\end{ex}

\begin{ex}
	Find the absolute extrema for $f(x)=xe^{-x^2/32}$ on $[0,2]$.
	\begin{sol}
		Absolute minimum $(0,0)$; Absolute maximum $(2,2e^{1/8})$.
	\end{sol}
\end{ex}

\begin{ex}
	Find the absolute extrema for $f(x)=x-\tan^{-1}(2x)$ on $[0,2]$.
	\begin{sol}
		Absolute minimum $(1/2,\frac{2-\pi}{4})$; Absolute maximum $(2,2-\tan^{-1}(4))$.
	\end{sol}
\end{ex}

\begin{ex}
	Find the absolute extrema for $f(x)=\frac{x}{x^2+1}$.
	\begin{sol}
		Absolute maximum $(1,1/2)$; Absolute minimum $(-1,-1/2)$.
	\end{sol}
\end{ex}

For the following exercises, sketch a potential graph of a continuous function on the closed interval $[0,4]$ with the given properties.

\begin{multicols}{2}
	\begin{ex}
		Absolute minimum at 0, absolute maximum at 2, local minimum at 3.
	\end{ex}

	\begin{ex}
		Absolute maximum at 1, absolute minimum at 2, local maximum at 3.
	\end{ex}

	\begin{ex}
		Absolute minimum at 4, absolute maximum at 1, local minimum at 2, local maxima at 1 and 3.
	\end{ex}
\end{multicols}

\end{enumialphparenastyle}


\chapter{Continuous Time Markov Chain}
\emph{“为了把连续的情况说清楚它还要发明一堆黑话。”}
\newpage

\section{Definition of Continuous Time Markov Chains}
    \subsection{Continuous Time Markov Chain}
        In the case of continuous time, it is technically difficult to specify the ``conditional probability given all of $X_r$ for all $r<s$''. Therefore instead, the Continuous Time Markov Chain is defined by

        \begin{definition}[Continuous Time Markov Chain]\label{def:ContinuousTimeMarkovChain}
            A stochastic process $X(t)$ is a \textbf{Continuous Time Markov Chain} if $\forall s,t \ge 0$, $\forall 0 \le s_0 < \cdots < s_n < s$
            \[ \mathbb{P}[X_{s+t} = j | X_s = i, X_{s_n} = i_n,\dots,X_{s_0}=i_0] = \mathbb{P}[X_t=j | X_0 = i] \]
        \end{definition}

        Given the current state, the previous states in the past is irrelevant for predicting the future, so we can simply ``throw away'' the previous states before $s$.

    \subsection{Transition Probability}
        In continuous case, the matrix-multiplication version of multistep transition probability cannot be applied directly. Instead we define a transition probability for each $t>0$.
        \[ p_t(i,j) = \mathbb{P}[X_t=j|X(0)=i] \]

        Recall the Chapman-Kolmogorov Equality \ref{thm:ChapmanKolmogorovEquality}, it still holds in the continuous case.

        \begin{theorem}[Chapman-Kolmogorov Equality, Continuous Case]\label{thm:ContinuousChapmanKolmogorovEquality}
            \[ \sum_{k}p_s(i,k)p_t(k,j) = p_{s+t}(i,j) \]
        \end{theorem}
        \begin{remark}
            This sugguests that if we know the transition probability for all $t<t_0$ (for some $t_0>0$), then we will be able to know the transition probability for all $t' \in \mathbb{R}$, by using the equality for sufficiently many times and keep doing the summation until $s+t=t'$.
        \end{remark}

        This further suggests that $p_t$ can be determined by its derivative at $t=0$.
        \begin{definition}[Jump Rate]\label{def:JumpRate}
            If the limit exists (assume it always does),
            \[ q(i,j) = \lim_{h \to 0}\frac{p_h(i,j)}{h} \quad \text{for $j \neq i$} \]
            then $q(i,j)$ is defined as the \textbf{jump rate} from $i$ to $j$.
        \end{definition}

    \subsection{A VERY Important Example}\label{sub:CTMCCoreExample}
        Let\footnote{“因为这个例子太重要了所以我给它起了个名字叫Example Star”--Chihao}
        \begin{itemize}
            \item $Y_n$ is a Markov Chain with transition matrix $u(i,j)$.
            \item $N(t)$ be a Poison Process with rate $\lambda$.
            \item $X(t)$ be a random variable defined as $X(t)=Y_{N(t)}$
        \end{itemize}
        That is, the discrete Markov chain $Y_n$ takes a jump according to the transition probability at each new arrival of the Poisson process $N(t)$.

        \subsubsection{Jump Rate of VERY Important Example.}
        By enumerating over all number of arrivals $n$
        \[ p_h(i,j) = \sum_{n=0}^\infty e^{-\lambda h}\frac{(\lambda h)^n}{n!} \cdot u^n(i,j) \]

        Notice that the probability of at least 2 jumps before time $h$ is 1 minus the probability of 0 and 1 jump,
        \[ 1 - (e^{-\lambda h}+\lambda h e^{-\lambda h}) \approx (\lambda h)^2/2! = o(h) \]
        So it converges to 0 when divided by $h$ and as $h \to 0$.
        Therefore
        \[ \frac{p_h(i,j)}{h} \approx \lambda e^{-\lambda h}u(i,j) \to \lambda u(i,j) \]

\section{More Examples}
    \subsection{The Poisson Process}
        Let $X(t)$ be the number of arrivals up to time $t$ in $Poisson(\lambda)$.

        Notice that at each new arrival, $X$ goes from $i$ to $i+1$ (with probability 1).

        Therefore $\forall n$
        \[ q(n, n+1) = \lambda \]

    \subsection{M/M/s Queue}
        We now consider a queue. Customers arrive according to a Poisson process of rate $\lambda$; the time to serve a customer is modeled by another Poisson process of rate $\mu$; there are only $s$ counters.

        Since the customers come in rate $\lambda$, we have
        \[ q(n, n+1) = \lambda \]

        Since the customers leave in rate $\mu$, we can use an exponential race in \ref{subs:ExponentialRace} to model the process, and therefore
        \[ q(n, n-1) = \begin{cases}
            \mu n & \quad n < s\\
            \mu s & \quad n \ge s
        \end{cases} \]

\section{Constructing a CT Markov Chain}
    \subsection{Informal Construction}
        Given a jump rate $q(i,j)$, the construction of a CTMC contains two steps, according to the formulation of \ref{sub:CTMCCoreExample}: 1) Choose $s\sim Exp(\lambda)$ to determine the next jump time, and 2) Choose $X(t+s) \sim u(i,\cdot)$ to determine the next state. Following this intuition, we define
        \[ \lambda_i = \sum_{j \neq i}q(i,j) \]
        $\lambda_i$ is the rate that the chain leaves $i$.
        \begin{itemize}
            \item If $\lambda_i = \infty$, then the chain leaves $i$ immediately.
            \item If $\lambda_i = 0$, then the chain will never leave $i$.
        \end{itemize}
        Since if the next state is chosen to be $i$, it is equivalent to state not moved, so we may choose different $\lambda_i$ so that in each jump we move into a different state. If we want identical $\Lambda$, we may choose $\Lambda = \sup \lambda_i$ and let $u(i,i)>0$.

        If $\lambda_i > 0$, let
        \[ u(i,j) = \frac{q(i,j)}{\lambda_i} \]
        be the probability that the chain goes to $j$ when it leaves $i$.

        \begin{itemize}
            \item If $X_t$ is in a state $i$ with $\lambda_i=0$, then the chain never leaves and we are done.
            \item If $X_t$ is in a state $i$ with $\lambda_i > 0$, then we first let the chain stay at $i$ according to an Exponential distribution with rate $\lambda_i$, then choose a destination according to $u(i,j)$.
        \end{itemize}

\section{Computing the Transition Probability}
    Given the jump rate $q(i,j)$, in the previous section we constructed a CTMC. In this section we move on to compute the transition probability.
    \begin{align*}
        p_{t+h}(i,j) - p_t(i,j) &= \sum_k p_h(i,k)p_t(k,j) - p_t(i,j) \quad \text{(CK Equation \ref{thm:ContinuousChapmanKolmogorovEquality})}\\
        &= \sum_{k \neq i} p_h(i,k)p_t(k,j) + (p_h(i,i)-1)p_t(i,j)\\
        &\triangleq A + B
    \end{align*}

    Multiply by $1/h$ and take limitation, then Expression A becomes
    \begin{align*}
        \lim_{h \to 0} \frac{1}{h}A &= \sum_{k \neq i}\lim_{h \to 0}\frac{1}{h}p_h(i,k)p_t(k,j)\\
        &= \sum_{k \neq i}q(i,k)p_t(k,j)
    \end{align*}

    and Expression B becomes
    \begin{align*}
        \lim_{h \to 0} \frac{1}{h}B &= \lim_{h \to 0}\frac{\sum_{k \neq i}p_h(i,k)}{h}p_t(i,j)\\
        &= \left(\sum_{k \neq i}q(i,k)\right)p_t(i,j)\\
        &= -\lambda_i p_t(i,j)
    \end{align*}

    And therefore
    \begin{equation}\label{eq:TransitionProbabilityKeyEq}
         \lim_{h \to 0}\frac{1}{h}\left(p_{t+h}(i,j) - p_t(i,j)\right) = \sum_{k \neq i}q_(i,k)p_t(k,j) - \lambda_i p_t(i,j)
    \end{equation}

    Notice that the LHS of equation (\ref{eq:TransitionProbabilityKeyEq}) is the derivative of $p_t(i,j)$, and therefore
    \[ p'_t(i,j) = \sum_{k \neq i}q_(i,k)p_t(k,j) - \lambda_i p_t(i,j) \]

    Further notice that  hte first term $\sum_{k \neq i}q_(i,k)p_t(k,j)$ in RHS of (\ref{eq:TransitionProbabilityKeyEq}) can be re-written in a matrix multiplication. Let
    \[ Q(i,j) = \begin{cases}
        q(i,j) &\quad i \neq j\\
        -\lambda_i &\quad i = j
    \end{cases} \]

    And therefore
    \begin{equation}\label{eq:KolmogorovBackwardEquation}
        P'_t = QP_t
    \end{equation}
    This equation is also known as the \textbf{Kolmogorov Backward Equation}.

    This looks similar to a differential equation and the solution is
    \[ P_t = e^{Qt} = \sum_{n=0}^{\infty}\frac{(tQ)^n}{n!} \]
    where $e^{Qt}$ is defined as
    \[ e^Q = \sum_{n=0}^{\infty} \frac{Q^n}{n!} \]

    Furthermore, if we write $\sum_{k}p_t(i,k)p_h(k,j)$ instead of $\sum_{k}p_h(i,k)p_t(k,j)$ in the first step of derivation, we will end up with the \textbf{Kolmogorov Forward Equation}.
    \begin{equation}\label{eq:KolmogorovForwardEquation}
        P'_t = P_tQ
    \end{equation}

    \subsection{Example: Poisson Process Revisited}
        In the previous sections, we know that given the jump rate $q(i,j)$, we are able to compute the transition probability and simulate the continuous time Markov chain.

        Let $X(t)$ be the number of arrivals of a Poisson process up to time $t$. $p_t(i,j)$ is the probability that the number rises from $i$ to $j$ in time interval $t$. By definition of the Poisson process, the arrival between $0$ and $t$ has a Poisson distribution of rate $\lambda t$, and $p_t(i,j)$ is simply the probability that the number of arrivals between $0$ and $t$ is $j-i$.
        \[ p_t(i,j) = e^{-\lambda t}\frac{(\lambda t)^{j-i}}{(j-i)!} \]

        We can use the result to verify the forward and backward equation \ref{eq:KolmogorovForwardEquation} and \ref{eq:KolmogorovBackwardEquation}.

    \subsection{Example: Two State Markov Chain}
        Suppose there are only two states $\Omega = \{1,2\}$. The two-state continuous-time Markov chain can be specified by
        \[ q(1,2) = \lambda \quad q(2,1) = \mu \]
        and
        \[ Q = \begin{bmatrix}
            -\lambda & \lambda\\
            \mu & -\mu
        \end{bmatrix} \]

        By Kolmogorov backward equation \ref{eq:KolmogorovBackwardEquation},
        \[
        \begin{bmatrix}
            p'_t(1,1) & p'_t(1,2)\\
            p'_t(2,1) & p'_t(2,2)
        \end{bmatrix} =
        \begin{bmatrix}
            -\lambda & \lambda\\
            \mu & -\mu
        \end{bmatrix}
        \begin{bmatrix}
            p_t(1,1) & p_t(1,2)\\
            p_t(2,1) & p_t(2,2)
        \end{bmatrix}
        \]

        Since there are only two states, it suffices to compute $p_t(1,1)$ and $p_t(2,1)$ only.
        \[ p'_t(1,1) = -\lambda p_t(1,1) + \lambda p_t(2,1) \]
        \[ p'_t(2,1) = \mu p_t(1,1) - \mu p_t(2,1) \]

        Therefore
        \[ \left(p_t(1,1)-p_t(2,1)\right)' = -(\lambda + \mu)\left(p_t(1,1)-p_t(2,1)\right) \]

        Solving the differential equation gives us
        \[ p_t(1,1) - p_t(2,1) = e^{-(\lambda + \mu)t} \]

        Plug the result back into the expression for $p'_t(1,1)$,
        \[ p'_t(1,1) = -\lambda e^{-(\lambda + \mu)t} \]

        Integrate over $t$
        \[ p_t(1,1) = p_o(1,1) - \lambda\int_0^t e^{-(\lambda + \mu)s}\mathrm{d}s = \frac{\mu}{\mu + \lambda}+\frac{\lambda}{\mu + \lambda} e^{-(\lambda + \mu)t} \]

\section{Limiting Behaviour: Continuous Time}
    \subsection{Irreducibility, Continuous Case}
        The intuition remains the same: the chain can jump from any state $i$ to any state $j$ in finite steps.
        \begin{definition}[Continuous Time Irreducibility]
            A continuous-time Markov chain is \textbf{irreducible} if there exists a sequence $k_0 = i, k_1, \dots, k_n=j$ such that
            \[ q(k_{m-1},k_m) > 0 \quad \forall 1 \le m \le n \]
        \end{definition}

        \begin{lemma}\label{lem:NecessaryConditionOfIrreducibility}
            If $X(t)$ is irreducible and $t>0$, then
            \[ p_t(i,j) > 0 \quad \forall i,j \]
        \end{lemma}

    \subsection{Stationary Distribution}
        \begin{lemma}[Stationary Distribution, Continuous Case]\label{lem:ContinuousTimeStationaryDistribution}
            A distribution $\pi$ is a stationary distribution if and only if
            \[ \pi^T Q = 0 \]
            or equivalently, iff
            \[ \pi^T P_t = \pi^T \quad \forall t \]
            but this one is difficult to verify.
        \end{lemma}
        \begin{remark}
            Using the definition of $Q$ we can prove that
            \[ \sum_{i \neq j}\pi(i)q(i,j) = \lambda_j\pi(j) \]
            The LHS is the rate that the chain ``gets into'' state $j$, and the RHS is the rate that the chain ``gets out of'' state $j$. So the intuition of continuous-time stationary distribution is that the rate that a chain goes in and goes out of a state are equal.
        \end{remark}

    \subsection{Convergence}
        \begin{theorem}[Convergence of CT Markov Chain]\label{thm:ConvergenceOfCTMarkovChain}
            If a continuous time Markov chain is irreducible and has a stationary distribution $\pi$, then
            \[ \lim_{t\to\infty}p_t(i,j) = \pi(j) \]
        \end{theorem}
        \begin{remark}
            Notice that we are not considering ``aperiodicity'' in CTMC, because a CTMC always has self-loops.
        \end{remark}

    \subsection{Detailed Balance Condition, Continuous Case}
        \begin{theorem}[Detailed Balance Condition]\label{thm:DetailedBalanceConditionOfCTMarkovChain}
            If
            \[ \forall i \neq j \quad \pi(i)q(i,j) = \pi(j)q(j,i) \]
            then $\pi$ is a stationary distribution.
        \end{theorem}


\section*{Exercises}

\begin{ex}
Let $B = \set{\begin{mymatrix}{r}
2 \\
-1
\end{mymatrix}, \begin{mymatrix}{r}
3 \\
2
\end{mymatrix} }$ be a basis of $\R^2$ and let $\vect{x} = \begin{mymatrix}{r}
5 \\
-7
\end{mymatrix}$ be a vector in $\R^2$. Find $C_B(\vect{x})$.
\end{ex}

\begin{ex}
Let $B = \set{\begin{mymatrix}{r}
1 \\
-1 \\
2
\end{mymatrix}, \begin{mymatrix}{r}
2 \\
1 \\
2
 \end{mymatrix}, \begin{mymatrix}{r}
-1 \\
0 \\
2
\end{mymatrix} }$
be a basis of $\R^3$ and let $\vect{x} = \begin{mymatrix}{r}
5 \\
-1 \\
4
\end{mymatrix}$ be a vector in $\R^2$. Find $C_B(\vect{x})$.
\begin{sol}
 $C_B(\vect{x}) =
\begin{mymatrix}{r}
2 \\
1 \\
-1
 \end{mymatrix}$.
\end{sol}
\end{ex}

\begin{ex}
Let $T: \R^2 \to \R^2$ be a linear transformation defined by $T \paren{\begin{mymatrix}{r}
a \\
b
\end{mymatrix}} = \begin{mymatrix}{r}
a+b \\
a-b
\end{mymatrix}$.

Consider the two bases
\[
B_1 = \set{\vect{v}_{1}, \vect{v}_{2} } = \set{\begin{mymatrix}{r}
1 \\
0
\end{mymatrix}, \begin{mymatrix}{r}
-1 \\
1
\end{mymatrix}
}
\]
 and
\[
B_2 = \set{\begin{mymatrix}{r}
1 \\
1
\end{mymatrix}, \begin{mymatrix}{r}
1 \\
-1
\end{mymatrix}
}
\]

Find the matrix $M_{B_2,B_1}$ of $T$ with respect to the bases $B_1$ and $B_2$.
\begin{sol}
$
M_{B_{2} B_{1}} = \begin{mymatrix}{rr}
\vspace{0.05in}1 & 0 \\
\vspace{0.05in}-1 & 1
\end{mymatrix} $
\end{sol}
\end{ex}


\section{Application: Cramer's rule}

\begin{outcome}
  \begin{enumerate}
  \item Use Cramer's rule to solve a system of equations with
    invertible coefficient matrix.
  \end{enumerate}
\end{outcome}

Another application of determinants is \textbf{Cramer's rule} for
solving a system of equations. Recall that we can represent a system
of linear equations in the form $A\vect{x}=\vect{b}$, where $\vect{x}$
is a vector of variables. Cramer's rule gives a formula for the
solutions $\vect{x}$ in the special case that the coefficient matrix
$A$ is a square invertible matrix. Note that Cramer's rule does not
apply if you have a system of equations in which there is a different
number of equations than variables (in other words, when $A$ is not
square), or when $A$ is not invertible.

\begin{theorem}{Cramer's rule}{cramers-rule}
  Suppose $A$ is an invertible $n\times n$-matrix and we wish to solve
  the system $A\vect{x}=\vect{b}$, where
  $\vect{x}=\mat{x_1,\ldots,x_n}$.  Then $x_i$ can be computed by the
  rule%
  \index{Cramer's rule}%
  \index{system of linear equations!Cramer's rule}%
  \index{determinant!Cramer's rule}
  \begin{equation*}
    x_i = \frac{\det(A_i)}{\det(A)},
  \end{equation*}
  where $A_i$ is the matrix obtained by replacing the $i\th$ column
  of $A$ with $\vect{b}$.
\end{theorem}

\begin{proof}
  Since $A$ is invertible, the solution to the system
  $A\vect{x}=\vect{b}$ is given by $\vect{x}=A^{-1}\vect{b}$.
  By Theorem~\ref{thm:inverse-and-determinant}, we have
  \begin{equation*}
    A^{-1} ~=~ \frac{1}{\det(A)} \adj(A),
  \end{equation*}
  and therefore
  \begin{equation*}
    \vect{x}=\frac{1}{\det(A)}\adj(A)\vect{b}.
  \end{equation*}
  Let $x_i$ be the $i\th$ component of $\vect{x}$ and $b_j$ be the $j\th$
  component of $\vect{b}$. Recall that the $\ijth$ entry of $\adj(A)$ is
  $\cofactor{A}{ji}$, the $ji\th$ cofactor of $A$. By definition of matrix
  multiplication, we have
  \begin{equation*}
    x_i = \frac{1}{\det(A)}(\cofactor{A}{1i}\,b_1+\ldots+\cofactor{A}{ni}\,b_n).
  \end{equation*}
  By the formula for the expansion of a determinant along a column,
  this is equal to
  \begin{equation*}
    x_i=\frac{1}{\det(A)}\begin{absmatrix}{ccccc}
      \ast & \cdots & b_1 & \cdots & \ast \\
      \vdots &  & \vdots &  & \vdots \\
      \ast & \cdots & b_n & \cdots & \ast
    \end{absmatrix},
  \end{equation*}
  where the $i\th$ column of $A$ is replaced with the column vector
  $\vect{b}$. But this last formula is exactly Cramer's rule.
\end{proof}

\begin{example}{Using Cramer's rule}{cramers-rule}
  Use Cramer's rule to solve the system of equations
  \begin{equation*}
    \begin{mymatrix}{rrr}
      1 & 2 & 1 \\
      3 & 2 & 1 \\
      1 & 4 & 1 \\
    \end{mymatrix} \begin{mymatrix}{c}
      x \\
      y \\
      z \\
    \end{mymatrix} =\begin{mymatrix}{r}
      3 \\
      5 \\
      6 \\
    \end{mymatrix}.
  \end{equation*}
\end{example}

\begin{solution}
  The matrices $A_1$, $A_2$, and $A_3$ are obtained by respectively
  replacing the first, second, and third column of $A$ by $\vect{b}$.
  We compute
  \begin{equation*}
    \det(A) = \begin{absmatrix}{rrr}
      1 & 2 & 1 \\
      3 & 2 & 1 \\
      1 & 4 & 1 \\
    \end{absmatrix}
    = 4,
    \qquad
    \det(A_1) = \begin{absmatrix}{rrr}
      3 & 2 & 1 \\
      5 & 2 & 1 \\
      6 & 4 & 1 \\
    \end{absmatrix}
    = 4
  \end{equation*}
  \begin{equation*}
    \det(A_2) = \begin{absmatrix}{rrr}
      1 & 3 & 1 \\
      3 & 5 & 1 \\
      1 & 6 & 1 \\
    \end{absmatrix}
    = 6,
    \qquad
    \det(A_3) = \begin{absmatrix}{rrr}
      1 & 2 & 3 \\
      3 & 2 & 5 \\
      1 & 4 & 6 \\
    \end{absmatrix}
    = -4.
  \end{equation*}
  Then by Cramer's rule,
  \begin{equation*}
    x = \frac{\det(A_1)}{\det(A)} = \frac{4}{4} = 1,\quad
    y = \frac{\det(A_2)}{\det(A)} = \frac{6}{4} = \frac{3}{2},\quad\mbox{and}\quad
    z = \frac{\det(A_3)}{\det(A)} = \frac{-4}{4} = -1.\quad
  \end{equation*}
  Thus, the solution is $(x,y,z)=(1,\frac{3}{2},-1)$.
\end{solution}

Cramer's rule is sometimes useful in situations where row operations
would be difficult to do. One such situation is when a system of
equations involves functions rather than numbers, as in the following
example.

\begin{example}{Using Cramer's rule for non-constant matrix}{cramers-rule-non-constant-matrix}
  Solve the following system of equations for $z$.
  \begin{equation*}
    \begin{mymatrix}{ccc}
      1 & 0 & 0 \\
      0 & e^{t}\cos t & e^{t}\sin t \\
      0 & -e^{t}\sin t & e^{t}\cos t
    \end{mymatrix}
    \begin{mymatrix}{c}
      x \\
      y \\
      z
    \end{mymatrix}
    = \begin{mymatrix}{c}
      1 \\
      t \\
      t^2
    \end{mymatrix}.
  \end{equation*}
\end{example}

\begin{solution}
  We are asked to find the value of $z$ in the solution. By Cramer's
  rule, we have
  \begin{equation*}
    z ~=~ \frac{
      \begin{absmatrix}{ccc}
        1 & 0 & 1 \\
        0 & e^{t}\cos t & t \\
        0 & -e^{t}\sin t & t^2
      \end{absmatrix}
    }{
      \begin{absmatrix}{ccc}
        1 & 0 & 0 \\
        0 & e^{t}\cos t & e^{t}\sin t \\
        0 & -e^{t}\sin t & e^{t}\cos t
      \end{absmatrix}
    }
    ~=~ \frac{e^t(t^2\cos t - t\sin t)}{e^{2t}}
    ~=~ e^{-t}(t^2\cos t - t\sin t).
  \end{equation*}
\end{solution}


\section{Definitions and Basic Properties}
We will make the following standing assumptions:
First, the state space $I$ is a countable set which we will almost always label as $\{1,2,\ldots\}$.
Also, we will be working in a probability space $(\Omega,\mathscr F,\mathbb P)$ where all relevant random variables are defined.
\begin{definition}
    A sequence of random variables $(X_n)_{n=0,1,\ldots}$ is a Markov chain if, for all $n\ge 0$ and $i_0,\ldots,i_{n+1}\in I$,
    $$\mathbb P[X_{n+1}=i_{n+1}|X_0=i_0,\ldots,X_n=i_n]=\mathbb P[X_{n+1}=i_{n+1}|X_n=i_n]$$
    given that these conditional probabilities are well-defined.\\
    A Markov chain is homogeneous if for all $i,j\in I$,
    $$\mathbb P[X_{n+1}=j|X_n=i]=\mathbb P[X_1=j|X_0=i]$$
\end{definition}
We are only interested in homogeneous Markov chains, so afterwards when we mention a Markov chain, we always mean a homogeneous one.\\
Then, a Markov chain is characterised by the initial distribution $\lambda=(\lambda_i)_{i\in I}$ given by $\lambda_i=\mathbb P[X_0=i]$ and the transition matrix $P=(p_{ij})_{i,j\in I}$ with $p_{ij}=\mathbb P[X_1=j|X_0=i]$.\\
Note that $\lambda$ is a distribution as $\lambda_i$ is always nonnegative and sums up to $1$.
$P$, at the same time, is a stochastic matrix, i.e. $p_{ij}$ is a distribution for every $i\in I$.
\begin{definition}
    $(X_n)$ is a Markov chain with initial distribution $\lambda$ and transition matrix $P$, or $(X_n)\sim\operatorname{Markov}(\lambda,P)$ if the above properties hold.
\end{definition}
\begin{theorem}\label{markov_alt_defn}
    $(X_n)\sim\operatorname{Markov}(\lambda,P)$ iff for all $n\ge 0$ and $i_0,\ldots,i_n\in I$,
    $$\mathbb P[X_0=i_0,\ldots,X_n=i_n]=\lambda_{i_0}p_{i_0i_1}\cdots p_{i_{n-1}i_n}$$
\end{theorem}
Pretty obvious but let's write this out.
\begin{proof}
    Suppose $(X_n)\sim\operatorname{Markov}(\lambda,P)$, then
    \begin{align*}
        &\phantom{=}\mathbb P[X_0=i_0,\ldots,X_n=i_n]\\
        &=\mathbb P[X_n=i_n|X_0=i_0,\ldots,X_{n-1}=i_{n-1}]\mathbb P[X_0=i_0,\ldots,X_{n-1}=i_{n-1}]\\
        &=\mathbb P[X_n=i_n|X_{n-1}=i_{n-1}]\mathbb P[X_0=i_0,\ldots,X_{n-1}=i_{n-1}]\\
        &=p_{i_{n-1}i_n}\mathbb P[X_0=i_0,\ldots,X_{n-1}=i_{n-1}]\\
        &=\cdots=\lambda_{i_0}p_{i_0i_1}\cdots p_{i_{n-1}i_n}
    \end{align*}
    Conversely, assume this is true, then set $n=0$ gives $\mathbb P[X_0=i_0]=\lambda_{i0}$, and that
    \begin{align*}
        \mathbb P[X_n=i_n|X_0=i_0,\ldots,X_{n-1}=i_{n-1}]&=\frac{\mathbb P[X_0=i_0,\ldots,X_n=i_n]}{\mathbb P[X_0=i_0,\ldots,X_{n-1}=i_{n-1}]}\\
        &=p_{i_{n-1}i_n}\\
        &=\mathbb P[X_n=i_n|X_{n-1}=i_{n-1}]
    \end{align*}
    which are exactly what we need.
\end{proof}
Let $\delta_i$ be the vector that has $1$ at $i^{th}$ entry and $0$ in other places.
\begin{theorem}
    Let $(X_n)\sim\operatorname{Markov}(\lambda,P)$, then conditioning on $X_m=i$, $(X_{m+n})_{n\ge 0}\sim\operatorname{Markov}(\delta_i,P)$ and is independent of $X_0,\ldots,X_m$.
\end{theorem}
\begin{proof}
    It suffices to show that\\
    (i) $\mathbb P[X_m=i_m,\ldots,X_{m+n}=i_{m+n}|X_m=i]=\delta_{ii_m}p_{i_mi_{m+1}}\cdots p_{i_{m+n-1}i_{m+n}}$.\\
    (ii) For every event $A$ determined by $X_1,\ldots,X_m$ and every event $B$ determined by $X_m,X_{m+1},\ldots$, we have
    $$\mathbb P[A\cap B|X_m=i]=\mathbb P[A|X_m=i]\mathbb P[B|X_m=i]$$
    In other words, ``independence of past and future given present''.\\
    The previous theorem implies both for the elementary events
    $$A=\{X_0=i_0,\ldots,X_m=i_m\},B=\{X_m=i_m,\ldots,X_{m+n}=i_{m+n}\}$$
    Indeed, after multiplying the both sides by $\mathbb P[X_m=i]$, (i) becomes
    $$\mathbb P[X_m=i_m,\ldots,X_{m+n}=i_{m+n}]=\delta_{ii_m}p_{i_mi_{m+1}}\cdots p_{i_{m+n-1}i_{m+n}}\mathbb P[X_m=i]$$
    and (ii) becomes
    $$\mathbb P[A\cap B]=\mathbb P[A]\mathbb P[B|X_m=i]=\delta_{ii_m}\mathbb P[A]\mathbb P[B]$$
    which are obviously true due to the theorem.
    Decomposing every events into elementary ones then proves the theorem.
\end{proof}
We say an array $(\lambda_i)$ as a distribution if $\sum_i\lambda_i=1$, and a measure if $\lambda_i\ge 0$ for all $i$.
When we mention arrays like these, we always mean row vectors.
\footnote{Staggering, I know.}
We write $(P^n)_{ij}=P^{(n)}_{ij}$ as convention.
When $\lambda_i>0$, we write $\mathbb P_i[A]=\mathbb P[A|X_0=i]$.
So $(X_n)_{n\ge 0}\sim\operatorname{Markov}(\delta_i,P)$ under $\mathbb P_i$.
\begin{theorem}
    Let $(X_n)_{n\ge 0}\sim\operatorname{Markov}(\lambda,P)$.
    Then for all $n,m\ge 0$, we have\\
    (a) $\mathbb P[X_n=j]=(\lambda P^n)_j$.\\
    (b) $\mathbb P_i[X_n=j]=p_{ij}^{(n)}$.
\end{theorem}
\begin{proof}
    Obvious.
\end{proof}
\begin{example}
    Consider the general two-state Markov chain with
    $$P=\begin{pmatrix}
        1-\alpha&\alpha\\
        \beta&1-\beta
    \end{pmatrix}$$
    As $P^{n+1}=P^nP$, we have $p_{11}^{(n+1)}=p_{12}^{(n)}\beta+p_{11}^{(n)}(1-\alpha)$.
    Also $p_{12}^{(n)}+p_{11}^{(n)}=1$, so $p_{11}^{(n+1)}=p_{11}^{(n)}(1-\alpha-\beta)+\beta$.
    As one can verify oneself, this gives
    $$p_{11}^n=\begin{cases}
        \frac{\beta}{\alpha+\beta}+\frac{\alpha}{\alpha+\beta}(1-\alpha-\beta)^n\text{, if $\alpha+\beta>0$}\\
        1\text{, if $\alpha+\beta=0$}
    \end{cases}$$
    And this basically told us the explicit formula of $P^n$, so this basic case of Markov chain can be solved easily.
\end{example}
The general method to compute the transition probability $p_{ij}^{(n)}$ for an $N$ state Markov chain is just as what we expect.
First, compute the eigenvalues $\lambda_1,\ldots,\lambda_N$ of $P$.
If the eigenvalues are all distinct, then $P$ is diagonalisable so we easily get
$$p_{ij}^{(n)}=a_1\lambda_1^n+\cdots+a_N\lambda_N^n$$ for some constants $(a_k)$ depending on $i,j$.
If an eigenvalue $\lambda_l$ has multiplicity $k$, we can simply replace $a_l$ by a degree $k$ polynomial in $n$.
This can be justified by considering the Jordan Normal form of $P$.
In general, the eigenvalues are not real numbers, but the values of $p_{ij}^{(n)}$ will are of course always real (and most of the times can be represented by trigonometric functions).
\begin{example}
    Consider the transition matrix
    $$P=\begin{pmatrix}
        &1&\\
        &1/2&1/2\\
        1/2&&1/2
    \end{pmatrix}$$
    We shall compute $p_{11}^{(n)}$.
    Now $\det(\lambda I-P)=(\lambda-1)(4\lambda^2+1)/4$, so the eigenvalues are $1,i/2,-i/2$.
    Therefore $p_{11}^{(n)}=a+b(i/2)^n+c(-i/2)^n$ for some constants $a,b,c$.
    Observe that
    $$\left(\pm\frac{i}{2}\right)^n=\frac{1}{2^n}\left(\cos\frac{\pi n}{2}\pm i\sin\frac{\pi n}{2}\right)$$
    Therefore we can get rid of $i$ and write
    $$p_{11}^{(n)}=\alpha+\frac{1}{2^n}\left[\beta\cos\frac{\pi n}{2}+\gamma\sin\frac{\pi n}{2}\right]$$
    for some constants $\alpha,\beta,\gamma$.
    Plugging in initial values reveals that $\alpha=1/5,\beta=4/5,\gamma=-2/5$, so
    $$p_{11}^{(n)}=\frac{1}{5}+\frac{1}{2^n}\left[\frac{4}{5}\cos\frac{\pi n}{2}-\frac{2}{5}\sin\frac{\pi n}{2}\right]$$
\end{example}
Now we can compute many data out of a Markov chain, but as good mathematicians we realise that there can be more properties in a Markov chain that can be explored.


\subsection{Classification of surfaces}

Given surfaces $M_{1}$ and $M_{2}$ we can form a new surface $M_{3}=M_{1}\#M_{2}$,
the connected sum, by removing from each the interior of ~disc, and
gluing together along the boundaries with a homeomorphism $f$.
\begin{fact}
Every closed connected surface is homeomorphic to one of $S^{2}$,
$M_{g}=T\#T\#\cdots\#T$ ($g$ copies of the torus $T=T^{2}$) or
$N_{h}=P\#P\#\cdots\#P$ (h copies of $P=\mathbb{R}P^{2})$.
\end{fact}
$M_{g}$ can also be understood in a planar model as the $4g$-gon
with edges identified in pairs, i.e. according to the word

\begin{align*}
[a_{1},b_{1}][a_{2},b_{2}]\cdots[a_{g},b_{g}].
\end{align*}

$N_{h}$ can be written as the $2h$-gon with edges identified according
to the word

\begin{align*}
a_{1}^{2}a_{2}^{2}\cdots a_{n}^{2}.
\end{align*}

Note: We can see that these surfaces are all different (not homeomorphic)
by considering the Abelianisations. We then have some corollaries.
\begin{cor}
No two $S^{2},M_{g},N_{h}$ are homeomorphic. In fact, two closed
surfaces are homeomorphic iff they have the same Abelian $\pi_{1}$,
iff they are homotopically-equivalent.
\end{cor}

\begin{cor}
A closed surface is simply connected iff it is homeomorphic to $S^{2}$.
\end{cor}


\subsection{Abelian groups}

A commutative group, that is where \(a\odot b=b\odot a\).

The following are abelian groups:

\begin{itemize}
\item Integers and addition
\item \(\{-1, 1\}\) and multiplication
\end{itemize}

The following are not abelian groups:

\begin{itemize}
\item Natural numbers above \(0\) and addition
\item Rational numbers and division
\item Natural numbers and addition
\item Natural numbers up to \(10\) and addition
\item \(n\times n\) matrices with determinants other than \(0\)
\end{itemize}


\documentclass[../PHYS306Notes.tex]{subfiles}

\begin{document}
\subsection{Worksheet - Review of Weeks 1-4}
\begin{p}
Which of these problems must be solved using the calculus of variations?
\newline 1. Find the period of small oscillations for a particle sliding (without friction) on the inside of a sphere.
\newline 2. Find the surface with fixed area that encloses the maximum volume.
\newline 3. Find the path between two points that minimizes the time for a particle to slide (without friction) between the points.
\newline 4. Find the path of a projectile (with no air resistance) that leads to the maximum range.
\end{p}
\begin{s}
Exactly two. For 1, we looked at the equations of motion and Taylor expanded around the minimum for small $\phi$. So, we aren't looking for the path that minimizes some functional in this case, hence its not particularly a variational problem. For 2, we have an optimization problem; we are trying to extremize/maximize a volume. We can write an expression for the surface area, and add in a constraint (e.g. we could use Lagrange multipliers for example). We could optimize this with the Calculus of variations. 3 is the brachistochrone problem, obviously yes. 4 does not require varations; this is just a question of initial conditions of the trajectory.
\end{s}

\begin{p}
A calculus of variations problem requires minimizing
$$
J[y(x)]=\int_{x_{1}}^{x_{2}} f\left[y(x) ; y^{\prime}(x) ; x\right] d x
$$
When we solve Euler's equation
$$
\frac{\partial f}{\partial y}-\frac{d}{d x} \frac{\partial f}{\partial y^{\prime}}=0
$$
what do we learn?
\end{p}
\begin{s}
We learn of the path $y(x)$ that minimizes $J[y(x)]$; the EL equation gives a differential equation for the path $y(x)$ which minimizes $J[y(x)]$.
\end{s}

\begin{p}
What is the Lagrangian of a particle of mass $m$ attached to a spring with spring constant $k$?
\end{p}
\begin{s}
$\LL = T - U = \frac{1}{2}m\dot{x}^2 - \frac{1}{2}kx^2$ (the kinetic energy term minus the potential energy term).
\end{s}

\begin{p}
What is the Lagrangian of a pendulum of mass $m$, length $l$? Assume the potential energy is zero when $\theta$ is zero.
\end{p}
\begin{s}
$\LL(\theta, \dot{\theta}, t) = \frac{1}{2}ml^2\dot{\theta}^2 - mgl(1-\cos\theta)$
\end{s}

\begin{p}
For which of these systems could you use Lagrange's equations of motion?
\newline 1. A double pendulum: a pendulum (mass $\mathrm{m}$, length 1 ) has a second pendulum (mass $\mathrm{m}$, length l) connected to its bob.
\newline 2. A projectile moves in two dimensions with gravity and air resistance.
\newline 3. A bead slides without friction on a circular, rotating wire.
\end{p}
\begin{s}
Everything but 2 works; with 2, we have friction/a non-conservative force and hence the Lagrange equations no longer apply (though we can add in a correction to account for this).
\end{s}

\begin{p}
A particle moves in one dimension with Lagrangian $\LL = T - U$. Suppose we shift the potential energy $U$ by a constant $C$. What changes?
\end{p}
\begin{s}
The value of $S$ changes ($S$ depends on the Lagrangian) but the physical path $x(t)$ taken by the particle remains invariant (that is, the path that gives $\delta S = 0$); one way of seeing this is the equations of motion are given by derivatives of the Lagrangian, which would remove the effects of any constants.
\end{s}

\begin{p}
A bead of mass $m$ slides on a circular wire of radius $R$, which rotates about a vertical axis with angular velocity $\Omega$. The equation of motion of the bead is \[\ddot{\theta} + \frac{g}{R}\sin\theta - \Omega^2\sin\theta\cos\theta = 0\]
What are the equilibrium values of $\theta$?
\end{p}
\begin{s}
$\theta = 0$ and $\theta = \arccos(\frac{g}{R\Omega^2})$.
\end{s}

\begin{p}
The equation of motion for small motions about the equilibrium $\theta = \arccos(\frac{g}{R\Omega^2})$ above is given by:
\[\ddot{\theta} +\Omega^2\sin^\theta_0\theta = 0\]
What is the oscillation frequency of the bead?
\end{p}
\begin{s}
$\omega = \Omega\sin\theta_0$
\end{s}

\begin{p}
Which of these systems are holonomic? E.g. which of these systems have a constraint that can be written as $f(q_1, \ldots, q_n, t) = 0$?
\newline 1. The double pendulum, but with the lower mass attached by a spring instead of a spring.
\newline 2. The motion of a hockeyu puck around a frictionless air hockey table
\newline 3. A bead moving frictionless on a circular wire hoop spinning at fixed angular velocity.
\end{p}
\begin{s}
A and C. With C, this is obviously possible (we constrain the radius). With B, the constraint is an inequality (e.g. the normal force is just such that the mass stays at the level of the table), which makes it non-holonomic. With A, we have that the lower mass can move more freely; the system is still holonomic, we just got rid of the constraint on the second mass. Note that if forces are dissipative, then we can also not write constrains as holonomic.
\end{s}

\begin{p}
What is the constraint equation?
\begin{center}
    \includegraphics[scale=0.75]{Lecture-13/w13-img1.png}
\end{center}
\end{p}
\begin{s}
$f(x,y) = \frac{y}{x} - \tan(\alpha) = 0$. A typical holonomic constraint.
\end{s}

\begin{p}
A particle of mass $m$ slides on the outside of a cylinder of radius $a$. A good choice of generalized coordinates is $(r, \theta)$. What is the constraint equation?
\end{p}
\begin{s}
$f(r,\theta) = r - a = 0$
\end{s}

\begin{p}
What can you conclude from the fact that $\dpd{\LL}{\dot{x}_i}$ is constant for all $i$?
\end{p}
\begin{s}
Momentum is conserved.
\end{s}

\begin{p}
How can we rewrite:
\[\int_{t_1}^{t_2}p_j\dod{}{t}\delta q_j dt\]
\end{p}
\begin{s}
By integrating by parts, we get:
\[-\int_{t_1}^{t_2}\dot{p}_j\delta q_j dt\]
(the variation vanishes at the boundary, so it is discarded).
\end{s}

\end{document}


\section{Inverse Trigonometric Functions}\label{sec:InvTrigFunctionsSection}
The trigonometric functions frequently arise in problems, and often it
is necessary to invert the functions, for example, to find an angle with
a specified sine. Of course, there are many angles with the same sine,
so the sine function doesn't actually have an inverse that reliably
``undoes'' the sine function. If you know that $\sin x=0.5$, you can't
reverse this to discover $x$, that is, you can't solve for
$x$, as there are infinitely many angles with sine
$0.5$. Nevertheless, it is useful to have something like an inverse to
the sine, however imperfect. The usual approach is to pick out some
collection of angles that produce all possible values of the sine
exactly once. If we ``discard'' all other angles, the resulting
function does have a proper inverse.

The sine takes on all values between $-1$ and $1$ exactly once on the
interval $[-\pi/2,\pi/2]$.
$$\includegraphics[width=4in]{images2/inv-trig-sin}$$
If we truncate the sine, keeping only the
interval $[-\pi/2,\pi/2]$, then this truncated sine has an inverse function. We call this
the inverse\index{inverse sine} sine or the arcsine\index{arcsine}, and
write it in one of two common notation: $y=\arcsin(x)$, or $y=\sin^{-1}(x)$.
$$\includegraphics[width=2in]{images2/inv-trig-arcsin}$$

Recall that a function and its inverse undo each other in either
order, for example, $\ds (\root3\of x)^3=x$ and $\ds \root3\of{x^3}=x$. This
does not work with the sine and the ``inverse sine'' because the
inverse sine is the inverse of the truncated sine function, not the
real sine function. It is true that $\sin(\arcsin(x))=x$, that is, the
sine undoes the arcsine. It is not true that the arcsine undoes the
sine, for example, $\sin(5\pi/6)=1/2$ and $\arcsin(1/2)=\pi/6$, so
doing first the sine then the arcsine does not get us back where we
started. This is because $5\pi/6$ is not in the domain of the
truncated sine. If we start with an angle between $-\pi/2$ and $\pi/2$
then the arcsine does reverse the sine: $\sin(\pi/6)=1/2$ and
$\arcsin(1/2)=\pi/6$.

\begin{example}{Arcsine of Common Values}{Arcsine of Common Values}
Compute $\sin^{-1}(0)$,\quad $\sin^{-1}(1)$\quad and\quad $\sin^{-1}(-1)$.
\end{example}

\begin{solution}
These come directly from the graph of $y=\arcsin x$:
$$\sin^{-1}\left( 0\right) =0\qquad \qquad \sin^{-1}(1)=\frac{\pi}{2}\qquad  \qquad \sin^{-1}(-1)=-\frac{\pi}{2}$$
\end{solution}

We can do something similar for the cosine function. As with the sine, we must
first truncate the cosine so that it can be inverted, in particular, we use the interval $[0,\pi]$.
$$\includegraphics[width=4in]{images2/inv-trig-cos}$$
Note that the
truncated cosine uses a different interval than the truncated sine, so
that if $y=\arccos(x)$ we know that $0\le y\le \pi$.
$$\includegraphics[width=2in]{images2/inv-trig-arccos}$$

\begin{example}{Arccosine of Common Values}{Arccosine of Common Values}
Compute $\cos^{-1}(0)$,\quad $\cos^{-1}(1)$\quad and\quad $\cos^{-1}(-1)$.
\end{example}

\begin{solution}
These come directly from the graph of $y=\arccos x$:
$$\cos^{-1}\left( 0\right) =\frac{\pi}{2}\qquad \qquad \cos^{-1}(1)=0\qquad  \qquad \cos^{-1}(-1)=\pi$$
\end{solution}

Finally we look at the tangent; the other trigonometric functions also
have ``partial inverses'' but the sine, cosine and tangent are enough
for most purposes. The truncated tangent uses an interval of $(-\pi/2,\pi/2)$.
$$\includegraphics[width=2.75in]{images2/inv-trig-tan}$$
Reflecting the truncated tangent in the line $y=x$ gives the arctangent function.
$$\includegraphics[width=3.5in]{images2/inv-trig-arctan}$$

\begin{example}{Arctangent of Common Values}{Arctangent of Common Values}
Compute $\tan^{-1}(0)$.
What value does $\tan^{-1}x$ approach as $x$ gets larger and larger?
What value does $\tan^{-1}x$ approach as $x$ gets large (and negative)?
\end{example}

\begin{solution}
These come directly from the graph of $y=\arctan x$.
In particular, $\tan^{-1}(0)=0$.
As $x$ gets larger and larger, $\tan^{-1}x$ approaches a value of $\frac{\pi}{2}$, whereas, as $x$ gets large but negative, $\tan^{-1}x$ approaches a value of $-\frac{\pi}{2}$.
\end{solution}

The cancellation rules are tricky since we restricted the domains of the trigonometric functions in order to obtain inverse trig functions:

\begin{formulabox}[Cancellation Rules]
\[ \sin ( \sin^{-1} x ) = x, \ \ \ x \in [-1, 1] \qquad\qquad \sin^{-1} ( \sin x) = x, \ \ \ x \in \left[-\frac{\pi}{2}, \frac{\pi}{2} \right] \]
\[ \cos ( \cos^{-1} x ) = x, \ \ \ x \in [-1, 1] \qquad\qquad \ \ \ \cos^{-1} ( \cos x) = x, \ \ \ x \in \left[0, \pi \right]\ \ \  \]
\[ \tan ( \tan^{-1} x ) = x, \ \ \ x \in (-\infty, \infty) \qquad\qquad \tan^{-1} ( \tan x) = x, \ \ \ x \in \left(-\frac{\pi}{2}, \frac{\pi}{2} \right) \]
\end{formulabox}

\begin{example}{Arcsine of $1/2$}{Arcsine of $1/2$}
Find $\sin ^{-1}\left( 1/2\right) $.
\end{example}

\begin{solution}
Since $\sin^{-1}(x)$ outputs values in $[-\pi/2,~\pi/2]$, the answer must be in this interval.
Let $\theta=\sin^{-1}(1/2)$.
We need to compute $\theta$.
Take the sine of both sides to get $\sin \theta = \sin(\sin^{-1}(1/2))=1/2$ by the cancellation rule.
There are many angles $\theta$ that work, but we want the one in the interval $[-\pi/2,~\pi/2]$.
Thus, $\theta=\pi/6$ and hence, $\ds\sin^{-1}\left(\frac{1}{2}\right)=\frac{\pi}{6}$.
\end{solution}

\begin{example}{Arccosine and the Cancellation Rule}{Arccosine and the Cancellation Rule}
Compute $\cos ^{-1}(\cos(0))$,\quad $\cos ^{-1}(\cos(\pi))$,\quad $\cos^{-1}(\cos(2\pi))$,\quad $\cos^{-1}(\cos(3\pi))$.
\end{example}

\begin{solution}
Since $\cos^{-1}(x)$ outputs values in $[0,~\pi]$, the answers must be in this interval.
The first two we can cancel using the cancellation rules:
$$\cos ^{-1}(\cos(0))=0\qquad\mbox{and}\qquad\cos ^{-1}(\cos(\pi))=\pi.$$
The third one we cannot cancel since $2\pi\notin[0,\pi]$:
$$\cos ^{-1}(\cos(2\pi))\mbox{\quad\red{is NOT equal to}\quad}2\pi.$$
But we know that cosine is a $2\pi$-periodic function, so $\cos(2\pi)=\cos(0)$:
$$\cos ^{-1}(\cos(2\pi))=\cos^{-1}(\cos(0))=0$$
Similarly with the fourth one, we can \red{NOT} cancel yet since $3\pi\notin[0,\pi]$. Using $\cos(3\pi)=\cos(3\pi-2\pi)=\cos(\pi)$:
$$\cos ^{-1}(\cos(3\pi))=\cos^{-1}(\cos(\pi))=\pi.$$
\end{solution}

\begin{example}{The Triangle Technique}{The Triangle Technique}
Rewrite the expression $\cos(\sin^{-1}x)$ without trig functions.
Note that the domain of this function is all $x\in[-1,1]$.
\end{example}

\begin{solution}
Let $\theta=\sin^{-1}x$.
We need to compute $\cos\theta$.
Taking the sine of both sides gives $\sin \theta = \sin(\sin^{-1}(x))=x$ by the cancellation rule.
We then draw a right triangle using $\sin\theta=x/1$:
$$\includegraphics[width=1.5in]{images2/inv-trig-triangle-1}$$
If $z$ is the remaining side, then by the Pythagorean Theorem:
$$z^2+x^2=1\qquad\to\qquad z^2=1-x^2\qquad\to\qquad z=\pm\sqrt{1-x^2}$$
and hence $z=+\sqrt{1-x^2}$ since $\theta\in[-\pi/2,~\pi/2]$.
Thus, $\ds\cos\theta=\sqrt{1-x^2}$ by SOH CAH TOA, so, $\ds\cos(\sin^{-1}x)=\sqrt{1-x^2}$.
\end{solution}

\begin{example}{The Triangle Technique 2}{The Triangle Technique 2}
For $x\in(0,1)$, rewrite the expression $\ds\sin(2\cos^{-1}x)$.
Compute $\sin(2\cos^{-1}(1/2))$.
\end{example}

\begin{solution}
Let $\theta=\cos^{-1}x$ so that $\cos\theta = x$.
The question now asks for us to compute $\sin(2\theta)$.
We then draw a right triangle using $\cos\theta=x/1$:
$$\includegraphics[width=1.75in]{images2/inv-trig-triangle-2}$$
To find $\sin(2\theta)$ we use the double angle formula $\ds \sin(2\theta)=2\sin\theta\cos\theta$.
But $\sin\theta=\sqrt{1-x^2}$, for $\theta\in[0,\pi]$, and $\cos\theta=x$.
Therefore, $\ds\sin(2\cos^{-1}x)=2x\sqrt{1-x^2}$.
When $x=1/2$ we have $\ds\sin(2\cos^{-1}(1/2))=\frac{\sqrt 3}{2}$.
\end{solution}

In Figure \ref{fig:domain_trig} we show the restrictions of the domains of the standard trigonometric functions that allow them to be invertible.\\

%\noindent %\hskip-110pt%
%\noindent\begin{minipage}{\textwidth+200pt}
\small\noindent
%\centering%\begin{center}
%\noindent\begin{minipage}[t]{.5\textwidth}%
\begin{tabular}{cccccc}
Function & Domain & Range &\parbox[b]{40pt}{\centering Inverse Function} & Domain & Range\\ \hline
\rule{0pt}{12pt} $\sin x$ & $[-\pi/2, \pi/2]$ & $[-1,1]$&$\sin^{-1} x$ & $[-1,1]$ & $[-\pi/2, \pi/2]$ \\
\rule{0pt}{12pt}$\cos x$ & $[0,\pi]$ & $[-1,1]$&$\cos^{-1}(x)$ & $[-1,1]$ & $[0,\pi]$ \\
\rule{0pt}{12pt}$\tan x$ & $(-\pi/2,\pi/2)$ & $(-\infty,\infty)$&$\tan^{-1}(x)$ & $(-\infty,\infty)$ & $(-\pi/2,\pi/2)$	\\
%\rule{0pt}{12pt} $\csc x$ & $[-\pi/2,0)\cup (0, \pi/2]$ & $(-\infty,-1]\cup [1,\infty)$&$\csc^{-1} x$ & $(-\infty,-1]\cup [1,\infty)$ & $[-\pi/2,0)\cup (0, \pi/2]$  \\
%the following makes integral of inverse function nicer
\rule{0pt}{12pt} $\csc x$ & $[-\pi/2,0)\cup (0, \pi/2]$ & $(-\infty,-1]\cup [1,\infty)$&$\csc^{-1} x$ & $(-\infty,-1]\cup [1,\infty)$ & $(0,\pi/2]\cup (\pi, 3\pi/2]$  \\
%\rule{0pt}{12pt}$\sec x$ & $[0,\pi/2)\cup (\pi/2,\pi]$ & $(-\infty,-1]\cup [1,\infty)$&$\sec^{-1}(x)$ & $(-\infty,-1]\cup [1,\infty)$ & $[0,\pi/2)\cup (\pi/2,\pi]$ \\
%the following makes integral of inverse function nicer
\rule{0pt}{12pt}$\sec x$ & $[0,\pi/2)\cup (\pi/2,\pi]$ & $(-\infty,-1]\cup [1,\infty)$&$\sec^{-1}(x)$ & $(-\infty,-1]\cup [1,\infty)$ & $[0,\pi/2)\cup [\pi,3\pi/2]$ \\
\rule{0pt}{12pt}$\cot x$ & $(0,\pi)$ & $(-\infty,\infty)$&$\cot^{-1}(x)$ &  $ (-\infty,\infty)$ & $(0,\pi)$
\end{tabular}
%\captionsetup{type=figure}
\caption{Domains and ranges of the trigonometric and inverse trigonometric functions. \label{fig:domain_trig}}
%\end{center}
%\normalsize
%\end{minipage}
%\captionsetup{type=figure}%
%\caption{Domains and ranges of the trigonometric and inverse trigonometric functions.}\label{fig:domain_trig}
%\end{minipage}
%}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:InvTrigFunctionsSection}}

\begin{enumialphparenastyle}

\begin{ex}
Compute the following:
\begin{multicols}{2}
\begin{enumerate}
	\item	$\sin^{-1}(\sqrt{3}/2)$
	\item	$\cos^{-1}(-\sqrt{2}/2)$
\end{enumerate}
\end{multicols}
\begin{sol}
\begin{enumerate}
	\item	$\pi/3$
	\item	$3\pi/4$
\end{enumerate}
\end{sol}
\end{ex}

\begin{ex}
Compute the following:
\begin{multicols}{2}
\begin{enumerate}
	\item	$\sin^{-1}\left(\sin(\pi/4)\right)$
	\item	$\sin^{-1}\left(\sin(17\pi/3)\right)$
	\item	$\cos\left(\cos^{-1}(1/3)\right)$
	\item	$\tan\left(\cos^{-1}(-4/5)\right)$
\end{enumerate}
\end{multicols}
\begin{sol}
\begin{multicols}{2}
\begin{enumerate}
	\item	$\pi/4$
	\item	$-\pi/3$
	\item	$1/3$
	\item	$-3/4$
\end{enumerate}
\end{multicols}
\end{sol}
\end{ex}

\begin{ex}
Rewrite the expression $\tan\left(\cos^{-1}x\right)$
without trigonometric functions. What is the domain of this function?
\begin{sol}
$\sqrt{1-x^2}/x$ with domain $[-1,0)\cup(0,1]$.
\end{sol}
\end{ex}

\end{enumialphparenastyle}


%================================
\section{Connectedness}
%================================

%--------------------------------
\begin{definition}
	[connectedness]
	\label{def: connectedness}
	Let $(X, \mathcal T)$ be a topological space. $(X, \mathcal T)$ is said to be \textit{connected} iff $X$ is not empty and it it not the union of any disjoint open sets. That is,
	$$
	\forall U, V \in \mathcal T : X = U \cup V : U \cap V \ne \emptyset.
	$$
\end{definition}
%--------------------------------

%--------------------------------
\begin{definition}
	[path-connectedness]
	\label{def: path-connectedness}
	Let $(X, \mathcal T)$ be a topological space.
	\begin{enumerate}[(i)]
		\item A map $\gamma: [0,1] \to X$ is called a \textit{path} in $X$ iff it is continuous. If $\gamma(0) = x$ and $\gamma(1)=y$, we say that $\gamma$ is path from $x$ to $y$ in $X$.
		\item $X$ is said to be \textit{path-connected} iff for all $x, y \in X$ there is a path from $x$ to $y$ in $X$.
	\end{enumerate}
\end{definition}
%--------------------------------


\documentclass[../algorithms.tex]{subfiles}
\begin{document}
This chapter is used to conclude all the algorithms and data structures relation by showing examples. For example, we show how we can model this problem using different data structure and end up solving this problem with different algorithms. For some example, we should how different algorithm matters.
\section{Complete Search VS Smart Search VS Dynamic Programming}
\begin{examples}
\item \textbf{956. Tallest Billboard (hard).} You are installing a billboard and want it to have the largest height.  The billboard will have two steel supports, one on each side.  Each steel support must be an equal height. You have a collection of rods which can be welded together.  For example, if you have rods of lengths 1, 2, and 3, you can weld them together to make a support of length 6. Return the largest possible height of your billboard installation.  If you cannot support the billboard, return 0.
\begin{lstlisting}[numbers=none]
Example 1:

Input: [1,2,3,6]
Output: 6
Explanation: We have two disjoint subsets {1,2,3} and {6}, which have the same sum = 6.

Example 2:

Input: [1,2,3,4,5,6]
Output: 10
Explanation: We have two disjoint subsets {2,3,5} and {4,6}, which have the same sum = 10.

Example 3:

Input: [1,2]
Output: 0
Explanation: The billboard cannot be supported, so we return 0.
\end{lstlisting}
Note: 0 <= rods.length <= 20, 1 <= rods[i] <= 1000. The sum of rods is at most 5000.

\textbf{Solution 1: Naive Complete Search.} We need to have two billboard: left and right side. Given a state (x, y), which means the sum of left and right billboard is x and y respectively. Now, to add a new rod v into the state, we have three options: 1) not use v, (x, y); 2) put v to left side: (x+v, y); 3) put v to the right side (x, y+v). If x' and y' end up to be equal, we can track the maximal value. Given the array size of n, and initial state (0, 0), we use search, each state will have three branches because of the three options, therefore, the state transfer graph(tree) will expand to $\sum(3^0, 3^1, 3^2, ..., 3^{n}) = 3^{n+1}-1$. The code is:
\begin{lstlisting}[language=Python]
def tallestBillboard(self, rods):
    if not rods:
        return 0
    n = len(rods)
    def helper(i, l, r, ans):
        if l == r:
            ans[0] = max(ans[0], l)
        if i == n:
            return
        helper(i+1, l, r, ans)
        helper(i+1, l+rods[i], r, ans)
        helper(i+1, l, r+rods[i], ans)
        return
    ans = [0]
    helper(0, 0, 0, ans)
    return ans[0]
\end{lstlisting}
\textbf{Solution 2: Smart Complete Search.} Similar to Bidirectional Search, which decrease the time complexty from $n$ power to $n/2$ by starting searching simulatously from soure and goal, and save the final nodes, and its path length from its starting node. The total length can be obtained by the common final nodes (its complexity depends on its total number of states), and sum up its path length of each side. For this problem, we can do the same by dividing the array into half and half. In each half, we enumerate all possible states and save them in a dictionary. If we have (x, y) in the left, and (y, x) in the right, then we have a possible value, which is $x+y$.  In this problem, we can compress the states: assume we have state (x, x+d) and (y, y+d), x < y, the optimal value will always be decided by the second set (y, y+d), therefore, instead of saving all possible (x,y), we can save it as (x-y) and the value is the maximum left length.
\begin{lstlisting}[language=Python]
def tallestBillboard(self, rods):
    def make(A):
        # enumerate all possible states
        states = {(0, 0)}
        for x in A:
            states |= ({(a+x, b) for a, b in states} |
                       {(a, b+x) for a, b in states})
        # compress states
        delta = {}
        for a, b in states:
            delta[a-b] = max(delta.get(a-b, 0), a)
        return delta
    N = len(rods)
    Ldelta = make(rods[:N//2])
    Rdelta = make(rods[N//2:])

    # meet in the middle
    ans = 0
    for d in Ldelta:
        if -d in Rdelta:
            ans = max(ans, Ldelta[d] + Rdelta[-d])
    return ans
\end{lstlisting}
\textbf{Snapsack\_based Dynamic Programming.}  In this solution, we define a two-dimensional dp array, dp[i][j] means using the first i rods to get a difference j which is the $y-x$. The saved value is the maximum common height for states with the same difference.  The time complexity is bounded by $O(21\times 5000)$. This is similar to the snapsack problem, the difference is within the state transfer function as shown in Fig~\ref{fig:dp_956}.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.9\columnwidth]{fig/956-ep234.png}
    \caption{Dynamic Programming}
    \label{fig:dp_956}
    \includegraphics[width=0.9\columnwidth]{fig/956_dp_table.png}
    \caption{The dp table: the arrows of different color means different operation. Red: not use current item, copy from the previous state; Blue: put the item on the taller side, the result is not affected; Green: put the item at the shorter side. }
    \label{fig:dp_956_2}
\end{figure}
\begin{lstlisting}[language=Python]
 def tallestBillboard(self, rods):
    maxHeight = sum(rods)
    maxRods = len(rods)
    dp = [float('-inf') for c  in range(maxHeight+1)]  # use difference
    dp[0] = 0
    for i in range(0, maxRods):
        old_dp = dp[:]
        for j in range(0, maxHeight+1-rods[i]):
            if old_dp[j] < 0:
                continue
            # green arrow
            if j >= rods[i]:
                dp[j-rods[i]] = max(dp[j-rods[i]], old_dp[j]+min(rods[i], j))
            else:
                dp[rods[i]-j] = max(dp[rods[i]-j], old_dp[j]+min(rods[i], j))
            # blue arrow
            dp[j+rods[i]] = max(dp[j+rods[i]], old_dp[j])
            # the red line is saved in dp indirectly
    return dp[0]
\end{lstlisting}
\end{examples}
\end{document}


\section[Permutations, binomial coefficients]{Permutations and binomial coefficients continued}\label{permutationsBinomialCoefficient}

The following lecture discusses some more problems in which binomial coefficients arise and will lead us
to the famous principle of exclusion and inclusion.

\subsection{The cab driver problem}

We will begin with the so called 'cab driver problem'. Suppose that we have a city with streets arranged in
a regular grid (like Manhattan) and suppose that our city contains $n$ blocks from north to south and $m$ blocks
from west to east. Now, suppose that we have a cab driver that wants to drive from one point on the grid to another
point, say from $A$ to $B$. In figure \ref{fig:cabdriverprobleminitialexample} the general setup is visualized.

\begin{figure}[ht]
    \centering
    \scalebox{.6}{\incfig{cabdriverprobleminitialexample}}
    \caption{The Cap Driver Problem.}
    \label{fig:cabdriverprobleminitialexample}
\end{figure}
\noindent
\textbf{Question:} In how many ways can we get from $A$ to $B$? We are only allowed to go east or north and we are not allowed to go backwards.
\\
\\
Let $P_{m,n}$ be the number of paths from $A$ to $B$. We notice first, that we can solve this problem by considering a sequence of letters. Let
us denote "going north" by the letter "N" and "going east" by the letter "E", then a path from $A$ to $B$ corresponds to a sequence of N's and E's
of length $m+n$ with $n$ N's and $m$ E's. The total number of such words is equal to the total number of paths from $A$ to $B$.
\\
\\
There is also another solution. Starting from any point in the above grid, there are only two choices: go north or go east. Let $A$ be the starting point
and denote "going north in the first step" by $A'$ and "going east in the second step" by $A''$, then the total number of paths from $A$ to $B$ is equal to
the total number of paths from $A'$ to $B$ plus the total number of paths from $A''$ to $B$. The result is a so called recurrence relation given by the following equation.
$$
P_{A \to B}=P_{A' \to B} + P_{A'' \to B}
$$
In general, if we go one step north the first time, then there are $P_{m, n-1}$ paths left from this position to $B$. Similarly, if we go east first, then there are $P_{m-1, n}$
paths left from this position to $B$. This results in the following relation.
$$
P_{m, n}=P_{m,n-1}+P_{m-1, n}
$$
Given the initial conditions $P_{0,n}=P_{m,0}=1$, and $P_{m,1}=m+1$ and $P_{1,n}=n+1$, our final solution can be found quickly.
$$
P_{m,n}={m + n \choose m}={m+n \choose n}
$$

\subsection{Balls in boxes and multisets}

Suppose that we have $k$ balls and $n$ boxes. In how many ways can we put $k$ balls into $n$ boxes? If the maximal capacity of each box is $1$, the answer would be
$n \choose k$ if $n \geq k$ and $0$ otherwise. What happens if there are no restrictions on the capacity of boxes?
\\
This question leads us to the definition of a multiset. Let $X$ be a set.

\begin{defn}
A multiset is a function $\mu: X \to \mathbb{Z}_{\geq  0}$. The size of a multiset is $\sum_{x \in X} \mu (x)$.
\end{defn}

\begin{exmp}
Assume that we have $5$ boxes, that is $X=\{1, 2, 3, 4 ,5\}$, and $\mu(1)=1, \mu(2)=0, \mu(3)=3$ and $\mu(4)=1$. Compare this with figure \ref{fig:multisetboxesexample}. Notice that box $3$ contains
$3$ balls. The size of $\mu$ is $5$.

\begin{figure}[ht]
    \centering
    \scalebox{0.4}{\incfig{multisetboxesexamples3}}
    \caption{Balls in boxes and mutlisets.}
    \label{fig:multisetboxesexample}
\end{figure}
\end{exmp}

It is easy to see that the number of configurations of $k$ balls into $n$ boxes corresponds to the number of multisets of $\{1,2,\cdots, n\}$
of size $k$. Now, let us interpret our problem in a slightly different way. Imagine $k$ balls and separators (visualized as "$|$") that correspond to a box. Then, the above configuration configuration
looks as follows.
$$
0\ |\ \ |\ 000\ |\ 0
$$
For $4$ boxes, we only need $3$ separators, or in general for $n$ boxes we need $n-1$ separators. Thus, configurations of balls in boxes correspond to configurations of bars and balls.
If we consider balls and bars to be objects, then we have $k+n-1$ objects in total.
\begin{tikzcd}
    & k \text{ balls}  \\
k+n-1 \arrow[ru] \arrow[rd] &                  \\
    & n-1 \text{ bars}
\end{tikzcd}

The above investigations encourage the following theorem.

\begin{theorem}
$k$ balls can be put into $n$ boxes in ${n+k-1 \choose k}$ ways. Equivalently, the number of $k$-element multisets in $\{1, \cdots, n\}$ is ${n+k-1 \choose k}$.
\end{theorem}
\noindent
The formal proof is left as an exercise for the reader.

\subsection{Integer compositions}

We are going to discuss integer compositions in the following section. We start with the formal definition.

\begin{defn}
An integer composition of $n \in \mathbb{Z}_{> 0}$ is a presentation of $n$ as  an ordered sum $n=n_1 + n_2 + \cdots + n_k, n_1,\cdots,n_k \in \mathbb{Z}_{> 0}$.
\end{defn}

\begin{exmp}
For $n=1$ there is only one trivial presentation, that is just $1$ itself. For $n=2$ there is one presentation, namely $2=1+1$. For $n=3$ there are $4$ presentations,
 namely $3=2+1=1+2=1+1+1$. Notice that this sums are ordered, thus $1+2$ and $2+1$ are considered to be different.
 For $n=4$ there are $8$ presentations, namely $4=3+1=1+3=2+2=2+1+1=1+2+1=1+1+2=1+1+1+1$.
\end{exmp}

\begin{theorem} The following statements are true.
\begin{enumerate}[(i)]
    \item The number of compositions of $n$ is $2^{n-1}$.
    \item The number of compositions of $n$ with exactly $k$ summands is ${n-1 \choose k-1}$.
\end{enumerate}
\end{theorem}

\begin{proof}
We start with $(ii)$. The composition of $n$ with exactly $k$ summands corresponds to configurations of $n$ balls
inside $k$ boxes, s.t. each box is nonempty. This is the same as the number of configurations of $n-k$ balls in $k$ boxes,
which is equal to
$$
{n-k+k-1 \choose k-1} = {n-1 \choose k-1}.
$$
We continue with $(i)$. The answer is trivial. The total number of compositions follows from the following equation.
$$
{n-1 \choose 0} + {n-1 \choose 1} + \cdots + {n-1 \choose n-1}=2^{n-1}.
$$
This completes the proof.
\end{proof}

\subsection[PIE]{Principle of inclusion and exclusion}

In the following section we are going to introduce the "Principle of Inclusion and Exclusion (PIE)". Let us introduce this topic
with an example.

\begin{exmp}
Suppose that we have a room full of students. $25$ of them speak spanish, $24$ of them speak french and $8$ of them speak both spanish
and french. How many students speak at least one foreign language? Denote the set of students speaking spanish by $S$ and the
set of students speaking french by $F$. Now, consider the following Venn-diagram.
\begin{figure}[H]
    \centering
    \scalebox{0.6}{\begin{tikzpicture}
        \begin{scope}[shift={(3cm,-5cm)}, fill opacity=0.45]
            \fill[green] \secondcircle;
            \fill[blue] \thirdcircle;
            \draw \secondcircle node [above] {$F$};
            \draw \thirdcircle node [below] {$S$};
        \end{scope}
    \end{tikzpicture}}
    \caption{Venn-diagram for two languages.}
\end{figure}

The number of students that speak at least one language is equal to $|S \cup F|=|S| + |F|-|S \cap F|=25+24-8=41$. Notice that we must substract their intersection
or we count the students speaking both french and spanish twice.\\\\
Now, suppose that $15$ students speak german, $6$ speak german and spanish, $7$ speak german and french and
$4$ speak all three languages. Let us denote the set of students who speak german by $G$, the set of students who speak french by $F$
and the set of students that speak spanish by $S$. Consider the following Venn-diagram.

\begin{figure}[H]
    \centering
    \scalebox{0.6}{\begin{tikzpicture}
        \begin{scope}[shift={(3cm,-5cm)}, fill opacity=0.45]
            \fill[red] \firstcircle;
            \fill[green] \secondcircle;
            \fill[blue] \thirdcircle;
            \draw \firstcircle node[below] {$G$};
            \draw \secondcircle node [above] {$F$};
            \draw \thirdcircle node [below] {$S$};
        \end{scope}
    \end{tikzpicture}}
    \caption{Venn-diagram for three languages.}
\end{figure}
Now the result is very similar to the one before. The number of students that speak at least one language is equal to the following.
$$
|G \cup S \cup F|=|G| + |F| + |S| - |S \cap F| - |S \cap G| - |G \cap F| + |S \cap F \cap G|=25+24+15-8-7-6+4=47.
$$
\end{exmp}

Let us generalize this by the following theorem.

\begin{theorem}(Principle of inclusion-exclusion)
Let $A_1\cdots, A_n$ be finite subsets of a set $X$. Then
\begin{align*}
|A_1 \cup \cdots \cup A_n|&=|A_1|+\cdots+|A_n|\\
&=\sum_{1 \leq i < j \leq n} |A_i \cap A_j|+\sum_{1 \leq i < j \leq n} |A_i \cap A_j \cap A_k| - \cdots + (-1)^{n-1}|A_1 \cap \cdots \cap A_n|.
\end{align*}
\end{theorem}

\begin{proof}
Let $x \in A_1 \cup \cdots \cup A_n$. How many times does $x$ appear on the right-hand-side? Suppose that $x \in A_1 \cap \cdots \cap A_p$ and
$x \notin A_{p+1}, A_{p+2}, \cdots, A_n$. Then $x$ contributes to the right-hand-side with multiplicity
$$
p- {p \choose 2} + {p \choose 3} - {p \choose 4} + \cdots + (-1)^{p-1} {p \choose p}=1+\biggl(-1 +p- {p \choose 2} + {p \choose 3} - \cdots + (-1)^{p-1} {p \choose p} \biggr )=1.
$$
\end{proof}

The standard proof of the above is via induction. This is left as an exercise for the reader.

\subsection{The derangement problem}

In this section we want to apply the principle of inclusion and exclusion to the so called derangement problem. But first, we note that sometimes it turns out that the
problem is symmetric in the following sense that $|A_{i1} \cap \cdots \cap A_{ik}|$ does not depend on $A_{i1}, \cdots, A_{ik}$, but only on $k$, that is $A^k=|A_{i1} \cap \cdots \cap A_{ik}|$.
Then,
$$
|A_1 \cup \cdots \cup A_n|=n \cdot A - {n \choose 2} A^2 + {n \choose 3} A^3 - \cdots + (-1)^{n-1} A^n.
$$

Now, suppose that $n$ people leave their coats in the coatroom, each of them get back a random coat.
What is the probability that everyone will get someone else's coat? Recall that the total number of permutations
(that is the total number of configurations of coats) is given by $n!$. We might ask: How many permutations have
not fixed points, i.e. never map a number into itself?\\
\\
We compute the number of permutations with fixed points. Define
$$
A_i=\{\text{permutations  bringing }i \text{ to } i\}
$$
and
$$
A_{i1} \cap \cdots \cap A_{ik}=\{\text{permutations bringing } i_1 \mapsto i_i, \cdots, i_k \mapsto i_k\}.
$$
Then, $|A_i|=(n-1)!$ and $|A_{i1} \cap \cdots \cap A_{ik}|=A^k=(n-k)!$, hence
\begin{align*}
|A_1 \cup \cdots \cup A_n|&=n(n-1)!- {n \choose 2} (n-2)! + \cdots + (-1)^{n-1} \cdot 0!\\
&=
n! - \frac{n!}{2!}+ \frac{n!}{3!} - \cdots + \frac{(-1)^{n-1}n!}{n!}
\end{align*}
It follows that
$$
n! - |A_1 \cup \cdots \cup A_k|=n!\biggr(1-1 + \frac{1}{2!}- \frac{1}{3!}+ \cdots + \frac{(-1)^n}{n!}\biggl)
$$

Thus, the desired probability follows from the following equations.
\begin{align*}
p(n)&=1-1+\frac{1}{2!}-\frac{1}{3!}+\cdots + \frac{(-1)^n}{n!}\\
e^x&=1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+ \cdots + \frac{x^n}{n!} + \cdots\\
&\implies \lim_{n \to \infty} p(n)=1-1+\frac{1}{2!}-\frac{1}{3!}+ \cdots = e^{-1}=\frac{1}{e}
\end{align*}
where $e=2.71828 \cdots$, thus $\frac{1}{e} \approx 0.37$.
\subsection{Exercises}

\begin{enumerate}
    \item Find the number of lattice paths from $(-2, -2)$ to $(3,3)$ passing through the point $(0,0)$ (each segment of a path goes north or east).
    \item In how many ways is it possible to permute the letters in the word \textit{EUCLID} in such a way that the order of the vowels (\textit{E,U,I}) is unchanged?
    \item Find the number of merry-go-rounds formed by $10$ carriages of two different colours.
    \item Find the number of positive integers not exceeding $300$ and \textbf{not} divisible by any of $2,3,$ and $5$.
\end{enumerate}