\section{Cauchy-Schwarz Inequality}

	\subsection{Theorem}
		\begin{align}
			\forall u, v \in \mathbb{R}^n |u \cdot v| \leq \|u\| \cdot \|v\|
		\end{align}

	\subsection{Proof}

		Let $tu + v$ be a family of vectors where t is in $\mathbb{R}$.

		\begin{align}
			\forall w \in \mathbb{R}^n 0 \leq w \cdot w \\
			0 \leq (tu + v) \cdot (tu + v) \\
			0 \leq tu \cdot v  + 2u \cdot v + v \cdot tu + v \cdot v \\
			\forall a = u \cdot u, b = 2u \cdot v: 0 \leq t^2
		\end{align}

	//TODO CJB REREAD THE FIRST PAGE (having trouble in low light)

\section{Triangle Inequality}

	\subsection{Definition}

		\begin{align}
			\|u+v\| \leq \|u\| + \|v\|
		\end{align}

	\subsection{Proof}

		We show that $\|u+v\|^2 \leq (\|u\| + \|v\|)^2$.

		\begin{align}
			\|u+v\|^2 &= (u+v) \cdot (u+v) = u \cdot u + 2u \cdot v + v \dot v \\
				&= \|u\|^2 + 2u \cdot v + \|v\|^2 \\
				&\leq \|u\|^2 + 2\|u \cdot v \| + \|v\|^2  \\
				&\leq \|u\|^2 + 2\|u|\ \|v\| + \|v\|^2 & \textrm{by Cauchy} \\
				&\leq (\|u\| + \|v\|)^2
		\end{align}

\section{Complex Numbers}

	\subsection{Introduction}

		Complex numbers introduce a unit $i$ such that $i^2 = -1$ and generate a set of numbers called $\mathbb{C}$.
		Usually complex numbers are represented as some combination $a + ib$ where $a, b \in \mathbb{R}$.

		In complex space you can solve any polynomial equation and the number of solution is the degree \footnote{No matter what people told me I found it hard to understand the importance of complex numbers, but it turns out they become really important in numerical analysis.  In stuff like predator-prey models, oscillations in population dynamics are results of complex eigenvalues.  You can determine whether or not the oscillations of such a population are stable by checking if the imaginary part of the complex values are positive or negative.  In black holes, everything spirals into the black hole along a circle, which can only mathematically be described using complex numbers.  It's surprisingly cool... gets a lot more cool with the stability of orbits for satellites.  You want a satellite to have a slightly unstable orbit so that it doesn't take much energy to shift the position of the satellite. -CJB}.
		In complex space //TODO return to 1/0 section.

		A complex number is a pair of real numbers with operations "+" and "*."
		The following demonstrate how complex numbers operate.
		\begin{align}
			(a,b) + (c,d) = (a+c, b+d) \\
			(a,b) * (c,d) = (ac - bd, ad + bc) \\
			(0,1) * (0,1) = (-1,0) = -(1,0) \implies i^2 = 1 \\
			a(1,0) + b(0,1) = (a,b) = a + ib \\
		\end{align}

		The following demonstrates the logic of complex numbers.
		\begin{align}
			(a+ib)(c+id) &=ac + aid + ibc + (ib)(id) \\
					&= ac + i(ad + bc) + i^2bd \\
					&= (ac - bd) + i(ad+bc)
		\end{align}

	\subsection{Definition}

		Formally:
		\begin{align}
			\forall a,b \in \mathbb{R}: z = a+ib \\
			a = realpart[z] = [Re(z)]\\
			b = imaginarypart[z] = [Im(z)]
		\end{align}

\section{Complex Conjugation}

	The complex conjugate of some complex number z is given by $\overline{z} = a - ib$.

	\subsection{Definitions}

		\begin{itemize}
			\item $z = a + ib, \overline{z} = (a - ib)$
			\item $z*\overline{z} = (a+ib)(a-ib) = a^2 + b^2$
			\item $|z| = \sqrt{a^2 + b^2} = sqrt{z*\overline{z}}$
		\end{itemize}

	\subsection{Theorems}

		\begin{itemize}
			\item $\forall z*\overline{z}: z*\overline{z} \in R$
			\item $\forall z*\overline{z}: z*\overline{z} \geq 0$
			\item $\forall z \in \mathbb{C}, z \neq 0: \exists w \in \mathbb{C} \mid zw =1$ \\
					Proof:
					\begin{align}
						&& z = a + ib, w = \dfrac{a-ib}{a^2 + b^2} \\
						zw &= (a + ib)\dfrac{a-ib}{a^2 + b^2} = \dfrac{a^2 - aib + aib - iib^2}{a^2 + b^2} \\
							&= \dfrac{a^2 + b^2}{a^2 + b^2} = 1
					\end{align}
			\item $|z_1z_2| = |z_1||z_2|$\\
					Proof:
					$|z_1z_2| = (z_1z_2)(\overline{z_1z_2}) = z_1z_2\overline{z_1}\overline{z_2} = z_1\overline{z_1}z_2\overline{z_2} = |z_1||z_2|$
		\end{itemize}

	\subsection{Visualisation}

		In the complex plane, complex conjugate is a reflection in the x-axis.
		//TODO generate figures for description of complex conjugate as a reflection of the complex number

\section{Complex Spaces}

	$\mathbb{C}^n$ is the complex n space that consists of n tuples of complex numbers.
	Numbers in $\mathbb{C}^n$ can be added, scaled, and otherwise manipulated much like $\mathbb{R}^n$.

	\subsection{Definitions}

	 	\subsubsection{Dot Product in $\mathbb{C}^n$}

		\begin{align}
			(z_1, \ldots, z_2) \cdot (w_1, \ldots, w_n) = z_1\overline{w_1} + \ldots + z_n\overline{w_n}
		\end{align}

		This agrees with the dot product on $\mathbb{R}^n$, as $\mathbb{R}^n$ is a subset of $\mathbb{C}^n$.  For $\mathbb{R}$, $a + 0i = a - 0i$, so the conjugate makes no difference.

	\subsection{Theorems}

		\subsubsection{$v \cdot v$ is in $\mathbb{R}^n$, and nonnegative; $v \cdot v = 0 \iff v = 0$}
		If $v$ is $\mathbb{C}^n$ then $v \cdot v$ is in $\mathbb{R}^n$, and nonnegative; $v \cdot v = 0 \iff v = 0$
				\begin{align}
					v = (z_1, \ldots, z_2) \\
					v \cdot v = z_1\overline{z_1} + \ldots + z_n\overline{z_n} \\
					z_k = a_k + ib_k \\
					z_k\overline{z_k} = a_k^2 + b_k^2 \\
					v \cdot v = a_1^2 + b_1^2 + \ldots + a_n^2 + b_n^2
				\end{align}

		This implies $v \cdot v$ is a non-negative i and 0 only if it equals 0 and it equals 0 only if $a_k = 0$ and $b_k = 0$ for all k between 1 and n.
		In this case, v is 0.

\section{Matrices as Arrays}

	We can look at matrices as arrays, where a matrix is an array with n rows each with m elements.

	The shape of a matrix is defined by n and m.
	If two matrices have the same shape they can be added, where each element of the matrices are added to the corresponding elements of the other matrix.

	Scaling a matrix consists of multiplying each element by some scalar.

	Multiplying a matrix consists of multiplying each row by a corresponding column in the other matrix.
	It's a little more complicated but presumably people are familiar with this from maths 133.

	The transposed matrix consists of a matrix where each row and column index has been switched.

	The matrix power is a matrix multiplied by itself some $n$ times.

	\subsection{Matrix Multiplication}

		\begin{align}
			(AB)C = A(BC) & \textrm{associative} \\
			A(B+C) = AB + AC & \textrm{distributive} \\
			(B+C)A = BA + CA \\
			AB \neq BA
		\end{align}

	\subsection{Definitions}

			\begin{description}
				\item[square matrix] An n by n matrix.
				\item[trace] The trace of a square matrix $A = [a_{ij}]$ is given by $tr(A) = \sum_{i=1}^{n} a_{ii}$.
				\item[identity matrix] The n by m identity matrix is the $I_n = [a_{ij}]$ where $\forall i = j, a_{ij} = 1 \wedge \forall i \neq j, a_{ij} = 0$
				\item[Matrix Power]
					$\forall square_matrix[A]\ \ A^0 = I \\
					\forall invertible[A]\ \ A^{-1} = inverse[A]$
				\item[Block Matrix] A matrix partitioned in submatrices.
				\item[Linear Equation]  A linear equation with unknowns is an expression of the form $a_1x_1 + \ldots + a_nx_n = b$ where some vector $a_n$ and the value $b$ are in some number system.
				A solution vector $x_n$ exists that satisfies this equation.
			\end{description}

	\subsection{Theorems}

		\subsubsection{$A_{mn}I_n = A,\ \ I_nB_{nm} = B$}

			\begin{align}
				\sum_{k=1}^{n} a_{ik}I_{ik} = a_{ij}
			\end{align}

		\subsubsection{$A_{ik}B_{kj}$}

			\begin{align}
				\forall A_{ik}B_{kj} \\
				AB = [C_{ij}] \\
				C_{ij} = \sum_{k = i}^{n} A_{ik}B_{kj}
			\end{align}

	\subsection{Note on Excluded Material}

		I have not included some theorems on addition and multiplication and the like.
		I also have not typeset notes on how to do proofs.
		The assignments seem to do an excellent job of covering this material.
		Should I find the time (probably will), these will be typeset after the midterm to complete the notes properly.


\subsection{Probability mass functions}

\(P(X=x)=P({\omega |X(\omega)=x})\)

For discrete probability, this is a helpful number. For example for rolling a die.

This is not helpful for continuous probability, where the chance of any specific outcome is \(0\).


\subsection{Lagrange polynomials}\label{subsec:lagrange_polynomials}

\begin{definition}\label{def:omega_polynomial}
  Given distinct elements \( x_0, \ldots, x_n \) of the field \( \BbbK \), we form the polynomial
  \begin{equation*}
    \omega(X) \coloneqq \prod_{k=0}^n (X - x_j).
  \end{equation*}
\end{definition}

\begin{proposition}\label{thm:omega_polynomial_derivative}
  For the polynomial \( \omega \) from \fullref{def:omega_polynomial}, for \( k = 0, \ldots, n \) we have
  \begin{equation*}
    \omega'(x_j) = \prod_{\substack{j = 0 \\ j \neq k}}^n (x_j - x_k),
  \end{equation*}
  where \( \omega' \) is the \hyperref[def:algebraic_derivative]{algebraic derivative} of \( \omega \).
\end{proposition}
\begin{proof}
  Fix \( k \in \{ 0, \ldots, n \} \) and denote
  \begin{equation*}
    q(X) \coloneqq \prod_{\substack{j = 0 \\ j \neq k}}^n (X - x_j).
  \end{equation*}

  Then
  \begin{equation*}
    \omega(X) = (X - x_k) q(X)
  \end{equation*}
  so
  \begin{equation*}
    \omega'(X) = [q(X) + X q'(X)] - x_k q'(X) = q(X) + (X - x_k) q'(X).
  \end{equation*}

  So for \( x_k \) we have
  \begin{equation*}
    \omega'(x_k) = q(x_k) = \prod_{\substack{j = 0 \\ j \neq k}}^n (x_k - x_j).
  \end{equation*}
\end{proof}

\begin{theorem}[Lagrange interpolation]\label{thm:lagrange_interpolation}
  Let \( x_0, x_1, \ldots, x_n \) be pairwise distinct elements of \( \BbbK \) and let \( y_0, y_1, \ldots, y_n \) be arbitrary elements of \( \BbbK \). Then there exists a unique \hyperref[def:polynomial_algebra]{polynomial} \( L(X) \) of degree at most \( n \) such that \( L(x_k) = y_k \) for \( k = 1, \ldots, n \).
\end{theorem}
\begin{proof}
  \SubProof{Proof of uniqueness} Suppose that \( p, q \) are polynomials of degree at most \( n \) that both satisfy \( L(x_k) = y_k \) for \( k = 1, \ldots, n \). Their difference \( p - q \) is a polynomial of degree at most \( n \) that has \( n + 1 \) roots. By \fullref{thm:def:integral_domain/root_limit}, \( p - q = 0 \).

  \SubProof{Proof of existence} We will construct the polynomial explicitly. Define the Lagrange basis polynomial
  \begin{equation*}
    L(X) = \sum_{m=0}^n y_m \prod_{\substack{j = 0 \\ j \neq m}}^n \frac {(X - x_j)} {(x_m - x_j)}.
  \end{equation*}

  For \( k = 0, 1, \ldots, n \) we have
  \begin{equation*}
    L(x_k) = y_k \underbrace{\prod_{\substack{j = 0 \\ j \neq k}}^n \frac {(x_k - x_j)} {(x_k - x_j)}}_{1} + \sum_{\substack{m = 0 \\ m \neq k}}^n y_m \overbrace{\frac{(x_k - x_m)}{(x_k - x_m)}}^{0} \prod_{\substack{j = 0 \\ j \neq k \\ j \neq m}}^n \frac {(x_k - x_j)} {(x_m - x_j)} = y_k.
  \end{equation*}

  Therefore, \( L \) is the desired polynomial.
\end{proof}

\begin{theorem}[Finite field Lagrange interpolation]\label{thm:finite_field_lagrange_interpolation}\mcite{MathOF:functions_over_finite_fields}
  For a multivariate function \( f: \BbbF_q^n \to \BbbF_q \) over the \hyperref[thm:finite_fields]{finite field} \( \BbbF_q \), there exists a unique multivariate polynomial \( L(X_1, \ldots, X_n) \) such that
  \begin{itemize}
    \item For any sequence of values \( x_1, \ldots, x_n \),
    \begin{equation*}
      f(x_1, \ldots, x_n) = L(x_1, \ldots, x_n).
    \end{equation*}

    \item For every monomial \( X_1^{\gamma_n} \cdots X_n^{\gamma_n} \) of \( L \), \( \gamma_i < q \) for \( i = 1, \ldots, n \).
  \end{itemize}
\end{theorem}
\begin{proof}
  For any point \( (x_1, \ldots, x_n) \in \BbbZ_q^n \), the characteristic polynomial
  \begin{equation*}
    c(X_1, \ldots, X_n) \coloneqq \prod_{i=0}^n \parens*{ \prod_{\substack{m=0 \\ m \neq x_i}}^{q - 1} \frac {X_i - m} {x_i - m} }
  \end{equation*}
  satisfies
  \begin{equation*}
    c(y_1, \ldots, y_n) = \begin{cases}
      1, &x_i = y_i \T{for all} i = 1, \ldots, n \\
      0, &\T{otherwise.}
    \end{cases}
  \end{equation*}

  As in \fullref{thm:lagrange_interpolation}, give us the desired polynomial, a linear combination of these basis polynomials with coefficients corresponding to the values of \( f \) give us the desired polynomial.
\end{proof}


\subsection{Sesquilinear forms}

\subsubsection{Bilinear form recap}

A bilinear form takes two vectors and produces a scalar from the underyling field.

The function is linear in addition in both arguments.

\(\phi (au+x, bv+y)=\phi (au,bv)+\phi (au,y)+\phi (x,bv)+\phi (x,y)\)

The function is also linear in multiplication in both arguments.

\(\phi (au+x, bv+y)=ab\phi (u,v)+a\phi (u,y)+b\phi (x,v)+\phi (x,y)\)

They can be represented as:

\(\phi (u,v)=v^TMu\)

\subsubsection{Sesquilinear forms}

Like bilinear forms, sesquilinear are linear in addition:

\(\phi (au+x, bv+y)=\phi (au,bv)+\phi (au,y)+\phi (x,bv)+\phi (x,y)\)

Sesqulinear forms however are only multiplictively linear in the second argument.

\(\phi (au+x, bv+y)=b\phi (au,v)+\phi (au,y)+b\phi (x,v)+\phi (x,y)\)

In the first argument they are "twisted"

\(\phi (au+x, bv+y)=\bar ab\phi (u,v)+\bar a\phi (u,y)+b\phi (x,v)+\phi (x,y)\)

\subsubsection{The real field}

For the real field, \(\bar b = b\) and so the sesqulinear form is the same as the bilinear form.

\subsubsection{Representing sesquilinear forms}

We can show the sesquilinear form as \(v^*Mu\)

\subsubsection{Stuff}

\(f(M)=f([v_1,v_2])\)

We introduce \(e_i\), the element vector. This is \(0\) for all entries except for \(i\) where it is \(1\). Any vector can be shown as a sum of these vectors multiplied by a scalar.

\(f(M)=f([\sum^m_{i=1}a_{1i}e_i,\sum^m_{i=1}a_{2i}e_i])\)

\(f(M)=\sum_{k=1}^mf([a_{1k}e_k,\sum^m_{i=1}a_{2i}e_i])\)

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}f([a_{1k}e_k,a_{2i}e_i])\)

Because this in linear in scalars:

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}^*a_{2i}f([e_k,e_i])\)

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}^*a_{2i}e_k^*Me_i\)

\subsubsection{Orthonormal basis and \(M=I\)}

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}^*a_{2i}e_k^*Me_i\)

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}^*a_{2i}e_k^*e_i\)

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}^*a_{2i}\delta_i^k\)

\(f(M)=\sum^m_{i=1}a_{1i}^*a_{2i}\)


\section{Deterministic Annealing
\hfill\normalfont\sffamily (\textit{\rmfamily Z} is tractable)}

\emph{Lemma:}\enspace
func's $\times$ domain $\to$ domain $\times$ co-dom.\\
\quad $\textcolor{gray}{\mathcal O(K^N) \rightarrow\,}
\highlight*{\sum_c \prod_i \epsilon_{i, \color{red} c(i)} = \prod_i \sum_k \epsilon_{ik}}
\textcolor{gray}{\,\leftarrow \mathcal O(NK)}$

$p(c\mid \theta, X) = \prod_{i\leq N} p_i(c(i) \mid \theta,X)$\\
\quad where\enskip $p_i(k\mid \theta, X) \propto \exp(-\frac1T \norm{x_i - \theta_k}^2)$
% $\phantom{p(c\mid \theta, X)} = \prod_{i\leq N} \frac{\exp(-\frac1T \norm{\bm x_i - \theta_{c(i)}}^2)}{\sum_{j\leq K} \exp(-\frac1T \norm{\bm x_i - \theta_j}^2)}$
% $\substack{~\\[10pt]\textcolor{gray}{\leftarrow Z(X)}}$

Max. entr.
$\implies \pderiv{\log Z}{\theta_k} = 0
\implies \highlight*{\theta_k^\ast = \frac{ \sum_i p_i(k\mid \theta^\ast, X) \:\cdot\: \bm x_i }{ \sum_i p_i(k\mid \theta^\ast, X) }}$

\begin{highlightbox}
\begin{itemize}
    \item[] do
    \begin{itemize}
        \item[]
            \vspace{-5pt}
            \textbf{E-step:\;} $p_i(k\vert \theta\ap{old}, X) = \frac{\exp\paren*{ -\frac1T \norm{\bm x_i - \theta_k}^2 }}{\sum_{j\leq K} \exp\paren*{ -\frac1T \norm{\bm x_i - \theta_j }^2}}$
            \vspace{-5pt}
        \item[] \textbf{M-step:} $\theta_k \leftarrow \ldots$
        \item[] $\theta\ap{old} \leftarrow \theta$
    \end{itemize}
    \item[] until convergence of $\theta$
    \item[] $\theta_k \leftarrow \theta_k + \epsilon$
        {\small\enspace\color{gray} (noise s.t. centroids can separate)}
\end{itemize}
\end{highlightbox}

\emph{Phase transitions:}\enspace
For $T {\to} \infty$ :\enspace $\theta_k^\ast = \overline{\bm X} \enspace \forall k\leq K$\\
Once $T=2\lambda\ped{max}$, more centroids appear, where $\lambda\ped{max} = \textrm{max. eigenvalue}$ of $\frac1N \bm X^\top \bm X$.
\enspace($\bm x_i$'s row-wise)


\subsection{Slant Asymptotes}\label{subsec:SlantAsymptotes}
Some functions may have slant (or {\it oblique}) asymptotes, which are neither vertical nor horizontal. If
\[\ds\lim_{x\to\infty}\left[f(x)-(mx+b)\right]=0\]
then the straight line $y=mx+b$ is a \dfont{slant asymptote} to $f(x)$. Visually, the vertical distance between $f(x)$ and $y=mx+b$ is decreasing towards 0 and the curves do not intersect or cross at any point as $x$ approaches infinity. Similarly when $x\to-\infty$.

\begin{example}{Slant Asymptote in a Rational Function}{SlantAsymptRationalFunction}
Find the slant asymptotes of $\ds f(x)=\frac{-3x^2+4}{x-1}$.
\end{example}
\begin{solution}
Note that this function has no horizontal asymptotes since $f(x)\to-\infty$ as $x\to\infty$ and $f(x)\to\infty$ as $x\to-\infty$.

In rational functions, slant asymptotes occur when the degree in the numerator is one greater than in the denominator. We use long division to rearrange the function:
\[\ds\frac{-3x^2+4}{x-1}=-3x-3+\frac{1}{x-1}.\]
The part we're interested in is the resulting polynomial $-3x-3$. This is the line $y=mx+b$ we were seeking, where $m=-3$ and $b=-3$. Notice that
\[\ds\lim_{x\to\infty}\frac{-3x^2+4}{x-1}-(-3x-3)=\lim_{x\to\infty}\frac{1}{x-1}=0\]
and
\[\ds\lim_{x\to-\infty}\frac{-3x^2+4}{x-1}-(-3x-3)=\lim_{x\to-\infty}\frac{1}{x-1}=0.\]
Thus, $y=-3x-3$ is a slant asymptote of $f(x)$.
\end{solution}

Although rational functions are the most common type of function we encounter with slant asymptotes, there are other types of functions we can consider that present an interesting challenge.

\begin{example}{Slant Asymptote}{SlantOtherFunction}
Show that $y=2x+4$ is a slant asymptote of $f(x)=2x-3^x+4$.
\end{example}
\begin{solution}
This is because
\[\lim_{x\to -\infty}[f(x)-(2x+4)]=\lim_{x\to -\infty}(-3^x)=0.\]
\end{solution}

We note that $\lim_{x\to \infty}[f(x)-(2x+4)]=\lim_{x\to \infty}(-3^x)=-\infty$.
So, the vertical distance between $%
y=f(x)$ and the line $y=2x+4$ decreases toward 0 only when $%
x\to -\infty $ and not when $x\to \infty$. The graph of $f$
approaches the slant asymptote $y=2x+4$ only at the far left and not at the
far right. One might ask if $y=f(x)$ approaches a slant
asymptote when $x\to \infty$. The answer turns out to be no, but we
will need to know something about the relative growth rates of the
exponential functions and linear functions in order to prove this.
Specifically, one can prove that when the base is greater than 1 the
exponential functions grows faster than any power function as $x\to
\infty$. This can be phrased like this: For any $a>1$ and any $n>0$,%
\begin{equation*}
\lim_{x\to \infty}\frac{a^x}{x^n}=\infty \text{ and }%
\lim_{x\to \infty}\frac{x^n}{a^x}=0.
\end{equation*}%
These facts are most easily proved with the aim of something called the L'H\^{o}pital's Rule.


\subsection{Independence and conditional independence}

\subsubsection{Independence}

\(x\) is independent of \(y\) if:

\(\forall x_i \in x,\forall y_j \in y (P(x_i|y_j)=P(x_i)\)

If \(P(x_i|y_j)=P(x_i)\) then:

\(P(x_i\land y_j)=P(x_i).P(y_j)\)

This logic extends beyond just two events. If the events are independent then:

\(P(x_i\land y_j \land z_j)=P(x_i).P(y_j \land z_k)=P(x_i).P(y_j).P(z_k)\)

Note that because:

\(P(x_i|y_j)=\dfrac{P(x_i\land y_j)}{P(y_j)}\)

If two variables are independent

\(P(x_i|y_j)=\dfrac{P(x_i)P(y_j)}{P(y_j)}\)

\(P(x_i|y_j)=P(x_i)\)

\subsubsection{Conditional independence}

\(P(A\land B|X)=P(A|X)P(B|X)\)

This is the same as:

\(P(A|B\land X)=P(A|X)\)


\documentclass[a4paper]{article}

\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage[colorinlistoftodos]{todonotes}

\title{MATH 542 Homework 3}
\author{Saket Choudhary\\skchoudh@usc.edu}

\begin{document}
\maketitle
\section*{Problem 1}

Given: $A_{n\times n}$ is idempotent $\implies AA=A$; $P$ is non-singular and $C_{n\times n}$ is orthogonal $\implies CC^T=C^TC=I$

\subsection*{1.a}
\begin{align*}
(I-A)(I-A) = I-A-A+AA\\
&= I-2A+A\ \text{ using } AA=A\\
&=I-A
\end{align*}

Hence $I-A$ is idempotent

\subsection*{1.b}
\begin{align*}
A(I-A) &= A-AA\\
&= A-A\ \text{ using } AA=A\\
&=0
\end{align*}

Similarly,
\begin{align*}
(I-A)A &= A-AA\\
&= A-A\ \text{ using } AA=A\\
&=0
\end{align*}

\subsection*{1.c}
\begin{align*}
(P^{-1}AP)(P^{-1}AP)  &=P^{-1}APP^{-1}AP\\
&= P^{-1}AAP \ \text{since }PP^{-1}=I\\
&= P^{-1}AP\ \text{ using } AA=A\\
\end{align*}
Hence $P^{-1}AP$ is idempotent

\subsection*{1.d}
\begin{align*}
(C'AC)(C'AC) &=C'ACC'AC\\
&= C'AAC \text{ since } CC'=C'C=I\\
&= C'AC \ \text{ using } AA=A\\
\end{align*}

\subsection*{1.e}
$A$ is a projection matrix and $C'C=C'C=I$
For $C'AC$ to be a projection matrix $C'ACz=z\ \forall z \in S$ for some vector space $S$

For some $z \in S$:
\begin{align*}
C'ACz &= C'Ay \text{ where } y=Az\\
&= C'Ay\\
&= C'y\text{ since $A$ is projection matrix }  \\
&= C'Cz\\
&= z\ \text{ since } C'C=I
\end{align*}

Thus, $C'ACz \in S$ and $C'ACz=z$ and hence $C'AC$ is a projection matrix

\section*{Problem 2}
\subsection*{2.a}

\begin{align*}
Q(x_1,x_2,x_3) &= 12x_1^2+3x_2^2+3x_3^2+2x_1x_2-10x_1x_3+4x_2x_3\\
&= 3(2x_1+x_2+x_3)^2-(12x_1x_2+6x_2x_3+12x_1x_3)+2x_1x_2-10x_1x_3+4x_2x_3\\
&= 3(2x_1+x_2+x_3)^2-10x_1x_2-2x_2x_3-22x_1x_3\\
&= (x_1+x_2)^2+ 2(x_2+x_3)^2+(5x_1-x_3)^2-14x_1^2\\
&=(x_1+x_2)^2+2(x_2+x_3)^2+x_3^2+11x_1^2-10x_1x_3\\
&= (x_1+x_2)^2+2(x_2+x_3)^2+25(x_1-x_3/5)^2-14x_1^2
\end{align*}

and hence $Q(x_1,x_2,x_3)$ is positive defnite, (the coefficient of the negative term(-$x_1^2$) is less than the positive coefficient of $x_1^2$
\subsection*{2.b}
\begin{align*}
x'Ax=\begin{pmatrix}x_1 & x_2 &x_3\end{pmatrix}\begin{pmatrix}
2 & 1 & 2\\
1 & 2 & 1\\
2 & 1 & 4\\
\end{pmatrix}\begin{pmatrix} x_1\\x_2\\x_3
\end{pmatrix}\\
&= (2x_1+x_2+x_3)x_1+(x_1+2x_2+x_3)x_2+(2x_1+x_2+4x_3)x_3\\
&= 2x_1^2+2x_2^2+4x_3^2+2x_1x_2+3x_1x_3+2x_2x_3\\
&= (x_1+x_2)^2+(x_2+x_3)^2+(x_1+\frac{3}{2}x_2)^2+\frac{3}{4}x_2^2\\
> 0
\end{align*}

$A$ is positive definite

\subsection*{2.c}

\begin{align*}
x'Ax=\begin{pmatrix}x_1 & x_2 &x_3\end{pmatrix}\begin{pmatrix}
1 & 2 & 3\\
2 & 1 & 1\\
3 & 1 & -2\\
\end{pmatrix}\begin{pmatrix} x_1\\x_2\\x_3
\end{pmatrix}\\
&= (x_1+2x_2+3x_3)x_1+(2x_1+x_2+x_3)x_2+(3x_1+x_2-2x_3)x_3\\
&= x_1^2+x_2^2-2x_3^2+4x_1x_2+6x_1x_3+2x_2x_3\\
&= (x_2+x_3)^2+3(x_1+x_3)^2+2(x_1+x_2)^2-6x_1^2-6x_3^2-4x_1^2-2x_2^2-6x_3^2\\
\end{align*}

$A$ is neither positive definite nor positive semidefinite.

\section*{Problem 3}

\begin{align*}
\begin{pmatrix}
a & b\\
c & d\\
\end{pmatrix}\times \begin{pmatrix}
a & b\\
c & d\\
\end{pmatrix} &= \begin{pmatrix}
2 & -1\\
-1 & 2
\end{pmatrix}\\
\begin{pmatrix}
a^2+b^2 & ac+bd\\
ac+bd & c^2+d^2
\end{pmatrix}&=\begin{pmatrix}
2 & -1\\
-1 & 2
\end{pmatrix}\\
\end{align*}

\begin{align*}
a^2+b^2 &=2\\
ac+bd&=-1\\
c^2+d^2&=2
\end{align*}

Let $a=1,b=1$ $\implies \ c+d=1 $

\begin{align*}
c^2+d^2 &=2\\
c^2+(-1-c)^2 & =2\\
2c^2+2c-1 &=0\\
c &= \frac{-2\pm\sqrt{4+8}}{4}\\
c &= \frac{-1\pm\sqrt{3}}{2}\\
d &= \frac{-1\pm\sqrt{3}}{2}
\end{align*}

and hence one possible $B$ is:
\begin{align*}
B &=\begin{pmatrix}
1 & 1\\
\frac{-1-\sqrt{3}}{2} & \frac{-1+\sqrt{3}}{2}\\
\end{pmatrix}
\end{align*}

\section*{Problem 4}
$G=(X'X)^-$

Let $Y=(X'X)$  to prove $GYG$ is symmetric we need to show $(GYG)^T=GYG$

NOTE: $Y^T=(X^TX)^T=X^TX=Y$

Using $YGY=Y$, we get:
\begin{align*}
YGY &= Y\\
YGYG &= YG\\
\implies GYG=G
\end{align*}

Now, again consider $YGY$ and take transpose on both sides:

\begin{align*}
(YGY)^T &= Y^T\\
Y^TG^TY^T &= Y^T\\
YG^TY &= Y \text{ using } Y^T=Y\\
\implies G&=G^T \ \text{ since }YG^TY=YGY=Y \text{Also see NOTE below}
\end{align*}

Now consider $GYG$

\begin{align*}
(GYG)^T &= G^TY^TG^T\\
&= GYG \ \text{ using } G=G^T and Y=Y^T
\end{align*}
NOTE:
\begin{enumerate}
\item
\end{enumerate}
However, this case is only true for the case when $G=G^T$ i.e. the generalized inverse in unique. I could not prove it for the general case

To show $GYG$ is a generalized inverse of $Y$ we need to show $YGYGY = Y$

\begin{align*}
Y(GYG)Y &= YG(YGY)\\
&= YGY\ \text{ using } YGY=Y, \text{ G is generalised inverse of Y}\\
&= Y\ \text{ using definition of generalised inverse for } G
\end{align*}
$\implies$ GYG is generalised inverse of $Y=X^TX$

\section*{Problem 5}

Given: $E[X]=1$ and $Var(X)=E[X^2]-E[X]^2 = 5$
Using $E[X]=1$ we get, $E[X^2]=5+E[X]^2=6$

\subsection*{5.a}

\begin{align*}
E[(1+2X)^2] &= E[1+4X+4X^2]\\
&= E[1]+E[4X]+E[4X^2]\\
&= 1+4E[X]+4E[X^2]\\
&= 1+4+4(6)\\
&= 30
\end{align*}

\subsection*{5.b}
\begin{align*}
var(3+4X) &= Var(3) + Var(4X) + 2Cov(3,4X)\\
&= 0 + 16Var(X) + 2(0)\\
&= 80
\end{align*}

\end{document}


\subsection{Perceptron (step function)}

\subsubsection{The function}

If the sum is above \(0\), \(a(z)=1\). Otherwise, \(a(z)=0\).

\subsubsection{The derivative}

This has a differential of \(0\) at all point except \(0\), where it is undefined.

\subsubsection{Notes}

This function is not smooth.

These is the activation function used in the perceptron.

Perceptron data needs to be linearly separable to train.

Even if linearly separable, doesn't necessarily get the best outcome?

\subsection{Perceptron}

Perceptron: one node neural network. is one or zero depednign if weightd inputs enough. therefore is classiication

If error, update weights

Only works if linearly separable. ie can draw linear line completely separting all inputs

Neural network has more layers

Works if data is linear

How to treat node inputs: raw, sigmoid, {0,1}

For all of these want the cost function have only one solution, like least squares doe. not guaranteed for all

For logistic, want to make it convex. loss = -log(f(x)) or -log(1-f(x)) depending on correct y. this is convex

How to create node inputs: sigmoid, binary cutoff


\section{Matrix Multiplication}

Given two matrices $\mathbf{A}\in\mathbb{R}^{m \times p }$ and $\mathbf{B}\in\mathbb{R}^{p \times n}$ with entries $a_{i,j}$ and $b_{i,j}$ in their $i$th row and $j$th column, the product $\mathbf{C} \in \mathbf{AB}\in\mathbb{R}^{m \times n}$ and with elements $c_{i,j} = \sum_{k = 1}^{p} a_{i,k}b_{k,j}$.

There are two ways to interpret how the rows or columns of $\mathbf{C}$ come about through this operation. The first is that a row or column vector in $\mathbf{C}$ is formed by one of the matrices acting on an individual vector from the other matrix. The second is that a row of a column vector in $\mathbf{C}$ is formed as the linear superposition of the vectors within one of the matrices, where the coefficients of the superposition are given by a row or a column within the other matrix.

Let:

\begin{equation}
\mathbf{A} = \left[
\begin{array}{ccc}
\vert & \vert & \vert \\
a^{(c)}_1 & a^{(c)}_2 & a^{(c)}_3\\
\vert & \vert & \vert \\
\end{array}
\right] =
\left[
\begin{array}{ccc}
- & a^{(r)}_1 & - \\
- & a^{(r)}_2 & - \\
- & a^{(r)}_3 & - \\
\end{array}
\right]
\end{equation}

\begin{equation}
\mathbf{B} = \left[
\begin{array}{ccc}
\vert & \vert & \vert \\
b^{(c)}_1 & b^{(c)}_2 & b^{(c)}_3\\
\vert & \vert & \vert \\
\end{array}
\right] =
\left[
\begin{array}{ccc}
- & b^{(r)}_1 & - \\
- & b^{(r)}_2 & - \\
- & b^{(r)}_3 & - \\
\end{array}
\right]
\end{equation}

\begin{equation}
\mathbf{C} = \left[
\begin{array}{ccc}
\vert & \vert & \vert \\
c^{(c)}_1 & c^{(c)}_2 & c^{(c)}_3\\
\vert & \vert & \vert \\
\end{array}
\right] =\left[
\begin{array}{ccc}
- & c^{(r)}_1 & - \\
- & c^{(r)}_2 & - \\
- & c^{(r)}_3 & - \\
\end{array}
\right]
\end{equation}

be the notation in terms of row and column vectors.

\subsubsection{Elementwise}
$[\mathbf{C}]_{i,j}$ is the dot product of the $i$th row vector of $\mathbf{A}$ and the $j$ column vector of $\mathbf{B}$.

\begin{equation}
c_{i,j} = \left[\begin{array}{c} \vert \\ a_i^{(r)}\\ \vert \end{array}\right] \cdot \left[\begin{array}{ccc} - &b_j^{(c)}& - \end{array}\right]
\end{equation}

\subsubsection{Columns of $\mathbf{C}$}

The $j$th column of $\mathbf{C}$ involves all of $\mathbf{A}$ and only the $j$th column of $\mathbf{B}$.

The $j$th column of $\mathbf{C}$ is the matrix product of $\mathbf{A}$ with the first column of $\mathbf{B}$.

\begin{equation}
\left[
\begin{array}{c}
\vert\\
c^{(c)}_j\\
\vert
\end{array}
\right]=
\left[
\begin{array}{ccc}
- & a^{(r)}_1 & - \\
- & a^{(r)}_2 & - \\
- & a^{(r)}_3 & - \\
\end{array}
\right]
\left[ \begin{array}{c} \vert \\ b^{(c)}_j  \\ \vert \end{array}\right]
\end{equation}

The $j$th column of the product $\mathbf{C}$ is a linear superposition of the columns of $\mathbf{A}$, with the coefficients given by the $j$th column of $\mathbf{B}$.

\begin{equation}
\left[
\begin{array}{c}
\vert\\
c^{(c)}_j\\
\vert
\end{array}
\right]=
\begin{array}{ccc}
\left[\begin{array}{c} \vert\\ a^{(c)}_1 \\ \vert \end{array}\right]b^{(c)}_{1,j}
+ \left[\begin{array}{c} \vert\\ a^{(c)}_2 \\ \vert \end{array}\right]b^{(c)}_{2,j}
+ \left[\begin{array}{c} \vert\\ a^{(c)}_3 \\ \vert \end{array}\right]b^{(c)}_{3,j}
\end{array}
\end{equation}

\subsubsection{Rows of $\mathbf{C}$}
The $i$th row of $\mathbf{C}$ involves all of $\mathbf{B}$ and only the $i$th row of $\mathbf{A}$.

The $i$th row of $\mathbf{C}$ is the matrix product of $\mathbf{B}$ acting backwards on the $i$th row of $\mathbf{A}$.

\begin{equation}
\left[\begin{array}{ccc} - & c^{(r)}_i & - \end{array}\right] =
\left[\begin{array}{ccc} - & a^{(r)}_i & - \end{array}\right]
\left[
\begin{array}{ccc}
\vert & \vert & \vert \\
b^{(c)}_1 & b^{(c)}_2 & b^{(c)}_3\\
\vert & \vert & \vert \\
\end{array}
\right]
\end{equation}

The $i$th row of $\mathbf{C}$ is a linear superposition of the rows of $\mathbf{B}$, with the coefficients given by the $i$th row of $\mathbf{A}$.

\begin{equation}
\begin{array}{rl}
\left[\begin{array}{ccc} - & c^{(r)}_i & - \end{array}\right] = & a^{(r)}_{i,1} \left[\begin{array}{ccc} - & b^{(r)}_1 & - \end{array}\right]\\
&+ a^{(r)}_{i,2} \left[\begin{array}{ccc} - & b^{(r)}_2 & - \end{array}\right]\\
&+ a^{(r)}_{i,3} \left[\begin{array}{ccc} - & b^{(r)}_3 & - \end{array}\right]
\end{array}
\end{equation}


\chapter{Hopfield Network}
Hopfield networks are recurrent neural networks with symmetric feedback connections. There are no self-feedback connections

\section{Energy minimizing Networks}
Output activation vector:
$$\mathbf{v} = (v_1 v_2 \ldots v_n)^{T}$$
The network energy is critical and network output is optimum or reaches a stable condition when:
$$Energy\ gradient, \nabla E(v) = 0$$
Hessian matrix is defined as:
$$H = \nabla^{2} E(\mathbf{v})$$
If $H$ is \emph{positive-definite}, then the energy is a \emph{minimum}.

\section{Analysis of Hopfield Model}
\begin{figure}[!h]
\centering
\includegraphics{chapter6_1}
\end{figure}
The total synaptic input:
\begin{equation}
\begin{split}
u_i &= \sum_{j=1,j\ne i}^{n} w_{ij} v_j + x_i -t_i \\
&= \mathbf{w^{T} v} + x_i - t_i
\end{split}
\label{hopfield_input}
\end{equation}
Rewritten in matrix notations:
$$\mathbf{u = W v + x - t}$$
Where matrix $\mathbf{W}$ is called the connectivity matrix:
$$ \mathbf{W}
\begin{bmatrix}
0 & w_{12} & \cdots & w_{1n} \\
w_{21} & 0 & \cdots & w_{2n} \\
\vdots & \vdots & \ddots & \vdots \\
w_{n1} & w_{n2} & \cdots & 0
\end{bmatrix}
$$
\begin{center}where $w_{ij} = w_{ji}$ and $w_{ii} = 0$ \end{center}
The unipolar Hopfield processing unit:
$$v_{i} =
\begin{cases}
    0 & u_i \le 0 \\
    1 & u_i > 0
\end{cases}
$$
The bipolar Hopfield processing unit:
$$v_{i} =
\begin{cases}
    -1 & u_i \le 0 \\
    1 & u_i > 0
\end{cases}
$$

\section{Updating Activation of Neurons}
Activity of each neuron of the Hopfield network is usually updated in an asynchronous manner. \\
The energy of node $i$ is defined as:
\begin{equation*}
\begin{split}
E_i &= -v_i u_i \\
&= - v_i \Big(\sum_{j} w_{ij} v_j + x_i - t_i \Big) \\
E &= -\frac{1}{2} \sum_{i=1}^{n} \sum_{j=1, i\ne j}^{n} w_{ji} v_{i} v_{j} - \sum_{i=1}^{n} x_i v_i + \sum_{i=1}^{n} t_i v_i \\
&= -\frac{1}{2} \mathbf{v^{T} W v - v^{T} x + v^{T}t} \\
\nabla E(\mathbf{v}) &= \frac{\partial E(\mathbf{v})}{\partial \mathbf{v}} \\
&= \mathbf{- Wv - x + t} \\
\Delta^{2} E(\mathbf{v}) &= - W
\end{split}
\end{equation*}

\section{Convergence Analysis}
If there is no change in its state, then:
$$\Delta E_i = 0$$
If there is a change in its state, then
\begin{equation}
\begin{split}
\Delta E_i &= E_i^{new} - E_i^{old} \\
&= - (v_i^{new} - v_i^{old}) u_i \\
&= - \Delta v_i u_i
\end{split}
\label{change_of_energy}
\end{equation}
If $v_i$ change from 0 to 1, then:
$$\Delta v_i = +1\ and\ u_i > 0$$
Substituting to \ref{change_of_energy} yields $\Delta E_i < 0$ \\ \\
If $v_i$ change from 1 to 0, then:
$$\Delta v_i = -1\ and\ u_i \le 0$$
Substituting to \ref{change_of_energy} yields $\Delta E_i < 0$

\section{Hopfield Model Design}

\subsection{Solution by calculation}
By using \ref{hopfield_input} and unipolar Hopfield processing unit, you will get several inequalities. From these inequalities, select one possible solutions. \\
\textit{Note: One of the problems in designing Hopfield network is false energy wells}

\subsection{Solution by learning}
Let the set of $P$ training patterns be:
$${\mathbf{s}(1), \mathbf{s}(2), \ldots, \mathbf{s}(P)}$$
$$\mathbf{s}(i) = s_1(p), s_2(p), \ldots , s_n(p)$$
Training is done using \emph{Hebbian learning rule}:
$$w_{ij} = \sum_{p=1}^{P} (2s_i(p) - 1)(2s_j(p) - 1)$$
\begin{center}for unipolar patterns \end{center}

$$w_{ij} = \sum_{p=1}^{P} s_i(p) s_j(p)$$
\begin{center}for bipolar patterns \end{center}

\noindent \textit{Note: you still need to use inequalities to find the threshold $t$}


%!TEX root = uber-driver-strategy.tex
\section{Problem Setup}
Consider a finite horizon Markov decision process with finite decision horizon $T = \{0,1,2, ..., N-1\}$. At each stage, the driver occupies a state $i \in \mathcal{X}$, where $n = |\mathcal{X}|$ is finite, and a driver is allowed to choose an action $a$ from finite set of allowable actions $\mathcal{A} = \{a_1, a_2, ..., a_m\}$. The system starts in a given initial state $i_0$. The states make Markov transitions according to a collection of time-dependent transition matrices $\tau := (P^{a}_t)_{a \in \mathcal{A}, t \in \mathcal{T}}$, the $n \times n$ transition matrix $P^{a}_t$ contains the probabilities of transitioning under action $a$ at stage $t$. Let $\pi = (\textbf{a}_0, ..., \textbf{a}_{N-1})$ denote a policy, where $\textbf{a}_t(i)$ denotes the action of the driver while at state $i$ at time $t$. Define by $r_t(i,a)$, the reward corresponding to state $i$ and action $a$ at time $t$, and by $r_N$ the reward at the final stage (end of time). We assume that $r_t(i,a)$ is non-negative and finite (negative cost of take empty ride action can be accounted for by considering all the rewards relative to the highest negative cost of empty ride i.e., moving the zero to the left.)

For a given set of transition matrices $\tau$, if $R_N(\pi, \tau)$ denotes the \textit{expected total rewards} under the policy $\pi$ and transition matrices $\tau$, then,
\begin{eqnarray*}
R_N(\pi, \tau) &:=& \textbf{E}\bigg(\sum_{t=0}^{N-1} r_t(i_t,\textbf{a}_t(i)) + r_N(i_N)\bigg)
\end{eqnarray*}
When the set of transition matrices $\tau$ is known,  $R_N(\pi, \tau)$ maximising policy $\pi$ can be found via dynamic programming algorithm, with a complexity of $O(mnN)$. The \textit{nominal} problem giving the corresponding maximum expected total reward is,
\begin{eqnarray*}
\phi_N(\Pi, \tau) &:=& \max_{\pi \in \Pi} R_N(\pi, \tau),
\end{eqnarray*}
where $\Pi = \mathcal{A}^{nN}$ is the entire policy space.

In our case, the transition matrix under each action $a$ and time $t$ is known to lie in some given subset $\mathcal{P}^{a}_t$, where $\mathcal{P}^{a}_t$ can be thought of as \textit{sets of confidence} for the transition matrices. Given this uncertainty in the transition matrices, we are interested in maximising the lower bound on the expected total rewards for the driver. This leads to a game between the driver and \textit{nature}, where the driver seeks to maximise the minimum expected reward, with nature being the minimizing player. Formally, \textit{policy of nature} refers to specific collection of time-dependent transition matrices $\tau = (P^{a}_t)_{a \in \mathcal{A}, t \in \mathcal{T}}$ chosen by nature, and the set of admissible policies of nature is $\mathcal{T} := (\otimes_{a \in \mathcal{A}} \mathcal{P}^{a}_t)^N$. The maximum lower bound can be given by,
\begin{eqnarray}
\phi_N(\Pi, \mathcal{T}) &:=& \max_{\pi \in \Pi} \min_{\tau \in \mathcal{T}} R_N(\pi, \tau)
\end{eqnarray}

\section{Robust Finite Horizon Dynamic programming} \label{section:robust-dynamic-programming}

For a given state $i \in \mathcal{X}$, action $a \in \mathcal{A}$ and $P^{a}_t \in \mathcal{P}^{a}_t$, we denote by $p^{a}_{i,t}$ the next-state distribution drawn from $P^{a}_t$ corresponding to state $i \in \mathcal{X}$, the $i$-th row of the matrix. Furthermore, we define $\mathcal{P}^{a}_{i,t}$ as the projection of the set $\mathcal{P}^{a}_{t}$ onto the set of $p^{a}_{i,t}$ variables. Using the robust dynamic programming theorem from \cite{nilim2003robustness}, we can show that perfect duality holds, i.e.,
\begin{eqnarray}
\phi_N(\Pi, \mathcal{T}) = \max_{\pi \in \Pi} \min_{\tau \in \mathcal{T}} R_N(\pi, \tau) = \min_{\tau \in \mathcal{T}} \max_{\pi \in \Pi} R_N(\pi, \tau) = \psi_N(\Pi, \mathcal{T})
\end{eqnarray}
The problem can be solved via recursion. The worst case optimal value function in state $i$ at time $t$, $v_t(i)$ is given by,
\begin{eqnarray}
v_t(i) &=& \max_{a \in \mathcal{A}} \bigg(r_t(i,a) + \sigma_{\mathcal{P}^{a}_{i,t}}(v_{t+1})\bigg), i \in \mathcal{X}, t \in T
\end{eqnarray}
where, $\sigma_{\mathcal{P}(v)} = \inf\{p^Tv : p \in \mathcal{P}\}$. It should be noted in that solving $\sigma_{\mathcal{P}(v)} = \inf\{p^Tv : p \in \mathcal{P}\}$ is the inner problem in every step of the above recursion. A corresponding optimal driver policy $\pi^* = (\textbf{a}^*_0, ..., \textbf{a}^*_{N-1})$ is obtained by,
\begin{eqnarray}
\textbf{a}^*_t(i) &=& \arg \max_{a \in \mathcal{A}} \bigg\{r_t(i,a) + \sigma_{\mathcal{P}^{a}_{i,t}}(v_{t+1})\bigg\}, i \in \mathcal{X}
\end{eqnarray}
Furthermore, the effect of uncertainty on some other policy $\pi = (\textbf{a}_0, ..., \textbf{a}_{N-1})$ can be evaluated as follows,
\begin{eqnarray}
v^{\pi}_t(i) &=& r_t(i,\textbf{a}_t(i)) + \sigma_{\mathcal{P}^{\textbf{a}_t(i)}_{i,t}}(v^{\pi}_{t+1}), i \in \mathcal{X}
\end{eqnarray}

\section{Likelihood Model} \label{section:likelihood}
We use a likelihood constraint to describe the uncertainty in the transition matrix for the action $a$ (when driver takes a passenger ride). Let $F^{a}_t$ denote the matrix of observed frequencies of transitions in our data (in time slice centered around time $t$), and let $f^{a}_{i,t}$ be its $i$-th row. We have $F^{a}_t \geq 0$ and $F^{a}_t\textbf{1} = \textbf{1}$, where \textbf{1} denotes the vector of ones. For simplicity, let us assume that $F^{a}_t > 0$, which is same as assuming $f^{a}_{i,t} > 0$, $f^{a}_{i,t} \in \textbf{R}^n_+$ and $(f^{a}_{i,t})^T\textbf{1} = 1$. To simplify the notation in this section, we shall drop the subscript $t$ denoting the time slice as we estimate the likelihood models independently for each time slice. Furthermore, we shall also drop the superscript $a$ denoting action, because there is uncertainty associated with only one of the actions in our model i.e, driver taking a passenger ride. \hc{The other category of actions is `taking empty ride to some other state'. There is no uncertainty involved in the transition matrix coresponding to this action, as exactly one value in every row is exactly 1. Another type of action in flexible work schedule involves driver going offline in the system. Again, there is either no transition in this action, or taking a definite transition to home node of the driver.}Plug-in estimates for the transition matrix $P$ with uncertainty level $\beta$ is given by,
\begin{eqnarray*}
\bigg\{P \in \textbf{R}^{n \times n}: P \geq 0, P\textbf{1} = \textbf{1}, \sum_{i,j}F(i,j)\log P(i,j) \geq \beta \bigg\}
\end{eqnarray*}
where $\beta < \beta_{max} = \sum_{i,j}F(i,j)\log F(i,j)$. From \cite{lehmann2006testing}, for a $(1-U_L)$ level of confidence, $\beta$ is solution of the equation,
\begin{eqnarray*}
(1 - U_L) &=& F_{\mathcal{X}^{2}_{n(n-1)}}(2(\beta_{max} - \beta))
\end{eqnarray*}
where $F_{\mathcal{X}^{2}_{d}}$ is the cumulative density function of the $\mathcal{X}^2$-distribution with $d$ degrees of freedom. In the inner problem, we only need to work with uncertainty on each row $p_{i}$. Due to separable nature of the log-likelihood function, the projection of the above set onto the $p_{i}$ variables of the matrix $P$ can be given as,
\begin{eqnarray*}
\mathcal{P}_{i}(\beta_i) &=& \bigg\{p \in \Delta^n: \sum_{j}f_{i}(j)\log p_{i}(j) \geq \beta_i \bigg\}
\end{eqnarray*}
where,
\begin{eqnarray*}
\beta_i &=& \beta - \sum_{k \neq i} \sum_{j} F(k,j) \log F(k,j). \\
\Delta^n &=& \{p \in \textbf{R}^n_+ : p^T\textbf{1} = 1\}
\end{eqnarray*}
In the following section, as we deal with each row of the matrix separately, we drop the subscript $i$ in the observed frequencies $f_i$ and in the lower bound $\beta_i$. Hence $\beta_{max} = \sum_{j} f(j)\log f(j)$. We assume that $\beta < \beta_{max}$ along with $f > 0$. \hc{Unsure what happens if we relax this constraint to $f \geq 0$, which would be more appropriate in our cases, as during some time slices there no transitions to/from airport zone of the city.} Without loss of generality, we can also assume that $v \in \textbf{R}^n_+$.


\section{Prob 4. b)}

The linear growth factor can be calculated indirectly from
\begin{equation}\label{eq:1}
    \dot D(t) = \frac{dD}{da} \dot a,
\end{equation}
where $\dot a(z) = H(z)a(z)$. Plug in $\dot a(z)$ and rewrite the equation with the chain rule, \cref{eq:1} then becomes
\begin{equation}\label{eq:2}
    \dot D(z) = \frac{dD}{da} H(z) a(z) = \frac{dD}{da} \frac{H(z)}{1+z} = \frac{dD}{dz} \frac{dz}{da} \frac{H(z)}{1+z},
\end{equation}
where $a(z) = \frac{1}{1+z}$. Take the derivative of $a$ with respect of $z$, we have
\begin{equation}
    \frac{da}{dz} = - \frac{1}{(1+z)^2} \quad
    \Rightarrow \quad \frac{dz}{da} = - (1+z)^2
\end{equation}
Thus, we can simplify \cref{eq:2} to
\begin{equation}\label{eq:4}
    \dot D(z) = - \frac{dD}{dz} (1+z) H(z).
\end{equation}
From part a) we get the linear growth factor and the Hubble parameter as a function,
\begin{gather}
    D(z) = \frac{5 \Omega_m {H_0}^2 }{2} H(z) \int_{z}^{\infty} \frac{1+z'}{H^3(z')} dz' \\
    H(z) = (\Omega_m (1+z)^3 + \Omega_{\Lambda})^{1/2}.
\end{gather}
Their derivatives with respect to $z$ are then,
\begin{gather}
    \label{eq:7}
    \frac{dD}{dz} = \frac{5 \Omega_m {H_0}^2 }{2} [\frac{dH(z)}{dz} \int_{z}^{\infty} \frac{1+z'}{H^3(z')} dz' + H(z) \frac{d}{dz} \int_{z}^{\infty} \frac{1+z'}{H^3(z')} dz'] \\
    \label{eq:8}
    \frac{dH}{dz} = \frac{3 \Omega_m (1+z)^2}{2[\Omega_m (1+z)^3 + \Omega_\Lambda]^{1/2}}.
\end{gather}
Substitute \cref{eq:7} and \cref{eq:8} into \cref{eq:4} and simplify, we get
\begin{equation}\label{eq:9}
\begin{split}
    \dot D(z) &= - \frac{5 \Omega_m {H_0}^2 }{2} [\frac{dH(z)}{dz} \int_{z}^{\infty} \frac{1+z'}{H^3(z')} dz' + H(z) \frac{d}{dz} \int_{z}^{\infty} \frac{1+z'}{H^3(z')} dz'] (1+z) H(z) \\
    &= - \frac{5 \Omega_m {H_0}^2 }{2}  (\frac{3 \Omega_m (1+z)^2}{2[\Omega_m (1+z)^3 + \Omega_\Lambda]^{1/2}} \int_{z}^{\infty} \frac{1+z'}{H^3(z')} dz' + \\
    &\qquad \qquad \qquad \frac{(1+z)^2}{[\Omega_m (1+z)^3 + \Omega_\Lambda]^{1/2}}) \cdot (1+z) [\Omega_m (1+z)^3 + \Omega_\Lambda]^{1/2} \\
    &= - \frac{5 \Omega_m {H_0}^2 }{2} (\frac{3 \Omega_m (1+z)^3}{2} \int_{z}^{\infty} \frac{1+z'}{H^3(z')} dz' + \frac{(1+z)^2}{[\Omega_m (1+z)^3 + \Omega_\Lambda]^{1/2}}).
\end{split}
\end{equation}
\cref{eq:9} is then what we need for the time derivative of the linear growth factor $D$ as a function of the redshift $z$.


\section{Biogeochemical models \label{sec:bio-intro}}

In this module, biogeochemical models are implemented,
in a two-way coupled mode.

\subsection{Mathematical formulation}\label{sec:bio-math}

The general structure of a biogeochemical model with
$I$ state variables expressed as ensemble averaged
concentrations is given by the
following set of equations:
\begin{equation}\label{FullSystem}
  \partial_t c_i + \partial_z
\left(m_i c_i - K_V \partial_z c_i\right)
=   P_i(\vec{c}) -D_i(\vec{c}), \;\; i = 1,\ldots,I, \quad j,k = 1,\ldots,3,
\end{equation}
with $c_i$ denoting the concentrations of state variables.
Furthermore, $m_i$ represents the autonomous motion of the ecosystem
component $c_i$ (e.g. sinking or active swimming),
and $K_V$ represents the eddy diffusivity.
The source and sink terms of the ecosystem component
$c_i$ are summarised in $P_i(\vec{c})$ and $D_i(\vec{c})$, respectively.
For three-dimensional models, advection with the flow field
and horizontal advection would have to be accounted for additionally.
In many biogeochemical models, some of the state variables have positive
lower limits. In order to account for this,
we defined all state variables as the difference between
the actual value and their lower limit, such that
(for non-negative state variables only)
the model value $c_i$ represents a concentration of
$c_i+c_i^{\min}$ where $c_i^{\min}$ is the lower limit of $c_i$.

The gradient term on the left hand side of (\ref{FullSystem})
is the total transport,
for which typically surface and bottom boundary conditions
\begin{equation}
K_V\partial_z c_i\big|_{z=\eta} = F^s_i,\qquad
K_V\partial_z c_i\big|_{z=-H} = -F^b_i,
\end{equation}
with surface and bottom fluxes, $F^s_i$ and $F^b_i$, respectively,
are applied.
The right hand side denotes the reaction terms,
which are composed of contributions
$d_{i,j}(\vec{c})$, which represent reactive fluxes from
$c_i$ to $c_j$, and in turn, $p_{i,j}(\vec{c})$ are reactive fluxes from
$c_j$ received by $c_i$:
\begin{equation}\label{eq:am:a}
P_i(\vec{c}) = \sum^I_{j=1} p_{i,j}(\vec{c}), \;\;\;\;
D_i(\vec{c}) = \sum^I_{j=1} d_{i,j}(\vec{c}),
\end{equation}
with $d_{i,j}\geq 0$ for all $i,j$ and $p_{i,j}\geq 0$
for all $i\not= j$.

We basically consider two types of ecosystem models.
In the simple NPZ (nutrient-phytoplankton-zooplankton) type models
all state variables are based on the same measurable unit
such as [mmol N m$^{-3}$] for nitrogen-based models.
In such NPZ models the reactive terms do only exchange mass
between state variables with
\begin{equation}\label{eq:am:symmetry}
p_{i,j}(\vec{c}) = d_{j,i}(\vec{c}), \quad \mbox{ for } i \not= j
\quad\mbox{ and }\quad
p_{i,i}(\vec{c}) = d_{i,i}(\vec{c})=0, \quad \mbox{ for } i = j.
\end{equation}
Neglecting for a moment all transport terms, it is easily seen that this
simple type of
model is conserving mass:
\begin{equation}
\begin{array}{l}
\displaystyle
d_t\left(\sum_{i=1}^I c_i    \right) =
\sum_{i=1}^I\left( P_i(\vec{c})- D_i(\vec{c})\right) =
\displaystyle
\sum_{i=1}^I\sum_{j=1}^I\left(p_{i,j}(\vec{c}) - d_{i,j}(\vec{c})\right)
= \sum_{i=1}^I\left(p_{i,i}(\vec{c}) - d_{i,i}(\vec{c})\right)=0.
\end{array}
\end{equation}
The NPZD model (see section \ref{sec:bio-npzd}) and the
\cite{Fashametal1990} model discussed in section \ref{sec:bio-fasham}
are such fully conservative models.

In many biogeochemical models most state variables are known to be positive
or at least non-negative quantities. For non-negative initial conditions
$c_i(0) \geq 0$ one can easily show by a simple
contradiction argument that the condition
\begin{equation}\label{eq:am:d1}
d_{i,j}(\vec{c}) \longrightarrow 0 \;\; \mbox{for} \;\; c_i \longrightarrow 0
\end{equation}
guarantees that the quantities
$c_i(t) \geq 0$, remain non-negative for all $t$.
A typical example is $d_{i,j}(\vec{c}) = f(\vec{c}) c_i$ with a non-negative,
bounded
function $f$ which might depend on all $c_i$.

However, for many applications such simple models are
too restrictive. Often different spatial references
are involved for the state variables, such as the
detritus concentration in the water column, measured in [mmol N m$^{-3}$]
and the fluff layer concentration at the bed, measured in [mmol N m$^{-2}$].
Many biogeochemical processes
involve more than two substances such as the photosynthesis
where different nutrients (e.g.\ nitrate and phosphorus) are taken
up by phytoplankta and oxygen is produced. The ratios between these
substances dissipated or produced are usually fixed, in the example
of photosynthesis uptake of 16 mmol m$^{-3}$ nitrate is
connected to an uptake of 1 mmol m$^{-3}$ phosphorus
and a production of 8.125 mmol m$^{-3}$ oxygen.

For state variables which may
be negative (such as oxygen concentration
which also includes oxygen demand units,
all sink and source
terms are added up in the production terms
$p_{i,j}$, with a negative sign for the sink terms.
For the \cite{Neumannetal2002} model discussed in sections
\ref{sec:bio-iow},
further deviations from the conservation formulation are introduced since
biogeochemical reactions include substances which are not budgeted by the
model (mostly because they are assumed to be not limiting).
One typical example is nitrogen fixated by blue-green algae which
builds up biomass by using atmospheric nitrogen which is later recycled to
nitrate. Such non-conservative terms are lumped into the diagonal terms
$p_{i,i}$ and $d_{i,i}$.

\subsection{Numerical aspects}\label{sec:bio-num}

Two basic aspects which are included in the mathematical formulation for
the biogeochemical equations discussed in section \ref{sec:bio-math}
are to be reproduced by the numerical methods applied: conservation and
positivity. Another constraint for the choice
of numerical methods is that they should be sufficiently stable and accurate.
In order to facilitate this, a split method is applied
separating the numerical solution of the transport part
(advection, diffusion) and the reaction part.
By doing so, we take splitting errors into account which should however
be not significant as long as the typical reaction time scales are
much longer than the constant model time step $\Delta t$.

In the transport step
in which the right hand side is set to zero,
finite volume discretisations
are used such that conservation of mass is guaranteed.
The spatial discretisation is carried out by separating the water column
into $N$ not necessarily equidistant intervals of height $h_k$.
The state variables, represented by layer-averaged values,
are located in the centres of these intervals, the advective and diffusive
fluxes are located at the interfaces in between.
The transport step itself is subject to operator splitting.
The autonomous motion of the state variables (including sinking or
rising due to negative or positive buoyancy, respectively)
is discretised by means of
TVD (Total Variation
Diminishing) advection schemes, for which several choices are
available, see \cite{Pietrzak98}.
These TVD schemes are positivity conserving due to their TVD
property.
The most accurate among those
schemes is the so-called
PDM-limited P$_2$ scheme which has been described in detail
by \cite{Leonhard91}.

For the diffusion, a
central in space scheme is used which is slightly biased towards
a backward in time scheme in order to avoid asymptotic instability
(see \cite{Samarskij84}). By doing so, positivity is obtained and the schemes
are practically second order in time and space.

With the discretisations of the transport terms given above,
accuracy, positivity and conservation of the state variables are
guaranteed by means of standard schemes. For the reaction terms,
\cite{Burchardetal2003b} recently developed schemes which also fulfil these
requirements.
Due to the operator split between transport and reaction terms,
only ordinary differential equations (ODEs)
have to be treated numerically for the latter terms.
For the case of conservative biogeochemical models with
$p_{i,i}=d_{i,i}=0$, these schemes are identical to those given
by \cite{Burchardetal2003b}.
For $p_{i,i}\not= 0$, some modifications are necessary.
Three classes of schemes are considered:
Explicit schemes such as the Euler-forward scheme and second- and
fourth-order Runge-Kutta schemes (see section \ref{sec:ode-solver}).
These schemes are known to be conservative, but for
sufficiently large time steps they may compute negative values
of state variables also for non-negative state variables.
This may be avoided by small time stepping, which however usually
leads to an enormous increase of the computational effort such that
these schemes lose their practical relevance in this context.
In order to solve this problem,
\cite{Patankar80} had suggested the first-order in time
positive definite scheme
(\ref{eq:am:patankar}), and \cite{Burchardetal2003b} have extended this
to second order, see (\ref{eq:am:PRK}).
However, these schemes are not conservative, since
source and sink terms are numerically treated in a different way.
Fully conservative and non-negative schemes in first- and second-oder
in time have thus been suggested and tested for
ordinary differential equations by \cite{Burchardetal2003b},
with $p_{i,i}=d_{i,i}=0$ in equations.\ (\ref{eq:am:MP}) and (\ref{eq:am:MPRK})
in section \ref{sec:ode-solver}.
This equal numerical treatment of sources and sinks
results in implicit linear systems of equations.
Since only ordinary differential equations are to be solved in each
grid point, these systems have small dimensions, for example $I=7$ for
the \cite{Fashametal1990} model (see section \ref{sec:bio-fasham}) and
$I=10$ for the \cite{Neumannetal2002} model
(see section \ref{sec:bio-iow}).
Thus, these linear systems may be directly solved
by Gaussian elimination schemes.
Nevertheless, one can also employ iterative methods.
Especially for the linear system arising in the context of the present
type of equations it is proven in \cite{Burchardetal2003b}
that the involved matrix is always non-singular and the
standard Jacobi-type method converge to the unique solution of the system.
Later, \cite{Bruggemanetal2005} found that the Modified Patankar
schemes as described in equations (\ref{eq:am:MP}) and (\ref{eq:am:MPRK})
are only conservative for systems with one model currency
(e.g.\ nitrogen in the model of \cite{Fashametal1990}),
but do not conserve stoichiometric ratios, when several
limiting nutrients are present. To solve that problem,
\cite{Bruggemanetal2005} developed first- and second-order
Extended Modified Patankar (EMP) schemes, which are
stoiciometrically conservative and explicit, such that they do
not need to solve implicit systems of linear equations.

\subsection{Computational aspects}\label{sec:bio-comp}

The computational structure of the coupled physical-biogeochemical
model system implemented here
has been designed under consideration of various objectives.
In this section
a description of the various design related decisions is given.
The major objectives are:
\begin{itemize}
  \item to provide a well-defined interface between the one-dimensional
   physical model and a sufficiently generic biogeochemical model,
  \item to allow for easy extensions of the system with new
     biogeochemical models without changing
  the over-all structure, such as the \cite{Fashametal1990} model
  (see section \ref{sec:bio-fasham}) and the \cite{Neumannetal2002} model,
  see section \ref{sec:bio-iow}),
  \item to provide a number of solution methods for the 'process part' of the
  biogeochemical model, i.e.\ solvers for ordinary differential equations
  as discussed in section \ref{sec:bio-num} and section \ref{sec:ode-solver},
  \item to obtain fast and efficient execution of the coupled model,
  \item to design the system in such a way that three-dimensional
   models can easily be interfaced with it.
\end{itemize}
For the biogeochemical model,
we have adopted the same design strategy as has been used for the turbulence
module of GOTM. The interface between an application using the turbulence
module consists of two subroutine calls only: \emph{init\_turbulence()} and
\emph{do\_turbulence()}. The subroutine \emph{init\_turbulence()} is
responsible for initialising the parameters of the turbulence module and
called as part of the
initialisation of the entire model. For the turbulence module the
initialisation includes reading \emph{namelists} with the model configuration,
allocating memory for all necessary variables and initialising
these variables with sensible values. \emph{init\_turbulence()} should only be
called once during program execution and after this call all public and private
variables of the turbulence module should be in a
consistently initialised state.
During the time integration of the model, \emph{do\_turbulence()} has to be
called at each time step. It is called with a number
of parameters to transfer information e.g.\ from the mean-flow module
to the turbulence module but also to receive the variables updated
by the turbulence module.

Using the same strategy for the biogeochemical module
has some problematic implications which are described here.
In analogy to the turbulence
module, the interface is given via the two subroutines \emph{init\_bio()} and
\emph{do\_bio()}.
The major difference between the turbulence module and the
biogeochemical module
in terms of implementation
is that in the former the number of variables are known at compilation time
and the dimensions are specified at run time where as in the latter both
the number of state variables and their dimension are known only at run time.
The general interface has to be able to handle not only the different
biogeochemical models implemented at present but also to provide a framework
for developing future models. There are two major
items to address: 1.\ how to initialise the biogeochemical model and 2.\ how to
select the right biogeochemical model during the time integration and use the
selected ordinary differential equation (ODE) solver.

To solve the initialisation problem we have chosen a two-level initialisation
approach.  At the first level variables not specific to any of the
biogeochemical
models are initialised. The single most important variable during this
phase is \emph{bio\_model}, which contains the identification number for all
implemented biogeochemical models.
Depending on the value of \emph{bio\_model}, the second level
of initialisation is started. At this level all model specific variables
(such as process rates) are initialised.
The most important variable at the second
level is $I$ (number of state variables).
After this step, the system returns to the first level, and
now all information is available for allocating memory and initialising
all variables. The most important data structure provided to the individual
biogeochemical models will briefly be mentioned here. $c_{i,k}$
with $1\leq i\leq I$ and $1\leq k\leq N$ (number of vertical layers)
is a two-dimensional array containing the concentrations of each
variable at each depth. $I$ is
supplied by the individual biogeochemical model and $N$
is transferred in the call to \emph{init\_bio()} from the physical model.

After the initialisation, all variables are initialised in a common
data structure where the only link to the specific model is via
\emph{bio\_model} and $I$. The next step is to design the actual
time integration in such a way that selected biogeochemical model
operates on the
common data structure using the selected ODE solver in a transparent way.

\begin{figure}
\begin{center}
\scalebox{0.5}{\includegraphics{figures/structure_gotmbio.eps}}
\caption{
The structure of the \emph{do\_bio()} subroutine. This subroutine is responsible
for updating all variables in the biogeochemical model in question at each time-step.
\emph{do\_bio()} essentially works as a wrapper around all biogeochemical models
implemented.
The hatched arrows from \emph{process model} to \emph{ODE solvers}
indicate that between one and four calls
of  \emph{process model}
per time step are performed, depending on the order of the chosen ODE solver.
}\label{fig_bio_code}
\end{center}
\end{figure}

Figure \ref{fig_bio_code} shows a sketch of how this is organised in the
model source code.
The sketch should be be read from left to right. At the left side we have
the interface \emph{do\_bio()}, which is the only connection to the
calling program. The next level shows a sequence of steps necessary to do the
time integration. It should be noted that not all biogeochemical
models necessarily
have to execute all the steps, some models do e.g.\
not need any surface fluxes or short-wave radiation.
For the diffusion/advection part a general subroutine is called
which is also used by
the physical model.
After having calculated $I_{PAR}$ and $B$ (see eqs.\ (\ref{Iz}) and
(\ref{B}),
the next step is the step at which the production and destruction terms
($p_{i,j,k}$ and $d_{i,j,k}$)
of the biogeochemical models are calculated. This is done via a call to
\emph{ode\_solver()}. After the call to \emph{ode\_solver()}, $c_{i,k}$
has been updated with the new values of all variables in the
biogeochemical model. Which ODE solver to use is determined during the
initialisation phase (\emph{ode\_method}).
It should be noted
that for some of the solution methods the biogeochemical processes have to
be evaluated more than once. Instead of having \emph{ode\_solver()} being
responsible for calling the chosen biogeochemical model,
an additional subroutine
has been introduced: \emph{process\_model}, is a simple
wrapper routine calling the selected biogeochemical model.

The implementation of this biogeochemical module into three-dimensional models
is straight-forward. The 3D model has to take care of storing all
three-dimensional state variables and calculate their advection with the
mean flow and the horizontal diffusion.
Settling, migration, vertical diffusion and the production/destruction
processes are calculated by the biogeochemical module which has to be
called by means of a loop over all horizontal grid boxes of the 3D
model.

This text has been adapted from \cite{Burchardetal05}.


\subsection{Positive-definite matrices}

The matrix \(M\) is positive definite if for all non-zero vectors the scalar is positive.

\(v^TMv\)

We know that the outcome is a scalar, so:

\(v^TMv=(v^TMv)^T\)

\(v^TMv=v^TM^Tv\)

\(v^T(M-M^T)v=0\)


\section{Central Limit Theorem}%
\label{sec:central_limit_theorem}

\begin{thm}
    If $X_1 \ldots X_n$ iid, $\E X_i = \xi$, $\Var X_i = \sigma^2 < \infty$, then
    \begin{equation*}
        \sqrt{n} (\bar{X} - \xi) \convl N(0, \sigma^2).
    \end{equation*}
\end{thm}

\begin{thm}[Berry-Esseen]
    If $X_1 \ldots X_n$ iid $\sim F$ with finite 3rd moment, then $\exists C$
    that is independent of $F$ such that for all $x$,
    \begin{equation*}
        |G_n(x) - \Phi| \le \frac{C}{\sqrt{n}} \frac{\E |X_i - \xi|^3}{\sigma^3}
    \end{equation*}
    where $G_n(x)$ is the CDF of $\sqrt{n} (\bar{X} - \xi) / \sigma$.
\end{thm}

\begin{corollary}
    Under the assumptions of Berry-Esseen's theorem,
    \begin{equation*}
        G_n(x) \to \Phi(x) \text{ as } n \to \infty
    \end{equation*}
    for any sequence $F_n$ with mean $\xi_n$ and $\sigma_n^2$ for which
    \begin{equation*}
        \frac{E_n |X_1 - \xi_n|^3}{\sigma_n^3} = o(\sqrt{n})
    \end{equation*}
    and in particular if it is bounded.
\end{corollary}


\section{Stochastic Boolean satisfiability}
\label{sect:background-ssat}

An SSAT formula $\Qf$ over variables $\{x_1,\ldots,x_n\}$ has the form:
$Q_1x_1,\ldots,Q_nx_n.\pf$,
where each $Q_i\in\{\exists,\random{p}\}$ and $\pf$ is quantifier-free.
Symbol $\exists$ denotes an existential quantifier with its conventional semantics.
Symbol $\random{p}$ denotes a randomized quantifier~\cite{Papadimitriou1985}, which requires
the quantified variable to evaluate to $\top$ with probability $p\in[0,1]$.
Given an SSAT formula $\Qf$, the quantification structure $Q_1 x_1, \ldots, Q_n x_n$ is called the \emph{prefix},
and the quantifier-free Boolean formula $\phi$ is called the \emph{matrix}.

Let $x$ be the outermost variable in the prefix of an SSAT formula $\Qf$.
The satisfying probability of $\Qf$, denoted by $\spb{\Qf}$, is defined by the following four rules:
\begin{enumerate}
    \item[a)] $\spb{\top}=1$,
    \item[b)] $\spb{\bot}=0$,
    \item[c)] $\spb{\Qf}=\max\{\spb{\ncf{\Qf}{x}},\spb{\pcf{\Qf}{x}}\}$, if $x$ is existentially quantified,
    \item[d)] $\spb{\Qf}=(1-p)\spb{\ncf{\Qf}{x}}+p\spb{\pcf{\Qf}{x}}$, if $x$ is randomly quantified by $\random{p}$,
\end{enumerate}
where $\ncf{\Qf}{x}$ and $\pcf{\Qf}{x}$ denote the SSAT formulas obtained by eliminating the outermost quantifier of $x$ via substituting the value of $x$ in the matrix with $\bot$ and $\top$, respectively.

The \textit{decision version} of SSAT is stated as follows.
Given an SSAT formula $\Qf$ and a threshold $\theta\in[0,1]$, decide whether $\spb{\Qf}\geq\theta$.
On the other hand, the \textit{optimization version} asks to compute the exact value of $\spb{\Qf}$.
The decision version of SSAT is PSPACE-complete~\cite{Papadimitriou1985}.

An SSAT formula can also be interpreted from a game-theoretical viewpoint.
The randomized quantifiers represent the nondeterministic factors in a stochastic game.
The existential quantifiers model the moves of an agent who plays under such uncertainty.
The satisfying probability of the SSAT formula corresponds to the maximum winning probability of the agent.
A \textit{Skolem function} for an existentially quantified variable is the agent's strategy to assign this variable.
Note that the Skolem function for a variable can only depend on its preceding variables in the prefix.
A set of optimal Skolem functions achieves the maximum winning probability.

\begin{example}
    Consider an SSAT formula $\Qf$:
    \begin{align*}
        \random{0.5}x_1,\exists y_1,\random{0.5}x_2,\exists y_2.
        (x_1\lor\lnot y_1)(\lnot x_1\lor y_1)
        (\lnot x_1\lor\lnot x_2\lor y_2)(x_1\lor\lnot y_2)(x_2\lor\lnot y_2).
    \end{align*}
    According to the computational rules for the satisfying probability of SSAT, we have $\spb{\Qf}=1$.
    The maximum winning probability can be achieved by assigning $y_1$ to $f_1(x_1)=x_1$ and $y_2$ to $f_2(x_1,x_2)=x_1\land x_2$.
    The set of functions $\{f_1,f_2\}$ is a set of optimal Skolem functions for $\Qf$.
\end{example}


\chapter{Stochastic Differential Equations and Training Algorithms}
\input{6DL/Bayesian}
\section{Simulated Annealing}
Given a function $\phi(x)$, we want to find the global minimum of this funcion, that is, we need to find $x^*=argmin(\phi(x))$
\subsection{Theorem}
For stochastic process $X^T_t$ which satisfies a stochastic differential equation
$$dX_t^T=-\frac{1}{\gamma}\nabla\phi(X_t^T)dt+\frac{\sigma}{\gamma} dW_t$$
where $T=\frac{\sigma^2}{2\gamma}$ and $\sigma$, $\gamma$ are constant.
~\\We have
$$\mathop{lim}\limits_{t\rightarrow\infty, T\rightarrow 0}E(X^T_t)=x^*$$
where $x^*=argmin (\phi(x))$
\subsection{Proof}For a stochastic differential equation
$$dX_t=A(X_t,t)dt+\sigma(X_t,t)dW_t \eqno{(1)}$$
By Kolmogorov forward equation, we have the transition probability $P(x,t;x',0)$ satisfies
$$\frac{\partial}{\partial t}P(x,t;x',0)=L_t^{+}P(x,t;x',0) \eqno{(2)}$$
where $L_t=A(X_t,t)\frac{\partial}{\partial x}+\frac{1}{2}\sigma^2(X_t,t)\frac{\partial^2}{\partial x^2}$ and $L_t^+$ is the adjoint of $L_t$.
~\\By the definition of probability density function w.r.t. transition probability:
$$f(x,t)=\int P(x,t;x',0)P(x')dx' \eqno{(3)}$$
where $P(x')$ is the density function at $t=0$.
~\\Combine equation (3) and (2), by simple computation, we get Fokker-Planck equation:
$$\frac{\partial}{\partial t}f(x,t)=L_t^{+}f(x,t) \eqno{(4)}$$
Now we set $A(x,t)=-\frac{1}{\gamma}\nabla\phi(x)$ and $\sigma(x,t)=\frac{\sigma}{\gamma}$ (constant)
~\\Then the Fokker-Planck equation [or equation (4)] becomes:
$$\frac{\partial f}{\partial t}=\frac{\partial}{\partial x}[\frac{1}{\gamma}\nabla\phi(x)\cdot f]+\frac{1}{2}\frac{\partial^2}{\partial x^2}(\frac{\sigma^2}{\gamma^2} f) \eqno(5)$$
When it comes to a steady state, that is, the density function would not change according to time:
$$\frac{\partial f}{\partial t}=0 \eqno(6)$$
(In physics, we often set $t\rightarrow \infty$ to represent the steady state)
~\\Combine equation (5) and (6), we get
$$\frac{\partial}{\partial x}[\frac{1}{\gamma}\nabla\phi(x)\cdot f]+\frac{1}{2}\frac{\partial^2}{\partial x^2}(\frac{\sigma^2}{\gamma^2} f)=0 \eqno(7)$$
Solve this ODE, we have:
$$f=\frac{e^{-\frac{2\gamma\phi(x)}{\sigma^2}}}{\int e^{-\frac{2\gamma\phi(x)}{\sigma^2}}dx} \eqno(8)$$
We define $T=\frac{\sigma^2}{2\gamma}$ as the temperature. We call this steady state distribution equilibrium distribution at temperature T. So just by changing variables, we rewrite equation (8):
$$f^T_{eq}=\frac{e^{-\frac{\phi(x)}{T}}}{\int e^{-\frac{\phi(x)}{T}}dx} \eqno(9)$$
We denote $X_t^T$ as the stochastic process which satisfies
$$dX_t^T=-\frac{1}{\gamma}\nabla\phi(X_t^T)dt+\frac{\sigma}{\gamma} dW_t \eqno(10)$$
where $T=\frac{\sigma^2}{2\gamma}$ [Because we have set $A(x,t)=-\frac{1}{\gamma}\nabla\phi(x)$ and $\sigma(x,t)=\frac{\sigma}{\gamma}$ (constant)]
~\\By previous argument, the density function of random variable $X_{\infty}^T$ is $f^T_{eq}$
~\\By Laplace Method, which says that:
~\\For  $\forall g\in C^{\infty}(\mathbb{R})$, we have
$$\int g\cdot f^T_{eq}\rightarrow \int g\cdot \delta(x-x^*)=g(x^*)~\ as ~\ T\rightarrow 0 \eqno(11)$$
where $x^*=argmin (\phi(x))$
~\\Because $g(x)=x\in C^{\infty}(\mathbb{R})$, we immediately get:
$$E(X^T_\infty)=\int x\cdot f^T_{eq}\rightarrow\int x\cdot \delta(x-x^*)=x^* ~\ as ~\ T\rightarrow 0 \eqno(12)$$
So we get:
$$\mathop{lim}\limits_{t\rightarrow\infty, T\rightarrow 0}E(X^T_t)=x^* \eqno(13)$$
So the basic idea of simulated annealing is to approximate $\mathop{lim}\limits_{t\rightarrow\infty, T\rightarrow 0}E(X^T_t)$
\subsection{Algorithm of simulated annealing}
Fix T=1, we fix a point $x_0$ and evolve the stochastic differential equation $dX_t^T=-\frac{1}{\gamma}\nabla\phi(X_t^T)dt+\frac{\sigma}{\gamma} dW_t$ [equation(10)]. When it comes to the steady state $(t>>1)$, we start sampling $\{x_1, x_2, x_3, ..., x_n\}$
when n is large enough, we get $E(X^1_\infty)\approx\frac{1}{n}\sum\limits^n_{i=1}x_i$
~\\Fix T=0.1, we set $x_0=\frac{1}{n}\sum\limits^n_{i=1}x_i$ and evolve the stochastic differential equation $dX_t^T=-\frac{1}{\gamma}\nabla\phi(X_t^T)dt+\frac{\sigma}{\gamma} dW_t$ [equation(10)]. When it comes to the steady state $(t>>1)$, we start sampling $\{x_1, x_2, x_3, ..., x_n\}$
when n is large enough, we get $E(X^{0.1}_\infty)\approx\frac{1}{n}\sum\limits^n_{i=1}x_i$.
~\\We keep lowering the temperature.
~\\Finally, we get $T=\epsilon$ ($\epsilon<<1$). After doing sampling $\{x_1, x_2, x_3, ..., x_n\}$, we have $\frac{1}{n}\sum\limits^n_{i=1}x_i\approx E(X^{\epsilon}_\infty)\approx x^*$

\subsection{From SGD (minibatch) to SA}
We first consider the following lost function
$$
F(x) = \frac{1}{N} \sum_{i=1}^N f_i(x),
$$
where $N$ is the number of data point. The Minibatch SGD is written
as
\begin{equation} \label{equ:minibatch}
x^{n+1} = x^n - \Delta t_{n} \nabla F_{B_n}(x^n),
\end{equation}
where
$$
F_{B_n}(x) = \frac{1}{|B_n|} \sum_{j \in B_n} f_j(x).
$$
Now, we rewrite \eqref{equ:minibatch} as
\begin{equation} \label{equ:minibach-SA}
X^{n+1} = X^n - \Delta t_{n} \nabla F(X^n) + \Delta {t}_n
\underbrace{[\nabla F(X^n) - \nabla F_{B_n}(X^n)]}_{Y^n}.
\end{equation}

\begin{equation} \label{equ:minibach-SA}
X^{n+1} = X^n - a_{n} \nabla F(X^n) + a_n
[\nabla F(X^n) - \nabla F_{B_n}(X^n)-\frac{b_n}{a_n}] + b_nW_n
\end{equation}
\begin{equation} \label{equ:minibach-SA}
X^{n+1} = X^n - a_{n} [\nabla F(X^n) + \xi_n] + b_nW_n
\end{equation}
where
$$
a_n=\Delta t_n, \xi_n=\nabla F(X^n) - \nabla F_{B_n}(X^n)-\frac{b_n}{a_n}W_n.
$$

Let $p_i$ is the probability of choosing the $i$-th component to
the minibatch. We note here that each component may be in the
minibatch for more than once. Then, we have
$$
\begin{aligned}
\mathbb{E}(Y^n) &= \nabla F(X^n) - \mathbb{E} [\nabla F_{B_n}(X^n)] \\
&= \frac{1}{N} \sum_{i=1}^N \nabla f_i(X^n) - \frac{1}{|B_n|}
\mathbb{E}[\sum_{j\in B_n} \nabla f_j(X^n)] \\
&= \frac{1}{N} \sum_{i=1}^N \nabla f_i(X^n) - \mathbb{E}[\nabla
f(X^n)] \qquad (\text{independence}) \\
&= \frac{1}{N} \sum_{i=1}^N \nabla f_i(X^n) - \sum_{i=1}^N p_i \nabla
f_i(X^n).
\end{aligned}
$$
Therefore, we have the following condition
\begin{equation} \label{equ:mean-condition}
\frac{1}{N}\sum_{i=1}^N \nabla f_i(x^n) = \mathbb{E}[\nabla f(X^n)].
\end{equation}
From the above condition, a simple choice is $p_i = \frac{1}{N}$,
namely uniform distribution. Condition \eqref{equ:mean-condition}
can also be written as $\mathbb{E}(Y^n) = 0$.

Note that ${\rm Var}(Y^n) = \mathbb{E}[Y^n (Y^n)^T] - \mathbb{E}(Y^n)
\mathbb{E}(Y^n)^T = \mathbb{E}[Y^n (Y^n)^T]$. For simplicity,
we remove the dependence of $X^n$ in the following. Then,
$$
\begin{aligned}
{\rm Var}(Y^n) &= \mathbb{E}[ (\nabla F - \nabla F_{B_n})
(\nabla F - \nabla F_{B_n})^T ]
\\
&= \mathbb{E}[\nabla F_{B_n} (\nabla F_{B_n})^T ] - \nabla F (\nabla
F)^T \\
&= \frac{1}{|B_n|^2} \mathbb{E}_{i_1} \mathbb{E}_{i_1} \cdots
\mathbb{E}_{i_{|B_n|}}[ (\sum_{j=1}^{|B_n|} \nabla f_{i_j})
(\sum_{j=1}^{|B_n|} \nabla f_{i_j})^T ] - \nabla F (\nabla F)^T \\
&= \frac{1}{|B_n|^2} \mathbb{E}_{i_1} \mathbb{E}_{i_2} \cdots
\mathbb{E}_{i_{|B_n|}}[ \sum_{j,r=1 , i\neq j}^{|B_n|} \nabla f_{i_j}
(\nabla f_{i_r})^T] \\
& ~~ + \frac{1}{|B_n|^2} \mathbb{E}_{i_1} \mathbb{E}_{i_2} \cdots
\mathbb{E}_{i_{|B_n|}} [\sum_{j=1}^{|B_n|} \nabla
f_{i_j} (\nabla f_{i_j})^T]- \nabla F (\nabla F)^T \\
& = (1 - \frac{1}{|B_n|}) \mathbb{E}(\nabla f) [\mathbb{E}(\nabla
f)]^T + \frac{1}{|B_n|} \mathbb{E}[ (\nabla f) (\nabla f)^T] -
\nabla F (\nabla F)^T \\
&= \frac{1}{|B_n|} \left( \mathbb{E}[(\nabla f) (\nabla f)^T] -
\mathbb{E}(\nabla f) [\mathbb{E}(\nabla f)^T]
\right) \\
&= \frac{1}{|B_n|} {\rm Var}(\nabla f).
\end{aligned}
$$

\begin{remark}
	Since the index here can be repeated, there is NO need that ${\rm
		Var}(Y^n) = 0$ when $|B_n| = N$.
\end{remark}

In summary, we have
\begin{equation} \label{equ:summary}
\mathbb{E}(Y^n) = 0, \qquad {\rm Var}(Y^n) = \frac{1}{|B_n|} {\rm
	Var}(\nabla f).
\end{equation}
It is conceivable that the result can be extended to the expectation
version of the lost function, namely,
$$
F(x) = \mathbb{E}_{\xi} f(x, \xi).
$$

\subsection{Convergence result}
In the SA scheme,
\begin{equation} \label{equ:SA}
X^{n+1} = X^n - \Delta t_n \nabla F(X^n) + \sigma_n W_{\Delta t_n},
\end{equation}
the convergence is proven under the following condition:
$$
\Delta t_n = \mathcal{O}(n), \qquad \sigma_n^2 =
\mathcal{O}(\frac{1}{n \log\log n})
$$
Compare the variance of $\sigma_n W_{\Delta t_n}$ and $\Delta t_n
Y^n$, we have
$$
\sigma_n^2 \Delta t_n \approx \Delta t_n^2 {\rm Var}(Y^n),
$$
which implies that
$$
\frac{1}{|B_n|} {\rm Var}(\nabla f) =
\mathcal{O}(\frac{\sigma_n^2}{\Delta t_n}) =
\mathcal{O}(\frac{1}{\log\log n}).
$$
We conclude that, if the SGD is stagnated, namely ${\rm Var}(\nabla
f)$ stays the almost the same, we need to slowly increase the size of
minibatch in the way that
\begin{equation} \label{equ:batch-size}
|B_n| = \mathcal{O}(\log\log n).
\end{equation}

\begin{remark}
	The above argument requires that $\nabla f(x, \xi)$ is Gaussian on
	$\xi$ for any $x$. This assumption holds for linear Gaussian
	regression.
\end{remark}


\subsection{lead compensation}
$$
G_c(s) = K_c\dfrac{s + \dfrac{1}{T}}{s+\dfrac{1}{\alpha T}}, \quad 0 < \alpha < 1
$$
$$
G_c(s) = K_c \alpha \dfrac{Ts + 1}{\alpha T s + 1}
$$
Define:
$$K_c\alpha = K$$
then:
$$
G_c(s) = K \dfrac{Ts + 1}{\alpha T s + 1}
$$
The open-loop transfer function of the compensated system is:
$$
G_c(s)G(s) = G_c(s) = K \dfrac{Ts + 1}{\alpha T s + 1} G(s)
= G_c(s) = \dfrac{Ts + 1}{\alpha T s + 1}KG(s) = G_c(s) = \dfrac{Ts + 1}{\alpha T s + 1}G_1(s)
$$
Where:
$$
KG(s)= G_1(s)
$$
Steady state error must be less than $0.05\%$
$$
e_{ss} = \lim_{s\to 0} = \dfrac{1}{1+G_1(0)} = 0.05 \to 0.05 + 0.05G_1(0) = 1 \to 0.95 = 0.05G_1(0) \to G_1(0) = 19
$$
$$
\lim_{s\to 0} G(s) = \lim_{s\to 0} \dfrac{50(s+0.5)}{(s+1)(s+1.5)^{3}(s+2)} = \dfrac{50\times0.5}{1\times1.5^3\times2} = 3.7037 \xrightarrow[G_1 = KG]{G_1(0) = 19} K = 5.1300
$$

The amplitude ratio:
$$
\left\vert G_1(j\omega) \right\vert = \left\vert KG(j\omega) \right\vert = 5.1300\dfrac{50\sqrt{\omega^2+0.5^2}}{\sqrt{\omega^2+1^2}\times(\sqrt{\omega^2 + 1.5^2})^3\times\sqrt{\omega^2 + 2^2}}
$$
Gain Crossover frequency:
$$
\left\vert G_1(j\omega_g) \right\vert = 1
$$
\newline
$$
5.1300\dfrac{50\sqrt{\omega_g^2+0.5^2}}{\sqrt{\omega_g^2+1^2}\times(\sqrt{\omega_g^2 + 1.5^2})^3\times\sqrt{\omega_g^2 + 2^2}} = 1
$$
$$
{26.3169\times2500({\omega_g^2+0.25}}) =
{({\omega_g^2+1})({\omega_g^2 + 2.25})^3({\omega_g^2 + 4}})
$$
This equation solved with MATLAB and code has attacked (Q1\_a.m).
$$
\omega_g = 3.6233
$$

The phase angle:

$$
\angle G(j\omega) = 0^{\circ} + \tan^{-1}\dfrac{\omega}{0.5} - \tan^{-1}\dfrac{\omega}{1} - 3\tan^{-1}\dfrac{\omega}{1.5} - \tan^{-1}\dfrac{\omega}{2}
$$

$$
\angle G(j\omega_g) = \tan^{-1}\dfrac{3.6233}{0.5} - \tan^{-1}\dfrac{3.6233}{1} - 3\tan^{-1}\dfrac{3.6233}{1.5} - \tan^{-1}\dfrac{3.6233}{2} = -4.46917_{rad} = -256.064^{\circ}
$$
Phase margin:
$$
\gamma = 180 + \angle G(j\omega_g) = 180 - 256.064 = -76.064
$$
Now we check above calculation with MATLAB margin function.
\begin{figure}[H]
	\caption{System Bode diagram using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/margin.png}
\end{figure}
Nyquist plot for system.
\begin{figure}[H]
	\caption{System Nyquist plot using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/system_nyquist.png}
\end{figure}
MATLAB bode diagram and our calculation are exactly the same.
$$
\bar{\gamma} = 45 + 5 = 50^{\circ},\quad \phi_m = \bar{\gamma} - \gamma = 126.0646^{\circ}.
$$
$$
\phi_m = 85^{\circ}
$$
This is too much!
best we can do is about $90^{\circ}$ in lead compensation so we add $85^{\circ}$.
$$
\sin(\phi_m) = \dfrac{1-\alpha}{1+\alpha} \to
\alpha = \dfrac{1-\sin(\phi_m) }{1+\sin(\phi_m) } = 0.0019
$$
$\alpha$ is very low.
$$
K_c = \dfrac{K}{\alpha} = \dfrac{5.1300}{0.0019} = 2691.1
$$
$$
\left\vert G_1(j\omega_m) \right\vert = \sqrt{\alpha} = \left\vert KG(j\omega_m) \right\vert = 5.13 \dfrac{50\sqrt{\omega_m^2+0.5^2}}{\sqrt{\omega_m^2+1^2}\times(\sqrt{\omega_m^2 + 1.5^2})^3\times\sqrt{\omega_m^2 + 2^2}} = \sqrt{0.0019}
$$
This equation solved with MATLAB and code has attacked (Q1\_a.m).
$$
\omega_m = 8.58898
$$
$$
\omega_m = \dfrac{1}{T\sqrt{\alpha}} \to
T = \dfrac{1}{\omega_m\sqrt{\alpha}} = \dfrac{1}{8.58898\sqrt{0.0019}} = 2.6666
$$
$$
G_c(s) = K_c \dfrac{s + \dfrac{1}{T}}{s + \dfrac{1}{\alpha T}} = 2691.1\dfrac{s + 0.3750}{s + 196.720}
$$
Bode diagram for lead compensation using MATLAB.
\begin{figure}[H]
	\caption{lead compensation Bode diagram using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/controller_bode.png}
\end{figure}
Nyquist plot for lead compensation using MATLAB.
\begin{figure}[H]
	\caption{lead compensation nyquist plot using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/controller_nyquist.png}
\end{figure}
Now add lead compensation to system.
$$
G_c(s)G(s) = 2691.1\dfrac{s + 0.3750}{s + 196.720} \dfrac{50(s+0.5)}{(s+1)(s+1.5)^{3}(s+2)}
$$
Bode diagram for system with adding lead compensation.
\begin{figure}[H]
	\caption{Bode diagram for system with lead compensation using MATLAB}
	\centering
	\label{margin with one lead}
	\includegraphics[width=12cm]{../Figure/Q1/a/new_margin.png}
\end{figure}
Nyquist plot for system with adding lead compensation.
\begin{figure}[H]
	\caption{Nyquist plot for system with lead compensation using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/new_nyquist.png}
\end{figure}
That was the best we can do with lead compensation!
All bode diagram in one figure:
\begin{figure}[H]
	\caption{all bode diagram using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/all_in_one.png}
\end{figure}
We didn't get requirement so we add new lead compensation.
we skip previous calculation and use MATLAB margin function in figure \ref{margin with one lead}.
$$
\bar{\gamma} = 45 + 5 = 50^{\circ},\quad \phi_m = \bar{\gamma} - \gamma = 98.9000^{\circ}.
$$
$$
\phi_m = 85^{\circ}
$$
This is too much!
best we can do is about $90^{\circ}$ in lead compensation so we add $85^{\circ}$.
$$
\sin(\phi_m) = \dfrac{1-\alpha}{1+\alpha} \to
\alpha = \dfrac{1-\sin(\phi_m) }{1+\sin(\phi_m) } = 0.0019
$$
$\alpha$ is very low.
With before lead compensation we currect steady state error so we assume $K = 1$.
$$
K_c = \dfrac{K}{\alpha} = \dfrac{1}{0.0019} = 524.5825
$$
From previous calculation and MATLAB calculation figure \ref{margin with one lead} $\omega_m = 8.58898$
$$
\omega_m = \dfrac{1}{T\sqrt{\alpha}} \to
T = \dfrac{1}{\omega_m\sqrt{\alpha}} = \dfrac{1}{8.58898\sqrt{0.0019}} = 2.6666
$$
$$
G_c(s) = K_c \dfrac{s + \dfrac{1}{T}}{s + \dfrac{1}{\alpha T}} = 524.5825\dfrac{s + 0.3750}{s + 196.720}
$$
Bode diagram for second lead compensation using MATLAB.
\begin{figure}[H]
	\caption{second lead compensation Bode diagram using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/new_controller_bode.png}
\end{figure}
Nyquist plot for second lead compensation using MATLAB.
\begin{figure}[H]
	\caption{second lead compensation nyquist plot using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/new_controller_nyquist.png}
\end{figure}
Now add second lead compensation to system and lag compensation.
$$
C_{c_1}(s)G_c(s)G(s) = 524.5825\dfrac{s + 0.3750}{s + 196.720}
2691.1\dfrac{s + 0.3750}{s + 196.720}
 \dfrac{50(s+0.5)}{(s+1)(s+1.5)^{3}(s+2)}
$$
Bode diagram for system with adding lead compensation.
\begin{figure}[H]
	\caption{Bode diagram for system with lead compensation using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/two_lead_new_margin.png}
\end{figure}
Nyquist plot for system with adding lead compensation.
\begin{figure}[H]
	\caption{Nyquist plot for system with lead compensation using MATLAB}
	\centering
	\includegraphics[width=12cm]{../Figure/Q1/a/two_lead_new_nyquist.png}
\end{figure}
We didn't get requirement so we add new lead compensation.
we skip previous calculation and use MATLAB margin function in figure \ref{margin with one lead}.
$$
\bar{\gamma} = 45 + 10 = 55^{\circ},\quad \phi_m = \bar{\gamma} - \gamma = 98.9000^{\circ}.
$$


% Project for Integration Workshop. Dept. Mathematics. UArizona
% Write a first-order ODE solver for a system of ODEs

\section{First-order ODE Integrators}
Consider the differential equation
\begin{equation}
\label{eq:ode_system}
\begin{cases} \dot{x}(t) = -y, & \quad x(0) = 1\\ \dot{y}(t) = \phantom{-}x, & \quad y(0) = 0 \end{cases}
\end{equation}
% which we can also write in matrix notation as
% \begin{equation}
% \label{eq:ode_matrix}
% \dot{\bm{x}}(t) = \begin{bmatrix*}[r] 0 & -1 \\ 1 & 0 \end{bmatrix*} \bm{x}, \quad \bm{x}(0) = \begin{pmatrix} 1\\0 \end{pmatrix}
% \end{equation}
Use pencil-and-paper to solve the ODE. Make a quiver plot. Sketch solutions.\\

In this project, you will build a numerical solver for differential equations like this one. You will test your solver on this ODE, and then use it to solve a more interesting ODE.
For this problem, discretize the time interval $0 \leq t \leq T$ into $n+1$ points. That is, set $\Delta t = T/n$ and $t_k = k \, \Delta t$, so $ t = \{t_0, t_1, \dots, t_n\}$. \\
\textit{Forward Differences:}
\begin{enumerate}[(a)]
    \item Use what's called a \textit{first-order forward difference} approximation to the derivatives $\dot{x}$ and $\dot{y}$ at each $t$ as follows:
    \begin{equation}
    \label{eq:forward_difference}
    \dot{x}(t) \approx \frac{x(t + \Delta t) - x(t)}{\Delta t} \qquad \text{and} \qquad \dot{y}(t) \approx \frac{y(t + \Delta t) - y(t)}{\Delta t}.
    \end{equation}
     Substitute equation (\ref{eq:forward_difference}) into equation (\ref{eq:ode_system}) and show that we can approximate the differential equation by the discrete time-stepping process:
    \begin{equation}
    \label{eq:forward_euler}
    \begin{cases} x_{k+1} = x_k - (\Delta t) y_k, & \quad x_0 = 1\\  y_{k+1} = y_k + (\Delta t) x_k, & \quad y_0 = 0\\  \end{cases}
    \end{equation}
    \item Implement equation \ref{eq:forward_euler} for $0 \leq t \leq 1$ and plot the trajectory given by the solution.
    \item Is your approximation qualitatively correct? Will it remain qualitatively correct on $0 \leq t \leq T$ as $T$ gets very large? Explain.\\
    \textit{Hint:} It may be useful to rewrite the system of difference equations in matrix form,
    \begin{equation*}
    \bm{x}_{k+1}
    % = \bm{x}_n + (\Delta t) \begin{bmatrix*}[r] 0 & -1 \\ 1 & 0 \end{bmatrix*} \bm{x}_n
    = \begin{bmatrix} 1 & -\Delta t \\  \Delta t & 1\end{bmatrix} \bm{x}_k, \quad \bm{x}_0 = \begin{pmatrix} 1\\0 \end{pmatrix}
    \end{equation*}
    and to analyze the eigenvalues of the matrix.
\end{enumerate}
\textit{Backward Differences:}
\begin{enumerate}[(a),resume]
    \item We could have solved this problem by a different approach. We could also have used what's called a \textit{first-order backward difference}:
    \begin{equation}
    \label{eq:backward_difference}
    \dot{x}(t) \approx \frac{x(t) - x(t - \Delta t)}{\Delta t} \qquad \text{and} \qquad \dot{y}(t) \approx \frac{y(t) - y(t  \Delta t)}{\Delta t}.
    \end{equation}
    Show that the backward difference gives the approximation
    \begin{equation}
    \label{eq:backward_euler}
     \begin{cases} x_{k} = x_{k-1} - (\Delta t) y_k, & \quad x_0 = 1\\  y_{k} = y_{k-1} + (\Delta t) x_k, & \quad y_0 = 0\\  \end{cases}
    \end{equation}
    which can be rewritten as
   \begin{equation*}
    \bm{x}_{k}
    % = \bm{x}_n + (\Delta t) \begin{bmatrix*}[r] 0 & -1 \\ 1 & 0 \end{bmatrix*} \bm{x}_n
    = \left(\begin{bmatrix} 1 & \Delta t \\  -\Delta t & 1\end{bmatrix}\right)^{-1} \bm{x}_{k-1}, \quad \bm{x}_0 = \begin{pmatrix} 1\\0 \end{pmatrix}
    \end{equation*}
    \item Implement equation (\ref{eq:backward_euler}) for $0 \leq t \leq 1$ and plot the trajectory given by the solution.
    \item Is your approximation qualitatively correct? Will it remain qualitatively correct on $0 \leq t \leq T$ as $T$ gets very large? Explain.
\end{enumerate}
\textit{Bonus:}
\begin{enumerate}[(a),resume]
    \item (The really fun stuff) Use your ODE solver to solve $\dot{x} = -y$, $\dot{y} = \sin x$.
\end{enumerate}
\newpage