\chapter{Theoretical Framework}
\section{Relativistic Electrons}
In a (linear) particle accelerator, charged particles, such as electrons, are accelerated to increase their total energy over their energy at rest.

Compared to heavier particles, such as protons ($m_p=\SI{938.27}{\mega\eV\per c\squared}$), electrons are light particles ($m_e=\SI{0.511}{\mega\eV\per c\squared}$). They are typically accelerated to speeds comparable to the speed of light to achieve kinetic energies usable for scientific experiments. For this reason, relativistic mechanics are needed to describe their movements.\footnote{As relativistic mechanics are a super set of classical mechanics, the equations also apply for slower particles.} \cite{Hinterberger1997}

With the speed of light $c=\SI{2.99792458e8}{\m\per\s}$ and the particle velocity $v$, it is common to define \cite{Wangler2008}:
\begin{align}\label{eq:theoreticalFramework_gammabeta}
\text{(normalized velocity)}\qquad\beta &= \frac{v}{c}\\
\text{(relativistic mass factor)}\qquad\gamma &= \frac{1}{\sqrt{1-\beta^2}} \\
\text{(relativistic momentum)}\qquad p &= \gamma m v
\end{align}

The total energy of a particle is \cite{Hinterberger1997}
\begin{equation}
W=\sqrt{(mc^2)^2+(pc)^2},
\end{equation}
with the rest energy $mc^2$ and the kinetic energy is
\begin{equation}
T=W-mc^2.
\end{equation}

With electrons (mass $m_e$) leaving the \gls{flute} electron gun with a momentum of $p=\SI{7}{\MeV\per c}$, this is equivalent to
\begin{align}
W=\sqrt{(m_ec^2)^2+(pc)^2} &= \SI{7.0186}{\MeV}\\
\gamma = \frac{W}{m_ec^2} &= \num{13.7351}\\
\beta = \sqrt{1-\gamma^{-2}} &= 0.9973
\end{align}

The force exerted on an electron (charge $q$) by an electric field $E$ and a magnetic flux density $B$ is given by the Lorentz force \cite[p.~19]{Hinterberger1997}
\begin{equation}\label{eq:fl}
F_L = q\left(E+v\times B\right).
\end{equation}
As the force due to the magnetic field is scaled with the velocity $v$, it is technically more viable to steer and focus a relativistic electron with magnets instead of electric fields. This is mainly because strong magnetic fields are easier to create and work with than high electric fields, which are difficult to handle. \cite[p.~27]{Krieger2013}

\autoref{eq:fl} also motivates the use need for stable electron energies. Since the velocity depends on the energy, the force on the electrons is also a function of their energy. The magnetic fluxes of electromagnets can only be changed slowly compared to the repetition rate of an accelerator, so corrections to $B$ are not possible, which is why $v$/$E$ needs to be stable for a stable trajectory/focusing.

\section{Signal Analysis}
\subsection{Auto- and Cross-Covariance}
The \textit{cross covariance} between two stochastic processes $x[n]$ and $y[n]$ is a measure of the similarity between $x[n]$ at index $n_1$ and $y[n]$ at index $n_2$ and is defined as
\begin{equation}\label{eq:crosscovariance}
r_{xy}[n_1,n_2] = \text{E}\left\{(x[n_1]-\mu_x[n_1])(y[n_2]-\mu_y[n_2])^\ast\right\}.
\end{equation}

For the special case of $y[n]:=x[n]$, $r_{xx}[n_1,n_2]$ is called \textit{auto covariance} and is a measure of self similarity of $x[n]$. \cite[p.~172]{Park2017}

The processes $x[n]$ and $y[n]$ are called \textit{\gls{wss}} if the following two properties hold. \cite[p.~167]{Park2017}
First, their means $\mu_{\xi}[n]$ are constant, i.e. they do not depend on the sample index:
\begin{align}\label{eq:wss}
\mu_{x}[n] &= \mu_x\\
\mu_{y}[n] &= \mu_y
\end{align}
Also, the auto covariance does not depend on the absolute sample indices $n_1$ and $n_2$, but merely on the difference between them:
\begin{equation}
r_{xy}[n_1,n_2] = r_{xy}[m],\qquad \text{with: } m:=n_2-n_1
\end{equation}

If both processes in \autoref{eq:crosscovariance} are \gls{wss}, \autoref{eq:crosscovariance} simplifies to
\begin{equation}
r_{xy}[m] = \text{E}\left\{(x[n]-\mu_x)(y[n-m]-\mu_y)^\ast\right\}.
\end{equation}

For the auto covariance both means are identical and can be moved outside the expectation operator:
\begin{equation}
r_{xx}[m] = \text{E}\left\{(x[n])(x[n-m])^\ast\right\}-\mu_{x}^2.
\end{equation}

When analyzing signals, the stochastic processes are often unknown and only one realization $x[n]$ is known. But if the process generating $x[n]$ is \textit{(weakly) ergodic}, then one realization is enough to determine the auto covariance of the process. \cite[p.~252]{Puente2019}
Then the auto covariance can be estimated with
\begin{equation}\label{eq:autocovarianveEstimation}
\hat{r}[m] = \frac{1}{N} \sum_{n=m+1}^{N} x[n]\,x^\ast[n-m]\qquad m \in [0,\,N-1].
\end{equation}

\subsection{Estimating the Spectrum of a Stochastic Process}
For a deterministic, time-discrete signal $x[n] \in \mathcal{L}_1$, the \gls{dft} exists (see \cite{Lapidoth2019}) and is defined as
\begin{equation}
X[k] = \sum_{n=0}^{N-1} x[n]\,\text{e}^{-j\frac{2\pi}{N}k\,n}\qquad k,n \in [0,\,N-1],
\end{equation}
using $k=\frac{N}{2\pi}\,\omega = N\,f$ as the independent, discrete frequency variable.
From the complex sequence $X[k]$, often only the magnitude (or energy) is of greater interest while the phase information are neglected.
Therefore, $S_{xx}$ is defined as
\begin{equation}
S_{xx} = \left|X[k]\right|^2,
\end{equation}
called the \textit{\gls{esd}}.

If $x[n]$ is the realization of a stochastic process, then it is of random nature rather than deterministic.
Because realizations of physical processes do not posses finite energy, they are not in the $\mathcal{L}_1$ set and their \gls{dft} is not defined. \cite[p.~5]{Stoica1997}

In this case instead of an energy spectral density, the spectrum of the average power of the process, called the \textit{\gls{psd}}, can be used instead.
To compute the \gls{psd}, there are two possibilities:
\begin{align}
\Phi_{xx}[k] &= \sum_{m=-\infty}^{\infty} r[m]\,\text{e}^{-j\frac{2\pi}{N}k\,m} \label{eq:phixx1}\\
\Phi_{xx}[k] &= \lim_{N\rightarrow\infty} \text{E}\left\{\frac{1}{N}\left|\sum_{n=0}^{N-1} x[n]\,\text{e}^{-j\frac{2\pi}{N}k\,n}\right|^2 \right\} \label{eq:phixx2}
\end{align}
When assuming $r[m]$ decays ``fast enough'', i.e.
\begin{equation}
\lim_{N\rightarrow\infty} \frac{1}{N} \sum_{m=-N}^{N} |m|\,\left|r[m]\right| = 0
\end{equation}
then \autoref{eq:phixx1} and \autoref{eq:phixx2} are equal. \cite[p.~7]{Stoica1997}

For measured data however neither equations can be used directly.
For \autoref{eq:phixx1} the auto covariance sequence $r[m]$ is unknown.
But it could be estimated with \autoref{eq:autocovarianveEstimation}. In case of \autoref{eq:phixx2} it is not possible to evaluate the limit, because only finite length data can be sampled and also the expectation can not be computed since in general there is only one realization available. Both operations can be neglected when doing an estimation.

With these practical changes in place, \autoref{eq:phixx1} and \autoref{eq:phixx2} become
\begin{align}
\hat{\Phi}_{c,\,xx}[k] &= \sum_{m=-(N-1)}^{N-1} \hat{r}[m]\,\text{e}^{-j\frac{2\pi}{N}k\,m} \label{eq:phiCxx1}\qquad\text{(Correlogram)}\\
\hat{\Phi}_{p,\,xx}[k] &= \frac{1}{N} \left| \sum_{n=0}^{N-1} x[n]\,\text{e}^{-j\frac{2\pi}{N}k\,n}\right|^2\qquad\text{(Periodogram)}\label{eq:periodogram}.
\end{align}

Both methods yield equal results, if $r[m]$ is estimated with the biased estimator $\hat{r}[m]$ in \autoref{eq:autocovarianveEstimation} in contrast to the unbiased estimator (compare \cite[p.~24]{Stoica1997})
\begin{equation}
\hat{r}_{\text{unbiased}}[m] = \frac{1}{N-m} \sum_{n=m+1}^{N} x[n]\,x^\ast[n-m]\qquad m \in [0,\,N-1].
\end{equation}

\cite{Rowell2008} shows one key weakness of the unmodified periodogram method in \autoref{eq:periodogram}: The variance does not decrease significantly with more samples $N$. Instead. the variance of the periodogram for each frequency approaches the square of the actual \gls{psd}:
\begin{equation}
\lim_{N\rightarrow\infty} \text{Var}\left\{\hat{\Phi}_{p,\,xx}[k]\right\} = \Phi_{xx}^2[k]
\end{equation}
Furthermore, the periodogram/correlogram suffer from the smearing and leakage effects because the limited length of the data samples always causes an implicit windowing, thus reducing frequency resolution.\\

There are several popular methods that improve on the periodogram/correlogram concepts:

\textbf{Blackman-Tukey:} Because of the poor accuracy of $\hat{r}[m]$ for $k\approx N$ in the definition of $\hat{\Phi}_{c,\,xx}[k]$ and the bigger the $N$, the more small errors in $\hat{r}[m]$ sum up, truncating/windowing of $\hat{r}[m]$ with $w[k]$ (length $M$) can be beneficial for the accuracy of the estimation.
\begin{equation}
\hat{\Phi}_{BT,\,xx}[k] = \sum_{m=-(M-1)}^{M-1} w[k]\hat{r}[m]\,\text{e}^{-j\frac{2\pi}{N}k\,m}
\end{equation}
The choice of the window $w[k]$ trades frequency resolution for variance and smearing for leakage reduction. \cite[p.~41]{Stoica1997}

\textbf{Barlett:} The Barlett method reduces the variance of the periodogram by splitting the $N$ data samples in $Q=\nicefrac{N}{M}$ blocks and averaging together the sub-periodograms:
\begin{align}
\hat{\Phi}_{q,\,xx}[k] &= \frac{1}{M} \left| \sum_{n=0}^{M-1} x_q[n]\,\text{e}^{-j\frac{2\pi}{M}k\,n}\right|^2\\
\hat{\Phi}_{B,\,xx}[k] &= \frac{1}{Q} \sum_{q=1}^{Q} \hat{\Phi}_{q,\,xx}[k]
\end{align}
The variance of the estimation scales with $Q$ (see \cite[p.~6]{Rowell2008}):
\begin{equation}
\text{Var}\left\{\hat{\Phi}_{B,\,xx}[k]\right\} = \frac{1}{Q}\Phi_{xx}^2[k]
\end{equation}

\textbf{Welch:} The Welch method combines splitting the data into $Q$ segments with windowing each segment and allowing the segments to overlap. With $P = \nicefrac{1}{M} \sum_{n=0}^{M-1} |w[n]|^2$ being the ``power'' of the window, the Welch method is computed as
\begin{align}
\hat{\Phi}_{s,\,xx}[k] &= \frac{1}{M P} \left| \sum_{n=0}^{M-1} x_s[n]\,\text{e}^{-j\frac{2\pi}{M}k\,n}\right|^2\\
\hat{\Phi}_{W,\,xx}[k] &= \frac{1}{Q} \sum_{s=1}^{Q} \hat{\Phi}_{s,\,xx}[k].
\end{align}
Compared to the Barlett method, the overlapping of up to \SI{50}{\percent} (see \cite{Welch1967}) allows increasing $Q$, thus reducing the variance.
\begin{equation}\label{eq:varWelch}
\text{Var}\left\{\hat{\Phi}_{W,\,xx}[k]\right\} = \frac{1}{Q}\Phi_{xx}^2[k]
\end{equation}

\paragraph{Spectogram}
If a stochastic process or a signal as a realization of that process $x[n]$ is not \gls{wss}, one possibility to analyze and display the spectral content is the use of the \gls{stft} and the spectrogram, which is a two dimensional power spectral density function mapping frequency and time to a third coordinate like height, intensity or color.

To calculate the spectrogram, the signal is split into segments with the sliding window $w[n-m]$ for which duration the signal is assumed to be stationary. For each segment at time index $m$, the periodogram is calculated according to
\begin{equation}
\hat{\Phi}_{xx}[k,m] = \frac{1}{N} \left| \sum_{n=0}^{N-1} w[n-m] x[n]\,\text{e}^{-j\frac{2\pi}{N}k\,n}\right|^2.
\end{equation}

\section{Feedback Control Systems}\label{sec:feedbackcontrol}
Feedback control systems are used to control a dynamic system (also called a plant) in such a way that its output $y(t)$ follows a certain input $x(t)$ and disturbances on the output $d(t)$ are rejected. The general structure of a closed-loop control system is shown in \autoref{fig:theoreticalFramework-feedback-architecture}. To achieve sufficient tracking of the input and stabilization of the output, a controller $G(s)$ uses the error $e(t)$ to control the plant $P(s)$ accordingly.
The error is defined as
\begin{equation}
e(t)=x(t)-r(t) = x(t)-[y(t)\ast h(t)]
\end{equation}
with $h(t)$ being the inverse Laplace transform of the filters transfer function $H(s)$. The signal $r(t)$ is the output of the measurement filter $h(t)$ or $H(s)$. It is most commonly a lowpass filter used to reject high-frequency noise on the system output $y(t)$.

Feedback control systems, or closed-loop systems, are to be differentiated from open-loop systems, in which there is no return path, so they cannot compensate for \textit{un}known disturbances. If $d(t)$ is known $\forall t$, then an open loop system would be possible and any errors could simply be compensated. But for real world application this approach is only usable for crude control tasks or if the system is very well understood or isolated from its surroundings.

\begin{figure}[tbh]
	\centering
	\includegraphics[width=\textwidth]{chap/TheoreticalFramework/img/feedbackControl/architecture.tikz}
	\caption[Control system structure]{General structure of a time-continuous feedback control system}
	\label{fig:theoreticalFramework-feedback-architecture}
\end{figure}

\subsection{Disturbance Rejection and Input Tracking}\label{sec:drandtrack}
Disturbance rejection and input tracking are two important characteristics to evaluate a stable controller. To calculate them, the block diagram in \autoref{fig:theoreticalFramework-feedback-architecture} and the Laplace transform of the inputs/outputs are used.\footnote{The Laplace transform of a function in time $f(t)$ is written as $F(s)=\mathcal{L}\left\{f(t)\right\}$.}

To calculate how the output $y(t)$ depends on the input $x(t)$, the input tracking transfer function can be used. \cite[p.~88]{Foellinger2016}
It is calculated as the transfer function $F_T=\nicefrac{Y(s)}{X(s)}$ by setting $d(t)=0$:
\begin{align}\label{eq:inputTracking}
Y(s)&=G(s)P(s)E(s)\quad\text{with:}\quad E(s)=X(s)-H(s)Y(s)\\
\Leftrightarrow Y(s)\left[1+G(s)P(s)H(s)\right]&=G(s)P(s)X(s)\\
\Leftrightarrow F_T:=\frac{Y(s)}{X(s)} &= \frac{G(s)P(s)}{1+G(s)P(s)H(s)}.
\end{align}

On the other hand, the transfer function $F_{DR}=\nicefrac{Y(s)}{D(s)}$ can be used to describe the systems response to a disturbance. \cite[p.~88]{Foellinger2016} It is defined by setting $x(t)=0$ and calculating
\begin{align}\label{eq:disturbanceRejection}
Y(s)&=G(s)P(s)E(s)+D(s)\quad\text{with:}\quad E(s)=-H(s)Y(s)\\
\Leftrightarrow Y(s)\left[1+G(s)P(s)H(s)\right]&=D(s)\\
\Leftrightarrow F_{DR}:=\frac{Y(s)}{D(s)}&=\frac{1}{1+G(s)P(s)H(s)}.
\end{align}

\subsection{Control System Stability}
The application of a controller to a system is only useful if the resulting system has a stable behavior.
One possible definition of stability is the \gls{bibo} criterion (see \cite[p.~82]{Foellinger2016}):
\begin{definition}\label{def:bibo}
(\textit{BIBO stability}) A \gls{lti} system is said to be \gls{bibo} stable if for some $M,N \in \mathbb{R}^+$, the response to a bounded input $|u(t)| \le M$ results in a bounded output $|y(t)| \le N$.
\end{definition}

For a given control system, one way to analyze its stability is to plot the locus $z=F_o(s=j2\pi f)$ of the open loop frequency response
\begin{equation}
F_o(s) = G(s)P(s)H(s)
\end{equation}
from $f=0$ to $f=\infty$ and using the Nyquist stability criterion.
For the special case of a stable open loop $F_o(s)$ \footnote{The stability of $F_o(s)$ can often easily be determined from the block diagram.} the Nyquist stability criterion can be stated as (see \cite[p.~111]{Foellinger2016})
\begin{definition}\label{def:Nyquist}
(\textit{Nyquist stability criterion}) If the open loop $F_o(s)$ is stable, then the closed loop is stable if $z=F_o(s=j2\pi f)$ does not go through or encircles $z=(-1,0j)$.
\end{definition}

\newpage
\section{Metrics to Quantify the Stability of a Signal}\label{sec:metrics}
``Stability'' can have different meanings depending on the context. In case of signal processing, a signal is usually said to be \textit{stable} if it has only little variation around its mean or some target value, i.e. the mean has to be constant and the variance stays below some threshold.
Stability is not to be confused with stationarity, which requires the mean, the variance and the autocorrelation to stay constant over time. \cite{Guthrie2020}
To express stability as a single numerical value, there are several possibilities, some are described in the following.

\paragraph{Relative Standard Deviation}
This measures the stability as the standard deviation but related to the mean value to make it comparable to other quantities with different scaling or units.

The relative, or percentual, standard deviation of the stationary stoachastic process $X$ is defined using the mean $\mu_X$ and the standard deviation $\sigma_X$ as
\begin{equation}
\op{\%STD_X} := \frac{\sigma_X}{\mu_X}.
\end{equation}
In general, especially if $X$ is non-stationary (see \autoref{eq:wss}), $\op{\%STD_X}$ depends on the absolute time $t$ and the window size $T$ for which the process is assumed to be stationary:
\begin{equation}
\op{\%STD_X} = \op{\%STD_X}(t,T)
\end{equation}
In that case, for a fixed window size $T=T_0$, a mean percentual standard deviation can be computed with
\begin{equation}\label{eq:stdp}
\op{\%STD_X}(T=T_0) = \frac{1}{N} \sum_{n=0}^{N-1} \op{\%STD_X}(t_n,T_0).
\end{equation}
This assumes discrete time steps $t_n$, $n\in[0,N-1]$.

\paragraph{Mean Squared Error}
A similar measure to the percentual standard deviation is the \gls{mse}, if, instead of the mean $\mu_X$, a fixed target $x_t$ is used.

The mean squared error sums up the squared errors $\left(x[n] - x_t[n]\right)^2$ of $x[n]$ from a set value $x_t$. To remove the effect of the length of the data sequence, the sum is devided by the length of the sequence $N$:
\begin{equation}
\op{MSE}_x := \frac{1}{N} \sum_{n=0}^{N-1} \left(x[n] - x_t\right)^2
\end{equation}

\paragraph{Relative Power of Most Prominent Noise}
This novel approach compares the power of the most prominent noise power source $P_\text{noise, max}$ of the signal $x$ with the total power $P_x$:
\begin{equation}\label{eq:mpn}
\op{MPN}_x := \frac{P_\text{noise, max}}{P_x}
\end{equation}

Compared to the relative standard deviation $\op{\%STD}(t,T)$, this method has the advantage of being time-independent and keeping periodic noise structures:\\

\paragraph{Comparison}
To make $\op{\%STD}(t,T)$ independent of time, it could be tempting to choose $T=T_\text{total}$ with $T_\text{total}$ being the time span of the data set. But because of the averaging effect of the standard deviation estimator, this washes out the maxima and thus could lead to an under-estimation of $\op{\%STD}$, so an over-estimation of the stability.

The $\op{MSE}$ uses the whole data set, but due to normalizing maxima can also be hidden.

Periodogram-based method have the advantage that for a fixed target value $x_t$, they are very easy to interpret: All power, that is not in the $f=0$ frequency bin, is definitely noise. Integrating over these noise bins yields the error power comparable to the $\op{MSE}$.
The described $\op{MPN}$ metric has the advantage of showing the largest contributor to the noise and allows for an easy validation of a controllers success on this noise component.


\subsection{Functionals of probabilities}

\(\phi (P)\in \mathbb{R} \) is a functional on \(P(X)\).

Examples include the expectation and variance.

We can define derivatives on these functionals.

\(\phi (P)\approx \phi (P^0)+D_\phi (P-P^0)\)

Where \(D_\phi \) is linear.


\section{Dynamic Programming}
  \subsection{Dynamic Convex Hull Trick}
    \code{dp/dynamic-ch-trick.cpp}
  \subsection{Divide and Conquer Optimization}
    For DP problems of the form $$dp(i,j) = min_{k \leq j} \{dp(i-1,k)+C(k,j)\}$$ where $C(k,j)$ is some cost function.
    \code{dp/divide-conquer-opt.cpp}


\Lecture{Jayalal Sarma}{Oct 19, 2020}{18}{Introduction to Ramsey Numbers}{Shivlal Gangesh}{$\alpha$}{JS}
\section{Introduction}
Till now we have seen advanced versions of the discrete mathematics topics we already know. Now we are going to get into Extremal combinatorics. Here we are interested in questions of the form
\begin{itemize}
\item \textit{If this structure appears, then what is the minimum/maximum size of the object?}
\item \textit{If the size is at least this much , then what kind of structures appear in the object?}
\item \textit{What is the minimum size of the collection such that it is guaranteed to have certain property?}
\end{itemize}
In general we are interested in the extreme behaviors in combinatorics. The classic example we start with is an extension to an example that we have done in the beginning of the course as an Application of Pigeon Hole Principle.
\section{Starting Point} \label{R(3,3)}
\begin{theorem}
Six people meet in a party. Then either there exist three people who are friends with each other or there exist three people who are strangers with each other.(Note : Any two people can either be friends or strangers)
\end{theorem}
We are interested in proving the above statement. Lets look into two different approaches
\subsection{Model 1 (Using Cliques and Independent Sets)}
\begin{description}
   \item[Model] Let us represent the problem as a $6$ vertex graph $G(V,E) $with each person corresponding to a vertex. $(u,v) \in E$ if and only if person $u$ is a friend of person $v$.
In this Model the original statement can be reformulated as
\item[Statement]
\textit{Any graph on $6$ vertices must either have a clique on $3$ vertices or an independent set on $3$ vertices.
\item}
\begin{proof}
 Consider any vertex $v$ in the graph G, without loss of generality we can assume that the degree of $v$ is greater than or equal to $3$  because suppose it is not the case then consider $\overline{G}$ ; as $\textrm{Cliques in }G \leftrightarrow \textrm{Independent Sets in } \overline{G} $.\\
 Let the $3$ neighbours of $v$ be $a$, $b$ and $c$. Consider the two exhaustive cases :
 \begin{description}
    \item[Case 1 : There are no edges among $a$, $b$ and $c$]
    $ $ \newline
    Here we have $\{a, b, c\}$ as the 3-Independent Set
    \item[Case 2 : There is at least one edge among $a$, $b$ and $c$ ]
    $ $ \newline
    Let $(a,b) \in E$ be that edge, then we have $\{v, a, b\}$ as the 3-clique
 \end{description}
Therefore the given statement holds true.
\end{proof}
\item[Proof for tightness]
To prove that this is tight we need to show there is a graph with $5$ vertices such that it does not have 3-clique and 3-Independent Set. Given below is one such  example
\begin{figure}[h!]
    \centering
    \includegraphics[width=0.2\linewidth]{images/r33counter_example.png}
    \caption{5-vertex graph with no 3-clique and no 3-Independent Set}
\end{figure}
\end{description}
\subsection{Model 2 (Using Graph Edge colouring)}
\begin{description}
   \item[Model]
   Let us represent the problem as 2-edge coloring of a $K_6$ graph with each vertex corresponding to a person. Color the edge $(u,v)$ with \textit{red} if $u$ and $v$ are friends, color it with \textit{blue} if $u$ and $v$ are strangers.
In this Model the original statement can be reformulated as
\item[Statement]
\textit{For any 2-edge colouring of $K_6$, there must exist either  a red $K_3$  or a blue $K_3$ }
\item
\begin{proof}
 Consider any Red,Blue-edge coloring of $K_6$. Consider any vertex $v$, the degree of $v$ is $5$ as the graph is a complete graph. By Pigeon Hole Principle , $v$ must have either $3$ red edges incident on it or $3$ blue edges incident on it. Consider the case when $v$ is incident on with $3$ red edges. Let the 3 neighbours of $v$ be $a$, $b$ and $c$.  Now there are 2 cases :
 \begin{description}
    \item[Case 1 : There is no red colored edge among $(a,b)$, $(b,c)$ and $(c,a)$ ]
    $ $ \newline
    Then all the three edges $(a,b)$, $(b,c)$ and $(c,a)$ are colored blue. Therefore $\{a, b, c\}$ forms a blue $K_3$
    \item[Case 2 : There is at least one red colored edge among $(a,b)$, $(b,c)$ and $(c,a)$ ]
    $ $ \newline
    Let $(a,b)$ be the red colored edge, then $\{v, a, b\}$ forms a red $K_3$
 \end{description}
 Therefore the given statement holds true.
\end{proof}
\item[Proof for tightness]
To prove that this is tight we need to show there is a 2-edge coloring of $K_5$ Such that it does not have red $K_3$ and blue $K_3$. Given below is one such  example
\begin{figure}[h!]
    \centering
    \includegraphics[width=0.2\linewidth]{images/k5counter_example.png}
    \caption{2-edge coloring of $K_5$ with no red $K_3$ and no blue $K_3$}
\end{figure}
\end{description}

Generalizing the above problem with arbitrary red $k_p$ and blue $k_q$ has been extensively studied by Ramsey and has led to the definition of Ramsey numbers.
\section{Ramsey numbers}
\begin{definition}[Ramsey number]
The Ramsey number denoted by $R(p,q)$ is the minimum number of vertices say $n$ such that any 2-edge coloring of $K_n$ must have either a red $K_p$ or a blue $K_q$.\\
(\textbf{Or equivalently as})\\
The minimum number of vertices ($n$) such that any graph on $n$ vertices must either have a clique on $p$ vertices or an independent set on $q$ vertices.
\end{definition}

\subsection{Some Observations}
\begin{property}
$R(3,3)=6$
\end{property}
This is the direct formulation of the example we have done previously in \ref{R(3,3)}
\begin{property}
$R(p,q) = R(q,p)$
\end{property}
The colors \textit{red} and \textit{blue} are just placeholders for two colors, thus swapping the colors will still preserve the Ramsey number property. Therefore $R(p,q) = R(q,p)$.
\begin{property}
$\forall l \geq 1 \quad R(l,1) = 1$
\end{property}
The existence of a blue $K_1$ is nothing but the presence of single vertex and any graph with a single vertex satisfies this property. Therefore $R(l,1) = 1$

\section{Existence of R(p,q)}
The Proof for the existence of $R(p,q)$ is due to Erdős–Szekeres. The existence was proved by providing an upper bound as a recurrence relation as follows :
\begin{theorem}
$$\forall p,q \geq 2 \quad R(p,q) \; \leq \; R(p,q-1) + R(p-1,q) $$
\end{theorem}
\begin{proof}
 Let us prove this by mathematical induction on $n$ where $n=p+q$.
 \begin{description}
    \item[Idea] To show the upper bound for $R(p,q) \leq n$ , we must argue that for any 2-edge coloring of $K_n$ there exist a red $K_p$ or blue $K_q$

   \item[Base case] $p=q=2$
$$R(2,2) \leq  R(2,1) + R(1,2)  $$
$$2 \leq 1+1$$
Hence it holds true for the base case.
   \item[Induction Hypothesis]
Assume the recurrence relation is true for $n<l$. Then we need to prove it for $n=l$. Let $n=R(p-1,q)+R(p,q-1)$. Let $w$ be any vertex in $G$ ($K_n$) and consider any $2$ -edge coloring of $G$. Let $H_1$ be the subgraph of $G$ formed from the vertices sharing a red-edge with $v$ and $H_2$ be the the subgraph of $G$ formed from the vertices sharing a blue-edge with $v$.
\begin{description}
   \item[Case 1 : There are at least $R(p-1,q)$ many red edges incident on vertex $w$]
   $ $ \newline
   $H_1$ is a complete graph on $R(p-1,q)$ vertices with 2-edge coloring. By definition and Induction Hypothesis we have that there exist a red $K_{p-1}$ or blue $K_q$ in $H_1$. So in graph $G$ (along with vertex $w$) there exist a red $K_p$ or blue $K_q$
   \item[Case 2 : There are at least $R(p,q-1)$ blue edges incident on vertex $w$]
      $ $ \newline
      $H_2$ is a complete graph on $R(p,q-1)$ vertices with 2-edge coloring. By definition and Induction Hypothesis we have that there exist a red $K_p$ or blue  $K_{q-1}$ in $H_2$. So in graph $G$ (along with vertex $w$) there exist a red $K_p$ or blue $K_q$.
\end{description}
   \end{description}

\end{proof}


\section{Sample algorithm}
\begin{algorithm}
	\caption{Kruskal's Algorithm}\label{alg:kruskal}
	\begin{algorithmic}[1]
		\Procedure{MakeSet}{$v$}
			\State Create new set containing $v$
		\EndProcedure \\

		\Function{FindSet}{$v$}
			\State\Return a set containing $v$
		\EndFunction \\

		\Procedure{Union}{$u$,$v$}
			\State Unites the set that contain $u$ and $v$ into a new set
		\EndProcedure \\

		\Function{Kruskal}{$V,E,w$}
			\State $A \gets \{\}$
			\For {each vertex $v$ in $V$}
				\State MakeSet($v$)
			\EndFor
			\State Arrange $E$ in increasing costs, ordered by $w$
			\For {each ($u$,$v$) taken from the sorted list}
				\If {FindSet($u$) $\neq$ FindSet($v$)}
					\State $A \gets A \cup \{(u,v)\}$
					\State Union($u,v$)
				\EndIf
			\EndFor
			\State \Return A
		\EndFunction
	\end{algorithmic}
\end{algorithm}

\newpage
\section{Sample Python code}
\begin{lstlisting}[language=Python]
import numpy as np

def incmatrix(genl1,genl2):
	m = len(genl1)
	n = len(genl2)
	M = None #to become the incidence matrix
	VT = np.zeros((n*m,1), int)  #dummy variable

	#compute the bitwise xor matrix
	M1 = bitxormatrix(genl1)
	M2 = np.triu(bitxormatrix(genl2),1)

	for i in range(m-1):
		for j in range(i+1, m):
			[r,c] = np.where(M2 == M1[i,j])
			for k in range(len(r)):
				VT[(i)*n + r[k]] = 1;
				VT[(i)*n + c[k]] = 1;
				VT[(j)*n + r[k]] = 1;
				VT[(j)*n + c[k]] = 1;

	if M is None:
		M = np.copy(VT)
	else:
		M = np.concatenate((M, VT), 1)

	VT = np.zeros((n*m,1), int)

	return M
\end{lstlisting}

\newpage
\section{Sample Matlab code}
\lstinputlisting[language=Octave]{codes/sample-code.m}


\subsection{Method of Undetermined Coefficients}
\noindent
The rules for undetermined coefficients are very similar for systems and single equations. Let's say we have a heterogeneous system of the form
\begin{equation*}
	\begin{cases}
		x_1' = a_{11}x_1 + \ldots + a_{1n}x_n + f_1(t) \\
		\vdots \\
		x_n' = a_{n1}x_n + \ldots + a_{nn}x_n + f_n(t)
	\end{cases}
\end{equation*}
written in a matrix form,
\begin{equation*}
	\begin{bmatrix}
		x_1' \\
		\vdots \\
		x_n'
	\end{bmatrix} = \begin{bmatrix}
		a_{11} & \ldots & a_{1n} \\
		\vdots & \ddots & \vdots \\
		a_{n1} & \ldots & a_{nn}
	\end{bmatrix} \begin{bmatrix}
		x_1 \\
		\vdots \\
		x_n
	\end{bmatrix} + \begin{bmatrix}
		f_1(t) \\
		\vdots \\
		f_n(t)
	\end{bmatrix}
\end{equation*}
or more compactly as
\begin{equation*}
	\vec{x}' = A\vec{x} + \vec{f}
\end{equation*}
All we need to do is look at each $f_i(t)$ and write in the $i^{th}$ blank the corresponding guess. This is like doing undetermined coefficients on each equation in the system.\\

\noindent
This is especially nice when all the $f_i(t)$'s have a similar form because we can write our guess as
\begin{equation*}
	\vec{g}(t) = f_c(t)\vec{v}
\end{equation*}
where $f_c(t)$ is the guess corresponding to $f$ and $\vec{c}$ is a vector of undetermined scalars.

\ifodd\includeLinearSystemsExamples\input{./linearSystems/heterogeneousSystems/undeterminedCoefficients_example.tex}\fi


%!TEX root =  ../main.tex

\objective{Solve arbitrary equations involving rational powers.}

Fractional exponents are nothing too special.  It is simply important to remember that in the
fraction, the numerator is what we've been thinking of as powers ($\frac{5}{1}=5$) and the
denominators are roots, i.e. $2^{\frac{1}{2}}=\sqrt{2}$.  As we will see next section, this isn't
a watertight definition, but it will do for now.

\subsection{Multiplying}\index{Exponents!rules of}
In situations where exponents abound, it may become confusing when one can and cannot
multiply, and what to do with exponents.  What do we mean when we write $5^2\cdot5^3$?
Is that $5^5$ or $5^6$?  Expanding the notation should help clear it up.  $5^2$ means
``multiply by five twice'', i.e. $5\cdot 5$.  $5^3$ means ``multiply by five three times'', $5\cdot
5\cdot 5$.  So we can see that $5^2\cdot5^3=5\cdot 5\cdot 5\cdot 5\cdot 5=5^5$.

\begin{derivation}{Multiplying with the Same Base}
$b^m\cdot b^n=b^{m+n}$.  Notice that $b$ must be consistent.
\end{derivation}

Students sometimes want to combine in impossible ways.  $2\cdot5^3$ becomes $10^3$
somehow, in their minds.  When we consider what the notation means, the contradiction
becomes clear.  $2\cdot5^3$ means ``two times this: five-times-five-times-five'', which is in no
way the same as ``ten times ten times ten''.

\subsection{Dividing}
If the exponents can be added when multiplying powers of the same base, what would you
expect when dividing?  Yes, it is subtraction.

\begin{derivation}{Dividing with the Same Base}
$\cfrac{b^m}{b^n}=b^{m-n}$.  Notice that $b$ must be consistent.
\end{derivation}

This is a good explanation for negative exponents.  For example, $\cfrac{2^3}{2^8}=2^{-5}$.
There are more two's in the denominator than the numerator.  This is the same as $\frac{1}{2^5}$,
which is a far more useful way to write the fraction.  When cancelling and simplifying are done,
it is conventional to expand the exponent, in this case writing $\frac{1}{32}$.

This also explains the origin of $b^0=1$, unless $b=0$.  Zero exponent arrises when there
are as many of the base in the denominator as there are in the numerator.  Anything
divided by itself is 1.

\subsection{Exponents}
What happens when there is an exponent on an exponent?  Easier than a dream within a
dream, the exponents continue to mean what they have always meant: ``have this many
of this bases be multiplied against themselves''.  For example, $(2^3)^4$ means ``four
groups of two-times-two-times-two'', or $2^12$.

\begin{derivation}{Distribution of Exponents over Multiplication}
$(b^m\cdot c^n)^p=b^{m\cdot p}\cdot c^{n\cdot p}$ and so for, on each element under
the power
\end{derivation}

\begin{example}
\exProblem
Solve $x^{\frac{3}{2}}=27$.

\exSolution
To get $x$ to have a simple exponent of 1, raise both sides to the two-third.\\
$\left(x^\frac{3}{2}\right)^{\frac{2}{3}}=27^{\frac{2}{3}}$
$x=\left(\sqrt[3]{27}\right)^2$\\
$x=3^2=9$
\end{example}

Notice too, there is no distributive property of exponents over addition.  $\sqrt{x^2+1}$
is irreducible, not $x+1$.  We would need to know what $x$ was to be able to proceed.

\subsection{Rational Exponents}\index{exponents!rational}
Fractional exponents must be the same as roots.  For example, we know from
the multiplication property that $4^\frac{1}{2}\cdot 4^\frac{1}{2}$ must equal $4^1$.
That must mean we are looking for a number times itself to equal 4, so $4^\frac{1}{2}$
must equal 2.  This means $\sqrt{4}=4^\frac{1}{2}$.  By the same logic, we would
find that $4^\frac{1}{3}=\sqrt[3]{4}$, $4^\frac{1}{4}=\sqrt[4]{4}$, etc.  By the power
rule above, $4^\frac{3}{2}$ must be the same as $(4^\frac{1}{2})^3$.  This could also
be written as $4^{1.5}$.

We must be careful with even rational exponents, because they can hide the sign of the base.
For example, $((-5)^2)^\frac{1}{2}=5$.  The safe answer then to $\sqrt[n]{x^n}$ for any
even $n$ is $|x|$.


\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage{amssymb,amsmath}

\usepackage{graphicx}

\usepackage[top=3cm, bottom=3cm, left=3cm, right=3cm, includefoot]{geometry}

\usepackage{amsthm}
\theoremstyle{definition}
\newtheorem{definition}{Definition}

\begin{document}

\begin{definition}
\emph{Convex polytope} $\mathbf{S}$ is the set of all convex combinations of a finite point set $ \mathcal{S}$, i.e.
\begin{displaymath}
 \mathbf{S} :=
 \left\lbrace
 v \in \mathbb{R}^n: v := \sum\limits_{i=1}^{\vert \mathcal{S} \vert} \alpha_i v_i;
	\forall i = 1,\dots,\vert \mathcal{S} \vert: \alpha_i \geq 0, v_i \in \mathcal{S},
	\sum\limits_{i=1}^{\vert \mathcal{S} \vert} \alpha_i = 1
 \right\rbrace.
\end{displaymath}
\end{definition}

Let us consider two convex polytopes $\mathbf{P}, \mathbf{Q}$ in $\mathbb{R}^d$ defined by the boundary point sets
\begin{displaymath}
 \begin{array}{rcll}
  \mathcal{P} & := & \lbrace p_1, \dots, p_{n_p} \rbrace \subset \mathbb{R}^d ~\mathrm{,} \\
  \mathcal{Q} & := & \lbrace q_1, \dots, q_{n_q} \rbrace \subset \mathbb{R}^d ~\mathrm{.}
 \end{array}
\end{displaymath}
The problem is to find the shortest distance between these two objects
\begin{equation}
 \label{eq:polytope_problem1}
 \min\limits_{p \in \mathcal{P}, q \in \mathcal{Q}} \Vert p - q \Vert ~\mathrm{.}
\end{equation}
Every interior point of the convex polytope can be expressed as a convex linear combination of given points in sets $\mathcal{P}$ and $\mathcal{Q}$
\begin{equation}
 \label{eq:polytope_linear_comb}
 \begin{array}{l}
  \forall p \in \mathbf{P} ~ \exists \alpha_1, \dots, \alpha_{n_p} \in \mathbb{R}: p = \sum\limits_{i = 1}^{n_p} \alpha_{i} p_i ~\mathcal{,} \\
	~~~~~~~~~~~~~~~~~~~~~~~ \mathrm{where} ~ \sum\limits_{i = 1}^{n_p} \alpha_i = 1 ~ \mathrm{and} ~ 0 \leq \alpha_i \leq 1 ~\forall i = 1,\dots,n_p ~\mathrm{,} \\
  \forall q \in \mathbf{Q} ~ \exists \beta_1, \dots, \beta_{n_q} \in \mathbb{R}: q = \sum\limits_{i = 1}^{n_q} \beta_{i} q_i ~\mathcal{,} \\
	~~~~~~~~~~~~~~~~~~~~~~~ \mathrm{where} ~ \sum\limits_{i = 1}^{n_q} \beta_i = 1 ~ \mathrm{and} ~ 0 \leq \beta_i \leq 1 ~\forall i = 1,\dots,n_q ~\mathrm{.}
 \end{array}
\end{equation}
We denote
\begin{displaymath}
 \begin{array}{rcl}
  y & := & [\alpha_1, \dots, \alpha_{n_p}, \beta_1, \dots, \beta_{n_q}]^T \in \mathbb{R}^{n_p + n_q} ~\mathrm{,} \\
  C & := & [p_1, \dots, p_{n_p}, -q_1, \dots, -q_{n_q}] \in \mathbb{R}^{d, n_p + n_q} ~\mathrm{,} \\
  B & := &
  \left[
   \begin{array}{cccccc}
    1 & \dots & 1 & 0 & \dots & 0 \\
    0 & \dots & 0 & 1 & \dots & 1
   \end{array}
  \right] \in \mathbb{R}^{2, n_p + n_q}  ~\mathrm{,} \\
  c & := & [1,1]^T \in \mathbb{R}^{2} \mathrm{.}
 \end{array}
\end{displaymath}
Afterwards, the cost function can be reformulated
\begin{displaymath}
 \Vert p - q \Vert  = \left\Vert \sum\limits_{i = 1}^{n_p} \alpha_{i} p_i - \sum\limits_{i = 1}^{n_q} \beta_{i} q_i \right\Vert = \Vert Cy \Vert
\end{displaymath}
and the feasible set conditions have the form
\begin{displaymath}
 By = c ~~~ \wedge ~~~ y \geq 0 ~\mathrm{.}
\end{displaymath}
After these notations, the problem \eqref{eq:polytope_problem1} can be reformulated as
\begin{equation}
 \label{eq:polytope_problem2}
 \begin{array}{rcl}
  \bar{y} & := & \arg \min\limits_{y \in \Omega_E \cap \Omega_I} y^T C^T C y ~\mathrm{,} \\
  \Omega_E & := & \left\lbrace y \in \mathbb{R}^{n_p + n_q}: By = c \right\rbrace ~\mathrm{,} \\
  \Omega_I & := & \left\lbrace y \in \mathbb{R}^{n_p + n_q}: y \geq 0 \right\rbrace ~\mathrm{.}
 \end{array}
\end{equation}
The next step consists of homogenization and orthogonalization. We introduce a substitution
\begin{equation}
 \label{eq:polytope_subs}
 x := y - y_{\mathrm{in}} ~~ \Rightarrow ~~ y = x + y_{\mathrm{in}} ~\mathrm{,}
\end{equation}
where $y_{in}$ is arbitrary point from $\Omega_E$. We can choose
\begin{displaymath}
 y_{\mathrm{in}} := \left[ \frac{1}{n_p}, \dots, \frac{1}{n_p}, \frac{1}{n_q}, \dots, \frac{1}{n_q} \right] \in \mathbb{R}^{n_p + n_q} ~\mathrm{.}
\end{displaymath}
Afterwards, the cost function and conditions have the form
\begin{displaymath}
 \begin{array}{l}
  f(x) := \frac{1}{2} \Vert C(x + y_{\mathrm{in}}) \Vert^2 = \frac{1}{2} x^T \overbrace{C^T C}^{=: A} x + x^T \overbrace{C^T C y_{\mathrm{in}}}^{=: - b} + ~c, ~~~ c =: \frac{1}{2} y_{\mathrm{in}}^T C^T C y_{\mathrm{in}} = ~\mathrm{const.} ~\mathrm{,}\\
  B(x + y_{\mathrm{in}}) = Bx + B y_{\mathrm{in}} = Bx + c ~~~ \Rightarrow ~~~ ( By = c ~ \Leftrightarrow ~ Bx = 0) ~\mathrm{,} \\
  y \geq 0 ~~ \Leftrightarrow ~~ x \geq - y_{\mathrm{in}}  ~\mathrm{.}
 \end{array}
\end{displaymath}
Moreover, the matrix $B$ can be orthonormalized using simple process
\begin{displaymath}
 \hat{B} :=
 \left[
  \begin{array}{cc}
   \frac{1}{\sqrt{n_p}} & 0 \\
   0 & \frac{1}{\sqrt{n_q}}
  \end{array}
 \right]
 B  ~\mathrm{.}
\end{displaymath}
We obtained QP with homogeneous orthogonal linear equality constraints and bound inequality constraints
\begin{equation}
 \label{eq:polytope_problem3}
 \begin{array}{rcl}
  \bar{x} & := & \arg \min\limits_{x \in \Omega_E \cap \Omega_I} \frac{1}{2} x^T A x - b^T x ~\mathrm{,} \\
  \Omega_E & := & \left\lbrace x \in \mathbb{R}^{n_p + n_q}: \hat{B}x = 0 \right\rbrace ~\mathrm{,} \\
  \Omega_I & := & \left\lbrace x \in \mathbb{R}^{n_p + n_q}: x \geq -y_{\mathrm{in}} \right\rbrace ~\mathrm{.}
 \end{array}
\end{equation}
After solving this problem, the original solution can be obtained using back substitution \eqref{eq:polytope_subs} to obtain $y$, i.e. the coeficients of linear combinations \eqref{eq:polytope_linear_comb} of the nearest points from each polytope.

%\newpage

\paragraph{Numerical example}

We consider two circles discretized by parameter $m \geq 3$, whose boundary points are defined by $P, Q \in \mathbb{R}^{2,m}$
with columns
\begin{displaymath}
 P_{\ast,i} =
 \left[
  \begin{array}{c}
   \cos (2i\pi / m ) - 2  \\
   \sin (2i\pi / m )
  \end{array}
 \right]
 , ~~~
 Q_{\ast,i} =
 \left[
  \begin{array}{c}
   \cos (\pi - 2i\pi / m ) + 2 \\
   \sin (\pi - 2i\pi / m )
  \end{array}
 \right], ~~~
 i = 0,\dots,m-1 ~\mathrm{.}
\end{displaymath}
Examples for $m = 5$ and $m=7$ can be found in Fig. \ref{fig:polytope1}.

\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.45\textwidth]{polytope_n5.eps}
\includegraphics[width=0.45\textwidth]{polytope_n7.eps}
\caption[Polytope distance: testing benchmark]{Testing benchmark for polytopes distance with discretization parameter $m = 5$ \emph{(left)} and $m = 7$ \emph{(right)}.}
\label{fig:polytope1}
\end{center}
\end{figure}

The solution of the problem for any $m$ is given by
\begin{displaymath}
 \bar{y} =
 \left[
  \alpha_1, \dots, \alpha_m, \beta_1, \dots, \beta_m
 \right]^T =
 [\underbrace{1 , 0, \dots, 0}_{\in \mathbb{R}^m}, \underbrace{1 , 0, \dots, 0}_{\in \mathbb{R}^m}]^T ~\mathrm{.}
\end{displaymath}

In this problem, we can directly compute the regular condition number of Hessian matrix
\begin{displaymath}
 \kappa (A) = \kappa (C^T C) = \kappa (CC^T) = \kappa
 \left( \left[
  \begin{array}{cc}
   \sum\limits_{i = 1}^{n_p} P_{i,1}^2 + \sum\limits_{i = 1}^{n_q} Q_{i,1}^2 & \sum\limits_{i = 1}^{n_p} P_{i,1} P_{i,2} + \sum\limits_{i = 1}^{n_q} Q_{i,1} Q_{i,2} \\
   \sum\limits_{i = 1}^{n_p} P_{i,2} P_{i,1} + \sum\limits_{i = 1}^{n_q} Q_{i,2} Q_{i,1} & \sum\limits_{i = 1}^{n_p} P_{i,2}^2 + \sum\limits_{i = 1}^{n_q} Q_{i,2}^2
  \end{array}
 \right] \right) ~\mathrm{.}
\end{displaymath}
Moreover, it holds
\begin{displaymath}
 \begin{array}{rcl}
 \sum\limits_{i = 1}^{n_p} P_{i,2} P_{i,1} + \sum\limits_{i = 1}^{n_q} Q_{i,2} Q_{i,1} & = &
	\sum\limits_{i = 0}^{m-1}
		\left( \cos \left( \frac{2i\pi}{m} \right) - 2 \right)
		\sin \left( \frac{2i\pi}{m} \right) \\
  & & ~~~~~ +
 \sum\limits_{i = 0}^{m-1}
  \left( \cos \left( \pi - \frac{2i\pi}{m} \right) + 2 \right)
  \sin \left( \pi - \frac{2i\pi}{m} \right)
 = 0~\mathrm{,}
 \end{array}
\end{displaymath}
so
\begin{displaymath}
 \kappa (A) = \kappa
 \left( \left[
  \begin{array}{cc}
   \sum\limits_{i = 1}^{n_p} P_{i,1}^2 + \sum\limits_{i = 1}^{n_q} Q_{i,1}^2 & 0 \\
   0 & \sum\limits_{i = 1}^{n_p} P_{i,2}^2 + \sum\limits_{i = 1}^{n_q} Q_{i,2}^2
  \end{array}
 \right] \right) ~\mathrm{.}
\end{displaymath}
Afterwards, the condition number can be expressed
\begin{displaymath}
\kappa (A) = \frac{\sum\limits_{i = 1}^{n_p} P_{i,1}^2 + \sum\limits_{i = 1}^{n_q} Q_{i,1}^2}{\sum\limits_{i = 1}^{n_p} P_{i,2}^2 + \sum\limits_{i = 1}^{n_q} Q_{i,2}^2} =
\frac{\sum\limits_{i=0}^{m-1} \left( \cos \left( \frac{2i\pi}{m} \right) - 2 \right)^2}{\sum\limits_{i=0}^{m-1} \sin^2 \left( \frac{2i\pi}{m} \right)} ~~~~ \forall m \geq 3 ~\mathrm{.}
\end{displaymath}

\noindent Let us consider $\alpha \in \mathbb{R}$ and let us present a complex number $z \in \mathbb{C}$ by prescription
\begin{displaymath}
 z := \cos \alpha + \mathbf{i} \sin \alpha ~\mathrm{,}
\end{displaymath}
where $\mathbf{i}$ is imaginary unit. Then by De Moivre's formula we can write
\begin{displaymath}
 \sum\limits_{i=0}^{m-1} \left( \cos ( i\alpha) + \mathbf{i} \sin ( i \alpha) \right) =
 \sum\limits_{i=0}^{m-1} z^i =
 \frac{z^m - 1}{z - 1} =
 \frac{\cos (m\alpha) + \mathbf{i} \sin (m\alpha) - 1}{z-1} ~\mathrm{.}
\end{displaymath}
If we choose specific $\alpha$ in previous equality, we obtain next
\begin{equation}
 \label{eq:polytope_ne_pom}
  \begin{array}{ccccc}
   \alpha := \frac{2 \pi}{m}& ~ \Rightarrow ~ & \sum\limits_{i=0}^{m-1} \left( \cos \left( \frac{2i\pi}{m} \right) + \mathbf{i} \sin \left( \frac{2i\pi}{m} \right) \right) = 0 & ~ \Rightarrow ~ & \sum\limits_{i=0}^{m-1} \cos \left( \frac{2i\pi}{m} \right) = 0\mathrm{,}\\
   \alpha := \frac{4 \pi}{m}& ~ \Rightarrow ~ & \sum\limits_{i=0}^{m-1} \left( \cos \left( \frac{4i\pi}{m} \right) + \mathbf{i} \sin \left( \frac{4i\pi}{m} \right) \right) = 0 & ~ \Rightarrow ~ & \sum\limits_{i=0}^{m-1} \cos \left( \frac{4i\pi}{m} \right) = 0\mathrm{.}
  \end{array}
\end{equation}
Now, we return back to regular condition number and $\forall m \geq 3$ we can write (using \eqref{eq:polytope_ne_pom})
\begin{displaymath}
 \begin{array}{rl}
   \kappa (A) = & \frac{\sum\limits_{i=0}^{m-1} \left( \cos \left( \frac{2i\pi}{m} \right) - 2 \right)^2}{\sum\limits_{i=0}^{m-1} \sin^2 \left( \frac{2i\pi}{m} \right)}
= \frac{\sum\limits_{i=0}^{m-1} \cos^2 \left( \frac{2i\pi}{m} \right) - 4 \cos \left( \frac{2i\pi}{m} \right) + 4}{\sum\limits_{i=0}^{m-1} \sin^2 \left( \frac{2i\pi}{m} \right)} \\
 & = \frac{\sum\limits_{i=0}^{m-1} \left( \frac{1 + \cos \left( \frac{4i\pi}{m} \right)}{2} - 4 \cos \left( \frac{2i\pi}{m} \right) + 4 \right)}{\sum\limits_{i=0}^{m-1}
  \frac{1 - \cos \left( \frac{4i\pi}{m} \right)}{2}}
 =  \frac{ \frac{1}{2} \sum\limits_{i=0}^{m-1} \cos \left( \frac{4i\pi}{m} \right) - 4 \sum\limits_{i=0}^{m-1} \cos \left( \frac{2i\pi}{m} \right) + \sum\limits_{i=0}^{m-1} \frac{9}{2}   }{-\frac{1}{2}\sum\limits_{i=0}^{m-1} \cos \left( \frac{4i\pi}{m} \right) + \sum\limits_{i=0}^{m-1} \frac{1}{2} } = 9 ~\mathrm{.}
 \end{array}
\end{displaymath}

\end{document}


\section{Introduction}

The objective of this exercise is to study damped and driven oscillations in
mechanical systems using the Pohl resonator. For driven oscillations, we will
also observe and quantify the mechanical resonance phenomenon.

If a periodically varying external force is applied to a damped harmonic
oscillator, the resulting motion is called forced (or driven) oscillations, and
the external force is called the driving force. Assuming that the driving force
is of the form
    \[
        F=F_0(sin\omega t+\delta),
    \]
with the amplitude F 0 and angular frequency ω, the resulting steady-state
forced oscillations will be simple harmonic with the angular frequency equal to
that of the driving force. The amplitude of these steady-state oscillations
turns out to depend on the angular frequency of the the driving force, in
particular on how far it is from the natural angular frequency, and the damping
coefficient. The amplitude may become quite large, and this phenomenon is known
as the mechanical resonance.

Another interesting property of driven steady-state oscillations is the fact
that there is a phase lag between the driving force and the displacement from
the equilibrium position of the oscillating particle. This phase lag reaches
$\pi/2$ (a quarter of the cycle) when the system is driven at the natural
angular frequency.

In this experiment, forced oscillation of a balance wheel will be studied. The
corresponding quantities (such as the force and the position) will be replaced
by their angular counterparts.

The driving torque $\tau_{dr}=\tau_0cos\omega t$ and a damping torque
$\tau_f=-b\frac{d\theta}{dt}$, Also, we know the restoring torque
$\tau=-k\theta$, its equation od motion is of the form

\begin{equation}
\label{1}
I\frac{d^2\theta}{dt^2}=-k\theta-b\frac{d\theta}{dt}+\tau_0cos\omega t,
\end{equation}

where I is the moment of inertia of the balance wheel, $\tau_0$ is the amplitude
of the driving torque, and $\omega$ is the angular frequency of the driving
torque. Introducing the symbols

\[
\omega_0=\frac{k}{I},\quad 2\beta=\frac{b}{I}, \quad \mu=\frac{\tau_0}{I},
\]

Eq. \ref{1} can be rewitten as
\begin{equation}
\label{2}
\frac{d^2\theta}{dt^2}+2\beta\frac{d\theta}{dt}+\omega_0^2\theta
=\mu cos\omega t.
\end{equation}

Thw solution to Eq \ref{2} is
\[
\theta(t)=\theta_{tr}(t)+\theta_{st}cos(\omega t+\varphi),
\]
where the former term $\theta_{tr}$ denotes the transient solution that vanished
exponentially as $t\rightarrow \infty$. The steady-state  oscillation is with
the amplitude
\[
\theta_{st}=\frac{\mu}{\sqrt{(\omega_0^2-\omega^2)^2+4\beta^2\omega^2}}
\]
For small values of the damping coefficient $\beta$, the resonance angular
frequency is close to the the natural angular frequency, and the amplitude of
steady-state oscillations becomes large. The dependence of both the amplitude
and the phase shift on the driving angular frequency are shown in the left and
right Figure 1, respectively, for different values of the damping coefficient.


\section{Basics of Reinforcement Learning}
\label{sec:basics_reinforcement_learning}

\emph{Reinforcement Learning} (RL) is a general class of algorithms in the field of \emph{Machine Learning} (ML) that allows an agent to learn how to behave in a stochastic and possibly unknown environment, where the only feedback consists of a scalar reward signal \cite{sutton1998introduction}. The goal of the agent is to learn by trial-and-error which actions maximize his long-run rewards. However, since the environment evolves stochastically and may be influenced by the actions chosen, the agent must balance his desire to obtain a large immediate reward by acting greedily and the opportunities that will be available in the future. Thus, RL algorithms can be seen as computational methods to solve sequential decision problems by directly interacting with the environment.\\

\subsection{Markov Decision Processes}
\label{sec:markov_decision_processes}
Sequential decision problems are typically formalized using \emph{Markov Decision Processes} (MDP). An MDP is a stochastic dynamical system specified by the tuple $<\S, \A, \calP, \calR, \gamma>$, where $(\S, \calS)$ is a measurable state space, $(\A, \calA)$ is a measurable action space, $\calP: \S \times \A \times \calS \to \R$ is a Markov transition kernel, $\calR: \S \times \A \to \R$ is a reward function and $0 < \gamma < 1$ is the discount factor. Suppose that at time $t$ the system is in state $S_t = s$ and that the agent takes action $A_t = a$, then, regardless of the previous history of the system, the probability to find the system in a state belonging to $B\in\calS$ at time $t+1$ is given by
\begin{equation}
	\calP(s, a, B) = \P{S_{t+1} \in B | S_t = s, A_t = a}
\end{equation}
Following this random transition, the agent receives a stochastic reward
$R_{t+1}$. The reward function $\calR(s, a)$ gives the expected reward
obtained when action $a$ is taken in state $s$, i.e.
\begin{equation}
	\calR(s, a) = \E{R_{t+1} | S_t = s, A_t = a}
\end{equation}
This feedback mechanism between the environment and the agent is illustrated in Figure \ref{fig:sequential_decision_problem}. At any time step, the agent selects his actions according to a certain policy $\pi: \S \times \calA \to \R$ such that for every $s \in \S$, $C \mapsto \pi(s,C)$ is a probability distribution over $(\A, \calA)$. Hence, a policy $\pi$ and an initial state $s_0 \in \S$ determine a random state-action-reward sequence ${\{(S_t, A_t, R_{t+1})\}}_{t\geq 0}$ with values on $\S \times \A \times \R$.
\begin{figure}[t]
	\centering
	\begin{tikzpicture}[node distance = 6em, auto, thick]
		\node [block] (Agent) {Agent};
		\node [block, below of=Agent] (Environment) {Environment};
		\path [line] (Agent.0) --++ (4em,0em) |- node [near start]{Action $a_t$} (Environment.0);
		\path [line] (Environment.190) --++ (-6em,0em) |- node [near start]{State  $s_{t}$} (Agent.170);
		\path [line] (Environment.170) --++ (-4.25em,0em) |- node [near start, right] {Reward $r_{t+1}$} (Agent.190);
	\end{tikzpicture}
	\caption{Agent-environment interaction in sequential decision problems.}
	\label{fig:sequential_decision_problem}
\end{figure}
In an infinite horizon task, the agent's performance is typically measured as the total discounted reward obtained following a specific policy
\begin{equation}
	G_t = \sum^{\infty}_{t=0} \gamma^t R_{t+k+1}
\end{equation}
Since this gain is stochastic, the agent considers its expected value, which is typically called \emph{state-value function}
\begin{equation}
	V_\pi(s) = \E[\pi]{G_t|S_t = s}
\end{equation}
where the subscript in $\mathbb{E}_{\pi}$ indicates that all the actions are selected according to policy $\pi$. The state-value function measures how good it is for the agent to be in a given state and follow a certain policy. Similarly, we introduce the \emph{action-value function}
\begin{equation}
	Q_\pi(s,a) = \E[\pi]{G_t|S_t = s, A_t = a}
\end{equation}
We have the following relationship between $V_\pi$ and $Q_\pi$
\begin{equation}
	V_\pi(s) = \int_\A \pi(s,a) Q_\pi(s,a) da
\end{equation}
Almost all reinforcement learning algorithms are designed to estimate these
value functions and are typically based on the Bellman equations.
\begin{equation}
	V_\pi(s) = \calR_\pi(s) + \gamma T_\pi V_\pi(s)
	\label{eq:bellman_expectation_eq_V}
\end{equation}
\begin{equation}
		Q_\pi(s,a) = \calR(s,a) + \gamma T_a V_\pi(s)
		\label{eq:bellman_expectation_eq_Q}
\end{equation}
where we denoted by $T_a$ (resp. $T_\pi$) the transition operator for action
$a$ (resp. for policy $\pi$)
\begin{equation}
	T_a F(s) = \E{F(S_{t+1})|S_t = s, A_t = a} = \int_\S \calP(s, a, s') F(s') ds'
\end{equation}
\begin{equation}
	T_\pi F(s) = \E[\pi]{F(S_{t+1})|S_t = s} = \int_\A \pi(s,a) \int_\S \calP(s,a,s') F(s') ds'	da
\end{equation}
These equations can be rewritten as fixed-point equations which, under some formal assumptions on the reward functions, admit a unique solution by the contraction mapping theorem. The agent's goal is to select a policy $\pi_*$ that maximizes his expected return in all possible states. Such a policy is called \emph{optimal} and the corrisponding value functions are called \emph{Optimal State-Value Function}
\begin{equation}
	V_*(s) = \sup_\pi V_\pi(s)
\end{equation}
and \emph{Optimal Action-Value Function}
\begin{equation}
	Q_*(s,a) = \sup_\pi Q_\pi(s,a)
\end{equation}
The optimal value functions satisfy the following Bellman equations.
\begin{equation}
	V_*(s) = \sup_a Q_*(s,a) = \sup_a \left\{\calR(s,a) + \gamma T_a V_*(s)\right\}
\end{equation}
\begin{equation}
	\begin{split}
		Q_*(s,a) &= \calR(s,a) + \gamma T_a V_*(s)\\
			 &= \calR(s,a) + \gamma \int_\S \calP(s,a,s') \sup_{a'} Q_*(s', a') ds'
	\end{split}
\end{equation}
Again, these are fixed-point equations for which the existence and uniqueness of a solution is guaranteed by the contraction mapping theorem. Given the optimal action-value function $Q_*$, an optimal policy is obtained by selecting in each state the action with maximizes $Q_*$
\begin{equation}
	a_* = \argsup_a Q_*(s,a)
\end{equation}
This greedy policy is deterministic and only depends on the current state of the system.

\subsection{Policy Gradient Methods}
The standard way to solve MDPs is through dynamic programming, which simply consists in solving the Bellman fixed-point equations discussed in the previous chapter. Following this approach, the problem of finding the optimal policy is transformed into the problem of finding the optimal value function. However, apart from the simplest cases where the MDP has a limited number of states and actions, dynamic programming becomes computationally infeasible. Moreover, this approach requires complete knowledge of the Markov transition kernel and of the reward function, which in many real-world applications might be unknown or too complex to use. \emph{Reinforcement Learning} (RL) is a subfield of Machine Learning which aims to turn the infeasible dynamic programming methods into practical algorithms that can be applied to large-scale problems. RL algorithms are based on two key ideas: the first is to use samples to compactly represent the unknown dynamics of the controlled system. The second idea is to use powerful function approximation methods to compactly estimate value functions and policies in high-dimensional state and action spaces. In this section we will only focus on a particular class of algorithms called \emph{Policy Gradient Methods}, which have proved successful in many applications. For a more complete introduction to RL, the reader may consult \cite{sutton1998introduction}, \cite{szepesvari2010algorithms} or \cite{wiering2012reinforcement}.\\
In \emph{policy gradient methods} \cite{peters2008reinforcement}, the optimal policy is approximated using a parametrized policy $\pi: \S \times \calA \times \Theta \to \R$ such that, given a parameter vector $\theta \in \Theta \subseteq \R^{D_\theta}$, $\pi(s, B; \theta) = \pi_\theta(s, B)$ gives the probability of selecting an action in $B \in \calA$ when the system is in state $s \in \S$.
The general goal of policy optimization in reinforcement learning is to
optimize the policy parameters $\theta \in \Theta$ so as to maximize a certain
objective function $J: \Theta \to \R$
\begin{equation}
	\theta^* = \argmax_{\theta \in \Theta} J(\theta)
\end{equation}
In the following, we will focus on gradient-based and model-free methods that exploit
the sequential structure of the the reinforcement learning problem. The idea of
policy gradient algorithms is to update the policy parameters using the gradient ascent direction of the objective function
\begin{equation}
	\theta_{k+1} = \theta_k + \alpha_k \nabla_\theta J\left(\theta_k\right)
\end{equation}
where $\{\alpha_k\}_{k\geq 0}$ is a sequence of learning rates. Typically, the
gradient of the objective function is not known and its approximation is the key component of every policy gradient algorithm. It is a well-know result from stochastic optimization \cite{kushner2003stochastic} that, if the gradient estimate is unbiased and the learning rates satisfy the \emph{Robbins-Monro conditions}
\begin{equation}
	\sum_{k=0}^\infty \alpha_k = \infty \;\;\;\;\;\; \sum^{\infty}_{k=0}
	\alpha_k^2 < \infty
\end{equation}
the learning process is guaranteed to converge at least to a local optimum of
the objective function. In an episodic environment where the system always starts from an initial state $s_0$, the typical objective function is the start value.
\begin{equation}
	J_{\text{start}}(\theta) = V_{\pi_\theta}(s_0) = \E[\pi_\theta]{G_0 |
 S_0 = s_0}
\end{equation}
In a continuing environment, where no terminal state exists and the task might go on forever, it is common to use either the average value
\begin{equation}
	J_{\text{avV}}(\theta) = \E[S \sim d^{\theta}]{V_{\pi_\theta}(S)} = \int_\S
	d^{\theta}(s) V_{\pi_\theta}(s) ds
\end{equation}
where $d^\theta$ is the stationary distribution of the Markov chain induced by $\pi_\theta$. Alternatively, one may use the average reward per time step
\begin{equation}
	J_{\text{avR}}(\theta) = \rho(\theta) = \E[\substack{S \sim d^{\theta}\\A \sim \pi_\theta}]{\calR(S,A)}
	= \int_\S d^{\theta}(s) \int_\A \pi_\theta(s,a) \calR(s,a) da ds
\end{equation}
Luckily, the same methods apply with minor changes to the three objective functions.

\subsubsection{Policy Gradient Theorem}
The \emph{policy gradient theorem} \cite{sutton1999policy} shows that the gradient can be rewritten in a form suitable for estimation from experience aided by an approximate action-value or advantage function.
\begin{theorem}[Policy Gradient]
\label{thm:risk_neutral_policy_gradient}
	Let $\pi_\theta$ be a differentiable policy. The policy gradient for the average reward formulation is given by
	\begin{equation}
		\nabla_\theta \rho(\theta) =
		\E[\substack{S \sim d^\theta\\A \sim \pi_\theta}]{\nabla_\theta\log
		\pi_\theta(S,A) Q_{\theta}(S, A)}
	\end{equation}
	where $d^\theta$ is the stationary distribution of the Markov chain induced by $\pi_\theta$. The policy gradient for the start value formulation is given by
	\begin{equation}
		\nabla_\theta J_{\text{start}}(\theta) =
		\E[\substack{S \sim d_\gamma^\theta(s_0, \cdot)\\A \sim \pi_\theta}]{\nabla_\theta\log
		\pi_\theta(S,A) Q_{\theta}(S, A)}
	\end{equation}
	where $d_\gamma^\theta(s_0, \cdot)$ is the $\gamma$-discounted visiting distribution over states starting from the initial state $s_0$ and following policy $\pi_\theta$
		\begin{equation}
			d_\gamma^\theta(s, x) = \sum_{k=0}^{\infty} \gamma^k \calP_\theta^{(k)}(s, x)
		\end{equation}
\end{theorem}
Let us notice that we can subtract a state-dependent baseline from the action-value function without changing the value of the expectation, indeed
\begin{equation*}
	\begin{split}
	\E[\substack{S \sim d^\theta\\A \sim \pi_\theta}]{\nabla_\theta\log
			\pi_\theta(S,A) B_\theta(S)}
	&= \int_\S d^\theta(s) \int_\A \pi_\theta(s,a) \nabla_\theta\log
				\pi_\theta(s,a) B_\theta(s) da ds\\
	&= \int_\S d^\theta(s)  B_\theta(s) \int_\A \nabla_\theta \pi_\theta(s,a) da ds\\
	&= \int_\S d^\theta(s)  B_\theta(s)  \nabla_\theta  \underbrace{\int_\A  \pi_\theta(s,a) da}_{= 1} ds = 0
	\end{split}
\end{equation*}
Hence, the policy gradient theorem can be rewritten as
\begin{equation}
\label{eq:pg_theorem_baseline}
	\nabla_\theta \rho(\theta) =
	\E[\substack{S \sim d^\theta\\A \sim \pi_\theta}]{\nabla_\theta\log
	\pi_\theta(S,A) \left(Q_{\pi_\theta}(S, A) - B_\theta(S)\right)}
\end{equation}
The baseline can be chosen so as to minimize the variance of the gradient estimate which can prove beneficial for the algorithm convergence \cite{peters2008reinforcement}. This result can be used as the starting point to derive several policy gradient methods that use different approximation of the action-value function, which is typically unknown. For instance, in an episodic MDP the action-value function can be estimated with the total return obtained on a sample trajectory
\begin{equation}
	Q_\theta(s_0,a_0) \approx \sum_{t=0}^{T^{(m)}} \gamma^t r_{t+1}^{(m)}
\end{equation}
Combining this remark with a Monte Carlo approximation of Eq. (\ref{eq:pg_theorem_baseline}), we obtain the \emph{Monte Carlo Policy Gradient} algorithm \cite{baxter2001infinite} (also known as GPOMDP) for which the pseudocode is reported in Algorithm \ref{algo:GPOMDP}.
\begin{algorithm}[t]
	\caption{GPOMDP}
	\label{algo:GPOMDP}
	\begin{algorithmic}[0]
		\Require{\\
			\begin{itemize}
				\item Initial policy parameters $\theta_0 = (\theta_0^1, \ldots, \theta_0^{D_\theta})^T$
				\item Learning rate $\{\alpha_k\}$
				\item Number of trajectories $M$
			\end{itemize}
		}
		\Ensure Approximation of the optimal policy $\pi_{\theta^*} \approx \pi_*$
		\begin{algorithmic}[1]
		\State Initialize $k = 0$
		\Repeat
			\State Sample $M$ trajectories $h^{(m)} = \{(s_t^{(m)}, a_t^{(m)}, r_{t+1}^{(m)}\}_{t = 0}^{T^{(m)}}$ of the MDP under policy $\pi_{\theta_k}$
			\State Compute the optimal baseline
			\begin{equation}
				\widehat{b}_k^n = \frac{\sum^{M}_{m=1} \left[ \sum_{i=0}^{T^{(m)}}
									\partial_{\theta_k} \log \pi_\theta\left(s_i^{(m)}, a_i^{(m)}\right) \right]^2
									\sum^{T^{(m)}}_{j=0} \gamma^j r_{j+1}^{(m)}}{\sum^{M}_{m=1} \left[ \sum_{i=0}^{T^{(m)}} \partial_{\theta_k} \log \pi_\theta\left(s_i^{(m)}, a_i^{(m)}\right) \right]^2}
			\end{equation}
			\State Approximate policy gradient
			\begin{equation}
				\frac{\partial}{\partial\theta^n} J_{\text{start}}(\theta_k) \approx \widehat{g}_k^n = \frac{1}{M} \sum^{M}_{m=1} \sum_{i=0}^{T^{(m)}}
					\frac{\partial}{\partial\theta^n} \log \pi_{\theta_k}\left(s_i^{(m)}, a_i^{(m)}\right) \left(
					\sum^{T^{(m)}}_{j=i} \gamma^j r_{j+1}^{(m)} - \widehat{b}_k^n \right)
			\end{equation}
			\State Update actor parameters $\theta_{k+1} = \theta_k + \alpha_k \widehat{g}_k $.

			\State $k \leftarrow k + 1$
		\Until{converged}
		\end{algorithmic}
	\end{algorithmic}
\end{algorithm}

\subsubsection{Parameter-Based Policy Gradient Methods}
In Monte Carlo Policy Gradient, trajectories are generated by sampling at each
time step an action according to a stochastic policy $\pi_\theta$ and the
objective function gradient is estimated by differentiating the policy with
respect to the parameters. However, sampling an action from the policy at each
time step leads to a large variance in the sampled histories and therefore in
the gradient estimate, which can in turn slow down the convergence of the
learning process. To address this issue, the \emph{policy gradient with parameter-based exploration} (PGPE) method \cite{sehnke2008policy} replaces the search in the policy space with a direct search in the model parameter space. Given an episodic MDP, PGPE considers a deterministic controller $F: \S \times \Theta \to \A$ that, given a set of parameters $\theta \in \Theta \subseteq \R^{D_\theta}$, maps a state $s \in \S$ to an action $a = F(s; \theta) = F_\theta(s) \in \A$. The policy parameters are drawn from a probability distribution $p_\xi$, with hyper-parameters $\xi \in \Xi \subseteq \R^{D_\xi}$. Combining these two hypotheses, the agent follows a stochastic policy $\pi_\xi$ defined by
\begin{equation}
	\forall B \in \calA ,\ \pi_\xi(s,B) = \pi(s, B; \xi) = \int_\Theta p_\xi(\theta)
	\ind{F_{\theta}(s)\in B} d\theta
\end{equation}
In this setting, the policy gradient theorem can be reformulated in the following way
\begin{theorem}[Parameter-Based Policy Gradient]
	Let $p_\xi$ be differentiable with respect to $\xi$, then the gradient of the average reward is given by
	\begin{equation}
		\nabla_\xi J(\xi) = \E[\substack{S \sim d^\xi\\\theta \sim p_\xi}]{\nabla_\xi \log p_\xi(\theta) Q_{\pi_\xi}(S, \theta)}
	\end{equation}
	where we denoted $Q_\xi(S, \theta) = Q_\xi(S, F_\theta(S))$.
\end{theorem}
This expression is very similar to the original policy gradient theorem, but
the expectation is taken over the controller parameters instead of the action space and we have the likelihood score of the controller parameters distribution instead of that of the stochastic policy. Thus, we might interpret this result as if the agent directly selected the parameters $\theta$ according to a policy $p_\xi$, which then lead to an action through the deterministic mapping $F_\theta$. Therefore, it is as if the agent's policy was in the parameters space and not in the control space. As in the standard policy gradient methods, we can subtract a state-dependent baseline $B_\xi(S)$ to the gradient without increasing the bias
\begin{equation}
	\nabla_\xi J(\xi) = \E{\nabla_\xi \log p_\xi(\theta) \left(Q_{\pi_\xi}(S,
						\theta) - B_\xi(S)\right)}
\end{equation}
The PGPE algorithm, which is outlined in Algorithm \ref{algo:PGPE}, employs a Monte Carlo approximation of this gradient, where the action-value function is estimated using the returns on a sampled trajectory of the MDP. The benefit of this approach is that the controller is deterministic and therefore the actions do not need to be sampled at each time step, with a consequent reduction of the gradient estimate variance. Indeed, It is sufficient to sample the parameters $\theta$ once at the beginning of the episode and then generate an entire trajectory following the deterministic policy $F_\theta$. As an additional benefit, the parameter gradient is
estimated by direct parameter perturbations, without having to backpropagate
any derivatives, which allows to use non-differentiable controllers. Again the baseline can be chosen so as to minimize the gradient estimate variance \cite{zhao2011analysis}.
\begin{algorithm}[t!]
	\caption{Episodic PGPE algorithm}
	\label{algo:PGPE}
	\begin{algorithmic}[0]
		\Require{\\
			\begin{itemize}
				\item Initial hyper-parameters $\xi_0 = (\xi_0^1, \ldots, \xi_0^{D_\xi})^T$
				\item Learning rate $\{\alpha_k\}$
				\item Number of trajectories $M$
			\end{itemize}
		}
		\Ensure Approximation of the optimal policy $F_{\xi^*} \approx \pi_*$
		\begin{algorithmic}[1]
			\State Initialize $k = 0$
			\Repeat
				\For {$m = 1, \ldots, M$}
					\State Sample controller parameters $\theta^{(m)} \sim p_{\xi_k}$
					\State Sample trajectory $h^{(m)} = \{(s_t^{(m)}, a_t^{(m)}, r_{t+1}^{(m)}\}_{t = 0}^{T^{(m)}}$ under policy $F_{\theta^{(m)}}$
				\EndFor
				\State Compute optimal baseline
				\begin{equation}
					\widehat{b}_k^n = \frac{\sum^{M}_{m=1} \left[\partial_{\xi^n} \log p_{\xi_k} \left(\theta^{(m)}\right)\right]^2 \sum^{T^{(m)}}_{j=0} \gamma^j r_{j+1}^{(m)}}{\sum^{M}_{m=1} \left[\partial_{\xi^n} \log p_{\xi_k} \left(\theta^{(m)}\right)\right]^2}
				\end{equation}
				\State Approximate policy gradient
				\begin{equation}
					\frac{\partial}{\partial\xi^n} J_{\text{start}}(\xi_k) \approx \widehat{g}_k^n = \frac{1}{M} \sum^{M}_{m=1}
						\frac{\partial}{\partial\xi^n} \log p_{\xi_k}\left(\theta^{(m)}\right) \left(
						\sum^{T^{(m)}}_{j=i} \gamma^j r_{j+1}^{(m)} - \widehat{b}_k^n \right)
				\end{equation}
				\State Update hyperparameters using gradient ascent $\xi_{k+1} = \xi_k + \alpha_k \widehat{g}_k^n$
				\State $k \leftarrow k + 1$
			\Until{converged}
		\end{algorithmic}
	\end{algorithmic}
\end{algorithm}


\subsection{Example: Four-bar linkage Mechanism}

\begin{frame}
	\begin{block}{Example 2: Four-bar linkage Mechanism}
		\begin{table}
			\begin{minipage}{0.5\linewidth}
				\begin{tabular}{l|l}
					& $l_{AB}=l_1=0.15m$\\
					& $l_{BC}=l_2=0.35m$\\
					& $l_{CD}=l_3=0.3m$\\
					Given & $l_{CE}=l_4=0.15m$\\
					& $x_D=0.3m$\\
					& $y_D=0.3m$\\
					& $\theta_1=45^ {\circ}$\\ \hline
					Find & $\vb{r}{B}$, $\vb{r}{C}$, $\vb{r}{E}$\\
				\end{tabular}
			\end{minipage}\hfill
			\begin{minipage}{0.5\linewidth}
				\begin{figure}
					\includegraphics[width=50mm]{images/R-RRR.png}
				\end{figure}
			\end{minipage}
		\end{table}
	\end{block}
\end{frame}

\begin{frame}
\emph{Solution}\vskip .25cm
Position of joint $B$:  $\displaystyle \vb{r}{B} = x_B\ih + y_B\jh = l_1\cos{\theta_1}\ih + l_1\sin{\theta_1}\jh$\\
Position of joint $C$:  $\displaystyle \vb{r}{C} = x_C\ih + y_C\jh = 0.1\jh$\\
Position of joint $D$: $\displaystyle \vb{r}{D} = x_D\ih + y_D\jh$\\
Position of joint $E$: $\displaystyle \vb{r}{E} = x_E\ih + y_E\jh$\\
\[
\Rightarrow\begin{cases}
(x_B-x_C)^2+(y_B-y_C)^2=l_1^2\\
(x_D-x_C)^2+(y_D-y_C)^2=l_3^ 2
\end{cases}
\]
Solving the system of equations yields $x_{C_1}$ and $x_{C_2}$. Notice that in the mechanism, $0>x_C$ is the condition to obtain correct solution.
\[
\Rightarrow \begin{cases}
(x_C-x_E)^2 + (y_C-y_E)^2 = l_4^2\\
\displaystyle \frac{y_C-y_E}{x_C-x_E}=\frac{y_D-y_E}{x_D-x_E}
\end{cases}
\]
Solving the system of equations yields $x_{E_1}$ and $x_{E_2}$. Notice that in the mechanism, $x_C>x_E$ is the condition to obtain correct solution.
\end{frame}

\begin{frame}{MATLAB R2019a code}
\lstinputlisting[style=Matlab-editor, basicstyle=\mlttfamily]{codes/RRRR-position1.m}
\end{frame}
\begin{frame}
\lstinputlisting[style=Matlab-editor, basicstyle=\mlttfamily]{codes/RRRR-position2.m}
\end{frame}
\begin{frame}{Plotting using MATLAB R2019a}
\lstinputlisting[style=Matlab-editor, basicstyle=\mlttfamily]{codes/RRRR-plot.m}
\end{frame}
\begin{frame}{Output figure}
\centering
\includegraphics[width=100mm]{images/RRRR-plot.png}
\end{frame}
\begin{frame}{Trajectory plotting using MATLAB R2019a}
\lstinputlisting[style=Matlab-editor, basicstyle=\mlttfamily]{codes/RRRR-trajectory.m}
\end{frame}
\begin{frame}{Output figure}
\centering
\includegraphics[width=100mm]{images/RRRR-trajectory.png}
\end{frame}


%! TEX root = master.tex

\lecture{5}{Wed 03 Nov 2021 10:45}{Determinants}

\subsection*{Definitions}

Let \matri{A} be a square matrix. The scalar $\det(A)$ or $\mod{A}$  is called the \textbf{determinant} of \matri{A}.

For a  $2 \times 2$ square matrix, the determinant is given by $a_{11}a_{22} - a_{12}a_{21}$. The plot thickens very quickly as we try to determine the determinant of larger $n \times n$ matrices.\\

The process of evaluating the determinant of an $n \times n$ matrix is described by the following steps:
\begin{enumerate}
  \item The \textbf{minor} $\matr{M_{jk}}$ is the matrix obtained from a square matrix when omitting the $j^{th}$ row and the $k^{th}$ column.
  \item The scalar quantity referred to as the \textbf{cofactor} $\alpha_{jk} = (-1)^{j+k} \det(\matr{M}_{jk})$
\end{enumerate}
The determinant of an $n \times n$ matrix \matri{A}, when choosing the first row as the starting point is therefore given by
    \begin{align*}
  \det(\matr{A}) &= a_{11}\matr A_{11} + a_{12}\matr A_{12} + \cdots + a_{1n}\matr A_{1n} \\
                 &= \sum_{i=1}^n a_{1r}\matr A_{1r}
\end{align*}
The same value of the determinant is obtained if the first point of reference is chosen to be any other row or column. To minimize computation complexity, the row or column with the most entries equal to 0 should be chosen with the result of reducing the maximum number of terms in the sequence to 0.

\subsection*{Properties of Determinants}

\begin{enumerate}
  \item [P1.] $\det(\matr{A}) = \det(\matr{A}^T)$
  \item [P2.] If \matri{A} is an $n \times n$ triangular matrix, then $\det(\matr{A}) = a_{11}a_{22}\cdots a_{nn}$.
  \item [P3.] The interchanging of any two consecutive rows only alters the sign of of the determinant without affecting its magnitude.
  \item [P4.] If any two rows consecutive rows or columns of an $n \times n$ matrix \matri{A} are equal, then $\det(\matr{A}) = 0$.
\end{enumerate}
\newpage
  \begin{theorem}\label{thm:scalar-out}
    If all elements of one row are multiplied by a constant $k$, then the value of the determinant is also multiplied by k.
  \end{theorem}

\begin{proof}
    Let \matri{K} be the general $n \times n$ matrix \matri{A} but with all entries in the first row multiplied by $k$. Then,
    \begin{align*}
      \mod{K} &=
    \begin{vmatrix}
      ka_{11} & ka_{12} & \cdots & ka_{1n}\\
      a_{21} & a_{22} & \cdots & a_{2n}\\
      \vdots & \vdots & \ddots & \vdots\\
      a_{n1} & a_{n2} & \cdots & a_{nn}
    \end{vmatrix}\\
              &= ka_{11}A_{11} + ka_{12}A_{12} + \cdots + ka_{1n}A_{1n}\\
              &= k(a_{11}A_{11} + a_{12}A_{12} + \cdots + a_{1n}A_{1n})\\
              &= k\mod{A}
    \end{align*}
\end{proof}

\corollary{If any two rows of an $n \times n$ matrix \matri A are multiples of the other (\emph{are linearlly dependent}), then the $\mod{A} = 0$.}
\corollary{If \matri{A} is an $n \times n$ matrix, then $\det{(k\matr A)} = k^n \det(\matr A)$.}
\vspace{2em}
\begin{theorem}
 If a distinct scalar $\alpha_{ij}$ is added to all the entries of a row of an $n \times n$ matrix \matri{A}, then $\det(\matr A)$ is equal to
 \[  \begin{vmatrix}
      a_{11} & a_{12} & \cdots & a_{1n}\\
      a_{21} & a_{22} & \cdots & a_{2n}\\
      \vdots & \vdots & \ddots & \ldots\\
      a_{n_1} & a_{n_2} & \cdots & a_{nn}\\
    \end{vmatrix}
    +
    \begin{vmatrix}
      \alpha_{11} & \alpha_{12} & \cdots & \alpha_{1n}\\
      a_{21} & a_{22} & \cdots & a_{2n}\\
      \vdots & \vdots & \ddots & \vdots\\
      a_{n_1} & a_{n_2} &\cdots& a_{nn}
      \end{vmatrix}.\]
\end{theorem}
\begin{proof}
  \[
    \text{Consider\ \ \ }
    \begin{vmatrix}
      a_{11}+\alpha_{11} & a_{12}+\alpha_{12} & \cdots & a_{1n}+\alpha_{1n}\\
      a_{21} & a_{22} & \cdots & a_{2n}\\
      \vdots & \vdots & \ddots & \ldots\\
      a_{n_1} & a_{n_2} & \cdots & a_{nn}\\
    \end{vmatrix}\\
  \]
  \begin{align*}
    &= \left(a_{11} + \alpha_{11}\right)\matr A_{11} + \left(a_{12} + \alpha_{12}\right)A_{12} + \cdots + \left( a_{1n} + \alpha_{1n} \right)\matr A_{1n}\\
    &= (a_{11}\matr A_{11} + a_{12}\matr A_{12} + \cdots + a_{1n}\matr A_{1n}) + \left(\alpha_{11}\matr A_{11} + \alpha_{12}\matr A_{12} + \cdots + \alpha_{1n}\matr A_{1n}\right)\\
  \end{align*}
\end{proof}
\newpage
\begin{theorem}\label{thm:row-distr}
  The value of a determinant is unchanged if we add to the entries of any row the same multiple of the corresponding entries of another row.

\begin{center}
  If $\mod{A} =
  \begin{vmatrix}
    a_{11} & a_{12} & \cdots & a_{1n}\\
    a_{21} & a_{22} & \cdots & a_{2n}\\
    \vdots & \vdots & \ddots & \vdots\\
    a_{n1} & a_{n2} & \cdots & a_{nn}
  \end{vmatrix}$, then $
  \begin{vmatrix}
    a_{11} + ka_{21} & a_{12}+ka_{22} & \cdots & a_{1n} + ka_{2n}\\
    a_{21} & a_{22} & \cdots & a_{2n}\\
    \vdots & \vdots & \ddots & \vdots\\
    a_{n1} & a_{n2} & \cdots & a_{nn}
  \end{vmatrix} = \mod{A}$
\end{center}

\end{theorem}
\begin{proof}
Let  $\matr K =\begin{vmatrix}
    a_{11} + ka_{21} & a_{12}+ka_{22} & \cdots & a_{1n} + ka_{2n}\\
    a_{21} & a_{22} & \cdots & a_{2n}\\
    \vdots & \vdots & \ddots & \vdots\\
    a_{n1} & a_{n2} & \cdots & a_{nn}
  \end{vmatrix}$. Then, by Theorem ~\ref{thm:row-distr}, \[\mod{K} = \mod{A} +
\begin{vmatrix}
    ka_{21} & ka_{22} & \cdots & ka_{2n}\\
    a_{21} & a_{22} & \cdots & a_{2n}\\
    \vdots & \vdots & \ddots & \vdots\\
    a_{n1} & a_{n2} & \cdots & a_{nn}
  \end{vmatrix}.\]
  Then, by Theorem \ref{thm:scalar-out} and P4,
$
  \begin{vmatrix}
    ka_{21} & ka_{22} & \cdots & ka_{2n}\\
    a_{21} & a_{22} & \cdots & a_{2n}\\
    \vdots & \vdots & \ddots & \vdots\\
    a_{n1} & a_{n2} & \cdots & a_{nn}
  \end{vmatrix}
  =
  k\begin{vmatrix}
    a_{21} & a_{22} & \cdots & a_{2n}\\
    a_{21} & a_{22} & \cdots & a_{2n}\\
    \vdots & \vdots & \ddots & \vdots\\
    a_{n1} & a_{n2} & \cdots & a_{nn}
  \end{vmatrix}
  = 0
$
~\\~\\~\\Therefore, $\mod{K} = \mod{A}$.
\end{proof}
\begin{theorem}\label{thm:det-distr}
  The determinant of the product of two matrices is equal to the product of the two determinants.
  $$\det(\matr{AB}) = \det(\matr A) \det(\matr B)$$
\end{theorem}
\begin{proof}
  This proof is left as an exercise to the reader for the $2 \times 2$ matrix case.
\end{proof}

\begin{theorem}
  If $\matr A \inverse$ is invertible, then $\mod{A \inverse} = \frac{1}{\det{(\matr A)}}$
\end{theorem}

\begin{proof}
  \matri A is invertible. Thus
  \begin{align*}
    \exists\ \matr B \text{ s.t. } \matr{AB} &= I\\
    \det{(\matr{AB})} &= \det{(I)}\\
    \det(\matr A) \cdot \det(\matr B) &= 1\\
    \det(\matr B) &= \frac{1}{\det(\matr A)}
 \end{align*}
\end{proof}
\remark{An $n \times n$ matrix is invertible $\Leftrightarrow \det(\matr A) \neq 0$}


\section{(Linear) Regression \quad
{\normalfont\sffamily model: $\color{section-text-color} \hat{\mathbf y} = \bm X \beta$}}

% ===

Assuming $\bm X^\top \bm X$ non-singular. %$\hat\beta\ped{Linear}$ has smallest variance among linear \underline{unbiased} estimates.

Bayesian view:\;
$(Y \mid X,\beta) \sim \Gauss{\bm x^\top \beta, \sigma^2 \mathbb I}$

Distrib. of estimator
$\hat\beta\ped{LS} \sim \Gauss{\beta, (\bm X^\top \bm X)^{-1} \sigma^2}$

\emph{Ridge:}
$\epsilon\ped{RSS}(\beta,{\color{OrangeRed}{\lambda}}) = \paren{\bm y - \bm X^\top \!\beta}^\top \paren{\bm y - \bm X^\top \!\beta} \color{OrangeRed} {+} \lambda \beta^\top \!\beta$
\\
$\hat\beta = (\bm X^\top \bm X \begingroup \color{OrangeRed} + \lambda \mathbb I \endgroup)^{-1} \bm X^\top \bm y$,
\quad prior: $\beta \sim \Gauss{0, \frac{\sigma^2}{\lambda} \mathbb I}$

\emph{(Ridge) Shrinkage:}
Decompose $\bm X = \bm{UDV}^\top$\\
$\bm X \hat\beta = \bm{UD} (\bm D^2 {+} \begingroup \color{OrangeRed} \lambda \mathbb I \endgroup)^{-1} \bm{DU}^\top \bm y = \sum\limits_{j\leq d} \bm u_j \frac{d_j^2}{d_j^2 \begingroup \color{OrangeRed} + \lambda \endgroup} \bm u_j^\top \bm y$

\emph{Lasso:}
%$\epsilon\ped{RSS}(\beta,\lambda) = \sum_{i\leq n} (y_i - x_i^\top \beta)^2 + \lambda \norm{\beta}_1$
$\hat\beta = \arg\min_\beta \sum_{i\leq n} (y_i - x_i^\top \beta)^2 \color{OrangeRed} + \lambda \norm{\beta}_1$
\\
\textit{(no closed form)},
\hfill prior: $p(\beta_i) {=} \frac{\lambda}{4\sigma^2} \exp(-\abs{\beta_i} \frac{\lambda}{2\sigma^2})$

\emph{Bias-variance:}
$\E[D]{ \E[Y\mid X=x]{ (\hat f(x) - Y)^2 } }$
{\small ${=} \E[D]{ (\hat f(x) {-} \E[D]{\hat f(x)})^2 } + \paren*{ \E[D]{\hat f(x)} {-} \E{Y\vert X=x} }^2$}
$+ \E{ (Y - \E{Y\vert X=x})^2 }$
$= variance + bias^2 + noise$

\emph{Gauss-Markov Theorem:}\\
For any linear estimator $\widetilde\theta = c\!^\top \bm y = a\!^\top (\hat\beta {+} \bm{Dy})$ that is unbiased for $a^\top \beta$, it holds: $\V{a^\top \hat\beta} \leq \V{c^\top \bm y}$.

Among all linear \textbf{u}-estimators, $\hat\beta\ped{LS}$ minimises the gen. error!
\enskip
What about \textbf{biased} estimators? We \rotatebox[origin=c]{45}{$\to$} bias a bit in the hope that the variance \rotatebox[origin=c]{-45}{$\to$}.

\emph{Combining Regressors:} $\hat f(x) \coloneqq \frac1B \sum_{i\leq B} \hat f_i(x)$\\
$\mathrm{bias} [\hat f(x)] = \frac1B \sum \mathrm{bias} [\hat f_i(x)]$\\
$\V{\hat f} = \frac{1}{B^2} \sum \V[D]{\hat f_i} + \frac{1}{B^2} {\sum\sum}_{i\neq j} \Cov{\hat f_i, \hat f_j} \approx \frac{\sigma^2}{B}$

% ===


%!TEX root =  ../main.tex

\subsection{Zeros}

\objective{Graph, read, and produce rational functions}

A rational function is one defined by $\frac{f(x)}{g(x)}$ where $f(x)$ and $g(x)$ are both
polynomials.  Rational functions exhibit some common behaviors with polynomials,
such as $x$-intercepts and one $y$-intercept.  But they also tend to have vertical and
horizontal asymptotes.  The most basic rational function --- $\frac{1}{x}$ --- is, in fact,
a hyperbola, with two branches.  But rational functions can have many more than two
branches, or even just have one.

There are three things one could consider with regard to 0 and a given rational function.
First, the entire quotient could be 0, which would make $x$-intercepts.  Fortunately,
a quotient is only equation to zero when the numerator is equal to zero, meaning
we only need to find when top equation's zeros to find the whole equation's.

When the denominator is equal to zero, much more bizarre behavior emerges.  A vertical
asymptote will occur, though it is not immediately clear if the left and right limits will be
to opposite infinities, or to the same one.  Factoring will make it clear: if the term
has an even multiplicity, the asymptote will be followed to the same infinity.  Otherwise,
one sided-limit will be $\infty$ and the other $-\infty$.

Lastly, we should consider what happens when we make the argument 0.  A $y$-intercept
is always a coordinate with the appearance $(0,y_0)$, so plugging in 0 is an efficient way
to find it.

Each of these three techniques for various ``zero's'', assumes that it is not occurring when
function is in an indeterminate form.  Holes and asymptotes can prevent intercepts from
actually occurring, and obfuscate one another

\subsection{Horizontal Asymptotes}
If a rational function looks like a fraction, a ratio, a quotient, that's because it is!  Just as
a polynomial looks more and more like an integer power function the further out one zooms,
so too a rational function will look more like the results if one simply divides the numerator
by the denominator.

Consider the function $r(x) = \dfrac{2x^2-4x-6}{x^2+x-2}$.  In the big picture,
it is a simple division problem.  Ultimately, we want to know $lim_{x\rightarrow\infty}
r(x)$:

\polylongdiv{2x^2-4x-6}{x^2+x-2}

and so the question becomes equivalent to

$$
\lim_{x\rightarrow\infty} 2-\frac{6x+2}{x^2+x-2}
$$

which is two.

We might have spared ourselves the work of polynomial long division by contrasting the degree
of the numerator and denominator beforehand.  Which has greater degree?
\begin{itemize}
\item[\textbf{denom.}] the long term behavior can be modeled by a horizontal asymptote at $y=0$
\item[\textbf{neither}] the long tern behavior will still be a horizontal asymptote at $y=\frac{a}{b}$, where $a$ and $b$ are the leading terms of the numerator and denominator respectively
\item[\textbf{numer.}] polynomial division cannot be avoided, and the long-run behavior trends towards an asymptote at $y=$ the quotient, ignoring the remainder
\end{itemize}

\subsection{Signs}
Rational functions are among the hardest for graphers to correctly display.
Even large computers can display misleading results if the window is not correctly
pre-programmed.  If it is necessary to map out the sign of the function beforehand,
a tabular approach can be very helpful.  We will combine a number line and a table
into a new kind of grid, using the following function as an example:

$$Y_1=\frac{x+1}{(10x-1)^2}$$

\begin{figure}[h]
\begin{centering}
\begin{tikzpicture}

\matrix (m) [matrix of math nodes,
             column sep=0cm, row sep=0pt,
     nodes={text width=15mm, align=center,
            text height=3ex, text depth=1.5ex}]
{
(x+1)       &   -   &   +   &   +   \\
(10x-1)^2    &   +   &   +   &   +   \\
\textbf{Q}        &   \textbf{-}  &  \textbf{+}    &  \textbf{+}    \\
};
\draw  (m-1-1.north west) -- (m-1-4.north east);
\draw (m-2-1.north west) -- (m-2-4.north east);
\draw (m-3-1.north west) -- (m-3-4.north east);
\draw (m-3-1.south west) -- (m-3-4.south east);

\draw  (m-1-1.north east) -- (m-3-1.south east);
\draw (m-1-2.north east) -- (m-3-2.south east);
\draw (m-1-3.north east) -- (m-3-3.south east);
\draw (m-1-4.north east) -- (m-3-4.south east);

\node[above] at (m-1-2.north east) {$-1$};
\node[above] at (m-1-3.north east) {$0.1$};

\node at (m-1-2.east) {$0$};
\node at (m-2-2.east) {$+$};
\node at (m-3-2.east) {\textbf{0}};

\node at (m-1-3.east) {$+$};
\node at (m-2-3.east) {$0$};
\node at (m-3-3.east) {\textbf{$\infty$}};
\end{tikzpicture}
\caption{A sign table and number-line combination}
\end{centering}
\end{figure}

The table is made by writing the factors of the numerator and denominator down the
left.  The bottom row is the function itself, which is the quotient of the rest.  Across
the top, you will need a division for each of the zeros of the factors and room
for all the spaces in-between and around.  On the lines, write the zero's if the factor
is zero there, or just the sign.  To complete the bottom, simply remember the
rules of  signs:

\begin{enumerate}
\item a positive multiplied or divided by a positive is a positive
\item a negative multiplied or divided by a negative is a negative
\item a negative multiplied or divided by a negative is positive
\end{enumerate}

The zero's require only a little more thought.  A zero in the numerator
makes the entire quotient zero.  If a zero in the numerator is surrounded by
like signs, then either side of the vertical asymptote will go to that sign infinity.
If the signs disagree, then both infinities will be approached, each from the
side matching their sign.

\begin{figure}
\begin{centering}
\includegraphics[scale=0.3]{\chapdir/pics/verticalasymptote}
\caption{An appropriately zoomed in graph of $\frac{x+1}{(10x-1)^2}$}
\end{centering}
\end{figure}

With this much more appropriate window, we can see that there is a local minimum,
perhaps in the vicinity of $x=-2$.  If it indeed occurs at a rational number, there is no
reason to find the derivative by hand with the TI-8* will do it for us!

\index{TI-8*!graph derivatives}
Set $Y_2$ to be \texttt{nDeriv(Y$_1$(X),X,X)} or $\left.\frac{d}{dx}(Y_1(X))\right|_{X=X}$,
depending upon your model of TI-8*.  (nDerive is choice 8 under
\Touche[style=function,principal=math].)  It seems have a
zero near 2.  The function is clearly concave and derivative is increasing, so it should be
a local minimum.  Using ZERO (under
\Touche[style=function,principal=trace,fontsize=7pt,position=0.9]
), we find it to be exactly -2.1.  Plugging
back into the original equation, we get $\frac{-1.1}{484}$, which $\triangleright$FRAC
converts to $-\frac{1}{440}$.


\chapter{Trigonometric Functions}
\section{Radians}
A radian is a measure of the central angle $\theta$ such that the arc $s$
intercepted by the angle is equal to the length of the radius, $r$.

One radian is the measure of a central angle $\theta$ that intercepts an arc $s$
equal to the length of the radis $r$ of the circle: ($\theta$ is measured in
radians)
\begin{equation}
  \theta=\frac{s}{r}
\end{equation}

There are a few key terms that are used when describing angles inside of a
circle:
\begin{description}
  \item[Initial side]{The starting position of the ray}
  \item[Terminal side]{The other side of the ray}
  \item[Vertex]{The intersection of the initial and terminal side}
\end{description}

\subsection{Converting beteen radians and degrees}
\begin{enumerate}
  \item{To convert degrees to radians, simply multiply degrees by
    $\frac{\pi\text{rad}}{180}$}
  \item{To convert radians to degrees, multiply radians by
    $\frac{180}{\pi\text{rad}}$}
\end{enumerate}

\subsection{Linear and Angular Speed}
Arc legth is defined for a length $s$ as the quantity of the length of the
radius, $r$ and the central angle that $s$ is intercepted by, $\theta$:
\begin{equation}
  s=r\theta
\end{equation}

Therefore, linear speed is defined as the quotient of the arc-length and the
time that has passed:
\begin{equation}
  \text{linear speed}=\frac{\text{arc length}}{\text{time}}=\frac{s}{t}
\end{equation}

And as follows, angular speed is the same formula, except we are looking at
radians per second, as in the following equation:
\begin{equation}
  \text{angular speed}=\frac{\text{central angle}}{\text{time}}=\frac{\theta}{t}
\end{equation}

\section{Unit Circle Trigonometry}
Consider a unit circle (a circle of radius $1$ centered around $(0,0)$) defined
by the following equation: $x^2+y^2=1$.

For any point \textit{on} the circle, the coordinate pair that will intersect
the edge of the circle is defined as $P(\theta)=(cos(\theta),sin(\theta))$.

\begin{itemize}
  \item{$sin(t)=y$}
  \item{$cos(t)=x$}
  \item{$tan(t)=\frac{y}{x}$}
  \item{$csc(t)=\frac{1}{y}$}
  \item{$sec(t)=\frac{1}{x}$}
  \item{$cot(t)=\frac{x}{y}$}
\end{itemize}

\subsection{Even and Odd Trigonometric Functions}
\begin{itemize}
  \item{Even:}
    \begin{itemize}
      \item{$cos(-t)=cos(t)$}
      \item{$sec(-t)=sec(t)$}
    \end{itemize}
  \item{Odd:}
    \begin{itemize}
      \item{$sin(-t)=-sin(t)$}
      \item{$csc(-t)=-csc(t)$}
      \item{$tan(-t)=-tan(t)$}
      \item{$cot(-t)=-cot(t)$}
    \end{itemize}
\end{itemize}

\section{Right Triangle Trigonmetry}
Consider a right triangle.  A geometric figure with three sides, two of which
meet at a $90\deg$ angle.  The other side is reffered to as the hypoteneuse, and
the length of that side is equal to $\sqrt{a^2+b^2}$.

There are six trigonometric functions that we will study that relate to right
triangles.  They are defined as follows:
\begin{itemize}
  \item{$sin(\theta)=\frac{\text{opp}}{\text{hyp}}$}
  \item{$cos(\theta)=\frac{\text{adj}}{\text{hyp}}$}
  \item{$tan(\theta)=\frac{\text{opp}}{\text{adj}}$}
  \item{$csc(\theta)=\frac{1}{sin(\theta)}$}
  \item{$sec(\theta)=\frac{1}{cos(\theta)}$}
  \item{$cot(\theta)=\frac{1}{tan(\theta)}$}
\end{itemize}

\subsection{Trigonometric Identities}

\subsubsection{Reciprocal Identities}
This set of identities defines the relationships between pairs of trigonometric
functions as inverses of each other.

\begin{itemize}
  \item{$sin(\theta)=\frac{1}{csc(\theta)}$}
  \item{$cos(\theta)=\frac{1}{sec(\theta)}$}
  \item{$tan(\theta)=\frac{1}{cot(\theta)}$}
  \item{$csc(\theta)=\frac{1}{sin(\theta)}$}
  \item{$sec(\theta)=\frac{1}{cos(\theta)}$}
  \item{$cot(\theta)=\frac{1}{tan(\theta)}$}
\end{itemize}

\subsubsection{Quotient Identites}
This set of identnties defines two trigonometric functions as quotients of other
pairs of trigonometric functions.

\begin{itemize}
  \item{$tan(\theta)=\frac{sin(\theta)}{cos(\theta)}$}
  \item{$cot(\theta)=\frac{cos(\theta)}{sin(\theta)}$}
\end{itemize}

\section{Graphs of Sine and Cosine Functions}
The sine function is defined for all $x\in(-\infty,\infty)$, and has a range of
$[-1,1]$.  It has a period of $2\pi$ and has $x$-intercepts at $(n\pi, 0)$ and
$y$-intercepts at the origin.  It is an odd-function with original symmetry.

On the other hand, the cosine is also defined for all $x\in(-\infty,\infty)$,
and has a range of $[-1,1]$.  It has a period of $2\pi$ and has $x$-intercepts
at $(\frac{\pi}{2}+n\pi,0)$.  It has a $y$-intercept at $(1,0)$ and is an even
function with $y$-axis symmetry.

\subsection{Amplitude}
The amplitude of a function is defined as one half of the distance between its
relative minimum and maximum points.  For any trigonmetric function of the form
$y=a\text{sin}(x)$ or $y=a\text{cos}(x)$, the amplitude is $|a|$.

\subsection{Period}
The period of a function is the time it takes the function to repeat itself, for
any point.  The period itself is not a function, it is a constant value for all
points \textit{across} a function.  For any function that is defined by the
pattern: $y=a\text{sin}(x)$ or $y=a\text{cos}(x)$, the period is defined by the
following equation:
\begin{equation}
  \text{Period}=\frac{2\pi}{b}
\end{equation}

\section{Graphs of other Trigonometric Functions}

\subsection{The Tangent and its Inverse}
The graph of the tangent function is defined for all real numbers $x$, such that
$x\neq\frac{\pi}{2}+n\pi$.  Its range is defined for all
$x\in(-\infty,\infty)$ and has $x$-intercepts at $(n\pi,0)$, and a
$y$-intercept at the origin.  It has vertical asymptotes at
$x=\frac{\pi}{2}+n\pi$ and is an odd function with origianl symmetry.

The function $f(x)=cot(x)$ is exactly the $tan(x)$ function, but shifted
$\frac{\pi}{2}$ units to the right.  That is to say that
$f(x)=cot(x)=tan(x)+\frac{\pi}{2}$.

\subsection{The Cosecant and Secant Functions}
These functions are very easy to reason about.  Take their inverse functions,
and draw plot them on a graph.  Define vertical asymptotes when the origianl
function is $f(x)=0$, and draw points at the relative minimum and maximum across
the original function.  Stretch those points to the asymptotes through
continuous curves, and you have the function that you are trying to sketch.

The same rules apply when applying transformations to these functions.

\section{Inverse Trignonometric Functions}
The inverse sine function, or $f(x)=arcsin(x)$ is defined over the interval
$[-\frac{\pi}{2},\frac{\pi}{2}]$.  Simply take the sine function at those two
endpoints and plot the inverse function.  Apply transformations as normal.

\subsection{Characteristics of Inverse Trigonometric Functions}
\begin{description}
  \item[$f(x)=arcsin(x)$]{Defined over all $x\in[-1,1]$; $-\frac{\pi}{2} \leq
    f(x) \leq \frac{pi}{2}$}
  \item[$f(x)=arccos(x)$]{Defined over all $x\in[-1,1]$; $0 \leq f(x) \leq \pi$}
  \item[$f(x)=arctan(x)$]{Defined over all $x\in(-\infty,\infty)$;
    $-\frac{\pi}{2} \leq f(x) \leq \frac{\pi}{2}$}
\end{description}