\section{Functions of Several Variables}\label{sec:FunctionsSeveralVariables}

In single-variable calculus we were concerned with functions that map
the real numbers $\mathbb{R}$ to $\mathbb{R}$, sometimes called ``real functions of
one variable'', meaning the ``input'' is a single real number and the
``output'' is likewise a single real number. Now we turn to functions of several
variables, where several input variables are mapped to one value: functions
$f\colon\mathbb{R}^n\to\mathbb{R}$. We will deal primarily with $n=2$ and to a lesser
extent $n=3$; in fact many of the techniques we discuss can be applied
to larger values of $n$ as well.

A function $f\colon\mathbb{R}^2\to\mathbb{R}$ maps a pair of values $(x,y)$ to a
single real number. The three-dimensional coordinate system we have
already used is a convenient way to visualize such functions: above
each point $(x,y)$ in the $x$-$y$ plane we graph the point $(x,y,z)$,
where of course $z=f(x,y)$.

\begin{example}{Plane}{PlaneFunction}
Describe the function $f(x,y)=3x+4y-5$.
\end{example}
\begin{solution}
Writing this as
$z=3x+4y-5$ and then $3x+4y-z=5$ we recognize the equation of a
plane. In the form $f(x,y)=3x+4y-5$ the emphasis has shifted: we now
think of $x$ and $y$ as independent variables and $z$ as a variable
dependent on them, but the geometry is unchanged.
\end{solution}

\begin{example}{Sphere}{SphereEquation}
Describe the equation $x^2+y^2+z^2=4$.
\end{example}
\begin{solution}
We have seen that $x^2+y^2+z^2=4$ represents a sphere of radius
2. We cannot write this in the form $f(x,y)$, since for
each $x$ and $y$ in the disk $x^2+y^2<4$ there are two corresponding
points on the sphere. As with the equation of a circle, we can resolve
this equation into two functions, $\ds f_1(x,y)=\sqrt{4-x^2-y^2}$ and
$\ds f_2(x,y)=-\sqrt{4-x^2-y^2}$, representing the upper and lower
hemispheres, respectively. Each of these is an example of a function with a
restricted domain: only certain values of $x$ and $y$ make sense
(namely, those for which $x^2+y^2\le 4$) and the graphs of these
functions are limited to a small region of the plane.
\end{solution}

\begin{example}{Square Root}{MultivariateSquareRootFunction}
Describe the function $f(x,y)=\sqrt x+\sqrt y$.
\end{example}
\begin{solution}
This function is defined only when
both $x$ and $y$ are non-negative. When $y=0$ we get $f(x,y)=\sqrt x$,
the familiar square root function in the $x$-$z$ plane, and when $x=0$
we get the same curve in the $y$-$z$ plane. Generally speaking, we see
that starting from $f(0,0)=0$ this function gets larger in every direction
in roughly the same way that the square root function gets
larger. For example, if we restrict attention to the line $x=y$, we
get $f(x,y)=2\sqrt x$ and along the line $y=2x$ we have $f(x,y)=\sqrt
x+\sqrt{2x}=(1+\sqrt2)\sqrt x$.
\end{solution}

\begin{figure}[H]
%\texonly
\centerline{
\vbox{\beginpicture
\normalgraphs
%\ninepoint
\setcoordinatesystem units <3truecm,3truecm>
\setplotarea x from 0 to 1.1, y from 0 to 1.1
\put {\hbox{\epsfxsize5cm\epsfbox{images/square_root.eps}}} at 0 0
\endpicture}}
%\endtexonly
\caption{$f(x,y)=\sqrt{x}+\sqrt{y}$}
\label{fig:double square root}
\end{figure}

A computer program that plots such surfaces can be very useful, as it
is often difficult to get a good idea of what they look like. Still,
it is valuable to be able to visualize relatively simple surfaces
without such aids. As in the previous example, it is often a good idea
to examine the function on restricted subsets of the plane, especially
lines. It can also be useful to identify those points $(x,y)$ that
share a common $z$-value.

\begin{example}{Elliptic Paraboloid}{EllipticParaboloidFunction}
Describe the graph of $f(x,y)=x^2+y^2$.
\end{example}
\begin{solution}
When $x=0$ this becomes
$f=y^2$, a parabola in the $y$-$z$ plane; when $y=0$ we get the
``same'' parabola $f=x^2$ in the $x$-$z$ plane.

Finally, picking a value $z=k$, at what points does
$f(x,y)=k$? This means $x^2+y^2=k$, which we recognize as the equation
of a circle of radius $\sqrt k$. So the graph of $f(x,y)$ has
parabolic cross-sections, and the same height everywhere on concentric
circles with center at the origin. This fits with what we have already
discovered.
\end{solution}

\begin{figure}[H]
%\texonly
\centerline{
\vbox{\beginpicture
\normalgraphs
%\ninepoint
\setcoordinatesystem units <3truecm,3truecm>
\setplotarea x from 0 to 1.1, y from 0 to 1.1
\put {\hbox{\epsfxsize5cm\epsfbox{images/parabolic_bowl.eps}}} at 0 0
\put {\hbox{\epsfxsize4cm\epsfbox{images/parabolic_level_curves.eps}}} at 2 0
\endpicture}}
%\endtexonly
\caption{$f(x,y)=x^2 + y^2$}
\label{fig:parabolic bowl}
\end{figure}

As in this example, the points $(x,y)$ such that $f(x,y)=k$ usually
form a curve, called a \dfont{level curve}\index{level curve} of the
function. A graph of some level curves can give a good idea of the
shape of the surface; it looks much like a topographic map of the
surface. In Figure~\ref{fig:parabolic bowl} both the surface and
its associated level curves are shown. Note that, as with a
topographic map, the heights corresponding to the level curves are
evenly spaced, so that where curves are closer together the surface is
steeper.

Functions $f\colon\mathbb{R}^n\to\mathbb{R}$ behave much like functions of two
variables; we will on occasion discuss functions of three variables.
The principal difficulty with such functions is visualizing them, as
they do not ``fit'' in the three dimensions we are familiar with. For
three variables there are various ways to interpret functions that
make them easier to understand. For example, $f(x,y,z)$ could
represent the temperature at the point $(x,y,z)$, or the pressure, or
the strength of a magnetic field.  It remains useful to consider those
points at which $f(x,y,z)=k$, where $k$ is some constant value. If
$f(x,y,z)$ is temperature, the set of points $(x,y,z)$ such that
$f(x,y,z)=k$ is the collection of points in space with temperature $k$;
in general this is called a \dfont{level set}; for
three variables, a level set is typically a surface, called a \dfont{level surface}.

\begin{example}{Level Surfaces}{LevelSurfacesExample}
Suppose the temperature at $(x,y,z)$ is
$T(x,y,z)=e^{-(x^2+y^2+z^2)}$. This function has a maximum value of 1
at the origin, and tends to 0 in all directions. If $k$ is positive
and at most 1,
the set of points for which $T(x,y,z)=k$ is those points satisfying
$x^2+y^2+z^2=-\ln k$, a sphere centered at the origin. The level
surfaces are the concentric spheres centered at the origin.
\end{example}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:FunctionsSeveralVariables}}

\begin{enumialphparenastyle}

\begin{ex}
Let $f(x,y)=(x-y)^2$.
Determine the equations and shapes of the cross-sections when
$x=0$, $y=0$, $x=y$, and describe the level curves.
Use a three-dimensional graphing tool to graph the surface.
\begin{sol}
$z=y^2$, $z=x^2$, $z=0$, lines of slope 1
\end{sol}
\end{ex}

\begin{ex}
Let $f(x,y)=|x|+|y|$.
Determine the equations and shapes of the cross-sections when
$x=0$, $y=0$, $x=y$, and describe the level curves.
Use a three-dimensional graphing tool to graph the surface.
\begin{sol}
$z=|y|$, $z=|x|$, $z=2|x|$, diamonds
\end{sol}
\end{ex}

\begin{ex}
Let $f(x,y)=e^{-(x^2+y^2)}\sin(x^2+y^2)$.
Determine the equations and shapes of the cross-sections when
$x=0$, $y=0$, $x=y$, and describe the level curves.
Use a three-dimensional graphing tool to graph the surface.
\begin{sol}
$z=e^{-y^2}\sin(y^2)$, $z=e^{-x^2}\sin(x^2)$,
$z=e^{-2x^2}\sin(2x^2)$, circles
\end{sol}
\end{ex}

\begin{ex}
Let $f(x,y)=\sin(x-y)$.
Determine the equations and shapes of the cross-sections when
$x=0$, $y=0$, $x=y$, and describe the level curves.
Use a three-dimensional graphing tool to graph the surface.
\begin{sol}
$z=-\sin(y)$, $z=\sin(x)$,
$z=0$, lines of slope 1
\end{sol}
\end{ex}

\begin{ex}
Let $f(x,y)=(x^2-y^2)^2$.
Determine the equations and shapes of the cross-sections when
$x=0$, $y=0$, $x=y$, and describe the level curves.
Use a three-dimensional graphing tool to graph the surface.
\begin{sol}
$z=y^4$, $z=x^4$,
$z=0$, hyperbolas
\end{sol}
\end{ex}

\begin{ex}
Find the domain of each of the following functions of two variables:
\begin{enumerate}
	\item	$\ds\sqrt{9-x^2}+\sqrt{y^2-4}$
	\item	$\arcsin(x^2+y^2-2)$
	\item	$\ds\sqrt{16-x^2-4y^2}$
\end{enumerate}
\begin{sol}
\begin{enumerate}
	\item	$\{(x,y)\mid |x|\le3\ \hbox{and}\ |y|\ge2\}$
	\item	$\{(x,y)\mid 1\le x^2+y^2\le3\}$
	\item	$\{(x,y)\mid x^2+4y^2\le16\}$
\end{enumerate}
\end{sol}
\end{ex}

\begin{ex}
Below are two sets of level curves.  One is for a cone, one
is for a paraboloid.  Which is which? Explain.

%\texonly
\nobreak
\hbox to \hsize{\hfill\epsfxsize4cm\epsfbox{images/parabolic_level_curves_2.eps}
\hfill\epsfxsize4cm\epsfbox{images/conical_level_curves.eps}\hfill}
%\endtexonly
%\htmlfigure{Partial_Differentiation_two_level_curve_graphs.html}
\end{ex}

\end{enumialphparenastyle}


\chapter{Triangle Quality Metrics}

All the metrics in this section are defined on a triangular element
as illustrated in Figure~\ref{f:tri}.

\begin{figure}[bhp]
  \centering
  \includegraphics[width=2in]{tri}
  \caption{Numbering of vertices and edges on a triangular element.%
                                                                  \label{f:tri}}
\end{figure}

Note that unlike all the other elements that follow,
we name edge vectors of the triangle by the vertex opposite the edge so that
\begin{equation*}
\begin{array}{lcl}
  \vec L_0 &=& \vec P_2 - \vec P_1\\
  \vec L_1 &=& \vec P_0 - \vec P_2\\
  \vec L_2 &=& \vec P_1 - \vec P_0.
\end{array}
\end{equation*}

The triangle edge lengths are denoted as follows:
\[
L_0 = \normvec{L_0}\quad
L_1 = \normvec{L_1}\quad
L_2 = \normvec{L_2}
\]
and the largest and smallest edge lengths are, respectively,
\[
L_{\min} = \min\left(L_0, L_1, L_2\right)
  \rule{2em}{0pt}
L_{\max} = \max\left(L_0, L_1, L_2\right)
\]

The area of a triangle is one half the magnitude of the cross product of any pair of adjacent edge vectors:
\begin{equation*}
  A
    = \frac{1}{2}\normvec{L_0\times\vec L_1}
    = \frac{1}{2}\normvec{L_1\times\vec L_2}
    = \frac{1}{2}\normvec{L_2\times\vec L_0}
\end{equation*}

In addition, we will let $r$ be the inradius
\begin{equation*}
\label{eq:Arp}
  r = \frac{2A}{\normvec{L_0} + \normvec{L_1} + \normvec{L_2}}
\end{equation*}
and $R$ the circumradius
\[
  R = \frac{\normvec{L_0} \normvec{L_1} \normvec{L_2}}%
           {2 r \left(\normvec{L_0} + \normvec{L_1} + \normvec{L_2}\right)}
\]
of the triangle.
These are respectively the radii of the inscribed and circumscribed circles of this triangle.

We will frequently use $n$ to represent some arbitrary edge $L_n$ or vertex $P_n$ of the triangle.
When referring to the next counterclockwise entry $n+1$ (or clockwise entry $n-1$),
we take the result modulo $3$ so that, for example, if $n = 1$, $n+1 = 2$ and $n+2 = 0$.

% -------------------Metric Table-------------------
\newcommand{\trimetrictable}[8]{%
  \begin{center}
  \begin{tabular}{ll}
    \multicolumn{2}{r}{\textbf{\sffamily\Large triangle #1}}\\\hline
    Dimension:                           & #2\\
    Acceptable Range:                    & #3\\
    Normal Range:                        & #4\\
    Full Range:                          & #5\\
    $q$ for equilateral unit triangle:   & #6\\
    Reference:                           & #7\\
    \verd\ function:       & \texttt{#8}\\ \hline
  \end{tabular}
  \end{center}
}

\newpage \input{TriArea}
\newpage \input{TriAspectRatio}
\newpage \input{TriAspectFrobenius}
\newpage \input{TriCondition}
\newpage \input{TriDistortion}
\newpage \input{TriEdgeRatio}
\newpage \input{TriMaximumAngle}
\newpage \input{TriMinimumAngle}
\newpage \input{TriScaledJacobian}
\newpage \input{TriRadiusRatio}
\newpage \input{TriRelativeSizeSquared}
\newpage \input{TriShape}
\newpage \input{TriShapeAndSize}


\documentclass[10pt]{article}
\usepackage{mycommands}
\usepackage{graphicx,wrapfig,listings,fancyhdr, bibentry}
\usepackage[all,dvips,arc,curve,color,frame]{xy}
\pagestyle{fancy}
% \usepackage[top=1.1in, bottom=1.0in, left=1.5in, right=1.5in]{geometry}
\doublespacing
\fancyhead[LO,LE]{T. Hopper}
\fancyhead[CO,CE]{Support Vector Machines}
\fancyfoot[CO,CE]{}
\fancyhead[RO, LE] {\thepage}

\renewcommand{\proofname}{Solution}
 \renewcommand{\labelenumi}{(\alph{enumi})}
\lstset{language=Mathematica}

\begin{document}
 % \unfinished

% \noindent OR 706 \hfill Support Vector Machines \hfill Tim Hopper

Support Vector Machines (SVMs) are statistical machine learning algorithms used, among other applications, to as a binary classifier. Classification algorithms such as the SVM, logistic regression, classification trees, linear discriminate analysis, and neural networks, ``learns'' to distinguish between classes of points in $\R^{n}$ given historic training data.

\subsection{Maximal Margin Separating Hyperplane} % (fold)
\label{ssub:maximal_margin_separating_hyperplane}

% subsubsection maximal_margin_separating_hyperplane (end)
For motivation, consider the two classes of points in $\R^{2}$ in Figure \ref{fig:sep_plot_2}. Note that there are infinitely many lines that could be drawn to separate the red from the green points. Any such line can be described by the equation
\begin{equation}
	f(x)=\beta^{T} x+\beta_{0}=0
\end{equation}
for some $\beta\in \R^{2}$, $\beta_{0}\in \R$, and $x\in \R^{2}$. Simple geometric arguments show that for the green points $x$, $f(x)<0$; likewise, for any red point $f(x)>0$.

\begin{figure}[htb]
				\begin{center}
				\includegraphics[width=300px]{Project/sep_plot_2.pdf}
				\caption{Linearly separable data}
					\label{fig:sep_plot_2}
				\end{center}
				\end{figure}

Moreover, for any two points $x_{1}$ and $x_{2}$ lying on the line $L$ described by $f(x)=0$,
 \[
 	\beta^{T}(x_{1}-x_{2})=-\beta_{0}+\beta_{0}=0,
 \]
and thus $\beta^{*}=\beta/\norm{\beta}$ is unit normal vector to $L$. The signed, normal distance from any point $x\in \R^{2}$ to $L$ is $\frac{1}{\norm{\beta}}(\beta^{T}x+\beta_{0})$.

Suppose our feature set of $N$ points is given by $(x_{i},y_{i})$ where the class of $x_{i}\in\R^{2}$ is given by $y_{i}$ such that $y_{i}=1$ if $x_{i}$  is red and $y_{i}=-1$ if $x_{i}$ is green. This way, for any $i$, $y_{i}(\beta^{T}x_{i}+\beta_{0})$ is the positive distance from $x_{i}$ to $L$. We define the margin of the separating line to be the distance from $L$ to the closest point in $\set{x_{i}}$.

The maximal margin separating hyperplane by solving the program:
\begin{align*}
	\operatorname*{maximize}_{\beta,\beta_{0}, \norm{\beta}=1}\qquad & M \\
	\text{subject to}\qquad & y_{i} (\beta^{T}x_{i}+\beta_{0})\geq M \\
	&i=1,...,N.
\end{align*}
We can integrate the $\norm{\beta}=1$ constraint into inequality constraints as
\[
	\frac{1}{\norm{\beta}}y_{i} (\beta^{T}x_{i}+\beta_{0})\geq M
\]
or
\[
	y_{i} (\beta^{T}x_{i}+\beta_{0})\geq M \norm{\beta}.
\]
Note that for any $\beta, \beta_{0}$ satisfying the inequalities, any positive multiple also satisfies. Thus, we let $\norm{\beta}=1/M$ and give an equivalent program:
\begin{align}
	\label{eq:sephypplan}\operatorname*{maximize}_{\beta,\beta_{0}}\qquad & \frac{1}{2}\norm{\beta}^{2} \\
	\text{subject to}\qquad & y_{i} (\beta^{T}x_{i}+\beta_{0})\geq 1 \nonumber\\
	&i=1,...,N.\nonumber
\end{align}

Observe that the objective function in \eqref{eq:sephypplan} is quadratic and the inequality constraints are nonlinear; thus, the program is convex and can be easily solved. The separating hyperplane program for arbitrary dimensional data can be naively solved in Mathematica:

\begin{lstlisting}
SeparatingHyperplane[x_List, y_List] :=
 Module[
  {conditions, vars, b, w, ans,
   dim = Length[x[[1]]] (* Dimension of training data *),
   n = Length[x](* Size of training data *)},
  vars = Table[w[i], {i, 1, dim}];
  conditions = Join[
    {Norm[vars]^2} (* Cost function *),
    Table[y[[i]] (vars.x[[i]] + b) >= 1, {i, 1, n}]
    ];
  ans = Minimize[conditions, Join[vars, {b}]];
  ans = #[[2]] & /@ ans[[2]];
  {ans[[1;;-2]], ans[[-1]]}
  ]
\end{lstlisting}

The function \textbf{SeparatingHyperplane} returns $\set{\beta_{0}, \beta}$. The problem is more effectively solved with its Lagrangian dual, but we leave that development for the non-separable case below.

\subsection{Non-Separable Data} % (fold)
\label{sub:non_separable_data}

Any hyperplane of parameters $\beta$ ($\norm{\beta}=1$) and $\beta_{0}$ given by
\[
	\set{x \;|\; f(x)={\beta^{T}x+\beta_{0}}=0}
\]
induces a binary classification rule given by
\[
	G(x)=\operatorname{sign}\bra{\beta^{T}x+\beta}.
\]

We now consider a set $\set{\parens{x_{i},y_{i}}}$ of $N$ points where $x_{i}\in R^{n}$ where each $x_{i}$ is of class $y_{i}\in\set{-1,1}$. Unlike our previous example, this data may not be separable by a hyperplane. Nevertheless, we want to find a hyperplane that, in some sense, optimally classifies the data. The program given above cannot handle non-separable data, because misclassifications cause it to be unbounded.

To construct a new program, we permit misclassifications by introducing positive slack variables $\xi=(\xi_{1},\xi_{2},\ldots,\xi_{N})$ and change our constraint to
\[
	y_{i}(\beta^{T}x_{i}+\beta_{0})\geq M(1-\xi_{i}).
\]

Since we want to minimize the number of misclassifications, we penalize the cost function for misclassified test points. Our resulting program is
\begin{align*}
	\operatorname*{minimize}_{\beta,\beta_{0}}\qquad &
	 	\frac{1}{2}\norm{\beta}^{2} +C\sum_{i=1}^{N}\xi_{i}\\
	\text{subject to}\qquad & 	y_{i}(\beta^{T}x_{i}+\beta_{0})\geq 1-\xi_{i} \\
	&\xi_{i}\geq 0\\
	&i=1,...,N
\end{align*}
for some penalizing constant $C$. This is again a convex problem.

\subsubsection{Dual Formulation} % (fold)
\label{ssub:dual_formulation}

We introduce Lagrange multipliers $\alpha_{i}, \mu_{1}$ (for $i=1,\ldots,N$) to give the Lagrangian
\[
	\phi=
		\frac{1}{2}\norm{\beta^{2}}
		+C\sum_{i=1}^{N}\xi_{i}
		-\sum_{i=1}^{N}\alpha_{i}
			\bra{	y_{i}(\beta^{T}x_{i}+\beta_{0})-(1-\xi_{i})}
		-\sum_{i=1}^{N}\mu_{i}\xi_{i}.
\]
We compute the Lagrange dual by setting the gradient with respect to the primal variables equal to zero:
\begin{align}
	\beta&=\sum_{i=1}^{N}\alpha_{i}y_{i}x_{i} \label{eq:dual_con_3}\\
	0&=\sum_{i=1}^{N}a_{i}y_{i} \label{eq:dual_con_1}\\
	\alpha_{i}&=C-\mu_{i}, \;\forall\, i.\label{eq:dual_con_2}
\end{align}

Substituting this into $\phi$, we obtain the Lagrangian dual
\[
	\phi(\alpha,\mu)=\sum_{i=1}^{N}\alpha_{i}-\frac{1}{2}\sum_{i=1}^{N}\sum_{j=1}^{N}\alpha_{i}\alpha_{j}y_{i}y_{j}x_{i}^{T}x_{j}.
	\]
Equations \eqref{eq:dual_con_1} and \eqref{eq:dual_con_2} give constraints for minimizing the dual function. Since $\mu_{i}\geq$, we minimize the dual subject to
\eqref{eq:dual_con_1} and $0\leq a_{i}\leq C$. Because the primal problem is convex, the dual optimal value is also primal optimal.

At the global optimal solution, the KKT conditions guarantee the gradient of the Lagrangian being zero (equations \eqref{eq:dual_con_3}-\eqref{eq:dual_con_2}). Also, primal feasibility and complementary slackness:
\begin{align}
	y_{i}(x_{i}^{T}\beta+\beta_{0})+(1-\xi_{i})&\geq 0\\
	\alpha_{i}\bra{y_{i}(x_{i})^{T}\beta+\beta_{0}-(1-\xi_{i})}&=0\label{eq:comp_slack_1}\\
	\mu_{i}\xi_{i}&=0,
\end{align}
for $i = 1,\ldots, N$.

Note that the dual problem only requires the training data in the form of an inner product. This inner product can be replaced by generalized (Mercer) kernel functions which transform the data into other spaces without significant computational difficulty. The dual formulation even gives a closed formed solution for transforming the data into certain infinite dimensional spaces. Further discussion of kernel methods is outside the scope of this project.

Given a optimal solution to the dual problem $\set{\hat{\alpha}_{i},\hat{\xi}_{i}}$, the primal solution is
\[
	\hat{\beta}=\sum_{i=1}^{N}\hat{\alpha}_{i}y_{i}x_{i},
\]
and $\beta_{0}$ is given by \eqref{eq:comp_slack_1} for any $i$ such that $\alpha_{i}= 0$.
% subsubsection dual_formulation (end)
% subsection non_separable_data (end)

\subsection{Sequential Minimal Optimization Algorithm} % (fold)
\label{sec:smo_algorithm}
In 1998 John Platt of Microsoft Research published a paper  \cite{platt1998sequential} describing his Sequential Minimal Optimization Algorithm for solving the dual problem. The SMO algorithm is a variation on the more general stochastic gradient descent algorithm and is very fast for solving the SVM.

Because of the linear dependence given by \eqref{eq:dual_con_1}, the parameters $\alpha_{i}$ cannot be optimized independently as in stochastic gradient descent. Platt recognized that the linear dependence requires optimization of two parameters $\alpha_{i}$ and $\alpha_{j}$ together.

I have included a \textbf{Mathematica} implementation of Andrew Ng's Simplified SMO algorithm described in \cite{ng_smo}. The simplified SMO does not guarantee under every possible training set. However, except in pathological cases, it provides a satisfactory SVM.

In Figure \ref{fig:svm_plot_2d}, I used my algorithm to find the separating hyperplane for 25 red points sampled from $N((1,1),(1.5,1.5))$ and 25 green points from $N((-1,-1),(1.5,1.5))$. The data has significant overlap and is thus not linearly separable; 45 points are correctly classified, and 5 are misclassified.

\begin{figure}[htb]
				\begin{center}
				\includegraphics[width=300px]{Project/svm_plot_2d.pdf}
				\caption{Non-separable data with linear separator}
					\label{fig:svm_plot_2d}
				\end{center}
				\end{figure}

My code generalizes to data in arbitrary dimension and arbitrary Mercer kernels. In figure \ref{fig:svm_plot_ker}, I used the polynomial kernel $K(x,x')=(1+\ang{x,x'})^{2}$ on the same data as \ref{fig:svm_plot_2d}; 47 points are correctly classified, and 3 are misclassified. Of course, kernel methods are prone to overfitting.

\begin{figure}[htb]
				\begin{center}
				\includegraphics[width=300px]{Project/svm_plot_ker.pdf}
				\caption{Non-separable data with quadratic kernel}
					\label{fig:svm_plot_ker}
				\end{center}
				\end{figure}
% section smo_algorithm (end)

\nocite{burges,platt1998sequential,ng,ng_smo}
\bibliographystyle{plain}
\bibliography{OR706Project}

\end{document}


% corrected VD 85

\subsubsection{Long short-term memory}~\\

An LSTM unit is composed of a memory cell, an input gate, an output gate and a
forget gate. The cell stores information over a long period and the three gates
control the flow of information through the cell. RNNs using LSTM units deal
with the vanishing gradient problem because these units allow gradients to flow
unchanged.\cite{doi:10.1162/neco.1997.9.8.1735}\\

The standard formulation of a single LSTM cell can be given by the following
equations:

\begin{align}
  f_t &= \sigma(W_f h_{t-1}+V_f x_t+b_f)\\
  i_t &= \sigma(W_i h_{t-1}+V_i x_t+b_i)\\
  C_t^{'} &= \tanh(W_C h_{t-1}+V_C x_t+b_C)\\
  C_t &= f_t C_{t-1}+i_t C_t^{'}\\
  o_t &= \sigma(W_o h_{t-1}+V_o x_t+b_o)\\
  h_t &= o_t \tanh(C_t)
\end{align}

where $\sigma$ is the sigmoid function, $\tanh$ is the hyperbolic tangent
function, $f,i,C^{'},C,o$ are the forget gate, input gate, new memory cell
content, and memory cell content, output gate respectively. See representation
of an LSTM cell in Figure~\ref{memorycell}.

\input{sections/scientific/fr2/lstmcell.tex}


\section{Normal Mapping}
Normal Mapping provides an enormous boost in detail for a relatively low cost. To implement
Normal Mapping, tangents and bitangents of each vertex are required. Tangents are computed in CPU
when a model is loaded, saving important time. Bitangents are computed in shader code, as a cross
product between re-orthogonalized Tangents and Normals.

Normals and Tangents for each vertex are calculated when loading the model and are first processed
on the Geometry pass where the TBN\footnote{TBN is a matrix that contains the Tangent, Bitangent
and Normal vectors} matrix is calculated and then written to a 2D texture, for light pass to use.

\noindent Normals used in light pass are calculated with the following equations:

$$norm(x)=\frac{2*x - 1}{\vert 2*x - 1 \vert}$$
$$N(n, t)=norm(t*norm(2*n - 1))$$

\noindent Where n is the normal vector and t, the TBN matrix.

On larger meshes the three TBN vectors could end up non-perpendiuclar to each other resulting in
a non orthogonal TBN matrix. That throws the normal mapping quality a bit off. To address that
issue a mathematical trick called \textit{Gram-Schmidt process} is used, to re-orthogonalize
the TBN matrix. First, the tangent needs to be re-orthogonalized with respect to N and then
the perpenducular bitangent vector can by calculated by the cross product of tangent and normal
vectors.

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.4,clip=true]{./image/nm1.png}
    \caption{Normal Mapping on a metallic wall}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[scale=0.4,clip=true]{./image/nm2.png}
    \caption{Normal Mapping on a house model}
\end{figure}

\newpage


\documentclass[../report/main.tex]{subfiles}

\begin{document}

% The asterix after \subsection disables section numbering
\subsection*{Problem 1 Part D}
Formulate a generalized linear programming model for the transshipment problem. Give the objective function and constraints as mathematical formulas.

\subsection*{Note}
I can't help but feel that I've already done this in Parts A through C but I'll repeat (and try to expand) here in a more general form.

\subsection*{Objective Function}
Whereas...\\\\\indent$n$ = number of plants $\geq 1$\\\indent$q$ = number of warehouses $\geq 1$\\\indent$m$ = number of retailers $\geq 1$\\\indent$np_{ij}$ = number of refrigerators shipped from plant $p_i$ to warehouse $w_j$ $\geq 0$\\\indent$cp_{ij}$ = cost of moving a refrigerator between plant $p_i$ and warehouse $w_j$ $> 0$\\\indent$nw_{jk}$ = number of refrigerators shipped from warehouse $w_j$ to retailer $r_k$ $\geq 0$\\\indent$cw_{jk}$ = cost of moving a refrigerator between warehouse $w_j$ and retailer $r_k$ $> 0$\\\\The \textbf{\underline{objective function}} is to...
\\\\\indent minimize $cost = \mathop{\sum_{i=1}^{n}\sum_{j=1}^{q}}np_{ij}*cp_{ij} +  \mathop{\sum_{j=1}^{q}\sum_{k=1}^{m}}nw_{jk}*cw_{jk}$

\subsection*{Constraints}
The objective function (designed to minimize $cost$) is \textbf{\underline{subject to}} the following contraints...\\\\\indent $0 \leq $ capacity of each plant $= s_i \leq$ some maximum\\\indent $0 \leq $ throughput capability of each warehouse $ = q_j \leq$ some maximum\\\indent $0 \leq $ demand of each retailer  $ = m_k \leq$ some maximum\\\\and also...\\\\\indent $\mathop{\sum_{i=1}^{n}}np_{ij} = $ capacity of a given plant $p_i$\\\\\indent $\mathop{\sum_{i=1}^{n}}np_{ij} = \mathop{\sum_{k=1}^{m}}nw_{jk}= $ throughput capability for a given warehouse $w_j$\\\\\indent $\mathop{\sum_{k=1}^{m}}nw_{jk} = $ demand of a given retailer $r_k$
\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS630: Database Management Systems
% Copyright 2014 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 1}

Consider a database schema with three relations:

\begin{terminal}
Students (@*\underline{sid}*@:integer, sname:string, age:integer)
Enrolled (@*\underline{sid}*@:integer, @*\underline{cid}*@:integer, grade:integer)
Courses  (@*\underline{cid}*@:integer, cname:string, credits:integer)
\end{terminal}

The keys are underlined in each relation.
Students are identified uniquely by \texttt{sid}, and courses by \texttt{cid}.
Students enroll to take courses, and for each course they obtain a grade which is an integer.
\texttt{sname} is the student name (string), \texttt{age} represents the student age and is an integer.
\texttt{cname} is the course name (string), and \texttt{credits} is the number of credits for a particular course (integer).

Write relational algebra expressions for the following queries:

\begin{enumerate}
  \item Find the names of students who got grade 10 in some course.

  \textbf{Solution:}
  $$\displaystyle \pi_{sname}((\sigma_{grade=10}Enrolled)\Join Students)$$
  \item Find the ages of students who take some course with 3 credits.

  \textbf{Solution:}
  $$\displaystyle \pi_{age} (\pi_{sid} (\pi_{cid} (\sigma_{credits=3}Courses) \Join Enrolled) \Join Students)$$
  \item Find the names of students who take a course named "Calculus".

  \textbf{Solution:}
  $$\displaystyle \pi_{sname} (\pi_{sid} (\pi_{cid} (\sigma_{cname="Calculus"}Courses) \Join Enrolled)\Join Students)$$
  \item Find the names of students who obtained grade at least 8 in some course that has less than 4 credits.

  \textbf{Solution:}
  $$\displaystyle \pi_{sname} (\pi_{sid} (\sigma_{grade>0}(\pi_{cid} (\sigma_{credits<4}Courses) \Join Enrolled)) \Join Students) $$

  \item Find the names of students who obtained only grades of 10 (implies that they took at least one course).

  \textbf{Solution:}
  \begin{enumerate}
  \item S1 is constructed to represent IDs of students who obtained grades of 10 at lease in one course; i.e. includes IDs of students who might have obtained a different grade in different course(s).
  $$\displaystyle \rho(S1, \pi_{sid}( \sigma_{grade=10}Enrolled)) $$
  \item S2 is constructed to represent IDs of all students who obtained a grade different than 10 at least in one course.
  $$\displaystyle \rho(S2, \pi_{sid}(\sigma_{grade<>10}Enrolled)) $$
  \item Desired list is obtained by looking up names of students whose ID is in S1 and not in S2.
  $$\displaystyle \pi_{sname}((S1-S2) \Join Students) $$
  \end{enumerate}

  \item Find the names of  students who took a course with three credits or who obtained grade 10 in some course.

  \textbf{Solution:}
  \begin{enumerate}
  \item S1 is constructed to represent IDs of students who took a course with three credits.
  $$\displaystyle \rho(S1, \pi_{sid} (\pi_{cid} (\sigma_{credits=3}Courses) ) \Join Enrolled)$$
  \item S2 is constructed to represent IDs of students who obtained grade 10 at least in one course.
  $$\displaystyle \rho(S2, \pi_{sid}(\sigma_{grade=10}Enrolled) ) $$
  \item Desired list is obtained by merging S1 and S2 lists and looking up names of students corresponding to IDs of merged list.
  $$\displaystyle \pi_{sname}((S1 \cup S2) \Join Students) $$
  \end{enumerate}

  \item Find the ages of students who attend "Calculus" but never took any 4-credit course (assume there is a course "Calculus" with 3 credits).

  \textbf{Solution:}
  \begin{enumerate}
  \item S1 is constructed to represent IDs of students who attend "Calculus".
  $$ \displaystyle \rho(S1,   \pi_{sid}( \pi_{cid}( \sigma_{cname="Calculus"}Courses) \Join Enrolled) ) $$
  \item S2 is constructed to represent IDs of students who took at lease one 4-credit course.
  $$ \displaystyle \rho(S2, \pi_{sid} (\pi_{cid}( \sigma_{credits=4}Courses) \Join Enrolled) ) $$
  \item Desired list is obtained by finding names of students whose ID is in S1 and not in S2.
  $$ \displaystyle \pi_{sname}((S1-S2)\Join Students) $$
  \end{enumerate}

  \item Find the names of students who have the lowest age.

  \textbf{Solution:}
  $$\displaystyle \rho(S1(1\rightarrow sid1, 2\rightarrow sname1, 3\rightarrow age1),Students)$$
  $$\rho(S2(1\rightarrow sid2, 2\rightarrow sname2, 3\rightarrow age2),Students)$$
  $$\rho(Elders(1\rightarrow sid),\pi_{sid1}(S1 \Join_{(age1>age2)} S2)) $$
  Desired list includes names of students whose ID is not among Elders.
  $$ \pi_{sname}((\pi_{sid}(Students)-Elders) \Join Students) $$

  \item Find the names of students who are enrolled in a single course.

  \textbf{Solution:}
  \begin{enumerate}
  \item{S1 is constructed to represent IDs of students who are not enrolled in any courses.}
  $$\displaystyle \rho(S1, \pi_{sid}Students-\pi_{sid}Enrolled)$$
  \item{S2 is constructed to represent IDs of students who are enrolled in more than one course.}
  $$\displaystyle \rho(E1(1\rightarrow sid1, 2\rightarrow cid1, 3\rightarrow grade1),Enrolled)$$
  $$\rho(E2(1\rightarrow sid2, 2\rightarrow cid2, 3\rightarrow grade2),Enrolled)$$
  $$\rho(S2,\pi_{sid}(E1 \Join_{(sid1=sid2\wedge cid1\neq cid2)} E2) ) $$
  \item{Students enrolled in a single course are those whose ID is neither in S1 nor in S2.}
  $$\displaystyle \pi_{sname}( ((\pi_{sid}Students-S1)-S2)\Join Students)$$
  \end{enumerate}

  \item Find the grades of students who are enrolled in course(s) with the highest number of credits.

  \textbf{Solution:}
  \begin{enumerate}
  \item C4 is constructed to give list of course IDs with highest number of credits.
  $$\displaystyle \rho(C1(1\rightarrow cid1, 2\rightarrow cname1, 3\rightarrow credits1),Courses)$$
  $$\rho(C2(1\rightarrow cid2, 2\rightarrow cname2, 3\rightarrow credits2),Courses)$$
  $$\rho(C3,\pi_{cid1}(C1 \Join_{(credits1<credits2)} C2)) $$
  $$\rho(C4(1 \rightarrow cid),\pi_{cid1}Courses-C3)$$
  \item Desired list is obtained using C4 table.
  $$ \pi_{grade}(C4 \Join Enrolled) $$
  \end{enumerate}

\end{enumerate}


\chapter{Description of the complete mathematical model}
\label{ap:model}
\textit{In this appendix we will analyse formally the proposed model, presented in Chapter \ref{ch::model}. Functions derivatives are trivial, so they will not be indicated except where otherwise indicated.}

%--------------------------------------------%
%--------------------------------------------%
\section*{Error propagation along the y axis}
Accordingly with the geometric model for \acs{SOL} systems, we can write:
  \begin{equation}
    \label{app:eq:triangulation}
	y(\alpha) = y_f + z_f \tan(\phi + \alpha)
  \end{equation}
where $y(\alpha)$ is the point $y$ coordinate; $(x_f, y_f, z_f)$ are the principal point projection coordinates in the laser plane (both in the world reference system); $\phi$ is the triangulation angle and $\alpha$ is the angle offset along $y$ axis.
Using the error propagation model for general non-trivial functions, the error is evaluated as follows:
  \begin{equation*}
    \sigma_{y_\alpha} = \sqrt{
      \left( \frac{\partial y}{\partial y_f} \right)^2 \sigma_{y_f}^2
      + \left( \frac{\partial y}{\partial z_f} \right)^2 \sigma_{z_f}^2
      + \left( \frac{\partial y}{\partial \phi} \right)^2 \sigma_\phi^2
      + \left( \frac{\partial y}{\partial \alpha} \right)^2 \sigma_\alpha^2
    }
  \end{equation*} \\

If laser plane doesn't lie on the target section plane, a common practise is applied the so called ``\textit{radial compensation}'' to \ref{app:eq:triangulation} output:
  \begin{equation*}
    y_w = y(\alpha) \cdot cos(\rho)
  \end{equation*}
where $\rho$ is the angle between laser plane and object plane of symmetry.
In this case the final error is:
  \begin{equation}
    \sigma_{y_w} = \sqrt{
      \left( \frac{\partial y_w}{\partial y(\alpha)} \right)^2 \sigma_{y_\alpha}^2
      + \left( \frac{\partial y_w}{\partial \rho} \right)^2 \sigma_\rho^2
    }
    \label{app:eq:sigma-yw}
  \end{equation} \\

A variation of $y_w$ in the world corresponds to a variation of $y_{s_i}$ in the image plane with an angle $\alpha_i$ according to the relation:
  \begin{equation}
    \label{app:eq:triang_angle}
  	\alpha_i = \arctan\left( \frac{y_{s_i}}{f} \right)
  \end{equation}
where $y_{s_i}$ is the point coordinate in the tilted image plane, and $f$ is the focal length. The error propagation is computed as:
  \begin{equation}
  	\sigma_{\alpha_i} = \sqrt{
  	  \left( \frac{\partial \alpha_i}{\partial y_{s_i}} \right)^2 \sigma_{y_{s_i}}^2
  	  + \left( \frac{\partial \alpha_i}{\partial f} \right)^2 \sigma_f^2
  	}
    \label{app:eq:sigma-alpha}
  \end{equation} \\

As it is known, Scheimpflug principle causes tilting of the image plane. Accordingly with \cite{SchCameraCalib}, the relation between tilted coordinates and parallel coordinates to the sensor is given by:
  \begin{equation}
    \label{app:eq:sch_y}
    y_{s_i} = \lambda_1 \frac{y_{p_i}}{\cos\upsilon} =
    \frac{f}{f - x_{p_i}\tan\chi - y_{p_i}\frac{\tan\upsilon}{\cos\chi}} \frac{y_{p_i}}{ \cos\upsilon}
  \end{equation}
where $\upsilon$ is the tilt angle with respect to $y$ axis, and $\chi$ is the swing angle with respect to $x$ axis, either in the image plane coordinates system, parallel to the camera sensor. The constant $\lambda_1$ is function of $f$, $\upsilon$ and $\chi$, and of point coordinates in the last image plane $\left( x_p, y_p \right)$. The error propagation follows:
  \begin{equation*}
    \sigma_{y_{s_i}} = \sqrt{
      \left( \frac{\partial y_{s_i}}{\partial y_{p_i}} \right)^2 \sigma_{y_{p_i}}^2 +
      \left( \frac{\partial y_{s_i}}{\partial x_{p_i}} \right)^2 \sigma_{x_{p_i}}^2 +
      \left( \frac{\partial y_{s_i}}{\partial \upsilon} \right)^2 \sigma_\upsilon^2 +
      \left( \frac{\partial y_{s_i}}{\partial \chi} \right)^2 \sigma_\chi^2 +
      \left( \frac{\partial y_{s_i}}{\partial f} \right)^2 \sigma_f^2
    }
  \end{equation*}
In this case we indicate the partial derivatives for completeness: \\
\begin{equation*}
  \begin{array}{rl}
    \frac{
      \partial y_s}{\partial y_p} = &
          \frac{f\cdot\sec\upsilon}{f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi} +
          \frac{f\cdot y_p\cdot \sec\chi\tan\upsilon\sec\upsilon}{\left( f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi \right)^2} \\~\\
      \frac{\partial y_s}{\partial x_p} = &
          \frac{f\cdot y_p\cdot \tan\chi\sec\upsilon}{\left( f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi \right)^2} \\~\\
      \frac{\partial y_s}{\partial \chi} = &
          \frac{f\cdot y_p\cdot \sec\upsilon \left( -y_p\cdot \tan\chi\sec\chi\tan\upsilon - x_p\cdot\sec^2\chi \right)}{\left( f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi \right)^2} \\~\\
      \frac{\partial y_s}{\partial \upsilon} = &
          \frac{f\cdot y_p^2\cdot \sec\chi\sec^3\upsilon}{ \left( f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi \right)^2} +
          \frac{f\cdot y_p\cdot \sec\upsilon\tan\upsilon}{f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi} \\~\\
      \frac{\partial y_s}{\partial f} = &
          \frac{f\cdot y_p\cdot \sec\upsilon}{ \left( f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi \right)^2} +
          \frac{y_p\cdot \sec\upsilon}{f -y_p\cdot\sec\chi\tan\upsilon - x_p\cdot\tan\chi}
  \end{array}
\end{equation*} \\

Developing this model, we took into account only the first order lens radial distortion:
  \begin{equation}
  	\label{app:eq:sensor_to_camera_distortion}
    \begin{matrix}
      y_{p_i} = y_{p_i}^{d} \left( 1 + k_1r^2 \right) \\~\\
      \sigma_{y_{p_i}} = \sqrt{
        \left( \frac{\partial y_{p_i}}{\partial y_{p_i}^d} \right)^2 \sigma_{y_{p_i}^d}^2
        + \left( \frac{\partial y_{p_i}}{\partial k_1} \right)^2 \sigma_{k_1}^2
      }
    \end{matrix}
  \end{equation}
Where $y_{p_i}^d$ is the distorted coordinate in the sensor plane, $k_1$ is the lens radial distortion first order coefficient and $r$ is the lens radius, estimated in the worst case as $r = \frac{image \, size}{2}$. \\

The distorted coordinate in sensor plane is evaluate accordingly with:
  \begin{equation}
      \label{app:eq:sensor_to_camera_center}
      y_{p_i}^d = d'_y (y_{c_i} - c_y)
  \end{equation}
where $c_y$ is the $y$ coordinate of the center image, $d'_y$ is the normalized center to center distance between adjacent sensor elements, and $y_c$ is the point coordinate in pixel. Its error is computed as:
  \begin{equation*}
    \sigma_{y_{p_i}^d} = \sqrt{
      \left( \frac{\partial y_{p_i}^d}{\partial y_{c_i}} \right)^2 \sigma_{y_{c_i}}^2
      + \left( \frac{\partial y_{p_i}^d}{\partial c_y} \right)^2 \sigma_{c_y}^2
    }
  \end{equation*} \\

At the end of the chain, the sub-pixel approximation error is evaluated as follows:
  \begin{equation*}
    \sigma_{y_{c_i}} = |\delta_i - \hat{\delta}_i|
  \end{equation*}
where $\delta_i$ is the real laser peak position inside the pixel, and $\hat{\delta}_i$ is the approximated peak position.

%--------------------------------------------%
%--------------------------------------------%
\section*{Error propagation along the x axis}
The analysis for error propagation along coordinate $x$ is similar to the previous one. For this reason, the parameters using the same notation won't be explained. \\

The point $x$ coordinate is evaluated accordingly with the geometric model:
  \begin{equation*}
    x(\alpha, \beta) = \frac{y_\alpha - y_f}{\sin(\phi + \alpha)}\tan(\beta)
  \end{equation*}
where $\beta$ is the triangulation angle with respect to the $x$ axis. As in the previous case, we have to balance the laser pitch rotation:
  \begin{equation*}
    x_w = x(\alpha, \beta) \cdot \cos(\gamma)
  \end{equation*}
where $\gamma$ is the laser pith angle. So the error is:
  \begin{equation}
    \sigma_{x_w} = \sqrt{
      \left( \frac{\partial x_w}{\partial y_\alpha} \right)^2 \sigma_{y_\alpha}^2 +
      \left( \frac{\partial x_w}{\partial \phi} \right)^2 \sigma_\phi^2 +
      \left( \frac{\partial x_w}{\partial \alpha} \right)^2 \sigma_\alpha^2 +
      \left( \frac{\partial x_w}{\partial \beta} \right)^2 \sigma_\beta^2 +
      \left( \frac{\partial x_w}{\partial \gamma} \right)^2 \sigma_\gamma^2
    }
    \label{app:eq:sigma-xw}
  \end{equation}
For clarity, we provide the partial derivatives below:
  \begin{equation*}
    \begin{array}{rl}
      \frac{\partial x_c}{\partial y} = & \tan\beta\sec\gamma\csc(\phi + \alpha) \\~\\
      \frac{\partial x_c}{\partial \phi} = & -y\cdot\tan\beta\cot(\phi + \alpha)\csc(\phi + \alpha) \\~\\
      \frac{\partial x_c}{\partial \alpha} = & -y\cdot\tan\beta\sec\gamma\cot(\phi + \alpha)\csc(\phi + \alpha) \\~\\
      \frac{\partial x_c}{\partial \beta} = & y\cdot\sec^2\beta\sec\gamma\csc(\phi + \alpha) \\~\\
      \frac{\partial x_c}{\partial \gamma} = & y\cdot \tan\beta\tan\gamma\sec\gamma\csc(\phi + \alpha)
    \end{array}
  \end{equation*} \\

How did in Equation \ref{app:eq:triang_angle}, the triangulation angle is computed as:
  \begin{equation*}
    \begin{matrix}
      \beta_i = \arctan\left( \frac{x_{s_i}}{f} \right)
      \qquad \qquad
  	  \sigma_{\beta_i} = \sqrt{
  	      \left( \frac{\partial \beta_i}{\partial y_{s_i}} \right)^2 \sigma_{y_{s_i}}^2
  	      + \left( \frac{\partial \beta_i}{\partial f} \right)^2 \sigma_f^2
  	  }
    \end{matrix}
  \end{equation*} \\

The image plane swing caused by Scheimpflug principle, is given by:
  \begin{equation}
    \label{app:eq:sch_x}
    \begin{matrix}
      x_{s_i} =
        \lambda_1\left( \frac{x_{p_i}}{\cos\chi} + y_{p_i}\tan\upsilon\tan\chi \right) = \\
        \frac{f}{f - x_{p_i}\tan\chi - y_{p_i}\frac{\tan\upsilon}{\cos\chi}} \left( \frac{x_{p_i}}{\cos\chi} + y_{p_i}\tan\upsilon\tan\chi \right)
    \end{matrix}
  \end{equation}
The error is computed as above:
  \begin{equation*}
    \sigma_{x_{s_i}} = \sqrt{
      \left( \frac{\partial x_{s_i}}{\partial y_{p_i}} \right)^2 \sigma_{y_{p_i}}^2 +
      \left( \frac{\partial x_{s_i}}{\partial x_{p_i}} \right)^2 \sigma_{x_{p_i}}^2 +
      \left( \frac{\partial x_{s_i}}{\partial \upsilon} \right)^2 \sigma_\upsilon^2 +
      \left( \frac{\partial x_{s_i}}{\partial \chi} \right)^2 \sigma_\chi^2 +
      \left( \frac{\partial x_{s_i}}{\partial f} \right)^2 \sigma_f^2
    }
  \end{equation*} \\

How in Equations \ref{app:eq:sensor_to_camera_distortion} and \ref{app:eq:sensor_to_camera_center}, the transformation from sensor coordinates to undistorted image plane coordinates, is computed as:
\begin{equation}
  \begin{matrix}
    \label{app:eq:camera_to_image_x}
    x_{p_i} = x_{p_i}^d (1 + k_1r^2) \qquad \qquad
    x_{p_i}^d = d_x'(x_{c_i} - c_x)
  \end{matrix}
\end{equation}
the errors of which are:
  \begin{equation*}
    \sigma_{x_{p_i}} = \sqrt{
        \left( \frac{\partial x_{p_i}}{\partial x_{p_i}^d} \right)^2 \sigma_{x_{p_i}^d}^2
        + \left( \frac{\partial x_{p_i}}{\partial k_1} \right)^2 \sigma_{k_1}^2
      }
	  \qquad
    \sigma_{x_{p_i}^d} = \sqrt{
      \left( \frac{\partial x_{p_i}^d}{\partial x_{c_i}} \right)^2 \sigma_{x_{c_i}}^2
      + \left( \frac{\partial x_{p_i}^d}{\partial c_x} \right)^2 \sigma_{c_x}^2
    }
  \end{equation*} \\

In this case, we can't perform a sub-pixel approximation, so the laser power distribution along the pixel was modelled as an \textit{uniform distribution} with standard deviation:
  \begin{equation*}
    \sigma_{x_{c_i}} = \frac{1}{2\sqrt{3}}
  \end{equation*} \\

Equations \ref{app:eq:sensor_to_camera_distortion}, \ref{app:eq:sensor_to_camera_center} and \ref{app:eq:camera_to_image_x} was used accordingly with \cite{TsaiTvLenses}, while Equations \ref{app:eq:sch_y} and \ref{app:eq:sch_x} was used accordingly with \cite{SchCameraCalib}.

%--------------------------------------------%
%--------------------------------------------%
\section*{Error propagation in diameter evaluation}
Accordingly with the Erone's formula, the diameter of the circle circumscribed to a triangle (in our case, defined by three rolling points in the rolling circle) is evaluable with the equation:
  \begin{equation}
    D = \frac{2\cdot a\cdot b\cdot c}{\sqrt{( a + b + c )( - a + b + c )( a - b + c )( a + b - c )}}
    \label{app:eq:diam-prop}
  \end{equation}
where $a$, $b$ and $c$ are the lengths of the edges of the triangle. \\

Projecting each vertex on a same coordinate reference system $(ZW)$, we can characterize that point with a vector $y_i$, that can be decomposed into its two components $z_i$ and $w_i$ as follows:
  \begin{equation}
    \begin{matrix}
      z_i = y_i \cdot \cos \theta_i + H_i \\
      w_i = y_i \cdot \sin \theta_i \\
    \end{matrix}
    \label{app:eq:components}
  \end{equation}
where $\theta_i$ is the triangulation angle and $H_i$ is the offset of the laser projector, with respect to the origin of the coordinate system. At this point, it is simple to compute the edges length as:
  \begin{equation}
    \left\{
    \begin{matrix}
      & a = \sqrt{(w_1 + w_2)^2 + (z_1 + z_2)^2} \\
      & b = \sqrt{(w_2 + w_3)^2 + (z_2 + z_3)^2} \\
      & c = \sqrt{(w_3 + w_1)^2 + (z_3 + z_1)^2}
    \end{matrix}
    \right.
    \label{app:eq:edges-len}
  \end{equation}
Hence, replacing in Equation \ref{app:eq:diam-prop} the results obtained in Equations \ref{app:eq:edges-len} and \ref{app:eq:components}, we can conclude that the error is propagated accordingly with:
  \begin{equation*}
    \sigma_D = \sqrt{
      \sum_{i = 1}^3 \left( \frac{\partial D}{\partial y_i} \sigma_{y_i} \right)^2 +
      \sum_{i = 1}^3 \left( \frac{\partial D}{\partial H_i} \sigma_{H_i} \right)^2 +
      \sum_{i = 1}^3 \left( \frac{\partial D}{\partial \theta_i} \sigma_{\theta_i} \right)^2
    }
  \end{equation*} \\

Because of there exists a linear relation between the reference system $(ZW)$ and the reference systems of each laser-camera pair, it is possible to define each vector $y_i$ as:
  \begin{equation*}
    |y_i| = \sqrt{x_w^2 + y_w^2 + z_w^2}
  \end{equation*}
where $\left( x_w, y_w, z_w \right)$ are the coordinates evaluated in the two sections above, and with $z_w = 0$, because each point lies on the laser plane. Thus, it is simple to determine $\sigma_{y_i}$ as:
  \begin{equation*}
    \sigma_{y_i} = \sqrt{
      \left( \frac{\partial y_i}{\partial x_w} \sigma_{x_w} \right)^2 +
      \left( \frac{\partial y_i}{\partial y_w} \sigma_{y_w} \right)^2
    }
    % = \frac{x}{\sqrt{x_w^2 + y_w^2}}
  \end{equation*}
where $\sigma_{x_w}$ and $\sigma_{y_w}$ are the value computed with Equations \ref{app:eq:sigma-xw} and \ref{app:eq:sigma-yw}, respectively. \\

Accordingly with the previous model, $\theta_i$ is the triangulation angle evaluated with respect to the $Y$ axis, thus $\sigma_{\theta_i}$ is the same error estimated with Equation \ref{app:eq:sigma-yw}. Vice-versa $H_i$ are a constructive parameters, hence we can set $\sigma_{H_i}$ reasonably, with respect to the requirements of product construction.


\chapter{Environment}

\section{Atmosphere}

US Standard Atmosphere 1976 is used to calculate air temperature, pressure, density, viscosity and speed of sound depending on altitude.

Mean molecular weight is given as follows:
\begin{equation}
  M_0 = \frac{ \sum_{j} M_j F_j }{ \sum_{j} F_j } = 28.9645
\end{equation}

Temperature is given by the following formula: \cite{NASA-TM-X-74335}
\begin{equation}
  T \left( h \right)
  =
  T_j + \left( \frac{dT}{dh} \right)_j \left( h - h_j \right)
\end{equation}

Pressure is given as follows: \cite{NASA-TM-X-74335}
\begin{align}
  p \left( h \right)
  =
  p_j \left( \frac{T_j}{ T \left( h \right) } \right)
  ^
  { \frac{gM_0}{ R \left( \frac{dT}{dh} \right)_j } }
  &\mathrm{~for~} \left( \frac{dT}{dh} \right)_j \neq 0 \\
    p \left( h \right)
  =
  p_j e^{ \frac{ g M_0 \left( h - h_j \right) }{RT_j} }
  &\mathrm{~for~} \left( \frac{dT}{dh} \right)_j = 0
\end{align}

Density is expressed by the following formula: \cite{NASA-TM-X-74335}
\begin{equation}
  \rho \left( h \right)
  =
  \frac{ p \left( h \right) M_0 }{ RT \left( h \right) }
\end{equation}

Speed of sound is given as follows: \cite{NASA-TM-X-74335}
\begin{equation}
  c_S \left( h \right)
  =
  \sqrt{ \frac{ \gamma RT \left( h \right) }{ M_0 } }
\end{equation}

Dynamic viscosity is given by the formula: \cite{NASA-TM-X-74335}
\begin{equation}
  \mu \left( h \right)
  =
  \frac{ 1.458 \cdot 10^{-6} \sqrt{ \left[ T \left( h \right) \right]^3 } }
  { T \left( h \right) + S }
\end{equation}

Kinetic viscosity is given as follows: \cite{NASA-TM-X-74335}
\begin{equation}
  \nu \left( h \right)
  =
  \frac{ \mu \left( h \right) }{ \rho \left( h \right) }
\end{equation}

\newpage

\vfill

\begin{table}[h!]
  \begin{center}
    \begin{tabular}{ S | S | S | S }
      \toprule
      \textbf{Altitude} & \textbf{Temperature gradient} & \textbf{Temperature} & \textbf{Pressure} \\
      {$h_j$} & {$\left( \cfrac{dT}{dh} \right)_j$} & {$T_j$} & {$p_j$} \\
      {[m]} & {[K/m]} & {[K]} & {[Pa]} \\ \midrule
          0 & -6.5e-3 & 288.15 & 101325.0    \\
      11000 &  0.0    & 216.65 &  22632.0    \\
      20000 &  1.0e-3 & 216.65 &   5474.8    \\
      32000 &  2.8e-3 & 228.65 &    868.01   \\
      47000 &  0.0    & 270.65 &    110.9    \\
      51000 & -2.8e-3 & 270.65 &     66.938  \\
      71000 & -2.0e-3 & 214.65 &      3.9564 \\
      \bottomrule
    \end{tabular}
    \caption{Reference levels \cite{NASA-TM-X-74335} }
  \end{center}
\end{table}

\vfill

\begin{table}[h!]
  \begin{center}
    \begin{tabular}{ l | S | S }
      \toprule
      \textbf{Gas species} & \textbf{Molecular weight} & \textbf{Fractional volume} \\
      {} & {[kg/kmol]} & {[-]} \\ \midrule
      Nitrogen       & 28.0134  & 0.78084     \\
      Oxygen         & 31.9988  & 0.209476    \\
      Argon          & 39.948   & 0.00934     \\
      Carbon Dioxide & 44.00995 & 0.000314    \\
      Neon           & 20.183   & 0.00001818  \\
      Helium         & 4.0026   & 0.00000524  \\
      Krypton        & 83.8     & 0.00000114  \\
      Xenon          & 131.3    & 0.000000087 \\
      Methane        & 16.04303 & 0.000002    \\
      Hydrogen       & 2.01594  & 0.0000005   \\
      \bottomrule
    \end{tabular}
    \caption{Molecular weights and fractional volume composition of S/L dry air \cite{NASA-TM-X-74335} }
  \end{center}
\end{table}

\vfill


% !TEX root = main.tex

%----------------------------------------------------------------------
\section{The method of moments}
%----------------------------------------------------------------------
Perhaps the simplest estimators are based on the \emph{method of moments}.

\bigskip
Let $X$ be a random variable, let $f(x;\mathbf{\theta})$ be its PMF/PDF and consider its $k$th moment:
\[\begin{array}{lll}
\expe(X^k) & = \displaystyle\sum_{i=1}^{\infty} x_i^k f(x_i;\mathbf{\theta}) \qquad	& \text{(discrete case),} \\[3ex]
\expe(X^k) & = \displaystyle\int_{-\infty}^{\infty} x^k f(x;\mathbf{\theta})\,dx		& \text{(continuous case)}.
\end{array}\]
Let $X_1,X_2,\ldots,X_n$ be a random sample from the distribution of $X$. To estimate $k$ parameters $\mathbf{\theta} = (\theta_1,\theta_2,\ldots,\theta_k)$, we equate the expressions for the first $k$ moments with the first $k$ empirical moments, then solve the resulting system of equations with respect to $\theta_1,\theta_2,\ldots,\theta_k$. The number of equations must be equal to the number of unknown parameters we wish to estimate: these are known as the \emph{moment equations}.
\[
\expe(X) = \frac{1}{n}\sum_{i=1}^n X_i,
\quad
\expe(X^2) = \frac{1}{n}\sum_{i=1}^n X_i^2,
\quad\ldots,\quad
\expe(X^k) = \frac{1}{n}\sum_{i=1}^n X_i^k.
\]
%
%\ben
%\it % single parameter
%To estimate a scalar parameter $\theta$ using the \emph{method of moments}, we equate the first theoretical moment with the first empirical moment (i.e.\ the sample mean), then solve this with respect to $\theta$:
%\[
%\expe(X;\theta) = \frac{1}{n}\sum_{i=1}^n X_i.% \equiv \bar{X}.
%\]
%\it % two parameters
%To estimate two parameters $\mathbf{\theta} = (\theta_1,\theta_2)$, we equate the first two theoretical moments with the first two empirical moments, then solve this system of two equations with respect to $\theta_1$ and $\theta_2$:
%\[
%\expe(X;\mathbf{\theta}) = \frac{1}{n}\sum_{i=1}^n X_i
%\qquad\text{and}\qquad
%\expe(X^2;\mathbf{\theta}) = \frac{1}{n}\sum_{i=1}^n X_i^2,
%\]
%\it % k parameters
%To estimate $k$ parameters $\mathbf{\theta} = (\theta_1,\theta_2,\ldots,\theta_k)$, we equate the first $k$ theoretical moments with the first $k$ empirical moments, then solve the resulting system of equations with respect to $\theta_1,\theta_2,\ldots,\theta_k$. The number of equations must be equal to the number of unknown parameters we wish to estimate.
%%\it In fact, we can equate the theoretical and empirical expected values of \emph{any} functions of $X$:
%%\begin{align*}
%%\expe\big[g_1(X);\mathbf{\theta}\big] & = \frac{1}{n}\sum_{i=1}^n g_1(X_i), \\
%%\expe\big[g_2(X);\mathbf{\theta}\big] & = \frac{1}{n}\sum_{i=1}^n g_2(X_i), \quad\text{etc.}
%%\end{align*}
%\een

% example: uniform
\begin{example}
Let $X_1,X_2,\ldots,X_n$ be a random sample from the continuous $\text{Uniform}(0,\theta)$ distribution, where $\theta>0$ is unknown. Find an estimator of $\theta$ using the method of moments.
\begin{solution}
Let $X\sim\text{Uniform}(0,\theta)$. In this case, $\expe(X) = \theta/2$ so the first moment equation is
\[
\frac{1}{2}\theta = \frac{1}{n}\sum_{i=1}^n X_i.
\]
Solving for $\theta$ we obtain $\hat{\theta}_{\text{\scriptsize{MME}}} = \displaystyle\frac{2}{n}\sum_{i=1}^n X_i$.
\end{solution}
\end{example}

% example: normal
\begin{example}
Let $X_1,X_2,\ldots,X_n$ be a random sample from the $N(\mu,\sigma^2)$ distribution. Find estimators of $\mu$ and $\sigma^2$ using the method of moments.
\end{example}

\begin{solution}
Let $X\sim N(\mu,\sigma^2)$ and let $\theta=(\mu,\sigma^2)$ be the parameter vector. Equating the first and second moments with the empirical first and second moments,
\[
\expe(X)	= \frac{1}{n}\sum_{i=1}^n X_i
\qquad\text{and}\qquad
\expe(X^2)	= \frac{1}{n}\sum_{i=1}^n X_i^2.
\]
The theoretical moments are $\expe(X) = \mu$ and $\expe(X^2) = \sigma^2 + \mu^2$. Solving the resulting equations for $\mu$ and $\sigma^2$, we obtain %the following MMEs of $\mu$ and $\sigma^2$:
\[
\hat{\mu}_{\text{\scriptsize{MME}}}	= \Xbar
\qquad\text{and}\qquad
\hat{\sigma}^2_{\text{\scriptsize{MME}}}
%= \frac{1}{n}\sum_{i=1}^n X_i^2 - \bar{X}^2
= \frac{1}{n}\sum_{i=1}^n (X_i-\bar{X})^2.
\]
%\begin{align*}
%\hat{\mu}			& = \Xbar, \\
%\hat{\sigma}^2	& = \frac{1}{n}\sum_{i=1}^n X_i^2 - \bar{X}^2 = \frac{1}{n}\sum_{i=1}^n (X_i-\bar{X})^2.
%\end{align*}
\bit
\it The MME of $\mu$ is the \emph{sample mean}.
\it The MME of $\sigma^2$ is the \emph{empirical mean squared deviation from the sample mean}.
\eit
Note that the latter is \emph{not} the sample variance.
\end{solution}

%----------------------------------------------------------------------
\begin{exercise}
\begin{questions}

% MME of Bernoulli distribution
\question
Let $X_1,X_2,\ldots,X_n$ be a random sample from the $\text{Bernoulli}(\theta)$ distribution. Find an estimator of $\theta$ using the method of moments.
\begin{answer}
The first moment of the $\text{Bernoulli}(\theta)$ distribution is $\theta$.
Equating this to the first empirical moment (i.e.\ the sample mean),
\[
\hat{\theta}_{\text{\scriptsize{MME}}} = \frac{1}{n}\sum_{i=1}^n X_i.
\]
The MME of $\theta$ is thus the proportion of successes observed in $n$ trials.
\end{answer}

% MME of exponential distribution
\question
Let $X_1,X_2,\ldots,X_n$ be a random sample from the $\text{Exponential}(\lambda)$ distribution, where $\lambda>0$ is an unknown rate parameter. Find an estimator of $\lambda$ using the method of moments.
\begin{answer}
Let $X\sim\text{Exponential}(\lambda)$. Then $\expe(X) = 1/\lambda$, so the first moment equation is
\[
\frac{1}{\lambda} = \bar{X}.
\]
Solving for $\lambda$, we obtain $\hat{\lambda}_{\text{\scriptsize{MME}}} = \bar{X}^{-1}$.
\end{answer}

% MME of Poisson distribution
\question
Let $X_1,X_2,\ldots,X_n$ be a random sample from the $\text{Poisson}(\lambda)$ distribution, where $\lambda>0$ is unknown. Find an estimator of $\lambda$ using the method of moments.
\begin{answer}
Let $X\sim\text{Poisson}(\lambda)$. The expected value of $X$ is $\expe(X)=\lambda$, so the first moment equation is simply
\[
\lambda = \bar{X}.
\]
Solving for $\lambda$, we obtain $\hat{\lambda}_{\text{\scriptsize{MME}}} = \bar{X}$.
\end{answer}
\end{questions}
\end{exercise}
%----------------------------------------------------------------------


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{tikz}
\setlength{\parindent}{0pt}

\newtheorem*{theorem}{Theorem}
\newtheorem*{definition}{Definition}
\newtheorem*{lemma}{Lemma}
\newtheorem*{corollary}{Corollary}
\newtheorem{example}{Example}
\newtheorem*{trick}{Trick}
\newtheorem*{question}{Question}

\title{Lecture 6: Kepler's Second Law}
\author{}
\date{}

\begin{document}

\maketitle

\section{Describe Motion with Vectors}

With parametric equations, we have a way not only to describe a motion, but also
to analyze the motion with more details.

\subsection{Position vectors}

Position vectors are used to describe the position of a point in a motion.
\begin{equation*}
  \vec{r}(t) = <x(t), y(t), z(t)>
\end{equation*}

Take the cycloid of the wheel of radius 1 rolling at unit speed as an example:
\begin{equation*}
  \vec{r}(t) = <t - \sin t, 1 - \cos t>
\end{equation*}

\subsection{Velocity vectors}

Velocity vectors are used to describe how fast and in what direction a point
moves in a motion.
\begin{equation*}
  \begin{split}
    \vec{v} &= \frac{d\vec{r}}{dt} \\
            &= <\frac{dx}{dt}, \frac{dy}{dt}, \frac{dz}{dt}> \\
  \end{split}
\end{equation*}
Speed is a scalar, and used to describe how fast a point moves in a motion.
\begin{equation*}
  speed = |\vec{v}|
\end{equation*}

Take the cycloid of the wheel of radius 1 rolling at unit speed as an example:
\begin{gather*}
  \begin{split}
    \vec{v} &= \frac{d\vec{r}}{dt} \\
            &= <\frac{d(t - \sin t)}{dt}, \frac{d(1 - \cos t)}{dt}> \\
            &= <1 - \cos t, \sin t> \\
  \end{split} \\
  \begin{split}
    |\vec{v}| &= \sqrt{(1 - \cos t)^2 + \sin^2 t} \\
              &= \sqrt{1 - 2\cos t + \cos^2 t + \sin^2 t} \\
              &= \sqrt{2 - 2\cos t} \\
  \end{split} \\
\end{gather*}
At $t = 0$, the velocity vector $\vec{v} = <0, 0>$, so the speed at that time is
0.

\subsection{Acceleration vectors}

Acceleration vectors are used to describe how velocity changes in a motion.
\begin{equation*}
  \vec{a} = \frac{d\vec{v}}{dt}
\end{equation*}

Take the cycloid of the wheel of radius 1 rolling at unit speed as an example:
\begin{equation*}
  \begin{split}
    \vec{a} &= \frac{d\vec{v}}{dt} \\
            &= <\frac{d(1 - \cos t)}{dt}, \frac{d(\sin t)}{dt}> \\
            &= <\sin t, \cos t> \\
  \end{split}
\end{equation*}
At $t = 0$, the acceleration vector $\vec{a} = <0, 1>$, which means the point at
that time has an acceleration in the positive direction of the $y$ axis.
Combining with the previous conclusion that the velocity at this time is 0, it
explains why the trajectory between two arches has a vertical tangent line.

\subsection{Arc length}

Arc length, usually denoted by $s$, is the distance traveled along the
trajectory of a motion. According to this definition,
\begin{equation*}
  \frac{ds}{dt} = speed = |\vec{v}|
\end{equation*}

Take the cycloid of the wheel of radius 1 rolling at unit speed as an example.
The length of an arch of the cycloid is
\begin{equation*}
  \int_{0}^{2\pi} \sqrt{2 - 2\cos t} dt
\end{equation*}

\subsection{Unit tangent vector}

Unit tangent vector, denoted by $\hat{T}$, is an unit vector in the direction of
the tangent line at a position in a motion.

Since $\vec{v}$ is a tangent vector,
\begin{equation*}
  \hat{T} = \frac{\vec{v}}{|\vec{v}|}
\end{equation*}

Another definition for the unit tangent vector:
\begin{gather*}
  \begin{split}
    \vec{v} &= \frac{d\vec{r}}{dt} \\
            &= \frac{d\vec{r}}{ds} \cdot \frac{ds}{dt} \\
            &= \frac{d\vec{r}}{ds} \cdot |\vec{v}| \\
  \end{split} \\
  \begin{split}
    \frac{d\vec{r}}{ds} &= \frac{\vec{v}}{|\vec{v}|} \\
                        &= \hat{T} \\
  \end{split} \\
  \vec{v} = \hat{T} \cdot \frac{ds}{dt} \\
\end{gather*}

Therefore, we can see the velocity vector can be interpreted as:
\begin{enumerate}
  \item direction: the unit tangent vector $\hat{T}$.
  \item magnitude: the speed $\frac{ds}{dt}$.
\end{enumerate}

The geometric interpretation of $\frac{d\vec{r}}{ds} = \hat{T}$: during the time
period of $\Delta t$,
\begin{gather*}
  \Delta \vec{r} = \vec{r}(t + \Delta t) - \vec{r}(t) \\
  \Delta \vec{r} \approx \hat{T} \cdot \Delta s \\
  \frac{\Delta \vec{r}}{\Delta t} \approx \hat{T} \cdot \frac{\Delta s}{\Delta t} \\
\end{gather*}
With $\Delta t$ approaches 0, the above formula becomes
\begin{gather*}
  \frac{d\vec{r}}{dt} = \hat{T} \cdot \frac{ds}{dt} \\
  \frac{d\vec{r}}{ds} = \hat{T} \\
\end{gather*}

\section{Example of Describing Motion with Vectors: Kepler's Second Law}

\subsection{Kepler's Second Law}

Kepler's Second Law: Motion of planets is in a plane, and the area swept out by
the line from sun to planet at a constant rate.

Newton later explain this law with gravitational attraction.

\subsection{Translation with Vector Notations}

The first part of Kepler's Second Law is: motion of planets is in a plane. The
plane of planet motion is determined by two vectors: the position vector of the
planet $\vec{r}$ and its velocity vector $\vec{v}$. The plane of planet motion
remains unchanged $\iff$ the direction of the normal vector of the plane remains
unchanged $\iff$ $dir(\vec{r} \times \vec{v})$ is constant.

The second part of Kepler's Second Law is: the area swept out by the line from
sun to planet at a constant rate. The area swept out in unit time can be
calculated as
\begin{gather*}
  \begin{split}
    dA &= \frac{1}{2}|\vec{r} \times d\vec{r}| \\
       &= \frac{1}{2}|\vec{r} \times \vec{v}| dt \\
  \end{split} \\
  \frac{dA}{dt} = \frac{1}{2}|\vec{r} \times \vec{v}| \\
\end{gather*}
Therefore, the area swept out by the line from sun to planet at a constant rate
$\iff$ $\frac{dA}{dt}$ is constant $\iff$ $|\vec{r} \times \vec{v}|$ is constant.

Therefore, Kepler's Second Law $\iff$ both $dir(\vec{r} \times \vec{v})$ and
$|\vec{r} \times \vec{v}|$ are constant $\iff$ $\vec{r} \times \vec{v}$ is
constant $\iff$ $\frac{d}{dt}(\vec{r} \times \vec{v}) = 0$.

\begin{gather*}
  \frac{d}{dt}(\vec{r} \times \vec{v}) = 0 \\
  \frac{d\vec{r}}{dt} \times \vec{v} + \vec{r} \times \frac{d\vec{v}}{dt} = 0 \\
  \vec{v} \times \vec{v} + \vec{r} \times \vec{a} = 0 \\
  0 + \vec{r} \times \vec{a} = 0 \\
  \vec{r} \times \vec{a} = 0 \\
  \vec{r} \parallel \vec{a} \\
\end{gather*}

Therefore, Kepler's Second Law holds for any moving object if and only if its
acceleration vector is always parallel to its position vector.

\end{document}


\textit{\noindent
Let $X$ be an infinite-dimensional Fréchet space. %
Prove that $X^\ast$, with its $\weakstar$topology, %
is of the first category in itself.
}
\newline\newline\noindent
This is actually a consequence of the below lemma,  %
which we prove first. %
The proof that $X^\ast$ is of the first category in itself comes right after, %
as a corollary.%
%
\paragraph{Lemma.}%
If $X$ is an infinite dimensional topological vector space whose dual %
%
  $X^\ast$ %
%
separates points on $X$, then the polar
%
  \begin{align}
    K_A\Def \{ \Lambda \in X^\ast:\, \magnitude{\Lambda} \leq 1 \text{ on } A\}
  \end{align}
%
of any absorbing subset $A$ is a $\weakstar$closed set that has empty interior.
%
\begin{proof}%
Let $x$ range over $X$. The linear form %
%
  $\Lambda \mapsto \Lambda x$ %
%
is $\weakstar$continuous; see \citeresultFA{3.14}. %
Therefore, %
%
  $P_x= \set{\Lambda \in X^\ast}{\magnitude{\Lambda x} \leq 1}$ %
%
is $\weakstar$ closed: %
%In particular, every $P_a$ ($a \in A$) is $\weakstar$closed:  %
As the intersection of $\set{P_a}{a\in A}$, %
$K_A$ is also a $\weakstar$closed set. %
We now prove the second half of the statement. % %
%
\newline\newline\noindent
%
From now on, $X$ is assumed to be endowed with its weak topology: %
$X$ is then locally convex, but its dual space is still %
%
  $X^\ast$ (see \citeresultFA{3.11}). %
%
Put %
%
  \begin{align}
    W \Def \bigcap_{x\in F} \set{\Lambda \in X^\ast}{\magnitude{\Lambda x} < r_x},
  \end{align}
%
where $r_x$ runs on $\R_+$, %
as $F$ runs through the nonempty finite subsets of $X$. %
%
Clearly, the collection of all such $W$ is a local base of $X^\ast$. %
Pick one of those $W$ and remark that the following subspace %
%
  \begin{align}
    M \Def \text{span}(F)
  \end{align}
%
is finite dimensional. %
Assume, to reach a contradiction, that $A\subset M$. %
So, every $x$ lies in $t_xM=M$ for some $t_x>0$, since $A$ is absorbing. %
As a consequence, $X=M$ is finite dimensional, which is a desired contradiction.
%
We have just established that $A\not\subset M$: %
Now pick $\varit{a}$ in $A\setminus M$ and so conclude that %f
%
  \begin{align}
    b\Def \frac{a}{t_a}\in A
  \end{align}
%
Remark that $b\notin M$ (otherwise, $a = t_a b \in t_a M=M$ would hold) %
and that $M$ is closed (see \citeresultFA{1.21 (b)}): %
By the Hahn-Banach theorem \citeresultFA{3.5}, %
there exists $\Lambda_a$ in $X^\ast$ such that
%
  \begin{align}\label{3.11. Polar_4.}
    \Lambda_a b > 2
  \end{align}
%
and
%
  \begin{align}\label{3.11. Polar_4bis.}
    \Lambda_a (M) = \singleton{0}.
  \end{align}
%
The latter equality implies that $\Lambda_a$ vanishes on $F$; %
hence $\Lambda_a$ is an element of $W$. %
On the other hand, given an arbitrary $\Lambda \in K_A$, %
the following inequalities  %
%
  \begin{align}
    \magnitude{\Lambda_a b + \Lambda b}
      \geq
    2 - \magnitude{\Lambda b}
      >
    1.
  \end{align}
%
show that $\Lambda + \Lambda_a$ is not in $K_A$. %
%
We have thus proved that
%
  \begin{align}
  \Lambda + W\not\subset K_A.
  \end{align}
%
Since $W$ and $\Lambda$ are both arbitrary, this achieves the proof. %
\end{proof}
%
%\newline\newline
\noindent
We now give a proof of the original statement. %
%
\paragraph{Corollary.}%
If $X$ is an infinite-dimensional Fréchet space, %
then $X^\ast$ is meager in itself.
%
\begin{proof}%
From now on, $X^\ast$ is only endowed with its $\weakstar$topology. %
Let $d$ be an invariant distance that is compatible with the topology of $X$, %
so that the following sets
%
  \begin{align}
    B_n \Def \set{x\in X}{d(0, x) <  1/n}\quad (\counting{n})
  \end{align}
%
form a local base of $X$. %
%
If $\Lambda$ is in $X^\ast$, then %
%
  \begin{align}
    \magnitude{\Lambda} \leq  m \text{ on } B_n
  \end{align}
%
for some $(n, m) \in \singleton{1, 2, 3, \dots}^2$; see \citeresultFA{1.18}. %
%
Hence, $X^\ast$ is the countable union of all %
%
  \begin{align}\label{3.11. Countable union.}
    m\cdot K_n \quad (\counting{m,n}),
  \end{align}
%
where $K_n$ is the polar of $B_n$. %
Clearly, showing that every $m\cdot K_n$ is nowhere dense %
is now sufficient. %
To do so, we use the fact that $X^\ast$ separates points; %
see \citeresultFA{3.4}. %
As a consequence, the above lemma implies %
%
  \begin{align}
    \left({\overline{K}_n}\right)^\circ = \left({{K}_n}\right)^\circ=\emptyset.
  \end{align}
%
Since the multiplication by $m$ is an homeomorphism (see \citeresultFA{1.7}), %
this is equivalent to %
%
  \begin{align}\label{3.11. Nowhere dense.}
    \left(\,{\overline{m\cdot K_n}}\, \right)^\circ
      =
    %\left(\, m \cdot \overline{{{K}_n}} \, \right)^\circ
    %  =
    %\left(m \cdot K_n \right)^\circ
    %  =
    m\cdot \left({{K}_n}\right)^\circ
      =
    \emptyset.
  \end{align}
%
So ends the proof.%
\end{proof}


\subsection{The partial differential operator}

\subsubsection{Differential}

When we change the value of an input to a function, we also change the output. We can examine these changes.

Consider the value of a function \(f(x)\) at points \(x_1\) and \(x_2\).

\(y_1=f(x_1)\)

\(y_2=f(x_2)\)

\(y_2-y_1=f(x_2)-f(x_1)\)

\(\dfrac{y_2-y_1}{x_2-x_1}=\dfrac{f(x_2)-f(x_1)}{x_2-x_1}\)

Let's define \(x_2\) in terms of its distance from \(x_1\):

\(x_2=x_1+\epsilon\)

\(\dfrac{y_2-y_1}{\epsilon }=\dfrac{f(x_1+\epsilon )-f(x_1)}{\epsilon }\)

We define the differential of a function as:

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x+\epsilon )-f(x)}{\epsilon }\)

If this is defined, then we say the function is differentiable at that point.

\subsubsection{Differential operator}

\subsubsection{Graph test}

\begin{tikzpicture}
\begin{axis}[
    axis lines = left,
    xlabel = $x$,
    ylabel = {$f(x)$},
]
%Below the red parabola is defined
\addplot [
    domain=-10:10,
    samples=100,
    color=red,
]
{x^2 + 5};
\addlegendentry{$x^2 + 5$}
%Here the blue parabloa is defined
\addplot [
    domain=-10:10,
    samples=100,
    color=blue,
    ]
{2 * x};
\addlegendentry{$2x$}

\end{axis}
\end{tikzpicture}


\chapter{Continuous-time Signals}

A continuous-time (CT) signal \index{CT Signal} is a function of one or more independent variables conveying information about a physical phenomena. This lecture gives an introduction to continuous-time signals as functions. You learn how to characterize such signals in a number of ways and are introduced to two very important signals: the unit impulse and the complex exponential.

\section{Signals as Functions}

In order to reason about signals mathematically we need a representation or {\it model}. Signals are modeled as functions, mappings between sets
\[
f: A \rightarrow B
\]
where $A$ is a set called the {\it domain} and $B$ is a set called the {\it co-domain}.

The most basic classification of signals depends on the sets that makeup the domain and co-domain. We will be interested in two versions of the domain, the reals denoted $\mathbb{R}$ and the integers denoted $\mathbb{Z}$. We will be interested in two versions of the co-domain, the reals $\mathbb{R}$ and the set of complex numbers $\mathbb{C}$.

\begin{definition}[Analog Signal]
  If the function $f: \mathbb{R} \rightarrow \mathbb{R}$, we call this an analog or real, continuous-time signal, e.g. a voltage at time $t \in \mathbb{R}$, $v(t)$. We will write these as $x(t)$, $y(t)$, etc. The units of $t$ are seconds. Fig.~\ref{fig:ctplots} shows some graphical representations, i.e. plots.
\end{definition}

\begin{figure}[ht]
  \begin{center}
  \frame{\includegraphics[scale=0.5]{graphics/ctsignals.pdf}}
  \end{center}
  \caption{Example plots of analog signals.}
  \label{fig:ctplots}
\end{figure}

\begin{definition}[Real, Discrete-time Signal]
  If the function $f: \mathbb{Z} \rightarrow \mathbb{R}$, we call this a real, discrete-time signal, e.g. the temperature every day at noon. We will write these as $x[n]$, $y[n]$, etc. Note $n$ is dimensionless.
\end{definition}

\begin{figure}[ht]
  \begin{center}
    \frame{\includegraphics[scale=0.5]{graphics/dtsignals.pdf}}
  \end{center}
  \caption{Example plots of real discrete-time signals.}
  \label{fig:dtplots}
\end{figure}

Some other possibilities:

\begin{itemize}
\item $f: \mathbb{R} \rightarrow \mathbb{Z}$, digital, continuous-time signals, e.g. the output of a general purpose pin on a microcontroller
\item $f: \mathbb{Z} \rightarrow \mathbb{Z}$, digital, discrete-time signals, e.g. the signal on a computer bus
\end{itemize}

The co-domain can also be complex.

\begin{itemize}
\item $f: \mathbb{R} \rightarrow \mathbb{C}$, complex-valued, continuous-time signals, e.g.\\
  \[
  x(t) = e^{j\omega t} = \cos(\omega t) + j\sin(\omega t)
  \]
\item $f: \mathbb{Z} \rightarrow \mathbb{C}$, complex-valued, discrete-time signals, e.g.\\
  \[
  x[n] = e^{j\omega n} = \cos(\omega n) + j\sin(\omega n)
  \]
\end{itemize}

Since the domains $\mathbb{R}$ and $\mathbb{Z}$ are usually interpreted as time, we will call these {\it time-domain} signals. In the time-domain, when the co-domain is $\mathbb{R}$ we call these real signals. All physical signals are real. However complex signals will become important when we discuss the frequency domain.

\section{Primitive Models}

We mathematically model signals by combining elementary/primitive functions, for example:

\begin{itemize}
\item polynomials: $x(t) = t$, $x(t) = t^2$, etc.
\item transendental functions: $x(t) = e^t$, $x(t) = \sin(t)$, $x(t) = \cos(t)$, etc.
\item piecewise functions, e.g.
  \[
     x(t) = \left\{  \begin{array}{cl}
       f_1(t) & t < 0\\
       f_2(t) & t \geq 0\\
     \end{array}\right.
     \]
\end{itemize}

\begin{example}[Modeling a Switch]
  Consider a mathematical model of a switch, which moves positions at time $t = 0$.

  \begin{center}
  \begin{circuitikz}[american voltages,scale=0.8, every node/.style={transform shape}]
    \draw
    (0,0) to[battery, l=$V$] (0,2.3)
    (3,2) node[spdt,xscale=-1,yscale=-1,anchor=in] (Sw) {}
    (0,2.3) to[short] (Sw.out 2)
    (Sw.out 1) to[short] (1.8,0)
    (0,0) to[short, -o] (4,0)
    (3,2) to[short, -o] (4,2)
    (4,2) to[open, v=$x(t)$] (4,0);;
  \end{circuitikz}
\end{center}

We use this model so much we give it it's own name and symbol: Unit Step, $u(t)$

    \[
     u(t) = \left\{  \begin{array}{cl}
        0 & t < 0\\
        1 & t \geq 0\\
      \end{array}\right.
  \]
  so a mathematical model of the switch circuit above would be $x(t) = V u(t)$.

  Note: some texts define the step function at $t=0$ to be $1$ or $\frac{1}{2}$. It is typically plotted like so:
  \begin{center}
    \begin{tikzpicture}[scale=0.8, every node/.style={transform shape}]
  \begin{axis}[ xlabel=$t$, axis x line=center, axis y line = center,
    xmin=-2, xmax=2, ymin=0, ymax=1.1, x label style={at={(axis cs:2,0)},anchor=west}]
    \draw[-,blue,style={ultra thick}] (axis cs:-2,0) -- (axis cs:0,0);
    \draw[-latex,blue,style={ultra thick}] (axis cs:0,1) -- (axis cs:2,1);
  \end{axis}
\end{tikzpicture}
    \end{center}
\end{example}

\begin{example}[Pure audio tone at "middle C"]
  A signal modeling the air pressure of a specific tone might be
  \[
  x(t) = \sin\left(2\pi (261.6) t\right)
  \]
\end{example}

\begin{example}[Chord]
    The chord "G", an additive mixture of tones at G, B, and D and might be modeled as
    \[
    x(t) = \sin\left(2\pi (392) t\right) + \sin\left(2\pi (494) t\right) + \sin\left(2\pi (293) t\right)
    \]
    This example shows we can use addition to build-up signals to approximate real signals of interest.
\end{example}

\section{Basic Transformations}

We can also apply transformations to signals to increase their modeling flexibility.

\begin{itemize}
\item  magnitude scaling
  \[
  x_2(t) = a x_1(t)
  \]
  for $a \in \mathbb{R}$.
\item derivatives
  \[
  x_2(t) = x_1^\prime(t)
  \]
\item integrals
  \[
  x_2(t) = \int\limits_{-\infty}^t x_1(\tau) \; d\tau
  \]
\item sums
  \[
  y(t) = \sum\limits_{i} x_i(t)
  \]
  an important example we will see is the CT Fourier series.
\item multiplication (modulation)
  \[
  y(t) = x_1(t) x_2(t)
  \]
  For example amplitude modulation $y(t) = x(t)\sin(\omega_0 t)$
\item time shift
  \[
  x_2(t) = x_1(t+\tau)
  \]
  \begin{itemize}
  \item if $\tau <0$ it is called a {\it delay}
  \item if $\tau >0$ it is called an {\it advance}
  \end{itemize}
  \item time scaling
    \[
    x_2(t) = x_1\left(\frac{t}{\tau}\right)
    \]
    \begin{itemize}
    \item if $\tau >0$ increasing $\tau$ expands in time, slows the signal
    \item if $\tau <0$ time reverses and decreasing $\tau$ expands in time
    \end{itemize}
    Example: time reversal $ x_2(t) = x_1(-t)$
\end{itemize}

\section{Characterization of Signals}

There are a few basic ways of characterizing signals.

\begin{definition}[Causal CT Signal]
  A CT signal is \emph{causal} if $x(t) = 0$ $\forall t < 0$.
\end{definition}
\begin{definition}[Anti-Causal CT Signal]
  A CT signal is \emph{anti-causal} or acausal if $x(t) = 0$ $\forall t \geq 0$.
\end{definition}

A signal can be written as the sum of a causal and anti-causal signal.

\begin{definition}[Periodic Signals]
  A CT signal is \emph{periodic} if $x(t) = x(t + T)$ $\forall t$ for a fixed parameter $T \in \mathbb{R}$ called the \emph{period}.
\end{definition}

The simplest periodic signals are those based on the sinusoidal functions.

\begin{definition}[Even Signal]
  A CT signal is \emph{even} if $x(t) = x(-t)$ $\forall t$.
\end{definition}

\begin{definition}[Odd Signal]
  A CT signal is \emph{odd} if $x(t) = -x(-t)$ $\forall t$.
\end{definition}

Any CT signal can be written in terms of an even and odd component
\[
x(t) = x_e(t) + x_o(t)
\]
where
\[
\begin{array}{ll}
  x_e(t) &= \frac{1}{2}\left\{x(t) + x(-t)\right\} \\
  & \\
  x_o(t) &= \frac{1}{2}\left\{x(t) - x(-t)\right\}
\end{array}
\]

\begin{definition}[Energy of a CT Signal]
  The \emph{energy} of a CT signal $x(t)$ is defined as a measure of the function
  \[
  E_x = \lim_{T\rightarrow\infty} \int\limits_{-T}^T \lvert x(t) \rvert^2 dt \; .
  \]
\end{definition}

\begin{definition}[Power of a CT Signal]
  The \emph{power} of a CT signal is the energy averaged over an interval as that interval tends to infinity.
  \[
  P_x = \lim_{T\rightarrow\infty} \frac{1}{2T} \int_{-T}^T \lvert x(t)\rvert^2 dt \; .
  \]
\end{definition}

Signals can be characterized based on their energy or power:
\begin{itemize}
\item Signals with finite, non-zero energy and zero power are called {\it energy signals}.
\item Signals with finite, non-zero power (and by implication infinite energy) are called {\it power signals}.
\end{itemize}

Note, these categories are non-exclusive, some signals are neither energy or power signals.

\section{Unit Impulse Function}
An important CT signal is the unit impulse function, also called the "delta" $\delta$ function for the symbol traditionally used to define it. Applying this signal to a system models a "kick" to that system. For example, consider striking a tuning fork. The reason this signal is so important is that it will turn out that the response of the system to this input tells us all we need to know about a linear, time-invariant system!

\begin{definition}[CT Impulse Function]
The CT impulse function is not really a function at all, but a mathematical object called a "distribution". Some equivalent definitions:

\[
\delta(t) = \lim_{\epsilon \rightarrow 0}\left\{
\begin{array}{ll}
  \frac{1}{2\epsilon} & |t| < \epsilon\\
  0 & \text{else}
\end{array}
\right.
\]

\[
\delta(t) = \lim_{\epsilon \rightarrow 0} \frac{1}{\sqrt{2\pi}\epsilon} e^{-\frac{t^2}{2\epsilon^2}}
\]
Note the area under each definition is always one.
\end{definition}

In practice we can often use the following definition and some properties, without worrying about the distribution functions.
\[
\delta(t) = \left\{
\begin{array}{ll}
  0 & t \neq 0\\
  \infty & t = 0
\end{array}
\right.
\]
which we draw an vertical arrow in plots:
\begin{center}
  \begin{tikzpicture}[scale=0.8, every node/.style={transform shape}]
    \begin{axis}[ xlabel=$t$, axis x line=center, axis y line = center,
        xmin=-2, xmax=2, ymin=0, ymax=1.1, x label style={at={(axis cs:2,0)},anchor=west}]
      \addplot+[ycomb,mark=triangle,style={ultra thick}] plot coordinates {(0,1)};
    \end{axis}
  \end{tikzpicture}
\end{center}
Note the height of the arrow is arbitrary. Often in the case of a non-unit impulse function the area is written in parenthesis near the arrow tip.

The following properties of the impulse function will be used often.

\begin{itemize}
\item The area under the unit impulse is unity since by definition
  \[
  \int\limits_{-\infty}^{\infty} \delta(t) \; dt = 1
  \]
\item Sampling property: $x(t)\delta(t-t_0) = x(t_0)\delta(t-t_0)$
\item Sifting Property:
  \[
  \int\limits_{a}^{b} x(t)\delta(t-t_0) \; dt = x(t_0)
  \]
  for any $a < t_0 < b$.
\end{itemize}

We previously defined the unit step function. The impulse can be defined in terms of the step:
\[
\delta(t) = \frac{du}{dt}
\]
and vice-versa
\[
u(t) = \int\limits_{-\infty}^{t} \delta(\tau) \; d\tau
\]
using the notion of distributions, e.g.

\[
u(t) = \int\limits_{-\infty}^{t} \delta(\tau) \; d\tau = \lim_{\epsilon \rightarrow 0} \int\limits_{-\infty}^{t} \frac{1}{\sqrt{2\pi}\epsilon} e^{-\frac{\tau^2}{2\epsilon^2}} \; d\tau = \lim_{\epsilon \rightarrow 0} \frac{1}{2}\left(1+\text{erf}\left( \frac{t}{\sqrt{2}\epsilon}\right)\right)
\]

The step and impulse function are related, but in many cases finding the response of a system to a step input is easier.

We can apply additional transformations to the impulse and step functions to get other useful signals, e.g.

\begin{itemize}
\item ramp
  \[
  r(t) = \int\limits_{-\infty}^{t} u(\tau) \; d\tau = tu(t)
  \]

\item causal pulse of width $\epsilon$
  \[
  p(t) = u(t) - u(t-\epsilon)
  \]

\item non-causal pulse of width $2\epsilon$
  \[
      p(t) = u(t+\epsilon) - u(t-\epsilon)
      \]
\end{itemize}

\section{CT Complex Exponential}

One of the most important signals in systems theory is the complex exponential:
\[
x(t) = C\, e^{a t}
\]
where the parameters $C, a \in \mathbb{C}$ in general.

When $C$ and $a$ are both real ($\Im(C) = \Im(a) = 0$), we have the familiar exponential. When $a > 0$, $x(t) = C e^{a t}$ looks like:

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[xmin=-2, xmax=2, ymin = 0, ymax=7, samples=50, xlabel=$t$, xticklabels={,,}, yticklabels={,,}]
      \addplot[blue, thick] (x,e^x);
      \addplot[mark=none, black] coordinates {(0,0) (0,7)};
      \addplot[mark=none, black] coordinates {(-2,1) (2,1)};
      \node at (axis cs:0,2) [anchor=north east] {$C$};
    \end{axis}
  \end{tikzpicture}
\end{center}

When $a < 0$, $x(t) = C e^{a t}$ looks like:

\begin{center}
  \begin{tikzpicture}
    \begin{axis}[xmin=-2, xmax=2, ymin = 0, ymax=7, samples=50, xlabel=$t$, xticklabels={,,}, yticklabels={,,}]
      \addplot[blue, thick] (x,e^-x);
      \addplot[mark=none, black] coordinates {(0,0) (0,7)};
      \addplot[mark=none, black] coordinates {(-2,1) (2,1)};
      \node at (axis cs:0,2) [anchor=north east] {$C$};
    \end{axis}
  \end{tikzpicture}
\end{center}

To get the pure sinusoidal case, let $C \in \mathbb{R}$ and $a$ be purely imaginary: $a = j\omega_0$:
\[
x(t) = Ce^{j\omega_0 t}
\]
where $\omega_0$ is the frequency (in radians/sec). This is called the complex sinusoid.

By Euler's identity:
\[
e^{j\omega_0 t} = \cos(\omega_0 t) + j\sin(\omega_0 t)
\]
and
\[
\Re(x(t)) = \cos(\omega_0 t) = \frac{1}{2}\left( e^{j\omega_0 t} + e^{-j\omega_0 t} \right)
\]
\[
\Im(x(t)) = \sin(\omega_0 t) = \frac{1}{2j}\left( e^{j\omega_0 t} - e^{-j\omega_0 t} \right)
\]
are both real sinusoids.

Note that the sinusoids are periodic. Recall a signal $x(t)$ is periodic with period $T$ if
\[
x(t) = x(t+T) \; \forall t
\]
In the case of the complex sinusoid
\[
Ce^{j\omega_0 t} = Ce^{j\omega_0 (t+T)}= Ce^{j\omega_0 t}\underbrace{e^{j\omega_0 T}}_{\text{must be 1}}
\]

\begin{itemize}
\item if $\omega_0 = 0$ this is true for all $T$
\item if $\omega_0 \neq 0$, then to be periodic $\omega_0 T = 2\pi m$ for $m = \pm 1, \pm 2, \cdots$. The smallest $T$ for which this is true is the {\it fundamental period} $T_0$
  \[
  T_0 = \frac{2\pi}{|\omega_0|}
  \]
  or equivalently $\omega_0 = \frac{2\pi}{T_0}$
\end{itemize}

Some useful properties of sinusoids:

\begin{itemize}
\item If x(t) is periodic with period $T$ and $g$ is any function then $g(x(t))$ is periodic with period $T$.
\item If $x_1(t)$ is periodic with period $T_1$ and $x_2(t)$ is periodic with period $T_2$, and if there exists positive integers $a,b$ such that
  \[
  aT_1 = b T_2 = P
  \]
  then $x_1(t) + x_2(t)$ and $x_1(t)x_2(t)$ are periodic with period $P$
\end{itemize}
The last property implies that both $T_1$ and $T_2$ must both be rational in $\pi$ or neither should be. For example

\begin{itemize}
\item $x(t) = \sin(2\pi t) + \cos(5\pi t)$ is periodic
\item $x(t) = \sin(2 t) + \cos(5 t)$ is periodic
\item $x(t) = \sin(2\pi t) + \cos(5 t)$ is {\bf not} periodic
\end{itemize}

When the parameter $C$ is complex we get a phase shift. Again let $a = j\omega_0$. When $C$ is complex we can write it as $C = Ae^{j\phi}$ where $A = |C|$ and $\phi = \angle C$. Then

\[
x(t) = Ae^{j\phi} e^{j\omega_0 t} = Ae^{j(\omega_0 t+\phi)}
\]
and
\[
\Re(x(t)) = A\cos(\omega_0 t+\phi)
\]
\[
\Im(x(t)) = A\sin(\omega_0 t+\phi)
\]

Since $\sin$ is a special case of $\cos$, i.e. $\cos(\theta) = \sin(\theta + \frac{\pi}{2})$, the general real sinusoid is

\[
A\cos(\omega_0 t + \phi)
\]

\begin{itemize}
\item $A$ is called the amplitude
\item $\omega_0$ is again the frequency in radians/sec.
\item $\phi$ is called the phase shift and is related to a time shift $T_s$ by
  \[
  \phi = \omega_0T_s
  \]
\end{itemize}

For example the signal graphically represented as follows
\begin{center}
  \includegraphics[scale=0.6]{graphics/ctsinusoid.pdf}
\end{center}
has the functional representation
\[
x(t) = 2\cos\left(\frac{\pi}{2} (t+\tfrac{1}{2}) \right) =  2\cos\left(\frac{\pi}{2} t +\frac{\pi}{4} \right)
\]

\subsection{Energy of CT complex sinusoid}

Recall the energy of a CT signal $x(t)$ is
\[
  E_x = \lim_{T\rightarrow\infty} \int\limits_{-T}^T \lvert x(t) \rvert^2 dt \; .
\]
Substituting $x(t) = e^{j\omega_0 t}$ and letting $T = N T_0$
  \[
    E_x = \lim_{N\rightarrow\infty} \int\limits_{-N T_0}^{N T_0} \underbrace{\lvert e^{j\omega_0 t} \rvert^2}_{\text{always 1}} \; dt = \lim_{N\rightarrow\infty} 2NT_0 = \infty
  \]

\subsection{Power of CT complex sinusoid}

Recall the power of a CT signal $x(t)$ is
\[
  E_x = \lim_{T\rightarrow\infty} \frac{1}{2T} \int\limits_{-T}^T \lvert x(t) \rvert^2 dt \; .
\]
Again, substituting $x(t) = e^{j\omega_0 t}$ and letting $T = N T_0$
\[
  E_x = \lim_{N\rightarrow\infty} \frac{1}{2NT_0} \int\limits_{-N T_0}^{N T_0} \underbrace{\lvert e^{j\omega_0 t} \rvert^2}_{\text{always 1}} \; dt = \lim_{N\rightarrow\infty} \frac{1}{2NT_0} 2NT_0 = 1
\]

\subsection{Harmonics}

Two CT complex sinusoids are {\it harmonics} of one another is both are periodic in $T_0$. This occurs when

\[
    x_k(t) = e^{jk\omega_0 t} \; \text{for} \; k = 0, \pm 1, \pm 2, \cdots
\]

The term comes from music where the vibrations of a string instrument are modeled as a weighted combination of harmonic tones.

\subsection{Geometric interpretation of the Complex Exponential}

In the general case we get a sinusoid signal modulated by an exponential. Let $C = Ae^{j\phi}$ and $a = r + j\omega_0$, then
\[
  x(t) = C e^{a t} =  Ae^{j\phi} e^{(r+j\omega_0)t}
\]
Expanding the terms and using Euler's identity gives:
\[
x(t) = \underbrace{Ae^{rt}\cos(\omega_0 t+\phi)}_{\Re \text{part}} + j \underbrace{Ae^{rt}\sin(\omega_0 t+\phi)}_{\Im \text{part}}
\]
Each part is a real sinusoid whose amplitude is modulated by a real exponential.

An important visualization of the general case is to view the signal x(t) as a vector rotating counter-clockwise in the complex plane for positive $t$.

\begin{center}
 \includegraphics[scale=0.7]{graphics/CT_complexsinusoid_visual}
\end{center}

For $r < 0$ the tip of the arrow traces out an inward spiral, whereas for $r > 0$ it traces out an outward spiral. For $r = 0$ it traces out the unit circle.

\section{Solved Problems}

\begin{enumerate}

\item Consider a signal described by the function
  \[
  x(t) = e^{-3t}\sin(10\pi t)u(t)
  \]

  a) Determine the magnitude and phase of $x\left( \frac{1}{20}\right)$\\
  \textbf{Solution:}
  Substituting $t = \frac{1}{20}$ gives
  \[
  x\left( \frac{1}{20}\right) = e^{-3\frac{1}{20}}\sin\left(10\pi \frac{1}{20}\right)u\left( \frac{1}{20}\right) = e^{-\frac{3}{20}} \approx 0.86
  \]
  Since the signal is purely real and exponential is always positive, the magnitude is
  \[
  \left|x\left( \frac{1}{20}\right)\right| = \left| e^{-\frac{3}{20}}\right| =  e^{-\frac{3}{20}}  \approx 0.86
  \]
  and the phase is
  \[
  \angle x\left( \frac{1}{20}\right) = 0
  \]

  b) Using Matlab, plot the signal $|x(t)|$ between $[-2, 2]$. Give your code and embed the plot.\\

  \textbf{Solution:}
\begin{verbatim}
% Solution to PS01 - Problem 1b
t = -2:0.001:2;
x = exp(-3*t).*sin(10*pi*t).*heaviside(t);
hp = plot(t,abs(x));
grid on;
xh = xlabel('t');
yh = ylabel('|x(t)|');
th = title('Plot for PS01 Problem 1b');

% make the plot more readable
set(gca, 'FontSize', 12, 'Box', 'off', 'LineWidth', 2);
set(hp, 'linewidth', 2);
set([xh, yh, th], 'FontSize', 12);

set(gcf, 'PaperPositionMode', 'auto');
print -dpng ps1p1b.png
\end{verbatim}
%% \begin{center}
%% \includegraphics[scale=0.8]{graphics/ps1p1b.png}
%% \end{center}

  \item Find a solution to the differential equation
  \[
  \frac{dy}{dt}(t) + 9y(t) = e^{-t}
  \]
  for $t \geq 0$, when $y(0) = 1$.\\

  \textbf{Solution:}
  The homogeneous equation is
  \[
  \frac{dy_h}{dt}(t) + 9y_h(t) = 0
  \]
  with initial condition $y_h(0) = 1$. It's solution is of the form
  \[
  y_h(t) = C\, e^{-9t}
  \]
  for constant $C$. Using the initial condition
  \[
  y_h(0) = C\, e^{-0} = C = 1
  \]
  gives
  \[
  y_h(t) = e^{-9t}
  \]
  The particular solution is of the form
  \[
  y_p(t) = C_1 e^{-t} + C_2 e^{-9t}
  \]
  Substitution and equating coefficients gives $C_1 = \frac{1}{8}$ and $C_2 = -\frac{1}{8}$. The total solution is the sum of the two solutions or
  \[
  y(t) = \frac{1}{8} e^{-t} - \frac{1}{8} e^{-9t} + e^{-9t} = \frac{1}{8} e^{-t} + \frac{7}{8} e^{-9t}
  \]

  \item Compute the integral
    \[
    \int\limits_{-\infty}^{\infty} e^{-t^2} \, \delta(t-10)\; dt
    \]
    where $\delta(t)$ is the delta function.\\
    \textbf{Solution:}

  Using the sifting property of the delta function
  \[
  \int\limits_{a}^{b} f(t) \, \delta(t-t_0)\; dt = f(t_0)
  \]
  for $a < t_0 < b$, we get
  \[
  \int\limits_{-\infty}^{\infty} e^{-t^2} \, \delta(t-10)\; dt = e^{-100} \approx 0
  \]
\end{enumerate}


\subsection{Method of moments}

\subsubsection{Introduction}

If we have \(k\) parameters to estimate, we can solve this if we have \(k\) equations.

We generate these

First, we link each first \(k\) moments to functions of the parameters.

Then we replace the momenets with sample estimates.

\subsubsection{Estimation}

The moments of this population distribution are:

\(\mu_i =E[X^i]=g_i(\theta_1,...,\theta_k)\)

We have a sample.

\(X=[X_1,...,X_n]\)

We now define the method of moments estimator

\(\hat \mu_i=\dfrac{1}{n}\sum_{j=1}^nx_j^i\)


\documentclass[fleqn]{article}

\usepackage{mathtools}
\usepackage{nccmath}
\usepackage{graphicx}
\usepackage{float}
\usepackage{booktabs}

\usepackage [english]{babel}
\usepackage [autostyle, english = american]{csquotes}
\MakeOuterQuote{"}

\DeclarePairedDelimiter\Floor\lfloor\rfloor
\DeclarePairedDelimiter\Ceil\lceil\rceil

\title{ODE model of the spread of measles in secondary schools}
\author{David Gurevich}
\date{}

\begin{document}
\maketitle

The following introduces a system of differential equations that is used to model the spread of measles through a secondary school.

\section*{List of Symbols}

	$\beta$ - Probability of disease transmission following an interaction \\
	$\sigma$ - Rate of maturation from exposed class to infected class \\
	$\alpha$ - Rate at which symptoms appear in an infected individual \\
	$\gamma$ - Rate at which infected individuals recover following the first appearance of \indent symptoms \\
	$\omega$ - Probability of an individual visiting the bathroom \\
	$q$ - Quantity of virus "shed" per minute \\
	$Q$ - Ventilation of air inside washroom (per cubic meter) \\
	$V$ - Volume of each washroom (cubic meters) \\
	$P$ - Pulmonary ventilation rate (cubic meters per minute) \\
	$T$ - Average duration of washroom visit \\
	$\mathcal{F}$ - Symmetrical contact matrix for friends between grades \\
	$\mathcal{C}$ - Symmetrical contact matrix for classmates between grades \\

\newpage

\section*{SVEIR Model}

$S_i$ - Proportion of population $i$ that is susceptible \\
$V_i$ - Proportion of population $i$ that is vaccinated \\
$E_i$ - Proportion of population $i$ that is exposed \\
$I_i$ - Proportion of population $i$ that is infected \\
$H_i$ - Proportion of population $i$ that is at home because of infection \\
$R_i$ - Proportion of population $i$ that has recovered \\
$W$ - Sum of all concentrations of measles virus in the air in washrooms\\ \\
$\tau$ - Day state: 1 if students are at home, 2 if students are in the halls, 3 if \indent students are in class
\begin{equation}
\dfrac{dS_i}{dt} =
\begin{dcases}
	0 & \text{if $\tau = 1$} \\
	- \Big[\Big(\sum_{j=1}^4 \dfrac{2}{20} \beta \mathcal{F}_{i,j} I_j \Big) + \dfrac{1}{6} \omega \beta W P \Big](\dfrac{S_i}{N_i}) & \text{if $\tau = 2$} \\
	- \Big[\Big(\sum_{j=1}^4 \dfrac{2}{20} \beta \mathcal{C}_{i,j} I_j \Big) + \dfrac{1}{6} \omega \beta W P \Big](\dfrac{S_i}{N_i}) & \text{if $\tau = 3$}
\end{dcases}
\end{equation}

\begin{equation}
\dfrac{dV_i}{dt} = 0
\end{equation}

\begin{equation}
\dfrac{dE_i}{dt} =
\begin{dcases}
	0 & \text{if $\tau = 1$} \\
	\Big[\Big(\sum_{j=1}^4 \dfrac{2}{20} \beta \mathcal{F}_{i,j} I_j \Big) + \dfrac{1}{6} \omega \beta W P \Big](\dfrac{S_i}{N_i}) - \sigma E_i & \text{if $\tau = 2$} \\
	\Big[\Big(\sum_{j=1}^4 \dfrac{2}{20} \beta \mathcal{C}_{i,j} I_j \Big) + \dfrac{1}{6} \omega \beta W P \Big](\dfrac{S_i}{N_i}) - \sigma E_i & \text{if $\tau = 3$}
\end{dcases}
\end{equation}

\begin{equation}
\dfrac{dI_i}{dt} = \sigma E_i - \alpha I_i
\end{equation}

\begin{equation}
\dfrac{dH_i}{dt} = \alpha I_i - \gamma H_i
\end{equation}

\begin{equation}
\dfrac{dR_i}{dt} = \gamma H_i
\end{equation}

\begin{equation}
\dfrac{dW}{dt} =
\begin{dcases}
\dfrac{-Q}{V} & \text{if $\tau = 1$} \\
\dfrac{-Q}{V} + \dfrac{1}{6} \omega q T \sum_{j=1}^4 I_j & \text{if $\tau = 2, 3$}
\end{dcases}
\end{equation}

\newpage

\begin{figure}[t!]
	\vspace*{-1.5in}
	\hspace{-2.7in}
	\includegraphics[scale=0.52]{fig.png}
\end{figure}
\begin{center}
Figure 1: Plot of ODE solution to model
\begin{table}[h]
\begin{tabular}{@{}cccc@{}}
$\beta$ = 0.91                                                                                & $\dfrac{1}{\gamma}$ = 4 & $Q$ = 5                                                          & $T$ = 2.2                                                        \\
$\dfrac{1}{\sigma}$ = 11                                                                                & $\omega$ = 0.00047     & $V$ = 32                                                         &                                                                  \\[0.4cm]
$\dfrac{1}{\alpha}$ = 4                                                                                & $q$ = 144              & $P$ = 0.00556                                                    &                                                                  \\
\multicolumn{1}{l}{}                                                                                & \multicolumn{1}{l}{}   & \multicolumn{1}{l}{}                                             & \multicolumn{1}{l}{}                                             \\
\multicolumn{1}{l}{$ \mathcal{F} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.03 & 0.97 & 0.03 & 0 \\ 0 & 0.03 & 0.097 & 0.03 \\ 0 & 0 & 0.03 & 0.97 \end{bmatrix}$} & \multicolumn{1}{l}{}   & \multicolumn{2}{l}{$ \mathcal{C} = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}$}
\end{tabular}
\end{table}

\end{center}

\end{document}