\def\mA{\ensuremath{{\bf A}}}
\def\mB{\ensuremath{{\bf B}}}
\def\mC{\ensuremath{{\bf C}}}
\def\mD{\ensuremath{{\bf D}}}
\def\mu{\ensuremath{{\bf u}}}
\def\mx{\ensuremath{{\bf x}}}
\def\my{\ensuremath{{\bf y}}}

\documentclass[12pt]{article}
\usepackage{siunitx}
\begin{document}
\title{Drive kinematics and simulation}
\author{sss}
\maketitle

A useful reference: http://fbsbook.org.

\section{Motor constants}

A DC motor typically consists of a permanent magnet rotating relative
to a conducting coil.  As the magnet rotates, it induces an EMF in the
coil.  This `back-EMF' is proportional to the angular velocity:
\begin{equation}
  E = K_e \omega .
\end{equation}

A DC motor can be modeled as a resistance $R$ in series with the back EMF.
Therefore, given the applied voltage $V$ and current $I$:
\begin{equation}
  V = IR + E = IR + K_e \omega.
\end{equation}

In other words, when a voltage is applied to the motor, it increases in speed
until the back-EMF (plus resistive losses) balance the applied voltage.

If the current with no load is $I_0$, then the maximum speed of the motor
is
\begin{equation}
  \omega_{\max} = {V-I_0R \over K_e}.
\end{equation}

Also note that when the motor is stalled, $E=0$, so the resistance
can be found using the current drawn at the stall:
\begin{equation}
  R = V / I_{\mathrm{stall}}.
\end{equation}

It is common to write $K_V = 1/K_e$.  Then the motor speed for a given applied
voltage is
\begin{equation}
  \omega = K_V (V-IR).
\end{equation}
In practice, the $IR$ term is usually small compared to $V$ and can be
neglected, giving
\begin{equation}
  \omega = K_V V.
\end{equation}
$K_V$ is sometimes called the motor velocity constant, and has SI units
of $1/Vs$ (though it is sometimes given as $\textrm{RPM}/V$).

The torque of the motor is related to the current via the motor
torque constant $K_T$:
\begin{equation}
  \tau = K_T I.
\end{equation}
This constant can be calculated from the torque required to stall the motor
and the current draw at the stall:
\begin{equation}
  K_T = \tau_{\textrm{stall}} / I_{\textrm{stall}}.
\end{equation}

$K_T$ and $K_V$ are not independent; they are inversely related.
The electrical power input to the motor is
\begin{equation}
  P_{\textrm{in}} = VI = (IR + K_e\omega)I = I^2R + IK_e\omega
\end{equation}
while the mechanical power out is
\begin{equation}
  P_{\textrm{out}} = \tau\omega = IK_T\omega,
\end{equation}
while losses are
\begin{equation}
  P_{\textrm{loss}} = I^2R.
\end{equation}.
But
\begin{equation}
  P_{\textrm{in}} = P_{\textrm{out}} + P_{\textrm{loss}},
\end{equation}
which implies that
\begin{equation}
  K_T = K_e = 1/K_V.
\end{equation}

To take a specific example, the Falcon 500  motor operates at $\SI{12}{\V}$,
with $I_0 = \SI{1.5}{\A}$, $I_{\textrm{stall}} = \SI{257}{\A}$,
$\tau_{\textrm{stall}} = \SI{4.69}{\N\m}$,
and $\omega_{\max} = \SI{6380}{RPM} = \SI{668}{/\s}$.
This then gives $R=V/I_{\textrm{stall}}=\SI{0.047}{\ohm}$,
$K_V = \omega_{\max} / (V-I_0R) = \SI{60.0}{/\V\s}$ and
$K_T = \tau_{\textrm{stall}} / I_{\textrm{stall}} = \SI{0.018}{\N\m/\A}$.
And $1/60 = 0.017$, so these are nearly inverse of each other.
(Note that since power is voltage times current,
$\textrm{VA} = \textrm{Nm}/\textrm{s}$ or $\textrm{Vs} = \textrm{NM}/\textrm{A}$.)

\section{A unicycle}

Let's now apply this to a putative vehicle with a single wheel,
constrained to move in one dimension.  In addition to
$R$, $K_V$, and $K_T$ of the motor, let $m$ be the mass,
$r_w$ be the wheel radius, and $g$ be the gear ratio between
the motor and the wheel.

Let $v$ be the (one-dimensional) velocity and $V$ be the voltage
applied to the motor.  Then we want to write:
\begin{equation}
  m \dot v = c_1 v + c_2 V.
\end{equation}
(Using the notation $\dot v = dv/dt$.)
So the right-hand side represents the force acting on the vehicle.

Consider the $c_2$ term first.  If the motor voltage is $V$, then the
current is $I = V/R$, the motor torque is $\tau_m = IK_T = K_T V/R$,
the wheel torque is $\tau_w = gK_T V/R$, and the force on the vehicle
is $F = \tau_w / r_w$, which implies
\begin{equation}
  c_2 = {g K_T \over R r_w}.
\end{equation}

Now, consider the case where the voltage on the motor is zero.
A rotating motor generates an EMF, so if the voltage is zero, then
something must be absorbing the power produced by the motor; hence,
the motor will act as a brake.  We can thus write
\begin{equation}
  m \dot v = - { g K_T \over R r_w } E,
\end{equation}
where $E$ is the back-EMF.  But $E = K_e\omega_m = g \omega_w / K_V =
g v / r_w K_V$.  Putting this together, we get
\begin{equation}
  c_1 = - { g^2 K_T \over K_V R r_w^2}.
\end{equation}

To specify the state of the vehicle, we also need its position $s$.
The equation for this is trivial:
\begin{equation}
  \dot s = v.
\end{equation}

\section{State space representation}

The dynamical variables describing the state of a system make up the
\emph{state space}.  These variables contain sufficient information
to find the future state of the system.

Formally, we write the state variables as an $n$-element vector $\mx$.
The system can also have control inputs, which we represent
as a $p$-element vector $\mu$.  We can also make measurements
on the system, which we represent as an $q$-element vector $\my$.
The system can then be described by
\begin{eqnarray}
  {\bf\dot x} &=& f(\mx, \mu)\\
  \my &=& h(\mx, \mu)
\end{eqnarray}

If $f$ and $h$ are time-invariant and linear, then we can write this as
\begin{eqnarray}
  {\bf\dot x} &=& \mA\mx + \mB\mu \\
  \my &=& \mC\mx + \mD\mu.
\end{eqnarray}
The matrix $\mA$ is called the `system' or `dynamics' or `state' matrix;
$\mB$ is called the `input' or `control' matrix;
$\mC$ is called the `output' or `sensor' matrix; and
$\mD$ is called the `feedthrough' or `feedforward' or `direct' matrix.

\section{Differential drive}

Here, the system consists of two identical motors with wheels separated
by distance $2r_b$, and the system can move in two dimensions.
The mass is again $m$, and the moment of inertia is $I$.

The state variables here are the $x$ and $y$ positions, $s_x$ and $s_y$,
the heading $\theta$, the positions of the two wheels
$d_L$ and $d_R$, and their velocities $\dot d_L$, $\dot d_R$.
We can write this as a vector:
$\mx = [s_x, s_y, \theta, \dot d_L, \dot d_R, d_L, d_R]^T$.
The control inputs are the motor voltages: $\mu = [V_L, V_R]^T$.

Using the results from the previous section, the forces on the
vehicle produced by the left and right motors are
\begin{equation}
  F_L = c_1 \dot d_L + c_2 V_L.
\end{equation}
and similarly for the other motor.  Given $F_L$ and $F_R$,
the total force is $F_{\textrm{tot}} = F_L + F_R$ and the torque is
$\tau = (F_L -F_R) r_b$.  Therefore, the acceleration for one
of the wheels is
\begin{eqnarray}
  \ddot d_L &=& {1\over m}F_{\textrm{tot}} + {r_b\over I}\tau\\
            &=& {1\over m}(F_L + F_R) + {r_b^2\over I}(F_L - F_R)\\
            &=& \left({1\over m} + {r_b^2\over I}\right) c_1 \dot d_L +
                \left({1\over m} - {r_b^2\over I}\right) c_1 \dot d_R +\\
            & & \left({1\over m} + {r_b^2\over I}\right) c_2 V_L +
                \left({1\over m} - {r_b^2\over I}\right) c_2 V_R \nonumber\\
            &=& c_3 c_1 \dot d_L + c_4 c_1 \dot d_R + c_3 c_2 V_L + c_4 c_2 V_R ,
\end{eqnarray}
where
\begin{eqnarray}
  c_3 &=& \left({1\over m} + {r_b^2\over I}\right) \\
  c_4 &=& \left({1\over m} - {r_b^2\over I}\right).
\end{eqnarray}
Similarly,
\begin{equation}
  \ddot d_R = c_4 c_1 \dot d_L + c_3 c_1 \dot d_R + c_4 c_2 V_L + c_3 c_2 V_R.
\end{equation}

In addition, if we call the forward velocity
$v = {1\over 2}(\dot d_L + \dot d_R)$, then
\begin{eqnarray}
  \dot s_x &=& v \cos \theta \\
  \dot s_y &=& v \sin \theta \\
  \dot \theta &=& (\dot d_R - \dot d_L) / 2r_b.
\end{eqnarray}

For the subset of state variables
$\mx = [\dot d_L, \dot d_R, d_L, d_R]^T$, with $\mu = [V_L, V_R]^T$,
the dynamics can then be written in the notation of the previous
section as
\begin{equation}
  \dot\mx = \mA\mx + \mB \mu
\end{equation}
with
\begin{equation}
  \mA = \left[ \begin{array}{cccc}
  c_1c_3 & c_1c_4 & 0 & 0 \\
  c_1c_4 & c_1c_3 & 0 & 0 \\
  1 & 0 & 0 & 0 \\
  0 & 1 & 0 & 0 \\
\end{array} \right]
\end{equation}
and
\begin{equation}
  \mB = \left[ \begin{array}{cc}
  c_2c_3 & c_2c_4 \\
  c_2c_4 & c_2c_3 \\
  0 & 0 \\
  0 & 0 \\
\end{array} \right].
\end{equation}

\section{Runge-Kutta integration}

Given
\begin{equation}
  {dy \over dt} = f'(y, t)
\end{equation}
with an initial value of $y$ at some $t$, one can use Euler's method
to integrate this in steps of $t$ of size $h$:
\begin{equation}
  y_{n+1} = y_n + hf'(y_n, t_n).
\end{equation}

In practice, however, this is not a good method to use.
The error per step is $O(h^2)$, as can be seen from a series expansion:
\begin{equation}
  y(t_n+h) = y(t_n) + {dy\over dt}(t_n) h + O(h^2).
\end{equation}
So over the $\sim 1/h$ steps needed to cover a fixed-size interval,
the error is $O(h)$.  This is not very accurate.

We can do better by first taking a trial step to the midpoint
of the interval and then using the derivatives calculated there
for entire step.  That is,
\begin{eqnarray}
  k_1 &=& h f'(t_n, y_n) \\
  k_2 &=& h f'(t_n + {h\over2}, y_n + {k_1\over2}) \\
  y_{n+1} &=& y_n + k_2.
\end{eqnarray}
This ends up canceling some the first order error terms, giving
a step error of $O(h^3)$.

One can go further in canceling error terms.  Fourth-order Runge-Kutta
is defined by
\begin{eqnarray}
  k_1 &=& h f'(t_n, y_n) \\
  k_2 &=& h f'(t_n + {h\over2}, y_n + {k_1\over2}) \\
  k_3 &=& h f'(t_n + {h\over2}, y_n + {k_2\over2}) \\
  k_4 &=& h f'(t_n + h, y_n + k_3) \\
  y_{n+1} &=& y_n + {k_1\over6} + {k_2\over3} + {k_3\over3} + {k_4\over6}
\end{eqnarray}
and has a step error of $O(h^5)$.  One can extend this further,
but that is usually not found to be worthwhile.

\end{document}


\documentclass[12pt,onecolumn,a4paper]{report}

\usepackage{mathtools}
\usepackage[top=60pt,bottom=60pt,left=50pt, right=50pt]{geometry}
\usepackage{graphicx}
\usepackage{wrapfig}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{listings}

\graphicspath{{../plots/}}
\renewcommand*\thesection{\arabic{section}}

\newfont{\pica}{cmpica scaled 800}

\begin{document}
\title{COMP3032 Assignment: Image Contour Extraction}
\author{ \textbf{ Dionisio Perez-Mavrogenis}\\
		School of Electronics and Computer Science \\
		Faculty of Physical and Applied Sciences \\
		University of Southampton}
\maketitle

\tableofcontents
\pagebreak

\section{Introduction}
\subsection{Equations used}

The equations used for extracting the optimal contour from the provided image were the following :
\begin{equation}
E(v) = \sum^{N-2}_{i=1} E_i (v_{i-1},v_i,v_{i+1})
\end{equation}

\begin{equation}
E_i (v_{i-1},v_i,v_{i+1}) = \lambda \frac{|v_{i+1} - 2v_i + v_{i+1}|^2}{|v_{i+1} - v_{i-1}|^2} + (1-\lambda)I(v_i)
\end{equation}

\begin{equation}
S_i(v_{i+1},v_i) = \min_{v_{i-1}}[S_{i-1}(v_i,v_{i-1}) + E_i (v_{i-1},v_i,v_{i+1})]
\end{equation}

\begin{equation}
E(v_i) = \lambda |v_i - v_{i-1}| + (1-\lambda)I(v_i)
\end{equation}

\begin{equation}
S_i(v_i) = \min_{v_{i-1}}[S_{i-1}(v_{i-1})+E_i(v_{i-1},v_i)]
\end{equation}

where $\lambda \in [0,1]$ is the regularization parameter and $S_0(v_1,v_0) = 0$.
\section{Approach}
The coursework was coded in Matlab. Although the approach I have chosen is not exactly the same as the one in the specification(with differences highlighted as encountered), my implementation works fairly well.
The image used was the file "tongue.png" provided with the specification, but the images displayed in the report are zoomed-in in the area enclosed be the contours because it is easier to spot details.
\subsection{Constructing the search space}
The construction of the search space is done by the function {\pica get\_search\_space}. Given a number $M$, the search space created will have $M*N$ points, where $M$ includes both points on the starting contours(only $M-2$ points are created).

The search space is constructed in the following manner :
for each point in the first contour, calculate the $x$ and $y$ difference with the corresponding point in the second contour ($ xdif_i = abs[v_{i,1}(x) - v_{i,2}(x)]/(M-1),ydif_i = abs[v_{i,1}(y) - v_{i,2}(y)]/(M-1) $) and then to get each new point $v'(i)$ between points $v_{i,1},v_{i,2}$ do :

\begin{displaymath}
v'_i = [xdif_i*(J-1)+v_{i,1}(x),ydif_i*(J-1)+v_{i,1}(y)], \forall J \in [1,M]
\end{displaymath}

where $v_{i,1}$ is point $i$ in contour $1$ and $dif_i$ is the absolute difference between points $(v_{i,1},v_{i,2})$(with $x$ and $y$ denoting the corresponding coordinate).I add $dif*(J-1)$ to $v_{i,1}$ because contour one is assumed to be "bellow" contour two, that is, it has $x_1 \leq x_2$ and $y_1 \leq y_2$.
The {\pica get\_search\_space} function is given as inputs an M, the vector of contour1, the vector of contour2 and the matrix of intensities of the image and returns the intensity matrix for the search space points and matrices $X$ and $Y$, which contain the corresponding $x$ and $y$ coordinates of the search space points.
Because the results of calculating the search space point coordinates might not always yield an integer, Matlab's inbuilt function {\pica round} function was used.

The code for {\pica get\_search\_space} is given in Appendix A 0.1, Listing A.2, and samples of the generated search space are shown in Appendix B 0.3, Figure B.1 .

\subsection{Construction of Energy and Position matrices}

The function {\pica get\_matrices} is responsible for calculating the energy and position matrices. It is supplied with arguments : $\lambda$ (the regularization parameter), the intensity matrix calculated by {\pica get\_search\_space}, and the $X$ and $Y$ matrices(also calculated by {\pica get\_search\_space}), and returns the energy matrix (an $M\times N\times M$ matrix) and a vector describing which points belong to the optimal contour (the backtracking is also done in this function), which then is handled by the plotting functions who locate the actual points on the image later on.

For a given point $v_i$ , the entry in the energy matrix is calculated according to equation(3), with resulting value being inserted in $en(y,x,z)$ and $z$ being equal to the $y$ index of $v_{i+1}$. The energy up to point $v_i$ is taken to be the energy between points $v_{i-1}$ and $v_i$, as this is assumed to be the optimal between these two and thus the optimal between the triplet.

Simultaneously, the position matrix is being built. At each position, $posm(y,x,z)= z' $, where $z'$ is the $y$-index of the optimal $v_{i-1}$ point and $z$ represents the $y$-index of the $v_{i+1}$ point for which this calculation is taking place.

The first and last columns of the position and energy matrices present a special case, since we need points $v_0$ and $v_{N+1}$ to calculate results for points in columns 1 and N, and such points do not exist. Therefore, column 1 is initialized to 0,  and the energy for points in column N is calculated by equations (4) and (5).

Backtracking involved searching through the N-1 column of the energy matrix for the row that contained the minimum value , record the position of the row($y$-index) and the position of the minimum value in the row($z$-index). The rest of the procedure involves setting $y=z$ and reading $posm(y,x,z)$ into  $z$, then repeating the process for every $x \in [2,N-1]$.

The code for this section is given in Appendix A, Listing A.3. The plotting functions are called in the invoking script, which is given in Appendix A, Listing A.1.

\section{Evaluation of the algorithm}
The evaluation following is not general and refers only to my implementation. Also, timings were performed using Matlab's {\pica tic toc} function, on machines in the Zepler labs.

\subsection{Robustness}
\paragraph{Robustness to different initialisations }

\begin{figure}[Ht]
        \centering
        \begin{subfigure}[b]{0.45\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{clcap-100-0.05000}.png}
                \caption{Initialized to 0.}
        \end{subfigure}%
        ~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
          %(or a blank line to force the subfigure onto a new line)
        \begin{subfigure}[b]{0.45\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{clcapnz-100-0.05000}.png}
                \caption{Initialized to intensity.}
        \end{subfigure}
        \caption{Optimal contour for different initialization of the energy matrix(M=100,$\lambda=0.05$.)}
\end{figure}

Two different initialization methods were tried to test the algorithms robustness. The first method set the energies of the first column to zero, as the coursework suggested, whereas the second set each point's energy to be equal to its intensity.

Figure 1 displays the resulting optimal contour for different initializations. The algorithm is robust enough so that the only difference is not on the actual contour being traced, but on the black space to the left of the tongue. Once the algorithm finds the optimal contour, the two contours are effectively identical.

\paragraph{Robustness for varying values of $\lambda$}
With an initial value of $\lambda=0.5$ the results were not satisfactory,as the result of tracing deviated a lot from the desired result, whereas when varying $\lambda$ it was found that acceptable results are obtained when $\lambda < 0.2$, and improved as $\lambda$ got smaller.

\begin{figure}[Ht]
        \centering
        \begin{subfigure}[b]{0.45\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{clcap-100-0.05000}.png}
                \caption{$\lambda=0.05$}
        \end{subfigure}%
        ~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
          %(or a blank line to force the subfigure onto a new line)
        \begin{subfigure}[b]{0.45\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{clcap-100-0.50000}.png}
                \caption{$\lambda=0.5$}
        \end{subfigure}
        \caption{Optimal contour for different values of $\lambda$(M=100)}
\end{figure}

Figure 2 shows a comparison for different $\lambda$ values and Appendix B 0.4 shows resulting traces for for different values of $\lambda$

\subsection{Performance}

\begin{figure}[Htb]
\includegraphics[scale=0.5]{3D/cw_3d_benchmark.png}
\centering
\caption{Benchmark of 3D matrix implementation for various values of M.}
\end{figure}

This algorithm performs for $N-1$ columns(where $N$ is the number of points in the initial contours), for $M$ points, $M\times M$ operations and thus a good approximation for the running time would be $N\times M^3$, and one hypothesis suggests that the algorithms has $O(M^3)$ running time, since N could be considered a constant. My implementation uses four {\pica for}-loops to perform the calculations and all calculations have been moved to the outermost loop possible .

Figure 3 is plot of the value of $M$ and the corresponding running times. The shape of the curve suggests an exponential function for the total running time. It has been observed that changing the value of $\lambda$ has no impact on the running time for any given $M$.

\subsection{Application to other images}

\begin{figure}[Ht]
        \centering
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{curve.png}
                \caption{The random curve}
        \end{subfigure}%
        ~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
          %(or a blank line to force the subfigure onto a new line)
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{curvez-150-0.05000}.png}
                \caption{The traced contour.}
        \end{subfigure}
        \caption{Application to a random curve(M=150)}
\end{figure}
The algorithm was applied to the image of a curve that I created, and the results are shown in Figure 4.
The result shown in Figure 4(b) is for M=150,$\lambda$=0.05 and a zero initialization of the energy matrix's first column.
There is a small part in the beginning of the curve that is not tracked (missing points from the initial contours), and the two initial points of the optimal contour that are completely off the curve, as well as the last point of the tracing. These might be attributed to the initialization of the energy matrix's first column and different calculation for the last column's energy. However, when the algorithm "finds" the black curve it tracks the curve correctly, confirming that the initial and last points pose a special case and could perhaps be ignored, or be dealt with in a different fashion.

\subsection{Comparison with a different algorithm}

\begin{figure}[Ht]
\includegraphics[scale=0.5]{2D/cw_2d_benchmark.png}
\centering
\caption{Benchmark of 2D matrix implementation for various values of M.}
\end{figure}

\begin{figure}[Ht]
        \centering
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{clcap-100-0.05000}.png}
                \caption{3D matrix, M=100,$\lambda=0.05$}
        \end{subfigure}%
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{2D/{2d-clcap-100-0.05000}.png}
                \caption{2D matrix, M=100,$\lambda=0.05$}
        \end{subfigure}

        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{clcap-300-0.05000}.png}
                \caption{3D matrix, M=300,$\lambda=0.05$.}
        \end{subfigure}

        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{2D/{2d-clcap-300-0.05000}.png}
                \caption{2D matrix, M=300,$\lambda=0.05$.}
        \end{subfigure}
        \caption{Comparison of the two algorithms}
\end{figure}

\begin{figure}[Ht]
\includegraphics[scale=1]{2D/{2d-curve-150-0.00500}.png}
\centering
\caption{Tracing of curve by 2D algorithm ($M=150,\lambda=0.005$), with portions of the contour being completely straight, illustrating this algorithm's linear sense of continuity.}
\end{figure}

A second algorithm was implemented in order to extract the contour of the tongue. This is the algorithm presented in the "Dynamic Programming" lecture, and uses Equations (4) and (5) to extract the optimal contour. The algorithm performs a total of $N\times M\times M$ operations; a benchmark of this algorithm's performance is given in Figure 5, and its time complexity could be characterized as $O(M^2)$, if we treat $N$ as a constant term.

Figure 6 shows a comparison of the contours extracted from the two algorithms. Both algorithms extracted the optimal contour fairly well, the 2D implementation being simpler and faster to implement and seems to extract a smoother contour. However, the 2D algorithm has got a sense of continuity that approximates a straight line because it only considers two points, something that is illustrated in Figure 7.

Code for this algorithm is given in Appendix A.2.

\section{Conclusion}
This implementation of the algorithm might not be the optimal, as I am not as proficient in Matlab, however it illustrates that the algorithm works and allows comparison with other algorithms.
Also, even though the background of the image containing the tongue is the original one, the algorithms run on the negated version of that image (so that the contour we extract has the minimum energy) and the result is plotted on the original image.

%

%		APPENDICES

%

\pagebreak
\appendix
\chapter{Code listings}
\subsection{Code for the 3D matrix algorithm}

\lstset{ %
  language=Matlab,                % the language of the code
  numbers=left,
  caption={Code used to initialize, run and plot the results for the 3D algorithm.},
}
\lstinputlisting{../start_coursework_3D.m}

\lstset{ %
  language=Matlab,                % the language of the code
  numbers=left,
  caption={Function generating the search space for the 3D algorithm.},
}
\lstinputlisting{../get_search_space.m}

\lstset{
  language=Matlab,
  numbers=left,
  caption={Function generating the position and energy matrices and doing the backtracking.},
}
\lstinputlisting{../get_matrices_3D.m}

\lstset{
  language=Matlab,
  numbers=left,
  caption={Function that gets the optimal points returned from {\pica get\_matrices} and retrieves their (x,y) coordinates, and adds them to an $N\times 2$ vector for the plotting functions.},
}
\lstinputlisting{../get_optimals_3D.m}

\subsection{Energy extraction function of 2D matrix.}
Since the search space extraction and plotting functions are identical to the 3D implementation, only the energy calculation and back-tracking code for the 2D implementations is shown.
\lstset{
  language=Matlab,
  numbers=left,
  caption={Function that generates the position and energy matrices for the 2D implementation of the algorithm.},
}
\lstinputlisting{../2D_implementation/fill_energy_matrix.m}

\lstset{
  language=Matlab,
  numbers=left,
  caption={Function that does the calculation for the transition between points $v_{i-1},v_i$.},
}
\lstinputlisting{../2D_implementation/point_energy.m}

\chapter{Some of the images generated}

\subsection{Images of the search space generated}
These are two sample images generated by the {\pica get\_search\_space} function. The search points are not plotted together with the optimal contour because the resulting image would get too cluttered to draw any results.
\begin{figure}[ht]
        \centering
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{sspace-50}.png}
                \caption{Search space generated for M=50}
        \end{subfigure}%
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{sspace-50}.png}
                \caption{Search space generated for M=100}
        \end{subfigure}
        \caption{Plotting of two sample search spaces.}
\end{figure}

\subsection{Traced contours for different values of $\lambda$}
\begin{figure}[H]
\centering
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{100-0.05000}.png}
                \caption{$\lambda=0.05$}
        \end{subfigure}%
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{100-0.15000}.png}
                \caption{$\lambda=0.15$}
        \end{subfigure}
        \begin{subfigure}[b]{0.5\textwidth}
	        	\centering
    		    \includegraphics[width=\textwidth]{3D/{100-0.30000}.png}
	        	\caption{$\lambda=0.3$}
        \end{subfigure}
        \begin{subfigure}[b]{0.5\textwidth}
                \centering
                \includegraphics[width=\textwidth]{3D/{100-0.50000}.png}
                \caption{$\lambda=0.5$}
        \end{subfigure}
        \caption{Plotting of two sample search spaces, with M=100.}
\end{figure}

These tracings are generated by the 3D energy matrix algorithm and have $M=100$.
\end{document}


\subsection{What is Pressure?}
While the Zeroth law of thermodynamics allows for the notion of "temperature" to be well defined, it isn't a particularly useful definition. While it can tell us that, macroscopically, "temperature is what you read from a thermometer", it is of interest to see how the phenomenon of temperature arises microscopically. For this, we will first introduce the notion of pressure\footnote{{\tt https://www.youtube.com/watch?v=a01QQZyl-\_I}}.

We will here define the pressure $P$ as:

\begin{equation}
    P = \frac{F}{A}
\end{equation}

Where $F$ is the force exerted over a certain area $A$. A good way to think about this might be if I have a gas of pressure $P$ contained in a box, that would tell you that if one of the walls of the box has area $A$, there is a perpendicular force of $PA$ being applied.\\

Just how does this help us define temperature? In order for us to connect the pieces, it will be helpful to figure out how the pressure of a gas arises from the microscopic properties of the gas. For this, let us consider a very simple scenario of a single molecule of gas bouncing back and forth in a 1-dimensional tube. This might seem like a strange place to start, but we will shortly see that this simple start leads to valuable information about more complicated scenarios! Note that the below derivation is mainly for your own understanding, and is not especially crucial for your success in Thermodynamics.

\begin{center}
    \begin{tikzpicture}[scale=4]
    \draw[draw=black] (0,0) rectangle(2,0.25);
    \filldraw (1,0.125) circle (0.75pt);
    \draw[red, ->, thick] (1.025,0.125) -- (1.5,0.125);
    \node[below] at (1,0.125) {$m$};
    \node[below] at (1.25,0.125) {$v$};
    \draw[<->] (0,-0.05) -- (2,-0.05);
    \draw[<->] (2.05,0) -- (2.05,0.25);
    \node[below] at (1,-0.05) {$L$};
    \node[right] at (2.05,0.125) {$A$};
    \end{tikzpicture}
\end{center}

Pictured above is our scenario; we have a single molecule of gas of mass $m$ and speed $v$ moving back and forth in a tube of length $L$, as it bounces off the ends of the tube that have area $A$. We assume that the molecule collides elastically off the walls, and hence it goes back and forth with velocities $\vec{v}$ and $-\vec{v}$. From our knowledge of kinematics, we obtain that the time it takes for the particle to go back and forth once is:
\begin{align*}
    \Delta t = \frac{2L}{v}
\end{align*}
Now, we want to figure out the average pressure that this particle applies on the wall; to obtain this, we, we return to how we have defined pressure above:
\begin{align*}
    P_{av} = \frac{F_{av}}{A}
\end{align*}
$A$ is just a constant here, so we then just need to figure out $F_{av}$! For this, we think back to the kinematics unit and the definition of force as the time derivative of momentum\footnote{Of course you know by know that force and momentum are vector and not scalar quantities, but since our problem is one dimensional, we can just consider the scalar version of the equation here!}:
\begin{align*}
    F = \frac{dp}{dt} = \frac{d(mv)}{dt}
\end{align*}
Since we are just concerned with the average force, we can just consider the change in momentum $\Delta p$ of the particle from hitting one of the ends over one cycle over the time it takes for one cycle (which is just $\Delta t$, above!). We consider that when the molecule hits the end of the tube with speed $v$, it reflects back elastically and starts going in the other direction with the same speed $v$, so its change in velocity is therefore $-2\Delta v$, and therefore the change in momentum is:
\[ \Delta p_{molecule} = -2m\Delta v\]
And therefore the average force applied to the wall is:
\[ F_{av} = \frac{-\Delta p_{molecule}}{\Delta t} = \frac{2mv}{\frac{2L}{v}} = \frac{mv^2}{L} \]
Then, getting the average pressure (from the one molecule) is simple, using the definition of pressure:
\[P_{av} = \frac{\frac{mv^2}{L}}{A} = \frac{mv^2}{V} \]
Where I have used the fact that the volume of the tube is given by $V = AL$. Now, in a real gas, we obviously don't just have one molecule, but a bunch of different molecules, all with different speeds! If we want the total pressure from a whole bunch of (say, $N$) molecules, we can just add up the average pressure of $N$ molecules:
\[P_{total} = \sum_{i=1}^N \frac{mv_i^2}{V}\]
As a reminder, the greek sigma $\Sigma$ above is telling us to take the "sum of N" things (in this case, molecules). It's clear that the volume can be taken out of the summation (because the volume of the tube is the same no matter which molecule we look at), and if we further assume that all of the masses of the molecules are the same (which is perfectly reasonable if we have a gas made of one element), then we obtain:
\[P_{total} = \frac{m}{V} \sum_{i=1}^N v_i^2 \]
Now I'm going to do a bit of a hack; I'm going to multiply this expression by $1 = \frac{N}{N}$:
\[P_{total} = N\frac{m}{V} \left(\frac{1}{N}\sum_{i=1}^N v_i^2 \right)\]
You might recognize from your physics labs that the quantity in brackets is just the \textit{average} of the squared velocities of all the molecules\footnote{To be clear, this is not the square of the averages!}; let us denote this by $v^2_{av}$. Now, let us multiply both sides by the volume $V$ of the box:
\[PV = Nmv^2_{av} \]
Now, if we multiply both sides by a half, we get something that looks familiar on the right hand side:
\[\frac{1}{2}PV = N\frac{1}{2}mv^2_{av} \]
You can see that we have exactly the average kinetic energy of our molecules (times the number of molecules in our system)! Now, at the beginning I said we were considering a one-dimensional system; but the direction we could have been talking about could have been the x direction, the y direction, or the z direction, and we would have got the same result! Therefore, we have that:
\[N\frac{1}{2}mv^2_{xav} = N\frac{1}{2}mv^2_{yav} = N\frac{1}{2}mv^2_{zav} = \frac{1}{2}PV \]
Therefore, for an ideal gas in three dimensions, all we have to do is add the three terms (the three kinetic energy contributions from each of the directions) together:
\begin{equation}
    \label{eqn:(2)}
    \frac{3}{2}PV = N\epsilon_{kavg}
\end{equation}
Where $\epsilon_{kavg}$ is the average kinetic energy of the molecules in three dimensions, defined as:
\begin{equation}
    \epsilon_{kavg} = \frac{1}{2}mv^2_{xav} + \frac{1}{2}mv^2_{yav} + \frac{1}{2}mv^2_{zav} = \frac{1}{2}m\vec{v}_{av} \cdot \vec{v}_{av}
\end{equation}
Now at this point, you might be holding your head in your hands because I spent three pages on a derivation\footnote{I sincerely apologize for your suffering.} and it seems like we're no closer to defining temperature than when we started. However, in the next section we will introduce the ideal gas law, and combining that with equation \ref{eqn:(2)} we just derived, we will see that we obtain exactly what we want to find.


\section{\module{cmath} ---
         Mathematical functions for complex numbers}

\declaremodule{builtin}{cmath}
\modulesynopsis{Mathematical functions for complex numbers.}

This module is always available.  It provides access to mathematical
functions for complex numbers.  The functions are:

\begin{funcdesc}{acos}{x}
Return the arc cosine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{acosh}{x}
Return the hyperbolic arc cosine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{asin}{x}
Return the arc sine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{asinh}{x}
Return the hyperbolic arc sine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{atan}{x}
Return the arc tangent of \var{x}.
\end{funcdesc}

\begin{funcdesc}{atanh}{x}
Return the hyperbolic arc tangent of \var{x}.
\end{funcdesc}

\begin{funcdesc}{cos}{x}
Return the cosine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{cosh}{x}
Return the hyperbolic cosine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{exp}{x}
Return the exponential value \code{e**\var{x}}.
\end{funcdesc}

\begin{funcdesc}{log}{x}
Return the natural logarithm of \var{x}.
\end{funcdesc}

\begin{funcdesc}{log10}{x}
Return the base-10 logarithm of \var{x}.
\end{funcdesc}

\begin{funcdesc}{sin}{x}
Return the sine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{sinh}{x}
Return the hyperbolic sine of \var{x}.
\end{funcdesc}

\begin{funcdesc}{sqrt}{x}
Return the square root of \var{x}.
\end{funcdesc}

\begin{funcdesc}{tan}{x}
Return the tangent of \var{x}.
\end{funcdesc}

\begin{funcdesc}{tanh}{x}
Return the hyperbolic tangent of \var{x}.
\end{funcdesc}

The module also defines two mathematical constants:

\begin{datadesc}{pi}
The mathematical constant \emph{pi}, as a real.
\end{datadesc}

\begin{datadesc}{e}
The mathematical constant \emph{e}, as a real.
\end{datadesc}

Note that the selection of functions is similar, but not identical, to
that in module \refmodule{math}\refbimodindex{math}.  The reason for having
two modules is that some users aren't interested in complex numbers,
and perhaps don't even know what they are.  They would rather have
\code{math.sqrt(-1)} raise an exception than return a complex number.
Also note that the functions defined in \module{cmath} always return a
complex number, even if the answer can be expressed as a real number
(in which case the complex number has an imaginary part of zero).


\documentclass[12pt]{article}

\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{float}
\usepackage{ulem}
\usepackage{bm}
\usepackage{anysize}
\usepackage{pythonhighlight}

\marginsize{2cm}{2cm}{0.9cm}{1.8cm}

\title{EECE 5639 Computer Vision\\ [2ex] \begin{large} Homework \#4 \end{large} }
\author{Jiyu Tian}
\date{}

\begin{document}
\maketitle
\pagestyle{empty}
%%---------------------------------------------------------------
%% Question 1
%%---------------------------------------------------------------
\section{Solution:}
(a) Since $Z=0$, it is a similarity transformation:
\begin{equation*}
\left[ \begin{array}{c}
u\\
v\\
1
\end{array} \right] \sim \left[ \begin{array}{ccc}
fcos\theta & -fsin\theta & ft_x\\
fsin\theta & fcos\theta & ft_y\\
0 & 0 & t_z
\end{array} \right]\left[ \begin{array}{c}
X\\
Y\\
1
\end{array} \right]
\end{equation*}
(b) It has 4 degree of freedom.\\
(c) 4 points are needed since it is a 2D to 2D transformation.\\
(d) Yes, more correspondences will be helpful for a more robust solution:
\begin{itemize}
    \item 4 correspondences may be collinear
    \item points may be inaccurate because of noise
\end{itemize}
(e) Cross ratio of collinear points.\\
(f) Parallellism.\\
(g) The building plane is parallel to the image plane if and only if:
\begin{itemize}
    \item lines of building parallel to $X$ axes are also parallel to each other on the image
    \item lines of building parallel to $Y$ axes are also parallel to each other on the image
\end{itemize}

%%---------------------------------------------------------------
%% Question 2
%%---------------------------------------------------------------
\section{Solution:}
(a) SSD
\begin{equation*}
SSD = \left[ \begin{array}{cccccccc}
\times & \times& \times& \times& \times &\times &\times & \times \\
\times &  24 & 28 & 24 & 12 & 8 & 8 & \times \\
\times &  16 & 12 & 16 & 8 & 12 & 8 & \times \\
\times &  8 & 8 & 12 & 8 & 16 & 8 & \times \\
\times &  10 & 7 & 9 & 12 & 28 & 20 & \times \\
\times &  9 & 12 & 9 & 12 & 24 & 20 & \times \\
\times &  0 & 7 & 7 & 8 & 12 & 8 & \times \\
\times & \times& \times& \times& \times &\times &\times & \times
\end{array} \right]
\end{equation*}
(b) Cross-Correlation
\begin{equation*}
C_fg = \left[ \begin{array}{cccccccc}
\times & \times& \times& \times& \times &\times &\times & \times \\
\times &  4 & 4 & 2 & 0 & 0 & 0 & \times \\
\times &  10 & 14 & 8 & 2 & 0 & 2 & \times \\
\times &  4 & 4 & 0 & 0 & 0 & 4 & \times \\
\times &  4 & 5 & 2 & 0 & 4 & 10 & \times \\
\times &  3 & 1 & 0 & 0 & 4 & 8 & \times \\
\times &  8 & 4 & 1 & 2 & 8 & 12 & \times \\
\times & \times& \times& \times& \times &\times &\times & \times
\end{array} \right]
\end{equation*}
(c) Normalized Cross-Correlation
\begin{equation*}
N_fg = \left[ \begin{array}{cccccccc}
\times & \times& \times& \times& \times &\times &\times & \times \\
\times &  0.0208 & 0.0179 & 0.0125 & 0 & 0 & 0 & \times \\
\times &  0.0446&  0.0547 & 0.0417 & 0.0625&  0 & 0.0625& \times \\
\times &  0.0625  &0.0625 & 0 & 0 & 0 & 0.0625 &\times \\
\times &  0.05 & 0.0694 & 0.05 & 0 & 0.0179 & 0.0391& \times \\
\times &  0.0536 & 0.0208 & 0 & 0 & 0.0208 & 0.0357& \times \\
\times &  0.125 & 0.0714 & 0.125 & 0.0625&  0.05 & 0.0625 &\times \\
\times & \times& \times& \times& \times &\times &\times & \times
\end{array} \right]
\end{equation*}

\end{document}


% !TEX root = index.tex

\section{The Cocomplex World of Cochain Complexes}
\epigraph{The introduction of the digit 0 or the group concept was general nonsense too, and mathematics was more or less stagnating for thousands of years because nobody was around to take such childish steps.}{Grothendieck}
\begin{align*}
 \xymatrix@R-2pc{
 \U \ar@{~>}[r] & \L^\bullet(\U)\\
 \mbox{Good cover of } X & \mbox{Cech Complex of } \U }
\end{align*}
To be able to use algebraic techniques, we need a way to convert topological information into algebraic information. We'll do this using locally constant functions and of course, linear algebra. All our vector spaces will be over the base field $ \F = \{0, 1\}$ i.e. all the scalars are either 0 or 1. This is mainly because $-1 = 1$ in $ \F$
and hence we do not have to worry about signs.\\

\noindent \textbf{Notation:} $V = \F \langle v_1, \dots, v_n \rangle = \F \langle \mathcal{B} \rangle$ stands for ``$V$ is a vector space over $\F$ with basis $ \mathcal{B} = \{ v_1, \dots, v_n\}$''.
  \begin{ques}
      Show that the elements of $ \F \langle \mathcal{B} \rangle$ can be identified with subsets of $\mathcal{B}$ and hence as a set $V$ has size $2^{\lvert\mathcal{B}\rvert}$, where $\lvert \mathcal{B} \rvert$ denotes the size of $\mathcal{B}$.
  \end{ques}

\subsection{Locally Constant Functions}
\begin{definition}
	For a topological space $X$, define the vector space of \textbf{locally constant functions}, denoted $\L(X)$, to be the space of continuous maps from $X$ to $\F$.
	\begin{align*}
		\L(X) := \{ \: f: X \rightarrow \F \: \mbox{ continuous } \}
	\end{align*}
  Here we are thinking of $\F$ as a topological space with 2 points.
\end{definition}
\begin{ques}
  Show that $\L(X)$ is naturally a vector space over $\F$.
\end{ques}
\begin{ques}
  Show that if $X$ is connected then every continuous function $f: X \rightarrow \F$ is a constant function and hence $\L(X) \cong \F$ as a vector space.
\end{ques}
\begin{ques}
	What is $\L(\phi)$ where $\phi$ is the empty set (which is a legit topological space)?
\end{ques}
\noindent Let $X^1, X^2, \dots, X^k$ be the connected components of $X$. Define $k$ functions $\delta^1, \delta^2, \dots, \delta^k : X \rightarrow \F$ as
\begin{align*}
  \delta^i(x) = \begin{cases}
    1 & \mbox{ if $x \in X^i$} \\
    0 & \mbox{ otherwise}
\end{cases}
\end{align*}
for $1 \le i \le k$.\\
\begin{ques}
	Show that
	\begin{align*}
		\L(X) = \F \langle \delta^1, \delta^2, \dots, \delta^k \rangle
	\end{align*}
	and hence $\dim \L(X) = k = \pi_0(X)$.
\end{ques}
\begin{ques}*
  Show that the above statement is false if $X$ has infinitely many connected components, for example, if $X$ is the set of integers.
\end{ques}
\begin{definition}
	With the notation as above, we call $\delta^1, \delta^2, \dots, \delta^k$ the \textbf{canonical basis} for $\L(X)$.
\end{definition}

The following exercise is extremely important, make sure you understand it well.
\begin{ques} \label{q:restrictions}
  For an inclusion of topological spaces $X \subseteq Y$,
	\begin{enumerate}
		\item We can restrict a function $f: Y \rightarrow \F$ to the subspace $X$ and get a function $f|_{X} : X \rightarrow \F$. Show that this induces a linear transformation
    \begin{align*}
			\res_{Y \rightarrow X}: \L(Y) \rightarrow \L(X)
		\end{align*}
    \item Explicitly compute the matrix for $\res_{Y \rightarrow X}$ in the canonical bases when $Y = \R$ and $X = \{ -1, 1\}$.
    \item Explicitly compute the matrix for $\res_{Y \rightarrow X}$ in the canonical bases when $Y = \R^2 $ minus the $y$-axis and $X = \{ (-1,0), (1,0), (2,0)\}$.
    \item* More generally, show that if $X^i$ and $Y^j$ are the connected components of $X$ and $Y$ respectively, for $1 \le i \le k$ and $1 \le j \le l$, then in the canonical basis $\res_{Y \rightarrow X}$ is a $k \times l$ matrix whose entry in the $i^{th}$ row and $j^{th}$ column is given by
    \begin{align*}
      [\res_{Y \rightarrow X}]_{i,j} &= \begin{cases}
        1 & \mbox{ if } X^i \subseteq Y^j \\
        0 & \mbox{ if } X^i \not \subseteq Y^j
    \end{cases}
    \end{align*}
		% \item Show that $\res_{U \rightarrow U}$ is the identity transformation.
		% \item Show that $\res_{V \rightarrow U} \circ \res_{W \rightarrow V} = \res_{W \rightarrow U}$.
    \end{enumerate}
\end{ques}

\newpage
\subsection{Cochain Complexes}
The various vector spaces $\L(U_I)$ naturally assemble themselves into a cochain complex, we'll get to that in the next section. First we need to understand what cochain complexes are.
\begin{definition}
  A \textbf{cochain complex} $ \V^\bullet$ consists of the following data:
  \begin{align*}
    &\xymatrix{0 \ar[r] & \V^0 \ar[r] & \cdots \ar[r] & \V^{i-1} \ar^-{d^{i-1}}[r] & \V^{i} \ar^{d^{i}}[r]& \V^{i+1} \ar[r] & \cdots \ar[r] & \V^n \ar[r] &0}
  \end{align*}
  \begin{enumerate}
    \item A vector space $ \V^i$ for each $ i \in \Z$, with $ \V^i \neq 0$  only if $ 0 \le i \le n$ for some positive integer $ n$.
    \item For each $ i \in \Z$ a linear transformation $ d^i: \V^i \rightarrow \V^{i+1}$ that satisfies
    \begin{align*}
      d^{i} \circ d^{i-1} = 0
      &&
      \xymatrix{
      \V^{i-1} \ar@/_1pc/_{d^{i} \circ d^{i-1} = 0}[rrrr] \ar^{d^{i-1}}[rr] && \V^i \ar^{d^{i}}[rr] && \V^{i+1}
      }
    \end{align*}
  \end{enumerate}
\end{definition}

\begin{ques}
  Show that $ \im d^{i-1} \subseteq \ker d^{i}$.
\end{ques}

\begin{definition}
  The $ i^{th}$ \textbf{cohomology} of $ \V^\bullet$ is the quotient vector space
    \begin{align*}
      H^i(\V) := \ker d^{i} / \im d^{i-1}
    \end{align*}
\end{definition}
\begin{remark}
  This is well-defined because of the previous exercise. We'll only be interested in the dimensions
  \begin{align*}
    \dim H^i(\V) = \dim \ker d^{i} - \dim \im d^{i-1}
  \end{align*}
\end{remark}
\noindent \textbf{Convention:} Even though in a cochain complex there is a vector space $ \V^i$ for all integers $ i$ it is a common convention to explicitly define $ \V^i$ only where it is non-zero, it is understood that the rest of the $ \V^i$ are all 0. The first non-zero vector space in the cochain complex is understood to be $ \V^0$ unless otherwise specified.
\begin{ques}
  A vector space $A$ can be thought of as a cochain complex as
  \begin{align*}
    \xymatrix{ \V^\bullet= 0 \ar[r] & A \ar[r] & 0}
  \end{align*}
  What are the cohomologies $ H^i(\V)$, for $ i \in \Z$?
\end{ques}

\begin{ques}
  A linear transformation $ f:A \rightarrow B$ can be thought of as a cochain complex as
  \begin{align*}
    \xymatrix{ \V^\bullet= 0 \ar[r] & A \ar^f[r] & B \ar[r] & 0}
  \end{align*}
  What are the cohomologies $ H^i(\V)$, for $ i \in \Z$?
\end{ques}
For the following exercises it'll be useful to invoke the {\bf Rank-Nullity / Dimension Theorem}.
\begin{ques} Verify that the following are cochain complexes and compute their cohomologies.
  \begin{enumerate}
    \item $ \xymatrix{0 \ar[r] & \F \ar^{0}[r] & \F \ar^{0}[r] & \F \ar[r] & 0 }  $
    \item $ \xymatrix{0 \ar[r] & \F \ar^{1}[r] & \F \ar^{0}[r] & \F \ar[r] & 0 }  $
    \item $ \xymatrix{0 \ar[r] & \F \ar^{0}[r] & \F \ar^{1}[r] & \F \ar[r] & 0 }  $
    \item $ \xymatrix{0 \ar[r] & \F \ar^{\begin{bmatrix}1 \\ 1 \end{bmatrix}}[rr]  && \F^2  \ar^{\begin{bmatrix}1 & 1 \end{bmatrix}}[rr] && \F \ar[r] & 0 }  $
  \end{enumerate}
\end{ques}

\begin{ques} Compute the cohomologies of the following cochain complexes.
  \begin{enumerate}
    \item $ \xymatrix{0 \ar[r] & \F^2 \ar^{\begin{bmatrix}1 & 1 \\ 1 & 1 \\ 1 & 1 \end{bmatrix}}[rr] && \F^3 \ar[r] & 0 } \qquad $ (this is computing $\check H^* (S^1 \vee S^1)$).
    \item $ \xymatrix{0 \ar[r] & \F^3 \ar^{\begin{bmatrix}1 & 1 & 0\\ 1 & 0 & 1 \\ 0 & 1 & 1 \end{bmatrix}}[rr] && \F^3 \ar^{\begin{bmatrix}1 & 1 & 1\\ 1 & 1 & 1 \end{bmatrix}}[rr] && \F^2 \ar[r] & 0 } \qquad $ (this is computing $\check H^* (S^2)$).
  \end{enumerate}
\end{ques}

\begin{ques}
  Given $\xymatrix{ \V^\bullet= 0 \ar[r] & A \ar^f[r] & B \ar^g[r] &  C \ar[r] & 0}$
  \begin{enumerate}
    \item Under what conditions on $ f,g$ is $ \V^\bullet$ a cochain complex.
    \item Under what conditions on $ f,g$ is $ H^i(\V) = 0$ for all $ i$. In this case, we say that $ \V^\bullet$ is a \textbf{short exact sequence}.
  \end{enumerate}
\end{ques}

\begin{definition}
  More generally, a cochain complex $ \V^\bullet$ is said to be \textbf{exact} (or \textbf{long exact}) if $H^i(\V) = 0$ for all $ i$.
\end{definition}
\newpage

\subsection*{Optional Problems}
\begin{definition}
  Define the \textbf{Euler characteristic} of a cochain complex to be
  \begin{align*}
    \chi(\V)
    &= \sum_{i \in \Z} (-1)^i \dim H^i(\V) \\
    &= \dim H^0(\V) - \dim H^1(\V) +  \dim H^2(\V) \pm \cdots + (-1)^n \dim  H^n(\V)
  \end{align*}
\end{definition}
% \begin{ques} $ $
%   \begin{enumerate}
%     \item Show that if $ \V^\bullet$ is exact then $ \chi(\V) = 0$.
%     \item Find an example of a cochain complex $ \V^\bullet$ which is not exact but for which $ \chi(\V) = 0$.
%   \end{enumerate}
% \end{ques}

\begin{ques}$ $
  \begin{enumerate}
    \item Express the Euler characteristic in terms of the $ \dim \ker(d^i)$ and $ \dim \im(d^i)$ for $i \in \Z$.
    \item Show that
      \begin{align*}
        \chi(V) = \sum_{i \in \Z} (-1)^i \dim \V^i
      \end{align*}
  \end{enumerate}
  \end{ques}
  \noindent Thus the Euler characteristic can be computed using the dimensions of the vector spaces of the original cochain complex, but it is really an invariant of the underlying cohomology!!! \\\\
\begin{ques}*
  The direct sum of cochain complexes $(\V_1 \oplus \V_2)^\bullet$ is defined as
  \begin{align*}
    (\V_1 \oplus \V_2)^i := \V_1^i \oplus \V_2^i
  \end{align*}
  and the differential is defined as
  \begin{align*}
    d^i := d_1^i \oplus d_2^i
  \end{align*}
  % \begin{align*}
  %   &\xymatrix{ \cdots \ar[r] & \V_1^{i-1} \oplus \V_2^{i-1} \ar^-{d_1^{i-1} \oplus d_2^{i-1}}[rr] && \V_1^{i} \oplus \V_2^{i} \ar^{d_1^{i} \oplus d_2^{i}}[rr]&& \V_1^{i+1} \oplus \V_2^{i+1} \ar[r] & \cdots}
  % \end{align*}
  Find $H^*(\V_1 \oplus \V_2)$ in terms of $H^*(\V_1)$ and $H^*(\V_2)$.
\end{ques}

\begin{ques}***
  The tensor product $(\V_1 \otimes \V_2)^\bullet$ of two cochain complexes $\V_1^\bullet$, $\V_2^\bullet$ is defined as
  \begin{align*}
    (\V_1 \otimes \V_2)^k := \bigoplus \limits_{i + j = k} \V_1^i \otimes \V_2^j
  \end{align*}
  and the differential is defined as
  \begin{align*}
    d^k := \bigoplus \limits_{i + j = k} d_1^i \otimes d_2^j
  \end{align*}
  Find $H^*(\V_1 \otimes \V_2)$ in terms of $H^*(\V_1)$ and $H^*(\V_2)$.
\end{ques}
\newpage
\begin{ques}***
  A \textbf{morphism of cochain complexes} $\phi: \V_1^\bullet \rightarrow \V_2^\bullet$ is a collection of maps $ \phi: \V_1^i \rightarrow \V_2^i$ for each $ i \in \Z$ such that the following diagram commutes
  \begin{align*}
    \xymatrix{
    \V_2^i \ar^-{d}[r]  & \V_2^{i+1}  \\
    \V_1^i \ar^-{d}[r] \ar_{\phi}[u] & \V_1^{i+1} \ar^{\phi}[u]
    }
  \end{align*}
  \begin{enumerate}
    \item Show that a morphism $\phi: \V_1^\bullet \rightarrow \V_2^\bullet$ between cochain complexes naturally induces a map between cohomologies $\phi^*: H^i(\V_1) \rightarrow H^i(\V_2)$, for all $ i \in Z$.
    \item Given two morphisms $\phi_1: \V_1^\bullet \rightarrow \V_2^\bullet$ and $\phi_2: \V_2^\bullet \rightarrow \V_3^\bullet$, show that their composition $\phi_2 \circ \phi_1: \V_1^\bullet \rightarrow \V_3^\bullet$ is also a morphism of cochain complexes. Further show that $ (\phi_2 \circ \phi_1)^* = \phi_2^* \circ \phi_1^*$.
  \end{enumerate}
\end{ques}

\begin{ques}**
  A cochain complex of cochain complexes (i.e. each $\V^i$ is itself a cochain complex and the differentials $d^i$ are morphisms of cochain complexes) is called a \textbf{double complex}. Unravel this description of a double complex and describe it more explicitly as a grid of vector spaces.
\end{ques}

\iffalse
\begin{ques}[Snake Lemma]**
  Show that a morphism $ \phi$ between short exact sequences
    \begin{align}
      \label{eq:double_complex}
      \xymatrix{
         0 \ar[r] & \V_1^0 \ar^{\phi^0}[d] \ar[r] & \V_1^1 \ar^{\phi^1}[d] \ar[r] & \ar^{\phi^2}[d] \V_1^2 \ar[r] & 0 \\
         0 \ar[r] & \V_2^0 \ar[r] & \V_2^1 \ar[r] &  \V_2^2 \ar[r] & 0
      }
    \end{align}
    induces a natural exact sequence (not a full cochain complex)
    \begin{align}
      \label{eq:snake_lemma}
      \xymatrix{\ker \phi^0 \ar[r] & \ker \phi^1 \ar[r] & \ker \phi^2 \ar[r] & \coker \phi^0 \ar[r] &\coker \phi^1 \ar[r] &\coker \phi^2 }
    \end{align}
    (Recall the cokernel of a map $ f:A \rightarrow B$ is $ B/(\im f)$.) This sequence does not naturally extend to a full cochain complex.
\end{ques}
\begin{remark}
  Note that \eqref{eq:double_complex} is a double complex which gives rise to the exact sequence \eqref{eq:snake_lemma}. Larger double complexes give rise to more complicated structures called \textbf{spectral sequences}.
\end{remark}
\fi


\documentclass{article}

\usepackage{amsmath}
\usepackage{numprint}

\author{Daniel Fernandes Martins (danielfmt)}
\title{Question \#9 Solution}

\begin{document}

\maketitle

\textbf{Disclaimer.} This is the reasoning I used to solve the problem; it
may be wrong though. This is intended just as food for thought.

\section{Parameters Of Neural Network}

Given a neural network with 10 input units (including the constant $x_0^{(0)}$
unit), one output unit, and 36 hidden units (including the necessary number of
constant units for a fully-connected network), this question asks what is the
minimum possible number of weights that such a network can have.

\subsection{Finding The Minimum}

The network must be fully connected, and each hidden layer must include a
constant (bias) unit. Since the constant units don't get any input from the
previous layer, one way to minimize the number of required weights is by
maximizing the number of constant units in the network.

This can be done by creating a network with 18 hidden layers, with 2 units each.
The number of weights in this network is given by:

\begin{equation*}
(1 \cdot 10) + (1 \cdot 2 \cdot 17) + (2) = 46
\end{equation*}

\end{document}


% conclusions and future work
\section{Discussions and Conclusions}
\label{sec:conclusions}

We presented an effective approach to mine approximate frequent subgraph
patterns from a single large graph database in the presence of
a label cost matrix.

There are two main parameters in our method: $K$, the number of random
walks, and $\alpha$ the cost threshold. The value of $K$ is directly
proportional to the number of maximal approximate patterns we desire,
and is relatively easy to set.
On the other hand, choosing an appropriate value of
$\alpha$ is very important as it affects the quality of
patterns mined. Depending on the application domain
and the purpose of the graph mining, let $t$ be the number of
vertices in the pattern for which we allow label mismatches
in the subgraph isomorphism. One reasonable value of
$\alpha$ is $t \times IMQ$ where $IMQ$ is the inter-quartile mean
i.e., the mean of the entries between the first quartile ( $25^{th}$
percentile) and the third quartile ( $75^{th}$ percentile)
of the entries in the cost matrix arranged in sorted order.
$t$ can be chosen by first
enumerating maximal patterns with $\alpha = 0$ and computing the
average size $m$ of the maximal patterns mined from the graph.
The value of $t$ then is a fraction of the average size $m$.
Care has to be taken not to choose a very large $\alpha$ as it leads to
patterns of poor quality and also increases the run time of the
algorithm significantly as can be seen in Table \ref{tab:scop_alpha}.

In terms of future work, we plan to increase the efficiency of our
method by exploiting parallelism. Obviously different walks can be
carried out in parallel. However, more interesting is the
parallelization of the approximate isomorphism generation and
label-based pruning steps, including verification. We also
want to explore the idea of label based pruning for more
general definitions of approximate isomorphism including
edge mismatches.


\onecolumn
\chapter{Measurement Models}

\begin{equation}
    \mathcal{R}_1^{(j)} = \rho^{(m,j)}_1(t_i) +c\left(b_r^{(m)}(t_i) - b_t^{(j)}(t_t)\right) + \Delta \rho^{(j)}_{iono}(t_i) + \Delta \rho^{(j)}_{tropo}(t_i) + \sigma_{SA}^{(j)} + \sigma_n^{(j)} + b_m
\end{equation}

\begin{tabbing}
12345678912345 \= Reynolds number based on length $s$ \kill
$\mathcal{R}_1^{(j)}$         \>  One-way pseudorange measurement from transmitting \\
$$                            \> antenna $k$ on satellite $j$ to receiving antenna $m$ on satellite $n$. \\
$\rho^{(m,j)}_1(t_i)$         \>  Geometric distance between transmitting and receiveing antenntas.\\
$b_r^{(m)}(t_i)$               \> Clock bias for receiver \\
$b_t^{(j)}(t_t)$                   \>  Clock bias for transmitter\\
$\Delta \rho^{(j)}_{iono}(t_i)$    \> Correction for ionspheric delay \\
$ \Delta \rho^{(j)}_{tropo}(t_i) $ \> Correction for  Tropospheric delay\\
$\sigma_{SA}^{(j)}$                \> Error due to selective availability \\
$\sigma_n^{(j)}$                   \> Measurement noise \\
$b_m$                              \> Measurement bias \\
$t_t$                              \> Trasmission time \\
$t_r$                              \> Receive time
\end{tabbing}

The transmission time $t_t$ and the geometric range are determined
iteratively by using fixed point iteration on the following equation
%
\begin{equation}
   t_t = t_i - \frac{\rho^{(m,j)}_1(t_i) + \Delta \rho^{(j)}_{iono}(t_i) + \Delta \rho^{(j)}_{tropo}(t_i)}{c}
\end{equation}

\section{Range}

GMAT supports two primary models for spacecraft range: geometric and
radiometric. In both cases the range is a measure of the distance
between an observer and a vehicle.
 The geometric range is calculated using vector geometry and ignores signal propagation
  times and error sources.  Most, if not all, ground trackers provide the user with
  the round trip signal propagation time.  The radiometric range model uses the best
   estimate spacecraft state to determine an expected value for the round trip signal
   propagation time from observer to spacecraft and back to the observer.  Hence,
   the raw radiometric value is a measure of round trip range.

The general form of the measurement model used by GMAT is
%
\begin{equation}
   \mathcal{O}_c = \mathbf{f}_k\left(\mathbf{r}_v(t + \delta t), \dot{\mathbf{r}}_v(t + \delta t),
   \mathbf{r}_o, \dot{\mathbf{r}}_o\right) + b + \delta_a + \delta_r
\end{equation}
%
%
\begin{tabbing}[htbp!]
12345678 \= dummy line \kill
$\mathbf{f}_k$ \> The kinematic model specific to a measurement type\\
$\mathbf{r}_v$ \> Vehicle position\\
$\dot{\mathbf{r}}_v$ \> Vehicle velocity\\
$\mathbf{r}_o$ \> Observer position\\
$\dot{\mathbf{r}}_o$    \> Observer velocity\\
$t$    \> Measurement time tag\\
$b$    \> Measurement bias\\
$\delta a$    \> Atmospheric correction\\
$\delta r$     \> Relativistic correction\\
\end{tabbing}

The kinematic model for geometric range is simply
%
\begin{equation}
    \rho_c = \| \mathbf{r}_v(t) - \mathbf{r}_g(t)\|
\end{equation}
%

The kinematic model for radiometric two way range is
%
\begin{equation}
     \rho_c= \frac{1}{2}\left(\| \mathbf{r}_v(t_{v}) -  \mathbf{r}_g(t_{gt})  \| +
      \| \mathbf{r}_v(t_{v}) -  \mathbf{r}_g(t_{gt})  \|\right) \label{Eq:ExpectedTwoWayRange}
\end{equation}
%
where
%
\begin{tabbing}[htbp!]
12345678 \= dummy line \kill
$t_{gt}$ \> Time the uplink signal is transmitted from ground station\\
$t_{vr}$ \> Time the uplink signal is received at vehicle\\
$t_{vt}$ \> Time the downlink signal is transmitted from vehicle\\
$t_{gr}$ \> Time the downlink signal is received at ground station\\
$t_u$    \> Uplink propagation time, ( $t_{vr}$ - $t_{gt}$ )\\
$t_d$    \> Downlink propagation time, ( $t_{vt}$ - $t_{gr}$ )\\
$\rho_u$    \> Uplink distance\\
$\rho_d$    \> Downlink distance\\
$d_T$     \> Distance traveled by vehicle during transponder delay\\
$\mathbf{r}_v(t)$ \> Position of vehicle at time $t$\\
$\mathbf{r}_t(t)$ \> Position of transmitter at time $t$\\
$\mathbf{r}_r(t)$ \> Position of receiver at time $t$\\
$t$           \>  time of geometric range measurement \\
$\delta T$  \>  Vehicle's transponder delay time\\
$\delta t_a$  \>  Atmospheric delays\\
$\delta t_r$  \>  Relativistic effects\\
\end{tabbing}
%

The radiometric model is derived from the measurement geometry shown
in Fig.~\ref{Fig:RangeMeasurement}. We see that the total signal
propagation time is the sum of three times, the uplink
 signal propagation time, $t_u$, the transponder delay time, $\delta T$, and
 the downlink propagation time, $t_d$.  Hence, the observed value for signal propagation time, $t_o$, is
%
\begin{equation}
     \Delta t_o = t_{gr} - t_{gt} = t_u + \delta T + t_d
\end{equation}
%
\begin{figure}[htbp!]
    \begin{center}
    \begin{picture}(270,215)
    \special{psfile= RangeMeasurement.eps
    hscale= 75 vscale= 75 hoffset = -85 voffset = -270}
        \makebox(175,295){ $\mathbf{r}_v(t_{vr})$}
        \makebox(-25,290){ $\mathbf{r}_v(t_{vt})$}
        \makebox(85,290){ $\rho_d$}
        \makebox(-100,90){ $\mathbf{r}_g(t_{gr})$}
        \makebox(-460,90){ $\mathbf{r}_g(t_{gt})$}
        \makebox(-485,290){ $\rho_u$}
        \makebox(-305,390){ $d_T$}
    \end{picture}
    \end{center}
    \vspace{0.2 in}
    \label{Fig:RangeMeasurement}
    \caption{ Geometry of Radiometric Range Measurement}
\end{figure}
%
The elapsed time is converted to a measure of the average range
using
%
\begin{equation}
     \rho_o = \frac{c}{2}\Delta t_o
\end{equation}
%
The computed elapsed time is rigorously expressed as
%
\begin{equation}
    \Delta t_c = \frac{1}{c}\| \mathbf{r}_v(t_{vr}) -  \mathbf{r}_g(t_{gt})  \| +
    \frac{1}{c}\| \mathbf{r}_v(t_{vt}) -  \mathbf{r}_g(t_{gr})  \| + \delta T
\end{equation}
%
Assuming that the transponder delay is modelled as a measurement
bias we can write $ t_{vr} = t_{vt} = t_v$ and convert the computed
round trip time to average range:
%
\begin{equation}
     \rho_c= \frac{1}{2}\left(\| \mathbf{r}_v(t_v) -  \mathbf{r}_g(t_{gt})  \| +
      \| \mathbf{r}_v(t_v) -  \mathbf{r}_g(t_{gr})  \|\right) \label{Eq:MeasuredTwoWayRange}
\end{equation}
%
To solve Eq.~\ref{Eq:MeasuredTwoWayRange}, we must know  $t_{v}$.
For applications
 that do not require high accuracy we can approximate $t_v$ as we describe below.
  For higher fidelity applications we must solve for the uplink and downlink
  propagation times using two iterative processes.  The downlink signal propagation
  time is calculated using the following fixed point iteration on the following equation:
%
\begin{equation}
     \delta t_d^{i+1} = \frac{1}{c}\| \mathbf{r}_v( t - t_d^{i}) - \mathbf{r}_g(t)   \|
\end{equation}
%
The uplink propagation time is calculated using fixed point
iteration on
%
\begin{equation}
     \delta t_u^{i+1} = \frac{1}{c}\| \mathbf{r}_v( t - t_d) - \mathbf{r}_g(t - t_d - t_u^{i} )   \|
\end{equation}
%

\begin{eqnarray}
     \frac{\partial \rho_c (t)}{\partial \mathbf{r}_v(t_v)} &=& \frac{1}{2\rho_u\rho_d}
     \left( \rho_d(\mathbf{r}_v^T(t_v) - \mathbf{r}_T^T(t_{gt}) + \rho_u(\mathbf{r}_v^T(t_v) - \mathbf{r}_R^T(t_{gr}  ) \right)\\
     %
     \frac{\partial \rho_c (t)}{\partial \dot{\mathbf{r}}_v(t_v)} &=& \mathbf{0}_{1x3}
\end{eqnarray}


\chapter{Beginning Combinatorics}

Discrete probabilities problems often include some counting. For
example, we figured out that there were 36 different ways the two dice
could come up, but all of them summed to some number 2 through 12. How
many different ways could 3 8-sided dice come up? We would need to
count them, right? As the numbers get big we will need some tricks so
we don't need to write them all down and count them one-by-one.

The branch of mathematics that focuses on tricks for counting is
called \textit{combinatorics}.\index{combinatorics}

How can be sure that there were 36 different configurations for the
two 6-sided dice? The first die could have come up as any one of six
numbers. For each of those, the second could have come up with any one
of six numbers.  Thus, the number of possibilities is $ 6 \times 6 =
36.$

How many different configurations for 3 8-sided dice?  $8 \times 8
\times 8 = 8^3 = 512$.

What about 7 dice, each with 20 sides? $20^7=1,280,000,000$
configurations.  See, aren't you glad we don't need to write them all
down?

Now, let's say that six people (Anne, Brock, Carl, Dev, Edgar, and Fred) are
going to run a race. You have to make a plaque that says who won first
place, who won second place, and who won third. If you want to get all
the possible plaques created beforehand, and just pull the right one
out as soon as the race ends, how many plaques would you need to get
engraved?

In this case, once someone has been given first place, they can't win
second or third place.  Thus, any of the 6 people can come in first,
but once you have engraved that person's name on the plaque, there are
only 5 people whose name can appear in second place.  Once you have
engraved that name, there are only 4 people whose name can appear in
third place.  Thus, you would get $6 \times 5 \times 4 = 120$ plaques
engraved.

What if the plaque includes all 6 places?  Then you would need $6 \times 5
\times 4 \times 3 \times 2 \times 1 = 720$ plaques engraved.  We use
this process often enough that we gave it a name.  We say ``I need 6
factorial plaques engraved.''  When we write a factorial, we use an
exclamation point:\index{factorial}

$$6! = 6 \times 5 \times 4 \times 3 \times 2 \times 1 = 720$$

The rule, then, is $n$ items can be ordered in $n!$ ways. We use the
word ``permutation'' to mean a particular ordering.  Thus
mathematicians actually say ``If you have a list of $n$ items then I
can generate $n!$ different permutations of those items''.

In Python, there is a \pyfunction{factorial} function in the math library:
\begin{Verbatim}[commandchars=\\\{\}]
> \textbf{python3}
>>> \textbf{import math}
>>> \textbf{math.factorial(6)}
720
\end{Verbatim}

Handy, right? Now you don't need to write loop to calculate factorials.

Remember when we only wanted the first three names on the plaque? We can do that problem using factorials:

$$6 \times 5 \times 4 = \frac{6 \times 5 \times 4 \times 3 \times 2 \times 1}{3 \times 2 \times 1} = \frac{6!}{3!}$$

This formulation makes it easy to figure out on any calculator with a ``!'' button.

The rule on this is to fill $m$ positions from $n$ items, it can be done this many ways:

$$\frac{n!}{(n-m)!}$$

\subsection{Choose}

Let's say that there are 12 kids in a classroom, and you need a team
of 4 to wipe down the desks.  How many different possible teams are
there? You know that if you were giving out four different positions
(Like the race gave out 1st, 2nd, and 3rd), the answer would be $12
\times 11 \times 10 \time 9$ or $12! / (12 - 4)!$.

However, once we pick the 4 people, we don't care what order they are
in, right?  In this problem, the team ``Anne, Brad, Carl, and Don'' is
the same as the team ``Carl, Don, Brad, and Anne''.

Thus, the quantity $12! / (12 - 4)!$ is many times too large because
it counts each permutation separately.  To get the right number, we
just divide this by the number of possible permuations for a group of
four people: $4!$

That gets us our answer: How many different teams of four can be chosen from 12 people?

$$\frac{12!}{(12-4)! 4!}= 495$$

In combinatorics, we use this quantity a lot, so we have given it a name: \textit{choose}\index{choose function}

We have also given it a notation.  ``12 choose 4'' is written like this:

$${12 \choose 4}$$

Python has the \pyfunction{math.comb} function:

\begin{Verbatim}[commandchars=\\\{\}]
> \textbf{python3}
>>> \textbf{import math}
>>> \textbf{comb(12, 4)}
495
\end{Verbatim}


\documentclass{article}
\input{commonheader}
\usepackage{comment}
\usepackage{enumitem}
\usepackage{algpseudocode}

\title{CS 5050: Homework 1}
\author{Andrew Pound}
\date{\today}

\begin{document}
\maketitle

I worked on this assignement with Chad Cummings.
\section{}
\begin{enumerate}[label=(\alph*)]
\item  Suppose we are comparing implementations of insertion sort and
merge sort on the same machine. For inputs of size n, insertion sort
runs in $8n^2$ steps, while merge sort runs in $64n \log n$ steps. For
which values of $n$ does insertion sort beats merge sort? {\bf (5
  points)}
\item[]
There are a few different ways to do this. The first way that we
attacked it was that we solved the inequality equation to easiest way
to solve this is to tabulate the answers for each side of the equatoin
and find the point that they cross.  (This would actually get a bit
long for this particular problem).  So instead a plot of the two
functions can be used to see where it crosses.

Equations:
\begin{equation}\label{eq:1a}
  \begin{split}
    8n^2 &\le 64n\log n\\
    \frac{1}{8} &\le  \frac{1}{n}\log n\\
    2^{\frac{1}{8}} &\le n^{\frac{1}{n}}
  \end{split}
\end{equation}

Plotting both sides of this, produces the graph below.
\begin{figure}[h!t]
  \centering
  \includegraphics[width=.7\linewidth]{prob1a}
  \caption{Plot of the inequality in equation \ref{eq:1a}.}
  \label{fig:prob1a}
\end{figure}

From this we can see that the cross over point is around 43.
Tabulating the values of $n^{1/n}$ in Table \ref{tab:1a}, the
neighborhood of 43 and
\begin{table}[h!t]
  \centering
  \caption{Tabulation of values around $n=43$.}
  \label{tab:1a}
  \begin{tabular}{cc}
    $n$ & $n^{1/n}$ \\
    \input{prob1a.tex}
  \end{tabular}
\end{table}
comparing them with the value of $2^{1/8} = 1.0905$, we can see that
any number greater than 43 satisfies the inequality.

\item  What is the smallest value of n such that an algorithm whose
running time is $100n^2$ runs faster than an algorithm whose running
time is $2^n$ on the same machine? {\bf (5 points) }
\item[] This can be solved in much the same manner as the previous
  problem. First, Let's examine the inequality.
  \begin{equation}\label{eq:1b}
    \begin{split}
      100 n^2 & \le 2^n\\
      \log_2(100n^2) & \le log_2(2^n) = n\\
      \frac{2}{n} \log(10n) &\le 1\\
      \frac{1}{n} \log(10n) &\le \frac1 2\\
      \log\left(n^{\frac{1}{n}}\right) &\le \frac{1}{2}\\
      \left(10n\right)^{\frac{1}{n}} &\le \sqrt{2}
    \end{split}
  \end{equation}
\end{enumerate}

The plot of this function is seen in Figure \ref{fig:1b}, and the
tabulation around the cross over is in Table \ref{tab:1b}.  This time
we ar comparing with the value $\sqrt{2} = 1.4142$, and we can see
that the cross over happens at $n=15$.
\begin{figure}[h!t]
  \centering
  \includegraphics[width=.7\linewidth]{prob1b}
  \caption{Plot of the inequality in equation \ref{eq:1b}.}
  \label{fig:1b}
\end{figure}

\begin{table}[h!t]
  \centering
  \caption{Tabulation of values around $n=15$.}
  \label{tab:1b}
  \begin{tabular}[h]{cc}
    $n$ & $(10n)^{1/n}$ \\
    \input{prob1b.tex}
  \end{tabular}
\end{table}

\vspace{3em}
\section{}
We have discussed the insertion sort algorithm in class and we showed
that in the worst case the algorithm runs in $\Theta(n^2)$ time. You
may wonder whether the algorithm can run asymptotically faster for
certain ``average'' cases. In this exercise, we will examine the running
time of the insertion sort algorithm on some average cases. Let $A$ be
the input array of n elements.

Note: For each of the questions, please give your answers in the
big-Theta notation and also briefly explain how you obtain your answer.

\begin{enumerate}[label=(\alph*)]
\item Consider the case where the first $n/2$ elements of $A$ have
already been sorted \emph{increasingly} and the last $n/2$ elements
have been sorted \emph{decreasingly}. For example, $A = \{1, 3, 5, 7,
8, 6, 4, 2\}$. Suppose we want to sort all elements of $A$ in
increasing order by using the insertion sort algorithm. What would be
the running time of the algorithm? What if the first $n/2$ elements of
$A$ have already been sorted decreasingly and the last $n/2$ elements
have been sorted increasingly? For example, $A = \{8, 6, 4, 2, 1, 3,
5, 7\}$. What would be the running time in this case? {\bf (10 points)}

\item[] The insertion sort algorithm that we discussed in class is
  given as
  \begin{algorithmic}
    \For{$i=2\dots n$}
    \State $x\gets A[i]$
    \State $j \gets i-1$
    \While{$A[j-1] < x \And j \ge 1$}
    \State $A[j+1]\gets A[j]$
    \State $j \gets j-1$
    \EndWhile
    \State $A[j+1]\gets x$
    \EndFor
  \end{algorithmic}
The outer for loop consists of $4c$ operations, and the inner loop
(when run) consists of $2c$ operations.
Now suppose that  the elements of our array are ordered in such a  way
that the first half of the array are already in ascending order, and
the second half is the worst case (descending order) and need to be
merged into the first half.  If that is the case, then the first half
of the algorithm is a best case scenario, and only uses
$4c\frac{n}{2}$ operations.  The second half, though will utilize the
inner for-loop to move some of the elements in the first half of the
array in order to insert in the correct place.
By considering the first elements in the second half, we can determine
a relation between the number of times in the inner loop and the
iteration index.  If the array is in the form specified, then the
first element of the second half will not require any moving to place
it.  The second will require 2 moves (the first element of the 2nd
half and the last element of the first half).  The third element in
the 2nd half will require 4 moves.  We see that for $i > \frac{n}{2}$,
the number of times in the inner loop will be $2(i-\frac{n}{2})$.
Thus, we can stick this into an expression for the total number of
operations
\begin{equation}
  \underbrace{4cn}_{\text{outer loop}} + \underbrace{\sum_{i =
    \frac{n}{2}}^n2\left(i-\frac{n}{2}\right)\left(2c\right).
}_{\text{inner loop}}
\end{equation}
Now expanding this, we get a total of
\begin{equation}
\begin{split}
  (4cn) + 4c\sum_{i=0}^{n/2}i &= 4cn +
  4c\frac{\frac{n}{2}\left(\frac{n}{2} + 1\right)}{2}\\
  &= 4cn + 2c\left(\frac{1}{4}n^2 + \frac{1}{2}n\right)\\
  &= 4cn + \frac{1}{2}n^2c + cn\\
  &= \frac{1}{2}cn^2 + 5cn\\
  &= O(n^2)
\end{split}
\end{equation}

%The second ordering

\item Consider the case where the elements of $A$ with odd indices have
already been sorted increasingly and elements with even indices have
been sorted decreasingly (we assume the index of $A$ starts from
1). For example, $A = \{1, 8, 2, 7, 3, 6, 4, 5\}$. Suppose we want to
sort all elements of $A$ in increasing order by using the insertion
sort algorithm. What would be the running time of the algorithm?

What if the elements of $A$ with odd indices have already been sorted
\emph{decreasingly} and elements with even indices have been sorted
\emph{increasingly}. For example, $A = \{8, 1, 7, 2, 6, 3, 5,
4\}$. What would be the running time in this case?  {\bf (10 points)}

\item[]
Ok, we'll begin by looking at how the algorithm works on the example
sequence.  The results of the steps of the algorithm are shown below.
The bolded numbers identify values that needed to be moved (i.e. the
values for which the inner loop was used). The first line is the
original sequence.
\begin{equation}
  \begin{matrix}
    1 & 8 & 2 & 7& 3  &6 & 4 & 5\\
    1 &&&&&&&\\
    1 & 8 &&&&&&\\
    1 & 2 & {\bf 8}&&&&&\\
    1 & 2 & 7 & {\bf 8}&&&&\\
    1 & 2 & 3 & {\bf 7}&{\bf 8}  &&&\\
    1 & 2 & 3 & 6& {\bf 7}&{\bf 8}  &&\\
    1 & 2 & 3 & 4& {\bf 6}& {\bf 7}&{\bf 8}  &\\
    1 & 2 & 3 & 4& 5  &{\bf 6} & {\bf 7}&{\bf 8} \\
  \end{matrix}
\end{equation}
From this we can see that for each pair of iterations, the number of
moves ends up being predictable.  In order to count the time, we then
start with the equation
\begin{equation}
  \underbrace{4cn}_{\text{outer loop}} + \underbrace{2 \sum_{i=1}^{n/2}
    (i-1)(2c)}_{\text{inner loop}}.
\end{equation}
The outer loop has $4c$ operations and happens for each element in the
array, thus we get $4cn$.   Each pair of iterations is hitting the
inner loop for the same number of times, namely $i-1$, for the $i$th
pair. The inner loop consists of $2c$ operations, thus we get the
summation over all the pairs, and multiply by 2 to get the full effect
of the inner loop.  This is
\begin{equation}
  \begin{split}
    2 \sum_{i=1}^{n/2}
    (i-1)(2c) &= 4c \sum_{i=1}^{n/2}(i - 1)\\
    &= 4c \sum_{i=1}^{n/2}i  - 4c\frac{n}{2}\\
    &= 4c \frac{\frac{n}{2}\left(\frac{n}{2} + 1\right)}{2} -
    4c\frac{n}{2}\\
    &= 2c\left(\frac{n^2}{4} + \frac{n}{2} - n\right)\\
    &= 2c\left(\frac{1}{4}n^2 - \frac{1}{2}n\right)
  \end{split}
\end{equation}
The entire operations count together then would be
\begin{equation}
  \begin{split}
    4cn + 2c\left(\frac{1}{4}n^2 - \frac{1}{2}n\right)&=
    4cn + \frac{1}{2}cn^2 - cn\\
    &= \frac{1}{2}cn^2 + 3cn\\
    &= O(n^2)
  \end{split}
\end{equation}
Now for the other example, let's see what it looks like:
\begin{equation}
  \begin{matrix}
    8& 1& 7& 2& 6& 3& 5 &4 \\
    8 &&&&&&&\\
    1 & {\bf 8} &&&&&&\\
    1 & 7 & {\bf 8}&&&&&\\
    1 & 2 & {\bf 7} & {\bf 8}&&&&\\
    1 & 2 & 6 & {\bf 7}&{\bf 8}  &&&\\
    1 & 2 & 3 & {\bf 6}& {\bf 7}&{\bf 8}  &&\\
    1 & 2 & 3 & 5& {\bf 6}& {\bf 7}&{\bf 8}  &\\
    1 & 2 & 3 & 4& {\bf 5}  &{\bf 6} & {\bf 7}&{\bf 8} \\
  \end{matrix}
\end{equation}
Essentially, the idea is the same, in that the iterations can be split
into pairs such that a pair will use the inner loop (to move larger
numbers) the same number of times.  Thus the analysis is (almost)
exactly the same: $ O(n^2)$.

\end{enumerate}

\section{}
For each of the following pairs of functions, indicate whether it is
one of the three cases: $f (n) = O(g(n))$, $f (n) = \Omega(g(n))$, or

$f (n) = \Theta(g(n))$. (30 points)
\begin{enumerate}[label=(\alph*)]
\item $f (n) = 100n + \log n$ and $g(n) = 6n + \log^2 n$.
\item[] $f(n) = \Omega(g(n))$.  My reasoning is this:
  \begin{equation}
    \begin{split}
      f(n) &\: ?\: g(n)\\
      100n + \log n &\: ? \: 6n+]log^2 n\\
      94n &\: ?\: \log^2 n - \log n
    \end{split}
  \end{equation}
Now, $\log^2 n - \log n < \log^2 n$, and comparing this as above, we
see that
\begin{equation}
  \begin{split}
    94n > \log^2 n > \log^2 n -\log n.
  \end{split}
\end{equation}
Thus, we can say that $f(n) = \Omega(g(n))$.
\item $f (n) = 20 \log n + 4$ and $g(n) = \log n^2 - 100$.
\item[] $f(n) = \Theta(g(n))$.  Playing with the equations a bit, we
  can see
  \begin{equation}
    \begin{split}
      f(n) & \: ? \: g(n)\\
      20\log n + 4 &\: ? \: \log n^2 - 100\\
      20 \log n + 4 &\: ? \: 2\log n - 100.
    \end{split}
  \end{equation}
Now, if we choose $c_1 = 20$ and $c_2 = 10$, then we get an asymptotic
upper and lower bound, respectively, by forming the equations $c_i
g(n)$.  Thus, we know that $f(n) = \Theta(g(n))$.
\item $f (n) =\frac{n^2}{\log n}$ and $g(n) = n \log^2 n$.
\item[] $f(n) = \Omega(g(n))$. Let's play with the equations:
  \begin{equation}
    \begin{split}
      \frac{n^2}{\log n} &\: ? \: n \log^2 n\\
      n &\: ? \: \log^3 n.
    \end{split}
  \end{equation}
Then using a nifty formula:
  \begin{equation}
    \log^k n = O(n^d) \quad \forall\: k>0, \: d > 0,
  \end{equation}
we can see that $d = 1$, and $k = 3$, and thus, $g(n) = O(f(n))$.
Or in other words, $f(n) = \Omega(g(n))$.
\item $f (n) =\sqrt{n}$ and $g(n) = \log^5 n$.
\item[] $f(n) = \Omega(f(n))$. Using the same nifty equation from
  above, we can see that $g(n) = O(f(n))$, as $k = 5$, and $d = 1/2$.
  Thus flipping this around, we see that $f(n) = \Omega(f(n))$.
\item $f (n) = n2^n$ and $g(n) = 3^n$ .
\item[] $f(n) = O(g(n))$.  We will make use of our other nifty
  formula
  \begin{equation}
    n^d = o(e^n), \quad \forall\: e > 1.
  \end{equation}
First, let's play a bit with our equation
\begin{equation}
\begin{split}
  n2^n &\: ?\: 3^n\\
  n &\: ? \:  \left(\frac{3}{2}\right)^n.
\end{split}
\end{equation}
This helps us to see that when $d = 1$ and $ e = 3/2$, then
we know that $f(n) = o(g(n))$ and therfore, $f(n) = O(g(n))$.
\item $f (n) = 4n \log n$ and $g(n) = n \log_3 n$.
\item[] $f(n) = \Theta(g(n))$. Playing with the equation, we get
  \begin{equation}
    \begin{split}
    4n \log n &\: ? \: n \log_3 n\\
    4 \log n &\: ? \: \frac{\log n}{\log 3}\\
    4 \log n &\: ? \: \frac{1}{\log 3} \log n\\
    4 &\: ? \: \frac{1}{\log 3}
    \end{split}
  \end{equation}
Now we see that $f(n) = \Theta(g(n))$.
\end{enumerate}

Note: For each question, you only need to give your answer and the
proof is not required.

\section{}
The \emph{knapsack problem} is defined as follows: Given as input a
knapsack of size $K$ and $n$ items whose sizes are $k_1 , k_2 ,
\dots , kn$ , where $K$ and $k_1 , k_2 , \dots , k_n$ are all real
numbers, find a full ``packing'' of the knapsack (i.e., choose a
subset of the n items such that the total sum of the sizes of the
items in the chosen subset is exactly $K$).

It is well known that the knapsack problem is NP-complete, which
implies that it is very likely that efficient algorithms (i.e., those
with a polynomial running time) for this problem do not exist. Thus,
people tend to look for good {\bf approximation algorithms} for
solving this problem. In this exercise, we relax the constraint of the
knapsack problem as follows. We still seek a packing of the knapsack,
but we need not look for a ``full'' packing of the knapsack; instead,
we look for a packing of the knapsack (i.e., a subset of the n input
items) such that the total sum of the sizes of the items in the chosen
subset is at least $K/2$ (but no more than $K$). This is called a
\emph{factor of 2 approximation solution} for the knapsack problem. To
simplify the problem, we assume that a factor of 2 approximation
solution for the knapsack problem always exists, i.e, there always
exists a subset of items whose total size is at least $K/2$ and at
most $K$.

Design a polynomial time algorithm for computing a factor of 2
approximation solution for this problem, and analyze the running time
of your algorithm (in the big-O notation). If your algorithm runs in
$O(n)$ time and is correct, then you get {\bf 5 extra points}. {\bf(20
  points)}

Note: You are required to clearly describe the main idea of your
algorithm. Although the pseudo-code is not required, you may also give
the pseudo-code if you feel it is helpful for you to describe your
algorithm. (The reason I want to see the algorithm description instead
of only the code or pseudo-code is that it would be difficult to
understand another person's code without any explanation.) You also
need to briefly explain why your algorithm works, i.e., why your
algorithm can produce a factor of 2 approximation solution. Finally,
please analyze the running time of your algorithm.

\paragraph{My algorithm}

In order to find a factor of 2 approximation, we will make use of the
assumptions that there definitely exists such an approximation, and
that all of the $k_i$ sizes are positive.

The algorithm is as follows:

Iterate through the list by pairs, finding the largest of each pair
and moving it to the lower position in the pair.
Then iterate and add the largest of each pair (unless it is too large
(i.e. makes the knapsack over full).
Then, if at the end of this loop, that knapsack is not at least 1/2
full, then iterate through the smaller of the pairs, adding them,
unless they overfill the knapsack.

Because there exists a factor of 2 approximation, this algorithm will
terminate when either the solution is found or the entire list is
added during the 3rd loop, which will then constitute a solution
(because one has to exist).

Pseudocode for the algorithm can be laid out as
{\singlespacing
\begin{algorithmic}
  \State $cap \gets$ Knapsack size/capacity
  \For{$i = 1,3,\dots,n$}
  \If{$k[i] \le k[i+1]$}
  \State Swap($k[i],k[i+1]$)
  \EndIf
  \EndFor
  \State $Sz = 0$
  \State // Let's fill in the larger sizes first
  \For{$i = 1,3,\dots,n$}
  \If{ $Sz + k[i] \le cap$}
  \State $Sz \gets Sz + k[i]$
  \EndIf
  \If{ $Sz \ge \frac{cap}{2}$}
  \State return
  \EndIf
  \EndFor
  \State // Now, start filling in the smaller sizes
  \For{$i = 2,4,\dots,n$}
  \If{ $Sz + k[i] \le cap$}
  \State $Sz \gets Sz + k[i]$
  \EndIf
  \If{ $Sz \ge \frac{cap}{2}$}
  \State return
  \EndIf
  \EndFor
\end{algorithmic}
}
The runtime for this algorithm is $O(\frac{3}{2}n)$, because at the
most it will utilize all three for-loops which are each over only half
the length of the array.
\end{document}