\subsection{Eigenvalues distribution}
\label{sec:dist}

One way to investigate if there is any significant difference between the regular and random SOMs is to compare their neural responses to the same random stimuli. Therefore, we measure the neural activity and build a covariance matrix out of it. Then, we compute the eigenvalues of the covariance matrix (or Gram matrix) and we estimate a probability distribution. Thus, we can compare the eigenvalues distributions of the two maps and compare them to each other. If the distributions are close enough in the sense of Wasserstein distance then the two SOMs are similar in terms of neural activation.  A Gram matrix is an $n \times n$ matrix given by where $n$ is the number of neurons of the map and ${\bf Y} \in \mathbb{R}^{n \times m}$ is a matrix for which each column is the activation of all $n$ neurons to a random stimulus.

From a computational point of view we construct the matrix ${\bf Y}$ by applying a set of stimuli to the self-organized map and computing the activity of each neuron within the map. This implies that ${\bf Y} \in \mathbb{R}^{m \times n}$, where $m=1024$ (the number of neurons) and $n={2, 3}$ (two- or three-dimensional input samples). Then we compute the covariance or Gram matrix as ${\bf M} = {\bf Y}{\bf Y}^T \in \mathbb{R}^{n \times n}$, where $n$ is the number of neurons. Then we compute the eigenvalues and obtain their distribution by sampling the activity of neurons of each experiment for $200$ different initial conditions using $50$ input sample each time. At the end of sampling we get an \emph{ensemble} of $200$ Gram matrices and finally we estimate the probability density of the eigenvalues on each \emph{ensemble} by applying a Kernel Density Estimation method~\citep{Parzen:1962} (KDE) with a Gaussian kernel and bandwidth $h=0.4$. This allows us to quantify any differences on the distributions of the regular and randomized SOMs by calculating the Earth-Mover or Wasserstein-1 distance over the two distributions (regular ($P$) and random SOM ($Q$)). The Wasserstein distance is computed as $W(P, Q) = \inf_{\gamma \in \Pi(P, Q)}\{\mathbb{E}_{(x, y) \sim \gamma}\Big[||x - y||\Big]\}$, where $\Pi(P, Q)$ denotes the set of all joint distributions $\gamma (x, y)$, whose marginals are $P$ and $Q$, respectively. Intuitively, $\gamma (x,y)$ indicates  how  much ``mass'' must be transported from $x$ to $y$ to transform the distribution $P$ into the distribution $Q$.

The distributions of the eigenvalues of the RSOM and the regular SOM are shown on figure~\ref{fig:eigenvalues}. We can conclude that the two distributions are alike and do not suggest any significant difference between the two maps in terms of neural activity. This implies that the RSOM and the regular SOM have similar statistics of their neural activities. This means that the loss of information and the \emph{stretch} to the input data from both RSOM and regular SOM are pretty close and the underlying  topology of the two maps do not really affect the neural activity. This is also confirmed
by measuring the Wasserstein distance between the two distributions. The blue curve shows the regular SOM or distribution $P$ and the black curve the RSOM or distribution $Q$. The Wasserstein distance between the two distributions $P$ and $Q$ indicates that the two distributions are nearly identical on all datasets. The Wasserstein distances in Table~\ref{table:distances}
confirm that the eigenvalues distributions of SOM and RSOM are almost identical indicating that both maps retain the
same amount of information after learning the representations of input spaces.

\begin{table}[!ht]
  \begin{center}
    \begin{tabular}{ll}
        \textbf{Experiment} & \textbf{Wasserstein Distance} \\
        \hline
        $2$D ring dataset               & $0.0000323$\\
        $2$D uniform dataset with holes & $0.0000207$  \\
        $3$D uniform dataset            & $0.0001583$ \\
        MNIST dataset                   & $0.0015$ \\
    \end{tabular}
      \caption{\textbf{Wasserstein distances of eigenvalues distributions.} We report here the Wasserstein
      distances between eigenvalues distributions of SOM and RSOM for each of the four major experiments we
      ran. The results indicate that the distributions are close pointing out that the SOM and RSOM capture
      a similar level of information during training. For more information regarding how we computed the
      eigenvalues distributions and the Wasserstein distance please see Section~\ref{sec:dist}.}
      \label{table:distances}
  \end{center}
\end{table}

\begin{figure}
  \includegraphics[width=\columnwidth]{eig-distributions-new.pdf}
  %
  \caption{Eigenvalues distribution for \textbf{A} 2D Ring dataset \textbf{B} 2D uniform dataset with holes \textbf{C} 3D uniform dataset and \textbf{D} MNIST Dataset
  }%
  \label{fig:eigenvalues}
 \end{figure}


\section{Calculations}
\label{sec:calc}
% CHANGE GB TO 1073741824 BYTES!!!

%----------------------------------------------------------------------
%           STORAGE
%----------------------------------------------------------------------
\subsection{Storage}

Storage requirements for 24 hours of 720p video data (.mp4):
\begin{equation*}
\begin{split}
Continual & = \left(\frac{24 \ hr}{1 \ day}\right)\left(\frac{60 \ min}{1 \ hr}\right)\left(\frac{60 \ sec}{ 1 \ min}\right)\left(\frac{17 \ mbit}{1 \ sec}\right)\left(\frac{1 \ byte}{8 \ bit}\right)\left(\frac{1 \ GB}{1737441824 \ byte}\right) \\
& = 170.99 \ \frac{GB}{day}
\end{split}
\end{equation*}

By limiting the active recording to moments where objects are detected, this storage requirement can be significantly lowered. Taking a theoretical family of 4 and a single dog, we can calculate for a minimum duration that the RPi2 is expected to record in 24 hours:

\begin{table}[H]
\centering
\caption{Known sighting during an average 24 hour day for a front door camera.}
\label{floor_table}
\begin{tabular}{@{}lcccc@{}}
\toprule
Type     & Qty & Duration (min) & Sighting/24 hr & Total  \\ \midrule
Person   & 4   & 1              & 2               & 8      \\
Dog      & 1   & 1              & 2               & 2      \\
Delivery & 2   & 1              & 1               & 2      \\ \midrule
         &     &                &                 & 12 min
\end{tabular}
\end{table}

With a minimum recording duration of 12 minutes based on the expected sightings (detected objects), the floor data requirements per 24 hours becomes:

\begin{equation*}
\begin{split}
Floor \ & = \left(\frac{12 \ min}{1 \ day}\right)\left(\frac{60 \ sec}{ 1 \ min}\right)\left(\frac{17 \ Mbit}{1 \ sec}\right)\left(\frac{1 \ byte}{8 \ bit}\right)\left(\frac{1 \ GB}{1737441824 \ byte}\right) \\
& = 0.000014254 \ \frac{GB}{day}\Rightarrow 0.014254 \ \frac{MB}{day} \Rightarrow 14.254 \ \frac{KB}{day}
\end{split}
\end{equation*}

%----------------------------------------------------------------------
%           NODS
%----------------------------------------------------------------------
\subsection{Night Vision}

\subsubsection{Analog Light Sensor}

\begin{equation*}
\begin{split}
    \log\left(I_{O}\right) &= mx + b \\
    I_{O} &= m\log\left(x\right) + b \\
    I_{O} &= 10\log\left(E_{v}\right)\left[l_{x}\right]
\end{split}
\end{equation*}

The sensor has an output current range of $0\mu A$--$50\mu A$ between $1$ lux--$100000$ lux. The breakout board converts this current into a voltage using a $68K\Omega$ resistor, giving a voltage output of $0V$--$3.4V$. Given the use-case of the light sensor, the $3.4V$ can be approximated to $3.3V$, which is a suitable voltage reference for the ATtiny85 microcontroller. With the output current range approximated to a voltage range of $0V$--$3.3V$ between $1$ lux--$100000$ lux,

\begin{equation*}
\begin{split}
    log_{i} &= \log\left(l_{x \ MAX} - l_{x \ MIN}\right) \\
    &= \log\left(100000\right) = 5 \\
\end{split}
\end{equation*}

To convert the analog value to a digital value, an 8-bit ADC is used:
\begin{equation*}
    ADC = \frac{log_{i}}{1024 \ steps} = 0.00488
\end{equation*}

Finally, the formula to obtain the final lux output, $l_{xo}$, on the RPi2:

\begin{equation*}
%\begin{split}
    l_{xo} = 10^{raw_{i} \times ADC} = 10^{raw_{i} \times 0.00488}
%\end{split}
\end{equation*}

where $raw_{i}$ is the raw value received by one of the analog pins on the ATtiny85.\\

\begin{figure}[H]
    \centering
    \makebox[\textwidth][c]{\includegraphics[width=0.2\textwidth]{ChauSze_SeniorDesign_VDR.PNG}}
    \caption{Voltage divider for sensor voltage reference.}
\end{figure}

Since 3.3V was chosen as the voltage reference value for the sensor, a voltage divider will be constructed in order to drop the 5V supply:

\begin{equation*}
    V_{1} = V_{CC} \frac{R_{2}}{R_{1} + R_{2}} = 5V \frac{3.3K\Omega}{1.7K\Omega + 3.3K\Omega} = 3.3V
\end{equation*}

%----------------------------------------------------------------------
%           LEDS
%----------------------------------------------------------------------
\subsubsection{Driving the LEDs}

\paragraph{Single LED Iteration}
The 8mm IR LEDs have a forward voltage drop of 1.3V and a current recommendation of 100mA. A resistor is added in series to each IR LED with the values chosen by:

\begin{equation*}
    R_{IR} = \frac{V_{CC} - V_{D}}{I_{D}} = \frac{5V - 1.3V}{100mA} = 37\Omega\approx 39\Omega
\end{equation*}

\begin{equation*}
    P_{D} = I_{D}^{2}R_{IR} = \left(100mA\right)^{2}\left(39\Omega\right) = 0.39W
\end{equation*}

\begin{figure}[H]
    \centering
    \makebox[\textwidth][c]{\includegraphics[width=0.5\textwidth]{ChauSze_SeniorDesign_IRLED.PNG}}
    \caption{Single LED circuit branch on the Night Vision block.}
\end{figure}

\paragraph{Series LED Iteration}
Another configuration that is being considered includes an additional IR LED onto each of the 3 ATtiny85 output pins. This changes the series resistor value to:

\begin{equation*}
    R_{IR} = \frac{V_{CC} - V_{D} - V_{D}}{I_{D}} = \frac{5V - 1.3V - 1.3V}{100mA} = 24\Omega\approx 27\Omega
\end{equation*}

\begin{equation*}
    P_{D} = I_{D}^{2}R_{IR} = \left(100mA\right)^{2}\left(27\Omega\right) = 0.27W
\end{equation*}

\begin{figure}[H]
    \centering
    \makebox[\textwidth][c]{\includegraphics[width=0.5\textwidth]{ChauSze_SeniorDesign_IRLEDII.PNG}}
    \caption{Double LED circuit branch on the Night Vision block.}
\end{figure}

%----------------------------------------------------------------------
%           CV PRE-PROCESSING
%----------------------------------------------------------------------
\subsection{Computer Vision: Pre-processing}

Before performing feature extraction, a given image--also known as a frame--is pre-processed in order to increase the accuracy of object-class detection. Some of these pre-processing steps also improve feature extraction during low-ambient light conditions. Some of these image processing techniques considers a given image in sections. This type of sectioning results in kernels, which is $m \times n$ matrix ``windows'' of a given frame. Convolution is then performed on the matrix, which averages all of the pixels around the center-most pixel.

\begin{figure}[H]
    \centering
    \begin{subfigure}[b]{0.475\textwidth}
        \includegraphics[width=\textwidth]{ChauSze_SeniorDesign_CalcEx0.PNG}
        \caption{Original image.}
        \label{fig:CalcEx0}
    \end{subfigure}
    ~ %add desired spacing between images, e. g. ~, \quad, \qquad, \hfill etc.
      %(or a blank line to force the subfigure onto a new line)
    \begin{subfigure}[b]{0.475\textwidth}
        \includegraphics[width=\textwidth]{ChauSze_SeniorDesign_CalcEx1.PNG}
        \caption{Converted to grayscale.}
        \label{fig:CalcEx1}
    \end{subfigure}
    \caption{An original image is converted to grayscale. A $3 \times 3$ matrix is then sectioned off for further pre-processing. A grid is used in (b) to show individual pixel boundaries.}\label{fig:CalcEx}
\end{figure}

\begin{figure}[H]
    \centering
    \makebox[\textwidth][c]{\includegraphics[width=0.875\textwidth]{ChauSze_SeniorDesign_CalcEx2.PNG}}
    \caption{The values for each pixel within the $3 \times 3$ matrix.}
    \label{fig:CalcEx2}
\end{figure}

As an arbitrary example, a $3 \times 3$ matrix in Fig. \ref{fig:CalcEx1}, bordered in red, will be considered for calculations.

%\begin{equation*}
%    118 + 90 + 57 + 105 + 69 + 59 + 167 + 113 + 63 = 841
%\end{equation*}

%----------------------------------------------------------------------
%           GAUSSIAN
%----------------------------------------------------------------------
\subsubsection{Gaussian Blurring}
% http://www.pixelstech.net/article/1353768112-Gaussian-Blur-Algorithm
% http://haishibai.blogspot.com/2009/09/image-processing-c-tutorial-4-gaussian.html
Blurring--also known as smoothing--is an image processing technique where high frequency noise is removed from a given image at the cost of detail reduction. For the purposes of this system, blurring will improve edge detection \cite[p.~28, 32]{mallat}.

Gaussian blurring was chosen due to its ability to utilize a weighted mean when averaging pixels in each frame. This specific blurring technique will allow for better performance when thresholding the frames. The name for this blurring technique comes from its use of a Gaussian function for calculations.

\begin{equation}
    G(x, y) = \frac{1}{\sqrt{2\pi\delta}}e^{-\frac{x^{2} + y^{2}}{2\delta^{2}}}
\end{equation}

Using the example arbitrary window in the previous section, each pixel is assigned a coordinate:

\begin{equation*}
    \begin{bmatrix}
        (-1, 1) & (0, 1) & (1, 1) \\
        (-1, 0) & (0, 0) & (1, 0) \\
        (-1, -1) & (0, -1) & (1, -1)
    \end{bmatrix}
\end{equation*}

Choosing an arbitrary $\delta$ value of 1.5 the center coordinate is calculated through the Gaussian function:

\begin{equation*}
    G(0, 0) = \frac{1}{\sqrt{2\pi 1.5}}e^{-\frac{0^{2} + 0^{2}}{2 \times 1.5^{2}}} = 0.0707
\end{equation*}

Calculating the rest of the coordinates results in a weighted matrix:

\begin{equation*}
    \begin{bmatrix}
        \textcolor{rp}{0.0454} & \textcolor{rb}{0.0566} & \textcolor{rp}{0.0454} \\
        \textcolor{rb}{0.0566} & 0.0707 & \textcolor{rb}{0.0566} \\
        \textcolor{rp}{0.0454} & \textcolor{rb}{0.0566} & \textcolor{rp}{0.0454}
    \end{bmatrix}
\end{equation*}

The sum of those values becomes:

\begin{equation*}
        (0.0435 \times 4) + (0.0566 \times 4) + 0.0707 =  0.4787
\end{equation*}

Dividing by the summation will result in a weighted average matrix:

\begin{equation*}
    \frac{1}{0.479}
    \begin{bmatrix}
        \textcolor{rp}{0.0454} & \textcolor{rb}{0.0566} & \textcolor{rp}{0.0454} \\
        \textcolor{rb}{0.0566} & 0.0707 & \textcolor{rb}{0.0566} \\
        \textcolor{rp}{0.0454} & \textcolor{rb}{0.0566} & \textcolor{rp}{0.0454}
    \end{bmatrix}
    =
    \begin{bmatrix}
        \textcolor{rp}{0.0948} & \textcolor{rb}{0.118} & \textcolor{rp}{0.0948} \\
        \textcolor{rb}{0.118} & 0.148 & \textcolor{rb}{0.118} \\
        \textcolor{rp}{0.0948} & \textcolor{rb}{0.118} & \textcolor{rp}{0.0948}
    \end{bmatrix}
\end{equation*}

Multiplying each coordinate of the $3 \times 3$ window will result in a Gaussian blur:

\begin{equation*}
    \begin{bmatrix}
        (118 \times 0.0948) & (90 \times 0.118)& (57 \times 0.0948) \\
        (105 \times 0.118) & (69 \times 0.148) & (59 \times 0.118)\\
        (167 \times 0.0948) & (113 \times 0.118) & (63 \times 0.0948)
    \end{bmatrix}
    =
    \begin{bmatrix}
        11.19 & 10.62 & 5.403 \\
        12.39 & 10.21 & 6.962 \\
        15.83 & 13.33 & 5.972
    \end{bmatrix}
\end{equation*}

Finally, summing up the values from the last matrix will result in the new value for the center-most pixel.

\begin{equation*}
        11.19 + 10.62 + 5.403 + 12.39 + 10.21 + 6.962 + 15.83 + 13.33 + 5.972 = 91.907
\end{equation*}

\begin{figure}[H]
    \centering
    \begin{subfigure}[b]{0.475\textwidth}
        \includegraphics[width=\textwidth]{ChauSze_SeniorDesign_CalcEx2.PNG}
        \caption{Center pixel before Gaussian blur: 69.}
        \label{fig:CalcEx3}
    \end{subfigure}
    ~ %add desired spacing between images, e. g. ~, \quad, \qquad, \hfill etc.
      %(or a blank line to force the subfigure onto a new line)
    \begin{subfigure}[b]{0.475\textwidth}
        \includegraphics[width=\textwidth]{ChauSze_SeniorDesign_CalcEx3.PNG}
        \caption{Center pixel after Gaussian blur: 93.}
        \label{fig:CalcEx4}
    \end{subfigure}
    \caption{Comparison of pixel values between the original $3 \times 3$ window against one filtered with Gaussian blur.}\label{fig:CalcEx5}
\end{figure}

As Fig. \ref{fig:CalcEx4} shows, the resulting pixel value from a Gaussian blur performed in GNU Image Manipulation Program (GIMP) is within range of the calculated value.

\begin{equation*}
    \frac{\mid GIMP - Calculated\mid}{Calculated} \times 100\% = \frac{\mid93 - 91.907\mid}{91.907} \times 100\% = 0.0119\%
\end{equation*}

%----------------------------------------------------------------------
%           THRESH
%----------------------------------------------------------------------
%\subsubsection{Thresholding}

%----------------------------------------------------------------------
%           CV DETECTION
%----------------------------------------------------------------------
\subsection{Computer Vision: Detection}

Facial detection is performed by searching a given image for features based on black and white rectangles \cite{ViolaJones}. The calculation for the regions again uses matrix convolution.

The first step is a single pass through the entirety of a given image that computes the sum of all pixel values above and to the left ($+y, -x$) of the position being examined, inclusive. This is known as an integral table, that is:

\begin{equation}
    I_{\sum}\left(x, y\right) = \sum_{\substack{
   x'\leq x \\
   y'\leq y
  }} i(x', y')
\end{equation}

Using the original window in Fig. \ref{fig:CalcEx2}:

\begin{equation*}
    \begin{bmatrix}
        (i_{(-1, 1)} = 118) & (i_{(0, 1)} = 208) & (i_{(1, 1)} = 265) \\

        (i_{(-1, 0)} = 223) & (\ddots) & (\vdots)\\

        (i_{(-1, -1)} = 390) & (\cdots) & (i_{(1, -1)} = 841)
    \end{bmatrix}
\end{equation*}

Feature identification is then performed using black and white rectangles over the integral table. The algorithm then passes feature types (various rectangles created by training the classifier using positive and negative results) over the integral table \cite{haarTri}. Identifying features from this point on is based on a simple summation of the rectangle corners:

\begin{equation}
    A_{M} = I(A) + I(C) - I(B) - I(D)
\end{equation}

\subsubsection{Haar}

The robustness of this method of object detection will be apparent with this final step. Consider Fig. \ref{fig:CalcHaar2}.

\begin{figure}[H]
    \centering
    \begin{subfigure}[b]{0.475\textwidth}
        \includegraphics[width=\textwidth]{ChauSze_SeniorDesign_CalcHaar1.png}
        \caption{Haar line feature being applied to Fig. \ref{fig:CalcEx1}. Points A to D corresponds to the white rectangle's corners.}
        \label{fig:CalcHaar1}
    \end{subfigure}
    ~ %add desired spacing between images, e. g. ~, \quad, \qquad, \hfill etc.
      %(or a blank line to force the subfigure onto a new line)
    \begin{subfigure}[b]{0.475\textwidth}
        \includegraphics[width=\textwidth]{ChauSze_SeniorDesign_CalcHaar2.png}
        \caption{Computing the leftmost white rectangle of Fig. \ref{fig:CalcEx1}'s integral image can be quickly achieved, allowing for real-time detection.}
        \label{fig:CalcHaar2}
    \end{subfigure}
    \caption{Facial detection through Haar-like features.}
    \label{fig:CalcHaar3}
\end{figure}

By analyzing the integral table of the original image, feature extraction becomes computationally inexpensive. If a $3 \times 3$ rectangle is covering the integral table that was solved earlier, the resulting calculation is:

\begin{equation*}
    A_{M} = I(A) + I(C) - I(B) - I(D) = 118 + 841 - 265 - 390 = 304
\end{equation*}

  This value is finally compared against the collection of rectangular Haar features collected during classifier training.

  Haar-like algorithm is grounded upon machine learning, a database of images containing positive and negative results for vehicles \cite{vehicleFeatures}. Since the detection algorithm based on Haar features is scale invariant, a set of images can then be used to train a Haar classifier for vehicles \cite{motionHaar}. This eliminates the use of blob analysis for vehicles.

%----------------------------------------------------------------------
%           ALGORITHMIC
%----------------------------------------------------------------------
%\subsection{Algorithmic Helper Functions}

%This section lists the calculations used in the software flowcharts.

%\subsubsection{Elapsed Time}

%\begin{equation*}

%\end{equation*}

%\newpage
%\begin{multicols}{2}
%\begin{onehalfspacing}
%\printglossary
%\end{onehalfspacing}
%\end{multicols}


%
% Chapter 3.2
%

\section*{3.2 The Mean Value Theorem}

\subsection*{Rolle's Theorem}

Let \(f\) be a function that satisfies the following three hypotheses:
\begin{enumerate}
    \item \(f\) is continuous on the closed interval \([a, b]\).
    \item \(f\) is differentiable on the open interval \((a, b)\).
    \item \(f(a)=f(b)\)
\end{enumerate}
Then there is a number \(c\) in \((a, b)\) such that \(f'(c)=0\).

\subsection*{Mean Value Theorem}

Let \(f\) be a function that satisfies the following hypothesis:
\begin{enumerate}
    \item \(f\) is continuous on the closed interval \([a, b]\).
    \item \(f\) is differentiable on the open interval \((a, b)\).
\end{enumerate}
Then there is \(c\) in \((a, b)\) such that
$$f'(c)=\frac{f(b)-f(a)}{b-a} \Leftrightarrow f(b)-f(a)=f'(c)(b-a)$$
\\
If \(f'(x)=0\) for all \(x\) in an interval \((a, b)\), then \(f\) is constant on \((a, b)\).


\section{Calculus in Real Vector Spaces}
\subsection{Stationary Points}
Let $C^2(\mathbb R^2)$ denote the collection of twice differentiable functions $\mathbb R^n\to\mathbb R$ with continuous second derivatives.
\begin{definition}
    Let $f\in C^2(\mathbb R^n)$.
    A point $\underline{a}\in\mathbb R^n$ is stationary (or is a stationary point of $f$) if $\nabla f(\underline{a})=\underline{0}$.
\end{definition}
If we expand $f$ near one of its stationary points $\underline{x}=\underline{a}$, then
\begin{align*}
    f(\underline{x})&=f(\underline{a})+(\underline{x}-\underline{a})\cdot\nabla f+\frac{1}{2}(x_i-a_i)(x_j-a_j)\frac{\partial^2f}{\partial x_i\partial x_j}(\underline{a})+o(|\underline{x}-\underline{a}|^2)\\
    &=f(\underline{a})+\frac{1}{2}(\underline{x}-\underline{a})^\top H(\underline{a})(\underline{x}-\underline{a})
\end{align*}
Where $H_{ij}(\underline{a})=\partial^2f/\partial x_i\partial x_j(\underline{a})$ is the Hessian, which is a real symmetric matrix for every $\underline{a}$.\\
Shift the origin to set $\underline{a}=\underline{0}$ and write $H=H(\underline{0})$.
As $H$ is real symmetric, we know that in another orthonormal basis $H$ has the form $H=\operatorname{diag}(\lambda_1,\ldots,\lambda_n)$, hence in this new coordinate system,
$$f(\underline{x})-f(\underline{0})=\frac{1}{2}\sum_{i=1}^n\lambda_ix_i^2+o(|x|^2)$$
Therefore,
\begin{theorem}
    Let $\lambda_i$ as before, then if $\forall i,\lambda_i>0$, then $\underline{0}$ is a local minimum.\\
    If instead $\forall i,\lambda_i<0$, then $\underline{0}$ is a local maximum.\\
    If some of the eigenvalues are positive and some are negative, then it is a saddle point.\\
    If some $i$ has $\lambda_i=0$,
\end{theorem}
If some $i$ has $\lambda_i=0$, we have to look at higher order terms in the Taylor series.
\begin{proof}
    Trivial.
\end{proof}
In the special case where $n=2$, we have $\det H=\lambda_1\lambda_2$ and $\operatorname{tr} H=\lambda_1+\lambda_2$, both of which are easier to calculate than the actual eigenvalues, so we can formulate a corollary to make things easier.
\begin{corollary}
    If $\det H>0,\operatorname{tr}H>0$, then it is a local minimum.\\
    If $\det H>0,\operatorname{tr}H<0$, it is a local maximum.\\
    If $\det H<0$, it is a saddle point.
\end{corollary}
\begin{proof}
    Obvious.
\end{proof}
Despite these results, one should note that the actual (global) maxima and minima might occur at boundaries where we may not have $\nabla f=0$.\\
Also, there are plenty of examples of functions who only have saddle points.
Consider a harmonic function $f$ on $D\subset\mathbb R^2$, i.e. $f_{xx}+f_{yy}=0$, then at a stationary point we necessarily have $\operatorname{tr}H=0$, hence either $\det H=0$ or it is a saddle point.\\
But of course, the theorem and corollary can help a lot in classifying stationary points.
\begin{example}
    Let $f(x,y)=x^3+y^3-3xy$, then $\nabla f=(3x^2-3y,3y^2-3x)^\top$, so the only stationary points are $(x,y)=(0,0),(1,1)$.
    $$H=\begin{pmatrix}
        6x&-3\\
        -3&6y
    \end{pmatrix}$$
    Hence, by the corollary, $(0,0)$ is a saddle point while $(1,1)$ is a minimum.
    One can easily produce a phase diagram using these information.
\end{example}
\subsection{Constraints and Lagrange Multipliers}
The optimisation problems in $\mathbb R^n$ are most likely coming with some sort of constraints.
We first illustrate this by way of an example
\begin{example}
    We want to find a circle centered at $(0,0)$ with smallest radius which intersects the parabola $y=x^2-1$.
    Naturally, there are many ways to do this.
    One can solve the problem directly by simply substitution
    $$x^2+y^2=x^2+(x^2-1)^2=x^4-x^2+1$$
    which minimum is $3/4$ by either calculus or completing square.
    So the smallest radius is $\sqrt{3}/2$.\\
    The second method to solve this probelm, which are are going to use, is called the Lagrange multipliers.
    Define a new function
    $$h(x,y,\lambda)=f(x,y)-\lambda g(x,y)$$
    where $f(x,y)=x^2+y^2$ is the function we want to minimise and $g(x,y)=y-x^2+1$ which vanishes if $(x,y)$ is on the parabola.
    $\lambda$ here is called the Lagrange multiplier.
    We now try to extremise $h$ over $x,y,\lambda$.
    Naturally, we take the particial derivatives
    $$\begin{cases}
        \partial h/\partial x=2x+2\lambda x\\
        \partial h/\partial y=2y-\lambda\\
        \partial h/\partial\lambda=y-x^2+1
    \end{cases}$$
    Hence the stationary point of $h$ are at $(x,y)=(0,-1),(\pm 1/\sqrt{2},-1/2)$.
    Plugging in yields the same solution.
\end{example}
Why does the method of Lagrange multiplier work?
Geometrically, if we want to minimise a function $f$ subject to $g=0$.
Now $\nabla g$ is always perpendicular to $g=0$.
Then, suppose the minimum of $f$ is $c$, then the graph of $f(x)=c$ would touch $g(x)=0$, therefore their gradients there are parallel, so $\nabla f=\lambda\nabla g$ for some $\lambda$, hence $\nabla h=0$ (note that $\partial h/\partial\lambda=0$ iff $g=0$), which is exactly the system we want to solve in the last part.\\
If there are many constraints $g_\alpha(\underline{x})=0$ of the minimisation problem of $f:\mathbb R^k\to\mathbb R$ where $\alpha$ ranges from $1$ to $k$, we can define an analogous
$$h(x_1,\ldots,x_k,\lambda_1,\ldots,\lambda_k)=f(\underline{x})-\lambda_\alpha g_\alpha(\underline{x})$$
then finding the stationary points of $h$ can help to determine the minimisation problem.


\documentclass{article}

\usepackage{stmaryrd}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{relsize}
\usepackage{bm}
\usepackage{IEEEtrantools}
\usepackage{graphicx}
\usepackage[font={small,it}, width=\textwidth]{caption}
\usepackage{subcaption}
\usepackage{hyperref}
\usepackage{cases}
\usepackage{xfrac}
\usepackage{comment}
\usepackage{framed}
\usepackage[ ddmmyyyy ]{datetime}
\usepackage{fancyhdr}
\usepackage{enumitem}
\usepackage{cite}
\usepackage{float}
\usepackage{multirow}

\newcommand{\source}[1]{\caption*{\hfill Source: {#1}} }

\usepackage{mathtools}
\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\sat}{sat}

\oddsidemargin = 20pt
\textwidth = 420pt

\hypersetup{
     colorlinks   = true,
     linkcolor    = blue
}

\title{The Beagle Project}
\author{Marcus Greiff}

\begin{document}
\maketitle

\section{Introduction}
This project is still in it's early stages, and the following document simply defines the equations of the DC motor model which is used to simulate the currently implemented regulators. A more detailed derivation and definitions will be added as soon as possible.
\section{Motor model}
\subsubsection*{Governing equations}
Assuming a simple DC motor model with constant magnetic field, Newton's and Kirchoff's laws can be used to derive the governing equations
\begin{flalign}
\frac{d^2\theta(t)}{dt^2} &= \frac{1}{J}\Big(K_t i(t) - b\frac{d\theta (t)}{d t}\Big)\\
\frac{di(t)}{dt} &= \frac{1}{L}\Big(-Ri(t) + V(t) - K_e\frac{d\theta(t)}{dt}\Big)
\end{flalign}
\subsubsection*{Statespace form}
\begin{flalign}
\dot{\mathbf{x}}(t) &= \mathbf{A}\mathbf{x}(t) + \mathbf{B}\mathbf{u}(t)\\
\mathbf{y}(t) &= \mathbf{C}\mathbf{x}(t)
\end{flalign}
Let
\begin{equation}
\mathbf{x}(t) = \begin{bmatrix}\theta(t) & \dot{\theta}(t) & i(t) \end{bmatrix}^T, \qquad \mathbf{u}(t) = V(t)
\end{equation}
then
\begin{equation}
\mathbf{A} =\begin{bmatrix} 0 & 1 & 0 \\ 0 & -b/J & K_t/J  \\ 0 & -K_e/L & -R/L  \end{bmatrix}, \quad
\mathbf{B} =\begin{bmatrix} 0  \\ 0  \\ 1/L  \end{bmatrix}, \quad
\mathbf{C} = \mathbb{I}_{3\times 3}
\end{equation}
\subsubsection*{Laplace domain equivalent}
Let
\begin{equation}
\mathcal{L}\begin{Bmatrix}\Theta(t)\end{Bmatrix}_s=\Theta(s), \quad\mathcal{L}\begin{Bmatrix}i(t)\end{Bmatrix}_s=I(s), \quad\mathcal{L}\begin{Bmatrix}V(t)\end{Bmatrix}_s=V(s), \quad
\end{equation}
then
\begin{flalign}
s^2\Theta(s) &= \frac{1}{J}\Big(K_t I(s) - b s\Theta(s)\Big)\\
sI(s) &= \frac{1}{L}\Big(-R I(s) + V(s) - K_e s\Theta(s)\Big)
\end{flalign}
\subsubsection*{transfer function from $V(s)$ to $\Theta(s)$}
\begin{flalign}
G(s)_{U\rightarrow\Theta} &= \frac{\Theta(s)}{U(s)} =  \frac{K_t}{s((sL+R)(J s +b) +K_tK_e)}U(s)
\end{flalign}
Using a partial fraction decomposition, this may be written
\begin{flalign}
G(s)_{U\rightarrow\Theta} = \frac{A}{s} - \frac{B + Cs}{s^2 + D s + E} = \frac{A}{s} + \frac{B}{s^2 + D s + E}+\frac{Cs}{s^2 + D s + E}
\end{flalign}
\begin{flalign}
\frac{Kt}{(s*(R*b + Ke*Kt))} - \frac{(J*Kt*R + Kt*L*b)/(R*b + Ke*Kt) + (J*Kt*L*s)/(R*b + Ke*Kt)}{J*L*s^2 + (L*b + J*R)*s + R*b + Ke*Kt}
\end{flalign}
\subsubsection*{transfer function from $V(s)$ to $I(s)$}

\subsubsection*{LS-estimation}
Using a Bilinear z-transform, it is evident that
\begin{flalign}
\mathcal{Z}\begin{Bmatrix}G(s)\end{Bmatrix}_z = \frac{a_2z^2 + a_1z^1 + a_0}{z^3 + b_2z^2 + b_1z^1 + b_0}
\end{flalign}
Letting
\begin{equation}
\begin{cases}
\theta =  \begin{bmatrix}a_2 & a_1 & a_0 & b_2 & b_1 & b_0 \end{bmatrix}^T\\
\varphi(t-1) = \begin{bmatrix}u(t-1)& u(t-2) & u(t-3) & -y(t-1)& -y(t-2) & -y(t-3) \end{bmatrix}^T
\end{cases}
\end{equation}
we note that the discrete time difference equation can be written
\begin{equation}
y(t) = \varphi(t-1)^T\theta
\end{equation}

\subsection{Controllers}
\begin{flalign}
\dot {x}_1 &= x_2\\
\dot {x}_2 &= (-bx_2 + K_tx_3)/J  \\
\dot {x}_3 &= (-K_ex_2 -Rx_3+ u)/L
\end{flalign}
with
\begin{equation}
V(x_1, x_2, x_3) = \frac{1}{2}(x_1^2 + x_2^2 + x_3^2)
\end{equation}
\begin{flalign}
\frac{\partial V(x_1, x_2, x_3)}{\partial t} &= {x}_1\dot{x}_1 + {x}_2\dot{x}_2 + {x}_3\dot{x}_3\\
&= {x}_1x_2 + {x}_2(-bx_2 + K_tx_3)/J+ {x}_3(-K_ex_2 -Rx_3+ u)/L\\
&= {x}_1x_2 -bx_2^2/J + (K_t/J -K_e/L)x_2{x}_3 -Rx_3^2/L+ u/L
\end{flalign}
\newpage\bibliography{Bibliography}{}
\bibliographystyle{IEEEtran}
\end{document}


% chapter included in vclmanual.tex
\documentclass[vcl_manual.tex]{subfiles}
\begin{document}

\chapter{Examples}\label{chap:Examples}
\flushleft

This example calculates the polynomial $x^3 + 2\cdot x^2 - 5\cdot x + 1$ on a floating point vector. The order of calculation is specified by parentheses in order to make shorter dependency chains.

\begin{example}
\label{examplePolynomial}
\end{example}
\begin{lstlisting}[frame=single]
Vec4f polynomial (Vec4f x) {
    return (x + 2.0f) * (x * x) + ((-5.0f) * x + 1.0f);
}
\end{lstlisting}
\vspacebig

In 64-bit Windows, you may add \codei{\_\_vectorcall} and use a Clang or Microsoft compiler. This makes sure that vector parameters are transferred in registers rather than in memory. This is not needed when the function is inlined or when compiling for other platforms than Windows:

\begin{example}
\label{examplePolynomialVectorcall}
\end{example}
\begin{lstlisting}[frame=single]
Vec4f __vectorcall polynomial (Vec4f x) {
    return (x + 2.0f) * (x * x) + ((-5.0f) * x + 1.0f);
}
\end{lstlisting}
\vspacebig

The next example transposes a 4x4 matrix, using the AVX2 instruction set.

\begin{example}
\label{exampleTranspose4x4}
\end{example}
\begin{lstlisting}[frame=single]
void transpose(float matrix[4][4]) {
    Vec8f row01, row23, col01, col23;
    // load first two rows
    row01.load(&matrix[0][0]);
    // load next two rows
    row23.load(&matrix[2][0]);
    // reorder into columns
    col01 = blend8f<0,4, 8,12,1,5, 9,13>(row01, row23);
    col23 = blend8f<2,6,10,14,3,7,11,15>(row01, row23);
    // store columns into rows
    col01.store(&matrix[0][0]);
    col23.store(&matrix[2][0]);
}
\end{lstlisting}
\vspacesmall

Same example with AVX512:

\begin{example}
\label{exampleTranspose4x4avx512}
\end{example}
\begin{lstlisting}[frame=single]
void transpose(float matrix[4][4]) {
    Vec16f rows, columns;
    // load entire matrix as rows
    rows.load(&matrix[0][0]);
    // reorder into columns
    columns = permute16f<0,4,8,12,1,5,9,13,
        2,6,10,14,3,7,11,15>(rows);
    // store columns into rows
    columns.store(&matrix[0][0]);
}
\end{lstlisting}
\vspacebig

The next example makes a matrix multiplication of two 4x4 matrixes.

\begin{example}
\label{exampleMatrixMul4x4}
\end{example}
\begin{lstlisting}[frame=single]
void matrixmul(float A[4][4], float B[4][4], float M[4][4]){
    // calculates M = A*B
    Vec4f Brow[4], Mrow[4];
    int i, j;
    // load B as rows
    for (i = 0; i < 4; i++) {
        Brow[i].load(&B[i][0]);
    }
    // loop for A and M rows
    for (i = 0; i < 4; i++) {
        Mrow[i] = Vec4f(0.0f);
        // loop for A columns, B rows
        for (j = 0; j < 4; j++) {
            Mrow[i] += Brow[j] * A[i][j];
        }
    }
    // store M
    for (i = 0; i < 4; i++) {
        Mrow[i].store(&M[i][0]);
    }
}
\end{lstlisting}
\vspacebig

The next example makes a table of the sin function and gets sin(x) and cos(x) by table lookup.

\begin{example}
\label{exampleSinTable}
\end{example}
\begin{lstlisting}[frame=single]

#include <cmath>

const double pi = 3.14159265358979323846;

// length of table. Must be a power of 2.
#define sin_tablelen 1024
// the accuracy of table lookup is +/- pi/sin_tablelen

class SinTable {
protected:
    float table[sin_tablelen];
    float resolution;
    float rres;  // 1./resolution
public:
    SinTable();  // constructor
    Vec4f sin(Vec4f x);
    Vec4f cos(Vec4f x);
};

SinTable::SinTable() {  // constructor
    // compute resolution
    resolution = float(2.0 * pi / sin_tablelen);
    rres = 1.0f / resolution;
    // Initialize table (No need to use vectors here because this
    // is calculated only once:)
    for (int i = 0; i < sin_tablelen; i++) {
        table[i] = sinf((float)i * resolution);
    }
}

Vec4f SinTable::sin(Vec4f x) {
    // calculate sin by table lookup
    Vec4i index = roundi(x * rres);
    // modulo tablelen equivalent to modulo 2*pi
    index &= sin_tablelen - 1;
    // look up in table
    return lookup<sin_tablelen>(index, table);
}

Vec4f SinTable::cos(Vec4f x) {
    // calculate cos by table lookup
    Vec4i index = roundi(x * rres) + sin_tablelen/4;
    // modulo tablelen equivalent to modulo 2*pi
    index &= sin_tablelen - 1;
    // look up in table
    return lookup<sin_tablelen>(index, table);
}

int main() {
    SinTable sintab;
    Vec4f a(0.0f, 0.5f, 1.0f, 1.5f);
    Vec4f b = sintab.sin(a);
    // b = (0.0000 0.4768 0.8416 0.9973)
    // accuracy +/- 0.003
    ...
    return 0;
}
\end{lstlisting}
\vspacesmall

\end{document}


\begin{tabular}{lcll}\hline
Variate    & $x$         & \ccode{double} & $\mu \leq x < \infty$ \\
Location   & $\mu$       & \ccode{double} & $-\infty < \mu < \infty$\\
Scale      & $\lambda$   & \ccode{double} & $\lambda > 0$ \\ \hline
\end{tabular}

The probability density function (PDF) is:

\begin{equation}
P(X=x) =  \lambda e^{-\lambda (x - \mu)}
\end{equation}

The cumulative distribution function (CDF) is:

\begin{equation}
P(X \leq x) = 1 - e^{-\lambda (x - \mu)}
\end{equation}

\subsection{Sampling}

An exponentially distributed sample $x$ is generated by the
transformation method, using the fact that if $R$ is uniformly
distributed on $(0,1]$, $1-R$ is uniformly distributed on $[0,1)$:

\[
   R = \mbox{uniform positive sample in (0,1]}\\
   x = \mu - \frac{1}{lambda} \log(R)
\]

\subsection{Maximum likelihood fitting}

The maximum likelihood estimate $\hat{\lambda}$ is $\frac{1}{\sum_i
x_i}$. The distribution of $\frac{\lambda}{\hat{\lambda}}$ is
approximately normal with mean 1 and standard error $\frac{1}{\sqrt{N}}$
\citep{Lawless82}.

% xref J1/p49 for derivation of standard error.


\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{dsfont}
\usepackage{amsthm}
\newtheorem*{theorem}{Theorem}
\begin{document}
\setlength{\parskip}{10pt plus 1pt minus 1pt}
\title{Coarse Geometry and Groups}
\author{Pranav Garg}
\maketitle
\section{Quasi-Isometry}
\subsection*{Definitions}
Let $(X, d_X)$ and $(Y, d_Y)$ be metric spaces.

$f:X\rightarrow Y$ is said to be K-Lipschitz if\[\forall x_1, x_2\in X, d_Y(f(x_1),f(x_2))\leq K\cdot d_X(x_1, x_2)\]

$f:X\rightarrow Y$ is said to be (K, L) Coarse Lipschitz if\[\forall x_1, x_2 \in X, d_Y(f(x_1),f(x_2)) \leq K\cdot d_X(x_1, x_2) + L\]

$(X,d_X)$ and $(Y, d_Y)$ are said to be bilipschitz if $\exists f:X\rightarrow Y$ and $g:Y\rightarrow X$, lipschitz such that $f\circ g=\mathds{1}_Y$ and $g\circ f=\mathds{1}_X$.

$f, g : X\rightarrow Y$ coarse lipschitz. We say $f\sim g$ if $\exists c>0$ such that $\forall x\in X, d_Y(f(x), g(x))<c$.

$(X,d_X)$ and $(Y, d_Y)$ are said to be quasi-isometric if $\exists f : X\rightarrow Y$ and $g : Y\rightarrow X$, coarse lipschitz such that $f\circ g\sim \mathds{1}_Y$ and $g\circ f\sim \mathds{1}_X$.

\subsection*{Examples}
\begin{itemize}

\item $\mathbb{Z}$ is quasi-isometric to $\mathbb{R}$.
\item A bounded metric space is quasi-isometric to a point.
\item If $\Gamma$ is a metric space, then the space of vertices is quasi-isometric to the space consisting of vertices and edges with the metric as defined in the section on graph metric.
\end{itemize}
\section{Word Metric}
Let $G$ be a finitely generated group and $S$ be a finite generator set for $G$.
For $g\in G$, define norm of $g$, \[\|g\| = min\{n \geq 0 : \exists \alpha_1, \alpha_2,..., \alpha_n \in S\cup S^{-1}, g=\alpha_1 \alpha_2 ...\alpha_n\} \]

This satisfies the following properties:
\begin{itemize}
\item $\|g\| = 0\Leftrightarrow g=e$ where $e$ is the identity of $G$.
\item $\|g^{-1}\|=\|g\|$
\item $\|g_1g_2\|\leq \|g_1\|+\|g_2\|$
\end{itemize}
Define the word metric $d$ on $G$ by \[\forall g,h\in G,\; d(g,h)=\|g^{-1}h\|\]
The properties of the norm listed above can be used to prove that this is a metric.
\begin{theorem}
If $S_1$ and $S_2$ are finite generating sets of $G$ and $d_1$ and $d_2$ are the corresponding word metrics, then $(G,d_1)$ is bilipschitz to $(G,d_2)$.
\end{theorem}
\begin{proof}
Consider $f:(G,d_1)\rightarrow(G,d_2)$ and $h:(G,d_2)\rightarrow(G,d_1)$, both identity maps. Clearly $f\circ h$ and $h\circ f$ are also identity maps.

Let $k_1=max\{\|\alpha\|_{S_2}:\alpha\in S_1\}.$
As $S_1$ is finite, so is $k_1$.
Now, if $\|g\|_{S_1}=n_1$, then $\exists \alpha_1, \alpha_2,...,\alpha_n \in S_1\cup S^{-1}_1$ such that $g=\alpha_1 \alpha_2 ...\alpha_n$
\[\Rightarrow \|g\|_{S_2} \leq \|\alpha_1 \|_{S_2} + |\alpha_2\|_{S_2}+...+|\alpha_n\|_{S_2}\]
\[\therefore\|g\|_{S_2}\leq \underbrace{k_1+k_1+...+k_1}_{\text{n times}}=k_1\cdot\|g\|_{S_1}\]

We have shown $\|g\|_{S_2}\leq k_1\|g\|_{S_1}.$ By replacing $g$ by $g_1 g^{-1}_2$, for $g_1,g_2\in G$, we get that $f$ is $k_1 -$lipschitz.

Similarly, we can construct a $k_2$ so that $h$ is $k_2 -$lipschitz.

Thus $(G,d_1)$ is bilipschitz to $(G,d_2)$.
\end{proof}
\subsection*{Cayley Graph}
The Cayley Graph on a finitely generated group $G$ with respect to a finite generator set $S$ is a graph with vertices as the set $G$ and edges as $E = \{(g, g\alpha ), \alpha \in S\cup S^{-1}\}$. The maps $i$ and $\tau$ are defined as: $i(g,g\alpha )=g$ and $\tau (g,g\alpha )=g\alpha$. The involution is $\overline{(g,g\alpha)}=(g\alpha,g)$.

The word metric could alternatively and equivalently be defined as the graph metric on the Cayley graph.
\section{Growth Function}
Let $G$ be a finitely generated group and $S$ be a finite generator set for $G$.
The growth function for $G$ and $S$, $\rho_{G,S}:\mathbb{N}\rightarrow\mathbb{N}$ is defined as:
\[\rho_{G,S}(r)=|\{g\in G\; \vert\; \|g\|\leq r \}|\]

Two growth functions $\rho_1$ and $\rho_2$ are defined to be equivalent, $\rho_1\sim\rho_2$, if $\exists k,c>0$ such that \[\forall r>0,\;\; \frac{1}{k}\rho_1(\frac{r}{c})\leq \rho_2(r)\leq k\rho_1(cr)\]
\begin{theorem}
If $S_1$ and $S_2$ are finite generating sets for $G$, then $\rho_{G,S_1}\sim\rho_{G,S_2}$
\end{theorem}
\begin{proof}
Construct $k_1$ and $k_2$ as in the proof of the previous theorem.

Let $c=max\{k_1,k_2\}$.
\[\|g\|_{S_2}\leq r\;\Rightarrow\;\frac{r}{c}\leq\|g\|_{S_1}\leq cr\]
\[\Rightarrow\{g\in G:\|g\|_{S_1}\geq \frac{r}{c}\}\subseteq\{g\in G:\|g\|_{S_2}\leq r\}\subseteq\{g\in G:\|g\|_{S_1}\leq cr\}\]
\[\therefore\rho_{G,S_1}(\frac{r}{c})\leq\rho_{G,S_2}\leq\rho_{G,S_1}(cr)\]
\end{proof}
\begin{theorem}
Let $\rho_n(r)=r^n$, $\rho_m(r)=r^m$. Then, $\rho_n\sim\rho_m\;\Rightarrow\;n=m$
\end{theorem}
\end{document}


% This LaTeX was auto-generated from an M-file by MATLAB.
% To make changes, update the M-file and republish this document.

\subsection*{dynamics\_cp.m}

\begin{par}
\textbf{Summary:} Implements ths ODE for simulating the cart-pole dynamics.
\end{par} \vspace{1em}

\begin{verbatim}  function dz = dynamics_cp(t, z, f)\end{verbatim}
    \begin{par}
\textbf{Input arguments:}
\end{par} \vspace{1em}

\begin{lstlisting}
%		t     current time step (called from ODE solver)
%   z     state                                                    [4 x 1]
%   f     (optional): force f(t)
%
% *Output arguments:*
%
%   dz    if 3 input arguments:      state derivative wrt time
%         if only 2 input arguments: total mechanical energy
%
%
% Note: It is assumed that the state variables are of the following order:
%       x:        [m]     position of cart
%       dx:       [m/s]   velocity of cart
%       dtheta:   [rad/s] angular velocity
%       theta:    [rad]   angle
%
%
% A detailed derivation of the dynamics can be found in:
%
% M.P. Deisenroth:
% Efficient Reinforcement Learning Using Gaussian Processes, Appendix C,
% KIT Scientific Publishing, 2010.
%
% Copyright (C) 2008-2013 by
% Marc Deisenroth, Andrew McHutchon, Joe Hall, and Carl Edward Rasmussen.
%
% Last modified: 2013-03-08

function dz = dynamics_cp(t,z,f)
\end{lstlisting}

\subsection*{Code}

\begin{lstlisting}
l = 0.5;  % [m]      length of pendulum
m = 0.5;  % [kg]     mass of pendulum
M = 0.5;  % [kg]     mass of cart
b = 0.1;  % [N/m/s]  coefficient of friction between cart and ground
g = 9.82; % [m/s^2]  acceleration of gravity

if nargin==3
  dz = zeros(4,1);
  dz(1) = z(2);
  dz(2) = ( 2*m*l*z(3)^2*sin(z(4)) + 3*m*g*sin(z(4))*cos(z(4)) ...
          + 4*f(t) - 4*b*z(2) )/( 4*(M+m)-3*m*cos(z(4))^2 );
  dz(3) = (-3*m*l*z(3)^2*sin(z(4))*cos(z(4)) - 6*(M+m)*g*sin(z(4)) ...
          - 6*(f(t)-b*z(2))*cos(z(4)) )/( 4*l*(m+M)-3*m*l*cos(z(4))^2 );
  dz(4) = z(3);
else
  dz = (M+m)*z(2)^2/2 + 1/6*m*l^2*z(3)^2 + m*l*(z(2)*z(3)-g)*cos(z(4))/2;
end
\end{lstlisting}


\chapter{Convolutional Neural Networks}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{6DL/Convolution}
\input{6DL/ConvolutionalOperator}
\input{6DL/convolution-filter}
\input{6DL/ClassicCNNs}
\input{6DL/DenseNet}
\input{6DL/Detail_ResNet}

\input{6DL/CNN-Examples}

\chapter{MgNet: a Unified Framework for CNN and MG}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{6DL/mgnet}
\input{6DL/mgnet_datafeature}
\input{6DL/mgnet_iterativescheme}
\input{6DL/mgnet_densenet}
\input{6DL/mgnet_DPN}
\input{6DL/mgnet_relation}
\input{6DL/mgnet_numerica}
\input{6DL/mgnet_summary}

\input{6DL/ConstrainedLinearModel}
\input{6DL/Bottleneck-MgNet}
\input{6DL/DenseNet-MgNet}


\subsection{Cesàro sum}

The Cesàro sum is the limit of the average of the first \(n\) partial sums.

That is:

\(\lim_{n\rightarrow \infty }\dfrac{1}{n}\sum_{k=1}^ns_k\)

Consider the sequence \(\{1,-1,1,-1,...\}\)

The partial sum is:

\(s_k=\sum_{i=1}^ka_i\)

\(s_k=k\mod(2)\)

The Cesàro sum is:
\(\lim_{n\rightarrow \infty }\dfrac{1}{n}\sum_{k=1}^ns_k\)

\(\lim_{n\rightarrow \infty }\dfrac{1}{n}\sum_{k=1}^nk\mod(2)\)

\(\dfrac{1}{2}\)


\subsection{Test for a Conservative Vector Field}
\noindent
Since we know that all conservative vector fields have potential function, we can create a test to see if a given vector field is conservative. $\vec{F_{\text{Conservative}}} = \langle f_x, f_y \rangle$, so $f_{xy} = f_{yx}$. That is, a vector field $\vec{F}(x,y) = \langle P(x,y), Q(x,y) \rangle$ is conservative if $P_y = Q_x$.\\
For 3D vector field, the test is a little more complicated. A vector field $\vec{F}\langle P(x,y,z), Q(x,y,z), R(x,y,z)\rangle$ is conservative if $P_y = Q_x$, $Q_z = R_y$, and $R_x = P_z$.\\

\noindent
For example, let see if $\langle yz, xz, xy\rangle$ is conservative.
\begin{equation*}
	\frac{\partial}{\partial y}yz = z = \frac{\partial}{\partial x}xz
\end{equation*}
\begin{equation*}
	\frac{\partial}{\partial z}xz = x = \frac{\partial}{\partial y}xy
\end{equation*}
\begin{equation*}
	\frac{\partial}{\partial y}xy = y = \frac{\partial}{\partial z}yz
\end{equation*}
So, the vector field is conservative.


\subsection{Transaction Privacy}
The basis for transactions is that the amount paid must equal the product cost plus fees, plus change returned to purchaser
$$ {paid} = {cost} + {fees} + {change}$$
or rather,
$$ {paid} - {change} = {cost} + {fees}$$

So using our pairing-based cryptography, we can encrypt the purchase offer as a triple
$(T_{buy},R_{sell}, P_{buy})$
$$ T_{buy} = \frac{k_{rand} \, s_{buy}}{paid - change} \, U \in G_1$$
$$ R_{sell} = \frac{1}{k_{rand}}\,P_{sell} \in G_2$$
for some random value $k_{rand}$ as a blinding factor, the buyer's secret key $s_{buy} \in Z_q$, buyer's public key $P_{buy} = s_{buy} \, V \in G_2$,  seller's public key $P_{sell} \in G_2$, and generators $U \in G_1$, and $V \in G_2$.

Blinding factors ensure that repeated transactions for the same amount will appear different each time. Only the buyer needs to know the amount paid, and the change returned. The seller only needs assurance that their cost plus fees is covered.

At the seller's end, this transaction can be completed by computing $C_{sell}$
$$ C_{sell} = \frac{cost+fees}{s_{sell}} \, R_{sell} \in G_2$$
for seller's private key $s_{sell} \in Z_q$,
and then verifying the pairing relation:
$$ e(T_{buy},C_{sell}) = e(U,P_{buy}) \in G_T$$
for buyer's public key $P_{buy}$.

This accomplishes three things. First, only the intended seller can verify the pairing relation, since it requires their private key, $s_{sell}$. Second, it proves to the seller that the buyer, identified by their public key, $P_{buy}$, originated the transaction. And third, it proves to the seller that the buyer paid the proper amount after receiving their change.

Any mismatch between the two pairings indicates either that the seller was illegitimate and could not decrypt the $R_{sell}$ value properly, or that a presumed buyer did not originate the transaction, or else they didn't send a proper amount of currency.

Buyer and seller are protected against forgery and fraud, and seller is protected against denial on the part of the buyer. The purchase offer is completely cloaked to third parties. And currency amounts, beyond what is already known to the seller, are hidden from  view.

To complete the transaction the seller must indicate acceptance and then the entire transaction must be published to the public blockchain with publicly verifiable proofs that the transaction was legitimate.

To accept the proposed purchase, the seller publishes the triple $(T_{buy}, C_{sell}, P_{buy})$ to the blockchain, where $T_{buy}$ was received from the buyer, and $C_{sell}$ was computed by the seller during verification. Together, these group elements form pairings that anyone can verify with the same pairing relation used by the seller:
$$ e(T_{buy},C_{sell}) = e(U,P_{buy}) \in G_T$$
This publication should be signed by the seller and posted to the blockchain. Doing so carries the signature weight of the seller as having accepted the purchase transaction. It would not have been published unless the seller accepted it. It proves that the buyer originated the transaction. And anyone can now verify that the transaction occurred. But all currency amounts remain hidden from view.

But more is needed. Nothing here prevents a buyer from claiming fraudulent values for amounts paid and change received. All we know is that their difference was matched by the seller's cost plus fees. Nor can we see if buyer and seller were colluding with fraudulent values on both sides of the transaction.

So accompanying the purchase transaction, we must provide cryptographic proofs that the individual currency values lie within legitimate ranges. In particular, the difference, $(paid - change)$, and $paid$,  $change$, $cost$, and $fees$, must all be nonnegative amounts within some range, like $[0..2^{64})$.   This is provided by Bulletproofs.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\section{Solving Equations}

\begin{frame}[fragile]
\frametitle{Solution of linear equations}
Consider,
  \begin{align*}
    3x + 2y - z  & = 1 \\
    2x - 2y + 4z  & = -2 \\
    -x + \frac{1}{2}y -z & = 0
  \end{align*}
Solution:
  \begin{align*}
    x & = 1 \\
    y & = -2 \\
    z & = -2
  \end{align*}
\end{frame}

\begin{frame}[fragile]
\frametitle{Solving using Matrices}
Let us now look at how to solve this using \texttt{matrices}
  \begin{lstlisting}
In []: A = array([[3,2,-1],
                  [2,-2,4],
                  [-1, 0.5, -1]])
In []: b = array([1, -2, 0])
In []: x = solve(A, b)
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]
\frametitle{Solution:}
\begin{lstlisting}
In []: x
Out[]: array([ 1., -2., -2.])
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
\frametitle{Let's check!}
\begin{small}
\begin{lstlisting}
In []: Ax = dot(A, x)
In []: Ax
Out[]: array([ 1.00000000e+00,  -2.00000000e+00,
              -1.11022302e-16])
\end{lstlisting}
\end{small}
\begin{block}{}
The last term in the matrix is actually \alert{0}!\\
We can use \texttt{allclose()} to check.
\end{block}
\begin{lstlisting}
In []: allclose(Ax, b)
Out[]: True
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
\frametitle{\texttt{roots} of polynomials}
\begin{itemize}
\item \texttt{roots} function can find roots of polynomials
\item To calculate the roots of $x^2-5x+6$
\end{itemize}
\begin{lstlisting}
  In []: coeffs = [1, -5, 6]
  In []: roots(coeffs)
  Out[]: array([3., 2.])
\end{lstlisting}
\vspace*{-.2in}
\begin{center}
\includegraphics[height=1.6in, interpolate=true]{images/roots}
\end{center}
\end{frame}

\begin{frame}[fragile]
\frametitle{SciPy: \texttt{fsolve}}
\begin{small}
\begin{lstlisting}
  In []: from scipy.optimize import fsolve
\end{lstlisting}
\end{small}
\begin{itemize}
\item Finds the roots of a system of non-linear equations
\item Input arguments - Function and initial estimate
\item Returns the solution
\end{itemize}
\end{frame}

\begin{frame}[fragile]
\frametitle{\texttt{fsolve} \ldots}
Find the root of $sin(z)+cos^2(z)$ nearest to $0$
\begin{lstlisting}
In []: def g(z):
 ....:     return sin(z)+cos(z)*cos(z)

In []: fsolve(g, 0)
Out[]: -0.66623943249251527
\end{lstlisting}
\begin{center}
\includegraphics[height=2in, interpolate=true]{images/fsolve}
\end{center}
\end{frame}

\section{ODEs}

\begin{frame}[fragile]
\frametitle{Solving ODEs using SciPy}
\begin{itemize}
\item Consider the spread of an epidemic in a population
\item $\frac{dy}{dt} = ky(L-y)$ gives the spread of the disease
\item $L$ is the total population.
\item Use $L = 2.5E5, k = 3E-5, y(0) = 250$
\item Define a function as below
\end{itemize}
\begin{lstlisting}
In []: from scipy.integrate import odeint
In []: def epid(y, t):
  ....     k = 3.0e-5
  ....     L = 2.5e5
  ....     return k*y*(L-y)
  ....
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
\frametitle{Solving ODEs using SciPy \ldots}
\begin{lstlisting}
In []: t = linspace(0, 12, 61)

In []: y = odeint(epid, 250, t)

In []: plot(t, y)
\end{lstlisting}
%Insert Plot
\end{frame}

\begin{frame}[fragile]
\frametitle{Result}
\begin{center}
\includegraphics[height=2in, interpolate=true]{images/epid}
\end{center}
\end{frame}

\begin{frame}[fragile]
\frametitle{ODEs - Simple Pendulum}
We shall use the simple ODE of a simple pendulum.
\begin{equation*}
\ddot{\theta} = -\frac{g}{L}sin(\theta)
\end{equation*}
\begin{itemize}
\item This equation can be written as a system of two first order ODEs
\end{itemize}
\begin{align}
\dot{\theta} &= \omega \\
\dot{\omega} &= -\frac{g}{L}sin(\theta) \\
 \text{At}\ t &= 0 : \nonumber \\
 \theta = \theta_0(10^o)\quad & \&\quad  \omega = 0\ (Initial\ values)\nonumber
\end{align}
\end{frame}

\begin{frame}[fragile]
\frametitle{ODEs - Simple Pendulum \ldots}
\begin{itemize}
\item Use \texttt{odeint} to do the integration
\end{itemize}
\begin{lstlisting}
In []: def pend_int(initial, t):
  ....     theta = initial[0]
  ....     omega = initial[1]
  ....     g = 9.81
  ....     L = 0.2
  ....     F=[omega, -(g/L)*sin(theta)]
  ....     return F
  ....
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
\frametitle{ODEs - Simple Pendulum \ldots}
\begin{itemize}
\item \texttt{t} is the time variable \\
\item \texttt{initial} has the initial values
\end{itemize}
\begin{lstlisting}
In []: t = linspace(0, 20, 101)
In []: initial = [10*2*pi/360, 0]
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
\frametitle{ODEs - Simple Pendulum \ldots}
%%\begin{small}
\texttt{}
%%\end{small}
\begin{lstlisting}
In []: from scipy.integrate import odeint
In []: pend_sol = odeint(pend_int,
                         initial,t)
\end{lstlisting}
\end{frame}

\begin{frame}[fragile]
\frametitle{Result}
\begin{center}
\includegraphics[height=2in, interpolate=true]{images/ode}
\end{center}
\end{frame}

%% \section{FFTs}

%% \begin{frame}[fragile]
%% \frametitle{The FFT}
%% \begin{itemize}
%%     \item We have a simple signal $y(t)$
%%     \item Find the FFT and plot it
%% \end{itemize}
%% \begin{lstlisting}
%% In []: t = linspace(0, 2*pi, 500)
%% In []: y = sin(4*pi*t)

%% In []: f = fft(y)
%% In []: freq = fftfreq(500, t[1] - t[0])

%% In []: plot(freq[:250], abs(f)[:250])
%% In []: grid()
%% \end{lstlisting}
%% \end{frame}

%% \begin{frame}[fragile]
%% \frametitle{FFTs cont\dots}
%% \begin{lstlisting}
%% In []: y1 = ifft(f) # inverse FFT
%% In []: allclose(y, y1)
%% Out[]: True
%% \end{lstlisting}
%% \end{frame}

%% \begin{frame}[fragile]
%% \frametitle{FFTs cont\dots}
%% Let us add some noise to the signal
%% \begin{lstlisting}
%% In []: yr = y + random(size=500)*0.2
%% In []: yn = y + normal(size=500)*0.2

%% In []: plot(t, yr)
%% In []: figure()
%% In []: plot(freq[:250],
%%   ...:      abs(fft(yn))[:250])
%% \end{lstlisting}
%% \begin{itemize}
%%     \item \texttt{random}: produces uniform deviates in $[0, 1)$
%%     \item \texttt{normal}: draws random samples from a Gaussian
%%         distribution
%%     \item Useful to create a random matrix of any shape
%% \end{itemize}
%% \end{frame}

%% \begin{frame}[fragile]
%% \frametitle{FFTs cont\dots}
%% Filter the noisy signal:
%% \begin{lstlisting}
%% In []: from scipy import signal
%% In []: yc = signal.wiener(yn, 5)
%% In []: clf()
%% In []: plot(t, yc)
%% In []: figure()
%% In []: plot(freq[:250],
%%   ...:      abs(fft(yc))[:250])
%% \end{lstlisting}
%% Only scratched the surface here \dots
%% \end{frame}


\documentclass[12pt]{article}
\usepackage[margin=1.25in]{geometry}
\usepackage[fleqn]{amsmath}
\usepackage{textcomp}
\usepackage{gensymb}
\usepackage{amsfonts}
\usepackage{enumitem}
%\usepackage{tikz}  % Include for figures.
%\usepackage{subfiles}  % Include for subfiles.

\newcommand{\HOMEWORKNUM}{8}
\newcommand{\NAME}{D. Choi}
\newcommand{\DATE}{2020-06-02}

\title{\vspace{-2\baselineskip}MATH 225 - Homework \#\HOMEWORKNUM}
\author{\NAME}
\date{\DATE}

%\pagenumbering{gobble}  % Include for single-page document.

% Augmented matrix environment.
% https://tex.stackexchange.com/a/2238
\newenvironment{amatrix}[1]{%
  \left(\begin{array}{@{}*{#1}{c}|c@{}}
}{%
  \end{array}\right)
}

\begin{document}
\maketitle

\section*{1.}
\textit{Solve.}
\begin{align*}
	x + 2y + 3z + 5w &= 8 \\
	2x + 4y + 6z + 10w &= 16 \\
	x + y + z + w &= 10
\end{align*}
This system can be rewritten in matrix form $A\vec{v} = \vec{b}$ as
\begin{equation*}
	\begin{pmatrix}
		1 & 2 & 3 & 5 \\
		2 & 4 & 6 & 10 \\
		1 & 1 & 1 & 1
	\end{pmatrix}
	\begin{pmatrix} x \\ y \\ z \\ w \end{pmatrix}
	=
	\begin{pmatrix} 8 \\ 16 \\ 10 \end{pmatrix}
\end{equation*}
The coefficient matrix $A$ and the target vector $\vec{b}$ can be concatenated
column-wise to create the augmented matrix $(A|\vec{b})$.
\begin{equation*}
	(A|\vec{b}) =
	\begin{amatrix}{4}
		1 & 2 & 3 & 5 & 8 \\
		2 & 4 & 6 & 10 & 16 \\
		1 & 1 & 1 & 1 & 10
	\end{amatrix}
\end{equation*}
The augmented matrix $(A|\vec{b})$ can be row-reduced into reduced row-echelon
form.
\footnotesize
\begin{align*}
	\begin{amatrix}{4}
		1 & 2 & 3 & 5 & 8 \\
		2 & 4 & 6 & 10 & 16 \\
		1 & 1 & 1 & 1 & 10
	\end{amatrix}
	&\xrightarrow{\frac{1}{2}r_2 \rightarrow r_2}
	\begin{amatrix}{4}
		1 & 2 & 3 & 5 & 8 \\
		1 & 2 & 3 & 5 & 8 \\
		1 & 1 & 1 & 1 & 10
	\end{amatrix}
	\xrightarrow{r_2 - r_1 \rightarrow r_2}
	\begin{amatrix}{4}
		1 & 2 & 3 & 5 & 8 \\
		0 & 0 & 0 & 0 & 0 \\
		1 & 1 & 1 & 1 & 10
	\end{amatrix}
	\\
	&\xrightarrow{r_2 \leftrightarrow r_3}
	\begin{amatrix}{4}
		1 & 2 & 3 & 5 & 8 \\
		1 & 1 & 1 & 1 & 10 \\
		0 & 0 & 0 & 0 & 0
	\end{amatrix}
	\xrightarrow{r_2 - r_1 \rightarrow r_2}
	\begin{amatrix}{4}
		1 & 2 & 3 & 5 & 8 \\
		0 & -1 & -2 & -4 & 2 \\
		0 & 0 & 0 & 0 & 0
	\end{amatrix}
	\\
	&\xrightarrow{-r_2 \rightarrow r_2}
	\begin{amatrix}{4}
		1 & 2 & 3 & 5 & 8 \\
		0 & 1 & 2 & 4 & -2 \\
		0 & 0 & 0 & 0 & 0
	\end{amatrix}
	\xrightarrow{r_1 - 2r_2 \rightarrow r_1}
	\begin{amatrix}{4}
		1 & 0 & -1 & -3 & 12 \\
		0 & 1 & 2 & 4 & -2 \\
		0 & 0 & 0 & 0 & 0
	\end{amatrix}
\end{align*}
\normalsize
\newpage
\noindent The reduced row echelon form of $(A|\vec{b})$,
\begin{equation*}
	\begin{amatrix}{4}
		1 & 0 & -1 & -3 & 12 \\
		0 & 1 & 2 & 4 & -2 \\
		0 & 0 & 0 & 0 & 0
	\end{amatrix}
	,
\end{equation*}
is analogous to the matrix equation
\begin{equation*}
	\begin{pmatrix}
		1 & 0 & -1 & -3 \\
		0 & 1 & 2 & 4 \\
		0 & 0 & 0 & 0
	\end{pmatrix}
	\begin{pmatrix} x \\ y \\ z \\ w \end{pmatrix}
	=
	\begin{pmatrix} 12 \\ -2 \\ 0 \end{pmatrix}
\end{equation*}
and also to the system
\begin{align*}
	x -z - 3w &= 12 \\
	y + 2z + 4w &= -2
	.
\end{align*}
In this system, $z$ and $w$ are free variables. \\
Thus, the system has \boxed{\text{infinitely many solutions}}.
\\[\baselineskip]
Solutions to the system are of the form
\begin{equation*}
	\boxed{
		\begin{pmatrix}
			12 + z + 3w \\
			-2 - 2z - 4w \\
			z \\
			w
		\end{pmatrix}
	}
	,
\end{equation*}
or, equivalently,
\begin{equation*}
	\boxed{
		\begin{pmatrix} 12 \\ -2 \\ 0 \\ 0 \end{pmatrix}
		+
		\begin{pmatrix} 1 \\ -2 \\ 1 \\ 0 \end{pmatrix} z
		+
		\begin{pmatrix} 3 \\ -4 \\ 0 \\ 1 \end{pmatrix} w
	}
	.
\end{equation*}
\newpage

\section*{2.}
\textit{Include an example of appropriate matrix as you justify your responses
to the following questions.}
\begin{enumerate}[label=(\alph*)]
	\item \textit{Suppose a linear system having six equations and three
	unknowns is consistent.
	Can you guarantee that the solution is unique?
	Can you guarantee that there are infinitely-many solutions?}
	\\[\baselineskip]
	Consider the row-reduced system:
	\begin{equation*}
		\begin{pmatrix}
			0 & 0 & 0 \\
			0 & 0 & 0 \\
			0 & 0 & 0 \\
			0 & 0 & 0 \\
			0 & 0 & 0 \\
			0 & 0 & 0
		\end{pmatrix}
		\begin{pmatrix} x \\ y \\ z \end{pmatrix}
		=
		\begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}
		.
	\end{equation*}
	\boxed{\text{A unique solution cannot be guaranteed.}}
	\\[\baselineskip]
	Consider the row-reduced system:
	\begin{equation*}
		\begin{pmatrix}
			1 & 0 & 0 \\
			0 & 1 & 0 \\
			0 & 0 & 1 \\
			0 & 0 & 0 \\
			0 & 0 & 0 \\
			0 & 0 & 0
		\end{pmatrix}
		\begin{pmatrix} x \\ y \\ z \end{pmatrix}
		=
		\begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}
		.
	\end{equation*}
	The only solution to such a system is
	\begin{equation*}
		\begin{pmatrix} x \\ y \\ z \end{pmatrix}
		=
		\begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}
	\end{equation*}
	Thus, \boxed{\text{infinitely-many solutions cannot be guaranteed}}.
	\newpage
	\item \textit{Suppose that a linear system having three equations and six
	unknowns is consistent.
	Can you guarantee that the solution is unique?
	Can you guarantee that there are infinitely-many solutions?}
	\\[\baselineskip]
	Consider the row-reduced system:
	\begin{equation*}
		\begin{pmatrix}
			0 & 0 & 0 & 0 & 0 & 0 \\
			0 & 0 & 0 & 0 & 0 & 0 \\
			0 & 0 & 0 & 0 & 0 & 0
		\end{pmatrix}
		\begin{pmatrix} a \\ b \\ c \\ d \\ e \\ f \end{pmatrix}
		=
		\begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}
		.
	\end{equation*}
	\boxed{\text{A unique solution cannot be guaranteed.}}
	\\[\baselineskip]
	For there to be a unique solution, there must be a pivot for each column
	of the coefficient matrix in the row-reduced augmented matrix. This is not
	possible when there are more columns than rows in the coefficient matrix.
	Thus, \boxed{\text{infinitely-many solutions are guaranteed}}.
	\item \textit{Suppose that a linear system is consistent and has a unique
	solution.
	What can you guarantee about the pivot positions in the augmented matrix?}
	\\[\baselineskip]
	For a row-reduced augmented matrix of a consistent linear system with a
	unique solution, \boxed{\text{a pivot must exist for each column of the
	coefficient matrix}}.
\end{enumerate}

\end{document}