\section{Introduction to error propagation}
Before studying the model, we briefly recall some notions on propagation of measurement errors. Accordingly with \cite{book:ivm}, the size is a property of an object that could be expressed by a number and a reference, i.e. it is measurable. Physical size measurement is generally followed by the error estimation associated with it, specially when they are indirect measure. In the general case, given the function $y = f(x)$, we can generalized its error as
  \begin{equation}
    \sigma_f = \begin{vmatrix}
      \frac{df}{dx}
    \end{vmatrix}_{x = x_0}
    \sigma_x
    \label{eq:er_prop_1}
  \end{equation}
If $y = f(\bar{x})$, function of $k$ variables, Equation \ref{eq:er_prop_1} can be generalize as:
  \begin{equation}
    \sigma_f = \sum_{j = 1}^k \begin{vmatrix}
      \frac{\partial f}{\partial x_j}
    \end{vmatrix}_{x = x_0}
    \sigma_x
    \label{eq:er_prop_2}
  \end{equation}
The propagation of the maximum errors by using the differential, is based on the assumption that infinitesimal variations of the variables are given by the respective errors. If we want to estimate the maximum error for $y$, it is appropriate to sum up all the terms consistently, i.e. taking the partial derivative modules.


\subsection{The probability function}

For all events \(E\) in \(F\), the probability function \(P\) is defined.

\subsection{Measure space}

This gives us the following measure space:

\((\Omega, F, P)\)

\subsection{First Kolmogorov axiom}

First axiom

The probability of all events is a non-negative real number.

\(\forall E \in F [(P(E)\ge 0)\land (P(E)\in \mathbb{R})]\)

\subsection{Second Kolmogorov axiom}

The probability of one of the elementary events occuring is \(1\).

The probability of the outcome set is \(1\).

\(P(\Omega )=1\)

\subsection{Third Kolmogorov axiom}

The probability of union for mutually exclusive events is:

\(P(\cup^\infty_{i=1}E_i)=\sum_{i=1}^\infty P(E_i)\)


Among the many methods developed for machine learning, neural networks, and especially deep neural networks, are among the most popular. Neural networks were suggested already in 1943 \cite{McCulloch1943} and have had many renaissances since. Currently we are experiencing such a renaissance, but in contrast to earlier periods of resurfaced interest, we now have the computer power to use neural nets efficiently.

A neural net bases itself loosely upon the biological model of neurons communicating together in the brain. A neuron cell contains most of what a normal cell contains, but it also has a long tail called an axon and some antenna like extension called dendrites. The axon of one cell can extend quite far and attach to some of the dendrites of another neural cell. Thus, the biological neural net consists of neural cells receiving input through their dendrites from many other cells and sending output through one output \citep[p. 257]{Geron}.

The computed neural networks works in a similar way. We construct "neurons" or "nodes" which are ordered in different layers where each neuron in one layer is connected to all neurons in the next layer. Initially, we start with an input layer which we feed information. Following this initial layer we have one or many hidden layers before we reach the output layer. A neural network with two or more hidden layers are called deep neural networks \citep[p. 263]{Geron}. Each neuron contains an activation function which determines the strength of the output. In the early days, a step function was used as the activation function. However, one has found that the use of a activation function with a gradient, such as the logistic function used in logistic regression, gives a better neural net. This is due to the fact that we now can apply gradient descent when optimizing the neural net which is discussed below.

To activate a neuron, it needs an input. This input is provided by all the neurons in previous layers through "wires" connecting the neurons (think of the axon to a dendrite). Each of these "wires" is weighted and all connections between one layer and the next is affected by a bias term. Thus, the output of a neuron is given as \citep[p. 260]{Geron}
\begin{equation}
    a = \sigma(z)
    \label{eqT:activation}
\end{equation}
with
\begin{align}
    z = \bm{w}^T \bm{x} + b
\end{align}
where $a$ is the output, $\bm{w}$ are the weights, $\bm{x}$ are the inputs and $b$ is the bias term. Note that if we had used a step function instead of the logit(sigmoid) function in the equation above, the neuron would either give an output of 1 or nothing, i.e. 0. When we instead use the sigmoid function, the output can be in the range of 0 to 1.

Once all neurons have been calculated in a layer, we can move on to the next and continue until we reach our output layer. Each layer can have as many neurons as the user wants. Optimizing the number of neurons in each layer is an art and requires both experience and a bit of luck. The output layer needs one neuron for each class we wish to identify.

After initial calculation of the outer layer, you will most likely have an answer that is completely rubbish. It is clear that we have to optimize the neural net. As each neurons activation function is calculated using equation \eqref{eqT:activation}, we see that we can optimize the weights between the neurons and the bias between the layers. This is done through a method called backwards propagation where one uses the cost function to identify the magnitude of the error (the cost) of a neural net and then one goes backwards through the neural net to update the weights and biases.

The backpropegation is then summed up as following(heavily influenced by the work of \citet{Nielsen}),
\begin{itemize}
    \item Compute the output error vector for the final layer (L) given by
    \begin{equation*}
        \delta^L = \nabla_a \mathcal{C} \odot \sigma^{'}(\bm{w}^T \cdot \bm{x} + b)
    \end{equation*}
    where $\mathcal{C}$ is the cost function and $\nabla_a$ is a vector who has components that are the partial derivatives $\frac{\partial \mathcal{C}}{\partial a_j^L}$ where $a_j$ is the j'th output found by using equation \eqref{eqT:activation} for $a$.
    \item Go back through all the previous layers l = L-1, L-2, ..., 2 and compute
    \begin{equation*}
        \delta^l ((\bm{w}^{l + 1})^T \delta^{l + 1}) \odot \sigma^{'}(\bm{w}^T \cdot \bm{x} + b)
    \end{equation*}
    this is where we back propagate the error.
    \item Finally, find the gradient of the cost function for the two parameters we want to change, $\bm{w}$ and $b$ by:
    \begin{align*}
        \frac{\partial \mathcal{C}}{\partial \bm{w}_{jk}^l} &= a_k^{l-1}\delta_{j}^{l} \\
        \frac{\partial \mathcal{C}}{\partial b_{j}^l} &= \delta_{j}^{l}
    \end{align*}
    where $k$ indicates the column of $\bm{w}^l$ as $j$ indicates the row.
\end{itemize}
Once all layers have been adjusted through back propagation, one can run through the whole network again and repeat the process.

In order to understand parts of the back propagation algorithm for $L$ layers and a general layer activation function $a^L(z)$, can take a quick look at how to proceed in deriving it. We start with some cost function $\CO(w^L, y)$, where $w^L$ is the output layer weights and $y$ is the \textit{true} values which we want to approach. In order to move towards optimal weights(and biases), we can perform a gradient descent by subtracting the gradient of the cost function w.r.t. the weights and biases. We begin by finding the gradient of the $\CO$ for $W^L$. Using the chain rule we get,
\begin{align}
    \frac{\partial \mathcal{C}}{\partial w^L} &= \frac{\partial z^L}{\partial w^L} \frac{\partial a^L}{\partial z^L} \frac{\partial \CO}{\partial a^L}
    \label{eq:mlp-gradient-chain}
\end{align}
From this we simply have to find each of the partial derivatives. Without specifying the cost function and the activation layer, we can only state that the first partial derivative is given as,
\begin{align*}
\frac{\partial z^L}{\partial w^L} = a^{L-1}
\end{align*}
From this, we quickly see that we need to properly defined the cost function in relation to the $\partial^L$ as we encountered earlier.

\subsubsection{Mean square error(MSE) cost function}
A common cost function may be the quadratic loss function(MSE) given by.
\begin{equation}
    \mathcal{C}_\mathrm{MSE} = \frac{1}{2N} \sum\limits_{j} (y_j - a_j^L)^2
\end{equation}
where $N$ are the total numbers of outputs in $j$ and $y$ are the true answers. For a single training sample, this becomes,
\begin{align}
    \mathcal{C}_\mathrm{MSE} = \frac{1}{2} (a^L - y)^2
    \label{eq:mse-mlp-cost}
\end{align}
We can then insert this expression into the cost chain rule expression \eqref{eq:mlp-gradient-chain}, and we get that the partial derivative
\begin{align*}
    \frac{\partial \mathcal{C}_\mathrm{CE}}{\partial w^L} &= a^{L-1}(a^L - y)\sigma'(z^L) \\
    &= a^{L-1} \delta^L
\end{align*}
If we now take the derivative for layer $L-1$, we get
\begin{align*}
    \frac{\partial \mathcal{C}_\mathrm{CE}}{\partial w^{L-1}} &= \frac{\partial a^{L-1}}{\partial w^{L-1}} \delta^L \\
    &= \frac{\partial z^{L-1}}{\partial w^{L-1}} \frac{\partial a^{L-1}}{\partial z^{L-1}} \delta^L \\
    &= a^{L-2}\sigma'(z^{L-1})\delta^L
\end{align*}

\subsubsection{Cross entropy(CE) cost function}
The CE cost function is given as
\begin{align}
    \mathcal{C}_\mathrm{CE} = - \sum_{i=1} y_i \log a^L
    \label{eq:ce-mlp-cost}
\end{align}
with its derivative as
\begin{align}
    \frac{\partial \mathcal{C}_\mathrm{CE}}{\partial a^L} = - \sum_{i=1} \frac{y_i} {a^L}
    \label{eq:ce-mlp-cost-der}
\end{align}
If we use softmax as the layer output activation function,
\begin{align}
    z_i = \frac{\exp{z_i}}{\sum_k \exp(z_k) }
    \label{eq:softmax}
\end{align}
with $i,k$ being output classes and its derivative,
\begin{align}
    z_i = z_i(\delta_{ij} - z_j),
\end{align}
we can write the initial gradient $\delta^L$ as
\begin{align}
    \delta^L = y - a^L
\end{align}
If we set the number of output classes to be 2 such that we get binary classification, we get the cost functions for logistic regression,
\begin{align*}
    \mathcal{C}_{lg} = - \left( y \log p + (1 - y)\log (1 - p)\right)
\end{align*}
for one sample.

\subsubsection{Activation layers} \label{sec:layer-acts}
The hidden layer activations can greatly affect the outcome of the neural network. We will focus on four different hidden layer activations, the first being \textbf{sigmoidal activation}.
\begin{align}
    \sigma (z) = \frac{1}{1 + \exp (-z)}
    \label{eq:act-sig}
\end{align}
with its derivative
\begin{align}
    \sigma'(z) = \sigma(z)(1 - \sigma(z))
    \label{eq:act-sig-der}
\end{align}
The \textbf{hyperbolic tangens} activation function is given as
\begin{align}
    \sigma_\mathrm{tanh}(z) = \tanh(z)
    \label{eq:act-tanh}
\end{align}
with its derivative
\begin{align}
    \sigma'_\mathrm{tanh}(z) = 1 - \tanh^2(z)
    \label{eq:act-tanh-der}
\end{align}
The \textbf{relu} or rectifier activation is given as,
\begin{align}
    \sigma_\mathrm{relu}(z) =
    \begin{cases}
        z & \text{if } z \geq 0 \\
        0 & \text{if } z < 0 \\
    \end{cases}
    \label{eq:act-relu}
\end{align}
with its derivative
\begin{align}
    \sigma'_\mathrm{relu}(z) =
    \begin{cases}
        1 & \text{if } z \geq 0 \\
        0 & \text{if } z < 0 \\
    \end{cases}
    \label{eq:act-relu-der}
\end{align}
The \textbf{Heaviside} activation function is given as
\begin{align}
    \sigma_\mathrm{Heaviside}(z) =
    \begin{cases}
        1 & \text{if } z \geq 0 \\
        0 & \text{if } z < 0 \\
    \end{cases}
    \label{eq:act-heaviside}
\end{align}
with its derivative
\begin{align}
    \sigma'_\mathrm{Heaviside}(z) = 0
    \label{eq:act-heaviside-der}
\end{align}

\subsubsection{Learning rate}
When updating the weights and biases with SGD(Stochastic gradient descent), we did so by a learning rate parameter $\eta$. There are several way to define $\eta$, with the simplest one having $\eta=\mathrm{constant}$. Another option is one that is inversely decreasing as a function of the epochs. That is, for a given epoch $i_e$ out a total $N_\mathrm{epochs}$, we set the learning rate as
\begin{align}
    \eta(t_e) = \eta_0 (1 - \frac{i_e}{1 + N_\mathrm{epochs}})
    \label{eq:inverse-eta}
\end{align}
This will force the step size to decrease toward 0 as we close in on the maximum number of epochs $N_\mathrm{epochs}$.

\subsubsection{Weight initialization} \label{sec:nn-weights}
When initializing weights and biases, we will look at two ways of how this can be done. The first is through a gaussian distribution, $\mathcal(0, 1)$ which we will call \textit{large}, as the biases will have large, spread-out distribution.

Then, we will use a gaussian distribution but divided by the number of training samples, $\mathcal(0, 1/{N_\mathrm{train}})$, dubbing that one to be called \textit{default}, as this is the one we will by default use in our neural network.

The effect of these two is essentially shrinking in the initial search space, and we should expect them to converge at large epoch times.

\subsubsection{Measuring the performance}
The performance of a neural network(or any classifier), can in its simplest form be measured by the accuracy, which is defined as
\begin{align}
    \mathrm{Accuracy} = \frac{\sum^{n}_{i=1}I(t_i = y_i)}{n},
    \label{eq:mlp-accuracy}
\end{align}
where $n$ is the number of samples we are testing against, $I$ is the indicater function, which returns 1 if the prediction $t_i$ equals the true values $y_i$.


\documentclass{article}[12pt]

\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{mathrsfs}
\usepackage{bm}

\begin{document}

\section*{The Heat Equation}
We postulate a fixed bar of a single and constant-density material, with length $L$; a uniform cross sectional area $A$ and a diffusivity of $\alpha = \frac{\kappa}{c\rho}$ (a parameter which depends on the rod material and its thermal conductivity/heat capacity). The bar is insulated on the outer surface and we assume at any point $x$ across the bar the temperature of the cross section is constant, but may be different to that of any other cross-section at a different point $x'$:\par
\begin{align}\label{eq:heateq}
 \frac{\partial \phi}{\partial t} &= \alpha \frac{\partial^2\phi}{\partial x^2} , &  0\leq x \leq L, &&  t\geq 0
\end{align}
We set the Dirichlet boundary conditions and the initial condition:\par
\begin{align}
  \phi(0,t) = \phi_0, & & \phi(L,t) = \phi_L, & & \phi(x,0) = f_0(x)
\end{align}
This is represented in Fig. \ref{fig:bar_representation}.\par
\begin{figure}[h]
  \centering
  \includegraphics[scale = 0.5]{bar_representation.png}
  \caption{Graphical representation of bar conditions.}
  \label{fig:bar_representation}
\end{figure}
We will use the Finite Difference (FD) method to solve Eq. \eqref{eq:heateq} numerically with a discrete approximation. We understand that the function $\phi$ and its derivatives $\partial_t\phi$, $\partial_x\phi$ and $\partial_{xx}\phi$ are all continuous in $x$ and $t$.
\subsection*{Continuous Form}
We imagine that the entire length of the bar $L$ is split in to a finite number $N$ of mesh points of equal length $\Delta x = L/N$. We then evaluate the heat at the boundary of each of these mesh points given the relevant system boundary conditions to evaluate the entire temperature profile across the whole bar.
\begin{equation}
  \lim_{N\rightarrow \infty}\frac{L}{N} \approx\partial x
\end{equation}
So we see:
\begin{equation}\label{eq:continuous_forward}
\lim_{\Delta x \rightarrow 0 } \frac{\phi(x+\Delta x) - \phi(x)}{\Delta x} \approx \frac{\partial \phi}{\partial x}
\end{equation}
This is a continuous form of the forward-difference numerical approximation for the PDE in $x$. We can see that the FD method is in fact very similar to a forward Euler method for solving non-first-order PDEs. In actuality we may not set $\Delta x = 0$ so a truncation error will exist in this limit. We  control the extent of this error by increasing the number of nodes present on the mesh.\par
We may also form backward-difference, centre-difference and second order approximations for the solution of this equation. Here, we simply present the first-order forward-difference method.
\subsection*{Discrete Form}
We may convert \eqref{eq:continuous_forward} in to a discrete form upon the mesh $(0,L) = (0,N\Delta x)$:
\begin{equation}\label{eq:discrete_forward_x}
  \frac{\partial \phi}{\partial x}\bigg\rvert_{x_i} = \lim_{\Delta x \rightarrow 0} \frac{\phi_{i+1} - \phi_i}{\Delta x}
\end{equation}
For $\Delta x = \Delta x_i = x_{i+1} - x_i$, given that all nodes on the mesh are equal in length.\par
Let us now look at the discretised time-derivative of the function $\phi_i^{m+1}$ at a point in time $t_{m+1}$:
\begin{equation}\label{eq:discrete_forward_t}
\frac{\partial \phi}{\partial t}\bigg\rvert_{t_{m+1},x_i} = \lim_{\Delta t \rightarrow 0}\frac{\phi_i^{m+1} - \phi_i^{m}}{\Delta t}
\end{equation}
For $ \Delta t = \Delta t_{m+1} = t_{m+1}  - t_{m}$ where we uniformly step through time.
from \eqref{eq:discrete_forward_x} we may now evaluate:
\begin{equation}\label{eq:discrete_forward_x_2o}
\frac{\partial^2 \phi}{\partial x^2}\bigg\rvert_{x_i,t_m} = \lim_{\Delta x \rightarrow 0} \frac{\phi_{i+1}^m - 2\phi_{i}^m + \phi_{i-1}^m}{\Delta x^2}
\end{equation}
So we now have a generalised forward-time/centred-space discrete form of \eqref{eq:heateq} by substituting \eqref{eq:discrete_forward_t} and \eqref{eq:discrete_forward_x_2o}:
\begin{equation}
\phi_i^{m+1} = \phi_i^m + \frac{\alpha\Delta t}{\Delta x^2} \left(\phi_{i+1}^m -2\phi_i^m + \phi_{i-1}^m\right)
\end{equation}
Where:
\begin{equation}
q = \frac{\alpha\Delta t}{\Delta x ^2}
\end{equation}
We may now solve this computationally, given the boundary conditions from \eqref{eq:heateq}, for each discrete point. Alternatively, we may form a matrix of all points.
\begin{equation}
\bm{\phi}^m+1 = A \bm{\phi}^{m}
\end{equation}
Where:
\begin{equation}
  A =
  \begin{bmatrix}
    1&0&0&\hdots&0&0\\
    q&(1-2q)&q&0&\ddots&0\\
    0&q&(1-2q)&q&\ddots&0\\
    0&\ddots&\ddots&\ddots&\ddots&0\\
    0&\ddots&q&(1-2q)&q&0\\
    0&\ddots&0&q&(1-2q)&q\\
    0&0&\hdots&0&0&1
  \end{bmatrix}
\end{equation}
So, we may evalute the column vector $\bm{\phi}^{0}$ across the length $L$ at $t=0$ given the intial conditions, and then step forward through time to calculate a general solution for any combination of boundary conditions and physical parameters.
\end{document}


\chapter{Fibonacci Numbers}

In mathematics, the Fibonacci numbers or Fibonacci series or Fibonacci sequence are the numbers in the following integer sequence:
%
\begin{equation*} % example of unnumbered equations
0, 1, 1, 2, 3,5,8,13,21,34,55,89,144,\ldots
\end{equation*}

By definition, the first two numbers in the Fibonacci sequence are 0 and 1, and each subsequent number is the sum of the previous two. In mathematical terms, the sequence $F_n$ of Fibonacci numbers is defined by the recurrence relation
%
\begin{equation}
F_n = F_{n-1} + F_{n-2},
\end{equation}
%
with seed values
%
\begin{equation}
F_0 = 0, F_1 = 1.
\end{equation}

The Fibonacci sequence is named after Leonardo Fibonacci. His 1202 book Liber Abaci introduced the sequence to Western European mathematics, although the sequence had been described earlier in Indian mathematics. \cite{Goonatilake:1998} By modern convention, the sequence begins either with $F_0 = 0$ or with $F_1 = 1$. The Liber Abaci began the sequence with $F_1 = 1$, without an initial 0.

\section{Origins}

The Fibonacci sequence appears in Indian mathematics, in connection with Sanskrit prosody. \cite{Singh:1985} In the Sanskrit oral tradition, there was much emphasis on how long (L) syllables mix with the short (S), and counting the different patterns of L and S within a given fixed length results in the Fibonacci numbers; the number of patterns that are m short syllables long is the Fibonacci number $F_{m + 1}$.
\citet{Goonatilake:1998} writes that the development of the Fibonacci sequence ``is attributed in part to Pingala (200 BC), later being associated with Virahanka (c.~700 AD), Gopala (c.~1135), and Hemachandra (c.~1150)''.

\begin{figure}[hbt!]
\centering
\includegraphics[width=.4\textwidth]{314px-Liber-abbaci-magliab-f124r}
\caption{A page of Fibonacci's Liber Abaci a very long title e just to prove my point more and more and more}
\source{Heinz L\"{u}neburg, Leonardi Pisani Liber Abaci oder Lesevergn\"{u}gen eines Mathematikers}
\end{figure}

\section{List of Fibonacci Numbers}

The first 11 Fibonacci numbers $F_n$ for $n = 0, 1, 2, \ldots, 10$ are:

\begin{table}[hbt!]\centering
\caption{First 11 Fibonacci Numbers for $n=0,1,\ldots$}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|}
\hline
$F_0$ & $F_1$ & $F_2$ & $F_3$ & $F_4$ & $F_5$ & $F_6$ & $F_7$ & $F_8$ & $F_9$ & $F_{10}$\\
\hline
0 & 1 & 1 & 2 & 3 & 5 & 8 & 13 & 21 & 34 & 55 \\
\hline
\end{tabular}
\end{table}

The sequence can also be extended to negative index n using the re-arranged recurrence relation
%
\begin{equation}
F_{n-2} = F_n - F_{n-1},
\end{equation}
%
which yields the sequence of ``negafibonacci'' numbers satisfying
%
\begin{equation}
F_{-n} = (-1)^{n+1} F_n.
\end{equation}
%
Thus the bidirectional sequence is
\begin{table}[hbt!]\centering
\caption{Bidirectional Fibonacci Numbers sequence}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|}
\hline
$F_{-5}$ & $F_{-4}$ & $F_{-3}$ & $F_{-2}$ & $F_{-1}$ & $F_0$ & $F_1$ & $F_2$ & $F_3$ & $F_4$ & $F_5$ \\\hline
5 & $-3$ & 2 & $-1$ & 1 & 0 & 1 & 1 & 2 & 3 & 5\\\hline
\end{tabular}
\end{table}

\citet{Rohl:1989} gives an account of how Fibonacci numbers can be computed efficiently.

\section{Applications}

\subsection{In Computation}

Fibonacci numbers have wide applications in mathematics as well as computer science:

\begin{itemize}
\item The Fibonacci numbers are important in the computational run-time analysis of Euclid's algorithm to determine the greatest common divisor of two integers: the worst case input for this algorithm is a pair of consecutive Fibonacci numbers.

\item Yuri Matiyasevich was able to show that the Fibonacci numbers can be defined by a Diophantine equation, which led to his original solution of Hilbert's tenth problem.

\item The Fibonacci numbers are also an example of a complete sequence. This means that every positive integer can be written as a sum of Fibonacci numbers, where any one number is used once at most.

\item Moreover, every positive integer can be written in a unique way as the sum of one or more distinct Fibonacci numbers in such a way that the sum does not include any two consecutive Fibonacci numbers. This is known as Zeckendorf's theorem, and a sum of Fibonacci numbers that satisfies these conditions is called a Zeckendorf representation. The Zeckendorf representation of a number can be used to derive its Fibonacci coding.

\item Fibonacci numbers are used by some pseudorandom number generators.

\item Fibonacci numbers are used in a polyphase version of the merge sort algorithm in which an unsorted list is divided into two lists whose lengths correspond to sequential Fibonacci numbers -- by dividing the list so that the two parts have lengths in the approximate proportion $\varphi$. A tape-drive implementation of the polyphase merge sort was described in The Art of Computer Programming.

\item Fibonacci numbers arise in the analysis of the Fibonacci heap data structure.

\item The Fibonacci cube is an undirected graph with a Fibonacci number of nodes that has been proposed as a network topology for parallel computing.

\item A one-dimensional optimization method, called the Fibonacci search technique, uses Fibonacci numbers.

\item The Fibonacci number series is used for optional lossy compression in the IFF 8SVX audio file format used on Amiga computers. The number series compands the original audio wave similar to logarithmic methods such as $\mu$-law.

\item Since the conversion factor 1.609344 for miles to kilometers is close to the golden ratio (denoted $\varphi$), the decomposition of distance in miles into a sum of Fibonacci numbers becomes nearly the kilometer sum when the Fibonacci numbers are replaced by their successors. This method amounts to a radix 2 number register in golden ratio base $\varphi$ being shifted. To convert from kilometers to miles, shift the register down the Fibonacci sequence instead.
\end{itemize}

\subsection{In Nature}

Fibonacci sequences appear in biological settings, in two consecutive Fibonacci numbers, such as branching in trees, arrangement of leaves on a stem, the fruitlets of a pineapple, the flowering of artichoke, an uncurling fern and the arrangement of a pine cone, and the family tree of honeybees. However, numerous poorly substantiated claims of Fibonacci numbers or golden sections in nature are found in popular sources, e.g.,~relating to the breeding of rabbits in Fibonacci's own unrealistic example, the seeds on a sunflower, the spirals of shells, and the curve of waves.

A model for the pattern of florets in the head of a sunflower was proposed by H.~Vogel in 1979. \cite{Vogel:1979} This has the form
\begin{equation}
\theta = \frac{2\pi}{\phi^2} n,\  r = c \sqrt{n}
\end{equation}
where $n$ is the index number of the floret and $c$ is a constant scaling factor; the florets thus lie on Fermat's spiral.

\begin{figure}[hbt!]\centering
\includegraphics[width=.5\textwidth]{FibonacciChamomile}

\caption{Yellow Chamomile head}
\end{figure}


\chapter{Formal specification of the gameplay}
\label{chap:formal}

\section{Grid universe}

The universe is made of a $m$ (horizontally) by $n$ (vertically) array of cases.
The cases are delimited by a grid.
We represent those with planar graphs:
\begin{itemize}
    \item The array of cases: $G_A = (V_A, E_A)$, with
        \begin{align*}
            V_A &= \left\{v_{a,ij} : 0 \le i < m, 0 \le j < n\right\} \\
            E_A &= \left\{\{v_{a,i_1j_1}, v_{a,i_2,j_2}\} :
            |i_1-i_2| + |j_1-j_2| = 1 \right\}
        \end{align*}
    \item The grid: $G_G = (V_G, E_G)$, with
        \[V_G = \left\{v_{g,ij} : 0 \le i \le m, 0 \le j \le n\right\} \]
        \[
            \begin{split}
                E_G = \{
                    \{v_{g,i_1j_1}, v_{g,i_2,j_2}\} :
                    & |i_1-i_2| + |j_1-j_2| = 1; \\
                    & 0 < i_1 < m; 0 < j_1 < n; \\
                    & 0 \le i_2 \le m, 0 \le j_2 \le n
                \}
            \end{split}
        \]
\end{itemize}
The graphs are depicted in figure~\ref{fig:graphs1}.
\begin{figure}
    \begin{center}
        \includegraphics[width=0.3\textwidth]{img/graphs_universe_draft.png}
    \end{center}
    \caption{Representation of the graphs: $G_A$ is in green, $G_G$ in blue
        (the dashed blue lines are not edges of the graph) and the global border
        in black.
    }
    \label{fig:graphs1}
\end{figure}
On this representation, all the edges of $G_G$ intersect
edges of $G_A$, which gives a correspondance bijection $c:E_G \rightarrow E_A$,
formally defined as follows
\begin{align*}
    c(\{v_{g,i,j}, v_{g,i+1,j}\}) &= \{v_{a,i,j-1}, v_{a,i,j}\} \\
    c(\{v_{g,i,j}, v_{g,i,j+1}\}) &= \{v_{a,i-1,j}, v_{a,i,j}\}.
\end{align*}

A border $B$ is a path in $G_G$, which starts at $v_{g,0j}$ and ends at
$v_{g,m+1,j'}$.

Due to the structure of both graphs, the edges of the border corresponds
(by $c$) to a set for edges which form a cut of $G_A$. This cut defines
two subgraphs: the free space $F$ (containing cases $v_{a,i,n-1}$) and
the structure $S$ (containing cases $v_{a,i0}$).

The following properties hold for any border:
\begin{itemize}
    \item the free space contains the upper row $v_{a,i,n-1}$;
    \item the structure contains the lower row of cases $v_{a,i0}$;
    \item the structure is connex;
    \item the free space is connex.
\end{itemize}

$F$ and $S$ are complementary, there is thus an trivial bijection between
the set of free spaces and the set of structures.

A free space, structure pair  which satisfies the preceding properties
is said to be well-formed.

A well-formed free space, structure pair always corresponds to a border:
taking the edges in $G_G$ corresponding to the edges of $G_A$
linking the structure and the free space gives
a set of edges which constitue a border.
There is thus a bijection between borders and well-formed free spaces (and structures).

\section{Players}

There is a global well-formed structure which is the same for all the players.
For all cases $v_{a,i,j}$ in the global structure, the condition $j < n - N_p$
must hold, where $N_p$ is the number of players.

The structure is thus conceptually dense (not holes in it). But this
does not imply anything on the display: the inner cases of the structure
can be anything, this doesn't impact the gameplay.

Each player $p_k, 0\le k < N_p$ is an entity, that occupies a case $v_{a,p_k}$.
Cases occupied by players must be in the global free space (ie. outside the global
structure). Only one player may occupy a given place at a given time.

Each player must be hooked to a neighbour case, which must be either occupied by
the global structure, either by another player. In any set of players, there must
be at least one player hooked to a case occupied by the global structure.

The hook point is the edge linking the case occupied by the player and the case
it is hooked to (or the corresponding edge through the bijection $c$).

\section{Movements}

At any time, each player has its own free space, structure, border triple $(F_k, S_k, B_k)$.
The free space is the largest connex component that contains the cases of the upper row,
but no cases of the global structure, and no cases occupied by other players.
The definition of its structure and border follows from that.

From the border $B_k$, extracting the edges gives a sequence of hook points $(h_k)_i$.
The player is always hooked to a point in this sequence.
To each hook point corresponds a unique case that can be occupied by the player
(the converse is not true).

When a player moves right, its new hook point is determined in two steps:
\begin{enumerate}
    \item Finding new case: from the current hook point, $(h_k)_i$ is traversed
        (increasing $i$), until the traversed hook point corresponds to a case
        different of the current case of the player. If no such hook point is
        found before the end of the sequence, the move is impossible.
    \item Deciding the new hook point: if possible, the player hooks to the
        global structure. To do this, the list is traversed, from the hook
        point found at previous step. One switches to the next hook point if
        the next hook point corrsponds to the same case as the same hook point
        and if the current hook point is hooking on another player. Otherwise,
        the traversing stops.
\end{enumerate}

\section{Mask}

The mask is a set of cases which must include all the cases of the global
structure. The connex component of the mask containing the global structure
must also include at least $N_p$ cases of the global free space.


\subsection{Plane Laminas}
\begin{definition}
	A plane lamina is an idealized 2D (0 thickness) object with mass that occupies a region $D \subset \mathbb{R}^2$.
\end{definition}

\noindent
Some questions one may ask about a plane lamina are "What is the total mass?" and "Where is the center of mass?".\\
We can think of the mass as
\begin{equation*}
	M = \iint\limits_{D}{\sigma(x,y)\mathrm{d}A}
\end{equation*}
where $\sigma(x,y)$ is the mass density of the lamina at $(x,y)$.\\
The center of mass is
\begin{equation*}
	\bar{x} = \frac{M_y}{M} = \frac{\iint\limits_{D}{x\sigma(x,y)\mathrm{d}A}}{M} \text{ and } \bar{y} = \frac{M_x}{M} = \frac{\iint\limits_{D}{y\sigma(x,y)\mathrm{d}A}}{M}
\end{equation*}
where $M_x$ is the moment about the $x$-axis and $M_y$ is the moment about the $y$-axis.


\chapter{E2E Evaluation Metrics}\label{ch:metrics}
\section{Audio Metrics}
\subsection{SNR - Signal to Noise Ratio}
The signal-to-noise ratio (SNR) metric evaluates how
distinct the desired signal is out of the overall noise.

Let \(y(t)\) denote a mixed time-domain signal consisting of
the desired speech signal, and some interferences
referred to as noise.
That mixture is given by:
\begin{align}
    y(t) = x(t) + n(t)
\end{align}

Where \(x(t)\) and \(n(t)\) denote the speech signal and the
interference noise.

Ideal speech separation of the mixture is characterized by
a perfect match between the predicted speech signal, \(\widehat{x}(t)\),
and the original (reference) speech signal \(x(t)\).
% Separating the speech out of the mixture,
% the predicted speech signal, \(\widehat{x(t)}\), has to match the
% original (target) speech signal \(x(t)\).

Such a problem can be modeled and optimized
by the MSE (L2) (Mean Square Error) loss function as follows:
\begin{align}
    \ell(\widehat{x}, x) & = \sum_{t=0}^{T-1} \left[\widehat{x}(t) - x(t)\right]^{2} \\
    & = \sum_{t=0}^{T-1} |r(t)|^{2}
\end{align}

The term \(\sum_{t} |r(t)|^{2}\)
is the total energy of the residual error between
the predicted signal and the target speech,
which translates to additive noise.

First, let's break \(\widehat{x}(t)\) to its fundamental components\cite{1643671}.
\begin{align}
    \widehat{x}(t) & = x_{_{s}} + e_{_{noise}} + e_{_{interf}} + e_{_{artif}}
\end{align}

Where \(x_{_{s}}\) stands for the
part of \(\widehat{x}(t)\) coming from the
wanted source(s), and \(e_{_{noise}}\) represents the part
of \(\widehat{x}(t)\) coming from the sensor's noise. The sensor
can be the microphone itself or one of its counterparts.
\(e_{_{interf}}\) notes the unwanted sources presented
in \(\widehat{x}(t)\), and the \(e_{_{artif}}\)
represents any other artifacts that cause
distortions in the prediction of \(x_{_{s}}\).

According to Parseval's theorem, the residual energy in time equals the sum of
the power difference between the predicted magnitude
and the magnitude of target speech
in the frequency domain\cite{1643671009}.
\begin{align}
    \sum_{t} |r(t)|^{2} & = \frac{1}{T}\sum_{\tau=0}^{T-1}\sum_{f=0}^{T-1} \left[ \widehat{X}(\tau, f) - X(\tau, f)\right]^{2}
\end{align}

Since the residual energy is referred to as the noise,
minimizing the residual,
which is minimizing the MSE loss function,
translates into an increase in SNR.

The SNR is therefore given by:
\begin{align}\label{eq:snr_equation}
    SNR & = 10\log_{10} \left( \frac{ \| x_{_{s}}\|^{2}}{\|\widehat{x} - x_{_{s}} \|^{2}}  \right) \nonumber \\
    & =  10\log_{10} \left( \frac{ \| x_{_{s}} \|^{2}}{\| r \|^{2}} \right)
\end{align}
\subsection{SI-SNR --- Scale Invariant SNR}
To ensure that the SNR is susceptible to scale invariance\cite{roux2018sdr},
both the target and estimated signals are normalized to \underline{zero-mean}.

\begin{align}
    SI-SNR & = 10\log_{10} \left( \frac{\left\| x_{_{s}} - \mathbf{E}[x_{_{s}}]\right\|^{2}}
    {\left\| (\widehat{x} - \mathbf{E}[\widehat{x}]) - (x_{_{s}} - \mathbf{E}[x_{_{s}}]) \right\|^{2}} \right) \nonumber \\
    & = 10\log_{10} \left( \frac{ \| x_{_{AC}}\|^{2}}{\|\widehat{x}_{_{AC}} - x_{_{AC}}\|^{2}}  \right) \nonumber \\
    & =  10\log_{10} \left( \frac{ \| x_{_{AC}}\|^{2}}{\| r_{_{AC}} \|^{2}} \right)
\end{align}

\subsection{Segmental SNR}
An SNR evaluation is basically the ratio between the overall
energies of the signal and those in the noise.
However, some portions of the signal are almost pure noise,
especially in the case of speech signals,
where there are gaps between phonemes,
articulation stops, and air aspiration breaks. As a result,
the SNR calculation may be impacted, and
it actually depends on
the length of the empty sections with respect
to the length of
the other sections where speech is present.

With Segmental SNR\cite{10.5555/912256}, instead of taking the entire signal,
the signal is segmented to relatively small chunks (segments),
each in length usually set to \(25ms\) long
with the option of setting an overlap
between segments.
Then, each SNR of each segment is extracted and, finally,
are averaged.
If the energy of the speech reference
in a segment is negligible, that segment is excluded,
thus limiting the evaluation only to sections
where speech is present.

Equation~\ref{eq:snr_equation} can be extended to:
\begin{align}
    SEG-SNR & = \frac{1}{M}\sum_{m=1}^{M}
                10\log_{10}
                \left(
                    \frac{ \| x_{_{s}} \|^{2}_{(m)}}{\| r \|^{2}_{(m)}}
                \right)
\end{align}

Where \(M\) denotes the number of segments the signal is divided by.

Despite being more accurate for speech signals, Segmental SNR
suffers from a limitation that can affect the actual results severely.
In speech enhancement evaluations, the signal's predicted (enhanced) version is
compared to a clean reference signal
concerning the noisy mixture.
Unfortunately, speech analysis for the extraction of
the Segmental SNR causes misalignments between
the reconstructed signal and the clean reference.
Moreover, the reconstructed signal is not aligned
with the noisy mixture either.
These misalignments
are a side effect of the time-domain
to the frequency-domain transformation,
the processing manipulations on the transformed signal,
and the reconstruction of the signal in
the time-domain using the inverse-transform technique.
Therefore, without any alignments, extraction of the Segmental-SNR
is meaningless and most probably inaccurate.
Due to that limitation, an alignment process should be applied
prior to taking the Segmental SNR calculation. These alignments usually
have a small marginal error that spans over a few sampling points.

\subsection{STOI --- Short-Time Objective Intelligibility}
STOI\cite{5495701} is a metric that is used to evaluate
the intelligibility of a speech signal.
The intelligibility is measured by taking the correlation
coefficient between the temporal envelopes of the clean
and degraded speech. In our case, the term degraded might
be confusing since the degraded speech input
is actually the outcome of the beamformer
following the T-F masking at the front-end.
However, relatively to the clean speech,
the beamformer's output is indeed degraded, altough
being considered as an enhanced version of the noisy mixture.

The naming convention \emph{Short-Time} comes from the time frame length
of the overlapping segments, which is \(384 ms\).
\begin{figure}[H]
    \centering
    \includegraphics[width=\linewidth]{Features/images/stoi_blocks_diagram}
    \caption{STOI flow diagram}\label{fig:stoi_blocks_diagram}
    \source{Adapted from \cite{5495701}}
\end{figure}

The STOI algorithm structure is demonstrated in
the blocks diagram shown in Figure\;\ref{fig:stoi_blocks_diagram}.

The short-time temporal envelop of the degraded
speech \(Y_{_{j,m}}\) is clipped and normalized
before the extraction of the correlation coefficient
with the short-time temporal envelop of the
clean speech \(X_{_{j,m}}\).
This clipped normalized version then be:
\begin{align}
    \mathcal{Y}[n] & = \min\Bigg\{
            \frac{||X_{_{j,m}}||}{||Y_{_{j,m}}||} Y_{_{j,m}}[n]
            ,\; (1+10^{\sfrac{-\beta}{20}})X_{_{j,m}}[n]
        \Bigg\}
\end{align}

Thus, the correlation coefficient can be noted as the
distance given in Equation\;\ref{eq:stoi_correl}.
\begin{align}\label{eq:stoi_correl}
    d_{_{j,m}} & = \frac{
            (X_{_{j,m}}-\bar{X}_{_{j,m}})^{tr}
            \cdot (\mathcal{Y}_{_{j,m}}-\bar{\mathcal{Y}}_{_{j,m}})
        }
        {
            ||X_{_{j,m}}-\bar{X}_{_{j,m}}||
            \cdot ||\mathcal{Y}_{_{j,m}}-\bar{\mathcal{Y}}_{_{j,m}}||
        }
\end{align}
Also, defining the intermediate intelligibility measure,
Equation\;\ref{eq:stoi_correl}, it stands for the
\(m^{th}\) time frame. Extending it to form
a definition for the entire signal,
we can take the average of \(d_{_{j,m}}\)
as in Equation\;\ref{eq:stop_dist_avg}.

\begin{align}\label{eq:stop_dist_avg}
    d & = \frac{1}{JM} \sum_{j,m} d_{_{j,m}}
\end{align}

Where \(J\) presents the total number of one-third octave bands,
and the averaging overlaps \(M\) number of time frames.

\subsection{PESQ --- Perceptual Evaluation of Speech Quality}
PESQ\cite{941023} is a measuring method adopted by
the ITU (International Telecommunication Union) to
test the speech quality of telephony and mobile stations.

This measuring metric evolved from different previous
measuring techniques such as Bark Spectral Distortion (BSD),
Perceptual Analysis Measurement System (PAMS),
and Perceptual Speech Quality Measure (PSQM).

The motivation behind the development of the PESQ metric
was the need to assess the speech quality in an E2E
communication channel that considers
the entire link rather than particular parts.

The evaluation of a speech signal quality by PESQ
follows the MOS (Mean Opinion Scores) model, where
the actual speech quality is ranked in-between 1 to 5
by a group of listeners.

Figure \ref{fig:pesq_blocks_diagram} shows the data flow
of the PESQ computation for a predicted signal, with respect to the
clean reference.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.85\linewidth]{Features/images/pesq_blocks_diagram_new}
    \caption{PESQ Algorithm Blocks Diagram}\label{fig:pesq_blocks_diagram}
    \source{Adapted from PESQ paper\cite{941023} and redesigned}
\end{figure}

\section{ASR Metrics}
\subsection{WER - Word Error Rate}
WER\cite{KLAKOW200219} metric is probably the most used evaluation
technique for speech recognition systems.

Evaluation of this metric occurs at the ASR engine's output,
where the predicted text is segmented into sentences.
Each word in the predicted
text is then matched with its counterpart in the
annotated reference transcript.
The sum of mismatches between a predicted sentence and the reference,
divided by the total counted words in the reference, indicates the WER.

However, in some cases, the predicted sentences differ in size compared
to the reference. Therefore, special care for \emph{Insertions}
and \emph{Deletions} should be carried out as well, without neglecting
the detected \emph{Substitutions}.

The WER is described by:
\begin{align}
    WER & = \frac{S + D + I}{N}
\end{align}

Where \(N\) is the total count of words in the reference,
and \(S,D,I\) are the number of \emph{Substitutions} (wrong word detection),
\emph{Deletions} (Omitting words),
and \emph{Insertions} (Wrong words insertions).

\subsection{CER - Character Error Rate}
CER is another metric with some similarities to the
WER evaluation metric but with a narrower resolution.
The change in resolution is due to comparing characters instead of words.
The same rules of \emph{Substitutions}, \emph{Deletions}, and \emph{Insertions}
apply, and therefore, the calculation of the CER is the same:
\begin{align}
    CER & = \frac{S + D + I}{N}
\end{align}

In many cases, the CER\cite{_isword} is a complementary
measuring metric to the WER
metric.
This extra measure shines especially
when there is a need to get a full perspective
with a greater differentiation capability of the
\emph{Substitutions} in the complete sentence
of the suggested predicted transcript,
 also known as the hypothesis.
While WER counts a mismatch between the reference and the predicted
word, even in cases where only single characters or worse,
punctuation marks are not correctly placed, CER can lead to
more accurate grading per word.

\section{HW Metrics}
% \subsection{Computation time}
% \subsection{Utilization Ratio}
\subsection{Power Estimation}
Electrical circuits, components, and systems require power to function.
The amount of power a device consumes from the power sources is subject
to various parameters and mainly describes the rate of energy delivery
from the source to the device or vice versa.

Due to the nature of conducting materials,
whenever an electrical potential is applied
between the conductor's terminals,
electrical current goes through the conductor.
The current that flows
in the system feeds the different components with energy.
However, the total supplied energy is not purely consumed over time,
and some energy is lost and wasted due to power dissipation.

Power dissipation is a side effect of a conductors' resistive nature,
which "resists" the transition of current through it.
As a result, part of the energy in the system is converted
to heat energy.

Since dissipated power is a waste of energy it is also
considered as one of the main causes
to electronic systems' performance degradation at high temperatures.
Therefore, engineers want to mitigate
as much as possible any dissipated power that
is not used for the main functionality of the system. % but dissipated and converted to heat.
For that end, power analysis is crucial in any system
design phase to ensure efficiency and correctness while maintaining
robustness over time and under different working conditions.

Electrical circuit power dissipation depends on many arguments. % mainly when speaking of digital logic designs.
However, in general, it can be modeled
accurately according to three scenarios divided into two main groups:
\begin{enumerate}
    \item Static Power
    \begin{itemize}
        \item Intrinsic Leakage Power
    \end{itemize}
    \item Dynamic Power
    \begin{itemize}
        \item Internal Power
        \item Switching Power
    \end{itemize}
\end{enumerate}

\subsubsection{Intrinsic Leakage Power}
Leakage power is the power that dissipates due to the
structure of a CMOS device,
where a thin layer of metal oxide isolates
between the semiconductor material and the
gate metal and thus forming a capacitor.
Leakage power dissipates statically regardless of the CMOS
device state,
whether it is the active state or the off state (idle).

\begin{figure}[H]
    \centering
    \includegraphics[width=0.75\linewidth]{Features/images/leak_power_schem}
    \caption{Leakage Power Illustration}\label{fig:leak_power_schem}
    \source{Adapted from Synopsys PrimePower Suit documentation\cite{lowpowerSoc}}
\end{figure}

Figure \ref{fig:leak_power_schem} describes three current leakages,
the reverse bias current of the diode (p-n junction), sub-threshold current leakage, and
the gate leakage.

With the recent advancement in process technologies,
CMOS devices are minimized in size,
but the leakage power is increasing as a side effect.
\begin{figure}[H]
    \centering
    \includegraphics[width=0.55\linewidth]{Features/images/leak_vs_nm}
    \caption{Leakage Power vs. Process Technology}\label{fig:leak_vs_nm}
    \source{Adapted from Soitec FinFet presentation\cite{processnodeleak}}
\end{figure}
The overall leakage power is a function
of the total number of voltage sources and their voltage levels,
the physical dimensions of the CMOS device,
and the threshold value set to switch between on-off states.
\begin{align}
    P_{leak} & = \mathcal{F} (V_{_{DD}}, V_{_{th}}, \frac{\mu_{n}\varepsilon_{ox} W}{L})
\end{align}

\subsubsection{Internal Power}
A CMOS device is a formation of two complementary MOS transistors,
a p-type and an n-type, formed together as a symmetrical
pair unit.
Internal power dissipation happens due to the structure of
CMOS devices.
Whenever a transition at the CMOS gate occurs,
both the NMOS and the PMOS drivers are active for a
relatively small duration of time. As a result,
a short circuit is formed directly from the power rail to the ground.
Although not lasting for long periods of time,
the amount of internal dissipated power in highly toggled
designs becomes significant over time.
To minimize the internal dissipated power,
or in other words, minimizing the time duration where
both devices are active and current flows from Vdd to GND,
the transition times (both rising and falling) are set to be very fast.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.75\linewidth]{Features/images/int_power_schem}
    \caption{Internal Power Illustration}\label{fig:int_power_schem}
    \source{Adapted from Synopsys PrimePower Suit documentation\cite{lowpowerSoc}}
\end{figure}

\subsubsection{Switching Power}
Switching power is the power dissipated as a result
of charging and discharging loads during transitions.
MOS devices introduce capacitance at their input gates
due to their structure.
Thus, whenever a low-to-high transition at the output occurs,
the driver pushes the current to charge the capacitive
load in order to set the desired logic level voltage.
Likewise, the load capacitance discharge and sink into the
device through the PMOS transistor to the ground
for a high-to-low transition at the output.
As a result, the charging and discharging currents
eventually dissipate and are not delivered to the external load.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.75\linewidth]{Features/images/sw_power_schem}
    \caption{Switching Power Illustration}\label{fig:sw_power_schem}
    \source{Adapted from Synopsys PrimePower Suit documentation\cite{lowpowerSoc}}
\end{figure}


\section*{Exercises}

\begin{ex}
  Which of the following vector functions are linear transformations?
  \begin{equation*}
    T_1\paren{\begin{mymatrix}{c} x \\ y \end{mymatrix}}
    = \begin{mymatrix}{c} 2x+y \\ x-2y \\ -x-y \end{mymatrix},\quad
    T_2\paren{\begin{mymatrix}{c} x \\ y \\ z \end{mymatrix}}
    = \begin{mymatrix}{c} x+y^2 \\ (x+y)z \\ 0 \end{mymatrix},\quad
    T_3\paren{\begin{mymatrix}{c} x \\ y \\ z \end{mymatrix}}
    = \begin{mymatrix}{c} 0 \\ 0 \end{mymatrix}.
  \end{equation*}
  \begin{sol}
    $T_1$ and $T_3$ are linear, and $T_2$ is not. The transformation
    $T_3$ is called the \textbf{zero transformation}%
    \index{zero transformation}%
    \index{linear transformation!zero transformation}.
  \end{sol}
\end{ex}

\begin{ex} Consider the following functions $T:\R^3\rightarrow \R^2$.
  Explain why each of these functions $T$ is not linear.

  \begin{enumerate}
  \item $T\begin{mymatrix}{c}
      x \\
      y \\
      z
    \end{mymatrix} =\begin{mymatrix}{c}
      x+2y+3z+1 \\
      2y-3x+z
    \end{mymatrix}$

  \item $T\begin{mymatrix}{c}
      x \\
      y \\
      z
    \end{mymatrix} =\begin{mymatrix}{c}
      x+2y^2+3z \\
      2y+3x+z
    \end{mymatrix}$

  \item $T\begin{mymatrix}{c}
      x \\
      y \\
      z
    \end{mymatrix} =\begin{mymatrix}{c}
      \sin x+2y+3z \\
      2y+3x+z
    \end{mymatrix}$

  \item $T\begin{mymatrix}{c}
      x \\
      y \\
      z
    \end{mymatrix} =\begin{mymatrix}{c}
      x+2y+3z \\
      2y+3x-\ln z
    \end{mymatrix}$
  \end{enumerate}
  % \begin{sol}
  % \end{sol}
\end{ex}

\begin{ex}
  Let $A$ be an $m\times n$-matrix. Show the vector function
  $T:\R^n\to \R^m$ defined by $T(\vect{v})=A\vect{v}$ is a
  linear transformation.
  \begin{sol}
    We have
    $T(a\vect{v}+b\vect{w}) = A(a\vect{v}+b\vect{w}) = a(A\vect{v}) +
    b(A\vect{w}) = aT(\vect{v}) + bT(\vect{w})$ by properties of
    matrix multiplication. Therefore, $T$ is linear by
    Proposition~\ref{prop:linear-transformation-alternative}.
  \end{sol}
\end{ex}

\begin{ex}
  Let $\vect{u}\in\R^n$ be a fixed vector.  Show that the function $T$
  defined by
  $T(\vect{v}) = \vect{v}-\proj_{\vect{u}}(\vect{v})$ is a
  linear transformation.
  \begin{sol}
    We have
    \begin{eqnarray*}
      T(a\vect{v}+b\vect{w})
      &=& a\vect{v}+b\vect{w}
          - \frac{\vect{u}\dotprod(a\vect{v}+b\vect{w})}{\norm{\vect{u}}^2}\vect{u} \\
      &=& a\vect{v}
          - a\frac{\vect{u}\dotprod\vect{v}}{\norm{\vect{u}}^2}\vect{u}
          + b\vect{w}
          - b\frac{\vect{u}\dotprod\vect{w}}{\norm{\vect{u}}^2}\vect{u} \\
      &=& aT(\vect{v}) + bT(\vect{w}).
    \end{eqnarray*}
    Therefore, $T$ is linear.
  \end{sol}
\end{ex}

\begin{ex}
  Let $\vect{u}\in\R^n$ be a fixed non-zero vector. The function $T$
  defined by $T(\vect{v})=\vect{u}+\vect{v}$ has the effect of
  translating all vectors by adding $\vect{u}$. Show this is not a
  linear transformation.
  \begin{sol}
    If $T$ were a linear transformation, it should satisfy
    $T(\vect{0}) = \vect{0}$, but it does not. Also
    $T(\vect{v}+\vect{w}) \neq T\vect{v}+T\vect{w}$.
  \end{sol}
\end{ex}


\chapter{Sliced Mixed-Marginal Wasserstein}

\section{Abstract}
	Multi-marginal optimal transport enables one to compare multiple probability measures, which increasingly finds application in multi-task learning problems.
	One practical limitation of multi-marginal transport is computational scalability in the number of measures, samples and dimensionality.
	In this work, we propose a multi-marginal optimal transport paradigm based on random one-dimensional projections, whose (generalized) distance we term the \emph{sliced multi-marginal Wasserstein distance}.
	To construct this distance, we introduce a characterization of the one-dimensional multi-marginal Kantorovich problem and use it to highlight a number of properties of the sliced multi-marginal Wasserstein distance.
	In particular, we show that (i) the sliced multi-marginal Wasserstein distance is a (generalized) metric that induces the same topology as the standard Wasserstein distance, (ii) it admits a dimension-free sample complexity, (iii) it is tightly connected with the problem of barycentric averaging under the sliced-Wasserstein metric.
	We conclude by illustrating the sliced multi-marginal Wasserstein on multi-task density estimation and multi-dynamics reinforcement learning problems.

\section{Introduction}

Optimal transport is a framework for defining meaningful metrics between probability measures \cite{villani, compopt}.
These metrics find a wide range of applications, such as generative modeling \cite{pmlr-v84-genevay18a,bunne2019}, Bayesian inference \cite{bayes}, imitation learning \cite{Dadashi2020PrimalWI}, graph matching and averaging \cite{pmlr-v97-xu19b,NIPS2019_8569}.
Multi-marginal optimal transport \cite{gangbo} studies ways of comparing more than two probability measures in a geometrically meaningful way.
Multi-marginal distances defined using this paradigm are often useful in settings where sharing geometric structure is useful, such as multi-task learning. In particular, they have been applied for training multi-modal generative adversarial networks \cite{mwgan}, clustering  \cite{generalizedmet}, and computing barycenters of measures \cite{altschuler_mm}.

Following the establishment of key theoretical results, including by \textcite{gangbo,journals/siamma/AguehC11, Pass2014MultimarginalOT}, research is shifting toward applications. This motivates a need for practical algorithms for the multi-marginal setting \cite{mw_compl}.
Standard approaches based on linear programming and entropic regularization scale exponentially with the number of measures, and/or the dimension of the space \cite{benamou:hal-01096124, accmmot}.
A number of recent works have therefore studied settings, where multi-marginal transport problems can be efficiently solved via low-rank structures on the underlying cost function \cite{altschuler_mm}, but exponential cost in the dimension remains \cite{altschulernpbary, altschuler_mm_np}.

In parallel, a number of works on \emph{sliced transport} \cite{bonnottee} developed techniques for scalable transport, which (i) derive a closed form for a problem in a single dimension, and (ii) extend it into higher dimensions via random linear projections (slicing) and thereby inherit the complexity of the one-dimensional problem.
This strategy has been shown effective in the classical Wasserstein \cite{bonnottee,bonneel, gensliced, distributionalsliced, maxsliced, orthsliced} and Gromov--Wasserstein \cite{sliced_gw} settings between pairs of measures, but has not yet been applied to settings with more than two measures.

In this paper, we address this gap and propose \emph{sliced multi-marginal transport}, providing a scalable analog of the multi-marginal Wasserstein distance.
To do so, we derive a closed-form expression for multi-marginal Wasserstein transport in one dimension, which lifts to a higher-dimensional analog via slicing.
This one-dimensional closed-form expression can be computed with a complexity of $\mathcal{O}(PN\log N)$, where $P$ is the number of measures and $N$ is the number of samples per measure.
Sliced multi-marginal Wasserstein ($\mathcal{SMW}$) can be estimated by Monte Carlo in $\mathcal{O}(KPN\log N)$, where $K$ is the number of Monte Carlo samples.

Furthermore, we study $\mathcal{SMW}$'s theoretical properties.
We prove that (i) it is a generalized metric, whose associated topology is the topology of weak convergence, (ii) its sample complexity is dimension free, just like the sliced Wasserstein case involving two measures, and (iii) sliced multi-marginal transport is closely connected with the problem of barycentric averaging under the sliced Wasserstein metric.
% Furthermore, we discuss projection complexity, and inequalities that relate sliced multi-marginal optimal transport with the Wasserstein distance between the marginals.
%
We also showcase applications, where we focus on multi-task learning on probability spaces, where sharing knowledge across tasks can be beneficial and sliced multi-marginal Wasserstein can be used as a regularizer between task-specific models.
We demonstrate this on a multi-task density estimation problem, where individual estimation tasks are corrupted and shared structure is needed to solve the problem, as well as a reinforcement learning problem, where certain agents receive no reward and must instead learn from other agents to solve their given task.

\section{Background}

\label{sec:background}

Multi-marginal optimal transport \cite{gangbo} is a class of optimization problems for comparing multiple measures $\mu_1,\ldots,\mu_P \in \mathcal{M}(\R^d)$, all supported on the metric space $(\R^d,  || \cdot||_2)$.
The most common such problem is computing the multi-marginal Wasserstein distance, defined as
\[
\label{eq:multi_marginal_w}
\mathcal{MW}^2(\mu_1,\ldots,\mu_P)=\min_{\pi \in \Pi(\mu_1,\ldots, \mu_P)} \int_{(\mathbb{R}^d)^P}  c(x_1,\ldots,x_P) \, d\pi(x_1,\ldots,x_P),
\]

where $c:\mathbb{R}^d \times\ldots\times \mathbb{R}^d \to \mathbb{R}$ is a cost function and $\Pi(\mu_1,\ldots, \mu_P)$ is the set of probability measures in $\mathcal{M}((\mathbb{R}^d)^P)$ with marginals $\mu_1, \ldots, \mu_P$.
We focus on the barycentric cost of \textcite{gangbo, journals/siamma/AguehC11}, given by
\[
c(x_1,\ldots,x_P) = \sum_{p=1}^P \beta_p \Big\Vert  x_p - \sum_{j=1}^P\beta_j x_j\Big\Vert ^2,
\quad\beta_1,\ldots,\beta_P \geq 0, \quad \sum_{p=1}^P \beta_p = 1
.
\]
This cost was originally motivated from an economics-inspired perspective, but is also often preferable because it leads to connections with barycentric averaging \cite{journals/siamma/AguehC11}, giving it a simple interpretation.
It also recovers the Wasserstein distance with squared $2$-Euclidean cost in the case $P=2$ (up to constants), referred to as $\mathcal{W}$.
Algorithms for estimating \eqref{eq:multi_marginal_w} from a set of samples  scale exponentially with the number of measures $P$ and/or the dimension $d$ of the ground space  \cite{altschuler_mm, altschulernpbary, benamou:hal-01096124}.

$\mathcal{MW}$ is useful in multi-task settings for regularizing measures $\mu_1,\ldots, \mu_P$ by adding $\mathcal{MW}(\mu_1,\ldots, \mu_P)$ to a multi-task loss.
It can also be used in a setting, where we aim for a model output $\mu$ to be close to a given set of measures $\nu_1,\ldots, \nu_P$, which can be done by introducing a loss of the form $\mathcal{MW}(\mu, \nu_1,\ldots, \nu_P)$ and minimizing it with respect to $\mu$.

% \begin{center}
% \begin{tabular}{llll}
% Method (incl. reference) & Computational complexity & Comments\\
% \end{tabular}
% \end{center}

\begin{figure}
	\centering
	\includegraphics[height=3cm]{pictures/diagrams/diagram_uniform.pdf}
	\hspace{5mm}
	\includegraphics[height=3cm]{pictures/diagrams/diagram_alignment.pdf}
	% \includegraphics[width=\hsize]{images/diagrams/sketch}
	\caption{Illustration of the optimal coupling's structure on $\R$ between discrete measures $\mu_1, \mu_2$ and $\mu_3$. Points are samples of each measures, with weights next to them.  Left: histogram of measures (horizontal); joint samples are obtained by sampling a (black) line uniformly (drawn vertically), and picking points that are associated with the bin intersected by that line. Right: Corresponding triples of points that are aligned according to the coupling are linked by a pair of lines.}
	\label{fig:illustr_mmot}
\end{figure}
%\subsection{Slicing Techniques for Scalability}
\textbf{Sliced transport.}  With the usual Euclidean-type cost structures, the Wasserstein distance between pairs of one-dimensional discrete measures can be computed efficiently using \emph{sorting}  with $\mathcal{O}(N \log N)$ complexity.
More generally, we can consider the average distance between measures projected onto $\mathbb{R}$ along random axis, which gives \cite{bonnottee, bonneel}
\[
\mathcal{SW}^2(\mu, \nu) = \int_{S_{d-1}}\mathcal{W}^2\big(M^{\theta}_{\#}(\mu), M^{\theta}_{\#}(\nu)\big) \, d\Theta(\theta),
\]
where $M^{\theta}(x) = x^T \theta$, $(.)_\#$ denotes the push-forward of measures, and $\Theta$ is the uniform distribution on the unit sphere $S_{d-1}$. We sample from $M^{\theta}_{\#}(\mu)$ by sampling from $\mu$ and  projecting onto $\theta$.

A fundamental result by \textcite{bonnottee} is that $\mathcal{SW}$ is a metric that metrizes the topology of weak convergence---the \emph{exact same} topology as $\mathcal{W}$.
$\mathcal{SW}$ can be estimated via Monte Carlo and preserves the computational complexity of estimating $\mathcal{W}$ on $\R$, which is $\mathcal{O}(N\log N)$.
Owing to the Monte Carlo nature, the sample complexity of $\mathcal{SW}$ is dimension free \cite{bonnottee, topstatprop}, in contrast with the exponential dependency of the Wasserstein distance on dimension.
The combination of good computational and statistical properties makes $\mathcal{SW}$ an attractive choice for  minimization problems on measure spaces, including generative modeling and imitation learning \cite{maxsliced, Dadashi2020PrimalWI}.
This immediately raises the question whether $\mathcal{SW}$ extends to the multi-marginal case so that it preserves its key appealing properties.

\subsection{Sliced Multi-Marginal Wasserstein Distance}

To define the sliced multi-marginal Wasserstein distance, we average the expressions given in \eqref{eq:mmwass_closed} along  one-dimensional random projections, which gives
\[
\label{eq:smmwass}
\mathcal{SMW}^2(\mu_1,\ldots,\mu_P) = \int_{S_{d-1}} \int_{0}^1 \sum_{p=1}^{P} \beta_p\Big| C_{\mu_p^{\theta}}^{-1}(x)-\sum_{j=1}^P \beta_j   C_{\mu_j^{\theta}}^{-1}(x) \Big|^2 \, dx \, d\Theta(\theta),
\]
where $ \mu_j^{\theta} = M^{\theta}_{\#} (\mu_j)$ for $ j=1,\ldots,P$.
$\mathcal{SMW}$ in \eqref{eq:smmwass} can be estimated via Monte Carlo in $O(KPN\log N)$, where $K$ is the number of Monte Carlo samples (projections).

\paragraph{Topological properties}
We now study $\mathcal{SMW}$'s  topological properties. We first show that  $\mathcal{SMW}$  is the weighted mean of sliced Wasserstein distances between pairs of measures.

\begin{prop}
	Let $\mu_1,\dots,\mu_P \in \mathcal{M}(\mathbb{R}^d)$. We have that
	\[
	\mathcal{SMW}^2(\mu_1, \ldots, \mu_P)  = \frac{1}{2} \sum_{i,j=1}^P \beta_i \beta_j \mathcal{SW}^2(\mu_i,\mu_j)
	.
	\label{eq:meansliced}
	\]
	\label{prop:meansliced}
\end{prop}

Proposition \ref{prop:meansliced} is useful in deriving statistical and topological properties of $\mathcal{SMW}$. It is however more efficient to estimate it via our closed-form formula for multi-marginal transport -- see \eqref{eq:smmwass}. This leads to a computational complexity of $O(KPN\log N)$, whereas naively implementing \eqref{eq:meansliced} scales in $\mathcal{O}(KP^2N\log N)$.
%
Furthermore, as the sliced-Wasserstein metric is upper-bounded by the Wasserstein \cite{bonnottee}, an immediate consequence of Proposition \ref{prop:meansliced} is that
\[
\mathcal{SMW}^2(\mu_1, \ldots, \mu_P) \stackrel{\eqref{eq:meansliced}}{=} \frac{1}{2} \sum_{i,j=1}^P\beta_i \beta_j \mathcal{SW}^2(\mu_i,\mu_j) \leq \frac{1}{2} \sum_{i,j=1}^P \beta_i \beta_j\mathcal{W}^2(\mu_i,\mu_j).
\]
This shows that $\mathcal{SMW}$ gives rise to the topology of weak convergence---one of the key properties that made $\mathcal{SW}$ an attractive choice in the first place.
% \paragraph{Metric properties}
We now study metric properties of $\mathcal{SMW}$.

\begin{prop}
	$\mathcal{SMW}$ is a generalized metric.
	\label{prop:metricprop}
\end{prop}

In particular, this means that $\mathcal{SMW}$ is (i) non-negative, (ii) zero if and only if all measures are identical, (iii) permutation-equivariant, and (iv) satisfies a generalized triangle inequality involving multiple measures. Hence, $\mathcal{SMW}$ is well-behaved topologically-wise as it is a generalized metric inducing weak convergence. We continue by studying $\mathcal{SMW}$'s statistical properties.

\paragraph{Statistical Properties}
In the following proposition, we assess the impact of the number of samples and random projections used to estimate $\mathcal{SMW}$.

\begin{prop}
	\label{prop:samplecomplexity}
	If $\mu_1,\ldots,\mu_P \in \mathcal{M}(\R^d)$, and assuming $\mathcal{W}^2$ has sample complexity $\rho(N)$ on $\R$, then,
	\[ &\quad \E [\mathcal{SMW}^2(\mu_1,\ldots,\mu_P) - \mathcal{SMW}^2(\h{\mu}_1,\ldots,\h{\mu}_P)]^2
	\leq\frac{1}{2}\rho(N),
	\]
	where $\h{\mu}_p$ refers to empirical measures with $N$ samples.
\end{prop}

% \begin{proof}
% Appendix \ref{sec:complexitiesproofs}.
% \end{proof}

Proposition \ref{prop:samples} shows that the sample complexity of $\mathcal{SMW}$ is dimension-free---this stands in contrast to the sample complexity of the multi-marginal Wasserstein, which is exponential in the dimension.
%
In practice, we use Monte Carlo sampling to compute $\mathcal{SMW}$, which introduces additional error.
To understand this error, we examine $\mathcal{SMW}$'s projection complexity.

\begin{proposition}
	\label{prop:projectioncomplexity}
	Let $\mu_1,\ldots,\mu_P \in \mathcal{M}(\R^d)$, and define $\widebar{\mathcal{SMW}}$ the approximation obtained by uniformly picking $L$ projections on $S_{d-1}$, then
	\[
	\E\left[ \widebar{\mathcal{SMW}}^2(\mu_1,\ldots,\mu_P) - \mathcal{SMW}^2(\mu_1,\ldots,\mu_P)\right]^2
	\leq
	L^{-1/2}\Var_{\v{\theta}}\Big[\mathcal{MW}^2\big(\mu_1^{\v{\theta}},\ldots,\mu_P^{\v{\theta}})\Big],%\int_{S_{d-1}} \Big\{\mathcal{MW}^2\big(M_{\theta\#}(\mu_1),\ldots,M_{\theta\#}(\mu_P)\big)-\delta \Big\}^2d\Theta(\theta),
	\]
	where  $\v{\theta}$ follows the uniform distribution on $S_{d-1}$ and $\mu_p^{\v{\theta}} = M_{\#}^{\v{\theta}}(\mu_p)$.

\end{proposition}

% \begin{proof}
% Appendix \ref{sec:complexitiesproofs}.
% \end{proof}

This shows that the quality of Monte Carlo estimates of $\mathcal{SMW}$ is controlled by number of projections and the variance of evaluations of the base multi-marginal Wasserstein in 1D.


\subsection{Trial division}

We have x

Divide by numbers between 2 and x

Only need to go to sqrt x

Don't need to divide by even numbers other than 2

algorithm for checking if number is a prime

loop up dividing number from 2

if divides, add factor list and divide target number by that

stop when i reaches number

eg for 45

divide 2? no

divide 3? yes :> 15

divide 3? yes :> 5

divide 4? no

divide 5? yes :> 1

6>1 so stop

number is prime if list just contains target

don't have to worry about including non primes in list, as will already have divded by that amount


\section{Data space, feature space and relevant mappings}\label{sec:spaces}
Given a data
\begin{equation}
	\label{data-f}
	f \in \mathbb{R}^{m \times n \times c},
	\quad \text{or}\quad [f]_i \in
	\mathbb{R}^{m\times n}, \quad i = 1:c,
\end{equation}
where $m\times n$ is called the spatial dimension and $c$ is the
channel dimension.

For the given data $f$ in \eqref{data-f}, we look for some
feature vector, denoted by $u$,  associated with $f$:
\begin{equation}
	\label{u}
	u \in \mathbb{R}^{m \times n \times h}.
\end{equation}
We make an assumption that the data $f$ and feature $u$ are related by a mapping
(which can be either linear or nonlinear)
\begin{equation}
	\label{u}
	A:  \mathbb{R}^{m \times n \times h}\mapsto \mathbb{R}^{m \times n \times c},
\end{equation}
so that
\begin{equation}
	\label{Auf}
	A(u)=f.
\end{equation}
A mapping
\begin{equation*}
	B : \mathbb{R}^{m\times n\times c} \mapsto \mathbb{R}^{m \times n\times h},
\end{equation*}
is called a feature extractor if $B \approx A^{-1}$ and
\begin{equation}\label{vBf}
	v = B(f),
\end{equation}
is such that $v \approx u$.

The data-feature relationship \eqref{Auf} or \eqref{vBf} is not
unique.   Different relationships give rise to different features.
We can view the data-feature relationship given in \eqref{Auf}
as a model that we propose.  Here the mapping $A$, which can be either
linear or nonlinear, is unknown and needs to be trained.

\subsection{Some linear and nonlinear mappings and extractors}
A data-feature map $A$ and feacture extractor $B$ can be either
linear or nonlinear.   The nonlinearity can be obtained from
appropriate application of an activation function
\begin{equation}
	\label{act}
	\sigma: \mathbb{R} \to \mathbb{R} .
\end{equation}
In this paper, we mainly consider a special activation function, known
as the {\it rectified linear unit} (ReLU), which is defined by
\begin{equation}
	\label{relu}
	\sigma(x)= {\rm ReLu}(x) :=\max(0,x), \quad x\in\mathbb{R}.
\end{equation}
By applying the function to each component, we can extend this
\begin{equation}
	\label{vector-act}
	\sigma:\mathbb R^{m\times n\times c}\mapsto \mathbb R^{m\times n\times c}.
\end{equation}

A linear data-feature mapping can simply given by a convolution as in \eqref{con1}:
\begin{equation}
	\label{linearA}
	A(f)=A\ast f
\end{equation}
A nonlinear mapping can be given by compositions of convolution and
activation functions:
\begin{equation}
	\label{nonlinearA}
	A=\xi\circ\sigma\circ\eta ,
\end{equation}

and
\begin{equation}
	\label{extractor}
	B=\sigma\circ \gamma \circ\sigma  .
\end{equation}
Here $\xi$, $\eta$ and $\gamma$ are all
appropriate convolution mappings.


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[legalpaper, portrait, margin=1in]{geometry}

\title{Mathinity Factorial Problem}

\author{Shreenabh Agrawal}

\date{\today}

\begin{document}

\maketitle

\section{Question}
$$Let \: S_{n}=1 !+2 !+\ldots+n !$$
Find all values of $n$ such that $S_{n}$ is a perfect square and prove that these are the only ones.
\section{Solution}
Let us start by calculating some values of this summation:
$$\begin{aligned}
S_{1} &=1 ! \\
&=1 \\
S_{2} &=1 !+2 ! \\
&=1+2 \\
&=3 \\
S_{3} &=1 !+2 !+3 ! \\
&=1+2+6 \\
&=9 \\
S_{4} &=1 !+2 !+3 !+4 !\\
&=1+2+6+24 \\
&=33
\end{aligned}$$
Here, we can easily find the trivial solutions $$n = \: 1 \: \& \: n = \: 3$$
Next, from $S_5$ onwards, there will always be multiples of 10 that would be added to $S_4$ (because $5!$ and consequently all larger factorials have 10 as a factor). Hence, we can write as (for $n > 4$)

$$\begin{aligned}
S_{n} & \equiv\left(S_{4}+10 k\right) \bmod 10 \\
& \equiv S_{4} \bmod 10 \\
& \equiv 3 \bmod 10
\end{aligned}$$
Which as we know can never be perfect squares. Thus our only answers are:

$$\boxed{n = 1, \: 3}$$
\end{document}

Number theory/Mathinity's questions/Congruence Factorial.tex


\documentclass{article}
\usepackage{amsmath}
\usepackage[margin=1.0in]{geometry}
\usepackage{xcolor}

\begin{document}

\noindent
Does $\displaystyle \sum_{n=1}^\infty n!$
diverge, converge absolutely, or converge conditionally?

\subsection*{Solution 1}

\begin{align*}
\lim_{n \to \infty} a_n
&= \lim_{n \to \infty} n!\\
&= \infty
\end{align*}
Since $\lim_{n \to \infty} a_n$ does not exist (infinity is not a number), by the Test for Divergence, the series $\displaystyle \sum_{n=1}^\infty n!$ diverges.

\subsection*{Solution 2}

\begin{align*}
L&=\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|\\
&= \lim_{n \to \infty} \left| \frac{(n+1)!}{n!}\right|\\
&= \lim_{n \to \infty} \left| \frac{(n+1) \cdot n!}{n!}\right|\\
&= \lim_{n \to \infty} \left| n+1 \right|\\
&= \lim_{n \to \infty} (n+1)\\
&= \infty.
\end{align*}
Since $L = \infty$, by the Ratio Test, the series $\displaystyle \sum_{n=1}^\infty n!$ diverges.

\end{document}%%%%%%%%%%%%%%%%%

\begin{align*}
L&=\lim_{n \to \infty} \sqrt[n]{|a_n|}\\
&= \lim_{n \to \infty} \sqrt[n]{\left| \right|}\\
\end{align*}

\begin{align*}
L&=\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|\\
&= \lim_{n \to \infty} \left| \right|\\
\end{align*}

\begin{align*}
\lim_{n \to \infty} a_n
&= \lim_{n \to \infty} \\
\end{align*}

Since $\sum |a_n| = \sum a_n$, the series $\displaystyle \sum_{n=1}^\infty AAAAAAAAAAAAAA$ converges absolutely.

Since $|r| < 1$, the series ...  converges by the Geometric Series Test.

Since $|r| \geq 1$, the series ...  diverges by the Geometric Series Test.

The function $f(x)=\frac{}{}$ is continuous, positive, and decreasing on $[1,\infty)$.

\subsection*{Solution}


\documentclass{article}
\usepackage{amsmath}
\usepackage[margin=1.0in]{geometry}
\usepackage{xcolor}

\begin{document}

\noindent
Does $\displaystyle \sum_{n=2}^\infty \left( \frac{1}{\ln n} - \frac{1}{\ln(n+1)} \right)$
diverge, converge absolutely, or converge conditionally?

\subsection*{Solution}

$\displaystyle \sum_{n=2}^\infty \left( \frac{1}{\ln n} - \frac{1}{\ln(n+1)} \right)$ appears to be a telescoping series. We look at the sequence of partial sums $s_n$, and note that after cancellation\footnote{This cancellation is much easier to describe in handwritten work than it is in typewritten work.... sorry!} we have
\[ s_n = \frac{1}{\ln 2} - \frac{1}{\ln(n+1)}\]

Then
\begin{align*}
\lim_{n \to \infty} s_n
&= \lim_{n \to \infty} \left(\frac{1}{\ln 2} - \frac{1}{\ln(n+1)}\right)\\
&= \frac{1}{\ln 2} - 0\\
&= \frac{1}{\ln 2}.
\end{align*}
{\color{red} It is tempting to get confused here with the Test for Divergence, but note that we found the limit of the sequence of partial sums.} So, (by definition), the series  $\displaystyle \sum_{n=2}^\infty \left( \frac{1}{\ln n} - \frac{1}{\ln(n+1)}\right)$ converges, and we furthermore know that the sum is $\frac{1}{\ln 2}$. Recall that it is unusual that we get to know the value of a convergent series.

Since $\sum |a_n| = \sum a_n$, the series $\displaystyle \sum_{n=2}^\infty \left( \frac{1}{\ln n} - \frac{1}{\ln(n+1)} \right)$ converges absolutely.

\end{document}%%%%%%%%%%%%%%%%%

\begin{align*}
L&=\lim_{n \to \infty} \sqrt[n]{|a_n|}\\
&= \lim_{n \to \infty} \sqrt[n]{\left| \right|}\\
\end{align*}

Since $\sum |a_n| = \sum a_n$, the series $\displaystyle \sum_{n=1}^\infty AAAAAAAAAAAAAA$ converges absolutely.

Since $|r| < 1$, the series ...  converges by the Geometric Series Test.

Since $|r| \geq 1$, the series ...  diverges by the Geometric Series Test.

The function $f(x)=\frac{}{}$ is continuous, positive, and decreasing on $[1,\infty)$.

\subsection*{Solution}


\section*{Chapter 5: The Emergence of Modern Number Theory}

\paragraph{Exercise 5.1}
Prove that if $n > 4$ is composite, then $(n-1)!$ is a multiple of $n$.

\begin{proof}
Let $n > 4$ be a composite integer with prime factorization
$n = p_1^{\alpha_1} \dots p_k^{\alpha_k}$. First note that, if $d$ is a
proper divisor of $n$, then $\Divides{d}{(n-1)!}$. Indeed, $d \leq n-1$,
and so $(n-1)! = (n-1) \cdot (n-2) \cdots d \cdot (d-1) \cdots 1$. \\

Suppose that $k > 1$. Then, $\Divides{p_i^{\alpha_i}}{(n-1)!}$,
$1 \leq i \leq k$. Since $p_1^{\alpha_1}, \dots, p_k^{\alpha_k}$ are
pairwise coprime, we have that
$\Divides{n = p_1^{\alpha_1} \dots p_k^{\alpha_k}}{(n-1)!}$.\\

Now, suppose that $k = 1$. Since
$n$ is composite and $n = p_1^{\alpha_1} > 4$, then either $p_1 > 2$ or
otherwise $\alpha_1 > 2$. In the latter case,
$(n-1)! = (n-1) \cdot (n-2) \cdots p_1^{\alpha_1 - 1} \cdots p_1 \cdots 1$,
and so $\Divides{n = p_1^{\alpha_1}}{(n-1)!}$. Otherwise, if $\alpha_1 = 2$,
$2 \cdot p_1 < p_1^2 = n$, and so
$(n-1)! = (n-1) \cdot (n-2) \cdots 2p_1 \cdots p_1 \cdots 1$, which means
that $\Divides{n = p_1^{2}}{(n-1)!}$.
\end{proof}


\subsection{Difference from Method Of Moments (MOM)}

More conditions than data.

\subsection{Generalised Method of Moments (GMM)}

We have a function on the output and a parameter:

\(g(y, \theta )\)

A moment condition is that the expectation of such a function is \(0\).

\(m(\theta )=E[g(y, \theta )]=0\)

To do GMM, we estimate this using:

\(\hat m(\theta )=\dfrac{1}{n}\sum_ig(y_i, \theta )\)

We define:

\(\Omega = E[g(y, \theta )g(y, \theta)^T]\)

\(G=E[\Delta_\theta g(y, \theta)]\)

And then minimise the norm:

\(||\hat m(\theta )||^2_W=\hat m(\theta )^TW\hat m(\theta )\)

Where \(W\) is a positive definite matrix for the norm.

\(\Omega ^{-1}\) is most efficient. But we don't know this. It depends on \(\theta \).

We can estimate it if IID:

\(\hat W(\hat \theta )= (\dfrac{1}{n}\sum_i g(y, \hat \theta)g(y, \hat \theta)^T)^{-1}\)

\subsection{Two-step feasible GMM}

Estimate using \(\mathbf W=\mathbf I\)

Consistent, but not efficient.

\subsection{Moment conditions}

OLS:

\(E[x(y-x\theta)]=0\)

WLS

\(E[x(y-x\theta)/\sigma^(x)]=0\)

IV

\(E[z(y-x\theta)]=0\)

MLE

\(E[\Delta_\theta \ln f(x, \theta)]=0\)

\subsection{New GMM}

\(m(\theta_0)=E[g(\mathbf x_i, \theta_0]\)

We replace this with sample moment

\(\hat m(\theta)=\frac{1}{n}\sum_ig(\mathbf x_i, \theta)\)

We have the "score"

\(\nabla_\theta g(\mathbf x_i, \theta_0)\)

Information

\(G=E[\nabla_\theta g(\mathbf x_i, \theta_0)]\)

Variance-covariance loss matrix

\(\Omega =E[g(\mathbf x_i, \theta_0)g(\mathbf x_i, \theta_0)^T]\)

We want to minimise moment loss

\(||\hat m(\theta)||^2_W=\hat m(\theta )^TW\hat m(\theta)\)

\(\hat \theta = argmin_\theta (\frac{1}{n}\sum_ig(\mathbf x_i, \theta))^T\hat W(\frac{1}{n}\sum_ig(\mathbf x_i, \theta))\)

\subsection{Asymptotic}

CLT means normal.

They are consistent IF moment condition is true.

There is an explicit formula for variance.

\(\sqrt n (\hat \theta -\theta_0)\rightarrow^d N[0, (G^TWG)^{-1}G^TW\Omega W^TG(G^TW^TG)^{-1}]\)

If we choose \(W\propto \Omega^{-1}\) then:

\(\sqrt n (\hat \theta -\theta_0)\rightarrow^d N[0, (G^T\Omega^{-1} G)^{-1}]\)

Problem: we need to estimate \(\Omega \) and \(G\).

\(\Omega \): estimate from sample. allows us to choose estimator, but still leaves variance unidentified.

Do the above from OLS? This is where robust etc stuff comes from

If it is specified. Moment conditions are equal to the number of moments, then \(W\) doesn't matter. This is normal Method of Moments.

Estimating the weighting matrix

\subsection{Iterated GMM}

\subsection{Moment-covariance matrix}

\subsection{Bias and variance of the GMM estimator}

page on Bias and variance of the GMM estimator (cluster assumption should be part of moment condition?) part of later calculation of weighting?

Can do robust, hac, clustering as part of GMM too.


\newpage
	\section*{Catalan sequence}
	\textbf{Task}\\
	Let $c_n$ be the Catalan sequence. Find the limit $\lim\limits_{n \to \infty} \frac{c_{n+1}}{c_{n}}$\\
	\\
	\textbf{Solution}\\
	\begin{gather*}
		c_n = \frac{1}{n+1} {{2n}\choose{n}}
	\end{gather*}
	Then
	\begin{gather*}
		\lim\limits_{n \to \infty} \frac{c_{n+1}}{c_{n}} =
		\lim\limits_{n \to \infty} \frac{\frac{1}{n+2} {{2n+2}\choose{n+1}}}{\frac{1}{n+1} {{2n}\choose{n}}} = \\ \\
		\lim\limits_{n \to \infty} \frac{\frac{1}{n+2} \frac{(2n+2)!}{(n+1)!((2n+2) - (n+1))!}}{\frac{1}{n+1} \frac{(2n)!}{n!((2n) - (n))!}} =
		\lim\limits_{n \to \infty} \frac{\frac{1}{n+2} \frac{(2n+2)!}{(n+1)!(n+1)!}}{\frac{1}{n+1} \frac{(2n)!}{n!n!}} = \\ \\
		\lim\limits_{n \to \infty} \frac{(n+1) \frac{(2n+2)!}{(n+1)!(n+1)!}}{(n+2) \frac{(2n)!}{n!n!}} =
		\lim\limits_{n \to \infty} \frac{(n+1) (2n+2)!n!n!}{(n+2) (2n)!(n+1)!(n+1)!} = \\ \\
		\lim\limits_{n \to \infty} \frac{(n+1)(2n+1)(2n+2)}{(n+2)(n+1)(n+1)} =
		\lim\limits_{n \to \infty} \frac{2(n+1)(2n+1)}{(n+2)(n+1)} = \\ \\
		\lim\limits_{n \to \infty} \frac{4n+2}{n+2} =
		\lim\limits_{n \to \infty} \frac{4+\frac{2}{n}}{1+\frac{2}{n}} = 4
	\end{gather*}