\problemname{Wiseguy}

\noindent Paulie Cicero is a made man who runs the underworld of Brooklyn along with his associates Jimmy ``the Gent" Conway, Tommy DeVito, and the young but ambitious Henry Hill. The crew just got wind of a one-in-a-lifetime opportunity to raid \$6 million from the Lufthansa vault at John F. Kennedy International Airport. If they plan carefully and succeed, this will go down as the greatest heist of all time.\\

Jimmy plans to hire $N$ new recruits for the operation, whom he plans to organize into a hierarchy of leadership. Recruits arrive one-by-one on a rolling basis. The first recruit has no leader, but every subsequent recruit that comes in must immediately be assigned to exactly one boss (who must be some previous recruit already in the hierarchy). Henry points out that for things to go smoothly, each recruit should in the end only be assigned $0$, $1$, or $2$ subordinates -- either a left-hand man, or a right-hand man, or neither, or both. Setting a limit at $2$ will reduce each individual's responsibility of having to manage lots of people.\\

Each recruit also has a distinct ``strength" level (say, as a distinct integer from $1$ to $N$), which Jimmy will evaluate and consider as he is assigning leadership. We can assume that the strength levels across all of the recruits are uniformly random, and that recruits arrive in no particular order of strengths. That is, the final sequence of strengths among the $N$ recruits ordered from oldest to newest is drawn uniformly randomly from the set of all $N!$ possible permutations.\\

Paulie knows from experience that having a strong recruit leading other weaker recruits is a bad idea (the boss might abuse the subordinates), as is having a weak recruit leading only stronger recruits (being more qualified than your boss can lead to insubordination). Henry came up with a simple rule to mitigate this problem and bring some balance: For each recruit $i$, the left-hand man (if he exists) should always be weaker than recruit $i$ himself. Conversely, the right-hand man (if he exists) should always be stronger than recruit $i$ himself.\\

Each new recruit that comes in is first handed to the first recruit (except the first recruit himself). Based on Henry's rule, the new recruit is passed down to become either the left- or right-hand man of the current recruit that has custody of him. If there is already a left- or right-hand man, then the recruit is passed down further. This repeats until there are no conflicts and the new recruit is settled in as a subordinate of an existing recruit who previously only had either $0$ or $1$ subordinates. The diagram below illustrates the boss assignment process for $N = 4$ recruits, who arrive in the following order of strengths: $3$, $1$, $4$, $2$.\\

\begin{center}
\includegraphics[width=300pt]{wiseguy}
\end{center}

While Henry's strategy of limiting the number of subordinates to $2$ per person is good for distributing responsibility, the hierarchy can also become quite ``vertical". Having excessively long chains of command can lead to miscommunication and broken telephones, which the crew cannot afford in a delicate operation like this. The ``height" of a hierarchy is defined as the maximum number of times that a message needs to be passed from the first recruit to be able to reach any other recruit in the hierarchy. For example, the height of a hierarchy with a single leaderless recruit is 0, and the height of the last hierarchy in the above diagram is $2$.\\

Given the uniform randomness of new recruit strengths as well as the rules above for assigning leaders, Henry needs your help in finding out the expected height of the final hierarchy. This can be thought of as the ``average" height of hierarchies across all possible arrival orders. For example, in the case of having to organize $N = 2$ recruits, both possible hierarchies have a height of $1$. In the case of $N = 3$, two of the possible hierarchies have a height of $1$, and the remaining four possible hierarchies have a height of $2$, resulting in an expected height of $(1 + 1 + 2 + 2 + 2 + 2)/6 = 5/3 \approx 1.66667$.\\

Please help Henry predict the height for a hierarchy of $N$ recruits, so he can decide whether the plan will be feasible. A true wiseguy never gets caught, but a bad judgment or miscommunication in the ranks can easily bring shame to your crew. You'd better not mess up, or you might as well listen to Billy Batt's advice for Tommy -- quit the mob life, go home and get your shine box.\\

\section*{Input}
The first line of input consists of a single integer $T$ ($1 \leq T \leq 500$), specifying the number of test cases to follow.\\
$T$ lines follow, each of which is a test case consisting of a single integer $N$ ($1 \leq N \leq 500$), specifying the number of recruits that will need to be organized for the heist.\\

\section*{Output}
For each test case, print, on a separate line, a single real number denoting the expected height for the hierarchy.\\
Note: your answer must have at most $10^{-5}$ absolute or relative error to be considered correct.\\


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{tcolorbox}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{xcolor}
\usepackage{float}
\usepackage[
top    = 2.50cm,
bottom = 2.50cm,
left   = 2.75cm,
right  = 2.75cm]{geometry}
\usepackage{fancyhdr}
\pagestyle{fancy}
\lhead{Analysis 1}
\rhead{EPFL/Alp Ozen}

\newtheorem{remark}{Remark}[section]
\newtheorem{theorem}{Theorem}[section]
\newtheorem{prop}{[Proposition]}
\newtheorem{definition}{Definition}
\numberwithin{equation}{subsection}

\title{Analysis}
\author{SemesterOne analysis at EPFL}
\date{\vspace{-5ex}}
\newtheorem{example}{Example}[section]
\newtheorem{axiom}{Axiom}
\newtheorem{cor}{Corollary}

\begin{document}

\maketitle

\section{Proofs and the reals}
\subsection{Some general proofs}
A valid proof is set of lines where each line logically follows from the next.
A most famous proof is that $\sqrt{2} $ is irrational.

\begin{tcolorbox}
\begin{proof}
	Suppose that $\sqrt{2} = \frac{a}{b}$ where $a,b \in \mathbb{Z}$ and $gcd(a,b) = 1$
	\\
	Now we have that $\sqrt{2}b = a$ which means that $2b^2 = a^2$.
	As result, $2 \vert a^2$ hence also $2 \vert a$. Thus we get that $a = 2k$ which also means that $b^2 = 2k^2$ hence $2 \vert b$. As result, $gcd(a,b) = 2$ which is a contradiction. Therefore, $\sqrt{2}$ must be irrational.
	\end{proof}
\end{tcolorbox}

Quite interestingly, we can also construct a 'wrong' proof just through one fallacious assumption and a set of correct steps.

\begin{tcolorbox}
    \textbf{Claim:} 1 is the largest integer.
    \\
    Proof:
    \\
    Let $n$ be the largest integer. Then we have $n \geq n^2$. Which also means $0 \geq n^2 - n = n(n-1)$. Now we have that either $n < 0$ or $n - 1 < 0$. But we know that $ n \nless 0$ as $n$ is at least 1. Hence, $n - 1 < 0$ giving us the result $n < 1$ proving our theorem. Note that the mistake here is solely the assumption we made at the start that there was a largest integer.
\end{tcolorbox}
\subsection{Proofs relating to infinite processes}
Consider the claim that $0.999\ldots = 1$.
One way to prove this claim, rather naively is this.

\begin{align*}
    9 \times 0.999\ldots \\
    = (10 - 1) \times 0.999\ldots\\
    = 9.999\ldots - 0.999\ldots
    = 1
\end{align*}

Now a more formal proof is to use an infinite sum and limits. Here it is.

\begin{tcolorbox}
\textbf{Analysis proof of 0.999\ldots = 1}
    \begin{align*}
        0.999\ldots = 9\lim_{k\to\infty}\sum_{i=1}^{k} (10^{-k})\\
        \lim_{k\to\infty}\sum_{i=1}^{k}(10^{-k}) = \frac{10^{-1} - 10^{-(k+1)}}{1-10^{-1}}\\
        = 9 \times \frac{1}{10} \times \frac{10}{9}\\
        = 1
    \end{align*}
\end{tcolorbox}

\subsection{Basic notions of sets}
The breakdown of sets used in 'standard' analysis are $\mathbb{N} \subseteq \mathbb{Z} \subseteq \mathbb{Q} \subseteq \mathbb{R}$.
\\
There are also some common set related notation that must be known.
\begin{itemize}
    \item a \textbf{subset} $a \subseteq b$ is defined as $\{x \in b \vert \text{"condition"}\}$
    \item a \textbf{open interval} is defined as $\left]a,b[; r\in A, a < r < b$
    \item an \textbf{open ball} $B(a,\lambda) = \left]a-\lambda,a+\lambda[$
\end{itemize}

\subsection{The Reals}

The reals, denoted \mathbb{R} are an ordered field. Here is a more precise definition.

\begin{tcolorbox}
The reals are a set that have the 3 following axioms:
\begin{itemize}
    \item \mathbb{R} is an abelian group under (+) and \mathbb{R*} is an abelian group under ($\times$). In addition to this, multiplication distributes over addition.
    \item The order relation $\leq$ holds $\forall x \in \mathbb{R}$ That is:
    \begin{align*}
        x \leq y \otimes y \leq x\\
        x \leq y, y \leq x \implies x = y\\
        x \leq y \implies \forall a \in \mathbb{R},\  x + a \leq y + a\\
        0 \leq x, 0 \leq y \implies 0 \leq xy
    \end{align*}
    \item The inf and sup axioms hold
\end{itemize}

We shall now come the \textbf{inf} and \textbf{sup} axioms. It should be intuitively clear that any subset of \mathbb{R}
\end{tcolorbox}

\subsection{Bounds}
Take some subset S in \mathbb{R}. An element B is called an upper bound of S if $\forall x \in S, B \geq x$. Similarly, it is a lower bound of S if $\forall x \in S, B \leq x$.
\\
The maximum B of a set S denoted $max(S)$ is such that $B \in S, \forall x \in S B \geq x$.
\\
The supremum of a set S(if it exists) is the lowest upper bound. That is $sup(S) = b$ is such that,

    \begin{align}
        \forall x \in S, b \geq x\\
        \forall \epsilon > 0, \exists x_{\epsilon}, b - x_{\epsilon} \leq \epsilon
    \end{align}

\begin{remark}

   In our above definition, b does not have to be in S.

\end{remark}

\begin{remark}
    Condition 1 states that b is an upper bound of S.

\end{remark}

\begin{remark}
       Given condition 1, b is the \textit{minimum} of the upper bounds of S.
\end{remark}

Some examples of $sup$ and $inf$

\begin{example}
\begin{align*}
 Sup ]a,b[ = b\\
 Inf ]a,b[ = a\\
 Sup \{ x\in \mathbb{R} \vert x = 2k\} \implies \text{Sup doesn't exist.}
\end{align*}
\end{example}

We now establish the infinimum axiom.
\begin{axiom}
All non-empty subsets of $\mathbb{R}_{+}^{*}$ have a highest lower boundary(aka.infinimum)
\end{axiom}

\subsection{\mathbb{Q} is dense in \mathbb{R}}
We claim that between every real number, one can find a rational number. Here's the proof,

\begin{tcolorbox}
\begin{proof}
    Let $x<y \in \mathbb{R}$ Suppose now that $\exists a \in \mathbb{Q}$ such that $x<a<y$. By the Archimedean principle(there is always a greater natural number, $n > \frac{1}{y-x}$ which implies $ny>nx+1$ Now since $ny>nx+1$, there is guaranteed to be some integer in the open bound $]nx,ny[$ which we denote $P$. Dividing by $n$, we get that $\frac{P}{n} \in ]x,y[$ which proves the theorem.
    \tag*{\qedhere}
\end{proof}
\end{tcolorbox}

\subsection{Integer and fractional part}

Any number $\in \mathbb{R}$ has a integer and fractional part(at least intuitively). Let's formally define these. For some $x \in \mathbb{R}$, let $S := \{n\in\mathbb{N} \vert n > x\}$ Now since $S$ is bounded from below, letting $N$ be the minimum of this set, we obtain that $N \not\in S$. N-1 is thus called the integer part of x denoted [x]. ie. [6.4] = 6
\\
Similarly, the fractional part of x denoted {x} is simply ${x} = x - [x]$

\subsection{Pinning it down:Sup/Inf, bounds, max/min}
\begin{definition}
for a given set $S \subseteq \mathbb{R}$, we have the following:
\\
\textbf{Sup s = b} $\iff \forall \epsilon > 0, \exists x_{\epsilon} \in S, s.t. b - x_{\epsilon} < \epsilon$ (resp. Inf s has the flipped argument)
\\
\textbf{Upper bound = b} $\iff \forall x \in S, b\geq x$(resp. lower bound)
\\
\textbf{Max s = b} $\iff$ b is an upper bound and $b \in S$

\end{definition}
\\

And for the sake of repeating the early axiom(but very important) the infimum axiom is:
\begin{axiom}
For all non-empty subsets of $\mathbb{R}$, $inf S$ exists.
\end{axiom}

Now we make the first claim in this course that uses an epsilon proof.
\begin{tcolorbox}

\begin{prop}
Whenever $S \subseteq \mathbb{N}$, then $inf S = min S$
\end{prop}

\begin{proof}
    Now, by our axiom, we have that $\mathbb{N} \subseteq \mathbb{R}$ hence we know that $inf S$ exists. We now have to show that $inf S = min S$.
    \\

    Suppose that $inf S \not =  min S$ and let $inf S = b$. Now clearly, $b+\epsilon$ is not a lower bound of S. Now, let $\epsilon = \frac{1}{2}$. Because, $b + \epsilon$ is not a lower bound, we know that $\exists s_{e} < b + \epsilon$.
    \\

    Now $s_{e$ is also not a lower bound, so let's pick $\epsilon^{\prime\prime} = s_{\epsilon} - d$. Now again, $s_{\epsilon^{\prime\prime}}$ must exist. We obtain yet the following:
    \\
    \begin{equation*}
        d < s_{\epsilon^{\prime\prime}} < s_{\epsilon} < d + 1/2
    \end{equation*}

    Now, two natural numbers clearly can not be in an interval which is only $\frac{1}{2}$ units long. Hence, contradiction which means that $d \in S$
    \tag*{\qedhere}
\end{proof}

\end{tcolorbox}
\\
Let's now prove that $\sqrt{2}$ belongs to the reals. For this, we need the following corollary and axiom.

\begin{cor}
Every non-empty subset of \mathbb{R} with an upper boundary admits a supremum.
\end{cor}

\begin{axiom}
An ordered field F, which \mathbb{R} is, has the Archimedean property if given any positive x and y in F, $\exists n \in \mathbb{Z}$ s.t. $nx > y$
\end{axiom}

\begin{tcolorbox}

\begin{prop}
$\sqrt{2} \in \mathbb{R}$
\end{prop}

\begin{proof}
Suppose we define a set $S = \{r \in \mathbb{R} \vert r \geq 0, r^{2} < 2\}$. Now, as $S$ is a non-empty subset of $\mathbb{R}$ bounded from above(i.e. 2 is an upper bound) we know that $sup S = x$ exists. Our goal is to show that both $x^{2} < 2$ and $x^{2} > 2$ lead to a contradiction.
\\

\textbf{Case 1:} Suppose $x^{2} < 2$. We want to find $x + \frac{1}{2} \in S$ which implies that $x$ is not an upper bound as $x < x + \frac{1}{2}$
\\
$$(x+\frac{1}{2})^2 = x^2 \frac{2}{x} + \frac{1}{n^2} \leq x^2 + \frac{2}{x} + \frac{1}{n} = x^2 + \frac{1}{n}(2x + 1) $$
\\
Now, we want to show that we can pick an $n$ s.t. $ x^2 + \frac{1}{n}(2x + 1) < 2$. If we can pick such an $n$, then we know by transitivity of $<$ that $(x+\frac{1}{n})^2 < 2$ as  $x^2 \frac{2}{x} + \frac{1}{n^2} \leq x^2 + \frac{2}{x} + \frac{1}{n}$
\\
Reordering the terms, we get $\frac{1}{n} < \frac{2 - x^2}{2x + 1}$ and clearly, $\frac{2 - x^2}{2x + 1}$ is positive as $x^2 < 2$ and $x \geq 0$. This way, we apply the archimedean property to know that $n$ exists s.t. $\frac{1}{n} < \frac{2 - x^2}{2x + 1}$. Given this, we now know that  $ x^2 + \frac{1}{n}(2x + 1) < 2$ which in turn implies $x + \frac{1}{n} \in S$ This contradicts that $x = sup S$ hence $x^2 \not < 2$
\\

\textbf{Case 2:} In turn, we consider the case where $x^2 > 2$ and try to derive a contradiction. We want to show that $\exists m \in \mathbb{N}$ s.t. $x - \frac{1}{m}$ is also an upper bound of $S$ which would mean that $x \not = inf S$. Now:

$$ (x - \frac{1}{m})^2 = x^2 - \frac{2x}{m} + \frac{1}{m^2} > x^2 - \frac{2x}{m}$$

We want to choose $m$ s.t. $x^2 - \frac{2x}{m} > 2$. This way, if $x^2 - \frac{2x}{m} > 2$ holds, then since $(x - \frac{1}{m})^2 > x^2 - \frac{2x}{m}$, we will have that $x - \frac{1}{m}$ is an upper bound. We obtain:
$$\frac{x^2 -2}{2x} > \frac{1}{m}$$
\\
Now as $\frac{x^2 -2}{2x}$ is positive, ${1}{m}$ dpes exist. Hence, $x - \frac{1}{m}$ is also an upper bound that implies $x \not = sup S$ if $x^2 > 2$.
\\
Therefore, $sup S = x = \sqrt{2}$ and since every $sup S \in \mathbb{R}$, $\sqrt{2} \in \mathbb{R}$

    \tag*{\qedhere}
\end{proof}
\end{tcolorbox}

\subsection{More theorems about \mathbb{R}}

\begin{tcolorbox}
\begin{prop}
If $a < b$ are real numbers, then $\exists c \in \mathbb{Q}, a < c < b$
\end{prop}

\begin{proof}
Now our goal is to show that for any real number $a,b$ we can always find such a $c$.
Now take some arbitrary $n$ and set it to $n = [ \frac{1}{b-a}] + 1$ Now clearly, $n>\frac{1}{b-a}$ and hence $\frac{1}{n} < b-a$. We will now use this result. Realize that $a = \frac{an}{n} < \frac{[an + 1]}{n} \leq \frac{an + 1}{n} = a + \frac{1}{n} < a + b - a = b$ Therefore, we have found that $a < \frac{[an]+1}{n}< b$ where $c = \frac{[an]+1}$

    \tag*{\qedhere}
\end{proof}
\end{tcolorbox}

\begin{tcolorbox}
\begin{prop}
If $a < b$ are real numbers, then $\exists c \in \mathbb{R\backslash Q}, a < c < b$
\end{prop}

\begin{proof}
Using the above proposition, we know that $\exists c$ for any $\frac{a}{\sqrt{2}} < \frac{b}{\sqrt{2}} $ and now we get that $a < \sqrt{2}c < b $ where $\sqrt{2}c$ is irrational as whenever one term in $x\times y = z$ is rational and the other irrational, we have (supposing $x$ is irrational) $x = \frac{z}{y}$ and if also $z$ were rational, it would make $x$ rational which is a contradiction.
    \tag*{\qedhere}
\end{proof}
\end{tcolorbox}

\begin{definition}
The absolute value function $x$ is a function $f: \mathbb{R} \to \mathbb{R^{+}}$ such that:

\[ \begin{cases}
      f(x) = x , x \geq 0 \\
      f(x) = - x , x < 0
   \end{cases}
\]
\end{definition}

Absolute value respects multiplication and division, that is:

\begin{align*}
    |a||b| = |ab|\\
    \frac{|a|}{|b|} = |\frac{a}{b}|
\end{align*}

But this doesn't hold for addition. For addition we have the triangle inequality:

\begin{equation*}
    |x+y| \leq |x| + |y|
\end{equation*}

To prove the above:

\begin{proof}
Take $|x+y| < 0$. Then $|x+y| = -(x+y) = -x -y \leq |x| + |y|$ Take $|x+y| \geq 0$. Then $|x+y| = x + y \leq |x| + |y|$
    \tag*{\qedhere}
\end{proof}
\clearpage
\section{Sequences}

\subsection{Basics}
\begin{tcolorbox}
Let's begin by formally defining sequences. A sequence is a function $f:\mathbb{N} \to \mathbb{R}$ generally denoted $(x_{n})_{n\geq0}$ And here are some more definitions on sequences:
\\
A sequence is:
\begin{itemize}
    \item constant if $\exists \ C \in \mathbb{R};\ x_{n} = C \ \forall \ n \in \mathbb{N}$
    \item bounded from below(resp. above) if $\exists \ m \in \mathbb{R} \ ; m \leq x_{n} \forall \ n \in \mathbb{N}$
    \item bounded if bounded from both directions
    \item increasing(resp. decreasing) if $x_{n+1} \geq x_{n} \forall \ n \in \mathbb{N}$
    \item strictly increasing(resp.decreasing) if $x_{n+1} \leq x_{n} \forall \ n \in \mathbb{N}$
    \item monotonous if it is increasing or decreasing(resp. strictly)
\end{itemize}
\end{tcolorbox}

Let's now consider a proof on the following proposition:

\begin{tcolorbox}
Define $x_{n} = \sqrt{4 + x_{n-1}}, \ x_{0} = 1$. We claim that $x_{n}$ is bounded and more precisely that $1 \leq x \leq 3$.

\begin{proof}
Base case: $x_{0} = 1$ hence holds.
\\
Now supposing proposition is true for all $n-1$ we get

$$ 1 \leq x_{n-1} \leq 3 $$
$$ 5 \leq x_{n-1} + 4 \leq 7$$

Now we get:

$$ \sqrt{5} \leq \sqrt{x_{n-1} + 4} \leq \sqrt{7}$$
$$ 1 \leq \sqrt{5} \leq \sqrt{x_{n-1} + 4} \leq \sqrt{7} \leq 3 $$
\tag*{\qedhere}
\end{proof}
\end{tcolorbox}

\begin{definition}
A sequence $x_{n} $\mathbb{conveges} to $x$ if $\forall \epsilon > 0$ we can find $n_{0} \in \mathbb{N}$ such that $n\geq n_{0} \rightarrow |x_{n} - x| \leq \epsilon $
\end{definition}

Now in an intuitive sense, suppose the sequence converges to $x$ from both the right and the left. $x$ being our central point, we move a distance $\epsilon$ away from this $x$. Now, if we can pick some $n_{0}$ such that for another $n \geq n_{0}$ we have that $|x_{n} - x| \leq \epsilon$ this means that no matter how small we make epsilon we are able to find some $x_{n}$ in this region.
\\
Having established convergence, any sequence that is not convergent is said to be \textbf{divergent}.
\\
Let's now prove that a sequence is divergent.

\begin{tcolorbox}

\begin{proof}

Take $x_{n} = (-1)^{n}$ Now suppose that $x_{n}$ converges to $x$. Then for $\epsilon = \frac{1}{2}, \ \exists n_{\frac{1}{2}} \in \mathbb{N}$ s.t. $\forall n \geq n_{\frac{1}{2}}$ we would have $|x_{n} - x| \leq \frac{1}{2}$. In particular if $n^{\prime}$ is any other integer $n^{\prime}>n_{\frac{1}{2}}$ then we would have $x_{n} - x_{n^{\prime}} \leq |x_{n} - x| + |x - x_{n^{\prime}}| \leq \frac{1}{2} + \frac{1}{2}$ which implies a contradiction as $|x_{n} - x_{n+1}| = 2 > 1$
\tag*{\qedhere}
\end{proof}
\end{tcolorbox}

\subsection{Limits and their algebra}
\begin{definition}
If a sequence $x_{n}$ converges to some $x$, we say that $x$ is the \textbf{limit} of the sequence and is denoted $\lim_{n\to \infty}{x_{n}} = x $
\end{definition}

Here are some properties of limits:

\begin{tcolorbox}
For sequences $x_{n}$ and $y_{n}$ with limits $x,y$ we have:
\begin{itemize}
    \item $$\lim_{n \to \infty} {x_{n} \cdot y_{n}} =  x\cdot y $$
    \item $$\lim_{n \to \infty} {x_{n} + y_{n}} =  x+y $$
    \item $$\lim_{n \to \infty} \frac{x_{n}}{y_{n}} = \frac{x}{y} ; y \not = 0$$
    \item if $\exists \ n_{0} \in \mathbb{N}, \ x_{n} \leq y_{n} \ \forall n \geq n_{0}$ then $x\leq y$
\end{itemize}
\end{tcolorbox}

We now introduce the famous \textbf{squeeze theorem} and prove it.

\begin{theorem}
Squeeze theorem
\\
Let $a_{n}$ and $b_{n}$ be both sequences that converge to $a$. In addition, let $c_{n}$ be such that $ \exists n_{0} \in \mathbb{N}, \ \forall n \geq n_{0}, \ a_{n} \leq c_{n} \leq b_{n}$ Then we clearly have the following:
\begin{proof}
\begin{equation}
\label{1}
\forall \epsilon > 0, \ \exists N_{1}, \ n \geq N_{1} \rightarrow |a_{n} - a| < \epsilon \equiv a - \epsilon < a_{n} < a + \epsilon
\end{equation}
Similarly, for $b_{n}$ we have that:
\begin{equation}
\label{2}
     \forall \epsilon > 0, \ \exists N_{2}, \ n \geq N_{2} \rightarrow |b_{n} - a| < \epsilon \equiv b - \epsilon < b_{n} < a + \epsilon
\end{equation}
Now set $N = \max{\{N_{1},N_{2},n_{0}\}}$. Now since $N \geq N_{1},N_{2},n_{0}$ we have that both \ref{1} and \ref{2} hold $\forall n > N$ This further gives us the result that:

\begin{align*}
    a - \epsilon < a_{n} \leq c_{n} \leq b_{N} < a + \epsilon \\
    |c_{n} - a| < \epsilon
\end{align*}
\tag*{\qedhere}
\end{proof}
\end{theorem}

We now list some useful inequalities that may be used along with the squeeze theorem and also a sample limit problem and a solution to it:

\begin{figure}[h]
    \centering
    \includegraphics[scale = 0.9]{epflSemesterOne/analysis/figures/usefula.JPG}
    \caption{Useful inequalities}
    \label{fig1}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics{epflSemesterOne/analysis/figures/usefula2.JPG}
    \caption{Solution to hard limit problem}
    \label{fig2}
\end{figure}

\subsection{More on sequences}
Suppose we want to show that some geometric sequence does not have a limit, simply that it is not converging.

We first establish the \textit{Bernoulli inequality} which we shall prove later. We will use this result immediately.

\begin{equation}
    q^n \geq 1 + n(q-1)
\end{equation}

\begin{tcolorbox}
Take $x_{n} = 4^n$. Now we claim that $x_{n}$ is not bounded.
\begin{proof}
It is not bounded if we can show that it increasing in increments that do not decrease. Suppose now that $b$ is some upper bound to $x_{n}$. Now by Bernoulli, we have that $4^n \geq 1 + n^\cdot3$. If we can show that $4^n \geq 1 + n\cdot3 > b$ we indeed get that there can be no upper bound $b$. We have that $n > \frac{b-1}{3}$ and such an $\n \in \mathbb{N}$ exists if we set it to $n:= [\frac{b-1}{3} + 1]$ Now since $4^n \geq 1 + n\cdot3$ we have that $x_{n}$ is not bounded.

\end{proof}
\end{tcolorbox}

\begin{theorem}
Every converging sequence is bounded(a lower or upper bound exists obviously and if convergent the latter also exists.)
\end{theorem}

\clearpage

Let's now get on to proving the rules we established for limit arithmetic.

\begin{proof}(Proof to sum rule)
Now we are given that $x_{n}$ converges to $x$ and that $y_{n}$ converges to $y$. We want to show that $x_{n} + y_{n}$ converges to $x+y$. By definition, this is true if we can show that
\begin{equation*}
    |(x_{n} + y_{n}) - (x+y)| \leq \epsilon, \ \forall \epsilon \in \mathbb{R}
\end{equation*}

Now luckily we have that $|(x_{y} + y_{n}) - (x+y)| = |(x_{n} - x) + (y_{n} - y)|$ And by the triangle inequality we know that $|(x_{n} - x) + (y_{n} - y)| \leq |x_{n}-x| + |y_{n} - y|$ Thus if we can show that $|x_{n}-x| + |y_{n} - y| < \epsilon$ we are guaranteed that $|(x_{n} - x) + (y_{n} - y)| < \epsilon$ We will succeed with the latter part if we can show that both parts $(x_{n} - x)$ and $(y_{n} - y)$ are smaller than $\frac{\epsilon}{2}$. Now we have by definition of convergence that:

$$ \exists n \geq n_{\frac{\epsilon}{2}}^{x} \rightarrow |x_{n} - x| \leq \frac{\epsilon}{2}$$
Similarly:

$$ \exists n \geq n_{\frac{\epsilon}{2}}^{y} \rightarrow |y_{n} - y| \leq \frac{\epsilon}{2}$$
Now fixing $n_{\epsilon} := \max{n_{\frac{\epsilon}{2}}^{x}, n_{\frac{\epsilon}{2}}^{y}}$(we do this since it assures both conditions to hold we have that because each of $(x_{n} - x)$ and $(y_{n} - x)$ are smaller than $\epsilon$, so must $|x_{n}-x| + |y_{n} - y|$ by the triangle inequality.
\end{proof}

Let's now do more applications of the squeeze theorem to show limits.
\begin{example}
We want to show that $aq^{n}$ converges to $0$ for $a\not = 0$ and $|q|<1$. Now as a property we use that $\lim_{n\to\infty}{x_{n}} = 0 \rightarrow \lim_{n\to\infty}{|x_{n}|}=0$ It is clear that we have an inequality of the form $0\leq|aq^{n}|\leq ?$ Doing more algebra(our goal is to find another sequence of form $\frac{1}{x}$ converging to 0 to get:

$$ \frac{1}{?} \leq \frac{1}{|a\cdot q^n|} = \frac{1}{|a|} \cdot (\frac{1}{|q|})^n$$

Now using Bernoulli we have:

$$(\frac{1}{|q|})^n \geq 1 + n(\frac{1}{|q|}-1)$$ which happily means:

$$(\frac{1}{a}\cdot\frac{1}{|q|})^n \geq (\frac{1}{a}\cdot(1 + n(\frac{1}{|q|}-1))$$
And finally taking the reciprocal all to get back to $aq^n$ we are left with

$$ 0 \leq |aq^n| \leq \frac{1}{\frac{1}{a}\cdot(1+n(\frac{1}{|q|}-1)}$$ and clearly we see that the RHS is also a sequence that converges to 0 since all terms in the denominator but $n$ are constants. Hence, we have \textbf{squeezed} our sequence. The key here was that we found a RHS sequence which we wanted to be of form $\frac{1}{x}$ And in addition, we took the reciprocal of the inequality at the start simply to be able to use the bernoulli inequality.
\end{example}

Let's now consider a harder example.

\begin{example}
Consider the sequence $x_{n} = \sqrt[n]{n}$ Now is this sequence converging? Well we know that $1\leq\sqrt[n]{n}$ and now another sequence we know which approaches to 1 is $1 + \frac{1}{\sqrt{n}}$. Now we only need to show that:

$$\sqrt[n]{n} \leq 1 + \frac{1}{\sqrt{n}} $$ holds and if we can show this, we'll have that our sequence converges to 1.

Now we get:
$$ n \leq (1 + \frac{1}{\sqrt{n}})^n$$ And notice how $(1 + \frac{1}{\sqrt{n}})^n$ is simply a binomial hence if any one of the terms in the binomial expansion is greater than $n$ our inequality will hold(this works as all $n$ are positive). Well for $\sum_{i=0}^{n}{n\choose i}(\frac{1}{\sqrt{n})^{i}}$ when observe that for $i=4$ we get:

$$\frac{(n-1)(n-2)(n-3)}{24n} \geq n$$ further reducing to:
$$\frac{24n^2}{(n-1)(n-2)(n-3)} \leq 1$$ and we know this is valid as

$$\lim_{n\to\infty}{\frac{24n^2}{(n-1)(n-2)(n-3)}} = 0$$

Hence we get that $$\sqrt[n]{n} \leq 1 + \frac{1}{\sqrt{n}} $$ meaning that $\sqrt[n]{n}$ converges to 1.
\end{example}

Let's do one last example:

\begin{example}
Consider $x_{n} = \frac{2^n}{x!}$ Now clearly $0\leq\frac{2^n}{x!}$ and observing that $\frac{2^n}{x!} = \frac{2}{1} \cdot \frac{2}{2} \cdot \frac{2}{3} \ldots $ it is obvious that $\frac{2^n}{x!} \leq 2 \cdot (\frac{2}{3})^{n-2}$ But we know that $\lim_{n\to\infty}{2 \cdot (\frac{2}{3})^{n-2}}=0$ hence we have again squeezed our sequence!
\end{example}

\begin{theorem}
If $\lim_{n\to\infty}{x_{n}}=0$ and $y_{n}$ is bounded, then $\lim_{n\to\infty}{x_{n}\cdot y_{n}}=0$
\end{theorem}

\begin{theorem}
\label{monotone}
If a sequence is bounded and increasing(monotone) it converges to the supremum (resp. infimum).
\end{theorem}

\begin{proof}
Now because $y_{n}$ is bounded, we have that $\exists M$ s.t. $M\geq y_{n} \forall n$. Hence we obtain that $x_{n}y_{n} \leq Mx_{n}$ and clearly since $Mx_{n}$ converges to $0$ we have squeezed $x_{n}y_{n}$ and have that it also converges to 0.
\end{proof}

We consider the famous sequence of \textbf{Fibonacci quotients}
\\
Now the fibonacci sequence is defined as $x_{0}=x_{1} = 1$, $x_{n+1} = x_{n} + x_{n-1}$ and we define the sequence of fibonacci quotients as $y_{n} = \frac{x_{n+1}}{x_{n}}$ Our first theorem is that fibonacci quotient sequence is bounded between 1 and 2. Let's now fnd it's limit. Notice firstly that $y_{n+1} = 1 + \frac{1}{y_{n}}$
    $$y = \lim_{n \to \infty}{y_{n}} = \lim_{n \to \infty}{y_{n+1}} = 1 + \frac{1}{\lim_{n\to \infty}{y_{n}}} $$ Hence we get:

    $$ y = 1 + \frac{1}{y}$$ which gives $\frac{1+\sqrt{5}}{2}$ as the only valid solution. But we still have to show that our sequence converges. A smart way to do so is to show that $z_{n} := |y_{n} - \frac{1+\sqrt{5}}{2}|$ converges which by limit arithmetic would imply $\lim_{n \to \infty}{y_{n}} = \frac{1+\sqrt{5}}{2}$ Our goal is to squeeze $z_{n}$ in doing so. Notice that $z_{n+1} = |y_{n+1} - \frac{1+\sqrt{5}}{2}|= \underbrace{1 + \frac{1}{y_{n}} - (1 + \frac{2}{1+\sqrt{5}})}_{\text{ using definition }} $ This yields the inequality $z_{n + 1} = \frac{|y_{n} - \frac{1+\sqrt{5}}{2}|}{y_{n}\frac{1+\sqrt{5}}{2}}= \frac{2}{1+\sqrt{5}}\frac{|y_{n - \frac{1+\sqrt{5}}{2}}|}{y_{n}}$ to give us that $z_{n+1} \leq \frac{2}{1+\sqrt{5}}|y_{n - \frac{1+\sqrt{5}}{2}}|}$ because we know that $y_{n} \geq 1$. Now finally this simplifies to $z_{n+1} \leq \frac{2}{1+\sqrt{5}}z_{n}$ which applying the definition of $z_{n}$ gives $z_{n} \leq \frac{2}{1+\sqrt{5}}^{2}z_{n-1} $ to result in $z_{n} \leq \frac{2}{1+\sqrt{5}}^{n}z_{0}$ which converges to 0. Hence we have squeezed $z_{n}$

    \clearpage

    In general this is the scheme one should use for finding limits of recursive sequences:

     \begin{tcolorbox}[drop shadow, title = (Limit of recursive sequence),lower separated=true]
     \begin{enumerate}
         \item Assuming there is a limit, compute it using limit algebra.
         \item Showing some upper and lower bound(using induction) exclude any extra answers.
         \item Finally show that the sequence converges by showing that $\lim_{n\to\infty}{x_{n}-x}=0$
     \end{enumerate}
\end{tcolorbox}

We now come to define \textbf{approaching infinities}.
\begin{definition}
The definition of approaching $\infty$ is as intuitive as saying for any real number I pick, the sequence has a term larger than it. Hence we define $\lim_{n\to \infty}{x_{n}} = \infty$ as $\forall A \in \mathbb{R}$ $\exists n_{A} \in \mathbb{N}$ such that $n\geq n_{A}$ and $x_{n} \geq A$. The similar definition applies for approaching $-\infty$
\end{definition}

\begin{example}
Notice that for a geometric sequence $x_{n} = aq^{n}$ if $a>0$ and $q>1$ it approaches $\infty$ and if $a<0$ and $q>1$ it approaches $-\infty$
\end{example}

A set of useful theorems on approaching infinities is the following:

\begin{theorem}(Theorems on approaching infinity)
\\
If $\lim_{n\to\infty}{x_{n}} = \infty$ and $y_{n}$ is bounded from below, then $\lim_{n\to\infty}{x_{n} + y_{n}} = \infty$
\\
If $x_{n}$ and $y_{n}$ both approach infinity, so does their product.
\\
If $y_{n}$ is bounded and $x_{n}$ approaches infinity, then $\lim_{n\to\infty}{\frac{y_{n}}{x_{n}}} = 0$
\end{theorem}

We note that when it is the case that $\lim_{n\to\infty}{x_{n}}=\infty$ and  $\lim_{n\to\infty}{x_{n}}=-\infty$ we may have different cases such as:

$$ \lim_{n\to\infty}\underbrace{n}_{x_{n}} + \underbrace{(-n)}_{y_{n}} = 0 $$

$$\lim_{n\to\infty}\underbrace{2n}_{x_{n}} + \underbrace{(-n)}_{y_{n}} = \infty$$

$$\lim_{n\to\infty}\underbrace{2n + (-1)^{n}n}_{x_{n}} + \underbrace{(-2n)}_{y_{n}} = (-1)^{n}n \text{ which is unbounded hence does not approach anything}$$

\begin{theorem}(Squeeze theorem for approaching infinities)
For sequences $x_{n}$ and $y_{n}$ if $\exists n_{0} \in \mathbb{N}$ such that $\forall n \geq n_{0}$, $x_{n} \leq y_{n}$ we have:
\\
(1) If $\lim_{n\to\infty}{x_{n}} = \infty$ then $\lim_{n\to\infty}{y_{n}} = \infty$
\\
(2) If $\lim_{n\to\infty}{y_{n}} = -\infty$ then $\lim_{n\to\infty}{x_{n}} = -\infty$
\end{theorem}

\begin{theorem}(D'Alembert Theorem)
\\
$\forall x_{n} \not = 0$ and $\lim_{n\to \infty}{|\frac{x_{n+1}}{x_{n}}|} = \infty$ then we have that $x_{n}$ diverges.
\end{theorem}

And now we provide an example of sandwich for infinities.

\begin{example}
Define $x_{n}:= \frac{x!}{2^{n}}$ We have that $\frac{n}{2} \cdot \frac{n-1}{2} \ldots \cdot \frac{3}{2} \ldots \geq \frac{n}{2} \cdot \frac{3}{2}^{q-1}$ for some $n$. And because the latter is a geometric sequence which noticeably approaches infinity we have that $x_{n}$ approaches infinity.
\end{example}

\begin{proof}(We present a proof to theorem \ref{monotone})
\\
Now set $S:=sup\{x_{n}|n\in\mathbb{N}\}$ and let $0<\epsilon\in\mathbb{R}$ By definition $S$ is the smallest bound hence $S-\epsilon$ is not the smallest bound. By def. again, $\exists n_{\epsilon}$ such that $S-\epsilon<x_{n_{\epsilon}}$ and we now get (for $n\geq n_{\epsilon}$
$$S - \epsilon < x_{n_{\epsilon}} < x_{n} < S < S+\epsilon$$ and this is exactly the definition of convergence hence $S$ is the limit.
\end{proof}
\clearpage

     \begin{tcolorbox}[drop shadow, title = (Exploration of e),lower separated=true]
   We define $x_{n} = (1 + \frac{1}{n})^n$ We first ask is $x_{n}$ increasing. Consider the claim $(1+\frac{1}{n})^{n} < (1+\frac{1}{n+1})^{n+1}$
   Now let's consider both the expansions of $(1 + \frac{1}{n})^n$ and of $(1 + \frac{1}{n+1})^{n+1}$

   $$ (1 + \frac{1}{n})^n = \sum_{i=0}^{n} \binom{n}{i} (\frac{1}{n^n}) = \sum_{i=0}^{n} \frac{1}{i!} \frac{n\cdot (n-1) \ldots (n-i+1)}{n^i} = \sum_{i=0}^{n} \frac{1}{i!} (1 - \frac{1}{n}) \ldots (1 - \frac{i-1}{n})    $$

   Similarly we have:
   $$ (1 + \frac{1}{n + 1})^{n+1} = \sum_{i=0}^{n} \frac{1}{i!} (1-\frac{1}{n+1}) \ldots  (1 - \frac{i-1}{n}) $$

   and because generally $\frac{
   q}{j} > \frac{q}{j+1}$ we have that terms on the RHS of the first expression are larger hence $(1+\frac{1}{n})^{n} < (1+\frac{1}{n+1})^{n+1}$ And we also have that our sequence is less than 3(which we do not show).

   Thus we define $$\lim_{x \to \infty}{(1+\frac{1}{n})^{n}} = e$$
\end{tcolorbox}

And now we present an example of a bounded sequence that is decreasing which converges to its infimum. Consider $x_{n+1} = \frac{1}{2}(x_{n} + \frac{1}{x_{2}})s$ and $x_{0}= 2$.
\\
We claim 1 is a lower bound which is true by induction(easy to show). Similarly, the sequence is decreasing as $x_{n} - x_{n+1} = \frac{1}{2}(x_{n}-\frac{1}{x_{2}} \geq 0$ since $x_{n} \geq 1$ And applying the recursive formula as usual we find that the limit is either -1 or 1 and ruling out -1 we get 1.

\subsection{More definitions and theorems}

We come to an interesting definition. That of \textit{LimSup & LimInf}.

\begin{definition}(LimSup and LimInf)
Let $x_{n}$ be a bounded sequence. Then:
$$y_{n} := \sup{\{x_{k}|n\leq k \in \mathbb{N}\}} \text{(resp. inf)}}$$
\end{definition}

Now the sequence $y_{n}$ is clearly decreasing as we are looking over a smaller set for each $n+1$. Similarly considering the respective inf definition, we have that it is increasing as each time we are removing elements from the largest set for $n=0$ meaning that our inf is at least as large as $y_{0}$ or bigger. Now since $y_{n}$ is valid sequence definition, it naturally has a limit as well defined as:

$$y_{n} := \lim_{n\to\infty}{\sup{\{x_{k}|n \leq k \in \mathbb{N}\}} \text{(resp. inf)}}}$$

Here's an example:

\begin{example}
Consider $x_{n} = (-1)^{n}$. Defining $y_{n} = \sup{\{x_{k}|n\leq k \in \mathbb{N}\}}$ we get that the limit of $y_{n}$ as n goes to infinity is 1.
\end{example}

And now we define what it means to be a subsequence.

\begin{definition}
Let $x_{n}$ be a sequence. Then $x_{n_{k}}$ is a subsequence of $x_{n}$ where each $k$ is mapped to some $n_{k}$ by some rule $f: \mathbb{N} \to \mathbb{N}$
\end{definition}

\begin{example}
Suppose we define $x_{n} = \frac{1}{n}\sin{n}$ And define the subsequence $x_{2\pi k}$ we notice that this subsequence is constant with all values mapping to $0$
\end{example}

And yet a meetier example is:

\begin{example}
Define $x_{n} = (1 + \frac{2}{n})^{n}$ and a subsequence $x_{2k}$. We notice that
$$ \lim_{k \to \infty}{(1 + \frac{1}{k})^{2k}} = \lim_{k \to \infty}{(1 + \frac{1}{k})^{k})^{2}} = e^{2}$$
\end{example}

\clearpage

\begin{theorem}
If a sequence $x_{n}$ converges to $a$, then so do all subsequences of $x_{n}$
\end{theorem}

\begin{proof}(Simply using invoking definition of convergence)
Now accepting the if true, we have that:

$$ \forall \epsilon > 0, \ \exists n_{\epsilon} \ \text{s.t.} n\geq n_{\epsilon} \rightarrow |x_{n} - a| \leq \epsilon$$

\textcolor{red}{We now fix $n_{\epsilon} = n_{k}$ hence proof. (this proof is to be revisited, it may be false.}
\end{proof}

\begin{example}
Consider a sequence that jumps between $e$ and $-e$ as $n \to \infty$ defined by $x_{n} = (-1)^{n}(1 + \frac{1}{n})^{n}$ For all subsequences with an even domain, the limit is $e$ and for subssequences with an odd domain, limit is $-e$
\end{example}

\begin{theorem}(Bolzano Weierstrass)
Every bounded sequence contains a convergent subsequence.
\end{theorem}

\begin{proof}
Let's define $y_{n} = \sup{x_{k}:k\geq n}$. Now given that $y_{n}$ is bounded from below(because $x_{n}$ is bounded) and that it is decreasing, we have that $y_{n}$ converges to some $y$ meaning:
$$ \forall \epsilon > 0, \forall N, \exists n_{\epsilon} \geq N \ , \underbrace{|y_{n} - y|\leq \frac{1}{2}\epsilon}_{\text{definition of convergence}}$$

And by definition of sup and how we defined $y_{n}$ we have that
$$ \forall \epsilon > 0, \exists n_{1} \geq n \rightarrow |x_{n_{1}} - y_{n}| \leq \frac{1}{2}\epsilon$$

Considering that we want an expression like $|x_{n_{1}} - y|\leq\epsilon$ And we get this by:

$$ |x_{n_{1}} - y| = |x_{n_{1}} - y_{n} + y_{n} - y| \leq \underbrace{|x_{n_{1}} - y_{n}| + |y_{n} - y_{n}|}_{\text{using triangle ineq.}} \leq \frac{1}{2}\epsilon + \frac{1}{2}\epsilon = \epsilon $$
Now the last part is enough for the proof because we are assured that for any subsequence choice we make, it holds that there is an $N$ such that $n_{k} \geq N$
\end{proof}

\begin{definition} (Cauchy convergence)
A sequence is Cauchy convergent if $\forall \epsilon > 0$, $\exists n_{\epsilon}$ such that $\forall n,m \geq n_{\epsilon}$ we have that $|x_{n} - x_{m}| \leq \epsilon$
\end{definition}

A very important theorem concerning Cauchy convergence is:

\begin{theorem}
$x_{n}$ is convergent iff. it is Cauchy convergent.
\end{theorem}

\begin{proof} (1)Suppose $x_{n}$ is convergent. Now set $x := \lim_{n \to \infty}{x_{n}}$ By definition we have that $\forall \epsilon$ $\exists n_{\frac{\epsilon}{2}}$ such that $\forall n,m \geq n_{\frac{\epsilon}{2}}$
\begin{align*}
    |x_{n} - x| \leq \frac{1}{2}\epsilon\\
    |x_{m} - x| \leq \frac{1}{2}\epsilon
\end{align*}
And noticing that $|x_{n} - x_{m}| = |x_{n} - x + x - x_{m}| \leq |x_{n} - x| + |x - x_{m}| \leq \epsilon$
\\
(2) We now prove the opposite direction.
\end{proof}

\end{document}


\chapter{Sphere collision}
\label{cha:spherecollision}

Spheres are the simplest of bounding shapes used in collision detection. This chapter presents tests for two versions of algorithms -- naive $O(N^2)$ approach and with partitioned space. While the simpler algorithm has a far greater number of collision checks per frame, it allocates almost no memory per frame. A more complex method will minimise the number of checks, but additional structure and steps added may influence overall execution time in an unexpected way.

\begin{figure}[h!]
  \caption{Example rendering of tested sphere collision system}
  \label{img:spheres}
  \centering
	\includegraphics[width=16cm]{spheres/render.png}
\end{figure}

\section{Algorithm description}
\label{sec:spherealgorithmdescription}

Collision detection for spheres is a trivial task. If distance between two spheres is smaller than the sum of their radiuses, spheres collide.

$\sqrt{(S_1.x - S_2.x)^2 + (S_1.y - S_2.y)^2 + (S_1.z - S_2.z)^2} < S_1.radius + S_2.radius$

While the equation is simple, with the large number N of colliding objects the complexity of this detection is $O(N^2)$. Methods of space partitioning are used to reduce the number of checks. The one used in this benchmark is Octree.
The base for the algorithm is a tree-like structure of bounding boxes. Whenever a box contains more than one colliding object, it is divided into eight smaller boxes, by partitioning each edge by 2. When a maximum tree depth is reached, multiple objects are stored in one box. One object may be referenced from multiple boxes, when its size and position make them intersect. Each movement requires a check if the object has already moved to one of the neighbour boxes.

\begin{figure}[h!]
  \caption{Octree structure. Source: http://en.wikipedia.org/wiki/File:Octree2.svg/}
  \label{img:octree2}
  \centering
	\includegraphics[width=10cm]{octree/octree2.png}
\end{figure}

Having objects grouped in boxes reduces the complexity of the collision check. Since an object may collide only with objects in the same box, the number of checks is much smaller. Overall complexity of Octree checks is $O(N log{N})$.

\begin{figure}[h!]
  \caption{Example of WebGL Octree debug rendering. Available online at http://pawlowski.it/octtree/}
  \label{img:octree}
  \centering
	\includegraphics[width=10cm]{octree/octree.png}
\end{figure}

When collision is detected, collision response is calculated. From the rule of conservation of momentum:

\begin{center}
$m_1 * \vec{v_1} + m_2 * \vec{v_2} = m_1 * \vec{v'_1} + m_2 * \vec{v'_2}$
\end{center}

Meaning that change of both momentums is of equal value.
\begin{center}
$m_1*\vec{v'_1} =  m_1*\vec{v_1} - \Delta P$

$m_2*\vec{v'_2} =  m_2*\vec{v_2} + \Delta P$

$\vec{v'_1} =  \vec{v_1} - \frac{\Delta P}{m_1}$

$\vec{v'_2} =  \vec{v_2} + \frac{\Delta P}{m_2}$
\end{center}

To simplify response, rotation and deformation of spheres are ignored. This does not affect performance analysis, since operations in tests are performed all in the same way.

Let
\begin{center}
$P = |\Delta P|$

$N = \hat{pos_1 - pos_2}$
\end{center}

Since transference of momentum occurs only along single points of contact:
\begin{center}
$ \Delta P = P * \hat{\vec{N}}$

$\vec{v'_1} =  \vec{v_1} - \frac{P}{m_1} * \vec{N}$

$\vec{v'_2} =  \vec{v_2} + \frac{P}{m_2} * \vec{N}$
\end{center}

Let us split each velocity into two scalars, the perpendicular and parallel value of velocity vector, and introduce $\vec{Q}$, similar to $\vec{N}$, a perpendicular normalised vector lining along the exchanged momentum.

 \begin{center}
$\vec{v_1} =  a_1 * \vec{N} + b_1 * \vec{Q}$

$\vec{v_2} =  a_2 * \vec{N} + b_2 * \vec{Q}$

$\vec{v'_1} =  a'_1 * \vec{N} + b'_1 * \vec{Q}$

$\vec{v'_2} =  a'_2 * \vec{N} + b'_2 * \vec{Q}$
\end{center}

\begin{figure}[h!]
  \caption{Illustration for collision response}
  \label{img:spheresbounce}
  \centering
	\includegraphics[width=8cm]{spheres/bounce.jpg}
\end{figure}

Deriving from previous equations:

 \begin{center}
$a_1' =  a_1 - \frac{P}{m_1}$

$b_1' = b_1$

$a_2' =  a_2 + \frac{P}{m_2}$

$b_2' = b_2$
\end{center}

Now let us use the rule of energy conservation to solve P:

 \begin{center}
$\frac{m_1}{2} * ||\vec{v_1}||^2 + \frac{m_2}{2} * ||\vec{v_2}||^2 = \frac{m_1}{2} * ||\vec{v'_1}||^2 + \frac{m_2}{2} * ||\vec{v'_2}||^2$

$\frac{m_1}{2} * ({a_1}^2 + {b_1}^2) + \frac{m_2}{2} * ({a_2}^2 + {b_2}^2) = \frac{m_1}{2} * ({a'_1}^2 + {b'_1}^2) + \frac{m_2}{2} * ({a'_2}^2 + {b'_2}^2)2$

$P = \frac{2*m_1*m_2*(a_1-a_2)}{m_1+m_2}$
\end{center}

and finally, using the result from the conservation of momentum:

 \begin{center}
$\vec{v'_1} =  \vec{v_1} - \frac{2*(a_1-a_2)}{m_1+m_2} * m_2 * \vec{N}$

$\vec{v'_2} =  \vec{v_2} + \frac{2*(a_1-a_2)}{m_1+m_2} * m_1 * \vec{N}$
\end{center}

From this result, using only the dot product of velocity vectors and normalised vector $pos_1 - pos_2$ the correct response to collision is calculated. In tested scenarios, the mass of all spheres is equal since it does not affect the complexity of calculations and produces less random results.

\section{{$O(N^2)$} approach}
\label{sec:sphereinitial}

The naive approach for collision detection proves to by easy to implement in JavaScript. Since almost no memory is allocated in each frame, no garbage collection issues appear. All methods are well defined and work mostly on floats. This results in a highly optimised binary code produced by the compiler, as shown on \ref{img:spheres1profile}.

\begin{figure}[h!]
  \caption{Chart of time used in optimised version of JavaScript}
  \label{img:spheres1profile}
  \centering
	\includegraphics[width=16cm]{spheres/spheres1-profile.png}
\end{figure}

Multiple tests with N=1000 and different number of frames rendered show that for simple mathematical tasks, the performance of JavaScript is very close to that of C++. On average, the JavaScript version of benchmark runs 15\% longer than a C++ one.

\begin{figure}[h!]
  \caption{Comparison of total execution time. N = 1000, varying number of frames.}
  \label{img:spheres1-time-total}
  \centering
	\includegraphics[width=16cm]{spheres/time-total.png}
\end{figure}
\begin{figure}[h!]
  \caption{Comparison of execution time per frame. N = 1000, varying number of frames.}
  \label{img:spheres1-time-per-frame}
  \centering
	\includegraphics[width=16cm]{spheres/time-per-frame.png}
\end{figure}

\section{Octree-partitioned space}
\label{sec:sphereoctree}

\begin{figure}[h!]
  \caption{Octree partitioned sphere collision system}
  \label{img:spheres}
  \centering
	\includegraphics[width=16cm]{spheres/render2.png}
\end{figure}

Tests with Octree partitioning were executed with N=1000 spheres and T=1000 frames. The varying value is the maximum depth of Octree, ranging from 1 to 10.
Changing the maximum depth reduces the number of collision checks between spheres, as shown on \ref{img:octree-collisions}.

\begin{figure}[h!]
  \caption{Number of collisions in Octree}
  \label{img:octree-collisions}
  \centering
	\includegraphics[width=16cm]{spheres/octree-collisions.png}
\end{figure}

For low values, the overall complexity of checks does not change significantly, since most spheres are in one or a few bounding cubes, and no checks are skipped. Additional operations related to Octree actually make this solution slower than $O(n^2)$ approach. For depth values in the optimal zone, the number of collisions is reduced by a factor of at least 10, while keeping Octree overhead reasonable. Interesting thing happens when the maximum level of Octree is very high and the edge of the smallest Octree cube approaches the size of the spheres. The number of transitions between partitioning cubes, related memory allocation and cleanups actually make this approach much slower, as shown on \ref{img:octree-time}. Moreover, some spheres are references in more than one cube, raising again the number of collision checks.

\begin{figure}[h!]
  \caption{Run times in Octree system}
  \label{img:octree-time}
  \centering
	\includegraphics[width=16cm]{spheres/octree-time.png}
\end{figure}

It's clearly visible that the number of collision checks and run time is correlated only up to a certain point. For deep Octrees, the number of checks does not improve further, but the overall run time gets longer. The performance of JavaScript in relation to C++ varies between 30\% to 80\% overhead. In comparison with $O(n^2)$ approach, optimal Octree in JavaScript runs over 92\% faster and C++ over 94\% faster.


\subsubsection{Specification of Time-Varying Parameters: Long Parameter Lines}

Time-varying specifications for a parameter are invoked using elements 8 - 14 in the \hyperlink{paraOrder}{long parameter line setup}. Each element and the options for selection related to time-varying parameters are as described below.

\hypertarget{EnvVar}{}
\begin{itemize}

\item Environmental Link and variable (env\_var\&link; element 8)

	\begin{itemize}
	   \item The environmental link and variable input is two inputs specified using a single three digit number. The hundreds place contains the option for the link function, while the tens and ones place is used to specify the environmental variable or derived quantity to which the parameter is linked. Note that environmental variables can only be included on an annual basis, so seasonal models would have the same effect applied to all seasons. If the environmental link and variable input is positive, then the parameter is linked to a variable specified in the data file environmental data; if it is negative, then the parameter is linked to a derived quantity. For example, env\_var\&link input 103 would use link type 1 and apply it to environmental data column 3, while the input -103  would use link type 1 and apply it to the "-3" column which is ln(relative summary biomass). The other options for both elements are enumerated below.
	   \item The link function options (hundreds place) for the env\_var\&link input are:
	   \begin{itemize}
	       \item 1 = exponential scalar: $P_{y} = P_{base}e^{P_{t}E_{y}}$
		   \item 2 = linear offset: $P_{y} = P_{base} + P_{t}E_{y}$
		   \item 3 = Bounded replacement: $P_{y} = min(P_{base})+\frac{max(P_{base})-min(P_{base})}{1+e^{P_tE_y+ln((P_{base}-min(P_{base})+0.0000001)/(max(P_{base})-P_{base}+0.0000001))}}$
		   \item 4 = Logistic: $P_{y} = P_{base}\frac{2}{1+e^{-P_{t2}(E_{y}-P_{t1})}}$
	   \end{itemize}
		where:
	   \begin{itemize}
	       \item $P_{y}$ = Parameter value in year $y$
           \item $P_{base}$ = Base parameter value
           \item $P_{t}$ = Link parameter value
           \item $P_{t1}$ = First of 2 link parameters (offset)
           \item $P_{t2}$ = Second of 2 link parameters (slope)
           \item $E_{y}$ = Environmental index value or derived quantity value in year $y$
           \item $min(P_{base})$ = the minimum parameter bound of base parameter
           \item $max(P_{base})$ = the maximum parameter bound of base parameter
        \end{itemize}
		\item The variable options (tens and ones place, or $E_{y}$) for the env\_var\&link input are either 1) a positive integer from 1 to 99 referencing a time-series located in the \hyperlink{env-dat}{environmental data section} of the data file, or 2) a negative value of -1 to -4 where $E_y$ is one of the following model-derived quantities:
		\begin{itemize}
			\item -1;  for ln(relative spawning biomass)
			\item -2;  for recruitment deviation
			\item -3;  for ln(relative summary biomass) (e.g., current year summary biomass divided by the unfished summary biomass)
			\item -4;  for ln(relative summary numbers)
		\end{itemize}
		\item The four derived quantities are all calculated at the beginning of each year within the model, so they are available to use as the basis for time-varying parameter links without violating any order of operations rules.
	\end{itemize}

\item Deviation Link (element 9). A positive integer invokes parameter deviations, but otherwise should be left as 0. SS3 expects the estimated deviations to be normal in distribution and the deviation values are multiplied by the standard error parameter as they are used. This differs from recruitment deviations and from the approach in SS3 v.3.24. Link options for parameter deviations are:
	\begin{itemize}
		\item 1 = multiplicative: $P_y = P_{base,y}e^{\text{dev}_y*\text{dev}_{se}}$,
		\item 2 = additive: $P_y = P_{base,y} + \text{dev}_y*\text{dev}_{se}$,
		\item 3 = random walk. Random walk options are implemented by using $\rho$ in the objective function. $P_y = P_{base,y} + \sum_{n=1}^{y} \text{dev}_n*\text{dev}_{se}$
		\item 4 = mean reverting random walk with $\rho$.
		\begin{itemize}
		    \item $X_1 = \text{dev}_1*\text{dev}_{se}$
			\item $P_1 = P_{base,y} + X_1$
			\item $X_y = \rho*X_{y-1} + \text{dev}_y*\text{dev}_{se}$
			\item $P_y = P_{base,y} + X_y$
		\end{itemize}
		\item 5 = mean reverting random walk with $\rho$ and a logit transformation to stay within the minimum and maximum parameter bounds (approach added in SS3 v.3.30.16)
	    \begin{itemize}
		    \item $X_1 = \text{dev}_1*\text{dev}_{se}$
			\item $R = P_{max} - P_{min}$
			\item $Y_1 = ln(\frac{P_{base,y} - P_{min} + \text{nil}}{P_{max} - P_{base,y} + nil})$
			\item $P_1 = P_{min} + \frac{R}{1 + e^{-Y_1 - X_1 }}$. For the first year.
			\item $X_y = \rho*X_{y-1} + \text{dev}_y*\text{dev}_{se}$
			\item $Y_y = ln(\frac{P_{base,y} - P_{min} + nil}{P_{max} + P_{base,y} + nil})$
			\item $P_1 = P_{min} + \frac{R}{1 + e^{-Y_y - X_y }}$. For years after the first year.
		\end{itemize}
		\item 6 = mean reverting random walk with penalty to keep the root mean squared error (RMSE) near 1.0. Same as case 4, but with penalty applied.
		\item The option of extending the final model year deviation value subsequent years (i.e., into the forecast period) was added in v. 3.30.13.  This new option is specified by selecting the appropriate deviation link option (1, 2, 3, or 4) and appending a 2 at the front (21, 22, 23, or 24) which will use the final year deviation value for all forecast years
	\end{itemize}
	where:
	\begin{itemize}
	     \item $P_{y}$ = Parameter value in year $y$
         \item $P_{base,y}$ = Base parameter value for year $y$
		 \item $\text{dev}_y$ = deviation in year $y$
		 \item $\text{dev}_{se}$ = standard error of the deviation
		 \item nil is a small value (e.g., 0.0000001)
	\end{itemize}
\item Deviation Minimum Year (element 10). Year deviations start for the parameter. This must be specified if using parameter deviations, but otherwise should be left as 0.

\item Deviation  Maximum Year (element 11). Year deviations end for parameter. This must be specified if using parameter deviations, but otherwise should be left as 0.

\item Deviation Phase (element 12). The phase in which the deviations for the parameter should be estimated. This must be specified if using parameter deviations, but otherwise should be left as 0.
%is there a recommended phase to use if wanting to estimate devs?

\item Use Time Blocks or Trends (element 13). Time blocks and trends are both specified using this input. If neither are used, this should be left as 0. For trend options, the cumulative normal distribution function is used as the shape of the trend in all cases, but the parameterization differs. In general, the trend used is:
    \begin{equation}
	    P_y = P_{base} + P_{\text{offset}}\phi(\frac{y - P_{\text{infl}}}{P_{width}})
	\end{equation}
	where
	\begin{itemize}
	    \item $P_y$ is the final parameter value in year $y$
		\item $P_{base}$ is the base parameter value
		\item $P_{\text{offset}}$ is the parameter offset value
		\item $\phi$ is the standard cumulative normal distribution function
		\item $P_{\text{infl}}$ is the inflection year (i.e., the year in which half of the total change from the base parameter has occured)
		\item $P_{width}$ is the standard deviation.
	\end{itemize}
In all cases, 3 parameters are estimated and hence 3 short parameter lines are required. These parameter lines differ amongst the trend options.

The input value options for element 13 are:
	\begin{itemize}
% TODO: add more information about how the above equation relates to theoptions -1 and -3.
		\item >0: time block index for parameter. See the \hyperlink{timeblocks}{time blocks section} of the control file for more information on specifying time blocks.
		\item -1: Trend Offset option. Three parameters are estimated: end trend value as a logistic offset (input as $ln(P_{\text{offset}})$), inflection year logistic offset (input as $ln(P_{\text{infl}})$, and width ($P_{width}$. Offset trend value is in natural log space. Inflection year is also in natural log space and offset from ln(0.5). Width is directly specified.
		\item -2: Trend Direct input option. In this case, $P_{\text{offset}} = 1$. Three parameters are input via short parameter lines: end trend parameter value ($P_y$ where $y$ is the final year), inflection year ($P_{\text{infl}}$, and width ($P_{width}$).
		\item -3: Trend Fractional option. In this case, $P_{\text{offset}} = 1$. Three parameters will be estimated: end trend parameter value as a fraction of base parameter maximum - minimum, inflection year as a fraction of end year - start year, and width ($P_{width}$). Width is directly input.
	\end{itemize}

\item Time Block Functional Form (element 14). Leave as 0, unless time blocks are used.
	\begin{itemize}
		\item 0: multiplicative parameter ($P_{block} = P_{base}*e^{P_t}$)
		\item 1: additive parameter ($P_{block} = P_{base} + P_t$)
		\item 2: replace parameter ($P_{block} = P_t$)
		\item 3: random walk across blocks ($P_{block} = P_{block,-1} + P_t$)
	\end{itemize}
	where:
	\begin{itemize}
        \item $P_{block}$ = Final parameter value in time block $block$
        \item $P_{base}$ = Base parameter value
		\item $P_{t}$ = Time-varying parameter value for a time block
		\item $P_{block,-1}$ = Final parameter value in the previous time block
     \end{itemize}
\end{itemize}

Code for the deviation link can be found in \href{https://github.com/nmfs-stock-synthesis/stock-synthesis/blob/main/SS_timevaryparm.tpl}{SS\_timevaryparm.tpl}, search for "SS\_Label\_Info\_14.3".

\subsubsection{Specification of Time-Varying Parameters: Short Parameter Lines}

If a time-varying specification set up in the long parameter lines for a particular section requires additional parameters, short parameter lines need to be created following the long parameter lines for the section (unless \hyperlink{autogen}{autogeneration} is used, which creates short parameter lines in control.ss\_new upon running the model). The number of parameter lines required depends on the time-varying parameter specification.

For example, if two parameters were specified to have environmental linkages in the MG parameter section, below the MG parameters would be two parameter lines (when not auto-generating these lines), which is an environmental linkage parameter for each time-varying base parameter:

\begin{longtable}{ p{0.7cm} p{0.7cm} p{0.7cm}  p{1cm}  p{1.4cm}  p{1cm} p{1cm} p{6.7cm}  }
	\hline
	&    &      & Prior &  Prior & Prior & & \Tstrut\\
	LO & HI & INIT & Value &  SD    & Type  & Phase & Parameter Label \Bstrut\\
	\hline
	\endfirsthead

	\hline
	&    &      & Prior &  Prior & Prior &  & \Tstrut\\
	LO & HI & INIT & Value &  SD    & Type  & Phase & Parameter Label \Bstrut\\
	\hline
	\endhead

	\endfoot

	\endlastfoot

	\multicolumn{7}{l}{COND: Only if MG parameters are time-varying} \Tstrut\\
	-99   & 99  & 1 & 0 & 0.01 & 0 & -1 &\#Wtlen\_1\_Fem\_ENV\_add\Tstrut\\
	-99   & 99  & 1 & 0 & 0.01 & 0 & -1 &\#Wtlen\_2\_Fem\_ENV\_add\Bstrut\\
	\hline
\end{longtable}

In SS3 v.3.30, the time-varying input short parameter lines are organized such that all parameters that affect a base parameter are clustered together with time blocks (or trend) first, then environmental linkages, then parameter deviations. For example, if the mortality-growth (MG) base parameters 3 and 7 had time varying changes, the order would look like:

 \begin{center}
 	\begin{longtable}{p{5cm} p{10cm}}
 		\hline
 		MG base parameter 3 & Block parameter 3-1\Tstrut\\
 		& Block parameter 3-2\\
 		& Environmental link parameter 3-1\\
 		& Deviation se parameter 3 \\
 		& Deviation $\rho$ parameter 3 \Bstrut\\
 		MG base parameter 7 & Block parameter 7-1 \\
 		& Deviation se parameter 7 \\
 		& Deviation $\rho$ parameter 7 \Bstrut\\
 		\hline
 	\end{longtable}
 \end{center}

The number of short parameter lines for each time-varying setup selected depends on the selection options. The \hyperlink{autogen}{autogeneration} feature can be used to figure out which parameter lines are needed. The short parameter lines needed for different time-varying options are:
\begin{itemize}
	\item Environmental Linkages: Requires 1 short parameter line ($P_{t}$), except for link option 4, which requires 2 short parameter lines ($P_{t1}$ and $P_{t2}$).
	\item Parameter deviations: Requires 2 short parameter lines, one for the standard error ($\text{dev}_{se}$), followed by one for $\rho$. Note that an input for $\rho$ is required but only used with random walk options. For the random walk options, $\rho$ can be set at 1 for a random walk with no drift or >1 for a random walk with drift.
	\item Time Blocks: One parameter for each time block ($P_{t}$) set up in the pattern.
	\item Trends: Requires 3 parameter lines. The interpretation of the parameters differs by the trend option selected, but in general they are a parameter for the final parameter value, a parameter for the inflection point year, and a parameter for the width (i.e., the standard deviation).
\end{itemize}

\subsubsection{Example Time-varying Parameter Setups}

The time-varying parameter options in Stock Synthesis are flexible. Below are some example setups that illustrate how the time-varying options could be used in a model, although there are many more possible setups.

\myparagraph{Environmental linkages}

\begin{itemize}
	\item Suppose growth rate is found to be linked with an index of water temperature. The water temperature proxy could be input into the data file as environmental data. If it is input as index number 1, the growth parameter $K$ (if using a von Bertalanffy growth equation) could be linked to the water temperature proxy data by specifying the code "201" in the env\_var\&link function input. This would establish an offset link between the parameter and the temperature proxy. One additional parameter line is required after the "MG parameter" long parameter lines section.
	\item Suppose for a fishery, selectivity is thought to shift depending on population size. Smaller fish are selected when there are lower population numbers, while larger fish are selected when there are higher population numbers. The selectivity parameter could be made time-varying using the code "-104" in the env\_var\&link option, which assumes a exponential scalar link between the base selectivity parameter and the time varying parameter value. One additional parameter line is required at the end of the selectivity long parameter lines section.
\end{itemize}

\myparagraph{Parameter Deviations}

\begin{itemize}
	\item Suppose a selectivity parameter is thought to drift every year during 2000-2010. This could be represented using a random walk link option available within the parameter deviations options. To implement this, the user could input 3 into the "dev link" input on the long parameter line for the selectivity parameter, and then input values 2000 and 2010 for "dev min yr" and "dev max yr", respectively. The dev phase could be set to 3. With this setup, 2 additional short parameter lines would be expected, one for the standard error and one for $\rho$. Both of these will be used since a random walk option is selected. To use a random walk without drift, $\rho$ is set at 1 with a negative phase.
\end{itemize}

\myparagraph{Time Blocks}

\begin{itemize}
	\item Offset approach: One or more time blocks are created and cover all or a subset of the years. Each block gets a parameter that is used as an offset from the base parameter (time block functional form 1). In this situation, typically the base parameter and each of the offset parameters are estimated. In years not covered by blocks, the base parameter alone is used.  However, if blocks cover all the years, then the value of the block parameter is completely correlated with the mean of the block offsets, so model convergence and variance estimation could be affected.  The recommended approach when using offsets is to not have all years covered by blocks or to fix the base parameter value at a reasonable level when doing offsets for all years.

	\item Replacement approach, Option A: Time blocks are created which cover a subset of the years. The base parameter is used in the non-block years and the value of the base parameter is replaced by the block parameter in each respective block (time block functional form 2). In this situation, typically the base parameter and each of the block parameters are estimated.

	\item Replacement approach, Option B: Replacement time blocks are created for all the years, so the base parameter is simply a placeholder that is always replaced by a block parameter (time block functional form 2). In this situation, do not allow the model to estimate the base parameter and only estimate the corresponding block replacement parameters, otherwise, the search algorithm will be attempting to estimate parameters that do not contribute to the log likelihood, so model convergence and variance estimation could be affected.
\end{itemize}

\myparagraph{Trends}

\begin{itemize}
	\item Suppose natural mortality was thought to increase from 0.1 to 0.2 during 2000 to 2010. This could be input as a trend. First, the natural mortality parameter would be fixed at an initial value of 0.1. Then, a value of -2 could be input into the "use block" column of the natural mortality long parameter line to indicate that the direct input option for trends should be used. The long parameter line for M could look like:
	\begin{center}
		\begin{longtable}{p{1cm} p{1cm} p{1cm}  p{1.5cm}  p{1cm}  p{1.5cm}  p{1.5cm} p{1.5cm} p{3cm}}

			\hline
			LO \Tstrut & HI & INIT & <other entries> & PHASE & <other entries> & Use\_Block & Block Fxn & Parameter Label\Bstrut\\
			\hline
			0          & 4 & 0.1 &  \multicolumn{1}{c}{...} & -1 & \multicolumn{1}{c}{...} & -2 & 0 & \#M \Bstrut\\
			\hline
		\end{longtable}
	\end{center}

	\item Three short parameter lines are then expected after the mortality-growth long parameter lines, one for the final value, one for the inflection year and one for the width. The final value could be fixed by using 0.2 as the final value on the short parameter line and a negative phase value. The inflection year could be fixed at 2005 by inputting 2005 for the inflection year in the short parameter line with a negative phase. Finally, the width value (i.e., standard deviation of the cumulative normal distribution) could be set at 3 years. The short parameter lines could look like:

	\begin{longtable}{ p{0.7cm} p{0.7cm} p{0.7cm}  p{1cm}  p{1.4cm}  p{1cm} p{1cm} p{6.7cm}}
	\hline
	&    &      & Prior &  Prior & Prior & & \Tstrut\\
	LO & HI & INIT & Value &  SD    & Type  & Phase & Parameter Label \Bstrut\\
	\hline
	\endfirsthead

	\hline
	&    &      & Prior &  Prior & Prior &  & \Tstrut\\
	LO & HI & INIT & Value &  SD    & Type  & Phase & Parameter Label \Bstrut\\
	\hline
	\endhead

	\endfoot

	\endlastfoot

	0.001 & 4    & 0.2  & 0 & 0.01 & 0 & -1 &\#M\_TrendFinal\Tstrut\\
	1999  & 2011 & 2005 & 0 & 0.01 & 0 & -1 &\#M\_TrendInfl\Bstrut\\
	-99   & 99   & 3    & 0 & 0.01 & 0 & -1 &\#M\_TrendWidth\_yrs\Bstrut\\
	\hline
\end{longtable}
\end{itemize}


\documentclass{article}
\usepackage{amsmath,amssymb,fouriernc,parskip,amsthm}
\usepackage{hyperref}
\hypersetup{
  citecolor=red,
  colorlinks=true
}
\title{Fixed Point}
\date{}
\author{}
\begin{document}
\theoremstyle{definition}
\newtheorem{thm}{Theorem}[section]
\newtheorem{lem}[thm]{Lemma}
\newtheorem{defn}[thm]{Definition}

\maketitle

Fix a valid fixed point format $fmt$, and let $Fixed$ be the set of all
fixed point numbers in this format (i.e., the set of all real numbers $x$ for
which $is\_fixed(fmt,x)$ is true).

\section{Rounding}

\begin{lem}
\label{fixedfinite}
\begin{equation*}
FINITE(Fixed)
\end{equation*}
\begin{proof} Let $I = \{0, 1, \ldots, 2r^{p - 1} - 1\}$. $I$ is finite
(FINITE\_NUMSEG). Let
\begin{equation*}
f(i) = \left \{
\begin{array}{ll}
0 & \text{if $i = 0$}\\
(i + 1)/2 \cdot r^{e - p + 1} & \text{if $i$ is odd}\\
i/2 \cdot r^{e - p + 1} & \text{if $i$ is even}
\end{array} \right .
\end{equation*}
Let $s \in Fixed$; then $|s| = f \cdot r^{e - p + 1}$ and $0 \leq f < r^{p - 1}$.
If $s = 0$, $s = f(0)$. If $s > 0$, $f > 0$, and $s = f(2f - 1)$. If $s < 0$,
then $f > 0$, and $s = f(2f)$.

We have shown $Fixed \subset f(I)$. $Fixed$ is therefore finite. (FINITE\_IMAGE)
\end{proof}
\end{lem}

\begin{defn}
\label{bounds}
For any $x \in \mathbb{R}$, let
\begin{align*}
S_x^- &= \{ \; s \in Fixed \; | \; s \leq x \; \}\\
S_x^+ &= \{ \; s \in Fixed \; | \; s \geq x \; \}
\end{align*}
and $x^- = \sup{\big ( \; S_x^- \; \big )}$, $x^+ = \inf{\big ( \; S_x^+ \;
\big )}$.
\end{defn}

\begin{lem}
\label{distance}
\begin{equation*}
\forall u, v \in Fixed \; . \; u \leq v \; \wedge \; v \leq u + fulp \;
\Longrightarrow \; v = u \; \vee \; v = u + fulp
\end{equation*}
\begin{proof} Assume the antecedent, so we have $u \leq v \leq u + fulp$.
Suppose the goal is not true. Then in fact
\begin{equation*}
u < v < u + fulp
\end{equation*}
which implies
\begin{equation*}
ff(u) \cdot r^{e - p + 1} < ff(v) \cdot r^{e - p + 1} < (ff(u) + 1) \cdot r^{e - p + 1}
\end{equation*}
Dividing by the non-zero $r^{e - p + 1}$ gives
\begin{equation*}
ff(u) < ff(v) < ff(u) + 1
\end{equation*}
a contradiction, as the fractions are natural numbers.
\end{proof}
\end{lem}

\begin{lem}
\label{bounddistance}
\begin{equation*}
\forall x \in \mathbb{R} \; . \; |x| \leq finf \;
\Longrightarrow \; x^- \leq x^+ \; \wedge \; x^+ \leq x^- + fulp
\end{equation*}
\begin{proof} Assume the antecedent. Since $|x| \leq finf$, $S_x^-$ and
$S_x^+$ are non-empty (they contain $-finf$ and $finf$ respectively).
Hence, $x^-$ and $x^+$ are maximal and minimal elements of $S_x^-$ and
$S_x^+$, respectively (SUP\_UNIQUE\_FINITE).

Let $s = x^- + fulp > x^-$. $s$ is a fixed point number with fraction
$ff(x^-) + 1$. Either $s \leq x$ or $s \geq x$. If $s \leq x$, then
$s \leq x^-$ by defn of $x^-$, which is impossible. If $s \geq x$, then
$s \geq x^+$.
\end{proof}
\end{lem}

\begin{lem}
\label{bounddistance2}
\begin{equation*}
\forall x \in \mathbb{R} \; . \; |x| \leq finf \;
\Longrightarrow \; x^+ = x^- \; \vee \; x^+ = x^- + fulp
\end{equation*}
\begin{proof} Simple combination of ~\ref{distance} and ~\ref{bounddistance}.
\end{proof}
\end{lem}

\begin{lem}
\label{boundbetw}
\begin{equation*}
\forall x \in \mathbb{R} \; . \; |x| \leq finf \;
\Longrightarrow \;
\forall s \in Fixed \; . \; s \leq x^- \; \vee \; s \geq x^+
\end{equation*}
\begin{proof} Assume the antecedent. Suppose, for a contradiction, that
there is some $s$ with $s > x^-$ and $s < x^+$. Then $s \geq x^-$ and
$s \leq x^+ \leq x^- + fulp$ from ~\ref{bounddistance2}. It then follows
from ~\ref{distance} that $s = x^-$ or $s = x^- + fulp$, both contradictions.
\end{proof}
\end{lem}

\begin{thm}
\label{rndnear}
\begin{align*}
&\forall x \in \mathbb{R} \; . \; |x| \leq finf \;
\Longrightarrow \\
& \quad \Big [ \; \forall s^* \in Fixed \; . \; s^* = round(to\_nearest,x) \;
\Longleftrightarrow \\
& \quad \Big [ \; \forall s' \in Fixed \; . \;
\neg closer(s^*, s', x) \;
\Longrightarrow
\; \big(\neg closer(s', s^*, x) \; \wedge \; even(ff(s^*)) \big) \; \Big ]
\; \Big ]
\end{align*}
\begin{proof} Assume the antecedent. $round(to\_nearest,x)$ is either
$x^-$ or $x^+$, with the corresponding assumptions and properties
(SUP\_UNIQUE\_FINITE again).

Suppose $round(to\_nearest, x) = x^-$, so $s^* = x^-$.

Suppose $closer(x^-, x^+,x)$ is true. Suppose there is an $s' \in Fixed$
with $\neg closer(s^*, s', x)$. If $s' = x^-$, contradiction. If
$s' = x^+$, contradiction. Otherwise, from ~\ref{boundbetw}, $s' < x^-$
or $s' > x^+$. In either case, contradiction (I'm assuming arith rule
can take over here).

Suppose $closer(x^-, x^+,x)$ is false, but $ff(x^-)$ is true, so
again $s^* = x^-$. Suppose there is an $s' \in Fixed$
with $\neg closer(s^*, s', x)$. If $s' = x^-$, contradiction. If
$s' = x^+$, then $\neg closer(s', s^*, x)$ and $ff(s^*)$; OK. Otherwise,
from ~\ref{boundbetw}, $s' < x^-$ or $s' > x^+$. In either case, contradiction.

The proof is similar for $x^+$.
\end{proof}
\end{thm}

The proofs for other rounding modes are similar.

\begin{thm}
\label{abserror}
\begin{equation*}
\forall x \in \mathbb{R} \; . \; |x| \leq finf \; \Longrightarrow \;
|round(to\_nearest, x) - x| \leq fulp/2
\end{equation*}
\begin{proof} Assume the antecedent. Since $|x| \leq finf$,
$x^-$ and $x^+$ have the desired property that $x^- \leq x$ and
$x \leq x^+$.

Suppose $round(to\_nearest,x) = x^-$ (with the corresponding assumptions).
Either $x > x^-$ or $x = x^-$. If $x = x^-$, the error is zero. If $x > x^-$,
then $x^- < x \leq x^+$ and $|x^- - x| \leq |x^+ - x|$ (defn of closer).
It then follows (I'm thinking arith tac can do it) that
$|x^-  - x| \leq |x^+ - x^-| / 2$. From ~\ref{bounddistance},
$|x^+ - x^-| \leq fulp$, whence $|x^- - x| \leq fulp/2$.

The proof for $x^+$ is similar.
\end{proof}
\end{thm}

The proof for the other rounding modes is similar, but easier, and the
bound on the absolute error is $fulp$ instead of $fulp/2$.

\begin{lem}
\label{rnddown}
\begin{equation*}
\forall x \in \mathbb{R} \; . \; |x| \leq finf \;
\Longrightarrow \;
\big [ \; x < x^- + fulp/2 \; \Longrightarrow \; round(to\_nearest, x) = x^-
\; \big ]
\end{equation*}
\begin{proof} Assume the antecedent. Then $x^-$ and $x^+$ exist with the
desired properties. From ~\ref{bounddistance2}, $x^+ = x^-$ or $x^+ =
x^- + fulp$. If $x^+ = x^-$, then no matter what, $round(to\_nearest,x)$
rewrites to $x^-$.

Suppose $x^+ = x^- + fulp$. It then follows that
\begin{equation*}
x - x^- < fulp/2 = (x^+ - x^-)/2
\end{equation*}
and hence (arith tac?) $x - x^- < x^+ - x$, or $|x - x^-| < |x^+ - x|$.
Therefore, $closer(x^-, x^+, x)$, and $round(to\_nearest, x) = x^-$
(probably some extra steps here in HOL light).
\end{proof}
\end{lem}

\begin{lem}
\label{rndup}
\begin{equation*}
\forall x \in \mathbb{R} \; . \; |x| \leq finf \;
\Longrightarrow \;
\big [ \; x > x^- + fulp/2 \; \Longrightarrow \; round(to\_nearest, x) = x^+
\; \big ]
\end{equation*}
\begin{proof} Similar proof.
\end{proof}
\end{lem}

\begin{thm}
\label{relerror}
\begin{equation*}
\forall x \in \mathbb{R} \; . \; |x| \leq finf \; \wedge \; x \neq 0 \;
\Longrightarrow \;
\left | \frac{round(to\_nearest, x) - x}{x} \right | \leq 1
\end{equation*}
\begin{proof} Assume the antecedent. Either $0 < |x| \leq fulp/2$ or
$|x| > fulp/2$. If $0 < |x| \leq fulp/2$, then $-fulp/2 \leq x \leq fulp/2$.
It then follows from ~\ref{rnddown} and ~\ref{rndup} with $x^- = 0$ and
$x^+ = 0$ (depending on the case) that $round(to\_nearest,x) = 0$, and so
\begin{equation*}
\left | \frac{round(to\_nearest, x) - x}{x} \right | = 1
\end{equation*}
Suppose $|x| > fulp/2$. Since $|x| \leq finf$, from
~\ref{abserror} we have
\begin{equation*}
\left | round(to\_nearest, x) - x \right | \leq fulp/2
\end{equation*}
It then follows that
\begin{equation*}
\left | \frac{round(to\_nearest, x) - x}{x} \right |
\leq \frac{fulp/2}{|x|} \leq 1
\end{equation*}
\end{proof}
\end{thm}

\end{document}


% 24 Feb 1997 mg
\chapter{Catalog of Program Packages and Entries}
\section*{Elementary Functions}
\begin{DLtt}{12345678901}
\item[B002 PRMFCT] Prime Numbers and Prime Factor Decomposition
\item[B100 RBINOM] Binomial Coefficient
\item[B101 ATG] Arc Tangent Function
\item[B102 ASINH] Hyperbolic Arcsine
\item[B105 RPLNML] Value of a Polynomial
\item[B300 RSRTNT] Integral of type $R(x,\sqrt{a+bx+cx^2})$
\end{DLtt}
\section*{Equations and Special Functions}
\begin{DLtt}{12345678901}
\item[C200 RZEROX] Zero of a Function of One Real Variable
\item[C201 RSNLEQ] Numerical Solution of Systems of Nonlinear Equations
\item[C202 RMULLZ] Zeros of a Real Polynomial
\item[C205 RZERO] Zero of a Function of One Real Variable
\item[C207 RRTEQ3] Roots of a Cubic Equation
\item[C208 RRTEQ4] Roots of a Quartic Equation
\item[C209 CPOLYZ] Zeros of a Complex Polynomial
\item[C210 NZERFZ] Number of Zeros of a Complex Function
\item[C300 ERF] Error Function and Complementary Error Function
\item[C301 FREQ] Normal Frequency Function
\item[C302 GAMMA] Gamma Function for Positive Argument
\item[C303 GAMMF] Gamma Function for Real Argument
\item[C304 ALGAMA] Logarithm of the Gamma Function
\item[C305 CGAMMA] Gamma Function for Complex Argument
\item[C306 CLGAMA] Logarithm of the Gamma Function for Complex Argument
\item[C309 CCLBES] Coulomb Wave, Bessel, and Spherical Bessel
Functions for Complex Argument(s) and \\ Order
\item[C312 BESJ0] Bessel Functions J and Y of Orders Zero and One
\item[C313 BESI0] Modified Bessel Functions I and K of Orders
Zero and One
\item[C315 RRIZET] Riemann Zeta Function
\item[C316 RPSIPG] Psi (Digamma) and Polygamma Functions
\item[C317 CPSIPG] Psi (Digamma) and Polygamma Functions for Complex
Argument
\item[C318 RELFUN] Jacobian Elliptic Functions sn, cn, dn
\item[C320 CELFUN] Jacobian Elliptic Functions sn, cn, dn for
Complex Argument
\item[C321 CGPLG] Nielsen's Generalized Polylogarithm
\item[C322 RFRSIN] Fresnel Integrals
\item[C323 RFERDR] Fermi-Dirac Function
\item[C324 RATANI] Arctangent Integral
\item[C326 RCLAUS] Clausen Function
\item[C327 BSIR4] Modified Bessel Functions I and K of
Order 1/4, 1/2 and 3/4
\item[C328 CWHITM] Whittaker Function M of Complex Argument and
Complex Indices
\item[C330 RASLGF] Legendre and Associated Legendre Functions
\item[C331 RFCONC] Conical Functions of the First Kind
\item[C332 RDILOG] Dilogarithm Function
\item[C334 RGAPNC] Incomplete Gamma Functions
\item[C335 CWERF] Complex Error Function
\item[C336 RSININ] Sine and Cosine Integrals
\item[C337 REXPIN] Exponential Integral
\item[C338 CEXPIN] Complex Exponential Integral
\item[C339 RDAWSN] Dawson's Integral
\item[C340 BSIR3] Modified Bessel Functions I and K of Order 1/3 and 2/3
\item[C341 BSKA] Modified Bessel Functions K of Certain Order
\item[C342 RSTRH0] Struve Functions of Orders Zero and One
\item[C343 BSJA] Bessel Functions J and I with Positive
Argument and Non-Integer Order
\item[C344 CBSJA] Bessel Functions J with Complex Argument
and Non-Integer Order
\item[C345 RBZEJY] Zeros of Bessel Functions J and Y
\item[C346 RELI1] Elliptic Integrals of First, Second, and Third Kind
\item[C347 RELI1C] Complete Elliptic Integrals of First,
Second, and Third Kind
\item[C348 CELINT] Elliptic Integral for Complex Argument
\item[C349 RTHETA] Jacobian Theta Functions
\end{DLtt}
\section*{Integration, Minimization, Non-linear Fitting}
\begin{DLtt}{12345678901}
\item[D101 SIMPS] Integration by Simpson's Rule
\item[D102 RADAPT] Adaptive Gaussian Quadrature
\item[D103 GAUSS] Adaptive Gaussian Quadrature
\item[D104 RCAUCH] Cauchy Principal Value Integration
\item[D105 RTRINT] Integration over a Triangle
\item[D106 RGS56P] Gaussian Quadrature with Five- and Six-Point Rules
\item[D107 RGQUAD] N-Point Gaussian Quadrature
\item[D108 TRAPER] Trapezoidal Rule Integration with an Estimated Error
\item[D110 RGMLT] Gaussian Quadrature for Multiple Integrals
\item[D113 CGAUSS] Adaptive Complex Integration Along a Line Segment
\item[D114 RIWIAD] Adaptive Multidimensional Monte-Carlo Integration
{\bf [Obsolete]}
\item[D120 RADMUL] Adaptive Quadrature for Multiple Integrals over
$N$-Dimensional Rectangular Regions
\item[D151 DIVON4] Multidimensional Integration or Random Number
Generation {\bf [Obsolete]}
\item[D200 RRKSTP] First-order Differential Equations (Runge-Kutta)
\item[D201 RDEQBS] First-order Differential Equations
(Gragg--Bulirsch--Stoer)
\item[D202 RDEQMR] First-order Differential Equations
(Runge--Kutta--Merson)
\item[D203 RRKNYS] Second-order Differential Equations
(Runge--Kutta--Nystr\"om)
\item[D300 EPDE1] Elliptic Partial Differential Equation
\item[D302 ELPAHY] Fast Partial Differential Equation Solver
\item[D401 RDERIV] Numerical Differentiation
\item[D501 LEAMAX] Constrained Non-Linear Least Squares and Maximum
                   Likelihood Estimation
\item[D503 RMINFC] Minimum of a Function of One Variable
\item[D506 MINUIT] Function Minimization and Error Analysis
\item[D510 FUMILI] Fitting Chisquare and Likelihood Functions
{\bf [Obsolete]}
\item[D601 RFRDH1] Solution of a Linear Fredholm Integral
Equation of Second Kind
\item[D700 RFT] Real Fast Fourier Transform
\item[D702 CFT] Complex Fast Fourier Transform
\item[D705 RFSTFT] Real Fast Fourier Transform
\item[D706 CFSTFT] Complex Fast Fourier Transform
\end{DLtt}
\section*{Interpolation, Approximations, Linear Fitting}
\begin{DLtt}{12345678901}
\item[E100 POLINT] Polynomial Interpolation
\item[E102 MAXIZE] Maximum and Minimum Elements of Arrays
\item[E103 AMAXMU] Largest Absolute Number in Scattered Vector
\item[E104 FINT] Multidimensional Linear Interpolation
\item[E105 DIVDIF] Function Interpolation
\item[E106 LOCATR] Binary Search for Element in Ordered Array
\item[E201 RLSQPM] Least Squares Polynomial Fit
\item[E208 LSQ] Least Squares Polynomial Fit {\bf [Obsolete]}
\item[E210 NORBAS] Polynomial Splines / Normalized B-Splines
\item[E211 RCSPLN] Cubic Splines and their Integrals
\item[E222 RCHEBN] Solution of Overdetermined Linear System in the
Chebychev Norm
\item[E230 TL] Constrained and Unconstrained Linear Least Squares Fitting
\item[E250 LFIT] Least-Squares Fit to Straight Line
\item[E255 PARLSQ] Least-Squares Fit to Parabola {\bf [Obsolete]}
\item[E406 RCHECF] Chebyshev Series Coefficients of a Function
\item[E407 RCHSUM] Summation of Chebyshev Series
\item[E408 RCHPWS] Conversion of Chebyshev to Power and Power to
Chebyshev Series
\item[E409 RTRGSM] Summation of Trigonometric Series
\end{DLtt}
\section*{Matrices, Vectors and Linear Equations}
\begin{DLtt}{12345678901}
\item[F001 LAPACK] Linear Algebra Package
\item[F002 RVADD] Elementary Vector Processing
\item[F003 RMADD] Elementary Matrix Processing
\item[F004 RMMLT] Matrix Multiplication
\item[F010 RINV] Linear Equations, Matrix Inversion
\item[F011 RFACT] Repeated Solution of Linear Equations,
Matrix Inversion, Determinant
\item[F012 RSINV] Symmetric Positive-Definite Linear Systems
\item[F105 POLROT] Rotate a Three-Dimensional Polar Coordinate System
\item[F110 MXPACK] TC Matrix Manipulation Package {\bf [Obsolete]}
\item[F112 TR] Manipulation of Triangular and Symmetric Matrices
\item[F116 DOTI] Scalar Product of Two Space-Time Vectors
\item[F117 CROSS] Vector Product of Two 3-Vectors
\item[F118 ROT] Rotating a 3-Vector
\item[F121 VECMAN] Vector Algebra
\item[F122 SCATTER] Search Operations on Sparse Vectors
\item[F123 BVSL] Bit Vector Manipulation Package
\item[F150 MXDIPR] Direct or Tensor Matrix Product
\item[F406 RBEQN] Banded Linear Equations
\item[F500 RLHOIN] Linear Homogenous Inequalities
\end{DLtt}
\section*{Statistical Analysis and Probability}
\begin{DLtt}{12345678901}
\item[G100 PROB] Upper Tail Probability of Chi-Squared Distribution
\item[G101 CHISIN] Inverse of Chi-Square Distribution
\item[G102 PROBKL] Kolmogorov Distribution
\item[G103 TKOLMO] Kolmogorov Test
\item[G104 STUDIS] Student's T-Distribution and Its Inverse
\item[G105 GAUSIN] Inverse of Gaussian Distribution
\item[G106 GAMDIS] Gamma Distribution
\item[G110 LANDAU] Landau Distribution
\item[G115 VAVLOV] Approximate Vavilov Distribution and its Inverse
\item[G116 VVILOV] Vavilov Density and Distribution Functions
\item[G900 RANF] Random Number Generator {\bf [Obsolete]}
\end{DLtt}
\section*{Operation Research Techniques and Management Science}
\begin{DLtt}{12345678901}
\item[H101 RSMPLX] Linear Optimization Using the Simplex Algorithm
\item[H301 ASSNDX] Assignment Problem
\end{DLtt}
\section*{Input/Output}
\begin{DLtt}{12345678901}
\item[I101 EPIO] EP Standard Format Input/Output Package
\item[I202 KUIP] KUIP - Kit for a User Interface Package
\item[I302 FFREAD] Format-Free Input Processing {\bf [Obsolete]}
\end{DLtt}
\section*{Output and Graphical Data Presentation}
\begin{DLtt}{12345678901}
\item[J200 VIZPRI] Print Large Characters
\item[J530 BINSIZ] Reasonable Intervals for Histogram Binning
\end{DLtt}
\section*{Executive Routines}
\begin{DLtt}{12345678901}
\item[L210 COMIS] COMIS - Compilation and Interpretation System
\item[L400 PATCHY] Source Code Maintenance
\end{DLtt}
\section*{Data Handling}
\begin{DLtt}{12345678901}
\item[M101 SORTZV] Sort One-Dimensional Array
\item[M103 FLPSOR] Sort One-Dimensional Array into Itself
\item[M104 SORCHA] Sort One-Dimensional Character Array into Itself
\item[M107 SORTR] Sort Rows of a Matrix
\item[M109 SORTRQ] Sort Rows of a Matrix
\item[M215 PSCALE] Find Power-of-Ten Scale for Printing
\item[M220 IE3CONV] Conversion To and From IEEE Number Format
\item[M400 CHTOI] Portable Conversion Between Type CHARACTER
and Type INTEGER
\item[M409 UBUNCH] Concentrate and Disperse Character Strings
{\bf [Partially obsolete]}
\item[M421 BITBYT] Package for Handling Bits and Bytes
\item[M422 PACBYT] Handling Packed Vectors of Bytes
\item[M423 INCBYT] Increment a Byte of a Packed Vector
\item[M426 BLOW] Unpack Full Words into Bytes
\item[M427 PKCHAR] Pack/Unpack Continuous Byte-strings
\item[M428 LOCBYT] Search for Byte-Content
\item[M429 NUMBIT] Number of One-Bits in a Word
\item[M431 IFROMC] Convert Between Character String and Packed
ASCII Form
\item[M432 CHPACK] Utility Routines for Character String Parsing
and Construction
\item[M433 INDEXX] Utility Package for Character Manipulation
\item[M434 VXINV] Fast VAX Byte Inversion
\item[M436 BUNCH] Pack Bytes into Full Words
\item[M437 GETBIT] Set or Retrieve a Bit in a String
\item[M438 BTMOVE] Move Bit String
\item[M439 GETBYT] Set or Retrieve a Bit String
\item[M441 BITPAK] Handling Bits and Bytes, Bit Zero the Least
Significant
\item[M442 NAMEFD] Fortran Emulation of VM/CMS NAMEFIND Command
\item[M501 IUSAME] Locating a String of Same Words
\item[M502 UOPTC] Decoding Options Characters
\item[M503 UBITS] Locate the One-Bits of a Word or an Array
\item[M507 LENOCC] Occupied Length of a Character String
\item[M508 BITPOS] Find One-Bits in a String
\end{DLtt}
\section*{Debugging, Error Handlng}
\begin{DLtt}{12345678901}
\item[N001 KERSET] Error Processing for Sections A-H of KERNLIB
{\bf [Partially obsolete]}
\item[N002 MTLSET] Error Processing for MATHLIB
\item[N100 LOCF] Address of a Variable
\item[N103 IUWEED] Detect Indefinite and Infinite in an Array
\item[N105 TRACEQ] Print Trace-Back
\item[N203 TCDUMP] Memory Dump
\end{DLtt}
\section*{Service or Housekeeping Programming Aids}
\begin{DLtt}{12345678901}
\item[Q100 ZEBRA] Dynamic Data Structure and Memory Manager
\item[Q120 HIGZ] High Level Interface to Graphics and Zebra
\item[Q121 PAW] PAW - Physics Analysis Workstation Package
\item[Q122 SIGMA] SIGMA - System for Interactive Graphical
Mathematical Applications
\item[Q123 FATMEN] Distributed File and Tape Management System
\item[Q124 CSPACK] Client Server Routines and Utilities
\item[Q180 HEPDB] Distributed Database Management System
\item[Q210 ZBOOK] Dynamic Memory Management {\bf [Obsolete]}
\item[Q901 INDENT] Indent Fortran Source
\item[Q902 FLOP] FLOP - Fortran Language Oriented Parser
\item[Q904 CONVERT] Fortran 77 to Fortran 90 source form conversion tool
\item[Q905 WYLBUR] Wylbur Phoenix - a Line Editor for ASCII Text Files
                   \textbf{[Obsolete]}
\end{DLtt}
\section*{Magnet and Beam Design, Electronics}
\begin{DLtt}{12345678901}
\item[T604 POISCR] Solution of Poisson's or Laplace's Equation in
Two-Dimensional Regions
\end{DLtt}
\section*{Quantum Mechanics, Particle Physics}
\begin{DLtt}{12345678901}
\item[U101 LOREN4] Lorentz Transformation
\item[U102 LORENF] Lorentz Transformations
\item[U111 RWIG3J] Wigner 3-j, 6-j, 9-j Symbols; Clebsch-Gordan,
Racah W-, Jahn U-Coefficients
\item[U112 RTCLGN] Clebsch-Gordan Coefficients in Rational Form
\item[U501 RDJMNB] Beta-Term in Wigner's D-Function
\end{DLtt}
\section*{Random Numbers and General Purpose Utilities}
\begin{DLtt}{12345678901}
\item[V104 RNDM] Uniform Random Numbers {\bf [Obsolete]}
\item[V105 NRAN] Arrays of Uniform Random Numbers {\bf [Obsolete]}
\item[V113 RANMAR] Uniform Random Number Generator
\item[V114 RANECU] Uniform Random Number Generator
\item[V115 RANLUX] Uniform Random Numbers of Guaranteed Quality
\item[V116 RM48]   Double Precision Uniform Random Numbers
\item[V120 RNORML] Gaussian-distributed Random Numbers
\item[V122 CORSET] Correlated Gaussian-distributed Random Numbers
\item[V130 RAN3D]  Random Three-Dimensional Vectors {\bf [Obsolete]}
\item[V131 RN3DIM] Random Three-Dimensional Vectors
\item[V135 RNGAMA] Gamma or Chi-Square Random Numbers
\item[V136 RNPSSN] Poisson Random Numbers
\item[V137 RNBNML] Binomial Random Numbers
\item[V138 RNMNML] Multinomial Random Numbers
\item[V149 RNHRAN] Random Numbers According to Any Histogram
\item[V150 HISRAN] Random Numbers According to Any Histogram
{\bf [Obsolete]}
\item[V151 FUNRAN] Random Numbers According to Any Function
{\bf [Obsolete]}
\item[V152 FUNLUX] Random Numbers According to Any Function
\item[V202 PERMU] Permutations and Combinations
\item[V300 UZERO] Preset Parts of an Array
\item[V301 UCOPY] Copy an Array
\item[V302 UCOCOP] Copy a Scattered Vector
\item[V304 IUCOMP] Search a Vector for a Given Element
\item[V306 PROXIM] Adjusting an Angle to Another Angle
\item[V401 GRAPH] Find Compatible Node-Nets in an Incompatibility Graph
\item[V700 RVNSPC] Volume of Intersection of a Circular Cylinder
with a Sphere
\end{DLtt}
\section*{High Energy Physics Simulation, Kinematics, Phase Space}
\begin{DLtt}{12345678901}
\item[W150 TRSPRT] Transport, Second-Order Beam Optics
\item[W151 TURTLE] Beam Transport Simulation, Including Decay
\item[W505 FOWL] General Monte-Carlo Phase-Space
\item[W515 GENBOD] N-Body Monte-Carlo Event Generator
\end{DLtt}
\section*{Statistical Data Analysis and Presentation}
\begin{DLtt}{12345678901}
\item[Y201 IUCHAN] Find Histogram-Channel
\item[Y250 HBOOK] Statistical Analysis and Histogramming
\item[Y251 HPLOT] HPLOT : HBOOK Graphics Interface for Histogram
Plotting
\end{DLtt}
\section*{Miscellaneous System-Dependent Facilities}
\begin{DLtt}{12345678901}
\item[Z001 KERNGT] Print KERNLIB Version Numbers
\item[Z007 DATIME] Job Time and Date
\item[Z009 CALDAT] Calendar Date Conversion
\item[Z020 UMON] Usage Monitor for VAX/VMS
\item[Z035 ABEND] Abnormal Termination of Fortran Programs
\item[Z036 ABUSER] Intercept a Fortran Abend on IBM
\item[Z037 VAXAST] Routines to Handle Control-C Interrupts on Vax
\item[Z041 QNEXTE] Restart of Next Event
\item[Z042 JUMPXN] Calling a Subroutine by its Address {\bf [Obsolete]}
\item[Z044 INTRAC] Identify Job as Interactive
\item[Z045 IFBATCH] Identify Job as Running in Batch Mode
\item[Z203 XINOUT] Short List Reading and Writing
\item[Z264 IARGC] Returns Command Line Arguments
\item[Z265 CINTF] Immediate Interface Routines to the C Library
\item[Z266 WHOAMI] Get the Name of the Executing Module
\item[Z267 FTOVAX] Convert File-name to and from UNIX Syntax
\item[Z301 VAXTIO] VAX Fortran Interface for Reading and
Writing 'Foreign' Tapes
\item[Z303 KAPACK] Random Access I/O Using Keywords {\bf [Obsolete]}
\item[Z310 CFIO] Handle Fixed-length Records on Unix Streams
\item[Z311 CIO] Handle Unix Disk Files
\item[Z313 TMREAD] Terminal Dialog Routines
\end{DLtt}


%File: feature.tex
%Date: Fri Jan 03 20:48:19 2014 +0800

\subsection{Feature Extraction}

\subsubsection{MFCC}
\label{sec:mfcc}
\textbf{Mel-Frequency Cepstral Coefficient} is a representation of the short-term power spectrum of a sound,
based on a linear cosine transform of a log power spectrum on a nonlinear mel-scale of frequency \cite{mfcc} .
MFCC is the mostly widely used features in Automatic Speech Recognition(ASR), and it can also be applied to Speaker Recognition task.

The process to extract MFCC feature is demonstrated in \figref{mfcc}
\begin{figure}

  \centering
  \includegraphics[width=\textwidth]{img/MFCC.png}
  \caption{MFCC feature extraction process\label{fig:mfcc}}

\end{figure}

First, the input speech should be divided into successive short-time frames of length $L$,
neighboring frames shall have overlap $R$.
Those frames are then windowed by Hamming Window, as shown in \figref{framming}.
\begin{figure}
  \centering
  \includegraphics[width=0.7\textwidth]{img/MFCC-windowing-frames.png}
  \caption{Framing and Windowing \label{fig:framming}}
\end{figure}

Then, We perform Discrete Fourier Transform (DFT) on windowed signals to compute their spectrums.
For each of $N$ discrete frequency bands we get a complex number $X[k]$ representing
magnitude and phase of that frequency component in the original signal.

Considering the fact that human hearing is not equally sensitive to all frequency bands, and especially,
it has lower resolution at higher frequencies.
Scaling methods like Mel-scale are aimed at scaling the frequency domain to better fit human auditory perception.
They are approximately linear below 1 kHz and logarithmic above 1 kHz, as shown below in \figref{melscale}:
\begin{figure}[H]
  \centering
  \includegraphics[width=0.5\textwidth]{img/mel-scale.png}
  \caption{Mel-scale plot \label{fig:melscale}}
\end{figure}

In MFCC, Mel-scale is applied on the spectrums of the signals.
The expression of Mel-scale warpping is as followed:
\[ M(f) = 2595 \log_{10}(1 + \dfrac{f}{700}) \]

\begin{figure}[H]
  \centering
  \includegraphics[width=0.5\textwidth]{img/bank.png}
  \caption{Filter Banks (6 filters) \label{fig:bank}}
\end{figure}
Then,  we appply the bank of filters according to Mel-scale on the spectrum,
calculate the logarithm of energy under each bank by $E_i[m] = \log (\sum_{k=0}^{N-1}{X_i[k]^2 H_m[k]}) $ and apply Discrete
Cosine Transform (DCT) on $E_i[m](m = 1, 2, \cdots M) $ to get an array $c_i $:
\[ c_i[n] = \sum_{m=0}^{M-1}{E_i[m]\cos(\dfrac{\pi n}{M}(m - \dfrac{1}{2}))} \]

Then, the first $k$ terms in $c_i $ can be used as features for future training.
The number of $k$ varies in different cases, we will further discuss the choice of $k$ in \secref{result}.

\subsubsection{LPC}
\textbf{Linear predictive coding} is a tool used mostly in audio signal processing and speech
processing for representing the spectral envelope of a
digital signal of speech in compressed form, using the information of a linear predictive model.\cite{lpc}

The basic assumption in LPC is that,
    in a short period, the $n$th signal is a linear combination of previous $p$ signals:
    $ \hat{x}(n) = \sum_{i=1}^pa_i x(n-i)$
    Therefore, to estimate the coefficients $ a_i$, we have to minimize the squared error
    $ \text{E}\left[ \hat{x}(n) - x(n)\right]$.
    This optimization can be done by Levinson-Durbin algorithm.\cite{levinson-durbin}

    Therefore, we first split the input signal into frames, as is done in MFCC feature extraction \secref{mfcc}.
    Then we calculate the $k$ order LPC coefficients for the signal in this frame.
    Since the coefficients is a compressed description for the original audio signal,
    the coefficients is also a good feature for speech/speaker recognition.
    The choice of $k$ will also be further discussed in \secref{result}.