\section{First-Order ODEs}
\begin{definition}
    An Ordinary Differential Equation (ODE) involves differentiable functions of $1$ variable, while Partial Differential Equations (PDEs) involve higher dimensional functions.\\
    The order of a differential equations is the highest order of derivative in the equation.
\end{definition}
\subsection{First-Order Linear ODEs}
An linear ODE is, obviously, a linear ODE.
\begin{example}
    $x^3y+y^\prime=0$ is a first-order linear ODE.
\end{example}
\subsubsection{Prelude: Exponential Function}
Consider the function $f(x)=a^x,a>1$, then $f^\prime(x)=a^x\lambda$ for some $\lambda>0$.
\begin{definition}
    We define $\exp(x)$ to be the solution to the differential equation $f^\prime=f$ with $f(0)=1$.
\end{definition}
By definition we have $(e^h-1)/h\to 1$ as $h\to 0$.
\begin{definition}
    One can show that $\exp$ is strictly increasing, therefore injective.
    So we can define $\ln$ to be the inverse function of $\exp$.
\end{definition}
So we have $\lambda=\ln a$.\\
The exponential function plays a central role in differential equations because it is the eigenfunction (i.e. a function that is only scaled under the operator) of the differential operator.\\
We obviously have $\mathrm de^{\lambda x}/\mathrm dx=\lambda e^{\lambda x}$, so it is indeed an eigenfunction.
Actually, all the eigenfunctions of the differential operator is of the form $Ce^{\lambda x}$ for some constant $C$ and $\lambda$.
So the behaviour of a differential equation can be somehow characterized by exponential functions.
\subsubsection{Rules of Linear ODEs}
Firstly, any homogeneous linear ODEs with constant coefficients (e.g. $af^\prime+bf=0, a,b\in\mathbb R$) have solutions of the form $Ce^{\lambda x}$ where $\lambda$ can be complex.\\
Secondly, for linear homogeneous ODEs, any contant multiple of a solution is also a solution.\\
Thirdly, an $n^{th}$ order linear ODE has only $n$ linearly independent solutions.\\
Lastly, an $n^{th}$ order ODE requires $n$ initial/boundary conditions to uniquely determine it.
\subsubsection{Forced (inhomogeneous) First Order ODEs with Constant Coefficient}
Case 1: Constant forcing.
For example, $5y'-3y=10$.
We will follow the following steps:\\
1. Find steady (equilibrium) solution where $y^\prime=0$.
In this case, it is $y=y_p=-10/3$.\\
2. Then, the general solution is in the form $y=y_p+y_c$ where $y_c$ is a complementary solution, any solution to the differential equation removing the inhomogeneous part.
in this case, $5y'-3y=0$.\\
3. Solve for $y_c$.
In this case, $y_c=Ae^{3x/5}$\\
4. Plug it back: $y=Ae^{3x/5}-10/3$\\
Here $A$ is any constant.
This way works since linearity of the equation granted that any two solutions to it must differ by a complementary solution.\\
Case 2: Eigenfunction forcing.\\
Suppose an isotope A decays into isotope B in a way that it is proportional to $a$, the number of nuclei of that isotope A.
While B decays into isotope C, its rate is proportional to $b$, the numbers of nuclei of isotope B.
So $\dot{a}=-k_aa\implies a=a_0e^{-k_at}$ for some constants $k_a,a_0$.
Now $\dot{b}=-k_bb+k_aa$ for some constant $k_b$, so
$$\dot{b}+k_bb=k_aa_0e^{-k_at}$$
This is an example where the forcing term is the eigenfunction of $\mathrm d/\mathrm dt$.
We can guess a particular solution
$$b_p=\frac{k_a}{k_b-k_a}e^{-k_at}$$
if $k_b\neq k_a$.
So the general solution is
$$b=\frac{k_a}{k_b-k_a}a_0e^{-k_at}+De^{-k_bt}$$
If $b(t)=0$, then
$$b=\frac{k_a}{k_b-k_a}a_0(e^{-k_at}-e^{-k_bt})$$
So
$$\frac{b}{a}=\frac{k_a}{k_b-k_a}(1-e^{(k_a-k_b)t})$$
we can solve it for $t$ and this solution allows us to date rocks etc by measuring the ratio of isotopes.
\subsubsection{First Order ODEs with Non-constant Coefficient}
The general form of these sort of equations are in the form $a(x)y^\prime+b(x)y=c(x)$ or the form $y^\prime+p(x)y=q(x)$ (given that $\forall x,a(x)\neq 0$).
We can solve it by integrating factor
$$\mu(x)=\exp\left(\int p\,\mathrm dx\right)$$
So
$$(\mu y)^\prime=\mu q\implies \mu y=\int \mu q\,\mathrm dx\implies y=\frac{1}{\mu}\int\mu q\,\mathrm dx$$
\subsection{Intermezzo: Discrete Equations}
Sometimes it is useful to consider functions evaluated at a discrete set of points, which could be useful to numerical integration and series solution.
\subsubsection{Numerical Integration}
One approximation to $\mathrm dy/\mathrm dx$ at $x=x_n$ can be written as
$$\left.\frac{\mathrm dy}{\mathrm dx}\right|_{y_n}\approx \frac{y_{n+1}-y_n}{h}$$
This is called the forward Euler approximation.
\begin{example}
    $5y'-3y=0$ again.
    Then it is approximately equal to the discrete equation
    $$5y_{n+1}-5y_n-3ny_n=0\implies y_{n+1}=\left(1+\frac{3h}{5}\right)y_n$$
    We can iterate it to approximate the solution given an initial value.
    This is the example of a recurrence relation.
    We can actually solve this by
    $$y_{n}=\left(1+\frac{3h}{5}\right)^ny_0$$
    Which can approximate the true solution pretty well if we let $h\to 0,n\to\infty$.
    Note that in this case, for finite value of $n$, $y_n$ is always less than the actual value of solution at that point.
\end{example}
\subsubsection{Series Solution}
A powerful way to solve DEs is to seek for solutions in the form of an infinite power series.
Let
$$y(x)=\sum_{n=0}^\infty a_nx^n$$
we can plug it in the differential equations to solve for $a_n$.
\begin{example}
    $5y'-3y=0$ yet again.
    Plug it in and we get
    $$5\sum_{n=0}^\infty (5(n+1)a_{n+1}-3a_n)x^{n}=0$$
    Hence
    $$5(n+1)a_n-3a_n=0\implies a_{n+1}=\frac{3a_n}{5n+5}\implies a_n=\left(\frac{3}{5}\right)^n\frac{1}{n!}a_0$$
    Thus
    $$y(x)=\sum_{n=0}^\infty a_nx^n=a_0\sum_{n=0}^\infty \left(\frac{3}{5}\right)^n\frac{1}{n!}x^n=a_0e^{3x/5}$$
    for some constant $a_0$.
\end{example}
\subsection{Non-linear First Order ODEs}
The general form of these sort of equations can be written in the following way:
$$Q(x,y)\frac{\mathrm dy}{\mathrm dx}+P(x,y)=0$$
We cannot guarantee that a nonlinear ODE can be solved in closed form.
But sometimes we can.
\subsubsection{Special Types of Non-linear First Order ODEs}
An ODE is said to be separable if it can be written in the form $q(y)\mathrm dy=p(x)\mathrm dx$, then we can solve for it by integrating both sides.\\
An ODE in the above form is called \textit{exact} if and only if $Q(x,y)\,\mathrm dy+P(x,y)\,\mathrm dx$ is an exact differential of some function $f(x,y)$, that is $\mathrm df=Q\,\mathrm dy+P\,\mathrm dx$.
If it is the case, our general form then give $\mathrm df=0$, so $f(x,y)=0$ is the solution.\\
To find it, we can make use of the multivariate chain rule to get
$$\frac{\partial f}{\partial x}+\frac{\partial f}{\partial y}\frac{\mathrm dy}{\mathrm dx}=0$$
$P=f_x,Q=f_y$, hence
$$\frac{\partial^2f}{\partial y\partial x}=\frac{\partial P}{\partial y},\frac{\partial^2f}{\partial x\partial y}=\frac{\partial Q}{\partial x}$$
So $P_y=Q_x$.
Conversely, if it is true in a simply connected domain, then $P\,\mathrm dx+Q\,\mathrm dy$ is an exact differential.
Therefore we can use it to test whether the given ODE is exact.
If so, we can find $f$ (hence a possibly implicit expression of $y$) back by integration.
\begin{example}
    We want to solve $6y(y-x)y'+(2x-3y^2)=0$.
    So $P=2x-3y^2,Q=6y(y-x)$, we can check that our preceding condition hold (i.e. $P_y=Q_x$), hence it is exact.\\
    To find $f(x,y)$, we notice
    $$
    \begin{cases}
        \left.\partial f/\partial x\right|_y=2x-3y^2\\
        \left.\partial f/\partial y\right|_x=6y(y-x)
    \end{cases}
    $$
    Integrating the first equation we have $f(x,y)=x^2-3xy^2+h(y)$ where $h$ is differentiable.
    Now plug it in the other equation we get $6y(y-x)=-6xy+h^\prime(y)$, thus $h^\prime(y)=6y^2\implies h(y)=2y^3-C$ where $C$ is a constant.
    So the general solution is
    $$x^2-3xy^2+2y^3=C$$
\end{example}
\subsubsection{Isoclines and Solution Curves}
Even if (most of the time) we cannot really solve the equation, we can still analyze its behaviour.
Now we consider an ODE of form $\dot{y}=f(y,t)$, each initial condition $y(t_0)=y_0$ will give a different solution curve (given existence).
\begin{example}
    Suppose $\dot{y}=t(1-y^2)$, of course this is separable and we can solve it, but without solving this, we can sketch solution curves as well.
\end{example}
\begin{definition}
    An isocline is a curve given by $\dot{y}$ being constant, that is, $t(1-y^2)=c$ for some constant $c$.
\end{definition}
Note that if $f(y,t)$ is single-valued (i.e. actually a function), then the solution curves cannot cross (unless as tangents of each other).
To sketch the solutions, we can first sketch all the Isoclines.
Note that along any isocline, the corresponding $\dot{y}$ is constant, so we can easily draw the vector field, so we could follow the directions with initial condition given to approximate the curve.\\
We can analysis the stability of a fixed point.
\begin{definition}
    A fixed point is a constant solution of $y$.
	That is, we have $\dot{y}=f(y,t)=0$.
\end{definition}
\begin{definition}
    A fixed point is called stable if the solution curve in a small neighbourhood of the fixed point converge to it.
\end{definition}
We now try to analyze the stability of fixed points using perturbation analysis.\\
Let $y=a$ be a fixed point of some DE $\dot{y}=f(y,t)$.
Consider a small perturbation of the fixed point $y=a+\epsilon(t)$, so $\dot{\epsilon}=f(a+\epsilon,t)=f(a,t)+\epsilon f_y(a,t)+O(\epsilon^2)$, so $\dot{\epsilon}\approx\epsilon f_y(a,t)$ which is linear.
The behaviour of $\epsilon$ obtained from this differential equation helps us to classify the fixed points.\\
If $\lim_{t\to\infty }\epsilon(t)=0$ then we call it a stable fixed point, if $\lim_{t\to\infty}\epsilon(t)=\pm\infty$, then we say it is unstable, otherwise we say it is neutral.
If $f_y(a,t)=0$, then we need higher order terms in that Taylor series in order to determine its behaviour.
\begin{example}
    $\dot{y}=t(1-y^2)$, so the fixed points are $y=\pm 1$.
    We have $f_y=-2ty$.\\
    For $y=1$, then $\dot{\epsilon}=-2t\epsilon\implies \epsilon_{0}e^{-t^2}$, so $\epsilon\to 0$ when $t\to\infty$.
    So it is stable.\\
    For $y=-1$, then $\dot{\epsilon}=2t\epsilon\implies \epsilon_{0}e^{t^2}$, so $\epsilon\to\pm\infty$ (for $\epsilon_0\neq 0$) when $t\to\infty$.
    So it is unstable.
\end{example}
\subsubsection{Autonomous DEs}
\begin{definition}
    A DE is called autonomous if $f$ does not depend on $t$, that is, it is of the form $\dot{y}=f(y)$.
\end{definition}
So in this case, we apply the perturbation analysis to get $\dot{\epsilon}=f^\prime(a)\epsilon$, thus $\epsilon=\epsilon_0e^{kt}$ where $k=f^\prime(a)$, so if $\epsilon_0\neq 0$,
$$\epsilon\to
\begin{cases}
    0\text{, if $f^\prime(a)<0$, so it is stable}\\
    \pm\infty\text{, if $f^\prime(a)>0$, so it is unstable}\\
    \text{Others, if $f^\prime(a)=0$, so it is neutral}
\end{cases}$$
\subsubsection{Phase Portraits}
Another way to analyze the behaviour of a given DE is from a geometric perspective represented by something called phase portrait.
\begin{example}
    Chemical kinetics is an important example why it is useful.
    Consider neutralization reaction ${\rm NaOH+HCl=H_2O+NaCl}$.
    Let $a(t),b(t)$ be the numbers of NaOH and HCl molecules at times $t$, and $c(t)$ be the number of ${\rm H_2O}$ which is equal to that of NaCl.\\
    So the initial condition is $a(0)=a_0,b(0)=b_0$, and the model is $\dot{c}=\lambda ab$, also $a=a_0-c,b=b_0-c$, so
    $$\frac{\mathrm dc}{\mathrm dt}=\lambda(a_0-c)(b_0-c)$$
    which is autonomous, so $a_0,b_0$ are the fixed points of the DE.
    We can sketch the graph of $\dot{c}$ against $c$, which is called the 2D phase protrait of the DE, where we can analyze the attraction vectors near the phase protrait by the trends of it to decide the stability.
\end{example}
\begin{example}\label{logistic_cont}
    Let $y(t)$ be the population at time $T$ and birth rate would be $\alpha y$ for some $\alpha$ and the death rate be $\beta y$ for some $\beta$.\\
    Case 1: Linear model.
    So $\dot{y}=\alpha y-\beta y\implies y=y_0e^{(\alpha -\beta)t}$, so $y\to\infty$ if $\alpha>\beta$.\\
    Case 2: Nonlinear model.
    So $\dot{y}=(\alpha-\beta)y-\gamma y^2$.
    the parameter $\gamma$ here could be due to the death due to overcrowdedness, etc.
    Equivalently, we can write it as $\dot{y}=ry(1-y/\lambda)$ where $r=\alpha-\beta,\lambda=(\alpha-\beta)/\lambda$, so we can sketch the phase protrait again.
    Note that the fixed points are $0$ and $\lambda$, the former is unstable but the latter is stable.
\end{example}
\subsubsection{Fixed Points in Discrete Equations}
We can introduce fixed points in discrete equations as well.
Consider a first order discrete equation (aka difference equation) of the form $x_{n+1}=f(x_n)$.
\begin{definition}
    The fixed point is a discrete equation is a fixed point of the function $f$.
\end{definition}
We can analyze its stability again by perturbation analysis.
Let $x_f$ be a fixed point and we perturb it by a small $\epsilon$, so we can expand $f$ in terms of Taylor series
$$f(x_f+\epsilon)=x_f+\epsilon f^\prime(x_f)+O(\epsilon^2),\epsilon\to0$$
So if $x_n=x_f+\epsilon, x_{n+1}=x_f+\epsilon f^\prime(x_f)$.
So $x_f$ is stable if $|f^\prime(x_f)|<1$, neutral if $|f^\prime(x_f)|=1$, unstable if $|f^\prime(x_f)|>1$.
\begin{example}[Discrete Logistic Equation]
    Some populations are born in distinct generations (e.g. lambs born in spring).
    A nonlinear model of this is
    $$\frac{x_{n+1}-x_n}{\Delta t}=\lambda x_n-\gamma x_n^2$$
    This is the discrete version of Example \ref{logistic_cont}, so
    $$x_{n+1}=(1+\lambda\Delta t)x_n-\gamma\Delta tx_n^2$$
    We can write alternatively $x_{n+1}=rx_n(1-x_n)=:f(x_n)$
    The function $f$ is called the logistic map.\\
    Note that the fixed point occurs at $0$ or $1-1/r$.
    We analyze its stability by perturbation.
    At $0$, $f^\prime=r$, so it is stable if $0<r<1$ and unstable if $r>1$.
    At $1-1/r$, $f^\prime=2-r$.
    We assume $r<0$ or $r>1$ since the other cases are not physical.
    \footnote{It's an applied course, so what the hell.}
    Then it is stable for $1<r<3$ and unstable for $r>3$.
\end{example}


\problemname{Pegs and Legs}

Pegs and Legs is a game where a disk slides down a nearly-vertical board.
At the bottom of the board are places for the disk to land, called \textit{legs}.
Each leg is worth a certain amount of points if your disk lands in it.

You start with a disk at the top and drop it onto some \textit{drop point}
peg or directly into some \textit{drop point} leg.
When your disk hits a peg, one of three things happen: (1) the disk falls to
the left with probability $\ell$, (2) the disk falls to the right with
probability $r$, or (3) it gets stuck with probability $1 - \ell - r$.
The probabilities may be different for each peg. If the disk falls to
the left or to the right, then it will either fall onto another peg or
into a leg. If the disk gets stuck, then you must drop it again from
some drop point on the top. The figure below illustrates the 3rd sample input,
the two dark pegs are the drop points.

\begin{figure}[h]
\begin{center}
 \includegraphics[width=0.5\textwidth]{pegsandlegs.pdf}
\end{center}
\end{figure}

Because of gravity it is not possible for a disk to hit the same peg
more than once unless the disk is dropped again. The game continues
until your disk lands in a leg, at which point you earn the value of
that leg. What is the maximum possible expected score the player can earn?

\section*{Input}
The first line of input contains two integers $L$
($1 \leq L \leq 100\,000$), which is the number of legs, and $P$
($1 \leq P \leq 100\,000$), which is the number of pegs. Legs
are numbered from $1$ to $L$ and pegs are labeled from $L+1$ to
$L+P$.

The next $L$ lines describe the legs, in order.
Each of these lines contains a single integer $v$ ($1 \leq v \leq 1\,000\,000$),
which is the value of this leg.

The next $P$ lines describe the pegs, in order.
Each of these lines starts with two real numbers $\ell$ ($0 < \ell < 1$),
which is the probability that the disk falls to the left after hitting this peg,
and $r$ ($0 < r < 1$), which is the probability that the disk falls
to the right after hitting this peg ($\ell + r \leq 1$), followed by two
integers $x$ ($1 \leq x \leq L+P$), which is the label of
the peg/leg the disk falls onto if it falls to the left, and $y$
($1 \leq y \leq L+P$), which is the label of the peg/leg the disk
falls onto if it falls to the right. It is guaranteed that $x$ and $y$
are smaller labels than the label of this peg.

All real numbers are specified to exactly 3 decimal places.
A peg or leg is a drop point if there are no pegs that drop into this peg or leg.

It is guaranteed that from any peg (whether a drop point or not),
the probability the disk eventually gets stuck after reaching this peg
is at most $0.9999$.

\section*{Output}
Display the maximum possible expected score the player can earn.
Answers with an absolute error or relative error of at most $10^{-6}$
will be accepted.


\documentclass[a4paper,10pt]{article}

%\usepackage[landscape]{geometry}

\usepackage{mathtools}
\usepackage{booktabs}
\usepackage{ltablex}
\usepackage{amsmath}
\usepackage{amsfonts}

\title{\vspace*{-4cm}Big \(\bigO\) Notation for A\&W Algorithms}
\date{}

\pagenumbering{gobble}

\newcommand{\bigO}{\mathcal{O}}
\DeclareMathOperator{\poly}{poly}
\DeclareMathOperator{\polylog}{polylog}

\begin{document}
    \maketitle
    \vspace*{-1.5cm}
    \renewcommand{\arraystretch}{1.25}
    \subsection*{Definitions}
    \begin{center}
        \begin{tabularx}{\textwidth}{l >{\raggedleft\arraybackslash}X}
            \toprule
            Number of vertices in a graph & \(n\) \\
            Number of edges in a graph & \(m\) \\
            Highest capacity in network & \(U\) \\
            Number of vertices at the edge (corners) of a convex hull & \(h\) \\
            Variable chosen for desired accuracy of Monte-Carlo algorithms & \(\lambda\) \\
            \bottomrule
        \end{tabularx}
    \end{center}

    \subsection*{Runtimes}

    \begin{center}
    \begin{tabularx}{\textwidth}{Xl}
        \toprule
        Find all cut vertices and edges in connected graph (DFS) & \(\bigO(m)\) \\
        Eulerian path in connected graph & \(\bigO(m)\) \\
        Hamiltonian cycle in general graph & \(\bigO(n^2 \cdot 2^n)\) \\
        Hamiltonian cycle in graph with $n \geq 3$ and $\text{minimum degree} \ \geq \frac{n}{2}$ & \(\bigO(n^2)\) \\
        Count Hamiltonian cycles in graph & \(\bigO(n^{2.81} \log n \cdot 2^n)\) \\
        Hamiltonian cycle in general graph & NP-complete \\
        Find augmenting path in bipartite graph (BFS) & \(\bigO(n + m)\) \\
        Find augmenting path in general graph (Blossom) & \(\bigO(nm)\) \\
        Maximal (inclusion) matching (greedy) & \(\bigO(m)\) \\
        Maximum (cardinality) matching in bipartite graph (Hopcroft-Karp) & \(\bigO(\sqrt{n} \cdot (n+m))\) \\
        2-approximation of Metric TSP & \(\bigO(n^2)\) \\
        $\frac{3}{2}$-approximation of Metric TSP & \(\bigO(n^3)\) \\
        Perfect matching in \(2^k\)-regular bipartite graph & \(\bigO(m)\) \\
        Perfect matching in complete graph with weight function $l: \: \binom{[n]}{2} \to \mathbb{N}_0$ & \(\bigO(n^3)\) \\
        Decide if $\chi(G) \leq k$ for $k \geq 3$ & NP-complete ($\bigO(2.2^n)$) \\
        Greedy coloring with $C(G)$ colors ($\chi(G) \leq C(G) \leq \Delta(G) + 1$) & \(\bigO(m)\) \\
        \(\bigO(\sqrt{n})\)-coloring for 3-colorable graphs & \(\bigO(m)\) \\
        Find duplicates (Hashmap) & \(\bigO(n \log n + |\text{Dupl}(\mathcal{S})|)\) \\
        Find duplicates (Bloom-Filter) & \(\bigO(kn)\) \\
        Miller-Rabin prime test & \(\bigO(\log n)\) \\
        Randomized Quicksort & \(\bigO(n \log n)\)\footnotemark \\
        Long path problem & NP-complete \\
        Colorful-Path with length \(\log n\) & \(\bigO(mn \log n)\) \\
        Colorful-Path with length \(\bigO(\log n)\) & \(\bigO(\poly(n))\) \\
        Maximal flow (Ford-Fulkerson) with max. capacity $U$ & \(\bigO(mnU)\)\\
        Maximal flow (Capacity Scaling) with max. capacity $U$ & \(\bigO(mn(1 + \log U))\) \\
        Maximal flow (Dynamic Trees) & \(\bigO(mn\log n)\) \\
        Minimal cut (naive) & \(\bigO(\lambda n^4)\) \\
        Minimal cut (single bootstrapping) & \(\bigO(\lambda n^3)\) \\
        Minimal cut (repeated bootstrapping) & \(\bigO(n^2\polylog(n))\) \\
        Smallest enclosing disc (naive deterministic or randomized) & \(\bigO(n^4)\)\footnotemark[\value{footnote}] \\
        Smallest enclosing disc (clever randomized) & \(\bigO(n \log n)\)\footnotemark[\value{footnote}] \\
        Convex hull (Jarvis-Wrap) & \(\bigO(nh)\) \\
        Convex hull with \emph{sorted ($\bigO(n \log n)$)} points (LocalRepair) & \(\bigO(n)\) \\
        \bottomrule
    \end{tabularx}
    \end{center}
    \footnotetext{Randomized algorithm; expected runtime is given}
\end{document}


\section{ Modeling Sequential Annotators }\label{sec:annomodels}

When combining multiple annotators with varying skill levels, we can improve performance by modelling their individual noise and bias using a probabilistic
model.
Here, we describe several models
that do not consider dependencies between annotations in a sequence,
before defining \emph{seq},
a new extension that captures sequential dependencies.
Probabilistic annotator models
each define a different function, $A$,
for the likelihood that the annotator chooses label $c_{\tau}$
given the true label $t_{\tau}$, for the $\tau$th token in a sequence.

\textbf{Accuracy model (acc)}:
 the basis of several previous methods~\cite{donmez2010probabilistic,rodrigues2013learning},
\emph{acc} uses a single parameter for each
annotator's accuracy, $\pi$:
\begin{flalign}
 & A = p( c_{\tau} \! = \! i | t_{\tau} \! = \! j, \pi ) = \left.
\begin{cases}
  \pi  \!&\!\!\!\text{ where } i = j \\
  \frac{1 - \pi}{J-1} \!&\!\!\!\text{ otherwise}
\end{cases}
\right\} \!, &&
\end{flalign}
where
%$c_{\tau}$ is the label given by the annotator for token $\tau$, $t_{\tau}$ is its true label
%and
$J$ is the number of classes.
%The limitation of this approach is that it
%It assumes reliability is constant,
%which means that
This may be unsuitable when one class label dominates the data,
since a spammer who always selects the most common label will nonetheless
have a high $\pi$.
%despite their labels being uninformative.
%Annotator models define a likelihood...
%\begin{flalign}
%& A = p(c_{\tau}, c_{\tau-1}, t_{\tau}) = p( c_{\tau} \!\!=\! i | c_{\tau-1}, t_{\tau} \!=\! j, \bs\pi ),&&
%\end{flalign}

\textbf{Spamming model (spam)}:
proposed as part of MACE~\cite{hovy2013learning}, this model also
assumes constant accuracy, $\pi$,
but that when an annotator is incorrect, they label according to
a spamming distribution, $\bs\xi$, that is independent of the true label, $t_{\tau}$.
\begin{flalign}
A & = p( c_{\tau} = i | t_{\tau} = j, \pi, \bs\xi) && \nonumber \\
& = \left.
\begin{cases}
  \pi + (1 - \pi) \xi_j  &\text{ where } i = j \\
  (1 - \pi) \xi_j &\text{ otherwise}
\end{cases}
\right\}.
\end{flalign}
This addresses the case where spammers choose the dominant label
% common label when the classes are imbalanced.
%While MACE can capture spamming patterns,
but does not explicitly model
different error rates in each class.
%which
%may be an issue for sequence tagging using the
%BIO encoding. For example, if an annotator frequently labels longer spans
% than the true spans by starting the spans early. In this
% case, they may more frequently
%mis-label the `B' tokens than the `I' or `O' tokens,  which cannot be modelled by MACE.
For example, if an annotator is better at detecting type `x' spans than type `y', or if they frequently miss the first token
in a span, thereby labelling the start of a span as `O' when the true label is `B-x',
this would not be explicitly modelled by \emph{spam}.

\textbf{Confusion vector (CV)}: this approach learns a separate accuracy
 for each class label~\cite{nguyen2017aggregating}
using parameter vector, $\bs\pi$, of size $J$:
\begin{flalign}
& A = p( c_{\tau} \!\!=\! i | t_{\tau} \!=\! j, \bs\pi ) = \left.
\begin{cases}
  \pi_j  \!\!\!\!\!\!&\text{ where } i \!=\! j \\
  \frac{1 \!- \!\pi_j}{J-1} \!\!\!\!\!\!&\text{ otherwise}
\end{cases}
\! \right\} \!.&&
\end{flalign}
%For the incorrect label cases where $i \! \neq \! j$,
% $p( c_{\tau} \!\!=\! i | t_{\tau} \!=\! j, \bs\pi )$ is constant for all values of $i$.
% Therefore, t
This model does not capture spamming
patterns where one of the incorrect labels has a much higher likelihood than the others.

\textbf{Confusion matrix (CM)}~\cite{dawid_maximum_1979}:
this model can be seen as an expansion of the confusion vector so that $\bs\pi$ becomes a
$J\times J$ matrix with values given by:
\begin{flalign}
& A = p( c_{\tau} \!\!=\! i | t_{\tau} \!=\! j, \bs\pi ) =
  \pi_{j,i} .&&
\end{flalign}
This requires a larger number of parameters, $J^2$, compared to the $J+1$ parameters of MACE or $J$ parameters
of the confusion vector.
Like \emph{spam},
\emph{CM} %represents the probability of each mistake, so it
can model spammers who frequently chose one label regardless
of the ground truth, but also
models different error rates and biases for each class.
%type of `B-x', `I-x' and `O' label.
However, \emph{CM} ignores dependencies between annotations in a sequence, % that affect these probabilities.
such as the fact that an `I' cannot immediately follow an `O'.
% Consider the following example where this may be a problem: three annotators produce sequences of labels as follows:
% O-B-I-I-I-O
% O-O-B-I-O-O
% O-O-O-O-O-O
% We can see that the first two annotators agree that the third token is part of the span,

\textbf{Sequential Confusion Matrix (seq)}: we introduce a new extension to the confusion matrix to model the dependency
of each label in a sequence on its predecessor,
giving the following likelihood:
\begin{flalign}
& A = p( c_{\tau} \!\!=\! i | c_{\tau-1} \!=\! \iota, t_{\tau} \!=\! j, \bs\pi ) =
  \pi_{j,\iota,i} ,&&
\end{flalign}
where $\bs\pi$ is now three-dimensional with size $J\times J\times J$.
In the case of disallowed transitions, e.g. from $c_{\tau-1}=$`O' to $c_{\tau}=$`I', the value $\pi_{j,c_{\tau-1},c_{\tau}}\approx 0$, $\forall j$
is fixed \textit{a priori}.
The sequential model can capture phenomena such as a tendency toward overly long sequences, by learning that I is more likely to follow another I, so that
$\pi_{O,I,I} > \pi_{O,I,O}$.
A tendency to split spans by inserting `B' in place of `I' can be modelled
by increasing the value of
$\pi_{I,I,B}$ without affecting $\pi_{I,B,B}$ and $\pi_{I,O,B}$.

The annotator models presented in this section
include the most widespread models for NLP annotation tasks,
and can be seen as extensions of one another.
The choice of annotator model for a particular annotator
 depends on the developer's understanding of the annotation task:
 if the annotations have sequential dependencies, this suggests the \emph{seq} model;
for non-sequential classifications \emph{CM} may be effective with small ($\leq 5$)
numbers of classes; \emph{spam} may be more suitable if there are many classes, as the number of parameters to learn is low.
However, there is also a trade-off between the expressiveness of the model and the
number of parameters that must be learned. Simpler models with fewer parameters
may be effective if there are only small numbers of annotations from each annotator.
%Our experiments in Section \ref{sec:expts_all} investigate this trade-off on NLP tasks involving sequential annotation.
The next section shows how these annotator models can be used as components of
a complete model for aggregating sequential annotations.
%The experiments in Section \ref{sec:expts_all}
% test whether the more expressive seq annotator model,
%which has more parameters to learn, is beneficial in a realistic setting.


\documentclass[letter]{article}

\usepackage{fullpage} % Package to use full page
%\usepackage{parskip} % Package to tweak paragraph skipping
\usepackage{tikz} % Package for drawing
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{mathtools}
\usepackage{listings}
\usepackage{mathrsfs}
\usepackage{amsfonts}
\usepackage{bbm}
\usepackage{xcolor}

\newcommand{\vecE}{\vec{E}}
\newcommand{\vecH}{\vec{H}}
\newcommand{\epsilonB}{\boldsymbol{\epsilon}}
\newcommand{\epsiloninv}{\boldsymbol{\epsilon^{-1}}}

\newcommand{\calE}{\vec{\mathcal{E}}}
\newcommand{\calH}{\vec{\mathcal{H}}}

\newcommand{\levi}{\mathcal{E}}

\newcommand{\Partialsqx}[1]{\frac{\partial^2 #1}{\partial x^2}}
\newcommand{\Partialsqy}[1]{\frac{\partial^2 #1}{\partial y^2}}

\usepackage{caption}
\captionsetup[table]{skip=10pt}

\title{2D Modes Draft}
\author{Panya Sukphranee}
\date{\today}

\begin{document}

	\maketitle

	\section*{Theory}

	Electric and magnetic fields propogating in the $\hat{z}$ direction have the form:

	\begin{align}
		\vecE &= \calE(x,y) e^{-j(\beta z + \omega t)} \\
		\vecH &= \calH(x,y) e^{-j(\beta z + \omega t)}
	\end{align}

	\begin{align}
		\Aboxed{\partial_z H^i &= -j \beta H^i}
		\label{eqn:partial_z}
	\end{align}

	\begin{align}
		\Aboxed{H_z &= \frac{1}{j \beta} \big( \frac{\partial H_x}{\partial x} + \frac{\partial H_y}{\partial y} \big)}
		\label{eqn:H_z}
	\end{align}

	For simplicity, assume that one of the material axes is along the propogation direction, $\hat{z}$. We also assume that the electric permittivity tensor has the form:
	\begin{align}
		\epsilonB &= \epsilon_0
		\begin{bmatrix}
			\epsilon_{xx} & \epsilon_{xy} & 0 \\
			\epsilon_{yx} & \epsilon_{yy} & 0 \\
			0 & 0 & \epsilon_{zz}
		\end{bmatrix}
		\label{matrix:diag_tensor}
	\end{align}
	implying that the transverse permitivitty is independent of $E_z$ and the longitudinal permittivity is independent of the transverse fields. Each entry above is an $M \times N$ matrix, where $M \times N$ is the dimension of the grid of points describing our waveguide. $0 = 0_{M \times N}$. Further, we deal with the simple case of a diagonal tensor. So that the electric permittivity along any coordinate axis is independent of orthogonal field components to that axis,  i.e.\\
	\begin{align*}
		\epsilonB &= \epsilon_0
		\begin{bmatrix}
			\epsilon_{xx} & 0 & 0 \\
			0 & \epsilon_{yy} & 0 \\
			0 & 0 & \epsilon_{zz}
		\end{bmatrix}
	\end{align*}

	$\epsilonB$ is a rank 4 tensor, 2 indices specify the point of evaluation, 2 indices specify xy, yx, xx, yy, etc.
	From Maxwell's equations, we start with
	\begin{equation}
		\nabla \times \vecH = j \omega \boldsymbol{\epsilon_p} \cdot \vecE
	\end{equation}

	Where $\epsilonB_p$ is $\epsilonB$ evaluated at a point, p.
	\begin{align*}
		\epsilonB_p^{-1} \cdot (\nabla \times \vecH) &= j \omega \vecE\\
		\nabla \times (\epsilonB_p^{-1} \cdot (\nabla \times \vecH)) &= j \omega (\nabla \times \vecE )\\
													&= j \omega ( - j \omega) \mu \vecH\\
													&=\omega^2 \mu \vecH
	\end{align*}

	Write in tensor notation, with $\partial_k = \frac{\partial}{\partial x^k}$ and $\epsilonB^{-1}=\bar{\epsilonB}$
	\begin{align}
		\big[\nabla \times (\bar{\epsilonB} \cdot (\nabla \times \vecH))\big]^i &= \omega^2 \mu \vecH^i \\
		\levi^{ij}_k \partial_j \bar{\epsilonB}^k_l \levi^{lm}_n \partial_m H^n &= \omega^2 \mu H^i\\
		\bar{\epsilonB}^k_l \levi^{ij}_k \levi^{lm}_n \partial_j \partial_m H^n &= \omega^2 \mu H^i
		\label{eqn:tensor_notation}
	\end{align}

	Where the inverse of $\epsilonB$ is $\footnote{Stover, Christopher and Weisstein, Eric W. "Matrix Inverse." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/MatrixInverse.html}$ \\
	\begin{align}
		\bar{\epsilonB} &= \frac{1}{\epsilon_0}
		\begin{bmatrix}
			\frac{\epsilon_{yy}}{\epsilon_{xx} \epsilon_{yy} - \epsilon_{xy}\epsilon_{yx}} & -\frac{\epsilon_{xy}}{\epsilon_{xx} \epsilon_{yy} - \epsilon_{xy}\epsilon_{yx}} & 0 \\[1ex]
			-\frac{\epsilon_{yx}}{\epsilon_{xx} \epsilon_{yy} - \epsilon_{xy}\epsilon_{yx}} & \frac{\epsilon_{xx}}{\epsilon_{xx} \epsilon_{yy} - \epsilon_{xy}\epsilon_{yx}} & 0 \\[1ex]
			0 & 0 & \frac{1}{\epsilon_{zz}}
		\end{bmatrix}
		\label{matrix:inverse}
	\end{align}

	\section*{Evaluate Transverse Fields for the Diagonal Case}
	For the diagonal case where $\epsilon_{xy} = \epsilon_{yx} = 0$, we have

	\begin{align}
		\bar{\epsilonB} &= \frac{1}{\epsilon_0}
		\begin{bmatrix}
			\frac{1}{\epsilon_{xx}} & 0 & 0 \\[1ex]
			0 & \frac{1}{\epsilon_{yy}} & 0 \\[1ex]
			0 & 0 & \frac{1}{\epsilon_{zz}}
		\end{bmatrix}  \\
		&\equiv \frac{1}{\epsilon_0}
		\begin{bmatrix}
			a^1 & 0 & 0 \\[1ex]
			0 & a^2 & 0 \\[1ex]
			0 & 0 & a^3
		\end{bmatrix}
		\label{matrix:inverse}
	\end{align}

	Using eqn \ref{eqn:tensor_notation} for the diagonal case (eqn \ref{matrix:inverse}), we evaluate $H^x$ and $H^y$ with the help of Table \ref{table:Hx} and \ref{table:Hy} to track indices

	\begin{align*}
		\bar{\epsilonB}^k_l \levi^{ij}_k \levi^{lm}_n \partial_j \partial_m H^n &= \omega^2 \mu H^i\\[1ex]
	\end{align*}

	\begin{table}[h]
	\centering
	\begin{tabular}{|c|c|c|c|c|c|c|c|}
	\hline
	\textbf{i} & \textbf{j} & \textbf{k} & \textbf{sgn} & \textbf{l} & \textbf{m} & \textbf{n} & \textbf{sgn} \\ \hline
	1          & 2          & 3          & +            & 3          & 1          & 2          & +            \\ \hline
	1          & 2          & 3          & +            & 3          & 2          & 1          & -            \\ \hline
	1          & 3          & 2          & -            & 2          & 3          & 1          & +            \\ \hline
	1          & 3          & 2          & -            & 2          & 1          & 3          & -            \\ \hline
	\end{tabular}
	\caption{$H^x$, $i=1$}
	\label{table:Hx}
	\end{table}

	\begin{table}[h]
	\centering
	\begin{tabular}{|c|c|c|c|c|c|c|c|}
	\hline
	\textbf{i} & \textbf{j} & \textbf{k} & \textbf{sgn} & \textbf{l} & \textbf{m} & \textbf{n} & \textbf{sgn} \\ \hline
	2          & 3          & 1          & +            & 1          & 2          & 3          & +            \\ \hline
	2          & 3          & 1          & +            & 1          & 3          & 2          & -            \\ \hline
	2          & 1          & 3          & -            & 3          & 1          & 2          & +            \\ \hline
	2          & 1          & 3          & -            & 3          & 2          & 1          & -            \\ \hline
	\end{tabular}
	\caption{$H^y$, $i=2$}
	\label{table:Hy}
	\end{table}

	\subsection*{Evaluate $i=1$, $H^x$}

	\begin{align}
		a^3 (\partial_2 \partial_1 H^2 - \partial_2 \partial_2 H^1) + a^2 (\partial_3 \partial_1 H^3 - \partial_3 \partial_3 							H^1) &= \omega^2 \mu H^1 \\
		a^3 (\partial_y \partial_x H^y - \partial_y \partial_y H^x) + a^2 (\partial_z \partial_x H^z - \partial_z \partial_z 							H^x) &= \omega^2 \mu H^x
	\end{align}

	Using the fact that $z$ dependence of the field components stands alone, we can evaluate $\partial_z$ any time. Using eqn \ref{eqn:partial_z} and \ref{eqn:H_z},

	\begin{align}
		a^3 (\partial_y \partial_x H^y - \partial_y \partial_y H^x) + a^2 (- \partial_x (\partial_x H^x + \partial_y H^y) - (- \beta^2) H^x) &= \omega^2 \mu H^x\\
		\frac{1}{e_{zz}} (\partial_y \partial_x H^y - \partial_y \partial_y H^x) + \frac{1}{e_{yy}} ( \beta^2 H^x- \partial_x\partial_x H^x - \partial_x \partial_y H^y) &= \omega^2 \mu \epsilon_0 H^x\\
		\frac{e_{yy}}{e_{zz}} (\partial_y \partial_x H^y - \partial_y \partial_y H^x) + \beta^2 H^x- \partial_x\partial_x H^x - \partial_x \partial_y H^y &= e_{yy} \omega^2 \mu \epsilon_0 H^x\\
		\Aboxed{e_{yy} \omega^2 \mu \epsilon_0 H^x + \partial_x\partial_x H^x + \partial_x \partial_y H^y - \frac{e_{yy}}{e_{zz}} \partial_y \partial_x H^y + \frac{e_{yy}}{e_{zz}} \partial_y \partial_y H^x &= \beta^2 H^x}
		\label{eqn:4a}
	\end{align}

	\subsection*{Evaluate $i=2$, $H^y$}

	\begin{align}
		a^1 (\partial_3 \partial_2 H^3 - \partial_3 \partial_3 H^2) + a^3 (\partial_1 \partial_2 H^1 - \partial_1 \partial_1 							H^2) &= \omega^2 \mu H^2 \\
		a^1 (\partial_z \partial_y H^z - \partial_z \partial_z H^y) + a^3 (\partial_x \partial_y H^x - \partial_x \partial_x 							H^y) &= \omega^2 \mu H^y \\
		a^1 ( - \partial_y (\partial_x H^x + \partial_y H^y) - (-\beta^2) H^y) + a^3 (\partial_x \partial_y H^x - \partial_x \partial_x H^y) &= \omega^2 \mu H^y \\
		\frac{1}{e_{xx}} (\beta^2 H^y - \partial_y \partial_x H^x - \partial_y \partial_y H^y) + \frac{1}{e_{zz}} (\partial_x \partial_y H^x - \partial_x \partial_x H^y) &= \omega^2 \mu \epsilon_0  H^y \\
		\Aboxed{e_{xx} \omega^2 \mu \epsilon_0 H^y + \partial_y \partial_x H^x + \partial_y \partial_y H^y - \frac{e_{xx}}{e_{zz}} \partial_x \partial_y H^x + \frac{e_{xx}}{e_{zz}} \partial_x \partial_x H^y &= \beta^2 H^y}
		\label{eqn:4b}
	\end{align}

	To be consistent Fallahkhair and Murphy, we assume transverse components have continuous second partials. Therefore, $\partial_x \partial_y = \partial_y \partial_x$. We can consolidate terms in \ref{eqn:4a} and \ref{eqn:4b} to be

	\begin{align}
		\Aboxed{\omega^2 \mu \epsilon_0 e_{yy} H^x + \partial_x\partial_x H^x + (1 - \frac{e_{yy}}{e_{zz}}) \partial_y \partial_x H^y + \frac{e_{yy}}{e_{zz}} \partial_y \partial_y H^x &= \beta^2 H^x}
		\label{eqn:4a*}\\
		\Aboxed{\omega^2 \mu \epsilon_0 e_{xx} H^y + \partial_y \partial_y H^y + (1 - \frac{e_{xx}}{e_{zz}}) \partial_x \partial_y H^x + \frac{e_{xx}}{e_{zz}} \partial_x \partial_x H^y &= \beta^2 H^y}
		\label{eqn:4b*}
	\end{align}

	\begin{figure}[h]
		\centering
		\includegraphics[scale=0.5]{grid.png}
		\caption{Fig 1}
		\label{fig:grid_points}
	\end{figure}

	We use the following approximations, using the convention in Fig $\ref{fig:grid_points}$.
	\begin{align*}
		\Partialsqx{H_x^p} &= \frac{H_x^w + H_x^e - 2H_x^p}{(\Delta x)^2} \\
		\Partialsqy{H_x^p} &= \frac{H_x^s + H_x^n - 2H_x^p}{(\Delta y)^2} \\
		\frac{\partial^2 H_x^p}{\partial x \partial y} &= \frac{H_x^{ne} + H_x^{sw} - H_x^{nw} -
															H_x^{se}}{4(\Delta x)(\Delta y)}\\
		\frac{\partial^2 H_x^p}{\partial x \partial y} &= \frac{\partial^2 H_x^p}{\partial y \partial x}
	\end{align*}

	to discretize eqn \ref{eqn:4a*} and \ref{eqn:4b*}

	\begin{align}
	\begin{split}
		\beta^2 H^x_p = \omega^2 \mu \epsilon_0 H^x_p &+ \frac{H_x^w + H_x^e - 2H_x^p}{(\Delta x)^2} + (1 - \frac{e_{yy}}{e_{zz}}) \frac{H_y^{ne} + H_y^{sw} - H_y^{nw} - H_y^{se}}{4(\Delta x)(\Delta y)} + \frac{e_{yy}}{e_{zz}} \frac{H_x^s + H_x^n - 2H_x^p}{(\Delta y)^2}
		\label{eqn:4a_descrete}
		\end{split}
	\end{align}

	\begin{align}
	\begin{split}
		\beta^2 H^y_p = \omega^2 \mu \epsilon_0 H^y_p &+ \frac{H_y^s + H_y^n - 2H_y^p}{(\Delta y)^2}  + (1 - \frac{e_{xx}}{e_{zz}}) \frac{H_x^{ne} + H_x^{sw} - H_x^{nw} - H_x^{se}}{4(\Delta x)(\Delta y)} + \frac{e_{xx}}{e_{zz}} \frac{H_y^w + H_y^e - 2H_y^p}{(\Delta x)^2}
		\label{eqn:4b_descrete}
	\end{split}
	\end{align}

	The coefficients in \ref{eqn:4a_descrete} and \ref{eqn:4b_descrete} are all to be evaluated at corresponding points they are multiplied with. We have an efficient scheme for this described in the next section.\\

	\begin{align}
	\begin{split}
		\beta^2 H^x_p = L_1 H^x_p &+ L_2(H^x_w + H^x_e - 2H^x_p) + L_3 (H^y_{ne} + H^y_{sw} - H^y_{nw} - H^y_{se}) + L_3 (H^x_s + H^x_n - 2H^x_p)
		\label{eqn:4a_descrete*}
		\end{split}
	\end{align}

	\begin{align}
	\begin{split}
		\beta^2 H^y_p = K_1 H^y_p &+ K_2(H^y_s + H^y_n - 2H^y_p)  + K_3(H^x_{ne} + H^x_{sw} - H^x_{nw} - H^x_{se})+ K_4(H^y_w + H^y_e - 2H^y_p)
		\label{eqn:4b_descrete*}
	\end{split}
	\end{align}

	\section*{Constructing the EigenMatrix - General Idea}

		\indent The general idea is to "unwrap" the columns of the calculation window into one long column vector and create a matrix of relationships between the elements. In the scalar wave code, we were able to execute a "brute force" method by supplying the entries of that matrix. But with the vector wave, the matrix gets large fast and we run out of memory fast. So the trick is to create a "sparse" matrix using sparse() which creates a list of entries and their values. Entries corresponding to values of 0 are effectively null and don't take up memory. \\
		\indent We will "unwrap" the computational window into a single column. So solving the transverse magnetic field becomes an eigenvalue problem. The "EigenMatrix" contains the coefficients relating the components of the transverse field. A transverse component, say $H^x$, has a value at every point of the $M \times N$ computational window. \\

			\begin{center}
			$H^x$ =
			$\begin{bmatrix}
				1 & M+1 &  \hdots &  \\
				2 & M+2 &    &    \\
				3 & M+3 &  \ddots &   \vdots \\
				\vdots & \vdots &    & \vdots \\
				M &  & \hdots   & MN
			\end{bmatrix}
			\rightarrow
			\begin{bmatrix}
			1 \\ 2 \\ 3 \\ \vdots \\ \vdots \\ \vdots \\ nx*ny
			\end{bmatrix}  $\\

		\end{center}

		We want to unwrap $H^x$ and $H^y$ and glue them together to get the following
		\begin{center}$
			\begin{bmatrix}
				EigenMatrix
			\end{bmatrix}
			\begin{bmatrix}
				H_x \\ H_y
			\end{bmatrix}
			\rightarrow
			\begin{bmatrix}
				EigenMatrix
			\end{bmatrix}
			\begin{bmatrix}
				H_1 \\ H_2 \\ \vdots \\ H_{MN} \\ H_{MN+1} \\ \vdots \\ H_{2MN}
			\end{bmatrix}
		$\end{center}

		 Table \ref{table:calculation_matrix_diagram} shows a diagram for the matrices we have to construct. The inner parts, marked x, are of size $M \times N$. They are $MN$ entries describing our waveguide, consistent with the refractive index matrix. But for calculation purposes, we have to construct an $M+2 \times N+2$ size matrix because sparse() requires arguments of consistent dimensions and because we need to approximate the edge points. The padding entries are constructed using the edge entries.

		 \begin{table}[h]
		\centering
		\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
		\hline
			 &   &   &   &   &   &   &   &   &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 & x & x & x & x & x & x & x & x &  \\ \hline
			 &   &   &   &   &   &   &   &   &  \\ \hline
			\end{tabular}
		\caption{Calculation Matrix Diagram}
		\label{table:calculation_matrix_diagram}
	\end{table}

		\section*{Coordinate Entries}
			We see from the diagram above Table \ref{table:calculation_matrix_diagram} that the $M \times N$ entries for both the $H_x$ and $H_y$ matrices correspond to some column of the eigenMatrix. We need to keep track of the relationship between the entries in those and their corresponding entries in our unwrapped column. So lets create two $M \times N$ matrix of "coordinates" that label each entry with their corresponding entry in the column vector.\\

			\begin{center}
			$Hx\_coord =
			\begin{bmatrix}
				1 & M+1 &  \hdots &  \\
				2 & M+2 &    &  \\
				3 & M+3 &  \ddots & \vdots \\
				\vdots &    &  & \vdots \\
				M &  & \hdots   & MN
			\end{bmatrix}
			\linebreak \linebreak
			Hy\_coord =
			\begin{bmatrix}
				MN+1  &  \hdots &  \\
				MN+2  &    &  \\
				MN+3   \ddots & \vdots \\
				\vdots &      & \vdots \\
				... &  \hdots   & 2MN
			\end{bmatrix} $
			\end{center}

%		\subsection*{Example}
%			Refer to Fig \ref{fig:grid_points} and Eqn \ref{eqn:H_x*}. Suppose some point $p$ under consideration corresponds %to column 12. Then the point $p$ itself for $H_x$, denote $H_x^p$, will correspond to the matrix coordinate (13,13). The $nw$ %point relative to $p$ for $H_x$, denote $H_x^{nw}$, will correspond to the matrix coordinate (13-(M+2)-1,13). The point $p$ %for $H_y$ relative to $p$, denote $H_y^p$, will correspond to the matrix coordinate (13+(M+2)(N+2), 13+(M+2)(N+2)).

%		\section*{Tensor Entries}
%			Matrices containing information on our waveguide include matrices corresponding to $\epsilon_{xx}, \epsilon_{yy}, %\epsilon_{zz}, n$. Each of these are MxN matrices containing their corresponding values at each point in the waveguide. They %will be padded the same way as above turning each into a M+2 x N+2 matrix.

%		\section*{The Indexer Matrix}
%			So far, our matrices have been padded along each edge and contain values we need for processing or calculating. We %now construct an indexing matrix to use for pulling these values. It will be an M+2 x N+2 matrix
%			\begin{center}
%			$ index =
%			\begin{bmatrix}
%				1 & M+3 &  \hdots &  \\
%				2 & M+4 &    &    \\
%				3 & M+5 &  \ddots &   \vdots \\
%				\vdots & \vdots &    & \vdots \\
%				M+2 &  & \hdots   & (M+2)(N+2)
%			\end{bmatrix}
%			$\\
%			\end{center}

%			\subsection*{Example}
%			$index(2:M-1, 2)$ contains entries corresponding to the left edge of our waveguide.

\end{document}


\documentclass{article}
\input{shared_preamble.tex}
\title{Exercises for Chapter 2: Linear Algebra}
\begin{document}
\maketitle

\section*{Exercises}
\begin{enumerate}
\item The expression $\alpha \vu$ for $\alpha \in \R$ and
unit vector $\vu \in \R^n$ defines a line of points that may be obtained by varying the
value of $\alpha$.
Derive an expression for the point $\vy$ that lies on this line that is
as close as possible to an arbitrary point $\vx \in \R^n$.
This operation of replacing a point by its nearest member within some set
is called {\em projection}.

{\em Exercise contributed by Ian Goodfellow}
\end{enumerate}

\section*{Solutions}
\begin{enumerate}
\item We begin by defining the distance from $\vy$ to $\vx$. We would like to
find the $\vy$ that minimizes this distance:
\begin{equation}
|| \vx - \vy||^2.
\end{equation}
Next, we need to enforce the constraint that $\vy$ lies on the line defined
by $\alpha \vu$. We can do this simply by defining $\vy$ to be $\alpha \vu$.
\begin{equation}
|| \vx - \alpha \vu ||^2.
\end{equation}
Next, we expand the expression:
\begin{align}
& || \vx - \alpha \vu ||^2 \\
=& ( \vx - \alpha \vu )^\top (\vx - \alpha \vu) \\
=& \vx^\top \vx - 2 \alpha \vx^\top \vu + \alpha^2 \vu^\top \vu \\
=& \vx^\top \vx - 2 \alpha \vx^\top \vu + \alpha^2.
\end{align}
In the last line, we used the fact that $\vu$ is a unit vector to make
the simplification $\vu^\top \vu = 1$.

We can minimize this distance by taking the derivative with respect to
$\alpha$ and setting it to zero:
\begin{align}
& - 2  \vx^\top \vu + 2 \alpha = 0 \\
\Rightarrow & \alpha = \vx\top \vu.
\end{align}

Recalling that $\vy = \alpha \vu$, we can conclude that $\vy = \vx^\top \vu \vu$.

{\em Solution contributed by Ian Goodfellow}

\end{enumerate}

\end{document}


\section{Evaluating Information Retrieval}
\label{sec:EvaluatingInformationRetrieval}

% ##############################################################################
\subsection{Evaluating Bounding Box Prediction}
\label{ssec:EvaluatingBoundingBoxPrediction}

\subsubsection{Intersection Over Union}
\label{sssec:IntersectionOverUnion}

\glslocalreset{iou}

\gls{iou} measures the overlap between two \glspl{bbox}. Let $\subsup{\vect{b}}{1}{T} = \sbrackets{x_1, y_1, w_1, h_1}$ and $\subsup{\vect{b}}{2}{T} = \sbrackets{x_2, y_2, w_2, h_2}$ be two \glspl{bbox} described by vectors containing $4$ elements. The respective elements are given by $x$, $y$ coordinates of the top-left corner and the \gls{bbox} width and height. The intersection area between $\vect{b}_1$ and $\vect{b}_2$ is defined as
\begin{equation}
    \begin{aligned}
        \vect{b}_1 \cap \vect{b}_2 =
         & \max \cbrackets{0,
            \min \cbrackets{x_1 + w_1, x_2 + w_2} - \max \cbrackets{x_1, x_2} + 1}
        \times                \\
         & \max \cbrackets{0,
            \min \cbrackets{y_1 + h_1, y_2 + h_2} - \max \cbrackets{y_1, y_2} + 1},
    \end{aligned}
\end{equation}
and the area of their union is given by
\begin{equation}
    \vect{b}_1 \cup \vect{b}_2 = w_1 h_1 + w_2 h_2 - \vect{b}_1 \cap \vect{b}_2.
\end{equation}
Then, the final \gls{iou} metric between $\vect{b_1}$ and $\vect{b_2}$ is computed as (\figtext{}~\ref{fig:IntersectionOverUnion})
\begin{equation}
    \label{eq:IntersectionOverUnion}
    \func{\IOU}{\vect{b}_1, \vect{b}_2} =
    \frac{\vect{b}_1 \cap \vect{b}_2}{\vect{b}_1 \cup \vect{b}_2},
\end{equation}
where $0 \leq \func{\IOU}{\vect{b}_1, \vect{b}_2} \leq 1$, such that value of $0$ represents no intersection, while value of $1$ represents a complete overlap. In terms of object detection or object tracking evaluation, an \gls{iou} threshold, $t$, such that $0 \leq t \leq 1$, can be associated with this metric, denoting the decision boundary between \gls{tp} and \gls{fp}.

% ------------------------------------------------------------------------------
\begin{figure}[t]
    \centerline{\includegraphics[width=0.2\linewidth]{figures/theoretical_foundations/intersection_over_union.pdf}}
    \caption[\gls{iou} visualization]{Computation of the \gls{iou} metric between two \glspl{bbox} using of ratio of the area of overlap and the area of the union.}
    \label{fig:IntersectionOverUnion}
\end{figure}
% ------------------------------------------------------------------------------

% ##############################################################################
\subsection{Mean Average Precision}
\label{ssec:MeanAveragePrecision}

\glslocalreset{map}

A commonly used metric for evaluating tracking algorithms, document searching systems, object detection, and object \gls{reid} is \gls{map}. It measures the success rate of an information retrieval algorithm.

\subsubsection{Object Re-Identification}
\label{sssec:ObjectReIdentification}

A frequent use case in the context of object \gls{reid} is to use \gls{map} to assess the search results for a particular query using Euclidean distance or cosine similarity as a metric. Oftentimes the model is trained with the intent to use one of these trivial metrics. Furthermore, this approach is often paired with \emph{top-k} accuracy, typically \emph{top-1}, \emph{top-2} and \emph{top-5}.

In a typical \gls{reid} evaluation setup, there is a query set and a gallery set. For each object in the query set the aim is to retrieve a similar identity from the gallery set. The computation of the \gls{ap} for a query image $q$ is thus defined as
\begin{equation}
    \label{eq:AveragePrecision}
    \func{\AP}{q} = \frac{1}{\func{N_{gt}}{q}} \sum_{k} \func{P}{k} \times {\delta}_k,
\end{equation}
where $\func{P}{k}$ represents precision at rank $k$, $\func{N_{gt}}{q}$ is the total number of true retrievals for the query $q$. The indicator ${\delta}_k$ is equal to $1$ when the matching of query image $q$ to a test image is correct at rank $r$, such that $1 \leq r \leq k$. The \gls{map} is then calculated as average over all query images, concretely
\begin{equation}
    \label{eq:MeanAveragePrecision}
    \MAP = \frac{1}{Q} \sum_q \func{\AP}{q},
\end{equation}
where $Q$ is the total number of query images, as described in~\cite{kuma2019vehiclereid}. \eqtext{}~\ref{eq:MeanAveragePrecision} tells us that, for a given query $q$, we calculate its corresponding \gls{ap} (\eqtext{}~\ref{eq:AveragePrecision}), and then take the mean of the all these \gls{ap} scores.

\subsubsection{Object Detection}

Object detectors seek to identify the presence of objects in images. The evaluation metric of such a model has to take the \gls{bbox} prediction into account, as there can be just a partial overlap of the predicted \gls{bbox} with the ground truth one.

In a ranked retrieval context, appropriate sets of retrieved documents are naturally given by the \emph{top-k} retrieved documents and for each such set, the \gls{pr} curve can be plotted. With this in mind, recall is defined as the proportion of all positive examples ranked above a given rank. Precision is the proportion of all examples above that rank which are from the positive class. In~\cite{everingham2010pascalvoc}, the \gls{ap} is computed for $11$ equally spaced discrete recall levels, specifically $\sbrackets{0.0, 0.1, 0.2, \dots, 1.0}$, using
\begin{equation}
    \AP = \frac{1}{11} \sum_{r \in \cbrackets{0.0, 0.1, \dots, 1.0}} \func{p_{interp}}{r},
\end{equation}
where the precision at each recall level $r$ is interpolated by taking the maximum precision measured for a method for which the corresponding recall exceeds $r$. Precision interpolation is used to remove the \emph{zig-zag} pattern by evaluating
\begin{equation}
    \func{p_{interp}}{r} = \underset{\tilde{r}:\tilde{r} \geq r}{\max} \quad \func{p}{\tilde{r}},
\end{equation}
with $\func{p}{\tilde{r}}$ representing the measures precision at a specific recall level $\tilde{r}$~\cite{salton1983introduction}.


%==============================================================================
\chapter{Harmonic Oscillators}
\label{chap:harosc}
%==============================================================================

%------------------------------------------------------------------------------
\section{Single Harmonic Oscillator}
\label{sec:single-harosc}
%------------------------------------------------------------------------------

A single harmonic oscillator is described by the Hamiltonian
\begin{equation}
H = \frac{1}{2}\rbr{\pi^2 + \Omega^2f^2},
\label{eq:hamiltonian-simple-harosc}
\end{equation}
where $\rbr{\pi, f}$ are conjugate momentum and position, respectively. Here
a unit system similar to the case in field theory is adapted. Canonical
quantisation uses the annihilation and creation operators
\begin{equation}
a^- \coloneqq \frac{1}{\sqrt{2}}\rbr{\sqrt{\Omega}f +
\frac{\ii}{\sqrt{\Omega}}\pi}, \qquad
a^+ \coloneqq \frac{1}{\sqrt{2}}\rbr{\sqrt{\Omega}f
-\frac{\ii}{\sqrt{\Omega}}\pi} \equiv \rbr{a^-}^\dagger;
\end{equation}
inverse expressions read
\begin{equation}
f = \frac{a^++a^-}{\sqrt{2\Omega}},\qquad
\pi = \ii\sqrt{\frac{\Omega}{2}}\rbr{a^+-a^-}.
\end{equation}
The ground state and the normalised $n$th excitation are defined by
\begin{equation}
a^-\Ket{0} \coloneqq 0,\qquad
\Ket{n} \coloneqq \frac{1}{\sqrt{n!}}\rbr{a^+}^n\Ket{0},\qquad
n \in \BbbZ_+,
\end{equation}
the wave functions of which are
\begin{equation}
\Braket{f | n} = \frac{1}{\sqrt{2^n n!}}
\ee^{-\Omega f^2/2} \rfun{H_n}{\sqrt{\Omega}f}, \qquad
n \in \BbbZ,
\end{equation}
where $\rfun{H_n}{x}$ is the $n$th Hermite polynomial. The normalising measure
\begin{equation}
\rfun{\dd \mu}{f} \coloneqq \sqrt{\frac{\Omega}{\pp}}\,\dd f
\end{equation}
is used throughout, so that the completeness relation holds,
\begin{align}
\int \rfun{\dd \mu}{f}\,\Braket{\alpha | f}\Braket{f | \beta} \equiv
\Braket{\alpha | \beta}.
\end{align}

In the present work, a general Gaussian state $\Ket{\omega}$ has also been
considered, the wave function of which reads
\begin{equation}
\Braket{f | \omega} = \rbr{\frac{\Re\omega}{\Omega}}^{1/4}
\rfun{\exp}{-\frac{\omega}{2}f^2},\qquad \Re \omega > 0
\label{eq:general-single-omega}
\end{equation}
Evaluating the expectation of $f^2$ yields
\begin{equation}
\abr{f^2}_\omega \coloneqq \Braket{\omega | f^2 | \omega}
= \rbr{2\Re \omega}^{-1}.
\label{eq:correlator-1}
\end{equation}
$\Ket{\omega}$ can also be expressed in terms of energy eigenstates,
\begin{align}
\Braket{n | \omega} &= \int \rfun{\dd \mu}{f}\, \Braket{n | f}
\Braket{f | \omega} \nonumber \\
&= \rbr{\frac{\Omega \Re\omega}{\pp^2}}^{1/4} \frac{1}{\sqrt{2^n n!}}
\int_{-\infty}^{+\infty} \dd f\,
\cfun{\exp}{-\frac{1}{2}\rbr{\Omega+\omega}f^2}\rfun{H_n}{\sqrt{\Omega}f}
\nonumber \\
&= \begin{cases} \rbr{\Omega  \Re(\omega )}^{\frac{1}{4}}
\dfrac{2^{\frac{1}{2}-m} \sqrt{(2 m)!} }{m!}
\dfrac{(\Omega -\omega )^m}{(\omega +\Omega )^{m+\frac{1}{2}}}
& n = 2m, \\
0 & n = 2m+1,
\end{cases}
\label{eq:n|omega}
\end{align}
thanks to \cite{Babusci2012}.

A thermal state of the oscillator at temperature $T$ can be described by the
density operator
\begin{equation}
\rho \coloneqq \frac{1}{Z} \cfun{\exp}{-\frac{H}{T}} = \frac{1}{Z}
\sum_{n=0}^{+\infty} \cfun{\exp}{-\frac{\Omega}{T}\rbr{n+\frac{1}{2}}}
\Ket{n}\Bra{n},
% \nonumber \\
%&= Z^{-1} \sum_{n=0}^{+\infty}
%\frac{\ee^{-\frac{\Omega}{T}\rbr{n+\frac{1}{2}}}}{n!}
%\rbr{a^+}^n\Ket{0}\Bra{0}\rbr{a^-}^n,
\label{eq:single-thermal-density-mat}
\end{equation}
where the partition function is
\begin{equation}
Z \coloneqq \tr{\ee^{-H/T}} = \frac{1}{2} \csch\frac{\Omega}{2 T}.
\end{equation}
One also obtains
\begin{align}
\abr{f^2}_\rho &= \frac{1}{2\Omega}\frac{1}{Z}\sum_{n=0}^{+\infty}
\cfun{\exp}{-\frac{\Omega}{T}\rbr{n+\frac{1}{2}}}
\Braket{n | \rbr{a^++a^-}^2 | n} \nonumber \\
%&= \frac{1}{2\Omega}\frac{1}{Z}\sum_{n=0}^{+\infty}
%\ee^{-\frac{\Omega}{T}\rbr{n+\frac{1}{2}}}
%\Braket{n | \rbr{\rbr{a^+}^2+a^+a^-+a^-a^++\rbr{a^-}^2} | n} \nonumber \\
&= \frac{1}{2\Omega}\frac{1}{Z}\sum_{n=0}^{+\infty}
\cfun{\exp}{-\frac{\Omega}{T}\rbr{n+\frac{1}{2}}}
\Braket{n | \rbr{2a^+a^-+1} | n} %\nonumber \\
%&= \frac{\coth\frac{\Omega}{2 T}}{2\Omega}
= \frac{1}{4T} \frac{\coth\frac{\Omega}{2 T}}{\frac{\Omega}{2 T}}.
\label{eq:sho-th-cor-1}
\end{align}
%\Cref{eq:single-thermal-density-mat} expressed in terms of position basis can
%also be calculated
%\begin{align}
%\rho &= \frac{1}{Z}
%\sum_{n=0}^{+\infty} \ee^{-\frac{\Omega}{T}\rbr{n+\frac{1}{2}}}
%\int\rfun{\dd\mu}{f_1}\,\rfun{\dd\mu}{f_2}\,
%\Ket{f_1}\Braket{f_1 | n}\Braket{n | f_2}\Bra{f_2}
%\nonumber \\
%&= \frac{1}{Z} \int\rfun{\dd\mu}{f_1}\,\rfun{\dd\mu}{f_2}\,\Ket{f_1}
%\rbr{\sum_{n=0}^{+\infty}\ee^{-\frac{\Omega}{T}\rbr{n+\frac{1}{2}}}
%\Braket{f_1 | n}\Braket{n | f_2}}
%\Bra{f_2};
%\label{eq:thermal-position-basis-1}
%\end{align}
%The summation in \cref{eq:thermal-position-basis-1} can be closed
%\begin{equation}
%\rfun{\dd\mu}{f_1}\,\rfun{\dd\mu}{f_2}\,
%\sum_{n=0}^{+\infty}\Braket{f_1 | n}\Braket{n | f_2}
%= \dd f_1\,\dd f_2 \rfun{\delta}{f_1 - f_2},
%\end{equation}
%due to orthonormality and completeness of the bases.

%------------------------------------------------------------------------------
\section{Multiple Harmonic Oscillators}
%------------------------------------------------------------------------------

Multiple harmonic oscillators are described by the Hamiltonian
\begin{equation}
H = \sum_i H_i,\qquad H_i = \frac{1}{2}\rbr{\pi_i^2 + \Omega^2f_i^2},
\end{equation}
where $\rbr{\pi_i, f_i}$ are conjugate momentum and position of the $i$th
oscillator.

When the general Gaussian state
\begin{equation}
\Braket{\cbr{f} | \sbr{\omega}} = \rbr{\det\frac{\omega}{\Omega}}^{1/4}
\rfun{\exp}{-\frac{1}{2}f_i \omega_{ij} f_j}
\label{eq:general-ij-omega}
\end{equation}
is considered, where $\omega_{ij}$ is a real symmetric positive-definite
matrix, the two-point correlator becomes
\begin{equation}
\Braket{\sbr{\omega} | f_i f_j | \sbr{\omega}} = \sbr{\rbr{2\omega}^{-1}}_{ij},
\label{eq:multi-har-cor}
\end{equation}
which is a generalisation of \cref{eq:correlator-1}.

A thermal state at temperature $T$ can be described by the density operator
\begin{equation}
\rho = \bigotimes_i \rho_i,\qquad \rho_i =
\frac{1}{Z} \rfun{\exp}{-\frac{H_i}{T}}.
\label{eq:multi-thermal-density-mat}
\end{equation}
Computing the two-point correlator of the state, one finds
\begin{equation}
\abr{f_if_j}_\rho = \frac{\coth\frac{\Omega_i}{2T}}{2\Omega_i}
\delta_{ij}
= \frac{1}{4T} \frac{\coth\frac{\Omega_i}{2T}}{\frac{\Omega_i}{2T}}
\delta_{ij}
\label{eq:correlator-multiple-thermal}
\end{equation}
from \cref{eq:sho-th-cor-1}.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../mythesis"
%%% End:


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\usepackage{amssymb}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{April 4, 2014}
\maketitle
\section*{7.2 generating functions}
we have fibonacci \#'s $0,1,1,2,3,5,8,\dots$. We wish to construct an algebraic function which encodes this sequence (or any sequence of numbers)

Given a sequence $h_0,h_1,h_2,h_3,\dots$ define it's generating function to be the infinite series $g(x)=h_0+h_1x+h_2x^2+h_3x^3+\dots$

\subsubsection*{example}
Generating function for the fibanocci numbers is $f(x)=0+1\cdot x+1\cdot x^2+2\cdot x^3+3\cdot x^4+\dots$.

\subsubsection*{goal}
express generating functions in closed form.

\subsubsection*{example}
find the generating function for the sequence $1,1,1,\dots$. Generating function is $f(x)=1+x+x^2+x^3+\dots=\frac{1}{1-x}$. this is the geometric series formula.

\subsubsection*{example}
Find the generating function for the sequence $1,1,1,\dots,1,0,0,\dots$. This is a finite series.
\[f(x)=1+x+x^2+\dots+x^{n-1}=\frac{1-x^n}{1-x}\]
,
\subsubsection*{example}
what is the generating function for the sequence for $\binom{n}{0},\binom{n}{1},\binom{n}{2},\binom{n}{3},\dots,\binom{n}{n},0,0,\dots$. binomial theorem
\[f(x)=(1+x)^n\]

or if $\alpha\in\mathbb{R}$ the generating function for $\binom{\alpha}{0},\binom{\alpha}{1},\binom{\alpha}{2},\binom{\alpha}{3},\dots,\binom{\alpha}{n},0,0,\dots$
\begin{enumerate}
\setcounter{enumi}{12}
\item
\begin{enumerate}
\item
$\frac{1}{1-cx}$
\item
$\frac{1}{1+x}$
\setcounter{enumii}{3}
\item
$e^x$
\end{enumerate}
\end{enumerate}

\subsubsection*{example}
the generating function for the fibonacci numbers is $f(x)=0+x+x^2+2x^3+3x^4+\dots$
\begin{align*}
  \sum\limits_{n=1}^\infty{f_nx^n}&=f_0+f_1x+\sum\limits_{n=2}^\infty{f_nx^n}\\
  &=0+x+\sum\limits_{n=0}^\infty{f_{n+2}x^{n+2}}\\
  &=x+x^2\sum\limits_{n=0}^\infty{(f_{n+1}+f_n)x^n}\\
  &=x+x^2\sum\limits_{n=0}^\infty{f_{n+1}x^n}+x^2\sum\limits_{n=0}^\infty{f_nx^n}\\
  &=x+x\sum\limits_{n=0}^\infty{f_{n}x^n}+x^2\sum\limits_{n=0}^\infty{f_nx^n}\\
  (1-x-x^2)\sum\limits_{n=0}^\infty{f_{n}x^n}&=x\\
  f(x)&=\frac{x}{1-x-x^2}
\end{align*}
\subsubsection*{example}
find the generating function for $h_0,h_1,h_2,h_3,\dots$ where $h_n=\#$ n-combinations fo $\{\infty\cdot e_1,\infty\cdot e_2,\infty\cdot e_3,\infty\cdot e_4\}$

find $g(x)=\sum\limits_{k=0}^\infty{h_kx^k}$ so $h_n$ is the coefficient of $x^n$. $h_n=\#$ non-negative solutions: $e_1+e_2+e_3+e_4=n=\binom{n+4-1}{n}$ so $x^n=x^{e_1}x^{e_2}x^{e_3}x^{e_4}$ so consider the product $(1+x+x^2+\dots)(1+x+x^2+\dots)(1+x+x^2+\dots)(1+x+x^2+\dots)$ with each term coming from $x^{e_n}$. Generating function is $\frac{1}{(1-x)^4}$
\end{document}


\chapter{Cryptography}

Authentication, Integrity, and Confidentiality are the three objectives of secure communications. Authentication guarantees that a message comes from the source that it claims to come from. Integrity ensures that messages are not altered in transit. Confidentiality ensures so that only the receiver can read the message.\\

Encryption and hashing are used to make certain that only authorized entities can read the message (Confidentiality). The receiver can verify that the received message is identical to the sent message and that no manipulation occurred (Integrity). \\

\section{Hash function}

A hash function takes the message and produces a fixed-length and condensed bit string, called the hash value or message digest. Hashing is based on a one-way mathematical function that is relatively easy to compute, but significantly harder to reverse. The cryptographic hashing function can provide proof of authenticity (IPsec, routing protocol, CHAP) and message integrity check proof.\\

MD5 produces a 128-bit hashed message digest and is now considered a legacy algorithm.\\

The SHA-1 algorithm produces a 160-bit message digest and is slightly slower than MD5, but the larger message digest makes it more secure against brute-force collision and inversion attacks. SHA-1 is now considered to be a legacy algorithm. It is recommended to use the SHA-2 family of hash functions, which are SHA-224 (224 bit), SHA-256 (256 bit), SHA-384 (384 bit), SHA-512 (512 bit). \\

A keyed-hash message authentication code (HMAC or KHMAC) use a secret key as input to the hash function. This adds authentication to integrity assurance. The mechanism of HMAC is that two parties share a secret key and use HMAC functions for authentication. Only parties who have access to that secret key can compute the digest of an HMAC function. This characteristic defeats man-in-the-middle attacks. Cisco technologies use two well-known HMAC functions: legacy Keyed MD5 (HMAC-MD5) and Keyed SHA-1 (HMAC-SHA-1).\\

Two terms that are used to describe keys are: Key length (key size) and Keyspace (the number of possibilities that can be generated by a specific key length). As key length increase, the keyspace increases exponentially, which affects the time it takes to crack the code. Longer keys are more secure; however, they are also more resource intensive.

\section{Encryption}

Cryptographic encryption can provide confidentiality at several layers of the OSI model. For example, the IPsec for network layer protocols. Secure Sockets Layer (SSL) or Transport Layer Security (TLS), provide session layer confidentiality. MD5, Keyed MD5, and Secure Hash Algorithm 1 are examples of hash functions. They provide data integrity but not data confidentiality.\\

There are two classes of encryption algorithms: Symmetric algorithms and Asymmetric algorithms.

\subsection{Symmetric algorithms}

Symmetric algorithms use the same pre-shared key on both parties and the key is shorter, meaning faster execution. The encryption and decryption keys are the same. The sender and the receiver must exchange the secret key using a secure channel before any encryption can occur. By obtaining the key, anyone can encrypt and decrypt messages. DES, 3DES, AES, Software Encryption Algorithm (SEAL), and the Rivest ciphers (RC) series are all well-known encryption algorithms that use symmetric keys.\\

The most commonly used techniques in symmetric encryption cryptography are block ciphers and stream ciphers. Block ciphers transform a fixed-length block of plaintext into a common block of ciphertext of 64 or 128 bits. Unlike block ciphers, stream ciphers encrypt plaintext one byte or one bit at a time.\\

Two main criteria that should be considered when selecting an encryption algorithm for an organization: The algorithm is trusted by the cryptographic community, The algorithm adequately protects against brute-force attacks.\\

%Data Encryption Standard (DES) is a block cipher that uses 64bit key, of which 56 bits are used for encryption. The remaining 8 bits are used for parity. When forced to use DES, change keys frequently, use a secure channel to communicate the DES key, and use cipher block chaining (CBC) mode. \\

One way to increase the DES effective key length is to use the same algorithm with different keys several times in a row. The technique of applying DES three times in a row to a plaintext block is called 3DES. The Cisco IPsec implementation uses DES and 3DES in CBC mode.\\

Although 3DES is very secure, it is also resource intensive. To better manage resources, AES was chosen to replace DES. It has stronger key and runs faster than DES. Despite these advantages, AES is a relatively young algorithm. A mature algorithm, like 3DES, is always more trusted.\\

The Software-Optimized Encryption Algorithm (SEAL) is a stream cipher that uses a 160-bit encryption key. Because it is a stream cipher, data to be encrypted is continuously encrypted, which makes it much faster than block ciphers. SEAL has several restrictions: IPsec and Security version IOS are required, The router and the peer must not have hardware IPsec encryption.\\

The stream cipher RC4 is often used in file encryption products and for secure communications, such as within SSL. It can be implemented insecurely, as in Wired Equivalent Privacy (WEP). RC5 is a fast block cipher that can be used as a drop-in replacement for DES if the block size is set to 64-bit. RC6 is a 128-bit to 256-bit block cipher that is based on RC5 and was designed to meet the requirement of AES.\\

\subsection{Asymmetric algorithms}

Asymmetric algorithms use different keys to encrypt and decrypt data. One key is called the private key, and the other is the public key. The private key is secret and known only to the user. The public key is openly shared and easily distributed. Secure messages can be exchanged without having to have a pre-shared key. Because neither party has a shared secret, very long key lengths must be used. These algorithms are resource intensive and slower to execute. Some well-known asymmetric algorithms: DH, RSA, DSS,	DSA. Asymmetric algorithms are slow, so they are commonly used in low-volume transactions such as making online purchases or logging into a financial website.\\

\textbf{Diffie-Hellman (DH)} Algorithm allows two computers to generate an identical shared secret on both systems, without having communicated before.  To start a DH exchange, both hosts must agree on two nonsecret numbers. The first number is a base number, also called the generator. The second number is a prime number that is used as the modulus. DH is commonly used in IKE (fundamental component of IPsec VPNs), SSL, TLS, and SSH. It is common to use DH algorithm to create and exchange pre-shared keys for symmetric algorithm (3DES or AES).\\

\textbf{Digital signatures} are commonly used in Code signing and Digital certificate. Code signing verifies the integrity of executable files downloaded from a vendor website. Digital certificates verify the identity of a vendor website and establish an encrypted connection. Digital signatures provide three basic security services: Authenticity, Integrity, and Nonrepudiation of the transaction\footnote{Nonrepudiation uses the unique characteristics of the sender of a message to confirm that the reputed sender is in fact the actual sender.}.\\

The \textbf{Public Key Infrastructure (PKI)} identifies a certificate authority which issues public key (certificates) for asymmetric algorithm. The PKI also identifies the encryption algorithms, levels of security, and distribution policy to users. \textbf{X.509} is a well-known standard that defines basic PKI formats. The X.509 version 3 (X.509v3) standard defines the format of a digital certificate. Another important set of PKI standards are the \textbf{Public-Key Cryptography Standards (PKCS}). PKCS defines the low-level formats for the secure exchange of arbitrary data, such as an encrypted piece of data or a signed piece of data.\\


\documentclass[11pt]{amsbook}

\usepackage{../HBSuerDemir}	% ------------------------

\begin{document}

% ++++++++++++++++++++++++++++++++++++++
\hPage{b2p2/312}
% ++++++++++++++++++++++++++++++++++++++
\begin{align*}
4A \frac{\hDif A}{\hDif t} = x(y^2+z^2) \frac{\hDif x}{\hDif t} + y(z^2+x^2) \frac{\hDif y}{\hDif t} + z(x^2+y^2)  \frac{\hDif z}{\hDif t}
\end{align*}

When P is at $P_{O}$(6, 0, 0), then Q is at $Q_{O}$(0, 9, 0) and R is at $R_{O}$(0, 0, 12) with $|P_{O}$  $Q_{O}$  $R_{O}|_{2}$ = 9$\sqrt{61}$. Then

\begin{align*}
4.9\sqrt{61}\frac{\hDif A}{\hDif t} = 6(225)2+ 9(680)3 + 12(117)4 \\
9\sqrt{61}\frac{\hDif A}{\hDif t} = 675 + 27.45 + 12.117 \\
\sqrt{61}\frac{\hDif A}{\hDif t} = 75 + 135 + 156 = 366 \\
\frac{\hDif A}{\hDif t} = \frac{766}{\sqrt{61}} = 6\sqrt{61}  unit^2/sec
\end{align*}

\subsection{TAYLOR'S FORMULA AND SERIES}

\begin{thm} If f(x, y) has continuous partial derivatives up to order n+1 in a neighborhood of (a, b)$\epsilon \upsilon_{f}$, then
\begin{align*}
f(x, y) = f(a, b) +  \sum_{k=1}^{n}  \frac{1}{k!} ((x-a) \frac{a}{ax} + (y-b) \frac{a}{ay})^k f(x, y) |_{(a, b)} + R_{n+1}
\end{align*}
where the remainder is given by
\begin{align*}
R_{n+1} = \frac{1}{(n+1)!} ((x-a) \frac{a}{ax} + (y-b) \frac{a}{ax})^{n+1}f(x, y)_{(x*, y*)}
\end{align*}
with (x*, y*) a point on the open segment ($P_{O}$P) joining $P_{O}$(a, b) to P(x, y)
\end{thm}
\begin{proof} Since every point of the line segment [$P_{O}$P] can be represented parametrically as

\begin{align*}
x = a+ht,\qquad   y = b+kt\qquad	  0\leq t \leq 1,
\end{align*}

\includegraphics[width=0.35\textwidth]{images/b2p2-312-fig01}

The end points of the segment correspond to t=0 and t=1 (observe that h, k are direction numbers of the line segment)
Substituting (2) in f(x, y) gives the function

\end{proof}

\end{document}

hw2/non-merged/EMRE ALPAGUT_38161_assignsubmission_file_/hw2 - 2016742000/suerdemir/pages/b2p2-312.tex


\subsubsection{Pade form}
\label{sec:onebodyjastrowpade}

While the spline Jastrow factor is the most flexible and most commonly used form implemented in \qmcpack,
there are times where its flexibility can make it difficult to optimize.  As an example, a spline jastrow
with a very large cutoff may be difficult to optimize for isolated systems like molecules due to the small
number of samples that will be present in the tail of the function.  In such cases, a simpler functional
form may be advantageous.  The second order Pade jastrow factor, given in Eq.\ref{padeeqn} is a good choice
in such cases.
\begin{equation}
\label{padeeqn}
u_{ab}(r) = \frac{a*r+c*r^2}{1+b*r}
\end{equation}
Unlike the spline jastrow factor which includes a cutoff, this form has an infinite range and for every particle
pair (subject to the minimum image convention) it will be applied.  It also is a cuspless jastrow factor,
so it should either be used in combination with a single particle basis set that contains the proper cusp or
with a smooth pseudopotential.

\paragraph{Input Specification}
\begin{table}[h]
\begin{center}
\begin{tabular}{l c c c l }
\hline
\multicolumn{5}{l}{Correlation element} \\
\hline
\bfseries name & \bfseries datatype & \bfseries values & \bfseries defaults & \bfseries description \\
\hline
elementType & text & name & see below & Classical particle target  \\
\hline
\multicolumn{5}{l}{elements}\\ \hline
& Coefficients & & & \\ \hline
\multicolumn{5}{l}{Contents}\\ \hline
& (None)  & & &  \\ \hline
\end{tabular}
%\end{tabular*}
\end{center}
\end{table}

\begin{table}[h]
\begin{center}
\begin{tabular}{l c c c l }
\hline
\multicolumn{5}{l}{parameter element} \\
\hline
\bfseries name & \bfseries datatype & \bfseries values & \bfseries defaults & \bfseries description \\
\hline
id & string & name & (required) & name for variable \\
name & string & A or B or C & (required) & see Eq.\ref{padeeqn}\\
optimize & text & yes or no & yes & if no, values are fixed in optimizations \\
\hline
\multicolumn{5}{l}{elements}\\ \hline
(None) & & & \\ \hline
\multicolumn{5}{l}{Contents}\\ \hline
 (no name) & real & parameter value & (required) & Jastrow coefficients \\ \hline
\end{tabular}
%\end{tabular*}
\end{center}
\end{table}

\paragraph{Example use case}
\label{sec:1bjpadeexamples}

Specify a spin independent function with independent jastrow factors for two different species (Li and H).
The name of the particleset holding the ionic positions is "i".
\begin{lstlisting}[language=xml]
<jastrow name="J1" function="pade2" type="One-Body" print="yes" source="i">
  <correlation elementType="Li">
    <var id="LiA" name="A">  0.34 </var>
    <var id="LiB" name="B"> 12.78 </var>
    <var id="LiC" name="C">  1.62 </var>
  </correlation>
  <correlation elementType="H"">
    <var id="HA" name="A">  0.14 </var>
    <var id="HB" name="B"> 6.88 </var>
    <var id="HC" name="C"> 0.237 </var>
  </correlation>
</jastrow>
\end{lstlisting}


\subsection{Integers}\label{subsec:integers}

\begin{definition}\label{def:set_of_integers}
  The set \( \BbbZ \) of \term{integers} is defined as the Grothendieck \hyperref[thm:monoid_completion_to_abelian_group]{completion} of the commutative monoid \( (\BbbN, +) \).

  Let \( \oplus \), \( \odot \) and \( \leq_N \) be the operations in \( \BbbN \) (see \fullref{thm:def:natural_number_multiplication/properties}). Since either \( x \in \BbbZ \) or \( -x \in \BbbZ \) is isomorphic to a natural number, we extend the operations to \( \BbbZ \) as follows:
  \begin{thmenum}
    \item addition is defined in the completion.
    \item multiplication is defined as follows:
          \begin{equation*}
            x \cdot y \coloneqq \begin{cases}
              x \odot y,    & x \geq 0 \iff y \geq 0      \\
              (-x) \odot y, & x < 0 \text{ and } y \geq 0 \\
              x \odot (-y), & x \geq 0 \text{ and } y < 0
            \end{cases}
          \end{equation*}

    \thmitem{def:set_of_integers/order} the order is inherited
    \item the additional absolute \hyperref[def:absolute_value]{value} operation is defined as
          \begin{equation*}
            \abs{x} \coloneqq \begin{cases}
              x,  & x \geq 0, \\
              -x, & x < 0.
            \end{cases}
          \end{equation*}
  \end{thmenum}
\end{definition}
\begin{proof}
  The proof that multiplication and absolute values are well-defined can be done similarly to the proof of \fullref{thm:monoid_completion_to_abelian_group}.

  Integer multiplication obviously generalizes natural number multiplication.
\end{proof}

\begin{proposition}\label{thm:integers_are_euclidean_domain}
  The \hyperref[def:semiring/integral_domain]{domain} of integers \( \BbbZ \) is \hyperref[def:semiring/euclidean_domain]{Euclidean} with \( \delta(n) \coloneqq \abs{n} \). Furthermore, the remainder and quotient are unique.
\end{proposition}
\begin{proof}
  Let \( a, b \in \BbbZ \) and \( b \neq 0 \). Suppose that \( b > 0 \). Define
  \begin{balign*}
    q & \coloneqq \max \{ q \in \BbbZ \colon bq \leq \abs{a} \} \\
    r & \coloneqq a - bq.
  \end{balign*}

  It remains to show that either \( r = 0 \) or \( \delta(r) < \delta(b) \).

  Note that \( r = 0 \) if and only if \( b \) is a \hyperref[def:commutative_ring_division]{divisor} of \( a \).

  Suppose that \( b \) is not a divisor of \( a \). Note that \( g \geq 0 \) and \( r > 0 \). If \( r \geq b \), this would imply \( r - b \geq 0 \) and
  \begin{equation*}
    a = bq + r = b(q + 1) + (r - b) \geq b(q + 1),
  \end{equation*}
  which would contradict the maximality of \( q \). Thus, \( r < b \).

  It remains to show uniqueness. Suppose that \( a = bq + r = bq' + r' \). Then
  \begin{equation*}
    0 = b(q - q') + (r - r').
  \end{equation*}

  Thus, \( b \mid r - r' \). But \( -b < r - r' < b \), which implies that \( r = r' \). Thus, also implies that \( q = q' \) since \( b \neq 0 \).

  Now suppose that \( b < 0 \). Define
  \begin{balign*}
    q & \coloneqq \min \{ q \in \BbbZ \colon -\abs{a} \leq bq \} \\
    r & \coloneqq a - bq.
  \end{balign*}

  Suppose that \( b \) is not a divisor of \( a \). Note that \( q \geq 0 \) and \( r < 0 \). If \( r \leq b \), this would imply \( r - b \leq 0 \) and
  \begin{equation*}
    a = bq + r = b(q + 1) + (r - b) \leq b(q + 1),
  \end{equation*}
  which would contradict the minimality of \( q \). Thus, \( r > b \) and, since both \( b \) and \( r \) are negative, \( \abs{r} < \abs{b} \).

  To see uniqueness, suppose that \( a = bq + r = bq' + r' \). Thus, \( b \mid r - r' \). But \( -\abs{b} = b < r - r' < -b = \abs{b} \), which implies that \( r = r' \) and \( q = q' \).

  In both cases, we obtained unique integers \( q \) and \( r \) such that \( a = bq + r \) with \( \abs{r} < \abs{b} \).
\end{proof}

\begin{remark}\label{rem:units_in_rings_etymology}
  An integer \( a \) is divisible by \( b \neq 0 \) if there exists a number \( q \) such that
  \begin{equation*}
    a = qb.
  \end{equation*}

  Obviously \( q = (-q)(-1) \), so the following also holds:
  \begin{equation*}
    a = [(-q)(-1)]b = (-q)(-b),
  \end{equation*}
  hence \( a \) is also divisible by \( -b \).

  For any nonzero number \( b = 1 \cdot b \) that divides \( a \), the number \( -b = (-1) \cdot b \) also divides \( a \). Both \( 1 \) and \( -1 \) have unit norm (that is, \( \abs{1} = \abs{-1} = 1 \)), so it is reasonable to call them \enquote{units}. They are the only integers \( e \) with the property that if \( b | a \), then \( eb | a \). This is probably the reason why invertible elements in arbitrary rings are named \enquote{units}. Another reason is that invertible elements divide the multiplicative identity, commonly denoted by \( 1 \).

  Consider fields, in which all nonzero elements are units. It makes no sense to speak of divisibility whatsoever because any real number \( a \) is divisible by any nonzero real number \( b \). Putting \( q \coloneqq \frac a b \) satisfies the divisibility condition. Now if \( e \) is any unit in \( \BbbR \), we have
  \begin{equation*}
    a = qb = q(e^{-1} e) b = (qe^{-1}) (eb),
  \end{equation*}
  hence \( eb \) also divides \( a \).
\end{remark}

\begin{lemma}[Euclid's lemma]\label{thm:euclids_lemma}
  An \hyperref[def:set_of_integers]{integer} is \hyperref[def:prime_ring_ideal]{prime} if and only if it is irreducible.
\end{lemma}
\begin{proof}
  Follows from \fullref{thm:ufd_prime_iff_irreducible}.
\end{proof}

\begin{definition}\label{def:prime_number}
  Despite negative integers being prime \hyperref[thm:euclids_lemma]{elements} of the ring \( \BbbZ \), we only call positive prime integers \term{prime numbers}. That is, a positive integer is prime if it has no divisors except \( 1 \) and itself.

  Non-prime integers are called \term{composite numbers}.
\end{definition}

\begin{definition}\label{def:coprime_numbers}
  Two integers \( n, m \) are called \term{coprime} (see \fullref{def:coprime_ring_ideals}) if \( \gcd(n, m) = 1 \).
\end{definition}

\begin{theorem}[Fundamental theorem of arithmetic]\label{thm:fundamental_theorem_of_arithmetic}
  Every positive integer greater than \( 2 \) can be \hyperref[def:factorization_in_ring]{factored} into a product of \hyperref[def:prime_number]{prime} powers.
\end{theorem}
\begin{proof}
  The ring \( \BbbZ \) is an Euclidean domain by \fullref{thm:integers_are_euclidean_domain}, which is a principal ideal domain by \fullref{thm:euclidean_domain_is_pid}, which is a unique factorization domain by \fullref{thm:pid_is_ufd}.
\end{proof}

\begin{proposition}[Fermat's little theorem]\label{thm:fermats_little_theorem}
  If \( p \) is a prime \hyperref[def:prime_number]{number}, for any integer \( x \) we have
  \begin{equation*}
    x^p \cong x \pmod p.
  \end{equation*}
\end{proposition}