\chapter{Simulation Models and General Approach}\label{chap:approach}
In this chapter the simulation models and their properties will be introduced.
These models will help to describe the physical effects used in the simulation.
The general simulation approach and an overview of ARTS will be presented and discussed in detail.

\section{Receiver and Source Model}\label{sec:receiverAndSource}
\textbf{3D Model}\newline
An acoustic source can be seen as a "speaker" and an acoustic receiver can be seen as a "microphone".
These two have common properties like a location, an orientation which defines their local coordinate system and a field-of-view (FOV) which represents their emission and reception characteristics depending on the outgoing/incoming angle.
A good receiver FOV example is the human ear.
Turning your ear towards someone who is talking to you will improve the speech reception whereas turning away will decrease it.
We will assume that sources and receivers can be modeled by a single point $(x,y,z)$ in the 3D cartesian coordinate system.
This has been illustrated for the 2D case in \figref{fig:SourceReceiver} in a global coordinate system with $x$-axis and $y$-axis.
\input{figures/approach/figSourceReceiver}
Let's define the following conventions for the receiver
\begin{conditions}
    x, y, z         & Location in the global coordinate system \\
    x_r, y_r, z_r   & Local coordinate system with $x_r$-axis, $y_r$-axis and $z_r$-axis \\
    \theta_0        & azimuth angle between global $x$-axis and local $x_r$-axis \\
    \varphi_0       & polar angle between global $z$-axis and local $z_r$-axis
\end{conditions}
and accordingly for the source as well.
Additionally let's define a receiver FOV function
\begin{equation}
    f_{FOV}(\theta_r, \varphi_r) \rightarrow [0, 1]
\end{equation}
for the reception characteristics and a source FOV function
\begin{equation}
    g_{FOV}(\theta_s, \varphi_s) \rightarrow [0, 1]
\end{equation}
for the emission characteristics where the respective angles are relative to the local coordinate system.
In \figref{fig:fov} two simple examples are given and how a receiver characteristic could look like.
Evenly distributed (blue rectangle) for a radial receiver without any angle penalty $f(\theta, \varphi)=1$ or linearly decreasing when moving away from the center (red triangle).
The center of the receiver and source is defined by the respective $x_r$- and $x_s$-axis.
\input{figures/approach/figFov}

\textbf{Receiver and Source Signals}\newline
Even though the geometry of receivers and sources is very similar, the signals are treated differently at the node positions.
A source signal is defined by
\begin{equation}
    s(t) = g_{FOV}(\theta_s, \varphi_s) \cdot x_s(t)
\end{equation}
for $t \in \mathbb{R}_0^+$ and a given direction $(\theta_s, \varphi_s)$ where
\begin{conditions}
    g_{FOV}(\theta_s, \varphi_s)    & Source FOV function \\
    x_s(t)                          & Continuous source signal function, $x_s(t) \rightarrow [-1,1]$ \\
    t                               & time in seconds $[s]$
\end{conditions}

The receiver signal on the other hand can be expressed as
\begin{equation}\label{approach:receiver}
    r[n] = f_{FOV}(\theta_r, \varphi_r) \cdot y(n \cdot T_r)
\end{equation}
for $n \in \mathbb{N}$ and a given direction $(\theta_r, \varphi_r)$ where
\begin{conditions}
    f_{FOV}(\theta_r, \varphi_r) & Receiver FOV function \\
    y_r(t) & Continuous receiver signal from direction $(\theta_r, \varphi_r)$, $y_r(t) \rightarrow \mathbb{R}$ \\
    T_r                          & Receiver Sampling Time, $T_r \in \mathbb{R}^+$ \\
    n                            & Sampling index
\end{conditions}

Note how the receiver signal is modeled as a discrete signal and not as a continuous signal.
This is a hint on how the signal is later going to be used.
A real world microphone does record continuously but the signal is later sampled at a given frequency in most systems.

\textbf{Definition}\newline
Given the introduced properties a source $S_i$ is defined by
\begin{itemize}
    \item a global coordinate $(x, y, z)$
    \item rotation $(\theta_0, \varphi_0)$ relative to global coordinate system
    \item output characteristic $g_{FOV}(\theta_s, \varphi_s)$
    \item a source signal $x_s(t)$
\end{itemize}
for $i \in {1,...,M}$ and a receiver $R_j$
\begin{itemize}
    \item a global coordinate $(x, y, z)$
    \item rotation $(\theta_0, \varphi_0)$ relative to global coordinate system
    \item input characteristic $f_{FOV}(\theta_r, \varphi_r)$
    \item sampling frequency $T_r$
\end{itemize}
for $j \in {1,...,N}$.

\section{Obstacle Model}\label{sec:obstacle}
In the model to be presented an "obstacle" is a surface that might reflect or absorb a sound wave.
Other suitable descriptions are "reflector" or "mirror".
There are stationary obstacles, like walls, windows, doors, tables, TVs and furniture in general and mobile obstacles, like chairs, laptops, bottles and so on.

\textbf{3D Model}\newline
In this approach all types of obstacles are treated the same way and their geometry is defined by (2D-)Polygons with at least three points. If the points form a 2D-polygon they must define a plane $\Phi: ax + by + cz = d$ as well.

\input{figures/approach/figObstacles}

Therefore the geometry of an obstacle $Q_k$ is given by
\begin{itemize}
    \item a set of points $B_k = \{P_1(x,y,z), P_2(x,y,z), ..., P_N(x,y,z)\}$
    \item such that $B_k$ defines a plane $\Phi_k$
\end{itemize}
where $N \geq 3$.

\textbf{Specular Reflection}\newline
When we apply the "geometric acoustic" approach mentioned by Niklas Röber et al in \cite{rober2007ray} sound waves are approximated as particles moving along directional rays.
The reflection of these rays will be modeled the same way as in optics where the reflection angle $\alpha^{'}$ is equal to the incident angle $\alpha$ relative to the surface normal.
\begin{equation}
    \alpha^{'} = \alpha
\end{equation}
Additionally a reflection coefficient $\gamma_k \rightarrow [0,1]$ will be defined, which represents the relation between the outgoing ray intensity level $I_2$ and the incoming ray intensity $I_1$.
\begin{equation}
    \gamma_k = \frac{I_2}{I_1}
\end{equation}
\input{figures/approach/figSpecularReflection}
\textbf{Other effects}\newline
As proposed in \cite{rober2007ray} wave phenomena such as "scattering", "diffraction" and the influence of the wavelength are worth mentioning but are usually discarded at higher frequencies.
Since we are looking into ultrasound ($f > 20\text{kHz}$) where the wavelength is much shorter than the dimensions of most obstacles this assumption will be applied for the scope of this thesis.
However these effects are still important for improving the reflection model and for noise generation and they should be introduced at some point.

\textbf{Definition}\newline
Given the discussed properties an obstacle $Q_k$ is defined by
\begin{itemize}
    \item a polygon of set $B_k$ of $N$ coplanar points lying on a plane $\Phi_k$
    \item a reflection coefficient $\gamma_k \rightarrow [0,1]$
\end{itemize}
where $N \geq 3$.

\section{Environment Model}\label{sec:environment}
In \secref{sec:backgroundBasic} we already defined an environment $E$, it is given by the tuple $(\vartheta, \phi, p)$ and a set $Q$ of $K$ obstacles.
The "simplest" environment therefore has no obstacles ($K=0$) and only relies on the temperature $\vartheta$, the relative humidity $\phi$ and atmospheric pressure $p$.
For the environment model these properties are assumed to be static and uniform across space for smaller indoor environments.
Their values will influence certain physical effects such as the speed of sound and mainly "path loss".

\textbf{Speed of sound}\newline
According to Reuben Thomas \cite{wayverb} speed of sound in air can be approximated by

\begin{equation}
    c_{\text{air}} = (331 + 0.6 \frac{\vartheta}{^{\circ}\textrm{C}})\frac{\text{m}}{\text{s}}
\end{equation}

\textbf{Path Loss}\newline
For spherical sound waves (point sources) the intensity in radial direction can be expressed as
\begin{equation}
    I(r) = \frac{P}{4 \cdot \pi \cdot r^2}\label{approach:intensity}
\end{equation}
where
\begin{conditions}
    P & sound power [W] \\
    r & sphere radius [m]
\end{conditions}
The relation $I(r) \propto 1/r^2$ is also referred to as the \textit{inverse-square law}.
Therefore we will approximate the damping of a source signal $s(t)$ by the following loss function
\begin{equation}
    \mathcal{L}(d) = \frac{1}{d+1}\label{approach:loss}
\end{equation}
where
\begin{conditions}
    d & distance travelled [m]
\end{conditions}
Friction and relaxation processes are another source for "path loss" and are described by Russel in \cite{drussell}.
Using Russel's absorption calculator and the values $\vartheta=20^{\circ}\textrm{C}, \phi=0\%, p=101\text{kPa}$ the result is $[6.53 \text{dB}/100\text{m}]$ for 20kHz and $[57.70 \text{dB}/100\text{m}]$ for 60kHz.
Russel shows that "air" can be described as a low-pass filter and that depending on the ultrasonic wave frequency the impact might be huge.
The current model however will for now only rely on the \eqref{approach:loss} as the loss function.

\newpage
\section{Approach Overview}
The general approach is to follow an architecture which is module based.
This will allow us to set clear boundaries for each type of problems while also allowing for optimizations to be made to each individual module independently.
For each module the input is clearly defined and usually results in a single output type which can then be fed to another module.
In the following sections each of these modules will be presented.
An overview is given in \figref{fig:overview}.

\newpage
\input{figures/approach/figOverview.tex}

\section{Model Parsing}
The Model Parser is the first module of the pursued approach.
It is important that all the properties of a setup $A$ can be loaded in the simulation.
We are dealing mainly with 3D data combined with some physical properties, see \secref{sec:receiverAndSource}, \secref{sec:obstacle} and \secref{sec:environment}.
Therefore a suitable description for these models should be chosen.
The generated 3D data structures and environmental properties will then be used in the following modules.

\input{figures/approach/figModelParser.tex}

\section{Ray Tracing}
The 3D data structures should represent the sources, receivers and obstacles in a way that simple geometrical operations can be performed, such as "point mirroring", "intersection of rays and obstacles", "distance calculation between points" and so on.
The Ray Tracer should only deal with the geometrical problem of finding paths between sources and receivers.
\input{figures/approach/figRayTracer.tex}
The problem to be solved can be stated relatively simple.
For each pair $(S_i, R_j)$, one source $S_i$ and one receiver $R_j$, all possible geometric paths need to be found in an Environment $E$ with $K$ obstacles.

Reflections will be categorized by an Order $L \in \mathbb{N}$.
They will be referenced as "reflections of order $L$".
The order $L$ tells us how often the ray bounces around until it reaches the receiver $R_j$.
As an example, the special case for $L=0$ is called "direct path" and does not bounce at all.

A Segment $A$ is defined by two locations $A_1(x,y,z)$ and $A_2(x,y,z)$ and a Path $P_{ij}$ is defined by concatenating a sequence of segments, where the starting point is a source $S_i$ and the end point is a source $R_j$. A path can therefore be expressed as

\begin{equation}
    P_{ij} = \{ S_i \rightarrow A_1 \rightarrow A_2 \rightarrow ... \rightarrow A_L \rightarrow R_j\}
\end{equation}

Let's start with simple examples to figure out the conditions that a source $S_i$ can actually reach the receiver $R_j$.

\textbf{First Condition: No collisions}\newline
A path can only exist if every segment contained does not collide with any obstacle $Q_i$.
Take a look at \figref{fig:collision} for the direct path scenario.
\input{figures/approach/figCollision.tex}
This condition is true for reflections as well, which consist of $L+1$ Segments.
If the segment starts or ends at a reflection point, the according obstacle should be excluded from this condition.

\textbf{Second Condition: Reflection point exists}\newline
In this example the first order reflection will be considered.
To ensure that a reflection actually exists two operations need to be performed:
\begin{itemize}
    \item Mirror the receiver point $R_1$ at plane $\Phi_1$ to get the mirroring point $H_1$
    \item Check if segment $(S_1, H_1)$ intersects with the obstacle $Q_1$ in $I_1$
\end{itemize}
\input{figures/approach/figReflection.tex}
Same as before, this check must hold for higher order reflections as well.

\textbf{Higher order reflections}\newline
For higher order reflections another notation is necessary to distinguish all possible paths and its members.
The previous notation might be enough for first order reflections but a more general description is needed for higher order reflections.
Let $T_l$ be a bounce tuple of length $l \in \{1,...,L\}$ describing a possible obstacle "bounce path".

\begin{itemize}
    \item For first order reflections $L = 1, T_1 = (k_1)$
    \item For second order reflections $L = 2, T_2 = (k_1, k_2)$
    \item For third order reflections $L = 3, T_3 = (k_1, k_2, k_3)$
\end{itemize}
where $k_p \in \{1,...,K\}$ such that $k_{p} \neq k_{p-1}$ for $L > 1$.

A bounce tuple only describes which obstacles $Q_k$ are being visited whereas a regular path describes the sequence of exact locations.
These exact locations for a given tuple $T_l$ are called reflection/intersection point $I_p^{T_l}$ for $p \in \{1,...,L\}$.
In order to get and construct these points the mirrored points $H_p^{T_l}$ for $p \in \{1,...,L\}$ of a receiver $R_j$ are needed.
The existence of these reflection points $I_p^{T_l}$ needs to be confirmed first as we know from the second condition.
An illustration of the notation is given by the example in \figref{fig:higherOrder}.
\input{figures/approach/figHigherOrder.tex}

To summarize the different types of paths:
\begin{itemize}
    \item Direct Path $P_{ij} = \{ S_i \rightarrow R_j \}$
    \item First order path $P_{ij}^{T_1} = \{ S_i \rightarrow I_1^{T_1} \rightarrow R_j \}$
    \item Second order path $P_{ij}^{T_2} = \{ S_i \rightarrow I_1^{T_2} \rightarrow I_2^{T_2}\rightarrow R_j \}$
    \item Third order path $P_{ij}^{T_3} = \{ S_i \rightarrow I_1^{T_3} \rightarrow I_2^{T_3} \rightarrow I_3^{T_3} \rightarrow R_j \}$
\end{itemize}

\textbf{Graphs}\newline
Since the goal is to get all the existing paths for a pair $(S_i, R_j)$ a graph $G_{ij}$ can be described as $G_{ij} = \{P_{ij}\} \cup \{P_{ij}^{T_1}\} \cup \{P_{ij}^{T_2}\} \cup \{P_{ij}^{T_3}\}$ for all existing bounce paths $T_1$, $T_2$, and $T_3$.
An illustration for a "full graph" $G_{ij}$ is given in \figref{fig:graph} for $K=2$ and $L=3$.
\input{figures/approach/figGraph.tex}

\textbf{Strategy Summary}\newline
For each possible path we need to perform the following actions:
\begin{enumerate}
    \item Mirroring $R_j$ backwards to get mirroring points $H_p^{T_l}$ for $p \in \{1,...,L\}$
    \item Travelling forward to get intersection points $I_p^{T_l}$
    \begin{itemize}
        \item Check if reflection point $I_p^{T_l}$ exists
        \item Check if new segment collides with other obstacles
        \item Add segments to path until $R_j$ is reached
    \end{itemize}
\end{enumerate}

\textbf{Run-time Complexity}\newline
Given that the current strategy is performed on all possible paths the following complexity $C_l$ for a specific order $l > 0, l \in \{1,...,L\}$ can be expected:
\begin{equation}
    C_l = M \cdot N \cdot K \cdot (K-1)^{l-1}
\end{equation}
where
\begin{conditions}
    M & The number of sources $S_i$ \\
    N & The number of receivers $R_j$ \\
    K & The number of obstacles $Q_k$
\end{conditions}

The total number of paths and the total complexity $C_L$ for all full graphs $G_{ij}$ can be expressed as
\begin{equation}\label{eq:complex}
    C_L = M \cdot N + \sum_{l=1}^{L}{C_l} =  M \cdot N \cdot ( 1 + K \cdot \sum_{l=1}^{L}{(K-1)^{l-1}})
\end{equation}

This sets an upper bound for the complexity of the current strategy.
A lot of paths will probably fail to satisfy one of the two checks and therefore terminate earlier.
The first step however is performed for every possible path, therefore run-times of order $C_L$ are expected.

\section{Path Sampling}
The "Ray Tracer" provides a set of paths in form of a Graph $G_{ij}$.
Each path might contribute to the final received signal depending on the properties of a path $P$ in $G_{ij}$ and the environmental properties introduced in the previous chapters.
The "Path Sampler" takes all these modeled effects and creates a set of samples for each $(S_i, R_j)$ pair.

\input{figures/approach/figPathSampler.tex}
In \eqref{approach:receiver} a first sneak peek was given on how an incoming signal $r[n]$ will be sampled.
A lot of different models have been introduced since then and they will now be condensed into a single equation for the total received signal $r_{ij}[n]$:
\begin{equation}\label{eq:receiver}
    r_{ij}[n] = \sum_{P \in G_{ij}} f_{FOV}(\theta_{r,P}, \varphi_{r,P}) \cdot \mathcal{L}(d_P) \cdot \prod_{\gamma_k \in P}{\gamma_k} \cdot g_{FOV}(\theta_{s,P}, \varphi_{s,P}) \cdot x_s(n \cdot T_r - \frac{d_P}{c_{\text{air}}})
\end{equation}
where
\begin{conditions}
    P & Path in Graph $G_{ij}$ \\
    d_P & Total Path length of $P$ in [m] \\
    f_{FOV}(\theta_{r,P}, \varphi_{r,P}) & Receiver $R_j$ FOV factor for incoming direction of path $P$ \\
    \mathcal{L}(d_P) & Distance $d_P$ loss factor \\
    \prod_{\gamma_k \in P}{\gamma_k} & Combined Reflection factor for all obstacles $Q_k$ in $P$ \\
    g_{FOV}(\theta_{s,P}, \varphi_{s,P}) & Source $S_i$ FOV factor for outgoing direction of path $P$ \\
    x_s(t) & Source signal of $S_i$ \\
    n  & Sampling index \\
    T_r & Receiver $R_j$ sampling time in [s] \\
    c_{\text{air}} & Speed of sound in [m/s]
\end{conditions}

\section{Estimation and Evaluation}
An estimator represents a system that is trying to solve a certain problem.
Indoor localization estimators will most likely try to estimate locations $(x,y,z)$ in the room like in \cite{bordoy2019exploiting}, angle of arrivals (AOA) $(\theta, \varphi)$, time difference of arrival (TDOA) $(t_1, t_2,...)$ and so on.

Indoor mapping applications will try to estimate the room geometry.
This can be expressed as a tuple of normal vectors $(\vec{n_1},\vec{n_2},...)$ for distances to walls and obstacles similar to \cite{ribeiro2011geometrically} or in the best case 3D data structures of the environment.

Another example could be that the simulator should be improved.
Therefore the estimation process can be skipped completely, acting as a pass-through.
This could be very useful for optimizing the simulator using real world recordings.

Since the nature, the algorithms and the approaches of these systems vary a lot, this part has been left open to be generalized. In \chapref{chap:experiments} an example will be shown for a LOS localization estimator.

\input{figures/approach/figEstimator.tex}

The evaluation of a system is highly coupled to the estimator.
Therefore it is up to the estimator to provide a suitable evaluation.
A well established error measure for vectors in general is the euclidean distance of two vectors
\begin{equation}
    \lVert \mathbf{\widetilde{x} - \mathbf{x}} \rVert_2 = \sqrt{(\tilde{x_1}-x_1)^2 + (\tilde{x_2}-x_2)^2 + ... + (\tilde{x_n}-x_n)^2}
\end{equation}
where
\begin{conditions}
    \mathbf{\widetilde{x}} & Estimation vector with length $n$ \\
    \mathbf{x} & True value vector with length $n$ \\
\end{conditions}
In the Evaluator Model the estimated values would be represented by $\mathbf{\widetilde{x}}$ and the real properties of the 3D data vectors by $\mathbf{x}$.
This approach will be used for the LOS localization estimator example in \chapref{chap:experiments}.
For non-vector types other approaches should be defined.

\input{figures/approach/figEvaluator.tex}


\documentclass{article}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{algorithm}
\usepackage{graphicx}
\usepackage[noend]{algpseudocode}
\usepackage{url}
\usepackage{listings}

\newlength\tindent
\setlength{\tindent}{\parindent}
\setlength{\parindent}{0pt}
\renewcommand{\indent}{\hspace*{\tindent}}

\newtheorem{thm}{Theorem}
\newtheorem{cor}{Corollary}[thm]
\newtheorem{lemma}{Lemma}[thm]

\title{COMP 312 Assignment 2}
\author{Daniel Braithwaite}

\begin{document}
	\pagenumbering{gobble}
	\maketitle
	\newpage
  	\pagenumbering{arabic}

  	\section{Python}
  		\subsection{Dice Problem}
  			\subsubsection{Part A}
  				\lstinputlisting[language=Python]{Programs/01.py}

  				The output of this program is $0.491206$ so yes the probability of the event occurring is less than $\frac{1}{2}$. The value of n was very large so we can be confident in this finding.

  			\subsubsection{Part B}
  				\lstinputlisting[language=Python]{Programs/02.py}

  				The modified code calculates the probability of the event occurring with a different number of throws. Over the range 24 to 26 it outputs the the following $\{24: 0.491443, 25: 0.506207, 26: 0.518883\}$

  				From this we see that it requires 25 throws for the probability to be $ > \frac{1}{2}$ but as close to $\frac{1}{2}$ as possible

  		\subsection{Optimized Table Look-up}
  			\lstinputlisting[language=Python]{Programs/03.py}

			The output from running this code is
			\newline

			\begin{center}
				\begin{tabular}{ c c c}
				Part I & Part II & Part III\\
				3.750354 & 3.750114 & 3.750704\\
				4.750354 & 2.249886 & 2.064436\\

				\end{tabular}
			\end{center}

			As we can see from this all results are mostly the same. We also see that option 2 and 3 take about half as many steps as option 1. However option 3 is just slightly faster

	\section{Queues}
		\subsection{Problem 3}
			We know that the service times are distributed exponentially. And we assume that all the servers have equal average service times, let this time be $m$. Once one of the other services has finished you can begin being served (you are now 'racing' against the remaining 6 people). As the exponential distribution has the property of being memory-less all the services have the same probability of taking time $m$. This means that $\frac{6}{7}$ probability of finishing before atleast one of the other customers being served.

		\subsection{Problem 4}
			$state \gets $ number of active pizza shops\newline

			For this problem I used python to compute the steady state vector. The following code is what I used to do this.
			\lstinputlisting[language=Python]{Programs/pizza.py}

			To get the average state of the system we take the following sum

			\begin{align*}
				\sum_{n = 0}^{32} n\pi_n
			\end{align*}

			for which we get the following result

  			\begin{align*}
  				Avg\ number\ of\ pizza\ shops:\ 27.6082698882
  			\end{align*}

			To get the fraction of time spent in state $i$ we know this value is $\pi_i$ so if we want to find the fraction of time with 20 or more restaurants this is

			\begin{align}
				\sum_{n = 20}^{32} \pi_n = 0.999741852504
			\end{align}

  			\begin{align}
  				Fraction\ of\ time\ with\ >\ 20\ restrants:\ 0.999741852504
  			\end{align}
\end{document}


% BLAST Extension
%
% Basic description of the extension stage of the BLAST algorithm

\subsection{BLAST Extension}
\begin{frame}
  \frametitle{Extension}
  \begin{itemize}
    \item The best-scoring seeds are extended in each direction
    \item BLAST does not explore the complete search space, so a rule (heuristic) to stop extension is needed
    \item Two-stage process:
    \begin{itemize}
      \item Extend, keeping alignment score, and \emph{drop-off} score
      \item When drop-of score reaches a threshold $X$, trim alignment back to top score
    \end{itemize}
  \end{itemize}
  \begin{center}
    \includegraphics[width=0.5\textwidth]{images/extension}
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Example}
  \begin{itemize}
    \item<1-> Consider two sentences (match=+1, mismatch=-1)
    \begin{itemize}
      \item \texttt{The quick brown fox jumps over the lazy dog.}
      \item \texttt{The quiet brown cat purrs when she sees him.}
    \end{itemize}
    \item<2-> Extend to the right from the seed \texttt{T}
    \begin{itemize}
      \item \texttt{The quic}
      \item \texttt{The quie}
      \item \texttt{123 4565 <- score}
      \item \texttt{000 0001 <- drop-off score}
    \end{itemize}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Example}
  \begin{itemize}
    \item Consider two sentences (match=+1, mismatch=-1)
    \begin{itemize}
      \item \texttt{The quick brown fox jumps over the lazy dog.}
      \item \texttt{The quiet brown cat purrs when she sees him.}
    \end{itemize}
    \item Extend to drop-off threshold
    \begin{itemize}
      \item \texttt{The quick brown fox jump}
      \item \texttt{The quiet brown cat purr}
      \item \texttt{123 45654 56789 876 5654 <- score}
      \item \texttt{000 00012 10000 123 4345 <- drop-off score}
    \end{itemize}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Example}
  \begin{itemize}
    \item Consider two sentences (match=+1, mismatch=-1)
    \begin{itemize}
      \item \texttt{The quick brown fox jumps over the lazy dog.}
      \item \texttt{The quiet brown cat purrs when she sees him.}
    \end{itemize}
    \item Trim back from drop-off threshold to get optimal alignment
    \begin{itemize}
      \item \texttt{The quick brown}
      \item \texttt{The quiet brown}
      \item \texttt{123 45654 56789 <- score}
      \item \texttt{000 00012 10000 <- drop-off score}
    \end{itemize}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Notes on implementation}
  \begin{itemize}
%    \item This example represents ungapped BLAST; gapped BLAST is more similar to dynamic programming
    \item $X$ controls termination of alignment extension, but dependent on:
    \begin{itemize}
      \item substitution matrix
      \item gap opening and extension parameters
    \end{itemize}
  \end{itemize}
  \begin{center}
    \includegraphics[width=0.5\textwidth]{images/extension}
  \end{center}
\end{frame}


\chapter{Background}
\label{chap:bg}
In this chapter, we first introduce the generative probability topic.
They are the basic of our framework.

\section{Convolutional Neural Network }

Convolutional Neural Network (CNN) is a feed-forward neural network. Its artificial neurons can respond to a part of the surrounding units in the coverage area and have excellent performance for large-scale image processing.

The convolutional neural network consists of one or more convolutional layers and a fully connected layer at the top, and also includes associated weights and a pooling layer. This structure enables convolutional neural networks to use the two-dimensional structure of the input data. Compared with other deep learning structures, convolutional neural networks can give better results in image and speech recognition. This model can also be trained using backpropagation algorithms. Compared with other deep, feed-forward neural networks, convolutional neural networks need to consider fewer parameters, making it an attractive deep learning structure.

\subsection{Convolutional Layers}
Each convolutional layer in a convolutional neural network consists of several convolutional units, and the parameters of each convolutional unit are optimized through the backpropagation algorithm. The purpose of the convolution operation is to extract different features of the input. The first layer of convolutional layer may only extract some low-level features such as edges, lines, and corners. More layers of networks can iteratively extract more complex features from low-level features. For example, regardless of the size of the input image, since each region has the same shared weight, using a 3x3 convolution kernel requires only 9 learnable parameters. Using regular weights on fewer parameters can avoid the vanishing gradient and explosive gradient problems that may occur during backpropagation.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.6\linewidth]{figures/conv}
	\caption[An example of two-dimensional convolution]{A simple two dimensional convolution with a 3x3 convolutional kernel and stride = 1, the output size is 3x3 which is decided by image size, kernel size and stride.}
	\label{fig:conv}
\end{figure}

\subsection{Pooling Layers}

Pooling is another important concept in convolutional neural networks, which is actually a form of downsampling. There are many different forms of non-linear pooling functions, of which max pooling (An example see in Fig.~\ref{fig:maxpool}) is the most common. It divides the input image into several rectangular areas, and outputs the maximum value for each sub-area. Intuitively, the reason this mechanism can be effective is that after a feature is discovered, its precise position is far less important than its relative position with other features. The pooling layer will continuously reduce the size of the data space, so the number of parameters and the amount of calculation will also decrease, which also controls over-fitting to a certain extent. Usually, the pooling layer is periodically inserted between the convolutional layers of CNN.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.6\linewidth]{figures/maxpool}
	\caption[An example of max pooling]{An example of max pooling with a 2x2 kernel and the stride is 2.}
	\label{fig:maxpool}
\end{figure}

\subsection{Fully Connection Layers}

In the CNN structure, after multiple convolutional layers and pooling layers, one or more fully connection layers are connected. Similar to Multilayer Perceptron (MLP), each neuron in the fully connection layer is fully connected to all neurons in the previous layer. The fully connection layer can integrate the local information with class distinction in the convolutional layer or the pooling layer. In order to improve the performance of the CNN network, the activation function of each neuron in the fully connected layer generally adopts the ReLU function. The output value of the last fully connected layer is passed to an output, which can be classified by softmax regression. This layer can also be called a softmax layer. For a specific classification task, it is very important to choose a suitable loss function. CNN has several commonly used loss functions, each with different characteristics. Generally, the fully connected layer of CNN has the same structure as MLP, and the training algorithm of CNN mostly uses BP algorithm.
\begin{figure}[H]
	\centering
	\includegraphics[width=0.4\linewidth]{figures/fc}
	\caption[The structure of fully connection layer]{The structure of fully connection layer.}
	\label{fig:fclayer}
\end{figure}

\section{Transformer}
\label{section:transformer}
The Transformer is a model proposed by Vaswani, Ashish et al. in 2017~\cite{vaswani2017attention}  that uses the attention mechanism to increase the speed of model training. Compared with recurrent networks, such as Long short-term memory (LSTM), the Transformer can model long dependencies between input sequence elements and support parallel processing of sequence, and get higher both accuracy and performance than the popular RNN recurrent neural network. It mainly consists of two parts: the Encoder and Decoder. The structure of the Transformer is shown below as Fig.~\ref{fig:transformer}.

The transformer model  was first applied to a wide range of language tasks such as text classification, machine translation, and question answering, and demonstrated exemplary performance. The breakthrough from Transformer network in the field of natural language processing (NLP) has aroused great interest in the computer vision community, meanwhile the Transformer model and its variants have been successfully applied to the various fields of computer vision, such as image recognition, object detection, segmentation, image super-resolution, video understanding, image generation and so on.

\begin{figure}[!htbp]
	\centering
	\includegraphics[width = 0.5 \textwidth]{figures/transformer.png}
	\caption[The Transformer - model architecture]
	{ The Transformer - model architecture.~\cite{vaswani2017attention}.}
	\label{fig:transformer}
\end{figure}

\subsection{Self-Attention}
An attention function can be described as mapping a query and a set of key-value pairs to an output, where the query, keys, values, and output are all vectors. The output is computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility function of the query with the corresponding key.

From the following Table~\ref{tab:complexity_compare}, we can know that self attention has a great improvement for longer sequences. For image data, we can think of it as a longer sequence, and it would be better to optimize it.
\begin{table}[!htbp]
	\resizebox{\textwidth}{15mm}{
	\begin{tabular}{cccc}
		\hline
		Layer Type                  & Complexity per Layer & Sequential Operations & Maximum Path Length \\ \hline
		Self-Attention              & $ O(n2\cdot d)  $           &$  O(1)  $                 & $ O(1)  $               \\
		Recurrent                   & $ O(n2\cdot d) $            & $ O(n)  $                 & $ O(n)  $               \\
		Convolutional               & $O(k\cdot n\cdot d^2)$       & $O(1) $              & $O(log_k(n)$)         \\
		Self-Attention (restricted) & $ O(r\cdot  n \cdot d) $         & $ O(1)  $                 & $ O(n/r)  $             \\ \hline
	\end{tabular}}

\caption[Compare between Self-Attention and Convolutional ]
{ Maximum path lengths, per layer complexity and minimum number of sequential operations for different layer types. n is the sequence length, d is the representation dimension, k is the kernel size of convolutions and r the size of the neighborhood in restricted self-attention. Obtained from ~\cite{vaswani2017attention}.}
	\label{tab:complexity_compare}
\end{table}

There are three very important vectors in attention, query, key, and value. For self-attention, these three vectors are obtained by multiplying the same vector by different matrices (see Fig.~\ref{fig:attention}).An attention function can be described as mapping a query and a set of key-value pairs to an output. The input consists of queries and keys of dimension $ d  $, and values of dimension $ d $, the softmax function to obtain the weights on the values.

\begin{equation}
	Attention(Q,K,V)=softmax(\frac{QK^T}{\sqrt{d_K}})V
	\label{equ:self_attention}
\end{equation}

\begin{figure}[!htbp]
	\centering
	\includegraphics[width = 0.8\textwidth]{figures/attention.png}
	\caption[Scaled Dot-Product Attention and Multi-Head Attention]
	{ (left) Scaled Dot-Product Attention. (right) Multi-Head Attention consists of several attention layers running in parallel. (Image source:~\cite{vaswani2017attention}).}
	\label{fig:attention}
\end{figure}

\subsection{Multihead Attention}

Multi-head attention allows the model to attend to different representation subspaces at different positions and therefore process richer information. The queries, keys and values are linearly projected h times with different, learned linear projections to $ dk $, $ dk $ and $ dv $ dimensions respectively. On each of these projections the attention function is computed in parallel, resulting in dv dimensional output values. These are concatenated and projected to obtain the final values, as depicted in Figure 2.3.

\begin{equation}
	\begin{aligned}
MultiHead(Q,K,V) = Concat(head_1,...,head_h)W^O, \\
where \ head_i = Attention(QW_i^Q,KW_i^K,VW_i^V)
\end{aligned}
\end{equation}

\subsection{Positional Encoding}
In the case of RNNs, we feed the words sequentially to the model, each token is aware of how it was ordered. However, multi-head attention computes the output of each item in the sequence independently with no notion of word order. It is inefficient to model the sequence information without any special order or position. To account for the order of the words in the input sequence, the Transformer model adds a vector to each input embedding called Positional Encoding. Positional Encoding from the Transformer model is computed by sine and cosine functions of different frequencies as:
\begin{equation}
	\begin{aligned}
	PE_{(pos,2i)} = sin(pos/10000^{2i/d_{model}}),\\
	PE_{(pos,2i+1)} = cos(pos/10000^{2i/d_{model}})
	\end{aligned}
	\label{equ:position_embedding}
\end{equation}

\subsection{Encoder-Decoder Architecture}
The Transformer model with its encoder and decoder components is illustrated in Figure 11. Both Encoder and Decoder are composed of multiple identical encoders and decoders that can be stacked on top of each other Nx times. The encoder stack and the decoder stack share the same number of Nx.

\textbf{Encoder}: The encoder block is a stack of Nx identical layers. Each layer has a multi-head self-attention mechanism sub-layer followed by a position-wise fully connected feed-forward network sub-layer. There are two on each floor sub-layer, the first is a multi-head self-attention mechanism, and the second is a simple feed-forward network with fully connected locations. The residual structure ~\cite{He_2016_CVPR} (see figure ~\ref{fig:transformer})is used on both sub-layers, and finally layer normalization ~\cite{ba2016layer}. That is, the output of each sub-layer is $LayerNorm(x + Sublayer(x))$, where $Sublayer(x)$ is the function implemented by the sub-layer itself. Since the model does not contain any recurrence and convolution, an embedding must be added to the input so that the model can be used in order of the sequence.

The input received by an encoder is a list of vectors, it will input the list of vectors to the self-attention layer, then pass through the feed-forward neural network layer(FFN), and finally get the output, which is passed to the next encoder. The word at each position passes through the self-attention layer, and each output vector obtained passes through the FFN separately, and the feed-forward neural network through each vector is the same.

\textbf{Decoder}: The decoder block is also a stack of Nx identical layers. In addition to the two sub-layers in each encoder layer, the decoder has an extra Masked Multi-Head Attention sub-layer to avoid this attention sub-layer looking into the future.

\section{Faster R-CNN}

Faster R-CNN~\cite{ren2016faster} is an advanced network for object detection and the cornerstone for the visual relationship detection models. Compared with other object detection models, Faster R-CNN performs well on localizing and classifying the objects in images. Therefore, it is used as the backbone of object detection in many works about visual relationship detection~\cite{zellers2018neural}. Faster R-CNN contains four parts: convolutional layers, region proposal network (RPN), ROI pooling and classifier. In visual relationship detection, the first three parts are utilized to extract the feature map and localize the bounding boxes of objects. The classifier of Faster R-CNN is generally replaced by the advanced model which is designed for visual relationship detection. The structure of Faster R-CNN is illustrated in Fig.~\ref{fig:fasterrcnn}.

\begin{figure}[!htbp]
	\centering
	\includegraphics[width = 0.7\textwidth]{figures/fasterrcnn.png}
	\caption[The architecture of Faster R-CNN]
	{ The architecture of Faster R-CNN.}
	\label{fig:fasterrcnn}
\end{figure}

The following are the key steps of Faster RCNN:
\begin{itemize}
	\item \textbf{Conv layers.} As a CNN network target detection method, Faster R-CNN first uses a set of basic conv, relu and pooling layers to extract image feature maps. The feature maps are shared for the subsequent RPN layer and fully connected layer.
	\item \textbf{Region Proposal Networks.} The RPN network is used to generate region proposals. This layer judges that anchors belong to foreground or background through softmax, and then uses bounding box regression to correct anchors to obtain accurate proposals.
	\item \textbf{Roi Pooling. }This layer collects the input feature maps and proposals, extracts the proposal feature maps after synthesizing the information, and sends them to the subsequent fully connected layer to determine the target category.
	\item \textbf{Classification}. Use proposal feature maps to calculate the category of the proposal, and again bounding box regression to obtain the final precise position of the bounding box.
\end{itemize}

\subsection{VGG16}

VGG16 is a convolutional neural network model proposed by K. Simonyan and A. Zisserman from the University of Oxford in the paper ``Very Deep Convolutional Networks for Large-Scale Image Recognition''~\cite{simonyan2015deep}. The model achieves 92.7\% top-5 test accuracy in ImageNet~\cite{ILSVRC15}, which is a dataset of over 14 million images belonging to 1000 classes.

The structure of VGG16 is shown in Fig.~\ref{fig:vgg16}.The input to cov1 layer is of fixed size 224 x 224 RGB image. The image is passed through a stack of convolutional (conv.) layers, where the filters were used with a very small receptive field: 3x3 (which is the smallest size to capture the notion of left/right, up/down, center). In one of the configurations, it also utilizes 1x1 convolution filters, which can be seen as a linear transformation of the input channels (followed by non-linearity). The convolution stride is fixed to 1 pixel; the spatial padding of conv. layer input is such that the spatial resolution is preserved after convolution, i.e. the padding is 1-pixel for 3x3 conv. layers. Spatial pooling is carried out by five max-pooling layers, which follow some of the conv.  layers (not all the conv. layers are followed by max-pooling). Max-pooling is performed over a 2x2 pixel window, with stride 2.

Three Fully-Connected (FC) layers follow a stack of convolutional layers (which has a different depth in different architectures): the first two have 4096 channels each, the third performs 1000-way ILSVRC classification and thus contains 1000 channels (one for each class). The final layer is the soft-max layer. The configuration of the fully connected layers is the same in all networks.

All hidden layers are equipped with the rectification (ReLU) non-linearity. It is also noted that none of the networks (except for one) contain Local Response Normalisation (LRN), such normalization does not improve the performance on the ILSVRC dataset, but leads to increased memory consumption and computation time.

\begin{figure}[!htbp]
	\centering
	\includegraphics[width=0.8\linewidth]{figures/vgg16}
	\caption[The Architecture of VGG-16]{The Architecture of VGG-16.}
	\label{fig:vgg16}
\end{figure}

The advantages of VGG:
\begin{itemize}
	\item The structure of VGGNet is very simple, the entire network uses the same size of the convolution kernel size (3x3) and maximum pooling size (2x2).
	\item The combination of several small filter (3x3) convolutional layers is better than a large filter (5x5 or 7x7) convolutional layer.
	\item It is verified that performance can be improved by continuously deepening the network structure.
\end{itemize}

The disadvantages of VGG:

\begin{itemize}
	\item VGG consumes more computing resources and uses more parameters, resulting in more memory usage. Most of the parameters are from the first fully connected layer. VGG has 3 fully connected layers.
\end{itemize}

\label{sec:roialign}
\subsection{ROI Align}
ROI Align is a regional feature aggregation method proposed in the Mask-RCNN~\cite{he2018mask} paper, which solves the problem of regional mis-alignment caused by two quantizations in the ROI Pooling operation. Experiments show that replacing ROI Pooling with ROI Align in the detection task can improve the accuracy of the detection model.

\begin{figure}[!htbp]
	\centering
	\includegraphics[width=1\linewidth]{figures/roi_align}
	\caption[Bilinear interpolation for ROI Align]{Bilinear interpolation for ROI Align.}
	\label{fig:roialign}
\end{figure}

ROI Align does not need to perform rounding operations like ROI pooling. If the decimal number is calculated, that is, it does not fall on the real pixel, then use the nearest pixel to perform bilinear interpolation on this virtual pixel to get this `pixel'  value.The step are processed as follow:

\begin{enumerate}[1.]
	\item Divide the bbox area into equal parts according to the size required by the output. It is likely that the vertices will not fall on the real pixels after the equal division.
	\item If the vertices do not fall on the real pixel points after equal division, then take a fixed 4 points in each bin, which is the blue point on the right side of Figure~\ref{fig:roialign}.
	\item For each blue point, the value of the 4 nearest real pixel points is weighted (bilinear interpolation) to obtain the value of this blue point.
	\item  Four new values are calculated in a bin, and max is selected from these new values as the output value of this bin Therefore we use bilinear interpolation to predict its value.
	\item Get the final 2x2 output.
\end{enumerate}

\section{End-to-End Object Detection with Transformers}
The DETR discards the traditional Faster R-CNN ROI-based object detection method, and uses Transformer and its proposed bipartite matching to achieve position-independent and unique bounding box for each object, thus eliminating the need for Anchor and NMS. With a very simple architecture, it achieves an accuracy comparable to or even surpassing Faster R-CNN. While being able to do object detection, the model also has better migration capabilities. In the original paper, the panoramic segmentation was achieved through the Transformer's Attention mechanism.

\begin{figure}[!htbp]
	\centering
	\includegraphics[width = 1 \textwidth]{figures/DETR.png}
	\caption[The framework of DETR model]
	{ The framework of DETR model. Figure obtained from ~\cite{carion2020end}.}
	\label{fig:detr}
\end{figure}

The network structure of DETR is very simple (see Fig.~\ref{fig:detr}.). It is divided into three parts: the first part is a traditional CNN to extract image features to higher dimensions. The second part is a Transformer Encoder and Decoder to extract the object features. Finally, the Bipartite matching loss is used to Train the network.

\subsection{Transformer Encoder}

As shown on the left side of the Fig.~\ref{fig:detr}, the DETR first enter the picture $ x_{img} \in  \mathbb{R} ^{3\times H_0\times W_0} $. After processing by CNN backbone, output feature map $ f \in \mathbb{R} ^{C\times H\times W } $, where $ C = 2048, H=\frac{H_0}{32}, W=\frac{W_0}{32}  $. Then add the feature map and position encoding, and input them into the Transformer Encoder for processing, and obtain the image embedding for input to the Transformer Decoder. Since the input of Transformer is serialized data, the feature map output by backbone is further processed and converted into serialized data.
As shown on the left side of the figure, the result of detr's transformer encoder is the same as the traditional transformer structure, except that the input is replaced with image features.

\subsection{Transformer Decoder}

DETR passes the predicted target sequence obtained by the Transformer encoder through the Transformer decoder shown in the right half of Figure~\ref{fig:detrtr}, and decodes it in parallel to obtain the output sequence (rather than outputting it element by element like machine translation). Different from the traditional autogreesive mechanism, each layer can decode N targets. Due to the position invariance of the decoder, that is, the result of switching the input order is unchanged, in addition to the information of each pixel itself, the position information is also very important, so these N targets The input embedding must be different to produce different results, so learn the method in NLP, add positional encoding and add it for each layer.

In addition, a fixed-size learnable object query is designed in DETR, which defines an embedding layer with a size of $ [100, 256] $, and uses its parameters as the object query. While training the network, the object query is also training, so that the most suitable query tensor is obtained and fixed.

\begin{figure}[h]
	\centering
	\includegraphics[width=0.7\linewidth]{figures/detr_tr}
	\caption[Architecture of DETR's transformer]{Architecture of DETR's transformer, figture obtained from ~\cite{carion2020end}.}
	\label{fig:detrtr}
\end{figure}

\subsection{Bipartite matching loss}
\label{sec:matchingloss}
Previously, the detector used the anchor and ground truth IoU to determine the positive and negative samples,  while DETR used bipartite matching loss to determine the positive and negative samples.

\begin{equation}
	\hat{\sigma}=\underset{\sigma \in \mathfrak{G}_{N}}{\arg \min } \sum_{i}^{N} \mathcal{L}_{\text {match }}\left(y_{i}, \hat{y}_{\sigma(i)}\right)
\end{equation}

where $ \mathcal{L}_{\text {match }} $ is the pair-wise matching cost between ground true and predicted bounding boxes. Through one-by-one pairing, there is no need to adopt traditional NMS, because a bounding box can only be matched with a ground true, which will inevitably introduce loss. The algorithm implementation uses the Hungarian algorithm.

DETR obtain the predicted distributions of classes and bounding boxes through two different feed forward networks(FFN). Let us denote by $ y $ the ground truth set of objects, and $  \hat{y}= \{ \hat{y}_i\} ^N_{i=1} $ the set of $ N  $predictions.To find a bipartite matching between these two sets we search for a permutation of $ N $ elements $sigma \in \Theta _N $ with the lowest cost:
\begin{equation}\label{equ:matcher}
	\hat{\sigma} = \underset{\sigma \in \Theta _N }{argmin} \sum_{i}^{N} \mathcal L_{match} (y_i,\hat{y}_{\sigma(i)})
\end{equation}

where $\mathcal L_{match} (y_i,\hat{y}_{\sigma(i)})$ is a pair-wise matching cost between ground truth $ y_i $ and a prediction with index $ \sigma(i) $.

Then we use the \textit{Hungarian loss }in DETR~\cite{carion2020end} to calculate the loss function:
\begin{equation}\label{equ:hungarianloss}
	\begin{aligned}
		&\mathcal L_{Hungarian}(y,\hat{y} ) = \sum^{N}_{i=1}[-\log{\hat{p}_{\hat{\sigma}(i)}(c_i)} + \mathbb{I}_{\{c_i \ne  \varnothing \}}\mathcal{L}_{box}(b_i,\hat{b}_{\hat{\sigma}(i)})   ], \\
		&wehre \\
		&\mathcal{L}_{box}(b_i,\hat{b}_{\hat{\sigma}(i)}) = \lambda_{iou}\mathcal{L}(b_i,\hat{b}_{\sigma(i)}) + \lambda_{L1}\left \| b_i - \hat{b}_{\sigma (i)}  \right \| _1
	\end{aligned}
\end{equation}

where $ \hat{\sigma} $ is the optimal assignment computed in the Equ.~\ref{equ:matcher}. we define probability of class $ c_i $ as $ \hat{p}_{\hat{\sigma}(i)}(c_i)$ and the predicted box as $ \hat{b}_{\sigma(i)} $. we use a linear combination of a negative log-likelihood for class prediction and a linear combination of the $  l_1  $ loss and the generalized IoU~\cite{rezatofighi2019generalized} loss for the box prediction.

\subsubsection{Hungarian matching }

Hungarian matching is binary matching using the Hungarian algorithm, it is also a maximum matching. The purpose of maximum matching is to match an edge for each node as much as possible, and the match with the largest number of edges is the maximum match. Usually, in binary matching, each node can only match one edge.

\begin{figure}
	\centering
	\includegraphics[width=0.3\linewidth]{figures/matcher}
	\caption[Bipartite matching loss]{Bipartite matching loss.}
	\label{fig:matcher}
\end{figure}

The basic steps of the Hungarian algorithm are as follows:
\begin{enumerate}[1.]
	\item Subtract the smallest entry in each row from all the other entries in the row. This will make the smallest entry in the row now equal to 0.
	\item Subtract the smallest entry in each column from all the other entries in the column. This will make the smallest entry in the column now equal to 0.
	\item Draw lines through the row and columns that have the 0 entries such that the fewest lines possible are drawn.
	\item  If there are nn lines drawn, an optimal assignment of zeros is possible and the algorithm is finished. If the number of lines is less than nn, then the optimal number of zeroes is not yet reached, then go to the next step.
	\item Find the smallest entry not covered by any line. Subtract this entry from each row that is not  crossed out, and then add it to each column that is crossed out. Then, go back to $ 3^{rd} $Step.
\end{enumerate}

\subsubsection{IoU and GIoU}

IoU is what we call cross-union ratio, which is the most commonly used index in target detection, in the anchor-based method. The function is not only used to determine the positive sample and the negative sample, but also can be used to evaluate the distance between the predict box and the ground-truth.

$$
I o U=\frac{|A \cap B|}{|A \cup B|}
$$

GIoU was proposed by Hamid et al.~\cite{rezatofighi2019generalized}, since IoU is the concept of ratio, it is not sensitive to the scale of the target object. However, the BBox regression loss (MSE loss, $ l_1 $ smooth loss, etc.) optimization and IoU optimization in the detection task are not completely equivalent, and the Ln norm is also sensitive to the scale of the object, and IoU cannot directly optimize the part that does not overlap.
$$
G I o U=I o U-\frac{\left|A_{c}-U\right|}{\left|A_{c}\right|}
$$

The above formula means: first calculate the area of the smallest closed area of the two boxes, that is, the area of the smallest box that contains both the predicted box and the ground truth box, then calculate the IoU, and then calculate the area of the closed area that does not belong to the two boxes. The area accounts for the proportion of the closure area, and finally IoU is used to subtract this proportion to get GIoU.

\subsection{Visualization of  attention}
As shown in Figure ~\ref{fig:detrattentionmap}, we can clearly see what detr pays more attention to a picture through the attention map, and the focus of each of his object features can be reflected in the attention map. Even DETR has good scalability and can be used for image segmentation.

\begin{figure}[h]
	\centering
	\includegraphics[width=1\linewidth]{figures/detr_attention_map}
	\caption[The attention map of the model DETR]{The attention map of the model DETR. The Encoder Self-Attention is able to separate individual instances, and it can be well visualized in the attention map. mage source from Carion et al.~\cite{carion2020end} page 11.}
	\label{fig:detrattentionmap}
\end{figure}

\label{sec:rankingloss}
\section{Ranking Loss}

Different from Cross-Entropy Loss ~\cite{Categorical_Loss} or Mean Square Error Loss(MSE) ~\cite{CHRISTOFFERSEN2004291}, their goal is to characterize the difference between the output of the model and the actual output. But ranking loss is actually a kind of metric learning, the relative distance they learn does not care about the actual value. Because there are different names in different scenes, including Contrastive Loss, Margin Loss, Hinge Loss or Triplet Loss.

Ranking loss is widely used, including two categories, such as face recognition, which is a person or not a person.

\begin{figure}[!htbp]
	\centering
	\includegraphics[width = 0.8\textwidth]{figures/pairwise_ranking_loss_faces.png}
	\caption[Example of a pairwise ranking loss ]
	{ Example of a pairwise ranking loss setup to train a net for image face verification. In this setup, the weights of the CNNs are shared. We call it siamese nets. But a pairwise ranking loss can be used in other setups, or with other nets.(Image source:~\cite{triplet_loss_em}.)}
	\label{fig:pairwise_ranking_loss}
\end{figure}

The objective is to learn representations with a small distance $d$ between them for positive pairs, and greater distance than some margin value $m$ for negative pairs. Pairwise Ranking Loss forces representations to have $0$ distance for positive pairs, and a distance greater than a margin for negative pairs. Being $r_a$ and $r_p$the samples representations and $d$ a distance function, we can write:

\begin{equation}
L=\begin{cases}
	d(r_a,r_p) & if \qquad positive\ pair \\
	max(0,m-d(r_a,r_p)) & if \qquad negative\  pair
\end{cases}
\end{equation}

Suppose $r_0,r_1 $ is used to represent the representation of the two elements of the sample, $y$is a two-valued value, when the input is a negative sample pair, it is 0, when a positive sample pair is input, it is 1, and the distance $d$ is Euclidean distance , We can have the final loss function expression:

\begin{equation}
L(r_0,r_1,y)=y\left || r_0-r_1 \right || + (1-y)max(0,m-\left || r_0-r_1 \right ||)
\end{equation}

This setup outperforms the former by using triplets of training data samples, instead of pairs. The triplets are formed by an anchor sample $x_a$, a positive sample $x_p$ and a negative sample $x_n$. The objective is that the distance between the anchor sample and the negative sample representations $d(r_a,r_n)$ is greater (and bigger than a margin $m$) than the distance between the anchor and positive representations $d(r_a,r_p)$. With the same notation, we can write:

\begin{equation}
L(r_a,r_p,r_n)=max(0,m+d(r_a,r_p)-d(r_a,r_p))
\label{equ:ranking_loss}
\end{equation}

\begin{figure}[!htbp]
	\centering
	\includegraphics[width = 0.8\textwidth]{figures/triplet_loss_faces.png}
	\caption[Example of a triplet ranking loss ]
	{ Example of a triplet ranking loss setup to train a net for image face verification. In this setup, the weights of the CNNs are shared. We call it triple nets. (Image source:~\cite{triplet_loss_em}).}
	\label{fig:triplet_ranking_loss}
\end{figure}

\section{Word Representation}

Same as location information, semantic information also helps a lot for understanding relations. The basic encoding method is one hot code.

One-hot representation is another natural approach to represent words, which assigns a unique index to each word. It is also not good enough to represent words with one-hot representation. First, one-hot representation could not capture the semantic relatedness among words. Second, one-hot representation is a high-dimensional sparse representation, which is very inefficient. Third, it is very inflexible for one-hot representation to deal with new words, which requires assigning new indexes for new words and would change the dimensions of the representation. The change may lead to some problems for existing NLP systems.

Recently, distributed word representation approaches are proposed to address the problem of one-hot word representation. The distributional hypothesis ~\cite{bojanowski2017enriching} that linguistic objects with similar distributions have similar meanings is the basis for distributed word representation learning. Based on the distributional hypothesis, various word representation models, such as CBOW and Skip-gram, have been proposed and applied in different areas.

\begin{figure}[!htbp]
	\centering
	\includegraphics[width = 0.5\textwidth]{figures/skip-gram-model.png}
	\caption[The architecture of skip-gram model]
	{ The architecture of skip-gram model.}
	\label{fig:skip-gram-model}
\end{figure}

One of the widely used distributed word representation is Skip-Gram model (see Fig.~\ref{fig:skip-gram-model}) which is part of the \textbf{Word2Vec} library. It was created by a team of researchers led by Tomas Mikolov at Google. The main idea is to represent words by means of its neighbors. It tries to predict all neighboring words(the context) of a given word. According to paper~\cite{mikolov2013distributed}, the objective of model is defined as follows:$$\frac{1}{T}\sum_{t=1}^{T}\sum_{-c\le j\le c,j\ne 0} \log_{ p(w_{t+j}\mid w_t) }$$
where w is training word and c is the size of context. So its objective is to find word representations that can predict surrounding words.

Another member of distributed word representations is \textbf{Global Vectors for Word Representation(GloVe)} which is an abbreviation for Global Vectors. While Word2Vec captures certain local context window, GloVe exploits overall co-occurrence statistics of words from corpus, which is a large collection of texts. It consists of two important steps. First one constructs a matrix of term co-occurrences. For each word we compute conditional probability, e.g. for word water $ P(k\mid water) $, where $ k $ is word from vocabulary. If $ k $ is stream, the value of $ P $ is high, and if $ k $ is fashion, then expected value is low as they do not usually co-occur together. After performing all statistical computations, the large matrix is formed. Then high-dimensional context matrix is reduced by normalizing counts and log-smoothing.


\section{Introduction}

OptimTraj is a matlab library designed for solving continuous-time single-phase trajectory optimization problems. I developed it while working on my PhD at Cornell, studying non-linear controller design for walking robots.

\subsection{What sort of problems does OptimTraj solve?}

\subsubsection*{Examples:}
\begin{itemize}
\item Cart-Pole Swing-Up: Find the force profile to apply to the cart to swing-up the pendulum that freely hangs from it.
\item Compute the gait (joint angles, rates, and torques) for a walking robot that minimizes the energy used while walking.
\item Find a minimum-thrust orbit transfer trajectory for a satellite.
\end{itemize}

\subsubsection*{Details:}

OptimTraj finds the optimal trajectory for a dynamical system. This trajectory is a sequence of controls (expressed as a function) that moves the dynamical system between two points in state space. The trajectory will minimize some cost function, which is typically an integral along the trajectory. The trajectory will also satisfy a set user-defined constraints. All functions in the problem description can be non-linear, but they must be smooth (C$^2$ continuous).

OptimTraj solves problems with
\begin{itemize} \setlength\itemsep{-0.1em}
\item continuous dynamics
\item boundary constraints
\item path constraints
\item integral cost function
\item boundary cost function
\end{itemize}

\subsection{Features:}
\begin{itemize} \setlength\itemsep{-0.1em}
\item {\bf Easy to Install: } no dependencies outside of Matlab \footnotemark
\item {\bf Lots of example: } look at the \texttt{deom/} directory to see for yourself!
\item {\bf Readable source code: } easy to debug your code and figure out how the software works
\item {\bf Analytic gradients: } most methods support analytic gradients
\item {\bf Rapidly switch between methods: } helpful for debugging and experimenting
 	\begin{itemize} \setlength\itemsep{-0.1em}
 	\item Trapezoidal Direct Collocation
 	\item Hermite-Simpson Direct Collocation
 	\item Runge--Kutta $4^\text{th}$-order Multiple Shooting
 	\item Chebyshev--Lobatto Orthogonal Collocation
 	\end{itemize}
\end{itemize}
\footnotetext{Chebyshev--Lobatto method requires ChebFun (\texttt{http://www.chebfun.org})}

\subsection{Installation:}
\begin{enumerate} \setlength\itemsep{-0.1em}
\item Clone or download the repository (\texttt{https://github.com/MatthewPeterKelly/OptimTraj})
\item Add the top level folder to your Matlab path
\item (Optional) Clone or download ChebFun (\texttt{http://www.chebfun.org}) to use the Chebyshev--Lobatto method
\end{enumerate}

\subsection{Usage: }
\begin{itemize} \setlength\itemsep{-0.1em}
\item Call the function \texttt{optimTraj} from inside matlab.
\item \texttt{optimTraj} takes a single argument: a struct that describes your trajectory optimization problem.
\item \texttt{optimTraj} returns a struct that describes the solution. It contains a full description of the problem, the transcription method that was used, and the solution (both as a vector of points and a function handle for interpolation).
\item For more details, type \texttt{>> help optimTraj} at the command line, or check out some of the examples in the \texttt{demo/} directory.
\end{itemize}

\subsection{License: }
OptimTraj is published under the MIT License, Copyright (c) 2016 Matthew P. Kelly.  \vspace{0.5em} \\

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:  \vspace{0.5em} \\

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.  \vspace{0.5em} \\

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.