\section{Review of Parallel Computational Models}
\label{sec:BackgroundModel}

The basic computer architecture is known as von Neummann architecture or von Neumman model. This model has the following components: a memory; an arithmetic-logic unit (ALU); a central processing unit (CPU), composed of several registers; and a control unit. New technologies and computational models began to be developed simultaneously with the evolution of the von Neumann model.

In 1972, Michael Flynn proposed a classification of parallel computing architectures. This classification distinguishes the number of instructions and the number of data that can be computed in parallel~\citep{flynn1996parallel}. This classification is presented in the Table \ref{tab:taxFlynn}.

\begin{table}[htpb]
\begin{center}
\begin{tabular}{|r|c|c|}
\hline
& \bf Single Instruction & \bf Multiple Instruction\\\hline
\bf Single Data & SISD & MISD \\\hline
\bf Multiple Data & SIMD & MIMD \\\hline
\end{tabular}
\end{center}
\caption{Classification of parallel architectures proposed by Michael Flynn (1972).}
\label{tab:taxFlynn}
\end{table}

We can illustrate the table above as follows. A von Neumann model machine that has only one processing core fits the SISD (Single Instruction; Single Data) classification; a machine with multiple processing cores can be classified as MIMD; GPUs are classified on the SIMD architecture, where each thread takes index to perform vector computations, for this the programming model of CUDA is named SIMT (Single Instruction; Multiple Threads).

Parallel computing models have been an active research topic since the development of modern computers~\citep{Gibbons1983:QRQW,Juurlink:1998,Skillicorn:1998:MLP}; their main goal is to provide a standard way of describing and evaluating the performance of parallel applications. For the success of a parallel computing model, it is paramount to also consider the characteristics of the underlying architecture of the hardware being used.

Mathematical models are simplified abstraction of a real situation. A important area of the computer science is related with the analysis, design and development of algoritmh that are implemented in real machines. A first and general example is the RAM model (Random-Access Memory). This model is an abstraction of a classic computer with a single processor. More and not a few models were created based on the RAM model.

In computing, granularity is associated with the amount of computation in relation to communication, that is, the ratio of computation to the amount of communication. Parallelism of fine granularity means relatively small amounts of computational work are done between communication events
Low computation to communication ratio. Coarse and Bulk granularity is the opposite: data transfers are less frequent, and present large amounts of computation. Parallelism of coarse or Bulk granularity means relatively large amounts of computational work are done between communication events, high computation to communication ratio. The finer granularity have greater potential for parallelism and consequently the increase in speed, but the overhead costs of synchronization and communication are expensive in terms of latency.

The main objective of a parallel computing model is to provide a set of parameters to be considered in the implementation of a parallel algorithm. These parameters can be used to simulate the behavior of applications that will run on different parallel platforms. To facilitate the programming and simulation of these applications, models with specific properties to parallel programming problems have been created~\citep{Skillicorn:1998:MLP}. The most important parallel models in the litearature are the PRAM (Parallel Random Access Memory), LogP, BSP (Bulk Synchonous Parallel) and CGM (Coarse Grained Multicomputer). They are explained below.

\subsection{Parallel Random Access Machine Model (PRAM)}
The PRAM model was created by~\cite{Fortune:1978:PRAM}. This model is a simple extension of the RAM model. It consists of an infinite set of processors and a centralized shared memory by all processors. Figure \ref{fig:Pram} shows graphically the PRAM model

\begin{figure}[htpb]
\centering
\includegraphics[scale=.6]{./images/Pram.png}
\caption{PRAM  model (\textit{Parallel Random Access Machine.})}
\label{fig:Pram}
\end{figure}

The advantage of the PRAM model is its simplicity and its similarity to the sequential model of von Neumann. The processor can only read or write a memory address in one cycle. The cost of writing is equal to the cost of reading, and is also equal to the cost of any operation performed by the processor. However, in spite of its simplicity, due to the increase in the distance between the processing and the speed of communication, this model has become more and more unrealistic.

Different submodels were created from the PRAM model. Researchers have made adaptations varying the way of access to memory, trying to avoid the maximum of conflicts in the communication. The different adaptations have arisen to propose concurrent or exclusive communications at the moment to access to shared memory~\citep{Gibbons19983:QRQW, Karp:CSD-88-408}.

\subsection{Bulk Synchronous Parallel Model (BSP)}
The BSP model was introduced by~\cite{Valiant:1990}. The BSP model offers a simple abstraction of parallel architectures, see Figure \ref{fig:BSP}. In Figure~\ref{fig:BSP} a set of processors are running local computations in a superstep and before the synchronization, all the messages are delivered and ready to be used in the next superstep.

\begin{figure}[htpb]
\centering
% \includegraphics[scale=.7]{./images/bspmodel.eps}
\includegraphics[scale=.7]{./images/bspmodel.png}
\caption{Superstep in a Bulk Synchronous Parallel Model.}
\label{fig:BSP}
\end{figure}

The BSP model bridges the essential characteristics of different kinds of machines as a combination of three attributes:

\begin{itemize}
\item a set of virtual processors, each associated to a local memory;
\item a router, that delivers the messages in a point-to-point manner;
\item a synchronization mechanism.
\end{itemize}

The execution of a parallel application is organized in a sequence of \emph{supersteps}, each one divided into three successive---logically disjointed---phases.
On the first phase, all processors use their local data to perform local sequential computations in parallel (i.e., there is no communication among the processors). The second phase is a communication phase, where all nodes exchange data performing personalized all-to-all communication. The last phase consists of a global synchronization barrier, that guarantees that all messages were delivered and all processors are ready to start the next superstep.

Figure~\ref{fig:BSP} depicts the phases of a BSP application. In this figure, a processor distributes tasks to a set of processors that execute local computations and communicate in a global form, if necessary. All processors wait for the others finish their tasks in a synchronization barrier, to be able to execute the next task. Sending and receiving messages between processors is only allowed at the end of each super-step.

On the BSP model there is no restriction on sending messages, but all of them should be received before the synchronization barrier. According to the execution model, the first and second phase may occur simultaneously. A BSP algorithm consists of an arbitrary number of super-steps. The BSP model has been widely used on different applications contexts. HPC practitioners have been using the BSP model to design algorithms and software that can run on any standard architecture with guaranteed performance \cite{AlgGPU,Goldberg2004,CamargoGKG06}. Consider a BSP program that runs on $S$ supersteps. Let $g$ be the bandwidth of the network and $L$ the latency---i.e., the minimum duration of a superstep---which reflects not only the latency of the network, but also the overhead of the synchronization step. The cost to execute the $i$-th superstep is then given by:

\begin{equation}
  \label{eq:superstep-cost}
  w_i + g h_i + l
\end{equation}

where $w_i$ is the maximum amount of local computations executed, and $h_i$ is the largest number of packets sent or received by any processor during the superstep. If $W = \sum_{i=1}^{S} w_i$ is the sum of the maximum work executed on all supersteps and $H = \sum_{i=1}^{S} h_i$ the sum of the maximum number of messages exchanged in each superstep, then the total execution time of the parallel application is given by:
\begin{equation}
  \label{ec:BSP}
  T = W + g H + L S
\end{equation}

A BSP algorithm, consequently, can be completely modeled by the parameters $(w, h, g, l)$. Using these parameters, the approximate execution time of a BSP algorithm can be characterized. One of the great advantages of the BSP model is that it facilitates to develop parallel programs on different systems and architectures, serving as a bridge between the programmer who develop parallel applications in massively parallel architectures.

\subsubsection{Multi-BSP model}
The Multi-BSP model is an adaptation of the BSP model, the BSP model is commonly used in a distributed memory parallel environment, and multi-BSP is a BSP extension for multi-core processors. ~\cite{Valiant:2011} created the Multi-BSP model which is used in a parallel shared memory environment. Computational models, such as multi-BSP, allow abstraction of the complexity of the problem in a simplification that is not significantly away from the reality of current computational  architectures.

Multi-BSP is a multi-level model that has explicit parameters at each level: number of processors $p$, memory/cache sizes $m$, communication latency costs $g$ and synchronization costs $L$. The multi-BSP model of an architecture with depth $d$ will be determined by $4d$ numeric parameters, $(p1, g1, L1, m1), (p2, g2, L2, m2), ( P3, g3, L3, m3), ..., (pd, gd, Ld, md)$. At each level the four parameters quantify, respectively, the number of subcomponents, processors, communication bandwidth, synchronization cost, and memory/cache size~\citep{BSPMeasures}.

Multi-BSP and BSP are important models that allow bridging the analysis, design and development of parallel algorithms, but they are not useful to design algorithms that are executed in massively parallel architectures. New models of performance prediction of applications that run on GPUs have arisen from adaptations of the models in this literature review.

\subsection{Coarse Grained Multicomputer Model (CGM)}
\cite{Dehne:2002} studied the problem of designing scalable parallel geometric algorithms for coarse grained cases. They called this model as Coarse Grained Multicomputer model (CGM), which is very efficient for a large set of problems of the ratio $\frac{n}{p}$, with $n$ the size of the problem and $p$ the number of processors. In other words, the CGM model is very good for addressing problems where the problem of size $n$ can be divided between an determined number of processors $p$. A CGM algorithm is a special case of a BSP algorithm where all communication operations of a super-step are done in $h$ relations. A fundamental difference between the BSP model and CGM is that the first captures real machine parameters while the CGM is an abstraction that allows to develop efficient algorithms in parallel machines.

An algorithm on a CGM machine can be modeled using only two parameters, $N$ and $p$. This algorithm consists of an alternating sequence of computing and communication rounds also separated by a synchronization barrier. A computational/communication round of the CGM model corresponds to a super-step of the BSP model with communication cost $g(N/p)$. A good performance of algorithms with the CGM model is achieved by minimizing the number of super-steps, the total time of local computations and the total size of the messages.

\subsection{LogP Model}
Synchronization of a large group of processes or threads is expensive in terms of latency, especially on architectures with classification MIMD. For these cases, parallel computing models without synchronization were created. The most popular of these models is the logP model.

LogP model was proposed by \citet{Culler:1993:LogP} and received its name exactly for the variables of the model. Culler et. al. perceived that the PRAM model was not realistic due to the lack of parameters to represent communication costs in parallel applications, especially for distributed applications. The parameters used to describe a parallel system according to LogP model are:

\begin{itemize}
\item[L:] \textit{Latency} - caused by communicating a message from a source to a destiny processor.
\item[o:] \textit{Overhead} - time during which a processor is busy sending or receiving a message, during that time it can not do
computations.
\item[g:] \textit{Gap} - minimum time between consecutive message transmissions or between receiving consecutive messages; the reciprocal of $g$ corresponds to the bandwidth of the system.
\item[P:] \textit{Processors}.
\end{itemize}


\subsection{Box and whisker plots}


\chapter{DERIVATION OF THE $\Upsilon$ FUNCTION}%
\label{appendixC}

%\clearpage %remove this command if your appendix doesn't start with a landscaped page!!!!!
%\thispagestyle{plain}
%\begin{landscape}
%\begin{figure}

 % \begin{center}
  %  \includegraphics[width=6in]{LaTeX2e_logo.eps}
   % \caption{\LaTeX 2\ensuremath{\epsilon.} logo}\label{biglogo}
  %\end{center}
%\end{figure}
%\end{landscape}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%ADD LABEL

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\proposition{The Upsilon Function}\label{first}

(1) If $\beta>0$ and $\alpha\neq0$, then for all $n\geq-1$,

$$I_{n}(c;\alpha; \beta; \delta) = - \frac{e^{\alpha c}}{\alpha} \sum_{i=0}^{n}(\frac{\beta}{\alpha})^{n-i} Hh_{i}(\beta c -\delta)$$

$$+ (\frac{\beta}{\alpha})^{n+1} \frac{\sqrt{2 \pi}}{\beta} e^{\frac{\alpha \delta}{\beta}+\frac{\alpha^{2}}{2\beta^{2}}} \phi(-\beta c + \delta + \frac{\alpha}{\beta})$$
(2) If $\beta<0$ and $\alpha<0$, then for all $x \geq -1$

$$I_{n}(c;\alpha; \beta; \delta) = - \frac{e^{\alpha c}}{\alpha} \sum_{i=0}^{n}(\frac{\beta}{\alpha})^{n-i} Hh_{i}(\beta c -\delta)$$

$$- (\frac{\beta}{\alpha})^{n+1} \frac{\sqrt{2 \pi}}{\beta} e^{\frac{\alpha \delta}{\beta}+\frac{\alpha^{2}}{2\beta^{2}}} \phi(\beta c - \delta - \frac{\alpha}{\beta})$$

\begin{proof}{Case 1.}

$\beta>0$ and $\alpha\neq0$. Since, for any constant $\alpha$ and $n \geq 0$, $e^{\alpha x} Hh_{n}(\beta x - \delta) \rightarrow 0$ as $x \rightarrow \infty$ thanks to (B4), integration by parts leads to

$$I_{n}=-\frac{1}{\alpha}Hh(\beta c -\delta) e^{\alpha c} + \frac{\beta}{\alpha}\int_{c}^{\infty} e^{\alpha x} Hh_{n-1}(\beta c - \delta)dx$$

In other words, we have a recursion, for $n \geq 0$, $I_{n}=-(e^{\alpha c}{\alpha})Hh_{n}(\beta c - \delta) + (\frac{\beta}{\alpha})I_{n-1}$ with

$$I_{-1}=\sqrt{2 \pi} \int_{c}{\infty}e^{\alpha x}\varphi(-\beta x +\delta)dx$$

$$=\frac{\sqrt{2 \pi}}{\beta} e^{\frac{\alpha \delta}{\beta}+\frac{\alpha^{2}}{2 \beta^{2}}}\phi(-\beta c + \delta +\frac{\alpha}{\beta})$$

Solving it yields, for $n \geq -1$,

$$I_{n}=-\frac{e^{\alpha c}}{\alpha}\sum_{i=0}^{n}(\frac{\beta}{\alpha})^{i}Hh_{n-i}(\beta c+\delta) + (\frac{\beta}{\alpha})^{n+1}I_{-1}$$

$$=-\frac{e^{\alpha c}}{\alpha}\sum_{i=0}^{n}(\frac{\beta}{\alpha})^{n-i} Hh_{i}(\beta c+\delta)$$

$$+ (\frac{\beta}{\alpha})^{n+1}\frac{\sqrt{2 \pi}}{\beta} e^{\frac{\alpha \delta}{\beta}+\frac{\alpha^{2}}{2 \beta^{2}}}\phi(-\beta c + \delta +\frac{\alpha}{\beta})$$

where the sum over an empty set is defined to be zero.
\end{proof}

Case2. $\beta<0$ and $\alpha<0$. In this case, we must also have, for $n \geq 0$ and any constant $\alpha<0, e^{\alpha x}Hh_{n}(\beta x -\delta) \rightarrow 0$ as

$x \rightarrow \infty$, thanks to (B5). Using integration by parts, we again have the same recursion, for $n \geq 0, I_{n}=-(e^{\alpha c}/\alpha)Hh_{n}(\beta c - \delta)+(\beta / \alpha)I_{n-1}$, but with a different initial condition

$$I_{-1}=\sqrt{2 \pi}\int_{c}^{\infty}e^{\alpha x}\varphi(-\beta x + \delta)dx$$

$$=-\frac{\sqrt{2 \pi}}{\beta} exp\{\frac{\alpha \delta}{\beta}+\frac{\alpha^{2}}{2 \beta^{2}}\}\phi(\beta c - \delta -\frac{\alpha}{\beta})$$

Solving it yields (B8), for $n \geq -1$.

Finally, we sum the double exponential and the normal random variables

Proposition B.3.

Suppose $\{\xi_{1},\xi_{2},...\}$ is a sequence of i.i.d. exponential random variables with rate $\eta>0$, and Z is a normal variable with distribution $N(0,\sigma^{2})$. Then for every $ n \geq 1$, we have: (1) The density functions are given by:

$$f_{Z+\sum_{i=1}^{n}\xi_{i}}(t)=(\sigma\eta)^{n}\frac{e^{(\sigma\eta)^{2}/2}}{\sigma\sqrt{2\pi}}e^{-t\eta}Hh_{n-1}(-\frac{t}{\sigma}+\sigma\eta)$$

$$f_{Z-\sum_{i=1}^{n}\xi_{i}}(t)=(\sigma\eta)^{n}\frac{e^{(\sigma\eta)^{2}/2}}{\sigma\sqrt{2\pi}}e^{-t\eta}Hh_{n-1}(\frac{t}{\sigma}+\sigma\eta)$$
(2) The tail probabilities are given by

$$P(Z+\sum_{i=1}^{n}\xi_{i}\geq x) = (\sigma\eta)^{n}\frac{e^{(\sigma\eta)^{2}/2}}{\sigma\sqrt{2\pi}}e^{-t\eta}I_{n-1}(x;-\eta,-\frac{1}{\sigma},-\sigma\eta)$$

$$P(Z-\sum_{i=1}^{n}\xi_{i}\geq x) = (\sigma\eta)^{n}\frac{e^{(\sigma\eta)^{2}/2}}{\sigma\sqrt{2\pi}}e^{-t\eta}I_{n-1}(x;\eta,\frac{1}{\sigma},-\sigma\eta)$$

Proof. Case 1. The densities of $Z+\sum_{i=1}^{n}\xi_{i}$, and $Z-\sum_{i=1}^{n}\xi_{i}$. We have

$$f_{Z+\sum_{i=1}^{n}\xi_{i}}(t)=\int_{-\infty}^{\infty}f_{\sum_{i=1}^{n}\xi_{i}}(t-x)f_{Z}(x)dx$$

$$=e^{-t\eta}(\eta^{n})\int_{-\infty}{t}\frac{e^{x\eta}(t-x)^{n-1}}{(n-1)!}\frac{1}{\sigma\sqrt{2\pi}}e^{-x^{2}/(2\sigma^{2})}dx$$

$$=e^{-t\eta}(\eta^{n})e^{(\sigma\eta)^{2}/(2)}\int_{-\infty}{t}\frac{(t-x)^{n-1}}{(n-1)!}\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\sigma^{2}\eta)^{2}/(2\sigma^{2})}dx$$

Letting $y=(x-\sigma^{2}\eta)/\sigma$ yields

$$f_{Z+\sum_{i=1}^{n}\xi_{i}}(t)=e^{-t\eta}(\eta^{n})e^{(\sigma\eta)^{2}/(2)}\sigma^{n-1}$$

$$\times\int_{-\infty}^{t/\sigma-\sigma\eta}\frac{(t/\sigma - y -\sigma\eta)^{n-1}}{(n-1)!}\frac{1}{\sqrt{2\pi}}e^{-y^{2}/2}dy$$

$$=\frac{e^{(\sigma\eta)^{2}/2}}{\sqrt{2\pi}}(\sigma^{n-1}\eta^{n})e^{-t\eta}Hh_{n-1}(-t/\sigma + \sigma\eta)$$

because $(1/(n-1)!)\int_{-\infty}{a}(a-y)^{n-1}e^{-y^{2}/2}dy=Hh_{n-1}(a)$. The derivation of $f_{Z+\sum_{i=1}^{n}\xi_{i}}(t)$ is similar.

Case 2. $P(Z+\sum_{i=1}^{n}\xi_{i}\geq x)$ and $P(Z-\sum_{i=1}^{n}\xi_{i}\geq x)$. From (B9), it is clear that

$$P(Z+\sum_{i=1}^{n}\xi_{i}\geq x)=\frac{(\sigma\eta)^{n}e^{(\sigma\eta)^{2}/2}}{\sigma\sqrt{2\pi}}\int_{x}^{\infty}e^{(-i\eta)}Hh_{n-1}(-\frac{t}{\sigma}+\sigma\eta)dt$$

$$=\frac{(\sigma\eta)^{n}e^{(\sigma\eta)^{2}/2}}{\sigma\sqrt{2\pi}}I_{n-1}(x;-\eta,-\frac{1}{\sigma},-\sigma\eta)dt$$

by (B6). We can compute
$P(Z-\sum_{i=1}^{n}\xi_{i}\geq x)$ similarly.

\theorem{Theorem} With $\pi_{n}:= P(N(t)=n)=e^{-\lambda T}(\lambda T)^{n}/n!$ and $I_{n}$ in Proposition \ref{first}.
, we have

$$P(Z(T)\geq a)=\frac{e^{(\sigma \eta_{1})^{2} T/2}}{\sigma \sqrt{2 \pi T}} \sum_{n=1}^{\infty} \pi_{n} \sum_{k=1}^{n} P_{n,k}(\sigma\sqrt{T}\eta_{1})^{k}\times I_{k-1}(a-\mu T; -\eta_{1},-\frac{1}{\sigma\sqrt{T}},-\sigma\eta_{1}\sqrt{T})$$

$$+\frac{e^{(\sigma\eta_{2})^{2}T/2}}{\sigma\sqrt{2\pi T}}\sum_{n=1}^{\infty}\pi_{n}\sum_{k=1}^{n}Q_{n,k}(\sigma\sqrt{T}\eta_{2})^{k}$$

$$\times I_{k-1}(a-\mu T; \eta_{2},\frac{1}{\sigma\sqrt{T}},-\sigma\eta_{2}\sqrt{T})$$

$$+\pi_{0}\phi(-\frac{a-\mu T}{\sigma\sqrt{T}})$$

Proof by the decomposition (B2)

$$P(Z(T) \geq a)= \sum_{n=0}^{\infty}\pi_{n} P(\mu T +\sigma\sqrt{T} Z + \sum_{j=1}^{n}Y_{j} \geq a)$$

$$=\pi_{0}P(\mu T +\sigma\sqrt{T} Z  \geq a)$$

$$+\sum_{n=1}^{\infty}\pi_{n}\sum_{k=1}^{n}P_{n,k} P(\mu T +\sigma\sqrt{T} Z + \sum_{j=1}^{n}\xi_{j}^{+} \geq a)$$

$$+\sum_{n=1}^{\infty}\pi_{n}\sum_{k=1}^{n}Q_{n,k} P(\mu T +\sigma\sqrt{T} Z - \sum_{j=1}^{n}\xi_{j}^{-} \geq a)$$

The result now follows via (B11) and (B12) for $\eta_{1} > 1$ and $\eta_{2} >0$.


\chapter{Quadtree}

% tikz
% http://www.texample.net/tikz/examples/merge-sort-recursion-tree/
% http://www.texample.net/tikz/examples/probability-tree/

% set overall layout of tree
\tikzstyle{level 1}=[level distance=3.0cm, sibling distance=5.0cm]
\tikzstyle{level 2}=[level distance=4.0cm, sibling distance=2.5cm]

% define styles for bags and leaves
\tikzstyle{bag} = [text width=4em, text centered]
\tikzstyle{end} = [circle, minimum width=3pt, inner sep=0pt]

\begin{figure}[htb]
\centering
\begin{tikzpicture}[grow=right, sloped,
	level 0/.style = {tabblue},
	level 1/.style = {taborange, sibling distance = 4cm},
    level 2/.style = {tabgreen, sibling distance = 1.0cm},
    level 3/.style = {teal, level distance = 1cm}]
%\begin{tikzpicture}[grow=right]
\node[circle, draw] {\tiny root}
	child {
		node[circle, draw] {\tiny \bf{11}}
			child {
				node[circle, draw] {\tiny \bf{0011}}
				edge from parent
			}
			child {
				node[circle, draw] {\tiny \bf{0010}}
				edge from parent
			}
			child {
				node[circle, draw] {\tiny \bf{0001}}
				edge from parent
			}
			child {
				node[circle, draw] {\tiny \bf{0000}}
				edge from parent
			}
		edge from parent
		% node[above] {\scriptsize $P(\overline{C})=0.99$}
	}
	child {
		node[circle, draw] {\tiny \bf{10}}
		edge from parent
		% node[above] {\scriptsize $P(\overline{C})=0.99$}
	}
	child {
		node[circle, draw] {\tiny \bf{01}}
		edge from parent
		% node[above] {\scriptsize $P(\overline{C})=0.99$}
	}
	child {
		node[circle, draw] {\tiny \bf{00}}
			child {
				node[circle, draw] {\tiny \bf{0011}}
				edge from parent
			}
			child {
				node[circle, draw] {\tiny \bf{0010}}
				edge from parent
			}
			child {
				node[circle, draw] {\tiny \bf{0001}}
				edge from parent
			}
			child {
				node[circle, draw] {\tiny \bf{0000}}
				edge from parent
			}
		edge from parent
		% node[above] {\scriptsize $P(C)=0.01$}
		% node[below] {\scriptsize prevalence}
	}; % remember trailing semi-colon after last child
\end{tikzpicture}
% \caption{Decision tree for mammogram as a test $T$ for breast cancer $C$.}
\label{fig:quadtree}
\end{figure}


\par
\chapter{{\tt SubMtx}: Submatrix object}
\par
The {\tt SubMtx} object was created to hold the data for and operate
with a submatrix of a sparse matrix.
The entries in a submatrix can be either double precision
real or complex.
\par
For example, the lower and upper triangular matrices $L$ and $U$
that are created during the factorization are stored as
submatrices,
e.g., $L_{I,I}$ and $L_{J,I}$ where $I$ and $J$ are index sets.
To be more precise, $I$ and $J$ are index sets associated with
fronts {\tt I} and {\tt J}.
We do not necessarily represent $L_{J,I}$,
because some of the rows in the submatrix may be zero.
Instead we keep $L_{\bnd{I}\cap J,I}$, where
$\bnd{I} \cap J$ are precisely those rows that may have nonzeros.
The situation is similar for $U$ where we keep $U_{I,\bnd{I}\cap J}$.
\par
The submatrices for $L$ and $U$ may be dense or sparse.
(A direct factorization typically generates dense submatrices
while a drop tolerance factorization produces sparse submatrices.)
We also use {\tt SubMtx} objects to represent submatrices of the
$D$ matrix, where $D$ is either diagonal or has $1 \times 1$ and
$2 \times 2$ blocks on its diagonal.
In the latter case, we support $D_{I,I}$ to be either
real symmetric, complex symmetric or complex Hermitian.
\par
The {\tt SubMtx} object has the following attributes.
\begin{itemize}
\item
A {\tt SubMtx} object has a row id and column id to identify itself
within the context of a larger block matrix.
\item
Each row and column of the block matrix corresponds
to a certain index set.
A {\tt SubMtx} object associated with block row {\tt J}
and block column {I} has row indices $J$ and column indices $I$.
\item
Matrix entries stored in one of the following ways.
\begin{itemize}
\item dense by rows, i.e., dense and row major
\item dense by columns, i.e., dense and column major
\item sparse using dense subrows
\item sparse using dense subcolumns
\item sparse using sparse rows
\item sparse using sparse columns
\item sparse using $(i,j,a_{i,j})$ triples
\item a diagonal matrix
\item a block diagonal symmetric matrix where the blocks are
      $1 \times 1$ or $2 \times 2$, used in the symmetric
      indefinite factorization.
\item a block diagonal Hermitian matrix where the blocks are
      $1 \times 1$ or $2 \times 2$, used in the hermitian
      indefinite factorization.
\end{itemize}
\item
The {\tt SubMtx} object can be self-contained, in the sense that
its structure contains a {\tt DV} object that manages a contiguous
vector of workspace that is used to store all information about the
{\tt SubMtx} object --- its scalar parameters, any integer index
or dimension information, and all matrix entries.
In a distributed environment,
this allows a {\tt SubMtx} object to be sent between processors
as one message, no copying to an internal buffer is needed,
nor any custom data type needs to be defined as for MPI.
In an out-of-core environment,
a {\tt SubMtx} object can be read from or written to a file
by a single operation.
\end{itemize}
\par
The {\tt SubMtx} object is a superset of the {\tt DenseMtx} object
in terms of data structure and functionality.
If we were working in a language that supports inheritance,
{\tt SubMtx} would be an abstract class and {\tt DenseMtx} would be
a subclass where entries would be stored by dense rows or columns.
At some point in the future we may deprecate the {\tt DenseMtx}
object in this library, replacing it with the {\tt SubMtx} object.
\par
Because the {\tt SubMtx} object wears so many hats, i.e., it supports
nine different storage formats, it has to be flexible in how it
responds to its environment.
For example, how we access the data is different depending on which
storage format.
Instead of accessing structure fields directly,
e.g., let {\tt mtx->entries} point to the start of the matrix entries,
we follow a convention that {\it instance} methods return
information.
For example, the function call
\begin{verbatim}
       SubMtx_columnIndices(mtx, &nrow, &rowind) ;
\end{verbatim}
is an instance method that fills {\tt nrow} with the number of
rows and {\tt rowind} with the first location of the row indices.
A more complex example is for the sparse storage by rows format,
\begin{verbatim}
       SubMtx_sparseRowsInfo(mtx, &nrow, &nent, &sizes, &indices, &entries) ;
\end{verbatim}
where the number of rows and entries are returned in {\tt nrow}
and {\tt nent}, the number of nonzero
entries in each row is contained in {\tt sizes[]},
and the column indices and nonzero entries are found in
{\tt indices[]} and {\tt entries[]}, respectively.
This convention of using instance methods to return information
is better than using explicit structure fields.
For example, if we want to extend the object by allowing another
storage format, we do not need to increase the size of the structure
at all --- it is only necessary to provide one or more instance methods
to return the new information.


\documentclass[simplex.tex]{subfiles}
% NO NEED TO INPUT PREAMBLES HERE
% packages are inherited; you can compile this on its own
\begin{document}
\subsection{Network Dependence Test via Diffusion Maps and MGC}
%% Jan
Deciphering the association between network structures and corresponding
nodal attributes of interest is a core problem in network science. We
propose a new nonparametric procedure for testing dependence between
network topology and nodal attributes, via diffusion maps and
\texttt{MGC}. Specifically, under an exchangeable graph, we verify that
the diffusion maps provide a set of conditionally independent
multivariate coordinates for the nodes, which can be combined with
\texttt{MGC} (or in general, any distance-based correlation measures) to
yield consistent statistic for network dependence testing. In
simulation, the new approach achieves superior testing performance under
a variety of common network models than existing benchmarks. The
diffusion maps provides a robust metric compared to adjacency matrix or
geodesic distance, while \texttt{MGC} can better capture nonlinear
dependencies, with their combined advantages shown in
Figure~\ref{fig:threeSBM201701}.

\begin{figure}[h!]
\begin{cframed}
		\centering
		\includegraphics[width=0.6\textwidth]{../../figs/ThreeSBM.png}
		\caption{Power comparison for all possible combinations of metrics and correlation measure, under the stochastic block model with three blocks. \texttt{MGC} with the diffusion maps (DM) yields the best power, comparing to using other metrics like adjacency matrix (AM), latent factors (LF), and other test statistics like distance correlation (mcorr), Heller-Heller-Gorfine (HHG) test, or Fosdick and Hoff (FH) method.}
		\label{fig:threeSBM201701}
		\end{cframed}
\end{figure}

%In order to show that \texttt{MGC} combined with diffusion maps as a network metrics perform better even in the case arbitrary noisy is added to edges or the attributes in real data, we are doing an experiment on brain network with physical locations as nodal attributes. Our proposed method not only detects the dependence between network topology and nodal attributes but also helps us to reveal possibly diverse dependence patterns through multiscale correlation maps or multiscale statistics as a function of diffusion time.

This month we made significant progress in writing the manuscript and improving the exposition. The current draft was submitted to ASA Nonparametric Statistics Section Student Paper Awards, and we are notified as finalists for awards and special presentation section in the Joint Statistical Meeting this year.

\clearpage

%% Feb

%% March
Deciphering the association between network structures and corresponding nodal attributes of interest is a core problem in network science. We propose a new nonparametric procedure for testing dependence between network topology and nodal attributes, via diffusion maps and \texttt{MGC}. Specifically, under an exchangeable graph, we verify that the diffusion maps provide a set of conditionally independent multivariate coordinates for the nodes, which can be combined with \texttt{MGC} (or in general, any distance-based correlation measures) to yield consistent statistic for network dependence testing. Moreover, our method is computationally inexpensive and robust against parameter mis-specifications, very efficient in capturing a wide variety of nonlinear and high-dimensional relationships, and readily extend-able to testing independence between two graphs.

Figure~\ref{fig:threeSBM201703} illustrates the advantage of the proposed method on testing dependency between two graphs. The graphs are simulated by the random dot product graph, with the underlying latent variables being related by a quadratic function. By repeatedly generating dependent sample graphs, the testing power equals the percentage of rejection of the independence hypothesis. Although all methods are consistent (having power $1$ as number of vertices increases), the proposed approach using \texttt{MGC} is able to achieve perfect testing power at a very small size, which is significantly better than other benchmarks.

\begin{figure}[h!]
\begin{cframed}
		\centering
		\includegraphics[width=0.7\textwidth]{../../figs/twoGraphs1.pdf}
		\caption{The power curve with respect to increasing number of vertices for the two-graph dependency testing simulation. The proposed approach quickly attains perfect power at a very small vertex size, while other benchmarks often require a much larger graph for perfect testing. }
		\label{fig:threeSBM201703}
		\end{cframed}
\end{figure}

An early draft is recently awarded the Best Student Paper Awards by the American Statistical Association Nonparametric Statistics Section, which will be presented in a special section in the Joint Statistical Meeting this year. We collected and addressed feedback from experts in graph inference, and submitted the complete manuscript this month.
%

\clearpage
\end{document}


%\documentclass{scrartcl}
%\usepackage{beamerarticle}
%%\usepackage{dtsc-beamer}
%\usepackage{fullpage}

\documentclass[9pt]{beamer}
\usetheme{boxes}
\usetheme{Boadilla}
\usecolortheme{beaver}
%\usecolortheme{sidebartab}
% \usefonttheme{structurebold}
\usefonttheme{serif}

%gets rid of bottom navigation bars
\setbeamertemplate{footline}[page number]{}

%gets rid of navigation symbols
\setbeamertemplate{navigation symbols}{}

% \usepackage{helvet}
\usepackage{amsmath, amssymb}
\usepackage{color}
%\usepackage{asymptote}
\usepackage{mathrsfs}
\usepackage{dsfont}
\usepackage{url}
\usepackage{cancel}
\usepackage{tikz}
\usetikzlibrary{fit,positioning}
\usetikzlibrary{shapes,matrix,decorations.markings,arrows}
\usetikzlibrary{graphs}
\usepackage{bbm}
\def\ind{\mathbbm{1}} %Indicator function
%
\definecolor{darkblue}{rgb}{0.0, 0.0, 0.55}
\definecolor{notsodarkblue}{rgb}{0.0, 0.0, 0.7}
\setbeamercolor{title}{fg=darkblue}
\setbeamercolor{frametitle}{fg=darkblue}
\newcommand{\myitem}{\item[$\bullet$]}
\definecolor{darkgreen}{rgb}{0, 0.55, 0}
\definecolor{lgray}{rgb}{0.9,0.9,0.9}

\newcommand{\LABFIG}[1]{\label{fig:#1}}%{\tt [fig:$\text{$#1$}$]}}

\newcommand{\ve}[1]{\boldsymbol{#1}}
\def\X{\ve{X}}
\def\x{\ve{x}}
\def\y{\ve{y}}
\def\m{\ve{m}}
\def\Sig{\ve{\Sigma}}
\def\E{\mathbb{E}}
\def\J{\ve{J}}
\def\h{\ve{h}}

\newcommand{\mc}[1]{\mathcal{#1}}
\def\sX{\mc{X}}
\def\N{\mc{N}}
\newcommand{\mxf}[3]{m^{#3}_{X_{#1}\rightarrow f_{#2}}(x_{#1})}
\newcommand{\mfx}[3]{m^{#3}_{f_{#2}\rightarrow X_{#1}}(x_{#1})}

\newcommand\Def[1]{{\textbf{Definition:}\\\emph{#1}\\\begin{center} ------------------------ \end{center}}}
\newcommand\Prop[1]{{\textbf{\textcolor{red}{Property:}}\\\emph{#1}\\\begin{center} \textcolor{red}{------------------------} \end{center}}}

\newcommand{\noteB}[1]{\textbf{\textcolor{notsodarkblue}{#1}}}

\newcommand{\noteR}[1]{\textbf{\textcolor{darkred}{#1}}}

\newcommand{\noteG}[1]{\textbf{\textcolor{darkgreen}{#1}}}

\newcommand{\snoteB}[1]{{\textcolor{darkblue}{#1}}}

\newcommand{\snoteR}[1]{{\textcolor{darkred}{#1}}}

\newcommand{\snoteG}[1]{{\textcolor{darkgreen}{#1}}}

\newcommand{\fs}[2]{#2}

\title[]{Gaussian BP}
\author[\textcolor{white}{Advanced Digital Communications}]{Introduction to Graphical Models and Inference for Communications\\\vspace*{3mm}{\small \textcolor{black}{UC3M}}
}
%\date[08/02/2016]{{08/02/2016}}
\institute{\textcolor{white}{}}
\AtBeginSection[]
{
  \begin{frame}<beamer>{Index}
    \tableofcontents[currentsection,currentsubsection]
  \end{frame}
}

\begin{document}

\frame{
\titlepage
\thispagestyle{empty}
\begin{center}
\includegraphics[scale=0.05]{Figuras/uc3m-logo.pdf}
\end{center}
}

\frame{
\frametitle{Today}

\begin{itemize}
\item One particular scenario: we want to perform inference over Gaussian probability distributions that factorize according to a graphical model
\item Factor Graphs.
\item Belief Propagation naturally extends to this scenario by replacing summations to integrals (\emph{Gaussian Belief Propagation}).
\item GaBP is exact for Gaussian tree factor graphs.
\item GaBP computes the exact mean for graphs with cycles, but approximated covariance matrix.
\end{itemize}

}

\section{Multivariate Gaussian distribution}

\frame{
\frametitle{Multivariate Gaussian distribution}
Let $\X$ be a Gaussian random vector ($\x\in\mathbb{R}^n$).
\begin{itemize}
\item \textbf{Covariance form:} the probability density function is
\begin{align*}
\mu(\x)=\frac{1}{(2\pi)^{n/2}|\Sig|^{1/2}}\exp\left\{-\frac{1}{2}(\x-\m) ^T \Sig^{-1}(\x-\m) \right\}
\end{align*}
denoted as $\x\sim\mathcal{N}(\m,\Sig)$ with mean $\m=\E[\x]$ and covariance matrix $\Sig=\E[(\x-\m)^T(\x-\m)]$.
\item \textbf{Natural form:} the probability density function is
\begin{align*}
\mu(\x)\propto \exp\left\{ -\frac{1}{2} \x^T \ve{J} \x+\ve{h}^{T}\x\right\}
\end{align*}
denoted as $\x\sim\mathcal{N}^{-1}(\ve{h},\ve{J})$ with potential vector $\h$ and \emph{precision matrix} $\J$.
\item Note $\J=\Sig^{-1}$ and $\h=\J\m$.
\end{itemize}

\begin{block}{}
Gaussian graphical models typically describe Gaussian joint probability density functions in \textbf{natural form}.
\end{block}

}

\frame{
\frametitle{Product of two Gaussians}

\begin{align*}
&\mathcal{N}(\m_a,\Sig_a)\mathcal{N}(\m_b,\Sig_b)\\
=&\mathcal{N}^{-1}(\ve{h}_a,\ve{J}_a)\mathcal{N}^{-1}(\ve{h}_b,\ve{J}_b)\\
=&\mathcal{N}^{-1}(\ve{h}_a+\h_b,\ve{J}_a+\J_b)=\N(\m,\Sig)
\end{align*}
where
\begin{align*}
\Sig=\left(\ve{J}_a+\J_b\right)^{-1}\qquad \m=\Sig(\ve{h}_a+\h_b)
\end{align*}

}

\frame{
\frametitle{Marginalization via algebraic manipulation}
\begin{itemize}
\item Let $\y=\ve{A} \x+ \ve{v}$, where $\x\sim\mathcal{N}(\m_x,\Sig_x)=\mathcal{N}^{-1}(\h_x,\J_x)$ and $\ve{v}\sim\mathcal{N}(\m_v,\Sig_v)=\mathcal{N}^{-1}(\h_v,\J_v)$, then
\begin{align*}
\mu(\y)&=\int_{-\infty}^{\infty}\mu(\y|\x)\mu(\x)d\x?
\end{align*}
\end{itemize}

\begin{align*}
\mu(\y|\x)\mu(\x)\propto &\exp\Big\{\frac{-1}{2}\y^T \J_v \y+(\J_v\m_v)^T\y +(\J_v\ve{A}\x)^T\y-\frac{-1}{2}\x^T \J_x \x+(\J_x\m_x)^T\x \Big\}\\
\propto&\Big\{ \frac{-1}{2} \left[\begin{array}{c}\y \\ \x \end{array} \right]^T \J_* \left[\begin{array}{c}\y \\ \x \end{array} \right] +†\h^T_* \left[\begin{array}{c}\y \\ \x \end{array} \right]  \Big\}
\end{align*}
where
\begin{align*}
\J_*=\left[\begin{array}{cc} \J_v & -\J_v\ve{A} \\ -(\J_v\ve{A})^T & \J_x \end{array}\right] \qquad \h_*=\left[\begin{array}{c}\h_v \\ \h_x \end{array}\right]
\end{align*}

}

\frame{
\begin{align*}
\J_*=\left[\begin{array}{cc} \J_x & -\J_v\ve{A} \\ -(\J_v\ve{A})^T & \J_v \end{array}\right] \qquad \h_*=\left[\begin{array}{c}\h_v \\ \h_x \end{array}\right]
\end{align*}

Marginalization its easy if we obtain the covariance form of the joint distribution. Applying the matrix inversion lemma:
\begin{align*}
\left[\begin{array}{cc} \J_v & -\J_v\ve{A} \\ -(\J_v\ve{A})^T & \J_x \end{array}\right]^{-1}=
\left[\begin{array}{cc}
\ve{S}^{-1} & \ve{S}^{-1}\J_v\ve{A}\J^{-1}_x \\  \J^{-1}_x(\J_v\ve{A})^T\ve{S}^{-1} & \J^{-1}_x+\J^{-1}_x(\J_v\ve{A})^T\ve{S}^{-1} \J_v\ve{A}\J^{-1}_x
\end{array}\right]
\end{align*}
where $\ve{S}=\J_v-\J_v\ve{A}\J_x^{-1}(\J_v\ve{A})^T$.

\begin{block}{Therefore}
\begin{align*}
p(\y)=\mathcal{N}^{-1}(\h_y,\J_y)
\end{align*}
where
\begin{align*}
 \J_y=\J_v-\J_v\ve{A}\Sig_x\ve{A}^T\J_v^T \qquad \h_y=\h_v+\J_v\ve{A}\Sig_x\h_x
\end{align*}
\end{block}

}

\section{Gaussian BP over a particular example}

\frame{
Let $X_i\in\sX$ $i=1,\ldots,5$ be a collection of real R.V.  with joint distribution of the form
\begin{align*}
\mu(\x)=\frac{1}{Z}f_1(x_1,x_2)f_2(x_2,x_3)f_3(x_2,x_4)f_4(x_4,x_5)\prod_{i=1}^{5}g(x_i)
\end{align*}
where
\begin{align*}
f_1(x_1,x_2)=\exp\{F_{1}x_1x_2\} \qquad f_2(x_2,x_3)=\exp\{F_{2}x_2x_3\}\\
f_3(x_2,x_4)=\exp\{F_{3}x_2x_4\} \qquad f_4(x_4,x_5)=\exp\{F_{4}x_4x_5\}
\end{align*}
 and
\begin{align*}
g(x_i)=\exp\{b_i x_i-\frac{1}{2}\pi_i x_i^2\}
\end{align*}

\begin{figure}
\begin{tabular}{c}
\includegraphics[scale=0.5]{Figuras/Graph_1.pdf}
\end{tabular}
\end{figure}
%
%\begin{block}{Assume we want to compute}
%\begin{align*}
%P(x_1)=\sum_{\x\sim x_1}P(x_1,x_2,x_3,x_4,x_5)
%\end{align*}
%\textbf{\textcolor{red}{By brute force... $|\sX|^{5}$ sums.}}
%\end{block}
%
%

}

\frame{
\begin{itemize}
\item $\mu(\x)$ is a Gaussian distribution defined in natural form!
\end{itemize}
\begin{align*}
\J&=\left[\begin{array}{ccccc}
\pi_1 & -F_{1} & 0 & 0 & 0 \\
-F_{1} & \pi_2 & -F_{2} & -F_{3} & 0\\
0 & -F_{2} & \pi_3 & 0 & 0 \\
0 &  -F_{3} & 0 & \pi_4 & -F_{4}  \\
0 & 0 & 0 & -F_{4} &  \pi_5
\end{array}
\right]\\\\
\h&=\left[\begin{array}{ccccc}
b_1 & b_2 & b_3 & b_4 & b_5
\end{array}
\right]^T
\end{align*}

\begin{block}{}
Computing the mean $\m$ and covariance matrix $\Sig$ requires computing $\Sig=\J^{-1}$ and $\m = \J^{-1} \h$. $O(n^3)$ cost!
\end{block}

\begin{exampleblock}{}
BP provides a way to exploit graph structure to perform this
computation in $\mathcal{O}(n)$ time instead of $O(n^3)$. It is only exact when the graph is a tree!
\end{exampleblock}

\begin{alertblock}{}
If the Gaussian graphical model has cycles, GBP still computes the exact $\m$, but only an estimation to $\Sig$.
\end{alertblock}

}

%\section{Gaussian BP}

\frame{
\frametitle{Gaussian BP algorithm}

\begin{itemize}
\item It is described as a \emph{message-passing algorithm}.
\item Messages represent \emph{local computations} at each node of the graph.
\item The FG has leave factors (only connected to one variable) and pairwise factors (connected to two variables).
\item Integrals instead of sums. The rest of the algorithm remains unaltered.
\end{itemize}

\begin{figure}
\begin{tabular}{c}
\includegraphics[scale=0.5]{Figuras/Graph_4.pdf}
\end{tabular}
\end{figure}

}

\frame{
\frametitle{Iteration 0}
\begin{itemize}
\item Messages send by variable nodes are initialized by their  leave factors
\end{itemize}
E.g.
\begin{align*}
m^{0}_{x_2\rightarrow f_2}(x_2)=g(x_2)=\exp\{b_2 x_2-\frac{1}{2}\pi_2 x_2^2\}\sim \mathcal{N}^{-1}(b_2,\pi_2) \quad   \text{\color{blue}{Gaussian message!}}
\end{align*}
\begin{itemize}
\item Messages send by the $f_j$ factors are computed as usual but replacing sums by integrals.
\end{itemize}
E.g.
\begin{align*}
m^{0}_{f_1\rightarrow x_1}(x_1)&=\int \exp\{F_{1}x_1x_2\}\exp\{b_2 x_2-\frac{1}{2}\pi_2 x_2^2\}  dx_2\\
&=\int \exp\Big\{-\frac{1}{2} \left[\begin{array}{c} x_1 \\ x_2 \end{array}\right]^T \left[\begin{array}{cc} 0 & -F_1 \\ -F_1 & \pi_2\end{array} \right] \left[\begin{array}{c} x_1 \\ x_2 \end{array}\right]+\left[\begin{array}{c} 0 \\b_2 \end{array}\right]^T\left[\begin{array}{c} x_1 \\ x_2 \end{array}\right]\Big\} dx_2\\
&\sim \mathcal{N}^{-1}(F_1\pi_2^{-1}b_2,F_1^2\pi_2^{-1})=\mathcal{N}^{-1}(h^{0}_{f_1\rightarrow x_1}, J^{0}_{f_1\rightarrow x_1})\quad \text{\color{blue}{Gaussian message!}}
\end{align*}
}

\frame{
\frametitle{At iteration $\ell$}

\begin{itemize}
\item \textbf{Step 1:} Variable nodes multiply their incoming messages to send a new message to factors.
\end{itemize}
E.g.
\begin{align*}
m^{\ell}_{x_2\rightarrow f_3}(x_2)&=\mathcal{N}^{-1}(h^{\ell-1}_{f_1\rightarrow x_2}, J^{\ell-1}_{f_1\rightarrow x_2})\mathcal{N}^{-1}(h^{\ell-1}_{f_2\rightarrow x_2}, J^{\ell-1}_{f_2\rightarrow x_2})\\
&=\mathcal{N}^{-1}(h^{\ell-1}_{f_1\rightarrow x_2}+h^{\ell-1}_{f_2\rightarrow x_2}, J^{\ell-1}_{f_1\rightarrow x_2}+J^{\ell-1}_{f_2\rightarrow x_2})\\
&=\mathcal{N}^{-1}(h^{\ell}_{x_2\rightarrow f_3}, J^{\ell}_{x_2\rightarrow f_3}).
\end{align*}
}

\frame{
\frametitle{At iteration $\ell$}
\begin{itemize}
\item \textbf{Step 2:} Factor nodes compute the messages to variable nodes by marginalization.
\end{itemize}
E.g.
\begin{align*}
&m^{\ell}_{f_3\rightarrow x_4}(x_4)=\int \exp\{F_{3}x_2x_4\}\mathcal{N}^{-1}(h^{\ell}_{x_2\rightarrow f_3}, J^{\ell}_{x_2\rightarrow f_3})  dx_2\\
&=\int \exp\Big\{-\frac{1}{2} \left[\begin{array}{c} x_2 \\ x_4 \end{array}\right]^T \left[\begin{array}{cc} 0 & -F_3 \\ -F_3 & J^{\ell}_{x_2\rightarrow f_3}\end{array} \right] \left[\begin{array}{c} x_2 \\ x_4 \end{array}\right]+\left[\begin{array}{c} 0 \\h^{\ell}_{x_2\rightarrow f_3} \end{array}\right]^T\left[\begin{array}{c} x_2 \\ x_4 \end{array}\right]\Big\} dx_2\\
&\sim \mathcal{N}^{-1}(\frac{F_3 h^{\ell}_{x_2\rightarrow f_3}}{J^{\ell}_{x_2\rightarrow f_3}},\frac{F_3^2}{J^{\ell}_{x_2\rightarrow f_3}})=\mathcal{N}^{-1}(h^{\ell}_{f_3\rightarrow x_4}, J^{\ell}_{f_3\rightarrow x_4})\quad \text{\color{blue}{Gaussian message!}}
\end{align*}

\begin{block}{}
All messages are Gaussian! The parameters of the Gaussian messages are computed in closed form without doing any integral!
\end{block}

\begin{exampleblock}{}
Convergence is guaranteed and achieved in a finite number  $\ell_*$ of iterations. The overall complexity is $\mathcal{O}(\ell_*n)$.
\end{exampleblock}
}

\section{Gaussian Hidden Markov Models}

\frame{
\frametitle{Gaussian Hidden Markov Models}

\begin{block}{}
\begin{itemize}
\item Wireless communication channels.
\item Speech processing.
\item Tracking applications.
\end{itemize}
\end{block}

\begin{figure}
\begin{tabular}{c}
\includegraphics[scale=0.75]{Figuras/HMM.pdf}
\end{tabular}
\end{figure}

}

\frame{
\frametitle{Gaussian Hidden Markov Models}
\begin{itemize}
\item States $\x_t\in\mathbb{R}^{d}$.
\item State transition matrix $\ve{A}\in\mathbb{R}^{d\times d}$.
\item Process noise $\ve{v}_t\in\mathbb{R}^{p}$ and $\sim\mathcal{N}(\ve{0},\Sig_v)$ for some $\Sig_v\in\mathbb{R}^{p\times p}$.
\item Dynamic equations:
\begin{align*}
\x_{t+1}=\ve{A}\x_t+\ve{B}\ve{v}_t\\
\x_0\sim \mathcal{N}(0,\Sig^{0}_{x})
\end{align*}
\end{itemize}

\begin{itemize}
\item Noisy observation  $\y_t\in\mathbb{R}^{d'}$:
\begin{align*}
\y_t=\ve{C}\x_t+\ve{w}_t
\end{align*}
where $\ve{C}\in\mathbb{R}^{d'\times d}$ and $\ve{w}_t\sim\mathcal{N}(0,\Sig_w)$.
\end{itemize}

\begin{figure}
\begin{tabular}{c}
\includegraphics[scale=0.6]{Figuras/HMM.pdf}
\end{tabular}
\end{figure}

}

\frame{
\frametitle{Gaussian Hidden Markov Models}
\begin{itemize}
\item In summary, for $\Sig_h=\ve{B}\Sig_v\ve{B}^T$ we have
\begin{align*}
\x_0&\sim \mathcal{N}(0,\Sig^{0}_{x})\\
\x_{t+1}|\x_t &\sim \mathcal{N}(\ve{A}\x_t,\Sig_h)\\
\y_t|\x_t &\sim \mathcal{N}(\ve{C}\x_t, \Sig_w)
\end{align*}
\item Factorization
\begin{align*}
\mu(\x,\y)=\mu(x_0)\mu(\y_0|x_0)\mu(\x_1|\x_0)\mu(\y_1|x_1)\mu(\x_2|\x_1)\mu(\y_2|x_2)\ldots
\end{align*}
\item Gaussian graphical model with no cycles!! GaBP is exact and cheap!
\item This factorization can be expanded as the product of leave factors and pairwise factors.
\end{itemize}

}

\frame{
\frametitle{Inference over Gaussian HMMs}

\begin{block}{Forward/Backward algorithm}
Given $\y_0,\y_2,\ldots,\y_L$, use GaBP to compute the (Gaussian) marginal for each state $\x_1,\x_2,\ldots,\x_L$.
\end{block}

\begin{block}{Kalman filter}
Given the actual observation $\y_t$ and the observations in the past $\y_0,\y_2,\ldots,\y_t-1$, use GaBP to compute the mean of $\x_t$, $\mathbb{E}[\x_t]$.
\end{block}

}

\section{Extension to any distribution?}

\frame{
\frametitle{BP over any graphical models}
\begin{itemize}
\item So far, we have applied BP to discrete and Gaussian graphical models.
\item In the Gaussian case, we can avoid integration by algebraic manipulation.
\item The same update rules can be applied to any Graphical model. However, integrals of the form
\begin{align*}
\int_{\x_j\sim x_i} f_j(\x_j) \prod_{u\in\N(f_j)}\m_{x_u\rightarrow f_j}(x_u) d(\x_j\sim x_i)
\end{align*}
are intractable in general! we cannot solve the integrals!!
\end{itemize}

}

\frame{
\begin{block}{Approximate message passing (APM)}
Approximate BP messages by Gaussian distributions:
\begin{itemize}
\item Compressed sensing \url{http://people.ee.duke.edu/~lcarin/AMP1.pdf}
\item Efficient multiuser detection in CDMA systems \url{http://arxiv.org/pdf/0810.1729.pdf}
\item Communications over Fading ISI channels:
\url{http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=04907469}
\item $\ldots$
\end{itemize}
\end{block}

\begin{exampleblock}{Alternatives}
Instead of approximating the BP messages, construct directly a tractable Gaussian approximation to the joint pdf.
\begin{itemize}
\item Variational Inference and Mean Field.
\item Expectation Propagation.
\end{itemize}
\end{exampleblock}

}

\end{document}


\documentclass[11pt]{article}

\usepackage{amsmath,amssymb,latexsym}
\usepackage{verbatim}
\usepackage{braket}
\usepackage{fullpage}
\usepackage{listings}
\usepackage{color}
\usepackage{graphicx}

%% disables automatic indentation
%% sets indent command to allow for manual indentation
\newlength\tindent
\setlength{\tindent}{\parindent}
\setlength{\parindent}{0pt}
\renewcommand{\indent}{\hspace*{\tindent}}

%% User-defined Commands %%

\newcommand{\cop}[2]{% \cop{<state index>}{<spin index>}
	\ensuremath{ \hat{a} _{#1 #2} ^{\dagger} }}

\newcommand{\aop}[2]{% \aop{<state index>}{<spin index>}
	\ensuremath{ \hat{a} _{#1 #2} }}

\newcommand{\pcop}[1]{% \pcop{<state index>}
	\ensuremath{ \cop{#1}{+} \cop{#1}{-} } }

\newcommand{\paop}[1]{% \paop{<state index>}
	\ensuremath{ \aop{#1}{-} \aop{#1}{+} } }

\newcommand{\cpm}[1]{% \cpm{<state index>}
	\ensuremath{\cop{#1}{+} \cop{#1}{-} }}

\newcommand{\amp}[1]{% \apm{<state index>}
	\ensuremath{ \aop{#1}{-} \aop{#1}{+} } }

\newcommand{\sz}[2]{% \sz{<<state index>}{<spin index>>}
	\ensuremath{ \frac{1}{2} \sum_{#1 #2} #2 a _{#1 #2} ^{\dagger} a _{#1 #2} } }

\newcommand{\splus}[1]{% \splus{<state index>}
	\ensuremath{ \sum_{#1}  a _{#1 +} ^{\dagger} a _{#1 -} } }

\newcommand{\sminus}[1]{% \sminus{<state index>}
	\ensuremath{ \sum_{#1}  a _{#1 -} ^{\dagger} a _{#1 +} } }

\newcommand{\hzero}[2]{% \hzero{<state index>}{<spin index>}
	\ensuremath{ \sum_{#1 #2} (#1-1) a _{#1 #2} ^{\dagger} a _{#1 #2} }}

\newcommand{\hfull}[4]{% \hfull{<index 1>}{<index 2>}{<index 3>}{<index 4>}
	\ensuremath{\sum_{#1 #2} (#1 - 1) \cop{#1}{#2} \aop{#1}{#2} - g \sum_{#3 #4} \hat{P} ^+ _{#3} \hat{P} ^- _{#4}  } }

\newcommand{\twobody}[2]{% \twobody{<first index>}{<second index>}
	\ensuremath{ \sum_{#1 #2} (-g) \cop{#1}{+} \cop{#1}{-} \aop{#2}{-} \aop{#2}{+}  } }

\newcommand{\caop}[2]{% \caop{<1st indx>}{<2nd indx>}
	\ensuremath{ \cop{#1}{#2} \aop{#1}{#2} } }

\newcommand{\numop}[2]{% \numop{<state index>}{<spin index>}
	\ensuremath{ \hat{n} _{#1 #2} }}

\newcommand{\nop}[2]{% \nop{<state index>}{<spin index>}
	\ensuremath{ \cop{#1}{#2} \aop{#1}{#2}} }

\newcommand{\sop}[1]{% \sop{<z,+,->}
	\ensuremath{ \hat{S}_{#1} } }

\newcommand{\krondelt}[2]{% \krondelt{<idx 1>}{<idx 2>}
	\ensuremath{ \delta _{#1 #2} }}

\newcommand{\sopz}{
	\ensuremath{ \hat{S} _{z} } }

\newcommand{\sopp}{
	\ensuremath{ \hat{S} _{+} } }

\newcommand{\sopm}{
	\ensuremath{ \hat{S} _{-} } }

\newcommand{\hop}{
	\ensuremath{ \hat{H} _0 }}

\newcommand{\ssop}{
	\ensuremath{ \hat{S} ^2} }

\newcommand{\vop}{
	\ensuremath{ \hat{V} } }

\newcommand{\ppop}[1]{% \ppop{<index>}
	\ensuremath{ \hat{P} _{#1} ^+ } }

\newcommand{\pmop}[1]{% \ppop{<index>}
	\ensuremath{ \hat{P} _{#1} ^- } }

\newcommand{\commutator}[2]{% \commutator{<1st op>}{<2nd op>}}
	\ensuremath{ \left [ #1,#2 \right ] }}

\newcommand{\commutatorexp}[2]{% \commutatorexp{1st op}{<2nd op>}
	\ensuremath{ #1 #2 - #2 #1 } }

\newcommand{\ssqr}{% sum form of S^2
	\ensuremath{ \sopz ^2 + \frac{1}{2} ( \sopp \sopm + \sopm \sopp ) } }

\newcommand{\vacuumb}{
	\ensuremath{ \Bra{0} } }

\newcommand{\vacuumk}{
	\ensuremath{ \Ket{0} } }

\title{Pairing Model}
\author{Xingze Mao \\ Zachary Matheson \\ Thomas Redpath}
%\date{}

\begin{document}

\maketitle

\section*{\hop commutation relations}

Show that the unperturbed Hamiltonian commutes with the spin projection \sop{z}:

\begin{align*}
	\left [ H _0, \sop{z} \right ] &=\\
	&=\left ( \hzero{p}{\sigma} \right ) \left ( \sz{p'}{\sigma'} \right ) -\\
	& \hspace{5mm} \left ( \sz{p''}{\sigma''} \right ) \left ( \hzero{p'''}{\sigma'''} \right )\\
	&= \frac{1}{2} \sum_{p \sigma p' \sigma'} (p-1) \sigma' \numop{p}{\sigma} \numop{p'}{\sigma'} - \frac{1}{2} \sum_{p'' \sigma'' p''' \sigma'''} (p'''-1) \sigma'' \numop{p''}{\sigma''} \numop{p'''}{\sigma'''}
\end{align*}

In this form it is apparent that these two sums are identical and cancel to give $[\hat{H}_0,\sop{z}]=0$.

Show that the unperturbed Hamiltonian commutes with the total spin squared operator $\hat{S}^2$:

\begin{align}
	\left [ \hat{H}_0, \hat{S}^2 \right ] &= \left [ \hop, \sop{z} ^2 \right ] + \frac{1}{2} \left [ \hop,(\sop{+} \sop{-} + \sop{-}\sop{+}) \right ]\\
	&= \commutator{\hop}{\sop{z}} \sop{z} + \sop{z} \commutator{\hop}{\sop{z}} +\frac{1}{2} \commutator{\hop}{(\sop{+}\sop{-} + \sop{-}\sop{+})}\\
	&= \frac{1}{2} \left ( \commutator{\hop}{\sop{+}} \sop{-} + \sop{+} \commutator{\hop}{\sop{-}} + \commutator{\hop}{\sop{-}} \sop{+} + \sop{-} \commutator{\hop}{\sop{+}} \right )
\end{align}

Now consider the commutators \commutator{\hop}{\sop{+}} and \commutator{\hop}{\sop{-}}.

\begin{equation}
	\commutator{\hop}{\sop{+}} = \left ( \hzero{p}{\sigma} \right ) \left ( \splus{p'} \right ) - \left ( \splus{p''} \right ) \left ( \hzero{p'''}{\sigma'''} \right )
	\label{eq:hsplus}
\end{equation}

Using the anti-commutation relations of the creation and annihilation operators, we can show that the first product of sums in eq.~\ref{eq:hsplus} contains the second and therefore, $ \commutator{\hop}{\sop{+}} =0$. Focusing on the first product of sums in eq.~\ref{eq:hsplus}:

\begin{align}
	& \sum_{p \sigma p'} (p-1) \cop{p}{\sigma} \aop{p}{\sigma} \cop{p'}{+} \aop{p'}{-}\\
	&= \sum_{p \sigma p'} (p-1) \cop{p}{\sigma} \left (\krondelt{p}{p'} \krondelt{\sigma}{+} -  \cop{p'}{+} \aop{p}{\sigma} \right ) \aop{p'}{-}\\
	&= \sum_{p \sigma p'} (p-1) \left ( \krondelt{p}{p'} \krondelt{\sigma}{+} \cop{p}{\sigma} \aop{p'}{-} - \cop{p}{\sigma}  \cop{p'}{+} \aop{p}{\sigma} \aop{p'}{-} \right )\\
	&= \sum_{p \sigma p'} (p-1) \left ( \krondelt{p}{p'} \krondelt{\sigma}{+} \cop{p}{\sigma} \aop{p'}{-} - \cop{p'}{+} \cop{p}{\sigma} \aop{p'}{-} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma p'} (p-1) \left ( \krondelt{p}{p'} \krondelt{\sigma}{+} \cop{p}{\sigma} \aop{p'}{-} - \cop{p'}{+} \left ( \krondelt{p}{p'} \krondelt{\sigma}{-} - \aop{p'}{-} \cop{p}{\sigma} \right ) \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma p'} (p-1) \left ( \krondelt{p}{p'} \krondelt{\sigma}{+} \cop{p}{\sigma} \aop{p'}{-} - \krondelt{p}{p'} \krondelt{\sigma}{-} \cop{p'}{+} \aop{p}{\sigma} + \cop{p'}{+} \aop{p'}{-} \cop{p}{\sigma} \aop{p}{\sigma} \right )\\
%	&= \sum_{p}  \left ( (p-1) \cop{p}{+} \aop{p}{-} \right ) + \sum_{p} \left ( (p-1) \cop{p}{+} \aop{p}{-} \right ) + \sum_{p \sigma p'} (p-1) \left ( \cop{p'}{+} \aop{p'}{-} \cop{p}{\sigma} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma p'} (p-1) \left ( \cop{p'}{+} \aop{p'}{-} \cop{p}{\sigma} \aop{p}{\sigma} \right )
\end{align}

Comparing the results of this calculation with the second product of sums in eq.~\ref{eq:hsplus}, which, reordering indicies, we can write as

\begin{align*}
	& \sum_{p'' p''' \sigma} (p''' -1) \left ( \cop{p''}{+} \aop{p''}{-}\cop{p'''}{\sigma'''}\aop{p'''}{\sigma'''} \right )\\
	&= \sum_{p \sigma p' } (p' -1) \left ( \cop{p}{+} \aop{p}{-} \cop{p'}{\sigma'} \aop{p'}{\sigma'} \right )
\end{align*}

\noindent we notice that the two terms cancel leaving $ \commutator{\hop}{\sop{+}} =0$.

Using the same procedure, we can show that $ \commutator{\hop}{\sop{-}} = 0$. With these results, we find that $\commutator{\hop}{\ssop} = 0$.

\section*{\vop commutation relations}

Show that \vop commutes with \sop{z}:

\begin{align}
\begin{split}
	\commutator{\vop}{\sop{z}} &= \left ( \twobody{q}{s} \right ) \left ( \sz{p}{\sigma} \right )\\
	& - \left( \sz{p'}{\sigma'} \right ) \left ( \twobody{q'}{s'} \right )
\end{split}
\label{eq:comvsz}
\end{align}

The first term may be re-written:

\begin{align*}
	& \frac{(-g)}{2} \sum_{qs p \sigma} \sigma \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} \cop{p}{\sigma} \aop{p}{\sigma}\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \sigma \cop{q}{+} \cop{q}{-} \aop{s}{-} \left ( \krondelt{s}{p} \krondelt{+}{\sigma} - \cop{p}{\sigma} \aop{s}{+} \right ) \aop{p}{\sigma}\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \krondelt{s}{p} \krondelt{+}{\sigma} \sigma \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{p}{\sigma} - \sigma \cop{q}{+} \cop{q}{-} \aop{s}{-} \cop{p}{\sigma} \aop{s}{+} \aop{p}{\sigma} \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \krondelt{s}{p} \krondelt{+}{\sigma} \sigma \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{p}{\sigma} - \sigma \cop{q}{+} \cop{q}{-} \left ( \krondelt{s}{p} \krondelt{-}{\sigma} - \cop{p}{\sigma} \aop{s}{-} \right ) \aop{s}{+} \aop{p}{\sigma} \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \krondelt{s}{p} \krondelt{+}{\sigma} \sigma \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{p}{\sigma} - \krondelt{s}{p} \krondelt{-}{\sigma} \sigma \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{p}{\sigma} + \sigma \cop{q}{+} \cop{q}{-} \cop{p}{\sigma} \aop{s}{-} \aop{s}{+} \aop{p}{\sigma}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( (+) \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} -  (-) \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{-} + \sigma \cop{p}{\sigma} \cop{q}{+} \cop{q}{-} \aop{p}{\sigma} \aop{s}{-} \aop{s}{+}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{-} + \sigma \cop{p}{\sigma} \cop{q}{+} \left ( \krondelt{q}{p} \krondelt{-}{\sigma} - \aop{p}{\sigma} \cop{q}{-} \right ) \aop{s}{-} \aop{s}{+}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{-} + \krondelt{q}{p} \krondelt{-}{\sigma} \sigma \cop{p}{\sigma} \cop{q}{+} \aop{s}{-} \aop{s}{+} - \sigma \cop{p}{\sigma} \cop{q}{+} \aop{p}{\sigma} \cop{q}{-} \aop{s}{-} \aop{s}{+}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{-} + (-) \cop{q}{-} \cop{q}{+} \aop{s}{-} \aop{s}{+} - \sigma \cop{p}{\sigma} \cop{q}{+} \aop{p}{\sigma} \cop{q}{-} \aop{s}{-} \aop{s}{+}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{-} - \cop{q}{-} \cop{q}{+} \aop{s}{-} \aop{s}{+} - \sigma \cop{p}{\sigma} \left ( \krondelt{q}{p} \krondelt{+}{\sigma} - \aop{p}{\sigma} \cop{q}{+} \right ) \cop{q}{-} \aop{s}{-} \aop{s}{+}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{-} - \cop{q}{-} \cop{q}{+} \aop{s}{-} \aop{s}{+} - \krondelt{q}{p} \krondelt{+}{\sigma} \sigma \cop{p}{\sigma} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \sigma \cop{p}{\sigma} \aop{p}{\sigma} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{-} - \cop{q}{-} \cop{q}{+} \aop{s}{-} \aop{s}{+} - \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \sigma \cop{p}{\sigma} \aop{p}{\sigma} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}  \right )\\
	& \frac{(-g)}{2} \sum_{qs p \sigma} \left ( \sigma \cop{p}{\sigma} \aop{p}{\sigma} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}  \right )\\
\end{align*}

Comparing this result to the second product of sums in eq.~\ref{eq:comvsz}, which may be re-written:

\begin{equation}
	\frac{+g}{2} \sum_{p \sigma qs} \sigma \cop{p}{\sigma} \aop{p}{\sigma} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} \nonumber
\end{equation}

the terms cancel so that $ \commutator{\vop}{\sop{z}} = 0$.

Show that \vop commutes with \ssop:

\begin{align*}
	\commutator{\vop}{\ssop} &= \commutator{\vop}{\sop{z} ^2} + \frac{1}{2} \left ( \commutator{\vop}{\sop{+} \sop{-}} + \commutator{\vop}{\sop{-} \sop{+}} \right )\\
	&= \sop{z} \commutator{\vop}{\sop{z}} +  \commutator{\vop}{\sop{z}} \sop{z} + \frac{1}{2} \left ( \sop{+} \commutator{\vop}{\sop{-}} + \commutator{\vop}{\sop{+}} \sop{-} + \sop{-} \commutator{\vop}{\sop{+}} + \commutator{\vop}{\sop{-}} \sop{+} \right )\\
	&= \frac{1}{2} \left ( \sop{+} \commutator{\vop}{\sop{-}} + \commutator{\vop}{\sop{+}} \sop{-} + \sop{-} \commutator{\vop}{\sop{+}} + \commutator{\vop}{\sop{-}} \sop{+} \right )
\end{align*}

Writing out \commutator{\vop}{\sop{-}}:

\begin{align}
\begin{split}
	\commutator{\vop}{\sop{-}} &= \left ( \twobody{q}{s} \right ) \left ( \sminus{m} \right )\\
	& - \left ( \sminus{m'} \right ) \left ( \twobody{q'}{s'} \right )
\end{split}
\label{eq:vsminus}
\end{align}

Re-writing the first prduct of sums using the anti-commutation relations and noting that in \ref{eq:vsmmk1} and \ref{eq:vsmmk2} the first terms give zero for the fermionic case since \aop{s}{+} \aop{s}{+} would act to remove a particle from an empty state and \cop{q}{-} \cop{q}{-} would act to add a particle to an occupied state.

\begin{align}
	& (-g) \sum _{q s m} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} \cop{m}{-} \aop{m}{+}\\
	& (-g) \sum _{q s m} \cop{q}{+} \cop{q}{-} \aop{s}{-} \cop{m}{-} \aop{m}{+} \aop{s}{+}\\
	& (-g) \sum _{q s m} \cop{q}{+} \cop{q}{-} \left ( \krondelt{s}{m} - \cop{m}{-} \aop{s}{-} \right ) \aop{m}{+} \aop{s}{+}\\
	& (-g) \sum _{q s m} \krondelt{s}{m} \cop{q}{+} \cop{q}{-} \aop{m}{+} \aop{s}{+} - \cop{q}{+} \cop{q}{-} \cop{m}{-} \aop{s}{-} \aop{m}{+} \aop{s}{+}\\
\label{eq:vsmmk1}	& (-g) \sum _{q s m} \cop{q}{+} \cop{q}{-} \aop{s}{+} \aop{s}{+} - \cop{q}{+} \cop{q}{-} \cop{m}{-} \aop{s}{-} \aop{m}{+} \aop{s}{+}\\
	& (-g) \sum _{q s m} - \cop{m}{-} \cop{q}{+} \aop{m}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
	& (-g) \sum _{q s m} - \cop{m}{-} \left ( \krondelt{q}{m} - \aop{m}{+} \cop{q}{+} \right ) \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
	& (-g) \sum _{q s m} - \krondelt{q}{m}\cop{m}{-} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{m}{-}  \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
\label{eq:vsmmk2}	& (-g) \sum _{q s m} - \cop{q}{-} \cop{q}{-} \aop{s}{-} \aop{s}{+} + \cop{m}{-}  \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
\label{eq:vsmres}	& (-g) \sum _{q s m} \cop{m}{-}  \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}
\end{align}

Comparing the result in \ref{eq:vsmres} to the second product of sums in eq.~\ref{eq:vsminus}, re-written here as

\begin{equation}
	g \sum _{m q s} \cop{m}{-} \cop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} \nonumber
\end{equation}

we see that $ \commutator{\vop}{\sop{-}} = 0$. Using the same procedure, we can show that $ \commutator{\vop}{\sop{+}} = 0$ and conclude that $ \commutator{\vop}{\ssop} = 0$.

\section*{Pair creation and annihilation operators}

We consider the system with total spin $S=0$ (no broken pairs) and define the pair creation and annihilation operators.

\begin{align*}
\begin{split}
	\hat{P} _p ^+ &= \pcop{p}\\
	\hat{P} _p ^- &= \paop{p}
\end{split}
\end{align*}

and the full Hamiltonian

\begin{equation}
	\hat{H} = \hfull{p}{\sigma}{q}{s}
\label{eq:fullham}
\end{equation}

Show that $\hat{H}$ commutes with the product of the pair creation and pair annihilation operators:

%\begin{itemize}
%	\item The indices on the pair creation and annihilation operators must be the same since otherwise, this would imply physically chaning the system.
%\end{itemize}

\begin{align}\label{eq:comhpac}
	\commutator{\hat{H}}{ \ppop{r} \pmop{r}} &= \commutator{\hat{H} _0}{ \ppop{r} \pmop{r}} + \commutator{\vop}{\ppop{r} \pmop{r}}
\end{align}

Taking the first commutator in eq.~\ref{eq:comhpac}:

\begin{align*}
	\commutator{\hop}{ \ppop{r} \pmop{r}} &= \left ( \hzero{p}{\sigma} \right ) \left ( \pcop{r} \paop{r} \right ) - \left ( \pcop{r} \paop{r} \right ) \left ( \hzero{p}{\sigma} \right )
\end{align*}

Re-writing the first product of sums to have the same form as the second product of sums leaves no remaining terms (\ref{eq:comhzpcap}). Thus $ \commutator{\hop}{\ppop{r} \pmop{r}}=0$.

\begin{align}
\begin{split}
	& \left ( \hzero{p}{\sigma} \right ) \left ( \pcop{r} \paop{r} \right )\\
	&= \sum_{p \sigma} (p-1) \cop{p}{\sigma} \aop{p}{\sigma} \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+}\\
	&= \sum_{p \sigma} (p-1) \cop{p}{\sigma} \left ( \krondelt{p}{r} \krondelt{\sigma}{+} - \cop{r}{+} \aop{p}{\sigma} \right ) \cop{r}{-} \aop{r}{-} \aop{r}{+}\\
	&= \sum_{p \sigma} (p-1) \left ( \krondelt{p}{r} \krondelt{\sigma}{+} \cop{p}{\sigma} \cop{r}{-} \aop{r}{-} \aop{r}{+} - \cop{p}{\sigma} \cop{r}{+} \aop{p}{\sigma} \cop{r}{-} \aop{r}{-} \aop{r}{+} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} - \cop{p}{\sigma} \cop{r}{+} \aop{p}{\sigma} \cop{r}{-} \aop{r}{-} \aop{r}{+} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{p}{\sigma} \aop{p}{\sigma} \cop{r}{-} \aop{r}{-} \aop{r}{+} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{p}{\sigma} \left ( \krondelt{p}{r} \krondelt{\sigma}{-} - \cop{r}{-} \aop{p}{\sigma} \right ) \aop{r}{-} \aop{r}{+} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \krondelt{p}{r} \krondelt{\sigma}{-} \cop{r}{+} \cop{p}{\sigma} \aop{r}{-} \aop{r}{+} - \cop{r}{+} \cop{p}{\sigma} \cop{r}{-} \aop{p}{\sigma} \aop{r}{-} \aop{r}{+} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \cop{p}{\sigma} \aop{r}{-} \aop{r}{+} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \left ( \krondelt{p}{r} \krondelt{\sigma}{-} - \aop{r}{-} \cop{p}{\sigma} \right ) \aop{r}{+} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \krondelt{p}{r} \krondelt{\sigma}{-} \cop{r}{+} \cop{r}{-} \aop{r}{+} \aop{p}{\sigma} - \cop{r}{+} \cop{r}{-} \aop{r}{-} \cop{p}{\sigma} \aop{r}{+} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \aop{r}{+} \aop{r}{-} - \cop{r}{+} \cop{r}{-} \aop{r}{-} \cop{p}{\sigma} \aop{r}{+} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} - \cop{r}{+} \cop{r}{-} \aop{r}{-} \cop{p}{\sigma} \aop{r}{+} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} - \cop{r}{+} \cop{r}{-} \aop{r}{-} \left ( \krondelt{p}{r} \krondelt{\sigma}{+} - \aop{r}{+} \cop{p}{\sigma} \right ) \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} -\krondelt{p}{r} \krondelt{\sigma}{+} \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{p}{\sigma} + \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} \cop{p}{\sigma} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} - \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} + \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} \cop{p}{\sigma} \aop{p}{\sigma} \right )\\
	&= \sum_{p \sigma} (p-1) \left ( \cop{r}{+} \cop{r}{-} \aop{r}{-} \aop{r}{+} \cop{p}{\sigma} \aop{p}{\sigma} \right )
\end{split}
\label{eq:comhzpcap}
\end{align}

Working out the second commutator from eq.~\ref{eq:comhpac}:

\begin{align}
\begin{split}
	\commutator{\vop}{\ppop{r} \pmop{t}} &= (-g) \sum _{qs} \commutator{ \ppop{q} \pmop{s} }{ \ppop{r} \pmop{r}}\\
	&= \ppop{q} \commutator{\pmop{s}}{\ppop{r}} \pmop{r} + \ppop{q} \ppop{r} \commutator{\pmop{s}}{\pmop{r}} + \commutator{\ppop{q}}{\ppop{r}} \pmop{s} \pmop{r} +  \ppop{r} \commutator{\ppop{q}}{\pmop{r}} \pmop{s}
%&= \left ( -g \sum_{q s} \hat{P} ^+ _{q} \hat{P} ^- _{s} \right ) \left ( \ppop{r} \pmop{r} \right ) - \left ( \ppop{r} \pmop{r} \right ) \left ( -g \sum_{q s} \hat{P} ^+ _{q} \hat{P} ^- _{s} \right )
\end{split}
\label{eq:ctpcom}
\end{align}

The commutation relations between pair creation and annihilation operators were found to be:

\begin{align*}
	\commutator{\ppop{p}}{\pmop{q}} &= \cop{p}{+} \cop{p}{-} \aop{q}{-} \aop{q}{+} - \aop{q}{-} \aop{q}{+} \cop{p}{+} \cop{p}{-}
\end{align*}

Re-writing the first term

\begin{align*}
	& \cop{p}{+} \cop{p}{-} \aop{q}{-} \aop{q}{+}\\
	& \cop{p}{+} \left ( \krondelt{p}{q} - \aop{q}{-} \cop{p}{-} \right ) \aop{q}{+}\\
	& \cop{p}{+} \aop{p}{+} - \cop{p}{+} \aop{q}{-} \cop{p}{-} \aop{q}{+}\\
	& \cop{p}{+} \aop{p}{+} - \aop{q}{-} \cop{p}{+} \aop{q}{+} \cop{p}{-}\\
	& \cop{p}{+} \aop{p}{+} - \aop{q}{-} \left ( \krondelt{p}{q} - \aop{q}{+} \cop{p}{+} \right ) \cop{p}{-}\\
	& \cop{p}{+} \aop{p}{+} -  \aop{p}{-} \cop{p}{-} + \aop{q}{-} \aop{q}{+} \cop{p}{+} \cop{p}{-}\\
\end{align*}

cancels the second leaving

\begin{equation}
	\commutator{\ppop{p}}{\pmop{q}} = \cop{p}{+} \aop{p}{+} -  \aop{p}{-} \cop{p}{-} \nonumber
\end{equation}

Generalizing the form for the commutation relation between \ppop{p} and \pmop{q}

\begin{align}
\begin{split}
	\commutator{\hat{P} _p ^{\pm} }{\hat{P} _q ^{\mp}} &= \pm ( \cop{p}{+} \aop{p}{+} - \aop{p}{-} \cop{p}{-})\\
	\commutator{\hat{P} _p ^{\pm}}{\hat{P} _q ^{\pm}} &= 0
\end{split}
\label{eq:ppmcom}
\end{align}

These results reduce the four terms in eq.~\ref{eq:ctpcom} to two, which can be expanded taking into account that the pair creation and annihilation operators act to select terms with their specific state index from the sum over all states.

\begin{align*}
	&\ppop{q} \commutator{\pmop{s}}{\ppop{r}} \pmop{r} + \ppop{q} \ppop{r} \commutator{\pmop{s}}{\pmop{r}} + \commutator{\ppop{q}}{\ppop{r}} \pmop{s} \pmop{r} +  \ppop{r} \commutator{\ppop{q}}{\pmop{r}} \pmop{s}\\
	&= \ppop{q} \commutator{\pmop{s}}{\ppop{r}} \pmop{r} +  \ppop{r} \commutator{\ppop{q}}{\pmop{r}} \pmop{s}\\
	&= \pcop{q} \left ( \aop{r}{-} \cop{r}{-} - \cop{r}{+} \aop{r}{+} \right ) \paop{r} + \pcop{r} \left ( \cop{r}{+} \aop{r}{-} - \aop{r}{-} \cop{r}{-} \right ) \paop{s}\\
	&= 0
\end{align*}

since the first term contains two \aop{r}{-} operators, the second term contains two \aop{r}{+} operators, the third contains two \cop{r}{+} operators and the fourth contains two \cop{r}{+} operators, therefore, action of \commutator{\vop}{\ppop{r} \pmop{t}} on any fermionic state produces 0. Combining this result with that of eq.~\ref{eq:comhzpcap} shows that the Hamiltonian commutes with the pair creation and annihilation operators.

\section*{The Hamiltonian Matrix}

Restricting the effective Hilbert space to only the two lowest single-particle states (they are doubly degenerate) and considering only two particles, we can construct a Hamiltonian matrix in the basis of 2-particle Slater determinants (eq.~\ref{eq:2ptclSD}).

\begin{align}
\begin{split}
	\Ket{\Phi _1 ^{\mathrm{SD}}} &= \pcop{1} \ket{0}\\
	\Ket{\Phi _2 ^{\mathrm{SD}}} &= \pcop{2} \ket{0}
\end{split}
	\label{eq:2ptclSD}
\end{align}

 Furthermore, our system is considered to have no broken pairs and total spin $S=0$. The matrix elements of the Hamiltonian can be calculated separately for the one- and two-body parts using Wick's theorem. For example, the one-body part of the first matrix element:

\begin{align*}
	& \Bra{0} \paop{1} \left ( \hzero{p}{\sigma} \right ) \pcop{1} \Ket{0}\\
	&= \Bra{0} \paop{1} \left ( 0 \nop{1}{+} + 0 \nop{1}{-} + 1 \nop{2}{+} + 1 \nop{2}{-} \right ) \pcop{1} \Ket{0}\\
	&= \Bra{0} \paop{1} \left ( 0 \nop{1}{+} + 0 \nop{1}{-} \right ) \pcop{1} \Ket{0}\\
	&= 0
\end{align*}

The two body part of the first matrix element:

\begin{align*}
	& \vacuumb \paop{1} \left ( \twobody{p}{q} \right ) \pcop{1} \vacuumk\\
	&= \vacuumb \paop{1} (-g) \left ( \cpm{1} \amp{1} + \cpm{1} \amp{2} + \cpm{2} \amp{1} + \cpm{2} \amp{2} \right ) \pcop{1} \vacuumk\\
	&= \vacuumb \paop{1} (-g) \left ( \cpm{1} \amp{1} \right ) \pcop{1} \vacuumk\\
	&= -g
\end{align*}

The second matrix element:

\begin{align*}
	& \vacuumb \paop{1} \left ( \hzero{p}{\sigma} \right ) \pcop{2} \vacuumk\\
	&= \vacuumb \paop{1} \left ( 0 \nop{1}{+} + 0 \nop{1}{-} + 1 \nop{2}{+} + 1 \nop{2}{-} \right ) \pcop{2} \vacuumk\\
	&= 0
\end{align*}

\begin{align*}
	& \vacuumb \paop{1} \left ( \twobody{p}{q} \right ) \pcop{2} \vacuumk\\
	&= \vacuumb \paop{1} (-g) \left ( \cpm{1} \amp{1} + \cpm{1} \amp{2} + \cpm{2} \amp{1} + \cpm{2} \amp{2} \right ) \pcop{2} \vacuumk\\
	&= \vacuumb \paop{1} (-g) \left ( \cpm{1} \amp{2} \right ) \pcop{2} \vacuumk\\
	&= -g
\end{align*}

The full $2 \times 2$ matrix where $d$ is the level spacing

\begin{equation}
	H =
	\begin{pmatrix}
		-g & -g\\
		-g & 2d - g
	\end{pmatrix}
	\label{eq:2ptclham}
\end{equation}

This matrix is diagonalized to yield eigenenergies (with $d=1$)

\begin{equation}
	E = 1 - g \pm \sqrt{1 + g^2}
	\label{eq:2ptcleng}
\end{equation}

The ground state energy, $E_0 = 1 - g - \sqrt{1 + g^2}$, as $g$ goes from $-1$ to $1$ goes from $\sim 0.5$ to $\sim -1.5$. This suggests that when the two-body interaction is attractive ($g$ positive), the one pair two level system has at least one bound state. Our shell model code (see Table~\ref{tab:1pr2lvl}) reproduces exactly the analytic result.

When we have four levels and four particles, the number of basis states is $\binom{4}{4/2}=6$, and the 6 Slater determinants are

\begin{align}
\begin{split}
	& \cpm{1} \cpm{2} \vacuumk\\
	& \cpm{1} \cpm{3} \vacuumk\\
	& \cpm{1} \cpm{4} \vacuumk\\
	& \cpm{2} \cpm{3} \vacuumk\\
	& \cpm{2} \cpm{4} \vacuumk\\
	& \cpm{3} \cpm{4} \vacuumk
\end{split}
\end{align}

%$\ket{1\pm2\pm},\ket{1\pm3\pm},\ket{1\pm4\pm},\ket{2\pm3\pm},\ket{2\pm4\pm},\ket{3\pm4\pm}$.

Action of the Hamiltonian on each Slater determinant results in

\begin{align*}
&\hat{H}\ket{1\pm2\pm}=(2-2g)\ket{1\pm2\pm}-g\ket{1\pm3\pm}-g\ket{1\pm4\pm}-g\ket{2\pm3\pm}-g\ket{2\pm4\pm}+0\ket{3\pm4\pm}   \\
&\hat{H}\ket{1\pm3\pm}=g\ket{1\pm2\pm}+(4-2g)\ket{1\pm3\pm}-g\ket{1\pm4\pm}-g\ket{2\pm3\pm}+0\ket{2\pm4\pm}-g\ket{3\pm4\pm}   \\
&\hat{H}\ket{1\pm4\pm}=-g\ket{1\pm2\pm}-g\ket{1\pm3\pm}+(6-2g)\ket{1\pm4\pm}-0\ket{2\pm3\pm}-g\ket{2\pm4\pm}-g\ket{3\pm4\pm}   \\
&\hat{H}\ket{2\pm3\pm}=-g\ket{1\pm2\pm}-g\ket{1\pm3\pm}-g\ket{1\pm4\pm}+(6-2g)\ket{2\pm3\pm}-g\ket{2\pm4\pm}-g\ket{3\pm4\pm}   \\
&\hat{H}\ket{2\pm4\pm}=-g\ket{1\pm2\pm}+0\ket{1\pm3\pm}-g\ket{1\pm4\pm}-g\ket{2\pm3\pm}+(8-2g)\ket{2\pm4\pm}-g\ket{3\pm4\pm}   \\
&\hat{H}\ket{3\pm4\pm}=0\ket{1\pm2\pm}-g\ket{1\pm3\pm}-g\ket{1\pm4\pm}-g\ket{2\pm3\pm}-g\ket{2\pm4\pm}+(10-2g)\ket{3\pm4\pm}
\end{align*}

where a short-hand notation $\ket{p \pm q \pm}$ is employed to represent a Slater determinant with one pair in level $p$ and one in level $q$.

So the Hamiltonian matrix is

\begin{equation}
\begin{bmatrix}
2-2g   &  -g   &   -g   &   -g    &   -g    &    0    \\
  -g   &4-2g   &   -g   &   -g    &    0    &   -g    \\
  -g   &  -g   & 6-2g   &    0    &   -g    &   -g    \\
  -g   &  -g   &   0   & 6-2g    &   -g    &   -g    \\
  -g   &   0   &   -g   &   -g    & 8-2g    &   -g    \\
   0   &  -g   &   -g   &   -g    &   -g    & 10-2g    \\
\end{bmatrix}
\label{eq:ham6mtx}
\end{equation}

We diagonalize this matrix numerically for $g=\{ -1.0, -0.5, 0.0, 0.5, 1.0 \}$ and ensure that our code's output, listed in Table~\ref{tab:2pr4lvl}, matches the results. This test ensures that our code correctly generates the Hamiltonian matrix. Similarly to the one pair two level case, we note that as $g$ becomes negative (as the two-body interaction becomes repulsive), the system becomes unbound (see Figure~\ref{fig:ezerovg}).

\begin{figure}
\center
	\includegraphics[width=0.6\textwidth]{lvl4pr2.png}
	\caption{Ground state energies for five different values of $g$ - calculated by diagonalizing the $6 \times 6$ Hamiltonian matrix of eq.~\ref{eq:ham6mtx}. These results match our code's output given in Table~\ref{tab:2pr4lvl}.}
	\label{fig:ezerovg}
\end{figure}

%Table~\ref{tab:1pr2lvl} lists the results of our shell model calculation with two particles (1 pair) and two levels.

\begin{table}[h]
\begin{center}
\begin{tabular}{|r|r|}
\hline
  g   &  $E_{gs}$ \\ \hline
-1.0  &  0.59 \\
-0.5  &  0.38 \\
 0.0  &  0.0 \\
 0.5  &  -0.62 \\
 1.0  &  -1.41 \\
\hline
\end{tabular}
\end{center}
\caption{Ground state energies as a function of interaction strength $g$ for the case of one pair in 2 single particle levels.}
\label{tab:1pr2lvl}
\end{table}

\begin{table}[h]
\begin{center}
	\begin{tabular}{|r|r|}
\hline
	g   &  $E_{gs}$ \\ \hline
	-1.0  &  3.30 \\
	-0.5  &  2.78 \\
	 0.0  &  2.0 \\
	 0.5  &  -0.64 \\
	 1.0  &  -1.49 \\
\hline
	\end{tabular}
\end{center}
	\caption{Ground state energies as a function of interaction strength $g$ for the case of two pairs in four levels.}
	\label{tab:2pr4lvl}
\end{table}

When we remove the single particle piece of the Hamiltonian and keep only
the two particle interaction, all particles are brought down to the same
degenerate energy level, with degeneracy $\Omega$ and ground state energy
\begin{equation}
	E_0 = - \frac{g}{4} n (\Omega - n +2)
	\label{eq:no1bdy}
\end{equation}

Our code reproduces the analytic result eq.~\ref{eq:no1bdy} for $g>0$ (an attractive two-body interaction). When
$g$ becomes negative, eq.~\ref{eq:no1bdy} gives the highest possible energy level due to the two-body interaction
and the ground state energy becomes 0.

As we then vary the
interaction strength $g$ we observe the following pattern: In the
negative $g$ case, the interaction is repulsive, and so the best the
system can do is break even. In this case, the ground state energy is
zero, regardless of $g$. When $g$ becomes positive, however, the
interaction is attractive, and as $g$ becomes more and more positive,
the interaction becomes increasingly attractive and the system
correspondingly becomes more and more bound. We illustrate this for
several cases in Table \ref{tab:data-nosinglepart}.

%%% Tables
\begin{table}
\begin{center}
%\label{tab:data-nosinglepart}
\begin{tabular}{|c|c|c|c|}
\hline
   g   &  $E_{gs}$(2,2)  &  $E_{gs}$(6,6)  & $ E_{gs}$(8,8) \\ \hline
-1.0  &  0  &  0  &  0   \\
-0.5  &  0  &  0  &  0   \\
  0.0  &  0  &  0  &  0   \\
  0.5  &  -1  & -6  &  -10 \\
  1.0  &  -2  & -12 &  -20 \\
\hline
\end{tabular}
\end{center}
\caption{Ground state energies $E_{gs}(N_l,N_{part})$ for the case with
the single-particle Hamiltonian removed. The energies are shown as a
function of interaction strength $g$ for the case of $N_{part}$
particles in $N_l=\frac{\Omega}{2}$ single particle ``levels.''}
\label{tab:data-nosinglepart}
\end{table}

\clearpage

\definecolor{mygreen}{rgb}{0,0.6,0}
\definecolor{mygray}{rgb}{0.5,0.5,0.5}
\definecolor{mymauve}{rgb}{0.58,0,0.82}

\lstset{ %
  backgroundcolor=\color{white},   % choose the background color; you must add \usepackage{color} or \usepackage{xcolor}
  basicstyle=\footnotesize,        % the size of the fonts that are used for the code
  breakatwhitespace=false,         % sets if automatic breaks should only happen at whitespace
  breaklines=true,                 % sets automatic line breaking
  captionpos=b,                    % sets the caption-position to bottom
  commentstyle=\color{mygreen},    % comment style
  deletekeywords={...},            % if you want to delete keywords from the given language
  escapeinside={\%*}{*)},          % if you want to add LaTeX within your code
  extendedchars=true,              % lets you use non-ASCII characters; for 8-bits encodings only, does not work with UTF-8
  frame=single,	                   % adds a frame around the code
  keepspaces=true,                 % keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
  keywordstyle=\color{blue},       % keyword style
  language=Octave,                 % the language of the code
  otherkeywords={*,...},           % if you want to add more keywords to the set
  numbers=left,                    % where to put the line-numbers; possible values are (none, left, right)
  numbersep=5pt,                   % how far the line-numbers are from the code
  numberstyle=\tiny\color{mygray}, % the style that is used for the line-numbers
  rulecolor=\color{black},         % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. comments (green here))
  showspaces=false,                % show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
  showstringspaces=false,          % underline spaces within strings only
  showtabs=false,                  % show tabs within strings adding particular underscores
  stepnumber=2,                    % the step between two line-numbers. If it's 1, each line will be numbered
  stringstyle=\color{mymauve},     % string literal style
  tabsize=2,	                   % sets default tabsize to 2 spaces
  title=\lstname                   % show the filename of files included with \lstinputlisting; also try caption instead of title
}

\lstinputlisting{shell_model.py}

%	&= \ppop{q} \left ( \commutatorexp{\pmop{s}}{\ppop{r}} \right ) \pmop{r} + \ppop{r} \left ( \commutatorexp{\ppop{q}}{\pmop{r}} \right ) \pmop{s}\\
%	&= \pcop{q} \left ( \commutatorexp{\paop{s}}{\pcop{r}} \right ) \paop{r} + \pcop{r} \left ( \commutatorexp{\pcop{q}}{\paop{r}} \right ) \paop{s}\\
%	&= \pcop{q} \paop{s} \pcop{r} \paop{r} - \pcop{q} \pcop{r} \paop{s} \paop{r}\\
%	& \hspace{4mm} + \pcop{r} \pcop{q} \paop{r} \paop{s} - \pcop{r} \paop{r} \pcop{q} \paop{s}\\

\begin{comment}

%%% SIDE TRACK - UNNECESSARY %%%

\begin{equation}
	\commutator{\hop}{\sop{-}} = 2 \sum_{p}  \left ( (p-1) \cop{p}{-} \aop{p}{+} \right )
	\label{eq:hsminusres}
\end{equation}

We can now re-write the remaining terms in $ \left [ \hat{H}_0, \hat{S}^2 \right ]$.

\begin{align*}
	\commutator{\hop}{\sop{+}} \sop{-} &= 2 \sum _{pp'} (p-1) \cop{p}{+} \aop{p}{-} \cop{p'}{-} \aop{p'}{+}\\
	\sop{+} \commutator{\hop}{\sop{-}} &= 2 \sum _{pp'} (p'-1) \cop{p}{+} \aop{p}{-} \cop{p'}{-} \aop{p'}{+}\\
	\commutator{\hop}{\sop{-}} \sop{+} &= 2 \sum _{pp'} (p-1) \cop{p}{-} \aop{p}{+} \cop{p'}{+} \aop{p'}{-}\\
	\sop{-} \commutator{\hop}{\sop{+}} &= 2 \sum _{pp'} (p'-1) \cop{p}{-} \aop{p}{+} \cop{p'}{+} \aop{p'}{-}
\end{align*}

Taking \commutator{\hop}{\sop{+}} \sop{-}:

\begin{align*}
	\commutator{\hop}{\sop{+}} \sop{-} &= \sum _{pp'} (p-1) \cop{p}{+} \aop{p}{-} \cop{p'}{-} \aop{p'}{+}\\
	&= \sum _{pp'} (p-1) \cop{p}{+} \left ( \krondelt{p}{p'} -  \cop{p'}{-} \aop{p}{-} \right ) \aop{p'}{+}\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{+} \aop{p'}{+} -  \cop{p}{+} \cop{p'}{-} \aop{p}{-} \aop{p'}{+} \right )\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{+} \aop{p'}{+} -  \cop{p'}{-} \cop{p}{+} \aop{p'}{+} \aop{p}{-} \right )\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{+} \aop{p'}{+} -  \cop{p'}{-} \left ( \krondelt{p}{p'} - \aop{p'}{+} \cop{p}{+} \right ) \aop{p}{-} \right )\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{+} \aop{p'}{+} - \krondelt{p}{p'} \cop{p'}{-} \aop{p}{-} - \cop{p'}{-} \aop{p'}{+} \cop{p}{+} \aop{p}{-} \right )\\
	&= \sum _{p} (p-1) \cop{p}{+} \aop{p}{+} - \sum _{p} (p-1) \cop{p}{-} \aop{p}{-} - \sum _{pp'} (p-1) \cop{p'}{-} \aop{p'}{+} \cop{p}{+} \aop{p}{-}\\
	&= \sum _{p} (p-1) \numop{p}{+} - \sum _{p} (p-1) \numop{p}{-} - \sum _{pp'} (p-1) \cop{p'}{-} \aop{p'}{+} \cop{p}{+} \aop{p}{-}\\
\end{align*}

Taking \commutator{\hop}{\sop{-}} \sop{+}:

\begin{align*}
	\commutator{\hop}{\sop{-}} \sop{+} &= \sum _{pp'} (p-1) \cop{p}{-} \aop{p}{+} \cop{p'}{+} \aop{p'}{-}\\
	&= \sum _{pp'} (p-1) \cop{p}{-} \left ( \krondelt{p}{p'} - \cop{p'}{+} \aop{p}{+} \right ) \aop{p'}{-}\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{-} \aop{p'}{-} - \cop{p}{-} \cop{p'}{+} \aop{p}{+} \aop{p'}{-} \right )\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{-} \aop{p'}{-} -  \cop{p'}{+} \cop{p}{-} \aop{p'}{-} \aop{p}{+} \right )\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{-} \aop{p'}{-} -  \cop{p'}{+} \left ( \krondelt{p}{p'} - \aop{p'}{-} \cop{p}{-} \right ) \aop{p}{+} \right )\\
	&= \sum _{pp'} (p-1) \left ( \krondelt{p}{p'} \cop{p}{-} \aop{p'}{-} - \krondelt{p}{p'} \cop{p'}{+} \aop{p}{+} - \cop{p'}{+} \aop{p'}{-} \cop{p}{-} \aop{p}{+} \right )\\
	&= \sum _{p} (p-1) \cop{p}{-} \aop{p}{-} - \sum _{p} (p-1) \cop{p}{+} \aop{p}{+} - \sum _{pp'} (p-1) \cop{p'}{+} \aop{p'}{-} \cop{p}{-} \aop{p}{+}\\
	&= \sum _{p} (p-1) \numop{p}{-} - \sum _{p} (p-1) \numop{p}{+} - \sum _{pp'} (p-1) \cop{p'}{+} \aop{p'}{-} \cop{p}{-} \aop{p}{+}\\
\end{align*}

Plugging these results in for their respective terms in the the remnant of $ \left [ \hat{H}_0, \hat{S}^2 \right ]$:

\begin{align*}
	& \left [ \hat{H}_0, \hat{S}^2 \right ]\\
	&= \frac{1}{2} \left ( \commutator{\hop}{\sop{+}} \sop{-} + \sop{+} \commutator{\hop}{\sop{-}} + \commutator{\hop}{\sop{-}} \sop{+} + \sop{-} \commutator{\hop}{\sop{+}} \right )\\
	&= \sum _{p} (p-1) \numop{p}{+} - \sum _{p} (p-1) \numop{p}{-} - \sum _{pp'} (p-1) \cop{p'}{-} \aop{p'}{+} \cop{p}{+} \aop{p}{-}\\
	& + \sum _{pp'} (p'-1) \cop{p}{+} \aop{p}{-} \cop{p'}{-} \aop{p'}{+}\\
	& + \sum _{p} (p-1) \numop{p}{-} - \sum _{p} (p-1) \numop{p}{+} - \sum _{pp'} (p-1) \cop{p'}{+} \aop{p'}{-} \cop{p}{-} \aop{p}{+}\\
	& + \sum _{pp'} (p'-1) \cop{p}{-} \aop{p}{+} \cop{p'}{+} \aop{p'}{-}
\end{align*}

%%% ALTERNATIVE FOR [V,SS] = 0 %%%

Writing the term \sop{-} \commutator{\vop}{\sop{+}} explicitly:

\begin{align}
\begin{split}
	\sop{-} \commutator{\vop}{\sop{+}} &= \left ( \sminus{m} \right ) \left ( \twobody{q}{s} \right ) \left ( \splus{p} \right )\\
	& - \left ( \sminus{m'} \right ) \left ( \splus{p'} \right ) \left ( \twobody{q'}{s'} \right )
\end{split}
\label{eq:svs}
\end{align}

The first product of sums may be re-written as follows, noting that in \ref{eq:svsmk1} and \ref{eq:svsmk2} the first terms are dropped since \aop{s}{-} \aop{s}{-} and \cop{q}{+} \cop{q}{+} acting on any state will give 0.

\begin{align}
	& (-g) \sum_{m q s p} \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} \cop{p}{+} \aop{p}{-}\\
	& (-g) \sum_{m q s p} \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \left ( \krondelt{s}{p} -  \cop{p}{+} \aop{s}{+} \right ) \aop{p}{-}\\
	& (-g) \sum_{m q s p} \krondelt{s}{p} \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{p}{-} - \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-}  \cop{p}{+} \aop{s}{+} \aop{p}{-}\\
\label{eq:svsmk1}	& (-g) \sum_{m q s p} \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{-} - \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-}  \cop{p}{+} \aop{s}{+} \aop{p}{-}\\
	& (-g) \sum_{m q s p}  - \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-}  \cop{p}{+} \aop{s}{+} \aop{p}{-}\\
	& (-g) \sum_{m q s p}  \cop{m}{-} \aop{m}{+} \cop{p}{+} \cop{q}{+} \cop{q}{-} \aop{p}{-} \aop{s}{-} \aop{s}{+}\\
	& (-g) \sum_{m q s p}  \cop{m}{-} \aop{m}{+} \cop{p}{+} \cop{q}{+} \left ( \krondelt{q}{p} -  \aop{p}{-} \cop{q}{-} \right ) \aop{s}{-} \aop{s}{+}\\
	& (-g) \sum_{m q s p} \krondelt{q}{p} \cop{m}{-} \aop{m}{+} \cop{p}{+} \cop{q}{+} \aop{s}{-} \aop{s}{+} - \cop{m}{-} \aop{m}{+} \cop{p}{+} \cop{q}{+}  \aop{p}{-} \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
\label{eq:svsmk2}	& (-g) \sum_{m q s p} \cop{m}{-} \aop{m}{+} \cop{q}{+} \cop{q}{+} \aop{s}{-} \aop{s}{+} - \cop{m}{+} \aop{m}{+} \cop{p}{+} \cop{q}{+}  \aop{p}{-} \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
	& (-g) \sum_{m q s p} - \cop{m}{-} \aop{m}{+} \cop{p}{+} \cop{q}{+}  \aop{p}{-} \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
	& (-g) \sum_{m q s p} - \cop{m}{-} \aop{m}{+} \aop{p}{-} \cop{p}{+} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}\\
\label{eq:svsmk3}	& (-g) \sum_{m q s p} \cop{m}{-} \aop{m}{+} \cop{p}{+} \aop{p}{-} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+}
\end{align}

Comparing the result from \ref{eq:svsmk3} to the second product of sums in eq.~\ref{eq:svs} which we re-write here as

\begin{equation}
	g \sum _{m p qs} \cop{m}{-} \aop{m}{+} \cop{p}{+} \aop{p}{-} \cop{q}{+} \cop{q}{-} \aop{s}{-} \aop{s}{+} \nonumber
\end{equation}

we find that $ \sop{-} \commutator{\vop}{\sop{+}} =0$

\end{comment}

\end{document}


\documentclass[11pt]{scrartcl}
\usepackage[sexy]{evan}
\usepackage{braket}
\usepackage{color}   %May be necessary if you want to color links
\usepackage{hyperref}
\usepackage{tikz}
\usepackage[compat=1.1.0]{tikz-feynman}
\usepackage{comment}

\DeclareOldFontCommand{\bf}{\normalfont\bfseries}{\mathbf}

\hypersetup{
	colorlinks=true, %set true if you want colored links
	linktoc=all,     %set to all if you want both sections and subsections linked
	linkcolor=blue,  %choose some color if you want links to stand out
}
\usepackage{geometry}
%\usepackage{showframe} %This line can be used to clearly show the new margins

\newgeometry{vmargin={30mm}, hmargin={20mm,20mm}}   % set the margins
\begin{document}
	\title{Digital Communication Theory} % Beginner
	\date{December 2021}
	\maketitle

	\begin{abstract}
		\sffamily\small
		Here is a collected notes on Digital Communications.
	\end{abstract}

	\vspace{1em}

	\tableofcontents
	\newpage

\section{Resources}
\begin{itemize}
	\item \emph{Thomas Cover, Elements of Information Theory}: Anything entropy, shannon capacity, refer to this.

	\item \emph{Cioffi notes}:   Each chapter covers the equivalent of a small book.  However lots of practical information nowhere else to be found (especially on equalization).
	\item \emph{Bane, Vasic}: Surprisingly good chapters on partial response channel capacity, coding (both RS and LDPC), and timing  recovery.
	\end{itemize}

\section{Foundations}

\subsection{Information Theory}

\textbf{Entropy, facts and definitions}:
\begin{itemize}
	\item Denote the \vocab{entropy} of random variable X with probability distribution $p(x)$ to be:
	\[ H(X) = - \braket{ \log (p(x)) } \]
	\item For a continuous random variable $X$ with density $f(x)$, the \vocab{differential entropy} $h(X)$ is denoted to be:
	\[h(X) = - \int_S f(x) \log f(x) dx \]
	where S is the support of X.
	\item Similarly denote the \vocab{joint entropy} for 2 random variables X, Y with distributions $p(x, y)$ to be:
    \[ H(X, Y) = -\sum_{x, y} p(x, y) \log(p(x, y)) = - \braket{ \log(p(x, y)) } \]
	\item  Denote the \vocab{conditional entropy} of $H(Y | X)$ to be:
	\[ H(Y| X) = \braket{ \log( p(y|x)) } \]
	\[ \sum_{x \in \mathcal{X}} p(x) \sum_{y \in \mathcal{Y}} p(y|x) \log(p(y | x)) \]
	\[ \sum_{x, y} p(x, y) \log(p(y|x)) \]
	\item \vocab{Chain rule}:
	\[H(X, Y) = H(X) + H(Y|X) \]
	\item The \vocab{kullback leibler} distance or \emph{relative entropy} between 2 probability mass distributions p(x) and q(x) is defined as:
	\[D(p || q) = \sum_x p(x) \log {p(x) \over q(x)} = \braket{\log {p(x) \over q(x)}} \]
	It is always non-negative, but it is not symmetric.
	\item  The \vocab{mutual information } I(X, Y) is the relative entropy between the joint and product distribution:
	\[ I(X, Y) = \sum_{x, y} p(x, y) \log {p(x, y) \over p(x) p(y)} \]
	\end{itemize}

\begin{example}
	\begin{itemize}
		\item The differential entropy for uniform $x \in [0, a]$ is:
		\[h(X) - \int_0^a  dx {1 \over a} \log({1 \over a}) = \log(a) \]
		\item The differential entropy for gaussian variable with
		$f(x) = {1 \over \sqrt{2 \pi \sigma^2}} \exp \left( -{x^2 \over 2 \sigma^2}\right)$ is:
		\[h(X) = \frac12 \log (2 \pi e \sigma^2)\]
		\end{itemize}
	\end{example}
\begin{lemma}
	The gaussian distribution between 2 random variable x, y maximizes the differential entropy for any given positive semi-definite autocorrelation $R_{x,y}$:
	\[ p_{x, y} = {1 \over \sqrt{\pi \det(R_{xy})} } \exp \left(- {\mathbf{u}}^T R_{xy}^{-1} \underbrace{\mathbf{u}}_{\equiv (x, y)} \right) \]
	\end{lemma}
\includegraphics{entropy1.png}

\begin{definition}
	A statistic t(X) of some random variable $X$ is \vocab{sufficient} \emph{for underlying parameter $\theta$}  if the conditional probability distribution of the data $X$ given t(X) does not depend on $\theta$.
	It means $I(X, \theta) = I(t(X), \theta)$
\end{definition}

\begin{definition}
	\end{definition}

\begin{theorem}
	Consider a signal $Y(t) = X(t) + N(t)$ where $X \subseteq \mathcal{L}_2$ with an orthonormal basis $ \mathcal{S} \{  \phi_k \}$.  Denote N(t) to be a white gaussian noise process with respect to $\mathcal{S}$.  Then the set of measurements $\braket{ \phi_k | X}$ form a set of sufficient statistics for detection of X(t) from Y(t).
	\end{theorem}

\begin{proof} See 6.451 notes, section 2.4
	\end{proof}

\subsection{Channel Capacity}
cioffic ch. 2

\begin{itemize}
	\item The \vocab{channel capacity} in bits/subsymbol for a channel described by $p(y|x)$ is defined by
	\[  \mathcal{C} = \mathrm{max}_{p(x)} I(x, y) \text{ } bits/subsymbol \]
	\item A slightly more fancy way to describe capacity is to describe the maximum mutual information with respect to a transmit and receive sequence of subsymbols: $\bf{x}^n \equiv (x_1, x_2, ..., x_{n})$ and $\bf{y}^n = (y_1, ..., y_n)$.
	\[ C = \lim_{n \rightarrow \infty} {1 \over n} \mathrm{max}_{p(\bf{x}^n)} I(\bf{x}^n, \bf{y}^n) \]
	The maximization $p(\bf{x}^n)$ is taken over all probability density functions $p(\bf{x}^n)$ which satisfy the symbol energy constraint given by
	\[ \braket{x_k^2} \leq E_s \]
	\item An \vocab{Additive White Gaussian Noise} channel is a channel where
	\[y(t) = x(t) + n(t) \]
	where $n(t)$ is white.
\end{itemize}

\begin{theorem}
	Given a channel with capacity $\mathcal{C}$, then there exists a code with bitrate $b < \mathcal{C}$ such that
	$P_e \leq \delta$ for any $ \delta > 0$.  Furthermore, if $b > \mathcal{C}$, then $P_e  \geq \text{positive constant}$, which is typically large even for b slightly greater than $\mathcal{C}$.
	\end{theorem}

\begin{theorem}
	Given an AWGN channel, the channel capacity is
	\[C = {W \over 2} \log_2 \left(1 + {SNR} \right)\]
	\end{theorem}

It's important to notice a few things about this formula:
\begin{itemize}
	\item  The channel capacity obviously depends on the constraint on the transmit probability distribution.  The probability $p(x)$ distribution that maximizes capacity and gives the formula ${W \over 2}  \log_2 \left(1 + {SNR} \right)$ is gaussian.  The capacity when constrained to be different (PAM-2 symbols etc...) is in general a difficult optimization problem with no closed form solution.
	\end{itemize}

\section{Detection and Estimation}
\newpage
\subsection{Biased SNR}

Consider a signal processing system where we would like to slice an output
\[ y =  \underbrace{\alpha}_{\text{adaptive gain}} (\underbrace{x}_{\text{symbol}} + \underbrace{n}_{\text{uncorrelated noise}})\]

Maximizing the SNR, or minimizing the MSE will lead to a \vocab{biased} gain factor $\alpha$ which is slightly less than 1.  To show this, we just need to minimize the error, defined as:
\[ \text{MSE} = \braket{(y - x)^2} = (\alpha - 1)^2 \epsilon_x + \alpha^2 \epsilon_n \]
\[ \pder{\alpha}{\text{MSE}} = 0 \rightarrow \alpha = {\epsilon_x \over \epsilon_x + \epsilon_n} = {SNR \over 1 + SNR} \]

It is straightforward to make the decisions unbiased by scaling $\alpha$ by ${SNR + 1 \over SNR}$, which leads to a relation between \vocab{biased} and \vocab{unbiased} SNR.
\[ SNR = SNR_{U} + 1 \]
The reason why one would care is that a biased decision rule, while maximizing SNR, may not optimize BER.  We will further make rather trivial comments

\begin{itemize}
\item The distinction between biased and unbiased decreases as the SNR improves.  This is why for SERDES links, people rarely care about the distinction.
\item  While the analysis was done for a simple gain adaptation loop, all the conclusion remains for FFE adaptation.  In that case, if one uses a MMSE algorithm for adaptation, one will end up with a \vocab{biased} decision rule, while if one uses a ZF algorithm, one will end up with a \vocab{unbiased} adaptation.
	\end{itemize}
\section{Math}

\end{document}