\documentclass[11pt, oneside]{article}   	% use "amsart" instead of "article" for AMSLaTeX format

% \usepackage{draftwatermark}
% \SetWatermarkText{Draft}
% \SetWatermarkScale{5}
% \SetWatermarkLightness {0.9}
% \SetWatermarkColor[rgb]{0.7,0,0}

\usepackage{geometry}                		% See geometry.pdf to learn the layout options. There are lots.
\geometry{letterpaper}                   		% ... or a4paper or a5paper or ...
%\geometry{landscape}                		% Activate for for rotated page geometry
%\usepackage[parfill]{parskip}    		% Activate to begin paragraphs with an empty line rather than an indent
\usepackage{graphicx}				% Use pdf, png, jpg, or eps� with pdflatex; use eps in DVI mode
								% TeX will automatically convert eps --> pdf in pdflat
								% TeX will automatically convert eps --> pdf in pdflatex
\usepackage{amssymb}
\usepackage{mathrsfs}
\usepackage{hyperref}
\usepackage{url}
\usepackage{subcaption}
\usepackage{authblk}
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{graphicx}
\usepackage[export]{adjustbox}
\usepackage{fixltx2e}
\usepackage{hyperref}
\usepackage{alltt}
\usepackage{color}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{float}
\usepackage{bigints}
\usepackage{braket}
\usepackage{siunitx}

%
% so you can do e.g., \begin{bmatrix}[r] (or [c] or [l])
%

\makeatletter
\renewcommand*\env@matrix[1][c]{\hskip -\arraycolsep
  \let\@ifnextchar\new@ifnextchar
  \array{*\c@MaxMatrixCols #1}}
\makeatother

\newcommand{\argmax}{\operatornamewithlimits{argmax}}
\newcommand{\argmin}{\operatornamewithlimits{argmin}}

\title{A Few Notes on Bell States, Superdense Coding, and Quantum Teleportation}
\author{David Meyer \\ dmm@\{1-4-5.net,uoregon.edu\}}

\date{Last update: \today}							% Activate to display a given date or no date

\begin{document}
\maketitle

\section{Introduction}

The Bell Circuit, shown in Figure \ref{fig:bell_circuit}, is comprised of two gates, $H$ and CNOT, which are defined as follows:

\begin{flalign*}
H = \frac{1}{\sqrt{2}}  \begin{bmatrix}[r] 1 & 1 \\ 1 &  -1 \end{bmatrix},
\text{CNOT} = \begin{bmatrix}[r]
1 & 0 & 0 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 0 & 1 \\
0 & 0 & 1 & 0 \\
\end{bmatrix}
\end{flalign*}

\bigskip
\noindent
and results in two maximally entangled qubits\footnote{This state is sometimes called an \emph{EPR} state.}.
How does this work?

\bigskip
\noindent
First, recall that

\begin{flalign*}
H \ket{0} &= \frac{1}{\sqrt{2}} \big ( \ket{0} + \ket{1} \big )  \text{ and }
H \ket{1} = \frac{1}{\sqrt{2}} \big ( \ket{0} - \ket{1} \big )
\end{flalign*}

\bigskip
\noindent
The Bell Circuit applies $H$ to $\ket{b_0}$ and then applies the CNOT gate to $H\ket{b_0}$ (control qubit)
and $\ket{b_1}$ (target qubit). The inputs and evolution of the Bell Circuit are shown are Table \ref{tab:bell_state}.

\bigskip
\begin{figure}
\center{\includegraphics[scale=0.45, frame] {images/bell_forward_circuit.png}}
\caption{Bell Circuit}
\label{fig:bell_circuit}
\end{figure}

\begin{table}[H]
\centering
\begin{tabular}{c | c | c | c | c}
$b_{0} b_{1}$  & $H \ket{b_0}$ & $\ket{b_1}$ & Bell Circuit evolution with inputs $\ket{b_0}$ and $\ket{b_1}$  & Bell State\\
\hline
00  & $H\ket{0}$ & $\ket{0}$ & $\ket{0} \xrightarrow{\scriptsize H}  \frac{1}{\sqrt{2}} (\ket{0} + \ket{1})
  \xrightarrow{\otimes \ket{0}}  \frac{1}{\sqrt{2}} (\ket{0} + \ket{1}) \ket{0} \xrightarrow{\scriptsize \text{CNOT}} \frac{1}{\sqrt{2}} (\ket{00} + \ket{11})$ & $\ket{\phi^+}$ \\
01  & $H\ket{0}$ & $\ket{1}$ & $ \ket{0} \xrightarrow{\scriptsize H} \frac{1}{\sqrt{2}} (\ket{0} + \ket{1})
  \xrightarrow{\scriptsize \otimes \ket{1}} \frac{1}{\sqrt{2}} (\ket{0} + \ket{1})  \ket{1} \xrightarrow{\scriptsize \text{CNOT}}  \frac{1}{\sqrt{2}} (\ket{01} + \ket{10})$\ & $\ket{\psi^+}$ \\
10  & $H\ket{1}$ & $\ket{0}$ & $\ket{1} \xrightarrow{\scriptsize H}  \frac{1}{\sqrt{2}} (\ket{0} - \ket{1})  \xrightarrow{\scriptsize \otimes \ket{0}} \frac{1}{\sqrt{2}} (\ket{0} - \ket{1})  \ket{0} \xrightarrow{\scriptsize \text{CNOT}} \frac{1}{\sqrt{2}}  (\ket{00} - \ket{11})$ & $\ket{\phi^-}$ \\
11    & $H\ket{1}$ & $\ket{1}$ & $\ket{1} \xrightarrow{\scriptsize H}  \frac{1}{\sqrt{2}} (\ket{0} - \ket{1})
  \xrightarrow{\scriptsize \otimes  \ket{1}}  \frac{1}{\sqrt{2}} (\ket{0} - \ket{1})  \ket{1} \xrightarrow{\scriptsize \text{CNOT}} \frac{1}{\sqrt{2}}  (\ket{01} - \ket{10})$ & $\ket{\psi^-}$
\end{tabular}
\caption{Bell States}
\label{tab:bell_state}
\end{table}

\bigskip
\noindent
What we can see from Table \ref{tab:bell_state} is that $b_0$ selects the "bit" ($\ket{\phi}$ or $\ket{\psi}$), and $b_1$ selects the "sign" ($\ket{+}$ or $\ket{-}$). Since there
are four orthonormal states, the \emph{Bell basis}, we can encode two bits ($b_0$ and $b_1$) in the four Bell States.

\bigskip
\noindent
Now,  if Alice wants to send two classical bits to  Bob (\emph{superdense coding}) using one qubit, she need only transform her qubit\footnote{Her half of the EPR pair, the two entangled qubits.}
into the Bell State corresponding to the two bits she wants to send, then send her half to Bob (this requires a \emph{quantum} channel). Bob can then recover Alice's two bit message.

\bigskip
\noindent
But how can Bob recover Alice's message? Recall that unitary quantum operations are reversible. So Bob can use the Reverse Bell Circuit shown in Figure \ref{fig:reverse_bell_circuit} to recover
Alice's 2 bit message.

\bigskip
\begin{figure}[H]
\center{\includegraphics[scale=0.45, frame] {images/bell_reverse_circuit.png}}
\caption{Reverse Bell Circuit}
\label{fig:reverse_bell_circuit}
\end{figure}

\section{Superdense Coding}

Suppose Alice wants to send Bob the message $00$. Alice can perform one or more unitary operations on her qubit (her half of the entangled pair) that will allow Bob,
when presented with Alice's qubit, to reconstruct Alice's message $b_0b_1$. If we run $\ket{\phi^+}$ through the circuit in Figure \ref{fig:reverse_bell_circuit},
that is, $\ket{\phi^+}  \xrightarrow {\scriptsize \text{CNOT}}  \xrightarrow {\scriptsize \text{  H  }}  \ket{b_0b_1}$,  Bob will recover Alice's message ($b_0b_1 = 00$). Why is this?

\begin{flalign*}
\ket{\phi^+} &=  \frac{1}{\sqrt{2}} (\ket{00} + \ket{11}) \longrightarrow  \\
& \frac{1}{\sqrt{2}} (\ket{0}  \ket{0} + \ket{1} \ket{1}) \xrightarrow {\scriptsize \text{CNOT}} \frac{1}{\sqrt{2}} (\ket{00} + \ket{10}) \longrightarrow \\
& \frac{1}{\sqrt{2}} (\ket{00} + \ket{10}) \xrightarrow {\scriptsize \text{  H  }}  \frac{1}{\sqrt{2}}  \Big ( \frac{1}{\sqrt{2}}  (\ket{0} + \ket{1}) \ket{0} + \frac{1}{\sqrt{2}}  (\ket{0} - \ket{1}) \ket{0} \Big) \\
&= \frac{1}{\sqrt{2}} \frac{1}{\sqrt{2}}  \Big ( (\ket{0} + \ket{1}) \ket{0} + (\ket{0} - \ket{1}) \ket{0} \Big ) \\
&= \frac{1}{2} \big (\ket{00} + \ket{10} + \ket{00} - \ket{10} \big ) \\
&= \frac{1}{2} \big  (2 \ket{00} + (\ket{10} - \ket{10}) \big ) \\
&=  \frac{1}{2} \cdot 2 \ket{00} \\
&= \ket{00}
\end{flalign*}

\bigskip
\noindent
Bob can now measure both qubits and recover Alice's message ($b_0b_1 = 00$).

\bigskip
\noindent
In general, Alice notices that

\begin{itemize}
\item To send \textbf{00}, apply the Identity matrix $\mathbf{I} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$ to her half of the EPR pair
\item To send \textbf{01}, apply the matrix $\mathbf{X} = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$ to her half of the EPR pair
\item To send \textbf{10}, apply the matrix $\mathbf{Z} = \begin{bmatrix}[r] 1 & 0 \\ 0 & -1 \end{bmatrix}$ to her half of the EPR pair
\item To send \textbf{11}, apply $i\mathbf{Y} = i \begin{bmatrix}[r] 0 & -i \\ i & 0 \end{bmatrix}$, i.e. both $\mathbf{X}$ and $\mathbf{Z}$,  to her half of the EPR pair
\end{itemize}

\bigskip
\noindent
where $\mathbf{I}$, $\mathbf{X}$, $\mathbf{Y}$ and $\mathbf{Z}$ are the \emph{Pauli} matrices \cite{wiki:pauli_matrices}.
\bigskip
\noindent
This transforms the EPR pair $\ket{\phi^+}$ into the four Bell States $\ket{\phi^+}$, $\ket{\psi^+}$, $\ket{\phi^-}$ and $\ket{\psi^-}$ respectively:

\begin{itemize}
\item \textbf{00}: $\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}  \frac{1}{\sqrt{2}} (\ket{00} + \ket{11} \longrightarrow  \frac{1}{\sqrt{2}} (\ket{00} + \ket{11}) =
\frac{1}{\sqrt{2}} \begin{bmatrix} 1 \\ 0 \\  0 \\ 1 \end{bmatrix} = \ket{\phi^+}$

\item \textbf{01:} $\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \frac{1}{\sqrt{2}} (\ket{00} + \ket{11} \longrightarrow  \frac{1}{\sqrt{2}} (\ket{01} + \ket{10}) =
\frac{1}{\sqrt{2}} \begin{bmatrix} 0 \\ 1 \\  1 \\ 0 \end{bmatrix} = \ket{\psi^+}$

\item \textbf{10:} $\begin{bmatrix}[r] 1 & 0 \\ 0 & -1 \end{bmatrix}  \frac{1}{\sqrt{2}} (\ket{00} - \ket{11} \longrightarrow  \frac{1}{\sqrt{2}} (\ket{00} - \ket{11}) =
\frac{1}{\sqrt{2}} \begin{bmatrix}[r] 1 \\ 0 \\  0 \\  -1 \end{bmatrix} = \ket{\phi^-}$

\item \textbf{11:}  $i  \begin{bmatrix}[r] 0 & -i \\ i & 0 \end{bmatrix} \frac{1}{\sqrt{2}} (\ket{00} + \ket{11} \longrightarrow  \frac{1}{\sqrt{2}} (\ket{01} - \ket{10}) =
\frac{1}{\sqrt{2}} \begin{bmatrix}[r] 0 \\ 1 \\  -1 \\ 0 \end{bmatrix} = \ket{\psi^-}$
\end{itemize}

\bigskip
\noindent
The four Bell states $\ket{\phi^+}$, $\ket{\psi^+}$, $\ket{\phi^-}$ and $\ket{\psi^-}$ are orthonormal and are hence distinguishable by
quantum measurement.  Thus after receiving Alice's transformed qubit (her half of the EPR pair),
Bob can measure both qubits and recover $b_0b_1$. Hence one qubit carries two classical bits of information; this is superdense coding.
We saw an example of this above in which Bob recovered $\ket{00}$ from $\ket{\phi^+}$ using
the Reverse Bell Circuit depicted in Figure \ref{fig:reverse_bell_circuit}.

\subsection{Aside: Spectral Decomposition of Pauli Matrices}
So far we've interpreted the Pauli matrices as a quantum gates. But note that a gate such \textbf{Z} is a Hermitean operator and as a
result can be interpreted as an observable. Somewhat surprisingly (notice the symmetry),  the spectral decomposition \cite{2014arXiv1405.5749S}
 of \textbf{Z}  is

\begin{flalign*}
\mathbf{Z} =  \ket{0}\bra{0} - \ket{1}\bra{1}
\end{flalign*}

\bigskip
\noindent
where  $\ket{u}\bra{v}$ is Dirac notation \cite{2000RPPh...63.1893G} for the outer product $\mathbf{u} \otimes \mathbf{v} =  \mathbf{u} \mathbf{v}^{\text{T}}$ of $m \times 1$ vector
\textbf{u} and  $n \times 1$ vector \textbf{v}, which yields a $m \times n$ matrix\footnote{The outer product is of vectors
\textbf{u} and \textbf{v}  is a special case of the tensor product  $\mathbf{u} \otimes \mathbf{v}$.
More generally, the outer product is an instance of a Kronecker product \cite{wiki:kronecker}.}.

\bigskip
\noindent
We can see that the eigenvalues of \textbf{Z}  are 1 and -1, corresponding to eigenvectors $\ket{0}$  and $\ket{1}$  respectively. So the measurement operators are the
projectors $\ket{0}\bra{0}$ and $\ket{1}\bra{1}$,  This means that a measurement of the Pauli observable \textbf{Z} is a measurement in the computational basis that has
eigenvalue +1 corresponding to $\ket{0}$  and eigenvalue -1 corresponding to $\ket{1}$.

\bigskip
\noindent
So ok, but why does $\mathbf{Z} = \ket{0}\bra{0} - \ket{1}\bra{1}$? Well, we know that the outer product $\mathbf{u} \otimes \mathbf{v}$ of a $m \times 1$ vector  \textbf{u} and a
$n \times 1$ vector  \textbf{v} is defined to be the $m \times n$ matrix\footnote{Contrast with the scalar inner product
$\langle \mathbf{u},  \mathbf{v} \rangle = \mathbf{u}^{\text{T}} \mathbf{v}$. Note also that $\langle \mathbf{u},  \mathbf{v} \rangle = \text{tr}(\mathbf{u} \otimes \mathbf{v})$,
where $\text{tr}(\mathbf{A})$ is the "trace" of matrix \textbf{A}. } $\mathbf{u} \mathbf{v}^{\text{T}}$.

\bigskip
\noindent
To see why $\mathbf{Z} = \ket{0}\bra{0} - \ket{1}\bra{1}$, first recall that  $\ket{0} = \begin{bmatrix} 1 \\0 \end{bmatrix}$ and $\ket{1} = \begin{bmatrix} 0 \\ 1 \end{bmatrix}$. Then

\begin{flalign*}
\ket{0}\bra{0}  &= \begin{bmatrix} 1 \\ 0 \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix}^{\text{T}} =
\begin{bmatrix} 1 \\ 0 \end{bmatrix} \begin{bmatrix} 1  & 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} \text{ and} \\
\ket{1}\bra{1}  &= \begin{bmatrix} 0 \\ 1 \end{bmatrix}\begin{bmatrix} 0 \\ 1 \end{bmatrix}^{\text{T}}  = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \begin{bmatrix} 0  & 1 \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} \text{ so that} \\
\ket{0}\bra{0} - \ket{1}\bra{1} &= \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}  - \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix}[r] 1 & 0 \\ 0 & -1 \end{bmatrix} = \mathbf{Z}
\end{flalign*}

\bigskip
\subsection{Back to Alice wanting to send a message to Bob}
Now suppose Alice want's to send Bob the message 01.  Alice then applies Pauli matrix $X$ to $\ket{\phi+}$ to get $\ket{\psi^+}$:

\begin{equation*}
\mathbf{X} \ket{\phi^+} = \begin{bmatrix}[r] 0 & 1  \\ 1 & 0\end{bmatrix} \frac{1}{\sqrt{2}} (\ket{00} + \ket{11} = \frac{1}{\sqrt{2}} (\ket{01} + \ket{10}) = \ket{\psi^+}
\end{equation*}

\bigskip
\noindent
Bob can now recover Alice's message as follows using the Reverse Bell Circuit (Figure \ref{fig:reverse_bell_circuit}). That is, Bob can
do the the unitary operations $\ket{\psi^+}  \xrightarrow {\scriptsize \text{CNOT}}  \xrightarrow {\scriptsize \text{H}}  \ket{01}$, as follows:

\begin{flalign*}
\ket{\psi^+} &= \frac{1}{\sqrt{2}} (\ket{01} + \ket{10}) \longrightarrow \\
& \frac{1}{\sqrt{2}} (\ket{0}  \ket{1} + \ket{1} \ket{0}) \xrightarrow {\scriptsize \text{CNOT}} \frac{1}{\sqrt{2}} (\ket{01} + \ket{11}) \longrightarrow \\
& \frac{1}{\sqrt{2}} (\ket{01} + \ket{11}) \xrightarrow {\scriptsize \text{  H  }}  \frac{1}{\sqrt{2}}  \Big ( \frac{1}{\sqrt{2}}  (\ket{0} + \ket{1}) \ket{1} + \frac{1}{\sqrt{2}}  (\ket{0} - \ket{1}) \ket{1} \Big) \\
&= \frac{1}{\sqrt{2}} \frac{1}{\sqrt{2}}  \Big ( (\ket{0} + \ket{1}) \ket{1} + (\ket{0} - \ket{1}) \ket{1} \Big ) \\
&= \frac{1}{2} \big (\ket{01} + \ket{11} + \ket{01} - \ket{11} \big) \\
&= \frac{1}{2} \big (2 \ket{01} + (\ket{11} - \ket{11}) \big) \\
&=  \frac{1}{2}  \cdot 2 \ket{01} \\
&=  \frac{2}{2}  \ket{01} \\
&= \ket{01}
\end{flalign*}

\bigskip
\noindent
Now Bob can measure the two qubits and recover Alice's message ($b_0b_1 = 01$).

\bigskip
\noindent
Similarly, suppose Alice wants to send the message 10 to Bob. Alice first transforms her qubit as follows

\begin{equation*}
\mathbf{Z} \ket{\phi^+} = \begin{bmatrix}[r] 1 & 0 \\ 0 & -1 \end{bmatrix}  \frac{1}{\sqrt{2}} (\ket{00} + \ket{11} = \frac{1}{\sqrt{2}} (\ket{00} -  \ket{11}) = \ket{\phi^-}
\end{equation*}

\bigskip
\noindent
Alice now sends her qubit to Bob over a quantum channel. Bob can now recover Alice's message, again using the Reverse Bell Circuit
($\ket{\phi^-}  \xrightarrow {\scriptsize \text{CNOT}}  \xrightarrow {\scriptsize \text{  H  }}  \ket{10}$). Again, why is this?

\begin{flalign*}
\ket{\phi^-} &=  \frac{1}{\sqrt{2}} (\ket{00} - \ket{11}) \longrightarrow \\
& \frac{1}{\sqrt{2}} (\ket{0}  \ket{0} - \ket{1} \ket{1}) \xrightarrow {\scriptsize \text{CNOT}} \frac{1}{\sqrt{2}} (\ket{00} - \ket{10}) \longrightarrow \\
& \frac{1}{\sqrt{2}} (\ket{00} - \ket{10}) \xrightarrow {\scriptsize \text{  H  }}  \frac{1}{\sqrt{2}}  \Big ( \frac{1}{\sqrt{2}}  (\ket{0} + \ket{1}) \ket{0} -  \frac{1}{\sqrt{2}}  (\ket{0} - \ket{1}) \ket{0} \Big) \\
&= \frac{1}{\sqrt{2}} \frac{1}{\sqrt{2}}  \Big ( (\ket{0} + \ket{1}) \ket{0} -  (\ket{0} - \ket{1}) \ket{0} \Big ) \\
&= \frac{1}{2} \big (\ket{00} + \ket{10} - \ket{00}  + \ket{10} \big ) \\
&= \frac{1}{2} \big (2 \ket{10} + (\ket{00} - \ket{00}) \big ) \\
&=  \frac{1}{2} \cdot 2 \ket{10} \\
&= \ket{10}
\end{flalign*}

\bigskip
\noindent
Now Bob can measure the two qubits and recover Alice's message ($b_0b_1 = 10)$.

\bigskip
\noindent
Finally, if Alice wants to send $11$ to Bob she first transforms her qubit

\begin{equation*}
i \mathbf{Y} \ket{\phi^+} =  \begin{bmatrix}[r] 0 & -i \\ i & 0 \end{bmatrix}   \frac{1}{\sqrt{2}} (\ket{00} + \ket{11} = \frac{1}{\sqrt{2}} (\ket{01} - \ket{10}) = \ket{\psi^-}
\end{equation*}

\bigskip
\noindent
Alice now transmits her qubit to Bob and Bob applies the Reverse Bell Circuit to recover Alice's message:

\begin{flalign*}
\ket{\psi^-} &= \frac{1}{\sqrt{2}} (\ket{01} -  \ket{10}) \longrightarrow \\
& \frac{1}{\sqrt{2}} (\ket{0}  \ket{1} - \ket{1} \ket{0}) \xrightarrow {\scriptsize \text{CNOT}} \frac{1}{\sqrt{2}} (\ket{01} -  \ket{11}) \longrightarrow \\
& \frac{1}{\sqrt{2}} (\ket{01} -  \ket{11}) \xrightarrow {\scriptsize \text{  H  }}  \frac{1}{\sqrt{2}}  \Big ( \frac{1}{\sqrt{2}}  (\ket{0} + \ket{1}) \ket{1} - \frac{1}{\sqrt{2}}  (\ket{0} - \ket{1}) \ket{1} \Big) \\
&= \frac{1}{\sqrt{2}} \frac{1}{\sqrt{2}}  \Big ( (\ket{0} + \ket{1}) \ket{1} -  (\ket{0} - \ket{1}) \ket{1} \Big ) \\
&= \frac{1}{2} \big (\ket{01} + \ket{11} -  \ket{01} +  \ket{11} \big) \\
&= \frac{1}{2} \big (2 \ket{11} + (\ket{01} - \ket{01}) \big) \\
&=  \frac{1}{2}  \cdot 2 \ket{11} \\
&=  \frac{2}{2}  \ket{11} \\
&= \ket{11}
\end{flalign*}

\bigskip
\noindent
Now Bob can measure the two qubits and recover Alice's message ($b_0b_1 = 11)$.

\section{Quantum Teleportation}
Quantum teleportation can be thought of as the dual task to super dense coding. Whereas super dense coding is concerned with conveying classical information
via a qubit, quantum teleportation is concerned with conveying quantum information with classical bits \cite{Bennett:1992tv}.

\subsection{A high-level view of the quantum teleportation algorithm}

\begin{enumerate}
\item Alice and Bob share an entangled (EPR) pair $\ket{\phi^+}$
\item Alice chooses a qubit $\ket{\psi}$ as the message she wants to convey to Bob
\item Alice performs operations on $\ket{\psi}$ and $\ket{\phi^{+}_A}$ (Alice's her half of $\ket{\phi^+}$)
\item Alice measures $\ket{\psi}$  and her half of  $\ket{\phi^+_A}$, destroying both of her qubits
\item Alice sends the two classical bits that were the results of her measurements to Bob
\item Bob uses the two classical bits to "correct" $\ket{\phi^{+}_B}$ (his half of $\ket{\phi^+}$) to be $\ket{\psi}$
\end{enumerate}

\bigskip
\noindent
Alice uses the circuit in Figure \ref{fig:a_reverse_bell_circuit} to prepare her two qubits (step 3 above). How exactly does this work?
First, notice that the input to the Reverse Bell Circuit shown in Figure \ref{fig:a_reverse_bell_circuit} is $\ket{\psi} \otimes \ket{\phi^{+}_A}$.
To see how this works, first recall that $\ket{\psi} = \alpha \ket{0} + \beta \ket{1}$.  Then

\begin{figure}[t]]
\center{\includegraphics[scale=0.45, frame] {images/a_reverse_bell_circuit.png}}
\caption{Reverse Bell Circuit}
\label{fig:a_reverse_bell_circuit}
\end{figure}

\begin{flalign*}
\ket{\psi} \otimes \ket{\phi^{+}_A} &= (\alpha \ket{0} + \beta \ket{1}) \otimes \frac{1}{\sqrt{2}} (\ket{00} + \ket{11}) \\
&= \frac{1}{\sqrt{2}} \Big ( \alpha (\ket{000} + \alpha \ket{011}) + \beta (\ket{100} + \ket{111}) \Big ) \xrightarrow {\scriptsize \text{CNOT}}
\quad\qquad \mathrel{\#} \ket{b_0b_1b_2}: b_0 \text{ is control, } b_1  \text{ is target} \\
& \frac{1}{\sqrt{2}} \Big ( \alpha \ket{000} + \alpha \ket{011})+ \beta (\ket{110} + \ket{101}) \Big ) \xrightarrow {\scriptsize \text{H}}
\; \qquad \qquad\qquad \mathrel{\#} \text{apply $H$ to $b_0$} \\
& \frac{1}{\sqrt{2}} \bigg [ \alpha \Big ( \frac{1}{\sqrt{2}} \ket{0} + \ket{1} \Big ) \ket{00} + \alpha \Big (  \frac{1}{\sqrt{2}} \ket{0} + \ket{1}) \Big )  \ket{11}  +
\beta \Big ( \frac{1}{\sqrt{2}} \ket{0} - \ket{1} \Big ) \ket{10} + \beta \Big (  \frac{1}{\sqrt{2}} \ket{0} - \ket{1} \Big )  \ket{01}  \bigg ]\\
&= \frac{1}{\sqrt{2}} \frac{1}{\sqrt{2}}
\bigg [ \alpha \Big ( \big (\ket{0} + \ket{1} \big ) \ket{00} +  \big (\ket{0} + \ket{1} \big )  \ket{11} \Big)  +
\beta \Big ( \big (\ket{0} - \ket{1} \big ) \ket{10} + \big ( \ket{0} - \ket{1} \big )  \ket{01} \Big ) \bigg] \\
&= \frac{1}{2} \bigg [ \alpha \Big (\ket{000} + \ket{100}  +  \ket{011} + \ket{111}) \Big)  +
\beta \Big (\ket{010} - \ket{110} + \ket{001} - \ket{101}  \Big ) \bigg ] \\
&= \frac{1}{2} \bigg [ \alpha \ket{000} + \alpha \ket{100}  +  \alpha \ket{011} + \alpha \ket{111})  +
 \beta \ket{010} - \beta\ket{110} + \beta \ket{001} - \beta \ket{101}  \bigg ] \\
\end{flalign*}

\bigskip
\noindent
Now Alice measures her two qubits ($\ket{\psi} \otimes \ket{\phi^{+}_A} $) and observes $b_0b_1 \in \{00, 01, 10, 11\}$ with $P(b_0b_1) = \frac{1}{4}$.

\bigskip
\noindent
Now here's the amazing thing. If Alice observes $00$, she communicates this to Bob (over a classical channel). As soon as
Bob sees the value $00$, he knows that his qubit $\ket{\phi^{+}_{B}} = \alpha \ket{0} + \beta \ket{1}$. How does Bob know this?

\bigskip
\noindent
First, as shown above

\begin{flalign}
\label{eqn:psi_otimes_phi+}
\ket{\psi} \otimes \ket{\phi^+} = \frac{1}{2} \bigg [ \alpha \ket{000} + \alpha \ket{100}  +  \alpha \ket{011} + \alpha \ket{111})  +
\beta \ket{010} - \beta\ket{110} + \beta \ket{001} - \beta \ket{101}  \bigg ]
\end{flalign}

 \bigskip
\noindent
Alice's measurement of the first two qubits collapses Bob's qubit to the third qubit\footnote{Recall that the original three qubits were
$\ket{\psi} \otimes \ket{\phi^{+}_{AB}}$.}.  The only terms in Equation \ref{eqn:psi_otimes_phi+} that are
consistent with the first two qubits being $\ket{00}$ (resulting from Alice's measurement)
are $\alpha \ket{000}$ and $\beta \ket{001}$. The "collapsed version" is $\alpha \ket{0}$ and $\beta \ket{1}$.
Hence Bob knows that his qubit, $\ket{\phi^{+}_B}$,  equals $\alpha \ket{0} + \beta \ket{1}$.

 \bigskip
\noindent
Since Alice sent the two bits she saw to Bob, he knows which operations to perform to transform $\ket{\phi^{+}_B} \rightarrow \ket{\psi}$.
In particular, $b_0 = 1$ Bob should apply Pauli matrix $Z$ to his qubit and $I$ otherwise,
and if $b_1 = 1$ he should apply $X$ and $I$ otherwise. This transforms $\ket{\phi^{+}_B}$, Bob's qubit, into $\ket{\psi}$.
This is shown in Table \ref{tab:bob}.

\bigskip
\noindent
Amazingly  this procedure teleports Alice's qubit $\ket{\psi}$ to Bob using the two classical bits that Alice learned by measuring her two qubits ($\ket{\psi}$ and $\ket{\phi^{+}_A}$).

\bigskip
\begin{table}[H]
\centering
\begin{tabular}{c | c | r | r}
$b_{0} b_{1}$  & $\ket{\phi^{+}_B}$  & \multicolumn{1}{c|}{Transformation} & \multicolumn{1}{c}{Computation}  \\
\hline
\textbf{00}  & $\alpha \ket{0} + \beta \ket{1}$ & $\mathbf{I} \begin{bmatrix} \alpha \\ \beta \end{bmatrix}$ & $\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}
 \begin{bmatrix} \alpha \\ \beta \end{bmatrix}  = \begin{bmatrix} \alpha \\  \beta  \end{bmatrix} = \alpha \ket{0} + \beta \ket{1} = \ket{\psi}$  \\
\textbf{01}  & $\beta \ket{0} + \alpha \ket{1}$ & $\mathbf{X}\begin{bmatrix} \beta \\ \alpha  \end{bmatrix}$ & $ \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}  \begin{bmatrix} \beta \\ \alpha  \end{bmatrix}
= \begin{bmatrix} \alpha \\  \beta  \end{bmatrix}  = \alpha \ket{0} + \beta \ket{1} = \ket{\psi}$ \\
\textbf{10}  & $\alpha \ket{0} - \beta \ket{1}$ & $\mathbf{Z} \begin{bmatrix}[r]   \alpha \\  -\beta  \end{bmatrix}$ & $\begin{bmatrix}[r] 1 & 0  \\ 0 & -1 \end{bmatrix}  \begin{bmatrix}[r] \alpha \\ -\beta \end{bmatrix}
= \begin{bmatrix} \alpha \\  \beta  \end{bmatrix}  = \alpha \ket{0} + \beta \ket{1} = \ket{\psi}$ \\
\textbf{11}  & $\beta \ket{0} - \alpha \ket{1} $ & $\mathbf{XZ} \begin{bmatrix}[r]  \beta \\  - \alpha  \end{bmatrix}$ & $  \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}  \begin{bmatrix}[r] 1 & 0  \\ 0 & -1 \end{bmatrix}
  \begin{bmatrix}[r] \beta \\ -  \alpha  \end{bmatrix} = \begin{bmatrix} \alpha \\  \beta  \end{bmatrix} = \alpha \ket{0} + \beta \ket{1} = \ket{\psi}$ \\
\end{tabular}
\caption{Bob's transformations on receiving classical bits $\mathbf{b_0b_1}$ from Alice}
\label{tab:bob}
\end{table}

\subsection{Curious Entry for \textbf{11} in Table \ref{tab:bob}?}
Note that the row for the result of Alice's measurement \textbf{11} in Table \ref{tab:bob} is curious. When Bob sees \textbf{11} from Alice he knows that his remaining qubit
$\ket{\psi^{+}_B}$, equals $- \beta \ket{0} + \alpha \ket{1}$. Why does the table say $\beta \ket{0} - \alpha \ket{1}$?

\bigskip
\noindent
Here is one way to look at this: First,
recall that when Bob receives classical bits \textbf{11} from Alice he knows that his qubit, $\ket{\psi^{+}_B}$,  is

\begin{flalign*}
\ket{\psi^{+}_B} &= - \beta \ket{0} + \alpha \ket{1} = \begin{bmatrix} -\beta \\ \alpha \end{bmatrix}
\end{flalign*}

\bigskip
\noindent
Now, if Bob now wants to transform $\ket{\psi^{+}_B} \rightarrow \ket{\psi}$, he would apply $\mathbf{ZX}$ as follows

\begin{flalign*}
\mathbf{ZX} \ket{\psi^{+}_B}  &=  \begin{bmatrix}[r] 1 & 0  \\ 0 & -1 \end{bmatrix}   \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}  \begin{bmatrix} -\beta \\ \alpha \end{bmatrix} \\
&= \begin{bmatrix}[r] 1 & 0  \\ 0 & -1 \end{bmatrix}   \begin{bmatrix} \alpha \\ - \beta \end{bmatrix} \\
&= \begin{bmatrix} \alpha \\ \beta \end{bmatrix} \\
&= \alpha \ket{0} + \beta \ket{1} \\
&= \ket{\psi}
\end{flalign*}

\bigskip
\noindent
But our rule (Table \ref{tab:bob})  tells Bob to apply $\mathbf{XZ}$ when he sees \textbf{11} from Alice. Why? Notice the following:

\begin{flalign*}
\mathbf{ZX} \begin{bmatrix} x_0 \\ x_1 \end{bmatrix} &= \mathbf{Z} \begin{bmatrix} x_1 \\ x_0  \end{bmatrix} =  \begin{bmatrix}[r] x_1 \\  - x_0 \end{bmatrix} \\
\mathbf{XZ} \begin{bmatrix} x_0 \\ x_1 \end{bmatrix} &= \mathbf{X} \begin{bmatrix}[r] x_0 \\ - x_1  \end{bmatrix}  = \begin{bmatrix}[r] - x_1 \\  x_0 \end{bmatrix}
\end{flalign*}

\bigskip
\noindent
which implies that

\bigskip

\begin{equation}
\mathbf{ZX} \begin{bmatrix} x_0 \\ x_1 \end{bmatrix}  = - \mathbf{XZ} \begin{bmatrix} x_0 \\ x_1 \end{bmatrix}
\label{eqn:equal}
\end{equation}

\bigskip
\bigskip
\noindent
So now let $x_0 = \beta$ and $x_1 =   \alpha$.  Then

\begin{flalign*}
\mathbf{XZ} \Big [\beta \ket{0} - \alpha \ket{1} \Big ] = \mathbf{XZ} \begin{bmatrix}[r] \beta  \\ - \alpha \end{bmatrix} = \mathbf{X} \begin{bmatrix} \beta \\ \alpha \end{bmatrix}
=  \begin{bmatrix} \alpha \\ \beta \end{bmatrix} = \alpha \ket{0} + \beta \ket{1} = \ket{\psi}
\end{flalign*}

\bigskip
\noindent
and $- \big (\beta \ket{0} - \alpha \ket{1} \big )= -\beta \ket{0} + \alpha \ket{1} \longrightarrow$

\begin{flalign*}
\mathbf{ZX}  \Big [ -\beta \ket{0} + \alpha \ket{1} \Big ] = \mathbf{ZX} \begin{bmatrix}[r] - \beta \\ \alpha \end{bmatrix} = \mathbf{Z} \begin{bmatrix}[r] \alpha \\ - \beta\end{bmatrix}
 = \begin{bmatrix} \alpha \\ \beta \end{bmatrix} = \alpha \ket{0} + \beta \ket{1} = \ket{\psi}
 \end{flalign*}

\bigskip
\bigskip
\noindent
The choice of the transformation rules shown in Table \ref{tab:bob} and Equation \ref{eqn:equal} allows us to write
$\beta \ket{0} - \alpha \ket{1}$ rather than $- \beta \ket{0} + \alpha \ket{1}$.

\bigskip
\noindent
Why do this? One thing it does is make the symmetry in Table \ref{tab:bob} more explicit, but hopefully there is a better reason...

\subsubsection{Cloning and/or Faster Than Light Communication?}
First, no faster-than-light communication occurs since Bob learns nothing from the changes until Alice actually sends the two classical bits to him (even
though Alice operating on $\ket{\phi^{+}_A}$ instantly affects $\ket{\phi^{+}_B}$).

\bigskip
\noindent
The No-Cloning Theorem \cite{2018arXiv180804213E}  is not violated since, even though Bob has an exact copy of $\ket{\psi}$, Alice had to destroy her copy (by
measuring it).

\bigskip
\noindent
Finally, an interesting point is that neither Alice or Bob ever "know"  what $\ket{\psi}$ is (in terms of its actual amplitudes); all they know
is that it was transferred (whatever it was).

\section{Bell and  CHSH}

\bigskip
\noindent

\section{Acknowledgements}

\newpage
\bibliographystyle{plain}
\bibliography{/Users/dmm/papers/bib/qc}

\end{document}


\documentclass[11pt]{scrartcl} % Font size
\input{structure.tex} % Include the file specifying the document structure and custom commands

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------

\title{
	\normalfont\normalsize
	\textsc{Harvard Privacy Tools Project}\\ % Your university, school and/or department name(s)
	\vspace{25pt} % Whitespace
	\rule{\linewidth}{0.5pt}\\ % Thin top horizontal rule
	\vspace{20pt} % Whitespace
	{\huge The Exponential Mechanism for Medians}\\ % The assignment title
	\vspace{12pt} % Whitespace
	\rule{\linewidth}{2pt}\\ % Thick bottom horizontal rule
	\vspace{12pt} % Whitespace
}

% \author{\LARGE} % Your name

\date{\normalsize\today} % Today's date (\today) or a custom date

\begin{document}
\maketitle

\section{The Exponential Mechanism}

Sometimes, the global sensitivity of a function is too great, so the Laplace mechanism will not produce meaningful results. The median is one such function. In many cases, the \textit{Exponential mechanism} is an alternate approach that gives reasonable utility.\footnote{This is not the \textit{only} advantage of the exponential mechanism. It is a way to compute differentially private queries on non-numeric data, unlike the Laplace mechanism it does not assume that the probability of outputting a response ought to be symmetric about the true response, etc.} Introduced in 2007 by McSherry and Talwar, the exponential mechanism posits that for a given database, users prefer some outputs over others. That those preferences may be encapsulated with a utility score, where a high utility score indicates a higher preference for that output. The exponential mechanism releases outputs with probability proportional (in the exponent) to the utility score and the sensitivity of the utility function.

\begin{definition}
Let $\mathcal{X}$ be a space of databases and let $[m,M]$ be an arbitrary range. Let $u: \mathcal{X} \times [m,M] \rightarrow \mathbb{R}$ be a utility function, which maps pairs of databases and outputs to a utility score. Let $\Delta u$ be the sensitivity of $u$ with respect to the database argument. The exponential mechanism outputs $r \in [m,M]$ with probability proportional to $\exp\left(\frac{\varepsilon u(x,r)}{2 \Delta u}\right)$ \cite{mcsherry2007mechanism, dwork2014algorithmic}.\footnote{The original definition is from \cite{mcsherry2007mechanism}, but here we state the version rewritten in \cite{dwork2014algorithmic} as it is slightly clearer.}
\end{definition}

\begin{theorem}
The exponential mechanism preserves $(\varepsilon,0)$-differential privacy \cite{mcsherry2007mechanism, dwork2014algorithmic}.\footnote{As written in \cite{mcsherry2007mechanism}, the mechanism actually preserves $(2\varepsilon\Delta u,0)$-differential privacy; the main difference in the $\cite{dwork2014algorithmic}$ version is that it has the extra factor of $2\Delta u$ to avoid these extra terms.}
\end{theorem}

Note that the exponential mechanism may not be tractable in many cases, as it assumes the existence of a utility function, and even if one exists it may not be tractable to compute it efficiently.

\section{An Exponential Mechanism for a quantile}

\subsection{Defining a sensible utility function}

Note that a user will prefer an output that is closer to the true quantile over one that is further away. Let $x$ be an (ordered) data set, let $r$ be a possible output, and let $N$ be the size of the data set. Let $\#(Z>r)$ refer to the number of points in $x$ above $r$. Then, the following is a reasonable utility function for a release $r$ for the $\alpha$-quantile of $x$.

\begin{equation}
u(x,r) = \max(\alpha, (1-\alpha))N - \vert (1-\alpha)\#(Z<r) - \alpha\#(Z>r)\vert.
\end{equation}

\subsection{Sensitivity of the utility function}

\subsubsection{Neighboring Definition: Change One}
\begin{lemma}
The above utility function $u$ has $\ell_11$ sensitivity bounded above by 1 in the change one model.
\end{lemma}

\begin{proof}
Let $c_1 = \#(Z<r)$ and $c_2 = \#(Z>r)$. In one worst case, $c_1$ increases by 1 and $c_2$ decreases by 1.
Then,
\begin{align*}
\Delta u &= \vert (1-\alpha) (c_1 + 1) - \alpha (c_2-1) \vert - \vert (1-\alpha) c_1 - \alpha c_2 \vert  \\
 &\le \vert (1-\alpha) (c_1 + 1) - \alpha (c_2-1) - (1-\alpha) c_1 + \alpha c_2 \vert\\
 & \le \vert c_1 + 1 - \alpha c_1 - \alpha - \alpha c_2 + \alpha - c_1 + \alpha c_1 + \alpha c_2 \vert\\
&= 1
\end{align*}
If instead $c_2$ decreases by 1 and $c_1$ increases by 1, the same thing will happen except with a negative sign that will not impact the final result due to the absolute values.
\end{proof}
\subsubsection{Neighboring Definition: Add/Drop One}

\begin{lemma}
The above utility function $u$ has $\ell_11$ sensitivity bounded above by $\max(1-\alpha, \alpha)$ in the add/drop one model.
\end{lemma}

\begin{proof}
Let $c_1 = \#(Z<r)$ and $c_2 = \#(Z>r)$.  Consider what happens if one point is added. There are two cases that would impact the utility function:
\begin{enumerate}
\item $c_1$ increases by one and nothing happens to $c_2$.
\item $c_2$ increases by one and nothing happens to $c_1$.
\end{enumerate}

Say the first case occurs. Then,

\begin{align*}
\Delta u &= | (1-\alpha) (c_1 + 1) - \alpha (c_2) | - | (1-\alpha) c_1 - \alpha c_2 |  \\
 	&\le | (1-\alpha) (c_1 + 1) - \alpha (c_2) - (1-\alpha) c_1 + \alpha c_2 | \\
	&= 1 - \alpha
\end{align*}

In the second case,

\begin{align*}
\delta u &= \vert (1-\alpha) (c_1) - \alpha (c_2 + 1) \vert - \vert (1-\alpha) c_1 - \alpha c_2 \vert \\
	&\le \vert c_1 -\alpha c_1 - \alpha c_2 - \alpha - c_1 + \alpha c_1 + \alpha c_2  \vert\\
	&= \alpha
\end{align*}
\end{proof}

Subtracting a point leads to the same results.

\bibliographystyle{alpha}
\nocite{*}
\bibliography{expMechMedian}
\end{document}


The following text is reprinted from \href{https://doi.org/10.21105/joss.00797}{Whalley, L. D. (2018). effmass: An effective mass package. \textit{Journal of Open Source Software}, 3(28), p.797.} © 2018 CC-BY

\subsection*{Summary}
\label{sec:summary}

Many semiconductor properties depend on the response of electrons to an external perturbation. This perturbation could take the form of an electric field, change in temperature or an applied lattice stress. In a crystal, this response depends on the interaction of the electrons with a periodic potential. The effective mass approximation assumes that the response of an electron in a periodic potential is equivalent to that of a free electron with a renormalised mass (called the `effective mass'). This makes the effective mass a critical parameter in models for the optical and transport properties of a semiconductor.

The effective mass has a number of definitions, depending on the perturbation under consideration. The conventional definition of effective mass is inversely proportional to the second derivative of electron energy with respect to electron momentum.\autocite[p.~227]{Ashcroft1976} This allows the effective mass to be easily calculated from ab-initio band structures, and there are existing codes which have implemented this (see the `Related packages' Section below).

We must approximate the band structure with a parabola for the previous definition to be valid.\autocite{Ariel2012} However, this approximation breaks down when there is a high concentration of electrons in the material - when, for example, the material is doped or excited under a laser. Instead, we can then approximate the band structure with the Kane quasi-linear dispersion,\autocite{Kane1957} and the definition of effective mass is adapted accordingly.

\textsc{effmass} is a Python 3 package for calculating various definitions of effective mass from the electronic bandstructure of a semiconducting material. It contains a core class that calculates the effective mass and other associated properties of selected band structure segments. effmass also contains functions for locating band structure extrema, calculating the Kane quasi-linear dispersion parameters and plotting approximations to the true dispersion. Parsing of electronic structure data is facilitated by the \textsc{vasppy} package.\autocite{Morgan2018}

\textsc{effmass} is aimed towards theoretical solid state physicists and chemists who have a basic familiarity with Python. Depending on the functionality and level of approximation you are looking for, it may be that one of the packages listed below will suit your needs better.

\subsection*{Related packages}
\label{sec:related}

Effective mass calculations are implemented in a number of other packages:
\begin{itemize}
    \item \textsc{vasppy} \cite{Morgan2018}: This is installed as a dependancy of effmass. Calculates the effective mass using a least-squares quadratic fit for parabolic dispersions.
    \item \textsc{sumo} \cite{Ganose2018}: Calculates the effective mass using a least-squares fit for parabolic and non-parabolic dispersions.
    \item \textsc{emc} \cite{Fornari2012}: Calculates the effective mass tensor using a finite-difference method for parabolic dispersions.
    \item \textsc{pymatgen} \cite{Ong2013}: This is installed as a dependancy of effmass. Calculates an average effective mass tensor for non-parabolic dispersions with multiple bands and extrema. Also calculates the Seebeck effective mass as defined here.
\end{itemize}

\subsection*{Unique features of \textsc{effmass}}
\label{sec:unique}

To our knowledge, the following features are unique to this package:
\begin{itemize}
    \item Easily compare the values of curvature effective mass calculated using multiple numerical techniques (least-squares and polynomial fitting)
    \item Tailor the polynomial fitting used to approximate the DFT calculated dispersion: by choosing the order of the polynomial and the energy range to fit over.
    \item Visualise the dispersions used to approximate the DFT calculated dispersion
    \item Quantify non-parabolicity through the Kane dispersion parameters: effective mass at band-edge and alpha
    \item Calculate the optical effective mass assuming a Kane dispersion.
\end{itemize}

\textbf{Acknowledgements}

LW would like to thank Aron Walsh, Benjamin Morgan and Jarvist Moore Frost for their guidance during this project. This package was written during a PhD funded by the EPSRC through the Centre for Doctoral Training in New and Sustainable Photovoltaics (grant no. EP/L01551X/1). The input data used for developing and testing this package was generated using the ARCHER UK National Supercomputing Service. We have access to Archer via our membership of the UK's HEC Materials Chemistry Consortium, which is funded by EPSRC (EP/L000202).


    \documentclass{article}
    \usepackage{amsfonts}
    \usepackage{amsmath,multicol,eso-pic}
    \begin{document}
    \title{Algebra 101 worksheet 1}
 \date{\vspace{-5ex}}
 \maketitle

        \section{Linear equations}
        Solve the following equations for the specified variable.
        \begin{multicols}{1}
        \begin{enumerate}
        \item Solve for $a$ : $$H a + 5 = 14 a + g$$
\item Solve for $N$ : $$N S + 5 = N w - 11$$
\item Solve for $A$ : $$A E - 4 = 6 A + h$$
\item Solve for $w$ : $$6 w + z = 21 w + 23$$
\item Solve for $b$ : $$K + 23 b = Y b + 5$$
\item Solve for $M$ : $$M S - 4 = 4 M + 6$$
\item Solve for $y$ : $$X y + z = T y + V$$
\item Solve for $p$ : $$- 2 p + 18 = Q p + 12$$
\item Solve for $p$ : $$g - 5 p = V + p t$$
\item Solve for $H$ : $$E H + 20 = H u + y$$
\item Solve for $M$ : $$M R - 10 = M k + R$$
\item Solve for $M$ : $$- 26 M + 9 = M j + b$$
\item Solve for $V$ : $$P + S V = Q V + X$$
\item Solve for $P$ : $$- 10 P + e = - 23 P + 1$$
\item Solve for $g$ : $$- 14 g + 16 = - 21 g - 21$$
\item Solve for $K$ : $$- 23 K + S = K T - 6$$
\item Solve for $d$ : $$Q - 10 d = N d - 24$$
\item Solve for $D$ : $$8 D - 5 = D m + 22$$
\item Solve for $E$ : $$E q + y = - 5 E + 2$$
\item Solve for $r$ : $$r x + 10 = c - 5 r$$
        \end{enumerate}
        \end{multicols}

        \section{Quadratic equations}
        Solve the following quadratic equations.
        \begin{multicols}{1}
        \begin{enumerate}
        \item $$4 y^{2} = - 15 y - 10$$
\item $$9 x^{2} = 19 x + 11$$
\item $$x^{2} - 13 x - 230 = 0$$
\item $$2 y^{2} - 8 = y + 14$$
\item $$2 x^{2} + 22 x = 3 x^{2}$$
\item $$- 16 x^{2} + 16 x - 23 = - 3 x^{2} + 2$$
\item $$x^{2} - 8 x - 20 = 0$$
\item $$- 7 x^{2} - 1 = 15 x^{2} + 2 x - 15$$
\item $$- 8 x^{2} - 24 x = - 13 x^{2}$$
\item $$- 7 y^{2} - 25 y = 3 y^{2} - 14 y + 23$$
\item $$- 3 x^{2} = 12 x^{2} - 4 x$$
\item $$x^{2} - 27 x + 180 = 0$$
\item $$y^{2} - 8 y + 12 = 0$$
\item $$x^{2} - 6 x - 391 = 0$$
\item $$x^{2} + 8 x + 7 = 0$$
\item $$x^{2} + 18 x + 65 = 0$$
\item $$- 20 x^{2} + 5 x - 21 = - 24 x^{2}$$
\item $$- 21 x^{2} + 20 x + 15 = 10 x$$
\item $$- 11 y^{2} = - 24 y^{2} + 25 y + 8$$
\item $$4 x^{2} + 10 x = 14 x$$
        \end{enumerate}
        \end{multicols}

        \section{Compute the derivative}
        ['x', 'y', 'z']
        \begin{multicols}{1}
        \begin{enumerate}
        \item $$\frac{d}{d x}\left(\frac{1}{x} \left(7 x^{2} + 12 x - 24\right)\right)$$
\item $$\frac{d}{d x}\left(\frac{2 \sqrt{x}}{- 9 x^{3} + 18 x + 7}\right)$$
\item $$\frac{d}{d x}\left(\frac{\log{\left (x \right )} + \tan{\left (x \right )}}{16 x^{3} - 23 x^{2} + 5 x}\right)$$
\item $$\frac{d}{d x}\left(\left(e^{x} + \tan{\left (x \right )}\right) e^{- x}\right)$$
\item $$\frac{d}{d x}\left(\left(19 x + e^{x}\right) e^{- x}\right)$$
\item $$\frac{d}{d x}\left(\frac{1}{x} \left(24 x^{2} + 7 x + \cos{\left (x \right )}\right)\right)$$
\item $$\frac{d}{d x}\left(\frac{1}{\tan{\left (x \right )}} \left(\log{\left (x \right )} + \sin{\left (x \right )}\right)\right)$$
\item $$\frac{d}{d x}\left(\frac{1}{x} \left(4 x^{3} + \log{\left (x \right )} - 17\right)\right)$$
\item $$\frac{d}{d x}\left(\frac{1}{\sin{\left (x \right )}} \left(- 17 x^{3} + 24 x^{2} + 14 x + \log{\left (x \right )} - 14\right)\right)$$
\item $$\frac{d}{d x}\left(\left(\sqrt{x} + \log{\left (x \right )}\right) e^{- x}\right)$$
        \end{enumerate}
        \end{multicols}

    \end{document}


\subsection{Power of tests}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\begmath 19.1 System Parameters

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

These subprograms provide values of various system parameters that
are needed in library subprograms and typically have different values
on different computer systems.  ``F. Supporting Information'' below
has instructions for customizing these programs for your system.

\subsection{Usage}

\subsubsection{Program Prototype}

\begin{description}
\item[REAL]  \ {\bf R1MACH, S}

\item[DOUBLE PRECISION]  \ {\bf D1MACH, D}

\item[INTEGER]  \ {\bf I1MACH, I, J}
\end{description}

Set J in the range 1 $\leq $ J $\leq $ 5 for R1MACH or D1MACH or 1 $\leq $ J
$\leq $ 16 for I1MACH. Then, use the appropriate one of the following
statements:
$$
\fbox{{\bf S = R1MACH(J)}}
$$
$$
\fbox{{\bf D = D1MACH(J)}}
$$
$$
\fbox{{\bf I = I1MACH(J)}}
$$
The results, S, D, or I are set as described in Section D.

\subsubsection{Argument Definitions}

\begin{description}
\item[J]  \ [in] Integer argument selecting the desired system parameter as
described in Section D.
\end{description}

\subsection{Examples and Remarks}

The program DRMACH lists all the values obtainable from R1MACH, D1MACH, and
I1MACH on the host computer system. Output is shown from several different
host systems.

\subsection{Functional Description}

For the purpose of this package a model of the Fortran~77 INTEGER, REAL, and
DOUBLE PRECISION number sets is characterized by a total of nine fundamental
parameters.

The model of Fortran~77 numbers of type INTEGER is parameterized by two
numbers, $a$ and $s$, where $a$ denotes the base (radix) of the number
system and $s$ denotes the maximum number of base $a$ digits available to
represent a Fortran integer. Thus the integers range from $-(a^s -1)$ to $%
a^s -1.$

The model of Fortran~77 numbers of type REAL is characterized by four
parameters, $b$, $t$, $emin$, and $emax$, where $b$ is the base of the
fraction part, $t$ is the number of base $b$ digits in the fraction part, $%
emin $ is the minimum exponent and $emax$ is the maximum exponent. The
magnitude of a floating-point number is thus of the form%
\begin{equation*}
b^e\left( c_1b^{-1}+c_2b^{-2}+...+c_tb^{-t}\right) ,
\end{equation*}
where, $emin\leq e\leq emax$, and the digits $c_i$ satisfy $0\leq c_i\leq
b-1.$ A nonzero floating-point number is normalized if and only if the digit
$c_1$ is nonzero. We shall consider only normalized floating-point numbers,
although numeric processors based on the IEEE standard also support a
range of unnormalized numbers.

Fortran~77 numbers of type DOUBLE PRECISION are modeled in the same form as
REAL numbers and are assumed to have the same base, $b$, but generally
different values of $t$, $emin$, and $emax$.

For some computer systems no setting of these parameters will make the model
system coincide exactly with the actual computer's number set. In such cases
the model parameters are selected so the model system will be as large a
subset of the actual number set as possible. In particular, for Cray
X/MP and Y/MP (but not T3D) systems the parameters $t$ and $emax$ are set
smaller and $emin$ is set larger than one might expect from the structure
of floating-point numbers on these systems. The reasons for this are
described in \cite{Schryer:1981:ATO}.

The values returned by this package are either prestored constants or are
computed at compile-time from prestored constants by use of expressions in
PARAMETER statements. Thus correct values must be determined and edited into
the package whenever this package is moved to a new computer system. Correct
values for many systems are present as comments in the source code.

On some systems, the Fortran compiler may not be able to compute
numbers in the full range of the model, in PARAMETER statements (this
disease usually strikes computation of the overflow limit).  On such
systems it may be necessary to restrict the model, $e.g.$ by reducing
$emax$.

{\bf Specification of Values Returned}
\begin{itemize}
\item[\bf J]  {\hspace{.5in} {\bf I1MACH (J)}}
\item[1]  Standard input unit number.
\item[2]  Standard output unit number.
\item[3]  Standard punch unit number.
\item[4]  Standard error message unit number.
\item[5]  Number of bits per Fortran integer storage unit.
\item[6]  Number of characters per Fortran integer storage unit.
\item[7]  $a$, the base for integers.
\item[8]  $s$, the number of base $a$ digits in an integer.
\item[9]  $a^s-1$, the largest integer magnitude.
\item[10]  $b$, the base for floating-point numbers. Assumed the same for
REAL and DOUBLE PRECISION arithmetic.
\item[11]  $t$, the number of base $b$ digits for REAL arithmetic.
\item[12]  $emin$, minimum exponent for REAL arithmetic.
\item[13]  $emax$, maximum exponent for REAL arithmetic.
\item[14]  $t$, the number of base $b$ digits for DOUBLE PRECISION
arithmetic.
\item[15]  $emin$, minimum exponent for DOUBLE PRECISION arithmetic.
\item[16]  $emax$, maximum exponent for DOUBLE PRECISION arithmetic.
\end{itemize}
\begin{itemize}
\item[\bf J]  {\hspace{.5in} {\bf R1MACH(J)}}
\item[1]  $b^{emin-1}$ smallest positive normalized REAL number,
(underflow limit).
\item[2]  $b^{emax}(1-b^{-t})$, largest REAL number, (overflow limit).
\item[3]  $b^{-t}$, smallest relative difference between two successive
nonzero REAL numbers. This is also the difference between~1.0 and the next
smaller REAL number.
\item[4]  $b^{-(t-1)}$ largest relative difference between two successive
nonzero REAL numbers. This is also the difference between~1.0 and the next
larger REAL number.
\item[5]  $\log _{10}b$, useful in certain conversions between base $b$ and
base~10.
\end{itemize}
The values returned by D1MACH are as described above for R1MACH with REAL
replaced by DOUBLE PRECISION.

\subparagraph{Historical perspective and relations to other languages}

The specifications of R1MACH, D1MACH, and I1MACH and the original
implementation were developed at the AT\&T Bell Laboratories, Murray Hill,
New Jersey, in the~1970's to support the development of portable
mathematical software, and specifically the PORT library,
\cite{Fox:1978:PMS}, which is a proprietary AT\&T Bell Laboratories
product.  These three subprograms were published as a subset of
Algorithm~528 in TOMS,~\cite{Fox:1978:AFP}, and are not proprietary.  The
MATH77 version has the same specification but is substantially different
in its implementation from the original versions.

The attributes associated with J = 1, 2, 3, 4, and~6 in I1MACH are less
relevant in Fortran~77 in the~90's than they were in Fortran~66 in the~70's.
In particular, only the DNLxxx and SNLxxx subroutines of
Chapter~9.3 access I1MACH(2).

Other MATH77 library subprograms use PRINT or WRITE(*,...) for printing.

Languages developed more recently than Fortran~77, such as Ada, ANSI C, and
Fortran~90, provide methods within the language to obtain certain
environmental parameters. Consider, for example, the underflow and overflow
limits, and precision for floating-point arithmetic. Using the present
package, these can be obtained for DOUBLE PRECISION arithmetic by
referencing D1MACH(1), D1MACH(2), and D1MACH(4), respectively, and for REAL
arithmetic by referencing R1MACH(1), R1MACH(2), and R1MACH(4). In Fortran~90
these parameters can be obtained by referencing the generic inquiry
functions TINY(X), HUGE(X), and EPSILON(X), where X may be any DOUBLE
PRECISION entity to obtain the values for DOUBLE PRECISION arithmetic, and
any REAL entity to obtain the values for REAL arithmetic. In ANSI C these
values for arithmetic of type $double$ are given by the macro names DBL\_MIN,
DBL\_MAX, and DBL\_EPSILON, for type $float$ there are FLT\_MIN, FLT\_MAX,
and FLT\_EPSILON, and for type $long\ double$ LDBL\_MIN, LDBL\_MAX, and
LDBL\_EPSILON.  All of these are defined in the standard
header file $float.h$.

\bibliography{math77}
\bibliographystyle{math77}

\subsection{Error Procedures and Restrictions}

If the argument is outside the range 1\ $\leq $ J $\leq $ 16 for I1MACH or
outside 1 $\leq $ J $\leq $ 5 for R1MACH or D1MACH, an error message is
printed and execution is terminated.

This package contains a partial protection against the inadvertent use of
the wrong version of one of these subroutines; say using the PC version on a
VAX. On the first call to any one of these subprograms, tests are done to
verify that two of the stored parameter values are not grossly wrong for the
current environment. These tests depend on assumptions about hardware,
compilers, and linkers that may be invalidated by technological changes.
Subroutines AMTEST and AMSUB1 are used to support these tests and are not
intended for any other usage.

\subsection{Supporting Information}

The source language is ANSI Fortran~77. All the program units are
grouped into a single file, AMACH.FOR. The filename may be different
on different systems, $e.g.$, ``amach.f" on UNIX systems.

One can either customize AMACH for a new system by commenting out
lines defining the parameters required for the system as it is
currently configured, and uncommenting lines required for the desired
system, or one can use the program {\tt m77con} described in
Chapter~19.4.  This requires making up a small control file {\tt
m77job}, and compiling, linking, and running {\tt m77con}, which is
self contained.  To make up {\tt m77job} for a VAX running UNIX, the
control file would contain the following.

{\tt SET SYS = VAX\newline
FILE amach.f}

Running {\tt m77con} with this control file and amach in the current
directory will generate a file {\tt amach.f} for the VAX.  If one
wants the extension ``{\tt .for}'' change the {\tt h.f} to {\tt
h.for}.  If one wants a machine other than the VAX, choose a value
for SYS (without any parenthetical remark) from the following table.
{\tt SYS=IEEE} covers any machine that uses the IEEE binary standard
for floating point arithmetic.  If your machine is not included in
this list, either pick a machine with the same parameters for the
floating point arithmetic as for a machine on this list, or enter
parameters for your machine as a new option into AMACH.

\begin{tabbing}
{\tt SYS = IEEE}\\
{\tt SYS = AMDAHL}\\
{\tt SYS = APOLLO\_10000}\\
{\tt SYS = BUR1700}\\
{\tt SYS = BUR5700}\\
{\tt SYS = BUR67\_7700}\\
{\tt SYS = CDC60\_7000}\\
{\tt SYS = CONVEXC\_1}\\
{\tt SYS = CRAY1}\\
{\tt SYS = CRAY1\_SD}\hspace{30pt}\=(Sngl prec.arith. used for dble.)\\
{\tt SYS = CRAY1\_64}\>(64 bit integers)\\
{\tt SYS = CRAY1\_SD\_64}\>(64 bit int, SP used for DP)\\
{\tt SYS = CRAY\_J90}\\
{\tt SYS = CRAY\_J90\_SD}\>(Sngl prec. used for dble.)\\
{\tt SYS = DG\_S2000}\\
{\tt SYS = HARRIS220}\\
{\tt SYS = HON600\_6000}\\
{\tt SYS = HON\_DPS\_8\_70}\\
{\tt SYS = HP700Q}\>(Q Precision on HP700 series)\\
{\tt SYS = IBM360\_370}\\
{\tt SYS = INTERDATA\_8\_32}\\
{\tt SYS = PDP10\_KA}\\
{\tt SYS = PDP10\_KB}\\
{\tt SYS = PDP11}\\
{\tt SYS = PRIME50}\\
{\tt SYS = SEQ\_BAL\_8000}\\
{\tt SYS = UNIVAC}\\
{\tt SYS = VAX}\\
{\tt SYS = VAX\_G}\\
{\tt SYS = ALPHA\_D3}
\end{tabbing}

Designed and programmed by P.A. Fox, A.D. Hall, and N.L. Schryer, AT\&T Bell
Laboratories, 1978. Adapted to the JPL MATH77 library, 1984 and~1987.

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.2in} {\bf Required Files}\vspace{2pt} \\
D1MACH & \hspace{.35in} AMACH\\
I1MACH & \hspace{.35in} AMACH\\
R1MACH & \hspace{.35in} AMACH\\\end{tabular}

\begcode

\medskip
\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRMACH}\vspace{0pt}
\lstinputlisting{\codeloc{mach}}

\centerline{{\bf \large Results from Various Machines\hspace{1in}}}

\begin{lstlisting}{}

              MACHINE CONSTANTS for IEEE Arithmetic
              -----------------------------------------
 J      I1MACH(J)       R1MACH(J)             D1MACH(J)

 1              5    0.11754944E-37    0.222507385850720100-307
 2              6    0.34028235E+39    0.179769313486231600+309
 3              7    0.59604645E-07    0.111022302462515700E-15
 4              6    0.11920929E-06    0.222044604925031300E-15
 5             32    0.30103001        0.301029995663981200
 6              4
 7              2
 8             31
 9     2147483647
10              2
11             24
12           -125
13            128
14             53
15          -1021
16           1024
\end{lstlisting}

\begin{lstlisting}{}

              MACHINE CONSTANTS for VAX
              -----------------------------------------
 J      I1MACH(J)       R1MACH(J)             D1MACH(J)
 1              5    0.29387359E-38    0.293873587705571877E-38
 2              6    0.17014117E+39    0.170141183460469229E+39
 3              7    0.59604645E-07    0.138777878078144568E-16
 4              6    0.11920929E-06    0.277555756156289135E-16
 5             32    0.30103001        0.301029995663981198
 6              4
 7              2
 8             31
 9     2147483647
10              2
11             24
12           -127
13            127
14             56
15           -127
16            127
\end{lstlisting}

\newpage
\begin{lstlisting}{}

              MACHINE CONSTANTS for CRAY J90
              -----------------------------------------
 J      I1MACH(J)       R1MACH(J)             D1MACH(J)
 1              5    0.73344155-2465   0.733441547021938866-2465
 2              6    0.13634352+2466   0.136343516952426991+2466
 3            102    0.71054274E-14    0.504870979341447555E-28
 4              6    0.14210855E-13    0.100974195868289511E-27
 5             64    0.30103000        0.301029995663981195
 6              8
 7              2
 8             46
 9 70368744177663
10              2
11             47
12          -8188
13           8189
14             94
15          -8188
16           8189

              MACHINE CONSTANTS for UNISYS 1100
              -----------------------------------------
 J      I1MACH(J)       R1MACH(J)             D1MACH(J)
 1              5     .14693679-038     .278134232313400172-308
 2              6     .17014118+039     .898846567431157951+308
 3              7     .74505806-008     .867361737988403547-018
 4              6     .14901161-007     .173472347597680709-017
 5             36     .30103000         .301029995663981194
 6              4
 7              2
 8             35
 9    34359738367
10              2
11             27
12           -128
13            127
14             60
15          -1024
16           1023
\end{lstlisting}
\end{document}


\graphicspath{{Chapter2/Figs/}}

\section{Model validation with simulated data} \label{section:mofa_simulated}

We used simulated data from the generative model to systematically test the technical capabilities of MOFA.

\subsection{Recovery of simulated factors}

First, we tested the ability of MOFA to recover simulated factors under varying number of views, features, factors and with different amounts of missing values.\\
For every simulation scenario we initialised a model with a high number of factors ($K=100$), and inactive factors were automatically dropped during model training by the ARD prior. In addition, to test the robustness under different random initialisations, 10 model instances were trained for every simulation scenario.\\
We observe that in most settings the model accurately recovers the correct number of factors (\Cref{fig:MOFA_learnK}). Exceptions occur when the dimensionality of the latent space is too large (more than 50 factors) or when an excessive amount of missing values (more than 80\%) is present in the data.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.9\textwidth]{MOFA_learnK}
	\caption{\textbf{Assessing the ability to recover simulated factors}.\\
	In all plots the y-axis displays the number of infered factors. (a) x-axis displays the number of true factors, and boxplots summarise the distribution of inferred factors across 10 model instances. For (b-d) the true number of factors was set to $K=10$ and each bar corresponds to a different model instance. (b) x-axis displays the number of features, (c) x-axis displays the number of views, (d) x-axis displays fraction of missing values. }
	\label{fig:MOFA_learnK}
\end{figure}

\subsubsection{View-wise sparsity on the weights}

One of the essential features of MOFA is the use of an ARD prior aimed at disentangling the activity of factors across views (see \Cref{section:ard} and \Cref{mofa:model_description}).\\
We simulated data from the generative model such that the factors were set to be active or inactive in specific views by sampling $\alpha_{k}^{m}$ from a discrete distribution with values $\{ 1, 1e3\}$. We compared the performance with a popular integrative clustering method (iCluster) that is also formulated as a latent variable model \cite{Mo2013}. In iCluster each factor shares the same sparsity constraint across all views, and hence the model is less accurate at detecting factors that show differential activity across different views:

\begin{figure}[H]
	\centering
	\includegraphics[width=0.9\textwidth]{MOFA_group_sparsity}
	\caption{\textbf{Evaluating the ability to recover differential factor activity across views}.\\
	(a) The true activity pattern, with factors sampled to display differential activity across views.
	(b) Percentage of variance explained for each factor in each view, for MOFA and iCluster \cite{Mo2013}.}
	\label{fig:MOFA_group_sparsity}
\end{figure}

\subsubsection{Feature-wise sparsity on the weights} \label{section:spike_slab}

In MOFA we implemented a spike-and-slab prior prior to enforce feature-wise sparsity on the weights with the aim of delivering a more interpretable solution (see \Cref{section:mofa_weights}).\\
To assess the effect of the spike-and-slab prior we trained a group of models with and without the spike-and-slab prior. Importantly, both models contain the ARD prior, which should provide some degree of regularisation. To compare both options to a non-sparse method, we also fit a Principal Component Analysis on the concatenated data set. As expected, we observe that the spike-and-slab prior induces more zero-inflated weights, although the ARD prior provided a moderate degree of regularisation. The PCA solution was notably more dense than both Bayesian models (\Cref{fig:MOFA_sparsity}).

\begin{figure}[H]
	\centering
	\includegraphics[width=0.7\textwidth]{MOFA_sparsity}
	\caption{\textbf{Assessing the sparsity priors on the weights}.\\
	The plot shows the empirical cumulative density function of the weights for an arbitrary factor in a single view. The weights were simulated with a sparsity level of $\theta_k^m=0.5$ (50\% of active features.)
	}
	\label{fig:MOFA_sparsity}
\end{figure}

\subsection{Non-Gaussian likelihoods}  \label{section:mofa_nongaussian_results}

A key improvement of MOFA with respect to previous methods is the use of non-Gaussian likelihoods to integrate data modalities with different types of readouts. In particular, as described in \Cref{section:mofa_ngaussian}, we implemented a Bernoulli likelihood to model binary data and a Poisson likelihood to model count data.\\
To validate both likelihood models, we simulated binary and count data using the generative model and we fit two sets of models for each data type: a group of models with a Gaussian likelihood and a group of models with a Bernoulli or Poisson likelihood, respectively.\\
Reassuringly, we observe that although the Gaussian likelihood is also able to recover the true number of factors, the models with the non-Gaussian likelihoods result in a better fit to the data:

\begin{figure}[H]
	\centering
	\includegraphics[width=0.8\textwidth]{MOFA_nongaussian}
	\caption{\textbf{Validation of the non-Gaussian likelihood models using simulated data}.\\
	(a-d) Comparison of Poisson and Gaussian likelihood models applied to count data.\\
	(e-h) Comparison of Bernoulli and Gaussian likelihood models applied to binary data.\\
	(a,e) The y-axis displays the ELBO for each model instance (x-axis). (b,f) The y-axis displays the mean reconstruction error for each model instance (x-axis). (c,g) The y-axis displays the number of estimated factrors for each model instance (x-axis). The horizontal dashed line marks the true number of factors $K=10$. (d,h) Distribution of reconstructed data. Plotted are the expected values of the inferred posterior distributions, not samples from the corresponding posteriors. This is why reconstructed measurements are continuous and not discrete.
	}
	\label{fig:MOFA_nongaussian}
\end{figure}

\subsection{Scalability}

Finally, we evaluated the scalability of the model when varying each of its dimensions independently, and we compared the speed with an implementation of GFA that uses Gibbs Sampling \cite{Leppaaho2017} and the popular Cluster+\cite{Mo2013}, which adopts a maximum-likelihood approach with grid search to optimise the hyperparameters. Overall, we observe that MOFA scales linear with respect to all dimensions and is significantly faster than any of the three evaluated techniques (\Cref{fig:MOFA_scalability}).

\begin{figure}[H]
	\centering
	\includegraphics[width=0.75\textwidth]{MOFA_scalability}
	\caption{\textbf{Evaluation of scalability in MOFA}.\\
	Shown is the time required for convergence (y-axis, in minutes). The x-axis displays the value of the dimension that was tested, either number of factors ($K$), number of features ($D$), number of samples ($N$) and number of views ($M$). Baseline parameters were $M=3, K=10, D=1000, N=100$. Each line represents a different model, GFA (red), MOFA (blue) and iCluster (green). Default convergence criteria where used for all methods. Each dot displays the average time across 10 trials with error bars denoting the standard deviation. iCluster is only shown for one value as all other settings required more than 200min for convergence.
	}
	\label{fig:MOFA_scalability}
\end{figure}

As a real application showcase, the training on the CLL cohort that is described below (\Cref{fig:MOFA_CLL_overview}) required 25 minutes using MOFA, 34 hours with GFA and 5-6 days with iCluster.

% \subsection{Class imbalance}  \label{section:class_imbalance}
% TO-DO.
%The objective function (the evidence lower bound, ELBO) does not weight the different data modalities according to the number of features. Hence, in general the larger the number of features
%Having said this, we find the model to be very robust to differences in the dimensionality of the feature space.

\newpage

\section{Application to a cohort of Chronic Lymphocytic Leukaemia patients} \label{section:mofa_cll}

Personalised medicine is an attractive field for the use of multi-omics, as dissecting heterogeneity across patients is a major challenge in complex diseases, and requires data integration from multiple biological layers \cite{Chen2013,Costello2014,Alyass2015}.

To demonstrate the potential of MOFA, we applied it to a publicly available study of 200 patient samples of Chronic Lymphocytic Leukaemia (CLL) profiled for somatic mutations, RNA expression, DNA methylation and \textit{ex vivo} drug responses \cite{Dietrich2018}, all of them at the bulk level. We selected this data set for three main reasons: (1) The complex missing data structure, with nearly 40\% samples having incomplete assays (\Cref{fig:MOFA_CLL_overview}). As described in \Cref{section:mofa_missing_values}, the inference framework implemented in MOFA should cope with large amounts of missing values, including missing assays. (2) After data processing, three assays had continuous observations whereas for the somatic mutations the observations were binary. As described in \Cref{section:mofa_ngaussian}, MOFA can combine different likelihood models. (3) The existence of clinical covariates provide an excellent test to evaluate whether the MOFA factors can capture the molecular variation that underlies clinically-relevant phenotypes.

\subsection{Data overview and processing}

Data processing and normalisation is essential for the model to work and it requires a few considerations. First, in the case of count-based assays such as RNA-seq one needs to remove differences in library size between samples. If not done correctly, the signal in the data will be dominated by this (undesired) source of variation, and more subtle heterogeneity will be harder to identify. Similarly, batch effects and other undesired technical sources of variation should be regressed out \textit{a priori}, although this was not the case for this particular data set. Second, feature selection must be performed by selecting highly variable features. A proper feature selection will increase the signal-to-noise-ratio, it will simplify model selection and it will speed up the training procedure.  Finally, as discussed above, the total number of features can influence the contribution of a data modality to the latent space. To mitigate this problem it is recommended to keep the number of features per view within the same order of magnitude, when possible.

Here we proceed to briefly describe the different data modalities and outline the basic data processing steps that we performed before applying MOFA:
\begin{itemize}
	\item \textbf{RNA expression} was profiled using bulk RNA-seq. Genes with low counts were filtered out and the data was subsequently normalized using DESeq2 \cite{Love2014}. Feature selection was performed by considering the top 5,000 most variable genes.
	\item \textbf{DNA methylation} was profiled using Illumina 450K arrays. We converted the beta-values to M-values, as it has better statistical properties when modelled with a Gaussian distribution \cite{Du2010}. Feature selection was performed by considering the top 1\% most variable CpG sites.
	\item \textbf{\textit{Ex vivo} Drug response} was screened using the ATP-based CellTiter-Glo assay. Briefly, the asay includes a panel of 62 drugs at 5 different concentrations each, for a total of 310 measurements. The readout is a number proportional to the fraction of viable cells in culture based on quantitation of the ATP present, which signals the presence of metabolically active cells.
	\item \textbf{Somatic mutations} were profiled using a combination of targeted and whole exome sequencing. Feature selection was performed by considering only mutations that were present in at least three samples, which resulted in a total of 69 mutations.
\end{itemize}

% For more details on the data generation steps we refer the reader to \cite{Dietrich2018}.

\subsection{Model overview}

In this data set, MOFA recovered $K=10$ factors, each one explaining a minimum of 3\% of variance in at least one assay. Interestingly, MOFA detected factors which are shared across several data modalities (Factors 1 and 2, sorted by variance explained). Some factors captured sources of covariation between two data modalities (Factor 3 and 5, active in the RNA expression and drug response). In addition, some factors captured variation that is unique to a single data modality (Factor 4, active in the RNA expression data).\\
All together, the 10 MOFA factors explained 41\% of variance in the drug response data, 38\% in the mRNA expression, 24\% in the DNA methylation and 24\% in somatic mutations.

\begin{figure}[H]
	\centering
	\includegraphics[width=1.0\textwidth]{MOFA_CLL_overview}
	\caption{\textbf{Application of MOFA to a study of chronic lymphocytic leukaemia. Model overview.}\\
	(a) Data overview. Assays are shown in different rows ($D$ = number of features) and samples ($N$) in columns, with missing samples shown using grey bars. Notice that some samples are missing entire assays.\\
	(b) Variance explained (\%) by each Factor in each assay.\\
	(c) Total variance explained (\%) for each assay by all factors.
	}
	\label{fig:MOFA_CLL_overview}
\end{figure}

The first two factors are the most interesting from a molecular perspective, as they capture a phenotypic effect that is manifested across multiple molecular layers. To annotate Factors 1 and 2 we proceeded to visualise the feature weights, starting by the (binary) somatic mutation data, as it is the simplest data modality to interpret. Inspection of the top weights revealed that Factor 1 was associated with the mutation status of the immunoglobulin heavy-chain variable (IGHV) region, while Factor 2 was aligned with trisomy of chromosome 12 (\Cref{fig:MOFA_CLL_factors12}).\\
Remarkably, in a completely unsupervised fashion, MOFA recovered the two most important clinical markers in CLL as the two major axes of molecular disease heterogeneity \cite{Fabbri2016,Bulian2017,Crombie2017}.

Next, we visualised the samples in the latent space spanned by Factors 1 and 2. A scatterplot based on these factors shows a clear separation of patients by their IGHV status on the first Factor and presence or absence of trisomy 12 on the second Factor (\Cref{fig:MOFA_CLL_factors12}). Interestingly, 24 patients lacked IGHV status measurements (grey crosses) due to quality control filtering in the DNA sequencing assay. Nonetheless, MOFA was able to pool information from the other molecular layers to map those samples to the latent space, and could be classified to the corresponding molecular subgroup.

\begin{figure}[H]
	\centering
	\includegraphics[width=1.0\textwidth]{MOFA_CLL_factors12}
	\caption{\textbf{Visualisation of the genetic signature underlying Factor 1 and 2}\\
	(a) Weights of the top somatic mutations for Factors 1 and 2.
	(b) Scatterplots of Factors 1 and 2. Each dot corresponds to one sample and the colours denote the IGHV status of the tumours samples; symbol shape indicate chromosome 12 trisomy status.
	}
	\label{fig:MOFA_CLL_factors12}
\end{figure}

IGHV status is currently the most important prognostic marker in CLL and has routinely been used to distinguish between two distinct subtypes of the disease\cite{Fabbri2016}. Molecularly, it is a surrogate of the level of activation of the B-cell receptor, which is in turn related to the differentiation state of the tumoral cells. Multiple studies have associated mutated IGHV with a better response to chemotherapy, whereas unmutated IGHV patients have a worse prognosis \cite{Fabbri2016,Bulian2017,Crombie2017}.\\
In clinical practice, the IGHV status has been considered binary. Our results suggest that this is a fairly good approximation, but a more complex structure with at least three groups or a potential underlying continuum is supported (\Cref{fig:MOFA_CLL_factors12,fig:MOFA_CLL_Factor1}), as also suggested in \cite{Queiros2015}.

% \subsection{Detection of outlier samples}

% Interestingly, there is some discrepancy between the IGHV status predicted by MOFA and the IGHV status reported in the clinical data. Out the 200 patients, MOFA classifies 176 in accordance with the clinical label, it classifies 12 patients that lacked the clinical marker and it re-classifies 12 patients to the opposite group.\\
% To validate the MOFA-based classification, we proceeded to inspect the molecular data in more detail.

% sample-to-sample correlation matrices for the individual layers suggest that for 3 of the cases where the inferred factor disagrees with the clinical label, the molecular data supports the predicted label. The other 9 cases showed intermediate molecular sgnatures now well captured by the binary classification.
% %Based on these results, we hypothesize that a multi-omics approach based on several molecular signatures could be a more precise and robust approach to predict clinical phenotypes than the use of single features such as mutations or expression of marker genes.

% \begin{figure}[H]
% 	\centering
% 	\includegraphics[width=1.0\textwidth]{MOFA_IGHV_outlier}
% 	\caption{XX}
% 	\label{fig:MOFA_IGHV_outlier}
% \end{figure}

\subsection{Molecular characterisation of Factor 1}
% \tabularnewline

An important step in the MOFA pipeline is the characterisation of the molecular signatures underlying each Factor. I will demonstrate this for Factor 1, although a similar strategy can be applied to Factor 2.

On the RNA expression, inspection of the top weights pinpoint genes that have been previously associated to IGHV status, some of which have been proposed as clinical markers\cite{Vasconcelos2005,Morabito2015}. Heatmaps of the RNA expression levels for these genes reveals clear differences between samples when ordered according to the Factor 1 values.

On the drug response data the weights highlight kinase inhibitors targeting the B-cell receptor pathway. Splitting the patients into three groups based on k-means clustering shows clear separation in the drug response curves.

% Copied
\begin{figure}[H]
	\centering
	\includegraphics[width=0.90\textwidth]{MOFA_CLL_Factor1}
	\caption{
	\textbf{Characterization of MOFA Factor 1 as IGHV status.}\\
	(a) Beeswarm plot of Factor 1 values, where each dot corresponds to a patient sample. Colours denote three groups found by applying 3-means clustering on the Factor values.\\
	(b) Genes with the largest weights (in absolute values) in the mRNA data. Plus or minus symbols on the right indicate the sign of the weight.\\
	(c) Heatmap of gene expression values for the genes with the largest weights displayed in (b).\\
	(d) Drugs with the largest weights (in absolute values) in the Drug response data, coloured by the drug's target category.\\
	(e) Drug response curves for two of the drugs with top weights, stratified by the clusters displayed in (a).
	}
	\label{fig:MOFA_CLL_Factor1}
\end{figure}

\subsection{Molecular characterisation of other factors}

Despite their clinical importance, Factor 1 (IGHV status) and Factor 2 (chr12 trisomy) they explain less than 20\% variability in each data modality, suggesting the existence of more subtle sources of variation. As an example, we will also characterise Factor 5, which explains 2\% of the variance in the mRNA and 6\% of variance in the drug response.\\
As mentioned in \Cref{mofa:downstream}, instead of exploring the feature weights individually, factors can be annotated using gene set annotations. This procedure is particularly appealing for RNA expression data, as a rich amount of resources exist that have categorised genes into ontologies in terms of biological pathways, molecular function and cellular components  \cite{Fabregat2015,Ashburner2000}.

Briefly, the idea is to aggregate the weights using prior information to obtain a single statistic for each gene set, which can be tested against a competitive null hypothesis. Inspired from \cite{Frost2015}, in MOFA we implemented several scoring schemes and a variety of parametric and unparametric statistical tests. By default we use the weights as feature statistics and the average difference in the weight values as the feature set statistic. P-values are then obtained per feature set and factor via a simple t-test.

Gene Set Enrichment Analysis on the RNA weights using the Reactome annotations \cite{Fabregat2015} reveals that Factor 2 is strongly enriched for oxidative stress and senescence pathways. Inspection of the top features highlights the importance of heat shock proteins (HSPs), a group of proteins that are essential for protein stability which are up-regulated upon stress conditions like high temperatures, pH shift or oxidative stress. Importantly, HSPs can be elevated in tumour cells and potentially contribute to prolonged tumour cell survival\cite{Dempsey2010}. In agreement with the findings from the mRNA view, the drugs with largest weights on Factor 5 belong to clinical categories associated with stress response, such as target reactive oxygen species and DNA damage response (\Cref{fig:MOFA_CLL_Factor5})

\begin{figure}[H]
	\centering
	\includegraphics[width=0.95\textwidth]{MOFA_CLL_Factor5}
	\caption{
	\textbf{Characterization of Factor 5 in the CLL cohort as oxidative stress response.}\\
	(a) Beeswarm  plot of Factor 5, where each dot corresponds to a patient sample. Colours represent the expression of TNF, an inflammatory stress marker that is present among the top mRNA weights.\\
	(b) Gene set enrichment analysis results using Reactome pathways. Displayed are the top pathways with the strongest enrichment.\\
	(c) Heatmap of mRNA expression values for representative genes among the top weights. Samples are ordered by their Factor 5 values.\\
	(d) Weights for the top drugs, annotated by target category.\\
	(e) Heatmap of drug response values for the top three drugs. Samples are ordered by their Factor 5 values, as in (c).
	}
	\label{fig:MOFA_CLL_Factor5}
\end{figure}

\subsection{Prediction of clinical outcomes}

We conjectured that the integration of multiple molecular layers could allow an improved prediction of the patients' clinical outcome. To evaluate the utility of the MOFA factors as predictors of clinical outcomes we fit Cox regression models \cite{Cox1972} using the patients' time to next treatment (TTT) as a response variable. Two types of analysis were performed: a univariate analysis where each Factor was independently associated with TTT, and a multivariate analysis where the combination of all factors were used to predict TTT (\Cref{fig:MOFA_CLL_Cox}). In the univariate Cox models, we observe  that Factor 1 (IGHV status), Factor 7 (associated with chemo-immunotherapy treatment prior to sample collection) and Factor 8 (enriched for Wnt signalling) were significant predictors of TTT. Accordingly, when splitting patients into binary groups based on the corresponding Factor values, we observe clear differences in the survival curves. In the multivariate Cox model, MOFA (Harrell's C-Index C=0.78) outperformed all other input settings, including PCA on single-omic data (C=0.68-0.72), individual genetic markers (C=0.66) as well PCA applied to the concatenated data matrix (C=0.74).

% Caption copied
\begin{figure}[H]
	\centering
	\includegraphics[width=0.9\textwidth]{MOFA_CLL_Cox}
	\caption{
	\textbf{Association analysis between MOFA factors and clinical putcome.}\\
	(a) Association of MOFA factors to time to next treatment using a univariate Cox regression model Error bars denote 95\% confidence intervals. Numbers on the right show p-values for each Factor.\\
	(b) Kaplan-Meier plots for the three MOFA factors that show a significant association with time to next treatment.\\
	(c) Prediction accuracy of time to treatment using multivariate Cox regression trained with the first 10 principal components applied to single data modalities, the full data set or the 10 MOFA factors. Shown are average values of Harrell's C-index from fivefold cross-validation. Error bars denote standard error of the mean.
	}
	\label{fig:MOFA_CLL_Cox}
\end{figure}

\subsection{Imputation of missing values}

A promising application of MOFA is the imputation of missing values, including the potential to impute of entire assays.\\
The principle of imputation in MOFA follows the same logic as simulating from the generative model: if the factors and weights are known, the input data can be reconstructed by a simple matrix multiplication:
\[
	\hat{\bfY} = \E[\bfZ] \E[\bfW]^T
\]
where $\E[\bfZ]$ and $\E[\bfW]$ denote the expected values of the variational distributions for the factors and the weights, respectively. Notice that, when using the expectations of the posterior distributions, the noise $\epsilon$ (\Cref{mofa_master_equation}) has a mean of zero and does not contribute to the predictions.\\
The equation above results in point estimates, but it ignores the uncertainity on $\bfZ$ and $\bfW$. Instead of relying in point estimates, one could adopt a more Bayesian approach and calculate the posterior predictive distribution by propagating the uncertainity \cite{Gelman2013}. Nonetheless, due to the nature of the optimisation problem in variational inference, the variance of the posterior distributions can be underestimated (see \Cref{section:expectation_propagation}). In addition, this would be substantially more complex to implement and would result in a significant increase in computational complexity, hence we did not implement this strategy.

To assess the imputation performance, we trained MOFA models using a data set of complete measurements (a total of N=121 samples) after masking parts of the drug response measurements. In a first experiment, we masked values at random, and in a second experiment we masked the entire drug response measurements. We compared the imputation accuracy of MOFA to some established imputation strategies, including imputation by feature-wise mean, SoftImpute \cite{Mazumder2010}, and a k-nearest neighbour method \cite{Troyanskaya2001}. For both imputation tasks, MOFA consistently yielded more accurate predictions, albeit the differences are less pronounced in the imputation of full assays, a significantly more challenging task.\\

\begin{figure}[H]
	\centering
	\includegraphics[width=0.9\textwidth]{MOFA_imputation}
	\caption{\textbf{Evaluation of imputation performance in the drug response assay.}\\
	The y-axis shows the mean-squared error (MSE) across 15 trials for increasing fractions of missing data (x-axis). Two experiments were considered: (a) values missing at random and (b) entire assays missing at random. Each point displays the mean across all trials and the error bars depict the corresponding standard deviations.}
	\label{fig:MOFA_imputation}
\end{figure}

\newpage

\section{Application to single-cell multi-omics} \label{section:mofa_scmt}

The emergence of single-cell multi-modal techniques has created opportunities for the development of novel computational strategies \cite{Stuart2019,Colome-Tatche2018,Chappell2018}.\\
To show case how MOFA can be used to integrate single-cell multi-omics data, we considered a simple data set that consists of 87 ESCs where RNA expression and DNA methylation were simultaneously measured using scM\&T-seq\cite{Angermueller2016}. Two populations of ESCs were profiled: the first one contains 16 cells grown in 2i media, which is known to induce a naive pluripotency state associated with genome-wide DNA hypomethylation \cite{Ficz2013}. The second population contains 71 cells grown in serum media, which contain stimuli that trigger a primed pluripotency state poised for differentiation \cite{Tosolini2016}.

\subsection{Data processing}

The RNA expression data was processed using \textit{scran}\cite{Lun2016b} to obtain log normalised counts adjusted by library size. Feature selection was performed by selecting the top 5,000 most overdispersed genes\cite{Lun2016a}. A Gaussian likelihod was used for this data modality. \\
The DNA methylation data was processed as described in Chapter 2. Briefly, for each CpG site, we calculated a binary methylation rate from the ratio of methylated read counts to total read counts. Next, CpG sites were classified by overlapping with genomic contexts, namely promoters, CpG islands and enhancers (distal H3K27ac peaks). Finally, for each annotation we selected the top 5,000 most variable CpG sites with a minimum coverage of 10\% across cells. Each of the resulting matrices was defined as a separate view for MOFA. A Bernoulli likelihod was used for this data modality.

\subsection{Model overview}

In this data set, MOFA inferred 3 factors with a minimum explained variance of 1\% (\Cref{fig:mofa_scMT}). Factor 1 captured the transition from naive to primed pluripotent states, which MOFA links to widespread coordinated changes between DNA methylation and RNA expression. Inspection of the gene weights for Factor 1 pinpoints important pluripotency markers including  \textit{Rex1/Zpf42} or \textit{Essrb} \cite{Mohammed2017}. As previously described both \textit{in vitro} \cite{Angermueller2016} and \textit{in vivo} \cite{Auclair2014}, the transition from naive to primed pluripotency state is concomitant with a genome-wide increase in DNA methylation levels. Factor 2 captured a second dimension of heterogeneity driven by the transition from a primed pluripotency state to a differentiated state, with RNA weights enriched with canonical differentiation markers including keratins and annexins \cite{Fuchs1988}.\\
Jointly, the combination of Factors 1 and 2 reconstruct the coordinated changes between the transcriptome and the epigenome along the differentiation trajectory from naive pluripotent cells to differentiated cells.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.9\textwidth]{MOFA_scMT}
	\caption{\textbf{MOFA recovers a differentiation process from a single-cell multi-omics data set.} \\
	(a) Overview of the data modalities. Rows indicate number of features ($D$) and columns indicate number of samples ($N$). Grey bars denote missing samples.\\
	(b) Fraction of variance explained per factor (column) and view (row).\\
	(c) Cumulative fraction of variance explained per view (across all factors).\\
	(d) mRNA weights of Factor 1 (bottom) and Factor 2 (top). The genes that are labelled are known markers of pluripotency (for Factor 1) or differentiation (for Factor 2). \\
	(e) Scatter plot of Factor 1 (x-axis) against Factor 2 (y-axis). Cells are colored based on the culture condition. Grey arrow illustrates the differentiation trajectory from a naive pluripotency state to a differentiated state.
	}
	\label{fig:mofa_scMT}
\end{figure}

% \begin{figure}[H]
% 	\centering
% 	\includegraphics[width=0.8\textwidth]{MOFA_scMT2}
% 	\caption{XX}
% 	\label{fig:MOFA_scMT2}
% \end{figure}

% \subsubsection{Comparison with clustering strategies}

% To illustrate the importance of learning continuous latent spaces before clustering samples, we applied popular integrative clustering algorithms \cite{Wang2014,Shen2009,Mo2013} to the data set. As expected, two clusters can be recovered that broadly match the culture conditions. However, no trajectory is recovered, illustrating the importance of

% \begin{figure}[H]
% 	\centering
% 	\includegraphics[width=1.0\textwidth]{MOFA_scMT_clustering}
% 	\caption{\textbf{Multi-omics clustering applied to scMT data set.}\\
% 	(a) Similarity matrix and dendogram obtained using Similarity Network Fusion\cite{Wang2014}\\
% 	(b) Dendrogram obtained using iClusterPlus\cite{Mo2013} with two clusters.
% 	}
% 	\label{fig:MOFA_scMT_clustering}
% \end{figure}


\documentclass[a4paper,12pt]{article}

% Standard configuration
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{hyperref}
\usepackage{setspace}
\onehalfspacing
\usepackage{upquote}
\usepackage{amsmath,amssymb}
\usepackage{tikz}
\usepackage[margin=1.3in]{geometry}
\usepackage[numbers]{natbib}

% Listings configuration
\usepackage{color}
\definecolor{listinggray}{gray}{0.9}
\usepackage{multirow,bigdelim}
\usepackage{listings}
\lstset{
	language=,
	literate=
		{æ}{{\ae}}1
		{ø}{{\o}}1
		{å}{{\aa}}1
		{Æ}{{\AE}}1
		{Ø}{{\O}}1
		{Å}{{\AA}}1,
	backgroundcolor=\color{listinggray},
	tabsize=3,
	rulecolor=,
	basicstyle=\scriptsize,
	upquote=true,
	aboveskip={1.5\baselineskip},
	columns=fixed,
	showstringspaces=false,
	extendedchars=true,
	breaklines=true,
	prebreak =\raisebox{0ex}[0ex][0ex]{\ensuremath{\hookleftarrow}},
	frame=single,
	showtabs=false,
	showspaces=false,
	showstringspaces=false,
	identifierstyle=\ttfamily,
	keywordstyle=\color[rgb]{0,0,1},
	commentstyle=\color[rgb]{0.133,0.545,0.133},
	stringstyle=\color[rgb]{0.627,0.126,0.941},
}
%captions on listings
\usepackage[center,font=small,labelfont=bf,textfont=it]{caption}

% Header configuration
\usepackage{fancyhdr}
\lhead[]{} %clear standard settings
\chead[]{} %clear standard settings
\rhead[]{} %\rightmark} %current section
\lfoot[]{} %clear standard settings
\cfoot[]{\thepage} %current page number
\rfoot[]{} %clear standard settings

\title{OSM\\G assignment 1}
\author{Tobias Hallundbæk Petersen (xtv657)\\Ola Rønning (vdl761)\\Nikolaj Høyer (ctl533)}
\date{February 17, 2014}

\begin{document}
\maketitle
\tableofcontents
\newpage
\section{A space-efficient doubly-linked list}
The trick in the exercise is how to save two pointer addresses in one pointer. This can, in the case of a doubly-linked list be done with XOR. Here every element holds the pointer to the element behind it XOR'ed with the pointer to the element in front of it, except for the head and the tail, as they would only need to point inwards or outwards. We use the notation that $P_n$ is the pointer of element $n$, and $p_n$ is the pointer to element $n$. With this system we can find the pointer to the element $p_n$ by the following logic: $ p_n = P_{n-1} \oplus p_{n-2} = P_{n+1} \oplus p_{n+2} $. And since we know the pointer to the head: $p_1$, as well as the pointer of the head $P_1$, which points to element 2 we can find: $p_3 = p_1 \oplus P_2$, and this way we can traverse the list.
\subsection{Insert}
For inserting we just need to set the pointer of the head or tail to the new element, and we need to XOR the pointer of the head/tail, with the pointer of the new head.
\begin{lstlisting}
void insert(dlist *this, item *thing, bool atTail) {
  node* n = (node*) malloc(sizeof(node));
  (*n).thing = thing;
  (*n).ptr = n;
  if((*this).head == NULL) {
    (*this).tail = n;
    (*this).head = n;
    return;
  }
  node* dlistNode = (atTail) ? (*this).tail : (*this).head;
  dlistNode -> ptr = (node*) ((long) dlistNode -> ptr ^ (long) n);
  (*n).ptr = dlistNode;
  (atTail) ? ((*this).tail = n) : ((*this).head = n);
}
\end{lstlisting}
\newpage
\subsection{Extract}
For extraction we simple remove the head or tail, and XOR the next element in line with the pointer to the head or tail, furthermore we change the lists head or tail with the new pointer.
\begin{lstlisting}
item* extract(dlist *this, bool atTail) {
  node* returnNode = (*this).head;
  item* value;
  if(returnNode == ((*returnNode).ptr)) {
    (*this).tail = NULL;
    (*this).head = NULL;
    value = (*returnNode).thing;
    free(returnNode);
    return value;
  }
  if (atTail) {
    returnNode = (*this).tail;
    (*this).tail = (*returnNode).ptr;
    (*this).tail -> ptr = (node*) ((long)(*this).tail -> ptr ^ (long) returnNode);
    value = (*returnNode).thing;
    free(returnNode);
    return value;
  }
  (*this).head = (*returnNode).ptr;
  (*this).head -> ptr = (node*) ((long)(*this).head -> ptr ^ (long) returnNode);
  value = (*returnNode).thing;
  free(returnNode);
  return value;
}
\end{lstlisting}
\newpage
\subsection{Search}
Search works by traversing the list in the manner described in the beginning of this section, and then using the function pointer of the parameters, to match on all of the elements. We have chosen to return \texttt{NULL} in the case where no element matches. As this search function at most looks at every single element, we can see that it holds a runtime of $O(n)$
\begin{lstlisting}
item* search(dlist *this, bool (*matches)(item*)){
  node* itNode = this -> head;
  node* oldPtr = (node*) 0;
  node* tempPtr;
  if (matches(itNode -> thing)) {
    return itNode -> thing;
  }
  while (itNode != this -> tail) {
    tempPtr = itNode;
    itNode = (node*) ((long) itNode -> ptr ^ (long) oldPtr);
    oldPtr = tempPtr;
    if (matches(itNode -> thing)) {
      return itNode -> thing;
    }
  }
  return NULL;
}
\end{lstlisting}

\subsection{Reverse}
Reverse is simple, as the only thing needed to be done is to swap the head with the tail. This of course yields a run time of $O(1)$.
\begin{lstlisting}
void reverse (dlist *this) {
  void* temp = ((*this).head);
  (*this).head = (*this).tail;
  (*this).tail = temp;
}
\end{lstlisting}
\newpage
\subsection{Testing}
The following main function holds the test cases that we run, with insertion and extraction at the head and tail, furthermore a search function that finds elements that have a thing that is equal to the integer 5, is implemented. And with that we test the search function.
\begin{lstlisting}
void main() {
  dlist list = {0};
  int i = 5;
  int i1 = 6;
  int i2 = 7;
  bool b = 0;
  bool b1 = 1;
  insert(&list,&i,b);
  insert(&list,&i1,b);
  insert(&list,&i2,b);
  bool (*match)(item*) = find;
  int* s = search(&list,match);
  printf("The search found:%2d\n",*s);
  int* p = extract(&list,b);
  printf("%d\n",*p);
  int* p1 = extract(&list,b);
  printf("%d\n",*p1);
  int* p2 = extract(&list,b);
  printf("%d\n",*p2);
}

bool find(item* thing) {
  return (*((int*)thing) == 5);
}
\end{lstlisting}
\newpage
\section{Buenos system calls for basic I/O}
\subsection{Implement system calls \texttt{read} and \texttt{write}}
We start by adding the appropriate system call cases to \texttt{proc/syscall.c}. In both \texttt{SYSCALL\_READ} and \texttt{SYSCALL\_WRITE} we call the appropriate functions defined in \texttt{kernel/read.h} and \texttt{kernel/write.h}. The parameters are extracted from \texttt{MIPS} registers \texttt{A1}, \texttt{A2} and \texttt{A3} and the result is placed in the return register \texttt{V0}.
\begin{lstlisting}
...
    switch(user_context->cpu_regs[MIPS_REGISTER_A0]) {
    case SYSCALL_HALT:
        halt_kernel();
        break;
    case SYSCALL_READ:
        user_context->cpu_regs[MIPS_REGISTER_V0] =
            syscall_read(
              (int) user_context->cpu_regs[MIPS_REGISTER_A1],
              (void*) user_context->cpu_regs[MIPS_REGISTER_A2],
              (int) user_context->cpu_regs[MIPS_REGISTER_A3]
            );
        break;
    case SYSCALL_WRITE:
        user_context->cpu_regs[MIPS_REGISTER_V0] =
            syscall_write(
              (int) user_context->cpu_regs[MIPS_REGISTER_A1],
              (const void*) user_context->cpu_regs[MIPS_REGISTER_A2],
              (int) user_context->cpu_regs[MIPS_REGISTER_A3]
            );
        break;
    default:
        KERNEL_PANIC("Unhandled system call\n");
    }
...
\end{lstlisting}
We now implement the functions defined in \texttt{kernel/read.h} and \texttt{kernel/write.h} in files \texttt{kernel/read.c} and \texttt{kernel/write.c}

\subsubsection{\texttt{kernel/read.c}}
The \texttt{read} function simply gets the generic \texttt{TTY} device driver and reads input into the \texttt{buffer}. We also make sure that our \texttt{fhandle} is 0 as we expect it would be in the read call - otherwise 0 characters are read and 0 returned.

\begin{lstlisting}
#include "kernel/read.h"
#include "drivers/gcd.h"
#include "drivers/device.h"

int read_kernel(int fhandle, void *buffer, int length){
  int len;
  device_t *dev;
  gcd_t *gcd;

  /* Checks that the file handle is the read code (0)
   * otherwise return 0 since we wont write any characters.
   * fhandle itself is not needed at this point of implementation.
   */
  if (fhandle != 0) {
    return 0;
  }

  // Get the default TTY and generic device driver
  dev = device_get(YAMS_TYPECODE_TTY, 0);
  gcd = (gcd_t *)dev->generic_device;
  // Read from the buffer
  len = gcd->read(gcd, buffer, length);
  return len;
}
\end{lstlisting}

\subsubsection{\texttt{kernel/write.c} }
The \texttt{write} function is very similar, except that it writes using the generic device driver instead of reading.

\begin{lstlisting}
#include "kernel/write.h"
#include "drivers/gcd.h"
#include "drivers/device.h"

int write_kernel(int fhandle, const void *buffer, int length){
  int len;
  device_t *dev;
  gcd_t *gcd;

  /* Checks that the file handle is the write code (1)
   * otherwise return 0 since we wont write any characters.
   * fhandle itself is not needed at this point of implementation.
   */
  if (fhandle != 1) {
    return 0;
  }

  // Get the default TTY and generic device driver
  dev = device_get(YAMS_TYPECODE_TTY, 0);
  gcd = (gcd_t *)dev->generic_device;
  // Write to the buffer
  len = gcd->write(gcd, buffer, length);
  return len;
}
\end{lstlisting}

\subsection{\texttt{tests/readwrite.c}}
The testing is executed as a small game, where the objective is to guess a number, a wrong answer results in a message telling the user, whether the number to be guessed is over, or under the guessed number. We test the read and write in this, as the game would not be playable, if either did not work.
\begin{lstlisting}
#include "tests/lib.h"

int main(void)
{
  char buffer[1];
  char newline[1] = {'\n'};
  char larger[12] = {'k',' ','i','s',' ','l','a','r','g','e','r','\n'};
  char smaller[13] = {'k',' ','i','s',' ','s','m','a','l','l','e','r','\n'};
  char correct[8] = {'c','o','r','r','e','c','t','\n'};
  int l, len;
  int k = 39;
  while (1) {
    syscall_read(0,buffer,1);
    syscall_write(1,buffer,1);
    l = (buffer[0] - '0')*10;
    syscall_read(0,buffer,1);
    syscall_write(1,buffer,1);
    syscall_write(1,newline,1);
    l += (buffer[0] - '0');
    if (l < k) {
      len = 12;
      syscall_write(1, larger, len);
    } else if (l > k) {
      len = 13;
      syscall_write(1, smaller, len);
    } else {
      len = 8;
      syscall_write(1, correct, len);
      break;
    }
  }
  syscall_halt();
  return 0;
}
\end{lstlisting}
\end{document}