\chapter{Solving the De Novo Genome Assembly}

In this last chapter, we approach the genome assembly problem with the use of quantum annealing. Let us first understand what this problem is, and then translate it to a QUBO model and solve it using quantum annealing.

\section{The Genome Assembly Problem}

The genome of an organism is all its genetic material \cite{Roth2019}. The deoxyribonucleic acid (DNA) is the carrier of that genetic information. It consists of two long chains twisted to form a double helix \cite{Alberts2007}. Each of these chains is composed of a series of nucleotides or bases: adenine (A), guanine (G), cytosine (C), and thymine (T). Since these bases are matched in pairs in the DNA double helix, they are called base pairs (bp).

A genome sequence is the complete list of nucleotides of every chromosome of an organism. With today's technology, automated sequence machines can read up to $10.000$ bp at a time \cite{Reinert1015} while the human genome contains 3 Mbp, so we cannot simply read the whole genome. This is where genome assembly comes in.

Genome assembly refers to the process of, given a large number of short DNA reads, stitch them together to form a large representation of the original chromosome where the reads came from. The two main techniques used to reconstruct these sequences are the ab initio reference-free alignment and the de novo reference-based assembly.

\subsection{Ab initio reference-based alignment}

In this method, the DNA reads are matched against a known trusted reference of the same organism. This is essentially a pattern matching problem, where we find the index of a given sub-string in a larger string. However, after the reconstruction is complete the result is compared to the reference in order to identify implications; therefore introducing bias based on the reference \cite{Sarkar2020}.

In the naive approach, the short sub-string is compared to the reference starting at the first index. If the end of the sub-string is reached with a positive, a match is obtained. Otherwise, the sub-string is shifted a single position and we compare again. Heuristic methods that improve on this idea are based on shifting a greater number of spaces after a mismatch.

Different number of strategies have been developed in this direction. For instance, the classic Boyer-Moore and Knuth-Pratt-Morris algorithms \cite{Holmes1999}. However, these in particular are not adequate for genome assembly since these are exact string matching algorithms and DNA reads usually need approximate matches due to reads errors. Other algorithms worth mentioning are the Needleman-Wunsch algorithm \cite{Needleman1970} and the Smith-Waterman algorithm \cite{Smith1981}, for global and local alignment respectively. These are dynamic programming algorithms designed specifically with DNA reads in mind.

State of the art algorithms trades off accuracy for speed and memory. Given enough computational power, the de novo reference-free method yields better results without introducing any reference bias.

\subsection{De novo reference-free assembly}
\label{sec:de-novo-genome-assembly}

On the other hand, the de novo reference-free method is, as its name suggests, reference free. Meaning, it is based only on the DNA reads. Thus, it has no reference bias but it is more computationally complex. It is usually used the first time a species DNA is read.

In this technique, multiple copies of the same DNA are made before slicing it. After chopping each copy at random places the data is redundant and the different reads overlap, making the assembly easier. There are multiple methods for de novo assembly based on different tools: Overlap-Layout Consensus (OLC) methods, de Bruijn graph (DBG) methods, string graphs, greedy and hybrid methods are some of the most famous examples (for a review see \cite{Sohn2018}). Depending on the reading method and the number and length of the DNA reads, different methods excel from the rest. For instance, short-read technologies with a large number of reads favor DBG methods while high-quality long reads favor OCL methods. For our purposes, we will focus on the OCL method.

In the OLC graph used for the de novo whole-genome assembly, each node represents a different DNA read. Directed edges are associated a weight depending on how well these two reads are stitched together in a certain order. For example, the directed edge going from reading $r_1$ to read $r_2$ will be assigned a weight depending on how well $r_1r_2$ can be stiched.

The weights computation depends on the implementation. For the purpose of this thesis, we will use exact matches, no taking reading errors into consideration. Then, the weight assigned to an edge is the length of the overlap between both reads without any errors, with a change of sign. For example, given the reads $r_1 = AATT$ and $r_2 = TTCC$, the perfect stitching will produce $AATTCC$, so the overlap between both reads is $2$, giving a weight of $-2$. We may call this the \emph{distance} between reads $r_1$ and $r_2$ (in that order). It does not fulfill the mathematical definition of distance since it is not even symmetric, but it will be useful for us anyway.

A Hamiltonian path in our overlap graph will represent a series of reads in a certain order. By minimizing the total cost of our Hamiltonian path, we maximize the overlap between reads, resulting in the shortest possible final chain. This is exactly the same as solving the Travelling Salesman problem associated with our overlap graph.

Figure \ref{de-novo-process} shows the whole problem resolution \cite{Boev2020}. Given the DNA reads (a) we compute the overlap graph (b) using a distance between the reads. We continue by viewing this problem as a traveling salesman and transforming it into a QUBO model (c). Then, either a simulated annealer (e) or the D-Wave quantum annealer (d) are used to obtain the Hamiltonian path/cycle of minimum cost (f). Finally, we traverse the cycle and build the resulting genome sequence.

\begin{figure}[H]
	\includegraphics[scale=0.4]{de-novo-process.png}
	\centering
	\caption{Resolution diagram for the genome assembly using quantum annealing \cite{Boev2020}}
	\label{de-novo-process}
\end{figure}

\newparagraph{Numerical example}

Let us study a final numerical example based on \cite{Sarkar2020} that shows the whole process. Suppose we are given the following reads:

\begin{itemize}
	\item $r_0 = ATGGCGTGCA$
	\item $r_1 = GCGTGCAATG$
	\item $r_2 = TGCAATGGCG$
	\item $r_3 = AATGGCGTGC$
\end{itemize}

We may compute the overlap between the different reads. This results in the overlap graph from figure \ref{fig:overlap-graph}.

\begin{figure}[h]
	\includegraphics[scale=1.1]{graphs/salesman-example.pdf}
	\centering
	\caption{OLC graph}
	\label{fig:overlap-graph}
\end{figure}

Which is the same graph studied in the traveling salesman section, \ref{sec:salesman-example}. From the study done in that section, we know that there are six types of cycles in the graph. They are displayedin table \ref{tbl:salesman-cycles}. In figure \ref{fig:overlap-cycles}, we see how these types of cycles represent different ordinations of our DNA reads, as well as their overlaps and the total length of the resulting chain.

\begin{figure}[h]
	\makebox[\textwidth][c]{
		\includegraphics[scale=0.35]{overlap-cycles.png}
	}
	\centering
	\caption{Type of cycles and their corresponding overlap analysis \cite{Sarkar2020}}
	\label{fig:overlap-cycles}
\end{figure}

As we know, the type of Hamiltonian cycle that minimizes the cost is type A. This type translates into the shortest chain, with a total length of $19$. We can easily compute the resulting assembly by traversing the graph and weaving the reads together.

\section{Experimentation}
\label{sec:experimentation}

In this last section, we will explain the experiments reproduced using the D-Wave systems.

\subsection{How to reproduce the experiments}

In order to configure the samplers and submit jobs to D-Wave quantum annealers we used the \emph{D-Wave Ocean Software}, a suite of tools provided by D-Wave to use their quantum systems \cite{DWave-OceanDoc}. Using python and the provided packages we may connect to \emph{Leap}, a 'quantum' cloud service that provides access the quantum computers \cite{DWave-Leap}.

Leap provides a minute of free QPU time with the (free) developer plan. We may extend this time by obtaining either commercial or research access. This can be done by contacting the Leap team.

For the pourpose of this thesis I used the developer plan. In total, I submitted $104$ jobs to Leap, out of the allowed $106$. I consumed $59.064$ QPU seconds, as can bee seen in Leap's breakdown in figure \ref{fig:leap-breakdown}. The number of jobs and QPU time used in each system can be seen in table \ref{tab:leap-breakdown}.

\begin{table}[H]
	\centering
	\begin{tabular}{lrr}
		\textbf{System} & \textbf{Number of jobs} & \textbf{QPU time (s)} \\
		\hline
		D-Wave 2000Q	& 26	& 11.015	\\
		Advantage		& 77	& 46.681	\\
		Total			& 104	& 59.064
	\end{tabular}
	\caption{Leap usage summary breakdown between systems.}
	\label{tab:leap-breakdown}
\end{table}

Apart from Leap, in order to reproduce these experiments you will need to install tthe following Python packages, available thorugh \emph{pip}: \textbf{\emph{dimod}}, \textbf{\emph{neal}}, \textbf{\emph{minorminer}}, \textbf{\emph{dwave}} and \textbf{\emph{dwave\_networkx}}. The code I used for these experiments can be found in \cite{thesis-code}, which are inspired on \cite{Sarkar2020}.

\begin{figure}[H]
	\includegraphics[scale=0.5]{experiments/leap-breakdown.png}
	\centering
	\caption{Leap monthly usage summary.}
	\label{fig:leap-breakdown}
\end{figure}

\subsection{Experiment 1: Data preparation and exact solving}

The first experiment aims to solve the previously discussed example using an exact solver provided by the Ocean's \emph{dimod} library \cite{Dimod}. The steps taken to transform the given reads to the QUBO matrix that Ocean's classes and functions use will be shared between the simulated annealer and the real quantum solvers. Therefore, the data preparation applied will be shared between experiments. The pseudocode of the experiment can be seen in the following snippet.

\begin{algorithm}
	\caption*{\textbf{Experiment 1}}

	Data preparation:
	\begin{itemize}
		\item Compute the distance between every two reads, creating an adjacency matrix.
		\item Transform the TSP adjancency matrix into a QUBO $Q$ matrix, as explained in section \ref{sec:tsp-qubo}.
		\item Transform the QUBO matrix into an adjacency dictionary.
	\end{itemize}

	Solving:
	\begin{itemize}
		\item Initiaize a sampler: \textbf{\emph{dimod.ExactSolver}}.
		\item Solve the prepared QUBO model using the selected solver.
	\end{itemize}

	Present the results.
\end{algorithm}

As we already know, using quantum annealing does not guarantee that we will obtain the optimal solution for a given cost function. In order to overcome this problem we use multi-sampling: running the experiment multiple times and and look at the best obtained solutions. Ocean already implements different types of \emph{samplers} to facilitate this task \cite{DWave-OceanDoc-Samplers}. In this first experiment we use an \emph{ExactSolver}, which simply check every possible solution. Although time-costly, this method will let us know if our data manipulation before solving the experiment is correct, and how the solutions landscape looks like.

The third step in data preparation is a formatting step. Ocean requires the QUBO and Ising models to be in an adjacency dictionary instead of a matrix. Let us look at an example of this transformation in order to better understand it. Consider a 2-reads, the associated TSP will have $2^2 = 4$ nodes: $n0t0$, $n0t1$, $n1t0$ and $n1t1$. Suppose the following (inconsistency) matrix is the associated $Q$ matrix:

$$
Q =
\left(
\begin{array}{cccc}
	1 & 2 & 3 & 4 \\
	5 & 6 & 7 & 8 \\
	9 & 10 & 11 & 12 \\
	13 & 14 & 15 & 16
\end{array}
\right)
$$

Then, it will be transformed into the dictionary, with $11$ missing entries:

\begin{minted}[bgcolor=bg]{python}
quboDict: {
	('n0t0', 'n0t0'): 1,
	('n0t0', 'n0t1'): 2,
	('n0t0', 'n1t0'): 3,
	('n0t0', 'n1t1'): 4,
	...
	('n1t1', 'n1t1'): 16,
}
\end{minted}

In particular, this experiment is applied to the already studied example with the following four reads:

\begin{itemize}
	\item $r_0 = ATGGCGTGCA$
	\item $r_1 = GCGTGCAATG$
	\item $r_2 = TGCAATGGCG$
	\item $r_3 = AATGGCGTGC$
\end{itemize}

The pair-wise distances are computed, providing a direct measure of much two reads overlap. Then, these values are normalized for easier use. The resulting normalized TSP matrix is transformed into a QUBO matrix using $1.6$ as multi-location and repetition penalties, and $-1.6$ for self-bias, as done in \cite{Sarkar2020}. Finally, we initialize an \emph{ExactSolver} and use it to sample every possible solution.

After the execution is completed, we find the lowest energy in the obtained solutions and print every solution with that energy:

\begin{minted}[bgcolor=bg]{python}
{'n0t0': 0, 'n0t1': 0, 'n0t2': 1, 'n0t3': 0,
 'n1t0': 0, 'n1t1': 0, 'n1t2': 0, 'n1t3': 1,
 'n2t0': 1, 'n2t1': 0, 'n2t2': 0, 'n2t3': 0,
 'n3t0': 0, 'n3t1': 1, 'n3t2': 0, 'n3t3': 0} --> -7.9811

{'n0t0': 0, 'n0t1': 1, 'n0t2': 0, 'n0t3': 0,
 'n1t0': 0, 'n1t1': 0, 'n1t2': 1, 'n1t3': 0,
 'n2t0': 0, 'n2t1': 0, 'n2t2': 0, 'n2t3': 1,
 'n3t0': 1, 'n3t1': 0, 'n3t2': 0, 'n3t3': 0} --> -7.9811

{'n0t0': 1, 'n0t1': 0, 'n0t2': 0, 'n0t3': 0,
 'n1t0': 0, 'n1t1': 1, 'n1t2': 0, 'n1t3': 0,
 'n2t0': 0, 'n2t1': 0, 'n2t2': 1, 'n2t3': 0,
 'n3t0': 0, 'n3t1': 0, 'n3t2': 0, 'n3t3': 1} --> -7.9811

{'n0t0': 0, 'n0t1': 0, 'n0t2': 0, 'n0t3': 1,
 'n1t0': 1, 'n1t1': 0, 'n1t2': 0, 'n1t3': 0,
 'n2t0': 0, 'n2t1': 1, 'n2t2': 0, 'n2t3': 0,
 'n3t0': 0, 'n3t1': 0, 'n3t2': 1, 'n3t3': 0} --> -7.9811
\end{minted}

As expected, these are the four minimums of the cost functions, representing the four ways of describing a type A loop in the used codification.

Finally, let us plot the landscape of solutions for the given reads. We sort the solutions by increasing energy and simply plot their energy, as seen in figure \ref{fig:exp1-landscape}.

\begin{figure}[H]
	\makebox[\textwidth][c]{
		\includegraphics[scale=0.7]{experiments/experiment1.png}
	}
	\centering
	\caption{Landscape of solutions}
	\label{fig:exp1-landscape}
\end{figure}

\subsection{Experiment 2: Simulated Quantum Annealing}

For our second experiment, we aim to solve our example using Simulated Annealing. Although \emph{dimod} provides a simulated annealing sampler, we use for our final experiments the  \emph{SimulatedAnnealingSampler} from \emph{Ocean}'s \emph{Neal} package as it yields more distributed results and has far better computing time. The same experiment was reproduced $10$ times using both samplers in my machine. Dimod's sampler run in a mean time of $11.76$ seconds while \emph{Neal}'s run in a mean time of $0.0610$ seconds.

The pseudo-code does not change much from the first experiment to the second one, we simply change the annealer:

\begin{algorithm}
	\caption*{\textbf{Experiment 2}}

	Data preparation:
	\begin{itemize}
		\item Compute the distance between every two reads, creating an adjacency matrix.
		\item Transform the TSP adjancency matrix into a QUBO $Q$ matrix, as explained in section \ref{sec:tsp-qubo}.
		\item Transform the QUBO matrix into an adjacency dictionary.
	\end{itemize}

	Solving:
	\begin{itemize}
		\item Initiaize a sampler: \textbf{\emph{neal.SimulatedAnnealingSampler}}.
		\item \textbf{Sample from} the prepared QUBO model using the selected \textbf{sampler}.
	\end{itemize}

	Present the obtained samples.
\end{algorithm}

It is worth mentioning that we do not \emph{solve} the QUBO model in these experiments, we \emph{sample} different results using simulated annealing.

Since \emph{Neal}'s sampler is quite efficient we were able to execute experiments with up to $10.000$ repetitions of the experiment, also called \emph{sample reads} or simply \emph{reads}. We developed an automated and scalable way to check whether a given cycle returned by the annealer was valid, and to recover its associated type. Using these tools we can easily check the number of occurrences each type of cycle appeared in the obtained sample reads. These results are presented, along with the associated energy to each result, in table \ref{tab:exp2} and figure \ref{fig:exp2-occ}. See figure \ref{fig:overlap-cycles} to recall the cycle types and cost computing.

\begin{table}[H]
	\centering
	\begin{tabular}{lrr}
		\textbf{Cycle type} & \textbf{Occurences} & \textbf{Energy} \\
		\hline
		Type A	& 3722	& -7.9811	\\
		Type C	& 1474	& -7.4541	\\
		Type D	& 1469	& -7.4541	\\
		Type F	& 1458	& -7.4541	\\
		Type E	& 1431	& -7.4541	\\
		Type B	& 446	& -6.927
	\end{tabular}
	\caption{Results of experiment 2}
	\label{tab:exp2}
\end{table}

\begin{figure}[H]
	\includegraphics[scale=0.8]{experiments/experiment2.png}
	\centering
	\caption{Ocurrences of each type in a $10.000$ reads experiment using Simulated Annealing}
	\label{fig:exp2-occ}
\end{figure}

We can appreciate in figure \ref{fig:exp2-occ} that the best type of cycle agglomerates most of the samples, $37.22\%$. In the second place, the four types of cycles that have the exact same energy also have almost exactly the same number of samples. This matches our quantum annealing theory: the physical system has an equal probability of ending in eigenstates with equal eigenenergies.

It is worth mentioning that there was not a single sample that encoded an invalid cycle. This means that the penalties values used for the experiment ($1.6$ for multi-location and repetition, and $-1.6$ for self-bias) are working to prevent invalid cycles.

Now that we are familiar with the type of results and codifications we may find, let us jump to the quantum realm by using the D-Wave's Quantum Computer Systems.

\subsection{Experiment 3: Quantum Annealing using D-Wave}

For our third experiment, we aim to solve our example using D-Wave's quantum annealers. For this purpose, we need to add some extra steps to our data processing. Section \ref{sec:embeddings} explained how not every graph can be directly mapped to the existing Chimera / Pegasus topologies that D-Wave systems use, and how we can overcome this problem by embedding our problems graphs into these topologies. The extra steps in our data processing are related to these embeddings: we will need to embed our graph into the fixed topology that will be used. Then, after the anneal takes place, we will translate the solutions from the embedded graph. Ocean provides a set of utilities to deal with these embeddings \cite{DWave-OceanDoc-Embedding}.

Additionally, we will need to connect to Leap to send our jobs to the quantum annealers using a \emph{client}. First, a configuration file is created, which includes your API key and some extra configuration details:

\begin{minted}[bgcolor=bg]{bash}
[ocete]
solver = {"qpu": true}
token = DEV-<api key goes here>
endpoint = https://cloud.dwavesys.com/sapi
\end{minted}

In our code we initialize a \emph{client} and a \emph{solver} using the configuration file. These are objects that encapsulate the connection functionality and solving problems in the associated machine respectively. We also initialize a sampler as we did before, but this time a production sampler is used: a \emph{DWaveSampler} that also needs our configuration file. The pseudo-code for this experiment can be found below. Each new step will be explained in detail below.

\begin{algorithm}
	\caption*{\textbf{Experiment 3}}

	Initialization:
	\begin{itemize}
		\item Initiaize a sampler: \textbf{\emph{system.samplers.DWaveSampler}}.
		\item \textbf{Initialize a client and obtain an associated solver}.
	\end{itemize}

	Data preparation:
	\begin{itemize}
		\item Compute the distance between every two reads, creating an adjacency matrix.
		\item Transform the TSP adjacency matrix into a QUBO $Q$ matrix, as explained in section \ref{sec:tsp-qubo}.
		\item Transform the QUBO matrix into an adjacency dictionary.
		\item \textbf{Find an embedding from our graph to the selected machine's topology (either Chimera or Pegasus)}.
		\item \textbf{Use the previous embedding to create a new QUBO model equivalent for the embedded graph}.
	\end{itemize}

	Solving:
	\begin{itemize}
		\item Use the solver to send a job to the client. This job samples from the prepared QUBO model multiple times.
	\end{itemize}

	Format the results:
	\begin{itemize}
		\item \textbf{Use the computed embedding to translate our answers}.
	\end{itemize}
\end{algorithm}

The first thing to be noticed is that the initialization now needs to be done before the data preparation. This is because, in order to find the embedding, we need to know which exact topology the quantum system will have. This information is provided through the \emph{solver}, previously initialized. The initialization step is as simple as follows:

\begin{minted}[bgcolor=bg]{python}
import dwave

# Create the solver (connecting to D-Wave) and the Sampler
config_file='../dwave.conf'
client = cloud.Client.from_config(config_file, profile='ocete')
solver = client.get_solver()
dwsampler = system.samplers.DWaveSampler(config_file=config_file)
\end{minted}

Suppose $Q$ already holds our computed QUBO model. We can compute an embedding and obtained the new associated model as follows:

\begin{minted}[bgcolor=bg]{python}
adjacency_dict = embedding.utils.edgelist_to_adjacency(solver.edges)
embedding = minorminer.find_embedding(Q, solver.edges)
Q_embedded = embed_qubo(Q, embedding, adjacency_dict)
\end{minted}

By using the same configuration file, the sampler already knows what machine it is associated to. We will sample from it, with the same syntax as in the second experiment:

\begin{minted}[bgcolor=bg]{python}
response = dwsampler.sample_qubo(Q_embedded, num_reads=num_reads)
\end{minted}

Where $num\_reads$ is a parameter that sets the number of samples to be read. However, these solutions are associated to the $Q\_embedded$, not to our original $Q$ model. We need to translate the responses to understand them:

\begin{minted}[bgcolor=bg]{python}
bqm = dimod.BinaryQuadraticModel.from_qubo(Q)
unembedded_response = embedding.unembed_sampleset(response, embedding, bqm)
\end{minted}

For this experiment, the D-Wave 2000Q was used (see section \ref{sec:d-wave-systems} to see its characteristics), since it was the default solver. After running the experiment with the same parameters as the simulated annealing experiment ($10.000$ reads and $(-1.6; 1.6; 1.6)$ for the QUBO parameters), we obtained that more than $90\%$ of the samples were invalid, a huge difference with the astonishing $0\%$ obtained in the second experiment.

My initial hypothesis was that the QUBO parameters were well adjusted for SA but not for QA, and thus allowed further tunning. After many attempts with different parameters values, this hypothesis was discarded. The only other set of parameters worth presenting is $(-1.5; 1.5; 1.5)$. These results are presented in table \ref{tab:exp3}.

\begin{table}[H]
	\centering
	\begin{tabular}{lrrr}
		\textbf{Cycle type} & \textbf{Occurences (param=1.5)} & \textbf{Occurences (param=1.6)} & \textbf{Energy} \\
		\hline
		Type A	& 131	& 50	& -7.9811	\\
		Type C	& 124	& 46	& -7.4541	\\
		Type D	& 40	& 91	& -7.4541	\\
		Type F	& 55	& 117	& -7.4541	\\
		Type E	& 112	& 118	& -7.4541	\\
		Type B	& 99	& 64	& -7.4541	\\
		Invalid & 9221	& 9194	& $>$ -5.6433
	\end{tabular}
	\caption{Results of experiment 3, $10.000$ reads using the quantum annealer.}
	\label{tab:exp3}
\end{table}

In figures \ref{fig:exp3-occ1} and \ref{fig:exp3-occ2} we see a comparison of the distribution between the different type of cycles in both experiments. We cannot see a distribution as we did in \ref{fig:exp2-occ}, potentially due to the low numbers of reads that actually represent a valid cycle. In fact, such distribution between the cycle types is not needed: a single sample with the lowest energy solution will completely solve our genome assembly problem.

\begin{figure}[H]
	\includegraphics[scale=0.75]{experiments/experiment3 (1.5).png}
	\centering
	\caption{Ocurrences of each cycle type in a $10.000$ reads experiment using Quantum Annealing, filtering out invalid cycles, with QUBO parameters $(-1.5; 1.5; 1.5)$}
	\label{fig:exp3-occ1}
\end{figure}

\begin{figure}[H]
	\includegraphics[scale=0.75]{experiments/experiment3 (1.6).png}
	\centering
	\caption{Ocurrences of each cycle type in a $10.000$ reads experiment using Quantum Annealing, filtering out invalid cycles, with QUBO parameters $(-1.6; 1.6; 1.6)$}
	\label{fig:exp3-occ2}
\end{figure}

With these results in mind, is it possible that we are sampling completely random solutions? In a 4-reads problem, we obtain a 16 binary variables QUBO model. That means there are up to $2^{16}$ possible solutions using our encoding, even more after we embed it in the Chimera topology. If we were to randomly sample from that solution space, we will never get almost $10\%$ of samples from a subset of $24$ solutions that represent our $6$ valid cycles. So we do know that the quantum system is working, just not as good as expected.

We have not found a clear improvement precision-wise in our small example, although QA does solve the problem. What about time-wise? In figure \ref{fig:exp3-time} we see the time breakdown provided by Leap after our last experiment. We can see a total QPU sampling time of $2.389$ seconds, while if we repeat the experiment with the same parameters using SA we obtain a mean of $2.525$ sampling seconds, over $10$ experiment repetitions. Keeping in mind that our QA time data comes from a single execution and the small difference between both time measures, we cannot conclude that either approach has any time advantage in such small problems.

However, does QA actually scales better than SA with the problem size? In the next experiment, we will study the Pegasus topology to understand what is the maximum problem size that can be tackled using the Advantage system. Finally, the last couple of experiments will try to answer the scalability comparison inquiry.

\begin{figure}[h]
	\includegraphics[scale=0.5]{experiments/exp3-time.png}
	\centering
	\caption{Time breakdown provided by Leap after the last experiment.}
	\label{fig:exp3-time}
\end{figure}

\subsection{Displaying chimera embeddings}

Let us plot some logical embeddings using the D-Wave 2000Q system, with relies on a Chimera graph. In figure \ref{fig:exp2_4reads} we see the embedding of a 4-nodes complete graph into the Chimera topology. The grey nodes represent unused nodes while nodes with the same color represent a single logical qubit mapped into multiple physical qubits.

\begin{figure}[h]
	\includegraphics[scale=0.6]{experiments/exp2_4reads.png}
	\centering
	\caption{Embedding of a 4-nodes complete graph into a Chimera topology.}
	\label{fig:exp2_4reads}
\end{figure}

The last embedding had a \emph{maximum chain length} of $2$. That is, the longest chain in a logical qubit is 2 physical qubits. This is a critical parameter to be minimized in embeddings: the longer the qubit chain is, the more probable it is for the qubits in a logical qubit to be desynchronized and end up in different states. Thus, the more difficult it is for the system to stay in the ground state.

In figure \ref{fig:exp2_13reads} we see an embedding of a 13-nodes complete graph into a Chimera topology. In this case, the maximum chain length is $5$, obtained by both the dark blue, purple, red, dark green, lime, and cyan logical qubits. In the rest of the experiments, I looked at the maximum chain length obtained in the pegasus embeddings, obtained lengths up to $15$ qubits. Since I did not have the time to study this characteristic in-depth, it will be a future line of work.

\begin{figure}[h]
	\includegraphics[scale=0.6]{experiments/exp2_13reads.png}
	\centering
	\caption{Embedding of a complete 13-nodes complete graph into a Chimera topology.}
	\label{fig:exp2_13reads}
\end{figure}

Recall that in every 16-nodes chimera cell, every node in the vertical axis is connected to every node in the horizontal axis, and to no other node from the vertical axis. For example, take the blue logical qubit, represented by 3 nodes at the very bottom in figure \ref{fig:exp2_13reads}. In the first cell, it connects with lime, pink, dark green, and cyan. In the second cell, it connects with light green, purple, red, and dark blue. In the third cell, it connects to brown, light blue, pink, and yellow. That makes a total of 12 connections, as expected since the embedded started with a complete 13-nodes graph.

\FloatBarrier
\subsection{Experiment 4: D-Wave's Advantage limits}

For our fourth experiment, we aim to test the limits of the Advantage architecture for our problem. That is, given a number of DNA reads, we will compute the associated QUBO problem and try to embed the obtained graph into the Advantage architecture. We will test these embeddings against a Pegasus 16 ($P_{16}$) topology since that is the one supported by the Advantage system.

In fact, given a set of $n$ reads, this is simply finding if a $n^2$-nodes complete graph ($K_{n^2}$) can be embedded into a $P_{16}$. The reads themselves are not relevant for the embedding. However, I have developed an automated way of creating fake tests, originally for a general-purpose, but it can also be used here.

This automated test production works as follows: Given a number of reads, the size of each chain (default value $150$), and the required overlap between adjacent chains (default value $50$), it randomly creates the original DNA chain, and then chop it using the given parameters. This makes sure that adjacent reads have enough overlap so it is basically impossible for two random reads to overlap more than $50$ bps.

Using these random tests we created graphs of an increasing number of reads until we were unable to find an embedding from $K_{n^2}$ to $P_{16}$. The results of the experiment can be seen in table \ref{tab:exp4}. We can see that a valid embedding was found to problem size up to 14 reads ($K_{196}$ associated graph). The embedding time, as well as the test generation time, are also displayed. I decided to display both since, as time grew larger and larger, I was not sure whether the test generation was adding too much of an overload.

\begin{table}[H]
	\centering
	\makebox[\textwidth][c]{
		\begin{tabular}{ccccc}
			\textbf{Number of reads} & \textbf{Nodes in graph} & \textbf{Test generation time} & \textbf{Embedding time} & \textbf{Total time} \\
			\hline
			2   & 4   &  0.000157   &  0.081310   &  0.081708 \\
			3   & 9   &  0.000216   &  0.108525   &  0.109205 \\
			4   & 16   &  0.000269   &  0.194807   &  0.196045 \\
			5   & 25   &  0.000335   &  0.377673   &  0.379814 \\
			6   & 36   &  0.000392   &  1.147603   &  1.154559 \\
			7   & 49   &  0.000433   &  3.514801   &  3.523473 \\
			8   & 64   &  0.000497   &  11.876785   &  11.888235 \\
			9   & 81   &  0.000558   &  19.226392   &  19.244367 \\
			10   & 100   &  0.000610   &  43.457666   &  43.477268 \\
			11   & 121   &  0.000676   &  62.499896   &  62.528545 \\
			12   & 144   &  0.000724   &  89.277396   &  89.313831 \\
			13   & 169   &  0.000812   &  63.782066   &  63.827706 \\
			14   & 196   &  0.000840   &  152.262340   &  152.316223
		\end{tabular}
	}
	\caption{Results of experiment 4}
	\label{tab:exp4}
\end{table}

It is worth noting that the algorithm used to find the embeddings does not guarantee to find a valid one if it exists. I tried to run the algorithm $50$ times with $15$ reads and could not find a valid embedding.

The reader might have already noticed that the embeddings are quite time-consuming. In fact, this is an NP problem itself and time will grow exponentially on input size. However, keep in mind that this embedding does not depend on the reads. Once a single embedding from a $K_{196}$ to a $P_{16}$ has been pre-computed it can be used -or further optimized- for different problems.

Now that we know the maximum number of reads that the Advantage system may tackle, let us put it to the test with 'big' input problems in the next experiment.

\subsection{Experiment 5: Scalability comparison between Simulated Annealing and Quantum Annealing}

In our fifth experiment, we put the D-Wave's quantum annealers to the test with bigger input problems and study how they behave compared with the simulated annealing experiments. We will start with a $3$ reads and then progressively increase this value, creating tests of this size and trying to solve them using both the simulated annealer and the Advantage system.

For each fixed number of reads, we will sample $10.000$ times using the annealers, with the same set of parameters used for the previous experiments. The number of valid solutions (i.e. valid cycles) will be displayed, along with the number of cycles that reached the real solution, which we know in advance since we created the test.

We will also display the energy of the best sample obtained, along with the difference between the real solution energy and the best sample obtained, which I called \emph{energy delta}. Finally, we will also display the sampling time, not including the time it took to create the test nor to prepare the data.

The results of this experiment using the simulated annealer can be seen in table \ref{tab:exp5_1}.

\begin{table}[H]
	\centering
	\makebox[\textwidth][c]{
		\begin{tabular}{cccccc}
			\textbf{N. reads} & \textbf{Valid cycles} & \textbf{Times sol. reached} & \textbf{Energy delta} & \textbf{Solution energy} & \textbf{Sampling time} \\
			\hline
			3	& 10000		& 9564			& 0				& -6.214214		& 1.881374 \\
			4	& 10000		& 6852			& 0				& -8.131935		& 2.178147 \\
			5	& 10000		& 3427			& 0				& -9.999001		& 3.487972 \\
			6	& 10000		& 1308			& 0				& -11.834192	& 5.121054 \\
			7	& 10000		& 351			& 0				& -13.656181	& 7.057235 \\
			8	& 10000		& 108			& 0				& -15.443562	& 9.680610 \\
			9	& 10000		& 21			& 0				& -17.232667	& 12.797996 \\
			10	& 10000		& 3				& 0				& -18.995278	& 16.080461 \\
			11	& 10000		& 0				& 0.625356		& -20.764680	& 20.118828 \\
			12	& 10000		& 0				& 0.601820		& -22.510011	& 25.351562 \\
			13	& 10000		& 0				& 0.570361		& -24.256735	& 30.612547 \\
			14	& 10000		& 0				& 1.378872		& -26.005003	& 34.897780
		\end{tabular}
	}
	\caption{Results of experiment 5, $10.000$ reads using the simulated annealer.}
	\label{tab:exp5_1}
\end{table}

We stopped at 15 reads since that is the maximum value that will be able to compare with using the quantum annealer. In the experiment results, we see how the number of valid cycles out of $10.000$, which turns out to be every single cycle up to 14 reads. However, the number of samples that reached the best solution go down from $95\%$ to ground cero from $11$ reads onward. Although we do not obtain the real solution, how close are these samples to it?

The columns related to the energy of the solutions try to answer this question. We may see how the solution's energy goes down to around $-26$, which is totally normal: the more reads in our graph, the higher the number of reads overlap, and the lower is the resulting solution energy. As soon as we cannot find the solution, the energy delta starts to increase, going up to around $1.37$. This means the energy of the best sample is quite close to the actual value. Is this enough?

When we tackle a classic TSP, a near-optimal solution is a really valid and useful solution. We do not necessarily need the actual minimum. However, although a valid cycle with low energy represents a valid way of sorting and sewing the genome reads, it does not provide the real genome the reads came originally from. It has some value since it could be further tuned to obtain the solution, but it does not solve our problem.

Lastly, the sampling time seems to grow linearly with the number of reads. However, using this simple simulated annealer for a real case with up to 3 Mbps would be simply impossible.

For the second part of this experiment, I run the same experiment but using the quantum annealer. Some extra steps come with it, like computing the embedding and connecting to Leap, but they have all been explained in previous experiments. We again sample $10.000$ times a maintain the same values for the annealing schedule self-bias, multi-location, and repetition parameters. The results of this experiment can be seen in table \ref{tab:exp5_2}.

\begin{table}[H]
	\centering
	\makebox[\textwidth][c]{
		\begin{tabular}{cccccc}
			\textbf{N. reads} & \textbf{Valid cycles} & \textbf{Times sol. reached} & \textbf{Energy delta} & \textbf{Solution energy} & \textbf{Sampling QPU time} \\
			\hline
			3	& 4574		& 3295		& 0				& -6.214214		& 1.035 \\
			4	& 683		& 156		& 0				& -8.131935		& 0.962 \\
			5	& 88		& 6			& 0				& -9.999900		& 1.272 \\
			6	& 2			& 0			& -1.322536		& -11.834014	& 1.272 \\
			7	& 1			& 0			& -1.647457		& -13.671186	& 1.206 \\
			8	& 0			& 0			& -3.867139		& -15.444996	& 1.339 \\
			9	& 0			& 0			& -36.592409	& -17.227226	& 1.570 \\
			10	& 0			& 0			& -50.03568		& -18.468571	& 1.276
		\end{tabular}
	}
	\caption{Results of experiment 5, $10.000$ reads using the quantum annealer.}
	\label{tab:exp5_2}
\end{table}

The first thing we may notice is that the experiment did not reach 14 reads. That is because I manually stopped it before it finished. The available QPU time was quite tight at the moment, and seeing the results of the experiment with a number of reads up to $10$ showed that we were not going to learn anything new by letting the experiment finished and wasting our remaining QPU time.

Back to the results, we can appreciate the number of valid cycles rapidly goes down to $0$, and even quicker the number of reads that reached the solution. At 4 reads we see similar results to what we saw in experiment 3: around $700$ valid cycles and $150$ times the real solution (type A cycle in the third experiment) was reached. Although we have developed this experiment in different machines, D-Wave 2000Q and Advantage, keep in mind that the improvements from one generation to another are size-related: we may tackle bigger (and thus harder) problems by using the Advantage system. It does not mean that its computation is somehow better.

Not only the responses are not valid, but they are also quite far from the solution. We can see that in how the delta grows hugely as we increase the number of reads. Leap provides a graph for each submitted experiment displaying the samples' distribution. In figures \ref{fig:exp5_1} and \ref{fig:exp5_2} we see this distribution for $4$ and $10$ reads respectively.

\begin{figure}[H]
	\includegraphics[scale=0.5]{experiments/exp5_1.png}
	\centering
	\caption{$10.000$ samples' distribuition for $4$ reads using the quantum annealer.}
	\label{fig:exp5_1}
\end{figure}

\begin{figure}[H]
	\includegraphics[scale=0.5]{experiments/exp5_2.png}
	\centering
	\caption{$10.000$ samples' distribuition for $10$ reads using the quantum annealer.}
	\label{fig:exp5_2}
\end{figure}

We can see how in the $4$ reads experiment, most of the samples group around $0$ energy, and only a very few are close to the solution, which is actually reached in this execution. However, in the $10$ reads experiment, most of the samples are gathered around $100/150$ energy, these solutions are not only invalid but really far from being so.

Lastly, let us comment on the QPU sampling time. This value is provided directly by Leap. From our code, we cannot separate the connection overload and other computations from the actual QPU sampling time. We may appreciate how time does not grow with the input. This is because the time it takes for the annealer to converge is fixed by the annealing schedule.

In conclusion, these results are astonishingly awful. The simulated annealing sampler not only provides closer samples to the solution, but in every seen case it reaches valid samples. My current hypothesis is that either the annealing schedule is not optimal, or that the values of the parameters can be improved for our specific problem. In our last experiment, I will spend all our remaining QPU time checking these two hypotheses.

\subsection{Experiment 6: Advantage tunning}

For our last experiment, we aim to test our previous hypothesis and try to solve a $10$ reads problem using the Advantage system. In the first set of iterations, I tried to increase the multi-location and repetition penalties in order to increase the number of valid cycles sampled. For this experiment, a single test with $10$ reads was created. Its solution energy was  $-18.994150$. The results of the experiment can be found in table \ref{tab:exp6_1}, where the mentioned parameters are \emph{(self-bias, multi-location, repetition)}.

\begin{table}[H]
	\centering
	\makebox[\textwidth][c]{
		\begin{tabular}{ccccc}
			\textbf{Parameters} & \textbf{Valid cycles} & \textbf{Times sol. reached} & \textbf{Energy delta} & \textbf{Sampling QPU time} \\
			\hline
			(-1.6, 1.6, 1.6)	& 0		& 0			& -76.344431	&	1.500 \\
			(-3, 3, 3)			& 0		& 0			& -87.263801	&	1.624 \\
			(-5, 5, 5)			& 0		& 0			& -83.657639	&	1.272 \\
			(-10, 10, 10)		& 0		& 0			& -258.290739	&	1.248 \\
			(-20, 20, 20)		& 0		& 0			& -357.917063	&	1.311 \\
			(-50, 50, 50)		& 0		& 0			& -1017.605383 	&	1.441 \\
			(-100, 100, 100)	& 0		& 0			& -1418.541757 	&	1.261
		\end{tabular}
	}
	\caption{Results of experiment 6, $10.000$ reads using the quantum annealer for different parameters configurations.}
	\label{tab:exp6_1}
\end{table}

As we can see in the table, we did not find a single valid cycle in these iterations. The energy delta naturally increases as we increase the parameter values: every single penalty is greater penalized. However, the energy delta barely increases until our parameters get a value of $10$. This could show how the responses for \emph{parameters} $= (-5, 5, 5)$ incurred in a lower number of penalties. In figure \ref{fig:exp6_1} we find the samples distribution for \emph{parameters} $= (-5, 5, 5)$. The overall energy is much higher than what we saw for \emph{parameters} $= (-1.6, 1.6, 1.6)$ in figure \ref{fig:exp5_2}, which occurs due to the higher penalty. This experiment concludes that either we have not been able to find the correct values for the parameters, or the values of the parameters are not the problem.

\begin{figure}[H]
	\includegraphics[scale=0.5]{experiments/exp6_1.png}
	\centering
	\caption{$10.000$ samples distribuition for \emph{parameters} $= (-5, 5, 5)$.}
	\label{fig:exp6_1}
\end{figure}

Finally, we try changing the annealing schedule. There are two possible parameters that we may modify in this sense: the \emph{annealing time} -time let for the annealer to converge-, or the \emph{annealing schedule} -which modifies the annealing functions as defined in figure \ref{fig:dwave-annealing-functions}. Since my initial hypothesis was that the annealer did not have enough time to find good solutions, let us focussed on the former for our last experiment.

The anneal time is meassured in microseconds, with a default value of $20$. Although the initial idea was to start with that value and make it progressively higher, at this point I was able to submit only one more worthy experiment, with a anneal time of $40$ microseconds. This was due to a bug in the D-Wave $get\_solver()$ function, which did not read the annealing time correctly and led to some invalid experiments at the last minute. We can see the obtained results in table \ref{tab:exp6_2}.

\begin{table}[H]
	\centering
	\makebox[\textwidth][c]{
		\begin{tabular}{ccccc}
			\textbf{Annealing time ($\mu$s)} & \textbf{Valid cycles} & \textbf{Times sol. reached} & \textbf{Energy delta} & \textbf{Sampling QPU time} \\
			\hline
			20	& 0		& 0			& -76.344431	& 1.500 \\
			40	& 0		& 0			& -57.944162	& 1.573
		\end{tabular}
	}
	\caption{Results of experiment 6, $10.000$ reads using different annealing times.}
	\label{tab:exp6_2}
\end{table}

Although the energy delta is reduced, the improvement is marginal: we still find no valid cycles and the responses have a high penalty. Probably with even more annealing time we will obtain further refined results.


\section{Preliminaries}\label{sec:prelim}
%
{\sc First Pass: assume fixed Godel T norm -- we can extend to other
  tnorms and to distribution semantics later.
}

Throughout this paper, we assume a general knowledge of logic
programming terminology, including tabled resolution and the
well-founded semantics including both default and explicit negation as
in \cite{AlDP95}.  Here, we include notation and definitions that will
be used for our proof of the correctness of Plow.

An {\em annotated atom} is an atom $A$ associated with an annotation
$n$ that is either a variable or $0 \leq n \leq 1$, denoted $A:n$.
From an annotated atom {\em A:n}, an {\em objective literal} $O$ is
formed as either $O = A:n$, termed a positive objective literal with
$sign(O) = pos$; or aa $O = \ourneg A:n$, termed a negative objective
literal with $sign(O) = \ourneg$, and the \ourneg{} symbol denotes
{\em explicit negation}.  In either case the annotation, $n$, is
denoted as $ annotation(O)$, while the underlying atom, $A$, is
$atom(A)$.
%
Two objective literals $O_1$ and $O_2$ with the same underlying atom
are {\em homologs} if they have the same sign and {\em conjugates}
otherwise.  An objective literal $A:n$ is ground if both $A$ and $n$
are ground.
%
% = A:n$ and $O_2 = \neg A:m$ are termed {\em conjugates}.
%denoted as $O_1 = conjugate(O_2)$ or $O_2 = conjugate(O_1)$;
%Otherwise if $atom(O_1) = atom(O_2)$ $O_1$ and $O_2$ are termed {\em
%  homologs}.
%
%, denoted as $O_1 = homolog(O_2)$ or $O_2 = homolog(O_1)$.
From an objective literal $O$ a default literal is formed as either a
positive default literal $O$ or as a negative default literal $\naf
O$, where \naf{} denotes {\em default negation}.  A default literal is
sometimes simply called a {\em literal}.
\footnote{
When convenient, an objective literal $A:1$ ($\neg A:1$) is denoted
simply as $A$ ($\neg A$).}

A rule has the form
\[r = O \mif{} L_1,\ldots,L_n\]
where $O$ is an objective literal and $L_0,\ldots,L_n$ are default
literals.

A rule $R$ is ground if all literals in $R$ are ground; a program
$\cP$ is ground if all rules in $\cP$ are ground.

%An objective literal $O$ is undefined in $\cI$ if $O \neg \hat{\in}

%\subsection{Three-Valued Models for Annotated Atoms}

Our attention is restricted to three-valued (partial) interpretations
and models such as those extending the well-founded model.  Each such
interpretation $\cI$ is represented as a pair of sets of ground
objective literals: ($\cT$,$za\cF$).

%{\sc TES: In the following, I dont yet feel confident about my $>$'s and my
%$\geq$'s.  ... conflicting truth values are mapped to U, but in the
%edge case, I'd like true to outweigh false (as with $r$ below).}

\begin{definition} \label{def:satisfiable}
Let $P$ be a ground objective literal $O$ with
% $atom(O) = A$ and
$annotation(O) = n$, and $\cI$ an interpretation.

$O$ is true in $\cI$ if
\begin{itemize}
\item $\cT$ contains an $O_T$ such that $O_T$ is a homolog of
  $O$ and $annotation(O_T) \geq n$; and
\item $\cT$ does not contain a conjugate $O_C$ of $O$ with
  $annotation(O_C) \geq (1-n)$; and
\item $\cF$ does not contain a homolog $O_F$ of $O$ with
  $annotation(O_F) \geq (1-n)$.
\end{itemize}

%----------------------------------------------------------------
$O$ is false in $\cI$ if
\begin{itemize}
\item $\cT$ does not contain a homolog $O_T$ of $O$ with
  $annotation(O_T) \geq (1-n)$; and either
\begin{itemize}
\item $\cF$ contains a homolog $O_F$ of $O$ with $annotation(O_F) \geq
  n$; or
\item $\cT$ contains a conjugate $O_C$ of $O$ with $annotation(O_C) \geq n$.
%Either $A:n_1 \in \cF$ for $n_1 \geq m$ or $\ourneg A:n_2 \in \cT$ for
%$m_2 \geq m$; and
\end{itemize}
\end{itemize}
$O$ is $\ourU$ in $\cI$ if it is neither true nor false in $\cI$.

A positive literal $O$ is true (false) in $\cI$ if $O$ is true (false)
in $\cI$; a negative literal $naf O$ is true in $\cI$ if $O$ is false
in $\cI$ and is false in $\cL$ if $O$ is true in $\cI$.  A literal is
$\ourU$ in $\cI$ if it is neither true nore false in $\cI$.
\end{definition}
%----------------------------------------------------------------

%\begin{definition} \label{def:satisfiable}
%A ground annotated atom $A:m$ is true in $\cI$ if
%\begin{itemize}
%\item There exists an $A:n_1 \in \cT$ with $n_1\geq m$, and
%\begin{itemize}
%\item $\cT$ does not contain $\ourneg A:n_2$ with $n_2 > (1-m)$ and
%\item   $\cF$ does not contain $A:n_3$ with $n_3 > (1-m)$.
%\end{itemize}
%\end{itemize}
%A ground annotated atom $A:m$ is false in $\cI$ if
%\begin{itemize}
%\item Either $A:n_1 \in \cF$ for $n_1 \geq m$ or $\ourneg A:n_2 \in \cT$
%  for $m_2 \geq m$; and
%\item $\cT$ does not contain $A:n_3$ for $n_3 \geq (1-m)$.
%\end{itemize}
%\end{definition}

Note that in the above definition the truth value \ourU{} captures
both the traditional case where a literal is undefined, along with the
case where a literal is overdefined.

\begin{example}
  Consider the interpretation $\cI_1$ where
\begin{itemize}
\item  $\cT = \{p:0.7,q:0.5, r:0.6,\ourneg r:0.5\}$
\item $\cF = \{p:0.4, q:0.5\}$.
\end{itemize}
$p:0.6$ is true in $\cI$ and $p:0.8$ false, but $p:0.7$ is \ourU ;
$q:0.5$ is \ourU{} in $\cI$. $r:0.4$ is true in $\cI$ but $r:0.7$ is
false; $r:n$ is \ourU{} for $0.5 \leq n \leq 0.6$.
\end{example}

%\cT$ and $O \neg \hat{\in} \cF$.

%----------------------------------------------------------------

%----------------------------------------------------------------
\subsubsection{Well-Founded Model}

Motivation: consider the program
\begin{verbatim}
p:0.8:- not p:0.8.
p:0.5.
neg p:0.3.
\end{verbatim}
We want a model with p:0.5 as true, p:0.7 as false and p:0.6 as u.
However, if the rule {\tt p:0.8:- not p:0.8.} were removed, p:0.6
would be false.

\comment{
%For two interpretations, $\cI$ and $\cJ$, $\cI
%\subseteq \cJ$ iff $true(\cI) \subseteq true(\cJ)$ and $false(\cI)
%%\subseteq false(\cJ)$.  Alternatively, a three-value interpretation
%can be represented as a set of literals.

A program $P$ is {\em safe} if each rule $r$ in $P$ is such that every
variable in $r$ occurs in a positive literal in the body of $r$.

Symbols within a term may be represented through {\em positions} which
are members of the set $\Pi$.  A {\em position\/} in a term is either
the empty string $\Lambda$ that reaches the root of the term, or the
string $\pi.i$ that reaches the $i$th child of the term reached by
$\pi$, where $\pi$ is a position and $i$ an integer.  For a term $t$
we denote the symbol at position $\pi$ in $t$ by $t_\pi$.  For
example, $p(a,f(X))_{2.1} = X$.  We assume that a program $P$ is
defined over a language $\cL$, containing a finite set $\functions$ of
predicate and function symbols, and a countable set of variables from
the set ${\cal V} \cup \posvar$.  Elements of the set $\cal V$ are
referred to as {\em program variables}.  Elements of the set
$\posvar$, called {\em position variables}, are of the form $X_\pi$,
where $\pi$ is a position.
%
These variables are used when it is convenient to mark certain
positions of interest in a term.  The Herbrand Universe of $\cL$ is
denoted $\cH_{\cL}$, or as $\cH_{P}$ if $\cL$ consists of the
predicate and function symbols in $P$; similarly the Herbrand Base is
denoted as $\cB_{\cL}$ or as $\cB_P$.  Throughout the paper variant
terms are considered to be equal.
}

One of the most important formulations of stratification is that of
{\em dynamic} stratification \cite{Przy89d}, which shows that a
program has a 2-valued well-founded model iff it is dynamically
stratified, so that it is the weakest notion of stratification
consistent with the well-founded semantics.  The original definition
of dynamic stratification included neither explicit negation (neg) nor
annotations; however the encapsulation of these features within the
interpretations of Definition~\ref{def:satisfiable} allows the
definitions in this section to be mostly unchanged from their original
formulation.

%
As presented in~\cite{Przy89d}, dynamic stratification computes strata
via operators on interpretations of the form ($\cT$,$\cF$) where $\cT$
and $\cF$ are subsets of $\cH_P$.  Given a set $\cS$ of ground
objective literals, a ground objective literal $A:m \hat{\in} \cS$ if
$A:n \in \cS$ with $n \geq m$.

%-----------------------------------------------------------------------
Dynamic stratification is based on a series of reduction of a program
in the usual manner.

\begin{definition}[Reduction of $\cP$ modulo $\cI$] \label{def:reduction}
%
Let $\cI$ be an interpretation and $\cP$ a program, both over the same
langage $\cL$.  By the {\em reduction of $P$ modulo $\cI$} we mean a
new program $\frac{P}{\cI}$ obtained from $P$ by performing the
following operations:
\begin{enumerate}
\item Remove from $\cP$ all rules that contain a literal that is false
  in $\cI$.
\item Remove from all the remaining rules those literals that are true
  in $\cI$
\end{enumerate}
\end{definition}

Each stratum is then based on the intepretation of the previous
stratum if it exists commbined with a reduction.

%-----------------------------------------------------------------------
\begin{definition} \label{def:dyn-ops}
For a normal program $P$, sets $\cT$ and $\cF$ of ground atoms and a
3-valued interpretation $I = (\cT,\cF)$ (sometimes called a pre-interpretation):
\begin{description} \item[$True^P_I(\cT) =$]
    $\{A|A$ is true in $I$; or there is a clause $B \leftarrow
  L_1,...,L_n$ in $P$, a grounding substitution $\theta$ such that $A
  = B\theta$ and for every $1 \leq i \leq n$ either $L_i\theta$ is
  true in $I$, or $L_i\theta \hat{\in} \cT$\};
  \item[$False^P_I(\cF) =$]
%$\{A : val_I(A) \neq {\tt f}$
$\{A|A$ is false in $I$; or for every clause $B \leftarrow
    L_1,...,L_n$ in $P$ and grounding substitution $\theta$ such that
    $A = B\theta$ there is some $i$ $(1 \leq i \leq n)$ such that
    $L_i\theta$ is false in $I$ or $L_i\theta \hat{\in} \cF\}$.
\end{description}
\end{definition}
%------------------------------------------------------------------------()
%
\cite{Przy89d} shows that $True^P_I$ and $False^P_I$ are both
monotonic, and defines $\kcaltrue^P_I$ as the least fixed point of $True^P_I(\emptyset)$
and $\calfalse^P_I$ as the greatest fixed point of
$False^P_I(\cH_P)$.
%\footnote{Below, we will sometimes omit the program $P$ in
%  these operators when the context is clear.}.
%, along with an
%operator $\cal I$ that assigns to every interpretation $I$ of $P$ a
%new interpretation ${\cal I}(I) = I \cup \langle \cT_I ; \cF_I \rangle$.
%
In words, the operator $\kcaltrue^P_I$ extends the interpretation $I$ to add
the new atomic facts that can be derived from $P$ knowing $I$; $\calfalse^P_I$
adds the new negations of atomic facts that can be shown false in $P$
by knowing $I$ (via the uncovering of unfounded sets).  An iterated
fixed point operator builds up dynamic strata by constructing
successive partial interpretations as follows.
%----------------------------------------------------------------------
\begin{definition}[Iterated Fixed Point and Dynamic Strata] \label{def:wfm}
\label{def:IFP}
For a normal program $P$ let

\begin{center}
$  \begin{array}{rcl}
          WFM_0 & = & \langle \emptyset ; \emptyset \rangle;      \\
 WFM_{\alpha+1} & = &       WFM_{\alpha} \cup
                                \langle \kcaltrue^P_{WFM_\alpha};\calfalse^P_{WFM_\alpha} \rangle; \\
     WFM_\alpha & = & \bigcup_{\beta < \alpha} WFM_\beta, \mbox{ for limit ordinal }\alpha.
  \end{array}
$
\end{center}

\noindent
  $WFM(P)$ denotes the fixed point interpretation $WFM_\delta$,
  where $\delta$ is the smallest (countable) ordinal such that both
  sets $\kcaltrue^P_{WFM_\delta}$ and $\calfalse^P_{WFM_\delta}$ are empty.
%($\delta$ exists, and is
%  a countable ordinal because both $\kcaltrue_I$ and $\calfalse^P_I$ are monotonically
%  increasing).
% We refer to $\delta$ as the {\em depth} of program $P$.
The {\em stratum} of atom $A$, is the least ordinal $\beta$ such that
   $A \in WFM_{\beta}$.
% (where $A$ may be either in the true or false
%   component of $WFM_{\beta}$).
\end{definition}
%------------------------------------------------------------------------()
%
\cite{Przy89d} shows that %the iterated fixed point
$WFM(P)$ is in fact the well-founded model and that any undefined
atoms of the well-founded model do not belong to any stratum --
i.e. they are not added to $WFM_{\delta}$ for any ordinal
$\delta$. Thus, a program is \emph{dynamically stratified} if every
atom belongs to at least one stratum.

This section has considered normal logic programs extended both with
explicit negation.  We note that if all atoms in a program $\cP$ have
the annotation 1, Definition~\ref{def:wfm} reduces to the definition
of the Well0founded Semantics with Explicit Negation~\cite{AlDP95},
and if $\cP$ also does not contain explicit negation, the definition
reduces to the definition of Well-Founded Semantics in \cite{Przy89d}.

%------------------------------------------------------------------------
% taking out fixed-order stuff until we put LPADs in (though maybe we
% wont even need it then )

%\input{fixed-order-dynstrat}


\documentclass[12pt, a4paper]{article}

\include{preamble}
\usepackage{lipsum} % <---------- remove me

\title{My lovely paper}
\include{author_vincent}
\date{\today}

\begin{document}

\maketitle

\begin{abstract}
Short introduction to subject of the paper \ldots
\end{abstract}

\section{Introduction}
This is a reference \citep{Vincent:2015dh}

\lipsum[1-2] % <---------- remove me

\begin{equation}
\label{eq1}
  y = mx+c
\end{equation}

\lipsum[2] % <---------- remove me

\begin{figure}[t!]
	\centering
	\includegraphics[]{bayes_net_general.pdf}
	\caption{This is a caption}
	\label{fig:first_figure}
\end{figure}

\lipsum[3-5] % <---------- remove me

% REFERENCE SECTION
\clearpage
\bibliographystyle{apacite}
\bibliography{my_references}

\end{document}


\chapter{Exponentials and Logarithms}

\section{Properties of the exponential function}

\section{Gradient of \(e^{kx}\)} % TODO: Typeset better
\section{Properties of the logarithm}
\section{Laws of logarithms}
\section{Equations involving exponentials}
\section{Reduction to linear form}
\section{Modelling using exponential functions}


%!TEX root = main.tex

This chapter describes the technical details of the implementation of signature-based partition refinement.
It contains a condensed version of several sections from~\cite{DBLP:conf/tacas/DijkP16}.

\section{Decision diagram algorithms in Sylvan}

In {symbolic model checking}, sets of states and transitions are represented by their characteristic function, rather than stored individually.
With states described by $N$ Boolean variables,
a set $S\subseteq \mathbb{B}^N$ can be represented by its characteristic function $f\colon \mathbb{B}^N\rightarrow \mathbb{B}$, where $S=\{s\mid f(s)\}$.
%
%
%
Binary decision diagrams (BDDs) are a concise and canonical representation of Boolean functions.

An (ordered) BDD is a directed acyclic graph with leaves $0$ and $1$. Each internal node has a variable label $x_i$ and two outgoing edges labeled $0$ and $1$. Variables are encountered along each path according to a fixed variable ordering. Duplicate nodes and nodes with two identical outgoing edges are forbidden.
%
%\begin{definition}
%An (ordered) BDD is a directed acyclic graph with the following properties:
%\begin{compactenum}
%\item There is a single root node and two terminal nodes 0 and 1.
%\item Each non-terminal node $p$ has a variable label $x_i$ and two outgoing edges, labeled 0 and 1; we write $\text{lvl}(p)=i$ and use $p[0]$ and $p[1]$ for the children.
%\item For each edge from node $p$ to non-terminal node $q$, $\text{lvl}(p)<\text{lvl}(q)$.
%%\item There are no {duplicate} nodes, i.e.,\\$\forall p\forall q\cdot(\text{lvl}(p)=\text{lvl}(q)\wedge p[0]=q[0]\wedge p[1]=q[1])\rightarrow p=q$.
%\item There are no duplicate nodes $\langle x_i,p[0],p[1]\rangle$ (i.e., maximum sharing)
%\item There are no redundant nodes (nodes with $p[0]=p[1]$)
%\end{compactenum}
%\end{definition}
%
%
%%
It is well known that for a fixed variable ordering, every Boolean function is represented by a unique BDD.

In addition to BDDs with leaves $0$ and $1$, multi-terminal binary decision diagrams have been proposed with leaves other than $0$ and $1$, representing functions from the Boolean space $\mathbb{B}^N$ onto any finite set. For example, MTBDDs can have leaves representing integers (encoding $\mathbb{B}^N\rightarrow\mathbb{N}$), floating-point numbers (encoding $\mathbb{B}^N\rightarrow\mathbb{R}$) and rational numbers (encoding $\mathbb{B}^N\rightarrow\mathbb{Q}$).
Partial functions are supported using a terminal leaf $\bot$.% (in practice we use the Boolean leaf $0$ for non-Boolean codomains).

Sylvan~\cite{DBLP:conf/tacas/DijkP15} implements parallelized operations on decision diagrams using parallel data structures and work-stealing.
Work-stealing is a load balancing method for task-based parallelism.
Recursive operations, such as most BDD operations, implicitly form a tree of tasks.
Independent subtasks are stored in queues and idle processors steal tasks from the queues of busy processors.

\begin{algorithm}[t!]
\Def{\Apply{$x$, $y$, $\textsf{F}$}}{
    \lIf{$(x,y,\textsf{F})\in\textsf{cache}$}{
        \Return{$\textsf{cache}[(x,y,\textsf{F})]$}\tcc*[f]{get from cache}
    }
    \lIf{$x$ and $y$ are terminals}{
        \Return{$\textsf{F}(x,y)$}\tcc*[f]{apply operator \textsf{F}}
    }
    $v$ = topVar($x$,$y$) \;
    low $\leftarrow$ \Apply{$x_{v=0}$, $y_{v=0}$, $\textsf{F}$} \tcc*[r]{execute in parallel}
    high $\leftarrow$ \Apply{$x_{v=1}$, $y_{v=1}$, $\textsf{F}$} \;
    result $\leftarrow$ \BDDnode{$v$, low, high} \tcc*[r]{compute result}
    $\textsf{cache}[(x,y,\textsf{F})]$ $\leftarrow$ result \tcc*[r]{put in cache}
    \Return{result}
}
\caption{Generic algorithm that applies a binary operator \textsf{F} to BDDs $x$ and $y$.}
\label{alg:generic}
\end{algorithm}

Algorithm~\ref{alg:generic} describes the implementation of a generic binary operation $\textsf{F}$.
%
%
%
BDD operations mainly consist of consulting an operation cache, performing some recursive step, and creating new BDD nodes using a unique table.
The operation cache is required to reduce the time complexity of BDD operations from exponential to polynomial in the size of the BDDs.
Sylvan uses a single shared unique table for all BDD nodes and a single shared operation cache for all operations.
To obtain high performance in a multi-core environment, the datastructures for the BDD nodes and the operation cache must be highly scalable.
Sylvan implements several non-blocking datastructures to enable good speedups.

To compute symbolic signature-based partition refinement, several basic operations must be supported by the BDD package (see also \cite{DBLP:conf/atva/WimmerHHSB06}).
%
%
%
Sylvan implements basic operations such as $\wedge$ and \texttt{if-then-else}, and existential quantification $\exists$.
Negation $\neg$ is performed in constant time using complement edges.
To compute relational products of transition systems, there are operations \texttt{relnext} (to compute successors) and \texttt{relprev} (to compute predecessors and to concatenate relations), which combine the relational product with variable renaming.
The operation \texttt{and\_exists} computes the traditional relational product without variable renaming.
Similar operations are also implemented for MTBDDs.
%
%
%
Sylvan is designed to support custom BDD algorithms.
We present two custom algorithms below.
%We also use custom extensions of traditional BDD algorithms to avoid variable renaming.

\section{Encoding of signature refinement}

We implement symbolic signature refinement similar to~\cite{DBLP:conf/atva/WimmerHHSB06}.
%
%
Unlike~\cite{DBLP:conf/atva/WimmerHHSB06}, we do not refine the partition with respect to a single block, but with respect to all blocks simultaneously.
% like in~\cite{DBLP:journals/entcs/BlomO03,DBLP:conf/qest/Derisavi07}.
%
%
%
We use a binary encoding with variables $s$ for the current state, $s'$ for the next state, $a$ for the action labels and $b$ for the blocks.
We order BDD variables $a$ and $b$ after $s$ and $s'$, since this is required
to efficiently replace signatures $(a,b)$ by new block numbers $b$ (see below).
Variables $s$ and $s'$ are interleaved, which is common in the context of transition systems.

\begin{figure}[tp]
\begin{center}
\scalebox{0.9}{
\begin{tikzpicture}
\coordinate (a) at (250:2);
\coordinate (b) at (290:2);
\coordinate (c) at (250:3);
\coordinate (d) at (290:3);
\draw (0,0) -- (c);
\draw (0,0) -- (d);
\draw (a) -- (b);
\draw (c) -- (d);
\draw (0,-1.5) node {$s,s'$};
\draw (0,-2.35) node {$a$};
\draw (0,-4.5) node {BDD $\mathcal{T}(s,s',a)$};

\coordinate (a0) at (3,0);
\coordinate (aa) at ($(a0) + (250:2)$);
\coordinate (ab) at ($(a0) + (290:2)$);
\coordinate (ac) at ($(a0) + (250:3)$);
\coordinate (ad) at ($(a0) + (290:3)$);
\coordinate (ae) at ($(a0) + (250:4)$);
\coordinate (af) at ($(a0) + (290:4)$);
\draw (a0) -- (ae);
\draw (a0) -- (af);
\draw (aa) -- (ab);
%\draw (ac) -- (ae);
\%draw (ad) -- (af);
\draw (ac) -- (ad);
\draw (ae) -- (af);
\draw (3,-1.5) node {$s$};
\draw (3,-2.35) node {$a$};
\draw (3,-3.25) node {$b$};
\draw (3,-4.5) node {BDD $\sigma_T(s,a,b)$};

\coordinate (b0) at (6.5,0);
\coordinate (ba) at ($(b0) + (250:2)$);
\coordinate (bb) at ($(b0) + (290:2)$);
\coordinate (bc) at ($(b0) + (250:3)$);
\coordinate (bd) at ($(b0) + (290:3)$);
\coordinate (be) at ($(b0) + (250:4)$);
\coordinate (bf) at ($(b0) + (290:4)$);
\draw (b0) -- (ba);
\draw (b0) -- (bb);
\draw (ba) -- (bb);
\draw (bc) -- (be);
\draw (bd) -- (bf);
\draw (bc) -- (bd);
\draw (be) -- (bf);
\draw ($(b0) + (0,-1.5)$) node {$s$};
\draw ($(b0) + (0,-3.25)$) node {$b$};
\draw ($(b0) + (0,-4.5)$) node {MTBDD $\sigma_R(s,b)$};

\coordinate (c0) at (-3,0);
\coordinate (ca) at ($(c0) + (250:2)$);
\coordinate (cb) at ($(c0) + (290:2)$);
\coordinate (cc) at ($(c0) + (250:3)$);
\coordinate (cd) at ($(c0) + (290:3)$);
\coordinate (ce) at ($(c0) + (250:4)$);
\coordinate (cf) at ($(c0) + (290:4)$);
\draw (c0) -- (ca);
\draw (c0) -- (cb);
\draw (ca) -- (cb);
\draw (cc) -- (ce);
\draw (cd) -- (cf);
\draw (cc) -- (cd);
\draw (ce) -- (cf);
\draw ($(c0) + (0,-1.5)$) node {$s$};
\draw ($(c0) + (0,-3.25)$) node {$b$};
\draw ($(c0) + (0,-4.5)$) node {BDD $\mathcal{P}(s,b)$};
\end{tikzpicture}
}
\end{center}
\caption{Schematic overview of the BDDs in signature refinement}
\label{fig:bdds}
\end{figure}

To perform symbolic bisimulation we represent a number of sets by their characteristic functions. See also Figure~\ref{fig:bdds}.
\begin{itemize}
\item A set of states is represented by a BDD $\mathcal{S}(s)$;
% with $\mathcal{S}(s)=1\Leftrightarrow s\in S$
\item Transitions are represented by a BDD $\mathcal{T}(s,s',a)$;
\item Markovian transitions are represented by an MTBDD $\mathcal{R}(s,s')$, with leaves containing rational numbers ($\mathbb{Q}$);
\item Signatures $\textbf{T}$ and $\textbf{B}$ are represented by a BDD $\sigma_T(s,a,b)$;
\item Signatures $\textbf{R}^s$ and $\textbf{R}^b$ are represented by an MTBDD $\sigma_R(s,b)$.
\end{itemize}

In the literature,
%~\cite{DBLP:conf/cav/BoualiS92,DBLP:conf/tacas/Derisavi07,DBLP:conf/qest/Derisavi07,DBLP:conf/atva/WimmerHHSB06},
three methods have been proposed to represent $\pi$.
%
%
%
\begin{enumerate}
\item As an equivalence relation, using a BDD $\mathcal{E}(s,s')=1$ iff $s\equiv_{\pi} s'$.
\item As a partition, by assigning each block a unique number, encoded with variables $b$, using a BDD $\mathcal{P}(s,b)=1$ iff $s\in C_b$.
\item Using $k=\ceil{\log_2 n}$ BDDs $\mathcal{P}_{0},\dotsc,\mathcal{P}_{k-1}$ such that $\mathcal{P}_i(s)=1$ iff $s\in C_b$ and the $i^\text{th}$ bit of $b$ is 1. This requires significant time to restore blocks for the refinement procedure, but can require less memory.
\end{enumerate}

%We use variables $t$ and $b$ instead of $s$ and $b$ in $\mathcal{P}$ to avoid creating a custom BDD operation for computing $\exists t\colon \mathcal{T}(s,t,a) \wedge \mathcal{P}(t,b)$.
%Also, \cite{DBLP:conf/atva/WimmerHHSB06} often requires renaming operations to get $\mathcal{P}(t,b)$ from $\mathcal{P}(s,b)$, which is unnecessary and expensive.

%Furthermore, the encoding of block numbers is such that the the lowest significant bit comes first...

%We represent the partition by the assignment to a block number.
%An alternative method to represent the partition, used by \cite{DBLP:conf/cav/BoualiS92} and \cite{DBLP:journals/ijfcs/MummeC13}, stores the equivalence relation in a BDD $\mathcal{R}(s,t)$.
We choose to use method 2, since in practice the BDD of $\mathcal{P}(s,b)$ is smaller than the BDD of $\mathcal{E}(s,s')$.
Using $\mathcal{P}(s,b)$ also has the advantage of straight-forward signature computation.
%In cases where only few blocks contain multiple states, $\mathcal{P}(s,b)$ blows up to a large tree, assigning each state to a different block.
%In such cases, the BDD for $\mathcal{E}(s,t)$ would be much smaller, since less states are equivalent.
%
%
The logarithmic representation is incompatible with our approach, since we refine all blocks simultaneously. Their approach involves restoring individual blocks to the $\mathcal{P}(s,b)$ representation, performing a refinement step, and compacting the result to the logarithmic representation. Restoring all blocks simply computes the full $\mathcal{P}(s,b)$.

We represent Markovian transitions using rational numbers, since they offer better precision than floating-point numbers.
The manipulation of floating-point numbers typically introduces tiny rounding errors, resulting in different results of similar computations.
This significantly affects bisimulation reduction, often resulting in finer partitions than the maximal bisimulation~\cite{DBLP:conf/mmb/WimmerB10}.

\section{The \texttt{refine} algorithm}
\label{sec:refine}
\input{refine}

\section{Computing inert transitions}
\label{sec:inert}
\input{inert}


\documentclass[12pt]{beamer}

\usepackage{tikz}
\usepackage{graphicx}
\usepackage[T1]{fontenc}
\usepackage{amsmath}
\usepackage{multicol}
\usepackage{booktabs, braket} % For formal tables
\usepackage{cleveref}
\usepackage{graphicx}
\usepackage{blkarray}
\usetikzlibrary{graphs, graphs.standard}

\DeclareMathOperator{\E}{\textrm{E}}		     % expected value
\DeclareMathOperator{\pr}{\mathrm{P}}		     % probability
\DeclareMathOperator{\cov}{t_{cov}}	             % cover time

\DeclareMathOperator{\X}{\mathbb{X}}		     % expected value
\DeclareMathOperator{\tr}{\text{tr}}		     % trace
\DeclareMathOperator{\Ap}{$A$ ^\prime} 		     % A'
\DeclareMathOperator{\bp}{$b$ ^\prime} 		     % b'
\DeclareMathOperator{\Xb}{\mathcal{X}}		     % big X

\title{Semidefinite Programming and Quantum Algorithms}
\author{Michael Czekanski \& R. Teal Witter}
\institute{Middlebury College}
\date{November 5, 2019}

\begin{document}
\graphicspath{{./../figures/}}

\frame{\titlepage}

\begin{frame}{Overview}
\tableofcontents
\end{frame}

\section{Query Complexity}
\begin{frame}{Query Complexity}
\begin{center}
The number of times an algorithm queries the input.
\end{center}
\begin{table}[]
\begin{tabular}{cc}
\hline
\textbf{Input} & \textbf{Output} \\ \hline
00             & 0               \\
01             & 1               \\
10             & 1               \\
11             & 1               \\ \hline
\end{tabular}
\end{table}
\end{frame}

\section{Semidefinite Programming}

\begin{frame}{Quantum Query Complexity as SDP: Reichardt \cite{reichardt2009span}}
\begin{align} \label{eq:reichardtObj}
    f_{\text{bound}} = \min_{\X} M(\X) = \max_{y \in D} \sum_{j \in [n]}
    \bra{y,j}\X\ket{y,j}
\end{align}
Subject to
\begin{align}\label{Eq:reichardtSemi}
    \X \succcurlyeq 0
\end{align}
\begin{align}\label{Eq:reichardtOffDiag}
    \forall (y,z) \in F \sum_{j \in [n]: y_j \ne z_j}
    \bra{y,j} \X \ket{z, j} = 1.
\end{align}
\end{frame}

\begin{frame}{A Standard Form: Boyd\cite{boyd2004convex}}
Minimize
\begin{align}\label{Eq:boyd_obj}
    M(\X) = \tr(C\X)
\end{align}
Subject to
\begin{align} \label{Eq:boydSemi}
    \X \succcurlyeq 0
\end{align}

\begin{align} \label{Eq:boydTraceCon}
    \tr(A_i \X) = b_i  \text{$\qquad$ for $i \in \{1,...,p\}$}
\end{align}
\end{frame}

\section{Alternating Direction Methods}
\begin{frame}{We use ADM \cite{adm}}
\centering
\includegraphics[scale=.15]{figures/adm_algorithm}
\bigskip
\begin{itemize}
    \item Easy to implement
    \item Exploits sparsity (yay zeroes!)
    \item Exploits constraint orthogonality
\end{itemize}
\end{frame}

\section{Our Work}
\begin{frame}{Numerial Results: Our Algorithm is Good.}
\centering
\includegraphics[scale=.5]{figure_all_or_complexity.eps}
\end{frame}

\begin{frame}{Run Time Results: Our Algorithm is Bad.}
\centering
\includegraphics[scale=.5]{figure_all_or_time.eps}
\end{frame}

\begin{frame}{We Have Plans (For the Future)}
\begin{itemize}
    \item Document
    \item More optimization
    \item Span program
\end{itemize}
\end{frame}

\begin{frame}{Thank you!}
\bibliographystyle{abbrv}
\bibliography{main}
\end{frame}

\end{document}


% Add a bit of space between paragraphs (copied from Karl Broman)
\setlength{\parskip}{0.5\baselineskip}

% Add abstract (copied from Karl Broman)
\begin{quote}
\small
\articleABSTRACT
\end{quote}

\section{Introduction}

\subsection{Historical context}

%Summarize the scientific question that your code was written to explore.
This paper aims to computationally reproduce my own statistical analysis from 16~years ago as part of the 10-year reproducibility challenge \supercite{Hinsen2019}. The analysis examined seasonal patterns in coronary heart disease in multiple countries \supercite{Barnett2004}. In most countries coronary heart disease deaths and emergency admissions to hospital have a strong seasonal pattern with a peak in winter and nadir in summer \supercite{Stewart2017}. There is an interesting variability between-countries in the size and timing of the winter peak. A better understanding of the differences between countries could help our understanding of the underlying causes of the winter peak in disease.

The analysis examined monthly time series of coronary  events (fatal and non-fatal) from 35~locations in 21~countries. The time series were 8 to 14 years in length and the earliest year of data was 1980. The analysis aimed to split the time series into a long-term trend, seasonal pattern(s) and remaining noise. There were two approaches, one that used a two-stage approach by first removing the trend, and another that estimated the trend and season together. The trend was estimated using the Kalman filter and the seasonal patterns were estimated using sinusoids. The estimates were made using Markov chain Monte Carlo.

%Describe the computational context: Which hardware was used to run the code? Which software infrastructure? Which constraints existed on software development? Which technical choices (language, libraries, ...) were made? Were reproducibility and/or re-usability important criteria?

\subsection{Original source code}

The original code was written in SAS (version~8.00 for Windows) and chain convergence was checked using the ``coda'' package (version unknown) in R \supercite{coda}. The code was based on Matlab code that I wrote during my PhD \supercite{Barnett2003} at The University of Queensland. I converted the code to SAS because I did not have access to Matlab after finishing my PhD and I hoped that more people would be able to re-use my SAS code to apply to their own time series data.
%It was not common practice at the time to share code and data in order to allow others to reproduce results.
The SAS code was first published in November 2004 with an update in August 2006, but I did not record what changes were made.
The paper states the code was published in September 2002 but this is likely a typo in the year given that the paper was not submitted until January 2004.

%About the original source code: Was it published? Was it archived somewhere? Was there a license for it?
The original code was published in an online appendix to the paper that was separate from the journal in November 2004 at this address: \url{http://www4.ktl.fi/publications/monica/chd_seasonal/appendix.htm}. However, the site has since moved to: \url{https://www.thl.fi/publications/monica/chd_seasonal/appendix.htm} (accessed 25 January 2020). The web site was created by the WHO MONICA Project specifically for appendices to papers that used the MONICA data. MONICA stands for monitoring trends and determinants in cardiovascular disease. The MONICA Project was a large multi-country study that aimed to examine several aspects of coronary heart disease \supercite{Tunstall2003}. It was a well-managed project that had staff who assisted with access to the data and helped add my code to the web.

%Please provide only information you are certain about. It's OK to forget technical details after many years.

\section{Results}

%Retrieval of the software
%Was it easy to find a copy of your source code?
%Was it easy to locate and setup the dependencies?
%Provide a list of all dependencies (libraries but also tools such as compilers).

\subsection{Retrieval of the software}

Most of my SAS code was easy to find, both on the web and on an old compact disc in a desk draw. I made this compact disc of my files when I moved jobs. I did not keep the data because I was more interested in other people using my code for their own data rather than replicating our published results. I believe that another reason for deleting the data was to save space, as the files were relatively large by the standards of the day (around 31,000~kilobytes for data in text format and 70,000~kilobytes for data in SAS format). Luckily the original data files were available in text files in fixed-width format on a compact disc attached to the MONICA monograph published in 2003 \supercite{Tunstall2003}.

My SAS code contained the macros needed to run the two statistical analyses. There was also SAS code to simulate a seasonal time series as an example data set. I found SAS code to read the MONICA data on my compact disc, but not on the web. I could not find the SAS code that applied the two methods to each location.

%Execution
%Describe what you did in order to run the software today.
%Did you succeed in running the software in a modern computational environment? Or did you have to search for old versions of tools and libraries?
%Describe the computational environment of the reproduction: hardware, operating system, compiler versions etc.
%Did you have to modify the software in order to make it run today?
%Were the original instructions that came with the software sufficient, or did you have to modify or extend them?

\subsection{Replication execution}

I used SAS version~9.4 for Windows (Microsoft Windows~10) to replicate the results; there was no need to use an earlier version of SAS.

My SAS files had relatively good instructions with a detailed header at the top of every file and comments throughout. However, some files had been adapted from the original analysis to secondary analyses, making the code a palimpsest with some commands commented out with notes such as, ``changed for weather analysis'' and ``sensitivity analysis of Ghent''. The correct data set was also ambiguous because alternative data sets had also been used (e.g., fatal vs non-fatal events).  In hindsight I should have kept the exact data and syntax files needed to re-create the published results.
%Mix of SAS files from different projects in the same folder.

The macros to run the analyses mainly needed only cosmetic edits, and I also put each macro in its own file rather than one overall file of macros. I also moved part of one macro, that estimated the standard deviation of the noise, into the macro that tested the periodogram for remaining seasonal structure. The largest change was having to re-write the files that applied the macros to the MONICA data. My SAS programming was rusty and I could not automate the process for each centre, so instead I created a new file for each analysis in each of three centres.

I did not repeat my analysis in all 35 locations, but instead did the three locations in the first figure as this felt sufficient to test the code.

The code to estimate the trend without the seasonal pattern used a slightly different parameterisation with ``tau'' as a ratio rather than an absolute value. ``tau'' controls the amount of change over time in the trend, with smaller values creating more linear trends. I cannot be sure that this version was the same code as the original.

\subsection{Closeness of the replicated results to the original}

%How close were the results you got to the originally published ones? Include the replicated data table and figure, as we do in other ReScience articles.
The original figure is shown in Figure~\ref{fig:original} and my replication in Figure~\ref{fig}. The figures show the estimates of the trend and season for three locations.
The results are similar except for the confidence intervals for the trend for the two-stage method, which are much narrower for the replication, but are centred on a similar mean.
This may be because I could not be certain that I had found the original code to perform the Kalman filter smoothing because of the difference in how ``tau'' was parameterised.
I also needed some trial-and-error to select ``tau'' which controls the smoothness of the trend and is defined by the user rather than being estimated by an algorithm.
%However, the estimates do look remarkably similar to my replicated estimates of the trend using the combined method. Hence another possibility is that the original figures were mislabeled as being from the two-stage method when in fact they came from the combined method.

\begin{figure}[!h]
    \centerline{\includegraphics[scale=0.9]{figures/original_figure.eps}}

\caption{Original results for three locations using the two-stage method (left and centre column) and combined method (right column). The left column shows the observed data, estimated trend and 95\% confidence interval. The middle column shows the periodogram of the de-trended data and limit for the test of seasonal structure (dotted line). The right column shows the observed data, estimated trend, and estimated annual seasonal pattern.}
	\label{fig:original}
\end{figure}

\begin{figure}[!h]
\centerline{Perth, Australia}
	\centerline{
    \includegraphics[scale=0.2]{figures/trend_twostage_perth.eps}
    \includegraphics[scale=0.2]{figures/periodogram_twostage_perth.eps}
    \includegraphics[scale=0.2]{figures/estimates_combined_perth.eps}
    }

\centerline{Belfast, UK}
    \centerline{
    \includegraphics[scale=0.2]{figures/trend_twostage_belfast.eps}
    \includegraphics[scale=0.2]{figures/periodogram_twostage_belfast.eps}
    \includegraphics[scale=0.2]{figures/estimates_combined_belfast.eps}
    }

\centerline{Warsaw, Poland}
    \centerline{
    \includegraphics[scale=0.2]{figures/trend_twostage_warsaw.eps}
    \includegraphics[scale=0.2]{figures/periodogram_twostage_warsaw.eps}
    \includegraphics[scale=0.2]{figures/estimates_combined_warsaw.eps}
    }

\caption{Replicated results for Figure~\ref{fig:original}}
	\label{fig}
\end{figure}

The estimates of the seasonal patterns for the three locations are replicated in Table~\ref{tab:season} which also shows the original results. The results are similar for the two-stage method, with most differences being within $\pm$~0.1. The biggest difference being for the estimate of the noise standard deviation in Belfast (9.0 versus 8.8).

\begin{table}[!h]
  \centering
  \begin{tabular}{rccc}\hline
 & Frequency,  & Amplitude,  &  Noise \\
Location & months & rate per 100,000 &  SD \\ \hline
\multicolumn{4}{c}{Two-stage method} \\ \hline
Perth, Australia & 2.3 & 2.0 (2.0) & 4.3 (4.2)\\
 & 12 & 2.6 (2.6) \\
Belfast, UK & 12 & 7.0 (7.0) & 8.7 (8.7) \\
Warsaw, Poland & 3.4 & 3.3 (3.3) & 7.5 (7.5)\\
               & 6  & 3.6 (3.6)\\
               & 12 & 3.2 (3.2)\\
\hline
\multicolumn{4}{c}{Combined method}  \\ \hline
Perth, Australia & 2.3 & 1.0 (1.0) & 3.9 (3.9)\\
 & 2.7 & ---- (0.8) \\
 & 3.8 & 1.3 (1.3)\\
 & 5.5 & 1.5 (1.4)\\
 & 12 & 2.7 (2.7)\\
Belfast, UK & 2.8 & ---- (1.6) & 9.0 (8.8) \\
            & 12 & 7.1 (7.0) &  \\
Warsaw, Poland & 3.4 & 3.4 (3.4) & 7.9 (7.8)\\
               & 6  & 3.7 (3.7)\\
               & 12 & 3.3 (3.3)\\
\hline
\end{tabular}
 \caption{Estimates of seasonal patterns using the two-stage and combined methods for three locations. The original results are in brackets. Each row in the table corresponds to a seasonal frequency at a location. SD = standard deviation.}
  \label{tab:season}
\end{table}

There were differences in the frequencies for the combined method, as two seasonal frequencies were not included in the new results, one in Perth and one in Belfast. Although the amplitudes were relatively small (1.6 or less), hence these are smaller seasonal patterns that are marginally important.

The difference in seasonal frequencies could be because of a key ambiguity in the original paper regarding the decision to add additional seasonal components for the combined approach. After fitting a candidate model, the residuals were tested to look for additional seasonal structure. If structure existed, then the model was modified to add an additional seasonal component as necessary, followed by another test of the residuals. The code produced both the periodogram and the spectrum, which is a smoothed version of the periodogram. The periodogram tended to flag significant seasonal patterns when the spectrum did not, and I used the spectrum to make the decisions because some of the statistically significant patterns found by the periodogram were only just above the 0.05 threshold used to judge statistical significance. This part of the replication was therefore somewhat subjective.

Some of the discrepancy in results in Table~\ref{tab:season} could be because the estimates are made using Markov chain Monte Carlo which uses random steps to make estimates. Hence the estimates will depend on the original random number seed. However, I used 5,000 estimates with a burn-in of 500, so these differences should be minor and may explain some of the small differences in the first decimal place for the amplitude and noise estimates. This potential discrepancy would have been avoided if the original code had specified the random number seed.

%If you had to make any modifications to the software or to the instructions that were supplied with it, try to describe which competence another researcher would need in order to do the same work. Would a general familiarity with your programming language and environment have been sufficient?

Overall the replication took about 21~hours. This was mostly the fiddly task of working out which macros to run and re-creating the figures and tables. Given the amount of code that had to be re-created, I think that only someone  familiar with SAS would have been able to replicate the results. My familiarity with the MONICA Project also helped me create the new code, hence I am not certain that an outsider could have replicated the results, perhaps only by trial-and-error.

%Software
%If the licenses applying to the original software allow, please make your old software available as a repository on GitHub or an equivalent site. In the best case scenario, in which the old code is publishable, here is the procedure:

%Add your old software as a single commit (ideally the initial one) of a new source repository.
%Edit the code as necessary to reproduce the results in today's computational environment, committing the changes in logical units in the process. Include the additional tools (scripts, makefiles, ...) that you wrote for re-running your software and the additional instructions.
%Make sure your repository contains a license.
%Finally, submit your repository to Software Heritage for archiving.

\section{Discussion}

%In addition to the content covered below, we are interested in the insight resulting from the reproducibility experience in relation to the choice of technology, and in your opinion on the original source code (clarity, documentation, etc).

It was an interesting exercise to bring this old code back to life. It would have been easier and faster if my original code had been properly arranged and prepared solely to repeat the original analyses. However, I was not aware of guidelines for curating code back in 2004. My original code had many comments which were useful, but there were problems with the overall file structure and how the files related to one another. There were also ambiguities with the dates of the code, and the changes I made after posting the original files. This is now solved by sites that track and date-stamp all changes, such as \textit{GitHub}.

My main reason for sharing the original code was so that others could adapt it to their own data. However, of the 21 papers that cited the original paper which I could access, none of them stated that they re-used my SAS code.
%I was not focused on the nitty-gritty of reproducing the original results and my code was in bad shape to do this.

% One instead cited the available R code to run the same analysis.
% Not confident it was exactly the same, but broadly similar.

Overall the replication was broadly successful in terms of re-creating the key figure and estimates for three locations. There was subjectivity in modelling decisions concerning the smoothness of the trend and number of seasonal components, which made perfect replication difficult.

%The ``combined'' model in the SAS files are available in the ``season'' package in R \supercite{season} using the ``nscosinor`` command.