\section{Abstract}
This article describes an attempt to reproduce key findings of a study of murine hippocampal oscillations published in 2007 \cite{hentschke_muscarinic_2007}. Being a submission to the 'Ten Years Reproducibility Challenge', it focuses on the process of reviving the Matlab code underlying the analyses, particularly the neuronal signal processing toolbox created for the study and its follow-ups.

\section{Introduction}
\subsection{Neuroscientific background}
\label{subsec:background}
Neuronal activity in mammalian central nervous systems is often oscillatory. Intensity, frequency, and other characteristics of the oscillations are closely associated with behavior and arousal state. As this association suggests a functional role of oscillations, they have been studied extensively. One of the most conspicuous cases of behavior-dependent oscillatory neuronal activity occurs in hippocampus, a brain region pivotal for navigation and memory formation: prominent rhythms exist in the theta (4-12 Hz) and gamma (30-90 Hz) bands.
Interestingly, the oscillations are coupled in a nonlinear fashion: the magnitude of gamma oscillations varies cyclically at theta frequency \cite{soltesz_low-_1993} (Fig.~\ref{fig:hip_oscill}). Such phase-amplitude coupling (PAC), also termed 'nesting' of oscillations, has been postulated to be important for memory formation and retrieval \cite{lisman_storage_1995, hyafil_neural_2015}.

A large body of research has shown that both theta and gamma rhythms depend on the neuromodulator acetylcholine, but before our study \cite{hentschke_muscarinic_2007} it had not been clear whether and how their PAC would change with cholinergic signalling. As a first step towards shedding light on this, we performed experiments on mice with extracellular multielectrode arrays implanted into area CA1 of hippocampus. Each extracellular electrode picked up local field potentials (LFPs) -- tiny, local, time-dependent changes in the electrical potential relative to a distant reference point (Fig.~\ref{fig:hip_oscill}B). The mice were behaviorally scored and the LFPs recorded for periods of ca.\ 90 min, including administration of a blocker of muscarinic cholinergic receptors 30 min into the experiments.

\subsection{Code base and scope of reproduction}
The code presented here consists of two major parts. One is a toolbox of bespoke data crunching routines which performed classical time and frequency domain analyses. These were geared towards revealing spatial and temporal relations between signals from the different recording electrodes (crosscorrelations, coherence, etc.), with a focus on within-site phase-amplitude coupling. The 'main' function of this toolbox -- termed 'rmouse' -- integrated behavioral scoring data, partitioned the computational results according to the animals' behavior, and created summary figures for each recording (Fig.~\ref{fig:rmouse_workingstyle}). Also included were numerous post-processing routines with specific jobs like aggregation of data.

The other part of the code consisted of project-specific routines for statistical analyses of the data, and for the generation of publication-style plots.

The major focus here is on getting the toolbox and associated post-processing code to run with a Matlab version as recent as possible. Moreover, I attempted to reproduce those figures of the original study which conveyed the key finding of the paper, namely a reduction of PAC in the presence of muscarinic blockers (Fig.~\ref{fig:repro}).

\subsection{Expectations and results in a nutshell}
As Matlab code is generally quite backwards compatible, I was optimistic about overall success. The optimism was justified in terms of a general repeatability of the analysis; both toolbox and post-processing code could be brought into a functional state, and figures be created from the newly computed analysis results. However, data could only be retrieved for three of the five experimental animals of the original study, so the figures shown here are not exact reproductions, but must rather be seen as proofs of concept.

\begin{figure}
	\centering
	\includegraphics[width=0.9\textwidth]{figures/figures_tenYears_01.png}
	\caption{Theta and gamma oscillations, two prominent hippocampal oscillations under investigation in the original study. A, sketch of a section of rodent hippocampus and the location of the multielectrode recording arrays (orange circles). Shown are also two pyramidal cells with their basal dendrites in \textit{stratum oriens} and apical dendrites in \textit{strata radiatum} and \textit{lacunosum-moleculare}. B, wide-band local field potential data (top trace), and filtered versions (center and bottom, respectively) of a short stretch of oscillatory activity in the hippocampus of a mouse exploring its surroundings. Note how troughs of the slower theta oscillation (4-12 Hz) coincide with peaks in the envelope of the faster gamma oscillations (30-90 Hz).}
	\label{fig:hip_oscill}
\end{figure}

\subsection{Computational context}
% Describe the computational context: Which hardware was used to run the code? Which software infrastructure? Which constraints existed on software development? Which technical choices (language, libraries, ...) were made? Were reproducibility and/or re-usability important criteria?
% About the original source code: Was it published? Was it archived somewhere? Was there a license for it?

I developed the largest part of the code during a postdoctoral stay 2003-2005 in the laboratory of Robert A. Pearce (University of Wisconsin at Madison). The core of the analysis strategy had been devised by Matthew I. Banks (also UW Madison), who had also implemented a first version in a spreadsheet-cum-script language (Origin). Due to limitations inherent in the language it was decided to implement and extend the analysis in a more versatile language. Matlab was the natural way to go, as it was (and still is) the standard programming environment in many electrophysiology laboratories, and I had used it extensively before. In our original study \cite{hentschke_muscarinic_2007}, Matlab versions 6.5 - 7.4 (R13 - R2007a) were used. For later projects, the code has been continuously adapted to Matlab versions up to 7.9 (R2009b), after which it lay dormant. Various Matlab toolboxes were required: most importantly, the Signal Processing Toolbox; depending on the exact post-processing routines, the Curve Fitting and Statistics Toolboxes were also required.

A pivotal issue for the project, and in general for any data-focused undertaking, was the import of raw data. The time series data existed in a proprietary format, the Axon Binary Format (ABF), in which each data point is stored as a 16-bit integer. A function for efficiently importing data of this format into Matlab did not exist. Although the software package provided by the vendor (former Axon Instruments, now Molecular Devices) allowed an export of the data into text format, this was not considered practical (tedium involved in manual conversion, inflation of data size). Hence, I wrote a routine for importing raw data in this format into Matlab and submitted it to the MathWorks Central File Exchange \cite{hentschke_abfload_nodate}. Initially considered a spin-off of minor importance, this routine and its upgrade \cite{collman_fcollmanabfload_nodate} found a widespread distribution in the neuroscience community, and also sparked implementations in other languages \cite{caldwell_abfload_nodate, harden_swhardenpyabf_nodate}. An ironic twist is that the more fail-safe way of implementing a data importing routine for the windows operating system, namely code employing Molecular Devices' dynamic link library (DLL), would have made sense at the project's start in 2003, but has since the introduction of 64-bit versions of Matlab been rendered impractical by the limitation of this DLL to 32 bit (confirmed via an exchange with Molecular Devices Customer Support in December 2019). Readers further interested in this topic may want to consult the "Unofficial Guide to the ABF File Format" by Scott Harden \cite{harden_scott_w_unofficial_nodate}.

As all code was written in pure Matlab, and data import had been solved within Matlab, there were no extraneous dependency issues.

Although reproducibility of the study results was not an imminent concern to me back then, reusability of the code had definitely been an important design aspect early on. Series of experiments with a similar setup were in the pipeline, and given the limited term of my stay it was clear that the code would also be used by my colleagues \cite{perouansky_amnesic_2007, hentschke_altered_2009, balakrishnan_midazolam_2014}. Therefore, I wrote a manual with illustrations, with the intent of providing sufficient information for lab members familiar with the scientific background to find their way through the code. I also commented the code prolifically, especially the scripts calling the toolbox functions. The source code has never been properly archived or published, except for the data importing routine (until the reproducibility challenge came along) for two reasons: first, I considered it too specialized and tailor-made to be of much use for other groups. Second, making the toolbox open access would have required making it more user-friendly and extending the documentation substantially. Absent an academic reward system for such efforts not only for myself but also the hosting lab, I considered the necessary investment of time inappropriate.

I developed the code without any kind of version control worth the term, and left it 'as is' on my then machine in the laboratory. The toolbox's core code was in one dedicated directory, but auxiliary code was scattered throughout my Matlab code base, which of course also contained a great deal of unrelated code. I never saw the necessity of properly packaging the code: it would not be published, and almost all auxiliary functions employed by the toolbox were quite general and therefore also part of other (and future) programming projects. So, my justification for this practice was to avoid redundancy and the usual problems associated with maintaining multiple copies of code.

Towards the end of my stay in the laboratory, I simply mirrored my Matlab directory on other users' machines to get the toolbox to run there. After I left, I developed it further and adapted it to the needs of the subsequent projects, which went on for quite a while.

Hardware was an issue, albeit an uncritical one. The analyses were run on 32-bit Windows PCs with 4 GB of RAM and not-too-bad Intel Pentium processors available at the time (more precise specs are not known anymore). The electrophysiological signals were sampled at ca. 1000 Hz from up to 16 channels and typically lasted for 30 minutes; the resulting individual raw data files occupied less than 100 Mb on disk. Their size in RAM, after conversion from 16-bit integers to the default 64-bit floating type in Matlab, posed no critical challenge. However, the code also implemented the creation of surrogate time series. Dealing with these, as well as multiple filtered copies of the data in RAM in parallel required some thought-through strategies and juggling with computer resources, and was taxing for the CPU. Yet, even full-blown analyses of the whole data set were accomplished overnight.

\section{Retrieval of the software}
% Was it easy to find a copy of your source code?
% Was it easy to locate and setup the dependencies?
% Provide a list of all dependencies (libraries but also tools such as compilers).

Retrieval of the source code was easy; in fact, most of it did not have to be retrieved. As code files are ridiculously small even by the standards of the mid-2000s, I generally leave them on the hard disks of the computers in my hands. The hard part was figuring out the organization of the code files. There were the toolbox folder, still in my Matlab code base, and the folder containing code files for generating the publication figures. Localizing both required just a few minutes. I was also aware that several auxiliary functions, including the data-importing function, were outside the toolbox folder but inside my Matlab code base. However, I forgot that back then I placed scripts defining analysis parameters for each individual data file in yet a different location: namely, in the root folder for the raw data, as well as the subfolders therein. Only after I tried to run the first analysis step did I realize this.

\section{Execution}
% Describe what you did in order to run the software today.
% • Did you succeed in running the software in a modern computational environment? Or did you have to search for old versions of tools and libraries?
% • Describe the computational environment of the reproduction: hardware, operating system, compiler versions etc.
% • Did you have to modify the software in order to make it run today?
% • Were the original instructions that came with the software sufficient, or did you have to modify or extend them?
% • How close were the results you got to the originally published ones? Include the replicated data table and figure, as we do in other ReScience articles.
% If you had to make any modifications to the software or to the instructions that were supplied with it, try to describe which competence another researcher would need in order to do the same work. Would a general familiarity with your programming language and environment have been sufficient?

Reproduction was performed on a PC with Windows 7 and Matlab R2020a with the Signal Processing, Statistics and Machine Learning, and Parallel Computing Toolboxes installed. The PC had 32 GB of RAM, ample space on both hard disk and solid state drive, an Intel Core i7-4770K CPU with four physical cores, and an nvidida GeForce GTX1060 graphics card.

Reproducing data analysis including selected figure panels of the original paper required several processing steps.
I started with the first step -- processing the raw data -- by trying to locate the scripts which defined the analysis parameters for each individual file to be processed. After finding only one, I read the manual, and realized that both the majority of the data and data-specific scripts were missing, contrary to my first cursory impression when entering the challenge. Thanks to the efforts of a former colleague (acknowledgements) I was able to retrieve six complete sets of data and data-specific scripts from three of five experimental animals used in the original study. With these available, I finally remembered how the toolbox was supposed to work.

Before actually trying to run it, I assembled all code files I could identify at this stage as belonging to the project into a directory structure such that submitting the code to a repository would be possible. Next, I ran a dependency analysis on the main function of the toolbox and could identify a number of additional code files that were missing. After copying these to the project directory, I excluded my general Matlab code base from the path. Then, I tried in earnest to start number crunching. It took me about three hours to get the analyses to run on three experiments (two data files each). The adjustments required were, in order of severity:

\begin{enumerate}
	\item Retrieval of functions that were still missing from my general Matlab code base (about two dozen of files).
	\item 'Downgrading' auxiliary functions which had been in my general code base and together with it had evolved over time to the point of being incompatible with the toolbox.
	\item Updating code to be compatible with the recent version of Matlab.
\end{enumerate}

I was shocked to learn how many of the required functions I had to retrieve from my '\_legacy' folder. Had I not exerted caution in getting rid of code and instead deleted them, I would have spent hours on end to recapitulate what they were supposed to do, and possibly given up. A pleasant confirmation of expectations was the little effort I had to spend adapting the code to the most recent Matlab version. Essentially, all I had to do was write a wrapper for function \texttt{diag}, which did not accept cell arrays as inputs anymore. Surprisingly, graphics, including one very simple GUI, worked without hiccups; thanks to a major revision of graphics in R2014b, it also produced more aesthetically pleasing figures.

Moreover, with little effort, it was possible to run number crunching in an embarassingly parallel way using \texttt{parfeval}. This, as well as moving the data to a solid state drive, cut down processing time from a baseline of 152 min to a very agreeable 66 min (three experimental subjects, two recordings each, 100 instances of surrogate data generation each). Although exact numbers are impossible to come by, my estimate is that the combination of recent software and more modern (certainly not recent) hardware sped up the computations by a factor of at least eight. Yet more speedup would have required fundamental re-coding of the toolbox.

\begin{figure}
	\centering
	\includegraphics[width=0.9\textwidth]{figures/example_summaryFig_rmouse.jpg}
	\caption{Example of a summary figure produced by the rmouse toolbox from one of the analyzed data files. These summary figures were meant to provide a quick overview of the data and a glimpse at the computed quantities. The top panel and pie chart on the left illustrate the temporal sequence and proportions of time spent in various behavioural states, respectively, and also quantify recording periods unsuitable for analysis. Examples of data segments considered artifacts are depicted on the left, below the pie chart. All other panels show the results of spectral and time domain analyses.}
	\label{fig:rmouse_workingstyle}
\end{figure}

The next task was aggregating the data from the different experiments, and (re-)producing selected figure panels of the original study. Numerous scripts for various visualizations of the results existed; as these were not strictly documented and kept up to date, a trial-and-error period of identifying key pieces of code responsible for producing the plots ensued. Apart from this, and on top of adjustments similar to those required for the toolbox, I expected two additional problems here. First, the original code made use of the Curve Fitting Toolbox, which was not at my disposition now. Second, code producing 'Christmas Tree' plots -- adorned horizontal bar plots often featuring a coniferous shape (Fig.~\ref{fig:repro}) -- featured low-level graphics commands. Due to substantial changes of Matlab graphics in the interim, I knew that this code would have to be modified.
Both challenges proved surmountable. For the simple fits of 1D data, I used function \texttt{fitnlm}, and refurbished the affected code accordingly. The function producing the horizontal bar plots could also be updated without major pains. Following these fixes, key findings of the original study could be reproduced, albeit only with said partial data (Fig.~\ref{fig:repro}). All other figure panels featuring above-mentioned horizontal bar plots could be reproduced with the same data base by  changing the name of a target variable in a script and adapting the curve-fitting code (data not shown).

\begin{figure}
	\centering
	\includegraphics[width=0.9\textwidth]{figures/figures_tenYears_02.png}
	\caption{Reproduction of key findings. A, exemplary power spectral densities of the wide-band LFP ('raw') and of the gamma envelope of the LFP recorded at the hippocampal fissure. The light shaded area corresponds to the power of the gamma envelope in the theta range; the gray area represents the power of the gamma oscillations. This graph corresponds to Figure 5B in the original paper. B, laminar profile of the power of the gamma envelope in the theta band for immobile and exploring  animals (three of five from which data could be retrieved). The vertical position of the bars illustrates the location of the recording sites underlying the data. Gray, control; black, with the muscarinic blocker atropine. Error bars are standard deviations. Dotted lines are exponential fits to the data. This graph corresponds to Figure 5C in the original paper. C, laminar profile of the crosscorrelation between theta and the gamma envelope. Same conventions as in B apply. Dotted lines are polynomial fits (order 2) to the data. This graph corresponds to Figure 7A in the original paper. D, artistic rendering of theta oscillations, illustrating their highly dynamic nature. Time runs horizontally from left to right. Each set of concentric rings represents a theta peak (green) or trough (orange) detected on each of the 16 electrodes spanning hippocampus (and a trifle of the surrounding tissue). The more rings in a set, the stronger the normalized peak or trough amplitude (as amplitudes differ strongly between the layers, they were normalized for each recording electrode separately). The orientation of recording sites is as in the sketch in B (topmost row corresponds to \textit{stratum oriens/alveus}). On average, there is a phase lag of almost half a theta period between the layers harboring the apical and basal dendrites of the pyramidal cells, but individual theta 'beats' are very variable. The increase of such variability in the presence of atropine (data not shown) is a major cause of the deteriorated coordination between theta and gamma (as shown in C). Modeled after Figure 9A in the original paper.}
	\label{fig:repro}
\end{figure}

\section{Conclusion and personal evaluation}
During my last years in academia, I focused increasingly on professional programming, and eventually left academia in 2019 to work as a Data Scientist. Having along the way picked up at least a modicum of formal software education, my opinion on my original source code is mixed.

On the positive side, I commented the code heavily, particularly the scripts which other users of the code were to use. Later on, I also spent a considerable amount of time on the documentation. A crude form of logging was implemented. Moreover, I had a knack for making code run fast, and consider some aspects of the implementation valid even today.

However, a few of my past code design choices strike me as odd now, to say the least. First and foremost, this is above-mentioned liberal dispersal of code files across my entire code base, and the placement of scripts defining parameters in data directories. The cascades of scripts required for running analyses with the toolbox required immersion into the code, and could have been replaced by a more user-friendly graphical user interface. For post-processing, there had been a proliferation of poorly documented and partly redundant helper or plotting functions. Over time, 13 (thirteen) variants of a 'combine\_X'-function had accumulated, the job of which was to assemble the results of the computations by the toolbox. Moreover, I would have compartmentalized the toolbox code much more. Finally, for a toolbox of this scope, more time should have been devoted to the documentation. All things considered, I believe that resurrecting the code would have been very arduous for anyone else, even a Matlab expert, due to a dearth of documentation.

Yet, I view these deficiencies in context. As a scientist-cum-programmer, one also has the science to do, not to mention actually making sense of the analysis results and writing papers about them. As mentioned before, spending a large amount of time on the documentation would have been hard to justify. Moreover, like many of my former colleagues, I was a self-taught programmer essentially without formal computer science education, so most of my design choices were not based on traded wisdom, but self-accrued, patchy knowledge.

My former self was certainly not alone in this position, as academia in general seems to be a fertile ground for 'organically home-grown' code that may or may not be 'good enough' to publish \cite{barnes_publish_2010}. It is heartening to see that programming scientists are increasingly given a hand in producing reproducible and efficient code \cite{wilson_good_2017}.

\section{Acknowledgements}
I owe profound thanks to Claudia Holt, Section of Experimental Anesthesiology, University Hospital of Tübingen, who unearthed the data for this study. Many thanks also go to Matthew I. Banks for discussions of the manuscript, and to Robert A. Pearce for consenting to the publication of the code.


\section{Analysis of results}

The amount of data collected is quite large, so directly plotting the results would be
unreadable. Instead, we processed the data selecting, for each algorithm and dataset,
the nondominated parameter choices --- that is those, for which there is no other
parameter choice that gives results that are better in both query time and precision.

Algorithms were always called with request for top 100 vectors, but we recorded
precision at a couple of points: 1, 5, 25 and 100. It turns out the precision vs. time
plots are very similar for each of those values --- to save space, we only show
graphs for precision at 100.

\begin{figure}
	\centering
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/nondominated/Amazon-3M-p100.eps}
	}
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/nondominated/sift-p100.eps}
	}
	\\
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/nondominated/WikiLSHTC-p100.eps}
	}
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/nondominated/siftsmall-p100.eps}
	}
\caption{Pareto frontiers for each algorithm.}
\end{figure}

The graphs show some similarity --- in all cases Hierarchical $K$-Means and IVF algorithms were much better
than the alternatives. ALSH algorithm gave very poor results on Amazon-3M and Wiki-LSHTC
datasets --- apparently the SIFT vectors have some regularity that ALSH could exploit better.
Still, the precision did not exceed approximately $0.60$ for reasonable speedups even on
this dataset. Finally, the quantization-based algorithm, although it was able to give
reasonable precision, it was so slow, that the only dataset it did not time
out\footnote{Timeout was set to twice the time of the brute force search.}
was SIFTsmall dataset. It contained only ten thousand vectors though and was included
just for completion.

IVF and Hierarchical $K$-Means curves follow each other quite closely --- it could be expected though,
since Hierarchical $K$-Means algorithm is generalization of IVF. In almost all places Hierarchical $K$-Means are even
better than IVF --- again, this can be explained by having more parameters to be tested.
To check whether this is caused by possibly faster implementation or the introduction of
hierarchy, we split the Hierarchical $K$-Means curve into three lines, each corresponding to a single
number of layers (for example, KMeans-2 means Hierarchical $K$-Means with two layers).

\begin{figure}
	\centering
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/nondominated-km/Amazon-3M-p100.eps}
	}
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/nondominated-km/sift-p100.eps}
	}
	\\
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/nondominated-km/WikiLSHTC-p100.eps}
	}
	\caption{Pareto frontiers for $K$-Means (split) and IVF.}
\end{figure}

From these graphs, we can see that in general IVF is approximately $30\% - 50\%$ faster
(on horizontal axis) than Hierarchical $K$-Means when using just a single layer. Both algorithms are
equivalent in this case, so this means the Faiss library is optimized for the flat clustering\footnote{
Checking the source of Faiss we can see it performs inner products of each query with all centroids
(from the first and only layer) all at once, before continuing to answer each query
using its candidates. This allows one to use heavily optimized matrix-matrix multiplication
routines for the first phase, with additional memory locality advantages.
In principle, this optimization
could also be implemented for Hierarchical $K$-Means (for the first layer only).
}.
In contrast, Hierarchical $K$-Means are much better (achieve same level of precision faster)
when allowed to use two or three layers. The use of hierarchy speeds the calculations to the level
of IVF --- for some parameters even surpassing
it, especially for Amazon-3M dataset.

The difference between two-layer and three-layer Hierarchical $K$-Means is not as amplified. Since
training time (not pictured here) usually rises quite significantly with the number of layers,
the minor additional speedup may not be worth it in practice. Four and more layers
would probably not have noticeable impact either.

It is also notable that two- and three-layer $K$-Means are able to achieve much higher speed up
than IVF, which manifests on graphs as $K$-Means curves starting earlier than $IVF$ curves.
Of course, such speed up comes with significant impact on accuracy, but
IVF cannot even reach this speed. In a sense, its accuracy is zero there.
A possible explanation for this cutoff comes from its complexity analysis as described
in the introduction to the Hierarchical $K$-Means algorithm chapter. No matter what precision is, the minimal
complexity of flat $K$-Means (equivalent to IVF) is $O(\sqrt{N})$. Two- or three-layer
$K$-Means bring this limit down to $O(\sqrt[3]{N})$, or $O(\sqrt[4]{N})$,
respectively.\footnote{Admittedly with a small multiplicative slowdown.}

It is interesting to find out which parameters correspond to particular levels
of precision or speedup. For that, we plotted precision vs. time for different
values of $n_{train}$ parameter (which determines shape of the $K$-Means tree as
described in Chapter 2.1 --- generally, higher values correspond to wider
trees).

\begin{figure}
	\centering
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/kmeans-specific/time-prec-WikiLSHTC-1.eps}
	}
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/kmeans-specific/time-prec-WikiLSHTC-2.eps}
	}
	\\
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/kmeans-specific/time-prec-WikiLSHTC-3.eps}
	}
	\caption{Precision vs. time for various values of the $n_{train}$ parameter.}
\end{figure}

The general trend is that the curves corresponding to lower $n_{train}$ perform
better than other at high-speed, low-precision parts of the chart. In other
words, if high precision is needed, high $n_{train}$ is a good choice (and vice
versa). Note that Auvolat version of this algorithm is equivalent to our
generalization with $n_{train}$ fixed to $1$. For higher precision
scenarios, it usually performs worse than higher $n_{train}$ runs.

This result is consistent with our theoretical calculations in Chapter 2.1, which
were performed for the case of $n_{train} \approx n_{query}$ (and called simply $n$).
In that special case, we would expect low $n$ to cause approximation to be very
quick (as a very small number of subtrees is considered), but rough. We can
check if this assumption holds by plotting $n_{query}$ vs. time and comparing
it with previous graphs.

\begin{figure}
	\centering
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/kmeans-specific/test-time-WikiLSHTC-1.eps}
	}
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/kmeans-specific/test-time-WikiLSHTC-2.eps}
	}
	\\
	\subfloat{
		\includegraphics[width=.48\textwidth]{../graphs/kmeans-specific/test-time-WikiLSHTC-3.eps}
	}
	\caption{Time vs. $n_{query}$ for various values of $n_{train}$ parameter.}
\end{figure}

The charts are quite noisy, since there is some random factor involved. Still,
let us consider the two-layer case. On the previous graphs, we could see
$n_{train} = 5$ was dominating for times in the range approximately from 0.02 to
0.04 (fraction of the full search time). Now, looking at the time vs. $n_{query}$ graph,
we get that range of $n_{query}$ corresponding to that time range is about
3 to 10. Similarly, for $n_{train} = 30$, the time range is about 0.04 to 0.12
(though the upper limit is hard to tell due to noise), with corresponding
$n_{query}$ range of about 10 to 60. In both cases, the $n_{query}$ range we
get is centered near $n_{train}$, meaning that for optimal
$(n_{train}, n_{query})$ choices, both parameters are approximately
equal.\footnote{
Note that the graphs only show a few chosen values of $n_{train}$. Plotting
more values would narrow their non-domination range, strengthening the
stated assumption. We did not do this due to graph readability issues
(lines being hard to tell apart).
}


\section{Miscellaneous Notes}
% the text width in a normal line is 11.80737cm
% To make a TODO note, start a comment with two or more consecutive capital letters.

% Figures are in the gfx folder. They should not have dots in the names
% since that interferes with the crossref function. They should be named
% something like '01-intro.png' where the first two digits are the
% chapter number and then some simple description of the figure.

% Potential arrangement (for 2nd ed???):
%		3 Background chapters
%		8 Chapters about ``Traditional'' procedures
%		4 Chapters about Business Analytics, datamining, etc.

% TODO
% Add whatever graphics I can to each chapter
% Add example research reports

% Use an ``Objective'' box for objectives and ``Summary'' box for summary. Both boxes are found below. Chapters 1 and 11 have an objectives box at the top and a summary box at the end. Use either as a guide for other chapters.

%******************************************************
\section{3-Line Equation}
%******************************************************
\begin{align}
	\label{03:eq:identity_example}
	1101_2 &= 123 \\
	\nonumber
	&= 123 \\
	\nonumber
	&= 123
\end{align}

\section{Solving an Equation Step-by-step}
\begin{align}
	\label{04:soln:solving_equation_one}
	AB+BC(B+C) && \text{Original Expression} \\
	\nonumber
	AB+BBC+BCC && \text{Distribute BC} \\
	\nonumber
	AB+BC+BC && \text{Idempotence: BB=B and CC=C} \\
	\nonumber
	AB+BC && \text{Idempotence: BC+BC=BC} \\
	\nonumber
	B(A+C) && \text{Factor} \\
\end{align}

%**************************************************************************
\section{Margin Paragraph}
%**************************************************************************
% Use to create a sidebar paragraph out in the margin
\marginpar{Whatever.}

%**************************************************************************
\section{Footnote Without Marker}
%**************************************************************************
% Use to create a footnote without a numbering marker
\blfootnote{Whatever.}

%***************************************************************************
\section{Text Box}
%***************************************************************************
% Creates a nice boxed text with a title and main section
\begin{tcolorbox}[colback=blue!5!white,colframe=blue!75!black]
	% Upper half of box: my "title" area
	\textcolor{blue}{\textbf{Interesing Note}}
	% Lower half of the box: the content
	\tcblower
	Whatever.
\end{tcolorbox}

%***************************************************************************
\section{Objectives Box}
% Relies on ``objbox'' in config file
%***************************************************************************
\begin{center}
	\begin{objbox}{Objectives}
		\begin{itemize}
			\setlength{\itemsep}{0pt}
			\setlength{\parskip}{0pt}
			\setlength{\parsep}{0pt}

			\item x1.
			\item x2.
			\item x3.
		\end{itemize}
	\end{objbox}
\end{center}

%***************************************************************************
\section{Summary Box}
% Relies on ``tkawybox'' in config file
%***************************************************************************
\begin{center}
	\begin{tkawybox}{Summary}
		\begin{itemize}
			\setlength{\itemsep}{0pt}
			\setlength{\parskip}{0pt}
			\setlength{\parsep}{0pt}

			\item x1.
			\item x2.
			\item x3.
		\end{itemize}
	\end{tkawybox}
\end{center}

%***************************************************************************
\section{Text Table}
%***************************************************************************
\begin{table}[H]
	\centering
	\begin{tabulary}{\linewidth}{LCCCC}
		\hline
		\multicolumn{5}{l}{\textbf{Do you support these propositions?}} \\
		\hline
		Prop & Strongly Support & Support & Do Not Support & Strongly Do Not Support  \\
		\hline
		100 & $\bigcirc$ & $\bigcirc$ & $\bigcirc$ & $\bigcirc$ \\
		115 & $\bigcirc$ & $\bigcirc$ & $\bigcirc$ & $\bigcirc$ \\
		220 & $\bigcirc$ & $\bigcirc$ & $\bigcirc$ & $\bigcirc$ \\
		\hline
	\end{tabulary}
	\caption{My great table.}
	\label{03:tab01}
\end{table}

%***************************************************************************
\section{Text Table v2}
% The 'p{}' values determine the width of each cell
%***************************************************************************
\begin{table}[H]
	\centering
	\definecolor{ltgray}{gray}{0.95} % this is a light gray
	\rowcolors{1}{}{ltgray} % zebra striping background
	\begin{tabularx}{0.95\linewidth}{
			p{0.22\linewidth}
			p{0.22\linewidth}
			p{0.22\linewidth}
			p{0.22\linewidth}}
		\toprule
		\textbf{Party} & \textbf{Clinton} & \textbf{Trump} & \textbf{Other} \\
		\midrule
		Democrats & $ 89\% $ & $ 8\% $ & $ 3\% $ \\
		Republicans & $ 8\% $ & $ 88\% $ & $ 4\% $ \\
		Independents & $ 42\% $ & $ 46\% $ & $ 12\% $ \\
		\bottomrule
	\end{tabularx}
	\caption{2016 Exit Poll.}
	\label{tab06.02}
\end{table}

%***************************************************************************
\section{ToDo}
%***************************************************************************
% Create a ToDo note in the text. This also creates a new clickable TODO
% section in the ``structure'' box on the left side of the page.
%TODO This is a todo note.

%***************************************************************************
\section{Code Snip}
%***************************************************************************
\lstset{ %
  caption={caption},
  label=SL:lst:listing01,
  numbers=left,
  language=Verilog
}
\begin{lstlisting}
  Code here
\end{lstlisting}

%***************************************************************************
\section{Introductory Figure}
%**************************************************************************
% Use this for figures at the top of each chapter

\begin{wrapfigure}{r}{0.4\textwidth}
	\centering
	\includegraphics[width=0.4\textwidth]{gfx/05-cake}
\end{wrapfigure}

First paragraph.\blfootnote{Photo by lindsay Cotter on Unsplash}

%***************************************************************************
\section{Wrap Figure}
%**************************************************************************
\begin{wrapfigure}{O}{0.2\textwidth}
	\caption{} % No text, wraps badly in very narrow space (does print fig number)
	% to not print a fig number use \caption*{}
	\label{03:fig02}
	\centering
	\includegraphics[width=0.2\textwidth]{gfx/99-placeholder}
\end{wrapfigure}

%***************************************************************************
\section{Sampling of Research}
%**************************************************************************
\begin{figure}[H]
	\centering
	\includegraphics[width=\maxwidth{.95\linewidth}]{gfx/Sampling_Of_Research}
	\caption*{}
	\label{05:sampling_of_research}
\end{figure}
\section{A Sampling of Research}

%***************************************************************************
\section{Regular Figure}
%**************************************************************************
\begin{figure}[H]
	\centering
	\includegraphics[width=\maxwidth{.95\linewidth}]{gfx/99-placeholder}
	\caption{Caption}
	\label{03:fig03}
\end{figure}

%***************************************************************************
\section{Lorem}
%**************************************************************************
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifend tellus. Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. Sed consequat, leo eget bibendum sodales, augue velit cursus nunc,

%***************************************************************************
\section{Style Guide}
%**************************************************************************

For menu selections:
Click \textsc{\fbox{Analyze $ \rightarrow $ Descriptive Statistics $ \rightarrow $ Frequencies}}

Labels should be a chapter, colon, verbal desc (all lc with underscores): \label{03:title}
For figures, label should be chapter, colon, ``fig'' and number: \label{03:fig01}

Dates do not include an apostrophe: In the 1960s this happened...

Wrap all numbers in an in-line math block: $ 123 $

The word ``data'' is plural, thus: ``Data are analyzed...''


\section{Class List}
Here are the classes, structs, unions and interfaces with brief descriptions\-:\begin{DoxyCompactList}
\item\contentsline{section}{{\bf Circle} }{\pageref{class_circle}}{}
\item\contentsline{section}{{\bf Vector} }{\pageref{class_vector}}{}
\end{DoxyCompactList}


% Chapter X

\chapter{Decisions} % Chapter title

\label{Decisions} % For referencing the chapter elsewhere, use

Figaro also contains the ability to solve and query structured decision problems. These types of models, also known as influence diagrams, are generalizations of Bayesian networks that contain additional decision and utility variables. Figaro generalizes ordinary decision models by allowing the information on which a decision is based to be an arbitrary data structure. Also, the full power of the programming language is available to build decision models. However, Figaro does require that the possible values of the decision variables themselves be discretely enumerated.

In this section, we first give a very brief introduction into decision models and decision-making. We then provide a small example of decision-making in Figaro. We also delve deeper into the decision- making implementation in Figaro. Finally, we discuss the different ways that decision-making can be performed on single and multiple decision models.

\section{Decision models}

Decision models are generalizations of Bayesian networks that contain two additional variable types. The first is a decision variable, which represents a set of actions that a decision-maker can perform. The parents of a decision variable represent the information available to the decision-maker at the time of the decision. Decision models also contain utility variables, which represent some gain or loss in the model that directly or indirectly depends upon some previous decisions or random variables.

The purpose of a decision model is usually to compute an optimal policy for each decision in the model, where a policy defines what action a decision-maker should take for every possible value of the decision's parent variable(s). An optimal policy is when every action specified by the policy for each value of the parents is optimal with respect to some measure. To measure the optimality of an action, Figaro uses the maximum expected utility of the action. That is, for each value of a decision's parents, Figaro determines the action that will result in the highest expected utility of the model.

\section{Basic example}

Using Figaro's decision-making capabilities is generally quite simple. For example, consider the code for a simple decision model shown below:

\begin{flushleft}
\marginpar{This example can be found in SingleDecision.scala}
\texttt{import com.cra.figaro.language.\_
\newline import com.cra.figaro.algorithm.decision.\_
\newline import com.cra.figaro.library.compound.\_
\newline import com.cra.figaro.library.decision.\_
\newline
\newline val market = Select(0.5 -> 0, 0.3 -> 1, 0.2 -> 2)
\newline val survey = CPD(market,
\newline \tab 0 -> Select(0.6 -> 0, 0.3 -> 1, 0.1 -> 2),
\newline \tab 1 -> Select(0.3 -> 0, 0.4 -> 1, 0.3 -> 2),
\newline \tab 2 -> Select(0.1 -> 0, 0.4 -> 1, 0.5 -> 2))
\newline
\newline val found = Decision(survey, List(true, false))
\newline
\newline def valueFcn(f: Boolean, m: Int): Double = \{
\newline \tab if (f) \{
\newline \tab m match \{
\newline \tab case 0 => -7.0
\newline \tab case 1 => 5.0
\newline \tab case 2 => 20.0
\newline \}
\newline \} else \{
\newline\tab  0.0
\newline \}
\newline \}
\newline
\newline val value = Apply(found, market, valueFcn)
\newline
\newline val alg = DecisionVariableElimination(List(value), found)
\newline alg.start()
\newline alg.setPolicy(found)
}
\end{flushleft}

The first four lines import the packages needed for decision models. The elements \texttt{market} and \texttt{survey} are random variables in a normal Figaro model. We create a decision variable called \texttt{found} that uses the element \texttt{survey} as a parent, with the possible actions of the decision as \texttt{true} or \texttt{false}. The element named \texttt{value} is a utility variable that computes a \texttt{Double} conditioned upon the action of the decision (\texttt{found}) and the current value of the \texttt{market} element. It uses the function \texttt{valueFcn} to compute current utility. Finally, we use Figaro's decision variable elimination to compute an optimal policy for the \texttt{found} decision, and set the policy in the \texttt{found} element when the algorithm completes so that it can be used for querying.

\section{Decisions in Figaro}

As can be seen in the previous section, decision-making can be implemented in Figaro with little effort. Decisions are created using the \texttt{Decision[T,U]} element. The \texttt{Decision[T,U]} element actually inherits from \texttt{Chain}; that is, a decision is simply an element that uses an \texttt{Element[T]} as a parent, and generates an \texttt{Element[U]} as the action. A new decision is instantiated simply as:

\begin{flushleft}
\texttt{Decision(Flip(0.7), List(0, 1, 2))}
\end{flushleft}

where the first argument is the parent of the decision, and the second argument is a list of the possible actions of the decision. The possible actions must always be finite and discrete. However, the parent of a decision may be an element over any Scala type. So, we could imagine making a decision based on a social network or a DNA sequence. One thing to note is that decision elements only support single parent decisions. However, multiple parent decisions can be easily created by grouping several parent elements into an element tuple. There are various other ways to instantiate a decision that can be found in the code for the \texttt{Decision} class.

Also, the no-forgetting assumption in decision models is not explicitly enforced in Figaro, hence Limited Memory Influence Diagrams (LIMIDs) can be represented in Figaro, though there is not an explicit LIMID reasoning algorithm implemented.

In decision models, there are also variables that represent the utility of the model. In Figaro, there is no need to explicitly create a utility element; this can be easily done using the \texttt{Apply} element, as shown in the example above. Utility elements must be of type \texttt{Element[Double]}.

A decision is similar to a chain, but unlike the chain, a decision element can change its functionality after an optimal policy has been computed for the decision. Most of the time, setting the policy of a decision can be done simply through the algorithm that computes the optimal policy. However, a user may manually set the policy of a decision element by calling the \texttt{setPolicy} function of the decision, defined as:

\begin{flushleft}
\texttt{def setPolicy(new\_fcn: (T => Element[U])): Unit}
\end{flushleft}

That is, setting the policy of a decision is just providing a new function from the value of a parent to an \texttt{Element[U]}. Users can also get the policy for a specific value of the parent by calling \texttt{getPolicy(p: T): Element[U]}. Various other ways to set the policy can also be found in the Decision code.

\section{Single decision models and policy generation}

Single decision models can be created in Figaro by simply inserting a \texttt{Decision} element into the model. Once the model has been created, the goal is usually to compute the optimal policy for the decision that maximizes the expected utility of the model. This is done as two explicit steps in Figaro;computing the expected utility of each parent and decision pair, then determining the decision that has the maximum expected utility for each parent value. The policy is then set as a function that returns the maximum expected utility decision as a \texttt{Constant}  for any parent value. This policy computation is performed using one of Figaro's built-in inference algorithms. Two alternative methods are provided. One is generally used when the support of the parent is finite, the other when it is infinite. However, there are some cases where the support is finite but very large and the infinite support method is preferable. Alternatively, for some distributions with infinite support, like Poisson or Geometric, only a small number of values are likely, and the finite support method can be used.

\subsection{Finite parent support}

In this case, computing the optimal policy can be performed using the variable elimination, importance sampling, or Metropolis-Hastings algorithms. In addition to the normal parameters that each algorithm takes (as explained in previous sections), the decision version of these algorithms also takes a \texttt{List[Element[Double]]} that indicates the utility nodes in the model. The target of the algorithm is always the decision you wish to compute an optimal policy for. To find the optimal policy for discrete decisions, you simply instantiate one of the algorithms, for example:

\begin{flushleft}
\texttt{val alg = DecisionVariableElimination(List(value), found)
\newline val alg = DecisionImportanceSampling(10000,	List(value), found)
\newline val alg = DecisionMetropolisHastings(10000, ProposalScheme.default,
1000, List(value), found)
}
\end{flushleft}

Where \texttt{List(value)} is the list of utilities in the model, and \texttt{found} is the decision. To compute the optimal policy, you simple start the algorithm, i.e., \texttt{alg.start()}. One the algorithm has completed running, you can call \texttt{alg.setPolicy(found)}, which will set the optimal policy in the \texttt{Decision} element that was computed from the algorithm.

\subsection{Infinite parent support}

When the parent(s) of a decision have infinite support, it is more difficult to compute an exact optimal policy. This is because it is not possible to compute the maximum expected utility action for each value of the parent since the range of the parent is infinite. In such a case, we use Figaro's sampling algorithms to compute an \emph{approximate} optimal policy that attempts to provide a maximal expected utility decision for any possible value of the parent. Since we use sampling algorithms to compute the approximation, only importance sampling and Metropolis-Hastings can be used with continuous decisions.

Instantiating a decision with infinite parent support is similar to finite parent support, except that one must explicitly instantiate a \texttt{NonCachingDecision}, which is based on \texttt{NonCachingChain}:

\begin{flushleft}
\texttt{NonCachingDecision(Normal(0.0, 1.0), List(0, 1, 2))}
\end{flushleft}

The creation of the algorithm and setting of the policy is the same as discrete decisions. Internally in Figaro, however, there are major differences between the implementation of policies for discrete and continuous decisions.

When a sampling algorithm is run on a continuous decision, the algorithm records the utility of the model for each parent and action value that is randomly sampled. When \texttt{setPolicy} is called on the algorithm, all of the generated samples are stored in the decision element. That is, no optimal policy is generated when \texttt{setPolicy} is called; the optimal action to take for a parent value is only computed when the model is queried for a decision with a particular parent value.

When the decision is queried, i.e., \texttt{getPolicy(p: T)} or \texttt{generate()} is invoked on the decision element, the optimal action for parent value p is computed using a nearest-neighbor method. The N closest samples to the parent value are retrieved from the stored samples, the expected utility is computed for each possible action, and the maximum is chosen as the optimal action for this parent value.

Since nearest-neighbor is used to find nearby parent values, a distance metric must also be defined for the parent type \texttt{T}. To use an \texttt{Element[T]} as the parent to a decision, the type \texttt{T} must implement the \texttt{Distance[T]} trait, defined as:

\begin{flushleft}
\texttt{trait Distance[T] \{
\newline \tab def distance(that : T) : Double
\newline \} }
\end{flushleft}

This trait just defines a function that computes a Double distance between two values of the type. For built-in types (Double, Int and Boolean), we use Scala's implicit conversion mechanism to automatically handle conversion to a class that implements the \texttt{Distance[T]} interface so that no changes are needed by the user. For examples, the Double implementation is:

\begin{flushleft}
\texttt{case class DoubleDistance(value : Double) extends Distance[Double] \{
\newline \tab def distance(that : Double) = math.abs(value-that)
\newline \}
\newline implicit def double2Dist(x : Double) = DoubleDistance(x)
}
\end{flushleft}

See the \texttt{Distance} class for more details on default conversions of basic types and parents that are element tuples. For user defined classes, all the user needs to do is implement a distance function in the \texttt{Distance} trait, and the type can be used as a parent to a decision element. For instance, we can use an element over the range of graphs as a parent to a decision by declaring the \texttt{dGraph} class as such:

\begin{flushleft}
\texttt{class dGraph() extends Distance[dGraph] \{
\newline \tab ...
\newline \tab def distance(that: dGraph): Double = \{
\newline \tab \tab ...
\newline \}
\newline \} }
\end{flushleft}

Since the number of samples generated from the algorithm may be large, and the optimal policy method retrieves the nearest neighbors for \emph{every} parent value that is queried from the decision, computing the optimal action can be quite slow. To ameliorate this slowdown, Figaro stores the samples in an index. The default implementation is a VP-index, used for metric distances. Different indices can be created an integrated as well. See the \texttt{Index} and \texttt{DecisionPolicy} classes for more information.

\section{Multiple decision models and policy generation}

Figaro also supports for multiple decision models using a backward induction algorithm.
In this algorithm, the optimal policies are computed in reverse order on a set of partially ordered decision variables. To create policies for multiple decision models, a user uses the multi-decision versions of the algorithms:

\begin{flushleft}
\marginpar{A multiple decision example can be found in MultiDecision.scala}
\texttt{val alg = MultiDecisionVariableElimination(List(utility1, utility2), decision1, decision2)
\newline val alg = MultiDecisionImportanceSampling(10000,	List(utility1, utility2), decision1, decision2)
\newline val alg = MultiDecisionMetropolisHastings(10000, maker: ProposalMakerType, 1000, List(utility1, utility2), decision1, decision2)
}
\end{flushleft}

Note that the interface for the \texttt{MultiDecisionMetropolisHastings} is different than the \texttt{DecisionMetropolisHastings} algorithm. \texttt{MultiDec\-isionMetropolisHastings} needs a \texttt{ProposalMakerType}, since inside the algorithm, \texttt{Decision\-MetropolisHastings} is run for each decision. The \texttt{ProposalMakerType} is defined as:

\begin{flushleft}
\texttt{type ProposalMakerType = (Universe, Element[\_]) => ProposalScheme}
\end{flushleft}

Only the one-time versions of the decision algorithms can be used for multi-decision models. To compute the optimal policy for every decision in the model, the user simply does, for example,:

\begin{flushleft}
\texttt{val propmaker = (mv: Universe, e: Element[\_]) => ProposalScheme.default(mv)
\newline val alg = MultiDecisionMetropolisHastings(200000, propmaker, 20000, List(value, cost), test, found)
\newline alg.start()}
\end{flushleft}

The ProposalMaker for this small example just uses the default proposal for each instantiation of \texttt{DecisionMetropolisHastings} for a decision. However, we could also change the proposal scheme for each decision. There is also no need to call \texttt{alg.setPolicy}, since the multi-decision algorithm will set the optimal policy for each decision as it is needed for backward induction. Figaro will automatically compute the partial order of the decisions that are in the parameter list.


\chapter{Conclusion and Future Work}
\label{ch:Conclusion-and-Future-Work}
In this study, we have shown that the application of a clustering step that utilizes dimensionality reduction techniques such as \gls{t-sne} and hierarchical, density-based clustering in the form of \gls{hdbscan} leads to significant improvements in forecasting accuracy when taking individual households into consideration. While this technique is certainly more complex, in particular with regards to the number of steps and moving parts associated with the entire pipeline, we maintain that the benefits in terms of improved forecasting accuracy outweigh the overall increase with regards to the time and effort it would take to train and set up such a model. The practicality of the model lies in the availability of the data that it requires to function -- primarily with respect to historical energy consumption data for the individual households in question (which is becoming easier and easier to obtain thanks to the prevalence of smart meters) and meteorological data that can easily be obtained from numerous sources. Furthermore, it is highly likely that, given enough historical data, the need to further train the model(s) after the initial setup is rather low further compounding the efficacy of our method.

\noindent \newline Furthermore, one of the benefits of our method that we previously discussed is that no prior knowledge of the number of clusters is required. As there is no guarantee that any 2 individual households contain a similar number of \textit{repeating} patterns we avoid running into the problem of overly generalizing a single working solution that may or may not work given said change in energy consumption patterns and instead present a solution that could potentially extend to a much larger scale. A potential issue with this implementation however, is that an individual household \textit{may} contain a large number of repeating consumption patterns which could possibly lead to an overall decline in what can already be considered sub-par performance from our classifier. That said, there is definitely room for improvement that could accommodate these potential risks, specifically with regards to the feature engineering step -- for example, improvements in classifier accuracy could be seen through the utilization of a more efficient classifier. Alternatively, the current lack of contextual information that serves to explain the emergence of the clusters as part of the clustering step could likely be the reason for obtaining sub-par accuracy scores as, in its current iteration, the premise of our clustering step was to group together days that exhibited the highest similarity purely in terms of their energy consumption patterns and, given that this information is not readily available to us when considering a new day, we are left reaching for straws when attempting to explain when any individual household is likely to observe energy consumption patterns that fall within any of the obtained clusters. Evidently, temporal and meteorological information is not enough to explain the emergence of said clusters and other information (perhaps patterns in terms of cluster labels leading up to the new sample) could serve to improve classifier accuracy. This is definitely an area of this study that could be looked into as part of future research. Additionally, regardless of the fact that the performance of our forecasting model is the highlight of this paper, it is interesting to note that a byproduct of our method is the potential to extract insights into variables that have an effect on the daily energy consumption patterns of unique households. A cursory glance at applying our method to a portion of the data at hand, as an example of the insights that we can obtain, shows us that some households have frequently occurring patterns that tend to deviate among the different days of the week while other households have an even bigger separation across months of the year or even among meteorological factors such as the temperature or chance of rain.

\noindent \newline To conclude, we note that, as a result of pre-clustering our data, and then training separate models on a per-cluster basis we achieved an improvement in overall forecasting accuracy with superior \gls{mape} scores in contrast to the current state-of-the-art (\gls{lstm} networks, clustering based on K-means, etc.).


\subsection{Continuous utility functions}

With non-discrete choice a consumer chooses how much of product \(x\) to consume.

The utility function is of the form:

\(U_i(x_1,...,x_m;d)\)

And the consumer chooses how much of \(x_i\) to consume to maximise this, subject to the budget constraint.

We include features relating to the individual, \(d\).

\subsection{Discrete choice}

The utility customer \(i\) gets from product \(j\) is:

\(U_{ij}=f_i(p,d)+\epsilon_{ij}\)

The customer chooses the product with the highest utility.

\subsection{The outside option}

We need to know the market share of the outside option. Do this theoretically. Eg number of customers in area, and 1 per day.


\chapter{Introduction}\label{chap:introduction}

A neural network, also known as an Artificial Neural Network (ANN), is an algorithm that mimics the way a human brain operates. It took millions of years of evolution for the human brain to achieve the level of intelligence we observe today. This intelligence helps us quickly interpret and evaluate what we see in our surroundings. For example, consider the following image of hand-written digits from the MNIST dataset \cite{mnist}:

\begin{figure}[h]
	\centering
	\includegraphics[width=0.5\linewidth]{images/introduction/digits.png}
	\caption[Hand-written digits]%
	{\textbf{Hand-written digits}: Each hand-written digit is a $28\times28$ pixel grayscale image. The entire MNIST dataset contains a total of $70000$ such images that split into a training set of $60000$ images and a test set of $10000$ images.}
	\label{fig:digits}
\end{figure}

One can quickly recognize these digits as $04192$, thanks to the network of billions of neurons available in our brain that makes it easy to identify visual patterns. Past experiences and memories stored in our brain make this process so simple that it happens subconsciously most of the time. This simple process becomes much more difficult when we try to write computer programs to identify similar patterns due to the lack of precise rules and hundreds of exceptions and varieties of a single pattern.

Neural networks tackle this problem by inferring rules from a large set of related training data. For example, to train a neural network to efficiently recognize hand-written digits shown in figure \ref{fig:digits}, a dataset large enough to include different styles of hand-written digits is needed.

Neural networks and their variations such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) are proven to be useful in many different application areas such as image recognition \cite{image_recon}, machine translation \cite{russian}, recommender systems \cite{recommender}. Despite this state-of-the-art performance, neural networks are computationally complex and memory-intensive due to their deep structures.

Sparse Neural Networks are a viable option to make neural networks less complex and make them memory efficient. Such sparsity in neural networks can be induced by pruning weights, utilizing skip connections, or generating random architectures based on graphs. Sparsity in a neural network can reduce that network's size by many times with little to no drop in performance \cite{sparse_nn, dey, liu}. Therefore, in this thesis, we implement two different ways to achieve sparsity in Recurrent Neural Networks and analyze its performance impact.

% -----------------------------------------------------------------------------------------------------------
% ------------------------------------------------ MOTIVATION -----------------------------------------------
% -----------------------------------------------------------------------------------------------------------

\section{Motivation}\label{section:motivation}

As stated before, deep neural networks are likely to have an increased performance but at the cost of higher complexity and increased fast memory requirements. One way to mitigate this issue while maintaining the performance is to introduce sparsity into a network's connections \cite{deep_res}. For example, Mao et al. in \cite{mao} report a higher-compression ratio with coarse-grained pruning without loss of accuracy while saving about twice the memory references. Similarly, Sun et al. in \cite{sparse_face} reported that the sparse ConvNet model, which only has 12\% of the original parameters, still performs similarly to the baseline model.

These are just a few examples where Sparse Neural Networks have proven advantageous in time, energy, and memory savings. Sparsity in traditional neural networks and CNNs is studied widely by many researchers but is not explored much in Recurrent Neural Networks that are difficult to train due to their nonlinear iterative nature. Sparse structures in traditional neural networks have shown a promise of training potential \cite{sparse_nn} which, if applied to Recurrent Neural Networks, can also make training RNNs less difficult by reducing their network size while retaining their performance.

% -----------------------------------------------------------------------------------------------------------
% -------------------------------------------- RESEARCH QUESTIONS -------------------------------------------
% -----------------------------------------------------------------------------------------------------------

\newpage
\section{Research questions}\label{section:research_questions}

The primary research goal of this master thesis is to investigate the effects of sparse structure on the performance of the Recurrent Neural Networks and its variants, namely Long Short-Term Memory and Gated Recurrent Unit. To do so, we have defined the following set of correlated research questions:

\begin{enumerate}
	\item\label{rq:q1} What is the effect of weights pruning on a recurrent network's accuracy?

	\item\label{rq:q2} What percentage of weights pruning is permissible without triggering a significant reduction in the performance?

	\item\label{rq:q3} After pruning a certain percent of weights, if we see a significant reduction in the accuracy,  how many re-training epochs can regain accuracy?

	\item\label{rq:q4} How does a randomly structured recurrent network's performance correlate with the graph properties of its internal structure?

	\item\label{rq:q5} Is it possible to predict a randomly structured recurrent network's performance using the graph properties of its base random graph?
\end{enumerate}

We will answer all these questions for four different recurrent networks, namely RNN with Tanh nonlinearity (RNN-Tanh), RNN with ReLU nonlinearity (RNN-ReLU), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU), by following the approach explained in section \ref{section:proposed_approach}.

% -----------------------------------------------------------------------------------------------------------
% -------------------------------------------- PROPOSED APPROACH --------------------------------------------
% -----------------------------------------------------------------------------------------------------------

\newpage
\section{Proposed approach}\label{section:proposed_approach}

To investigate the effects of sparse structures in recurrent networks, we propose the following three experiments:
\begin{enumerate}
	\item \textbf{Investigating the effects of weights pruning on recurrent networks}: \\
	    In this experiment, we prune input-to-hidden and hidden-to-hidden weights, both simultaneously and individually.

	    We start by training a recurrent network on a dataset for a certain number of epochs. Once this training is complete, we follow the process shown in the following flowchart:

	    \begin{figure}[h]
            \centering
            \includegraphics[width=0.5\linewidth]{images/introduction/flow_1.png}
            \caption[Flowchart for the pruning experiment]{The basic flowchart depicting the process of pruning and re-training a trained recurrent network.}
            \label{fig:flowchart_pruning}
        \end{figure}

    	 To outline this flowchart, once the initial training is complete, retrieve the learned weights of the trained model, prune the lower $p\%$ of weights (where $p \in \mathbb{Z}:p \in [1, 100]$), and re-train this pruned model to check for the number of epochs required to regain accuracy.

    	 This experiment will help answer research questions \ref{rq:q1}, \ref{rq:q2}, and \ref{rq:q3}.

	\item \textbf{Analyzing the performance of recurrent networks with randomly structured recurrent networks}: \\
	    We start by generating randomly structured neural networks by following the technique described by Stier et al. in \cite{julian} that generate Sparse Neural Networks.

        After generating Sparse Neural Networks, we introduce recurrent connections to generate Sparse RNNs. These recurrent connections are necessary to make each subsequent run dependent on the previous run.

        Once we have Sparse RNNs, we train them on a dataset and compute the correlation between its accuracy and its internal structure's graph properties.

        This experiment will help answer the \ref{rq:q4}th research question.

    \item \textbf{Performance prediction of a randomly structured neural network:}: \\
        Once we have trained Sparse RNNs, we train three different regressor algorithms with graph properties as features and corresponding accuracy as the target. Since this is a regression problem, we finally report an R-squared value for each RNN variant and each regressor algorithm.

        This experiment will help answer the \ref{rq:q5}th research question.
\end{enumerate}

All these three approaches are described in detail in the later chapters.

% -----------------------------------------------------------------------------------------------------------
% ------------------------------------------------ STRUCTURE ------------------------------------------------
% -----------------------------------------------------------------------------------------------------------


\chapter{Hartree-Fock}

\begin{Exercise}
    Answer each of the following in one sentence, using words only.
    \Question{Define canonical Hartree-Fock orbitals.}\label{canon_hf}
    \Question{Explain why the choice of Hartree-Fock orbitals is not unique.}\label{unique_hf}
\end{Exercise}

\begin{Answer}
    \ref{canon_hf}. Hartree-Fock orbitals are ``canonical'' when the Lagrange multiplier matrix is diagonal.
    \ref{unique_hf}. The Hartree-Fock energy and the orbital overlaps are invariant to a unitary transformation, so any unitary variation of the canonical orbitals satisfies the Hartree-Fock optimization conditions.
\end{Answer}

\begin{Exercise}
    Briefly explain how the Lagrangian approach to constrained optimization works. Draw pictures where necessary.
\end{Exercise}

\begin{Exercise}
  Determine the functional derivatives of the Hartree-Fock Lagrangian,
  $\dfr{\d\mc{L}}{\d\y_k^*}$
  and
  $\dfr{\d\mc{L}}{\d\y_k}$.
\end{Exercise}

\begin{Exercise}
    Derive the following expression for the energy expectation value of a Slater determinant, known as the \textit{first Slater rule}.
    \begin{align*}
      \ip{\F|\op{H}_e|\F}
    =
    \sum_{i}^n
      \ip{\y_i|\op{h}|\y_i}
    +\fr{1}{2}\sum_{ij}^n
      \ip{\y_i\y_j||\y_i\y_j}
    \end{align*}
\end{Exercise}