\chapter{Long-term surface water change detection}
\label{ch5}

\begin{abstract}
	Methods to detect long-term surface water changes are discussed. A simple method based on reflectance percentile composites and linear regression is presented. As an example, Siling Lake in the Tibetan Plateau is used as a study area. The advantages and drawbacks of the method are discussed.

	\begin{center}
		\begin{tikzpicture}[every node/.style={inner sep=0,outer sep=0}]
		\node[draw=none,shade,blur shadow={shadow blur steps=5}] {
			\includegraphics[width=3in]{01.4-change-detection/figures/change-detection-dubai}
		};
		\end{tikzpicture}
	\end{center}

	\textbf{Keywords:} $M_3$, reflectance percentiles, NDWI, linear regression, long-term surface water changes.

\end{abstract}

{
	\setlength{\parindent}{0pt}This chapter is based on \bibentry{donchyts2016earth}.
}

%% Start the actual chapter on a new page.
\newpage

\section{Introduction}
\dropcap{A}{s} we have seen in the previous three chapters, accurate surface water detection is a complex task and may require many steps. However, different land use changes may occur at different temporal scales. If we are interested only in long-term (inter-annual) land-use changes, more generic statistical methods can be applied to detect these changes. With sufficient observations available and under the assumption that statistics of reflectance values for different years remain the same, we can use relatively simple methods, when compared to those discussed in the Chapter \ref{ch4}.

The first step that should be done when analyzing optical passive sensor satellite imagery is the elimination of clouds and cloud shadow effects. Cleaning images from cloud, cloud shadows, and snow can be a challenging and time-consuming process, especially when images partially covered by clouds need to be considered. Many methods were developed to do this properly, as has been discussed in chapters \ref{ch2}, \ref{ch3}, and \ref{ch4}. These methods can be used to construct completely cloud-free images, or to extract information required to analyze temporal changes in different land cover classes.

As an alternative to this, we can generate inter-annual composites directly from the top-of-atmosphere or surface reflectance values. In recent years, this approach was used in \citet{Hansen2013} to study forest cover changes by employing a combination of metrics, such as:
\begin{enumerate*}[label=(\emph{\alph*})]
	\item selected percentile values
	\item mean reflectance values for selected percentile ranges
	\item slope of linear regression of band reflectance values versus image date
\end{enumerate*}. Furthermore, they make use of supervised classification methods based on decision trees to relate these metrics to homogeneously varying subsets of data.

Here, a similar approach to estimate long-term surface water changes is introduced. However, instead of developing multiple metrics like in \citet{Hansen2013}, a single combined metric employing percentiles is used to estimate average cloud-free reflectance values and the slope of linear regression to identify pixels where long-term changes have been occurring.

The resulting method is also more resource efficient because applying linear regression after averaging is cheaper to compute. The cost of applying linear regression directly to reflectance values is $O(n^3)$, and the cost to calculate the average value for every pixel is $O(n)$, where n is the number of analyzed images. The $O$ notation is frequently used in computer science to indicate scalability of the algorithm with the growth of the dataset. By employing a two-step approach, surface water changes can be estimated in almost $O(n)$ time, making it attractive to be applied across multiple sensors, assuming similarity in reflectance values among different satellite sensors for some spectral bands. For surface water change studies, the most used spectral index is \gls{NDWI}. The index uses near-infrared and green bands. These bands have similar spatial and spectral resolution across multiple Landsat missions and seem to be in agreement with regards to spectral response and spectral resolution, except for Landsat 8. For Landsat 8, the OLI sensor covers the thinner spectral range for the near-infrared band when compared to TM and ETM+ sensors used by Landsat 4, 5, and 7 \citet{roy2016characterization, angal2014cross}. A detailed comparison of spectral responses between Landsat 5 and 7 can be found in \citet{teillet2001radiometric} and between Landsat 7 and 8 in \citet{flood2014continuity}. Additionally, the literature says that data produced from Landsat was cross-calibrated to ensure consistency and continuity of values between different missions \citet{mishra2014radiometric}. This makes combining images across different Landsat sensors, and the use of statistical methods to process their values, very promising.

\begin{figure}[H]
	\includegraphics[width=1.0\textwidth,left]{01.4-change-detection/figures/aral}
	\caption{Aral Sea, Uzbekistan. False-color reflectance percentile composite (swir1, nir, green, 20\%). The red circle indicates the location where reflectance and \gls{NDWI} values shown on charts were plotted. The composite images and the time series were computed from 4325 Landsat scenes between 1984 and 2017. \opt{source}{Source: \url{https://code.earthengine.google.com/c1ccc09c6b5e7d73bd3a3601f8edfaf3}}}
	\label{fig:ch4-aral-sea}
\end{figure}

Surface water is very dynamic and may change on a daily, monthly, and annual basis. To estimate long-term, inter-annual surface water changes from raw satellite images, we have to exclude effects caused by both clouds and by intra-annual or seasonal variations, assuming these changes remain the same across years. Examples of intra-annual changes can be seasonal changes in surface water due to natural processes, a regular water cycle, changes of surface water levels caused by tidal effects in coastal areas, and variability in surface water levels due to man-made activities, such as the operation of reservoirs, which may also cause surface water changes in the downstream rivers. Long-term surface water changes may be due to natural changes of water boundaries, such as erosion or accretion, natural changes caused by climate changes, or manmade changes in rivers or coastal zones (embankments, land reclamation, construction of reservoirs).

In reality, a combination of these surface water changes may occur and interfere with each other, making the task of surface water change detection extremely difficult.

To demonstrate the variability of reflectance values in different bands for an area where both long and short-term changes took place, a long-term time series in the middle of the Aral Sea is obtained for pixels which have dried up during the past few decades, as shown in Figure \ref{fig:ch4-aral-sea}. This area has faced dramatic changes regarding surface water and, as can be seen from the reflectance values, this has influenced both reflectance and corresponding \gls{NDWI} value changes. While it would be possible to detect the trend of \gls{NDWI} values from the raw reflectance value time series, doing this may be less efficient, mainly because \gls{NDWI} values for many pixels corresponding to clouds may look very similar to the values corresponding to water. Additionally, it would require significantly more computing resources.

\section{Long-term surface water change detection}

To address long-term surface water changes, we can first try to exclude short-term variability of reflectance values by computing average images for a long time intervals, excluding effects of clouds and cloud shadows in this way. By estimating this type of composite images for multiple time intervals, followed by the application of spectral water indices, such as \gls{NDWI}, we should be able to reconstruct long-term variability of the surface water changes, which will be reflected in the variability of the \gls{NDWI} values. One of the important questions arising using this method is how to select the best percentile, allowing to minimize the effect of clouds and cloud shadows, and at the same time, avoiding the appearance of cloud shadows in the images.

The method will work only if a sufficient number of observations is available. A large number of images is required to ensure that the distribution of values, used to compute reflectance percentile composites, converges to the actual distribution representing different land or atmosphere values for a given location  is based on similar distributions. Ideally, the resulting percentile should look the same for two areas when no long-term surface water changes take place.

Because surface water has a very distinctive spectral signature, with very low reflectance values in almost all bands, the presence or absence of water in a given pixel will be reflected in the final distribution represented as a probability distribution function (PDF) or a cumulative distribution function (CDF) for a given location.

\afterpage{%

	\begin{figure}
		\includegraphics[width=1.0\textwidth,left]{01.4-change-detection/figures/Tibet-lake-changes}
		\caption{Siling Lake, Tibetan Plateau, China. Two reflectance composite images at the top-left show how the lake looked like in 80's (left) and how it looks like today (right). The blue color indicates submerged land due to climate changes. \opt{source}{Source: \url{https://code.earthengine.google.com/87d0057d8b3d28817b1da7cfc1bd6b6e}}}
		\label{fig:ch4-tibet-lake}
	\end{figure}

	\begin{figure}
		\includegraphics[width=1.0\textwidth,left]{01.4-change-detection/figures/Tibet-time-series}
		\caption{Estimating surface Siling Lake, Tibetan Plateau, China. The actual TOA reflectance values (left-top), annually-averaged (20\%) reflectance values (left-middle) and \gls{NDWI} values, estimated from annually-averaged reflectance values (left-bottom). The three charts on the left indicate values and distributions of the sun parameters for the images used to compute these values.}
		\label{fig:ch4-tibet-lake-time-series}
	\end{figure}

	\clearpage
}

To demonstrate the main steps used in the methods, surface water changes which have occurred around the Siling Lake are analyzed - one of the lakes of Tibetan Plateau, China. In the last few decades, an enormous area of new surface water has been created due to climate changes. Figure \ref{fig:ch4-tibet-lake} shows the final estimate for the submerged land, equal to $22.5 km^2$.

A straightforward and efficient way to estimate long-term surface water changes may be to use a two-step approach. First, the intensity percentile composites are computed for all temporal intervals $T$ used for the change detection analysis:

\begin{equation}
F(\rho) = \int_{-\infty}^{\rho^i} P(\rho)d\rho
\end{equation}

where $F$ - cumulative distribution function, the superscript index $i$ denotes the $i^{th}$ percentile, $\rho$ - reflectance values for a given band.

Then, for every percentile reflectance value $\rho^i$, the spectral index is computed as an image:

\begin{equation}
I = \frac{\rho_{green}^i-\rho_{nir}^i}{\rho_{green}^i+\rho_{nir}^i} \\
\end{equation}

After that, the linear regression is performed on a set of all index values for all intervals $T$:

\begin{equation}
I = \beta_0 + \beta_1 T + \epsilon
\end{equation}

The resulting slope of the linear regression $\beta_1$ is then analyzed to identify pixels, where significant surface water changes took place.

Usually, the percentile is determined empirically and corresponds to low reflectance values, but not too low, to avoid confusing water with cloud shadows, which can also be very dark for green and nir (or swir) bands used to estimate spectral indices. For those pixels, where the slope $\beta_1$ of the linear regression is significant. The spectral index values $I$ are also tested to exclude false positive changes, detected for images where most images correspond to water or land:

\begin{equation}
\begin{split}
\begin{aligned}
I &= min\left(I, I_{min}^{water}\right) \\
I &= max\left(I, I_{max}^{land}\right)
\end{aligned}
\end{split}
\label{eq:water-land-suppression}
\end{equation}

where $I_{min}^{water}$ and $I_{max}^{land}$ indicate minimum and maximum values of the spectral index to be considered as water and land correspondingly. The equation \ref{eq:water-land-suppression} is applied for every interval used during temporal averaging.

The first equation of the \ref{eq:water-land-suppression} allows to filter pixels, where the slope of the regression is significant, with all values still belonging to water. The second equation of the \ref{eq:water-land-suppression} does the same to eliminate locations where all of the temporal intensity values belong to the land.

Additionally, \gls{NDVI} vegetation index is used similarly to remove locations, where, for example, deforestation took place. In this case, \gls{NDWI} values may also change significantly in time, but the actual changes do not correspond to surface water changes.

\afterpage{%

	\begin{figure}
		\includegraphics[width=1.0\textwidth,left]{01.4-change-detection/figures/images_real}
		\caption{Examples of false-color (swir1, nir, green) images from 1998 acquired by Landsat TM sensor for Siling Lake, Tibetan Plateau, China. \opt{source}{Source: \url{https://code.earthengine.google.com/882ab2929f536676d56bc79cbc895ded}}
		}
		\label{fig:ch4-tibet-lake-false}
	\end{figure}

	\begin{figure}
		\includegraphics[width=1.0\textwidth,left]{01.4-change-detection/figures/images_percentiles}
		\caption{False-color (swir1, nir, green) reflectance percentile images estimated from 751 images acquired by multiple NASA Landsat missions during 1984-2017, Siling Lake, Tibetan Plateau, China}
		\label{fig:ch4-tibet-lake-percentiles}
	\end{figure}
	\clearpage
}

The actual number of images available need to be adjusted based on the cloud frequency present for a given area. For the area around this lake, the average cloud frequency is 13\%, with relatively low intra-annual variation, as estimated by \citet{wilson2016remotely} from MODIS images. However, for some areas on Earth, mainly near the equator, 95\% of the images may be covered by clouds. This means, that to apply this method, many more images need to be analyzed. In this case, only long-term surface water changes may be estimated using passive optical sensors.

% t-test using regression \url{https://en.wikipedia.org/wiki/Student%27s_t-test#Slope_of_a_regression_line}
% non-parametric tests \url{ftp://cran.r-project.org/pub/R/web/packages/trend/vignettes/trend.pdf}

% CDF sun parameters mismatch, topographic errors \url{https://code.earthengine.google.com/c9ed4d65ee90db7d4944e4c257969e1b}
% \url{https://code.earthengine.google.com/5ddaeea0946642afe23d7557c3c0c657}

\section{Topographic noise for inconsistent image collections}
Even though the method described in Chapter \ref{ch5} allows detection of surface water changes at high accuracy, a substantial commission error may occur in mountain regions. These errors are caused mainly by combined errors of elevation and azimuth, causing differences in hill shadows in composite images, used to detect surface water changes. For global analysis, most of these errors were eliminated by masking-out the final surface water changes images with the topographic index derived from \gls{HAND}. The topographic mask, constructed by HAND < 40m, was used to filter out these errors. The \gls{HAND} dataset used for the processing was generated from 30m \gls{SRTM} and some auxiliary 90m datasets to cover areas where it was not acquired (>60 degree north latitude) \citet{donchyts2016hand}. However, for very accurate surface water change estimates this may also result in omission errors in the areas when DEM values are incorrect. For the time being, no free high-resolution DEM exists to be used as a topographic index (via \gls{HAND}). Therefore, the Aqua Monitor website does not perform this correction during on-the-fly estimation of surface water changes. This may change in future versions, when a more appropriate method is introduced.

\begin{figure}
	\centering
	\includegraphics[width=1\textwidth]{01.8-aqua-monitor/figures/problem1}
	\caption{False-positive composites (swir1, nir, green) and surface water changes commission errors due to inconsistent sun parameters in the samples used to compute changes in the hilly areas (2000-2015). Top: upstream of the Lake Mead (\href{http://aqua-monitor.appspot.com/?from=2000&to=2013&view=36.103119715292486,-113.93795967102045,13z&max_doy=365}{36.10, -113.90}); bottom: river near Yogongxiang, China (\href{http://aqua-monitor.appspot.com/?mode=dynamic&from=2000&to=2013&view=30.269128374988604,94.89677429199217,12z&max_doy=365&averaging_months1=36&averaging_months2=36}{30.27, 94.90}).}
	\label{fig:am-topographic-errors}
\end{figure}

Examples of these errors are demonstrated in Figure \ref{fig:am-topographic-errors}.

The same errors were reported in the \citet{Hansen2013} dataset, which uses similar methods to estimate surface water changes (interval mean percentile composites versus percentile composites in Aqua Monitor).

A time consuming approach to eliminate these errors could be to perform topographic correction on every image before passing them to the change detection algorithm. However, this may significantly increase the amount of resources needed, making this type of analysis hardly achievable today.

\section{Conclusions and Discussion}

The method presented in this chapter showed surprisingly good performance for most of the places on Earth and was used to analyze global surface water changes over the last thirty years. Despite its simplicity, the method can very accurately detect long-term surface changes. Additionally, it is easy to implement using the Google Earth Engine parallel processing platform.

Another advantage of the method is that it is less resource intensive when compared to a more streamlined approach to detecting surface water changes. Hence it can be used for large planetary-scale studies and for a wide range of users, which is demonstrated in Chapter \ref{ch6}. For example, all images are first classified e.g. using supervised machine learning methods, and then the changes are computed from the resulting thematic maps.

However, it may be difficult to reliably detect surface water changes when:

\begin{enumerate}[label=(\alph*)]
	\item \label{enum:changes-low-count} the number of images used to compute percentiles is low
	\item \label{enum:changes-complex-trend} surface water changes follow a more complex pattern than a one directional trend
	\item \label{enum:changes-multple-classes} multiple land-use changes are present, represented by low reflectance values but irrelevant to surface water changes
	\item \label{enum:changes-topo} complex topographic conditions combined with unequally distributed sun parameters in different time intervals
\end{enumerate}

Some of these issues can be easily addressed, for example, by changing empirically chosen percentiles to ensure one captures pixels free from cloud and snow effects.

Additionally, image samples used to compute percentiles can also be selected to only represent certain seasons, where some of the effects are less present, filtering image collections by day-of-year or by sun parameters, to avoid the most evident topographic effects.

The methods presented here work best when reflectance values, representing complex land-use and atmospheric changes, are similarly distributed. While for surface water this is frequently true, it may be less trivial under very complex land-use change \ref{enum:changes-multple-classes}. To detect these kind of changes automatically, additional inference steps may be required, for example, where different patterns can be recognized in the distributions such as those presented in Chapter \ref{ch4}, Figure \ref{fig:prob-sampled-water-cloud-snow}.

While linear regression may be a good choice to detect monotonic long-term changes, surface water may follow much more complex trends, requiring the use of more complex methods to capture them. For example, logistic regression can be used instead of linear regression to detect abrupt changes such as construction or decommissioning of reservoirs, or to analyze autocorrelation to detect recurring surface water changes.

With the increasing number of satellite images available, the use of statistical methods may be a powerful instrument, providing an easy way for change or anomaly detection before the use of more complex methods.


\section{Essential ML ideas}
\smallskip \hrule height 2pt \smallskip

\begin{itemize}
	\item Never ever \underline{ever} touch the test set
	\item You know you are overfitting when there is a big test between train and test results.  E.g. metric of percent wrong.
	\item Need to be comfortable taking a hit on fitting accuracy if you can get a benefit on the result.
	\item Bias vs variance trade-off.
		High bias when the model is too simple \& doesn't fit the data well.
		High variance is when small changes to the data set lead to large solution changes.
	\item If features are non discriminative in the beginning, they don't work for any classifier.  % week 4 reminder
	\item Your feature vector often has a smaller dimension that the feature space.    % week 4, Friday.
		If you have too long of a feature vector, you may get overfitting.
	\item You need to prevent the optimizer from getting an easy way out.  % week 7 audio
	\item Whenever we are building a discriminative classifier,
        		we should not expect it to reason about things it has never seen before
    		You can't classify on something that has only seen + points.
    		The classifier needs to see some + and some - or it will fail miserably.   % week 8 audio
	\item We should always avoid making hard decisions early.  % week 8 audo.
		 E.g. don't put a lot of trust in classifiers early on in bagging.
\end{itemize}

You can do $l_2$ normalization for a feature vector to get a unit vector:   % week 6 audio.
	Convert $x$ to $\hat{x}$ so that if you form $||\hat{x}||_2^2 = 1$
	Can also do $l_1$


\section{Setting the map projection} \label{subsec:adv_mapproj}
%------------------------------------------------------

In \scalerm, grids are allocated based on actual distance. The latitude and longitude for all grids are calculated by using the latitude and longitude of a certain reference point using a map projection. All information pertaining to the latitude and longitude of the grids are output to all output files in \netcdf format. The locations of the domain and the map projection can be configured in \nmitem{PARAM_MAPPROJECTION}. \textcolor{blue}{This configuration must be shared among the configuration files \texttt{pp.conf}, \texttt{init.conf}, and \texttt{run.conf}.} An example is as follows:
\editboxtwo{
\verb|&PARAM_MAPPROJECTION| & \\
\verb| MAPPROJECTION_basepoint_lon = 138.727778D0,| & \\
\verb| MAPPROJECTION_basepoint_lat = 35.360556D0,|  & \\
\verb| MAPPROJECTION_type          = 'MER',|        & ; Choose from Table \ref{tab:map_proj}.\\
\verb|/| & \\
}

\begin{table}[htb]
\begin{center}
\caption{Map projections selectable in \scalerm}
\begin{tabularx}{150mm}{|l|X|} \hline
 \rowcolor[gray]{0.9} \verb|MAPPROJECTION_type| & Map projection\\ \hline
 \verb|NONE| & No map projection for ideal experiment ( default ) \\ \hline
 \verb|LC|   & Lambert conformal conic projection   \\ \hline
 \verb|PS|   & Polar stereo projection           \\ \hline
 \verb|MER|  & Mercator projection               \\ \hline
 \verb|EC|   & Equi-rectangular projection        \\ \hline
\end{tabularx}
\label{tab:map_proj}
\end{center}
\end{table}

\nmitem{MAPPROJECTION_basepoint_lat, MAPPROJECTION_basepoint_lon} are the latitude and longitude at a reference point. This reference point is the center of the calculation domain by the default setting.
In \scalerm, the north and south latitudes are expressed by positive and negative values, respectively. The east and west longitudes are also expressed by positive and negative values, respectively. It is possible to express longitude using an angle greater than $180^{\circ}$.
In above setting, the center of the domain is configured at $35.360556^{\circ}$N and $138.727778^{\circ}$E, where N and E denote the north latitude and east longitude, respectively. The entire calculation domain is placed at the center with a specified size.

\nmitem{MAPPROJECTION_type} provides the kind of map projection and \verb|MER| the Mercator projection.
Table \ref{tab:map_proj} shows the selectable map projections in the current version of \scalerm. In the case of the Mercator projection, the standard latitude, to which a cylinder is tangent, is specified as \nmitem{MAPPROJECTION_M_lat} in units of degrees. In general, the standard latitude is set to the Equator.
In \scalerm, \nmitem{MAPPROJECTION_basepoint_lat} is used as standard latitude, unless \nmitem{MAPPROJECTION_M_lat} is explicitly specified. This is because this method is more precise with less distortion than the usual Mercator projection.

The text below explains the Lambert conformal conic projection, the usage frequency of which is highest. The example is the same as that in the file used in the tutorial for the real atmospheric experiment.
\editbox{
\verb|&PARAM_MAPPROJECTION| \\
\verb| MAPPROJECTION_basepoint_lon = 135.220404,| \\
\verb| MAPPROJECTION_basepoint_lat = 34.653396,| \\
\verb| MAPPROJECTION_type          = 'LC',| \\
\verb| MAPPROJECTION_LC_lat1       =  30.0,| \\
\verb| MAPPROJECTION_LC_lat2       =  40.0,| \\
\verb|/| \\
}
In \scalerm, two standard parallels are used for this projection.
The north and south of them are specified in \nmitem{MAPPROJECTION_LC_lat1, MAPPROJECTION_LC_lat2}
in units of degrees.
In the region between the latitudes, the ratio of the latitudinal to longitudinal length is adjusted to be close to the Earth’s ellipsoid face.

Furthermore, it is possible for the reference point (\nmitem{MAPPROJECTION_basepoint_lon},\\
\nmitem{MAPPROJECTION_basepoint_lat}) to be displaced from the default setting, i.e., domain center as follows:

\editbox{
\verb|&PARAM_MAPPROJECTION| \\
\verb| MAPPROJECTION_basepoint_lon = 135.220404,| \\
\verb| MAPPROJECTION_basepoint_lat = 34.653396,| \\
\verb| MAPPROJECTION_basepoint_x   = 100.0,| \\
\verb| MAPPROJECTION_basepoint_y   = 100.0,| \\
\verb| MAPPROJECTION_type          = 'LC',| \\
\verb| MAPPROJECTION_LC_lat1       = 30.0,| \\
\verb| MAPPROJECTION_LC_lat2       = 40.0,| \\
\verb|/| \\
}

The location of the reference point is specified as the distance from the south-west corner (bottom-left corner).;
\nmitem{MAPPROJECTION_basepoint_x} and \nmitem{MAPPROJECTION_basepoint_y} are the distances in units of meters between the south-west corner and the reference point in \XDIR and in \YDIR, respectively.
If they are not specified, the center of projection corresponds to that of the domain.
Figure \ref{fig:map_lc} shows both cases.

\begin{figure}[t]
\begin{center}
  \includegraphics[width=0.8\hsize]{./../../figure/LC_latlon_xy.pdf}\\
  \caption{The relationship between projection center and calculation domain.; (a) Default setting and (b) the case where the center of projection is displaced at the center of the domain.
    The red line indicates the boundary of the calculation domain.}
  \label{fig:map_lc}
\end{center}
\end{figure}


\subsection{Dirac distribution}


In this section, we discuss dynamic steady-state power and temperature analysis
under process variation. Unlike transient analysis, this analysis cannot be done
one time step at a time, since the repetitive workload needs to be taken into
account at once in order to calculate the corresponding dynamic steady state.
Thus, similarly to the contrast between \sref{transient-analysis} and
\sref{dynamic-steady-analysis}, the solutions in \sref{chaos-transient-analysis}
and this section differ. However, they still rely on the same methodology
outlined in \sref{chaos-uncertainty-analysis} and shown in
\fref{chaos-overview}.

\subsection{\problemtitle}

The system model is the same as the one in \sref{chaos-transient-analysis}. The
only difference is that it is more convenient to define the power model as
follows:
\begin{equation} \elab{chaos-power-model-bulk}
  \mp = \f(\vu, \mq).
\end{equation}
The function $\f: \real^\nu \times \real^{\np \times \ns} \to \real^{\np \times
\ns}$ is supposed to return the periodic power profile that corresponds to the
periodic workload being analyzed.

\inputalgorithm{chaos-dynamic-steady-solution-iterative}
The solution to probabilistic dynamic steady-state analysis is based on the
deterministic one presented in \sref{dynamic-steady-solution}. The power,
temperature, and other vectors that appear in \sref{dynamic-steady-solution}
become stochastic in the present context, which also concerns the boundary
condition given in \eref{dynamic-steady-condition}. The pseudocode for a
procedure that delivers \mq for a fixed \vu is listed in
\aref{dream-dynamic-steady-solution}. The algorithm does not take account of the
interdependence between power and temperature; however, this interdependence can
be addressed via one of the techniques presented in \sref{power-temperature}. In
this section, we use the iterative approach illustrated in
\aref{dream-dynamic-steady-solution-iterative}. For clarity, this algorithm is
rewritten here as shown in \aref{chaos-dynamic-steady-solution-iterative}; the
main difference is that the calculation of power on line~3 is now based on
\eref{chaos-power-model-bulk}.

\begin{remark}
The application of the linear approximation described in
\sref{power-temperature} is problematic in this case. This technique is suitable
when the only varying parameter is temperature, and all other parameters have
nominal values. In that case, it is relatively easy to decide on a
representative temperature range and apply a curve-fitting procedure. In this
case, however, the power model has multiple parameters that range far from their
nominal values.
\end{remark}

To recapitulate, the quantity of interest \g in \fref{chaos-overview} is the
dynamic steady-state power and temperature profiles, which are denoted by \mp
and \mq, respectively. The calculation of this quantity is shown in
\aref{chaos-dynamic-steady-solution-iterative}.

\subsection{Surrogate Construction}

At Stage~3 in \fref{chaos-overview}, the procedure delineated in
\sref{chaos-construction} is applied to
\aref{chaos-dynamic-steady-solution-iterative} from Stage~1 with respect to the
output of Stage~2. Concretely, \aref{chaos-dynamic-steady-solution-iterative} is
utilized inside \aref{chaos-construction} via Algorithm~G. In this case,
Algorithm~G calls \aref{chaos-dynamic-steady-solution-iterative} and returns \mp
or \mq or both, depending on what is actually needed for the subsequent
calculations, in a suitable format.

Suppose, for instance, that the designer is interested in analyzing solely the
dynamic steady-state temperature profile \mq. Following
\rref{chaos-multidimensional-output} concerning vector-valued quantities, \g is
treated as an $\np \ns$-element row vector, in which case each coefficient
$\hat{\g}_{\vi}$ in \eref{chaos-expansion} is also such a vector. The projection
matrix, which is defined in \eref{chaos-projection-matrix}, and
\aref{chaos-construction} should be reinterpreted accordingly: \vg is an $\nq
\times \np \ns$ matrix whose row~$j$ is \mq computed at point~$j$ of the
quadrature in \eref{chaos-coefficient} and reshaped into a row vector.
Similarly, $\hat{\vg}$ should be understood as an $\nc \times \np \ns$ matrix
whose row~$i$ is coefficient~$i$ of the expansion in \eref{chaos-expansion}.
Recall that a certain ordering is assumed to be imposed on quadrature points and
polynomial terms.

The constructed \ac{PC} expansion can now be post-processed as required, which
is already the topic of Stage~4 in \fref{chaos-overview}; see
\sref{chaos-processing}.


\documentclass[11pt]{article}
\usepackage{geometry}
\geometry{a4paper,left=2.5cm,right=2.5cm,top=2.5cm,bottom=2.5cm}
\usepackage{natbib}
\usepackage{color}
\definecolor{mygreen}{RGB}{28,172,0} % color values Red, Green, Blue
\definecolor{mylilas}{RGB}{170,55,241}
\usepackage{epsfig}
\usepackage{amssymb,amsmath}
\usepackage{enumerate}
\usepackage{enumitem}
\usepackage[utf8]{inputenc}
\usepackage{hyperref}
\usepackage{mathtools}

\newcommand{\ssd}{\text{ssd}}
\newcommand{\sS}{\mathsf{S}}
\newcommand{\tot}{\text{tot}}

\begin{document}

\input{symbols}

\section*{One-Layer Rotating Shallow Water Model}

\subsection*{Classical Form}

The one-layer, two-dimensional rotating shallow water model
on a rotating $f$-plane can be written as,
\begin{align}
\frac{\partial u}{\partial t} + \left(\bf{u} \cdot \nabla\right) u - f v
&= - g \frac{\partial h}{\partial x},\\
\frac{\partial v}{\partial t} + \left(\bf{u} \cdot \nabla\right) v + f u
&= - g \frac{\partial h}{\partial y},\\
\frac{\partial h}{\partial t} + \nabla \cdot \left( h \bf{u} \right) &= 0.
\end{align}
This describes the motion of a pancake like fluid in that it is thin in the vertical
and much longer in the horizontal.  It contains pressure forces due to the free surface
and a Coriolis pseudo-force because of the rotating frame of reference.

The fluid moves as columns that can be
translated in the horizontal and stretched/contracted in the vertical.  If the
height of a column changes then the voriticty must change, as can be reflected
in the fact that, in the absence of forcing and dissipation,
Potential Vorticity is conserved following the motion,
$$
\frac{D}{Dt} \left(
\frac{ \frac{\partial v}{\partial x} - \frac{\partial u}{\partial y} + f }{h} \right) = 0.
$$

\subsection*{Conservation Form}

There are a variety of forms, these equations can be written.
One of them is conservation where the three fields that have time
derivatives are $U = hu$, $V = hv$ and $h$.
\begin{align}
\frac{\partial U}{\partial t}
+ \frac{\partial}{\partial x}\left( \frac{U^2}{h} + \frac{g h^2}{2} \right)
+ \frac{\partial}{\partial y}\left( \frac{U V}{h}  \right)
- f V
&= 0,\\
\frac{\partial V}{\partial t}
+ \frac{\partial}{\partial x}\left( \frac{U V}{h}  \right)
+ \frac{\partial}{\partial y}\left( \frac{V^2}{h} + \frac{g h^2}{2} \right)
+ f U
&= 0,\\
\frac{\partial h}{\partial t} + \nabla \cdot \left( h \bf{u} \right) &= 0.
\end{align}

Conservation form is attractive because it can help to ensure that, using a clever numerical
scheme, some quantitites are conserved.  Note that if you have topography then there
are source terms that appear on the right-hand side and this causes some problems.

\subsection*{Vorticity-Bernoulli Form}

A third form arises from rewriting the nonlinear
acceleration terms as a gradient and a cross product term.  Using a vector identity,
it can be shown that the above system is mathematically equivalent to the following,
\begin{align}
\frac{\partial u}{\partial t}  - q h v
&= - \frac{\partial B}{\partial x},\\
\frac{\partial v}{\partial t}  + q h u
&= - \frac{\partial B}{\partial y},\\
\frac{\partial h}{\partial t} + \nabla \left( h \bf{u} \right) &= 0.
\end{align}
Note that above we have defined the vorticity and the Bernoulli function,
\begin{align}
q &= \frac{\zeta + f}{h}
 = \frac{\frac{\partial v}{\partial x} - \frac{\partial u}{\partial y} + f}{h},\\
B & = g h + \frac12 \left( u^2 + v^2 \right).
\end{align}

Before we look at solving this complicated set of equations

\subsection*{ $1 \frac12$ Dimensional Limit}

We assume that none of the variables depend on one horizontal direction, say $y$.
But, it is very important to realize that the velocity in that direction is not necessarily zero.
Indeed, if you have flow in the $x$-direction, the Coriolis force will deflect it to the right,
which will then generate a flow that is perpendicular.  This will continue and often give rise
to inerital oscialltions in the horizontal.  So we can have motion in either direction but
the motion only changes with respect to $x$.

If we simply the governing equations we get
\begin{align}
\frac{\partial u}{\partial t}  - q h v
&= - \frac{\partial B}{\partial x},\\
\frac{\partial v}{\partial t}  + q h u
&= 0,\\
\frac{\partial h}{\partial t} + \frac{\partial}{\partial x} \left( h u \right) &= 0.
\end{align}
where the vorticity and the Bernoulli function simplify to,
\begin{align}
q & = \frac{\zeta + f}{h} = \frac{\frac{\partial v}{\partial x} + f}{h},\\
B & = g h^2 + \frac12 \left( u^2 + v^2 \right).
\end{align}

\subsection*{Conserved Quantities}

In the purely conservative or nondissipative limit,
there are three quantities that are exactly conserved.

1) Mass:
$$
M = \int_D h \, dA
$$

2) Total Energy: sum of potential and kinetic energies
$$
E = \frac12 \int_D \left(g h^2 + h( u^2 + v^2)\right) \, dA
$$

3) Potenal Enstrophy:
$$
Q = \frac12 \int_D  h q^2 \, dA
$$

In a numerical model we cannot expect these to be conserved
but we would like them to be close to be conserved.
If they are very badly conserved than this could reflect that
the numerical scheme is behaving badly.
However, just because these are conserved that does not
guarantee that the solution is correct.
But they are usually good indicators as to how our method is
doing in the conservative limit.

Of course when nonconservative forces are introduced things will change.
One might argue that since the world is non-dissipative then we don't
need to worry about conserving these.  However, it is desirable to
know that basis of your model is well behaved and therefore why
we should worry about conserved quantities.

\end{document}


%-----------------------------------------------------------------------------------------------
\section{Introduction}

    Polytropic        processes---\emph{poly}:        many,        \emph{tropic}:         forms,
    ways~\cite{2020-NaaktgeborenC-engrXiv}---are a versatile class of equilibrium thermodynamics
    processes          with          many          applications          in          engineering
    thermodynamics~\cite{2002-MoranMJ+ShapiroHN-LTC,                 2013-CengelYA+BolesMA-AMGH,
    2015-KroosKA+PotterMC-Cengage,  1986-JonesJB+HawkinsGA-Wiley},  given   by   the   following
    relationship
    %
    \begin{equation}
        Pv^n = \mathsf{c} = \mbox{const.},
        \label{eq:polytropic}
    \end{equation}
    %
    \noindent where $P$ is the system pressure, $v$ the system specific volume, and $n$  is  the
    polytropic exponent.

    Polytropic       processes       are       said       to       find        support        in
    measurements~\cite{2013-CengelYA+BolesMA-AMGH,                   2002-MoranMJ+ShapiroHN-LTC,
    1985-WylenG-Wiley};         however,          theory~\cite{2012-ChristiansJ-IntJMechEngEduc,
    2020-NaaktgeborenC-engrXiv} predicts a fairly restrictive set of conditions---on the process
    boundary and internal conditions and on the underlying substance model---are required for  a
    process to follow Eq.~(\ref{eq:polytropic}) \emph{exactly}---see, for instance, Theorem~3 of
    reference~\cite{2020-NaaktgeborenC-engrXiv}.

    Thus, in non-ideal  settings,  polytropic  processes  either  (i)~hold  approximately  in  a
    \emph{local} neighborhood of  a  given  state,  for  constant-$n$,  or  (ii)~the  polytropic
    exponent must be generalized into a function of a state property, such  as  temperature,  or
    even further generalized.

    Reference~\cite{2020-NaaktgeborenC-engrXiv} brings useful definitions for  discussions  like
    this, such as that of a \emph{logical process},  and  also  those  of  \emph{exact}  and  of
    \emph{local} polytropic processes.

    In \emph{exact polytropic process}, Eq.~(\ref{eq:polytropic}) holds exactly with a constant,
    unique polytropic exponent, for the entire duration of the \emph{logical process}. They  are
    shown to be able to represent any straight line segment process in $\log P  \times  \log  v$
    coordinates~\cite{2020-NaaktgeborenC-engrXiv}.

    Moreover, process whose representations in $\log P \times \log v$  coordinates  are  curved,
    are shown to be able to be  approximated  by  a  finite  number  of  \emph{local  polytropic
    processes} within  finite  error  intervals~\cite{2020-NaaktgeborenC-engrXiv}.  Observations
    like these attest the flexibility of processes based on the  polytropic  relationship,  thus
    justifying the flexibility encoded in their name.

    Despite their enormous flexibility, the defining concept  of  polytropic  processes  can  be
    further generalized as to allow for increased flexibility, so that some  processes  families
    with curved representation in $\log P  \times  \log  v$  coordinates  might  have  an  exact
    representation in such further generalized process relation  concept.  The  introduction  of
    such generalizations can be of theoretical and of applied interest.

    Therefore, this work proposes one such generalization, named \emph{polyekthetic  processes},
    that form a \emph{larger process set} of which  the  polytropic  process  set  is  a  proper
    subset, so that any polytropic process is a particular case of a polyekthetic  process,  but
    not necessarily the opposite.

    The origin  (etymology)  for  the  proposed  `polyekthetic'  term  is  also  given,  and  an
    application yielding exact polyekthetic processes  (but  not  exactly  polytropic  ones)  is
    provided.

%-----------------------------------------------------------------------------------------------


    \filetitle{rngcmp}{Compare two IRIS date ranges}{dates/rngcmp}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
Flag = rngcmp(R1,R2)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{R1}, \texttt{R2} {[} numeric {]} - Two IRIS date ranges that
  will be compared.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{Flag} {[} \texttt{true} \textbar{} \texttt{false} {]} - True
  if the two date ranges are the same.
\end{itemize}

\paragraph{Description}\label{description}

An IRIS date range is distinct from a vector of dates in that only the
first and the last dates matter. Often, date ranges are context
sensitive. In that case, you can use \texttt{-Inf} for the start date
(meaning the earliest possible date in the given context) and
\texttt{Inf} for the end date (meaning the latest possible date in the
given context), or simply \texttt{Inf} for the whole range (meaning from
the earliest possible date to the latest possible date in the given
context).

\paragraph{Example}\label{example}

\begin{verbatim}
r1 = qq(2010,1):qq(2020,4);
r2 = [qq(2010,1),qq(2020,4)];

rngcmp(r1,r2)
ans =
    1
\end{verbatim}


\section{Results}

\subsection{Convincing tuning curves}
\begin{figure}[H]
    \centering
    \begin{subfigure}[b]{0.4\textwidth}
        \includegraphics[width=\textwidth]{figs/Mouse12-120806_awakedata.mat/T3C10.eps}
        \caption{Cell with convining tuning curve. There appears to be a strong reaction to stimuli.}
        \label{fig:1a_high_mut_info}
    \end{subfigure}
    \begin{subfigure}[b]{0.4\textwidth}
        \includegraphics[width=\textwidth]{figs/Mouse12-120806_awakedata.mat/T2C2.eps}
        \caption{Cell with unconvincing tuning curve, and the stimuli appears to be uncorrelated with the response.}
        \label{fig:1a_low_mut_info}
    \end{subfigure}
    \caption{Two cells from mouse 12 with different tuning curves}
    \label{fig:1a_two_tuning_curves}
\end{figure}
To understand how different cells reacts to different stimuli, we created tuning curves from \cref{eq:firing_rate}. Some of the cells showed very convincing tuning properties with strong peaks for specific stimuli with little background, such as in \cref{sub@fig:1a_high_mut_info}. Others appears to mostly be noise, and does not appear to contain much information such as shown in \cref{fig:1a_low_mut_info}.
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.8]{figs/1b.eps}
    \caption{Promising tuning curves from two different mice and different brain areas. }
    \label{fig:1b_convincing_tuning}
\end{figure}
In \cref{fig:1b_convincing_tuning} there are 4 different neurons from different mice and brain areas, with different convincing tuning properties. Neuron T4C14 also showes how the tuning curve wraps around when the stimuli goes beyond $360^\circ$. Most of the neurons appears to have unique tuning properties, and tend to prefer different stimulus. By combining multiple neurons it appears to be possible to at least perform an edjucated guess about the current head direction by knowing the activitity in different HD cells. The different tuning curve spikes appear to be well distributed around a circle, i.e. the cells seems to cover most head directions well.
\subsection{Peculiar tuning properties}
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.8]{figs/peculiar_tuning.eps}
    \caption{Cell with peculiar tuning properties. Notice how the cells have multiple firing patterns.}
    \label{fig:peculiar_tuning}
\end{figure}
Some cells also had more peculiar tuning properties with multiple firing patterns. \cref{fig:peculiar_tuning} shows three cells with multiple firing patterns, but which still appears to contain useful information about head direction. These cells may not indicate one specific head angle, but by knowing when these cells are active we can still use the information to rule out other possibilites and shape the distribution of possible head directions.

\subsection{Mutual information}
To quantify the amount of information about head direction for each cell, the mutual information score were calculated using \cref{eq:mutinfo_disc}. Each of the plots include the mutual information score in the title, and from \cref{fig:1a_two_tuning_curves} we can see how the mutual information tend to be higher for the more convincing tuning curves. The mutual information score is in bits (shannons) per unit time, and can the thought of as how many yes/no answers (bits) we gain on average after observing a cell during a sufficiently short time interval \cite{mutualinfo}.

\begin{figure}[H]
    \centering
    \begin{subfigure}[b]{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/weird_mutual.eps}
        \caption{Convincing tuning curves with low mutual information}
        \label{fig:weird_mutual}
    \end{subfigure}
    \begin{subfigure}[b]{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/probs.eps}
        \caption{Prior probability distribution}
        \label{fig:probs}
    \end{subfigure}
\end{figure}

However, low mutual information score does however not imply that the corresponding tuning curve is of no interest. In \cref{fig:weird_mutual} there are three tuning curves which looks convincing, but which results in a low mutual information score. By also looking at the prior probability density function in \cref{fig:probs}, we can see how these tuning curves have their peak in a head direction which was less visited by the mice. The prior indicate what we believed the head angle distribution were before observing any cell activity. When observing cell activity which contradicts our prior belief (i.e. high activity in cells which triggers at an unlikely head direction), we simply need more information to "accept it". Observing high activity in these neruons contradicts our prior belief and would make us more uncertain about what the true head direction really is (i.e. all possibilites becomes more equally likely).

\subsection{Principal component analysis}
\begin{figure}[H]
    \centering
    \begin{subfigure}[b]{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/Mouse12-120806_awakedata.mat/pca.eps}
        \caption{PCA for mouse 12}
    \end{subfigure}
    \begin{subfigure}[b]{0.49\textwidth}
        \includegraphics[width=\textwidth]{figs/Mouse28-140313_awakedata.mat/pca.eps}
        \caption{PCA for mouse 28}

    \end{subfigure}
    \caption{Principal components analysis for the two datasets. The principal components are visualized both in cartesian coordinates as well as in polar coordinates. The plots clearly shows similar stimuli clustering together. The polar coordinate angle $\theta$ appears to explain the difference head direction well.}
    \label{fig:pca}
\end{figure}
To better visualize and utilize the data, PCA analysis were used to reduce the number of variables, while still keep as much information as possible. In the rather beautiful plot \cref{fig:pca} it shows the two first principal components, where the scores are color-coded according to the head direction. The plots clearly shows how the different stimuli is grouped by color, and there appears to be a strong relationship between the principal components and the head direction. The two components explains about $15\%$ of the variance which considering the amount of cells (variables in the original dataset) makes this a very useful visualization.
It also appears to be a slight trend in the samples, where the colour appears to get brighter with clockwise motion. By applying a non-linear transformation to polar coordinates, we see how the head direction is well explained by the rotation $\theta$ (not to be confused with the head direction), and only barely depends on the radius $\rho$. Notice how the color increase in intensity almost linearly as principal component scores rotate around the circle. This indicates a circular relationship between the first two principal components and the head direction.


\section{Correlation coefficients}\label{app:correlations}

A graphical representation of the correlation coefficients for the features in
Table~\ref{table:manual_features}.

\begin{figure}[!htbp]
        \begin{center}
            \includegraphics[width=.75\linewidth]{../images/standard_correlation_plot.pdf}
        \end{center}
        \caption{Correlation coefficients of features in Table~\ref{table:manual_features}
        for standard tournaments}
\end{figure}
\begin{figure}[!htbp]
    \begin{center}
        \includegraphics[width=.75\linewidth]{../images/noise_correlation_plot.pdf}
    \end{center}
    \caption{Correlation coefficients of features in Table~\ref{table:manual_features}
    for noisy tournaments}
\end{figure}
\begin{figure}[!htbp]
    \begin{center}
        \includegraphics[width=.75\linewidth]{../images/probend_correlation_plot.pdf}
    \end{center}
    \caption{Correlation coefficients of features in Table~\ref{table:manual_features}
    for probabilistic ending tournaments}
\end{figure}
\begin{figure}[!htbp]
    \begin{center}
        \includegraphics[width=.75\linewidth]{../images/probend_noise_correlation_plot.pdf}
    \end{center}
    \caption{Correlation coefficients of features in Table~\ref{table:manual_features}
    for noisy probabilistic ending tournaments}
\end{figure}
\begin{figure}[!htbp]
    \begin{center}
        \includegraphics[width=.75\linewidth]{../images/merged_correlation_plot.pdf}
    \end{center}
    \caption{Correlation coefficients of features in Table~\ref{table:manual_features}
    for data set}
\end{figure}


\section{Problem statement}
Automatic music transcription consists of analyzing digital recordings of songs and identify things like rhythm, melodies, pitch, bass lines. The main subtasks of AMT are:

\begin{itemize}
	\item \textbf{Pitch estimation}
	\item \textbf{Beat detection}
	\item \textbf{Instrument detection}
\end{itemize}

Because of the polyphonic approach of music, AMT is still unresolved today. The state of the art approaches struggle to get an accuracy greater than 70\%. It's a problem that requires a lot of digital signal processing knowledge in order to optimize the data and split the subtask even further in onset and offset detection, genre detection and velocity detection. In order to have a full prediction, you need to cover all of these tasks.

The proposed solution attempts the \textbf{Pitch estimation} part of AMT, using a heuristic approach to find the beat and creating spectrograms of the audio in order to get the instrument out of the equation.

Fundamentally, AMT is about identifying the pitch and duration of played notes, so they can be converted in traditional music notation on a sheet

\section{Data set and input representation}
The data set used was the MAPS database~\cite{maps}. It is a piano database for multi-pitch estimation and automatic transcription of music. It contains MIDI-annotated piano recordings, composed of isolated notes, random-pitch chords, usual musical chords and pieces of music. It provides a diverse range of sounds from various recording conditions.
\par
The recordings are CD quality (16-bit, $44-kHz$ sampled stereo audio) and the related aligned MIDI files contain the ground truth. The overall size of the database is 40GB.
\par
Working with sound in neural networks is different from dealing with images. The input contains audio files so before feeding it to the network we need to turn it into a visual representation. The most common way to represent a sound is the audio representation in the time domain. (Figure \ref{fig:waveform})

\begin{figure}[h!]
	\caption[Example of audio representation in the time domain]{ Example of audio representation in the time domain~\cite{genre_class} }
	\centering
	\label{fig:waveform}
	\includegraphics[width=1\textwidth, height=\textheight, keepaspectratio]{"resources/waveform"}
\end{figure}

Figure \ref{fig:waveform} shows the evolution in time of the song and you can see the oscillation of the signal. The $Y$ axis represents the amplitude while the $X$ axis is the time. In this representation, it's not possible to distinguish the notes that are playing. Because of this, we need a better representation. This is where the Fourier Transformations come in handy. Using the transformation on the data we get a representation over frequency instead of time. This is also known as a spectrum. The spectrum reveals relevant information that's crucial to analyze the audio. (Figure \ref{fig:cq_vs_stft})
\par

The input MIDI file will be split into $\frac{1}{16}\cdot second$ window frames. For example, a note lasting 1 second will have 16 consecutive windows created. Each window is then turned into a .wav file, using fluidsynth~\cite{fluidsynth}, which in turn will be transformed using the Constant-Q transform.
One window transformation takes ~0.3 seconds making the process slow and punishing if a bug is found in the preprocessing.

\par
To compute the Constant-Q transform, the library \textit{Librosa}~\cite{librosa} will be used, in particular, the method called \textit{librosa.cqt}.
The parameters that we will use are:
\begin{itemize}
	\item \textbf{y}: Audio signal
	\item \textbf{sr}: Sampling rate
	\item \textbf{fmin}: Minimum frequency
	\item \textbf{n\_bins}: Number of frequency bins
	\item \textbf{bins\_per\_octave}: Number of bins per octave
	\item \textbf{hop\_length}: Number of samples between successive CQT columns
\end{itemize}
\par

The result of the Librosa function will be plotted, resulting in a logarithmic scale spectrogram of size $145\times49$ (Figure \ref{fig:q_spec}). This way from a song of length $x$, $x \cdot 16$ spectrograms will be extracted. These are the input of the network.

\begin{figure}[H]
	\caption[Example of a constant-q spectrogram]{ Example of a constant-q spectrogram }
	\centering
	\label{fig:q_spec}
	\includegraphics[width=1\textwidth, height=\textheight, keepaspectratio]{"resources/q_spec"}
\end{figure}

\par

For the cross validation, a train-validation-test split approach was used. The learning set was split $80\%$ for training, $20\%$ for testing and $10\%$ of the training set is used for validation. A visual representation can be seen in Figure \ref{fig:split}

\begin{figure}[H]
	\caption[Visual representation of the split]{ Visual representation of the split}
	\centering
	\label{fig:split}
	\includegraphics[width=1\textwidth, height=\textheight, keepaspectratio]{"resources/train_split"}
\end{figure}

\section{Labeling}
A supervised machine learning model needs the data to be labeled. As it was explained in the subsection above, the input song is going to be represented in the frequency domain. The labeling will be provided using the MIDI files associated with the input \textit{.wav} files. They contain information about the notes playing at a certain time. The labels will be arrays showing the played notes represented with \textit{one-hot encoding}.
\par

One-hot encoding is a widely used technique in neural networks. It consists of creating an array of boolean values~\cite{one-hot}. In this case, every column symbolizes a possible note that can be played at a certain moment. Because of this, there are as many columns as possible notes (128). This will be done per window, so the goal is to see which notes are played in a certain window (Table \ref{table:one_hot}). A value of 1 in a cell means that the specific musical note has been played during that window, while a value of 0 is the opposite.

\begin{table}
	\centering
	\caption{Example of one hot representation of a MIDI file}
	\begin{tabular}{||c c c c c||}
		\hline
		Note & Window 1 & Window 2 & ... & Window n \\ [0.5ex]
		\hline\hline
		1 & 0 & 0 & ... & 0 \\
		\hline
		2 & 1 & 0 & ... & 0 \\
		\hline
		3 & 0 & 0 & ... & 1 \\
		\hline
		... & ... & ... & ... & ... \\
		\hline
		128 & 0 & 1 & ... & 0 \\ [1ex]
		\hline
	\end{tabular}
	\label{table:one_hot}
\end{table}

\section{First CNN architecture}
The first neural network has been created with the aim of obtaining some initial results and it has served to learn how a model developed in Keras~\cite{keras} behaves.
\par
An architecture similar to the Keras MNIST is evaluated~\cite{keras_mnist}. It has been created to predict handwriting digits, the input and output data are completely different, but this is a standard architecture for image recognition with small patches. No optimization was done on this one as its aim was to perform a first approach that deals with the input data.
\par

The architecture is shown in Table \ref{table:initial}. It performed poorly but it helped in the preprocessing progress as it learned really fast on a GPU. Making changes to the input sizes and different types of transformations and retesting was really quick.

\begin{table} [h!]
	\centering
	\caption{Initial CNN architecture}
	\begin{tabular}{ |c|c|}
		\hline
		Layer (type) & Output Shape  \\ \hline
		Conv2D &  (None, 72, 24, 32) \\ \hline
		Conv2D & (None, 70, 20, 64)\\ \hline
		MaxPooling2 & (None, 35, 10, 64) \\ \hline
		Dropout(0.25) & (None, 64)\\ \hline
		Flatten & (None, 2048) \\ \hline
		Dense & (None, 128) \\ \hline
		Dense & (None, 128) \\ \hline
	\end{tabular}
	\label{table:initial}
\end{table}

\section{Proposed CNN architecture}
The proposed architecture is similar to AlexNet~\cite{alexnet} (Figure \ref{fig:alexnet}). This architecture won the Image Classification Challenge in 2012\cite{challenge_2012}. It was used on high-resolution images into 1000 different classes. This was too big for our input data of $145\times49\times3$ size images. The kernel and stride sizes were adjusted to fit our data. Some layers were dropped and some dropout layers were added in order to reduce overfitting. The dropout layers have a $0.5$ chance to deactivate a neuron. This forces the layer to learn the same concept with different neurons, improving generalization.
\par

\begin{figure}[h!]
	\caption[Loss progression over the epochs]{ Loss for the final architecture}
	\centering
	\label{fig:loss}
	\includegraphics[width=1\textwidth, height=\textheight, keepaspectratio]{"resources/loss"}
\end{figure}

Three activation functions were tested in the hidden layers for the monophonic scenario as this was the first milestone. Without good accuracy here there's no point going further:
\begin{itemize}
	\item \textbf{Sigmoid}: 30\% accuracy
	\item \textbf{ReLu}: 83\% accuracy
	\item \textbf{Tanh}: 99.9\% accuracy
\end{itemize}

The model performed better using Tanh as the activation function inside the hidden layers. As for the output layer, Softmax was used as this is a multi class classification problem.

\par
The accuracy of the model was tested using custom made song with predetermined note overlap percentage.
The results can be seen in Table \ref{table:results}. As the amount of overlapping notes increases the accuracy of the model decreases. This happens because the system can't predict with high confidence all the notes from a window when their number increases. It works almost perfectly for monophonic songs, reaching an accuracy of 99.9\%. The sweet spot of the system is a combination of one, two and three overlapping notes with no more than 50\% and 10\% overlapping time for two and three notes respectively. This combination provides an overall accuracy of about 70\%.

\begin{table} [H]
	\centering
	\caption{Initial CNN architecture}
	\begin{tabular}{ |c|c|c|}
		\hline
		Max number of overlapping notes & Overlapping Time & Accuracy\\ \hline
		1 & 100\% & 99.9\%  \\ \hline
		2 & 10\% & 95.3\% \\ \hline
		2 & 50\% & 78.1\% \\ \hline
		2 & 100\% & 65.0\% \\ \hline
		3 & 10\% & 90.3\% \\ \hline
		3 & 50\% & 67.2\% \\ \hline
		3 & 100\% & 43\% \\ \hline
	\end{tabular}
	\label{table:results}
\end{table}

\begin{figure}[H]
	\caption[AlexNet Architecture]{ Original AlexNet architecture~\cite{alexnet}}
	\centering
	\label{fig:alexnet}
	\includegraphics[width=1\textwidth, height=\textheight, keepaspectratio]{"resources/alexnet"}
\end{figure}

\begin{table} [H]
	\centering
	\caption{Proposed CNN architecture}
	\begin{tabular}{ |c|c|c|}
		\hline
		Layer (type) & Output Shape & Param \# \\ \hline
		Conv2D &  (None, 72, 24, 32) & 896 \\ \hline
		MaxPooling2 & (None, 36, 12, 32) & 0\\ \hline
		Conv2D & (None, 17, 5, 64) & 18496 \\ \hline
		Conv2D & (None, 16, 4, 128) & 32896 \\ \hline
		MaxPooling2 & (None, 8, 2, 128) & 0 \\ \hline
		Flatten & (None, 2048) & 0 \\ \hline
		Dense & (None, 4096) & 8392704 \\ \hline
		Dropout & (None, 4096) & 0 \\ \hline
		Dense & (None, 4096) & 16781312 \\ \hline
		Dropout & (None, 4096) & 0 \\ \hline
		Dense & (None, 128) & 524416 \\ \hline
	\end{tabular}
	\label{table:cnn_architecture}
\end{table}
\newpage

\section{Application architecture}

\begin{figure}[H]
	\caption[Class diagram]{ Application class diagram }
	\centering
	\label{fig:class_diag}
	\includegraphics[width=1\textwidth, height=1\textheight, keepaspectratio]{"resources/class_diagram"}
\end{figure}
\clearpage

Theia7 has a layered application without the data layer. For the model and loggers creation, the abstract factory design pattern was used in order to facilitate easy creation and scaling for these features. The main business logic happens in the controller with the trainer and preprocessor at its disposal.
\par

The configuration of the application is contained inside a json configuration file. A wrapper over it was created to facilitate reading fields from it inside the classes (Figure \ref{fig:utils_diag}).

\begin{figure}[H]
	\caption[Utils diagram]{ Static classes diagram }
	\centering
	\label{fig:utils_diag}
	\includegraphics[width=1\textwidth, height=1\textheight, keepaspectratio]{"resources/utils_diagram"}
\end{figure}

\section{Output}
The output of the prediction will be a one-hot encoding of the song as described in Table \ref{table:one_hot}. This will be transformed into a MIDI file using the \textit{pretty-MIDI} library~\cite{pretty_midi}. Knowing that each window is a $\frac{1}{16}$ of a second we can find an approximation of the original note length by concatenating all consecutive windows predicted for it. The MIDI is then converted to wav using fluidsynth~\cite{fluidsynth} to have a sound comparison and the spreadsheet is created using the Sheet software~\cite{sheet}. An output example can be found in Figure \ref{fig:sheet_output}.

\begin{figure}[H]
	\caption[Sheet output example]{ Sheet output example }
	\centering
	\label{fig:sheet_output}
	\includegraphics[width=1\textwidth, height=\textheight, keepaspectratio]{"resources/sheet_example"}
\end{figure}

\section{Technologies used}
List of used technologies:
\begin{itemize}
	\item Python~\cite{python}
	\item Anaconda~\cite{anaconda}
	\item Librosa~\cite{librosa}
	\item Keras~\cite{keras}
	\item Tensorflow~\cite{tensorflow}
	\item Pretty-midi~\cite{pretty_midi}
	\item Sheet~\cite{sheet}
	\item Numpy~\cite{numpy}
	\item Pydub~\cite{pydub}
	\item Mido~\cite{mido}
	\item Cuda~\cite{cuda}

\end{itemize}


\chapter{A Machine Learning Framework for Register Allocation}
\label{chap:ch5}

In this chapter, we will talk about MLRA - \textit{A Machine Leaning Framework for Register Allocation}. Its main objectives are the following: to perform semantically correct code generation,  to support multiple backend architectures,
and to form proper communication between LLVM and ML model.

The main focus of this work will be on the \textit{infrastructure development}.

In section~\ref{sec:mlra:bg}, we will have an overview of Register Allocation and graph coloring problem. In section~\ref{sec:mlra:intro} and ~\ref{sec:mlra:pre_work}, we will talk about our proposal and the previous work respectively. In section~\ref{sec:mlra:components}, we discuss about different components of the framework like Capture Interference graph, ML model interface and Code generation.

In Section~\ref{sec:mlra:arch}, we discuss about our Support for multiple architecture.
In Section~\ref{sec:mlra:correct}, we present the correctness results.
In Section~\ref{sec:mlra:mlra_action}, we run the framework with an actual Model and report the (initial) results.
In Section~\ref{sec:mlra:conclusion}, we will conclude this chapter also outlining the future work.

\section{Background}\label{sec:mlra:bg}
Register Allocation is an old and classic optimization problem. Many researchers provided many solutions for it, some exact solutions, but many heuristics as well. The success of a heuristic  depends upon the domain expertise of its developer. There is always a possibility that some important heuristics are missed, leading to sub-optimal performance.

The allocation should be done efficiently as the number of the registers is limited in number and may lead to poor performance if the registers are not assigned to a memory-bound variable properly. This leads to an increase in the use of the memory load and store operations. The memory operations are costlier than the register operations in case of performance.

The popular register allocations are greedy, basic, fast, and pbqp register allocation. In this we are interested in the register allocation via Interference graph coloring. In the fig~\ref{fig:mlra-coloring}, there is a sample code for which we have the live interval interference chart. The Live interval are the scope of the variable when the variable was first defined and last used. Every variable has its own live interval and may interfere with other variable's live intervals. We can formulate this problem as the graph color program where the node is the variable or live interval and edges are the inference among the nodes. The Rule of Thumb, no two adjacent nodes be assigned the same color(register). The optimal graph coloring is a classic NP-Hard problem.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/mlra_coloring.png}
    \caption{Register Allocation as Interference graph coloring}
     \label{fig:mlra-coloring}
\end{figure}
The LLVM has Greedy allocator  based on the graph coloring. There are four strategies used by this allocator respectively namely splitting, coalescing, eviction, and spilling (fig~\ref{fig:mlra-strat}).

\paragraph{Splitting} - A selected node or live interval of the graph is divided into two sub-live intervals depending upon the point of splitting. The newly created live interval may or not form the edge between the existing nodes.

\paragraph{Coalescing} - In these two nodes which share the same variable value can be combined to form one node or single live interval.

\paragraph{Eviction} - If some node of the graphs is already assigned a colored, but we wanted to uncolor the respective node. This process is known as eviction.

\paragraph{Spilling} - If the allocator ran out of register or heuristically suggests to assign the given variable to the memory. Memory load-store are costlier operations.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/mlra_strategies.png}
    \caption{Register Allocation - Strategies}
     \label{fig:mlra-strat}
\end{figure}

The \textit{greedy allocator} uses all the four strategies in a heuristic manner and \textit{Basic allocator} uses splitting and spilling.

The current allocators are heuristically defined and provide sub-optimal solutions. Machine learning could help to improve the solutions.

\section{Introduction}\label{sec:mlra:intro}
We have developed a Machine learning framework for register allocation. It's main objectives are to perform semantically correct code generation, support multiple backend architecture and form proper communication between LLVM and ML model. The main focus of the work will be the infrastructure.

\section{Previous Work}\label{sec:mlra:pre_work}
\vk{More details before drawbacks}

In previous work by Das et al.~\cite{dlra:LLVMHPC_2020}, they tried to use the LSTM based deep learning model to predict the color (or register) for each node of the interference graph. The graphs with atmost 100 nodes are being supported by this work. \fixme{NOT CLEAR CLEANUP PLEASE}

There were several drawbacks to the above work. The solution was not integrated back to the LLVM pipeline. The proper modeling of the interference rule was not done. Because of this, the model could predict adjacent nodes to be assigned the same color, and this leads to incorrect register assignments. They have added a \textit{correction phase} to overcome this problem. Finally, the dataset used for the training was not realistic as it was sampled from random graphs using the graph based C library.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/mlra_components.png}
    \caption{Components of correct ML based register allocation}
     \label{fig:mlra-components}
\end{figure}

\section{Framework Components}\label{sec:mlra:components}
The communication between the Capture interference graph, ML model, and code generation is done via gRPC~\cite{grpc}. The gRPC is a medium for cross-platform communication between C++ (LLVM) and Python (ML Model). See Fig.\ref{fig:mlra-components}
\subsection{Capture Interference Graph}
We have written a pass in LLVM which is responsible for capturing the interference graphs for the given input files. We annotate the graph with more information which would be later used by the other components. Fig.~\ref{fig:mlra-cig}.

The node comprises of node label, information regarding type of register physical or virtual, color and register Id if physical register, the class of the register, and embedding representation.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/mlra_cig.png}
    \caption{Capture interference graphs}
     \label{fig:mlra-cig}
\end{figure}

The edge or interference that we are considering are between Virtual -- Physical registers and between Virtual -- Virtual registers. There are some physical register which are already in used and can’t be used to respect the interference rule, the edge is added. The Virtual -- Virtual is the trivial case.
\subsection{ML Model interface}
The Model interface could be any ML setting i.e. supervised, unsupervised, reinforcement and etc. The choice of the model depend upon the developer. The Model will be used in training as well as inference phase.

\subsubsection{Training}
From the previous Capture Interference graph component, we get the interference graphs and pass to the training paradigm. The interference graphs go to the model written in python where we request the LLVM to start the server through gRPC. After the server got started, the ML model runs and share its prediction with the CodeGen component through another gRPC call and receive some information as response. The response is used by model for it's proper execution. See Fig.~\ref{fig:mlra-training}.

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/mlra_training.png}
    \caption{Train the model using gRPC}
     \label{fig:mlra-training}
\end{figure}

After all the interference graphs are visited, training is finished and the trained model used for the inference.
\subsection{Code generation or inference}
After the model is trained, we have added the integrated model with LLVM so that code generation could be done using the prediction suggested by the configured trained model.

While inference,the input file is given the LLVM and in the pipeline,  capture interference graph pass is invoke and the graph is then send to the Python through gRPC where it is feed to the trained model. The trained model send a color prediction json to the LLVM. In LLVM, the predicted colors/registers are mapped to the virtual registers and code is generated. See Fig.~\ref{fig:mlra-inference}.
\begin{figure}[t]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/mlra_inference.png}
    \caption{Code generation using the trained model}
     \label{fig:mlra-inference}
\end{figure}

\section{Architecture}\label{sec:mlra:arch}
Different backends have their own instruction set and register set. In LLVM, this information is managed by TD files. TableGen takes the TD files and applies them to the backend so that this information is available at the code generation.

The TD files consist of information regarding different types of registers, classes information to which the register belongs, aliasing information, sub-registers details, and many more.
We have implemented a pass that captures information from the TD files and generates the configuration files which are used in different components. In the pass, we can configure the classes of the registers which we want to support, mapping between the color and the registers and overlapping information of the subregisters.

These configuration files are used in the capture interference graph component to get the register id, color, register classes and other information. In the ML model inference, overlapping information, register color mapping configuration is used. Similarly in the code generation, color register mapping is used to get the color corresponding to the register.

\section{Correctness Results}\label{sec:mlra:correct}
To check the correctness of work framework, we have chosen GCC Torture benchmark. It is also part of llvm-testsuite. GCC Torture is a stress testing benchmark. For the experimentation, we have selected 1483 input files. We have selected two architectures X86 and AArch64 and successfully generated the code followed by successful execution.

For AArch64, we used QEMU\cite{qemu} to run the generated code on X86 machine. QEMU is a simulator or virtualization software that provide the libraries needed by assembly code of different architecture and executing on others. See the Tab.~\ref{tab:mlra:correction}.

% Please add the following required packages to your document preamble:

\begin{table}[h]
\begin{tabular}{llllll}
\hline
\textbf{Architecture} & \textbf{\begin{tabular}[c]{@{}l@{}}Program \\ files\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Base \\ compilable files\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Assembly\\ files generated\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Binary \\ files generated\end{tabular}} & \textbf{\begin{tabular}[c]{@{}l@{}}Successful \\ execution\end{tabular}} \\ \hline
\textbf{X86} & 1483 & 1483 & 1483 & 1483 & 1483 \\ \hline
\textbf{\begin{tabular}[c]{@{}l@{}}AArch64\\ (QEMU)\end{tabular}} & 1483 & 1479 & 1479 & 1479 & 1479 \\ \hline
\end{tabular}
\centering
\caption{Successful execution}
\label{tab:mlra:correction}
\end{table}

The errors undertaken will evaluating the framework are code generation error, linking error, runtime error, and semantic errors. A lot of iteration went into fixing these errors. We were able to achieve zero errors. See the Tab.~\ref{tab:mlra:error}.

\begin{table}[h]
\begin{tabular}{lllll}
\hline
\textbf{Architecture} & \textbf{\begin{tabular}[c]{@{}l@{}}Code \\ generation error\end{tabular}} & \textbf{Linking error} & \textbf{\begin{tabular}[c]{@{}l@{}}Runtime\\ error\end{tabular}} & \textbf{Semantic error} \\ \hline
\textbf{X86} & 0 & 0 & 0 & 0 \\ \hline
\textbf{AArch64} & 0 & 0 & 0 & 0 \\ \hline
\end{tabular}
\centering
\caption{Errors or Failures}
\label{tab:mlra:error}
\end{table}

\section{MLRA in Action}\label{sec:mlra:mlra_action}
Here we want to show how the framework can be used to register prediction using a naive reinforcement learning model on X86 architecture. We have taken 50K interference graphs from the SPEC 2017\cite{spec17-Bucek:2018:SCN:3185768.3185771} benchmarks. Embedding are generated using the IR2Vec on machine instructions representation(MIR). We are using reinforcement learning as the ML model interface and the objective of the model is to minimize the spill cost. The spill cost is the weight associated with each variable. Higher the value of spill cost more costly is load-store memory operations. See Fig.~\ref{fig:mlra-fw}

\begin{figure}[t]
    \centering
    \includegraphics[scale=0.45]{figures/chapter-5/mlra_action.png}
    \caption{Framework configurations}
     \label{fig:mlra-fw}
\end{figure}

\subsection{Reinforcement Learning (RL) Model}
Reinforcement learning is the machine learning branch where datapoints do not have any label associated to them. During the training, a score is given to the model(agent) depending upon the intermediate and final action taken. The score is known as the reward which depend upon the action taken given an environment state. After applying action on the environment, its state or may not change. This change in the state would be good or bad. If the change is good, then a positive value is return as reward else negative value is returned as reward. The agent receive the reward from the environment and use this knowledge for future decision.

We have implemented a policy using the DQN algorithm in that the agent predicts which node should be assigned the register first.  While assigning the register, the policy is constraint by the interference rule. The objective of the model is to minimize the spillcost.

\subsection{Results}

Run the inference with trained model and showed our results on PolyBench and MiBench\cite{mibench}.
\subsubsection{PolyBench}
It is a Loop benchmark which have different computational intensive programs and a standard benchmark suites for polyhedral optimization. See the Fig.~\ref{fig:mlra-polybench} and Tab.~\ref{tab:mlra:polybench} for the performance comparison

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/polybench.png}
    \caption{PolyBench - Runtime}
     \label{fig:mlra-polybench}
\end{figure}

\begin{table}[h!]
\begin{tabular}{|l|l|l|l|l|}
\hline
 & \textbf{Greedy} & \textbf{Greedy ML} & \textbf{Basic} & \textbf{Basic ML} \\ \hline
GeoMean & \multicolumn{1}{r|}{9.306} & \multicolumn{1}{r|}{9.050} & \multicolumn{1}{r|}{9.259} & \multicolumn{1}{r|}{8.891} \\ \hline
\end{tabular}
\centering
\caption{PolyBench - Runtime GeoMean}
\label{tab:mlra:polybench}
\end{table}

\subsubsection{MiBench}
This Benchmark is a micro-kernel benchmarks which are suitable for embedded devices.  See the Fig.~\ref{fig:mlra-mibench} and Tab.~\ref{tab:mlra:mibench} for the performance comparison

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.4]{figures/chapter-5/MiBench.png}
    \caption{Runtime - MiBench}
     \label{fig:mlra-mibench}
\end{figure}

\begin{table}[h!]
\begin{tabular}{|l|l|l|l|l|}
\hline
 & \textbf{Greedy} & \textbf{Greedy ML} & \textbf{Basic} & \textbf{Basic ML} \\ \hline
GeoMean & \multicolumn{1}{r|}{0.0227} & \multicolumn{1}{r|}{0.0220} & \multicolumn{1}{r|}{0.0242} & \multicolumn{1}{r|}{0.0237} \\ \hline
\end{tabular}
\centering
\caption{MiBench - Runtime GeoMean}
\label{tab:mlra:mibench}
\end{table}

\section{Conclusion}\label{sec:mlra:conclusion}
We have successfully developed a framework for register allocation which generates semantically correct code, which supports multiple architectures, and which can communicate between LLVM (in C++) and ML model (in Python).

This framework can also be used to support the ML based optimizations in LLVM for other optimization problems, in which such kind of information to-from is required.


\subsubsection*{Singular Value Decomposition\hfill \hyperlink{linearAlgebraFunctions}{(up)}\hypertarget{singularValueDecompostion}{}} \addcontentsline{toc}{subsubsection}{Singular Value Decomposition}
\begin{table}[H]
\caption{Singular Value Decomposition}
\label{tab:singularValueDecompostion}
\begin{center}
\begin{tabular}{|l|l|}
\multicolumn{2}{c}{\hyperlink{svdFunc}{\rmfamily \bfseries SVD Function Set}}\\
\hline\Ts
svd & Matrix Singular Value Decomposition\Bs\\
svd\_create & Create Singular Value Decomposition Object \Bs\\
svd\_destroy & Destroy SVD Object\Bs\\
svd\_getattr & SVD Get Attributes\Bs\\
svd\_produ & Product with U from SV Decomposition\Bs\\
svdprodv & Product with V from SV Decomposition\Bs\\
svdmatu & Return with U from SV Decomposition\Bs\\
svdmatv & Return with V from SV Decomposition\Bs\\
\hline\end{tabular}
\end{center}
%\label{default}
\end{table}%


% vim: set spell spelllang=en
\documentclass[10pt]{article}
\usepackage[pdftex]{graphicx}
\usepackage{amssymb, amsmath}
\usepackage{url}

\usepackage{fullpage}
\setlength{\parindent}{0pt}
\setlength{\parskip}{\baselineskip}

\title{Implementation of the Density Evolution method with
Fokker-Planck for L-NLIF Model in Python}
\author{Valentin Haenel \\
valentin.haenel@gmx.de \\
BCCN Berlin}

\begin{document}

\maketitle

\section{Introduction}

This report is a description of the work done by Valentin Haenel
for a Lab Rotation within the Bernstein Center for Computational
Neuroscience in Berlin. The rotation was completed under the
supervision of  Gabriel and Bartosz Telenczuk.

Throughout the project I have implemented a variant of the Leaky
Integrate and Fire Neuron (LIF), and a method to fit the parameters of this
neuron using a model of the input and extracellularly recorded spikes.
Primarily this involved solving a partial differential equation and constructing
a maximum likelihood objective function.

This project is motivated by the fact that recording intracellularly from awake
behaving animals is usually not possible, and extracellularly recorded spike
trains are much more common in this setting. The model implemented allows us to
gain at least some partial insight into the subthreshold voltage of such
neurons. A further motivation comes from the fact, that algorithms described are
not publicly available in the form of an easy to use software package.

\section{Model}
The original model is a L-NLIF model, which consists of a Linear
(L) Filter, followed by a probabilistic or Noisy (N) form of Leaky
Integrate and Fire spike generation (LIF) (Paninski et
al.\cite{PaninskiPillowSimoncelli}). For sake of completeness the
model is briefly described here.

The evolution of the Voltage $V$ is given by:
\begin{equation}
    dV = (-g(V(t) -V_{leak}) +I_{stim}(t) + I_{hist}(t)) dt + W_{t}
\end{equation}
Where $g$ is the leak conductance $V_{leak}$ is the leak reversal
potential $I_{stim}$ is the convolution of linear filter with the
input signal, $I_{hist}$ is the spike current history and $W_t$ is
standard Gaussian white noise. More precisely:
\begin{equation}
    I_{stim}(t) = \vec{k} \cdot  \vec{x}(t)
\end{equation}
\begin{equation}
    I_{hist}(t) = \sum_{j=0}^{i-1}h(t-t_j)
\end{equation}
Where $k$ is a linear filter and $h$ is a fixed postspike current waveform.

\section{Algorithms}

\subsection{PDE Solver}

\subsubsection{The Problem}
Let
\begin{equation}
    P(V,t) \equiv P(V(t) \cap  V(s) < V_{th} \forall s < t)
\end{equation}
This refers to the probability of $V(t)$, the membrane potential,
being less than $V_{th}$ the firing threshold until time $t$.

As stated in \cite{PaninskiPillowSimoncelli} the method of density
evolution requires us to solve the
following Fokker-Planck drift diffusion equation numerically.
\begin{equation}
    \frac{\partial P(V,t)}{\partial t} =
    \frac{\sigma^2}{2} \frac{\partial^2 P(V,t) } {\partial V^2} +
    g\frac{\partial[(V-V_{rest})P(V,t)]}{\partial V}
    \label{fokkerplanck}
\end{equation}

We will be solving the evolution of the probability density for a
given inter-spike interval (ISI). And therefore we will constrain the range
of $V$ and $t$.
\begin{equation}
    P(V_{th},t) = 0
    \label{boundarytop}
\end{equation}
\begin{equation}
    P(V,0) = \delta(V-V_{reset})
    \label{boundaryleft}
\end{equation}

Condition (\ref{boundarytop}) ensures that the probability of obtaining a spike
during an ISI is zero. Condition (\ref{boundaryleft}) ensures
that at the beginning of an ISI, i.e. at the time of the last spike,
the probability of the neuron being at reset potential is exactly one.
Thereby we have obtained the constraints for the top and the left hand
side of our solution grid, however, when solving numerically we also
need a lower bound on the voltage $V_{lb}$, so this becomes an
additional boundary condition, for the  bottom:
\begin{equation}
    P(V_{lb},t) = 0
    \label{boundarybottom}
\end{equation}

Ideally $V_{lb} = -\infty $.

$V_{rest}$ is defined as the stationary point of the noiseless
subthreshold dynamics:
\begin{equation}
    V_{rest}(t) = V_{leak} + \frac{1}{g}(\vec{k} \cdot \vec{x}(t) +
    \sum_{j=0}^{i-1}h(t-t_j))
\end{equation}

% This was just wasting space
%
%We can rewrite (\ref{fokkerplanck}) as:
%\begin{equation}
%    \frac{\sigma^2}{2} \frac{\partial^2 P(V,t) } {\partial V^2} +
%    g(V-V_{rest})\frac{\partial P(V,t)}{\partial V} +
%    gP(V,t) -
%    \frac{\partial P(V,t)}{\partial t} =
%    0
%\end{equation}
%Since we know that:
%\begin{equation}
%    \frac{\partial[(V-V_{rest})P(V,t)]}{\partial V} =
%    \frac{\partial (V-V_{rest})}{\partial V}  P +
%    \frac{\partial P}{\partial V} (V-V_{rest})
%\end{equation}
%and
%\begin{equation}
%    \frac{\partial (V-V_{rest})}{\partial V} = 1
%\end{equation}

\subsubsection{Finite Differencing}

In order to solve (\ref{fokkerplanck}) numerically we discretized time
and potential. We adopt the notation that Potential is discretized
into $W$ intervals of length $w$ and indexed
by $\nu= 0,1, \dots W $.  Time is discretized  into $U$ intervals of
length $u$ and indexed by: $\tau= 0,1, \dots U $.
We adopt the notation: $P_{\nu,\tau} = P(\nu w,\tau u)$

A forward finite differencing scheme would like:
\begin{eqnarray}
    \hat{P}(v,t) =& P_{\nu,\tau}\\
%
\frac{\partial \hat{P}(v,t)}{\partial t} =& \frac{P_{\nu,\tau +1 } -
    P_{\nu,\tau}}{u} \\
%
    \frac{\partial \hat{P}(v,t)}{\partial V} =&
    \frac{P_{\nu +1,\tau } -
    P_{\nu - 1,\tau } }
    {2w} \\
%
    \frac{\partial^2 \hat{P}(v,t)}{\partial V^2} =&
    \frac{P_{\nu+1,\tau} - 2 P_{\nu,\tau} + P_{\nu-1,\tau}}
    {w^2}
\end{eqnarray}

However as suggested in \cite{PaninskiHaithSzirtes} we will use the
computationally more stable Crank-Nicolson scheme \cite{press}.
We rewrite the derivatives using the new scheme which is centered around $t +
u/2$.

\begin{eqnarray}
    P_{CN}(v,t) =& \frac{P_{\nu,\tau} + P_{\nu,\tau + 1}}{2} \\
    %
    \frac{\partial P_{CN}(v,t)}{\partial t} =& \frac{P_{\nu,\tau +1 } -
    P_{\nu,\tau}}{u} \\
    %
    \frac{\partial P_{CN}(v,t)}{\partial V} =&
    \frac{P_{\nu +1,\tau } + P_{\nu +1,\tau +1 } -
    P_{\nu - 1,\tau } - P_{\nu -1,\tau +1}}
    {4w} \\
    %
    \frac{\partial^2 P_{CN}(v,t)}{\partial V^2} =&
    \frac{P_{\nu+1,\tau} - 2 P_{\nu,\tau} + P_{\nu-1,\tau} +
    P_{\nu+1,\tau+1} - 2 P_{\nu,\tau+1} + P_{\nu-1,\tau+1}}
    {2w^2}
\end{eqnarray}

\subsubsection{Tridiagonal Equations}

If we now let:
\begin{align*}
a &= \frac{\sigma^2}{2} \\
b &= g(V - V_{rest}) \\
c &= g \\
\end{align*}
and multiply throughout with $4w^2u$
we may rearrange all the $P_{*,\tau+1} $ terms on the left hand side:
\begin{multline}
    \overbrace{-(2au+bwu)}^{A_\nu} P_{\nu+1,\tau+1} +
    \overbrace{(4au - 2cw^2u + 4w^2)}^{B_\nu} P_{\nu,\tau+1}
    \overbrace{-(2au-bwu)}^{C_\nu} P_{\nu-1,\tau+1}
    =  \\
    \underbrace{(2au+bwu) P_{\nu+1,\tau} +
    (-4au +2cw^2u + 4w^2) P_{\nu,\tau} +
    (2au-bwu) P_{\nu-1,\tau}}_{D_{\nu}}
    \label{rearrange}
\end{multline}
For each $ \nu = 1 , \dots , W-1 $

Using (\ref{rearrange}) and incorporating the boundary conditions
\ref{boundarytop} and \ref{boundaryleft} we have obtain a complete system
of $W-1$ linear equations which we may now rewrite in the following
tridiagonal matrix notation.

\begin{equation}
\underbrace{
\begin{pmatrix}
    & B_1   & A_1    & 0      & \cdots   \\
    & C_2   & B_2    & A_2    & 0        \\
    &       & \ddots & \ddots & \ddots   \\
    & \cdots & 0      & C_{W-1}& B_{W-1} \\
\end{pmatrix}}_{\Lambda}
\underbrace{
\begin{pmatrix}
    P_{1,\tau+1}   \\
    P_{2,\tau+1}   \\
    \vdots         \\
    P_{W-1,\tau+1} \\
\end{pmatrix}}_{\chi}
=
\underbrace{
\begin{pmatrix}
    D_1     \\
    D_2     \\
    \vdots  \\
    D_{W-1} \\
\end{pmatrix}}_{\beta}
\end{equation}

We can then use a tridiagonal matrix algorithm to solve $\Lambda \chi
= \beta$. This allows us to iteratively obtain the density evolution $P(V,t)$ of the
membrane potential for a given ISI. Where the initial $\beta$ for an ISI is given by
(\ref{boundaryleft}) and the halting criterion is the time of the next
spike. So for the $i$th spike we obtain values for $V$ in the range
$[V_{lb}, V_{th}]$ and for $t$ in the range $[t_{i-1},t_{i}]$. An
example can be seen in figure \ref{P_vt}.

\subsection{First Passage Time}

The maximum likelihood optimizer proposed in
\cite{PaninskiPillowSimoncelli} however uses the first passage time
density at a given time point which is defined as:

\begin{equation}
    \mathrm{FPT}(t)  = -\frac{\partial}{\partial t} \int P(V,t)dV
    \label{eq:fpt}
\end{equation}

and this can be easily computed from a $P(V,t)$ matrix by a finite differences
/ sums scheme.

\subsection{Maximum Likelihood Estimator}

Putting together the PDE Solver and method for computing the First
Passage Time
The final likelihood is then simply

\begin{equation}
    L_{\vec{x},t_{i}} = \prod_{i} \mathrm{FPT}(t_{i})
\end{equation}

This is the product of the first passage time at the times of
spiking for all spikes.

The pseudocode for the objective function can be described as follows:

{\tt
for each spike interval do:
\begin{enumerate}
    \item compute $P(V,t)$ as matrix
    \item compute FPT$(t)$ as vector
\end{enumerate}
return the product of the last scalars in the FPT vectors.}

This objective function could be presented to a standard numerical
optimizer, for example the Nelder-Mead Downhill simplex \cite{press},
this needs no gradients and is fairly robust but slow.

\subsection{Monte Carlo Method}

To double check the results of the PDE Solver and the resulting First
Passage Time, we additionally implemented some basic Monte Carlo
simulations. The basic idea is to simulate a few thousand traces of the neuron
up to the first spike, with noise. The resulting potential traces, and spike
times were used to obtain both $P(V,t)$ and $fpt$. The density evolution is
compared qualitatively to the density evolution technique as can bee seen in:
\ref{P_vt}. The FPT is compared using a Kolmogorov-Smirnov (K-S)
test\cite{press}.

\section{Implementation}

This section describes the concrete implementation

\subsection{Availability and Design}

The algorithm has been implemented in Python, and the full source code
has been made available within a revision control system under a free
software license at: http://github.com/esc/molif/tree/master. Given
that the Python Language was named after the British Comedians Monty
Python we decided to call the software {\it Meaning Of LIF},
(abbreviated to {\it molif}) which is a reference to the Monty Python
movie {\it Meaning of Life}.

The software is structured into a series of modules contained within
the package {\tt molif}

\begin{itemize}
    \item {\tt model} implementation of the neuron model
    \item {\tt density} implementation of the PDE solver
    \item {\tt likelihood} connects the optimizer and the PDE solver
    \item {\tt montecarlo} implementation of Monte Carlo simulator
    \item {\tt plotting} methods used to generate plots
    \item {\tt util} methods to determine timing
\end{itemize}

The software uses the additional packages numpy, scipy for vector
based and scientific computing, and pylab/matplotlib for plotting the
results. We used the modules {\it scipy.sparse} and {\it
scipy.linsolve} for computing the solution to tridiagonal equations, and the
method {\it optimize.fmin}, which implements the Nelder-Mead Downhill
Simplex, for minimizing the negative likelihood.

\subsection{Limitations and Future Work}

The PDE solver uses sparse matrix algorithms to store and solve the
tridiagonal matrix. Ideally we would like to use the routine $TRIDAG$
described in \cite{press}, however an open source implementation in
python was not available, although feasible. This improvement could
possibly speed up the computation of $P(V,t)$.

There is a serious bug, whereby the value of sigma,
must be within a specific range of $0.1 - 0.05$, otherwise strange
oscillatory instabilities occur. Among the possible causes, are a
numerical instability in the finite differencing scheme, a bug in our
code, a bug in the solver for the linear equations or the
discretization of $V$ and $t$ being to large.

Currently the gradient of the objective function isn't implemented,
and any gradient based optimizer would need to compute their
approximation from the function values.

We used a Kolmogorov-Smirnov (K-S) test to check that the first
passage time computed by the PDE-solver and the Monte Carlo method
come from the same distribution. However the test currently returns a
negative result, although at first glance the distributions look very
similar.

Lastly we should mention that the objective function isn't fully
operational yet. A simple test is executed as follows: first we
generate some spikes with given parameters, and then we attempt to fit
parameters to these spikes. If the code operates correctly the
optimizer should not deviate away from the initial parameters vector.
However currently this is not the case and the optimizer finds a
minimum at a different position in parameter space.

\subsection{Results}

This section presents some plots.

Figure \ref{P_vt} shows a comparison of PDE Solver (top) and Monte Carlo
(bottom) method when computing the density evolution. As expected the two plots
look the same, and corresponds very nicely to what we would expect from the
iterative solution to a Fokker-Planck equation. The drift term is responsible for
the shift of the mean, and the diffusion is responsible for the flattening.

Figure \ref{fpt} shows the comparison of the PDE Solver and Monte
Carlo method when computing the first passage time. Unfortunately a
K-S test allows us to reject the hypothesis that these two come from
the same distribution, even though they look very much alike at a
first glance.

\begin{figure}[htp]
\centering
\includegraphics[scale=0.60]{P_vt}
\caption{Density Evolution}
\label{P_vt}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[scale=0.60]{fpt.png}
\caption{First Passage Time Comparison}
\label{fpt}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[scale=0.60]{traces.png}
\caption{Voltage Traces for 5000 Monte Carlo simulations }
\label{traces}
\end{figure}

\section{Conclusion}

Not all of the initial Project Goals were reached to my full
satisfaction. We did implement an Integrate and Fire
neuron, and a method to fit the parameters, however we did not get as
far as actually fitting real experimental data. None the less
useful code has been produced and solid numerical techniques (Solving
Partial Differential Equations) were
learnt, and I consider the lab rotation to be successful.

\bibliography{documentation}{ \bibliographystyle{abbrv} }

\end{document}