\chapter{Generalized Adaptive Intelligent Binning of Multiway Data}

\begin{quote}
{\it
  The art of doing mathematics consists in finding that special case which
  contains all the germs of generality.}
\\\\
 -- David Hilbert
\end{quote}

\section{Introduction}

\begin{figure}[hb!]
\includegraphics[width=6in]{figs/gaibin/01-scheme.png}
\caption
      [Generalization of Adaptive Intelligent Binning.]{
  {\bf Generalization of Adaptive Intelligent Binning.}
  \\
  ({\bf A}) In the one-dimensional case, the bin containing regions 1 and 2
  is optimally subdivided ({\it asterisk}) when the sum of the objective values
  in regions 1 and 2 is maximal and greater than the original bin's objective
  value. ({\bf B}) In the $D$-dimensional case, there are now $D$ possible
  dimensions along which an optimal subdivision may exist. The optimal
  subdivision along the \hnmr{} dimension ({\it triangle}) occurs when the sum
  of the objective values in regions 3+6 and 4+5 is maximal and greater than
  that of the original bin. Similarly, the optimal subdivision along the
  \cnmr{} dimension ({\it circle}) occurs when the sum of the objective values
  in regions 3+4 and 5+6 is maximal and above the original bin objective. A
  comparison between all possible optimal subdivisions along all dimensions
  yields the best possible subdivision (\cnmr{}, {\it circle}).
}
\label{figure.8.1}
\end{figure}

\begin{doublespace}
By and large, the phase ``NMR metabolic fingerprinting'' implies the use of
one-dimensional (1D) \hnmr{} NMR spectroscopic methods, due in no small part
to the ease and speed of 1D data collection and the large natural abundance
of NMR-active protons found in metabolomics samples
\cite{lindon:cmr2000,worley:cmb2013}. Before processed spectra are
submitted to PCA or PLS for modeling, they are often subdivided into bins
to simplify multivariate analyses. Spectral binning, introduced and described
in detail in \hyperlink{chapter.3}{Chapter 3}, reduces the dimensionality of
a data matrix and masks chemical shift variability between samples at the
expense of decreased model interpretability: any given bin in a 1D \hnmr{} NMR
spectral dataset may contain several overlapped signals from multiple distinct
metabolites \cite{aberg:abc2009}. Thus, without utilizing
computationally intensive methods of deconvolution to tease apart signal
contributions from individual metabolites
\cite{astle:jasa2012,zheng:binf2011}, the resulting fingerprint from a
binned 1D dataset is usually limited to high-level inference about metabolic
trends.
\\\\
By leveraging the connectivities between \hnmr{} and \cnmr{} nuclei in
metabolites, two-dimensional (2D) heteronuclear NMR methods reduce spectral
overlap by spreading \hnmr{} information over a second (\cnmr) chemical shift
dimension \cite{mandal:cmr2004}. Heteronuclear single quantum coherence (HSQC)
experiments are commonly performed in NMR metabolic profiling studies, and
provide an NMR singlet or multiplet for each directly bonded \hcnmr{} pair
in the sample. Developments in NMR hardware and acquisition techniques have
brought natural abundance \hcnmr{} HSQC experiment times down to values
compatible with high-throughput metabolic fingerprinting studies
\cite{motta:anchem2010,rai:anchem2012}. However, multivariate analysis
of 2D NMR datasets is still a nontrivial undertaking that requires either
vectorization \cite{hedenstrom:cils2008}, which breaks the inherent
structure of the data, or the use of multilinear factorizations
\cite{lu:ieee2009,lu:pr2011}, which are more computationally intensive
and difficult to cross-validate.
\\\\
Spectral binning is another potential means of preparing 2D NMR datasets for
multivariate analysis that holds several advantages over binning 1D spectra.
First, multiple integration of bins maps each spectrum to an observation
vector regardless of its original dimensionality, allowing bilinear PCA and
PLS algorithms to be used without concern for loss of the inherent structure
of the data. Second, binning of 2D spectral data yields more well-conditioned
data matrices than simple vectorization. Finally, because signals are better
resolved in 2D spectra, each bin contains substantially fewer signals from
distinct metabolites. Multiple different algorithms have been developed to
bin 1D NMR data \cite{
  anderson:metab2008,
  anderson:metab2011,
  davis:cils2007,
  demeyer:anchem2008,
  sousa:cils2013}, and the use of uniform binning on 2D NMR data has also been
reported \cite{van:jpr2008}. However, at the time of this writing, no methods
exist to \emph{intelligently} bin multidimensional data for use in multivariate
analyses. This motivated the development of a generalization of Adaptive
Intelligent (AI) binning \cite{demeyer:anchem2008} to spectral data
of any dimensionality, called Generalized Adaptive Intelligent (GAI) binning
(\figref{8.1}{Figure 8.1}).
\end{doublespace}

\section{Theory}

\subsection{AI-binning}

\begin{doublespace}
Generalized AI-binning (GAI-binning) is a logical extension of AI-binning to
two or more dimensions. In the AI algorithm (\figref{8.1}{Figure 8.1A}), bins
are recursively subdivided until a stopping criterion or minimum bin width is
reached \cite{demeyer:anchem2008}. For a 1D dataset containing $N$ spectra,
the following objective function is used to assess the quality of each bin:
\begin{equation}
V_b = \frac{1}{N}
  \sum_{n=1}^N \left[
    (max_{n,b} - I_{n,b,1})
    (max_{n,b} - I_{n,b,end}) \right]^\frac{R}{2}
\end{equation}
where $max_{n,b}$ is the maximum intensity inside the bin $b$ in spectrum $n$,
and $I_{n,b,1}$ and $I_{n,b,end}$ are the bin edge intensities. The exponent
$R$ in the AI objective function is referred to as a ``resolution parameter'',
which offers a means of tuning the binning result based on signal-to-noise and
peak resolution of a dataset. The replacement of $R$ with $\frac{R}{2}$ in the
exponent of equation 6.1, enables a slightly modified interpretation of each
summed term in the AI objective function as a relaxed form of a geometric mean
of the differences between the bin edge intensities and the maximum bin
intensity. At each subdivision step, new bin edges are chosen to maximize the
combined (summed) objective values of the two resulting bins over the objective
value of the original bin. If no bin subdivision exists with a combined
objective value greater than that of the original bin, recursive subdivision
within that bin is terminated, and the AI algorithm terminates once all bins
may no longer be subdivided.
\end{doublespace}

\subsection{GAI-binning}

\begin{doublespace}
In two or more dimensions, the set of bin boundary points expands to include
all points that lie on the edges (or faces, hyperfaces, etc.) of the bin.
By denoting the set of all edge points in bin $b$ as $E_b$, a new objective
function may be constructed:
\begin{equation}
V_b = \frac{1}{N}
  \sum_{n=1}^N \left[
    \prod_{e \in E_b} (max_{n,b} - I_e)
  \right]^\frac{R}{||E_b||}
\end{equation}

Thus, the GAI algorithm computes the ``relaxed'' geometric mean of the
differences between the bin maximum and all points on the boundary. In the
case of one-dimensional data, it is apparent that equation 6.2 reduces to
equation 6.1, and GAI-binning operates identically to AI-binning. As
dimensionality increases, the risk of floating-point overflow or underflow
increases due to the larger bin edge set $E_b$. To avoid this, the following
``log-objective'' may be used in lieu of equation 6.2:
\begin{equation}
V_{b,ln} = \frac{R}{N ||E_b||}
  \sum_{n=1}^N \sum_{e \in E_b}
    \ln(max_{n,b} - I_e)
\end{equation}

Like AI-binning, GAI-binning initializes a bin around the entire dataset and
proceeds to recursively subdivide each bin until a minimum bin size is reached
or no bin may be divided to yield an increase in the objective value. Because
the number of ways to subdivide each bin increases with dimensionality, all
possible dimensions are tested, and the new bin boundary that maximizes the
objective over all possible subdivision dimensions is selected
(\figref{8.1}{Figure 8.1B}). Therefore, the GAI algorithm may be considered
a form of binary space partitioning (BSP) which limits its partition
hyperplanes to lying orthogonally to the basis vectors of the coordinate
system \cite{deberg2000}.
\end{doublespace}

\subsection{Noise Bin Elimination}

\begin{doublespace}
It is important that noise bins be removed from the data matrix prior to
multivariate analysis, as their presence is known to negatively impact the
interpretability and reliability of multivariate models
\cite{halouska:jmr2006,bro:anmeth2014}. Because the integration of a
noisy space of increasing dimensionality (i.e. double or triple
integration) results in a random variable having a similarly increasing
variance, the importance of noise removal is compounded in multidimensional
binning. Therefore, a noise bin removal step based on spectral intensity was
added to the GAI algorithm. A running mean and variance calculation was
performed to estimate the noise floor of each spectrum. The initial mean
$\mu_n$ and standard deviation $\sigma_n$ of the noise were computed using the
first 32 points on one edge of the spectrum, which were assumed to contain only
baseline noise. Every other data point was then classified as signal or noise
based on whether its intensity exceeded the current running noise floor,
$\mu_n + 3 \sigma_n$. Upon inclusion of a new noise data point, the mean and
standard deviation of the noise were appropriately updated. Once the estimated
noise floor was determined for each spectrum in the dataset, a threshold for
bin removal was computed as the median noise floor of all the spectra:
\begin{equation}
I_{th} = \mathrm{med}_n (\mu_n + k \sigma_n)
\end{equation}
where $k$ is a user-selectable parameter to adjust the noise threshold. Only
bins whose maximum intensity fell above $I_{th}$ were retained in the final
data matrix.
\end{doublespace}

\section{Materials and Methods}

\subsection{Human Liver Dataset}

\begin{doublespace}
Two independently collected \hcnmr{} HSQC NMR datasets from ongoing
metabolomics studies were used as test cases for the GAI-binning algorithm.
For the first dataset, twenty-four 1.0 mL samples of SK-Hep1 human liver cells
were provided for metabolic fingerprinting, half of which were treated with
50 $\mu$M tetrathiomolybdate (TTM). The cells were extracted into 80:20
methanol:water to collect the water-soluble metabolites, spun in a rotary
evaporator for two hours, lyophilized at $-50^\circ$C and 0.02 mBar for 24
hours, and finally redissolved in 600 $\mu$L of 50.0 mM phosphate buffer in
99.8\% D$_2$O (Isotec, St. Louis, MO) adjusted to pH 7.4. The redissolved,
pH-adjusted samples were then collected into NMR tubes.
\\\\
Experiments were collected on a Bruker Avance III HD 700 MHz spectrometer
equipped with a 5 mm inverse quadruple-resonance (\hnmr{}, \cnmr{}, \nnmr{},
\pnmr{}) cryoprobe with cooled \hnmr{} and \cnmr{} channels and a {\it z}-axis
gradient. A Bruker SampleJet and ICON-NMR were used to automate NMR data
collection. A 2D gradient-enhanced \hcnmr{} HSQC with improved sensitivity
\cite{palmer:jmr1991,kay:jacs1992} ({\it hsqcetgpsi}) was collected for
each sample. Spectra were collected with 4 scans and 16 dummy scans over a
uniform Nyquist grid of 512 and 64 complex points along the \hnmr{} and \cnmr{}
dimensions, respectively. Spectral windows were set to 3,285 $\pm$ 4,545 Hz
along \hnmr{} and 12,677 $\pm$ 14,620 Hz along \cnmr{}. All spectra were
collected at a sample temperature of 298.0 K.
\end{doublespace}

\begin{figure}[hb!]
\includegraphics[width=6in]{figs/gaibin/02-hsqc-liver.png}
\caption
      [Binned Liver Dataset.]{
  {\bf Binned Liver Dataset.}
  \\
  Processed \hcnmr{} HSQC mean spectrum of the liver data tensor, with
  overlaid uniform ({\bf A}) and GAI ({\bf B}) bin boundaries.
}
\label{figure.8.2}
\end{figure}

\subsection{Mouse Embryonic Fibroblast Dataset}

\begin{doublespace}
A second set of samples from kinase suppressor of Ras 1 (KSR1) knockout mouse
embryonic fibroblast (MEF) cells was also provided to generate a test
\hcnmr{} HSQC dataset for GAI-binning. For this second dataset, ten cell
samples from $ksr^{-/-}$ MEFs and ten samples from KSR1-rescued $ksr^{-/-}$
MEFs were used to produce metabolite extracts. The cells were washed, extracted
into 80:20 methanol:water, spun in a rotary evaporator, lyophilized and
redissolved according to the procedures used to extract metabolites from the
liver cell samples.
\\\\
Experiments were collected on a Bruker Avance DRX 500 MHz spectrometer equipped
with a 5 mm inverse triple-resonance (\hnmr{}, \cnmr{}, \nnmr{}) cryoprobe with
a {\it z}-axis gradient. A Bruker BACS-120 sample changer and ICON-NMR software
were used to automate data collection. A 2D gradient-enhanced \hcnmr{}
HSQC ({\it hsqcetgp}) was collected for each sample. Spectra were collected
with 128 scans and 16 dummy scans over a uniform grid of 1024 and 32 complex
points along the \hnmr{} and \cnmr{} dimensions, respectively. Spectral windows
were set to 2,359 $\pm$ 2,367 Hz along \hnmr{} and 8,174 $\pm$ 8,803 Hz along
\cnmr{}. All spectra were collected at a sample temperature of 293 K.
\end{doublespace}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/gaibin/03-hsqc-mef.png}
\caption
      [Binned Fibroblast Dataset.]{
  {\bf Binned Fibroblast Dataset.}
  \\
  Processed \hcnmr{} HSQC mean spectrum of the MEF data tensor, with
  overlaid uniform ({\bf A}) and GAI ({\bf B}) bin boundaries.
}
\label{figure.8.3}
\end{figure}

\subsection{NMR Processing and Multivariate Analysis}

\begin{doublespace}
All processing, treatment and statistical modeling were performed in GNU
Octave 3.6 \cite{eaton2008} using routines currently available in the MVAPACK
toolbox for NMR chemometrics \cite{worley:acscb2014}, discussed in
\hyperlink{chapter.4}{Chapter 4}. The 2D raw serial files were loaded
\cite{delaglio:jbnmr1995}, apodized with a squared-sine window,
zero-filled once along \hnmr{} and twice along \cnmr{}, and
Fourier-transformed. Spectra from the liver cell extracts were manually
phase-corrected and cropped (1.0 -- 6.6 ppm along \hnmr{}; 16 -- 112 ppm
along \cnmr{}), and spectra from the MEF extracts were similarly phase
corrected and cropped (1.25 -- 6.2 ppm along \hnmr{}; 8 -- 102 ppm
along \cnmr{}). Both uniform and GAI-binning were performed on each data tensor
using minimum \hnmr{} and \cnmr{} bin widths of 0.025 and 2.5
ppm, respectively, and a GAI resolution parameter of 0.1. Binned regions
identified to be less intense than three times the standard deviation of the
spectral noise ($k = 3$) were removed after binning. The mean spectra of the
entire processed liver and MEF datasets, superimposed with bins identified by
both uniform and GAI-binning, are shown in \figref{8.2}{Figure 8.2} and
\figref{8.3}{Figure 8.3}.
\\\\
The applicability of GAI-binning to bilinear factorizations was demonstrated
by modeling the data tensors using both PCA and OPLS-DA. For PCA modeling of
the data, the spectral regions identified by each binning method were doubly
integrated. Scores and loadings were then calculated using the Nonlinear
Iterative Partial Least Squares (NIPALS) algorithm
\cite{jolliffe2002}. Internal leave-one-out cross-validation (LOOCV)
of each computed PCA model was performed to yield model fit (\rsqx{}) and
predictive ability (\qsq{}) statistics
\cite{krzanowski:biom1987,eshghi:cils2014}. For OPLS-DA, spectral data
points within the identified bins were vectorized row-wise into a data matrix
as previously described \cite{hedenstrom:cils2008}. During
vectorization, all data points within each binned region are stacked into an
observation vector, and data points not within bins are excluded. The use of
vectorization prior to supervised modeling facilitates the creation of
backscaled pseudospectral OPLS loadings, which hold greater ease of
interpretation over binned loadings \cite{wiklund:anchem2008}.
Modeling by an OSC-filtered NIPALS algorithm
\cite{trygg:jchemo2002} and 100 rounds of seven-fold Monte Carlo
cross-validation (MCCV) \cite{xu:jchemo2004} were performed to
compute data fit (\rsqx{}), response fit (\rsqy{}) and model predictive ability
(\qsq{}) statistics. The binned data matrices produced via double integration
were also subjected to OPLS-DA modeling in the same manner as the vectorized
data. All OPLS-DA models were further validated using CV-ANOVA
\cite{eriksson:jchemo2008} and 1,000 iterations of response
permutation testing \cite{westerhuis:metab2008a} to rigorously ensure
model reliability. Backscaled predictive OPLS loadings were computed from the
vectorized bins according to previously published works
\cite{cloarec:anchem2005a,hedenstrom:cils2008}. During backscaling,
OPLS loading vectors were scaled by the inverse of their original Pareto
scaling coefficients and then unstacked into a two-dimensional pseudospectrum
using bin information. Data points not included in the vectorized loadings were
set to zero in the backscaled pseudospectrum. All data matrices were normalized
using Probabilistic Quotients (PQ) \cite{dieterle:anchem2006} and then
Pareto scaled \cite{vandenberg:bmcg2006} prior to modeling.
\end{doublespace}

\section{Results and Discussion}

\begin{doublespace}
Processing of the liver extract spectra yielded a real data tensor of 24
\hcnmr{} HSQC spectra having 442$\times$149 points each, and processing of the
fibroblast spectra yielded a tensor of 17 spectra having 1,071$\times$172
real data points each. The observation counts ($N$), variable counts ($K$)
and PCA/OPLS cross-validation statistics (\rsq{}, \qsq{}) for each dataset and
variable reduction method are summarized in Table 8.1. Further validation
results from the OPLS models, all of which indicate varying degrees of high
model reliability, are also summarized in Table 8.2. Through examination of the
variable counts within Table 8.1, it is readily apparent that GAI-binning is
dramatically more effective than uniform binning at discriminating between
signal and noise regions within spectral data. On average, GAI-binning
segmented each data tensor into less than half the number of bins produced by
uniform binning, and produced PCA models with markedly higher \rsqx{} and
\qsq{} statistics. Moreover, even with the greatly reduced variable counts
produced by GAI-binning relative to uniform binning, the OPLS \qsq{} statistics
between the two methods are statistically indistinguishable. In fact, the
variable counts resulting from GAI-binning these third-order tensors are
substantially lower than the few hundred variables typically produced by
binning {\it one}-dimensional spectra. Resulting scores from PCA modeling
of the GAI-binned liver data tensor are shown in \figref{8.4}{Figure 8.4}.
\end{doublespace}

\begin{table}[h!]
\caption{Data Matrices and PCA/OPLS Model Statistics.}
\begin{center}
\begin{tabular}{l l | l l l l l | l l l}
  \hline
              &         &
    \multicolumn{5}{c|}{{\bf Integration}} &
    \multicolumn{3}{c}{{\bf Vectorization}} \\
              &         &
    \multicolumn{3}{c}{PCA} &
    \multicolumn{2}{c|}{OPLS} & &
    \multicolumn{2}{c}{OPLS} \\
  \hline
              &         &
    $K$       & \rsqx{} & \qsq{} & \rsqy{} & \qsq{} &
    $K$                          & \rsqy{} & \qsq{} \\
  \hline
  {\bf Liver} & Unif.   &
    248       & 0.82    & 0.71   & 0.993   & 0.938 $\pm$ 0.002 &
    11,160                       & 0.993   & 0.929 $\pm$ 0.003 \\
  $N = 24$    & GAI     &
    113       & 0.89    & 0.75   & 0.991   & 0.928 $\pm$ 0.003 &
    10,474                       & 0.994   & 0.933 $\pm$ 0.003 \\
  \hline
  {\bf MEF}   & Unif.   &
    334       & 0.48    & 0.40   & 0.994   & 0.974 $\pm$ 0.004 &
    18,348                       & 0.994   & 0.963 $\pm$ 0.005 \\
  $N = 17$    & GAI     &
    93        & 0.71    & 0.56   & 0.994   & 0.973 $\pm$ 0.005 &
    18,789                       & 0.996   & 0.962 $\pm$ 0.006
\end{tabular}
\end{center}
\end{table}

\begin{table}[h!]
\caption{OPLS-DA Cross Validation $p$-values.}
\begin{center}
\begin{tabular}{l l | l l | l l}
  \hline
              &       &
    \multicolumn{2}{c|}{{\bf Integration}} &
    \multicolumn{2}{c}{{\bf Vectorization}} \\
              &       & Permutation & CV-ANOVA & Permutation & CV-ANOVA \\
  \hline
  {\bf Liver} & Unif. &
    $< 0.001$ & $3.24 \times 10^{-11}$ & $< 0.001$ & $4.70 \times 10^{-11}$ \\
  $N = 24$    & GAI   &
    $< 0.001$ & $3.34 \times 10^{-10}$ & $< 0.001$ & $9.74 \times 10^{-11}$ \\
  \hline
  {\bf MEF}   & Unif. &
    $< 0.001$ & $3.56 \times 10^{-10}$ & $< 0.001$ & $1.73 \times 10^{-9}$ \\
  $N = 24$    & GAI   &
    $< 0.001$ & $1.37 \times 10^{-9}$  & $< 0.001$ & $2.34 \times 10^{-9}$
\end{tabular}
\end{center}
\end{table}

\begin{SCfigure}
\includegraphics[width=3.25in]{figs/gaibin/04-scores.png}
\caption
      [PCA Scores of a GAI-binned Tensor.]{
  {\bf PCA Scores of a GAI-binned Tensor.}
  \\
  Principal component analysis scores resulting from modeling the GAI-binned
  \hcnmr{} HSQC liver data matrix, indicating a high degree of separation
  between experimental groups. Model \rsqx{} and \qsq{} were 0.68 and 0.64 for
  the first principal component ($t_1$) and 0.12 and 0.09 for the second
  ($t_2$). Class separations of this magnitude are readily achievable using
  data matrices generated by GAI-binning, due in large part to the low variable
  counts it generally produces.
}
\label{figure.8.4}
\end{SCfigure}

\begin{doublespace}
Backscaled predictive OPLS-DA loadings of the vectorized \hcnmr{} HSQC spectral
data tensors (\figref{8.5}{Figure 8.5}) lend further support for the use of
multidimensional binning in metabolic fingerprinting studies. Even when
vectorization is performed in place of integration to produce a data matrix,
binning offers an effective means of variable selection: only 10,474 of 65,858
variables (16\%) were retained when GAI-binning was used as a pre-filter prior
to modeling the liver data. A similar reduction was observed in the fibroblast
dataset, where GAI-binning retained 18,789 of 184,212 total variables for
a 90\% reduction in dimensionality. These substantially reduced variable
counts offered by binning translate to more well-conditioned bilinear
modeling problems. As the dimensionality of the input dataset is
increased further, the reductions in variable count afforded by
multidimensional binning are expected to become even
more dramatic. While the variable counts produced by vectorization of uniformly
binned data tensors are comparable to those from GAI-binning, it is critical to
recognize that the uniformly binned regions contain more noise data points than
their GAI-binned counterparts, and thus offer a less efficient dimensionality
reduction.
\\\\
Spectral regions produced by GAI-binning (\figref{8.2}{Figure 8.2})
demonstrate several important properties of the combined binning and noise
removal processes. Because $t_1$ noise and truncation artifacts yield
phase-incoherent negative spectral excursions after Fourier
transformation, ``unrelaxed'' GAI-binning ($R = 1$) tends to
preferentially subdivide near such regions, producing
elongated bins along the $F_1$ dimension. Decreasing the resolution parameter
from its maximum value shrinks these bins to contain only true signals. Thus,
an objective rule for determining an optimal resolution parameter during
binning is to decrease $R$ until all bins shrink to contain a minimal amount
of noise. Once an optimal resolution parameter has been identified, a suitable
noise threshold ($k$) must be determined such that all noise bins are removed
without loss of bins containing weak signals. However, once $R$ and $k$ have
been determined for a given set of experimental conditions, they may be applied
during GAI-binning to any data collected at later times under the same
conditions to achieve ideal results. The selections of resolution parameter
($R = 0.1$) and noise threshold ($k = 3$) made in this work were identified
according to the above criteria through a manual visual examination of the
binning results, but it is conceivable that objective metrics of these criteria
could be constructed that facilitate automated determination of these
parameters.
\\\\
Finally, like AI-binning, the execution time of GAI-binning scales
quadratically with the number of spectral data points, and scales approximately
linearly with both the number of spectral dimensions and the number of
observations. Typical run times for binning two-dimensional datasets range from
seconds to a few minutes, depending mostly on the data point count. Thus,
while zero-filling may be used to increase the digital resolution of data
being input into GAI-binning, it should be applied sparingly to avoid
unnecessarily long computation times during bin region determination.
\end{doublespace}

\begin{figure}[hb!]
\includegraphics[width=6in]{figs/gaibin/05-loadings.png}
\caption
      [Pseudospectral HSQC Loadings.]{
  {\bf Pseudospectral HSQC Loadings.}
  \\
  Backscaled full-resolution pseudospectral loadings from OPLS-DA modeling of
  the GAI-reduced ({\bf A}) liver and ({\bf B}) fibroblast \hcnmr{} HSQC data
  tensors. Positive and negative loadings are represented by red and blue
  contours, respectively.
}
\label{figure.8.5}
\end{figure}

\section{Conclusions}

\begin{doublespace}
Generalized Adaptive Intelligent binning is a logical extension of the
previously described Adaptive Intelligent binning algorithm
\cite{demeyer:anchem2008} to multidimensional datasets, and provides a
model-free alternative to peak-fitting and peak-picking as a means of variable
selection in multivariate analyses. Furthermore, GAI-binning is a more
intelligent method to extract signal regions from multidimensional spectral
data tensors than uniform binning, and may be used to generate very
low-dimensionality data matrices via multiple integration or efficiently
noise-filtered data matrices via vectorization. The C++ implementations of
1D and 2D GAI-binning used in this work are freely available as part of the
MVAPACK software package \cite{worley:acscb2014} introduced in
\hyperlink{chapter.5}{Chapter 5}.
\end{doublespace}

\newpage
\section{Permutation Test Results}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/gaibin/06-perm-unif-int.png}
\caption
      [Response Permutation Test: Uniform integration.]{
  {\bf Response Permutation Test: Uniform integration.}
  \\
  Response permutation test results for OPLS-DA models from the uniformly
  binned (integrated) liver ({\bf A}, {\bf B}) and fibroblast
  ({\bf C}, {\bf D}) data tensors. Model fit (\rsqy{}) statistics
  ({\bf A}, {\bf C}) are shown in blue, and model predictive ability (\qsq{})
  statistics ({\bf B}, {\bf D}) are shown in green. True values of \rsqy{} and
  \qsq{} are represented by vertical bars, and null distributions are computed
  through kernel density estimation of the values from permutation. Scatter
  plots of the permutation (null) \rsqy{} and \qsq{} statistics are shown in
  the lower panes.
}
\label{figure.8.6}
\end{figure}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/gaibin/07-perm-gai-int.png}
\caption
      [Response Permutation Test: GAI-integration.]{
  {\bf Response Permutation Test: GAI-integration.}
  \\
  Response permutation test results for OPLS-DA models from the GAI-binned
  (integrated) liver ({\bf A}, {\bf B}) and fibroblast
  ({\bf C}, {\bf D}) data tensors. See the caption of Figure 8.6 for a
  complete description of the figure contents.
}
\label{figure.8.7}
\end{figure}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/gaibin/08-perm-unif-vec.png}
\caption
      [Response Permutation Test: Uniform vectorization.]{
  {\bf Response Permutation Test: Uniform vectorization.}
  \\
  Response permutation test results for OPLS-DA models from the uniformly
  binned (vectorized) liver ({\bf A}, {\bf B}) and fibroblast
  ({\bf C}, {\bf D}) data tensors. See the caption of Figure 8.6 for a
  complete description of the figure contents.
}
\label{figure.8.8}
\end{figure}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/gaibin/09-perm-gai-vec.png}
\caption
      [Response Permutation Test: GAI-vectorization.]{
  {\bf Response Permutation Test: GAI-vectorization.}
  \\
  Response permutation test results for OPLS-DA models from the GAI-binned
  (vectorized) liver ({\bf A}, {\bf B}) and fibroblast
  ({\bf C}, {\bf D}) data tensors. See the caption of Figure 8.6 for a
  complete description of the figure contents.
}
\label{figure.8.9}
\end{figure}

\newpage
\bibliographystyle{abbrv}
\bibliography{bworley}


\section{Loss functions for point predictions}


\section{Types of Machine Learning}
For a machine to learn it is necessary to have something for it to learn, some set of data that can point it to the desired behaviour. Depending on the type of data available it is possible to divide learning into 4 different categories: Supervised; Unsupervised; Semi-supervised; and Reinforcement Learning.

\subsection{Supervised Learning} \label{sub:supervised_learning}
The simplest way to teach a computer is to train it with both the input data and the expected output. For example, suppose a classifier that is trained on the \gls{MNIST} dataset, the goal of this classifier is to take a gray-scale image as input and return the corresponding digit as output. Since \gls{MNIST} also contains the labels, the training would consist of feeding the images to the learning algorithm and comparing the predicted digits with those given by the labels. The difference between the prediction and the real output can then be used to adjust the classifier in order to make future predictions more likely to be correct.

Learning problems where the data contains both the input and the desired output are known as Supervised Learning problems, this is currently the most common form of learning. The advantage of this approach is that it makes very explicit to the computer what is expected from it, resulting in generally easier learning when compared to other forms of learning that have incomplete data.

The biggest problem with supervised learning is producing the datasets in the first place, obtaining the input data is generally simple, however producing the corresponding outputs (e.g. labels for classification problems) can be very difficult. For problems like image colorization \cite{colorization2016} or super resolution \cite{ganSuperResolution2016} this is trivial (i.e. convert a colorized image to gray-scale, downscale high resolution images), however for a dataset like \gls{MNIST} it is necessary to have a human label every single one of the $70,000$ images. For bigger datasets like ImageNet \cite{imageNet2015} that contain millions of images, classified into thousands of different classes, and that have annotated bounding boxes for a significant number of the images, the process is very slow and expensive. Amazon Mechanical Turk
\footnote{
    Mechanical Turk page: \url{https://www.mturk.com}
}%
is often used for such tasks.

\subsection{Unsupervised Learning} \label{sub:unsupervised_learning}
In contrast with supervised learning where all the data is often costly labelled, unsupervised learning refers to problems where only the input data is available and the network has no explicit notion of the desired outputs. These types of problems have a huge potential since the amount of unlabeled data is much larger, however, learning without labels is also much more difficult and it is still an active area of research.

Situations where unsupervised learning can be used are more rare and may require some specific prior knowledge of the datasets to work properly. The \gls{MNIST} dataset can be used again as a simple example to understand how this type of learning works. Suppose that it is desired to build a classifier for handwritten digits, but the only dataset available is \gls{MNIST} stripped of its labels; it may seem impossible to learn anything from this since all the machine sees are just images without any notion of right or wrong classification.

The trick is to exploit the prior knowledge of the problem and the distributions of the dataset in order to make progress. Since the objective is to build a digit classifier, then one prior information is that the number of possible classes will be 10, or 11 if a class ``Not a digit'' is also included.

Also consider the size of the input space, for \gls{MNIST} this is a $28{\times}28 = 784$ dimensional space, and with gray-scale images the number of different possible inputs is $256^{28\cdot28} = 2^{6,272} \approx 10^{1888}$. In such high dimensional spaces, all of the sensible inputs (e.g. images of digits) are just a tiny fraction of the whole input volume. This holds true for basically all situations, any random sample from an input space (pixel values, letters, audio amplitude) will almost always fail to produce the structured data present in the real world (pictures, words and phrases, human voice) \cite[chap. 5]{deepLearningBook2016} \textbf{--} it can be said that real life data is \textit{sparse} on the input space.

Along with sparsity, real data is also not evenly spaced on these high dimensional spaces, but it is instead concentrated around a relatively small number of clusters. \textcite[chap. 5]{deepLearningBook2016} argue this point by noting that it is possible to take similar images and apply some transformations, like moving objects or changing the light, in order to move from one image to another in the cluster. For example, one could take an image of the number 0 in \gls{MNIST}, add some slight rotations or change the width of the strokes, and end up with a different image that still represented the number 0. However a transformation from the number 0 to the number 3 does not seem to be so simple, so at least in an intuitive sense this example gives an idea of why the sparse data is also clustered around some points.

Using this knowledge of clusters and the number of classes expected in a problem like \gls{MNIST}, one can build an algorithm like \gls{KNN} to divide the input space into a desired number of regions, using a measure of similarity between inputs from the dataset as a guide for the divisions. Without using any labels the resulting algorithm is able to tell in which of the regions a given input belongs. For the \gls{MNIST} case, these regions have strong correlation with the corresponding digit \textbf{--} \cite{fashionMNIST2017} obtained accuracies of above 95\% with this technique.

Of course this approach is very dependent of the dataset and type of problem, it is not always that unsupervised learning will be so straight forward or that the divided regions will correlate well with the desired output.

\acp{GAN}, the main focus of this document, are also primarily unsupervised learning approaches. In \autoref{cha:gans} it will be discussed in detail how they work and learn from the underlying data distribution.

\subsection{Semi-supervised and Reinforcement Learning} \label{sub:semi-supervised_and_RL}
Semi-supervised learning is an intermediate between fully supervised and unsupervised learning, it concerns situations where the dataset is only partially labeled, the idea behind it is to use the vast amounts of unlabelled data to support the training from the relatively small number of labeled data. Since unlabeled data is much more easily available, semi-supervised learning has a lot of potential for building better models by fully leveraging the available data \cite{semi_supervised2005}.

Reinforcement learning is particularly different from the other types of learning, its objective is to teach an agent to interact with a changing environment; learning does not occur with a dataset, but is instead achieved through trial-and-error where the agent is rewarded or punished depending on its actions \cite{reinforcement_learning1996}. AlphaZero is an example of a reinforcement learning agent, it learned to play chess, shogi, and Go entirely through self-play \cite{alphaZero2017}.


\chapter{Experiments}
\label{chap:experiments}
This chapter explains how the models were trained in order to obtain the best performances and an explanation map.
\section{Training}
Training of the models was done using the Adam\cite{kingma2014adam} optimizer and the pytorch\cite{pytorch_paszke2017automatic} library. An implementation in tensorflow\cite{tensorflow_45166} has been done but PyTorch was chosen as we needed to explain the model using the fullGrad\cite{fullgradient} GitHub repository. For the loss we chose Binary Cross Entropy as the task we are solving is a binary classification.

Training the different models required to fine turn some hyperparameter. Figure~\ref{fig:fine_tune_learning_rate} shows how the test loss can be used to compare the performances of the model when trained with different learning rates. Here we see that choosing a learning rate of \num{5e-05} seems to be the best decision to train the model.

\begin{figure*}
    \centering
    \begin{subfigure}[b]{0.475\textwidth}
        \centering
        \includegraphics[width=\textwidth]{figures/Experiements/training_3D_CNN_GAP_lr_1e-05.pdf}
        \caption[Network2]%
        {{\small lr=\num{1e-05}}}
    \end{subfigure}
    \hfill
    \begin{subfigure}[b]{0.475\textwidth}
        \centering
        \includegraphics[width=\textwidth]{figures/Experiements/training_3D_CNN_GAP_lr_5e-05.pdf}
        \caption[]%
        {{\small lr=\num{5e-05}}}
        \label{fig:best_lr}
    \end{subfigure}
    \vskip\baselineskip
    \begin{subfigure}[b]{0.475\textwidth}
        \centering
        \includegraphics[width=\textwidth]{figures/Experiements/training_3D_CNN_GAP_lr_0.0001.pdf}
        \caption[]%
        {{\small lr=0.0001}}
    \end{subfigure}
    \quad
    \begin{subfigure}[b]{0.475\textwidth}
        \centering
        \includegraphics[width=\textwidth]{figures/Experiements/training_3D_CNN_GAP_lr_0.0002.pdf}
        \caption[]%
        {{\small lr=0.0002}}
    \end{subfigure}
    \caption[ The average and standard deviation of critical parameters ]
    {\small Comparison of the train and test loss for different learning rate across 200 epochs of training. }
    \label{fig:fine_tune_learning_rate}
\end{figure*}

Once trained, a network should be able to predict the correct class for any input. Unfortunately, the prediction of a model cannot be blindly trust. Nonetheless, the output of the network can be interpreted as a confidence score on the prediction of the network. Therefore even in the case where the network outputs $0.45$, which means that the patient does not have dementia, it can still be worth double-checking his case. In fact, for this particular prediction, the network is not so confident and has more chances of being wrong. Unfortunately, when trained for too long, the network tends to become overconfident and its outputs become almost binary (values close to zero or close to one), losing part of its interpretability. Label Smoothing \cite{label_smoothing_szegedy2015rethinking} aims at reducing the overconfidence of the network by changing the label of the train set from $0$ to $0 + \epsilon$ and from $1$ to $1 - \epsilon$. For the experiments, we chose $\epsilon$ to be $0.1$.

The loss used for training is Binary Cross Entropy. But as shown in figure~\ref{fig:OASIS_CDR_table} the data we are training on is unbalanced. Instead of training the model with the standard binary cross-entropy loss as defined in section~\ref{sec:binary_cross_entropy}, we used a variant that penalizes more the mistakes made on the underrepresented class. In our case, as we have approximately 70 percent of control samples and 30 percent of samples from people with dementia, the loss for a demented sample would be weighted by $\frac{1}{0.3} = 3.33$ and the other by $\frac{1}{0.7} = 1.43$.

Note that more could have been done in order to better train the model such as trying another optimizer or even using a learning rate scheduler, but we preferred to focus on explainability instead of performances.

\section{Evaluation}
The performance of the different model being very similar, we will perform the evaluation on the standard convolution model described in section~\ref{sec:standard_cnn}. This section purely evaluates the model's performance and resistance to age bias but is not interested in the explainability of the model. For evaluation of the age predictor please refer to annex~\ref{chap:age_pred}.

\subsection{Bias due to age}
\begin{wrapfigure}{r}{0.6\textwidth}
 \centering
 \includegraphics[width=.9\linewidth]{figures/Experiements/Eval/prediction_age.pdf}
 \captionsetup{width=.9\linewidth}
 \caption[PredPerAge]{Prediction of dementia per age.}
 \label{fig:prediction_per_age}
\end{wrapfigure}
As shown in the section~\ref{sec:OASIS}, the OASIS dataset presents a bias to age. To evaluate the model bias we can look at figure~\ref{fig:prediction_per_age}. We can see a trend that the model tends to more easily predict dementia to an old person than a young one.

\subsection{Metrics}
In order to evaluate and compare the different model, we used the metrics defined in section~\ref{sec:losses_metrics}. The confusion matrix in figure~\ref{fig:test_conf_matrix} allows us to compute the accuracy which is of $81.87\%$. This metric is quite confusing as it seems to be a good performance, but random guessing already has an accuracy of $70.76\%$. Instead, we can compute the precision which is of $74\%$ and the recall to be of $67.27\%$.

Figure~\ref{fig:roc_and_pr_curve} shows the ROC and PR curves to gain a better sense of the model performance. Note that as the dataset is unbalanced it is recommended to look at the PR curve.

\begin{wrapfigure}{r}{0.6\textwidth}
 \centering
 \includegraphics[width=.9\linewidth]{figures/Experiements/Eval/test_confusion_matrix.pdf}
 \captionsetup{width=.9\linewidth}
 \caption[ConfMatrix]{Confusion Matrix on the test set.}
 \label{fig:test_conf_matrix}
\end{wrapfigure}

\begin{figure}
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=1\linewidth]{figures/Experiements/Eval/ROC_curve.pdf}
  \caption{ROC curve.}
  \label{fig:roc_curve}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=1\linewidth]{figures/Experiements/Eval/pr_curve_curve.pdf}
  \caption{Precision and Recall curve.}
  \label{fig:pr_curve}
\end{subfigure}
\caption[Curve]{Evaluation curves.}
\label{fig:roc_and_pr_curve}
\end{figure}

\newpage
\section{Model Output Explanation}
\label{sec:model_out_exp}

As shown in figure~\ref{fig:alzheimerbrain}, Alzheimer disease tends to severely damage some specific regions of the brain. One of these regions is the hippocampus which serves at the creation of new memories. We would expect from a good explainer that indeed it highlights this specific region when explaining the output for a damaged brain. Figure~\ref{fig:dem_vs_control_age_pred} compares the outputs of the 3 algorithms we implemented on different slices of a damaged brain. In fact, in figure~\ref{fig:explainer_compared} we chose slices where the hippocampus is visible and observed that while Shap explanation is difficult to interpret, both GradCam and FullGrad focus on the hippocampus. It is especially visible with the output of the fullgrad algorithm where its maximal attention on this slice is actually located inside the hippocampus. For the rest of the report, we chose to work with fullgrad as its output makes more sense to us and is also sharper. The brain viewer implemented in annex~\ref{chap:brainviewer} is designed to work with a fullgrad saliency map.

Looking at the fullgrad output in figure~\ref{fig:fullgrad_output}, we realized that the model tends to focus more on the right hippocampus that the left one. In fact, the analysis performed in annex~\ref{sec:ml_result_stat_analysis} seems to confirm the difference with respect to the right hippocampus between healthy and demented patients.

Further analyses of the fullgrad outputs show the focus that the model has on the ventricles that typically grows larger for damaged brains.

\begin{figure}
    \centering
    \includegraphics[width=0.9\linewidth]{figures/Experiements/explainer_coparaison.pdf}
    \caption{Outputs of the explainer algorithm on one patient with dementia.}
    \label{fig:explainer_compared}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=1\linewidth]{figures/Experiements/output_fullgrad.pdf}
    \caption{Outputs of the fullgrad algorithm,the first row is composed of control patients and the second one of dementia patients. }
    \label{fig:fullgrad_output}
\end{figure}


\documentclass[11pt,a4paper,oneside]{article}
\usepackage[english]{babel}

\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}

\usepackage{geometry}
\usepackage{color}
\usepackage{graphicx}
\usepackage{pifont}
\usepackage[babel]{csquotes}
\usepackage{textcomp}
\usepackage{upgreek}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{latexsym}
\usepackage{pgf}
\usepackage{nicefrac}
\usepackage{enumerate}
\usepackage{stmaryrd}

\usepackage{tgpagella}

\DeclareFontFamily{OT1}{pzc}{}
\DeclareFontShape{OT1}{pzc}{m}{it}{<-> s * [1.10] pzcmi7t}{}
\DeclareMathAlphabet{\mathpzc}{OT1}{pzc}{m}{it}
\DeclareUnicodeCharacter{10230}{$\rightarrow$}

\newcommand{\ie}{i.\,e.}
\newcommand{\wuppdi}[0]{\hfill\ensuremath{\square}}
\newcommand{\qed}[0]{\vspace{-3mm}\begin{flushright}\textit{q.e.d.}\end{flushright}\vspace{3mm}}
\newcommand{\bred}{\ensuremath{\longrightarrow_\beta}}
\newcommand{\acos}{\textrm{arccos}}
\newcommand{\determ}[1]{\textrm{det}(#1)}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\BB}{\mathbb{B}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\CC}{\mathbb{C}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\kernel}[1]{\textrm{ker}(#1)}
\renewcommand{\epsilon}{\varepsilon}
\renewcommand{\phi}{\varphi}
\renewcommand{\theta}{\vartheta}
\newcommand{\atan}{\mathrm{arctan}}
\newcommand{\rot}{\mathrm{rot}}
\newcommand{\vdiv}{\mathrm{div}}
\newcommand{\shouldbe}{\stackrel{!}{=}}
\newcommand{\sturm}{\texttt{sturm}}
\newcommand{\lemma}{\textbf{lemma}}
\newcommand{\card}{\textrm{card}}
\newcommand{\real}{\textrm{real}}

\newcommand{\isabellehol}{\mbox{Isabelle}\slash HOL}

\geometry{a4paper,left=30mm,right=30mm, top=25mm, bottom=30mm}

\title{\LARGE User's Guide for the \texttt{sturm} Method\\[4mm]}
\author{\Large Manuel Eberl <eberlm@in.tum.de>\\[1mm]\large Institut für Informatik, Technische Universität München\\[4mm]}

\begin{document}
\begin{center}
\vspace*{20mm}
\includegraphics[width=4cm]{isabelle.pdf}
\end{center}
\vspace*{-5mm}
{\let\newpage\relax\maketitle}
\vspace*{10mm}
\tableofcontents
\newpage

\section{Introduction}

The \sturm\ method uses Sturm's theorem to determine the number of distinct real roots of a polynomial (with rational coefficients) within a certain interval. It also provides some preprocessing to decide a number of statements that can be reduced to real roots of polynomials, such as simple polynomial inequalities and logical combinations of polynomial equations.

\section{Usage}

\subsection{Examples}
The following examples should give a good overview of what the \sturm\ method can do:
\begin{align*}
&\lemma\ "\card\ \{x::\real.\ (x - 1)^2 * (x + 1) = 0\}\  =\  2"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "\mathrm{card}\ \{x::\mathrm{real}.\ -0.010831 < x\ \wedge\ x < 0.010831\ \wedge\\
&\hskip20mm \mathrm{poly}\ [:0, -17/2097152, -49/16777216, 1/6, 1/24, 1/120:]\ \ x\ =\ 0\}\  =\  3"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "\card\ \{x::\real.\ x^3 + x = 2*x^2\ \wedge\ x^3-6*x^2+11*x=6\}\  =\  1"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "\card\ \{x::\real.\ x^3 + x = 2*x^2\ \vee\ x^3-6*x^2+11*x=6\}\  =\  4"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "(x::\real)^2+1 > 0"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "(x::\real) > 0\ \Longrightarrow\ x^2+1 > 0"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "\llbracket (x::\real) > 0; x \leq 2/3\rrbracket\ \Longrightarrow\ x*x \neq\ x"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "(x::\real) > 1\ \Longrightarrow\ x*x > x"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "x \geq -0.5\ \Longrightarrow\ x \leq 1\ \Longrightarrow\ x^3 - 5*x^2 + x + 3 \geq (0::\mathrm{real})"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "\textrm{mono}\ (\lambda x::\mathrm{real}.\ x^3)"\ \textrm{\textbf{by}\ sturm}\\
&\lemma\ "\textrm{strict\_mono}\ (\lambda x::real.\\\
&\hskip20mm\ x^7/7 - 14/5*x^5 + 3*x^4 + 49/3*x^3 - 42*x^2 + 36*x - 13)"\ \textrm{\textbf{by}\ sturm}\\
\end{align*}

\subsection{Determining the number of real roots}
The \enquote{classical} application of Sturm's theorem is to count the number of real roots of a polynomial in a certain interval. The \sturm\ method supports this for any polynomial with rational coefficients and any real interval, \ie $[a;b]$, $(a;b]$, $[a;b)$, and $(a;b)$ where $a\in\QQ\cup\{-\infty\}$ and $b\in\QQ\cup\{\infty\}$.\footnote{The restriction to rational numbers for the coefficients and interval bounds is to the fact that the code generator is used internally, which, of course, does not support computations on irrational real numbers.} The general form of the theorems the method expects is:
$$\card\ \{x::\real.\ a < x \wedge x < b \wedge p\ x = 0\}\ =\ ?n$$
$?n$ should be replaced by the actual number of such roots and $p$ may be any polynomial real function in $x$ with rational coefficients. The bounds $a < x$ and $x < b$ can be omitted for the \enquote{$\infty$} case.\\

Furthermore, the \sturm\ method can instantiate the number $?n$ on the right-hand side automatically if it is left unspecified (as a schematic variable in a schematic lemma). However, due to technical restrictions this also takes twice as long as simply proving that the specified number is correct.

\subsection{Inequalities}

A simple special case of root counting is the statement that a polynomial $p\in\RR[X]$ has no roots in a certain interval, which can be written as:
$$\forall x::\real.\ x > a \wedge x < b \longrightarrow p\ x \neq 0$$
The \sturm\ method can be directly applied to statements such as this and prove them.

\subsection{More complex expressions}

By using some simple preprocessing, the \sturm\ method can also decide more complex statements:
$$\card\ \{x::\real.\ x > a\ \wedge\ x < b\ \wedge\ P\ x\}\ =\ n$$
where $P\ x$ is a \enquote{polynomial expression}, which is defined as:
\begin{enumerate}
\item $p\ x= q\ x$, where $p$ and $q$ are polynomial functions, such as $\lambda x.\ a$, $\lambda x.\ x$, $\lambda x.\ x^2$, $\mathrm{poly}\ p$, and so on
\item $P\ x\ \wedge\ Q\ x$ or $P\ x\ \vee\ Q\ x$, where $P\ x$ and $Q\ x$ are polynomial expressions
\end{enumerate}

Of course, by reduction to the case of zero roots, the following kind of statement is also provable by \sturm\ :
$$\forall x::\real.\ x > a\ \wedge\ x < b\ \longrightarrow\ P\ x$$
where $P\ x$ is a \enquote{negated polynomial expression}, which is defined as:
\begin{enumerate}
\item $p\ x\neq q\ x$, where $p$ and $q$ are polynomial functions
\item $P\ x\ \wedge\ Q\ x$ or $P\ x\ \vee\ Q\ x$, where $P\ x$ and $Q\ x$ are negated polynomial expressions
\end{enumerate}

\subsection{Simple strict linear inequalities}
For any polynomial $p\in\RR[X]$, the question whether $p(x) > 0$ for all $x\in I$ for a non-empty real interval $I$ can obviously be reduced to the question of whether $p(x) \neq 0$ for all $x\in I$, \ie $p$ has no roots in $I$, and $p(x) > 0$ for some arbitrary fixed $x\in I$, the first of which can be decided using Sturm's theorem and the second by choosing an arbitrary $x\in I$ and evaluating $p(x)$.\\

Using this reduction, the \sturm\ method can also decide single \enquote{less than}/\enquote{greater than} inequalities of the form
$$\forall x::\real.\ x > a\ \wedge\ x < b\ \longrightarrow\ p\ x < q\ x$$

\subsection{Simple nonstrict linear inequalities and monotonicity}

The \sturm\ method can also decide nonstrict inequalities, e.g. $p(x) \geq 0$ for all $x\in I$ where $I$ is a non-empty real interval. In contrast to the previous cases, proving nonstrict inequalities is done with help from an ML function that computes a witness, which is then used in the proof by the actual proof method. The witness is a decomposition of the interval in question into disjoint intervals such that every interval contains exactly one root, which makes it possible to check nonnegativity by verifying it on each interval.

Note that while this does not compromise soundness in any way (as witnesses still have to go through checking in Isabelle), it may compromise completeness if the ML code, due to a bug, fails to provide an adequate witness.\newpage

Additionally, the \sturm\ method can also prove monotonicity and strict monotonicity of polynomials (over the entire real line) by reducing the problem to nonnegativity of the derivative.

\subsection{A note on meta logic versus object logic}

While statements like $\forall x::\real.\ x^2+1>0$ were expressed in their HOL notation in this guide, the \sturm\ method can also prove the meta logic equivalents $\bigwedge x::\real.\ x^2+1>0$ and $(x::\real)^2+1>0$ directly.

\section{Troubleshooting}

Should you find that the \sturm\ method fails to prove a statement that it should, according to the above text, be able to prove, please go through the following steps:
\begin{enumerate}
\item ensure that your function is indeed a \emph{real} polynomial. Add an appropriate type annotation if necessary.
\item use a computer algebra system to ensure that the property is indeed correct
\item if this did not help, send the statement in question to \texttt{eberlm@in.tum.de}; it may be a bug in the preprocessing of the proof method or
\end{enumerate}

Note: in case of monotonicity and nonnegativity, the proof method will output a counterexample if the statement is false. Should proving nonnegativity or monotonicity fail without a counterexample being given, please also report this as a bug.

\end{document}


% !TEX TS-program = pdflatexmk
\documentclass{article} % For LaTeX2e
\usepackage{nips15submit_e,times}
\usepackage{hyperref}
\usepackage{url}
%\documentstyle[nips14submit_09,times,art10]{article} % For LaTeX 2.09

\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{mathtools}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}[theorem]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{example}{Example}[theorem]

\theoremstyle{definition}
\newtheorem{definition}{Definition}

\newcommand\myeq{\stackrel{\mathclap{\tiny\mbox{d}}}{=}}

\newcommand\mc{\mathcal} %calligraphic
\newcommand\ts{\mathcal} %tensor
\newcommand\mt{} %matrix
\newcommand\vt{\mathbf} %vector
\newcommand\fn{} %function
\newcommand\triple[3]{(#1 \stackrel{#2}\rightarrow #3)}
%\newcommand\triple[3]{(#1, #2, #3)}

\title{Knowledge Graph Construction with External Task}

\author{
Dongwoo Kim
}

% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.

\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}

\nipsfinalcopy % Uncomment for camera-ready version

\begin{document}

\maketitle

%\begin{abstract}
%\end{abstract}
\section{Notation}
We denote vectors by bold lower case letter, e.g. $\vt{a}$, matrices by bold upper case letters, e.g. $\mt{R}$, and mode-3 tensors by calligraphic upper case letters, e.g. $\ts{G} \in \mathbb{R}^{d_1 \times d_2 \times d_3}$. An index of the tensor is represented by a triple $\triple{i}{k}{j}$, and calligraphic letters are also used as a set of triples $\ts{T}$ if it is clear in the context. We use $r_{ij}$ to represent the $(i,j)$ entry of matrix $\mt{R}$ and $g_{ijk}$ to represent the $(i,j,k)$ entry of tensor $\ts{G}$. In addition, we use $\vt{r}_i$ to represent the row $i$ of matrix $\mt{R}$ and $G_k$ to represent $k$th slice of tensor $\ts{G}$.

\section{Knowledge Graph Construction with External Task}

Many active learning algorithms for knowledge graph construction only focus on obtaining as many positive triples as possible. This is not a crazy idea, however, it does not agree with a practical usage of knowledge graphs. For example, a content provider may want to use the knowledge graph to recommend items that fit to user's preference. This could be achieved by considering the similarity between different items in the knowledge graph based on a history of a user. If the active knowledge graph construction does not care the performance of following tasks, it may end up to discover some facts that do not lead any improvement in recommendation. Therefore, `good' active construction algorithms should reflect the following usages of the knowledge graph.

\subsection{General Framework}
A score function $s_\mt{X}: \mathbb{M} \rightarrow \mathbb{R}_+$.
$s_\mt{X}: \mathbb{T} \rightarrow \mathbb{R}_+$

\begin{align}
L_{\mt{X},\ts{G}} = L_\mt{X}(s_\mt{X}) + L_\ts{G}(s_\ts{G})
\end{align}
Assume that both loss function shares some parameter $\vt{v}_j$ for each entity $j$

\subsection{Co-factorisation}

A recent study \cite{Zhang2016} examines how a knowledge graph can be used for a movie recommendation. Their approach resembles a co-factorisation where two different loss functions for user-movie matrix and knowledge graph tensor are used to estimate the latent embeddings of movies and users.
Let $\vt{u}_i \in \mathbb{R}^d$ be a latent embedding of user $i$, $\vt{v}_j, \vt{v}_l \in \mathbb{R}^d$ be a latent embedding of entity $j$ and $l$, and $\mt{R}_{k} \in \mathbb{R}^{d\times d}$ be a latent embedding of relation $k$. $X \in \mc{X}^{U \times E}$, where $U$ is a number of users and $E$ is a number of entities, is a user-movie rating matrix, and $\ts{G} \in \{0,1\}^{E\times E \times K}$, where $K$ is a number of relations, is a tensor-represented knowledge graph. We use $g_{jlk}$ to denote the value of triple $t=\triple{j}{k}{l}$. The goal of co-factorisation is to minimise the following loss
\begin{align}
\label{eqn:two_losses}
L_{X, \ts{G}}(\mt{U}, \mt{V}, \ts{R}) & = L_X(\mt{U}, \mt{V}) + \lambda L_\ts{G}(\mt{V}, \ts{R})\\
& = \sum_{ij}\ell_X(x_{ij}, \vt{u}_{i}^\top \vt{v}_j) + \lambda \sum_{jlk}\ell_\ts{G}(g_{jlk}, \vt{v}_j^\top \mt{R}_{k} \vt{v}_{l}), \label{eqn:two_losses}
\end{align}
where we use standard bilinear model to score each triple given latent variables.
A squared error is typically used as a loss for the matrix factorisation. We may consider the recommendation as a binary classification problem. In this case, we can choose an appropriate loss $\ell_X$ to optimise.  We often measure the performance of these problems using AUC to take into account an imbalanced distribution of labels. In this case, it would be preferable to directly optimise AUC on a training set, or in practice, optimise a surrogate loss. However, the current objective optimise both losses for recommendation and knowledge graph construction, which may not be an optimal choice. In other words, we want to optimise the first term in RHS of Equation \ref{eqn:two_losses} instead of optimising both terms while considering the structure of a knowledge graph.
One way to achieve this goal is to design a nested objective function. One possible candidate is
\begin{align}
\label{eqn:obj}
L_{X, \ts{G}}(\mt{U}) & = \sum_{ij}\ell(x_{ij}, \vt{u}_{i}^\top \fn{f}_j(\ts{G}))
\end{align}
where feature function $\fn{f}_j : \{0,1\}^{E\times E\times K} \rightarrow \mathbb{R}^d$ extracts $d$-dimensional feature of entity $j$ from the entire knowledge graph $\ts{G}$. Two questions are naturally occurring from here: 1) How to extract feature of entity $j$ from the entire knowledge graph? 2) How does the back-propagated loss help to construct a KG?
%Before answering these questions, let's derive the partial derivative of the objective. If we use the squared loss, the derivative w.r.t. function $\fn{f}_j$ has a form
%\begin{align}
%\frac{\partial L_{X, \ts{G}}}{\partial \fn{f}_j} = -\sum_{i}(x_{ij} - \vt{u}_i^\top \fn{f}_j) \vt{u}_i.
%\end{align}
There might be several ways to design a feature function $f$. However, instead of designing a feature function from scratch, it would be better to incorporate some ideas of existing statistical relational models since the goal of these models are highly focused on how to infer latent semantics of entities and relations. More specifically, if we can match the function $f_j$ to the latent representation of entity $j$ of some statistical model, then the Equation \ref{eqn:obj} will naturally encompass both the statistical relational model and recommendation problem.

Alternatively, we can use the co-factorisation objective in Equation \ref{eqn:two_losses}. One problem of combining two datasets in a single objective is that an optimal loss may be different across different datasets. For example, if we cast the recommendation problem as a classification problem, then it would be preferable to learn a pair-wise scorer between positive and negative data points based on the hinge loss.

%How to model feature function of an entity? The feature function $\fn{f}_j$ should reflect some local (or global) structure of the knowledge graph.  Let $\mc{T}_j$ be a set of triples that contain entity $j$. We may also use the bilinear structure to model features as follow:
%\begin{align}
%\label{eqn:feature_fn}
%\fn{f}_j(\ts{G}) = \frac{1}{|\mc{T}_j|} \bigg( \sum_{l,k:\triple{j}{k}{l} \in \mc{T}_j} \mt{R}_{k} \vt{v}_{l}/g_{jkl} + \sum_{l,k:\triple{j}{k}{l} \in \mc{T}_j}\mt{R}_{k}^\top \vt{v}_{l}/g_{lkj} \bigg).
%\end{align}
%This feature function is derived from a bilinear assumption. In the bilinear model, the ultimate goal is to infer entity vectors and relation matrices which satisfy the following equation
%\begin{align}
%g_{jkl} = \vt{v}_j ^\top \mt{R}_{k} \vt{v}_l, \quad \forall \triple{j}{k}{l}.
%\end{align}
%We have used $\vt{v}_j = g_{jkl} \mt{R}_{k} \vt{v}_l$ and let $\fn{f}_j$ be an average of these vectors across all possible triples which contain entity $j$.
%\begin{align}
%\frac{\partial \fn{f}_{jd}}{\partial\vt{v}_l} = \frac{|\mc{T}_{jl}|}{|\mc{T}_j|} \bigg( \sum \mt{R}_{k} g_{jkl} + \sum \mt{R}_{k}^\top g_{lkj} \bigg)
%\end{align}
%To compare with the second loss in Equation \ref{eqn:two_losses}, we only took the structure of knowledge graph (which entities are connected from/to entity $j$ through which relations) to model the feature function. $e$ and $R$ in Equation \ref{eqn:feature_fn} are not estimated based on some loss w.r.t. the knowledge graph.

\subsection{Active learning}
\textbf{Which triple should be queried next?} Our goal is to minimise the empirical loss in Equation \ref{eqn:obj}. Therefore, next triple $t$ should be chosen to minimise the expected loss
\begin{align}
\arg\min_t \mathbb{E}_{g_t}\bigg[\sum_{ij}\ell(x_{ij}, \vt{u}_{i}^\top  \fn{f}_j(\ts{G} \cup t))\bigg],
\end{align}
or maximise the expected gain
\begin{align}
\arg\max_t \sum_{ij}\ell(x_{ij}, \vt{u}_{i}^\top  \fn{f}_j(\ts{G})) - \mathbb{E}_{g_t}\bigg[\sum_{ij}\ell(x_{ij}, \vt{u}_{i}^\top  \fn{f}_j(\ts{G} \cup t))\bigg],
\end{align}
where we take an expectation over the possible value of triple i.e. $g_t \in \{0, 1\}$. Again, this requires some model that estimates the probability of triple such as \textsc{prescal}.

\section*{Statistical relational models}
\subsection*{RESCAL}
The original \textsc{Rescal} paper uses squared error loss to optimise the latent embeddings. Below we list some alternative losses and its derivative w.r.t. the latent variables.

\textbf{Negative log logistic loss}:
A total loss of the \textsc{Rescal} can be decomposed into a sum of individual loss
\begin{align}
L_\ts{G}(\mt{V}, \ts{R}) = \sum_{jkl} \ell_\sigma(g_{jlk}, \vt{v}_j^\top \mt{R}_{k} \vt{v}_l).
\end{align}
If the individual loss is the negative logarithm of the logistic loss, the function can be written
\begin{align}
\ell_\sigma(g_{jlk}, \vt{v}_j^\top \mt{R}_{k} \vt{v}_l) = - \bigg\{g_{jlk} \log \sigma(\vt{v}_j^\top \mt{R}_{k} \vt{v}_l) + (1- g_{jlk}) \log (1- \sigma(\vt{v}_j^\top \mt{R}_{k} \vt{v}_l))\bigg\},
\end{align}
and the derivative of the total loss w.r.t the entity vector $\vt{v}_j$ is
\begin{align}
\frac{\partial L_\ts{G}}{\partial \vt{v}_j} =  \sum_{lk}\bigg[\Big\{\sigma(\vt{v}_j^\top \mt{R}_{k} \vt{v}_l) - g_{jlk}\Big\} \mt{R}_{k} \vt{v}_l + \Big\{\sigma(\vt{v}_l^\top \mt{R}_{k_p} \vt{v}_j) - g_{ljk}\Big\} \mt{R}_{k}^\top \vt{v}_l \bigg],
\end{align}
where we consider both direction from/to entity $j$.

\textbf{Surrogate AUC loss (hinge loss)}
Let $\mc{T}^{(p)}$ be a set of positive triples and $\mc{T}^{(n)}$ be a set of negative triples.
\begin{align}
L_\ts{G}(\mt{V}, \ts{R}) = \sum_{\triple{j_p}{k_p}{l_p} \in \mc{T}^{(p)}}\sum_{\triple{j_n}{k_n}{l_n} \in \mc{T}^{(n)}} \ell_+(\vt{v}_{j_p}^\top \mt{R}_{k_p} \vt{v}_{l_p}, \vt{v}_{j_n}^\top \mt{R}_{k_n} \vt{v}_{l_n}),
\end{align}
where $\ell_+(x, y) = y - x$ if $y - x \geq 0$ else 0. Let $\mc{T}_+^{(p,n)}$ be a set of pairs of triples whose hinge loss is greater than 0. Taking the derivative of the hinge loss w.r.t. $v_j$ is
\begin{align}
\frac{\partial L_\ts{G}}{\partial \vt{v}_j} = &
- \sum_{(t_p, t_n) \in \mc{T}_+^{(p,n)} : j_p = j} \mt{R}_{k_n}\vt{v}_{l_p} - \sum_{t_p, t_n: l_p = j} \mt{R}_{k_p}^\top \vt{v}_{j_p} \\
& \quad + \sum_{t_p, t_n: j_n = j} \mt{R}_{k_n}\vt{v}_{l_n}  + \sum_{t_p, t_n: l_n = j} \mt{R}_{k_n}^\top \vt{v}_{j_n}.
\end{align}
The derivative remains the same even if we put some additional margin on the loss.

\subsection*{TranseE}
\textsc{TransE} model assumes that every entity can be represented by a linear function of relation and other entity in each triple.
\begin{align}
\vt{v}_j =  \vt{v}_l -\vt{r}_k \quad \forall \triple{j}{k}{l},
\end{align}
where both entities and relations are represented by length-$d$ vector unlike \textsc{Rescal}.
Since above equations are overdetermined system in general, the goal of \textsc{TransE} is to minimise the sum of squared distance between $\vt{v}_j$ and $\vt{v}_l -\vt{r}_k$:
\begin{align}
L_\ts{G}(\mt{V}, \mt{R}) = \sum_{\triple{j}{k}{l}}|\vt{v}_j + \vt{r}_k - \vt{v}_l|^2
\end{align}

\section{}

\begin{align}
&\mathbb{D}_{\phi}(\mt{X}, \hat{X}) + \sum_k \mathbb{D}_{\phi}(\mt{G}_k, \hat{G}_k)\\
=&\mathbb{D}_{\phi}(\mt{X}, \mt{U}\mt{V}^\top) + \sum_k \mathbb{D}_{\phi}(\mt{G}_k, \mt{V}\mt{R}_k\mt{V}^\top)\\
=& \sum_{ij} d_\phi(x_{ij}, \vt{u}_i^\top \vt{v}_j) + \sum_{\triple{j}{k}{l}} d_\phi(g_{jkl}, \vt{v}_j^\top R_k \vt{v}_l)\\
=& \sum_{ij} \phi(x_{ij}) - \phi(\vt{u}_i^\top \vt{v}_j) - \langle x_{ij}, \vt{u}_i^\top \vt{v}_j \rangle \nabla\phi(\vt{u}_i^\top \vt{v}_j) \\
&+ \sum_{\triple{j}{k}{l}} \phi(g_{jkl}) - \phi(\vt{v}_j^\top R_k \vt{v}_l) - \langle g_{jkl}, \vt{v}_j^\top R_k \vt{v}_l \rangle \nabla\phi(\vt{v}_j^\top R_k \vt{v}_l)\\
=& \sum_{ij} \phi(x_{ij}) + \phi^\star(\hat{x}_{ij}^\star) - \langle x_{ij}, \hat{x}_{ij}^\star \rangle \\
&+ \sum_{\triple{j}{k}{l}} \phi(g_{jkl}) + \phi^\star(\hat{g}_{jkl}^\star) - \langle g_{jkl}, \hat{g}_{jkl}^\star\rangle ,
\end{align}
where $\phi^\star$ is a conjugate of $\phi$, and $\hat{x}^\star = (\nabla \phi)^{-1}(\hat{x})$ is related to $\hat{x}$ by the Legendre transformation.

\section{Does the domain adaptation or transfer learning help?}
The domain adaptation often provides a systematic way to transfer knowledge from one domain to the other domain. For example, a model trained on a set of synthesised images with varying font styles is used to identify a house-number on street-view dataset with the domain adaptation \cite{Ganin2015}. In domain adaptation, each dataset comes from different source, but all datasets share the same task i.e. input distribution $p(X)$ changes between different datasets, but label distribution $p(Y|X)$ remains the same. In our case, the domain adaptation does not fit well because we tackle to solve two separate tasks: knowledge graph completion and recommendation. The transfer learning might be more appropriate in this case. The assumption behind transfer learning is that the input distribution $p(X)$ remains the same but the label distribution $p(Y|X)$ changes between different datasets, and typically, it is assumed that we have one data set that is labeled for both problems. The difference between our problem and general transfer learning problems is that the data are i.i.d. distributed in general transfer learning, but our data is not i.i.d. distributed and can be represented by multiple relations. Markov logic networks have been used to tackle the transfer learning problem in relational datasets. For example, \cite{Mihalkova2007} try to find mapping between relations from two different knowledge graphs via mapping two Markov logic networks. The MLN approach only works for homogeneous datasets does not suit in our case.

\bibliographystyle{apalike}
\bibliography{ref}

\end{document}


\section{Cryptographic Details}
\label{sec:crypto-details}

\subsection{Hashing}
We present the (informal) properties of cryptographically safe hash functions:
\begin{description}
\item[Preimage resistance] It should be hard to find a message with a given hash value.
\item[Second preimage resistance] Given one message, it should be hard to find another message with the same hash value.
\item[Collision resistance] It should be hard to find two messages with the same hash value.
\end{description}

\noindent In practice, several cryptographic protocols use hash functions as random oracles.
However, the properties of our scheme do not depend on it.

The hashing algorithm we use for all verification keys and multi-signature scripts is BLAKE2b-224\footnote{Note that for the signature
and verification algorithms, as well as the proving and verification algorithms of VRFs, we use the hash function as defined in the IETF standard (see Section~\ref{sec:app-addresses} and Section~\ref{sec:app-vrf}).}.
Explicitly, this is the payment and stake credentials (Figure~\ref{fig:defs:addresses}),
the genesis keys and their delegates (Figure~\ref{fig:ts-types:pp-update}),
stake pool verification keys (Figure~\ref{fig:delegation-transitions}),
and VRF verification keys (Figure~\ref{fig:delegation-defs}).
The only property we require of this hash function is that of Second Preimage resistance. Given the hash of verification key or multi-signature script, it should be hard to find a different key or script with the same hash.

Everywhere else we use BLAKE2b-256. In this case we do require second preimage resistance and collision resistance. We use the output of these hash functions to produce signatures (see Figure~\ref{fig:rules:utxow-shelley}), meaning that if an adversary can find two messages with the same hash, it could break the intended EUF-CMA (Existentially UnForgeable against a Chosen Plaintext Attack) property of the underlying signature scheme.

In the CDDL specification in Appendix~\ref{sec:cddl},
$\mathsf{hash28}$ refers to BLAKE2b-224 and
and $\mathsf{hash32}$ refers to BLAKE2b-256.
BLAKE2 is specified in RFC 7693 \cite{rfcBLAKE2}.

\subsection{Addresses}
\label{sec:app-addresses}
The \fun{sign} and \fun{verify} functions from Figure~\ref{fig:crypto-defs-shelley}
use Ed25519. See \cite{rfcEdDSA}.

\subsection{KES}
The \fun{sign_{ev}} and \fun{verify_{ev}} functions from Figure~\ref{fig:kes-defs-shelley}
use the iterated sum construction from Section 3.1 of \cite{cryptoeprint:2001:034}.
We allow up to $2^7$ key evolutions, which is larger than the maximum number
of evolutions allow by the spec, \MaxKESEvo, which will be set to $90$.
See Figure~\ref{fig:rules:ocert}.

\subsection{VRF}
\label{sec:app-vrf}
The \fun{verifyVrf} function from Figure~\ref{fig:defs-vrf}
uses ECVRF-ED25519-SHA512-Elligator2 as described in the draft IETF specification
\cite{rfcVRFDraft}.

More generally, we expect the following properties from a Verifyable Random Function:
\begin{description}
\item[Full uniqueness] For any fixed public VRF key and for \textit{any} input seed, there is a unique VRF output that can be proved valid.
\item[Full collision resistance] It should be computationally infeasible to find two distinct VRF inputs that have the same VRF output, even if the adversary knows the private key.
\item[Pseudorandomness] Assuming the public-private key pair has been generated honestly, the VRF hash output on any adversarially-chosen VRF input looks indistinguishable from random for a computationally bounded adversary who does not know the private key.
\end{description}

\subsection{Abstract functions $\seedOp$ and \fun{seedToSlot}}
\label{sec:mkseed-slottoseed}
The seed operation $x \seedOp y$ from Figure~\ref{fig:defs-vrf} is implemented
as the BLAKE2b-256 hash of the concatenation of $x$ and $y$.

The function \fun{slotToSeed} from Figure~\ref{fig:defs:blocks} is implemented
as the big-endian encoding of the slot number in 8 bytes.

\subsection{Multi-Signatures}
As presented in Figure~\ref{fig:types-msig}, Shelley realizes multi-signatures
in a native way, via a script-like DSL. One defines the conditions required to
validate a multi-signature, and the script takes care of verifying the
correctness of the request. It does so in a na\"ive way, i.e. checking every
signature individually. For instance, if the requirement is to have $n$ valid
signatures out of some set $\mathcal{V}$ of public keys, the na\"ive
script-based solution checks if: (i) the number of submitted signatures is
greater or equal to $n$, (ii) the distinct verification keys are part of the set
$\mathcal{V}$, and (iii) at least $n$ signatures are valid. However, there are
more efficient ways to achieve this using more advanced multi-signature schemes,
that allow for aggregating both the signatures and the verification procedure.
This results in less data to be stored on-chain, and a cheaper verification
procedure. Several schemes provide these properties~\cite{musigBoneh, musig,
musig2, pixel}, and we are currently investigating which would be the best fit
for the Cardano ecosystem. We formally introduce multi-signature schemes.

\sloppy
A multi-signature scheme~\cite{musigs} is defined as a tuple of algorithms
$\textsf{MS} = (\textsf{MS-Setup}, \textsf{MS-KG}, \textsf{MS-AVK},
\allowbreak\textsf{MS-Sign}, \textsf{MS-ASign}, \textsf{MS-Verify)}$ such that
$\Pi\leftarrow\textsf{MS-Setup}(1^k)$ generates public parameters---where $k$ is
the security parameter. Given the public parameters, one can generate a
verification-signing key pair calling,
$(\mathsf{vk,sk})\leftarrow\textsf{MS-KG}(\Pi)$. A multi-signature scheme
provides a signing and an aggregate functionality. Mainly
\begin{itemize}
\item $\sigma\leftarrow\textsf{MS-Sign}(\Pi, \mathsf{sk}, m)$, produces a
signature, $\sigma$, over a message $m$ using signing key $\mathsf{sk}$, and
\item $\tilde{\sigma}\leftarrow\textsf{MS-ASig}(\Pi, m, \mathcal{V, S)}$, where
given a message $m$, a set of signatures, $\mathcal{S}$, over the message $m$,
and the corresponding set of verification keys, $\mathcal{V}$, aggregates all
signatures into a single, aggregate signature $\tilde{\sigma}$.
\end{itemize}

To generate an aggregate verification key, the verifier calls the function
$\mathsf{avk}\leftarrow\textsf{MS-AVK}(\Pi, \mathcal{V})$, which given input a
set of verification keys, $\mathcal{V}$, returns an aggregate verification key
that can be used for the verification of the aggregate siganture:
$\textsf{MS-Verify}(\Pi, \mathsf{avk}, m,
\tilde{\sigma})\in\{\textsf{true},\textsf{false}\}$.

Note the distinction between multi-signature schemes (as described above) and
multi-signatures as defined in Figure~\ref{fig:types-msig}. In
Figure~\ref{fig:types-msig} we allow scripts to consider valid the
\type{RequireAllOf}, \type{RequireAnyOf} or \type{RequireMOfN} typed signatures.
In the definition above, given a set of public keys $\mathcal{V}$, a signature
is considered valid if \textit{all} key owners participate. However, such
multi-signature schemes together with a simple script-like DSL can achieve the
properties defined in Figure~\ref{fig:types-msig} while still providing the
benefits of a simple verification procedure, and a smaller signature size.