\section{Integration of scalar fields}


\documentclass[12pt]{article}
\usepackage{xspace,times,a4wide,command,environment}
\usepackage{skaff}
\usepackage{optionman}
\usepackage{comment}

\newcommand{\Linsearch}[0]{{\small \textsf{linsearch}}\xspace}
\newcommand{\Binsearch}[0]{{\small \textsf{binsearch}}\xspace}
\newcommand{\Vmbucket}[0]{{\small \textsf{vmbucket}}\xspace}
\newcommand{\Vmctlook}[0]{{\small \textsf{vmctlook}}\xspace}
\newcommand{\Mkvtree}[0]{{\small \texttt{mkvtree}}\xspace}
\newcommand{\Showfile}[1]{{\small \texttt{#1}}}
\newcommand{\Weight}[0]{w}
\newcommand{\Sol}[1]{\mathit{Sol}(\Weight)}
\newcommand{\Minmass}{600 \cdot 10^{p}}
\newcommand{\Maxmass}{4000 \cdot 10^{p}}
\newcommand{\Defaultrange}{\lbrack\Minmass,\Maxmass\rbrack}
\newcommand{\WSMP}{weighted substring matching problem\xspace}

\author{Stefan Kurtz\thanks{\SKaffiliation}}
\title{\textbf{Implementing Algorithms for}\\
       \textbf{the Weighted String Matching Problem}}

\begin{document}
\maketitle
\section{Introduction}
Let \(\Sigma\) be a finite alphabet. Strings are indexed from \(0\),
i.e.\ any \(w\in\Sigma^{l}\) is written as \(w=w_{0}\ldots w_{l-1}\).
Let \(\sigma:\Sigma\to\Reals_{+}\) be a function.
For any \(l\in\Reals_{+}\) and any
string \(w\in\Sigma^{l}\) we define \(\sigma(w)=\sum_{i=0}^{l-1}\sigma(w_{i})\).
Let \(S\in\Sigma^{n}\). For any \((j,l)\in[0,n-1]\times[1,n-j]\)
we define \(\Weight_{j,l}=\sigma(S_{j}\ldots S_{j+l-1})\), i.e.\
\(\Weight_{j,l}\) is the weight of the substring of \(S\) of length \(l\)
starting at position \(j\). Let \(\alpha,\beta\in\Reals_{+}\),
\(\alpha\leq\beta\) and \(\Weight\in[\alpha,\beta]\). The
\emph{\WSMP for} \(\Weight\) is to enumerate the
following set:
\[\Sol{\Weight}=\Set{(j,l)\mid j\in[0,n-1], l\in[1,n-j],\Weight_{j,l}=\Weight}\]

\section{Algorithms}
Cieliebak et.\ al \cite{CIE:ERL:LIP:STO:WEL:2001} describe several algorithms
for solving the \WSMP. The simplest of these
are the Algorithms \Linsearch and \Binsearch. This manuscript reports
on the implementation of these Algorithms or variations thereof.

\subsection{An $O(n)$ Time Algorithm}
Algorithm \Linsearch performs a linear search over the input string \(S\).
It manipulates three integers \(l\), \(r\), and \(z\) such that
\(0\leq l\leq r\leq n\)  and \(z=\Weight_{l,r-l}\). That is, \(z\) is the
weight of the substring \(S_{l}\ldots S_{r-1}\). In the initialization step
we set \(l=r=z=0\). The main loop of the algorithm iterates
the following case distinction, until \(r=n\):

\begin{itemize}
\item
If \(z<\Weight\), then \(z\) is incremented
by \(\sigma(S_{r})\) and \(r\) is incremented by \(1\).
\item
If \(z=\Weight\), then \((l,r-l)\in\Sol{\Weight}\). Hence \((l,r-l)\) is output.
Moreover, \(z\) is decremented by \(\sigma(S_{l})\) and \(l\) is incremented
by \(1\).
\item
If \(z>\Weight\), then \(z\) is decremented by \(\sigma(S_{l})\) and \(l\) is
incremented by \(1\).
\end{itemize}
The correctness of the algorithm is easily established.
In each step either \(l\) or \(r\) is incremented. Thus the main loop has at
most \(2(n+1)\) iterations. Each loop iteration takes constant time. Thus
the running time of \Linsearch is \(O(n)\).

\subsection{An $O(\log n^{2})$ Time Algorithm}
Algorithm \Binsearch, as suggested in \cite{CIE:ERL:LIP:STO:WEL:2001},
first preprocesses \(S\). It enumerates all \(\Weight_{j,l}\),
\(j\in[0,n-1]\), \(l\in[1,n-j]\), and stores all triples
\((\Weight_{j,l},j,l)\) in an array \(wtab\) which is sorted w.r.t.\ the first
component, i.e.\ the weight. Now \(wtab\) can be searched for a given weight
\(\Weight\), using two binary searches: One binary search
determines the smallest \(i\), say \(i_{l}\), satisfying
\begin{equation}
wtab(i)=(\Weight,j,l)\mbox{ for some }j\mbox{ and some }l,\label{Miexists}
\end{equation}
if it exists. Similarly, the other binary search determines the
largest \(i\), say \(i_{r}\), satisfying (\ref{Miexists}), if it exists.
The triples in \(wtab\) between the two boundaries \(i_{l}\) and \(i_{r}\)
give the solutions to the \WSMP for \(\Weight\). The size of \(wtab\) is
on the order of \(O(n^{2})\) in the worst case. Hence the running time
of \Binsearch is \(O(\log n^{2})\).

\subsection{An $O(1)$ Time Algorithm for Integer Weights}\label{OptimalAlgorithm}
We have developed a new method similar to Algorithm \Binsearch.
Our method makes the following assumptions:
\begin{itemize}
\item
The weights delivered by the weight function \(\sigma\) are non-negative
integers.
\item
The weights to be searched are in the range \(\lbrack\alpha,\beta\rbrack\)
for some fixed constants \(\alpha\) and \(\beta\). These are known in advance.
\end{itemize}
Note that \Binsearch does not makes these assumptions.

Our method naturally splits into two phases, a preprocessing phase and a
lookup phase. Hence we describe the two algorithms
\Vmbucket and \Vmctlook.\footnote{Cieliebak et.\ al
\cite{CIE:ERL:LIP:STO:WEL:2001} describe
an algorithm \emph{LOOKUP} which has nothing to do with \Vmctlook}
We exploit the fact, that the upper and lower boundaries \(\alpha\) and

Algorithm \Vmbucket performs the preprocessing step by
traversing the suffix tree for \(S\)
in a depth first strategy to efficiently collect all substring positions with
weight between \(\alpha\) and
\(\beta\). Whenever visiting a branching node \(\overline{v}\),
\Vmbucket maintains a stack holding the weights for all non-empty prefixes
of \(v\). The weight stack is used to determine the weight for all prefixes
of \(v\) (including \(v\)) whose length is larger than the depth of the
father of \(\overline{v}\). Call these prefixes the \emph{edge prefixes}
of \(v\). Leaves in the subtree below \(\overline{v}\) tell the position where
the prefix of \(v\) starts. For all weights \(\Weight\in[\alpha,\beta]\)
of the edge prefixes of \(v\), algorithm \Vmbucket stores the rank of the
leftmost leaf in the subtree below \(\overline{v}\). The rank refers to an
enumeration of all leaves in a depth first traversal of the suffix tree. That
is, the leftmost leaf gets rank 0, the next leaf reached in the depth
first traversal gets rank 1, etc. If \(i\) is the rank of the leftmost leaf,
then we say that \(i\) \emph{goes into bucket} \(B_{\Weight}\).

Whenever Algorithm \Vmbucket visits a leaf edge, say outgoing from
\(\overline{v}\), it takes the characters labeling the leaf edge and
incrementally adds up corresponding weights, until the end of
the label is reached or the weight becomes larger than \(\beta\).
Each leaf represents a suffix of \(S\$\) and hence for each weight
\(\Weight\in[\alpha,\beta]\) obtained in this way, Algorithm \Vmbucket stores
the start position, say \(j\), of this suffix in \(S\$\). We say that \(j\)
\emph{goes into bucket} \(B_{\Weight}\).

To efficiently obtain and retrieve the ranks and the start positions,
the suffix tree is traversed twice. In a first traversal Algorithm
\Vmbucket determines the size of each bucket \(B_{\Weight}\), i.e.\ the number
of ranks and suffix positions that go into bucket \(B_{\Weight}\). Then an array
\(B\) is allocated large enough to store all ranks and suffix positions.
From the size of the buckets, boundaries in \(B\) are determined where
the elements of each bucket are stored. For each \(\Weight\in[\alpha,\beta]\),
the elements of \(B_{\Weight}\) are stored beginning at position
\(b_{\Weight}=\sum_{i=\alpha}^{\Weight-1}\Size{B_{i}}\) in \(B\). That is, all
elements of \(B_{\Weight}\) will be stored in consecutive positions of \(B\),
and the buckets will be placed in consecutive order of their weights, where
\(\Weight\) is the weight of \(B_{\Weight}\). Finally, Algorithm \Vmbucket
again traverses the suffix tree and stores all elements into array \(B\)
between the corresponding bucket boundaries.
Algorithm \Vmbucket can be implemented such
that it runs in \(O(\beta-\alpha+\Size{B})\) time.

The bucket boundaries \(b_{\alpha},b_{\alpha+1},\ldots,b_{\beta-1},b_{\beta}\)
and array \(B\) are stored on a file. This is mapped into main memory, so
that all necessary information is readily available when searching for a
weight. Suppose we have to compute \(\Sol{\Weight}\) for
some \(\Weight\in[\alpha,\beta]\). Algorithm \Vmctlook determines
the bucket boundaries \(b_{\Weight}\) and \(b_{\Weight+1}\).
If \(\Weight=\beta\), then \(b_{\Weight+1}=b_{\Weight}+\Size{B_{\Weight}}\).
Then all \(B[l]\) for \(l\in[B_{\Weight},B_{\Weight+1}-1]\) are processed. If
\(B[l]\) stores a suffix position, say \(j\), then we know that a
substring of \(S\) with weight \(\Weight\) starts at position \(j\).
To determine the length of this substring we add up weights starting at
position \(j\) until we reach weight \(\Weight\).
If \(B[l]\) is a rank, say \(i\), then we know that the \(i\)th
leaf in the suffix tree represents a suffix which has a prefix with
weight \(\Weight\). This prefix, say \(u\), can easily be determined.
Suppose \(v\) is the shortest prefix of that suffix such that \(\overline{v}\)
is a branching node and \(v\) is a prefix of \(u\). By construction, the
leaf with rank \(i\) is the left most leaf in the subtree below
\(\overline{v}\). We only have to enumerate the other leaves in the subtree
below \(\overline{v}\) to obtain the positions where \(v\) occurs. This can be
done in time proportional to the number of these leaves. For each leaf
obtained in this way we determine the start position of the corresponding
suffix. This gives us the start positions of \(v\) and \(\sigma(v)=\Weight\).
The length of \(v\) is determined once for the suffix corresponding to the
left most leaf. For all other start positions of \(v\), the length is the same.

Algorithm \Vmctlook takes constant time to find the boundaries. Each of
the \(b_{\Weight+1}-b_{\Weight}\) integers (ranks or suffix positions)
between the boundaries are processed one after the other.
Note that any string with weight at most \(\beta\) is at most of length
\(\Rounddown{\beta/\sigma_{\min}}\)
where \(\sigma_{\min}=\min\Set{\sigma(a)\mid a\in\Sigma}\). Hence
each suffix position is processed in \(O(\Rounddown{\beta/\sigma_{\min}})\)
steps. This is independent of \(n\) and can thus be considered a constant. Each
integer representing a rank requires to find all, say \(q\), leaves in the
corresponding subtree. These can be found in \(O(q)\) time. Each leaf is
processed in constant time. Altogether \(\Sol{\Weight}\) is
determined in \(O(\Size{\Sol{\Weight}})\) time. This is optimal.
An important disadvantage of algorithm \Vmctlook is the fact that the size of
the precomputed bucket information is in \(O(n^{2})\). However, preliminary
experiments suggest that for the domain of protein identification the
size of the index grows linearly with \(n\).
\begin{comment}
For more details see
Section \ref{Experiments}.
\end{comment}

\section{Weights and Masses and their Representation}
The \WSMP is motivated by applications in protein identification. In this
application \(\Sigma\) is the alphabet of aminoacids and the weights are the
\emph{masses} of the aminoacids. Table \ref{Masstab} shows these masses.
Moreover, the assumption that only weights in some range
\([\alpha,\beta]\) are sought holds true. We currently assume that
\(\alpha=600\) and \(\beta=4000\). If you think this range is too
restricitve, please contact us.

\begin{table}
\begin{center}
\begin{small}
\begin{tabular}{llll}
\begin{tabular}{|l|r|}\hline
\texttt{G} & 57.02146\\\hline
\texttt{A} & 71.03711\\\hline
\texttt{S} & 87.03203\\\hline
\texttt{P} & 97.05276\\\hline
\texttt{V} & 99.06841\\\hline
\texttt{T} &101.04768\\\hline
\end{tabular}&
\begin{tabular}{|l|r|}\hline
\texttt{C} &103.00919\\\hline
\texttt{L} &113.08406\\\hline
\texttt{I} &113.08406\\\hline
\texttt{X} &113.08406\\\hline
\texttt{N} &114.04293\\\hline
\texttt{O} &114.07931\\\hline
\end{tabular}&
\begin{tabular}{|l|r|}\hline
\texttt{B} &114.53494\\\hline
\texttt{D} &115.02694\\\hline
\texttt{Q} &128.05858\\\hline
\texttt{K} &128.09496\\\hline
\texttt{Z} &128.55059\\\hline
\texttt{E} &129.04259\\\hline
\end{tabular}&
\begin{tabular}{|l|r|}\hline
\texttt{M} &131.04049\\\hline
\texttt{H} &137.05891\\\hline
\texttt{F} &147.06841\\\hline
\texttt{R} &156.10111\\\hline
\texttt{Y} &163.06333\\\hline
\texttt{W} &186.07931\\\hline
\end{tabular}
\end{tabular}
\label{Masstab}
\caption{The Masses of the Aminoacids}
\end{small}
\end{center}
\end{table}
For a given \emph{precision value} \(p\in[0,5]\),
we multiply the masses given in Table \ref{Masstab} by
\(10^{p}\) and round to the next integer. Then the maximal mass is
$18607931\leq 2^{25}$ and we represent it by a 32-bit integer. For
efficiency reasons, \(p\) is determined at compile time. Several other
constants depend on the choice of \(p\).
\begin{comment}
Hence we have implemented a
program \Showfile{mkmasstab.x} which outputs appropriate C-definitions,
see Appendix \ref{Mkmasout} for an example.
Besides the table storing the rounded masses, \Showfile{mkmasstab.x}
outputs constant definitions for the maximal and minimal masses
\(600\cdot 10^{p}\) and \(4000\cdot 10^{p}\). The output of
\Showfile{mkmasstab.x} is stored in a header file
\Showfile{masstab.h}, which is used by all programs described below.
\end{comment}

%\begin{comment}

\section{Programs}
We have implemented the algorithms described above. These and some
other useful programs related to the \WSMP are described in the following.
The distributions contains for all programs depending on \(p\),
all versions of the program for \(p\in[0-5]\). For example,
there are programs
\(\texttt{findmass0.x}\),
\(\texttt{findmass1.x}\),
\(\texttt{findmass2.x}\),
\(\texttt{findmass3.x}\),
\(\texttt{findmass4.x}\),
\(\texttt{findmass5.x}\). To always work with a consistent
collection of programs (for the same value of \(p\)), we recommend
to call the shell-script \texttt{Linkmassprogs.sh} with the appropriate
value for \(p\) as first argument. This creates softlinks
for all programs, so that you can use them as described below.
The examples given below are for \(p=2\).

\subsection{The Program \texttt{evalmass.x}}
This program takes a sequence of aminoacids and outputs the total mass
of this sequence.  For example:

\begin{verbatim}
$ evalmass.x GASPVTCLNOBDQKZEMHFRYW
262018
\end{verbatim}

\subsection{The Program \texttt{callmkvtree.sh}}
To solve the \WSMP, we first perform
an indexing step of the protein sequences to be searched. In
particular, we generate an enhanced suffix array of the protein sequences
(see \cite{ABO:KUR:OHL:2002}) using the program \Mkvtree.
We use the shell script \Showfile{callmkvtree.sh} to call \Mkvtree with the
proper arguments. The arguments to \Showfile{callmkvtree.sh} specify the
names of the files to be indexed (following the option \Showoption{db}). If
more than one file is to be indexed, then the additional
option \Showoption{indexname} followed by the name of the index must be
specified.

\paragraph{Example}
As a running example we suppose a file \Showfile{swiss1MB} containing a
subsection of the Swissprot protein sequence database.
Then we call the shell script \Showfile{callmkvtree.sh} as follows:

\begin{footnotesize}
\begin{verbatim}
reading file "swiss1MB"
total length of sequences: 1000120 (including 2549 separators)
alphabet of size 23: GASPVTCLNOBDQKZEMHFRYW*
creating file "swiss1MB.ssp"
creating file "swiss1MB.tis"
creating file "swiss1MB.ois"
creating file "swiss1MB.des"
creating file "swiss1MB.sds"
creating file "swiss1MB.lcp"
initializing data structures
sorting suffixes according to prefix of length 4
sorting all buckets
creating file "swiss1MB.llv"
creating file "swiss1MB.suf"
creating file "swiss1MB.prj"
creating file "swiss1MB.al1"
overall space peak: main=6.11 MB (6.40 bytes/symbol), secondary=1.08 MB
\end{verbatim}
\end{footnotesize}

Now we have the enhanced suffix array stored in different files
with the prefix \Showfile{swiss1MB}.

\begin{small}
\begin{verbatim}
$ ls -l swiss1MB.*
-rw-r-----    1 kurtz    users         171 Mar  7 22:08 swiss1MB.al1
-rw-r-----    1 kurtz    users      118867 Mar  7 22:08 swiss1MB.des
-rw-r-----    1 kurtz    users     1000121 Mar  7 22:08 swiss1MB.lcp
-rw-r-----    1 kurtz    users       85576 Mar  7 22:08 swiss1MB.llv
-rw-r-----    1 kurtz    users     1000120 Mar  7 22:08 swiss1MB.ois
-rw-r-----    1 kurtz    users         187 Mar  7 22:08 swiss1MB.prj
-rw-r-----    1 kurtz    users       10204 Mar  7 22:08 swiss1MB.sds
-rw-r-----    1 kurtz    users       10196 Mar  7 22:08 swiss1MB.ssp
-rw-r-----    1 kurtz    users     4000484 Mar  7 22:08 swiss1MB.suf
-rw-r-----    1 kurtz    users     1000120 Mar  7 22:08 swiss1MB.tis
\end{verbatim}
\end{small}
See \cite{KUR:2002A} for more details on the information contained in the
different files comprising the index.

\subsection{The Program \texttt{vmbucket.x}}
This program implements algorithm \Vmbucket, as described above.
However, while \Vmbucket is based on suffix trees,
\Showfile{vmbucket.x} it is based on enhanced suffix arrays as constructed by
\Mkvtree. In \cite{ABO:KUR:OHL:2002} it is shown how to simulate the depth
first traversal of the suffix tree on an enhanced suffix array.
\Showfile{vmbucket.x} computes the bucket information for a given index.

\begin{Showprogramwithoptions}{\Showfile{vmbucket.x}}{
\emph{indexname} is the name of the index produced by the
\Showfile{callmkvtree.sh}. Unless called with the option \Showoption{size},
\Showfile{vmbucket.x} generates an additional mass-index file
\Showfile{indexname.mas} storing the bucket information.}

\Option{minmass}{$\alpha$}{
Specify the minimal mass value $\alpha$. This must be in the range
$\Defaultrange$. If this option is not specified, then the default value
for $\alpha$ is $\Minmass$.
}

\Option{maxmass}{$\beta$}{
Specify the maximal mass value $\beta$. This must be in the range
$\Defaultrange$. If this option is not specified, then the default value
for $\beta$ is $\Maxmass$.
}

\Option{size}{}{
Only report the size of the bucket information, but do not generate it.}

\Option{help}{}{
Show a summary of all options and terminate.}

\end{Showprogramwithoptions}

\paragraph{Example (continued)}
Applying \Showfile{vmbucket.x} to the index \Showfile{swiss1MB}, we obtain the
following:

\begin{small}
\begin{verbatim}
number of bucket boundaries = 340002 (0.34 integers per input char)
suffix positions = 19054185 (19.05 integers per input char)
ranks = 2762138 (2.76 integers per input char)
total size of buckets = 22156325 (22.15 integers per input char)
size of index: 84.52 megabytes
create index file "swiss1MB.mas" of 84.52 megabytes
# space peak in megabytes: 83.22
# mmap space peak in megabytes: 5.81
\end{verbatim}
\end{small}

Now we have the mass-index file \Showfile{swiss1MB.mas} available.

\begin{small}
\begin{verbatim}
$ ls -l swiss1MB.mas
-rw-r-----    1 kurtz    users    107633732 Dec  4 01:10 swiss1MB.mas
\end{verbatim}
\end{small}

\subsection{The Program \texttt{findmass.x}}
This program implements the different algorithms to solve the
\WSMP, namely algorithm \Linsearch and
\Vmctlook. For each algorithm it provides different output formats.

\begin{Showprogramwithoptions}{\Showfile{findmass.x}}{}

\Option{search}{$alg$}{
specify the search algorithm.
If $alg$ is \texttt{lin}, then \Linsearch is used.
If $alg$ is \texttt{ctl}, then \Vmctlook is used. If this option
is not specified, then the default search algorithm is \Vmctlook.
}

\Option{trials}{$t$}{
run $t$ trials. In each trial, the \WSMP is solved for a randomly chosen
mass in the range $\lbrack \alpha,\beta\rbrack$.
This option is mainly used for testing purposes.
}

\Option{massfile}{$\emph{filename}$}{
read the masses from the file named \emph{filename}. This file
contains one mass per line and nothing else. For each mass value
found in the file, the \WSMP is solved.
}

\Option{output}{$outmode$}{
specify the form of the output.
\begin{itemize}
\item
If $outmode$ is \texttt{no}, then no
output is shown, but the search is still performed. This output mode is
mainly integrated to measure the running time of the program
without disturbing it by the generation of the output.
\item
If $outmode$ is \texttt{plain}, then all solutions to the
\WSMP are output, one per line.
\item
If $outmode$ is \texttt{group}, then all matches are grouped according
to the sequence in which they match. The groups are output ordered by their
size. For each group, its consecutive number, the corresponding sequence
number and its size is reported, as well as all matches ordered by their
relative position in the sequence.
\end{itemize}
If $outmode$ is \texttt{plain} or \texttt{group}, then an additional
optional argument \texttt{seq} triggers the additional output of the
matching sequence.
}

\Option{help}{}{
Show a summary of all options and terminate.}

\end{Showprogramwithoptions}

Exactly one of the two options \Showoption{trials} and
\Showoption{massfile} is mandatory.

\paragraph{Example (continued)}
Here is the partial output for several calls of \Showfile{findmass.x}.
The arguments of the program was called with are shown in the first line
of the output.

\begin{footnotesize}
\begin{verbatim}
# findmass.x -search ctl -output plain seq -trials 100 swiss1MB
# indexfile "swiss1MB.mas" (88625308 bytes) read
# each line reporting a match is in the following format:
# mass sequence_header relative_position match_length matching_sequence
# run 100 trials of masses in the range [60000,400000]
314415 sp|AEGP_RAT|APICAL  27   30 HCRSPTEATCNFVCDCGDCSDEAQCGFHGA
314415 sp|A2AB_DIDMA|ALPHA-2B 259   28 SVGPEDGSQKQEEEEEEEEEEEEECGPP
314415 sp|ADH1_KLUMA|ALCOHOL  95   30 GSCMSCEECELSNEPNCPKADLSGYTHDGS
383933 sp|41BB_MOUSE|4-1BB 182   34 SLQVLTLFLALTSALLLALIFITLLFSVLKWIRK
 97434 sp|ACH4_HUMAN|NEURONAL 222    8 YECCAEIY
 97434 sp|ACH4_RAT|NEURONAL 228    8 YECCAEIY
...
\end{verbatim}
\end{footnotesize}

If we use a different search mode, then the results are reported in a
order of the position of the matched in the indexed file.

\begin{footnotesize}
\begin{verbatim}
# findmass.x -search lin -output plain seq -trials 100 swiss1MB
# each line reporting a match is in the following format:
# mass sequence_header relative_position match_length matching_sequence
# run 100 trials of masses in the range [60000,400000]
314415 sp|A2AB_DIDMA|ALPHA-2B 259   28 SVGPEDGSQKQEEEEEEEEEEEEECGPP
314415 sp|ADH1_KLUMA|ALCOHOL  95   30 GSCMSCEECELSNEPNCPKADLSGYTHDGS
314415 sp|AEGP_RAT|APICAL  27   30 HCRSPTEATCNFVCDCGDCSDEAQCGFHGA
383933 sp|41BB_MOUSE|4-1BB 182   34 SLQVLTLFLALTSALLLALIFITLLFSVLKWIRK
 97434 sp|A2HS_BOVIN|ALPHA-2-HS-GLYCOPROTEIN  30    9 ACDDPDTEQ
 97434 sp|A2HS_BOVIN|ALPHA-2-HS-GLYCOPROTEIN  31    9 CDDPDTEQA
...
\end{verbatim}
\end{footnotesize}

If we choose to group the output, then the output is independent of
the chosen search mode.

\begin{footnotesize}
\begin{verbatim}
# findmass.x -search lin -output group -trials 100 swiss1MB
# each line reporting a match is in the following format:
# mass sequence_header relative_position match_length
# run 100 trials of masses in the range [60000,400000]
# output 2159 groups in order of their size
# group 0 for sequence 1671 has 38 elements
  288629 sp|ACVS_CEPAC|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 116   29
  126958 sp|ACVS_CEPAC|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 133   13
  124665 sp|ACVS_CEPAC|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 312   11
...
# group 1 for sequence 1673 has 36 elements
  232817 sp|ACVS_NOCLA|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 130   20
  124360 sp|ACVS_NOCLA|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 278   10
  355276 sp|ACVS_NOCLA|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 380   34
...
# group 2 for sequence 1672 has 33 elements
  149685 sp|ACVS_EMENI|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 268   14
  213109 sp|ACVS_EMENI|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 476   20
  213109 sp|ACVS_EMENI|DELTA-(L-ALPHA-AMINOADIPYL)-L-CYSTEINYL 477   20
...
\end{verbatim}
\end{footnotesize}

\section{Release Notes}
\subsection{Changes made on March 7, 2003}
\begin{enumerate}
\item
change minimum mass from 800 to 600
\item
compile programs for different values of \(p\) and add
helpful script \texttt{Linkmassprogs.sh}
\item
output original protein sequence instead of transformed sequence
\item
output sequence headers (up to the first blank) instead of
sequence numbers
\item
update manual accordingly
\end{enumerate}

\section{Preliminary Experimental Results}\label{Experiments}
In our experiments we have applied the programs from above to six protein
files: \Showfile{swiss.$i$MB.fna}, for \(i\in[1,5]\) and \Showfile{sprot38}.
The latter is the complete swissprot database release 38.
Each file \Showfile{swiss.$i$MB.fna} is a collection of randomly selected
protein sequences from \Showfile{sprot38} such that the total length of
the sequences is about \(i\) megabytes.
Table \ref{PPresults} shows the results when generating the index
for the different input files. In particular,
file sizes, number of suffix positions, number of ranks, and index sizes
are shown.
In all cases, the number of bucket boundaries is
\(10^{p}(\beta-\alpha)+1+1=10^{2}(4000-600)+2=340002\). Note that the
index size does not include the size of the
files produced by \Mkvtree. These are however small compared to the
mass-index file. Similarly, the construction time for the suffix array is
small compared to the construction of the mass index.
Table \ref{Findresults} shows the running time of the different searching
algorithms for the different indexes.

\begin{table}
\begin{center}
\begin{small}
\begin{tabular}{|*{6}{r|}}\hline
\emph{file}&$n$&\emph{suffix positions}$/n$&\emph{ranks}$/n$&\emph{size of index} (MB)&\emph{time}\\\hline
\Showfile{swiss1MB} & 1002835 & 26.17 & 0.35 & 102.65&10.18\\\hline
\Showfile{swiss2MB} & 2006050 & 25.68 & 0.53 & 201.80&21.61\\\hline
\Showfile{swiss3MB} & 3008338 & 25.36 & 0.66 & 299.93&33.43\\\hline
\Showfile{swiss4MB} & 4011052 & 25.03 & 0.80 & 396.37&38.02\\\hline
\Showfile{swiss5MB} & 5013972 & 24.64 & 0.93 & 490.36&54.73\\\hline
\Showfile{sprot38}  &31791220 & 20.59 & 2.19 &2764.05&??
\\\hline
\end{tabular}
\end{small}
\end{center}
\caption{Results when computing the mass-index. The first column shows
the file, the second its size. The next columns show the number of
stored suffix positions as well as the number of ranks, relative to the
size of the file. The last two column show the total size of the mass-index
as well as the time to construct it. The reported time include the
output to a file. All time results are in seconds and refer to a computer
with a 1123~Mhz Pentium III processor and 614 MB RAM running Linux.}
\label{PPresults}

\begin{center}
\begin{small}
\begin{tabular}{|*{5}{c|}}\hline
           &\multicolumn{2}{c|}{\Linsearch}
           &\multicolumn{2}{|c|}{\Vmctlook}\\\hline
\emph{file}&\emph{time}&\emph{space}&\emph{time}&\emph{space}\\\hline
\Showfile{swiss1MB}&182.44 &0.97 &0.52&108.40\\\hline
\Showfile{swiss2MB}&374.83 &1.93 &1.11&213.30\\\hline
\Showfile{swiss3MB}&556.20 &2.90 &1.29&317.18\\\hline
\Showfile{swiss4MB}&746.24 &3.87 &2.10&419.39\\\hline
\Showfile{swiss5MB}&935.78 &4.83 &2.18&519.13\\\hline
\end{tabular}
\end{small}
\end{center}
\caption{Results when searching 10000 masses that are randomly sampled from
the interval $[60000,400000]$. The first column shows the file.
The second and third column show the running time (in seconds) and space
requirement (in MB) for the \Linsearch algorithm.
The fourth and fifth column show the running time (in seconds) and space
requirement (in MB) for the \Vmctlook algorithm.}
\label{Findresults}
\end{table}
%\end{comment}

\bibliographystyle{plain}
\bibliography{defines,strings,kurtz}

\begin{comment}
\newpage
\appendix
\section{The output of \texttt{mkmasstab.x 2}}\label{Mkmasout}
\begin{footnotesize}
\begin{verbatim}
#ifndef MASSTAB_H
#define MASSTAB_H

        /* This file is generated by mkmas.x 2, do not edit */

        /* the precision value */
#define PRECISIONVALUE 2
        /* number of aminoacids */
#define NUMOFAMINOACIDS 22
        /* minimal mass */
#define MINMASS 60000
        /* maximal mass */
#define MAXMASS 400000
        /* size of table countmasstab */
#define SIZECOUNTMASSTAB (MAXMASS-MINMASS+1+1)
        /* number of digits to output mass */
#define DIGITSFORMAXMASS 6
        /* aminoacids ordered by mass */
#define AMINOCHARACTERS "GASPVTCLNOBDQKZEMHFRYW"

#define MASSFORMAT "%*u"
        /* type for masses */
typedef unsigned int Masstype;

        /* defining this means to ignore masstab */
#ifndef IGNOREMASSTAB

        /* The masses of the amino acids */
static Masstype masstab[] =
{
  5702,  /* G -> 0 */
  7104,  /* A -> 1 */
  8703,  /* S -> 2 */
  9705,  /* P -> 3 */
  9907,  /* V -> 4 */
  10105,  /* T -> 5 */
  10301,  /* C -> 6 */
  11308,  /* L -> 7 */
  11404,  /* N -> 8 */
  11408,  /* O -> 9 */
  11453,  /* B -> 10 */
  11503,  /* D -> 11 */
  12806,  /* Q -> 12 */
  12809,  /* K -> 13 */
  12855,  /* Z -> 14 */
  12904,  /* E -> 15 */
  13104,  /* M -> 16 */
  13706,  /* H -> 17 */
  14707,  /* F -> 18 */
  15610,  /* R -> 19 */
  16306,  /* Y -> 20 */
  18608  /* W -> 21 */
};

#endif
#endif
\end{verbatim}
\end{footnotesize}
\end{comment}

\end{document}
\newpage

\section{Ideas and Future Work}
\begin{enumerate}
\item
Divide \(S\) into non-overlapping sections of length \(n/2^{16}\). Instead of
storing a suffix position only store the section in which the suffix position
starts. The section numbers can be stored in 16 bits. This would reduce the
space requirement for the suffix positions by 50\%. Searching would require
to search a complete section using \Linsearch. However, applying
\Linsearch on a small section will not cost too much. Note that
in each bucket a position can only be stored once.
\item
Compute the smallest common multiple of all pairs of weights to determine
if the partitions of the weights is unique.
If the scm of two weights is smaller, then the partition is is not unique.
\item
The index becomes so large due to the large number of suffix positions
to be stored. The basic problem is for each mass to find the sets
of leaf-weights it is contained in. Each set of leaf weights can be
stores in constant space. But this does not allow fast retrieval.
\item
If there are two isomorphic subtrees below \(\overline{aw}\) and
\(\overline{w}\), then the masses in the subtree are identical up to
the addition with \(\sigma(a)\).
\item
Encode the precision value in the index. Check if the programs
fits to the index.
\end{enumerate}

kurtz@mahagoni[57] time massbck -size sprot38.fas
countleaf = 654521611 (20.59 per input char)
countfirstbranch = 69737236 (2.19 per input char)
total number of integers = 724258847 (22.78 integers per input char)
size of index: 2764.05 megabytes
# space peak in megabytes: 0.00
# mmap space peak in megabytes: 184.01
160.79u 1.14s 2:41.96 99.9%


\documentclass[../../../../main]{subfiles}
\graphicspath{{images/sfm/}}

\begin{document}

\paragraph{Structure from motion}

Structure from motion problem is the case of finding a set of 3D points with projection matrices and translation vectors for corresponding views from the set of images. This process is a 3D reconstruction from a sequence of images.

Sequential methods are widely used, they are iterative thus reconstruction is done partially, step by step. The process starts with the first image, when the new image is registered algorithm processes new portion of data, performs triangulation and adds new 3D data to the reconstruction model. Initialization is usually achieved by finding fundamental matrix from the first two views and decomposing it \cite{RichardSzeliski2010}.

These methods have some complications, first of all, they require huge amount of corresponding points per each image in a sequence. Usually seven or more correspondences must be present at three or more views. Large set of images require too much computational power in order to process these. Secondly, there is a number of structure and motion combinations that are not appropriate for the mentioned methods. These cases might be the camera rotation without any translation or planar scenes. It is impossible to avoid those cases without an expert planning on how to take pictures for the structure from motion sequence \cite{sfm_theia}. However grouping of pictures by the feature similarities might be a partial answer to the raised problem.

The most common strategy for registering images is epipolar constraints. It is achieved by using the correspondence of the image from the current view to the image of the previous view. Essential matrix is typically used but intrinsic camera parameters must be known. Its decomposition gives relative camera orientation and translation vector. Figure \ref{fig:incrementalSfm} illustrates iterative nature of incremental structure from motion.

\begin{figure} [ht]
    \begin{center}
        \includegraphics[width=200pt]{incremental_sfm}
        \caption{Demonstration of incremental structure from motion \cite{sfm_theia}.}
        \label{fig:incrementalSfm}
    \end{center}
\end{figure}

On the other hand, factorization methods do the job simultaneously. These methods belong to the family of batch methods. \ac{SVD} factorization based linear methods have been created for many affine camera models like orthographic, para-perspective or weak perspective and etc. These methods distribute reconstruction error among all measurements but, unfortunately, they are not applicable for the real world situations because camera lenses have too wide angle thus cannot be approximated as linear \cite{factorization_sfm}.

Lastly, after receiving initial estimations for 3D points and projection matrices it is needed to minimize function cost with the non-linear iterative optimization. This optimization is called bundled adjustment. It, basically, refines camera and structure parameters initial estimations to get those parameters that predict the locations of features among all images in the most efficient way.

\begin{figure} [!ht]
  \centering
    \begin{equation}
        \underset{a_j b_i}{min} = \sum_{i=1}^{n} \sum_{j=1}^{m} v_{i j} d(Q(a_j, b_i), x_{i j})^2
        \label{bundleAdjustmentFormula}
    \end{equation}
  \caption{Bundle adjustment minimization function.}
\end{figure}

In the equation \ref{bundleAdjustmentFormula} $a$ is a vector that parametrizes camera and $b$ is a vector for 3D point so that $Q(a_j, b_i)$ is the prediction of $i$ point projection onto $j$ image. Euclidean distance is represented as $d(,)$ where its parameters are vectors.

\end{document}


\section{Shock wave simulation}
We studied the influence of fractions of heavy ions on the shock wave, in particular, the spectrum of particles and the shape of the shock wave. We present results of two simulations with different compositions of the plasma : pure protons and electrons in the first case and with the admixture of alpha particles in the second case. Initially the homogeneous flux is  moving from the right free boundary to the left. On the left side there is a reflecting super-conducting wall, which causes formation of shock wave. Simulations are one-dimensional and have following parameters: initial velocity $v = 0.2c$, number densities $n_e = 10^{-4} \rm{cm}^{-3}$, $n_p = 10^{-4} \rm{cm}^{-3}$ in the first simulation and $n_e = 10^{-4} \rm{cm}^{-3}$, $n_p = 0.6\cdot10^{-4} \rm{cm}^{-3}$, $n_\alpha = 0.2\cdot10^{-4} \rm{cm}^{-3}$ in the second, temperature $5\cdot10^8 \rm{K}$, magnetic field $B_\parallel = 10^{-4} \rm{G}, B_\perp= 0.7\cdot10^{-4} \rm{G}$, the full size of the box $L = 1\cdot10^{12} \rm{cm}$, the number of cells $N=2\cdot10^4$. Electron mass is reduced to $m_e = \frac{m_p}{20}$. The full time of simulation is $T = 2000 {\omega_p}^{-1}$.
\begin{figure}[h!]
	\centering
	\begin{minipage}{0.49\textwidth}
		\centering
		%\includegraphics[width=0.98\textwidth]{fig/protons.eps}
		\caption{Distribution of protons in the simulation without alpha particles.}
		\label{protons}
	\end{minipage}\hfill
	\begin{minipage}{0.49\textwidth}
		\centering
		%\includegraphics[width=0.98\textwidth]{fig/protons_with_He.eps}
		\caption{Distribution of protons in the simulation with alpha particles.}
		\label{protons_with_alpha}
	\end{minipage}
\end{figure}
\begin{figure}[h!]
	\centering
	\begin{minipage}{0.49\textwidth}
		\centering
		%\includegraphics[width=0.98\textwidth]{fig/electrons.eps}
		\caption{Distribution of electrons in the simulation without alpha particles.}
		\label{electrons}
	\end{minipage}\hfill
	\begin{minipage}{0.49\textwidth}
		\centering
		%\includegraphics[width=0.98\textwidth]{fig/electrons_with_He.eps}
		\caption{Distribution of electrons in the simulation with alpha particles.}
		\label{electrons_with_alpha}
	\end{minipage}
\end{figure}

The results presented in Figures \ref{protons}-\ref{electrons_with_alpha} show that the values of $F(p)p^4$ for high energies are much greater for protons, than for electrons in both simulations. Also electrons need much more time to be accelerated and to form spectrum $\propto p^{-4}$. It means that protons are injected into the acceleration process more efficiently, and it is consistent with the work of Park et al. {\cite{Park2015}}. Also we have shown that minority of heavy ions increases the spectrum of protons and do not have influence on the spectrum of electrons. It can be explained by the fact, that heavy ions form the large scale turbulence and protons can efficiently scatter on this turbulence. Otherwise for electrons turbulence produced by protons is already enough large-scale  and the influence of heavy ions is neglectable.


\documentclass{article}

\usepackage{graphicx}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{authblk}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{titling}
\usepackage{float}
\usepackage{mathtools}
\usepackage{tabularx}

\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}

\newtheorem{theorem}{Theorem}[section]
\newtheorem{proposition}{Proposition}[section]
\newtheorem{corollary}{Corollary}[theorem]
\newtheorem{lemma}[theorem]{Lemma}

\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]

\theoremstyle{definition}
\newtheorem{example}{Example}[section]

\newtheorem{xca}[theorem]{Exercise}

\newtheorem{remark}[theorem]{Remark}

\setlength{\abovedisplayskip}{5pt}
\setlength{\belowdisplayskip}{5pt}
\setlength{\abovedisplayshortskip}{5pt}
\setlength{\belowdisplayshortskip}{5pt}

\renewcommand*{\Affilfont}{\normalsize\normalfont}
\renewcommand*{\Authfont}{\bfseries}

\newcounter{protocol}
\newenvironment{protocol}[1]
  {\par\addvspace{\topsep}
   \noindent
   \tabularx{\linewidth}{@{} X @{}}
    \hline
    \refstepcounter{protocol}\textbf{Protocol \theprotocol} #1 \\
    \hline}
  { \\
    \hline
   \endtabularx
   \par\addvspace{\topsep}}

\newcommand{\sbline}{\\[.5\normalbaselineskip]}% small blank line

\title{Algorithmic Governance Design for Cryptocurrency Protocols}

\author{Drew Stone}

\begin{document}
\maketitle
\begin{abstract}
We tryna do adaptive, autonomous governance yo.
\end{abstract}
\section{Introduction}
Governance is a topic that spans numerous fields like political science, economics, and, for our matter, computer science. In the world of blockchains, governance is the topic that embodies how cryptocurrencies evolve and grow. Participants act in a variety of ways according to ever-changing incentives according to a mechanism that defines the feasible set of actions. Oftentimes, blockchain governance garners dogmatic behavior from maximalists of different currencies, who flout the benefits of certain governance structures. In this paper, we discuss this and try to ascertain whether this behavior carries any weight when designing incentivized technology. We systematically build governance mechanisms using tools from algorithmic game theory, mechanism design, and computer science. Specifically, we formalize a number of governance objectives and solve them using distributed optimization techniques.

\section{Background}
Blockchain governance in the context of this paper describes the set of interactive mechanisms that control a cryptocurrency protocol's evolution. For example, in existing blockchains, miners or validators selectively decide which transactions to process by selecting the highest-paying transactions (or none at all). In other protocols, miners and validators decide how large blocks should be within certain ranges. The mechanisms available to these agents largely define the dynamics of the underlying protocol.

For settings such as these, we, as the mechanism designers, want to build a governance mechanism for blockchain protocols that optimize certain properties. First and foremost is defining the mechanism designer's goal.

\begin{definition}
A \textit{blockchain governance mechanism designer} $\mathcal{M}(\mathcal{P})$ is primarily concerned with finding a feasible policy $\pi^*\in \Pi_f$ over the system's state $s\in \mathcal{S}$ that maximizes the following quantities:
\begin{itemize}
    \item The throughput or transactions per second (\text{\fontfamily{lmss}\selectfont tps}) of the protocol $\mathcal{P}$.

    \item The decentralization (\text{\fontfamily{lmss}\selectfont dec}) of the protocol $\mathcal{P}$.
\end{itemize}
\end{definition}
\begin{proposition}
$\mathcal{M}(\mathcal{P})$ is concerned with finding a policy $\pi^*\in \Pi_f$ for any state $s\in\mathcal{S}$ that optimizes an objective function $\varphi$ of the quantities above:
\begin{align*}
    \pi^*(s) \in \argmax_{\pi \in \Pi_f}
    \varphi(\text{\fontfamily{lmss}\selectfont tps($\mathcal{P}(\pi))$}, \text{\fontfamily{lmss}\selectfont dec($\mathcal{P}(\pi))$})
\end{align*}
\end{proposition}
Previous work \cite{previous work} has shown that increasing the throughput of many protocols leads to both a decrease in security and decentralization. This is due largely in the heterogeneity of participants in the protocol who will fail to keep up with higher throughput demand. As a result, less participants secure the information contained in the ledger and the network becomes more susceptible to malicious attacks such as 51\% attacks.

While much of the research in the blockchain space has focused on building protocols whose tradeoff between throughput and decentralization is less severe along any axis, none of the research has tried to optimize the governance policy utilized for balancing any such tradeoff. By casting the protocol designer's objective as a multi-objective optimization problem, any interested designer can select the tradeoff within the hyperparameters of the model. Therefore, finding an optimal policy for a given set of hyperparameters remains the main goal.

\section{A model for governance mechanisms}
\begin{definition}
We define a blockchain protocol $\mathcal{P}$ over an abstract state space $B$ to be a tuple $\langle \mathcal{V}, \mathcal{O}, P, \pi, \mathcal{R}\rangle$ such that at any round $t$ there exists the following:
\begin{itemize}
    \item A set of $n$ active nodes $V\subseteq \mathcal{V}$.
    \item A leader election oracle $\mathcal{O}(V)$ with distribution $\mathcal{D}$.
    \item A payment rule $P:B^n\longrightarrow \mathbb{R}^n$.
    \item A governance rule $\pi:B^n\longrightarrow \mathcal{A}$ where $\mathcal{A}$ denotes the set of alternatives.
    \item A reward $\mathcal{R}$ for the leader elected at each round.
\end{itemize}
\end{definition}

\begin{definition}
A \textit{miner, validator, node} or \textit{agent} $i\in V$ is a participant of a blockchain protocol with the following properties and responsibilities:
\begin{itemize}
    \item $i$ has some probability of being selected to mine a block $\mathcal{O}(i)\in [0,1]$ such that $\sum_{i\in V}\mathcal{O}(i) = 1$.
    \item $i$ has private, local information $b_i\in B$.
    \item $i$ has a valuation function over alternatives $v_i: \mathcal{A}\longrightarrow \mathbb{R}$.
    \item $i$ has some utility $\forall \textbf{b}=(b_1,\cdots,b_n)\in B^n$ defined with respect to a given payment and governance rule.
    \begin{align*}
        u_i(\pi(\textbf{b}),P(\textbf{b})) = \mathcal{R}\cdot\textbf{I}[\mathcal{O}(V)=i] + v_i(\pi(\textbf{b})) - P(\textbf{b})_i
    \end{align*}
    \textit{We use the subscript $i$ notation over vectors to denote the value at the $i$'th index of the vector.}
\end{itemize}
\end{definition}

\subsubsection{Ideal functionality}
\begin{definition}
We call $\mathcal{P}$ \textit{incentive-compatible} if $\forall i\in V$ with private state $\textbf{b}\in B^n,~\forall b'\in B$, the following holds:
\begin{align*}
    \mathbb{E}[u_i(\pi(\textbf{b}), P(\textbf{b}))] \geq \mathbb{E}[u_i(\pi(b',\textbf{b}_{-i}), P(b', \textbf{b}_{-i}))]
\end{align*}
\textit{i.e. for any reports of private states, a node always maximizes its expected utility by reporting its true private state when all other agents do so as well.}
\end{definition}
\begin{definition}
We call $\mathcal{P}$ \textit{individually-rational} if $\forall i\in V$ with private state $\textbf{b}\in B^n$, the following holds:
\begin{align}
    \mathbb{E}[u_i(\pi(\textbf{b}), P(\textbf{b}))] \geq 0
\end{align}
\end{definition}

As is typical with mechanism design, our ideal functionality is an incentive compatible and individually rational protocol $\mathcal{P}$. We could also strive for budget-balanced or feasible mechanisms, defined as never running a deficit with respect to payments, but we have access to a monetary minting process that provides new functionality than traditionally available. We gloss over this concept primarily because a blockchain protocol allows us to mint a quantity $\mathcal{R}$ of money each round. Therefore, it is not unreasonable to imagine the mechanism using inflationary processes to balance incentives with payment rules that incur deficits.

\subsubsection{A taxonomy of governance rules}
\begin{definition}
We call $\pi$ a \textit{constant} governance rule if at each round $t$ and $\forall \textbf{b}\in B^n$, $\pi$ always outputs the same alternative $a\in \mathcal{A}$:
\begin{align*}
    \pi(\textbf{b}) = a
\end{align*}
\end{definition}
This describes the governance mechanism of Bitcoin since it requires a hard fork--through a new mechanism--to change the fixed alternative/block size chosen. There is no adaptability to different environments and as such presents both interesting and contentious discussion in different ecosystems.
\begin{definition}
We call $\pi$ a \textit{leader-based} governance rule if at each round $t$ and $\forall \textbf{b}\in B^n$, $\pi$ always outputs the alternative signalled by the chosen leader:
\begin{align*}
    \pi(\textbf{b}) &= b_{i}\text{ such that }i = \mathcal{O}(V)
\end{align*}
\textit{i.e. the governance rule selects the information/alternative signalled by the elected leader $\mathcal{O}(V)$. For simplicity, let $\mathcal{A}=B$ so that leaders bid for their desired future alternatives. If leaders are rational or "selfish" then we get that:}
\begin{align*}
\pi(\textbf{b}) = a^*\text{ such that }a^* \in \argmax_{a\in \hat{\mathcal{A}}} v_i(a)
\end{align*}
\textit{where $\hat{\mathcal{A}}$ denotes the feasible alternative set according to some globally known restriction.}
\end{definition}
In Ethereum for example, the miner $i$ that wins the proof of work process can set the next block gas limit within some bounded range with respect to the previous block's gas limit, usually 0.1\% higher or lower. Therefore, this is analogous to a protocol $\mathcal{P}$ receiving the vector $(\_,\dots,b_i,\dots,\_,)$ where the $i$'th entry corresponds to miner $i$ participating in the protocol. While we abuse notation since in proof of work blockchains we don't exactly know how many miners are participating, it is sufficient that we know there is some number $n$ of participants as described in the mechanism and only one is elected as a leader each round.
\begin{definition}
We call $\pi$ a \textit{social-welfare maximizing} governance rule if at each round $t$ and $\forall \textbf{b}\in B^n$, $\pi$ always outputs an alternative that maximizes (resp. minimizes) the valuations (resp. costs) of all participants:
\begin{align*}
    \pi(\textbf{b}) &= a^*\text{ such that } a^*\in \argmax_{a\in \hat{\mathcal{A}}} \sum_{i\in V} v_i(a)
\end{align*}
\end{definition}
\section{Single Parameter Governance}
We begin our search for \textit{incentive-compatible} blockchain protocols over single parameter governance rules where $\mathcal{A}=\mathbb{R}$ or $\mathbb{Z}$. Single parameter governance rules are useful for determining parameters such as a block size or a block gas limit each round. Additionally, we are interested in mechanisms that incentivize truthfulness of reporting private state.

\subsubsection{Single peaked preferences}
We consider a model where agents have single peaked preferences over alternatives $a\in \mathcal{A}$ with the following structure:
\begin{align*}
v_i(a) \begin{cases}
      a & a\leq b_i \\
      0 & otherwise
   \end{cases}
\end{align*}
\begin{definition}
For each agent $i\in V$, the value $b_i$ as used above is known as the \textit{bliss point}. We will often describe the agent's \textit{bliss point} as their \textit{tolerance} or \textit{capacity}.
\end{definition}

Since agents have single-peaked utilities at the bliss points $\textbf{b} = (b_1,\dots, b_{|V|})$, we can consider the agents in sorted order by their points: $b_1\leq b_2\leq \dots\leq b_{|V|}$. Oftentimes, we represent bold-faced variables such as \textbf{b} and \textbf{v} as sets (in most cases ordered), so that an agent $i$'s report $b_i\in \textbf{b}$ and valuation $v_i\in\textbf{v}$.

\subsection{Mechanisms without dropout}
We start by considering a model with the following premise: if the current parameter is $\hat{b}$, any agent $i\in V$ with bliss point $b_i< \hat{b}$ does not drop out of the mechanism. They still provide their bids, albeit earning no positive utility.
\subsubsection{Median Mechanisms}
Given single-peaked preferences over alternatives, a natural thing to analyze is the median mechanism, which chooses the alternative denoted by the median bliss point over a set of agents or voters $V$. This immediately leads us to our first mechanism and the following proposition:

\begin{theorem}[\textbf{Moulin 1980}]
The Median mechanism over single-peaked preferences \textbf{v} that selects the median bliss point $b_{median}$ is truthful and Pareto-efficient.
\end{theorem}
% \begin{proof}
% Using a plurality voting protocol, where agents vote for a single alternative, the median alternative chosen coincides with the median bliss point and bliss points are reported truthfully as follows.

% Consider a voter $i$ whose bliss point $b_i<b_{median}$, i.e. voter $i$ falls on the left of voter $median$ according to the order of \textbf{b}. The only way for $i$ to manipulate the vote is by reporting $\hat{b}_i > b_{median}$ since anything different but still to the left of $b_{median}$ will result in the same median. However, if $i$ reports, untruthfully, a value greater than $b_{median}$ it will increase the median, leading to a weakly worse-off outcome since $v_i(a)=0,~\forall a> b_i$. The same argument proves truthfulness in the reverse direction and so we have truthfulness. The median is also Pareto-efficient from this analysis, since any change in a vote that might make one voter better off, certainly makes another voter worse off.

% Instead of a plurality voting protocol, we can also consider a majority voting protocol over the entire preference profiles of reporters. Given a feasible, ordered set of alternatives $\textbf{a}=\{a_1,\cdots, a_k\}\subseteq \mathcal{A}$ such that $\textbf{b}\subseteq \textbf{a}$, for each agent $i\in V$, we can represent their preferences as
% $$
% v_i(a_{i_1}) \preceq v_i(a_{i_2}) \preceq \cdots \preceq v_i(b_i)
% $$
% Then since \textbf{b} is ordered, the median reporter's preference profile restricted to \textbf{b} looks like the following. Let $m$ denote the index of the \textit{median} reporter.
% $$
% v_{m}(b_{m+1}) \preceq v_{m}(b_{m+2}) \preceq \cdots \preceq v_{m}(b_{m})
% $$
% From single-peakedness, we know $v(b_{m+i})=0,~\forall i>0$. We also know that each reporter of index $m+i,~\forall i > 0$ prefers $b_m$ to any lower bliss point. Therefore, the median report $b_m$ has the majority of votes according to the preference profiles of all reporters; that is, the median receives at least half the votes, coinciding with exactly half the votes when all bliss points are distinct.
% \end{proof}

Contrary to median mechanisms in other domains \cite{DBLP:journals/corr/BlumrosenD16}, the median mechanism may perform very poorly under certain distributions of bliss point vectors or player capacities. This occurs when the capacity of a single agent far outweighs the capacity of the rest of the agents. Therefore, while the median ensures the number of players with non-zero utility is at least half and that they report truthfully, it does not guarantee that any approximation to optimal social welfare under any distribution.

\begin{proposition}
There exist distributions $\mathcal{D}$ over bliss points \textbf{b} such that the Median mechanism is not a 2-approximation of optimal social welfare.
\end{proposition}
\begin{proof}
Given a bliss point vector \textbf{b} = $\{1,2,k\},~\forall k > 8$, the median mechanism does substantially worse than $\frac{1}{2}$ the optimal social welfare as $k\longrightarrow \infty$. The median $\forall k
\geq 2$ will be 2, leading to a social welfare of $4$. Therefore, for any value $k > 8$, the median mechanism achieves less than half the optimal welfare which is $k$.
\end{proof}

A natural question to ask is under what assumptions do median mechanisms perform well with respect to the optimal social welfare. We now introduce some definitions that will allow us to design a modified median mechanism and argue for a 2-approximation to optimal social welfare.
\begin{definition}
The \textit{Weighted Median} mechanism is defined over an ordered multi-set of weighted bliss points \textbf{b} = $\{(b_1, w_1),\cdots,(b_n, w_n)\}$ outputs the element $b_k$ that is incident where at most half the weight falls.
$$
b_k = \sup\{b_i~|~\sum_{j=1}^i w_j \leq \frac{\sum_{j=1}^n w_j}{2}\}
$$
\end{definition}
\begin{proposition}
For weight distributions $\mathcal{D}$ proportional to a node's bliss point or capacity where no player has $\geq 50\%$ of weight, the Weighted Median mechanism achieves a 2-approximation to optimal social welfare.
\end{proposition}
\begin{proof}
Let the weights \textbf{w} be defined for each player as
$$
\forall i\in V,~w_i = \frac{b_i}{\sum_{j} b_j},~w_i\leq \frac{1}{2}
$$. We want to show that if $b_k=\pi(\textbf{b})$ that $\sum_{i\in V}v_i(b_k)\geq \frac{1}{2} OPT$. If $\exists w_j = \frac{1}{2}$, notice that $b_k=b_{j-1}$. The social welfare is then
$$
\sum_{i\in V} v_i(\pi(\textbf{b})) \geq 2\cdot \pi(\textbf{b})
$$
Since $b_j \leq 2\cdot \pi(\textbf{b})$ by definition, we have that the welfare attained by $\pi(\textbf{b})$ is at least $\frac{1}{2}OPT$.

If no bliss point has half the weight, then this implies that it is closer to $\pi(\textbf{b})$. Therefore, if $\pi(\textbf{b})$ is not the optimal bliss point, the bliss point yielding the optimal welfare yields the following:
\begin{align*}
OPT &\leq 2\cdot \sum_{i\in V} v_i(\pi(\textbf{b})) \\
\frac{1}{2} OPT &\leq \sum_{i\in V} v_i(\pi(\textbf{b}))
\end{align*}
\end{proof}
This claim tells us that we have a good approximation to the optimal social welfare when we have the assumptions set forth by blockchain protocols. We usually assume that no more than half of the computational power concentrates in the hands of a single individual. Therefore, as long as that holds over a Proof of Work based cryptocurrency, the median selection of parameters will give us a 2-approximation of the optimal social welfare.

\subsubsection{VCG Mechanisms}
In many settings, especially if there is a cost to reporting one's bliss point, we need to properly incentivize the reports while also preserving truthfulness. Here, we turn another realm of mechanism design that uses money to elicit reports from players such that the reports are truthful.
\begin{theorem}
The VCG mechanism over bids \textbf{b} is truthful, individually rational, and social-welfare maximizing with $\pi(\textbf{b})\in \argmax_{b\in \textbf{b}} \sum_{i\in V} v_i(b)$. If $i^*$ is the index of $\pi(\textbf{b})\in \textbf{b}$, the mechanism charges the following payment to each agent $i$:
\begin{align*}
    P(\textbf{b})_i &= (\max_{j\neq i,~j\in V} (n-j+1)\cdot b_j - v_i(b_j)) - ((n-i^*)\cdot b_{i^*} - v_i(b_{i^*}))
\end{align*}
\end{theorem}
\begin{proof}
The VCG mechanism charges as payment the externality that individual players impose on the social welfare of the entire group and is known to incentivize truthfulness. Starting from the VCG objective, we have that $\pi(\textbf{b}) = \argmax_{b\in \textbf{b}}\sum_{i\in V} v_i(b)$, which by definition maximizes the social welfare.

Using similar notation as above, we let $\pi_{-i}(\textbf{b}_{-i})$ be the social-welfare maximizing alternative for the mechanism without agent $i\in V$. We let $j^*$ be the index of $\pi_{-i}(\textbf{b}_{-i})\in \textbf{b}$. The VCG payment is computed as:
\begin{align*}
    P(\textbf{b})_i &= \sum_{j\neq i}v_j(\pi_{-i}(\textbf{b}_{-i})) - \sum_{j\neq i}v_j(\pi(\textbf{b})) \\
    &= \sum_{j\neq i} v_j(\pi_{-i}(\textbf{b}_{-i})) - (n-i^* + 1)\cdot b_{i^*} - v_i(b_{i^*}) \\
    &= (n-j^* + 1)\cdot b_{j^*} - v_i(b_{j^*}) - (n-i^* + 1)\cdot b_{i^*} - v_i(b_{i^*}) \\
    &= (\max_{j\neq i,~j\in V} (n-j+1)\cdot b_j - v_i(b_{j})) - ((n-i^*+1)\cdot b_{i^*} - v_i(b_{i^*}))
\end{align*}
For each agent $i$ with $b_i < \pi(\textbf{b})$, it follows that when $i$ has insufficient capacity to participate, $i$ has no affect whether he/she participates or not.. The welfare from $\pi_{-i}(\textbf{b}_{-i}) = \pi(\textbf{b})$ and consequently generate the same social welfare. Furthermore, agent $i$ is charged zero payment.
\end{proof}
% \subsection{Byzantine Agreement}
% Consider a Byzantine Agreement protocol $\mathcal{P}$ that builds a blockchain and manages a governance mechanism over its blocksize as described above. Further, let each miner also be identical in terms of computational power. As is common with these protocols, $|V|=3f+1$ where $f$ denotes the number of byzantine nodes.

% \begin{definition}
% A miner $i$ is identical to a miner $j$ if $\hat{b}_i=\hat{b}_j$; that is, miners are identical if their true private states are equal.
% \end{definition}
% \begin{definition}
% $\forall i\in V$ the private state $b_i\in \mathbb{R}$ is the true processing capacity of miner $i$'s hardware.
% \end{definition}
% We now describe the protocol in detail. We assume the existence of digital signatures for each agent $i\in V$, denoted by $\sigma_i$. The protocol works by aggregating at least $n-f$ private state reports $\textbf{b}_{n-f}$ and then computing the new alternative $a$. We inherit the logic of the Practical Byzantine Fault Tolerant protocol introduced by Castro and Liskov for building the underlying state machine. To that end, we modify the protocol to include new messages that enable the aggregation of state reports \textbf{b} and to build a blockchain.
% \begin{protocol}{Byzantine Fault Tolerant Governance}
% \textit{Inputs.} At round $t$, $\forall i\in V$ with $b^t_i$, $H=\{(a^0,\emptyset),\dots,(a^{t-2},\textbf{b}^{t-1})\}$.
% \sbline
% \textit{Goal.} $\mathcal{O}(V)$ uses block size $a^t_{\mathcal{O}(V)}=\pi(\textbf{b}, H\cup\{(a^{t-1}, \textbf{b}^t\})$ to build block $t$.
% \sbline
% \textit{The protocol:}
% \begin{enumerate}
%   \item \textbf{Setup.}
%   \begin{enumerate}
%     \item $\forall i\in V,~i$ computes $b^t_i$ from $a^{t-1}$ and broadcasts $\langle (a^{t-1}, b^t_i)\rangle_{\sigma_i}$
%     \item $\forall i\in V,~i$ collects $n-f$ reports $\{\langle(a^{t-1}, b^t_j)\rangle_{\sigma_j}\}_{j\in I\subseteq V}$
%     \item $\forall i\in V,~i$ builds $\langle (a^{t-1}, \textbf{b}^t)_{\sigma}\rangle$ with at most $f$ empty slots in $\textbf{b}^t$.
%     \item $\forall i\in V,~i$ computes $a^t_i = \pi(\textbf{b}^t, H\cup\{(a^{t-1}, \textbf{b}^t)\}$.
%     \item $\forall i\in V,~i$ broadcasts $\langle a^t_i, H\cup\{(a^{t-1}, \textbf{b}^t)_{\sigma}\}\rangle$
%   \end{enumerate}
% \end{enumerate}

% \end{protocol}
\end{document}


\section{\uppercase{Best views estimation}}\label{sec:best-views-estimation}

\noindent Estimation of the best views for a constellation of sensors requires both the ability to generate accurate sensor data for each type of sensor and also an efficient approach to compute the surface coverage that we are trying to maximize. The next sections explain how the sensor data is analyzed and also present the approaches used to estimate the best constellation of sensors for a given simulation world.

\subsection{Reference surface point cloud}

The first step in the processing pipeline includes the generation of the multi-object reference point cloud (described in \cref{alg:reference-pointcloud}) that is built by transforming the point cloud associated with the target \gls{cad} model into each target object within the simulation world (example shown in \cref{fig:reference-cloud}). Later on, the reference point cloud in the world coordinate frame is filtered with a voxel grid algorithm in order to perform a regular space partition and extract the surface voxels centroids that contain points. This approach allows to generate a reference point cloud with a constant surface point density, which will be critical later on when computing the surface coverage percentage achieved with a given sensor constellation.

\begin{algorithm}
	\caption{Generation of the reference point cloud}
	\label{alg:reference-pointcloud}
	\begin{algorithmic}[1]
		\State \textbf{Input:}
		\State $P \gets$ CAD model point cloud
		\State $N \gets$ target objects simulation names
		\State $F \gets$ cell size for the voxel grid filter
		\Procedure{ReferencePointCloud}{$P,N,F$}
			\State $r \gets Empty$\Comment{merged point cloud}
			\ForAll{n names in N}
				\State $w \gets GetModelWorldPose(n)$
				\State $p \gets TransformPointCloud(P,w)$
				\State $r \gets Merge(r,p)$
			\EndFor
			\State $r \gets FilterWithVoxelGrid(r,F)$
			\State \textbf{return} $(r)$
		\EndProcedure
	\end{algorithmic}
\end{algorithm}

\begin{figure}
	\centering
	\includegraphics[height=.14\textheight]{sensor-data-processing/multimodel-environment}\hspace{2em}
	\includegraphics[height=.1\textheight]{sensor-data-processing/cad-model-pointcloud}
	\includegraphics[height=.14\textheight]{sensor-data-processing/multimodel-pointclouds-with-cad}\hspace{2em}
	\includegraphics[height=.14\textheight]{sensor-data-processing/multimodel-pointclouds}
	\caption{The first image illustrates the color scene rendering in Gazebo with the target objects in green while the third and fourth images display the reference point cloud that was generated using the CAD point cloud shown on the second image.}
	\label{fig:reference-cloud}
\end{figure}

\subsection{Sensors data analysis}

After loading the simulation world 3D models, deploying the sensors populations on the environment and building the filtered reference point cloud, the proposed system generates a color and depth image for every sensor. Then, for each pixel in the color images that have the target objects unique color, the corresponding pixel in the depth image is retrieved and using the pinhole model equations shown in \cref{eq:pointcloud}, the 3D point is computed from the 2D pixel coordinates and depth value. Later on, the 3D point is transformed from the sensor coordinate system into the world coordinate frame (having all sensor data in the same coordinate system allows fast merging of point clouds from several sensors). After processing all pixels of a given color image, the associated point cloud in the world coordinate frame is filtered with a voxel grid algorithm with a cell size tuned for the objects geometry we are trying to observe (given that too many points on a small area for a large object do not provide a significant advantage for 3D perception and require more processing time). This allows to perform a regular space partition for extracting the centroid of each voxel containing sensor points. This step is critical for allowing consistent evaluation of the object(s) observed surface area percentage, given that sensors with different resolution or at varying distances may generate point clouds with different point density even when observing the same surface area. Moreover, given that both the reference point cloud and the sensor data point cloud were filtered in the same coordinate frame and with the same voxel grid cell size, the surface coverage percentage can be computed very efficiently by simply dividing the number of surface points in the filtered sensor data point cloud by the number of surface points in the filtered reference point cloud.

In the end of the sensor analysis stage (presented in \cref{alg:sensor-analysis}), each sensor is associated with a filtered point cloud in the world coordinate frame containing only points belonging to the target objects surface.

\begin{algorithm}
	\caption{Sensor analysis}
	\label{alg:sensor-analysis}
	\begin{algorithmic}[1]
		\State \textbf{Input:}
		\State $S \gets$ deployed sensors
		\State $C \gets$ target objects unique color
		\State $F \gets$ cell size for the voxel grid filter
		\Procedure{SensorAnalysis}{$S,C,F$}
			\State $v \gets Empty$\Comment{sensors filtered point clouds}
			\ForAll{s sensors in S}
				\State $c \gets RenderColorImage(s)$
				\State $d \gets RenderDepthImage(s)$
				\State $w \gets GetSensorWorldPose(s)$
				\State $u \gets Empty$\Comment{target objects points}
				\ForAll{y image rows in c}
					\ForAll{x image columns in c}
						\State $p \gets GetPixel(c,x,y)$
						\If{$p = C$}
							\State $k \gets GetDepth(d,x,y)$
							\If{$InValidRange(k,s)$}
								\State $j \gets 3DPoint(k,x,y,s)$
								\State $m \gets TransformPt(j,w)$
								\State $AppendPoint(u,m)$
							\EndIf
						\EndIf
					\EndFor
				\EndFor
				\State $z \gets FilterPointCloud(u,F)$
				\State $AppendPointCloud(v,z)$
			\EndFor
			\State \textbf{return} $(v)$
		\EndProcedure
	\end{algorithmic}
\end{algorithm}

\footnotesize
\begin{equation}\label{eq:pointcloud}
	\begin{split}
		X = \frac{(PixelCol - XPrincipalPoint) \times PixelDepth}{XFocalLenght}\\
		Y = \frac{(PixelRow - YPrincipalPoint) \times PixelDepth}{YFocalLenght}\\
		Z = PixelDepth
	\end{split}
\end{equation}
\normalsize

\begin{figure}
	\centering
	\includegraphics[height=.132\textheight]{sensor-data-processing/sensors-best-view}\\
	\vspace{0.5em}
	\includegraphics[height=.132\textheight]{sensor-data-processing/rviz-sensor-view}\hspace{2em}
	\includegraphics[height=.132\textheight]{sensor-data-processing/rviz-sensor-view-without-cad-with-model}
	\caption{Color image rendered with the Gazebo simulator (top image containing the scene and sensor) along with the generated point cloud for the target object taking into consideration the environment occlusions (bottom images, in which the green spheres are the observed points and the blue spheres are from the filtered point cloud of the associated CAD model).}
	\label{fig:sensor-data-processing}
\end{figure}

\subsection{Estimation of the best sensors views}

When only one sensor is needed for the task at hand (for example when we are trying to perform 3D perception of the environment in which the sensor is attached to a robotic arm), the estimation of the best sensor can be performed by simply selecting the one that achieved the best surface coverage percentage. On the other hand, if several sensors are available or we want a single sensor to observe the target objects from a set of N best views, then it is used a \gls{ransac} approach to estimate the constellation of sensors that can achieve the best surface coverage. This approach allows to mitigate and bound the combinatorial explosion that happens when we need to estimate a high number of best views from a large population of sensors. As can be seen in \cref{alg:best-n-views}, this approach runs at most a fixed number of iterations. In each iteration, a set of N sensors are chosen randomly, their sensor data is merged and filtered, and if the surface coverage percentage achieved by this set of views is higher than a given threshold, then the search is stopped. In the end, it is returned the best sensor constellation found along with its associated point could (with the merged sensor data) and the best surface coverage percentage that was achieved.

\begin{algorithm}
	\caption{Estimation of the best N sensors views}
	\label{alg:best-n-views}
	\begin{algorithmic}[1]
		\State \textbf{Input:}
		\State $N \gets$ number of desired sensors
		\State $P \gets$ point clouds from each deployed sensor
		\State $F \gets$ cell size for the voxel grid filter
		\State $C \gets$ minimum surface coverage percentage
		\State $I \gets$ maximum number of iterations
		\Procedure{BestSensorsViews}{$N,P,F,C,I$}
			\State $s \gets Empty$\Comment{best coverage sensors}
			\State $p \gets Empty$\Comment{best merged point cloud}
			\State $c \gets 0$\Comment{best coverage percentage}
			\State $i \gets 0$\Comment{current iteration}
			\While{$i < I$ and $c < C$}
				\State $x \gets SelectSensorsRandomly(P,N)$
				\State $m \gets MergePointClouds(P,x)$
				\State $f \gets FilterPointCloud(m,F)$
				\State $k \gets ComputeSurfaceCoverage(f)$
				\State $i \gets i + 1$
				\If{$k > c$}
					\State $s \gets x$
					\State $p \gets f$
					\State $c \gets k$
				\EndIf
			\EndWhile
			\State \textbf{return} $(s,p,c)$
		\EndProcedure
	\end{algorithmic}
\end{algorithm}


\chapter{Results}
\label{ch:results}

\section{Neutron Stars}
Using \chromos, power colours can be obtained for a large number of observations. Applying \chromos to the population of neutron stars given in Tab.~\ref{tab:objects} reveals that most objects follow similar tracks in the \ac{PCC}~diagram. This can be seen in Fig.~\ref{fig:pc_all_ns}, where a distinct elliptical shape emerges after overplotting these tracks. In this plot $PC1$ is defined as the variance in (0.25-2.0~Hz)/(0.0039-0.031~Hz) and $PC2$ as the variance in (0.031-0.25~Hz)/(2.0-16.0~Hz). Only objects with more than three data points were included, and two objects were excluded due to their peculiar behaviours. A full discussion on the nature of those objects can be found in section~\ref{sec:dis_ns}. For sake of clarity, only points where the variance is $>\!3 \sigma$ in all four frequency bands are plotted, and error bars are omitted. Typical error bars are on the order of 17\% of the power colour values. Individual tracks can be seen more clearly in appendix~\ref{ch:pccds}, where each object has been plotted with the neutron star \ac{PCC} tracks in reference. \\

While using power colours is useful in comparing evolutionary tracks of systems, power colours require two dimensions ($PC1$ and $PC2$) to classify the state of a system. Reducing this down to a single parameter can be helpful in comparing the state of a system against other parameters. To this end, the `hue'~parameter \marginpar{In colour theory, hue is defined as the attribute by virtue of which a colour is red, green, etc \citep{oed}, and is often determined using a colour wheel. This makes hue an excellent analogy to the angle within a \ac{PCC}~diagram.} can be introduced \citep{heil2015power}. Defined as the angle of a point in the \ac{PCC}~diagram with respect to a central point, hue runs in a clockwise direction from a line in the Northwest direction from $0^\circ$ to $360^\circ$. Following the original classification of the hue centre given in \citet{heil2015power}, a central point with the coordinates (4.51920, 0.453724) is chosen as reference point. An example of this classification can be seen in Fig.~\ref{fig:pc_hue_bins}, where the neutron star \ac{PCC}~diagram has been divided into hue bins of $20^\circ$. \\

These hue bins allow an overview of power spectra to be created per hue bin, showing typical power spectra for a variety of objects. Comprising of various types of neutron star systems, these power spectra can be found in appendix~\ref{ch:psds}. Intrinsic scatter in the power, especially at the high frequencies, is reduced by binning data points linear in log-frequency and errors reduced accordingly. \\

\begin{landscape}
\begin{figure}
	\myfloatalign
	{\vspace*{-3cm}\includegraphics[width=0.8\linewidth]{pc/all_ns}}
	\caption[Neutron stars in a \acs{PCC}~diagram]{A \ac{PCC}~diagram showing tracks for neutron stars. While providing an excellent overview of the general trend, tracks of individual objects can perhaps be best pursued in appendix~\ref{ch:pccds}, where \ac{PCC}~diagrams can be found for each system.}\label{fig:pc_all_ns}
\end{figure}
\end{landscape}

\begin{figure}[p]
	\myfloatalign
	{\includegraphics[width=0.8\linewidth]{pc/hue_bins}}
	\caption[Defining the hue in a \acs{PCC}~diagram]{A \ac{PCC}~diagram showing the division of $20^\circ$ hue bins for neutron star \acp{PCC}. The starting angle is defined as the angle at $45^\circ$ in a counter-clockwise direction from the axes. Based on Fig.~2b from \citet{heil2015power}.}\label{fig:pc_hue_bins}
\end{figure}

An interesting parameter to compare hue with, is the energy spectral hardness. Linking timing information back to spectral information can provide valuable insights into properties of systems such as the relative spectral evolution and therefore changes in inner region structure. A \ac{HH}~diagram for neutron stars can be seen in Fig.~\ref{fig:hh_all_ns}, with the hardness defined as the ratio of the total count rate in 9.7-16.0~keV over the total count rate in 6.4-9.7~keV. Next to the selection methods for \ac{PCC}~points described in the first paragraph of this section, only points with a hue-error $<\!30^\circ$ are included in this graph, where errors are propagated through from the \ac{PCC}-errors. In a similar fashion to the \ac{PCC}~diagrams in appendix~\ref{ch:pccds}, individual \ac{HH}~diagrams can be found in appendix~\ref{ch:hhds}, allowing the evolution of an object within a \ac{HH}~diagram to be traced against other neutron stars.\\

\begin{figure}[p]
	\myfloatalign
	{\includegraphics[width=\linewidth]{hh/all_ns}}
	\caption[Neutron stars in a \acs{HH}~diagram]{A \ac{HH}~diagram showing the evolution of \ac{PCC}-tracks for neutron stars through use of the hue. Supplementary \ac{HH}~diagrams can be found in appendix~\ref{ch:hhds}, showing the tracks of individual objects with neutron star \ac{HH} tracks as reference.}\label{fig:hh_all_ns}
\end{figure}

An often-used method in X-ray astronomy to trace the overall secular evolution of \acp{LMXB} is the \ac{HI}~diagram (e.g. \citet{done2003observing}, \citet{klein2008identification} or \citet{fridriksson2015common}). In a \ac{HI}~diagram the energy spectral hardness is plotted against the relative intensity of an object. Appendix~\ref{ch:hids} shows \ac{HI}~diagrams for the full population of systems given in Tab.~\ref{tab:objects}, allowing a broad range of tracks to be compared. A number of objects show unusual tracks, a point discussed in more detail in section~\ref{sec:dis_ns}.\\

In order to fully encapsulate the evolution of neutron star \acp{LMXB}, \ac{ECC}~diagrams are preferred over \ac{HI}~diagrams. The tracks in the \ac{ECC}~diagram are commonly divided into various states --- the \ac{EIS}, the \ac{IS} and the banana branch, with the \ac{LLB}, the \ac{LB} and the \ac{UB}. Linking the states of atoll source Aql~X-1 with the \ac{PCC}~diagram reveals an interesting link as shown in Fig.~\ref{fig:cc}. Here the softness is defined as the ratio of the total count rate in 3.5-6.0~keV over the total count rate in 2.0-3.5~keV, with the hardness retaining the same definition as earlier. For clarity, error bars on the energy colours have been left off the \ac{ECC}~diagram, typically being small in comparison to the \ac{ECC}~values. A clear distinction can be made in the \ac{PCC}~diagram between sources in the \ac{EIS}, and sources in the banana states. While the \ac{IS} does not show up in the \ac{PCC}~diagram, other objects showed this state to be located left of the lower apex, a region with relatively few \ac{PCC} due to the rapid state transitions.

\begin{figure}[p]
\myfloatalign%
\makebox[\textwidth][r]{%
\subfloat{\includegraphics[width=.5\largefigure,valign=t]{cc/ns_states_aquila_X1}}%
\quad%
\subfloat{\includegraphics[width=.51\largefigure,valign=t]{pc/ns_states_aquila_X1}}%
}%
\caption[Comparing energy and power colours]{\spacedlowsmallcaps{left} A \ac{ECC}~diagram for Aql~X-1, showing the commonly-adopted division into various states: the \acf{EIS}, the \acf{IS} and the banana branch, with the \acf{LLB}, the \acf{LB} and the \acf{UB} \spacedlowsmallcaps{right} A \ac{PCC}~diagram showing observations linked to the various states defined using the \ac{ECC}~diagram.}\label{fig:cc}
\end{figure}

\section{Neutron Stars \& Black Holes}
In order to conduct a model-independent comparison of accreting black hole and neutron star variability, \ac{PCC}~diagrams can be constructed to show the evolution of both black hole and neutron star systems. In the left panel of Fig.~\ref{fig:ns_bh}, three representative transient black holes have been plotted for comparison with neutron stars, with additional information on these systems in Tab.~\ref{tab:objects}. Both types of system show similar paths, yet a clear distinction is found on the right-hand side of the diagram where the black holes systematically follow a higher path with respect to the neutron stars. In the right panel of Fig.~\ref{fig:ns_bh}, a \ac{HH}~diagram is shown for the same systems, where the hardness is classified as the same ratio of (9.7-16.0~keV)/(6.4-9.7~keV). With the hue washing out any radial differences in \acp{PCC}, in the \ac{HH}~diagram the black holes closely follow the neutron stars albeit with a broader coverage of angles. \\

\begin{figure}[p]
\myfloatalign%
\makebox[\textwidth][r]{%
\subfloat{\includegraphics[width=.51\largefigure,valign=t]{pc/ns_bh}}%
\quad%
\subfloat{\includegraphics[width=.49\largefigure,valign=t]{hh/ns_bh}}%
}%
\caption[Comparison of neutron stars and black holes]{\spacedlowsmallcaps{left} A \ac{PCC}~diagram showing black hole systems, with in grey the neutron star \acp{PCC} in Fig.~\ref{fig:pc_all_ns} as reference. \spacedlowsmallcaps{right} The same systems plotted in a \ac{HH}~diagram.}\label{fig:ns_bh}
\end{figure}

While black hole and neutron star \acp{LMXB} have similar broad-band spectral shapes, the frequency at which power spectral features occur can be a factor five higher for black holes than for neutron stars \citep{kleinwolt}. Thus comparing \ac{PCC}~values of black holes with neutron stars could perhaps best be done by shifting the frequency ranges for neutron star power colours up by a factor of five in the power spectrum. This causes the original frequency band boundaries to shift up to 0.0195, 0.155, 1.25, 10 and 64~Hz, where the final frequency is limited by the resolution in which light curves were extracted. Fig.~\ref{fig:shiftedpc} shows the result of shifting the frequency bands for neutron stars. In the left panel, the original \ac{PCC}~values for neutron stars can be seen in red against the black hole \ac{PCC}~values in grey. In the right panel are respectively the shifted \ac{PCC}~values, shown against the unaffected black hole \ac{PCC}~values. While the shifted \ac{PCC}~values show a greater overlap between neutron star and black hole \acp{PCC}, both tracks can still be distinguished.\\

\begin{figure}[p]
	\myfloatalign
	{\includegraphics[width=\largefigure]{pc/shiftedpc}}
	\caption[Effect of shifting power colour frequency bands]{\spacedlowsmallcaps{left} A \ac{PCC}~diagram showing neutron stars in red against black hole systems in grey. \spacedlowsmallcaps{right} \acp{PCC} for neutron stars where the frequency bands have been shifted up by a factor of five. The black hole systems in grey have retained the original frequency bands for their \ac{PCC} values.}\label{fig:shiftedpc}
\end{figure}

\section{Effects of Neutron Star Properties}

\enlargethispage{2\baselineskip}
\subsection{Inclination}
Power colours present the unique ability to compare various parameters of multiple systems throughout different energy spectral states. Using power colours in such a fashion, \citet{heil2015inclination} found black holes systems followed an inclination-dependent track in the \ac{HH}~diagram. Applying the same technique to neutron stars results in Fig.~\ref{fig:inclination}, where sources have been split into either a low ($i\!\leq\!60^\circ$)  or high ($i\!>\!60^\circ$) binary orbit inclination group using Tab.~\ref{tab:objects}, following the division adopted in \citet{heil2015inclination}. Errorbars have been omitted for clarity and sources with an undefined inclination plotted in grey. While no particular trend can be discerned from the \ac{PCC}~diagram, the resulting \ac{HH}~diagram shows signs of an offset dependent on inclination, with low inclination sources showing higher hardness per hue than high inclination sources. Implications of this trend are discussed in section~\ref{sec:dis_incl}, including suggestions on possible origins.\\

\begin{figure}[p]
\myfloatalign%
\makebox[\textwidth][l]{%
\subfloat{\includegraphics[width=.51\largefigure,valign=t]{pc/inclination}}%
\quad%
\subfloat{\includegraphics[width=.49\largefigure,valign=t]{hh/inclination}}%
}%
\caption[Inclination effects]{\spacedlowsmallcaps{left} A \ac{PCC}~diagram with neutron star systems split into high and low inclination groups. Sources with an undefined inclination have been plotted in grey. Details on the inclination of individual sources can be found in Tab.~\ref{tab:objects}. \spacedlowsmallcaps{right} A \ac{HH}~diagram with neutron star systems divided in the same manner.}\label{fig:inclination}
\end{figure}

\subsection{Atoll \& Z Sources}
Based on the path \acp{LMXB} trace out in a \ac{ECC}~diagram, neutron star systems can be classified into two subclasses: atoll and Z sources \citep{hasinger1989two}. In Tab.~\ref{tab:objects}, systems classified as either an atoll or a Z source have been denoted with respectively an $A$ or a $Z$. A comparison of these systems in the \ac{PCC} and \ac{HH}~space can be seen in Fig.~\ref{fig:atoll_z}. Plotted together with unclassified neutron stars, the Z sources in the \ac{PCC}~diagram rarely cross into the upper-right half of diagram. This is reflected in the \ac{HH}~diagram, where almost all Z source values remain above a hue of $180^\circ$. No particular difference can be discerned between atoll and as of yet unclassified sources, with atoll sources tracing a similar path to the latter.\\

\begin{figure}[p]
\myfloatalign%
\makebox[\textwidth][l]{%
\subfloat{\includegraphics[width=0.51\largefigure,valign=t]{pc/atoll_z}}%
\quad%
\subfloat{\includegraphics[width=0.49\largefigure,valign=t]{hh/atoll_z}}%
}%
\caption[Comparing Atoll and Z sources]{\spacedlowsmallcaps{left} \ac{PCC}~diagram with systems classified as atoll sources in red, Z sources in blue and unclassified sources in grey. An overview showing the type of each individual system can be found in Tab.~\ref{tab:objects}. \spacedlowsmallcaps{right} Replotting the same systems in a \ac{HH}~diagram.}\label{fig:atoll_z} % \TODO Check Z-source outlier
\end{figure}

A further division can be made in Z sources on the basis of spectral and timing behaviour, allowing sources to be split into Cyg- and Sco-like sources \citep{kuulkers1997gx}. Nonetheless, there are only a few sources which have been classified as Cyg- or Sco-like sources, as seen in Tab.~\ref{tab:objects}. These can be plotted in a \ac{PCC}~diagram, as seen in Fig.~\ref{fig:pc_sco_cyg}. With Z sources rarely straying beyond the lower left corner, a large degree of scatter is expected, and is found in the \ac{PCC}~values. It is interesting that there is seemingly a lower degree of scatter for the Cyg-like sources in comparison to the Sco-like sources.\\

\begin{figure}[p]
	\myfloatalign
	{\includegraphics[width=0.8\linewidth]{pc/sco_cyg}}
	\caption[\acs{PCC}~diagram with Sco- and Cyg-like sources]{\ac{PCC}~diagram with Z sources. Sco-like sources are plotted in red and with Cyg-like sources in blue. Note the large contributions of Sco X-1 and Cyg X-2 to each respective group.}\label{fig:pc_sco_cyg}
\end{figure}

\subsection{Pulsations \& Spin}
Since the discovery of the first \ac{AMSP} by \citet{wijnands1998millisecond}, systematic searches have revealed fourteen other systems of this nature \citep[see][for a review]{patruno2012accreting}. One such source, HETE J1900.1-2455, was the first to show `quasi-persistent' activity \citep{galloway2006intermittent}. Believed to be due to a build-up and subsequent suppression of magnetic field through channeling of the accreting flow \citep[e.g.][]{cumming2001magnetic}, it provides an effective test for a possible correlation between magnetic fields and \acp{PCC}. Fig.~\ref{fig:hete_pulsations} shows a selection of observations split into time intervals of standard luminosity levels and time intervals with pulsations. Selections were made on basis of the fractional pulse amplitudes between MJD~53520--53690 as given in \citet{galloway2006intermittent}. Though the \ac{PCC}~diagram hints at higher PC2 values during pulsation periods, no conclusive correlation can be determined, with relatively few observations available to distinguish any trends.\\

\begin{figure}[p]
	\myfloatalign
	{\includegraphics[width=0.8\linewidth]{pc/hete_pulsations}}
	\caption[\acsp{PCC} during intermittent pulsations]{A \ac{PCC}~diagram with observations of HETE~J1900.1-2455 split into pulsation time intervals and periods of standard flux-levels. The division between these time intervals was made on basis of information in \citet{galloway2006intermittent}.}\label{fig:hete_pulsations}
\end{figure}
\newpage
Pulsations in neutron star \acp{LMXB} are commonly used in an attempt to establish the spin frequency of the neutron star, resulting in groups of objects referred to as respectively bursters and pulsars. It is generally accepted that the rapid increase in flux observed in bursters is linked to unstable nuclear burning on the neutron-star surface \citep[e.g.][]{klis2000millisecond}, providing a means to determine the spin frequency of the neutron star through hotspots on the neutron star surface \citep[see][]{chakrabarty2003nuclear,strohmayer2003new}. In contrast, pulsars show regular pulsations, resulting in extremely precise spin frequencies. While spin frequencies typically fall far from the frequency bands that power colours probe, some effect of the higher spin frequencies could perhaps be expected to be seen in the lower power colour frequencies. The spin could for instance affect the accretion flow via the pulsar magnetosphere. Coupling spin frequencies with \acp{PCC} results in Fig.~\ref{fig:spin}, with objects colour-coded according to their spin frequency. Objects with less than four points have been removed from the sample, to ensure clarity. Bursters show no distinct effects on \acp{PCC}, however a tentative link could perhaps be found in pulsar systems where a shift in \acp{PCC} according to spin frequency can be observed. An in-depth discussion on this relation can be found in section~\ref{sec:dis_ps}, as well as an hypothesis on the origin of this tentative relationship.\\

\begin{figure}[p]
	\myfloatalign
	\makebox[\textwidth][l]{%
	{\includegraphics[width=\largefigure]{pc/bursters_pulsars}}
	}%
	\caption[The effect of spin frequency on \acsp{PCC}]{\spacedlowsmallcaps{left} Bursters plotted in order of frequency, with neutron star \acp{LMXB} without a defined spin frequency in comparison. \spacedlowsmallcaps{right} \ac{PCC}~diagram showing pulsars per spin frequency together with the \ac{PCC} values of the other neutron stars.}\label{fig:spin}
\end{figure}


\documentclass[10pt, handout]{beamer}
\setbeamertemplate{navigation symbols}{}
\usefonttheme{serif}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{cite}
\usepackage{color}
\usepackage{setspace}
\usepackage{hyperref}

\newcommand{\xx}{{\bf{x}}}

\begin{document}
\title{Machine Learning I Lecture III:\\ Gaussian models}
\author{Jakob H Macke\\ Max Planck Institute for Biological Cybernetics\\ Bernstein Center for Computational Neuroscience}
\date{XY.XY.2012}

\frame{\titlepage}

%\frame{\frametitle{Today: Back to basics of probability theory}}

\frame{\frametitle{Plan for today}\tableofcontents}

\section{Wrap up: Continuous random variables}

\frame{\frametitle{Mean, variance, and conditioning on events are the same as the discrete case, just with sums replaced by integrals.}
\begin{itemize}
\item Mean: $E(X)= \int_x x  \cdot p(x) dx$\\
\item Variance: $\mbox{Var}(X)= E(X^2)- E(X)^2$
\item Example: Uniform, Exponential [on board]
\item \pause If $X$ has pdf $p(x)$, then $X | (X \in A)$ has pdf
\begin{align}
p_{X|A}(x)=\frac{p(x)}{P(A)}=\frac{p(x)}{\int_{x \in A} p(x) dx}
\end{align}
\item \pause Only makes sense if $P(A)>0$~!
\item Examples: Uniform, Exponential [on board]
\end{itemize}
}

\frame{\frametitle{Bivariate continuous distributions: Marginalization, Conditioning and Independence}
\begin{itemize}
\item $p_{X,Y}(x,y)$, joint probablity density function of $X$ and $Y$
\item $\int_x \int_y p(x,y)dx dy=1$
\item \pause \alert{Marginal distribution:} $p(x)= \int_{-\infty}^\infty p(x,y) dy$
\item \pause \alert{Conditional distribution:} $p(x|y)= \frac{p(x,y)}{p(y)}$
\item Note: $P(Y=y)=0$! Formally, conditional probability in the continuous case can be derived using infinitesimal events.
\item \pause \alert{Independence:} $X$ and $Y$ are independent if $p_{X,Y}(x,y)=p_X(x)p_Y(y)$
\end{itemize}
}

\section{Gaussians}
\frame{\frametitle{The univariate Gaussian}
\begin{align}
t &\sim \mathcal{N}(\mu,\sigma^2)\\
p(t|\mu, \sigma^2)&=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left( -\frac{1}{2}\left(\frac{t-\mu}{\sigma} \right)^2\right)
\end{align}
\begin{itemize}
\item
The Gaussian has \alert{mean} $\mu$ and \alert{variance} $\sigma^2$ and \alert{precision} $\beta=1/\sigma^2$
\item \pause Q: What are the \alert{mode} and the \alert{median} of the Gaussian?
\item \pause Maximum Likelihood estimation of $\mu$ and $\beta$: [on board]
\item \pause Q: How would you find the conjugate prior for the Gaussian?

\end{itemize}
}

\frame{\frametitle{A (very important) aside: Products of Gaussian pdfs are (unnormalized) Gaussians pdfs}
\begin{itemize}
\item Suppose $p_1(x)=\mathcal{N}(x,\mu_1, 1/\beta_1)$ and $ p_2(x)=\mathcal{N}(x,\mu_2, 1/1\beta_2)$, then
\pause \begin{align}
p_1(x) p_2(x) &\propto \mathcal{N}(x, \mu, 1/\beta)\\
\beta&=\beta_1+\beta_2\\
\mu&=\frac{1}{\beta}(\beta_1 \mu_1 +  \beta_2 \mu_2)
\end{align}

\pause
In general:
\begin{align}
p_1(x) p_2(x) ... p_n(x) &\propto \mathcal{N}(x, \mu, 1/\beta)\\
\beta&=\sum_n \beta_n\\
\mu&=\frac{1}{\beta} \sum_n \mu_n \beta_n
\end{align}

\pause
This is also true for multivariate Gaussians!

\end{itemize}
}

\section{Bayesian inference for Gaussians}

\frame{\frametitle{Bayesian Inference for the Gaussian}
\begin{itemize}
\item Suppose we are given data $D=\{x_1, \ldots, x_N\}$.
\item We assume that the data is Gaussian-distribution with known variance $\sigma^2$ and unknown mean $\mu$.
\item Our prior for $\mu$ is Gaussian: $\mu \sim \mathcal{N}(\mu_o, \sigma^2_o)$
\item Posterior distribution over $\mu$ given the data: [on board]
\end{itemize}
~
\pause
{\centering
\includegraphics[width=.58\textwidth]{Figure212.pdf}
\\\vspace{-.5cm}}
{\tiny[Bishop PRML Figure 2.12]}

\pause Behaviour for large $N$: [on board]
}

\frame[shrink=0]{\frametitle{What if the variance is not given?}
\begin{itemize}
\item For simplicity, assume mean to be known.
\item More convenient to work with precision $\lambda=1/\sigma^2$.
\item Conjugate prior: Gamma distribution $\mbox{Gam}(\lambda|a,b)$
\begin{align}
p(\lambda|a, b)=\frac{1}{\Gamma(a)}b^a \lambda^{a-1} exp(-b\lambda)
\end{align}
\item Posterior is $\mbox{Gam}(\lambda|a_N,b_B)$
\begin{align}
a_N &= a + \frac{N}{2}\\
b_N &= b + \frac{1}{2} \sum_{n=1}^N (x_n -\mu)^2
\end{align}
\pause
\includegraphics[width=.38\textwidth]{Figure213b.pdf}
\includegraphics[width=.38\textwidth]{Figure213c.pdf} \\\tiny[Bishop PRML Page 100]

\end{itemize}
}

\frame[shrink=0]{\frametitle{What if both the mean and the variance are unknown?}
\begin{itemize}
\item Conjugate prior: Gaussian-Gamma distribution
\begin{align}
p(\mu,\lambda) & = \mathcal{N}\left(\mu |\mu_o (\beta\lambda)^{-1}\right) \mbox{Gam}(\lambda|a,b)
\end{align}s
\includegraphics[width=.68\textwidth]{Figure214.pdf} \\\tiny[Bishop PRML Page 102]
\end{itemize}
}

\end{document}