\documentclass{article}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{hyperref}
\usepackage{float}

\title{Assignment for CS 696-04 \\ (Deep Learning)}
\author{Diwas Sharma}
\date{\today}
\begin{document}

\maketitle
\newpage

\section{Design}
The code that is used to construct and train the Neural network is highly modular and
can be used as a library for other programs. The interface is quite similar to the one
provided by keras \footnote{\url{https://keras.io/}}.

The code enable the user to construct a neural network with any number of dense
layers. And, supports activation functions such as sigmoid, hyperbolic tangent,
ReLu, Leaky ReLu, and linear. An example of a neural network that can be construct
using the code is given below.

\begin{lstlisting}[language=python]
from depth.sequential import NeuralNet

nn_object = NeuralNet()
nn_object.add_layer(units=32, activation_function="relu",
                    input_dimension=10)
nn_object.add_layer(units=32, activation_function="tanh")
nn_object.add_layer(units=32, activation_function="sigmoid")

\end{lstlisting}

It also provides a choice between cross entropy loss and mean squared
error to be used as a loss function for the output layer.

\section{MNIST dataset}
\subsection{Network}
The network consists of 3 hidden layers each with 32 neurons.  The first and the last
hidden layer uses hyperbolic tangent as the activation function. And the second
hidden layer uses the ReLu function as the activation function. The output layer
is a hyperbolic tangent layer with 10 neurons with softmax output.

As the MNIST dataset is a multiclass classification problem, the cross entropy function
is used as the loss function along with gradient descent optimization for adjusting
parameters.

\subsubsection{Source Code}
\begin{lstlisting}[language=python]
# First construct an optimizer to use
optimizer = SGD(lr=self.learning_rate, momentum=self.momentum)

# Create L2 regularizer
regularizer = L2Regularizer(self.regularization_coefficient)

self.nn = NeuralNet()
self.nn.add_layer(units=32, activation_function="tanh",
                    input_dimension=self.input_data_dimension,
                    regularizer=regularizer)
self.nn.add_layer(units=32, activation_function="relu",
                    regularizer=regularizer)
self.nn.add_layer(units=32, activation_function="tanh",
                    regularizer=regularizer)
self.nn.add_layer(units=self.output_data_dimension,
                    activation_function="tanh", regularizer=regularizer)
self.nn.compile(loss="cross_entropy",
                error_threshold=self.error_threshold,
                optimizer=optimizer)
\end{lstlisting}

\subsection{Training}
At first a learning rate of $\eta = 0.001$ was chosen as a starting point
to observe the performance of the network. The training was done using gradient
descent algorithm with momentum where the hyperparameters were selected as
momentum coefficient$(\gamma) = 0.9$ and regularization coefficient$(\lambda)=0.01$.  The plot in figure \ref{fig:mnist_three_loss} shows
the regularized loss function at the end of each epoch during the training. Similarly, figure \ref{fig:mnist_three_accuracy} shows the
training accuracy at the end of every epoch.

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{m1e-3_m_loss.png}
  \caption{Regularized training loss for $\eta = 0.001, \gamma = 0.9, \lambda=0.01$}
  \label{fig:mnist_three_loss}
\end{figure}

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{m1e-3_m_accuracy.png}
  \caption{Training accuracy for $\eta = 0.001$}
  \label{fig:mnist_three_accuracy}
\end{figure}

It is clear from the plots that the learning rate of $0.001$ is too small for training the network. A
significant improvement can be seen in the training performance if the learning rate is increased to $\eta = 0.5$.
The performance of network for the increased learning rate is show in figures \ref{fig:mnist_one_loss} and \ref{fig:mnist_one_accuracy}.

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{m5e-1_m_loss.png}
  \caption{Regularized training loss for $\eta = 0.5, \gamma = 0.9, \lambda=0.01$}
  \label{fig:mnist_one_loss}
\end{figure}

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{m5e-1_m_accuracy.png}
  \caption{Training accuracy for $\eta = 0.5$}
  \label{fig:mnist_one_accuracy}
\end{figure}

\subsection{Testing}
The accuracy obtained for the network on the test data is 0.89403.

\section {CIFAR-10 dataset}
\subsection{Network}
The network consists of 3 hidden layers each with 128 neurons.  The first and the last
hidden layer uses hyperbolic tangent as the activation function. And the second
hidden layer uses the ReLu function as the activation function. The output layer is
a hyperbolic tangent layer with 10 neurons with softmax output.

The loss function used is cross entropy loss and gradient descent is used for
optimizing the parameters.

\subsubsection{Source Code}
\begin{lstlisting}[language=python]
# First construct an optimizer to use
optimizer = SGD(lr=self.learning_rate, momentum=self.momentum)

# Create L2 regularizer
regularizer = L2Regularizer(self.regularization_coefficient)

self.nn = NeuralNet()
self.nn.add_layer(
    units=128, activation_function="tanh",
    input_dimension=self.input_data_dimension, regularizer=regularizer)
self.nn.add_layer(units=128, activation_function="relu",
                    regularizer=regularizer)
self.nn.add_layer(units=128, activation_function="tanh",
                    regularizer=regularizer)
self.nn.add_layer(units=self.output_data_dimension,
                    activation_function="tanh", regularizer=regularizer)
self.nn.compile(loss="cross_entropy",
                error_threshold=self.error_threshold,
                optimizer=optimizer)
\end{lstlisting}

\subsection{Training}
The performance of the network for a learning rate of $\eta = 0.001$ is shown in figures
\ref{fig:cifar_three_loss} and \ref{fig:cifar_three_accuracy}.

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{c1e-3_m_loss.png}
  \caption{Regularized training loss for $\eta = 0.001, \gamma = 0.9, \lambda=0.01$}
  \label{fig:cifar_three_loss}
\end{figure}

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{c1e-3_m_accuracy.png}
  \caption{Training accuracy for $\eta = 0.001$}
  \label{fig:cifar_three_accuracy}
\end{figure}

Increasing the learning rate to $\eta = 0.5$ yields the training performance shown in figures
\ref{fig:cifar_one_loss} and \ref{fig:cifar_one_accuracy}

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{c5e-1_m_loss.png}
  \caption{Regularized training loss for $\eta = 0.5, \gamma = 0.9, \lambda=0.01$}
  \label{fig:cifar_one_loss}
\end{figure}

\begin{figure}[!ht]
  \includegraphics[width=\textwidth,height=0.35\textheight,keepaspectratio]{c5e-1_m_accuracy.png}
  \caption{Training accuracy for $\eta = 0.5$}
  \label{fig:cifar_one_accuracy}
\end{figure}

\subsection{Testing}
The accuracy obtained for the network on the test data is 0.403.

\end{document}


\documentclass{subfile}

\begin{document}
	\section{BuNO}\label{sec:buno}

		\begin{problem}[$2020$ Day $1$, problem $2$]
			Let $b_{1},\ldots,b_{n}$ be non-negative integers and $a_{0},a_{1},\ldots,a_{n}$ be real numbers such that $b_{1}+\ldots+b_{n}=2$ and $a_{0}=a_{n}=0,|a_{i}-a_{i-1}|\leq b_{i}$ for $1\leq i\leq n$. Prove that
				\begin{align*}
					\sum_{i=1}^{n}(a_{i}+a_{i-1})b_{i}
						& \leq 2
				\end{align*}
		\end{problem}

		\begin{problem}[$2018$, problem $3$]
			Prove that
				\begin{align*}
					\left(\dfrac{6}{5}\right)^{\sqrt{3}}
						& > \left(\dfrac{5}{4}\right)^{\sqrt{2}}
				\end{align*}
		\end{problem}

		\begin{problem}[$2016$, problem $3$]
			For positive real numbers $a,b,c$ and $d$, prove that
				\begin{align*}
					\dfrac{a+\sqrt{ab}+\sqrt[3]{abc}+\sqrt[4]{abcd}}{4}
						& \leq \sqrt[4]{a\cdot\dfrac{a+b}{2}\cdot\dfrac{a+b+c}{3}\cdot\dfrac{a+b+c+d}{4}}
				\end{align*}
		\end{problem}

		\begin{problem}[$2009$, problem $6$]
			Let $a_{1},\ldots,a_{n},b_{1},\ldots,b_{n}$ be arbitrarily taken real numbers and $c_{1},\ldots,c_{n}$ be positive real numbers, then
				\begin{align*}
					\left(\sum_{i,j=1}^{n}\dfrac{a_{i}a_{j}}{c_{i}+c_{j}}\right)\left(\sum_{i,j=1}^{n}\dfrac{b_{i}b_{j}}{c_{i}+c_{j}}\right)
						& \geq \left(\sum_{i,j=1}^{n}\dfrac{a_{i}b_{j}}{c_{i}+c_{j}}\right)^{2}
				\end{align*}
		\end{problem}

		\begin{problem}[$2008$, problem $3$]
			Let $n$ be a natural number and $a_{1},\ldots,a_{n},b_{1},\ldots,b_{n}$ be real positive numbers such that $0\leq a_{1}\leq\ldots\leq a_{n}\leq\pi$ and
				\begin{align*}
					\left|\sum_{i=1}^{n}b_{i}\cos{ka_{i}}\right|
						& < \dfrac{1}{k}
				\end{align*}
			for all positive integer $k$. Prove that $b_{1}=\ldots=b_{n}=0$.
		\end{problem}

		\begin{problem}[$2007$ Team Selection Test, problem $3$]
			Let $n\geq2$ be a positive integer. Find the best constant $C(n)$ such that
				\begin{align*}
					\sum_{i=1}^{n}x_{i}
						& \geq C(n)\sum_{1\leq j<i\leq n}(2x_{i}x_{j}+\sqrt{x_{i}x_{j}})
				\end{align*}
			is true for all $x_i\in(0,1)$ such that
				\begin{align*}
					(1-x_{i})(1-x_{j})
						& \geq \dfrac{1}{4}
				\end{align*}
			for $1\leq i<j\leq n$.
		\end{problem}

		\begin{problem}[$1997$, problem $1$]
			Let $a,b,c$ be positive real numbers such that $abc=1$. Prove that
				\begin{align*}
					\dfrac{1}{1+b+c}+\dfrac{1}{1+c+b}+\dfrac{1}{1+a+b}
						& \leq \dfrac{1}{2+a}+\dfrac{1}{2+b}+\dfrac{1}{2+c}
				\end{align*}
		\end{problem}
\end{document}


\section{Task 6}
% Given a n-by-n tori and an n-dimensional hypercube decide things for node
% count 4, 16, 64, 256

% N   | tori n | cube k
% 4       2        2
% 16      4        4
% 64      8        6
% 256    16        8
\subsection{Task 6.a}
% At what scale does the hypercube provide (strictly) higher bisection width than the torus?

Table \ref{tab:task6a1} shows the different dimensions of the two patterns for
the specified networks sizes, this is used to calculate the bisection widths
which can be seen in Table \ref{tab:task6a2}. We can see that at $N=64$ the
hypercube provides a strictly greater bisection width.

\begin{table}[H]
    \centering
    \begin{tabular}{|r|r|r|}
        \hline
        $N$ & Tori $n$ & Cube $k$ \\\hline
          4 &  2       & 2        \\
         16 &  4       & 4        \\
         64 &  8       & 6        \\
        256 & 16       & 8        \\\hline
    \end{tabular}
    \caption{The different variables needed for calculating the dimensions of
        the two patterns.}
    \label{tab:task6a1}
\end{table}

\begin{table}[H]
    \centering
    \begin{tabular}{|l|l|l|}
        \hline
        $N$     & Tori & HyperCube \\ \hline
        Formula & $2n$ & $2^{k-1}$ \\
        4       &  4   &   2       \\
        16      &  8   &   8       \\
        64      & 16   &  32       \\
        256     & 32   & 128       \\ \hline
    \end{tabular}
    \caption{My caption}
    \label{tab:task6a2}
\end{table}

\subsection{Task 6.b}
% Determine the network diameter and the switch degree for the scale at which the hypercube
% provides a higher bisection width than the torus.

% Scale: N=64.
% n/k = 8/6
% Network Diameter: 8/6
% Switch degree: 4/6

Using the formulas from \cite[slide 38]{l7Interconnect} for scale $N=64$ as
calculated in the previus sub task, we have a $8$-by-$8$ tori and a
$6$-dimensional hypercube, so we have $n=8$ and $k=6$. This gives the tori a
network diameter of $8$ and the hypercube a network diameter of $6$. The switch
degree for the tori is $4$ regardless of the size of the tori, and for the
hypercube it is $6$.

\subsection{Task 6.c}
% What can you say about the relative merits of the two topologies?
\begin{itemize}
    \item[Hyper-cybe] The high bisection width allows for potential high
    bandwidth through the many links, however this also comes with a high
    switching degree, and for higher dimensions hyper-cubes, they layout of the
    cube itself becomes highly complicated, which makes the hyper-cube a very
    costly interconnection pattern.

    \item[Torus] A constant switching degree means that nodes can be added
    without adding complexity to the rest of the system. The slower growing
    bisection width means that potential bandwidth doesn't increase as fast as
    with the hyper-cube.
\end{itemize}


\subsection{Type graphs}
\label{subsec:formalisations:groove_formalisation:type_graphs}

In GROOVE, type graphs are used to constrain the valid instance graphs within the grammar. From a type graph follows a set of valid instance graphs that can be used for verification.

\begin{defin}[Type graph]
\label{defin:formalisations:groove_formalisation:type_graphs:type_graph}
A type graph is modeled as tuple $TG$:
\begin{equation}
    TG = \langle NT, ET, \sqsubseteq, abs, \mathrm{mult}, contains \rangle
\end{equation}
with
\begin{itemize}
    \item $NT \subseteq Lab_t \cup Lab_{prim}$ is the set of nodes in the type graph. The nodes can consist of type labels (see \cref{defin:formalisations:groove_formalisation:definitions:labels}) or primitive type labels (see \cref{defin:formalisations:groove_formalisation:definitions:primitive_type_labels}).

    \item $ET \subseteq NT \times (Lab_e \cup Lab_f) \times NT$ is the set of (directed) edges in the type graph, which is a set of triples containing the source and target node, as well as the edge label or flag label (see \cref{defin:formalisations:groove_formalisation:definitions:labels}) used to identify the edge.

    \item $\sqsubseteq\ \subseteq NT \times NT$ is the inheritance relation, the set of tuples of nodes between which an inheritance relation exists.

    \item $abs \subseteq NT$ is the (possibly empty) subset of nodes in the type graph which are considered abstract. An instance graph cannot instantiate abstract nodes.

    \item $\mathrm{mult}: ET \Rightarrow \mathbb{M} \times \mathbb{M}$ is the function which maps edges to their multiplicity pair. See \cref{defin:formalisations:groove_formalisation:type_graphs:multiplicity_pair} for the definition.

    \item $contains \subseteq ET$ is the set of edges which identify an containment relation.
\end{itemize}

\isabellelref{type_graph}{GROOVE.Type_Graph}
\end{defin}

\begin{figure}[p]
    \centering
    \begin{subfigure}{\textwidth}
        \centering
        \input{images/03_formalisations/03_groove_formalisation/type_graph_example.tikz}
        \caption{Type graph in GROOVEs visual notation. Multiplicities are omitted for clarity.}
    \end{subfigure}

    \begin{subfigure}{\textwidth}
        \centering
        \begin{align*}
            NT_{TG} =\ & \{
                \type{House},
                \type{PaymentInterval},
                \type{PaymentInterval\$MONTH},\\&
                \type{PaymentInterval\$QUARTER},
                \type{Person},
                \type{Renter},
                \type{Room},
                \type{int},
                \type{string}
            \}\\
            ET_{TG} =\ & \{
                ( \type{House}, \type{name}, \type{string} ),
                ( \type{House}, \type{rooms}, \type{Room} ),\\&
                ( \type{Person}, \type{age}, \type{int} ),
                ( \type{Person}, \type{name}, \type{string} ),\\&
                ( \type{Renter}, \type{payment\_interval}, \type{PaymentInterval} ),
                ( \type{Renter}, \type{rents}, \type{Room} ),\\&
                ( \type{Room}, \type{number}, \type{int} ),
                ( \type{Room}, \type{renter}, \type{Renter} )
            \}\\
            \sqsubseteq_{TG}\: =\ & \{
                ( \type{House}, \type{House} ),
                ( \type{PaymentInterval}, \type{PaymentInterval} ),\\&
                ( \type{PaymentInterval\$MONTH}, \type{PaymentInterval} ),\\&
                ( \type{PaymentInterval\$MONTH}, \type{PaymentInterval\$MONTH} ),\\&
                ( \type{PaymentInterval\$QUARTER}, \type{PaymentInterval} ),\\&
                ( \type{PaymentInterval\$QUARTER}, \type{PaymentInterval\$QUARTER} ),\\&
                ( \type{Person}, \type{Person} ),
                ( \type{Renter}, \type{Person} ),
                ( \type{Renter}, \type{Renter} ),
                ( \type{Room}, \type{Room} ),
                ( \type{int}, \type{int} ),
                ( \type{string}, \type{string} )
            \}\\
            abs_{TG} =\ & \{
                \type{PaymentInterval}, \type{Person}
            \}\\
            \mathrm{mult}_{TG} =\ & \big\{
                \big( ( \type{House}, \type{name}, \type{string} ), ( 0..*, 1..1 ) \big),
                \big( ( \type{House}, \type{rooms}, \type{Room} ), ( 1..1, 1..* ) \big),\\&
                \big( ( \type{Person}, \type{age}, \type{int} ), ( 0..*, 1..1 ) \big),
                \big( ( \type{Person}, \type{name}, \type{string} ), ( 0..*, 1..1 ) \big),\\&
                \big( ( \type{Renter}, \type{payment\_interval}, \type{PaymentInterval} ), ( 0..*, 1..1 ) \big),\\&
                \big( ( \type{Renter}, \type{rents}, \type{Room} ), ( 0..*, 0..* ) \big),\\&
                \big( ( \type{Room}, \type{number}, \type{int} ), ( 0..*, 1..1 ) \big),
                \big( ( \type{Room}, \type{renter}, \type{Renter} ), ( 0..*, 0..1 ) \big)
            \big\}\\
            contains_{TG} =\ & \{
                ( \type{House}, \type{rooms}, \type{Room} )
            \}
        \end{align*}
        \caption{Formal definition of the type graph}
    \end{subfigure}
    \caption{Example of a type graph corresponding with \cref{defin:formalisations:groove_formalisation:type_graphs:type_graph}}
    \label{fig:formalisations:groove_formalisation:type_graphs:type_graph_example}
\end{figure}

An example of a type graph is given in \cref{fig:formalisations:groove_formalisation:type_graphs:type_graph_example}. This example is similar to the type model example discussed in \cref{subsec:formalisations:ecore_formalisation:type_models}. There is a node $\type{House}$ which contains $\type{Room}$s. A $\type{House}$ also has an edge to a primitive type label $\type{string}$ under edge label $\type{name}$ which represents the name of the house. Please note that in the visual representation, syntactic sugar is used to represent this edge. Instead of an extra node and edge, it is represented as part of the $\type{House}$ node. This syntactic sugar can be used for edges to primitive types and are in reality still treated as an edge to a separate node type. A $\type{Room}$ has an edge $\type{number}$, targeting the primitive type label $\type{int}$, which represents the number of the room within the house. A $\type{Room}$ can be rented by a $\type{Renter}$. The $\type{Renter}$s have edges to the $\type{Room}$s they rented under the edge label $\type{rents}$, while a $\type{Room}$ can access its $\type{Renter}$ through the edge with edge label $\type{renter}$. A $\type{Renter}$ extends the abstract $\type{Person}$ node type, which has 2 edges $\type{age}$ and $\type{name}$, targeting the primitive type labels $\type{int}$ and $\type{string}$ respectively. These edges represent the age and the name of the $\type{Person}$. Finally, a $\type{Renter}$ has an edge under the edge label $\type{payment\_interval}$, which points to a $\type{PaymentInterval}$ node type. This node type is abstract and the edge should therefore point to one of its subtypes, $\type{PaymentInterval\$MONTH}$ or $\type{PaymentInterval\$QUARTER}$. This represents the interval in which the $\type{Renter}$ pays the rent.
Notable from the definition is that the nodes set $N$ can contain primitive type labels. As a consequence, primitive type labels need to be added explicitly to a type graph in order to use primitive type values in an instance graph.

Furthermore, each edge has a multiplicity pair tied to it, which is defined as the $\mathrm{mult}$ function in the type graph definition. The multiplicity pair consists of an incoming multiplicity and an outgoing multiplicity. The incoming multiplicity determines the allowed amount of nodes that share the same target node with this edge type. On the other hand, the outgoing multiplicity determines the number of edges a single source node may have to its target nodes.

\begin{defin}[Multiplicity pair]
\label{defin:formalisations:groove_formalisation:type_graphs:multiplicity_pair}
A multiplicity pair is defined as a tuple of two multiplicities, $\mathbb{M} \times \mathbb{M}$, in which the first value denotes the incoming multiplicity and the second value the outgoing multiplicity.

For any multiplicity pair, we define two functions:
\begin{align*}
    \mathrm{in}\!:&\: \mathbb{M} \times \mathbb{M} \Rightarrow \mathbb{M} \\
    \mathrm{out}\!:&\: \mathbb{M} \times \mathbb{M} \Rightarrow \mathbb{M}
\end{align*}
The $\mathrm{in}$ function being the function which from a multiplicity pair returns the incoming multiplicity and the $\mathrm{out}$ function being the function that returns the outgoing multiplicity, so:
\begin{equation*}
\forall m = (m_{in}, m_{out}) \in \mathrm{mult}_{TG}: \mathrm{in}(m) = m_{in} \land \mathrm{out}(m) = m_{out}
\end{equation*}

\isabellelref{multiplicity_pair}{GROOVE.Multiplicity_Pair}
\end{defin}

With all definitions in place, it is possible to define a valid type graph.
The definition of a valid type graph introduces some new constraint that should hold for a type graph to be valid.

\begin{defin}[Type graph validity]
\label{defin:formalisations:groove_formalisation:type_graphs:type_graph_validity}
For a type graph to be valid, the following properties must hold:
\begin{enumerate}
    \item There may not be any ambiguity in the use of edges: $\forall (s_1, l, t_1) \in ET_{TG}\,\land\, (s_2, l, t_2) \in ET_{TG}\!: \big((s_1, s_2) \in\ \sqsubseteq_{TG} \lor\ (s_2, s_1) \in\ \sqsubseteq_{TG}\!\!\big) \land \big((t_1, t_2) \in\ \sqsubseteq_{TG} \lor\ (t_2, t_1) \in\ \sqsubseteq_{TG}\!\!\big) \Longrightarrow s_1 = s_2 \land t_1 = t_2$.
    \item Flags should have the same source and target node: $\forall (s, l, t) \in ET_{TG}: l \in Lab_f \Longrightarrow s = t$.
    \item $\sqsubseteq_{TG}$ is a partial order ($\sqsubseteq_{TG}$ is reflexive, transitive and anti-symmetric on $N\!$).
    \item The incoming multiplicities of edges that identify a containment relation are valid: $\forall e \in contains_{TG}: \mathrm{in}(\mathrm{mult}_{TG}(e)) = (0, 1) \lor \mathrm{in}(\mathrm{mult}_{TG}(e)) = (1, 1)$.
\end{enumerate}

\isabellelref{type_graph}{GROOVE.Type_Graph}
\end{defin}

\begin{figure}
    \centering
    \begin{subfigure}{0.45\textwidth}
        \centering
        \input{images/03_formalisations/03_groove_formalisation/edge_ambiguity_example_valid.tikz}
        \caption{Valid type graph without ambiguity}
        \label{fig:formalisations:groove_formalisation:type_graphs:ambiguous_edges_example:valid}
    \end{subfigure}
    \begin{subfigure}{0.45\textwidth}
        \centering
        \input{images/03_formalisations/03_groove_formalisation/edge_ambiguity_example_invalid.tikz}
        \caption{Invalid type graph with an ambiguous edge type labelled $\type{f}$}
        \label{fig:formalisations:groove_formalisation:type_graphs:ambiguous_edges_example:invalid}
    \end{subfigure}
    \caption{Example of ambiguity within edge types}
    \label{fig:formalisations:groove_formalisation:type_graphs:ambiguous_edges_example}
\end{figure}

The last 3 properties presented here are mostly self-explanatory. The first property might be unclear at first. This property prevents type graphs from having ambiguous edge types. \cref{fig:formalisations:groove_formalisation:type_graphs:ambiguous_edges_example} shows an example of such an ambiguity. In essence, when creating edges within an instance graph, there should be an unique solution for typing the edge. In \cref{fig:formalisations:groove_formalisation:type_graphs:ambiguous_edges_example:valid}, this is always the case, even though both edges are labelled $\type{f}$. If an edge labelled $\type{f}$ references a node of type $\type{B}$, then the edge type should be $( \type{Y}, \type{f}, \type{B} )$. When an edge labelled $\type{f}$ references a node of type $\type{A}$, the edge type should be $( \type{X}, \type{f}, \type{A} )$. There is no ambiguity possible.

\cref{fig:formalisations:groove_formalisation:type_graphs:ambiguous_edges_example:invalid} shows an example of a type graph where ambiguity is possible. When a node of type $\type{Y}$ references a node of $\type{B}$ using an edge labelled $\type{f}$, it is unclear which edge type was meant. Both $( \type{Y}, \type{f}, \type{B} )$ and $( \type{X}, \type{f}, \type{A} )$ would be valid edge types here. This means there is ambiguity in how edges are typed. The first property of \cref{defin:formalisations:groove_formalisation:type_graphs:type_graph_validity} excludes this case, since $( \type{Y}, \type{f}, \type{B} )$ and $( \type{X}, \type{f}, \type{A} )$ are both edge types, while $(\type{Y}, \type{X}) \in\ \sqsubseteq_{TG}$ and $(\type{B}, \type{A}) \in\ \sqsubseteq_{TG}$. Then according to the first property, $\type{Y}$ should be equal to $\type{X}$ and $\type{B}$ should be equal to $\type{A}$, which is not the case, so \cref{fig:formalisations:groove_formalisation:type_graphs:ambiguous_edges_example:invalid} violates the first property, hence the example is invalid.


%%
%% This is file `./samples/shortsample.tex',
%% generated with the docstrip utility.
%%
%% The original source files were:
%%
%% apa6.dtx  (with options: `shortsample')
%% ----------------------------------------------------------------------
%%
%% apa6 - A LaTeX class for formatting documents in compliance with the
%% American Psychological Association's Publication Manual, 6th edition
%%
%% Copyright (C) 2011-2017 by Brian D. Beitzel <brian at beitzel.com>
%%
%% This work may be distributed and/or modified under the
%% conditions of the LaTeX Project Public License (LPPL), either
%% version 1.3c of this license or (at your option) any later
%% version.  The latest version of this license is in the file:
%%
%% http://www.latex-project.org/lppl.txt
%%
%% Users may freely modify these files without permission, as long as the
%% copyright line and this statement are maintained intact.
%%
%% This work is not endorsed by, affiliated with, or probably even known
%% by, the American Psychological Association.
%%
%% ----------------------------------------------------------------------
%%
\documentclass[jou]{apa6}

\usepackage[american]{babel}

\usepackage{csquotes}
\usepackage[style=apa,sortcites=true,sorting=nyt,backend=biber]{biblatex}
\DeclareLanguageMapping{american}{american-apa}
\addbibresource{bibliography.bib}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Discrete Structures
%% The start of RBS stuff
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Working internal and external links in PDF
\usepackage{hyperref}
% Extra math symbols in LaTeX
\usepackage{amsmath}
\usepackage{gensymb}
\usepackage{amssymb}
% Enumerations with (a), (b), etc.
\usepackage{enumerate}

\let\OLDitemize\itemize
\renewcommand\itemize{\OLDitemize\addtolength{\itemsep}{-6pt}}

\usepackage{etoolbox}
\makeatletter
\preto{\@verbatim}{\topsep=3pt \partopsep=3pt }
\makeatother

% These sizes redefine APA for A4 paper size
\oddsidemargin 0.0in
\evensidemargin 0.0in
\textwidth 6.27in
\headheight 1.0in
\topmargin -24pt
\headheight 12pt
\headsep 12pt
\textheight 9.19in

\title{Sample Quiz for Week02}
\author{Discrete Structures, Fall 2020}
\affiliation{RBS}

\leftheader{Discrete Structures (W2): Sample Quiz}

\abstract{%
}

%\keywords{}

\begin{document}

\twocolumn

\section{Worksheet 2: Predicates}

{\bf Question 1.} Let $a,b \in \mathbb{Z}^{+}$ be two positive integers.
Translate into predicate logic: ``$d$ is the greatest common divisor
of $a,b$.'' (That is,
the greatest number that divides both $a$ and $b$).\\
{\em Note 1.} "Translate into predicate logic" means - use
predicates and quantifiers to express that statement.
{\em Note 2.} Use ``infix'' notation for common predicates: write
$a\,\mid\,b$ whenever $a$ divides $b$; write $x < y$, if $x$ is less than $y$. For example, $3\,|\,6$ \hspace{1ex}
($3$ divides $6$) is preferred compared to
``$\mathtt{divides}(3,6)$''. Also \hspace{1ex}
$10 < 17$ is more readable than
``$\mathtt{lessThan(10,17)}$''.

\vspace{10pt}
{\bf Question 2.} Use quantifiers to write a
statement to tell that a quadratic
function $f(x) = ax^2 +bx+c$
has two different integer roots.

\vspace{10pt}
{\bf Question 3.} Assume that for some argument {\tt x},
Python functions {\tt a(x)} and {\tt b(x)}
return value {\tt True}, but other two functions
{\tt c(x)} and {\tt d(x)} return value {\tt False}.
Which functions are called and evaluated, if you run the following conditional statement:
\begin{verbatim}
if (a(x) or b(x)) and (c(x) & d(x)):
    ## ... some Python code ...
\end{verbatim}
{\em Note.} Note that Boolean operators {\tt and}, {\tt or}
use {\em short-circuit
evaluation}, but operators {\tt \&}, {\tt |} do not.

\vspace{10pt}
{\bf Question 4.} There is a set $C$ of several children and a
set $H$ of several hats. There is a predicate $W(c,h)$ which is {\tt True}
iff the child $c \in C$ has ever worn the hat $h \in H$.\\
{\bf (a)} Write the domain set and the range set of the predicate function $W$. \\
{\bf (b)} Use quantifiers to write a statement: ``Every two
children have at least one hat in common (that is, both of
them have worn it).''

\vspace{10pt}
{\bf Question 5.}
Let $\mathcal{H}$ be the set of all humans, and the predicate $F(x,y)$ is
true iff $x$ is the father of $y$. Express these sentences in plain English:\\
{\bf (a)} $\forall y \in \mathcal{H}\;\exists x \in \mathcal{H},\;F(x,y)$.\\
{\bf (b)} $\exists x \in \mathcal{H}\;\forall y \in \mathcal{H},\;F(x,y)$.\\
{\bf (c)} Which of the two statements {\bf (a)}, {\bf (b)} (if any)
is true, if we replace the non-empty domain of all humans $\mathcal{H}$ by an
empty domain $\mathcal{Z} = \emptyset$ of all zombies?

\vspace{10pt}
{\bf Question 6.} We define predicates $P(x,y)$,
$Q(x,y)$ with two arguments,
where $x,y$ can be any of the three letters:
$\mathtt{a},\mathtt{b},\mathtt{c}$.
(The type for all these predicates is
$\{ \mathtt{a},\mathtt{b},\mathtt{c} \}^2
\rightarrow \{ \mathtt{T},\mathtt{F} \}$.)

They have these following truth tables:
\begin{center}
\begin{tabular}{c|ccc}
$P$ & $a$ & $b$ & $c$ \\ \hline
$a$ & $\mathtt{T}$ & $\mathtt{F}$ & $\mathtt{T}$ \\
$b$ & $\mathtt{F}$ & $\mathtt{F}$ & $\mathtt{F}$ \\
$c$ & $\mathtt{T}$ & $\mathtt{T}$ & $\mathtt{T}$
\end{tabular}
\hspace{2ex}
\begin{tabular}{c|ccc}
$Q$ & $a$ & $b$ & $c$ \\ \hline
$a$ & $\mathtt{F}$ & $\mathtt{T}$ & $\mathtt{F}$ \\
$b$ & $\mathtt{F}$ & $\mathtt{F}$ & $\mathtt{T}$ \\
$c$ & $\mathtt{T}$ & $\mathtt{F}$ & $\mathtt{F}$
\end{tabular}
\end{center}
Find the truth values of these statements:\\
{\bf (a)} $\exists x \forall y, P(y,x)$\\
{\bf (b)} $\exists x \forall y, P(x,y)$\\
{\bf (c)} $\exists x \exists y, Q(x,y)$\\
{\bf (d)} $\forall x \exists y, Q(x,y)$\\
{\em Note.} In all truth tables the first argument is
represented by row, the second is represented by column.
For example $P(\mathtt{b}, \mathtt{c}) = \mathtt{F}$ (2nd row
and 3rd column). Meanwhile
$P(\mathtt{c}, \mathtt{b}) = \mathtt{T}$ (3rd row and 2nd column).

\vspace{10pt}
{\bf Question 7.}
For a real number $x$ we know that $\lfloor x \rfloor \neq
\lfloor x + 0.5 \rfloor$. Write this statement in
predicate logic without using any $\lfloor \ldots \rfloor$
notation. (Write instead that $x$ and $x+0.5$ have some integer number
betwen $x$  and $x+0.5$.)\\
{\em Note.} By $\lfloor x \rfloor$ we denote the
largest integer number that does not exceed $x$.
For example $\lfloor 3.14 \rfloor = 3$, $\lfloor 17 \rfloor = 17$,
$\lfloor -4.5 \rfloor = -5$.

\newpage

\subsection{Answers}

{\bf Question 1.} Answer:
$$\boxed{\forall k \in \mathbb{Z}^{+},\;\left(k\,\mid\,a \wedge
k\,\mid\,b\right)\,\rightarrow\,k \leq d.}$$
{\em Recite:} ``For all positive integers $k$, if $k$ divides
both $a$ and $b$, then $k$ does not exceed
$d = \operatorname{gcd}(a,b)$.''\\
This also means that $d$ is the largest number among all
common divisors of $a,b$ ({\em greatest common divisor}, GCD).\\
{\em Note:} In the above formula $a,b,d$ are ``free variables''
(they need to be assigned independently; and if
$d \neq \operatorname{gcd}(a,b)$, then the statement is false).
On the other hand, $k$ is a ``bound variable''. You can rename
$k$ into $x$ \textendash{} and nothing will change.

\vspace{10pt}
{\bf Question 2.} Answer:
\begin{align}
 & \exists x_1 \in \mathbb{Z}^{+}\,\exists x_2 \in \mathbb{Z}^{+}\,
\forall x_3 \in \mathbb{Z}^{+}, \nonumber \\
 & \left( x_1 \neq x_2 \wedge f(x_1)=0 \wedge f(x_2) = 0 \right) \wedge \nonumber \\
\wedge & \left( f(x_3) = 0 \rightarrow x_3 = x_1 \vee x_3 = x_2 \right).
\nonumber
\end{align}
{\em Recite:} ``There exist two positive integers $x_1,x_2$
such that they are different and
both of them are roots of $f(x)=0$ and,
furthermore, for any other root $x_3$, it
equals either $x_1$ or $x_2$.''\\
{\em Note:} The above statement says that there are {\em exactly}
two roots $x_1$ and $x_2$. You can easily write a modified statement
saying that the equation $f(x)=0$ has {\em at least} two roots \textendash{}
in this case you can skip the $x_3$ part. For quadratic equations it
is the same (since no equation has more than $2$ roots), but
the idea expressed here is slightly different:
\begin{align}
 & \exists x_1 \in \mathbb{Z}^{+}\,\exists x_2 \in \mathbb{Z}^{+} \nonumber \\
 & \left( x_1 \neq x_2 \wedge f(x_1)=0 \wedge f(x_2) = 0 \right). \nonumber
\end{align}

\vspace{10pt}
{\bf Question 3.} Answer:\\
\underline{The following functions are called: {\tt a(x)},
{\tt c(x)}, {\tt d(x)}.}\\
Function {\tt b(x)} is not called, because
short-circuit operator {\bf or}
skips the second argument in ``{\tt a(x) or b(x)}'',
if the first argument is {\tt True}.

\vspace{10pt}
{\bf Question 4.} Answer: {\bf (a)}
$$\boxed{W\,:\,C \times H \rightarrow \{ \mathtt{T}, \mathtt{F} \}.}$$
{\em Note.} In Coq the set of ``True'' and ``False''
is denoted by {\tt Prop} or $\{ \mathtt{T}, \mathtt{F} \}$.\\
{\bf (b)}
$$\boxed{\forall c_1 \in C \; \forall c_2 \in C \; \exists h \in H,
W(c_1,h) \wedge W(c_2,h).}$$
Every two
children have at least one hat in common (that is, both
of them have worn it).

\vspace{10pt}
{\bf Question 5.} Answers:\\
{\bf (a)} \underline{Every human has a father.}\\
{\bf (b)} \underline{There exists someone, who is the father of everyone.}\\
{\bf (c)} \underline{Both (a) and (b) are false for empty sets.}\\
Every time we write $\exists x \in \mathcal{Z}$
regarding anything (where $\mathcal{Z} = \emptyset$ \textendash{}
an empty set) it is false.

On the other hand, this statement is {\tt True}:
$$\forall x \in \mathcal{Z} \; \forall y \in \mathcal{Z},\; F(x,y) \equiv \mathtt{True}.$$
{\em Recite:} ``In the (empty) set $\mathcal{Z}$ of all
zombies, any zomby is a father of any other zomby.''\\
{\em Note.} For empty domains we do not need to know anything
about the predicate values, because their truth tables have
zero rows and zero columns. You can also safely state the negation:
``No zomby is a father of another zomby.''
$$\forall x \in \mathcal{Z} \; \forall y \in \mathcal{Z},\; \neg F(x,y) \equiv \mathtt{True}.$$

\vspace{10pt}
{\bf Question 6.} Answers:\\
{\bf (a)} False. In the truth table of $P$, there is NO
column (denoted by $x$) containing $\mathtt{T}$ in
every row.\\
{\bf (b)} True. In the truth table of $P$ there is a row $x$
(it is the last row $x = \mathtt{c}$), containing
$\mathtt{T}$ in every column.\\
{\bf (c)} True. We can indeed make $Q(x,y)$ true, if we pick
row and column. For example $Q(\mathtt{c},\mathtt{a}) = \mathtt{T}$.\\
{\bf (d)} True. For any row in the truth table of $Q$,
we can find at least one $\mathtt{T}$ in some column.

\vspace{10pt}
{\bf Question 7.} Answer:\\
$$\exists k \in \mathbb{Z}, x < k \wedge x+0.5 \geq k.$$
In this example $\lfloor x \rfloor = k-1$, but
$\lfloor x+0.5 \rfloor = k$, so they are different.

\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% End of RBS stuff
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%
%% Copyright (C) 2011-2017 by Brian D. Beitzel <brian at beitzel.com>
%%
%% This work may be distributed and/or modified under the
%% conditions of the LaTeX Project Public License (LPPL), either
%% version 1.3c of this license or (at your option) any later
%% version.  The latest version of this license is in the file:
%%
%% http://www.latex-project.org/lppl.txt
%%
%% Users may freely modify these files without permission, as long as the
%% copyright line and this statement are maintained intact.
%%
%% This work is not endorsed by, affiliated with, or probably even known
%% by, the American Psychological Association.
%%
%%
%% This work is "maintained" (as per LPPL maintenance status) by
%% Brian D. Beitzel.
%%
%% This work consists of the file  apa6.dtx
%% and the derived files           apa6.ins,
%%                                 apa6.cls,
%%                                 apa6.pdf,
%%                                 README,
%%                                 APAamerican.txt,
%%                                 APAbritish.txt,
%%                                 APAdutch.txt,
%%                                 APAenglish.txt,
%%                                 APAgerman.txt,
%%                                 APAngerman.txt,
%%                                 APAgreek.txt,
%%                                 APAczech.txt,
%%                                 APAturkish.txt,
%%                                 APAendfloat.cfg,
%%                                 apa6.ptex,
%%                                 TeX2WordForapa6.bas,
%%                                 Figure1.pdf,
%%                                 shortsample.tex,
%%                                 longsample.tex, and
%%                                 bibliography.bib.
%%
%%
%% End of file `./samples/shortsample.tex'.


% !TEX root = ../zeth-protocol-specification.tex

\section{Ethereum}\label{preliminaries:ethereum}

In a nutshell, \ethereum~is a distributed deterministic state machine, consisting of a globally accessible singleton state (``the World state'') and a virtual machine that applies changes to that state~\cite{mastering-eth}.
State transitions in the state machine are represented by transactions on the system. As such, each transaction represents a change in the global state represented as a Merkle Patricia Tree~\cite{patricia-tree} whose nodes are objects called ``accounts'' (\cref{preliminaries:ethereum:eth-account}). The Ethereum Virtual Machine (\evm) allows state transitions to be specified by creating a type of accounts which are associated with a piece of code (smart-contracts). The code of such accounts, and so, the corresponding state transitions, can be executed to transition to another state in the automata, by creating a transaction that calls the given piece of code (\cref{preliminaries:ethereum:eth-tx}).

To prevent unbounded state transitions in the state machine, each instruction executed by the \evm~is associated with a cost in \wei, referred to as ``the gas necessary to run the operation''. The ``gas cost'' of a transaction needs to be paid by the transaction originator (deduced from their account balance), and is awarded to the miner (added to their account balance) who successfully mines the block containing the transaction.
In addition to the cost of every instruction executed as part of a state transition, every transaction has an intrinsic cost of $\txDefaultGas$ gas~\cite[Appendix G]{ethyellowpaper}. Bounding modifications to the $\ethereum{}$ state by the amount of \wei~held in the transaction originator's account allows the system to avoid the Halting problem\footnote{\url{https://en.wikipedia.org/wiki/Halting\_problem}} and protects against a range of Denial of Service (\dos) attacks.

\subsection{Ethereum account}\label{preliminaries:ethereum:eth-account}

An \ethereum~account~\cite[Section 4.1]{ethyellowpaper} is an object containing 4 attributes, as represented~\cref{preliminaries:tab:eth-account}.
We distinguish two types of accounts:
\begin{itemize}
    \item ``Externally Owned Accounts'' (\eoa), that are created by derivation of an \ecdsa~secret key; and
    \item Smart-contract accounts, that are derived from \evm~code specifying a state transition on the state machine.
\end{itemize}

Each account object is accessible in the Merkle Patricia Tree representing the ``World state'' by a unique $\addressLen$-bit long identifier called the address.
In the context of \eoa, the address is obtained by generating a new \ecdsa~\cite{johnson2001elliptic} key pair $\smalltuple{\sk, \vk}$ over curve \secpCurve~\cite{qu1999sec} and taking the rightmost $\addressLen$ bits of the \keccak{256} hash of the verification key $\vk$.

\begin{table}[H]
    \centering
    \begin{tabular}{cp{25em}c}
        Field           & Description & Data type\\ \toprule
        $\nonce$        & The nonce of an account is a scalar value representing the number of transactions that have originated from the account, starting at 0. & $\NN_\ethWordLen$ \\ \midrule
        $\balance$      & The balance of an account is a scalar value representing the amount of \wei~in the account. & $\NN_\ethWordLen$\\ \midrule
        $\sroot$        & The storage root is the \keccak{256} hash representing the storage of the account. & $\BB^{\keccakTwoDigestLen}$\\ \midrule
        $\codeh$        & The code hash is the hash of the \evm~code governing the account. If this field is the \keccak{256} hash of the empty string, then the account is said to be an ``Externally owned Account'' (\eoa), and is controlled by the corresponding \ecdsa~private key. If, however, this field is not the \keccak{256} hash of the empty string, the account represents a smart contract whose interactions are governed by its \evm~code. & $\BB^{\keccakTwoDigestLen}$\\ \bottomrule
    \end{tabular}
    \caption{Ethereum Account structure}\label{preliminaries:tab:eth-account}
\end{table}

\begin{notebox}
In the rest of this document, we will refer to an \emph{Ethereum user} $\eparty{U}$ as a person, modeled as an object, holding \emph{one}\footnote{The same physical person may correspond to multiple ``$\ethereum{}$ users'' and thus control multiple accounts in the Merkle Patricia Tree.} secret key, $\sk$ (object attribute), associated with an existing \eoa~in the ``World state''. We denote by $\eparty{U}.\addr$ the \ethereum~address of $\eparty{U}$ derived from $\eparty{U}.\sk$, and which allows $\eparty{U}$ to access the state of their account $\wstate[\eparty{U}.\addr]$.

We denote by $\contractstyle{SmartC}$ a smart-contract instance/object (i.e.~deployed smart-contract with an address,~\cref{preliminaries:ethereum:eth-tx}), and denote by $\contractstyle{SmartC}.\addr$ its address.
\end{notebox}

\subsection{Ethereum transaction}\label{preliminaries:ethereum:eth-tx}

We now briefly mention what \ethereum~transactions~\cite[Section 4.2]{ethyellowpaper} are, and how they are created, signed and validated. Once more, the reader is highly encouraged to refer to~\cite{ethyellowpaper} for a detailed presentation.
%\subsubsection{Transaction}
Informally, a transaction object ($\tx$) is a signed message originating from an \ethereum~user $\eparty{U}$ (the \emph{transaction originator}, or simply \emph{sender}) that represents a state transition on the distributed state machine (i.e.~a change in the ``World state'' $\wstate$).

\subsubsection{Raw transaction}\label{preliminaries:ethereum:eth-tx:raw}
In the following, we define a raw transaction as an unsigned transaction (\cref{preliminaries:tab:eth-unsigned}).

\begin{table}[H]
    \centering
    \begin{tabular}{ccc}
        Field			    & Description & Data type \\\toprule
        $\nonce$            & Transaction nonce & $\NN_{\ethWordLen}$\\\midrule
        $\gasp$		        & gasPrice & $\NN_{\ethWordLen}$\\\midrule
        $\gasl$		        & gasLimit & $\NN_{\ethWordLen}$\\\midrule
        $\tto$		        & Recipient's address & $\BB^{\addressLen}$\\\midrule
        $\val$		        & Value of the transaction in $\wei$ & $\NN_{\ethWordLen}$\\\midrule
        $\init$ / $\data$	& \begin{tabular}{@{}c@{}}Contract Creation data $\init$ \\ Message call data $\data$\end{tabular} & $\BB^{*}$\\\bottomrule
    \end{tabular}
    \caption{Structure of a \emph{raw transaction data type} $\txRawDType$}\label{preliminaries:tab:eth-unsigned}
\end{table}

\subsubsection{Finalizing raw transactions}\label{preliminaries:ethereum:eth-tx:final}
A raw transaction needs to be finalized to be accepted. In the context of this document, ``finalizing a raw transaction'' will be a synonym of ``signing a raw transaction''. The transaction structure is represented in~\cref{preliminaries:tab:eth-signed}.

\begin{table}[H]
    \centering
    \begin{tabular}{ccc}
        Field           & Description & Data type \\ \toprule
        $\rawTx$      & Raw transaction object & $\txRawDType$ \\ \midrule
        $\sigv$         & Field $\sigv$ of $\ecdsa$ signature used for public key recovery & $\BB^{\byteLen}$\\ \midrule
        $\sigr$         & Field $\sigr$ of $\ecdsa$ signature~\cite{rfc6979} & $\FFx{\rSecp}$\\ \midrule
        $\sigs$         & Field $\sigs$ of $\ecdsa$ signature~\cite{rfc6979} & $\FFx{\rSecp}$\\ \bottomrule
    \end{tabular}
    \caption{Structure of a (finalized) \emph{transaction data type} $\txDType$}\label{preliminaries:tab:eth-signed}
\end{table}

We define the transaction generation function, cf.~\cref{preliminaries:fig:txgen}, as the function taking the sender's \ecdsa~signing key and the components of a raw transaction as arguments, and returning a signed (or finalized) transaction ($\finalTx$ or $\tx$ for short).
\begin{align*}
    \finalTx &= \txgen(\sk_{\ecdsa}, \inp{\nonce}, \inp{\gasp}, \inp{\gasl}, \inp{\tto}, \inp{\val}, \inp{\init}, \inp{\data})\\
    \finalTx &= \{ \\
                & \left.
                \begin{array}{l@{}l}
                    \nonce & {}: \inp{\nonce},\\
                    \gasp & {}: \inp{\gasp},\\
                    \gasl & {}: \inp{\gasl},\\
                    \tto & {}: \inp{\tto},\\
                    \val & {}: \inp{\val},\\
                    \init/\data & {}: \inp{\init}/\inp{\data},
                \end{array}
                \right\rbrace~\rawTx\\
                & \left.
                \begin{array}{l@{}l}
                    \sigv & {}: \sigma_\ecdsa.\sigv,\\
                    \sigr & {}: \sigma_\ecdsa.\sigr,\\
                    \sigs & {}: \sigma_\ecdsa.\sigs \\
                \end{array}
                \right\rbrace~\sigma_\ecdsa\\
                \}
\end{align*}

To sign a transaction, the sender first computes the hash of the raw transaction using $\keccak{256}$, cf.~\cref{preliminaries:eq:tx-sig-hash}, and then uses their \ecdsa~signing key, $\sk_\ecdsa$, to sign the obtained digest. cf.~\cref{preliminaries:eq:tx-sig-sig}. The signature is then appended to the raw transaction to obtain a finalized transaction, cf.~\cref{preliminaries:fig:txgen}.

\begin{align}
    \digest_\ecdsa &= \keccak{256}(\inp{\nonce}, \inp{\gasp}, \inp{\gasl}, \inp{\tto}, \inp{\val}, \inp{\init}/\inp{\data}) \label{preliminaries:eq:tx-sig-hash} \\
    \sigma_{\ecdsa} &= \ecdsasigscheme.\sig(\sk_\ecdsa, \digest_\ecdsa)\ (= \smalltuple{\sigv, \sigr, \sigs}) \label{preliminaries:eq:tx-sig-sig}
\end{align}

\begin{figure}[H]
    \centering
    \procedure[linenumbering]{$\txgen(\sk_\ecdsa, \inp{\nonce}, \inp{\gasp}, \inp{\gasl}, \inp{\tto}, \inp{\val}, \inp{\init}, \inp{\data})$}{%
    \pcif \inp{\tto} = \emptyset \pcdo\\
    \t \rawTx \gets \{\nonce: \inp{\nonce}, \gasp: \inp{\gasp}, \gasl: \inp{\gasl}, \tto: \inp{\tto}, \val: \inp{\val}, \init: \inp{\init}\}; \\
    \pcelse \\
    \t \rawTx \gets \{\nonce: \inp{\nonce}, \gasp: \inp{\gasp}, \gasl: \inp{\gasl}, \tto: \inp{\tto}, \val: \inp{\val}, \data: \inp{\data}\}; \\
    \pcendif \\
    \sigma_\ecdsa \gets \ecdsasigscheme.\sig(\sk_{\ecdsa}, \keccak{256}(\rawTx)); \\
    \finalTx \gets \{ \rawTx, \sigv: \sigma_\ecdsa.\sigv, \sigr: \sigma_\ecdsa.\sigr, \sigs: \sigma_\ecdsa.\sigs \}; \\
    \pcreturn\ \finalTx;
}
    \caption{Transaction generation function \txgen}\label{preliminaries:fig:txgen}
\end{figure}

\begin{remark}\label{preliminaries:recovering-msg-sender}
    As one can see, there is no ``from'' attribute in a transaction. The sender's \ethereum~address can be recovered from the \ecdsa~signature. This method is defined in the \ethereum~yellow paper as a ``sender function'' $S$~\cite[Appendix F]{ethyellowpaper} which maps each transaction to its sender.
\end{remark}

\subsubsection{Types of transactions}\label{preliminaries:ethereum:eth-tx:tx-types}

While only two types of transactions are described in~\cite[Section 4.2]{ethyellowpaper}; namely those which result in message calls and those which result in the creation of new accounts with associated code, we will instead differentiate the types of transactions based on their purpose. The reader is encouraged to read~\cite{ethyellowpaper} for a formal discussion.

\medskip

Informally, a transaction can be used to achieve three things: transferring \wei~from an \eoa~to another \eoa, creating a new account with associated code (i.e.~``deploying a smart-contract''), and calling a function of a smart-contract. We will detail here the differences between these usages.
\begin{description}
    \item[Creating a contract] The $\tx.\tto$ address is set to $\emptyset$ in the transaction. The contract creation data ($\tx.\init$) includes the new contract's code. The contract address is computed as the rightmost $\addressLen$ bits of the \keccak{256} hash of the \rlp~encoding~\cite{ethrlp} of the transaction originator's address and account nonce~\cite[Section 6]{ethyellowpaper}.
    \item[Calling a contract function] The $\tx.\tto$ address is set to the address of the contract. The message call data byte array ($\tx.\data$) is set to the contract's function address (or \emph{``Function Selector''}~\cite{abi-function-selector}) which are the first 4 bytes of the \keccak{256} hash of the function signature, and the function input arguments ($\ethWordLen$ bits per input)~\cite[Section 8]{ethyellowpaper}.
    \item[Transferring \wei~from an \eoa~to another \eoa] This corresponds to a ``plain transaction'' spending \wei~from an address to send them to another. In that case the $\tx.\tto$ address corresponds to the recipient's address while the transaction data is left empty.
\end{description}

\begin{notebox}
    In order to keep notations simple, we assume, in the rest of the document, that smart-contract functions are uniquely determined by their name. As such, we denote by $\funcSelec{\cdot} \colon \BB^{*} \to \BB^{4 \cdot \byteLen}$ the function that takes a function name as input and returns its function selector.
\end{notebox}

\subsubsection{Transaction validity}\label{preliminaries:ethereum:eth-tx:tx-validity}

Importantly, not all finalized transactions constitute valid state transitions on the state machine~\cite[Section 6]{ethyellowpaper}.
We denote by \ethVerifyTx~the function that takes an \ethereum~transaction object $\tx$ as input and return $\true$ (resp.~$\false$) if $\tx$ is valid (resp.~invalid). To be deemed valid, a transaction $\MUST$ satisfy \emph{all} the following conditions:
\begin{enumerate}
    \item The transaction is correctly \rlp~encoded, with no additional trailing bytes;
    \item the transaction signature $\smalltuple{\sigv, \sigr, \sigs}$ is valid;
    \item the transaction nonce ($\tx.\nonce$) is valid, i.e.~it is equal to the account nonce of the transaction originator;
    \item the gas limit is no smaller than the gas used by the transaction;
    \item the transactor has enough funds on his account balance to cover at least the cost $\tx.\val + \tx.\gasp \cdot \tx.\gasl$.
\end{enumerate}

\subsubsection{Lifecyle of a transaction, and miners' incentives}\label{preliminaries:ethereum:eth-tx:tx-life}

After the creation of an \ethereum~transaction \tx~by a user from an \ethereum~client (machine running a piece of software that enables to be connected to the \ethereum~network), the transaction is broadcasted to the network and received by a set of peers/nodes.

The transaction is then stored in each node's transaction pool, which is a data structure containing all transactions that should be validated (pending transactions) by the node and mined. To maximize miners' returns, the transaction pools are ordered according to the gas price of the transactions. As such, transactions with the highest $\tx.\gasp$ are subject to be validated and included into a block first.
Once \tx~is selected from the transaction pool, it is validated (fed into \ethVerifyTx), executed, and included into a block (i.e.~``mined''). The block is then broadcasted to all the nodes of the network and is used as the predecessor for the next block to be mined on the network (i.e.~``it is added to the chain'').

\subsection{Ethereum events and Bloom filters}\label{sssec:ethereum-events}

The \evm~contains the set of ``LOGX'' instructions enabling smart-contract functions to ``emit events'' (i.e.~log data) when they are executed\footnote{see~\url{https://ethgastable.info/}}

As such, when a block is generated by a miner or verified by the rest of the network, the address of any logging contract, and all the indexed fields from the logs generated by executing those transactions are added to a Bloom filter~\cite{DBLP:journals/cacm/Bloom70}, which is included in the block header~\cite[Section 4.3]{ethyellowpaper}. Importantly, the actual logs \emph{are not included in the block data} in order to save space.
    As such, when an application wants to find (``consume'') all the log entries from a given contract, or with specific indexed fields (or both), the node can quickly scan over the header of each block, checking the Bloom filter to see if it may contain relevant logs. If it does, \emph{the node re-executes the transactions from that block, regenerating the logs, and returning the relevant ones to the application}~\cite{eth-bloom-filters}.

\begin{notebox}
    The ability for a smart-contract function to ``emit'' some pieces of data when executed, and for an application to ``consume'' such pieces of data, is used in \zeth~in order to construct a \emph{confidential receiver-anonymous channel}~\cite{DBLP:conf/pet/KohlweissMOTV13}.
\end{notebox}


% ------------------------------------------------------------------------
% bjourdoc.tex for birkjour.cls*******************************************
% ------------------------------------------------------------------------
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\documentclass{birkjour}
%
%
% THEOREM Environments (Examples)-----------------------------------------
%
 \newtheorem{thm}{Theorem}[section]
% \newtheorem{cor}[thm]{Corollary}
% \newtheorem{lem}[thm]{Lemma}
% \newtheorem{prop}[thm]{Proposition}
% \theoremstyle{definition}
 \newtheorem{defn}[thm]{Definition}
% \theoremstyle{remark}
% \newtheorem{rem}[thm]{Remark}
% \newtheorem*{ex}{Example}
 \numberwithin{equation}{section}

\usepackage[noadjust]{cite}
\usepackage{amsfonts}
\usepackage{listings}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{booktabs}
\usepackage{float}
\usepackage{caption}

\begin{document}

%-------------------------------------------------------------------------
% editorial commands: to be inserted by the editorial office
%
%\firstpage{1} \volume{228} \Copyrightyear{2004} \DOI{003-0001}
%
%
%\seriesextra{Just an add-on}
%\seriesextraline{This is the Concrete Title of this Book\br H.E. R and S.T.C. W, Eds.}
%
% for journals:
%
%\firstpage{1}
%\issuenumber{1}
%\Volumeandyear{1 (2004)}
%\Copyrightyear{2004}
%\DOI{003-xxxx-y}
%\Signet
%\commby{inhouse}
%\submitted{March 14, 2003}
%\received{March 16, 2000}
%\revised{June 1, 2000}
%\accepted{July 22, 2000}
%
%
%
%---------------------------------------------------------------------------
%Insert here the title, affiliations and abstract:
%

\title[Robust Quaternion Estimation with Geometric Algebra]
 {Robust Quaternion Estimation with \\Geometric Algebra}

%----------Author 1
\author[Mauricio Cele Lopez Belon]{Mauricio Cele Lopez Belon}
\address{Madrid, Spain}
\email{mclopez@outlook.com}

%----------classification, keywords, date
\subjclass{Parallel algorithms 68W10; Clifford algebras, spinors 15A66}

\keywords{Geometric Algebra, Quaternion Estimation, Wahba Problem}

\date{October 21, 2019}
%----------additions
%\dedicatory{To my wife}
%%% ----------------------------------------------------------------------

\begin{abstract}

Robust methods for finding the best rotation aligning two sets of corresponding vectors are formulated in the linear algebra framework, using tools like the SVD for polar decomposition or QR for finding eigenvectors. Those are well established numerical algorithms which on the other hand are iterative and computationally expensive. Recently, closed form solutions has been proposed in the quaternion's framework, those methods are fast but they have singularities i.e., they completely fail on certain input data. In this paper we propose a robust attitude estimator based on a formulation of the problem in Geometric Algebra. We find the optimal eigen-quaternion in closed form with high accuracy and with competitive performance respect to the fastest methods reported in literature.

\end{abstract}

%%% ----------------------------------------------------------------------
\maketitle
%%% ----------------------------------------------------------------------
%\tableofcontents
\section{Introduction}

\indent The estimation of rotations has been studied for over half a century \cite{Wahba1965}. The problem consist on finding the optimal rotation aligning two sets of corresponding vectors. Many effective methods have been developed \cite{Arun1987, Horn1987, Mortari1996, Shuster1981, Yang2015} using $3\times3$ matrices and quaternions. Formulations based on quaternions solves a max-eigenvalue problem, while formulations based on linear algebra relies on Singular Value Decomposition (SVD). In the last decade formulations based on geometric algebra \cite{Perwass2009, Dorst2011} were introduced but they also rely on linear algebra numerical algorithms such as SVD due to the lack of native numerical algorithms. Closed form solutions for finding the optimal quaternion has been proposed \cite{Yang2013, Wu2017, Wu2018FA3R, Wu2018FS3R} based on analytic formulas for solving the roots of the quartic polynomial associated with eigenvalue problem.

Accuracy and speed of prominent methods have been compared in  \cite{Eggert1997, Markley1999, Wu2017} evidencing a trade-off between performance and robustness. In particular SVD based methods exhibit the best accuracy but low performance and quaternion based methods are faster but less accurate. Regarding the later methods, the closed form solutions exhibit the best performance so far but they have singularities i.e., they completely fail on certain input data.

In this paper we propose a robust estimator of the best quaternion aligning two sets of corresponding vectors. We maximize a convex quadratic energy functional formulated in the $\mathbb{G}_{3,0,0}$ geometric algebra which allow us to find an optimal quaternion in a robust way without resorting to linear algebra numerical algorithms. Geometric algebra rotors are isomorphic to quaternions, we find geometric algebra to be a more natural choice for studying this problem since rotations and subspaces of $\mathbb R^3$ are treated in the same manner, facilitating meaningful algebraic manipulations. We primarily work with bivectos instead of vectors for the sake of mathematical convenience. Due to mathematical (and geometric) duality of vectors and bivectors in $\mathbb{G}_3$ our formulation is also valid for vectors.

\section{Geometric Algebra $\mathbb{G}_3$}

A geometric algebra $\mathbb{G}_3$ is constructed over a real vector space $\mathbb R^3$, with basis vectors $\{e_1, e_2, e_3\}$. The associative geometric product is defined on vectors so that the square of any vector $a$ is a scalar $a a = a^2 \in \mathbb{R}$ and the geometric product of two vectors $a$ and $b$ is $a b = a \cdot b + a \wedge b$ and $b a = b \cdot a - a \wedge b$. From the vector space $\mathbb R^3$, the geometric product generates the geometric algebra $\mathbb{G}_3$ with elements $\{ X, R, A...\}$ called multivectors.

For a pair of vectors $a$ and $b$, a symmetric inner product $a \cdot b = b \cdot a$ and antisymmetric outer product $a \wedge b = -b \wedge a$ can be defined implicitly by the geometric product. It is easy to prove that $a \cdot b = \frac{1}{2}(a b + b a)$ is a scalar, while the quantity $a \wedge b = \frac{1}{2}(a b - b a)$, called a bivector or $2$-vector, is a new algebraic entity that can be visualized as the two-dimensional analogue of a direction i.e., a planar direction. Similar to vectors, bivectors can be decomposed in a bivector basis $\{ e_{12}, e_{13}, e_{23} \}$ where $e_{ij} = e_i \wedge e_j$.

The outer product of three vectors $a \wedge b \wedge c$ generates a $3$-vector also known as the pseudoscalar, because the trivector basis consist of single element $e_{123} = e_1 \wedge e_2 \wedge e_3$. Similarly, the scalars are regarded as $0$-vectors whose basis is the number $1$. It follows that the outer product of $k$-vectors is the completely antisymmetric part of their geometric product: $a_1 \wedge a_2 \wedge ... \wedge a_k = \langle a_1 a_2 ... a_k \rangle_k$ where the angle bracket means $k$-vector part, and $k$ is its grade. The term grade is used to refer to the number of vectors in any exterior product. This product vanishes if and only if the vectors are linearly dependent. Consequently, the maximal grade for nonzero $k$-vectors is $3$. It follows that every multivector $X$ can be expanded into its $k$-vector parts and the entire algebra can be decomposed into $k$-vector subspaces:
\begin{equation*}
\mathbb G_3 = \sum_{k=0}^n{\mathbb{G}^k_3} = \{ X = \sum_{k=0}^n { \langle X \rangle_k } \}
\end{equation*}
This is called a \emph{grading} of the algebra.

Reversing the order of multiplication is called reversion, as expressed by $(a_1 a_2 ... a_k)\tilde{} = a_k ... a_2 a_1$ and $(a_1 \wedge a_2 \wedge ... \wedge a_k)\tilde{} = a_k \wedge ... \wedge a_2 \wedge a_1$, and the reverse of an arbitrary multivector is defined by $\tilde{X} = \sum_{k=0}^n { \langle \tilde{X} \rangle_k }$.

\subsection{Rotors}

Rotations are even grade multivectors known as rotors. We denote the subalgebra of rotors as $\mathbb{G}^{+}_3$. A rotor $R$ can be generated as the geometric product of an even number of vectors. A reflection of any $k$-vector $X$ in a plane with normal $n$ is expressed as the sandwitch product $(-1)^k n X n$. The most basic rotor $R$ is defined as the product of two unit vectors $a$ and $b$ with angle of $\frac{\theta}{2}$. The rotation plane is the bivector $L = \frac{a \wedge b}{\| a \wedge b \|}$.
\begin{equation}
a b = a \cdot b + a \wedge b = \cos\left( \frac{\theta}{2} \right) + L \sin\left( \frac{\theta}{2} \right).
\end{equation}
Rotors act on all $k$-vectors using the sandwitch product $X' = R X \tilde R$, where $\tilde R$ is the reverse of $R$ and can be obtained by reversing the order of all the products of vectors.

\subsection{Bivector products}

We define the commutator product of two bivectors $p_j$ and $q_j$ as $p_j \times q_j = \frac{1}{2}(p_j q_j - q_j p _j)$. The commutator product of bivectors in $\mathbb{G}_3$  can be interpreted as a cross-product of bivectors i.e., the resulting bivector $B = p_j \times q_j$ is orthogonal to both $p_j$ and $q_j$. The commutator product allow us to define the geometric product of two bivectors as $A B = A \cdot B + A \times B$. The inner product of bivectors differs from the inner product of vectors on the sign, since the square of bivectors is negative, the inner product of bivectors is a negative scalar e.g., $(a e_{12} + b e_{13} + c e_{23}) \cdot (d e_{12} + e e_{13} + f e_{23}) = -a d - b e - c f$.

\subsection{Quaternions}

A quaternion $Q = w + \vec v$ consists of two parts: a scalar part $w$ and vector part $\vec v$ which denotes the axis of rotation. The vector part $\vec v$ is defined in a basis of complex vectors $\{ i, j, k \}$ that squares to $-1$ and anticommute i.e., $i^2 = j^2 = k^2 = -1$ and $i j k = -1$. In geometric algebra they corresponds to bivectors:
\begin{eqnarray}
i = e_{23} \ \ \ j = e_{13} \ \ \  k = e_{12} \\
ijk = e_{23} e_{13} e_{12} = -1 \nonumber
\end{eqnarray}
Rotors can easily be transformed to quaternions and vice versa. A rotor $R = w + L$ corresponds with a quaternion $Q = w + \vec v$, where $L = \alpha e_{12} + \beta e_{13} + \gamma e_{23}$ and $\vec v = \gamma i + \beta j + \alpha k$.

\section{Geometric Algebra Rotor Estimation}

Given two sets of $n$ corresponding bivectors $P = \{p_j\}_{j=1}^n$ and $Q = \{q_j\}_{j=1}^n$, we attempt to maximize the following energy function:
\begin{eqnarray}
	E(R) = \max_{R \in \mathbb{G}^{+}_3 } \sum_j { c_{j} \|p_j + \tilde R q_i R \|^2 }\\
	s.t. \ R \tilde R = 1 \nonumber
\end{eqnarray}
where $\{c_{j}\}_{j=1}^n$ are scalar weights such that $\sum_j^n{c_j} = 1$. It is a quadratic maximization problem with a non-linear constraint in $R$. Notice that the term $\|p_j + \tilde R q_i R\|^2$ is dual to the traditional least squares error $\|q_j - R p_j \tilde R\|^2$. Duality in the sense that the optimal $R$ is a critical point of both energies.
Notice also that $p_j + \tilde R q_i R$ is equivalent to $R p_j + q_j R$ by multiplying by $R$ on the left and using the fact that $R \tilde R = 1$. The equivalent problem is:
\begin{eqnarray}
	\label{eqn:max_energy}
	E(R) = \max_{R \in \mathbb{G}^{+}_3 } \sum_j { c_{j} \|R p_j + q_j R\|^2 }\\
	s.t. \ R \tilde R = 1 \nonumber
\end{eqnarray}
which exposes the that $R$ is only quadratic in \ref{eqn:max_energy}.

The constraint $R p_j + q_j R$ can be rewriten as $(w + L) p_j  + q_j (w + L)$. For some a scalar $w$ and bivector $L$. Expanding the geometric product of bivectors in terms of the inner product and the communtator product we get:
\begin{eqnarray}
	w  (p_j + q_j) + L \cdot (p_j + q_j)  + (q_j - p_j) \times L
\end{eqnarray}

In matrix language we can define the following matrix system $M_j R = 0$:
\begin{eqnarray}
	M_j R =
	\left[\begin{array}{cc}
		0      &       -s_j^T \\
		s_j    &   \left[ d_j \right]_\times \\
	\end{array}\right]
	\left[\begin{array}{c}
		w \\
		L
	\end{array}\right] =
	\left[\begin{array}{c}
		-s_j^T L \\
		w s_j + d_j \times L
	\end{array}\right]\\
	d_j = q_j - p_j \ \ s_j = p_j + q_j  \nonumber
\end{eqnarray}
where $d_j$ and $s_j$ are $3 \times 1$ column vectors holding bivector's coefficients, $M_j$ is a skew-symmetric $4\times 4$ real matrix, so that $M_j^T = -M_j$. The rotor $R$ is represented as $4 \times 1$ column vector made of the scalar $w$ and the $3 \times 1$ column vector $L$ holding the bivector's components. The $3\times 3$ matrix $\left[ d_j \right]_\times$ is representing the skew-symmetric cross-product matrix as usually defined for vectors in $\mathbb R^3$.

We can express $E(R)$ as the following quadratic form:
\begin{eqnarray}
E(R) = \max_R R^T M R\\
s.t. \ R^T R = 1  \nonumber
\end{eqnarray}
where $M = \sum_j^n { c_j M_j^T M_j}$. Note that since $M_j$ is skew-symmetric, the product $M_j^T M_j$ is symmetric and positive semi-definite.
Consequently the matrix $M$ is also symmetric positive semi-definite. It follows that all eigenvalues of $M$ are real and $\lambda_i \geq 0$.
\begin{eqnarray}
	M_j^T M_j =
	\left[\begin{array}{cc}
		\| s_j \|^2       &         (s_j \times d_j)^T \\
		s_j \times d_j  &    s_j s_j^T - \left[ d_j \right]^2_\times \\
	\end{array}\right]\\
	d_j = q_j - p_j \ \ s_j = p_j + q_j  \nonumber
\end{eqnarray}

By the spectral theorem the maximizer of $E(R)$ is the eigenvector of $M$ associated with the largest eigenvalue which is a positive number.

\section{Convexity}

The convexity of the energy $E(R)$ can be proof by showing that its Hessian matrix of second partial derivatives is positive semi-definite. The Hessian matrix of $E(R)$ is $\frac{\partial^2 E(R)}{\partial R} = \sum_j^n { c_j M_j^T M_j}$ which is symmetric, moreover since $M_j$ is skew-symmetric matrix, the product $M_j^T M_j$ is symmetric positive semi-definite. Then follows that $\sum_j^n { c_j M_j^T M_j}$ is positive semi-definite and therefore convex, provided that the sum of weights is convex i.e., $\sum_j^n { c_j } = 1$.

\section{Optimal Quaternion using 4D Geometric Algebra}

The most time consuming task of the estimation is to compute the eigenvector of $M$ associated with the greatest eigenvalue. In this section we show how to find the largest eigenvalue and its corresponding eigenvector i.e., the required quaternion, using the 4D Geometric Algebra $ \mathbb{G}_4$ which is robust, efficient and accurate.

Let us define four vectors $\textbf m_1$, $\textbf m_2$, $\textbf m_3$, $\textbf m_4$ corresponding to the columns of matrix $M$:
\begin{eqnarray}
	M = \sum_j^n { c_j M_j^T M_j} =
	\left[\begin{array}{cccc}
		\\
		\textbf m_1 & \textbf m_2 & \textbf m_3 & \textbf m_4 \\
		\\
	\end{array}\right]
\end{eqnarray}

The matrix system that we want to solve is $M R = \lambda R$ for $\lambda$ corresponding to the largest eigenvalue of $M$. We can write the system in its homogeneous form:
\begin{eqnarray}
	\label{eqn:eigensystem}
	\left[\begin{array}{cccc}
		\\
		\textbf m_1 - \lambda e_1 & \textbf m_2 - \lambda e_2 & \textbf m_3 - \lambda e_3 & \textbf m_4- \lambda e_4 \\
		\\
		\\
	\end{array}\right]
	\left[\begin{array}{c}
		w \\
		L_1\\
		L_2\\
		L_3
	\end{array}\right] =
	\left[\begin{array}{c}
		0 \\
		0 \\
		0 \\
		0
	\end{array}\right]
\end{eqnarray}

The matrix in equation~\ref{eqn:eigensystem}, called \emph{characteristic matrix}, is of rank $3$ and thus singular i.e., the vectors $(\textbf m_1 - \lambda e_1)$, $(\textbf m_2 - \lambda e_2)$, $(\textbf m_3 - \lambda e_3)$ and $()\textbf m_4- \lambda e_4)$ are linearly dependent. It follows that its outer product must be zero:
\begin{eqnarray}
\label{eqn:characteristic_outer_product}
(\textbf m_1 - \lambda e_1)\wedge(\textbf m_2 - \lambda e_2)\wedge(\textbf m_3 - \lambda e_3)\wedge(\textbf m_4- \lambda e_4) = 0
\end{eqnarray}

Which is called the \emph{charactetistic outer porduct} and is equivalent to the characteristic polynomial $P(\lambda) = det(M - \lambda I)$. A simple way to find the largest eigenvalue is using Newton-Raphson method $\lambda_{i+1} = \lambda_i - P(\lambda) / P'(\lambda)$. This method is indeed robust given that all eigenvalues are non-negative real numbers, it wasn't however for methods based on Davenport's matrix \cite{Davenport1968} which eigenvalues can be negative. We found that $Trace(M)$ is a robust guess for Newton-Raphson iteration because it is larger than the largest eigenvalue and is also close enough to converge in few iterations. For the sake of completeness we show the first derivative of \ref{eqn:characteristic_outer_product}:
\begin{eqnarray*}
P'(\lambda) =
- e_1\wedge(\textbf m_2 - \lambda e_2)\wedge(\textbf m_3 - \lambda e_3)\wedge(\textbf m_4- \lambda e_4)\\
- (\textbf m_1 - \lambda e_1)\wedge e_2\wedge(\textbf m_3 - \lambda e_3)\wedge(\textbf m_4- \lambda e_4)\\
- (\textbf m_1 - \lambda e_1)\wedge(\textbf m_2 - \lambda e_2)\wedge e_3\wedge(\textbf m_4- \lambda e_4)\\
- (\textbf m_1 - \lambda e_1)\wedge(\textbf m_2 - \lambda e_2)\wedge(\textbf m_3 - \lambda e_3)\wedge e_4
\end{eqnarray*}

Following \cite{DeKeninck2019} we solve the system~\ref{eqn:eigensystem} using outer products. For a linear system $A \textbf x = b$ the authors in \cite{DeKeninck2019} defines $N$ linear equations of the form $A^T_j \textbf x = b_j$ each of which corresponds to a dual hyper-plane of the form $\textbf a_j \equiv A_j - b_j e_0$ where the solution $\textbf x$ must lie on. The $e_0$ is an \emph{homogeneous} basis vector needed for enabling projective geometry, enlarging the base space to $N+1$, which interpretation is to be the offset of the hyper-plane. So solution of the linear system is the intersection of $N$ dual hyper-planes, which is given by its outer product.
\begin{eqnarray}
\alpha (\textbf x + e_0)^* = \textbf a_1 \wedge \textbf a_2 \wedge ... \wedge \textbf a_N
\end{eqnarray}

Where the symbol $^*$ is the dual operator of the $N+1$ space and $\alpha$ is a weight factor. After taking the dual of $\alpha (\textbf x + e_0)^*$ and divide by the coefficient of $e_0$ (which is $\alpha$), the solution $\textbf x$ can be read off the coefficients of the $1$-vector.

We know that the null space of $(M - \lambda I)$ is of rank one. Algebraically this means that one of its column vectors is redundant i.e., it can be written in term of the others. In linear algebra this means that the system has infinitely many solutions. The geometric interpretation is that all hyper-planes intersect in a line passing through the origin. Since all solutions lie on the same line and they only differ by a scaling term, a particular solution can be found by fixing the scale. Actually, it is enough to constrain the scale of a single hyper-plane. The homogeneous component $e_0$ can be interpreted as scale of solution (instead of hyper-plane's offset as in \cite{DeKeninck2019}) since it affects only that aspect of the solution. In linear algebra language it is equivalent to set one value at the right hand side of \ref{eqn:eigensystem}, however that system can't be solved in linear algebra because it requires to invert a singular matrix.

We define the dual hyper-planes passing through the origin as:
\begin{eqnarray}
   \textbf a_i \equiv \textbf m_i - \lambda e_i
\end{eqnarray}

Although in most cases it is enough to set the scale of a single hyper-plane go get a solution it is inconvenient to do so, as will be explained in Section~\ref{section:robustness}. It is more robust to set the scale of all hyper-planes to some $\gamma \neq 0$ which is a scalar value. Intersection can then be found by taking the outer product as:
\begin{eqnarray}
\label{eqn:hyperplanes_intersection}
\alpha (\textbf x + 1)^* = (\textbf a_1 + \gamma) \wedge  (\textbf a_2 + \gamma) \wedge  (\textbf a_3  + \gamma) \wedge (\textbf a_4 + \gamma)
\end{eqnarray}

Distributing the outer product and keeping the terms of grade-$3$ we get:
\begin{eqnarray}
\alpha (\textbf x + 1)^* =  \gamma (\textbf a_1 \wedge \textbf a_3 \wedge \textbf a_4
+ \textbf a_1 \wedge \textbf a_2 \wedge \textbf a_4
+ \textbf a_1 \wedge \textbf a_2 \wedge \textbf a_3
+ \textbf a_2 \wedge \textbf a_3 \wedge \textbf a_4)
\end{eqnarray}

Here the symbol $^*$ is the dual operator of the $4$D space (not $5$D as in \cite{DeKeninck2019} which allow us to be more efficient) and $\alpha$ is a weight factor. After taking the dual of $\alpha (\textbf x + 1)^*$ the eigenvector $\textbf x$ can be read off the coefficients of the $1$-vector. Notice that solution needs to be normalized.

\section{Optimal Computation of M}

The symmetric matrix $M_j^T M_j$ has a simple form:

\begin{eqnarray*}
	M_j^T M_j =
	\left[\begin{array}{cc}
		\| s_j \|^2        &         (s_j \times d_j)^T \\
		s_j \times d_j  &    s_j s_j^T - d_j d_j^T + \| d_j \|^2 I \\
	\end{array}\right]\\
	d_j = q_j - p_j \ \ s_j = p_j + q_j
\end{eqnarray*}

Writing it in terms of $p_j$ and $q_j$ we get:

\begin{eqnarray}
   \label{eqn:matrix_fast}
   	M_j^T M_j = 2
	\left[\begin{array}{cc}
		p_j^T q_j       &         (p_j \times q_j)^T \\
		p_j \times q_j  &    p_j q_j^T + q_j p_j^T - p_j^Tq_j I_{3\times3} \\
	\end{array}\right]
    + (\| p_j \|^2 + \| q_j \|^2) I_{4\times4}
\end{eqnarray}
All terms of \ref{eqn:matrix_fast} can be derived from the covariance matrix $B = p_j q_j^T$ plus the quantity $\| p_j \|^2 + \| q_j \|^2$. Since matrix $B$ is of $3\times3$ its computation is more efficient than the whole \ref{eqn:matrix_fast}. Details can be found in Section~\ref{section:algoritms}.

\section{Robustness and Singularities}
\label{section:robustness}

As stated before, setting the scale of a single hyperplane is enough to get a valid eigenvector in most cases. However, the choice of which hyper-plane to constraint is problematic. For instance,  assuming input vectors without noise, constraining the hyperplane corresponding to the $w$ component of the rotor won't work because when the angle of rotation is $\pi$ the $w = \cos(\pi/2) = 0$ and so its scale cannot be constrained. Similarly, constraining one of the hyper-planes corresponding to $L_1$, $L_2$ or $L_3$ won't work because when the angle of rotation is $0$ or $4 \pi$ the $\sin(2 \pi) = 0$ and so on. Then, it is complex task to avoid all problematic situations. By constraining all hyper-planes as in \ref{eqn:hyperplanes_intersection} our geometric algebra method does not suffer from any of those singularities.

Some of the above mentioned singularities are present in classic methods such as QUEST \cite{Shuster1981} and FOMA \cite{Markley1993} but also on methods derived from those, including recent descendants based on analytic formulas \cite{Yang2013, Wu2016, Wu2017, Wu2018FA3R, Wu2018FS3R}. All those methods are based on finding the eigenvector corresponding to largest eigenvalue of Davenport's matrix \cite{Davenport1968}. Since that is an indefinite matrix, some eigenvalues are positive and some negative, the Newton-Raphson can fail to find the max eigenvalue (which can be negative). So significant effort has been put on finding fast and robust analytic solutions to the quartic polynomial but no advances has been made on improving robustness on finding the associated eigen-vector besides doing \emph{sequential rotations} i.e., a $\pi$ angle rotation of input data accomplished by changing the signs of two columns of the covariance matrix of \cite{Shuster1981}. The QUEST method performs sequential rotations one axis at a time, until an acceptable reference coordinate system is found.

\section{Algorithms}
\label{section:algoritms}

The pseudo-code of proposed method is shown in Algorithm~\ref{alg:FastRotorEstimation}.

\begin{algorithm}
\begin{algorithmic}[1]
\REQUIRE {$P = \{p_{j}\}_{j=1}^n, Q = \{q_{j}\}_{j=1}^n, C = \{c_{j}\}_{j=1}^n$}
\STATE{$S = B = 0, \gamma = 1$}
\FOR{$j = 1$ \TO $n$}
\STATE{$S = S + c_j (p_j \cdot p_j + q_j \cdot q_j)$}
\STATE{$B = B + c_j p_j q_j^T$}
\ENDFOR
\STATE{$\textbf m_1 = (\frac{1}{2}S + Tr(B)) e_1 + (B_{12} - B_{21}) e_2 + (B_{20} - B_{02}) e_3 + (B_{01} - B_{10}) e_4$}
\STATE{$\textbf m_2 = (B_{12} - B_{21})e_1 + (2 B_{00} + \frac{1}{2}S - Tr(B)) e_2 + (B_{01} + B_{10}) e_3 + (B_{20} + B_{02}) e_4$}
\STATE{$\textbf m_3 = (B_{20} - B_{02})e_1 + (B_{01} + B_{10}) e_2 + (2 B_{11} + \frac{1}{2}S - Tr(B)) e_3 + (B_{12} + B_{21}) e_4$}
\STATE{$\textbf m_3 = (B_{01} - B_{10})e_1 + (B_{20} + B_{02} ) e_2 + (B_{12} + B_{21}) e_3 + (2 B_{22} + \frac{1}{2}S - Tr(B)) e_4$}
\COMMENT{Newton-Raphson}
\STATE{$\lambda_0 = 7 S - 3 Tr(B)$}
\REPEAT
\STATE{$\lambda_{i+1} = \lambda_i - P(\lambda_i) / P'(\lambda_i)$}
\UNTIL{$\|\lambda_{i+1} - \lambda_i\| < \epsilon$}
\STATE{$\textbf a_1 = \textbf m_1 - \lambda e_1$
}
\STATE{$\textbf a_2 = \textbf m_2 - \lambda e_2$
}
\STATE{$\textbf a_3 = \textbf m_3 - \lambda e_3$
}
\STATE{$\textbf a_4 = \textbf m_4 - \lambda e_4$
}
\STATE{$\textbf X = \gamma (\textbf a_1 \wedge \textbf a_3 \wedge \textbf a_4
	+ \textbf a_1 \wedge \textbf a_2 \wedge \textbf a_4
	+ \textbf a_1 \wedge \textbf a_2 \wedge \textbf a_3
	+ \textbf a_2 \wedge \textbf a_3 \wedge \textbf a_4)$}
\STATE{$R = normalize( \langle \textbf X^* \rangle_1 )$}
\RETURN { $R(0) + R(1) e_{12} + R(2) e_{13} + R(3) e_{23}$ }
\end{algorithmic}
\caption{Fast Rotor Estimation}\label{alg:FastRotorEstimation}
\end{algorithm}
An optimized C++ code using the Eigen library \cite{Eigen} and GAALOP  \cite{Gaalop} is publicly available on GitHub \cite{GARotorEstimator}

\section{Comparisons}

We selected several representative methods e.g. FLAE \cite{Wu2017}, SVD \cite{Arun1987}, QUEST \cite{Shuster1981}, FA3R \cite{Wu2018FA3R}, Symbolic \cite{Wu2018FS3R}, Q-Method \cite{Shuster1981} and FOMA \cite{Markley1993} for comparison. The Eigen library \cite{Eigen} was employed to implement all of them. The tests were ran on a MacBook Pro laptop with Intel Core i7 CPU running at 2,5 GHz. The Clang C++ compiler was used with -Ofast option enabled.

We first show that some methods fails on producing meaningful rotations when certain input data is provided. Table~\ref{tab:failure_cases} shows four cases which causes error. First two cases consist of having input vectors rotated $\pi/2$ and $\pi$ radians without noise. Other case is when the 3D vectors are projected to the $YZ$-plane and rotated  around the $X$-axis without noise. The last case is similar but projected to the $XZ$-plane. The root mean-squared error (RMSE) is used to describe the difference of accuracy.

 \begin{table} [H]
 \centering
 \resizebox{1.0\textwidth}{!}{
 \begin{tabular}{cccccc}
 \toprule
 {Algo}&{$\pi/2$}&{$\pi$}&{$YZ$Plane}&{$XZ$Plane}&{$XY$Plane}\\
 \midrule
 {FLAE}&{$4.41 \times 10^{-28}$}&{$696$}&{$392$}&{$1328$}&{$3.25 \times 10^{-28}$}\\
 {Symbolic}&{$1.78 \times 10^{-26}$}&{$1053$}&{$392$}&{$1321$}&{$4.70 \times 10^{-27}$}\\
 {FA3R}&{$4.00 \times 10^{-28}$}&{$5.41 \times 10^{-29}$}&{$2.25 \times 10^{-09}$}&{$2.76 \times 10^{-08}$}&{$4.98 \times 10^{-08}$}\\
 {Proposed}&{$7.32 \times 10^{-18}$}&{$1.79 \times 10^{-18}$}&{$4.05 \times 10^{-28}$}&{$3.43 \times 10^{-28}$}&{$2.22 \times 10^{-24}$}\\
 {Q-Method}&{$3.78 \times 10^{-28}$}&{$2.89 \times 10^{-29}$}&{$7.10 \times 10^{-28}$}&{$1.45 \times 10^{-28}$}&{$9.57 \times 10^{-29}$}\\
 {QUEST}&{$82.6$}&{$2.18 \times 10^{-29}$}&{$6.09 \times 10^{-26}$}&{$9.93 \times 10^{-28}$}&{$8.36 \times 10^{-29}$}\\
 {FOMA}&{$6.08 \times 10^{-28}$}&{$2.95 \times 10^{-28}$}&{$3.93 \times 10^{-26}$}&{$2.13 \times 10^{-27}$}&{$1.19 \times 10^{-28}$}\\
 {SVD}&{$4.41 \times 10^{-28}$}&{$7.51 \times 10^{-29}$}&{$3.35 \times 10^{-28}$}&{$9.83 \times 10^{-29}$}&{$1.09 \times 10^{-28}$}\\
\bottomrule
 \end{tabular}}
 \caption{RMSE of one million vector alignments of a thousand input vectors without noise. Some methods failed when 3D vectors were rotated $\pi/2$ and $\pi$  and when vectors are projected to the coordinate planes}
 \label{tab:failure_cases}
 \end{table}

We also compared the robustness of the methods when input data has extreme noise applied to the length of input vectors. Figure~\ref{fig:robustness} shows that some methods often fails to find meaningful rotations. We took SVD as reference. Notice that the amount of RMSE is almost the same on successes and on failures, evidencing that almost all methods builds, in essence, the Q-Method's matrix.

\begin{figure}[H]
\centering
\includegraphics[width=1.0\textwidth]{robustness.png}
\caption{Sum of RMSE of one million vector alignments of a thousand input vectors. We applied Gaussian random noise to angles and random noise to lengths}
\label{fig:robustness}
\end{figure}

Among the compared methods SVD, Q-Method and the ours are the most robust. Performance is shown in Figure~\ref{fig:time} compared to the other methods. Our experiments shows that when the number of input vectors is up to 1000 our method is faster than Q-Method. Beyond that point the trend shows that Q-Method is faster.

\begin{figure}[H]
	\centering
	\includegraphics[width=1.0\textwidth]{time.png}
	\caption{Performance comparison: number of vectors v.s. execution time ($\mu$s)}
	\label{fig:time}
\end{figure}

\section{Conclusion}

We presented a novel method for estimating the best quaternion aligning two sets of corresponding vectors and bivectors. Since we maximize a convex energy functional we only deal with non-negative eigenvalues, which makes Newton-Raphson iteration robust. We find the eigenvector corresponding to the largest eigenvalue intersecting hyper-planes in the language of geometric algebra. Results shows that our method is a good alternative in terms of robustness and accuracy. Its speed is competitive when the number of input vectors is relatively small.

\bibliographystyle{abbrv}
\bibliography{rotorestimation}

% ------------------------------------------------------------------------
\end{document}
% ------------------------------------------------------------------------