\acrshort{rankfromsets} is a classifier whose output is used to rank items for a
user $u$ based on each item's set of attributes $x_m \in I$ and learned user
embeddings $\theta_u$. Every item $x_m \in I$ is represented by a subset of
attributes $v \in V$ from a vocabulary of attributes $V$. Item $m$ is
represented by the set of attributes $x_m \subseteq V$.

The \acrshort{rankfromsets} model is a classifier parameterized by a neural
network $f$ that predicts whether user $u$ consumed item $m$, demarcated by the
binary indicator $\yum = 1$. The model is trained to discriminate items a user
consumed, $(x_m, \yum=1)$, from items a user is unlikely to consume,
$(x_k, \yuk=0)$. In implicit feedback data, only the former datapoints
$(x_m, \yum = 1)$ are observed. (We describe how to define negative samples
$(x_k, \yuk=0)$ later.)

The classifier outputs the probability of a user $u$ consuming an item $m$:
\begin{align}
p(\yum = 1 \mid x_m) = \sigma\left(f\left(\theta_u, \sum_{v\in x_m}
  \beta_v,~~h(x_m)\right)\right).
\label{eq:rankfromsets}
\end{align}
The learned user embedding $\theta_u$, item attribute embeddings $\beta_v$, and
item intercept function $h(x_m)$ are used to make the prediction of item
consumption. The item intercept function maps items to
intercepts based on the item attributes (we describe how to construct this
function later). The sigmoid function is denoted $\sigma$ and squashes the
output of the neural network to be between $0$ and $1$, yielding the probability
that user $u$ consumes item $m$.

Fitting the model amounts to learning the user embeddings, the item attribute
embeddings, parameters of the item intercept function, and any parameters of the
neural network $f$ such as weights and biases. We define the loss function in
\Cref{sec:fitting}.

Does \acrshort{rankfromsets} satisfy the modeling criteria laid out in
\Cref{sec:desiderata}? The model is order-invariant: the summation of the item
attribute embeddings $\beta_v$ and the construction of the item intercept
function (detailed below) guarantees that the model output is invariant to the
order of set elements. Second, the model provably improves recall, as described
in \Cref{sec:theory}. Third, the choice of neural network $f$ to parameterize
the model enables universal approximation of other order-invariant models.
Finally, the model satisfies parameter-sharing, as the item attribute embeddings
$\beta_v$ and the intercept function $h$ are shared across items and allow the
model to scale.

\paragraph{Theoretical results.} \acrshort{rankfromsets} is supported by
propositions in \Cref{sec:theory}. We derive a result related to the
recommendation performance of the model. A common evaluation metric for
recommendation models is recall. Recall can be viewed as a binary task, and we
prove that this immediately leads to the conclusion that a classifier such as
\acrshort{rankfromsets} can maximize this metric. The second result concerns
universal approximation. The structure of the neural network used to
parameterize \acrshort{rankfromsets} enables it to approximate other models such
as matrix factorization or permutation-marginalized recurrent neural networks.

\subsection{Cold start and collaborative filtering}

The cold start problem arises when new items or new users appear.
Recommendations of such items or for such users are only possible if there
exists metadata about items (such as sets of attributes) or metadata about users
(such as age). As \acrshort{rankfromsets} recommends items based on their
attributes, we focus on the cold start setting of new items, where there is item
metadata available.

\paragraph{\acrshort{rankfromsets} item attribute embeddings leaven the cold
  start problem.} If an item has not been consumed by any user, it is considered
a `cold start` item, as the recommendation model does not have any historical
information with which to rank the item. The model must use the item attributes
to inform the rankings of such cold start items. \acrshort{rankfromsets}
addresses the cold-start problem by leveraging item attribute embeddings, which
allow the ranking of new items through their sets of attributes. Consider the
recommendation of cold start items that contain common attributes, such as meals
with commmon foods (e.g. bananas, tomatoes, bread), or arXiv papers that use
standard terms (e.g. recommendation, deep, learning). Item attribute embeddings
for such attributes that are shared across items enable information to be shared
between items with zero consumption data and items that have been consumed and
have similar attributes.

As our focus is on the setting where users have a history of previously-consumed
items, we need to ensure that \acrshort{rankfromsets} supports collaborative
filtering. In other words, the model needs to make recommendations for a user by
filtering items based on the consumption patterns of other users.

\paragraph{Collaborative filtering via item intercepts.}
\acrshort{rankfromsets} shares information across users through the item
intercept function $h$. The item intercept function enables the ranking of items
for a user based on similar users' patterns of consumption. The item intercept
function is shared across users, allowing one user's consumption choices to
inform another's ranking. This is in contrast to the item attribute embeddings,
which depend on the attributes of every item (these content-based attributes
cannot capture patterns in usage independent of content). The parameterization
of the item intercept function $h$ depends on the size of the data. If the data
is small enough, $h$ can function as a lookup for unique intercepts for every
item. However, if the number of items is so large that unique item intercepts
lead to overfitting, we can define $h$ using additional information about every
item. For example, if the data consists of foods in meals, we can define a meal
intercept as the sum of food intercepts, yielding a scalable item intercept
function. These two choices of parameterization satisfy the desideratum of
order-invariance. The input to the item intercept function is a set of
attributes, so the above constructions (using item-level parameters or sums)
ensure every item intercept is independent of the order of the item's set of
attributes.

\subsection{Neural network architectures}
\label{sec:parameterizations}
There are several choices for parameterizing the neural network $f$ in the
\acrshort{rankfromsets} recommendation model. The following choices of neural
network architectures enable universal approximation of other order-invariant
models; we outline the technical requirements for this in \Cref{sec:theory}. In
these parameterizations, the neural network $f$ outputs logits that predict item
consumption; these are used to train the classifier. For recommendation, items
are ranked using the logits.

In the inner product parameterization, the classifier makes predictions using
the dot product of the user embedding $\theta_u$ and the sum of the item
attribute embedding sand item intercept function.  This architecture is equivalent to
a neural network with user-dependent weights $\theta_u$. The neural network
outputs logits that predict whether user $u$ consumed item $m$:
\begin{align}
  f\left(\theta_u, x_m\right) = \theta_u^\top\left(\sum_{v\in x_{m}}
  \beta_v + h(x_m)\right)
  \label{eq:inner-product}
\end{align}

As an alternative parameterization, $f$ can be a deep neural network:
\begin{align}
  f(\theta_u, x_m) = \phi\left(\theta_u, \sum_{v\in x_m}
  \beta_v, h(x_m)\right),
  \label{eq:neural-network}
\end{align}
where the deep network $\phi$ has weights and biases and takes as inputs the
user embedding, sum of item attribute embeddings, and item intercept. This
architecture contrasts the inner product parameterization: ex ante, it is
unclear whether a finite-depth, finite-size neural network can learn the inner
product function.

A third parameterization is a combination of the above, using an idea borrowed
from residual networks~\citep{DBLP:journals/corr/HeZRS15}. A neural network can
be used to learn the residual in the inner product model:
\begin{align}
  f\left(\theta_u, x_m\right) = \theta_u^\top\left(\sum_{v\in x_m}
  \beta_v + h(x_m)\right) + \phi(\theta_u, x_m).
  \label{eq:residual}
\end{align}

We discuss the generalization and universal approximation capabilities of these
parameterizations in the following section.

\subsection{Fitting the model}
\label{sec:fitting}
\acrshort{rankfromsets} is fit to data using a classification objective function
and a stochastic optimization algorithm.

The model can be represented using a Bernoulli distribution, with generative
process $\yum~\sim~\textrm{Bernoulli}\left(\yum; \sigma(f(u, x_m))\right)$. The
probability of a user consuming an item is given by \Cref{eq:rankfromsets}.

As \acrshort{rankfromsets} is a classifier, it requires both positively- and
negatively-labeled training data. But in implicit feedback data, there are no
negative labels. Users explicitly consume items $(x_m, \yum = 1)$, but do not
indicate dislike of items $(x_m, \yuk = 0)$. For training the
\acrshort{rankfromsets} classifier, we assign negative labels to items a user is
unlikely to consume. These items are called negative samples. Negative samples
are drawn uniformly at random from the collection of items $I$. (We justify this
choice of negative sampling distribution in the next section.)

Denote the parameters of the model by $\theta$, and let $K$ be the number of
negative samples. The model parameters are the user embeddings, item attribute
embeddings, parameters of the item intercept function, and any weights and
biases of the neural network in \Cref{eq:rankfromsets}. The maximum likelihood
objective function for a single datapoint $(x_m, \yum=1)$ is
\begin{align}
\begin{split}
  \cL(\theta, & (x_m, \yum), \{(x_k, \yuk)\}) =\\
  &\log p(\yum = 1 \mid x_m; \theta) + \sum_{k=1}^K \log p(\yuk = 0 \mid x_k;
  \theta),\\
  &\textrm{with}~x_k\sim\textrm{Uniform}(I).
  \label{eq:objective}
\end{split}
\end{align}

% \begin{align}
%   \cL_{\textrm{total}}\left(\theta, \{(x_m,
%         \yum)\}\right) = & \sum_u \sum_{m: \yum = 1}\cL(\theta, (x_m, \yum))
% \end{align}
%\input{algo_rankfromsets}

\paragraph{Optimization.} We use stochastic gradient ascent~\citep{Robbins:1951}
to maximize the objective. This algorithm subsamples datapoints
$(x_m, \yum = 1)$, and optimizes the objective with respect to the model
parameters by taking stochastic gradient steps in the parameter space. We use
minibatches of datapoints to improve convergence.

\section{theory}
\label{sec:theory}

Having exhibited several parameterizations of the \acrshort{rankfromsets}
recommendation model, we prove that these parameterizations satisfy the
desideratum of improved recall in \Cref{sec:desiderata}. By noting that recall
is a binary task, it follows that a classifier such as \acrshort{rankfromsets}
can maximize this metric (\Cref{prop:1}). We also prove that
\acrshort{rankfromsets} can approximate any order-invariant recommendation model
such as matrix factorization (\Cref{prop:2}).

Recall measures how accurately a recommendation model ranks items consumed by a
user. It is computed as follows. First, for a specific user $u$, use the model
to rank all items $x_m \in I$. Denote the top $M$ items returned by the model as
${x_1, x_2, \ldots, x_M}$. Recall is defined as the fraction of items the user
$u$ consumed in these top-ranked items:
\begin{align}
  \textrm{Recall@}M &= \frac{\lvert \{x_m : \yum = 1 \} \rvert }{M}
\label{eq:recall}
\end{align}

\subsection{Classifiers maximize recall}
\label{sec:optimal}

The central insight here is that recall can be viewed as classification: a
recommendation model does or does not recall an item consumed by a user. This
connection between the performance of a recommender and its parameterization as
a classifier forms the scaffolding for \acrshort{rankfromsets}.

Datapoints in a classifier are of the form $(x, y) \sim F$ where $F$ is the
empirical data-generating distribution. The generalization error of a binary
classifier $p(y \mid x)$ is the misclassification rate,
\begin{equation}
E = \sum_{x \in F} \left[ \hat y(x) \neq y \right] \, .
\end{equation}
\begin{proposition}
  A binary classifier is trained with positive examples from a user's history
  and negative samples drawn uniformly from the collection of items $I$. If such
  a classifier achieves zero generalization error, then it achieves maximum
  recall.
  \label{prop:1}
\end{proposition}
\begin{proof}
  A model with zero generalization error is a perfect classifier; it assigns
  greater probability to positively-labeled datapoints than to
  negatively-labeled datapoints. In other words, it ranks positive examples
  above negative examples. Recall is measured by the fraction of
  positively-labeled items in a ranking returned by the model. In a classifier
  that achieves zero generalization error, positively-labeled datapoints muts be
  ranked higher than other datapoints, maximizing recall.
  %Recall is maximized up to negative samples being
%  identical to positive examples (such examples will have the same rank).
\end{proof}

\Cref{prop:1} connects recommender systems to classification. This connection may
seem vacuous or tautological. But the message is subtle---in terms of recall, a
good classifier makes for a good recommender.

This theory also justifies the choice of negative samples for implicit feedback
data in the objective, \Cref{eq:objective}. For good recall performance, it is
sufficient to assign negative labels uniformly over the items $I$. Note that the
distribution of negative samples can be other than uniform, as long as every
item in the collection is assigned positive weight. In the infinite data limit,
any such distribution of negative samples will result in a classifier with
maximum recall.

\subsection{\acrshort{rankfromsets} can approximate any order-invariant model}

Having shown that a classifier such as \acrshort{rankfromsets} can yield good
performance as a recommendation model in terms of recall, we turn to the model's
flexibility. The desideratum of order-invariance for models that rank from sets
in \Cref{sec:desiderata} leads to a specific class of models. We show that
\acrshort{rankfromsets} can approximate any model in this class.

The criterion of order-invariance in \Cref{sec:desiderata} requires that a
recommendation model that takes as input a set of attributes be invariant to the
order the set elements are fed to the model. We formally define this class of
recommendation models as order-invariant models, and show that
\acrshort{rankfromsets} can approximate any model in this class.

\paragraph{Order-invariant models.} A set is unordered, and therefore invariant
to permutation. Recommendation models that rank items from their sets of
attributes must be invariant to the order that set elements are fed to the
model. For a model $f$ that is used to rank item $m$ for user $u$, order
invariance with respect to the attributes $v_i \in x_m$ is written
\begin{align}
  f(u, {v_1, v_2, v_L}) &= f(u, {v_{\pi(1)}, v_{\pi(2)}, \ldots
                                     v_{\pi(L)}})~\forall~\pi,
\label{eq:order-invariance}
\end{align}
where $\pi$ is a permutation of the indices of individual attributes.

We can verify that a recommendation model is in the class of order-invariant
models by checking that it satisfies \Cref{eq:order-invariance}. Examples of
order-invariant models are matrix factorization, recommendation models based on
word embeddings, and permutation-marginalized recurrent neural networks.
We show that a Bayesian matrix factorization model \citep{Gopalana} is
order-invariant in \Cref{sec:order-invariance}. These models are evaluated in
\Cref{sec:rfs-experiments}.

The class of order-invariant models is large. The next proposition allows us to
focus on comparing architectures for a single model rather than comparing
models. This simplifies design of recommendation models evaluated using recall.
\acrshort{rankfromsets} is parameterized by neural networks, and \Cref{prop:2}
says that the model can approximate any other model in the class of
order-invariant models. This result is derived from the framing of the
parameterizations of \acrshort{rankfromsets} in \Cref{sec:parameterizations} as
neural networks with learned item-dependent covariates and user-dependent
weights, and use of the fact that neural networks are universal approximators.

\begin{proposition}
  Assume the vocabulary of attributes (set elements) is countable,
  $\lvert V \rvert < \lvert \bbN_0 \rvert$. \acrshort{rankfromsets}, with
  parameters on the order of the size of the vocabulary can approximate any
  order-invariant recommendation model.
  \label{prop:2}
\end{proposition}
The proof follows directly from Theorem~2 in
\citet{DBLP:journals/corr/ZaheerKRPSS17} and we will not restate it here. (The
only change to the proof is the mapping from set elements to one-hot vectors,
$c \colon V \to \left\{0, 1\right\}^{\lvert V \rvert}$ to yield a unique
representation of every member of the powerset.)

\Cref{prop:2} supports using \acrshort{rankfromsets} in practice: not only does
the model improve recall (\Cref{prop:1}), but it can also approximate other
models, including matrix factorization. Matrix factorization and other
order-invariant models may not have theoretical guarantees of improved
evaluation metrics or universal approximation. Focusing on building efficient
architectures to parameterize \acrshort{rankfromsets} rather than designing
extensions of models may simplify the design of recommenders.

\subsection{Comparing parameterizations of \acrshort{rankfromsets} using
  generalization}
\label{sec:generalization}

Is there an optimal architecture for \acrshort{rankfromsets} between the inner
product, \Cref{eq:inner-product}, or residual parameterization in
\Cref{eq:residual}? It depends. In the regime of infinite data and infinite
parameters, the inner product and residual architectures are equivalent:
\Cref{prop:2} states that given infinite data, both are universal approximators.
In theory, they can approximate any order-invariant model such as matrix
factorization. But in the finite data, finite parameter regime, there are
tradeoffs that may favor one architecture over another. Depending on assumptions
about the data-generating distribution and parameter-sharing choices, one
architecture may be more efficient than another.

With finite data and finite paramaters, the optimal parameterization of
\acrshort{rankfromsets} is dependent on the data-generating distribution. Recall
that observations of user-item interactions are generated by
$ \yum~\sim~\textrm{Bernoulli}\left(\yum; \sigma(f(u, x_m)\right)$, where $f$ is
a function that outputs logits. We describe how different assumptions about the
logit function $f$ lead to the residual architecture or inner product
architecture yielding the best predictive performance. Note that the number of
parameters across architectures must be equal for a fair comparison.

To illustrate a scenario where the inner product architecture outperforms the
residual architecture, suppose the logit function of the data-generating
distribution has the following form. With two-dimensional embeddings (and a
single attribute per item) say the logit function is given by
$f(u, x_m) = \theta_1\beta_1 + \theta_2\beta_2^{10}$. The second dimension of
the item attribute embedding $\beta$ has been raised to the power of $10$.
Consider fitting the inner product model with a dimensionality of $2$ to data
from this generative process. The optimal setting of the item attribute
embeddings is $\beta^* = (\beta_1, \beta_2^{10})$. Now consider fitting the
residual model. The requirement that the number of parameters be equal across
architectures forces the neural network to have less parameters. This limits the
ability of the residual architecture to learn that the second component of the
item attribute embedding should be $\beta_2^{10}$, as neural networks with small
hidden layer size may not be able to approximate complex polynomials.

For an example where the residual model would outperform the inner product
model, consider the following generative process. The logit function is given by
$f(u, x_m) = \theta_u^\top \sum_{v\in \xum}\beta_v + \eta(\theta_u, \sum_{v\in
  \xum} \beta_v)$. Here $\eta$ is some nonlinear function. This generative
process mirrors \Cref{eq:residual} and is best approximated by the residual
model. We verified both of these claims with simulation studies.

% How can we ensure our choice of model does not lead to overfitting and enables
% prediction of held-out data? In recommendation data, one way to define
% generalization is to consider the matrix of observations of shape users by items
% $(U, I)$. As $U\rightarrow \infty$ and $I\rightarrow \infty$, it is clear that a
% model can memorize the training data if the number of parameters is
% $\mathcal{O}(UI)$. As long as the number of model parameters grows slower than
% $U$ or $I$, and the model predicts well, generalization is possible.

The above two examples show that the choice of architecture in
\acrshort{rankfromsets} is data-dependent. To ensure that the model does not
overfit as new users or items are included in the training data, we need to
compare the number of parameters to the number of datapoints. A model with
parameters the size of the training data can overfit by memorizing the training
data. For generalization to be possible, overfitting can be avoided if the
number of parameters grows slower than the size of the data. The technical
backing for this comes from asymptotic statistics and the concept of sieved
likelihoods~\citet{vaart_1998}. Specifically, the maximum likelihood estimation procedure with the
objective function in \Cref{eq:objective} can be replaced by maximization of a
sieved likelihood function. The `sieve' refers to filtering information as the
number of parameters (in this case, user and item representations) grows with
the number of observations. The sieved likelihood function enables the analysis
of asymptotic behavior as the number of users grows $U\rightarrow \infty$ and
the number of items grows $I\rightarrow \infty$, An example of a technique to
grow the number of parameters in a way that supports generalization is given in
Chapter 25 of \citet{vaart_1998}.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "set_recommendation"
%%% End:


\section{Runtime Example}

\toclesssubsection{Minsort}

%-------------------------------------------------------------------------------

\begin{frame}{Runtime analysis - Minsort}
  \vspace{-1em}
  \begin{figure}[!h]
    \includegraphics[width=0.75\linewidth]{Images/MinSort/Minsort.png}
    \label{fig:introduction:minsort_runtime}
  \end{figure}
  \vspace{-0.5em}
  \textbf{How long does the program run?}
  \begin{itemize}
    \item<2- |handout:1>
      In the last lecture we had a schematic
    \item<2- |handout:1>
      \textbf{Observation:} it is going to be \enquote{disproportionately} slower
      the more numbers are being sorted
    \item<3- |handout:1>
      How can we say more precisely what is happening?
  \end{itemize}
\end{frame}

%-------------------------------------------------------------------------------

\begin{frame}{Runtime analysis - Minsort}
  \textbf{How can we analyze the runtime?}
  \begin{itemize}
    \item<1- |handout:1>
      Ideally we have a formula which provides the runtime of the program for
      a specific input
    \item<2- |handout:1>
      \textbf{Problem:}
      the runtime is depends on many variables, especially:
      \begin{itemize}
        \item
          What kind of computer the code is executed on
        \item What is running in the background
        \item Which compiler is used to compile the code
      \end{itemize}
    \item<3- |handout:1>
      \textbf{Abstraction 1:}
      analyze the number of basic operations, rather than analyzing the runtime
  \end{itemize}
\end{frame}


\chapter{Full Experimental Results}

\section{One Argument Programs}

\subsection{Factorial}
\underline{\textbf{Input Examples}}
\begin{lstlisting}
example(call(fac, 0), 1).
example(call(fac, 1), 1).
example(call(fac, 2), 2).
example(call(fac, 3), 6).
\end{lstlisting}

\begin{multicols*}{2}

\underline{\textbf{Interpreted approach results}}
\begin{lstlisting}
choose_match(1,1,0)
choose_match(2,2)
choose(1,3,1)
choose(2,9)
choose_where(34)
choose_where(45,1)

Models      : 1
Optimization: 2 3
Time        : 1.640
  Prepare   : 0.060
  Prepro.   : 0.020
  Solving   : 1.560

\end{lstlisting}

\begin{lstlisting}
fac x
  | x == 0 = 1
  | otherwise = x * x0
  where x0 = f x1
  	where x1 = x - 1
\end{lstlisting}
\vspace*{\fill}
\columnbreak
\underline{\textbf{Constraint approach results}}

\begin{lstlisting}
choose_match(1,1,0)
choose_match(2,2)
choose(1,1,1)
choose(2,37,1)

Models      : 1
Optimization: 1
Time        : 1.240
  Prepare   : 0.200
  Prepro.   : 0.140
  Solving   : 0.900
\end{lstlisting}

\begin{lstlisting}
fac x
  | x == 0 = 1
  | otherwise = (fac (x - 1)) * x
\end{lstlisting}
\end{multicols*}

\subsection{Fibonacci}
\underline{\textbf{Input Examples}}
\begin{lstlisting}
example(call(f, 1), 1).
example(call(f, 2), 1).
example(call(f, 3), 2).
example(call(f, 4), 3).
example(call(f, 5), 5).
example(call(f, 6), 8).
\end{lstlisting}

\begin{multicols*}{2}
\underline{\textbf{Interpreted approach results}}
\begin{lstlisting}
choose_match(1,1,1)
choose_match(2,1,2)
choose_match(3,2)
choose(1,1)
choose(2,13,1)
choose(3,84)
choose_where(30,1)
choose_where(49)
choose_where(83)
choose_where(75,2)

Models      : 1
Optimization: 4 1
Time        : 807.240
  Prepare   : 0.060
  Prepro.   : 0.020
  Solving   : 807.160

fib x
  | x == 1 = x
  | x == 2 = x - 1
  | otherwise = x0 + x2
  where x0 = fib x1
  	where x1 = x - 1
  		where x2 = fib x3
  			where x3 = x - 2
\end{lstlisting}
\vspace*{\fill}
\columnbreak
\underline{\textbf{Constraint approach results}}
\begin{lstlisting}
UNSATISFIABLE

Models      : 0
Time        : 0.020
  Prepare   : 0.020
  Prepro.   : 0.000
  Solving   : 0.000
\end{lstlisting}
\end{multicols*}

\pagebreak
\subsection{Powers of 2}
\underline{\textbf{Input Examples}}
\begin{lstlisting}
example(call(f, 0), 1).
example(call(f, 1), 2).
example(call(f, 2), 4).
example(call(f, 3), 8).
\end{lstlisting}

\begin{multicols*}{2}
\underline{\textbf{Interpreted approach results}}
\begin{lstlisting}
choose_match(1,1,0)
choose_match(2,2)
choose(1,3,1)
choose(2,84)
choose_where(30,1)
choose_where(49)
choose_where(64)

Models      : 1
Optimization: 3 3
Time        : 35.840
  Prepare   : 0.040
  Prepro.   : 0.040
  Solving   : 35.760

power2 x
  | x == 0 = x + 1
  | otherwise = x1 + x2
  	where x1 = f x0
  		where x2 = f x0
  			where x0 = x - 1
\end{lstlisting}
\vspace*{\fill}
\columnbreak
\underline{\textbf{Constraint approach results}}
\begin{lstlisting}
choose_match(1,1,0)
choose_match(2,2)
choose(1,1,1)
choose(2,38,1,2)

Models      : 1
Optimization: 1
Time        : 5.580
  Prepare   : 0.380
  Prepro.   : 0.340
  Solving   : 4.860

power2 x
  | x == 1 = 1
  | otherwise = (f (x - 1)) * 2
\end{lstlisting}
\end{multicols*}
\pagebreak
\section{Two Argument Programs}

\subsection{Tail Recursive Factorial}
\underline{\textbf{Input Examples}}
\begin{lstlisting}
example(call(f, (0, 1)), 1).
example(call(f, (1, 1)), 1).
example(call(f, (2, 1)), 2).
example(call(f, (3, 1)), 6).
example(call(f, (2, 3)), 6).
\end{lstlisting}

\begin{multicols*}{2}
\underline{\textbf{Interpreted approach results}}
\begin{lstlisting}
choose_match(1,1,0)
choose_match(2,7)
choose(1,2)
choose(2,27)
choose_where(51,1)
choose_where(69)

Models      : 1
Optimization: 2 2
Time        : 419.860
  Prepare   : 2.060
  Prepro.   : 0.840
  Solving   : 416.960
\end{lstlisting}

\begin{lstlisting}
fac x y
  | x == 0 = y
  | otherwise = fac x0 x1
  where x0 = x - 1
  	where x1 = x * y
\end{lstlisting}
\vspace*{\fill}
\columnbreak
\underline{\textbf{Constraint approach results}}
\begin{lstlisting}
choose_match(1,2,0)
choose_match(2,1)
choose(1,3)
choose(2,184,1)

Models      : 1
Optimization: 3
Time        : 173.900
  Prepare   : 5.540
  Prepro.   : 1.440
  Solving   : 166.920

\end{lstlisting}
\begin{lstlisting}
fac x y
  | x == 0 = y
  | otherwise = fac (x - 1) (x * y)
\end{lstlisting}
\end{multicols*}
\pagebreak
\subsection{Greatest Common Divisor}
\underline{\textbf{Input Examples}}
\begin{lstlisting}
example(call(gcd, (1, 1)), 1).
example(call(gcd, (2, 1)), 1).
example(call(gcd, (4, 3)), 1).
example(call(gcd, (3, 6)), 3).
example(call(gcd, (9, 6)), 3).
example(call(gcd, (4, 7)), 1).
example(call(gcd, (9, 3)), 3).
\end{lstlisting}
\begin{multicols*}{2}
\underline{\textbf{Interpreted approach results}}
\begin{lstlisting}
choose_match(1,2,0)
choose_match(2,6)
choose_match(3,7)
choose(1,1)
choose(2,30)
choose(3,81)
choose_where(53)
choose_where(77)

Models      : 1
Optimization: 1 2
Time        : 1546.820
  Prepare   : 1.940
  Prepro.   : 0.980
  Solving   : 1543.900

gcd x y
  | y == 0 = x
  | x > y = gcd y x0
  | otherwise = gcd x1 x
  where x0 = x - y
  	where x1 = y - x
\end{lstlisting}
\vspace*{\fill}
\columnbreak
\underline{\textbf{Constraint approach results}}
\begin{lstlisting}
choose_match(1,3)
choose_match(2,4)
choose_match(3,6)
choose(1,2)
choose(2,150, 0)
choose(3,159, 1, 1)

Models      : 1
Optimization: 2
Time        : 1599.420
  Prepare   : 11.720
  Prepro.   : 2.100
  Solving   : 1585.600

gcd x y
	| x == y = x
	| x > y	 = gcd (x - y) y
	| x < y	 = gcd y x
\end{lstlisting}
\end{multicols*}
\pagebreak

\pagebreak
%\renewcommand\bibname{{References}}
%\bibliography{References}
%\bibliographystyle{plain}


\section{Avalanche involving a dry area}

An avalanche problem involving a dry area is solved using shallow water approach. This problem is very similar to the dry dam break, but it is on a sloping topography. The debris could be snow, sand, or even rock. The simulation should show a rarefaction and wetting process, just like the dry dam break problem. The analytical solution of this problem was derived by Mungkasi and Roberts~\cite{MR2011DA}. This shallow water approach to solve debris avalanche problems was also implemented by a number of researchers, such as Mangeney et al.~\cite{MHR2000} and Naaim et al.~\cite{NVC1997}.

The initial condition is
\begin{equation} \label{eq:dap_init}
u(x,0)=0, ~~v(x,y)=0, ~~\textrm{and}~~
h(x,0) = \left\{ \begin{array}{ll}
h_1 & \textrm{if $x < 0$}\\
0 & \textrm{if $x > 0$}\\
\end{array} \right.
\end{equation}
where $h_1>0$. The topography is a flat bed with positive slope.

The analytical solution~\cite{MR2011DA} at time $t>0$ is
\begin{equation}
h(x) = \left\{ \begin{array}{ll}
0 & \textrm{if $x \leq -2 c_0 t + \frac12 mt^2$}\\
h_R=\frac{1}{9g} \left( \frac{x}{t} + 2c_0 - \frac12 mt \right)^2 & \textrm{if $-2 c_0 t + \frac12 mt^2 \leq x \leq c_0 t + \frac12 mt^2$}\\
h_0 & \textrm{if $x \geq c_0 t + \frac12 mt^2$}\\
\end{array} \right.
\end{equation}
which is the free surface and
\begin{equation}
u(x) = \left\{ \begin{array}{ll}
0 & \textrm{if $x \leq -2 c_0 t + \frac12 mt^2$}\\
u_R=\frac23 \left( \frac{x}{t} - c_0 + mt \right) & \textrm{if $-2 c_0 t + \frac12 mt^2 \leq x \leq c_0 t + \frac12 mt^2$}\\
mt & \textrm{if $x \geq c_0 t + \frac12 mt^2$}\\
\end{array} \right.
\end{equation}
which is the velocity. Here $m=-g\tan{\theta}+F$, where $\tan{\theta}$ is the slope of the topography. Variable $F$ is the Coulomb-type friction given by
\begin{equation}
F=g \cos^2{\theta} \tan{\delta},
\end{equation}
in which $\tan{\delta}$ is a given value of friction slope such that $\tan{\delta} \leq \tan{\theta}$.

\subsection{Results}

For our test, we consider $h_0=20$ in (\ref{eq:dap_init}).
The following figures show the stage, $x$-momentum, and $x$-velocity at several instants of time. We should see excellent agreement between the analytical and numerical solutions. The wet/dry interface is difficult to resolve and it usually produces large errors, similar to the dry dam break problem.
\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{stage_plot.png}
\end{center}
\caption{Stage results}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{xmom_plot.png}
\end{center}
\caption{Xmomentum results}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{xvel_plot.png}
\end{center}
\caption{Xvelocity results}
\end{figure}

\endinput


We implement all the graph mining algorithms using Postgres's embedded PL/pgSQL programming language, which supports many advanced features, like user defined function, aggregate, etc. It also has a sophisticated query execution engine, which we think is the most critical component of Postgres.

The following are the algorithms we plan to implement. The reason we choose them is that there are a lot of implementation in other platform like MapReduce, we can compare our SQL version with them to draw a clusion about SQL's unique charastic in solving data analytic tasks.

\begin{description}
  \item[Degree Distribution:] Plot the distribution of each node's degree.
  \item[PageRank:] Determine the importance of every node of a graph based on its connectivity.
  \item[Connected Components:] Partition the nodes of a graph also based on its connectivity.
  \item[Radius of Every Node:] Compute the radius of every node in a graph. The radius is defined as the number of hops that a node needs to reach to its furthest neighbor.
  \item[Belief Propagation:] Calculate the marginal probability of each node in a graphical model.
  \item[Eigenvalue:] Using approximation method to estimate the top-k eigenvalues of a matrix.
  \item[Count of triangle] Using approcimate method to calculate the number of triangles in a social network.
  \item[Shortest Path] Calculate the shortest path from each node in a graph to a single source node.
  \item[Minimum spanning tree] Construct a tree which is a subgraph of original graph with the minimum sum of weight of its edges.
\end{description}

\subsection{Degree Distribution}
We employ Postgres's group by command according to either source node or target node to count the degree distribution according to each node. The pseudo code is in Algorithm \ref{algo1:1}, \ref{algo1:2}.

\begin{algorithm}[!htbf]
\caption{Out Degree distribution}
\begin{algorithmic}
\STATE{Group edges according to source node's id}
\STATE{Count the number of members of each group}
\end{algorithmic}
\label{algo1:1}
\end{algorithm}

\begin{algorithm}[!htbf]
\caption{In Degree distribution}
\begin{algorithmic}
\STATE{Group edges according to target node's id}
\STATE{Count the number of members of each group}
\end{algorithmic}
\label{algo1:2}
\end{algorithm}

\subsubsection{Math}
First we need to count degree of each node. Then we count the frequency of each degree count.

\subsubsection{Idea of SQL implementation}
The only operation we need from SQL is its group by clause. We can aggregate the edges according with source node or target node.

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Pagerank}
The algorithm of pagerank is Power method. We do matrix multiplication continuously until the change in pagerank is small. The most important equation for calculating pagerank is as follows:
\begin{equation}
  PR(i) = \alpha \frac{1}{N} + (1 - \alpha) \sum_{j \in InNeighbor(i)} \frac{PR(j)}{OutDegree(j)}
\end{equation}
\begin{algorithm}
\caption{Pagerank}
\begin{algorithmic}
\STATE{Bulk load graph into an edge table in database.}
\STATE{Initialization(damper factor=0.85, max iteration = 100, epsilon = 0.0001)}
\STATE{Build a weight matrix trans, initialize pagerank p}
\REPEAT
\STATE{For each node i, update its pagerank with its old value and its income node's pagerank.}
\UNTIL{Convergence}
\end{algorithmic}
\end{algorithm}

\subsubsection{Math}
According to the definition of pagerank, we can gain an intuitive idea of how to calculate the pagerank of a node.
We just take weighted sum of its incoming neighbors. We can encode this operation as matrix vector multiplication.
We can do this multiple times. And the fix point of the equation is the pagerank of the graph. And by the results
from linear algebra, we know that the stable vector is the eigenvector with biggest eigenvalue. Thus we can get
the eigenvector by power method, which just do multiplication until convergence.

\subsubsection{Idea of SQL implementation}
The implementation consists of several components.
First, we have a \emph{graph} which has the schema \emph{(from\_id, to\_id, weight)}. Then we will build a \emph{rank} table has the schema \emph{(node\_id, rank)}, all the rank are initialized randomly. After we enter loop, we will do a large matrix vector multiplication, which is implemented by SQL \emph{select} and \emph{join}. The new pagerank is stored in a temporary table. After the loop is over, we calclate the updates to every node, if the change is little, then abort.

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Weakly connected components}
In terms of the implementation of weakly connected components. We borrow the idea of HCC method from the "PEGASUS" paper.\cite{Kang09}

\subsubsection{method}
The key idea of this algorithm is that for every node $v_i$ in the graph, we maintain a component id $c_i^h$ which is the minimum node id within $h$ hops from $v_i$. Initially, $c_i^h$
of $v_i$ is set to its own node id. For each iteration, each node sends its current component id to its neighbors. Then $c_i^{h+1}$ is set to the minimum value among its current component id and the received component ids from its neighbors. Finally, when the update converges, all nodes in the same connected component will share the same component id.

\subsubsection{Idea of SQL implementation}
The algorithm can be described in algorithm \ref{algo:wcc}. The key step that updates the
component id to the minimum of its neighbors' is accomplished in SQL using join and group by
clause.
\begin{algorithm}
\caption{Weakly Connected Component}
\begin{algorithmic}
\STATE{Bulk load graph into an edge table in database.}
\STATE{Create a component table, where each entry contains a node id, and the component id.}
\STATE{Initialize the component table where component id equals node id.}
\REPEAT
\STATE{For each node, assign the minimum component id of its neighbors as the new component id of this node.}
\UNTIL{Convergence}
\end{algorithmic}
\label{algo:wcc}
\end{algorithm}
After several rounds of iteration, the nodes in the same connected component will share the same component id.
The number of iterations for convergence can be proved to be upper bounded by the diameter of the graph.

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Radius of every node}
We discard the traditional algorithm because it is extremely infeasible for large graphs, since it uses a set to record every neghbors within n hops for a node during iteration, which requires a O($n^2$) space.
\subsubsection{method}
Since the exact algorithm is hopeless, we use the approximation algorithm described in "HADI" paper\cite{DBLP:journals/tkdd/KangTAFL11} instead. Specifically, we use the Flajolet-Martin algorithm for counting the number of distinct members in a multiset. It is guaranteed to give an unbiased estimate and a tight O($log(N)$) bound for space complexity. The basic idea of Floajolet-Martin algorthm is to use a bitstrings of length $L$ to encode the set. For each element to add, we randomly pick up a index according to a specified distribution, and assign BITMAP[index] to 1. Following this procedure to add element, the size of the final set can be estimated by $\frac{1}{\phi} 2^{\frac{1}{k}\sum_{i=1}^k R_i}$, where $\phi$ = 0.77351, $R_i$ denotes the index of the leftmost 0 in the the $k$th bitstring.

In our proposed method for computing radius of every node, we use the Flajolet-Martin(FM) bitstrings to encode the neighbors of every node. Formally, we use $k$ FM-bitstrings $b(h, i)$ to represent the set of neighbor nodes reachable from $node_i$ within h hops. And for each iteration, we use the following way to update each FM-bitstring:$$b(h,i) = b(h-1,i)  \quad BIT-OR \quad  {b(h-1,j)|(i,j)\in E}$$

Given the above description of how to encode neighbors of a node, and the method to update bitstrings, we can describe the approximation method we used to compute the raidus of every node in algorithm \ref{radius:algo3}.

\begin{algorithm}
\caption{Radius of Every Node}
\begin{algorithmic}
\STATE{Bulk load graph into an edge table in database.}
\STATE{Preprocess the edge table, add a self loop edge to every node in the graph.}
\STATE{Initialize the vertex table, which contains node id, and a column of bitstring array, the bitstrings are initialized using the FM algorithm}
\REPEAT
\STATE{For every node update the bitstring according to formula: $b(h,i) = b(h-1,i) \quad BIT-OR \quad  {b(h-1,j)|(i,j)\in E}$. }
\STATE{For every node, check whether the bitstrings is unchanged before and after updates, if it's not changed, output i as the radius for this node.}
\UNTIL{The bitstrings of every node stabilizes or it reaches the maximum rounds of iteration.}
\end{algorithmic}
\label{radius:algo3}
\end{algorithm}

\subsubsection{Idea of SQL implementation}
In order to store the fm-string array in the table, we use the array type which is supported by PostgreSQL. To initialize the fm string and update the fm string array, we defined some user defined functions in PostgreSQL. The key step of algorithm that updates the FM-bitstring array is accomplished by using aggBitOr in the join and group by clause. Specifically, we join the edge table and vertex table on dst id, group by src id, and then call the aggBitOr to update the fm string array for all nodes. We summarize the user defined functions in Table \ref{table:radius}.

\begin{table}[[!htbf]
\caption{User defined functions for task 4}
\begin{center}
\begin{tabular}{|c|c|}
\hline \hline
function name & description \\
\hline
fmAssign & Assign the k FM-bitstrings for a node \\
bit-or & Execute the OR operations between two bitstring arrays \\
aggBitOr & Aggregate function for bit-or \\
fmSize & Estimate the size of a set encoded by FM-bitstring \\
\hline

\end{tabular}
\end{center}
\label{table:radius}
\end{table}%

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Eigenvalue}
We adopt the method propose in \cite{kang2011spectral}. There are several methods to solve part of eigenvalue computation problems, for instance, power method\cite{langville2004deeper}. While it has the limitation that it can only extract the eigenvector with biggest eigenvalue. Several method have been proposed to extract top k eigenvectors simultaneously. The approach we use is Lanczos algorithm\cite{lanczos1950iteration}. The general idea about this algorithm is that instead of directly work on an $N \times N$ matrix, we first generate a skinny $N \times m$ matrix(M $\ll$ N). Then it computes a small $M \times M$ dense matrix which has good approximation to the eigenvalues of the original matrix. In this case, we directly apply quadratic algorithm to top-k eigenvalues. Notice that k $<$ M.

\begin{algorithm}
{\bf Input:} Matrix $A^{n \times m}$\\
random n-vector $b$,\\
number of steps m\\
{\bf output:} Orthogonal matrix $ V^{v \times m}_{m} = [v_{1}\cdots v{m}]$,\\
coefficients $\alpha[1..m]$ and $\beta[1..m-1]$
\begin{algorithmic}[1]
\caption{Lanczos algorithm}
\STATE $\beta_{0} \leftarrow 0, v_{0} \leftarrow 0, v_{1} \leftarrow \frac{b}{\parallel b \parallel}$
\FOR {$i=1$ to $m$}
	\STATE $v \leftarrow Av_{i}$
	\STATE $\alpha_{i} \leftarrow v^{T}_{i}v $
	\STATE $v \leftarrow v - \beta_{i-1}v_{i-1} - \alpha_{i}v_{i}$
	\STATE $\beta_{i} \leftarrow \parallel v\parallel $
	\IF {$\beta_{i} = 0$}
	\STATE break for loop
	\ENDIF
	\STATE $ v_{v+1} \leftarrow \frac{v}{\beta_{i}} $
\ENDFOR
\end{algorithmic}
\label{eigen:algo1}
\end{algorithm}

\begin{algorithm}
\caption{Build tridiagonal matrix}
{\bf Input:} $\alpha, \beta$
{\bf Output:} $T^{m\times m}_{m}$
\begin{algorithmic}[1]
\FOR {$i=1$ to m}
	\STATE $T[i, i] \leftarrow \alpha_{i} $
	\STATE $T[i, i+1] = T[i+1, i] \leftarrow \beta_{i}$
\ENDFOR
\end{algorithmic}
\label{eigen:algo2}
\end{algorithm}

\begin{algorithm}
\caption{Compute Ritz values}
{\bf Input:}Orthogonal matrix $V^{n\times m}_{m}$\\
coefficients $\alpha[1..m]$ and $\beta[1..m-1]$
\begin{algorithmic}[1]
\STATE $T_{m} \leftarrow$ (build a tridiagonal matrix from $\alpha$ and $\beta$)
\STATE $QDQ^{T} \leftarrow EIG(T_{m})$
\STATE $\lambda_{1..k} \leftarrow$ (top k eigenvalues from D)
\STATE $Q_{k} \leftarrow $ (k columns of Q corresponding to $\lambda_{1..k})$
\STATE $R_{k} \leftarrow V_{m}Q_{k}$
\end{algorithmic}
\label{eigen:algo3}
\end{algorithm}

\subsubsection{Math}
Different from power method, the intermediate multiplication matrix is used to construct a set of orthonormal base of \emph{Krylv subspace $K_{m}$} which follows the definition:
\begin{equation}
K_{m} = < b, Ab, \cdots, A^{m-1}b>.
\end{equation}
The sub procedure to construct orthonormal bases may be any standard algorithm, for example Gram-schmidt algorithm. We can view Lanczos algorithm as an iterative method which incrementally construct Krylov subspace. The pseudo-code is shown in Algorithm \ref{eigen:algo1}.

After Lanczos factorization, we get a few matrices that satisfy the following equation:
\begin{equation}
	AV_{m} = V_{m}T_{m} + f_{m}e^{T}_{m}
\end{equation}
To name a few, $A^{n\times m}$ is input matrix, $V^{n\times m}_{m}$ contains the m orthonormal bases, $T^{m\times m}_{m}$ is a tridiagonal matrix, $f_{m}$ is new n-vector orthogonal to all columns of $V_{m}$, $
e_{m}$ is a vector that \emph{m}th element is 1, and others 0. After algorithm \ref{eigen:algo1}, we need to construct the matrix $T^{m\times m}_{m}$. The algorithm is quite simple, it is listed in algorithm \ref{eigen:algo2}.

The eigenvalues of $T_{m}$ are called Ritz values, and $V_{m}Y$'s columns are called Ritz vector. It is constructed by Algorithm \ref{eigen:algo3}. We expect the Ritz values and Ritz vectors to be good approximation of the eigenvalues and eigenvectors of original matrix. The computation of eigenvalues of $T_{m}$ can be done by standard quadratic algorithms, such as QR method.

\subsubsection{Idea of SQL implementation}
Since the algorithm of Lanczos algorithm is matrix calculation intensive, so we wrap all matrix related operation in our host language Python. I'll list the most important routines that appears very often in my high level implementation of Lanczos. Then in the final python code, I'l just call these wrappers instead of using raw SQL again and again.
\begin{description}
  \item[create\_vector\_or\_matrix:]{declare a vector/matrix variable}
  \item[assign\_to:]{Copy a variable's value to another variable}
  \item[vetorr\_length:]{Return the length of a vector}
  \item[vector\_dot\_product:]{take the dot product of two vectors}
  \item[reverse\_matrix:]{Append reverse of every edge into original graph}
  \item[matrix\_multiply\_matrix\_overwrite:]{Multiply a matrix with a matrix}
  \item[matrix\_multiplt\_vector\_overwrite:]{Multiply a matrix with a vector}
  \item[normalzed\_vector:]{Normalize a vector}
\end{description}

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Belief Propagation}
For belief propagation, we use the fabp method proposed in paper\cite{DBLP:conf/pkdd/KoutraKKCPF11}.

\subsubsection{method}
It can be shown that the solution of belief propagation can be approximated by the linear system:$$[\mathbf{I}	 + a\mathbf{D} - c\mathbf{A}]\mathbf{b_h} = \mathbf{\phi_h}$$
where $\mathbf{A}$ is the n by n symmetric adjacency matrix, $\mathbf{D}$ is the diagonal matrix of degrees, $b_h$ corresponds to the vector of final beliefs for each node, $\phi_h$ is prior belief vector, and $h_h$ is the homophily factor,  $a = 4h_h^2/(1 - h_h^2)$ and $c = 2h_h / (1-4h_h^2)$.

To solve this linear system, we can see :$\mathbf{I} + a\mathbf{D} - c\mathbf{A}$ as the form $\mathbf{I} - \mathbf{W}$, where $\mathbf{W} = -a\mathbf{D} + c\mathbf{A}$, and using the expansion:$$(\mathbf{I} - \mathbf{W})^{-1} = \mathbf{I} + \mathbf{W} + \mathbf{W}^2 + \mathbf{W}^3 + ...$$

and the solution of the linear system is given by the formula:
$$\mathbf{b_h} = (\mathbf{I} - \mathbf{W}^{-1})\mathbf{\phi_h} =\mathbf{\phi_h}  + \mathbf{\phi_h} \mathbf{W} + \mathbf{\phi_h} \mathbf{W}^2 + \mathbf{\phi_h} \mathbf{W}^3 + ...$$

Given this power method, the implementation is pretty straightforward as described in algorithm \ref{bp:algo4}.
\begin{algorithm}[!htbf]
\caption{Belief Propagation}
\begin{algorithmic}
\STATE{Bulk load graph into an edge table in database.}
\STATE{Initialize $h_h = 0.001 $}
\STATE{Initialize the initial belief of every node as prior belief}
\REPEAT
\STATE{Update the belief of node by $b_h(i) = b_h(i-1)\mathbf{W} + \mathbf{\phi_h}$ }
\UNTIL{Convergence}
\end{algorithmic}
\label{bp:algo4}
\end{algorithm}

\subsubsection{Idea of SQL Implementation}
The major computation involved is matrix vector multiplication, which is easy to implement in SQL using join and group by step. Furthermore we've wrapped all the matrix related operation in our host language Python, as described in previous task.

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Count of Triangle}
We use a simple technique proposed by \cite{tsourakakis2008fast}, its general idea is build upon a theorem that the count of triangles in a graph is proportional to the sum of cubes of eigenvalues of the graph.

\subsubsection{Global triangle}
The algorithm to calculate global triangles is as follows:

\paragraph{Math}
The formula to count global triangle is sum of cubes of eigenvalues, which is:
\begin{equation}
    \Delta(G) \gets \frac{1}{6} \sum_{j=1}^{i-1}\lambda_{j}^{3}
\end{equation}

\begin{algorithm}[!htbf]
\caption{The EigenTriangle algorithm}
{\bf Require: } Adjacency matrix A (n X n)\\
{\bf Require: } Tolerance \emph{tol}\\
{\bf Output: } $\bigtriangleup'(G)$ global triangle estimation
\begin{algorithmic}
\STATE{$\lambda_{i} \leftarrow LanczosMethod(A, 1)$}
\STATE{$\overrightarrow{\Lambda} \gets [\lambda_{1}]$}
\STATE{$i \gets 2 \{ $ {initialize i, $\overrightarrow{\Lambda}$} \} }
\REPEAT
    \STATE{$\lambda_{i} \leftarrow LanczosMethod(A, i)$}
    \STATE{$\overrightarrow{\Lambda} \gets [\overrightarrow{\Lambda} \lambda_{i}]$}
    \STATE{$i \gets i + 1$}
\UNTIL{0 $\leq$ $\frac{|\lambda_{i}^3|}{\sum_{j=1}^{i} |\lambda_{i}|^3} \leq$ tol}
\STATE{$\bigtriangleup'(G) \gets \frac{1}{6} \sum_{j = 1}^{i} \lambda_{i}^3$}
\RETURN{$\bigtriangleup'(G)$}
\end{algorithmic}
\end{algorithm}

\subsubsection{Local triangle}
The algorithm to calculate the local triangle is as follows.

\paragraph{Math}
$\Delta_{i}$ is the number of local triangles that node i participated in. The formula of local triangle count is based on the following theorem:
\begin{equation}
    \Delta_{j} = \frac{\sum_{k=1}^{i-1}u_{jk}^{2}\lambda_{k}^{3}}{2}
\end{equation}

\begin{algorithm}[!htbf]
\caption{The local eigentriangle algorithm\cite{tsourakakis2008fast}}
\begin{algorithmic}
\REQUIRE Adjacency matrix $A(n \times n)$
\REQUIRE Tolerance $tol$\\
{\bf OUTPUT: } $\Delta'(G)$  per node triangle estimation
\STATE $\langle \lambda_{1},\vec{u_{1}} \rangle \leftarrow LanczosMethod(A,1)$
\STATE $\vec{\Lambda} \leftarrow [\lambda_{1}]$
\STATE $\bigcup \leftarrow [\vec{u_{1}}]$
\STATE $i \leftarrow 2$
\REPEAT
\STATE $\langle \lambda_{i},\vec{u_{i}} \rangle \leftarrow LanczosMethod(A,i)$
\STATE $\vec{\Lambda} \leftarrow [\vec{\Lambda}\lambda_{i}]$
\STATE $\bigcup \leftarrow [\bigcup \vec{u_{1}}]$
\STATE $i \leftarrow i + 1$
\UNTIL $0 \leq \frac{|\lambda_{i}^3|}{\Sigma_{j=1}^{i-1}\lambda_{j}^3} \leq tol$
\FOR   {$j=1$ to $n$}
\STATE $\bigtriangleup_{j} = \frac{\Sigma_{k=1}^{i-1}u_{jk}^2\lambda_{k}^3}{2}$
\ENDFOR
\STATE $\bigtriangleup(G)\leftarrow[\bigtriangleup_{1},\ldots,\bigtriangleup_{n}]$
\RETURN $\bigtriangleup(G)$
\end{algorithmic}
\end{algorithm}

\subsubsection{Idea of SQL implementation}
We will call Lanczos to get the eigenvalues and eigenvectors of the graph, and sum up the number using SQL \emph{select}.

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Shortest Path}
We adopt Dijkstra's shortest algorithm to compute shortest path from the source node. It works for directed weighted
graph which has $O(|V| log |V| + |E|)$ time complexity. The pseudo code is listed in algorithm \ref{algo:dijkstra}.

\subsubsection{Math}
The core idea of Dijkstra's idea is that it greedily select a candidate node which has already obtained its minimum distance, then update its neighbors' current
best distance.

\begin{algorithm}[!htbf]
{\bf Input:} Source node and directed weighted graph. \\
{\bf Output:} Shortest path of every node from source node.
\begin{algorithmic}
\caption{Dijkstra shortest path algorithm}
\FORALL{node V in graph}
    \STATE{dist[V] $\gets$ infinity}
    \STATE{visited[V] $\gets$ false }
\ENDFOR
\STATE{dist[source] $\gets$ 0}
\STATE{insert source into Q}
\WHILE{Q is not empty}
    \STATE{u $\gets$ vertex in Q with smallest distance}
    \STATE{remove u from Q}
    \STATE{visited[u] $\gets$ true}
    \FORALL{neighbour v of u}
        \STATE{alt $\gets$ dist[u] + dist\_between(u, v)}
        \IF{alt $<$ dist[v] \&\& !visited[v]}
            \STATE{dist[v] $\gets$ alt}
            \STATE{insert v into Q}
        \ENDIF
    \ENDFOR
\ENDWHILE
\end{algorithmic}
\label{algo:dijkstra}
\end{algorithm}

\subsubsection{Idea of SQL implementation}
This is implemented purely in SQL. Just like ordinary C code, I have a table to record the current status(visited, distance) of each node, and update one node at a time.

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.

\subsection{Minimum Spanning Tree}
We use the classical Prim's algorithm for this additional task. We abandon the Kruskal's algorithm because the disjoint set is not easy to implement using SQL.

\subsubsection{method}
The general idea of Prim's algorithm is to first initialize a tree with a single vertex, chosen arbitrarily from the graph, Then we grow the tree by one edge, the one that connect
the tree to vertices not yet in the tree with minimum cost(weight). We repeat this process until all of the nodes in a graph is in the tree(of course we're assuming that the graph is weighted and connected).

\subsubsection{Idea of SQL implementation}
The implementation in SQL can be described in algorithm \ref{algo:prim}.

\begin{algorithm}
{\bf Input:} Edge Table E of a undirected connected graph \\
{\bf output:} Edge Table MST containing edges of the minimum spanning tree
\begin{algorithmic}
\caption{Prim's algorithm}
\STATE Create a node table N
\STATE Randomly insert a node into N
\FOR {$i=1$ to $number of nodes - 1$}
	\STATE Insert into MST an edge from E with minimum weight where src node is in N and destination node is not in N
	\STATE Insert into N with the destination node of the edge selected in last step
\ENDFOR
\end{algorithmic}
\label{algo:prim}
\end{algorithm}

\subsubsection{SQL code}
Please refer to the code in {\bf Appendix}.


\documentclass[jou]{apa6}

\usepackage[american]{babel}

\usepackage{csquotes}
\usepackage[style=apa,sortcites=true,sorting=nyt,backend=biber]{biblatex}
\DeclareLanguageMapping{american}{american-apa}
\addbibresource{bibliography.bib}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Discrete Structures
%% The start of RBS stuff
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Working internal and external links in PDF
\usepackage{hyperref}
% Extra math symbols in LaTeX
\usepackage{amsmath}
\usepackage{gensymb}
\usepackage{amssymb}
% Enumerations with (a), (b), etc.
\usepackage{enumerate}
\usepackage[framemethod=TikZ]{mdframed}
\usepackage{xcolor}
\usepackage{graphicx}
\usepackage[justification=centering]{caption}
\usepackage{fancyvrb}

\let\OLDitemize\itemize
\renewcommand\itemize{\OLDitemize\addtolength{\itemsep}{-6pt}}

\usepackage{etoolbox}
\makeatletter
\preto{\@verbatim}{\topsep=3pt \partopsep=3pt }
\makeatother

% These sizes redefine APA for A4 paper size
\oddsidemargin 0.0in
\evensidemargin 0.0in
\textwidth 6.27in
\headheight 1.0in
%\topmargin -24pt
\topmargin -32pt
\headheight 12pt
\headsep 12pt
%\textheight 9.19in
\textheight 9.35in

\title{Sample Quiz 8}
\author{Discrete Structures, Spring 2020}
\affiliation{RBS}

\leftheader{Discrete Sample Quiz 8}

\abstract{%
}

%\keywords{}

\setlength\parindent{0pt}

\begin{document}

\twocolumn

\section{Appendix: Quiz on Individual Topics}

\vspace{4pt}
{\bf Question 1.} A hacker wants to
use {\em Arithmetic coding}
that would send a virus to the client's computer and unpack itself there.
His favorite virus uses this RNS sequence: $\textcolor{blue}{\mathtt{GACGU\$}}$,
where $\textcolor{blue}{\mathtt{A}},\textcolor{blue}{\mathtt{C}},\textcolor{blue}{\mathtt{G}},\textcolor{blue}{\mathtt{U}}$ are four nucleobases (the useful data payload),
but the symbol $\textcolor{blue}{\$}$ is used just once to mark
the end of the RNS string.

He uses the following {\em a priori} frequencies for the symbols:

\begin{tabular}{ccccc}
$\textcolor{blue}{\mathtt{A}}$ & $\textcolor{blue}{\mathtt{C}}$ & $\textcolor{blue}{\mathtt{G}}$ & $\textcolor{blue}{\mathtt{U}}$ & $\textcolor{blue}{\mathtt{\$}}$ \\
$30\%$ & $10\%$ & $30\%$ & $20\%$ & $10\%$ \\
\end{tabular}

\vspace{4pt}
He starts with the half-closed line segment $S_0 = [0;1)$ and at every step (for all $i = 0,1,2,3,4,5$) creates the
next segment $S_{i+1}$ from $S_i$ by dividing $S_i$ into five parts of lengths proportional
to the frequencies (the proportions of subdivisions are $3\!\!:\!\!1\!\!:\!\!3\!\!:\!\!2\!\!:\!\!1$). Then $S_{i+1}$ is the
subdivision of the previous $S_i$ corresponding to the newly encoded character.
After encoding all six characters in the virus message $\textcolor{blue}{\mathtt{GACGU\$}}$, the hacker
gets the segment $S_6$.

\begin{figure}[!htb]
\center{\includegraphics[width=3in]{quiz-on-individual-topics/arithmetic-coding.png}}
\caption{\label{fig:arithmetic-coding} Building arithmetic code}
\end{figure}

Find the binary fraction $\beta$ belonging to $S_6$.\\
\textcolor{teal}{\em Select one answer.}

{\bf (A)} $\beta = 0.011011101100011_2$\\
{\bf (B)} $\beta = 0.011011101100110_2$\\
{\bf (C)} $\beta = 0.011011101101001_2$\\
{\bf (D)} $\beta = 0.011011101101100_2$\\
{\bf (E)} $\beta = 0.011011101101111_2$\\

{\em Note.} In arithmetic coding any sequence of
$n$ bits represents an interval of length $\frac{1}{2^n}$.
For example the bit sequence {\tt 010} stands for the segment of real numbers:
$[0.010_2; 0.011_2) = [0.25; 0.375)$ having length $\frac{1}{2^3} = \frac{1}{8}$.
If the arithmetic coding yields a segment $[a;b)$ such that
$[0.25; 0.375) \subseteq [a;b)$, then {\tt 010} is the
result of the arithmetic coding.

The number $\beta = 0.010_2 \in [a;b)$, and the extra "0"
character ensures that the interval is not too long and it fits inside $[a;b)$.
For example, {\tt 01} would represent a different interval $[0.25; 0.5) \neq [0.25; 0.375)$.

\vspace{10pt}
{\bf Question 2.} We want to use a {\em regular expression}
to find all phone numbers with the Latvian country code.
Assume that the phones can have one of the following formats (here
symbol {\tt D} denotes any digit (0-9)). The space symbols
are used exactly as written (they are single space characters).

\begin{verbatim}
(+371) DDDDDDDD
+371 DDDDDDDD
\end{verbatim}

Pick a regular expression that would recognize just these 2 phone formats.\\
\textcolor{teal}{\em Select one answer.}

\begin{verbatim}
(A)   (+371|\(+371\)) \d{8}
(B)   (\+371|\(\+371\)) [0-9]{8}
(C)   (+371|\(+371\)) \d\d\d\d\d\d\d\d
(D)   \+371|(\+371\) [0-9]{8,8}
\end{verbatim}

{\em Note.} If you wish, you can use text editor such as Notepad++ search dialogue
to verify which regex works. (Click {\bf Ctrl+F},
enter your regular expression, switch ``Search mode'' to Regular Expression,
and click the button  ``Find All in All Opened Documents''):

\begin{figure}[!htb]
\center{\includegraphics[width=2in]{quiz-on-individual-topics/notepad-regex.png}}
\caption{\label{fig:regex-search} Regex Search in Notepad++}
\end{figure}

If you are on a Linux machine, you can also
test regex search with the command "grep" (and flag -E).

\vspace{10pt}
{\bf Question 3.} About 70\% of the entries in a bit array of a {\em Bloom filter} are
equal to $1$ as it is initialized with the words
from some dictionary $D$.
The Bloom filter computes $n=8$ independent hash functions to check,
if some entry belongs to the dictionary $D$.
What is an (approximate) chance $P$ to get a false positive?
A false positive is an event where one picks
a random word $w \not\in D$,
and Bloom filter incorrectly states that $w$ belongs to $D$\\

\vspace{4pt}
\textcolor{teal}{\em Select one answer.}\\
{\bf (A)} $P = 30.00\%$,\\
{\bf (B)} $P = 5.76\%$,\\
{\bf (C)} $P = 3.75\%$,\\
{\bf (D)} $P = 0.66\%$.

\vspace{10pt}
{\bf Question 4.}
A testing laboratory tests the same number of people daily, and on day $i$, the number
of people who tested positively for some health condition was $n_i$. The laboratory knows that
the numbers $n_i$ are distributed according to {\em Poisson distribution} with the
expected value $\lambda = 13.5$.

By $P$ we denote the probability that on the given day
$n_i < 5$ (i.e. less than $5$ people test
positive for that condition). Which is the closest approximate value for this probability?

{\bf (A)} $P = 0.26\%$,\\
{\bf (B)} $P = 0.51\%$,\\
{\bf (C)} $P = 5.78\%$,\\
{\bf (D)} $P = 10.89\%$.

{\em Note.} Another typical illustration of the same Poisson distribution:
Imagine the ice cream ``R\={u}jienas sald\={e}jums'' with raisins. (In this case
the average number of raisins in one package is $\lambda = 13.5$; and you have to find the
probability that a given ice cream package has at most $4$ raisins.
See \url{https://bit.ly/2KanwIf}.

\begin{figure}[!htb]
\center{\includegraphics[width=1in]{quiz-on-individual-topics/rujiena-icecream.png}}
\caption{\label{fig:rujiena-icecream} An Ice Cream Package with Raisins satisfying Poisson Distribution}
\end{figure}

% ff <- function(k) {  return((lam^k*exp(-lam))/factorial(k) }

\vspace{10pt}
{\bf Question 5.} Find the minimum number of colors to paint the $12$ vertices
from the graph shown in Figure~\ref{fig:graph-coloring} so
that any two vertices connected with an edge are having different colors.
(This number $n$ is called the {\em chromatic number} for the graph $G$.)

\begin{figure}[!htb]
\center{\includegraphics[width=1.6in]{quiz-on-individual-topics/durer-graph.png}}
\caption{\label{fig:graph-coloring} Graph for Vertex Coloring}
\end{figure}

\vspace{10pt}
{\bf Question 6.} Assume that the ``World Wide Web'' contains only
$4$ pages $A,B,C,D$ that link to each other as shown in the picture below.

\begin{figure}[!htb]
\center{\includegraphics[width=1in]{quiz-on-individual-topics/pageranks.png}}
\caption{\label{fig:pageranks} Four webpages with links.}
\end{figure}

% https://matrix.reshish.com/multiplication.php

In the Iteration $0$ initialize the page ranks with equal values
$$\text{PR}_0(A) = \ldots = \text{PR}_0(D) = \frac{1}{4}.$$
Compute the first two iterations of these pageranks, using the formulas:
$$\left\{ \begin{array}{l}
\text{PR}_{i+1}(A) = (1 - d) + d\left( \frac{\text{PR}_i(D)}{2} \right)\\
\text{PR}_{i+1}(B) = (1 - d) + d\left( \frac{\text{PR}_i(A)}{2} + \frac{\text{PR}_i(C)}{1} + \frac{\text{PR}_i(D)}{2}  \right)\\
\text{PR}_{i+1}(C) = (1 - d) + d\left( \frac{\text{PR}_i(B)}{2} \right)\\
\text{PR}_{i+1}(D) = (1 - d) + d\left( \frac{\text{PR}_i(A)}{2} + \frac{\text{PR}_i(B)}{2} \right)
\end{array} \right.$$
Set the value of the damping factor $d=0$.

Write the values of the second iteration for all the pages:
$\text{PR}_2(A),\ldots,\text{PR}_2(D).$.

\textcolor{teal}{\em Write $4$ comma-separated numbers; round them to
the nearest thousandth.}

\vspace{4pt}
{\em Note.} You can also use vector algebra, if you are
familiar with multiplying matrices with vectors
\textendash{} \url{https://bit.ly/2RJTNtC}.
$$\left( \begin{array}{c}
\text{PR}_2(A)\\
\text{PR}_2(B)\\
\text{PR}_2(C)\\
\text{PR}_2(D)
\end{array} \right) = \left(
\begin{array}{cccc}
\textcolor{blue}{0}   & \textcolor{green}{0}   & \textcolor{red}{0} & \textcolor{purple}{1/2} \\
\textcolor{blue}{1/2} & \textcolor{green}{0}   & \textcolor{red}{1} & \textcolor{purple}{1/2} \\
\textcolor{blue}{0}   & \textcolor{green}{1/2} & \textcolor{red}{0} & \textcolor{purple}{0} \\
\textcolor{blue}{1/2} & \textcolor{green}{1/2} & \textcolor{red}{0} & \textcolor{purple}{0}
\end{array} \right)^2 \cdot \left( \begin{array}{c}
1/4 \\
1/4 \\
1/4 \\
1/4
\end{array} \right).$$
In this formula a square matrix $4 \times 4$ is twice multiplied to a $4 \times 1$
vector $(1/4, 1/4, 1/4, 1/4)$ from the left side.
%See \url{https://bit.ly/2VkZHnh}.

\vspace{4pt}
{\em Note.} \url{https://checkpagerank.net/check-page-rank.php}
shows that:\\
{\tt https://www.bitl.lv/} has PageRank $2/10$,\\
{\tt https://www.delfi.lv/} has PageRank $6/10$.\\
This does {\bf not} mean that there are three times more
inbound links to Delfi than to BITL (or that these links are three times more ``valuable'').
The value returned by this web resource is a {\em logarithmic measure}
of its iterative value. In fact, the difference between $2/10$ and $6/10$
means that the popularity of these pages differs by many orders of magnitude.
See \url{https://bit.ly/2yrRMf7}.

\vspace{10pt}
{\bf Question 7.} As you probably know, {\em Karatsuba's algorithm} can express
the multiplication of two numbers of length $n$ digits as
three multiplications of numbers of length $n/2$ (i.e. the number of
operations are three times larger, but the operands become two times shorter).
This ultimately means that Karatsuba's algorithm requires
only $O(n^{1.585})$ operations to multiply numbers of length $n$.

Imagine that somebody has invented a new operation $a \otimes b$ for
some objects $a,b$ (both $a,b$ have the same size $n$).
Assume that s/he knows how to express
$a \otimes b$ using $7$ operations $a_i \otimes b_i$ (where $i = 1,2,\ldots,7$, and
all $a_i,b_i$ have size $n/2$, i.e. half the size of the original operands $a,b$).
({\em We do not care, what the operation $\otimes$ does; but we know that we
can compute it for arguments $a,b$ of length $1$ in constant time; it is
therefore easy for very short arguments.})

Find the best Big-O-Notation estimate for the time needed to compute $a \otimes b$, if $a,b$ are
both of size $n$.

{\bf (A)} $O(n^2)$\\
{\bf (B)} $O(n^2 \log n)$\\
{\bf (C)} $O(n^{2.646})$\\
{\bf (D)} $O(n^{2.808})$\\
{\bf (E)} $O(n^3)$\\
{\bf (F)} $O(n^3 \log n)$\\
\textcolor{teal}{\em Select one answer.}

\vspace{4pt}
{\em Note 1.} One can use Master's theorem (Rosen2019, p.558) for
this problem and also for the Karatsuba's algorithm.

\vspace{4pt}
{\em Note 2.} If the algorithm falls in multiple Big-O complexity classes, pick
the one that shows the slowest growth. For example, if a speed of an algorithm is
both in $O(n^2)$ and $O(n^3)$,
then $O(n^2)$ would be a more precise and a more useful estimate.

\vspace{10pt}
{\bf Question 8.} Assume that two players $A$ and $B$ play a {\em matrix game}.
They simultaneously guess one number each. Either player can guess
one of these three numbers: $\{ 1,2,5 \}$.
The payoff matrix is shown below.
In each cell the first number is what is paid to player $A$,
the second number is paid to player $B$.

\begin{figure}[!htb]
\center{\includegraphics[width=2.4in]{quiz-on-individual-topics/matrix-game.png}}
\caption{\label{fig:matrix-game} Matrix game with payoffs.}
\end{figure}

Expressed in human language, the rules are as follows.
Assume that the player $A$ just guessed a number $a$, and player $B$ guessed
a number $b$.
\begin{itemize}
\item If $a=b$, then it is a tie; nobody pays anything.
\item If $a>b$ (yet $a < 3b$), then $B$ pays to $A$ one euro. (And also,
if $b>a$ yet $b < 3a$, then $A$ pays to $B$ one euro.)
\item If $a \geq 3b$, then $A$ pays to $B$ two euros. (And also, if $b \geq 3a$,
then $B$ pays to $A$ two euros.)
\end{itemize}

{\em In this number guessing game one can win by guessing a number which is a little
bit larger than the other player's number. But one should not guess a number which is
larger than the other by ``a lot'' (if you exceed the other player's number
three times or more, then you suffer a double loss.).}

Which can be {\em Nash equilibrium} for this number guessing game?
(You can assume that one of the answer variants is correct \textendash{}
the same optimal strategy for both players. It is enough to find the
one that beats all the other strategies.)
Each strategy lists the probabilities for guessing the number $x$:

{\bf (A)} $P(x = 1) = 1/3$, $P(x = 2) = 1/3$, $P(x = 5) = 1/3$.\\
{\bf (B)} $P(x = 1) = 0$, $P(x = 2) = 1/2$, $P(x = 5) = 1/2$.\\
{\bf (C)} $P(x = 1) = 1/2$, $P(x = 2) = 1/2$, $P(x = 5) = 0$.\\
{\bf (D)} $P(x = 1) = 1/4$, $P(x = 2) = 1/2$, $P(x = 5) = 1/4$.\\
{\bf (E)} $P(x = 1) = 2/6$, $P(x = 2) = 3/6$, $P(x = 5) = 1/6$.\\
\textcolor{teal}{\em Select one answer.}

\vspace{10pt}
{\bf Question 9.} The first iterations using Lindermayer system are given:\\
{\bf Iteration 0:} {\tt A}\\
{\bf Iteration 1:} {\tt AB}\\
{\bf Iteration 2:} {\tt ABBA}\\
{\bf Iteration 3:} {\tt ABBABAAB}\\
{\bf Iteration 4:} {\tt ABBABAABBAABABBA}

({\em To get Iteration 5: Take Iteration 4,
change all A's into B's and
vice versa, and append such string to the end of Iteration 4.})
Find the correct set of rules to generate this L-system.

{\bf (A)} ${\displaystyle \left\{ \begin{array}{l}
\mathtt{A} \rightarrow \mathtt{B}\\
\mathtt{B} \rightarrow \mathtt{BA}
\end{array} \right. }$\\
{\bf (B)} ${\displaystyle \left\{ \begin{array}{l}
\mathtt{A} \rightarrow \mathtt{AB}\\
\mathtt{B} \rightarrow \mathtt{AA}
\end{array} \right. }$\\
{\bf (C)} ${\displaystyle \left\{ \begin{array}{l}
\mathtt{A} \rightarrow \mathtt{AB}\\
\mathtt{B} \rightarrow \mathtt{BA}
\end{array} \right. }$\\
{\bf (D)} ${\displaystyle \left\{ \begin{array}{l}
\mathtt{A} \rightarrow \mathtt{AB}\\
\mathtt{B} \rightarrow \mathtt{AB}
\end{array} \right. }$

\textcolor{teal}{\em Select one answer.}

{\em Note.} We can have a {\em turtle} that reads this
sequence and performs actions:
\begin{itemize}
\item Letter $\mathtt{A}$: Step $1$ unit ahead, turn
$60^{\circ}$ counterclockwise.
\item Letter $\mathtt{B}$: Turn $180^{\circ}$.
\end{itemize}
In this case the iterations $0,2,4,\ldots$ would produce
a fragment of Koch snowflake (Figure~\ref{fig:lindenmayer-system}).

\begin{figure}[!htb]
\center{\includegraphics[width=2.5in]{quiz-on-individual-topics/lindenmayer-system.png}}
\caption{\label{fig:lindenmayer-system} Koch curve as an L-system}
\end{figure}

\vspace{10pt}
{\bf Question 10.} Assume that someone uses
a {\em secure hash} algorithm $h(x)$ that for any file $x$
outputs a hash value consisting of exactly $100$
bits. (The typical SHA-256 algorithm would return $256$ bits.)

Assume that we want to use brute force to find
hash collision \textendash{} two different files $x_1,x_2$
such that $h(x_1) = h(x_2)$.
You can estimate, how many hash values we need to compute before we
get at least $50\%$ probability to find a hash collision.
Estimation can be done using Square aproximation from
the Birthday paradox:
\begin{equation}
\label{eq:square-approximation}
p_{\text{collision}} \approx \frac{n^2}{2m},
\end{equation}
Formally: If a hash function $h(x)$ can take $m$ different values and
we randomly pick $n$ different integer numbers $x_1,\ldots,x_n$, then the probability
that there is at least one collision ($h(x_i) = h(x_j)$ and $x_i \neq x_j$) is approximately expressed by
the formula (\ref{eq:square-approximation}). See \url{https://bit.ly/2RNjhGB}.

Assume that a single hash value $h(x)$ can be computed in one microsecond
($1\,\mu{}s = 10^{-6}\,s$). Estimate the number of years it would take to produce a
collision for a 100-bit secure hash algorithm with probability at least $50\%$.

{\bf (A)} The expected time is $0.11$ years.\\
{\bf (B)} The expected time is $35.7$ years.\\
{\bf (C)} The expected time is $71.4$ years.\\
{\bf (D)} The expected time is $856$ years.\\
{\bf (E)} The expected time is $4.02\cdot 10^{16}$ years.\\
\textcolor{teal}{\em Select one answer.}

\vspace{10pt}
{\bf Question 11.} Consider the following
problem solving strategies:

{\footnotesize
{\bf (A)} {\bf Drawing a picture.} Can you
write down all the things you need to consider on paper?
Can you order them nicely in a list or a table?
Can you show them in a two-dimensional or a three-dimensional drawing?\\
{\bf (B)} {\bf Getting hands dirty.} Can you start experimenting with the
problem, plug in specific values, see where they lead you?\\
{\bf (C)} {\bf Going to the extremes.} Can you pick some ``borderline case''?
Is there the smallest or the largest item that is possible in the problem?\\
{\bf (D)} {\bf Lateral thinking.} Could it happen that your current solving approach
is not applicable or is too inefficient? Can you pretend that you
have not spent many years studying mathematics at school;
can you apply lateral/divergent thinking out of the box to
come up with something unexpected?\\
{\bf (E)} {\bf Looking for symmetries.} Can we switch two numbers or two letters in our
notation? Can we inspect just one item and notice that many others are identical?\\
{\bf (F)} {\bf Making it easier.} Can we make a simpler version of this problem and
solve it first? Insert a smaller number? Solve only one particular case of it?\\
{\bf (G)} {\bf Penultimate step.} What precondition must take place before
the final solution step is possible? Imagine, which result you would need
in order to say that you are ``almost done''.\\
{\bf (H)} {\bf Wishful thinking.} Can you apply some outrageous simplification to your
initial problem. Imagine for a while that you have already solved it: What would that imply?
% http://courses.cs.vt.edu/~cs4104/shaffer/Fall2010/PSintro.pdf
}

Now consider the following problem:

\begin{figure}[!htb]
\center{\includegraphics[width=2in]{quiz-on-individual-topics/tower-of-hanoi.png}}
\caption{\label{fig:tower-of-hanoi} Tower of Hanoi}
\end{figure}

\begin{mdframed}[roundcorner=6pt]
{\footnotesize
{\bf Problem.} A Tower of Hanoi (Figure~\ref{fig:tower-of-hanoi})
has three pegs ($A$, $B$, $C$) and
four disks initially on the peg $A$. The task is to move all the four disks to the peg $B$, where
the following rules apply:\\
Rule 1: Only one disk can be moved at a time.\\
Rule 2: Each move consists of taking the upper disk from any peg and moving it to
another peg.\\
Rule 3: No larger disk may be placed on top of a smaller disk.

{\em The solver wants to come up with the sequence of moves. S/he has tried
a similar game with just three disks with some trial and error, but
is not sure how to proceed in the case with four disks. Somebody suggests
a few ``natural looking'' hints.}

{\bf Hint 1.} Find the disk that is the hardest to move anywhere or moved least frequently?\\
{\bf Hint 2.} To which peg all the other disks need to go before we move this disk?\\
{\bf Hint 3.} Assume that you know how to move three disks from the peg $A$ to the peg $B$.
Can you move them between any other pegs? How?
}
\end{mdframed}

What kind of problem solving strategies are contained in the hints?

\textcolor{teal}{\em Select up to three relevant strategies (A-H).}

\begin{center}
\includegraphics[width=2in]{quiz-on-individual-topics/thinking-outside-the-box.png}\\
\textcopyright{} {\em Leo Cullum}, \url{https://www.newyorker.com/}
\end{center}

\vspace{10pt}
{\bf Question 12.} Someone wants to compute a MD5 checksum for the following file:

\textcolor{blue}{
{\tt 95.211.48.179~~~bitl.lv}
}

The following is true:
\begin{itemize}
\item File {\tt hosts.txt} is exactly $23$ bytes long.
\item The IP address is seperated from {\tt bitl.lv} by
a single horizontal tab (byte in hexadecimal: {\tt 0x09}).
\item The only line is ends with a Windows-style line ending (carriage return, line feed:
bytes in hexadecimal: {\tt 0x0D}, {\tt 0x0A}).
\end{itemize}

Figure~\ref{fig:hosts-file} shows file displayed by {\em Total Commander};
button {\bf F3}, then menu item {\bf Options $>$ Hex} (and also in the Notepad++ editor).

\begin{figure}[!htb]
\center{\includegraphics[width=3in]{quiz-on-individual-topics/hosts-file.png}}
\caption{\label{fig:hosts-file} Bytes in file {\tt hosts.txt}}
\end{figure}

\textcolor{teal}{\em Copy the whole MD5 checksum in your answer.}

{\em Note.} In this exercise it is important to have exactly the
same file content as shown in the picture.
For example, replacing the {\bf TAB} character
by one or more spaces (or Windows-style line ending with a UNIX-style
line ending) would totally change MD5.
(For secure hashes there is absolutely no string tokenization \textendash{}
unlike plagiarism detection they are very sensitive against
the smallest changes in their input.)

\vspace{10pt}
{\bf Question 13.} There are two people playing a game: Player $A$ (he is the Maximizer \textendash{} wants
to go down to a leaf with maximum payoff), and Player $B$ (he is the Minimizer \textendash{} wants
to minimize the $A$'s payoff). The current position is the root of the tree (Figure~\ref{fig:minimax}) and it is Player's $A$
turn to make the first move (to any of the root's children). After that Player $B$ moves (going down one more level) and so on
\textendash{} until they reach a leaf, which shows the payoff for Player $A$.\\
Find the maximum payoff for Player $A$ (you could use minimax algorithm with or without Alpha-Beta pruning
speedup to find out).

\textcolor{teal}{\em Write the payoff as a positive integer.}

\begin{figure}[!htb]
\center{\includegraphics[width=3in]{quiz-on-individual-topics/minimax.png}}
\caption{\label{fig:minimax} Game positions in a tree.}
\end{figure}

\vspace{10pt}
{\bf Question 14.} If you verify the Conway's game of life the configuration $P_0$
of a straight line with $4$ live cells (Figure~\ref{fig:conway}),
you need $N = 2$ steps until you reach ``periodic state'' $P_2$ that will
return after period $T = 1$ (i.e.\ returning needs just one step $P_3 = P_2$,
since ``beehive'' configuration is stable). So in this case $(N,T)=(2,1)$ \textendash{}
there are $N=2$ preliminary steps, and after that there is a period of length $T=1$.

\begin{figure}[!htb]
\center{\includegraphics[width=2.2in]{quiz-on-individual-topics/conway.png}}
\caption{\label{fig:conway} Conway game for a line of $4$.}
\end{figure}

Now consider a different starting position $P_0$ that contains a straight line
of $5$ live cells (Figure~\ref{fig:conway2}). Determine the number of steps $N$ needed
to reach the first position $P_N$ that would repeat infinitely often, and
the length of period $T$ with which the subsequent steps repeat, i.e.
the smallest positive integer $T$ with the property:
$$\forall k \in \mathbb{Z}_{0+},\;(k \geq N)\;\rightarrow\; (P_{k+T} = P_k).$$

\begin{figure}[!htb]
\center{\includegraphics[width=0.6in]{quiz-on-individual-topics/conway2.png}}
\caption{\label{fig:conway2} Starting position $P_0$ for a line of $5$.}
\end{figure}

\textcolor{teal}{\em Write two comma-separated integers $N,T$.}

{\em Note.} In Conway's game there are some positions
that are not periodic (glider guns that
constantly create new stuff), but most simple positions eventually reach periodic state.
Therefore the numbers $N$ and $T$ are defined in these cases.

\vspace{10pt}
{\bf Question 15.} A mathematical theory $\mathcal{T}$ (in a similar way as Coq software)
provides rules to prove various mathematical statements.
Assume that in this theory $\mathcal{T}$ one can prove some statement $A$ and also the
statement $\neg A$. Which description is true for this theory:

{\bf (A)} $\mathcal{T}$ is consistent.\\
{\bf (B)} $\mathcal{T}$ is not consistent.\\
{\bf (C)} $\mathcal{T}$ is complete.\\
{\bf (D)} $\mathcal{T}$ is not complete.\\
{\bf (E)} $\mathcal{T}$ is effectively axiomatized.\\
{\bf (F)} $\mathcal{T}$ is not effectively axiomatized.\\
\textcolor{teal}{\em Select one answer.}

\vspace{10pt}
{\bf Question 16.} Some banks can issue
19-digit credit card numbers (instead of the more typical 16-digit ones).
Assume that there is a 19-digit number that satisfies the Luhn check (mod $10$):

$$\mathtt{557367054456450571\ast}.$$

Please find the digit that is written in the place of the last $\ast$ symbol.

\textcolor{teal}{\em Write a single digit.}

\vspace{10pt}
{\bf Question 17.}
Some text $T$ has been tokenized into a sequence of $N$ words ($w_0,\ldots,w_{N-1}$).
Assume that you assigned
unique numbers to the stemmed words (each word $w_i$ in the text $T$ is replaced by
a number $n(w_i)$); and then computed rolling hash values for
five consecutive words in this text using this formula:\\

%\begin{align}
% & H(w_1,w_2,w_3,w_4,w_5) = \nonumber \\
%= & \left( n(w_1) \cdot a^4 + n(w_2) \cdot a^3 + n(w_3) \cdot a^2 + \right. \nonumber \\
%+ & \left. n(w_4) \cdot a^1 + n(w_5) \cdot a^0 \right)\;\; \text{mod}\;\; q. \nonumber
%\end{align}

{\footnotesize
\begin{align}
 & H(w_1,w_2,w_3,w_4,w_5) = \nonumber \\
= & \left( n(w_1) \cdot a^4 + n(w_2) \cdot a^3 + n(w_3) \cdot a^2 + n(w_4) \cdot a^1 + n(w_5) \right)\, \text{mod}\,q. \nonumber
\end{align}
}

This is a polynomial value for the argument $a$ followed by a remainder when dividing by $q$.
Parameters $a$ and $q$ are two large primes.

We compute all such hash values:
$$\left\{ \begin{array}{l}
v_0 = H(w_0,w_1,w_2,w_3,w_4),\\
v_1 = H(w_1,w_2,w_3,w_4,w_5),\\
\ldots\\
v_{N-5} = H(w_{N-5},w_{N-4},w_{N-3},w_{N-2},w_{N-1}).
\end{array} \right.$$

It turned out that $10\%$ of these hash values were found in an existing
hashtable $H$ (built from some existing texts using the same hash function) \textendash{}
about $5\%$ of the values in that hashtable are marked (the others are empty).
What is the most likely explanation for this?

{\bf (A)} Text $T$ contains large chunks of text that is copy-pasted from other sources.\\
{\bf (B)} Overlaps of the size $10\%$ can happen by chance. On the other hand, overlaps
exceeding $1/5$ (the size of the rolling hash window) would be highly unusual and
would require manual inspection.\\
{\bf (C)} This is not an effective way to detect copying and plagiarism.
Rolling hash should instead run on characters (not entire words),
since multiple authors may use the same words.\\
\textcolor{teal}{\em Select one answer.}

\vspace{10pt}
{\bf Question 18.} Jane took an ordinary soccer ball made from an elastic material
(Figure~\ref{fig:icosahedron3d}).

\begin{figure}[!htb]
\center{\includegraphics[width=2in]{quiz-on-individual-topics/truncated-icosahedron-3d.png}
\caption{\label{fig:icosahedron3d} 3D Soccer Ball}
}
\end{figure}

She stretched one of its faces so that it became a planar graph
(Figure~\ref{fig:icosahedron2d}).
Then she marked a Hamiltonian cycle in this graph (not shown).

\begin{figure}[!htb]
\center{\includegraphics[width=2.4in]{quiz-on-individual-topics/truncated-icosahedron.png}}
\caption{\label{fig:icosahedron2d} Planar Soccer Ball}
\end{figure}

How many edges of this graph do {\bf not} belong to the Hamiltonian cycle?\\
\textcolor{teal}{\em Write a positive number.}

\mbox{}
\newpage

\subsection{Answers}

\vspace{4pt}
{\bf Question 1.} Answer {\bf (D)}\\
The intervals encoding the string $\textcolor{blue}{\mathtt{"GACGU\$"}}$
form the following sequence:
$$\left\{ \begin{array}{l}
S_0 = [0.000000; 1.000000]\;\text{encodes}\;\textcolor{blue}{\mathtt{""}}\;\text{(empty string)}, \\
S_1 = [0.400000; 0.700000]\;\text{encodes}\;\textcolor{blue}{\mathtt{"G"}}, \\
S_2 = [0.400000; 0.490000]\;\text{encodes}\;\textcolor{blue}{\mathtt{"GA"}}, \\
S_3 = [0.427000; 0.436000]\;\text{encodes}\;\textcolor{blue}{\mathtt{"GAC"}}, \\
S_4 = [0.430600; 0.433300]\;\text{encodes}\;\textcolor{blue}{\mathtt{"GACG"}}, \\
S_5 = [0.432490; 0.433030]\;\text{encodes}\;\textcolor{blue}{\mathtt{"GACGU"}}, \\
S_6 = [0.432976; 0.433030]\;\text{encodes}\;\textcolor{blue}{\mathtt{"GACGU\$"}}. \\
\end{array} \right.$$

To make the sequence of intervals $S_0 \supset S_1 \supset \ldots \supset S_6$,
we can define iterative sequences $\text{Left}_i$, $\text{Length}_i$ describing
the left endpoint and the length of each successive interval so that for every
$i = 0,1,\ldots,6$:
$$S_i = \left[ \text{Left}_i; \text{Left}_i + \text{Length}_i \right].$$

These sequences are defined in terms of the offsets and frequencies
of the characters $c_0, c_1, \ldots, c_5$.

$$\left\{ \begin{array}{l}
\text{Left}_{i+1} = \text{Left}_{i} + \text{Left}_{i} \cdot \text{Offset}(c_i) \\
\text{Length}_{i+1} = \text{Length}_{i}  \cdot \text{Frequency}(c_i) \\
\end{array} \right.$$

For each encodable character the offsets and frequencies are defined in this table:

\begin{tabular}{|l|l|l|} \hline
Character & Frequency & Offset \\ \hline
$\mathtt{A}$ & $0.3$ & $0.0$ \\ \hline
$\mathtt{C}$ & $0.1$ & $0.3$ \\ \hline
$\mathtt{G}$ & $0.3$ & $0.4$ \\ \hline
$\mathtt{U}$ & $0.2$ & $0.7$ \\ \hline
$\mathtt{\$}$ & $0.1$ & $0.9$ \\ \hline
\end{tabular}

The only binary number $\beta$ that belongs to
$S_6 = [0.432976; 0.433030]$ is shown in answer {\bf (D)}:
$$\beta = 0.011011101101100_2 = 0.4329834_{10}.$$

\vspace{10pt}
{\bf Question 2.} Answer {\bf (B)}\\
The only syntactically correct regular expression is
\begin{verbatim}
(\+371|\(\+371\)) [0-9]{8}
\end{verbatim}

\vspace{10pt}
{\bf Question 3.} Answer {\bf (B)}\\
The probability that all $8$ hash values
will happen to be $1$ is
$$0.7^8 = 0.05764801.$$

\vspace{10pt}
{\bf Question 4.} (None of the answers is right)\\
We can add up the probabilities, using Poisson distribution formula:
$$P(X = k) = \frac{\lambda^k \cdot e^{-\lambda}}{k!}.$$
By adding up the first $5$ values of this distribution we get this expression with $\lambda = 13.5$:
$$P(X < 5) = \sum\limits_{k=0}^{4} \frac{\lambda^k \cdot e^{-\lambda}}{k!} \approx 0.000707.$$
Therefore the probability to get less than $5$ raisins is $0.0707\%$.

Since all the answer variants were wrong, everyone gets full credit for this.

\vspace{10pt}
{\bf Question 5.} Answer: $3$\\
Three colors are sufficient as shown in the picture.
But two colors are impossible (since the graph contains a triangle \textendash{}
a full graph $K_3$).

\begin{figure}[!htb]
\center{\includegraphics[width=1.5in]{quiz-on-individual-topics/durer-graph-colored.png}}
\caption{\label{fig:durer-graph-colored} D\"{u}rer Graph colored in $3$ colors.}
\end{figure}

This graph is planar (you can draw it without intersecting edges);
it also has a famous
polyhedron (D\"{u}rer solid); it was shown in an engraving
made in 1514 called {\em Melencolia I}. See
\url{https://bit.ly/2KE2QZl}.

\vspace{10pt}
{\bf Question 6.} Answer: $0.125,0.313,0.250,0.313$\\
We write the vectors as they are multiplied with the
$4 \times 4$ matrix. The components in the vector
correspond to the pageranks (iterations $0$, $1$ and $2$).
$$v_0 =  \left( \begin{array}{c}
1/4 \\
1/4 \\
1/4 \\
1/4 \\
\end{array} \right);\;\;
v_1 = \left( \begin{array}{c}
1/8 \\
1/2 \\
1/8 \\
1/4 \\
\end{array} \right);\;\;
v_2 = \left( \begin{array}{c}
1/8 \\
5/16 \\
1/4 \\
5/16 \\
\end{array} \right).$$

\vspace{10pt}
{\bf Question 7.} Answer {\bf (D)}\\
The time complexity of such algorithm is $O(n^{\log_2 7})$ by Master's theorem. Since
$\log_2 7 \approx 2.807355 < 2.808$, we have the time complexity in $O(n^{2.808})$ (it is the best estimate that is given
among the arguments).

\vspace{10pt}
{\bf Question 8.} Answer {\bf (D)}\\
Since we can assume that one of the five strategies is optimal, it is sufficient to compare the strategies.
We assume that Players $A$ and $B$ adopt one of the five suggested strategies
plus one more simple strategy {\bf (F)} (always guess number $2$).

There are altogether $36$ variants.
For each pair of strategies we compute the payoff for the Player $A$.
We want to find the strategy for Player $A$
that beats any strategy chosen by $B$ (or at least achieves a tie).

{\bf (A)} $P(x = 1) = 1/3$, $P(x = 2) = 1/3$, $P(x = 5) = 1/3$.\\
{\bf (B)} $P(x = 1) = 0$, $P(x = 2) = 1/2$, $P(x = 5) = 1/2$.\\
{\bf (C)} $P(x = 1) = 1/2$, $P(x = 2) = 1/2$, $P(x = 5) = 0$.\\
{\bf (D)} $P(x = 1) = 1/4$, $P(x = 2) = 1/2$, $P(x = 5) = 1/4$.\\
{\bf (E)} $P(x = 1) = 2/6$, $P(x = 2) = 3/6$, $P(x = 5) = 1/6$.\\
{\bf (F)} $P(x = 1) = 0$, $P(x = 2) = 1$, $P(x = 5) = 0$.\\

The strategies of Player $A$ are shown in rows; strateges of Player $B$ are shown in columns:

\begin{tabular}{|l|c|c|c|c|c|c|} \hline
 & {\bf (A)} & {\bf (B)} & {\bf (C)} & {\bf (D)} & {\bf (E)} & {\bf (F)} \\ \hline
{\bf (A)} & $0$ & $1/6$ & $-1/6$ & $0$ & $-1/18$ & $0$ \\ \hline
{\bf (B)} & $-1/6$ & $0$ & $0$ & $0$ & $0$ & $1/2$ \\ \hline
{\bf (C)} & $1/6$ & $0$ & $0$ & $0$ & $0$ & $-1/2$ \\ \hline
{\bf (D)} & $0$ & $0$ & $0$ & $0$ & $0$ & $0$ \\ \hline
{\bf (E)} & $1/18$ & $0$ & $0$ & $0$ & $0$ & $-1/6$ \\ \hline
{\bf (F)} & $0$ & $-1/2$ & $1/2$ & $0$ & $1/6$ & $0$ \\ \hline
\end{tabular}

As you can see from the table, every strategy (except {\bf (D)})
loses against some other strategy (there is at least one negative
number on every line of the table). The only strategy that never loses
is strategy {\bf (D)}, i.e. picking the numbers $1,2,5$ with probabilities
$\frac{1}{4}, \frac{1}{2}, \frac{1}{4}$ respectively.
(In fact, this is also Nash equilibrium \textendash{} no other
probabilistic strategy fares better than this.)

Note that without the strategy {\bf (F)} we did not have enough
evidence to see that strategies {\bf (C)} and {\bf (E)}
are not optimal (because they only lose to strategy {\bf (F)}).
Moreover, the relationships between strategies are non-transitive:
\begin{itemize}
\item {\bf (A)} beats {\bf (B)}
\item {\bf (B)} beats {\bf (F)}
\item {\bf (F)} beats {\bf (C)} and {\bf (E)}
\item {\bf (C)} and {\bf (E)} beat {\bf (A)}
\end{itemize}

The Nash equilibrium {\bf (D)} does not beat anything in the list;
it does not try to exploit ``foolish'' choices of the
opponent.

\vspace{10pt}
{\bf Question 9.} Answer {\bf (C)}\\
This is known as Thue-Morse sequence. It is usually described at ``marcro-level'' - how to obtain
new iterations of this sequence (by appending a new chunk of the previous iteration, where
all letters have changed places). See \url{https://bit.ly/3aExEnq}.

But it is also possible to build Thue-Morse sequence at a ``micro-level'' (how to expand individual letters into
pairs of letters):\\
${\displaystyle \left\{ \begin{array}{l}
\mathtt{A} \rightarrow \mathtt{AB}\\
\mathtt{B} \rightarrow \mathtt{BA}
\end{array} \right. }$

{\em Note.} \url{https://bit.ly/2ypZ2rI} shows other nice
Lindenmayer images that can be created from the Thue-Morse sequence.

\vspace{10pt}
{\bf Question 10.} Answer {\bf (B)}\\
The number of hash values to be computed in order to have a $\frac{1}{2} = 50\%$ chance of a hash collision
can be estimated using the square approximation. If we compute $n$ values (and each value is
a non-negative integer less than $m$), then
$$\frac{1}{2} \approx \frac{n^2}{m};\;\;n \approx \sqrt{m}.$$
If we have $100$-bit hash values, then $m = 2^{100}$. And $n = \sqrt{2^{100}} = 2^{50}$.

We therefore need $n = 2^{50} \approx 1.1259 \cdot{10}^{15}$. Now divide this number to convert from
microseconds into years (divide by the number of milliseconds in a second; the number of seconds in an hour;
the number of hours in a day; a number of days in year):
$$n = 1.1259 \cdot{10}^{15} : 10^6 : 3600 : 24 : 365 \approx 35.70205.$$

Therefore the estimate is $35$ years. (For SHA-256 the estimate would be 10.8 Septillions or
about $10.8\cdot 10^{24}$ years.) The collisions of secure hash functions exist (and by the
Pigeonhole principle there should be many collisions for sufficiently short files). In fact,
SHA-256 collisions are inevitable even when the size of the input file exceeds $256$ bits (i.e.
$32$ bytes).

\vspace{10pt}
{\bf Question 11.} Answer: {\bf (C)}, {\bf (E)}, {\bf (G)}\\
\begin{itemize}
\item Hint 1 asks to consider the largest or the least frequently moved disk.
(Going to the extremes, answer {\bf (C)}).
\item Hint 2 asks what should happen right before we can move the largest disk
(Penultimate step, answer {\bf (G)}).
\item Hint 3 asks to see the symmetry in the problem (if you know how to move
three disks from peg $A$ to peg $B$, then you can also move them from peg $A$ to peg $C$,
and thus free the way for the fourth disk.
(Looking for symmetries, answer {\bf (E)}.
\end{itemize}

Other strategies are not directly used in the hints. Learning how to move
three disks (before doing it with four disks) would mean making it easier (answer {\bf (F)}),
but it is already done by the learner.

\vspace{10pt}
{\bf Question 12.} Answer:\\
{\tt a787b563a07713fad9b68fb1d1370f5e}

It can be computed from the command-line:
\begin{verbatim}
md5sum hosts.txt
\end{verbatim}

\vspace{10pt}
{\bf Question 13.} Answer: $6$\\

We can compute the best moves moving bottom up, starting with the
leaves and computing the best payoff in every internal node (see Figure~\ref{fig:minimax2}).

\begin{figure}[!htb]
\center{\includegraphics[width=3in]{quiz-on-individual-topics/minimax2.png}}
\caption{\label{fig:minimax2} Minimax in a tree.}
\end{figure}

\vspace{10pt}
{\bf Question 14.} Answer: $6,2$\\
There are $N=6$ steps to go from $P_0$ to $P_6$.
After that there is a periodic repeat of positions $P_6$ and $P_7$
with period $T = 2$ (see Figure~\ref{fig:conway-line5}).

\begin{figure}[!htb]
\center{\includegraphics[width=3in]{quiz-on-individual-topics/conway-line5.png}}
\caption{\label{fig:conway-line5} Conway Positions.}
\end{figure}

\vspace{10pt}
{\bf Question 15.} Answer: {\bf (B)}\\

A theory which can prove a statement and its negation is called {\bf not consistent}.
(On the contrary, theories where this can never happen are called {\bf consistent}).

BTW the first G\"{o}del's Incompleteness Theorem states that any theory about integer numbers that
is effectively axiomatizable and consistent should also be incomplete
(i.e.\ there are true results that cannot be proven). So the theory $\mathcal{T}$ from our problem has a chance to
be complete. But being inconsistent makes it completely useless (if a statement and its negation are
both provable, then virtually anything can be proven, since
both provable statements $A$ and $\neg A$ also imply
$(A \wedge \neq A) = \mathtt{false}$. But
such {\tt false} theorem would imply anything \textendash{} whether it makes sense or not.

\vspace{10pt}
{\bf Question 16.} Answer: $4$\\
The full 19-digit credit card number is this:
$$\mathtt{557367054456450571}\textcolor{red}{\mathtt{4}}.$$

See \url{https://bit.ly/2Y6NaWv} for online Luhn check.
The procedure is as follows:

\begin{itemize}
\item Drop the last digit from the number. (It is initially unknown in our case.)
\item Reverse the digits.
\item Multiply the digits in odd positions ($1$, $3$, $5$, etc.) by $2$ and subtract $9$ to all any result higher than $9$.
\item Add all the obtained numbers together
\item The last (unknown) number is the amount that you would need to add to get a multiple of 10.
\end{itemize}

\begin{verbatim}
5,5,7,3,6,7,0,5,4,4,5,6,4,5,0,5,7,1
1,7,5,0,5,4,6,5,4,4,5,0,7,6,3,7,5,5
2,7,10,0,10,4,12,5,8,4,10,0,14,6,6,7,10,5
2,7,1,0,1,4,3,5,8,4,1,0,5,6,6,7,1,5
\end{verbatim}

The sum of all digits on the last line is $66$. By adding digit $4$ this number becomes divisible by $10$.

\vspace{10pt}
{\bf Question 17.} Answer {\bf (A)}\\

We should apply our intuition about natural language texts.
If there are many identical sequences of five consecutive words, then they
cannot appear by pure chance (there might be some common proverbs or short quotes,
but they would not make $10\%$ overlap.

Answer {\bf (B)} is not credible \textendash{} if random lookups in the hashtable lead to $5\%$ matching,
then $10\%$ overlap cannot be explained by chance.\\
Answer {\bf (C)} is not applicable either; rolling hash on words (especially, if it detects considerable
number of matches) is more useful than rolling hash on characters \textendash{} matching the characters
is less robust and the hash windows are typically shorter.

\vspace{10pt}
{\bf Question 18.} Answer $30$\\
Truncated icosahedron (soccer ball graph) has the following number of
faces, edges and vertices:
$$F = 32, E = 90, V = 60.$$

Since Hamilton graph visits all $60$ vertices,
it should also have $60$ edges.
For this reason, there are $E - V = 90 - 60 = 30$ edges
that are not part of the Hamiltonian cycle.

\end{document}


\section{Conclusions and Future Directions}

ITK is a large and powerful framework for medical (and general) image processing.  However, the lack of filters for understanding, manipulating, and unwrapping phase data is a current limitation of the library.  We have here presented an ITK module which we hope will begin to bridge that gap.  The two most significant contributions are the \code{itk::QualityGuidedPhaseUnwrappingImageFilter} and \code{itk::DCTPhaseUnwrappingImageFilter} classes, which implement efficient $n$-dimensional unwrapping algorithms.  The quality-guided approach has the advantage of giving a result that is congruent to the input.  Moreover, this approach avoids low-quality phase data, given an adequate quality map.  The unweighted $L^2$-norm approach has the advantage of giving a smooth result throughout, but is not congruent with the input and weights all pixels equally regardless of quality.

These algorithms both gave servicable results when presented with the SWI data, which had few residues within the region of interest, and in which the low quality data was largely relegated to the periphery.  However, both algorithms failed to produce an adequate result when presented with the more difficult HARP image.  In the case of the quality-guided approach, this is likely due to the inadequacy of phase derivative variance as a quality map, because in HARP images phase varies quite smoothly even in regions where there is little to no signal.  In the case of the DCT algorithm, this is likely because the region of interest is relatively small compared to the image as a whole.

In the future, it would be of great benefit to allow for other quality maps (such as maximum phase gradient, pseudocorrelation, and user-defined masks) in addition to phase derivative variance.  This would allow for finer control over the path the algorithm takes in the case of difficult cases such as HARP images.  Additionally, it would be of benefit to implement a weighted $L^2$-norm method, so that the DCT approach could also exclude low-quality or uninteresting regions.  Weighted $L^2$-norm phase unwrapping algorithms have been described which iteratively apply unweighted algorithms to weighted wrapped phase Laplacians.  The preconditioned conjugate gradient (PCG) approach in particular makes use of this method \cite{Ghiglia1998}, and would be an important next step in the development of this module.

This submission has also described \code{itk::DCTImageFilter} and \code{itk::DCTPoissonSolverImageFilter}, which are efficient implementations of general-purpose utilities important in image compression, gradient image editing, and phase unwrapping.  The former is a simple wrapper to the FFTW library, allowing for the discrete cosine transform to be integrated into an ITK pipeline.  The latter makes use of the DCT class to recover an image from its Laplacian.  We refer the interested reader to the appendices for a proper discussion.