\section{Method}
\label{sec:method}

This section describes our approach to solving the page rank estimation problem. We present our methodology as we have applied it to the problem at hand, as opposed to extracting the generic, underlying principles, as these are mostly covered in the background section of this document.

The page rank estimation problem is, given a web page represented as an attributed graph with screenshots, to estimate the global popularity of that web page.
Our ground-truth for \textit{popularity} is a list containing 100,000 domains, created by Open PageRank.
More details can be found in Section~\ref{OpenPageRank}.
The given data has several characteristics which have influenced our way of tackling the problem:

\begin{itemize}
    \item With approximately 100,000 samples, the dataset is relatively large. Each sample, i.e. a domain, is made up of up to 16 images (eight desktop and eight mobile screenshots).
    \item The problem is a ranking task. Samples need to be ordered and during inference the rank of a (possibly) unseen sample must be predicted. Besides the web page ranking list (containing rank and domain) there is no other information available.
    \item Images make up the majority of the information contained in the dataset. NNs, in particular CNNs \cite{lecun1989backpropagation} are performing well at processing images and at extracting features from them. With \cite{krizhevsky:imagenet} they have become the de-facto standard way of solving image classification problems wherever large datasets are available.
    \item The samples are variably-sized and can be easily represented as a graph: The number of screenshots taken from a domain may vary depending on the number of subpages. Links between subpages can be represented as edges in a graph. Naturally, the samples can be seen as graphs and a model capable of using the entire information from the graph seems best suited.
    \item The data is relatively inconsistent, e.g. samples might be mostly black or ranks might be inaccurate, as the global page ranking list is subject to change over time.
\end{itemize}

Motivated by the stated observations, we use a combination of CNN and GN, trained with ranking loss functions, to predict page ranks.

Learning features as opposed to hand-engineering them seems reasonable, because it is unknown to us what the visual elements are that distinguish popular and unpopular web pages and whether they exist at all. Until this point it is unknown to us, whether it is feasible to predict a domain's rank based on the visual appearance of its web pages. A dataset containing more than 1,000,000 images provides enough training data to work with high-capacity, deep models, capable of learning complex features.

CNNs are the most common choice for image processing in the deep learning domain. While many of today's CNN-papers deal with natural images, e.g. \cite{gu2018ava,szegedy2017inception,openai:learningdexterity}, there have been applications with computer generated images as well: \cite{beltramelli:pix2code} converts a screenshot into a fixed size feature vector to infer the design layout of given user interface from a screenshot. This work as well as the general attention CNNs get indicate that they are a suitable choice for web page screenshot processing and feature extraction.

GNs are designed to process graphs, possibly ones where a node's attribute is an image. GNs naturally scale to graphs of arbitrary size and type of content present in the graph. While processing the graph, information can flow between nodes (message passing). While we focus on visual information (screenshots) of websites, future extensions could augment nodes with textual or other data as well.

Ranking problems are a well studied field and losses for different setups have been defined in previous works. We incorporate in particular the probabilistic ranking loss by \cite{Burges:learningtorankwithsgd} into our model.

We define an accuracy metric that allows us to evaluate the performance of our model. It measures how accurately the model ranks any two given web pages from a subset of the dataset relative to each other. This accuracy metric is also the primary way of comparing models to each other, e.g. for hyperparameter tuning.

While our dataset contains web pages from domains which are (or have been) publicly available on the web, we want it to work with unseen inputs as well. The goal of our model's inference mode is to either estimate the absolute popularity of a web page with respect to the top 100,000 web pages or, secondly, compare two (or more) given web pages relative to one and another.

\begin{figure}
    \centering
    \includegraphics[scale=0.64]{resources/model-architecture}
    \caption[Model architecture]{Web page rank estimation model architecture. Input to the model is a graph representing the web pages of a domain (1). Each node is attributed with two screenshots and the nodes are interconnected by web links. A screenshot feature extractor CNN is (2) applied node-wise, thereby converting the screenshot graph into a graph with feature vectors (3). The feature vector graph is the input to a (4) GN which looks at the graph as a whole (as opposed to node-wise processing). It reduces the graph to a single scalar (5) from which the relative ranking can be inferred.
    The internals of module (2) are explained in Section~\ref{sec:screenshotfeatextr}; the ones of (4) in Section~\ref{sec:gnblocks}.}
    \label{fig:methodcomponents}
\end{figure}

The remainder of this section describes the components of our model in greater detail; Figure~\ref{fig:methodcomponents} gives an overview of the model's internal processing pipeline. We also introduce our loss function and the evaluation metric we use to measure the performance of our model. The dataset is described separately in Section~\ref{sec:datasets}.

\subsection{Screenshot Feature Extractor}
\label{sec:screenshotfeatextr}

The screenshot feature extractor is an integral component of the model. Its task is to extract a vector of fixed size from a given website screenshot. The vector is supposed to contain a condensed version of the information contained in the original image. Condensed with respect to the specific task the feature extractor is trained on.

The idea of feature extraction is a reoccurring scheme in the field of deep learning. Recurrent neural networks such as LSTMs (\cite{hochreiter1997lstm}) can extract a fixed size vector from a variable length sequence, used e.g. to map a sentence into a representation space \cite{salesforceabstractivesummarization}. Similarly, CNNs are used to extract features from images: It is a common architectural choice to classify an image with a CNN, the feature extraction, followed by fully connected layers, referred to as classification head, see e.g. \cite{krizhevsky:imagenet,Girshick15:fastrcnn,szegedy2017inception}. We borrow this idea and extract feature vectors from screenshots. The extracted features serve as input to the following stage, a GN.

Mathematically, a screenshot feature extractor is a function mapping a screenshot into a smaller vector space of dimensionality $d_\text{featvec}$: $f:\mathbb{R}^{480\times270\times3}\rightarrow\mathbb{R}^{d_\text{featvec}}$. We are working with two different types of screenshots, namely desktop and mobile web page screenshots. The screenshots are similar in that they contain similar content (a website), however, potentially of different size. Also, mobile screenshots might have a different look overall, commonly having a menu drawer in the corner and a single-column layout. We decide to process a tuple of corresponding mobile and desktop screenshots with a single function to share some of the function's parameters. Specifically, the employed feature extractor is a function \begin{equation}
f_\text{dm}:\left(\mathbb{R}^{480\times270\times3},\mathbb{R}^{187\times333\times3}\right)\rightarrow\left(\mathbb{R}^{d_\text{featvec}},\mathbb{R}^{d_\text{featvec}}\right)\,,
\end{equation} where \enquote{dm} stands for desktop-mobile. The input tensors are two screenshots (desktop and mobile respectively), with three color channels. The output is a tuple consisting of two vectors. They are supposed to be mappings of the screenshot into a $d_\text{featvec}$-dimensional feature space.

\begin{figure}
    \centering
    \begin{tikzpicture}[>=stealth, thick]
    \node (I_d) at (2,-16) [draw, process, align=flush center]
        {Desktop screenshot\\$\tens{I}_\text{d}\in\mathbb{R}^{480\times270\times3}$};

    \node (I_m) at (6,-16) [draw, process, align=flush center]
        {Mobile screenshot\\$\tens{I}_\text{m}\in\mathbb{R}^{187\times333\times3}$};

    \node (block1_d) at (2,-14) [draw, double, align=flush center]
    {Block1\_d\\$n_\text{filters}=32$};

    \node (block1_m) at (6,-14) [draw, double, align=flush center]
    {Block1\_m\\$n_\text{filters}=32$};

    \node (block2_d) at (2,-12) [draw, double, align=flush center]
    {Block2\\$n_\text{filters}=64$};
    \node (block2_m) at (6,-12) [draw, double, align=flush center]
    {Block2\\$n_\text{filters}=64$};

    \node (block3_d) at (2,-10) [draw, double, align=flush center]
    {Block3\\$n_\text{filters}=128$};
    \node (block3_m) at (6,-10) [draw, double, align=flush center]
    {Block3\\$n_\text{filters}=128$};

    \node (block4_d) at (2,-8) [draw, double, align=flush center]
    {Block4\\$n_\text{filters}=256$};
    \node (block4_m) at (6,-8) [draw, double, align=flush center]
    {Block4\\$n_\text{filters}=256$};

    \node (dense_d) at (2,-6) [draw, double, align=flush center]
    {Dense\\$n_\text{units}=256$};
    \node (dense_m) at (6,-6) [draw, double, align=flush center]
    {Dense\\$n_\text{units}=256$};

    \node (dense2_d) at (2,-4) [draw, double, align=flush center]
    {Dense\_d\\$n_\text{units}=d_\text{featvec}$};
    \node (dense2_m) at (6,-4) [draw, double, align=flush center]
    {Dense\_m\\$n_\text{units}=d_\text{featvec}$};

    \node (feat_d) at (2,-2) [draw, process, align=flush center]
        {Desktop feature vector\\$\bm{x}_\text{d}\in\mathbb{R}^{d_\text{featvec}}$};

    \node (feat_m) at (6,-2) [draw, process, align=flush center]
        {Mobile feature vector\\$\bm{x}_\text{m}\in\mathbb{R}^{d_\text{featvec}}$};

    \draw[->] (I_d) -- (block1_d);
    \draw[->] (I_m) -- (block1_m);

    \draw[->] (block1_d) -- (block2_d);
    \draw[->] (block1_m) -- (block2_m);
    \draw[dashed,<->] (block2_m) -- (block2_d);

    \draw[->] (block2_d) -- (block3_d);
    \draw[->] (block2_m) -- (block3_m);
    \draw[dashed,<->] (block3_m) -- (block3_d);

    \draw[->] (block3_d) -- (block4_d);
    \draw[->] (block3_m) -- (block4_m);
    \draw[dashed,<->] (block4_m) -- (block4_d);

    \draw[->] (block4_d) -- (dense_d);
    \draw[->] (block4_m) -- (dense_m);
    \draw[dashed,<->] (dense_m) -- (dense_d);

    \draw[->] (dense_d) -- (dense2_d);
    \draw[->] (dense_m) -- (dense2_m);

    \draw[->] (dense2_d) -- (feat_d);
    \draw[->] (dense2_m) -- (feat_m);

    \end{tikzpicture}
    \caption[Feature extractor]{Feature extractor $f_\text{dm}$; inputs are at the bottom, outputs at the top. Blocks with double borders are parameterized, dashed arrows indicate parameter sharing, normal arrows information flow. The weight sharing between inner layers allows for similar, yet different feature extraction because the weight matrices of the first convolutional block and the last dense block are not shared.}
    \label{fig:featextr}
\end{figure}

We model the function $f_\text{dm}$ with a CNN. The high-level architecture is described and visualized in Figure~\ref{fig:featextr}. Each \textbf{Block} consists of the following components:
\begin{enumerate}
    \item 2D convolutional layer with $n_\text{filters}$ filters, preserving the spatial size of the tensor and increasing the number of channels.
    \item ReLU activation function, $\operatorname{ReLU}(x)=\max(0,x)$.
    \item Second 2D convolutional layer with $n_\text{filters}$ filters, preserving the size of the tensor along all dimensions.
    \item ReLU activation function.
    \item 2D max-pooling with a kernel size of $(3,3)$. This reduces the spatial size of the tensor by a factor of $3$ along both, width and height.
    \item 2D dropout \cite{srivastava2014:dropout} with drop probability $p_\text{drop}$, enabled during training, disabled during evaluation and inference.
\end{enumerate}
Exceptions to this are the first block which has a max-pooling size of $(2,2)$ and the fourth block which is followed by global average pooling. These deviations from \cite{beltramelli:pix2code} are made to account our specific screenshot dimensionality which varies from desktop to mobile.

The \textbf{Dense} blocks each contain a fully connected layer followed by ReLU activation and dropout. The last layer outputs the raw result of the matrix multiplication, without activation or dropout.

The parameters of the feature extraction model can be trained with backpropagation. The following modes of training are conceivable:
\begin{itemize}
    \item \textbf{End-to-end training [e2e]}: The feature extractor is trained alongside with the other components of the model, gradients are computed by backpropagating through the following components. This method will train the CNN such that the extracted vectors contain features most helpful/relevant for the overall task. The downside is the additional computational complexity introduced by backpropagating through a GN.
    \item \textbf{Pre-training without fine-tuning [woft]}: The weights of the feature extractor are learnt on a separate auxiliary task and transferred to be used for the actual problem. This method requires an auxiliary task, which (in this setting) may be the prediction of a page rank based solely on a single desktop-mobile screenshot tuple. If the model parameters are kept frozen, i.e. are not being fine-tuned, the feature vectors for the dataset can be cached, once the entire model is trained on the main task. That way the computational overhead of passing images through the CNN can be omitted, greatly increasing the training speed and decreasing GPU memory requirements. Even though the described auxiliary task is similar to the page rank prediction based on an entire graph of screenshots, its difference may lead to slightly less meaningful feature vectors.
    \item \textbf{Pre-training with fine-tuning [wft]}: Same as \textit{pre-training without fine-tuning} except the weights are kept trainable and are fine-tuned on the main task as well. Caching of feature vectors is not possible.
\end{itemize}

\subsection{Graph Network Blocks}
\label{sec:gnblocks}

After converting the graph of screenshots into a graph of feature vectors, the entire graph is fed into the GN; component (4) in Figure~\ref{fig:methodcomponents}. The GN module is an integral component of our model: It utilizes the graph structure of the dataset samples and outputs the actual model prediction, thereby converting a graph into a continuous scalar, interpretable as a rank estimation.

The GN consists of several GN blocks, which were introduced in \cite{deepmind:graphnets}. A summary of the most important aspects as well as the notation we use in the following can be found in Section~\ref{sec:graphnetworks}.

Formally, our GN module converts a graph into a scalar, it is a function \begin{equation}
    \label{eq:gnmodule}
    f_\text{gn}:\mathbb{G}\rightarrow\mathbb{R}\,,
\end{equation} where $\mathbb{G}$ denotes the set of all possible graphs. The input graph represents a website and consists of up to eight nodes. Each node contains two feature vectors which are the screenshot feature extractor's output. The directed edges point from nodes to nodes which contain hyperlinks. Initially, the edges are (opposed to the nodes which contain feature vectors) not attributed.

On a semantic level the GN module consists of three different types of blocks: encoder, core, and decoder. For each of these blocks variants can be used but the role remains the same. This idea is dubbed \textit{encode-process-decode} and depicted in Figure~\ref{fig:gn-enc-proc-dec}.

Our \textbf{encoder block} $\text{GN}_\text{enc}$ converts the raw input graph into a homogeneous graph where nodes, edges, and global state are all attributed with a single vector. In our case this is achieved by defining the following update and aggregation functions for the encoder:\begin{align}
    \phi^e_\text{enc}\left(\bm{e}_k,\bm{v}_{r_k},\bm{v}_{s_k},\bm{u}\right)=&\bm{v}_{s_k}\label{eq:enc:edgeupdate}\\
    \rho^{e\rightarrow u}_\text{enc}\left(\mathbb{E}'\right)_j=&\operatorname{avg}\left(\left\{x_j:\bm{x}\in\mathbb{E}'\right\}\right)\label{eq:enc:edgeaggreforu}\\
    \phi^v_\text{enc}\left(\bm{\overline{e}}'_i,\bm{v}_i,\bm{u}\right)=&\bm{v}_i\label{eq:enc:nodeupdate}\\
    \phi^u_\text{enc}\left(\bm{\overline{e}}',\bm{\overline{v}},\bm{u}\right)=&\bm{\overline{e}}'\label{eq:enc:uupdate}\,,
\end{align}where $\rho^{e\rightarrow v}$ and $\rho^{v\rightarrow u}$ remain undefined because they are not needed. The edges are initially populated with the feature vector of the sender node (Equation~\ref{eq:enc:edgeupdate}). Equation~\ref{eq:enc:edgeaggreforu}, the edge aggregation for the global state, is an element-wise averaging of the edge vectors. To ensure there is at least one element in $\mathbb{E}'$ we add reflexive edges to the graph so every node has at least one outgoing (and incoming) edge, namely the one pointing to itself. Equation~\ref{eq:enc:nodeupdate} is an identity mapping, thus the node attributes remain unchanged. The global state is set to be the aggregation of all edges (Equation~\ref{eq:enc:uupdate}). Alternatively, the node aggregation $\bm{\overline{v}}$ could be chosen; both options are equally valid and we did not perform experiments comparing both options within the scope of this work.

Intuitively, the encoder block can be seen as a preparation stage. It transforms the graph into the desired state and adds missing attributes. In fact, the entire feature extractor can be seen as an encoder block, too: It is a node-wise application of the CNN: $\phi^v_\text{dm}\left(\bm{\overline{e}}'_i,\bm{v}_i,\bm{u}\right)=f_\text{dm}(\bm{v}_i)$, where $\bm{v}_i$ is a tuple made up of desktop and mobile screenshot. We chose not to formulate the overall model that way to aid understanding, but it illustrates the expressive power of the GN framework.

Before applying the encoder block we concatenate desktop and mobile feature vectors so node attributes are a single vector $\bm{v}=\begin{bmatrix}\bm{x}_\text{d} & \bm{x}_\text{m}\end{bmatrix}$.

Our \textbf{core block} $\text{GN}_\text{core}$ performs the graph \textit{understanding} part by performing the \textit{message passing} (see Section 4.3 in \cite{deepmind:graphnets}). It may be stacked $M$ times.

We choose the aggregation functions $\rho^{e\rightarrow v}\left(\mathbb{E}'_i\right)$, $\rho^{e\rightarrow u}\left(\mathbb{E}'\right)$, $\rho^{v\rightarrow u}\left(\mathbb{V}'\right)$ to average the vectors element-wise, analogous to Equation~\ref{eq:enc:edgeaggreforu}. Alternatively, max-pooling could be used; we did not perform experiments comparing both options within the scope of this work.

The update functions for node attributes, edge attributes, and the global state are fully-connected layers with bias, followed by a ReLU activation function ($\operatorname{ReLU}(x) = \max(0,x)$) and dropout (see \cite{srivastava2014:dropout}; denoted by the function $\operatorname{drop}(\cdot)$). The exact definitions are the following:
\begin{align}
    \phi^e_\text{core}\left(\bm{e}_k,\bm{v}_{r_k},\bm{v}_{s_k},\bm{u}\right)=&
        \operatorname{drop}\left(\operatorname{ReLU}\left(
            \bm{W}_e\begin{bmatrix}\bm{e}_k & \bm{v}_{r_k} & \bm{v}_{s_k} & \bm{u}\end{bmatrix}^\mathsf{T}+\bm{b}_e
        \right)\right)\\
    \phi^v_\text{core}\left(\bm{\overline{e}}'_i,\bm{v}_i,\bm{u}\right)=&
        \operatorname{drop}\left(\operatorname{ReLU}\left(
            \bm{W}_v\begin{bmatrix}\bm{\overline{e}}'_i & \bm{v}_i & \bm{u}\end{bmatrix}^\mathsf{T}+\bm{b}_v
        \right)\right)\\
    \phi^u_\text{core}\left(\bm{\overline{e}}',\bm{\overline{v}},\bm{u}\right)=&
        \operatorname{drop}\left(\operatorname{ReLU}\left(
            \bm{W}_u\begin{bmatrix}\bm{\overline{e}}' & \bm{\overline{v}} & \bm{u}\end{bmatrix}^\mathsf{T}+\bm{b}_u
        \right)\right)\,,
\end{align}where $\begin{bmatrix}\cdot\end{bmatrix}$ denotes concatenation into a row vector. The core block can be repeated $M$ times. The trainable weight matrices $\bm{W}$ and biases $\bm{b}$ may either be shared between the $M$ core block replicas or different matrices/vectors can be used.

Our \textbf{decoder block} $\text{GN}_\text{dec}$ converts the graph returned from the last core block into the desired output. In our case the output of the GN module is a scalar (see Equation~\ref{eq:gnmodule}). We extract a scalar from the graph by muliplying the global state vector $u$ of the last core block with a weight vector of matching size and add a bias:
\begin{equation}
    \label{eq:dec:nodeupdate}
    \phi^u_\text{dec}\left(\bm{\overline{e}}',\bm{\overline{v}},\bm{u}\right)=\bm{w}_\text{dec}\cdot\bm{u}+\bm{b}_\text{dec}\,.
\end{equation}
The trainable parameters are $\bm{w}_\text{dec}$ and $\bm{b}_\text{dec}$.

Intuitively, the global state can be understood as an aggregation of the information contained in the graph. This is an elegant solution to the problem of converting a set into a vector, which must be solved when converting a graph of variable size into a single scalar.

\subsection{Loss Function}
\label{sec:loss}

Our model is trained with gradient descent which requires the definition of a loss function. We choose a probabilistic loss function, suitable for ranking problems, that is heavily inspired by \cite{Burges:learningtorankwithsgd}. We recap their approach as well as more information on ways of tackling ranking problems in our Section \textit{Learning to Rank} (\ref{sec:learningtorank}).

Let $\mathbb{D}=\left\{\left(G^{(i)},r^{(i)}\right)\right\}_{i=1}^N$ denote a dataset consisting of $N\ge2$ graphs and corresponding ranks.
Let $f_\text{model}(G)$ be the model function mapping a graph to a scalar in $\mathbb{R}$ (visualized in Figure \ref{fig:methodcomponents}).
$\bm{f}\in\mathbb{R}^N$ is the vector induced by applying the model function to the entire dataset:
\begin{equation}
    f_i=f_\text{model}\left(G^{(i)}\right)\,.
\end{equation}

Given the samples with indices $i$ and $j$, we interpret the model output as follows:
\begin{align}
    f_i>f_j&\implies G^{(i)}\triangleright G^{(j)}&G^{(i)}\text{ is higher ranked,}\\
    f_i<f_j&\implies G^{(i)}\triangleleft G^{(j)}&G^{(j)}\text{ is higher ranked,}\\
    f_i=f_j&\implies G^{(i)}\equiv G^{(j)}&\text{equal rank.}
\end{align}

All pairwise predictions (how the model ranks any two samples relative to each other) are stored in the matrix $\bm{O}\in\mathbb{R}^{N\times N}$, whose entries are defined as
\begin{equation}\label{eq:outputmatrix}
    O_{i,j}=f_i-f_j\,.
\end{equation}

The ground truth matrix $\bm{P}\in\mathbb{R}^{N\times N}$ can be derived from the data. $\bm{P}$ stands for probability. Its entries\begin{equation}\label{eq:groundtruthmatrix}
    P_{i,j}=\begin{cases}
        0&\text{if }r^{(i)}\triangleleft r^{(j)}\\
        0.5&\text{if }r^{(i)}\equiv r^{(j)}\\
        1&\text{if }r^{(i)}\triangleright r^{(j)}
    \end{cases}
\end{equation}indicate the probability of $r^{(i)}$ to be a higher rank (e.g.~\#5) than $r^{(j)}$ (e.g.~\#100). Note that a greater rank number is a lower rank; the highest rank is~\#1. This can be confusing from time to time, we use the wording of a \textit{higher rank} to be a lower number, which is a higher position in the ranking.

The loss is defined as
\begin{equation}
    \label{eq:loss}
    \frac{1}{N^2}\sum_{i=1}^N\left(
        w_b\left(r^{(i)}\right)\sum_{j=1}^N
            \underbrace{\left(-P_{i,j}O_{i,j}+\log\left(1+e^{O_{i,j}}\right)\right)}_{C_{i,j}}
    \right)\,,
\end{equation}where $w_b:\mathbb{N}\rightarrow\mathbb{R}$ is a rank-dependent weighting function, which is explained below. The loss term can be understood as a computation of the pairwise loss, denoted by $C_{i,j}$, for each pair of samples in the dataset. The leading factor $\frac{1}{N^2}$ normalizes so the number of samples $N$ does not affect the loss magnitude.

The loss asymptotes to a linear function. According to \cite{Burges:learningtorankwithsgd}, this is likely to be more robust with noisy labels than a quadratic cost. This behavior is illustrated in Figure~\ref{fig:costfn}.

We introduce the additional weighting factor
\begin{equation}\label{eq:weightingfactor}
    w_b(r)=1-\left(\frac{\log\left(r\times r_\text{max}\right)}{\log r_\text{max}}-1\right)^b\,,
\end{equation}which counteracts the imbalance of samples for high and low ranks: The first 10,000 samples are presumably more distinctive than e.g. the samples from 80,000 to 90,000. Therefore the samples with high ranks should also have a higher contribution to the loss. The hyperparameter $b$ balances this favoring. $r_\text{max}$ is the maximum rank in the dataset, in our case $r_\text{max}=100,000$. Equation~\ref{eq:weightingfactor} is plotted for several values of $b$ in Figure~\ref{fig:weightingfactor}.

\begin{figure}
\begin{minipage}[t]{.45\textwidth}
  \centering
  \resizebox {\columnwidth} {!} {
  \begin{tikzpicture}

    \pgfplotsset{
        scale only axis,
    }

    \begin{axis}[
        xlabel=$O_{i,j}$,
        ylabel=$C_{i,j}$,
        samples=100,
        xmin=-5,xmax=5,
        ymin=0
        ]
        \addplot[style=dotted][domain=-5:5]{-0*x+ln(1+e^x)};
        \addlegendentry{$P_{i,j}=0$}
        \addplot[style=dashed][domain=-5:5]{-0.5*x+ln(1+e^x)};
        \addlegendentry{$P_{i,j}=0.5$}
        \addplot[][domain=-5:5]{-1*x+ln(1+e^x)};
        \addlegendentry{$P_{i,j}=1$}
    \end{axis}

    \end{tikzpicture}
  }
  \captionof{figure}[Loss function plotted for values of the target probability]{The loss function plotted for three different values of the target probability \cite{Burges:learningtorankwithsgd}. The curves asymptote to linear functions.}
  \label{fig:costfn}
\end{minipage}\hfill%
%
\begin{minipage}[t]{.45\textwidth}
    \centering
    \resizebox {\columnwidth} {!} {
    \begin{tikzpicture}

      \pgfplotsset{
          scale only axis,
      }

      \begin{axis}[
          xlabel=$r$,
          ylabel=$w_b(r)$,
          samples=500,
          ]
          \addplot[][domain=1:100000]{1-pow(ln(x*100000) / ln(100000) - 1., 1)};
          \addlegendentry{$b=1$}
          \addplot[style=dashed][domain=1:100000]{
              1-pow(ln(x*100000) / ln(100000) - 1., 2)};
          \addlegendentry{$b=2$}
          \addplot[style=dotted][domain=1:100000]{1-pow(ln(x*100000) / ln(100000) - 1., 10)};
          \addlegendentry{$b=10$}
          \addplot[dash pattern=on 1pt off 3pt on 3pt off 3pt][domain=1:100000]{1-pow(ln(x*100000) / ln(100000) - 1., 100)};
          \addlegendentry{$b=100$}
      \end{axis}

      \end{tikzpicture}
    }
    \captionof{figure}[Weighting factor $w_b$ plotted for several values of $b$]{Weighting factor $w_b$ plotted for several values of $b$.}
    \label{fig:weightingfactor}
  \end{minipage}%
\end{figure}

It is noteworthy that a change of $b$ will also affect the average magnitude of the loss and thereby the effective step size. When comparing different weighting factors with each other, the loss term should therefore be divided by
\begin{equation}\label{eq:losswnormalization}
\frac{2}{r_\text{max}}\int_1^{r_\text{max}}w_b(r)dr\,,
\end{equation}
to keep the lost magnitude consistent. The term evaluates to $\approx0.95$ for $b=10$ and $\approx0.17$ for $b=1$. Intuitively, the loss must be scaled up more (divided by a lower value) if the area under curve is lower. For $b=1$ it is lower than it is e.g. for $b=10$ (see Figure~\ref{fig:weightingfactor}).

During training, the gradient induced by the loss term (Equation~\ref{eq:loss}) can be approximated with fewer than $N$ samples, so called mini-batches. This is less costly than computing the gradient for the exact problem and has a regularizing effect \cite{DBLP:journals/corr/KeskarMNST16}.

Note that this is single scalar supervision (or binary classification) and it effectively trains a CNN plus GN successfully.

\subsection{Accuracy}
\label{sec:accuracy}

During training we evaluate the performance of our model by computing an accuracy metric. Given all possible $N\times N$ pairs of samples in the dataset, the accuracy is the percentage of pairs the model ranks correctly. It is defined as \begin{equation}\label{eq:acc}
    \frac{1}{N^2-N}\left(\sum_{i=1}^N\sum_{j=1}^N\bm{1}\left[\left(O_{i,j}<0\land P_{i,j}=0\right)\lor\left(O_{i,j}>0\land P_{i,j}=1\right)\right]\right)\,,
\end{equation}with $\bm{O}$ being the output matrix (defined in Equation~\ref{eq:outputmatrix}) and $\bm{P}$ being the ground truth (defined in Equation~\ref{eq:groundtruthmatrix}). $\bm{1}\left[\cdot\right]$ is $1$ if the boolean expression enclosed in the brackets is true, $0$ otherwise. The additional $(-N)$ term in the denominator of the leading normalization factor accounts for the main diagonal of the matrix which is always correctly ranked because $O_{i,i}=0$, so we disregard it.

For performance purposes the symmetric property of the matrix may be exploited. The evaluation of the lower triangular part suffices to compute the accuracy score with adjusted normalization factor.

Verbally described, the accuracy measure is the percentage of correct pairwise rankings. It is easy to interpret and can be computed for humans as well, by showing them the sample pairs and requesting a ranking.

The accuracy evaluation metric has the downside of not taking the difficulty into consideration: Ranking e.g. sample \#55,000 relative to sample \#54,590 is presumably much harder than samples \#100 and \#40,000, because the sample ranks are closer together.

\subsection{Inference}

Inference of the presented model can be done in two ways: (1) absolute and (2) relative. In our experiments we do not need the rank inference mode (1) because the accuracy on validation and test set can be computed with Equation~\ref{eq:acc}. We present both modes anyways for use in future research or in applications. The two methods differ in that (1) requires the ground truth dataset $\mathbb{D}$, whereas (2) works with any set of two or more websites.

\begin{enumerate}
    \item[(1)] A single sample $G$ can be ranked with respect to the samples of the dataset $\mathbb{D}$. This can be done by computing $f_G=f_\text{model}(G)$ and comparing it to all entries in the vector $\bm{f}$. The estimated rank of $G$ is \begin{equation}
        \label{eq:inference}
        r^{(G)}\approx N-\sum_{i=1}^N\bm{1}\left[f_G>f_i\right]\,.
    \end{equation}
    \item[(2)] The relative inference compares a set of samples $\mathbb{S}$ relative to each other, where $\left\lvert\mathbb{S}\right\rvert\ge2$. Instead of computing the matrix $\bm{O}$ for samples in the dataset we compute it for $\mathbb{S}$. The entry at $O_{i,j}$ indicates how the samples with index $i$ and $j$ are ranked relative to each other.
\end{enumerate}


\documentclass[a4paper,10pt]{article}
\usepackage{graphicx}
\usepackage{lscape}
\title{Output tables for 1xN statistical comparisons.}
\author{}
\date{\today}
\begin{document}
\begin{landscape}
\pagestyle{empty}
\maketitle
\thispagestyle{empty}

\section{Average rankings of Friedman test}

Average ranks obtained by each method in the Friedman test.

\begin{table}[!htp]
\centering
\begin{tabular}{|c|c|}\hline
Algorithm&Ranking\\\hline
features&4.4194\\voting&3.2742\\bagging&4.8548\\svm-lin&4.5161\\svm-rbf&3.2258\\knn3&3.2258\\knn5&6.4032\\mlp&6.0806\\\hline\end{tabular}
\caption{Average Rankings of the algorithms (Friedman)}
\end{table}

Friedman statistic (distributed according to chi-square with 7 degrees of freedom): 56.849462. \newline P-value computed by Friedman Test: 0.\newline

Iman and Davenport statistic (distributed according to F-distribution with 7 and 210 degrees of freedom): 10.649255. \newline P-value computed by Iman and Daveport Test: 0.000000000019.\newline

\newpage

\section{Post hoc comparison (Friedman)}

P-values obtained in by applying post hoc methods over the results of Friedman procedure.

\begin{table}[!htp]
\centering\footnotesize
\begin{tabular}{cccc}
$i$&algorithm&$z=(R_0 - R_i)/SE$&$p$\\
\hline7&knn5&5.106987&0\\6&mlp&4.588511&0.000004\\5&bagging&2.618303&0.008837\\4&svm-lin&2.073903&0.038088\\3&features&1.918361&0.055065\\2&voting&0.077771&0.93801\\1&knn3&0&1\\\hline
\end{tabular}
\caption{Post Hoc comparison Table for $\alpha=0.05$ (FRIEDMAN)}
\end{table}
\newpage

\section{Adjusted P-Values (Friedman)}

Adjusted P-values obtained through the application of the post hoc methods (Friedman).

\begin{table}[!htp]
\centering\small
\begin{tabular}{ccc}
i&algorithm&unadjusted $p$\\
\hline1&knn5&0\\2&mlp&0.000004\\3&bagging&0.008837\\4&svm-lin&0.038088\\5&features&0.055065\\6&voting&0.93801\\7&knn3&1\\\hline
\end{tabular}
\caption{Adjusted $p$-values (FRIEDMAN) (I)}
\end{table}
\begin{table}[!htp]
\centering\small
\begin{tabular}{ccc}
i&algorithm&unadjusted $p$\\
\hline1&knn5&0\\2&mlp&0.000004\\3&bagging&0.008837\\4&svm-lin&0.038088\\5&features&0.055065\\6&voting&0.93801\\7&knn3&1\\\hline
\end{tabular}
\caption{Adjusted $p$-values (FRIEDMAN) (II)}
\end{table}

\newpage
\end{landscape}\end{document}


% !TeX document-id = {c74a84f4-c0b2-4978-a479-47211c753752}
% !BIB TS-program = biber

\documentclass[twocolumn]{article}

\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{biblatex}
\usepackage{caption}
\usepackage{dirtytalk}
\usepackage{hyperref}
\usepackage{mathtools}
\usepackage{multicol}
\usepackage{placeins}
\usepackage{siunitx}
\usepackage{subcaption}
\usepackage{tikz}
\usepackage{url}
\usepackage{xypic}

\title{Skip Ratchet \\[1ex] \large A Hierarchical Hash System}
\author{Brooklyn Zelenka \\ \href{https://fission.codes}{Fission Codes} \\ \href{mailto:brooklyn@fission.codes}{brooklyn@fission.codes} }
\addbibresource{spiral-ratchet.bib}
\date{\today\\v1.0.0}

\begin{document}
    \maketitle

%    \begin{multicols}{2}

   	\begin{abstract}\
   		Hash chains are a simple way to generate pseudorandom key material. However, they are inefficient in many situations, including when used as logical clocks, or for synchronizing the heads of a forward secret stream. This paper presents the ``skip ratchet'', a novel symmetric key derivation function that can be efficiently incremented by arbitrary intervals.
   	\end{abstract}

   	\section{Introduction}

	Hash algorithms provide a way to generate pseudorandom data from an input that is impractical to reverse. Many applications iteratively hash a value (``ratcheting'' \cite{pond}) for pre-computation resistance, forward secrecy (FS), one-time schemes (e.g. S/KEY \cite{rfc1760}), deterministically generating unique names, key derivation functions (KDFs), digital payment schemes \cite{multidimensional}, and so on.

	This paper introduces the ``skip ratchet'', a hierarchical and iterative hashing system suitable for symmetric key derivation, and capable of efficiently making large leaps in hash count while preserving forward secrecy in a passive setting.

    \section{Motivation}

    The intersection of private data and location-independent access control in open protocols is only beginning to be explored. Decentralization and trust minimization are increasingly important, but admit to many unsolved problems including how to secure changing data in an unknown and unstable topology. Not knowing the number or attributes of peers implies the use of passive security methods based directly on data.

    An increasing number of applications in open decentralized networks have no fixed topology, and require zero-interaction key agreement on a large number of files, the ability to share a single file (and no others), a range of files, or elements from a point in time onwards (but not prior versions). As such, there is also a need to keep the historical information of the internal state secret from even those with current access.

    One approach for securing history is to use a simple ratchet function, iteratively hashing on each update to produce the next state (a "hash chain"). This works well if the number of changes to synchronize is small, but $\mathcal{O}(n)$ ratchet steps is prohibitive as the difference grows. It allows a malicious participant to force others to a large amount of work in order to access the latest update.

    The skip ratchet presented here improves the situation. A skip ratchet is able to synchronize in sublinear time, supports efficient arbitrary access in the forward direction, maintains backwards secrecy of its internal state, and does not leak metadata such as the number of updates or participants. Further, jump size and granularity are configurable.

    \section{Numeral Intuition}

    The problem of how to efficiently calculate the $n^{\text{th}}$ step of a ratchet in less than $n$ steps is structurally similar to how we efficiently represent large numbers. There are many systems for doing so. The two most common are tally marks (unary) and positional numeral systems. The skip ratchet uses both of these concepts.

	\subsection{Unary Hashing}

	Hash chains are structurally very similar to unary counting systems.\footnote{Two examples are tally marks and Peano numbers \cite{peano-wikipedia}.} Unary is very concrete: the symbol count is equivalent to the number being represented. Replacing the successor function with a hash function $h$, one may represent unary counting with hashes.\footnote{By convention, this paper will use $I$, $J$, and $K$ as independent fixed initial values, $x$ as an arbitrary variable, and $h$ for hashing} Importantly, this does not make reference to a concrete ``zero'' value; merely an interval from the input $x$.

	While this method is very straightforward, it is not time efficient. A unary digit is included in the skip ratchet for forward security reasons. However, the remaining digits are more efficient.

    \begin{figure}[h]
		\centering

		\begin{flalign}
			Peano(3) & \Rightarrow S(S(S(0))) \nonumber \\
			h_{chain}(3)  & \Rightarrow h(h(h(I))) \equiv h^{3}(I) \nonumber
		\end{flalign}

		\caption{Peano Number $\leftrightarrow$ Hash Chain}
		\label{fig:peano-hash-equiv}
	\end{figure}

    \subsection{Positional Hashing}

    A major advantage of positional numerals is the ability to express large jumps with minimal effort. In positional systems (such as the familiar binary, decimal, and hexadecimal) each position corresponds to some factor for the numeral at that position.\footnote{While most systems use fixed-base exponential factors, there exist systems where there is no common relationship between positions.}

    \begin{figure}[h]
       	\centering
        $246_{16}$ = 2$\times{16^{2}}$ + 4$\times{16^{1}}$ + 6$\times{16^{0}}$ = $582_{10}$
     \caption{Componentized Hexadecimal}
    \end{figure}

    Just as positional numerals combine multiple numbers to represent a single sum, a KDF state can have multiple components that are combined to create a single symmetric key.\footnote{Different directions of this core idea include hash calendars \cite{distributed-hash-calendar}, and multidimensional hash chains (MDHC) \cite{multidimensional}}

    \begin{figure}[h]
       	\begin{flalign}
       		h_{10}(x+0)   & \Rightarrow I \nonumber \\
       		h_{10}(x+11)  & \Rightarrow \langle h(J), h(I) \rangle \nonumber \\
       		h_{10}(x+582) & \Rightarrow \langle h^{5}(K), h^{8}(J), h^{2}(I) \rangle \nonumber
       	\end{flalign}

       	\caption{Compound Hash State}
       	\label{fig:compound-hash}
    \end{figure}

    Unlike a positional number system used in arithmetic, a KDF ``forgets'' its internal state, combining elements and flattening its structure. There are many methods of combination, including further hashing; this paper will use bitwise \textsf{XOR} (Figure \ref{fig:compound-kdf}).

	\begin{figure}[h]
		\begin{flalign}
			h_{pos10, \oplus}(0)   & \Rightarrow I \nonumber \\
			h_{pos10, \oplus}(11)  & \Rightarrow h(J) \oplus h(I) \nonumber \\
			h_{pos10, \oplus}(582) & \Rightarrow h^{5}(K) \oplus h^{8}(J) \oplus h^{2}(I) \nonumber
		\end{flalign}

		\caption{Compound Hash Key Derivation}
		\label{fig:compound-kdf}
	\end{figure}

    \section{Bounding}

    Unlike numeral systems useful for arithmetic, the skip ratchet's internal state is composed of a fixed number of ``digits''. This limits the jumps to fixed intervals, with a maximum jump interval. While it is possible to create variants that deterministically generate increasingly large digits as needed, but this would leak data about the range of the counter. Being computationally indistinguishable is desirable in many use cases, such as for securing updatable documents in an public network.

    While advancing the ratchet by arbitrary intervals is possible, the jump operation is not the same as adding arbitrary integers. Incrementing all but the lowest digit cascades down to the lower values, ``zeroing'' them out. Therefore, arbitrary jumps require a carry (Algorithm \ref{alg:jump}). This is structurally similar to deterministic skip lists \cite{deterministic-skip-lists} (Figure \ref{fig:skip-list}), but with each step pointing to a monotonically increasing value, rather than previous ones.\footnote{While the analogy to fingered DAGs such as skip lists in the name is apt, the skip ratchet is distinct from other DAG-structured hash constructions like the Merkle tree \cite{merkle-tree} or hash calendar \cite{distributed-hash-calendar}. A skip ratchet does not aggregate digests into a signature, but rather counts by iteratively hashing the same value repeatedly, in tiers.} \\

	\begin{figure*}[h]
		\centering

		$\xymatrix{
			  Nil
			& A \ar[l]
			& B \ar[l] , \ar@/^1pc/[ll]
			& C \ar[l]
			& D \ar[l] , \ar@/^1pc/[ll] , \ar@/_2pc/[llll]
			& E \ar[l]
			& F \ar[l] , \ar@/^1pc/[ll]
			& G \ar[l]
			& H \ar[l] , \ar@/^1pc/[ll] , \ar@/_2pc/[llll]
		}$

		\caption{Deterministic Skip List}
		\label{fig:skip-list}
	\end{figure*}

	\begin{figure*}[h]
		\centering

		$\xymatrix{
		    \underset{000_{2}}{\left[ { \begin{array}{cc} L \\ M  \\ S \end{array} } \right]}
				\ar[r]^{+1}
				\ar@/_3pc/[rr]_{skip_{M}}
				\ar@/^7pc/[rrrr]^{skip_{L}}
    	 & \underset{001_{2}}{\left[ { \begin{array}{cc} L \\ M  \\ h(S) \end{array} } \right]}
	      		\ar[r]^{+1}
	      		\ar@/^4.75pc/[rrr]^{skip_{L}}
	     & \underset{010_{2}}{\left[ { \begin{array}{cc} L \\ h(M)  \\ S' \end{array} } \right]}
	      		\ar[r]^{+1}
	      		\ar@/_3pc/[rr]_{skip_{M}}
	      		\ar@/^3pc/[rr]^{skip_{L}}
    	 & \underset{011_{2}}{\left[ { \begin{array}{cc} L \\ h(M)  \\ h(S') \end{array} } \right]}
	      		\ar[r]^{+1}
	     & \underset{100_{2}}{\left[ { \begin{array}{cc} h(L) \\ M'  \\ S'' \end{array} } \right]}
		}$

		\caption{Simplified Skip Ratchet}
		\label{fig:compound-ratchet}
	\end{figure*}

	Each digit is given a label and maximum value for all but the largest, which acts as a linear ``spine''. This largest element is analogous to an unbounded unary digit. The remaining elements are arranged positionally, and are bounded by their numerical base. Since the unary digit is a hard limit on the number of digits, it conceals information about the range of numbers that the current internal state represents. Any range within a fixed larger value is called an ``epoch'' (e.g. ``the 300s epoch'' or ``the $h^{42}(x)$ epoch'').\footnote{The term ``epoch'' is used here in the generic or Ethereum\cite{ethereum-yellowpaper} sense, and is a slightly different from some other ratchet constructions such as the Double Ratchet\cite{double-ratchet-modularization}.}

	By geometric analogy, the possible steps in this construction are similar to moving along a spiral: one component returns to its initial value, but the other dimension is always moving forward. Following a spiral in single steps is smooth, but leaps are possible by jumping to adjacent rings (Figure \ref{fig:spiral-metaphor}). Unlike the repeating numerals found in most counting systems, the skip ratchet never repeats a value despite the counter being kept to a limited number of elements. We call this the ``spiral property.''.

	\begin{figure}[h]
		\centering

		\begin{tikzpicture}
			\draw [->, domain=0:29, variable=\t, smooth, samples=100]
			plot ({\t r}: {0.0015 * \t * \t});
			\draw [->] (0, 0.3) -- (0, 0.6);
			\draw [->, style=densely dotted, rotate=-30] (-0.15, 0.250) arc (0:60:21pt);
		\end{tikzpicture}

		\caption{Spiral With Leaps}
		\label{fig:spiral-metaphor}
	\end{figure}

	\section{Skip Ratchet}

	\subsection{Internal State}

	The skip ratchet's state is built from a unary digit $U$, a fixed number $n$ of positional digits, and their base $b$. The positional digits are bounded by the base, and so must track a natural number count. The unary digit explicitly does not track its count.

	\begin{figure*}[h]
		\centering

		$S : \langle \langle Count_{0}, Value_{0} \rangle, \langle Count_{1}, Value_{1} \rangle \ldots \langle Count_{n-1}, Value_{n-1} \rangle, U\rangle_{b}$

		\caption{Skip Ratchet Counter}
		\label{fig:counter}
	\end{figure*}

	Note that Figure \ref{fig:counter} is given as little-endian. The order of the state does not strictly matter, but it is convenient to associate the index of the state with the exponent for the base that the position represents.

	\subsection{Initialization}

	A skip ratchet is deterministically derived from a hash function $h$, a numeric base $b$, a positional digit count $n$, a function $k$ to derive a key the state, and an initialization vector $IV$. We distinguish between configurations by referring to their $\langle n, b, h, k \rangle$ triple, such as $\langle 3, 256, \textsf{SHA}3, \oplus \rangle$ skip ratchet.\footnote{This also implies that a $\langle 1, b, h, k \rangle$ skip ratchet would merely be a hash chain.}

	\begin{figure}[h]
		\centering

		$(\langle n, b, h, k \rangle, IV) \rightarrow S$

		\caption{Seed Generation}
		\label{fig:config}
	\end{figure}

	To prevent leaking the iteration count, all of the positional values in the seed state are immediately incremented by a random value. This randomized origin is treated as the ratchet's starting state. Given the backward secrecy constraint, only relative values may be used unless this initial value is known.

	Each positional value is generated from the \emph{binary complement ($\sim$) of the preimage} of its larger neighbour, in a recursive cascade starting with the unary digit (Algorithm \ref{alg:setup}). This protects the information needed to derive the current state, and thus prevents leaking all of the values in that range.

	\begin{algorithm*}
		\begin{algorithmic}[1]
			\Require $\langle n, b, k \rangle \in \mathbb{N}_{1}\times{\mathbb{N}_{1}\times{\mathbb{N}_{1}}}$
			\State $seed \overset{\$}{\gets} \lbrace 0, 1 \rbrace^{k}$
			\State $pos   \coloneqq []$
			\State $unary \coloneqq hash(seed)$
			\For{$i \gets (n - 1) \ldots 0$} \Comment{Descending to associate index with degree}
				\State $\delta \overset{\$}{\gets} 0\ldots{b - 1}$
				\State $seed \coloneqq hash(\sim{seed})$ \Comment{Secretly derive from the larger value}
				\State $pos[i].count \coloneqq \delta$
				\State $pos[i].value \coloneqq hash^{\delta + 1}(seed)$
			\EndFor
			\\
			\Return{$b, unary, pos$}
		\end{algorithmic}

		\caption{Skip Ratchet Initialization}
		\label{alg:setup}
	\end{algorithm*}

	The number of digits is configurable. These should be chosen to a balance jump control (the number of intervals that can be skipped by), and step-function performance. If the number of digits is large, increment on the unary element or higher digits will have require multiple hash operations in each zero cascade.

	In practice, $\langle 3, 256, \textsf{SHA}3, \oplus \rangle$ and $\langle 3, 256, \textsf{BLAKE}3, \oplus \rangle$ skip ratchets have performed well. The maximum leap is $256^{2}$, so the state $i + 256^{3}$ is (approximately) the same amount of work as $i + 256$. Given the small constant factor of hardware-accelerated hashing and the monotonicity of a ratchet, relatively large bases are often viable.

	\subsection{Basic Operations}

	Basic operations on the skip ratchet follow from the rules set out during initialization. It can be incremented sequentially, or leap to the next ``zero'' of any digit. Any interval can be efficiently found by combining increments and skips (Figure \ref{fig:sr-ops}).

	It is sometimes helpful to distinguish between a small and large skip, and so the terms ``skip'' and ``leap'' may be used for this purpose, despite them functioning identically but on different positions. The term ``increment'' (or ``inc'') is only used for the special case of the smallest movement (by one).

	\begin{figure*}[h]
		\centering

		$\xymatrix{
			IV \ar[r]^<<<<<{inc}
				& *+[F]{State_{0}}
					\ar[d]|\oplus
					\ar[r]^{inc}
					\ar@/^1.5pc/[rr]^{skip}
					\ar@{.>}@/^5pc/[rrrr]^{leap}
				& *+[F]{State_{1}}
					\ar[d]|\oplus
					\ar[r]^{inc}
					\ar@{.>}@/^3pc/[rrr]^{leap}
				& *+[F]{State_{2}}
					\ar[d]|\oplus
					\ar[r]^{inc}
					\ar@/^1.5pc/[rr]^{skip}
				& *+[F]{State_{3}}
					\ar[d]|\oplus
					\ar[r]^{inc}
				& *+[F]{State_{4}}
					\ar[d]|\oplus
					\ar[r]^{inc}
				& \cdots
			\\
				& \text{Key}_{0}
				& \text{Key}_{1}
				& \text{Key}_{2}
				& \text{Key}_{3}
				& \text{Key}_{4}
		}$

		\caption{Binary Skip Ratchet Operations}
		\label{fig:sr-ops}
	\end{figure*}

	Being a ratchet, it can never be ``unwound'' (there is no inverse or subtractive analogue). Access to earlier keys requires knowing an even earlier state, and moving forward to the desired state from there. This is important for break-in resistance and backwards secrecy (e.g sharing a document from a point in time but no earlier).

	\subsubsection{Key Derivation}

	Generating output key material (OKM) from the ratchet state can be done in a myriad of ways. This paper will use the bitwise \textsf{XOR} function, as it is straightforward and efficient.

	\begin{algorithm*}
		\caption{Generating a Key}
		\label{alg:toKey}

		\begin{algorithmic}[1]
			\Function{toKey}{ratchet}
				\State $sum \coloneqq ratchet.unary$
				\For{$digit \in ratchet.pos$}
					\State $sum \coloneqq acc \oplus digit.value$
				\EndFor
				\State $\textbf{return}\ sum$
			\EndFunction
		\end{algorithmic}
	\end{algorithm*}

%	\FloatBarrier

	\subsubsection{Increment}

	As with positional numerals, when a digit reaches its maximum, the next larger digit is incremented, and all lower digits are zeroed out. The zero cascade works similarly for skip ratchets, but never repeats a value (i.e. the \hyperref[{spiral-metaphor}]{spiral property}). As during initialization, the initialization, each zero is derived from the preimage of the higher digit.

	The special case of incrementing by one is the simplest way to demonstrate this (Algorithm \ref{alg:inc}). It functions by checking the lowest digit's counter, and if it's saturated, mark a carry and move to the next digit and recurse, bounded by the unary digit. Any carried digits are then re-zeroed by the preimage of their larger neighbour.

	\begin{algorithm*}
		\caption{Incrementing a Skip Ratchet}
		\label{alg:inc}

		\begin{algorithmic}[1]
			\Function{inc}{$ratchet$}
				\State $\{base, unary, pos\} \gets ratchet$
				\State $seed \coloneqq unary$
				\State $counter \coloneqq length(pos)$
				\\
				\For{$i \gets 0 \ldots length(pos) - 1$}
					\State $\{count, value\} \gets pos[i]$
					\If{$count \overset{?}{<} base - 1$} \Comment{Position not saturated}
						\State $seed \coloneqq value$
						\State $counter \coloneqq i$
						\State $ratchet.pos[i].count \coloneqq count + 1$ \State $ratchet.pos[i].value \coloneqq hash(value)\}$
						\State \textbf{break}
					\EndIf
				\EndFor
				\\
				\If{$counter\overset{?}{=} length(pos)$} \Comment{All positional values were saturated}
					\State $ratchet.unary \coloneqq hash(unary)$
				\EndIf
				\\
				\If{$counter \overset{?}{>} 0$}
					\For{$j \gets (counter - 1) \ldots 0$} \Comment{N.B. Descending}
						\State $seed' \coloneqq pos[j].value$
						\State $ratchet[pos][j].count \coloneqq 0$
						\State $ratchet[pos][j].value \coloneqq hash(\sim{seed})$
						\State $seed \coloneqq seed'$
					\EndFor
				\EndIf
				\\
				\State $ratchet$
				\EndFunction
		\end{algorithmic}
	\end{algorithm*}

	\subsubsection{Arbitrary Jumps}

	Jumps by an arbitrary interval require a little more calculation (Algorithm \ref{alg:jump}). The interval $\delta$ is componentized, and each digit incremented in ascending order. Every positional digit has an upper bound, so any remainder saved for the final step, where any zeroed digits are incremented to match the final count. Arbitrary jumps run in $\mathcal{O}(\log_{b} \delta)$ when $\delta < b^{b(n+1)}$ (the normal operating interval that the skip ratchet is tuned for). As the unary hash chain dominates as $\delta$ becomes very large, the complexity becomes $\mathcal{O}(\frac{\delta}{b^{n}})$.

	\begin{algorithm*}
		\caption{Skip Ratchet Arbitrary Jump}
		\label{alg:jump}

		\begin{algorithmic}[1]
			\Function{jump}{$ratchet, amount$}
				\State $\{base, unary, pos\} \gets ratchet$
				\State $remaining \coloneqq amount$
				\State $seed \coloneqq unary$
				\State $carry \coloneqq []$
				\State $n \coloneqq length(pos)$
				\\
				\For{$i \gets 0 \ldots n - 1$}
					\If{$remaining \overset{?}{=} 0$}
						\State \textbf{break}
					\EndIf
					\\
					\State $component_{i} \coloneqq remaining\mod{base^{i}}$
					\State $remaining \coloneqq remaining - component_{i}$
					\State $\delta_{i} \coloneqq \frac{component}{base^{i}}$ \Comment{No remainder because ascending in steps}
					\\
					\State $\{count, value\} \gets pos[i]$
					\State $headroom \coloneqq base - count - 1$
					\\
					\If{$\delta_{i} \overset{?}{>} headroom$}
						\State $carry[i] \coloneqq steps - headroom$
					\Else
						\State $ratchet.pos[i].count \coloneqq count + \delta_{i}$
						\If{$remaining \overset{?}{=} 0$}
							\State $seed \coloneqq hash^{\delta_{i} - 	1}(value)$
							\State $ratchet.pos[i].value \coloneqq hash(seed)$
						\Else
							\State $carry[i] \coloneqq 0$
						\EndIf
					\EndIf
				\EndFor
				\\
				\If{$remaining \overset{?}{>} 0$}
					\State $component_{u} \coloneqq remaining \mod{base^{n}}$
					\State $\delta_{u} \coloneqq \frac{component_{u}}{base^{n}}$
					\State $ratchet.unary \coloneqq hash^{\delta_{u}}(unary)$
				\EndIf
				\\
				\For{$j \gets (length(carry) - 1) \ldots 0$} \Comment{N.B. Descending}
					\State $seed \coloneqq hash^{carry[j] - 1}(\sim{seed})$
					\State $ratchet.pos[j].value \coloneqq hash(seed)$
				\EndFor
				\\
				\State $\textbf{return}\ ratchet$
			\EndFunction
		\end{algorithmic}
	\end{algorithm*}

	\section{Security}

	There is no inbuilt way to revoke this access once the key is compromised. Updates rely on forward secrecy for protection, and key rotation in the case where a breach becomes known. Algorithms like the Double Ratchet\cite{double-ratchet} have post-compromise security in a two-party semi-trusted setup, where there key is automatically rotated as a matter of course.

	The skip ratchet has two security domains: its external key generation, and its internal state. The internal state is itself based on hierarchical hash chains to assure forward secrecy, even for those that have been given access to the KDF's internal state. Not only is the previous state not calculable, other metadata such as the number of steps is also hidden, even when the internal state is known.

	\subsection{Interval Bounding}

	The skip ratchet's internal state permits bounding derivation to a fixed range in a decentralized system. This is accomplished by only sharing a few of the lower positions, and only sharing the rest of te state as a flattened hash.

	Revealing one OKM provides access to the cleartexts encrypted with that key (e.g. a single version of one file). Providing the \textsf{XOR} of the unary digit and a nonzero number of positional elements gives (post-compromise) access to the remaining number of updates in that epoch. Access to the entire internal state grants access from a point onwards, including potentially the initial state.

	For example, in a $\langle 4, 256, \textsf{SHA}3, \oplus \rangle$ skip ratchet, sharing $\langle S, M, L \oplus U \rangle$ allows the recipient to move freely in the ``medium'' jump range, up to $i + 256^{3} - 1$. Keys can still be fully generated thanks to \textsf{XOR}'s associativity, but the recipient will need to ask for the next $L' \oplus U$ to continue.

	\subsection{Post-Compromise Security}

	The skip ratchet trades off general post-compromise self-healing (of the unary digit) for permissionless key agreement between an unbounded and growable set of peers. If the secured communications are between exactly two parties, the Double Ratchet is typically a better choice due to its post-compromise security. Such a model could be adapted to use a skip ratchet for decentralized key agreement over time. In essence, this involves distributing a freshly initialized skip ratchet to a list of trusted peers on every $i$ steps \cite{decentralized-key-agreement}, or any other method of rotating the unary digit.

	\section{Conclusion}

	This paper has presented the skip ratchet, an algorithm for decentralized key agreement on streams of encrypted data with unbounded and changing participants. Arbitrary forward search in the expected operating range can be performed in $\mathcal{O}(\log_{b} \delta)$ time, backwards access is impossible without the earlier internal state, and total counts can never be inferred from the internal state.

    \section{Acknowledgements}

    The skip ratchet algorithm was developed at \href{https://fission.codes}{Fission Codes} to support work on the \href{https://github.com/fission-suite/webnative}{WebNative File System (WNFS)}. Many thanks to the Fission Codes team for providing the support and freedom to develop this algorithm.

    Thanks to \href{https://github.com/matheus23/}{Philipp Krüger} at Fission Codes for  feedback on the design, for writing the bulk of the TypeScript implementation\cite{ts-wnfs-ratchet}, and for his patience as various parts of the algorithm were tweaked while that code was already in-flight.

    Many thanks to \href{https://github.com/QuinnWilton}{Quinn Wilton} at Fission Codes for her feedback on various drafts of this paper, research into the broader literature, and suggestions on intended audience.

    Thanks to \href{https://github.com/b5/}{Brendan O'Brien} at \href{https://qri.io/}{Qri} for the {Go implementation}\cite{go-wnfs-ratchet}, and for many discussions on the best way to explain the concept.

    Thanks to \href{https://github.com/dholms}{Daniel Holmgren} at \href{https://blueskyweb.xyz}{Bluesky} for his feedback on early versions of this algorithm.

    Thanks to \href{https://github.com/jbenet/}{Juan Benet} at \href{https://protocol.ai/}{Protocol Labs} for encouraging me to write a fuller description of the skip ratchet than what was available in the WebNative online documentation.

    \printbibliography[title={References}]
\end{document}


%\documentclass[UTF8]{ctexart} % use larger type; default would be 10pt
\documentclass[a4paper]{article}
\usepackage{../mqc}

\renewcommand\thesection{\Alph{section}}

\title{\textbf{Modern Quantum Chemistry, Szabo \& Ostlund}\\HW}
\author{wsr
\vspace{5pt}\\
}
\date{\today} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
% \boldmath

\maketitle

\tableofcontents

\newpage

%\setcounter{section}{6}

\section{Integral Evaluation with 1s Primitive Gaussians}

\section{2-Electron Self-consistent-field Program}

\section{Analytic Derivative Methods and Geometry Optimization}
\subsection{Introduction}

\subsection{General Considerations}

\subsection{Analytic Derivatives}

\subsection{Optimization Techniques}

\subsection{Some Optimization Algorithms}
\ex{C.1}
\subex{(a)}
\begin{align}
\vb{H} &= \mqty(\dis\pdv[2]{E}{x} & \dis\pdv{E}{x}{y} \vspace{5pt}\\ \dis\pdv{E}{y}{x} & \dis\pdv[2]{E}{y}) \notag\\
&= \mqty(K & K'' \\ K'' & K')
\end{align}
\begin{align}
\vb{f}(\vb{X}) &= \mqty(\dis\pdv{E}{x} \vspace{5pt}\\ \dis\pdv{E}{y}) \notag\\
&= \mqty(K(x-a) + K''y \\ K'(y-b) + K''x) \notag\\
&= \mqty(K & K'' \\ K'' & K')\mqty(x\\ y) - \mqty(K a \\ K' b) \notag\\
&= \mqty(K & K'' \\ K'' & K')\vb{X} - \mqty(K a \\ K' b)
\end{align}
\begin{align}
\vb{q} &= -\vb{H}^{-1}\vb{f} \notag\\
&= -\vb{X} + \mqty(K & K'' \\ K'' & K')^{-1} \mqty(K a \\ K' b) \notag\\
&= -\vb{X} + \dfrac{1}{KK' - K''^2} \mqty(K' & -K''\\ -K'' & K) \mqty(K a \\ K' b) \notag\\
&= -\vb{X} + \dfrac{1}{KK' - K''^2} \mqty(KK' a - K'K'' b \\ -KK'' a + KK' b) \notag\\
&= -\vb{X} + \dfrac{1}{KK' - K''^2} \mqty(KK' & -K'K'' \\ -KK'' & KK' ) \mqty(a\\ b)
\end{align}

\subex{(b)}
Since $ \vb{q} = \vb{X}_e - \vb{X} $,
\begin{align}
\vb{X}_e &= \dfrac{1}{KK' - K''^2} \mqty(KK' & -K'K'' \\ -KK'' & KK' ) \mqty(a\\ b) \notag\\
&= \dfrac{1}{0.1 - K''^2} \mqty(0.1 & -0.1 K'' \\ -K'' & 0.1 ) \mqty(3\\ 2) %\notag\\
\end{align}

\begin{table}
	\centering
	\begin{tabular}{cc}
		\hline
		$ K'' $ & $ \vb{X}_e = (x_e, y_e) $\\ \hline
		0 & (3.000, 2.000)\\
		0.010 & (2.983, 1.702)\\
		0.030 & (2.967, 1.110)\\ \hline
	\end{tabular}
\end{table}

\ex{C.2}
\begin{align}
\vb{H}
&= \mqty(K & K'' \\ K'' & K') \notag\\
&= \mqty(1.000 & 0.030\\ 0.030 & 0.100)
\end{align}
\begin{align}
\vb{f}
&= \mqty(K & K'' \\ K'' & K')\vb{X} - \mqty(K a \\ K' b) \notag\\
&= \mqty(1.000 & 0.030\\ 0.030 & 0.100)\mqty(3.3\\ 1.8)
- \mqty(1.000\times 3.00\\ 0.100\times 2.00) \notag\\
&= \mqty(0.354\\ 0.079)
\end{align}
\begin{align}
\vb{q} &= -\vb{H}^{-1}\vb{f} \notag\\
&= -\mqty(1.000 & 0.030\\ 0.030 & 0.100)^{-1} \mqty(0.354\\ 0.079) \notag\\
&= \mqty(-0.333, -0.690)
\end{align}
thus
\begin{align}
\vb{X}_e &= \vb{q} + \vb{X} \notag\\
&= \mqty(2.967, 1.110)
\end{align}
which agrees with the result in Ex C.1(b).

\ex{C.3}
A program is written to solve this problem, which is \code{C-3.py}.

For example, run the program by
\code{python C-3.py 0.03}, and the Nelder-Mead optimization steps will be printed for $ K'' = 0.03 $.

\ex{C.4}
A program is written to solve this problem, which is \code{C-4.py}.

For example, run the program by
\code{python C-4.py}, and the MS optimization steps will be printed.

\subsection{Transition States}

\subsection{Constrained Variation}

\section{Molecular Integrals for \ce{H_2} as a Function of Bond Length}

\end{document}


\section{On-policy Control with Approximation}

\subsection{Exercise 10.1}
\subsubsection*{Q}
We have not explicitly considered or given pseudocode for any Monte Carlo methods or in this chapter. What would they be like? Why is it reasonable not to give pseudocode for them? How would they perform on the Mountain Car task?

\subsubsection*{A}
\begin{itemize}
    \item Monte Carlo is $n$-step Sarsa with $n \to \infty$
    \item This is same pseudocode as given, but with full episodes and $G_t$ rather than $G_{t:t+n}$.
    \item Could have been very poor on the mountain car as may never have finished the first episode and does not learn within an episode (online)
\end{itemize}

\subsection{Exercise 10.2}
\subsubsection*{Q}
Give pseudocode for semi-gradient one-step \emph{Expected Sarsa} for control.

\subsubsection*{A}
Expected sarsa is the same but the target is
\[
    \sum_{k=t}^{t +n -1} \gamma^{k-t} R_{k + 1} + \sum_a \pi(a \vert{} S_{t+n}) q_{t + n -1}(S_{t+n}, a)
\]

\subsection{Exercise 10.3}
\subsubsection*{Q}
Why do the results shown in Figure 10.4 have higher standard errors at large $n$ than at small $n$?
\subsubsection*{A}
The longer the step length then the greater the variance in initial runs, this is because the agent needs to wait for $n$ steps to start learning. Some initial episodes of high $n$ cases could have been very poor.

\subsection{Exercise 10.4}
\subsubsection*{Q}
Give pseudocode for a differential version of semi-gradient Q-learning.

\subsubsection*{A}
Same as others but with the target
\[
    R_{t+1} - \bar{R}_{t+1} - \max_a \hat{q}(S_{t+1}, a, \vec{w}_t)
\]

\subsection{Exercise 10.5}
\subsubsection*{Q}
What equations are needed (beyond 10.10) to specify the differential version of TD(0)?

\subsubsection*{A}
Just need the semi-gradient update
\[
    \vec{w}_{t+1} = \vec{w}_{t} + \alpha \delta_t \grad_{\vec{w}_t} \hat{v}(S_t, \vec{w}_t)
\]
where
\[
    \delta_t = R_{t+1} - \bar{R}_{t+1} + \hat{v}(S_{t+1}, \vec{w}_{t}) - \hat{v}(S_{t}, \vec{w}_t)
\]

\subsection{Exercise 10.6}
\subsubsection*{Q}
Consider a Markov reward process consisting of a ring of three states A, B, and C, with state transitions going deterministically around the ring. A reward of 1 is received upon arrival in A and otherwise the reward is 0. What are the differential values of the three states?
\subsubsection*{A}
The average reward is $\bar{R} = \frac13$. To calculate the differential return we have
\[
    V(A) = \sum_t (a_t - \bar{R})
\]
where $a_i = \mathds{1}\{i + 1 \equiv 0 \pmod 3\}$. This doesn't converge in the normal way, so to attempt to calculate it let's consider
\[
    V(A; \gamma) = \sum_t \gamma^t \left( a_t - \frac13\right)
\]
then, formally, we have
\[
    \lim_{\gamma \to 1} V(A; \gamma) = V(A).
\]
Now
\begin{align*}
    V(A; \gamma) &= - \frac13 - \frac13 \gamma  + \frac23 \gamma^2 + \sum_{t=3}^{\infty} \gamma^t \left(a_t - \frac13\right) \\
                 &= \frac13 (2 \gamma^2 - \gamma - 1) + \gamma ^3 \sum_{t=0}^{\infty} \gamma^t \left(a_t - \frac13\right)
\end{align*}
so
\begin{align*}
    V(A; \gamma) &= \frac13 \frac{2 \gamma^2 - \gamma - 1}{1 - \gamma^3} \\
                 &= -\frac13 \frac{2 \gamma + 1}{\gamma^2 + \gamma +1}
\end{align*}
which leads to $V(A) = -\frac13$. \\

Then we have
\[
    V(A) = -\frac13 + V(B) \quad \implies \quad V(B) = 0
\]
and
\[
    V(B) = -\frac13 + V(C) \quad \implies \quad V(C) = \frac13.
\]

\subsection{Exercise 10.7}
\subsubsection*{Q}
Suppose there is an MDP that under any policy produces the deterministic sequence of rewards 1, 0, 1, 0, 1, 0, . . . going on forever. Technically, this is not allowed because it violates ergodicity; there is no stationary limiting distribution $\mu_\pi$ and the limit (10.7) does not exist. Nevertheless, the average reward (10.6) is well defined; What is it? Now consider two states in this MDP. From A, the reward sequence is exactly as described above, starting with a 1, whereas, from B, the reward sequence starts with a 0 and then continues with 1, 0, 1, 0, . . .. The differential return (10.9) is not well defined for this case as the limit does not exist. To repair this, one could alternately define the value of a state as

\[
    v_\pi(s) \doteq \lim_{\gamma \to 1} \lim_{h \to \infty} \sum_{t=0}^h \gamma^t \left( \Epi{}[R_{t+1} \vert{} S_0 = s] - r(\pi) \right).
\]

Under this definition, what are the values of states A and B?
\subsubsection*{A}
Define
\[
    f(h) = \frac{1}{2h} \sum_{t=0}^{2h} \mathds{1}\{t \equiv 0 \pmod 2\} = \frac{h + 1}{2h}
\]
then
\[
    \bar{R} = \lim_{h \to \infty}f(h/2) = \lim_{h \to \infty}f(h) = \frac12.
\]
Now to compute the differential state values we write
\[
    V(S; \gamma) = \lim_{h\to \infty} \sum_{t=0}^h \gamma^t \left( \E{}[R_{t+1} \vert{} S_0 = s] - \bar{R} \right)
\]
then
\begin{align*}
    V(A; \gamma) &= 1 - \bar{R} + \gamma V(B; \gamma) \\
    V(B; \gamma) &= - \bar{R} + \gamma V(A; \gamma)
\end{align*}
so
\[
    V(A; \gamma) = \frac12 ( 1 - \gamma ) - \gamma^2 V(A; \gamma)
\]
and
\begin{align*}
    V(A; \gamma) &= \frac12 \frac{1 - \gamma}{1 - \gamma^2} \\
                 &= \frac{1}{2(1 + \gamma)}.
\end{align*}
Finally, $V(A) = \lim_{\gamma \to 1} V(A; \gamma) = \frac14$ and $V(B) = - \frac14$.

\subsection{Exercise 10.8}
\subsubsection*{Q}
The pseudocode in the box on page 251 updates $\bar{R}_{t+1}$ using $\delta_t$ as an error rather than simply $R_{t+1} - \bar{R}_{t+1}$. Both errors work, but using $\delta_t$ is better. To see why, consider the ring MRP of three states from Exercise 10.6. The estimate of the average
reward should tend towards its true value of $\frac13$. Suppose it was already there and was held stuck there. What would the sequence of $R_{t+1} - \bar{R}_{t+1}$ errors be? What would the sequence of $\delta_t$ errors be (using (10.10))? Which error sequence would produce a more stable estimate of the average reward if the estimate were allowed to change in response to the errors? Why?

\subsubsection*{A}
$\bar{R} = \frac13$ fixed. \\

The sequence of errors from $R_t - \bar{R}_t$ starting in A would be
\[
    -\frac13, -\frac13, \frac23, -\frac13, -\frac13, \frac23, \dots
\]
while the sequence of TD errors starting in A (taking differential values from Exercise 10.6) would be
\[
    0, 0, 0, 0, 0, 0, \dots
\]
which is clearly of much lower variance and would therefore give more stable updates. Once $\bar{R}$ gets to the correct value it never leaves.

\subsection{Exercise 10.9}
\subsubsection*{Q}
In the differential semi-gradient $n$-step Sarsa algorithm, the step-size parameter on the average reward, $\beta$, needs to be quite small so that $\bar{R}$ becomes a good long-term estimate of the average reward. Unfortunately, $\bar{R}$ will then be biased by its initial value for many steps, which may make learning inefficient. Alternatively, one could use a sample average of the observed rewards for $\bar{R}$ . That would initially adapt rapidly but in the long run would also adapt slowly. As the policy slowly changed, $\bar{R}$ would also change; the potential for such long-term non-stationarity makes sample-average methods ill-suited. In fact, the step-size parameter on the average reward is a perfect place to use the unbiased constant-step-size trick from Exercise 2.7. Describe the specific changes needed to the boxed algorithm for differential semi-gradient $n$-step Sarsa to use this trick.

\subsubsection*{A}
We define a parameter $\beta$ and seed a sequence $u_n$ with $u_0=0$. The under the if statement where $\tau \geq 0$ we place the following:
\begin{align*}
    u &\leftarrow u + \beta (1 - u) \\
    \bar{R} & \leftarrow \bar{R} + \frac{\beta}{\mu} (R - \bar{R})
\end{align*}


\documentclass[11pt]{article}

\usepackage{booktabs}
\usepackage{float}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{multirow}
\usepackage[parfill]{parskip}
\usepackage[capitalise]{cleveref}

\begin{document}
    \title{HPCSE II - Exercise 6}
    \author{Anian Ruoss}
    \maketitle

    \section*{Task 1}
    \label{sec:Task1}

    We parallelize the heat2D model for the single-grid case.
    For ease of communication with MPI we change the \lstinline{gridLevel}
    struct to store the grids with \lstinline{double *} instead of
    \lstinline{double **}.
    First, we establish the problem geometry with
    \lstinline{MPI_Dims_create()}, \lstinline{MPI_Cart_create()}, and
    \lstinline{MPI_Cart_shift()}.
    Since the grid size is not a multiple of two we have to pay attention to
    the boundary cases.
    Every rank allocates its own subgrids for \lstinline{U}, \lstinline{Un},
    \lstinline{Res}, and \lstinline{f}.
    The root rank then initializes the problem on the entire grid and
    distributes the initial values to each rank.
    Since this step is only performed once we do not employ custom data
    types even though this incurs some memory overhead due to data duplication.

    \subsection*{a)}
    \label{subsec:Task1a}

    After every Jacobi iteration we need to exchange the boundaries between
    neighboring ranks.
    Since we cannot pass the grid communicator to \lstinline{applyJacobi()}
    (as this method is declared in \lstinline{heat2d_mpi.hpp} where MPI is
    not initialized yet), we move the loop over the
    \lstinline{downRelaxations} out of \lstinline{applyJacobi()}.
    Next, we define data types for the contiguous and non-contiguous
    boundaries.
    Finally, we only need to adapt the loop boundaries to the subgrid sizes.

    \subsection*{b)}
    \label{subsec:Task1b}

    The \lstinline{calculateResidual()} method is trivially parallelizable
    by adapting the loop boundaries to the subgrid sizes.
    For \lstinline{calculateL2Norm()} we need to be careful not to loop over
    the ghost cells (we make use of the fact that the Dirichlet boundary
    condition is zero).
    After computing the sum of squared residuals for every subgrid we reduce
    the sum with MPI before computing the $L_2$-norm.

    \subsection*{Evaluation}
    \label{subsec:Task1Eval}

    We run the baseline sequential model and compare it with the execution
    times from running the parallelized model on one, two, and four full nodes
    on Euler.
    We display the running times
    in~\cref{lst:Baseline,lst:OneNode,lst:TwoNodes,lst:FourNodes}.

    \begin{lstlisting}[basicstyle=\scriptsize, caption={Output from running
    the baseline sequential model on one full node.}, float, floatplacement=h,
    frame=single, label={lst:Baseline}]
    Time (s)     Grid0 Total
    -------------|---------|---------
    Smoothing | 570.226 | 570.226
    Residual | 204.614 | 204.614
    Restriction | 0.000 | 0.000
    Prolongation | 0.000 | 0.000
    L2Norm | 219.811 | 219.811
    -------------|---------|---------
    Total | 994.652 | 994.652
    -------------|---------|---------

    Running Time : 994.954s
    L2Norm: 70812.9164
    \end{lstlisting}
    \begin{lstlisting}[basicstyle=\scriptsize, caption={Output from running
    the parallelized model on one full node.}, float, floatplacement=h,
    frame=single, label={lst:OneNode}]
    Time (s)    Grid0 Total
    -------------|---------|---------
    Smoothing | 18.228 | 18.228
    Residual | 6.370 | 6.370
    Restriction | 0.000 | 0.000
    Prolongation | 0.000 | 0.000
    L2Norm | 9.528 | 9.528
    -------------|---------|---------
    Total | 34.126 | 34.126
    -------------|---------|---------

    Running Time : 35.273s
    L2Norm: 70812.9164
    \end{lstlisting}
    \begin{lstlisting}[basicstyle=\scriptsize, caption={Output from running
    the parallelized model on two full nodes.}, float, floatplacement=h,
    frame=single, label={lst:TwoNodes}]
    Time (s)    Grid0 Total
    -------------|---------|---------
    Smoothing | 8.896 | 8.896
    Residual | 3.271 | 3.271
    Restriction | 0.000 | 0.000
    Prolongation | 0.000 | 0.000
    L2Norm | 7.684 | 7.684
    -------------|---------|---------
    Total | 19.851 | 19.851
    -------------|---------|---------

    Running Time : 19.926s
    L2Norm: 70812.9164
    \end{lstlisting}
    \begin{lstlisting}[basicstyle=\scriptsize, caption={Output from running
    the parallelized model on two full nodes.}, float, floatplacement=h,
    frame=single, label={lst:FourNodes}]
    Time (s)    Grid0 Total
    -------------|---------|---------
    Smoothing | 3.993 | 3.993
    Residual | 1.598 | 1.598
    Restriction | 0.000 | 0.000
    Prolongation | 0.000 | 0.000
    L2Norm | 6.009 | 6.009
    -------------|---------|---------
    Total | 11.600 | 11.600
    -------------|---------|---------

    Running Time : 13.509s
    L2Norm: 70812.9164
    \end{lstlisting}

    It can be observed that all runs compute the same final $L_2$-norm value.
    We display the strong scaling speed-ups and efficiencies
    in~\cref{tab:Runtimes}.
    Both \lstinline{applyJacobi()} and \lstinline{calculateResidual()}
    achieve speedups that exceed $100$\%.
    This is due to the fact that we do not include the communication time in
    the measured kernel time for simplicity of the implementation.
    Hence, it is more meaningful to analyze the speed-up and efficiency for
    the total running time.
    We observe that for one and two nodes we achieve perfect
    efficiency\footnote{We believe that the efficiency is slightly above
    $100$\% as we avoid the loop overhead by commenting out the prolongation-
    and restriction-loops since we only consider the single-grid case.},
    whereas we are probably limited by an increased communication overhead
    when running on four full nodes.
    Lastly, we note that the sharp efficiency decline for
    \lstinline{calculateL2Norm()} is due to the expensive collective
    \lstinline{MPI_Allreduce()} operation.

    \begin{table}
        \caption{Strong scaling analysis for the parallelized heat2D model.}
        \label{tab:Runtimes}
        \vspace{.25cm}
        \begin{center}
            \resizebox{.5\textwidth}{!}{ \input{./task1/results/runtimes} }
        \end{center}
    \end{table}

    \section*{Task 2}
    \label{sec:Task2}

    \subsection*{a)}
    \label{subsec:Task2a}

    We implement the first hybrid approach with MPI and OpenMP by adding
    \lstinline{#pragma omp parallel for collapse(3)} in front of the jacobi
    step loop.
    This does not induce a race condition since all iterations are
    independent from each other.

    \subsection*{Evaluation}
    \label{subsec:Task2Eval}

    To avoid inconsistent $L_2$-norm values when dividing the grid by $12$ or
    $24$ MPI ranks, we increase the grid size to $768$.
    We use the following command to run hybrid jobs on Euler with $N$ nodes,
    $M$ MPI ranks and $T$ OpenMP threads:

    \begin{lstlisting}[basicstyle=\tiny]
    bsub -R fullnode -n 24*N
    "unset LSB_AFFINITY_HOSTFILE;
    OMP_NUM_THREADS=T mpirun -n M --map-by node:PE=T --report-bindings ./hybrid"
    \end{lstlisting}

    We first try to gauge the performance loss incurred by network
    communication.
    To this end, we limit the total number of processes to $24$ and measure
    the running times when distributing these processes across MPI/OpenMP
    and multiple nodes.
    We display the running times in~\cref{tab:StrongScaling}.

    \begin{table}
        \caption{Comparison of intra-node and network communication effects.}
        \label{tab:StrongScaling}
        \vspace{.25cm}
        \begin{center}
            \resizebox{\textwidth}{!}{ \input{./task2/results/strong_scaling} }
        \end{center}
    \end{table}

    The first row displays our baseline which uses $24$ MPI ranks distributed
    evenly among both sockets of one node.
    Hence, our baseline relies on intra-node communication only.
    The second row displays the output from running our hybrid code with $4$
    MPI ranks with $6$ OpenMP threads each, all running on the same node.
    The first two ranks are assigned to the first socket, the last two ranks
    to the second socket.
    We observe that --- although the number of computed entries increases
    by a factor of $6$ for every rank --- the total compute time increases
    only by a factor of $\approx 2.2$ meaning that we achieve an efficiency
    of $\approx 45$\% when considering only OpenMP.\@
    Compared to the baseline, this hybrid model uses $6$ times fewer ranks
    which means that the number of communications decreases from $92$ to $8$.
    However, every rank has to exchange $\approx 3.5$ times more boundary
    entries and we observe that this incurs an increase in the overall
    \lstinline{MPI_Waitall} time for the hybrid intra-node case.
    This effect becomes even stronger when we distribute the MPI ranks
    across multiple nodes.
    The third row displays the execution times for a run with $4$ MPI ranks
    with $6$ OpenMP threads each, of which $2$ ranks are on a separate node.
    The last row displays the execution times for with $4$ MPI ranks
    with $6$ OpenMP threads each, where every rank is on a node of its own.
    For this case, where all communication is performed over the network, we
    observe a substantial increase in \lstinline{MPI_Waitall} time compared to
    the second row which relies on intra-node communication only.

    Next, we try to measure whether using a hybrid model has a positive effect
    on the total running time.
    We run the hybrid model with $2$ MPI ranks per node and $12$ OpenMP
    threads per rank, making sure that each rank is assigned to its own socket.
    As a consequence we reduce the amount of network communication between MPI
    ranks.
    We compare our hybrid model against a pure MPI model on one, two, and
    four nodes and display the running times in~\cref{tab:WeakScaling}.

    \begin{table}
        \caption{Runtime comparison of MPI and hybrid MPI/OpenMP models.}
        \label{tab:WeakScaling}
        \vspace{.25cm}
        \begin{center}
            \resizebox{\textwidth}{!}{ \input{./task2/results/weak_scaling} }
        \end{center}
    \end{table}

    It can be observed that for one and two nodes the increased message
    sizes --- due to the smaller number of MPI ranks --- supersedes the
    reduced cost of less communication over the network.
    However, for four nodes the \lstinline{MPI_Waitall} time is already
    smaller for the hybrid case and we would except the hybrid model to
    significantly outperform the MPI model on more nodes.

\end{document}


\documentclass[12pt, a4paper]{article}
\usepackage[a4paper, bindingoffset=0.2in, %
left=0.5in,right=0.5in,top=0.5in,bottom=0.5in,%
footskip=.25in]{geometry}
\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{physics}

\title{PSet7 Report}
\author{Ali Abolhassanzadeh Mahani}

\begin{document}
	\maketitle
	\section{Implementing Code to Simulate the Ising Model}
	I started this code by making the class \texttt{Ising} with an \texttt{\_\_init\_\_} function that
	takes in the size of the platform, $L$, and $\beta$, and then creates a grid of size $L$ and fills it
	randomly with $\pm1$ as spin up/down. ($\ket{\uparrow}, \ket{\downarrow}$)\\

	We know that the distribution for this system is canonical, of the form
	$Z = \sum_{i}^{} e^{-\beta E_i}$. So now, I create a method \texttt{metropolis()}, which uses the metropolis algorithm to make the distribution of energy $E_i$ to be of the canonical form.
	This method, loops for $L^2$ times and picks a cell on the grid randomly and performs the metropolis conditions on it and if it's works, the method give the cell a bit flip.

	Here, in file \texttt{ising\_example.py}, I initialized an Ising model of size $L = 200$ and called the \texttt{metropolis()}
	function on it 200 times. the results for $\beta = 0.3$ and $\beta = 0.6$ are available in Fig\ref{fig:ising_grid}
	\begin{figure}[h!]
		\centering
		\includegraphics[width=0.9\linewidth]{../ba3.jpg}
		\includegraphics[width=0.9\linewidth]{../ba6.jpg}
		\label{fig:ising_grid}
		\caption{Ising model of size 200, before and after using the \texttt{metropolis()} function 200 times (top: $\beta = 0.3$, bottom: $\beta = 0.6$).}
	\end{figure}

	\section{Data Acquisition}
	In order to collect data from the system, first we need a method to check if the system has reached equilibrium. In order to do that, I made the method \texttt{equalize()} that uses
	\texttt{metropolis()} $n = 100$ times and takes the energy samples along the process. Then checks if the auto-correlation of energies for a gap of $\frac{n}{10}$ is less that $e^{-3}$.
	If \texttt{True}, the system has reached equilibrium, otherwise repeat the process 100 more
	times and check for $n = 200$. and so on until we reach equilibrium.

	After the system has reached equilibrium, we need to take the auto-correlation of energies
	to see for how many steps our data are \emph{almost} not correlated to pick a sample.
	This is made possible using the \texttt{corr\_len()} function.
	Then, I pick the data in intervals of $h =$ \texttt{corr\_len(energies)}.

	This process is repeated for lengths $\{100, 110, 130, 160, 200\}$ and a linear space of
	$\beta$ of size $40$, from $0.1$ to $0.7$ . I saved the data to CSV files and analyzed and
	plotted them using the file \texttt{data\_analysis.ipynb}.\\
	The plots are available in Figs \ref{fig:data}
	\begin{figure}[h!]
		\centering
		\includegraphics[width=0.45\linewidth]{../energy_plot.jpg}
		\includegraphics[width=0.45\linewidth]{../magnet_plot.jpg}
		\includegraphics[width=0.45\linewidth]{../ksi_plot.jpg}
		\includegraphics[width=0.45\linewidth]{../heat_cap_plot.jpg}
		\includegraphics[width=0.45\linewidth]{../spin_cor_plot.jpg}
		\label{fig:data}
		\caption{Data vs. beta for $\beta$ from $0.1$ to $0.7$ and ising size in
			$\{100, 110, 130, 160, 200\}$. from left to right and top down, we have \emph{energy}, \emph{magnetization}, \emph{$\chi$}, \emph{heat capacity $C_v$}, $\xi$ \emph{only for $L = 100$}.}
	\end{figure}

	As for how I selected and changed the temperature, I started from high temp.,
	$\beta =\frac{1}{k_B T} = 0.1$ to $0.7$, and each time I was done taking data for each
	$\beta$, I the equalized state of the system with the previous value of $\beta$ for the initial
	condition of the system with new $\beta$. This way the necessary time to reach equilibrium is decreased and so is the risk of the system freezing.

	\section{Finding the critical exponents $\nu, \mu, \beta$ and the Coefficient $c_0$}
	The value for $c_0$ was found by fitting a log-log polynomial of degree 1 to the data. The function for $c_0$ is:
	\begin{equation}
		c_v = c_0 \ln(|T - T_c|), \qquad c_0 = 7.24 \pm 0.07
	\end{equation}

	For the other parameters since we have already shown the plots, we'll just report the numerical values here.
	\begin{equation}
	\begin{aligned}
		m &\sim L^{-\frac{\beta}{\nu}}, \qquad \beta = \pm \\
		\chi &\sim L^{-\frac{\gamma}{\nu}}, \qquad \frac{\gamma}{\nu} = - 1.49 \pm 0.56\\
		C_v &\sim L^{-\frac{\alpha}{\nu}}, \qquad \frac{\alpha}{\nu} = - 0.79 \pm 0.12\\
		\xi & \sim L^{-\frac{1}{\nu}}, \qquad \frac{1}{\nu} = 0.02\pm 0.08
	\end{aligned}
	\end{equation}

	\section{Animating the Ising model (Bonus Question)}
	I finally got the hang of \texttt{plt.animation.FuncAnimation()}, I suppose. I used this to make a function \texttt{animate\_ising()} in
	the \texttt{anime.py} file. Then I used it to animate the ising model from a uniform random distribution to equilibrium. I rendered the animations as GIFs using \emph{imagemagick} and saved them using the value for $\beta$ as a signature for the file names.
	(\texttt{'ising<beta>.GIF'}). The time steps are each equal to one iteration of the \texttt{Ising.metropolis()} function.

\end{document}