\chapter{Background}
\label{chap:background}
Image classification refers to a task that requires from method to determine the categories to which picture belongs. Such challenges that utilize standard datasets like ImageNet and PASCAL VOC limit this problem to a question if objects of particular classes presented on the image \cite{Russakovsky2015ImageNet, Everingham2010PASCAL-VOC}. However, real-world settings might require extending this question to more general one \cite{Wang2016CNN-RNN:Classification}. For instance, in this study, the search extended beyond merely objects to include other image properties including color, weather, action, and sceneries. %In other words, image classification system should be able to find a correct placement of a picture in a multi-dimensional space of categories.

% \section{Image classification}
% definition of image classification

\section{Deep convolutional neural networks}

A standard neural network consists of computational units called neurons. Neurons are connected to each other in particular way to form a network. Neurons in such networks are organized in groups or ``layers'' for computational efficiency reasons since it allows to apply vector operations \cite{cs231n-nn1}.  Each input connection to a neuron has dedicated weight. Non-liner \textit{activation function} use these input weights to determine when to activate particular neuron and send a signal further \cite{Schmidhuber2015DeepOverview}. Activation function can be different, however recent years ReLU function is mostly used in modern DNN due to its computational efficiency and good output results \cite{relu, Krizhevsky2012ImageNetDNN}. The ``depth'' of the neural network is identified as an amount of layers except input one \cite{Schmidhuber2015DeepOverview}.

The process of \textit{training} the neural network is optimization of each neuron input weights in such way that the particular input to a network will produce desired output. This optimization is achieved by first introducing loss function which represents how ``far'' given output is from the desired one. After that, on each iteration of training, by calculating gradients of a loss function on given weights, the system determines how it should adjust weights to decrease the computed loss \cite{cs231n-opt1}. Modern DNNs use backpropagation approach of gradient calculation due to its efficiency \cite{cs231n-opt2}.

Convolutional Neural Networks (CNN) have the same core ideas as any other neural networks except that CNN architectures assume that the inputs are images \cite{cs231n-conv}. Unlike conventional neural networks, instead of processing images pixel-by-pixel CNN is searching for patterns in image patches of different sizes. Each convolution can also be perceived as a feature extractor, such as the one that detects particular edge angle or color. Such convolution functions are also organized in layers where output features from one layer can be an input data for next convolutional filters. Therefore, an end system consists of filters cascade, where, similar to human vision, low-level features, like edges, are recognized on the fist levels, and higher levels trigger on more complex features like different shapes and patterns~\cite{Zeiler2014VisualizingNets}

The main difference between machine learning approach and deep convolutional neural networks is that the former one has an additional step of supervised feature extraction. In contrast, in deep learning approach feature extraction happens automatically by the internal network itself. Therefore DNN can be considered as a ``black box'' system. For instance, in the case of image classification, such system will have raw picture pixels as an input and classified category numbers as output.

To improve the generalizability of results and to be able to compare networks with different hyperparameters the common approach is to use dataset splitting where an image collection is partitioned into three parts \cite{cs231n-cls}. The first part, \textit{called training} set, is employed directly for training, while the second part, called \textit{validation set}, is used to compare different architectures and control overfitting.  Overfitting is a state of the system when it optimizes to a noise in a training data instead of searching the underlying patterns \cite{cs231n-nn1}. Such optimization can potentially reduce the generalizability of the system and should be avoided \cite{cs231n-nn1}. The usual way to control overfitting of the model is to check system performance on a validation set periodically during the training process \cite{cs231n-nn3}. If the loss computed on a training set gets smaller with more iterations, but validation loss increases or stays the same, then it is an evidence that the model is starting to overfit on the training set \cite{cs231n-nn3}. To be able to monitor the generalizability of the method, validation set should not contain any images from the training set. The third part of the dataset, which is called \textit{test set}, is used for a final evaluation of the method. The separation between validation and test sets is important for the similar reason as with separation between training and validation sets. When different network implementations are compared using the validation set, and the best one is selected, hyperparameters can overfit on the validation set similar to the state when network is overfitting on the training set \cite{cs231n-cls}. Meaning that chosen parameters will be optimized for the particular images in the validation set. Therefore, it is important to test the final system on a separate set of pictures, which CNN has not seen at any time during the training process. According to Andrew Ng, The common ratio between these tree sets is 60\%/20\%/20\%, where the training set gets 60\% of images while validation and test sets receive 20\% each \cite{Ng2016NutsLearning}.
% train, val, test split
% different layers
% overfitting
% backpropagation

\section{Multi-label image classification}
Opposite to a single-label image classification, in multi-label case picture can belong to more than one category. Several labels can both describe all objects presented on the picture and characterize the objects or picture itself.

There are two group of multi-label image classification methods:
\begin{itemize}
    \item That process parts of the images and use CNN to find a single label for each of the part \cite{Wei2016HCP, Ren2016,  Yang2015},
    \item That train CNN on the whole picture, similarly to single label methods, but use different loss functions that take several labels into account \cite{Gong2013DeepRanking}.
\end{itemize}

An example of the first type of network is described by Wei et al. in their paper ``HCP: A Flexible CNN Framework for Multi-Label Image Classification''  \cite{Wei2016HCP}. The core idea of the proposed method is to reuse already developed CNNs that were trained to classify single-labeled pictures in a multi-label scenario. The authors proposed deep CNN infrastructure called ``Hypotheses-CNN-Pooling'' (HCP) to achieve this. Its architecture is shown on Figure \ref{fig:hcp-net}.

\begin{figure}[h!]
    \centering
    \includegraphics[width=\textwidth]{hcp-net}
    \caption{Hypotheses-CNN-Pooling infrastructure \cite{Wei2016HCP}}
    \label{fig:hcp-net}
\end{figure}

The whole classification pipeline proposed by the authors consists of the following steps:
\begin{enumerate}
    \item Applying state of the art object detection technique (for example BING \cite{Cheng2014} or EdgeBoxes \cite{Zitnick2014}) to get the set of hypothesis,
    \item Using proposed hypothesis selection method reduce the number of initial hypotheses,
    \item Use pre-trained CNN to assign single label for each hypothesis,
    \item Use cross-hypothesis max-pooling to create the vector of labels found on the whole picture.
\end{enumerate}

Authors claim of getting up to 90.9\% of Mean Average Precision (which is defined in PASCAL Challenge \cite{Everingham2010PASCAL-VOC}) on PASCAL VOC 2012 image dataset.

Yang et al. proposed similar infrastructure, but extended it with additional large-margin nearest neighbor (LMNN) CNN that incorporates knowledge of ground truth bounding boxes of the training set to improve results  \cite{Yang2015}. This extra layer tries to find similar objects in the test image to the ones extracted from the training set (ground-truth object set) independently from the other layer that employs more usual CNN. Results from the both layers are taken into account in the final prediction vector.

The method described by Girshik et al. in ``Rich feature hierarchies for accurate object detection and semantic segmentation'' paper also highly rely on the ground-truth bounding box during the training phase \cite{Girshick2014}. In particular, generated hypothesis are ranged based on the overlap with ground-truth box.

Oquab et al. proposed method that also use bounding box information to produce labels for training images \cite{Oquab2014TransferringMidLevel}. Specifically, they propose to generate around 500 square patches from the picture and put a single label on each patch dependently on the overlap with the ground truth bounding box. Patches which intersect with two or more objects are filtered out, and the once which do not intersect with any objects are labeled as a background. Example training data generated from the image and ground-truth information is shown on Figure \ref{fig:patches-example}.

\begin{figure}[H]
    \centering
    \includegraphics[scale=0.35]{patches-example}
    \caption{Training data generation example using patching method \cite{Oquab2014TransferringMidLevel}}
    \label{fig:patches-example}
\end{figure}

Currently, the first type of multi-label classification methods with image partitioning show improved performance compared to the ones that use the whole pictures \cite{Wei2016HCP, Ren2016, Yang2015}. However, such methods require having either ground truth bounding box encoded in metadata in addition to labels which identifies the location of all objects on the image \cite{Chen2015ContextualizingClassification, Dong2013Subcategory-AwareClassification}, or pre-trained single-label classifier for the same set of categories \cite{Wei2016HCP}. In addition, this approach might be more suitable for searching for objects in the picture, but not perform as good to assign labels describing the whole image since it processes only part of a picture at a time. For instance, the dataset used in this study had label \textit{alone} that separates images with single objects, or \textit{landscapes} which characterize the whole picture. These kinds of categories might be challenging to classify using method that splits an image.

In contrast to previous papers, Gong et al. applied CNN to the whole picture without partitioning \cite{Gong2013DeepRanking}. The main contribution of this study was not to produce new multi-label classification method, but to compare different loss functions in multi-label classification context. Results showed that weighted approximated-ranking loss (WARP) improved the classification performance of the categories with a small number of pictures which can be an essential factor for unbalanced datasets.

Wang et al. in their research were also trying to improve multi-label image classification performance for realistic image collections \cite{Wang2016CNN-RNN:Classification}. Authors point out on issues connected with methods that transforming multi-label task into a single-label one. Researchers identify the lack of modeling semantic dependencies between categories in such methods. The main reason for this is that such approaches treat image categories completely independently, while some of the labels might occur together more often than others. To solve this problem, the authors propose to utilize Recurrent Neural Network (RNN) in addition to CNN, which can model semantic relevance between images.

Most of the presented methods use so-called transfer learning technique in different ways to improve their results and to utilize the work from the other studies to their benefit. Brief description and motivation behind this approach presented in the next section.

% \section{Caffe framework}

% pipeline of computations going through layers
% sigmoid cross entropy

\section{Transfer learning}
\label{sec:transfer-learning}

Usually, the usage of deep learning approach adds additional requirements: to have enough processing power and big enough image dataset. One of the most common methods to reduce these constraints called \textit{transfer learning}. The main principal behind it is to reuse ``knowledge'' of a pre-trained neural network on one task and transfer this knowledge to another one \cite{Pan2010TransferLearningSurvey, Oquab2014TransferringMidLevel}. This idea appeared when people noticed that first layers of most CNNs learn to trigger on the same features like corners, edges, color conjunctions, therefore there is no need to discover them in each network \cite{Zeiler2014VisualizingNets}.

There are two types of transfer learning \cite{Yosinski2014HowTransferable}:
\begin{itemize}
    \item With frozen layers, when layers copied from the pre-trained network do not change their weights during the backpropagation phase, in other words, their learning rates are set to zero,
    \item Fine-tuning, when copied layers only start with pre-trained weights, but during the training phase adjust just their values like all other layers.
\end{itemize}

Oquab et al. describe in detail the usage of transfer learning for multi-label classification. In particular, they show how pre-trained network on one category set can be used to find labels for more general categories (for example source categories might include different dog breeds while target set may contain only general label ``dog'') or even entirely new ones \cite{Oquab2014TransferringMidLevel}. Insights provided in this research will be applied in the project since target NTB dataset also has a different set of categories compared to the ones available in the standard image collections.

Recent studies also suggest that the usage of a pre-trained model can improve the generalizability of the final image classification system in contrast to the system trained on the target dataset from scratch \cite{Oquab2014TransferringMidLevel, Yosinski2014HowTransferable}. The transfer learning method employed in this project, which is described in the next chapter, allowed to reduce dataset size and performance requirements needed for successful system training.

\section{Real-world datasets}
Some datasets like PASCAL VOC or NUS WIDE are created from realistic pictures that contain several objects or people in different combination and interactions between them \cite{Everingham2010PASCAL-VOC, Chua2009NUS-WIDE}. Creators of such datasets, as well as other researchers, argue that real-world images are more likely to have multi-label nature by having both multiple entities in the picture as well as such category set that can describe image or object in several prospectives \cite{Everingham2010PASCAL-VOC, Chua2009NUS-WIDE, Wang2016CNN-RNN:Classification, Dong2013Subcategory-AwareClassification}. Wang et al. poit out that in addition to objects, such images can contain labels that represent scenes, actions, and parts of some entities \cite{Wang2016CNN-RNN:Classification}. Authors also identify the challenge of small objects on such images that can be ignored by classification methods, especially when full pictures are processed in a neural network.

Except for the picture collection itself, dataset should also contain metadata for image and a defined set of categories. While datasets described earlier consist of realistic pictures, the set of categories and therefore images were designed and moderated for the purpose of further training and comparison of image classification systems. Therefore, there is still a question on how different state of the art methods can be applied to datasets that were created for other reasons and how their metadata utilization can be maximized. In addition to realistic pictures, such datasets can have additional challenges connected with a unique set of labels, a less balanced size of categories, a big number of both systematic and random errors.

\subsection{Current state of the NTB dataset}
As it was mentioned earlier, NTB provided their labeled image collection for the purpose of these project. Currently, NTB dataset contains around one million of manually annotated pictures and more than ten millions of images without labels. Therefore, the fist usage of the automatic image classification system for the company is to propagate annotations on other images from the collection. NTB has its unique set of categories organized in a tree structure. The company uses these tags to provide a solution for pictures search to their customers. Therefore the second application of the system would be to incorporate an image classification in this search solution. The dataset is specific to Norway and to particular topics including sports, finances, politics, and media. More in-depth analysis of the NTB dataset will be presented in Section \ref{sec:dataset-analysis}.

% own set of categories, have to trafsfer them as many as possible


%
% CMPT 454: Database Systems II - A Course Overview
% Section: Query Optimization
%
% Author: Jeffrey Leung
%

\section{Query Optimization}
	\label{sec:query-optimization}
\begin{easylist}

& Notation: Given a schema $R$:
	&& The size in pages is denoted as $|R|$
	&& The number of records per page is denoted as $p_R$
	&& The size of the buffer is $B$

& Clustered indices are only applicable for terms which form a prefix of the search key
	&& E.g. The index $<A, B, C>$ can be used for the search key $A = 1, B = 3$

\clearpage
\end{easylist}
\subsection{Optimizing Equality Joins}
	\label{subsec:optimizing-equality-joins}
\begin{easylist}

& \textbf{Join:} Database operation where a single set of rows are returned, created from two relations

& E.g. \textrm{SELECT * FROM Sailors S, Reserves R WHERE R.id = S.id}

& \textbf{Simple Nested Loops (SNL) join:} Equality join implementation where nested loops compare equality values using 1-page buffers
	&& Consists of multiple nested loops, each of which loops through a table
		&&& \textbf{Outer table:} Table in an SNL join which is scanned outside the other nested loop(s)
		&&& \textbf{Inner table:} table in an SNL join which is scanned inside the other nested loop(s)
	&& Runtime:
		\begin{align*}
			|Outer| + |Outer| \cdot |Inner|
		\end{align*}
		&&& $|Outer|$ is the cost of loading each value from the outer table into memory
		&&& $|Outer| * |Inner|$ is the cost of loading each value from the inner table into memory, and comparing with an outer table value
		&&& To optimize, smaller tables should be implemented as the outer

& \textbf{Block Nested Loops (BNL) join:} Equality join implementation where nested loops compare equality values using multi-page buffers
	&& \textbf{Block:} Fixed-size set of pages representing the size of a buffer
	&& Runtime:
		\begin{align*}
			|Outer| + |Outer\ Blocks| \cdot |Inner| \\
			= |Outer| + \left\lceil \frac{|Outer|}{B-2} \right\rceil \cdot |Inner|
		\end{align*}
		&&& Two buffer pages are used; one as the output buffer and one as the input buffer for the inner table
		&&& To optimize, smaller tables should be implemented as the outer

& \textbf{Index Nested Loops (INL) join:} Equality join implementation where nested loops compare equality values, and inner loops use an index to search matching values from the outer loop
	&& Runtime:
		\begin{align*}
			|Outer| + |Outer| \cdot p_{outer} \cdot \textrm{probing costs}
		\end{align*}
		&&& $|Outer| \cdot p_{outer}$ represents the number of records in table $R$
		&&& \textbf{Probing cost:} I/O cost to search for an entry and record using an index
			&&&& Probing cost of data entries in a hash index: 1.2
			&&&& Probing cost of data entries in a B+ tree: 3-4
			&&&& Probing cost of data records when the search key is a candidate key: 1
			&&&& Probing cost of data records when the search key is not a candidate key, in a clustered index: 1
			&&&& Probing cost of data records when the search key is not a candidate key, in an unclustered index: $\frac{\textrm{records in inner}}{\textrm{records in outer}}$

	&& Equality with multiple attributes: Index must apply to one or more of the attributes
	&& Inequality: Index must be clustered B+ tree

& \textbf{Sort-merge-join (SMJ):} Equality join implementation where tables are sorted externally, written back to the disk, merged by equality, then sent to output
	&& Runtime:
		&&& Sorting:
		\begin{align*}
			  2 \cdot |T_1| \cdot (\left\lceil \log_{B-1} (\frac{|T_1|}{B}) \right\rceil + 1) \\
			+ 2 \cdot |T_2| \cdot (\left\lceil \log_{B-1} (\frac{|T_2|}{B}) \right\rceil + 1)
		\end{align*}
		where $T_1$, $T_2$ are tables and $B$ is the buffer size
		&&& Merging:
		\begin{align*}
			|T_1| + |T_2|
		\end{align*}
		&&& Optimization 1: Find the tables which have the fewest maximum count of any repeating value, then use those those tables as the outer loop (to fit as much outer loop data as possible into the input buffer)
		&&& Optimization 2: Join the runs of the sorted tables after only the first pass of the external sort
			&&&& When joining, each run is allocated a single buffer
			&&&& Only possible if $B \geq \sqrt{\textrm{max} \{ |T_1|, |T_2| \} }$
			&&&& Runtime:
			\end{easylist}
			\begin{align*}
				& \textrm{(Cost of pass 0)} + \textrm{(Merging all runs)} \\
				& = 2 (|R|+|S|) + (|R|+|S|) \\
				& = 3 (|R|+|S|)
			\end{align*}
			\begin{easylist}

	&& Equality with multiple attributes: Sort based on all joined attributes
	&& Inequality: Not applicable

& \textbf{In-memory probing by hash table:} CPU runtime optimization which hashes database table pages into buckets
	&& Reduces excess page retrieval and optimizes access speed to specific pages

& \textbf{Hash-join (HJ):} Equality join implementation where tables are partitioned into hash buckets, then the values in each hash bucket is joined with the matching values in the hash bucket from the other table
	&& Hash bucket notation:
		&&& $T_i$ represents the contents of hash bucket $i$ created by partitioning $T$
		&&& Assuming partitions are roughly equally sized: $|T_i| = \frac{|T|}{B-1}$
	&& Tables can be hash-joined in one scan if and only if $B-2 \geq \textrm{min} \{ |R_i|, |S_i| \}$ for all $R_i$, $S_i$ given tables $R$, $S$
		&&& Implies $B \geq \sqrt{\textrm{min} \{ |R_i|, |S_i| \} }$
		&&& If tables cannot be hash-joined in one scan, recursively apply hash-join to the bucket(s) which are too large to fit in memory
	&& Runtime (assuming $R_i$ or $S_i$ can fit in memory):
	\end{easylist}
	\begin{align*}
		& \textrm{Cost of partitioning both relations - read and write} + \textrm{Cost of joining relations} \\
		& = 2( |R|+|S|) + (|R|+|S|) \\
		& = 3( |R|+|S|)
	\end{align*}
	\begin{easylist}
	&& Parallelizable
	&& Sensitive to data skew

	&& Equality with multiple attributes: Hash  based on all joined attributes
	&& Inequality: Not applicable

\clearpage
\end{easylist}
\subsection{Optimizing Selections}
	\label{subsec:optimizing-selections}
\begin{easylist}

& \textbf{Selection:} Database operation where a subset of rows are returned from a relation

& Notation:
	&& \textbf{Reduction factor (RF):} Percent of data that a table is reduced by through a selection operation

& Size of result:
\begin{align*}
	|T| \cdot \textrm{RF}
\end{align*}

& Runtime:
	&& No index or not clustered on the attributes: $|T|$
	&& Clustered on the attributes: $|T| \cdot \textrm{RF}$

& Page I/O:
	&& No index or not clustered on the attributes: $|T| \cdot \textrm{RF} \cdot p_T$
		&&& Optimization: Collect and sort record IDs before retrieving them
	&& Clustered on the attributes: $|T| \cdot \textrm{RF}$
		&&& Clustered records are more likely to be on the same page

& Converted to conjunctive normal form (CNF) before processing

& Optimizing with multiple indices:
	&& Approach 1: Retrieve a set of records using the most selective/restrictive index (minimizing returned values), then apply remaining terms
	&& Approach 2: Retrieve sets of record IDs using all indices, intersect them, sort them, retrieve the records, then  apply remaining terms

\clearpage
\end{easylist}
\subsection{Optimizing Projections}
	\label{subsec:optimizing-projections}
\begin{easylist}

& \textbf{Projection:} Database operation where a subset of columns are returned from a relation
	&& Often includes a $\textrm{DISTINCT}$ operation

& \textbf{Standard projection:} Projection implementation using a modified version of the external sort Pass 0
	&& When writing during external sort Pass 0, drop unnecessary fields
	&& During external sort merging of runs, drop duplicate rows

	&& Page I/O:
	\end{easylist}
	\begin{align*}
	& \textrm{(Reading all records)} + \textrm{(Writing partial columns)} \\
	& + \textrm{(Reading the same data)} \\
	& + \textrm{(Merging non-duplicate rows)} \\
	= & |T| + |T| \cdot \textrm{(\% of data projected)} * 2 \\
	  & + |T| \cdot \textrm{(\% of data projected)} \cdot \textrm{(\% of distinct data merged)}
	\end{align*}
	\begin{easylist}

& \textbf{Hash projection:} Projection implementation where hashing is used to eliminate duplicates
	&& Process: Use a hash function on all wanted attributes to partition the table into $B-1$ partitions (while discarding unprojected fields), use a new hash function on all fields of each partition to discard duplicates

& Index terms:
	&& \textbf{Index-only projection:} Projection operation where all projected attributes are in the index's search key
		&&& Requires sorting afterwards if the projected attributes are not a prefix of the search key
		&&& Requires duplicate removal
	&& \textbf{Index-only, no sort needed projection:} Projection operation where all projected attributes constitute a prefix of the index's search key
		&&& Requires duplicate removal

\clearpage
\end{easylist}
\subsection{Optimizing Unions}
	\label{subsec:optimizing-unions}
\begin{easylist}

& \textbf{Union:} Database operation where two relations with matching columns are combined
	&& Includes removal of duplicates

& \textbf{Hash-based union:} Union implementation which utilizes two hash functions
	&& Process:
		&&& Hash both relations into partitions
		&&& Re-hash a partition from the first relation, into new partitions
		&&& Re-hash the matching partition from the second relation, into the same re-partitions (avoiding adding duplicates)

\end{easylist}
\subsection{Optimizing Differences}
	\label{subsec:optimizing-differences}
\begin{easylist}

& \textbf{Difference:} Database operation between two relations where any row from the first relation which exists in the second relation is removed

& \textbf{Hash-based difference:} Difference implementation which utilizes two hash functions
	&& Process:
		&&& Hash both relations into partitions
		&&& Re-hash a partition from the first relation, into new partitions
		&&& For each value in the second relation's matching partition:
			&&&& Re-hash it to find the corresponding partition
			&&&& Remove it from the partition if it exists

\end{easylist}
\subsection{Optimizing Aggregation}
	\label{subsec:optimizing-aggregation}
\begin{easylist}

& \textbf{Aggregation:} Database operation where numerical values in rows are combined

& If the index is clustered based on the attribute to be grouped, then no sorting will be required

& \textbf{Index-only aggregation:} Aggregation case where the search key contains all attributes to be searched and grouped, therefore no record access is required (only entries)

\end{easylist}
\subsection{Combined Operations}
	\label{subsec:combined-operations}
\begin{easylist}

& If grouping by a specific attribute:
	&& Ensure the relation is clustered/sorted on the attribute (if not, sort it externally)
	&& Use the relation with the attribute as the outer loop to preserve order

\end{easylist}
\clearpage


\subsection{Mint}
\label{sec:20Mint}

Suppose Alice wishes to be able to transfer ownership of an amount of ERC-20 tokens under zero-knowledge, so that the following become private:
	\begin{center}
		\begin{framed}
      \begin{enumerate}
        \item The value of ERC-20 tokens being transferred.
        \item The identity of the sender of the tokens (`Alice').
        \item The identity of the receiver of the tokens.
      \end{enumerate}
    \end{framed}
  \end{center}
In order to achieve this, Alice must first convert her ERC-20 tokens into a private ERC-20 commitment. We call this act of conversion \textbf{`minting'} an ERC-20 commitment.\\
\\
In this section, we outline Nightfall's protocol for minting an ERC-20 commitment, but first, an important privacy warning is given in Figure~\ref{fig:fMintWarning}.

\begin{figure}[h]
  \begin{center}
    \begin{mdframed}[backgroundcolor=verylightred]
      \noindent
      PRIVACY WARNING\\
      \\
      \textit{
      Privacy is NOT achieved during the minting stage!
      }\\
      \\
      Minting an ERC-20 commitment initially requires Alice to transfer a certain value of ERC-20 tokens to a `Shield' contract (which thereafter holds this value in escrow). This transfer reveals the Ethereum address of the sender (Alice) as well as the value. Therefore everyone will know the owner and the underlying value being represented by the initial ERC-20 commitment which is created at this `minting' stage.\\
      \\
      Only during subsequent `transfers' of the new ERC-20 commitment, will we achieve the privacy intentions of Figure~\ref{fig:fPrivacyIntentions}
    \end{mdframed}
  \end{center}
  \caption{Privacy warning: minting alone does not achieve privacy}
  \label{fig:fMintWarning}
\end{figure}

\noindent
Suppose Alice owns ERC-20 tokens of value $c$
(denominated in the ERC-20 token's currency).
Suppose Alice wishes to create a private token commitment, representing her ownership of value $c$.\\
\\
For Alice to mint a token commitment $Z_c$ representing value $c$ on the blockchain, under zero knowledge, she follows the steps in Figure~\ref{fig:fMintAlgorithm}. Note: We avoid using $Z_A$ (which was used to stress Alice's ownership in the non-fungible section), because Alice will own more than one token commitment when we explain `transfers'.
\\

\begin{figure}[htp]
	\begin{center}
		\begin{framed}
      \begin{tabular}{p{16cm}}
        \textbf{Fungible mint algorithm} \\
        \\
        \midrule
        \textbf{Alice's steps:}\\
        \begin{enumerate}
				  \item Generate a random salt $\sigma_c$.
				  \item Compute $Z_c := h(\;c\;|\;pk^Z_A\;|\;\sigma_c\;)$, a token commitment which represents $c$.
          \item Set public inputs $x = (\;c,\;Z_c\;)$
          \item Set private inputs $\omega = (\;pk_A^Z,\;\sigma_c\;)$
          \item Select $C_{nft-mint}(\;\omega,\;x\;)$ -- the set of constraints which are satisfied if and only if:
          \begin{enumerate}
            \item $Z_c$ equals $h(\;c\;|\;pk_A^Z\;|\;\sigma_c\;)$ (Proof that the commitment $Z_c$ hides the correct value $c$)
          \end{enumerate}
          \item Generate $\pi := P(\;p_C\;,\;x,\;\omega\;)$; a proof of knowledge of satisfying arguments $(\omega, x)\;s.t.\;C(\omega, x) = 1$. Recall: $p_C$ -- the proving key for $C$ -- will be stored on Alice's computer.

          The pair $(\pi, x)$ is the zk-SNARK which attests to knowledge of private inputs $\omega$ without revealing them.
          \item Send $(\pi, x)$ to the Shield contract for verification.

          Using web3: \texttt{fTokenShield.mint(proof, inputs, vkId)}
          %remember where the count (enumi) is up to and store it in ongoingEnumCounter:
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Shield contract's steps:}\\
        \begin{enumerate}
          %resume counter
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Verify the proof as correct: call a Verifier contract to verify the \texttt{(proof, inputs)} pair against the verification key represented by \texttt{vkId}.
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Verifier contract's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Compute \texttt{result = verify(proof, inputs, vkId)}.

          I.e. Verify the \texttt{(proof, inputs)} pair against the verification key.
          \item Return \texttt{result}$\in$\texttt{\{false, true\}} to the Shield contract.
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Shield contract's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item If \texttt{result = false}, revert.
          \item Else:
          \begin{enumerate}
            \item Transfer a value of $c$, on behalf of Alice, to the Shield Contract. I.e. store $c$ in escrow.
            \item Add $Z_c$ to the next empty leaf of the Merkle Tree.
            \item Recalculate the path to the root of the Merkle Tree from $Z_c$ for future users.
          \end{enumerate}
          \setcounter{ongoingEnumCounter}{\value{enumi}}
        \end{enumerate}
        \ \\
        \midrule
        \textbf{Alice's steps:}\\
        \begin{enumerate}
          \setcounter{enumi}{\value{ongoingEnumCounter}}
          \item Store relevant data in local database, including the leafindex of $Z_c$.
          \setcounter{ongoingEnumCounter}{0} %reset for next figure
        \end{enumerate}
			\end{tabular}
		\end{framed}
	\end{center}
\caption{Fungible Mint Algorithm}
\label{fig:fMintAlgorithm}
\end{figure}

\newpage
\subsubsection{Details}
\label{sec:20MintDetails}

We refer to the numbered steps of Figure~\ref{fig:fMintAlgorithm}.\\
\\
\textbf{Step $1$}
\ \\
This is handled within the \hyperref[sec:ui]{UI} microservice (or within the api-gateway).\\
\\

\textbf{Steps $2 - 4$}
\ \\
These steps are handled within \hyperref[sec:f-token-controller]{\texttt{f-token-controller.js}}.\\
\\

\textbf{Steps $5 - 6$}
\ \\
These steps are handled within a \hyperref[sec:zokrates]{ZoKrates} container.\\
\\

\textbf{Step $7$}
\ \\
This transaction is handled within \hyperref[sec:f-token-zkp]{\texttt{f-token-zkp.js}}.\\
\\

\textbf{Steps $8 - 10$}
\ \\
The Verifier contract already has stored within it the object $vk_C$ (see \hyperref[sec:trustedSetup]{Trusted Setup}). It runs a verification function $V(vk_C, \pi , x)$.
\begin{align*}
  V: (vk_C, \pi_{C,x,\omega}, x) \to \{0,1\}
\end{align*}
where:
\[
    V=
\begin{cases}
    1,& \text{if } \pi_{C,x,\omega} \text{ and } x \text{ satisfy } vk_C\\
    0,& \text{otherwise}
\end{cases}
\]
\ \\

\textbf{Steps $11 - 12$}
\ \\
If the Verifier contract returns $1$ (\texttt{true}) (verified) to the Shield contract, then the Shield contract will be satisfied with Alice's commitment, and will update its persistent states:\\
\\
Suppose the Shield contract stores an ever-increasing array, $\bm{Z}$, of all token commitments which have ever been submitted by anyone.\\
\\
Suppose, prior to Alice's mint, there are $n-1$ tokens in the tree:
\begin{align*}
  \bm{Z}_{n-1} = (Z_0, Z_1, ..., Z_{n-1})
\end{align*}
The information held within $\bm{Z}_{n-1}$ may be represented by the root hash $\roott_{n-1}$ of a Merkle Tree $M_{n-1}$:\\

\begin{align*}
  \begin{forest}
    [{$\roott_{n-1}:= h\br*{
                      h\br*{
                        h\br*{
                          h\br*{
                            Z_0,Z_1
                          },
                          ...
                        },
                        h\br*{
                          h\br*{
                            Z_{n-1},0
                          },
                          0
                        }
                      },
                      0
                    }
                  $}
      [{$ h\br*{
            h\br*{
              h\br*{
                Z_0,Z_1
              },
              ...
            },
            h\br*{
              h\br*{
                Z_{n-1},0
              },
              0
            }
          }
        $}
        [{$ h\br*{
              h\br*{
                Z_0,Z_1
              },
              ...
            }
          $}
          [{$ h\br*{
                Z_0,Z_1
              }
            $}
            [{$Z_0$}][{$Z_1$}]
          ]
          [...
            [...][...]
          ]
        ]
        [{$ h\br*{
              h\br*{
                Z_{n-1},0
              },
              0
            }
          $}
          [{$ h\br*{
                Z_{n-1},0
              }
            $}
            [{$Z_{n-1}$}][0]
          ]
          [0
            [0][0]
          ]
        ]
      ]
      [0
        [0
          [0
            [0][0]
          ]
          [0
            [0][0]
          ]
        ]
        [0
          [0
            [0][0]
          ]
          [0
            [0][0]
          ]
        ]
      ]
    ]
  \end{forest}
\end{align*}

\noindent
Now that the Shield contract has been given verification that Alice's commitment, $Z_c$, does indeed hide the value $c$, the Shield contract will do the following:\\

\begin{itemize}
  \item Append the commitment $Z_c$ to the ever-increasing array of tokens, $\bm{Z}_{n-1}$, so that $\bm{Z}_{n}=(Z_0, Z_1, ... Z_{n-1}, Z_c)$
  \item Recalculate a Merkle Root $\roott_{n}$ of $M_{n}$:
  \\
  \begin{align*}
    \begin{forest}
      [{$\roott_{n}:= h\br*{
                      h\br*{
                        h\br*{
                          h\br*{
                            Z_0,Z_1
                          },
                          ...
                        },
                        h\br*{
                          h\br*{
                            Z_{n-1},Z_c
                          },
                          0
                        }
                      },
                      0
                    }
                  $}
        [{$ h\br*{
              h\br*{
                h\br*{
                  Z_0,Z_1
                },
                ...
              },
              h\br*{
                h\br*{
                  Z_{n-1},Z_c
                },
                0
              }
            }
          $}
          [{$ h\br*{
                h\br*{
                  Z_0,Z_1
                },
                ...
              }
            $}
            [{$ h\br*{
                  Z_0,Z_1
                }
              $}
              [{$Z_0$}][{$Z_1$}]
            ]
            [...
              [...][...]
            ]
          ]
          [{$ h\br*{
                h\br*{
                  Z_{n-1},Z_c
                },
                0
              }
            $}
            [{$ h\br*{
                  Z_{n-1},Z_c
                }
              $}
              [{$Z_{n-1}$}][{$Z_c$}]
            ]
            [0
              [0][0]
            ]
          ]
        ]
        [0
          [0
            [0
              [0][0]
            ]
            [0
              [0][0]
            ]
          ]
          [0
            [0
              [0][0]
            ]
            [0
              [0][0]
            ]
          ]
        ]
      ]
    \end{forest}
  \end{align*}

  \item Append $\roott_{n}$ to an ever-increasing array $\rootsList=(\roott_0, \roott_1,..., \roott_{n-1}, \roott_{n})$
\end{itemize}

\textbf{Step $13$}
\ \\
Alice will store all important information in her private database.\\


\section{The Inverse Method}

\subsection{Requirements}

\begin{enumerate}
\item Typechecker
  \begin{itemize}
  \item Internal api, e.g. ``\%doublecheck''
  \item External api, for Deepak say.  Perhaps support explicit sharing of terms.
  \end{itemize}
\item Term Reconstruction
\item Logic programming engine (Bottom up)
\item M2 Checker (\%mode,\%worlds,\%terminates,\%covers,\%total)
\item logic programming engine (Top down)
\item Meta-theorem prover (produce .prf files from .thm files)
\end{enumerate}

\subsection{The Subformula Property}

\begin{Theorem} The cut rule is admissible in LF \end{Theorem}
\XXX{Needs reference}

It is a property of cut-free sequent calculi that all propositions occurring
in a derivation are \emph{subformulas} of the endsequent in the following sense.

We define \emph{signed subformulas} of a formula $F$ with judgments:

\begin{align*}
\PSubform{A} & & \mbox{$A$ is a positive subformula of the initial goal.}\\
\NSubform{A} & & \mbox{$A$ is a negative subformula of the initial goal.}
\end{align*}

where
$$\Psi ::= \cdot \Spb \Psi, v:A \Spb \Psi, x:A$$

Here $v$ ranges over arbitrary terms and $x$ ranges over
fixed parameters.

These rules are defined as a forward logic program by the following inference
rules:

\bigskip
\begin{tabular}{cc}
\infer{\NSubform{A}}{\PSubform{(A\LFArrow B)}} &
\infer{\NSubform{B}}{\PSubform{(A\LFArrow B)}} \\
\infer{\PSubform{A}}{\NSubform{(A\LFArrow B)}} &
\infer{\PSubform{B}}{\NSubform{(A\LFArrow B)}} \\
\infer{\PSubform[\Psi,x:A]{B}}{\PSubform{(\PiTyp{x}{A}{B})}} &
\infer{\NSubform[\Psi,u:A]{B}}{\NSubform{(\PiTyp{u}{A}{B})}} \\
\end{tabular}
\bigskip

Note that there are no rules for families of kind $\Type$.

To find the subformulas of a given goal formula $G$, we run
the above rules as a forward logic program to saturation on $\PSubform{G}$.

\subsection{Admissible Substitutions}

Explicit substitutions are an integral part of the theory and
implementation of our development.  They are necessary because
when logic variables arise during proof search, it is not always
possible to apply a substitution immediately.  While contexts
do the necessary bookkeeping of tracking the types of bound variables,
substitutions track variable (term) instantiations.
The language of substitutions is

$$\sigma ::= \circ \Spb \sigma,(M/u) \Spb \sigma, (y//x)$$

where $M$ ranges over arbitrary terms, and $y$ ranges over
parameters.

An invariant(*) of admissible substitutions is that if $\AdmSub{\sigma}{\Psi}$ then
the domain of $\sigma$ is exactly the free variables of $\Psi$.  (The codomain
need not mention all of $\Sigma$.)  Similarly, whenever we apply
$A[\Theta]$ it will be the case that $\Theta$ substitutes exactly for the
free variables of $A$.

The judgment on admissible substitutions will be
$$\AdmSub{\sigma}{\Psi}$$
read ``$\sigma$ is an admissible substitution from $\Psi$ to $\Sigma$''.
and meaning that $\sigma$ substitutes for all the free variables in
$\Psi$, but can mention variables in $\Sigma$.

\bigskip

\infer{\AdmSub{\cdot}{\cdot}}{}
\bigskip

\infer{\AdmSub{(\sigma,M/u)}{\Psi,u:A}}{\AdmSub{\sigma}{\Psi} & \CheckTy[\Sigma]{M}{A[\sigma]}}
\smallskip
when $u\not\in\Psi$
\bigskip

\infer{\AdmSub{(\sigma,y//x)}{\Psi,x:A}}{\AdmSub{\sigma}{\Psi} & \SynthTy[\Sigma]{y}{A'} & A' = \HerSub{A}}
\smallskip
when $x\not\in\Psi$
\bigskip

\subsection{Forward Sequents}

For the inverse method, we think of the usual sequent calculus rules, which work bottom-up,
backwards to work top-down.

The judgment

$$\Sequent{\Sigma}{A_1[\tau_1],\ldots,A_n[\tau_n]}{C[\sigma]}$$

where for all $i$,

\begin{align*}
  &\AdmSub{\tau_i}{\Psi_i}\\
  &\NSubform[\Psi_i]{A_i}\\
  &\AdmSub{\sigma}{\Psi}\\
  &\PSubform[\Psi]{C}
\end{align*}

can be read as (modulo constraints on substitutions)
``Given the parameters $\Sigma$ and hypotheses and derived formulas
$A_i$ under admissible substitutions $\tau_i$, the goal $C$ under admissible
substitution $\sigma$ is derivable.''

\subsection{Subsumption}

$$
(\Sequent{\Sigma}{A_1[\tau_1],\ldots,A_n[\tau_n]}{C[\sigma]}) \leq
(\Sequent{\Sigma'}{A_1[\tau_1'],\ldots,A_k[\tau_k']}{C[\sigma']})
$$

if there exists an admissible substitution $\AdmSub[\Sigma']{\Theta}{\Sigma}$
such that
$\tau_1' = \tau_1 \Comp \Theta, \ldots, \tau_k' = \tau_k \Comp \Theta, \sigma' = \sigma \Comp \Theta$

for some $n\leq k$.

\subsection{Unification}

\subsection{Most General Unifiers}
In the pattern fragment of LF, most general unifiers exist.  Therefore the
function $\mgu(x,y)$ is well defined.  Keeping our invariant (*) in mind,
we will write $\mgu = (\Theta_1,\Theta_2)$ when the domain of the intended
terms to which we apply the substitutions have different sets of free variables.
That is, if the most general unifier of $A_1,A_2$ is $\Theta$ and
$fv(A_1) \neq fv(A_2)$ then
\begin{align*}
  \Theta_1 &= \mathbf{filter}\ (\lambda(x,\_).\ \mathbf{mem}(fv(A_1,x),x))\ \Theta \\
  \Theta_2 &= \mathbf{filter}\ (\lambda(x,\_).\ \mathbf{mem}(fv(A_2,x),x))\ \Theta
\end{align*}

\subsection{Forward Sequent Rules, Pattern Fragment}

\infer[\textbf{Init}]{\Sequent{\Sigma}{P_1[\Theta_1]}{P_2[\Theta_2]}}{}
when
\begin{align*}
  &\NSubform[\Psi_1]{P_1}\\
  &\PSubform[\Psi_2]{P_2}\\
  &\MGU{(\Theta_1,\Theta_2)}{\NSubform[\Psi_1]{P_1}}{\PSubform[\Psi_2]{P_2}}\\
  &\AdmSub{\Theta_1}{\Psi_1}\\
  &\AdmSub{\Theta_2}{\Psi_2}
\end{align*}

\infer[\textbf{Contract}]{\Sequent{\Sigma'}{\Gamma[\Theta],A[\tau_1] \Comp \Theta}{C[\sigma \Comp \Theta]}}
                         {\Sequent{\Sigma}{\Gamma,A[\tau_1],A[\tau_2]}{C[\sigma]}}
when
\begin{align*}
  &\AdmSub{\tau_1}{\Psi}\\
  &\AdmSub{\tau_2}{\Psi}\\
  &\MGU[\Sigma']{\Theta}{\tau_1}{\tau_2}
\end{align*}

\infer[\textbf{ArrowR1}]{\Sequent{\Sigma'}{\Gamma[\Theta]}{(A\LFArrow B)[\tau \Comp \Theta]}}
                       {\Sequent{\Sigma}{\Gamma,A[\tau]}{B[\sigma]}}
when
\begin{align*}
  &\MGU[\Sigma']{\Theta}{\tau}{\sigma} \\
  &\AdmSub{\Theta}{\Psi} \\
  &\AdmSub{\sigma}{\Psi}\\
  &\PSubform{A \LFArrow B}
\end{align*}

\infer[\textbf{ArrowR2}]{\Sequent{\Sigma}{\Gamma}{(A\LFArrow B)[\sigma]}}
                        {\Sequent{\Sigma}{\Gamma}{B[\sigma]}}
when
\begin{align*}
  &\AdmSub{\sigma}{\Psi}\\
  &\PSubform{A \LFArrow B}
\end{align*}

\infer[\textbf{ArrowL}]{\Sequent{\Sigma}{\Gamma_1[\Theta_1],\Gamma_2[\Theta_2],(A\LFArrow B)[\tau_1 \Comp \Theta_1]}{C[\sigma \Comp \tau_2]}}
                       {\Sequent{\Sigma_1}{\Gamma_1}{A[\tau_1]} & \Sequent{\Sigma_2}{\Gamma_2,B[\tau_2]}{C[\sigma]}}
when
\begin{align*}
  &\MGU{(\Theta_1,\Theta_2)}{\AdmSub{\Theta_1}{\Sigma_1}}{\AdmSub{\Theta_2}{\Sigma_2}}\\
  &\AdmSub{\tau_1}{\Psi}\\
  &\AdmSub{\tau_2}{\Psi}\\
  &\NSubform{A \LFArrow B}
\end{align*}

\infer[\textbf{PiL}]{\Sequent{\Sigma}{\Gamma, (\PiTyp{u}{A}{B})[\tau]}{C[\sigma]}}
                    {\Sequent{\Sigma}{\Gamma, B[\tau,M/u]}{C[\sigma]}}
when
\begin{align*}
  \NSubform{\PiTyp{u}{A}{B}}
\end{align*}

\infer[\textbf{PiR}]{\Sequent{\Sigma}{\Gamma}{(\PiTyp{x}{A}{B})[\sigma]}}
                    {\Sequent{\Sigma,y:A'}{\Gamma}{B[\sigma,y//x]}}

when
\begin{align*}
  &y\not\in \Gamma \\
  &A' = A[\sigma] \\
  &\PSubform{\PiTyp{x}{A}{B}}
\end{align*}

%% \subsection{Junk}

%% \newcommand{\init}{(\Gamma_0^-,C_0^+)}

%% \infer[$Atom$]{P \vdash P}{}
%% \bigskip
%% when $P \leq^+ \init$ and $P \leq^- \init$
%% \bigskip

%% \infer[$Contract$]{\Gamma,A \vdash B}{\Gamma,A,A\vdash B}
%% \bigskip

%% \infer[\to$R1$]{\Gamma\vdash A \LFArrow B}{\Gamma,A \vdash B & (A\LFArrow B \leq^+ \init)}
%% \bigskip

%% \infer[\LFArrow$R2$]{\Gamma\vdash A \LFArrow B}{\Gamma \vdash B & A \not\in\Gamma & (A\LFArrow B \leq^+ \init)}
%% \bigskip

%% \infer[\LFArrow$L$]{\Gamma_1,\Gamma_2,A \LFArrow B\vdash C}{\Gamma_1,\vdash A,& \Gamma_2,A \vdash B & (A\LFArrow B \leq^- \init)}
%% \bigskip

%% \subsection{The First Order Case}

%% \newcommand{\seq}[3][\Sigma]{#1 ; #2 \vdash #3}
%% \newcommand{\unif}[3][\Sigma]{#1 ; #2 = #3}
%% \newcommand{\renaming}[1]{#1\ \mathsf{renaming}}

%% General Form:

%% $$\Sigma; \Gamma[\tau] \vdash C[\sigma]$$

%% which is shorthand for

%% $$\Sigma; A_1[\tau_1],\ldots,A_n[\tau_n] \vdash C[\sigma]$$

%% \bigskip

%% \infer[$Atom$]{\seq{P_1^-[\rho\circ\tau]}{P_2^+[\tau]}} {}
%% \bigskip
%% when $\unif{P_1^-[\rho\circ\tau]}{P_2^+[\tau]}$ and $\renaming{\rho}$
%% \bigskip

%% \infer[\Pi$-R$]{\seq{\Gamma}{}}{}
%% \bigskip


\subsection{Functions between topologies}

We can define a function from topology to another.

\(f(X)=Y\)

\subsubsection{Continuous functions between topologies}

If \(f(X)\) is continous, then we have a continous function between topologies.

\subsubsection{Inverse functions between topologies}

If \(f(X)\) is invertible then there is a inverse mapping.


\documentclass[journal=jpcbfk,manuscript=article,layout=singlecolumn,articletitle=true]{achemso}
\setkeys{acs}{articletitle = true}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Place any additional packages needed here.  Only include packages
%% which are essential, to avoid problems later.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\usepackage{graphicx,bm,lineno,mathpazo}
\usepackage{amssymb,amsfonts,amsmath,mathtools}
\usepackage{algorithm,algpseudocode,booktabs,dcolumn}
\captionsetup{labelfont=bf}
\usepackage[input-decimal-markers={.},separate-uncertainty=true,table-auto-round,scientific-notation = false,zero-decimal-to-integer=true]{siunitx}
\DeclareMathOperator*{\tr}{tr}
\DeclareMathOperator*{\argmax}{arg\,max}
\usepackage[caption=false,font=normalsize,position=top]{subfig}
\usepackage[kerning,spacing,tracking]{microtype}
\microtypecontext{spacing=nonfrench}
\usepackage{indentfirst}
\overfullrule=5pt
\usepackage[normalem]{ulem}
\usepackage{color}
\usepackage{empheq}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% New command
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newcommand{\boxedeq}[2]{\begin{empheq}[box={\fboxsep=6pt\fbox}]{align}\label{#1}#2\end{empheq}}

\title{EM: From Langevin-Dynamics}
\author{Yi-Tsao Chen}
\date{December 2019}

\begin{document}

\maketitle

\section{Langevin-Dynamics}
Given a potential mean force $V(x)$ and a diffusion coefficient $D$, we simulate a trajectory, $X(t)$, from
\begin{equation}
\label{eq:ODLD}
dx_t=D F(x_t)dt  + \sqrt{2D}d{W}_t,
\end{equation}

\section{Formulate the problem as Hidden Markov Model}
We denote a particular configuration as $(\textbf{x}, \textbf{y})$:
\begin{equation}
(\textbf{x}, \textbf{y})=(x_0, x_1, \cdots, x_T, y_0, y_1, \cdots, y_T)
\end{equation}
The joint probability is:
\begin{equation}
p(\textbf{x}, \textbf{y})= p(x_0)\prod_{t=0}^{T-1} p(x_{t+1}|x_t) \prod_{t=0}^{T}p(y_t|x_t)
\end{equation}

\begin{figure}[h]
\includegraphics{figures/graphical.pdf}
\caption[Graphical Model]{Graphical Model}
\label{domaindecomp}
\end{figure}

\section{Inference}
\subsection{state representation}
We represent the state at time $t$ as a multinomial random varialbe $x_{t}$, with components $x_t^i$ for $i=0,...,M$. Thus $x_t^i$ is equal to one for a particular value of $i$ and is equal to zero for $j \neq i$.
\begin{equation}
x_t =
\begin{bmatrix}
x_t^0 & x_t^1 & x_t^2 &\cdots & x_t^M
\end{bmatrix}^T
\end{equation}
For example, if the state is in the second state,
\begin{equation}
x_t =
\begin{bmatrix}
0 & 0 & 1 & \cdots & 0
\end{bmatrix}^T
\end{equation}
\subsection{alpha-beta}
We focus on a particular state node $x_t$ and ask to calculate its posterior probability, $p(x_t| \textbf{y})$. And by Bayes' theorem,
\begin{equation}
p(x_t | \textbf{y}) = \frac{p(\textbf{y}|x_t) p(x_t)}{p(\textbf{y})}
\end{equation}
Then, conditioning on $x_t$ and use conditional independence:
\begin{equation}
p(x_t | \textbf{y}) = \frac{p(y_0,\cdots,y_t|x_t) p(y_{t+1},\cdots,y_T|x_t) p(x_t)}{p(\textbf{y})}
\end{equation}
Regroup the terms
\begin{equation}
\begin{split}
p(x_t | \textbf{y}) &= \frac{p(y_0,\cdots,y_t,x_t) p(y_{t+1},\cdots,y_T|x_t)}{p(\textbf{y})}\\
&=\frac{\alpha(x_t)\beta(x_t)}{p(\textbf{y})}
\end{split}
\end{equation}
where
\begin{equation}
\alpha(x_t) \triangleq p(y_0,\cdots,y_t,x_t)
\end{equation}
is the probability of emitting a partial sequence of outputs $y_0,\cdots,y_t$ and ending up in state $x_t$, and
\begin{figure}[h]
\includegraphics{figures/alpha_example.pdf}
\caption[Alpha]{The illustration of $\alpha(x_2)=p(y_0,y_1,y_2,x_2)$}
\label{alpha}
\end{figure}
\begin{equation}
\beta(x_t) \triangleq p(y_{t+1},\cdots,y_T|x_t)
\end{equation}
is the probability of emitting a partial sequence of outputs $y_{t+1},\cdots,y_T$ given that the system starts in state $x_t$.
\begin{figure}[h]
\includegraphics{figures/beta_example.pdf}
\caption[Beta]{The illustration of $\beta(x_2)=p(y_3,y_4|x_2)$}
\label{beta}
\end{figure}

\begin{figure}[b]
\includegraphics{figures/graphical_example.pdf}
\caption[Graphical Model]{The example of forward-backward}
\label{domaindecomp}
\end{figure}

\subsection{Likelihood function}
The sum of $p(x_t|\textbf{y})$ over the possible values of $x_t$ must equal one.
\begin{equation}
1 = \sum_{x_t} p(x_t|\textbf{y}) = \frac{\sum_{x_t} \alpha(x_t)\beta(x_t)}{p(\textbf{y})}
\end{equation}
So, the likelihood function $p(\textbf{y})$ can be written as
\begin{equation}
p(\textbf{y})=\sum_{x_t} \alpha(x_t)\beta(x_t)
\end{equation}

\subsection{Posterior probability $p(x_t | \textbf{y})$}
We denote the posterior probablity $p(x_t | \textbf{y})$ as $\gamma(x_t)$
\begin{equation}
\gamma(x_t) \triangleq \frac{\alpha(x_t)\beta(x_t)}{p(\textbf{y})}
\end{equation}

\subsection{Forward algorithm}
\begin{equation}
\alpha(x_{t+1}) = \sum_{x_t} \alpha(x_t) a_{x_t,x_{t+1}} p(y_{t+1}|x_{t+1})
\end{equation}
where $a_{x_t,x_{t+1}}$ is the transition probability from $x_t$ to $x_{t+1}$. The definition of alpha at the first time step yields:
\begin{equation}
\alpha(x_0) = p(y_0, x_0) = p(y_0|x_0)p(x_0) = p(y_0|x_0) \pi_{x_0}
\end{equation}
and these values are used to initialize the recursion.

\subsection{Backward algorithm}
\begin{equation}
\beta(x_t) = \sum_{x_{t+1}} \beta(x_{t+1}) a_{x_t,x_{t+1}} p(y_{t+1}|x_{t+1})
\end{equation}
The beta recursion is a backwards recursion; that is, we start at the final time step $T$ and proceed backwards to the initial time step.

Some remarks:
\begin{enumerate}
\item $\beta(x_T)$ is unhelpful because it makes reference to a non-existent $y_{T+1}$
\item If we assign $\beta(x_T)$ to be a vector of ones, i.e., $\beta(x_T)=\textbf{1}$, we can compute $\beta(x_{T-1})$ correctly
\begin{equation}
\begin{split}
\beta(x_{T-1}) &= p(y_T|x_{T-1}) \\
&= \sum_{x_T} \beta(x_T) a_{x_{T-1}, x_T} p(y_T|x_T) \\
&= \sum_{x_T} a_{x_{T-1}, x_T} p(y_T|x_T)
\end{split}
\end{equation}
\end{enumerate}

\subsection{Computing $p(\textbf{y})$ by a single forward pass for the alphas}
\begin{equation}
\begin{split}
p(\textbf{y}) &= \sum_{i} \alpha(x_T^i) \beta(x_T^i) \\
&= \sum_{i} \alpha(x_T^i) \\
&= \sum_{i} p(y_0,y_1,\cdots,y_T,x_T^i)
\end{split}
\end{equation}
\end{enumerate}

\section{Expectation maximization}
Preparing an initial potential mean force $V_0(x)$ and a diffusion coefficient $D_0$, and try to using the framework in the 2013 JPCB paper to learn $V(x)$ and $D$ back.

\section{Question}
\begin{enumerate}
\item If we ignore the information of photons, what is the observed data, $Y(t)$?
\item What is the likelihood functional?
\begin{equation}
\label{eq:PathP}
\begin{split}
\mathcal{P}(Y(t) ; F(x),D) &= \mathcal{P}(Y(t) ; \theta)= \mathcal{L}[\theta]\\
&=\langle \alpha_{t_0} |e^{-\bm{H}\Delta t_1}\bm{y}_1e^{-\bm{H}\Delta t_2}\bm{y}_2 \ldots e^{-\bm{H}\Delta t_{N_{\mathrm{P}}}}\bm{y}_{N_{\mathrm{P}}} | \beta_{t_{\mathrm{exp}}} \rangle
\end{split}
\end{equation}
\end{enumerate}

\section{Appendix A: Bayes' theorem}
\begin{equation}
p(A|B) = \frac{p(B|A)p(A)}{p(B)}
\end{equation}

\end{document}


\chapter{Rational Equations}

\section{Unit 4}


% arara: pdflatex
% arara: bibtex
% arara: pdflatex
% arara: pdflatex

\documentclass[a4paper,11pt]{article}
\pdfoutput=1 % if your are submitting a pdflatex (i.e. if you have
             % images in pdf, png or jpg format)

\usepackage[a4paper,
total={170mm,257mm},
left=20mm,
top=20mm,]{geometry}

\usepackage[dvipsnames]{xcolor}
\usepackage{longtable}
\usepackage{authblk}
\usepackage{hyperref}

\usepackage{titlesec}
\titleformat{\section}[block]{\color{blue}\Large\bfseries\filcenter}{}{1em}{}
\titleformat{\subsection}[hang]{\bfseries}{}{1em}{}

\title{\textcolor{red}{\texttt{\Huge TpMp}} - topology mapping}

\author{Andrey Pikelner}
\affil{II.~Institut f\"ur Theoretische Physik, Universit\"at Hamburg,\\
Luruper Chaussee 149, 22761 Hamburg, Germany}

% \emailAdd{andrey.pikelner@desy.de}

% Mathematica commands in blue
\newcommand{\mma}[1]{\textcolor{BlueGreen}{\texttt{#1}}}
% Color tt
\newcommand{\ctt}[1]{\textcolor{OliveGreen}{\texttt{#1}}}

\setcounter{secnumdepth}{0}
\begin{document}
\maketitle

\abstract{We present program for diagram generation and mapping on
  topology working with \texttt{Mathematica} interface. Available from
  \url{https://github.com/apik/TpMp}}

\section*{Design ideas}

\subsection*{QGRAF input conventions}
\begin{itemize}
\item We use all particles as internal in QGRAF and having momentum
  $p_1,\dots,p_n$. Field out kept blank.
\begin{verbatim}
   in  = q, Q ;
   out = ;
\end{verbatim}

\item All fields in model file must contain field \ctt{type} with
  possible values \ctt{F,M,S,C,A} in addition to
  comutativity flag $\pm$.
\begin{verbatim}
   [ phi, phi, + ; type='S']
\end{verbatim}

\item We set internal momentum to be \ctt{k}
\end{itemize}

\subsection*{DB storage}
\label{sec:dbstore}

\section*{Main commands available in package }

\subsection*{Loading diagrams}
\begingroup
\renewcommand\arraystretch{2}
\begin{longtable}{p{4cm}|p{11cm}}
  \mma{LoadQGRAF["qlist"]}
  & Load \texttt{QGRAF}\cite{Nogueira:1991ex} output \texttt{qlist.yaml} in YAML format produced with the help of
    yaml.sty style file and produce SQLite 3 DB with generated
    diagrams and name \texttt{qlist.sqlite3}.\\
  \mma{nDB=LoadDB["db"]}
  & Load SQLite 3 DB from file \texttt{db.sqlite3} with diagrams and
    return numeric descriptor \texttt{nDB} of open DB.\\
  \mma{GetDia[nDia,nDB]}
  & Retrive diagram with number \texttt{nDia} from DB with descriptor
    \texttt{nDB}.\\
\end{longtable}
\endgroup

When DB loaded all commands need DB decriptor to be specified. If DB
descriptor not specified explicitly - all commands applyed to first
open DB.

\subsection*{Selecting diagrams}

\begingroup
\renewcommand\arraystretch{2}
\begin{longtable}{p{4cm}|p{11cm}}
  \mma{WithField["f"]}
  & Select diagrams with field named "f".\\
  \mma{WithFieldType[T]}
  & Select diagrams with field type \texttt{T}, where \texttt{T} is
  one of: \texttt{F} - for Dirac fermion, \texttt{M} - for Majorana
  fermion, \texttt{S} - for scalar, \texttt{C} - for ghost and
  \texttt{A} - for auxiliary field type.\\
\end{longtable}
\endgroup

\subsection*{Plotting diagrams}

\section{Mapping}
\subsection{\boldmath $U$ and $F$ polynomials}

If we define $P_i=-q_j^2+m_j^2$, than:
\begin{equation}
  \label{eq:MQJdefMass}
  \sum\limits_{j=1}^{n}x_j(-q_j^2+m_j^2)=-\sum\limits_{r=1}^{l}\sum\limits_{s=1}^{l}k_rM_{rs}k_s+\sum\limits_{r=1}^{l}2k_r\cdot
  Q_r+J
\end{equation}
Term $J$ contain all dependence on external momentum products and
masses and can be decomposed:

\begin{equation}
  \label{eq:Jdecompose}
  J=J^{p}+J^{m} = J^{p}(p_i^2,p_i\cdot p_j)+\sum\limits_{j=1}^{n}x_jm_j^2
\end{equation}

If only information about momentum distribution needed we can neglect
masses and use $P$ polynomial instead of $F$

\begin{equation}
  \label{eq:MQJdefNoMass}
  -\sum\limits_{j=1}^{n}x_jq_j^2=-\sum\limits_{r=1}^{l}\sum\limits_{s=1}^{l}k_rM_{rs}k_s+\sum\limits_{r=1}^{l}2k_r\cdot
  Q_r+J^{p}
\end{equation}

And Feynman graph polymnomials defined as
\begin{equation}
  \label{eq:UFMdef}
  U=\det(M),\quad P=\det(M)(J^p+QM^{-1}Q),\quad F=\det(M)(J+QM^{-1}Q).
\end{equation}

$U$ polynomial determine internal structure of diagram, $P$ polynomial
determines external momenta distribution and kinematic of the diagram
and $F$-polynomial determines how masses are distributed between lines.

\begin{verbatim}
<|-x[1] ->
        <|{} -> <|{1} -> {{{}, T12[{1, k1, m1}]}}|>|>,

-x[1] - x[2] ->
        <|{{1, 1}} ->
             <|{1,  1} -> {{{{1, 1} -> -p1^2}, T12[{1, k1,      m1}, {2, k1 + p1, m2}]}},
               {0,  1} -> {{{{1, 1} -> -p1^2}, T02[{2, k1 + p1, m2}, {1, k1, 0}]}},
               {1,  0} -> {{{{1, 1} -> -p1^2}, T10[{1, k1,      m1}, {2, k1 + p1, 0}]}},
               {0,  0} -> {{{{1, 1} -> -p1^2}, T00[{1, k1,       0}, {2, k1 + p1, 0}]}}|>|>|>
\end{verbatim}

\subsection{External momentum flow}
\label{sec:extflow}

It is possible to map diagram with $p_1,\dots,p_n$ external momentum on topology
with $p_1,\dots,p_k,k<n$ external momenta. This mean that momenta flow
for $p_{k+1},\dots,p_n$ kept according to flow in original
diagram. Momentum to be kept should be marked as optional parameter of
\ctt{MapOnAux} function as \ctt{SplitMomenta->\{p2,p3,...\}}.

\subsection{Internal structure}

For set of diagrams it is possible to define ``projections'' i.e.
diagrams with contraints on kinematical variables: masses and external
momentums.

Projections created with the help of command \texttt{Project["pr",subs]} where
parameters \texttt{"pr"} - unique name of projection and \texttt{subs}
- substitution rules.

% \appendix
% \section{Additions}
% \acknowledgments

\bibliographystyle{plain}
\bibliography{TpMp}

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\documentclass[11pt]{scrartcl} % Font size
\input{structure.tex} % Include the file specifying the document structure and custom commands

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------

\title{
	\normalfont\normalsize
	\textsc{Harvard Privacy Tools Project}\\ % Your university, school and/or department name(s)
	\vspace{25pt} % Whitespace
	\rule{\linewidth}{0.5pt}\\ % Thin top horizontal rule
	\vspace{20pt} % Whitespace
	{\huge Generalized Resize Notes}\\ % The assignment title
	\vspace{12pt} % Whitespace
	\rule{\linewidth}{2pt}\\ % Thick bottom horizontal rule
	\vspace{12pt} % Whitespace
}

\author{} % Your name

\date{} % Today's date (\today) or a custom date

\begin{document}

\maketitle

\textbf{This is a work in progress.}

\section{Goals}
The generalized resize component is a means of jointly achieving a few different goals:
\begin{enumerate}
    \item guarantee known $n$,
    \item give users flexibility in how they trade off between bias and privacy usage, and
    \item allow for a combination of c-stability and privacy amplification from subsampling.
\end{enumerate}

\section{Algorithm Statement}
\label{sec:algorithm_statement}
The function will take the following inputs:
\begin{enumerate}
    \item $X$: The private underlying data.
    \item $\tilde{n}$: The size of the private underlying data.
    \item $n$: The desired size of the new data.
    \item $p$: The proportion of the underlying data that can be used to construct the new data. Can be $> 1$.
    \item $...$: Various arguments explaining imputation rules (not of interest for this doc)
\end{enumerate}

Let $sample(Y, m)$ be a function that samples $m$ elements from data set $Y$ without replacement.
Let $Aug(Y, m, \hdots)$ be a function that imputes new elements independent of the data (using imputation parameters given by $\hdots$) for a data set $Y$ until it is of size $m$.
The algorithm will look something like the following:
\begin{algorithm}[H]
    \caption{Generalized Resize: resize(X, n, p, neighboring, ...)}
    \label{alg:gen_resize}
    \begin{algorithmic}[1]
        \State $c \gets \lceil p \rceil$ \Comment{sets c-stability property}
        \State $s \gets p/c$ \Comment{sets subsampled\_proportion property}
        \State $X_c \gets \bigcup_{i=1}^{c}X$ \Comment{create new database of size $c\tilde{n}$, composed of $c$ copies of $X$}
        \If{neighboring == ``replace one''}
            \State $m \gets \lfloor sc\tilde{n} \rfloor$ \Comment{number of records that can be filled using subsampled private data}
        \ElsIf{neighboring == ``add/remove one''}
            \State $m \gets Binomial(c\tilde{n}, s)$ \Comment{number of records that can be filled using subsampled private data}
        \EndIf
        \State $(\epsilon', \delta') \gets \left(\log\left(1+s\left(e^{c\epsilon}-1\right) \right), s\left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)\delta \right)$ \Comment{privacy amplification via subsampling}
        \State $X' \gets sample\left( X_c, \max(m, n), neighboring \right) \bigcup \left( Aug(\varnothing, \max(0, n - m), \hdots) \right)$
        \\ \Return $(X', \epsilon', \delta')$
    \end{algorithmic}
\end{algorithm}

The $\epsilon', \delta'$ terms come from first applying the group privacy definition with group size $c$ to the
database $X_c$ to get $\left(c\epsilon, \left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)\delta \right)$ \cite{Vad17}
and then applying privacy amplification by subsampling results from Theorems 8 and 9 of \cite{BBG18}.
Note that, for the ``replace one'' definition, we could be using $\frac{m}{c\tilde{n}}$ instead of $s$ in the privacy calculation. Using $s$ gives us a very slightly
worse privacy guarantee (the only difference is the $\lfloor \cdot \rfloor$ we used to get $m$), but is nice for
consistency between the methods and not having to keep track of $m$ as an extra property.

\section{Functional Privacy Parameters}
We established in section~\ref{sec:algorithm_statement} that a user asking for an $(\epsilon, \delta)$-DP guarantee will get an
$(\epsilon', \delta')$-DP guarantee with respect to the original private data. What we'd really like, however, is for the user to ask for an $(\epsilon, \delta)$-DP
gurantee and have the library come up with what we will call a \emph{functional $(\epsilon'', \delta'')$} that will ensure $(\epsilon, \delta)$-DP
on the original data. Any components that operate on the resized data will use the \emph{functional $(\epsilon'', \delta'')$} internally instead of
the parameters passed by the user.

\begin{theorem}
    A mechanism that respects
    \[ \left( \frac{1}{c}\log\left(\frac{e^{\epsilon'}-1}{s} + 1 \right), \frac{\delta'}{s\left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)} \right)\text{-DP} \]
    on the resized data respects $(\epsilon, \delta)$-DP on the true private data.
    \begin{proof}
        We know that an $(\epsilon, \delta)$-DP on the resized data corresponds to a
        \[ (\epsilon', \delta') = \left(\log\left(1+s\left(e^{c\epsilon}-1\right) \right), s\left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)\delta \right) \]
        guarantee on the private data. We just need to invert the function to find $(\epsilon, \delta)$ in terms of $(\epsilon', \delta')$. \newline

        Let's start with $\epsilon, \epsilon'$:
        \begin{align*}
            \epsilon' &= \log\left(1+s\left(e^{c\epsilon}-1\right)\right) \\
            e^{\epsilon'} &= 1+s\left(e^{c\epsilon}-1\right) \\
            \frac{e^{\epsilon'}-1}{s} &= e^{c\epsilon}-1 \\
            \frac{1}{c}\log\left(\frac{e^{\epsilon'}-1}{s} + 1 \right) &= \epsilon.
        \end{align*}

        We carry out a similar calculation for $\delta, \delta'$:
        \begin{align*}
            \delta' &= s\left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)\delta \\
            \frac{\delta'}{s\left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)} &= \delta.
        \end{align*}
    \end{proof}
    So, in order for a mechanism to respect $(\epsilon, \delta)$-DP on the original data, it must respect
    $\left( \frac{1}{c}\log\left(\frac{e^{\epsilon}-1}{s} + 1 \right), \frac{\delta}{s\left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)} \right)$-DP
    on the resized data.
\end{theorem}

We now present an mini-algorithm for finding the \emph{functional} $(\epsilon, \delta)$.
\begin{algorithm}
    \caption{Finding Functional $(\epsilon_f, \delta_f)$: get\_func\_priv($p, \epsilon, \delta$)}
    \label{alg:finding_functional_privacy}
    \begin{algorithmic}[1]
        \State $c \gets \lceil p \rceil$ \Comment{sets c-stability property}
        \State $s \gets p/c$ \Comment{sets subsampled\_proportion property}
        \State $\epsilon_{f} \gets \frac{1}{c}\log\left(\frac{e^{\epsilon}-1}{s} + 1\right)$
        \State $\delta_{f} \gets \frac{\delta}{s\left(\sum_{i=1}^{c-1}e^{i \epsilon}\right)}$
        \\ \Return $(\epsilon_{f}, \delta_{f})$
    \end{algorithmic}
\end{algorithm}

\section{Examples} \label{sec:examples}
Let $X$ be such that $\tilde{n} = \vert X \vert = 100$. We can look at a few examples of calls to the generalized resize function (which will return $X'$) and check the behavior.
\begin{enumerate}
    \item resize(X, 150, 1, ...): $X'$ will be made up of the 100 true elements of $X$ and 50 imputed values. The functional privacy parameters are identical to the ones the user provides.
    \item resize(X, 100, 0.75, ...): $X'$ will be made up of 75 true elements of $X$ and 25 imputed values. The functional privacy parameters will benefit (lower noise) from amplification via subsampling.
    \item resize(X, 90, 1.5, ...): $X'$ will be a random sample of $X \bigcup X$ of size 90.
            The functional privacy parameters will lead to greater noise than what the user provides, as they have to take into account the new c-stability of $2$.
            This example is illustrative in that is shows that the functional privacy usage is affected only by $p$ -- it has nothing to do with the relative sizes of the $X$ and $X'$.
\end{enumerate}

\bibliographystyle{alpha}
\bibliography{generalized_resize}

\end{document}


\subsection{Multilinear forms}


\section{\textbf{TreeTest}}

\subsection{Particular Case (problem)}
This problem belongs to chapter 12, where the idea is to present
problems which look inherently serial but that surprisingly accept
interesting concurrent solutions (though not always easy to
explain). The particular problem this test is exercising is that of
shared
counting, where we have \C{N} threads to increase a shared numeric
variable up to certain value; but with the goal of producing less
contention than a serial solution would generate.

\subsection{Solution}

The java program is called \C{Tree.java}, although it corresponds to
the \C{CombiningTree} from the book, which is a binary tree structure
where each thread is located at a leaf and at most two threads can
share a leaf. The shared counter is at the root of the tree, and the
rest of the nodes serve as intermediate result points. When a couple
of threads collide in their attempt to increment, only one of them
will serve as the representative and go up in the tree with the mission
of propagating the combined increment (2) up to the shared counter at
the root of the tree. In its way up, it may encounter further threads
that collide again with it, making it wait or continue its journey to
the root. \\

When a thread reaches the root it adds its accumulated result to the
shared counter, and propagates down the news that the job is done to
the rest of the threads that waited along the way. This
solution to the counting problem has worse latency than lock-based
solutions (\bigO{1} vs \bigO{log(p)}, where $p$ is the number of
threads or processors; but it offers a better throughput.

\subsection{Experiment Description}

The program consists of a single unit test \C{testGetAndIncrement},
which creates 8 threads to perform each $2^20$ increments. The program
uses an auxiliary array called \C{test} to record the individual
increments done by each thread; assertions are made to ensure that
none of the attempts results in a duplicated value, as well as to
ensure that all threads completed their task.

\subsection{Observations and Interpretations}

The test performs without much controversy in an elapsed time between
3 and 4 seconds (laptop computer with i5 processor). Below a sample
output:

\begin{verbatim}
.Parallel, 8 threads, 1048576 tries

Time: 3.641
\end{verbatim}

To make the test a little more interesting, we created another couple
of tests reusing sample template of existing one; difference was the
Thread class used on each case. \\

The first additional test uses a thread class based on Java provider \\
\C{java.util.concurrent.atomic.AtomicInteger}: \\

\begin{lstlisting}[style=nonumbers]
  cnt2 = new AtomicInteger();

  class MyThread2 extends Thread {
    public void run() {
      for (int j = 0; j < TRIES; j++) {
        int i = cnt2.getAndIncrement();
        if (test[i]) {
          System.out.printf("ERROR duplicate value %d\n", i);
        } else {
          test[i] = true;
        }
      }
    }
  }
\end{lstlisting}
\hfill

while the second additional test was based on a custom class that used
synchronized \C{getAndIncrement} method:\\

\begin{lstlisting}[style=nonumbers]
  class MyThread3 extends Thread {
    public void run() {
      for (int j = 0; j < TRIES; j++) {
        int i = cnt3.getAndIncrement();
        if (test[i]) {
          System.out.printf("ERROR duplicate value %d\n", i);
        } else {
          test[i] = true;
        }
      }
    }
  }

  class MyCounter {
    private int cnt = 0;

    public synchronized int getAndIncrement()
    {
      return cnt++;
    }
  }
\end{lstlisting}
\hfill

The expectation was that the test test based on a synchronized method
would be the slowest (\C{Parallel3} label on output), the one based on
the \C{CombineTree} would become second (\C{Parallel1} label on
output) and the fastest would be the one based on \C{AtomicInteger}
(\C{Parallel2}); as the latest is likely to take advantage of hardware
atomic instruction of 32bits. Surprisingly, the test based on
\C{CombineTree} was the worst of all, by far: \\

\begin{verbatim}
.Parallel1, 8 threads, 1048576 tries took 3878
.Parallel2, 8 threads, 1048576 tries took 162
.Parallel3, 8 threads, 1048576 tries took 723

Time: 4.827

OK (3 tests)
\end{verbatim}
\hfill

Most likely we are not comparing apples with apples, as the theory
does not match the experiment; perhaps the \C{CombineTest} class is
not meant for real usage, so this comparison is not fair. Anyway, the
test does not hang nor fails on neither the 2 cores nor the 24 machines.