\documentclass{article}

\usepackage{geometry}
\usepackage{graphicx}

\usepackage{amsmath}
\usepackage{amssymb}

\newcommand{\ex}{\mathbb{E}}
\newcommand{\pr}{\mathbb{P}}

\usepackage{natbib}
\usepackage{hyperref}
\usepackage{cleveref}

\title{CS267 Final Project: Distributed Stochastic Gradient Langevin Dynamics on MPI}
\author{Feynman Liang}
\date{2018/05/08}

\begin{document}

\maketitle

\section{Introduction}

In Bayesian statistics, one oftentimes need to estimate expectations with respect
to a posterior distribution $\pr(\theta \mid X)$ over parameters $\theta$ after
observing data $X$. For example, the posterior mean
\begin{align}
  \hat\theta(X) &= \ex[\theta \mid X]
\end{align}
is a common quantity of interest because it minimizes mean squared error loss.

Bayesian statistical models consist of a specification of a prior distribution $\pr(\theta)$
over the model parameters as well as a likelihood $\pr(X \mid \theta)$ yielding the probability
of the data $X$ under parameter settings $\theta$. By Bayes rule, the posterior distribution
is given by
\begin{align}
  \label{eq:bayes}
  \pr(\theta \mid X) = {\pr(X \mid \theta) \pr(\theta) \over \int \pr(X \mid \theta') \pr(\theta') d \theta'}
\end{align}
Unfortunately, closed-form solutions of the integral in the denominator
(aka the \emph{partition function}) are not always available, precluding
analytical computation of expectations with respect to the posterior.

In light of this complication, a common practice is to perform Monte Carlo.
If a method to generate posterior samples $\theta_i \sim \pr(\cdot \mid X)$ were available,
then the law of large numbers and continuous mapping theorem can be used to argue
that for any continuous function $f$
\begin{align}
  n^{-1} \sum_{i=1}^n f(\theta_i) \overset{a.s.}{\to} \ex_{P(\cdot \mid X)} f(\theta_i)
\end{align}
as $n \to \infty$.

Generating posterior samples $\theta_i$ is again complicated by the intractable denominator
in \cref{eq:bayes}, as the inability to compute probability densities / cumulative distribution
functions prevents the usage of common sampling methods such as rejection sampling / inverse transform sampling.
Instead, researchers have traditionally used Monte Carlo Markov Chain (MCMC) \citep{metropolis1949monte}
which avoids computation of the denominator.

While MCMC has enabled significant progress, it crucially requires a computation of an
acceptance probability
\begin{align}
  \alpha_\theta = {P(\theta' \mid X) \over P(\theta \mid X)}
\end{align}
where the entire dataset $X$ is utilized. As data sets grow larger in this era of big data,
this global computation required for each iteration of the algorithm
becomes a bottleneck and limits the applicability of MCMC.

A recently developed method which avoids this problem is Stochastic Gradient Langevin Dynamics (SGLD)
\citep{welling2011bayesian}, which is capable of posterior sampling while leveraging only a subset
of the data at each iteration. This property makes SGLD a prime candidate for solving
the scalability problem in Bayesian statistics, resulting in a large amount of recent researcher
interest and development of extensions such as distributed implementations (D-SGLD) \citep{ahn2014distributed}
and samplers where $\theta$ is restricted to the probability simplex
(Riemannian Langevin dynamics, SGRLD) \citep{patterson2013stochastic}.

\section{Project Goals and Deliverables}

We develop an implementation of D-SGLD which leverages MPI for coordination of multiple processors.
We also investigate the following extensions and modifications:
\begin{itemize}
  \item Trajectory sampling, to solve problems resulting from short communication cycles by
    trading off communication overhead with chain mixing rates
  \item Load balancing using trajectory lengths, to mitigate underutilization
    resulting from situations where different processors operate at different
    speeds or where data is not equally partitioned
  \item Riemannian Langevin dynamics \citep{patterson2013stochastic}, to enable sampling of
    parameters $\theta$ constrained to a probability simplex
    ($0 \leq \theta_i \leq 1$ for all $i$, $\sum_i \theta_i = 1$)
    which is required for models such as latent Dirichlet allocation (LDA) \citep{blei2003latent}
\end{itemize}

In contrast to \citet{ahn2014distributed}, where the implementation targeted a
cloud environment (provided by AWS), our implementation uses OpenMPI and
targets a HPC computing cluster (provided by NERSC). Compared to a HPC cluster,
a cloud environment has higher communication costs (requiring network I/O) but
greater scalability (can easily provision a large number of EC2 instances). As
a result, we expect the effects of the above modifications to yield
different tradeoffs than those reported in \citet{ahn2014distributed}.

\section{Our Method}
\label{sec:our-method}

We follow the D-SGLD algorithm described by \citet{ahn2014distributed}. This is a data-parallel
algorithm which first partitions data into disjoint shards $(X_s)_{s \in S}$ distributed across
workers and then runs $S$ parallel chains (one on each worker) using the update
\begin{align}
  \theta_{t+1} = \theta_t \frac{\epsilon_t}{2}\left( \nabla \log P(\theta) + N g(\theta_t, X_s) \right) + \nu_t
\end{align}
where $\nu_t \overset{\text{iid}}{\sim} N(0, \epsilon_t)$, $\epsilon_t \to 0$ sufficiently slowly, and
$g(\theta_t, X_s)$ is an unbiased estimator of $\nabla \log P(\theta \mid X)$.
Notice that the unbiased condition
\begin{align}
  \label{eq:sgld}
  \ex g(\theta_t, X_s) = \nabla \log P(\theta \mid X) = \nabla \log P(\theta \mid \cup_{s \in S} X_s)
\end{align}
requires chains to be exchanged between different workers $s \in S$, resulting
in communication.

\section{Parallel chains and chain exchange}

In this section and the following two, we use a two-component Gaussian mixture model (GMM)
over $\mathbb{R}$ with priors $\theta_1 \sim \mathcal{N}(0,10)$, $\theta_2 \sim \mathcal{N}(0,1)$,
and likelihood
$x_i \overset{\text{iid}}{\sim} \frac{1}{2}\mathcal{N}(\theta_1, 2) + \frac{1}{2}\mathcal{N}(\theta_1 + \theta_2, 2)$.
The posterior has two modes at $\theta = (0,1)$ and $\theta = (-1,1)$ and is negatively correlated,
allowing easy verification of expected behavior.

As discussed in \cref{sec:our-method}, correctness of the algorithm requires
the $S$ parallel chains to be exchanged across workers. One possible
implementation of $g(\theta_t, X_s)$ \cref{eq:sgld} is to randomly sample a
permutation $\sigma \in S^\lvert S \rvert$ and send the chain on worker $s$ to
$\sigma(s)$ after every iteration. \Cref{fig:samples} illustrates the necessity of this
exchange. Without it, each of the chains only samples from the posterior $\pr(\theta \mid X_s)$
induced by the data local to the worker it is assigned to rather than the global posterior
$\pr(\theta \mid \cup_s X_s)$.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.49\linewidth]{poster-figures/gaussian_no_exchange.png}
  \includegraphics[width=0.49\linewidth]{poster-figures/gaussian_exchange.png}
  \caption{Samples drawn from local posteriors $P(\theta \mid X_s)$ (left, e.g.\ due to lack
    of chain exchange) may not well approximate the global posterior
    $P(\theta \mid \cup_s X_s)$ (right). Different colors corresponds to different chains.}
  \label{fig:samples}
\end{figure}

As we increase the number of workers $S$, we expect the rate at which samples are collected to increase.
Indeed, \cref{fig:speedup-partitioning} confirms that it takes less time to generate the same number
of samples when we have more workers. However, the scaling is not very good as
\cref{fig:speedup-partitioning} shows that the time to generate $800000$ samples is only cut down by
$\sim 1/3$ when three workers are added (perfect scaling would expect this to be a $2/3$ reduction).
We can attribute this poor scaling to communication overhead, which we will address in the following section.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.5\linewidth]{poster-figures/speedup-parallel.png}
  \caption{Speedup in sampling due to parallelism.}
  \label{fig:speedup-partitioning}
\end{figure}

\section{Trajectory sampling}

As illustrated in \cref{fig:speedup-partitioning}, exchanging chains after each
iteration incurs significant communication overhead and impacts the algorithms
scaling behavior. To circumvent this, SGLD can be modified to perform
trajectory sampling. Instead of drawing a single sample, a trajectory of length
$\tau$ is sampled on each worker prior to exhanging chains. This can reduce the
amount of inter-worker communication by a factor of $1/\tau$,
but it must be used with care as too little chain exchange results in biased samples
(setting $\tau$ equal to the number of samples is equivalent to never exchanging chains).

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.5\linewidth]{poster-figures/traj_length_slowdown.png}
  \caption{Increasing trajectory length reduces how often communication happens, resulting in less
    communication overhead.}
  \label{fig:less-comm}
\end{figure}

\Cref{fig:less-comm} shows the fraction of time spent communication during an execution of the
algorithm. As expected, we see that longer trajectory lengths reduces this quantity and leads to
a greater proportion of time spent generating samples.

Although we do not explore it in this work, one interesting direction for
future work would be to explore the tradeoff between mixing rate and trajectory
length in more detail. For example, one could try to optimize $\tau$ by
choosing it such that it generates the largest effective sample size in a fixed
amount of time.

\section{Trajectory length load balancing}

Workers may not all finish sampling trajectories at the same times.
For example, workers may have different hardware and data partitioning may be imbalanced.
As a result, faster workers must block on slower workers on every chain exchange, leading to
low utilization (see \cref{fig:ahn-lb}).

\begin{figure}[htbp]
  \centering
  \includegraphics[width=1.0\linewidth]{poster-figures/ahn-lb.png}
  \caption{Figure from \cite{ahn2014distributed} illustrating trajectory length load balancing.
    The fast worker $m_2$ must sit idle waiting for the slow worker $m_4$ without load balancing (left),
    whereas after (right) it utilizes this time to generate a longer tarjectory.}
  \label{fig:ahn-lb}
\end{figure}

To improve utilization, we can modify the global trajectory length $\tau$ into
a worker-specific trajectory lengths $(\tau_s)_{s \in S}$ and use $\tau_s$ to
balance work. Faster workers can be assigned a larger $\tau_s$, allowing them
to continue generating samples while waiting. We set these worker-specific
trajectory lengths following \citet{ahn2014distributed} by reporting the time spent
within the trajectory sampling loop to the master and adjusting $\tau_s$ in an
inversely proportional manner. The results of this modification are shown in
\cref{fig:load-balancing}, where we simulated imbalanced data partitioning by
assigning 95\% of the data to worker 1. The reduction in latencies after the first
iteration (where initial timing results are gathered) confirm that trajectory load
balancing results in improved utilization.

\begin{figure}[htbp]
  \includegraphics[width=0.49\linewidth]{poster-figures/load-balancing-none.png}
  \includegraphics[width=0.49\linewidth]{poster-figures/load-balancing.png}
  \caption{Before (top) and after (bottom) trajectory length load balancing. Experiments used
    the GMM with an imbalanced data partitioning which assigned 95\% to worker 1. The lower variance
    in latency after the first iteration (where initial timing results are gathered)
    confirm that trajectory length load balancing results in improved
    utilization.}
  \label{fig:load-balancing}
\end{figure}

\section{Latent Dirichlet Allocation with Riemannian Langevin Dynamics}

So far, we have only explored a toy GMM. One real world application of posterior
sampling is inference in LDA \citep{blei2003latent}, a probabilistic topic model
used to discover structure in documents. As ``topics'' in LDA represent
discrete distributions over words, they are constrained to lie on the probability simplex.
Accordingly, we leverage stochastic gradient Riemannian Langevin dynamics \citep{patterson2013stochastic}
to sample the probability simplex and adapt it to run in a distributed MPI environment
over partitioned data. The result of training a LDA model using our implementation is shown in
\cref{fig:perplexity-sgrld}, where we see that the parameters sampled are decreasing in perplexity.
This indicates that the sampler is being drawn towards a posterior mode, and suggests that the implementation
is behaving as expected.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.5\linewidth]{poster-figures/perplexity-lda.png}
  \caption{Distributed SGRLD training on NIPS corpus ($1740$ documents,
    $19889$ unique words) of a 10 topic LDA model. Each color corresponds to a different chain.}
  \label{fig:perplexity-sgrld}
\end{figure}

\section{Conclusion}

We provide an implementation of D-SGLD on top of MPI and investigate the performance
characteristics of various optimizations. Our results demonstrate that D-SGLD enjoys
significant speedups due to parallelism, can sample posteriors even when data is partitioned
disjointedly across workers, and can be controlled using trajectory lengths to trade off
between communication versus mixing time as well as load balance workers.

\bibliographystyle{plainnat}
\bibliography{final-report}

\end{document}


\chapter{Introduction}
\label{chap:introduction}

Two definitions of Machine Learning are offered at the start of the course:
\begin{description}
\item[Arthur Samuel] `The field of study that gives computers the ability to learn without being explicitely programmed.'
\item[Tom Mitchell] `A computer program is said to learn from experience $E$ with respect to some class of tasks $T$ and performance measure $P$, if its performance at tasks $T$, as measured by $P$, improves with experience $E$.'
\end{description}

\section{Supervised Learning}
\label{chapintro-sect:suplearn}

In supervised learning, we have a data set and already know what the correct output should be,  knowing that there is an existing relationship between the input and output. There are two types of supervised learning: \textbf{classification}, and \textbf{regression}.

In a regression problem, we're trying to predict results with a continuous output; i.e. we are trying to map input variables to some continuous function. In a classification problem, we are instead trying to predict results with a discrete output. Let's look at some examples:

\begin{description}
\item[Regression] If we're trying to predict the price of a house given data about the house such as its size, location, etc., this is a regression problem.
\item[Regression] Given a picture of a person, try to predict his/her age.
\item[Classification] Given a picture of a person, try to predict if he/she is of high school, college, or graduate age.
\item[Classification] As a bank, decide whether or not to give a loan to a potential borrower.
\end{description}

\section{Unsupervised Learning}
\label{chapintro-sect:unsuplearn}

Unsupervised learning allows us to approach problems with little or no idea about what our results should look like. We can try and derive structure from data where we don't necessarily know the effect of the variables. We can derive this structure by clustering the data based on relationships among the variables in the data. With unsupervised learning, there is no feedback based on the prediction results.

One example of unsupervised learning is clustering. Imagine we take 1000 essays written by US college students. We can try and automatically group these essays into a smaller number that are somehow similar or related by different variables; word frequency, sentence length, page count, etc.

Here is an example of unsupervised learning that isn't clustering: the cocktail party algorithm. This is a way to find structure in messy data, such as the identification of individual voices and music from a mesh of sounds at a cocktail party.\footnote{For more information about auditory filtering, look at Wikipedia's \href{https://en.wikipedia.org/wiki/Cocktail_party_effect}{Cocktail Party Effect} article.}


\subsection{Mapping 2D manifolds to Riemann surfaces}

Needs to be orientable and metricisable.

\subsection{Connections of topological manifolds}

Connected vs path-connected topological manifods.


%!TEX root = inversion.tex
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Copyright (c) 2003-2018 by The University of Queensland
% http://www.uq.edu.au
%
% Primary Business: Queensland, Australia
% Licensed under the Apache License, version 2.0
% http://www.apache.org/licenses/LICENSE-2.0
%
% Development until 2012 by Earth Systems Science Computational Center (ESSCC)
% Development 2012-2013 by School of Earth Sciences
% Development from 2014 by Centre for Geoscience Computing (GeoComp)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{DC Resistivity Forward modelling}\label{Chp:cook:Dc Resistivity inversion}
\section{Introduction}
DC resistivity surveys involve placing electrodes into the ground and injecting a current
into them. The current propagates through the ground and generates a potential field.
The change in potential between an other set of electrodes can then be measured.

As the separation between the electrodes increases, the current has a longer
distance to travel and can potentially travel deeper. This is not necessarily the case
as a highly conductive layer will keep the current close to the surface.

The final objective is to perform an inversion and develop a resistivity image of the subsurface.
This image can then be compared to known resistivities of material and used
to make inferences about the material contained in the subsurface.
There are a number of different ways to set up a DC resistivity survey, each providing
different spatial information\cite[pg 5]{LOKE2014}.

Escript currently supports the forward modelling of DC resistivity surveys. Forward modelling
involves performing a survey artificially by solving the PDEs which describe the underlying
physics. Escript provides a number of classes for solving forward modelling problems, these are
detailed in section \ref{sec:forward DCRES}.

\section{Example}
In this section we will look at an example forward problem. The domain consists of
a homogeneous half-space with a half-sphere embedded within it (Figure~\ref{fig:HalfSphere}).
In this example\footnote{The script is similar to
\examplefile{dc_forward.py} within the \escript example file directory.}
 a Schlumberger survey is used.

\begin{figure}
\centering
\includegraphics[width=0.7\textwidth]{HalfSphere.png}
\caption{
    (file \examplefile{data/HalfSphere_v1.4.geo}). model created using gmsh.}
\label{fig:HalfSphere}
\end{figure}

\begin{pyc}\label{code: dc1}
\
\begin{python}
#Header
import esys.finley      as finley
import esys.escript     as escript
from esys.downunder     import *
import math

#Constants
pi = math.pi

#Setup Input
mesh_file = "data/HalfSphere_v1.4.msh"
# Tag volume names and conductivity values (S/m) for primary
# and secondary potential:
tag_p = {"domain" : 1/10.0, "sphere" :  1/10.0} # Primary (homogeneous).
tag_s = {"domain" : 1/10.0, "sphere" :  1/1.0 } # Secondary.

xe_0 = -5.0 # start X-coordinate
numEle =  21  # number of electrodes
a =  0.5 # step size
n = 9 #max electrode step
midPoint = [xe_0 + (((numEle-1)*estp)/2), 0, 0]
current = 1.0 # (Ampere)
domain = finley.ReadGmsh(mesh_file, 3)
mesh_tags = escript.getTagNames(domain)
directionVector = [1,0]
sig_p = escript.Scalar(0,escript.ContinuousFunction(domain))
sig_s = escript.Scalar(0,escript.ContinuousFunction(domain))
for tag in tag_p:
    # All initially defined tags must be in the tag list.
    # Print an error if it doesn't and exit the program.
    if tag in mesh_tags:
        # Assign value:
        sig_p.setTaggedValue( tag, tag_p[tag] )
        sig_s.setTaggedValue( tag, tag_s[tag] )
    else:
        print("Error: the defined tag is not defined in the mesh: " & tag)
        sys.exit()

# Expand the data objects for output.
sig_p.expand()
sig_s.expand()
#solve for result
schs=SchlumbergerSurvey(domain, sig_p, sig_s, current, a, n, midPoint,
    directionVector, numEle)
pot=schs.getPotential()
totalApparentRes=schs.getApparentResistivityTotal()
#print result
n=1
print ("Total:\n")
for i in totalApparentRes:
    print ("n = %d:"%n)
    print (i,"\n")
    n=n+1
\end{python}
\end{pyc}

The example begins with constructing the domain, loaded from a pre-prepared
\emph{gmsh} model. The \emph{gmsh} script used can be found in \examplefile{data/HalfSphere_v1.4.geo}.
\emph{gmsh} can be used to generate the \texttt{msh} file. The values for primary and secondary
conductivity, in Siemens per meter, are specified for the different regions. These regions have been tagged
in the \emph{gmsh} script. The survey is constructed to have 21 electrodes spanning from -5m to 5m
in the $x$-axis, with a fixed interval between each electrode and the next in
line. These electrodes, once placed, are not moved for the remainder of the
survey. The potentials and total apparent resistivity is then calculated.

The SchlumbergerSurvey class uses four electrodes at a time, beginning with the
electrode at location \texttt{xe_0}. In each set of four electrodes, electrodes
1 and 4 are used as current electrodes and
2 and 3 used as potential electrodes. The next set of electrodes are then used
for the next measurement, beginning with the previous electrode 2. This is
repeated until the last four electrodes are used.

This process is itself repeated with a continually increasing step size between
the electrodes pairs at each end of the set. As an example, the first time the
process is repeated, the initial set will be made up of electrodes 1, 3, 4, and
6. During the second repeat the initial set will be electrodes 1, 4, 5, and 8.
The maximal step size in the above script is given as $n$.


The VNF catalogue supported by VeriGraph is composed of several network function models, which are: End-host, Mail Server/Client, Web Client/Server, Anti-spam, NAT, Web Cache, ACL firewall and Learning Firewall. Here we describe the formulas that model the functional behaviour of each function in catalog.

\paragraph{End-host model}
An end-host is a network node which sends packets towards a destination and receives packets from a source. The sent packets must satisfy some conditions (Formula~\ref{end-host_constr1}): \textit{(i)} the end-host address is the source address; \textit{(ii)} \textit{origin} is the end-host itself; \textit{(iii)} \textit{origin\_body} and \textit{body}  must be equal. The received packet (Formula~\ref{end-host_constr2}) must have the end-host address as destination.
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%Antispam model:
			\begin{align}
				\begin{split}
					\label{end-host_constr1}
					(send(end\_host, n_{0}, p_{0}, t_{0}) &) \implies (nodeHasAddr(end\_host, p_{0}.src) \wedge \\
					& p_{0}.origin == end\_host \wedge p_{0}.origin\_body == p_{0}.body) \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split} \\
				\begin{split}
					\label{end-host_constr2}
					(recv(n_{0}, end\_host, p_{0}, t_{0}) &) \implies (nodeHasAddr(end\_host, p_{0}.dest)),\\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{End-host model.}
	\label{end-host_model}
\end{figure}
\\
At this version of VeriGraph, the tool does not support end-host configurations, as well as the other end-host-based models (i.e., servers and clients). In other words, we cannot specify which traffic flow an end-host sends, without changing its model (e.g., a client can generate packet with specific port number, destination address etc.). It could be useful to extend VeriGraph to support end-host configurations.

\paragraph{Mail Server model} A mail server is a complex form of end-host. In fact, this kind of server can generate only \textit{POP3\_RESPONSE} or \textit{SMTP\_RESPONSE} messages addressed to a mail client (Formula~\ref{mail-server_constr1}), but the type of the response depends on the previous messages. In particular, a \textit{POP3\_RESPONSE} packet is sent only if previously a \textit{POP3\_REQUEST} was received (Formula~\ref{mail-server_constr2}) and similarly for the \textit{SMTP\_RESPONSE} packets (Formula~\ref{mail-server_constr3}). A mail server is also modelled for receiving just \textit{POP3\_REQUEST} or \textit{SMTP\_REQUEST} messages (Formula~\ref{mail-server_constr4}).
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%MailServer model:
			\begin{align}
				\begin{split}
					\label{mail-server_constr1}
					(se& nd(mail\_server , n_{0}, p_{0}, t_{0})) \implies (nodeHasAddr(mail\_server, p_{0}.src) \wedge \\
					& p_{0}.origin = mail\_server \wedge p_{0}.origin\_body = p_{0}.body \wedge \\
					& (p_{0}.proto = POP3\_RESPONSE \vee p_{0}.proto = SMTP\_RESPONSE) \wedge \\
					& p_{0}.dst = ip\_mail\_client \wedge p_{0}.emailFrom = mail\_server), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split} \\
			\end{align}
		\end{subequations}
	}
\end{figure}
\begin{figure}
	{\footnotesize
		\begin{subequations}
		\begin{align}
				\begin{split}
					\label{mail-server_constr2}
					(se& nd(mail\_server , n_{0}, p_{0}, t_{0}) \wedge p_{0}.proto = POP3\_RESPONSE) \implies \\
					& \exists (p_{1}, t_{1}) | recv(n_{0}, mail\_server, p_{1}, t_{1})  \wedge p_{1}.proto = POP3\_REQUEST),\\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split} \\
				\begin{split}
					\label{mail-server_constr3}
					(se& nd(mail\_server , n_{0}, p_{0}, t_{0}) \wedge p_{0}.proto = SMTP\_RESPONSE) \implies \\
					& \exists (p_{1}, t_{1}) | recv(n_{0}, mail\_server, p_{1}, t_{1}) \wedge  p_{1}.proto = SMTP\_REQUEST),\\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split} \\
				\begin{split}
					\label{mail-server_constr4}
					(re& cv(n_{0}, mail\_server, p_{0}, t_{0})) \implies  (nodeHasAddr(mail\_server, p_{0}.dst) \\
					& (p_{0}.proto = POP3\_REQUEST \vee p_{0}.proto = SMTP\_REQUEST)), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{Mail Server model.}
	\label{mail-server_model}
\end{figure}

\paragraph{Mail client model} A mail client is a particular kind of end-host (Formulas~\ref{mail-client_constr1} and~\ref{mail-client_constr2}). This node is modelled so that it can send \textit{POP3\_REQUEST} or \textit{SMTP\_REQUEST} messages only. Those messages must have a mail server address as destination (\textit{$p_{0}.dst = ip\_mail\_server$}) and the \textit{emailFrom} field must indicate the mail client node itself (Formula~\ref{mail-client_constr1}).
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%MailClient model:
			\begin{align}
				\begin{split}
					\label{mail-client_constr1}
					(se& nd(mail\_client , n_{0}, p_{0}, t_{0})) \implies (nodeHasAddr(mail\_client, p_{0}.src) \wedge \\
					& p_{0}.origin = mail\_client \wedge p_{0}.origin\_body = p_{0}. body \wedge \\
					& (p_{0}.proto = POP3\_REQUEST \vee p_{0}.proto = SMTP\_REQUEST) \wedge \\
					& p_{0}.dst = ip\_mail\_server \wedge p_{0}.emailFrom = mail\_client), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split} \\
				\begin{split}
					\label{mail-client_constr2}
					(re& cv(n_{0}, mail\_client, p_{0}, t_{0})) \implies  (nodeHasAddr(mail\_client, p_{0}.dst)), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{Mail Client model.}
	\label{mail-client_model}
\end{figure}

\paragraph{Web Client model} The web client is based on the end-host model (Formulas~\ref{web-client_constr1} and~\ref{web-client_constr2}), but it can generate only \textit{HTTP\_REQUEST} packets with a web server address as destination (Formula~\ref{web-client_constr1}).
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%WebClient model:
			\begin{align}
				\begin{split}
					\label{web-client_constr1}
					(se& nd(web\_client , n_{0}, p_{0}, t_{0})) \implies (nodeHasAddr(web\_client, p_{0}.src) \wedge \\
					& p_{0}.origin = web\_client \wedge p_{0}.origin\_body = p_{0}.body \wedge \\
					& p_{0}.proto = HTTP\_REQUEST \wedge p_{0}.dst = ip\_web\_server), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split} \\
				\begin{split}
					\label{web-client_constr2}
					(re& cv(n_{0}, web\_client, p_{0}, t_{0})) \implies  (nodeHasAddr(web\_client, p_{0}.dst)), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{Web Client model.}
	\label{web-client_model}
\end{figure}

\paragraph{Web Server model} The web server model is built to send \textit{HTTP\_RESPONSE} packets only if the server has previously received a \textit{HTTP\_REQUEST} packet (Formula~\ref{web-server_constr1}). The sent and received packets must refer to the same \textit{url} field (\textit{$p_{1}.url = p_{0}.url$}):
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%webServer model:
			\begin{align}
				\begin{split}
					\label{web-server_constr1}
					(se& nd(web\_server , n_{0}, p_{0}, t_{0})) \implies (nodeHasAddr(web\_server, p_{0}.src) \wedge \\
					& p_{0}.origin = web\_server \wedge p_{0}.origin\_body = p_{0}.body \wedge \\
					& p_{0}.proto = HTTP\_RESPONSE \wedge \exists (p_{1}, t_{1}) | (t_{1} < t_{0} \\
					& recv(n_{0}, web\_server, p_{1}, t_{1}) \wedge p_{1}.url = p_{0}.url \wedge \\
					& p_{1}.proto = HTTP\_REQUEST \wedge p_{0}.dst = p_{1}.src \wedge p_{0}.src = p_{1}.dst)), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split} \\
				\begin{split}
					\label{web-server_constr2}
					(re& cv( n_{0}, mail\_server, p_{0}, t_{0})) \implies (nodeHasAdrr(web\_server, p_{0}.dst)),\\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{Web Server model.}
	\label{web-server_model}
\end{figure}

\paragraph{Anti-Spam model}An anti-spam function was modelled to drop packets from blacklisted mail clients and servers. In fact, the anti-spam behaviour was based on the assumption that each client interested in receiving a new message addressed to it, sends a \texttt{POP3\_REQUEST} to the mail server in order to retrieve the message content. The server, in turn, replies with a \texttt{POP3\_RESPONSE} which contains a special field (\textit{emailFrom}) representing the message sender. The process of sending an email is similarly modelled through SMTP request and response messages. As evident from Formula \ref{anti-spam_constr1}, an anti-spam rejects any message containing a black listed email address (that are set during the creation of the VNF chain model). On the other hand, Formula \ref{anti-spam_constr2} is needed in order to state that a \texttt{POP3\_REQUEST} message is forwarded only after having received it in a previous time instant.
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%Antispam model:
			\begin{align}
				\begin{split}
					\label{anti-spam_constr1}
					(send(anti\_spam, n_{0}, p_{0}, t_{0}) & \wedge p_{0}.protocol = POP3\_RESPONSE) \implies \\
					& \neg isInBlackList(p_{0}.emailFrom) \wedge \exists (n_{1}, t_{1}) \: | \: (t_{1} < t_{0} \\
					& \wedge recv(n_{1}, anti\_spam, p_{0}, t_{1})) \\
					& \forall n_{0}, p_{0}, t_{0}
				\end{split} \\
				\begin{split}
					\label{anti-spam_constr2}
					(send(anti\_spam, n_{0}, p_{0}, t_{0}) & \wedge p_{0}.protocol = POP3\_REQUEST) \implies \\
					& \exists (n_{1}, t_{1}) \: | \: (t_{1} < t_{0} \wedge recv(n_{1}, anti\_spam, p_{0}, t_{1})) \\
					& \forall n_{0}, p_{0}, t_{0}
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{Anti-spam model.}
	\label{anti-spam_model}
\end{figure}
\\
The set of blacklist email addresses is configured through a public function (e.g., \textit{parseConfiguration()}), which gives an interpretation to the \textit{isInBlackList()} uninterpreted function. In particular if the blacklist is empty, VeriGraph will build a constraint like Formula~\ref{anti-spam_configuration1}. Otherwise, let us suppose that the blacklist contains two elements (\textit{BlackList=[mail1, mail2]}), VeriGraph will build the Formula~\ref{anti-spam_configuration2}. In this case, VeriGraph is imposing that the \textit{inInBlackList()} function returns \textit{TRUE} if either \textit{$(emailFrom == mail1)$} or \textit{$(emailFrom == mail2)$} are \textit{TRUE}.
	\begin{figure}[h]
		{\footnotesize
			\begin{subequations}
				%Antispam model:
				\begin{align}
					\begin{split}
						\label{anti-spam_configuration1}
						(isInBlackList(emailFrom) == \;& False), \forall emailFrom
					\end{split} \\
					\begin{split}
						\label{anti-spam_configuration2}
						(isInBlackList(emailFrom) == (& emailFrom == mail1) \; \vee \\
						& (emailFrom == mail2)), \forall emailFrom
					\end{split}
				\end{align}
			\end{subequations}
		}%
	\end{figure}

\paragraph{NAT model}
A different type of function is a NAT, which needs the notion of internal and external networks. This kind of information is modelled by means of a function (\textit{isPrivateAddress}) that checks if an address is registered as private or not. Private addresses are configured when the VNF chain model is initialized. As example of configuration, let us suppose that end-hosts \textit{nodeA} and \textit{nodeB} are internal nodes, hence VeriGraph must add Formula~\ref{nat_configuration1} among its constraint to verify. Here the \textit{isPrivateAddress()} function returns \textit{TRUE} if both \textit{$( address == nodeA\_addr)$} and \textit{$(address == nodeB\_addr)$} are \textit{TRUE}.
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%NAT model:
			\begin{align}
				\begin{split}
					\label{nat_configuration1}
					(isPrivateAddress(address) == (& address == nodeA\_addr \wedge \\
					& address == nodeB\_addr)), \forall address
				\end{split}
			\end{align}
		\end{subequations}
	}%
\end{figure}

In details, the NAT behaviour is modelled by two formulas. Formula~\ref{nat_constr1} states for an internal node which initiates a communication with an external node. In this case, the NAT sends a packet (\textit{$p_0$}) to an external address (\textit{$\neg isPrivateAddress(p_{0}.dst)$}), if and only if it has previously received a packet (\textit{$p_1$}) from an internal node (\textit{$isPrivateAddress(p_{1}.src)$}). The received and sent packets must be equal for all fields, except for the \textit{src}, which must be equal to the NAT public address (\textit{$ip\_nat$}).\\

\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%NAT model:
			\begin{align}
			\begin{split}
			\label{nat_constr1}
			(se& nd(nat, n_{0}, p_{0}, t_{0}) \wedge \neg isPrivateAddress(p_{0}.dst)) \implies p_{0}.src = ip\_nat \\
			& \wedge \exists (n_{1}, p_{1}, t_{1}) \: | \: (t_{1} < t_{0} \wedge recv(n_{1}, nat, p_{1}, t_{1}) \wedge isPrivateAddress(p_{1}.src) \\
			& \wedge p_{1}.origin = p_{0}.origin \wedge p_{1}.dst = p_{0}.dst \wedge p_{1}.seq\_no = p_{0}.seq\_no \\
			& \wedge p_{1}.proto = p_{0}.proto \wedge p_{1}.emailFrom = p_{0}.emailFrom \wedge p_{1}.url = p_{0}.url), \\
			& \forall (n_{0}, p_{0}, t_{0})
			\end{split} \\
%			\end{align}
%		\end{subequations}
%	}%
%\end{figure}
%\begin{figure}[t!]
%	{\footnotesize
%		\begin{subequations}
%			%NAT model:
%			\begin{align}
			\begin{split}
			\label{nat_constr2}
			(se& nd(nat, n_{0}, p_{0}, t_{0}) \wedge isPrivateAddress(p_{0}.dst)) \implies \neg isPrivateAddress(p_{0}.src) \\
			& \wedge \exists  (n_{1}, p_{1}, t_{1}) \: | \: (t_{1} < t_{0} \wedge recv(n_{1}, nat, p_{1}, t_{1}) \wedge \neg isPrivateAddress(p_{1}.src) \\
			& \wedge p_{1}.dst = ip\_nat \wedge p_{1}.src = p_{0}.src \wedge p_{1}.origin = p_{0}.origin \\
			& \wedge p_{1}.seq\_no = p_{0}.seq\_no \wedge p_{1}.proto = p_{0}.proto \wedge p_{1}.emailFrom = p_{0}.emailFrom \\
			& \wedge p_{1}.url = p_{0}.url) \wedge \exists (n_{2}, p_{2}, t_{2}) \: | \: (t_{2} < t_{1} \wedge recv(n_{2}, nat, p_{2}, t_{2}) \\
			& \wedge isPrivateAddress(p_{2}.src) \wedge p_{2}.dst = p_{1}.src \wedge p_{2}.dst = p_{0}.src \\
			& \wedge p_{2}.src = p_{0}.dest), \forall (n_{0}, p_{0}, t_{0})
			\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{NAT model.}
	\label{nat_model}
\end{figure}

On the other hand, the traffic from the external network to the private is modelled by Formula ~\ref{nat_constr2}. In this case, if the NAT is sending a packet to an internal address (\textit{$isPrivateAddress(p_{0}.dst)$}), this packet (\textit{$p_0$}) must have an external address as its source (\textit{$\neg isPrivateAddress(p_{0}.src)$}). Moreover, \textit{$p_0$} must be preceded by another packet (\textit{$p_1$}), which is, in turn, received by the NAT and it is equal to \textit{$p_0$} for all the other fields. It is worth noting that, generally, a communication between internal and external nodes cannot be started by the external node in presence of a NAT. As a consequence, this condition is expressed in the Formula~\ref{nat_constr2} by imposing that \textit{$p_1$} must be preceded by another packet \textit{$p_2$} (\textit{$(t_{2} < t_{1} \wedge recv(n_{2}, nat, p_{2}, t_{2})$}), sent to the NAT from an internal node (\textit{$isPrivateAddress(p_{2}.src)$}).

\paragraph{Web Cache model} A simple version of web cache can be modelled with three formulas (Formulas~\ref{cache_constr1},~\ref{cache_constr2} and~\ref{cache_constr3}), where we have a notion of internal addresses (\textit{$isInternal$} function), which are configured when the chain model is created. VeriGraph follows a similar approach to the NAT model (Formula~\ref{nat_configuration1}) for configuring the internal nodes. For instance, if the internal network is composed of two nodes \textit{nodeA} and \textit{nodeB}, VeriGraph will give an interpretation to the \textit{isInternal} function by means of Formula~\ref{web-cache_configuration1}.
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%Web cache model:
			\begin{align}
				\begin{split}
					\label{web-cache_configuration1}
					(isInternal(node) == (& node == nodeA \wedge \\
					& node == nodeB)), \forall node
				\end{split}
			\end{align}
		\end{subequations}
	}%
\end{figure}

This model was designed to work with web end-hosts (i.e., web client and server). In details, formula~\ref{cache_constr1} states that: a packet sent from the cache to a node belonging to the external network (\textit{$\neg isInternal(n_{0})$}), implies a previous HTTP request packet (\textit{$p_{0}.proto = HTTP\_REQ$}) and received from an internal node, which cannot be served by the cache (\textit{$\neg isInCache(p_{0}.url, t_{0})$}), otherwise the request would have not been forwarded towards the external network.

\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%Web cache model:
			\begin{align}
			\begin{split}
			\label{cache_constr1}
			(send(cache&, n_{0}, p_{0}, t_{0}) \wedge \neg isInternal(n_{0})) \implies \neg isInCache(p_{0}.url, t_{0}) \\
			& \wedge p_{0}.proto = HTTP\_REQ \wedge \exists (t_{1}, n_{1}) \: | \: (t_{1} < t_{0} \\
			& \wedge isInternalNode(n_{1}) \wedge recv(n_{1}, cache, p_{0}, t_{1})), \\
			& \forall (n_{0}, p_{0}, t_{0})
			\end{split} \\
			\begin{split}
			\label{cache_constr2}
			(send(cache&, n_{0}, p_{0}, t_{0}) \wedge isInternal(n_{0})) \implies isInCache(p_{0}.url, t_{0}) \\
			& \wedge p_{0}.proto = HTTP\_RESP \wedge p_{0}.src = p_{1}.dst \wedge p_{0}.dst = p_{1}.src \wedge \\
			& \wedge \exists  (p_{1}, t_{1}) \: | \: (t_{1} < t_{0} \wedge p_{1}.proto = HTTP\_REQ  \\
			& \wedge p_{1}.url = p_{0}.url \wedge recv(n_{0}, cache, p_{1}, t_{1})), \\
			& \forall (n_{0}, p_{0}, t_{0})
			\end{split} \\
			\begin{split}
			\label{cache_constr3}
			isInCache&(u_{0}, t_{0}) \implies \exists (t_{1}, t_{2}, p_{1}, p_{2}, n_{1}, n_{1}) \: | \: (t_{1} < t_{2} \wedge t_{1} < t_{0} \wedge t_{2} < t_{0} \\
			& \wedge recv(n_{1}, cache, p_{1}, t_{1}) \wedge recv(n_{2}, cache, p_{2}, t_{2}) \wedge p_{1}.proto = HTTP\_REQ \\
			& \wedge p_{1}.url = u_{0} \wedge p_{2}.proto = HTTP\_RESP \wedge p_{2}.url = u_{0} \wedge isInternal(n_{2})) \\
			& \forall (u_{0}, t_{0})
			\end{split}
			\end{align}
			\end{subequations}
			}%
			\caption{Web cache model.}
			\label{cache_model}
\end{figure}

Formula~\ref{cache_constr2} states that a packet sent from the cache to the internal network contains a \texttt{HTTP\_RESPONSE} for an URL which was in cache when the request has been received. We also state that the packet received from the internal network is a \texttt{HTTP\_REQUEST} and the target URL is the same as the response (\textit{$p_{1}.protocol = HTTP\_REQ \wedge p_{1}.url = p_{0}.url$}).

The final formula (Formula~\ref{cache_constr3}) expresses a constraint that the \textit{isInCache()} function must respect. In particular, we state that a given URL ($u_{0}$) is in cache at time $t_{0}$ if (and only if) a request packet was received at time $t_{1}$ (where $t_{1} < t_{0}$) for that URL and a subsequent packet was received at time $t_{2}$ (where $t_{2} < t_{0} \wedge t_{2} > t_{1}$) carrying the corresponding \texttt{HTTP\_RESPONSE}.

\paragraph{ACL firewall model} An ACL firewall is a simple firewall that drops packets based on its internal Access Control List (ACL), configured when the chain model is initialized. In particular the ACL list is managed through the uninterpreted function \textit{$acl\_func()$}. A possible interpretation is given by VeriGraph though the Formula~\ref{acl-firewall_configuration1}, when the ACL list contains two entries, like for example \textit{$ACL = [<src_1, dst_1>,<src_2,dst_2>]$}.
	\begin{figure}[h]
		{\footnotesize
			\begin{subequations}
				%Antispam model:
				\begin{align}
					\begin{split}
						\label{acl-firewall_configuration1}
						(acl\_func(a, b) == \;& ((a == src_1 \wedge b == dst_1 ) \; \vee \\
						& (a == src_2 \wedge b == dst_2))), \forall a, b
					\end{split}
				\end{align}
			\end{subequations}
		}%
	\end{figure}

Hence, if an ACL firewall sends a packet, this implies that the firewall has previously received a packet of which the source and destination address are not contained in the ACL list.
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			\begin{align}
				\begin{split}
					\label{acl-fw_constr1}
					(send(fw, n_{0}, p_{0}, t_{0})) \implies (&\exists (t_{1}, n_{1}) | t_{1} < t_{0} \wedge recv(n_{1}, fw, p_{0}, t_{1}) \wedge \\
					&\neg acl\_func(p_{0}.src, p_{0}.dst)), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{ACL Firewall model}
	\label{acl-fw_model}
\end{figure}

\paragraph{Learning firewall model} The learning firewall behaviour follows the fundamental principals of network forwarding: if the firewall is sending a packet, this implies that the function has previously received that packet (Formula~\ref{learning-fw_constr1}). However the forwarding decisions of a learning firewall depend on an internal Access Control List: each ACL entry specifies a pair of addresses allowed to exchange packets and are configured through the same mechanism defined for the ACL firewall model (Formula~\ref{acl-firewall_configuration1}). The ACL entries define also the direction of the allowed traffic flows, which is, for instance, if the ACL list contains an entry like \textit{$<nodeA\_addr, nodeB\_addr>$}, the learning firewall will forward packet from node \textit{A} to \textit{B}, but no the vice-versa. The traffic in the opposite direction (i.e., from \textit{B} to \textit{A}) is allowed if and only if node \textit{A} has previously opened a connection toward \textit{B} (that is \textit{A} has sent a packet to \textit{B} - Formula~\ref{learning-fw_constr2}):
\begin{figure}[h]
	{\footnotesize
		\begin{subequations}
			%Antispam model:
			\begin{align}
				\begin{split}
					\label{learning-fw_constr1}
					(send(fw, n_{0}, p_{0}, t_{0})) \implies (&\exists (t_{1}, n_{1}) | \; t_{1} < t_{0} \wedge recv(n_{1}, fw, p_{0}, t_{1})),\forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
			%	\end{subequations}
			%	\begin{subequations}
			\begin{align}
				\begin{split}
					\label{learning-fw_constr2}
					(send(fw, n_{0}, & p_{0}, t_{0})  \wedge \neg acl\_func(p_{0}.src, p_{0}.dst)) \implies (\exists ( n_{1},t_{1}) \; | \; send(fw, n_{1}, p_{1},t_{1}) \wedge \\
					& t_{1} + 1 < t_{0} \wedge acl\_func(p_{1}.src, p_{1}.dst) \wedge  p_{0}.src = p_{1}.dst \wedge  p_{0}.dst = p_{1}.src \wedge \\
					& p_{0}.src\_port = p_{1}.dst\_port \wedge p_{0}.dst\_port = p_{1}.src\_port), \\
					& \forall (n_{0}, p_{0}, t_{0})
				\end{split}
			\end{align}
		\end{subequations}
	}%
	\caption{Learning Firewall model}
	\label{learning-fw_model}
\end{figure}