\section{Compilation}
\label{sec:compilation}

The front-end (FE) of \sysname simplifies operators' task of describing preferred paths. That simplicity, however, comes at the cost of compilation complexity. The compiler must efficiently compute the sets of paths represented by the intersection of preferences and topology, determine which ones can be honored under failures, and ensure policy compliance under all possible failure cases.

\begin{figure}[t!]
  \centering
  \includegraphics[width=.75\columnwidth]{figures/pipeline}
  \caption{Compilation pipeline stages for Propane.}
  \label{fig:pipeline}
  \vspace{-1em}
\end{figure}

We handle these challenges by breaking compilation into multiple stages and developing efficient algorithms for each (See Figure~\ref{fig:pipeline}). The first stage of the pipeline involves simple rewriting rules and substitutions from the FE to the core Regular Intermediate Representation (RIR). Policies in RIR are checked for well-formedness (\EG, never constraining traffic that does not enter the network), before being combined with the network topology to obtain the Product Graph Intermediate Representation (PGIR). The PGIR is a data representation that compactly captures the flow of BGP announcements subject to the policy and topology restrictions. We develop efficient algorithms that operate over the PGIR to ensure policy compliance under failures, avoid BGP instability, and prevent aggregation-induced black holes. Once the compiler determines that the PGIR is safe, it translates it to an abstract BGP (ABGP) representation. Vendor-specific adaptors to translate from ABGP to concrete device configurations are easily added to generate concrete configurations.

\subsection{Regular IR (RIR)}
\label{sec:rir}

%Regular expressions are an expressive formalism that have been used extensively for describing paths through graphs~\cite{bib:todo}, with applications to networks~\cite{bib:todo}. Our regular expression language differs from others by allowing operators to describe preferences between paths.

% grammar
\newcommand{\BNFALT}{\;\;|\;\;}
\newcommand{\hdr}[2]{\flushleft \chdr{\hspace{5mm}#1}{#2}}
\newcommand{\chdr}[2]{\textbf{#1} {#2} \\ \centering}

\begin{figure*}\small
  \begin{minipage}[t]{.45\linewidth}
  \hdr{\large Syntax}{}
  \vspace*{-1\baselineskip}
  %
  \[ \begin{array}{rclr}
    \hline

     pol     &::=& p_1, \dots, p_n & \textit{constraints} \\
     p       &::=& t \hspace{.3em} \Path \hspace{.3em} r_1 \Prefer \dots \Prefer r_m & \textit{preferences} \\
     x       &::=& a.b.c.d/n & \textit{prefix} \\
     c       &::=& \cdots & \textit{community value} \\
     t       &::=& \True & \textit{true} \\
         &\BNFALT& \NOT t & \textit{negation} \\
         &\BNFALT& t_1 \OR{} t_2 & \textit{disjunction} \\
         &\BNFALT& t_1 \AND{} t_2 & \textit{conjunction} \\
         &\BNFALT& \textit{prefix} = x & \textit{prefix test} \\
         &\BNFALT& \textit{comm} = c & \textit{community test} \\
     r       &::=& n & \textit{AS number} \\
         &\BNFALT& \emptyset & \textit{empty set} \\
         &\BNFALT& \In & \textit{internal loc} \\
         &\BNFALT& \Out & \textit{external loc} \\
         &\BNFALT& r_1 \cup r_2 & \textit{union} \\
         &\BNFALT& r_1 \cap r_2 & \textit{intersection} \\
         &\BNFALT& r_1 \cdot r_2 & \textit{concatenation} \\
         &\BNFALT& \NOT(r) & \textit{path negation} \\
         &\BNFALT& r^* & \textit{repetition} \\
     l       &::=& r_1 \rightarrow r_2 & \textit{link pairs} \\
     cc     &::=& agg(x, l) \BNFALT tag(c, t, l) & \textit{control constraints} \\
  \end{array} \]

  \end{minipage}
  %
  ~~
  \vrule
  ~~
  %
  \begin{minipage}[t]{.5\linewidth}\small
  \hdr{\large Propane Expansions}{}
  \vspace*{-1\baselineskip}
  %
  \[\begin{array}{rcl}
    \hline
    \Any           & = & \Out^* \cdot \In^+ \cdot \Out^* \\
    \None          & = & \emptyset \\
    \Internal      & = & \In^+ \\
    \Only(X)       & = & \Any \cap X^* \\
    \Never(X)      & = & \Any \cap (!X)^* \\
    \Through(X)    & = & \Out^* \cdot \In^* \cdot X \cdot \In^* \cdot \Out^* \\
    \Later(X)      & = & \Out^* \cdot (X \cap \Out) \cdot \Out^* \cdot \In^+ \cdot \Out^* \\
    \Before(X)     & = & \Out^* \cdot \In^+ \cdot \Out^* \cdot (X \cap \Out) \cdot \Out^* \\
    \End(X)        & = & \Any \cap (\Sigma^* \cdot X) \\
    \Start(X)      & = & \Any \cap (X \cdot \Sigma^*) \\
    \Exit(X)       & = & (\Out^* \cdot \In^* \cdot (X \cap \In) \cdot \Out \cdot \Out^*) \cup \\
                  &        & (\Out^* \cdot \In^+ \cdot (X \cap \Out) \cdot \Out^*) \\
    \Enter(X)      & = & (\Out^* \cdot \Out \cdot (X \cap \In) \cdot \In^* \cdot \Out^*) \cup \\
                  &        & (\Out^* \cdot (X \cap \Out) \cdot \In^+ \cdot \Out^*) \\
    \LinkKW(X,Y)     & = & \Any \cap (\Sigma^* \cdot X \cdot Y \cdot \Sigma^*) \\
    \PathKW(\vec{X}) & = & \Any \cap (\Sigma^* \cdot X_1 \dots X_n \cdot \Sigma^*) \\
    \Novalley(\vec{X}) & = & \Any ~ \cap \\
                  &   & \NOT\PathKW(X_2,X_1,X_2) ~ \cap \dots \cap \\
                  &   & \NOT\PathKW(X_n,X_{n-1},X_n) \\
  \end{array} \]

  \end{minipage}

  \hrulefill

  \caption{Regular Intermediate Language (RIL) syntax (left), and
           Propane language expansions (right).}
  \label{fig:rir-syntax}
  \vspace{-1em}
\end{figure*}

\para{Syntax}
The \sysname FE is just a thin layer atop a core, regular-expression language (RIR) for describing preference-based path constraints.
Figure~\ref{fig:rir-syntax} shows the RIR syntax. A policy consists of one or more constraints, each of which have a test on the type of route, and a corresponding set of preferred regular paths. Regular paths are regular expressions where the base characters are abstract locations---either a router or an AS. Special \In{} and \Out{} symbols refer to any internal or external location respectively. In addition, $\Sigma$ refers to any symbol. We also use other standard abbreviations
for regular expressions such as $r^+$, a sequence of one or more occurrences
of $r$, when appropriate.
%% For example, the constraint
%% \[(\textit{prefix}=74.3.28.0/24) \Link \\
%% (as200 \cdot \In^+) \Prefer (as100 \cdot \In^+)
%% \]
%% describes a more-preferred set of paths for traffic announced by a prefix no less specific than \CD{74.125.28.0/24}, which starts at AS 200, before entering and staying inside the user's network to get to the destination, and a less-preferred set of paths that start at AS 100 and are otherwise the same. The plus operator $\In^+$ stands for $\In \cdot \In^*$, at least one internal hop. Tests over route types use standard boolean connectives, and can refer to both prefixes and route community values.

\sysname also supports constraints purely on the control-plane behavior of BGP. For example, prefix aggregation is an important optimization to reduce routing table size and churn in practice.
%% Aggregation, for example, from internal to external locations, is specified using the same regular syntax as before:
%% $$\Agg(128.17.0.0/16, \In \Link \Out)$$
%% where the expression $\In \Link Out$ refers to control messages flowing from any internal to any external location.
We list only the route aggregation and community tagging constraints in Figure~\ref{fig:rir-syntax}, but we also support other constraints such as limiting the maximum number of routes allowed between ASes, or enabling BGP multipath.

\para{Semantics}

The semantics of RIR is in terms of ranked paths. Each preference-based regular path constraint (of the form $r_1 \Prefer \dots \Prefer r_j$) maps to a set of concrete paths in the network that match one of $r_i$. We denote a network path as a string of abstract locations (routers or external ASes) of the form: $n_1 n_2 \dots n_k$. A regular expression $r$ matches path $p$, if $p \in \mathcal{L}(r)$, that is the path is in the language of the regular expression, and $p$ is a topologically-valid path. We denote the length of a path $p$ as $\abs{p}$. A path $p$ will have a rank:
$$(\min_i \set{ p \in \mathcal{L}(r_i) }, \abs{p})$$
where the rank is lexicographically ordered according to (1) the most preferred regular expression matched, and (2) as a tie breaker, the path length. Lower ranks indicate \emph{more} preferred paths. Traffic may be sent on any of the most preferred paths for each pair of starting and ending locations that appear in some specified path. There is an implicit lowest preference to drop traffic: $\emptyset$ when no other route is possible.

The set of ranked paths depends on which paths are valid in the topology, and thus when failures occur, the most preferred routes change. The \sysname compiler ensures that generated configurations for a policy always achieve the most preferred path possible given the failures in the topology, using only distributed mechanisms.

\para{Propane FE to RIR}

The main differences between the FE and RIR are: $i)$ FE allows the programmer to specify constraints using a series of (modular) definitions, and combine them later, $ii)$ FE provides high-level names that abstract sets of routes and groups of prefixes/neighbors, and $iii)$ FE allows the preference operator to be used more flexibly.

A key constraint when translating FE to RIR is to ensure that all specified routes are well-formed. In particular, each regular path constraint $r$ must satisfy $r \subseteq \Out^* \cdot \In^+ \cdot \Out^*$. This ensures that the user only control traffic that goes through the user's network at some point, and that such traffic does not loop back multiple times to the user network.

The translation from \sysname to RIR is based on a set of rewriting rules.
The first step merges separate constraints. It takes the cross product of per-prefix constraints, where logical conjunction ($a \AND b$) is replaced by intersection on regular constraints ($a \Intersect b$), logical disjunction is replaced by union, and logical negation ($\NOT a$) is replaced by ($\Any \cap \NOT(a)$), where $\Any$ ensures the routes are well-formed.
%
For example, in the data center configuration from \S\ref{sec:propane}, combining the \CD{Locality} and \CD{Ownership} policies results in the following RIR:

%% \begin{lstlisting}[mathescape=true]
%% $\path{p_{g1}}{any \cap end(A)}$
%% $\path{p_{g2}}{any \cap end(B)}$
%% $\path{p_{l1}}{\neg enter(out) \cap end(E)}$
%% $\path{p_{l2}}{\neg enter(out) \cap end(F)}$
%% $\path{true}{exit(out)}$
%% \end{lstlisting}

\begin{code}
PG1 \Path \End(A)
PG2 \Path \End(B)
PL1 \Path \textbf{only}(\In) \Intersect \End(E)
PL2 \Path \textbf{only}(\In) \Intersect \End(F)
\True \Path \Exit(\Out)
\end{code}

The next step rewrites the high-level constraints such as \textit{enter} according to the equivalences in Figure~\ref{fig:rir-syntax}. Since preferences can only occur at the outermost level for an RIR expression, the final step is to ``lift" occurrences of the preference operator in each regular expression. For example, the regular expression $a \cdot (b \Prefer c) \cdot d$ is lifted to $(a \cdot b \cdot d) \Prefer (a \cdot c \cdot d)$ by distributing the preference over the sequence operator. In general, we employ the follwing distributivity equivalences:
%
\[
\begin{array}{c}
  x \odot (y_1 \Prefer \dots \Prefer y_n) = (x \odot y_1) \Prefer \dots \Prefer (x \odot y_n) \\
  (y_1 \Prefer \dots \Prefer y_n) \odot x = (y_1 \odot x) \Prefer \dots \Prefer (y_n \odot x)
\end{array}
\]
%
where $\odot$ stands for an arbitrary regular binary operator. Preferences nested under a unary operator, \textit{star} or \textit{negation}, are flagged by the compiler as invalid policies.

\subsection{Product Graph IR}

\newcommand{\state}[4]{\node[state,#3](#1)[#4]{#2};}
\newcommand{\transition}[4]{\path[->] (#1) edge [#4] node {#3} (#2);}

\begin{figure*}
  \begin{minipage}[t]{.5\linewidth}

  \hdr{\large Topology}{}
  \vspace*{-2\baselineskip}

  \includegraphics[width=.6\columnwidth]{figures/topology}

  \hdr{\large Policy Automata}{}
  \vspace*{-0.5\baselineskip}

  \begin{tikzpicture}[>=stealth',shorten >=1pt,auto,node distance=1.4cm,minimum size=1cm]
    \state{0}{$0$}{              }{}
    \state{1}{$1$}{right of=0}{}
    \state{2}{$2$}{right of=1}{}
    \state{3}{$3$}{right of=2}{}
    \state{4}{$4$}{right of=3}{}
    \state{5}{$5$}{right of=4}{accepting}
    \transition{0}{1}{out}{}
    \transition{1}{2}{D}{}
    \transition{2}{3}{C}{}
    \transition{3}{4}{A}{}
    \transition{4}{5}{W}{}
  \end{tikzpicture}

  \begin{tikzpicture}[>=stealth',shorten >=1pt,auto,node distance=1.6cm]
    \state{0}{$0$}{              }{}
    \state{1}{$1$}{right of=0}{}
    \state{2}{$2$}{right of=1}{}
    \state{3}{$3$}{right of=2}{}
    \state{4}{$4$}{right of=3}{accepting}
    \transition{0}{1}{out}{}
    \transition{1}{2}{in}{}
    \transition{2}{2}{A,C,D,E}{loop above}
    \transition{2}{3}{B}{}
    \transition{3}{3}{B}{loop above}
    \transition{3}{2}{A,C,D,E}{bend left}
    \transition{3}{4}{W}{}
  \end{tikzpicture}
  \end{minipage}
  %
  ~~
  ~~
  %
  \begin{minipage}[t]{.5\linewidth}
  \hdr{\large Product Graph IR}{}
  \vspace*{-1\baselineskip}
  %
  \includegraphics[width=.65\columnwidth]{figures/productgraph}
  \end{minipage}

  \hrulefill
  \vspace*{.4em}

  \caption{Example Product Graph IR construction for policy: $(\CD{W} \cdot \CD{A} \cdot \CD{C} \cdot \CD{D} \cdot \Out) \Prefer (\CD{W} \cdot \CD{B} \cdot \In^+ \cdot \Out)$.}
  \label{fig:example-compilation}
  \vspace{-1em}
\end{figure*}

Now that the user policy exists in a simplified form, we must consider topology. In particular, we want a compact representation that describes all the possible ways BGP route announcements can flow through the network subject to the policy and topology constraints. Our PGIR captures these dependencies by ``intersecting" each of the regular automata corresponding to the RIR path preferences, and the topology. Paths through the PGIR correspond to real paths through the topology that satisfy the user constraints.

%Finding paths through a graph subject to regular constraints has been studied extensively in the database literature~\ref{bib:todo}, and has been applied to networks in the past~\ref{bib:todo}.

\para{Formal definition}

% Formally define automata
The RIR policy is an ordered sequence of regular expressions $r_1 \Prefer \dots \Prefer r_j$. While paths talk about the direction traffic flows through the network, to implement the policy with BGP we are concerned about the way control-plane information is disseminated (i.e., route announcements flowing in the opposite direction). To capture this idea, for each regular expression $r_i$, we construct a deterministic finite state machine on the reversed regular expression. Each automaton is a tuple: ($\Sigma, Q_i, F_i, q_{0_i}, \sigma_i$). The alphabet $\Sigma$ consists of all all abstraction locations (routers or ASes), $Q_i$ is the set of states for automaton i, $F_i$ is the set of final states, $q_{0_i}$ is the initial state, and $\sigma_i \colon Q_i \times \Sigma \rightarrow Q_i$ is the state transition function.
%
% Formally define topology
The topology is represented as a graph ($V, E$), which consists of a set of vertices $V$, and a set of directed edges $E \colon 2^{V \times V}$.
%
% Formally define product graph
The combined PGIR is a tuple: ($V'$, $E'$, $s$, $e$, $P$) with
vertices $V' \colon V \times Q_1 \times \dots \times Q_j$,
edges $E' \colon 2^{V' \times V'}$,
a unique starting vertex $s$,
a unique ending vertex $e$,
and a preference function $P \colon V \rightarrow 2^{\set{1, \dots, j}}$ , which maps nodes in the product graph to a set of preference ranks.
For a vertex $v' = (v, \dots) \in V'$, we say $loc(v') = v$.
Two nodes $x$ and $y$ \emph{shadow} each other in the product graph when $x \in V'$ and $y \in V'$ and $loc(x) = loc(y)$ but $x \neq y$. That is, $x$ and $y$ shadow each other when they are different nodes in the PGIR that represent the same topology location.

\para{From RIR To PGIR}

% Formal product construction
Let $a_i$ and $b_i$ denote states in the regular policy automata.
The PGIR is constructed by adding an edge from $v_1 = (x, a_1, \dots, a_m)$ to $v_2 = (y, b_1, \dots, b_m)$ whenever $\sigma_i(a_i, y) = b_i$ for each $i$ and $(x,y) \in E$ is a valid topology link.
%
Additionally, we add edges from the start node $s$ to any $v = (x, a_1, \dots, a_m)$ when $\sigma_i(q_{0_i}, x) = a_i$ for each $i$.
%
The preference function $P$ for node $v = (x, a_1, \dots, a_m)$ is defined as $P(v) = \set{i~\vert~a_i \in F_i} $. That is, it records which preferences are achieved from each policy automaton for every node in the product construction.
%
Finally, there is an edge from each node in the PGIR such that $P(v) \neq \emptyset$ to the special end node $e$.

Intuitively, PGIR tracks which states of each automaton the policy is in as route announcements move between routers.
%
Consider the topology in Figure~\ref{fig:example-compilation}. Suppose we want a primary route from W that enters the network from $A$, and utilizes the A--C and C--D links. We also want a backup route that enters from $B$, and utilizes the B--C link, but is otherwise unconstrained. For simplicity, we assume that the route can end in either $X$, $Y$, or $Z$. The RIR for the policy is:
%
$$(\CD{W} \cdot \CD{A} \cdot \CD{C} \cdot \CD{D} \cdot \Out) \Prefer (\CD{W} \cdot \CD{B} \cdot \In^+ \cdot \Out)$$
%
Figure~\ref{fig:example-compilation} shows the policy automata for each regular expression preference. Since we are interested in the flow of control messages, the automata match backwards.
%
The figure also shows the PGIR after intersecting the topology and policy automata. Every path in it corresponds to a concrete path in the topology. In particular, every path through the PGIR that ends at a node $v$ such that the preference function $P(v) = \set{i_1, \dots, i_m}$ is non-empty, is a valid topological path that satisfies the policy constraints and results in a particular path with preferences $i_1$ through $i_m$.
%
For example, the path $X \cdot D \cdot C \cdot A \cdot W$ is a valid path in the topology that BGP route announcements might take, which would lead to obtaining the most prefered preference $1$.
BGP control messages can start from peer X, which would match the $\Out$ transition from both automata, leading to state $1$ in the first automaton, and state $1$ in the second automaton. This is reflected in the product graph by the node with state $(X,1,1)$. From here, if X were to advertise this route to D, it would result in the path $D \cdot X$, which would lead to state $2$ in the first automaton, and state $2$ in the second automaton, and so on.
%
The ``--" state indicates the corresponding automaton can not accept the current path or any extension of it. Since node $(W,5,-)$ is in an accepting state for the first automata, it indicates this path has preference 1.

\para{Minimization}

Before checking if policies captured by the PGIR are safe, we minimize the PGIR to improve our safety analysis.
The minimization is based on the observation that, although every path in the PGIR is a valid path in the topology, we do not want to consider paths that form loops. In particular, BGP's loop prevention mechanism forces an AS to reject any route for which it is already in the AS path.
%
For example, in Figure~\ref{fig:example-compilation}, the path $W \cdot A \cdot C \cdot B \cdot W$ is a valid topological path, leading to a path that satisfies the preference 1 policy, but which contains a loop.
%
It often improves the precision of the failure-safety analysis to eliminate cases that are not possible due to loops.
%For example, if we can safely remove a PGIR edge, then we do not have to consider what happens when the edge has failed.
We use graph dominators~\cite{tarjan-dominance} as a cheap approximation for removing many nodes and edges in the PGIR that are never on any \emph{simple} (loop free) path between the start and end nodes.
In the PGIR, a node $x$ dominates a node $y$ if $x$ appears on every path leading to $y$ from the start node.
%We can safely remove any PGIR node or edge as long as it never appears on any \emph{simple} path from the \textit{start} node to the \emph{end} node.
For example, node $(W,1,1)$ in Figure~\ref{fig:example-compilation} is never on a simple path to the end node since it must go through another $W$ in either case.
Colored nodes and dashed edges are removed after minimization since they are irrelvant to the BGP decision process.
%
%Although fully minimizing this graph is an NP-complete problem,  a simple and efficient algorithm based on graph dominators achieves largely the same effect, greatly simplifying the failure safety analysis.

%A node $x$ dominates $y$ if $x$ appears on every path leading to $y$ from a source node. Many efficient algorithms exist for finding graph dominators~\ref{bib:todo}. For minimization, we compute the dominator set for each node in both the product graph $G$ with respect to the start node, and in the reversed product graph $G^R$ with respect to the end node. The following rules enable efficient minimization of the product graph.
%
%\begin{itemize}
%  \item Remove nodes that never reach the end node
%  \item Remove nodes not reachable from the $start$ node
%  \item Remove any node $x$ such that $x$ is dominated by $y$ in $G$ or $G^R$, and $loc(x) = loc(y)$
%  \item Remove any edge from $x$ to $y$ in $G$ or $G^R$ if there is some node $z$ dominated by $y$ such that $loc(x) = loc(z)$
%\end{itemize}
%
%Repeated application of the above rules leads to removing the colored nodes and dashed lines in Figure~\ref{fig:example-compilation}.

\para{Failure Safety}

To implement path preferences in routing, BGP uses local preferences on a per-device basis. However, the distributed nature of BGP makes setting preferences locally to achieve a network-route routing policy difficult, particularly in the presence of failures. For example, imagine an extremely simple policy for the topology in Figure~\ref{fig:example-compilation}, which says to prefer one route over another:
%
$(\CD{W} \cdot \CD{A} \cdot \CD{C} \cdot \CD{D}) \Prefer (\CD{W} \cdot \CD{B} \cdot \CD{C} \cdot \CD{E})$.
%

How could such a policy be implemented in BGP? Suppose we set the local preference at router $C$ to prefer $D$ over $E$, and use the BGP MED attribute to influence $W$ to prefer $A$ over $B$. This might work as expected under normal conditions, however, if the A -- C link fails, then suddenly $C$ has made the wrong decision. Traffic will now follow the
$W \cdot B \cdot C \cdot D$ path, even though this path was not specified by the policy. This is a violation of \emph{soundness}: The distributed implementation has used a route not specified by the policy. To make matters worse, the second preference $W \cdot B \cdot C \cdot E$ is available in the network but not being used. This is a violation of \emph{completeness}: A path for the best possible route available exists in the network but is not being used by the distributed implementation.
We could solve the first problem by filtering routes that go through $D$ at $B$, however, we can not fix the second problem---$C$ has made the wrong choice. In fact, this policy can not be implemented in BGP in a way that is policy compliant under all failures.

%and can not get either the primary or backup route specified, despite the fact that the $W \cdot B \cdot C \cdot E$ path is available in the network.

At a first cut, any time a router must make a decision locally between several route options, there is the possibility it might choose incorrectly. The PGIR captures this notion of choice precisely. For example, router $D$ in Figure~\ref{fig:example-compilation} appears only once in the PGIR, possibly receiving announcements from $X$ or $Y$. However, regardless of whether $D$ chooses a route from $X$ or $Y$, the set of paths allowed by the policy after going through $D$ will be the same in either case. This remains true despite any failures that might have occurred in the network. Thus $D$ can safely prefer $X$ and $Y$ equally.

The more challenging case is when a topology location occurs in mutliple contexts in the PGIR. For example, in the compilation example from Figure~\ref{fig:example-compilation}, the topology node $C$ can receive an announcement from $E$ and later achieve the backup path, or it can receive an announcement from its neighbor $D$ and later achieve either the primary or backup path. Is it safe for $C$ to prefer its neighbor $D$ over its other neighbor $E$? The important observation is that, if $C$ prefers $D$, then it is never worse off---it will always achieve at least as good a path as if it had choosen $E$.
For example, suppose $C$ chooses a route from $D$, but cannot achieve its primary path because the $A$ -- $W$ link has failed. In this case, the advertisement from $C$ will still be sent along towards the ultimate backup location $W$. Since the $(C,3,2)$ node has the same one-step and two-step next hops as node $(C,-,2)$ in the product graph, no possible failure will prevent a route advertisment from reaching $(W,-,4)$ that wouldn't have otherwise prevented it if $C$ had choosen $E$.

In general, ensuring that an individual router's preferences respect the policy under all possible failures is hard, and enumerating all failures is intractable. We thus adopt a conservative analysis based on the observations above. The high-level idea is to order each PGIR node according to a \textit{can prefer} relation. For example, node $(C,3,2)$ can be preferred to node $(C,-,2)$, but not the other way around. Intuitively, a node $N_1$ can be preferred to another node $N_2$ when, for each preference $N_2$ might achieve, there is a better preference that $N_1$ will achieve regardless of failures. Formally this means that $N_1$ can be preferred to $N_2$ when:
%
$$\forall i, \exists j, j \leq i \wedge protect(G_j, N_1, G_i, N_2)$$
%
where \textit{protect} means that, from $N_1$ on the product graph restricted to nodes that can potentially achieve preference $j$ or better,
there is always a path to each ending location whenever this is a path from $N_2$ restricted to $G_i$.

\begin{algorithm}[t!]
\caption{Failure Protection}
\label{alg:failures}
\begin{algorithmic}[1]
\Procedure{Protect($G_1$, $N_1$, $G_2$, $N_2$)}{}
  \If {$loc(N_1) \neq loc(N_2)$} \Return false
  \EndIf
  \State $q \gets Queue()$
  \State $q.Enqueue (N_1, N_2)$
  \While {$!q.Empty()$}
    \State $(n_1,n_2) \gets q.Dequeue()$
    \For {$x$ in adj($G_2$, $n_2$)}
      \If {\big($\exists y \in$ adj($G_1$,$n_1$), shadows $x$\big) \textbf{or} \\
            \hspace{5.2em} \big($\exists y \in$ $G_1$, dominates $n_1$, shadows $x$\big)}
        \If {$(x,y)$ not marked}
          \State {mark $(x,y)$ as seen}
          \State $q.Enqueue(x,y)$
        \EndIf
      \Else { \Return $false$}
      \EndIf
    \EndFor
  \EndWhile
  \Return true
  \EndProcedure
\end{algorithmic}
\end{algorithm}

Algorithm~\ref{alg:failures} defines what it means for one node to \textit{protect} against the failures of another. It walks from nodes $N_1$ and $N_2$ in $G_j$ and $G_i$ respectively, and ensures that for every \textit{step} $N_2$ can take to some new topology location, $N_1$ can, at the very least, take an equivalent step.
When there is no equivalent step, the algorithm checks to see if there is an equivalent dominator (a topology location the advertisement must have passed through earlier) that can protect against failures instead.
Thus any time $N_2$ can get to a final location for preference $i$, $N_1$ can get to at least the same final locations for a better preference $j$. The algorithm terminates since the number of related states $(x,y)$ that can be expolored is finite.

For each router in the topology, local preferences are now obtained by  sorting the corresponding PGIR nodes according to the \textit{can prefer} relation. If two nodes are incomparable, the compiler rejects the policy as unimplementable.

\para{Avoiding Loops}

The checks for failure safety described above overlook one critical point: Paths for a higher preference might not exist due to loops. To avoid this situation, the compiler checks that, when node $x$ protects against failures for node $y$, its does so without using any nodes that shadow another node that appears ``above'' $x$ in the PGIR.
%It is fine however, to reuse the same nodes that appear above $x$, since any path that went through another node $z$ to get to $x$, but then looped through $z$ again, would be obtainable from $z$ itself.
%even if the analysis determines that a node $x$ protects against failures of another node $y$, it may be tricked into thinking $x$ has a particular path that will actually be discarded by BGP's loop prevention mechanism due to locations previously traversed before arriving at $x$. To avoid this situation, the compiler conservatively checks that $x$ protects against $y$ without using any nodes that shadow another node that appears above $x$ in the PGIR (i.e., nodes reachable from $x$ in $G_j^R$). It is fine however, to reuse the same nodes that appear above $x$, since any path that went through another node $z$ to get to $x$, but then looped through $z$ again, would be obtainable from $z$ itself.

\para{Aggregation-Induced Black Holes}
Because the \sysname compiler has the complete user policy and topology, it is also able to efficiently check that aggregates do not black hole traffic for up to $k$ failures. The algorithm for aggregation failure safety is detailed in Section~\ref{sec:black-holes} of the appendix.
\subsection{Abstract BGP}

%\begin{figure}[t!]
%\centering
%\includegraphics[width=.9\columnwidth]{figures/config}
%\caption{Abstract BGP configuration.}
%\label{fig:abgp-config}
%\end{figure}

\newcommand{\highlight}[1]{%
  \colorbox{red!50}{$\displaystyle#1$}}
\newcommand{\Router}[1]{\KW{Router} #1:}
\newcommand{\REGEX}[1]{\texttt{regex}(#1)}
\newcommand{\PEER}{\texttt{peer}}
\newcommand{\COMM}{\texttt{comm}}
\newcommand{\MED}{\texttt{MED}}
\newcommand{\Arrow}{\ensuremath{\leftarrow}}

%\begin{figure}[t!]
%\begin{lstlisting}[frame=single, mathescape=true]
%$\Router{A}$
%  Match $\PEER=C$
%    Export $\MED \leftarrow 80, \PEER \leftarrow W$
%$\Router{B}$
%  Match $\PEER = C$
%    Export $\COMM \leftarrow \text{noexport}, \MED \leftarrow 81, \PEER \leftarrow W$
%$\Router{C}$
%  Match$[LP=99]$ $\PEER = E $
%    Export $\PEER \leftarrow B$
%  Match $\PEER = D$
%    Export $\PEER \leftarrow A,B$
%$\Router{D}$
%  Match $\REGEX{X + Y}$
%    Export $\PEER \leftarrow C$
%$\Router{E}$
%  Match $\REGEX{Z}$
%    Export $\PEER \leftarrow C$
%\end{lstlisting}
%\label{fig:config-min}
%\caption{Abstract BGP minimized configurations}
%\end{figure}

The final stage of our compiler translates policies from PGIR to a vendor-neutral abstraction of BGP (ABGP).
% and then from ABGP to actual device configurations.

\para{From PGIR to ABGP}

Once we have the ordering on node preferences in the PGIR from the failure safety analysis, the translation to ABGP is straightforward. The idea is to encode the state of the automaton using BGP community values. Each router will match based on its peer and a community value corresponding to the state of the PGIR, and then update the state before exporting to the neighbors permitted by the PGIR. For example, router $A$ in Figure~\ref{fig:example-compilation} will allow an announcement from $C$ with a community value for state $(3,2)$ (and deny anything else). If it sees such an announcement, it will remove the old community value, and add a new one for state $(4,2)$ before exporting it to $W$.

To ensure preferred paths are always obtained, for each router $r$ in the topology, the compiler sets a higher local preference for neighbors of a more-preferred node for $r$ in the PGIR. For example, $C$ will prefer an advertisement from $D$ in state $(2,2)$ over an advertisement from $E$ in state $(-,2)$.

Since the compiler is only able to control community tagging only for routers under the control of the AS being programmed, it cannot match on communities for external ASes. Instead, it translates matches from external ASes into a BGP regular expression filter. For example, node $D$ in Figure~\ref{fig:example-compilation} would match the single hop external paths $X$ or $Y$. In general, if routes are allowed from beyond $X$ or $Y$, these will also be captured in the BGP regular expression filters. The unknown AS topology is modelled as a special node in the PGIR that generates a filter to match any sequence of ASes.

Finally, the external AS $W$ should prefer our internal router $A$ over $B$. In general, it is not possible to reliably control traffic entering the network beyond certain special cases. In this example, however, assuming our network and $W$ have an agreement to honor MEDs, the MED attribute can influence $W$ to prefer $A$ over $B$. Additionally, the compiler can use the BGP no-export community to ensure that no other AS beyond $W$ can send us traffic. The compiler can perform a simple analysis to determine when it can utilize BGP special attributes to ensure traffic enters the network in a particular way by looking at links in the product graph that cross from the internal topology to the external topology.
%Aggregation is another tool commonly used to influence how traffic can enter the network, by relying on BGP's longest prefix matching default. Since aggregates are provided as additional user constraints, the compiler can also make use of this information to check that incoming preferences can be met given the aggregates specified (e.g., to ensure that all external ASes prefer to enter from one location over another).
Figure~\ref{fig:abgp-config} shows the full configuration from the compilation example.

After configuration generation, the compiler further processes the ABGP policy, removing community tags when possible, combining filters, removing dead filters, and so on. In the compilation example, all community tags can be removed since there is never any ambiguity based on the router importing the route, and the neigbor the route is being imported from. Similarly, after removing the communities, the two rules at router $B$ are merged into a single rule.

\begin{figure}[t!]
%% \begin{lstlisting}[frame=single, mathescape=true]
%% $\Router{A}$
%%   Match $\PEER=C, \COMM=(3,2)$
%%     Export $\COMM \leftarrow (4,2), \MED \leftarrow 80, \PEER \leftarrow W$
%% $\Router{B}$
%%   Match $\PEER = C, \COMM = (-,2)$
%%     Export $\COMM \leftarrow (-,3), \COMM \leftarrow \text{noexport},$
%%            $\MED \leftarrow 81, \PEER \leftarrow W$
%%   Match $\PEER = C, \COMM = (3,2)$
%%     Export $\COMM \leftarrow (-,3), \COMM \leftarrow \text{noexport},$
%%            $\MED \leftarrow 81, \PEER \leftarrow W$
%% $\Router{C}$
%%   Match$[LP=99]$ $\PEER = E, \COMM = (-,2) $
%%     Export $\COMM \leftarrow (-,2), \PEER \leftarrow B$
%%   Match $\PEER = D, \COMM = (2,2)$
%%     Export $\COMM \leftarrow (3,2), \PEER \leftarrow A,B$
%% $\Router{D}$
%%   Match $\REGEX{X + Y}$
%%     Export $\COMM \leftarrow (2,2), \PEER \leftarrow C$
%% $\Router{E}$
%%   Match $\REGEX{Z}$
%%     Export $\COMM \leftarrow (-,2), \PEER \leftarrow C$
%% \end{lstlisting}
\begin{code}
  \Router{A}
    Match \PEER=C, \COMM=(3,2)
      Export \COMM \Arrow (4,2), \MED \Arrow 80, \PEER \Arrow W
  \Router{B}
    Match \PEER = C, \COMM = (-,2)
      Export \COMM \Arrow (-,3), \COMM \Arrow \text{noexport},
             \MED \Arrow 81, \PEER \Arrow W
    Match \PEER = C, \COMM = (3,2)
      Export \COMM \Arrow (-,3), \COMM \Arrow \text{noexport},
             \MED \Arrow 81, \PEER \Arrow W
  \Router{C}
    Match[LP=99] \PEER = E, \COMM = (-,2)
      Export \COMM \Arrow (-,2), \PEER \Arrow B
    Match \PEER = D, \COMM = (2,2)
      Export \COMM \Arrow (3,2), \PEER \Arrow A,B
  \Router{D}
    Match \REGEX{X + Y}
      Export \COMM \Arrow (2,2), \PEER \Arrow C
  \Router{E}
    Match \REGEX{Z}
      Export \COMM \Arrow (-,2), \PEER \Arrow C
  \end{code}
  \vspace{-2em}
  \caption{Abstract BGP router configurations. \label{fig:abgp-config}}
  \vspace{-1em}
\end{figure}


% $Id: 4ab38b73d20edef7032a785ba8fc6325fc362bd8 $

\section{SAT Solving Modulo Tableau and Rewriting Theories}
\label{sec:smt}

In this section, we recall the classic architecture of SMT solvers and we then
introduce the tableau and rewriting theories.

Compared to genuine tableau automated theorem provers, like \princess{} or
\zenon{} for example, our approach has the benefit of being versatile since the
tableau rules are actually integrated as a regular SMT theory. This way, the
tableau rules can be easily combined with other theories, such as equality logic
with uninterpreted functions or arithmetic. The way we integrate the tableau
rules into the SAT solver (mainly by boxing/unboxing first order formulas) is
close to what is done in the \satallax{} tool~\cite{CEB12}. The difference
resides in the fact that we are in a pure first order framework, which has
significant consequences in the management of quantifiers and unification in
particular (see Sec.~\ref{sec:super}).

Regarding the integration of rewriting, automated theorem provers rely on
several solutions (superposition rule for first order provers, triggers for SMT
solvers, etc.). But deduction modulo theory~\cite{DA03} is probably the most
general approach, where a theory can be partly turned in to a set of rewrite
rules over both terms and propositions. Several proof search methods have been
extended to deduction modulo theory, resulting in tools such as \iproverm{} and
\zenm{}. This paper can be seen as a continuation of these previous experiments
adapted to the framework of SMT solving.

\subsection{SAT Solving Modulo Theories}

In this subsection, we recall the classic architecture of SMT
solvers~\cite{BEA06}. We introduce $\mathcal{T}$ and $\mathcal{F}$ respectively
the sets of first order terms and formulas over the signature
$\mathcal{S}=(\mathcal{S}_\mathcal{F},\mathcal{S}_\mathcal{P})$, where
$\mathcal{S}_\mathcal{F}$ is the set of function symbols, and
$\mathcal{S}_\mathcal{P}$, the set of predicate symbols, such that
$\mathcal{S}_\mathcal{F}\cap\mathcal{S}_\mathcal{P}=\emptyset$. The set
$\mathcal{T}$ is extended with two kinds of terms specific to tableau proof
search, i.e. $\epsilon{}$-terms (used instead of Skolemization) of the form
$\epsilon(x).P(x)$, where $P(x)$ is a formula, and which means some $x$ that
satisfies $P(x)$, if it exists, and metavariables (often named free variables in
the tableau-related literature) of the form $X_P$, where $P$ is the formula that
introduces the metavariable, and which is either $\forall{}x.Q(x)$ or
$\neg\exists{}x.Q(x)$, with $Q(x)$ a formula.

A boxed formula is of the form $\lfloor{}P\rfloor$, where $P$ is a formula. A
boxed formula is called atom, and a literal is either an atom, or the negation
of an atom. A literal is such that there is no negation on top of the boxed
formula (which means that $\lfloor\neg{}P\rfloor=\neg\lfloor{}P\rfloor$, and
$\neg\neg\lfloor{}P\rfloor=\lfloor{}P\rfloor$). A clause is a disjunction of
literals. It should be noted that SAT solving usually reasons over sets of
clauses composed of first order literals; here, a literal is a first order
formula (possibly with quantifiers), which requires to box formulas to get a
regular SAT solving problem where boxed formulas are propositional variables.

An assignment of the boxed formulas is a partial function that assigns to boxed
formulas an element of $\{\top,\bot\}$ (values of the Boolean algebra). Given a
clause $C$, we say that $C$ is valid and write $\models{}C$, if $C$ evaluates to
$\top$ for every assignment. Given a set of clauses $S$, we say that $S$ is
valid and write $\models{}S$, if each clause of $S$ is valid. Given two sets of
clauses $S$ and $S'$, we say that $S$ implies $S'$ and write $S\models{}S'$ if,
for every assignment, if each clause of $S$ evaluates to $\top$ then each clause
of $S'$ evaluates to $\top$. If $S$ is a set of literals, then we say that $S$
is a model of $S'$.

A theory $T$ is a set of formulas over a given signature $\mathcal{S}$, which
are called axioms of the theory $T$. A clause $C$ is a tautology in the theory
$T$, written $\models_TC$, if the axioms of $T$ imply the formula $C'$, where
$C'$ is the clause $C$ where all the boxed formulas have been unboxed. A set of
clauses $S$ is a tautology in the theory $T$ if each clause of $S$ is a
tautology in $T$. Given two sets of clauses $M$ and $S$, we say that $M$ is a
model of $S$ modulo the theory $T$ and write $M\models_TS$, if $M$ is a model of
$S$ and $M$ is a tautology in $T$.

A SAT solver modulo theories is characterized by a set of theories $S_T$ and an
internal state of the form $M\parallel{}S$, where $M$ is an ordered list of
literals (if $M=l_1,l2,\ldots,l_n$ then $l_1<l_2<\ldots<l_n$) and $S$ a set of
clauses. Intuitively, $M$ represents the model of $S$, which will be built
progressively (initially, we have $M=\emptyset$) by the application of a set of
rules over the internal state of the solver. This set of rules is presented in
Fig.~\ref{fig:smt}. Each time a literal is propagated (rule
``$\mathrm{unit~prop}$'') or decided (rule ``$\mathrm{decide}$''), it is given
to each theory, which may generate new clauses (rule ``$\mathrm{learn}$''). If a
clause is unvalidated by the model, then it is possible to backtrack the
decision over a given literal of the model if it exists (rule
``$\mathrm{backjump}$''). There are two final states in this transition
system. The first one is ``$\mathrm{unsat}$'', which means that $S$ is
unsatisfiable (rule ``$\mathrm{unsat}$''). The second one is $M\parallel{}S$,
where $M$ is a model of $S$ and a tautology for every theory in $S_T$. In this
last case, if $M$ is not a tautology in a theory $T$, this theory generates a
set of conflict clauses (rule ``$\mathrm{learn}$'').

\begin{figure}[t]
\parbox{\textwidth}
{\small
\begin{center}
$\begin{array}{l@{\hspace{0.4cm}}l@{\hspace{0.4cm}}l}
M\parallel{}S,C\lor{}l\longrightarrow{}M,l\parallel{}S,C\lor{}l &
(\mathrm{unit~prop}) & \mbox{if }l\not\in{}M\mbox{ and }M\models{}\neg{}C\\\\

M\parallel{}S\longrightarrow{}M,l^\mathrm{d}\parallel{}S & (\mathrm{decide}) &
\mbox{if }l\not\in{}M\mbox{, and }l\in{}S\mbox{ or }\neg{}l\in{}S\\\\

M\parallel{}S,C\longrightarrow\mathrm{unsat} & (\mathrm{unsat}) &
\mbox{if }M'\models{}\neg{}C\mbox{ s.t. }M'\subseteq{}M\\
&& \mbox{and there is no }l^\mathrm{d}\leq{}l'\mbox{ in }M\\
&& \mbox{for all }l'\in{}M'\\\\

M,l^\mathrm{d},M'\parallel{}S,C\longrightarrow{}M,l'\parallel{}S,C &
(\mathrm{backjump}) & \mbox{if }M,l^\mathrm{d},M'\models\neg{}C
\mbox{, and there is}\\
&& \mbox{some clause }C'\lor{}l' s.t.:\\
&& l'\not\in{}M\mbox{, and }l'\in{}S\mbox{ or }\neg{}l'\in{}S\\
&& \mbox{or }l'\in{}M,l^\mathrm{d},M'\mbox{ or }
\neg{}l'\in{}M,l^\mathrm{d},M',\\
&& \mbox{and }S,C\models{}C'\lor{}l'\mbox{, and }M\models\neg{}C'\\\\

M\parallel{}S\longrightarrow{}M\parallel{}S,S'
& (\mathrm{learn}) & \models_TS'\mbox{, where }T\mbox{ is a theory}
\end{array}$
\end{center}}
\caption{Rules of SAT Solving Modulo Theory}
\label{fig:smt}
\end{figure}

\subsection{The Tableau Theory}

In the SAT solver previously described, the tableau proof search method is
integrated as a regular theory. When a literal is propagated or decided, we
generate a set of clauses corresponding to the application of a tableau rule
over the logical connective at root of the formula in the box of the
literal. More precisely, for a literal $l$, we generate the set of clauses
$\llbracket{}l\rrbracket$, where the function $\llbracket\cdot\rrbracket$ is
described by the rules of Fig.~\ref{fig:tabth}. When a literal is propagated or
decided, we use all the rules of Fig.~\ref{fig:tabth} except the instantiation
$\gamma$-rules (rules $\gamma_{\forall\mathrm{inst}}$ and
$\gamma_{\neg\exists\mathrm{inst}}$). It should be noted that we use the same
names for the rules than in tableau calculus ($\alpha$-rules, $\beta$-rules,
etc.), but there is no precedence between rules and therefore no priority in the
application of the rules contrary to the tableau proof search method (where
$\alpha$ rules are applied before $\beta$-rules, and so on).

When the SAT solver reaches a state $M\parallel{}S$, where $M$ is a model of
$S$, we look for a conflict in $M$ between two literals by unification and we
generate the clauses corresponding to the instantiation of the metavariables
using the result of the unification. More precisely, if there exist two literals
$l$ and $\neg{}l'$ in $M$ such that $l=\lfloor{}Q\rfloor$ and
$l'=\lfloor{}R\rfloor$, with $Q$ and $R$ two formulas, then for each
substitution $(X_{\forall{}x.P(x)}\mapsto{}t)\in\mathrm{mgu}(Q,R)$ (resp.
$(X_{\neg\exists{}x.P(x)}\mapsto{}t)\in\mathrm{mgu}(Q,R)$) such that there is no
$Y_T\in\forall{}x.P(x)$ (resp. $Y_T\in\neg\exists{}x.P(x)$), we can generate the
clauses $\llbracket\lfloor\forall{}x.P(x)\rfloor{}\rrbracket$
(resp. $\llbracket\neg\lfloor\exists{}x.P(x)\rfloor{}\rrbracket$) using the rule
$\gamma_{\forall\mathrm{inst}}$ (resp. $\gamma_{\neg\exists\mathrm{inst}}$) of
Fig.~\ref{fig:tabth}.

\begin{figure}[t]
\parbox{\textwidth}
{\small
\underline{Analytic Rules}
\begin{center}
$\begin{array}{lll@{\hspace{0.5cm}}l}
\llbracket\lfloor{}P\land{}Q\rfloor{}\rrbracket & = &
\neg\lfloor{}P\land{}Q\rfloor\lor\lfloor{}P\rfloor,
\neg\lfloor{}P\land{}Q\rfloor\lor\lfloor{}Q\rfloor & (\alpha_\land)\\\\

\llbracket\neg\lfloor{}P\land{}Q\rfloor{}\rrbracket & = &
\lfloor{}P\land{}Q\rfloor\lor\neg\lfloor{}P\rfloor\lor\neg\lfloor{}Q\rfloor &
(\beta_{\neg\land})\\\\

\llbracket{}\lfloor{}P\lor{}Q\rfloor{}\rrbracket & = &
\neg\lfloor{}P\lor{}Q\rfloor\lor\lfloor{}P\rfloor\lor\lfloor{}Q\rfloor &
(\beta_\lor)\\\\

\llbracket\neg\lfloor{}P\lor{}Q\rfloor{}\rrbracket & = &
\lfloor{}P\lor{}Q\rfloor\lor\neg\lfloor{}P\rfloor,
\lfloor{}P\lor{}Q\rfloor\lor\neg\lfloor{}Q\rfloor &
(\alpha_{\neg\lor})\\\\

\llbracket{}\lfloor{}P\Rightarrow{}Q\rfloor{}\rrbracket & = &
\neg\lfloor{}P\Rightarrow{}Q\rfloor\lor\neg\lfloor{}P\rfloor\lor
\lfloor{}Q\rfloor & (\beta_\Rightarrow)\\\\

\llbracket\neg\lfloor{}P\Rightarrow{}Q\rfloor{}\rrbracket & = &
\lfloor{}P\Rightarrow{}Q\rfloor\lor\lfloor{}P\rfloor,
\lfloor{}P\Rightarrow{}Q\rfloor\lor\neg\lfloor{}Q\rfloor &
(\alpha_{\neg\Rightarrow})\\\\

\llbracket{}\lfloor{}P\Leftrightarrow{}Q\rfloor{}\rrbracket & = &
\neg\lfloor{}P\Leftrightarrow{}Q\rfloor\lor\lfloor{}P\Rightarrow{}Q\rfloor,
\neg\lfloor{}P\Leftrightarrow{}Q\rfloor\lor\lfloor{}Q\Rightarrow{}P\rfloor &
(\beta_\Rightarrow)\\\\

\llbracket{}\neg\lfloor{}P\Leftrightarrow{}Q\rfloor{}\rrbracket &
= &
\lfloor{}P\Leftrightarrow{}Q\rfloor\lor\neg\lfloor{}P\Rightarrow{}Q\rfloor\lor
\neg\lfloor{}Q\Rightarrow{}P\rfloor & (\beta_{\neg\Rightarrow})
\end{array}$
\end{center}

\underline{$\delta$-Rules}
\begin{center}
$\begin{array}{lll@{\hspace{0.5cm}}l}
\llbracket\lfloor\exists{}x.P(x)\rfloor{}\rrbracket & = &
\neg{}\lfloor\exists{}x.P(x)\rfloor\lor\lfloor{}P(\epsilon(x).P(x))\rfloor &
(\delta_\exists)\\\\

\llbracket\neg\lfloor\forall{}x.P(x)\rfloor{}\rrbracket & = &
\lfloor\forall{}x.P(x)\rfloor\lor\neg\lfloor{}P(\epsilon(x).\neg{}P(x))\rfloor &
(\delta_{\neg\forall})
\end{array}$
\end{center}

\underline{$\gamma$-Rules}
\begin{center}
$\begin{array}{lll@{\hspace{0.5cm}}l}
\llbracket\lfloor\forall{}x.P(x)\rfloor{}\rrbracket & = &
\neg\lfloor\forall{}x.P(x)\rfloor\lor\lfloor{}P(X_{\forall{}x.P(x)})\rfloor &
(\gamma_{\forall{}M})\\\\

\llbracket\neg\lfloor\exists{}x.P(x)\rfloor{}\rrbracket & = &
\lfloor\exists{}x.P(x)\rfloor\lor
\neg\lfloor{}P(X_{\neg\exists{}x.P(x)})\rfloor &
(\gamma_{\neg\exists{}M})\\\\

\llbracket\lfloor\forall{}x.P(x)\rfloor{}\rrbracket & = &
\neg\lfloor\forall{}x.P(x)\rfloor\lor\lfloor{}P(t)\rfloor &
(\gamma_{\forall\mathrm{inst}})\\\\

\llbracket\neg\lfloor\exists{}x.P(x)\rfloor{}\rrbracket & = &
\lfloor\exists{}x.P(x)\rfloor\lor\neg\lfloor{}P(t)\rfloor &
(\gamma_{\neg\exists\mathrm{inst}})
\end{array}$
\end{center}}
\caption{Rules of Tableau Theory}
\label{fig:tabth}
\end{figure}

To show how this theory works, let us prove that
$\exists{}x.P(x)\Rightarrow{}P(a)\land{}P(b)$, where $P$ is a predicate symbol
and $a,b$ two constants. The SAT solver is initiated with the state
$\emptyset\parallel
\neg\lfloor\exists{}x.P(x)\Rightarrow{}P(a)\land{}P(b)\rfloor$, and the proof is
described in Fig.~\ref{fig:tab}, where $X$ is a shortcut for
$X_{\neg\exists{}x.P(x)\Rightarrow{}P(a)\land{}P(b)}$. It should be noted that
to do this proof in sequent calculus, a right contraction is necessary to
instantiate the formula twice (with $a$ and $b$). In Fig.~\ref{fig:tab}, it is
done by propagating the literal $\lfloor{}P(X)\rfloor{}$, which can provide as
many instantiations (by unification) as necessary.

\begin{figure}[t!]
\parbox{\textwidth}
{\small
\begin{center}
$\begin{array}{lcl}
\emptyset\parallel\neg{}A & \longrightarrow & (\mathrm{unit~prop})\\

\boldsymbol{\neg{}A}\parallel{}\neg{}A & \longrightarrow & (\mathrm{learn})\\

\neg{}A\parallel\neg{}A,\boldsymbol{A\lor\neg{}B} & \longrightarrow &
(\mathrm{unit~prop})\\

\neg{}A,\boldsymbol{\neg{}B}\parallel\neg{}A,A\lor\neg{}B & \longrightarrow &
(\mathrm{learn})\\

\neg{}A,\neg{}B\parallel\neg{}A,A\lor\neg{}B,\boldsymbol{B\lor{}C},
\boldsymbol{B\lor\neg{}D} & \longrightarrow & (\mathrm{unit~prop})\times{}2\\

\neg{}A,\neg{}B,\boldsymbol{C},\boldsymbol{\neg{}D}\parallel\neg{}A,
A\lor\neg{}B,B\lor{}C,B\lor\neg{}D & \longrightarrow & (\mathrm{learn})\\

\neg{}A,\neg{}B,C,\neg{}D\parallel\neg{}A,A\lor\neg{}B,B\lor{}C,B\lor\neg{}D,\\
~~~~\boldsymbol{D\lor\neg{}E\lor\neg{}F} & \longrightarrow & (\mathrm{decide})\\

\neg{}A,\neg{}B,C,\neg{}D,\boldsymbol{\neg{}E^d}\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C, & \longrightarrow &
(\mathrm{learn})\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F && \{X\mapsto{}a\}=\mathrm{mgu}(C,E)\\

\neg{}A,\neg{}B,C,\neg{}D,\neg{}E^d\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C,\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,\boldsymbol{A\lor\neg{}G} &
\longrightarrow & (\mathrm{unit~prop})\\

\neg{}A,\neg{}B,C,\neg{}D,\neg{}E^d,\boldsymbol{\neg{}G}\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C,\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,A\lor\neg{}G & \longrightarrow &
(\mathrm{learn})\\

\neg{}A,\neg{}B,C,\neg{}D,\neg{}E^d,\neg{}G\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C,\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,A\lor\neg{}G,\boldsymbol{G\lor{}E},
\boldsymbol{G\lor\neg{}D} & \longrightarrow & (\mathrm{backjump})\\

\neg{}A,\neg{}B,C,\neg{}D,\boldsymbol{E}\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C,\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,A\lor\neg{}G,G\lor{}E,G\lor\neg{}D &
\longrightarrow & (\mathrm{unit~prop})\times{}2\\

\neg{}A,\neg{}B,C,\neg{}D,E,\boldsymbol{\neg{}F},\boldsymbol{\neg{}G}\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C, & \longrightarrow & (\mathrm{learn})\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,A\lor\neg{}G,G\lor{}E,G\lor\neg{}D &&
\{X\mapsto{}b\}=\mathrm{mgu}(C,F)\\

\neg{}A,\neg{}B,C,\neg{}D,E,\neg{}F,\neg{}G\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C,\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,A\lor\neg{}G,G\lor{}E,G\lor\neg{}D\\
~~~~\boldsymbol{A\lor\neg{}H} & \longrightarrow & (\mathrm{unit~prop})\\

\neg{}A,\neg{}B,C,\neg{}D,E,\neg{}F,\neg{}G,\boldsymbol{\neg{}H}\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C,\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,A\lor\neg{}G,G\lor{}E,G\lor\neg{}D\\
~~~~A\lor\neg{}H & \longrightarrow & (\mathrm{learn})\\

\neg{}A,\neg{}B,C,\neg{}D,E,\neg{}F,\neg{}G,\neg{}H\parallel
\neg{}A,A\lor\neg{}B,B\lor{}C,\\
~~~~B\lor\neg{}D,D\lor\neg{}E\lor\neg{}F,A\lor\neg{}G,G\lor{}E,G\lor\neg{}D\\
~~~~A\lor\neg{}H,\boldsymbol{H\lor{}F},\boldsymbol{H\lor{}\neg{}D} &
\longrightarrow & (\mathrm{unsat})\\

\mathrm{unsat}
\end{array}$
\end{center}
\begin{flushleft}
$\begin{array}{l}
\mbox{where:}\\
\begin{array}{ll}
~~~~A\equiv\lfloor\exists{}x.P(x)\Rightarrow{}P(a)\land{}P(b)\rfloor &
~~~~B\equiv\lfloor{}P(X)\Rightarrow{}P(a)\land{}P(b)\rfloor\\
~~~~C\equiv\lfloor{}P(X)\rfloor{} &
~~~~D\equiv\lfloor{}P(a)\land{}P(b)\rfloor{}\\
~~~~E\equiv\lfloor{}P(a)\rfloor &
~~~~F\equiv\lfloor{}P(b)\rfloor\\
~~~~G\equiv{}\lfloor{}P(a)\Rightarrow{}P(a)\land{}P(b)\rfloor &
~~~~H\equiv{}\lfloor{}P(b)\Rightarrow{}P(a)\land{}P(b)\rfloor
\end{array}
\end{array}$
\end{flushleft}}
\caption{Example of Proof Using the Tableau Theory}
\label{fig:tab}
\end{figure}

\subsection{The Rewriting Theory}
\label{sec:rew}

A rewriting theory allows us to introduce a computational behavior to the SAT
solver. We aim to integrate rewriting in the broadest sense of the term as
proposed by deduction modulo theory. Deduction modulo theory~\cite{DA03} focuses
on the computational part of a theory, where axioms are transformed into rewrite
rules, which induces a congruence over propositions, and where reasoning is
performed modulo this congruence. In deduction modulo theory, this congruence is
then induced by a set of rewrite rules over both terms and propositions.

In the following, we borrow some of the notations and definitions
of~\cite{DA03}. We call $\mathrm{FV}$ the function that returns the set of
free variables of a term or a formula. A term rewrite rule is a pair of terms
denoted by $l\rew{}r$, where $\mathrm{FV}(r)\subseteq\mathrm{FV}(l)$. A
proposition rewrite rule is a pair of propositions denoted by $l\rew{}r$, where
$l$ is an atomic proposition and $r$ is an arbitrary proposition, and where
$\mathrm{FV}(r)\subseteq\mathrm{FV}(l)$. A class rewrite system is a pair of
rewrite systems, denoted by $\mathcal{RE}$, consisting of $\mathcal{R}$, a set
of proposition rewrite rules, and $\mathcal{E}$, a set of term rewrite rules.

Given a class rewrite system $\mathcal{RE}$, the relations $=_\mathcal{E}$ and
$=_\mathcal{RE}$ are the congruences generated respectively by the sets
$\mathcal{E}$ and $\mathcal{R}\cup\mathcal{E}$. In the following, we use the
standard concepts of subterm and term replacement: given an occurrence $\omega$
in a proposition $P$, we write $P_{|\omega}$ for the term or proposition at
$\omega$, and $P[t]_\omega$ for the proposition obtained by replacing
$P_{|\omega}$ by $t$ in $P$ at $\omega$. Given a class rewrite system
$\mathcal{RE}$, the proposition $P$ $\mathcal{RE}$-rewrites to $P'$, denoted by
$P\rew_\mathcal{RE}P'$, if $P=_\mathcal{E}Q$, $Q_{|\omega}=\sigma(l)$, and
$P'=_\mathcal{E}Q[\sigma(r)]_\omega$, for some rule $l\rew{}r\in\mathcal{R}$,
some proposition $Q$, some occurrence $\omega$ in $Q$, and some substitution
$\sigma$.

The relation $=_\mathcal{RE}$ is not decidable in general, but there are some
cases where this relation is decidable depending on the class rewrite system
$\mathcal{RE}$ and the rewrite relation $\rew_\mathcal{RE}$. In particular, if
the rewrite relation $\rew_\mathcal{RE}$ is confluent and (weakly) terminating,
then the relation $=_\mathcal{RE}$ is decidable.

The rewriting theory is integrated into the SAT solver in a similar way than for
the tableau theory. When a literal is propagated or decided, we generate a
clause introducing a new formula that is congruent to the formula in the box of
the literal. More precisely, given a literal $\lfloor{}P\rfloor$, where $P$ is a
formula, and a formula $P'$ such that $P=_\mathcal{RE}P'$, we generate the
following clause:

$$\left(\bigvee_{(l,r)\in\mathcal{R}}
\neg\lfloor\forall{}\vec{x}.l\Leftrightarrow{}r\rfloor\right)\lor
\left(\bigvee_{(l,r)\in\mathcal{E}}\neg\lfloor\forall{}\vec{x}.l=r\rfloor\right)\lor
\lfloor{}P\Leftrightarrow{}P'\rfloor$$

where $\vec{x}=\mathrm{FV}(l)\cup\mathrm{FV}(r)$.

It should be noted that in usual SMT solvers, rewriting can be emulated by means
of triggers that are actually the left-hand side members of the class rewrite
system $\mathcal{RE}$ introduced above. But in our rewriting theory, we can
generate the formula where rewritings have been done, while triggers can just
generate bindings, i.e. instances of the rewrite rules, which are used later to
relate the initial and rewritten formulas. Moreover, in our case, we can perform
several rewritings at once, while a trigger can only emulate one rewriting at a
time.

Let us illustrate the use of the rewriting theory by means of an example in set
theory. Let us prove that
$(\forall{}s,t.s\subseteq{}t\Leftrightarrow{}\forall{}x.x\in{}s\Rightarrow{}
x\in{}t)\Rightarrow{}a\subseteq{}a$, where $a$ is a constant. The proof is given
by Fig.~\ref{fig:rew}, where the rules ``$\mathrm{learn~tab}$'' and
``$\mathrm{learn~rew}$'' represent the rule ``$\mathrm{learn}$'' using
respectively the tableau and rewriting theories.

\begin{figure}[t!]
\parbox{\textwidth}
{\small
\begin{center}
$\begin{array}{lcl}
\emptyset\parallel
\neg{}A & \longrightarrow & (\mathrm{unit~prop})\\

\boldsymbol{\neg{}A}\parallel{}\neg{}A & \longrightarrow &
(\mathrm{learn~tab})\\

\neg{}A\parallel
\neg{}A,\boldsymbol{A\lor{}B},\boldsymbol{A\lor\neg{}C} & \longrightarrow &
(\mathrm{unit~prop})\times{}2\\

\neg{}A,\boldsymbol{B},\boldsymbol{\neg{}C}\parallel
\neg{}A,A\lor{}B,A\lor\neg{}C & \longrightarrow & (\mathrm{learn~rew})\\

\neg{}A,B,\neg{}C\parallel\neg{}A,A\lor{}B,A\lor\neg{}C,
\boldsymbol{\neg{}B\lor{}D} & \longrightarrow & (\mathrm{unit~prop})\\

\neg{}A,B,\neg{}C,\boldsymbol{D}\parallel\neg{}A,A\lor{}B,A\lor\neg{}C,
\neg{}B\lor{}D & \longrightarrow & (\mathrm{learn~tab})\\

\neg{}A,B,\neg{}C,D\parallel\neg{}A,A\lor{}B,A\lor\neg{}C,\neg{}B\lor{}D,
\boldsymbol{\neg{}D\lor{}E},\boldsymbol{\neg{}D\lor{}F} & \longrightarrow &
(\mathrm{unit~prop})\times{}2\\

\neg{}A,B,\neg{}C,D,\boldsymbol{E},\boldsymbol{F}\parallel
\neg{}A,A\lor{}B,A\lor\neg{}C,\neg{}B\lor{}D,\neg{}D\lor{}E,\\
~~~~\neg{}D\lor{}F & \longrightarrow & (\mathrm{learn~tab})\\

\neg{}A,B,\neg{}C,D,E,F\parallel\neg{}A,A\lor{}B,A\lor\neg{}C,\neg{}B\lor{}D,
\neg{}D\lor{}E\\
~~~~\neg{}D\lor{}F,\boldsymbol{\neg{}F\lor\neg{}G\lor{}C} & \longrightarrow &
(\mathrm{unit~prop})\\

\neg{}A,B,\neg{}C,D,E,F,\boldsymbol{\neg{}G}\parallel\neg{}A,A\lor{}B,
A\lor\neg{}C,\neg{}B\lor{}D,\neg{}D\lor{}E\\
~~~~\neg{}D\lor{}F,\neg{}F\lor\neg{}G\lor{}C & \longrightarrow &
(\mathrm{learn~tab})\\

\neg{}A,B,\neg{}C,D,E,F,\neg{}G\parallel\neg{}A,A\lor{}B,A\lor\neg{}C,
\neg{}B\lor{}D,\neg{}D\lor{}E\\
~~~~\neg{}D\lor{}F,\neg{}F\lor\neg{}G\lor{}C,\boldsymbol{G\lor\neg{}H} &
\longrightarrow & (\mathrm{unit~prop})\\

\neg{}A,B,\neg{}C,D,E,F,\neg{}G,\boldsymbol{\neg{}H}\parallel\neg{}A,A\lor{}B,
A\lor\neg{}C,\neg{}B\lor{}D\\
~~~~\neg{}D\lor{}E,\neg{}D\lor{}F,\neg{}F\lor\neg{}G\lor{}C,
G\lor\neg{}H & \longrightarrow & (\mathrm{learn~tab})\\

\neg{}A,B,\neg{}C,D,E,F,\neg{}G,\neg{}H\parallel\neg{}A,A\lor{}B,A\lor\neg{}C,
\neg{}B\lor{}D\\
~~~~\neg{}D\lor{}E,\neg{}D\lor{}F,\neg{}F\lor\neg{}G\lor{}C,
G\lor\neg{}H,\boldsymbol{H\lor{}I},\boldsymbol{H\lor\neg{}I} & \longrightarrow &
(\mathrm{unit~prop})\\

\neg{}A,B,\neg{}C,D,E,F,\neg{}G,\neg{}H,\boldsymbol{I}\parallel\neg{}A,A\lor{}B,
A\lor\neg{}C,\neg{}B\lor{}D\\
~~~~\neg{}D\lor{}E,\neg{}D\lor{}F,\neg{}F\lor\neg{}G\lor{}C,
G\lor\neg{}H,H\lor{}I,H\lor\neg{}I & \longrightarrow & (\mathrm{unsat})\\

\mathrm{unsat}
\end{array}$
\end{center}
\begin{flushleft}
$\begin{array}{l}
\mbox{where:}\\
\begin{array}{ll}
\multicolumn{2}{l}{
~~~~A\equiv\lfloor(\forall{}s,t.s\subseteq{}t\Leftrightarrow{}
\forall{}x.x\in{}s\Rightarrow{}x\in{}t)\Rightarrow{}a\subseteq{}a\rfloor}\\
~~~~B\equiv\lfloor\forall{}s,t.s\subseteq{}t\Leftrightarrow{}
\forall{}x.x\in{}s\Rightarrow{}x\in{}t\rfloor &
~~~~C\equiv\lfloor{}a\subseteq{}a\rfloor\\
~~~~D\equiv\lfloor{}a\subseteq{}a\Leftrightarrow
\forall{}x.x\in{}a\Rightarrow{}x\in{}a\rfloor &
~~~~E\equiv\lfloor{}a\subseteq{}a\Rightarrow
\forall{}.x\in{}a\Rightarrow{}x\in{}a\rfloor\\
~~~~F\equiv\lfloor(\forall{}.x\in{}a\Rightarrow{}x\in{}a)
\Rightarrow{}a\subseteq{}a\rfloor &
~~~~G\equiv\lfloor\forall{}.x\in{}a\Rightarrow{}x\in{}a\rfloor\\
~~~~H\equiv\lfloor\epsilon_x\in{}a\Rightarrow\epsilon_x\in{}a\rfloor &
~~~~I\equiv\lfloor\epsilon_x\in{}a\rfloor
\end{array}\\\\
\mbox{with: }\epsilon_x=\epsilon(x).\neg(x\in{}a\Rightarrow{}x\in{}a)
\end{array}$
\end{flushleft}}
\caption{Example of Proof Using the Rewriting Theory}
\label{fig:rew}
\end{figure}


\section{Conclusions}
\frame{\tableofcontents[currentsection, hideothersubsections]}

\begin{frame}
\frametitle{Conclusions}

SGD can directly minimize the risk function
\begin{itemize}
    \item by sampling a point i.i.d from D \textbf{and}
    \item using a subgradient of the loss of the current hypothesis at this point as
    an unbiased estimate of the (sub)gradient of the risk function.
\end{itemize}
\vspace{5mm}

SGD's \textbf{number of iterations} \\
guarantees an expected objective of at most $\epsilon$ plus the optimal objective
\vspace{5mm}

WARN: Skipped (Subsub)sections:\\
\begin{itemize}
    \item any proofs
    \item 14.2.2 Subgradients of Lipschitz Functions
    \item 14.4 Variants
\end{itemize}

\end{frame}


\section{Real: 32-bit IEEE Reals}

The basic library architecture is based on 64-bit reals.  But it
is sometimes handy to use 32-bit reals.  And if you are going to use
them, it helps to have the standard constants and functions
available.  We have provided wrappers for the normal Math.i3 functions,
in order to hide the type conversions to and from 64-bit reals.

However, that means the 32-bit versions of the functions
are {\em slower} than the 64-bit versions.  Eventually, perhaps
we should have direct calls to a library of 32-bit real assembler
code.  For now, Real32.i3/m3 is just a convenience.


\section{Notations}
\begin{table}[H]
  \centering
  \caption{Important Notations.}
  \label{Notation}
  \begin{tabular*}{\hsize}{@{\extracolsep{\fill}}clc}
    \toprule
    Symbol & Description & Unit \\
    \midrule
    $S_{cylinder}$ & side surface area of cylinder & $m^2$ \\
    $\bm{E}$ & electric field intensity on the surface of uniformly charged cylinder & $N/C$ \\
    $\Phi$ & electric flux & $N\cdot m^2/C$ \\
    $DR$ & decomposition rate & Unitless \\
    $\alpha$ & fungi linear density & Unitless \\
    $\varepsilon_{DR}$ & environmental decomposition constant & Unitless \\
    $ACT$ & fungal activity factor & Unitless \\
    $\beta$ & fungi areal density & Unitless \\
    $HER$ & hyphal extension rate & $mm/day$ \\
    $RMT$ & relative moisture tolerance & Unitless \\
    $R^2$ & sample correlation coefficient & Unitless \\
    $N(t)$ & current population size & Unitless \\
    $RGB$ & relative growth blocking index & Unitless \\
    $X$ & time series data & Unitless \\
    \bottomrule
  \end{tabular*}
\end{table}


\section{Preface}
\xymatrix{
*+[F]{Problems}\ar@//[r]_<{\txt{\\$\downarrow$\\modeling error}} & *+[F]{Models}\ar@//[r]^<{\txt{approximation error\\$\uparrow$\\}} & *+[F]{Algorithms}\ar@//[d]^(0.3){\txt{$\rightarrow$Round-off}} \\
& & *+[F]{Promgramming}\ar@//[d] \\
& & *+[F]{Results}\ar@/^3.2pc/[luu]_{\txt{$\leftarrow$feedback}}
}


\documentclass{article}
\pagestyle{empty}
\textwidth155mm
\oddsidemargin2.1mm
\topmargin-8mm
\textheight225mm

\def\rgtbox#1#2{\phantom{#1}\hbox to0pt{\hss#2}}
\def\lftbox#1#2{\hbox to0pt{#2\hss}\phantom{#1}}

\begin{document}

\subsection*{The Apriori Algorithm for Finding Association Rules}

\vskip5mm
\begin{tabbing}
00 \= 00 \= 00 \= 00 \= \hskip60mm \= \kill
{\bf function} apriori $(I, T, s_{\min}, c_{\min}, k_{\max})$
   \>\>\>\>\> $(*$ apriori algorithm for association rules $*)$ \\
{\bf begin} \\
\> $\lftbox{C_k}{$k$} := 1$;
   \>\>\>\> $(*$ --- find frequent item sets $*)$ \\
\> $C_k := \bigcup_{i \in I} \{i\};$
   \>\>\>\> $(*$ start with single element sets $*)$ \\
\> $\lftbox{C_k}{$F_k$} := \mbox{prune}(C_k, T, s_{\min})$;
   \>\>\>\> $(*$ and determine the frequent ones $*)$ \\
\> {\bf while} $F_k \neq \emptyset$
   {\bf and}   $k \le k_{\max}$ {\bf do begin}
   \>\>\>\>   $(*$ while there are frequent item sets $*)$ \\
\> \> $C_{k+1} := \mbox{candidates}(F_k)$;
   \>\>\>     $(*$ create item sets with one item more $*)$ \\
\> \> $\lftbox{C_{k+1}}{$F_{k+1}$}
               := \mbox{prune}(C_{k+1}, T, s_{\min})$;
   \>\>\>     $(*$ and determine the frequent ones $*)$ \\
\> \> $\lftbox{C_{k+1}}{$k$} := k+1$;
   \>\>\>     $(*$ increment the item counter $*)$ \\
\> {\bf end}; \\
\> $R := \emptyset$;
   \>\>\>\>   $(*$ --- generate association rules $*)$ \\
\> {\bf forall} $f \in \bigcup_{j=2}^k F_j$ {\bf do begin}
   \>\>\>\>   $(*$ traverse the frequent item sets $*)$ \\
\> \> $\lftbox{H_m}{$m$} := 1$;
   \>\>\>     $(*$ start with rule heads (consequents) $*)$ \\
\> \> $H_m := \bigcup_{i \in f} \{i\}$;
   \>\>\>     $(*$ that contain only one item $*)$ \\
\> \> {\bf repeat}
   \>\>\>     $(*$ traverse rule heads of increasing size $*)$ \\
\> \> \> {\bf forall} $h \in H_m$ {\bf do}
   \>\>       $(*$ traverse the possible rule heads $*)$ \\
\> \> \> \> {\bf if} $\frac{s(f)}{s(f-h)} \ge c_{\min}$
   \>         $(*$ if the confidence of the rule $*)$ \\
\> \> \> \> {\bf then} $\lftbox{H_m}{$R$}
                       := R \cup \{ [(f-h) \to h] \}$;
   \>         $(*$ is high enough, add it to the result, $*)$ \\
\> \> \> \> \lftbox{\bf then}{\bf else} $H_m := H_m - \{h\}$;
   \>         $(*$ otherwise discard the rule head $*)$ \\
\> \> \> $H_{m+1} := \mbox{candidates}(H_m)$;
   \>\>       $(*$ create rule heads with one item more $*)$ \\
\> \> \> $\lftbox{H_{m+1}}{$m$} := m+1$;
   \>\>       $(*$ increment the head item counter $*)$ \\
\> \> {\bf until} $H_m = \emptyset$ {\bf or} $m \ge |f|$;
   \>\>\>     $(*$ until there are no more rule heads $*)$ \\
\> {\bf end};
   \>\>\>\>   $(*$ or the antecedent would become empty $*)$ \\
\> {\bf return} $R$;
   \>\>\>\>   $(*$ return the rules found $*)$ \\
{\bf end} $(*$ apriori $*)$ \\
\\
{\bf function} candidates $(F_k)$
   \>\>\>\>\> $(*$ generate candidates with $k+1$ items $*)$\\
{\bf begin} \\
\> $C := \emptyset$;
   \>\>\>\>   $(*$ initialize the set of candidates $*)$ \\
\> {\bf forall} $f_1, f_2 \in F_k$
   \>\>\>\>   $(*$ traverse all pairs of frequent item sets $*)$ \\
\> \lftbox{\bf forall}{\bf with} $f_1 = \{i_1,\ldots,i_{k-1},i_k\}$
   \>\>\>\>   $(*$ that differ only in one item and $*)$ \\
\> \lftbox{\bf forall}{\bf and}  $f_2 = \{i_1,\ldots,i_{k-1},i_k'\}$
   \>\>\>\>   $(*$ are in a lexicographic order $*)$ \\
\> \lftbox{\bf forall}{\bf and}  $i_k < i_k'$ {\bf do begin}
   \>\>\>\>   $(*$ (the order is arbitrary, but fixed) $*)$ \\
\> \> $f := f_1 \cup f_2 = \{i_1,\ldots,i_{k-1},i_k,i_k'\}$;
   \>\>\>     $(*$ the union of these sets has $k+1$ items $*)$ \\
\> \> {\bf if} $\;\forall i \in f:\; f -\{i\} \in F_k$
   \>\>\>     $(*$ only if all $k$ element subsets are frequent, $*)$ \\
\> \> {\bf then} $C := C \cup \{f\}$;
   \>\>\>     $(*$ add the new item set to the candidates $*)$ \\
\> {\bf end};
   \>\>\>\>   $(*$ (otherwise it cannot be frequent) $*)$ \\
\> {\bf return} $C$;
   \>\>\>\>   $(*$ return the generated candidates $*)$ \\
{\bf end} $(*$ candidates $*)$ \\
\\
{\bf function} prune $(C, T, s_{\min})$
   \>\>\>\>\> $(*$ prune infrequent candidates $*)$ \\
{\bf begin} \\
\> {\bf forall} $c \in C$ {\bf do}
   \>\>\>\>   $(*$ initialize the support counters $*)$ \\
\> \> $s(c) := 0$;
   \>\>\>     $(*$ of all candidates to be checked $*)$ \\
\> {\bf forall} $t \in T$ {\bf do}
   \>\>\>\>   $(*$ traverse the transactions $*)$ \\
\> \> {\bf forall} $c \in C$ {\bf do}
   \>\>\>     $(*$ traverse the candidates $*)$ \\
\> \> \> {\bf if} $c \in t$
   \>\>       $(*$ if the transaction contains the candidate, $*)$ \\
\> \> \> {\bf then} $s(c) := s(c) +1$;
   \>\>       $(*$ increment the support counter $*)$ \\
{\bf end} $(*$ prune $*)$
\end{tabbing}

\end{document}