\section{Related Work}
\label{sec:relatedwork}

In the past, many algorithms have been proposed to mine subgraphs from a
given database of graphs. These algorithms can be mainly divided in to
two classes depending on how the candidate patterns are generated.
Algorithms like those in \cite{IWM03,FSG01,HWP03} are Apriori
based methods, i.e., a candidate pattern of size $k+1$ is generated by
combining two frequent graphs of size $k$ that have a common $k-1$ sized
subgraph.  Algorithms like those in \cite{gSpan,borgelt2002mmf,HWP03},
on the other hand, belong to the class of pattern growth algorithms in
which a candidate pattern is generated by extending a frequent pattern
with an edge.

Mining subgraphs from a single graph is a related problem which is
surprisingly difficult compared to mining from a database of graphs.  In
\cite{kuramochi2005ffp}, they defined the support of a pattern
in a single graph as the maximum number of edge disjoint isomorphisms,
which is itself a $NP$-Hard problem.  In \cite{fiedler2007support}, they
proposed a definition of support based on overlapping ancestor
isomorphisms.  In \cite{2012-kais}, they proposed
CMDB-Miner\xspace to mine frequent patterns from a single large graph.
Support of a pattern is defined as the maximum flow in an appropriately
constructed flow network with capacities. This method estimates the
support of a pattern without enumerating its isomorphisms. The authors
also proposed methods to summarize the maximal frequent patterns
extracted from the graph. In \cite{li2010dessin}, they proposed an
algorithm to extract frequent patterns from dense graphs. It uses
GADDI index proposed in \cite{Gaddi2009} to efficiently extract the
isomorphisms of a subgraph.

There been little work in approximate subgraph
mining.  In \cite{gapprox}, they proposed $gApprox$ to mine approximate
frequent subgraphs.  The degree of approximation between a pattern and its
isomorphism includes label mismatches and missing edges. The search space
is explored in a depth first order and the support of a pattern is
computed by enumerating its isomorphisms. This approach is not feasible
for large graphs with label multiplicities as there are potentially
exponential number of isomorphisms \cite{2012-kais}.
In \cite{JiaZH11}, they proposed APGM to approximate frequent
subgraphs from a database of graphs. The method is similar to the
$gApprox$ method in that it stores the complete set of approximate
embeddings of the current frequent pattern. The difference, however, is
that in \cite{JiaZH11} the entire $1$-hop neighborhood of the current
embeddings is explored to enumerate all extensions of the frequent
pattern and their corresponding embeddings, whereas $gApprox$ enumerates
the embeddings for a single extension in each step.
In \cite{SpeedUpFAS}, the authors proposed strategies
to speed up the existing mining algorithms, by
limiting the number of candidates and also the number of duplicate
checks performed.
In \cite{RAM2008}, they proposed a randomized algorithm to mine
approximate patterns from a database graphs. In this method, an edge in
the approximate pattern is required to have at least a given number of
occurrences.

Graph querying is another problem that is related to
subgraph mining. The goal is to find matches of a given query graph in a
single graph or database of graphs.  In \cite{TALE}, they proposed an
indexing method to extract the approximate occurrences of a given graph
query in large graph databases.  The algorithm proposed in
\cite{yan.icde:2006} extracts selective fragments from the query graph
and queries against an index constructed from the fragments of the
database graphs. In \cite{RandomMatching}, they proposed a polynomial
time algorithm for detecting isomorphism between spectrally
distinguishable graphs. An isomorphism, if it exists,
is obtained by matching
the steady state vectors of Markov chains in both the graphs.  In
\cite{BerettiIndexing}, they proposed indexing and retrieval methods for
graph models. The problem with most of the indexing approaches is that
they are efficient in retrieving a single match for the query graph but
fail at retrieving all matches, and thus are not suited to mine frequent
patterns. Furthermore, they assume that the query given, and thus they
do not perform pattern enumeration as required in graph mining.

Frequent pattern mining algorithms usually return a large number of
patterns and interpreting them is a big challenge. This is especially
true if the output is presented to a human user for further analysis.
Sampling approaches like those proposed in
\cite{2009-graphsampling,2012-kais,RAM2008} mine a representative set of
maximal patterns from a database of graphs or a single graph.
In our work, we perform a random walk in the search space
to enumerate a maximal pattern.

%The concept of using derived labels based on the
%structure and the attributes is frequently used in detecting graph
%isomorphism \cite{zampelli} and computing graph kernels
%\cite{shervashidzeJmlr,shervashidzeNips}.
%Our methods to prune representative sets are somewhat similar to
%the wiesfieler lehman kernel to test isomorphism between two graphs
%$G_1$ and $G_2$ \cite{weisfeiler}, where after every iteration the labels are
%sorted and renamed with a different string in such a way that the pairwise
%relationship is maintained. $G_1$ is not isomorphic to $G_2$
%if the label set of the graphs differ. The specifics of how and what
%information we update is different, and also we use the information for
%subgraph isomorphism instead of graph isomorphism.


\documentclass[a4paper,twocolumn]{article}

\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{fullpage}
\usepackage[small]{titlesec}

% patch titlesec bug of not showing (sub)title numbering http://tex.stackexchange.com/a/300259
\usepackage{etoolbox}
\makeatletter
\patchcmd{\ttlh@hang}{\parindent\z@}{\parindent\z@\leavevmode}{}{}
\patchcmd{\ttlh@hang}{\noindent}{}{}{}
\makeatother

\newcommand*\mean[1]{\overline{#1}}

\title{The Cascade-Correlation Learning Architecture $-$ summary}
\author{Matěj Nikl}

\begin{document}
\maketitle
\noindent
The Cascade-Correlation Architecture (CCA) is a architecture as well as a supervised learning algorithm for artificial neural networks (ANNs). Conventional way of training ANNs is to \textit{somehow} choose a fixed network topology and then train it using back-propagation algorithm.
CCA eliminates the need of choosing a fixed network topology and provides a way of training this kind of a dynamic network.

\section{Principles of growing}
The network starts with no hidden units, only output units are present. Then, one hidden unit is added at a time, receiving all network inputs and all outputs of the previously added hidden units, each time adding more complexity to the network, possibly creating more and more complex feature detectors.

\begin{figure}[!h]
    % \centering
    \includegraphics[width=0.475\textwidth]{cascade.png}
    \caption{The CCA after two hidden units have been added. The vertical lines sum all incoming activations. Boxed connections are frozen, X connections are trained repeatedly.}
\end{figure}

\section{Training}
The process of training can be divided into two parts, which will be described in the following subsections, however one constraint holds through both of them: once a new hidden unit is added, its input weights are forever frozen.
% \begin{itemize}
%     \item \ref{ssec:nn} training the network output
%     \item \ref{ssec:unit} training a new hidden unit, to-be added to the network
% \end{itemize}
\subsection{Network output training} \label{ssec:nn}
Having everything except output units' weights frozen translates into optimization of a single layer network, which was one of the authors' goals (to avoid slow back-propagation). The following steps take place:
\begin{enumerate}
    \item optimize the single layer network, until a convergence is achieved
    \item evaluate the test error and if it is low enough, stop
    \item otherwise, proceed to subsection \ref{ssec:unit}
\end{enumerate}

\subsection{New unit training} \label{ssec:unit}
If network's test error is not low enough, we need to add a new hidden unit. To do so, the following steps take place:
\begin{enumerate}
    \item create a pool of \textit{candidate} units, having their input weights randomly initialized, each receiving all available inputs
    \item maximize $S$, the sum of covariances between the candidate unit's value and the residual output error of all output units, for each candidate unit, by adjusting their input weights, until convergence
\[
    S = \sum_{o \in O} \left| \sum_{p \in P} (V_p - \mean{V})(E_{p,o} - \mean{E_o}) \right|
\]
    \begin{itemize}
        \item $O$ is a set of output units
        \item $P$ is a set of training patterns (training data)
        \item $V_p$ is candidate unit's value for pattern $p$
        \item $E_{p,o}$ is the residual output error at output unit $o$ for pattern $p$
        \item averages are computed across $P$
    \end{itemize}

    \item choose the candidate unit with the largest $S$ and add it to the network, proceed to \ref{ssec:nn}
\end{enumerate}

\section{Principles of modularization}
The only sense of modularization in the CCA can be seen in viewing individual hidden units as modules, since they are being added throughout the learning process. However, the network as a whole is not modular in the full sense. Hidden units cannot be detached from a network and reattached to a different one. A hidden unit can only be used in conjunction with the hidden units it has been trained with and nowhere else.
\paragraph{Similar design}
A similar partially modular design can be seen in Progressive Neural Networks, where new columns (full-blown NNs) are being added with lateral connections to all of the previously trained columns, showing the same aspects of \textit{(non)modularity} as CCAs.

\end{document}


% !TEX root = ../00_thesis.tex

% ------------------------------------------------------------------------------
\section{Performance of the \TTW Scheduler}
\label{sec:ttw_evaluation_sched}
% ------------------------------------------------------------------------------

The following two sections present the performance evaluation of our \TTW implementation, presented in~\cref{sec:ttw_implementation}.
We first evaluate the performance of the scheduler. In particular, we illustrate the benefits of the minimal inheritance strategy presented in \cref{sec:multi_mode} and we show that the complexity of the schedule synthesis is tractable.

% ------------------------------------------------------------------------------
\subsection{Benefits of Minimal Inheritance}

\begin{figure}
  \centering
  \href{\ttwfig{Figure-11}}{%
  \includegraphics[scale=1]{inheritance_eval}}
  \caption{Number of rounds in the different modes' schedule, depending on the inheritance approach considered.
  \capt{We consider the number of rounds scheduled over 80\s, which is the least common multiple of the modes' hyperperiod.}}
  \label{fig:inheritance_eval}
\end{figure}

Every round introduces some overhead (mainly from  sending the beacon), which consumes energy.
To reduce the energy consumption, \TTW aims to minimize the number of rounds \objective{1}.
The schedule synthesis for a single mode is optimal in this respect; that is, the procedure guarantees that the schedule minimizes the number of communication rounds~(\cref{sec:single_mode}).

\squarepar{%}
  The second objective of the scheduler is to allow persistent applications to keep the same schedule in different operation modes \objective{2}.
  This creates additional constraints that break the optimality guarantee: in other words, the schedule of mode \modej, when constrained to be compatible with mode \modei, may contain more rounds than required to schedule the mode \modej alone.
  A naive solution to meet \objective{2} is to completely ``reserve the space'' of previously scheduled modes. This is equivalent to consider that all applications executing in mode \modei are also executing in \modej, even if it is not actually the case. We call this the \emph{full inheritance} approach.
  This approach does guarantee compatibility but it is very pessimistic: it leads to an excessive increase of the number of rounds and find problems to be non-schedulable when they may in fact be feasible.%
}

\begin{figure}
  \centering
  \includegraphics[scale=1]{modeGraph_eval}
  \caption{The mode graph \modeGraph' used in the inheritance evaluation scenario.
  Mode \modei has priority $i$. The applications executing in each mode are listed in~\cref{append:inheritance_eval}.}
  \label{fig:modeGraph_eval}
\end{figure}

In \cref{sec:multi_mode}, we derived the minimal set of constraints that are necessary to guarantee the compatibility between modes \objective{2}, which we refer to as \emph{minimal inheritance}.
We now illustrate with a simple example that the minimal inheritance does not overly increase the number of communication round required and performs much better than the full inheritance.
%
We consider the following configuration (fully detailed in \cref{append:inheritance_eval}): The system is composed of 13 nodes, running 15 different applications including 45 tasks and 30 messages.
The periods and deadlines vary between 10 and 80\s. The applications are executing in 5 different modes connected by the mode graph shown in~\cref{fig:modeGraph_eval}.
Finally, all applications are considered persistent.
We synthesize the schedule for the 5 modes while considering
(i)~no inheritance,%
%
\footnote{Considering no inheritance is equivalent to set all applications as non-persistent. In other words, there are no constraints between the different modes and the individual mode schedules are guaranteed to be optimal in terms of number of rounds~(\cref{sec:single_mode}).}
%
(ii)~our minimal inheritance approach, and (iii)~the naive full inheritance approach.
The results are shown in \cref{fig:inheritance_eval}.

One important observation is that the number of rounds steadily increases with the full inheritance approach, which is expected: the full inheritance assumes that all previously scheduled applications are still executing. Thus, the number of applications to schedule only increases, and so does the number of round required. Ultimately, this not only wastes energy, it also limits scalability with the number of operation modes.
In comparison, our minimal inheritance approach performs much better: Since only the required constraints are included, the minimal inheritance does not suffer from the scalability issue mentioned above. In this example, the minimal inheritance performs optimally (\ie it does not schedule more rounds that the minimum, captured by the ``no inheritance'' case); but note that this is {\bf not true in general}, it simply happens to be the case in this example.

\fakepar{Conclusion}
The minimal inheritance approach derived in \cref{sec:multi_mode} efficiently addresses the challenge of synthesizing compatible schedules~\objective{2} while minimizing the energy impact in terms of number of rounds scheduled~\objective{1}.
This approach does increase the complexity of the synthesis formulation; however, it is implemented in our \TTW scheduler, it induces no overhead for the user, and it does not affect the synthesis solving time, as discussed below.

% ------------------------------------------------------------------------------
\subsection{Offline Solving Time}

\squarepar{%}
  We computational complexity of the schedule synthesis is made tractable by \TTW's sequential approach: modes are scheduled individually, in order of priority~(\cref{sec:multi_mode}) and for each mode, the number of rounds to schedule is kept fixed then incremented until a solution is found~(\cref{sec:single_mode}).

  For the evaluation scenario described above, the solving time for one mode grows up to ten minutes~(\cref{table:solvingTimes}) and is generally correlated with the complexity of the mode to schedule: mode \mode{3} and \mode{4} contains the most applications~(\cref{append:inheritance_eval}), leading to more constraints in the formulation.
  Furthermore, we note that the minimal inheritance strategy does not increase the overall solving time compared to ``no inheritance''. The intuition is that, by reserving some applications' schedule, we fix the value of some of the problem variables, thereby reducing the complexity of the problem.
  However, as shown by the full inheritance approach, if too many variables are fixed, the resulting problem might become harder to solve: more communication rounds become required, which increases the number of variables and thus the complexity.

  \fakepar{Conclusion}
  The evaluation scenario is simple but representative of a middle-sized \CPS. Our evaluation shows that the computational complexity may grow to the scale of minutes for challenging modes, which remains perfectly tractable for a task that needs to be performed only once and before deployment.%
}

\begin{table}
  \centering
  \caption{Approximate solving time for the different modes of the inheritance evaluation~(\cref{sec:ttw_evaluation_sched}).
  \capt{Time expressed in seconds; all computation performed on a commodity laptop.}}
  \label{table:solvingTimes}
  {\smaller \input{\TablePath/solvingTimes.csv}}
  \vspace{-5pt}
\end{table}

\vspace{-10pt}
% -----------------------------------------------------------------------------
\section{Performance of \TTnet}
\label{sec:ttw_evaluation_implem}
% ------------------------------------------------------------------------------

After the evaluation of the \TTW scheduler~(\cref{sec:ttw_evaluation_sched}), we now consider the performance of our \TTnet implementation, described in \cref{subsec:implem_ttnet}.

% ------------------------------------------------------------------------------
\subsection{Memory Utilization}

First, we consider the memory utilization induced by storing the scheduling tables in the nodes' memory.
For a given operation mode, the entire schedule contains the task offsets, the message offsets and deadlines, the round starting times, and the allocations of messages to rounds~(\cref{table:ttw_inputs_outputs}).
In addition, nodes must know the task periods and the mode hyperperiod to compute the absolute start time of the tasks and rounds.

Since we dedicate the execution of tasks and the wireless communication to different processors~(\cref{sec:ttw_implementation}), the memory cost for storing the schedule can be splitted.
On the application side, we must store the task offsets and periods, which are required to know when to execute the tasks; \ie 2 variables per task.
On the communication side, we must store the mode hyperperiod and the rounds information, \ie the offset and allocation of the rounds scheduled within the mode's hyperperiod; \ie $(\nslotsmax+1)$ variables per round.
The message deadlines are not required at runtime and do not need being stored.
As a result, we can generally estimate that the scheduling tables represent tens to hundreds of variables per mode for each processor.

\squarepar{%}
  \fakepar{Conclusion}
  Our application and communication processors feature 64\kB~\cite{msp432} and 4\kB\cite{CC430F6137} of RAM, respectively. Thus, considering an average size of two\bytes per variable, storing the scheduling tables represent a significant overhead and limits the scalability of the system, in particular on the communication processor.
  This limitation would be significantly relaxed with newer platforms, which commonly feature 256\kB of RAM~\cite{nRF52840}.%
}

% ------------------------------------------------------------------------------
\subsection{\TTnet Model}
\label{subsec:model}

\squarepar{%}
  As discussed in \cref{sec:ttw_implementation}, we implement \TTnet using \baloo, which allows to derive a precise model of
  (i)~the execution time of a communication round and
  (ii)~the time spent with the radio turned on, which correlates with the energy consumed for communication.
  Estimating the communication time is necessary to synthesize the schedules since the scheduler must know how long the rounds last.
  This model should be as tight as possible not to ``waste'' time and thus minimize the end-to-end deadlines schedulable by \TTW, but it must be a safe upper-bound in order to prevent deadline misses.
  This section presents our \TTnet model and derives the theoretically achievable performance in terms of minimal message latency and the energy savings expected from using rounds.%
}

Let $\app.\delta$ denote the latency of an application \app. This latency represents the delay for a complete execution of \app; that is, the completion of all tasks in \app.\predG. Let $\app.c$ be a \emph{chain} in \app.\predG.
A chain is defined as a path of \app.\predG starting with a task without predecessor and ending with a task without successor.%
%
\footnote{For example, $(\tau_2, m_2, \tau_4)$ is a chain of \predG in \cref{fig:precedence_graph}.}
%
The minimum achievable latency for a single message in \TTW is the length of a round composed of only one slot, denoted $\Tround(L,1)$ where $L$ is the payload size. Thus $\app.\delta$ is lower-bounded by
\begin{align}
\label{eq:min_deadline}
\app.\delta \;
	& \geq \; \max_{\app.c \,\in\, \app.\predG}
		\left(
			\; \sum_{\tau \, \in \, \app.c} \tau.e \,+\, \sum_{m \, \in \, \app.c} \Tround(L,1) \;
		\right)
\end{align}

\begin{remark}
  By comparison, the best possible guarantee for the latency of a single message provided by  \DRP~(\cref{ch:drp}) is of the order of $2*\Tround(L,\nslotsmax)$.
  Since $\Tround(L,\nslotsmax) \approx \nslotsmax*\Tround(L,1)$, \TTW reduces the minimal guarantee on message latency by a factor of approximately $2*\nslotsmax$.
  For a relatively small number of slots per round, such as $\nslotsmax=5$, this represents an order of magnitude improvement.
  This difference stems from the loose coupling between the task and message schedules in \DRP, whereas \TTW statically schedules all tasks and messages.
\end{remark}

\afterpage{
\begin{figure}
\centering
\includegraphics[scale=1]{Tslot}
\caption{%
Break-down of a communication round.
\capt{%
At the slot level, the colored boxes identify phases where the radio is on.
In the ``idle'' phase, the radio is turned off in practice, but this idle time depends on each node's distance to the initiator.
To estimate the energy saving of rounds (\cref{fig:energy_ratio}), we assume that the radio stays on for the whole time of \Tglossy, as specified in~\cref{eq:Ton}.
}
}
\label{fig:Tslot}
\end{figure}}

A round \Tround is composed of up to $(\nslotsmax+1)$ slots in which Glossy floods~\cite{ferrari2011Glossy} are executed.
An entire slot completes in time $\Tslot$, decomposed into
\begin{equation}
  \Tslot = \Twakeup + \Tstart + \Tglossy + \Tgap
\end{equation}
The composition of a slot in our implementation is detailed in~\cref{fig:Tslot}.
First, all nodes wake up (\Twakeup) and switch on their radio (\Tstart).
Then the message flood starts. We denote by \Thop the time required for one protocol step, \ie a one-hop transmission. The total length of the flood is
%
\begin{align}
\Tglossy = (H+2N-1)*\Thop
\end{align}
with $H$ the network diameter and $N$ the number of times each node transmits each packet.%
%
\footnote{Glossy achieves more than 99.9\% packet reception rate using $N =2$~\cite{ferrari2011Glossy}.}
%
\Thop is itself divided into
%
\begin{align}
\Thop = \Td + \Tcal + \Theader + \Tpayload
\end{align}
where \Td is a radio delay, and \Tcal, \Theader and \Tpayload are the transmission times of the clock calibration message, the Glossy header and the message payload, respectively.
With a bit rate of \Rbit, the transmission of $L$\bytes takes
%
\begin{align}
T(L) = 8L/\Rbit
\end{align}
Once the flood is completed, some gap time \Tgap is necessary to process the received packet.
This time is used (among other things) to execute \baloo's \texttt{on\_slot\_post()} callback, where the received messages are written into \bolt.
We divide \Tslot into \Ton and \Toff, which denote the time spent with radio on and off, respectively.
%
\begin{align}
\Toff &\,=\,
	\Twakeup + \Tgap \\
\nonumber
\Ton(L) &\,=\,
	\Tstart + \\
\label{eq:Ton}
  & \qquad (H+2N-1) * \left( \Td + 8(\Lcal + \Lheader + L)/\Rbit \right) \\
\Tslot(L) &\,=\,
  \Toff + \Ton(L) \\
\Tround(L)
	&\,=\,
	\Tslot(\Lbeacon) + B*\Tslot(L) + \Tpreprocess
\end{align}

Sending beacons is necessary to let the nodes know about the current state of the system; \ie which mode is executing and ``how far'' is the system in the scheduling table. Without that information, it is impossible for a failing node to recover and resume its normal operation.
Moreover, beacons prevent message collisions by guaranteeing that the nodes always know the system's state when a round starts.
In a design \emph{without} round, each message transmission should be preceded by its own beacon to provide the same guarantees. Thus, the transmission time for \nslots messages of size $L$, denoted $\Tworound(L)$, would take
%
\begin{align}
\Tworound(L) =  B*( \, \Tslot(\Lbeacon) + \Tslot(L) \, )
\end{align}
%
We can then derive the relative energy savings $E$ granted by using a round-based design, which we compute as $E= (\Tworoundon - \Troundon)/\Tworoundon$.

\afterpage{
\begin{figure}
  \begin{subfigure}{\linewidth}
    \centering
    \href{\ttwfig{Figure-14}}{%
    \includegraphics[scale=1]{T=f(H,B)}}
  \end{subfigure}
  \begin{subfigure}{\linewidth}
    \centering
    \href{\ttwfig{Figure-14}}{%
    \includegraphics[scale=1]{Overhead=f(H,B)}}
  \end{subfigure}
  \caption{Example values of round length (top) and protocol overhead (bottom) computed using the \TTnet model~(\cref{subsec:model}).
  The protocol overhead is computed as the percentage of time spent to send the beacons relative to the overall communication time for a round containing \nslots slots.
  Payload is set to 16\bytes and we use $N = $2 transmissions in the Glossy floods~\cite{ferrari2011Glossy}.}
  \label{fig:TTWmodel}
\end{figure}
}

The complete \TTnet model is available in Appendix~(\cref{appendix:ttw_artifacts}). We use this model to compute the round length \Tround and the energy savings $E$ for different values of number of slots per rounds (\nslots), message payload size ($L$), network diameter $H$, and number of transmissions in Glossy floods ($N$).
Selected results are shown in \cref{fig:TTWmodel,fig:energy_ratio}.
\linebreak
For example, with $N$ set to 2, it takes less than 100\ms to complete a 10-slot round sending 16-bytes messages over a 4-hop network~(\cref{fig:TTWmodel}, top).

% ------------------------------------------------------------------------------
\subsection{Model Validation}

\squarepar{%}
  We now evaluate the runtime execution of our implementation and aim to validate our \TTnet model.
  In particular, it is important that the round length model gives safe upper-bounds since the \TTW scheduler relies on the model to schedule messages and tasks: if a round overruns, this may delay the execution of subsequent tasks and cause deadline misses.
  We test our \TTnet implementation for different number of slots per round \nslots and payload size $L$, we measure the round length and radio-on time experienced by the different nodes in the network, and we compare the results with the \TTnet model.%
}

\fakepar{Evaluation scenario}
We program the network to execute, one round with \nslots slots, followed by \nslots rounds with one slot.
For each of these rounds, we collect the round length and the radio-on time. Both values are measured in software (\ie the measurement is implemented in the firmware) and use a 32\kHz timer, leading to a measurement accuracy of about 30\us.

\begin{table}
  \centering
  \caption{\triscale parameters for the experimental validation of \TTnet's model}
  \label{table:ttw_triscale_param}
  {\smaller \input{\TablePath/triscale_param.csv}}
\end{table}

\fakepar{Experiment design}
We design the evaluation using the \triscale framework (introduced in \cref{ch:triscale}). The evaluation parameters are listed in \cref{table:ttw_triscale_param}.
Our evaluation scenario is terminating (there is a finite task to accomplish); thus there is no need to test for convergence.
The round length evaluation aims to validate that the \TTnet model is a safe upper-bound; thus, we use the maximum measured round length across all nodes as metric for a run.
For the same reason, we choose a large KPI (95th percentile) and a high confidence level (95\%), which leads to a minimal number of 59 runs per series.
To investigate the reproducibility of the results, we choose the median and a 75\% level of confidence for the variability scores, leading to a minimal number or 3 series.
To evaluate the average savings provided by using communication rounds, we use the median values across nodes as metric.
We perform the evaluation on FlockLab~\cite{FlockLab}, an indoor testbed located in an office building. It has been shown that the experimental conditions on FlockLab exhibits weekly seasonal components~(\cref{subsec:network_profiling}); therefore, to avoid biasing our evaluation, we perform our series of runs using a span of one week, during which we schedule randomly 60 runs per set of parameters. We test our \TTnet implementation using 5, 10, and 30 slots per round, and payloads of 8, 16, and 64\bytes.
The three series of tests were performed between May and October 2019.
The KPI values from our evaluation are listed in~\cref{table:KPIs}.

\afterpage{
\begin{figure}
  \centering
  \href{\ttwfig{Figure-15}}{%
  \includegraphics[scale=1]{energy_savings}}
  \caption{Relative radio-on time savings by using rounds compared to single messages.
  \capt{The energy savings induced by a round-based design grow with the number of number of slots per round (X-axis). Conversely, these savings become less significant as the payload size increases (lighter colors).
  The diamonds show our evaluation KPIs and thus estimate, with a probability of 95\%, the average energy savings expected in 95\% of the test runs. $H = 4$, $N = 2$.}}
  \label{fig:energy_ratio}
\end{figure}}

\begin{remark}
  \squarepar{%}
  Observe that certain values in~\cref{table:KPIs} are reported with a {\ssymbol{1}} or {\ssymbol{2}} symbol.
  The {\ssymbol{1}} marks series where \triscale independence test fails; this indicates that the metric data do not appear to be \iid and therefore the KPI value loose its predictive power (\ie it does not allow to infer what is the expected performance).
  However, in our evaluation, the autocorrelation plots show no significant differences between the series that passes the test and those that do not~(data available in~\cref{appendix:ttw_artifacts}).
  Moreover, the round length KPI values are almost the same in all series.
  Together, these two facts increase our confidence in the results and suggests that the reported KPIs are robust estimates of the expected performance.\\
  The {\ssymbol{2}} marks series where we could not collect enough data in order to compute the KPIs. This appended in Series 3 due to construction work taking place in the FlockLab building, where many test runs were lost due to sporadic power outages.
  In both cases, the table shows the maximum round length or minimum energy savings metric values obtained across all the runs in the series.%
  }
\end{remark}

\fakepar{Results -- Round length}
The results for the round length are extremely stables~(\cref{table:KPIs}):
the differences of KPI values between series are at most one time tick ($\approx$ 30\us), which is our measurement accuracy.
Concretely, this means that, in all series, the largest round length measured by any node is essentially the same.

Furthermore, the KPI values are (i)~very close to and (ii)~consistently lower than the model. By definitions of the KPI, we can estimate with 95\% probability that (at least) 95\% of runs will yield a maximal round length smaller than the KPI value, and thus smaller than the model value.
\cref{fig:exampleSeries} (top) shows the distribution of the round length measurements from all the nodes collected during one series of 60 runs. We observe that the distribution is narrow (less than 300\us of spread), which is expected. Indeed, the \TTnet rounds are fully time-triggered; thus, the measurement differences between nodes mainly come from the difference in execution time of \baloo's end-of-round operations, which is expected to be small.

\squarepar{%}
  In our entire evaluation, there was one case where a node reported a value (77.85\ms) larger than the model (77.52\ms). This concerned only one node: in this run%
  \footnote{FlockLab test number 66992; data available in~\cref{appendix:ttw_artifacts}.}
  the second highest value reported (77.12\ms) was smaller than the model value.
  It is hard to know a posteriori what may have cause this.
  However, we argue that this one overshoot is more likely imputable to some sporadic hardware delay than due to a miscalibration of the model.%
}

\afterpage{
\begin{figure}
  \centering
  \begin{subfigure}{\linewidth}
    \centering
    \href{\ttwfig{Figure-16}}{%
    \includegraphics[scale=1]{serie2_T_round_H4_N2_L16_B5}}
  \end{subfigure}
  \begin{subfigure}{\linewidth}
      \centering
      \href{\ttwfig{Figure-16}}{%
      \includegraphics[scale=1]{serie2_T_on_round_H4_N2_L16_B5}}
  \end{subfigure}
  \caption{Distributions of round length (top) and radio-on time (bottom) measurements from all the nodes, collected in one series of 60 runs (Serie 2, $L=16$, $\nslots=5$).
  \capt{
    While the distribution of round length is very narrow, the radio-on time exhibits a much larger spread. We can see that the \TTnet model provides a generally overestimate the radio-on time, which is expected since it assumes that nodes keep their radio on for the entirety of \Tflood~\textup{(}\cref{eq:Ton}\textup{)}, which is not the case in practice: nodes turn the radio off when they have transmitted a packet $N$ times.
  }}
  \label{fig:exampleSeries}
\end{figure}}

\fakepar{Results -- Energy savings}
The energy savings results show more fluctuations than the round length, which is not surprising: (i)~the energy model is less precise and (ii)~the dynamic interference conditions affect the radio-on time, as nodes may need to keep their radio on for a longer share of \Tglossy.
\cref{fig:energy_ratio} shows the model and our energy savings KPIs together.
\cref{fig:exampleSeries} (bottom) shows the distribution of radio-on time measurements from all the nodes, collected in one series of 60 runs: nodes experience significant differences in radio-on time during a round. This is expected since nodes terminate a flood as soon as they have transmitter a packet $N$ times, which happens earlier for nodes that are closer to the initiator,

Overall, the energy savings come from the ``distribution'' of the overhead from sending beacons between the slots. Thus, the more slots (increasing \nslots) and the smaller the slots (decreasing $L$), the more radio-on time is spared by using rounds.
For a payload of 16\bytes, we obtain an average energy savings of about 30\% with only 10 slots per round.

\afterpage{
\begin{landscape}
\begin{table}
  \centering
  \caption{
  KPIs from the performance evaluation described in~\cref{sec:ttw_evaluation_implem} and corresponding model values for the \TTnet round length \Tround and energy savings $E$; other settings: $H=4$ and $N=2$.
  \capt{
    The value marked in bold corresponds to the one case where the round length KPI is larger than the model value.
    {\ssymbol{1}} marks series where \triscale independence test failed.
    {\ssymbol{2}} marks series without enough samples for computing the KPIs.
    In these two cases, reported values are the maximum round length or minimum energy savings metric values for all the runs in the series.
  }}
  \label{table:KPIs}
  {\smaller \input{\TablePath/KPIs.csv}}
\end{table}
\end{landscape}
}

\fakepar{Conclusion}
We validated the tightness and safeness of \TTnet round length model, which was found to be an upper-bound of the effective round length for all but one in about 14k measurements collected.
Furthermore, we showcased that, even with small beacons (2\bytes in our implementation), a round-based design yields significant reduction of radio-on time, and therefore helps minimizing the overall energy consumption~(\feature{Efficiency}).


\section{Related Work}
\label{sec:related-work}
% different classes of CHC solvers:
A significant body of previous work has presented solvers for
different classes of Constrained Horn Clauses, or finding inductive
invariants of programs that correspond to solutions of CHCs.
% solving linear systems:
\impact attempts to verify a given sequential procedure by iteratively
selecting paths and synthesizing invariants for each path. This
approach corresponds to solving a recursive linear CHC
system~\cite{mcmillan06}.

% interprocedural verification:
Previous work also proposed a verifier for recursive
programs~\cite{heizmann10}.
%
The proposed approach selects interprocedural paths of a program and
synthesizes invariants for each as nested interpolants.
%
Such an approach corresponds to attempting to solve a recursive CHC
system $S$ by selecting derivation trees of $S$
and solving each tree.

% solving recursive systems:
Previous work has proposed solvers for recursive systems that, given a
system $S$, attempt to solve $S$ by generating and
solving a series of recursion-free unwindings of $S$.
%
In particular, \eldarica attempts to solve each unwinding
$S'$ by reducing to and solving body-disjoint systems~\cite{rummer13a,rummer13b}.
%
\duality attempts to avoid solving all derivation-trees (i.e
body-disjoint systems) by using lazy annotation~\cite{bjorner13}.
%
Other optimizations select derivation trees to solve using
symbolic analogs of Prolog evaluation with
tabling~\cite{jaffar09,mcmillan14}.
%

\whale attempts to verify sequential recursive programs by generating
and solving hierarchical programs, which correspond to recursion-free CHC
systems~\cite{albarghouthi12b}.
%
To solve a particular recursion-free system, \whale solves a linear
inlining of the input using a procedure named
\vinta~\cite{albarghouthi12a}.
%
In general, the linear inlining may be exponentially larger than the
input.

\sys is similar to the recursion-free CHC approaches given above in
that it reduces the problem to solving a CHC system in a
directly-solvable class.
%
\sys is distinct in that it reduces to solving Clause-Dependent
Disjoint (CDD) systems.
%
As discussed in \autoref{sec:overview}, the class of CDD systems is a
superset of classes used by the approaches above. CDD systems can
also be solved directly.

\sys solves general CHC systems using the same strategy as proposed by
the above approaches.
%
Specifically, it solves a series of recursion-free unwindings of the
original system, and tries to synthesize a general solution from the
recursion-free solutions.

Previous work describes solvers for non-linear Horn clauses over
particular theories.
%
In particular, verifiers have been proposed for recursion-free systems
over the theory of linear arithmetic~\cite{komuravelli14}.
%
Because the verifier relies on quantifier elimination, it is not clear
if it can be extended to richer theories that support interpolation,
such as the combination of linear arithmetic with uninterpreted
functions.
%
Other work describes a solver for the class of \emph{timed pushdown
systems}, a subclass of CHC systems over the theory of linear real
arithmetic~\cite{hoder12}.
%
Unlike these approaches, \sys can solve systems over any theory that
supports interpolation.

% DAG inlining:
DAG inlining attempts to generate compact verification conditions for
hierarchical programs~\cite{lal-qadeer15}.
%
\sys attempts to solve recursion-free CHC systems by reducing them to
compact CDD systems.
%
Because hierarchical programs and recursion-free CHC systems are
closely related, algorithms that operate on hierarchical programs
correspond to algorithms that operate on recursion-free Horn Clauses.
%
However, it is not apparent whether such algorithms can be used directly
to synthesize solutions.


\chapter{Introduction}

The goal of this Capita Selecta is to investigate the possibilities for a fast MATLAB based implementation of the Block MLDU algorithm.\\

\noindent High performance solvers are usually written in languages like C/C++ because they are compiled, avoiding the sometimes costly step of the interpreter. Languages like MATLAB are dynamically typed and interpreter based, which make them great for prototyping purposes.\\

\noindent The Block MLDU algorithm shows great possibilities, but the lack of a high performance implementation makes it prohibitive to use for large matrices. This is a problem because research projects sometimes encounter very large matrices that need to be solved with the Block MLDU algorithm.\\

\noindent This project tries to alleviate the problem by investigating the performance bottlenecks of a MATLAB based implementation and provide workarounds for them. This should result in a much faster implementation without requiring the extra work associated with a full C/C++ version.\\

\noindent The biggest advantage of this workflow is that the time required to implement a workaround for a bottleneck will be much lower in MATLAB than in C/C++, making it easier to try different approaches to the problems. The lessons learned at this stage will still be useful when the code does finally get ported to C/C++.\\

\noindent The biggest disadvantage to sticking to MATLAB is that the interpreter will always provide some overhead compared to C/C++. The hope is that this overhead will be small compared to the other bottlenecks encountered.

\newpage

\section{The Block MLDU algorithm}

The Block MLDU algorithm is very comparable to the well known LU decomposition. The main difference between the Block MLDU algorithm and LU decomposition is the "block" nature of the splitting.

\subsection{Example}

A simple example will be provided to highlight the differences between LU and Block MLDU. Let matrix $A$ be $4 \times 4$:\\

$
\hspace{20mm} A =
\left\lbrack\begin{array}{rrrr}
           1&           0&           1&           0\\
           2&           1&          -1&           0\\
           0&          -1&           0&           1\\
          -2&           0&           1&           3
\end{array}\right\rbrack
$
$
\hspace{30mm} s =
\left\lbrack\begin{array}{rrr}
           2&           1&           1
\end{array}\right\rbrack
$\\

\noindent As stated by row $s$, matrix $A$ will first be split using a $2 \times 2$ block, followed by two $1 \times 1$ blocks.\\

\noindent \textbf{Step one:}\\

$
\left\lbrack\begin{array}{rr|rr}
          D1&          D1&          U1&          U1\\
          D1&          D1&          U1&          U1\\ \hline
          L1&          L1&          M1&          M1\\
          L1&          L1&          M1&          M1
\end{array}\right\rbrack
$
$
\hspace{10mm} L1 =
\left\lbrack\begin{array}{rr}
           0&          -1\\
          -2&           0
\end{array}\right\rbrack
$
$
\hspace{6mm} D1 =
\left\lbrack\begin{array}{rr}
           1&           0\\
           2&           1
\end{array}\right\rbrack
$
$
\hspace{6mm} U1 =
\left\lbrack\begin{array}{rr}
           1&           0\\
          -1&           0
\end{array}\right\rbrack
$\\

$
Schur1 = L1 D1^{-1} U1 =
$
$
\left\lbrack\begin{array}{rr}
           3&           0\\
          -2&           0
\end{array}\right\rbrack
$
$
\hspace{20mm} A - Schur1 =
\left\lbrack\begin{array}{rrrr}
           1&           0&           1&           0\\
           2&           1&          -1&           0\\
           0&          -1&  \textbf{-3}&  \textbf{1}\\
          -2&           0&   \textbf{3}&  \textbf{3}
\end{array}\right\rbrack
$\\

\noindent \textbf{Step two:}\\

$
\left\lbrack\begin{array}{rr|r|r}
          D1&          D1&          U1&          U1\\
          D1&          D1&          U1&          U1\\ \hline
          L1&          L1&          D2&          U2\\ \hline
          L1&          L1&          L2&          M2
\end{array}\right\rbrack
$
$
\hspace{10mm} L2 =
\left\lbrack\begin{array}{r}
           3
\end{array}\right\rbrack
$
$
\hspace{18mm} D2 =
\left\lbrack\begin{array}{r}
           -3
\end{array}\right\rbrack
$
$
\hspace{10mm} U2 =
\left\lbrack\begin{array}{r}
           1
\end{array}\right\rbrack
$\\

$
Schur2 = L2 D2^{-1} U2 =
$
$
\left\lbrack\begin{array}{r}
           -1
\end{array}\right\rbrack
$
$
\hspace{12mm} A - Schur1 - Schur2 =
\left\lbrack\begin{array}{rrrr}
           1&           0&           1&           0\\
           2&           1&          -1&           0\\
           0&          -1&          -3&           1\\
          -2&           0&           3&   \textbf{4}
\end{array}\right\rbrack
$\\

\noindent \textbf{Step three:}\\

$
\left\lbrack\begin{array}{rr|r|r}
          D1&          D1&          U1&          U1\\
          D1&          D1&          U1&          U1\\ \hline
          L1&          L1&          D2&          U2\\ \hline
          L1&          L1&          L2&          D3
\end{array}\right\rbrack
\\$\\

\noindent \textbf{Result:}\\

$
L =
\left\lbrack\begin{array}{rrrr}
           0&           0&           0&           0\\
           0&           0&           0&           0\\
   \textbf{0}& \textbf{-1}&          0&           0\\
  \textbf{-2}&  \textbf{0}&  \textbf{3}&          0
\end{array}\right\rbrack
$
$
\hspace{10mm} D =
\left\lbrack\begin{array}{rrrr}
   \textbf{1}&  \textbf{0}&          0&           0\\
   \textbf{2}&  \textbf{1}&          0&           0\\
           0&           0&  \textbf{-3}&          0\\
           0&           0&           0&   \textbf{4}
\end{array}\right\rbrack
$
$
\hspace{10mm} U =
\left\lbrack\begin{array}{rrrr}
           0&           0&   \textbf{1}&  \textbf{0}\\
           0&           0&  \textbf{-1}&  \textbf{0}\\
           0&           0&           0&   \textbf{1}\\
           0&           0&           0&           0
\end{array}\right\rbrack
\\$\\

$
\\
A = (L + D)D^{-1}(D + U)
$


\chapter{Conclusion}\label{chap:conclusion}

We looked at how the give token and random token orders solved the two main problems: Deadlocks and infinite executions. We discussed that there are three different ways to introduce a token, two of them solve both problems. The table token does not solve the deadlock problem because there is the possibility that no agent takes the token off the table. This is illustrated in the following table:

\begin{table}[h!]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{|l|l|l|l|}
\hline
 & table token & random token & give token \\ \hline
empower token & \begin{tabular}[c]{@{}l@{}}solves infinite executions \\ $\longrightarrow$ Proposition 1 \\ does not solve deadlocks\end{tabular} & \begin{tabular}[c]{@{}l@{}}solves infinite executions\\ $\longrightarrow$ Proposition 1 \\ solves deadlocks \\ $\longrightarrow$ Proposition 4 \end{tabular} & \begin{tabular}[c]{@{}l@{}}solves infinite executions\\ $\longrightarrow$ Proposition 1 \\ solves deadlocks \\ $\longrightarrow$ Proposition 4 \end{tabular} \\ \hline
force action token & \begin{tabular}[c]{@{}l@{}}solves infinite executions \\ $\longrightarrow$ Proposition 2 \\ does not solve deadlocks\end{tabular} & \begin{tabular}[c]{@{}l@{}}solves infinite executions\\ $\longrightarrow$ Proposition 2 \\ solves deadlocks \\ $\longrightarrow$ Proposition 3  \end{tabular} & \begin{tabular}[c]{@{}l@{}}solves infinite executions\\ $\longrightarrow$ Proposition 2 \\ solves deadlocks \\ $\longrightarrow$ Proposition 3 \end{tabular}  \\ \hline
\end{tabular}%
}
\end{table}

This means that there are positive results with the introduction of a token based order. In every case, if an agent that has found a plan gets the token, the goal will be reached in a finite number of steps.

For future work, the way the token changes the search of a plan should be researched. With tokens, the search tree of a plan might have a smaller width because the agents can be prevented from acting. The tree will also have a deeper depth because the agents each have a new action of handing the token to the next player.\\
We also did not look at how a tokenized planning task changes the existence of i-strong policies. It is unclear if some planning problems become unsolvable through the tokenize function.

There could also be future research in researching the fairness of the tokens.\\
Another field of research for the future could be impatient players and the table token execution order. An impatient player will wait for the other player to take the token first, but only for a limited amount of time. For this we would need to extend the formalism since we cannot model time or a waiting agent explicitly.


% BEGIN LICENSE BLOCK
% Version: CMPL 1.1
%
% The contents of this file are subject to the Cisco-style Mozilla Public
% License Version 1.1 (the "License"); you may not use this file except
% in compliance with the License.  You may obtain a copy of the License
% at www.eclipse-clp.org/license.
%
% Software distributed under the License is distributed on an "AS IS"
% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied.  See
% the License for the specific language governing rights and limitations
% under the License.
%
% The Original Code is  The ECLiPSe Constraint Logic Programming System.
% The Initial Developer of the Original Code is  Cisco Systems, Inc.
% Portions created by the Initial Developer are
% Copyright (C) 2006 Cisco Systems, Inc.  All Rights Reserved.
%
% Contributor(s):
%
% END LICENSE BLOCK

\chapter{Getting started with Finite Domains}

%----------------------------------------------------------------------
\section{Using the Finite Domains Library}
To use the Finite Domains Library, load the library using either of
\begin{quote}\begin{verbatim}
:- lib(fd).
:- use_module(library(fd)).
\end{verbatim}\end{quote}
Specify this at the beginning of your program.

%----------------------------------------------------------------------
\section{Structure of a Constraint Program}
The typical top-level structure of a constraint program is
\begin{quote}\begin{verbatim}
solve(Variables) :-
        read_data(Data),
        setup_constraints(Data, Variables),
        labeling(Variables).
\end{verbatim}\end{quote}
where setup_constraints/2 contains the problem model. It creates the
variables and the constraints over the variables.
This is often deterministic, but not necessarily.
The labeling/1 procedure is the search part of the program. It
tries to find solutions by trying all instantiations for the
variables. This search is being pruned by constraint propagation.

The above program will find all solutions.
If the best solution is wanted, we can just wrap a branch-and-bound
procedure around the search component of the program:
\begin{quote}\begin{verbatim}
solve(Variables) :-
        read_data(Data),
        setup_constraints(Data, Variables),
        min_max(labeling(Variables), Objective).
\end{verbatim}\end{quote}

%----------------------------------------------------------------------
\section{Modelling}
The modelling code need to do the following:
    \begin{itemize}
    \item Create the variables with their initial domains
    \item Setup the constraints between the variables
    \end{itemize}
%\htmladdnormallink{Example: Send more money}{../../examples/sendmore.pl.txt}

A simple example is the ``cryptarithmetic'' puzzle,
\verb0SEND+MORE = MONEY0.
The idea is to associate a digit (0-9) with each letter so the
equation is true.

The { \eclipse } code is as follows:
\begin{quote}\begin{verbatim}

:- lib(fd).

sendmore1(Digits) :-
    Digits = [S,E,N,D,M,O,R,Y],

% Assign a finite domain with each letter - S, E, N, D, M, O, R, Y -
% in the list Digits
    Digits :: [0..9],

% Constraints
    alldifferent(Digits),
    S #\= 0,
    M #\= 0,
                 1000*S + 100*E + 10*N + D
               + 1000*M + 100*O + 10*R + E
    #= 10000*M + 1000*O + 100*N + 10*E + Y,

% Search
    labeling(Digits).
\end{verbatim}\end{quote}

%----------------------------------------------------------------------
\section{Simple User-defined Constraints}
Conceptual constraints that are just conjunctions of primitive
constraints can easily be defined.
%\htmladdnormallink{Example: Zebra}{../../examples/zebra.pl.txt}

For example, let us assume that we have a set of colours and we want
to define that some colours fit with each other and others do
not. This should work in such a
way as to propagate possible changes in the domains as soon as this
becomes possible.

Let us assume we have a symmetric relation that defines which colours
fit with each other:

\begin{quote}\begin{verbatim}
% The basic relation
fit(yellow, blue).
fit(yellow, red).
fit(blue, yellow).
fit(red, yellow).
fit(green, orange).
fit(orange, green).
\end{verbatim}\end{quote}

The predicate \verb0nice_pair(X, Y)0 is a constraint and any change of the
possible values of $X$ or $Y$ is propagated to the other variable. There
are many ways in which this pairing
can be defined in ECLiPSe. They are different solutions with different
properties, but they yield the same results.

\subsection{Using Evaluation Constraints}

We can also encode directly the relations between elements in the
domains of the two variables:
\begin{quote}\begin{verbatim}
nice_pair(A, B) :-
    np(A, B),
    np(B, A).

np(A, B) :-
    [A,B] :: [yellow, blue, red, orange, green],
    A #= yellow #=> B :: [blue, red],
    A #= blue #=> B #= yellow,
    A #= red #=> B #= yellow,
    A #= green #=> B #= orange,
    A #= orange #=> B #= green.
\end{verbatim}\end{quote}

This method is quite simple and does not need any special analysis; on
the other hand it potentially creates a huge number of auxiliary
constraints and variables.

\subsection{Using Propia}
The simplest way is to load the Generalised Propagation library ({\em
propia} - see \ref{propia} above) and use
arc-consistency ({\em ac}) propagation, viz:
\begin{quote}\begin{verbatim}
?- nice_pair(X,Y) infers ac
\end{verbatim}\end{quote}

\subsection{Using the {\em element} Constraint}

In this case we use the \verb0element/30 predicate, available in the fd
library.  It is rather awkward to use, because additional
variables are required, but it propagates efficiently:

\begin{quote}\begin{verbatim}
nice_pair(A, B) :-
    element(I, [yellow, yellow, blue, red, green, orange], A),
    element(I, [blue, red, yellow, yellow, orange, green], B).
\end{verbatim}\end{quote}

We define a new variable $I$ which is an index into the clauses
of the fit predicate. The first colour list contains colours in the
first argument of \verb0fit/20 and the second list
contains colours from the second argument.

Behind the scenes, this is exactly the implementation used for
arc-consistency propagation by the Generalised Propagation library.

Because of the specific and efficient algorithm implementing the
\verb0element/30 constraint,  it is usually faster than the first
approach, using evaluation constraints.

%----------------------------------------------------------------------
\section{Built-in Constraints}

\begin{description}
\item[?Vars :: ?Domain]\ \\
Domain declaration, in fact just a very simple form of a constraint.

\item[?T1 \#\bsl= ?T2 or ?T1 \#\# ?T2]\ \\
\index{\#\verb+\=+/2}
The value of the rational term {\it T1} is not equal to the value of the
rational term {\it T2}.

\item[?T1 \#\lt ?T2]\ \\
\index{\#$<$/2}
The value of the rational term {\it T1} is less than the value of the
rational term {\it T2}.

\item[?T1 \#$<=$ ?T2]\ \\
\index{\#$<=$/2}
The value of the rational term {\it T1} is less than or equal to the value of the
rational term {\it T2}.

\item[?T1 \#= ?T2]\ \\
\index{\#=/2}
The value of the rational term {\it T1} is equal to the
value of the rational term {\it T2}.

\item[?T1 \#$>$ ?T2]\ \\
\index{\#$>$/2}
The value of the rational term {\it T1} is greater than the
value of the rational term {\it T2}.

\item[?T1 \#$>=$ ?T2]\ \\
\index{\#$>=$/2}
The value of the rational term {\it T1} is greater than or equal to the
value of the rational term {\it T2}.

\item[element(?Index, +List, ?Value)]\ \\
\index{element/3}
The {\it Index}'th element of the ground list {\it List}
is equal to {\it Value}.

\item[alldifferent(?List)]
\index{alldifferent/1}
All elements of {\it List} (domain variables and ground terms) are pairwise
different.

\item[atmost(+Number, ?List, +Val)]\ \\
\index{atmost/3}
At most {\it Number} elements of the list {\it List} of domain variables
and ground terms are equal to the ground value {\it Val}.
\end{description}

%----------------------------------------------------------------------
%\section{Propagation}

%----------------------------------------------------------------------
\section{Labelling}
\begin{description}
\item[indomain(+DVar)]\ \\
\index{indomain/1}
This predicate instantiates the domain variable {\it DVar} to
elements of its domain, on backtracking the subsequent value is taken.
It is used e.g. to find a value of {\it DVar} which is consistent
with all currently imposed constraints.
If {\it DVar} is a ground term, it succeeds.
Otherwise, if it is not a domain variable, an error is raised.

\item[labeling(+List)]
\index{labeling/1}
\index{labeling!fd}
The elements of the {\it List} are instantiated using the
\bipref{indomain/1}{../bips/lib/fd/indomain-1.html} predicate.

\item[deleteff(?Var, +List, -Rest)]
\index{deleteff/3}
This predicate is used to select a variable from a list of domain variables
which has the smallest domain.
{\it Var} is the selected variable from {\it List},
{\it Rest} is the rest of the list without {\it Var}.

\item[deleteffc(?Var, +List, -Rest)]
\index{deleteffc/3}
This predicate is used to select the most constrained variable from a list
of domain variables.
{\it Var} is the selected variable from {\it List} which has the least domain
and which has the most constraints attached to it.
{\it Rest} is the rest of the list without {\it Var}.

\item[deletemin(?Var, +List, -Rest)]
\index{deletemin/3}
This predicate is used to select a variable from a list of domain variables
which has the smallest lower domain bound. This is useful e.g.\ when trying to
schedule tasks with their earliest possible starting time.
\end{description}

The {\em fd\_search} library provides a variety of search routines
based on FD.

%----------------------------------------------------------------------
\section{Optimization}

There are optimization predicates in the { \eclipse } finite
domains library.
However it is recommended to use the generic optimization predicates
in the {\em branch\_and\_bound} library.
These predicates support optimization in conjunction with all the
different solvers in {\eclipse} which employ variable domains.

\begin{description}
\item[minimize(+Goal,-Cost)]
\index{minimize/2}
The simplest predicate in the {\em branch\_and\_bound} library is
\verb0minimize/20, which behaves as follows.
A solution of the goal Goal is found that minimizes the value of
Cost. Cost should be a variable that is affected, and eventually
instantiated, by Goal. Usually, Goal is the search
procedure of a constraint problem and Cost is the variable
representing the cost. The solution is found using the branch and
bound method: as soon as a solution is found, it
is recorded and the search is continued or restarted with an
additional constraint on the Cost variable which requires the next
solution to be better than the previous one.
Iterating this process yields an optimal solution in the end.

\item[bb_min(+Goal, -Cost, ++Options)]
\index{bb\_min/3}
The user can take more control over the branch and bound behaviour by
invoking the predicate \verb0bb_min/30 which supports a variety of
different options within the branch and bound framework.
\end{description}

%----------------------------------------------------------------------

\section{Bin Packing}
This section presents a worked example using finite domains to solve a
bin-packing problem.

\subsection{Problem Definition}
In this type of problems the goal is to pack a certain amount of
different items into the minimal number of bins under specific constraints.
Let us solve an example given by Andre Vellino in the Usenet
group comp.lang.prolog, June 93:
\begin{itemize}
\item There are 5 types of items:

        glass, plastic, steel, wood, copper

\item There are three types of bins:

        red, blue, green

\item        whose capacity constraints are:

\begin{itemize}
\item        red   has capacity 3
\item        blue  has capacity 1
\item 	     green has capacity 4
\end{itemize}

\item containment constraints are:
\begin{itemize}
\item        red   can contain glass, wood, copper
\item        blue  can contain glass, steel, copper
\item        green can contain plastic, wood, copper
\end{itemize}

\item and requirement constraints are (for all bin types):

        wood requires plastic

\item Certain component types cannot coexist:

\begin{itemize}
\item        glass and copper exclude each other
\item        copper and plastic exclude each other
\end{itemize}

\item and certain bin types have capacity constraint for certain
components

\begin{itemize}
\item red   contains at most 1 wood item
\item green contains at most 2 wood items
\end{itemize}

\item Given an initial supply of:
\begin{itemize}
\item 1 glass item
\item 2 plastic items
\item 1 steel items
\item 3 wood items
\item 2 copper items
\end{itemize}
what is the minimum total number of bins required to
contain the components?
\end{itemize}

\subsection{Problem Model - Using Structures}

For modelling this problem we need to refer to an array of quantities
of glass items, plastic items, steel items, wood items and copper
items.
We therefore introduce a structure to hold this array:
\begin{verbatim}
:- local struct(contents(glass,plastic,steel,wood,copper))
\end{verbatim}

To represent a bin, with its colour, capacity and contents we use
another structure:
\begin{verbatim}
:- local struct(bin(colour,capacity,contents:contents))
\end{verbatim}
The {\bf contents} attribute of {\bf bin} is itself a {\bf contents}
structure.

The predicate {\bf solve_bin/2} is the general predicate
that takes an amount of components packed into a {\bf contents}
structure and it returns the solution.
\begin{quote}
\begin{verbatim}
?- Demand = contents\{glass:1,plastic:2,steel:1,wood:3,copper:2\},
   solve_bin(Demand, Bins).
\end{verbatim}
\end{quote}

\subsection{ Handling an Unknown Number of Bins}

{\bf solve_bin/2}  calls {\bf bin\_setup/2} to
generate a list {\bf Bins}.
It adds some redundant constraint to remove symmetries (two
symmetrical solutions are
the same, but with the bins in a different order).
Finally it labels all decision variables in the problem.
\begin{quote}
\begin{verbatim}
solve_bin(Demand, Bins) :-
    bin_setup(Demand,Bins),
    remove_symmetry(Bins),
    bin_label(Bins).
\end{verbatim}
\end{quote}

The usual pattern for solving finite domain problems is to state
constraints on a set of variables, and then label them.
However, because the number of bins needed is not known initially, it
is awkward to model the problem with a fixed set of variables.

One possibility would be to take a fixed, large enough, number of bins
and to try to find a minimum number of non-empty bins.
However, for efficiency, we choose to solve a sequence of problems,
each one with a - larger - fixed number of bins,
until a solution is found.

The predicate {\bf bin_setup/2} to generate a list of bins with appropriate
constraints works as follows.
First it tries to match the (remaining) demand with zero,
and use no (further) bins.
If this fails, a new bin is added to the bin list;
appropriate constraints are imposed on all the new bin's
variables;
its contents are subtracted from the demand;
and the {\bf bin_setup/2} predicate calls itself recursively:

\begin{quote}
\begin{verbatim}
bin_setup(Demand,[]) :-
        all_zeroes(Demand).
bin_setup(Demand,[Bin|Bins]) :-
        constrain_bin(Bin),
        reduce_demand(Demand,Bin,RemainingDemand),
        bin_setup(RemainingDemand,Bins).

all_zeroes(
           contents\{glass:0,plastic:0,wood:0,steel:0,copper:0\}
          ).

reduce_demand(
              contents\{glass:G,plastic:P,wood:W,steel:S,copper:C\},
              bin\{glass:BG,plastic:BP,wood:BW,steel:BS,copper:BC\},
              contents\{glass:RG,plastic:RP,wood:RW,steel:RS,copper:RC\}
             ) :-
       RG #= G - BG,
       RP #= P - BP,
       RW #= W - BW,
       RS #= S - BS,
       RC #= C - BC.

\end{verbatim}
\end{quote}

\subsection{Constraints on a Single Bin}

The constraints imposed on a single bin correspond exactly to the
problem statement:
\begin{quote}
\begin{verbatim}
constrain_bin(bin\{colour:Col,capacity:Cap,contents:C\}) :-
        colour_capacity_constraint(Col,Cap),
        capacity_constraint(Cap,C),
        contents_constraints(C),
        colour_constraints(Col,C).
\end{verbatim}
\end{quote}

\paragraph{colour\_capacity\_constraint}
The colour capacity constraint relates the colour of the bin to its
capacity.  It uses generalised propagation to apply
arc-consistency propagation.
\begin{quote}
\begin{verbatim}
colour_capacity_constraint(Col,Cap) :-
	capacity(Col,Cap) infers ac.

capacity(blue, 1).
capacity(green,4).
capacity(red,  3).
\end{verbatim}
\end{quote}
At the end of this section we will present another way to implement the
colour\_capacity\_constraint, using the explicit "glass-box"
functionality of fd.

\paragraph{capacity\_constraint}
The capacity constraint states:
\begin{itemize}
\item that the number of items of each
kind in the bin is non-negative,
\item
their sum does not exceed the capacity of the bin,
\item and the bin is non-empty (an empty bin serves no purpose)
\end{itemize}

\begin{quote}
\begin{verbatim}
capacity_constraint(Cap, contents\{glass:G,
                                   plastic:P,
                                   steel:S,
                                   wood:W,
                                   copper:C\}) :-
        G #>= 0, P #>= 0, S #>= 0, W #>= 0, C #>= 0,
        Cap #>= G+P+W+S+C,
        G+P+W+S+C #> 0.
\end{verbatim}
\end{quote}

\paragraph{contents\_constraints}
The contents_constraints directly enforce the restrictions on items in
the bin: wood requires paper, glass and copper exclude each other, and
copper and plastic exclude each other:
\begin{quote}
\begin{verbatim}
contents_constraints(contents\{glass:G,plastic:P,wood:W,copper:C\}) :-
        requires(W,P),
        exclusive(G,C),
        exclusive(C,P).
\end{verbatim}
\end{quote}

These constraints are expressed as logical combinations of constraints
on the number of items.
"requires" is expressed using entailment, \verb0#=>0.
"Wood requires paper" is expressed in logic as "If the number of wood
items is greater than zero, then the number of paper items
is also greater than zero":
\begin{quote}
\begin{verbatim}
requires(W,P) :-
        W #> 0 #=> P #> 0.
\end{verbatim}
\end{quote}

Exclusion is expressed using disjunction, \verb0#\/0.
"X and Y are exclusive" is expressed as "Either the number of items of
kind $X$ is zero, or the number of items of kind $Y$ is zero":
\begin{quote}
\begin{verbatim}
exclusive(X,Y) :-
        X #= 0 #\/ Y #= 0.
\end{verbatim}
\end{quote}

\paragraph{colour\_constraints}
The colour constraint limits the number of wooden items in bins of
different colours.
Like the capacity constraint, the relation between the colour and
capacity ($WCap$ is expressed using generalised propagation to enforce
arc-consistency.  The number of wooden items is then constrained not to
exceed the capacity:
\begin{quote}
\begin{verbatim}
colour_constraints(Col,contents\{wood:W\}) :-
        colour_wood_cap(Col,WCap) infers ac,
        WCap #>= W.

colour_wood_cap(blue, Cap) :- capacity(blue, Cap).
colour_wood_cap(green,2).
colour_wood_cap(red,1).
\end{verbatim}
\end{quote}

This model artificially introduces a capacity of blue bins for
wood items (set simply at its maximum capacity for all items).

\subsection{Symmetry Constraints}
To make sure two solutions are not just different permutations of the
same bins, we fix the sequence of variables specifying each bin, and
lexically order their values.

\begin{quote}
\begin{verbatim}
remove_symmetry(Bins) :-
        ( fromto(Bins,[B1,B2|Rest],[B2|Rest],[_Last])
        do
            lex_ord(B1,B2)
        ).
\end{verbatim}
\end{quote}

Since colours don't have a built-in ordering, we map them to integers
and apply the lexicographic ordering to the integers instead of the
colours.  Since the mapping is defined before values are chosen for the
variables, the mapping is turned into a constraint, using generalised
propagation.
\begin{quote}
\begin{verbatim}
lex_ord(bin\{colour:Col1,contents:Conts1\},
        bin\{colour:Col2,contents:Conts2\}) :-
        colour_map(Col1,Int1) infers ac,
        colour_map(Col2,Int2) infers ac,
        term_variables(Conts1,Vars1),
        term_variables(Conts2,Vars2),
        lexico_le([Int1|Vars1],[Int2|Vars2]).

colour_map(blue,1).
colour_map(green,2).
colour_map(red,3).
\end{verbatim}
\end{quote}
(Another way to associate integers with colours is
described at the end of this section.)

Though lexicographical ordering is defined in the fd\_global library of
{ \eclipse }, Warwick
Harvey has written a very elegant version that enforces arc-consistency,
using a few features of the fd library.  Here it is:

\begin{quote}
\begin{verbatim}
lexico_le(Xs, Ys) :-
        lexico_le_bool(Xs, Ys, 1).

lexico_le_bool([], [], 1).
lexico_le_bool([X | Xs], [Y | Ys], B) :-
        B isd (X #< Y + B1),
        lexico_le_bool(Xs, Ys, B1).
\end{verbatim}
\end{quote}

\subsection{Search}

The search is done by first choosing a colour for each bin, and then
labelling the remaining variables.
\begin{quote}
\begin{verbatim}
bin_label(Bins) :-
        ( foreach(bin\{colour:C\},Bins) do indomain(C) ),
        term_variables(Bins,Vars),
        labeleff(Vars).
\end{verbatim}
\end{quote}

The remaining variables are labelled using the first fail heuristic
(using the fd built-in \verb0deleteff0).  This code illustrates the use
of \verb0fromto0 for dynamically ordering the variables to be labelled.
\begin{quote}
\begin{verbatim}
labeleff(Vars) :-
        ( fromto(Vars,InVars,OutVars,[])
        do
            deleteff(Var,InVars,OutVars),
            indomain(Var)
        ).
\end{verbatim}
\end{quote}

\subsection{Glass-Box Implementation of colour\_capacity\_constraint}
To illustrate how the facilities of fd can be used to directly encode
constraint behaviour, we have reimplemented the
colour\_capacity\_constraint without using generalised propagation.

The first requirement is to ensure the capacity variable has a finite
domain.  One way to do this is as follows:
\begin{quote}
\begin{verbatim}
colour_capacity_constraint(Col,Cap) :-
    ( is_domain(Cap) -> true ; Cap #>= 0 ),
    col_cap_cons(Col,Cap).
\end{verbatim}
\end{quote}

If the colour $Col$ is already known, the constraint is enforced simply by
invoking the capacity predicate.  If, however, the colour $Col$ is a
variable, then the more complex var\_col\_cap predicate is called:

\begin{quote}
\begin{verbatim}
col_cap_cons(Col,Cap) :-
    ( nonvar(Col) -> capacity(Col,Cap) ;
      var(Col) -> var_col_cap(Col,Cap)
    ).
\end{verbatim}
\end{quote}

If the required capacity (i.e. the minimum value in the domain of $Cap$)
is more than 1, then the colour cannot be blue.
If, moreover, the required capacity is more than 3, then the colour must
be green.
If the required capacity is either 2 or 3, then any change to the $Col$
or $Cap$ variables must instantiate them, because each variable has only
two
possible values).  Since there is a unique
colour for each capacity, as well as vice versa, it suffices to call the
capacity predicate as soon as either variable is instantiated.
If, however the minimum capacity is still 1, then the col\_cap\_cons
constraint is suspended again, until the colour is instantiated
\verb0Col->inst0 or the required
capacity increases \verb0Cap->min0:
\begin{quote}
\begin{verbatim}
var_col_cap(Col, Cap) :-
    mindomain(Cap,MinC),
    (MinC > 1 ->
        Col #\= blue,
        (MinC > 3 ->
            Col = green
        ;
            suspend(capacity(Col, Cap), 3, (Col, Cap)->inst)
        )
    ;
        suspend(col_cap_cons(Col, Cap), 3, [Col->inst, Cap->min])
    ).

\end{verbatim}
\end{quote}

\subsection{Using Macros instead of a Colour-Integer Mapping}

As we discussed in the section on symmetries, above, there is no
built-in ordering on colours.

To keep the lexicographic ordering predicate simple and still have a
symbolic
representation of the colour in the program, we can define
input macros that transform the colour atoms into integers:

\begin{quote}
\begin{verbatim}
:- define_macro(no_macro_expansion(blue)/0, tr_col/2, []).
:- define_macro(no_macro_expansion(green)/0, tr_col/2, []).
:- define_macro(no_macro_expansion(red)/0, tr_col/2, []).

tr_col(no_macro_expansion(red), 1).
tr_col(no_macro_expansion(green), 2).
tr_col(no_macro_expansion(blue), 3).

\end{verbatim}
\end{quote}

Now the symmetry removal can be programmed using the following
lexical ordering predicate:

\begin{quote}
\begin{verbatim}
lex_ord(Bin1,Bin2) :-
       term_variables(Bin1,V1),
       term_variables(Bin2,V2),
       lexico_le(V1,V2).
\end{verbatim}
\end{quote}


%!TEX root = paper.tex
\subsection{Nonlinear shoaling wave over submerged bar}
In this laboratory experiment an incident sin wave travels over a trapezoidal underwater obstacle and the surface elevations of the non-breaking wave are measured at several gauges spread along the obstacle.
This experiment conducted in different versions by \cite{BejiBattjes.1993} and \cite{BejiBattjes.1994}, \cite{Dingemans.1994} (more cases, here only case A considered), and by Luth et al. (1994) (cited in \cite{Dingemans.1994}, but I do not have access to the report).
In this experiments, gauges and setups were different (see tables \ref{tab:bejibattjes_gauges_SZ} -- \ref{tab:bejibattjes_gauges_BB1994} and figures \ref{fig:bejibattjes_setup_SZ} -- \ref{fig:bejibattjes_setup_BB1994}).

The setup is as described in \cite{StellingZijlema.2003}, and M. Zijlema provided the data. According to the gauges, this should be the data described in \cite{Dingemans.1994, StellingZijlema.2003} as the data from Luth et al. (1994), but scaled (divided) by a factor of 2.

The initial condition is the unperturbed state. The boundary condition is described as in incident wave at the left boundary of the computational domain, where the surface elevation is set to $\xi=a \text{sin}\left(\frac{2\pi t}{T}\right)$ with period $T=2.02 \,$s and amplitude $a=1.0\,$cm.
We impose reflecting boundary conditions at the boundary in x-direction and periodic boundary conditions in y-direction. For the setup see figure \ref{fig:bejibattjes_setup_SZ}. The computational domain is enlarged to 40m to ensure no reflecting waves disturbing the solution during the simulation time of 40s.

Note: No boundary conditions are specified for w and q at the moment, computations which set them to zero are running. This hopefully explains the wrong behavior.

\begin{table}[htbp]
\begin{tabular}{lllllllll}
\textbf{gauge} & G4 & G5 & G6 & G7 & G8 & G9 & G10 & G11 \\
\toprule
\textbf{x [m]} & 10.5 & 12.5 & 13.5 & 14.5 & 15.7 & 17.3 & 19.9 & 21.0 \\
\bottomrule
\end{tabular}
\caption{Locations of gauges used in \cite{StellingZijlema.2003}}
\label{tab:bejibattjes_gauges_SZ}
\end{table}

\begin{table}[htbp]
\begin{tabular}{lllllllll}
\textbf{gauge} & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 \\
\toprule
\textbf{x [m]} & 6.0 & 11.0 & 12.0 & 13.0 & 14.0 & 15.0 & 16.0 & 17.0 \\
\bottomrule
\end{tabular}
\caption{Locations of gauges used in \cite{BejiBattjes.1993}}
\label{tab:bejibattjes_gauges_BB1993}
\end{table}

\begin{table}[htbp]
\begin{tabular}{llllllll}
\textbf{gauge} & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
\toprule
\textbf{x [m]} & 6 & 10.8 & 12.8 & 13.8 & 14.8 & 16.0 & 17.6 \\
\bottomrule
\end{tabular}
\caption{Locations of gauges used in \cite{BejiBattjes.1994}}
\label{tab:bejibattjes_gauges_BB1994}
\end{table}

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{bejibattjes_setup_SZ}
\caption{Setup of the experiment of Beji and Battjes according to \cite{StellingZijlema.2003}}
\label{fig:bejibattjes_setup_SZ}
\end{figure}

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{bejibattjes_setup_BB1993}
\caption{Setup of the experiment of Beji and Battjes according to \cite{BejiBattjes.1993}}
\label{fig:bejibattjes_setup_BB1993}
\end{figure}

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{bejibattjes_setup_BB1994}
\caption{Setup of the experiment of Beji and Battjes according to \cite{BejiBattjes.1994}}
\label{fig:bejibattjes_setup_BB1994}
\end{figure}

\subsubsection{Results of \nh\ model}
The model results can be found in figure \eqref{fig:nh_bejjibattjes_nh}.
The results are shifted in time by 4.5s to match the laboratory data at gauge 10 in phase after 20s.

\begin{figure}[htbp]
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_4}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_5}
\end{minipage} \\
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_6}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_7}
\end{minipage} \\
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_8}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_9}
\end{minipage} \\
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_10}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_11}
\end{minipage}
\caption{Comparison of the laboratory (black) sea surface height at gauges with the simulation results of the \nh\ model with linear pressure profile (yellow) and quadratic pressure profile (red)}
\label{fig:nh_bejjibattjes_nh}
\end{figure}


% !TEX TS-program = lualatex
\documentclass{article}

%\usepackage{fontspec}

\begin{document}

\section{Introduction}
\label{Intro}

Therefore we still aim to compute the traces in the drift term exactly. For this we note that the derivative in the trace is an extremely sparse matrix only containing two nonzero $3\times 3$ blocks located at block-entries $(i,i+\hat d)$ and $(i+\hat d,i)$. On multiplication with $D^{-1}$ the product matrix will be sparse, existing of 2 full column-blocks at $(k,i)$ and $(k,i+\hat d$), $k=1,\ldots,V$ (a column block has size $3V\times 3$). To compute such a trace the same columns of $D^{-1}$ are needed. Hence, it looks as if the complete matrix $D^{-1}$ is needed when computing all traces. However, the situation is not as dramatic as it seems. Indeed, on taking the trace of the product matrix only the \textit{diagonal elements} of the product is needed. To compute these diagonal elements we do not need the complete columns of $D^{-1}$ but only the $3\times 3$ blocks located at the transposed positions of the $3\times 3$ derivative blocks $\partial D/\partial a$. For example, the derivative block at location $(i+\hat d,i)$ will only give a contribution to the trace when multiplied with the $3\times 3$ block of $D^{-1}$ located at $(i, i+\hat d)$.

Therefore we still aim to compute the traces in the drift term exactly.
For this we note that the derivative in the trace is an extremely sparse matrix only containing two nonzero $3\times 3$ blocks located at block-entries $(i,i+\hat d)$ and $(i+\hat d,i)$.
On multiplication with $D^{-1}$ the product matrix will be sparse, existing of 2 full column-blocks at $(k,i)$ and $(k,i+\hat d$), $k=1,\ldots,V$ (a column block has size $3V\times 3$).
To compute such a trace the same columns of $D^{-1}$ are needed. Hence, it looks as if the complete matrix $D^{-1}$ is needed when computing all traces.
 However, the situation is not as dramatic as it seems.
 Indeed, on taking the trace of the product matrix only the \textit{diagonal elements} of the product is needed.
 To compute these diagonal elements we do not need the complete columns of $D^{-1}$ but only the $3\times 3$ blocks located at the transposed positions of the $3\times 3$ derivative blocks $\partial D/\partial a$.
 For example, the derivative block at location $(i+\hat d,i)$ will only give a contribution to the trace when multiplied with the $3\times 3$ block of $D^{-1}$ located at $(i, i+\hat d)$.

\end{document}


%!TEX root = paper.tex
%
% AMRClaw
%
% Lead currently:  Marsha Berger
%

\subsection{\amrclaw} \label{sec:amrclaw}
Fortran code in the \amrclaw repository performs block-structured adaptive mesh
refinement \cite{BO,BC} for both \clawpack and \geoclaw  applications.
The algorithms implemented in \amrclaw are discussed in detail in
\cite{mjb-rjl:amrclaw,LeVequeGeorgeBerger:an11}, but a  short description is
given here to set the stage for a description of recent changes. \revised{
This type of refinement solves the PDE on a hierarchy of logically rectangular
grids.  One (or more) level 1 grids comprise the entire domain, while grids at
finer level are created and destroyed (as opposed to moving these grids) to
follow important features in the solution.}

\begin{figure}[tb]
    \centering
    \includegraphics[width=0.6\textwidth, trim=40 80 40 70, clip]{final_submission/Fig1.png}
    \caption{\revised{An illustration showing grid cells on levels one and two, and only grid outlines on levels three and four.}}
    \label{fig:figure1}
\end{figure}

\amrclaw includes the functionality for:
\begin{itemize}
\item Coordinating the flagging of points where refinement is needed,
with a variety of criteria possible for flagging cells that need refinement
from each level to the next finer level (including Richardson extrapolation,
gradient testing, or user-specified criteria)\footnote{See
\url{http://www.clawpack.org/flag.html}},
\item Organizing the flagged points into efficient grid
patches at the next finer level, using the algorithm of
\cite{mjb-rig:cluster},
\item Interpolating the solution to newly created fine grids and initializing
auxiliary data (topography, wind velocity, metric data and so on) on  these
grids,
\item Averaging fine grid solutions to coarser grids,
\item Orchestrating the adaptive time stepping (i.e. sub-cycling in time),
\item Interpolating coarse grid solution to fine grid ghost cells, and
\item Maintaining conservation at patch boundaries between resolution levels.
\end{itemize}

\amrclaw now allows users to specify ``regions'' in space-time
$[x_1,x_2] \times [y_1,y_2] \times [t_1,t_2]$ in which refinement is forced to
be at least at some level $L_1$ and is allowed to be at most $L_2$.  This can be
useful for constraining refinement, e.g. allowing or ensuring resolution of only
a small coastal region in a global tsunami simulation. Previously the user could
enforce such conditions by writing a custom flagging routine, but now this is
handled in a general manner so that the parameters above can all be specified in
the Python problem specification. Multiple regions can be specified, and a
simple rule is used to determine the constraints at a grid cell that lies in
multiple regions.

Auxiliary arrays are often used in \clawpack to store data that
describes the problem and the routine.
The routine \texttt{setaux} must then be provided by the user to set these values each time a
new grid patch is created.  For some applications computing these values can be time-consuming.  In \clawpack 5.2,
this code was improved to allow reuse of values from previous patches at
the same level where possible at each regridding time.
This is backward compatible, since no harm is done if previously
written routines are used that still compute and overwrite instead of
checking a mask.

In \clawpack 5.3 the capability to specify spatially varying boundary
conditions was added. For a single grid, it is a simple matter to
compute the location of the ghost cells that extend outside the
computational domain and set them appropriately.  With AMR however,
the boundary condition routine can be called for a grid located
anywhere in the domain, and may contain fewer or larger numbers of
ghost cells. For this reason, the boundary condition routines
 do not assume a fixed number of ghost cells.

Anisotropic refinement is allowed in both two and three dimensions.
This means that the spatial and temporal refinement ratios can be
specified independently from one another (as long as the temporal
refinement satisfies the CFL condition).  In addition, capabilities
have been added to automatically select the refinement ratio in time  on each
level based on the CFL condition.  This has only been implemented in
\geoclaw where the wave speed in the shallow water equations
depends on the local depth. The finest grids are often located only in
shallow coastal regions, so a large refinement ratio in space does not
lead to a large refinement ratio in time.

\amrclaw has been parallelized using OpenMP directives. \revised{The main
paradigm in structured AMR is an outer loop over levels of refinement, and an
inner loop over all grids at that level, where \revised{the same} operation is
performed on each grid (i.e. taking a time step, finding ghost cells,
conservation updates, etc.).  This inner loop is parallelized using a {\tt
parallel for} loop construct.  One thread is assigned to operate on one grid.
Dynamic scheduling is used with a chunk size of one.  To help with load
balancing, grids at each level are sorted from largest to smallest, using the
total number of cells in the grid as an indicator of work.  In addition the
grids are limited to a maximum of 32 cells in each dimension, otherwise they
are bisected until this condition is met.}  Note that this approach causes a
memory bulge. Each thread must have its own scratch arrays to save the incoming
and outgoing waves and fluxes for future conservation fix-ups. The bulge is
directly proportional to the number of threads executing. For stack-based memory
allocation per thread, the use of the environment variable {\tt OMP\_STACKSIZE}
to increase the limit may be necessary.

\cref{fig:shockbubble} shows two snapshots of the solution to a
three-dimensional shock-bubble interaction problem found in the \clawpack
\texttt{apps} repository, illustrating localized phenomena requiring adaptive
refinement. In \cref{fig:amr_scaling} we show scalability tests and some timings
for this example, when run on a 24 core Intel Xeon Haswell machine
\revised{(E5-2670v3 at 2.3 GHz)}, using
\texttt{KMP\_AFFINITY compact} \revised{with one thread per core}.
For timing purposes, the only modifications made to the input parameters was to
turn off check-pointing and graphics output. The plot on the left shows that most
of the wall clock time is in the integration routine (\texttt{stepgrid}), which
closely tracks the total time. The second chunk of time is in the regridding,
which contains algorithms that are not completely scalable. Very little time is
in the filling of ghost cells, mostly from other patches but also includes those
at domain boundaries. The efficiency is above 80\% until 24 cores.
Note that there are only two grids on level, and an average of
22.8 level 2 grids. Most of the work is on level 3 grids, where there are an
average of 138.1 grids over all the level 3 timestep.  At 24 cores, there are
on average 5.8 grids per core, and the grids are very different sizes.

\begin{figure}[t]
  \begin{center}
    \includegraphics[width=\textwidth]{figures/final_submission/Fig2}
    % \plotbox{\includegraphics[width=0.45\textwidth]{f1.png}}\hfil
    % \plotbox{\includegraphics[width=0.45\textwidth]{f4.png}}
  \end{center}
\caption{\amrclaw example demonstrating a
  shock-bubble interaction in the Euler equations of compressible
  gas-dynamics at two times, illustrating the need for adaptive refinement
  to capture localized behavior. \revised{There are two $20 \times 10 \times 10$
  grids at
  level 1.  They are refined} where needed
  by factors of 4 and then 2 in this 3-level run.}
\label{fig:shockbubble}
\end{figure}

\begin{figure}[h]
  \begin{center}
    % \includegraphics[width=0.45\textwidth]{amr_scaling}
    % \plotbox{\includegraphics[width=0.4\textwidth,
    %   clip=true,trim=0cm 1cm 1cm 2cm]{newcpu.pdf}}\hfil
    % \plotbox{\includegraphics[width=0.4\textwidth,
    %   clip=true,trim=0cm 1cm 1cm 2cm]{efficiency.pdf}}
    \includegraphics[width=\textwidth]{final_submission/Fig3.pdf}
  \end{center}
  \caption{(a) Strong scaling results for the \amrclaw example shown in
    \cref{fig:shockbubble}.
    (b) Plot of efficiency based on total computational time.}
  \label{fig:amr_scaling}
\end{figure}

\revised{The target architecture for \amrclaw and \geoclaw are multi-core
machines.  \pyclaw on the other hand scales to tens of
thousands of cores using MPI via PETSc \cite{petsc-user-ref} but is not
adaptive.}