% BEGIN LICENSE BLOCK
% Version: CMPL 1.1
%
% The contents of this file are subject to the Cisco-style Mozilla Public
% License Version 1.1 (the "License"); you may not use this file except
% in compliance with the License.  You may obtain a copy of the License
% at www.eclipse-clp.org/license.
%
% Software distributed under the License is distributed on an "AS IS"
% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied.  See
% the License for the specific language governing rights and limitations
% under the License.
%
% The Original Code is  The ECLiPSe Constraint Logic Programming System.
% The Initial Developer of the Original Code is  Cisco Systems, Inc.
% Portions created by the Initial Developer are
% Copyright (C) 2006 Cisco Systems, Inc.  All Rights Reserved.
%
% Contributor(s):
%
% END LICENSE BLOCK

\chapter{Getting started with Finite Domains}

%----------------------------------------------------------------------
\section{Using the Finite Domains Library}
To use the Finite Domains Library, load the library using either of
\begin{quote}\begin{verbatim}
:- lib(fd).
:- use_module(library(fd)).
\end{verbatim}\end{quote}
Specify this at the beginning of your program.

%----------------------------------------------------------------------
\section{Structure of a Constraint Program}
The typical top-level structure of a constraint program is
\begin{quote}\begin{verbatim}
solve(Variables) :-
        read_data(Data),
        setup_constraints(Data, Variables),
        labeling(Variables).
\end{verbatim}\end{quote}
where setup_constraints/2 contains the problem model. It creates the
variables and the constraints over the variables.
This is often deterministic, but not necessarily.
The labeling/1 procedure is the search part of the program. It
tries to find solutions by trying all instantiations for the
variables. This search is being pruned by constraint propagation.

The above program will find all solutions.
If the best solution is wanted, we can just wrap a branch-and-bound
procedure around the search component of the program:
\begin{quote}\begin{verbatim}
solve(Variables) :-
        read_data(Data),
        setup_constraints(Data, Variables),
        min_max(labeling(Variables), Objective).
\end{verbatim}\end{quote}

%----------------------------------------------------------------------
\section{Modelling}
The modelling code need to do the following:
    \begin{itemize}
    \item Create the variables with their initial domains
    \item Setup the constraints between the variables
    \end{itemize}
%\htmladdnormallink{Example: Send more money}{../../examples/sendmore.pl.txt}

A simple example is the ``cryptarithmetic'' puzzle,
\verb0SEND+MORE = MONEY0.
The idea is to associate a digit (0-9) with each letter so the
equation is true.

The { \eclipse } code is as follows:
\begin{quote}\begin{verbatim}

:- lib(fd).

sendmore1(Digits) :-
    Digits = [S,E,N,D,M,O,R,Y],

% Assign a finite domain with each letter - S, E, N, D, M, O, R, Y -
% in the list Digits
    Digits :: [0..9],

% Constraints
    alldifferent(Digits),
    S #\= 0,
    M #\= 0,
                 1000*S + 100*E + 10*N + D
               + 1000*M + 100*O + 10*R + E
    #= 10000*M + 1000*O + 100*N + 10*E + Y,

% Search
    labeling(Digits).
\end{verbatim}\end{quote}

%----------------------------------------------------------------------
\section{Simple User-defined Constraints}
Conceptual constraints that are just conjunctions of primitive
constraints can easily be defined.
%\htmladdnormallink{Example: Zebra}{../../examples/zebra.pl.txt}

For example, let us assume that we have a set of colours and we want
to define that some colours fit with each other and others do
not. This should work in such a
way as to propagate possible changes in the domains as soon as this
becomes possible.

Let us assume we have a symmetric relation that defines which colours
fit with each other:

\begin{quote}\begin{verbatim}
% The basic relation
fit(yellow, blue).
fit(yellow, red).
fit(blue, yellow).
fit(red, yellow).
fit(green, orange).
fit(orange, green).
\end{verbatim}\end{quote}

The predicate \verb0nice_pair(X, Y)0 is a constraint and any change of the
possible values of $X$ or $Y$ is propagated to the other variable. There
are many ways in which this pairing
can be defined in ECLiPSe. They are different solutions with different
properties, but they yield the same results.

\subsection{Using Evaluation Constraints}

We can also encode directly the relations between elements in the
domains of the two variables:
\begin{quote}\begin{verbatim}
nice_pair(A, B) :-
    np(A, B),
    np(B, A).

np(A, B) :-
    [A,B] :: [yellow, blue, red, orange, green],
    A #= yellow #=> B :: [blue, red],
    A #= blue #=> B #= yellow,
    A #= red #=> B #= yellow,
    A #= green #=> B #= orange,
    A #= orange #=> B #= green.
\end{verbatim}\end{quote}

This method is quite simple and does not need any special analysis; on
the other hand it potentially creates a huge number of auxiliary
constraints and variables.

\subsection{Using Propia}
The simplest way is to load the Generalised Propagation library ({\em
propia} - see \ref{propia} above) and use
arc-consistency ({\em ac}) propagation, viz:
\begin{quote}\begin{verbatim}
?- nice_pair(X,Y) infers ac
\end{verbatim}\end{quote}

\subsection{Using the {\em element} Constraint}

In this case we use the \verb0element/30 predicate, available in the fd
library.  It is rather awkward to use, because additional
variables are required, but it propagates efficiently:

\begin{quote}\begin{verbatim}
nice_pair(A, B) :-
    element(I, [yellow, yellow, blue, red, green, orange], A),
    element(I, [blue, red, yellow, yellow, orange, green], B).
\end{verbatim}\end{quote}

We define a new variable $I$ which is an index into the clauses
of the fit predicate. The first colour list contains colours in the
first argument of \verb0fit/20 and the second list
contains colours from the second argument.

Behind the scenes, this is exactly the implementation used for
arc-consistency propagation by the Generalised Propagation library.

Because of the specific and efficient algorithm implementing the
\verb0element/30 constraint,  it is usually faster than the first
approach, using evaluation constraints.

%----------------------------------------------------------------------
\section{Built-in Constraints}

\begin{description}
\item[?Vars :: ?Domain]\ \\
Domain declaration, in fact just a very simple form of a constraint.

\item[?T1 \#\bsl= ?T2 or ?T1 \#\# ?T2]\ \\
\index{\#\verb+\=+/2}
The value of the rational term {\it T1} is not equal to the value of the
rational term {\it T2}.

\item[?T1 \#\lt ?T2]\ \\
\index{\#$<$/2}
The value of the rational term {\it T1} is less than the value of the
rational term {\it T2}.

\item[?T1 \#$<=$ ?T2]\ \\
\index{\#$<=$/2}
The value of the rational term {\it T1} is less than or equal to the value of the
rational term {\it T2}.

\item[?T1 \#= ?T2]\ \\
\index{\#=/2}
The value of the rational term {\it T1} is equal to the
value of the rational term {\it T2}.

\item[?T1 \#$>$ ?T2]\ \\
\index{\#$>$/2}
The value of the rational term {\it T1} is greater than the
value of the rational term {\it T2}.

\item[?T1 \#$>=$ ?T2]\ \\
\index{\#$>=$/2}
The value of the rational term {\it T1} is greater than or equal to the
value of the rational term {\it T2}.

\item[element(?Index, +List, ?Value)]\ \\
\index{element/3}
The {\it Index}'th element of the ground list {\it List}
is equal to {\it Value}.

\item[alldifferent(?List)]
\index{alldifferent/1}
All elements of {\it List} (domain variables and ground terms) are pairwise
different.

\item[atmost(+Number, ?List, +Val)]\ \\
\index{atmost/3}
At most {\it Number} elements of the list {\it List} of domain variables
and ground terms are equal to the ground value {\it Val}.
\end{description}

%----------------------------------------------------------------------
%\section{Propagation}

%----------------------------------------------------------------------
\section{Labelling}
\begin{description}
\item[indomain(+DVar)]\ \\
\index{indomain/1}
This predicate instantiates the domain variable {\it DVar} to
elements of its domain, on backtracking the subsequent value is taken.
It is used e.g. to find a value of {\it DVar} which is consistent
with all currently imposed constraints.
If {\it DVar} is a ground term, it succeeds.
Otherwise, if it is not a domain variable, an error is raised.

\item[labeling(+List)]
\index{labeling/1}
\index{labeling!fd}
The elements of the {\it List} are instantiated using the
\bipref{indomain/1}{../bips/lib/fd/indomain-1.html} predicate.

\item[deleteff(?Var, +List, -Rest)]
\index{deleteff/3}
This predicate is used to select a variable from a list of domain variables
which has the smallest domain.
{\it Var} is the selected variable from {\it List},
{\it Rest} is the rest of the list without {\it Var}.

\item[deleteffc(?Var, +List, -Rest)]
\index{deleteffc/3}
This predicate is used to select the most constrained variable from a list
of domain variables.
{\it Var} is the selected variable from {\it List} which has the least domain
and which has the most constraints attached to it.
{\it Rest} is the rest of the list without {\it Var}.

\item[deletemin(?Var, +List, -Rest)]
\index{deletemin/3}
This predicate is used to select a variable from a list of domain variables
which has the smallest lower domain bound. This is useful e.g.\ when trying to
schedule tasks with their earliest possible starting time.
\end{description}

The {\em fd\_search} library provides a variety of search routines
based on FD.

%----------------------------------------------------------------------
\section{Optimization}

There are optimization predicates in the { \eclipse } finite
domains library.
However it is recommended to use the generic optimization predicates
in the {\em branch\_and\_bound} library.
These predicates support optimization in conjunction with all the
different solvers in {\eclipse} which employ variable domains.

\begin{description}
\item[minimize(+Goal,-Cost)]
\index{minimize/2}
The simplest predicate in the {\em branch\_and\_bound} library is
\verb0minimize/20, which behaves as follows.
A solution of the goal Goal is found that minimizes the value of
Cost. Cost should be a variable that is affected, and eventually
instantiated, by Goal. Usually, Goal is the search
procedure of a constraint problem and Cost is the variable
representing the cost. The solution is found using the branch and
bound method: as soon as a solution is found, it
is recorded and the search is continued or restarted with an
additional constraint on the Cost variable which requires the next
solution to be better than the previous one.
Iterating this process yields an optimal solution in the end.

\item[bb_min(+Goal, -Cost, ++Options)]
\index{bb\_min/3}
The user can take more control over the branch and bound behaviour by
invoking the predicate \verb0bb_min/30 which supports a variety of
different options within the branch and bound framework.
\end{description}

%----------------------------------------------------------------------

\section{Bin Packing}
This section presents a worked example using finite domains to solve a
bin-packing problem.

\subsection{Problem Definition}
In this type of problems the goal is to pack a certain amount of
different items into the minimal number of bins under specific constraints.
Let us solve an example given by Andre Vellino in the Usenet
group comp.lang.prolog, June 93:
\begin{itemize}
\item There are 5 types of items:

        glass, plastic, steel, wood, copper

\item There are three types of bins:

        red, blue, green

\item        whose capacity constraints are:

\begin{itemize}
\item        red   has capacity 3
\item        blue  has capacity 1
\item 	     green has capacity 4
\end{itemize}

\item containment constraints are:
\begin{itemize}
\item        red   can contain glass, wood, copper
\item        blue  can contain glass, steel, copper
\item        green can contain plastic, wood, copper
\end{itemize}

\item and requirement constraints are (for all bin types):

        wood requires plastic

\item Certain component types cannot coexist:

\begin{itemize}
\item        glass and copper exclude each other
\item        copper and plastic exclude each other
\end{itemize}

\item and certain bin types have capacity constraint for certain
components

\begin{itemize}
\item red   contains at most 1 wood item
\item green contains at most 2 wood items
\end{itemize}

\item Given an initial supply of:
\begin{itemize}
\item 1 glass item
\item 2 plastic items
\item 1 steel items
\item 3 wood items
\item 2 copper items
\end{itemize}
what is the minimum total number of bins required to
contain the components?
\end{itemize}

\subsection{Problem Model - Using Structures}

For modelling this problem we need to refer to an array of quantities
of glass items, plastic items, steel items, wood items and copper
items.
We therefore introduce a structure to hold this array:
\begin{verbatim}
:- local struct(contents(glass,plastic,steel,wood,copper))
\end{verbatim}

To represent a bin, with its colour, capacity and contents we use
another structure:
\begin{verbatim}
:- local struct(bin(colour,capacity,contents:contents))
\end{verbatim}
The {\bf contents} attribute of {\bf bin} is itself a {\bf contents}
structure.

The predicate {\bf solve_bin/2} is the general predicate
that takes an amount of components packed into a {\bf contents}
structure and it returns the solution.
\begin{quote}
\begin{verbatim}
?- Demand = contents\{glass:1,plastic:2,steel:1,wood:3,copper:2\},
   solve_bin(Demand, Bins).
\end{verbatim}
\end{quote}

\subsection{ Handling an Unknown Number of Bins}

{\bf solve_bin/2}  calls {\bf bin\_setup/2} to
generate a list {\bf Bins}.
It adds some redundant constraint to remove symmetries (two
symmetrical solutions are
the same, but with the bins in a different order).
Finally it labels all decision variables in the problem.
\begin{quote}
\begin{verbatim}
solve_bin(Demand, Bins) :-
    bin_setup(Demand,Bins),
    remove_symmetry(Bins),
    bin_label(Bins).
\end{verbatim}
\end{quote}

The usual pattern for solving finite domain problems is to state
constraints on a set of variables, and then label them.
However, because the number of bins needed is not known initially, it
is awkward to model the problem with a fixed set of variables.

One possibility would be to take a fixed, large enough, number of bins
and to try to find a minimum number of non-empty bins.
However, for efficiency, we choose to solve a sequence of problems,
each one with a - larger - fixed number of bins,
until a solution is found.

The predicate {\bf bin_setup/2} to generate a list of bins with appropriate
constraints works as follows.
First it tries to match the (remaining) demand with zero,
and use no (further) bins.
If this fails, a new bin is added to the bin list;
appropriate constraints are imposed on all the new bin's
variables;
its contents are subtracted from the demand;
and the {\bf bin_setup/2} predicate calls itself recursively:

\begin{quote}
\begin{verbatim}
bin_setup(Demand,[]) :-
        all_zeroes(Demand).
bin_setup(Demand,[Bin|Bins]) :-
        constrain_bin(Bin),
        reduce_demand(Demand,Bin,RemainingDemand),
        bin_setup(RemainingDemand,Bins).

all_zeroes(
           contents\{glass:0,plastic:0,wood:0,steel:0,copper:0\}
          ).

reduce_demand(
              contents\{glass:G,plastic:P,wood:W,steel:S,copper:C\},
              bin\{glass:BG,plastic:BP,wood:BW,steel:BS,copper:BC\},
              contents\{glass:RG,plastic:RP,wood:RW,steel:RS,copper:RC\}
             ) :-
       RG #= G - BG,
       RP #= P - BP,
       RW #= W - BW,
       RS #= S - BS,
       RC #= C - BC.

\end{verbatim}
\end{quote}

\subsection{Constraints on a Single Bin}

The constraints imposed on a single bin correspond exactly to the
problem statement:
\begin{quote}
\begin{verbatim}
constrain_bin(bin\{colour:Col,capacity:Cap,contents:C\}) :-
        colour_capacity_constraint(Col,Cap),
        capacity_constraint(Cap,C),
        contents_constraints(C),
        colour_constraints(Col,C).
\end{verbatim}
\end{quote}

\paragraph{colour\_capacity\_constraint}
The colour capacity constraint relates the colour of the bin to its
capacity.  It uses generalised propagation to apply
arc-consistency propagation.
\begin{quote}
\begin{verbatim}
colour_capacity_constraint(Col,Cap) :-
	capacity(Col,Cap) infers ac.

capacity(blue, 1).
capacity(green,4).
capacity(red,  3).
\end{verbatim}
\end{quote}
At the end of this section we will present another way to implement the
colour\_capacity\_constraint, using the explicit "glass-box"
functionality of fd.

\paragraph{capacity\_constraint}
The capacity constraint states:
\begin{itemize}
\item that the number of items of each
kind in the bin is non-negative,
\item
their sum does not exceed the capacity of the bin,
\item and the bin is non-empty (an empty bin serves no purpose)
\end{itemize}

\begin{quote}
\begin{verbatim}
capacity_constraint(Cap, contents\{glass:G,
                                   plastic:P,
                                   steel:S,
                                   wood:W,
                                   copper:C\}) :-
        G #>= 0, P #>= 0, S #>= 0, W #>= 0, C #>= 0,
        Cap #>= G+P+W+S+C,
        G+P+W+S+C #> 0.
\end{verbatim}
\end{quote}

\paragraph{contents\_constraints}
The contents_constraints directly enforce the restrictions on items in
the bin: wood requires paper, glass and copper exclude each other, and
copper and plastic exclude each other:
\begin{quote}
\begin{verbatim}
contents_constraints(contents\{glass:G,plastic:P,wood:W,copper:C\}) :-
        requires(W,P),
        exclusive(G,C),
        exclusive(C,P).
\end{verbatim}
\end{quote}

These constraints are expressed as logical combinations of constraints
on the number of items.
"requires" is expressed using entailment, \verb0#=>0.
"Wood requires paper" is expressed in logic as "If the number of wood
items is greater than zero, then the number of paper items
is also greater than zero":
\begin{quote}
\begin{verbatim}
requires(W,P) :-
        W #> 0 #=> P #> 0.
\end{verbatim}
\end{quote}

Exclusion is expressed using disjunction, \verb0#\/0.
"X and Y are exclusive" is expressed as "Either the number of items of
kind $X$ is zero, or the number of items of kind $Y$ is zero":
\begin{quote}
\begin{verbatim}
exclusive(X,Y) :-
        X #= 0 #\/ Y #= 0.
\end{verbatim}
\end{quote}

\paragraph{colour\_constraints}
The colour constraint limits the number of wooden items in bins of
different colours.
Like the capacity constraint, the relation between the colour and
capacity ($WCap$ is expressed using generalised propagation to enforce
arc-consistency.  The number of wooden items is then constrained not to
exceed the capacity:
\begin{quote}
\begin{verbatim}
colour_constraints(Col,contents\{wood:W\}) :-
        colour_wood_cap(Col,WCap) infers ac,
        WCap #>= W.

colour_wood_cap(blue, Cap) :- capacity(blue, Cap).
colour_wood_cap(green,2).
colour_wood_cap(red,1).
\end{verbatim}
\end{quote}

This model artificially introduces a capacity of blue bins for
wood items (set simply at its maximum capacity for all items).

\subsection{Symmetry Constraints}
To make sure two solutions are not just different permutations of the
same bins, we fix the sequence of variables specifying each bin, and
lexically order their values.

\begin{quote}
\begin{verbatim}
remove_symmetry(Bins) :-
        ( fromto(Bins,[B1,B2|Rest],[B2|Rest],[_Last])
        do
            lex_ord(B1,B2)
        ).
\end{verbatim}
\end{quote}

Since colours don't have a built-in ordering, we map them to integers
and apply the lexicographic ordering to the integers instead of the
colours.  Since the mapping is defined before values are chosen for the
variables, the mapping is turned into a constraint, using generalised
propagation.
\begin{quote}
\begin{verbatim}
lex_ord(bin\{colour:Col1,contents:Conts1\},
        bin\{colour:Col2,contents:Conts2\}) :-
        colour_map(Col1,Int1) infers ac,
        colour_map(Col2,Int2) infers ac,
        term_variables(Conts1,Vars1),
        term_variables(Conts2,Vars2),
        lexico_le([Int1|Vars1],[Int2|Vars2]).

colour_map(blue,1).
colour_map(green,2).
colour_map(red,3).
\end{verbatim}
\end{quote}
(Another way to associate integers with colours is
described at the end of this section.)

Though lexicographical ordering is defined in the fd\_global library of
{ \eclipse }, Warwick
Harvey has written a very elegant version that enforces arc-consistency,
using a few features of the fd library.  Here it is:

\begin{quote}
\begin{verbatim}
lexico_le(Xs, Ys) :-
        lexico_le_bool(Xs, Ys, 1).

lexico_le_bool([], [], 1).
lexico_le_bool([X | Xs], [Y | Ys], B) :-
        B isd (X #< Y + B1),
        lexico_le_bool(Xs, Ys, B1).
\end{verbatim}
\end{quote}

\subsection{Search}

The search is done by first choosing a colour for each bin, and then
labelling the remaining variables.
\begin{quote}
\begin{verbatim}
bin_label(Bins) :-
        ( foreach(bin\{colour:C\},Bins) do indomain(C) ),
        term_variables(Bins,Vars),
        labeleff(Vars).
\end{verbatim}
\end{quote}

The remaining variables are labelled using the first fail heuristic
(using the fd built-in \verb0deleteff0).  This code illustrates the use
of \verb0fromto0 for dynamically ordering the variables to be labelled.
\begin{quote}
\begin{verbatim}
labeleff(Vars) :-
        ( fromto(Vars,InVars,OutVars,[])
        do
            deleteff(Var,InVars,OutVars),
            indomain(Var)
        ).
\end{verbatim}
\end{quote}

\subsection{Glass-Box Implementation of colour\_capacity\_constraint}
To illustrate how the facilities of fd can be used to directly encode
constraint behaviour, we have reimplemented the
colour\_capacity\_constraint without using generalised propagation.

The first requirement is to ensure the capacity variable has a finite
domain.  One way to do this is as follows:
\begin{quote}
\begin{verbatim}
colour_capacity_constraint(Col,Cap) :-
    ( is_domain(Cap) -> true ; Cap #>= 0 ),
    col_cap_cons(Col,Cap).
\end{verbatim}
\end{quote}

If the colour $Col$ is already known, the constraint is enforced simply by
invoking the capacity predicate.  If, however, the colour $Col$ is a
variable, then the more complex var\_col\_cap predicate is called:

\begin{quote}
\begin{verbatim}
col_cap_cons(Col,Cap) :-
    ( nonvar(Col) -> capacity(Col,Cap) ;
      var(Col) -> var_col_cap(Col,Cap)
    ).
\end{verbatim}
\end{quote}

If the required capacity (i.e. the minimum value in the domain of $Cap$)
is more than 1, then the colour cannot be blue.
If, moreover, the required capacity is more than 3, then the colour must
be green.
If the required capacity is either 2 or 3, then any change to the $Col$
or $Cap$ variables must instantiate them, because each variable has only
two
possible values).  Since there is a unique
colour for each capacity, as well as vice versa, it suffices to call the
capacity predicate as soon as either variable is instantiated.
If, however the minimum capacity is still 1, then the col\_cap\_cons
constraint is suspended again, until the colour is instantiated
\verb0Col->inst0 or the required
capacity increases \verb0Cap->min0:
\begin{quote}
\begin{verbatim}
var_col_cap(Col, Cap) :-
    mindomain(Cap,MinC),
    (MinC > 1 ->
        Col #\= blue,
        (MinC > 3 ->
            Col = green
        ;
            suspend(capacity(Col, Cap), 3, (Col, Cap)->inst)
        )
    ;
        suspend(col_cap_cons(Col, Cap), 3, [Col->inst, Cap->min])
    ).

\end{verbatim}
\end{quote}

\subsection{Using Macros instead of a Colour-Integer Mapping}

As we discussed in the section on symmetries, above, there is no
built-in ordering on colours.

To keep the lexicographic ordering predicate simple and still have a
symbolic
representation of the colour in the program, we can define
input macros that transform the colour atoms into integers:

\begin{quote}
\begin{verbatim}
:- define_macro(no_macro_expansion(blue)/0, tr_col/2, []).
:- define_macro(no_macro_expansion(green)/0, tr_col/2, []).
:- define_macro(no_macro_expansion(red)/0, tr_col/2, []).

tr_col(no_macro_expansion(red), 1).
tr_col(no_macro_expansion(green), 2).
tr_col(no_macro_expansion(blue), 3).

\end{verbatim}
\end{quote}

Now the symmetry removal can be programmed using the following
lexical ordering predicate:

\begin{quote}
\begin{verbatim}
lex_ord(Bin1,Bin2) :-
       term_variables(Bin1,V1),
       term_variables(Bin2,V2),
       lexico_le(V1,V2).
\end{verbatim}
\end{quote}


%!TEX root = paper.tex
\subsection{Nonlinear shoaling wave over submerged bar}
In this laboratory experiment an incident sin wave travels over a trapezoidal underwater obstacle and the surface elevations of the non-breaking wave are measured at several gauges spread along the obstacle.
This experiment conducted in different versions by \cite{BejiBattjes.1993} and \cite{BejiBattjes.1994}, \cite{Dingemans.1994} (more cases, here only case A considered), and by Luth et al. (1994) (cited in \cite{Dingemans.1994}, but I do not have access to the report).
In this experiments, gauges and setups were different (see tables \ref{tab:bejibattjes_gauges_SZ} -- \ref{tab:bejibattjes_gauges_BB1994} and figures \ref{fig:bejibattjes_setup_SZ} -- \ref{fig:bejibattjes_setup_BB1994}).

The setup is as described in \cite{StellingZijlema.2003}, and M. Zijlema provided the data. According to the gauges, this should be the data described in \cite{Dingemans.1994, StellingZijlema.2003} as the data from Luth et al. (1994), but scaled (divided) by a factor of 2.

The initial condition is the unperturbed state. The boundary condition is described as in incident wave at the left boundary of the computational domain, where the surface elevation is set to $\xi=a \text{sin}\left(\frac{2\pi t}{T}\right)$ with period $T=2.02 \,$s and amplitude $a=1.0\,$cm.
We impose reflecting boundary conditions at the boundary in x-direction and periodic boundary conditions in y-direction. For the setup see figure \ref{fig:bejibattjes_setup_SZ}. The computational domain is enlarged to 40m to ensure no reflecting waves disturbing the solution during the simulation time of 40s.

Note: No boundary conditions are specified for w and q at the moment, computations which set them to zero are running. This hopefully explains the wrong behavior.

\begin{table}[htbp]
\begin{tabular}{lllllllll}
\textbf{gauge} & G4 & G5 & G6 & G7 & G8 & G9 & G10 & G11 \\
\toprule
\textbf{x [m]} & 10.5 & 12.5 & 13.5 & 14.5 & 15.7 & 17.3 & 19.9 & 21.0 \\
\bottomrule
\end{tabular}
\caption{Locations of gauges used in \cite{StellingZijlema.2003}}
\label{tab:bejibattjes_gauges_SZ}
\end{table}

\begin{table}[htbp]
\begin{tabular}{lllllllll}
\textbf{gauge} & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 \\
\toprule
\textbf{x [m]} & 6.0 & 11.0 & 12.0 & 13.0 & 14.0 & 15.0 & 16.0 & 17.0 \\
\bottomrule
\end{tabular}
\caption{Locations of gauges used in \cite{BejiBattjes.1993}}
\label{tab:bejibattjes_gauges_BB1993}
\end{table}

\begin{table}[htbp]
\begin{tabular}{llllllll}
\textbf{gauge} & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
\toprule
\textbf{x [m]} & 6 & 10.8 & 12.8 & 13.8 & 14.8 & 16.0 & 17.6 \\
\bottomrule
\end{tabular}
\caption{Locations of gauges used in \cite{BejiBattjes.1994}}
\label{tab:bejibattjes_gauges_BB1994}
\end{table}

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{bejibattjes_setup_SZ}
\caption{Setup of the experiment of Beji and Battjes according to \cite{StellingZijlema.2003}}
\label{fig:bejibattjes_setup_SZ}
\end{figure}

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{bejibattjes_setup_BB1993}
\caption{Setup of the experiment of Beji and Battjes according to \cite{BejiBattjes.1993}}
\label{fig:bejibattjes_setup_BB1993}
\end{figure}

\begin{figure}[htbp]
\includegraphics[width=\textwidth]{bejibattjes_setup_BB1994}
\caption{Setup of the experiment of Beji and Battjes according to \cite{BejiBattjes.1994}}
\label{fig:bejibattjes_setup_BB1994}
\end{figure}

\subsubsection{Results of \nh\ model}
The model results can be found in figure \eqref{fig:nh_bejjibattjes_nh}.
The results are shifted in time by 4.5s to match the laboratory data at gauge 10 in phase after 20s.

\begin{figure}[htbp]
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_4}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_5}
\end{minipage} \\
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_6}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_7}
\end{minipage} \\
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_8}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_9}
\end{minipage} \\
\begin{minipage}{\textwidth}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_10}
\includegraphics[width=0.48\textwidth]{BejiBattjes_nh_x=loc_11}
\end{minipage}
\caption{Comparison of the laboratory (black) sea surface height at gauges with the simulation results of the \nh\ model with linear pressure profile (yellow) and quadratic pressure profile (red)}
\label{fig:nh_bejjibattjes_nh}
\end{figure}


\chapter{Baseline} \label{baseline}

\section{Syntax}
\note{
it's the description of the syntax of the programs in the search space as well as how to derive programs from programs.
}

\note{Rewrite this without mixing syntax with semantics and with the baseline algorithm}
In the context $\Gamma$ we store the bindings from library components and input variables to their types.

In the context $\Delta$ we store the bindings from active and inactive holes to their types.

Terms are applications of library components or input variables (denoted by $x$ in the grammar), values (denoted by $v$) and holes ($?x$ and $?\underline{v}$).

We distinguish between active holes $?x$, that can be substituted with other programs, from inactive holes $?\underline{v}$, that can only be replaced with values. To determine the exact value, a heuristic using a mix of symbolic execution and brute force is applied later. Every active hole having a value type (denoted by $V$ in the grammar) can be turned into an inactive hole.

Programs without active holes are considered \emph{closed}. Those are the programs in which we plug in the input and execute as far as possible to see whether it agrees with the expected output.

Values are integer constants (denoted by $n$ for brevity), lists of values and tuples of values. \note{Does it make sense to have a special treatment for list values and tuples? Wouldn't it be better to use library components to construct them?}

We use a standard type system featuring functional and universal types, as well as integers, lists and tuples. We distinguish between two kinds of type variables: $?X$ are the type variables that can be instantiated with other types when it comes to unification, $X$ are the type variables that are already fixed by the goal type and have to remain like that. Universal types are assumed to be used like in Haskell only as outer wrapping of the types of library components and the goal type specified by the user.

\TODO{Find a way to format t t as t t and not a tt, same for VV and TT}
\TODO{Find a way to format it vertically and adding comments to each line. For example, write "inactive hole" on the right of $?\underline{v}$.}
\note{Maybe it's better to write $\Gamma$ and $\Delta$ as sets and not as grammars, as you are using them as sets. If you leave it like this, $\Delta \setminus \{?x : T\}$ would be undefined.}
\begin{plstx}
(library components): \Gamma ::= \emptyset | \Gamma \cup x : T \\
(holes): \Delta ::= \emptyset | \Delta \cup ?x : T | \Delta \cup ?\underline{v} : T \\
(terms): t ::= v | ?\underline{v} | ?x | x | t t \\
(values): v ::= n | v : v | [] | (v,v) \\
(types): T ::= \text{Int} | \text{List } T | \text{Tuple } T T | T \rightarrow T | \forall X. T | X | ?X \\
(value types): V ::= \text{Int} | \text{List } V | \text{Tuple } V V \\
\end{plstx}

\note{It may appropriate to move this to the next section}
We assume that the goal program is normalized in the following sense. If the goal type specified by the user is a universal type, then we substitute each universally quantified type variable with a fresh fixed type variable $X$. If the goal type is a function type, then we abstract the type as much as possible and add input variables to the context $\Gamma$.
For example, if the user specifies the goal program as
\begin{lstlisting}[mathescape]
length :: $\forall X$. List $X$ -> Int
length [1,2,3] == 3
length [2,2,2] == 3
length [] == 0
length [5] == 1
\end{lstlisting}
then we start our search from the partial program $\Gamma \cup \{xs : \texttt{List } X \} \vdash ?x :: \texttt{Int}$ where $\Gamma$ already contains all bindings from library components to their types. Note that it must be possible to instantiate the type variable $X$ with every possible type, therefore we do not have the right to prefer one instantiation over another and must treat $X$ as an uninterpreted type.

\TODO{Add the rules of the typing judgement}
The typing judgement is standard. To define the search graph we are going to explore in order to find the goal program, we also need the \emph{derive} judgement, which says between which nodes of the graph (programs of the form $\Gamma, \Delta \vdash t :: T$) there is an edge.
To express the rules in a more compact form, we introduce \emph{evaluation contexts}. An context is an expression with exactly one syntactic hole $[]$ in which we can plug in any term. For example, if we have the context $\mathcal{E}$ we can place the term $t$ into its hole and denote this new term by $\mathcal{E}[t]$.

\TODO{Make a figure of all rules}
We can turn an active hole into an inactive hole if the active hole has a value type.
\note{do we need a premise? Wouldn't it be enough to write ?x :: V below the line?}
%D-VarVal, if we can turn an active hole into an inactive hole
\begin{prooftree}
\AxiomC{$\Gamma, \Delta \vdash ?x :: V$}
	\RightLabel{D-VarVal}
	\UnaryInfC{$\Gamma, \Delta \vdash ?x :: T \Mapsto \Gamma, \Delta \setminus \{?x : T\} \cup \{?\underline{v} : V\} \vdash ?\underline{v} :: V$}
\end{prooftree}

An active hole $?x$ can be turned into a library component or an input variable $x$ from the context $\Gamma$. The procedure fresh($T$) transforms universally quantified type variables into fresh type variables $?X$ not used in $\Delta$.
The notation $\sigma(\Delta)$ denotes the application of the substitution $\sigma$ to all types contained in the context $\Delta$.
%D-VarComp, an active hole can be turned into a library component or an input variable
\begin{prooftree}
\AxiomC{$x : T_x \in \Gamma$}
\AxiomC{$\sigma$ unifies $T$ with fresh($T_x$)}
	\RightLabel{D-VarLib}
	\BinaryInfC{$\Gamma, \Delta \vdash ?x :: T \Mapsto \Gamma, \sigma(\Delta \setminus \{?x : T\}) \vdash x :: \sigma(T) $}
\end{prooftree}

An active hole can also be turned into a function application of two new active holes.
%D-VarApp, we can turn one hole into two holes :)
\begin{prooftree}
\AxiomC{$?X$ is a fresh type variable}
	\RightLabel{D-VarApp}
	\UnaryInfC{$\Gamma, \Delta \vdash ?x :: T \Mapsto \Gamma, \Delta \setminus \{?x:T\} \cup \{?x_1 : ?X \rightarrow T, ?x_2 : ?X\} \vdash ?x_1$ $?x_2 ::T$}
\end{prooftree}

In all other cases we just choose an active hole and expand it according to the three rules above.
%D-App, for all other cases
\begin{prooftree}
\AxiomC{$\Gamma, \Delta \vdash ?x :: T_1 \Mapsto \Gamma, \Delta' \vdash t_1' :: T_1'$}
	\RightLabel{D-App}
	\UnaryInfC{$\Gamma, \Delta \vdash t[?x] :: T \Mapsto \Gamma, \Delta' \vdash t[t_1] :: T[T_1/T_1']$}
\end{prooftree}

\section{Baseline Algorithm}
\note{
The breadth (or best) first search of the search graph, nothing fancy.
}

The baseline algorithm is a simple best first search implemented using a priority queue. One possibility is to order the enqueued elements according to the number of library components and active holes.

The root node has the form $\Gamma, \{?x:T\} \vdash ?x :: T$ where $T$ is neither functional nor universal type and $\Gamma$ contains the input variables along with the library components.
The successors of a node are the nodes reachable in one step of the derive judgement. That is, $\Gamma, \Delta' \vdash t' :: T'$ is a successor of $\Gamma, \Delta \vdash t :: T$ if it holds $\Gamma, \Delta \vdash t :: T \Mapsto \Gamma, \Delta' \vdash t' :: T'$.

We don't need to explore nodes that are equivalent up to alpha conversion to already visited nodes.

A term is considered \emph{closed} if it does not contain active holes. Closed terms are tested on the input-output examples. Programs with inactive holes are symbolically executed on the input-output examples and, if possible, the concrete value of the inactive hole is determined. Otherwise we try to solve it by brute force with a small timeout.
\note{Does it make sense in terms of performance? Usually the values we need are really simple like \lstinline?[], 0, 1?. Wouldn't it be more efficient to try some simple values first?}

The input-output examples are given as a vector of inputs \lstinline?I? and the vector of corresponding expected outputs \lstinline?O?.

\TODO{Typeset in a nicer way, for example with a vertical line instead of all those 'end' and nicer keyword formatting.}
\begin{lstlisting}[mathescape]
BFS(root, I, O)
	queue $\leftarrow$ {root}
	visited $\leftarrow$ {}
	while($\text{timeout not reached}$)
		current $\leftarrow$ queue.dequeue
		if (current.closed)
			if (current.test(I) == O)
				return current
			end
		else
			for (s in current.successors)
				if (!visited.alphacontains(s))
					queue $\leftarrow$ queue.push(s)
				end
			end
		end
	end
end
\end{lstlisting}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "thesis"
%%% End:


% !TEX TS-program = lualatex
\documentclass{article}

%\usepackage{fontspec}

\begin{document}

\section{Introduction}
\label{Intro}

Therefore we still aim to compute the traces in the drift term exactly. For this we note that the derivative in the trace is an extremely sparse matrix only containing two nonzero $3\times 3$ blocks located at block-entries $(i,i+\hat d)$ and $(i+\hat d,i)$. On multiplication with $D^{-1}$ the product matrix will be sparse, existing of 2 full column-blocks at $(k,i)$ and $(k,i+\hat d$), $k=1,\ldots,V$ (a column block has size $3V\times 3$). To compute such a trace the same columns of $D^{-1}$ are needed. Hence, it looks as if the complete matrix $D^{-1}$ is needed when computing all traces. However, the situation is not as dramatic as it seems. Indeed, on taking the trace of the product matrix only the \textit{diagonal elements} of the product is needed. To compute these diagonal elements we do not need the complete columns of $D^{-1}$ but only the $3\times 3$ blocks located at the transposed positions of the $3\times 3$ derivative blocks $\partial D/\partial a$. For example, the derivative block at location $(i+\hat d,i)$ will only give a contribution to the trace when multiplied with the $3\times 3$ block of $D^{-1}$ located at $(i, i+\hat d)$.

Therefore we still aim to compute the traces in the drift term exactly.
For this we note that the derivative in the trace is an extremely sparse matrix only containing two nonzero $3\times 3$ blocks located at block-entries $(i,i+\hat d)$ and $(i+\hat d,i)$.
On multiplication with $D^{-1}$ the product matrix will be sparse, existing of 2 full column-blocks at $(k,i)$ and $(k,i+\hat d$), $k=1,\ldots,V$ (a column block has size $3V\times 3$).
To compute such a trace the same columns of $D^{-1}$ are needed. Hence, it looks as if the complete matrix $D^{-1}$ is needed when computing all traces.
 However, the situation is not as dramatic as it seems.
 Indeed, on taking the trace of the product matrix only the \textit{diagonal elements} of the product is needed.
 To compute these diagonal elements we do not need the complete columns of $D^{-1}$ but only the $3\times 3$ blocks located at the transposed positions of the $3\times 3$ derivative blocks $\partial D/\partial a$.
 For example, the derivative block at location $(i+\hat d,i)$ will only give a contribution to the trace when multiplied with the $3\times 3$ block of $D^{-1}$ located at $(i, i+\hat d)$.

\end{document}


%!TEX root = paper.tex
%
% AMRClaw
%
% Lead currently:  Marsha Berger
%

\subsection{\amrclaw} \label{sec:amrclaw}
Fortran code in the \amrclaw repository performs block-structured adaptive mesh
refinement \cite{BO,BC} for both \clawpack and \geoclaw  applications.
The algorithms implemented in \amrclaw are discussed in detail in
\cite{mjb-rjl:amrclaw,LeVequeGeorgeBerger:an11}, but a  short description is
given here to set the stage for a description of recent changes. \revised{
This type of refinement solves the PDE on a hierarchy of logically rectangular
grids.  One (or more) level 1 grids comprise the entire domain, while grids at
finer level are created and destroyed (as opposed to moving these grids) to
follow important features in the solution.}

\begin{figure}[tb]
    \centering
    \includegraphics[width=0.6\textwidth, trim=40 80 40 70, clip]{final_submission/Fig1.png}
    \caption{\revised{An illustration showing grid cells on levels one and two, and only grid outlines on levels three and four.}}
    \label{fig:figure1}
\end{figure}

\amrclaw includes the functionality for:
\begin{itemize}
\item Coordinating the flagging of points where refinement is needed,
with a variety of criteria possible for flagging cells that need refinement
from each level to the next finer level (including Richardson extrapolation,
gradient testing, or user-specified criteria)\footnote{See
\url{http://www.clawpack.org/flag.html}},
\item Organizing the flagged points into efficient grid
patches at the next finer level, using the algorithm of
\cite{mjb-rig:cluster},
\item Interpolating the solution to newly created fine grids and initializing
auxiliary data (topography, wind velocity, metric data and so on) on  these
grids,
\item Averaging fine grid solutions to coarser grids,
\item Orchestrating the adaptive time stepping (i.e. sub-cycling in time),
\item Interpolating coarse grid solution to fine grid ghost cells, and
\item Maintaining conservation at patch boundaries between resolution levels.
\end{itemize}

\amrclaw now allows users to specify ``regions'' in space-time
$[x_1,x_2] \times [y_1,y_2] \times [t_1,t_2]$ in which refinement is forced to
be at least at some level $L_1$ and is allowed to be at most $L_2$.  This can be
useful for constraining refinement, e.g. allowing or ensuring resolution of only
a small coastal region in a global tsunami simulation. Previously the user could
enforce such conditions by writing a custom flagging routine, but now this is
handled in a general manner so that the parameters above can all be specified in
the Python problem specification. Multiple regions can be specified, and a
simple rule is used to determine the constraints at a grid cell that lies in
multiple regions.

Auxiliary arrays are often used in \clawpack to store data that
describes the problem and the routine.
The routine \texttt{setaux} must then be provided by the user to set these values each time a
new grid patch is created.  For some applications computing these values can be time-consuming.  In \clawpack 5.2,
this code was improved to allow reuse of values from previous patches at
the same level where possible at each regridding time.
This is backward compatible, since no harm is done if previously
written routines are used that still compute and overwrite instead of
checking a mask.

In \clawpack 5.3 the capability to specify spatially varying boundary
conditions was added. For a single grid, it is a simple matter to
compute the location of the ghost cells that extend outside the
computational domain and set them appropriately.  With AMR however,
the boundary condition routine can be called for a grid located
anywhere in the domain, and may contain fewer or larger numbers of
ghost cells. For this reason, the boundary condition routines
 do not assume a fixed number of ghost cells.

Anisotropic refinement is allowed in both two and three dimensions.
This means that the spatial and temporal refinement ratios can be
specified independently from one another (as long as the temporal
refinement satisfies the CFL condition).  In addition, capabilities
have been added to automatically select the refinement ratio in time  on each
level based on the CFL condition.  This has only been implemented in
\geoclaw where the wave speed in the shallow water equations
depends on the local depth. The finest grids are often located only in
shallow coastal regions, so a large refinement ratio in space does not
lead to a large refinement ratio in time.

\amrclaw has been parallelized using OpenMP directives. \revised{The main
paradigm in structured AMR is an outer loop over levels of refinement, and an
inner loop over all grids at that level, where \revised{the same} operation is
performed on each grid (i.e. taking a time step, finding ghost cells,
conservation updates, etc.).  This inner loop is parallelized using a {\tt
parallel for} loop construct.  One thread is assigned to operate on one grid.
Dynamic scheduling is used with a chunk size of one.  To help with load
balancing, grids at each level are sorted from largest to smallest, using the
total number of cells in the grid as an indicator of work.  In addition the
grids are limited to a maximum of 32 cells in each dimension, otherwise they
are bisected until this condition is met.}  Note that this approach causes a
memory bulge. Each thread must have its own scratch arrays to save the incoming
and outgoing waves and fluxes for future conservation fix-ups. The bulge is
directly proportional to the number of threads executing. For stack-based memory
allocation per thread, the use of the environment variable {\tt OMP\_STACKSIZE}
to increase the limit may be necessary.

\cref{fig:shockbubble} shows two snapshots of the solution to a
three-dimensional shock-bubble interaction problem found in the \clawpack
\texttt{apps} repository, illustrating localized phenomena requiring adaptive
refinement. In \cref{fig:amr_scaling} we show scalability tests and some timings
for this example, when run on a 24 core Intel Xeon Haswell machine
\revised{(E5-2670v3 at 2.3 GHz)}, using
\texttt{KMP\_AFFINITY compact} \revised{with one thread per core}.
For timing purposes, the only modifications made to the input parameters was to
turn off check-pointing and graphics output. The plot on the left shows that most
of the wall clock time is in the integration routine (\texttt{stepgrid}), which
closely tracks the total time. The second chunk of time is in the regridding,
which contains algorithms that are not completely scalable. Very little time is
in the filling of ghost cells, mostly from other patches but also includes those
at domain boundaries. The efficiency is above 80\% until 24 cores.
Note that there are only two grids on level, and an average of
22.8 level 2 grids. Most of the work is on level 3 grids, where there are an
average of 138.1 grids over all the level 3 timestep.  At 24 cores, there are
on average 5.8 grids per core, and the grids are very different sizes.

\begin{figure}[t]
  \begin{center}
    \includegraphics[width=\textwidth]{figures/final_submission/Fig2}
    % \plotbox{\includegraphics[width=0.45\textwidth]{f1.png}}\hfil
    % \plotbox{\includegraphics[width=0.45\textwidth]{f4.png}}
  \end{center}
\caption{\amrclaw example demonstrating a
  shock-bubble interaction in the Euler equations of compressible
  gas-dynamics at two times, illustrating the need for adaptive refinement
  to capture localized behavior. \revised{There are two $20 \times 10 \times 10$
  grids at
  level 1.  They are refined} where needed
  by factors of 4 and then 2 in this 3-level run.}
\label{fig:shockbubble}
\end{figure}

\begin{figure}[h]
  \begin{center}
    % \includegraphics[width=0.45\textwidth]{amr_scaling}
    % \plotbox{\includegraphics[width=0.4\textwidth,
    %   clip=true,trim=0cm 1cm 1cm 2cm]{newcpu.pdf}}\hfil
    % \plotbox{\includegraphics[width=0.4\textwidth,
    %   clip=true,trim=0cm 1cm 1cm 2cm]{efficiency.pdf}}
    \includegraphics[width=\textwidth]{final_submission/Fig3.pdf}
  \end{center}
  \caption{(a) Strong scaling results for the \amrclaw example shown in
    \cref{fig:shockbubble}.
    (b) Plot of efficiency based on total computational time.}
  \label{fig:amr_scaling}
\end{figure}

\revised{The target architecture for \amrclaw and \geoclaw are multi-core
machines.  \pyclaw on the other hand scales to tens of
thousands of cores using MPI via PETSc \cite{petsc-user-ref} but is not
adaptive.}


\chapter{Experiments}\label{chap:experiments}
A series of experiments were carried out to determine the performance of the network in its stated goal of classifying traversability. To this end, the network was tested on the Mapillary Vistas dataset \cite{mapillary} and its hyperparameters optimized using the Bayesian optimization with hyperband (BOHB) procedure\cite{bohb}

\section{Evaluation Metrics} \label{section:experiments-evaluationmetrics}
The mean Intersection over Union (mIoU) is the main evaluation metric chosen.
IoU measures the intersection of the prediction and the ground truth data, divided by the union of the prediction and the ground truth data.
This is formally defined as
\begin{align}
	J(A_m,B_m) &= \frac{|A_m \cap B_m|}{|A_m \cup B_m|}
\end{align}
where $A$ is the prediction, $B$ the ground truth, and $m$ the class.
The mIoU is thus defined as
\begin{align}
	J(A,B) &= \frac{1}{|m|} \sum_{m}\frac{|A_m \cap B_m|}{|A_m \cup B_m|}
\end{align}
and can be intuitively understood as the average of the class-wise IoU accuracy values.

\section{Dataset} \label{section:experiments-dataset}
We evaluate our approach on the Mapillary Vistas dataset as it was, to our knowledge, the only street-level dataset that included ground truth segmentations of both curbs and curb cuts at the time of writing.
This dataset contains images from all around the world and includes images captured from different imaging devices including mobile phones, action cameras, and professional imaging solutions.
These images are captured during various weather, seasonal, and daylight conditions.
As such, the dataset provides a challenging level of diversity.

Some image samples from the dataset can be seen in \figref{fig:experiments-datasetsamples}.
These images show some examples of how diverse the images in the dataset can be.
The dataset also contains images taken from sidewalks, such as in the case of Figure \ref{fig:dataset-3}.
This is especially useful for our end goal of training a segmentation network for an autonomous pedestrian robot.
The diversity of the dataset improves the generalizability of our model.

\input{figures/experiments/dataset}

The image dimensions in the dataset are also not standardized, with 156 different image dimensions ranging from $(640 \times 480)$ to $(6528 \times 5248)$.
With such a diverse range of dimensions, the images must first be preprocessed.
We process all the images to conform to a 4:3 size ratio, eliminate images without images of curbs or curb cuts, and extract only road, curb, and curb cut classes.

To optimize training speed with respect to wall clock time, and given the computational constraints, images were resized for training.
The chosen image resolution was $360 \times 320$ pixels.
We use an NVIDIA Titan X GPU with 12 GB of VRAM.
This allows a batch size of 16 per GPU.
To further optimize training, the images are resized prior to training and the resized images stored.
This allows all training and validation images to be loaded into memory once at the beginning of training, reducing the number of file accesses required and reducing training wall clock time by a factor of two.

After running an analysis of the dataset with respect to their curb and curb cut content and image dimensions, we found that there were 15,160 usable images in the training set and 1610 usable images in the validation set.
Usable images in this case refers to images that at least contain both curbs and curb cuts.
A detailed results of the analysis of the dataset can be found in Appendix section \ref{appendix:dataset}.

We also applied image augmentation to our dataset.
The augmentations we have applied are Gaussian blur, dropout, brightness adjustment, and contrast adjustment.

\section{Network Evaluation}\label{section:experiments-networkevaluation}
Experiments were done to evaluate the performance of different networks on our dataset.
Specifically, we identified the small size of curbs relative to the rest of the image causes a severe class imbalance and increases the difficulty in recognizing the smaller features.

Relative to the entire image, curb and curb cut classes make up a relatively small proportion of the image, making up an average of 0.986\% and 0.196\% of images respectively.
As such, three networks with good known performance for urban scene segmentation were chosen and evaluated for their performance classifying traversability classes.
We experimented with GoogLeNet, described in the paper "Going Deeper with Convolutions"~\cite{googlenet}; FCN16s, described in "Fully Convolutional Networks for Semantic Segmentation"~\cite{fcn}; and DeepLab v3+, described in the paper "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation"~\cite{deeplab}.

To evaluate which network would have the most potential, we trained each network on a small subset of the whole dataset to find which networks would yield the highest overall accuracy given a certain budget.
We use a subset of 64 images, sampled randomly from the Mapillary Vistas training subset, for training and a subset of 64 images, also sampled randomly from the Mapillary Vistas validation subset, for validation.
For each of these runs, the default recommended hyperparameters from the DeepLab v3+ paper were used.
Due to time constraints, conduct hyperparameter optimization was not done on each of the tested networks.
The results can be seen in \figref{chart:experiments-networkcomparison} and in \tabref{tab:network-comparison}.

We observe that DeepLab v3+ starts with the lowest mIoU performance but quickly achieves 28.61\% mIoU accuracy after only 80 iterations, already outperforming the final result of GoogLeNet, after which training begins to plateau.
After 1000 iterations, it was able to achieve 30.83\% mIoU accuracy.
GoogLeNet is able to also train relatively quickly, achieving 22.05\% mIoU accuracy after 80 iterations, after which performance also began to plateau achieving a final 25.08\% mIoU accuracy.
FCN16s was able to achieve 19.97\% mIoU overall after 1000 iterations.
We believe this may be due to FCN16s not being designed to identify smaller structures and features, such as in the case of curbs and curb cuts.

Given these results, we chose to use DeepLab v3+ as our network.

\input{tables/experiments/network-comparison-table}
\input{figures/experiments/network-comparison-chart}

\section{Loss Function Evaluation}\label{section:experiments-loss}
Our custom loss function masked cross entropy (MCE) loss was evaluated against weighted cross entropy (WCE) loss.
We evaluated the two loss functions using the same network hyperparameters and a pretrained backbone.
The backbone is pretrained on the ImageNet database, published in 2009 by J. Deng et al~\cite{imagenet}.
Training results, shown in \figref{chart:experiments-losstraining}, show that both lost functions show similar results until the 40\textsuperscript{th} iteration, after which the network using MCE loss is able to produce better results.
Validation accuracy is plotted in \figref{chart:experiments-lossvalidation}.
We observe the validation accuracy to be similar for the first 2370 iterations.
After this point, MCE loss shows a clear advantage in validation accuracy compared to WCE loss.
After 4740 iterations, a clear performance advantage can be seen, with a difference of 5.56 mIoU percentage points.
As such, we have chosen to implement our network using MCE loss as our loss function.

\input{figures/experiments/losscomparison}

\section{Hyperparameters} \label{section:experiments-hyperparameters}
Hyperparameter tuning was done by using Bayesian Optimization with HyperBand.
The hyperparameters that were tuned were the learning rate; optimizer; momentum, if the optimizer chosen was stochastic gradient descent; epsilon, if the optimizer chosen was Adam; and the loss criterion.

The learning rate is optimized between the interval $\left[1 \times 10^{-5}, 1 \times 10^{-2}\right]$, varied on a logarithmic scale.
The optimizer is a categorical choice between stochastic gradient descent or Adam.
The momentum is optimized between the interval $\left[0, 0.99\right]$.
The epsilon value is optimized between the interval $\left[1 \times 10^{-2}, 1\right]$.
The weight ratio for the loss weight determines the ratio of the weights between the curb and curb cut.
The loss criterion is a categorical choice between weighted cross entropy loss or masked cross entropy loss.
Further details of the hyperparameters optimized can be found in \tabref{tab:hyperparameters}.

\input{tables/experiments/hyperparameters}

The hyperparameters are optimized according to the above parameters using a cluster of 4 GPUs over a budget of 4 iterations.
The resulting optimized values for the hyperparameters are in \tabref{tab:hyperparameterresults}.
More detailed results of each run of the hyperparameter optimization can be found in Appendix section \ref{appendix:hpoptimresults}.

\input{tables/experiments/hyperparameter-results}

\section{Implementation} \label{section:experiments-trainingpipeline}
The training pipeline is a main script which calls the training loop with a set of parameters defined in either a JSON file or through command-line arguments.
Using a JSON file makes it simpler to change parameters as nothing is hard-coded and command-line arguments do not have to be memorized or meticulously typed out each time.

A Graphical User Interface (GUI) is also available, made using the tkinter framework, which is capable of visualizing the ground truth segmentation, the network output, a live display of the current loss and mIoU accuracy, and other statistics.
This GUI can be seen in \figref{fig:experiments-gui}.

A Command Line Interface (CLI) is also available, made using the ncurses framework, which is capable of displaying various statistics during training.
The CLI also allows training remotely using secure shell (SSH), as the GUI would fail to start if invoked remotely through the command line.
A screen capture of the CLI can be seen in \figref{fig:experiments-cli}

\input{figures/experiments/ui}

The different components necessary to train the network are written in such a way as to be modular, with nearly everything being easily configurable.
For example, the network, optimizer, and loss criterion can easily be swapped by changing command-line arguments or parameters in the JSON file.
This makes running different experiments straightforward.

\section{Results}\label{section:experiments-results}
Running our network with the tuned hyperparameters chosen by the hyperparameter search in Section \ref{section:experiments-hyperparameters} for 30,336 iterations, we were able to achieve a mIoU accuracy of 50.188\% on the validation dataset.

\input{tables/experiments/results}

The network is able to produce the segmentations seen in \figref{fig:experiments-resultsmapillary}.
We also notice that the network is able to achieve very high IoU accuracy for the road class, as seen in \tabref{tab:mapillary-results}.
This is due to the original purpose of DeepLab v3+ as a road segmentation network \cite{deeplab}.
The IoU accuracy of curbs, curb cuts, and sidewalks are lower than of the road class.
One possible reason for this is that IoU is especially hard to fulfill when there are only a few pixels belonging to the class.
Cases where the class has only a few pixels and the borders do not match exactly would return a low IoU.

\input{figures/experiments/mapillary}
\input{figures/experiments/obelix}

In these results we can observe that the network is quite capable of identifying each of the traversability classes.
\figref{fig:mapresult-1} shows an example of how the network is able to identify each of the traversibility classes.
The network can also identify traversability classes, specifically curbs and curb cuts, which are in the distance and thus have a relatively small size, as can be observed in Figures \ref{fig:mapresult-2} - \ref{fig:mapresult-5}.
In these cases, the segmentation is significantly larger than the size of the curb itself, i.e. even though the intersection is significant, there is a significant area around the curb that is falsely identified as also being a part of the curb.
Figure \ref{fig:mapresult-5} shows a failure case where the road marking in the bottom right corner is mislabeled as a curb.
Figures \ref{fig:mapresult-4} and \ref{fig:mapresult-5} also show failure cases where the road is labeled as the ignore class.
Figure \ref{fig:mapresult-6} shows another failure case where the entire road is mislabeled as sidewalk.

We also ran inference against a dataset gathered by the onboard cameras on the Obelix robot using the network.
The results of this experiment can be seen in \figref{fig:experiments-resultsobelix}.
Figures \ref{fig:obresult-1} and \ref{fig:obresult-2} show that the network is able to generalize to the Obelix dataset and is able to identify the different traversability classes in the images, despite never having been trained on them.
Figure \ref{fig:obresult-3} shows although curb cuts are correctly identified, as in the case of the curb cut across the street, they are also sometimes mislabeled, as in the case of the curb cut directly ahead of the robot.
We can observe in Figures \ref{fig:obresult-4} and \ref{fig:obresult-5} that the network seems unable to decide whether the asphalt sidewalk is a road or a sidewalk.
The same failure case can be seen in Figure \ref{fig:obresult-6} where the left side of the sidewalk is mislabeled as road while the right side is labeled correctly as sidewalk.

We observe that the main failure cases of the network are falsely identifying certain road markings as curbs and identifying certain roads as sidewalks or vice versa.
Certain road markings which are located near road edges are sometimes segmented as curbs, as can be seen in the bottom right corner of \figref{fig:mapresult-5}.
We believe that this is due to the texture of the marking being colored similarly to some curbs as well as its shape which does resemble a curb.
In some cases, the sidewalk is identified as a road, as in \figref{fig:obresult-5} or the entire road is identified as a sidewalk, as in \figref{fig:mapresult-6}.
We believe that this may occur when sidewalks have an asphalt-like texture, as in the case in \figref{fig:obresult-5}.


% !TEX root=../report.tex

\section{Giving more choice}

\begin{grammar}
  Pretasks
    & p & ::=& \ldots       & \\
    &   &\mid& e_1 \Or e_2  & – task choosing \\
    &   &\mid& e_1 \Xor e_2 & – user choosing \\
  Tasks
    & t & ::=& \ldots       & \\
    &   &\mid& t_1 \Or t_2  & – task choosing \\
    &   &\mid& t_1 \Xor t_2 & – user choosing \\
  Actions
    & a & ::=& \ldots       & \\
    &   &\mid& \Pick r      & – pick route \\
  Routes
    & r & ::=& \Left r      & – go left \\
    &   &\mid& \Right r     & – go right \\
    &   &\mid& \Here        & – stay here \\
\end{grammar}

\begin{equation*}
  \userule{T-Or} \qquad \userule{T-Xor}
\end{equation*}

\begin{equation*}
  \userule{E-Or} \qquad \userule{E-Xor}
\end{equation*}

\begin{equation*}
  \userule{S-OrLeft} \qquad \userule{S-OrRight} \qquad \userule{S-OrNone}
\end{equation*}

\begin{equation*}
  \userule{S-Xor}
\end{equation*}

\begin{equation*}
  \begin{array}{lcl}
    \Value(t_1 \Or t_2) &=& \left\{
      \begin{array}{ll}
        v_1  & \when\ \Value(t_1) = v_1 \\
        v_2  & \when\ \Value(t_1) = \bot \lor \Value(t_2) = v_2 \\
        \bot & \otherwise
      \end{array}
    \right.
  \end{array}
\end{equation*}

\begin{equation*}
  \begin{array}{lcl}
    \Failing(t_1 \Or t_2) &=& \Failing(t_1) \lor \Failing(t_2)
  \end{array}
\end{equation*}

\begin{margintext}{Aside: Pairing in iTasks}
  $\Or$ is similar to \type{-||-}
  and $\Xor$ does not exist in iTasks.
\end{margintext}

\begin{equation*}
  \userule{H-FirstOr} \qquad \userule{H-SecondOr}
\end{equation*}

\begin{equation*}
  \userule{H-PickLeft} \qquad \userule{H-PickRight} \qquad \userule{H-PickHere}
\end{equation*}

\subsection{Three principles of choosing}

\subsection{Intermezzo: alternative functor and its laws}

\begin{equation*}
  \begin{array}{rclr}
    \Fail \Or t
      &\approxeq& t
      & \text{(left identity)} \\
    t \Or \Fail
      &\approxeq& t
      & \text{(right identity)} \\
    r \Or\ (s \Or t)
      &\approxeq& (r \Or s) \Or t
      & \text{(associativity)} \\
    & & & \\
    \Fail \Then g
      &\approxeq& \Fail
      & \text{(left absorption)} \\
    \Edit x \Or t
      &\approxeq& \Edit x
      & \text{(left catch)}
  \end{array}
\end{equation*}

\subsection{Example: a vending machine}

Let us model a vending machine.
In a simple process algebra we can write
\begin{equation*}
  ?\text{2euro}; (!\text{tea}\ \oplus\ ?\text{1euro}; !\text{coffee}).
\end{equation*}
Which reads:
users should insert 2 Euros,
after which they need to make a choice between tea,
or entering another Euro which results in coffee.
So $?\text{2euro}$ and $?\text{1euro}$ mean an input of respectively 2 and 1 Euro,
and $!\text{tea}$ and $!\text{coffee}$ are the outputs.
We use $\oplus$ as an external choice operation.

We can model this vending machine using tasks as follows.
\begin{equation*}
  \begin{array}{l}
    \Enter \Euro \Then \lambda n.
    \IF n \equiv 2.00 \THEN \\
     \quad \Edit \str{tea}
      \Xor
        \Enter \Euro \Then \lambda m.
        \IF m \equiv 1.00 \THEN
          \Edit \str{coffee}
        \ELSE
          \Fail \\
    \ELSE \Fail
  \end{array}
\end{equation*}
Where we use a new type $\Euro$ to represent amounts of money.

We see that in the task world we reason with concrete \emph{values} of a specified \emph{type}.
Only when the user inserted exactly two euros,
we will step to the next task.
In all other cases,
the step will \emph{fail} and we stay where we are.
After the first step,
users need to make an \emph{explicit} choice by selecting tea or selecting the option to enter more coins.

The task world explicitly captures that users can, for example, insert five euros:
The step will not be taken.
Also, it implicitly captures that the user can get his money back.
This can be implemented by a button on the vending machine which sends the \emph{empty} event to the task.
Next to an input field in a graphical user interface,
a coin insertion component is another way to present the concept of an editor to an end user.

% \begin{TASK}
%   fill Euro >>= \ n.
%     if n == 2 then
%         edit "tea"
%       <?>
%         fill Euro >>= \ m.
%         if m == 1 then
%           edit "coffee"
%         else
%           fail
%     else
%       fail
% \end{TASK}


\documentclass[a4paper,10pt]{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{pifont}
\usepackage[utf8]{inputenc}
\usepackage[left=2.5cm,right=2.5cm,top=2.5cm,bottom=2.0cm]{geometry}
\usepackage{multirow}
\usepackage{geometry}
\usepackage{pdflscape}
\usepackage{xcolor}
\usepackage{listings}
\lstset{basicstyle=\ttfamily,
  showstringspaces=false,
  commentstyle=\color{red},
  keywordstyle=\color{blue}
}
\usepackage{float}
\newcommand{\xmark}{\ding{55}}%

\title{Elements of Computational Biology\\ \Large
Subject 15: Distance phylogenetics: UPGMA and NJ}
\author{Agata Radys, Paweł Cejrowski, Łukasz Myśliński}
\date{\today}

\begin{document}
\newgeometry{margin=1.8cm}
\maketitle

\section{Usage}
Program was developed in Java programming language without any external libraries. All sources are available on Github and compiled using \texttt{Maven}.
\begin{lstlisting}[language=bash,caption={Building project}]
git clone git@github.com:MiSSLab/BioComp15.git
mvn package
\end{lstlisting}
Created Java archive can be run using \texttt{JRE}. Sample data can be found in directory \texttt{resources/}.
\begin{lstlisting}[language=bash,caption={Running project using data1.matrix}]
java -jar -Dfilename="resources/data1.matrix" \
    target/distance-phylogenetics-jar-with-dependencies.jar
\end{lstlisting}

\section{Data formats}
\subsection{Input}
Application requires \texttt{CSV} data format and quadratic matrix of distances.

\begin{lstlisting}[caption={Example data file content}]
a,b,c,d,e
0,8,8,5,3
8,0,3,8,8
8,3,0,8,8
5,8,8,0,5
3,8,8,5,0
\end{lstlisting}
Labels in header has to be lexicographically sorted, dense vector with every column matching \texttt{"[a-z]+"}.
\newpage
\subsection{Output}
Resulting trees are printed in \texttt{ASCII}-art to the \texttt{STDOUT}.
\begin{lstlisting}[caption={UPGMA output}]
UPGMA(resources/data1.matrix)
[[8.0]]
|-[[5.0]]
|  |-[[3.0]]
|  |  |---- [[a]]
|  |  `---- [[e]]
|  `---- [[d]]
`-[[3.0a]]
   |---- [[b]]
   `---- [[c]]

     a b c d e 8.0 5.0 3.0 3.0a
a   |0 0 0 0 0 0   0   1   0
b   |0 0 0 0 0 0   0   0   1
c   |0 0 0 0 0 0   0   0   1
d   |0 0 0 0 0 0   1   0   0
e   |0 0 0 0 0 0   0   1   0
8.0 |0 0 0 0 0 0   1   0   1
5.0 |0 0 0 1 0 1   0   1   0
3.0 |1 0 0 0 1 0   1   0   0
3.00|0 1 1 0 0 1   0   0   0

UPGMA-canonical
[[a]]
`-[[3.0]]
    |---- [[e]]
    `-[[5.0]]
        |---- [[d]]
        `-[[8.0]]
            `-[[3.0a]]
                |---- [[b]]
                `---- [[c]]
\end{lstlisting}
\newpage
Despite the fact that NJ returns unrooted tree it is presented as a rooted one with particular node choosen as a root.
\begin{lstlisting}[caption={NJ output}]
NJ(resources/data1.matrix)
[[g]]
|---- [1.5-<-[a]]
|---- [1.5-<-[e]]
`-[1.75-<-[h]]
    |---- [1.0-<-[d]]
    `-[4.0-<-[f]]
        |---- [1.5-<-[b]]
        `---- [1.5-<-[c]]

   a b c d e g h f
a|0 0 0 0 0 1 0 0
b|0 0 0 0 0 0 0 1
c|0 0 0 0 0 0 0 1
d|0 0 0 0 0 0 1 0
e|0 0 0 0 0 1 0 0
g|1 0 0 0 1 0 1 0
h|0 0 0 1 0 1 0 1
f|0 1 1 0 0 0 1 0

NJ-canonical
[[a]]
`-[[g]]
    |---- [[e]]
    `-[[h]]
        |---- [[d]]
        `-[[f]]
            |---- [[b]]
            `---- [[c]]
\end{lstlisting}
\section{Algorithms}

\subsection{UPGMA (ang. Unweighted Pair Group Method with Arithmetic Mean)}
Data: ultrametric matrix \textit{d} for set \textit{L}.
\begin{lstlisting}[caption={UPGMA pseudocode}]
clusters[|L|]
while (clusters.length > 1):
    calculate distances between clusters
        (sum of distances between cluster members
            divided by product of cluster cardinalities)
    find the lowest distance
    merge the closest clusters
\end{lstlisting}

\subsection{NJ}
Data: ultrametric matrix \textit{d} for set \textit{L}.\\ \\
Q - matrix: $ Q(i,j) = (n-2)d(i,j) - \sum_{k=1}^{n} d(i,k) - \sum_{k=1}^{n} d(j,k)$ \\ \\
Distance from the pair members to the new node: \\ \\
$d'(f,u) = \frac{1}{2}d(f,g) + \frac{1}{2(n-2)}(\sum_{k=1}^n d(f,k) - \sum_{k=1}^n d(g,k))$ \\ \\
$d(g,u) = d(f,g) - d'(f,u)$

\begin{lstlisting}[caption={NJ pseudocode}]
clusters[|L|]
while (number of clusters > 2):
    calculate Q-matrix
    find the lowest q-distance
    merge the q-closest clusters
    update distances
merge last 2 clusters
\end{lstlisting}

\subsection{Creating adjacency matrix}
Both rooted and unrooted trees created via UPGMA and NJ algorithms can be transformed to adjacency matrix. The algorithm for that is as follows:
\begin{lstlisting}[caption={Tree to adjacency matrix.}]
function walk(Node rootNode):
    int x = adjacencyMatrix.establishPosition(rootNode)
    for (node : rootNode.getChildren ):
        int y = adjacencyMatrix.establishPosition(node)
        adjacencyMatrix[x][y] = 1;
        adjacencyMatrix[y][x] = 1;
        walk(node);

init adjacency matrix with 0;
walk(tree.rootNode)
\end{lstlisting}

\subsection{Creating canonical tree}
To compare trees, we need its canonical form. It will be a tree with the first node from input data as a root.
\begin{lstlisting}[caption={Tree to adjacency matrix.}]
function toCanonicalTree:
    List<Integer> visited = new ArrayList<>();
    Node rootNode = createNodes(0, visited);
    return new Tree(rootNode);

function Node createNodes(int i, List<Integer> visited):
    visited.add(i);
    Node node = new Node(adjacencyMatrix.header[i]);
    for (j = 0; j < adjacencyMatrix.header.length; j++):
        if (visited.notContains(j) && adjacencyMatrix[i][j] = 1):
            node.children.add(createNodes(j, visited));
    return node;
\end{lstlisting}
\newpage
\subsection{Tree comparison}
Trees in canonical form can be easily compared using the following algorithm.
\begin{lstlisting}[caption={Comparing trees.}]
function equals(Node aNode, Node bNode):
    if (aNode.children.size = bNode.children.size):
        if (aNode.children.size == 0 && .children.size  == 0):
            if (aNode.label = bNode.label):
                return true
            else:
                return false
        else:
            comparisons = []
            for (aChild : aNode.children):
                comparisons2 = []
                for (bChild : bNode.children):
                    boolean equals = equals(aChild, bChild)
                    comparisons2.add(equals)
                comparisons.add(comparisons2.reduce(OR))
            return comparisons.reduce(AND)
    else:
        return false;
\end{lstlisting}
An important assumption in this algorithm is that we can compare labels of leaf-nodes, because they are those given as an input so they have to be equal.

\end{document}


\documentclass[a4paper]{article}
\usepackage{graphicx,subcaption}
\usepackage{amsmath,amsfonts}
\usepackage{qtree}
\title{Notes on Sequence Parsing}
\author{G.A. Jarrad}
\begin{document}
\maketitle
\numberwithin{equation}{section}
\numberwithin{figure}{section}
\numberwithin{table}{section}
\section{Introduction}\label{sec:intro}
The purpose of {\em sequence parsing} is to provide a hierarchically structured interpretation
of a sequence of tokens,
$\vec{\tau}=(\tau_1,\tau_2,...,\tau_n)$.
Our primary example is the English sentence $\vec{\tau}=({\tt The},{\tt cat},{\tt sat},{\tt on},{\tt the},{\tt mat})$,
the parse of which has a variety of representations, as shown by (but not restricted to)
Figures~\ref{fig:bracketing}--\ref{fig:parse-tree}. It is important to note, however, that some
natural languages are not so strictly ordered, and so a general sequence parse need not necessarily follow the same
ordering as the token sequence.
\begin{figure}[h]
\centering
\{
 \{{\tt The\, cat}\}\,
 \{{\tt sat}\,
  \{{\tt on}\,
   \{{\tt the\, mat}\}
  \}
 \}
\}
\caption{The parse represented as a hierarchical partitioning of the tokens.}
\label{fig:bracketing}
\end{figure}
\begin{figure}[h]
\centering
\(
\underset{\rho_5}{\underline{
   \underset{\rho_1}{\underline{\tt The\;\;cat}}\;\;
   \underset{\rho_4}{\underline{{\tt sat}\;\;
     \underset{\rho_3}{\underline{{\tt on}\;\;
       \underset{\rho_2}{\underline{\tt the\;\;mat}}
     }}
  }}
}}
\)
\caption{The parse represented as an ordered set of combination rules.}
\label{fig:nesting}
\end{figure}
\begin{figure}[h]
\centering
\Tree [.S [.NP [.DET The ] [.N cat ] ] [.VP [.V sat ] [.PP [.P on ] [.NP [.DET the ] [.N mat ] ] ] ] ]
\caption{The parse represented as a tree of nodes with  part-of-speech categories.}
\label{fig:parse-tree}
\end{figure}

Sequence parsing typically comprises two distinct stages: (i) {\em token analysis}, discussed more fully in
Section~\ref{sec:token-analysis};
and (ii) {\em structure analysis}, discussed in Section~\ref{sec:structure-analysis}.
Briefly, the purpose of token analysis is to deduce, for each token $\tau_i$, the set $\Lambda(\tau_i)$ of {\em leaf nodes}
that represent plausible interpretations of the token. For example, in natural language understanding the leaf nodes
might include categories such as part-of-speech, as shown in Figure~\ref{fig:parse-tree}.
The purpose of structure analysis is then to deduce a set $\Pi(\vec{\tau})$ of
rules that recursively combine sequences of nodes into higher-order {\em derived nodes},
until a single derived node spans the entire token sequence $\vec{\tau}$,
again as shown in Figure~\ref{fig:parse-tree}.

In summary, $\Pi(\vec{\tau})$ may be characterised as a parse tree graph with a
set ${\cal V}$ of nodes (both leaf and derived), and
a set ${\cal R}$ of rules linking these nodes.
Each leaf node represents known information about the corresponding
token, including the position of that token in the token sequence.
Likewise, each derived node represents a
sequence of leaf and/or derived nodes, which includes knowledge of the positions of all corresponding tokens.
In general, therefore, every node $\nu\in{\cal V}$ {\em spans} a set of tokens. Specifically, we define
the span $\sigma(\nu)$ of node $\nu$ to be the ordered set of
indices of the underlying tokens.
Consequently, each rule $\rho\in{\cal R}$ then takes the form
\begin{eqnarray}
\nu_{i_1}\;\nu_{i_2}\;\cdots\;\nu_{i_m} & \stackrel{\rho}{\rightarrow} & \nu_*\,,
\end{eqnarray}
where the derived node $\nu_*=\delta(\rho)$ combines the {\em predecessor} nodes
$\vec{\pi}(\rho)=(\nu_{i_1},\ldots,\nu_{i_m})$
with a resulting ordered span of
\begin{eqnarray}
\sigma(\delta(\rho))=\bigcup_{\nu\in\vec{\pi}(\rho)}\sigma(\nu)\,.
\end{eqnarray}
 For example, if $\sigma(\nu_1)=\{2,1\}$ and $\sigma(\nu_2)=\{3\}$, then the rule
$\nu_1\;\nu_2\rightarrow\nu_3$ implies that $\sigma(\nu_3)=\{2,1,3\}$.

\end{document}


\section{Uncertainty in PIV measurements}
\label{sec:piv_uncert}

 There are two major sources of measurement uncertainty associated with PIV.
 The first is "particle lag", or the ability of the seed particles to follow
 the
 flow, and is related to the inertia of the particle and the force exerted by
 the fluid upon the particle. The mineral oil droplets used in the present
 study are capable of following unsteady movements in the fluid flow on a scale
 much smaller than the PIV is capable of resolving, and so the uncertainty from
 this source is negligible. Sustained rotation similar to a rigid body
 inside the vortex core boundary has the effect of ejecting the relatively
 heavy particles away from the centerline. This particle ejection introduces a
 slight radial velocity bias in all measurements. This bias however is also
 negligible when compared to the second source of uncertainty: the process of
 extracting velocity vector information from the PIV images.

 Understanding the uncertainty in a PIV measurement can be accomplished with
 analysis of the PIV optical geometry \cite{lawson1997b}. Alternatively, Monte
 Carlo techniques for evaluating PIV uncertainty can be used by creating
 artificial image pairs with simulated particle displacements, then passing
 them through the PIV processing chain. In PIV, as previously discussed, each
 camera
 utilizes its own set of coordinate transform equations which allow pixel
 coordinates to be mapped to the coordinates of the interrogation plane.
 Simulated particles are given random coordinates within the views of both
 cameras in the interrogation plane, then mapped onto the coordinate system of
 both cameras. The intensity of each pixel can be determined by summing the
 intensity function of every randomly generated particle as in
 \cite{adeyinka2005,fouras2007}. This approach has been used successfully once
 before in the same wind tunnel facility with the same PIV system
 \cite{doan2012}.

There are a number of factors that contribute to uncertainty in PIV
measurements. Both bias and precision errors can be estimated by considering
detailed information about the optical geometry of the PIV setup. Monte-Carlo-
based error estimation techniques can be applied by creating artificially
simulated images with randomly distributed particles \cite{adeyinka2005}.
The distribution of these particles can be modeled using a Gaussian intensity
profile as described in Equation
\ref{eq:piv_gaussian_uncertainty} \cite{raffel1998}.

\begin{equation}
	I(x,y) = I_0exp \left( \frac{-(x_{img} - x_p)^2 - (y_{img} - y_p)^2}
	{\frac{1}{8}d_\tau^2} \right)
	\label{eq:piv_gaussian_uncertainty}
\end{equation}

\noindent
where $x_p$ and $y_p$ are the locations of the particle centroid, $d_\tau$ is
the diameter of the particle, and $I_0$ is particle intensity. Particle
intensity is directly related to the intensity of the light sheet, which is
modeled as a Gaussian distribution \cite{PIVuncertAIAA}. This assumption allows
us
to express particle intensity as \ref{eq:particle_intensity_gaus}

\begin{equation}
	I_0(z_p) = (q)exp\left(- \frac{z_p^2}{\frac{1}{8}\Delta Z_L^2}\right)
	\label{eq:particle_intensity_gaus}
\end{equation}

\noindent
where $z_p$ is the particles position within the thickness of the light sheet,
$q$ is the particle light scattering efficiency, and $\Delta Z_L$ is the
thickness of the light sheet.

These formulae were used to generate artificial image pairs for a single
camera.
A sufficient number of particles were created with $x$, $y$ and $z$ coordinates
to meet particle density parameters, these coordinates are then
used to generate light intensities according to Equation
\ref{eq:particle_intensity_gaus}, which populate the image plane of the first
image $A$. Subsequently, a displacement image, $B$, was generated by shifting
all the
particles in a predetermined direction in three dimensional space. It is worth
noting that for a single camera setup, particle movements in the $z$ direction
do not produce pixel displacements, but simply determine the intensity of the
light reflected from the particle. The known particle displacements can then be
compared against outputs calculated with PIV capture and processing software.

To translate this concept to stereo PIV, an additional step was required.
Instead of directly distributing particles with known coordinates onto the
image
plane of one camera, they were placed on a conceptual version of the
interrogation plane. The coordinate transforms obtained from PIV calibration
were used to map the displacements from the conceptual plane into the image
plane of each camera. These coordinate transforms are unique to each camera,
and depend upon the optical geometry of the PIV setup. Uncertainty is
calculated using the recommended AIAA calibration procedure
\cite{PIVuncertAIAA}. To determine the system bias, the mean difference between
the velocity standard established by the Monte Carlo simulation and the
velocity calculated by the PIV software are compared as follows

\begin{equation}
\overline{\Delta U} = \frac{1}{N} \left(\sum_{i=1}^N \Delta U_i \right),
\label{eq:Uerror}
\end{equation}

\begin{equation}
\overline{\Delta V} = \frac{1}{N} \left(\sum_{i=1}^N \Delta V_i \right),
\label{eq:Verror}
\end{equation}

\begin{equation}
\overline{\Delta W} = \frac{1}{N} \left(\sum_{i=1}^N \Delta W_i \right)
\label{eq:Werror}
\end{equation}

\noindent
which is simply the average difference between the known velocity components
and the measured velocity components $\Delta U$, $\Delta V$, and $\Delta W$,
for a large number of simulations. This is referred to as the \textit{bias},
and the three bias components are denoted as

\begin{equation}
\beta_{U} = \overline{\Delta U},
\label{eq:Ubias}
\end{equation}
\begin{equation}
\beta_{V} = \overline{\Delta V},
\label{eq:Vbias}
\end{equation}
\begin{equation}
\beta_{W} = \overline{\Delta W},
\label{eq:Wbias}
\end{equation}

The measurement precision is reported as the
standard deviation, calculated as in

\begin{equation}
S_{\Delta U} = \sqrt{\frac{1}{N-1} \left(\sum_{i=1}^N (\Delta U_i -
\overline{\Delta U})^2 \right)},
\label{eq:Usd}
\end{equation}

\begin{equation}
S_{\Delta V} = \sqrt{\frac{1}{N-1} \left(\sum_{i=1}^N (\Delta V_i -
	\overline{\Delta V})^2 \right)},
\label{eq:Vsd}
\end{equation}

\begin{equation}
S_{\Delta W} = \sqrt{\frac{1}{N-1} \left(\sum_{i=1}^N (\Delta W_i -
	\overline{\Delta W})^2 \right)},
\label{eq:Wsd}
\end{equation}

\noindent
resulting in precision calculations given by

\begin{equation}
P_{\overline{U}} = \frac{2 S_{\Delta U}}{\sqrt{N}},
\label{eq:Uprec}
\end{equation}

\begin{equation}
P_{\overline{V}} = \frac{2 S_{\Delta V}}{\sqrt{N}},
\label{eq:Vprec}
\end{equation}

\begin{equation}
P_{\overline{W}} = \frac{2 S_{\Delta W}}{\sqrt{N}}
\label{eq:Wprec}
\end{equation}

Total uncertainty for each component at the 95\% confidence level was
calculated by combining the bias and precision via to obtain

\begin{equation}
U_{\overline{\Delta U}} = \sqrt{\beta_{U}^2 + P_{\overline{U}}^2}
\label{eq:Uuncert}
\end{equation}
\begin{equation}
U_{\overline{\Delta V}} = \sqrt{\beta_{V}^2 + P_{\overline{V}}^2}
\label{eq:Vuncert}
\end{equation}
\begin{equation}
U_{\overline{\Delta W}} = \sqrt{\beta_{W}^2 + P_{\overline{W}}^2}
\label{eq:Wuncert}
\end{equation}

For these experiments, uncertainty analysis was conducted after the
experimental data were taken. Vortices were characterized by velocities at key
locations that allow each vortex to be described by one of the common vortex
models. Characterization velocities of particular interest include the maximum
tangential velocity about the vortex core, the distance of this maximal
tangential velocity region from the core axis, which defined the core radius,
and the typical
axial velocity distribution near the free stream velocity. Understanding the
uncertainty of these measurements required a Monte Carlo approach from
synthetically created particle imagery as shown in Figures \ref{fig:la} and
\ref{fig:lb}. Artificial pixel displacements were specified to approximate the
typical displacement associated with the velocities of greatest interest.

\vspace{32pt}
\begin{figure}[H]
	\centering
	\includegraphics[width=4in]{figs/artificial_images/la}
	\caption{Left artificial image subset example, station 1, $t=0$.}
	\label{fig:la}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[width=4in]{figs/artificial_images/lb}
	\caption{Left artificial image subset example, station 1, $t=dt$.}
	\label{fig:lb}
\end{figure}

%ensures every page has "Figure         Page" at the top of it
\makeatletter
\addtocontents{lof}{%
	\protect\afterpage{\protect\hbox to \linewidth%
		{\noindent Figure \hfill Page}\par%
		\protect\vspace{12\p@}}}
\makeatother

To add complexity, the time between frame captures, $dt$ was expected to have a
significant impact on uncertainty. Since the range of velocities used in this
study required the use of multiple values of $dt$, artificial images were
generated for a scenario at each value of $dt$ with appropriate associated
velocities.
With the exception of measurements at station one, all tests were conducted
with a $dt$ value of 25, or 40 $\mu s$. At station one $dt$ values of 35
$\mu s$ were also used. During the experimentation period, great difficulty
was encountered in tuning PIV parameters to achieve well-resolved vector fields
at station one. The uncertainty analysis demonstrated that the actual geometry
of the viewing angles was also unfavorable at this station, and the quality of
measurements taken this far upstream was poor. Hence, that data were entirely
disregarded.

Creation of these artificial images by Monte Carlo was performed with custom
software written in Python\texttrademark. This software parses the calibration
files output
from INSIGHT software and constructs the set of equations needed for all
coordinate transformations. In order to simulate as accurately as possible,
artificial images were generated at the full resolution of the specific PIV
cameras (1280 x 1024). Particles were only generated randomly at coordinates
that were within the field of view of both cameras, to eliminate wasted
computation time generating particles which would not aid in the production of
a three-dimensional vector. An excess of 100,000 particles were simulated for
each image set in order to ensure that the particle density was sufficient to
resolve a vector in the majority of sectors, and to approximate the
experimental seed density as closely as possible. The most accurate way to
ensure that the intensities of each particle was represented accurately was to
evaluate the intensity for every particle at every point in the full image
space of 1280 x 1024 pixels, and then add them together. Since this produces a
three-dimensional space in excess
of five billion values for a set of stereo image pairs, $La, Lb, Ra$ and $Rb$
images, computation time for uncertainty images was a limiting factor. At a
minimum, one set of uncertainty images for each combination of viewing geometry
and time step $dt$ was required. Therefore, simulated velocity values were
chosen for each of the 14 cases to approximate the velocities of greatest
interest.

Tables \ref{table:experiment_results_1} through
\ref{table:experiment_results_7} show a summary of results from each of the 70
tests conducted, including the maximum
observed azimuthal velocity, the average measured axial velocity, and the low
axial velocity at the vortex core. At the three furthest positions downstream
where the vortex appears to have stabilized, the typical value of maximum
tangential velocity, $\overline{v}_{\theta,max}$ in component notation, ranges
from 6.0 to 7.9 $m/s$ for each run with a $dt$ of $25 \mu s$. This tangential
velocity could align with the $X$ or $Y$-axis, and
it was desirable to simulate displacements in both directions at once to limit
the number of Monte Carlo simulations to be performed, so simulated velocities
$\overline{u}_{sim}$ and $\overline{v}_{sim}$
of $4.9 m/s$ was employed so that the in-plane magnitude would be equal
to the the middle of this range. Likewise, a simulated $\overline{u}_{sim}$ and
$\overline{v}_{sim}$  velocity of
$3.3 m/s$ was used to create synthetic images for testing experiments with a
$dt$ of $40 \mu s$. In the axial direction, $Z$ , mean values of $19 m/s$ and
$29
m/s$ were used for the high and low velocity experiments respectively. These
conditions are summarized in Table \ref{table:uncertainty_sim_table}.

\vspace{32pt}
\input{tables/uncertainty_sim_table}

Even though many samples were taken at every vector location, the uncertainty
in each individual measurement was of great importance for studying the
unsteady component of the velocity, and thus turbulent phenomena. Uncertainty
in the fluctuating component were best represented by using an $N$-value of one
in precision Equations \ref{eq:Uprec} through \ref{eq:Wprec}.
Uncertainty in the stable component was lower, since this measurement is a
result of averaging many measurements, and was calculated by using an $N$-value
of 200 in the precision Equations \ref{eq:Uprec} through \ref{eq:Wprec}.

\clearpage
\subsection{Uncertainty Analysis Results}
The Monte Carlo analysis was performed for each station employing two
scenarios. Uncertainty in the measurements for $u, v$ and $w$
components are tabulated in Tables \ref{table:uncertainties_u} and
\ref{table:uncertainties_v}. For the $u$ velocity components in the $X$
direction, bias is reported as $\beta_u$,
precisions are reported for both single-sample measurements and with
measurements comprised of 200 averages by $P_{u^\prime}$ and $P_{\bar{u}}$
respectively. Total uncertainties are reported as $U_{u^\prime}$ and
$U_{\bar{u}}$.

\vspace{32pt}
\input{tables/uncertainties_u}
\input{tables/uncertainties_v}

Uncertainties in
the $X$ and $Y$ direction were comparable in some cases, while in others $Y$
uncertainties were half as high as those in $X$. At all velocities, poor
precision in the individual measurements was the primary driver of total
uncertainty, while high bias was the driver for uncertainties in the
measurements comprised of time averages of 200 samples. Spectral analysis of
the turbulence over the observation time is subject to the very high
uncertainties associated with that of individual measurements. Time averaged
velocities, and the derived Reynolds stresses are best characterized by the
better precision offered by many samples. While 200 is nominally used as $N$,
each grid point may have between 20 and 200 valid vectors making up the set, so
some areas where vectors were less frequently resolved successfully may be
characterized with greater uncertainty.

In the axial or $Z$-direction, the PIV was found to exhibit an extreme bias
toward underestimating velocities based on the Monte Carlo simulation.
However, experimental data
showed that the average axial velocity was consistently a few percent above
free stream velocity measured in the wind tunnel using dynamic pressure
instrumentation, as expected, as shown in Tables
\ref{table:experiment_results_1} through \ref{table:experiment_results_7}.
Therefore, uncertainties in the $Z$ direction were overestimated by the
technique. The presence of the bias is peculiar, and could not be explained
despite best efforts by the author. The mean component of the Reynolds
decomposition components are
strongly influenced by a high bias, but the fluctuating components are entirely
sensitive to precision. The Reynolds stress and turbulence values which are of
great importance in this study included contributions of velocity fluctuations
in the axial direction, which are not impacted by any bias that does exist.
Experimental data were examined for possible misrepresentation of measurement
uncertainty in the $X$ and $Y$ direction, but no meaningful
discrepancies were found. Histograms the distribution of measurements for each
of the 14 cases were included in Appendix \ref{appendix:uncertainty_histograms}.

\subsection{Bias from Uneven sampling}

Every grid location with at least 20 successfully resolved vectors was
subject to a Reynolds decomposition into average and fluctuating components.
With 200 total samples taken for each test, the results are actually made up of
many grid points with between 20 and 200 samples. Uncertainty is related to the
number of samples making up a measurement, but visualizing the potential
impacts of uncertainty as it varies in two and three dimensional space is
challenging. Furthermore, the likelihood of successfully resolving a vector is
dependent upon aspects of the interaction between tracer particles and the
fluid flow. A very clear interdependence between areas of high turbulence, and
the number of successful vector calculations was observed as shown in Figures
\ref{fig:run_55_num_contour_nolab} and \ref{fig:run_55_ctke_contour_nolab}.

\input{figs/run_55/run_55_num_contour_nolab}
\input{figs/run_55/run_55_ctke_contour_nolab}

It was assumed that, as long as the number of samples making up any given
measurement remains above the threshold of 20, every vector in the field is
accurately represented. A sampling technique could be employed by
which a random selection of 20 different values is used at every location, to
result in a more statistically homogeneous vector field. However,
the uncertainty in a PIV measurement also depends significantly on the
magnitude of the velocity measured, thus the benefit of such a sampling scheme
is arguable.