\documentclass[12pt,a4paper]{article}

\usepackage[centertags,reqno]{amsmath}
\usepackage{verbatim}

\newcommand{\OCCAL}{{\sf\bf OCCAL}}
\newcommand{\REDUCE}{{\sf\bf REDUCE}}
\newcommand{\R}{\mbox{{\sf\bf R}}}
\newcommand{\FORTRAN}{{\sf FORTRAN}}
\newcommand{\RATFOR}{{\sf RATFOR}}
\newcommand{\Pascal}{{\sc Pascal}}
\newcommand{\C}{{\sf C}}
\newcommand{\GENTRAN}{{\sf GENTRAN}}
\newcommand{\MULCON}{{\sf MULCON}}
\newcommand{\SCOPE}{{\sf SCOPE}}
\newcommand{\perl}{{\sf perl}}

\renewcommand{\and}{\quad}

\newlength{\dummy}
\settowidth{\dummy}{Appendix~\mbox{}}
\addtolength{\dummy}{3mm}

\title{OCCAL\\
       A mixed symbolic-numeric\\
       Optimal Control CALculator}

\date{ }

\author{Rainer Sch\"opf
        \and
        Peter Deuflhard
        \\
        Konrad-Zuse-Zentrum\\
        f\"ur Informationstechnik Berlin\\
        Heilbronner Str. 10\\
        W-1000 Berlin 31\\
        Federal Republic of Germany}

\begin{document}

\maketitle

\begin{abstract}

  The numerical solution of optimal control problems by indirect
  methods (such as multiple shooting or collocation) requires a
  considerable amount of analytic calculation to establish a
  numerically tractable system.  These analytic calculations, though
  being rather tedious in realistic examples, are nowadays mostly
  still done by hand---and thus prone to calculation errors.  The
  paper aims at automating this analytic processing to a reasonable
  extent by means of a modern symbolic manipulation language (here:
  \REDUCE).  In its present stage of development the package \OCCAL{}
  (mnemotechnically for \underline{O}ptimal \underline{C}ontrol
  \underline{CAL}culator) permits an interactive use, covering tasks
  like automatic determination of control and, in case of a singular
  control, of its order.
%
  In simpler problems, the present version of \OCCAL{} automatically
  produces the full subroutine input for a MULtiple shooting code
  (\MULCON) with adaptive numerical CONtinuation.\\
%
  In more complicated problems where singular sub-arcs may occur or
  where the sequence of sub-arcs of the optimal trajectory is unclear
  \OCCAL{} is a significant help in reducing analytic pre-processing.
  Examples illustrate the performance of \OCCAL/\MULCON.

%  We describe \OCCAL, a integrated symbolic-numeric calculus system
%  for the treatment of optimal control problems. The system is not
%  only able to do a lot of simple but tedious analytic calculations,
%  but uses also advanced techniques to calculate as much as possible
%  automatically. Optimized code for the numeric boundary value problem
%  solver is generated as well.
\end{abstract}
\vspace*{0.5cm}
Preprint SC 91-13
\newpage

\tableofcontents

\newpage

\section{Introduction}

Optimal Control problems appear in a wide variety of important
scientific and technical applications.  In spite of the rather high
standards of {\em numerical\/} algorithms for such problems
\cite{Bock1,Bock2,Bock3,bulirsch,DFK2} tackling such a problem is
still a complicated task.  The reason for that is that certain steps in
the analytical preparation of the calculation, which are simple in
principle, may be very elaborate and can lead to rather complex
expressions.  Implementation of these into numerical code by hand is
tiresome and error-prone.

The purpose of our system \OCCAL{} to be presented here is to overcome
this tedious preparation phase. We intend to solve as many of the
arising problems as possible.

%To this end we employ a combined symbolic-numeric system, where the
%first step is to symbolically analyze the given problem, using the
To this end we employ a combined symbolic system that
symbolically analyzes the given problem, using the
Computer Algebra System \REDUCE.  In passing, this approach
additionally permits to generate {\em optimized\/} numerical
subprograms for use with existing numerical solvers for boundary value
problems. These are employed in the second step to obtain numerical
solutions.

Symbolic computation (as most computations) is often applied to tasks
that are simple in principle, but tedious to carry out by hand.
Consequently, often only some well defined steps in a calculation are
done with the help of a Computer Algebra System. The results are then
treated more or less by hand.

%\OCCAL{}, however is an integrated system in which the symbolic and
%numeric processors interact with each other, thereby allowing
%interchange of information between each other and with the user. The
%advantage of this approach is obvious: at each point in a calculation,
%the optimal method to solve the task at hand can be used. Examples of
%this are differentiation of functions, which can be ill-behaved if
%done numerically, or matrix computations, which can usually be done
%much faster by a numerical processor. Additionally, the automatic
%exchange of information reduces the number of errors the user may
%introduce into the calculation.

It should be stressed at this point that the symbolic processor in
\OCCAL{} is not only used for the tasks mentioned as ``tedious, but
simple in principle''.  Nowadays, advanced symbolic techniques are
available, e.g., for the solution of systems of nonlinear equations or
for computations with inequalities. This can be used, for example, to
determine the sign of a function of several variables over a given
range.

The need for these advanced methods within \OCCAL{} has not only led
to their improvement and adaptation to the problems at hand, but also
to their inclusion into the recently released new version of the
Computer Algebra System \REDUCE. Therefore these techniques are now
available to a much greater community than before, and hopefully also
for a much wider area of applications.

In Section \ref{sec:intro} below the typical structure of optimal
control problems is presented\allowbreak---mainly to fix notations and to
elucidate the kind of analytical work involved in the pre-numerical
stage. On this basis those tasks that can be automated are listed in
detail in Section \ref{sec:operation}---together with the
corresponding details of symbolic devices. Finally in Section
\ref{sec:examples} the performance of \OCCAL{} in a few well-known
examples is documented.

Of course, the full use of such a software system can only be
estimated by actually trying it!

\section{A Short Introduction \protect\\ to Optimal Control Problems}

\label{sec:intro}

Consider a system that is described by a number of time-dependent
state variables $y$. Let $y\in K \subset C^{1}$, where $K$ is a
suitable class of continously differentiable functions, and $y\colon
[a,b]\rightarrow \R^{n}$.  This system is controlled by certain {\em
control variables\/} $u\in C^{0}$ piecewise, $u\colon [a,b]\rightarrow
\R^{k}$, and governed by the following system of differential
equations
\begin{equation}
  y' = f(t,y,u), \label{dyn-sys}
\end{equation}
the with separated boundary conditions $y(a)=y_{a}$ and
\begin{equation}
  r(b,y(b)) =0, \quad r\colon \R^{n+1}\rightarrow \R^{p},
   \quad  p\leq n.
\end{equation}
The task is to minimize the integral
\begin{equation}
  I[u] := \int\limits_{a}^{b} f_{0}(y,u,t) dt
\end{equation}
over $K$.
%We couple the differential equations and the boundary conditions
%to the functional using Lagrange multipliers $\lambda$:
%\begin{equation}
%  \hat I[u,y,y',\lambda] := \Phi(b,y(b)) + \nu^{T} r
%      + \int_{a}^{b} \left(\sum_{i=1}^{n} \lambda_{i} (y'_{i} -
%      f_{i}(t,y,u)) \right) dt
%      \label{full-func}
%\end{equation}
Similar to the calculus of variations there is the so-called {\em
Hamiltonian\/}
\begin{equation}
  H(t,y,\lambda,u) := \sum_{i=1}^{n} \lambda_{i} f_{i}(t,y,u)
     +\lambda_{0} f_{0}(t,y,u) \label{Hamiltonian},
\end{equation}
where $\lambda(t) = (\lambda_{1}(t),\ldots,\lambda_{n}(t))$
denotes the {\em adjoint variables}.

The minimum principle of Pontrjagin states then that a necessary condition for
$(y_{0}, u_{0})$ being a solution of the optimal control problem is
\begin{equation}
  H(t,y_{0},\lambda,u_{0}) = \min_{v} H(t,y_{0},\lambda,v) \label{pontrjagin}
\end{equation}
with $v$ fulfilling the conditions.  Calculating the first variation
leads to the so-called {\em canonical equations\/}
\begin{eqnarray}
  y'_{i} &=& H_{\lambda_{i}} = f_{i}(t,y,u), \quad i = 1,\ldots,n, \nonumber\\
  \lambda'_{i} &=& - H_{y_{i}} = - \frac{\partial f_{0}}{\partial y_{i}}(t,y,u)
        - \sum_{j=1}^{n} \lambda_{i}
          \frac{\partial f_{j}}{\partial y_{i}}(t,y,u), \quad i =
          1,\ldots,n.           \label{can-eqs}
\end{eqnarray}
We number the control variables $u_{i}$ in such a way that the first
$m$ components (denoted by $u^{(n)}$) appear nonlinearly in $f$, while
the others (denoted by $u^{(l)}$) appear only linearly, so that $f$
splits in the form
\begin{eqnarray}
  f(t,y,u) &=:& g(t,y,u^{(n)}) + h(t,y,u^{(n)}) u^{(l)}\:, \nonumber \\
    && u^{(n)} \in \R^{m}, u^{(l)} \in \R^{q}, \quad m+q=k \label{u-dist}\:.
\end{eqnarray}
This splitting carries over to the Hamiltonian $H$ as well.  Since
$u^{(l)}$ appears only linearly in $f$ and $H$ (cf.\ (\ref{u-dist}))
minimization of $H$ can be done in two steps:
\begin{enumerate}
  \item Determination of $u^{(n)}$ from
  \begin{eqnarray}
    && H_{u^{(n)}} = 0 \nonumber \\
    && H_{u^{(n)}u^{(n)}} \mbox{ positive (semi-)definite} \label{sys-nonlin}
  \end{eqnarray}
  (Here $H_{u^{(n)}}$ denotes the Fr\'echet derivatives.)  One obtains
  an analytic expression $u^{(n)} = u^{(n)}(t,y,\lambda)$ which can be
  substituted into the original expressions (\ref{can-eqs}). Thus
  $u^{(n)}$ essentially drops out of the set of variables.
  \item Determination of $u^{(l)}$:\\
        After substitution of $u^{(n)}$ the Hamiltonian can be written as
        \begin{equation}
          H(t,y,\lambda,u^{(n)}(t,y,\lambda),u^{(l)})
            =: H_{0}(t,y,\lambda)
            +  \sum_{i=1}^{q} S_{i}(t,y,\lambda) u_{m+i}\:.
        \end{equation}
        Obviously we have
        \begin{equation}
          S_{i}(t,y,\lambda) = H_{u_{m+i}}(t,y,\lambda,u^{(n)}(t,y,\lambda),u^{(l)})
        \end{equation}
        with $i=1,\ldots,q$.
        The condition (\ref{pontrjagin}) is then equivalent to
        \begin{equation}
          S_{i}(t,y_{0},\lambda) \cdot u_{0}(t) = \min_{v} S_{i}(t,y_{0},\lambda) \cdot v(t),
        \end{equation}
      which is a linear optimization problem.
      It is therefore necessary that the control variables $u^{(l)}$ are subject to
      control constraints usually of the form
        \begin{eqnarray}
          && \alpha_{i} \leq u_{i} \leq \beta_{i}, \quad i=m+1,\dots,k
          \nonumber \\
          && \alpha_{i}, \beta_{i} \in \R.
        \end{eqnarray}
      The further procedure
      depends on which one of the following two cases holds:
        \begin{enumerate}
          \item $S_{i} \not\equiv 0$, i.e.\ $S_{i}$ has only isolated zeros.
               This is the case of a {\em bang-bang control}.
               From the minimum principle we deduce
                 \begin{equation}
                   u_{m+i} = \left\{\begin{array}{l@{\quad}l}
                                  \alpha_{m+i} & \mbox{for $S_{i} > 0$} \\
                                  \beta_{m+i} & \mbox{for $S_{i} < 0$}
                                \end{array}\right.
                 \end{equation}
               The $S_{i}$ are called {\em switching functions}.
               A change of sign can only occur in the {\em switching points\/}
               $\tau^{i}$:
                 \begin{equation}
                   S_{i}(\tau^{i},y(\tau^{i}),\lambda(\tau^{i})) = 0.
                 \end{equation}
          \item $S_{i} \equiv 0$ within a non-empty interval. This is the so-called {\em singular\/} case.
                Let us assume, for simplicity, that $q=1$, i.e.\ there
                is only one linear control $u$, and that there is only
                one such interval.  Define the following functions
                recursively by
                 \begin{eqnarray}
                   S^{(0)}(t,y,\lambda) &:=& S(t,y,\lambda) \\
                   S^{(k+1)}(t,y,\lambda) &:=& \frac{d
                   S^{(k)}(t,y,\lambda)}{dt} \nonumber \\
                        &=& S^{(k)}_{y} \cdot \dot{y} +
                        S^{(k)}_{\lambda} \cdot \dot{\lambda} + S^{(k)}_{t}\\
                        &=&  S^{(k)}_{y} \cdot H_{\lambda} -
                             S^{(k)}_{\lambda} \cdot H_{y}  +  S^{(k)}_{t} \nonumber
                 \end{eqnarray}
                Define $m$ to be the smallest index such that
                $\partial S^{(m)}/\partial u \not\equiv 0$. Then
                theory \cite{th1,th2,th3} shows that $m = 2r$ is an even
                number. $r$ is called {\em order\/} of the singular
                control.

                Since $u^{(l)}$ appears only linearly in $H_{y}$ and $H_{\lambda}$ it follows
                that $S^{(m)}$ is of the form
                 \begin{equation}
                   S^{(m)}(t,y,\lambda) = A(t,y,\lambda) + B(t,y,\lambda)\cdot u(t).
                 \end{equation}
                For $B(t,y,\lambda)\not=0$ the singular control can be
                determined to be
                 \begin{equation}
                   u_{0}(t,y,\lambda) = - \frac{A(t,y,\lambda)}{B(t,y,\lambda)} \label{singular-control}
                 \end{equation}
                In certain cases $u_{0}$ does not depend on the adjoint
                variables $\lambda$; this is called a {\em feedback
                control}.  For an optimal trajectory $y(t)$ and an
                optimal control $u_{0}(t)$ of order $k = 2r$ holds the
                {\em generalized Legendre-Clebsch condition\/} \cite{th3}
                  \begin{equation}
                    0 \leq (-1)^{r} \cdot \frac{\partial}{\partial u}
                     \left(\frac{d^{2r}}{dt^{2r}}H_{u}(t)\right)
                     =  (-1)^{r} \cdot  B(t,y(t),\lambda(t)),
                  \end{equation}
                In the case of a feedback control a strict inequality holds.
        \end{enumerate}
\end{enumerate}
Finally we can determine the boundary condition for the $\lambda_{i}$ as
\begin{equation}
  \left. \lambda_{i} \right |_{b} = \Phi_{y_{i}(b)} + \nu^{T} r_{y_{i}(b)},
\end{equation}
where the $\nu$ are Lagrange multipliers.

\vspace*{3mm}
\noindent {\bf Numerical solution by means of Multiple Shooting.}\\
After $u^{(n)}$ has been substituted back into the canonical equations
(\ref{can-eqs}) we can proceed to a {\em numerical\/} solution of the
resulting boundary value problem.  If there is a linear control
$u^{(l)}$ we have to take the structure of the optimal control problem
into account, especially the number $m_{s}$ and position of the switching
points $\tau_{j}$ defined by the equations
\begin{equation}
  S_{i}(\tau_{j},y(\tau_{j}),\lambda(\tau_{j})) = 0,\quad
   j = 1,\ldots,m_{s}.
\end{equation}
Since the position of these points is in general not known beforehand,
we introduce them as additional parameters
\begin{displaymath}
  \tau_{1} < \tau_{2} < \ldots < \tau_{m_{s}}
\end{displaymath}
and transform every sub-interval into an interval of the same length,
i.e.,
\begin{eqnarray}
         [a,\tau_{1}] \rightarrow [0,1] \colon
                 \bar t &:=& \frac{t - a}{\tau_{1} - a} \nonumber\\ \relax
  [\tau_{1},\tau_{2}] \rightarrow [1,2] \colon
                 \bar t &:=& \frac{t - \tau_{1}}{\tau_{2} - \tau_{1}} + 1 \nonumber\\ \relax
    \vdots \\ \relax
  [\tau_{m_{s}}, b] \rightarrow [m_{s},m_{s}+1] \colon
                 \bar t &:=& \frac{t - \tau_{m_{s}}}{b - \tau_{m_{s}}} + m_{s} \nonumber\\
  y(t) \rightarrow \bar y(\bar t),&& \lambda(t) \rightarrow \bar\lambda(\bar t) \nonumber
\end{eqnarray}
This leads to extra factors in the right hand side of the differential
equations which then additionally depend on one parameter (for the
border intervals) or on two parameters (for the intermediate
intervals). The parameters $\tau_{1},\ldots,\tau_{m_{s}}$ are
determined by the {\em inner point conditions}
\begin{equation}
  S_{i}(\tau_{j},\bar y(\tau_{j}),\bar \lambda(\tau_{j})) = 0,\quad
   j = 1,\ldots,m_{s}.
\end{equation}
It may happen that the switching structure changes during the
numerical iteration. In that case the whole procedure has to be
restarted with the new structure taken into account.

Thus the problem has been reduced to a parameter dependent boundary
value problem of the general form:
\begin{eqnarray}
  y' &=& f(y;p) \\
  r(y(a),\ldots{},y(b);p) &=& 0.
\end{eqnarray}

The associated multiple shooting Jacobian then has the general block
structure
\begin{equation}
  \left[
    \begin{array}{cccccc}
      G_{1} & - I & \cdot & \cdot & P_{1} \\
      \cdot & \cdot & \cdot & \cdot & \cdot \\
      \cdot & \cdot & \cdot & \cdot & \cdot \\
      \cdot & \cdot & G_{m-1} & -I & P_{m-1} \\
      R_{1} & \cdot & R_{m-1} & R_{m} & P_{m}
    \end{array}
  \right] \label{Jacobian}
\end{equation}
in terms f the sub-matrices
\begin{eqnarray}
  G_{j} &:=& \left. W(t_{j+1},t_{j}) \right|_{y(t|x_{j},p)} \nonumber \\
  R_{j} &:=& \left. \frac{\partial r}{\partial y(t_{j})} \right|_{y(t|x_{j},p)} \nonumber \\
  P_{j} &:=& \left. P(t_{j+1},t_{j}) \right|_{y(t|x_{j},p)} \\
  P_{m} &:=& \left. \frac{\partial r}{\partial p} \right|_{y(t|x_{j},p)} \nonumber.
\end{eqnarray}
Here the matrices $W(t_{j+1},t_{j})$ are the {\em Wronskian matrices}
\begin{equation}
  W(t,t_{j}) = \frac{\partial y(t)}{\partial y(t_{j})}
\end{equation}
which are the solutions of the {\em variational equation}
\begin{equation}
  \frac{dW(t,t_{j})}{dt} = f_{y}(y(t|x_{j},p);p) W(t,t_{j}),\quad W(t_{j},t_{j}) = I,
\end{equation}
and the {\em sensitivity matrices\/} $P_{j} = P(t_{j+1},t_{j})$ are
the solutions of the generalized variational equation
\begin{equation}
  \frac{dP(t,t_{j})}{dt} = f_{y}(y(t|x_{j},p);p) P(t,t_{j}) + f_{p}(y(t|x_{j},p);p)
\end{equation}
with initial value
\begin{equation}
  P(t_{j},t_{j}) = 0.
\end{equation}

\section{Application of symbolic computation\protect\\
         to optimal control problems}

\label{sec:operation}

Summarizing the calculation steps outlined in the previous section we
have the following sub-tasks
\begin{itemize}
  \item Calculation of Hamiltonian.
  \item Identification of nonlinear versus linear control.
  \item Determination of nonlinear control by solving the system (\ref{sys-nonlin}).
  \item Substitution into differential equations (\ref{can-eqs}).
  \item Determination of linear control.
  \item In the case of bang-bang control: determination of the switching points.
  \item In the case of singular control: determination of the order of the control
        and test of the generalized Legendre-Clebsch condition.
  \item Generation of boundary conditions for the adjoint variables.
  \item Determination of the variational equations.
  \item Code generation (with optional optimization) for the differential equations,
        the boundary conditions, and the variational equations.
\end{itemize}
Some of these steps can easily be executed by {\em any\/} Computer
Algebra System (). Examples of this type of tasks are the computation of
the Hamiltonian (\ref{Hamiltonian}), of the canonical equations
(\ref{can-eqs}), of the derivatives $H_{u^{(n)}}$ and $H_{u^{(n)}u^{(n)}}$,
or the substitution of the solution $u^{(n)}$ into the Hamiltonian.

The \OCCAL{} system, however, does much more than that. We head at
accomplishing all the aforementioned tasks automatically, and have
already succeeded to a great extent.  Let us first give a short
overview of the operation of our system.

The first thing for the user to do is to formulate his problem in a
form that is suitable for processing with \OCCAL.
Since the symbolic program is written in \REDUCE, input must be given
in \REDUCE's Algol-like syntax. Even though this syntax is not too
difficult to learn and understand, it is an unnecessary burden to the
user who is not necessarily familiar with this type of program.

Therefore we decided to allow the input to be in a very natural,
user-friendly form, with automatic translation into \REDUCE{} syntax.
To this end we developed a pre-processor, using the utility \perl{}
\cite{perl}. We chose \perl{} due to its wide-spread availability and
excellent string manipulation facilities that allowed us to implement
the program in about one day's time.

The basic idea is to provide an easy-to-understand input file format,
with keywords to identify the different sections, and some freedom for
the user to choose a formulation that is most suitable to his or her
own way of thinking. Furthermore, we want to keep in mind the
possibility of a future system where this input file is only an
intermediate step in a much more sophisticated system with, e.g., a
graphical user interface. A description of the input file format is
given in appendix~\ref{app:perl-file}.

This pre-processing step does already some simple consistency checks:
constants, parameters and variables may be defined only once,
differential equations may only be given for already defined dynamic
variables, and so on. If there was no error, it outputs a complete
definition of the problem in \REDUCE{} syntax to a file, taking care
of arranging the statements in the correct order.  This file is
then read by the \REDUCE{} program and processed symbolically.

It should be stressed that this output is well-readable: the
experienced user may want to intervene at that point instead of
changing the input file.

In the second step, the symbolic program tackles the tasks outlined
above.  First the Hamiltonian (\ref{Hamiltonian}) is calculated. This
object is analyzed and the nonlinear and linear controls $u^{(n)}$ and
$u^{(l)}$ identified.

The determination of the nonlinear control is done in three steps, of
which only the first one is straightforward in general:
\begin{itemize}
  \item Calculation of $H_{u^{(n)}}$.
        This involves only a series of symbolic differentiations, a
        standard problem for any Computer Algebra System.
  \item Solution of the system of equations.
        Here advanced symbolic techniques are used: algebraic
        equations are solved using the methods of Groebner bases,
        transcendental equations may be reduced to algebraic ones by a
        suitable change of variables. For example, the transcendental
        equation
        \begin{equation}
          A \sin u + B \cos u = C \label{sin-cos}
        \end{equation}
        may be converted to a system of algebraic equations by substituting
        \begin{eqnarray*}
          x = \sin u \\
          y = \cos u
        \end{eqnarray*}
      and adding the relation $x^{2}+y^{2}-1=0$.
      \typeout{******************************}
      \typeout{Einfuegen: test der direkten steuerung delta.}
      \typeout{******************************}
  \item Determination of the sign of $H_{u^{(n)}u^{(n)}}$.
        This is done using new symbolic techniques developed at ZIB.
        Two different methods are available: the simpler one of these
        tries to deduce the sign of the expression at hand by breaking
        it down into its parts and determining the sign of those.
        Though this seems to be a rather simple algorithm, it is
        nevertheless very powerful and can be applied to a wide range
        of expressions.  If this is not sufficient, the user will be
        queried for the missing information.
%
%        If this is not sufficient, a more complicated method can be
%        used: given a function of several variables, and the domains of
%        these variables, it determines the range of the function.
\end{itemize}
Finally, the solution is substituted into the Hamiltonian and the
differential equations. This uses again very much the standard
facilities of the \REDUCE{} system for differentiation, substitution
and extraction of parts of expressions.

If there is no linear control $u^{(l)}$, all that remains to do is to
solve the system of differential equations. To this end, \OCCAL{}
generates code for use with a numerical code for solving boundary
value problems. The code generation step uses the \GENTRAN{}
translator for its work; this package has interfaces for \FORTRAN,
\RATFOR, \Pascal, and \C{} programming languages.

In our examples we use exclusively \C; our numerical BVP solver is
\MULCON{} \cite{mulcon} which was translated to \C{}
semi-automatically, first using a FORTRAN-to-C translator and then
beautifying the automatically produced code by hand to make it better
readable.

Several functions are generated in the target language:
\begin{itemize}
  \item Calculation of boundary values $r$.
  \item Calculation of differential equations $f$.
  \item Calculation of $f_{y}$ to solve the variational equation
        (for Wronskian approximations $G_{j}$, $P_{j}$).
  \item Calculation of the matrices of boundary condition derivatives
        $R_{1} = r_{y(a)}$ and $R_{m} = r_{y(b)}$.
\end{itemize}
These are exactly the functions that are needed to compute the
Jacobian (\ref{Jacobian}).

\begin{figure}[bt]
  \setlength{\unitlength}{0.5mm}
  \newcommand{\smiley}{%
    \fbox{\setlength{\unitlength}{0.5pt}%
          \begin{picture}(30,40)
            \put(8,30){\circle*{2}}
            \put(22,30){\circle*{2}}
            \put(15,10){\line(0,1){20}}
            \put(1,5){\line(1,0){20}}
          \end{picture}}}

  \begin{center}
  \begin{picture}(120,170)(0,0)
    \put(10,140){\framebox(40,25){User input}}
    \put(30,110){\oval(45,15)}
    \put(10,110){\makebox(40,0){\small Pre-processor}}
    \put(30,070){\oval(45,15)}
    \put(10,070){\makebox(40,0){\small Symbolic}}
    \put(10,010){\framebox(40,25){\small Numeric}}
    \put(80,100){\dashbox(40,25){\Large User\ \smiley}}
   \thicklines
    \put(97,127){\vector(-3,2){44}}
    \put(97,097){\vector(-3,-2){44}}
    \put(53,070){\vector(+3,+2){44}}
    \put(29,138){\vector(0,-1){19}}
    \put(31,138){\vector(0,-1){19}}
    \put(29,100){\vector(0,-1){20}}
    \put(31,100){\vector(0,-1){20}}
    \put(29,060){\vector(0,-1){24}}
    \put(31,060){\vector(0,-1){24}}
  \end{picture}
  \end{center}
  \caption{An overview of the \OCCAL\ system}
  \label{fig:overview}
\end{figure}

The symbolic calculation of the matrix entries is of some importance
for the whole computation, since this avoids the (external) numerical
differentiation of $f$ and $r$, which might well lead to problems in
the numerical calculation.  Furthermore, one may reorder the variables
to take advantage of the possible block structure of $R_{1}$ and
$R_{m}$.

Without further manipulation, the resulting code is generated directly
from the expressions obtained in the previous phase, i.e.\ in a fully
expanded form. This is, of course, non-optimal, except for really
simple cases.  Therefore several different optimization strategies can
be applied.

We developed a very simple one that separates the calculation of
algebraic and transcendental functions like $\sqrt {x}$ or ${\mathrm
e}^{x}$, (so that their values are computed only once) and uses
Horner's scheme to evaluate polynomials.  In the case of sparse
polynomials in many variables, however, using Horner's scheme is not
straightforward, since it is not a priori clear how the variables
should be ordered.

A more elaborate optimization scheme was developed and implemented by
{\sc van Hulzen et al.}\ \cite{scope}. It attempts to minimize the
number of arithmetic operations. The optimizer performs heuristic
searches on arithmetic expressions, detecting and extracting common
subexpressions, and replacing them by temporary variable names.

The choice of optimization strategy depends mainly on the problem at
hand. Complete optimization takes a considerable amount of computing
time, but this may be payed off by the generation of more efficient
and better readable numerical code. For every new problem, it is
therefore necessary to exploit all possibilities to arrive at an
optimal solution.

If, on the other hand, we have a problem with linear control $u^{(l)}$,
the next step is to try to solve the remaining linear optimization
problem. The present \OCCAL{} system is not yet capable to distinguish
between the singular and the bang-bang case. This is being worked on.

Figure~\ref{fig:overview} depicts the overall structure of the system.

\section{Examples}

\label{sec:examples}

In this section we present a few simple and well-known examples to
illustrate the performance of our system,

\vspace*{3mm}\noindent
{\bf Example 1: Turbo Generator \cite{Wulkow}}.

This example is simple enough to do all necessary calculations by
hand, and therefore well-suited to check the performance of \OCCAL{}.

We will not discuss the technical background of the turbo generator as
a machine. Rather we concentrate on the mathematical model derived
from these technical considerations.

The dynamic model of the turbo generator consists of the following system of
ordinary differential equations:
\begin{eqnarray}
  \dot x_{1} &=& x_{2} x_{4} \nonumber \\
  \dot x_{2} &=& \frac{1}{M} (u_{1} - s_{4} x_{1} x_{4} - s_{5} x_{1}
                  x_{3} - \kappa_{d} x_{2}) \nonumber \\
  \dot x_{3} &=& u_{2} - A x_{3} + c x_{4} \nonumber \\
  \dot x_{4} &=& - x_{1} x_{2}
\end{eqnarray}
Here $x_{i}$ are the dynamic variables, $u_{i}$ the control variables,
$s_{4}$, $s_{5}$, $\kappa_{d}$, $A$, $M$, and $c$ are constants.  The
stationary state of the machine is characterized by $x_{i} =
x_{i}^{s}$ and $u_{i}=u_{i}^{s}$, with certain constants $x_{i}^{s}$,
$u_{i}^{s}$.

The functional to be minimized describes the deviation from a
stationary state
\begin{equation}
  \begin{split}
  J_{1}(x,u) = \int\limits_{0}^{T}
                (& \alpha ((x_{1} - x_{1}^{s})^{2} +
                          (x_{4} - x_{4}^{s})^{2}) +
                  \alpha_{2} x_{2}^{2} +
                  \alpha_{3} (x_{3} - x_{3}^{s})^{2} + \\
              &    \beta_{1} (u_{1} - u_{1}^{s})^{2} +
                  \beta_{2} (u_{2} - u_{2}^{s})^{2}) dt
  \end{split}
\end{equation}
with certain weight factors $\alpha$, $\alpha_{2}$, $\alpha_{3}$,
$\beta_{1}$, and $\beta_{2}$.  Boundary conditions are
\begin{eqnarray}
  x(0) &=& (x_{1}^{s}, x_{2}^{s}, x_{3}^{s}, x_{4}^{s}) =: x_{0}^{t} \\
  x(T) && \mbox{free}
\end{eqnarray}
The control variables $u^{(n)} = (u_{1},u_{2})$ appear quadratically in
the Hamiltonian, which leads to a system of two linear equations with
a unique solution. After substituting the solution into the
differential equations we end up with a boundary value problem with
four dynamical variables. Due to its simplicity, no optimization of
the code is necessary.

A corresponding \OCCAL{} sample session, the input file for \OCCAL{}
itself, the \REDUCE{} input file generated from this by the
pre-processor, and the \C{} source file are shown in \ref{app-turbo}.
It can be seen that the intermediate \REDUCE{} input file is
well-readable, so that the experienced user may want to edit this file
directly. This example takes about $25$ seconds of real time on a
typical (Sun 4) workstation.

The file of generated \C{} code shows several interesting points:
\begin{itemize}
  \item The constants defined in the CONSTANTS section of the input
        file are rendered as \C{} pre-processor macros.

  \item There are three \C{} functions defined: for the boundary
        values (\verb|R1|), for the functions $f_{i}$ (\verb|FCN1|),
        and for the functions $f_{y}$ that are needed in the solution
        of the variational equation (\verb|DFCN1|).

        The functions for the calculation of the matrices $R_{1}$ and
        $R_{m}$ have been skipped in this example.
  \item The code optimization process introduces several intermediate
        variables to reduce the number of operations in the program.
        These are easily identified by their name which consists
        always of the letter \verb|G| (short for generated) followed
        by a number. It is possible to choose other names as well.
\end{itemize}

With a rather simple main program to drive the calculation it is easy
to reproduce the numerical results of \cite{Wulkow}.

\vspace*{3mm}

\noindent
{\bf Example 2: Re-entry problem \cite{SB}.}

This is a well-known test problem of somewhat higher degree of complexity.
Here, the expressions arising from the calculation turn out
to be rather lengthy.

The re-entry of an Apollo vehicle into the Earth's atmosphere is
governed by the following system of differential equations:
\begin{eqnarray}
  \dot v &=& V(v,\gamma,\xi,u) = - \frac{S \rho v^{2}}{2m} C_{W}(u) -
             \frac{g \sin \gamma}{(1+\xi)^{2}}\\
  \dot \gamma &=& \Gamma(v,\gamma,\xi,u) = \frac{S \rho v}{2m}
             C_{A}(u) + \frac {v \cos \gamma}{R(1+\xi)}
             - \frac{g \cos \gamma}{v(1+\xi)^{2}}\\
  \dot \xi &=& \Xi(v,\gamma,\xi,u) = \frac{v\sin\gamma}{R} \\
  \dot \zeta &=& Z(v,\gamma,\xi,u) = \frac{v \cos \gamma}{1+xi}
\end{eqnarray}
where $u$ is the control variable; and state variables are: $v$
(tangential velocity), $\gamma$ (flight path angle), $\xi = h/R$
(relative height over ground), $\zeta$ (velocity over ground).
Moreover, the expressions $\rho = \rho_{0} \exp(-\beta R \xi)$ (),
$C_{W}(u)$ (aerodynamical resistance), $C_{A}(u)$ (drag or lift
coefficient), $S$ (front area), and $m$ (mass of the vehicle) appear
in the above equations..

The boundary values at the begin of the reentry into the atmosphere
and at the time of landing are
\begin{alignat}{3}
 v(0) &= 0.36,& \quad
 \gamma(0) &= -8.1^{\circ} \frac{\pi}{180^{\circ}},& \quad
 \xi(0) &= \frac{4}{R}\\
 v(T) &= 0.27,& \quad
 \gamma(T) &= 0,& \quad
 \xi(T) &= \frac{2.5}{R}
\end{alignat}
The final time $T$ is free; $\zeta$ is not considered for the
optimization. The functional to be minimized is the heating of the
capsule, approximated by
\begin{equation}
  J = \int_{0}^{T} 10\, v^{3} \sqrt{\rho} dt \:.
\end{equation}
The problem has only one control variable, but equation
(\ref{sys-nonlin}) turns out to be transcendental, of the type
(\ref{sin-cos}) discussed earlier. It is possible, by a
change of variables, to turn it into a system of algebraic equations
which can be solved by computing the Groebner base of the polynomials.
This is done automatically by the \REDUCE{} equation solver. As a
result, we get two solutions, only one of which leads to a positive
definite determinant. This can be determined automatically if the
additional information of the variable $v > 0$ is given to the system.
Otherwise the user is prompted for the sign of $v$.

The further calculation then proceeds, in principle, just as in the
previous example. However, the expressions to be turned into numerical
code are quite large, so that we apply optimization techniques during
the code generation process. Of the two levels described above, the
\SCOPE{} package by van Hulzen \cite{scope} is able to reduce the size
of the \C{} source code from $35$ kBytes ($808$ lines) to $12$ kBytes
($378$ short lines). The simpler package which only extracts simple
common subexpression reduces the size to $27$ kBytes ($570$ lines).
Using the latter, the calculation takes about $50$ seconds of real time
on a Sun 4 workstation.

% Two examples including singular control
%
% \cite{chemo,zellen} are two examples of optimal control problems with
% non-regular control. The first of these is completely solvable
% analytically in the absence of state constraints \cite{chemo}.
% \OCCAL{} is not yet capable to do this completely since this solution
% relies on heuristic methods which are very difficult to make into an
% algorithm. Nevertheless, \OCCAL{} goes very far and is able to
% correctly determine the singular control.
%
% The second example shows other difficulties: due to the presence of a
% lot of inequalities the calculation of the singular control can be
% simplified significantly. However, dealing with inequalities is a weak
% point in most Computer Algebra Systems (see also next section).

\section{Conclusion}

As documented above, \OCCAL{} has reached a first stage of
development where it can be successfully used to help in the solution
of optimal control problems. The tasks already accomplished are:
\begin{itemize}
  \item Calculation of Hamiltonian.
  \item Determination of nonlinear and linear control.
  \item Code generation with optional optimization.
\end{itemize}
Next steps of development will include the following features that we
are planning to incorporate in the future:
\begin{itemize}
  \item Symbolic-numeric determination of nonlinear control: It is rather obvious
        that not for all systems of equations an analytic solution can
        be found. In these cases the algebraic equations have to be
        coupled to the dynamical system (\ref{dyn-sys}) and the whole
        must be solved numerically.

        The numeric treatment of algebro-differential equations is
        being worked on.

  \item Determination of inequalities: The determination of the sign
        of the determinant (\ref{sys-nonlin}) is only a first simple
        step in the treatment of inequalities. There are a number of
        other places where inequalities arise, e.g., when inequalities
        involving the state variables appear. This is especially
        important in order to find the intervals where the switching
        funtions vanish identically.

  \item Inclusion of state constraints: these may be very important as
        they may determine the structure of the linear control (i.e.,
        singular vs.\ bang-bang). As far as inequalities are
        concerned, this is related to the previous point, and has
        probably to be treated in a common framework.

  \item Treatment of singular sub-arcs.
\end{itemize}

\noindent
{\bf Acknowledgments.} The authors wish to thank M.~Wulkow for several
helpful discussions.

\begin{thebibliography}{99}

  \bibitem{Bock1} H.~G.~Bock:
  \newblock Randwertproblemmethoden zur Parameteridentifizierung
            in Systemen nichtlinearer Differentialgleichungen,
  \newblock Universit\"at Bonn, Dissertation (1985).

  \bibitem{Bock2} H.~G.~Bock:
  \newblock Numerical Solution of Nonlineaer Multipoint Boundary Value
            Problems with Applications to Optimal Control,
  \newblock GAMM, Copenhagen 1977.

  \bibitem{Bock3} H.~G.~Bock:
  \newblock Numerische Berechnung zustandsbeschr\"ankter optimaler
            Steuerungen mit der Mehrzielmethode,
  \newblock Carl-Cranz-Gesellschaft 1978.

  \bibitem{bulirsch} R.~Bulirsch:
  \newblock Variationsrechnung und optimale Steuerung.
  \newblock Lectures given at the Universit\"at zu K\"oln (1971).
  \newblock Unpublished.

  \bibitem{th1} R.~Bulirsch et al.~(eds.):
  \newblock Optimization and Optimal Control,
  \newblock LN Math.~477 (1975).

  \bibitem{th2} A.~E.~Bryson and Y.~C.~Ho:
  \newblock Applied optimal control,
  \newblock Ginn and Company,
  \newblock Waltham, Massachusetts (1969).

  \bibitem{mulcon} P.~Deuflhard, B.~Engquist (Ed.):
  \newblock Large Scale Scientific Computing,
  \newblock Birkhauser/Boston, Series ``Progress In Scientific Computing'',
  \newblock Vol.~7 (1987).

  \bibitem{DFK2} P.~Deuflhard, B.~Fiedler, P.~Kunkel:
  \newblock Efficient Numerical Pathfollowing Beyond Critical Points
            in ODE models,
  \newblock in: \cite{mulcon}, pp.~97--113 (1987).

  \bibitem{scope} J.~A.~van~Hulzen:
  \newblock Code optimization of multivariate polynomial schemes:
            A pragmatic approach,
  \newblock Proceedings EUROCAL '83,
  \newblock Springer LNCS 162, pp.~286--300.

  \bibitem{th3} H.~Maurer, U.~Heidemann:
  \newblock Optimale Steuerprozesse mit Zustandsbeschr\"ankungen,
  \newblock in: \cite{th1}.

  \bibitem{SB} J.~Stoer, R.~Bulirsch:
  \newblock Einf\"uhrung in die Numerische Mathematik II,
  \newblock Springer-Verlag,
  \newblock Berlin 1978,
  \newblock ISBN 3--540--08840--7.

  \bibitem{perl} L.~Wall, R.~L.~Schwartz:
  \newblock Programming \perl,
  \newblock O'Reilly \& Associates, Inc.,
  \newblock 1990.

  \bibitem{Wulkow} M.~Wulkow:
  \newblock Vergleich numerischer Verfahren zur Berechnung der
  optimalen Steuerung eines Turbogenerators,
  \newblock Diploma Thesis, Univ.\ M\"unster, 1987.

%  \bibitem{f2c}

\end{thebibliography}

\newpage

\appendix

\renewcommand{\thesection}{Appendix~\Alph{section}}
\renewcommand{\thesubsection}{\Alph{section}.\arabic{subsection}}

\section[\protect\hspace*{\dummy}An \OCCAL{} sample session\protect\hspace*{-\dummy}]%
        {An \OCCAL{} sample session}

\label{app-turbo}
As an example we treat the example of the turbo generator. We assume
that the following input file has been prepared.\\
\rule{\textwidth}{0.6mm}
\begin{small}
  \verbatiminput{turbo.dsc}
\end{small}
\rule{\textwidth}{0.6mm}\\[3mm]
It is further assumed that the name of this file ends in
\verb|.dsc|, e.g., \verb|turbo.dsc|.
Type in:
\begin{verbatim}
occal turbo
\end{verbatim}
\OCCAL{} displays
\begin{verbatim}
Parsing input file...done.
Starting symbolic processor...
REDUCE 3.4, 15-Jul-91 ...

1:

\end{verbatim}
and then a protocol of the following \REDUCE{} statements that were
generated by the pre-processor.\\
\rule{\textwidth}{0.6mm}
\begin{small}
  \verbatiminput{turbo.red}
\end{small}
\rule{\textwidth}{0.6mm}\\[3mm]
After a while it displays
\begin{verbatim}
============================================================
 That's all, folks!
============================================================
\end{verbatim}
and then
\begin{verbatim}
Quitting
OCCAL finished.
\end{verbatim}
There is now a file with name \verb|turbo1.c|. We show its contents for illustration.\\
\rule{\textwidth}{0.6mm}
\begin{small}
  \verbatiminput{turbo1.c}
\end{small}
\rule{\textwidth}{0.6mm}

\newpage

\section[\protect\hspace*{\dummy}Files for the reentry problem]%
        {Files for the reentry problem}

\label{app-SB}

\small

\subsection{Problem description}
\verbatiminput{SB.dsc}

\subsection{\REDUCE{} input file}
\verbatiminput{SB.red}

\newpage

\section[\protect\hspace*{\dummy}Input file format for the pre-processor]%
        {Input file format\\ for the pre-processor}

\label{app:perl-file}

The input file consists of a number of sections, each identified by a
keyword on a line of its own, and ended by an empty line. Possible
keywords are:
\begin{description}
  \item [COMMENT] Used only to explain what is going on. Everything up
        to the next empty line is ignored.
  \item [NAME] A short name to identify the problem, used for file
        names, etc.
  \item [TITLE] Description of the problem, can consist of several
        lines of text.
  \item [CONSTANTS] Constants of the problem, i.e., identifiers whose
        values do not change. Can only be set at compile time. A value
        is mandatory.
  \item [PARAMETERS] Parameters whose values may be set at run time.
        A value is optional.
  \item [DYNAMIC\_VARIABLES] The dynamic variables, called $y_{i}$ in a
        previous section, each on a line of its own.
  \item [CONTROL\_VARIABLES] The control variables, called $u_{i}$ in
        a previous section, each on a line of its own.
  \item [TIME\_VARIABLE] The name of the independent variable.
  \item [BOUNDARIES] Two expressions in \REDUCE{} syntax that denote
        the boundaries of the interval. Must be delimited by a
        semicolon.
  \item [FUNCTIONAL] The functional to be minimized.
  \item [DIFFERENTIAL\_EQUATIONS] The dynamical system.
  \item [END\_POINT\_IS\_FREE] Specifies that the right boundary is free
        and to be calculated.
  \item [BOUNDARY\_CONDITIONS] The boundary conditions at the points
        specified in the BOUNDARIES section.
  \item [ABBREVIATIONS] Arbitrary \REDUCE{} expressions, each
        delimited by a semicolon.
  \item [RESTRICTIONS] Additional conditions, such as a variable being
        bounded from below.
  \item [TARGET\_LANGUAGE] The programming language in which the numerical
        programs are generated. Possible are: \C, \FORTRAN, \RATFOR, and \Pascal.
  \item [TEMPVAR\_PREFIX] The prefix used for constructing variable names during
        code generation. Default is \verb|ANS|.
\end{description}
The order of these sections is largely arbitrary, except for one
restriction: the dynamic variables need to be defined before the
corresponding differential equations.

\end{document}


% STRAINS --------------------
\section{Discussion}\label{sec:discussion:clustering}

In general, we observe two trends in our data. For the \isols{} that get clustered into strains,
our approach correctly identifies the \spec{} with over 80-85\% accuracy. This accuracy
is sufficient to conduct sophisticated \mst{} studies. Most of the strains discovered
in the \cplop{} data show high degree of purity, and even considering the presence of
a few large impure clusters, most of the clustered \isols{} fall into strains of high purity.

At the same time, the pure strain-based approach suffers from a drop in the coverage as the size of a cluster grows.
This means that in general \cplop{} \isols{} tend to be very diverse
and come from strains for which not enough DNA material has been collected and pyrosequenced.
Identifying the \spec{} for \isols{} that do not fall into strains/clusters using the pure strain-based
method is impossible. In future work, our goal is to combine the \kNN{}-based \mst{} method
of \cite{DBLP:conf/bibm/McGovernDKBVG15} with the strain-based approach discussed in this paper
to increase coverage while preserving the high \mst{} accuracy.

%%%%%%%

%Considering how many \isols{} end up in clusters of high purity, as \autoref{fig:clust_purity_dist_3} shows, this occurrence appears to be infrequent.
%Nevertheless, viewing the progression of graphs in \autoref{fig:clust_pure} implies that the clusters morph and change into each other as we increase \minneigh{}.

One factor explaining the large impure clusters is the possibility that these clusters represent what the  biologists call  ``transient'' strains, i.e., strains that  persist in more than one \spec{}.
%That is, certain strains might show up in many \spec{} and not just relegated to one \spec{}.
Such a characteristic can compound \mst{} by making certain strains of \ecoli{} less reliable as \fib{} for identifying \spec{}.
In \autoref{fig:clust_purity_dist}, we see evidence of that and it is revealed in \autoref{fig:clust_pure}.
One mitigation strategy may be to reduce the presence of these strains in the library holding the \fib{}.
Another may be to fall back to an alternative \mst{} technique that works with \cplop{} when an unknown \isol{} falls into an impure cluster.
Finally, if a true transient strain is indeed discovered, and an \isol{} is mapped to it, our \mst{} procedure can simply acknowledge that the query \isol{} belongs to a transient strain and provide information about the \spec{} that show high frequency of \ecoli{} incidence from this strain.
In order to handle the lack of complete clustering coverage --- when \dbscan{} marks an \isol{} as noise --- we propose a fallback method : the \kraplong{}, described in \autoref{chap:krap}.

%It also is possible that a more complicated or biologically-motivated strategy may work best.
%One avenue of validation we chose not to pursue was $k$-fold cross-validation.
%Previous work \cite{DBLP:conf/bibm/McGovernDKBVG15} has used it for validation.
%Future work may include it, but large $k$ values are likely to partition the dataset into groups different enough to create widely varying clusterings.


\documentclass{spec}

\usepackage{amsfonts,bm,amsmath}
\usepackage{verbatim}
\usepackage{algorithm, algpseudocode}
\usepackage{caption}
\usepackage{calc}

\input{nlls_header.tex}
% data types for the derived types
\newcommand{\scalarinteger}[1]{\itt{#1} is a scalar of type {\tt INTEGER} }
\newcommand{\scalarlogical}[1]{\itt{#1} is a scalar of type {\tt LOGICAL} }
\newcommand{\scalarcharacter}[2]{
  \itt{#1} is a scalar of type {\tt CHARACTER} and length {\tt #2}
}
\newcommand{\scalarreal}[1]{\itt{#1} is a scalar of package type}
\newcommand{\ronearrayinteger}[1]{is a rank-1 array of type {\tt INTEGER} and size {\tt #1} }
\newcommand{\ronearrayreal}[2]{\itt{#1} is a rank-1 array of package type and size {\tt #2}\ }
% data types for the calling sequences
\newcommand{\scalarintegerio}{is an \intentout scalar of type {\tt INTEGER} }
\newcommand{\scalarintegerii}{is an \intentin scalar of type {\tt INTEGER} }

\newcommand{\ronearrayrealii}[1]{is an \intentin rank-1 array of package type and size {\tt #1} }
\newcommand{\ronearrayrealiiopt}[1]{is an optional \intentin rank-1 array of package type and size {\tt #1} }
\newcommand{\ronearrayrealio}[1]{is an \intentout rank-1 array of package type and size {\tt #1} }
% Fortran calling routine
\newcommand{\mainroutine}{{\tt nlls\_solve}}
\newcommand{\onestep}{{\tt nlls\_iterate}}
% Fortran customization
\newcommand{\ct}{\%}
\newcommand{\fortranonly}[1]{#1}
\newcommand{\conly}[1]{}
\newcommand{\vecentry}[2]{\texttt{#1(#2)}}
\begin{document}

\hslheader

\hslsummary

\input{summary.tex}

%!!!!!!!!!!!!!!!!!!!!!!!!!!!!
\hslattributes
\hslversions{\versionum\ (\versiondate)}.
\hslIRDCZ Real (single, double).
\hsllanguage Fortran 2003 subset (F95+TR155581).
\hsldate January 2016.
\hslorigin The Numerical Analysis Group, Rutherford Appleton Laboratory.
\hslremark The development of this package was
partially supported by EPSRC grant EP/M025179/1.

%!!!!!!!!!!!!!!!!!!!!!!!!!!!
\newpage
\hslhowto

\subsection{Calling sequences}

Access to the package requires a {\tt USE} statement \\ \\
\indent\hspace{5mm}{\tt use ral\_nlls\_double}
% \noindent
% If it is required to use more than one module at the same time, the derived
% types  (see Section~\ref{derived types})
% must be renamed in one of the {\tt USE} statements.
\medskip

\noindent The user can then call one of the procedures:
\vspace{-0.1cm}
\begin{description}
\item {\tt NLLS\_SOLVE}  solves the non-linear least squares problem (\ref{eq:nlls_problem}).
\item {\tt NLLS\_ITERATE} performs one iteration for the non-linear least squares solver.
\end{description}

%%%%%%%%%%%%%%%%%%%%%% derived types %%%%%%%%%%%%%%%%%%%%%%%%

\hsltypes
\label{derived types}
For each problem, the user must employ the derived types defined by the
module to declare scalars of the types {\tt NLLS\_inform} and
{\tt NLLS\_options}.  If {\tt nlls\_iterate} is to be used, then
a scalar of the type {\tt NLLS\_workspace} must also be defined.
The following pseudocode illustrates this.
\begin{verbatim}
      use nlls_module
      ...
      type (NLLS_inform) :: inform
      type (NLLS_options) :: options
      type (NLLS_workspace) :: work ! needed if nlls_iterate to be called
      ...
\end{verbatim}
The components of {\tt NLLS\_options} and {\tt NLLS\_inform} are explained
in Sections~\ref{typeoptions} and \ref{typeinform}.

%%%%%%%%%%%%%%%%%%%%%% argument lists %%%%%%%%%%%%%%%%%%%%%%%%
\hslarguments
\subsubsection{Optional arguments}\label{Optional arguments}

We use square brackets {\tt [ ]} to indicate \optional\ arguments.
In each
call, optional arguments follow the argument {\tt inform}.  Since we
reserve the right to add additional optional arguments in future
releases of the code, {\bf we strongly recommend that all optional
arguments be called by keyword, not by position}.

\subsubsection{Integer and package types}
%{\tt INTEGER} denotes default {\tt INTEGER} and
%{\tt INTEGER(long)} denotes {\tt INTEGER(kind=selected\_int\_kind(18))}.
The term {\bf package type} is used to mean
default real if the single precision version is being used and
double precision real for the double precision version.

\subsubsection{To solve the non-linear least squares problem}
\label{sec:solve}

To solve the non-linear least squares problem a call of the following form should be made:

\begin{verbatim}
   call nlls_solve(n, m, X, eval_r, eval_J, eval_Hf, params, options, inform[, weights])
\end{verbatim}

\begin{description}
\itt{n} \scalarintegerii that holds the number $n$ of
variables to be fitted; i.e., $n$ is the length of the unknown vector $\bm x$.
\textbf{Restriction:} \texttt{n}$>${\tt 0}.

\itt{m} \scalarintegerii that holds the number $m$ of
data points available; i.e., $m$ is the number of residuals $r_i$.
\textbf{Restriction:} \texttt{m}$\geq$\texttt{n}

\itt{X} is an \intentinout\  rank-1 array of package type
and size {\tt n}.  On entry, it must hold the initial guess for $\bm x$, and on
successful exit it holds the solution to the non-linear least squares problem.

\itt{eval\_r} is a {\tt PROCEDURE} that, given a point $\iter{\vx}$, returns the vector $\vr(\iter{\vx})$.
Further details of the format required are given in Section~\ref{sec::function_eval}.

\itt{eval\_J} is a {\tt PROCEDURE} that, given a point $\iter{\vx}$,
returns the $m \times n$ Jacobian matrix, $\iter{\vJ}$, of $\vr$ evaluated at $\iter{\vx}$.
Further details of the format required are given in Section~\ref{sec::function_eval}.

\itt{eval\_Hf} is a {\tt PROCEDURE} that, given vectors $\vx \in \mathbb{R}^n$
and $\vr \in \mathbb{R}^m$, returns the quantity $\sum_{i=1}^m (\vr)_i \nabla^2 \vr_i (\vx)$.
Further details of the format required are given in Section~\ref{sec::function_eval}.  If {\tt NLLS\_options\%exact\_second\_derivative = .false.}, then this is not referenced.

\itt{params} is an \intentin\ scalar of class {\tt params\_base\_type} that holds parameters to
be passed to the user-defined routines {\tt eval\_r}, {\tt eval\_J}, and {\tt eval\_Hf}.
Further details of its use are given in Section~\ref{sec::function_eval}.

\itt{options}is an \intentin\  scalar  of type {\tt NLLS\_options}
(see Section~\ref{typeoptions}).

\itt{inform} is an \intentinout\ scalar of type
{\tt NLLS\_inform}. Its components provide information about the execution
of the subroutine, as explained in Section~\ref{typeinform}.

\itt{weights} \ronearrayrealiiopt{m}.  If present, {\tt weights} holds the square-roots of the
diagonal entries of the weighting matrix, $\vW$, in (\ref{eq:nlls_problem}).  If absent, then the norm in (\ref{eq:nlls_problem}) is taken to be the 2-norm, that is, $\vW = I$.

\end{description}

\subsection{To iterate once}
\label{sec:iterate}
Alternatively, the user may step through the solution process one iteration at
a time by making a call of the following form:

\begin{verbatim}
   call nlls_iterate(n, m, X, w, eval_F, eval_J, eval_HF, params, options, inform[, weights])
\end{verbatim}

\begin{description}

\item[\texttt{n}, \texttt{m}, \texttt{eval\_F}, \texttt{eval\_J}, \texttt{eval\_HF}, \texttt{params}, \texttt{info} and \texttt{options}] are as described in Section~\ref{sec:solve}.

\itt{X} is an \intentinout\  rank-1 array of package type
and size {\tt n}.  On the first call, it must hold the initial guess for
$\bm x$. On return it holds the value of $\bm x$ at the current iterate, and
must be passed unaltered to any subsequent call to \texttt{nlls\_iterate()}.

\itt{w} is an \intentinout\ scalar of type \texttt{NLLS\_workspace}. It is used
to store the current state of the iteration and should not be altered by the
user.

\end{description}

The user may use the components \texttt{info\%convergence\_normf} and
\texttt{info\%convergence\_normg} to determine whether the iteration has
converged.

\subsection{User-supplied function evaluation routines}
\label{sec::function_eval}
To evaluate the residual, Jacobian and Hessian at a point, the user
must supply subroutines that perform this operation that the package
{\tt ral\_nlls} will call internally.

In order to pass user-defined data into the evaluation calls, {\tt params\_base\_type} is extended to a {\tt user\_type}, as follows:
\begin{verbatim}
   type, extends( params_base_type ) :: user_type
      ! code declaring components of user_type
   end type user_type
\end{verbatim}
We recommend this type is wrapped in a module with the user-defined routines
for evaluating the function, Jacobian, and Hessian.

The components of the extended type are accessed through a \texttt{select type}
construct:
\begin{verbatim}
   select type(params)
   type is(user_type)
     ! code that accesses components of params that were defined within user_type
   end select
\end{verbatim}

\subsubsection{For evaluating the function $\vr(\vx)$}

A subroutine must be supplied to calculate $\vr(\vx)$ for a given vector $\vx$.
It must implement the following interface:

\begin{verbatim}
abstract interface
   subroutine eval_r(n, m, params, x, r, status)
      integer, intent(in) :: n
      integer, intent(in) :: m
      class(params_base_type), intent(in) :: params
      double precision, dimension(n), intent(in) :: x
      double precision, dimension(m), intent(out) :: r
      integer, intent(inout) :: status
   end subroutine eval_r
end interface
\end{verbatim}

\input{eval_r.tex}

\subsubsection{For evaluating the function $\vJ = \nabla \vr(\vx)$}
A subroutine must be supplied to calculate $\vJ = \nabla \vr(\vx)$ for a given vector $\vx$. It
must implement the following interface:

\begin{verbatim}
abstract interface
   subroutine eval_J(n, m, params, x, J, status)
      integer, intent(in) :: n
      integer, intent(in) :: m
      class(params_base_type), intent(in) :: params
      double precision, dimension(n), intent(in)  :: x
      double precision, dimension(n*m), intent(out) :: J
      integer, intent(inout) :: status
  end subroutine eval_J
end interface
\end{verbatim}

\input{eval_j.tex}

\subsubsection{For evaluating the function $Hf = \sum_{i=1}^m r_i(\vx) \vW \nabla^2 r_i(\vx)$}
A subroutine must be supplied to calculate $Hf = \sum_{i=1}^m (\vr)_i \nabla^2 r_i(\vx)$ for given vectors $\vx \in \mathbb{R}^n$ and $\vr \in \mathbb{R}^m$; here \((\vr)_i\) denotes the $i$th component of the vector $\vr$. The subroutine must implement the following interface:

\begin{verbatim}
abstract interface
   subroutine eval_Hf_type(n, m, params, x, r, Hf, status)
       integer, intent(in) :: n
       integer, intent(in) :: m
       class(params_base_type), intent(in) :: params
       double precision, dimension(n), intent(in)  :: x
       double precision, dimension(m), intent(in)  :: r
       double precision, dimension(n*n), intent(out) :: Hf
       integer, intent(inout) :: status
     end subroutine eval_Hf_type
end interface
\end{verbatim}

\input{eval_hf.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{The options derived data type}
\label{typeoptions}

\input{options_type.tex}

\subsection{The derived data type for holding information}
\label{typeinform}

\input{info_type.tex}

%%%%%%%%%%%%%%%%%%%%%% Warning and error messages %%%%%%%%%%%%%%%%%%%%%%%%

\hslerrors

A successful return from a subroutine in the package is indicated by
{\tt NLLS\_inform\%status} having the value zero.
A non-zero value is associated with an error message that by default will
be output on unit {\tt NLLS\_options\%error}.

\input{errors.tex}

\hslgeneral

\hslio
   Error messages on unit {\tt NLLS\_options\%error} and warning
   and diagnostic messages on unit \newline{\tt NLLS\_options\%out},
   respectively. These have default value {\tt 6};
   printing of these messages is suppressed if the relevant unit number
   is negative or if {\tt NLLS\_options\%print\_level} is negative.
\hslrestrictions {\tt m$\ge$n$\ge$1}.

\hslmethod
\label{method}

\input{method.tex}
% todo!

\hslexample

\input{example_text.tex}

\verbatiminput{../example/Fortran/nlls_example.f90}

\end{document}


\chapter{EXAMINATION OF PREVIOUSLY-EXISTING GLOBAL DP LIBRARIES}

The first goal of this Thesis is to examine previously existing programming libraries and APIs that provide the application of Differential Privacy to a dataset. This has been achieved by many companies, such as Google and IBM, but also from research programs like ARX that study the benefits of data privacy. We separate those implementations regarding their output. The possible outputs of a mechanism that adds D.P. to a dataset can be:
\begin{itemize}
    \item An answer to a query, in a private manner.
    \item An anonymized dataset, that meets the criteria of D.P.
\end{itemize}

In the first category, we can distinguish libraries such as Google's and IBM's, that have functions which if applied on a dataset, and given a specific query, can return a single answer.

In the second one, we can find libraries such as the ARX tool, that given a dataset and a group of privacy settings (such as the amount of noise to be inserted), produces an anonymized version of a dataset, that has obviously reduced information in comparison to the original one, but is usable by the final user.

In this chapter, we are going to test those libraries by providing different kinds of datasets, in order to determine the advantages and the disadvantages of each category.

We are going to conduct all of our testings using noise generated by the \emph{Laplace Mechanism}, thus we must first define its theoretical behavior.

\section{The Laplace Mechanism}

The Laplace Mechanism is used widely in applications of Differential Privacy regarding \emph{numerical queries}, which are actually functions that match a query to a number, or a vector of numbers, thus answering to it. The mechanism uses noise produced by the Laplace probabilistic distribution, which is proportional to the query's sensitivity.

\subsection{Query Sensitivity}
The $l_1$ sensitivity of a query $f$, is defined as following:

\begin{align*}
    \Delta f = \max_{\{||x-y||_1 = 1\}} ||f(x) - f(y)||_1
\end{align*} where $x,y \in N^{|X|}$.

This quantity shows the effect by which a single participant's data can change in the worst case during the query $f$, and thus, the uncertainty that we must insert to to the response in order to protect them.

\subsection{The Laplace Distribution}
The Laplace Distribution with a scale $b$, is the distribution with probability density function:
\begin{align*}
Lap(x|b) = \frac{1}{2b}exp(-\frac{|x|}{b})
\end{align*}

who's variance is $\sigma^2 = 2b^2$, and is actually a symmetric version of the exponential distribution.

\subsection{Use of Laplace in D.P.}

In order to be of use in our definition, the scale of the noise will be calibrated to the sensitivity of the query $f$, divided by epsilon. Thus, the noise used will be drawn from

\begin{align*}
Lap(\frac{\Delta f}{\epsilon})
\end{align*}

Of course, many other probabilistic distributions can be used to ensure differential privacy, but during our testings we prefer to use Laplace.

\section{Query Answering Libraries}

We are going to begin by testing libraries that belong in the first category, and specifically the \emph{IBM's diffprivlib}, which is written in python, and is publicly available \href{https://github.com/IBM/differential-privacy-library}{here}. The library includes a host of mechanisms, the building blocks of differential privacy, alongside a number of applications to machine learning and other data analytics tasks. A full explanation of the library's functionalities is available in [3]. We are going to focus our testings in the simple queries, such as the \emph{mean value}, the \emph{extreme values} and the \emph{histograms} of a numerical dataset. The library consists of three modules:

\begin{itemize}
    \item \textbf{Mechanisms}, as known from the theoretical foundations of D.P.
    \item \textbf{Models}, especially machine learning models, that will not concern us during this thesis
    \item \textbf{Tools} that will allow us to apply D.P. in datasets.
\end{itemize}

We are going to use the tools available, in order to apply differential privacy in a dataset of our own, guided of course by the mechanisms provided by diffprivlib. First, we are going to take a look at the dataset that we are going to use going forward.

\subsection{Setup of the mechanism}

The first step in order to test the library, is to setup the mechanism by defining its properties and parameters.

\subsubsection{Bounds' Selection}

One of the most important aspects for us if we want to apply DP algorithms, is to define the bounds, i.e. the range that a variable can be in. It would be very convenient in our case to just take the tuple of the smallest and the largest value in the column that we are interested in. However, in the real world, the person who asks the queries is not supposed to know this info about the dataset. Thus, since a solution is not provided by the library, we must define our own bounds by guessing the lowest and the highest values in the fields that we want to examine.

Thus, the user must have somewhat of a previous knowledge regarding the dataset, in order to decide the minimum and maximum value. Those values do not need to be precise, although the more close they are to real ones, the best the protocol will function. At the same time, we must be sure that during our selection we do not leave some of the dataset's values outside of the bounds, as they will be ignored in the final results.

\subsubsection{Privacy Budget}

In this form of D.P., someone trying to breach the users' privacy, could theoretically ask an infinite number of questions, and thus each time gain more and more information about their private data. In the same manner, an untrusted user of the library could ask the same question many times, aiming to determine how much noise is added each time, in order to find out the actual answer to the query, as the way that the noise is drawn is already known.

In order to eliminate this problem, a special parameter call the \emph{privacy budget} is implemented. The library offers the ability to initialize this budget before asking any queries. During the queries, this budget is each time decreased, according to how much data the answer to the question reveals. For example, the answer to the "mean value of the charges for a surgery", costs less than the answer to the "histogram values of heart transplant surgeries in the West coast".

This parameter is implemented by the library as the budget accountant. This variable tracks the privacy spent, so that our system is not left exposed after lots of "expensive queries". The system will allow someone to ask one question that uses the whole privacy budget, or a series of questions whose total impact is less or equal to the initial budget.

\subsection{Testings' goal}

When applying D.P. mechanisms to our data, we provide the privacy settings of our choice (epsilon variable), and obtain an answer to each of our questions. Thus, in our testings, our goal is to \emph{determine the accuracy of the answers}, given a specific ε, or some other settings, and comparing them to the true answer, using some metrics. Those metrics are different for each query type. In this section we are going to focus on two types of queries: statistical, and histograms.

\subsection{General techniques}
In each one of our following testings, we are going to run the query \emph{many times}. As we already know, D.P. relies on probabilistic algorithms that can some times produce extreme results. This may be rare, but we want our testings and conclusions to be accurate. So, we are going to run each query 100 times, and return as a result the mean value of those runs.

\subsection{Statistical Queries}

The first type of queries that we are going to test are those that answer questions like "What is the mean cost of a surgery?", or "What is the largest fee paid by medicare for a transplant?", known as statistical queries.

\subsubsection{Metrics used}

In the case of statistical queries, their answer is usually a real number, so in order to check their alteration with the true answer, we are going to take into account the \emph{absolute difference between the truth and the query answer}.

\subsubsection{Bounds Definition}

We are considering fees for surgeries in our example, thus a logical lower bound would be 0\$ (surgeries could be done pro bono too!), and an upper bound would be 1 million dollars. Either way, we are trying to be extreme with our picks, in order to not find ourselves in the unfortunate situation that a value taken into consideration by the DP query would be out of bounds.

\subsubsection{The identity of the testing Dataset}

The dataset chosen to test the library, is the publicly available "Surgery Charges Across the U.S.", that contains many different kinds of surgeries in a plethora of different hospitals. Our goal is to protect each hospital's data when it comes down to a specific surgery, while helping a patient choose one, depending on the charges that can be found all over the United States. The data provided in this dataset, is going to help a potential patient balance his need of top care, and the need to spend less money. The columns contained in the dataset are:

\begin{itemize}
    \item Surgery code and definition
    \item Provider hospital name
    \item Provider city
    \item Average total payments
    \item Average medicare payments
\end{itemize}

We are going to focus on the last two columns, in order to approximate the charges of a surgery. The above table gives us an image of the containers of the dataset.

The dataset contains a total of 200,000 entries, a more than satisfying number for running D.P. algorithms.

\begin{table}[!htb]

    \caption{"Surgery Charges Across the U.S." dataset columns}
    \label{numbers}

    \begin{tabular}{| c | c | c | c | c| c |}
      \hline
      ID & Surgery Type & Hospital Name & Hospital City & Total & Medicare \\
      \hline
      1 & TRANSPLANT & MAYO CLINIC & PHOENIX & \$240422.80 & \$133509.55\\
      \hline
      2 & ECMO &  GROSSMONT HOSP & LA MESA & \$193617.86 & \$192003.43 \\
      \hline
      3 & CRANIOTOMY & STANFORD HOSP &  STANFORD & \$32597.87 & \$29347.12  \\
      \hline
    \end{tabular}

\end{table}

\subsubsection{General Dataset Utilities Queries}

Our first experiment is just to ask for some of the utilities of the dataset, and specifically its cost column: the \emph{mean value}, the \emph{variance}, the \emph{sum} and the \emph{standard deviation} values of the surgeries' cost.

All of those queries can be executed using the following command (specifically for the mean value query):
\bigskip

\begin{lstlisting}[language=Python]
mean_with_dp = dp.tools.mean(df["Average_Total_Payments"].tolist())
\end{lstlisting}
\bigskip

where the dataframe column is the one containing the cost of each surgery, and its values should be in a list in order for the library to function.

By running the above mentioned queries, we got the following results:

\begin{table}[!htb]
    \centering
    \caption{General Queries results for Surgeries Dataset}
    \label{numbers}

    \begin{tabular}{| c | c | c |}
      \hline
      Query & True answer & Private Answer \\
      \hline
      Mean value & 13168.5 & 13167.3 \\
      \hline
      Sum & 262754253.1 &  262935459.3 \\
      \hline
      Variance & 262754253.1 & 261940796.5\\
      \hline
      Standard Deviation & 18855.1 & 25825.0\\
      \hline
    \end{tabular}

\end{table}

The answers are almost perfect, for example on the mean value, considering that the cost is thousands of dollars, and the error is just 1 dollar. The simplicity of the query just lies to the following instruction:

However, this is just a simple example, executed only once, hence it can not provide us with safe conclusions for the library. In order to do so, we are going to run more complicated examples moving forward.

\subsubsection{Lowering the Dataset size}

As we mentioned above, the entries that the dataset contains are a very large number, and we know as a fact that D.P. functions well when this is the case. How is the library going to respond though if the dataset size is smaller? We are going to run for 4 different values of epsilon, and get the results for an increasing number of entries, starting from 10, and moving to 2000. Thus, the X axis of each plot represents the increasing dataset size, the Y axis the accuracy error, and each plot has a title of the epsilon setting used for the measurements. We can see the results in \textbf{Figure 3.1} below.

\begin{figure}[!htb]\centering
    \includegraphics[width=1\textwidth]{images/increasing_ds_size.png}
    \caption{Accuracy Error for Increasing Dataset Sizes}
\end{figure}

By observing the plots, we can make a couple of conclusions:

\begin{itemize}
    \item \emph{The smaller the epsilon gets, the bigger the accuracy error in the case of small datasets.} This, according to the definition makes sense, because small epsilon indicates higher privacy, thus for small datasets it can mean lower accuracy, due to the high amount of noise added.
    \item \emph{The accuracy error stabilizes near 0 as the size of the dataset gets over 1000 entries.} Of course, depending to the epsilon value, this point could be earlier in the dataset sizes, as we observe for ε $= 1$. This again lays in the above mentioned property of the definition.
\end{itemize}

\subsubsection{Epsilon measurements}

The most important aspect when applying Differential Privacy to a dataset, is the selection of epsilon. This number is held accountable of the trade-off that DP offers: how much accuracy are we going to sacrifice in order to have less privacy loss, and vice versa. Given the library and our surgeries' costs dataset, we are going to measure the accuracy changes with the selection of different epsilon values.

The size of the dataset is too big, and thus the computation of the sensitivity will take too long. We assume that the data are somewhat equally distributed, thus we chose only 1\% of the dataset (which is a significant number of members), to take part in our sensitivity calculations.

In order to observe if the library functions well, we are going to compare the results produced with the \emph{theoretical bounds of the Laplace mechanism}.

In theory, the accuracy of an ε-differential private query, when using the Laplace distribution, is equal to the sensitivity of the query divided by epsilon, thus $$\frac{\Delta f}{\epsilon}$$ where $\Delta f$ denotes the sensitivity, and ε is our current privacy setting.

Sensitivity is defined as the maximum difference that can be found if we alter a single entry in the dataset. We are interested in global sensitivity in order to conduct our testings, and is defined as:
    \begin{align*}
        \Delta f = \frac{upper - lower}{length(DB)}
    \end{align*} where upper and lower are the highest and lowest values of the column we are interested in our dataset, and length is the size of the database.

In order to compute the local sensitivity, we must check the maximum difference in the query result that occurs if we remove a single person from the database. We are going to define a function to do so, so we can check the theoretical bounds during our tests.

The results of those testings are shown in the \textbf{Figure 3.2}.

\begin{figure}[!htb]\centering
    \includegraphics[width=1\textwidth]{images/epsilon_measurements.png}
    \caption{Real and Optimal Accuracy Error for Increasing epsilon values}
\end{figure}

It is clear that as we increase the epsilon value, the privacy loss gets bigger. On the other hand, if epsilon is too small, as we can see in the above plot, extreme errors in accuracy in our queries will emerge.

The optimal value of epsilon varies, there is no general rule for the perfect epsilon. It depends on many different aspects, such as:

\begin{itemize}
    \item The noise generated by the probabilistic mechanism used
    \item The implementation of the algorithm
    \item The size of the dataset
    \item The query itself.
\end{itemize}

Moreover, the selection of epsilon depends on the dataset. For example, we might have a dataset that is extremely important to have minimal privacy loss. In that case, we will opt to use a rather small epsilon, thus we do not disclose the sensitive data included. An other dataset could be less sensitive, but the analyst might have a need for extreme accuracy every time, so the epsilon selected should be rather big.

Regarding the comparison with the Laplace bounds, we can see that the library's query performs significantly well throughout the different epsilon values selected. This is due to the fact that IBM uses the same formula to compute the sensitivity, as well as a similar mechanism (Laplace truncated noise), in order to answer to our queries.

\subsection{Histogram Queries}

Histogram graphs are a very handy way to visualize numerical data, compare different values of a specific field, and thus extract conclusions about the dataset. We are going to study the `diffprivlib`'s method of creating an histogram, and its accuracy when changing the epsilon factor.

The IBM DP library offers a differential private way to create histograms. The difference with the simple queries that we tested, is that now, \emph{geometric truncated} noise is added in order to satisfy DP.

\subsubsection{Metrics Used}
The result of a histogram query on a dataset is a vector containing how many entries belong on a specific range. Thus, the comparison between 2 histograms can be held out by comparing those vectors.

There are plenty of metrics used to compare vectors, but we are going to focus on the \emph{Euclidean Metric} and the \emph{Kantorovich metric} (also known as Wasserstein or EMD metric).

The Euclidean metric is one of the most simple metrics, as it takes into account the distance between each pair of the two vectors. Thus, the distance of the vectors is defined by:

\begin{align*}
    d = \sqrt{\sum_{i=1}^n (x_i - y_i) ^ 2}
\end{align*}

where $n$ are the total elements of the vectors (must be of equal size), and $x$ and $y$ are the 2 vectors that we are comparing.

The Kantorovich metric is a more complex one, as it examines the cost to move a specific quantity of the one vector to the other, in order for the two to be similar, and thus figuring out their distance. For example, the Kant. distance is larger if we have the vectors $[0 0 0 1]$ and $[1 0 0 0]$ than having the vectors $[0 0 0 1]$ and $[0 0 1 0]$, as in the first case the first element of the histogram is moved 3 places in order to be similar to the second one. In order to determine the cost, a base metric is used, which in our case will be the euclidean.

The way that the metric works in a naive approach, is explained in \textbf{Figure 3.3.}

\begin{figure}[!htb]\centering
    \includegraphics[width=0.7\textwidth]{images/emd.png}
    \caption{Kantorovich Metric Application on 2 histograms}
\end{figure}

This metric is much more suitable for D.P. as we are not just interested in the alteration of the results, but on the amount of alteration. If, for example, we are examining a histogram that contains the age of a hospital's patients, it is not the same for the private histogram to deem a 10 year old patient as 90 year old, than deeming him as 11.

In order for the Kantorovich metric to be computed, we must solve a Dynamic Programming Problem, and make complex calculations, which are beyond the subject of this thesis. There are many implementations of the metric, and since IBM's library is written in Python, we are going to use the QIF library from [13] which is also available as a Python library.

\subsubsection{Bounds Definition}

In the following testings we are examining salaries, thus we must set the lower and the higher bounds such as to support the most extreme salaries that could possibly be present in the dataset. Therefore, the lower bound is set to 0 dollars, and the higher to 500,000 dollars.

\subsubsection{The identity of the testing Dataset}

During our histogram testings, we are going to use a different dataset, which contains  sensitive data regarding employee's salaries in the state of Baltimore, while stating other facts about the members of the dataset.

The columns contained are:
\begin{itemize}
    \item Name
    \item Job Title
    \item Annual earnings
    \item Gross earnings
\end{itemize}

We are going to focus on the last 2 columns, containing the employees' salaries. The dataset has 13000 entries, which are more than enough in order for our histograms to be realistic and accurate. The above table gives us an image of the containers of the dataset.

\begin{table}[!htb]
    \centering

    \caption{"Baltimore State Employees Salaries" dataset columns}
    \label{numbers}

    \begin{tabular}{| c | c | c | c |}
      \hline
      Name & Job Title & Annual Earnings & Gross \\
      \hline
      Aaron,Kareem D & Utilities Inst Repair I	 & 32470.0 & 25743.94 \\
      \hline
      Abadir,Adam O	 & Police Officer &  60200.0 & 57806.13  \\
      \hline
      Abbeduto,Mack & Council Technician & 53640.0 &  59361.55 \\
      \hline
    \end{tabular}
\end{table}

\subsubsection{Simple queries}

At first, we are going to run simple instances of histogram queries, so we can take a look at the amount of error that we expect moving forward, and also familiarize with the execution of the commands. To create a private histogram using the library, we must provide the bins. Those should be identical to our bounds, that we have earlier defined. While creating the non-private histograms we are going to use the same bounds, in order for our comparisons to make sense. The creation of the bins and the private histogram are achieved using the following Python instructions:

\bigskip
\begin{lstlisting}[basicstyle= \footnotesize,
language=Python]
bins = np.linspace(0, 300000, 20)[1:]
result = dp.tools.histogram(input_list, bins = bins,
         epsilon = epsilon, range=bounds_range)[0]
\end{lstlisting}
\bigskip

where epsilon is the float number selected as the epsilon parameters, and bounds range is the tuple of our dataset bounds.

The result of the execution of both the private and the non-private histogram queries for the annual earnings column are shown below, in the \textbf{Figure 3.4}.

\begin{figure}[!htb]\centering
    \includegraphics[width=1\textwidth]{images/simple_hists.png}
    \caption{Private and Non-private histograms for the Salaries Dataset}
\end{figure}

The results are quite impressive. This is probably due to the large dataset size: we are not able to locate small changes. In order to do so, we are going to check our error using the accuracy error function that we have defined.

While applying our metrics, the Euclidean error is $14.81$, and the Kantorovich error is $91.63$. The euclidean distance error determines how many entries were wrongly classified in a bin (in average), when the differentially private query was run. So, less than 20 people out of 13.000 were wrongly classified, whereas their privacy was secured. This is quite a good trade-off!

\subsubsection{Epsilon Measurements}

Once again, we are going to run the histogram queries for different values of epsilon, in order to check their behavior as the parameter increases. The results of the execution for the Euclidean metric are shown in the \textbf{Figure 3.4}, and for the Kantorovich metric in the \textbf{Figure 3.5}.

\begin{figure}[!htb]\centering
    \includegraphics[width=1\textwidth]{images/hist_metrics_euclidean.png}
    \caption{Euclidean Metric Error for increasing values of epsilon}
\end{figure}

\begin{figure}[!htb]\centering
    \includegraphics[width=1\textwidth]{images/hist_metrics_kantorovich.png}
    \caption{Kantorovich Metric Error for increasing values of epsilon}
\end{figure}

We observe that the error curves follow the same ratio as the previous ones, indicating what we already know, that the error decreases as the epsilon values increase. This is another case where the library produces accurate and definition-aware results.

\subsubsection{Histogram queries in theory}
Based on [1] the histogram queries are very high sensitivity queries, thus a slight change to the bounds could be critical for their result.

The authors suggest that we use noise generated by the Laplace mechanism, but with a slight change. In detail they suggest the following:

\textit{"In the special (but common) case in which the queries are structurally disjoint we can do much better — we do not necessarily have to let the noise scale with the number of queries. An example is the histogram query. In this type of query the universe $N^X$ is partitioned into cells, and the query asks how many database elements lie in each of the cells. Because the cells are disjoint, the addition or removal of a single database element can affect the count in exactly one cell, and the difference to that cell is bounded by 1, so histogram queries have sensitivity 1 and can be answered by adding independent draws from $Lap(\frac{1}{\epsilon})$ to the true count in each cell."}
\clearpage

\subsection{Conclusions}

After a rather satisfying amount of testing in a large, real world dataset, we suggest that the IBM DP library has quite impressive results when it comes down to the trade-off between privacy and accuracy during both simple counting queries and histogram ones. However, we are cautious about 2 problems that were observed while using the library:

\begin{itemize}
 \item \emph{Bounds checking}. The user must define himself the bounds, a fact that causes for speculations on the variance of the values in the dataset. It would be convenient to take the lowest and highest value in the field that we are examining, in fact that is how IBM demonstrates those examples, but that violates the rule that prevents the user from having any info of the dataset before the DP processing.

 \item \emph{Non-DP preprocessing}. If we ask complicated queries (ex Surgeries performed in Stanford), the library does not offer a way to preprocess the data, thus we trust python in doing so, which results in a non-DP way of shrinking the dataset. The result obtained is of course differential private, but what happens if the dataset has only 1 record in it? That, while being a very extreme case, violates the definition of differential privacy.
\end{itemize}

The bounds' problem can be solved if the users have prior knowledge in the domain of the dataset, in order to safely define the bounds themselves.

The Non-DP preprossessing is a more tricky one, that needs special techniques in order to cover the edge case mentioned. However, we are going to check such techniques in the following section, which is based on returning the whole dataset after the application of D.P.