\setlength{\footskip}{8mm}

\chapter{Incremental Behavior Modeling and Suspicious
Activity Detection}
\label{ch:incremental}

\textit{We propose and evaluate an efficient method for automatic
identification of suspicious behavior in video surveillance data that
incrementally learns scene-specific statistical models of human
behavior without requiring storage of large databases of training
data. The approach begins by building an initial set of models
explaining the behaviors occurring in a small bootstrap data set. The
bootstrap procedure partitions the bootstrap set into clusters then
assigns new observation sequences to clusters based on statistical
tests of HMM log likelihood scores. Cluster-specific likelihood
thresholds are learned rather than set arbitrarily. After
bootstrapping, each new sequence is used to incrementally update the
sufficient statistics of the HMM it is assigned to. In an evaluation
on a real-world testbed video surveillance data set, we find that
within one week of observation, the incremental method's false alarm
rate drops below that of a batch method on the same data. The
incremental method obtains a false alarm rate of 2.2\% at a 91\% hit
rate. The method is thus a practical and effective solution to the
problem of inducing scene-specific statistical models useful for
bringing suspicious behavior to the attention of human security
personnel.}

\section{Introduction}
\label{incremental-intro}

Human monitoring of video surveillance channels is increasingly
ineffective as the number of channels increases.  \textit{Anomaly
detection} aims to alleviate this problem by filtering out typical
events and only bringing suspicious events to the attention of human
security personnel.

We propose a new incremental method for learning behavior models and
detecting anomalies using hidden Markov model (HMM)-based clustering
on a small bootstrap set of sequences labeled as normal or suspicious.
After bootstrapping, we assign new observation sequences to behavior
clusters using statistical tests on the log likelihood of the sequence
according to the corresponding HMMs.  We label a sequence as
suspicious if it maps to an existing model of suspicious behavior or
does not map to any existing model.  After labeling, we either
incrementally update the sufficient statistics for the most likely
HMM's parameters or create a new HMM for the new sequence.

In an evaluation on a real-world video surveillance situation, we find
that the method is very effective at identifying suspicious
behavior. A batch method achieves a false positive rate of 8.6\% at a
100\% hit rate. The incremental method decreases the false alarm rate
to 2.2\% at a 91\% hit rate, and detection threshold tuning can
achieve an equal error rate of 95.8\% or a false alarm rate of 3.7\%
at a 100\% hit rate.  The experimental results demonstrate that our
incremental method is better able to learn new forms of normal
behavior, leading in turn to more effective anomaly detection. The
method is thus a practical and effective solution to the problem of
inducing scene-specific statistical models useful for bringing
suspicious behavior to the attention of human security personnel.

\section{Related Work}
\label{incremental-related-work}

In this section, we explore methods for incremental learning of
behavior models and active detection of anomalous deviations from the
learned typical behavior.  We build upon previous research in human
behavior
understanding \shortcite{davis98activity,masoud03recognition,cao04motion,sminchisescu05motion,li06behavior,du06activity,wang06gesture},
the use of dynamic graphical models such as hidden Markov models
(HMMs) and conditional random fields (CRFs) for behavior understanding
in specific
scenarios \shortcite{yamato92hmm,gao04dining,xiang05profiling,sminchisescu05motion,vail07activity},
and classification of behavior in videos to a priori known categories
\shortcite{nair02surveillance,lee03modeling,wu05behavior,arsic07behavior}.

Several research groups have investigated methods for video clustering
\shortcite{sakoe78dtw,zhong04detection,xiang05profiling,hautamaki08clustering,xiang08clustering,swears08clustering,xiang08modeling}. Especially effective are those methods using HMMs
to model different classes of behavior in video surveillance
data \shortcite{xiang05profiling,swears08clustering,xiang08modeling}.

There has been some work on anomalous time series detection outside
the context of video surveillance using relatively simple statistical
models \shortcite{turner10detection,yao10detection}.  However, these
methods learn single comprehensive models without addressing the
special requirement in video surveillance to capture an extremely wide
variety of typical behaviors.  This requires construction of a
collection of statistical models.  Intrusion detection requires
similar diversity in the statistical models.  Some of the work uses
ensembles of classifiers \shortcite{rasoulifard08incremental}, but
most of the research has focused on anomaly detection methods using
incremental
clustering \shortcite{burbeck04adwice,burbeck07incremental,ren08incremental,zhong08incremental,ekman08incremental}.
All of these methods generally work by comparing a new pattern against
a collection of clusters representing historically typical behavior
and classifying the new pattern as an anomaly if its distance from the
nearest cluster is above threshold.  We take a similar approach, with
a specific approach to clustering, distance measurement, and threshold
learning specific to the case of surveillance video.

Some incremental learning approaches have been applied to human
behavior modeling. \shortciteA{vasquez09incremental} incrementally
model vehicle and pedestrian trajectories using growing hidden Markov
models (GHMMs). The method builds a topological map and corresponding
GHMM using an incremental variant of the Baum-Welch algorithm. For
each new node in the map, a corresponding state and connectivity are
added to the GHMM. \shortciteA{kulic10incremental} model movement
primitives using HMMs and piece the primitives together with a higher
level HMM. Similar to Vasquez et al., the method incrementally adds
new hidden states when a new primitive model is built.

Besides human behavior modeling, \shortciteA{stenger01topology} first
propose the incremental Baum-Welch (IBW) algorithm on continuous
observations for background modeling. This algorithm is a
straight-forward adaptation of the original Baum-Welch algorithm to
incremental learning. They update the parameters by taking into
account the sufficient statistics. Although they use a single
observation sequence to update the parameters, but the sufficient
statistics still can represent the information computed for all of the
observed data. Later, \shortciteA{larrahondo05incremental} adapt this
algorithm to model discrete
observations. \shortciteA{cavalin08incremental} use the ensemble
training (ET) algorithm \shortcite{mackay97hmm} to the incremental
learning problem in alphanumeric character recognition. This algorithm
has never been used in incremental learning setting. The authors claim
that it is easy to adapt to this kind of setting since the parameters
of the final HMM are computed for each observation sequence
independently.

The existing work most similar to ours is the incremental learning
approach of \shortciteA{xiang08incremental}, who also model activity
in a scene incrementally after initialization from a small bootstrap
dataset. They build explicit probabilistic models of normal and
abnormal activity patterns in the bootstrap set then classify new
observations using a likelihood ratio test.  The models are global
joint models over the entire scene.  It is not clear how the
likelihood ratio test could handle completely new anomalous events
with low likelihoods under both models, and it is not clear whether a
global model is appropriate for detecting isolated anomalous behavior
in a scene.

Our method does not require an a priori model of anomalous human
behavior. It constructs an ensemble of simple models, adding to the
ensemble when the existing set is insufficient to represent new
cases. We associate each new observation with an individual model
using separate statistical tests on the observation's likelihood
according to each individual model.  Rather than globally modeling the
entire scene, our method takes a more local approach, separately
analyzing each individual moving object's behavior.  In an
experimental comparison of the two methods, we find statistical
testing of the likelihood to be superior to likelihood ratio tests,
and we find the local representation to be superior to the global
representation.

%A preliminary report on our clustering method appeared in a conference
%paper \shortcite{kan10clustering}, but it was limited to tracking only
%one human in a scene.  In this chapter, we have extended the work to
%include multiple human tracking, anomaly detection, and incremental
%learning.

The main contribution of the work described in this chapter is a new
modeling method for detection of anomalous events in surveillance
video based on simply-structured models and incremental learning that
is demonstrably able to evolve the models over time to adapt to new
behavior and also outperforms current techniques on the same data set.

\section{Behavior Model Bootstrapping}
\label{sec:incremental-bootstrap}

Simply-structured statistical models cannot hope to capture the
diversity of ``normal'' behavior in a given scene --- multiple models
are required.  However, it is difficult to determine how many and what
activities will occur in a scene a priori or based on manual
monitoring.  We therefore propose a method to automatically determine,
from a small bootstrap set, the common similar behaviors occurring in
a scene. The entire process of the method used in this section is
previously described in Section \ref{sec:batch-modeling} including
blob extraction, appearance-based blob tracking, blob feature vector
discretization, and behavior clustering.

This process results in a set of $K$ different typical behavior
clusters ${\cal C} = \{ c_1, c_2, \ldots, c_K \}$ with a set of $K$
corresponding HMMs ${\cal M} = \{ M_1, M_2, \ldots, M_K \}$.

\section{Anomaly Detection}
\label{incremental-detection}

We classify new sequences as abnormal if the most likely HMM for the
input sequence is associated with one of the abnormal
models, \textit{or} the $z$-scored per-observation log likelihood of
the sequence under that most likely model is less than a global
empirically determined threshold $\theta_z$. We previously described
this method in Section~\ref{sec:batch-anomaly-detection}.

One problem with this approach is that it never learns any new typical
behavior over time.  Given a small bootstrap set, the method may work
well for a short period of time, but as typical behavior evolves and
becomes more diverse, the false alarm rate may increase.

The naive solution to this problem is to retain all of the observation
data and retrain the system periodically.  However, this would require
storing all of the incoming data, requiring too much disk space. For
lifelong anomaly detection, we need a more adaptive approach that
learns new behaviors over time but does not require storing all of the
historical data. We solve this problem in the next section.

\section{Incremental Behavior Modeling for Anomaly Detection}
\label{incremental-inc-modeling}

In this section, we extend the basic anomaly detection method
previously described in Section~\ref{incremental-detection} using the
incremental maximum likelihood (IML) algorithm for HMMs
\shortcite{gotoh96incremental}. The incremental expectation maximization (EM)
procedure was originally proposed by \shortciteA{neal93em}. The basic
idea is to update the sufficient statistics for the parameters of the
HMMs as new events occur. In our approach, given an initial set of
observation sequence clusters and corresponding HMMs modeling the
behavior in the scene, when a new observation sequence arrives, we
first determine which cluster it falls into by performing statistical
tests on the sequence's likelihood according to each HMM model. Since
likelihood scores for different HMMs cannot be compared directly, we
select the most likely model based on the same $z$-scored
per-observation log likelihood measure introduced in
Section~\ref{sec:clustering-behavior-clustering} for the initial
behavior model clustering.

When the most likely model for a new observation sequence is a model
for a cluster of atypical behaviors, or when the new observation
sequence's likelihood according to the most likely model is not
sufficiently high, we raise an alert. If the operator gives feedback
that a new observation sequence marked as suspicious by the system is
in fact not suspicious, we create a new typical behavior cluster and
corresponding HMM for the false alarm sequence. If the likelihood
according to the most likely model is sufficiently high for one of the
pre-existing models and the operator confirms that the classification
is correct, we incrementally update the sufficient statistics for that
model; otherwise, we start a new cluster for the sequence, build a new
HMM for the new cluster, and label it according to the operator
feedback. This approach allows the system to raise alerts when unusual
behaviors are detected while adapting to gradual changes in behavior
patterns over time. The method is detailed in Algorithm
\ref{anomaly-detection-with-iml-algorithm}.

\begin{algorithm}
  \caption{Anomaly Detection with Incremental Learning}
  \label{anomaly-detection-with-iml-algorithm}
  \begin{algorithmic}
    \REQUIRE $\vec{O}$: behavior sequence
    \REQUIRE ${\cal M}$: set of HMMs
    \REQUIRE ${\cal S}$: set of sufficient statistics
    \ENSURE $\widetilde{\cal M}$: set of revised HMMs
    \ENSURE $\widetilde{\cal S}$: set of revised sufficient statistics

    \STATE $\widetilde{\cal M} \gets {\cal M}; \; \widetilde{\cal S} \gets {\cal S}$
    \STATE ${\cal M}_{ab} \gets \{ M \mid M \in {\cal M} \text{~and $M$ is marked abnormal} \}$

    \STATE $( M_{ml}, S_{ml}, L_{ml} ) \gets
    \textsc{Find-Most-Likely-Model}(\vec{O}, {\cal M})$
    \STATE $d_{\text{feedback}} \gets \emptyset$

    \IF{$M_{ml} \in {\cal M}_{ab}$ or $L_{ml} \leq \theta_z$}
      \STATE $d_{\text{feedback}} \gets \textsc{Alert-Security-Personnel}(\vec{O})$
    \ENDIF

    \IF{$L_{ml} > \theta_z$ and $d_{\text{feedback}} \neq$ false positive}
      \STATE $( M, S ) \gets \textsc{Incrementally-Update}(M_{ml}, S_{ml})$
      \STATE $\widetilde{\cal M} \gets \{ \widetilde{\cal
      M}\;$\textbackslash$\;M_{ml} \} \cup \{ M \}; \; \widetilde{\cal S} \gets \{ \widetilde{\cal
      S}\;$\textbackslash$\;S_{ml} \} \cup \{ S \}$
    \ELSE
      \STATE $( M, S ) \gets \textsc{Create-New-Model}(\vec{O})$

%      \IF{$L_{ml} \le \theta_z$}
      \IF{$L_{ml} \le \theta_z$ and $d_{\text{feedback}} \neq$ false positive}
        \STATE Mark $M$ as abnormal
      \ENDIF

      \STATE $\widetilde{\cal M} \gets \widetilde{\cal M} \cup \{ M \}; \; \widetilde{\cal S} \gets \widetilde{\cal S} \cup \{ S \}$
    \ENDIF
  \end{algorithmic}
\end{algorithm}

When a new behavior sequence $\vec{O}$ arrives, we find the cluster
$k$ whose HMM $M_{ml}$ assigns $\vec{O}$ the highest log likelihood
$L_{ml}$. $\vec{O}$ is considered consistent with $c_k$ if $L_{ml}$ is
above threshold $\theta_z$ and the operator does not mark $\vec{O}$ as
a false positive.  In this case, we use $\vec{O}$ to update the
sufficient statistics ${\cal S}^{(k)} = \{ \vec{\Gamma}^{(k)},
\vec{\Xi}^{(k)} \}$ for cluster $k$, allowing us to throw away the
original observation sequence.  $\vec{\Gamma}^{(k)}$ accumulates the
$\vec{\gamma}^{(k)}$ probabilities (defined in Rabiner's tutorial;
Rabiner, 1989\nocite{rabiner89hmm}) needed to estimate the emission
and transition probabilities of $M_k$.  The update rule is simply
\[
  \vec{\Gamma}_{\text{new}}^{(k)} = \alpha
  \vec{\Gamma}_{\text{old}}^{(k)} + \vec{\gamma}^{(k)},
\]
where $\alpha$ is a forgetting factor \shortcite{nowlan91incremental}.
$\vec{\Xi}^{(k)}$ accumulates the $\vec{\xi}^{(k)}$ probabilities
needed to estimate the transition probabilities.  It is updated the
same way $\vec{\Gamma}^{(k)}$ is. In the experiments reported upon in
this work, we use $\alpha = 0.9$.

When an HMM is first trained, we initialize the sufficient statistics
${\cal S}^{(k)}$ over all of the observation sequences in cluster
$c_k$ using batch training. During incremental learning, in each
iteration of the E-step, we calculate the sufficient statistics
$S^{(k)}$ for the single input observation sequence then update ${\cal
S}^{(k)}$ accordingly to be used in the M-step.  We repeat the E and M
updates for a fixed number of iterations $I$. \shortciteA{neal93em}
prove monotonic convergence of incremental EM under certain
circumstances and find that in practice it is much faster than the
batch EM algorithm. In our case, since we are incorporating a
completely new observation sequence at each increment, the likelihood
over all the sequences may increase or decrease.  The method is
summarized in Algorithm \ref{incremental-em-algorithm}.

\begin{algorithm}[t]
  \caption{Incremental EM Algorithm}
  \label{incremental-em-algorithm}
  \begin{algorithmic}
    \REQUIRE $\vec{O}$: behavior sequence
    \REQUIRE $M$: HMM model
    \REQUIRE ${\cal S} = \{ \vec{\Gamma}, \vec{\Xi} \}$: set of sufficient statistics
    \ENSURE $M^*$: revised HMM model
    \ENSURE ${\cal S}^* = \{ \vec{\Gamma}^*, \vec{\Xi}^* \}$: set of revised sufficient statistics
    \STATE $M^* \gets M$
    \FOR{$i = 1 \to I$}
      \STATE \COMMENT{E-step}
      \STATE $(\vec{\gamma}, \vec{\xi}) \gets \textsc{Compute-Sufficient-Statistics}(\vec{O}, M^*)$
      \STATE $\vec{\Gamma}^* \gets \alpha \vec{\Gamma} + \vec{\gamma}$; \;
      $\vec{\Xi}^* \gets \alpha \vec{\Xi} + \vec{\xi}$
      \STATE \COMMENT{M-step}
      \STATE $M^* \gets \textsc{Reestimate-Model-Parameters}(M^*, \vec{\Gamma}^*, \vec{\Xi}^*)$
    \ENDFOR
  \end{algorithmic}
\end{algorithm}

\section{Experimental Results}
\label{incremental-results}

We recorded video\footnote{Freely available for others to experiment
with at \url{http://www.kanouivirach.com/#downloads}.}
from the scene in front of a building during working hours
(9:00--17:00) for one week.  We started a new event whenever the
number of foreground pixels exceeded 300.  We used a threshold of
0.995 for shadow detection NCC.  We discarded any blobs smaller than
300 pixels in area. We obtained 423 video segments containing 660
observation sequences.

Figure \DIFdelbegin \DIFdel{\ref{fig:example-behavior} }\DIFdelend \DIFaddbegin \DIFadd{\ref{fig:batch-example-behavior} }\DIFaddend shows examples of four common
activities: people entering the building, people leaving the building,
people parking bicycles, and people riding bicycles out\DIFaddbegin \DIFadd{, and people walking
pass by or having interactions with each other}\DIFaddend .
For the purpose of analyzing results, we manually labeled each of the videos
with the ``normal'' categories ``walk-in,'' ``walk-out,''
``cycle-in,'' and ``cycle-out,'' or, for other activities such as
walking around looking for an unlocked bicycle, with the ``abnormal''
category.

\DIFdelbegin %DIFDELCMD < \begin{figure}[t]
%DIFDELCMD <   \centering
%DIFDELCMD <   \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior01.pdf}\label{fig:example-of-scene}}
%DIFDELCMD <   %%%
\DIFdel{\hspace{0.05in}
  }%DIFDELCMD < \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior02.pdf}}
%DIFDELCMD <   %%%
\DIFdel{\hspace{0.05in}
  }%DIFDELCMD < \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior03.pdf}}
%DIFDELCMD <   %%%
\DIFdel{\hspace{0.05in}
  }%DIFDELCMD < \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior04.pdf}}
%DIFDELCMD <   %%%
%DIFDELCMD < \caption[Examples of common human activities in our testbed
%DIFDELCMD <     scene.]{%
{%DIFAUXCMD
%DIFDELCMD < \small %%%
\DIFdel{Examples of common human activities in our testbed
    scene.  (a) Walking in. (b) Walking out. (c) Cycling in. (d)
    Cycling out.}}
  %DIFAUXCMD
%DIFDELCMD < \label{fig:example-behavior}
%DIFDELCMD < \end{figure}
%DIFDELCMD < %%%
\DIFdelend %DIF > \begin{figure}[t]
%DIF >   \centering
%DIF >   \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior01.pdf}\label{fig:example-of-scene}}
%DIF >   \hspace{0.05in}
%DIF >   \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior02.pdf}}
%DIF >   \hspace{0.05in}
%DIF >   \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior03.pdf}}
%DIF >   \hspace{0.05in}
%DIF >   \subfloat[]{\includegraphics[width=0.2\textwidth]{figures/example-behavior04.pdf}}
%DIF >   \caption[Examples of common human activities in our testbed
%DIF >     scene.]{\small Examples of common human activities in our testbed
%DIF >     scene.  (a) Walking in. (b) Walking out. (c) Cycling in. (d)
%DIF >     Cycling out.}
%DIF >   \label{fig:example-behavior}
%DIF > \end{figure}

We performed experiments in three parts: model configuration selection
(finding an optimal set of HMMs to model the bootstrap set), anomaly
detection based on the HMM bootstrap set, and anomaly detection with
incremental learning.

\subsection{Model Configuration Selection}
\label{incremental-model-selection}

Towards model identification, we performed five-fold cross validation
with different bootstrap parameter settings and selected the
configuration with the highest accuracy in separating the normal
sequences from the abnormal sequences on the bootstrap set, as
measured by the false positive rate for abnormal sequences over all
cross validation test folds. (Every bootstrap cluster containing an
abnormal sequence is considered abnormal, so we always obtain 100\%
detection on the bootstrap set; the only discriminating factor is the
false positive rate.) The set of parameters that we varied were 1) the
number of states in each bootstrap HMM (3--8), 2) the number of
symbols/$k$-means clusters (3--8), and 3) the number $N_c$ of deviant
patterns allowed in a bootstrap cluster (1--10). To find the
distribution (parameters $\mu_c$ and $\sigma_c$) of the
per-observation log likelihood for a particular HMM, we always
generated 1,000 sequences of average length (the average length was
computed over the bootstrap set; it was 205 observations in our
experiment). For the rejection threshold $p_c$, we used a
$z$-threshold of 2.0 (i.e., $p_c = \mu_c - 2\sigma_c$), corresponding
to a Type I error rate (probability of misclassifying a sequence
generated by the HMM as not generated by the HMM) of 0.0228.

The setting of the $z$-score threshold is important. With a $z$-score
of 2, on different runs, we find that the method generates 15--30
clusters from our bootstrap set, usually perfectly classifies the
bootstrap set, and performs well on the test set. With a lower
threshold (0.5 to 1.5), only extremely similar series are clustered
together, leading to a large number of clusters and a large number of
false positives on typical behaviors that vary only slightly from what
has been seen in the bootstrap set. With higher thresholds (2.5 to 3),
patterns that are quite dissimilar get clustered, oftentimes leading
to very few clusters that do not cleanly separate the bootstrap set
and have low accuracy on the test set.

To reduce the variance in the measured false positive rate due to
random initialization, we ran the entire cross validation procedure
three times and averaged the results.

\begin{table}[t]
  \caption[Example human behavior pattern bootstrapping
    results.]{\small Example human behavior pattern bootstrapping
    results. We used linear HMMs with five states and seven
    symbols. The model consists of 17 clusters. ``W'' means ``walk''
    and ``C'' means ``cycle.'' For the six clusters containing more
    than one sequence, shown is the distribution of the patterns in
    the cluster over the activities.  The last row shows the
    distribution of the 11 clusters containing only a single sequence
    over the activity categories.}
  \begin{center}
    \begin{tabular}{c|c|c|c|c|c}
      \hline
      Cluster \# & W-in & W-out & C-in & C-out & Other \\
      \hline \hline
      1 & 44 & 0  & 20 & 0  & 0 \\ \hline
      2 & 0  & 38 & 0  & 24 & 0 \\ \hline
      3 & 0  & 0  & 3  & 0  & 0 \\ \hline
      4 & 0  & 0  & 2  & 0  & 0 \\ \hline
      5 & 0  & 0  & 0  & 0  & 6 \\ \hline
      6 & 0  & 0  & 0  & 0  & 2 \\ \hline
      One-seq clusters & 0 & 1 & 6 & 1 & 3 \\ \hline
    \end{tabular}
  \end{center}
  \label{tab:bootstrapping-results}
\end{table}

Based on the false positive rate criterion described above, we
selected the model configuration consisting of five states, seven
symbols, and $N_c = 6$ and trained a new HMM ensemble on all 150
bootstrap sequences.  We repeated the training process several times
until we obtained a model that \DIFdelbegin \DIFdel{obtained perfect accuracy }\DIFdelend \DIFaddbegin \DIFadd{perfectly maps anomalous and typical
sequences to different clusters }\DIFaddend on the bootstrap set.
Table \ref{tab:bootstrapping-results} shows the
distribution of bootstrap sequences across this model's 17 behavior
clusters. We used this model with the remaining 510 sequences in the
anomaly detection experiments.

\subsection{Anomaly Detection (Batch Processing)}
\label{incremental-against-global-method}

Here we evaluate the ability of our anomaly detection method to
identify anomalous events in the data not included in the bootstrap
set. Since the main concern in video surveillance is to detect every
unusual event while minimizing the false positive rate, we calculate
an ROC curve and select the detection threshold yielding the best
false positive rate at a 100\% hit rate. We reported preliminary
results for the proposed method compared to traditional machine
learning algorithms in Chapter~\ref{ch:batch}.

%a conference paper
%\shortcite{kan12detection}.

In this experiment, we compare the proposed method against HMM-based
methods using alternative representations and scoring methods similar
to those of \shortciteA{xiang08incremental}. This allows us to
determine the extent to which different event representations (our
local representation or Xiang and Gong's global representation) and
different scoring methods (our $z$-scoring method or the standard
likelihood ratio test (LRT)) contribute to anomaly detection
performance on our test set. The four methods are as follows:

\begin{enumerate}
  \item Method I: The proposed method.
  \item Method II: The proposed method, but using Xiang and Gong's
    likelihood ratio test \shortcite{xiang08incremental} rather than
    our $z$-scored likelihood method.
  \item Method III: An HMM-based method using a global event
    representation similar to that of Xiang and Gong , with our
    $z$-scored likelihood method.
  \item Method IV: The global method (Method III) using Xiang and
    Gong's likelihood ratio test rather than our $z$-scored likelihood
    method.
\end{enumerate}

%Testing methods II--IV together with our method allows us to determine
%the extent to which different event representations (our local
%representation or Xiang and Gong's global representation) and
%different scoring methods (our z-scoring method or the likelihood
%ratio test) contribute to anomaly detection performance on our test
%set.

\DIFdelbegin %DIFDELCMD < \begin{figure}[t]
%DIFDELCMD <   \centering
%DIFDELCMD <   \begin{tabular}{ccccc}
%DIFDELCMD <     \includegraphics[scale=0.21]{figures/case-1-suspicious-0187} &
%DIFDELCMD <     \includegraphics[scale=0.21]{figures/case-1-suspicious-0208} &
%DIFDELCMD <     \includegraphics[scale=0.21]{figures/case-1-suspicious-0225} &
%DIFDELCMD <     \includegraphics[scale=0.21]{figures/case-1-suspicious-0249} &
%DIFDELCMD <     \includegraphics[scale=0.21]{figures/case-1-suspicious-0267}
%DIFDELCMD <     \\
%DIFDELCMD <     \small %%%
\DIFdel{Frame 187 }%DIFDELCMD < &
%DIFDELCMD <     \small %%%
\DIFdel{Frame 208 }%DIFDELCMD < &
%DIFDELCMD <     \small %%%
\DIFdel{Frame 225 }%DIFDELCMD < &
%DIFDELCMD <     \small %%%
\DIFdel{Frame 249 }%DIFDELCMD < &
%DIFDELCMD <     \small %%%
\DIFdel{Frame 267
  }%DIFDELCMD < \end{tabular}
%DIFDELCMD <   %%%
%DIFDELCMD < \caption[Example anomaly detected by the proposed method (Method
%DIFDELCMD <     I).]{%
{%DIFAUXCMD
%DIFDELCMD < \small %%%
\DIFdel{Example anomaly detected by the proposed method
    (Method I). The sequence contains a person walking around looking
    for an unlocked bicycle.}}
  %DIFAUXCMD
%DIFDELCMD < \label{fig:suspicious-behavior-detected}
%DIFDELCMD < \end{figure}
%DIFDELCMD < %%%
\DIFdelend %DIF > \begin{figure}[t]
%DIF >   \centering
%DIF >   \begin{tabular}{ccccc}
%DIF >     \includegraphics[scale=0.21]{figures/case-1-suspicious-0187} &
%DIF >     \includegraphics[scale=0.21]{figures/case-1-suspicious-0208} &
%DIF >     \includegraphics[scale=0.21]{figures/case-1-suspicious-0225} &
%DIF >     \includegraphics[scale=0.21]{figures/case-1-suspicious-0249} &
%DIF >     \includegraphics[scale=0.21]{figures/case-1-suspicious-0267}
%DIF >     \\
%DIF >     \small Frame 187 &
%DIF >     \small Frame 208 &
%DIF >     \small Frame 225 &
%DIF >     \small Frame 249 &
%DIF >     \small Frame 267
%DIF >   \end{tabular}
%DIF >   \caption[Example anomaly detected by the proposed method (Method
%DIF >     I).]{\small Example anomaly detected by the proposed method
%DIF >     (Method I). The sequence contains a person walking around looking
%DIF >     for an unlocked bicycle.}
%DIF >   \label{fig:suspicious-behavior-detected}
%DIF > \end{figure}

For method I and II, we used the 150-sequence bootstrap sequence set
from Section \ref{incremental-model-selection} as training data and
tested on the remaining 510
sequences. Figure \ref{fig:suspicious-behavior-detected} shows an
example of a sequence classified as abnormal.  When evaluating methods
III and IV, we extract global features rather than individual blob
features. The local method considers the single blob motion at a time,
whereas the global method simultaneously considers multiple blobs
appearing at the same time. The data set for the global method
comprises 401 sequences (366 negatives and 35 positives). We used the
first 150 sequences (139 negatives and 11 positives) for the bootstrap
set and the remaining 251 sequences (227 negatives and 24 positives)
for the test set. In all four conditions, we performed a separate
five-fold cross validation to select the best model configuration as
described in Section \ref{incremental-model-selection}.

\begin{figure}[t]
  \centering
  \includegraphics[width=0.8\linewidth]{figures/roc-ours-vs-global-results}
  \caption[Anomaly detection ROC curves. Solid red, dotted green,
    dashed blue, and dash-dot magenta lines represent ROCs for methods
    I, II, III, and IV, respectively.]{\small Anomaly detection ROC
    curves. Solid red, dotted green, dashed blue, and dash-dot magenta
    lines represent ROCs for methods I, II, III, and IV,
    respectively.}
  \label{fig:roc-ours-vs-global-results}
\end{figure}

\begin{table}[t]
  \caption[Anomaly detection results for the local method with the
    $z$-scoring method and the likelihood ratio test, and the global
    method with the $z$-scoring method and likelihood ratio
    test.]{\small Anomaly detection results for the local method with
    the $z$-scoring method and the likelihood ratio test, and the
    global method with the $z$-scoring method and likelihood ratio
    test. TP, FP, TN, and FN are the number of true \DIFdelbeginFL \DIFdelFL{positive}\DIFdelendFL \DIFaddbeginFL \DIFaddFL{positives}\DIFaddendFL ,
    false \DIFdelbeginFL \DIFdelFL{positive}\DIFdelendFL \DIFaddbeginFL \DIFaddFL{positives}\DIFaddendFL , true \DIFdelbeginFL \DIFdelFL{negative}\DIFdelendFL \DIFaddbeginFL \DIFaddFL{negatives}\DIFaddendFL , and false \DIFdelbeginFL \DIFdelFL{negative}\DIFdelendFL \DIFaddbeginFL \DIFaddFL{negatives}\DIFaddendFL , respectively.
    TPR and FPR stand for true positive rate and false positive rate,
    respectively.}
  \begin{center}
    \begin{tabular}{c|c|c|c|c|c|c}
      \hline Batch method & TP & FP & TN & FN & TPR & FPR \\ \hline \hline
      Local ($z$-scoring) & 24 & 42 & 444 & 0 & 1 & 0.086 \\ \hline
      Local (LRT) & 24 & 486 & 0 & 0 & 1 & 1 \\ \hline
      Global ($z$-scoring) & 24 & 217 & 10 & 0 & 1 & 0.956 \\ \hline
      Global (LRT) & 24 & 223 & 4 & 0 & 1 & 0.982 \\ \hline
    \end{tabular}
  \end{center}
  \label{tab:hmm-based-detection-results}
\end{table}

Figure \ref{fig:roc-ours-vs-global-results} shows ROC curves for
methods I--IV.  The solid red line represents the ROC curve for method
I as we vary the threshold ($z$-score or likelihood ratio) at which a
sequence is considered anomalous. Note that the ROC does not intersect
the point $(0, 0)$ because any sequence that is most likely under one
of the HMMs modeling anomalous sequences in the bootstrap set is
automatically classified as anomalous regardless of the threshold. The
ROC reveals that a $z$-score per-observation log likelihood $\theta_z$
of $-2.293$ achieves zero false negatives at a false alarm rate of
0.086. The dotted green line is the ROC curve for method II, the
dashed blue line represents method III, and the dash-dot magenta line
represents method IV.  Table
\ref{tab:hmm-based-detection-results} shows the detailed performance
of each detection method at 100\% hit rate.

The ROC curves and FP rates show that our method clearly outperforms
the other three methods, but there is also a surprising interaction:
for the global method, $z$-scored likelihood and LRT are equally
effective, but for our local event based method, $z$-scoring is much
more effective than the LRT.  This may reflect that the LRT's use of a
mixture model is less appropriate for the local method than the global
method.

The reason that the local method is better than the global method
(albeit when combined with $z$-scoring of the likelihood) is that many
of the abnormal sequences in the test set tend to be locally similar
to the abnormal sequences in the bootstrap set but globally
different. The local method treats concurrent but spatially separate
local events as being independent, whereas the global method attempts
to construct a joint model over the entire scene.  The global method
might be improved with a larger bootstrap set.

%For example, a person is
%looking for an unlocked bicycle while another person is leaving a
%building. The local method considers the former event as anomalous;
%the latter event as normal. Whereas, the global method considers it as
%only one anomalous event.

The log likelihood ratio test fails to detect completely new anomalous
patterns when the pattern has a very low likelihood according to the
abnormal model and the normal model but has a slightly higher
likelihood according to the normal model. In order for the LRT test to
detect such patterns, we need to adjust the LR threshold, causing
higher false positives (increasing the number of false positives to
about 30--40).  Since our approach creates a new abnormal model for
any low-likelihood observation, regardless of the relative likelihood,
it does not suffer from this problem.

\subsection{Anomaly Detection with Incremental Learning}

In this experiment, we evaluate the use of incremental HMM learning
for anomaly detection. We used the same settings and model
configuration as in the batch method experiments described in Section
\ref{incremental-against-global-method}.  Since we incorporate a completely
new observation sequence at each increment, the likelihood over all
the sequences may increase or decrease. In the current work, we
reestimate $\mu_c$ and $\sigma_c$ after every update by generating
1,000 new sample sequences.  The update process could obviously be
optimized.

Overall results of incremental learning are shown in Table
\ref{tab:iml-detection-results}. Compared to the batch method, the
incremental version of the proposed method (local method with
$z$-scoring) obtains a nearly four-fold decrease in the false positive
rate (2.2\% vs.\ 8.6\%) at the cost of a 9\% decrease in the hit rate.
The results shown in the table are based on a fixed $z$-score
threshold of $\theta_z=2.0$.  To better enable comparison of the
results for batch and incremental learning, we varied $\theta_z$ to
obtain false positive rates at 100\% detection rates and to obtain
equal error rates (EERs). The incremental method achieves a 3.7\%
false alarm rate at a 100\% hit rate, compared to 8.6\% for the batch
method.  It achieves an EER of 95.8\%, compared to 91.8\% for the
batch method.

Overall, these results demonstrate that the incremental algorithm
detects anomalies more effectively with lower false alarm rates.  It
is better able than the batch method to learn new forms of normal
behavior.  New behavior classes are always called anomalous at first,
so completely new behaviors (suspicious or not) will always trigger a
false positive and lead to the creation of a new model trained on the
new sequence.  These one-sequence HMMs tend to generate sequences with
very low variance in the log likelihood. Such models do not often get
updated with new sequences, but they do from time to time. In this
experiment, new sequences were assigned to existing multi-sequence
HMMs 466 times, existing single-sequence HMMs 44 times, and were used
to create new sequences 5 times.  The improvement in performance of
the incremental method over the batch method thus comes first from
tuning the parameters of the existing behavior classes (both typical
and anomalous), but it also comes from the use of additional typical
labels for the false positive anomalous series.

\begin{figure}[t]
  \centering
  \subfloat[]{\includegraphics[width=3in]{figures/cFPR-chunk-results-5-trials.pdf}}
  \subfloat[]{\includegraphics[width=3in]{figures/cTPR-chunk-results-5-trials.pdf}}
  \caption[Comparison of batch learning and incremental learning over
    time.]{\small Comparison of batch learning (dotted red line) and
    incremental learning (solid green line) over time.  (a) Cumulative
    false positive rates. (b) Cumulative true positive rates.  Each
    point is an average of the false positive or true positive rates
    over 5 trials.}
  \label{fig:chunking-results}
\end{figure}

To further understand the performance differences between the batch
and incremental methods, we partitioned the test data sequentially
into chunks and computed the false and true positive rates for each
chunk of observations.  Initially, we set the chunk size to 10. If our
system detects a suspicious pattern within a chunk, we record it and
compute the false positive rate for the chunk. If the method does not
detect any suspicious pattern, on the other hand, we increase the
chunk's size by 1 until a true positive is given, then we reset the
next chunk's size back to 10. This resulted in 22 chunks for our test
set. Cumulative false and true positive rate data are shown in
Figure \ref{fig:chunking-results}. Initially, the incremental
method has the same false alarm rate as the batch method.  However,
the incremental method's false alarm rate drops quickly to achieve the
eventual 2.2\% false alarm rate at a 91\% hit rate.

\begin{table}[t]
  \caption[Anomaly detection results on average for incremental HMM
    learning for our local method and the global method over 5
    trials.]{\small Anomaly detection results on average for
    incremental HMM learning for our local method and the global
    method over 5 trials. TP, FP, TN, and FN are the number of true positive,
    false positive, true negative, and false negative, respectively.
    TPR and FPR stand for true positive rate and false positive rate,
    respectively.}
  \begin{center}
    \begin{tabular}{c|c|c|c|c|c|c}
      \hline
      Incremental method & TP & FP & TN & FN & TPR & FPR \\
      \hline \hline
      Local ($z$-scoring) & 21.8 & 9.8  & 476.2  & 2.2 & 0.91 & 0.022 \\ \hline
      Local (LRT) & 16 & 243.4 & 212.6 & 8 & 0.6664 & 0.564 \\ \hline
      Global ($z$-scoring)  & 3 & 13 & 214 & 21 & 0.125 & 0.057 \\ \hline
      Global (LRT)  & 18 & 184.4 & 42.6 & 6 & 0.75 & 0.813 \\ \hline
    \end{tabular}
  \end{center}
  \label{tab:iml-detection-results}
\end{table}

For comparison, we also tested incremental versions of the three
alternative approaches previously described in Section
\ref{incremental-against-global-method}: local method with LRT, global method
with $z$-scoring, and global method with LRT. In every condition, we
used the same model configuration and detection threshold found best
in the batch experiment.  For our incremental approach, both typical
and atypical behavior models get updated during incremental learning.
The results are shown in Table \ref{tab:iml-detection-results}.
Similar to the incremental method results, none of the three alternative
methods performed as well as the proposed method. Our experimental
results also demonstrate that over 510 new patterns, our approach
requires security personnel to correct the system a mere 9--10 times.

The incremental version of the global method has lower false alarm
rates and lower detection rates than the batch version of the global
method. To an extent, the fixed $z$-score threshold of 2.0 places the
global method at an inopportune position on the ROC curve. However,
generally, we find that the global method with $z$-scoring performs
poorly regardless of the threshold. With the global representation,
the abnormal models tend to be quite specific whereas the normal
models tend to be a little more general, so that we often see new
anomalous patterns receiving relatively good scores from some normal
models but no good scores from abnormal models. The anomalies thus
tend to get incorrectly classified as normal more often.

Besides improved detection, another advantage of our incremental
approach is that it is more efficient than the batch method over time.
To demonstrate this, we performed a brief experiment on the runtime
requirements on a Core 2 Duo processor at 2.2 GHz. We first trained a
model with 1,000 synthetic sequences then re-trained that model with
1,000 sequences plus 10 new sequences using batch learning or
incremental learning. The batch method required 33816 ms, whereas our
incremental method with only the 10 new sequences required only 1056
ms. Obviously, the incremental approach uses far less time than the
batch method to integrate new data into the training set, because it
does not require retraining on old data.

As a final manipulation, we performed a brief experiment to determine
whether incremental learning alone without a bootstrap set is
effective on these data.  We found that test set performance improves
with the size of the bootstrap set.  With more bootstrap data we tend
to discover more compact behavior classes, leading to better
performance on incremental updates. Conversely, incremental updates
without a good bootstrap model lead to poor detection rates throughout
the experiment. So while a no-bootstrap method could lessen the user's
up-front workload, we conclude that the risk of missed anomalies is
too high.

\section{Discussion}
\label{incremental-conclusion}

We have proposed and evaluated an efficient method for bootstrapping
scene-specific anomalous human behavior detection systems that
incrementally learns behavior models without requiring storage of
large databases of training data. The method requires minimal
involvement of a human operator; the only required action is to label
the patterns in a small bootstrap set as normal or anomalous and then
to label false positive alarms as normal when they occur.

On a testbed data set acquired in a real-world video
surveillance situation, with a bootstrap set of 150 sequences,
the incremental method achieves a false positive rate of 2.2\% at a
91\% hit rate.  With threshold tuning, the method can yield an equal
error rate of 95.8\% or a false positive rate of 3.7\% at a 100\% hit
rate. The experiments show that it is possible to learn a complex set
of varied behaviors occurring in a specific scene with a collection of
simple HMMs while allowing evolution of the learned typical behavior
model over time.  This can lead in turn to more effective anomaly
detection.

\FloatBarrier

%%% --- Maybe use later ---

%%% WE MAY USE THIS INFO %%%
%For Method V, the dash-dot magenta line in
%\figurename~\ref{fig:roc-results} represents the ROC curve for the
%global method as we vary the likelihood threshold at which a sequence
%is considered anomalous. We found that our local approach performs
%better than the global approach on our small dataset. The performance
%of the global method is shown in Table
%\ref{tab:detection-results}. Since the global approach aims to model
%the whole scene, it may perform better with more training data
%covering all possible typical activities in a scene.

% remove the comparison with the ML algorithms due to limitied length
% --- beginning of the comparison with the ML algorithms ---
% \subsubsection{Comparison with Traditional Machine Learning Algorithms}
%
% In this experiment, we apply the proposed method and traditional
% machine learning algorithms ($k$-nearest neighbors, Mahalanobis
% classification, and support vector machines) to the same problem.
% Since the main concern in video surveillance is to detect every
% unusual event while minimizing the false positive rate, in every
% experiment except $k$-NN, we calculate an ROC curve and select the
% detection threshold yielding the best false positive rate at a 100\%
% hit rate. ($k$-NN does not have a detection threshold.) Our
% experimental hypothesis was that the proposed method for modeling
% scene-specific behavior patterns should obtain better false positive
% rates than the traditional methods.
%
% \begin{figure}[t]
%   \begin{center}
%     \begin{tabular}{ccccc}
%       \includegraphics[scale=0.21]{suspicious-behavior-0187.pdf} &
%       \includegraphics[scale=0.21]{suspicious-behavior-0208.pdf} &
%       \includegraphics[scale=0.21]{suspicious-behavior-0225.pdf} &
%       \includegraphics[scale=0.21]{suspicious-behavior-0249.pdf} &
%       \includegraphics[scale=0.21]{suspicious-behavior-0267.pdf}
%       \\
%       Frame 187 & Frame 208 & Frame 225 & Frame 249 & Frame 267
%     \end{tabular}
%   \end{center}
%   \caption{Example anomaly detected by the proposed method (Method
%     I). The sequence contains a person walking around looking for an
%     unlocked bicycle.}
%   \label{fig:suspicious-behavior-detected}
% \end{figure}
%
% \paragraph{Proposed method} We used the 150-sequence bootstrap
% sequence set from Section \ref{sec:model-selection} as training data
% and tested on the remaining 510 sequences.
% \figurename~\ref{fig:suspicious-behavior-detected} shows an example of
% a sequence classified as abnormal.
%
% \paragraph{$k$-NN} We used the same division into training and test data as
% our proposed method. For the distance measure, we used the same DTW
% distance measure used in our hierarchical clustering method. We varied
% $k$ from 1 to 5 and retained the best result.
%
% \paragraph{Mahalanobis classifier} We used the 139 typical
% sequences in the bootstrap set for training and used all abnormal
% sequences (including the abnormal sequences in the bootsrap set) for
% testing. To get a fixed-dimension measurement vector for each
% sequence, we calculated a summary vector consisting of the means and
% standard deviations of each observation vector element over the entire
% sequence. With seven features in the observation vector, we obtained a
% 14-element vector summarizing each sequence then $z$-score normalized
% each component of the summary vector. Since we are performing
% probability density estimation for the normal patterns, we applied PCA
% to the 139 normal sequences in the bootstrap set.  We chose the number
% of principal components accounting for 80\% of the variance in the
% bootstrap data. Finally, we classified the remaining 521 test
% sequences using the PCA model to calculate the Mahalanobis distance of
% each sequence's summary vector to the mean of the normal bootstrap
% patterns' summary vectors.
%
% \paragraph{SVMs} We used the same summary vector technique used for
% Mahalanobis classifier but performed supervised classification using
% support vector machines. We used the radial basis function kernel
% implementation in LIBSVM \cite{CC01a} with grid search for the optimal
% hyperparameters using five-fold cross validation on the training set
% (150 sequences).
%
% \begin{figure}[t]
%   \begin{center}
%     \includegraphics[width=3in]{figures/roc-ours-vs-ml-results.pdf}
%   \end{center}
%   \caption{Anomaly detection ROC curves. The proposed method
%     outperforms the Mahalanobis and SVM classifiers.}
%   \label{fig:roc-ours-vs-ml-results}
% \end{figure}
%
% \begin{table}[t]
%   \begin{center}
%     \begin{tabular}{ | c | c | c | c | c | c | c | }
%       \hline
%       Method                 & TP & FP  & TN  & FN & TPR   & FPR \\
%       \hline\hline
%       Proposed method        & 24 & 42  & 444 & 0  & 1     & 0.086 \\ \hline
%       $k$-NN ($k$ = 1)       & 19 & 1   & 485 & 5  & 0.792 & 0.002 \\ \hline
%       Mahalanobis classifier & 35 & 421 & 65  & 0  & 1     & 0.866 \\ \hline
%       SVM                    & 24 & 228 & 258 & 0  & 1     & 0.469 \\ \hline
%     \end{tabular}
%   \end{center}
%   \caption{Anomaly detection results for the proposed method, $k$-NN,
%     Mahalanobis classifier, and SVM, respectively. For the Mahalanobis
%     classifier, we include 11 abnormal sequences from the bootstrap
%     set in the test set, so the total number of positives is 35.}
%   \label{tab:detection-results}
% \end{table}
%
% \paragraph{Results}
% \figurename~\ref{fig:roc-ours-vs-ml-results} shows the anomaly
% detection results in the form of ROC curves for our proposed method,
% Mahalanobis classifier, and SVM.  The solid red line represents the
% ROC curve for our method as we vary the likelihood threshold at which
% a sequence is considered anomalous.  Note that the ROC does not
% intersect the point $(0, 0)$ because any sequence that is most likely
% under one of the HMMs modeling anomalous sequences in the bootstrap
% set is automatically classified as anomalous regardless of the
% threshold. The ROC reveals that a $z$-score threshold of $-2.293$
% achieves zero false negatives at a false alarm rate of 0.086.  The
% dotted green line is the ROC curve for Mahalanobis classifier,
% obtained by varying the Mahalanobis distance threshold, and the dashed
% blue line is the ROC curve for the SVM, obtained by varying the
% threshold on the signed distance to the separating hyperplane used for
% classification as normal or abnormal.
%
% Table~\ref{tab:detection-results} shows the detailed performance of
% the best models with 100\% hit rate for our method, Mahalanobis, and
% SVM, as well as the $k$-NN model with the best hit rate.  The high
% false positive rate of the Mahalanobis and SVM methods at the 100\%
% hit rate threshold as well as the the overall poor performance in the
% ROC analysis show that the Mahalanobis classifier and SVMs are clearly
% inferior to our proposed method.  While $k$-NN's false positive rates
% are much lower than those obtained in our method, the hit rates are
% unacceptable.
% --- end of the comparision with the ML algorithms ---

%%% MAYBE USEFUL TEXT %%%

%(a person walking around looking for an unlocked bicycle).

% \begin{table}[t]
%   \begin{center}
%     \begin{tabular}{ | c | c | c | c | c | c | }
%       \hline
%       TP & FP & TN & FN & TPR & FPR \\
%       \hline\hline
%       24 & 36 & 450 & 0 & 1 & 0.074 \\ \hline
%     \end{tabular}
%   \end{center}
%   \caption{Anomaly detection results at 100\% hit rate for the
%     proposed method in Experiment I.}
%   \label{tab:detection-results}
% \end{table}

%Since PCA requires a fixed-length input vector, we calculated, for
%each sequence in the testbed data set, a summary vector consisting of
%the means and standard deviations of each observation vector element
%over the entire sequence.

% \begin{table}[t]
%   \begin{center}
%     \begin{tabular}{ | c | c | c | c | c | c | }
%       \hline
%       TP & FP & TN & FN & TPR & FPR \\
%       \hline\hline
%       35 & 421 & 65 & 0 & 1 & 0.87 \\ \hline
%     \end{tabular}
%   \end{center}
%   \caption{PCA anomaly detection results in Experiment III. We include
%   11 abnormal sequences from the bootstrap set in the test set, so the
%   total number of positives is 35.}
%   \label{tab:pca-detection-results}
% \end{table}

% \begin{table}[t]
%   \begin{center}
%     \begin{tabular}{ | c | c | c | c | c | c | }
%       \hline
%       TP & FP & TN & FN & TPR & FPR \\
%       \hline\hline
%       24 & 228 & 258 & 0 & 1 & 0.469 \\ \hline
%     \end{tabular}
%   \end{center}
%   \caption{SVM anomaly detection results in Experiment IV.}
%   \label{tab:svm-detection-results}
% \end{table}

%Although the results are clearly better than those obtained from
%$k$-NN or PCA, they are also clearly inferior to those obtained in
%Experiment I.

%\ifnum\short=0 For $k$-NN to be a
%practical anomaly detection method, we would have to adjust it.  For
%example, we could impose a distance threshold beyond which a pattern
%is considered anomalous even if a majority of the nearest neighbors
%are normal. \fi

% \begin{table}[t]
%   \begin{center}
%     \begin{tabular}{ | c | c | c | c | c | c | c | }
%       \hline
%       $k$ & TP & FP & TN & FN & TPR & FPR \\
%       \hline\hline
%       1 & 19 & 1 & 485 & 5 & 0.792 & 0.002 \\ \hline
%       2 & 19 & 2 & 484 & 5 & 0.792 & 0.004 \\ \hline
%       3 & 16 & 0 & 486 & 8 & 0.667 & 0 \\ \hline
%       4 & 16 & 1 & 485 & 8 & 0.667 & 0.002 \\ \hline
%       5 & 14 & 0 & 486 & 10 & 0.583 & 0 \\ \hline
%     \end{tabular}
%   \end{center}
%   \caption{$k$-NN anomaly detection results in Experiment II.}
%   \label{tab:kNN-results}
% \end{table}

%      $k$-NN ($k$ = 2) & 19 & 2 & 484 & 5 & 0.792 & 0.004 \\ \hline
%      $k$-NN ($k$ = 3) & 16 & 0 & 486 & 8 & 0.667 & 0 \\ \hline
%      $k$-NN ($k$ = 4) & 16 & 1 & 485 & 8 & 0.667 & 0.002 \\ \hline
%      $k$-NN ($k$ = 5) & 14 & 0 & 486 & 10 & 0.583 & 0 \\ \hline
%      Proposed incremental method & 23.9 & 13.6 & 472.4 & 0.067 &
%      0.997 & 0.028 \\ \hline


\section{Results}
\label{results}
Both the BVH builder and path tracer were implemented in Go and only utilize the CPU. Code is optimized moderately without exploiting any SIMD instructions. A series of tests was conducted to compare the build times, ray tracing performance and resulting time per frame rendered between different hyperparameter configurations. LBVH was used as reference, PHR-Fast and PHR-HQ used the parameters proposed by Hendrich et al.\cite{hendrich_parallel_2017}, namely $\alpha=0.5, \delta=6$ and $\alpha=0.55, \delta=9$, respectively. PHR-Grid uses parameters based on the proposed grid search approach over the search space $\alpha\in\{0.4,0.45,0.5.0.55\}, \delta\in\{6,7,8,9\}$ and PHR-BO uses parameters resulting from a Bayesian optimization over the equivalent interval $\alpha\in[0.4,0.55], \delta\in[6,9]$. The Bayesian optimization itself is executed utilizing the bo framework\cite{ou19bo} and is based on a Gaussian process and expected improvement as exploration strategy.

To make results more reliable, all numbers were averaged over ten executions using the bench framework\cite{ou20bench}. Furthermore, the CPU, an AMD Ryzen 2600 eight core processor with 3.4 GHz, was locked to 90 percent capacity to prevent irregularities due to overheating or other high performance fluctuations. Note that the deviation percentages are left out for clarity in the tables presented in this thesis, but the full results are available in the attached files.

Render times are also averaged over three representative views for each scene. To keep times in an interactive window, only the relatively small resolutions 256x256 and 512x512 were tested with one sample per pixel.
\subsection{Multi-Bounding Volume Hierarchies}
\label{multi_bvh}
As mentioned in section \ref{phr_algorithm}, the PHR algorithm allows the construction of bounding volume hierarchies with higher branching factors, which is especially useful for SIMD path tracers. Even though the evaluated path tracer does not utilize any SIMD instructions, I compared the build and trace performance of different multi-BVHs. As expected, the performance difference between branching factors was insignificant in most cases. 4-ary BVHs had slightly faster trace times, while 16-ary BVH construction was slightly slower. The following tests were all performed on 2-ary BVHs.
\subsection{Frame Performance}
The main part of the experiment was about comparing the resulting frame times of all configurations. Table \ref{tab:frametime} shows build time, SAH cost, average render time over the compared view points and the resulting frame times. Note that the PHR build times do not include construction of the auxiliary bounding volume hierarchy, as those would be reused over several frames.

First of all, the numbers clearly show the impact different PHR parameters can have on the build and trace time of the algorithm. PHR-Fast was indeed fairly fast, but the achieved trace speed is even below the baseline, LBVH, in some cases. PHR-HQ had the fastest trace speed in most cases, but the high build duration often leads to higher frame times. A noticeable difference can be seen between the different scene types. PHR performed comparably worse in single object scenes like Bunny, Dragon and Happy Buddha, often not exceeding the render performance of LBVH by much. Sibenik's and Sponza's render times on the other hand, were improved more significantly by applying PHR. Finally, PHR-Grid and PHR-BO were able to improve frame times significantly in a number of cases. However, Bayesian optimization delivered rather inconsistent results and no approach was able to find the optimal parameters in every case. This is probably a result of overfitting the evaluation function and their parameters to certain scenes. Nonetheless, both PHR-Grid and PHR-BO are able to improve frame times compared to PHR-Fast and PHR-HQ. Figure \ref{fig:difference} shows the average relative performance compared to LBVH in percent. While PHR-Fast only improves frame times by 33\% on average, both optimization approaches surpass an average improvement of 50\%.
\begin{figure} [H]
    \centering
    \includegraphics[width=300pt]{images/performance_difference.pdf}
    \caption{Relative difference to LBVH in percent.}
    \label{fig:difference}
\end{figure}
\subsection{Optimization Performance}
Figure \ref{fig:optimization} shows optimization times divided through average frame time, or in other words, how many frames could potentially be rendered instead of executing the optimization. Considering that the average performance increase of both methods amounted around 50\%, i.e. can potentially half the rendering time, optimizations start to become viable at half a frames duration and below. So even though the search space used in grid search is relatively low, the achieved times never reached what would be feasible in interactive applications. Bayesian optimization performed considerably better, but only reached competitive times in a few cases. Note that reaching this time does not equal a performance increase but just a hypothetical chance that frame times are increased.

This shows that the presented optimization approaches still lack in performance and are not yet viable. PHR needs to be executed to evaluate the cost function, which is especially costly for parameters that result in high build times. This could be improved by limiting the search spacer further or making it dynamic and related to the scene's complexity. An interrupt after a maximum execution time might also be a solution. Bayesian optimization uses a costly run with maxed out parameters to determine the maximum build cost. This could be solved by reusing max values from previous optimizations. This topic is discussed further in section \ref{discussion_execute}.
\begin{figure}[H]
    \centering
    \includegraphics[width=300pt]{images/frame_per_optimization.pdf}
    \caption{Number of potential frames during optimization.}
    \label{fig:optimization}
\end{figure}
\clearpage
\begin{table}
\caption{Performance comparison of a representative selection of tested configuration at 256x256 resolution.}
\label{tab:frametime}
\centering
\begin{tabular}{ | c | m{3.5em} | m{3.5em} | m{3.5em} | m{3.5em} | m{3.5em} |  m{3.5em}|}
\hline
& build time (ms) & render time (ms) & frame time ms & build time (ms) & render time (ms) & frame time ms\\
\hline
& \multicolumn{1}{|m{4.5em}}{\includegraphics[width=60pt]{images/bunny.png}} &     \multicolumn{2}{m{4em}|}{Bunny \#triangles 144k}
& \multicolumn{1}{|m{4.5em}}{\includegraphics[width=60pt]{images/sponza.png}} & \multicolumn{2}{m{4em}|}{Sponza \#triangles 66k}\\
\hline
LBVH & 152.0 & 21.9 & 173.9            & 70.4 & 227.0 & 297.5 \\
PHR-Fast & 70.5 & 20.1 & 90.6          & 27.6 & 184 & 211.6 \\
PHR-HQ & 273.0 & 20.7 & 293.7          &  94.7 & 182 & 276.7\\
\hline
PHR-Grid & 14.4 & 20.8 & 35.2        &  17.2 & 207 & 224.2\\
PHR-BO & 14.2 & 20.7 & 34.97         &  21.7 & 200 & 221.7\\
\hline
& \multicolumn{1}{|m{4.5em}}{\includegraphics[width=60pt]{images/dragon.png}} &     \multicolumn{2}{m{4em}|}{Dragon \#triangles 817k}

& \multicolumn{1}{|m{4.5em}}{\includegraphics[width=60pt]{images/sibenik.png}} & \multicolumn{2}{m{4em}|}{Sibenik \#triangles 75k}\\

\hline
LBVH & 900.0 & 28.1 & 928.2              &  72.1 & 226.1 & 298.1\\
PHR-Fast & 150.0 & 32.5 & 182.5            &  32.5 & 205 & 237.5\\
PHR-HQ & 1090.0 & 29.0 & 1119.0              &  98.4 & 187  &285.4\\
\hline
PHR-Grid & 31.7 & 63.5 & 95.2            & 19.1 & 236 & 255.1  \\
PHR-BO & 30.8 & 66.1 & 96.9              & 28.1 & 220 & 248.1 \\
\hline
& \multicolumn{1}{|m{4.5em}}{\includegraphics[width=60pt]{images/buddha.png}} & \multicolumn{2}{m{4em}|}{Happy Buddha \#triangles 1087k}

& \multicolumn{1}{|m{4.5em}}{\includegraphics[width=60pt]{images/san_miguel.png}} &     \multicolumn{2}{m{4.5em}|}{San Miguel \#triangles 5617k}\\

\hline
LBVH & 1080 & 25 & 1105                    & 6010.0 & 816.0 & 6826.6 \\
PHR-Fast & 190 & 27.6 & 217.6                   & 432.0 & 7830.0 & 8262.0 \\
PHR-HQ & 1250 & 23.7 & 1273.7                     & 2600.0 & 812.3 & 3412.3 \\
\hline
PHR-Grid & 40.5 & 57.5 &98                   & 1300.0 & 1856.6 & 3156.6  \\
PHR-BO & 42.1 &58 &100.1                     & 815.0 & 3203.3 & 4018.3 \\
\hline
\end{tabular}
\end{table}
\cleardoublepage


\chapter{Introduction}
Access control, \ie, restricting the access to information or
resources, is an important pillar of today's information security
portfolio. Thus the large number of access control models
(\eg,~\cite{moyer.ea:generalized:2001,sandhu.ea:arbac97:1999,
  sandhu.ea:nist:2000,ansi:rbac:2004,bell:looking:2005,bell.ea:secure:1996,oasis:xacml:2005,li.ea:critique:2007})
and variants thereof
(\eg,~\cite{ferreira.ea:how:2009,wainer.ea:dw-rbac:2007,ardagna.ea:access:2010,samuel.ea:context-aware:2008,bertino.ea:trbac:2001,ardagna.ea:access:2010,becker:information:2007})
is not surprising. On the one hand, this variety of specialized access
control models allows concise representation of access control
policies. On the other hand, the lack of a common foundations makes it
difficult to compare and analyze different access control models
formally.

We present formalization of the Unified Policy Framework
(UPF)~\cite{bruegger:generation:2012} that provides a formal semantics
for the core concepts of access control policiesb. It can serve as a
meta-model for a large set of well-known access control
policies and moreover, serve as a framework for analysis and test
generation tools addressing common ground in policy models.
Thus, UPF for comparing different access control models,
including a formal correctness proof of a specific embedding, for
example, implementing a role-based access control policy in terms of a
discretionary access enforcement architecture. Moreover, defining
well-known access control models by instantiating a unified policy
framework allows to re-use tools, such as test-case generators, that
are already provided for the unified policy framework. As the
instantiation of a unified policy framework may also define a
domain-specific (\ie, access control model specific) set of policy
combinators (syntax), such an approach still provides the usual
notations and thus a concise representation of access control
policies.

UPF was already successful used as a basis for large scale access
control policies in the health care
domain~\cite{brucker.ea:model-based:2011} as well as in the domain of
firewall and router
policies~\cite{brucker.ea:formal-fw-testing:2014}. In both domains,
the formal policy specifications served as basis for the generation,
using {HOL-TestGen}~\cite{brucker.ea:theorem-prover:2012}, of test
cases that can be used for validating the compliance of an
implementation to the formal model. UPF is based on the following four
principles:
\begin{enumerate}
\item policies are represented as \emph{functions} (rather than relations),
\item policy combination avoids conflicts by construction,
\item the decision type is three-valued (allow, deny, undefined),
\item the output type does not only contain the decision but also a
      `slot' for arbitrary result data.
\end{enumerate}

UPF is related to the state-exception monad modeling failing computations;
in some cases our UPF model makes explicit use of this connection, although it
is not central. The used theory for state-exception monads can be found in the
appendix.


\subsection{Privacy Preserving C4.5}\label{s:pp-c45}
C4.5 is a decision tree generating algorithm developed by Ross Quinlan \cite{quinlan1993c4}.
The C4.5 algorithm is an extension of the ID3 algorithm described in section \ref{s:id3} also developed by Quinlan \cite{quinlan1986induction}.
It was ranked \#1 in the \textit{Top 10 Algorithms in Data Mining} pre\hyp eminent paper published by Springer Knowledge and Information Systems in 2008 \cite{wu2008top}.

The main difference between C4.5 and ID3 is that the former has native support for both continuous (\textit{i.e.} numerical) data and discrete (\textit{i.e.} categorical) data. The idea behind C4.5 classification is to find the best splitting point for an attribute and split the dataset on that point.
Transactions that are below that threshold belong to one branch and transaction above that threshold belong to the other branch.
The splitting criterion is based on the information gain \ref{eq:gain} (difference in entropy \ref{eq:entropy}) given by the splitting point.
The attribute that most efficiently splits the dataset is chosen by the algorithm at each level of the tree.
After that the algorithm recurses on the two subsets created by that split.

In algorithm \ref{a:c45-pp} we present the privacy\hyp preserving \f{C45} procedure.
It is quite similar to the \f{ID3} one in algorithm \ref{a:id3-pp}.
The main differences include that the call of \f{Best} procedure (see algorithm \ref{a:c45-best-pp}), now not only returns the ``best'' attribute, but also the ``best'' threshold that this attribute should split on, and the subsets that result from that split.
In case that the chosen attribute is categorical, then the $bestSplitted$ variable will contain the subsets resulting from each possible value $v_i$ of that attribute. ($\{example \in examples \mid example[bestAttribute] = v_i\}$ for each possible $v_i$), and the algorithm creates that many branches.
In case that $bestAttribute$ is a numerical attribute, then the $bestSplitted$ variable will contain the two subsets -- the set of transactions $t$ having $t[bestAttribute] <= bestThreshold$ ($less$), and the set of transactions $t$ having $t[bestAttribute] > bestThreshold$ ($greater$).
Also, in that case creates two branches one for each aforementioned subsets.
Since the algorithm creates two branches for every split for numerical attributes) the output tree is a binary tree (branching factor $= 2$), unlike ID3 where the branching factor of the tree is equal to the number of possible values for every attribute.

\import{./}{algorithms/c45_pp.tex}

% \import{./}{algorithms/c45_gain_textbook.tex}
% \import{./}{algorithms/c45_best_textbook.tex}
% \import{./}{algorithms/c45_textbook.tex}

Here we present the privacy\hyp preserving algorithms for \f{C45} and \f{Best} procedures.
As in section \ref{s:pp-id3} we face the same restrictions regarding the subset creation, for the same reasons as described before.
We omit the rest procedures, as well as their textbook equivalents, as they are similar to the ones presented in sections \ref{s:id3} and \ref{s:pp-id3}.
One difference for example is that the \f{InformationGain} procedure takes the disjoint subsets that occur from a split as an argument instead of taking the attribute and computing the subsets inside.
Also, another argument is the current entropy of the dataset that gets computed once in the \f{Best} procedure for optimization purposes.

\import{./}{algorithms/c45_best_pp.tex}


\subsubsection{Forward Rate Agreement}

A forward rate agreement (trade type \emph{ForwardRateAgreement} is set up using a \\
{\tt ForwardRateAgreementData} block as shown in listing \ref{lst:ForwardRateAgreementdata}. The forward rate agreement specific elements
are:

\begin{itemize}
\item StartDate: A FRA expires/settles on the startDate. \\

Allowable values:  See \lstinline!Date! in Table \ref{tab:allow_stand_data}.

\item EndDate: EndDate is the date when the forward loan or deposit ends. It follows that (EndDate - StartDate) is the tenor/term of the underlying loan or deposit.

Allowable values:  See \lstinline!Date! in Table \ref{tab:allow_stand_data}.
\item Currency: The currency of the FRA notional. \\

Allowable values:  See \lstinline!Currency! in Table \ref{tab:allow_stand_data}.
\item Index: The name of the interest rate index the FRA is benchmarked against.

  Allowable values: An alphanumeric string of the form CCY-INDEX-TERM. CCY, INDEX and TERM must be separated by dashes (-). CCY and INDEX must be among the supported currency and index combinations. TERM must be an integer followed by D, W,
  M or Y. See Table \ref{tab:indices}.

\item LongShort: Specifies whether the FRA position is long (one receives the agreed rate) or short (one pays the agreed rate).

Allowable values: \emph{Long}, \emph{Short}.
\item Strike: The agreed forward interest rate.

Allowable values: Any  real number. The strike rate is
  expressed in decimal form, e.g. 0.05 is a rate of 5\%.
\item Notional: No accretion or amortisation, just a constant notional. \\
Allowable values:  Any positive real number.
\end{itemize}

\begin{listing}[H]
%\hrule\medskip
\begin{minted}[fontsize=\footnotesize]{xml}
    <ForwardRateAgreementData>
        <StartDate>20161028</StartDate>
        <EndDate>20351028</EndDate>
        <Currency>EUR</Currency>
        <Index>EUR-EURIBOR-6M</Index>
        <LongShort>Long</LongShort>
        <Strike>0.001</Strike>
        <Notional>1000000000</Notional>
    </ForwardRateAgreementData>
\end{minted}
\caption{Forward Rate Agreement Data}
\label{lst:ForwardRateAgreementdata}
\end{listing}


\documentclass[8pt,letter,twocolumn]{article}
\usepackage[margin=0.6in]{geometry}
\usepackage[utf8]{inputenc}
\usepackage{fancyvrb}
\usepackage{changes} % For \textsubscript

\usepackage[protrusion=true,expansion=true,activate={true,nocompatibility},final,factor=1200,stretch=35,shrink=35]{microtype}
\doublehyphendemerits=10000

\title{\small \textbf{\textsc{{A Micro-Manual for LISP - Not the whole truth}}}}

\author{\footnotesize \bfseries John McCarthy\\
            \footnotesize \bfseries Artificial Intelligence Laboratory\\
            \footnotesize \bfseries Stanford University}

\date{}

\newcommand{\args}[1] {#1\textsubscript{i} \dots \ #1\textsubscript{n}}
\newcommand{\ei}[0] {e\textsubscript{i}}
\newcommand{\en}[0] {e\textsubscript{n}}

\begin{document}
\setcounter{page}{215}
\maketitle

% \section{Introduction}

LISP data are symbolic expressions that can be either \textit{atoms} or
\textit{lists}. \textit{Atoms} are strings of letters and digits and other
characters not otherwise used in LISP. A list consists of a left parenthesis
followed by zero or more atoms or lists separated by spaces and ending with a
right parenthesis. Examples: \texttt{a}, \texttt{onion}, \texttt{()},
\texttt{(a)}, \texttt{(a onion a)}, \texttt{(plus 3 (times x pi) 1)},
\texttt{(car (quote (a b)))}.

The LISP programming language is defined by rules whereby certain LISP
expressions have other LISP expressions as \textit{values}. The function called
\textit{value} that we will use in giving these rules is not part of the LISP
language but rather part of the informal mathematical language used to define
LISP. Likewise, the italic letters \textit{e} and \textit{a} (sometimes with
subscripts) denote LISP expressions, the letter \textit{v} (usually subscripted)
denotes an atom serving as a variable, and the letter \textit{f} stands for a
LISP expression serving as a function name.

\begin{enumerate}
\setlength\itemsep{0em}
\item \textit{value} \texttt{(quote \textit{e})} = \textit{e}. Thus the value of
  \texttt{(quote a)} is \texttt{a}.

\item \textit{value} \texttt{(car \textit{e})}, where \textit{value e} is a
  non-empty list, is the first element of \textit{value e}. Thus \textit{value}
  \texttt{(car (quote (a b c)))} = \texttt{a}.

\item \textit{value} \texttt{(cdr e)}, where \textit{value} \textit{e} is a
  non-empty list, is the the list that remains when the first element of
  \textit{value} \textit{e} is deleted. Thus \textit{value} \texttt{(cdr (quote
    (a b c)))} = \texttt{(b c)}.

\item \textit{value} \texttt{(cons el e2)}, is the list that results from
  prefixing \textit{value e1} onto the list \textit{value e2}. Thus
  \textit{value} \texttt{(cons (quote a) (quote (b c)))} = \texttt{(a b c)}.

\item \textit{value} \texttt{(equal el e2)} is T if \textit{value e1} =
  \textit{value e2}. Otherwise, its \textit{value} is \texttt{NIL}. Thus
  \textit{value} \texttt{(equal (car (quote (a b))) (quote a))} = \texttt{T}.

\item \textit{value} \texttt{(atom e)} = \texttt{T} if \textit{value e} is an
  atom; otherwise its \textit{value} is \texttt{NIL}.

\item \textit{value} \texttt{(COND(p\textsubscript{i} \ei) \dots
    (p\textsubscript{n} \en))} = \textit{value} \texttt{\ei}, where
  \texttt{p\textsubscript{i}} is the the first of the \textit{p}\rq s whose
  \textit{value} is not \texttt{NIL}. Thus \textit{value} \texttt{(cond ((atom
    (quote a)) (quote b)) ((quote t) (quote c)))} = \texttt{b}.

\item An atom \textit{v}, regarded as a variable, may have a value.

\item \textit{value} \texttt{((lambda (\args{v}) e) \args{e})} is the same as
  \textit{value e} but in an environment in which the variables \args{v} take
  the values of the expressions \args{e} in the original environment. Thus
  \textit{value} \texttt{((lambda (x y) (cons (car x) y)) (quote (a b)) (cdr
    (quote (c d))))} = \texttt{(a d)}.

\item Here\rq s the hard one. \textit{value} \texttt{((label \textit{f} (lambda
    (\args{v}) e)) \args{e})} is the same as \textit{value} \texttt{((lambda
    (\args{v}) e) \args{e})} with the additional rule that whenever
  \texttt{(\textit{f} \args{a})} must be evaluated, \textit{f} is replaced by
  \texttt{(LABEL \textit{f} (lambda (\args{v}) e))}. Lists beginning with
  \texttt{label} define functions recursively.

\end{enumerate}

This is the core of LISP, and here are more examples:  \\

\textit{value} \texttt{(car x)} = \texttt{(a b)} if \textit{value} \texttt{x} =
\texttt{((a b) c)}, and \textit{value} \texttt{((label ff (lambda (x) (cond
  ((atom x) x) ((quote t) (ff (car x)))))) (quote ((a b) c)))}= \texttt{a}

Thus \texttt{((label ff (lambda (x) (cond ((atom x) x) ((quote t) (ff (car
  x))))))}, is the LISP name of a function \textit{ff} such that \textit{ff e}
is the first atom in the written form of \textit{e}. Note that the list
\textit{ff} is substituted for the atom \texttt{ff} twice.

Difficult mathematical type exercise: Find a list \textit{e} such that
\textit{value} \textit{e} = \textit{e}.

\vspace*{1\baselineskip}
\textbf{Abbreviations}
\vspace*{1\baselineskip}

The above LISP needs some abbreviations for practical use.

\begin{enumerate}
\setlength\itemsep{0em}

\item The variables \texttt{T} and \texttt{NIL} are permanently assigned the
  values \texttt{T} and \texttt{NIL}, and \texttt{NIL} is the name of the null
  list \texttt{()}.

\item So as not to describe a LISP function each time it is used, we define it
  permanently by typing \texttt{(defun f (\args{v}))) e)} . Thereafter
  \texttt{(f \args{e})} is evaluated by evaluating \textit{e} with the variables
  \args{v} taking the values value \textit{value} \ei, \dots, \textit{value} \en
  respectively. Thus, after we define \texttt{(defun ff (x) (cond ((atom x) x)
    (t (ff (car x)))))}, typing \texttt{(ff (quote (ca b) c)))}, gets \texttt{a}
  from LISP.

\item We have the permanent function definitions.

  \texttt{(defun null (x) (equal x nil))} and \texttt{(defun cadr (x) (car (cdr
    x)))}.

  and similarly for arbitrary combinations of A and D.

\item \texttt{(list \args{e})} is defined for each n to be \texttt{(cons \ei
    (cons \dots \ (cons \en nil)))} .

\item \texttt{(and p q)} abbreviates \texttt{(cond (p q) (t nil))}.
  \textit{AND}s with more terms are defined similarly, and the propositional
  connectives \texttt{or} and \texttt{not} are used in abbreviating
  corresponding conditional expressions.

\end{enumerate}

Here are more examples of LISP function definitions:

\texttt{(defun alt (x) (cond ((or (null x) (null (cdr x))) x) (t (cons (car x)
  (alt (cddr x))))))}

defines a function that gives alternate elements of a list starting with the
first element. Thus \texttt{(alt (quote (a b c d e)))} = \texttt{(a c e)}

\texttt{(defun subst (x y z) (cond ((atom z) (cond ((equal z y) x) (t z))) (t
  (cons (subst x y (car z))(subst x y ( c d r z))))))},

where Y is an atom, gives the result of substituting X for Y in Z. Thus
\texttt{(subst (quote (plus x y)) (quote v) (quote (times x v)))} =
\texttt{(times x (plus x y))}.

You may now program in LISP. Call LISP on a time-sharing computer, define some
functions, type in a LISP expression, and LISP will output its value on your
terminal.

\vspace*{1\baselineskip}
\textbf{The LISP interpreter written in LISP}
\vspace*{1\baselineskip}

The rules we have given for evaluating LISP expressions can themselves be
expressed as a LISP function \texttt{(eval e a)}, where \textit{e} is an
expression to be evaluated, and \textit{a} is a list of variable-value pairs,
\textit{a} is used in the recursion and is often initially \texttt{NIL}. The
long LISP expression that follows is just such an evaluator. It is presented as
a single LABEL expressions with all auxiliary functions also defined by LABEL
expressions internally, so that it references only the basic function of LISP
and some of abbreviations like CADR and friends. It knows about all the
functions that are used in its own definition so that it can evaluate itself
evaluating some other expression. It does not know about DEFUNs or any features
of LISP not explained in this micro-manual such as functional arguments,
property list functions, input-output, or sequential programs.

The function \texttt{eval} can serve as an interpreter for LISP, and LISP
interpreters are actually made by hand-compiling EVAL into machine language or
by cross-compiling it on a machine for which a LISP system already exists.

The definition would have been easier to follow had we defined auxiliary
functions separately rather than include them using LABEL. However, we would
then have needed property list functions in order to make the \texttt{eval}
self-applicable. These auxiliary functions are \texttt{evlis} which evaluates
lists of expressions, \texttt{evcond} which evaluates conditional expressions,
\texttt{assoc} which finds the value associated with a variable in the
environment, and \texttt{pairup} which pairs up the corresponding elements of
two lists.

\newpage
Here is \texttt{eval}.

\VerbatimInput[fontfamily=tt, fontsize=\scriptsize, samepage=true]{eval.lisp}

\end{document}