\chapter*{Abstract}

Maximising the economic effectiveness of a wind farm is essential in making wind a more economic source of energy. This effectiveness can be increased through the reduction of operation and maintenance costs, which can be achieved through continuously monitoring the condition of wind turbines. An alternative to expensive condition monitoring systems, which can be uneconomical especially for older wind turbines, is to implement classification algorithms on supervisory control and data acquisition (SCADA) signals, which are collected in most wind turbines. Several publications were reviewed, which were all found to use separate algorithms to predict specific faults in advance. In reality, wind turbines tend to have multiple faults which may happen simultaneously and have correlations with one another. This project focusses on developing a methodology to predict multiple wind turbine faults in advance simultaneously by implementing classification algorithms on SCADA signals for a wind farm with 25 turbines rated at 2,500 kW, spanning a period of 30 months. The data, which included measurements of wind speed, active power and pitch angle, was labelled using corresponding downtime data to detect normal behaviour, faults and varying timescales before a fault occurs. Three different classification algorithms, namely decision trees, random forests and k nearest neighbours were tested using imbalanced and balanced training data, initially to optimise a number of hyperparameters. The random forest classifier produced the best results. Upon conducting a more detailed analysis on the performance of specific faults, it was found that the classifier was unable to detect the varying timescales before a fault with accuracy comparable to that of normal or faulty behaviour. This could have been due to the SCADA data, which are used as features, being unsuitable for detecting the faults, and there is potential to improve this by balancing only these classes.
\\[.5cm]
\noindent\textbf{\textit{Keywords:}} \keywords


\chapter{parelm}

\section{Introduction}

An LPE may have parameters that do not affect the behavior of that process in any way.
These parameters are said to be \emph{inert}.
Removing inert parameters from an LPE reduces the size of state vectors and the state space of that LPE, which both benefits performance.

\section{Algorithm}

The algorithm consists of the following steps \cite{groote2001computer}:

\begin{enumerate}

\item Make it so that all parameters of the LPE are \labeledas{inert}.

\item Consider the guards of all summands of the LPE, and \removelabelfrom{inert} from all LPE parameters that occur in the guard of a summand.

\item Let $X$ be the set of all LPE parameters that are \labeledas{inert}, and consider all expressions $v_s(p)$ that define the value of an LPE parameter $p$ after application of a summand $s$.
For each such expression, if $p$ is \emph{not} \labeledas{inert}, \removelabelfrom{inert} all LPE parameters that occur in $v_s(p)$.

\item Repeat the previous step until the fixpoint of $X$ is reached.
Then remove all LPE parameters that are still \labeledas{inert} from the LPE, substituting references to those parameters by their initial values.

\end{enumerate}

\clearpage
\section{Example}

Consider the following LPE:

\begin{lstlisting}
//Process definition:
PROCDEF example[A :: Int, B](x, y, z :: Int)
  = A ? i [[x==0]] >-> example[A, B](i, y, z)
  + A ? i [[x==1]] >-> example[A, B](0, i, z)
  + B [[x==2]] >-> example[A, B](0, y, z)
  + B [[true]] >-> example[A, B](z, y, x)
  ;

//Initialization:
example[A, B](0, 0, 0);
\end{lstlisting}

First, the `inert' label is removed from $x$ because $x$ occurs in the guards of the first three summands.

In the fourth summand, $z$ is used in the expression of which the value is assigned to $x$.
Therefore $z$ is also no longer \labeledas{inert}.

$y$ remains labeled with `inert': it does not occur in a guard, nor is it used in the assignment to a process parameter other than itself.
Removing $y$ gives

\begin{lstlisting}
//Process definition:
PROCDEF example[A :: Int, B](x, z :: Int)
  = A ? i [[x==0]] >-> example[A, B](i, z)
  + A ? i [[x==1]] >-> example[A, B](0, z)
  + B [[x==2]] >-> example[A, B](0, z)
  + B [[true]] >-> example[A, B](z, x)
  ;

//Initialization:
example[A, B](0, 0);
\end{lstlisting}

\section{Benchmark results}

The following durations were measured with a benchmark for several models:
\begin{itemize}
\item The average duration of \txs{} to make 500 steps in a model after it has been converted to LPE form;
\item The average duration of \txs{} to make 500 steps in a model after it has been converted to LPE form and after the \texttt{parelm} operation has been applied.
\end{itemize}

When plotting the second series of measurements against the first (see Figure~\ref{parelm-vs-lpe-only:fig}), it is easy to see that the impact is insignificant in most cases.
The only model for which a significant performance increase has been measured is the \texttt{ControlLoop} model, which has indeed lost part of its state vector.

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=0.5\linewidth]{charts/parelm-vs-lpe-only}
\caption{Benchmark results: parelm vs LPE transformation}
\label{parelm-vs-lpe-only:fig}
\end{center}
\end{figure}


\section{Building NWAs from other NWAs (namespace \texttt{opennwa::construct})}
\label{Se:Building NWAs}

The library provides functions for performing automata-theoretic operations upon
NWAs. The supported operations are union, intersection, concatenation,
reversal, Kleene star, complement, and determinization.

The library supports two interfaces to each of these operations. In one, the
operation allocates an NWA with \texttt{new}, performs the construction, and
returns a \texttt{NwaRefPtr} to the result.  In the other, the operation takes
a reference to an NWA, clears it, and constructs the result in-place. The
first form is usually more convenient to use, and creates a mini language for
set expressions; the second form makes it possible to store the result of an
operation in a subclass of \texttt{Nwa}, or an existing \texttt{Nwa} object.

Each of these functions is in the namespace \texttt{opennwa::construct}:

\begin{functionlist}

  \functionDefEarly{NwaRefPtr}{unionNwa}{Nwa const \& a, Nwa const \& b}{}
  \functionDef{void}{unionNwa}{Nwa \& out, Nwa const \& a, Nwa const \& b}{}
    Computes the union of the NWAs \texttt{a} and \texttt{b},
    either returning the result or storing it in \texttt{out}.
    See Section
    \ref{Se:Union}. (This function is called \texttt{unionNwa} instead of
    \texttt{union} because the latter is a C++ keyword.) The two NWAs must not
    have any states in common, and the output will be nondeterministic even
    if both input NWAs are deterministic. The running time\footnote{We count
      state lookups, additions, etc.\ as constant time even though they are
      actually logarithmic, and occasionally linear.} is
    $O(|Q^a|+|Q^b|+|\delta^a|+|\delta^b|)$.

  \functionDefFirstEarly{NwaRefPtr}{intersect}{Nwa const \& a, Nwa const \& b}{}
  \functionDef{void}{intersect}{Nwa \& out, Nwa const \& a, Nwa const \& b}{}
    Computes the intersection of the NWAs \texttt{a} and \texttt{b},
    either returning the result or storing it in \texttt{out}.
    See Section \ref{Se:Intersection}. If both input NWAs are deterministic,
    the output will be deterministic. The worst-case running time is
    $O(|Q^a| |Q^b| d_a d_b)$, where $d_a$ is the maximum out-degree
    of a state in \texttt{a} and $d_b$ is the maximum out-degree of a state
    in \texttt{b}.

  \functionDefFirstEarly{NwaRefPtr}{concat}{Nwa const \& left, Nwa const \& right}{}
  \functionDef{void}{concat}{Nwa \& out, Nwa const \& left, Nwa const \& right}{}
    Computes the concatenation of the NWAs \texttt{left} and
    \texttt{right}, either returning the result or storing it in
    \texttt{out}.
    See Section \ref{Se:Concatenation}. The output automaton will be
    nondeterministic. The running time is
    $O(|Q^a|+|Q^b|+|\delta^a|+|\delta^b|+|Q^a||\delta_r^b|)$.

  \functionDefFirstEarly{NwaRefPtr}{star}{Nwa const \& orig}{}
  \functionDef{void}{star}{Nwa \& out, Nwa const \& orig}{}
    Computes the Kleene star of the NWA \texttt{orig}, either
    returning the result or storing it in \texttt{out}. See Section
    \ref{Se:Star}. The output automaton will be nondeterministic. The
    running time is $O(|Q|+|\delta|+|Q_0||Q_f||\delta_c|)$.

  \functionDefFirstEarly{NwaRefPtr}{reverse}{Nwa const \& orig}{}
  \functionDef{void}{reverse}{Nwa \& out, Nwa const \& orig}{}
    Computes the NWA that accepts the reverse of each nested word
    accepted by the NWA \texttt{orig}, either returning the result or
    storing it in \texttt{out}. See Section \ref{Se:Reverse}. The running
    time is $O(|Q|+|\delta|)$.

  \functionDefFirstEarly{NwaRefPtr}{determinize}{Nwa const \& orig}{}
  \functionDef{NwaRefPtr}{determinize}{Nwa \& out, Nwa const \& orig}{}
    Computes a determinization of \texttt{orig}, either returning the
    result or storing it in \texttt{out}.
    See Section \ref{Se:Determinize}. The worst-case running time for this
    algorithm is at least $O({|Q|^3} \cdot2^{|Q|^2})$.

\clearpage
  \functionDefFirstEarly{NwaRefPtr}{complement}{Nwa const \& orig}{}
  \functionDef{void}{complement}{Nwa \& out, Nwa const \& orig}{}
    Computes the complement of the determinization of \texttt{orig},
    either returning the result or storing it in \texttt{out}.
    See Section
    \ref{Se:Complement}. %For complementing the set of final states to work
    %correctly, two things must hold: \texttt{orig} must be deterministic and
    %the transition functions must be total. (That is, for a given state $q$
    %and symbol $\sigma$, there must be exactly one internal transition $(q,
    %5\sigma, q') \in\delta_i$, exactly one call transition $(q, \sigma,
    %q')\in\delta_c$, and exactly one return transition $(q, p, \sigma,
    %q')\in\delta_r$ for each state $p$.)
    \texttt{complement()} determinizes \texttt{orig}, so its worst-case
    running time is also at least $O({|Q|^3} \cdot2^{|Q|^2})$.

    %\vspace{0.5\baselineskip}
    %If \texttt{determinize} is \texttt{true}, it will
    %automatically determinize the input NWA before complementation, which
    %also guarantees the input machine will be total. If you
    %know that \texttt{orig} is already deterministic and total, you can set
    %\texttt{determinize} to
    %\texttt{false} to avoid this step. (Complementing a nondeterministic or
    %non-total NWA can easily lead to an incorrect result.) If
    %\texttt{determinize} is \texttt{true}, the worst-case running time is the
    %same as for the \texttt{determinize} function (roughly
    %$O(2^{|Q|^2})$). If \texttt{determinize} is \texttt{false}, the running
    %time is $O(|Q|+|\delta|)$.

\end{functionlist}

As mentioned above, these functions create a small language of set
expressions. For instance, to compute an automaton \texttt{M} whose language
is $\texttt{A} \cup (\texttt{B} \cap \texttt{C})*$ (where \texttt{A},
\texttt{B}, and \texttt{C} are \texttt{NwaRefPtr}s), one can write
\begin{center}
  \texttt{NwaRefPtr M = unionNwa(*A, *star(*intersect(*B, *C)))}
\end{center}

\begin{figure}[tb]
  \centering
  \begin{minipage}{0.3\textwidth}
    \centering
    \nwaimage[.6]{Figures/figure-1}
    \caption{An example NWA.}
    \label{Fi:Example1}
  \end{minipage}
  \hspace{0.025\textwidth}
  \begin{minipage}{0.3\textwidth}
    \centering
    \nwaimage[0.49]{Figures/union-other}
    \caption{A second example NWA.}
    \label{Fi:Union1}
  \end{minipage}
  \hspace{0.025\textwidth}
  \begin{minipage}{0.3\textwidth}
    \centering
    \nwaimage[0.8]{Figures/union-result}
    \caption{The NWA resulting from the union of the NWA in Figure
      \ref{Fi:Example1} and the NWA in Figure \ref{Fi:Union1}. Note that there
      are two initial states.}
    \label{Fi:Union2}
  \end{minipage}
\end{figure}

\subsection{Union}
\label{Se:Union}
The union of two NWAs is constructed by taking the union of each of the
components of the NWAs. (In
particular, it does \textsl{not} do a cross-product construction, and will
\textsl{always} produce a nondeterministic automaton as a result as long as
both machines have at least one initial state.)

Formally, the union of NWAs $N = (Q_N, \Sigma_N, Q_{0,N}, \delta_N, F_N)$
and $M = (Q_M, \Sigma_M,$ $Q_{0,M}, \delta_M, F_M)$ is
$N \cup M = (Q_N \cup Q_M, \Sigma_N \cup \Sigma_M, Q_{0,N} \cup
   Q_{0,M}, \delta_N \cup \delta_M, F_N \cup F_M)$.

As an example, \figref{Union2} illustrates the union of \figref{Example1} and
\figref{Union1}.

The state sets of the NWAs must not overlap,
i.e., $Q_1 \cap Q_2 = \emptyset$. \textsl{Client code should not rely on
  this condition being checked} or any particular behavior occurring if it
does not hold.

Client information is copied directly from the original NWAs using
\texttt{ClientInfo::clone()}.

\subsection{Intersection}
\label{Se:Intersection}

The intersection of two NWAs is computed in the standard cross-product
fashion, using a worklist algorithm to only compute those states that
are reachable.

The algorithm traverses the original NWAs starting at
the initial states and incrementally adds transitions for each pair of
``intersectable'' transitions that are encountered. By default, transitions
are intersectable when the transitions are the same kind (internal, call,
or return) and the symbols on the edge are identical or one is wild.

For example, \figref{Intersect1} shows the intersection of \figref{Example1} and
\figref{Intersect1}, and \figref{Intersect4} shows the intersection of the
two NWAs in \figref{Intersect3}.

\begin{figure}[tp]
  \centering
  \begin{minipage}{0.48\textwidth}
    \centering
    \nwaimage[0.45]{Figures/intersection-simple-other}
    \caption{Simple NWA to intersect with the NWA in Figure \ref{Fi:Example1}.}
    \label{Fi:Intersect1}
  \end{minipage}\hspace{0.2cm}
  \begin{minipage}{0.49\textwidth}
    \centering
    \nwaimage{Figures/intersection-simple-result}
    \caption{The NWA resulting from the intersection of the NWA in Figure
      \ref{Fi:Example1} and the NWA in Figure \ref{Fi:Intersect1}. Note that
      because each of the input NWAs only accepts a single word and those
      words are different, the language of the intersection is empty. The NWA
    built up by \texttt{intersect()} got as far as it
    could. \texttt{(key\#2,3)} is the pair of states \texttt{(start,start)}
    from the two input automata.}
    \label{Fi:Intersect2}
  \end{minipage}
\end{figure}

\begin{figure}[p]
  \centering
    \adjustbox{width=\textwidth,height=\textheight,keepaspectratio=true}{
    \nwaimage{Figures/intersection-complex-first}
    \nwaimage{Figures/intersection-complex-second}}
  \caption{Two complex NWAs to intersect.}
  \label{Fi:Intersect3}
\end{figure}

\begin{figure}[p]
  \centering
    \nwaimage[.45]{Figures/intersection-complex-result}
  \caption{The NWA resulting from the intersection of the NWAs in Figure \ref{Fi:Intersect3}.}
  \label{Fi:Intersect4}
\end{figure}

\antistupidfloats

It is possible to customize what symbols are considered equivalent, or
otherwise impose constraints on what transitions can be intersected, by
overriding the \texttt{transitionIntersect} function in a subclass of
\texttt{Nwa}. In addition, it is possible to impose additional constraints on
what states can be combined by overriding \texttt{stateIntersect}. Both
functions also produce the result of the intersection:
\texttt{transitionIntersect} produces the symbol that will be used on the
resulting edge, and \texttt{stateIntersect} produces the state that will be
used as the target.

The default behavior of
\texttt{transitionIntersect} is that two transitions are intersectable if
neither symbol is epsilon and either the symbols are the same or at least one of
the symbols is a wild. (Epsilon transitions are dealt with in
\texttt{intersect()} itself. If you override \texttt{transitionIntersect}, it
should return \texttt{false} if either input is epsilon.)

The default behavior of \texttt{stateIntersect} is that any two
states can be combined, the resulting state is labeled with a
key that is uniquely generated from the pair of the keys of
the two states under consideration, and the client information associated
with the resulting state is \texttt{null}.

Client information is initially generated by the helper method \texttt{stateIntersect},
but can be altered through the use of the helper methods
\texttt{intersectClientInfoInternal}, \texttt{intersectClientInfoCall}, and
\texttt{intersect\-Client\-InfoReturn}, which are invoked by
\texttt{intersect()} as transitions of the three different kinds involving the
associated state are added.  The default behavior of these three functions is
to perform no changes to the \texttt{ClientInfo}.  These methods can be
overridden in subclasses to specify alternative behaviors.

\goodbreak
The following operations are virtual functions in class \texttt{Nwa} and are intended
to be overridden to customize behavior:

\begin{functionlist}
  \functionDefNoCloseParen{bool}{stateIntersect}{%
                  \parbox[t]{4in}
                              {Nwa const \& first, State state1,\\
                               Nwa const \& second, State state2,\\
                               State \& resSt, ref\_ptr<ClientInfo> \& resCI)}}
    Determines whether the given states can be combined and,
    if so, creates the combined state. If the two states are incompatable,
    returns \texttt{false}. If the two states are compatable, it computes the
    key of the combined state (storing it in \texttt{resSt} and the client
    information (storing it in \texttt{resCI}), then returns \texttt{true}.

  \functionDefFirstNoCloseParen{bool}{transitionIntersect}{%
                  \parbox[t]{4in}
                              {Nwa const \& first, Symbol sym1,\\
                               Nwa const \& second, Symbol sym2,\\
                               Symbol \& resSym )}}
    Determines whether the given symbols are considered to be equivalent for
    the purposes of intersection. If so, it computes the symbol that should
    be associated with the combined transition (storing it in
    \texttt{resSym}) and returns true. If not, returns false.

  \functionDefFirstEarlyNoCloseParen{void}{intersectClientInfoInternal}{%
                  \parbox[t]{4in}
                     {Nwa const \& first, State src1, State tgt1,\\
                      Nwa const \& second, State src2, State tgt2,\\
                      Symbol resSym, State resSt )}}
    \nopagebreak
  \functionDefEarlyNoCloseParen{void}{intersectClientInfoCall}{%
                  \parbox[t]{4in}
                      {Nwa const \& first ,State call1, State entry1,\\
                       Nwa const \& second, State call2, State entry2,\\
                       Symbol resSym, State resSt )}} \nopagebreak
  \functionDefNoCloseParen{\small void}{\small intersectClientInfoReturn}{%
                  \parbox[t]{4in}
                      {\small Nwa const \& first, State exit1, State call1,State ret1,\\
                       Nwa const \& second, State exit2,State call2, State ret2,\\
                       Symbol resSym, State resSt )}}
    Called after a transition of the corresponding type is added to the
    automaton. It is intended to be used to
    alter the client information associated with \texttt{resSt} given the
    endpoints of the new transition.

\end{functionlist}

\subsection{Concatenation}
\label{Se:Concatenation}

The concatenation of two NWAs is constructed by taking the union of all
states and transitions of the two automata, and adding
internal epsilon transitions from each final state of the first NWA to each
initial state of the second NWA.  In the resulting NWA, the initial states
are the initial states from the first NWA, and the final states are the final
states of the second NWA.

However, the concatenation construction is a bit more complicated than just this,
because we need to address the issue of an unbalanced-left
word being concatenated with an unbalanced-right word.  (Recall the
definition of what happens when an NWA reads a pending return: it is
allowed to take a return transition where the call predecessor is an initial
state of the automaton.)

To deal properly with the case where the first operand generates strings with
pending calls and the second operand generates strings with pending returns,
the
transitions from the second automaton are augmented in the
following manner. When computing the concatenation of \texttt{left} and
\texttt{right}, for each transition $(q, p_0, a, q')
\in\delta_r^{\texttt{right}}$ where $p_0 \in Q_0^\texttt{right}$ (these are
transitions that \texttt{right} can take when reading a pending return) and
each $p \in Q^\texttt{left}$, we add $(q, p, a, q')$ to $\delta_r$ of the
result. (It is actually only necessary to add such a transition for states
$p$ that either appear in the call position of a call transition in
\texttt{left} or are in $Q_0^\texttt{left}$.)

 % If the original NWAs are $(Q_1, \Sigma_1,
%{Q_0}_1, \delta_1, {Q_f}_1)$ and $(Q_2, \Sigma_2, {Q_0}_2, \delta_2,
%{Q_f}_2)$, then the resulting NWA is $(Q, \Sigma, Q_0, \delta, Q_f)$ where $Q
%= Q_1 \cup Q_2$, $\Sigma = \Sigma_1 \cup \Sigma_2$, $Q_0 = {Q_0}_1$, $\delta
%= \delta_1 \cup \delta_2 \cup \delta_\epsilon$ (where $\delta_\epsilon =
%\{(q,\epsilon,q') | q \in {Q_f}_1, q' \in {Q_0}_2\}$, and $Q_f = {Q_f}_2$ .
%The NWA resulting from the concatenation of the NWA in Figure
%\ref{Fi:Example1} and the NWA shown in Figure \ref{Fi:Concat1} is shown in
%Figure \ref{Fi:Concat2}.

\begin{figure}[p]
  \centering
  \begin{minipage}{0.5\textwidth}
    \begin{minipage}{\textwidth}
      \centering
      \nwaimage[1]{Figures/concat-other}
      \caption{Simple NWA to concatenate onto the NWA in Figure \ref{Fi:Example1}.}
      \label{Fi:Concat1}
    \end{minipage}

    \vspace{2\baselineskip}
    \begin{minipage}{0.8\textwidth}
      \centering
      \nwaimage[0.55]{Figures/reverse-of-figure-1}
      \caption{The NWA resulting from performing reverse on the NWA in Figure \ref{Fi:Example1}.}
      \label{Fi:Reverse1}
    \end{minipage}
  \end{minipage}
  \begin{minipage}{0.49\textwidth}
    \centering
    \nwaimage[0.75]{Figures/concat-result}
    \caption{The NWA resulting from the concatenation of the NWA in Figure
      \ref{Fi:Example1} with the NWA in Figure \ref{Fi:Concat1}.}
    \label{Fi:Concat2}
  \end{minipage}
\end{figure}

\antistupidfloats

The state sets of the NWAs must not overlap,
i.e., $Q_1 \cap Q_2 = \emptyset$. \textsl{Client code should not rely on
  this condition being checked.}

Client information is copied directly from the original NWAs using
\texttt{ClientInfo::clone()}.

\figref{Concat2} shows the result of concatenating \figref{Example1} and
\figref{Concat1}.

\subsection{Kleene star}
\label{Se:Star}

Like concatenation, Kleene-star is complicated in the case of NWAs because of
the ability to have unbalanced words in the automaton's
language. Relative to standard FAs, in the concatenation construction we only needed
to add extra transitions; in this construction, we must add additional states
as well.

The NWA resulting from performing Kleene-Star on the NWA shown in
\figref{Star1} is shown in \figref{Star2}.

The construction presented in \cite{JACM:AM2009} has a minor error. The error is
analogous to not adding a distinguished start state in the traditional
Thompson construction,\footnote{The initial state of the automaton $A^*$
  must accept, because $\epsilon$ is in $L^*$; and because of this property it is
  incorrect to merely add epsilon transitions from the old final states to
  the old initial states. If you do this and there is a cycle from the
  initial state back to itself (for example, a self-loop on the initial
  state), the word corresponding to that path would be accepted even though
  it should not be.} and in
fact can be exhibited using the same example (it is not necessary to use NWA
calls or returns). Alur confirmed that our interpretation of the construction
in \cite{JACM:AM2009} is
correct~\cite{AlurNwaStarBroken}. Below, we present
a version that uses $\epsilon$-transitions, and thus it looks a bit
different from the version in \cite{JACM:AM2009}.

When computing $R = A^*$ for some
NWA $A$, the resulting NWA has two ``copies'' of $A$. These are denoted by
primed and unprimed version of states from $A$ in the definition below.

\begin{figure}[p]
  \centering
    \nwaimage[.4]{Figures/star-simple}
  \caption{An NWA on which to perform Kleene-Star.}
  \label{Fi:Star1}
\end{figure}

\begin{figure}[p]
  \centering
    \nwaimage[1.1]{Figures/star-simple-result}
  \caption{The NWA resulting from performing Kleene-Star on the NWA in Figure \ref{Fi:Star1}.}
  \label{Fi:Star2}
\end{figure}
\antistupidfloats

Suppose that $R$ is reading a word $w=w_1w_2\cdots w_n$, where each $w_i \in
L(A)$. $R$ begins in a start state of $A'$. Henceforth it maintains the
following invariant on the state that $R$ is in with respect to the portion
of $w$ read so far: if the next symbol $\sigma$ is in a return position, then
that
symbol is a \emph{pending} return in the current $w_i$ iff $R$ is in the $A'$
portion, i.e., if the current state is primed. (Note that this return only
needs to be
pending in the current $w_i$. In the full string $w$, $\sigma$ may match a
call in an earlier $w_j$, or it may be pending in the whole string.)

Internal transitions thus keep $R$ in the same copy of $A$, and call
transitions always take $R$ to the unprimed copy of $A$ (because if it then
reads a return, the return will match that call). Return transitions can
target either copy of $A$: if the call predecessor is unprimed, then the
target will be unprimed; if the call predecessor is primed, then the target
will be primed.

The description above describes $R$'s operation under ``normal''
conditions. If $R$ is in a final state (either primed or unprimed) of the
automaton $A$, it is also allowed to guess that it should ``restart'' by taking
an epsilon transition to a distinguished start and final state $q_0$. This
guess is correct if it just read the
last character in $w_i$ (making the next character the first one in
$w_{i+1}$). Note that $q_0$ only has transitions to
the $A'$ portion of $R$, maintaining the invariant.

The reason for the two copies of $A$ comes into play when $R$ reads a return
$\sigma$ while in the $A'$ portion. By the invariant, $\sigma$ is pending in the
current $w_i$. In the original automaton $A$, the transitions that the
machine can use are return transitions $(q,r,\sigma,p)$ where the call predecessor
$r$ is in $Q_0$. We need to make sure that $R$ can take those same
transitions. There are two cases we need to consider:
\begin{enumerate}
\item For the cases where
$\sigma$ is pending in the whole string $w$, we need to have a version of the
return transition with $q_0$ in the call-predecessor position, so we add $(q',
q_0, \sigma, p')$.
\item For the cases where $\sigma$ is matched with a call in some
earlier $w_j$, it does not matter what state the machine was in before that
call; thus we add $(q', s, \sigma, p')$ for each state $s$ in $Q \cup Q'$.
\end{enumerate}
\vspace{0.25\baselineskip}

``Normal'' operation corresponds to the transitions introduced by the
\textsc{Internal}, \textsc{Call}, and \textsc{Locally-Matched-Return}
inference rules given below. The source states of both transitions added by
\textsc{Locally-Matched-Return} are unprimed, because if the current symbol
is a return that matches a call in the same $w_i$, $R$ will be in
the $A$ portion by the invariant.
The \textsc{Restart} rule allows $R$ to restart, and the
\textsc{Start} rule allows $R$ to get from $q_0$ to the $A'$ portion; these
transitions target the $A'$ portion because there have been no calls read in
the current $w_i$, and thus a return symbol would be pending.
The \textsc{Globally-Pending-Return} rule addresses the situation where the
current symbol is a return that is pending in the whole string $w$. (This is
the first case in the previous paragraph.) The \textsc{Locally-Pending-Return} rule
addresses the situation where the current symbol is a return that is pending
in the current $w_i$ but matches a call from a previous $w_j$.

Formally, if the original NWA is $(Q, \Sigma, Q_0, \delta, Q_f)$,
then the result of performing Kleene-Star on that NWA is $(Q^*, \Sigma,
Q_0^*, \delta^*, Q_f^*)$. The sets of states are defined by $Q^* = Q \cup
Q' \cup \{q_0\}$ (with $Q' = \{q'\, |\, q \in Q\}$ and $q_0 \not \in Q$),
and  $Q_0^* = Q_f^* = \{q_0\}$.
The transitions in $\delta^*$ are defined by the following rules:

\begin{mathpar}
{\inferrule*[left=\textsc{Internal}]
             {(q,\sigma,p) \in \delta_i }
  { (q,\sigma,p) \in  \delta_i^* \\ (q',\sigma,p') \in \delta_i^*}
}
\and
{\inferrule*[right=\textsc{Call}]
           { (q,\sigma,p) \in \delta_c  }
  { (q,\sigma,p) \in  \delta_c^* \\ (q',\sigma,p) \in \delta_c^*   }
}
\and
{\inferrule*[left=\textsc{Locally-Matched-Return}]
              { (q,r,\sigma,p) \in \delta_r }
  {(q,r,\sigma,p) \in  \delta_r^* \\ (q,r',\sigma,p') \in \delta_r^* }
}
\\
\and
{\inferrule*[left=\textsc{Start}]
  { q \in Q_0 }
  {(q_0, \epsilon, q') \in \delta_i^*}
}
\and
{\inferrule*[right=\textsc{Retart}]
  { q \in Q_f }
  {(q, \epsilon, q_0) \in \delta_i^* \\ (q', \epsilon, q_0) \in \delta_i^* }
}
\and
{\inferrule*[left=\textsc{Globally-Pending-Return}]
  { (q,r,\sigma,p) \in \delta_r \\ r \in Q_0 }
  {(q',q_0,\sigma,p') \in  \delta_r^* }
}
\and
{\inferrule*[left=\textsc{Locally-Pending-Return}]
  { (q,r,\sigma,p) \in \delta_r \\ r \in Q_0 \\ s \in Q \cup Q' }
  {(q',s,\sigma,p') \in \delta_r^* }
}
\end{mathpar}

Client information is copied directly from the original NWA (using
\texttt{ClientInfo::clone()}) such that for each $q \in Q$, $q$
and $q'$ have (different copies of) the same client information.

\emph{Note:} The key for state $q'$ is generated from the key for state $q$
using the expression \texttt{getKey(q, getKey("prime"))}. The input automaton
to the Kleene star function must not already contain both $q$ and $q'$ for
any $q$.

%Consider the slightly more complex example of computing the Kleene-Star of the NWA shown in Figure \ref{Fi:Star3}.  The resulting NWA is shown in Figure \ref{Fi:Star4}.

%\begin{figure}[htbp]
%  \centering
%    \includegraphics[angle=270,width=10cm]{Figures/Figure13.pdf}
%  \caption{Complex NWA on which to perform Kleene-Star.}
%  \label{Fi:Star3}
%\end{figure}

%\begin{figure}[htbp]
%  \centering
%    \includegraphics[angle=270,width=12cm]{Figures/Figure14.pdf}
%  \caption{The NWA resulting from performing Kleene-Star on the NWA in Figure \ref{Fi:Star3}.}
%  \label{Fi:Star4}
%\end{figure}

\subsection{Reverse}
\label{Se:Reverse}

A nested word $n = (w, \rightsquigarrow)$ is reversed by reversing the linear
word $w$ and exchanging calls and returns. Formally,
$n^{rev} = (w^{rev}, \{(|w|+1-r, |w|+1-c) | (c,r)
\in\rightsquigarrow\})$. (Pending calls and returns are handled by defining
$|w|+1-(+\infty) = -\infty$ and $|w|+1-(-\infty) = +\infty$.) Roughly
speaking, call transitions in $A$ correspond to return transitions in
$A^{rev}$ and vice versa, and we reverse the direction of all transitions as
in the standard FA construction. We describe the construction from the
perspective of $A^{rev}$ --- that is, a ``call transition'' is a call
transition in $A^{rev}$, and a ``call'' is a call in the reversed string.

Perhaps unsurprisingly, pending returns pose a problem because the
role of initial and final states are exchanged. Because of this complication,
the algorithm for reversing an NWA has a similar flavor to that of the
Kleene-star procedure. The automaton $A^{rev}$ has two ``copies'' of $A$
(primed and unprimed), and maintains the same invariant as the Kleene-star
construction: if
the next symbol $\sigma$ is in a return position, then that symbol is a
\emph{pending} return iff $A^{rev}$ is in the $A'$ portion.
(For those familiar with the construction in \cite{JACM:AM2009}, ours is more
complicated because the version in \cite{JACM:AM2009} will not work as stated
with a weakly-hierarchical NWA.)

If the original NWA is $(Q, \Sigma, Q_0, \delta, Q_f)$, then the result of
reversing that NWA is $(Q \cup Q', \Sigma, Q_f', \delta^{rev}, Q_0)$ obtained using
the following rules:

\begin{mathpar}
{\inferrule*[left=\textsc{Internal}]
     { (p,\sigma,q) \in \delta_i  }
  { (q,\sigma,p)  \in \delta^{rev}_i \\ (q',\sigma,p')  \in \delta^{rev}_i }
}
\and
{\inferrule*[left=\textsc{Call-Return}]
       { (q_c,\sigma_c,q_e) \in  \delta_c \\ (q_x,\_\!\_\,,\sigma_r,q_r) \in \delta_r }
  { (q_r, \sigma_r, q_x), (q_r', \sigma_r, q_x) \in \delta^{rev}_c \\
    (q_e, q_r, \sigma_c, q_c), (q_e, q_r', \sigma_c, q_c') \in \delta^{rev}_c }
}
\and
{\inferrule*[left=\textsc{Pending-Return}]
       { (q_c,\sigma,q_e) \in  \delta_c \\ q_f \in Q_f }
  { (q_e', q_f, \sigma, q_c') \in \delta^{rev}_R }
}
\end{mathpar}
%The \textsc{Pending-Return} rule is the most perplexing one. The name
%\textsc{Pending-Return} is written from the perspective of the reversed
%automaton -- in other words, the reversed automaton will take a transition
%added by this rule when it reads a pending return, and that pending return
%would have been a pending call in the original automaton. However, $q_f \in
%Q_f$ is written from the perspective of the original automaton: because
%$q_f$ is a final state in the original it is an initial state in the
%reversed automaton, and this is why the transition can be taken on a pending
%return.

The NWA resulting from performing reverse on the NWA shown in
Figure \ref{Fi:Example1} is shown in Figure \ref{Fi:Reverse1}.

Client
information is copied directly from the original NWA using
\texttt{ClientInfo::clone()}.

\subsection{Determinize}
\label{Se:Determinize}

\begin{definition}
An NWA, $(Q,\Sigma,Q_0,\delta,Q_f)$, is \textbf{deterministic} iff

\begin{enumerate}

\item $|Q_0| \leq 1$,

\item For all $q \in Q$, there is never a choice between reading $\sigma$ and
  following a $\sigma$ transition or following a wild (\wild) transition:
  \begin{itemize}
    \item if $(q,\wild,q') \in \delta_i$ then $|\{q'|(q,\sigma,q') \in
      \delta_i, {\sigma\neq@}\}| = 0$; \\ otherwise, for all $\sigma \in \Sigma - \{\wild\}$,
      $|\{q'|(q,\sigma,q') \in \delta_i\}| \leq 1$,

    \item if $(q,\wild,q') \in \delta_c$ then $ |\{q'|(q,\sigma,q') \in
      \delta_c, {\sigma\neq@}\}| = 0$;\\
      otherwise, for all $\sigma \in \Sigma - \{\wild\}$,
      $|\{q'|(q,\sigma,q') \in \delta_c\}| \leq 1$, and

    \item \mbox{for $q' \in Q$, if $(q,q',\wild,q'') \in \delta_r$ then
      $|\{q''|(q,q',\sigma,q'') \in \delta_r, {\sigma\neq@}\}| = 0$;} \\
      otherwise, for all
      $\sigma \in \Sigma - \{\wild\}$, $|\{q''|(q,q',\sigma,q'') \in \delta_r\}|
      \leq 1$,
  \end{itemize}
\item There are no $\epsilon$ transitions:
 \begin{itemize}
   \item for all $(q,\sigma,q') \in \delta_i, \sigma \neq \epsilon$,
   \item for all $(q,\sigma,q') \in \delta_c, \sigma \neq \epsilon$, and
   \item for all $(q,q',\sigma,q'') \in \delta_r, \sigma \neq \epsilon$.
 \end{itemize}
\end{enumerate}
If an NWA is not deterministic, then it is \textbf{non-deterministic}.
\end{definition}

Determinizing an NWA operates like a
generalization of the classical subset construction.  Instead of the states
in the determinized NWA being subsets of states in the original NWA, states of the
determinized NWA are sets of state pairs (i.e., binary relations on states)
\cite{JACM:AM2009}.  To support determinization, the library provides a
typedef of \texttt{std::set$<$pair$<$State, State$>>$} as
\texttt{Nwa::BinaryRelation}. See also \texttt{opennwa/RelationOps.hpp}.

We present the algorithm we use for determinization in
\appref{DeterminizeAlgorithm}.

%There is very experimental support for using the BuDDY

The result of determinizing the automaton in \figref{Det1} is shown in
\figref{Det2}.

\begin{figure}[p]
  \centering
    \nwaimage[0.45]{Figures/determinize}
  \caption{Simple nondeterministic NWA.}
  \label{Fi:Det1}
\end{figure}

\begin{figure}[p]
  \centering
    \nwaimage[.75]{Figures/determinize-result}
    \caption{The NWA resulting from determinizing the NWA in Figure
      \ref{Fi:Det1}. As mentioned in the text, states in the determinized NWA
      are relations on the states in the original NWA. The state
      $\varnothing$ has been removed from this diagram.}
  \label{Fi:Det2}
\end{figure}

Client information is generated through the use of the helper method
\texttt{mergeClientInfo}, but can be altered through the use of the helper
methods \texttt{mergeClientInfoInternal}, \texttt{mergeClientInfoCall}, and
\texttt{mergeClientInfoReturn}, which are invoked by \texttt{determinize} as
transitions of the three kinds involving the associated state are added.  The
default behavior of \texttt{mergeClientInfo} is that the \texttt{ClientInfo}
associated with the resulting state is \texttt{null}.  The default behavior
of \texttt{mergeClientInfoInternal}, \texttt{mergeClientInfoCall}, and
\texttt{mergeClientInfoReturn} is to make no changes to the the
\texttt{ClientInfo}.  These methods can be overridden to specify alternative
behaviors.  As determinization is performed, \texttt{mergeClientInfo} is
called each time a new state is created.  Then, as each transition is added,
\texttt{mergeClientInfoInternal}, \texttt{mergeClientInfoCall}, or
\texttt{mergeClientInfoReturn} is called (depending on the type of transition
being added) to update the \texttt{ClientInfo} associated with the target
state of the transition being added.

The following functions can be overridden in a subclass of \texttt{Nwa} to
customize the behavior of determinization:
\begin{functionlist}
  \functionDefNoCloseParen{void}{mergeClientInfo}{%
     \parbox[t]{4in}{
       Nwa const \& nondet, BinaryRelation const \& binRel, \\
       St resSt, ref\_ptr<ClientInfo>\& resCI)}}
  Callback that gets called when a new state \texttt{resSt} (representing the
  binary relation \texttt{binRel}) is added to the determinized automaton.
  Intended to provide a hook for computing the client information that should
  be associated with the new state; the client information should be set in
  the output parameter \texttt{resCI} (i.e., \texttt{setClientInfo} should not
  be called directly). \texttt{nondet} is the NWA being determinized.

  \functionDefFirstEarlyNoCloseParen{void}{mergeClientInfoInternal}{%
     \parbox[t]{4in}{
       Nwa const \& nondet,\\
       BinaryRelation const \& binRelSource,\\
       BinaryRelation const \& binRelTarget,\\
       State sourceSt, Symbol resSym, State resSt,\\
       ref\_ptr<ClientInfo>\& resCI )}}
  \functionDefEarlyNoCloseParen{void}{mergeClientInfoCall}{%
     \parbox[t]{4in}{
       Nwa const \& nondet,\\
       BinaryRelation const \& binRelCall,\\
       BinaryRelation const \& binRelEntry,\\
       State callSt, Symbol resSym, State resSt,\\
       ref\_ptr<ClientInfo>\& resCI )}}
  \functionDefNoCloseParen{void}{mergeClientInfoReturn}{%
     \parbox[t]{4in}{
       Nwa const \& nondet,\\
       BinaryRelation const \& binRelExit,\\
       BinaryRelation const \& binRelCall,\\
       BinaryRelation const \& binRelReturn,\\
       State exitSt, State callSt, Symbol resSym,\\
       State resSt, ref\_ptr<ClientInfo>\& resCI )}}
    Callbacks that get called when a new transition is added to the given
    automaton. The endpoints and their associated binary relations are
    given.
    Alters the client information associated with \texttt{resSt} given
    information about the transition being added to the determinized
    automaton.
 \end{functionlist}

%Consider the slightly more complex determinization of the NWA shown in Figure \ref{Fi:Det3}.  The resulting NWA is shown in Figure \ref{Fi:Det4}.

%\begin{figure}[htbp]
%  \centering
%    \includegraphics[angle=270,width=12cm]{Figures/Figure18.pdf}
%  \caption{Complex nondeterministic NWA.}
%  \label{Fi:Det3}
%\end{figure}

%\begin{figure}[htbp]
%  \centering
%    \includegraphics[angle=270,width=12cm]{Figures/Figure19.pdf}
%  \caption{The NWA resulting from determinizing the NWA in Figure \ref{Fi:Det3}.}
%  \label{Fi:Det4}
%\end{figure}

\subsection{Complement}
\label{Se:Complement}

Complementing an NWA is performed by determinizing the automaton and then
complementing the set of final states.
\begin{comment}
 In our implementation, an extra flag
to \texttt{complement} controls whether the determinization step is to be
performed, so it can be bypassed if you have \textsl{a priori} knowledge that
the input NWA must already be deterministic.
\end{comment}
The result of
complementing the NWA shown in Figure \ref{Fi:Det1} is
shown in Figure \ref{Fi:Comp1}.

Client information is computed during determinization as described in the
previous section, then cloned during the complement step.

\begin{figure}[h]
  \centering
    \nwaimage[1]{Figures/complement-of-determinize}
  \caption{The complement of the NWA in Figure \ref{Fi:Det1} (the
    determinization of which is shown in Figure \ref{Fi:Det2}).  We omit
    transitions to the state $\{\}$; any action that does not appear in the
    diagram goes to the state $\{\}$.
  }
  \label{Fi:Comp1}
\end{figure}


\documentclass[11pt, a4paper, oneside]{article}

\usepackage{color}
\usepackage{tikz}
\usepackage{showframe}
\usepackage{blindtext}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{bm}
\usepackage{calc}
\usepackage{enumitem}
\usepackage{xcolor}
\usepackage{lipsum}
\usepackage{setspace}
\usepackage{glossaries}
\usepackage{latexsym}

\tikzset{
    vertex/.style = {
        circle,
        fill = black,
        outer sep = 2pt,
        inner sep = 1pt,
    }
}

\SetLabelAlign{parright}{\parbox[t]{\labelwidth}{\raggedleft#1}}
\setstretch{0.5}

\DeclareMathOperator*{\argmin}{argmin}

\begin{document}

\title{}
\author{}
\date{}

%Dataset
%name: LS3D-W.
%download: https://www.adrianbulat.com/face-alignment/

\part*{Studies dependency}.

\section*{Face alignment 2D}

Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. In: CVPR. (2012)

Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: CVPR. (2013) 532–539

Jourabloo, A., Liu, X.: Pose-invariant 3d face alignment. In: CVPR. (2015) 3694–3702

Zhu, S., Li, C., Change Loy, C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: CVPR. (2015) 4998–5006

Bulat, A., Tzimiropoulos, G.: Convolutional aggregation of local evidence for large pose face alignment. In: BMVC. (2016)

\section*{Face alignment 3D}

Bulat, A.,Tzimiropoulos, G.: Two-stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge. ???? (2016)

\part*{Bulat 2017 Face Alignment}

\noindent
Normalized Mean Error:

\begin{equation}
\text{NME}
= \frac{1}{N} \sum_{k=1}^N \frac{\| \mathbf{x}_k - \mathbf{y}_k \|^2}{d}
\tag{}
\end{equation}

\begin{description}[labelwidth=\widthof{\bfseries 123456789012345},align=parright]
	\item[Input variable: ]	$d = \sqrt{w_{ \mathit{bbox}} * h_{\mathit{bbox}} }$
	\item[$\mathit{bbox}:$] bounding box.
\end{description}

\bigskip

\part*{Bulat 2016 Two stage 3D pose}

\begin{equation}
l_2 = \frac{1}{N} \sum_{n=1}^N (\tilde{z}_n - z_n) _2
\tag{}
\end{equation}

\bigskip

\noindent
Ground Truth Error(GTE):

\begin{equation}
E( \mathbf{X}, \mathbf{Y})
= \frac{1}{N} \sum_{n=1}^N \frac{\| \mathbf{X}_n - \mathbf{Y}_n \|_2}{d_i}
\tag{}
\end{equation}

\begin{description}[labelwidth=\widthof{\bfseries 123456789012345},align=parright]
	\item[$\mathbf{X}$:] Predicted set of points.
	\item[$\mathbf{Y}$:] Corresponding ground truth.
	\item[$d_i$:] Interocular distance for the $i$th image.
\end{description}

\bigskip

\noindent
Cross View Ground Truth Consistency Error(CVGTCE):
\begin{equation}
E_{vc}( \mathbf{X}, \mathbf{Y}, T )
=\frac{1}{N} \sum_{n=1}^1 \frac{\| s\mathbf{Rx}_n - \mathbf{y}_n \|_2}{d_i}
\tag{}
\end{equation}

\begin{equation}
P = \{ s, \mathbf{R}, \mathbf{t} \}
\tag{}
\end{equation}

\begin{equation}
\{ s, \mathbf{R}, \mathbf{t} \}
= \argmin_{s, \mathbf{R}, \mathbf{t}}
\sum_{n=1}^N \| \mathbf{y}_k - (s \mathbf{Rx} + \mathbf{•}mathbf{t}) \|_2 ^2
\end{equation}

\end{document}


\chapter[Agent Based Simulation]{Agent Based Simulation}\label{chp:agent_based_simulation}

\chapterinitial{S}{ometimes} individual behaviours and interactions are well
understood, and an understanding of how a whole population of such individuals
might behave needed. For example psychologists and economists may know a lot
about how individual spenders and vendors behave in response to given stimuli,
but an understanding of how these stimuli might effect the macro-economy is
necessary. Agent based\index{agent based} simulation is a paradigm of thinking
that allows such emergent population level behaviour to be investigated from
individual rules and interactions.

\section{Problem}\label{sec:agent_based_simulation_problem}

Consider a city populated by two categories of household, for example a household
might be fans of Cardiff City FC or Swansea City AFC\footnote{Swansea and
Cardiff are two cities in South Wales with rival football clubs.}.
Each household has a preference for living close to households of the same kind,
and will move around the city while their preferences are not satisfied.
How will these individual preferences affect the overall distribution of fans in
the city?

\section{Theory}\label{sec:agent_based_simulation_theory}

The problem considered here is considered a `classic' one for the paradigm of
agent based simulation, and is referred to as Schelling\index{Schelling}'s
segregation model. It features in Thomas Schelling's book `Micromotives and
Macrobehaviours'~\cite{schelling2006micromotives}, whose title neatly summarises
the world view of agent based modelling: we know, understand, determine, or can
control individual micromotives; and from this we'd like to observe and
understand macrobehaviours.

In general an agent based model consists of two components, agents, and an
environment:

\begin{itemize}
    \item agents are autonomous entities that will periodically choose to take one
        of a number of actions.
        These are chosen in order to maximise that agent's own given utility
        function;
    \item an environment contains a number of agents and defines how their
        interactions affect each other.
        The agents may be homogeneous or heterogeneous, and the
        relationships may change over time, possibly due to the actions taken
        by the agents.
\end{itemize}

In general, an agent will first observe a subset of its environment, for
example it will consider some information about the agents it is currently
close to.
Then it will update some information about itself based on these observations.
This could be recording relevant information from the observations, but could
also include some learning, maybe considering its own previous actions.
It will then decide on an action to take, and carry out this action. This
decision may be deterministic or random and/or based on its own attributes from
some learning process\index{learning process}; with the ultimate aim of
maximising its own utility. In practice, a utility\index{utility} can be
represented by a function that maps the environment to some numeric value.
This process happens to all agents in the environment, possibly simultaneously.
This is summarised in Figure~\ref{fig:abm_diagram}

\begin{figure}
    \begin{center}
        \includestandalone[width=\textwidth]{./assets/abm-diagram}
    \end{center}
    \caption{Representation of an agent interacting with its environment.}
    \label{fig:abm_diagram}
\end{figure}

For the football team supporters problem, each household is an agent.
The environment is the city.
Each household's utility function is to satisfy their preference of living next
to at least a given number of households supporting the same team as them.
Their choices of action are to move house or not to move house.

As a simplification the city will be modelled as a \(50 \times 50\) grid.
Each cell of the grid is a house that can either contain a household of Cardiff
City FC supporters, or contain a household of Swansea City AFC supporters.
A house's neighbours are assumed to be the houses adjacent to it, horizontally,
vertically, and diagonally.
For mathematical simplicity, it is also assumed that the grid is a torus,
where houses in the top row are vertically adjacent to the bottom row, and
houses in the rightmost column are horizontally adjacent to the leftmost column.

Every household has a preference \(p\).
This corresponds to the minimum proportion of neighbours they are happy to live
Figure~\ref{fig:schelling_happyunhappy} shows a household of Cardiff City FC
supporters that are happy with their neighbours, and not happy with their
neighbours, when \(p=0.5\). Households supporting Cardiff City FC are shaded
grey.

\begin{figure}
    \begin{center}
        \subfigure[A happy household, with 6 similar neighbours (\(\frac{6}{8} > p = 0.5\))]{\includestandalone[width=0.4\textwidth]{./assets/schelling_happy}}
        \subfigure[An unhappy household, with 2 similar neighbours (\(\frac{2}{8} < p = 0.5\))]{\includestandalone[width=0.4\textwidth]{./assets/schelling_unhappy}}
    \end{center}
    \caption{Example of a household happy and unhappy with its neighbours, when
    \(p=0.5\). Households supporting Cardiff City FC are shaded grey, households
    supporting Swansea City AFC are white.}
    \label{fig:schelling_happyunhappy}
\end{figure}

The original problem stated that households move around the city whenever they
are unhappy with their neighbours.
This long process of selling, searching for, and buying houses can be simplified
to randomly pairing two unhappy households and swapping their houses.
In fact, this can be simplified to consider the houses themselves as agents,
who swap households with each other.

Therefore the model logic is:

\begin{enumerate}
  \item initialise the model: fill each house in the grid with either a
  household of Cardiff City FC or Swansea City AFC supporters with
  probability \(0.5\) each;
  \item at each discrete time step, for every house:
  \begin{enumerate}
    \item consider their household's neighbours (\textit{observe});
    \item determine if the household is happy (\textit{update});
    \item if unhappy (\textit{decide}), swap household with another randomly
    chosen house with an unhappy household (\textit{action}).
  \end{enumerate}
\end{enumerate}

After a number of time steps the overall structure of the city can be observed.
This agent based model only explicitly defines individual behaviours
and interactions, therefore any population level behaviour that may have emerged
would occur without explicit definition.

\section{Solving with Python}\label{sec:agent_based_simulation_solving-with-python}

Agent based modelling lends itself well to a programming paradigm called
object-orientated programming\index{object-orientated programming}.
This paradigm lets a number of \textit{objects} from a set of instructions
called a \textit{class} to be built.
These objects can both store information (in Python these are called
\textit{attributes}\index{attribute}), and do things (in Python these are called
\textit{methods}\index{method}).
Object-orientated programming allow for the creation of new classes which can be
used to implement the individual behaviours of an agent based model.

For this problem two classes will be built: a
\mintinline{python}{House} and a \mintinline{python}{City} for them to live in.

The following libraries will be used:

\begin{pyin}
import random
import itertools
import numpy as np
\end{pyin}

Now to define the \mintinline{python}{City}:

\begin{pyin}
class City:
    def __init__(self, size, threshold):
        """Initialises the City object.

        Args:
            size: an integer number of rows and columns
            threshold: float between 0 and 1 representing the
            minimum acceptable proportion of similar neighbours
        """
        self.size = size
        sides = range(size)
        self.coords = itertools.product(sides, sides)
        self.houses = {
            (x, y): House(x, y, threshold, self)
            for x, y in self.coords
        }

    def run(self, n_steps):
        """Runs the simulation of a number of time steps.

        Args:
            n_steps: an integer number of steps
        """
        for turn in range(n_steps):
            self.take_turn()

    def take_turn(self):
        """Swaps all sad households."""
        sad = [h for h in self.houses.values() if h.sad()]
        random.shuffle(sad)
        i = 0
        while i <= len(sad) / 2:
            sad[i].swap(sad[-i])
            i += 1

    def mean_satisfaction(self):
        """Finds the average household satisfaction.

        Returns:
            The average city's household satisfaction
        """
        return np.mean(
            [h.satisfaction() for h in self.houses.values()]
        )
\end{pyin}

This defines a class, a template or a set of instructions that can be used to
create instances, called objects.
For the considered problem only one instance of the \mintinline{python}{City}
class will be needed.
However, it is useful to be able to produce more in order to run multiple trials
with different random seeds.
This class contains four methods: \mintinline{python}{__init__},
\mintinline{python}{run}, \mintinline{python}{take_turn} and
\mintinline{python}{mean_satisfaction}.

The \mintinline{python}{__init__} method is run whenever the object is first
created, and initialises the object.
In this case it sets a number of attributes.

\begin{itemize}
     \item First the square grid's \mintinline{python}{size} is defined, which
           is the number of rows and columns of houses it contains;
     \item next the \mintinline{python}{coords} are defined, a list of tuples
           representing all the possible coordinates of the grid, this uses the
           itertools library for efficient iteration;
     \item finally \mintinline{python}{houses} is defined, a dictionary with
           grid coordinates as keys, and instances of the
           \mintinline{python}{House} class.
\end{itemize}

The \mintinline{python}{run} method runs the simulation. For each
\mintinline{python}{n_steps} number of discrete time steps, the city runs the
method \mintinline{python}{take_turn}.
In this method, first a list of all the houses is created with households that
are unhappy with their neighbours; these are put in a random order using the
random library; and then working inwards from the boundary
houses with sad households are paired up and swap households.

The last method defined here is the \mintinline{python}{mean_satisfaction}
method, which is only used to observe any emergent behaviour\index{emergent behaviour}.
This calculates the average satisfaction of all the houses in the grid, using
the numpy library for convenience.

In order to be able to create an instance of the above class, we need to define
a \mintinline{python}{House} class:

\begin{pyin}
class House:
    def __init__(self, x, y, threshold, city):
        """Initialises the House object.

        Args:
            x: the integer x-coordinate
            y: the integer y-coordinate
            threshold: a number between 0 and 1 representing
              the minimum acceptable proportion of similar
              neighbours
            city: an instance of the City class
        """
        self.x = x
        self.y = y
        self.threshold = threshold
        self.kind = random.choice(["Cardiff", "Swansea"])
        self.city = city

    def satisfaction(self):
        """Determines the household's satisfaction level.

        Returns:
            A proportion
        """
        same = 0
        for x, y in itertools.product([-1, 0, 1], [-1, 0, 1]):
            ax = (self.x + x) % self.city.size
            ay = (self.y + y) % self.city.size
            same += self.city.houses[ax, ay].kind == self.kind
        return (same - 1) / 8

    def sad(self):
        """Determines if the household is sad.

        Returns:
            a Boolean
        """
        return self.satisfaction() < self.threshold

    def swap(self, house):
        """Swaps two households.

        Args:
            house: the house object to swap household with
        """
        self.kind, house.kind = house.kind, self.kind
\end{pyin}

It contains four methods: \mintinline{python}{__init__},
\mintinline{python}{satisfaction}, \mintinline{python}{sad} and
\mintinline{python}{swap}.

The \mintinline{python}{__init__} methods sets a number of attributes at the
time the object is created: the house's \mintinline{python}{x} and
\mintinline{python}{y} coordinates (its column and row numbers on the grid);
its \mintinline{python}{threshold} which corresponds to \(p\); its
\mintinline{python}{kind} which is randomly chosen between having a Cardiff City
FC supporting household or a Swansea City AFC supporting household; and finally
its \mintinline{python}{city}, an instance of the \mintinline{python}{City}
class, shared by all the houses.

The \mintinline{python}{satisfaction} method loops though each of the house's
neighbouring cells in the city grid, counts the number of neighbours that are
of the same kind as itself, and returns this as a proportion.
Then the \mintinline{python}{sad} method returns a boolean indicating if the
household's satisfaction is below the minimum threshold.

Finally the \mintinline{python}{swap} method takes another house object, and
swaps their household kinds.

A function to create and run one of these simulations will now be written
with a given random seed\index{random seed}, threshold, and number of steps.
This function returns the resulting mean happiness:

\begin{pyin}
def find_mean_happiness(seed, size, threshold, n_steps):
    """Create and run an instance of the simulation.

    Args:
        seed: the random seed to use
        size: an integer number of rows and columns
        threshold: a number between 0 and 1 representing
            the minimum acceptable proportion of similar
            neighbours
        n_steps: an integer number of steps

    Returns:
        The average city's household satisfaction after
        n_steps
    """
    random.seed(seed)
    C = City(size, threshold)
    C.run(n_steps)
    return C.mean_satisfaction()
\end{pyin}

Now consider each household with a threshold of
0.65, and compare the mean happiness after 0 steps and 100 steps.
First 0 steps:

\begin{pyin}
initial_happiness = find_mean_happiness(
    seed=0, size=50, threshold=0.65, n_steps=0
)
print(initial_happiness)
\end{pyin}

\begin{pyout}
0.4998
\end{pyout}

This is well below the minimum threshold of \(0.65\), and so on average
households are unhappy.
After 100 steps:

\begin{pyin}
final_happiness = find_mean_happiness(
    seed=0, size=50, threshold=0.65, n_steps=100
)
print(final_happiness)
\end{pyin}

\begin{pyout}
0.9078
\end{pyout}

After 100 time steps the average satisfaction level is much higher.
In fact, it is much higher than each individual household's threshold.
Now consider that this satisfaction level is really a level of how similar
each households' neighbours are, it is actually a level of segregation.
This was the central premise of Schelling's original
model~\cite{schelling2006micromotives}
that overall emergent segregation levels are much higher than any individuals'
personal preference for segregation.

More analysis methods can be added, including plotting functions.
Figure~\ref{fig:schelling_python_plot} shows the grid at the beginning, after 20
time steps, and after 100 time steps, with households supporting Cardiff City FC
in grey, and those supporting Swansea City AFC in white.
It visually shows the households segregating over time.

\begin{figure}
\begin{center}
\subfigure[At the beginning.]{\includegraphics[width=0.32\textwidth]{./assets/python_schelling_0}}
\subfigure[After 20 time steps.]{\includegraphics[width=0.32\textwidth]{./assets/python_schelling_20}}
\subfigure[After 100 time steps.]{\includegraphics[width=0.32\textwidth]{./assets/python_schelling_100}}
\end{center}
\caption{Plotted results from the Python code.}
\label{fig:schelling_python_plot}
\end{figure}

\section{Solving with R}\label{sec:agent_based_simulation_solving-with-R}

Agent based modelling lends itself well to a programming paradigm called
object-orientated programming\index{object-orientated programming}.
This paradigm lets a number of \textit{objects} from a set of instructions
called a \textit{class} to be built.
These objects can both store information (in the R package used here these are called
\textit{fields}\index{field}), and do things (in the R package used here these are called
\textit{methods}\index{method}).
Object-orientated programming allow for the creation of new classes which can be
used to implement the individual behaviours of an agent based model.

There are a number of ways of doing object-orientated programming in R.
In this chapter, a package called R6\index{R6} will be used here.

For this problem two classes will be built: a
\mintinline{R}{House} and a \mintinline{R}{City} for them to live in.

Now to define the \mintinline{R}{City}\footnote{
Note that no documentation is included in the definition
of the class and the style is inconsistent with the other chapters in this book.
The authors do not know of any widely accepted convention for documenting classes
in R, furthermore the text of this chapter serves as detailed documentation in this
context.
}

\begin{Rin}
library(R6)
City <- R6Class("City", list(
  size = NA,
  houses = NA,
  initialize = function(size, threshold) {
    self$size <- size
    self$houses <- c()
    for (x in 1:size) {
      row <- c()
      for (y in 1:size) {
        row <- c(row, House$new(x, y, threshold, self))
      }
      self$houses <- rbind(self$houses, row)
    }
  },
  run = function(n_steps) {
    if (n_steps > 0) {
      for (turn in 1:n_steps) {
        self$take_turn()
      }
    }
  },
  take_turn = function() {
    sad <- c()
    for (house in self$houses) {
      if (house$sad()) {
        sad <- c(sad, house)
      }
    }
    sad <- sample(sad)
    num_sad <- length(sad)
    i <- 1
    while (i <= num_sad / 2) {
      sad[[i]]$swap(sad[[num_sad - i]])
      i <- i + 1
    }
  },
  mean_satisfaction = function() {
    mean(sapply(self$houses, function(x) x$satisfaction()))
  }
) )
\end{Rin}

This defines an R6 class, a template or a set of instructions that can be used
to create instances of it, called objects.
For this model only one instance of the \mintinline{R}{City} class is needed,
although it may be useful to be able to produce more in order to run multiple
trials with different random seeds.
This class contains four methods: \mintinline{R}{initialize},
\mintinline{R}{run}, \mintinline{R}{take_turn} and
\mintinline{R}{mean_satisfaction}.

The \mintinline{R}{initialize} method is run at the time the object is first
created.
It initialises the object by setting a number of its fields:

\begin{itemize}
     \item First the square grid's \mintinline{R}{size} is defined, which
           is the number of rows and columns of houses it contains;
     \item then the \mintinline{R}{houses} are defined by iteratively repeating
           the \mintinline{R}{rbind} function to create a two-dimensional vector
           of instances of the, yet to be defined, \mintinline{R}{House} class,
           representing the houses themselves.
\end{itemize}

The \mintinline{R}{run} method runs the simulation. For each discrete time
step from 1 to \mintinline{R}{n_steps}, the world runs the method
\mintinline{R}{take_turn}.
In this method, a list of all the houses with households that
are unhappy with their neighbours is created;
these are put in a random order and then working inwards from the boundary,
houses with sad households are paired up and swap households.

The last method defined here is the \mintinline{R}{mean_satisfaction}
method, which is used to observe the emergent behaviour\index{emergent behaviour}.
This calculates the average satisfaction of all the houses in the grid.

In order to be able to create an instance of the above class,
a \mintinline{R}{House} class is needed:

\begin{Rin}
House <- R6Class("House", list(
  x = NA,
  y = NA,
  threshold = NA,
  city = NA,
  kind = NA,
  initialize = function(x = NA,
                        y = NA,
                        threshold = NA,
                        city = NA) {
    self$x <- x
    self$y <- y
    self$threshold <- threshold
    self$city <- city
    self$kind <- sample(c("Cardiff", "Swansea"), 1)
  },
  satisfaction = function() {
    same <- 0
    for (x in -1:1) {
      for (y in -1:1) {
        ax <- ( (self$x + x - 1) %% self$city$size) + 1
        ay <- ( (self$y + y - 1) %% self$city$size) + 1
        if (self$city$houses[[ax, ay]]$kind == self$kind) {
          same <- same + 1
        }
      }
    }
    (same - 1) / 8
  },
  sad = function() {
    self$satisfaction() < self$threshold
  },
  swap = function(house) {
    old <- self$kind
    self$kind <- house$kind
    house$kind <- old
  }
) )
\end{Rin}

It contains four methods: \mintinline{R}{initialize},
\mintinline{R}{satisfaction}, \mintinline{R}{sad} and \mintinline{R}{swap}.

The \mintinline{R}{initialize} methods sets a number of the class' fields when
the object is created: the house's \mintinline{R}{x} and \mintinline{R}{y}
coordinates (its column and row numbers on the grid); its
\mintinline{R}{threshold} which corresponds to \(p\); its \mintinline{R}{kind}
which is randomly chosen between having a Cardiff City
FC supporting household or a Swansea City AFC supporting household; and finally
its \mintinline{R}{city}, an instance of the \mintinline{R}{City} class, shared
by all the houses.

The \mintinline{R}{satisfaction} method loops though each of the house's
neighbouring cells in the city grid, counts the number of neighbours that are of
the same kind as itself, and returns this as a proportion.
The \mintinline{R}{sad} method returns a boolean indicating of the
household's satisfaction is below its minimum threshold.

Finally the \mintinline{R}{swap} method takes another house object, and
swaps their household kinds.

A function to create and run one of these simulations will now be written
with a given random seed, threshold, and number of steps. This function return
the resulting mean happiness:

\begin{Rin}
#' Create and run an instance of the simulation.
#'
#' @param seed: the random seed to use
#' @param size: an integer number of rows and columns
#' @param threshold: a number between 0 and 1 representing
#'   the minimum acceptable proportion of similar neighbours
#' @param n_steps: an integer number of steps
#'
#' @return The average city's household satisfaction
#'   after n_steps
find_mean_happiness <- function(seed,
                                size,
                                threshold,
                                n_steps){
  set.seed(seed)
  city <- City$new(size, threshold)
  city$run(n_steps)
  city$mean_satisfaction()
}
\end{Rin}

Now consider each household with a threshold of
0.65, and compare the mean happiness after 0 steps and 100 steps.
First 0 steps:

\begin{Rin}
initial_happiness <- find_mean_happiness(
  seed = 0,
  size = 50,
  threshold = 0.65,
  n_steps = 0
)
print(initial_happiness)
\end{Rin}

\begin{Rout}
[1] 0.4956
\end{Rout}

This is well below the minimum threshold of \(0.65\), and so on average
households are unhappy here.
Let's run the simulation for 100 generations and see how this changes:

\begin{Rin}
final_happiness <- find_mean_happiness(
  seed = 0,
  size = 50,
  threshold = 0.65,
  n_steps = 100
)
print(final_happiness)
\end{Rin}

\begin{Rout}
[1] 0.9338
\end{Rout}

After 100 time steps the average satisfaction has increased.
It is now actually much higher that each individual household's threshold.
This satisfaction level can be considered as a level of how similar each
households' neighbours are, and so it is actually a level of segregation.
This was the central premise of Schelling's original
model~\cite{schelling2006micromotives}, that overall
emergent segregation levels are much higher than any individuals' personal
preference for segregation.

More analysis methods can be added, including plotting functions.
Figure~\ref{fig:schelling_R_plot} shows the grid at the beginning, after 20
time steps, and after 100 time steps, with households supporting Cardiff City FC
in grey, and those supporting Swansea City AFC in white.
It shows the households segregating over time.

\begin{figure}
\begin{center}
\subfigure[At the beginning.]{\includegraphics[width=0.32\textwidth]{./assets/schelling_R_0}}
\subfigure[After 20 time steps.]{\includegraphics[width=0.32\textwidth]{./assets/schelling_R_20}}
\subfigure[After 100 time steps.]{\includegraphics[width=0.32\textwidth]{./assets/schelling_R_100}}
\end{center}
\caption{Plotted results from the R code.}
\label{fig:schelling_R_plot}
\end{figure}

\section{Wider context}\label{sec:agent_based_simulation_wider_context}

The simulations described in this chapter come under the larger umbrella term of
multi agent systems, which discusses the theory of systems with multiple
independent agents interacting with one another. A good source on the topic
is~\cite{shoham2008multiagent}.

The model described here is called Schelling's\index{Schelling} segregation
model, and was first described in~\cite{schelling2006micromotives}. Another model
considered as classic in this domain is a model of a flock of birds presented
in~\cite{reynolds1987flocks}, otherwise referred to as Boids\index{Boids}, where the
behaviours of flocks of birds are understood by capturing the individual
interactions between individual birds. Conway's Game of
Life\index{Conway's Game of Life}, described in~\cite{gardener1970mathematical} is
another classic, which comes under the banner of cellular
automata\index{cellular automata}. Here cells on a grid either become alive or
dead depending on a certain simplistic set of rules. Emergent behaviours
observed due to these rules include self replicating as well as oscillating
structures. In the 1970s agent based tournaments were held by Robert
Axelrod\index{Axelrod}~\cite{axelrod1990evolution},
which was the first of a number of studies using agent based modelling and game
theory\index{game theory} (see Chapter~\ref{chp:game_theory}) to understand the
emergence of cooperative behaviours.

In recent years, similar methodologies have been used in a variety of
applications such as~\cite{diaz2021agent} which models parents' choice of school,
in~\cite{romanowska2019agent} archaeological population migration and trade
dynamics are modelled, and~\cite{jing2020agent} offers a systematic
literature review for the use of agent based modelling of autonomous vehicles.


%===============================================================================
% COVID-19 Reproduction Numbers in South Africa
% Kevin Durant
% May 2020
%===============================================================================

\documentclass[12pt,a4paper]{article}

% \usepackage{booktabs} % better looking tables.
\usepackage{mathtools} % also loads amsmath.
\usepackage{microtype}
\usepackage{iftex}
\ifPDFTeX
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
  \usepackage{amssymb}
\else
  % Unicode-math should be loaded after other maths- or font-related packages.
  % It loads amsmath and fontspec if necessary, and enables a custom version of
  % the Latin Modern Math font by default.
  \usepackage{unicode-math}
%   \setmainfont{STIX Two Text}
%   \setmathfont{STIX Two Math}
\fi
\usepackage{biblatex}
\usepackage{svg}
\usepackage{hyperref}
\addbibresource{cvza.bib}

% General maths commands %======================================================

\newcommand\ds{\displaystyle}                 % large maths.
\newcommand\ts{\textstyle}                    % small maths.
\newcommand\mb[1]{\mathbb{#1}}                % mathbb shorthand.
\newcommand\mc[1]{\mathcal{#1}}               % mathcal shorthand.
\newcommand\ub[1]{\symbf{#1}}                 % unicode-math symbf shorthand.
\newcommand\ff[1]{^{\underline{#1}}}          % falling factorial.
\newcommand\rf[1]{^{\overline{#1}}}           % rising factorial.
\newcommand\ul[1]{\underline{#1}}             % underline.
\newcommand\ol[1]{\overline{#1}}              % overline.
\DeclareMathOperator\Pb{P}                    % probability.
\DeclareMathOperator\Ex{E}                    % expected value.
\DeclareMathOperator\Va{V}                    % variance.
\DeclarePairedDelimiter\lr{\lparen}{\rparen}  % sized parentheses.
\DeclarePairedDelimiter\lrb{\lbrack}{\rbrack} % sized brackets.
\DeclarePairedDelimiter\abs{\lvert}{\rvert}   % absolute value symbol.
\DeclarePairedDelimiter\cl{\lceil}{\rceil}    % ceiling symbol.
\DeclarePairedDelimiter\fl{\lfloor}{\rfloor}  % floor symbol.
\DeclareMathOperator\B{B}                     % beta distribution.
\DeclareMathOperator\BP{BP}                   % beta prime distribution.
\DeclareMathOperator\NB{NB}                   % negative binomial distribution.

% Title %=======================================================================

\title{Estimating COVID-19 Reproduction Numbers}
\author{Kevin Durant}
\date{June 2020}

% Document %====================================================================

\begin{document}

\maketitle

\begin{abstract}
This is a short description of an analytic, \emph{almost} fully Bayesian
solution to the problem of inferring effective reproduction numbers for an
epidemic from reported infection counts. Under the assumptions of a negative
binomial likelihood function and a simple beta prime distributed predictive
prior, the posterior distribution of the daily infection rate follows a beta
prime distribution whose parameters can be expressed in closed form. These
posteriors can in turn be used to estimate effective reproduction numbers.
\end{abstract}

\section{Introduction} %========================================================

The effective reproduction number of an epidemic, $R_t$, refers to the expected
number of new infections caused by a single infected individual at a given stage
of the epidemic's course. Along with other, complementary measures, these
numbers can help give an indication of the rate at which the epidemic is
spreading.

The method for estimating effective reproduction numbers discussed here is a
slightly different take on one described elsewhere by Kevin
Systrom~\cite{systrom2020}, which is itself based on work
by~\citeauthor{bettencourt2008}~\cite{bettencourt2008}. Unlike those two
approaches, the solution described here is almost entirely analytic---the only
numerical computation required being that of percentiles of a beta prime
distribution. All three methods are based on a key point made by the latter
authors mentioned above: that under the assumptions of a standard epidemic
susceptible-infected~(SIR) model, the effective reproduction number of a virus
at time~$t$ can be estimated from the number of new cases recorded between
times~$t-1$ and~$t$, and~$t$ and~$t+1$.

More specifically, let~$R_t$ be the effective reproduction number at time~$t$,
and assume that this number remains constant over the interval~$(t-1, t]$ (most
likely a single day)\footnote{Note that our notation differs slightly from that
used by~\citeauthor{bettencourt2008}---our~$R_t$ and~$\lambda_t$ correspond to
their~$R_{t-1}$ and~$\Delta T(t)$ respectively.}. Likewise, let~$\lambda_t$ be
the average number of new infections that occur during this interval---i.e., the
current rate of infection. One then has the following
approximation~\cite{bettencourt2008}:
\begin{equation*}
  \lambda_t \approx \lambda_{t-1}\exp(\gamma(R_t - 1)),
\end{equation*}
in which $\gamma$ is the reciprocal of the infectious period of the virus.
Equivalently,
\begin{equation}\label{eqn:R_t}
  R_t \approx \frac{1}{\gamma} \log\lr*{\frac{\lambda_t}{\lambda_{t-1}}} + 1.
\end{equation}

The approach outlined here involves modelling the number of observed
infections~$k_t$ as a stochastic process, assuming that $k_t$ depends on an
underlying rate of infection~$\lambda_t$ via a negative binomial distribution (a
Poisson distribution can also be used). One can then infer the infection
rates~$\lambda_t$ analytically, and use them to estimate effective reproduction
numbers by applying equation~\eqref{eqn:R_t}.

Specifically, one finds that the posterior rate of infection~$\lambda_t$ can be
described using a beta prime distribution:
\begin{equation*}
  \Pb\lr*{\frac{\lambda_t}{r} \Bigm\vert k_1, \dots, k_{t-1}}
  = \BP\lr*{\frac{\lambda_t}{r} \Bigm\vert \alpha_t, \beta_t},
\end{equation*}
in which
\begin{align}\label{eqn:recursion}
\begin{split}
    \alpha_t &= \frac{a_1}{c^{t-1}} + \sum_{i=0}^{t-1} \frac{k_{t-i}}{c^i}, \\
    \beta_t &= \frac{b_1}{c^{t-1}} + \sum_{i=0}^{t-1} \frac{r}{c^i}
    = \frac{b_1}{c^{t-1}} + r\frac{1 - \frac{1}{c^t}}{1 - \frac{1}{c}}.
\end{split}
\end{align}
Both $r$ and $c$ are model parameters that can easily be optimised, because
their marginal likelihood~$\Pb(k_1, \dots, k_{t-1} \mid r, c)$ is available in
closed form. The constants $a_1$ and~$b_1$ are parameters of the initial prior
on $\lambda_1$.

We apply this model to the estimation of reproduction numbers in
section~\ref{sec:application}, by making use of a simplified version of
equation~\eqref{eqn:R_t} in which $\lambda_{t-1}$ is replaced with a point
estimate~$\lambda_{t-1}^*$. We use, for example, the median value of
$\Pb(\lambda_{t-1} \mid k_1, \dots, k_{t-2})$ as such an estimate.

\section{The stochastic process} %==============================================

Let~$\ub{k} = k_1, \dots, k_{t-1}$ be a sequence of observed infection counts,
and~$\ub{\lambda}$ the corresponding sequence of unknown infection rates. The
primary assumption is that each~$k$ and~$\lambda$ are related via a negative
binomial distribution:
\begin{equation}\label{eqn:likelihood}
  \Pb(k_t \mid \lambda_t, \ub{k}) = \Pb(k_t \mid \lambda_t)
  \sim \NB\lr*{k_t \Bigm\vert r, \frac{\lambda_t}{r + \lambda_t} = p_t},
\end{equation}
where $r$ is an unknown dispersion parameter and $p_t$ is a reparameterisation
of $\lambda_t$ as a `success' probability. Parameterised in this way, the
negative binomial distribution converges to a Poisson distribution of
rate~$\lambda_t$ as~$r \to \infty$, and by adjusting $r$ we can control the
level of variance inherent to the distribution (smaller values of $r$ result in
higher variance).

Inference of the rate sequence~$\ub{\lambda}$ is performed iteratively, by
repeated application of Bayes' rule:
\begin{equation*}
  \Pb(\lambda_t \mid k_t, \ub{k}) \propto \Pb(k_t \mid \lambda_t)
    \Pb(\lambda_t \mid \ub{k}).
\end{equation*}
The first term on the right-hand side---the likelihood function---is simply the
negative binomial distribution given above. The second term is a predictive
prior on~$\lambda_t$ given only the \emph{past} infection counts~$\ub{k}$.
Technically the inference will be done with respect to $\ub{p} = p_1, \dots,
p_{t-1}$, not $\ub{\lambda}$, but with the right choice of prior the translation
between the two is seamless.

The conjugate prior for the negative binomial likelihood function (with known
dispersion) is the beta distribution, so our second assumption is that the prior
distribution on $p_t$ is of this form. Note that the change-of-variable formula
for probability density functions implies that when $p_t$ follows a beta
distribution, $\lambda_t/r$ is distributed according to a beta prime
distribution with identical parameters:
\begin{equation*}
  p_t \sim \B(p_t \mid \alpha, \beta)
  \Rightarrow \frac{\lambda_t}{r}
    \sim \BP\lr*{\frac{\lambda_t}{r} \Bigm\vert \alpha, \beta}.
\end{equation*}
As mentioned above, this allows us to work with $p_t$ instead of $\lambda_t$
while deriving posteriors and marginal likelihoods, but still consider
$\lambda_t$ when computing $R_t$.

The third and final assumption we make involves the way in which the predictive
prior $\Pb(p_t \mid \ub{k})$ is derived from the previous posterior $\Pb(p_{t-1}
\mid \ub{k})$. In the case of a Gaussian stochastic process, one would derive
the prior by assuming additive Gaussian noise on the previous latent variable,
resulting in a distribution that has the same mean as the previous posterior,
but higher variance. Doing so involves solving an integral of the form
\begin{equation*}
  \Pb(p_t \mid \ub{k}) = \int \Pb(p_t \mid p_{t-1})
    \Pb(p_{t-1} \mid \ub{k})\, dp_{t-1},
\end{equation*}
which is tractable in the Gaussian case.

Although the situation is not quite as straightforward here, we can achieve a
similar outcome by simply assuming the relationship to the previous posterior
directly: specifically, if
\begin{equation}\label{eqn:posterior form}
  \Pb(p_{t-1} \mid \ub{k}) \sim \B(p_{t-1} \mid \alpha_{t-1}, \beta_{t-1}),
\end{equation}
we might assume a predictive prior of the form
\begin{equation}\label{eqn:prior}
  \Pb(p_t \mid \ub{k}) \sim \B(p_t \mid \alpha_{t-1}/c, \beta_{t-1}/c)
  = \B\lr*{p_t \mid a_t, b_t}.
\end{equation}
This is a straightforward prior that has the same mean as the predictive
posterior on~$p_{t-1}$, but a variance that is larger \emph{roughly} by a
factor~$c$---since the mean and variance of a beta distribution with parameters
$\alpha$ and $\beta$ are given by
\begin{equation*}
  \Ex[X] = \frac{\alpha}{\alpha + \beta}, \quad
  \Va[X] = \frac{\alpha\beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)},
\end{equation*}
and in general $c$ will be small relative to $\alpha + \beta$.

One could just as easily make use of a predictive prior that introduces additive
noise, unlike the multiplicative noise described above. The main reason for
choosing multiplicative noise here is that changes in scale made to~$p_t$ (and
$\lambda_t$) result in additive changes to~$R_t$, so in this way one is
effectively introducing additive noise into the overarching reproduction number
process.

The remaining details of the stochastic process now follow from
assumptions~\eqref{eqn:likelihood} and~\eqref{eqn:prior}. Firstly--and most
importantly---the posterior on~$p_t$ is given by
\begin{align*}
  \Pb(p_t \mid k_t, \ub{k}) &\propto \Pb(k_t \mid p_t) \Pb(p_t \mid \ub{k}) \\
  &= \NB(k_t \mid r, p_t)\, \B(p_t \mid a_t, b_t) \\
  &\sim p_t^{k_t} (1 - p_t)^r \cdot p_t^{a_t-1} (1 - p_t)^{b_t-1} \\
  &\Rightarrow \B(p_t \mid a_t + k_t, b_t + r).
\end{align*}
That is (extending equation~\eqref{eqn:posterior form}):
\begin{equation}\label{eqn:posterior}
  \Pb(p_t \mid k_t, \ub{k}) \sim \B(p_t \mid a_t + k_t, b_t + r)
  = \B(p_t \mid \alpha_t, \beta_t).
\end{equation}
Combining this with equation~\eqref{eqn:prior} one can solve for $\alpha_t$
and~$\beta_t$ recursively, leading to equation~\eqref{eqn:recursion}. Note
however that these solutions presume that the time series of data points is
complete; in the presence of a missing datum $k_t$ one will need to specify
$\alpha_t$ and~$\beta_t$ explicitly---for example simply by setting them to
$a_t$ and~$b_t$ respectively.

Secondly, we can derive the marginal likelihood of observation~$k_t$ given the
previous observations:
\begin{align}\label{eqn:marginal}
  \Pb(k_t \mid \ub{k}) &= \int \Pb(k_t, p_t \mid \ub{k})\, dp_t \nonumber \\
  &= \int \NB(k_t \mid r, p_t)\, \B(p_t \mid a_t, b_t)\, dp_t \nonumber \\
  &= \binom{k_t + r - 1}{k_t} \frac{1}{\Beta(a_t, b_t)}
    \int p_t^{a_t+k_t-1} (1 - p_t)^{b_t+r-1}\, dp_t \nonumber \\
  &= \begin{dcases}
    \frac{\Beta(a_t, b_t + r)}{\Beta(a_t, b_t)} &\text{if } k_t = 0, \\
    \frac{1}{k_t\Beta(k_t, r)}\frac{\Beta(a_t + k_t, b_t + r)}{\Beta(a_t, b_t)}
      &\text{if } k_t > 0,
  \end{dcases}
\end{align}
where~$\Beta(x, y)$ denotes the beta function. (The final expression can also be
rephrased in terms of $\alpha_t$ and $\beta_t$ using
equation~\eqref{eqn:posterior}.) This allows us to compute the overall marginal
likelihood iteratively, since
\begin{equation*}
  \Pb(\ub{k}) = \prod_{i=1}^{t-1} \Pb(k_i \mid k_1, \dots, k_{i-1}).
\end{equation*}
The overall marginal likelihood will in turn allow us to compare the relative
likelihoods of values of~$r$ and~$c$, which are the model's two tunable
parameters.

Equations~\eqref{eqn:prior} and~\eqref{eqn:posterior} allow us to derive,
via~$p_t$, a posterior distribution for each~$\lambda_t$. Ideally one would hope
for a distribution on~$\lambda_t/\lambda_{t-1}$, since this is the ratio on
which~$R_t$ depends (equation~\eqref{eqn:R_t}), however this would require one
to either specify~$\Pb(p_t \mid p_{t-1})$---something we explicitly avoiding
doing above to keep things tractable---or treat $\lambda_t$ and~$\lambda_{t-1}$
as independent for the purposes of deriving $R_t$.

Here we adopt a simpler approach: replace $\lambda_{t-1}$ in
equation~\eqref{eqn:R_t} with a point estimate~$\lambda_{t-1}^*$, resulting in
the approximation
\begin{equation}\label{eqn:R_t approximation}
  R_t \approx \frac{1}{\gamma} \log\lr*{\frac{\lambda_t}{\lambda_{t-1}}} + 1
    \approx \frac{1}{\gamma}
    \log\lr*{\frac{\lambda_t}{r}\frac{r}{\lambda_{t-1}^*}} + 1,
\end{equation}
to which our inferred posterior on $\lambda_t/r$ can directly be applied. This
is not unlike a simplification made by the other
authors~\cite{bettencourt2008,systrom2020}, who set~$\lambda_{t-1}^* = k_{t-1}$.
In what follows, we have set $\lambda_{t-1}^*$ to the median of the posterior
distribution~$\Pb(\lambda_{t-1} \mid k_1, \dots k_{t-1})$ (which is simply $r$
times the median of the posterior on $\lambda_{t-1}/r$).

Before moving on, we note that~\citeauthor{bettencourt2008} also describe a
negative binomial process in which $r$ is not constant, but rather $r_t =
k_{t-1}$. We have briefly tested this approach, and in practice it performs
similarly to the one we have described above; the main difference being that the
precision of the likelihood function (and thus posterior) varies with the
observed counts~$\ub{k}$. The approach outlined in this section can still be
applied, and the resulting analytic solution differs only slightly.

\subsection*{An application to South African data}\label{sec:application} %=====

In this section we apply the steps described above to data stemming from the
COVID-19 epidemic in South Africa. The data set used here is obtained from South
Africa's National Institute for Communicable Diseases, via the University of
Pretoria~\cite{dsfsi2020}. The data take the form of daily cumulative infection
counts per province, from which we derive new infection counts per day.

\begin{figure}[htb]
  \centering
  \includeinkscape{img/counts}
  \caption{Daily new infection counts for the COVID-19 epidemic in South Africa,
    smoothed using a Gaussian window with a standard deviation of~$\sigma =
    3.5$. Shaded areas indicate different levels of the nationwide lockdown,
    with the first, grey area depicting the period between the announcement of
    the lockdown and its initiation.}
  \label{fig:counts}
\end{figure}

Before inferring daily infection rates, we first need to decide whether or not
the raw counts should be smoothed, and if so, to what extent (see
figure~\ref{fig:counts}). The obvious argument against doing so is that the
variance of $R_t$ depends indirectly on the variance of the $k_t$, so smoothed
infection counts may result in artificially precise posteriors.

On the other hand, one might argue that the underlying assumption of the model
is that $k_t$ represents the \emph{true} number of infections on a given day, of
which smoothed counts are likely a more appropriate indication. A second, more
technical argument for reducing the variance of the reported infection counts is
the fact that the equation $\lambda_t \approx \lambda_{t-1}\exp(\gamma(R_t -
1))$ contains an implicit assumption: that $R_t \ge 0$, and thus
$\lambda_t/\lambda_{t-1} \ge \exp(-\gamma)$.

\begin{figure}[htb]
  \centering
  \includeinkscape{img/ratio}
  \caption{Ratios of consecutive infection counts~$k_t/k_{t-1}$ for various
    levels of Gaussian smoothing. The red line indicates the bound implied by
    equation~\eqref{eqn:R_t}.}
  \label{fig:ratio}
\end{figure}

In figure~\ref{fig:ratio} we plot the ratio~$k_t/k_{t-1}$ for various levels of
Gaussian averaging---roughly corresponding to smoothing windows of 0.5, 1, and~2
weeks (if one uses four standard deviations as a guide). One sees that in this
case, windows with scales greater than $\sigma = 7/3$ appear both to respect the
implicit bound mentioned above, as well as alleviate much of the periodicity
visible in the raw count sequence.

For the remainder of this example we will counts that have been averaged using a
Gaussian window with standard deviation~$\sigma = 3.5$. This value will not
necessarily be suitable for data sets from other regions or sources, however.

The rest of the application is straightforward: we apply
equation~\eqref{eqn:marginal} to select values for $r$ and~$c$ that maximise the
marginal likelihood (for the countrywide South African data this yields $r =
1592$ and~$c \approx 3.56$), and then use equation~\ref{eqn:posterior} to
compute the parameters of the posteriors on $\lambda_1, \lambda_2, \dots$. These
posteriors, shown in figure~\ref{fig:posterior}, track the smoothed $k_t$
relatively closely, as one might expect.

Finally, equation~\eqref{eqn:R_t approximation} allows us to plot the estimated
evolution of $R_t$ over time, simply by mapping percentiles of $\lambda_t/r$ to
those of~$R-t$. This plot is shown in figure~\ref{fig:countrywide} for the
entire country, and figure~\ref{fig:provincial} on a provincial level.

\begin{figure}[htbp]
  \centering
  \includeinkscape{img/posterior}
  \caption{Daily posterior infection rates~$\lambda_t$, plotted as a median and
    5--95th percentile interval.}
  \label{fig:posterior}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includeinkscape{img/countrywide}
  \caption{Estimated reproduction numbers~$R_t$ for the COVID-19 epidemic in
    South Africa, plotted as a median and 5--95th percentile interval.}
  \label{fig:countrywide}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includeinkscape{img/provincial}
  \caption{Reproduction numbers for the COVID-19 epidemic in South Africa,
    estimated at a provincial level.}
  \label{fig:provincial}
\end{figure}

The only comment we will make on these figures is that the effect of the hard
lockdown (implemented on 28 March) on the estimated reproduction numbers is
clearly visible, as is---to a lesser extent---the slow relaxation of lockdown
restrictions and gradual reopening of economic activity that follows.

% Bibliography %================================================================

\printbibliography

\end{document}