\documentclass[main.tex]{subfiles}
\begin{document}

\subsection{The physical effects of Comptonization}

\marginpar{Thursday\\ 2020-8-27, \\ compiled \\ \today}

As we have said it is hard to find numerical solutions to the Kompaneets equation, however there is an interesting quantity whose evolution we can study analytically: the photon energy density.
The energy density can be calculated as
%
\begin{align}
u = \int \dd[3]{p} h \nu \frac{N}{ \dd[3]{p}\dd[3]{x}}
= \int \dd[3]{p} h \nu   \frac{2 n}{h^3}
= \frac{8 \pi (k_B T)^{4}}{h^3c^3} \int_0^{\infty } \dd{x} x^3 n
\,,
\end{align}
%
where, as usual, \(x = h \nu / k_B T\), and the last equality comes about by substituting this and \(\dd[3]{p} = p^2 \dd{p} \dd{\Omega }\) and assuming isotropy.

In order to see how it changes in the first stage of the Comptonization, when the photon energy is low compared to the electron temperature, let us differentiate this energy density with respect to time (rescaled, as in the Kompaneets equation, by the mean time between two scatterings):
%
\begin{align}
\dv{u}{t_s}
= \frac{8 \pi (k_B T)^{4}}{h^3c^3} \int_0^{\infty } \dd{x} x^3 \pdv{n}{t_s}
= \frac{8 \pi (k_B T)^{4}}{h^3c^3} \int_0^{\infty } \dd{x} x^3 \frac{\Theta}{x^2} \pdv{}{x} \qty[x^{4 } (n' + n + n^2)]
\marginnote{Used the Kompaneets equation.}
\,.
\end{align}

Now, let us suppose that \(n\) is very small, so that we can neglect \(n\) and \(n^2\) inside the derivative, keeping \(n'\): this yields
% \todo[inline]{Not super clear why \(n \ll n'\) if \(n\) is small should hold\dots }
%
\begin{align}
\dv{u}{t_s} \approx \frac{8 \pi (k_B T)^{4}}{h^3 c^3} \Theta \int_0^{\infty } \dd{x} x \pdv{[x^{4} n']}{x}
\,.
\end{align}

We integrate by parts twice and assume that \(x^{k} n' \to 0 \) for \(k> 3\): this give us
%
\begin{align}
\dv{u}{t_s} \approx \frac{8 \pi (k_B T)^{4}}{h^3 c^3} 4 \Theta \int_0^{\infty } \dd{x} x^3 n = 4 \Theta u
\,.
\end{align}

The solution to this is an exponential:
%
\begin{align}
u(t_s) = u(0 ) \exp( 4 \Theta t_s) = u(0 ) \exp( t / t_c )
\,,
\end{align}
%
where \(t_c = 1/ (4 \Theta n \sigma _T c)\) is called the Compton time.
This means that the radiative energy density increases \textbf{exponentially fast}, over a characteristic time of the order \(t_c\).

\todo[inline]{So, to see if I understand correctly: the contributions \(n + n^2\) are relevant as we move closer to equilibrium, while the \(n'\) term dominates the first stage of the evolution; and the exponential describes the first stage of an evolution which is shaped like some kind of sigmoid?}

\subsection{Relativistic Kompaneets equation}

We made the hypotheses that \(\Theta \ll 1\) and \(\epsilon \ll 1\) at the start: the electrons and photons were nonrelativistic.
If we keep all the other hypotheses (isotropy, homogeneity, small fractional energy change and so on) can we generalize the Kompaneets equation to relativistic particles?

We will need to use a relativistic Maxwellian (Jüttner distribution) instead of a Maxwellian, and the Klein-Nishina cross section instead of the Thomson one. This will allow us to deal with generic \(\Theta \) and \(\epsilon \): the result is
%
\begin{align}
\pdv{n}{t} = \frac{1}{\epsilon^2} \pdv{}{\epsilon } \qty[ \alpha (\epsilon , \Theta ) \qty(\Theta \pdv{n}{\epsilon } + n + n^2)]
\,,
\end{align}
%
where \(\alpha \) is a certain known function,
%
\begin{align}
\alpha (\epsilon , \Theta ) = \frac{\epsilon^4}{2 K_2 (1 / \Theta )} \int_0^{\infty } \dd{z} z^2 \alpha_0 (\epsilon z) \exp( - \frac{z + 1/z}{2 \Theta })
\,,
\end{align}
%
where \(\alpha_0\) is a known analytical function, whose expression is omitted here.

The form of the relativistic Kompaneets equation is very similar to the nonrelativistic one.

The shape of \(\alpha \) is quite similar to \(\epsilon^4\), its nonrelativistic counterpart, for low photon energies; while for high photon energies \(\alpha \) is lower than \(\epsilon^4\).

The take-away, here, is that if we were to need the relativistic form of the equation we can find it in the literature and apply it to our problem.

\subsection{The emerging spectrum}

In astrophysical (non-cosmological) settings we will not find infinite homogeneous media; instead, we need to treat clouds of finite size.
However, we cannot simply use the Kompaneets equation for them.

What is the spectrum we expect to observe at infinity?
Let us suppose we have a source inside a spherical cloud of radius \(R\); this source will emit a pulse of radiation at a time \(t\), so that the density at a time \(t_0 \) is
%
\begin{align}
n(\nu , t_0 ) = n_0 (\nu ) \delta (t_0  - t )
\,.
\end{align}

Photons will propagate and scatter in the cloud, until finally they are emitted and some of them can be observed. What will be the spectral distribution of these?

Roughly speaking, we can express the observed number density as
%
\begin{align}
n _{\text{obs}}(\epsilon ) = \sum _{i} p_i n(\epsilon , t_i)
\,,
\end{align}
%
where \(t_i\) are different times after \(t_0 \), \(n(\epsilon, t_i)\) are solutions of the Kompaneets equation at these times, and \(p_i\) are the probabilities that a photon emitted at \(t\) will exit the cloud at \(t_i\).
Properly speaking this should be an integral, but we are just giving the rough idea. We will turn it into an integral later.

The escape probability will satisfy
%
\begin{align}
p(t) \dd{t} = \frac{ \dd{n}}{n}
\,,
\end{align}
%
so that it describes the fraction of photons (\(\dd{n} / n\)) which escape the medium in a time \(\dd{t}\).
The observed intensity will then be
%
\begin{align}
n _{\text{obs}}(\epsilon ) = \int_0^{\infty } p(\tau) n(\epsilon , \tau ) \dd{\tau }
\,.
\end{align}

The typical timescale for a photon to leave the cloud will be given by the mean number of scatterings times the typical time per scattering: if the medium is assumed to be optically thick we get
%
\begin{align}
t _{\text{esc}} \sim N_s t_s \sim \max \frac{\qty(\tau,\tau^2) }{n \sigma _T c} \sim \frac{\tau^2 R}{nR \sigma _T c} \sim \frac{\tau R}{c}
\,,
\end{align}
%
since \(n R \sigma _T = \tau \).
This result makes sense: it is the light travel time \(R / c\) times the optical depth.

The form of \(p(t)\) depends on the shape of the cloud; for example in our spherical cloud we have
%
\begin{align}
p(t) = \frac{1}{\tau } \sum _{n=1}^{\infty } \lambda_n \sin \lambda_n \exp( - \frac{\lambda _n^2}{3} \frac{t}{ t _{\text{esc}}})
\,,
\end{align}
%
where the coefficients \(\lambda _n\) are the solutions of \(\tan \lambda _n  = \lambda _n / (1 - 3 \tau / 2)\).

It is important to apply this procedure instead of blindly plugging the distribution inside the Kompaneets equation: we know that in that case the photons will eventually thermalize, which is not of physical interest in most situations.

\subsubsection{Unsaturated Comptonization and power-law tails}

Let us consider a phenomenologically-motivated modified Kompaneets equation, which accounts for the injection and escape of photons in a finite medium:
%
\begin{align}
\dot{H} - \dot{S} = \frac{\Theta}{x^2} \pdv{[x^{4} (n' + n + n^2)]}{x}
\,,
\end{align}
%
where \(\dot{S} = Q(x)\) quantifies the photons gained per unit time, while \(\dot{H} = n / \max(\tau , \tau^2)\) quantifies the photons lost per unit time.

\todo[inline]{Are the signs of the \(\dot{S}\) and \(\dot{H}\) not the other way around?}

\todo[inline]{How does this work dimensionally? Are \(\dot{H}\) and \(\dot{S}\) not supposed to be inverse times?}

Suppose that the input is \emph{soft}, that is, low-energy.
This means that we will have \(Q(x)\) different from zero only for \(x\) smaller than a certain threshold, \(x< x_s \ll 1\).
We then look at the high-energy tail in emissions, \(x \gg 1\).

We will neglect the \(n^2\) term. The equation  will then read
%
\begin{align}
0 = \frac{\Theta}{x^2} \pdv{[x^{4}(n' + n)]}{x} + \underbrace{Q(x) - \frac{n}{\max (\tau , \tau^2)}}_{\text{vanish in the high \(x\) regime}}
\,,
\end{align}
%
so we are left with \(n' + n = \const\), meaning that \(n \sim \exp(- x)\).
This means that at very high energies the decay of the emission is exponential.

\todo[inline]{Why does the term proportional to \(n\) vanish?}

Now, instead, we look for mid-energy solutions, in the range between the threshold for the emission \(x_s\) and 1. We will suppose that \(x_s \ll x \ll 1\).
We will then neglect the \(n\) and \(n^2\) terms, meaning that we can ignore \(Q(x)\) but not the escape term, so we have
%
\begin{align}
0 = \frac{\Theta}{x^2} \pdv{[x^{4} (n')]}{x} - \frac{n}{\max (\tau , \tau^2)}
\,.
\end{align}

If we take a powerlaw ansatz, \(n \propto x^{m}\), we find that the equation for \(m\) becomes
%
\begin{align}
m(m+3) - \frac{4}{4 \Theta \max (\tau, \tau^2)}= 0
\,,
\end{align}
%
where we recognize the Compton parameter \(4 \Theta \max (\tau , \tau^2) = y\), which quantifies how much the scatterings affect the spectrum.
The second-order equation for \(m\) then becomes
%
\begin{align}
m = \frac{-3 \pm \sqrt{9 + 16 / y^2}}{2}
\,.
\end{align}

If we take the positive solution it will be \(m_+>0\), so the spectrum will increase at high energies, which is unphysical.
So, we consider the negative solution: expanding the square root under the assumption that \(y < 1\)\footnote{As \(a\) and \(b\) get further from one another in magnitude, the approximation \(\sqrt{a + b} = \sqrt{a} + \sqrt{b}\) gets closer to being true. For small \(y\) we have quite a good result, while for \(y\) closer to 1 the exponent is slowly-varying anyway.} we get
%
\begin{align}
m_- \approx -3 - \frac{2}{y}
\,.
\end{align}

This is then a powerlaw spectrum: for any soft photon input, then, we have found a qualitative result. It is a powerlaw for medium photon energies and an exponential for high photon energies.
This then gives a typical signature for the observational spectrum we observe in Compton scattering.

Note that, like in the Planck function, the spectrum of the observed intensity scales like \(I_\nu \sim n \nu^3\), so we expect to see \(I_\nu \sim \nu^{3+m} \sim \nu^{-2/y}\) for medium energies, and \(I_\nu \sim \nu^3 e^{-x}\).

\subsubsection{Dynamical Comptonization}

We have so far discussed the effects of repeated scattering of photons onto electrons which are moving due to thermal motion. This is however not the only reason electrons may move!

An interesting situation is one in which the temperature \(T\) of the electrons is low, but the bulk velocity of the plasma \(\vec{v}\) is high, so that interaction with photons is still capable of causing inverse Compton scattering.

Suppose we have a flow of electrons moving with a uniform \(\vec{v}\), constant for all of them. Then, the Electron Rest Frame is the same for all the electrons.

So, in order to deal with the first scattering we must boost to the ERF, but then we are done: we can stay in that frame for all subsequent scatterings.
The energy the photon will emerge with will be \(\epsilon_2 \approx \gamma^2 \epsilon_1 \) regardless of the number of scatterings.

If, on the other hand, the velocity is not uniform we need to insert a relative \(\gamma \) factor for each new electron.
This effect will build up for each scattering.
So, dynamical Comptonization requires a nonvanishing velocity gradient to work.

\todo[inline]{He writes \(\nabla \cdot \vec{v} \neq 0\), which is not the same\dots why would the \emph{divergence} not vanishing specifically be the condition? what about the curl, or the other components of \(\partial_{i} v_j\) anyway?}

This is similar to \emph{second-order Fermi acceleration.}
\todo[inline]{what is that?}

\end{document}


\section{Time scale}\label{sec:time_scale}

In Sect. \ref{subsec:time_definition} we describe the physical time scale and
the trade time scale. In Sect. \ref{subsec:trade_time} and Sect.
\ref{subsec:physical_time} we define the trade and the physical time scales,
respectively.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Time definition}\label{subsec:time_definition}

Due to the nature of the data, there are several options to define time for
analyzing data. In general, the time series are labeled in calendar time
(hours, minutes, seconds, milliseconds).Moreover, tick-by-tick data available
on financial markets all over the world are time stamped up to the nanosecond
\cite{xetra}, but the order of magnitude of the guaranteed precision is much
larger, usually one second or a few hundreds of milliseconds
\cite{market_digest,empirical_facts}. In several papers are used different time
definitions (calendar time, physical time, event time, trade time, tick time)
\cite{empirical_facts,sampling_returns,market_making}. The spot foreign
exchange market data used in the analysis only has the quotes. In consequence,
we have to infer the trades during the market time. As we have tick-by-tick
resolution, we can use either trade time scale or physical time scale.

The trade time scale increases by one unit each time a transaction happens,
which in our case is every time the quotes change. The advantage of this count
is that limit orders far away in the order book do not increase the time by one
unit. The main outcome of trade time scale is its ``smoothing" of data and the
aggregational normality \cite{empirical_facts}.

The physical time scale is increased by one unit each time a second passes.
This means that computing the responses in this scale involves sampling
\cite{Wang_2016_cross,sampling_returns}, which has to be done carefully when
dealing for example with several stocks with different liquidity. This sampling
is made in the trade signs and in the midpoint prices.

We use these two definitions of time scale to compute the price response
function. Between these two scales, there is not a judgment which is better or
worse.Their use directly depends on the application. Thus, our aim is to
present how the price response function behaves under these two different time
scales.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Trade time scale}\label{subsec:trade_time}

As a first approximation, we use the trade sign classification in trade time
scale proposed in Ref. \cite{Wang_2016_cross} and used in Refs.
\cite{my_paper_response_financial,Wang_2016_avg,Wang_2017,Wang_2018_copulas}
that reads
\begin{equation}\label{eq:trade_signs_trade}
    \varepsilon^{\left(\textrm{t}\right)}\left(t,n\right)=\left\{
    \begin{array}{cc}
    \text{sgn}\left(m\left(t,n\right)-m\left(t,n-1\right)\right),
    & \text{if }\\ m\left(t,n\right) \ne m\left(t,n-1\right)\\
    \varepsilon^{\left(\textrm{t}\right)}\left(t,n-1\right),
    & \text{otherwise}
    \end{array}\right..
\end{equation}
Here, $t$ distinguish each second where one or more trades can happen.
$\varepsilon^{\left(\textrm{t}\right)}\left( t,n \right) = +1$ implies a
trade triggered by a market order to buy, and a value
$\varepsilon^{\left(\textrm{t}\right)}\left( t,n \right) = -1$ indicates a
trade triggered by a market order to sell.

In the second case of Eq. (\ref{eq:trade_signs_trade}), if two consecutive
trades with the same trading direction do not exhaust all the available volume
at the best quote, the trades would have the same price, and they will thus
have the same trade sign.

With this classification we obtain trade signs for every single trade in the
data set. According to Ref. \cite{Wang_2016_cross}, the average accuracy of the
classification is $85\%$ for the trade time scale.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Physical time scale}\label{subsec:physical_time}

We use the trade sign definition in physical time scale proposed in Ref.
\cite{Wang_2016_cross} and used in Refs.
\cite{Wang_2016_avg,Wang_2017}, that depends on the classification in
Eq. (\ref{eq:trade_signs_trade}) and reads
\begin{equation}\label{eq:trade_signs_physical}
    \varepsilon^{\left(\textrm{p}\right)}\left(t\right)=\left\{
    \begin{array}{cc}
    \text{sgn}\left(\sum_{n=1}^{N\left(t\right)}
    \varepsilon^{\left(\textrm{t}\right)} \left(t,n\right)\right),
    & \text{If }N \left(t\right)>0\\
    0, & \text{If }N\left(t\right)=0
    \end{array}\right. ,
\end{equation}
where $N \left(t \right)$ is the number of trades in an interval of one second
following t. Here, $\varepsilon^{\left(\textrm{p}\right)}\left( t \right) = +1$
implies that the imbalance of trades in the second $t$ are triggered by a
market order to buy, and a value
$\varepsilon^{\left(\textrm{p}\right)}\left( t \right) = -1$ indicates an
imbalance of sell market orders. In this definition, there are two ways to
obtain $\varepsilon^{\left(\textrm{p}\right)}\left( t \right) = 0$.  First,
there are no trades in a particular second and thus no trade sign. Second, the
sum of the trade signs in a given second amounts to zero, indicating an exact
balance of buy and sell market orders.

Market orders show opposite trade directions as compared to limit orders
executed simultaneously. An executed sell limit order corresponds to a
buyer-initiated market order. An executed buy limit order corresponds to a
seller-initiated market order. In this case we do not compare every single
trade sign in a second, but the net trade sign obtained for every second with
the definition, see Eq. (\ref{eq:trade_signs_physical}). According to Ref.
\cite{Wang_2016_cross}, this definition has an average accuracy up to $82\%$ in
the physical time scale.


\input assets/cs-pre
\bibliography{assets/references.bib}

\begin{document}
\MYTITLE{CMPSC 383: Multi-Agent and Robotic Systems}{Final Project Progress Report}
\MYHEADERS{}

\section{Introduction}
    \textit{Flocking}, in the context of multi-agent systems, involves the collective behavior of a group of interacting, mobile agents. Flocking patterns are exhibited by such a group when the individual agents attempt to move together with their neighbors. This behavior in multi-agent systems mimics the similar flocking behaviors of groups of animals such as birds or fish. Applications of this type of behavior are diverse: multi-robot teams can use flocking to coordinate their movement towards a goal while maintaining a consistent formation or level of proximity~\cite{saska2014swarms,min2011design}, finding optimized solutions to mathematical problems~\cite{cui2009boid}, and simulating the behavior and appearance of flocks or swarms of animals.% TODO: more examples
    \par Boids particular simulation model for multi-agent flocking behaviour~\cite{hartman2006autonomous}. It was first described by Craig Reynolds in 1987~\cite{reynolds1987flocks}. Boids uses a set of three rules for the manipulation: separation, cohesion, and alignment~\cite{reynolds1987flocks,hartman2006autonomous}. More detail on this model can be found in~\Cref{boids}. This model has been used in a number of computer graphics applications, including motion pictures and video games, as well as in robotics applications~\cite{saska2014swarms,min2011design}.
    \par Our project is an implementation of this Boids simulation. To do so, we used the Haskell programming language, a purely functional language with lazy evaluation. This document describes some particulars of Haskell and how they relate to the task of agent programming, the details of the mechanism of our implementation, and the results we acquired from our implementation.

\section{Method}
\subsection{Language Choice}

We chose Haskell as the implementation language for our Boids simulation.
Haskell is a purely functional language with lazy evaluation~\cite{hudak1994haskell,jones2003haskell}. It has been observed that Haskell supports the rapid prototyping of software systems, allowing working systems to be implemented quickly and with minimal complexity~\cite{hudak1994haskell}. The functional programming paradigm in general has been noted to support modularity and allow problems to be decomposed easily~\cite{hughes1989functional}. These observations, along with our previous experience in the language, influenced our choice of Haskell as a platform for our project.

As our class experience with implementing multi-agent simulations has been solely through the object-oriented programming (OOP) paradigm, our language choice presented a challenge, but also a valuable learning experience. Our
implementation of the Boid agents involves the definition of an abstract data
type, which contains the position, velocity, and neighborhood radius of an
individual boid. In Haskell, this is defined as follows:
\begin{listing}[h!]
\begin{minted}{haskell}
data Boid = Boid { position :: Point
                 , velocity :: Vector
                 , radius   :: Float
                 }
\end{minted}
\end{listing}
\FloatBarrier
Note that Haskell, as a purely functional programming language, does not allow mutation of existing values. Therefore, once a
\mintinline{haskell}|Boid| is created, it cannot be mutated, and updating the \mintinline{haskell}|Boid|'s state
requires the creation of an entirely new \mintinline{haskell}|Boid| instance.

Also, in contrast to a corresponding OOP implementation, which might define some \mintinline{haskell}|Boid|behaviour (i.e. methods) to accompany this basic data structure, this \mintinline{haskell}|Boid| data type is kept distinct from the functions which act upon it.. Instead, we define a type called \mintinline{haskell}|Update|:

\begin{minted}{haskell}
type Update = Boid -> Boid
\end{minted}

Thus, a function of type \mintinline{haskell}|Update| is a function that takes a \mintinline{haskell}|Boid| and returns
a new \mintinline{haskell}|Boid|. We use \mintinline{haskell}|Update| to define \mintinline{haskell}|Behaviour|:

\begin{minted}{haskell}
type Perception = [Boid]
type Behaviour = Perception -> Update
\end{minted}

This defines a \mintinline{haskell}|Behaviour| as a function that maps an agent's perception of its environment (in this case, a \mintinline{haskell}|Boid|'s visible neighborhood) to
a function for updating its environment. These examples demonstrate how implementing a multi-agent
simulation in a purely-functional language requires a different conception of
what it means programmatically for an Agent to behave.

\subsection{Boids}\label{boids}

Boids models the behaviour of a flock in as being effected by three primary steering forces: \textit{cohesion}, the tendency of an individual to stay close to the centre of the flock; \textit{separation}, the tendency of an individual to avoid collision with other individuals, and \textit{alignment}, the tendency of an individual to match velocities with its neighbors~\cite{hartman2006autonomous,reynolds1987flocks}. Each of these steering forces is modeled as a vector, which are then summed to compute the position of a given boid at each time interval.

The separation steering vector $\vec{s}_i$ for a given boid $b_i$ may be calculated as the negative sum of the position vector of $b_i$ and each visible boid $b_j$, using the following formula:

\[ \vec{s}_i = - \sum\limits_{\forall b_j \in V_i} (p_i - p_j) \]

where $V_i$ is the set of boids visible by $b_i$ (i.e. the neighborhood)~\cite{hartman2006autonomous}. In our implementation, this formula corresponds to the following Haskell source code:

\begin{minted}{haskell}
separation :: Boid -> Perception -> Vector
        -- :: Boid -> [Boid] -> V3 Float
separation self neighbors =
    let p = position self
    in sumV . map (^-^ p) $ positions neighbors
\end{minted}

The cohesion steering vector $\vec{k}_i$ for a given boid $b_i$ may be calculated by finding the centre of density $c_i$ of the visible boids $V_i$ using the formula

\[ c_i = \sum\limits_{\forall b_j \in V_i} \frac{p_j}{m} \]

where $m$ is the cardinality of $V_i$. The steering vector may then be calculated by subtracting $b_i$'s position from $c_i$~\cite{hartman2006autonomous}:

\[ \vec{k}_i = c_i - p_i \]

In our implementation, these formulae corresponds to the following Haskell source code:

\begin{minted}{haskell}
centre :: Perception -> Vector
    -- :: [Boid] -> V3 Float
centre boids =
    let m = fromIntegral $ length boids :: Float
    in sumV $ map (^/ m) $ positions boids

cohesion :: Boid -> Perception -> Vector
      -- :: Boid -> [Boid] -> V3 Float
cohesion self neighbors =
    let p = position self
    in centre neighbors - p
\end{minted}

Finally, the alignment steering vector $\vec{m}_i$ for a boid $b_i$ may be calculated by averaging the velocities of the set of visible boids $V_i$ using the following formula

\[ \vec{m}_i = \sum\limits_{\forall b_j \in V_i} \frac{\vec{v}_j}{m} \]

where $\vec{v}_j$ is the velocity of $b_j$. If the cardinality of $V_i$ is zero, then $\vec{v}_i = 0$~\cite{hartman2006autonomous}. In our implementation, this formula corresponds to the following Haskell source code:

\begin{minted}{haskell}
alignment :: Boid -> Perception -> Vector
       -- :: Boid -> [Boid] -> V3 Float
alignment _ []        = V3 0 0 0
alignment _ neighbors =
    let m = fromIntegral $ length neighbors :: Float
    in (sumV $ map velocity neighbors) ^/ m
\end{minted}

Once all three steering vectors have been calculated, they are combined to find the velocity $\vec{v_i\prime}$ of a boid

\[ \vec{v_i\prime} = \vec{v_i} + S.\vec{s_i} + K.\vec{k_i} + M.\vec{m_i} \]

where $S$, $K$, and $M$ are coefficients which control the weight of each steering force and are typically global parameters to the simulation.

The position of that boid at time $t + \Delta t$ maythen  be updated using $\vec{v_i\prime}$

\[ p\prime_i = p_i + \Delta t\vec{v_i}\]

In our implementation, these formulae corresponds to the following Haskell source code:

\begin{minted}{haskell}
steer :: Weights -> Behaviour
   -- :: Weights -> [Boid] -> Boid -> Boid
steer (s, c, m) neighbors self =
    let s_i  = s *^ separation self neighbors
        c_i  = c *^ cohesion self neighbors
        m_i  = m *^ alignment self neighbors
        v'   = velocity self ^+^ s_i ^+^ c_i ^+^ m_i
        p    = position self
        p'   = p ^+^ v'
    in self { position = p', velocity = v'}
\end{minted}

\subsection{Program Usage}

Our implementation supports a number of command-line arguments to set simulation parameters and control the resulting visualization. The following options for controlling the simulation parameters are supported:

\begin{description}
    \item[Height] of the space can be set using the options \texttt{-x HEIGHT} or \texttt{--height=HEIGHT}.
    \item[Width] of the space can be set using the options \texttt{-y WIDTH} or \texttt{--width=WIDTH}.
    \item[Number of Boids] can be  set using the options \texttt{-n BOIDS} or \texttt{--num=BOIDS}.
    \item[Boid visibility radius] can be set using the options \texttt{-v RADIUS} or \texttt{--visibility=RADIUS}. The default is 50.
    \item[Simulation speed] can be set using the options \texttt{-p SPEED} or \texttt{--speed=SPEED}. This value sets a coefficient by which the movement vector of each boid is divided every simulation tick. The default is 1000 at 30 frames per second.
\end{description}

Additionally, our implementation allows the user to select one of a set of preset behavior types. These control the values of the weight constants for the three boid steering forces, as discussed in \Cref{boids}.

\begin{description}
    \item[Equal-Weight Behavior] selected with the command-line flag \texttt{-e} or \texttt{--equal}, sets all weight coefficients to 1.
    \item[Cohesive Behavior] selected with the flag \texttt{-c} or \texttt{--cohesive}, sets weights as follows: $S = 1, K = 0.5, M = 0.5$.
    \item[Swarming Behavior] selected with the flag \texttt{-s} or \texttt{--swarm}, sets the alignment coefficient ($M$) equal to zero, creating true swarming behavior.
\end{description}

Our program may also be run in `debug mode,' using the \texttt{-d} or \texttt{--debug} option. In debug mode, the program will run the visibility radius and velocity vector of each boid, so that the user can gain a greater understanding of how the simulation functions. Finally, the \texttt{-h} or \texttt{--help} flag will print out a summary of the available command-line options.

\section{Results}

    The results of this simulation are visible in the following figures:
    \begin{itemize}
        \item Figure~\ref{fig:no-debug} demonstrates our simulation without our ``debug-mode''. Of course, as a static image without any representation of boid vectors, it is difficult to tell how the swarm is behaving.
        \item Figure~\ref{fig:eqweight} demonstrates our ``debug mode'' with equal weights for cohesion, separation, and alignment. A large degree of agent clustering is visible.
        \item However, with smaller radii as in Figure~\ref{fig:eqweight-small}, we see a significantly decreased level of clustering of the agents.
        \item On the other hand, increasing the radii as in Figure~\ref{fig:eqweight-large}, gives an even greater degree of clustering than before.
        \item With our ``cohesive'' mode, as in Figure~\ref{fig:eqweight-large}, we see that the boids form a much more even distribution throughout the space. This is due the fact that the separation impulse is still present, but does not dominate the vectors of the boids, preventing them from separating a great distance apart.
    \end{itemize}

    \begin{figure}
        \centering
        \includegraphics[width=0.5\textwidth]{assets/img/nodebug}
        \caption{Simulation run in swarm mode, with debugging off.}
        \label{fig:no-debug}
    \end{figure}

    \begin{figure}
        \centering
        \includegraphics[width=0.5\textwidth]{assets/img/eqweight}
        \caption{Simulation run in equal-weight mode, with debugging and radii of size 20.}
        \label{fig:eqweight}
    \end{figure}

    \begin{figure}
        \centering
        \includegraphics[width=0.5\textwidth]{assets/img/eqweight-small}
        \caption{Simulation run in equal-weight mode, with debugging and radii of size 10.}
        \label{fig:eqweight-small}
    \end{figure}

    \begin{figure}
        \centering
        \includegraphics[width=0.5\textwidth]{assets/img/eqweight-large}
        \caption{Simulation run in equal-weight mode, with debugging and radii of size 30.}
        \label{fig:eqweight-large}
    \end{figure}

    \begin{figure}
        \centering
        \includegraphics[width=0.5\textwidth]{assets/img/cohesion}
        \caption{Simulation run in cohesion mode.}
        \label{fig:cohesion}
    \end{figure}
    % describe behavior for each mode
    \FloatBarrier
\section{Conclusion}
    \subsection{Challenges}
    One of our major challenges was the existence of a bug in the ``alignment'' behavior of our code. This bug caused all boids in the scene to speed up erratically, eventually causing overflow in the vector components of the boids' velocities, and causing the program to crash. This was fixed after we realized that the neighborhood selection code for each boid included the boid itself, and thus added on the boid's own velocity to itself in the alignment function.
    \par Another challenge we faced early on was our early ambitions of creating a 3D simulation using the Haskell OpenGL bindings. These bindings caused us a number of problems, and abandoning them in favor of the 2D \texttt{gloss} library allowed us to focus on the details of our implementation with greater detail. Neither of us is particularly experienced with computer graphics, and we found that the higher level of abstraction provided by \texttt{gloss} made our task much easier. Furthermore, using OpenGL directly requires writing code that could be considered fairly unidiomatic in Haskell.

    \subsection{Takeaways}
    Both group members enjoyed the experience of implementing a multi-agent system using a purely functional programming language. As Hawk had significantly less experience programming in Haskell than Will, he found this assignment to be a particularly good learning experience in that language, but both members of the team enjoyed the opportunity to try something different than the previous course assignments. Furthermore, we found experimenting with Haskell's libraries for vector mathematics, as necessitated by the boids algorithm, to be interesting and fun.
    \par We also enjoyed being able to implement swarming and flocking behavior in a multi-agent system. Our previous experiences with swarming in class had only involved using preexisting programs, so we were interested in how such programs were written. Although Boids is not a particularly complex algorithm, the simulations it produces provide a fairly convincing approximation of real-life bird behavior. Both team members are very interested in `artificial life' simulations in general --- for his first final project in a computer science course at Allegheny, Will wrote an implementation of Conway's Game of Life in Processing. As this project is his last final project for a computer science course here at Allegheny, artificial life provides `book ends,' in a sense, on his Allegheny career.

\printbibliography

\end{document}


\documentclass[runningheads]{llncs}

\usepackage{graphicx}
\usepackage{placeins}
\usepackage{hyperref,xcolor}
\renewcommand\UrlFont{\color{blue}\rmfamily}
\usepackage{listings}
\definecolor{very-light-gray}{gray}{0.95}

\begin{document}

\title{Solving Combinatorial Puzzles with Parallel Evolutionary Algorithms \thanks{This work was supported by a private funding of Velbazhd Software LLC.}}
\titlerunning{Solving Puzzles with PEA}

\author{Todor Balabanov\orcidID{0000-0003-3139-069X} \and
Stoyan Ivanov \and
Rumen Ketipov}
\authorrunning{T. Balabanov et al.}

\institute{Institute of Information and Communication Technologies \\
Bulgarian Academy of Sciences \\
acad. Georgi Bonchev Str., Block 2, 1113 Sofia, Bulgaria \\
\email{todorb@iinf.bas.bg} \\
\url{http://iict.bas.bg/}}

\maketitle

\begin{abstract}
Rubik's cube is the most popular combinatorial puzzle. It is well known that solutions of the combinatorial problems are generally hard to find. If 90 degree clockwise rotations of the cube's sides are taken as operations it will give a minimal cube's grammar. By building formal grammar sentences with the usage of the six operations ([L]eft, [R]ight, [T]op, [D]own, [F]ront, [B]ack) all cube's permutations can be achieved. In an evolutionary algorithms (like genetic algorithms for example) set of formal grammar sentences can be represented as population individuals. Single cut point crossover can be efficiently applied when population individuals are strings. Changing randomly selected operation with another randomly selected operation can be used as efficient mutation operator. The most important part of such global optimization is the fitness function. For better individuals fitness value evaluation a combination between Euclidean and Hausdorff distances is proposed in this research. The experiments in this research are done as parallel program written in C++ and Open MPI.

\keywords{Distributed evolutionary algorithms \and Combinatorial puzzles \and Integer optimization.}
\end{abstract}

\section{Introduction}

A parallel implementation of a genetic algorithm-based solver of the Rubik's cube was implemented by Balabanov in \cite{balabanov01} and it was presented in \cite{balabanov02}. Rubik's cube was invented and introduced by Erno Rubik in the 70s of the 20th century. After its creation the cube became the most popular combinatorial puzzle all over the world. In its original version it has 3x3x3 cubical segments. There are stickers in six different colors on each subcube square of the exposed sides. Each of the six planes (3x3x1) can be rotated in 90, 180, 270 or 360 degrees, relative to the other part of the puzzle. In the original initial state all sides of the cube are in single color. Scumbling of the puzzle is done by many random rotations of the (3x3x1) sides. The optimization task aims to restore the cube in its original state. Such combinatorial optimization problem is quite difficult because there are billions of combinations. The real number of combinations is $4.3252*10^{19}$ \cite{korf01} and all of them can be reached from any starting combination. The puzzle is successfully resolved when a sequence of moves is applied such that all subcubes are matched to each other by their color on each side of the cube. According to an estimation in \cite{korf01} resolutions sequences variate from 50 to 100 moves when the cube is well scrambled.

When there is an optimization problem with a sequence of commands it is a perfect candidate for evolutionary algorithms as optimizers. This research addresses the application of parallel genetic algorithms for Rubik's cube optimal or suboptimal solutions findings. The source code of the experiments is written in C++ with OpenMPI for parallel calculations and it can be found in a public source code repository \cite{balabanov01}. Modification of the evaluation function presented in \cite{balabanov02} is upgraded by addition of Hausdorff distance component.

The rest of this paper is organized as follows: Section 2 briefly describes theoretical details. Hence, section 3 presents the proposed modifications into a practical software example and section 4 is devoted to some experiments and results. Finally, Section 5 concludes and presents some ideas for further research.

\section{Parallel Genetic Algorithms}

Genetic algorithms are global optimization strategy inspired from the evolution process in the Biology theories. Place of the genetic algorithms and the genetic programming in the family of the metaheuristics is well presented in Fig.\ref{fig02}.

\begin{figure}
\includegraphics[width=1.0\textwidth,height=0.65\textwidth]{fig02.png}
\centering
\caption{Euler diagram of the different classifications of metaheuristics.} \label{fig02}
\end{figure}
%\FloatBarrier

Solutions of a particular problem are represented as vectors of values into the solution space. All selected solution vectors are the population of the algorithm. The most common way of initial population establishment is by generation of random vectors. Each new generation appears in the population after recombination of selected individuals. In genetic algorithms recombination is done by two consequent operators - crossover and mutation. Which individuals to participate in the recombination process is decided by application of selection operator. It is very common during selection process elitism rule to be applied. Elitism means that some percent of the best found solutions survive to the real end of the optimization process. Because genetic algorithms based optimization is an iterative optimization a stopping criteria is needed. The most used stopping criteria is initially given number of genetic algorithm generations.

Genetic algorithms are the base of the genetic programming which is used in this research. Each element of the solution vector actually is an operation applied over the state of the Rubik's cube. Ordered set of such instructions is actually an algorithmic program. Because there is no direct intermediate relation between the individuals in a particular population genetic algorithms are highly appropriate for implementation in parallel computing or distributed computing. Population of the genetic algorithm can be easily divided in many sub-population and it can be distributed on many processors/cores or even heterogeneous computers in a cluster. Separation of the global population is the preferred approach, but in cases where only the fitness value calculation is time consuming, population is kept in the central processor/computer and only fitness value calculation is sent to the other contributing processors/computers.

When sub-populations distribution calculation scheme is selected some strategy for individual migration should be applied \cite{balabanov02}. Migration between different islands is needed in order best found solutions to be available in some or in all sub-populations. When the implementation of the calculation is organized as donated distributed computing project in the inclusion of a new remote contributing computer fresh subset of the global population can be supplied. With such strategy solutions space is much better investigated.

\section{Modified Rubik's Cube Solver}

The core of the optimization code is Rubik's cube representation into the computer memory. For the needs in this research the cube is presented as six (one for each side) two-dimensional (3x3) arrays. Values in these arrays are integer numbers which correspond to cube's colors. There are better ways for digital representation \cite{korf01}, but it is much more practical in this way from algorithmic aspect.

Data structures are the first side of the modeling proccess. On the second side are the algorithmic operations done over the data structures. The cube has six sides that is why the minimum number of operations over the cube is six. Six capital letters are used for 90 degrees clockwise rotations, as proposed in \cite{randall01,balabanov02}: \\
\\
T (Top) –90 degrees clockwise rotation of the top side; \\
L (Left) –90 degrees clockwise rotation of the left side; \\
B (Back) –90 degrees clockwise rotation of the back side; \\
R (Right) –90 degrees clockwise rotation of the right side; \\
F (Front) –90 degrees clockwise rotation of the front side; \\
D (Down) –90 degrees clockwise rotation of the down side. \\

This set of six operations is the minimal fully functional grammar for the Rubik's cube. Extended grammars are also possible, for example if counter-clockwise operators are included (+T, +L, +B, +R, +F, +D, –T, –L, –B, –R, –F, –D). Next level of extension is addition as number of turns (+1T, +2T, +3T, +1L, +2L, +3L, +1B, +2B, +3B, +1R, +2R, +3R, +1F, +2F, +3F, +1D,+2D, +3D, –1T, –2T, –3T, –1L, –2L, –3L, –1B, –2B, –3B, –1R, –2R, –3R, –1F, –2F, –3F, –1D, –2D, –3D) \cite{balabanov02}.

With the presented ideas for a formal Rubik's cube grammar the neutral choice is genetic algorithm individuals to be represented as formal grammar sentences with variable length. Each of the letters can appear at any position many times repeated in the chromosome. As it was appointed in \cite{korf01} the average expected length of the chromosomes can be between 50 and 100.

\begin{figure}
\includegraphics[width=0.75\textwidth]{fig01.png}
\centering
\caption{Fitness value evaluation by combination between Euclidean and Hausdorff distance.} \label{fig01}
\end{figure}
\FloatBarrier

Single cut point is selected as crossover population, but other options \cite{poli01} are also applicable. As mutation operator random change of a single instruction is selected. Selection is done by randomly selected parents, but elitism rule is applied. For the evaluation of the newly created individuals instructions encoded in the individual are applied over the scrambled cube. After that the state of the cube is compared with the target state (cube in the solved state). The listing in Fig.\ref{fig01} shows the proposed in this paper modification of fitness evaluation function. For each pairs of cube's sides Euclidean distance is calculated. After that according to Hausdorff distance rules the maximum of the minimums is found. Evaluated fitness value is positive because the Euclidean distance is calculated with positive integers (cube's colors are mapped to integers) and the Hausdorff distance is a calculation of a maximum of the minimums. The puzzle is as better solved as the fitness value is smaller.

\section{Experiments and Results}

All experiments were done on a single processor desktop machine - Intel Core i5, 2.3 GHz, 2 Cores, 8GB RAM and Mac OS X 10.13.6, Apple LLVM version 9.1.0. For the parallel implementation Open MPI is used.

\begin{figure}
\includegraphics[width=0.75\textwidth]{fig03.png}
\centering
\caption{Algorihtm convergence with star topology.} \label{fig03}
\end{figure}
\FloatBarrier

Experiments are done in two groups with 30 independent runs for each. The source code originally is divided for test with star topology and incident nodes participation as migration strategies. That is why there are two groups of experiments. Each experiment compares pure Euclidean distance implementation and the proposed Hausdorff distance modification. Parameters of the genetic algorithm are listed in Tab. \ref{tab01}.

\begin{table}
\caption{Genetic algorithm parameters.}
\label{tab01}
\begin{tabular}{p{6.9cm}p{4.4cm}}
\hline\noalign{\smallskip}
\textbf{Parameter} & \textbf{Value} \\
\hline\noalign{\smallskip}
generation gap & 0.93 \\
crossover rate & 0.98 \\
mutation rate & 0.01 \\
maximum generations & 10000 \\
number of individuals & 37 \\
number of variables & floating \\
inserted rate & 100 \% \\
\noalign{\smallskip}\hline\noalign{\smallskip}
\end{tabular}
\end{table}
\FloatBarrier

\begin{figure}
\includegraphics[width=0.75\textwidth]{fig04.png}
\centering
\caption{Algorihtm convergence with incident nodes participation.} \label{fig04}
\end{figure}
\FloatBarrier

Fig.\ref{fig03} shows that when star topology is used the advantage of Hausdorff modification is not so great, but as it is shown in Fig.\ref{fig04} when an incident nodes participation is used the proposed modification leads to convergence seep-up. Calculation of a Euclidean distance between two cubes uses six calculations of the sides for the cube (Listing \ref{lst01}). In the case of Hausdorff distance there are six times more calls of a single Euclidean distance calculation (Listing \ref{lst02}).

\begin{lstlisting}[caption={Euclidean distance.},label={lst01},backgroundcolor=\color{very-light-gray},frame={single}]
double euclidean(const int side1[3][3],
                 const int side2[3][3]) const {
  double distance = 0.0;

  for(int i=0; i<3; i++) {
    for(int j=0; j<3; j++) {
      distance += (side1[i][j]-side2[i][j]) *
                  (side1[i][j]-side2[i][j]);
    }
  }

  return sqrt(distance);
}
\end{lstlisting}
\FloatBarrier

\begin{lstlisting}[caption={Hausdorff distance.},label={lst02},backgroundcolor=\color{very-light-gray},frame={single}]
double hausdorff(const RubiksCube &cube) const {
  double min[] = {INT_MAX, INT_MAX, INT_MAX,
                  INT_MAX, INT_MAX, INT_MAX};

  for(int s1=0; s1<6; s1++) {
    for(int s2=0; s2<6; s2++) {
      double distance =
      euclidean(*sides[s1], *(cube.sides)[s2]);

      if(min[s1] > distance) {
        min[s1] = distance;
      }
    }
  }

  double result = min[0];
  for(int s1=0; s1<6; s1++) {
    if(result < min[s1]) {
      result = min[s1];
    }
  }

  return(result);
}
\end{lstlisting}
\FloatBarrier

\section{Conclusions}

The experiments show that the addition of Hausdorff distance component improves the performance of the genetic algorithm. A calculation of the Hausdorff distance is a little bit slower than the calculation of the Euclidean distance, but better solution fitness estimation generally leads to genetic algorithm convergence improvement. Further investigations in the field of the soft computing can be done for the proposed fitness value evaluation as it was done in \cite{angelova01}.

As further research it will be interesting an evaluation of the fitness value to be additionally filtered with Kalman filter \cite{alexandrov01}, for example. Another interesting direction would be an involvement of the artificial neural networks such as \cite{tashev01,atanasova01} for preliminary solutions evaluation.

\begin{thebibliography}{8}
\bibitem{balabanov01}
MPI Parallel Implementation of Genetic Algorithm Based Rubik’s Cube Solver, \url{http://github.com/TodorBalabanov/RubiksCubeGeneticAlgorithmsSolver}. Last accessed 10 Feb 2019
\bibitem{balabanov02}
Balabanov, T., Zankinski, I., Barova, M.: Strategy for Individuals Distribution by Incident Nodes Participation in Star Topology of Distributed Evolutionary Algorithms. Cybernetics and Information Technologies \textbf{16}(1), 80--88 (2016)
\bibitem{korf01}
Korf, R.: Finding Optimal Solutions to Rubik’s Cube Using Pattern Databases. In: AAAI-98 Proceedings, pp. 700--705. AAAI Press, Menlo Park, CA, USA (1998)
\bibitem{randall01}
Randall, K.: Cilk - Efficient Multithreaded Computing. Doctor of Philosophy Thesis in Computer Science and Engineering, Massachusetts Institute of Technology, USA (1998)
\bibitem{poli01}
Poli, R., Kozak J.: Genetic Programming. In: Burke, EK., Kendall, G. (eds.) Search Methodologies, pp. 143--185. Springer US (2014)
\bibitem{angelova01}
Angelova, V.: Investigations in the Area of Soft Computing Targeted State of the Art Report. Cybernetics and Information Technologies \textbf{9}(1), 18--24 (2009)
\bibitem{alexandrov01}
Alexandrov, A.: AD HOC Kalman filter based fusion algorithm for real-time Wireless Sensor Data Integration. In: FQAS-2015 Proceedings, pp. 151--160. Springer, Heidelberg (2015)
\bibitem{tashev01}
Tashev, T., Hristov, H.: Modeling of synthesis of information processes with generalized nets. Cybernetics and Information Technologies, \textbf{3}(2), 92--104 (2003)
\bibitem{atanasova01}
Atanasova T., Barova M.: Exploratory analysis of Time Series for hypothesize feature values. In: UniTech17 Proceedings,  \textbf{16}(2), 399--403 (2017)
\end{thebibliography}
\end{document}


% This LaTeX was auto-generated from an M-file by MATLAB.
% To make changes, update the M-file and republish this document.

      \subsection{propagated.m}

\begin{par}
\textbf{Summary:} Propagate the state distribution one time step forward            with derivatives
\end{par} \vspace{1em}

\begin{verbatim}function [Mnext, Snext, dMdm, dSdm, dMds, dSds, dMdp, dSdp] = ...
  propagated(m, s, plant, dynmodel, policy)\end{verbatim}
    \begin{par}
\textbf{Input arguments:}
\end{par} \vspace{1em}
\begin{verbatim}m                 mean of the state distribution at time t           [D x 1]
s                 covariance of the state distribution at time t     [D x D]
plant             plant structure
dynmodel          dynamics model structure
policy            policy structure\end{verbatim}
\begin{par}
\textbf{Output arguments:}
\end{par} \vspace{1em}
\begin{verbatim}Mnext             predicted mean at time t+1                         [E x 1]
Snext             predicted covariance at time t+1                   [E x E]
dMdm              output mean wrt input mean                         [E x D]
dMds              output mean wrt input covariance matrix         [E  x D*D]
dSdm              output covariance matrix wrt input mean        [E*E x  D ]
dSds              output cov wrt input cov                       [E*E x D*D]
dMdp              output mean wrt policy parameters                  [E x P]
dSdp              output covariance matrix wrt policy parameters  [E*E x  P]\end{verbatim}
\begin{verbatim}where P is the number of policy parameters.\end{verbatim}
\begin{par}
Copyright (C) 2008-2013 by Marc Deisenroth, Andrew McHutchon, Joe Hall, Henrik Ohlsson, and Carl Edward Rasmussen.
\end{par} \vspace{1em}
\begin{par}
Last modified: 2013-01-23
\end{par} \vspace{1em}

\subsection*{High-Level Steps}

\begin{enumerate}
\setlength{\itemsep}{-1ex}
   \item Augment state distribution with trigonometric functions
   \item Compute distribution of the control signal
   \item Compute dynamics-GP prediction
   \item Compute distribution of the next state
\end{enumerate}

\begin{lstlisting}
function [Mnext, Snext, dMdm, dSdm, dMds, dSds, dMdp, dSdp] = ...
  propagated(m, s, plant, dynmodel, policy)
\end{lstlisting}

\subsection*{Code}

\begin{lstlisting}
if nargout <= 2                                  % just predict, no derivatives
  [Mnext, Snext] = propagate(m, s, plant, dynmodel, policy);
  return
end

angi = plant.angi; poli = plant.poli; dyni = plant.dyni; difi = plant.difi;

D0 = length(m);                                        % size of the input mean
D1 = D0 + 2*length(angi);          % length after mapping all angles to sin/cos
D2 = D1 + length(policy.maxU);          % length after computing control signal
D3 = D2 + D0;                                         % length after predicting
M = zeros(D3,1); M(1:D0) = m; S = zeros(D3); S(1:D0,1:D0) = s;   % init M and S

Mdm = [eye(D0); zeros(D3-D0,D0)]; Sdm = zeros(D3*D3,D0);
Mds = zeros(D3,D0*D0); Sds = kron(Mdm,Mdm);
X = reshape(1:D3*D3,[D3 D3]); XT = X'; Sds = (Sds + Sds(XT(:),:))/2;
X = reshape(1:D0*D0,[D0 D0]); XT = X'; Sds = (Sds + Sds(:,XT(:)))/2;

% 1) Augment state distribution with trigonometric functions ------------------
i = 1:D0; j = 1:D0; k = D0+1:D1;
[M(k) S(k,k) C mdm sdm Cdm mds sds Cds] = gTrig(M(i), S(i,i), angi);

[S Mdm Mds Sdm Sds] = ...
  fillIn(S,C,mdm,sdm,Cdm,mds,sds,Cds,Mdm,Sdm,Mds,Sds,[ ],[ ],[ ],i,j,k,D3);

mm=zeros(D1,1); mm(i)=M(i); ss(i,i)=S(i,i)+diag(exp(2*dynmodel.hyp(end,:))/2);
[mm(k), ss(k,k) C] = gTrig(mm(i), ss(i,i), angi);     % noisy state measurement
q = ss(j,i)*C; ss(j,k) = q; ss(k,j) = q';

% 2) Compute distribution of the control signal -------------------------------
i = poli; j = 1:D1; k = D1+1:D2;
[M(k) S(k,k) C mdm sdm Cdm mds sds Cds Mdp Sdp Cdp] = ...
  policy.fcn(policy, mm(i), ss(i,i));

[S Mdm Mds Sdm Sds Mdp Sdp] = ...
  fillIn(S,C,mdm,sdm,Cdm,mds,sds,Cds,Mdm,Sdm,Mds,Sds,Mdp,Sdp,Cdp,i,j,k,D3);

% 3) Compute distribution of the change in state ------------------------------
ii = [dyni D1+1:D2]; j = 1:D2;
if isfield(dynmodel,'sub'), Nf = length(dynmodel.sub); else Nf = 1; end
for n=1:Nf                               % potentially multiple dynamics models
  [dyn i k] = sliceModel(dynmodel,n,ii,D1,D2,D3); j = setdiff(j,k);

  [M(k) S(k,k) C mdm sdm Cdm mds sds Cds] = dyn.fcn(dyn, M(i), S(i,i));

  [S Mdm Mds Sdm Sds Mdp Sdp] = ...
    fillIn(S,C,mdm,sdm,Cdm,mds,sds,Cds,Mdm,Sdm,Mds,Sds,Mdp,Sdp,[ ],i,j,k,D3);

  j = [j k];                                   % update 'previous' state vector
end

% 4) Compute distribution of the next state -----------------------------------
P = [zeros(D0,D2) eye(D0)]; P(difi,difi) = eye(length(difi));  P = sparse( P);
Mnext = P*M; Snext = P*S*P'; Snext = (Snext+Snext')/2;

PP = kron(P,P);
dMdm =  P*Mdm; dMds =  P*Mds; dMdp =  P*Mdp;
dSdm = PP*Sdm; dSds = PP*Sds; dSdp = PP*Sdp;

X = reshape(1:D0*D0,[D0 D0]); XT = X';                          % symmetrize dS
dSdm = (dSdm + dSdm(XT(:),:))/2; dMds = (dMds + dMds(:,XT(:)))/2;
dSds = (dSds + dSds(XT(:),:))/2; dSds = (dSds + dSds(:,XT(:)))/2;
dSdp = (dSdp + dSdp(XT(:),:))/2;

% A1) Separate multiple dynamics models ---------------------------------------
\end{lstlisting}

\begin{lstlisting}
function [dyn i k] = sliceModel(dynmodel,n,ii,D1,D2,D3) % separate sub-dynamics
if isfield(dynmodel,'sub')
  dyn = dynmodel.sub{n}; do = dyn.dyno; D = length(ii)+D1-D2;
  if isfield(dyn,'dyni'), di=dyn.dyni; else di=[]; end
  if isfield(dyn,'dynu'), du=dyn.dynu; else du=[]; end
  if isfield(dyn,'dynj'), dj=dyn.dynj; else dj=[]; end
  i = [ii(di) D1+du D2+dj]; k = D2+do;
  dyn.inputs = [dynmodel.inputs(:,[di D+du]) dynmodel.target(:,dj)];   % inputs
  dyn.target = dynmodel.target(:,do);                                 % targets
else
  dyn = dynmodel; k = D2+1:D3; i = ii;
end

% A2) Apply chain rule and fill out cross covariance terms --------------------
function [S Mdm Mds Sdm Sds Mdp Sdp] = ...
  fillIn(S,C,mdm,sdm,Cdm,mds,sds,Cds,Mdm,Sdm,Mds,Sds,Mdp,Sdp,dCdp,i,j,k,D)

if isempty(k), return; end

X = reshape(1:D*D,[D D]); XT = X';                         % vectorized indices
I=0*X; I(i,i)=1; ii=X(I==1)'; I=0*X; I(k,k)=1; kk=X(I==1)';
I=0*X; I(j,i)=1; ji=X(I==1)'; I=0*X; I(j,k)=1; jk=X(I==1)'; kj=XT(I==1)';

Mdm(k,:)  = mdm*Mdm(i,:) + mds*Sdm(ii,:);                           % chainrule
Mds(k,:)  = mdm*Mds(i,:) + mds*Sds(ii,:);
Sdm(kk,:) = sdm*Mdm(i,:) + sds*Sdm(ii,:);
Sds(kk,:) = sdm*Mds(i,:) + sds*Sds(ii,:);
dCdm      = Cdm*Mdm(i,:) + Cds*Sdm(ii,:);
dCds      = Cdm*Mds(i,:) + Cds*Sds(ii,:);
if isempty(dCdp) && nargout > 5
  Mdp(k,:)  = mdm*Mdp(i,:) + mds*Sdp(ii,:);
  Sdp(kk,:) = sdm*Mdp(i,:) + sds*Sdp(ii,:);
  dCdp      = Cdm*Mdp(i,:) + Cds*Sdp(ii,:);
elseif nargout > 5
  aa = length(k); bb = aa^2; cc = numel(C);
  mdp = zeros(D,size(Mdp,2)); sdp = zeros(D*D,size(Mdp,2));
  mdp(k,:)  = reshape(Mdp,aa,[]); Mdp = mdp;
  sdp(kk,:) = reshape(Sdp,bb,[]); Sdp = sdp;
  Cdp       = reshape(dCdp,cc,[]); dCdp = Cdp;
end

q = S(j,i)*C; S(j,k) = q; S(k,j) = q';                           % off-diagonal
SS = kron(eye(length(k)),S(j,i)); CC = kron(C',eye(length(j)));
Sdm(jk,:) = SS*dCdm + CC*Sdm(ji,:); Sdm(kj,:) = Sdm(jk,:);
Sds(jk,:) = SS*dCds + CC*Sds(ji,:); Sds(kj,:) = Sds(jk,:);
if nargout > 5
  Sdp(jk,:) = SS*dCdp + CC*Sdp(ji,:); Sdp(kj,:) = Sdp(jk,:);
end
\end{lstlisting}


\section{Learning to Optimally Segment Point Clouds}\label{header-n863}

\emph{IEEE ROBOTICS AND AUTOMATION LETTERS, VOL. 5, NO. 2, APRIL 2020}
{[}28{]}

\subsection{Introduction}\label{header-n865}

This letter focuses on the application of autonomous vehicles.
Perception for autonomous robots presents a series of challenges. First,
the right representation of the 3D data obtained by a LiDAR sensor still
remains an open question. Secondly, contemporary approaches to object
detection and scene understanding tend to be closed-world, where the
task is predicting 1-of-N possible labels. Finally, practical autonomous
robotics makes heavy use of perceptual priors in the forms of geometric
maps and assumptions on LiDAR geometry. In this work, the authors focus
on the problem of class-agnostic instance segmentation of LiDAR point
clouds in an open-world setting.

\begin{figure}[h!]
\centering
\includegraphics[width=0.7\linewidth]{images/pointcloudseg.png}
\caption{The class-agnostic instance-level segmentation over all foreground points performed by the proposed method }
\end{figure}

They carefully mix graph-theoretic algorithms with data-driven learning.
While data-driven learning is widely used due to its performance, it is
difficult to guarantee good results when processing out-of-sample data
from an open world. The proposed method searches over an exponentially
large space of possible segmentations and returns the one most similar
to the data-driven point-based model of ``objectness". First, the search
is restricted into a subset of segmentations that are consistent with a
hierarchical grouping of a point cloud sweep. Such hierarchical groups
can be readily produced with agglomerative clustering or hierarchical
graph-based algorithms. Since that a segmentation algorithm can produce
an exponentially-large set of segmentations, the authors introduce
efficient algorithms that search over a space of tree-consistent
segmentations and return the one that maximizes a global segmentation
score.

\subsection{Proposed approach}\label{header-n869}

For 3D object point segmentation, the input is a 3D point cloud, which
contains an \emph{unknown} number of objects. The goal is to produce a
point segmentation, in which every segment contains points from one and
only one object.

\subsubsection{Definitions}\label{header-n871}

A \emph{global segmentation} $P_X$ is a partition of a set of points
$X$ into subsets of points $C_i$ (called local segment), i.e
$P_X = \{C_i\}_{i = 1}^{M}$, where $M$ denotes the number of
segments and $C_i \subset X$. Importantly, every point exists in one
and only one segment.

The concept of \emph{tree-consistent segmentation} is defined as
follows. The set of all possible global segmentations on $X$ is
defined as $S_X$. Without constraints, the size of $S_X$ is
exponential but in practice the authors reduce the number of candidates
by enforcing geometric constraints. All points are grouped
hierarchically into a tree structure $T_X$. Now, the tree-consistent
segmentation is $S_{X,T}$ and contains all possible segmentation
defined by the tree $T$. The following figure illustrates the
relationship between $S_X$ and $S_{X,T}$.

\begin{figure}[h!]
\centering
\includegraphics[width=0.7\linewidth]{images/treesegment.png}
\caption{Tree-consistent segmentation}
\end{figure}

Any tree-consistent segmentation from corresponds to a vertex cut set of
the tree $T$, i.e. a set of tree nodes, which satisfy the following
constraints:

\begin{itemize}
\item
  for each node in the vertex cut, its ancestor and itself cannot both
  be in the cut and
\item
  each leaf node must have itself or its ancestor in the cut.
\end{itemize}

The \emph{segment score} is defined as a function $f(C, \theta)$,
where $C$ is the segment and $\theta$ are the function parameters.
$f$ predicts a given segment's ``objectness" and can be implemented by
a PointNet++, where $\theta$ are the network's weights.

The \emph{segmentation score} is calculated as the aggregation of the
score of all segments in a global segmentation $P_X$. Formally, given
a global segmentation $P_X = \{Ci\}^{M}_{i=1}$, its score is
$F(P_X; \theta) : PX→ [0, 1]$ by aggregating over local objectness of
all its segments.

\subsubsection{Worst-case segmentation}\label{header-n884}

The \emph{worst-case segmentation} score of a global segmentation is the
worst objectness among its local segments:

$ F_{\min }(P_X; \theta) = \min _i{f(C_i; \theta)}, i\in {1 \ldots M}.$

Now, the optimal worst-case segmentation is

$ P_{X,\min }^* = \mathop{\operatorname{argmax}}\limits_{P_X\in S_{X,T}} {F_{\min } (P_X; \theta)} $.

It turns out the problem of finding optimal worst-case segmentation has
optimal substructure, allowing us to find the global optimum efficiently
with dynamic programming (see the following figure).

\begin{figure}[h!]
\centering
\includegraphics[width=0.8\linewidth]{images/worstcaseseg.png}
\caption{Optimal worst case segmentation}
\end{figure}

The algorithm starts from the root node $X$ and chooses between a
coarse segmentation and a fine one. The fine segmentation will be the
union of all $X$`s children's optimal worst-case segmentation, which
can be computed recursively. The algorithm would first traverse down to
the leaf nodes, representing the finest segmentation. Then it will make
its way up, during which it finalizes optimal segmentation for each
intermediate node by making local coarse vs. fine decisions. Eventually,
it returns to the root node and produces an optimal worst-case global
segmentation. This algorithm might not visit all nodes. Instead, it
skips a sub-trees whenever one sub-tree exhibits a lower score. The
algorithm's complexity is linear in $N$ despite the fact that the
search space is exponential in $N$.

\subsubsection{Average-case segmentation}\label{header-n892}

Average-case segmentation score of a global segmentation is the average
objectness among its local segments:
\newline
$ F_{\operatorname{avg}{}}(P_X; \theta) = \frac{1}{M} \sum _{i=1}^M f(C_i; \theta).$
\newline
$P_{X,\operatorname{avg}{}}^*$ can be defined as \emph{optimal
average-case segmentation} if
\newline
$ P_{X,\operatorname{avg}{}}^* = \mathop{\operatorname{argmax}}\limits_{P_X\in S_{X,T}} {F_{\operatorname{avg}{}} (P_X; \theta)}. $
\newline
It is important to specify that the problem of finding the optimal
average-case segmentation does not have optimal substructure, unlike
worst-case segmentation.

\subsubsection{Learning the Objectness Function}\label{header-n898}

Until now, the segmentation algorithms have been discussed under the
assumption that the objectness function $f(C; \theta)$ is defined (it
predicts an objectness score for a given point cloud). First of all, the
ground-truth function must be defined. Suppose to have the ground truth
segmentation $P^{gt} = \{C_1^{gt},...,C_L^{gt}\}$. To define the
target objectness of a segment, we have to consider that 3D sensors
(e.g. LiDAR) tend to produce denser points near the sensor. In
consequence, the objectness will be heavily influenced by the
partitioning of points closer to the sensor. A segment objectness
function of a segment is the largest point between itself and any ground
truth segment:
\newline
$ Objectness(C, P^{gt}) = \max _{l=1,\ldots,L} \frac{\sum _{x \in C \cap C^{gt}_l} x^T x }{\sum _{x \in C \cup C^{gt}_l} x^T x }, $
\newline
where $x^T x$ represents the squared distance of a point $x$ to
sensor origin. The authors train a PointNet++ model for learning the
objectness function. Each segment is re-sampled to 1024 points.

\subsubsection{Building tree hierarchies}\label{header-n902}

This section explains how to build a tree structure given a set of
points $X$. One natural approach is agglomerative clustering, merging
points in classes according to their spatial features. This approach
tends to create tree hierarchies with very fine granularity, e.g. one
node may differ from another with only one point of difference. In this
work, the authors propose a method which builds a coarser tree whose
leaf nodes are segments (rather than individual points) and adjacent
nodes should differ from each other much more. To build such a tree,
Euclidean Clustering algorithm is used recursively in a top-down fashion
with a list of decreasing $\epsilon$. In the beginning, the algorithm
is run with the largest $\epsilon$ value, defining the most coarse
connected components. Then, the procedure is repeated a smaller
$\epsilon$ within each connected component. This produces a
multiple-tree top-down hierarchy.

\subsection{Experiments}\label{header-n904}

For evaluation, the authors repurpose the KITTI object detection
benchmark for point cloud segmentation, following the setup in {[}30{]}.
The ground truth segmentation is composed of a series of point segments
marked with a bounding box. All the background points (those outside the
bounding box) are removed. To evaluate the performance are considered
two metrics: the \emph{under-segmentation error} $U$ and the
\emph{over-segmentation error} $O$, defined as follows:
\newline
$ U = \frac{1}{L} \sum _{l=1}^L \mathbf {1} \left(\frac{|C_{i^*} \cap C_l^{gt}|}{|C_{i^*}|} < \tau _U\right) \\ O = \frac{1}{L} \sum _{l=1}^L \mathbf {1} \left(\frac{|C_{i^*} \cap C_l^{gt}|}{|C_l^{gt}|} < \tau _O\right)$
\newline
with
\newline
$ i^* = \operatorname{argmax}_{i=1}^M{|C_i \cap C^{gt}_l|}$
\newline
and $\tau _U, \tau _O$ as constant thresholds, set to
$\frac{2}{3}, 1$.
\newline
The Euclidean clustering algorithm is applied with an
$\epsilon \in \{2 m, 1 m, 0.5 m, 0.25 m\}$. The authors include it as
a baseline to find better solutions. As baselines, other
state-of-the-art 3D detectors are included: AVOD, PointPillars,
PointRCNN, and SECOND. Since these detectors output class-specific
bounding box detection, the authors simply ignore the class label to
produce class-agnostic segmentations. In addition, the authors include
for evaluation modified versions of the proposed baselines, in order to
improve their performance. A much better approach, called
\emph{\{Detector\}++} (e.g. AVOD++ etc.), check if the leftover segments
can be included in an existing detection segment. Another approach,
SECOND++, consists of re-train and re-evaluate the best baseline, i.e.
SECOND, with background removal. These baselines are marked with ``+ BG
Removal". In addition, the authors discover that, by extending the
SECOND's detection range from 50 m to 80 m, the SECOND's performance is
significantly improved. The affected baselines are marked with ``+ Ext.
Range". Finally, they re-train and re-evaluate SECOND on all 8 classes.
The new baselines are labeled as ``SECOND++(8)", while the off-the-shelf
SECOND baselines are labeled as ``SECOND++(4)". The following figure
resumes the results obtained.

\begin{figure}[h!]
\centering
\includegraphics[width=1\linewidth]{images/segmentationerrors.png}
\caption{Segmentation errors on the proposed method (*Ours*) and the baselines}
\end{figure}

The table shows that the average-case segmentation, \emph{Ours(avg)},
consistently outperforms the optimal worst-case segmentation
\emph{Ours(min)}. \emph{Ours(min)} produces a much lower
over-segmentation error but a much higher under-segmentation error,
suggesting it makes more mistakes of grouping different objects into one
segment and fewer mistakes of splitting points from one single object
into multiple segments. The authors label Euclidean Clustering as
``EC()," where $\epsilon$ represents the distance threshold (meter) and
constructs a pool of segments that contains every node (segment) in the
hierarchy and call this ``EC(all)*". This serves as an unreachable
upper-bound. The gap between our proposed method and the upper bound is
relatively small (3--4\%), suggesting plenty of room left for
improvement in creating better hierarchies. The confront between a
\emph{Detector} algorithm and its variation \emph{Detector++}, shows
that the improved version obtains better performance (see the AVOD's
case). SECOND++ performs the best among all Detector++ baselines.
SECOND++ performs better on common classes such as cars while the
proposed method perform better on rare ones such as misc. Another
evaluation metric is measure how objectness generalizes over different
classes. The authors apply the learned objectness onto truth segments
from the validation set. The graph below shows the results.

\begin{figure}[h!]
\centering
\includegraphics[width=0.8\linewidth]{images/objectnessgen.png}
\caption{Learned objectness generalization}
\end{figure}

As the number of training data decreases dramatically, the average score
tends to drops slightly and the variance tends to rise slightly. In
conclusion, the authors prove that their algorithm is guaranteed to
achieve optimality to a specific definition. On KITTI, the proposed
approach significantly outperforms past bottom-up approaches and
top-down object-based algorithms for segmenting point clouds.


\documentclass[a4paper,11pt]{article}
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage[usenames, dvipsnames]{color}

\begin{document}

\begin{flushright}

\vspace{1.1cm}

{\bf\Huge Problem Set 10}

\rule{0.25\linewidth}{0.5pt}

\vspace{0.5cm}
%Put Authors
Justin Ely
\linebreak
\newline
%Put Author's affiliations
\footnotesize{605.411 Foundations of Computer Architecture \\}
\vspace{0.5cm}
% Date here below
15 November, 2016
\end{flushright}

\noindent\rule{\linewidth}{1.0pt}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{1a)}
\begin{itemize}
  \item A: $time = 5s + \frac{3145728 \, bytes \times s^{-1}}{5120s} = 619.4s$
  \item B: $time = 4s + \frac{3145728 \, bytes \times s^{-1}}{3072s} = 1028s$
\end{itemize}

\section*{1b)}
\begin{eqnarray}
5 + \frac{N}{5120} &=& 4 + \frac{N}{3072} \\
\frac{N}{5120} &=& -1 + \frac{N}{3072} \\
N &=& -5120 + \frac{5N}{3} \\
 \frac{-2N}{3}  &=& -5120 \\
 N  &=& 7680 \, bytes
\end{eqnarray}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{2a)}
With a device generating 150 requests per second, and each request requiring 12,000 cycles to handle, the total number of
cycles per second spent handling interrupts and processing requests will be: \\

$150 \, \frac{requests}{second} \times 12,000 \, cycles = 1,800,000 \frac{cycles}{second}$.

\section*{2b)}
For just a polling scenario, the CPU will use 500 cycles once every .5ms.  \\

$ \frac{1 \, poll}{5e^{-4} \, seconds} \times 500 \, cycles= 1,000,000 \frac{cycles}{second}$

\section*{2c)}

$ \frac{1 \, poll}{N \, seconds} \times (500 + 12000)\, cycles = 1800000 $, thus N = .0069s or 6.9ms.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{3a)}
Memory-mapping implements I/O by allowing peripheral device access through the same address bus as normal memory.  This has the benefit of not needing special I/O commands; rather the same load instruction can be used to access system memory or an external device, at the cost of decreasing the amount of addresses reserved for memory.  Consequently, CPUs are allowed to remain simpler, cheaper, smaller, and faster without the overhead of the extra logic.

\section*{3b)}
DMA is an improvement over CPU programmed I/O because it requires fewer active CPU cycles to complete memory access.  CPU Programmed I/O requires the CPU to be active during the entire read or write sequence.  With DMA, a device is allowed to directly access memory and bypass the need for the CPU to act as middleman.  In this scheme the CPU is only need at the beginning and end of an access, the rest of the time it is free to perform other functions.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\end{document}


\subsection{Functional programming}

With functional programming, we write functions to be called. These functions should not depend on the state, outside of local variables.