{ % localize the newcommands
\newcommand{\Tlabin}{T_{i,\textrm{lab}}}
\newcommand{\plabin}{p_{i,\textrm{lab}}}
\newcommand{\plaba}{p_{\textrm{lab},1}}
\newcommand{\plabb}{p_{\textrm{lab},2}}
\newcommand{\plabc}{p_{\textrm{lab},3}}
\newcommand{\Tlabe}{T_{e,\textrm{lab}}}
\newcommand{\plabe}{p_{e,\textrm{lab}}}
\newcommand{\plabea}{p_{e,\textrm{lab},1}}
\newcommand{\plabeb}{p_{e,\textrm{lab},2}}
\newcommand{\plabec}{p_{e,\textrm{lab},3}}
\newcommand{\Tcmi}{T_{i,\textrm{cm}}}
\newcommand{\Tcmt}{T_{t,\textrm{cm}}}
\newcommand{\Tcme}{T_{e,\textrm{cm}}}
\newcommand{\Tcmr}{T_{r,\textrm{cm}}}
\newcommand{\pcmi}{p_{i,\textrm{cm}}}
\newcommand{\pcma}{p_{\textrm{cm},1}}
\newcommand{\pcmb}{p_{\textrm{cm},2}}
\newcommand{\pcmc}{p_{\textrm{cm},3}}
\newcommand{\pcme}{p_{e,\textrm{cm}}}
\newcommand{\pcmea}{p_{e,\textrm{cm},1}}
\newcommand{\pcmeb}{p_{e,\textrm{cm},2}}
\newcommand{\pcmec}{p_{e,\textrm{cm},3}}
\newcommand{\pcmta}{p_{t,\textrm{cm},1}}
\newcommand{\pcmia}{p_{i,\textrm{cm},1}}
%\newcommand{\mucm}{\mu_{\textrm{cm}}}
%\newcommand{\mulab}{\mu_{\textrm{lab}}}

\chapter{Relativistic 2-body problems}
\label{Appendix-relativity}
In this appendix, relativistic 2-body mechanics is examined from the point
of view of computational physics.  That is, the subtraction
nearly equal numbers is avoided as much as is possible.
The analysis starts with a collision of an incident particle with
a stationary target.  This determines the mapping between
the laboratory frame and the center-of-mass frame.
The appendix closes with a discussion of
emission after the reaction.

As is customary in discussions of relativity, the units are
such that the speed of light has the value~$c = 1$.

\section{Initial collision}
For this appendix, $E$ is the total energy of
a system
and $p$ its total momentum.  Thus, for a particle with rest mass~$m_0$
and kinetic energy~$T$, it follows that $E = m_0 + T$.
\textit{The convention $c = 1$ implies that the data must be such that
particle rest masses and kinetic energies must be given in the same units.}
The analysis makes repeated use of the invariance under Lorentz
transformations of the quantity
\begin{equation}
  S_0 = E^2 - p^2.
  \label{spacetime}
\end{equation}

If the system is a single particle
in a frame in which the particle is stationary, then $S_0 = m_0^2$.
Consequently, for a single particle in
any frame Eq.~(\ref{spacetime}) takes the form
\begin{equation}
  m_0^2 = (m_0 + T)^2 - p^2,
 \label{spacetime-1}
\end{equation}
or
\begin{equation}
  p^2  = 2m_0 T + T^2.
  \label{one-p}
\end{equation}
When it is desired to solve Eq.~(\ref{one-p}) for~$T$ corresponding
to a known value of~$p^2$, it is recommended to use the formula
\begin{equation}
  T = \frac{p^2 }
          {m_0 + \sqrt{ m_0^2 + p^2 }}.
 \label{good-T}
\end{equation}
The relation Eq.~(\ref{good-T}) is computationally more reliable
than the more obvious solution of the quadratic equation Eq.~(\ref{one-p})
$$
  T = -m_0 + \sqrt{ m_0^2 + p^2 }.
$$

Consider the application of Eq.~(\ref{spacetime}) to the system
consisting of a moving incident particle and a target
at rest in
the laboratory frame.  Suppose that the incident
particle has rest mass $\myi$ and kinetic
energy $\Tlabin$, and let $\mtarg$ be the rest mass of the
target.  Then, for a target at rest it follows from Eq.~(\ref{one-p})
that the initial laboratory-frame momentum is given by
\begin{equation}
  \plabin^2  = 2\myi \Tlabin + \Tlabin^2.
 \label{p-lab-in}
\end{equation}
Consequently, for the system of consisting of a particle
incident on a stationary target
in the laboratory frame, the energy-momentum invariant is
$$
  S = (\mtarg + \myi + \Tlabin)^2 -
      ( 2\myi \Tlabin + \Tlabin^2 ),
$$
This expression simplifies to
\begin{equation}
  S = (\myi + \mtarg )^2 + 2\mtarg \Tlabin.
  \label{S-lab}
\end{equation}

The value of $S$ must be the same when this
system of two particles is considered in the center-of-mass
frame.  Denote the center-of-mass kinetic energy of
the incident particle by $\Tcmi$ and its momentum by~$\pcmi$.
Similarly, let the target have center-of-mass kinetic energy $\Tcmt$,
and its momentum is~$-\pcmi$.  The energy-momentum invariant
for the system is therefore
\begin{equation}
  S = (\myi + \Tcmi + \mtarg + \Tcmt)^2,
 \label{spacetime-cm}
\end{equation}
the square of the total energy of the system in the center-of-mass frame.
By using Eq.~(\ref{spacetime-1}) on each of the particles, it
is possible to rewrite this as
$$
  S = \left(
    \sqrt{\myi^2 + \pcmi^2 } +
       \sqrt{\mtarg^2 + \pcmi^2 }
   \right)^2.
$$
Upon solving this equation for $\pcmi^2$, it is found that
\begin{equation}
  \pcmi^2 = \frac{ \left[S - (\myi^2 + \mtarg^2)\right]^2 -
                 4 \myi^2 \mtarg^2 }
                {4 S }.
  \label{p-cm}
\end{equation}
An expression for $\pcmi^2$ in terms of the
laboratory incident kinetic energy $\Tlabin$ is obtained by substituting
in Eq.~(\ref{p-cm}) the value of $S$ given by Eq.~(\ref{S-lab}),
\begin{equation}
  \pcmi^2 = \frac{\mtarg^2( 2 \myi \Tlabin + \Tlabin^2)}
              {(\mtarg + \myi )^2 + 2\mtarg \Tlabin}.
  \label{p-cm-1}
\end{equation}
It follows from Eq.~(\ref{one-p}) that this equation may also
be written as
$$
  \pcmi^2 = \frac{\mtarg^2 \plabin^2 }
                {(\mtarg + \myi )^2 + 2\mtarg \Tlabin}.
$$

\section{Mapping between frames}
Consider a coordinate system in which the momentum
$\plabin$ of the incident particle is in the direction of
the first spatial axis.  The boost of the energy-momentum
vector from
the laboratory to the center-of-mass frame then takes the
form
\begin{equation}
  \begin{bmatrix}
    E_{\text{cm}} \\
    \pcma \\
    \pcmb \\
    \pcmc
  \end{bmatrix}
   =
    \begin{bmatrix}
     \cosh \chi & -\sinh \chi & 0 & 0 \\
     -\sinh \chi & \cosh \chi & 0 & 0 \\
     0  & 0 & 1 & 0 \\
     0  & 0 & 0 & 1
  \end{bmatrix}
  \begin{bmatrix}
    E_{\text{lab}} \\
    \plaba \\
    \plabb \\
    \plabc
  \end{bmatrix}.
  \label{R-3}
\end{equation}

Upon applying the second component of the boost Eq.~(\ref{R-3}) to the
target,
it is found that
$$
  \pcmta = - \mtarg  \sinh \chi.
$$
With the notation that $ | p | $
is the length of the vector~$p$,
it follows that
$$
   \pcmta =  -\pcmia = - | \pcmi |,
$$
so that
\begin{equation}
   \sinh \chi = \frac{|\pcmi|}{\mtarg}.
 \label{lab-to-cm}
\end{equation}
By using Eq.~(\ref{p-cm-1}), one may conclude that
\begin{equation}
  \sinh \chi = \frac{\sqrt{ 2 \myi \Tlabin + \Tlabin^2}}
    {\sqrt{( \mtarg + \myi )^2 + 2 \mtarg \Tlabin }}.
  \label{lab-to-cm-2}
\end{equation}
Note that except for incident gammas, $\Tlabin$ is much smaller
than the rest mass $\myi$, so that $\chi$ is a
small, positive number.

In the next section of this appendix, for 2-body problems
the center-of-mass energy and momentum of the
emitted particle and residual are determined.  In order to boost these 4-vectors
to the laboratory frame, one may use the inverse of the matrix in
Eq.~(\ref{R-3}), so that
\begin{equation}
  \begin{bmatrix}
    E_{\text{lab}} \\
    \plaba \\
    \plabb \\
    \plabc
  \end{bmatrix}
   =
    \begin{bmatrix}
     \cosh \chi & \sinh \chi & 0 & 0 \\
     \sinh \chi & \cosh \chi & 0 & 0 \\
     0  & 0 & 1 & 0 \\
     0  & 0 & 0 & 1
  \end{bmatrix}
  \begin{bmatrix}
    E_{\text{cm}} \\
    \pcma \\
    \pcmb \\
    \pcmc
  \end{bmatrix}.
  \label{R-3-inv}
\end{equation}

\subsection{Incident photons}
\label{Sec:photon-in}
When the incident particle is a photon, the boost from the
center-of-mass frame to the laboratory frame must be determined
relativistically, because the mass of the incident particle is zero
but its momentum is nonzero.

In this case, Eq.~(\ref{p-lab-in}) simplifies to
$$
  | \plabin |  =  \Tlabin,
$$
and Eq.~(\ref{lab-to-cm-2}) becomes
$$
  \sinh \chi = \frac{ \Tlabin}
    {\sqrt{ \mtarg^2 + 2 \mtarg \Tlabin }}.
$$
It follows that
$$
  \cosh \chi = \frac{ \mtarg + \Tlabin}
    {\sqrt{ \mtarg^2 + 2 \mtarg \Tlabin }}.
$$

\section{Outgoing particles}
Denote by $\myo$ the rest mass of the
emitted particle and
$\Tcme$ its kinetic energy in the center-of-mass frame.
The convention in \xendl\ is that the energy~$Q$ of the reaction
is specified by the data, and the rest mass  $\mres$ of the residual
is calculated from
\begin{equation}
  \mres = \mtarg + ( \myi - \myo ) - Q.
 \label{Q-2body}
\end{equation}
Let $\Tcmr$ be the kinetic energy of the residual in the center-of-mass frame.
In terms of these variables, the energy-momentum invariant
for the system is the square of the total energy
$$
    S = (\myo + \Tcme + \mres + \Tcmr)^2,
$$
with the same value of $S$ as in Eq.~(\ref{spacetime-cm}).
The argument leading to Eq.~(\ref{p-cm}) shows that the
momentum~$\pcme$ of the emitted particle in the center-of-mass frame
has magnitude given by
\begin{equation}
  \pcme^2 = \frac{ \left[S - (\mres^2 + \myo^2)\right]^2 -
                 4 \mres^2 \myo^2 }
                {4 S }.
  \label{p-cm-e}
\end{equation}

It is not a good idea to use Eq.~(\ref{p-cm-e}) in a
computation, because of its subtraction of nearly equal
numbers.  It is therefore desirable to do some algebraic manipulation
in order to mitigate this problem as much as possible.
As a first step, Eq.~(\ref{p-cm-e}) is rewritten in the
form
\begin{equation}
  4 S \pcme^2 = \left[S - (\mres + \myo)^2\right]
              \left[S - (\mres - \myo)^2\right].
 \label{p-cm-e-1}
\end{equation}
In this expression, the subtraction of nearly equal
numbers is confined to the first factor on the right-hand
side.  For photon emission the two factors are identical.
An analysis of photon emission later, because it offers some
simplifications.

By using the expression for~$S$ in Eq.~(\ref{S-lab}), one
obtains the relation
$$
  S - (\mres + \myo)^2 =
  (\mtarg + \myi)^2 - (\mres + \myo)^2  + 2\mtarg \Tlabin.
$$
In terms of the energy $Q$ of the discrete 2-body reaction
and the parameter
\begin{equation}
  M_T = \mtarg + \mres + \myi + \myo,
 \label{def-M}
\end{equation}
it follows that
$$
    S - (\mres + \myo)^2 =
     M_T Q + 2\mtarg \Tlabin.
$$
Consequently, it is seen that Eq.~(\ref{p-cm-e}) may be
replaced by
\begin{equation}
  \pcme^2 = \frac{ (M_T Q +  2\mtarg \Tlabin )
                 ( M_T Q +  2\mtarg \Tlabin + 4\mres \myo)}
                {4 S }.
  \label{p-cm-e-OK}
\end{equation}

\textbf{Remark.}
It is clear from Eq.~(\ref{p-cm-e-OK}) that for endothermic
reactions ($Q < 0$), the threshold occurs when the incident
particle has kinetic energy
$$
  \Tlabin = \frac{-M_T Q}{2\mtarg}.
$$

In Eq.~(\ref{p-cm-e-OK}) there is subtraction of nearly equal
numbers when the kinetic energy $\Tlabin$ of the incident
particle is just above the threshold in endothermic reactions.
That operation is unavoidable in
the analysis of nuclear reactions.

Now that $\pcme^2$ has been obtained in Eq.~(\ref{p-cm-e-OK}), one may use
Eq.~(\ref{good-T}) to determine the kinetic energy of the
emitted particle in the center-of-mass frame as
\begin{equation}
  \Tcme = \frac{\pcme^2 }
          {\myo + \sqrt{ \myo^2 + \pcme^2 }}.
 \label{T-cm-e}
\end{equation}

\subsection{The boost to the laboratory frame}
It is often desired to determine the kinetic energy~$\Tlabe$
and momentum~$\plabe$ of the emitted particle in the laboratory frame
for given direction cosine~$\mucm$ in the center-of-mass frame.
It is possible to use the boost
Eq.~(\ref{R-3-inv}) to determine $\plabe$ as follows.  Recall that
the form of Eq.~(\ref{R-3-inv}) is determined by the requirement that
the first axis of the coordinate system was chosen parallel to~$\plabin$.
Consequently, one has
$$
  \pcmea = \mucm |\pcme|.
$$
If the orientation of the coordinate system is such that
$$
  \pcmec = 0
  \quad \textrm{and} \quad
  \pcmeb \ge 0,
$$
then
$$
  \pcmeb =  |\pcme| \sqrt{ 1 - \mucm^2}.
$$
The momentum components of the boost Eq.~(\ref{R-3-inv}) then
take the form
\begin{equation*}
\begin{split}
  \plabea &= (\myo + \Tcme)\sinh \chi +  \mucm |\pcme| \cosh \chi, \\
  \plabeb &= |\pcme| \sqrt{ 1 - \mucm^2}, \\
  \plabec & = 0.\\
\end{split}
\end{equation*}

The magnitude of the momentum in the laboratory frame is
$$
  |\plabe| = \sqrt{ \plabea^2 + \plabeb^2 + \plabec^2 }.
$$
If $|\plabe| = 0$, the direction cosine $\mulab$ in the laboratory
frame is undetermined.  Otherwise, it is given by
$$
  \mulab = \frac{\plabea}{|\plabe|}.
$$
The kinetic energy $\Tlabe$ is calculated from $|\plabe|$
by using Eq.~(\ref{good-T}).

\subsection{Photon emission}
When the emitted particle is a photon, because $\myo = 0$,
Eqs.~(\ref{p-cm-e-OK}) and (\ref{T-cm-e}) take the simpler form
$$
  E_{e, \textrm{cm}} = \Tcme = |\pcme| =
   \frac{ M_T Q +  2\mtarg \Tlabin }
                {2 \sqrt{S} }.
$$
For given direction cosine~$\mucm$ in the center-of-mass frame,
the energy component of the boost Eq.~(\ref{R-3-inv}) gives the
Doppler shift
$$
   E_{e, \textrm{lab}} = E_{e, \textrm{cm}}
     \left( \cosh \chi + \mucm \sinh \chi \right).
$$
The first component of the momentum of the photon in the laboratory
frame is
$$
  \plabea =  E_{e, \textrm{cm}}
     \left( \sinh \chi + \mucm \cosh \chi \right),
$$
so the direction cosine is
$$
  \mulab = \frac{\sinh \chi + \mucm \cosh \chi}
                {\cosh \chi + \mucm \sinh \chi}.
$$

\newcommand{\mayo}{m_{1,e}}
\newcommand{\mares}{m_{1,r}}
\newcommand{\mbyo}{m_{2,e}}
\newcommand{\mbres}{m_{2,r}}

\newcommand{\muacm}{\mu_{\text{1,cm}}}
\newcommand{\mubcm}{\mu_{\text{2,cm}}}
\newcommand{\mualab}{\mu_{\text{1,lab}}}
\newcommand{\mublab}{\mu_{\text{2,lab}}}

\newcommand{\Vacm}{\textbf{V}_{\text{1,cm}}}
\newcommand{\Valab}{\textbf{V}_{\text{1,lab}}}

\newcommand{\Talabe}{T_{1,e,\textrm{lab}}}
\newcommand{\Tacme}{T_{1,e,\textrm{cm}}}
\newcommand{\Tacmr}{T_{1,r,\textrm{cm}}}
\newcommand{\pacmi}{p_{1,i,\textrm{cm}}}
\newcommand{\pacme}{p_{1,e,\textrm{cm}}}
\newcommand{\pacmea}{p_{1,e,\textrm{cm},1}}
\newcommand{\pacmeb}{p_{1,e,\textrm{cm},2}}
\newcommand{\pacmec}{p_{1,e,\textrm{cm},3}}
\newcommand{\plabi}{p_{i,\textrm{lab}}}
\newcommand{\palabe}{p_{1,e,\textrm{lab}}}
\newcommand{\palabea}{p_{1,e,\textrm{lab},1}}
\newcommand{\palabeb}{p_{1,e,\textrm{lab},2}}
\newcommand{\palabec}{p_{1,e,\textrm{lab},3}}

\newcommand{\Tblabe}{T_{2,e,\textrm{lab}}}
\newcommand{\Tbcme}{T_{2,e,\textrm{cm}}}
\newcommand{\Tbcmr}{T_{2,r,\textrm{cm}}}
\newcommand{\pbcmi}{p_{2,i,\textrm{cm}}}
\newcommand{\pbcme}{p_{2,e,\textrm{cm}}}
\newcommand{\pbcmea}{p_{2,e,\textrm{cm},1}}
\newcommand{\pbcmeb}{p_{2,e,\textrm{cm},2}}
\newcommand{\pbcmec}{p_{2,e,\textrm{cm},3}}
\newcommand{\pbThetae}{p_{2,e,{\Theta_1}}}
\newcommand{\pbThetaea}{p_{2,e,{\Theta_1},1}}
\newcommand{\pbThetaeb}{p_{2,e,{\Theta_1},2}}
\newcommand{\pbThetaec}{p_{2,e,{\Theta_1},3}}
\newcommand{\pblabe}{p_{2,e,\textrm{lab}}}
\newcommand{\pblabea}{p_{2,e,\textrm{lab},1}}
\newcommand{\pblabeb}{p_{2,e,\textrm{lab},2}}
\newcommand{\pblabec}{p_{2,e,\textrm{lab},3}}

\section{Two-step 2-body reactions}
\label{Sec:2-step-2-body-rel}
This section presents a relativistic treatment of the 2-step
discrete 2-body reaction discussed in Section~\ref{Sec:2-step-2-body}.
The first step of the reaction is inelastic scattering, with an
excited outgoing particle.  This excited particle then splits into a final outgoing particle
and a residual.  The notation used for this reaction is as in
Section~\ref{Sec:2-step-2-body} so that for the first step, $\mayo$ is the rest mass of
the excited outgoing particle, $\mares$ is the residual, and $Q_1$ is
the $Q$-value.  For the second step of the reaction, $\mbyo$ denotes the rest mass of
the final outgoing particle, $\mbres$ is the residual, and $Q_2$ is
the $Q$-value.

For the first step of the reaction, the energy-momentum invariant $S_1$
is as in Eq.~(\ref{spacetime-cm}),
\begin{equation}
  S_1 = (\myi + \Tcmi + \mtarg + \Tcmt)^2.
 \label{spacetime-cm-step1}
\end{equation}
As mentioned in Eq.~(\ref{2-step-mayo}), the rest mass of
the excited outgoing particle from the first step is given by
$$
  \mayo = \myi + \mtarg - \mares - Q_1.
$$

For the excited particle emitted in the first step of the
reaction, the boost Eq.~(\ref{R-3-inv}) from the center-of-mass frame to the
laboratory frame takes the form
\begin{equation}
  \begin{bmatrix}
    \mayo + \Talabe \\
    \palabea \\
    \palabeb \\
    \palabec
  \end{bmatrix}
   =
    \begin{bmatrix}
     \cosh \chi_1 & \sinh \chi_1 & 0 & 0 \\
     \sinh \chi_1 & \cosh \chi_1 & 0 & 0 \\
     0  & 0 & 1 & 0 \\
     0  & 0 & 0 & 1
  \end{bmatrix}
  \begin{bmatrix}
    \mayo + \Tacme \\
    \pacmea \\
    \pacmeb \\
    \pacmec
  \end{bmatrix}.
  \label{R-3-inv-step1}
\end{equation}
As in Eq.~(\ref{lab-to-cm-2}),
the parameter~$\chi_1$ for this boost
is is given by
\begin{equation}
  \sinh \chi_1 = \frac{\sqrt{ 2 \myi \Tlabin + \Tlabin^2}}
    {\sqrt{( \mtarg + \myi )^2 + 2 \mtarg \Tlabin }}.
  \label{lab-to-cm-2-step1}
\end{equation}

For step~1 of the reaction, the total mass of the particles
in Eq.~(\ref{def-M}) is given by
\begin{equation}
  M_{1,T} = \mtarg + \mares + \myi + \mayo,
 \label{def-M-step1}
\end{equation}
and for the momentum of the excited outgoing particle in
the frame of the center of mass of the initial collision the
relation Eq.~(\ref{p-cm-e-OK}) takes the form
\begin{equation}
  |\pacme|^2 = \frac{ (M_{1,T} Q_1 +  2\mtarg \Tlabin )
                 ( M_{1,T} Q_1 +  2\mtarg \Tlabin + 4\mayo \mares)}
                {4 S_1 }.
  \label{p-cm-e-OK-step1}
\end{equation}
Then by Eq.~(\ref{T-cm-e}), the kinetic energy of the excited
particle emitted in the first step is
\begin{equation}
  \Tacme = \frac{|\pacme|^2 }
          {\mayo + \sqrt{ \mayo^2 + |\pacme|^2 }}.
 \label{T-cm-e-step1}
\end{equation}

The remaining items to be determined on the right-hand
side of Eq.~(\ref{R-3-inv-step1}) are the components of
the momentum~$\pacme$.
Let $\theta_1$ denote the angle between $\pacme$ and the
center-of-mass momentum~$\pacmi$ of the incident particle.
The coordinate system may be
chosen so that $\pacmec = 0$ and so that~$\theta_1$
satisfies the condition
$$
  0 \le \theta_1 \le \pi.
$$
Then with $\muacm = \cos \theta_1$, the components
of~$\pacme$ are
\begin{equation*}
\begin{split}
  \pacmea &= | \pacme | \, \muacm, \\
  \pacmeb &=     |   \pacme | \sqrt{ 1 - \muacm^2 }, \\
  \pacmec &=     0.
\end{split}
%\label{pcme-abc}
\end{equation*}

Now that all of the variables on the  right-hand
side of Eq.~(\ref{R-3-inv-step1}) are identified, those on
the left-hand side are determined as well.  In particular,
the magnitude of the momentum~$\palabe$ in the laboratory
frame of the excited
outgoing particle from step~1 of the reaction is given by
\begin{equation}
  | \palabe | = \sqrt{ \palabea^2 + \palabeb^2 + \palabec^2 }\,.
 \label{plab-rel-step1}
\end{equation}
The direction cosine~$\mualab$ for step~1 of the reaction is
defined as
\begin{equation}
  \mualab =
  \begin{cases}
     \palabea / | \palabe | &
       \text{for $| \palabe | \ne 0$,} \\
     1 & \text{otherwise.}
   \end{cases}
 \label{mu-lab-step1}
\end{equation}
Consequently, for $| \palabe | \ne 0$ the angle between
$\palabe$ and the laboratory-frame momentum~$\plabi$
of the incident particle is
\begin{equation}
   \Theta_1 = \cos^{-1} \mualab.
 \label{def-Theta1}
\end{equation}

For the breakup of the excited particle emitted
by step~1, the initial analysis takes place in the
frame in which this particle is stationary.  In this frame
the energy-momentum invariant is
$$
  S_2 = \mayo^2.
$$
The total mass for the breakup is
$$
  M_{2,T} = \mayo + \mbres + \mbyo.
$$
For step~2 of the reaction, Eq.~(\ref{p-cm-e-OK}) for the
magnitude of the momentum takes the form
\begin{equation}
  |\pbcme|^2 = \frac{ M_{2,T} Q_2
                 ( M_{2,T} Q_2 + 4\mbyo \mbres)}
                {4 S_2 }.
  \label{p-cm-e-OK-step2}
\end{equation}
In this frame, the
kinetic energy of the final emitted particle is
\begin{equation}
  \Tbcme = \frac{|\pbcme|^2 }
          {\mbyo + \sqrt{ \mbyo^2 + |\pbcme|^2 }}.
  \label{T-cm-e-OK-step2}
\end{equation}

For the relativistic treatment of this 2-step reaction,
the elements $\Inum_{g,h,\ell}$ of the transfer matrix are calculated
as in the Newtonian case Eq.~(\ref{2-step-muEint}), namely by
\begin{equation}
    \Inum_{g,h,\ell} =
     \int_{\calE_g} dE \, \sigma ( E ) w(E) \widetilde \phi_\ell(E)
    \int_{\muacm} d\muacm  \, g(\muacm \mid E)
    \int_{\Sigma_{1,h}} d\sigma_1 \, P_\ell( \mulab ).
  \label{2-step-muEint-2step}
\end{equation}
One difference from the Newtonian case is that, here
the isotropic emission in the breakup step is reflected
in the uniform distribution of the momentum
$$
   \pbcme =
  \begin{bmatrix}
    \pbcmea \\
    \pbcmeb \\
    \pbcmec
  \end{bmatrix}
$$
over the hemisphere~$\Sigma_1$ given by the relation
\begin{equation}
  \pbcmec =
  \sqrt{ |  \pbcme |^2 -  \pbcmea^2 -  \pbcmeb^2 }
  \label{2-step-def-Sigma0}
\end{equation}
for
$$
  \pbcmea^2 +  \pbcmeb^2 \le |  \pbcme |^2.
$$
The differential $ d\sigma_1$ in Eq.~(\ref{2-step-muEint-2step})
is the differential surface area
on the hemisphere~$\Sigma_1$ normalized so that
$$
  \int_{\Sigma_1} d\sigma_1 = 1.
$$

Another difference in Eq.~(\ref{2-step-muEint-2step}) from the
Newtonian case is the need to construct a boost of the
energy-momentum vector
\begin{equation}
  \begin{bmatrix}
    \mbyo + \Tbcme \\
    \pbcmea \\
    \pbcmeb \\
    \pbcmec
  \end{bmatrix}
  \label{2-step-final-Ep-cm}
\end{equation}
to the laboratory frame of the initial collision
\begin{equation}
  \begin{bmatrix}
    \mbyo + \Tblabe \\
    \pblabea \\
    \pblabeb \\
    \pblabec
  \end{bmatrix}.
  \label{2-step-final-Ep-lab}
\end{equation}
This boost will be constructed in such a way that the $\pblabea$
momentum axis in Eq.~(\ref{2-step-final-Ep-lab}) is parallel to
the momentum $\plabin$ of the initial incident particle.
In the integral in Eq.~(\ref{2-step-muEint-2step}), $\Sigma_{1,h}$
denotes the portion of $\Sigma_1$ on which the kinetic energy
$\Tbcme$ of the final emitted particle in the laboratory frame
is in the outgoing energy bin~$\calE_h'$.

The direction cosine~$\mulab$
in Eq.~(\ref{2-step-muEint-2step}) is taken to be
\begin{equation}
  \mulab =
  \begin{cases}
    \pblabea / | \pblabe | &
       \text {if $ | \pblabe | > 0$,} \\
     1 & \text{otherwise.}
  \end{cases}
  \label{2-step-final-mu-lab}
\end{equation}

The boost of the energy-momentum vector for the final
emitted particle from the frame of Eq~(\ref{2-step-final-Ep-cm})
to the laboratory frame of Eq.~(\ref{2-step-final-Ep-lab})
will be constructed as an initial boost followed by a rotation
of the momentum subspace.

For the excited particle emitted in step~1, the boost
\begin{equation}
  \begin{bmatrix}
    \mayo + \Talabe \\
    |\palabe|  \\
    0 \\
    0
  \end{bmatrix}
   =
    \begin{bmatrix}
     \cosh \chi_2 & \sinh \chi_2 & 0 & 0 \\
     \sinh \chi_2 & \cosh \chi_2 & 0 & 0 \\
     0  & 0 & 1 & 0 \\
     0  & 0 & 0 & 1
  \end{bmatrix}
  \begin{bmatrix}
    \mayo \\
    0 \\
    0 \\
    0
  \end{bmatrix}
  \label{boost-chi2}
\end{equation}
maps the energy and momentum from the center-of-mass
frame of the breakup reaction to a laboratory frame.
The value of~$\chi_2$ in the boost Eq.~(\ref{boost-chi2})
is determined by
$$
  \sinh \chi_2 = \frac{  |\palabe| }{ \mayo}
$$
with~$|\palabe|$ given by Eq.~(\ref{plab-rel-step1}).

An application of the boost Eq.~(\ref{boost-chi2}) to the
energy-momentum vector for the
particle emitted in step~2 of the reaction gives the result
\begin{equation}
  \begin{bmatrix}
    \mbyo + \Tblabe \\
    \pbThetaea \\
    \pbThetaeb \\
    \pbThetaec
  \end{bmatrix}
   =
    \begin{bmatrix}
     \cosh \chi_2 & \sinh \chi_2 & 0 & 0 \\
     \sinh \chi_2 & \cosh \chi_2 & 0 & 0 \\
     0  & 0 & 1 & 0 \\
     0  & 0 & 0 & 1
  \end{bmatrix}
  \begin{bmatrix}
    \mbyo + \Tbcme \\
    \pbcmea \\
    \pbcmeb \\
    \pbcmec
  \end{bmatrix}.
  \label{R-3-inv-step2}
\end{equation}

The mapping Eq.~(\ref{R-3-inv-step2}) takes the
center-of-mass energy-$\pbcmea$ plane onto the laboratory
energy-$\pbThetaea$ plane.  The construction Eq.~(\ref{boost-chi2})
ensures that this plane contains the vector~$\palabe$.  Both
$\palabe$ and the momentum~$\plabin$ of the initial incident particle
lie in the original laboratory momentum subspace, but Eqs.~(\ref{mu-lab-step1})
and~(\ref{def-Theta1}) show that these vectors are separated by
the angle~$\Theta_1$.  It is therefore desirable to rotate the
$$
  \pbThetae =
   \begin{bmatrix}
     \pbThetaea \\
    \pbThetaeb \\
    \pbThetaec
   \end{bmatrix}
$$
momentum subspace by the mapping
\begin{equation}
  \begin{bmatrix}
    \mbyo + \Tblabe \\
    \pblabea \\
    \pblabeb \\
    \pblabec
  \end{bmatrix}
   =
    \begin{bmatrix}
     1 & 0 & 0 & 0 \\
     0 & \cos \Theta_1 & -\sin \Theta_1 & 0 \\
     0  & \sin \Theta_1 & \cos \Theta_1 & 0 \\
     0  & 0 & 0 & 1
  \end{bmatrix}
  \begin{bmatrix}
    \mbyo + \Tblabe \\
    \pbThetaea \\
    \pbThetaeb \\
    \pbThetaec
   \end{bmatrix}.
  \label{R-3-inv-rotate}
\end{equation}
Recall that $\Theta_1$ is set equal to~$0$ if $|\palabe | = 0$, so that
Eq.~(\ref{R-3-inv-rotate}) is also valid in
this case.  For step~2 of the reaction, the mapping of the
energy-momentum vector from the center-of-mass frame
to the initial laboratory frame is accomplished by the boost
Eq.~(\ref{R-3-inv-step2}) followed by the rotation Eq.~(\ref{R-3-inv-rotate}).

One should not compute the kinetic energy~$\Tblabe$ by subtracting
the rest mass~$\mbyo$ from the total energy in the laboratory frame.
It is better to first calculate the square of the length of the momentum
$$
  |\pblabe|^2 = \pblabea^2 + \pblabeb^2 + \pblabec^2
$$
and to use
\begin{equation*}
  \Tblabe = \frac{|\pblabe|^2 }
          {\mbyo + \sqrt{ \mbyo^2 + |\pblabe|^2 }}.
  %\label{T-lab-e-OK-step2}
\end{equation*}

It remains to describe the details of the computation of
the integral
$$
    \int_{\Sigma_{1,h}} d\sigma_1 \, P_\ell( \mulab ),
$$
which appears in Eq.~(\ref{2-step-muEint-2step}).
This is an integral over a subset of the hemisphere
Eq.~(\ref{2-step-def-Sigma0}) of the center-of-mass
momentum subspace for the breakup step of the reaction.
In the Newtonian analysis of this reaction in Section~\ref{Sec:2-step-2-body},
a uniform measure~$d\sigma_0$ was derived for the
hemisphere~$\Sigma_0$ of Eq.~(\ref{2-step-def-zeta}) in
terms of the variables $\mubcm$ and~$w$ of Eqs.~(\ref{2-step-def-xi})
and~(\ref{2-step-def-eta}).  The analogous uniform measure~$d\sigma_1$
on the hemisphere~$\Sigma_1$ of Eq.~(\ref{2-step-def-Sigma0})
is given by the relations
\begin{alignat}{2}
  \pbcmea &= |\pbcme| \, \mubcm
    \quad &\text{for $-1 \le \mubcm \le 1$,}
      \label{2-step-def-xi-rel} \\
  \pbcmeb &= |\pbcme| \sqrt{ 1 - \mubcm^2 } \, \sin w
    \quad &\text{for $-\pi/2 \le w \le \pi/2$}.
   \label{2-step-def-eta-rel}
\end{alignat}
With this notation, the expression
$$
  d\sigma_1 = \frac{ d \mubcm \, dw}{2 \pi}
$$
gives a uniform measure on~$\Sigma_1$.

\begin{figure}
\input{figA-1}
\end{figure}

It is clear from the mappings Eqs.~(\ref{R-3-inv-step2}) and~(\ref{R-3-inv-rotate})
that for fixed values of the kinetic energy~$\Tlabin$ of the incident
particle and for fixed direction cosines $\muacm$ and~$\mubcm$,
the kinetic energy~$\Tblabe$ of the final emitted particle in the initial
laboratory frame is independent of the value of~$\pbcmeb$.
That is, $\Tblabe$ is constant for $\pbcmea$ and~$\pbcmeb$ on
the segment from $A$ to~$B$ in the $(\pbcmea, \pbcmeb)$-plane
in Figure~\ref{Fig:2-step-2-body-step2-rel}.
Note also that for fixed $\Tlabin$ and~$\muacm$, the relations
Eqs.~(\ref{R-3-inv-step2}) and~(\ref{2-step-def-xi-rel}) imply that
$\Tblabe$ is an increasing function of~$\mubcm$.  It follows
that if $\Tblabe$ lies in an energy bin~$\calE_h'$ for given
$\Tlabin$ and~$\muacm$, then it does so for $\mubcm$ on
an interval $[a_h, b_h]$ with
$$
  -1 \le a_h \le \mubcm \le b_h \le 1.
$$
This situation is of interest only if $a_h < b_h$.  In conclusion,
it has been shown that the integral over~$\Sigma_{1,h}$ in
Eq.~(\ref{2-step-muEint-2step}) may be written as
\begin{equation*}
   \int_{\Sigma_{1,h}} d\sigma_1 \, P_\ell( \mulab ) =
   \frac{1}{2 \pi }
   \int_{ a_h}^{b_h} d\mubcm \int_{- \pi/2}
                           ^{\pi/2} dw \,
          P_\ell( \mulab ) .
% \label{2-step-int-sigma-rel}
\end{equation*}

} % end of this appendix


\documentclass{standalone}
\begin{document}

\subsection{Classifiers Approach}

	Classifiers approaches use statistical pattern recognition techniques to segment images by using a mixture model which assumes that each pixel belongs to one of a prior known set of classes~\cite{INP:Withey}. To assign each pixel to the corresponding class, it uses the so-called \emph{feature space}, which is the space of any function of the image. An example of a 1D feature space is the image histogram.

	The features of each pixel form a pattern that is classified by assigning a probability measure for the inclusion of each pixel in each class~\cite{INP:Withey}.

	This approach assumes prior knowledge about the total numbers of features in the image and the probability of occurrence of each class. Generally, this quantity is not prior known, so we need a set of training data to use as a reference. There are different techniques which use this approach:

	\begin{itemize}

	\item \textbf{k-Nearest Neighborhood} : each pixel is classified in the same class as the training data with the closest intensity;

	\item \textbf{Maximum likelihood or Bayesian} : Assume that pixel intensities are independent samples from a mixture of probability distributions and the  classification is obtained by assigning each pixel to the class with the highest posterior probability.
	\end{itemize}

	This approach requires a structure to segment with distinctive and quantifiable features. It is computationally efficient and can be applied to multichannel images.
	This approach does not consider spatial modelling and it requires a manual interaction to obtain the training data that must be several since the use of the same training set for a large number of scans can lead to biased results.

\end{document}


\section{Discussion and Conclusions}\label{sec:conclusion}

In this work, we apply a machine-learning algorithm that involves a Gaussian process for the purpose of augmenting a stellar grid. We train GP models for the stellar grid and obtain continuous functions that map fundamental inputs to observable outputs on a good accuracy level. We also train another set of GP models which describe the local systematic uncertainty of each output parameter. A GP-trained model set is then generated and we use it to model fake stars. The GP-based modelling is obviously advanced in modelling individual stars compared with the gird-based modelling because it provides continuous sampling. We lastly test GP-determined masses and ages by comparing them with true values and find good consistence.

\subsubsection*{Advantages}
GP is a very efficient tool to augment a stellar grid instead of computing massive evolutionary tracks. For the case in this study, we train 10 GP-Grid models and 1 GP-SYS model for each output parameter and hence 55 GP models in total. The training process takes approximate one hour per GP model on a NVidia Tesla V100 graphics processing unit (GPU).
%
We also show that GP is able to manage the augmentation in high-demissions data (5 demissions in this work), which is very difficult for the interpolation approach.
%
Moreover, the section scenario overcomes the limitation of training data size (20,000 for this case) and offers flexibility to apply GP on stellar grids with different size.
%
When modelling individual stars, GP significantly improves the sampling especially for initial metallicity, helium fraction, and the mixing-length parameter, which are always well spaced in a stellar grid.
%

{\bf Any more points?}

\subsubsection*{Limitations}
GP is efficient for training global parameters but not for training the stellar structure. It hence not able to replace a stellar model. To obtain comprehensive stellar model, an optimal path would be using GP to constrain the range of fundamental parameters and then computing models with stellar code in that range.
%

{\bf add more limitations}

\subsubsection*{Future works}

This work presents an application of GP on augmenting the whole grid. However, this method is not efficient when modelling a single star. Our future work is developing a fast tool that only trains a small set of models in the grid around a star. This trained-GP model can be easily sticked with an Markov chain Monte Carlo simulator for a delicate bayesian analysis.
%

{\bf more future works}


\section{Results and discussion}
    \label{sec:results}

    \subsection{Latent Dirichlet allocation}

        Latent Dirichlet allocation was adjusted in our documents using five topics (components), the number of precedents we are considering. \autoref{fig:lda_topics} shows the top-20 words for each topic. From the barplot, we can see the following topics associated with the following BPs, justified by the following words:
        \begin{itemize}
                \item topic 1 \& BP 20: gratificação, inativos, atividade, desempenho, avaliação, gdata;
                \item topic 3 \& BP 4: salário, base, decisão, mínimo, constituição, cálculo;
                \item topic 4 \& BP 10: decisão, tribunal, constitucional, 10;
                \item topic 5 \& BP 14: acesso, direito, penal, inquérito, defeso, 14.
        \end{itemize}

        \begin{figure*}[!h]
                \includegraphics[width=\linewidth]{lda_topics.png}
                \caption{Top-20 words for each topic estimated with latent Dirichlet analysis.}
                \label{fig:lda_topics}
        \end{figure*}

        We can verify if the ``predicted'' topics match the real BPs being cited. Using this topic-to-BP assignment presented before, we get an accuracy of 0.76. That is, this topic modeling predicts the cited precedent correctly in 76\% of documents. We would expect 0.2 if it were a random assignment. So, LDA can extract the subjects of the BPs being cited.

    \subsection{Truncated SVD dimensionality reduction}

        \autoref{fig:svd_1} shows the resulting truncated SVD dimensionality reduction to 2. The color indicates the cited precedents, but the algorithm did not know that. We can see a pattern of data distribution, indicating that documents that cite the same BP are near each other.

        \begin{figure}[H]
                \includegraphics[width=\linewidth]{svd_1.png}
                \caption{Truncated singular value decomposition (SVD) dimensionality reduction to 2.}
                \label{fig:svd_1}
        \end{figure}

        \autoref{fig:svd_2_3}(a) and (b) present the result of dimensionality reduction to 3 dimensions. This graphic complements the last graphic, in the sense that it is more clear from this plot the fact that the documents can be separable in high dimensional space according to precedent citations.

        \begin{figure*}[!h]
                \centering
                \subfloat[\centering ]{{\includegraphics[width=0.4\linewidth]{svd_2.png} }}
                \qquad
                \subfloat[\centering ]{{\includegraphics[width=0.4\linewidth]{svd_3.png} }}
                \caption{Truncated singular value decomposition (SVD) dimensionality reduction to 3.}
                \label{fig:svd_2_3}
        \end{figure*}

    \subsection{Linear regression}

        The results of classification using linear regression are presented in \autoref{tab:ridge}. Cross-validation to choose the Ridge regularization parameter was performed. All test metrics, \ie, precision, recall, and accuracy, are very high, which indicates the data is \textbf{linear separable}. \autoref{fig:rigde} shows the test confusion matrix, presenting how many documents from each BP were predicted as each BP. It is almost a diagonal matrix, indicating a very well-adjusted model.

        % Please add the following required packages to your document preamble:
        % \usepackage{multirow}
        \begin{table}[H]
                \centering
                \caption{Linear regression test metrics.}
                \label{tab:ridge}
                \begin{tabular}{c|cc|c}
                BP & Precision & Recall & Accuracy              \\ \hline
                4  & 0.96      & 0.91   & \multirow{5}{*}{0.95} \\
                10 & 0.94      & 0.93   &                       \\
                14 & 0.94      & 0.99   &                       \\
                20 & 0.99      & 0.95   &                       \\
                37 & 0.90      & 0.95   &
                \end{tabular}
        \end{table}

        \begin{figure}[H]
                \centering
                \includegraphics[width=0.8\linewidth]{ridge.png}
                \caption{Linear regression test confusion matrix.}
                \label{fig:rigde}
        \end{figure}

    \subsection{Logistic regression}

        \autoref{tab:lr} presents the results of classification using logistic regression. Cross-validation to choose $C$ regularization parameter was performed. All test metrics, \ie, precision, recall, and accuracy, are very high. Comparing to linear regression, the accuracy is even higher. \autoref{fig:lr} (in \autoref{sec:confusion_matrices}, from now on) shows the test confusion matrix. It is also almost a diagonal matrix, indicating a very well-adjusted model.

        % Please add the following required packages to your document preamble:
        % \usepackage{multirow}
        \begin{table}[H]
                \centering
                \caption{Logistic regression test metrics.}
                \label{tab:lr}
                \begin{tabular}{c|cc|c}
                BP & Precision & Recall & Accuracy              \\ \hline
                4  & 0.98      & 0.97   & \multirow{5}{*}{0.97} \\
                10 & 0.96      & 0.96   &                       \\
                14 & 0.98      & 0.99   &                       \\
                20 & 0.97      & 0.97   &                       \\
                37 & 0.95      & 0.96   &
                \end{tabular}
        \end{table}

    \subsection{Linear discriminant analysis}

        The results of classification using linear discriminant analysis are presented in \autoref{tab:lda}. All test metrics, \ie, precision, recall, and accuracy, are very high, which indicates the data is \textbf{linearly separable}. \autoref{fig:lda} shows the test confusion matrix, almost a diagonal matrix.

        % Please add the following required packages to your document preamble:
        % \usepackage{multirow}
        \begin{table}[H]
                \centering
                \caption{Linear discriminant analysis test metrics.}
                \label{tab:lda}
                \begin{tabular}{c|cc|c}
                BP & Precision & Recall & Accuracy              \\ \hline
                4  & 0.97      & 0.91   & \multirow{5}{*}{0.95} \\
                10 & 0.91      & 0.95   &                       \\
                14 & 0.97      & 0.99   &                       \\
                20 & 0.99      & 0.93   &                       \\
                37 & 0.90      & 0.96   &
                \end{tabular}
        \end{table}

    \subsection{K-nearest neighbors}

        \autoref{tab:k_nn} presents the results of using k-nearest neighbors for classification. For choosing the number $k$ of neighbors, it was performed cross-validation, and the chosen hyperparameter was $k = 1$. Test metrics, \ie, precision, recall, and accuracy, are high. \autoref{fig:k_nn} shows the test confusion matrix, which indicates high performance.

        % Please add the following required packages to your document preamble:
        % \usepackage{multirow}
        \begin{table}[H]
                \centering
                \caption{K-nearest neighbors test metrics.}
                \label{tab:k_nn}
                \begin{tabular}{c|cc|c}
                BP & Precision & Recall & Accuracy              \\ \hline
                4  & 0.93      & 0.95   & \multirow{5}{*}{0.93} \\
                10 & 0.87      & 0.90   &                       \\
                14 & 0.95      & 0.96   &                       \\
                20 & 0.97      & 0.95   &                       \\
                37 & 0.92      & 0.90   &
                \end{tabular}
        \end{table}

    \subsection{Random forest}

        The results of using a random forest for classification are shown in \autoref{tab:rf}. Hyperparameters of maximum tree depth and the number of estimators were chosen using cross-validation, resulting in no maximum depth and number of trees equal to 100. From test metrics, we see a very high performance. The test confusion matrix is also excellent, almost a diagonal matrix (\autoref{fig:rf}).

        % Please add the following required packages to your document preamble:
        % \usepackage{multirow}
        \begin{table}[H]
                \centering
                \caption{Random forest test metrics.}
                \label{tab:rf}
                \begin{tabular}{c|cc|c}
                BP & Precision & Recall & Accuracy              \\ \hline
                4  & 0.98      & 0.96   & \multirow{5}{*}{0.96} \\
                10 & 0.91      & 0.95   &                       \\
                14 & 0.98      & 0.99   &                       \\
                20 & 0.98      & 0.96   &                       \\
                37 & 0.94      & 0.93   &
                \end{tabular}
        \end{table}

    \subsection{Support vector machine}

        Test metrics relative to support vector machine used for classification are presented in \autoref{tab:svm}. From the metrics, we see very high performance in test data. The hyperparameters, which includes kernel function (linear vs. RBF), regularization parameter $C$, and $\gamma$, were chosen by CV, and the result was $C = 100$, $\gamma = 0.001$, and kernel RBF. The test confusion matrix is excellent, which indicates a very high model performance (\autoref{fig:svm}).

        % Please add the following required packages to your document preamble:
        % \usepackage{multirow}
        \begin{table}[H]
                \centering
                \caption{Support vector machine test metrics.}
                \label{tab:svm}
                \begin{tabular}{c|cc|c}
                BP & Precision & Recall & Accuracy              \\ \hline
                4  & 0.99      & 0.97   & \multirow{5}{*}{0.97} \\
                10 & 0.95      & 0.97   &                       \\
                14 & 0.98      & 0.99   &                       \\
                20 & 0.98      & 0.97   &                       \\
                37 & 0.96      & 0.96   &
                \end{tabular}
        \end{table}

    \subsection{Further discussion}

        \autoref{fig:accuracies} resumes the supervised tests. It is clear from the experiments that our dataset is separable, in the sense of being able to be separated according to the Binding Precedent citation. Even more, the data seems to be \textbf{linearly separable}, as indicated by linear regression, logistic regression, and linear discriminant analysis. One very interesting property is that the documents, in very high dimensional space, are closer if they cite the same precedent, which is indicated by the results of truncated SVD dimensionality reduction and k-NN.

        \begin{figure}[H]
                % \centering
                \includegraphics[width=0.8\linewidth]{accuracies.png}
                \caption{Test accuracies of supervised models.}
                \label{fig:accuracies}
        \end{figure}

        Using latent Dirichlet allocation, and analyzing the results, we could verify that the topics, in general, extracted words and the subject of the cited precedents. We saw that, with five BPs and five topics, each topic could be associated with a BP, and the accuracy of this association was surprisingly high, over 0.75, against 0.2, the accuracy of a random assignment.

        With these models in hand, one could expand these experiments, considering more documents, more precedents, more sophisticated models, and build a model that confidently predicts the precedent being cited.


\section{Removing Some Assumptions and Mistakes from CLAuDE}
The first half of this stream was spent looking through the code and fixing some mistakes. To spare you dear reader from making these same mistakes, they have already been incorporated into
the previous sections, hooray! This does not only save you some work, but it also spares you from staring at a model that does not function due to wrongly defined constants or using the wrong
values.

\subsection{Adding a Spin-Up Time}
Instead of having a static start (having the planet start from rest, so no rotations allowed) we will have the model start up for some time before we start simulating the climate extensively.
To accomodate for this, we have to make some changes in the code. First we need to add two booleans (variables that can only take two values, either \texttt{TRUE} or \texttt{FALSE}) that we use
to indicate to the model whether we want to simulate the wind and whether we want to simulate advection. This means that the main loop will have some changes made to it. After performing the
calculations in \autoref{alg:temperature with density} we would calculate the velocities and afterwards we would calculate the advection. Instead let us change it to what is shown in
\autoref{alg:stream4v1}.

\begin{algorithm}
    \While{\texttt{TRUE}}{
        \autoref{alg:temperature with density} \;
        \If{$velocity$}{
            \autoref{alg:stream3} \;
            \If{$advection$}{
                \autoref{alg:advectionv2} \;
            }
        }
    }
    \caption{Main loop that can simulate flow and advection conditionally}
    \label{alg:stream4v1}
\end{algorithm}

Now to dynamically enable/disable the simulation of flow and advection we need to add the spin-up calculations to the main loop. So in \autoref{alg:stream4v1}, before
\autoref{alg:temperature with density} we add \autoref{alg:spinup}. What it does is it changes the timestep when spinnning up and disables flow simulation, and when a week has passed it reduces
the timestep and enables flow simulation. At this point in time, the advection is not dynamically enabled/disabled but it is done by the programmer. Currently it will break the model, so I
recommend leaving it on \texttt{FALSE} until it is fixed in \autoref{sec:advectionfix}.

\begin{algorithm}
    \eIf{$t < 7day$}{
        $\delta t \leftarrow 60 \cdot 47$ \;
        $velocity \leftarrow \texttt{FALSE}$ \;
    }{
        $\delta t \leftarrow 60 \cdot 9$ \;
        $velocity \leftarrow \texttt{TRUE}$ \;
    }
    \caption{The spin-up block dynamically enabling or disabling flow simulation}
    \label{alg:spinup}
\end{algorithm}

\subsection{Varying the Albedo}
The albdeo (reflectiveness of the planet's surface) is of course not the same over the whole planet. To account for this, we instead vary the albedo slightly for each point in the latitude
longitude grid. The algorithm that does this is shown in \autoref{alg:albedo variance}. The uniform distribution basically says that each allowed value in the interval has an equal chance of
being picked \cite{uniformdist}.

\begin{algorithm}
    $V_a \leftarrow 0.02$ \;
    \For{$lat \in [-nlat, nlat]$}{
        \For{$lon \in [0, nlon]$}{
            $R \leftarrow \text{ Pick a random number (from the uniform distribution) in the interval } [-V_a, V_a]$ \;
            $a[lat, lon] \leftarrow a[lat, lon] + V_a \cdot R$\;
        }
    }
    \caption{Varying the albedo of the planet}
    \label{alg:albedo variance}
\end{algorithm}

\subsection{Fixing the Advection} \label{sec:advectionfix}
Currently the advection does not work like it should. This is probably due to boundary issues, where we get too close to the poles and it starts freaking out there \cite{simon}. So to fix this
we are going to define boundaries and assume that the advection only works within those boundaries. We only let it change by half of the values. The changes are incorporated in
\autoref{alg:advectionfix}. The reason why we mention this seperately, in contrast to the other fixes that we have incorporated throughout the manual already, is the accompanying change with the
boundary.

\begin{algorithm}
    $\alpha_a \leftarrow 2 \cdot 10^{-5}$ \;
    $\alpha_p \leftarrow 1.5 \cdot 10^{-6}$ \;
    $boundary \leftarrow 7$ \;
    \While{\texttt{TRUE}}{
        $T_{add} \leftarrow T_a + \delta t \alpha_a \nabla^2(T_a) + \nabla(T_a)$ \;
        $T_a \leftarrow T_a - 0.5T_{add}[boundary:-boundary, :] \text{ //Only subtract } T_{add} \text{ to } T_a \text{ for indices in the interval } [-nlat + boundary, nlat - boundary]$. \;
        $\rho[boundary: -boundary, :] \leftarrow \rho - 0.5(\delta t \nabla \rho) \text{ //Only change the density for indices in the interval } [-nlat + boundary, nlat - boundary]$ \;
        $T_p \leftarrow T_p + \delta t \alpha_p \nabla^2(T_p)$ \;
    }
    \caption{The main loop for calculating the effects of advection}
    \label{alg:advectionfix}
\end{algorithm}

\subsection{Adding Friction}
In order to simulate friction, we multiply the speeds $u$ and $v$ by $0.99$. Of course there are equations for friction but that gets complicated very fast, so instead we just assume that we
have a constant friction factor. This multiplication is done directly after \autoref{alg:stream3} in \autoref{alg:stream4v1}.


\chapter{Characteristic Length Scales}
\label{sec:lengths}
%This table lists lengths used in the dissertation.

\captionsetup[table]{list=no}
%\begin{center}
\begin{table} % for label and caption purposes
%\begin{tabular}{|l|l|l|} % table runs off page
%\begin{tabular}{|l|l|p{7cm}|} % center column too large
\caption{Length scales used in this dissertation}
\begin{tabular}{|l|p{4cm}|p{8.5cm}|}
%\begin{tabular*}{0.75\textwidth}{|l|l|l|}{@{\extracolsep{\fill}} | c | c | c | }
\hline Symbol & Name & Description \\ \hline
$\lambda$ & wavelength & Wavelength of incident light \\
$L$ &  system length & Length of waveguide along direction of propagation ($z$-axis) \\
$W$ & system width & Dimension of waveguide perpendicular to direction of propagation ($y$-axis) \\
$L_{\phi}$ & phase coherence length & Length over which phase remains coherent. Equivalent to $L_{inelastic}$\cite{1988_Stone,1986_Imry}. Applicable only to electron transport. \\
$L_D$ & path length & How far a particle (i.e. ray optics) travels in the media in ballistic and diffusive regimes \\
$\ell_{scat}$ & scattering length & Average distance between scattering events. Often referred to as $\ell_{mfp}$ (mean free path) or the inelastic length\cite{1984_John_prl}, or extinction length\cite{1999_van_Tiggelen}. \\
$\ell_{tmfp}$ & transport mean free path & Average distance over which phase and direction are randomized.  $\ell_{tmfp} = \frac{\ell_{scat}}{1-\langle cos\ \theta \rangle}$. Sometimes referred to as elastic mfp\cite{1991_John}. % page 34, col 2
Measured with respect to $L$. \\
$\xi$ & localization length & Probability of diffusive path forming loop is 1. $\xi~=~N~\ell_{tmfp}$. Measured with respect to $L$. \\
$\ell_{a,g}$ & ballistic absorption/gain length & Average distance over which intensity decreases by two/increases by two. \\
$\xi_{a,g}$ & absorption/gain length & How far, on average, a particle travels in the diffusive regime before being absorbed (or doubled), measured with respect to path length $L_D$ in the diffusive regime \\
$z_p$ & penetration depth & Applies to diffusive regime only. $z_p \approx \ell_{tmfp}$ \\
\hline
\end{tabular}

\ \\

\label{tabl:lengths}
\end{table}
%\end{center}
All length scales (except $\lambda$) are normalized by wavelength. % lengths can be converted to equivalent times by $\ell = c t$
%\captionsetup[table]{list=yes}


%%
%% /docs/report/content/chapters/implementation.tex
%%
%% Created by Paul Warkentin <paul@warkentin.email> on 21/07/2018.
%% Updated by Paul Warkentin <paul@warkentin.email> on 26/07/2018.
%%

\section{Implementation}
\label{section:implementation}

We will discuss two different sizes of SSD networks in this project. The SSD 300 and SSD 512 networks accept image inputs of size $300 \times 300$ and $512 \times 512$ respectively. The smaller SSD 300 network is not as precise as the bigger SSD 512 network but can therefore score with a better inference time. \\

Some of the tricks in this implementation will be based on the original Caffe implementation \cite{ssdcfgit}. All presented methods are implemented in Python 3.6\footnote{\url{https://www.python.org/}} making prevalent use of TensorFlow 1.9\footnote{\url{https://www.tensorflow.org/}} and NumPy 1.14\footnote{\url{https://www.numpy.org/}}. The source code is publicly available at \url{https://github.com/paulwarkentin/tf-ssd-vgg}. The \texttt{README.md} provides additional information on how to get started.

\subsection{Model structure}

The model of the SSD network consists of three parts. \\

First, the input image is fed into a classifier network, here called the base network. In this project we use the VGG 16 network for this purpose. The fully connected layers in the standard architecture are replaced by convolutional feature layers.

Secondly, the output of the modified base network is fed into a series of extra feature layers. These layers decrease in size and allow the prediction of detections at multiple scales.

Finally, the output of the extra feature layers plus a selected layer from the base network are used as detection predictors. Each of these layers can produce a fixed set of predictions using a new set of convolutional layers.

\begin{center}
  \includegraphics[width=1.0\linewidth]{ssd_architecture.png}
  \captionof{figure}{Architecture of the SSD 300 network.}
\end{center}

The structure above is about the SSD 300 network. The slightly bigger SSD 512 network has two more extra feature layers.

\subsection{MultiBox priors}

MultiBox starts with a fixed set of default anchor boxes as priors for the predictions and attempt to approach the groundtruth bounding boxes using the priors. \\

For each feature layer, we compute default anchor boxes for a given set of aspect ratios and scales. The number of anchor boxes for a feature layer of size $(h, w)$ of $b$ different scales and ratios can be calculated by $b \cdot h \cdot w$. In the implementation of this project we use the following presets for the SSD 300 network.

\begin{center}
  \begin{tabular}{l|c|c|c}
    \textbf{Layer} & \textbf{Height} & \textbf{Width} & \textbf{\# Ratios} \\ \hline
    \texttt{conv4\_3} & 38 & 38 & 4 \\
    \texttt{conv7} & 19 & 19 & 6 \\
    \texttt{conv8\_2} & 10 & 10 & 6 \\
    \texttt{conv9\_2} & 5 & 5 & 6 \\
    \texttt{conv10\_2} & 3 & 3 & 4 \\
    \texttt{conv11\_2} & 1 & 1 & 4
  \end{tabular}
  \captionof{table}{Presets for the SSD 300 network.}
\end{center}

This sums up to a total of 8732 default anchor boxes. The SSD 512 network is defined by the following presets.

\begin{center}
  \begin{tabular}{l|c|c|c}
    \textbf{Layer} & \textbf{Height} & \textbf{Width} & \textbf{\# Ratios} \\ \hline
    \texttt{conv4\_3} & 64 & 64 & 4 \\
    \texttt{conv7} & 32 & 32 & 6 \\
    \texttt{conv8\_2} & 16 & 16 & 6 \\
    \texttt{conv9\_2} & 8 & 8 & 6 \\
    \texttt{conv10\_2} & 4 & 4 & 6 \\
    \texttt{conv11\_2} & 2 & 2 & 4 \\
    \texttt{conv12\_2} & 1 & 1 & 4
  \end{tabular}
  \captionof{table}{Presets for the SSD 300 network.}
\end{center}

This sums up to a total of 24564 default anchor boxes.

\subsection{Matching strategy}

Before the training step, we need to match the groundtruth boxes to the previous calculated default anchor boxes. The image below describes the logic of this step well. The bounding box of the dog is matched to the red dotted anchor box in the $4 \times 4$ feature layer but not to any shown box in the $8 \times 8$ feature layer. The criteria of a match is given by the Intersection of Union (in short: IoU) and a given threshold, here $\frac{1}{2}$. The IoU is dependent on the location, size and aspect ratios of the boxes. The bounding box of the cat on the other hand is matched to the blue dotted anchor box on the $8 \times 8$ feature layer.

\begin{center}
  \includegraphics[width=0.8\linewidth]{default_anchor_boxes.png}
  \captionof{figure}{Matching default anchor boxes.}
\end{center}

In this project, we follow a certain strategy for finding those matches. First, we match each groundtruth box to the default anchor box with the highest IoU. Then, each other default anchor box is matched to any groundtruth box with an IoU over a given threshold, here $\frac{1}{2}$.

\subsection{Training objective}

The loss function is the sum of two components.

\begin{itemize}
  \item \textbf{Confidence loss:} this loss measures the confidence of the predicted class of the computed bounding box. To compute this loss, categorical softmax cross-entropy is used.
  \item \textbf{Localization loss:} this loss measures the distance of the computed bounding box from the groundtruth box. To compute this loss, the L2-Norm is used.
\end{itemize}

Let $x_{ij}^p \in \{0, 1\}$ be the status of matching the $i$-th default anchor box to the $j$-th groundtruth box of class $p$. The set $X_P$ contains all default anchor boxes that were matched to a groundtruth box, the set $X_N$ on the other hand contains all unmatched anchor boxes. \\

The confidence loss is calculated for positive and negative samples by
\begin{equation*}
  L_{conf}(x, c) = - \sum_{i \in X_P} x_{ij}^p \log(\hat{c}_i^p) - \sum_{i \in X_N} \log(\hat{c}_i^0),
\end{equation*}
where $\hat{c}_i^p$ is the softmax function over confidences $c_i^p$ of class $p$,
\begin{equation*}
  \hat{c}_i^p = \frac{\exp(c_i^p)}{\sum_p \exp(c_i^p)}.
\end{equation*}
After the matching step, most of the default anchor boxes are not matched with any object, i.e. they are negatives. To compensate this imbalance of positive and negative samples, we perform hard negative mining. We sort the confidence losses of the negative samples in descending order and take only the samples with the highest loss so that the ratio of positive to negative samples is 1:3. \\

The localization loss is calculated for positive samples by using the predicted box $b$ and the groundtruth box $g$,
\begin{equation*}
  L_{loc}(x, b, g) = \sum_{i \in X_P} \sum_m x_{ij}^p \text{smooth}_{L1}(b_i^m - \hat{g}_j^m),
\end{equation*}
where $m \in \{cx, cy, w, h\}$ and $\hat{g}_j$ is the SSD encoding of the $j$-th groundtruth box using the $i$-th default anchor box $d_i$ defined by
\begin{alignat*}{2}
  & \hat{g}_j^{cx} = \frac{1}{d_i^w} (g_j^{cx} - d_i^{cx}),\ \ \ && \hat{g}_j^{cy} = \frac{1}{d_i^h} (g_j^{cy} - d_i^{cy}), \\
  & \hat{g}_j^{w} = \log\Big(\frac{g_j^w}{d_i^w}\Big), && \hat{g}_j^{h} = \log\Big(\frac{g_j^h}{d_i^h}\Big).
\end{alignat*}
The smooth L1-Norm is defined as
\begin{equation*}
  \text{smooth}_{L1}(y) = \begin{cases}
    \frac{1}{2} y^2 & \text{if} \ \lvert y \rvert \leq 1 \\
    \lvert y \rvert - \frac{1}{2} & \text{otherwise}.
  \end{cases}
\end{equation*}

The overall training objective is then calculated by adding and normalizing the latter two losses,
\begin{equation*}
  L(x, c, b, g) = \frac{1}{N} (L_{conf}(x, c) + L_{loc}(x, b, g)).
\end{equation*}
If the number of positive samples $N$ is equal to zero, we set the loss to zero.

\subsection{Data augmentation}

Before training, various data augmentation tricks are randomly applied to the input images to increase the number of samples and to reduce overfitting of the network. \\

First, the colors of the image are randomly distorted choosing one of four distortion packages. Each package adjusts the brightness, saturation, hue and contrast of the image with random parameters. Each package applies the distortions in a different order. After the color distortions, the RGB channels of the image are randomly reordered. \\

Secondly, the image is expanded, i.e. the image is randomly placed on a canvas up to four times bigger than the image itself. The background color of the canvas is set to the mean value of the dataset that was used to train the base network. All bounding boxes are transformed accordingly. \\

Thirdly, the image is randomly cropped so that the cropped image contains at least $\frac{1}{2}$ of any bounding box. All bounding boxes are transformed accordingly. \\

Lastly, the image is randomly flipped horizontally.

\subsection{Evaluation metric}

To evaluate the training results the mean average precision (in short: mAP) is used. mAP is often used in object detection. The metric is calculated using the precision and the recall of objects of one class.
\begin{itemize}
  \item \textbf{Precision:} measures how accurate the prediction is, i.e. the percentage of the positive predictions that are correct,
  \begin{equation*}
    \text{Precision} = \frac{TP}{TP + FP}.
  \end{equation*}
  \item \textbf{Recall:} measures how good all the positives were found,
  \begin{equation*}
    \text{Recall} = \frac{TP}{TP + FN}.
  \end{equation*}
\end{itemize}

The average precision is then calculated by computing the area under thre precision-recall curve. This value is computed for every class. The mAP is then the mean of the average precision values of all classes.


\chapter{Conclusions and Future Work} \label{chapter5:Conclusions}

This paper presents a method for estimating the covariance function of a stationary isotropic Gaussian process that does not rely on choosing a parametric family that may or may not fit the data well. It allows the data alone to drive the estimation, yet still guarantees that the estimated covariance function will turn out to be positive definite.

It is generally believed~\cite{Stein1999} that fitting a Mat\'ern covariance function, via maximum likelihood or something similar, will give good interpolation performance even if the covariance of the true underlying Gaussian process can't be approximated very closely by a function from the Mat\'ern family. Our simulations in Chapter~\ref{chapter3:Simulation-Study} support that claim. However, our method has performed comparably with regard to prediction (Figure~\ref{fig:pred-log}). In addition, we observed that our method can more closely match the covariance function (Figure~\ref{fig:boxplot-integrals}) in cases where a "hole effect", or distances that correspond to a negative covariance, exists. One example of this phenomenon is the damped cosine covariance function \eqref{eq:dampedcos}.

Our method involves estimating the covariance matrix element by element, which is an extremely computationally intensive problem, and one that would scale quite poorly if it were executed sequentially. However, this issue is largely alleviated through the use of parallel computing on a GPU. For a moderately sized problem (here $n = 400$), the GPU approach results in a speedup of roughly $100\times$. %Also, we expect that as the number of observations increases, our method would outperform other methods in terms of speed

% \section{Future Work} % (fold)
% \label{sec:future_work}

This semiparametric covariance estimation method could be adapted to allow us to relax the isotropy and stationarity assumptions. Modeling an anisotropic Gaussian process would be a straightforward extension; it would involve replacing the simplification in \eqref{eq:bochner2} with a two-dimensional Fourier transform
\[
	C(\bm{h}) = \iint_{-\infty}^{\infty} \exp(i\bm{h}^T\bm{\omega}) \; f(\bm{\omega}) \; d\bm{\omega}.
\]

It is possible to relax the stationarity assumption as well. In the literature there are several methods for dealing with spectral densities of nonstationary Gaussian processes. One notable example is the work of Fuentes~\cite{fuentes2002spectral}, who proposed a nonstationary periodogram that is a nonparametric estimator of the spectral density $f(\omega_1, \omega_2)$.

% This project is still a work in progress. As mentioned in Section~\ref{sec:calculating_the_likelihood}, I am working on a formal proof that the estimated likelihood converges to the true likelihood. In addition, I believe the performance of Algorithm~\ref{alg:mcmc} could be improved. The results in Figure~\ref{fig:result} are disappointing, but I think it could be more accurate with better proposal distributions and/or better knot placement. It could use more tinkering.

% The overall speed could be improved as well. Currently the MCMC algorithm runs in R, and invokes a CUDA C function to perform the Monte Carlo integrations. Translating all of the code into C/C++ would improve the performance even further. In particular, there is a matrix inversion via Cholesky decomposition in the likelihood calculation. It's possible to implement that in parallel on a GPU as well, which would allow $n$ to be larger than $400$ without harming performance too much.

% Finally, we would like to be able to relax the assumptions on the Gaussian process. We still need stationarity, but it should be a straighforward extension to eliminate the need to assume the process is isotropic.

% subsection future_work (end)


\documentclass{llncs}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{document}

\title{Permutations}
\titlerunning{Permutations}
\author{Jacque Carette \and Amr Sabry}
\authorrunning{Carette and Sabry}
\institute{? \\
\email{...@...}
\and
...}
\maketitle

\begin{abstract}

\keywords{}
\end{abstract}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Common Representations}

Say we want to write the permutation between \verb|Fin 3 x Fin 2| and
\verb|Fin 2 x Fin 3|. First let's enumerate the two sets in some
canonical order:
\begin{verbatim}
Fin 3 x Fin 2 = { (0,0), (0,1), (1,0), (1,1), (2,0), (2,1) }
Fin 2 x Fin 3 = { (0,0), (0,1), (0,2), (1,0), (1,1), (1,2) }
\end{verbatim}

Then the permutation can be visualized as:

\begin{verbatim}
Fin 3 x Fin 2 = { (0,0), (0,1), (1,0), (1,1), (2,0), (2,1) }
                    |      |      |      |      |      |
                    |      |      |      +---------+   |
                    |      |      |             |  |   |
                    |      +-------------+      |  |   |
                    |             |      |      |  |   |
                    |      +------+      |      |  |   |
                    |      |             |      |  |   |
                    |      |      +-------------+  |   |
                    |      |      |      |         |   |
                    |      |      |      |      +--+   |
                    |      |      |      |      |      |
Fin 2 x Fin 3 = { (0,0), (0,1), (0,2), (1,0), (1,1), (1,2) }
\end{verbatim}

How to represent this in a way that amenable to efficient and
convenient manipulation. There are several common representations:

\begin{itemize}
\item Truth table
\item Matrix
\item Reed-Muller expansion
\item Cycle expansion
\item Decision diagram
\item Vectors
\item Factorial representation
\item Terms of a programming language
\end{itemize}

As vectors we would get:
\begin{verbatim}
p = [ 0, 3, 1, 4, 2, 5 ]
\end{verbatim}
which says that the item at position \verb|i| goes to the item at
position \verb|p(i)|.

As a sequence of transpositions, we would get several possible
representations... etc.

Very difficult to work with such representations. Imagine trying to
write big permutations for a reversible adder. Imagine trying to prove
properties of such permutations (bimonoidal structure). Etc.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Essence}

Our point is that the essence of the permutation above is a type
equivalence that is trivial to prove. Indeed the equivalence between
\verb|A x B| and \verb|B x A| is trivial to prove. We can once and for
all derive |Fin m x Fin n ~= Fin (m * n)| from such a type
equivalence. We can derive the two monoidal structures and their
interactions. Our main theorem is that:
\begin{verbatim}
Fin, ~=, ~~~ is a weak rig groupoid
\end{verbatim}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\bibliographystyle{acm}
\bibliography{cites}
\end{document}


\section{Future Work}\label{sec:futurework}
For future work we formulate some specific ideas to further improve our machine learning model and the usability for end users.
On one hand we've created an android application prototype, which can read the accelerometer data readings which are needed when using our model.

Improving the machine learning model presented in this paper could be done in several ways. One way could be to extract even more features, specifically using the accelerometer data as signal data and extract more audio features such as: Continuous wavelet transform, Mel-frequency cepstrum and many more. Together with extracting different statistical features the result could be improved and features that are deemed more usable for our regression can be used. Similarly, analyzing the accelerometer data could also be used to extract features that for example indicate if the participant is walking, sitting or standing. And finally, features with a low correlation could be removed.

Additionally, a new dataset could be collected by recreating the original application from \cite{DBLP:conf/ijcai/KillianPNMC19} and eliminating some shortfalls they had in collecting the data, e.g. loss of phone power, missing or completely unusable data points. Furthermore, using a more solid method of collecting either the transdermal or blood alcohol content could be used and more features could be collected, such as if the participant is taking a drink or if he is present in a bar.

Varying the sliding window length and step size could improve the regression. Using a bigger value range for the hyperparameter optimization and more trials could also help to improve the results as well. And last but not least, evaluating other types of machine learning algorithms such as deep learning models with LSTM cells is feasible.


\section{Power series}


\section{Natural Language Generation}

Natural Language Generation (NLG) is a sub-field of Natural Language Processing that attempts to generate sequences of words that resemble natural human languages. Traditionally, this was done either by using production rules of a predefined grammar, or by performing statistical analyses of existing human-written texts to predict sequences of words, based on their occurrence probabilities \citep{cambria2014jumping}. Markov-chain text generators are an example of the latter, popularized by their usage in parody text generation \citep{jelinek1985markov}.

This broad classification of problems has multiple applications including, but not limited to:
\begin{itemize}
	\item Neural machine translation (NMT), in which the generation objective is to produce a semantically similar sentence in a target language, given a sentence in a source language \citep{bahdanau2014neural,cho2014properties,luong2015effective,wu2016google}.
	\item Dialogue generation, in which the objective is to produce a natural and syntactically correct response to a provided utterance \citep{cavazza2005dialogue,li2016persona,li2016deep,li2017adversarial}.
	\item Text summarization, which eliminates superfluous and non-pertinent information in a body of text, to express the idea using fewer words \citep{barzilay1999using,gong2001generic,conroy2001text}.
	\item Data-to-text report generation, which utilizes structured data, sourced from a relational data format, to fill out a text template \citep{goldberg1994using,reiter2007architecture,gatt2009data}.
\end{itemize}

\section{Multi-Layer Perceptrons}

Multi-layer perceptrons (MLP) are widely used in the neural network literature as the simplest vanilla artificial neural network. An MLP typically comprises one or more hidden layers that learn intermediate representations before producing an output layer. A simple MLP classification architecture for MNIST digits \citep{lecun2010mnist,deng2012mnist} is depicted in Figure \ref{fig:mlp-network}. The network is represented as a directed graph containing nodes and edges, where each node represents a neuron and each edge represents a learned weight. Each weight here is a model parameter. In this example, the weight between two successive layers are densely interconnected. Given a layer of $m$ nodes and a subsequent layer of $n$ nodes that are densely connected, we require $m*n$ edges (or weights) to connect the nodes of these two layers. These $m*n$ edges form a matrix representing the model parameters that transform a vector represented in the first layer to a vector represented in the second layer.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/mlp-network}
	\imgsrc{\url{http://corochann.com/mnist-training-with-multi-layer-perceptron-1149.html}}
	\caption{\label{fig:mlp-network} Multi-Layer Perceptron}
\end{figure}

In terms of function approximation, a multilayer perceptron can be thought of as nested function calls on the input $x$. For example, assume that the intermediate layers approximate the functions $f_1$ and $f_2$, and the output layer approximates the function $f_3$ in the example presented in Figure \ref{fig:mlp-network}. We can then say that the relation of the output distribution $y$ with respect to the features in the input sample $x$ would be described as $y = f_3(f_2(f_1(x)))$. The back-propagation of the objective loss over the weights of the network is achieved by using the chain-rule to differentiate this nested function. The gradients thus obtained are then used to update the weights, thereby changing how the output layer is computed during the next iteration of computation through the directed graph \citep{lecun1989backpropagation}.

The activation functions applied at each layer could either be linear or non-linear. The most popularly used non-linear activation functions are sigmoid, tanh, ReLU etc., which are typically used in the intermediate layers of the network. In the case of multi-label prediction problems, the output layer is typically a softmax distribution over the possible output classes. The softmax activation is simply the binary logistic regression classifier that generalizes to multiple classes, since the output of the softmax function can be used to represent a categorical distribution (Equation \ref{eqn:softmax-function}).

\begin{equation} \label{eqn:softmax-function}
	P(y=j \mid \mathbf{x}) = \frac{e^{\mathbf{x}^\mathsf{T}\mathbf{w}_j}}{\sum_{k=1}^K e^{\mathbf{x}^\mathsf{T}\mathbf{w}_k}}
\end{equation}

An MLP classifier is trained by minimizing a cross-entropy loss between the predicted softmax distribution $y$ and the true distribution $y'$
\begin{equation}
	\mathcal{H}_{y'} (y) := - \sum_{i}^K y_{i}' \log (y_i)
\end{equation}
where $K$ is the total number of classes.

\subsection{Dropout}

Dropout is a regularization technique used during the training of neural networks. It was proposed by \cite{srivastava2014dropout} as a simple method to prevent a model from over-fitting the approximately learned function on the training data. The idea of dropout is to randomly ignore a set of neural network weights i.e. randomly set function parameters to 0 during each instance of the forward pass, as shown in Figure \ref{fig:dropout}.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/dropout}
	\imgsrc{\cite{srivastava2014dropout}}
	\caption{\label{fig:dropout} Dropout}
\end{figure}

Each neuron / unit is dropped out with a predefined probability, which can be viewed as a model hyperparameter. This allows for more generalized and robust functions being learned by minimizing the co-dependence of neurons on each other. Dropout has also been empirically shown to improve experiment results on several supervised learning tasks in vision, speech recognition, document classification and computational biology \citep{srivastava2014dropout}.

\section{Convolutional Neural Networks}

A convolutional neural network (CNN) \citep{lecun1995convolutional} is a specialized neural network that is optimized for feature extraction from local regions in data, for instance a specific object in an image, or a specific attribute in a body of text. Using densely connected multi-layer perceptrons for feature extraction from all the pixels of an image increases the number of parameters and slows down the training process. Instead, convolutional networks use shared weights over the source data that are called `filters'.

The `convolution' within a convolutional neural network is implemented using a filter. A filter is a learned function that transfers an arbitrary spatial representation of the image, say $3\text{px} * 3\text{px}$, into a single value and this is done for all the similar spatial zones in the image. 2-dimensional convolutions are used to process features in images and video because of their inherent spatial nature. A 1-dimensional equivalent could be used to extract features from contiguous regions of text, usually 3 to 5 words \citep{kim2014convolutional}. These features are then sub-sampled to reduce the dimensionality for subsequent layers. This is typically achieved by max-pooling or average-pooling, which simply requires aggregating all the spatially proximal region activations produced by the convolutional layer.

The strategy of stacking convolutional and pooling layers allows the network to learn parameters that are translation invariant. In simpler words, an object can be identified regardless of its location in an image, and an attribute can be identified regardless of its position in a body of text.

An example CNN architecture is presented in Figure \ref{fig:cnn}.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/cnn}
	\imgsrc{\url{https://commons.wikimedia.org/wiki/File\%3ATypical_cnn.png}}
	\caption{\label{fig:cnn} Convolutional Neural Network}
\end{figure}

Since their inception, CNNs have been proven to achieve state-of-the-art results in multiple task settings within vision and language, especially with the use of deep convolutional architectures \citep{lawrence1997face,krizhevsky2012imagenet,karpathy2014large,kalchbrenner2014convolutional,kim2014convolutional,hu2014convolutional}.

\section{Recurrent Neural Networks}

Recurrent neural networks (RNNs) are a sub-class of artificial neural networks that are commonly used for sequence processing. Its units forms a directed graph that operates on a sequence of inputs in a temporally-distributed manner. This makes them a useful tool for extracting features from arbitrary length sequences of input like audio or text. The optimization problem an RNN tries to solve is to predict the next element in a sequence given the historical context, as shown in Equation \ref{eqn:rnn-next}.
\begin{equation} \label{eqn:rnn-next}
	P(w_1, \cdots, w_T) = \prod_{i=1}^T P(w_i | w_1, \cdots, w_{i-1})
\end{equation}

The language model is built in such a way that the features extracted from the sequence at time-step $t$ depend on the features observed during the time-steps $0 \cdots t-1$. A graphical depiction of a temporally-unrolled RNN is shown in Figure \ref{fig:recurrent-neural-network-unfold}.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/recurrent-neural-network-unfold}
	\imgsrc{\url{https://commons.wikimedia.org/wiki/File:Recurrent_neural_network_unfold.svg}}
	\caption{\label{fig:recurrent-neural-network-unfold} Recurrent Neural Network}
\end{figure}

The most recent and useful variants of recurrent units provide the ability to retain `memory' of the context in which the current features are to be extracted. In the domain of language processing, this typically implies that the recurrent unit has a stored memory of the previously observed words in a sequence, and this property grants it the ability to learn context as part of the feature space. The prominent variants of recurrent units used in neural networks to extract features from sequences using a memory mechanism, are Long Short-Term Memory (LSTM) units \citep{gers2001lstm} and Gated Recurrent Units (GRU) \citep{chung2014empirical}.

Recurrent networks in the domain of natural language processing are used frequently for both natural language understanding (NLU) tasks, as well natural language generation (NLG) tasks \citep{bengio2003neural,morin2005hierarchical,mikolov2010recurrent}. Similar to the manner in which recurrent networks can extract features from arbitrarily long sequences of vectorized text, they can also be used to produce text sequences from a unit vector representation of text, as represented in Figure \ref{fig:rnn-nmt}.

This is achieved by conditioning the generation of the first word of text either on some latent variable produced by a model, or by sampling from a generative distribution, and conditioning the generation of each subsequently predicted word on the word that was predicted in the previous time-step, until the model predicts an end-of-sentence (EOS) token.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/rnn-nmt}
	\imgsrc{\url{http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns}}
	\caption{\label{fig:rnn-nmt} Recurrent Encoder-Decoder Model}
\end{figure}

\subsection{Long Short-Term Memory}

As described in the previous section, recurrent neural networks (RNN) learn representations of the data over temporal sequences, like text or audio features. However, in practice, RNNs don't seem to be able to assign priorities to which of the past data they choose to ascribe a higher importance to. This is detrimental to their usage in tasks that require processing of long sequences of data like in text, audio or video processing. This lack of ability to learn long-term dependencies in the input features is shown in previous work by \cite{bengio1994learning}.

Long Short-Term Memory (LSTM) units, as described first by \cite{hochreiter1997long}, seek to address this issue by explicitly modelling how much to retain and forget at each time-step during the RNN's training procedure.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/lstm}
	\imgsrc{\url{https://isaacchanghau.github.io/post/lstm-gru-formula}}
	\caption{\label{fig:lstm} Long Short-Term Memory Unit}
\end{figure}

The structure of an LSTM is depicted in Figure \ref{fig:lstm}. An LSTM is comprised of three distinct gating layers internally - namely the forget gate, input gate and output gate - that determine its outputs: the new cell state and hidden state. As can be seen in Figure \ref{fig:lstm}, the cell state $C_t$ passes through the LSTM mostly unperturbed, which is intended to address the problem of vanishing gradients \citep{hochreiter1999vanishing}.

The forget gate uses a sigmoid activation to squash the values of the output of the previous time-step between 0 and 1, which can be understood as being semantically equivalent to deciding how much of the previous cell state to forget.
\begin{equation*}
	f_t = \sigma(W_f*[h_{t-1}, x_t] + b_f)
\end{equation*}

The input gate also uses a sigmoid activation on the previous hidden state, which is multiplied by the candidate cell state $\hat{C_t}$. The value thus obtained is then multiplied by the forget-gated previous cell state to form the next cell state.
\begin{align*}
	i_t =
	 & \sigma(W_i*[h_{t-1}, x_t] + b_i) \\
	\hat{C_t} =
	 & \tanh(W_c*[h_{t-1}, x_t] + b_c)  \\
	C_t =
	 & f_t * C_{t-1} + i_t * \hat{C_t}
\end{align*}

Now that we have obtained the cell state, we propagate that through to the next time-step. To obtain the hidden state, we use an output gate, again with a sigmoid activation, to gate the cell-state and create the hidden-state for the next time-step.
\begin{align*}
	o_t =
	 & \sigma(W_o*[h_{t-1}, x_t] + b_o) \\
	h_t =
	 & \tanh(C_t)
\end{align*}

The efficacy of LSTM-backed RNNs is evident from the prevalence of their usage in language modelling literature \citep{sundermeyer2012lstm,gers2001lstm,graves2005framewise} \footnote{\url{https://karpathy.github.io/2015/05/21/rnn-effectiveness}}.

\subsection{Gated Recurrent Units}

Gated Recurrent Units (GRU) \citep{cho2014learning} are similar to LSTMs in that they possess gating mechanisms to assist with long-term dependency learning. They differ from LSTMs in a few aspects, like the merging of the input and forget gates into a single update gate. They also merge cell state and hidden state, emitting only a single output vector after the computations within the cell. Figure \ref{fig:gru} depicts the internal GRU states and gates.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/gru}
	\imgsrc{\url{https://isaacchanghau.github.io/post/lstm-gru-formula}}
	\caption{\label{fig:gru} Gated Recurrent Unit}
\end{figure}

GRUs are simpler to work with in practice due to their having only a single output. They have been used interchangeably with LSTMs as memory units for recurrent neural networks \citep{chung2015gated,fu2017style}.

\section{Autoencoders}

Autoencoders are models that are parameterized to convert arbitrary data into a latent representation (encoder), and recover the original data back from the latent representation (decoder). In this setup, the dimensionality of the latent representation is usually much smaller than that of the actual data. A simple autoencoder architecture is depicted in Figure \ref{fig:autoencoder-structure}.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/autoencoder-structure}
	\imgsrc{\url{http://mlexplained.com/2017/12/28/an-intuitive-explanation-of-variational-autoencoders-vaes-part-1}}
	\caption{\label{fig:autoencoder-structure} Model Architecture: Autoencoder}
\end{figure}

By training a model to reconstruct data that is funnelled through a lower dimensional representation, two objectives can be achieved simultaneously:
\begin{itemize}
	\item The encoder weights of the model could be used to extract the most salient features of the data in a compressed representation, which is a better input feature format for downstream processing or learning algorithms. \citep{hinton2006reducing}
	\item The decoder weights of the model could be used as a generator. Given that we can sample from the distribution of the existing latent representations learned, or from a predefined prior (in a variational autoencoder), we can generate plausible novel data.
\end{itemize}

In the context of natural language, the encoder can be utilized as a sentence-encoding feature-extractor and the decoder can be utilized as a generative model. Autoencoders are also able to de-noise data given pairs of noisy and regular data, by learning a de-noising function, which can then be used for actual noisy data. These properties make autoencoders a good framework to utilize to implement sequence-to-sequence models, wherein both the encoder and decoder weights of the model are parameterized by neural networks that process and generate sequences of data, respectively.

\subsection{Variational Autoencoders}

Variational autoencoders (VAEs) \citep{kingma2013auto} are a probabilistic variant of autoencoders. The general structure of a VAE is the same as that of a deterministic autoencoder, with an encoder that learns a compressed latent space and a decoder that learns a function to map the latent space into the original data. In contrast with the deterministic autoencoder, a variational autoencoder parameterizes representations of the latent mean and variance as separate spaces. The method of variational inference requires placing conjugate priors on the mean and variance of the hitherto unknown latent representation. The encoder is then trained to learn the approximate posterior distribution, and the decoder is trained to generate novel data using samples taken from the prior distribution. The family of distributions used for this purpose is Gaussian. A simple VAE architecture is depicted in Figure \ref{fig:vae-structure}.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/vae-structure}
	\imgsrc{\url{http://mlexplained.com/2017/12/28/an-intuitive-explanation-of-variational-autoencoders-vaes-part-1}}
	\caption{\label{fig:vae-structure} Model Architecture: Variational Autoencoder}
\end{figure}

In addition to the reconstruction loss that a deterministic autoencoder uses, a metric of distance is used to evaluate how closely the learned representations of mean and variance resemble a Gaussian distribution with mean 0 and unit variance i.e. $\mathcal{N}(0,I)$. The co-variance matrix is treated as an identity matrix for the sake of simplicity. The Kullback-Leibler (KL) divergence \citep{kullback1951information} is used to penalize learned representations that do not resemble the prior. However, we cannot compute the KL divergence directly. Instead we compute an alternative objective that is equivalent to the KL divergence up until an added constant. This is called the evidence lower-bound (ELBO). The ELBO objective maximized in a variational autoencoder model is given in Equation \ref{eqn:elbo}:
\begin{equation} \label{eqn:elbo}
	ELBO(q) = \mathbb{E}_{q(z)} [(\log(p(x|z))] - \mathbb{KL}(q(z)||p(z))
\end{equation}

Simultaneously, for the decoding step, we sample from the prior, utilizing the re-parameterization trick to ensure that the model remains differentiable. As a result, the decoder is trained as a generative network that can randomly sample from the prior and generates plausible data that is similar to the input distribution. The decoder being able to act as a generative model independently is what sets a VAE apart from a deterministic autoencoder.

\section{Word Embeddings}

Prior to the usage of matrix factorization and neural models to learn continuous word representations, the numerical representations used for words and documents were one-hot representations, word-ngrams \citep{brown1992class} or tf-idf weighted statistics. Word-ngrams are useful for expressing probability distributions of word occurrences in a corpus. Higher order ngrams (bigrams, trigrams, four-grams etc.) also take into account the context of nearby words, and can be used to construct language models.

However, these methods use discrete word or character occurrence counts to represent words and documents, and these representations scale with the size of the input corpus vocabulary for word-based models. This makes discrete word representation based language models computationally inefficient for large corpora with diverse vocabularies. Character-based models, on the other hand, are liable to produce invalid words and don't work as well with recurrent memory units as words do \citep{bojanowski2015alternative}. These methods are also ineffective at modelling semantic inter-word relationships.

Word2Vec, which aims to alleviate these problems, is a vector space model that embeds words into a continuous space. Also, semantically similar words in Word2Vec are embedded in nearby spaces \citep{mikolov2013distributed,mikolov2013linguistic,le2014distributed}. Word2Vec can model the corpus vocabulary in two different ways, namely CBOW and Skip-gram. The CBOW model predicts a target word from context words, whereas the Skip-gram model does the opposite and predicts target words from a single context word. Both model types are shown in Figure \ref{fig:efficient-models}

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/efficient-models}
	\imgsrc{\cite{mikolov2013efficient}}
	\caption{\label{fig:efficient-models} Word2Vec Model}
\end{figure}

The Word2Vec model is a simple shallow neural network, with single input, hidden and output layers. The hidden layer is of arbitrary size, called the embedding size. The embedding size is usually much lower that the vocabulary size, which is reminiscent of an autoencoder that learns a compressed salient representation of the words. The one-hot encoded input is transformed into its context or target word, depending on whether the algorithm chosen is CBOW or Skip-gram. This transformation is parameterized by the shallow neural network. After the model converges, the network weights connecting the input and the hidden layer can be used as parameters to obtain dense, continuous representations of words.

A surprising outcome of this embedding model is that translations within the learned vector space seem to emulate semantic (gender, tense) as well as factual relationships (country-capital), as shown in Figure \ref{fig:word2vec-linear-relationships}.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/word2vec-linear-relationships}
	\imgsrc{\url{https://www.tensorflow.org/tutorials/word2vec}}
	\caption{\label{fig:word2vec-linear-relationships} Word2Vec Relationships}
\end{figure}

Word embedding neural models are also shown to be implicitly factorizing a word-context matrix by subsequent work in this area \citep{levy2014neural}.

\section{Sequence-to-Sequence Modelling}

Also abbreviated in the literature as Seq2Seq, this is a class of models that learns functions to transform one sequence into another. First introduced by \cite{sutskever2014sequence}, Seq2Seq is a flexible framework for modelling transformations made to arbitrary length sequences, typically using a neural encoder-decoder framework. In the field of natural language processing, the main tasks that benefit from such an encoder-decoder framework are those for which there exists two distinct distributions of text, and the model is trained to learn the mapping from one to the other. Similarly it can also be used for sequence autoencoding tasks, by learning to reconstruct sequences.

\cite{sutskever2014sequence} define a sequence-to-sequence model that learns a function (Equation \ref{eqn:seq2seq}) to map a sequence of inputs $x_1, \cdots , x_T$ to a sequence of outputs $y_1, \cdots , y_{T'}$, where the initial state $h$ is set to the hidden LSTM representation of $x_1, \cdots , x_T$.

\begin{equation} \label{eqn:seq2seq}
	p(y_1, \cdots , y_{T'} | x_1, \cdots , x_T) =	\prod_{t=1}^T p(y_t | h, y_1, \cdots , y_{t-1})
\end{equation}

This makes the Seq2Seq model an ideal framework using which one can implement solutions to several natural language generation tasks like neural machine translation, dialogue generation, text summarization, etc.

\section{Generative Adversarial Networks}

\cite{goodfellow2014generative} presented a novel generative model that utilizes the idea of a game-theoretic competition between a generative model which is a decoder, and a discriminative model which is a classifier. The general idea is to train the discriminative model, also known as the adversary, to be able to discriminate between the synthetic distribution that the generator model produces, and a true distribution that the generator trying to mimic. Figure \ref{fig:gans} depicts a sample GAN model architecture.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/gans}
	\imgsrc{\url{https://deeplearning4j.org/generative-adversarial-network}}
	\caption{\label{fig:gans} Model Architecture: Generative Adversarial Networks}
\end{figure}

In the original formulation of adversarial training, which involves using a binary classifier, the generator learns weights to map a randomly initialized latent variable $z$ to a data distribution $x$ that it is unaware of. The discriminator is trained using the true distribution and samples generated by the generative model, with the objective to discriminate the true samples from the generated samples. The generator on the other hand, attempts to maximize the discriminator's classification error without directly modifying the discriminator's parameters, thereby producing plausible samples that mimic the true distribution. In this manner, both the generator and discriminator are simultaneously trained to both produce better synthetic samples, as well as differentiate between real and synthetic samples respectively. This min-max game is represented by Equation \ref{eqn:gan-minimaxgame}, where $D$ is the discriminator, $G$ is the generator and $V$ is the value-function or loss-function of the generative model.

\begin{equation}
	\label{eqn:gan-minimaxgame}
	\min_G \max_D V(D, G) = \mathbb{E}_{\bm{x} \sim p_{\text{data}}(\bm{x})}[\log D(\bm{x})] + \mathbb{E}_{\bm{z} \sim p_{\bm{z}}(\bm{z})}[\log (1 - D(G(\bm{z})))].
\end{equation}

This property the generator model's parameters possess - of translating random noise into plausible data samples - is used in areas of research within computer vision and natural language processing to produce novel data samples. Also, adversarial learning as a principle is an effective tool to ensure that a feature space is devoid of any arbitrarily chosen set of known attributes. We use this property of adversarial learning in our model to assist with the disentanglement of stylistic attributes.

\section{Visual Style Transfer}

The idea of neural style transfer was originally proposed by \cite{gatys2016image}. In the task described in the paper, the authors use two distinct images as input. The first image contributes the content, and the second contributes the style. The objective is to generate a final image that contains all of the physical objects visible in the first image, but depicted in the style (hue, brush strokes, texture, etc.) visible in the second.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\textwidth]{images/image-style-transfer}
	\imgsrc{\cite{gatys2016image}}
	\caption{\label{fig:image-style-transfer} Artistic Style Transfer in Images}
\end{figure}

The features, as with most of the state-of-the-art image processing techniques, are extracted from the images using Convolutional Neural Networks (CNNs) \citep{lecun1998gradient}, the deep neural network variants of which have been widely successful at image recognition tasks such as object recognition in the ImageNet dataset \citep{krizhevsky2012imagenet}.

As depicted in Figure \ref{fig:image-style-transfer}, subsequent layers of the CNN learn incrementally higher level features in both the style image and the content image. These learned features for the style and content images are stored for future use. To generate the final image, the authors begin with a random white noise image. This image is passed through both networks, the parameters of which extract both its style features and its content features.

This model uses two training objectives:
\begin{itemize}
	\item Minimize the the element-wise mean squared difference between the style features computed for the style image and the white-noise image.
	\item Minimize the the mean squared difference between the content features of the content image and the white noise image.
\end{itemize}

The overall training loss is a linear combination of the above objectives, and is used to iteratively modify the white-noise image until it has the style features of the style image and the content features of the content image.

In this chapter, we discussed the background needed to understand the methods used in our research and other related work. The next chapter builds on the fundamentals established here, and discusses previous attempts at neural disentanglement in the context of linguistic style transfer, as well as methods that do not use disentanglement as the basis of their style transfer model.