% $Id: PropagatorOverview.tex,v 1.1 2008/10/09 16:16:11 dconway Exp $
\chapter{\label{chapter:PropagatorOverview}Propagation in GMAT}
\chapauthor{Darrel J. Conway}{Thinking Systems, Inc.}

\section{Propagator Overview}

GMAT's model contains a subsystem that is used to move a user defined mission through time,
following the sequence of events defined in the Mission Control Sequence.  That subsystem is
referred to as the propagation subsystem.  In its most generic form, GMAT's propagation subsystem
can be thought of as taking a mission state $\textbf{X}_i$ at some time $t_i$ and moving that state
by some amount $\delta t$ to some new time $t_f$, resulting in a new state $\textbf{X}_f$.  The
components that are used to perform this change of state are the elements of the propagation
subsystem.  In other words, GMAT's propagation subsystem performs the actions needed to calculate
the new state by performing the operation

\begin{equation}\label{eq:abstractProp}
\textbf{X}_f(t_f) = F(\textbf{X}_i(t_i), \delta t)
\end{equation}

\noindent This chapter provides an overview of the definition of the quantities in this equation:
the definition of the mission state vector and all of its components, the evolution operator $F$
that moves the state vector, and an overview of the construction and decomposition of these
quantities in GMAT.

GMAT's propagation subsystem is used to perform all analytic propagation and numerical integration
needed by the system.  That includes modeling the time lines for spacecraft and formations, mass
depletion modeling during finite duration maneuvers, evolution of the state transition matrix,
attitude propagation, evolution of physical properties and their variances as needed for orbit and
attitude determination problems, and evolution of costate variables needed for optimal control
problems.  The diversity of propagation issues encountered in the coverage provided by the
propagation subsystem makes it a potentially quite complicated collection of components.  For that
reason, the subsystem is broken into a collection of different elements each tailored to specific
aspects of the propagation problem regime.  The propagation problem requires a consistent, complete
description of the evolution operators and the propagation vector, which are discussed briefly below
and in detail in Chapters~\ref{chapter:Propagators} and~\ref{chapter:PropagatorStates},
respectively.  This chapter concludes with a description of the scripting elements used to drive
propagation in GMAT, using the Propagate command as an example of this scripting.

\section{Propagator Overview}

The function $F$ in equation~\ref{eq:abstractProp} defines the process that takes a vector of
numbers defining system properties at one time and finds the values of those same properties at a
different time.  This process of moving the properties through time is performed using one
or more evolution operators.  The objects in GMAT that perform this evolution are the propagators.
GMAT's propagators are designed to support evolution either forwards or backwards in time.  The
propagators can be set to run in a synchronized manner or independently of one another.  The
selection of the category and type of propagator that is used is selected by the user based on the
types of parameters that need to be propagated.

GMAT Supports three different categories of propagators, defined by the type of data available for
the propagation.  The categories are

\begin{description}
\item[Analytic] GMAT's analytic propagators take a known, closed form solution to the evolution
problem and apply that solution to the propagation vector specified at some initial time to find its
values at some other time.  GMAT's analytic propagators use embedded evolution equations to
calculate the values of parameters at the desired times.
\item[Numerical] The numerical propagators find a solution of the differential equations of motion
for the parameters using a precision numerical integrator and derivative models.  Propagators in
this category consist of a pair of objects: a numerical integrator derived from the Integrator class
and a derivative model derived from the PhysicalModel class.
\item[Precalculated]  Precalculated parameter values are retrieved -- and, if needed, interpolated
-- from files containing the time history of the parameter values.
 \end{description}

Each category of propagator supplies implementations of core methods designed to standardize
the propagation interface.  The core propagation interfaces are defined in the Propagator base
class.  Every GMAT propagator uses these interfaces to perform the following actions:

\begin{itemize}
\item Map the propagation vector to known evolution operators
\item Initialize all data structures and object references needed to propagate
\item Load the initialized data structure immediately before propagating
\item Propagate the propagation vector
\item Reset the propagator to use its initial settings
\end{itemize}

\noindent The Propagator class defines these interfaces to perform the actions listed above:

\begin{description}
\item[GetPropagatorOrder]  GMAT includes both first and second order numerical integrators.  This
interface is used to determine which is needed for a particular integrator.  The non-numerical
integrators all report order 0.
\item[SetPhysicalModel]  The SetPhysicalModel method sets the derivative model used for numerical
integration, and -- for all propagators -- performs pre-initialization tasks required before
initialization can proceed.  This method ensures that all of the elements needed for evolution have
been set for the propagator.
\item[Initialize]  All of the reference object pointers and other interconnections necessary for
propagation are set in the Initialize method.  Propagators that do not allow for dynamic resizing of
the propagation vector also use this method to initialize the propagation vector mapping needed for
the evolution.  Those that may require dynamic resizing perform a preliminary mapping during
initialization.
\item[Update]  The Update method is used to reset the propagator when the underlying physical model
changes.  This method is used to refresh the propagation vector mapping if the propagation vector
has changed -- for instance, when a transient derivative is applied or removed to model mass flow
during a finite maneuver.  It also resets some of the propagator's properties if needed, like the
initial step size for variable step propagators.
\item[ResetInitialData]  The ResetInitialData method sets a flag telling the propagator that at the
next call to propagate the propagation vector, the propagator's initial data needs to be updated
prior to propagation.
\item[Step]  State evolution is driven through calls to the Step method.  Different categories and
subcategories of propagators implement different versions of this method based on the needs and
capabilities of the propagators.
\end{description}

\noindent These methods define the minimum level of interface required for every GMAT propagator.
The Propagator subclasses extend this set based on the needs of the category of the propagator
defined in the subclass.

The design details for the propagators are provided in Chapter~\ref{chapter:Propagators}.

\section{Propagation Vector Overview}

The propagators evolve a vector of data from one epoch to another.  The data that evolves is
contained in a column vector sized to match the evolving data.  This vector is assembled and
populated prior to propagation based on specifications scripted by the user.  The propagation
vector provides methods that support the following actions:

\begin{itemize}
\item Supply epoch information
\item Supply propagation vector information, including all of the following:
\begin{itemize}
\item Size of the vector
\item Element by element types for mapping the vector onto the propagator
\end{itemize}
\item Supply the propagation vector
\end{itemize}

\noindent GMAT uses a class, PropVector, to model the propagation vector.  The PropVector objects
supply the information used to assemble the derivative vector needed for numerical integration, the
epoch and state data needed for analytic propagation, or the epoch and time step information used
for precalculated propagation.

The PropVector class defines these interfaces to perform the actions listed above:

\begin{description}
\item[GetEpoch and SetEpoch]  A PropVector contains data calculated at a single epoch.  These
GetEpoch and SetEpoch methods provide access to the epoch data in the PropVector.
\item[GetDimension]  Retrieves the current size of the PropVector -- that is, the number of
elements that evolve over time.  The dimension is used to initialize the propagator, and, in
conjuction with the vector map, to ensure that the correct operator is used to evolve each element
of the vector.
\item[GetVectorMap]  Retrieves the mapping of the PropVector, element by element, so that the
evolution components can be set correctly.
\item[GetVector]  Retireves a pointer to the Real array of data for the vector.  The retrieved
vector has the size given by the dimension of the PropVector, mapped as specified in the vector
map.  Components that use the PropVector for propagation can both read and write to this retrieved
vector.  This violation of encapsulation is intentional, in order to make the propagation process
as fast as possible in GMAT.
\end{description}

\noindent These interfaces define the minimal interfaces necessary for PropVector objects.  They
are defined as overridable interfaces so that classes can be derived form PropVector in the future
as needed.

The PropVector class is managed through a mapping class, the MissionState.  The MissionState class
is a container class -- it contains references to the objects providing state data that evolves
using GMAT's evolution operators, as defined in its Propagator classes.  The MissionState class
provides methods accessed by the propagators to construct, retrieve, and update the PropVector.  It
manages the data flow between these objects and the propagation vectors manipulated by the
Propagators.  The MissionState builds the PropVectors used by the propagators, setting up the data
vector and associated component mappings used by the propagator to assemble the elements needed for
propagation.

The MissionState class defines a set of interfaces used to manage the data soureces used to assemble
the PropVector.  The interfaces supplied by the MissionState for these tasks are:

\begin{description}
\item[AddObject]  Adds an object that supplied data to one or more PropVectors.
\item[Initialize]  Constructs the PropVector or PropVectors used by a propagator.
\item[PrepareToPropagate] Completes PropVector initialization needed prior to propagation.
\item[GetPropVector]  Retrieves an assembled PropVector designed for propagation.
\end{description}

\noindent Each one of GMAT's propagators uses a MissionState object to manage the data that is
propagated.

The PropVector and MissionState classes are described in detail in
Chapter~\ref{chapter:PropagatorStates}.

\section{Scripting Propagation}

As is described above, GMAT provides three approaches to propagation.  The system can model the
evolution of the state data by numerically integrating the equations of motion, providing a high
precision model of the evolution, by reading the data from a pre-calculated file of data points
(e.g. a SPICE file), or by using an analytical propagator that provides a fast, lower fidelity
vector of propagated data.  The propagation subsystem can perform these tasks for all of the
parameters represented in the propagation vector: for the spacecraft orbit and attitude data, mass
properties, for the state transition matrix and covariance matrices, and for any other parameters
placed in the propagation vector, as long as the underlying propagation model supporting the
parameter exists.  For the numerical integrators, that means that the  set of ordinary differential
equations describing the evolution have been coded into the system.  Similarly, analytic
propagators require encoding of the analytic equations of motion.  The precalculated data
propagators require an ephemeris or similar time based file of the evolution of the paramteres that
they propagate.  GMAT is designed to support mixed mode propagation as well; elements propagated in
a Mission Control Sequence command need not all be propagated using the same approach.  Some can be
propagated analytically or using a file while others are numerically integrated.  The design
implementing this mode is described in Chapter~\ref{chapter:Propagators} and in the descriptions of
the commands that support propagation.

Commands that drive propagation follow the process shown in
Figure~\ref{figure:CommandPropagationOverview}.  The command is initialized in the Sandbox along
with the rest of the Mission Control Sequence.  During this initialization, specific pieces
required to initialize each propagator driven by that command are performed.  There are portions of
the propagation process that cannot be initialized at this point in the mission run because they
depend on data that may be changed when commands that precede the one invoking propagation fire.
This final piece of initialization is performed in a method named PrepareToPropagate(), which is
called when the command's Execute() method is called, immediately before the actual propagation.
Finally, the propagation is performed in the body of the Execute() method associated with the
command.

\begin{figure}[htb]
\begin{center}
\includegraphics[scale=0.5]{Images/PropagationintheMCS.eps}
\caption[Command and Propagation Processes]{\label{figure:CommandPropagationOverview}Command and
Propagation Processes}
\end{center}
\end{figure}

\subsection{Example: The Propagate Command}

Propagation is scripted in GMAT through the creation of propagator objects and the incorporation of
these objects into the Mission Control Sequence through propagator enabled commands.  The
prototypical propagator enabled command is the Propagate command (described more fully in
Section~\ref{section:Propagate}), scripted in its simplest form like this:

\begin{quote}
\begin{verbatim}
Propagate prop(sat)
\end{verbatim}
\end{quote}

\noindent The command shown here consists of three elements: the Propagate keyword, the name of the
propagator that is used (``prop'' in this case), and the object that is propagated, ``sat'' in
this example.  In order for this command to run, the objects used in teh propagation must exist, so
there must be matching creation commands.  Assuming that the command exists in the main script
rather than a function, that means these lines must occur before the Propagate command line:

\begin{quote}
\begin{verbatim}
Create Spacecraft sat;
Create Propagator prop;
\end{verbatim}
\end{quote}

\noindent The propagated object can, of course, be any object that supplies a propagation vector for
propagation.

The type of propagator is specified by setting the ``Type'' property on the propagator object, like
this:
\begin{quote}
\begin{verbatim}
prop.Type = PrinceDormand78;  % Use a Prince-Dormand RK integrator
\end{verbatim}
\end{quote}

\noindent The Type setting for the propagator must be made before setting any other propagator
properties because the properties for the propagator depend on the type of Propagator object that is
being used.  Every GMAT propagator has default values for each of its properties.  Users override
these settings to tailor the behavior of the propagator.

The following sections describe the scripting for each category of Propagator.

\subsubsection{Scripting Numerical Integrators}

The numerical integrators require an additional object defining the forces and other differential
equations that are used to model the propagation.  These pieces are gathered in a container class,
the ODEModel class derived from PhysicalModel.  The ODEModel class plays several roles in the
propagation subsystem:

\begin{itemize}
\item It works with a MissionState object to create the propagation vector
\item It coordinates the data mapping between the objects that are propagated and the propagation
vector
\item It handles the superposition tasks when the differential equations defining propagation
consist of multiple components
\end{itemize}

\noindent The ODEModel class is described more fully in Section~\ref{section:TheODEModel}.
Scripting for ODEModel objects depends on context.  For orbit propagation, the ODEModel is scripted
as a ForceModel component, using this syntax:

\begin{quote}
\begin{verbatim}
Create ForceModel forces;
forces.CentralBody = Earth;
forces.PrimaryBodies = {Earth};
forces.Drag = MSISE90;
forces.SRP = On;
forces.ErrorControl = RSSStep;
forces.GravityField.Earth.Degree = 4;
forces.GravityField.Earth.Order = 4;
forces.GravityField.Earth.PotentialFile = 'JGM2.cof';
forces.PointMasses = {Sun, Luna, Mercury, Venus, Mars};

Create Propagator prop;
prop.Type = PrinceDormand78;
prop.FM = forces;
\end{verbatim}
\end{quote}

\noindent The forces that are included in the differential equations of motion are defined by
specifying the desired elements and, when needed, properties of those elements.  The resulting
ODEModel is assigned to the integrator by assigning its name to the FM property of the Propagator.

\subsubsection{Scripting Analytic Propagators}

This section is TBD, based on requirements and design for analytic propagators when they are added
to GMAT.

\subsubsection{Scripting Precalculated Propagators}

\textit{Detailed information for this section is TBD, based on requirements and design for the SPICE
file component and discussions of other ephemeris based propagators planned for GMAT.  However, the
scripting discussion is underway, and included here, subject to extensive revision.}

Propagation based on precalculated data is used in GMAT to propagate data for spacecraft, celestial
bodies not modeled elsewhere, and other elements that are described using file-based time-indexed
data.

Scripting for the precalculated propagators is similar to that for the numerical integrators.  The
precalculated propagators identify the propagator type as one of the supported file based propagator
types, and identify a file -- the file containing the ephemeris data -- rather than a force model.
An example of the setup for a SPICE file based precalculated propagator for the Clementine mission
is given here:

\begin{quote}
\begin{verbatim}
Create Spacecraft Clementine
Clementine.EphemID = -40      % Clementine's NAIF ID

Create Asteroid Geographos    % Asteroid support is a future enhancement;
                              % use a Spacecraft with the current code base
Geographos.EphemID = 2006513  % No clue about the real NAIF ID for Geographos...

Create Propagator prop
prop.type = SPICE
prop.StepSize = 300    % seconds
prop.Ephemeris = DSPSE.SPK        % Ephem containing Clementine and Geographos
prop.Ephemeris = SolarBodies.SPK  % Second ephem used to add vectors together
\end{verbatim}
\end{quote}

\noindent Propagation with a SPICE file propagator is handled identically to that performed using
other propagators.  For the asteroid encounter phase of the mission above, a user could script the
propagaton like this:

\begin{quote}
\begin{verbatim}
Create Variable i
For i = 0 : 2000
   Propagate prop(Clementine, Geographos)
EndFor
\end{verbatim}
\end{quote}

\noindent One constraint imposed in GMAT is that for a single ephemeris based propagator, each
object that is propagated must be contained in the same ephemeris file.  That means that in the
example above, the ephemeris for both the Clementine spacecraft and the asteroid Geographos must
exist in the SPICE file DSPSE.SPK.  For this case, the same result could be achieved with separate
ephemerides for the spacecraft and asteroid with this scripting:

\begin{quote}
\begin{verbatim}
Create Spacecraft Clementine
Clementine.EphemID = -40   % Clementine's NAIF ID

Create Asteroid Geographos    % Asteroid support is a future enhancement;
                              % use a Spacecraft with the current code base
Geographos.EphemID = 2006513  % No clue about the real NAIF ID for Geographos...

Create Propagator prop
prop.type = SPICE
prop.StepSize = 300    % seconds
prop.Ephemeris = Clementine.SPK   % Ephem containing Clementine
prop.Ephemeris = SolarBodies.SPK  % Second ephem used to add vectors together

Create Propagator geogProp
geogProp.type = SPICE
geogProp.StepSize = 300    % seconds
geogProp.Ephemeris = Asteroids.SPK    % Ephem containing Geographos
geogProp.Ephemeris = SolarBodies.SPK  % Second ephem used to add vectors together

Create Variable i
For i = 0 : 2000
   Propagate Synchronized prop(Clementine) geogProp(Geographos)
EndFor
\end{verbatim}
\end{quote}

\noindent Finally, you may have noted that a second ephemeris source is identified for the SPICE
propagator.  This option is scripted in these examples to allow conversion of the epheris data for
propagated objects to other bodies in the model.  For example, the ephemeris for Geographos is
likely to be calculated with respect to the Sun.  GMAT may need Earth-centered states, so the SPICE
propagator needs to load the SPICE kernel that describes the Earth's location with respect to the
Sun in order to add the position vectors together to build the state vector.

The following chapters provide details of the propagation subsystem components.
Chapter~\ref{chapter:PropagatorStates} describes the PropVector and MissionState classes, and
includes descriptions of the data mapping for vector elements and diagrams describing hte layout of
the PropVector data for single and multiple objects.  Chapter~\ref{chapter:Propagators} describes
the design of the propagator classes.  The commands that control the propagation subsystem are
described in Chapters~\ref{chapter:Commands} and~\ref{chapter:SpecificCommands}.

% \subsection{The Equations of Motion}
%
% \subsection{Division of Labor: Integrators and Forces}
%
% \section{Integrators}
%
% \section{\label{section:ForceModelOverview}The GMAT Force Model}
%
% \subsection{The PhysicalModel Class}
%
% \subsection{The ForceModel Class}
%
% \subsubsection{Adding and Removing Forces}
%
% \subsection{Applying Forces to Spacecraft}
%
% \section{The State Vector}
%
% \section{The PropSetup Container}


\chapter{Basics}\label{ch:basics}
\input{basics/point.tex}
\input{basics/transfo.tex}
\input{basics/products.tex}
\input{basics/line.tex}
\input{basics/segment.tex}
\input{basics/polygon.tex}
\input{basics/circle.tex}


\subsection{Applying Models to New York City}

So what about applying the same models to climate data of a different city? We consider to use the weather data from New York City in 2017 as test data with the models trained by Seattle data. The same data preprocessing approach is followed as what we did on Seattle data. After that, we only apply SVM and random forest here as they have better performance than $k$-NN. The confusion matrices are shown respectively in Table \ref{ny1} and Table \ref{ny2}.

For random forest, the Type \uppercase\expandafter{\romannumeral1} error is significantly higher. Although the Type \uppercase\expandafter{\romannumeral2} error is lower, it is actually due to the model classified to many days into rain days mistakenly. The model predicted 289 days that rain in 2017 and the average raining day in New York based on historical record is 122. Overall, this model trained with Seattle data is not suitable to predict precipitation in New York. Again, we apply SVM with whole Seattle data as training set and test on New York. As shown in Table \ref{ny2}, the Type \uppercase\expandafter{\romannumeral1} error is significantly higher as well as the Type \uppercase\expandafter{\romannumeral2} error. Overall those two model performed badly in the New York data.

\begin{table}[h]
\setlength{\belowcaptionskip}{5pt}
\caption{Confusion Matrix for New York, Random Forest}
\label{ny1}
\centering
\renewcommand\arraystretch{1.5}
\begin{tabular}{rrrrr}
\hline
\hline
 & & \multicolumn{2}{c}{True Condition} & \\
\hline
 & & Non-Precipitation & Precipitation & \\
\cline{1-4}
\multirow{2}{*}{Prediction} & {Non-Precipitation} & 62 & 13 & \\
\cline{2-4}
&Precipitation&176&113&\\
\hline
&Error Rate & 0.7394 & 0.1031 & 0.5192\\
\cline{2-5}
& & Type \uppercase\expandafter{\romannumeral1} & Type \uppercase\expandafter{\romannumeral2} & Overall\\
\hline
\end{tabular}
\end{table}

\begin{table}[h]
\setlength{\belowcaptionskip}{5pt}
\caption{Confusion Matrix for New York, SVM}
\label{ny2}
\centering
\renewcommand\arraystretch{1.5}
\begin{tabular}{rrrrr}
\hline
\hline
 & & \multicolumn{2}{c}{True Condition} & \\
\hline
 & & Non-Precipitation & Precipitation & \\
\cline{1-4}
\multirow{2}{*}{Prediction} & {Non-Precipitation} & 96 & 45 & \\
\cline{2-4}
&Precipitation&142&81&\\
\hline
&Error Rate & 0.596 & 0.357 & 0.5137\\
\cline{2-5}
& & Type \uppercase\expandafter{\romannumeral1} & Type \uppercase\expandafter{\romannumeral2} & Overall\\
\hline
\end{tabular}
\end{table}

There are several possible explanations for this. First, New York located on the east coast may have totally different meteorological environment compares to Seattle. So the variables and the threshold of those variables that matter in prediction may change. Second, New York has a rather imbalanced situation which has more days without precipitation (in fact, 126 days with precipitation, and 238 days without).


\section{Substitute Vectors}
\label{sec:lm}

In this study, we predict the part of speech of a word in a given
context based on its substitute vector.  The dimensions of the
substitute vector represent words in the vocabulary, and the entries
in the substitute vector represent the probability of those words
being used in the given context.  Note that the substitute vector is a
function of the context only and is indifferent to the target word.
This section details the choice of the data set, the vocabulary and
the estimation of substitute vector probabilities.

% what is the test data
The Wall Street Journal Section of the Penn Treebank \cite{treebank3}
was used as the test corpus (1,173,766 tokens, 49,206 types).
% what is the tag set
The treebank uses 45 part-of-speech tags which is the set we used as
the gold standard for comparison in our experiments.
% what is the LM training data
%Train => 5181717 126019973 690121813
To compute substitute probabilities we trained a language model using
approximately 126 million tokens of Wall Street Journal data
(1987-1994) extracted from CSR-III Text \cite{csr3text} (we excluded
the test corpus).
% how is the language model trained
We used SRILM \cite{Stolcke2002} to build a 4-gram language model with
Kneser-Ney discounting.
% what is the vocabulary
Words that were observed less than 20 times in the language model
training data were replaced by \textsc{unk} tags, which gave us a
vocabulary size of 78,498.
% perplexity
The perplexity of the 4-gram language model on the test corpus is 96.

% how are the substitutes computed
It is best to use both left and right context when estimating the
probabilities for potential lexical substitutes.  For example, in
\emph{``He lived in San Francisco suburbs.''}, the token \emph{San}
would be difficult to guess from the left context but it is almost
certain looking at the right context.  We define $c_w$ as the $2n-1$
word window centered around the target word position: $w_{-n+1} \ldots
w_0 \ldots w_{n-1}$ ($n=4$ is the n-gram order).  The probability of a
substitute word $w$ in a given context $c_w$ can be estimated as:
\begin{eqnarray}
  \label{eq:lm1}P(w_0 = w | c_w) & \propto & P(w_{-n+1}\ldots w_0\ldots w_{n-1})\\
  \label{eq:lm2}& = & P(w_{-n+1})P(w_{-n+2}|w_{-n+1})\nonumber\\
  &&\ldots P(w_{n-1}|w_{-n+1}^{n-2})\\
  \label{eq:lm3}& \approx & P(w_0| w_{-n+1}^{-1})P(w_{1}|w_{-n+2}^0)\nonumber\\
  &&\ldots P(w_{n-1}|w_0^{n-2})
\end{eqnarray}
where $w_i^j$ represents the sequence of words $w_i w_{i+1} \ldots
w_{j}$.  In Equation \ref{eq:lm1}, $P(w|c_w)$ is proportional to
$P(w_{-n+1}\ldots w_0 \ldots w_{n+1})$ because the words of the
context are fixed.  Terms without $w_0$ are identical for each
substitute in Equation \ref{eq:lm2} therefore they have been dropped
in Equation \ref{eq:lm3}.  Finally, because of the Markov property of
n-gram language model, only the closest $n-1$ words are used in the
experiments.

Near the sentence boundaries the appropriate terms were truncated in
Equation \ref{eq:lm3}.  Specifically, at the beginning of the sentence
shorter n-gram contexts were used and at the end of the sentence terms
beyond the end-of-sentence token were dropped.

For computational efficiency only the top 100 substitutes and their
unnormalized probabilities were computed for each of the 1,173,766
positions in the test set\footnote{The substitutes with unnormalized
  log probabilities can be downloaded from \mbox{\url{http://goo.gl/jzKH0}}.
  For a description of the {\sc fastsubs} algorithm used to generate
  the substitutes please see \mbox{\url{http://arxiv.org/abs/1205.5407v1}}.
  {\sc fastsubs} accomplishes this task in about 5 hours, a naive
  algorithm that looks at the whole vocabulary would take more than 6
  days on a typical 2012 workstation.}.  The probability vectors for
each position were normalized to add up to 1.0 giving us the final
substitute vectors used in the rest of this study.


% -*- latex -*-

% This file can be used for troubleshooting problems.

% Adjust this file to your liking and then build using:
%    make mathics-test.pdf
\chapter{Test Chapter}
\chapterstart
\chaptersections
\section*{TestSection}
\sectionstart

Here is what we want to test:

\begin{asy}
if(!settings.multipleView) settings.batchView=false;
settings.tex="xelatex";
defaultfilename="mathics-test-1";   % Note that filename line has been altered!
if(settings.render < 0) settings.render=4;
settings.outformat="";
settings.inlineimage=true;
settings.embed=true;
settings.toolbar=false;
viewportmargin=(2,2);

% Copy stuff here from mathics-xxx.asy
% The below is from the "Darker" section around mathics-114.asy

import three;
import solids;
size(6.6667cm, 6.6667cm);
currentprojection=perspective(2.6,-4.8,4.0);
currentlight=light(rgb(0.5,0.5,1), specular=red, (2,0,2), (2,2,2), (0,2,2));
// Sphere3DBox
draw(surface(sphere((0, 0, 0), 1)), rgb(0.0,0.6666666666666667,0.0));
draw(((-1,-1,-1)--(1,-1,-1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((-1,1,-1)--(1,1,-1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((-1,-1,1)--(1,-1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((-1,1,1)--(1,1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((-1,-1,-1)--(-1,1,-1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((1,-1,-1)--(1,1,-1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((-1,-1,1)--(-1,1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((1,-1,1)--(1,1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((-1,-1,-1)--(-1,-1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((1,-1,-1)--(1,-1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((-1,1,-1)--(-1,1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
draw(((1,1,-1)--(1,1,1)), rgb(0.4, 0.4, 0.4)+linewidth(1));
\end{asy}
\sectionend
\chapterend


\chapter{Limits \& Continuity}

Limits are a way of describing what happens to a function $f(x)$ as $x$ gets arbitrarily close to a value from some direction (positive or negative).
This allows us not only to deal with "holes" in some functions but describe some of the building blocks of calculus, namely the derivative.

\input{./limits_continuity/limit_definition.tex}
\input{./limits_continuity/limit_properties.tex}
\input{./limits_continuity/left_right_hand_limits.tex}
\input{./limits_continuity/sandwich_theorem.tex}
\input{./limits_continuity/infinite_limits.tex}
\input{./limits_continuity/continuity_definition.tex}
\input{./limits_continuity/discontinuity_types.tex}
\input{./limits_continuity/continuity_properties.tex}
\input{./limits_continuity/intermediate_value_theorem.tex}


\documentclass[a4paper, 11pt]{article}
\input{../preamble.tex}

\DeclareMathOperator{\N}{\mathbb{N}}
\DeclareMathOperator{\R}{\mathbb{R}}
\DeclareMathOperator{\Q}{\mathbb{Q}}
\DeclareMathOperator*{\Z}{\mathbb{Z}}

\title{Existance of Nth Roots and Real Powers}

\begin{document}

\section{Difference of Powers formula}
\section{Bernoulli Inequality}
\section{Every Real number has $n^{th}$ root for all natural numbers}
\end{document}


\documentclass[11pt,a4paper]{report}
\usepackage{amsmath,amsfonts,amssymb,amsthm,epsfig,epstopdf,titling,url,array}
\usepackage{enumitem}
\usepackage{changepage}
\usepackage{graphicx}
\usepackage{caption}
\theoremstyle{plain}
\newtheorem{thm}{Theorem}[section]
\newtheorem{lem}[thm]{Lemma}
\newtheorem{prop}[thm]{Proposition}
\newtheorem*{cor}{Corollary}
\theoremstyle{definition}
\newtheorem{defn}{Definition}[section]
\newtheorem{conj}{Conjecture}[section]
\newtheorem{exmp}{Example}[section]
\newtheorem{exercise}{Exercise}[section]
\theoremstyle{remark}
\newtheorem*{rem}{Remark}
\newtheorem*{note}{Note}
\def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]}
\let\endchangemargin=\endlist
\begin{document}

\section*{Problem}
The function graphed below is a polynomial with only real roots. What is the function?
\begin{figure}[h!]
  \includegraphics[width=4in]{poly.png}
  {\caption*{}}
  \label{}
\end{figure}
\section*{Bonus}
Find all roots of the polynomial $x^3-x^2+x-1$.  Explain why this example illustrates how the ``only real roots'' assumption simplifies the answer to the first part.

\newpage
\section*{Solution}
From the graph, you can see that the polynomial has roots at $x=-1$, $x=0$, and $x=1$.  Since the polynomial has only real roots, its equation must be $p(x) = (x - - 1)(x - 0)(x - 1) = x^3 - x$.

\section*{Bonus Solution}
$x^3 - x^2 + x - 1 = (x^2 + 1)(x - 1) = (x - i)(x -- i)(x - 1)$
This polynomial has roots at $\pm i$ in the complex plane and at the real number 1.  Its graph shows only the real root at $x = 1$.  So in general, just multiplying together $(x - r_i)$ where the $r_i$ are the real roots (what you can see in its graph) will not always give you the equation of the polynomial.  If you have all of the complex roots, it will always work (up to a constant). A beautiful fact about the Complex numbers is that they make up what is called an \textit{algebraically closed field} with the consequence that every polynomial of degree $n$ with real or complex coefficients can be factored into a product like the above, giving $n$ roots.  Some may be ``repeated'' as in $p(x) = x^2 = (x - 0)(x - 0)$.

\section*{Bonus Bonus Solution}
There is actually an error in the solution above. The graph is in fact the graph of the given function, but that can't actually be established from the information given.  If $p(x)$ is a polynomial and $c$ is a non-zero constant, the $r(x) = c \mathord{\cdot} p(x)$ is another polynomial with the same roots.  So for example, $2x^3 - 2x$ would also have the same roots as the polynomial above.

\end{document}


\documentclass[a4paper]{report}

\usepackage[margin=3.0cm]{geometry}
\usepackage{amsmath}
\usepackage[pdftex]{graphicx}
%\usepackage{graphics}
\usepackage{subfig}
\setlength\parindent{0pt}
%\usepackage{natbib}           % required for bibliography

\title{HPIPM reference guide}
\author{Gianluca Frison}

\begin{document}

\maketitle
\tableofcontents

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Introduction}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

HPIPM, which stands for High-Performance Interior Point Method, is a library providing a collection of quadratic programs (QP) and routines to manage them.
Aim of the library is to provide both stand-alone IPM solvers for the QPs and the building blocks for more complex optimization algorithms.

At the moment, three QPs types are provided: dense QPs, optimal control problem (OCP) QPs, and tree-structured OCP QPs.
These QPs are defined using C structures.
HPIPM provides routines to manage the QPs, and to convert between them.

HPIPM is written entirely in C, and it builds on top of BLASFEO~\cite{Frison2018}, that provides high-performance implementations of basic linear algebra (LA) routines, optimized for matrices of moderate size (as common in embedded optimization).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Dense QP}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The dense QP is a QP in the form
\begin{align*}
\min_{v,s} & \quad \frac 1 2 \begin{bmatrix} v \\ 1 \end{bmatrix}^T \begin{bmatrix} H & g \\ g^T & 0 \end{bmatrix} \begin{bmatrix} v \\ 1 \end{bmatrix} + \frac 1 2 \begin{bmatrix} s^l \\ s^u \\ 1 \end{bmatrix}^T \begin{bmatrix} Z^l & 0 & z^l \\ 0 & Z^u & z^u \\ (z^l)^T & (z^u)^T & 0 \end{bmatrix} \begin{bmatrix} s^l \\ s^u \\ 1 \end{bmatrix} \\
\text{s.t.} & \quad A v = b, \\
& \quad \begin{bmatrix} \underline v \\ \underline d \end{bmatrix} \leq \begin{bmatrix} J_{b,v} \\ C \end{bmatrix} v + \begin{bmatrix} J_{s,v} \\ J_{s,g} \end{bmatrix} s^l, \\
& \quad \begin{bmatrix} J_{b,v} \\ C \end{bmatrix} v - \begin{bmatrix} J_{s,v} \\ J_{s,g} \end{bmatrix} s^u \leq \begin{bmatrix} \overline v \\ \overline d \end{bmatrix}, \\
& \quad s^l\geq \underline s^l, \\
& \quad s^u\geq \underline s^u,
\end{align*}
where $v$ are the primal variables, $s^l$ ($s^u$) are the slack variables of the soft lower (upper) constraints.
The matrices $J_{\dots}$ are made of rows from identity matrices.
Furthermore, note that the constraint matrix with respect to $v$ is the same for the upper and the lower constraints.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{OCP QP}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The OCP QP is a QP in the form
\begin{align*}
\min_{x,u,s} & \quad \sum_{n=0}^N \frac 1 2 \begin{bmatrix} u_n \\ x_n \\ 1 \end{bmatrix}^T \begin{bmatrix} R_n & S_n & r_n \\ S_n^T & Q_n & q_n \\ r_n^T & q_n^T & 0 \end{bmatrix} \begin{bmatrix} u_n \\ x_n \\ 1 \end{bmatrix} + \frac 1 2 \begin{bmatrix} s^l_n \\ s^u_n \\ 1 \end{bmatrix}^T \begin{bmatrix} Z^l_n & 0 & z^l_n \\ 0 & Z^u_n & z^u_n \\ (z^l_n)^T & (z^u_n)^T & 0 \end{bmatrix} \begin{bmatrix} s^l_n \\ s^u_n \\ 1 \end{bmatrix} \\
\text{s.t}  & & \\
     & \quad x_{n+1} = A_n x_n + B_n u_n + b_n, \qquad \qquad \qquad \qquad \qquad \qquad n=0,\dots,N-1, &\\
     & \quad \begin{bmatrix} \underline u_n \\ \underline x_n \\ \underline d_n \end{bmatrix} \leq \begin{bmatrix} J_{b,u,n} & 0 \\ 0 & J_{b,x,n} \\ D_n & C_n \end{bmatrix} \begin{bmatrix} u_n \\ x_n \end{bmatrix} + \begin{bmatrix} J_{s,u,n} \\ J_{s,x,n} \\ J_{s,g,n} \end{bmatrix} s^l_n, \quad \qquad \quad \,n=0,\dots,N, &\\
& \quad \begin{bmatrix} J_{b,u,n} & 0 \\ 0 & J_{b,x,n} \\ D_n & C_n \end{bmatrix} \begin{bmatrix} u_n \\ x_n \end{bmatrix} - \begin{bmatrix} J_{s,u,n} \\ J_{s,x,n} \\ J_{s,g,n} \end{bmatrix} s^u_n \leq \begin{bmatrix} \overline u_n \\ \overline x_n \\ \overline d_n \end{bmatrix} , \qquad \qquad n=0,\dots,N, & & \\
& \quad s^l_n\geq \underline{s}^l_n, \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \,\,\,  n=0,\dots,N, & &\\
& \quad s^u_n\geq \underline{s}^u_n, \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \,\,\,  n=0,\dots,N, & &\\
\end{align*}
where $u_n$ are the control inputs, $x_n$ are the states, $s^l_n$ ($s^u_n$) are the slack variables of the soft lower (upper) constraints
and $\underline{s}^l_n$ and $\underline{s}^u_n$ are the lower bounds on lower and upper slacks, respectively.
The matrices $J_{\dots,n}$ are made of rows from identity matrices.
Note that all quantities can vary stage-wise.
Furthermore, note that the constraint matrix with respect to $u$ and $x$ is the same for the upper and the lower constraints.

%%%%%%%%%%%%%%%%
\section{QP dimensions structure}
%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%
\subsection{Create structure}
%%%%%%%%%%%%%%%%

\begin{verbatim}
int d_ocp_qp_dim_memsize(int N);
\end{verbatim}

\begin{verbatim}
void d_ocp_qp_dim_create(int N, struct d_ocp_qp *qp, void *memory);
\end{verbatim}

%%%%%%%%%%%%%%%%
\subsection{Populate structure}
%%%%%%%%%%%%%%%%

Once created, an OCP QP dimmensions structure can be populated using the global setter routine
\begin{verbatim}
void d_ocp_qp_dim_set_all(int *nx, int *nu,
    int *nbx, int *nbu, int *ng,
    int *nsbx, int *nsbu, int *nsg,
    struct d_ocp_qp_dim *dim);
\end{verbatim}
which is useful when all structure fields have to be populated at onece.

Alternatively, it is possible to set the individual structure fields with the setter routine
\begin{verbatim}
void d_ocp_qp_dim_set(char *field, int *stage, int value,
    struct d_ocp_qp_dim *dim);
\end{verbatim}
where {\tt field} can be one of {\tt nx, nu, nbx, nbu, ng, nsbx, nsbu, nsg}.

%%%%%%%%%%%%%%%%
\section{QP structure}
%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%
\subsection{Create structure}
%%%%%%%%%%%%%%%%

\begin{verbatim}
int d_ocp_qp_memsize(struct d_ocp_qp_dim *dim);
\end{verbatim}

\begin{verbatim}
void d_ocp_qp_create(struct d_ocp_qp_dim *dim, struct d_ocp_qp *qp, void *memory);
\end{verbatim}

%%%%%%%%%%%%%%%%
\subsection{Populate structure}
%%%%%%%%%%%%%%%%

Once created, an OCP QP structure can be populated using the global conversion routine
\begin{verbatim}
void d_ocp_qp_set_all(double **A, double **B, double **b,
    double **Q, double **S, double **R, double **q, double **r,
    int **idxbx, double **lbx, double **ubx,
    int **idxbu, double **lbu, double **ubu,
    double **C, double **D, double **lg, double **ug,
    double **Zl, double **Zu, double **zl, double **zu,
    int **idxs, double **lls, double **lus,
    struct d_ocp_qp *qp);
\end{verbatim}
which is useful when all the structure fileds have to be populated at once.

Alternatively, it is possible to set the individual structure fields with the setter routine
\begin{verbatim}
void d_ocp_qp_set(char *field, int *stage, void *value,
    struct d_ocp_qp *qp);
\end{verbatim}
where {\tt filed} can be one of {\tt A, B, b, Q, S, R, q, r, idxb, lb, ub, Jbx, idxbx, lbx, ubx, Jbu, idxbu, lbu, ubu, C, D, lg, ug, Zl, Zu, zl, zu, idxs, lls, lus}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \chapter{Tree OCP QP}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%\bibliographystyle{plain}
%\bibliography{biblio}             % bib file to produce the bibliography

\begin{thebibliography}{1}

\bibitem{Frison2018}
G.~Frison, D.~Kouzoupis, T.~Sartor, A.~Zanelli, and M.~Diehl.
\newblock {BLASFEO}: Basic linear algebra subroutines for embedded
  optimization.
\newblock {\em ACM Transactions on Mathematical Software (TOMS)}, 2018.
\newblock (accepted).

\end{thebibliography}

\end{document}


\section{Efficiency Analysis}

Here we try to analyze the speedup of Parareal analytically, and then through
scalability studies conducted on the supercomputer Prince, here at NYU.

\subsection{Theoretical Results}

We try to directly estimate the speedup of the Naive OpenMP implementation. We
denote our speedup $S$ by: (\cite{fieldstalk} slide 17)
\begin{equation*}
  S = \frac{\text{Time taken by fine solver}}{\text{Parareal time}}
\end{equation*}
Let the time taken by the fine and coarse solvers respectively be denoted by
$T_f$ and $T_g$. Furthermore, suppose we have the optimal amount of processors
$P$, such that $\Delta t = T/P$, where $T$ is the time to integrate up to.

Considering this, each step parareal makes a predictor and corrector
computation. The predictor is just a run of the coarse solver, costing $PT_g$.
The corrector has two parts, the fine and coarse portion. The fine portion is
parallelized optimally, and therefore costs just $T_f$, instead of $PT_f$.  In
addition, the coarse portion of the corrector takes advantage of the
\textit{first same as last} property, and doesn't have to be counted sepreately.
Recall, also, before the parareal iteration, we make an initial guess using one
coarse solve, $PT_g$. Therefore, for a parareal with $k$ iterations it would
have speedup:
\[
  S = \frac{PT_f}{PT_g + k(PT_g + T_f)}
  = \frac{1}{\frac{T_g}{T_f} + k(\frac{T_g}{T_f} + \frac{1}{P})}
  = \frac{1}{\frac{T_g}{T_f}(1+k) + \frac{k}{P}}
\]
Notice, under the limit:
\[
  \lim_{P \to \infty} S = \frac{1}{\frac{T_g}{T_f}(1+k)} = \frac{T_f}{T_g(1+k)}
\]
Supposing we have the idealized case of $k = 1$, then we would find that this
would result in speedup $\frac{T_f}{2T_g}$, which, given that $T_g << T_f$,
could be significant. However, in practice, $k$ has to be taken to be large
enough so that $S$ isn't too great. This is one of the first indicators we have
of why $T_g << T_f$ must be true.

\subsection{Scalability}

Since this algorithm is designed to take our numerical ODE techniques to a high
performance computing context, we would like to perform the standard scalability
tests using this algorithm, measuring how it scales to massively parallel
hardware.

All tests are performed on a single Prince node, having requested 28 processors.
We would be worried about other processes running on this node if this problem
was memory bound, however since the fine computation is performed in place and
the coarse operator has few points, the computation is compute bound.

\subsubsection{Strong Scaling Study}

A strong scaling study consists of fixing a total problem size, and increasing
the number of processors necessary. See figure \ref{fig:strong_scaling} for our
performed analysis.
\begin{figure}[!htb]
  \centering
  \includegraphics[width=.8\textwidth]{./resources/strong_scaling}
  \caption{This is a strong scaling study performed on the ODE $u' = u, u(0) =
  1$ integrating on time scale $[0, 4]$. Here we divided our time scale into
  pieces of size $4/P$, and then ran Parareal restricting the number of
  processors. Notice how the time taken doesn't improve from $14 \to
  28$.}\label{fig:strong_scaling}
\end{figure}
Note that figure \ref{fig:strong_scaling} presents a very interesting, and
dismal, phenomena. Notice how from processors $14 \to 27$ the speedup recieved
is negligable from the previous. Why does this occur? Recall that the optimal
number of processors for parareal iteration is the number of coarse steps, so
that we can process all of the $\fine$ integrators in one step. However,
supposing we have one less than optimal, the entire compute time is bounded by
that one thread that has to process two $\fine$ integrators. This is
troublesome.
In fact, suppose we desire to cut our time domain $[0, T]$ into $2N$ pieces.
Then any computation with number of processors $P$ taking value $N \to 2N-1$
will have the same computation time, theoretically.

Notice, however, that the general speedup does follow the theoretical linear
speedup we hope to see with $P$ processors, which is great.

\subsubsection{Weak Scaling Study}

The weak scaling study examines how the runtime increases as we increase the
amount of processors, working on a similar sized problem. See figure
\ref{fig:weak_scaling} for the results.
\begin{figure}[!htb]
  \centering
  \includegraphics[width=.8\textwidth]{./resources/weak_scaling}
  \caption{}\label{fig:weak_scaling}
\end{figure}
We see that the Weak Scaling of parareal is alright, seeming to stay roughly
constant, or atleast sublinear as we increase the number of processors doing
equal amounts of work. Note, this can only be true if $T_g << T_f$, otherwise we
will be bounded by Amadahl's law. When we increase the amount of processors
working, what happens is that the fine portion is computed in the same amount of
time, but the amount of points in general increase. This implies that the serial
portion of the computation will increase linearly in the number of points, so if
the course computation is not very cheap with respect to the fine computation,
we will lose the weak scaling performance.


\documentclass{article}  %Need this.

\usepackage{amsmath,amsthm,amssymb}

\newtheorem*{thm}{Theorem}
\newtheorem*{cnj}{Conjecture}
\newtheorem*{lem}{Lemma}
\newtheorem*{cor}{Corollary}
\newtheorem*{prop}{Proposition}

\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\R}{\mathbb{R}}

\title{Proof Portfolio Problem \#  1--  Example--}
\author{--Dr. Keough --}
\date{}

\begin{document}
\maketitle  %This will add the title, author, and date located above

Here's the problem: You work for a cell phone company which has just invented a new cell phone protector and wants to advertise that it can be dropped from the $n^{th}$ floor without breaking.
If you are given 2 phones and a 100 story building, how do you guarantee you know the highest floor it won't break with the smallest number of trial drops?

You might want to try this problem before you read the proof. On the next page I include my scratchwork.

\newpage
...

\newpage

\begin{thm}
If there are 2 cell phones and a 100 story building available then one can test dropping cell phones from 14 stories and find the maximum number of stories the cell phone can be dropped from without breaking.
\end{thm}

\begin{proof}
We will describe an algorithm for needing only to drop from 14 stories. Our first step will be to drop the first phone from the $14^{th}$ floor. If the phone breaks from this floor, we will test floors $1$ through $13$, in order, until the cell phone breaks, giving us up to a total of $14$ drops. If the phone does not break on a drop from the $14^{th}$ floor, we will drop the phone from the $27^{th}$ floor. If the phone breaks from the $27^{th}$ floor, we'll need to test floors $15$ through $26$. This is 12 more floors, in addition to the drop from the $14^{th}$ and the drop from the $27^{th}$ again giving us $14$ drops.

We'll continue this process by dropping from floors $39, 50, 60, 69, 77, 84, 90, 95,$ and finally $99$. If we make it all the way to the $99^{th}$ floor then we will have done $11$ drops. In any other case, we will do $14$ total drops as seen by the following cases, which consider the first floor the phone breaks on from the list $39, 50, 60, 69, 77, 84, 90, 95$:
	\begin{itemize}
	\item The phone first breaks on the $39^{th}$ floor: In this case we test floors $14, 27, 39$ and floors $28-38$ giving $14$ total drops.
	\item The phone first breaks on the $50^{th}$ floor: In this case we test floors $14, 27, 39, 50$ and floors $40-49$ giving $14$ total drops.
	\item The phone first breaks on the $60^{th}$ floor: In this case we test floors $14, 27, 39, 50, 60$ and floors $51-59$ giving $14$ total drops.
	\item The phone first breaks on the $69^{th}$ floor: In this case we test floors $14, 27, 39, 50, 60, 69$ and floors $60-68$ giving $14$ total drops.
	\item The phone first breaks on the $77^{th}$ floor: In this case we test floors $14, 27, 39, 50, 60, 69, 77$ and floors $70-76$ giving $14$ total drops.
	\item The phone first breaks on the $84^{th}$ floor: In this case we test floors $14, 27, 39, 50, 60, 69, 77, 84$ and floors $78-83$ giving $14$ total drops.
	\item The phone first breaks on the $90^{th}$ floor: In this case we test floors $14, 27, 39, 50, 60, 69, 77, 84,90$ and floors $85-89$ giving $14$ total drops.
	\item The phone first breaks on the $95^{th}$ floor: In this case we test floors $14, 27, 39, 50, 60, 69, 77, 84,90,95$ and floors $91-94$ giving $14$ total drops.
	\end{itemize}

Thus we see that we can figure out the maximum floor where the phone breaks in $14$ total drops.
\end{proof}

\newpage

\section*{A Couple of Notes}

\begin{enumerate}
\item Note the proof does not claim that there isn't a better solution. Don't feel like you need to do every piece of a problem. Figure out something cool, and try to prove it.
\item On that note it's best to check your conjectures with me. I can help you if they are too hard, too obvious, too false, and also brainstorm proof ideas.
\item Remember that  a proof, at its heart, is just an explanation that other mathematicians believe. Don't feel like you need to adhere to strictly to a specific proof technique. Give a convincing explanation.
\item Pick a problem you like and start early so that this can be fun and not something you're worried about in the last week of the semester.
\end{enumerate}

\end{document}


\appendix
\chapter*{Additional figures}
\paragraph{Chapter 4 : Looking into financial data} \textcolor{white}{Below, the cumulated residuals between }
\begin{table}[h]
	\centering
	\begin{tabular}[]{ | p{3cm} || m{9cm} ||}
		\hline
		Stock & Cumulative residuals between $\exp(R_t)$ and $1 + R_t$  \\ \hline
		BNP & 3.22E-15  \\ \hline
		Carrefour & 1.44E-15  \\ \hline
		LVMH & 4.75E-15  \\ \hline
		Sanofi & 1.55E-15  \\ \hline
		Total & 3.86E-15  \\ \hline
	\end{tabular}
		\caption{Cumulative residuals between $\exp(R_t)$ and $1 + R_t$ for the BNP, Carrefour, LVMH, Sanofi, Total stocks}
		\label{tab:cumulativeResidualsStocks}
\end{table}


\chapter{Summary and Discussion}
\label{chapter.summary}

We have first discussed entropy for percolation. Note that percolation
is a probabilistic model and hence Shannon entropy is the only hope if we want to measure entropy
for percolation. To measure the Shannon entropy for percolation we
have defined the cluster picking probability $\mu_i$ that a site is picked at random belongs to
the labelled cluster $i$. It gives entropy which is consistent with the behaviour
of the order parameter. Essentially entropy measures
the degree of disorder while order parameter measures the extent of order. Thus, entropy and order parameter cannot be minimum or maximum
at the same state since the system cannot be in most disordered and most ordered state at the same time.
However, by measuring entropy and order parameter using existing definition for site percolation, we find
that at $p=0$ both order parameter and entropy equal
to zero which is absurd. It demands immediate correction to the definition of entropy and we obliged.
Note that in the bond percolation we occupy bond to connect sites and measure clusters by the
number of sites. In analogy with that we redefine the site percolation as follows. We occupy sites to connect
bonds which are assumed to exist already in the system and measure clusters in terms of the number of bonds. On the other hand, occupation probability in the bond (site) percolation is the fraction of bonds (sites) occupied
in the system. With this new definition we have found the entropy behaves exactly in the same way as
it does in the case of its bond counterpart. Thus the conflict that the system is in ordered and disordered
at the same state is resolved.

The question that arises then is: Do we recover all the known results? To verify
this we obtained all the necessary critical exponents with the new definition for
site percolation. Earlier it was well-known that bond and site percolation belong to the
same universality class regardless of the nature of lattice but have the same dimension.
We have confirmed that bond and redefined site percolation still belong to the same universality class.
Note that scaling theory predicts that the various critical exponents cannot just assume values
independently  rather they are bound by some scaling and hyperscaling relations. One
of the most interesting relations is the Rushbrooke inequality $\alpha+2\beta+\gamma\geq 2$.
Substituting our values of $\alpha=0.906$, $\gamma=0.8543$ and already
known value of $\beta=0.137$ we find $\alpha+2\beta+\gamma=2.0347$.
We can thus conclude that the RI holds almost as equality but marginally greater
than $2$.

 %%%%%%%%%%%% RSBD summary
 Then we have investigated percolation by random sequential ballistic deposition (RSBD) on a square lattice with interaction range upto second nearest neighbors. The critical points $p_c$ and all the necessary critical exponents $\alpha$, $\beta$, $\gamma$, $\nu$ etc. are obtained numerically for each range of interactions. Like  in its thermal counterpart, we find that the critical exponents of RSBD depend on the range of interactions and for a given range of interaction they obey the Rushbrooke inequality. We obtain  the fractal dimension $d_f$ that characterizes the spanning cluster at $p_c$. Our results suggest that the RSBD for each range of interaction belong to a new universality class which is in sharp contrast to earlier results of the only work that exhist on RSBD.

We denote $L_0,L_1,L_2$ for expressing direct, first nearest neighbor and second nearest neighbor interaction respectively. Obviously $L_0$ denotes the regular kind of site percolation where we choose a site randomly with uniform probability and occupy it if it is empty else we skip the step. And $L_1$ is the class where we choose one of the four neighbor to occupy whenever we fail to do $L_0$ but only if the neighbor is empty else we skip the step. Finally in $L_2$ we choose the neighbor in the direction of the second neighbor, which was picked but was not empty, to occupy if it is empty else we skip the step. We have found that for $L_1$ and $L_2$ the exponents $\alpha, \beta, \gamma, \nu$ are consistent and they belong to a universality class respectively.
Note that  we can use new feature of $L_1$ only if the feature of $L_0$ is unavailable, i.e., the selected site is already occupied. Similarly we can use new feature of $L_2$ only if the feature of $L_0$ and $L_1$ is unavailable. Using this in mind we perform simulation and we obtain the critical exponents which agree with the laws of thermodynamics and the Rushbrooke inequality is satisfied in all cases.

\clearpage
\newpage
\section{Results}
Here we list all the exponents found in our investigation of site percolation after redefining it and the RSBD model. Table (\ref{tab:exponents-combined}) Lists all the critical values and exponents as we find them in our exponents. Note that we can get the exponent $a$, for example, from exponent $a/\nu$ simply by dividing them by $1/\nu$ which is shown in table (\ref{tab:rushbrooke}) where we also show that the Rushbrooke inequality is satisfied.
\begin{table}[h]
\centering
\begin{tabular}{|c|c|c|c|c|c|}
	\hline
	Interaction & $p_c$ & $1/\nu$ & $\alpha/\nu$ & $\beta/\nu$ & $\gamma/\nu$ \\ \hline
	$L_0$ & 0.5927 & 0.75  & 0.6799 & 0.103  & 0.64071  \\ \hline
	$L_1$ & 0.5782 & 0.736 & 0.6712 & 0.1026 & 0.6287  \\ \hline
	$L_2$ & 0.5701 & 0.721 & 0.6631 & 0.0982 & 0.6362  \\ \hline
\end{tabular}
\caption{List of combined exponents}
\label{tab:exponents-combined}
\end{table}

\begin{table}[h]
\centering
\begin{tabular}{|c|c|c|c|c|}
	\hline
	Interaction & $\alpha$ & $\beta$ & $\gamma$ & $\alpha+2\beta+\gamma$ \\ \hline
	$L_0$  & 0.906 & 0.137 & 0.8543 & 2.0347   \\ \hline
	$L_1$  & 0.911 & 0.139 & 0.8542 & 2.044   \\ \hline
	$L_2$  & 0.919 & 0.136 & 0.882  & 2.07    \\ \hline
\end{tabular}
\caption{Exponents Satisfying Rushbrooke Inequality}
\label{tab:rushbrooke}
\end{table}

Finally we list all the fractal dimensions, $d_f$, for different interactions in table (\ref{tab:cluster-info}).
\begin{table}[h]
	\centering
	\begin{tabular}{|c|c|c|}
		\hline
		Interaction 	& $d_f$     \\ \hline
		$L_0$ (standard) 	& 91/48     \\ \hline
		$L_0$ (obtained)	& 1.8939   \\ \hline
		$L_1$				& 1.8994      \\ \hline
		$L_2$				& 1.9081     \\ \hline
	\end{tabular}
	\caption{Fractal dimensions for $L_0,L_1,L_2$ respectively.}
	\label{tab:cluster-info}
\end{table}

Here we notice that the critical point decreases as we increase the range of interaction. But the fractal dimension increases. This is reasonable since my occupying nearest and second nearest neighbor we are increasing the change of any individual cluster to grow faster. This is the reason for the $p_c$ value to decrease. But it grows in area not in length average meaning when the spanning cluster appears it will contain more sites and bonds than in regular percolation which is evident from the fractal dimension $d_f$.
All other exponents changes a bit but their shape is not different. That's why change is not visible to the naked eye and it requires a thorough investigation.

\clearpage
\section{Further Research}
	The idea of RSBD molde can be applies to many other lattice structures. Notice that in square lattice we only have four neighbor for a site but in other structures of lattice we may have more or less than four neighbors for one site, e.g. honeycomb lattice, triangular lattice. Here we have only considered interaction up to second nearest neighbor which can be extended to an arbitrary range but it will only be effective if we take the lattice size to be large. Instead of single layer formation we can go for multi-layer formation since in real world if we drop particle (for example sand or grain) in a confined space then it will create multiple layer form. Controlling the velocity parameter at which the particle move to a certain direction can also be an interesting thing to do. We can try to involve particles of different sizes and apply the RSBD model which might give some interesting results.


One of the nice properties of Yampa is that it can handle continuous and discrete signals (which are realised as a type of continuous signal) in one framework. A discrete signal is thought of as a signal whose value is either ``Nothing happened'', or ``Something happened, and here is some information''. More prescisely, a discrete signal is a signal with values of type \hask{Event a} where

\begin{lstlisting}
data Event a = NoEvent | Event a
\end{lstlisting}

Examples of discrete events might be mouseclicks or key presses, or when the ambient temperature exceeds a certain level.

We refer to signals of \hask{Event} type as \emph{event streams}. A signal function of type \hask{SF a (Event b)} is called an \emph{event source}.

A sample of an event stream will take two (types of) value: either \hask{NoEvent} or \hask{Event a} for some value of type \hask{a}. However, events should not occur with infinite density on a signal, and indeed, the sampling rate provides an upper bound for how often events can occur on a signal. If this is a problem, one should implement some buffering for events.

\section{Switching and events}

Events are used to initiate changes in signal functions. Yampa provides combinators which allow us to describe switching in one signal function for another when an event occurs. The most basic of these is \hask{switch}:

\begin{lstlisting}
switch :: SF a (b, Event c) -> (c -> SF a b) -> SF a b
\end{lstlisting}

\noindent The first argument of switch is best thought of as taking the form \hask{(sigFun \&\&\& eventSource)}, where \hask{sigFun :: SF a b} and \hask{eventSource :: SF a (Event c)}. The idea is that \hask{switch} says to use the signal function \hask{sigFun}, until \hask{eventSource} supplies an event. The event will be a sample of type \hask{Event c}, and this wrapped value of type \hask{c} is used to determine what the replacement signal function should be --- this is what the second input of type \hask{c -> SF a b} does. The result is a signal function of type \hask{SF a b}.

In a more intuitive language, \hask{sigFun} is switched for a different signal function when an event occurs. Let's provide an example. We will rewrite our first hello program to use a switch to output a \hask{True} signal when a time of two seconds elapses.

\lstinputlisting[caption={HelloEvent.hs}, label=lst:helloEvent]{./src/HelloEvent.hs}

Our new version is certainly no simpler than the original, but it demonstrates basic use of events. \hask{waitTwo} is still our signal function, but now it makes use of a \hask{switch} to change its output. First \hask{waitTwo} outputs a constant signal false. When an event occurs from \hask{twoElapsed} however, it takes the value from this event (in this case \hask{()}) and uses the passed function to decide what signal function to switch to. In our case, the supplied function has a constant value \hask{constant True}, so upon an event from \hask{twoElapsed}, our signal function changes to take the value \hask{True} constantly.

So now the timing work must be done in \hask{twoElapsed}. This looks much the same as before, except now \hask{time >>> arr (>=2)} is fed into a new signal function \hask{edge}. \hask{edge} has type \hask{SF Bool (Event ())}. \hask{edge} takes an input signal of value type \hask{Bool}, and when the signal value changes (from \hask{True} to \hask{False}, or \hask{False} to \hask{True}), it ouputs an \hask{Event ()} on its output; it otherwise outputs \hask{NoEvent}. In sum then, this means an event is generated when (at least) two seconds have elapsed.

The sum effect is that our program prints its conclusion after waiting for two seconds. Notice that \hask{switch} is written in infix notation. One of the benefits of this is that it the initial signal function occurs on the left of \hask{switch} and the replacement upon an event on the right.

\begin{exercise}
When a signal function \hask{sf} is switched for a signal function \hask{sf'}, the time for \hask{sf'} starts again from zero. In other words signal functions have their own local time. Write a small program to demonstrate this.
\end{exercise}

\subsection{Other event generating functions}

We saw above how to use \hask{edge} to generate an event when a signal changes from \hask{True} to \hask{False} or \hask{False} to \hask{True}, but Yampa provides a few other functions to generate events also.

\begin{lstlisting}
never :: SF a (Event b)
\end{lstlisting}

\noindent simply never generates an event. More useful perhaps is

\begin{lstlisting}
now :: b -> SF a (Event b)
\end{lstlisting}

\noindent which generates an event (with value) of type \hask{b} immediately. In other words \hask{now v} is a signal function which has value \hask{Event v} immediately, and \hask{NoEvent} afterwards.

\begin{lstlisting}
after :: Time -> b -> SF a (Event b)
\end{lstlisting}

\noindent generates an event after some specified period time, with a value of type \hask{b}. \hask{after t v} will generate a signal of value \hask{Event v} in \hask{t} seconds, with a value of \hask{NoEvent} at all other times.

\begin{exercise}
Rewrite listing \ref{lst:helloEvent} to use \hask{after}.
\end{exercise}

\noindent For those situations where having an event reoccur at regular intervals, Yampa provides

\begin{lstlisting}
repeatedly :: Time -> b -> SF a (Event b)
\end{lstlisting}

\noindent \hask{repeatedly t v} generates a signal which has value \hask{NoEvent} at all times except whole number multiplicities of \hask{t} seconds, at which point the signal has value \hask{Event v}. Zero time is when \hask{repeatedly} starts running.

Note that \hask{edge} produces an \hask{Event ()} signal. Sometimes we might want to attach or force an event to take on certain value. This is what the function \hask{tag} is for.

\begin{lstlisting}
tag :: Event a -> b -> Event b
\end{lstlisting}

\noindent In fact since \hask{Event} is a functor (it is isomorphic to \hask{Maybe}), \hask{tag} can be defined by

\begin{lstlisting}
tag e v = fmap (const v) e.
\end{lstlisting}.

\subsection{More switches}

\subsection{Other ways of interacting with events}
Switches are the main way in which event signals interact with other signals (or discrete signals interact with continuous signals if you prefer), but there are other ways we can interact with events.

\begin{lstlisting}
hold :: a -> SF (Event a) a
\end{lstlisting}

\noindent \hask{hold v} initially takes on the value \hask{v}, but when an event occurs on the input signal, it switches to the value wrapped in the event. In other words \hask{hold v} outputs the value wrapped by the previous event, and if no event has previously occured, it outputs value \hask{v}.

Instead of changing a value, we also have the possibility of transformation.

\begin{lstlisting}
accum :: a -> SF (Event (a -> a)) (Event a)
\end{lstlisting}

\noindent \hask{accum v} receives a stream of events of type \hask{a->a}, and upon receiving an event, takes the wrapped function and applies it to the value of the previous event which was dispatched, and \hask{v} if none exists. Since it is natural to want to generate a signal consisting of just the value and not an event, Yampa also provides \hask{accumHold}:

\begin{lstlisting}
accumHold :: a -> SF (Event (a->a)) a
accumHold v = accum v >>> hold v
\end{lstlisting}

\noindent This is useful for creating counters and other similar devices.


\chapter{Paper Errata}\label{app:paperErrata}
This appendix shows some errors that appeared in the published papers.
\\
\subsubsection{Real-Time Implementation of an Elasto-Plastic Friction Model applied to Stiff Strings using Finite Difference Schemes \citeP[C]}
\begin{itemize}
    \item The authors in reference [15] are a bit shuffled.
\end{itemize}
\subsubsection{Real-time Implementation of a Physical Model of the Tromba Marina \citeP[D]}
\begin{itemize}
    \item The minus sign in Eq. (28) (and thus Eqs. (31) and (35)) should be a plus sign.
    \item $\sigma_{1,\text{s}}$ in Eq. (21) should be $\sigma_{1,\text{p}}$
    \item The unit of the spatial Dirac delta function $\delta$ should be m$^{-1}$
\end{itemize}
%
\subsubsection{DigiDrum: A Haptic-based Virtual Reality Musical Instrument and a Case Study \citeP[F]}
\begin{itemize}
    \item $\sigma_0$ and $\sigma_1$ should be multiplied by $\rho H$ in order for the stability condition to hold.
    \item stability condition is wrong. Should be:
    \begin{equation}
        h \geq \sqrt{c^2k^2 + 4\sigma_1k + \sqrt{(c^2k^2+4\sigma_1k)^2 + 16\kappa^2k^2}}
    \end{equation}
    \item Unit for membrane tension is N/m.
\end{itemize}
% %
% \subsubsection{Dynamic grids \citeP[G]}
% \begin{itemize}
%     \item Reference in intro for `recently gained popularity' should go to \cite{Bilbao2019CMJa} \\
%     \textit{Note: not really an error, but should be changed before resubmission}
% \end{itemize}


% !TEX root = ../main.tex
\section{Introduction}
In statistical physics, building a model corresponding to a physical system is useful: comparing how the model's predictions differ from experiment can be used to understand how a system behaves. For computing properties corresponding to a model's behavior, the normalization constant of the model's Boltzmann distribution in \Cref{eq:boltzmann} is a central quantity. The partition function can be used to derive properties of physics models that can be measured in experimental realizations, such as specific heat or magnetization. Such properties of models can be compared to experimental values, which can inform how a model might be improved to better mirror reality. But the partition function is intractable for many probabilistic models of interest, as described in \Cref{ch:background}.

One workaround to the problem of an intractable partition function is to use an approximate inference algorithm such as \gls{mcmc}. \gls{mcmc} relies on sampling likely configurations of a system and does not require calculating the partition function. In theory, these samples will be draws from the probability model of interest~\citep{metropolis1953equation,andrieu2003an-introduction}. For example, samples from the Boltzmann distribution can be used to approximate physical quantities derived from the probability model, such as specific heat.

However, with limited computation \gls{mcmc} has limitations. This method requires practitioners to use convergence diagnostics~\citep{brooks1998general} to assess whether samples from the algorithm are independent. Scalable \gls{mcmc} requires careful consideration. While some scalable versions of \gls{mcmc} have been developed~\citep{neal2011mcmc,welling2011bayesian}, they are biased samplers that may not have guaranteed convergence to samples from the probability model of interest. This is similar to how in \gls{vi} performance must often be assessed empirically. But in comparison to \gls{vi}, unless a model-specific algorithm has been developed~\citep{wolff1989comparison}, generic \gls{mcmc} methods do not readily scale to large numbers of random variables.
% - describe the contributions of our paper: (1) show that
% gibbs-bogoliubov-feynman inequality is equivalent to variational inference.
% (1) show how the reinforcement learning approach of wu (2019) is more
% naturally expressed in terms of variational inference. (2) show that a modern
% tool of variational inference, HVMs, is useful and improves on wu (2019) by
% scaling to larger system sizes.

In \Cref{ch:background} we showed that the machine learning framework of \gls{vi} is equivalent to the \acrlong{gbf} variational principle. This has allowed practitioners to study statistical physics models using many variational approximations, including \glspl{van}. As an example of a variational method enabled by \gls{vi}, we study \glspl{hvm} as approximations to the Boltzmann distribution of statistical physics models. We find that \glspl{hvm} scale to larger systems sizes than \glspl{van} in Sherrington-Kirkpatrick and Ising models. Testing the feasibility of \gls{vi} methods in statistical physics is a twofold opportunity. Statistical physics problems might serve as benchmarks for \gls{vi}, and using \gls{vi} for these problems can lead to improved computational methods in statistical physics. % and where exact solutions ar eknown?
\paragraph{Related Work.} The \gls{gbf} variational principle has been used to study Markov random fields~\citep{jun-zhang1996the-application} and the connection between variational inference and statistical physics has been well-documented~\citep{blei2017variational,hoffman2013stochastic,mackay2003information}. But this equivalence between \gls{vi} and the \gls{gbf} inequality might serve as an introduction to \gls{vi} for physicists. \citet{wu2019solving} implicitly use \gls{vi}, by developing \glspl{van} and a reinforcement learning policy gradient algorithm (however, \gls{vi} is not mentioned). Further, for a system of size $L$, autoregressive neural networks require $\cO(L^2)$ forward passes to sample a system configuration, making \glspl{van} intractable in larger systems. The use of \glspl{hvm} can be advantageous for statistical physics as these models can sample from a system in $\cO(L)$ time and yield results for larger systems.

\paragraph{Variational Inference.}
\gls{vi} is equivalent to the \gls{gbf} variational principle and requires similar choices of a practitioner. The variational family $q(\mbz; \mbnu)$ to approximate a model must be chosen, in addition to a method to maximize the variational lower bound in \Cref{eq:llbo}.

The \gls{vi} literature provides several choices of variational family, such as a mean field, factorized variational distribution with independent latent variables. Another choice of variational family is the Bethe approximation, which constrains the variational distribution to the polytope of mean parameters that captures correlations between any two latent variables~\citep{wainwright2008graphical}. Some machine learning research focuses on developing variational approximations that capture correlations between latent variables~\citep{hoffman2015stochastic,kingma2016improved,maaloe2016auxiliary,wu2019solving}. An example of a variational family that can model correlations between latent variables is the \gls{van} family~\citep{wu2019solving}, which uses autoregressive neural networks to parameterize the variational distribution ${q(\mbz_i \mid \mbz_1, \ldots, \mbz_{i-1})}$. We explore the \gls{hvm} class of variational approximations~\citep{ranganath2018black}.

The second choice required to employ \gls{vi} is how to optimize the variational lower bound in \Cref{eq:llbo}. The choice of variational family can limit the available optimization techniques. For a simple variational family like the mean field approximation, it may be possible to analytically evaluate the expectations in \Cref{eq:llbo}. Then derivatives of the variational bound with respect to the variational parameters $\mbnu$ and manual calculation can maximize the lower bound, as derived in \Cref{sec:ising-mean-field}. If more expressive variational families are used (e.g. \gls{van}s with thousands or millions of variational parameters), the analytic approach is infeasible. Stochastic optimization and automatic differentiation software have been used to develop several approaches to computing gradients of the variational lower bound, such as \acrlong{bbvi}~\citep{ranganath2018black,mohamed2019monte}.
\input{ch-hvm/fig/graphical_model}

The choice of variational family $q(\mbz; \mbnu)$ and optimization method for maximizing the variational lower bound leads to a trade-off intrinsic to \gls{vi}. Simple variational approximations such as the mean field family may be computationally feasible but inaccurate. The cost of increased accuracy, say by using a structured variational approximation, is increased computation. We illustrate the use of \gls{vi} in statistical physics by comparing two choices of variational approximation, \glspl{hvm}~\citep{ranganath2018black} and \glspl{van}~\citep{wu2019solving}. Many other variational approximations can be explored in future work.


% !Mode:: "TeX:UTF-8"
% !TEX program  = xelatex
\appendix
\section*{Appendix --- Quadratic Equation in \texttt{Python}}\label{A:python}
If you are familiar with \texttt{Python}, you can use \texttt{Python} to draw the pictures of trajectories, and you will find it easy to use \texttt{Python} to solve quadratic equations, especially in symbolic calculations.

\Python{quadratic\_equation.py}{code/quadratic_equation.py}

\Python{trajectory.py}{code/trajectory.py}


% Copyright 2017-2019 Burlen Loring, Remi Lehe
%
% This file is part of WarpX.
%
% License: BSD-3-Clause-LBNL

\input{newcommands}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Moving window and optimal Lorentz boosted frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The simulations of plasma accelerators from first principles are extremely computationally intensive, due to the need to resolve the evolution of a driver (laser or particle beam) and an accelerated particle beam into a plasma structure that is orders of magnitude longer and wider than the accelerated beam. As is customary in the modeling of particle beam dynamics in standard particle accelerators, a moving window is commonly used to follow the driver, the wake and the accelerated beam. This results in huge savings, by avoiding the meshing of the entire plasma that is orders of magnitude longer than the other length scales of interest.

\begin{figure}
%\begin{centering}
\includegraphics[scale=0.6]{Boosted_frame.png}
%\par\end{centering}
\caption{\label{fig:PIC} A first principle simulation of a short driver beam (laser or charged particles) propagating through a plasma that is orders of magnitude longer necessitates a very large number of time steps. Recasting the simulation in a frame of reference that is moving close to the speed of light in the direction of the driver beam leads to simulating a driver beam that appears longer propagating through a plasma that appears shorter than in the laboratory. Thus, this relativistic transformation of space and time reduces the disparity of scales, and thereby the number of time steps to complete the simulation, by orders of magnitude.}
\end{figure}

Even using a moving window, however, a full PIC simulation of a plasma accelerator can be extraordinarily demanding computationally, as many time steps are needed to resolve the crossing of the short driver beam with the plasma column. As it turns out, choosing an optimal frame of reference that travels close to the speed of light in the direction of the laser or particle beam (as opposed to the usual choice of the laboratory frame) enables speedups by orders of magnitude \cite{Vayprl07,Vaypop2011}. This is a result of the properties of Lorentz contraction and dilation of space and time. In the frame of the laboratory, a very short driver (laser or particle) beam propagates through a much longer plasma column, necessitating millions to tens of millions of time steps for parameters in the range of the BELLA or FACET-II experiments. As sketched in Fig. \ref{fig:PIC}, in a frame moving with the driver beam in the plasma at velocity $v=\beta c$ (where $c$ is the speed of light in vacuum), the beam length is now elongated by $\approx(1+\beta)\gamma$ while the plasma contracts by $\gamma$ (where $\gamma=1/\sqrt{1-\beta^2}$ is the relativistic factor associated with the frame velocity). The number of time steps that is needed to simulate a ``longer'' beam through a ``shorter'' plasma is now reduced by up to $\approx(1+\beta) \gamma^2$ (a detailed derivation of the speedup is given below).

The modeling of a plasma acceleration stage in a boosted frame
involves the fully electromagnetic modeling of a plasma propagating at near the speed of light, for which Numerical Cerenkov
\cite{Borisjcp73,Habericnsp73} is a potential issue, as explained in more details below.
In addition, for a frame of reference moving in the direction of the accelerated beam (or equivalently the wake of the laser),
waves emitted by the plasma in the forward direction expand
while the ones emitted in the backward direction contract, following the properties of the Lorentz transformation.
If one had to resolve both forward and backward propagating
waves emitted from the plasma, there would be no gain in selecting a frame different from the laboratory frame. However,
the physics of interest for a laser wakefield is the laser driving the wake, the wake, and the accelerated beam.
Backscatter is weak in the short-pulse regime, and does not
interact as strongly with the beam as do the forward propagating waves
which stay in phase for a long period. It is thus often assumed that the backward propagating waves
can be neglected in the modeling of plasma accelerator stages. The accuracy  of this assumption has been demonstrated by
comparison between explicit codes which include both forward and backward waves and envelope or quasistatic codes which neglect backward waves
\cite{Geddesjp08,Geddespac09,Cowanaac08}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Theoretical speedup dependency with the frame boost}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The derivation that is given here reproduces the one given in \cite{Vaypop2011}, where the obtainable speedup is derived as an extension of the formula that was derived earlier\cite{Vayprl07}, taking in addition into account the group velocity of the laser as it traverses the plasma.

Assuming that the simulation box is a fixed number of plasma periods long, which implies the use (which is standard) of a moving window following
the wake and accelerated beam, the speedup is given by the ratio of the time taken by the laser pulse and the plasma to cross each other, divided by the shortest time scale of interest, that is the laser period. To first order, the wake velocity $v_w$ is set by the 1D group velocity of the laser driver, which in the linear (low intensity) limit, is given by \cite{Esareyrmp09}:

%
\begin{equation}
v_w/c=\beta_w=\left(1-\frac{\omega_p^2}{\omega^2}\right)^{1/2}
\end{equation}
%
where $\omega_p=\sqrt{(n_e e^2)/(\epsilon_0 m_e)}$ is the plasma frequency, $\omega=2\pi c/\lambda$ is the laser frequency, $n_e$ is the plasma density, $\lambda$ is the laser wavelength in vacuum, $\epsilon_0$ is the permittivity of vacuum, $c$ is the speed of light in vacuum, and $e$ and $m_e$ are respectively the charge and mass of the electron.

In practice, the runs are typically stopped when the last electron beam macro-particle exits the plasma, and a measure of the total time of the simulation is then given by
%
\begin{equation}
T=\frac{L+\eta \lambda_p}{v_w-v_p}
\end{equation}
%
where $\lambda_p\approx 2\pi c/\omega_p$ is the wake wavelength, $L$ is the plasma length, $v_w$ and $v_p=\beta_p c$ are respectively the velocity of the wake and of the plasma relative to the frame of reference, and $\eta$ is an adjustable parameter for taking into account the fraction of the wake which exited the plasma at the end of the simulation.
For a beam injected into the $n^{th}$ bucket, $\eta$ would be set to $n-1/2$. If positrons were considered, they would be injected half a wake period ahead of the location of the electrons injection position for a given period, and one would have $\eta=n-1$. The numerical cost $R_t$ scales as the ratio of the total time to the shortest timescale of interest, which is the inverse of the laser frequency, and is thus given by
%
\begin{equation}
R_t=\frac{T c}{\lambda}=\frac{\left(L+\eta \lambda_p\right)}{\left(\beta_w-\beta_p\right) \lambda}
\end{equation}
%
In the laboratory, $v_p=0$ and the expression simplifies to
%
\begin{equation}
R_{lab}=\frac{T c}{\lambda}=\frac{\left(L+\eta \lambda_p\right)}{\beta_w \lambda}
\end{equation}
%
In a frame moving at $\beta c$, the quantities become
\begin{eqnarray}
\lambda_p^*&=&\lambda_p/\left[\gamma \left(1-\beta_w \beta\right)\right] \\
L^*&=&L/\gamma \\
\lambda^*&=& \gamma\left(1+\beta\right) \lambda\\
\beta_w^*&=&\left(\beta_w-\beta\right)/\left(1-\beta_w\beta\right) \\
v_p^*&=&-\beta c \\
T^*&=&\frac{L^*+\eta \lambda_p^*}{v_w^*-v_p^*} \\
R_t^*&=&\frac{T^* c}{\lambda^*} = \frac{\left(L^*+\eta \lambda_p^*\right)}{\left(\beta_w^*+\beta\right) \lambda^*}
\end{eqnarray}
where $\gamma=1/\sqrt{1-\beta^2}$.

The expected speedup from performing the simulation in a boosted frame is given by the ratio of $R_{lab}$ and $R_t^*$

\begin{equation}
S=\frac{R_{lab}}{R_t^*}=\frac{\left(1+\beta\right)\left(L+\eta \lambda_p\right)}{\left(1-\beta\beta_w\right)L+\eta \lambda_p}
\label{Eq_scaling1d0}
\end{equation}

We note that assuming that $\beta_w\approx1$ (which is a valid approximation for most practical cases of interest) and that $\gamma<<\gamma_w$, this expression is consistent with the expression derived earlier \cite{Vayprl07} for the laser-plasma acceleration case, which states that $R_t^*=\alpha R_t/\left(1+\beta\right)$ with $\alpha=\left(1-\beta+l/L\right)/\left(1+l/L\right)$, where $l$ is the laser length which is generally proportional to $\eta \lambda_p$, and $S=R_t/R_T^*$. However, higher values of $\gamma$ are of interest for maximum speedup, as shown below.

For intense lasers ($a\sim 1$) typically used for acceleration, the energy gain is limited by dephasing \cite{Schroederprl2011}, which occurs over a scale length $L_d \sim \lambda_p^3/2\lambda^2$.
Acceleration is compromised beyond $L_d$ and in practice, the plasma length is proportional to the dephasing length, i.e. $L= \xi L_d$. In most cases, $\gamma_w^2>>1$, which allows the approximations $\beta_w\approx1-\lambda^2/2\lambda_p^2$, and $L=\xi \lambda_p^3/2\lambda^2\approx \xi \gamma_w^2 \lambda_p/2>>\eta \lambda_p$, so that Eq.(\ref{Eq_scaling1d0}) becomes
%
\begin{equation}
S=\left(1+\beta\right)^2\gamma^2\frac{\xi\gamma_w^2}{\xi\gamma_w^2+\left(1+\beta\right)\gamma^2\left(\xi\beta/2+2\eta\right)}
\label{Eq_scaling1d}
\end{equation}
%
For low values of $\gamma$, i.e. when $\gamma<<\gamma_w$, Eq.(\ref{Eq_scaling1d}) reduces to
%
\begin{equation}
S_{\gamma<<\gamma_w}=\left(1+\beta\right)^2\gamma^2
\label{Eq_scaling1d_simpl2}
\end{equation}
%
Conversely, if $\gamma\rightarrow\infty$, Eq.(\ref{Eq_scaling1d}) becomes
%
\begin{equation}
S_{\gamma\rightarrow\infty}=\frac{4}{1+4\eta/\xi}\gamma_w^2
\label{Eq_scaling_gamma_inf}
\end{equation}
%
Finally, in the frame of the wake, i.e. when $\gamma=\gamma_w$, assuming that $\beta_w\approx1$, Eq.(\ref{Eq_scaling1d}) gives
%
\begin{equation}
S_{\gamma=\gamma_w}\approx\frac{2}{1+2\eta/\xi}\gamma_w^2
\label{Eq_scaling_gamma_wake}
\end{equation}
Since $\eta$ and $\xi$ are of order unity, and the practical regimes of most interest satisfy $\gamma_w^2>>1$, the speedup that is obtained by using the frame of the wake will be near the maximum obtainable value given by Eq.(\ref{Eq_scaling_gamma_inf}).

Note that without the use of a moving window, the relativistic effects that are at play in the time domain would also be at play in the spatial domain \cite{Vayprl07}, and the $\gamma^2$ scaling would transform to $\gamma^4$. Hence, it is important to use a moving window even in simulations in a Lorentz boosted frame. For very high values of the boosted frame, the optimal velocity of the moving window may vanish (i.e. no moving window) or even reverse.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Numerical Stability and alternate formulation in a Galilean frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The numerical Cherenkov instability (NCI) \cite{Godfreyjcp74}
is the most serious numerical instability affecting multidimensional
PIC simulations of relativistic particle beams and streaming plasmas
\cite{Martinscpc10,VayAAC2010,Vayjcp2011,Spitkovsky:Icnsp2011,GodfreyJCP2013,XuJCP2013}.
It arises from coupling between possibly numerically distorted electromagnetic modes and spurious
beam modes, the latter due to the mismatch between the Lagrangian
treatment of particles and the Eulerian treatment of fields \cite{Godfreyjcp75}.

In recent papers the electromagnetic dispersion
relations for the numerical Cherenkov instability were derived and solved for both FDTD \cite{GodfreyJCP2013,GodfreyJCP2014_FDTD}
and PSATD \cite{GodfreyJCP2014_PSATD,GodfreyIEEE2014} algorithms.

Several solutions have been proposed to mitigate the NCI \cite{GodfreyJCP2014,GodfreyIEEE2014,GodfreyJCP2014_PSATD,GodfreyCPC2015,YuCPC2015,YuCPC2015-Circ}. Although
these solutions efficiently reduce the numerical instability,
they typically introduce either strong smoothing of the currents and
fields, or arbitrary numerical corrections, which are
tuned specifically against the NCI and go beyond the
natural discretization of the underlying physical equation. Therefore,
it is sometimes unclear to what extent these added corrections could impact the
physics at stake for a given resolution.

For instance, NCI-specific corrections include periodically smoothing
the electromagnetic field components \cite{Martinscpc10},
using a special time step \cite{VayAAC2010,Vayjcp2011} or
applying a wide-band smoothing of the current components \cite{VayAAC2010,Vayjcp2011,VayPOPL2011}. Another set of mitigation methods
involve scaling the deposited
currents by a carefully-designed wavenumber-dependent factor
\cite{GodfreyJCP2014_FDTD,GodfreyIEEE2014} or slightly modifying the
ratio of electric and magnetic fields ($E/B$) before gathering their
value onto the macroparticles
\cite{GodfreyJCP2014_PSATD,GodfreyCPC2015}.
Yet another set of NCI-specific corrections
\cite{YuCPC2015,YuCPC2015-Circ} consists
in combining a small timestep $\Delta t$, a sharp low-pass spatial filter,
and a spectral or high-order scheme that is tuned so as to
create a small, artificial ``bump'' in the dispersion relation
\cite{YuCPC2015}. While most mitigation methods have only been applied
to Cartesian geometry, this last
set of methods (\cite{YuCPC2015,YuCPC2015-Circ})
has the remarkable property that it can be applied
\cite{YuCPC2015-Circ} to both Cartesian geometry and
quasi-cylindrical geometry (i.e. cylindrical geometry with
azimuthal Fourier decomposition \cite{LifschitzJCP2009,DavidsonJCP2015,Lehe2016}). However,
the use of a small timestep proportionally slows down the progress of
the simulation, and the artificial ``bump'' is again an arbitrary correction
that departs from the underlying physics.

A new scheme was recently proposed, in \cite{KirchenARXIV2016,LeheARXIV2016}, which
completely eliminates the NCI for a plasma drifting at a uniform relativistic velocity
-- with no arbitrary correction -- by simply integrating
the PIC equations in \emph{Galilean coordinates} (also known as
\emph{comoving coordinates}). More precisely, in the new
method, the Maxwell equations \emph{in Galilean coordinates} are integrated
analytically, using only natural hypotheses, within the PSATD
framework (Pseudo-Spectral-Analytical-Time-Domain \cite{Habericnsp73,VayJCP2013}).

The idea of the proposed scheme is to perform a Galilean change of
coordinates, and to carry out the simulation in the new coordinates:
\begin{equation}
\label{eq:change-var}
\vec{x}' = \vec{x} - \vgal t
\end{equation}
where $\vec{x} = x\,\vec{u}_x + y\,\vec{u}_y + z\,\vec{u}_z$ and
$\vec{x}' = x'\,\vec{u}_x + y'\,\vec{u}_y + z'\,\vec{u}_z$ are the
position vectors in the standard and Galilean coordinates
respectively.

When choosing $\vgal= \vec{v}_0$, where
$\vec{v}_0$ is the speed of the bulk of the relativistic
plasma, the plasma does not move with respect to the grid in the Galilean
coordinates $\vec{x}'$ -- or, equivalently, in the standard
coordinates $\vec{x}$, the grid moves along with the plasma. The heuristic intuition behind this scheme
is that these coordinates should prevent the discrepancy between the Lagrangian and
Eulerian point of view, which gives rise to the NCI \cite{Godfreyjcp75}.

An important remark is that the Galilean change of
coordinates (\ref{eq:change-var}) is a simple translation. Thus, when used in
the context of Lorentz-boosted simulations, it does
of course preserve the relativistic dilatation of space and time which gives rise to the
characteristic computational speedup of the boosted-frame technique.

Another important remark is that the Galilean scheme is \emph{not}
equivalent to a moving window (and in fact the Galilean scheme can be
independently \emph{combined} with a moving window). Whereas in a
moving window, gridpoints are added and removed so as to effectively
translate the boundaries, in the Galilean scheme the gridpoints
\emph{themselves} are not only translated but in this case, the physical equations
are modified accordingly. Most importantly, the assumed time evolution of
the current $\vec{J}$ within one timestep is different in a standard PSATD scheme with moving
window and in a Galilean PSATD scheme \cite{LeheARXIV2016}.

In the Galilean coordinates $\vec{x}'$, the equations of particle
motion and the Maxwell equations take the form
\begin{subequations}
\begin{align}
\frac{d\vec{x}'}{dt} &= \frac{\vec{p}}{\gamma m} - \vgal \label{eq:motion1} \\
\frac{d\vec{p}}{dt} &= q \left( \vec{E} +
\frac{\vec{p}}{\gamma m} \times \vec{B} \right) \label{eq:motion2}\\
\left( \Dt{\;} - \vgal\cdot\nab\right)\vec{B} &= -\nab\times\vec{E} \label{eq:maxwell1}\\
\frac{1}{c^2}\left( \Dt{\;} - \vgal\cdot\nab\right)\vec{E} &= \nab\times\vec{B} - \mu_0\vec{J} \label{eq:maxwell2}
\end{align}
\end{subequations}
where $\nab$ denotes a spatial derivative with respect to the
Galilean coordinates $\vec{x}'$.

Integrating these equations from $t=n\Delta
t$ to $t=(n+1)\Delta t$ results in the following update equations (see
\cite{LeheARXIV2016} for the details of the derivation):
%
\begin{subequations}
\begin{align}
\fb^{n+1} &= \theta^2 C \fb^n
 -\frac{\theta^2 S}{ck}i\vec{k}\times \fe^n \nonumber \\
& + \;\frac{\theta \chi_1}{\epsilon_0c^2k^2}\;i\vec{k} \times
                     \fj^{n+1/2} \label{eq:disc-maxwell1}\\
\fe^{n+1} &=  \theta^2 C  \fe^n
 +\frac{\theta^2 S}{k} \,c i\vec{k}\times \fb^n \nonumber \\
& +\frac{i\nu \theta \chi_1 - \theta^2S}{\epsilon_0 ck} \; \fj^{n+1/2}\nonumber \\
& - \frac{1}{\epsilon_0k^2}\left(\; \chi_2\;\mc{\rho}^{n+1} -
  \theta^2\chi_3\;\mc{\rho}^{n} \;\right) i\vec{k} \label{eq:disc-maxwell2}
\end{align}
\end{subequations}
%
where we used the short-hand notations $\fe^n \equiv
%
\fe(\vec{k}, n\Delta t)$, $\fb^n \equiv
\fb(\vec{k}, n\Delta t)$ as well as:
\begin{subequations}
\begin{align}
&C = \cos(ck\Delta t) \quad S = \sin(ck\Delta t) \quad k
= |\vec{k}| \label{eq:def-C-S}\\&
\nu = \frac{\vec{k}\cdot\vgal}{ck} \quad \theta =
  e^{i\vec{k}\cdot\vgal\Delta t/2} \quad \theta^* =
  e^{-i\vec{k}\cdot\vgal\Delta t/2} \label{eq:def-nu-theta}\\&
\chi_1 =  \frac{1}{1 -\nu^2} \left( \theta^* -  C \theta + i
  \nu \theta S \right) \label{eq:def-chi1}\\&
\chi_2 = \frac{\chi_1 - \theta(1-C)}{\theta^*-\theta} \quad
\chi_3 = \frac{\chi_1-\theta^*(1-C)}{\theta^*-\theta} \label{eq:def-chi23}
\end{align}
\end{subequations}
Note that, in the limit $\vgal=\vec{0}$,
(\ref{eq:disc-maxwell1}) and (\ref{eq:disc-maxwell2}) reduce to the standard PSATD
equations \cite{Habericnsp73}, as expected.
As shown in \cite{KirchenARXIV2016,LeheARXIV2016},
the elimination of the NCI with the new Galilean integration is verified empirically via PIC simulations of uniform drifting plasmas and laser-driven plasma acceleration stages, and confirmed by a theoretical analysis of the instability.