\chapter{Experiments}

\section{Benchmark}

We perform benchmarks on five bivariate cause-effect datasets\footnote{ The TCEP dataset
can be found \href{https://webdav.tuebingen.mpg.de/cause-effect/}{\color{blue}{here}} and
all the other datasets can be found
\href{https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/3757KX}{\color{blue}{here}}
}, covering a wide range of associations:

% \begin{enumerate}[itemsep=-1.8em, topsep=0pt]
\begin{enumerate}[noitemsep, topsep=0pt]

    \item \textbf{Cha} (300 cause-effect pairs) pairs from the challenge of \cite{chalearn}.
    \item \textbf{Net} (300 cause-effect pairs) artificial cause-effect pairs generated using
            random distributions as causes, and neural networks as causal mechanisms.
    \item \textbf{Gauss} (300 cause-effect pairs) generated by \cite{Mooij2016jmlr}, using random
      mixtures of Gaussians as causes, and Gaussian process priors as causal mechanisms.
    \item \textbf{Multi} (300 cause-effect pairs) built with random linear and
    polynomial causal mechanisms (by \cite{goudet2017causal}). In this dataset,
    additive or multiplicative noise, is applied before or after the causal mechanism.
    \item \textbf{TCEP} (108 cause-effect pairs)\footnote{Note that 6 of these pairs are not bivariate; the
    dataset is also weighted as some samples come from similar data generation processes. }
     is the Tübingen Cause Effect Pair data set which consists of various
    domains such as climatology, finance, and medicine (\cite{Mooij2016jmlr}).

\end{enumerate}

We use the results reported by \cite{goudet2017causal}. We compare against the following algorithms
in the benchmark: ANM (\cite{Mooij2016jmlr}) with Gaussian Process regression and HSIC score as described
in the introduction. IGCI \cite{daniusis2012inferring} with entropy estimator and Gaussian reference measure.
A  pairwise version of LiNGAM \cite{shimizu2011directlingam}.  Jarfo (Fonollosa, 2016), using a random forest
causal classifier trained from the ChaLearn Cause-effect pairs on top of 150 features including ANM, IGCI,
CDS, LiNGAM, regressions, HSIC tests (\cite{fonollosa2019conditional}). GCNN without the approximated MMD (\cite{goudet2017causal})

In the first setting we want to see how well we perform in terms of accuracy with real date, i.e.
TCEP. For each  competitor method, a leave-one-dataset-out scheme
is used to select the best hyperparameters for each method (as reported by \cite{goudet2017causal}).
We report the scores in table \ref{tab:acc}; as we can see see, twin test is competitive with
other methods.

\vspace{0.03\textheight}
\begin{table}[H]
    \centering
    \begin{tabular}{lccccc}
        \hline method  & TCEP \\
        LiNGAM &   44.3 \\
        IGCI  &  62.6 \\
        ANM  &  59.5 \\
        Jarfo  &  59.5 \\
        CGNN   &  74.4 \\
        \hline TT & 62.4 \\
        \hline
    \end{tabular}

    \caption{ weighted accuracy
     in parenthesis for TCEP.}
    \label{tab:acc}
\end{table}

The standard TwinTest (TT) algorithm that we use has the following setup:

The score is the maximum pairwise distance:

$$
    C(P_k) = \operatorname{max}_{i, j} d(p_i, p_j)
$$

with $d(x, y) = \norm{x - y}_1$.

We use polynomial regression via model selection as described in section 4.1 For the cluster size
we use a greedy approach: given some $\rho > 0$, $m \in \N$, and data $\mathcal{D}$,
we start by finding $m$ clusters
using k-means, say $\mathcal{D}_1, ..., \mathcal{D}_m$, if
$\min_i |\mathcal{D}_i| \geq \rho n$, where $n = |\mathcal{D}|$, then we are done;
otherwise we try again with $m - 1$ clusters, and we keep going until it halts because of the
condition being met or when $m = 2$.

In practice we set $m = 8$, and we let $\rho$ vary\footnote{More details about the implementation can be found
\href{https://github.com/Nacho114/bivariate-causal-inference}{\color{blue}{here}}.}
with sample size, intuitively if we have many samples, then we can support a lower $\rho$.
For the bin size we choose roughly $\log(k)$ where $k$ is the size of a cluster.

We also try the following variants of TT:

\begin{itemize}
    \item[--] TT\textsubscript{mmd}: Standard TT with
        $d(x, y) = \operatorname{MMD}(x, y)$, using a Gaussian kernel and the median heuristic\footnote{
            As a reminder, the median heuristic is:
            $\hat{\gamma}(\mathbf{u}):=\operatorname{median}\left\{\left\|u_{i}-u_{j}\right\|:  i<j ,\left\|u_{i}-u_{j}\right\| \neq 0\right\}$}
         to determinre the bandwidth.
    \item[--] TT\textsubscript{2-partitions}: Standard TT with fixed partition size set to $2$.
    \item[--] TT\textsubscript{Neural net}: Standard TT using neural network as regression.
    \item[--] TT\textsubscript{$\mu$-score}: Standard TT with $d(x, y) = \norm{x - y}_1$ and score:
    $$
    C(P_k) = \frac{1}{\binom{k}{2}} \sum_{i < j} d(p_i, p_\mu)
    , \quad p_\mu = \frac{1}{k} \sum_i p_i
    $$
\end{itemize}

In order to see how the variants compare we run them on all the benchmarks and report the
scores in table \ref{tab:own_perf}. We also include ANM and IGCI but do not do any fine
tuning and use them right out of the box --- we do so since we do not fine tune any
of the variants of TT as well as TT.

We remark that overall, TwinTest in its standard form has the best performance on TCEP,
and is close to the top in most of the benchmarks. It seems
that using partition numbers adaptively is better than when fixing it with 2, as is
done in TT\textsubscript{2-partitions}. Surprisingly, the MMD does not seem to
offer much advantage compared to the $l_1$ distance.
When using neural networks with TT\textsubscript{Neural net}, performance
is also lost; this could probably be improved by using a better model selection process
for the neural network. Using a different score as in TT\textsubscript{$\mu$-score}
does not seem to make a big difference.

\vspace{0.03\textheight}
\begin{table}[H]
    \centering
    \begin{tabular}{lccccc}
        \hline method & Cha & Net & Gauss & Multi & TCEP \\
        ANM & 67.3 & 76.3 & 80.0 & 35.3 & 52.5\\
        IGCI & 55.0 & 57.0 & 21.3 & 68.0 & 60.3\\
        % A1 & 65.0 & 75.3 & 78.7 & 36.3 & 57.2\\
        TT\textsubscript{mmd} & 65.7 & 75.7 & 77.0 & 36.3 & 57.3\\
        TT\textsubscript{2-partitions} & 67.0 & 72.0 & 73.0 & 43.7 & 60.6\\
        TT\textsubscript{Neural net} & 62.0 & 68.0 & 76.7 & 39.7 & 59.7\\
        TT\textsubscript{$\mu$-score}  & 67.3 & 70.7 & 76.0 & 48.0 & 62.3\\
        TT & 67.3 & 68.3 & 76.7 & 42.3 & 62.4\\
        \hline
    \end{tabular}

    \caption{Cause-effect relations: Accuracy
     on 5 benchmarks for the cause-effect experiments (weighted accuracy
     for TCEP).}
    \label{tab:own_perf}
\end{table}

% \begin{table}[H]
%     \centering

%     \begin{tabular}{lcc}
%         \hline Model & TCEP & TCEP with 75 samples \\
%         \hline BCI & 0.64 & 0.60 \\
%         ANM-HSIC & 0.63 & 0.54 \\
%         ANM-MML & 0.58 & 0.56 \\
%         IGCI & 0.66 & 0.62 \\
%         CGNN & 0.70 & 0.69 \\
%         \hline TwinTest & 0.62 & TODO \\
%         \hline
%     \end{tabular}
%     \caption{Accuracy for TCEP Benchmark}
%     \label{tab:acc}
% \end{table}

% Table \ref{tab:acc} is taken from \cite{kurthen2018bayesian}
% \begin{table}[H]
%     \centering
%     \begin{tabular}{lccccc}
%         \hline method & Cha & Net & Gauss & Multi & TCEP \\
%         % \hline Best fit & 56.4 & 77.6 & 36.3 & 55.4 & 58.4 (44.9) \\
%         LiNGAM & 54.3 & 43.7 & 66.5 & 59.3 & 39.7 (44.3) \\
%         % CDS & 55.4 & 89.5 & 84.3 & 37.2 & 59.8 (65.5) \\
%         IGCI & 54.4 & 54.7 & 33.2 & 80.7 & 60.7 (62.6) \\
%         ANM & 66.3 & 85.1 & 88.9 & 35.5 & 53.7 (59.5) \\
%         % PNL & 73.1 & 75.5 & 83.0 & 49.0 & 68.1 (66.2) \\
%         Jarfo & 79.5 & 92.7 & 85.3 & 94.6 & 54.5 (59.5) \\
%         % GPI & 67.4 & 88.4 & 89.1 & 65.8 & 66.4 (62.6) \\
%         CGNN  & 73.6 & 89.6 & 82.9 & 96.6 & 79.8 (74.4) \\
%         \hline TwinTest & 66.3 & 81.9 & 85.1 & 39.8 & 82.0 (62.4) \\
%         \hline
%     \end{tabular}

%     \caption{Cause-effect relations: Area Under the Precision Recall curve
%      on 5 benchmarks for the cause-effect experiments (weighted accuracy
%      in parenthesis for TCEP).}
%     \label{tab:perf}
% \end{table}


\subsubsection{\stid{3.06} PETSc-TAO} \label{subsubsect:petsc}
\paragraph{Overview}

Algebraic solvers (generally nonlinear solvers that use sparse linear solvers) and integrators form the
core computation of many numerical simulations. No scalable ``black box'' sparse solvers or integrators
work for all applications, nor are there single implementations that work well for all problem sizes.
Hence, algebraic solver and integrator packages provide a wide variety of algorithms and implementations
that can be customized for the application and range of problem sizes. PETSc/TAO~\cite{petsc:homepage,petsc-man}
is a widely used numerical library for the scalable solution of linear, nonlinear, and variational systems,
for integration of ODE/DAE systems and computation of their adjoints, and for numerical optimization.
This project focuses on three topics: (1) partially matrix-free scalable solvers to efficiently use
many-core and GPU-based systems; (2) reduced synchronization algorithms that can scale to larger
concurrency than solvers with synchronization points; and (3) performance and data structure
optimizations for all the core data structures to better utilize many-core and GPU-based
systems as well as provide scalability to the exascale systems.

The availability of systems with over 100 times the processing power of today's machines compels the utilization
of these systems not just for a single ``forward solve'' (as discussed above), but rather within a tight loop
of optimization, sensitivity analysis (SA), and uncertain quantification (UQ). This requires the implementation
of a new scalable library for managing a dynamic hierarchical collection of running scalable simulations, where
the simulations directly feed results into the optimization, SA, and UQ solvers.  This library, which we call
libEnsemble, directs the multiple concurrent ``function evaluations'' through the tight coupling and
feedback. This work consist of two parts: (1) the development of libEnsemble; and (2) the development
of application-relevant algorithms to utilize libEnsemble.

\paragraph{Key Challenges}

A key challenge for scaling the PETSc/TAO numerical libraries to Exascale systems is that traditional
``sparse-matrix-based'' techniques for linear, nonlinear, and ODE solvers, as well as optimization
algorithms, are memory-bandwidth limited.  Another difficulty is that any synchronizations
required across all compute units---for example, an inner product or a norm---can
dramatically affect the scaling of the solvers.  Another challenge is the need to
support the variety of accelerators that will be available on the exascale systems
and the programming models that application teams use for performance
portability.

Running an ensemble of simulations requires a coordination layer that handles load balancing and
allows the collection of running simulations to grow and shrink based on feedback. Thus, our
libEnsemble library must be able to dynamically start simulations with different parameters,
resume simulations to obtain more accurate results, prune running simulations that the solvers
determine can no longer provide useful information, monitor the progress of the simulations,
and stop failed or hung simulations, and collect data from the individual simulations both
while they are running and at the end.

\paragraph{Solution Strategy}

To address the scalability of the numerical libraries, we implemented new solvers and data
structures including: pipeline Krylov methods that delay the use of the results of inner
products and norms, allowing overlapping of the reductions and other computation; partially
matrix-free solvers using high-order methods that have high floating-point-to-memory-access
ratios and good potential to use many-core and GPU-based systems; and in-node optimizations
of sparse matrix-matrix products needed by algebraic multigrid to better utilize many-core
systems.

Our strategy for coordinating ensemble computations has been to develop libEnsemble
to satisfy our needs.  This library should not be confused with workflow-based
scripting systems; rather it is a library that, through the tight coupling and
feedback, directs the multiple concurrent ``function evaluations'' needed by
optimization, SA, and UQ solvers.

\paragraph{Recent Progress}

In the past year, we have released PETSc/TAO 3.16 (available at \url{http://www.mcs.anl.gov/petsc}), which
features enhanced GPU support.  We have full support for a Kokkos plus KokkosKernels backend for
performance portability that includes vector, matrix, and matrix-matrix operations and full
support for a native CUDA backend.  We have partial support for a native HIP and OpenCL
backends.  We have been updating and profiling the GAMG solver, the native algebraic
multigrid solver in PETSc, to use the enhanced GPU support.  Numerical results on
this work is available in \cite{mills2021toward}.  We have also been updating
our scalable communication layers, PetscSF, which allows us to use GPU-aware
MPI, thus allowing direct communication of data between Summit GPUs, bypassing
the previously needed step of first copying the data to the CPU memory.  Numerical
results on this work is available in \cite{zhang2021petscsf}.

\begin{figure}
\centering
\includegraphics[trim = 0in .2in 1.7in .2in, clip, width=0.9\textwidth]{projects/2.3.3-MathLibs/2.3.3.06-PETSc-TAO/petsc_arch}
\caption{The PETSc/TAO architecture enables users to utilize a variety of programming
models for GPUs independently of PETSc's internal programming model.}
\label{fig:petsc-tao-fig}
\end{figure}

We have also release libEnsemble 0.7.2 (available at \url{https://github.com/Libensemble/libensemble}).
This release includes new generator functions and examples, improved testing across available platforms,
and a new tutorial.  A paper documenting libEnsemble and providing use cases is available
in \cite{hudson2021libensemble}.

\paragraph{Next Steps}

Our next efforts are:
\begin{enumerate}
  \item \textbf{Application readiness}:
  We will complete a status review of our applications for the early access hardware.  We will
  complete software quality initiatives related to the build system, architecture specific spack
  recipes, and spack smoke tests for build and accelerator usage validation.
  \item \textbf{PETSc/TAO + Kokkos + KokkosKernels release}:
  We will release a version of PETSc/TAO with full Kokkos and KokkosKernels integration.  We will
  provide a tutorial on these features, characterize the performance, and suggest optimizations
  and best practices.
  We will also make improvements to the PETSc/TAO GAMG solver and provide updated performance results.
  \item \textbf{libEnsemble + Balsam2 release}:
  We will release a version of libEnsemble with full Balsam2 support and updated API.
  We will follow continuous integration best practices and continue testing on pre-exascale DOE systems.
  \item \textbf{PETSc/TAO + Optimized communication layer release}:
  We will release a version of PETSc/TAO with an optimized communication layer for the early access systems.
  This version will include support for AMGx on NVIDEA GPUs and a batch LU factorization and solver.
  We will optimize some numerical methods (such as methods in DMNetwork and numerical optimization methods)
  to use the communication layer and better utilize the accelerators.
  We will provide initial benchmark results for GAMG and AMGx.
\end{enumerate}


\documentclass[12pt]{article}
\usepackage{graphics}
\begin{document}

\section*{NYU Physics 1---In-class Exam 3}

\vfill

\paragraph{Name:} ~

\paragraph{email:} ~

\paragraph{recitation:} ~

\vfill

This exam consists of two problems.  Write only in this booklet.  Be
sure to show your work.

\vfill ~

\clearpage

\section*{Problem 1}

A Japanese Shinkansen (bullet train) (of mass $M=10^6~\mathrm{kg}$),
moving at $v_S=300~\mathrm{km\,h^{-1}}$ with respect to the tracks,
hits and collides elastically with a superball (of mass
$m=3~\mathrm{g}$), which is initially at rest.  The front face of the
train is inclined at an angle of $\theta=45~\mathrm{deg}$ to the
horizontal, as shown.
\\ \rule{0.1\textwidth}{0pt}
\resizebox{0.8\textwidth}{!}{\includegraphics{../mp/shinkansen.eps}}

(a) Draw a diagram of the ball--train system, just before the
collision, in the center-of-mass rest frame.  Clearly show the
velocities of the ball and train in this frame.

\vfill

(b) What is the final speed and direction of the ball, immediately
after the collision, in the rest frame of the tracks (ie, \emph{not}
in the center-of-mass frame)?

\vfill ~

\clearpage

\section*{Problem 2}

A four-wheeled cart rolls without slipping at speed $v$ on a flat
surface.  Each of the cart's four wheels has mass $m$, radius $R$ and
moment of inertia $I$.  The total mass of the cart, including all four
wheels, is $(M+4m)$.

(a) When the cart is moving at speed $v$, what is its total kinetic
energy $K$?  Don't forget to include the kinetic energies of the
rotating wheels, and the rolling-without-slipping condition which
relates wheel angular speed $\omega$ to speed $v$.  Give your answer
in terms of quantities named.

\vfill

(b) Now---more difficult---imagine that the cart rolls without
slipping down a ramp inclined at an angle of $\theta$ to the
horizontal.  What is the magnitude of the cart's linear acceleration
$a$ down the ramp?  You will need to draw a free body diagram for the
whole cart (body plus wheels on one diagram), consider torques on the
four wheels, and consider the sum of forces on the whole cart.  Assume
that the friction on the axles is negligible.

\vfill ~

\clearpage

[This page intentionally left blank for calculations or other work.]

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
%  colloid.tex
%
%  Largely colloid hydrodynamics and colloid-colloid interactions
%
%  Edinburgh Soft Matter and Statistical Physics Group and
%  Edinburgh Parallel Computing Centre
%
%  Contributing authors:
%  Kevin Stratford (kevin@epcc.ed.ac.uk)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Colloid Dynamics}

\label{section:colloids}

\subsection{Hydrodynamics}

\label{section-colloid-hydrodynamics}

Having computed the total force and torque on an individual colloid
by the process of bounce-back-on-links,
is is possible to update the the linear velocities
\begin{equation}
m_0 \mathbf{U}(t + \Delta t) = m_0 \mathbf{U}(t) + \Delta t \mathbf{F} (t),
\end{equation}
where the mass of the colloid is related to the input radius by
$m_0 = (4/3)\pi\rho_0 a_0^3$ and the angular
velocity
\begin{equation}
I_0 \mathbf{\Omega} (t + \Delta t) = I_0 \mathbf{\Omega}(t) + \Delta t
\mathbf{T}(t),
\end{equation}
where the moment of inertia is $I_0 = (2/5) m_0 a_0^2$.
However, this explicit update is generally found to have poor stability
properties \cite{l94b, nguyen-ladd2002}. The alternative is to use a
velocity update
which is implicit \cite{heemels,nguyen-ladd2002}.

The total force and torque on a colloid can be split into a velocity-dependent
and a velocity-independent part by combining Equations~(\ref{eq-colloid-ub})
and~(\ref{eq-colloid-fb}) to eliminate the boundary velocity $\mathbf{u}_b$.
In this way, one can write
\begin{equation}
\left( \begin{array}{rr} \mathbf{F} \\ \mathbf{T}
\end{array} \right) =
\left( \begin{array}{rr} \mathbf{F}_0 \\ \mathbf{T}_0
\end{array} \right) -
\left( \begin{array}{rr}
\mathbf{\boldsymbol{\zeta}}^{FU} & \mathbf{\boldsymbol{\zeta}}^{F\Omega} \\
\mathbf{\boldsymbol{\zeta}}^{TU} & \mathbf{\boldsymbol{\zeta}}^{T\Omega}
\end{array} \right)
\left( \begin{array}{rr} \mathbf{U} \\ \mathbf{\Omega}
\end{array} \right).
\end{equation}
%%
Here, the velocity independent parts of the force and the torque
(appropriate for a colloid at rest) are
\begin{eqnarray}
\mathbf{F}_0(t + {\scriptstyle\frac{1}{2}}\Delta t) =
\frac{2\Delta x^3}{\Delta t} \sum_b \big[ f_b^\ast(\mathbf{r}; t)
- f_{b'}^\ast(\mathbf{r} + \mathbf{c}_i \Delta t; t) \big] \mathbf{c}_b,
\\
\mathbf{T}_0(t + {\scriptstyle\frac{1}{2}}\Delta t) =
\frac{2\Delta x^3}{\Delta t} \sum_b \big[ f_b^\ast(\mathbf{r}; t)
- f_{b'}^\ast(\mathbf{r} + \mathbf{c}_i \Delta t; t) \big]
(\mathbf{r}_b\times \mathbf{c}_b),
\end{eqnarray}
and the matrices $\boldsymbol{\zeta}$ are interpreted as drag
coefficients and can be written as
\begin{eqnarray}
\mathbf{\boldsymbol{\zeta}}^{FU} &=& \frac{4\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} \mathbf{c}_b \mathbf{c}_b,
\\
\mathbf{\boldsymbol{\zeta}}^{F\Omega} &=&
\frac{4\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} \mathbf{c}_b (\mathbf{r}_b \times \mathbf{c}_b),
\\
\mathbf{\boldsymbol{\zeta}}^{TU} &=& \frac{4\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} (\mathbf{r}_b \times \mathbf{c}_b) \mathbf{c}_b,
\\
\mathbf{\boldsymbol{\zeta}}^{T\Omega} &=&
\frac{4\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} (\mathbf{r}_b \times \mathbf{c}_b)
(\mathbf{r}_b \times \mathbf{c}_b).
\end{eqnarray}

The velocity updates can now be rewritten using an
implicit (or self-consistent) approach, where the velocity
at the new time step is used in computing the velocity-dependent
terms:
\begin{equation}
\label{eq-colloid-implicit-update}
\left( \begin{array}{rr}
m_0 \mathbf{U}(t + \Delta t) \\ I_0\mathbf{T}(t + \Delta t)
\end{array} \right) =
\left( \begin{array}{rr} m_0 \mathbf{U}(t) \\ I_0\mathbf{T}(t)
\end{array} \right) +
%%
\Delta t
\bigg[
\left( \begin{array}{rr} \mathbf{F}_0 \\ \mathbf{T}_0
\end{array} \right) -
\left( \begin{array}{rr}
\mathbf{\boldsymbol{\zeta}}^{FU} & \mathbf{\boldsymbol{\zeta}}^{F\Omega} \\
\mathbf{\boldsymbol{\zeta}}^{TU} & \mathbf{\boldsymbol{\zeta}}^{T\Omega}
\end{array} \right)
\left( \begin{array}{rr}
\mathbf{U}(t + \Delta t) \\ \mathbf{\Omega}(t + \Delta t)
\end{array} \right)
\bigg].
\end{equation}

If the particle has a symmetric distribution of boundary links (the
special case where the centre of mass is on a symmetry point of the
lattice) the $\boldsymbol{\zeta}^{FU}$ and
$\boldsymbol{\zeta}^{T\Omega}$ matrices are
diagonal, while the $\boldsymbol{\zeta}^{F\Omega}$
and $\boldsymbol{\zeta}^{TU}$ matrices are zero. In general, however,
this is not the case, and the full drag matrix must be used to form
the update of the dynamic quantities in which case
Equation~\ref{eq-colloid-implicit-update} is solved
as a 6$\times$6 matrix inversion.

Finally, the position of the particle can be updated: an Euler
forward step is taken using the mean of the old and updated velocity
\begin{equation}
\label{eq-colloid-position-update}
\mathbf{r} (t + \Delta t) = \mathbf{r} (t) + (1/2)\Delta t
\big[ \mathbf{U}(t) + \mathbf{U}(t + \Delta t) \big].
\end{equation}

\subsubsection{Rotational motions}

For a number of applications it is useful to update not only
position, as in Equation~\ref{eq-colloid-position-update},
but also orientation. This is relevant for magnetic
dipoles, swimming directions,  and so on. A convenient way to
implement the required rotations is by considering quaternions.

A quaternion can be thought of as an extended complex number
\begin{equation}
q = w + xi + yj + zk
\end{equation}
where $i^2 = j^2 = k^2 = -1$ and $ijk = -1$. The norm, or length
of a quaternion is
\begin{equation}
N(q) = (w^2 + x^2 + y^2 + z^2)^{1/2}.
\end{equation}
Addition and subtraction of quaternions proceeds as normal, but
care is required for multiplication, which is associative but not
commutative. If the quaternion is written as $q = (w, {\bf v})$ where
$\bf{v}$ is a normal 3-vector, then multiplication can be written
\begin{equation}
q_1 q_2 = (w_1 w_2 - {\bf v}_1 . {\bf v}_2, w_1 {\bf v}_2 + w_2 {\bf v}_1
+ {\bf v}_1 \times {\bf v}_2),
\end{equation}
where the usual scalar and vector products are used. Note that the
conjugate of a quaternion $q^\star = (w, -{\bf v})$ so that
$qq^\star = N^2(q)$.
It can be shown that a rotation of an arbitrary vector though an
angle $\theta$ around the unit axis of rotation $\hat{\bf w}$ can
be represented by the quaternion
\begin{equation}
q = (\cos(\theta/2), \hat{\bf w} \sin(\theta/2))
\end{equation}
which ensures that $N(q) = 1$. If the arbitrary vector is (0, {\bf v}),
then the rotated vector ${\bf v}^{'}$ can be written as
\begin{equation}
(0, {\bf v}^{'}) = q (0, {\bf v}) q^\star
\end{equation}
which can be expanded as
\begin{equation}
{\bf v}^{'} = (1 - \cos\theta)({\bf v}.{\hat{\bf w}) \hat{\bf w} +
{\bf v}\cos\theta} + (\hat{\bf w} \times {\bf v})\sin\theta.
\end{equation}
This equation is used to update appropriate vector quantities at
each time step by a rotation
$\mathbf{w} = \Delta t \mathbf{\Omega}(t +\Delta t)$,
where the angular velocity is that determined by the implicit
procedure described in Section~\ref{section-colloid-hydrodynamics}.

\subsubsection{Changes in particle shape}

One consequence is that the discrete shape of the particle fluctuates
as the colloid centre moves relative to the
lattice. This means that the size of the particle as seen by the
fluid changes, despite the fact that the input radius of the colloid
is fixed.  However, these fluctuations are small for input radii
greater than about 5 lattice units \cite{l96a}. Furthermore, the
fluctuations are greater for some input radii than others
(leading to preferred values of $a_0$ 1.25, 2.33, and so on).

In the original approach of Ladd \cite{l96a}, changes in the map of
boundary links are accommodated by the internal fluid. If the particle
shape changes so that an internal node is exposed, the internal fluid
at that site rejoins the fluid proper with the solid body momentum
(assuming the internal fluid is relaxed to solid-body rotation).
If a fluid node is covered by particle movement, then it simply
joins the internal fluid, and will relax to solid-body rotation.
Without internal fluid, changes in particle shape in which fluid nodes
are either covered or uncovered must be accompanied by the removal or
addition of fluid with appropriate properties at the nodes in question.
It is important that this is done in a way which minimises the
perturbation to the fluid flow around the particle.

If a fluid node at $\mathbf{r}$ is covered by the movement of a
solid particle, the fluid loses a mass $\rho(\mathbf{r}) \Delta x^3$,
of which an excess $\Delta M_c = (\rho(\mathbf{r}) - \rho_0)\Delta x^3$
must be replaced explicitly so that the overall mean fluid density is
unchanged. At the same time, the colloid must assume the momentum lost
by the fluid
$\Delta x^3 \sum_i f_i(\mathbf{r}) \mathbf{c}_i$.

Similarly, when a fluid node is exposed by particle movement,
fluid must be replaced with the appropriate distribution, density
and velocity. If the
new density is $\rho$ (to be determined by some method) then
the fluid gains an excess of mass $\Delta M_u = (\rho - \rho_0)
\Delta x^3$ which must be balanced elsewhere. If the new fluid
is assumed to have velocity $\mathbf{u}$ then the particle must
give up momentum $\Delta x^3 \rho\mathbf{u}$.

Following Nguyen and Ladd \cite{nguyen-ladd2002}, these changes owing
to change in particle shape are implemented by adding a small
correction to the bounce-back at the following time step.
The excess mass from a covered fluid site is redistributed over all the
boundary nodes by redefined the momentum transfer at bounce-back
\begin{equation}
f_{b'}(\mathbf{r}; t + \Delta t) = f_b^\ast (\mathbf{r}; t)
- \frac{2 w_{c_b} \rho_0 \mathbf{u}_b . \mathbf{c}_b}{c_s^2}
+ \frac{w_{c_b} \rho_0\Delta M_c }{A},
\label{eq:q1}
\end{equation}
where $A = \rho_0 \Delta x^3 \sum_b w_{c_b}$. The accompanying
force on the particle owing to the change in shape is then
that lost by the fluid plus the contributions from the final term
in (\ref{eq:q1}) summed over all the boundary links
\begin{equation}
\Delta \mathbf{F}_c = \frac{\Delta x^3}{\Delta t}\sum_{i} f_i \mathbf{c}_i
+ \frac{\Delta x^3}{\Delta t} \sum_b \frac{w_{c_b} \rho_0 \Delta M_c}{A}
\mathbf{c}_b.
\label{eq:q2}
\end{equation}

Note that for a closed surface,
$\sum_b w_{c_b}\mathbf{c}_b = 0$, and the second term on the right-hand
side of Eq.~(\ref{eq:q2}) is zero, i.e., the redistribution of mass
does not contribute to the net force and torque on the particle.
The corresponding change in torque is
\begin{equation}
\Delta \mathbf{T}_c = \frac{\Delta x^3}{\Delta t} \mathbf{r}_c
\times \sum_i f_i \mathbf{c}_i
+ \frac{\Delta x^3}{\Delta t}
\sum_b \frac{w_{c_b}\rho_0 \Delta M_c}{A} \mathbf{r}_b \times \mathbf{c}_b,
\label{eq:q2a}
\end{equation}
where $\mathbf{r}_c$ is the boundary vector at the position where the
fluid has been removed. Again, for a closed surface, the
redistribution of mass does not contribute to the net torque
on the particle.

Likewise, the contribution to the bounce-back from newly uncovered nodes
is
\begin{equation}
- \frac{w_{c_b} \rho_0 \Delta M_u}{A}
\label{eq:q3}
\end{equation}
leading to a change in force of
\begin{equation}
\Delta \mathbf{F}_u = -\frac{\Delta x^3 \rho(\mathbf{r})
\mathbf{u}(\mathbf{r})}{\Delta t} - \frac{\Delta x^3}{\Delta t} \sum_b
\frac{w_{c_b} \rho_0 \Delta M_u}{A} \mathbf{c}_b,
\label{eq:q4}
\end{equation}
with a corresponding change in the torque.
If more than one lattice node is either covered or uncovered by
the movement of the particle, then these contributions add in a simple
way. The overall contributions can be added to the
velocity-independent terms $\mathbf{F}_0$ and $\mathbf{T}_0$ appearing in
Equation~\ref{eq-colloid-implicit-update}.

\subsubsection{Particles near contact}

Particles near contact may not possess a full set of boundary
links. This leads to a potential
non-conservation of mass associated with the particle motion
which must be corrected.

Again following Nguyen and Ladd \cite{nguyen-ladd2002}, mass conservation is
enforced explicitly using the following procedure. There is a mass
transfer associated with the bounce-back at each link of
$(2w_{c_b}\rho_0 \mathbf{u}_b . \mathbf{c}_b / c_s^2) \Delta x^3$.
The net mass transport into the particle is the sum of this
quantity over all the links, and can be written
\begin{equation}
\Delta M_s = - \frac{2\Delta x^3 \rho_0}{c_s^2}
\Big[ \mathbf{U}.\sum_b w_{c_b} \mathbf{c}_b +
  \mathbf{\Omega} . \sum_b w_{c_b} \mathbf{r}_b \times \mathbf{c}_b \Big].
\end{equation}
For any particle with a full set of boundary links, both
$\sum_b w_{c_b} \mathbf{c}_b$
and $\sum_b w_{c_b} \mathbf{r}_b \times \mathbf{c}_b$ are zero.
However, if the set of boundary links is incomplete
$\Delta M_s$ is not zero and
a correction is required. This is again made by redistributing the
excess or deficit of mass over the other existing boundary nodes.
The additional contribution to the bounce-back here is
$-w_{c_b} \rho_0 \Delta M_s/A$, where
$A = \rho_0 \Delta x^3 \sum_b w_{c_b}$ (cf.\ Equation~\ref{eq:q1}).

This contribution
now adds to the velocity-dependent part of the force and torque and
leads to a slightly different form of the drag matrices
\begin{eqnarray}
\boldsymbol{\zeta}^{FU} &=& \frac{-2\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} (\mathbf{c}_b - \overline{\mathbf{c}_b}) \mathbf{c}_b
\\
\boldsymbol{\zeta}^{F\Omega} &=& \frac{-2\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} \mathbf{c}_b (\mathbf{r}_b \times \mathbf{c}_b -
\overline{\mathbf{r}_b\times\mathbf{c}_b}),
\\
\boldsymbol{\zeta}^{TU} &=& \frac{-2\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} (\mathbf{r}_b\times\mathbf{c}_b)
(\mathbf{c}_b - \overline{\mathbf{c}_b}),
\\
\boldsymbol{\zeta}^{T\Omega} &=& \frac{-2\rho_0 \Delta x^3}{c_s^2 \Delta t}
\sum_b w_{c_b} (\mathbf{r}_b\times\mathbf{c}_b)
(\mathbf{r}_b\times\mathbf{c}_b - \overline{\mathbf{r}_b\times\mathbf{c}_b}).
\end{eqnarray}
The mean quantities
\begin{equation}
\overline{\mathbf{c}_b} = \frac{\sum_b w_{c_b} \mathbf{c}_b}{ \sum_b w_{c_b}}
\end{equation}
and
\begin{equation}
\overline{\mathbf{r}_b\times\mathbf{c}_b} =
\frac{\sum_b w_{c_b} \mathbf{r}_b\times\mathbf{c}_b}{\sum_b w_{c_b}}.
\end{equation}
A little algebra will show that both occurrences of $\mathbf{c}_b$ and/or
$\mathbf{r}_b\times\mathbf{c}_b$ in the definition of the drag matrices
can be replaced by their deviation from the mean. This provides a
convenient way to compute the drag matrices (maintaining symmetry)
and computing the correct force and torque on the particle when close
to contact.

\subsection{Colloid-Colloid interactions}

Surface-surface separation between two colloids is denoted
$h = r_{12} - a_1 - a_2$ where $r_{12}$ is the centre-centre
separation, and is based on the hydrodynamic radius $a_h$.
For ease of notation all references to $a$ in this section refer to
the hydrodynamic radius $a_h$.

\subsubsection{Lubrication corrections}

Hydrodynamic lubrication interactions between two particles may
not be fully resolved on the lattice if the particles are close
to contact. The degree to which this is the case depends on the
size of the particles, and their exact discrete position on the
lattice. There will be a cut-off separation $h_c$ associated
with corrections which must be determined by calibration
\cite{nguyen-ladd2002}; the cut-off is different for the
different contributions to the interaction.

The force and torque between the pair may be corrected
by adding back a contribution taken from the known theoretical
expression for two spheres. Two contributions are handled:

\begin{enumerate}
\item {\bf Normal force}

The appropriate correction term between colloids with hydrodynamic
radii $a_1$ and $a_2$, and having separation $h < h_c$, is
\begin{equation}
\mathbf{f}_{12} = -6\pi\eta \frac{ a_1^2 a_2^2}{(a_1 + a_2)^2}
\left[ \frac{1}{h} - \frac{1}{h_c} \right]
\hat{\mathbf{r}}_{12}. ( \mathbf{U}_1 - \mathbf{U}_2)\,
\hat{\mathbf{r}}_{12}.
\end{equation}

\item
{\bf Tangential force}

The corresponding tangential component is a somewhat complex
expression:
\begin{equation}
\mathbf{f}_{12} =
-(24/15) \pi\eta A
\big[
(\mathbf{U}_1 - \mathbf{U}_2)
-  \hat{\mathbf{r}}_{12} . (\mathbf{U}_1 - \mathbf{U}_2) \hat{\mathbf{r}}_{12}
\big],
\end{equation}
where
\begin{equation}
A = \frac{a_1 a_2 (2a_1^2 + a_1 a_2 + 2a_2^2)}{(a_1 + a_2)^3}
\left[
\ln\!\left(\frac{a_1 + a_2}{2h}\right)
- \ln\!\left(\frac{a_1 + a_2}{2h_c}\right)
\right].
\nonumber
\end{equation}
\end{enumerate}

In the presence of isothermal fluctuations, there is an additional
consideration related to the dissipation related to the correction
itself. This adds an additional random force with a variance
$(2k_BT |\mathbf{f}_{12}|)^{1/2}$ to each lubrication component.

We do not implement the collective implicit update required for
stability when many particles are in mutual lubrication contact
\cite{nguyen-ladd2002}. This treats the velocity-dependent
corrections as effectively velocity independent and not as part
of the implicit update.

This means that mutual contact must be prevented
via a repulsive potential of sufficient strength to keep particles
from close approach (the update may become unstable if not). For
many purposes it may then be preferable to ignore the lubrication
corrections completely. Long-range lubrication ($h > h_c$) will
still be well represented by the lattice fluid.

\subsubsection{Hard sphere interaction}

Particles do not overlap, i.e., there is always
a hard-sphere interaction which is a function of the separation $h$:
\begin{equation}
v^{hs}(h) = \left\{
\begin{array}{ll}
\infty & h \leq 0,\\
0   & h > 0.
\end{array} \right.
\end{equation}
The hard-sphere interaction does not give rise to a force, but gives rise
to program termination if $h<0$.

\subsubsection{Soft-sphere interaction}

\label{section-colloids-soft-sphere}

A simple soft-sphere interaction is available, the basic form
of which is:
\begin{equation}
v^{ss}(r) = \epsilon (\sigma / r)^{\nu},
\end{equation}
where $\epsilon$ sets the energy scale, and $\sigma$ sets the
characteristic width. The steepness of the potential is set by
the exponent $\nu (> 0)$.

To prevent the need for calculation of long-range interactions,
the soft-sphere potential is truncated in a ``cut-and-shift''
approach. This is done in such a way as to smoothly match
both the potential and the force at the cut-off distance
$r_c$. For $r < r_c$ the potential is then
\begin{equation}
v^{ss}(r) - v^{ss}(r_c) - (r - r_c) \left.\frac{d v^{ss}}{dr}\right|_{r=r_c}.
\label{eq_ss_shift}
\end{equation}
Clearly, this potential is not exactly what we first thought of as
soft-sphere. Matching the potential smoothly ensures conservation
of energy, while matching the force smoothly prevents potential
instabilities in the molecular dynamics update.

The soft-sphere potential can be useful as a mechanism to
keep the particles from touching (which risks hard-sphere interactions),
in which case is is possible to compute the interaction as a function
of the separation $h$, instead of $r$. The force between
two particles is computed via the derivative of equation~\ref{eq_ss_shift}
with respect to $r$:
\begin{equation}
\mathbf{F}_{ij}(r) = -\left\{\frac{d v^{ss}(r)}{dr} -
\left. \frac{d v^{ss}(r)}{dr}\right|_{r=r_c} \right\} \mathbf{\hat{r}}_{ij}
\end{equation}
where $\mathbf{\hat{r}}_{ij}$ is the unit vector joining the centre of
particle $i$ to the centre of $j$.

\subsubsection{Lennard-Jones interaction}

\label{section-colloids-lennard-jones}

A cut-and-shifted Lennard-Jones interaction is available based on the
centre-centre separtion of pairs. The basic form
of the potential is
\begin{equation}
v^{lj}(r) = 4\epsilon
\left[ \left(\frac{\sigma}{r}\right)^{12} -
\left( \frac{\sigma}{r} \right)^6 \right]
\end{equation}
where $\epsilon$ sets the energy scale, and $\sigma$ is a characteristic
width. The cut-and-shift of both the potential and the associated force
is as described in the previous section.

If using the Lennard-Jones potential, some care may be required to
ensure that no hard-sphere overlaps occur at the level of the
hydrodynamic radius $a_h$.

\subsubsection{Dipole-dipole interactions}

For particles with magnetic dipoles $\mathbf{m}_i$, long-range
dipole-dipole interactions may be computed using a standard
Ewald sum for a periodic system
(e.g., \cite{allen-tildesley}, \cite{rapaport1995}).
We describe this here only briefly.

We consider particles with uniform dipoles
$\mathbf{m}_i = \mu \mathbf{s}_i$ where $\mu$ is the dipole
strength and $\mathbf{s}_i$ is
a unit vector describing the orientation of the dipole. The
basic dipole-dipole interaction is then
\begin{equation}
v^{dd} (r) = \frac{\mu_0 \mu^2}{4\pi}
\big( \mathbf{s}_i . \mathbf{s}_j
- 3(\mathbf{s}_i.\mathbf{r}_{ij})( \mathbf{s}_j .\mathbf{r}_{ij})
\big),
\end{equation}
where $\mu_0$ is the permeability. The Ewald sum decomposes this
into a short-range part contributing forces and torques computed
in real space and reciprocal-space part computed as a sum over
reciprocal space vectors. The choice of the real-space cut-off
$r_c$ determines the number of reciprocal space vectors required
in the sum.

A relevant parameter for magnetic colloid systems is the
dipolar coupling constant which measures the strength of
the dipole-dipole interaction compared with the thermal
energy $k_bT$ (which may be introduced via isothermal
fluctuations in the fluid). If we take $\mu_0 / 4\pi$ to be
unity in lattice units, the the dipolar coupling constant
is
\begin{equation}
\lambda = \mu^2 / 8k_bT a^3.
\end{equation}
\subsection{External Forces}

\subsubsection{Gravity}

For the purposes of sedimentation and other driven motions, the code
allows a uniform gravitational acceleration on colloids which
may be written as $\mathbf{g} = (g_x, g_y, g_z)$. We will consider the
simple situation where $\mathbf{g} = (0, 0, -g)$, i.e., the
negative $z$-direction is ``downwards.''

A colloid of density $\rho$ sedimenting in this gravitational field
experiences a net buoyancy force density $(\rho-\rho_0)g = \Delta\rho g$,
where $\rho_0$ is the fluid density. (If the colloid and fluid density
are equal, one can still think in the limit that $\Delta\rho \rightarrow 0$
and $g \rightarrow \infty$, but where the product $\Delta\rho g $ is finite.)

In a fully periodic system, the application of the force on the colloids
will cause a continuous and unboundaed acceleration. To prevent this, and
to balance the momentum budget,
an equal an opposite body force density is applied uniformly to fluid
sites at each time step. In a system with a ``bottom'' (for example, a
flat wall perpendicular to the gravitational force) no such correction
is required.

\subsubsection{Magnetic field}

The the presence of a uniform external magnetic field $\mathbf{B}$,
a particle with dipole $\mathbf{s}$ experiences a torque
$\mathbf{s} \times \mathbf{B}$.

\subsubsection{Electric Field}

In the presence of a uniform external electric field $\mathbf{E}$, a
particle with charge $q$ experiences a body force $q\mathbf{E}$.

% TODO
% Yukawa
% Bonds, angles, etc
% Squirmer
% Sub grid dynamics
% solid-solid links

% End section
\vfill
\pagebreak


\section{The fundamental theorem of calculus}


%===============================================================
%   A Brief History of the Deflection of Light
%===============================================================
\section{A Brief History of the Deflection of Light}

In the beginning, there was Newtonian gravity. For nearly 400 years, Newton's Laws of Motion and Gravity accurately predicted the movements of celestial bodies. The nature of light up until the late 19th and early 20th century was still a mystery, as well as its relationship with gravity. John Michell first posited that light could behave similarly as massive particles and be deflected in the vicinity of a strong gravitational potential. The computations for the deflection of light were carried out by Henry Cavendish in 1784 \citep{Cavendish:2011ts} and later Johann von Soldner in 1804 \citep{Jaki:1978wc}. This deflection angle was computed to be

\begin{equation}
\alpha_\mathrm{Newton} = \frac{2GM}{c^2b}
\end{equation}

\noindent where $M$ is the mass of the object and $b$ is the impact parameter, or distance of closest approach to $M$, $G$ is the gravitational constant, and $c$ is the speed of light.

However, Albert Einstein's relativity does not allow for massive objects and light to travel through spacetime in a similar manner, as light particles always have a spacetime interval of $ds^2=0$ (light always travels at $c$ in all reference frames). Thus, light will follow geodesics unique to those of massive particles when influenced by the bending of space and time by a massive object \citep{Einstein:1911rf}. Using the Schwarzchild metric for a point mass and following the weak field limit (i.e., $GM/c^2r$ \textless \textless\ 1 along the entire trajectory), one obtains an alternative result for the deflection of light:

\begin{equation}
\alpha_\mathrm{Einstein} = \frac{4GM}{c^2b} =  2\alpha_\mathrm{Newton}.
\label{intro:eqn:deflection}
\end{equation}

This postulate of general relativity provided the first opportunity to test the controversial theory. The best measuring man in England, Arthur Eddington, was intrigued by the German physicist's theory, and despite facing barriers put in place to prevent collusions with enemy scientists during the time of World War I, he collaborated with Einstein and devised an experiment to measure the deflection of background stars by the Sun during the 1919 solar eclipse. Einstein predicted a deflection of 1.74 seconds of arc of the stars in the background Hyades cluster. On May 29, two teams in Sobral, Brazil and the island of Principe off the coast of Africa measured a deflection of $1.98\pm0.16$" (see Figure~\ref{intro:fig:eclipse}) and $1.6\pm0.40$", respectively, both well within two sigma of the prediction by Einstein and far ruling out the Newtonian prediction of 0.87". When the article was published on November 6th, 1919, Einstein became a world-wide celebrity overnight \citep{Dyson:1920zl}.

\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{Intro/1919_eclipse_negative.jpg}
\caption[Photometric plates from the 1919 solar eclipse]{From \citet{Dyson:1920zl}, original caption: ``From the report of Sir Arthur Eddington on the expedition to verify Albert Einstein's prediction of the bending of light around the sun. In Plate 1 is given a half-tone reproduction of one of the negatives taken with the 4-inch lens at Sobral. This shows the position of the stars, and, as far as possible in a reproduction of this kind, the character of the images, as there has been no retouching. A number of photographic prints have been made and applications for these from astronomers, who wish to assure themselves of the quality of the photographs, will be considered as as far as possible acceded to."}
\label{intro:fig:eclipse}
\end{figure}

%===============================================================
%   Extragalactic Gravitational Lensing
%===============================================================

\section{Extragalactic Gravitational Lensing}

Fritz Zwicky and Albert Einstein both suggested that galaxies rather than stars would make for much stronger gravitational lenses \citep{Zwicky:1937yq,Einstein:1936cl}. And indeed, later in the 20th century, astronomers began discovering this deflection of light, or gravitational lensing as it became known, by extragalactic sources. In rare cases, the deflection can be strong enough such that the light from the background source can travel multiple paths around the deflecting, or lensing, object, thus creating multiple images of the background source. In 1979, the ``Twin Quasar" was discovered -- two quasars located unusually close together in the sky, both behind a massive foreground galaxy, shown in Figure~\ref{intro:fig:quasar}. Due to the similar redshifts and spectra of the quasars, it was deduced that they were lensed images of the same background quasar \citep{Chang:1979jk,Porcas:1979qf,Weymann:1979yu}.

\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{Intro/twin_quasar.png}
\caption[\hst\ image of the Twin Quasar]{{\it Hubble Space Telescope} (\hst) image of the famous Twin Quasar. From  ``Seeing double". ESA/Hubble Picture of the Week. Retrieved 30 November 2017. The two bright blue objects with diffraction spikes close to the central galaxy are the two images of the quasar. The other redder objects with diffraction spikes are foreground stars.}
\label{intro:fig:quasar}
%http://www.spacetelescope.org/images/potw1403a/
\end{figure}

Galaxy clusters are the most massive gravitationally-bound objects in the Universe, consisting of hundreds of galaxies clustered within a virial radius of roughly a megaparsec in size. Zwicky used galaxy clusters to hypothesize the existence of an invisible mass in galaxy clusters, as the velocity dispersion of the galaxies suggested the total masses of these clusters to be $400\times$ higher than the mass implied by the star light of the galaxies \citep{Zwicky:1937yq}. This dark matter, as it has come to be known, has been proven time and time over to exist. Only $\sim1\%$ of of the mass of a typical galaxy cluster consists of baryons contained in cluster member galaxies. Nearly $\sim90\%$ of the mass consists of dark matter. The remaining $\sim9\%$ is contained in the 10 million degree intergalactic medium, which would have been ``invisible" to Zwicky, but today is visible with Xray telescopes.

Any galaxy cluster is capable of being a weak lens; however, fewer clusters are capable of behaving as strong lenses.  In order to find these rare strong lenses, the lensed galaxies would need to be highly magnified and distorted significantly as to be seen as obvious signs of lensing. This work would have been impossible on photographic plates as faint lensed galaxies could easily be mistaken for cluster member galaxies or optical defects in the plate. Ultimately, it took the sensitivity and resolution of the CCD camera that allowed for the discovery of gravitational lenses. And indeed, the first spectroscopically-confirmed cluster lensed galaxy was discovered in the field of Abell 370 in 1986 \citep{Soucail:1988kx,Soucail:1987sf,Soucail:1987rz}, shown in Figure~\ref{intro:fig:a370}. Interestingly, we will learn more about this cluster using sophisticated gravitational lens modeling in Chapter~\ref{chap:hff_clusters}.

\begin{figure}
\raisebox{50pt}{\includegraphics[width=0.35\textwidth, trim=0 10pt 0 0, clip, angle=-28]{Intro/soucail88.png}}
\includegraphics[width=0.55\textwidth, trim=100pt 200pt 100pt 200pt, clip]{Intro/a370_hff.jpg}
\caption[Images of Abell 370 -- the first cluster lens]{Left: From \citet{Soucail:1988kx}, original caption: ``CCD frame of the giant arc in A370. This picture was obtained by J.L.~Prieur at the prime focus of the 3.60m Canada-France-Hawaii telescope on October 25th, 1987. The CCD was a $640\times1024$ RCA2: scale 0.2"/pixel -- seeing 0.7", with an exposure time of 10 minutes in white light. Note the shape of the object \#37, which was already suspected based on its spectrum \citet{Soucail:1987sf}", meaning that since the object is not at the cluster redshift, but indeed behind it and being lensed into a fantastic arc. Right: 140-orbit \hst\ image of Abell 370 from the Hubble Frontier Fields Director's Discretionary Program. From ``The last of the Frontier Fields -- Abell 370. STScI. Retrieved 30 November 2017.}
% https://www.spacetelescope.org/images/heic1711a/
\label{intro:fig:a370}
\end{figure}

%===============================================================
%   Gravitational Lensing Theory
%===============================================================

\section{Gravitational Lensing Theory}

\subsection{The one-dimensional lensing equation}

\begin{figure}
\includegraphics[width=\textwidth, trim=75pt 75pt 75pt 75pt]{Intro/lens_diagram.pdf}
\caption[Diagram of gravitational lensing parameters]{Diagram showing the definitions of angles and angular diameter distances used for deriving the lens equation.}
\label{intro:fig:diagram}
\end{figure}

Since the distances between the lens, source, and observer are much larger than the size of the lens itself, it is safe to use the ``thin lens approximation" popular in optics where we assume an instantaneous deflection of the light on a plane of lensing mass. Therefore, we envision the geometry of the lensing as shown in Figure~\ref{intro:fig:diagram}. The distances $\dl$, $\ds$, and $\dls$ are the angular diameter distances between the observer-lens, observer-source, and lens-source, respectively. We define the angle between the lens and source in the observer's frame to be $\beta$ and the angle between the lens and of the apparent position of the image of the source to be $\theta$. The angle $\hat\alpha$ is the deflection angle computed by Einstein in (\ref{intro:eqn:deflection}). Using the small-angle approximation, we can write

\begin{equation}
\theta \ds = \beta \ds + \hat\alpha \dls,
\end{equation}

\noindent and after rearranging, we get that

\begin{equation}
\beta(\theta) = \theta - \frac{\dls}{\ds} \hat\alpha,
\end{equation}

\noindent known as the lensing equation. Here, we have redefined $\beta$ as a function of $\theta$ as every image will map to only a single source, which should be fairly intuitive. We can also define another variable

\begin{equation}
\alpha \equiv  \frac{\dls}{\ds} \hat\alpha,
\label{intro:eqn:dlsds}
\end{equation}

\noindent also shown in Figure~\ref{intro:fig:diagram}, which is the angular offset between the image and the source at the location of the observer. While $\hat\alpha$ will always remain constant for a given impact parameter, the observed deflection depends on the geometry of the lensing system, i.e., the relative distances to the source and lens.

\subsection{Lensing in two-dimensions and magnification}
\label{intro:sec:magnification}

We have so far been working in one dimension considering positions of sources and images on singular axes. The solutions hold for any axisymmetrical lens; however, in nature, we must consider that mass distributions are more complex, and thus will cause a dependence on position angle of the source rather than purely the impact parameter, $\theta\dl$. In the simple case shown in Figure~\ref{intro:fig:diagram}, our one-dimensional lens has produced two point source images. This is also true if we extend the solution to two-dimensions, except in the instance where source is directly behind the lens ($\beta=0$). In this case, the solution is a circle (or ring) of radius

\begin{equation}
\theta = \alpha = \frac{\dls}{\ds} \frac{4GM}{c^2} \frac{1}{\theta \dl}
\end{equation}

\noindent from which we define the Einstein radius as

\begin{equation}
\theta_E^2 \equiv \frac{4GM}{c^2} \frac{\dls}{\dl\ds}.
\label{intro:eqn:einstein_radius}
\end{equation}

\subsection{Lens mapping}

Most sources in the Universe are not point sources and can be magnified, in that the solid angle subtended by their images is larger than that of the source. The exact definition of the magnification is the ratio of these solid angles,
$\mu \equiv \Delta \theta/\Delta \beta$. We can also treat lensing as a mathematical transformation of the shape of a source into the shapes its observed images. We can rewrite the lens equation as a transformation matrix

\begin{equation}
A_{ij} = \frac{\partial \beta_{ij}}{\partial \theta_{ij}} = \delta_{ij} - \frac{\partial{\alpha_{i}}}{\partial{\theta_{j}}} =
	\begin{bmatrix}
		1-\kappa-\gamma_1 & \gamma_2 \\
		\gamma_2 & 1-\kappa+\gamma_1
	\end{bmatrix},
\end{equation}

\noindent where we have defined the new terms

\begin{align}
\kappa &= \frac{1}{2} \left(\frac{\partial{\alpha_{1}}}{\partial{\theta_{1}}}+\frac{\partial{\alpha_{2}}}{\partial{\theta_{2}}}\right) = \frac{1}{2} \nabla_{ij} \alpha_{ij} = \frac{\Sigma}{\Sigma_{crit}} \label{intro:eqn:kappa} \\
\gamma_1 &= \frac{1}{2} \left(\frac{\partial{\alpha_{1}}}{\partial{\theta_{1}}}-\frac{\partial{\alpha_{2}}}{\partial{\theta_{2}}}\right) \\
\gamma_2 &= \frac{\partial{\alpha_{1}}}{\partial{\theta_{2}}} = \frac{\partial{\alpha_{2}}}{\partial{\theta_{1}}} \\
\gamma^2 &= \gamma_1^2 + \gamma_2^2
\end{align}

\noindent with $\Sigma$ representing the projected surface mass density of the lens plane, with critical surface mass density equal to

\begin{equation}
\Sigma_\mathrm{crit} = \frac{c^2}{4\pi G} \frac{\ds}{\dls\dl}.
\end{equation}

\noindent The formation of multiple and/or highly distorted images, occurs when $\kappa>1$. We see in (\ref{intro:eqn:kappa}) that lensing depends not only on how massive the lens is ($\Sigma$), but also its proper geometrical alignment with the source ($\Sigma_\mathrm{crit}$).

The magnification map can be determined by taking the inverse determinant of the transformation matrix:

\begin{equation}
\mu^{-1} = |\det A| = |(1-\kappa)^2 - \gamma^2|.
\end{equation}

\noindent It is possible for the magnification to be smaller than one (i.e., demagnified images) as well as become infinite. However, the area in the image plane where this occurs is infinitesimally small, so infinite magnifications fall on lines called critical curves. The eigenvectors of $A$ tell us where these lines occur:

\begin{align}
1-\kappa-\gamma &= 0 \\
1-\kappa+\gamma &= 0.
\end{align}

\noindent These describe the tangential and radial critical curves in the image plane, respectively. These are the lines of symmetry between image pairs, reflected either tangentially or radially. Image pairs will have opposite parity; they are mirror images of one another. We can use the lens equation to map these curves to the source plane, producing caustics that inform us on the number of images the lens will produce. The number of images must always be odd, with two additional images being formed for each caustic the source falls within. Often one of these images is highly demagnified and/or lies behind the bright lensing galaxy, thus is not always observed and the reason there are reports of ``double" or ``quad" configurations in the literature. Typical image configurations and their respective source locations are shown in Figure~\ref{intro:fig:image_config}.

For the remainder of this work, we will refer to ``strong" gravitational lensing as the phenomenon in which lensing results in multiple and/or highly-distorted images of the background source.

\begin{figure}
\centering
\includegraphics[height=0.8\textheight]{Intro/image_config.png}
\caption[Strong lensing caustics, critical curves, and multiple image configurations]{From \citet{Kneib:2011qy}, original caption: ``Multiple-image configurations produced by a simple elliptical mass distribution. The panel (S) shows the caustic lines in the source plane and the positions numbered 1 to 10 denote the source position relative to the caustic lines. The panel (I) shows the image of the source without lensing. The panels (1) to (10) show the resulting lensed images for the various source positions. Certain configurations are very typical and are named as follows: (3) radial arc, (6) cusp arc, (8) Einstein cross, (10) fold arc."}
\label{intro:fig:image_config}
\end{figure}

\section{Strong gravitational lens modeling}

\subsection{First-order approximation for the mass of the lens: Einstein radius}

As we found in \S~\ref{intro:sec:magnification}, sources which are almost perfectly aligned with the lens will produce an Einstein ring. By assuming that the lens is axisymmetric, we can determine the total mass within this ring from (\ref{intro:eqn:einstein_radius}), which yields

\begin{equation}
M(<\theta_E) = \pi (\theta_E \dl)^2 \Sigma_\mathrm{crit}.
\label{intro:eqn:einstein_mass}
\end{equation}

Figure~\ref{intro:fig:einstein_ring} shows examples of nearly perfect Einstein rings where this approximation can be made. Typical values for $\theta_E$ in clusters are on the order of $\sim$15", translating to physical scales on the order of a few hundred kiloparsecs. All other estimates for masses based on astronomical observables (ex., X-ray, Sunyaev-Zel'dovich effect, galaxy dynamics, weak lensing, galaxy counts, etc.) are sensitive to the masses at much larger radii ($\sim0.5-1.5$ Mpc). Thus, strong lensing provides a unique estimate for the masses at the very cores of clusters. In conjunction with other techniques, strong lensing can be used to measure the mass-concentration relationship of galaxy clusters, which helps reveal their formation history. Earlier halos tend to be more highly concentrated \citep[e.g., ][]{Wechsler:2002rz}; however, an over-concentration of clusters may be explained by exotic physics such as non-gaussianity of primordial density fluctuations \citep{Fedeli:2007sf,Mathis:2004ty} or the effects of baryons on the innermost profiles of clusters \citep{Rozo:2008mz}.

\begin{figure}
\centering
\includegraphics[height=0.3\textheight]{Intro/einstein_ring_1.png}
\includegraphics[height=0.3\textheight]{Intro/einstein_ring_2.png}
\caption[\hst\ Einstein Rings]{Two famous Einstein rings imaged with the \hst. (Left): LRG~3-757 nearly forms a perfect ring around the massive elliptical galaxy. (Right): Galaxy cluster SDSS~J1038+4849 forms a pseudo-Einstein ring with partial rings formed from several lensed background sources.}
% http://apod.nasa.gov/apod/ap111221.html
% https://en.wikipedia.org/wiki/Einstein_ring#cite_note-NASA-20150210-6
\label{intro:fig:einstein_ring}
\end{figure}

Galaxy clusters are not necessarily axisymmetric; they contain a lot of substructure and galaxies, which perturb the local lensing potential. While it is a close approximation, a precise determination of the mass distribution requires more careful modeling which can take into account shape and substructure of the cluster.

\subsection{Lens modeling}

A more precise model of the lensing mass can be achieved using the positions of multiple images. By finding a deflection field that maps all multiple images to a single position in the source plane, an estimate for the mass can be computed using (\ref{intro:eqn:kappa}). Using more sets of multiple images will help to further constrain the deflection field at different radii. After examining (\ref{intro:eqn:einstein_mass}), we see that $\Sigma_\mathrm{crit}$ is linear with $\alpha$ and thus scales in the same manner as (\ref{intro:eqn:dlsds}). Having a range of redshifts from different sources will produce a range of Einstein radii, which will help to constrain the slope of the mass distribution (such as the cluster in the right image of Figure~\ref{intro:fig:einstein_ring}).

The general approach to lens modeling involves finding the mass distribution which creates the deflection field that best reproduces the observed image configuration. These approaches fall into two categories: ``parametric" and ``non-parametric". The former involves defining the surface mass density of the cluster and its galaxies in terms of physically-motivated potentials such as an isothermal sphere or a Navarro, Frenk, and White profile \citep[NFW; ][]{Navarro:1997qa}. The cluster dark matter halo is represented by a physically-motivated potential, with cluster member galaxies added as additional perturbers. The methods for assigning mass to cluster member galaxies rely heavily on the assertion that ``light traces mass." Therefore, the total mass of the galaxy is scaled by the total light from the galaxy, usually following the empirical fundamental plane of luminosity, radius, and velocity dispersion of elliptical galaxies \citep{Gudehus:1973kq}. ``Non-parametric" methods allow for more flexible characterization of the mass distribution by employing either a pixelized array of masses or a grid of radial basis functions that can each fluctuate in total mass to affect the localized deflection near multiple images as well as the overall shape of the mass distribution. Generally, this method does not force the notion that galaxies must have mass, but allows for the image configurations to constrain the masses on smaller-scale structures as needed to reproduce the lensing effect.

The modus operandi of this author is a parametric method established by the publicly-available \texttt{LENSTOOL} software \citep{Jullo:2007lr} and embodies the lens modeling inquiry presented in this dissertation. Therefore, detailed discussions of the variants of non-parametric methods is beyond the scope of this work.

\subsubsection{\texttt{LENSTOOL}}

In \texttt{LENSTOOL}, the lensing potential is constrained using the positions (i.e., right ascension and declination) of the multiple images to determine the parameterized model of the mass which best produces the observed image configurations. While several mass distributions are built into the software, this author uses the psuedo-isothermal elliptical mass distribution \citep[PIEMD; ][]{Eliasdottir:2007ve} exclusively to map the lensing mass. The PIEMD has seven tunable parameters: two positional parameters for the location of the halo on the sky, an ellipticity, a position angle, a fiducial velocity dispersion, and a core radius and cut radius, which act to determine the inner and outer slopes of the distribution, respectively.

Models made in \texttt{LENSTOOL} consist of two types of mass components: cluster scale and galaxy scale. Cluster halos represent the overall smooth dark matter distribution of the cluster, and tend to have velocity dispersions on the order of several hundred km s$^{-1}$ and core and cut radii on the order of a few 100 kpc and 1 Mpc, respectively. As we discussed above, strong lensing is most sensitive to total mass of the cluster within a few hundred kpc, so the PIEMD is well-tuned to the scale of strong lensing. Many clusters have a single cluster scale halo while cluster mergers can have several. Additionally, dozens to a hundreds of galaxy scale halos are superposed onto the cluster representing the mass in individual galaxies. The position and shape parameters of these galaxies are extracted from their light profiles. The core radius, cut radius, and velocity dispersion are set based on the estimates of these parameters for a $L_\star$ galaxy at the cluster redshift. Following the fundamental plane relation, each galaxy halo is scaled such that $luminosity \propto (core/cut\ radius)^2 \propto (velocity\ dispersion)^4$.

Any of the parameters of the halos can be set as free or fixed. Free parameters are explored in the optimization process while fixed parameters remain constant. It is typical to allow all parameters of the cluster halo to be free with the exception of the cut radius, as it is too far away from the cluster center to be constrained by strong lensing. Galaxy halos are usually kept fixed as they provide only small perturbations to the lensing mass. However, halos of individual galaxies can be optimized when in close proximity to lensing constraints. \texttt{LENSTOOL} also has the ability to include the parameter normalizations of the scaling relations for the galaxy halos in the optimization; however, effectively constraining these parameters requires many hundreds of multiple images \citep[e.g., ][]{Richard:2014gf}.

The redshifts of the multiple images used as constraints are also considered parameters in the optimization. If the spectroscopic redshift can be measured, the redshift parameter is fixed to that value. If the redshift has been measured photometrically or not at all, then the redshift can be set to a free parameter with a prior (defined and discussed below) set according to the measurement error.

The optimization of \texttt{LENSTOOL} is a Bayesian Markov Chain Monte Carlo (MCMC). This method is extremely common and used to build astrophysical models as well as more ``down-to-earth" models, such as predicting the outcomes of political elections and sports tournaments. Below is a description of the method as it applies to strong lens modeling.

\subsubsection{Bayesian statistics and model optimization}

Regardless of the modeling method, nearly all have settled on exercising a Bayesian approach, where optimizing a model depends not only on its fitness, but also information about the parameters obtained a priori. The Bayes' theorem is defined

\begin{equation}
P(\vec{p} | D ) = \frac{P(\vec{p}) P(D | \vec{p} ) }{P(D)}
\label{intro:eqn:bayes}
\end{equation}

\noindent where $D$ are the observed data, and $\vec{p}$ contains the parameters of the model. Let us break down each component of (\ref{intro:eqn:bayes}):

\begin{itemize}
\item $P(\vec{p} | D )$: The posterior probability distribution -- the probability of the model given the data. In this instance, the data are the locations of the multiple images along with their distance from the lens and observer (i.e., redshift).
\item $P(\vec{p})$: The prior. This term is the a probability that a particular model is accurate based on {\it a priori} information.
\item $P(D | \vec{p} )$:  The likelihood of getting the data given the parameters of the model. Non-Bayesian modeling focuses this term in optimization (i.e., maximum likelihood estimation) and de-regulates the prior.
\item $P(D)$: The evidence. This term is the probability that a particular model is valid independent of all other variables, which effectively acts as Occam's razor: ``All things being equal, the simplest solution tends to be the best one." The evidence normalizes the posterior probability distribution.
\end{itemize}

Bayes' theorem can easily be applied to strong lens modeling once we define a likelihood function to include in our optimization. The likelihood function will take the form

\begin{equation}
P(D | \vec{p} ) = \prod_{i=1}^N \frac{1}{  \prod_{j=1}^{n_i} \sigma_{ij} \sqrt{2\pi} } \exp^{-\frac{\chi_i^2}{2}},
\label{intro:eqn:likelihood}
\end{equation}

\noindent where $N$ is the number of sources, $n_i$ is the number of images of each source $i$, and $\sigma_{ij}$ is the measurement of error of image $j$ of source $i$. By minimizing $\chi^2$, we will produce the maximum likelihood model. Our goal in lens modeling is to find a solution with the smallest scatter between the source positions mapped to each of its images. Defining the likelihood in this manner is called source plane optimization. While this still solves the lens equation, it is not ideal as the position of the sources are unknown to the observer. The more appropriate method would be image plane optimization, which involves an extra step of ray tracing the source position of each image back out to the image plane and measuring the scatter of the predicted images. However, this second step is much more computationally intensive due to the ray tracing, which is required because the lens equation is not invertible. The $\chi_i^2$ function for each source $i$ will then take the form

\begin{equation}
\chi^2_i = \sum_{j=1}^{n_i} \frac{[\theta_\mathrm{obs}^j - \theta^j (\vec{p})]^2}{\sigma_{ij}^2},
\label{intro:eqn:chi2}
\end{equation}

\noindent where $\theta^j (\vec{p})$ is the predicted image position ray traced to the observed image position $\theta_\mathrm{obs}^j$. Examining this equation further, we can see that image plane optimization will ensure that models which produce additional images than those observed are likely to be rejected as they will produce much higher $\chi^2$ than models with fewer images (smaller $n_i$).

It should be noted that this method could be improved by including additional lensing information in the optimization, such as magnification, time delays, and/or flexion (i.e., shape), which could be added in quadrature to (\ref{intro:eqn:chi2}). Some lens modeling methodologies do include these terms; however, for this work, we will only consider image positions as the primary constraint.

\subsubsection{Computational methods}

The \texttt{LENSTOOL} software utilizes a Markov Chain Monte Carlo (MCMC) to sample parameter space to determine the shape of the posterior probability distribution. In the MCMC burn-in phase, ``walkers" of models are initialized with a set of parameters randomly sampled from the current posterior probability distribution. At the first stage, the likelihood has not been sampled yet, so it is not included in the initial determination of the posterior. This is done by replacing the likelihood in Bayes' theorem (\ref{intro:eqn:bayes}) with the term $P(D | \vec{p} )^\lambda$, where $\lambda$ is a ``cooling" parameter. At the beginning, $\lambda=0$, so in the analogy to thermodynamics, the model selection is ``hot" and samples from a wide range of values in parameter space, ensuring that the MCMC can find a global maximum likelihood. The posterior probability for each walker is computed, and a new $\vec{p}$ is selected for each walker and given a probability to jump based on the ratio of the posterior probability of the current position and the new position. Walkers will tend to jump to higher posterior probabilistic positions in parameter space as the chain continues to form, all while $\lambda$ is slowly increasing from 0 to 1 at a rate chosen to insure a smooth convergence towards a colder posterior probability distribution. After $\lambda=1$, the burn-in phase is complete and the chains are re-initialized from the last position in parameter space for all the walkers. Using the full Bayes' theorem, a new MCMC runs to sample the posterior probability distribution.

\subsection{Data for cluster lensing}

Lens modeling has benefited from the imaging CCDs aboard the {\it Hubble Space Telescope} (\hst). The high sensitivity and high resolution allow for easier detection and identification of multiple image systems to be used in the modeling. The first remarkable instance of using \hst\ for cluster lensing of Abell 2218 \citep{Kneib:1996kb} imaged with the Wide Field Planetary Camera 2 (WFPC2) is shown in Figure~\ref{intro:fig:lensing_clusters} (with more recent imaging added).

\begin{figure}
\centering
\includegraphics[height=0.3\textheight]{Intro/a2218.jpg}
\includegraphics[height=0.3\textheight]{Intro/a1689.jpg}
\caption[Abell 2218 and Abell 1689]{\hst\ imaging of Abell~2218 (left) and Abell~1689 (right).}
\label{intro:fig:lensing_clusters}
\end{figure}

The servicing mission for \hst\ in 2002 gave us the new Advanced Camera for Surveys (ACS), which greatly enhanced optical imaging capabilities. \citet{Broadhurst:2005qy} and \citet{Halkola:2006ss} were able to map the mass distribution in fine detail using over a hundred multiple images of background sources of Abell 1689 with the new ACS data, shown in Figure~\ref{intro:fig:lensing_clusters}. Furthermore, the Wide Field Camera 3 (WFC3) installed during the final servicing mission in 2009 ignited the field of cluster strong lensing with several groups working to develop new lens modeling techniques. This camera has both ultraviolet, optical, and near-infrared imaging sensitivity, allowing for a wide bandwidth coverage of lensing clusters.

The {\it Spitzer Space Telescope} has also greatly added in the field of cluster lensing. While it cannot achieve the same resolution as \hst, it can greatly help distinguish between foreground and background objects in the field of view with adding data to galaxies' spectral energy distribution (SED) in the far-infrared.

\section{Fantastic lenses and where to find them}

The selection techniques for galaxy clusters exhibiting strong gravitational lensing can generally be split into two categories: ``high mass" and ``high magnification." Galaxy clusters may be selected by mass for lensing, as those more massive will tend to provide a larger cosmological volume for which $\kappa>1$, thus having high cross sections for lensing many background galaxies. However, galaxies need not be massive to lens a single galaxy to a very high magnification ($\mu>30$). The techniques for finding these clusters are different as are their scientific applications.

\subsection{Surveys for finding clusters}

Some of the first galaxy clusters were identified in wide-field optical surveys. For example, the Abell clusters \citep{Abell:1958mz,Zwicky:1968rm,Abell:1989ly} were found in large surveys of photographic plates and identified based on the clustering of several massive red galaxies located within a few arcminutes of one another. Generally speaking, more massive clusters tend to have higher galaxy membership, or ``richness." Abell 1689 and Abell 370, as we discussed earlier, are prime examples. With digitized data, clusters can be found in optical/near-infrared surveys using algorithms aimed at identifying clusterings of galaxies close in photometric colors and by identifying the red sequence of early-type galaxies on a color-magnitude diagram \citep{Rykoff:2014rz,Gladders:2000kq}.

With the advent of Xray telescopes \citep{Giacconi:1968ly}, more massive clusters were able to be identified. A hot ionized intercluster medium fills the space between galaxies. In order for this gas to maintain hydrostatic equilibrium within the gravitational potential of the cluster, it must be at a temperature of order $10^7$ to $10^8$ K, with $M\propto T_X^{3/2}$ based on theoretical arguments for a relaxed system \citep{Horner:1999rz}. Plasmas at this temperature will emit high energy photons on the order of a few keV via thermal bremsstrahlung radiation. Massive clusters ($T>5$ keV) can be targeted and identified as lensing clusters with optical imaging. The Massive Cluster Survey \citep[MACS; ][]{Ebeling:2001rt} carried out this method by applying Xray brightness and hardness cuts to sources in the ROSAT All Sky Survey and cross-matched with optical surveys and follow-up observations to find the most massive galaxy clusters at $z>0.3$. The selection effect for Xray-selected clusters is redshift-dependent as a result of cosmological dimming of the emission.

The hot intercluster medium also provides a second means for detection. Photons from the cosmic microwave background (CMB) are cooler than the gas surrounding galaxy clusters and can receive an energy boost from inverse Compton scattering. This phenomenon is known as the Sunyaev-Zel'dovich effect \citep{Sunyaev:1972lr}. As a result, the CMB photons passing through a galaxy cluster will ``disappear" at the frequencies below the thermal null frequency of the CMB at $\sim220$~GHz and ``reappear" at higher frequencies. Over a thousand clusters have been found in the Planck all-sky survey of the CMB \citep{Planck-Collaboration:2014gf}. More massive clusters will have a stronger CMB decrement and there is no cosmological dimming of the signal, thus the sample is mass-limited. However, the large seven arcminute beam can cause significant beam-dilution for sources with angular sizes smaller than the beam, and therefore also takes a hit at detecting clusters at higher redshifts. Other surveys work with smaller beam sizes and thus, while still mass-limited, are able to push cluster detections out past $z=1$. The most massive galaxy at $z=0.87$, nicknamed ``El Gordo" \citep{Menanteau:2012ul, Menanteau:2010fu}, was found along with nearly 100 other clusters by the Atacama Cosmology Telescope \citep[ACT; ][]{Hasselfield:2013pd,Marriage:2011qf}. The South Pole telescope \citep[SPT; ][]{Bleem:2015gf} has detected nearly 900 clusters over a 2500 square degrees.

\begin{figure}
\centering
\includegraphics[width=\textwidth]{Intro/spt.png}
\caption[Masses and redshifts of cluster detections]{From \citet{Bleem:2015gf}: The masses and redshifts of optically-confirmed clusters discovered in the SPT, ACT, Planck and ROSAT surveys.}
\label{intro:fig:cluster_detections}
\end{figure}

Figure~\ref{intro:fig:cluster_detections} shows these clusters plotted in mass and redshift. It is important to note that mass is not a direct quantity that any of the methods above can measure. Our best way of measuring the total mass of a galaxy cluster is through weak gravitational lensing, where the integrated surface mass density along the line of sight is measured using the slight distortions in the shapes of background galaxies to map the shear. Since galaxies can intrinsically have elongated shapes, thousands of background sources are needed in order to determine a statistically significant measurement for the shear across the entire field of the cluster. Therefore, high-resolution ($<$1"), wide-field (several square degrees) imaging is needed for these measures, as well as accurate estimates for the redshifts of the background sources, in order to eliminate the mass sheet degeneracy \citep{Schneider:1995vn}. The ``Weighing the Giants" project has measured weak lensing mass and systematics for 51 massive clusters based on imaging from both the {\it Subaru} and {\it Canada-France-Hawaii Telescopes} \citep{von-der-Linden:2014wd}, which can be compared with other scaling relations for mass.

\subsection{Discovering galaxies at cosmic dawn}

``High mass" clusters are the best for finding the most distant galaxies in the Universe. These techniques combine two powerful tools: massive lensing clusters and \hst\ along with deep integrations on the scales of hundreds of orbits. Using lensing can be powerful but makes analysis of determining a luminosity function more difficult. Lensing magnifies not only the sources, but the cosmological volume for which galaxies can be significantly magnified. Because of the smaller survey volume, the counts of lower redshift galaxies in lensed fields can be slightly lower than in unlensed fields, like those of Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey \citep[CANDELS; ][]{Grogin:2011ly} and the Hubble Ultra Deep Field \citep{Beckwith:2006rt}. However, the cosmological volume containing high magnifications is larger for higher redshifts, so we will see a boost in counts of galaxies at $z>8$. Our best hopes at finding sources with double digit redshifts rely on cluster lensing.

The Cluster Lensing and Supernova Search with {\it Hubble} \citep[CLASH; ][]{Postman:2012lr} used this technique on 20 Xray-selected galaxy clusters and 5 known lensing clusters to be imaged in 16 bands of \hst\ ACS and WFC3. The high photometric resolution allows for more accurate lens modeling, both strong and weak, as well as more accurate photometric redshifts obtained from the well constrained spectral energy distributions of the galaxies from the photometry \citep{Jouvel:2014qy}. CLASH provided the means to set more stringent limits on the star formation rate density at $z=9-10$ \citep{Bouwens:2014zp} as well as precise measurements of the mass-concentration relationships of the lensing clusters themselves \citep{Merten:2015rz,Meneghetti:2014ys}. Several exciting discoveries were also made: a triply-imaged $z\sim11$ candidate was found to be lensed by a cluster MACSJ0647.7+7015 at $z = 0.591$ \citep{Coe:2013tg}, a spectroscopically confirmed, quintuply-imaged $z=6.11$ galaxy was found lensed by Abell S1063 \citep{Monna:2014lr,Balestra:2013uq}, and three lensed supernovae at $z=0.85,1.14,1.28$ were discovered \citep{Patel:2014kl}, two of which were Type Ia and could be used to challenge the magnification predictions of the lens models.

In 2013, a director's discretionary time (DDT) was approved for \hst\ over the course of Cycles 22-24 to image 6 massive clusters, 140 orbits each and in 7 ACS and WFC3/IR bands. This survey, the Frontier Fields \citep{Lotz:2017gd} aimed at pushing beyond the limits of CLASH and predicted to yield up to 70 $z>9$ candidates \citep{Coe:2015qf}. As this was a DDT program, the data are publicly available immediately when downloaded from the telescope, and after a short while, high level data reductions provided by the Space Telescope Science Institute (STScI). Therefore, in efforts to level the playing field for the high redshift community with or without lensing experience, the lens models were also made publicly available (a discussion on this process and the creation of the models is given in Chapter~\ref{chap:hff_clusters} of this dissertation).

\subsection{High-resolution galaxies at cosmic noon}

Finding galaxies lensed to extraordinary magnifications is extremely rare; however, it is possible to find them buried in data sets cataloging thousands of clusters. Scanning through thousands of cut out images of galaxy clusters in optical surveys for the signatures of strong lensing can be a painstakingly tedious process for any individual to endure. But, believe it or not, a lot of dedicated astronomers have carried out this task on data. These projects include the Red Sequence Cluster Survey \citep[RCS;][]{Gladders:2003zr}, the Sloan Giant Arc Survey \citep[SGAS; Gladders~et~al. in preparation, ][see Figure~\ref{intro:fig:gallery}]{Hennawi:2008mz}, SPT optical follow-up \citep{Bleem:2015gf}, and the Dark Energy Survey \citep[DES; ][]{Diehl:2017zh}. Ways of making these tasks less daunting include taking advantage of public interest using citizen science projects like Space~Warps\footnote{https://spacewarps.org/} and by utilizing deep learning algorithms to search large datasets for the signatures of strong lensing \citep{Lanusse:2018zv,Nord:2016kc}.

\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{Intro/gallery.png}
\caption[Gallery of lensing galaxy clusters discovered in SGAS]{A selection of the best lensing clusters found in the Sloan Giant Arc Survey. Each image is $gri$ 5 min exposures with {\it Gemini}/GMOS. Image credit: M.~Bayliss.}
\label{intro:fig:gallery}
\end{figure}

As difficult as it can be to find strong lensing clusters, it is worth it. While it could take weeks or months to visually inspect a dataset for strong lenses, this effort provides us with data necessary to do high resolution galaxy morphology studies at $z\sim2$ that will not be possible to obtain for the general population of galaxies for several years or decades when better telescope technologies come online. Lensing amplifies the sizes of galaxies in the sky by factor of roughly the square root of the magnification, allowing us to peer into the fine details of a galaxy, even with only a few orbits of \hst\ data. Chapters~\ref{chap:s1110}-\ref{chap:clumps} of this dissertation show an example of how lensing can allow us to break the kiloparsec resolution limit for galaxies at $z\sim2$.

As we will discuss in the next section, determining an accurate magnification can be difficult and imparts systematic errors into several derived quantities. However, in most cases, any quantities derived from colors are unbiased because lensing is color-independent. Therefore, lensed galaxies are great laboratories for spectroscopic studies, as observing a single giant arc achieves the same signal-to-noise ratio as a field galaxy in 1/1000th of the exposure time. This allows for a case-by-case study of high redshift galaxies \citep[e.g., ][]{Rigby:2018hs,Rigby:2017yb} rather than relying on stacked spectra from thousands of galaxies \citep[e.g., ][]{Shapley:2003fk}.

\section{Systematics of strong lens modeling}

Strong lens modeling of galaxy clusters has come a long way from where it began over a decade ago. The earliest models often did not provide any statistical errors as doing so at the time would be extremely computationally intensive. Computing power has greatly increased and nearly all lens modeling codes now implement some version of a Bayesian MCMC to adequately sample the parameter space of a lens model from which calculating statistical errors is trivial.

The Frontier Fields lens models were a great experiment for the lens modeling community in the context of beginning to understand systematics. Several teams constructed models with their own techniques using the same lensing information and their models did not produce the same results. Figure~\ref{intro:fig:hff_models} show the differences in the regions of high magnification for the cluster Abell 2744. These modeling variations were of concern as the systematic errors in magnification across the field of the cluster translate directly to systematics in the derived luminosity functions: both in the corrections for intrinsic source luminosity as well as for the size of the survey volume. This prompted the Frontier Fields lens modeling comparison project \citep{Meneghetti:2016xe}, where several lens modeling softwares were put to the challenge of modeling two simulated clusters synthetically ``observed" through the eyes of \hst\ to the same depths as the Frontier Fields. The models produced reliable mass models of the clusters, likely as a result of the many multiple images and constraints provided with full knowledge of the source redshifts. This test was reassuring as the Frontier Fields lens models, with full depth data and several spectroscopic campaigns, are less vulnerable to systematics.

\begin{figure}
\includegraphics[width=\textwidth]{Intro/hff_models.png}
\caption[Magnification maps of Abell~2744 produced by Frontier Fields lens modeling teams]{From \citet{Coe:2015qf}, original caption: ``Magnification maps (log grayscale) for $z=9$ galaxies lensed by A2744 according to all seven submitted gravitational lensing models. The Frontier Fields WFC3/IR FOV (136"$\times$123") is outlined in red. At bottom right is a color HST image \citep[produced with Trilogy; ][]{Coe:2012px} showing the Frontier Fields WFC3/IR observations (red channel) within the prior ACS observations (blue-green). North is up and East is to the left."}
\label{intro:fig:hff_models}
\end{figure}

It is clear, though, that there was significant systematic error in the lens models of the Frontier Fields produced prior to when the data were taken. These models were built with far few constraints and many only had a few spectroscopic redshifts of the background galaxies. Therefore, a major source of the systematics is in the selection of constraints themselves. The systematic errors in mass and magnification for a cluster like those of the Frontier Fields benefit from increased numbers of constraints and redshifts. However, there is a point where the global systematic errors begin to saturate and are likely a product of the positions and redshift distributions of the constraints themselves. Areas in the image plane where constraints are nearby will have lower statistical and systematic errors, especially if the redshift of that source galaxy is known. This will be discussed further in Chapter~\ref{chap:ares_systematics} and Chapter~\ref{chap:sim}.

\section{Dissertation overview}

This dissertation consists of five chapters demonstrating the use of strong gravitational lens modeling to study the intermediate and high redshift universe, as well as how to better understand and improve our methods when it comes to systematic errors.

Chapter~\ref{chap:hff_clusters} describes the lens models of the six Frontier Fields clusters. These models were created in 2014 using archival \hst\ imaging supplemented by ground-based spectroscopic observations, including some provided within this chapter. The Frontier Fields lens models created by this author are complementary to several others created using various other lens modeling techniques, all of which are publicly available to the users of the Frontier Fields data sets.

Chapter~\ref{chap:ares_systematics} begins to quantify the systematics involved in this author's lens modeling techniques based on constraint selection and redshift information. Hundreds of lens models of the simulated cluster Ares were realized using different numbers of images and varying levels of precision for the redshift information of the sources. Therefore, we were able to quantify the dependency of errors on mass, magnification, and image plane scatter as they relate to the availability of constraints.

Chapter~\ref{chap:s1110} provides a full lensing analysis of the cluster \cluster. It begins with a description of the hybrid lens modeling technique used to model the mass of the cluster. Then, a forward modeling technique is described and implemented on the giant arc \giantarc\ to create a source plane reconstruction of the source galaxy.

Chapter~\ref{chap:clumps} builds on the previous chapter's source plane reconstruction to report both the sizes and star formation rates of the clumps within the galaxy. These clumps are then compared to the literature, showing an unprecedented level of resolution compared to field galaxies and other source plane reconstructions of lensed galaxies using different methods.

Chapter~\ref{chap:sim} looks briefly at quantifying the systematic errors in a simulated galaxy cluster not all that different from \cluster, where there are very few constraints and spectroscopic redshifts. We will investigate how the effects of adding more spectroscopic redshifts changes the systematic errors in both mass and magnification.

Chapter~\ref{chap:future} provides insight into the future directions of lens modeling in ``the era of precision lensing", where our lens modeling techniques are sophisticated and we have now the computational power to compute the statistical errors on quantities on the levels of a few percent. Fully quantifying systematic errors in the realm of clusters with few constraints and redshifts will be vital in the era of large surveys to come.


\documentclass{article}

%%  Latex generated from POD in document /cygdrive/i/tim/src/pl/reason/mypod_tmp.pl
%%  Using the perl module Pod::LaTeX
%%  Converted on Sat Aug 30 22:29:54 2003

\usepackage{makeidx}
\makeindex

\input{mypodstyle}

\tableofcontents

\clearpage
\section{Demo of \texttt{Randoms}\label{Demo_of_Randoms}\index{Demo of Randoms}}

Demo stuff

\section{Header\label{Header}\index{Header}}
\subsection{Loads */\label{Loads_}\index{Loads */}}
\begin{verbatim}
 :- ensure_loaded([randoms,demo]). /*
\end{verbatim}
\section{Body */\label{Body_}\index{Body */}}
\begin{verbatim}
  eg(any1) :-
    forall(any(member(X,[every,good,boy,deserves,fruit])),
           format('~w\n',[X])).
\end{verbatim}
\begin{verbatim}
  eg(rand0) :-
    Rand1 is rand,
    format('~w is a random number between 0 and 1.\n', [Rand1]).
\end{verbatim}
\begin{verbatim}
  eg(rand2) :-
    Rand2 is rand(10,20),
    format('~w is a random number between 10 and 20.\n', [Rand2]).
\end{verbatim}
\begin{verbatim}
  eg(rand3) :-
    R1 is rand(10,20,0.9),    R2 is rand(10,20,0.9),
    R3 is rand(10,20,0.9),    R4 is rand(10,20,0.9),
    R5 is rand(10,20,0.9),    R6 is rand(10,20,0.9),
    R7 is rand(10,20,0.9),    R8 is rand(10,20,0.9),
    R9 is rand(10,20,0.9),    R10 is rand(10,20,0.9),
    Nums=[R1,R2,R3,R4,R5,R6,R7,R8,R9,R10],
    format('~w\nare random numbers 90\% between 10 and 20.\n', [Nums]).
\end{verbatim}
\begin{verbatim}
  eg(normal2) :-
    R1 is normal(10,2),    R2 is normal(10,2),
    R3 is normal(10,2),    R4 is normal(10,2),
    R5 is normal(10,2),    R6 is normal(10,2),
    R7 is normal(10,2),    R8 is normal(10,2),
    R9 is normal(10,2),    R10 is normal(10,2),
    Nums=[R1,R2,R3,R4,R5,R6,R7,R8,R9,R10],
    format('~w\nare random numbers from normal(10,2).\n', [Nums]).
\end{verbatim}
\begin{verbatim}
 eg(gamma2) :-
    R1 is gamma(10,2),    R2 is gamma(10,2),
    R3 is gamma(10,2),    R4 is gamma(10,2),
    R5 is gamma(10,2),    R6 is gamma(10,2),
    R7 is gamma(10,2),    R8 is gamma(10,2),
    R9 is gamma(10,2),    R10 is gamma(10,2),
    Nums=[R1,R2,R3,R4,R5,R6,R7,R8,R9,R10],
    format('~w\nare random numbers from gamma(10,2).\n', [Nums]).
\end{verbatim}
\begin{verbatim}
 eg(gammas) :-
        N=1000,
        Mean=10,
        Alpha=1,
        eggammas(N,Mean,Alpha).
\end{verbatim}
\begin{verbatim}
 eg(seed) :-
        N=1000,
         egseed(N).
\end{verbatim}
\begin{verbatim}
  eggammas(N,Mean,Alpha) :-
        bagof(X,I^(between(1,N,I),
                   X is gamma(Mean,Alpha)), L),
         sum(L,Sum),
         Temp is Sum/N,
         format('got ~w, expected ~w\n',[Temp,Mean]).
\end{verbatim}
\begin{verbatim}
 egseed(N) :-
        bagof(X,I^(between(1,N,I),
                   X is rand), L),
         sum(L,Sum),
         Temp is Sum/N,
         format('got ~w, expected 0.5\n',[Temp]).
\end{verbatim}
\begin{verbatim}
 sum([H|T],Sum) :- sum(T,H,Sum).
\end{verbatim}
\begin{verbatim}
 sum([],Out,Out).
 sum([H|T],Temp,Out) :- New is H+Temp, sum(T,New,Out). /*
\end{verbatim}
\section{Footer\label{Footer}\index{Footer}}
\subsection{Start-ups */\label{Start-ups_}\index{Start-ups */}}
\begin{verbatim}
 %:- egs.
\end{verbatim}
\printindex

\end{document}


\section{Multi-Point Search}\label{sec:multi_point}
As has been mentioned, the multi-point selection perturbative hyper-heuristic employs a genetic algorithm which explores the space of heuristic combinations. This approach is similar to that of Raghavjee and Pillay \cite{raghavjee2015genetic} in that low-level heuristic combinations are evolved to produce the combinations that best improve on an initial solution. In addition, Evohyp \cite{pillay2017evohyp} - a Java toolkit for evolutionary algorithm hyper-heuristics - was used for the implementation of the genetic algorithm.

A single chromosome in the genetic algorithm population is represented as a heuristic combination - for example, \emph{AEBDACA}. To evaluate the chromosome, the heuristic combination is applied to the initial solution to produce a new solution. Each low-level heuristic is applied to the solution until the solution has been changed \emph{n} times, where \emph{n} has again been arbitrarily chosen as the number of exams that need to be scheduled for the problem instance. The means of applying a perturbative heuristic sequence is shown in algorithm \ref{alg:multi_point}.

Following the evaluation a heuristic sequence, the fitness of a solution is calculated in the same manner as suggested by Pillay \cite{pillay2010evolving} in that fitness is a measure of the soft constraint cost, multiplied by the hard constraint cost incremented by one.

\begin{algorithm}[H]\label{alg:multi_point}
\SetAlgoLined
 currentSolution = initialSolution\;
 \BlankLine
 \For{$i\gets1$ \KwTo $n$}{
    heuristic = H.elementAt($i \mod H.length$)\;
    currentSolution = heuristic.applyTo(currentSolution)\;
 }
 \BlankLine
 return currentSolution\;
 \caption{Applying a Perturbative Heuristic Sequence}
\end{algorithm}

Furthermore, the genetic algorithm employs tournament selection as the selection method and uses crossover and mutation as genetic operators - these techniques are all supported by the Evohyp \cite{pillay2017evohyp} toolkit. Of course, as is the case with any genetic algorithm approach, there are specific parameters required by the algorithm. The parameters, in this case, were largely chosen based on proportions that have been used previously with similar approaches to the same benchmark set \cite{pillay2010evolving, raghavjee2015genetic}. However, many parameters were scaled down to smaller values. The genetic parameters used were selected as follows:

\begin{itemize}
    \item The Population Size was selected as 50.
    \item The Tournament Size was selected as 5.
    \item The Number of Generations was selected as 10.
    \item The Mutation Rate was selected as 0.3.
    \item The Crossover rate was selected as 0.7.
    \item The Maximum Initial Length was selected as 20.
    \item The Maximum Offspring Length was selected as 20.
    \item The Mutation Length was selected as 5.
\end{itemize}