\chapter{{\tt DSTree}: \break A Domain/Separator Tree Object}
\label{chapter:DSTree:intro}
\par
The {\tt DSTree} object represents a recursive partition of a
graph, as is constructed during a nested dissection procedure.
The graph is split by a vertex {\it separator} into subgraphs,
and this process continues recursively up to some point.
A subgraph which is not split is a {\tt domain}.
The {\tt DSTree} object is normally created by the {\tt GPart}
graph partitioning object and then used to determine the
stages vector used as input to the {\tt MSMD} multistage minimum
degree object.
\par
The {\tt DSTree} object contains a {\tt Tree} object that stores
the natural tree links between separators and domains.
The top level separator has no parent.
Once a separator $S$ splits a graph, each subgraph is either split
again (in this case $S$ is the parent of the separator that splits the
subgraph) or $S$ is the parent of the subgraph (which is a
domain).
The {\tt DSTree} object also contains an {\tt IV} object that stores
a map from the vertices to the domains and separators.
\par
The {\tt DSTree} object is a natural output from a nested
dissection or other graph partitioning algorithm that uses vertex
separators.
Presently it has only one active function --- it creates a map from
the vertices to the {\it stages} needed as input for the
multi-stage minimum degree algorithm (see the {\tt MSMD} object).
Multisection orders the vertices in two stages: all vertices in the
domains first, then the vertices in the separators.
Nested dissection orders the vertices in as many stages as there
are levels in the {\tt DSTree} object.


\documentclass[]{interact}
\usepackage[outdir=./]{epstopdf}
\usepackage[natbibapa,nodoi]{apacite}
\usepackage{mathtools, fix-cm}
%\usepackage[nomarkers, nolists, tablesfirst]{endfloat}
%\renewcommand{\efloatseparator}{\mbox{}}

\title{A variation on the Chamberlin trimetric map projection}

\begin{document}
\maketitle
\begin{abstract}%max 250 words
   We present a variation on the Chamberlin trimetric map projection. This new
   projection, which we call the matrix trimetric projection, consists of a
   linear transformation of the squares of the distances between a given point
   and three control points. The formula of the forward projection is simpler
   than the Chamberlin projection, and admits an inverse formula which requires
   numerical iteration of only one parameter. We make comparisons between the
   two projections using a representative list of control points. The Chamberlin
   trimetric projection outperforms the matrix trimetric projection on measures
   of angle deformation and scale deformation, but the opposite is true for a
   measure of distance deformation, and the difference between the results of
   the projections is small over all measures. The forward Matrix trimetric
   projection can be calculated in half the time of the Chamberlin trimetric
   projection. We conclude that the matrix
   trimetric projection is a viable alternative to the Chamberlin trimetric
   projection, especially if an inverse is required or speed is important.

   \textit{Keywords: Chamberlin trimetric, matrix trimetric, projection,
   cartography, distortion, distance, triangulation}

\end{abstract}
\newpage

\section{Introduction}
Any map projection must introduce some form of distortion. In particular, no
map projection can be both conformal (angle-preserving) and equal-area.
Projections that eliminate one form of distortion often introduce a great deal
of another form of distortion. For example, many conformal projections
drastically enlarge some areas compared to others, most notoriously the
Mercator projection, but also the stereographic and the Lambert conformal conic
projections. \citep{snyder87} A compromise map projection is one that seeks to
balance different kinds of distortion: allowing some amount of different kinds
of distortion can produce a map of a portion of the globe
in which none of the distortion is conspicuous.

The Chamberlin trimetric projection (CTP) is such a compromise map projection.
It is named for Wellman Chamberlin, a chief cartographer for the National
Geographic Society, which has published wall and atlas maps using his
projection. \citep{christensen} The CTP is appropriate for mapping whole
continents and large portions of continents. It is based on a simple geometric
construction, using three arbitrary control points on the globe. The control
points are chosen so that the control triangle contains or mostly contains the
geographic region of interest. The CTP may be thought of as a form of
triangulation. It can also be considered part of an ``$n$-metric'' family of
projections with the azimuthal equidistant projection and the two-point
equidistant projection, which depend on measurements from a single point and
pair of points, respectively. \citep{snyder87} Being a compromise projection,
the CTP does not perfectly preserve angle, area, or distance. Most often, the
CTP is used with the spherical approximation of the Earth: for practical
ellipsoids, distortions due to the spherical approximation are negligible
compared to distortions due to the projection itself.

Modern computerized GISes require algebraic formulas, which are somewhat
involved for the CTP. These formulas are presented in \citet{christensen},
and can be seen in the source code to its implementation in \citet{proj}.
In general, the length of the algorithm relates to its speed of execution.
In particular, the algorithmic form of the CTP has branching logic depending on
where the given point lies with respect to the geodesics between control points,
which \citet{christensen} observed can cause numerical issues.

We present a new map projection called the matrix trimetric projection
(MTP) as an alternative to the CTP. A geometric construction related to the
CTP results in a map projection that is similar but has a simpler formula. This
formula is a closed-form matrix function of the squares of the distances, and
allows for a tractable inverse. As the derivations involve matrices and linear
algebra, refer to a text such as \citet{strang80} if needed.

\section{Preliminaries}
These projections will almost always be implemented using the spherical
approximation. Formulas for that approximation are given shortly, and we'll
refer to ``the sphere'' instead of ``the ellipsoid''. In fact, the forward
formulas for the CTP and MTP are agnostic of the surface to be projected,
depending only on the result of the distance function.

We denote points on the sphere by $\mathbf v$ or subscripted versions.
$\mathbf v$ is a 3-vector with unit length lying on the unit sphere.
Using this vector form allows us to take advantage of linear algebra.
Points specified by latitude $\varphi$ and longitude $\lambda$
can be converted to a unit vector as so: \citep{kent}
\begin{equation}
\mathbf v =
\begin{bmatrix*}
  x \\ y \\ z
\end{bmatrix*}
=
\begin{bmatrix*}
 \cos(\varphi) \cos(\lambda) \\
 \cos(\varphi) \sin(\lambda) \\
 \sin(\varphi)
\end{bmatrix*}.
\end{equation}
The reverse conversion is:
\begin{equation}\begin{split}
  \varphi =& \arctan\left(\frac{z}{\sqrt{y^2 + x^2}}\right), \\
  \lambda =& \arctan\left(y, x\right),
\end{split}\end{equation}
where the 2-variable form of $\arctan$, commonly called \texttt{arctan2} or
\texttt{atan2} in numeric software libraries, is used for $\lambda$.

One statement of the distance function on the sphere is
\begin{equation}
d\left(\mathbf v_i, \mathbf v_j\right)
= R \arccos\left(\mathbf v_i \cdot \mathbf v_j\right)
\end{equation}
where $R$ is the radius of the sphere. We use a radius of 6,371 km,
approximately equal to the most common mean radii of the Earth. \citep{snyder87}
There are equivalent functions in terms of vectors
that are more numerically stable for certain range of arguments,
but this is the form we will use for derivations.

We denote points in the Euclidean plane by $\mathbf p = [x, y]$ and
subscripted versions. Planar distances are found using the usual
Euclidean norm: $\|\mathbf p_a - \mathbf p_b\|$.

Let $\mathbf v_1$, $\mathbf v_2$, $\mathbf v_3$ be control points on the
sphere, in counter-clockwise order, and let $\mathbf V$ be the matrix having
$\mathbf v_i$ as its $i$th column. Let $\mathbf p_1$ etc. be the control points
on the plane, in corresponding order, and $\mathbf P$ be the matrix having
$\mathbf p_i$ as its $i$th column. The triangles with vertices at $\mathbf v_i$
or $\mathbf p_i$ are called the control triangles: spherical or planar control
triangle, respectively, if the distinction is important. For this paper,
triangles with small or zero area are excluded, as are triangles with areas of
an entire hemisphere or more: neither are typical use cases for the CTP.
Note that the planar control triangle is \textbf{not} the image of the
spherical control triangle under either projection! That image is
slightly larger, containing the planar control triangle, and has curved edges.

For both projections, $\mathbf p_i$ must be arranged such that
$d\left(\mathbf v_i, \mathbf v_j\right) = \|\mathbf p_i - \mathbf p_j\|$
for all $i$ and $j$ in $\{1, 2, 3\}$: the spherical length of the edges of the
spherical control triangle are equal to the Euclidean lengths of the planar
control triangles. Without loss of generality, also assume that
$\|\mathbf p_i\|$ is the same for all $i$, such that the circumcenter of the
control triangle lies at the origin. This assumption just removes a translation
in the plane in order to simplify the formula for the MTP: false northing and
easting can be added later. Given $\mathbf v_i$, $\mathbf p_i$ can be
constructed as follows. Let $i, j, k$ be a cyclic permutation of
$\{1, 2, 3\}$, and let $s_k = d\left(\mathbf v_i, \mathbf v_j\right)$,
such that side $k$ with length $s_k$ is opposite point $k$. The circumradius
of the Euclidean triangle with sides of length $s_k$ is \citep{isaacs}
\begin{equation}\label{eq:circumradius}
  C = \frac{\prod_i s_i}
            {\sqrt{\sum_i 2 s_i^2 s_j^2 - s_i^4}}.
\end{equation}
From the (Euclidean) law of cosines,
the interior angle $\phi_i$ at each vertex is
\begin{equation}\label{eq:phi}
  \phi_i = \arccos \left( \frac{s_k^2 + s_j^2 - s_i^2}{2 s_k s_j}\right).
\end{equation}
Then a set of points satisfying the requirements are given as so:
\begin{equation}\label{eq:planarctrlpts}
  \mathbf P = \begin{bmatrix*}[r]
  C \cos \left(2 \phi_2 \right) &
  C \cos \left(2 \phi_1 \right) &
  C \\
  C \sin \left(2 \phi_2 \right) &
  -C \sin \left(2 \phi_1 \right) &
  0 \\
\end{bmatrix*}.
\end{equation}
These points can be rotated about the origin as desired. Note that some
intermediate values in Equation \ref{eq:circumradius} may be large. This
should not cause a problem on modern computers using common geographic
units of distance, but one may wish to use a unit of measurement such
as radians of arc in these calculations and then scale the result accordingly.

\section{Forward projection}
Figure \ref{fig:chamberlin} shows the geometric construction of the CTP and
MTP. We start with the CTP. Let $\mathbf v$ denote the point to project, and
let $r_i = d(\mathbf v_i, \mathbf v)$ be the true distances from each control
point to $\mathbf v$. Figure \ref{fig:chamberlin}a shows the control triangle
on the sphere with vertices $\mathbf v_i$, the given point $\mathbf v$, and
circles of radius $r_i$ centered on $\mathbf v_i$ and passing through
$\mathbf v$. Moving to the plane, Figure \ref{fig:chamberlin}b shows circles of
the same radius $r_i$ centered on the planar control points $\mathbf p_i$. The
circles may appear to all intersect at a single point, but due to the difference
in curvature between the sphere and the plane they in fact form a small triangle
with curved edges, as shown in Figure \ref{fig:chamberlin}c. In the time of
manual plotters the exact definition of this point was less important, but
\citet{christensen} and modern implementations (e.g. \citealp{proj}) use the
centroid of the small triangle, denoted $\mathbf p_c$.
Note that each pair of circles intersects in (at most) two points,
so the algorithm must choose which point makes up the small triangle.
In \citet{christensen} this is done by comparing the azimuths between each pair
of control points to the azimuths from each control point to the given point.

\begin{figure}
  \includegraphics[width=\textwidth]{construction}
  \caption{Construction of the CTP and MTP. Plot 1a is in equirectangular
  projection. Plot 1b is the result of the CTP or MTP in the plane. Plot 1c is
  the same as plot 1b, zoomed in to the small triangle created by the three
  circles. $\mathbf p_c$ is the point projected by the CTP and $\mathbf p_m$ is
  the point projected by the MTP. Green dotted lines indicate the control
  triangles, the circles are blue solid lines,
  and the perpendiculars are red dashes.}
  \label{fig:chamberlin}
\end{figure}

Construction of the MTP starts like the CTP. If one draws a line through the two
points of intersection of each pair of circles, that line is perpendicular to
the triangle edge. These are the dashed red lines in Figure
\ref{fig:chamberlin}b and c. The three lines, one for each pair of circles,
appear to meet at a single point: this observation can be proven with a simple
triangle theorem sometimes attributed to Carnot. (\citealp{posamentier};
\citealp{wohlgemuth}) That point of intersection is denoted $\mathbf p_m$. For
most points within the control triangle $\mathbf p_m$ lies within the small
triangle, although this is not true in general and is not necessary to create a
valid map projection. (If $\mathbf v$ lies on a control triangle edge,
one pair of circles has a single point of intersection. In that case,
draw a perpendicular to the control triangle edge through that
point of intersection and continue as in the general case.)

The equations of each perpendicular line, taken together, create a linear
system. It is an overdetermined system of 3 equations in 2 variables, but since
all 3 lines meet at the same point, it has a solution. Ultimately this system
can be solved for $\mathbf p_m$ to define a forward map projection as follows.
\begin{equation}\label{eq:forward}
\mathbf p_m =
\mathbf M \begin{bmatrix*} r^2_1 & r^2_2 & r^2_3 \end{bmatrix*}^\top,
\end{equation}
\begin{equation}\label{eq:forwardm}
\mathbf M = \frac{1}{2T}
\begin{bmatrix*} y_3 - y_2 & y_1 - y_3 & y_2 - y_1 \\
x_2 - x_3 & x_3 - x_1 & x_1 - x_2 \end{bmatrix*} = \frac{1}{2T}
\begin{bmatrix*}[r] 0 & -1  \\
1 & 0 \end{bmatrix*}
\mathbf P
\begin{bmatrix*}[r] 0 & -1 & 1 \\
1 & 0 & -1 \\
-1 & 1 & 0 \end{bmatrix*},
\end{equation}
\begin{equation}\label{eq:forwardt}
T = \begin{vmatrix*} x_1 & x_2 & x_3 \\
 y_1 & y_2 & y_3 \\
 1 & 1 & 1
\end{vmatrix*}.
\end{equation}
$T$ is equal to twice the area of the Euclidean control triangle.

The matrix $\mathbf M$ has a (right) nullspace spanned by the vector
$[1, 1, 1]^\top$. This implies the MTP is not one-to-one for all possible values
of $r_i$. For example, for any values of $r_i$ such that $r_1 = r_2 = r_3$, then
$\mathbf p_m = [0, 0]^\top$. Like the CTP, the MTP projects the entire sphere to
a bounded portion of the plane: the boundary of that portion can be termed the
boundary of the projection. There is a region of overlap that is mapped into
the same area but in reverse orientation.
That region includes the antipodes of the control points. In practice,
the overlap region of the sphere is not of interest and can be excluded.

\section{Inverse projection}
Given $\mathbf p_m$, start to invert the projection as so:
\begin{equation}\label{eq:inverse}
\begin{bmatrix*} k_1 & k_2 & k_3
\end{bmatrix*}^\top = \mathbf M^+ \mathbf p_m \,,
\end{equation}
\begin{equation}\label{eq:inversem}
\mathbf M^+ = \frac{2}{3}
\begin{bmatrix*}[r] -2x_1 + x_2 + x_3 & -2y_1 + y_2 + y_3 \\
x_1 - 2x_2 + x_3 & y_1 - 2y_2 + y_3 \\
x_1 + x_2 - 2x_3 & y_1 + y_2 - 2y_3
\end{bmatrix*} = \frac{2}{3}
\begin{bmatrix*}[r] -2 & 1 & 1 \\
1 & -2 & 1 \\
1 & 1 & -2
\end{bmatrix*}
\mathbf P^\top .
\end{equation}
$k_i = r^2_i - h$ for some value of the free parameter $h$. This is a general
solution to inverting Equation \ref{eq:forward}: $\mathbf M^+$ is the
pseudoinverse of $\mathbf M$ and vice versa. Because $\mathbf M^+$ has a left
nullspace spanned by the vector $[1, 1, 1]$, it follows that $\sum_i k_i = 0$,
and then that $h = \frac{1}{3}\sum_i r^2_i$.
These equations are not enough to determine $r_i$ and $\mathbf v$:
information about the sphere needs to be introduced.

To simplify the derivation, use $R=1$, such that distance on the
sphere has units of radians of arc. The circle of points $\mathbf v$ at
distance $r_i$ from a point $\mathbf v_i$ is simply the circle created by a
plane intersecting the sphere. Rearranging the distance function, this plane
may be specified as $\mathbf v_i \cdot \mathbf v = \cos\left(r_i\right).$
These planes combined together give a linear system. Thus,
\begin{equation}\label{eq:inversev}
  \mathbf v = \mathbf V^{-1} \begin{bmatrix*} \cos\left(r_1\right) &
  \cos\left(r_2\right) &
  \cos\left(r_3\right)
  \end{bmatrix*}^\top .
\end{equation}

For the point to lie on the unit sphere, $\|\mathbf v\| = 1$. Let $\mathbf c$
be a vector with $i$th component $\cos\left(r_i \right)$. Then,
$\mathbf c^\top \left(\mathbf V^\top \mathbf V\right )^{-1} \mathbf c = 1$.
Make the substitution
\begin{equation}\label{eq:inverser}
  r_i = \sqrt{k_i + h} \,.
\end{equation} We now have an equation with one unknown, $h$.

Some obvious bounds can be placed on $h$. In units of radians,
$0 \le r_i \le \pi$. Since this must hold for each $r_i$, it follows that
\begin{equation}
   h_{\min} = -\min_i k_i \le h \le \pi^2 - \max_i k_i = h_{\max}\,.
\end{equation}
Within these bounds, there may be at most two solutions for $h$. The solution
with smaller $h$ is the desired one, and the one with larger $h$ corresponds to
the overlap region.

Let $\mathbf A = \left(\mathbf V^\top \mathbf V\right )^{-1}$ and
\begin{equation}\label{eq:inversefh}
f(h) = \mathbf c^\top \mathbf A \mathbf c - 1 \,.
\end{equation}
The derivative of $f(h)$ is
\begin{equation}\label{eq:inversefph}
  f'(h) = -\mathbf c^\top \mathbf A \mathbf b \,,
\end{equation}
where $\mathbf b$ is a vector with $i$th component
$\mathrm{sinc}\left(\sqrt{k_i + h}\right)$.
Note that $f'(h)$ and $f(h)$ share many of the same terms, which can be
exploited for more efficient calculation. Given all the preceding, Newton's
method can be applied to solve for $h$. A suitable initial condition is
$h = h_{\min}$, which satisfies $f(h) = 0$ at the control points. This initial
condition consistently results in convergence to the lower root of $f(h)$.

The error is roughly proportional to the final value of $|f(h)|$. A stopping
condition of $|f(h)| < 10^{-4}$ results in an accuracy of 1 meter or better.
For points in or near the control triangle, this condition is met in only a
few iterations. Convergence is somewhat slower farther away from the
control triangle. At the boundary of the projection,
the lower solution and upper solution are the same and $f(h)=f'(h)=0$
so Newton's method converges at a merely linear rate. \citep{burden}

\section{Comparison}
The open-source C++ package PROJ \citep{proj} contains an implementation of the
CTP. We implemented the MTP as a patch to PROJ, to enable a fair comparison of
computation time, and to take advantage of its existing framework.

We use a set of control triangles, recorded in Table \ref{table:ctrlpts}, that
is the same as \citet[p.~90]{christensen}. These serve as a set of
test cases for comparing the CTP and MTP. The length of each side and the area
of each triangle (with the spherical approximation given earlier) is also given,
and the control triangles are sorted by area.

\begin{table}
\begin{tabular}{ p{2.5cm} | p{1.3cm} p{1.3cm} p{1.4cm} | r r r | r }
Region & \multicolumn{3}{c}{Point} &
  \multicolumn{3}{c}{Side length} &  Area \\
& 1 & 2 & 3 & 1 & 2 & 3 & \\
\hline
Africa Wall & 19°3'W, 24°25'N & 20°E, 35°S & 59°3'E, 24°25'N &
  7,783 & 7,785 & 7,783 & 32.38 \\
North \mbox{America} Wall & 150°W, 55°N & 92°30'W, 10°N & 35°W, 55°N &
  7,064 & 6,434 & 7,064 & 23.71 \\
South \mbox{America} Wall & 80°W, 9°N & 71°W, 53°S & 35°W, 6°S &
  6,161 & 5,259 & 6,947 & 17.70 \\
Europe Wall & 15°E, 72°N & 8°W, 33°N & 38°E, 33°N &
  4,254 & 4,541 & 4,541 & 9.09 \\
E South \mbox{America} & 63°33'W, 8°8'N & 58°33'W, 34°35'S & 35°13'W, 5°47'S &
  4,000 & 3,502 & 4,779 & 7.25 \\
S South \mbox{America} & 43°W, 18°S & 72°W, 18°S & 72°W, 56°S &
  4,225 & 4,874 & 3,064 & 6.77 \\
Australia & 134°E, 8°S & 110°E, 32°S & 158°E, 32°S &
  4,487 & 3,643 & 3,643 & 6.76 \\
NW South \mbox{America} & 69°W, 25°S & 55°W, 10°N & 85°W, 10°N &
  3,284 & 4,261 & 4,177 & 6.70 \\
Canada Wall & 150°W, 60°N & 97°30'W, 50°N & 45°W, 60°N &
  3,423 & 5,197 & 3,423 & 6.11 \\
Canada \mbox{Atlas} & 98°13'W, 61°39'N & 135°W, 40°N & 55°W, 40°N &
  6,560 & 3,761 & 3,449 & 5.28
\end{tabular}
\caption{Table of control triangles and their measurements. Coordinates are from
\citet{christensen}. Side $n$ is opposite point $n$. Lengths are in km,
and areas are in millions of square km.
Measurements use spherical approximation.}
\label{table:ctrlpts}
\end{table}

To measure distortion of area and angle, we use the areal scale factor $s$ and
maximum angular deformation $\omega$ as defined in equations 12--15, 27, and 28
in section 4 of \citet{snyder87}. These are implemented in the
\verb|proj_factors()| function of PROJ, which estimates the derivatives
numerically. For the projections in this text it makes sense to measure distance
distortions with regards to the distance between a given point and the control
points. Let $r_i = d(\mathbf v_i, \mathbf v)$ be the distance from
$\mathbf v_i$ to $\mathbf v$ on the sphere, as earlier, and let
$\ell_i = \|\mathbf p_i - \mathbf p\|$ be the distance from $\mathbf p_i$ to
$\mathbf p$ in the plane.
Our new measure, the total distance deviation or $D$, is the sum of
differences in the distances measured on projected and unprojected distances:
\begin{equation}
 D = \sum_i \left| r_i - \ell_i \right|.
\end{equation}

For each spherical control triangle, summary statistics for $\omega$, $D$, and
$s$ are shown in Figures \ref{fig:omegap}--\ref{fig:scalep}. Note that the
control triangles are sorted by area in these figures as well.
These statistics are measured within the spherical control triangles,
and should not be taken to summarize the entire map: in most cases,
the region of interest extends outside the triangle.
Rather, the statistics allow a quantitative comparison of the two projections.

\begin{figure}
  \includegraphics[width=\textwidth]{omegaplot}
  \caption{Comparison of maximum angular deformation $\omega$.}
  \label{fig:omegap}
\end{figure}

As shown in Figure \ref{fig:omegap}, the MTP consistently has a larger $\omega$
than the CTP: in fact, the maximum $\omega$ for the CTP is near the average
$\omega$ for the MTP. Maximum and average $\omega$ trend upwards with control
triangle area, although asymmetry of the control triangle has some influence
too, as can be seen in the Canada triangles. The maximum $\omega$ for the MTP
is about 3 times that for the CTP, and the average $\omega$ is about 3 to 4
times. However, for moderately-sized triangles,
the maximum $\omega$ values are small for both projections, not exceeding 6°.
Even for the large triangles -- Africa, North America, and South America Wall
Maps -- the maximum distortion is tolerable, and the average does not exceed 6°.

\begin{figure}
  \includegraphics[width=\textwidth]{distanceplot}
  \caption{Comparison of total distance deviation $D$.}
  \label{fig:distancep}
\end{figure}

Total distance deviation $D$ in Figure \ref{fig:distancep} shows an even clearer
trend, monotonically increasing with triangle area. Both projections have the
same maximum $D$ values. The MTP has consistently lower average $D$ values than
the CTP: among these triangles, the maximum $D$ for the CTP is about
1.35 to 1.4 times the maximum $D$ for the MTP. The worst distortion,
Africa Wall Map's maximum $D$, is less than 5\% of its edge lengths, and
smaller control triangles have a maximum $D$ that is an even smaller percent.

\begin{figure}
  \includegraphics[width=\textwidth]{scaleplot}
  \caption{Comparison of areal scale ratio $\sigma$.}
  \label{fig:scalep}
\end{figure}

As true 1:1 scale maps are impractical, absolute values of $s$ are less
important than relative values: the quantity
$\sigma = \frac{\max s}{\min s} - 1$ is plotted in Figure
\ref{fig:scalep} instead. The trend of $\sigma$ is less clear. A combination of
control triangle area and asymmetry influences this value. In general, this
value is higher for MTP than CTP, but not by any consistent factor,
and for the large and symmetric Africa Wall Map triangle the MTP is
actually lower than the CTP. The amount of scale distortion is small for both:
except for the very obtuse Canada Atlas triangle, all values are below 10\%.

Figure \ref{fig:proj} is a comparison of the two projections using the South
America Wall Map control points. This control triangle is fairly representative,
being somewhat asymmetric but not too obtuse. The small part of Central America
in the upper left is somewhat shifted between the two maps, as is the western
area near Ecuador and Peru, but no features on either map are conspicuously
distorted compared to the other. Figure \ref{fig:tissot} shows ellipses of
distortion with centers on a grid at steps of 15° latitude and longitude.
Distortion is somewhat more visible in this figure: one can see that the
MTP introduces slightly more angle distortion near the control points.

\begin{figure}
  \includegraphics[width=\textwidth]{South_America_Wall_Map_zoom}
  \caption{Projection of South America and surroundings. In this and the
  following figures, red dots indicate the control points.}
  \label{fig:proj}
\end{figure}

\begin{figure}
  \includegraphics[width=\textwidth]{South_America_Wall_Map_tissot}
  \caption{Ellipses of distortion, on a 15° grid.}
  %FIXME cheating a little here, use a, b, theta instead?
  \label{fig:tissot}
\end{figure}

\begin{figure}
  \includegraphics[width=\textwidth]{South_America_Wall_Map_omega}
  \caption{Maximum angular deformation $\omega$, in degrees. Blue dashed lines
  indicate where the two projections have equal $\omega$ values. The white
  $+$ symbol indicates where $\omega$ attains the minimum value of 0°.}
  \label{fig:angle}
\end{figure}

Contour lines of $\omega$ are shown in Figure \ref{fig:angle}. Those in the CTP
are ovals, while those for the MTP extend outward through each edge of the
control triangle. Both reach a minimum $\omega$ of 0 at a point inside the
control triangle. Highly obtuse control triangles, like Canada Atlas, may have
two local minima for $\omega$ within the control triangle. $\omega$ is higher
for MTP than CTP, except for a small region in the control triangle where both
are near 0 and some regions near the boundary of the map.

\begin{figure}
  \includegraphics[width=\textwidth]{South_America_Wall_Map_distance}
  \caption{Total distance deviation $D$. Blue dashed lines indicate where the
  two projections have equal $D$ values. The black $+$ symbol indicates where
  $D$ attains a local maximum of 147 meters.}
  \label{fig:distance}
\end{figure}

Figure \ref{fig:distance} depicts contour lines of $D$ for these two
projections. Both projections have a local minimum of 0 at the control points,
which follows from the geometric construction. Both have a local maximum inside
the control triangle. $D$ for the MTP is low along the control triangle edges,
while for the CTP it is larger. $D$ is lower for MTP than CTP in a region
including the control triangle and nearly all of South America, except for a
small region near the shared maximum. It is also lower along the lines extending
out along the edges of the control triangle. $D$ does increase more rapidly for
the MTP than the CTP in the other regions outside the triangle as one moves away
from the control triangle.

\begin{figure}
  \includegraphics[width=\textwidth]{South_America_Wall_Map_scale}
  \caption{Areal scale factor $s$. The white $+$ symbol indicates where
  $s$ attains a local minimum: 0.974 for CTP and 1.022 for MTP.}
  \label{fig:scale}
\end{figure}

Contour lines of $s$ are shown in Figure \ref{fig:scale}.
The contour lines have the same elliptical structure in both projections,
centered on a point where $s$ reaches a local minimum. This point, and the
minimum value there, is different for each projection. That influences the
differences in aggregate area distortion in Figure \ref{fig:scalep}. The
spherical control triangle is projected to a slightly different shape for each
projection, which influences the unscaled values of $s$ as well.

In most practical applications the area pictured in these figures will be
sufficient, but we can state trends as both projections extend outward from the
control triangle, outside the area shown in these figures. $\omega$ increases to
180° at the projection boundary, and increases further in the overlap region,
representing the inversion there. $D$ generally increases away from the center,
but for the MTP $D$ remains low on the control triangle edges.
$s$ is bounded for the entire sphere:
it increases to a maximum and then decreases to 0 at the projection boundary.
In the overlap region, $s$ becomes negative and reaches a minimum value.

\begin{table}
\begin{tabular}{ l r r}
  Projection & Time (s) & Time minus overhead (s)\\
\hline
  CTP & 3.42 & 3.40 \\
  MTP & 1.63 & 1.62 \\
  No-op & 0.02 & 0.00
\end{tabular}
\caption{Computation time comparison.}
\label{table:time}
\end{table}

To determine computation time, the \texttt{timeit} package in Python
\citep{python} was used with the PyProj interface to PROJ. \citep{pyproj} A
data set of 64,800 points across the sphere was projected 100 times. This was
repeated 5 times for each control triangle, and the lowest total time of those 5
was recorded, as recommended by the \texttt{timeit} documentation. This was
done for the two projections and a ``no-op'' projection that returns the input
unchanged, used to estimate overhead. Only the transformation itself was timed,
not the initialization of the projection: the initialization time is negligible
compared to the time cost of projecting a large number of points repeatedly. As
expected, the timing differences between control triangles were not significant
for either projection. Table \ref{table:time} shows the minimum
runtimes across all control triangles. Accounting for overhead,
the MTP takes slightly less than half the time of the CTP.

\section{Conclusion}
We have presented a new compromise map projection, the matrix trimetric
projection (MTP), and demonstrated that it is a suitable alternative to the
Chamberlin trimetric projection (CTP). In general, the differences between the
results of the CTP and MTP are small. The MTP has the advantage if an inverse
is needed, if one wishes to reduce the distance distortion described earlier,
or if processing time is a concern. The CTP is useful for creating maps of
continents with low levels of various types of distortion,
but is uncommon in contemporary cartography:
the advantages of the MTP may open it up to new uses.

The forward formula for the MTP is given in Equations
\ref{eq:forward}--\ref{eq:forwardt}. Equation \ref{eq:forward} is a simple
product of a matrix and a vector. The vector depends on the point being
transformed. The matrix calculated by the latter two equations depends only on
the control points, so can be calculated once and reused. Measuring points
transformed per unit time, the MTP is about twice as fast as the CTP. Of course,
details of the software influence the speed comparison, but PROJ's
implementation of the CTP is itself fairly short, does not have any obvious
coding inefficiencies,
and has changed little in more than two decades, suggesting that
PROJ's implementation of the CTP is as efficient as it could possibly be.

The inverse formula for the MTP is calculated by Equations
\ref{eq:inverse}--\ref{eq:inverser} and a single-variable Newton's method
iteration using Equations \ref{eq:inversefh} and \ref{eq:inversefph}. The
matrices in Equations \ref{eq:inversem} and \ref{eq:inversev}, as well
as matrix $\mathbf A$ in Equations \ref{eq:inversefh} and \ref{eq:inversefph},
also depend only on the control points and can be reused. The CTP,
in contrast, has no known inverse aside from the general methods used to invert
a two-variable function, which are slower and often unstable.

Figures \ref{fig:omegap}--\ref{fig:scalep} plot aggregate measures of distortion
within each of the listed control triangles. The CTP outperforms the MTP in
terms of angular distortion and (except in one case) scale distortion, but the
reverse is true for distance distortion. In most cases, the differences between
the two are small in absolute terms. For both projections, distortion tends to
be larger for a larger control triangle.
Figures \ref{fig:proj}--\ref{fig:tissot} show the projections applied to South
America, and Figures \ref{fig:tissot}--\ref{fig:distance} show distortions
across the mapped area. Differences between the two projections in
Figure \ref{fig:proj} are difficult to notice visually.

The same control points were used for both projections in this work, but the
optimal placement of control points may be somewhat different for the
application of different projections to the same geographic feature. The
angular distortion of the MTP is lower in a region extending through the middle
of the control triangle edges, so one may wish to orient the triangle to take
advantage of that. One may also wish to avoid the area outside the triangle
where $D$ increases more rapidly for the MTP than the CTP.

The form of the MTP's forward formula suggests a possible family of projections
that are simple functions of distances to control points, such as a polynomial
function. Additionally, there may exist similar projections using measurements
other than distance, such as area.

\section{Data availability statement}
A software implementation of the MTP, as well as code used to
produce the calculations and figures in this text,
%is available on the author's Github site. \citep{trimetric}
is available at the private link https://figshare.com/s/d2f92fa8ae678cdd382c.
On acceptance this link will be replaced with an openly accessible site,
subject to Creative Commons license.

Figures in this text were created using the Python packages matplotlib
\citep{matplotlib} and GeoPandas \citep{geopandas}, and their dependencies
including PROJ \citep{proj} and PyProj \citep{pyproj} as referenced earlier.
Land mass shape data come from \citet{natearth}.

%\section{Funding}
%None.

%\section{Disclosure statement}
%The author declares no potential competing interest.

\bibliographystyle{apacite}
\bibliography{references}

\end{document}


%%% BAC exercise template %%%
% template modified by Derek Huang, original by Sean Cox.
\documentclass{article}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath, amssymb, amsfonts, enumitem, fancyhdr, tikz}
% get rid of paragraph indent
\setlength{\parindent}{0 pt}
% allow section.equation numbering
\numberwithin{equation}{section}
% allows you to copy-paste code segments. requires pygments to be installed
% natively, i.e. not just in a Python venv! for example, on WSL Ubuntu you
% need to sudo apt-get install python3-pygments (unfortunately, the version
% will be a bit dated) and cannot rely on your venv Pygments version.
\usepackage{minted}
% makes clickable links to sections
\usepackage{hyperref}
% make the link colors blue, as well as cite colors. urls are magenta
\hypersetup{
    colorlinks, linkcolor = blue, citecolor = blue, urlcolor = magenta
}
% fancy pagestyle so we can use fancyhdr for fancy headers/footers
\pagestyle{fancy}
% add logo in right of header. note that you will have to adjust logo path!
\fancyhead[R]{\includegraphics[scale = 0.15]{./bac_logo1.png}}
% don't show anything in the left and center header
\fancyhead[L, C]{}
% give enough space for logo by reducing top margin height, head separator,
% increasing headerheight. see Figure 1 in the fancyhdr documentation. if
% \topmargin + \headheight + \headsep = 0, original text margins unchanged.
\setlength{\topmargin}{-60 pt}
\setlength{\headheight}{50 pt}
\setlength{\headsep}{10 pt}
% remove decorative line in the fancy header
\renewcommand{\headrulewidth}{0 pt}

% title, author + thanks, date
\title{Exercise -1}
\author{Derek Huang\thanks{NYU Stern 2021, BAC Advanced Team.}}
% the extra thanks is also useful for making notes about previous versions.
% you can remove this thanks if there are no version revisions.
\date{June 9, 2021\thanks{Original version released on February 23, 2021.}}

\begin{document}

\maketitle
% need to include this after making title to undo the automatic
% \thispagestyle{plain} command that is issued.
\thispagestyle{fancy}

% remove some space between title and first section to keep text on one page.
% comment this line out or adjust the space reduction as needed. generally,
% try to remove as little space as possible if you are trying to keep content
% on one page, else comment this line out if content spans multiple pages.
\vspace{-20 pt}

\section{PCA whitening}

Suppose we have centered data matrix $ \mathbf{X} \in
\mathbb{R}^{N \times d} $, where each of the $ N $ rows is a transposed
$ \mathbb{R}^d $ data sample. In principal components analysis we want to find
matrices $ \mathbf{W} \in \mathbb{R}^{d \times d},
\mathbf{Z} \in \mathbb{R}^{N \times d} $, such that
\begin{enumerate}
    \item
    $ \mathbf{W}^\top\mathbf{W} = \mathbf{WW}^\top = \mathbf{I} $, i.e.
    $ \mathbf{W} $ is an orthonormal matrix.

    \item
    The columns of $ \mathbf{W} $ are eigenvectors of the sample covariance
    matrix $ \mathbf{C}_\mathbf{X} \triangleq \frac{1}{N}
    \mathbf{X}^\top\mathbf{X} $ ordered in descending order by the magnitude
    of their corresponding eigenvalue.

    \item
    $ \mathbf{X} = \mathbf{ZW}^\top $, i.e. for each data point
    $ \mathbf{x}_k $, we want $ \mathbf{W}, \mathbf{Z} $ such that
    $ \mathbf{x}_k = \mathbf{Wz}_k $, $ \mathbf{z}_k $ the $ k $th row of
    $ \mathbf{Z} $.
\end{enumerate}

Suppose we find the solution by performing [compact] singular value
decomposition on $ \mathbf{X} $. That is, we find $ \mathbf{U} \in
\mathbb{R}^{N \times d} $, $ \mathbf{\Sigma} \in \mathbb{R}^{d \times d} $,
$ \mathbf{V}^\top \in \mathbb{R}^{d \times d} $ where $ \mathbf{U} $ has
orthonormal columns, $ \mathbf{\Sigma} $ is the diagonal matrix of singular
values\footnotemark\footnotetext{
    Recall that the singular values of $ \mathbf{X} $ are the square roots of
    the eigenvalues of $ N\mathbf{C}_\mathbf{X} = \mathbf{X}^\top\mathbf{X} $.
} ordered in descending order, and $ \mathbf{V} $ is an orthonormal matrix
such that $ \mathbf{X} = \mathbf{U\Sigma V}^\top $. Recall that
$ \mathbf{X} = \mathbf{ZV}^\top $, which implies that
$ \mathbf{Z} = \mathbf{XV} $, where $ \mathbf{C}_\mathbf{Z} $ is diagonal.

\medskip

What $ \tilde{\mathbf{V}} \in \mathbb{R}^{d \times d} $ can we derive from
$ \mathbf{V} $ such that $ \tilde{\mathbf{Z}} \triangleq
\mathbf{X}\tilde{\mathbf{V}}^\top $ has
$ \mathbf{C}_{\tilde{\mathbf{Z}}} = \mathbf{I} $?

\medskip

\textit{Hint.} $ \mathbf{C}_\mathbf{Z} $ is a rescaling of a particular
diagonal matrix we already computed.

\section{Flag checking}

Consider the C function using the \href{
    https://numpy.org/devdocs/reference/c-api/array.html
}{NumPy C array API} shown on the next page.

\begin{minted}{c}
static PyObject *
npy_sigmoid(PyObject *self, PyObject *args)
{
    PyArrayObject *ar;
    if (!PyArg_ParseTuple("O!", &PyArray_Type, (PyObject **) &ar)) {
        return NULL;
    }
    if (PyArray_TYPE(ar) != NPY_DOUBLE) {
        PyErr_SetString(PyExc_TypeError, "ndarray type not NPY_DOUBLE");
        return NULL;
    }
    double *data = (double *) PyArray_DATA(ar);
    npy_intp ar_size = PyArray_SIZE(ar);
    for (npy_intp i = 0; i < ar_size; i++) {
        data[i] = 1 / (1 + exp(-data[i]));
    }
    Py_RETURN_NONE;
}
\end{minted}

Why is this code dangerous? What flags must \texttt{ar} satisfy in particular
for it to be safe?

\end{document}


\documentclass{article}

\title{Introduction to Deep Learning}
\author{Vladimir Feinberg}

\input{../old/defs}

\begin{document}

\maketitle

\section{Introduction}

Many-layer neural networks networks, or deep neural networks, constitute a diverse form of machine learning strategies, which span a broad spectrum of unsupervised, semi-supervised, reinforced, and supervised learning. Inspired by biology, this class of methods is broadly referred to as artificial neural networks (ANNs) or deep learning. Occasionally the term deep neural network comes up, but deep is a relative term useful for distinguishing between two networks. I'll refrain from using such terminology because it creates a false dichotomy in certain situations.

At the highest level, for a given network architecture, an ANN forms a family of possibly nonlinear functions. Such functions may be mappings from a possibly variable-length tensor to another possibly variable-length tensor of real values.

An early, simple example of ANNs is the feed-forward neural network (FFNN), which focuses on fixed-size input and output tensors. This family defines a hypothesis class for empirical risk minimization (ERM) in supervised learning.

These notes follow Dr. Goodfellow's \nurl{http://www.deeplearningbook.org/contents/mlp.html}{Deep Learning Book}. As mentioned there, there are two essential components responsible and necessary for the success ANNs:
\begin{enumerate}
\item Expressiveness through depth: iterative compositions of linear functions followed by activations can efficiently model mostly continuous functions (specifics below).
\item Tractable learning: compositions of simple functions as a graph allow for linear-time evaluation, and, critically, linear-time gradients via back-propagation.
\end{enumerate}

Interestingly, these two frequently-cited facts give rise to two perplexing questions about ANNs, which don't seem to yet have a good answer:
\begin{enumerate}
\item If DNNs are so expressive, then isn't their VC dimension extremely large? Wouldn't this make them extremely prone to overfitting, to the point that they would be unusable? Perhaps DNNs can model ``real distributions'' well, so the distribution-independent analysis of VC-based statistical learning theory is insufficient. How can we justify the previous sentence? (\nurl{https://arxiv.org/abs/1611.03530}{Zhang et al 2017}, \nurl{https://openreview.net/pdf?id=rJv6ZgHYg}{Krueger et al 2017})
\item Why exactly is learning so tractable? Efficient derivatives aren't good enough in a non-convex enviornment: we know we'll reach a local min, but what makes a local min good? (\nurl{https://arxiv.org/abs/1602.04915}{Lee et al 2016}, \nurl{https://arxiv.org/abs/1703.00887}{Jin et al 2017}, \nurl{https://arxiv.org/abs/1412.0233}{Choromanska et al 2014}, \nurl{https://arxiv.org/abs/1705.08292}{Wilson et al 2017})
\end{enumerate}

\section{Definition}

An ANN $f$ of depth $d$ is a a composition of $d$ vector-valued functions:
$$
f(\vx)=f^{(d)}\circ f^{(d-1)}\circ\cdots\circ f^{(2)}\circ f^{(1)}\pa{\vx}
$$

Note that the input and output dimension of each $f^{(i)}$ may differ; the maximum such dimension is the width of the network. Each sub-function $f^{(i)}$, or layer, may differ, but has some known and well-structured parameterization. ANN methods then typically diverge with the training algorithm and what they consider their inputs and outputs to be. These iterative compositions of more fundamental layer-wise operations construct what is accurately labeled by \nurl{http://karpathy.github.io/neuralnets/}{Dr. Karpathy} as ``real valued circuits,'' where simple building blocks construct complex output behaviors through tractable operations.

For instance, a FFNN treats inputs $\vx$ as those inputs from a supervised learning problem, and attempts to find a mapping $f$ between observed pairs $(\vx, y)$. A multi-layer perceptron (MLP), a type of FFNN, is the composition of a vectorized activation function $g$ and an affine function. The dimensionality of the affine functions and the choice of activation are hyperparameters.

$$
f^{(i)}(\vx)=g\pa{W^{(i)}\vx+\vb^{(i)}}
$$
$W^{(i)},\vb^{(i)}$ define parameters for $f$. The activation $g$ is usually a parameter-free (but perhaps not hyperparameter-free) nonlinearity. The MLP differs from other ANNs in that it is fully connected (all $W^{(i)}$ are unconstrained, so there may be arbitrary linear relationships between the output of $f^{(i-1)}$ and the input to $g$). A network whose $W^{(i)}$ matrices correspond to convolutions is a convolutional neural network (CNN) and refers to a much more restricted family of functions.

Other networks, such as recurrent neural networks (RNNs) handle variable-length inputs. The Asimov Institute has compiled \nurl{http://www.asimovinstitute.org/neural-network-zoo/}{an extensive list} of the networks and acronyms used over the years.

A cost function $J(\vtheta)$ must be defined over the vector of aggregate parameters $\vtheta$. MLE informs the shape of $J$ in most cases: for $(\vx,\vy)$ distributed according to some data distribution, the log likelihood is the cross entropy $\E p_\text{model}(\vy|\vx,\vtheta)$. With $p_\text{model}(\vy|\vx,\vtheta)=N(f_\vtheta(\vx),I)$, for instance, MLE is equivalent to minimizing MSE. This kind of approach has the ANN specify the parameters for an output distribution, instead of predicting the output directly: this enables us to encode additional statistical knowledge about the inputs.

Training is typically performed with first-order optimization of $J(\vtheta)$; this leverages back-propagation for efficient derivatives. For a derivation of backpropagation in common architectures, see \nurl{https://github.com/tomepel/Technical_Book_DL}{these technical notes}.

\section{Activations}

Activations like $g$ above are an important design choice for all ANNs. General well-performing ones are listed, though problem-specific considerations will take priority.
\\\\
\noindent
\textbf{ReLU} (\nurl{http://proceedings.mlr.press/v15/glorot11a.html}{Glorot et al 2011}). The ReLU $g(\vx)=\max\pa{\vx,\vzero}$ induces activation sparsity, which is more biologically plausible. Their use improves accuracy when compared to smoother activations in practice, and their linear regime keeps magnitude scaling behavior reasonable as we traverse the net, which is a boon to optimization.
\\\\
\noindent
\textbf{Maxout} (\nurl{https://arxiv.org/abs/1302.4389}{Goodfellow et al 2013}, \nurl{https://arxiv.org/abs/1312.6211}{Goodfellow et al}). A maxout layer is a generalized ReLU $g$ with input dimension $n$ and fixed $p$ and $k$ as hyperparameters has fixed subsets $G^{(i)}=\set{ik+j}{j\in[p]}$ such that $g(\vz)_i=\max_{j\in G^{(i)}}z_j$. In addition to being more theoretically expressive (allowing for fewer units in the next layer), maxout enables units to be driven by several ``filters'' (activations in the previous layer), preventing catastrophic forgetting. Moreover, maxout is complimentary to dropout.

\section{Theory Overview}

\noindent
\textbf{Universal Approximation Theorem}. A feed-forward network with a linear output layer, at least one hidden layer, and the logistic sigmoid activation can approximate any Borel measurable function and its derivatives, where they exist, arbitrarily well, given sufficient hidden units.
\\\\
\noindent
\textbf{Deep Rectifier Network Efficiency Theorem} (\nurl{https://arxiv.org/abs/1402.1869}{Mont\'{u}far et al 2014}). A deep rectifier network of constant width $w$, input dimension $i$ and depth $d$ may have up to $\Omega\pa{\pa{\nicefrac{w}{i}}^{di}i^i}$ distinct linear regions and a maxout network with $k$ filters per unit may have up to $\Omega(k^{i+l-1})$. Visually demonstrated in Figure~\ref{fig:montufar}.

\begin{figure}[!h]
\centering
{\includegraphics[width=\textwidth]{montufar2014.pdf}}
  \caption{Visual demonstration of how composing an absolute value rectifier, which in effect allows for a higher-level function to be mirrored over some hyperplane in the input space, can result in exponentially many different linear regions by relying on (learned) symmetry.}
\label{fig:montufar}
\end{figure}

\end{document}


\documentclass{notes}

\title{Adaptive time steps in DREAM}
\author{Mathias Hoppe}

\newcommand{\Deltats}{\Delta t^\star}
\newcommand{\epst}{\epsilon_{\Delta t}}
\newcommand{\epstol}{\epsilon_{\rm tol}}
\newcommand{\Eps}{\mathcal{E}}

\begin{document}
    \maketitle

    \noindent
    The solution to the \DREAM\ equations can evolve on a wide range of time
    scales, even during the same simulation. For example, the thermal quench
    and current quench are usually separated by one or more orders of magnitude,
    as are the current quench and runaway plateau. To ensure that the simulation
    is always well-resolved, without requiring the user to manually split the
    simulation into separate and carefully adjust parameters, \DREAM\ provides
    functionality for manually adjusting the time step during the simulation.
    This document describes the theoretical basis for this adaptive timestepper,
    as well as the details of how it is implemented in \DREAM.

    \section{Theory}
    The purpose of the adaptive time stepper is to balance two objectives: first
    and foremost, we want the solution to be correct and accurate, and so the
    adaptive time stepper must ensure that the truncation error
    $\epst$ is always sufficiently small, i.e. below some set
    tolerance $\epstol$. We could always ensure this by keeping the time step
    $\Delta t$ fixed and have $\Delta t\to 0$, but this would also cause the
    wall time of the simulation to approach infinity. Rather, we would prefer
    the simulation to finish as quickly as possible. Therefore, the second
    objective that the adaptive time stepper must achieve is to make $\Delta t$
    as large as possible, since the wall time of the simulation can usually be
    assumed to be proportional to $\Delta t$.

    The way to balance these two objectives is to, first, always ensure that
    $\epst\leq\epstol$. When $\epst<\epstol$, this indicates that the time step
    $\Delta t$ is a bit too small and we could increase it without violating the
    accuracy condition.

    In order to devise an adaptive time stepper we need to determine two
    quantities, namely the truncation error $\epst$ in any given time step,
    as well as the optimal time step $\Deltats(\epst)$ to take. The bible,
    commonly also known as \emph{Numerical recipes}, provides the basic building
    blocks for both of these.

    We will determine the local error $\epst$ by taking a total of three steps.
    First, we take a full step of length $\Delta t$ in order to obtain the
    solution $y_1$. Next, we revert the solver and take two separate steps, each
    of size $\Delta t/2$, and obtain the solution $y_2$. The difference between
    the two solutions can be taken as the truncation error:
    \begin{equation}
        \epst\equiv \left| y_2 - y_1 \right|.
    \end{equation}
    For checking convergence, we evaluate this truncation error for each unknown
    quantity separately. In order to adjust the time step, however, we select
    the largest error:
    \begin{equation}
        \Eps = \max_i\frac{\epst^{(i)}}{\epstol^{(i)}},
    \end{equation}
    where $i$ identifies the unknown quantity. If $\Eps > 1$, the error is too
    large, and vice versa. If the step taken to achieve a too large error
    $\Eps_1$ was $\Delta t_1$, and we strive for an error $\Eps^\star$, then the time step
    that would have given the correct error (in Euler backward) is
    \begin{equation}
        \Deltats = \Delta t_1\left|\frac{\Eps^\star}{\Eps_1}\right|.
    \end{equation}
    In \DREAM, we take $\Eps^\star = 1$ (note that this does not mean that enforce
    \emph{no error}, but rather that we require the error to be within the
    previously enforced tolerances). Since the error estimate is not exact, we
    also introduce a safety factor $S\approx0.95$ and obtain the updated time
    step
    \begin{equation}\label{eq:timestep}
        \Deltats_{n+1} = S\frac{\Delta t_n}{\Eps_n}.
    \end{equation}

    \paragraph{Improved stability}
    The stability of the stepper can be somewhat improved by noting
    that~\eqref{eq:timestep} can be considered as an
    \emph{integrating controller} in standard control theory. The $1/\Eps$
    indicates that the control variable is obtained by ``integrating'' the
    error. It is well known in control theory that in order to improve the
    stability of this simple PI controller, one can incorporate the error in
    the previous step as well:
    \begin{equation}
        \Deltats_{n+1} = S\Delta t_n \frac{\Eps_{n-1}^\beta}{\Eps_n^{\alpha}}.
    \end{equation}
    The exponents $\alpha$ are typically chosen as
    \begin{equation}
        \beta\approx 0.4/k,\quad \alpha\approx 1/k-0.75\beta = 0.7/k,
    \end{equation}
    where $k$ denotes the order of the method ($k=1$ for backward Euler). Hence,
    in \DREAM, we use $\alpha = 0.4$ and $\alpha = 0.7$.

\end{document}


\subsection{Independence of irrelevant alternatives}

This is not always observed. For example if a firm offers two products, a cheap \(a\) option and an expensive \(b\) option, then a preference may be:

\(a\prec b\)

The firm may add another product \(c\) slightly less expensive than \(b\), with fewer features, such if there were only \(2\) of the goods on sale:

\(a\prec b\)

\(a\prec c\)

\(b\prec c\)

In particular, the last two imply the first, through the axiom of transitivity.

However with all three, the marketing effectively makes the consumer choose \(b\), by making it look like a better deal, so we observe:

\(a\prec s\)

\(e\prec c\)

\(c\prec s\)

Even though the elements have not changed.

Axioms of revealed preference

Add weak axiom of revealed preference

Strong axiom. can derive both (?)


\documentclass{article}

\usepackage[final]{style}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc}    % use 8-bit T1 fonts
\usepackage{hyperref}       % hyperlinks
\usepackage{url}            % simple URL typesetting
\usepackage{booktabs}       % professional-quality tables
\usepackage{amsfonts}       % blackboard math symbols
\usepackage{nicefrac}       % compact symbols for 1/2, etc.
\usepackage{microtype}      % microtypography
\usepackage{verbatim}
\usepackage{graphicx}       % for figures
\usepackage{float}
\usepackage{amsmath}

\title{Lecture \#5: Edge detection}

\author{
  Stephen Konz, Shivaal Roy, Charlotte Munger, Christina Ramsey, Alex Iyabor Jr \\
  Department of Computer Science\\
  Stanford University\\
  Stanford, CA 94305 \\
  \texttt{\{swkonz, shivaal, cmunger, cmramsey, aiyabor\}@cs.stanford.edu} \\
}

\begin{document}

\maketitle

\section{Continuing from Last Lecture}
\subsection{Linear Systems}
Linear Systems (Filters) form new images, the pixels of which are a weighted sum of select groupings of the original pixels. The use of different patterns and weights amplifies different features within the original image. A system  $S$ is a linear system If and Only If it satisfies the Superposition Property of systems:
\begin{center}
$S[\alpha f_i[n,m] + \beta f_j[h,m]]$ $=$ $\alpha S[f_i[n,m]] + \beta S[f_j[h,m]]$
\end{center}

As previously introduced in the last lecture, the process of applying a filter to an input image is referred to as \textit{convolution}.

\subsection{LSI (Linear Shift Invariant Systems) and the impulse response}
A shift invariant system is one in which shifting the input also shifts the output an equal amount. A linear shift-invariant (LSI) system's is characterized by its response to an impulse; this response is known as the \textit{impulse response}. The impulse response helps us to understand the output of an LSI system for any given input signal.

\subsection{Why are Convolutions Flipped?}
Proof of $2D$ cross correlation's commutativity:
\begin{align*}
f[n,m] * * h[n,m] &= \sum_{k}^{} \sum_{l}^{} f[k,l] \cdot h[n-k,m-l]\\
\text{let $N = n - k$, $M = m-l$ so $k = n - N$ and $l = m - M$} \\
&= \sum_{k}^{} \sum_{l}^{} f[n-N, m-M] \cdot h[N,M]\\
&= \sum_{N}^{} \sum_{M}^{} h[N,M] \cdot f[n-N, m-M]\\
% &= \sum_{k}^{} \sum_{l}^{} f[n-k,m-l] \cdot  h[k,l] &\\
&= h[n,m] * * f[n,m]
\end{align*}
\subsubsection{Example}
Apply kernel k to matrix M.
$$ M = \begin{bmatrix}
0 & 0 & 0\\
0 & 1 & 0\\
0 & 0 & 0
\end{bmatrix}, \\
k = \begin{bmatrix}
x_{1,1} & x_{1,2} & x_{1,3} \\
x_{2,1} & x_{2,2} & x_{2,3}\\
x_{3,1} & x_{3,2} & x_{3,3}\\
\end{bmatrix}$$
$$ M * k = \begin{bmatrix}
x_{3,3} & x_{3,2} & x_{3,1} &\\
x_{2,3} & x_{2,2} & x_{2,1} &\\
x_{1,3} & x_{1,2} & x_{1,1} &\\
\end{bmatrix}$$
Here, the kernel is expected to match the convolution. Instead, the output is equal to the kernel flipped in the x and y direction. To rectify this, the kernel is flipped at the initial step to ensure correct output form.
\subsection{Convolution vs Cross Correlation}
A convolution is an integral that expresses the amount of overlap of one function as it is shifted over another function. We can think of the convolution as a filtering operation. \newline

The \textit{Correlation} calculates a similarity measure for two input signals (e.g., two image patches). The output of correlation reaches a maximum when the two signals match best. Correlation can be though of as a measure of relatedness of two signals.

\section{Edge Detection in Mammals}

\subsection{Hubel \& Wiesel}
In a series of experiments conducted by Hubel and Wiesel \cite{hubel1960receptive}, the neuron responses in a cat's brain were recorded to observe how each part of the brain responds to different stimuli. They revealed that the cat’s neurons were most excited by the edges at different orientations; namely, certain neurons correlated to specific orientations of edges or edges that moved in specific directions. Having one of these edges moving in its field of vision would cause that particular neuron to fire excitedly.

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{hubel_wiesel_cat.jpg}
\caption{The Hubel and Wiesel's experiment. Source: \cite{hubel1960receptive}; Lecture 5, slide 34}
\end{figure}

\subsection{Biederman}
Biederman investigated the rate at which humans can recognize the object they're looking at. To test this, he drew outlines of common and recognizable objects and split them into two halves, with each line segment divided in only one of the halves. These outlines were then shown to partipants to test whether they could recognize the original objects while only seeing half of the original outline. \newline

Surprisingly, he observed no difference in terms of the speed with which people recognized the objects. It was easy for them to recognize an object via only parts of its edges. This study benefits computer vision by providing an insight: even if only part of the original image is shown, a system theoretically should still be able to recognize the whole object or scene.

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{biederman_outlines.png}
\caption{The examples of the Biederman's outlines. Source: Lecture 5, slide 36}
\end{figure}

\subsection{Walther, Chai, Caddigan, Beck \& Fei-Fei}
In similar experiments, another group of researchers took two variations of the same image - the original color image and the outline of that image - to test color image vs line image recognition in humans. They traced recognition through different layers, each one a different level of processing in the visual pcortex of the brain. They found that lower layers could recognize the scenes faster via the line image, but as it moved up the layers of the brain (which encode increasingly higher level concepts), the color drawings were much more helpful in allowing people to recognize the scene as compared to the line drawing. This is believed to have happened because lower layers are better at recognizing pieces, like edges, while higher layers are better at recognizing concepts (like "human," "chair," or "tiger").

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{fei-fei_images.png}
\caption{The visualization of the images and outcomes. Source: \cite{walther2011simple};Lecture 5, Slide 37}
\end{figure}

\section{Edge Detection for Computer Vision}
The goal of edge detection is to identify sudden changes (\textit{discontinuities}) in an image. Intuitively, most semantic and shape information from the image can be encoded in its edges. \newline

The edges help us extract information, recognize objects, and recover geometry and viewpoint. They arise due to discontinuities in surface normal, depth, surface color, and illumination . \newline

\subsection{Types of Discrete Derivative in 1D}
There are three main types of derivatives that can be applied on pixels. Their formulas and corresponding filters are:\\
Backward
$$\frac{df}{dx} = f(x) - f(x-1) = f'(x)$$
$$[0, 1, -1]$$
Forward:
$$\frac{df}{dx} = f(x) - f(x+1) = f'(x)$$
$$[-1, 1, 0]$$
Central:
$$\frac{df}{dx} = f(x+1) - f(x-1) = f'(x)$$
$$[1, 0, -1]$$
\subsection{Discrete Derivative in 2D}
Gradient vector
$$ \nabla f(x,y) = \begin{bmatrix} f_x\\
f_y\\
\end{bmatrix}  $$
Gradient magnitude
$$ |\nabla f(x,y)| = \sqrt{f_x^2 + f_y^2}$$
Gradient direction
$$ \theta = tan^{-1}\Big( \frac{\frac{df}{dy}}{\frac{df}{dx}}\Big)$$

\subsection{Example}
The gradient at a matrix index can be approximated using neighboring pixels based on the central discrete derivative equation expanded to $2D$. A filter like
$$ \frac{1}{3}\begin{bmatrix} -1 & 0 & 1\\
-1 & 0 & 1 \\
-1 & 0 & 1 \\\end{bmatrix}$$
When overlapped on top of a pixel $x[m,n]$, such that the center of the filter is located at $x[m,n]$ shown with its neighbors below
$$ \begin{bmatrix}
...&...&...&...&...\\
...&x_{m-1,n-1} & x_{m-1,n} & x_{m-1,n+1}&... \\
...&x_{m,n-1} & x_{m,n} & x_{m,n+1}&...\\
...&x_{m+1,n-1} & x_{m+1,n} & x_{m+1,n+1}&...\\
...&...&...&...&...\\
\end{bmatrix}
$$
Produces an output of
$$\frac{1}{3}\Big((x_{m-1,n+1} - x_{m-1,n-1})+ (x_{m,n+1}-x_{m,n-1}) + (x_{m+1,n+1} - x_{m+1,n-1})\Big) = $$
Which is equivalent to an approximation of the gradient in the horizontal ($n$) direction at pixel $(m,n)$. This filter detects the horizontal edges, and a separate kernel is required to detect vertical ones.

\section{Simple Edge Detectors}

\subsection{Characterizing Edges}
Characterizing edges (i.e., characterizing them properly so they can be recognized) is an important first step in detecting edges. For our purposes, we will define an edge as a rapid place of change in the image intensity function. If we plot the intensity function along the horizontal scanline, we see that the edges correspond to the extra of the derivative. Therefore, noticing sharp changes along this plot will likely give us edges.

\begin{figure}[H]
\centering
\includegraphics[height=4cm]{lazebnik.png}
\caption{An image with intensity function and first derivative. Source: Lecture 5, slide 66}
\end{figure}

\subsection{Image Gradient}
The gradient of an image has been defined as the following:
$$\nabla f = \left[ \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}\right],$$
while its direction has been defined as:
$$\theta = \tan^{-1} \left( \frac{\partial f}{\partial y} / \frac{\partial f}{\partial x} \right).$$

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{seitz_gradient_direction.png}
\caption{The gradient vector directions. Source: Lecture 5, slide 67}
\end{figure}

The gradient vectors point toward the direction of the most rapid increase in intensity. In vertical edge, for example, the most rapid change of intensity occurs in the $x$-direction.

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{lazebnik_tiger.png}
\caption{The gradients as applied to the image of a tiger. Source: Lecture 5, slide 68}
\end{figure}

\subsection{Effects of Noise}

If there is excessive noise in an image, the partial derivatives will not be effective for identifying the edges. \newline

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{seitz_noise.png}
\caption{The derivative of an edge in a noisy image. Source: Steve Seitz; Lecture 5, slide 70}
\end{figure}

In order to account for the noise, the images must first be smoothed. This is a process in which pixel values are recalculated so that they more closely resemble their neighbors. The smoothing is achieved by means of convolving the image with a filter (e.g., gaussian kernel). \newline

There are, of course, some concerns to keep in mind when smoothing an image. The image smoothing does remove noise, but it also blurs the edges; the use of large filters can result in the loss of edges and the finer details of the image.

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{seitz_smoothing.png}
\caption{Smoothing with different filters and filter sizes. Source: Steve Seitz; Lecture 5, slide 75}
\end{figure}

The image smoothing facilitates the process of edge detection. After smoothing an image $f$, the search for peaks is initiated by calculating $f * \frac{d}{dx}g$ with kernel $g$.

\subsection{Gaussian Blur}

The Gaussian blur is the result of blurring an image by a Gaussian function to reduce image noise. It is a low-pass filter, meaning it attenuates high frequency signals. You will generally use Gaussian blurring as a preliminary step.
\newline
One dimension:
$$G(x) = \frac{1}{\sqrt{2\pi\sigma}}e^{-\frac{x^2}{2\sigma^2}}$$
Two dimension:
$$G(x,y) = \frac{1}{2\pi\sigma}e^{-\frac{x^2+y^2}{2\sigma^2}}$$

\section{Designing a Good Edge Detector}
An optimal edge detector must have certain qualities:
\begin{enumerate}
    \item Good detection
    \begin{itemize}
        \item It must minimize the probability of detecting false positives (which are spurious edges, generally caused by noise) and false negatives (missing real edges, which can be caused by smoothing, among other things). If it detects something as an edge, it should be an edge.
    \end{itemize}
    \item Good localization
    \begin{itemize}
        \item The detected edges must be as close as possible to the actual edges in the original image. The detector must identify where the edges occur and pinpoint the exact location of the edge; it must also be consistent in determining which pixels are involved in each edge.
    \end{itemize}
    \item Silent response
    \begin{itemize}
        \item It must minimize the number of local maxima around the true edge (returning one point only for each true edge point). It should tell you that there is one very specific edge instead of splitting one edge into multiple detected edges. In other words, only the real border of the edge is captured; other possibilities are suppressed.
    \end{itemize}
\end{enumerate}

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{fei-fei_detection.png}
\caption{Sample problems of bad edge detectors. Source: Lecture 5, slide 84}
\end{figure}

\small
\bibliographystyle{plain}
\bibliography{bibliography}
\end{document}


\chapter{\uppercase{Conclusions, Future Work, and Preliminary Results} \label{chapter:future}}

\section{Concluding Remarks}
In the Chapter~\ref{chapter:mud} we demonstrate that the MUD solution retains the accuracy of least-squares solutions while simultaneously offering the flexibility of specifying initial beliefs.
Normally in order to incorporate such beliefs, practitioners in the machine-learning field would perform Tikhonov regularization, usually with the inclusion of a hyper-parameter which scales the additional parameter-space norm in the objective function.
Mathematically, this scaling factor applied to the norm is equivalent to scaling the matrix representation of an initial (prior) covariance and searching for the MAP point of a Bayesian posterior.
Increasing this scaling factor is interpreted as having less confidence in these initial assumptions.
Conversely, decreasing it is equivalent to putting more emphasis on the prior beliefs than the evidence provided by the data, which causes MAP solutions to drift away from the solution contour (equivalence class) to which $\paramref$ belongs.

The MUD point is not impacted by scaling of the initial covariance, providing \emph{consistent} solutions which demonstrate levels of accuracy that MAP points only exhibit for larger values of scaling factors.
Not only is it robust to the specification of prior assumptions, but it manages to offer the flexibility of such specifications without paying the additional cost of hyper-parameter optimization that would be required for the Tikhonov solution to achieve comparable results; any choice of $\alpha$ would have sufficed.

By contrast, the Tikhonov-regularized solution selects a point that is biased in directions defined by the initial density (covariance).
The data-consistent solution is an update to the initial mean in this same direction but will always exist on the contour $Q^{-1}(\observedMean)$, where $\observedMean$ is the mean of the observed density.

We also show that regardless of how well-informed the initial beliefs are, the convergence rate of the MUD solutions as more data are incorporated\---either by dimension or rank)\---will match those of the Least-Squares solutions.
Moreover, unlike MAP solutions, the MUD point is not sensitive to scaling of the initial covariance (how strongly initial beliefs are held).
This insensitivity provides a strong motivating factor for the consideration of the data-consistent approach within the standard set of solution methods available to scientists and modelers who seek to perform parameter-identification.
We leave the investigation of more connections to the removal of hyper-parameter estimation to future work.

The trouble is, none of these regularization approaches actually guarantee that in under-determined problems, the unique solutions that are selected are close to $\paramref$.
The equivalence-class nature of the solution contour means that by definition there are directions in which uncertainty is unresolved.
Thus, the goal is to aggregate data into components of a vector-valued QoI map to improve estimates of $\paramref$ across more dimensions.
We use skewness as a guide for constructing the QoI and demonstrate its utility in improving estimates to a $\paramref$ related to estimating an uncertain function.

\section{Future Work and Preliminary Results}

\input{extensions/mud_oed.tex}
\FloatBarrier

%\subsection{Leveraging Data in Different Ways}\label{sec:ch05-data}
%Mention the other map we can use (SSE) here.
%
%\subsection{Addressing Model Assumptions}\label{sec:ch05-variance}
%Both the maps required us to have knowledge of the variance in the measurements.
%What if we got it wrong?
%\emph{What if we don't know the variance? How does mis-estimating it affect our solutions?}
%In this section we pose some questions and provide a brief hint at a research direction but really we do not have adequate time to flesh out the answers to these, just acknowledge that they're similar concerns shared by the Bayesians.
%
%Multiplicative noise - handled in a straightforward way, maybe put an example here and leave it at that? Put it in appendix?

\input{extensions/sequential_inversion.tex}
\FloatBarrier


\chapter{Related Works}
\label{Related Works}

\section{Standard Lane Following Algorithm}
As autonomous cars evolve into reality, there is a need for algorithms
to assist them in navigating and staying inside the lines. These
algorithms detect lines on the pavement, and
instructing the car to stay in the center. This is similar to
a two line crosswalk, the goal is to keep the user between the two
lines. However, with a crosswalk, the goal is to keep them inside the crosswalk, not necessarily centered.
One algorithm for lane following is as follows:
\begin{enumerate}
  \item Convert the image to grayscale
  \item Crop to the center portion of the image
  \item Identify the edges using an edge detection algorithm, and then draw the
  edges onto a new image
  \item Apply Hough line detection to find shapes in the edges \cite{HoughTransform}
  \item Delete extraneous lines that were detected
  \item Apply and draw the discovered lines on the main image
\end{enumerate}
By following these steps, the image is processed and the lanes are returned,
which can then be used to direct the car to remain inside the lane \cite{SingleLane1}.

%https://www.researchgate.net/publication/4207389_Bipolarity_and_Projective_Invariant-Based_Zebra-Crossing_Detection_for_the_Visually_Impaired

\section{Crosswalk Detection Approach Using Pixel Bipolarity}

Kyoto Institute of Technology's Uddin and Shioyama created a method of detecting zebra crosswalks using their geometric properties and the bipolarity of the images \cite{relatedworkbipolarity}. Their paper focuses on grouping large areas together that have high values of dark and/or light inside of them. Also, they use the projective invariant aka cross product to determine if the areas of light and dark in their image fit the pattern of a crosswalk. They then use the Fischer criterion to extract feature points in the image where the change from white to black (and vice versa) occur.

The steps are as follows:
\begin{enumerate}
   \item Convert to grayscale and split the image into 16x16 pixel groups.
   \item Find regions that are strongly bipolar by merging pixel groups that have a distance ratio within a threshold. The distance ratio is a measure of how similar the regions intensities are to each other in regards to bipolarity. A merged image is shown in figure \ref{fig:BipolarCrosswalk} section  (b).
   \item Once grouped, discard regions that don't exhibit enough bipolarity.
   \item Refine the segmentation by allowing larger groups to absorb smaller groups inside of them.
   \item Check where the region is in the image. If it is far to the left or right, classify it as such. If the bottom of the region is too high up in the image, discard it.
   \item Use a Fourier transform to check if the crosswalk angle is too extreme. If it is, throw it out.
   \item Use Fischer criteria to find the feature points where the intensity changes drastically, determine that there are at least seven (which means there are four or more crosswalk stripes in the image), and then check the projective invariant criteria to see if it passes or is thrown out\cite{relatedworkbipolarity}.
   \item Repeat for each region until a match is found or all regions are exhausted.
\end{enumerate}
This method had decent results, with a high true positive rate (74 / 79), and zero false positives out of 34 other images. One downside of their method is that the image is required to have a minimum of four crosswalk stripes in the image to detect it, which means the majority of the crosswalk must be in the frame. Theoretically this method will work with a moderate amount of the crosswalk covered up (by other pedestrians or cars) as long as there are enough feature points in a vertical line in at least one area to detect the crosswalk.

\begin{figure}[t]
\begin{center}
\includegraphics[width=10cm]{figures/BipolarCrosswalk.png}
\captionfonts
\caption{Steps of detecting a zebra crosswalk using Bipolarity and Projective Invariant: (a) gray scale image, (b) bipolarity of each segmented region, (c) original image highlighting the largest detected region, (d) extracted feature points shown on region}
\label{fig:BipolarCrosswalk}
\end{center}
\end{figure}


\documentclass[simplex.tex]{subfiles}
% NO NEED TO INPUT PREAMBLES HERE
% packages are inherited; you can compile this on its own
\begin{document}
\subsection{Robust Law of Large Graphs}

We had theorems for the ML$q$E under the exponential distribution. Actually the results can be generalized to a broader class of distribution families, and even a different entry-wise robust estimator other than ML$q$E with the following conditions:
\begin{compactenum}
\item Let $A_{ij} \stackrel{ind}{\sim} (1-\epsilon) f_{P_{ij}} + \epsilon f_{C_{ij}}$, then $E[(A_{ij} - E[\hat{P}_{ij}^{(1)}])^k] \le \mathrm{const} \cdot k!$, where $\hat{P}^{(1)}$ is the entry-wise MLE as defined before;
This is to ensure the that observations will not deviate from the expectation too far away, such that the concentration inequality can apply.
\item There exists $C_0(P_{ij}, \epsilon) > 0$ such that under the contaminated model with $C > C_0(P_{ij}, \epsilon)$,
\[
	\lim_{m \to \infty} \left| E[\hat{P}_{ij}] - P_{ij} \right| <
    \lim_{m \to \infty} \left| E[\hat{P}^{(1)}_{ij}] - P_{ij} \right|;
\]\\
It requires the contamination of the model to be large enough (a restriction on the distribution) and $\hat{P}$ to be robust enough with respect to the contamination (a condition on the estimator).
\item $\hat{P}_{ij} \le \mathrm{const} \cdot \hat{P}_{ij}^{(1)}$; (This might be generalized to with high probability later)\\
Since we use the results of $\hat{P}^{(1)}$ to bound $\hat{P}^{(q)}$, the proof can apply directly with this condition for an arbitrary $\hat{P}$.
\item $\mathrm{Var}(\hat{P}_{ij}) = O(m^{-1})$, where $m$ is the number of observations.\\
We will get exactly the same results under this condition. However, even if the variance of the new estimator is not of order $O(m^{-1})$, we will get similar results with a different term related to $m$.
\end{compactenum}

\clearpage
\end{document}


% mainfile: ../../Refinement.tex
The Object-Z, shortly \oz{}, is a specification language used to describe an entity through specifying its data, operations and the effects of those operations on the data. This section introduces the Object-Z as described in \cite{olderog}.
\subsection{Intuition}
\label{sec_oz_intuition}
\input{chapters/mainpart/preliminaries/sections/oz/subsections/oz_intuition/oz_intuition}

\subsection{Semantics}
\label{sec_oz_sem}
\input{chapters/mainpart/preliminaries/sections/oz/subsections/oz_semantics/oz_semantics}

\subsection{Dynamic OZ}
\label{sec_oz_dynamic_oz}
\input{chapters/mainpart/preliminaries/sections/oz/subsections/oz_dynamic_oz/oz_dynamic_oz}