\section{Hierarchical Information Content}
\label{sec:hic}

We construct a measure for the structural complexity of a system. As motivated
in section~\ref{sec:finding_complexity}, we want the complexity to grow at
the compositions of ordered and disordered subsystems in a hierarchical
structure. We construct the overall complexity from the sum of the complexity at
each composition.

Assume we are given a hierarchical system with $L$ levels. We construct the
global system complexity from the local complexity computed at each of the
$L-1$ compositions between consecutive levels. The local complexity measures
are given by the difference in mutual information of the system at level
$\ell$ and that of the subsystem at level $\ell - 1$. We denote by $X^{\ell}$ the
state variable at the $\ell$-th level of the hierarchy. Following
\citet{simon1991architecture}, we define the span of a level as the
number of elements from the $\ell - 1$ subsystem used to construct the state
$X^\ell$. For now we assume the span to be constant with a value of two for
every level. We later generalize to larger spans.

As an example, the state variable $X^\ell$ could be a molecule in an artificial
chemistry constructed from two atoms. Alternatively, $X^{\ell}$ could be an $n
\times n$ state of a cellular automata constructed from two $(\nicefrac{n}{2})
\times n$ sub-states. As an example from nature, $X^{\ell}$ could be a string
of $2^\ell$ base pairs of DNA constructed from two substrings of length $2^{\ell
- 1}$.

The complexity at the $\ell$-th composition is computed from the squared
difference of the mutual information of the span between level $\ell$ and level
$\ell + 1$:
\begin{equation}
    C_\ell = \left[ I(X^{\ell + 1}; Y^{\ell + 1}) - I(X^{\ell}; Y^{\ell}) \right]^2
\end{equation}
The mutual information can be computed from the Shannon entropy and conditional
entropy when $X^\ell$ is discrete or the differential entropy when $X^\ell$ is
continuous~\citep{cover1999elements}:
\begin{equation}
\label{eq:mutual_information}
I(X^\ell; Y^\ell) = H(X^\ell) - H(X^\ell \mid Y^\ell).
\end{equation}
The mutual information is large when the $\ell$-th level is highly ordered and
small when it is highly disordered. Hence, the values $C_\ell$ will be large
exactly at the transition points between order and disorder (or vice-versa).  We
use the level complexities $C_\ell$ to construct the overall system complexity.

\begin{definition}[Hierarchical Information Content]
\label{def:hic}
  The Hierarchical Information Content (HIC) of a hierarchical system $S$ with
  $L$ levels is given by:
  \begin{equation}
    \label{eq:hic}
    \hic(S) = \sum_{l=1}^{L-1} C_\ell = \sum_{l=1}^{L-1} \left[ I(X^{\ell+1}; Y^{\ell+1}) - I(X^\ell; Y^\ell) \right]^2.
  \end{equation}
\end{definition}

\paragraph{Generalizing to larger spans:} Using a larger span requires
generalizing the mutual information to multiple variables. The interaction
information is one such generalization~\citep{mcgill1954multivariate}. The
interaction information of a set of variables is defined recursively as:
\begin{equation}
    \label{eq:interaction_information}
    I(X_1^\ell; \ldots; X_{n+1}^\ell) = I(X_1^\ell; \ldots; X_n^\ell) - I(X_1^\ell; \ldots; X_n^\ell \mid X_{n+1}^\ell),
\end{equation}
where the conditional mutual information is given by:
\begin{equation}
    \label{eq:conditional_mutual_information}
    I(X; Y \mid Z) = \sum_{Z} P(Z) I(X; Y).
\end{equation}

\paragraph{Estimating information criteria:} Computing the mutual information
in equation~\ref{eq:mutual_information} requires computing the entropy of the
state variables $H(X^\ell)$ and the conditional entropy $H(X^\ell \mid
Y^\ell)$. Estimating these quantities directly requires models for the
distributions $P(X^\ell)$ and $P(X^\ell \mid Y^\ell)$. For state variables with
a small domain these distributions can likely be estimated efficiently from
counts. However, for larger state variables sample efficiency becomes an issue.
In these cases more sophisticated density estimation techniques should be used.
Suitable techniques can be found in most modern statistical learning or machine
learning texts (see for example \citet{friedman2001elements}).

\paragraph{Specifying the hierarchy:} In some cases, the hierarchical structure
of the system may be self-evident. This might be the case if the system has a
natural tree-like structure. However, in the general case, both the boundaries
between levels and the sub-modules which make up the state at the next level up
must be specified. This is introduces a degree subjectivity to the HIC which
effects its ability to faithfully capture the complexity of the system.

As an example consider measuring the HIC of a sequence of DNA. We might define
the state $X^\ell$ to be a subsequence of $2^\ell$ nucleotides. On the other
hand there may be some natural structure in the DNA to take advantage of. For
example, codons are sequences of three nucleotides which each code for a
specific amino acid.  These would make a good definition for a state variable.

\subsection{Examples}

We attempt to build intuition for the definition of HIC through some simple
examples.

\begin{example}
  \label{ex:constant}
  As a first example, consider the state $S = [0, 0, \ldots, 0]$,
  a constant sequence of all $0$s. At any level for any neighborhood, the mutual
  information $I(X^\ell; Y^\ell) = 0$. Hence the terms $C_\ell = 0$ for all
  $\ell$ and the overall $\hic(S) = 0$.
\end{example}

\begin{example}
  \label{ex:uniform}
  Let $S = [x_1, x_2, \ldots]$ consist of a sequence of independent draws from a
  multinomial uniform distribution over $K$ categories. Assuming a span of $2$,
  the variable $X^\ell$ consists of $2^\ell$ of the $x_i$ primitives and hence
    can take on any of $K^{(2^\ell)}$ values. Consider the mutual information of level
  $\ell$:
  \begin{equation}
    I(X^\ell; Y^\ell) = H(X^\ell) - H(X^\ell \mid Y^{\ell}) = 2^\ell \log K - 2^\ell \log K = 0
  \end{equation}
  Hence all of the $C_\ell = 0$ and $\hic(S) = 0$.
\end{example}

As examples~\ref{ex:constant} and \ref{ex:uniform} show, the HIC behaves as
expected when the system exhibits complete order or complete disorder. In both
of these examples the HIC was zero because the mutual information at each level
was zero. In the following example, we see slightly more interesting behavior,
where the mutual information at each level is nonzero, but the resulting HIC
is.

\begin{example}
  \label{ex:repeats}
  Let $S = [0, 1, 0, 1, \ldots]$ consist of a sequence of alternating zeros and
  ones.

  The first level $X^1$ takes on the two values $\{0, 1\}$ with equal
  probability hence $H(X^1) = \log 2$. However given the neighbor $Y^1$ the
  value of $X^1$ is deterministic and hence $H(X^1 \mid Y^1) = 0$. Thus the
  overall mutual information is $I(X^1; Y^1) = \log 2$.

  The second level $X^2$ takes on two values $\{[0, 1], [1, 0]\}$ with
  equal probability, and we have $H(X^2) = \log 2$. Again $X^2$ is fully
  determined by $Y^2$ thus $H(X^2 \mid Y^2) = 0$ and $I(X^2, Y^2) = \log 2$.
  Combining these to produce $C^2$ yields:
  \begin{equation}
    C_2 = (I(X^2; Y^2) - I(X^1; Y^1))^2 = (\log 2 - \log 2)^2 = 0.
  \end{equation}
  The above argument generalizes to all levels of the hierarchy so $\hic(S) = 0$.
\end{example}

In example \ref{ex:repeats} we used overlapping windows to construct $X^2$. If
we had chosen disjoint windows the HIC would be small but nonzero. Depending on
the problem this may be desirable in that the alternating sequence should be
considered slightly more complex than a constant sequence. This is where
subjectiveness comes in. The designer should select from these choices
based on the setting at hand.

\subsection{Comparison to Statistical Complexity}
\label{statistical_complexity}

The statistical complexity of \citet{crutchfield1989inferring} measures the
complexity of a system by observing the number of distinct future distributions
given the present and past states.  Statistical complexity infers a set of
``causal states'' by collapsing the state variables representing the current
and past time into equivalence classes which all lead to indistinguishable
future distributions. The entropy of the distribution of these causal states
then serves as a measure of the complexity of the process. For futures which
are ordered or unchanging, the system will have one or a small number of causal
states and the entropy will be small. Similarly, for futures which are
uniformly random, the future distributions will be indistinguishable given the
past and present, and the states will similarly collapse into a single or small
number of causal states. Thus random processes will also have low statistical
complexity.

Let $X_t$ represent the state variable at the $t$-th time step for a system
with $N$ state variables per time step. The past light cone
$\mathcal{X}_{\textrm{P}}$ of $X_t$ with a history of $T_{\textrm{P}}$ time
steps is given by:
\begin{equation}
    \mathcal{X}_{\textrm{P}} =
        \{X_{i, j} \mid i = t - 1, \ldots, t - T_{\textrm{P}}
            \;\;\land\;\; j = 1, \ldots, N
            \;\;\land\;\; X_{i, j} \rightarrow X_t \},
\end{equation}
where $A \rightarrow B$ means the state $A$ can influence $B$ (\emph{i.e.} $B$
is a function of $A$). The future light cone $\mathcal{X}_{\textrm{F}}$ of
$X_t$ with a future of $T_{\textrm{F}}$ time steps is similarly defined as:
\begin{equation}
    \mathcal{X}_{\textrm{F}} =
        \{X_{i, j} \mid i = t + 1, \ldots, t + T_{\textrm{F}}
            \;\;\land\;\; j = 1, \ldots, N
            \;\;\land\;\; X_t \rightarrow X_{i, j} \}.
\end{equation}
In words, the past light cone of $X_t$ is the set of states which can influence
$X_t$ and the future light cone is the set of states which can be influenced by
$X_t$.

Each past light cone and state variable $X_t$ defines a conditional
distribution over future light cones $P(\mathcal{X}_{\textrm{F}} \mid
\mathcal{X}_{\textrm{P}}, X_t)$. We can measure the similarity between two such
conditional distributions using a statistical divergence $D(P_i \| P_j)$. A
causal state $C$ represents a set of states $C = \{(\mathcal{X}_{\textrm{P}},
X_t)\}$ where $D(P_i \| P_j) = 0$ for every pair $(\mathcal{X}_{\textrm{P}},
X_t)_i$ and $(\mathcal{X}_{\textrm{P}}, X_t)_j$ in $C$. In other words, the
distribution of future light cones of $(\mathcal{X}_{\textrm{P}}, X_t)_i$ and
$(\mathcal{X}_{\textrm{P}}, X_t)_j$ are indistinguishable. In practice, we
relax $D(P_i \| P_j) < \tau$ for some predefined threshold due to finite
sample sizes. Furthermore, we only require that $D(P_i \| P_j) < \tau$ be
satisfied for a single $(\mathcal{X}_{\textrm{P}}, X_t)_j$ in $C$ instead of
all such states.

Unlike HIC, statistical complexity is a measure of process complexity. We can
use it to observe how the complexity of a system changes over time. We can also
use statistical complexity as a measure of the global complexity of the system
by applying it to every state at every time step; as we do in this work.

Perhaps more importantly, statistical complexity can be computed in many
cases without much difficulty. The distributions of future light cones given
past light cones and state variables can be estimated for reasonable values of
$T_\textrm{P}$ and $T_\textrm{F}$. \citet{shalizi2004quantifying} estimate the
local statistical complexity of circular cellular automata and show
qualitatively that the measure correlates well to with the perceived complexity
of the automata.


%
% 111
%
\chapter{The Theory of Residues; Application to the Evaluation of Definite Integrals}

\Section{6}{1}{Residues}

If the function $f(z)$ has a pole of order $m$ at $z=a$, then, by the
definition of a pole, an equation of the form
$$
f(z)
=
\frac{a_{-m}}{ (z-a)^{m} }
+ \frac{ a_{-m+1} }{ (z-a)^{m-1} }
+ \cdots
+ \frac{a_{-1}}{z-a}
+ \phi(z),
$$
where $\phi(z)$ is analytic near and at $a$, is true near $a$.

The coefficient $a_{-1}$ in this expansion is called the \emph{residue}\index{Residues!defined} of the
function $f(z)$ relative to the pole $a$.

Consider now the value of the integral $\int_{\alpha} f(z) \dmeasure z$, where the path of
integration is a circle\footnote{The existence of such a circle is
  implied in the definition of a pole as an isolated singularity.}
$\alpha$, whose centre is the point $a$ and whose radius $\rho$ is so small
that $\phi(z)$ is analytic inside and on the circle.

We have
$$
\int_{\alpha} f(z) \dmeasure z
=
\sum_{r=1}^{m}
a_{r}
\int_{\alpha} \frac{\dmeasure z}{ (z-a)^{r}}
+
\int_{\alpha} \phi(z) \dmeasure z.
$$

Now $\int_{\alpha} \phi(z) \dmeasure z = 0$ by \hardsectionref{5}{2};
and (putting $z-a = \rho e^{i\theta}$) we have, if $r \neq 1$,
$$
\int_{\alpha} \frac{ \dmeasure z}{ (z-a)^{r} }
=
\int_{0}^{2\pi} \frac{\rho e^{i\theta} i \dmeasure \theta}{ \rho^{r}
  e^{ri\theta}}
=
\rho^{-r+1}
\int_{0}^{2\pi} e^{(1-r)i\theta} i \dmeasure \theta
=
\rho^{-r+1}
\thebracket{ \frac{e^{(1-r) i \theta}}{1-r} }_{0}^{2\pi}
=
0.
$$

But, when $r = 1$, we have
$$
\int_{\alpha}
\frac{\dmeasure z}{z - a}
=
\int_{0}^{2\pi} i \dmeasure \theta
=
2\pi i.
$$

Hence finally
$$
\int_{\alpha} f(z) \dmeasure z = 2 \pi i a_{-1}.
$$

Now let $C$ be any contour, containing in the region interior to it a
number of poles $a,b,c,\ldots$ of a function $f(z)$, with residues
$a_{-1}, b_{-1}, c_{-1},\ldots$
respectively; and suppose that the function $f(z)$ is
analytic throughout $C$ and its interior, except at these poles.

Surround the points $a,b,c,\ldots$ by circles
$\alpha,\beta,\gamma,\ldots$
so small that their respective centres are the only singularities
inside or on each circle; then the function $f(z)$ is analytic in the
closed region bounded by $C$, $\alpha,\beta,\gamma,\ldots$.
%
% 112
%

Hence, by \hardsectionref{5}{2} corollary TODO,
\begin{align*}
  \int_{C} f(z) \dmeasure z
  =&
  \int_{\alpha} f(z) \dmeasure z
  + \int_{\beta} f(z) \dmeasure z
  + \cdots
  \\
  =&
  2\pi i a_{-1} + 2\pi i b_{-1} + \cdots.
\end{align*}

Thus we have the \emph{theorem of residues}, namely that
\emph{if $f(z)$ be analytic throughout a contour $C$ and its interior
  except at a number of poles inside the contour, then
  $$
  \int_{C} f(z) \dmeasure z = 2\pi i \sum R,
  $$
where $\sum R$ denotes the sum of the residues of the function $f(z)$ at
those of its poles which are situated within the contour $C$.}

This is an extension of the theorem of \hardsubsectionref{5}{2}{1}.

%\begin{Remark} TODO:\alpha or a?
Note. If $a$ is a simple pole of $f(z)$ the residue of $f(z)$ at that pole
is $\lim_{z\rightarrow a} (z-a) f(z)$.
%\end{Remark}

\Section{6}{2}{The evaluation of definite integrals.}

We shall now apply the result of \hardsectionref{6}{1} to evaluating
various classes of definite integrals; the methods to be employed in
any particular case may usually be seen from the following typical
examples.

\Subsection{6}{2}{1}{The evaluation of the integrals of certain
  periodic functions taken between the limits $0$ and
  $2\pi$.\index{Integrals!of periodic functions}\index{Periodic
    functions!integrals involving}}
An integral of the type
$$
\int_{0}^{2\pi} R(\cos \theta, \sin \theta) \dmeasure \theta,\index{Trigonometrical integrals}
$$
where the integrand is a rational function of $\cos\theta$ and
$\sin\theta$ finite on the range of integration, can be evaluated by
writing $e^{i\theta}=z$; since
$$
\cos\theta = \half(z+z^{-1}),
\quad
\sin\theta = \frac{1}{2i} (z-z^{-1}),
$$
the integral takes the form $\int_{C} S(z) \dmeasure z$, where $S(z)$
is a rational function of $z$ finite on the path of integration $C$, the
circle of radius unity whose centre is the origin.

Therefore, by \hardsectionref{6}{1}, \emph{the integral is equal to
  $2\pi i$ times the sum of the residues of $S(z)$ at those of its
  poles which are inside that circle.}

\begin{wandwexample}
  If $0 < p < 1$,
  $$
  \int_{0}^{2\pi}
  \frac{\dmeasure \theta}{ 1 - 2p\cos\theta + p^{2}}
  =
  \int_{C} \frac{ \dmeasure z}{ i (1-pz) (z-p) }.
  $$
  The only pole of the integrand inside the circle is a simple pole at
  $p$; and the residue there is
  $$
  \lim_{z \rightarrow p} \frac{z-p}{ i (1-pz)(z-p)}
  =
  \frac{1}{i (1-p^{2})}.
  $$
%
% 113
%
  Hence
  $$
  \int_{0}^{2\pi} \frac{\dmeasure\theta}{1 - 2p\cos\theta + p^{2}}
  =
  \frac{2\pi}{1-p^{2}}.
  $$
\end{wandwexample}

\begin{wandwexample}
  If $0 < p < 1$,
  \begin{align*}
    \int_{0}^{2\pi} \frac{ \cos^{2} 3\theta}{1-2p\cos 2\theta + p^{2}}
    \dmeasure \theta
    =&
    \int_{C} \frac{\dmeasure z}{iz}
    \theparen{ \half z^{3} + \half z^{-3}}^{2}
    \frac{1}{ (1-pz^{2})(1-pz^{-2})}
  \end{align*}
  where $\sum R$ denotes the sum of the residues of
  $ \frac{ (z^{6} + 1)^{2} }{ 4z^{5} (1-pz^{2})(z^{2}-p)} $ at its poles
  inside $C$; these poles are $0, -p^{\half}, p^{\half}$;
  and the residues at them are
  $ -\frac{1+p^{2}+p^{4}}{4p^{3}},
  \frac{(p^{3}+1)^{2}}{ 8p^{3}(1-p^{2})},
  \frac{(p^{3}+1)^{2}}{ 8p^{3}(1-p^{2})}$; %TODO:same?
  and hence the integral is equal to
  $$
  \frac{\pi (1-p+p^{2})}{1-p}.
  $$
\end{wandwexample}
\begin{wandwexample}
  If $n$ be a positive integer,
  $$
  \int_{0}^{2\pi} e^{\cos\theta} \cos(n\theta - \sin\theta)
  \dmeasure \theta
  =
  \frac{2\pi}{n!},
  \quad
  \int_{0}^{2\pi}
  e^{\cos\theta} \sin(n\theta - \sin\theta) \dmeasure \theta
  =
  0.
  $$
\end{wandwexample}
\begin{wandwexample}
  If $a>b>0$,
  $$
  \int_{0}^{2\pi}
  \frac{\dmeasure \theta}{ (a+b\cos\theta)^{2}}
  =
  \frac{2\pi a}{ (a^{2}-b^{2})^{3/2} },
  \quad
  \int_{0}^{2\pi}
  \frac{\dmeasure\theta}{ (a+b\cos^{2}\theta)^{2}}
  =
  \frac{\pi(2a+b)}{ a^{3/2} (a+b)^{3/2}}.
  $$
\end{wandwexample}
\Subsection{6}{2}{2}{The evaluation of certain types of integrals
  taken between the limits $-\infty$ and $+\infty$.}
We shall now evaluate $\int_{-\infty}^{\infty} Q(x) \dmeasure x$,
where $Q(z)$ is a function such that (i) it is analytic when the
imaginary part of $z$ is positive or zero (except at a finite number
of poles), (ii) it has no poles on the real axis and (iii) as
$\absval{z}\rightarrow\infty$, $zQ(z) \rightarrow 0$ uniformly for all
values of $\arg z$ such that $0 \leq \arg z \leq \pi$; provided that
(iv) when $x$ is real, $x Q(x) \rightarrow 0$, as
$x \rightarrow \pm\infty$, in such a way\footnote{The condition
  $x Q(x)\rightarrow 0$ is not in itself sufficient to secure the
  convergence of
  $\int^{\infty} Q(x) \dmeasure x$;
  consider $Q(x) = (x \log x)^{-1}$.}
that $\int_{0}^{\infty} Q(x) \dmeasure x$ and
$\int_{-\infty}^{0} Q(x) \dmeasure x$ both converge.

Given $\eps$, we can choose $\rho_{0}$ (independent of $\arg z$) such
that $\absval{z Q(z)} < \eps/\pi$ whenever
$\absval{z} > \rho_{0}$ and $0 \leq \arg z \leq \pi$.

Consider $\int_{C} Q(z) \dmeasure z$ taken round a contour $C$
consisting of the part of the real axis joining the points $\pm\rho$
(where $\rho > \rho_{0}$) and a semicircle $\Gamma$, of radius $\rho$,
having its centre at the origin, above the real axis.

Then, by \hardsectionref{6}{1}, $\int_{C} Q(z) \dmeasure z = 2 \pi i
\sum R$, where $\sum R$ denotes the sum of the residues of $Q(z)$ at
its poles above the real axis.\footnote{$Q(z)$ has no poles above the
  real axis outside the contour.}

%
% 114
%
Therefore
$$
\absval{
  \int_{-\rho}^{\rho} Q(z) \dmeasure z
  -
  2 \pi i \sum R
}
=
\absval{
  \int_{\Gamma} Q(z) \dmeasure z
}.
$$

In the last integral write $z = \rho e^{i\theta}$, and then
\begin{align*}
  \absval{ \int_{\Gamma} Q(z) \dmeasure z  }
  &=
  \absval{
    \int_{0}^{\pi} Q(\rho e^{i\theta}) \rho e^{i\theta} i
    \dmeasure\theta
  }
  \\
  &<
  \int_{0}^{\pi} (\eps / \pi) \dmeasure \theta
  \\
  &= \eps
\end{align*}
by \hardsubsectionref{4}{6}{2}.

Hence
$$
\lim_{\rho\rightarrow\infty}
\int_{-\rho}^{\rho} Q(z) \dmeasure z
=
2\pi i \sum R.
$$

But the meaning of $\int_{-\infty}^{\infty} Q(x) \dmeasure x$;
is $\lim_{\rho,\sigma\rightarrow\infty}\int_{-\rho}^{\sigma} Q(x)
\dmeasure x$;
and since
$\lim_{\sigma\rightarrow\infty}\int_{0}^{\sigma} Q(x) \dmeasure x$
and
$\lim_{\rho\rightarrow\infty}\int_{-\rho}^{0} Q(x) \dmeasure x$
both exist, this double limit is the same as
$\lim_{\rho\rightarrow\infty}\int_{-\rho}^{\rho} Q(x) \dmeasure x$.

Hence we have proved that
$$
\int_{-\infty}^{\infty} Q(x) \dmeasure x = 2\pi i \sum R.
$$

This theorem is particularly useful in the special case when $Q(x)$ is a
rational function.

%\begin{smallfont}
[Note. Even if condition (iv) is not satisfied, we still have
$$
\int_{0}^{\infty}
\thebrace{
  Q(x) + Q(-x)
} \dmeasure x
=
\lim_{\rho\rightarrow\infty} \int_{-\rho}^{\rho} Q(x) \dmeasure x
= 2 \pi i \sum R.]
$$
%\end{smallfont}

\begin{wandwexample}
  The only pole of $(z^{2} + 1)^{-3}$ in the upper half plane is a
  pole at $z=i$ with residue there $-\frac{3}{16} i$. Therefore
  $$
  \int_{-\infty}^{\infty} \frac{\dmeasure x}{ (x^{2}+1)^{3} }
  =
  \frac{3}{8} \pi.
  $$
\end{wandwexample}
\begin{wandwexample}
If $a > 0, b > 0$, shew that
$$
\int_{-\infty}^{\infty}
\frac{x^{4} \dmeasure x}{ (a + bx^{2})^{4} }
=
\frac{\pi}{ 16 a^{3/2} b^{5/2}}.
$$
\end{wandwexample}
\begin{wandwexample}
  By integrating $\int e^{-\lambda z^{2}} \dmeasure z$ round a
parallelogram whose corners are $-R, R, R + ai, -R + ai$ and making $R
\rightarrow \infty$, shew that, if $\lambda > 0$, then
  $$
  \int_{-\infty}^{\infty} e^{-\lambda x^{2}} \cos (2\lambda a x)
\dmeasure x = e^{-\lambda a^{2}} \int_{-\infty}^{\infty} e^{-\lambda
x^{2}} \dmeasure x = 2 \lambda^{-\half} e^{-\lambda a^{2}}
\int_{0}^{\infty} e^{-x^{2}} \dmeasure x.
  $$
\end{wandwexample}
\Subsubsection{6}{2}{2}{1}{Certain infinite integrals involving sines
and cosines.}
If $Q(z)$ satisfies the conditions (i), (ii) and (iii) of
\hardsectionref{6}{2}{2}, and $m > 0$,
then $Q(z) e^{miz}$ also satisfies those conditions.

%
% 115
%

Hence
$
\int_{0}^{\infty}
\thebrace{
  Q(x) e^{mix} + Q(-x) e^{-mix}
}
\dmeasure x
$
is equal to $2\pi i \sum R'$, where
$\sum R'$ means the sum of the residues of $Q(z) e^{mix}$ at its poles
in the upper half plane; and so
\begin{itemize}
\item % (i)
  If $Q(x)$ is an even function\index{Even functions}, i.e. if $Q (- x) = Q (x)$,
  $$
  \int_{0}^{\infty} Q(x) \cos(mx) \dmeasure x
  =
  \pi i \sum R'.
  $$
\item %(ii)
  If $Q(x)$ is an odd function\index{Odd functions},
  $$
  \int_{0}^{\infty} Q(x) \sin(mx) \dmeasure x
  =
  \pi \sum R'.
  $$
\end{itemize}
\Subsubsection[Jordan's lemma.]{6}{2}{2}{2}{Jordan's lemma.\footnote{TODO:Jordan, Cours d'Aiiahjse, ii. (1894), pp. 285, 286.}\index{Jordan's lemma}}

The results of \hardsubsubsectionref{6}{2}{2}{1} are true if $Q(z)$ be
subject to the less stringent condition $Q(z) \rightarrow 0$ uniformly
when
$O \leq \arg z \leq \pi$ as $\absval{z} \rightarrow \infty$ in
place of the condition $z Q(z) \rightarrow 0$ uniformly.

To prove this we require a theorem known as Jordan's lemma, viz.

\emph{If $Q(z) \rightarrow 0$ uniformly with regard to $\arg z$ as
  $\absval{z} \rightarrow \infty$ when $0 \leq \arg z \leq \pi$,
  and if $Q(z)$ is analytic when both $\absval{z} > c$ (a constant)
  and $0 \leq \arg z \leq \pi$, then
  $$
  \lim_{\rho\rightarrow\infty}
  \theparen{ \int_{\Gamma} e^{miz} Q(z) \dmeasure z  }
  =
  0,
  $$
  where $\Gamma$ is a semicircle of radius $\rho$ above the real axis with centre at
  the origin.}

Given $\eps$, choose $\rho_{0}$ so that $\absval{Q(z)} < \eps/\pi$
when $\absval{z} > \rho_{0}$ and
$0 \leq \arg z \leq \pi$;
then, if $\rho > \rho_{0}$,
$$
\absval{ \int_{\Gamma} e^{miz} Q(z) \dmeasure z }
=
\absval{
  \int_{0}^{\pi}
  e^{mi (\rho \cos\theta + i\rho \sin\theta)}
  Q(\rho e^{i\theta})
  \rho e^{i \theta}
  i \dmeasure \theta
}.
$$
But $\absval{ e^{mi\rho\cos\theta} } = 1$, and so
\begin{align*}
  \absval{ \int_{\Gamma} e^{miz} Q(z) \dmeasure z }
  <&
  \int_{0}^{\pi} (\eps/\pi) \rho e^{-m\rho\sin\theta} \dmeasure\theta
  \\
  =&
  (2\eps/\pi)
  \int_{0}^{\half\pi} \rho e^{-m\rho\sin\theta} \dmeasure\theta.
\end{align*}

Now $\sin\theta \geq 2\theta/\pi$, when\footnote{This inequality
  appears obvious when we draw the graphs $y = \sin x$, $y = 2x/\pi$;
  it may be proved by shewing that $(\sin\theta)/\theta$ decreases as
  $\theta$ increases from $0$ to $\half\pi$.
} $0 \leq \theta \leq \half\pi$, and so
\begin{align*}
  \absval{ \int_{\Gamma} e^{miz} Q(z) \dmeasure z }
  <&
  (2\eps/\pi)
  \int_{0}^{\half\pi}
  \rho e^{-2m\rho\theta/\pi} \dmeasure\theta
  \\
  =&
  (2\eps/\pi) \cdot (\pi/2m)
  \thebracket{ -e^{-2m\rho\theta/\pi} }_{0}^{\half\pi}
  \\
  <&
  \eps/m.
\end{align*}

%
% 116
%

Hence
$$
\lim_{\rho\rightarrow\infty} \int_{\Gamma} e^{miz} Q(z) \dmeasure z = 0.
$$

This result is Jordan's lemma.

Now
$$
\int_{0}^{\rho}
\thebrace{
  e^{mix} Q(x) + e^{-mix} Q(-x)
}
\dmeasure x
=
2 \pi i \sum R'
-
\int_{\Gamma} e^{miz} Q(z) \dmeasure z,
$$
and, making $\rho\rightarrow\infty$, we see at once that
$$
\int_{0}^{\infty}
\thebrace{
  e^{mix} Q(x) + e^{-mix} Q(-x)
}
\dmeasure x
=
2 \pi i \sum R',
$$
which is the result corresponding to the result of \hardsubsubsectionref{6}{2}{2}{1}.

\begin{wandwexample}
  Shew that, if $a > 0$, then
  $$
  \int_{0}^{\infty}
  \frac{ \cos x}{x^{2} + a^{2}} \dmeasure x
  =
  \frac{\pi}{2a} e^{-a}.
  $$
\end{wandwexample}
\begin{wandwexample}
  Shew that, if $a \geq 0, b \geq 0$, then
  $$
  \int_{0}^{\infty}
  \frac{ \cos 2ax - \cos 2bx}{x^{2}}
  \dmeasure x
  =
  \pi (b-a)
  $$
  (Take a contour consisting of a large semicircle of radius $\rho$, a
  small semicircle of radius $\delta$, both having their centres at the
  origin, and the parts of the real axis joining their ends; then
  make $\rho \rightarrow \infty, \delta \rightarrow 0$.)
\end{wandwexample}
\begin{wandwexample}
  Shew that, if $b > 0, m \geq 0$, then
  $$
  \int_{0}^{\infty}
  \frac{3x^{2} - a^{2}}{ (x^{2} + b^{2})^{2} }
  \cos mx
  \dmeasure x
  =
  \frac{\pi e^{-mb}}{4b^{3}}
  \thebrace{
    3b^{2} - a^{2} - mb(3b^{2}+a^{2})
  }.
  $$
\end{wandwexample}
\begin{wandwexample}
  Shew that, if $k > 0, a > 0$, then
  $$
  \int_{0}^{\infty}
  \frac{ x \sin ax }{x^{2} + k^{2}}
  \dmeasure x
  =
  \half \pi e^{-ka}.
  $$
\end{wandwexample}
\begin{wandwexample}
  Shew that, if $m \geq 0$, $a > 0$, then
  $$
  \int_{0}^{\infty}
  \frac{ \sin mx }{ x (x^{2} + a^{2})^{2} }
  \dmeasure x
  =
  \frac{\pi}{2a^{4}}
  -
  \frac{\pi e^{-ma}}{4a^{3}}
  \theparen{ m + \frac{2}{a} }.
  $$
  (Take the contour of example TODO.)
\end{wandwexample}
\begin{wandwexample}
Shew that, if the real part of $z$ be positive,
$$
\int_{0}^{\infty} (e^{-t} - e^{-tz}) \frac{\dmeasure t}{t} = \log z.
$$
[We have
\begin{align*}
  \int_{0}^{\infty} (e^{-t} - e^{-tz}) \frac{\dmeasure t}{t}
  =&
  \lim_{\delta\rightarrow 0,\, \rho\rightarrow\infty}
  \thebrace{
    \int_{\delta}^{\rho} \frac{e^{-t}}{t} \dmeasure t
    -
    \int_{\delta}^{\rho} \frac{e^{-tz}}{t} \dmeasure t
  }
  \\
  =&
  \lim_{\delta\rightarrow 0,\, \rho\rightarrow\infty}
    \thebrace{
    \int_{\delta}^{\rho} \frac{e^{-t}}{t} \dmeasure t
    -
    \int_{\delta z}^{\rho z} \frac{e^{-u}}{u} \dmeasure u
  }
  \\
  =&
  \lim_{\delta\rightarrow 0,\, \rho\rightarrow\infty}
    \thebrace{
    \int_{\delta}^{\delta z} \frac{e^{-t}}{t} \dmeasure t
    -
    \int_{\rho}^{\rho z} \frac{e^{-t}}{t} \dmeasure t,
  },
\end{align*}
since $t^{-1} e^{-t}$ is analytic inside the
quadrilateral whose corners are $\delta, \delta z, \rho z, \rho$.

%
% 117
%

Now $\int_{\rho}^{\rho z} t^{-1} e^{-t} \dmeasure t \rightarrow 0$
when $\Re(z) > 0$; and
$$
\int_{\delta}^{\delta z} t^{-1} e^{-t} \dmeasure t
= \log z
-
\int_{\delta}^{\delta z} t^{-1} (1 - e^{-t}) \dmeasure t
\rightarrow
\log z,
$$
since $t^{-1} (1 - e^{-t}) \rightarrow 1$ as $t \rightarrow 0$.]
\end{wandwexample}
\Subsection{6}{2}{3}{Principal values of
  integrals.\index{Integrals!principal values of}\index{Principal part
  of a function!value of an integral}}
%\begin{smallfont}
It was assumed in TODO that the function $Q(x)$ had no
poles on the real axis; if the function has a finite number of \emph{simple}
poles on the real axis, we can obtain theorems corresponding to those
already obtained, except that the integrals are all principal values
(\hardsectionref{4}{5}) and $\sum R$ has to be replaced by
$\sum R + \half \sum R_{0}$, where $\sum R_{0}$ means the
sum of the residues at the poles on the real axis. To obtain this
result we see that, instead of the former contour, we have to take as
contour a circle of radius $\rho$ and the portions of the real axis joining
the points
$$
-\rho, \ a-\delta_{1};
\quad
a+\delta_{1}, \ b-\delta_{2};
\quad
b+\delta_{2}, \ c-\delta_{3},
\ \ldots
$$
and small semicircles above the real axis of radii
$\delta_{1},\delta_{2},\ldots$ with
centres $a, b, c, \ldots$ where $a, b, c, \ldots$ are the poles of
$Q(z)$ on the real axis; and then we have to make
$\delta_{1}, \delta_{2}, \ldots \rightarrow 0$;
call these semicircles $\gamma_{1},\gamma_{2},\ldots$.
Then instead of the equation
$$
\int_{-\rho}^{\rho} Q(z) \dmeasure z
+
\int_{\Gamma} Q(z) \dmeasure z
=
2 \pi i \sum R,
$$
we get
$$
P \int_{-\rho}^{\rho} Q(z) \dmeasure z
+
\sum_{n}
\lim_{\delta_{n} \rightarrow 0}
\int_{\gamma_{n}} Q(z) \dmeasure z
+
\int_{\Gamma} Q(z) \dmeasure z
=
2 \pi i \sum R.
$$

Let $a'$ be the residue of $Q(z)$ at $a$; then writing
$z = a + \delta_{1} e^{i\theta}$ on $\gamma_{1}$ we get
$$
\int_{\gamma_{1}} Q(z) \dmeasure z
=
\int_{\pi}^{0}
Q(a + \delta_{1} e^{i\theta})
\delta_{1} e^{i\theta} i \dmeasure \theta.
$$
But $Q(a + \delta_{1} e^{i\theta}) \rightarrow a'$
uniformly as $\delta_{1} \rightarrow 0$; and therefore
$
\lim_{\delta_{1} \rightarrow 0} \int_{\gamma_{1}} Q(z) \dmeasure z
=
- \pi i a'
$;
we thus get
$$
P \int_{-\rho}^{\rho} Q(z) \dmeasure z
+
\int_{\Gamma} Q(z) \dmeasure z
=
2 \pi i \sum R
+
\pi i \sum R_{0},
$$
and hence, using the arguments of \hardsubsectionref{6}{2}{2}, we get
$$
P \int_{-\infty}^{\infty} Q(x) \dmeasure x
=
2 \pi i \theparen{ \sum R + \half \sum R_{0} }.
$$

The reader will see at once that the theorems of TODO have
precisely similar generalisations.

The process employed above of inserting arcs of small circles so as to
diminish the area of the contour is called \emph{indenting} the contour.
%\end{smallfont}
\Subsection{6}{2}{4}{Evaluation of integrals of the form
  $\int_{0}^{\infty} x^{a-1} Q(x) \dmeasure x.$}
Let $Q(x)$ be a rational function of $x$ such that it has no poles on the
positive part of the real axis and
$x^{a} Q(x) \rightarrow 0$ both when $x \rightarrow 0$ and when
$x \rightarrow \infty$.

%
% 118
%

Consider $\int (-z)^{a-1} Q(z) \dmeasure z$ taken round the contour
$C$ shewn in the figure TODO, consisting of the arcs of circles of
radii $\rho,\delta$ and the straight lines joining their end points;
$(-z)^{a-1}$ is
to be interpreted as
$$
\exp \thebrace{ ( a-1) \log (- z) }
$$
and
$$
\log (-z) = \log \absval{z} + \arg (-z),
$$
where
$$
-\pi \leq \arg (-z) \leq \pi; %TODO:is overlap correct?
$$
with these conventions the integrand is one-valued and analytic on and
within the contour save at the poles of $Q(z)$.

Hence, if $\sum r$ denote the sum of the residues of $(-z)^{a-1} Q(z)$
at all its poles,
$$
\int_{C} (-z)^{a-1} Q(z) \dmeasure z = 2 \pi i \sum r.
$$

On the small circle write $-z = \delta e^{i\theta}$, and the integral
along it becomes
$- \int_{\pi}^{-\pi} (-z)^{a} Q(z) i \dmeasure \theta$,
which tends to zero as $\delta \rightarrow 0$.

On the large semicircle write $-z = \rho e^{i\theta}$, and the
integral along it becomes $- \int_{-\pi}^{\pi} (-z)^{n} Q(z) i
\dmeasure\theta$, which tends to zero as $\rho \rightarrow \infty$.

On one of the lines we write $-z = x e^{\pi i}$ on the other
$-z = x e^{-\pi i}$ and $(-z)^{a-1}$ becomes $x^{a-1} e^{\pm (a-1) \pi i}$.

Hence
$$
\lim_{\delta\rightarrow 0,\ \rho\rightarrow\infty}
\int_{\delta}^{\rho}
\thebrace{
  x^{a-1} e^{-(a-1)\pi i} Q(x)
  -
  x^{a-1} e^{(a-1)\pi i} Q(x)
}
\dmeasure x
=
2 \pi i \sum r;
$$
and therefore
$$
\int_{0}^{\infty} x^{a-1} Q(x) \dmeasure x
=
\pi \cosec (a\pi) \sum r.
$$

%TODO: change corollary environment
Corollary. If $Q(x)$ have a number of simple poles on the positive
part of the real axis, it may be shewn by indenting the contour that
$$
P \int_{0}^{\infty} x^{a-1} Q(x) \dmeasure x
=
\pi \cosec (a\pi) \sum r
-
\pi \cot (1\pi) \sum r',
$$
where $\sum r'$ is the sum of the residues of $z^{a-1} Q(z)$ at these poles.
\begin{wandwexample}
If $0 < a < 1$,
$$
\int_{0}^{\infty} \frac{x^{a-1}}{1+x} \dmeasure x
=
\pi \cosec a\pi,
\quad
P \int_{0}^{\infty} \frac{x^{a-1}}{1+x} \dmeasure x
=
\pi \cot a\pi
$$
\end{wandwexample}
%
% 119
%
\begin{wandwexample}
  If $0 < z < 1$ and $-\pi < a < \pi$,
  $$
  \int_{0}^{\infty} \frac{t^{z-1}}{ t+e^{ia} } \dmeasure t
  =
  \frac{ \pi e^{i(z-1)a}}{\sin \pi z}.
  $$
\addexamplecitation{Minding.}
\end{wandwexample}
\begin{wandwexample}
  Shew that, if $- 1 < z < 3$, then
  $$
  \int_{0}^{\infty} \frac{ x^{z} }{ (1+x^{2})^{2} } \dmeasure x
  =
  \frac{ \pi (1-z) }{ 4 \cos \half\pi z}.
  $$
\end{wandwexample}
\begin{wandwexample}
  Shew that, if $-1 < p < 1$ and $-\pi < \lambda < \pi$, then
  $$
  \int_{0}^{\infty}
  \frac{ x^{-p} \dmeasure x }{ 1 + 2x \cos\lambda + x^{2} }
  =
  \frac{\pi}{\sin p\pi} \frac{\sin p\lambda}{\sin \lambda}.
  $$
  \addexamplecitation{Euler.}
\end{wandwexample}
\Section{6}{3}{Cauchy's integral.}
We shall next discuss a class of contour-integrals which are
sometimes found useful in analytical investigations.

Let $C$ be a contour in the $z$-plane, and let $f(z)$ be a function
analytic inside and on $C$. Let $\phi(z)$ be another function which is
analytic inside and on $G$ except at a finite number of poles; let the
zeros of $\phi(z)$ in the interior\footnote{$\phi(z)$ must not have
  any zeros or poles on $C$.} of $C$ be $a_{1},a_{2},\ldots$, and let
their degrees of multiplicity be $r_{1},r_{2},\ldots$; and let its
poles in the interior of $C$ be $b_{1},b_{2},\ldots$, and let their
degrees of multiplicity be $s_{1},s_{2},\ldots$.

Then, by the fundamental theorem of residues,
$ \frac{1}{2\pi i} \int_{C} f(z) \frac{\phi'(z)}{\phi(z)} \dmeasure z $
is equal to the sum of the residues of
$f(z) \frac{\phi'(z)}{\phi(z)}$
at its poles inside $C$.

Now
$f(z) \frac{\phi'(z)}{\phi(z)}$
can have singularities only at the poles and zeros of $\phi(z)$.
Near one of the zeros, say $a_{1}$, we have
$$
\phi(z)
=
A (z-a_{1})^{r_{1}}
+ B (z-a_{1})^{r_{1}+1}
+ \cdots.
$$

Therefore
$$
\phi'(z)
=
A r_{1} (z-a_{1})^{r_{1}-1}
+ B (r_{1}+1) (z-a_{1})^{r_{1}}
+ \cdots,
$$
and
$$
f(z) = f(a_{1}) + (z-a_{1}) f'(a_{1}) + \cdots.
$$

Therefore
$
\thebrace{f(z) \frac{\phi'(z)}{\phi(z)}
  -
  \frac{ r_{1} f(a_{1}) }{ z - a_{1} }
}
$ is analytic at $a_{1}$.

Thus the residue of $f(z) \frac{\phi'(z)}{\phi(z)}$, at the point
$z=a_{1}$, is $r_{1} f(a_{1})$.

Similarly the residue at $z=b_{1}$ is
$-s_{1} f(b_{1})$; for near $z=b_{1}$, we have
$$
\phi(z)
=
C (z-b_{1})^{-s_{1}}
+ D (z-b_{1})^{-s_{1} + 1}
+ \cdots,
$$
and
$$
f(z)
=
f(b_{1})
+ (z - b_{1}) f'(b_{1})
+ \cdots,
$$
so
$
f(z) \frac{ \phi'(z) }{\phi(z)}
+
\frac{ s_{1} f(b_{1}) }{ z-b_{1} }
$
is analytic at $b_{1}$.

Hence
$$
\frac{1}{2\pi i}
\int_{C} f(z) \frac{ \phi'(z) }{\phi(z)} \dmeasure z
=
\sum r_{1} f(a_{1})
-
\sum s_{1} f(b_{1}),
$$
the summations being extended over all the zeros and poles of
$\phi(z)$.

\Subsection{6}{3}{1}{The number of roots of an equation contained
  within a contour.\index{Roots of an equation, number of!(inside a contour)}}
The result of the preceding paragraph can be at once applied to find
how many roots of an equation $\phi(z) = 0$ lie within a contour $C$.

For, on putting $f(z) = 1$ in the preceding result, we obtain the
result that
$$
\frac{1}{2\pi i}
\int_{C} \frac{ \phi'(z) }{\phi(z)} \dmeasure z
$$
is equal to the excess of the number of zeros over the number of poles
of $\phi(z)$ contained in the interior of $C$, each pole and zero
being reckoned according to its degree of multiplicity.

%
% 120
%
\begin{wandwexample}
  Shew that a polynomial $\phi(z)$ of degree $m$ has $m$ roots.
  \index{Equation of degree $m$ has $m$ roots}\index{Roots of an
    equation!number of}

Let
$$
\phi(z)
=
a_{0} z^{m} + a_{1} z^{m-1} + \cdots + a_{m},
\quad
(a_{0} \neq 0).
$$
Then
$$
\frac{ \phi'(z) }{ \phi(z) }
=
\frac{ m a_{m}z^{m-1} + \cdots + a_{m-1} }{ a_{0}z^{m} + \cdots a_{m}}.
$$
Consequently, for large values of $\absval{z}$,
$$
\frac{ \phi'(z) }{ \phi(z) }
=
\frac{m}{z} + O\theparen{ \frac{1}{z^{2}} }
$$
Thus, if $C$ be a circle of radius $\rho$ whose centre is at the origin, we
have
$$
\frac{1}{2 \pi i} \int_{C} \frac{\phi'(z)}{\phi(z)} \dmeasure z
=
\frac{m}{2 \pi i} \int_{C} \frac{ \dmeasure z }{z}
+
\frac{1}{2 \pi i} \int_{C} O\theparen{ \frac{1}{z^{2}} } \dmeasure z
=
m + \frac{1}{2 \pi i} \int_{C} O\theparen{ \frac{1}{z^{2}} } \dmeasure z.
$$

But, as in \hardsectionref{6}{2}{2},
$$
\int_{C} O\theparen{ \frac{1}{z^{2}} } \dmeasure z
\rightarrow
0
$$
as $\rho\rightarrow\infty$; and hence as $\phi(z)$ has no poles in the
interior of $C$, the total number of zeros of $\phi(z)$ is
$$
\lim_{\rho\rightarrow\infty}
\frac{1}{2\pi i}
\int_{C} \frac{ \phi'(z) }{\phi(z)} \dmeasure z
=
m.
$$
\end{wandwexample}
\begin{wandwexample}
If at all points of a contour $C$ the inequality
$$
\absval{ a_{k} z^{k} }
>
\absval{
  a_{0}
  + a_{1} z
  + \cdots
  + a_{k-1} z^{k-1}
  + a_{k+1} z^{k+1}
  + \cdots
  + a_{m} z^{m}
}
$$
is satisfied, then the contour contains $k$ roots of the equation
$$
a_{m} z^{m} + a_{m-1} z^{m-1} + \cdots a_{1} z + a_{0} = 0.
$$

For write
$$
f(z)
=
a_{m} z^{m} + a_{m-1} z^{m-1} + \cdots a_{1} z + a_{0}.
$$

Then
\begin{align*}
  f(z)
  &=
  a_{k} z^{k}
  \theparen{ 1 +
    \frac{ a_{m}z^{m}
      + \cdots
      + a_{k+1}z^{k+1}
      + a_{k-1}z^{k-1}
      + \cdots
      + a_{0}
    }{ a_{k} z^{k} }
  }
  \\
  &= a_{k} z^{k} (1 + U),
\end{align*}
where $\absval{U} \leq a \leq 1$ on the contour, $a$ being
independent\footnote{$\absval{U}$ is a continuous function of $z$ on
  $C$, and so \emph{attains} its upper bound
  (\hardsubsectionref{3}{6}{2}). Hence its upper bound $a$ must be
  less than $1$.} of $z$.

Therefore the number of roots of $f(z)$ contained in $C$
$$
= \frac{1}{2 \pi i} \int_{C} \frac{f'(z)}{f(z)} \dmeasure z
= \frac{1}{2 \pi i}
\int_{C} \theparen{ \frac{k}{z}
  + \frac{1}{1+U} \frac{ \dd U }{ \dd z }
} \dmeasure z
$$

But $\int_{C} \frac{\dd z}{z} = 2 \pi i$; and, since $\absval{U} < 1$,
we can expand $(1 + U)^{-1}$ in the uniformly convergent series
$$
1 - U + U^{2} - U^{3} + \cdots,
$$
so
$$
\int_{C} \frac{1}{1+U} \frac{\dd U}{\dd z} \dmeasure z
=
\thebracket{
  [U - \half U^{2} + \frac{1}{3} U^{3} - \cdots]
}_{C}
=
0.
$$

Therefore the number of roots contained in $C$ is equal to $k$.
\end{wandwexample}
\begin{wandwexample}
Find how many roots of the equation
$$
z^{6} + 6z + 10 = 0
$$
lie in each quadrant of the Argand diagram.
\addexamplecitation{Clare, 1900.}
\end{wandwexample}
%
% 121
%

\Section{6}{4}{Connexion between the zeros of a function and the zeros of its
derivative.}\index{Zeros of a function!connected with zeros of its derivative}
Macdonald\footnote{TODO} has shewn that \emph{if $f(z)$ be a function of $z$
analytic throughout the interior of a single closed contour $C$, defined
by the equation $\absval{f(z)} = M$, where $M$ is a constant, then the
number of zeros of $f(z)$ in this region exceeds the number of zeros of
the derived function $f'(z)$ in the same region by unity.}

On $C$ let $f(z) = M e^{i\theta}$; then at points on $C$
$$
f'(z) = M e^{i\theta} i \frac{\dd \theta}{\dd z},
\quad
f''(z)
=
M e^{i\theta}
\thebrace{
  i \frac{ \dd^{2} \theta }{ \dd z^{2} }
  -
  \theparen{ \frac{\dd \theta}{\dd z} }^{2}
}.
$$
Hence, by \hardsectionref{6}{3}{1}, the excess of the number of zeros
of $f(z)$ over the number of zeros of $f'(z)$
inside\footnote{$f'(z)$ does not vanish on $C$ unless $C$ has a node
  or other singular point; for, if $f = \phi + i\psi$,
  where $\phi$ and $\psi$ are real, since
  $i \frac{\partial f}{\partial x} = \frac{\partial f}{\partial y}$,
  it follows that if $f'(z) = 0$ at any point, then
  $\frac{\partial \phi}{\partial x},
  \frac{\partial \phi}{\partial y},
  \frac{\partial \psi}{\partial x},
  \frac{\partial \psi}{\partial y}$
  all vanish; and these are sufficient conditions for a singular point
  on  $\phi^{2} + \psi^{2} = M^{2}$.
} $C$ is
$$
\frac{1}{2\pi i}
\int_{C} \frac{ f'(z) }{f(z)} \dmeasure z
-
\frac{1}{2\pi i}
\int_{C} \frac{ f''(z) }{ f'(z) } \dmeasure z
=
-\frac{1}{2 \pi i}
\int_{C} \theparen{
  \left.
    \frac{ \dd^{2} \theta }{ \dd z^{2} }
  \right/
  \frac{\dd\theta}{\dd z}
  } \dmeasure z.
$$
Let $s$ be the arc of $C$ measured from a fixed point and let $\psi$
be the angle the tangent to $C$ makes with $Ox$; then
\begin{align*}
-\frac{1}{2 \pi i}
\int_{C} \theparen{
  \left.
    \frac{ \dd^{2} \theta }{ \dd z^{2} }
  \right/
  \frac{\dd\theta}{\dd z}
  } \dmeasure z
  &=
  - \frac{1}{2 \pi i}
  \thebracket{ \log \frac{\dd\theta}{\dd z} }_{C}
  \\
  &=
  - \frac{1}{2 \pi i}
  \thebracket{
    \log \frac{\dd\theta}{\dd s}
    -
    \log \frac{\dd z}{\dd s}
  }_{C}
  .
\end{align*}
Now $\log \frac{\dd \theta}{\dd s}$ is purely real and its initial
value is the same as its final value; and
$\log \frac{\dd z}{\dd s} = i \psi$; hence the excess of the number of
zeros of $f(z)$ over the number of zeros of
$f'(z)$ is the change in $\psi/2\pi$ in describing the curve $C$; and it is
obvious\footnote{For a formal proof, see TODO} that if $C$ is any
ordinary curve, $\psi$ increases by $2\pi$ as the point of contact of
the tangent describes the curve $C$; this gives the required result.
\begin{wandwexample}
  Deduce from Macdonald's result the theorem that a
  polynomial of degree $n$ has $n$ zeros.
\end{wandwexample}
\begin{wandwexample}
  Deduce from Macdonald's result that if a function $f(z)$,
  analytic for real values of $z$, has all its coefficients real, and all
  its zeros real and different, then between two consecutive zeros of
  $f(z)$ there is one zero and one only of $f'(z)$.
\end{wandwexample}
REFERENCES.

M. C. Jordan, Cours dJ Analyse, 11. (Paris, 1894), Ch. vi.

E. Goursat, Cours d' Analyse (Paris, 1911), Ch. xiv.

E. LiNDELOF, Le Calcid des Residus (Paris, 1905), Ch. il.

%
% 122
%

Miscellaneous Examples.
\begin{enumerate}
\item  A function $\phi(z)$ is zero when $z=0$, and is real when $z$
  is real, and is analytic when $\absval{z} \leq 1$; if $f(x,y)$ is
  the coefficient of $i$ in $\phi(x + iy)$, prove that if $-1 < x < 1$,
  $$
  \int_{0}^{2\pi}
  \frac{x \sin\theta }{ 1 - 2x\cos\theta + x^{2}}
  f(\cos\theta, \sin\theta)
  \dmeasure\theta
  =
  \pi \phi(x).
  $$
  \addexamplecitation{Trinity, 1898.}
\item
  By integrating $\frac{e^{\pm aiz}}{e^{2\pi z}-1}$ round a contour formed by the rectangle whose
  corners are $0, R, R+i, i$ (the rectangle being indented at $0$ and
  $i$) and making $R\rightarrow\infty$,
  shew that
  $$
  \int_{0}^{\infty}
  \frac{ \sin ax }{e^{2 \pi x} - 1}
  \dmeasure x
  =
  \frac{1}{4}
  \frac{ e^{a} + 1 }{ e^{a} - 1 }
  -
  \frac{1}{2a}.
  $$
  \addexamplecitation{Legendre.}

\item
  By integrating $\log (-z) Q(z)$ round the contour of
  \hardsubsectionref{6}{2}{4}, where $Q(z)$ is a rational function such
  that  $z Q(z) \rightarrow 0$ as $\absval{z} \rightarrow 0$ and as
  $\absval{z} \rightarrow \infty$, shew that if $Q(z)$ has no poles
  on the positive part of the real axis,
  $\int_{0}^{\infty} Q(x) \dmeasure x$ is equal to minus
  the sum of the residues of $\log(-z) Q(z)$ at the poles of
  $Q(z)$; where the imaginary part of $\log(-z)$ lies between $\pm\pi$.

\item
  Shew that, if $a > 0, b > 0$,
  $$
  \int_{0}^{\infty}
  e^{a \cos bx} \sin (a \sin bx) \frac{\dmeasure x}{x}
  =
  \half \pi \theparen{ e^{a} - 1 }.
  $$

\item
  Shew that
  $$
  \int_{0}^{\half\pi}
  \frac{a \sin 2x}{1 - 2a \cos 2x + a^{2}}
  x \dmeasure x
  =
  \begin{cases}
    \frac{1}{4} \pi \log (1+a), & (-1<a<1)
    \\
    \frac{1}{4} \pi \log (1+a^{-1}), & (a^{2} > 1).
  \end{cases}
  $$
%  \addexamplecitation{Cauchy.}
\item Shew that
  $$
  \int_{0}^{\infty}
  \frac{\sin \phi_{1}x}{x}
  \frac{\sin \phi_{2}x}{x}
  \cdots
  \frac{\sin \phi_{n}x}{x}
  \cos a_{1} x
  \cdots
  \cos a_{m} x
  \frac{ \sin ax }{x}
  \dmeasure x
  =
  \frac{\pi}{2} \phi_{1} \phi_{2} \cdots \phi_{n},
  $$
  if $\phi_{1}, \phi_{2}, \ldots \phi_{n}, a_{1}, a_{2}, \ldots a_{m}$
  be real and $a$ be positive and
  $$
  a
  >
  \absval{\phi_{1}}
  + \absval{\phi_{2}}
  + \cdots
  + \absval{\phi_{n}}
  + \absval{a_{1}}
  + \cdots
  + \absval{a_{m}}.
  $$
  \addexamplecitation{TODO Stormer, Acta Math, xix.}
  %
\item
  If a point $z$ describes a circle $C$ of centre $a$, and if
  $f(z)$ be analytic throughout $C$ and its interior except at a
  number of poles inside $C$, then the point $u=f(z)$ will
  describe a closed curve $\gamma$ in the $u$-plane.
  Shew that if to each element of $\gamma$ be attributed
  a mass proportional to the corresponding element of $C$, the centre of
  gravity of $\gamma$ is the
  point $r$, where $r$ is the sum of the residues of
  $\frac{f(z)}{z-a}$ at its poles in the interior of $C$.
  \addexamplecitation{TODO Amigues, Noiiv. Ann. de Math. (.3), xii. (1893), pp. 142-148.}
  %
\item  Shew that
  $$
  \int_{-\infty}^{\infty}
  \frac{\dmeasure x}{ (x^{2}+b^{2}) (x^{2}+a^{2})^{2} }
  =
  \frac{\pi (2a+b)}{ 2a^{3}b(a+b)^{2}}.
  $$
\item
  Shew that
  $$
  \int_{0}^{\infty}
  \frac{\dmeasure x}{(a+bx^{2})^{n}}
  =
  \frac{\pi}{ 2^{n} b^{\half} }
  \frac{1 \cdot 3 \cdots (2n-3)}{1 \cdot 2 \cdots (n-1)}
  \frac{1}{ a^{n-\half} }.
  $$
%
% 123
%

\item
  If $F_{n}(z)= \prod_{m=1}^{n-1} \prod_{p=1}^{n-1} (1-z^{mp})$, shew that the series
  $$
  f(z)
  =
  - \sum_{n=2}^{\infty}
  \frac{F_{n}(zn^{-1})}{ (z^{n}n^{-n} -1) n^{n-1} }
  $$
  is an analytic function when $z$ is not a root of any of the equations
  $z^{n} = n^{n}$; and that the sum of the residues of $f(z)$ contained in the
  ring-shaped space included between two circles whose centres are at
  the origin, one having a small radius and the other having a radius
  between $n$ and $n + 1$, is equal to the number of prime numbers less than
  $n + 1$. \addexamplecitation{TODO Laurent, A' oiiv. Aim. de Math. (3), xviii. (1899), pp.
  234-241.}

\item
  If $A$ and $B$ represent on the Argand diagram two given roots (real
  or imaginary) of the equation $f(z) = 0$ of degree $n$, with real or
  imaginary coefficients, shew that there is at least one root of the
  equation $f'(z) = 0$ within a circle whose centre is the middle point
  of $AB$ and whose radius is $\half AB \cot \frac{\pi}{n}$.
  \addexamplecitation{TODO Grace, Proc. Camb. Phil. Soc. XI.}
\item
  Shew that, if $0<\nu<1$,
  $$
  \frac{e^{2\pi i \nu x}}{1 - e^{2\pi i x}}
  =
  \frac{1}{2\pi i}
  \lim_{n\rightarrow\infty}
  \sum_{k=-n}^{n} \frac{e^{2k\nu\pi i}}{k-x}.
  $$
  [Consider
  $
  \int \frac{e^{(2\nu-1) z \pi i}}{\sin \pi z}
  \frac{\dmeasure z}{z-x}
  $
  round a circle of radius $n+\half$; and make $n\rightarrow\infty$.]
  \addexamplecitation{TODO Kronecker, Journal fiir Math, cv.}
\item
  Shew that, if $m > 0$, then
  \begin{align*}
    \int_{0}^{\infty}
    \frac{\sin^{n} mt}{t^{n}} \dmeasure t
    \\
    =
    \frac{\pi m^{n-1}}{2^{n} (n-1)!}
    \theparen{
      n^{n-1}
      -
      \frac{n}{1} (n-2)^{n-1}
      +
      \frac{n(n-1)}{2} (n-4)^{n-1}
      -
      \frac{n(n-1)(n-2)}{3!} (n-6)^{n-1}
      +
      \cdots
    }.
  \end{align*}
  Discuss the discontinuity of the integral at $m = 0$.

\item
  If $A + B + C + \cdots = 0$ and $a, b, c, \ldots$ are positive, shew that
  $$
  \int_{0}^{\infty}
  \frac{A \cos ax + B \cos bx + \cdots + K \cos kx}{x}
  \dmeasure x
  =
  -A \log a
  -B \log b
  - \cdots
  -K \log k.
  $$
  \addexamplecitation{Wolstenholme.}
\item
  By considering
  $
  \int \frac{e^{x(k+ti)}}{k+ti} \dmeasure t
  $
  taken round a rectangle indented at the origin, shew
  that, if $k > 0$,
  $$
  i \lim_{\rho\rightarrow\infty}
  \int_{-\rho}^{\rho}
  \frac{e^{x(k+ti)}}{k+ti} \dmeasure t
  =
  \pi i
  +
  \lim_{\rho\rightarrow\infty}
  P \int_{-\rho}^{\rho} \frac{e^{xti}}{t} \dmeasure t,
  $$
  and hence deduce, by using the contour of
  \hardsubsubsectionref{6}{2}{2}{2} example
  TODO:citeEx2, or its reflexion in the real axis
  (according as $x \geq 0$ or $x < 0$), that
  $$
  \lim_{\rho\rightarrow\infty}
  \frac{1}{\pi}
  \int_{-\rho}^{\rho}
  \frac{e^{x(k+ti)}}{k+ti} \dmeasure t
  =
  2,\, , 1,\, \textrm{or } 0,
  $$
  according as $x>0$, $x = 0$, or $x < 0$.

  [This integral is known as Cauchy's discontinuous factor.]
%
\item
  Shew that, if $0 < a < 2, b > 0, r > 0$, then
  $$
  \int_{0}^{\infty}
  x^{a-1}
  \sin (\half a\pi - bx )
  \frac{r \dmeasure x}{x^{2} + r^{2}}
  =
  \half \pi r^{a-1} e^{-br}.
  $$
%
% 124
%

\item
  Let $t>0$ and let
  $
  \sum_{n=-\infty}^{\infty}
  e^{-n^{2} \pi t}
  =
  \psi(t).
  $

  By considering
  $
  \int \frac{e^{-z^{2} \pi t}}{e^{2\pi i z} - 1} \dmeasure z
  $
  round a rectangle whose corners are
  $\pm (N+\half) \pm i$, where $N$ is an integer,
  and making $N \rightarrow \infty$, shew that
  $$
  \psi(t)
  =
  \int_{-\infty - i}^{\infty - i}
  \frac{e^{-z^{2} \pi t}}{e^{2\pi i z} - 1} \dmeasure z
  -
  \int_{-\infty + i}^{\infty + i}
  \frac{e^{-z^{2} \pi t}}{e^{2\pi i z} - 1} \dmeasure z.
  $$

  By expanding these integrands in powers of
  $e^{-2\pi i z}, e^{2\pi i z}$
  respectively
  and integrating term-by-term, deduce from \hardsubsectionref{6}{2}{2} example TODO that
  $$
  \psi(t)
  =
  \frac{1}{ \sqrt{\pi t} }
  \psi(1/t)
  \int_{-\infty}^{\infty} e^{-x^{2}} \dmeasure x.
  $$

  Hence, by putting $t = 1$ shew that
  $$
  \psi(t) = t^{-\half} \psi(1/t).
  $$

  (This result is due to TODOPoisson, Journal de VEcole foly technique, xii.
  (cahier xix), (1823), p. 420; see also Jacobi, Journal fiir Math,
  xxxvi. (1848), p. 109 [Ges. Werke, ii. (1882), p. 188].)

\item
  Shew that, if $t>0$,
  $$
  \sum_{-\infty}^{\infty}
  e^{-n^{2} \pi t - 2 n \pi a t}
  =
  t^{-\half}
  e^{\pi a^{2} t}
  \thebrace{
    1
    +
    2
    \sum_{n=1}^{\infty}
    e^{-n^{2} \pi / t} \cos 2n\pi a
  }.
  $$
  \addexamplecitation{TODOPoisson, Mem. de I'Acad. des Sci. vi. (1827), p. 592; Jacobi,
  Journal fur Math. ill. (1828), pp. 403-404 [Ges. Werke, i. (1881), pp.
  264-265]; and Landsberg, Journal fur Math. CXI. (1893), pp. 234-253;
  see also \hardsubsectionref{21}{5}{1}.)}

\end{enumerate}


\subsection{Atomic formulae}

Atomic formulae are those without operators taking more than one input.

Literals, and negative literals, are types of atomic formula.

A literal is a formula with no operators.

\(\theta \)

These are also known as positive literals.

Negative literals are the negation of a literal.

\(\neg \theta \)

\subsection{Well-formed formulae}

A well-formed formula is one which can be given a truth value.

The following is not a well-formed formula:

\(\theta \land \)


In this section it is explained how to impose an external flux or atmospheric exchange of one or a few constituents into the model. This feature can be used to for example incorporate the flux of dissolved oxygen as a result of aeration in a water quality model or to model the influx of contaminant, organic matter and nutrients separate from the inflows. Several influx models are provided in GIFMod as described below:
\begin{itemize}
\item \textbf{Constant influx model: } In this model the flux is assumed to be constant:
\begin{equation}
\label{eq:28}
J_{ext} = k_{ext}
\end{equation}
So the external influx of a constituent will be calculated as:
\begin{equation}
\label{eq:29}
\dot{m} = A_s k_{ext}
\end{equation}

\item \textbf{Constant rate model: } In this model the flux is assumed to be proportional to the difference between the concentration of a constituent and a saturation concentration $C_s$:
\begin{equation}
\label{eq:30}
J_{ext} = k_{ext}(C_s-C)
\end{equation}
Which is applied volumetrically. So the external influx of a constituent will be calculated as:
 \begin{equation}
\label{eq:30a}
\dot{m} = k_{ext}(C_s-C) \sout{V}
\end{equation}

\item \textbf{Free surface model: } The free surface model is used for open-channel reaeration where the flow depth and velocity affect the rate of aeration. The for of the flux equation is consistent with the forms suggested by \citep{oconnor1956,churchill1964,owens1964some,isaacs1968,langbein1967}:
\begin{equation}
\label{eq:31}
J_{ext} = k_{ext}v^{\alpha_{ext}}H^{\beta_{ext}}(C_s-C)
\end{equation}

\item \textbf{Soil model: } Aeration in soils is expected to be limited by the area of air-water interface. In this model it is assumed that the area of the air-water interface is proportional to the moisture content and air content of the soil. The external flux equation for soils is therefore considered as:

\begin{equation}
\label{eq:32}
J_{ext} = k_{ext}\theta (\theta_s-\theta) (C_s-C)
\end{equation}
Which is applied volumetrically. So the external influx of a constituent will be calculated as:
 \begin{equation}
\label{eq:33}
\dot{m} = k_{ext}\theta (\theta_s-\theta) (C_s-C) \sout{V}
\end{equation}
\end{itemize}
\subsubsection{External Flux Properties: }
An external flux model must be added by right-clicking \textbf{Project Explorer}$\rightarrow$\textbf{Water Quality}$\rightarrow$\textbf{External Fluxes} and clicking on \textbf{Add External Flux} and then be assigned to individual blocks through the \textbf{External Flux} property of the block.

Below properties that can be assigned to a \textbf{External flux} model are described:

\begin{itemize}
\item \textbf{Name: } This is the name of the external flux model. The name is used to assign the model to individual blocks.
\item \textbf{Coefficient: } Here the value of $k_{ext}$ in Eqs. (\ref{eq:28}, \ref{eq:30}, \ref{eq:31}, \ref{eq:32}) can be entered. This is needed for all external flux models.
\item \textbf{Constituent: } This indicate the constituent that will be released as a result of the external flux.
\item \textbf{Depth exponent: } The value of $\beta_{ext}$ in the free surface model.
\item \textbf{Model: } Here the type of the model to be used for the external flux will be selected.
\item \textbf{Saturation: } The value of saturation concentration $C_s$ used in the \textbf{constant rate}, \textbf{free surface}, \textbf{soil} will be entered here.
\textbf{Solid: } Indicates the solid phase (soil or particles) that the external flux will be assigned to. If left empty the influx occur as dissolved.
\item \textbf{Transfer Rate Expression: } This property allows the user to type in an expression to be used for the calculation of the external flux rates.
\item \textbf{Velocity Exponent: } The value of $\alpha_{ext}$ in free surface model (Eq. \ref{eq:31}).
\end{itemize}

\input{Aeration_ex.tex}


\section{Extensions to the Problem}
For the bulk of this thesis, we have looked at many-to-one matching problems with one-sided preferences, as this setting makes the most sense for the student-seminar application. However, there is a wide range of similar problems and extensions that are also worth mentioning. This section will present some of those problems and key results from the literature.

\subsection{Two-Sided Preferences}\label{extensions:two-sided}
In many practical one-to-many matching schemes, both parties will express preferences over the other. For instance, many countries use such schemes for matching students to university programs, which can be viewed as an instance of a one-to-many matching problem with two-sided preferences. A similar problem exists in the United States, where graduating medical students need to be matched to residency positions in hospitals.

Since we are considering two-sided preferences now, it is not possible to simply transform this problem into a weighted bipartite graph anymore, since edges now have two weights. It could be possible to use the product of those two weights as an edge weight; however, in practice different mechanisms are being used that do not rely on the graph representation of the problem.

\subsubsection{Stable Matchings}
In the context of two-sided preferences, \emph{Stability} is a commonly used optimality criteria that matchings should fulfill. It can be seen as the two-sided preference extension to Pareto-optimality and is defined as follows. Given a set $H$ of hospitals and a set $R$ of residents, a matching $M$ of an instance $I$ is stable iff there is no pair $(r_i, h_j) \in M, r_i \in R, h_j \in H$ with:
\begin{itemize}
    \item $r_i$ and $h_j$ are on each other's preference lists
    \item either $r_i$ is assigned in $M$, or $r_i$ prefers $h_j$ to $M(r_i)$
    \item either $h_j$ is undersubscribed in $M$, or $h_j$ prefers $r_i$ to it is least preferred assigned resident in $M(h_j)$
\end{itemize}
Simply put, in a stable matching $M$, no pair of agents $(r_i, h_j)$ should have an incentive to give up their current match to get matched to each other.

\subsubsection{Algorithm}
Gale and Shapley \cite{GaleShapleyOrig} proved that for every instance of the problem, there exists a stable matching that can be computed using a linear-time algorithm. There are two variants of the algorithm, one that computes a resident-optimal, hospital pessimal matching and one that does the opposite. That means that in the first case, each resident is matched to the best hospital among all stable matchings, while each hospital is matched to the worst set of residents among all stable matchings \cite{Gusfield}. In the context of student seminar matchings, it would make sense to use the resident/student optimal algorithm, which goes as follows \cite{Gusfield}:
\begin{algorithm} % enter the algorithm environment
    \caption{Resident-oriented Deferred Acceptance Algorithm}
    \label{alg:resident-oriented-algorithm}
    \begin{algorithmic} % enter the algorithmic environment
        \Require set of residents with preferences $R$, set of hospitals $H$
        \Ensure Stable Matching M
        \While{Some resident $r$ is free and has a non-empty preference list}
        \State $h:=$ first hospital on $r$'s preference list
        \If{$h$ is fully subscribed}
            \State $r':=$ worst resident provisionally assigned to h
            \State unassign $r'$ from $h$
        \EndIf
        \State provisionally assign $r$ to $h$

        \If{$h$ is fully subscribed}
            $s:=$ worst resident provisionally assigned to $h$
            \ForEach{successor $s'$ of $s$ on $h$'s list}
                \State remove $s'$ and $h'$ from each other's list
            \EndFor
        \EndIf

        \EndWhile
    \end{algorithmic}
\end{algorithm}

An advantage of using this algorithm for matching students to seminars is that the order in which students are processed does not matter. Gusfield shows that for any permutation of the input, each hospital is assigned the same number of residents, the same set of residents is unassigned and each undersubscribed hospital is matched to the same set of residents \cite{Gusfield}.

\subsection{Many-to-Many Matchings}
A natural extension of the many-to-one matching setting is the many-to-many matching problem. It could very well be a requirement that students need to be matched to more than one seminar. In that case, we need to differentiate between one- and two-sided preferences again:

\subsubsection{One-sided Preferences}
Many-to-many matching scenarios with one-sided preferences are also commonly referred to as the \emph{Course allocation} problem, which is a combinatorial assignment problem with the goal of assigning students to courses, based on the students' preferences \cite{CourseAllocation}. Compared to the student and seminar scenario we now allow students to get matched to more than one course.

A common mechanism for solving course allocation problems is using an \emph{Auction model} or \emph{Bidding points mechanism} in which students are given a set amount of artificial currency, which they then use to bid on seats in classes. After all bids are submitted, the system assigns the seats to the highest bidders. The bids can be used to infer students' preferences; however, the true preferences may differ significantly, which makes this mechanism less desirable \cite{Bidding}.

Another commonly used mechanism is the \emph{Draft}, in which students are asked to pick a course with remaining seats based on a draft order. Intuitively, this mechanism is an extension of RSD (Section \ref{algo-rsd}), where the whole process is repeated until no student makes a pick anymore. Such draft mechanisms are commonly used in course allocation settings; one example being Harvard Business School \cite{CourseAllocation}. However, it has been shown that students have successfully manipulated the draft due to the fact that this mechanism is not strategy-proof.

A mechanism that is more robust towards strategic manipulation is a \emph{proxy bidding mechanism} presented by Kominers et al. \cite{CourseAllocation}. The mechanism uses proxies that act on behalf of the students by using their true preferences in the draft.

\subsubsection{Two-sided Preferences}
For the case of two-sided preferences, we have to slightly modify the definition of Stability again to accommodate for the fact that each entity can be assigned more than once: Given the set of hospitals $H$ and residents $R$, let $h \in H$ and $r \in R$, so that $h$ and $r$ are acceptable to each other, unmatched and the following holds true:
\begin{itemize}
    \item either $h$ has unfilled places or prefers $r$ to one of his matched partners \textbf{and}
    \item either $r$ has unfilled places or prefers $h$ to one of his matched partners
\end{itemize}
Using this definition of stability, Gusfield \cite{Gusfield} proposes that an algorithm can be constructed using ideas from the hospital-oriented and resident-oriented (see Algorithm \ref{alg:resident-oriented-algorithm}) algorithm that finds a stable matching that's optimal for either of the sets. Again, such a stable matching exists for any instance of the problem \cite{Gusfield}.

\subsection{Online Variants}\label{sec:online-variants}
When solving the \emph{online-variant} of the problem, the whole input is not available from the start. That means that the input needs to be processed piece by piece, or more formally: Given a bipartite weighted graph $(U, V, E)$, where $U$ is known to the algorithm, vertices in $V$ are unknown, but arrive one at a time, while also revealing their incident edges, find a matching that maximizes some objective function. These algorithms could be of interest in the case of a first-come first-serve course allocation system, or in other areas such as DVD-rental or online-advertisement allocation systems \cite{Mehta:Online}.

In the case of student-seminar assignments, we would assume that the set of courses and their capacities is known beforehand, and the students arrive later. One of the algorithms we have seen in Section \ref{chapter:algorithms} can be used for this problem, namely the RSD-algorithm. As a matter of fact, the algorithm will produce the same results for the offline and online case, given that the order in which students are processed is identical.

\subsubsection{Online Maximum Cardinality Matching}
There has been lots of research in particular on finding maximum-cardinality matchings with online inputs. The online-inputs are classified by how much information the algorithm possesses about the input order. For now, we will only consider the \emph{adversarial order}, where we assume no knowledge of the query sequence, which means that only $U$ is known at the beginning of the algorithm, while we have no knowledge of $V$ and $E$ or the order they appear in \cite{Mehta:Online}. To measure performance, we will use the \emph{competitive ratio} of an algorithm which is defined as follows. Given an instance of the problem $I$, the value of the objective function for the online algorithm is given as $ALG(I)$, and the value of the objective function for the best offline algorithm is given as $OPT(I)$. The competitive ratio is now computed as follows: $C.R.=\frac{ALG(I)}{OPT(I)}$ \cite{Mehta:Online}.

A simple algorithm for this online problem is a greedy algorithm, which matches arriving vertices to any available neighbor, or a random approach that matches arriving vertices to a random neighbor. These mechanisms achieve a competitive ratio of $\frac{1}{2}$ \cite{Mehta:Online}. An optimal, yet simple algorithm was introduced by Karp et al. \cite{Karp:Online}, which achieves a competitive ratio of $1 - \frac{1}{e} \simeq 0.63$. The algorithm, called Ranking, begins by permuting the known vertices of $U$ in a random permutation $\pi$, i.e. we assign a random priority number to each $u \in U$. Each incoming vertex $v \in V$ is then assigned to an available neighbor, with the smallest value of $\pi(u)$. In detail, the algorithm looks like this:

\begin{algorithm} % enter the algorithm environment
    \caption{Ranking}
    \label{alg:ranking} % and a label for \ref{} commands later in the document
    \begin{algorithmic} % enter the algorithmic environment
        \State \textbf{Offline:} Pick a random, uniform permutation $\pi$ of U
        \ForEach {arriving vertex $v \in V $}
            \If{$v$ has no available neighbors}
                \State continue
            \EndIf
            \State Match $v$ to the neighbor $u \in U$ with the smallest value $\pi(u)$
        \EndFor
    \end{algorithmic}
\end{algorithm}


\documentclass{article}
\usepackage[margin=2.5cm, includefoot, footskip=30pt]{geometry}
\pagestyle{plain}
\setlength{\parindent}{0em}
\setlength{\parskip}{1em}
\renewcommand{\baselinestretch}{1}

\usepackage{amsmath}
\usepackage{amsfonts}

\newcommand{\R}{\mathbb{R}}

\title{The effect of homophily in hierarchical systems}

\renewcommand{\S}{\mathcal{S}}

\begin{document}

\maketitle

\section{Mathematical formulation of base model}

\begin{itemize}
    \item Given a hierarchical system with \(K\) levels.
    \item Level \(0\leq k < K\) has capacity \(C_k\)
    \item The first level (\(k=0\)) has the most capacity and capacity is
        monotonically decreasing: \(C_0 \geq C_1 \geq \dots \geq C_{k-2} \geq C_{k-1} = 1\).
    \item There are 2 types of agents: \(j\in\{0, 1\}\).
\end{itemize}

Consider a state space \(\S\):

\begin{equation}\label{eqn:state_space}
    \S = \left\{s \in \mathbb{Z} ^ {K \times 2} _ {\geq 0}\left|
        \begin{array}{l}
        s_{i0} + s_{i1} \leq C_i\text{ for all }0\leq i \leq K - 1\\
        s_{K-1} = (1, 0)\\
        \sum_{i=0}^{K - 1}s_{i0} + s_{i1} \in\left\{\sum_{i=0}^{K - 1}C_i, \sum_{i=0}^{K - 1}C_i - 1\right\}\\

        \end{array}
                \right.\right\}
\end{equation}
%TODO comment on top row
Where \(s_{ij}\) denotes the number of individuals of type \(j\) at level \(i\).

For example,

\begin{itemize}
    \item Let \(K = 3\)
    \item Let \(C = (4, 3, 1)\)
\end{itemize}

Then:

\[
    s = \begin{pmatrix}
        3 & 1 \\
        2 & 1 \\
        1 & 0 \\
    \end{pmatrix}
\]

corresponds to a system with 3 agents of first type and 1 of second type at the
first level, 2 of first type and 1 of second type at the second level and 1 of
each type at the 3rd level.

The constraints on \(\S\) ensure that either all positions are filled or a
single position is available. Thus at any stage either all spots are full and
someone will retire or there will be a spot available and someone will be
hired/promoted.

The size of the state space is then given by:

\begin{equation}
    |S| = \prod_{i=0}^{K - 2}\left(2C_i + 1\right)
\end{equation}

Given two elements \(s^{(1)}, s^{(2)}\in \S\) the transition rates are given by:

\begin{equation}\label{eqn:transition_rates}
    Q_{s_1, s_2} =
        \begin{cases}
            \mu_{ij},& \text{ if }s^{(2)} - s^{(1)} = -e_{ij}
            \text{ and }s^{(1)}_{i0} + s^{(1)}_{i1} = C_i\text{ for all }i\\
           \text{max}(rs_{i + 1,j} + s_{i + 1, \bar j}, 1),& \text{ if }s^{(2)} - s^{(1)} = e_{ij} - e_{i-1, j}
                                     \text{ and }s^{(1)}_{i0} + s^{(1)}_{i1} < C_{i}
                                     \text{ and }i > 0\\
           \lambda_{j},&\text{ if }s^{(2)} - s^{(1)} = e_{0j}
                        \text{ and }s^{(1)}_{00} + s^{(1)}_{01} = C_{0} - 1\\
        \end{cases}
\end{equation}

Where:

\begin{itemize}
    \item \(\mu_{ij}\) is the retirement rate of agents of type \(j\) at level
        \(i\).
    \item \(r > 1\) is a constant that reflects the homophily effect.
    \item \(\lambda_j\) is the hiring rate of individuals of type \(j\).
\end{itemize}

% TODO Add some examples.

\section{Mathematical formulation of model with competence}

The base model described in the previous section does not take into account the
competence of the individuals. Let's consider the above example of:

\begin{itemize}
    \item \(K = 3\) and \(C = (4, 3, 1)\)
\end{itemize}

A possible state \(s\) for the above configuration when competence is taken into
account is:

\[
    s = \begin{pmatrix}
        (1, 0.1) & (1, 0.2) & (1, 0.3) & (0, 0.5) \\
        (1, 0.2) & (1, 0.3) & (0, 0.8) \\
        (1, 0.7) \\
    \end{pmatrix}
\]

Note that this is a possible state for the configurations of \(K\) and \(C\) but
not the only one. The competence of each individual can be any real number between
\([0, 1]\), thus there are an infinite number of possible states.

The set of possible states, denoted as \(S\), is given by,

\begin{equation}\label{eqn:state_space}
    \S = \left\{
        s \in (\{0, 1\}, \R^{[0,1]}) ^ {K \times C}  \left|
        \begin{array}{l}
         s_{i,j} = (l_{i, j}, c_{i, j}) \quad \forall i \in [0, K -1] \quad \forall j \in [C_i - 1, C_i] \\
        \text{ where } l_{i, j} \in \{0, 1\} \text{ and } c_{i, j} \in \R^{[0,1]} \\
        |s_{k - 1}| = 1  \\
        l_{k -1, 1} = 0
    \end{array}
    \right.\right\}
\end{equation}

where \(s_{i, j}\) is a tuple of \((l_{i, j}, c_{i, j})\) where \(l_{i, j}\) is the
type of individual \(j\) at level \(i\) and \(c_{i, j}\) is the individual's
competence.

Promotion in the competence model assumes that individuals of the same type give a random
bonus to the competence of individuals like them.
Thus for a state \(s\) where a promotion is happening at level \(i\), the promotion
probability of an individual \(j\) at level \(i - 1\) is given by:

\begin{equation}
    P(i, j) =
    \frac{
    \displaystyle \sum_{t = 1} ^ {C_{i -1} - 1} \text{max}((1 - |l_{i-i, t} - l_{i, j}|) \gamma_t, 1) \ c_{i, j}}
    {
    \displaystyle \sum_{\bar{j}=1}^{C_i} \sum_{t = 1} ^ {C_{i -1} - 1} \text{max}((1 - |l_{i-i, t} - l_{i, j}|) \gamma_t, 1) \ c_{i, \bar{j}}
    },
\end{equation}

where \(\gamma_{t} > 1\) is randomly sampled at each \(t\) from \([0, \Gamma]\).

Thus, the transition rates between two states are given by:

\begin{equation}\label{eqn:transition_rates}
        \begin{cases}
            \mu_{ij},& |s_i| = C_i\text{ for all }i\\
            P(i, j), & |s_i| < C_{i} \text{ and }i > 0\\
           \lambda_{(l, c)},& |s_{0}| = C_0 - 1\\
        \end{cases}
\end{equation}

Where:

\begin{itemize}
    \item \(\mu_{ij}\) is the retirement rate of the agent \(j\) at level
        \(i\).
    \item \(\lambda_{(l, c)}\) is the hiring rate of an individual \(l\) that has
    a competence \(c\).
\end{itemize}

The competence of the system is the total competence of the individuals in the
system regarding their type.

\begin{equation}\label{eqn:state_space}
    \sum_{i = 0} ^ {K - 1} \sum_{j=1} ^ {C_i} c_{i, j}
\end{equation}

\end{document}


\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{txfonts}

%- Abbreviations.
\newcommand{\psf}{\mathbf{H}}
\newcommand{\G}{\mathbf{G}}
\newcommand{\p}{\mathbf{p}}
\newcommand{\phat}{\mathbf{\hat p}}
\newcommand{\hhat}{\mathbf{\hat h}}
\newcommand{\x}{\mathbf{x}}
\newcommand{\xhat}{\mathbf{\hat x}}
\newcommand{\y}{\mathbf{y}}
\newcommand{\rvect}{\mathbf{r}}
\newcommand{\W}{\mathbf{W}}
\newcommand{\A}{\mathbf{A}}
\newcommand{\avec}{\mathbf{a}}
\newcommand{\F}{\mathbf{F}}
\newcommand{\V}{\mathbf{V}}
\newcommand{\diag}{\mathrm{diag}}

\newcommand{\ddx}{\frac{\partial}{\partial \mathbf{x}}}
\newcommand{\ddxi}{\frac{\partial}{\partial x_i}}

\newcommand{\Nsn}{\textbf{XXX}}
\newcommand{\Nspec}{\textbf{YYY}}

\newcommand{\rem}[1]{\textbf{[\textsl{#1}]}}
\newcommand{\szf}[1]{\textbf{\textsl{SZF: #1}}}

% -----------------------------------------------------------------------------
% Document

\begin{document}

\title{CubeFit $\chi^2$ gradient calculation}
\author{S.~Bongard, Kyle~Barbary, Clare~Saunders}
\maketitle
\begin{abstract}
In CubeFit, one of the fitting steps is to fit the galaxy model to the
data while holding other parameters (such as positions) fixed. The
galaxy model has on the order of one million parameters. The ability to
analytically calculate the gradient (a vector of length $\sim$ 1
million) on the fit $\chi^2$ is essential in order to carry out the fit
efficiently. This note explains the derivation of the gradient function.
\end{abstract}

\section{The Gradient}

The $\chi^2$ in the galaxy fit is given by

\begin{equation}
\chi^2 = (\y - \G\x)^T \W (\y - \G\x)
\end{equation}

\noindent where $\y$ is the data, $\x$ is the model parameters, $\W$
is the weight matrix (inverse covariance matrix; diagonal in our case)
and $\G$ is the operator that translates the model into the data
frame. $\G$ includes the effect of PSF convolution, shift operation
and cropping. Note that we are supposing that we can represent all
these operations as a matrix $\G$, but we will not show this until a
later section. Remember also that this is all in real space.  Carrying
on,

\begin{eqnarray*}
  \label{eq:5}
  \ddx \chi^{2} & = & \ddx \left( (\y - \G\x)^{T} \W (\y - \G\x) \right)\\
                & = & \ddx \left( (\G \x)^{T} \W \G \x \right)
                    - \ddx \left( (\G \x)^{T} \W \y \right)
                    - \ddx \left( \y^{T} \W \G \x \right) \\
                & = & \ddx \left( \x^{T} \G^{T} \W \G \x \right)
                    - \ddx \left( \x^{T} \G^{T} \W \y \right)
                    - \ddx \left( (\G^{T} \W \y)^{T} \x \right)
\end{eqnarray*}

\noindent where in the second line we have simply separated terms, dropping the
constant term $\y^T \W \y$, and in the third line we have used the property
that $(\mathbf{A}\mathbf{B})^T = \mathbf{B}^T\mathbf{A}^T$ and assumed that
$\W = \W^T$ (the weight matrix is symmetric).

We note that $(\G^{T} \W \y)^{T}$ and $\G^{T} \W \y$ are vectors ($\y$ is a
vector). The first two terms of the equation are thus of the form
$\ddx \x^T \avec$ and $\ddx \avec^T \x$. These are both simply equal to
$\avec$ as $\x^T \avec$ and $\avec^T \x$ are dot products between the
constant vector $\avec$ and $\x$, so the partial derivative with respect to
$\x$ is just $\avec$. Making this substitution, we have

\begin{equation}
\ddx \chi^{2} = \ddx \left( \x^{T} \G^{T} \W \G \x \right) - 2 \G^T \W \y
\end{equation}

\noindent The first term has the form $\ddx \x^T\A\x$, with
$\A = \G^T \W \G$. It helps to write this out:

\begin{eqnarray*}
  \ddxi \x^{T}\A\x & = & \ddxi \left( \sum_{j} x_{j} \sum_l A_{j,l} x_{l}  \right) \\
                   & = & \sum_l A_{i,l} x_{l} + \sum_j A_{j,i} x_{j} \\
                   & = & \left( \A \x + \A^{T} \x␇ \right)_{i}
\end{eqnarray*}

\noindent or without the $i$ subscripts,

\begin{equation}
  \ddx \x^T\A\x = (\A + \A^T) \x
\end{equation}

\noindent Substituting this in (with $\A = \G^T \W \G$), we finally get:

\begin{eqnarray*}
\ddx \chi^{2} & = & \left( \G^T \W \G \x + (\G^T \W \G)^T \x \right)
                 - 2 \G^T \W \y \\
             & = & 2 \G^T \W \G \x - 2 \G^T \W \y \\
             & = & -2 \G^T \W ( \y - \G \x ) \\
             & = & -2 \G^T \W \rvect \\
\end{eqnarray*}

\noindent where $\rvect$ is shorthand for the residual
$\y - \G \x$. But what is the matrix $\G$ and what is $\G^T$?
For that we need to digress about
Fourier transforms as matricies (in the next section).

\section{Discrete Fourier Transform: Matrix Form}

The discrete Fourier transform of a vector $x_n$ is

\begin{equation}
\hat{x}_k = \sum_{n=0}^{N-1} x_n e^{-i 2 \pi k n / N}
\end{equation}

\noindent We can write this in matrix form,

\begin{equation}
\xhat = \left( \begin{array}{cccc}
   e^{-i 2\pi 0 \cdot 0 / N} & e^{-i 2\pi 0\cdot 1 / N} & e^{-i 2\pi 0 \cdot 2 / N} & \cdots \\
   e^{-i 2\pi 1 \cdot 0 / N} & e^{-i 2\pi 1 \cdot1 / N} & e^{-i 2\pi 1 \cdot 2 / N} & \cdots \\
   e^{-i 2\pi 2 \cdot 0 / N} & e^{-i 2\pi 2 \cdot 1 / N} & e^{-i 2\pi 2 \cdot 2 / N} & \cdots \\
   \vdots & \vdots & \vdots & \ddots
   \end{array} \right)
   \left( \begin{array}{c} x_0 \\ x_1 \\ x_2 \\ \vdots \end{array} \right)
\end{equation}

\noindent For a shorthand, we will call the square matrix $\mathbf{F}$

\begin{equation}
\xhat = \F \x
\end{equation}

Similarly, the inverse transform can be written

\begin{equation}
x_n = \frac{1}{N} \sum_{k=0}^{N-1} \hat{x}_k e^{i 2 \pi k n / N}
\end{equation}

\noindent or

\begin{equation}
   \mathbf{x} = \frac{1}{N} \left( \begin{array}{cccc}
   e^{i 2\pi 0 \cdot 0 / N} & e^{i 2\pi 1 \cdot 0 / N} & e^{i 2\pi 2 \cdot 0 / N} & \cdots \\
   e^{i 2\pi 0 \cdot 1 / N} & e^{i 2\pi 1 \cdot 1 / N} & e^{i 2\pi 2 \cdot 1 / N} & \cdots \\
   e^{i 2\pi 0 \cdot 2 / N} & e^{i 2\pi 1 \cdot 2 / N} & e^{i 2\pi 2 \cdot 2 / N} & \cdots \\
   \vdots & \vdots & \vdots & \ddots
   \end{array} \right)
   \left( \begin{array}{c} \hat{x}_0 \\ \hat{x}_1 \\ \hat{x}_2 \\ \vdots \end{array} \right)
\end{equation}

\noindent or as shorthand,

\begin{equation}
\x = \F^{-1} \xhat
\end{equation}

\noindent Now, note that the matrix $\F$ has the property

\begin{equation}
\F^{-1} = \frac{1}{N} \bar{\F}
\end{equation}

\noindent where $\bar{\F}$ is the element-wise complex conjugate of $\F$.

\section{The $\G$ matrix }

The $\G$ matrix represents the operations applied to the model to put
it "in the data frame". These are (1) convolve model by PSF (2) shift
model to align with data (3) crop model indicies that extend past the
data (model is spatially larger than data). Convolution in real space
is equivalent to element-wise multiplication (the Hadamard product) in
Fourier space (and this is indeed how the convolution is implemented
in CubeFit). While element-wise multiplication can be written
$\mathbf{a} \circ \mathbf{b}$, we will find it useful to write it as
$\diag(\mathbf{a}) \mathbf{b}$, where diag() is an operator
that translates a vector into a square diagonal matrix with the same
entries.

Convolution of the galaxy model $\x$ by a real-space PSF $\p$ can thus be
written as
\begin{equation}
\F^{-1} \diag(\F \p) \F \x
\end{equation}

Similarly, spatial shifts are implemented as element-wise multiplication in
Fourier space. the key point being that we can
subsume this shift vector (or rather the Fourier transform of it) into
$\F \p = \phat$. Therefore, we can write $\G$ as

\begin{equation}
\G = \F^{-1} \diag(\phat) \F
\end{equation}

We are now interested in calculating $\G^T$ as it appears in the
equation for $\ddx \chi^{2}$. Simply taking the transpose, we get
$\G^{T} = \F^T \diag(\phat)^{T} \F^{-1T}$, but we would like
to rewrite this in a form that has the Fourier transforms in the same
order as in $\G$. Since $\G$ is a real operator, we have $\G =
\bar{\G}$, and thus $\G^T = \bar{\G}^T$. We will use this along with
the relation between the forward and reverse Fourier transform derived
above ($\F^{-1} = \frac{1}{N} \bar{F}$), and the fact that the
transpose of a diagonal matrix is the matrix itself. We find:

\begin{eqnarray*}
  \G^{T} & = & \F^{T} \diag(\phat)^{T} \F^{-1T}\\
        & = & \overline{\F^{T} \diag(\phat)^{T} \F^{-1T}} \\
        & = & \bar{\F}^T \overline{\diag(\phat)^T} \bar{\F}^{-1T} \\
        & = & N\F^{-1} \overline{\diag(\phat)} \frac{1}{N}\F \\
        & = & \F^{-1} \overline{\diag(\phat)} \F\\
\end{eqnarray*}

\noindent If we now write the residual $\rvect = \y - \G\x$, we
finally find that the derivative of $\chi^{2}$ is:

\begin{eqnarray*}
 \ddx \chi^2 & = & -2 \G^T \W \rvect \\
             & = & -2 \F^{-1} \overline{\diag(\phat)} \F \W \rvect
\end{eqnarray*}

\noindent In CubeFit, the operator $\G^T = \F^{-1}
\overline{\diag(\phat)} \F$ is currently implemented by the method
\verb|AtmModel.gradient_helper()|.

\section{Fitting galaxy and sky simultaneously}

In another step in CubeFit, we fit the galaxy model to multiple epochs
of data simultaneously, while adjusting the sky for each epoch on each
iteration.  On each iteration, the sky is calculated by taking the
weighted average of the residual ($\y - \G \x$) at each wavelength and
subtracting that to form the new residual. We need to take this into
account in the $\chi^2$ gradient: qualitatively, increasing any given
model pixel will result in a lower sky estimate, and the sky estimate
feeds back into the $\chi^2$. The actual $\chi^2$ we're using involves
the residual $\y - \G\x - \mathrm{avg}(\y - \G\x)$. where avg() is a
weighted average over wavelength. Fortunately, this weighted average
can be represented by a matrix multiply. For example,

\begin{equation}
  \left( \begin{array}{cccc}
   \frac{w_0}{w_0 + w_1} & \frac{w_1}{w_0 + w_1} & 0 & 0 \\
   \frac{w_0}{w_0 + w_1} & \frac{w_1}{w_0 + w_1} & 0 & 0 \\
   0   & 0   & \frac{w_2}{w_2 + w_3} & \frac{w_3}{w_2 + w_3} \\
   0   & 0   & \frac{w_2}{w_2 + w_3} & \frac{w_3}{w_2 + w_3} \\
   \end{array} \right)
   \left( \begin{array}{c} x_0 \\ x_1 \\ x_2 \\ x_3 \end{array} \right)
\end{equation}

\noindent yields a vector whose first two entries are the weighted
average across the first two entries of $\x$ and whose last two
entries are the weighted average across the last two entries of
$\x$. Call this matrix $\V$. The weighted average of the residual is then
$\V (\y - \G\x)$ and the $\chi^2$ for each epoch is

\begin{eqnarray*}
\chi^2 &=& (\y - \G\x - \V(\y - \G\x))^T \W (\y - \G\x - \V(\y - \G\x)) \\
       &=& ((I - \V)(\y - \G\x))^T \W ((I - \V)(\y - \G\x)) \\
       &=& (\y - \G\x)^T (I - \V)^T \W (I - \V) (\y - \G\x)
\end{eqnarray*}

\noindent where in the third line we have used the equality
$(\mathbf{A}\mathbf{B})^T = \mathbf{B}^T \mathbf{A}^T$. This is the
same as the original $\chi^2$ equation, but with $\W$ replaced by $(I
- \V)^T \W (I - \V)$. This is a constant, so our original result for the
$\chi^2$ gradient holds under this replacement:

\begin{eqnarray*}
\W \rvect &\to& (I - \V)^T \W (I - \V) \rvect \\
          &\to& (I - \V)^T \W \mathbf{r'} \\
          &\to& \W \mathbf{r'} - \V^T \W \mathbf{r'}
\end{eqnarray*}

\noindent where $\mathbf{r'} = (I - \V)(\y - \G\x)$ is the residual after
subtracting the ``sky.''

Practically, how do we calculate $\V^T \W \mathbf{r'}$? In other words, if
multiplication by $\V$ is equivalent to a weighted average by wavelength,
what is $\V^T$? In our minimal example from above, it would be

\begin{equation}
  \V^T \x
  = \left( \begin{array}{cccc}
    \frac{w_0}{w_0 + w_1} & \frac{w_0}{w_0 + w_1} & 0 & 0 \\
    \frac{w_1}{w_0 + w_1} & \frac{w_1}{w_0 + w_1} & 0 & 0 \\
    0   & 0   & \frac{w_2}{w_2 + w_3} & \frac{w_2}{w_2 + w_3} \\
    0   & 0   & \frac{w_3}{w_2 + w_3} & \frac{w_3}{w_2 + w_3} \\
    \end{array} \right)
    \left( \begin{array}{c} x_0 \\ x_1 \\ x_2 \\ x_3 \end{array} \right)
  = \left( \begin{array}{c}
    \frac{w_0}{w_0 + w_1} (x_0 + x_1) \\
    \frac{w_1}{w_0 + w_1} (x_0 + x_1) \\
    \frac{w_2}{w_2 + w_3} (x_2 + x_3) \\
    \frac{w_3}{w_2 + w_3} (x_2 + x_3)
    \end{array} \right)
\end{equation}

\noindent Looking at the result, we can see this is equivalent to, for
each wavelength, taking the ratio of the sum of the input vector and
the sum of $\W$ and multiplying the result by $\W$. In Python (where
we are representing $\W$ and $\x$ with 3-d arrays), $\V^T \W \mathbf{r'}$
would be

\begin{verbatim}
tmp = np.sum(w * rprime, axis=(1, 2)) / np.sum(w, axis=(1, 2))
vtwrprime = w * tmp[:, None, None]
\end{verbatim}

\noindent where $\W$ is \verb|w| and $\mathbf{r'}$ is \verb|rprime|, and the
full replacement for $\W \rvect$ would be:

\begin{verbatim}
w * rprime - vtwrprime
\end{verbatim}

\section{Fitting SN and Data Positions}

In the final step of CubeFit, we fit simultaneously the position of
the data with respect to the galaxy and the position of the SN with
respect to the galaxy. Here, we calculate the analytic gradient of the
$\chi^2$ function used in this fit.

\begin{eqnarray*}
\chi^2 &=& \sum_{t, \lambda} \sum_i w_i (d_i - (scene)_i)^2 \\
       &=& \sum_{t, \lambda} \sum_i w_i (d_i - (sky + g_i + SN s_i))^2
\end{eqnarray*}

\noindent where the first sum is over epochs and wavelengths and the
second sum is over pixels at a give wavelength. $g_i$ and $s_i$ depend
on the positions that we are optimizing, as do $sky$ and $SN$, which
are determined based on the galaxy model, $\vec{g}$ and SN model,
$\vec{s}$.  The derivative of $\chi^2$ with respect to the position
$x_j$ of the data is

\begin{eqnarray*}
\frac{d\chi^2}{dx_\ast} &=& \sum_{t, \lambda} \sum_i -2 w_i (d_i - (scene)_i) \frac{d(scene)_i}{dx_\ast} \\
                     &=& \sum_{t, \lambda} \sum_i -2 w_i (d_i - (scene)_i) \frac{d}{dx_\ast} (sky + g_i + SN s_i) \\
                     &=&  \sum_{t, \lambda} \sum_i -2 w_i (d_i - (scene)_i) \left(\frac{d(sky)}{dx_\ast} + \frac{dg_i}{dx_\ast} + \frac{d(SN)}{dx_\ast} s_i + SN \frac{ds_i}{dx_\ast}\right)
\end{eqnarray*}

\noindent If $x_\ast$ is an SN position parameter then $dg_i/dx_\ast = 0$,
but otherwise all terms above are nonzero: $sky$ and $SN$
depend on $x_\ast$ through their dependence on $\vec{g}$ and
$\vec{s}$. The derivatives $dg_i/dx_\ast$ and $ds_i/dx_\ast$ are fairly
simple to compute. $\vec{g}$ and $\vec{s}$ are computed via
multiplication by a phase shift array in Fourier space. The phase
shift array is the only part that depends on position and everything
else is just multiplication. So all we need to do is to subsitute the
derivative on the phase shift array, then carry out all calculations
as we normally would.

We must also determine the gradient in $sky$ and $SN$ with respect to
position.  We start by defining some quantities for
convenience (in the left column), and taking their derivatives with respect to
data and SN positions. (Note that $s_i$ is affected by both positions, $g_i$ is only affected by the data position.)

\begin{displaymath}
%denom = AC - B^2
\begin{array}{ccc}
A = \sum_i w_i s_i^2    & \frac{dA}{dx_\ast} = \sum_i 2 w_i s_i \frac{ds_i}{dx_\ast}   &   (x_\ast \in \{x_{data}, x_{sn}\}) \\
B = \sum_i w_i s_i    & \frac{dB}{dx_\ast} = \sum_i w_i \frac{ds_i}{dx_\ast} \\
C = \sum_i w_i         & \frac{dC}{dx_\ast} = 0 \\
D = \sum_i w_i d_i     & \frac{dD}{dx_\ast} = 0 \\% wd
E = \sum_i w_i d_i s_i  & \frac {dE}{dx_\ast} = \sum_i w_i d_i  \frac{ds_i}{dx_\ast} \\ % wdsn
F = \sum_i w_i g_i     & \frac{dF}{dx_{data}} = \sum_i w_i \frac{dg_i}{dx_{data}}   &    \frac{dF}{dx_{sn}} = 0 \\ % wgal
G = \sum_i w_i g_i s_i  & \frac{dG}{dx_{data}} = \sum_i w_i \left(g_i \frac{ds_i}{dx_{data}} + s_i \frac{dg_i}{dx_{data}} \right) & \frac{dG}{dx_{sn}} = \sum_i w_i g_i \frac{ds_i}{dx_{sn}} % wgalsn
\end{array}
\end{displaymath}

\noindent where $i$ ranges over all pixels at the given wavelength,
$s_i$ is the SN model (scaled to 1) $g_i$ is the galaxy model, $d_i$
is the data and $w_i$ is the weight of the data. The SN amplitude, a
scalar here, is given by

\begin{equation}
SN = (-DB + EC + FB - GC) / (AC - B^2)
\end{equation}

\noindent (Derivation not shown.) Taking the derivative with
respect to $x_\ast$,

\begin{eqnarray*}
\frac{d(SN)}{dx_\ast} &=& \frac{-D'B - DB' + E'C + EC' + F'B + FB' - G'C - GC'}{AC - B^2} + \\
                     & & (SN) \frac{A'C + AC' - 2BB'}{AC-B^2} \\
                     &=& \frac{-DB' + E'C + F'B + FB' - G'C}{AC - B^2} + (SN) \frac{A'C - 2BB'}{AC-B^2}
\end{eqnarray*}

Similar to the SN amlitude, the sky is given by

\begin{equation}
sky = (DA - EB - FA + GB) / (AC - B^2)
\end{equation}

\noindent The derivative is

\begin{eqnarray*}
\frac{d(sky)}{dx_\ast} &=& \frac{D'A + DA' - E'B - EB' - F'A - FA' + G'B + GB'}{AC - B^2} + \\
                   & &  + (sky) \frac{A'C + AC' - 2BB'}{AC-B^2} \\
                   &=& \frac{DA' - E'B - EB' - F'A - FA' + G'B + GB'}{AC - B^2} + (sky) \frac{A'C - 2BB'}{AC-B^2}
\end{eqnarray*}

\end{document}


\subsection{Position space representation}

First we define the time integral of the Lagrangian of the classical oscillator given in Eq.~(\ref{eq_5}), over a period $T=2\pi/\omega$ as
\begin{equation} \label{eq_b1}
  \Delta_{\varepsilon} = \frac{1}{T} \int_0^T dt' \; L(\zeta,\dot{\zeta},t').
\end{equation}
Additionally, performing this integral, we can obtain a more simplified result
\begin{equation} \label{eq_b2}
  \Delta_{\varepsilon} = \frac{(eE)^2}{4m_e(\omega_0^2 - \omega^2)}.
\end{equation}
Next, we define another parameter
\begin{equation} \label{eq_b3}
  \xi =
  \int_0^t dt' \; L(\zeta,\dot{\zeta},t') -
  \Delta_{\varepsilon} t,
\end{equation}
and after simplifying, we can identify
\begin{equation} \label{eq_b4}
  \xi =
  \frac{(eE)^2\qty(3\omega^2 - \omega_0^2)}{8m_e\omega(\omega_0^2 - \omega^2)^2} \sin(2\omega t),
\end{equation}
which is a periodic function in time. Using these parameters, we can factorize the wave function given in Eq.~(\ref{eq_2}) as linearly time dependent part and periodic time dependent part as follows
\begin{equation} \label{eq_b5}
  \begin{aligned}
    \psi_{\alpha}&(x,y,t)  =
    \exp(\frac{i}{\hbar}\qty[-\epsilon_nt + \Delta_{\varepsilon} t ])
    \frac{1}{\sqrt{L_x}} \chi_n\big(y - y_0 - \zeta(t)\big)
    \\
    & \times
    \text{exp}\bigg(
     \frac{i}{\hbar}\bigg[
     p_x x +
     \frac{eEy}{\omega}\cos(\omega t) \\
     & \quad+
     m_e\dot{\zeta(t)}\big[y-\zeta(t)\big]
     + \int_0^{t}dt'L(\zeta,\dot{\zeta},t') - \Delta_{\varepsilon} t  \bigg]
     \bigg).
  \end{aligned}
\end{equation}
This leads to separate the linear time dependent phase component as the quasienergies
\begin{equation} \label{eq_b6}
  \varepsilon_{n} =
  \hbar \omega_0\qty(n + \frac{1}{2}) - \Delta_{\varepsilon},
\end{equation}
while rest of the components as time-periodic Floquet modes
\begin{equation} \label{eq_b7}
  \begin{aligned}
    \phi_{n,m}(x,y,t) =  &
    \frac{1}{\sqrt{L_x}} \chi_{n}\left(y - y_0 - \zeta(t)\right)
    \text{exp}\bigg(
     \frac{i}{\hbar}\bigg[
     p_x x \\
     & +
     \frac{eE[y - y_0]}{\omega}\cos(\omega t) \\
     & +
     m_e\dot{\zeta}(t)\big[y - y_0 -\zeta(t)\big]
     + \xi \bigg]\bigg).
  \end{aligned}
\end{equation}

\subsection{Momentum space representation}

We perform continuous Fourier transform over the considering confined space $A=L_xL_y$ on the Floquet modes given in Eq.~(\ref{eq_7}) to realize the Floquet modes in momentum space
\begin{equation} \label{eq_b8}
  \begin{aligned}
    \phi_{n,m}&(k_x,k_y,t) \\
    & =
    \exp(
     \frac{-i\gamma(t)}{\hbar}
     y_0)
    \exp(\frac{-i}{\hbar}
    \qty[
    m_e \dot{\zeta}(t) \zeta(t) - \xi
    ])\\
    & \quad\times
    \int_{-L_y/2}^{L_y/2} dy\; \exp(-i\qty[k_y - \gamma(t)]y)
    \chi_{n}\qty[y - \mu(t)] \\
     & \quad\times
     \frac{1}{\sqrt{L_x}}
     \int_{-L_x/2}^{L_x/2} dx\;
     \exp(-ik_x x)
     \exp( \frac{i p_x }{\hbar}x ).
  \end{aligned}
\end{equation}
Here we used new two parameters
\begin{equation} \label{eq_b9a}
  \mu(t) = \frac{eE\sin(\omega t)}{m_e(\omega_0^2 - \omega^2)} + y_0,
\end{equation}
and
\begin{equation} \label{eq_b9b}
  \gamma(t) =
  \frac{eE\omega_0^2\cos(\omega t)}{\hbar\omega(\omega_0^2 - \omega^2)}.
\end{equation}
Subsequenty, using the Fourier transform identity \cite{bruus04}
\begin{equation} \label{eq_b10}
  \int_{L_x} dx\;
  \exp( -ik_x x + \frac{i p_x }{\hbar}x ) =
  L_x \delta_{k_x,\frac{p_x}{\hbar}},
\end{equation}
we can derive
\begin{equation} \label{eq_b11}
  \begin{aligned}
    \phi_{n,m}&(k_x,k_y,t)  =
    \Phi_{n,m}(k_y,t)
    \delta_{k_x,\frac{p_x}{\hbar}}\\
    & \times
    \exp(
     \frac{-i\gamma(t)}{\hbar}
     y_0)
    \exp(\frac{-i}{\hbar}
    \qty[
    m_e \dot{\zeta}(t) \zeta(t) - \xi
    ]),
  \end{aligned}
\end{equation}
where we can define $\Phi_{n,m}(k_y,t)$ as
\begin{equation} \label{eq_b12}
  \begin{aligned}
    \Phi_{n,m}(k_y,t) =
    \sqrt{L_x}
    \int_{-L_y/2}^{L_y/2} dy\; &
    \chi_{n}\qty[y - \mu(t)] \\
    & \times
    \exp(
      -i\qty[k_y - \gamma(t)]
      y).
  \end{aligned}
\end{equation}
Substituting ${k'_y} = k_y -\gamma(t)$ with $y' = y -\mu(t)$, and assuming that the size of the considered 2DEG sample in $y$-direction is considerably large ($L_y \rightarrow \infty$), we can obtain
\begin{equation} \label{eq_b13}
  \Phi_{n,m}({k'_y} ,t) =
  {\sqrt{L_x}} e^{-i {k'_y}\mu}
  \int_{-\infty}^{\infty} dy'\;
  \chi_{n}\qty(y')
  \exp(-i{k'_y} y').
\end{equation}
Moreover, we can identify that the above integral represents the Fourier transform of $\{\chi_n\}$ functions. In addition, using the symmetric conditions of the Fourier transform for Gauss-Hermite functions $\theta_n(x)$ \cite{celeghini21}
\begin{equation} \label{eq_b14}
  \mathcal{FT}[\theta_n(\kappa x),x,k] = \frac{i^n}{|\kappa|}\theta_n(k/\kappa),
\end{equation}
we can simply the Eq.~(\ref{eq_b13}) as
\begin{equation} \label{eq_b15}
  \Phi_{n,m}({k'_y} ,t) =
    \sqrt{L_x}e^{-i {k'_y}\mu}
    \tilde{\chi}_{n}\qty({k'_y}),
\end{equation}
with
\begin{equation} \label{eq_b16}
  \tilde{\chi}_{n}\qty(k) =
  \frac{i^n}{\sqrt{2^{n} n! \sqrt{\pi}}}
  \qty(\frac{1}{\kappa})^{1/2}
  e^{-\frac{k^2}{2 \kappa^2}}
  \mathcal{H}_{\alpha} \qty(\frac{k}{\kappa}).
\end{equation}
Finally, substitute Eq.~(\ref{eq_b15}) back into Eq.~(\ref{eq_b11}) and this leads to
\begin{equation} \label{eq_b17}
  \begin{aligned}
    \phi_{n,m}(k_x,k_y,t)  = &
    {\sqrt{L_x}}
    \tilde{\chi}_{n}\qty(k_y - b\cos(\omega t)) \\
    & \times
    \text{exp}\left(
      i\xi
      -ik_y  \qty[d\sin(\omega t) + \frac{\hbar k_x}{eB}]
    \right),
  \end{aligned}
\end{equation}
where
\begin{equation} \label{eq_b18}
  b \equiv
  \frac{eE\omega_0^2}{\hbar\omega(\omega_0^2 - \omega^2)},
\end{equation}
and
\begin{equation} \label{eq_b19}
  d \equiv
 \frac{eE}{m_e(\omega_0^2 - \omega^2)}.
\end{equation}
It is necessary to notice that $k_x$ is quantized with $k_x = 2\pi m/L_x ~,~ m \in \mathbb{Z}$.