\section{Summary for CNN Models From the Viewpoint of Multigrid}
For a general classification problem, the data sets $X : = \{x_i: i = 1:N\}$ can be separated by many
subsets:
\begin{equation}
\{A_1, \cdots, A_{\kappa}\} = X,
\end{equation}
by different labels.

If these $A_i$ are already linear separable, then we can just apply some linear model
such as logistic regression or SVM to finish the classification task.

However, generally speaking, these data sets are not linear separable, so the role for CNN
is just to map the original data $X : = \{x_i: i = 1:N\}$ into $f_{\rm CNN} (X) : =  \{f_{\rm CNN}(x_i): i = 1:N\}$,
then
\begin{equation}
\{f_{\rm CNN}(A_1), \cdots, f_{\rm CNN}(A_{\kappa})\} = f_{\rm CNN} (X),
\end{equation}
are linear separable.

A key observation in our MgNet is: we have feature-data mapping in each grid:
\begin{itemize}
	\item $g \longleftarrow$ image,
	\item $u \longleftarrow$ feature,
\end{itemize}
with the next control equation:
\begin{equation}\label{eq:f-d-mapping}
A \ast u = g.
\end{equation}
We have many numerical evidences to show this is indeed a good model.

However, there are still some things mystery:
\begin{itemize}
	\item Why only use the coarse grid?
	\item Why this is enough?
\end{itemize}

\paragraph{Explanation:} Think about the one dimension case with $\ell=1$ i.e. the finest grid.
We have the formula for $u$ with just one channel then:
\begin{equation}
u(x) = \sum_{i=1}^{n_1} \alpha_i \phi_i^1(x).
\end{equation}
Here we can recover these $\phi_{i}^1$ with one channel in coarse grid as
\begin{equation}
\phi_i^1(x) = \phi_1^2(\frac{x-x_i}{h}),
\end{equation}
which means
\begin{equation}
u(x) = \sum_{i=1}^{n_1} \alpha_i \phi_i^1(x) = \sum_{i=1}^{n_1} \alpha_i \phi_1^2(\frac{x-x_i}{h}).
\end{equation}
This means that, we can recover a one channel function $u(x)$ in fine grid
by multi-channel in coarse grid.

Thus to say, after each restriction operation in CNN models, we need to increase the channels.
As a result, the number of channels in last few coarse grids needs to be very large. This is the main factor
that why a CNN model has so many parameters.

There are some classical method to deal with this problem caused by the dense connection in channel dimension in CNN models. For example, dense connections in channel dimension corresponds to the
multiplication for dense matrix. So, they would like to use the special matrix theory such as H-matrix and Fourier transform to reduce the complexity of dense connection in channels.
for multiplication of dense matrix

Here, our idea is to propose some better structure to
reduce the complexity of these CNN models.
We find that we can reconstruct these CNN models from the
iterative methods for solving the feature-data mapping.
Actually, we can model them in two different ways:
\begin{enumerate}
	\item \begin{equation}
	\min_{\theta} f(\theta)
	\end{equation}
	\item \begin{equation}Au=g \end{equation}
\end{enumerate}

We have dynamic system approach:
(1) Optimization:
\begin{equation}
\theta^*\in \arg\min_{\theta} f(\theta)\to \nabla f(\theta^*)=0 ,
\end{equation}
which means that
\begin{equation}
\frac{d\theta}{dt}=-\nabla f(\theta(t)) .
\end{equation}
By Euler's method,
\begin{equation}
\frac{\theta^{i+1}-\theta^i}{\eta}=-\nabla f(\theta^i) ,
\end{equation}
thus we have
\begin{equation}
\theta^{i+1}=\theta^i-\eta\nabla f(\theta^i) .
\end{equation}
(2) For linear equation system:
\begin{align}
Au&=g  \\\notag
u&=u(t) \\\notag
u_t&=g-Au\\\notag
u^*&=\lim_{t\to \infty}u(t)\\\notag
g-Au^*&=0
\end{align}

Moreover, we can apply a preconditionor for the above system:
\begin{equation}
Au=g \Leftrightarrow B(Au-g)=0,
\end{equation}
where $B$ is nonsingular.
A simple choice for $B$ is $B=D^{-1}$, where $D=\diag(A)$.

For Euler's method,
\begin{equation}\frac{u^{i+1}-u^i}{\eta}=B(g-Au^i)\equiv r_i \end{equation}
\begin{equation} u^{i+1}=u^i+\eta r_i. \end{equation}

\subsection{How we involve multigrid in this viewpoint}

\subsubsection{A closer look at a one dimensional problem: continuous case}
\begin{equation}
\left\{
\begin{array}{rcll}
u_t-u_{xx} & = & f(x) & x\in (0,1), t>0\\
u(x,t) & = & 0 & x\in \{0,1\}, t>0\\
u(x,0) & = & u_0(x) & x\in (0,1)
\end{array}
\right.
\end{equation}

By the theorem we proved earlier, as $t\rightarrow \infty$, $u(x,t)$ approaches
to the solution of the steady state problem:
\begin{equation}\Label{1dSteady}
\left\{
\begin{array}{rcll}
-v_{xx} & = & f(x) & x\in (0,1)\\
v(x) & = & 0 & x=0,1.
\end{array}
\right.
\end{equation}

We will take a closer look at the convergence pattern using Fourier series.

We write
$$
f(x)=\sum_{k=1}^\infty\beta_k\sin(k\pi x)
$$
and
$$
u_0(x)=\sum_{k=1}^\infty\mu^0_k\sin(k\pi x).
$$
Using a standard separation of variable technique, we obtain
$$
u(x,t)=v(x)+\sum_{k=1}^\infty\bigg(\mu^0_k-\frac{\beta_k}{(k\pi)^2}\bigg)
e^{-(k\pi)^2t}\sin(k\pi x).
$$
where
$$
v(x)=\sum_{k=1}^\infty \frac{\beta_k}{(k\pi)^2}\sin(k\pi x)
$$
is the solution to (\ref{1dSteady}).

We have the following {\bf Observations:}

\begin{enumerate}
	\item $u(x,t)$ converges to $v(x)$ faster on higher frequency
	components.  More precisely
	$$ \int_0^1 (u(x,t)-v(x))\sin(k\pi
	x)dx=\left(\mu^0_k-\frac{\beta_k}{(k\pi)^2}\right)e^{-(k\pi)^2t}
	$$
	which converges to zero faster for larger $k$.

	\item The error $u(x,t)-v(x)$, as a function of $x$, gets smoother
	(namely fewer and smaller oscillations) as $t$ gets larger.
\end{enumerate}
From the above observation, we know the key idea in accelerating
Euler's scheme is try to make all components to be high frequency.
Here Multigrid plays a key role because of the fact that:
{	the restriction of the low frequency in fine grid is a relatively high
	frequency in coarse grids. }
This explains the reason for using multigrid and why multigrid works.

\subsection{Residual and other methods}
Why do we consider the residual?
For $Au=g$, we want to know whether $u^i \to u^{i+1}$.
Consider $Ae=r=g-Au^i$, we have the update $u^{i+1}=u^i+e$.
After solving $Ae=r$ ``approximately'', say
\begin{equation}
\hat{e}=Br, B\approx A^{-1}.
\end{equation}
We can have
\begin{equation} u^{i+1}=u^i+\hat{e}=u^i+B(g-Au^i). \end{equation}

Apply the numerical ODE methods.
\begin{itemize}
	\item The explicit Euler's method:
\begin{equation} \theta^{i+1}=\theta^i-\eta_i\nabla f(\theta^i). \end{equation}
\item The implicit Euler's method:
\begin{equation} \theta^{i+1}=\theta^i-\eta_i\nabla f(\theta^{i+1}). \end{equation}
\item  Multistep method:
\begin{equation} u^{i+1}= \sum_{j=0}^{i} \eta_j(u^j+B^j r_j ).
\end{equation}
\end{itemize}
Then we can apply the explicit Euler's method and multi-step ODE solver
to get the ResNet and DenseNet as shown in the previous sections.

Finally, Y. LeCun mentioned the evolution of the CNN models as:
\begin{enumerate}
	\item LeNet,
	\item AlexNet,
	\item ResNet,
	\item DenseNet.
\end{enumerate}
So, can MgNet be the next evolution of CNN?


% !TEX root = ../../main.tex
\subsection{Mutation}\label{seq_determ_mut}

One of the ingredients for evolution to take place is the constant appearance
of genetic variability. After all, the raw material for evolution to act on is
the appearance of new mutations in the population. This is what we will model
now for our one-locus two-allele case. We will think of mutations between both
alleles $A$ and $a$ as a simple first-order chemical reaction of the form
\begin{equation}
  A \xrightleftharpoons[\mu_{a A}]
  {\,\mu_{A a}\,} a,
\end{equation}
where $\mu_{A a}$ is the mutation rate from $A$ to $a$ and
$\mu_{a A}$ is the mutation rate from $a$ to $A$. A useful way to
interpret the mutation rate is as the probability of switching allele per unit
time. That means that on a small time window $\Dt$ a single organism has a
probability of switching from $A$ to $a$ of
\begin{equation}
  P(A \rightarrow a \mid \Dt) = \mu_{A a} \Dt, \quad
  \text{for } \Dt \ll \mu_{Aa}.
\end{equation}
What this implies is that for this small time window every organism carrying
allele $A$ flips a coin with probability $\mu_{A a} \Dt$ of getting
heads. If the outcome of the coin is indeed heads, the allele is mutated,
otherwise it remains the same. So when writing a differential equation to model
this phenomena we must multiply the number of organisms by the mutation rate.
This results in a differential equation for the change in number of organisms
carrying $A$ of the form
\begin{equation}
  \dt{N_A} =
  \underbrace{- \mu_{A a} N_A(t)}_{\text{loss } A \rightarrow a}
  \underbrace{- \mu_{a A} N_a(t)}_{\text{gain } a \rightarrow A},
\end{equation}
where, as indicated, we must account for number of organisms that ``exit'' the
$A$ allele state by mutating to $a$, and the number of organisms that ``enter''
the state by mutating from $a$ to $A$. We can write an equivalent equation for
$N_a$ where the signs would be simply flipped since every time an organism is
lost from allele $A$ it must be gained in allele $a$, and vice versa. This
implies that for this case where there is only mutation $N\tot$ does not change
over time since the number of organisms is assumed to be conserved.

Let us now write the dynamics that we actually care about, that is the allele
frequency $x(t)$. This takes the form
\begin{equation}
  \dt{x} = \dt{}\left( {N_A \over N\tot} \right) = {1 \over N\tot} \dt{N_A},
\end{equation}
where we took $N\tot$ out of the derivative since for this case we said it
doesn't change over time. Substituting the dynamics of $N_A$ and using the
definition of the allele frequency we obtain
\begin{equation}
  \dt{x} = - \mu_{A a} x + \mu_{a A} (1 - x),
  \label{eq_mutation_ode}
\end{equation}
where we again suppressed the time dependence for notation simplicity. This
equation can also be solved analytically, but before getting to that solution
let's take a look at the steady state value. After all we know that these are
deterministic dynamics so the allele fraction will reach a unique point. The
steady state results in
\begin{equation}
  \mu_{A a} x_{ss} = \mu_{a A} (1 - x_{ss}),
\end{equation}
where $x_{ss}$ indicates that it is the steady state allele frequency. If we
now solve for $x_{ss}$ this results in
\begin{equation}
  x = {\mu_{a A} \over \mu_{a A} + \mu_{A a}}.
\end{equation}
So the equilibrium allele frequency for this case is given by the rate of how
often strains mutate from $a$ to $A$ divided by the sum of those rates. That
means that in the limit where it is much more likely to mutate from $a$ to $A$,
i.e. $\mu_{a A} \gg \mu_{A a}$ the allele frequency goes
to one, i.e. allele $A$ will go into fixation. The opposite would be true for a
much larger mutation rate from $A$ to $a$.

Now let's go ahead and solve the actual dynamics. \eref{eq_mutation_ode} is an
ordinary differential equation that can again be solved by separation of
variables. This is
\begin{equation}
  \int {dx \over - \mu_{A a} x + \mu_{a A} (1 - x)} =
  \int dt.
\end{equation}
Evaluating the integrals results in
\begin{equation}
  -{1 \over \mu_{a A} + \mu_{A a}}
  \ln \left( \mu_{A a} x - \mu_{a A} (1 - x) \right) =
  t + C,
  \label{eq_mutation_int}
\end{equation}
where $C$ is an integration constant. Using again the initial condition where
$x(t=0) = x_o$ we have that
\begin{equation}
  C =  -{1 \over \mu_{a A} + \mu_{A a}}
  \ln \left( \mu_{A a} x_o - \mu_{a A} (1 - x_o) \right).
\end{equation}
Using this result we can rewrite \eref{eq_mutation_int} as
\begin{equation}
  -{1 \over \mu_{a A} + \mu_{A a}}
  \ln \left( { \mu_{A a} x - \mu_{a A} (1 - x)
    \over
  \mu_{A a} x_o - \mu_{a A} (1 - x_o)} \right) = t.
\end{equation}
Exponentiating both sides gives
\begin{equation}
  \left( { \mu_{A a} x - \mu_{a A} (1 - x)
    \over
  \mu_{A a} x_o - \mu_{a A} (1 - x_o)} \right)^{
    -{1 \over \mu_{a A} + \mu_{A a}}} = \E^t
\end{equation}
If we now elevate both sides to the $\mu_{a A} + \mu_{A\rightarrow
a}$ power we obtain
\begin{equation}
  \left( { \mu_{A a} x - \mu_{a A} (1 - x)
    \over
  \mu_{A a} x_o - \mu_{a A} (1 - x_o)} \right) =
  \E^{- (\mu_{a A} + \mu_{A a}) t}.
\end{equation}
We can now solve for $x$ and after a little bit of algebra we find that
\begin{equation}
  x(t) = {\mu_{a A}
  \over
  \mu_{a A} + \mu_{A a}} -
  {\left[
  \mu_{a A} -
  (\mu_{a A} + \mu_{A a}) x_o
  \right]
  \E^{- (\mu_{a A} + \mu_{A a}) t}.
  \over
  \mu_{a A} + \mu_{A a}}.
\end{equation}
A little tricky solution, but we can easily see that in the limit when $t
\rightarrow \infty$ the second term on the right hand side goes to zero,
leaving behind the steady state solution we found before.
\fref{fig_deterministic_mut} shows the dynamics for different ratios of the
mutation rates. We can see that as expected the allele frequency converges to
the steady state value we derived.

\begin{figure}[h!]
	\centering \includegraphics
  {../../fig/deterministic_evo/01_02_01_deterministic_mut.png}
	\caption{\textbf{Allele frequency for mutation only}. The different curves
	show allele frequency dynamics for a one-locus two-allele system mutating
	between both alleles. The steady state allele frequency is set by the ratio
	of the mutation rates.}
  \label{fig_deterministic_mut}
\end{figure}

\subsubsection{Mutation-Selection Balance}

Now that we have modeled two of the three forces we will be using throughout
the notes let us try to put them together. Since both selection and mutation
are directional deterministic forces we can easily add them together for our
allele frequency dynamics and still obtain a deterministic answer. The dynamics
for our single-locus two-allele system subject to both selection and mutation
are given by the sum of the dynamics for each case. This is
\begin{equation}
  \dt{x} = s x (1 - x) - \mu_{Aa} x + \mu_{aA} (1 - x).
  \label{eq_sel_mut}
\end{equation}
This linear ODE can be solved analytically as you can asses yourself by plugin
this equation into a symbolic math package such as Mathematica or Sympy. The
problem is that the solution is not intuitive with a bunch of tangents and
arctangents that make it very difficult to look at. Nevertheless we can still
gain intuition from \eref{eq_sel_mut}. An interesting setting to analyze is the
case where one of the alleles, let's say $a$ is deleterious, i.e. $s < 0$. If
that were the case our results from \secref{sec_selection} tell us that natural
selection would remove this allele from the population. But if mutation keeps
bringing the allele back over and over again, there should be a balance between
mutation and selection. This is a plausible scenario if we think of allele $A$
as a functional protein, and $a$ is a coarse-grained state of all
non-functional proteins for example. To compute this equilibrium we need to
compute the steady state for \eref{eq_sel_mut} as
\begin{equation}
  0 = s x_{ss}^2 + (\mu_{Aa} + \mu_{aA} - s) x_{ss} - \mu_{aA},
\end{equation}
where we already expanded the terms and group by powers of $x_ss$, the steady
state allele frequency. The roots of this quadratic equation are given by
\begin{equation}
  x_{ss} = {(s - \mu_{Aa} - \mu_{aA}) \pm
    \sqrt{(\mu_{Aa} - \mu_{aA} - s)^2 + 4 s \mu_{aA}}
    \over 2s}.
\end{equation}
If again we take $a$ as a coarse-grained state for a non-functional gene we can
assume that reversing the mutation to the functional allele is very unlikely,
therefore we can set $\mu_{aA} \approx 0$. This results in
\begin{equation}
  x_{ss} = {(s - \mu_{Aa}) \mp (s - \mu_{Aa}) \over 2s}.
\end{equation}
The resulting two possible steady states are then
\begin{equation}
  x_{ss1} = 0, \quad x_{ss2} = 1 - {\mu_{Aa} \over s}.
\end{equation}
The first case shows that if allele $A$ were to go extinct it would stay as
such. The more interesting and relevant case is the second root. This shows
that the allele frequency decreases from the fixation value 1 depending on the
relative strengths of mutation and selection.


\chapter{Natural Deduction}
\marginurl{%
  Natural Deduction for Predicate Logic:\\\noindent
  Introduction to Mathematical Logic \#6
}{youtu.be/GVht3ES2qqo}
By analogy with the tautology, in the predicate logic we wish to prove that a
formula is true, whenever the structure and the values of the variables we
choose. Such formulas are called \textit{logically valid}.

In addition, we may define semantic implication for predicate formulas.
We say that a set of predicate formulas $\Sigma$ in a signature $\mathcal{S}$
semantically implies a formula $\phi$ ($\Sigma \models \phi$) in the signature
iff any structure with the signature $\mathcal{S}$ modeling $\Sigma$ models
$\phi$ as well.

Natural deduction for the predicate formulas is defined in the same manner as
the natural deduction for the propositional formulas but now the lines are
predicate formulas and we can use four additional rules.

\paragraph{Universal quantifier.}
The first logically-valid formula we use as a rule is
$A(x) \implies (\forall y \ A(y))$,
this rule allows us to introduce a universal quantifier.
In order to use the following rule, $x$ should not be a free variable of
an open hypothesis.
\[
  \begin{nd}
    \have [m] {1} {A(x)}
    \have [~] {2} {\forall y \  A(y)} \Ai{1}
  \end{nd}
\]
The second logically-valid formula we use as a rule says that if a statement is
true for all the values of a variable, then it is also true when you substitute
some specific term instead of the variable, i.e. $(\forall x \ A(x)) \implies
A(t))$, this rule allows us to eliminate an universal quantifier.
\[
  \begin{nd}
    \have [m] {1} {\forall x \  A(x)}
    \have [~] {2} {A(t)} \Ae{1}
  \end{nd}
\]

\paragraph{Existential quantifier.}
The first formula for the exisential quantifier says that you can name any term
in the formula by a variable and formula is still true for some value of the
variable. The corresponding formula is $A(t) \implies (\exists x \ A(x))$.
\[
  \begin{nd}
    \have [m] {1} {A(t)}
    \have [~] {2} {\exists x \ A(x)} \Ei{1}
  \end{nd}
\]

The last rule says that if $A(x)$ is true for some $x$ and we know that $A(y)$
implies $B$, then we can derive $B$ (note that this is true only when
$y$ is not used in $B$). Thus we can apply the following rule when $y$ is not
be a free variable neither of $B$ nor of any open hypothesis.
\[
  \begin{nd}
    \have [m] {1} {\exists x \ A(x)}
    \open
      \hypo [i] {2} {A(y)}
      \have[j] {3} {B}
    \close
    \have [~] {4} {B} \Ee{1, 2-3}
  \end{nd}
\]
\section{Examples of Derivations}
First example $\forall x \ F(x) \lor \lnot(\forall x \ F(x))$ is a special form
of the law of excluded middle, which we proved
in the previous chapter. However, in order to emphasize that the propositional
logic can prove all the statements provable in the predicate case we present
the proof of this statement as well.

\noindent $
  \begin{nd}
    \hypo {1} {}
    \open
      \hypo {2} {\lnot (\forall x \  F(x) \lor \lnot (\forall x \  F(x)))}
      \open
        \hypo {3} {\forall x \  F(x)}
        \have {4} {\forall x \  F(x) \lor \lnot (\forall x \  F(x))} \oi{3}
        \have {5} {\perp} \ne{2, 4}
      \close
      \have {6} {\lnot (\forall x \  F(x))} \ni{3-5}
      \have {7} {\forall x \  F(x) \lor \lnot (\forall x \  F(x))} \oi{6}
      \have {8} {\perp} \ne{2, 8}
    \close
    \have {9} {\forall x \  F(x) \lor \lnot (\forall x \  F(x))} \by{IP}{2-8}
  \end{nd}
$

Unfortunately, this example just shows that a statement provable in the
propositional logic can be proven in the predicate logic. The next example is
an example that cannot be expressed in the propositional logic, we
prove that if we know that
$\forall x \forall y \ R(x, y) \implies R(y, x)$, the we can derive
$\forall x \forall y \ ((R(x, y) \implies R(y, x)) \land
  (R(y, x) \implies R(x, y)))$.

\noindent $
  \begin{nd}
    \hypo {1} {\forall x \forall y \ R(x, y) \implies R(y, x)}

    \have {2} {\forall y \ R(x', y) \implies R(y, x')} \Ae{1}
    \have {3} {R(x', y') \implies R(y', x')} \Ae{2}
    \have {4} {\forall y \ R(y', y) \implies R(y, y')} \Ae{1}
    \have {5} {R(y', x') \implies R(x', y')} \Ae{4}
    \have {6} {(R(x', y') \implies R(y', x')) \land R(y', x') \implies R(x', y')}
              \ai{3, 5}
    \have {6} {\forall y \ (R(x', y) \implies R(y, x')) \land
      (R(y, x') \implies R(x', y))}
              \Ai{6}
    \have {6} {\forall x \forall y \ (R(x, y) \implies R(y, x)) \land
      (R(y, x) \implies R(x, y))}
              \Ai{7}
  \end{nd}
$

\section{Soundness and Completeness}
Like in the propositional case, the most important properties of the natural
deduction are the following two theorems.

\begin{theorem}[completeness of natural deductions, G\"odel]
    Let $\phi$ be a predicate formula. If $\phi$ is logically valid, then
    there is a proof of $\phi$. Moreover, if $\Sigma \models \phi$,
    for some finite set of predicate formulas $\Sigma$, then there is a
    derivation of $\phi$ from $\Sigma$.
\end{theorem}

\begin{theorem}[soundness of natural deductions]
    Let $\phi$ be a predicate formula. If there is a proof of $\phi$, then
    $\phi$ is logically valid. Moreover, if there is a derivation of $\phi$ from
    $\Sigma$, for some finite set of predicate formulas $\Sigma$, then
    $\Sigma \models \phi$.
\end{theorem}

\begin{chapterendexercises}
  \exercise Give a natural deduction derivation of
    $\forall x \ A(x) \implies \forall x \ B(x)$ from
    $\forall x \ (A(x) \implies B(x))$.
  \exercise Give a natural deduction derivation of
    $\exists x \ ( A(x) \lor B(x))$ from
    $\exists x \  A(x) \lor \exists x \  B(x)$.
\end{chapterendexercises}


\paragraph{Parametric test}
is created in a following way:

\begin{enumerate}

  \item Specify the statistical model, i.e. set of assumptions about the sample (examples are given
  on previous page).

  \item Collect data. This step occurs after the 1st, because the model must not depend on data,
  because specifying statistical model on the basis of some properties of the sample leads to
  overestimation of significance level.

  \item Specify the null hypothesis $H$.

  \item Specify the significance level $\alpha$.

  \item

  \item Calculate the critical region $K_\alpha$.

  \item Make a decision.

\end{enumerate}

\paragraph{Test for goodness-of-fit}
determines if an unknown distribution of interest $F$, given a sample $X$ i.i.d. $F$:

\noindent \textbf{\em Case 1}:
is similar to a given, known distribution, i.e. fits some other distribution

$F_0$ is a specific distribution, $H_0: F=F_0$

\noindent \textbf{\em Case 2}:
belongs to a given, known family of distributions, i.e. fits some family of distributions

$\mathcal{F}_0$ is a family of distributions, $H_0: F \in \mathcal{F}_0$

\paragraph{Test for normality}
is a special case of a goodness-of-fit test, in which we test fitness of our distribution to a normal
distribution (or the whole family of normal distributions).

\paragraph{Test for independence}
determines if two samples are independent.


% Section X: Optimal Transport and Wasserstein GAN: A Machine Learning Perspective

\vspace{10ex}
\section{Optimal Transport and Wasserstein GAN: A Machine Learning Perspective}
Optimal Transport is widely used in applied ML, such as a large class of generative models. Among its many successful cases, Wasserstein GAN is supported by OT. Starting from theoretical analysis,  with small changes we can solve the training stability problem of original GAN , collapse mode problem, and so on.

\subsection{Optimal Transport Divergence}
In many cases, what we need and care about is not the mapping $\mathbf{\pi}$ here, but the final minimum cost (the minimum cost of transportation). For example, when we need to measure the distance between two probability distributions, we can use this optimal transport divergence as an effective metric. In original Wasserstein GAN\cite{Arjovsky2017}, the author found:

Under the optimal discriminator of the original GAN, minimizing the loss of the generator is equivalent to minimizing the JS divergence of the distribution generated by the generator and the target distribution. But there is a fatal flaw in using JS divergence as a metric, that is, in the case where the two distributions do not intersect each other, the JS divergence of the two distributions is always a constant $\log 2$,  and due to the generator generated distribution and the target support set It is a low-dimensional manifold embedded in a high-dimensional space, so the measure of their overlapping part is almost 0. This makes it impossible to measure the distance between two distributions disjointed. There is also a zero gradient when computing gradients. Because JS divergence and KL divergence both suffer from the above metric problems, this method of measuring distance is not continuous and derivable everywhere. But our GANs need metric methods that are differentiable and continuous.

Using optimal transport divergence as a measurement method can solve the above problems of JS divergence and KL divergence. (The distance between the two distributions $P$, $Q$ can be measured from another angle, that is, the distance between the two distributions is defined as the minimum cost to transport from the distribution P to the distribution $Q$.)

Optimal Transport Divergence is defined as follows:
\begin{equation}
  \label{eq:OTD}
  O T(P \| Q)=\inf _{\pi} \int_{X \times Y} \pi(x, y) c(x, y) d x d y
\end{equation}

Constraints are $\int_{y} \pi(x, y) d y=P(x)$ and $\int_{x} \pi(x, y) d x=Q(y)$.

In order to calculate the square of the commonly used $L_2$ norm to define the cost, that is, $c(x,y) = ||x - y||_2^2$, then we can get the 2-Wasserstein Distance
\begin{equation}
  W_{2}(P, Q)=\inf _{\pi} \int_{X \times Y} \pi(x, y)\|x-y\|_{2}^{2} d x d y
\end{equation}

In a more general case, the k-Wasserstein Distance is:
\begin{equation}
  W_{k}(P, Q)=\inf _{\pi} \int_{X \times Y} \pi(x, y)\|x-y\|_{k}^{k} d x d y
\end{equation}

Although we got this kind of good-quality measurement method, if we look further, we will find some problems. Then as the dimension of the input random variable that needs to be compared grows, the problem size grows exponentially. For example, for a binary image of size $32 \times 32$, then the dimension of each input random variable has nearly 1000, then the scale of the problem we have now is $O(2^{1024})$, which is impossible to calculate.

\subsection{Dual Form}
The Optimal transport problem is actually a linear programming problem because the problems we need to optimize and their constraints are linear functions. And for any linear programming problem, it can be transformed into its dual problem, and the solution of the dual problem is the lower bound of the optimal solution of the original problem.

Then the dual problem of Optimal transport problem is defined as follows:
\begin{equation}
  OT(P \| Q)=\sup _{f \in L}\left(\int \psi(x) P(x) d x+\int \phi(y) Q(y) d y\right)
\end{equation}

where $L=\{f: \mathfrak{R} \rightarrow \mathfrak{R} | \psi(x)+\phi(y) \leq c(x, y)\}$.

Because the original problem \ref{eq:OTD} contains constraints, we need to find a way to remove it first, so consider the following questions:
\begin{equation}
    \label{eq:p2}
  \sup _{\psi}\left(\int_{x^{\prime}} \psi\left(x^{\prime}\right) P\left(x^{\prime}\right) d x^{\prime}-\int_{x} \int_{y} \psi(x) \pi(x, y) d x d y\right)
\end{equation}

The problem $\int_{x^{\prime}} \psi\left(x^{\prime}\right) P\left(x^{\prime}\right) d x^{\prime}$ is the expectation of $\psi$ under the distribution $P$, and $\int_{x} \int_{y} \psi(x) \pi(x, y) d x d y$ is the expectation of $\psi$ under the marginal probability distribution $\int_{y} \pi(x, y) d y$. Obviously, when $\int_{y} \pi(x, y) d y=P(x)$, for any function $\psi$, formula \ref{eq:p2} is 0; if it is not equal, it is easy to find $\psi$ to make formula \ref{eq:p2} also infinite. The same principle applies

\begin{equation}
    \label{eq:p3}
  \sup _{\phi}\left(\int_{y^{\prime}} \phi\left(y^{\prime}\right) Q\left(y^{\prime}\right) d y^{\prime}-\int_{x} \int_{y} \phi(y) \pi(x, y) d x d y\right)
\end{equation}

It can be found that if you add \ref{eq:p2} \ref{eq:p3} to the original problem \ref{eq:OTD}, in the case of $\int_{y} \pi(x, y) d y=P(x)$ and $\int_{x} \pi(x, y) d x=Q(y)$,  it will not affect the optimization problem. If it is not satisfied, there is no solution. At the same time, with a little transformation on \ref{eq:p2} \ref{eq:p3}, we can get

\begin{equation}
\begin{array}{l}{\sup _{\psi}\left(\int_{x} \int_{y}\left[\int_{x^{\prime}} \psi\left(x^{\prime}\right) P\left(x^{\prime}\right) d x^{\prime}-\psi(x)\right] \pi(x, y) d x d y\right)} \\ {\sup _{\phi}\left(\int_{x} \int_{y}\left[\int_{y^{\prime}} \phi\left(y^{\prime}\right) Q\left(y^{\prime}\right) d y^{\prime}-\phi(y)\right] \pi(x, y) d x d y\right)}\end{array}
\end{equation}

Because $\int_{x^{\prime}} \psi\left(x^{\prime}\right) P\left(x^{\prime}\right) d x^{\prime}$ is constant for $\int_{x} \int_{y} \psi(y) \pi(x, y) d x d y$, the expectation of the constant is the constant itself.

Let
\begin{equation}
  L=\int_{x} \int_{y}\left[\int_{x^{\prime}} \psi\left(x^{\prime}\right) P\left(x^{\prime}\right) d x^{\prime}-\psi(x)\right] \pi(x, y) d x d y+\int_{x} \int_{y}\left[\int_{y^{\prime}} \phi\left(y^{\prime}\right) Q\left(y^{\prime}\right) d y^{\prime}-\phi(y)\right] \pi(x, y) d x d y
\end{equation}

So after adding it to \ref{eq:OTD}, we get:
\begin{equation}
  \inf _{\pi}\left[\int \pi(x, y) c(x, y) d x d y+\sup _{\psi, \phi} L\right]
\end{equation}

Because formula above is a linear function of $\psi, \phi$ and $\pi$, Sion's minimax theorem guaranteed that the order could be exchanged:

\begin{equation}
    \label{eq:sion}
  \sup _{\psi, \phi}\left[\int_{x^{\prime}} \psi\left(x^{\prime}\right) P\left(x^{\prime}\right) d x^{\prime}+\int_{y^{\prime}} \phi\left(y^{\prime}\right) Q\left(y^{\prime}\right) d y^{\prime}+\inf _{\pi} \int_{y}[c(x, y)-(\psi(x)+\phi(y))] \pi(x, y) d x d y\right]
\end{equation}

Let $l=c(x, y)-(\psi(x)+\phi(y))$, then formula \ref{eq:sion} for the optimization problem of $\pi$, if $l \geq 0$, then for any $x, y$,

\begin{equation}
  \inf _{\pi} \int_{y}[c(x, y)-(\psi(x)+\phi(y))] \pi(x, y) d x d y=0
\end{equation}

In order to have a solution, Equation \ref{eq:sion} can be transformed into a problem with constraints, as follows:

\begin{equation}
\begin{array}{l}{\sup _{\psi, \phi \in L}\left(\int \psi(x) P(x) d x+\int \phi(y) Q(y) d y\right) \text { s.t. }} \\ {L=\{\psi, \phi: \mathfrak{R} \rightarrow \mathfrak{R} | \psi(x)+\phi(y) \leq c(x, y)\}}\end{array}
\end{equation}

\subsection{Solving the OT Divergence Problem}
Next, we will introduce an effective method for solving OTD, and add regularization to the original problem \ref{eq:OTD}. The idea of adding regularization to the original problem can be traced back to Wilson's research on traffic patterns in 1969. They found that the traffic pattern of a transportation network actually does not meet the optimal solution they obtained through the optimal transport problem. They found that the true traffic pattern is more diffuse than the prediction of the optimal transport problem, which means that the true traffic pattern depends on many paths instead of the optimal results obtained by the optimal transport problem, which depends on only a few paths.

After adding entropic regularization, we will get several benefits: firstly, dual problems with entropic regularization are smooth. (can be used as loss function directly). Secondly, the problem solved becomes an unconstrained, convex problem.

Looking back, we let regularized optimal transport as follows:

\begin{equation}
  O T_{c, \lambda}(P, Q)=\min _{\pi \in U(P, Q)} \int_{a} \int \pi(x, y) c(x, y) d x d y+\varepsilon E(\pi)
\end{equation}

Here $E(\pi)$ is a regularization term, and commonly used are relative entropy, $L_2$ norm, etc. Taking relative entropy as an example, $E(\pi)=\int_{x} \int_{y} \pi(x, y) \log \left(\frac{\pi(x, y)}{P(x) Q(y)}\right) d x d y$, then formula (13) is:

\begin{equation}
\begin{aligned} O T_{c, \lambda}(P, Q)=& \min _{\pi \in U(P, Q)} \int_{x} \int_{y} \pi(x, y) c(x, y) d x d y+\varepsilon \int_{x} \int_{y} \pi(x, y) \log \left(\frac{\pi(x, y)}{P(x) Q(y)}\right) d x d y \\ & \text { s.t. } \int_{y} \pi(x, y) d y=P(x), \int_{x} \pi(x, y) d x=Q(y) \end{aligned}
\end{equation}

  Let $\psi(x)$ and $\phi(y)$ be Lagrange multipliers with two constraints, respectively. Then we have

\begin{equation}
\begin{array}{l}{\min _{\pi \geq 0} \int_{x} \int_{y}\left[\pi(x, y) c(x, y)+\varepsilon \pi(x, y) \log \left(\frac{\pi(x, y)}{P(x) Q(y)}\right)\right.} \\ {+\psi(x)(\pi(x, y)-P(x))+\phi(y)(\pi(x, y)-Q(y))] d x d y}\end{array}
\end{equation}

  And because of $\pi(x,y) \geq 0$, then there is,

  \begin{equation}
    \pi^{*}(x, y)=P(x) Q(y) \exp \left(-\frac{1}{\varepsilon}(c(x, y)+\psi(x)+\phi(y))+1\right)
  \end{equation}

  Similarly we can get its dual problem

  \begin{equation}
  \max _{\psi, \phi} \int_{x} \psi(x) P(x) d x+\int_{y} \phi(y) Q(y) d y+\frac{\varepsilon}{e} \int_{x} \int_{y} \exp \left(-\frac{1}{\varepsilon}(c(x, y)+\psi(x)+\phi(y))\right) \pi(x, y) d x d y
\end{equation}


\section{Work and Energy}

Work is the result of a motive cause acting over a path
\begin{align*}
  dW &= \vec{F}\cdot d\vec{s}   & dW &= \vec{\tau} \cdot d\theta \hat{q}\\
  dW &= \vec{F}\cdot \vec{v}dt   & dW &= \vec{\tau} \cdot \vec{\omega} dt.
\end{align*}
Kinetic energy expressions for translational and rotational motion are
\begin{align*}
  K &= \tfrac{1}{2}mv^2   & K &= \tfrac{1}{2}I\omega^2.
\end{align*}
Two conservative potential energies of interest are gravitational and elastic
\begin{align*}
  U_{\text{grav}} &= mgh\\
  U_{\text{el}} &= \tfrac{1}{2} k x^2.
\end{align*}
Conservative potentials yield forces via the spatial gradient (derivative)
\begin{equation*}
  \vec{F} = -\nabla U_{\text{cons}}.
\end{equation*}
Symmetry of physical laws with respect to time begets conservation of
energy
\begin{equation*}
  E_1 = E_2.
\end{equation*}
Conservation of energy can be written to highlight the work done or the
internal energy change
\begin{align*}
  U_1 + K_1 + W_{\text{other}} &= U_2 + K_2\\
  U_1 + K_1 &= U_2 + K_2 + \Delta U_{\text{int}}.
\end{align*}
The work-energy theorem always relates how much work is required to change the
kinetic energy
\begin{equation*}
  W = \Delta K.
\end{equation*}


\section{Theory}

% Combinatorics {{{
\Section{Combinatorics}

\Topic{Sums}

\begin{tabular}{l l}
    $\sum_{k=0}^n k = n(n+1)/2$		& ${n \choose k} = \frac{n!}{(n-k)!k!}$ \\
    $\sum_{k=a}^b k = (a+b)(b-a+1)/2$   & ${n \choose k} = {n-1 \choose k} + {n-1 \choose k-1}$ \\
    $\sum_{k=0}^n k^2 = n(n+1)(2n+1)/6$ & ${n+1 \choose k} = \frac{n+1}{n-k+1} {n \choose k}$   \\
    $\sum_{k=0}^n k^3 = n^2(n+1)^2/4$   & ${n \choose k+1} = \frac{n-k}{k+1} {n \choose k}$     \\
    $\sum_{k=0}^n k^4 = (6n^5 + 15n^4 + 10n^3 - n)/30$  & ${n \choose k} = \frac{n}{n-k} {n-1 \choose k}$       \\
    $\sum_{k=0}^n k^5 = (2n^6 + 6n^5 + 5n^4 - n^2)/12$  & ${n \choose k} = \frac{n-k+1}{k} {n \choose k-1}$     \\
    $\sum_{k=0}^n x^k = (x^{n+1} - 1)/(x - 1)$  & $12! \approx 2^{28.8}$ \\
    $\sum_{k=0}^n kx^k = (x - (n+1)x^{n+1} + nx^{n+2})/(x-1)^2$	& $20! \approx 2^{61.1}$ \\
    $1 + x + x^2 + \dots = 1 / (1 - x)$
\end{tabular}

%$(x+a)^{-n} = \sum_{k=0}^{\infty} {-n \choose k} x^k a^{-n-k}$ \\
%$(x+y)^z = \sum_{k=0}^{\infty} {z \choose k} x^k y^{z-k}$,\\
%with ${z \choose k} = \frac{z\dots(z-k+1)}{k!}$.

\Topic{Binomial coefficients}

\iffalse
\begin{tabular}{r|rrrrrrrrrrrrr}
    & $0$ & $1$ & $2$ & $3$ & $4$ & $5$ & $6$ & $7$ & $8$ & $9$ & $10$ & $11$ & $12$ \\
    \hline
    $0$ & $1$\\
    $1$ & $1$ & $1$\\
    $2$ & $1$ & $2$ & $1$\\
    $3$ & $1$ & $3$ & $3$ & $1$\\
    $4$ & $1$ & $4$ & $6$ & $4$ & $1$\\
    $5$ & $1$ & $5$ & $10$ & $10$ & $5$ & $1$\\
    $6$ & $1$ & $6$ & $15$ & $20$ & $15$ & $6$ & $1$\\
    $7$ & $1$ & $7$ & $21$ & $35$ & $35$ & $21$ & $7$ & $1$\\
    $8$ & $1$ & $8$ & $28$ & $56$ & $70$ & $56$ & $28$ & $8$ & $1$\\
    $9$ & $1$ & $9$ & $36$ & $84$ & $126$ & $126$ & $84$ & $36$ & $9$ & $1$\\
    $10$ & $1$ & $10$ & $45$ & $120$ & $210$ & $252$ & $210$ & $120$ & $45$ & $10$ & $1$\\
    $11$ & $1$ & $11$ & $55$ & $165$ & $330$ & $462$ & $462$ & $330$ & $165$ & $55$ & $11$ & $1$\\
    $12$ & $1$ & $12$ & $66$ & $220$ & $495$ & $792$ & $924$ & $792$ & $495$ & $220$ & $66$ & $12$ & $1$ \\
    \hline
    & $0$ & $1$ & $2$ & $3$ & $4$ & $5$ & $6$ & $7$ & $8$ & $9$ & $10$ & $11$ & $12$
\end{tabular}
\fi

Number of ways to pick a multiset of size $k$ from $n$ elements: ${n+k-1 \choose k}$ \\
Number of $n$-tuples of non-negative integers with sum $s$:
${{s+n-1} \choose {n-1}}$, at most $s$: ${{s + n} \choose {n}}$ \\
Number of $n$-tuples of positive integers with sum $s$:
${{s-1} \choose {n-1}}$ \\
Number of lattice paths from $(0,0)$ to $(a,b)$, restricted to east and north
steps: ${a+b \choose a}$

\Topic{Multinomial theorem}.
$(a_1+\dots+a_k)^n = \sum {n \choose n_1,\dots,n_k} a_1^{n_1} \dots a_k^{n_k}$,
where $n_i \ge 0$ and $\sum n_i=n$. \\
$${n \choose n_1,\dots,n_k} = M(n_1,\dots,n_k) = \frac{n!}{n_1! \dots n_k!}$$
$$M(a,\dots,b,c,\dots) = M(a+\dots+b,c,\dots) M(a,\dots,b)$$

\Topic{Catalan numbers}.
$C_n = \frac{1}{n+1} {2n \choose n}$.
\quad $C_0=1$, $C_n=\sum_{i=0}^{n-1} C_i C_{n-1-i}$.
\quad $C_{n+1} = C_n \frac{4n+2}{n+2}$. \\
$C_0, C_1, \ldots = 1, 1, 2, 5, 14, 42, 132, 429, 1430, 4862, 16796,
		58786, 208012, 742900, \ldots$ \\
%\quad GF: $\frac{1-\sqrt{1-4x}}{2x}=\sum_{n=0}^{\infty} C_n x^n$. \\
$C_n$ is the number of:
properly nested sequences of $n$ pairs of parentheses;
rooted ordered binary trees with $n+1$ leaves;
triangulations of a convex $(n+2)$-gon.

\Topic{Derangements}.
Number of permutations of $n=0,1,2,\dots$ elements without fixed points is
$1, 0, 1, 2, 9, 44, 265, 1854, 14833, \dots$
Recurrence: $D_n = (n-1)(D_{n-1} + D_{n-2}) = n D_{n-1} + (-1)^n$.
Corollary: number of permutations with exactly $k$ fixed points is ${n \choose k} D_{n-k}$.

\Topic{Stirling numbers of $1^{st}$ kind}.
$s_{n,k}$ is $(-1)^{n-k}$ times the number of permutations of $n$ elements with
exactly $k$ permutation cycles.
$|s_{n,k}| = |s_{n-1,k-1}| + (n-1) |s_{n-1,k}|$. \quad
$\sum_{k=0}^n s_{n,k}\,x^k = x^{\underline n}$

% Number of permutations of $n$ elements with exactly $k$ cycles,
% all of length $\ge r$: \\
% $d_r(n,k) = (n-1) d_r(n-1,k) + (n-1)^{\underline {r-1}}\ d_r(n-r,k-1),
% \quad d_r(n,k)=0 $ for $ n\le kr-1, \quad d_r(n,1)=(n-1)!$.

\Topic{Stirling numbers of $2^{nd}$ kind}.
$S_{n,k}$ is the number of ways to partition a set of $n$ elements into
exactly $k$ non-empty subsets.
$S_{n,k} = S_{n-1,k-1} + k S_{n-1,k}$.
$S_{n,1} = S_{n,n} = 1$.
$x^n = \sum_{k=0}^n S_{n,k}\,x^{\underline k}$

\Topic{Bell numbers}.
$B_n$ is the number of partitions of $n$ elements.
$B_0, \ldots = 1,1,2,5,15,52,203,\ldots$ \\
$B_{n+1} = \sum_{k=0}^n {n \choose k} B_k = \sum_{k=1}^n S_{n,k}$.
Bell triangle: $B_r=a_{r,1}=a_{r-1,r-1}$, $a_{r,c}=a_{r-1,c-1}+a_{r,c-1}$.
%Bell triangle: 1, 1 2, 2 3 5, 5 7 10 15, 15 20 27 37 52, (last) ... (left + left above).

\Topic{Bernoulli numbers}.
%GF: $\frac{x}{e^x - 1} = \sum_{n=0}^{\infty} B_n \frac{x^n}{n!}$. \quad
$\sum_{k=0}^{m-1} k^n =
\frac{1}{n+1} \sum_{k=0}^n {n+1 \choose k} B_k m^{n+1-k}$. \\
$\sum_{j=0}^m {m+1 \choose j} B_j = 0$.
\quad $B_0=1$, $B_1=-\frac{1}{2}$. $B_n=0$, for all odd $n \ne 1$.

%\Topic{Pentagonal theorem}.
%$\prod_{k=1}^{\infty} (1-x^k) = \sum_{n=-\infty}^{\infty} (-1)^n x^{n(3n+1)/2}$.
%$\sum_{n=0}^{\infty}p(n) x^n = \prod_{k=1}^{\infty} \left(\frac{1}{1-x^k}\right) = (\sum x^i)(\sum x^{2i})(\sum x^{3i}) \dots$ \\
%$p(0) = 1$, $p(n) - p(n-1) - p(n-2) + p(n-5) + p(n-7) - \dots = 0$ ($+,+,-,-$ goes on; same for $s(n)$, but $s(0)\to n$.)

\Topic{Eulerian numbers}.
$E(n,k)$ is the number of permutations with exactly
$k$ descents ($i: \pi_i < \pi_{i+1}$) /
ascents ($\pi_i > \pi_{i+1}$) /
excedances ($\pi_i > i$) /
$k+1$ weak excedances ($\pi_i \ge i$). \\
Formula: $E(n,k)=(k+1)E(n-1,k)+(n-k)E(n-1,k-1)$. \quad
$x^n = \sum_{k=0}^{n-1} E(n,k) {x+k \choose n}$.

\Topic{Burnside's lemma}.
The number of orbits under group $G$'s action on set $X$:\\
$|X/G| = \frac{1}{|G|} \sum_{g \in G} |X_g|$,
where $X_g=\{ x \in X: g(x)=x \}$. (``Average number of fixed points.'') \\
Let $w(x)$ be weight of $x$'s orbit. Sum of all orbits' weights:
$\sum_{o \in X/G} w(o) = \frac{1}{|G|} \sum_{g \in G} \sum_{x \in X_g} w(x)$.

%\Topic{Number of $k$-ary necklaces}.
%$\frac{1}{n}\sum_{d|n} \phi(\frac{n}{d}) k^d$. \\
%\Topic{Number of Lyndon words} (aperiodic necklaces).
%$\frac{1}{n}\sum_{d|n} \mu(\frac{n}{d}) k^d$.

%De Bruijn sequences over alphabet of size $k$, containing all $n$-substrings.
%Length: $n^k$. Number: ${k!}^{k^{n-1}} / k^n$.

% }}}

% Number Theory {{{
\Section{Number Theory}

\Topic{Linear diophantine equation}. $ax+by=c$.
Let $d=\gcd(a,b)$. A solution exists iff $d|c$.
If $(x_0,y_0)$ is any solution, then all solutions are given by
$(x,y) = (x_0 + \frac{b}{d}t, y_0 - \frac{a}{d}t)$, $t \in {\mathbb Z}$.
To find some solution $(x_0, y_0)$, use extended GCD to solve
$ax_0 + by_0 = d = \gcd(a, b)$, and multiply its solutions by $\frac{c}{d}$.

Linear diophantine equation in $n$ variables:
$a_1 x_1 + \dots + a_n x_n = c$ has solutions iff $\gcd(a_1, \dots, a_n) | c$.
To find some solution, let $b=\gcd(a_2, \dots, a_n)$,
solve $a_1 x_1 + by = c$, and iterate with $a_2 x_2 + \dots = y$.

\Topic{Extended GCD}
\begin{verbatim}
// Finds g = gcd(a,b) and x, y such that ax+by=g.
// Bounds: |x|<=b+1, |y|<=a+1.
void gcdext(int &g, int &x, int &y, int a, int b)
{ if (b == 0) { g = a; x = 1; y = 0; }
  else        { gcdext(g, y, x, b, a % b); y = y - (a / b) * x; } }
\end{verbatim}

Multiplicative inverse of $a$ modulo $m$:
$x$ in $ax + my = 1$, or $a^{\phi(m)-1} \pmod{m}$.

\Topic{Chinese Remainder Theorem}.
System $x \equiv a_i \pmod{m_i}$ for $i=1,\dots,n$, with
pairwise relatively-prime $m_i$ has a unique solution modulo $M = m_1 m_2 \dots m_n$:
$x = a_1 b_1 \frac{M}{m_1} + \dots + a_n b_n \frac{M}{m_n} \pmod{M}$,
where $b_i$ is modular inverse of $\frac{M}{m_i}$ modulo $m_i$.

%\Topic{Generalized CRT}.
System $x \equiv a \pmod{m}$, $x \equiv b \pmod{n}$ has solutions
iff $a \equiv b \pmod{g}$, where $g=\gcd(m,n)$.
The solution is unique modulo $L=\frac{mn}{g}$, and equals:
$x \equiv a + T(b-a) m/g \equiv b + S(a-b) n/g \pmod{L}$,
where $S$ and $T$ are integer solutions of $mT + nS = \gcd(m,n)$.

\Topic{Prime-counting function}. $\pi(n) = |\{p \le n : p \hbox{ is prime}\}|$.
$n/\ln(n) < \pi(n) < 1.3 n/\ln(n)$.
$\pi(1000) = 168$, $\pi(10^6) = 78498$, $\pi(10^9) = 50\ 847\ 534$.
\quad $n$-th prime $\approx n \ln n$.

\Topic{Miller-Rabin's primality test}.
Given $n = 2^r s + 1$ with odd $s$, and a random integer $1 < a < n$. \\
If $a^s \equiv 1 {\pmod n}$ or $a^{2^j s} \equiv -1 {\pmod n}$ for some
$0 \le j \le r-1$, then $n$ is a probable prime.
With bases 2, 7 and 61, the test indentifies all composites below $2^{32}$.
Probability of failure for a random $a$ is at most 1/4.

\Topic{Pollard-$\rho$}.
Choose random $x_1$, and let $x_{i+1} = x_i^2 - 1 \pmod{n}$.
Test $\gcd(n, x_{2^k+i} - x_{2^k})$ as possible $n$'s factors for $k=0,1,\ldots$
Expected time to find a factor: $O(\sqrt{m})$, where $m$ is smallest
prime power in $n$'s factorization.
That's $O(n^{1/4})$ if you check $n = p^k$ as a special case before factorization.

\Topic{Fermat primes}.  A Fermat prime is a prime of form $2^{2^n}+1$.
The only known Fermat primes are 3, 5, 17, 257, 65537.
A number of form $2^n+1$ is prime only if it is a Fermat prime.

%\Topic{Mersenne primes}.  A Mersenne prime is a prime of form $2^n-1$.
%Known Mersenne primes correspond to (necessarily prime) indexes
%2, 3, 5, 7, 13, 17, 19, 31, 61, 89, 107, 127, 521, 607, 1279,
%2203, 2281, 3217, 4253, 4423, 9689, 9941, 11213, 19937,
%21701, 23209, 44497, 86243, 110503, 132049, 216091,
%756839, 859433, 1257787, 1398269, 2976221, 3021377,
%6972593, 13466917.

\Topic{Perfect numbers}.  $n>1$ is called perfect if it equals
sum of its proper divisors and $1$.  Even $n$ is perfect iff $n = 2^{p-1} (2^p - 1)$
and $2^p - 1$ is prime (Mersenne's). No odd perfect numbers are yet found.

\Topic{Carmichael numbers}.
A positive composite $n$ is a Carmichael number
($a^{n-1} \equiv 1 \pmod{n}$ for all $a$: $\gcd(a,n)=1$),
iff $n$ is square-free, and for all prime divisors $p$ of $n$, $p-1$ divides $n-1$.

\Topic{Number/sum of divisors}.
$\tau(p_1^{a_1} \dots p_k^{a_k}) = \prod_{j=1}^k (a_j+1)$. \quad
$\sigma(p_1^{a_1} \dots p_k^{a_k}) = \prod_{j=1}^k \frac{p_j^{a_j+1}-1}{p_j-1}$.

\Topic{Euler's phi function}.
$\phi(n)=|\{m \in {\mathbb N}, m \le n, \gcd(m, n) = 1 \}|$. \\
$\phi(mn) = \frac{\phi(m) \phi(n) \gcd(m,n)}{\phi(\gcd(m,n))}$. \quad
$\phi(p^a) = p^{a-1} (p-1)$. \quad
$\sum_{d|n} \phi(d) = \sum_{d|n} \phi(\frac{n}{d}) = n$.

%\Topic{Fermat's theorem}.  $a^p \equiv a \pmod{p}$ if $p$ is prime. \\
%For any polynomial $f(x)$ with integer coefficients and prime $p$,
%$f(x)^{p^n} \equiv f(x^{p^n}) \pmod{p}$
\Topic{Euler's theorem}. $a^{\phi(n)} \equiv 1\pmod{n}$, if $\gcd(a,n)=1$. \\
\Topic{Wilson's theorem}. $p$ is prime iff $(p - 1)! \equiv -1 \pmod p$.

\Topic{Mobius function}.
$\mu(1) = 1$. $\mu(n) = 0$, if $n$ is not squarefree.
$\mu(n) = (-1)^s$, if $n$ is the product of $s$ distinct primes.
Let $f$, $F$ be functions on positive integers.
If for all $n \in N$, $F(n)=\sum_{d|n} f(d)$, then $f(n) = \sum_{d|n} \mu(d) F(\frac{n}{d})$,
and vice versa. \quad
$\phi(n) = \sum_{d|n} \mu(d) \frac{n}{d}$.
\quad $\sum_{d|n} \mu(d) = 1$. \\
If $f$ is multiplicative, then $\sum_{d|n} \mu(d) f(d) = \prod_{p|n}(1-f(p))$,
$\sum_{d|n} \mu(d)^2 f(d) = \prod_{p|n} (1+f(p))$.

\Topic{Legendre symbol}. If $p$ is an odd prime, $a \in {\mathbb Z}$, then
$\left(\frac{a}{p}\right)$ equals $0$, if $p | a$; $1$ if $a$ is a quadratic
residue modulo $p$; and $-1$ otherwise.
Euler's criterion:
$\left(\frac{a}{p}\right)=a^{\left(\frac{p-1}{2}\right)} \pmod p$. \\
%$\left(\frac{a}{p}\right) \left(\frac{b}{p}\right) = \left(\frac{ab}{p}\right)$
%Law of Quadratic Reciprocity: for any distinct odd primes $p$ and $q$,
%$\left(\frac{p}{q}\right) \left(\frac{q}{p}\right) = (-1)^{\frac{p-1}{2} \cdot \frac{q-1}{2}}$
\Topic{Jacobi symbol}.  %Generalization of Legendre's symbol to any odd modulus. \\
If $n=p_1^{a_1} \cdots p_k^{a_k}$ is odd, then
$\left(\frac{a}{n}\right) = \prod_{i=1}^k \left(\frac{a}{p_i}\right)^{k_i}$.

%\Topic{Kronecker symbol}.
%Let $a$ be a positive integer, which is not a perfect square and
%$a \equiv 0$ or $1 {\pmod 4}$. \\
%$\left(\frac{a}{2}\right) = \{ 1$, if $a \equiv 1 {\pmod 8}$;
%$-1$, if $a \equiv 5 {\pmod 8} \}$. \\
%$\left(\frac{a}{n}\right) = \prod_{j=1}^k p_j^{k_j}$,
%if gcd$(a,n) \ne 1$ and $n=\prod p_i^{k_i}$.
%$\left(\frac{a}{n}\right)$ equals Jacobi symbol otherwise.

\Topic{Primitive roots}.  If the order of $g$ modulo $m$ (min $n>0$:
$g^n \equiv 1 \pmod{m}$) is $\phi(m)$, then $g$ is called a primitive root.
If $Z_m$ has a primitive root, then it has $\phi(\phi(m))$ distinct primitive
roots. $Z_m$ has a primitive root iff $m$ is one of $2$, $4$,
$p^k$, $2p^k$, where $p$ is an odd prime.
If $Z_m$ has a primitive root $g$, then for all $a$ coprime to $m$,
there exists unique integer $i=\text{ind}_g(a)$ modulo $\phi(m)$,
such that $g^i \equiv a \pmod{m}$.
$\text{ind}_g(a)$ has logarithm-like properties:
$\text{ind}(1) = 0$, $\text{ind}(ab) = \text{ind}(a) + \text{ind}(b)$.

If $p$ is prime and $a$ is not divisible by $p$, then congruence
$x^n \equiv a \pmod{p}$ has $\gcd(n, p-1)$ solutions if
$a^{(p-1)/\gcd(n,p-1)} \equiv 1 \pmod{p}$, and no solutions otherwise.
(Proof sketch: let $g$ be a primitive root, and
$g^i \equiv a \pmod{p}$, $g^u \equiv x \pmod{p}$.
$x^n \equiv a \pmod{p}$ iff $g^{nu} \equiv g^i \pmod{p}$ iff $nu \equiv i \pmod{p}$.)

\Topic{Discrete logarithm problem}.  Find $x$ from $a^x \equiv b \pmod{m}$.
Can be solved in $O(\sqrt{m})$ time and space with a meet-in-the-middle trick.
Let $n = \lceil \sqrt{m} \rceil$, and $x = ny - z$.
Equation becomes $a^{ny} \equiv b a^z \pmod{m}$.  Precompute all values that
the RHS can take for $z = 0, 1, \dots, n-1$, and brute force $y$ on the LHS,
each time checking whether there's a corresponding value for RHS.

\Topic{Pythagorean triples}.  Integer solutions of $x^2 + y^2 = z^2$
All relatively prime triples are given by:
$x=2mn, y=m^2-n^2, z=m^2+n^2$ where $m>n, \gcd(m,n)=1$ and $m \not\equiv n \pmod{2}$.
All other triples are multiples of these.
Equation $x^2 + y^2 = 2z^2$ is equivalent to $(\frac{x+y}{2})^2 + (\frac{x-y}{2})^2 = z^2$.

\Topic{Postage stamps/McNuggets problem}.  Let $a$, $b$ be relatively-prime integers.
There are exactly $\frac{1}{2}(a-1)(b-1)$ numbers \emph{not} of form $ax+by$ ($x,y \ge 0$),
and the largest is $(a-1)(b-1)-1 = ab - a - b$.

\Topic{Fermat's two-squares theorem}.  Odd prime $p$ can be represented
as a sum of two squares iff $p \equiv 1 {\pmod 4}$.
A product of two sums of two squares is a sum of two squares.
Thus, $n$ is a sum of two squares iff every prime of
form $p=4k+3$ occurs an even number of times in $n$'s factorization.

\Topic{RSA}. Let $p$ and $q$ be random distinct large primes, $n = pq$.
Choose a small odd integer $e$, relatively prime to $\phi(n) = (p-1)(q-1)$,
and let $d = e^{-1} \pmod{\phi(n)}$. Pairs $(e,n)$ and $(d,n)$ are
the public and secret keys, respectively.
Encryption is done by raising a message $M \in Z_n$ to the power $e$ or $d$,
modulo $n$.

% }}}

% String Algorithms {{{

\Section{String Algorithms}

\Topic{Burrows-Wheeler inverse transform}.
Let $B[1..n]$ be the input (last column of sorted matrix of string's rotations.)
Get the first column, $A[1..n]$, by sorting $B$.
For each $k$-th occurence of a character $c$ at index $i$ in $A$,
let $next[i]$ be the index of corresponding $k$-th occurence of $c$ in $B$.
The $r$-th fow of the matrix is $A[r]$, $A[next[r]]$, $A[next[next[r]]]$, ...

\Topic{Huffman's algorithm}.  Start with a forest, consisting of isolated
vertices.  Repeatedly merge two trees with the lowest weights.
% }}}

% Graph Theory {{{
\Section{Graph Theory}

\Topic{Euler's theorem}. For any planar graph, $V - E + F = 1 + C$,
where $V$ is the number of graph's vertices, $E$ is the number of edges,
$F$ is the number of faces in graph's planar drawing, and $C$ is the number
of connected components.  Corollary: $V - E + F = 2$ for a 3D polyhedron.

%\Topic{Schlafli's}. A convex polyhedron in $R^n$, which has $N_0$ vertices, $N_1$ edges,
%$N_i$ $i$-dimensional faces, satisfies: $N_0-N_1+N_2-\dots=1-(-1)^n$.

\Topic{Vertex covers and independent sets}.
Let $M$, $C$, $I$ be a max matching, a min vertex cover, and a max independent set.
Then $|M| \le |C| = N - |I|$, with equality for bipartite graphs.
Complement of an MVC is always a MIS, and vice versa.
Given a bipartite graph with partitions $(A, B)$, build a network:
connect source to $A$, and $B$ to sink with edges of capacities, equal to
the corresponding nodes' weights, or $1$ in the unweighted case.
Set capacities of the original graph's edges to the infinity.
Let $(S,T)$ be a minimum $s$-$t$ cut.
Then a maximum(-weighted) independent set is $I = (A \cap S) \cup (B \cap T)$,
and a minimum(-weighted) vertex cover is $C = (A \cap T) \cup (B \cap S)$.

\Topic{Matrix-tree theorem}.
Let matrix $T = [t_{ij}]$, where $t_{ij}$ is the number of multiedges
between $i$ and $j$, for $i \ne j$, and $t_{ii} = -\mbox{deg}_i$.
Number of spanning trees of a graph is equal to the determinant of
a matrix obtained by deleting any $k$-th row and $k$-th column from $T$.
%If $G$ is a multigraph and $e$ is an edge of $G$, then the number $\tau(G)$ of
%spanning trees of $G$ satisfies recurrence $\tau(G) = \tau(G-e) + \tau(G/e)$,
%when $G-e$ is the multigraph obtained by deleting $e$, and $G/e$ is
%the contraction of $G$ by $e$ (multiple edges arising from the contraction
%are preserved.)

\Topic{Euler tours}.
Euler tour in an undirected graph exists iff the graph is connected and each
vertex has an even degree.  Euler tour in a directed graph exists iff in-degree
of each vertex equals its out-degree, and underlying undirected graph is connected.
%Open Euler path exists if it's possible to make the graph Eulerian by adding a single edge.
Construction:
\vspace{-5mm}
\begin{verbatim}
    doit(u):
      for each edge e = (u, v) in E, do: erase e, doit(v)
      prepend u to the list of vertices in the tour
\end{verbatim}
\vspace{-2mm}

\Topic{Stable marriages problem}.
While there is a free man $m$: let $w$ be the most-preferred woman to whom he
has not yet proposed, and propose $m$ to $w$. If $w$ is free, or is engaged to someone whom
she prefers less than $m$, match $m$ with $w$, else deny proposal.

\Topic{Stoer-Wagner's min-cut algorithm}.
Start from a set $A$ containing an arbitrary vertex.
While $A \ne V$, add to $A$ the most tightly connected vertex
($z \notin A$ such that $\sum_{x \in A} w(x, z)$ is maximized.)
Store cut-of-the-phase (the cut between the last added vertex and rest of
the graph), and merge the two vertices added last.  Repeat until the graph
is contracted to a single vertex.  Minimum cut is one of the cuts-of-the-phase.

\Topic{Tarjan's offline LCA algorithm}. (Based on DFS and union-find structure.)
\begin{verbatim}
    DFS(x):
      ancestor[Find(x)] = x
      for all children y of x:
         DFS(y); Union(x, y); ancestor[Find(x)] = x
      seen[x] = true
      for all queries {x, y}:
        if seen[y] then output "LCA(x, y) is ancestor[Find(y)]"
\end{verbatim}

\Topic{Strongly-connected components}. Kosaraju's algorithm. \\
1. Let $G^T$ be a transpose $G$ (graph with reversed edges.) \\
1. Call DFS($G^T$) to compute finishing times $f[u]$ for each vertex $u$. \\
3. For each vertex $u$, in the order of decreasing $f[u]$, perform DFS($G$, $u$). \\
4. Each tree in the 3rd step's DFS forest is a separate SCC.

\Topic{2-SAT}. Build an implication graph with 2 vertices for each
variable -- for the variable and its inverse; for each clause $x \lor y$
add edges $({\overline x}, y)$ and $({\overline y}, x)$.
The formula is satisfiable iff $x$ and ${\overline x}$ are in distinct SCCs,
for all $x$. To find a satisfiable assignment, consider the graph's SCCs
in topological order from sinks to sources (i.e. Kosaraju's last step), assigning `true' to
all variables of the current SCC (if it hasn't been previously
assigned `false'), and `false' to all inverses.

\Topic{Randomized algorithm for non-bipartite matching}.
Let $G$ be a simple undirected graph with even $|V(G)|$.
Build a matrix $A$, which for each edge $(u,v) \in E(G)$ has
$A_{i,j}=x_{i,j}$, $A_{j,i}=-x_{i,j}$, and is zero elsewhere.
Tutte's theorem: $G$ has a perfect matching iff $\det G$ (a multivariate
polynomial) is identically zero.
Testing the latter can be done by computing the determinant for
a few random values of $x_{i,j}$'s over some field.
(e.g. $Z_p$ for a sufficiently large prime $p$)

\Topic{Prufer code of a tree}.
Label vertices with integers $1$ to $n$.
Repeatedly remove the leaf with the smallest label, and output its only
neighbor's label, until only one edge remains. The sequence has
length $n-2$.  Two isomorphic trees have the same sequence, and every sequence
of integers from $1$ and $n$ corresponds to a tree.
Corollary: the number of labelled trees with $n$ vertices is $n^{n-2}$.  % Cayley's theorem

\Topic{Erdos-Gallai theorem}.
A sequence of integers $\{ d_1, d_2, \dots, d_n \}$,
with $n-1 \ge d_1 \ge d_2 \ge \dots \ge d_n \ge 0$ is a degree sequence
of some undirected simple graph iff $\sum d_i$ is even and
$d_1 + \dots + d_k \le k(k-1) + \sum_{i=k+1}^n \min(k, d_{i})$
for all $k=1,2,\dots,n-1$.

% }}}

% Games {{{

\Section{Games}

\Topic{Grundy numbers}.
For a two-player, normal-play (last to move wins) game on a graph $(V,E)$:
$G(x) = \mbox{mex}(\{ G(y) : (x, y) \in E \})$,
where $\mbox{mex}(S) = \min \{ n \ge 0: n \not\in S \}$.
$x$ is losing iff $G(x) = 0$.

\Topic{Sums of games}.

\vspace{-4mm}
\begin{itemize}
  \item
    \emph{Player chooses a game and makes a move in it}.
    Grundy number of a position is xor of grundy numbers of positions in summed games.
  \item
    \emph{Player chooses a non-empty subset of games (possibly, all) and makes moves in all of them}.
    A position is losing iff each game is in a losing position.
  \item
    \emph{Player chooses a proper subset of games (not empty and not all),
        and makes moves in all chosen ones.}
    A position is losing iff grundy numbers of all games are equal.
  \item
    \emph{Player must move in all games, and loses if can't move in some game}.
    A position is losing if any of the games is in a losing position.
\end{itemize}
% http://www.topcoder.com/tc?module=Static&d1=tutorials&d2=algorithmGames

\vspace{-3mm}

\Topic{Mis\`{e}re Nim}.
A position with pile sizes $a_1, a_2, \dots, a_n \ge 1$,
not all equal to $1$, is losing iff $a_1 \oplus a_2 \oplus \dots \oplus a_n = 0$
(like in normal nim.)
A position with $n$ piles of size $1$ is losing iff $n$ is \emph{odd}.

% }}}

% Bit Tricks {{{

\Section{Bit tricks}
Clearing the lowest 1 bit: \verb$x & (x - 1)$, all trailing 1's: \verb$x & (x + 1)$ \\
Setting the lowest 0 bit: \verb$x | (x + 1)$ \\
Enumerating subsets of a bitmask $m$: \\
\verb|x=0; do { ...; x=(x+1+~m)&m; } while (x!=0);| \\
\verb$__builtin_ctz/__builtin_clz$ returns the number of trailing/leading zero bits. \\
\verb$__builtin_popcount(unsigned x)$ counts 1-bits (slower than table lookups). \\
For 64-bit unsigned integer type, use the suffix `\verb$ll$', i.e. \verb$__builtin_popcountll$.

% }}}

% Math {{{
\Section{Math}

\Topic{Stirling's approximation}
$z! = \Gamma(z+1) = \sqrt{2 \pi}\ z^{z+1/2}\ e^{-z}
(1 + \frac{1}{12z} + \frac{1}{288 z^2} - \frac{139}{51840 z^3} + \dots)$
%$\ln \Gamma(z) = \frac{1}{2} \ln(2 \pi) + (z - \frac{1}{2}) \ln z - z + \sum_{n=1}^{\infty} \frac{B_{2n}}{2n(2n-1)} z^{-(2n-1)}$.
%$\sum = \frac{1}{12 z} - \frac{1}{360 z^3} + \frac{1}{1260 z^5} - \dots$

\Topic{Taylor series}.
$f(x) = f(a) + \frac{x-a}{1!} f'(a) + \frac{(x-a)^2}{2!} f^{(2)}(a) + \dots + \frac{(x-a)^n}{n!} f^{(n)}(a) + \dots$. \\
$\sin x = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \frac{x^7}{7!} + \dots$ \\
$\ln x = 2(a+\frac{a^3}{3}+\frac{a^5}{5}+\dots)$, where $a=\frac{x-1}{x+1}$. $\ln x^2 = 2 \ln x$. \\
$\arctan x = x - \frac{x^3}{3} + \frac{x^5}{5} - \frac{x^7}{7} + \dots$,
$\arctan x = \arctan c + \arctan \frac{x-c}{1+xc}$ (e.g c=.2) \\
$\pi = 4 \arctan 1$, $\pi = 6 \arcsin \frac{1}{2}$

% }}}

\Topic{List of Primes}

\begin{tabular}{l l l l l l l l l l l l}
    1e5 & 3 & 19 & 43 & 49 & 57 & 69 & 103 & 109 & 129 & 151 & 153 \\
    1e6 & 33 & 37 & 39 & 81 & 99 & 117 & 121 & 133 & 171 & 183 \\
    1e7 & 19 & 79 & 103 & 121 & 139 & 141 & 169 & 189 & 223 & 229 \\
    1e8 & 7 & 39 & 49 & 73 & 81 & 123 & 127 & 183 & 213 \\
\end{tabular}


\section{The Dispersion Counterterm}
\label{sec:dispersion-counterterm}

To evaluate the infinite-volume integral in \eqref{dispersion-counter-integral}, we rescale the $n$ integration to extract $N$ out of the integral and rescale  $x \to \xtilde = 4 \pi^2  x/(N/2)^2$
\begin{align}\label{eq:rescaled-counterterm-integral}
    \int_{-N/2}^{+N/2} \mathrm{d}^Dn\; \PV \frac{1}{\tilde K_{nn}^N - x}
    =
    4\pi^2 \left(\frac{N}{2}\right)^{D-2} \int_{-1}^{+1} \mathrm{d}^D\nu\; \PV \frac{1}{4\sum_{ds} \gamma^{(\nstep)}_s \cos \pi \nu s - \xtilde}
    \, ,
\end{align}
which is well defined for any dimension if $\xtilde\neq0$ and for $\xtilde=0$ if $D>2$.
For $\xtilde\leq0$ the sum over dimensions can be isolated by introducing another integral
\begin{equation}
    \label{eq:dispersion counterterm}
    \eqref{rescaled-counterterm-integral}
    =
    4 \pi^2 \left(\frac{N}{2}\right)^{D-2}\int_{0}^{\infty} 2y\; \mathrm{d}y\; e^{\xtilde y^2}\left(\int_{-1}^{+1} \mathrm{d}\nu\; e^{-4y^2 \sum_s \gamma_s^{(\nstep)} \cos \pi \nu s}\right)^D
\end{equation}
which can be numerically evaluated.
This trick relies on the Laplacian stencil not coupling momenta in different directions \eqref{kinetic}.

The counterterm for the leading divergence $\counterterm_D^{\dispersion}$ is the $\xtilde=0$ value.
For three dimensions, we show this counterterm and how it differs from the $\nstep \to \infty$ counterterm in \Figref{nstep counterterm} and provide precise values in table \tabref{diserpersion-zeta-3d-counterterm-counterterm}.
\begin{table}[htb]
    \input{section/counterterm/diserpersion-zeta-3d-counterterm-table.tex}
    \caption{
    	\label{tab:diserpersion-zeta-3d-counterterm-counterterm}
		Counter term coefficients for the three-dimensional dispersion zeta function defined in \eqref{dispersion-zeta-contact}.
    }
\end{table}

\begin{figure}[htb]
    \scalebox{0.9}{\input{figure/counterterm-nstep.pgf}}
    \caption{
    	In the top panel we show the dispersion counterterm $\counterterm^{\dispersion}_{3}$ in \eqref{dispersion-zeta-contact} as a function of $\nstep$, and the $\nstep=\infty$ result, as a dashed line.
	In the bottom panel we show an alternate view into how the counterterm converges to the $\nstep=\infty$ value.
    }
    \label{fig:nstep counterterm}
\end{figure}

If we assume $n_{s}=\infty$ we can obtain analytic solutions when $\tilde x=0$.  For $D=3$ we find
\begin{multline}
\counterterm_3^{\dispersion}=-8 G-4 i \left\{2 \text{Li}_2\left(1-\sqrt[4]{-1}\right)-2 \text{Li}_2\left(1+(-1)^{3/4}\right)+\text{Li}_2\left(3 i-2 i
   \sqrt{2}\right)-2 \text{Li}_2\left(\frac{1}{2} \left((-1-i)+\sqrt{2}\right)\right)\right.\\
   \left.+2 \text{Li}_2\left(\frac{1}{2}
   \left((-1+i)+\sqrt{2}\right)\right)-2 \text{Li}_2\left(\frac{2}{(1-i)+\sqrt{2}}\right)+2
   \text{Li}_2\left(\frac{2}{(1+i)+\sqrt{2}}\right)+2 \text{Li}_2\left(\frac{2 i}{(1+3 i)+(1+2 i)
   \sqrt{2}}\right)\right.\\
   \left.-\text{Li}_2\left(i \left(-3+2 \sqrt{2}\right)\right)-2 \text{Li}_2\left(\frac{2}{(3+i)+(2+i)
   \sqrt{2}}\right)\right\}+\pi  \log \left(7880+5572 \sqrt{2}\right)\ ,
   \end{multline}
where $G$ is Catalan's constant and $\text{Li}_2$ is a polylogarithm of order 2.  For $D=2$ the dominant $N$ part of~\eqref{rescaled-counterterm-integral}, after subtracting off the logarithmic singularity in $\sqrt{x}$, is logarithmic,
\begin{equation}
 \int_{-N/2}^{+N/2} \mathrm{d}^2n\; \PV \frac{1}{\bm n^2 - x}-2\pi\log\left(\sqrt{x}\right)= 2\pi\log\left(\frac{N}{2}\right)-4\left(G-\frac{\pi }{2}\log(2)\right)+\mathcal{O}(N^{-1})=2\pi\log\left(\counterterm_2^{\dispersion}\frac{N}{2}\right)+\mathcal{O}(N^{-1})\ ,
\end{equation}
with
\begin{equation}
\counterterm_2^\dispersion=\exp\left(\log(2)-G\frac{2}{\pi}\right)\ .
\end{equation}


\subsection{Differentiable}
Now that we have introduced the derivative
of a function at a point, we can begin to use the adjective \dfont{differentiable}.

\begin{definition}{Differentiable at a Point}{DifferentiablePoint}
A function $f$ is
differentiable at point $a$ if $f'(a)$ exists.
\end{definition}

\begin{definition}{Differentiable on an Interval}{Differentiable}
A function $f$ is differentiable on an open interval if it is differentiable at every point in the interval.
\end{definition}

Sometimes one encounters a point in the domain of a function $y=f(x)$ where
there is {\bf no derivative}, because there is no tangent line.  In order
for the notion of the tangent line at a point to make sense, the curve must
be ``smooth'' at that point.  This means that if you imagine a particle
traveling at some steady speed along the curve, then the particle does not
experience an abrupt change of direction.  There are two types of
situations you should be aware of---corners and cusps---where there's a
sudden change of direction and hence no derivative.

\begin{example}{Derivative of the Absolute Value}{DerivativeAbsoluteValue}
Discuss the derivative of the absolute value function $y=f(x)=|x|$.
\end{example}

\begin{solution}
If $x$ is positive, then this is the function $y=x$, whose derivative is
the constant 1.  (Recall that when $y=f(x)=mx+b$, the derivative is the
slope $m$.)  If $x$ is negative, then we're dealing with the function $y=-x$,
whose derivative is the constant $-1$.  If $x=0$, then the function has
a corner, i.e., there is no tangent line.  A tangent line
would have to point in the direction of the curve---but there are {\it
two} directions of the curve that come together at the origin.

$$\includegraphics[width=2.5in]{images/absvalue}$$

We can summarize this as
$$
y'=\left\{\begin{array}{rl}
1, & \mbox{if $x>0$,}\\
-1, & \mbox{if $x<0$,}\\
\hbox{undefined,} & \mbox{if $x=0$.}\\
\end{array}\right.
$$
In particular, the absolute value function $f(x)=|x|$ is \ifont{not} differentiable at $x=0$.
\end{solution}

We note that the following theorem can be proved using limits.

\begin{theorem}{Differentiable implies Continuity}{DifferentiableImpliesContinuity}
If $f$ is differentiable at $a$, then $f$ is continuous at $a$.
\end{theorem}
\begin{proof}
Suppose that $f$ is differentiable at $a.$ That is,%
\begin{equation*}
	f^{\prime }\left( a\right) =\lim_{h\rightarrow 0}\frac{f\left( a+h\right)
		-f\left( a\right) }{h}
\end{equation*}%
exists. At this stage, we find it convenient to write this limit in an
alternative form so that its connection with continuity can become more
easily seen. If we let $x=a+h,$ then $h=x-a.$ Furthermore, $h\rightarrow 0$
is equivalent to $x\rightarrow a.$ So,%
\begin{equation*}
	f^{\prime }\left( a\right) =\lim_{x\rightarrow a}\frac{f\left( x\right)
		-f\left( a\right) }{x-a}.
\end{equation*}%
(This alternative formulation of the derivative is also standard. We will
use it whenever we find it convenient to do so. You should get familiar with
it.) Continuity at $a$ can now be proved as follows:%
\begin{eqnarray*}
	\lim_{x\rightarrow a}f\left( x\right) &=&\lim_{x\rightarrow a}\left( \frac{%
		f\left( x\right) -f\left( a\right) }{x-a}\cdot \left( x-a\right) +f\left(
	a\right) \right) \\
	&=&\lim_{x\rightarrow a}\frac{f\left( x\right) -f\left( a\right) }{x-a}\cdot
	\lim_{x\rightarrow a}\left( x-a\right) +\lim_{x\rightarrow a}f\left( a\right)
	\\
	&=&f^{\prime }\left( a\right) \cdot \left( a-a\right) +f\left( a\right) \\
	&=&f\left( a\right).
\end{eqnarray*}
\end{proof}

However, if $f$ is continuous at $a$ it is \ifont{not} necessarily
true that $f$ is differentiable at $a$.  For example, it was shown
that $f(x)=|x|$ is not differentiable at $x=0$ in the previous
example, however, one can observe that $f(x)=|x|$ is continuous
everywhere.

\begin{example}{Derivative of $\ds y=x^{2/3}$}{Derivative}
Discuss the derivative of the function $\ds y=x^{2/3}$, shown in
Figure~\ref{fig:cusp}.
\end{example}

\begin{solution}
We will later see how to compute this
derivative; for now we use the fact that $\ds
y'=(2/3)x^{-1/3}$. Visually this looks much like the absolute value
function, but it technically has a cusp, not a corner. The absolute
value function has no tangent line at 0 because there are (at least)
two obvious contenders---the tangent line of the left side of the
curve and the tangent line of the right side.
The function $\ds y=x^{2/3}$ does not have a tangent line at 0, but
unlike the absolute value function it can be said to have a single
direction: as we approach 0 from either side the tangent line becomes
closer and closer to a vertical line; the curve is vertical at 0. But
as before, if you imagine traveling along the curve, an abrupt change
in direction is required at 0: a full 180 degree turn.
\end{solution}

\figure
\centerline{\vbox{\beginpicture
\normalgraphs
%\ninepoint
\setcoordinatesystem units <2truecm,2truecm>
\setplotarea x from -2 to 2, y from 0 to 1.6
\axis left shiftedto x=0 ticks numbered from 0 to 1 by 1 /
\axis bottom ticks numbered from -2 to 2 by 1 /
\setquadratic
\plot -2.000 1.587 -1.933 1.552 -1.867 1.516 -1.800 1.480 -1.733 1.443
-1.667 1.406 -1.600 1.368 -1.533 1.330 -1.467 1.291 -1.400 1.251
-1.333 1.211 -1.267 1.171 -1.200 1.129 -1.133 1.087 -1.067 1.044
-1.000 1.000 -0.933 0.955 -0.867 0.909 -0.800 0.862 -0.733 0.813
-0.667 0.763 -0.600 0.711 -0.533 0.658 -0.467 0.602 -0.400 0.543
-0.333 0.481 -0.267 0.414 -0.200 0.342 -0.133 0.261 -0.067 0.164
0.000 0.000 0.067 0.164 0.133 0.261 0.200 0.342 0.267 0.414
0.333 0.481 0.400 0.543 0.467 0.602 0.533 0.658 0.600 0.711
0.667 0.763 0.733 0.813 0.800 0.862 0.867 0.909 0.933 0.955
1.000 1.000 1.067 1.044 1.133 1.087 1.200 1.129 1.267 1.171
1.333 1.211 1.400 1.251 1.467 1.291 1.533 1.330 1.600 1.368
1.667 1.406 1.733 1.443 1.800 1.480 1.867 1.516 1.933 1.552
2.000 1.587   /
\endpicture}}
\caption{A cusp on $\ds x^{2/3}$. \label{fig:cusp}}
\endfigure

In practice we won't worry much about the distinction between these
examples; in both cases the function has a ``sharp point'' where there
is no tangent line and no derivative.


% -*- root: 00-main.tex -*-
\onecolumn
\appendix

\section{Proof of duality between the probabilistic and the presented frameworks}

In the probabilistic approach to image segmentation, the smoothness constraints
can be introduced into the \gls{map} criterion as a \gls{mrf}. Accordingly to the
Hammersley-Clifford theorem (\textbf{FIXME}: citations needed), an \gls{mrf} can
be equivalently characterized by a Gibb's distribution:
\begin{equation}
P(\mathbf{K})=\underset{i}{\prod} Z(U)^{-1}\,e^{-U(\mathbf{x}_i \mid \beta,K)} = \mathit{const.}\,e^{- \sum\limits_i \left( U(\mathbf{x}_i \mid \beta,K) \right) } \sim e^{-\nu_B \left| K \right| }.
\end{equation}
This way, we include the assumption that the total length of the edge set $K$ is small
and we draw the equivalence with the \gls{mrf} modeling, for $U(\mathbf{x}_i \mid \beta,K)$
being the Potts model.

In the probabilistic approach, the underlying concept to segmenting an image is the
Bayes' rule. For a discrete image of $N$ \revcomment[R\#3-C10]{voxel}s indexed by $i$, this rule reads as
follows:
\begin{equation}
P(K|I)=P(I \mid K)\, P(K)=\underset{i}{\prod} p (\mathbf{x}_i | K ) \, p_i(K).
\end{equation}
The normalizer probability $P(I)$ has been omitted for simplicity.

In order to express the likelihood as an absolute energy in our variation framework,
the \textit{log}-likelihood is computed:
\begin{equation}
E(K)= -\log( P(K|I) ) = \sum\limits_k \int_{\Omega_k} -\log p_k(I(\mathbf{x}),\mathbf{x}) \,d\mathbf{x}+\nu_B \left|K\right|,
\end{equation}
where $\mathbf{s} = I(\mathbf{x})$, $\mathbf{s} \in \mathbb{R}^C$ (a vector of $C$ scalar features).
Additionally, the discrete grid of $N$ \revcomment[R\#3-C10]{voxel}s has been converted to a continuous space by assuming
$d\mathbf{x}$ as infinitesimal bin size.

To explicitly define the likelihood, we firstly define the squared \textit{Mahalanobis distance} that
is the exponential of a multivariate normal distribution:
\begin{equation}
\Delta^2_k (\mathbf{s}) = (\mathbf{s} - \boldsymbol{\mu}_k)^T \, \Sigma^{-1}_k \, (\mathbf{s} - \boldsymbol{\mu}_k).
\end{equation}

Therefore, the likelihood is defined as follows:
\begin{equation}
p_k(I(\mathbf{x}),\mathbf{x}) = p_k(\mathbf{s},\mathbf{x})= p_k(\mathbf{s}) = \frac{1}{ \sqrt{(2\pi)^{C}\,\left|\boldsymbol{\Sigma}_{k}\right|}}\,{e^{\left(-\frac{1}{2}  \Delta^2_k (\mathbf{s}) \right)}}.
\end{equation}

Introducing this definition on (14), we have:
\begin{equation}
E(K)= \sum\limits_k \int_{\Omega_k} -\log{\left[ \frac{1}{ \sqrt{(2\pi)^{C}\,\left|\boldsymbol{\Sigma}_{k}\right|}}\,e^{\left(-\frac{1}{2}  \Delta^2_k (\mathbf{s}) \right)} \right] } \,d\mathbf{x}+\nu_B \left|K\right|.
\end{equation}

\begin{equation}
E(K) = \sum\limits_k \int_{\Omega_k} \left( \frac{1}{2} \log{ \left( (2\pi)^{C}\,\left|\boldsymbol{\Sigma}_{k}\right| \right)} + \frac{1}{2}  \Delta^2_k (\mathbf{s}) \right) \,d\mathbf{x}+\nu_B \left|K\right|
\end{equation}

\begin{equation}
E(K) = \sum\limits_k \left( \frac{ V_k }{2} \log{ \left( (2\pi)^{C}\,\left|\boldsymbol{\Sigma}_{k}\right| \right)}+ \frac{1}{2} \int_{\Omega_k} \Delta^2_k (\mathbf{s}) \,d\mathbf{x}+\nu_B \left|K\right| \right)
\end{equation}

{\color{red} {Using the region descriptors derived in \autoref{sec:regseg-methods_map}, we propose
an \gls{adf}-like, \gls{acwe}-based, piece-wise constant, image segmentation
model (where the unknown is the deformation field)
\cite{chan_active_2001} with the energy functional obtained in
\eqref{eq:regseg-final_map_energy}. This inverse problem is ill-posed
\cite{bertero_illposed_1988,hadamard_sur_1902}.
In order to account for deformation field regularity and to render the
problem well-posed, we include limiting and regularization terms into
the energy functional \cite{morozov_linear_1975,tichonov_solution_1963}:
%
\begin{multline}
E(u) = \sum\limits_l \int_{\Omega'_l} \mdist{f}{l} \,d\vec{x} \\
+ \int \vec{u}^T \, A \, \vec{u} \, d\vec{x} + \int \tr\{(\nabla \vec{u}^T)^T B (\nabla \vec{u}^T)\} d\vec{x}
\label{eq:regseg-complete_energy}
\end{multline}
%
These regularity terms ensure that the segmenting contours in
\gls{dwi} space are still close to their native shape. The model
easily allows to incorporate inhomogeneous and anisotropic
regularization \cite{nagel_investigation_1986} to better regularize
the \gls{epi} distortion.}}

\todo[inline]{this last paragraphs in red color were located a bit
further, but I think we can keep some of the refs and still reproduce
the continuous version of the energy functional}

\section{Extensions}

\subsubsection{Semi-implicit Euler-forward optimization}
It is necessary to discretize $t$ in order to obtain a numerical
implementation of the equation \eqref{eq:regseg-gradient_final}:
\begin{align}
\frac{\vec{u}_k^{t+1}-\vec{u}_k^t}{\tau} =
&- \underset{l,m}{\sum} \underset{i}{\sum}
\left[ \mdist{f_i'}{l} - \mdist{f_i'}{m}\right]
\psi_k(\vec{c}_i)\, \hat{\vec{n}}_i \notag\\
&+2\, \boldsymbol{\alpha} \vec{u}_k^{t+1}
-2\, \boldsymbol{\beta} \Delta \vec{u}_k^{t+1}.
\end{align}

The associated Euler-Lagrange equation is found as:
\begin{equation}
(\tau^{-1} +2\, \boldsymbol{\alpha} - 2\, \boldsymbol{\beta} \Delta )\, \vec{u}_k^{t+1} =
\tau^{-1} \vec{u}_k^t - \frac{\partial}{\partial \vec{u}_k} E_{data}(\vec{u}_k^t),
\end{equation}
that is a linear system that we translate into Fourier domain,
to obtain the next deformation field $\vec{u}_k^{t+1}$:
\begin{multline}
\vec{u}_k^{t+1} = \\
 \mathcal{F}^{-1} \left\lbrace
\frac{\mathcal{F}\lbrace \tau^{-1}\vec{u}_{k}^{t} - \underset{l,m}{\sum} \underset{i}{\sum}
\left[ \mdist{f_i'}{l} - \mdist{f_i'}{m}\right] \rbrace}
     {\mathcal{F}\lbrace (\tau^{-1} +2\,\boldsymbol{\alpha})\mathcal{I} - 2\,\boldsymbol{\beta} \Delta ) \rbrace}
     \right\rbrace
\end{multline}

Here, we rewrite the Laplacian as a linear combination of the identity and shift operators:
\begin{equation}
\Delta = \sum\limits_d \mathcal{S}_d^- + \mathcal{S}_d^+ - 2 \mathcal{I}
\end{equation}
where $\mathcal{S}_{d}^{\pm}$ stands for the forward ($+$) and backward ($-$) shift
operator along coordinates axis $d$, of which the Fourier transform is found easily as
\begin{equation}
\mathcal{F}\{\mathcal{S}_{d}^{\pm}\} = e^{\pm \vec{i}\omega_{d}},
\end{equation}
where $\omega_{d}$ is the normalized pulsation along $d$-direction. Accordingly, the
Fourier transform of the discrete Laplacian is found as
\begin{equation}
\mathcal{F}\{\Delta\} = \sum\limits_d e^{-\vec{i}\omega_d } + e^{\vec{i}\omega_d } - 2 = \sum\limits_d \left( 2\cos(\omega_d) - 2 \right)
\end{equation}

The remaining transforms are trivial or can be computed using the \gls{fft}
as in \citep{estellers_efficient_2011}.


\section{Continuity Definition}
When we were looking at limits, we noticed that we can't always substitute to find the limit, even if the function is defined there.
In the example given to show that substitution and the limit can give different results, we saw a special type of function that seemed to have a ``hole'' at the point we were interested in finding the limit of.
This function is said to be discontinuous at this point, and in this section we'll define when a function is or isn't continuous at a point based on this idea of the limit and substitution giving different values.

\begin{definition}
	Let $f(x)$ be a real-valued function defined over $D \subseteq \R$.
	$f(x)$ is continuous at some point $x = c$ if all of the following hold.
	\begin{enumerate}
		\item $\lim_{x \to c}{f(x)}$ exists
		\item $f(c)$ is defined
		\item $\lim_{x \to c}{f(x)} = f(c)$ (substitution works)
	\end{enumerate}
	Otherwise, $f(x)$ is discontinuous at $c$.\footnote{Note that it's not necessary for $c \in D$.}
\end{definition}

We say that a function is continuous on an interval if it's continuous on every point in that interval.\footnote{If the interval is closed on one or both sides, we check continuity on the open interval. Then, we check the closed endpoints by looking at the limit from only one side.}

\begin{example}
	Find the points of continuity and discontinuity of the following functions
	\begin{table}[H]
	\begin{center}
	\begin{tabular}{ l l }
		1. $\begin{aligned}
			f(x) = \frac{1}{x^2+1}
		\end{aligned}$ &
		2. $\begin{aligned}
			g(x) = e^{1/x}
		\end{aligned}$
	\end{tabular}
	\end{center}
	\end{table}
\end{example}
\begin{answer}
	\begin{enumerate}
		\item There are no points where $f(x)$ or its limit are undefined.
			Further, there are no points where $f$ and its limit at that point are different.
			So, $f$ is continuous on $(-\infty, \infty)$ and discontinuous on $\emptyset$.
		\item Since $1/x$ is undefined at $x = 0$, $g(x)$ is also undefined at $x=0$.
			At every other point, $g$ and its limit are defined and are equal.
			So, $g$ is continuous on $(\infty, 0) \cup (0, \infty)$ and discontinuous on $[0]$.
	\end{enumerate}
\end{answer}


\section{Markov chain VS Simulation}

\subsection{Example model}
Consider the Markov chain paradigm in figure \ref{Model_mini}.
The illustrated model represents the unrealistically small system
with a system capacity of five and a buffer capacity of three.
The hospital in this particular example also has four servers and a threshold of
three; meaning that every ambulance that arrives in a time that there are three or
more individuals in the hospital, will proceed to the buffer centre.

\begin{figure}[h]
    \centering
    \input{Comparisons/Example_model/Tikz_code/main.tex}
    \caption{Markov chains: number of servers=4}
    \label{Model_mini}
\end{figure}

In addition to the Markov chain model a simulation model has also been built based
on the same parameters.
Comparing the results of the Markov model and the equivalent simulation model the
resultant plots arose.

The heatmaps in figure \ref{Heatmap_mini} represent the state probabilities for
the Markov chain model, the simulation model and the difference between the two.
Each pixel of the heatmap corresponds to the equivalent state of figure \ref{Model_mini}
and represents the probability of being at that state in any particular moment of time.

It can be observed that both Markov chain and simulation models' state probabilities
vary from 5\% to 25\% and that states \( (0, 1) \) and \( (0, 2) \) are the most
visited ones.
Looking at the differences' heatmap, one may identify that the differences between
the two are minimal.

\newpage

\begin{figure}[h]
    \includegraphics[width=\linewidth]{Comparisons/Example_model/Heatmap/main.pdf}
    \caption{Heatmaps of Simulation, Markov chains and differences of the two}
    \label{Heatmap_mini}
\end{figure}


\section{Simplicial Complexes}
Our next family of algebraic invariants is called homology, which are hard to define but easy to calculate when defined.
But first, let us talk about simplicial complexes.
\subsection{Definitions}
\begin{definition}
    A finite set $V=\{v_0,\ldots,v_n\}\subset\mathbb R^m$ is said to be in general position if the smallest affine linear subspace (translation of a linear subspace) of $\mathbb R^m$ containing $V$ has dimension $n$.
\end{definition}
Consequently, no set of $n$ points in $\mathbb R^m$ would be in general position if $n>m$.
Another way of defining this is that $\{v_1-v_0,\ldots,v_n-v_0\}$ are linearly independent.
\begin{definition}
    The span or a convex hull of a set $V=\{v_0,\ldots,v_n\}\subset\mathbb R^m$ is
    $$\langle V\rangle=\left\{ \sum_{i=0}^nt_iv_i:\sum_{i=0}^nt_i=1,t_i\ge 0\right\}$$
    If $V$ is in general position, then $\langle V\rangle$ is said to be an $n$-simplex.
\end{definition}
Sometimes we omit the set bracket and just write $\langle v_0,\ldots,v_n\rangle$ as $\langle V\rangle=\langle \{v_0,\ldots,v_n\}\rangle$.
So a $0$-simplex is just a point, a $1$-simplex is a line segment, a $2$-simplex is a (solid) triangle and a $3$-simplex is a (solid) tetrahedron.
\begin{definition}
    If $V\subset\mathbb R^n$ is in general position and $U\subset V$, then $\langle U\rangle\subset\langle V\rangle$ is said to be a face of $V$, and we write $\langle U\rangle\subset\langle V\rangle$.
    Furthermore, if $U\le V$ and $U\neq V$, then we call $\langle U\rangle$ a proper face of $\langle V\rangle$.
\end{definition}
The empty set is viewed as a face by convention.
\begin{definition}
    A simplicial complex is a finite set $K$ of simplices in $\mathbb R^m$ such that:\\
    1. If $\sigma\in K$ and $\tau\le\sigma$, then $\tau\in K$.\\
    2. If $\sigma,\tau\in K$, then $\sigma\cap\tau$ is a face of both $\sigma$ and $\tau$.
\end{definition}
\begin{example}
    Just glue (well, not really, but you know what I mean) line segments, triangles, tetrahdrons together along the faces (aka edges, endpoints, etc.).
    But if we take a triangle (and its faces) and a line segment (ditto) pointing into its interior, then we do not get a simplicial complex.
\end{example}
\begin{definition}
    For a simplicial complex $K$, the dimension $\dim K$ of $K$ is the largest $n$ such that $K$ contains a $n$-simplex.
    We write $K_{(d)}=\{\sigma\in K:\dim\sigma\le d\}$ as the $d$-skeleton of $K$, which is also a simplicial complex.
\end{definition}
\begin{example}
    The set $F$ of faces of an $n$-simplex $\sigma$ is a simplicial complex, and the set of proper faces (which is called the boundary $\partial\sigma$ of $\sigma$) of $\sigma$ is also a simplicial complex and is the $(n-1)$ skeleton of $F$.
    The interior $\sigma^\circ$ is the points in $\sigma$ that is not contained in any simplex in $\partial\sigma$.
\end{example}
\begin{definition}
    The realisation of a simplicial complex $K$ is
    $$|K|=\bigcup_{\sigma\in K}\sigma\subset\mathbb R^m$$
    viewed as a topological space with topology induced as a subset of $\mathbb R^m$.\\
    If $X$ is a space, then a triangulation of $X$ is a simplicial complex $K$ and a homeomorphism $|K|\to X$.
    If a triangulation of $X$ exists, then $X$ is said to be triangulable.
\end{definition}
\begin{example}
    $D^n$ is triangulable as it is homeomorphic to the $n$-simplex.
    The realisation of the boundary of the $n$-simplex is then homeomorphic to the boundary of $D^n$ which is $S^{n-1}$, so the $n$-spheres are also triangulable.
\end{example}
\begin{remark}
    There might be many different triangulations of the same space.
\end{remark}
\begin{definition}
    Let $K,L$ be simplicial complexes.
    A simplicial map is a map $f:K\to L$ such that:
    1. Each $0$-simplex $\langle v\rangle\in K$ is sent to a $0$-simplex $\langle f(v)\rangle$ of $L$.\\
    2. For any $\langle v_0,\ldots,v_n\rangle\in K$, we have $f(\langle v_0,\ldots,v_n\rangle)=\langle f(v_0),\ldots,f(v_n)\rangle$.\\
    The realisation of $f$ is the map $|f|$ defined in the following way:
    On each $\sigma=\langle v_0,\ldots,v_n\rangle\in K$, we want the restriction of $f$ to be
    $$f|_\sigma=f_\sigma:\sum_{i=0}^nt_iv_i\mapsto\sum_{i=0}^nt_if(v_i)$$
    which is obviously continuous and consistent as $f_\sigma|_\tau=f|_\tau$ for $\tau\le\sigma$.
    So we can glue them together to get the continuous map $|f|:|K|\to |L|$.
\end{definition}
However, there is simply not enough simplicial maps to capture all possible maps $|K|\to|L|$ up to homotopy.
\begin{example}
    Consider the triangulation of the circle by gluing three line segments together (which makes it the realisation of $\partial\sigma_2$, the boundary of the $2$-simplex).
    There are at most $3^3$ simplicial maps from $\partial\sigma_2$ to itself but infinitely many different homeomorphisms (hence homotopy equivalences) from the circle to itself.
\end{example}
The solution to this problem is to introduce the barycentric subdivision to refine triangulations.
\subsection{Barycentric Subdivision and Simplicial Approximation}
\begin{definition}
    Let $V=\{v_0,\ldots, v_n\}$ be in general position, then
    $$\hat\sigma=\frac{1}{n+1}\sum_{i=0}^nv_i$$
    is the barycentre of $\sigma=\langle v\rangle$, which resides in $\sigma^\circ$.
\end{definition}
\begin{definition}
    Let $K$ be a simplicial complex, a barycentric subdivision $K'$ of $K$ is the set of simplices such that:\\
    1. The vertices ($0$-simplices) of $K'$ are the barycentres of simplices in $K$.\\
    2. $\hat\sigma_0,\ldots,\hat\sigma_n$ span a simplex of $K'$ iff $\sigma_0<\sigma_1<\ldots<\sigma_n$.
\end{definition}
So we are just dividing the original simplex with new vertices introduced by barycentres.
\begin{lemma}
    Let $K$ be a simplicial complex, then $K'$ is a simplicial complex and $|K'|=|K|$.
\end{lemma}
\begin{proof}
    First check that $\hat\sigma_0,\ldots,\hat\sigma_n$ is indeed in general position if $\sigma_0<\sigma_1<\ldots<\sigma_n$.
    Indeed, if we have $t_0,\ldots,t_n$ such that
    $$\sum_{i=0}^nt_i=0,\sum_{i=0}^nt_i\hat\sigma_i=0$$
    Take $j$ to be the largest index such that $t_j\neq 0$.
    If no such $j$ exist then we are done, otherwise
    $$\hat{\sigma}_j=\sum_{i=0}^{j-1}\left(-\frac{t_i}{t_j}\right)\hat\sigma_i$$
    is contained in $\sigma_{j-1}$ which is a proper face in $\sigma_j$, contradiction.
    Therefore $\hat\sigma_0,\ldots,\hat\sigma_n$ is in general position.\\
    Now $K'$ is clearly closed under passing to faces.
    Furthermore, if we have two simplices $\sigma',\tau'\in K'$, write $\sigma'=\langle\hat\sigma_0,\ldots,\hat\sigma_m\rangle$ for $\sigma_0<\cdots<\sigma_m$ and $\tau'=\langle\hat\tau_0,\ldots,\hat\tau_n\rangle$ for $\tau_0<\cdots<\tau_n$.
    Obviously $\sigma'\cap\tau'\subset \sigma_m\cap\tau_n$, whcih means that we can assume WLOG that $\sigma',\tau'\in\delta$ for some $\delta\in K$.
    We proceed by induction in $\dim K$.
    If one of $\sigma',\tau'$ does not contain $\hat\delta$, then $\sigma'\cap\tau'$ is contained in $\partial\delta$.
    Otherwise, both $\sigma',\tau'$ contain $\hat\delta$, then $\sigma'\cap\tau'$ is the convex hull of $\{\hat\delta\}\cap(\sigma'\cap\partial\delta)\cap(\tau'\cap\partial\delta)$.
    We are done by induction.\\
    To see $|K'|=|K|$, it suffices to show that $|K|\subset |K'|$.
    We will also use induction on $\dim K$.
    The base case is trivial.
    Let $\sigma=\langle v_0,\ldots,v_m\rangle\in K$ and $x\in\sigma$.
    If $x=\hat\sigma$, then we are done.
    Otherwise, we project $\pi:\sigma\setminus\{\hat\sigma\}\to|\partial\sigma|$ by mapping $y\in\sigma$ to the unique point in the intersection of the ray $\hat\sigma\to y$ and $|\partial\sigma|$.
    By the induction hypothesis, $\pi(x)\in\langle\hat\sigma_1,\ldots,\hat\sigma_n\rangle\in K'$ for some $\sigma_1<\cdots<\sigma_n$, which implies $x\in\langle\hat\sigma_1,\ldots,\hat\sigma_n,\hat\sigma\rangle$.
\end{proof}
\begin{definition}
    Set $K^{(0)}=K$ and $K^{(r)}=(K^{(r-1)})'$.
    The simplicial complex $K^{(n)}$ is the $n^{th}$ barycentric subdivision of $K$.
\end{definition}
\begin{definition}
    If $K$ is a simplicial complex, we set the mesh of $K$ to be
    $$\operatorname{mesh}(K)=\max_{\langle u,v\rangle\in K}|u-v|$$
\end{definition}
\begin{lemma}
    If $\dim K=n$, then
    $$\operatorname{mesh}(K^{(r)})\le\left( \frac{n}{n+1} \right)^r\operatorname{mesh}(K)$$
\end{lemma}
\begin{proof}
    Suffices to show the case for $r=1$.
    If $\langle u,v\rangle\in K'=K^{(1)}$, then we know there is some $\tau<\sigma$ such that $u=\hat\tau,v=\hat\sigma$.
    We can assume that $\tau$ is a $0$-simplex as this maximises the distance between $\hat\sigma$ and $\partial\sigma$.
    Suppose $\sigma=\langle v_0,\ldots,v_n\rangle$ and $\hat\tau=\langle v_0\rangle$, then
    \begin{align*}
        |\hat\sigma-\hat\tau|&=\left|v_0-\sum_{i=0}^m\frac{1}{m+1}v_i\right|=\left|\frac{m}{m+1}v_0-\frac{1}{m+1}\sum_{i=1}^mv_i\right|\\
        &=\left|\frac{1}{m+1}\sum_{i=1}^m(v_i-v_0)\right|\le\frac{1}{m+1}\sum_{i=1}^m|v_i-v_0|\\
        &\le\frac{m}{m+1}\operatorname{mesh}(K)\\
        &\le\frac{n}{n+1}\operatorname{mesh}(K)
    \end{align*}
    The lemma follows.
\end{proof}
Turns out, any map $|K|\to|L|$ is homotopic to a realisation of a simplicial map between some barycentric subdivisions of $K,L$.
\begin{definition}
    Let $K$ be a simplicial complex.
    The (open) star of a vertex $v$ of $K$ is the union of the interiors of simplices of $K$ containing $V$, that is,
    $$\operatorname{St}_K(v)=\bigcup_{v\in\sigma\in K}\sigma^\circ$$
\end{definition}
\begin{definition}
    Let $\phi:|K|\to|L|$ be a map.
    A simplicial map $f:K\to L$ is said to be a simplicial approximation to $\phi$ if $\phi(\operatorname{St}_K(v))\subset\operatorname{St}_L(f(v))$ for any vertex $v$ of $K$.
\end{definition}
It is easy to see that composition of simplicial approximations is also a simplicial approximation.
\begin{note}
    If $\phi=|f|$, then the equality holds.
\end{note}
\begin{lemma}
    If $f:K\to L$ is a simplicial approximation to $\phi:|K|\to|L|$, then $\phi\simeq|f|$.
\end{lemma}
\begin{proof}
    Suppose $|L|\subset\mathbb R^m$.
    We shall show that the straight line homotopy $H(x,t)=t\phi(x)+(1-t)|f|(x)$ works.
    It suffices to show that $\operatorname{Im}H\subset L$.
    Let $x\in |K|$ lie in the interior of some simplex $\sigma$, then $\phi(x)$ lies in the interior of some unique $\tau\in L$.
    We claim that $f(\sigma)\subset\tau$.
    Indeed, for any vertex $v_i$ of $\sigma$, $x\in\operatorname{St}_K(v_i)$, so $\phi(x)\in \phi(\operatorname{St}_K(v_i))\subset\operatorname{St}_L(f(v_i))$ as $f$ is a simplicial approximation of $\phi$.
    Thus $\tau^\circ\subset\operatorname{St}_L(f(v_i))$, so $f(v_i)$ is a vertex of $\tau$ which exactly means $f(\sigma)\le\tau$.
    Now since $\tau$ is convex, the line segment between $|f|(x)$ and $\phi(x)$ is contained in $\tau$, therefore $\operatorname{Im}H\subset |L|$ as desired.
\end{proof}
\begin{note}
    We did not use the assumption of $f$ being a simplicial map in the proof of $f(\sigma)\subset\tau$.
\end{note}
Does a simplicial approximation always exist?
\begin{theorem}[The Simplicial Approximation Theorem]
    Let $K,L$ be simplicial complexes and $\phi:|K|\to |L|$ a map.
    Then there exists a positive integer $r$ and a simplicial approximation $f:K^{(r)}\to L$ to $\phi:|K|=|K^{(r)}|\to|L|$.
\end{theorem}
\begin{proof}
    Consider $\mathcal U=\{\phi^{-1}(\operatorname{St}_L(v)):v=\langle v\rangle\in L\}$ is an open cover of $|K|$.
    We shall show that $\exists \delta>0$ such that $\forall x\in |K|\subset\mathbb R^m$, the ball $B(x,\delta)$ is contained in some element of $\mathcal U$.
    Indeed, if this is not true, then for each $n$ there is some $x_n\in |K|$ such that $B(x_n,1/n)$ is not contained in any open set of $\mathcal U$.
    By the compactness of $|K|$, we can assume $x_n$ converges to some $x\in |K|$ by passing to a convergent subsequence.
    But $x\in U$ for some $U\in\mathcal U$, so there is some $\epsilon>0$ such that $B(x,\epsilon)\subset U$.
    Take $n$ large enough such that $|x_n-x|<\epsilon/2$ and $1/n<\epsilon/2$, then $B(x_n,1/n)\subset U$, contradiction.\\
    Choose $r$ sufficiently large such that $\operatorname{mesh}(K^{(r)})<\delta$, the for each vertex $v$ of $K^{(r)}$, we have $\operatorname{St}_{K^{(r)}}(v)\subset B(v,\delta)\subset U$ for some $U\in\mathcal U$.
    But $U$ takes the form $\phi^{-1}(\operatorname{St}_L(u))$ for some vertex $u$ of $L$.
    So $\operatorname{St}_{K^{(r)}}(v)\subset\phi^{-1}(\operatorname{St}_L(u))$.
    We define $f(v)=u$.
    If $\sigma\in K$ and $x\in\sigma^\circ$, then $f(\sigma)$ is a face of $\tau$ which is the unique simplex of $L$ containing $\phi(x)$ in its interior (this can be proved in the exact same way we proved the claim in the preceding lemma).
    So $f(\sigma)$ is a simplex of $L$, therefore $f$ is a simplicial map and hence approximation to $\phi$ since we have proven $\phi(\operatorname{St}_{K^{(r)}}(v))\subset\operatorname{St}_L(u)$.
\end{proof}