\mychapter{8}{Lesson 8} %181019

\section{Domain extension}

Up until now, encryption has been dealt with messages of fixed size around a polynomial function to $\lambda$. How to deal with messages with arbitrary size? Setting a maximum bound to message length seems impractical, both for waste reasons when messages are too short, and for practicality when messages eventually get too long. The solution takes the form of a ``block-cipher'', where a message of a given size is split into equally-sized blocks, and then encrypted using a fixed-size encryption scheme. Various instances of this technique, called \emph{modes}, have been devised.

\subsection{Electronic Codebook mode}

The operation of \textsc{ecb}-mode is straightforward: Given a message split into blocks $(m_1, \dots, m_t)$, apply the scheme's encryption routine to each block, as shown in figure \ref{fig:ecb}:
\[
    c_i = F_k(r) \oplus m_i \quad \forall i \in \{0, \dots, t\}
\]
Decryption is trivially implemented by \textsc{xor}-ing the ciphered blocks with $F_k(r)$.

\begin{figure}[ht]
    \centering
    \begin{tikzpicture}[node distance = 12mm, >=latex']

        \path[->]
            node (m1) {$m_1$}
            node (x1) [right of = m1] {$\oplus$}
            node (c1) [below of = x1] {$c_1$}
            node (f1) [box, above of = x1] {$f_k$}
            node (r1) [above of = f1] {$r$}

            node (m2) [right of = x1] {$m_2$}
            node (x2) [right of = m2] {$\oplus$}
            node (c2) [below of = x2] {$c_2$}
            node (f2) [box, above of = x2] {$f_k$}
            node (r2) [above of = f2] {$r$}

            node (xd) [right of = x2, node distance = 15mm] {$\dots$}
            node (cd) [right of = c2, node distance = 15mm] {$\dots$}
            node (fd) [right of = f2, node distance = 15mm] {$\dots$}
            node (rd) [right of = r2, node distance = 15mm] {$\dots$}

            node (mt) [right of = xd] {$m_t$}
            node (xt) [right of = mt] {$\oplus$}
            node (ct) [below of = xt] {$c_t$}
            node (ft) [box, above of = xt] {$f_k$}
            node (rt) [above of = ft] {$r$}

            (r1) edge (f1)
            (r2) edge (f2)
            (rt) edge (ft)

            (f1) edge (x1)
            (f2) edge (x2)
            (ft) edge (xt)

            (m1) edge (x1)
            (m2) edge (x2)
            (mt) edge (xt)

            (x1) edge (c1)
            (x2) edge (c2)
            (xt) edge (ct)
        ;

    \end{tikzpicture}
    \caption{\textsc{ecb}-mode block-cipher in action, using a \prf{} as the encryption routine}
    \label{fig:ecb}
\end{figure}

This approach has the advantage of being completely parallelizable, as each block can clearly be encrypted separately; however there is a dangerous flaw in being not \cpa-secure, even when using a \prf-based encryption scheme. To understand why, observe that random nonces for ciphertext randomization are chosen per-message; this means the encryption of message blocks become deterministic in the message scope, enabling an adversary to attack the scheme within a single plaintext. It is sufficient to choose an all-0 or all-1 message to realize that all its blocks would encrypt to the same ciphered block.

\subsection{Cipher block chaining mode (\textsc{cbc})}

% AP190105: The explanation anticipates PRPs, which may be a problem. Should be reviewed at later times...
This mode serializes block encryption by using the preceding ciphered block in the formula:
\[
    c_i = P_k(r) \oplus m_i \quad \forall i \in \{0, \dots, t\}
\]
This time, a \emph{pseudorandom permutation}(\prp) is used instead of a \prf; they will be discussed later on. The diagram in figure \ref{fig:cbc} shows a general view of \textsc{cbc}-mode's operation. The decryption process is analogous but in a reversed fashion, by computing the preimage of a ciphered block and \textsc{xor}-ing it with the preceding ciphered block:
\[
    m_i = P_k^{-1}(c_i) \oplus c_{i-1}
\]

\begin{figure}[ht]
    \centering
    \begin{tikzpicture}[node distance = 12mm, > = latex']

        \draw[->]
            node (r) {$r$}
            node (c0) [below of = r, node distance = 24mm] {$c_0$}

            node (c1) [right of = c0, node distance = 24mm] {$c_1$}
            node (p1) [box, above of = c1] {$P_k$}
            node (x1) [above of = p1] {$\oplus$}
            node (m1) [above of = x1] {$m_1$}

            node (c2) [right of = c1, node distance = 24mm] {$c_2$}
            node (p2) [box, above of = c2] {$P_k$}
            node (x2) [above of = p2] {$\oplus$}
            node (m2) [above of = x2] {$m_2$}

            node (cd) [right of = c2, node distance = 24mm] {$\dots$}
            node (pd) [above of = cd] {$\dots$}
            node (xd) [above of = pd] {$\dots$}
            node (md) [above of = xd] {$\dots$}

            node (ct) [right of = cd, node distance = 24mm] {$c_t$}
            node (pt) [box, above of = ct] {$P_k$}
            node (xt) [above of = pt] {$\oplus$}
            node (mt) [above of = xt] {$m_t$}

            (r) edge (c0)

            (m1) edge (x1)
            (m2) edge (x2)
            (mt) edge (xt)

            (x1) edge (p1)
            (x2) edge (p2)
            (xt) edge (pt)

            (p1) edge (c1)
            (p2) edge (c2)
            (pt) edge (ct)

            (c0) -| +(12mm, 24mm) edge (x1)
            (c1) -| +(12mm, 24mm) edge (x2)
            (c2) -| +(12mm, 24mm) edge (xd)
            (cd) -| +(12mm, 24mm) -- (xt)
        ;

    \end{tikzpicture}
    \caption{\textsc{cbc}-mode block-cipher in action, using a \prp{} as the encryption routine}
    \label{fig:cbc}
\end{figure}

\subsection{Counter mode}

Also denoted as \textsc{ctr} in short, this mode closely resembles \textsc{ecb}-mode but uses a ``rolling'' nonce instead of a static one, as shown in figure \ref{fig:ctr}. At each successive block, the nonce is incremented by 1 and then used in a single block encryption. Since the nonce is in $\binary^n$, the increment is done modulo $2^n$ so that the value will wrap around to 0 if it ever overflows. Decryption is analogous.

\begin{figure}[ht]
    \centering

    \begin{tikzpicture}[node distance = 12mm, > = latex']

        \draw[->]
            node (c0) {$c_0$}

            node (c1) [right of = c0, node distance = 24mm] {$c_1$}
            node (x1) [above of = c1] {$\oplus$}
            node (m1) [left of = x1] {$m_1$}
            node (f1) [box, above of = x1] {$f_k$}
            node (r1) [above of = f1] {$r$}

            node (c2) [right of = c1, node distance = 24mm] {$c_2$}
            node (x2) [above of = c2] {$\oplus$}
            node (m2) [left of = x2] {$m_2$}
            node (f2) [box, above of = x2] {$f_k$}
            node (r2) [above of = f2] {$r + 1$}

            node (cd) [right of = c2, node distance = 19mm] {$\dots$}
            node (xd) [above of = cd] {$\dots$}
            node (fd) [above of = xd] {$\dots$}
            node (rd) [above of = fd] {$\dots$}

            node (ct) [right of = cd, node distance = 24mm] {$c_t$}
            node (xt) [above of = ct] {$\oplus$}
            node (mt) [left of = xt] {$m_t$}
            node (ft) [box, above of = xt] {$f_k$}
            node (rt) [above of = ft] {$r + t - 1$}

            (r1) edge (r2)
            (r2) edge (rd)
            (rd) edge (rt)

            (r1) edge (f1)
            (r2) edge (f2)
            (rt) edge (ft)

            (f1) edge (x1)
            (f2) edge (x2)
            (ft) edge (xt)

            (m1) edge (x1)
            (m2) edge (x2)
            (mt) edge (xt)

            (x1) edge (c1)
            (x2) edge (c2)
            (xt) edge (ct)

            (r1) -| (c0)
        ;

    \end{tikzpicture}
    \caption{Counter-mode block-cipher in action, using a \prf{} as the encryption routine}
    \label{fig:ctr}
\end{figure}

This apparently innocuous change to \textsc{ebc} is enough to ensure \cpa-security, at the cost of perfect parallelization.

\begin{theorem}
    Assume $f_k$ is a \prf, then the counter-mode block cipher is \cpa-secure for variable length messages\footnotemark.
\end{theorem}
\footnotetext{\emph{Variable length messages} exactly means every message $m = (m_1, \dots, m_t)$ is made of $t$ blocks, and $t$ can change from any message to a different one.}

\begin{proof}
    Figure \ref{cryptogame:ctrcpa} models a \cpa{} attack to a counter-mode block-cipher. The proof will proceed by hybrid argument starting from this game, therefore the statement to verify will be $\cryptog{cpa}[\textsc{ctr}](\lambda, 0) \compindist \cryptog{cpa}[\textsc{ctr}](\lambda, 1)$.

    \begin{cryptogame}
        {ctrcpa}
        {A chosen-plaintext attack to counter-mode block-cipher}
        {cpa}

        \cseqbeginloop

        \send{}{$m = (m_1, \dots, m_t)$}{}
        \receive{\shortstack[l]{
            $r = c_0 \pickUAR \binary^n$ \\
            $c_i = f_k(r + i - 1) \oplus m_i \quad \forall i$
        }}
        {$c = (c_0, c_1, \dots, c_t)$}{}

        \cseqendloop

        \cseqdelay

        \send{$|\mu_0| = |\mu_1| \in \mathcal{M}$}{$\mu_0, \mu_1$}{}
        \receive{\shortstack[l]{
            $\rho = \gamma_0 \pickUAR \binary^n$ \\
            $b \pickUAR \binary$ \\
            $(\gamma_b)_i = f_k(\rho + i - 1) \oplus (\mu_b)_i \quad \forall i$
        }}
        {$\gamma_b = (\gamma_0, \gamma_1, \dots, \gamma_t)$}{}

        \cseqdelay

        \cseqbeginloop

        \send{}{$m = (m_1, \dots, m_t)$}{}
        \receive{\shortstack[l]{
            $r = c_0 \pickUAR \binary^n$ \\
            $c_i = f_k(r + i - 1) \oplus m_i \quad \forall i$
        }}
        {$c = (c_0, c_1, \dots, c_t)$}{}

        \cseqendloop

        \cseqdelay

        \send{}{$b'$}{\textsc{Output 1 iff} $b' = b$}

    \end{cryptogame}

    Define the two hybrid games from the original \cpa{} game as follows:

    \begin{itemize}
        \item $\hybridg{1}[\textsc{ctr}](\lambda, b)$: A random function $R$ is chosen \uar{} from $\mathfrak{R}(n, n)$ at the beginning of the game, and is used in place of $F_k$ in all block encryptions;
        \item $\hybridg{2}[\textsc{ctr}](\lambda, b)$: The challenger will pick random values from $\binary^n$ as ciphered blocks, disregarding any encryption routine.
    \end{itemize}

    \begin{lemma}
        $\cryptog{cpa}[\textsc{ctr}](\lambda, b) \compindist \hybridg{1}[\textsc{ctr}](\lambda, b) \quad \forall b \in \binary$
    \end{lemma}

    \begin{proof} The proof is left as exercise.

        \emph{Hint}: Since the original game and the first hybrid are very similar, we can use a distinguisher which plays the \cpa-game; since this is a lemma, our goal in the reduction is to break the precondition contained in the theorem statement.
    \end{proof}

    \begin{lemma}
        $\hybridg{1}[\textsc{ctr}](\lambda, b) \compindist \hybridg{2}[\textsc{ctr}](\lambda, b) \quad \forall b \in \binary$
    \end{lemma}

    \begin{proof}

        Since $m_i$ doesn't affect the distribution of the result at all, for any $i$, if $R(r^{*})$ behaves like a true random extractor, then the two hybrids are indistinguishable in the general case ($R(r + i) \oplus m_i \approx R(r + i)$). However, there is a sneaky issue: if in both games it happens that a given nonce $r_i$ is used in both one query encryption and the challenge message encryption at any step, the subsequent encrypted blocks will be completely random in the second hybrid, whereas in the first hybrid the function's images, albeit random, become predictable, enabling a \cpa.

        Nevertheless, it can be proved that these ``collisions'' happen with negligible probability within $\hybridg{1}[\textsc{ctr}]$. Let:
        \begin{itemize}
            \item $q$ = number of encryption queries in a game run
            \item $t_i$ = number of blocks for the $i$-th query
            \item $\tau$ = number of blocks for the challenge ciphertext
            \item \textsc{Overlap} event: $\exists i, j, \iota : r_i + j = \rho + \iota$
        \end{itemize}

        The \textsc{Overlap} event exactly models our problematic scenario. Now it suffices to show that it occurs negligibly. For simplicity, assume the involved messages are of the same length, that is $t_i = \tau =: t$. Denote with $\textsc{Overlap}_i$ to be the event that the $i$-th query overlaps the challenge sequence as specified above.

        Fix some $\rho$. One can see that $\textsc{Overlap}_i$ happens if:
        \[
            \rho - t + 1 \leq r_i \leq \rho + t - 1
        \]
        which means that $r_i$ should be chosen \emph{at least} in a way that:
        \begin{itemize}
            \item the sequence $\rho, \dots, \rho + t - 1$ comes before the sequence $r_i, \dots, r_i + t - 1$, and they overlap just for the last element $\rho + t - 1 = r_i$ or
            \item the sequence $r_i, \dots, r_i + t - 1$ comes before the sequence the sequence $\rho, \dots, \rho + t - 1$, and they overlap just for the last element $r_i + t - 1 = \rho$.
        \end{itemize}

        Then:

        \begin{align*}
            \Pr[\textsc{Overlap}_i] &= \frac{(\rho + t - 1) - (\rho - t + 1) + 1}{2^n} \\
            &= \frac{2t-1}{2^n} \\
            \Pr[\textsc{Overlap}] &\leq \sum_{i = 1}^{t} \Pr[\textsc{Overlap}_i] \\
            &\leq 2 \frac{t^2}{2^n} \in \negl{\lambda}
        \end{align*}

        which proves that our collision scenario happens with negligible probability, thus the two hybrids are indistinguishable.

    \end{proof}

    Having proven the indistinguishability between the hybrids, the conclusion is reached:
    \[
        \cryptog{cpa}[\textsc{ctr}](\lambda, 0) \compindist \cryptog{cpa}[\textsc{ctr}](\lambda, 1)
    \]

\end{proof}


% Preamble.
\documentclass[12pt]{article}
\usepackage[margin=1.25in]{geometry}
\usepackage[fleqn]{amsmath}
\usepackage{textcomp}
\usepackage{gensymb}
\usepackage{amsfonts}
\usepackage{enumitem}
%\usepackage{tikz}  % Include for figures.
%\usepackage{subfiles}  % Include for subfiles.

%% Title macros.
\newcommand{\HOMEWORKNUM}{21}
\newcommand{\NAME}{D. Choi}
\newcommand{\DATE}{2020-06-24}

\title{\vspace{-2\baselineskip}MATH 225 - Homework \#\HOMEWORKNUM}
\author{\NAME}
\date{\DATE}

%% Formatting options.
%\pagenumbering{gobble}  % Include for single-page document.

% Macros.
%% Contextualized by input/output bases.
\newcommand{\based}[3]{{\{#1\}}_{#2}^{#3}}

% Document.
\begin{document}
\maketitle

\section*{1.}
\begin{equation*}
	A = \begin{pmatrix} 5 & 1 \\ 1 & 5 \end{pmatrix}
\end{equation*}
\begin{enumerate}[label=(\alph*)]
	\item \textit{Let}
	\begin{equation*}
		\vec{a} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}, \quad
		\vec{b} = \begin{pmatrix} 2 \\ 2 \end{pmatrix}, \quad
		\vec{c} = \begin{pmatrix} -1 \\ 1 \end{pmatrix},\quad
		\vec{d} = \begin{pmatrix} 5 \\ -5 \end{pmatrix},\quad
		\vec{e} = \begin{pmatrix} 10 \\ 0 \end{pmatrix}.
	\end{equation*}
	\textit{Which are eigenvectors of $A$? Find their corresponding
	eigenvalues.} \\[\baselineskip]
	For a square matrix $A$, a vector $\vec{v}$ is an eigenvector of $A$ if
	there is a scalar $\lambda$ such that $A\vec{v} = \lambda \vec{v}$.
	\\[\baselineskip]
	For $\vec{a} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$,
	\begin{equation*}
		A \vec{a}
		=
		\begin{pmatrix} 5 & 1 \\ 1 & 5 \end{pmatrix}
		\begin{pmatrix} 1 \\ 1 \end{pmatrix}
		=
		\begin{pmatrix} 6 \\ 6 \end{pmatrix}
		=
		6
		\begin{pmatrix} 1 \\ 1 \end{pmatrix}
		=
		6 \vec{a}
		.
	\end{equation*}
	Thus, $\vec{a}$ is an eigenvector of $A$ with a corresponding eigenvalue
	of \boxed{6}.
	\\[\baselineskip]
	For $\vec{b} = \begin{pmatrix} 2 \\ 2 \end{pmatrix}$,
	\begin{equation*}
		\vec{b} = 2 \vec{a}
		.
	\end{equation*}
	Because $\vec{b}$ is a scalar multiple of $\vec{a}$, an eigenvector of
	$A$, $\vec{b}$ is also an eigenvector of $A$. Furthermore, $\vec{b}$
	shares $\vec{a}$'s corresponding eigenvalue of \boxed{6}.
	\\[\baselineskip]
	For $\vec{c} = \begin{pmatrix} -1 \\ 1 \end{pmatrix}$,
	\begin{equation*}
		A \vec{c}
		=
		\begin{pmatrix} 5 & 1 \\ 1 & 5 \end{pmatrix}
		\begin{pmatrix} -1 \\ 1 \end{pmatrix}
		=
		\begin{pmatrix} -4 \\ 4 \end{pmatrix}
		=
		4
		\begin{pmatrix} -1 \\ 1 \end{pmatrix}
		=
		4 \vec{c}
		.
	\end{equation*}
	Thus, $\vec{c}$ is an eigenvector of $A$ with a corresponding eigenvalue
	of \boxed{4}.
	\\[\baselineskip]
	For $\vec{d} = \begin{pmatrix} 5 \\ -5 \end{pmatrix}$,
	\begin{equation*}
		\vec{d} = -5 \vec{c}
		.
	\end{equation*}
	Because $\vec{d}$ is a scalar multiple of $\vec{c}$, an eigenvector of
	$A$, $\vec{d}$ is also an eigenvector of $A$. Furthermore, $\vec{d}$
	shares $\vec{c}$'s corresponding eigenvalue of \boxed{4}.
	\\[\baselineskip]
	For $\vec{e} = \begin{pmatrix} 10 \\ 0 \end{pmatrix}$,
	\begin{equation*}
		A \vec{e}
		=
		\begin{pmatrix} 5 & 1 \\ 1 & 5 \end{pmatrix}
		\begin{pmatrix} 10 \\ 0 \end{pmatrix}
		=
		\begin{pmatrix} 50 \\ 10 \end{pmatrix}
		\neq
		\lambda
		\begin{pmatrix} 10 \\ 0 \end{pmatrix}
		\text{ for any $\lambda$.}
	\end{equation*}
	Thus, $\vec{e}$ is not an eigenvector of $A$.

	\item \textit{Let $\mathcal{S}$ be the standard basis, and let}
	\begin{equation*}
		\based{T}{\mathcal{S}}{\mathcal{S}} = A.
	\end{equation*}
	\textit{Find a basis $\mathcal{B}$ made of the eigenvectors of $A$, and
	find $\based{T}{\mathcal{B}}{\mathcal{B}}$.} \\[\baselineskip]
	Let
	\begin{equation*}
		\mathcal{B}
		=
		\left\{
			\begin{pmatrix} 1 \\ 1 \end{pmatrix},
			\begin{pmatrix} -1 \\ 1 \end{pmatrix}
		\right\}
		.
	\end{equation*}
	Using $\mathcal{B}$,
	\begin{align*}
		\based{T}{\mathcal{B}}{\mathcal{B}}
		&=
		\based{T}{\mathcal{S}}{\mathcal{B}}
		\based{T}{\mathcal{S}}{\mathcal{S}}
		\based{I}{\mathcal{B}}{\mathcal{S}}
		\\
		&=
		\begin{pmatrix}
			\frac{1}{2} & \frac{1}{2} \\
			-\frac{1}{2} & \frac{1}{2}
		\end{pmatrix}
		\begin{pmatrix} 5 & 1 \\ 1 & 5 \end{pmatrix}
		\begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix}
		\\
		&=
		\boxed{
			\begin{pmatrix} 6 & 0 \\ 0 & 4 \end{pmatrix}
		}
		.
	\end{align*}
	Note the diagonal elements of $\based{T}{\mathcal{B}}{\mathcal{B}}$ are
	the eigenvalues of $A$.
\end{enumerate}
\newpage

\section*{2.}
\textit{Suppose that $A$ is a $3 \times 4$ matrix.}
\begin{enumerate}[label=(\alph*)]
	\item \textit{Is it possible that $\text{dim}(\text{nul}(A)) = 0$?}
	\\[\baselineskip]
	\boxed{\text{No.}} By the rank-nullity theorem,
	\begin{equation*}
		\text{rank}(A) + \text{nullity}(A) = \text{dim}(\text{domain}(A))
		.
	\end{equation*}
	The dimension of the domain of $A$ is $4$, because $A$ has $4$ columns. \\
	The rank of $A$ is at maximum 3, because $A$ is a $3 \times 4$ matrix. \\
	As a result of this, the nullity of $A$ must be at least $1$.

	\item \textit{If $\text{dim}(\text{nul}(A)) = 1$, what can you say about
	$\text{col}(A)$?} \\[\baselineskip]
	\boxed{\text{dim}(\text{col}(A)) = \text{rank}(A) = 4 - 1 = 3.}

	\item \textit{If $\text{dim}(\text{nul}(A)) = 2$, what can you say about
	$\text{col}(A)$?} \\[\baselineskip]
	\boxed{\text{dim}(\text{col}(A)) = \text{rank}(A) = 4 - 2 = 2.}

	\item \textit{If $\text{dim}(\text{nul}(A)) = 3$, what can you say about
	$\text{col}(A)$?} \\[\baselineskip]
	\boxed{\text{dim}(\text{col}(A)) = \text{rank}(A) = 4 - 3 = 1.}

	\item \textit{If $\text{dim}(\text{nul}(A)) = 4$, what can you say about
	$\text{col}(A)$?} \\[\baselineskip]
	\boxed{\text{dim}(\text{col}(A)) = \text{rank}(A) = 4 - 4 = 0.}
\end{enumerate}
\newpage

\section*{3.}
\textit{Let}
\begin{equation*}
	A = \begin{pmatrix} 8 & -10 \\ 5 & -7 \end{pmatrix},
	\qquad
	\vec{v}_1 = \begin{pmatrix} 2 \\ 1 \end{pmatrix}, \quad
	\vec{v}_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}.
\end{equation*}
\begin{enumerate}[label=(\alph*)]
	\item \textit{Show that the vectors $\vec{v}_1$ and $\vec{v}_2$ are
	eigenvectors of $A$ and find their associated eigenvalues.}
	\begin{gather*}
		A \vec{v}_1
		=
		\begin{pmatrix} 8 & -10 \\ 5 & -7 \end{pmatrix}
		\begin{pmatrix} 2 \\ 1 \end{pmatrix}
		=
		\begin{pmatrix} 6 \\ 3 \end{pmatrix}
		=
		3 \begin{pmatrix} 2 \\ 1 \end{pmatrix}
		=
		3 \vec{v}_1
		, \\
		A \vec{v}_2
		=
		\begin{pmatrix} 8 & -10 \\ 5 & -7 \end{pmatrix}
		\begin{pmatrix} 1 \\ 1 \end{pmatrix}
		=
		\begin{pmatrix} -2 \\ -2 \end{pmatrix}
		=
		-2 \begin{pmatrix} 1 \\ 1 \end{pmatrix}
		=
		-2 \vec{v}_2
		.
	\end{gather*}
	Thus, $\boxed{\lambda_1 = 3}, \boxed{\lambda_2 = -2}$.

	\item \textit{Express the vector}
	\begin{equation*}
		\vec{x} = \begin{pmatrix} -4 \\ -1 \end{pmatrix}
	\end{equation*}
	\textit{as a linear combination of $\vec{v}_1$ and $\vec{v}_2$.}
	\\[\baselineskip]
	Solving
	\begin{equation*}
		\begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix}
		\begin{pmatrix} a \\ b \end{pmatrix}
		=
		\begin{pmatrix} -4 \\ -1 \end{pmatrix}
		,
	\end{equation*}
	$a = -3$, and $b = 2$. \\
	Thus,
	\begin{equation*}
		\boxed{
			\vec{x}
			=
			-3 \vec{v}_1 + 2 \vec{v}_2
		}
		.
	\end{equation*}
\end{enumerate}
\newpage

\section*{Extra.}
\textit{Consider the Fibonacci sequence and the following:}
\begin{equation*}
	\begin{pmatrix} 0 & 1 \\ 1 & 1 \end{pmatrix}
	\begin{pmatrix} 2 \\ 3 \end{pmatrix}
	=
	\begin{pmatrix} 3 \\ 5 \end{pmatrix}
	, \quad
	\begin{pmatrix} 0 & 1 \\ 1 & 1 \end{pmatrix}
	\begin{pmatrix} 3 \\ 5 \end{pmatrix}
	=
	\begin{pmatrix} 5 \\ 8 \end{pmatrix}
	.
\end{equation*}
\textit{Find the 1000th Fibonacci number.} \\[\baselineskip]
Let $0$ be the $0$th Fibonacci number.
\begin{equation*}
	\begin{pmatrix} 0 & 1 \\ 1 & 1 \end{pmatrix}^n
	\begin{pmatrix} 0 \\ 1 \end{pmatrix}
	=
	\begin{pmatrix}
		n\text{th Fibonacci number} \\
		(n+1)\text{th Fibonacci number}
	\end{pmatrix}
	.
\end{equation*}
Thus, let
\begin{equation*}
	\begin{pmatrix} 0 & 1 \\ 1 & 1 \end{pmatrix}^n
	\begin{pmatrix} 0 \\ 1 \end{pmatrix}
	=
	\begin{pmatrix}
		a \\
		\_
	\end{pmatrix}
	.
\end{equation*}
\boxed{a} is the 1000th Fibonacci number.

\end{document}


\subsection{Heuristic approach}
A heuristic method has been developed basing on imitating the visual selection of L-shapes (see figure)
\begin{itemize}
\item Overimpose a $3\times 3$ grid on the scatterplot.
\item Score points on each subgrid in such a way that
\begin{itemize}
	\item Points in permitted regions increase score
	\item Points in non-desired regions decrease score
	\item Points in non-allowed regions set score to $\-infty$.
\end{itemize}
\item Use cross-validation to tune scoring parameters.
\end{itemize}

\subsection{The Naive method}

Traditionally selection has been based on searching for genes with negative correlation betweem expression and correlation. This ``Naïve approach" is used to compare with other approaches.


\documentclass[]{article}

%opening
\title{MTH 343 Numerical Analysis: Lecture 12}
\author{Sheikh Abdul Raheem Ali}

\begin{document}

\maketitle

\section*{Interpolation}

	\begin{enumerate}
		\item Solving Systems of Equations
		\item Lagrange Polynomials
	\end{enumerate}

\begin{tabular}{c c}
	$ x_i $ & $ f_i $ \\
	$ x_0 $ & $ f_0 $ \\
	$ x_1 $ & $ f_1 $ \\
	$ x_2 $ & $ f_2 $
\end{tabular}

\[ P_2(x) = \frac{(x-x_1)(x - x_2)}{(x_0 - x_1)(x_0 - x_2)}f_0 + \frac{(x-x_0)(x - x_2)}{(x_1 - x_0)(x_1 - x_2)}f_1 + \frac{(x-x_0)(x - x_1)}{(x_2 - x_0)(x_2 - x_1)}f_2 \]

\begin{tabular}{c c}
	$ x_i $ & $ f_i $ \\
	$ 0 $ & $ 1 $ \\
	$ -1 $ & $ 2 $ \\
	$ 2 $ & $ 3 $
\end{tabular}

\[  P_2(x) = \frac{(x+1)(x - 2)}{(0 + 1)(0 - 2)} + \frac{(x-0)(x - 2)}{(-1 - 0)(-1 - 2)}2 + \frac{(x-0)(x + 1)}{(2 - 0)(2 + 1)}3 \]

$ P_n(x) = \sum_{0}^{n}L_{n,i}f(x_i) $

\section*{Lagrange Polynomial Error Function}

\[ E(x) = |f(x) - P_n(x)| = \frac{|(x-x_0)(x-x_1)\dots(x-x_n)f^{n+1}(\xi)|}{(n+1)!}  \]

Where $ \xi  $ is the element in the smallest interval containing $ x_0,x_1,\dots,x_n $

\end{document}


%!TEX root = main.tex
\section{Simulation setup}
\subsection{Variance reduction}
Because the simulation can be time consuming to run, we want to lower the variance of the estimated parameters as much as possible relative to the computational resources used.

\subsubsection{Control Variates}
Control variates is a variance reduction technique which can be used to reduce the variance of the estimated expected value $\hat{\mu} = \mathbb{E}[Y]$. It works by drawing/calculating a variable $X_i$ such that $Y_i \approx f(X_i)$. One can then estimate $\hat{\mu} = \mathbb{E}[Z]$ where $Z$ is given by:
\begin{align}
Z_i = Y_i - c (X_i - \mathbb{E}[X])
\end{align}

Choosing $c = \frac{Cov(X, Y)}{Var(X)}$ gives the largest variance reduction of:
\begin{equation}
\frac{Var[Z]}{Var[Y]} = \frac{Var[Y]}{Var[Y]} - \frac{Cov(X, Y)^2}{Var[X] Var[Y]} = 1 - \rho_{x, y}^2 \label{eq:variance-reduction}
\end{equation}

In our case there is no analytical expression for the covariance or variance of the random variables. Instead $c$ is estimated from the dataset itself. This introduces some overfitting, thus one should take care to reduce the degrees of freedom in the variance, covariance estimates:
\begin{equation}
\hat{c} = \frac{Cov_{N-1}(X, Y)}{Var_{N-1}(X)}, \quad \hat{\sigma}^2 = Var_{N-2}(Z)
\end{equation}

The subscript denotes the degrees of freedom, and $\hat{\sigma}^2$ has $N - 2$ degrees of freedom, because $\hat{c}$ is estimated from the dataset.

\subsubsection{Estimating infection rate}

From \eqref{eq:variance-reduction} it seen that to get a high variance reduction, $X_i$ just need to be heavily correlated with $Y_i$. In our case we would like to estimate the infection peak time and how many people where infected at the peak. From the theoretical differential equation model those values should be correlated with the infection rate $\beta$.

Assuming the stochastic model have a behaviour somewhat similar to the differential equation model, there exists a fairly simple method for estimating $\beta$ and $\gamma$ \cite{wiki-sir}.
\begin{equation}
\gamma = \frac{R'_{max}}{I_{max}} \text{ and } \beta = -\frac{\ln(p)}{1 - p} \gamma \quad \text{where: } p = \frac{N - R_\infty}{N}
\end{equation}

$R'_{max}$ is the maximum removed difference, and $I_{max}$ is the maximum infected. Due to the construction of the differential equation, these are found at the same time $t$. $R_\infty$ is how many people where removed in the end, thus $p$ is the survival rate.

\subsection{Effect of Global Events}
We want to estimate what effect a global event can have on a virus outbreak. We define an event $e$ as simply being a transfer of people from one or more regions $r_{i}$ to region $r_t$ at a time $t_0$, and the reverse transfer of people from $r_t$ to $r_{i}$ at time $t_1$.

\subsubsection{Motivation: Zika virus}
Recently, a new virus has been observed in South and Latin America. The Zika virus affects pregnant women's fetuses and can cause serious birth defects. The Zika virus is primarily spread by mosquitoes, but can also spread from a man to his sex partners (and from mother to fetus). Some political discussions have been made as to whether or not the Olympic Games should be canceled to prevent the spreading of the virus.

In this report we will run a "hypothetical" simulation, showing how the a virus could spread assuming inter-human infections. As such this simulation is primarily an attempt of showing how such a model would work and not a realistic Zika virus predictor.

\subsubsection{Event definition: Olympic Games in Rio 2016}
In our simulation we define the Olympic Games to take place during $t_0=$ `2016-07-03' and $t_1=$ `2016-07-21'. According to The Gaurdian \cite{theguardian-olympics} Rio expects to recieve 380,000 tourist. We choose to transfer people from all over the world to Rio, such that the transfer is proportional to the population size of each region:

\begin{align}
	\text{Transfer}_0(i, j) &= 380000 \cdot \frac{\text{population}_i}{N} \\
	\text{Transfer}_1(j, i) &= \text{transfer}_0(i, j)
\end{align}

where $\text{Transfer}_0(i, j)$ denotes the number of people transferred from region $i$ to region $j$ at the start of the Olympics and $ \text{Transfer}_1(j, i) $ the people transferred from $j$ to $i$ at the end of the Olympics.


%Note for this test case:
%Simple water flow example using ANUGA: Water flowing down a channel.
%It was called "steep_slope" in an old validation test.

\section{Shallow flow down a mild slope}
This case simulates very shallow flow running down a mild slope topography. It represents an idealisation of the rainfall-runoff problem, which will often involve very shallow flows down such a topography. This case has an analytical solution, and in particular, we consider the steady-uniform solution with the values of bed slope and friction slope are the same.

Suppose that we are given a one dimensional domain. The steady state conditions with a contant water depth everywhere make the shallow water equations to the single identity
\begin{equation}
z_x = - S_f,
\end{equation}
where $z_x$ is the bed slope, and $S_f$ is the symbol for the force of bottom friction.  We take Manning's friction
\begin{equation}
S_f = n^2 \frac{q|q|}{h^{10/3}}
\end{equation}
where $n$ is the Manning's coefficient and $q$ is the discharge $uh$.
If $q$, $n$, and $z_x$ are given, then the analytical solution for $u$ and $h$ is
\begin{equation}
u(x)= \left[- n^{-2} q^{4/3} z_x\right]^{3/10},
\end{equation}
\begin{equation}
h(x)= \frac{q}{u}\,.
\end{equation}

\subsection{Results}
For our test, we consider a square dimensional domain with length and width 100. We take $q=0.2$, $n=0.03$, and $z_x=-0.1$.
The topography is
\begin{equation}
z(x, y)= -0.1 x\,.
\end{equation}
The initial condition is $u=v=0$ and
\begin{equation}
w(x,y,0)= -0.1 x + 0.01\,.
\end{equation}

Some simulation results are as follows.
Figures~\ref{fig:depthdownchan} shows the steady state depth in the downstream direction.
There should be a good agreement with the analytical solution, at least away from the boundaries.

Figures~\ref{fig:depthacrosschan} shows the steady state depth across the slope around the line x = 50m.
There should be a good agreement with the analytical solution, at least away from the boundaries.

Figures~\ref{fig:xvelscrosschan} and~\ref{fig:yvelscroschan} show the steady state $x$- and $y$-velocities, along a slice in the cross slope direction (near $x=50$). The $x$-velocities should agree well with the analytical solution, and the $y$-velocities should be zero.

\begin{figure}
\begin{center}
\includegraphics[width=0.8\textwidth]{depth_x.png}
\caption{Depth in the downstream direction}
\label{fig:depthdownchan}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.8\textwidth]{depth_y.png}
\caption{Depth across the slope around $x$ = 50m}
\label{fig:depthacrosschan}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.8\textwidth]{x_velocity.png}
\caption{$x$-velocity along the cross-section $x=50$ (i.e. a cross-section with constant bed elevation)}
\label{fig:xvelscrosschan}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.8\textwidth]{y_velocity.png}
\caption{$y$-velocity along the cross-section $x=50$ (i.e. a cross-section with constant bed elevation)}
\label{fig:yvelscroschan}
\end{center}
\end{figure}

\endinput


% !TEX root = ../thesis.tex

\chapter{Preliminaries}
\label{chap:preliminaries}
There are several topological spaces that play a key role
when describing properties of the Hopf map $h : S^3 \to S^2\!$.
These include of course the domain $S^3$ and codomain $S^2\!$,
which are traditionally defined as subspaces of $\R^4$ and $\R^3$ respectively.
As we will see later,
it is useful to consider $S^3$ and $S^2$ as quotient spaces of $\C^2 \setminus \set{0}$
or subspaces of the quaternion algebra $\H$ instead.
Both $\C^2$ and $\H$ are isomorphic to $\R^4$ as a real vector space,
but their additional structure sheds light on various properties of the Hopf map.
We will therefore briefly examine these spaces before defining the Hopf map.
In later chapters we will explore the differentiable structure of these spaces,
but for now we focus on their topological properties.
Furthermore, group actions are used extensively throughout this chapter,
so we quickly recall some of the terms involved.

\section{Definitions}

\definition
Let $X$ be a set with additional structure,
such as a vector space or a topological space.
The \emph{automorphism group} of $X$,
denoted $\Aut(X)$ is the group of bijections $X \to X$
that preserve its structure.
More formally we can say that $X$ is an object in a concrete category $\mathcal{A}$,
a category equipped with a faithful functor $\mathcal{A} \to \mathbf{Sets}$,
the forgetful functor.
The automorphism group is the group of invertible morphisms $X \to X$
in this category.

\example
For a group $G$, $\Aut(G)$ is the group of group isomorphisms $G \to G$.
For a module $M$ over a ring $R$,
or for a vector space $V$ over a field $F$,
the automorphism group consists of $R$-linear bijections $M \to M$
and $F$-linear bijections $V \to V$ respectively.
For a topological space $X$,
$\Aut(X)$ is the group of homeomorphisms $X \to X$.
For sets, the automorphism group is simply the group of bijections
from the set to itself.

\definition
Let $G$ be a group and $X$ a set with additional structure.
A \emph{group action} of $G$ on $X$ is a group homomorphism $\phi : G \to \Aut(X)$.
If $X$ is an object in the category $\mathcal{A}$ \,(e.g. the category of vector spaces,
topological spaces, etc.),
we say $G$ acts on $X$ in this category.
Given an element $x \in X$ and $g \in G$,
we will often write $g \cdot x$ for the element $\phi(g)(x)$.

In the definition above,
the group $G$ acts \emph{from the left} on $X$.
For $g, h \in G$ and $x \in X$,
we have $(gh) \cdot x = g \cdot (h \cdot x)$.
Sometimes we encounter a natural \emph{antihomomorphism} $G \to \Aut(X)$.
In this case,
we say that $G$ acts \emph{from the right} on $X$ in the category $\mathcal{A}$,
and to make the notation more natural we write $x \cdot g$ instead of $g \cdot x$,
so that $x \cdot (gh) = (x \cdot g) \cdot h$.

\definition
Let $G$ and $X$ be as before, and $x \in X$.
The \emph{orbit} of $x$ is the set
\[ Gx = \set{ g \cdot x \mid g \in G } \]
Having the same orbit defines an equivalence relation on $X$,
and the quotient with respect to this relation is called the \emph{orbit space},
written $X/G$.

\definition
Let $G$ and $X$ be as before, and $x \in X$.
The \emph{stabiliser} of $x$ is the subgroup
\[ G_x = \set{ g \in G \mid g \cdot x = x } \]

\proposition[prop:open-quotient-map]
Let $X$ be a topological space, $G$ a group that acts on $X$.
When $X/G$ is endowed with the quotient topology,
the quotient map $q : X \surj X/G$ is an open map.

\proof
Let $U \subseteq X$ be open,
define $V = q(U)$.
$V$ is open if and only if $q^{-1}(V)$ is open
by definition of the quotient topology.
We have
\[ q^{-1}(V)
 = \bigcup_{g \in G} g \cdot U \]
Because the automorphisms of the group action are homeomorphisms,
they are open maps,
so $g \cdot U$ is open for all $g \in G$.
Hence, $q^{-1}(V)$ is the union of open sets,
so it is open.
\qed

\definition
A \emph{topological group} is a group $G$ that is also a Hausdorff space,
such that the map $G \times G \to G$, $(g, h) \mapsto gh^{-1}$ is continuous
when $G \times G$ is endowed with the product topology.
This is equivalent to the statement that multiplication and inversion are continuous;
see for example \parencite[ch.~\textsc{iii}, \S~1.1]{bourbaki1971} or \parencite[p.~276]{szekeres2004}.

\definition
Let $G$ be a topological group and $X$ a topological space,
such that $G$ acts on $X$ in the category of sets.
The action is said to be \emph{continuous} if
\[ G \times X \longto X,
\quad (g, x) \longmapsto g \cdot x \]
is a continuous map.
It follows immediately from this definition that
$x \mapsto g \cdot x$ is a homeomorphism for all $g \in G$ when $G$ acts continuously on $X$,
therefore $G$ acts on $X$ in the category of topological spaces;
the group homomorphism $G \to \Aut(X)$ is actually a group homomorphism $G \to \Homeo(X)$.

\theorem{Universal property of the quotient topology}[thm:universal-property-quotient-topology]
Let $X$ and $Y$ be topological spaces and $\sim$ an equivalence relation on $X$.
Denote by $q : X \surj X\modsim$ the quotient map.
Let $f : X \to Y$ be a continuous map such that
for all $x, y \in X$ it holds that $x \sim y$ implies $f(x) = f(y)$.
(Such $f$ is said to be \emph{compatible} with the equivalence relation.)
Then there exists a unique continuous map $g : X\modsim \to Y$ that makes the following diagram commute:
\vspace{-\parskip}
\begin{center}
\begin{tikzcd}[column sep = small] &
X        \ar[dl, two heads, swap, "q"]
         \ar[dr, "f"] & \\
X\modsim \ar[rr, dashed, swap, "\exists_! g"] & & Y
\end{tikzcd}
\end{center}

\proof
See for example \parencite[ch.~\textsc{i}, \S~3.4]{bourbaki1971}.

\proposition
Let $G_1$ and $G_2$ be groups, and $X$ a set.
Suppose that $G_1 \times G_2$ acts on $X$ in the category of sets.
This implies that the subgroups $G_1$ and $G_2$ act on $X$ individually as well.
Then the following holds:
\begin{enumerate}
\item There is a natural action of $G_1$ on $X / G_2$ in the category of sets.
\item If $X$ is a topological space and $G_1 \times G_2$ acts in the category of topological spaces,
      $G_1$ acts on $X / G_2$ in this category.
\item If $G_1$ and $G_2$ are topological groups such that $G_1 \times G_2$ acts continuously on $X$,
      then $G_1$ acts continuously on $X / G_2$.
\end{enumerate}

\proof
Let $x \in X$ such that $[x] \in X / G_2$, and $g \in G_1$.
Define $g \cdot [x] = [g \cdot x]$.
We have to show that this action is well-defined.
Suppose that $[x] = [y]$ for some $y \in X$.
Then there exists an $h \in G_2$ such that $x = h \cdot y$.
Because $h$ and $g$ commute in $G_1 \times G_2$,
we have
\[ g \cdot x = g \cdot (h \cdot y) = (g, h) \cdot y = h \cdot (g \cdot y) \]
Thus, we find $[g \cdot x] = [g \cdot y]$.
That this defines a homomorphism $G_1 \to \Aut(X / G_2)$ follows
from the fact that $G_1 \to \Aut(X)$ is a homomorphism.
This proves statement \textbbf{i}.

Suppose that $X$ is a topological space
and $G_1 \times G_2$ acts on $X$ in the category of topological spaces.
Let $g \in G_1$, then $g$ induces a homeomorphism $\phi : X \to X$,
and a bijection $\psi : X / G_2 \to X / G_2$.
Denote by $q : X \surj X / G_2$ the quotient map,
then $q \circ \phi$ is a continuous map $X \to X / G_2$
that satisfies $\psi \circ q = q \circ \phi$ due to statement \textbbf{i}.
This means that $q \circ \phi$ is compatible with the quotient map,
so by the universal property of the quotient topology (theorem \ref{thm:universal-property-quotient-topology}),
there exists a unique continuous map $\psi'$ such that $q \circ \phi = \psi' \circ q$.
Uniqueness implies that $\psi' = \psi$,
therefore $\psi$ is continuous.
The same argument applies to $\psi^{-1}$,
hence $\psi$ is a homeomorphism.
This shows that $G_1$ acts on $X / G_2$ in the category of topological spaces,
which proves statement \textbbf{ii}.

To prove statement \textbbf{iii},
we will use the following diagram:
\begin{center}
\begin{tikzcd}[row sep = large]
&
X \ar[dr, two heads, "q"] & \\
G_1 \times X           \ar[ur, "a"]
                       \ar[r, "f"]
                       \ar[dr, two heads, swap, "r"] &
G_1 \times {(X / G_2)} \ar[r, "T"] &
X / G_2 \\ &
{(G_1 \times X)} / G_2 \ar[u, dashed, "\exists_! \phi"]
                       \ar[ur, dashed, swap, "\exists_! \psi"] &
\end{tikzcd}
\end{center}
The map $a : G_1 \times X \to X$,
$(g, x) \mapsto g \cdot x$
is continuous because it is the restriction of $G_1 \times G_2 \times X \to X$
that is continuous by assumption.
Let $q: X \surj X / G_2$ denote the quotient map.
Define $f : G_1 \times X \to G_1 \times {(X / G_2)}$ as $f = (\id, q)$.
This map is continuous because both of its coordinates are.
(See proposition 1 of \parencite[ch.~\textsc{i}, \S~4.1]{bourbaki1971}.)
Furthermore $f$ is open,
for $\id$ and $q$ are open. (See proposition~\ref{prop:open-quotient-map}.)
Let $G_2$ act on $G_1 \times X$ by $h \cdot (g, x) = (g, h \cdot x)$
where $g \in G_1, h \in G_2, x \in X$,
and let $r$ denote the quotient map.
$f$ is compatible with $r$,
so by the universal property of the quotient topology (theorem \ref{thm:universal-property-quotient-topology}),
there exists a unique continuous map
$\phi$ that makes the bottom left triangle of the diagram commute.
The map is given by $[g, x] \mapsto (g, [x])$
and its inverse is given by $(g, [x]) \mapsto [g, x]$.
An open set in $(G_1 \times X) / G_2$
is the image under $r$ of an open set in $G_1 \times X$,
so from commutativity it follows that $\phi$ is an open map.
Hence, $\phi$ is a homeomorphism.

On the top of the diagram, we have the map $q \circ a : G_1 \times X \to X / G_2$,
given by $(g, x) \mapsto [g \cdot x]$.
As the composition of continuous maps it is continuous,
and it is compatible with $r$.
Thus, by the universal property of the quotient topology,
there exists a unique continuous map $\psi$
such that $\psi \circ r = q \circ a$.
Composing with $\phi^{-1}$, we find that the map
\[ T : G_1 \times {(X / G_2)} \longto X / G_2,
\quad (g, [x]) \longmapsto [g \cdot x] \]
is continuous,
which proves claim \textbbf{iii}.
Furthermore, the above diagram commutes.
\qed

\theorem[thm:quotient-map-factors]
Let $G_1$ and $G_2$ be groups and $X$ a topological space
such that $G_1 \times G_2$ acts on $X$.
Then $X / (G_1 \times G_2)$ is canonically homeomorphic to $(X / G_1) / G_2$.
In particular, the quotient map $X \surj X / (G_1 \times G_2)$ factors over $X / G_1$.

\proof
Let
$q_1 : X \surj X / G_1$,
$q_2 : (X / G_1) \surj (X / G_1) / G_2$,
and $q_{12} : X \surj X / (G_1 \times G_2)$ denote the quotient maps.
Then we have the following commutative diagram:
\begin{center}
\begin{tikzcd}[row sep = huge, column sep = large]
 X                    \ar[r,  two heads, "q_1"]
                      \ar[d,  two heads, "q_{12}"] &
 X / G_1              \ar[d,  two heads, "q_2"]
                      \ar[dl, dashed, bend right = 7, swap, "\phi_1"] \\
 X / (G_1 \times G_2) \ar[r,  dashed, bend left = 13, "\phi_{12}"] &
(X / G_1) / G_2       \ar[l,  dashed, bend left = 13, "\phi_2"]
\end{tikzcd}
\end{center}
The map $q_2 \circ q_1$ is continuous and compatible with $q_{12}$,
so by the universal property of the quotient topology (theorem \ref{thm:universal-property-quotient-topology})
there exists a unique continuous map $\phi_{12}$ that makes the diagram commute.
Because $q_{12}$ is compatible with $q_1$,
there exists a unique continuous map $\phi_1$ such that $q_{12} = \phi_1 \circ q_1$.
It follows that $\phi_1$ is compatible with $q_2$,
so there exists a unique continuous map $\phi_2$ that makes the diagram commute.
Now we see that $\phi_{12}$ and $\phi_2$ are continuous inverses of one another,
hence $X / (G_1 \times G_2)$ and $(X / G_1) / G_2$ are homeomorphic.
\qed

\subsection*{Physical interpretation}
Groups are prevalent in mathematics.
In physics, groups are often encountered in the context of symmetries.
In that case one may think of a group as a set of transformations of a system,
transformations under which a certain property is invariant.
For instance, angular momentum is invariant under rotation of space,
and four-momentum is invariant under Lorentz transformations.
A \emph{group action} generalises this idea.
Elements of the group induce a transformation of a system.
By applying all possible transformations to a point,
we obtain the \emph{orbit} of a point.
For instance, when we let the Lorentz group act on Minkowski space,
the orbit of a timelike vector is all of the light cone (past and future).
Often, a group encodes transformations that we are \emph{not} interested in.
The \emph{orbit space} is what remains if we consider points that differ
by such a transformation to be equal.
For example, the orbit space of the Lorentz group action on Minkowski space
consists of four elements:
the origin, the class of null (or light-like) vectors,
the class of timelike vectors,
and the class of spacelike vectors.
The \emph{stabiliser} of a point is the subgroup of transformations under which the point is invariant.

Topology is the branch of mathematics that studies abstract properties of space.
It gives us the tools to study properties that do not depend
on exact distances, but rather on overall shape.
For instance, one would like to think of a garden hose
as a one dimensional system where water can move back and forth,
regardless of how the hose is bent or twisted.
Topology allows us to ignore the bending and twisting.
Virtually all spaces that occur in physics are topological spaces:
$\R^3\!$, Minkowski space, Hilbert spaces, etc.
Often these spaces have additional structure
such as a metric or inner product,
but many properties can be derived from the topology alone.
An important example of such a property
is \emph{continuity} of a map between topological spaces,
a notion that is prevalent throughout physics.
Many of the groups encountered in this thesis happen to have a natural topology as well.
In this case, an action on another topological space can be \emph{continuous}.
The definition given in this section codifies our intuition:
if two group elements that are near act on a point,
the resulting points should be near as well.

\section{Projective space}
\label{sec:projective-space}
It is possible to identify $\R^4$ and $\C^2$ as four-dimensional real vector spaces,
by identifying the standard basis $(e_1, e_2, e_3, e_4)$ with the basis $((1, 0), (i, 0), (0, 1), (0, i))$.
The space $\C^2 \setminus \set{0}$ will be prevalent in the rest of this section,
so we introduce a shorthand notation. Furthermore, we embed $S^1$ in $\C$.

\definition
$\CZ = \C^2 \setminus \set{0}$.

\definition
The \emph{unit circle} is defined by
\[ S^1 = \set{ \, z \in \C \mid 1 = |\,z\,| \, } \]
This is a group under multiplication.

\definition[def:s3-real]
The \emph{three-sphere} is defined by
\[ S^3 = \set{ \, x \in \R^4 \mid 1 = \nsq{x} } \]
Here $\|{}\cdot{}\|$ denotes the regular Euclidean norm.
By identifying $\R^4$ with $\C^2$ as above,
we can consider $S^3$ to be a subset of $\CZ$.

Consider the multiplicative group $\Rpos$ of positive real numbers.
It acts continuously on $\C^2$ (in the category of real vector spaces) by scalar multiplication,
and this action can be restricted to $\CZ$ (in the category of sets).
This allows us to give an alternative definition of $S^3$ as a quotient:

\definition[def:s3-complex]
$\SC$ is the orbit space of $\CZ$ with respect to the $\Rpos$ action.
Denote by $r : \CZ \surj \SC$ the quotient map.
$\C^2$ is endowed with its regular topology induced by the Euclidean metric,
and $\SC$ is endowed with the quotient topology.

Intuitively, this definition is not that different from definition~\ref{def:s3-real}.
Every point $p$ at the three-sphere defines a ray from the origin through $p$.
This ray, except for the origin, is the orbit of $p$ under the $\Rpos$ action.
In other words, every orbit can be represented by a point at unit distance from the origin.
The quotient map $r$ corresponds to projection onto the sphere.

\proposition[prop:s3-equivalence]
$S^3$ and $\SC$ as defined in definition~\ref{def:s3-real} and \ref{def:s3-complex} are homeomorphic.

\proof
\newcommand*{\RZ}{{\R^4_{\,\circ}}}
Write $\R^4 \setminus \set{0} = \RZ$.
Let $i : S^3 \to \RZ$ be the inclusion,
and let $\phi : \R^4 \to \C^2$ be the vector space isomorphism
induced by the identification of the bases given earlier in this section.
The inclusion $i$ is continuous, and the restriction $\phi|_\RZ = \phi_\circ$ is a homeomorphism.
Therefore, the composition $\psi = r \circ \phi_\circ \circ i : S^3 \to \SC$ is continuous.
Consider the map
\vspace{-0.3\parskip}
\[ \CZ \longto S^3,
\quad x \longmapsto \phi_\circ^{-1} \left( \frac{x}{\|x\|} \right) \vspace{0.3\parskip} \]
which is continuous and compatible with $r$.
By the universal property of the quotient topology (theorem~\ref{thm:universal-property-quotient-topology}),
this map induces a unique continuous map $\psi^{-1} : \SC \to S^3$ that is the inverse of $\psi$.
Thus, $\psi$ is a homeomorphism.
\qed

Consider the multiplicative group $\C^*$ (the complex plane minus the origin).
It acts continuously on $\C^2$ (in the category of complex vector spaces) by scalar multiplication,
and this action can be restricted to $\CZ$.
This allows us to define the projective space:

\definition[def:complex-projective-line]
The \emph{complex projective line} $\PC$ is the orbit space of $\CZ$ with respect to the $\C^*$ action.
Denote by $q : \CZ \surj \PC$ the quotient map.
$\PC$ is endowed with the quotient topology.

Elements of $\PC$ are indicated by \emph{homogeneous coordinates}:
if $(z_1, z_2)  \in \C^2$ is nonzero,
then we write $(z_1 : z_2)$ for $q(z_1, z_2)$.
We can embed $\C$ in $\PC$ via $z \mapsto (z : 1)$.
The only point that is not reached in this manner is $(1 : 0)$.

\theorem[thm:s2-homeom-p1c]
There exists a homeomorphism between $S^2$ and $\PC$.

\proof
We will postpone the proof until section \ref{sec:hopf-quaternionic},
and prove this with the aid of quaternions in theorem~\ref{thm:hopf-map-equivalence}.
For an alternative proof, see \parencite[ch.~\textsc{viii}, \S~4.3]{bourbaki1974}.

The general linear group $\GLC$
of invertible complex $2 \times 2$ matrices
acts on $\C^2$ by matrix multiplication.
This induces a group action of $\GLC$ on $\CZ$.
Furthermore, the groups $\Rpos$, $S^1$, and $\C^*$ are isomorphic to subgroups of $\GLC$:
given an element $z \in \C^*\!$,
we can identify it with the matrix
\[ \begin{pmatrix} z & 0 \\ 0 & z \end{pmatrix} \]
in the centre of $\GLC$.
$\C^*$ is isomorphic to the direct product $\Rpos \times S^1$:
this is the decomposition of a complex number into its modulus and argument.
It follows that $S^1$ and $\Rpos$ are central in $\GLC$, because their elements correspond to scalar matrices.
Consequently, $S^1$ and $\Rpos$ are normal in $\GLC$.

\subsection*{Informal summary}
The \emph{projective space} $\PC$ is a construction with several interpretations.
For starters, $\PC$ can be thought of as $\C$ with one extra point,
a point “at infinity”.
This allows us to talk about $z_1/z_2$ even when $z_2$ is zero.
Instead of $z_1/z_2$, we write $(z_1 : z_2)$, called \emph{homogeneous coordinates}.
Secondly, theorem \ref{thm:s2-homeom-p1c} tells us that $\PC$ can be thought of as the unit sphere $S^2\!$.
(In fact, $\PC$ is sometimes called the \emph{Riemann sphere}.)
A \emph{homeomorphism} between two spaces
is a function, both one-to-one and onto,
that preserves all topological properties.
From a topological point of view, $\PC$ and $S^2$ are the same space.
This means that when we formulate the Hopf map later on
— a function from $S^3$ to $S^2$ —
we can express it as a function to $\PC$.
This expression is significantly simpler than the one involving Cartesian coordinates on $S^2\!$.

\section{Quaternions}
\label{sec:quaternions}

\definition
The \emph{quaternion algebra} $\H$ is
the real noncommutative algebra with basis $(1, i, j, k)$.
Multiplication is given by the identities
\[ i^2 = j^2 = k^2 = -1,
\quad i\!j =  k, \ jk =  i, \ ki =  j,
\quad j  i = -k, \ kj = -i, \ ik = -j \]
and $1$ commutes with all elements.
In particular, $\H$ is a ring and a four-dimensional real vector space.
Analogously to complex numbers,
this algebra has an involution $\overline{\raisebox{0pt}[0.5em]{${}\cdot{}$}}$
called \emph{conjugation}
that flips the sign of the $i$, $j$, and $k$ components.

\definition
The \emph{trace} is the map $\Tr : \H \to \R,\ q \mapsto q + \overline{q}$.
Because the imaginary parts cancel, the trace of a quaternion is real.
Furthermore, the trace is $\R$-linear.

The reals commute with all quaternions,
so $\Tr(q)$ commutes with $q$ for all $q \in \H$.
Because $\overline{q} = \Tr(q) - q$,
it follows that $q$ and $\overline{q}$ commute.

\definition
The standard inner product on $\H$ is given by
\[ \inp{{}\cdot{}}{{}\cdot{}} : \H \times \H \longto \R,
\quad (p, q) \longmapsto \tfrac{1}{2} \Tr(p\overline{q}) = \tfrac{1}{2}(p\overline{q} + q \overline{p}) \]
Symmetry is clear from the definition,
and bilinearity follows from the linearity of the trace.
For positive definiteness,
remark that for $q = a + bi + cj + dk$,
we have $q \overline{q} = a^2 + b^2 + c^2 + d^2\!$.
Therefore $\inp{q}{q} \geq 0$,
and $\inp{q}{q} = 0 \implies a = b = c = d = 0 \implies q = 0$.

\definition
The \emph{norm} of $q \in \H$ is given by $\nsq{q} = q \overline{q}$.
Because $q$ and $\overline{q}$ commute,
$q\overline{q} = \smallfrac{1}{2}(q \overline{q} + \overline{q} q)$,
so the norm is induced by the inner product.
This norm coincides with the Euclidean norm on $\H$
as real vector space with orthonormal basis $(1, i, j, k)$.
Therefore,
$\H$ with the topology induced by the norm is homeomorphic to $\R^4\!$.

Because conjugation reverses the order of multiplication,
the norm is multiplicative:
for $p, q \in \H$, we have
\[ \nsq{pq}
 = (pq)\overline{(pq)}
 = p \, q \overline{q} \, \overline{p}
 = p \nsq{q} \, \overline{p}
 = \nsq{q} p \overline{p}
 = \nsq{q} \nsq{p} \]

Because $q \overline{q} = \nsq{q}\!$,
we have $q^{-1} = \overline{q} \, \|q\|^{-2}$ for $\|q\| \neq 0$.
Therefore, $\H$ is a division algebra:
every nonzero element has an inverse.

\proposition
$\H^* = \H \setminus \set{0}$ is a topological group.

\proof
Multiplication is continuous,
because for $p, q \in \H^*\,$,
the components of the product $pq$
can be written as a polynomial in the components of $p$ and $q$.
Inversion is continuous,
because the components of $q^{-1}$ are rational functions of the components of $q$,
which do not vanish because $\nsq{q} \neq 0$.
See also \parencite[ch.~\textsc{viii}, \S~1.4]{bourbaki1974}.
\qed

With this machinery,
we can give a quaternionic definition of $S^3$ and $S^2\!$.
Whereas the definitions in section \ref{sec:projective-space} emphasise
how $S^3$ and $S^2$ are quotients with respect to a group action,
the quaternionic definitions emphasise the group structure on the three-sphere itself,
and the action of $S^3$ on $S^2\!$.

Let us revisit the three-sphere as defined in definition~\ref{def:s3-real}.
By identifying $\H$ with $\R^4$ as a normed real vector space
via the basis given earlier in this section,
we can consider $S^3$ to be a subset of $\H$,
the set of quaternions with unit norm:
\[ S^3 = \set{ q \in \H \mid 1 = \nsq{q} } \]
This set is closed under multiplication due to the multiplicativity of the norm,
and it contains $1$.
Therefore, this is a subgroup of $\H^*\!$.
We can embed $S^2$ in $S^3\!$,
but in $\R^4$ there is no preferred way of doing so.
For quaternions, there is one natural choice:

\definition[def:s2-quaternion]
The \emph{two-sphere} $S^2 = \set{ q \in S^3 \mid \Tr(q) = 0 }$,
the set of pure imaginary quaternions with unit norm.
This definition coincides with the conventional definition of $S^2$
when $\R^3$ is identified with the subspace of $\H$ spanned by $i$, $j$, and $k$.
$S^2$ may alternatively be written as $\set{q \in S^3 \mid \inp{1}{q} = 0} = 1^\perp \cap S^3$.

The group $\H^*$ acts
on $\H$ in the category of $\R$-algebras via the following homomorphism:
\[ \phi : \H^* \longto \Aut(\H),
   \quad
   p \longmapsto (q \mapsto p q p^{-1}) \]
Because quaternion multiplication is continuous,
this is a continuous action.
By restriction to the subgroup $S^3\!$,
we get a continuous action of $S^3$ on $\H$.

\proposition[prop:invariant-inner-product]
The inner product on $\H$ is invariant under the action of $\H^*\!$.

\proof
Let $p \in \H^*\!$, $q_1, q_2 \in \H$,
then we have
\begin{align*}
   2 \, \inp{p \cdot q_1}{p \cdot q_2}
&= p q_1 p^{-1} \overline{p q_2 p^{-1}} + p q_2 p^{-1} \overline{p q_1 p^{-1}} \\
&= p q_1 \nsq{p^{-1}} \overline{q_2} \, \overline{p} + p q_2 \nsq{p^{-1}} \overline{q_1} \, \overline{p} \\
&= \nsq{p^{-1}} p (q_1  \overline{q_2} + q_2 \overline{q_1}) \overline{p} \\
&= \nsq{p^{-1}} \, \nsq{p} \, 2 \, \inp{q_1}{q_2} \\
&= 2 \, \inp{q_1}{q_2}
   \tag*{\qed}
\end{align*}

\corollary[cor:s3-action]
Identify $\R^3$ with the subspace of $\H$ spanned by $i$, $j$, and $k$.
Then $\R^3 = 1^\perp$ and $S^2 = 1^\perp \cap S^3$ are invariant under the action of $\H^*\!$,
which means $\H^*$ and its subgroup $S^3$ act continuously on $\R^3$ and $S^2\!$.

$\C$ is a commutative subring of~$\H$.
As real vector spaces with bases $(1, i)$ and $(1, i, j, k)$,
$\C$ can be identified with the subspace of $\H$ spanned by $1$ and $i$.
The stabiliser of $i \in \H$ consists of the nonzero elements that commute with $i$.
These elements are linear combinations of $1$ and $i$,
so we have $\H^*_i = \C^*$ and $S^3_i = S^1\!$.

\proposition[prop:s3-isom-su2c]
$S^3$ is isomorphic to $\SUC$, the group of unitary $2 \times 2$ matrices with determinant $1$.

\proof
Define the unitary matrices
\[            I = \begin{pmatrix}1 &  0 \\ 0 &  1\end{pmatrix}
\qquad \sigma_1 = \begin{pmatrix}0 &  1 \\ 1 &  0\end{pmatrix}
\qquad \sigma_2 = \begin{pmatrix}0 & -i \\ i &  0\end{pmatrix}
\qquad \sigma_3 = \begin{pmatrix}1 &  0 \\ 0 & -1\end{pmatrix} \]
These matrices are sometimes called the \emph{Pauli spin matrices}.
Let $\phi: \H \to \Mat(2 \times 2, \C)$ be the $\R$-linear extension of
\[    1 \longmapsto I,
\quad i \longmapsto i \sigma_1,
\quad j \longmapsto i \sigma_2,
\quad k \longmapsto i \sigma_3 \]
Let $\psi$ be the restriction of $\phi$ to $S^3\!$.
All matrices in the image of $\psi$ are unitary,
and a little computation shows that for $q \in S^3\!$,
$\det \psi(q) = 1$.
The matrices $I, i\sigma_1, i\sigma_2, i\sigma_3$
satisfy the same multiplication rules as $1, i, j, k$.
That is, $i\sigma_1 i\sigma_2 = i\sigma_3$, etc.
Therefore, $\psi$ is a group homomorphism $S^3 \to \SUC$.
This homomorphism is surjective (see \parencite[p.~173]{szekeres2004}),
and $1$ is the only element in its kernel.
Therefore, $\psi$ is an isomorphism.
\qed

\theorem[thm:s3-surj-so3r]
The map
\[ \phi: S^3 \longto \SOR,
\quad q \longmapsto (x \mapsto q \cdot x) \]
is a surjective group homomorphism with kernel $\set{\pm 1}$.
Here $x \in \R^3 \cong \Span(i, j, k)$.

\proof
The map $x \mapsto q \cdot x$ is linear,
and orthogonality follows from
the fact that the inner product is invariant under the action,
as shown in proposition~\ref{prop:invariant-inner-product}.
To show that $x \mapsto q \cdot x$ is not a reflection,
note that $\det : \textup{O}_3(\R) \to \R$ is a continuous map
(see for example \parencite[p.~281]{hatcher2002}).
We can express $\phi$ as a polynomial on all coordinates
when elements of $\SOR$ are written as matrices,
so $\phi$ is continuous.
By composition we get a continuous map $S^3 \to \set{\pm 1}$.
Because $S^3$ is connected, this map must be constant.
% (See for instance proposition~3.4.11 of \parencite[p.~92]{runde2005}.)
The determinant of $\id$ is $1$,
so all $q \in S^3$ induce an orthogonal map with positive determinant.

To show that the kernel of $\phi$ is $\set{\pm 1}$,
suppose that $q \in S^3$ is such that $q \cdot x = q x q^{-1} = x$ for all $x \in \R^3\!$.
Then $q$ commutes with all $x \in \R^3\!$,
so $q$ must be real.
Because $\nsq{q} = 1$, it follows that $q = 1$ or $q = -1$.

To prove surjectivity,
suppose that $\rho \in \SOR$ is an anticlockwise rotation
of $\alpha$ radians about an axis spanned by $u \in \R^3\!$,
where $\nsq{u} = 1$.
Then the quaternion $q = \cos(\smallfrac{1}{2}\alpha) + u \sin(\smallfrac{1}{2}\alpha)$
will map to $\rho$.
To see this,
note that all points on the axis of rotation are fixed points,
for $u$ commutes with $q$.
Furthermore, suppose that $v \in \R^3$ is such that $\inp{u}{v} = 0$.
Set $q_0 = \cos(\smallfrac{1}{2}\alpha)$ and $\vec{q} = u \sin(\smallfrac{1}{2}\alpha)$.
By using identities from \parencite[p.~157]{szekeres2004},
we find
\newcommand*{\vv}{v} % Could switch to v with an arrow, but to me it is just noise.
\newcommand*{\vq}{\vec{q}}
\begin{align*}
q \cdot v &= (q_0 + \vq) \vv (q_0 - \vq)
%      \\ &= (q_0 + \vq) (\inp{v}{\vq} + q_0 \vec{v} - \vec{v} \times \vq)
       \\ &= (q_0 + \vq) (q_0 \vv - \vv \times \vq)
       \\ &= -\inp{\vq}{q_0 \vv - \vv \times \vq} + q_0 (q_0 \vv - \vv \times \vq) + \vq \times {(q_0 \vv - \vv \times \vq)}
       \\ &= q_0^2 \vv - q_0 \vv \times \vq + q_0 \vq \times \vv - \vq \times {(\vv \times \vq)}
       \\ &= q_0^2 \vv - 2 q_0 \vv \times \vq - \vv \inp{\vq}{\vq} + \vq \inp{\vq}{\vv}
       \\ &= (\cos^2(\smallfrac{1}{2}\alpha) - \sin^2(\smallfrac{1}{2}\alpha)) v
           - 2 \cos(\smallfrac{1}{2}\alpha)\sin(\smallfrac{1}{2}\alpha) \, {v \times u}
       \\ &= \cos(\alpha) v + \sin(\alpha) \, {u \times v}
\end{align*}
This demonstrates that $q$ rotates $v$ anticlockwise by $\alpha$ radians about $u$.
We saw already that $x \mapsto q \cdot x$ is an orthogonal map with determinant $1$.
Therefore, $q$ maps to $\rho$.
\qed

\corollary[cor:transitive-s3-action]
$S^3$ acts transitively on $S^2\!$,
for every point on $S^2$ can be mapped
into any other point on $S^2$ by a rotation of the sphere.

The proof of theorem~\ref{thm:s3-surj-so3r} gives us a way to explicitly get
a $q \in S^3$ such that $q \cdot i = p$ for any $p \in S^2$:
we rotate $i$ onto $p$ with a rotation of $\R^3\!$.
If $p = -i$, $q = j$ will suffice,
so suppose $p \neq -i$.
Then an axis that we can rotate about is the one spanned by $i + p$,
which bisects the angle between $i$ and $p$,
so we need to rotate by $\pi$ radians.
We find
\begin{equationref}
\label{eqn:transitive-s3-action}
q = \frac{i + p}{\|i + p\|}
\end{equationref}
To verify that this works,
note that for $p \in S^2$ we have
$p \overline{p} = 1$ and $\overline{p} = -p$,
so $p^2 = -1$.
It then follows that
\[ p^2 = -1
% \implies pi + p^2 = -1 + pi
\enskip \implies \enskip p(i + p) = (i + p)i
\enskip \implies \enskip p(i + p)\overline{(i + p)} = (i + p)i\overline{(i + p)} \]
Multiplying by $\|i + p\|^{-2}$ on both sides then yields $p = q i q^{-1}$.

\subsection*{Physical interpretation}
Just like complex numbers are an extension of the real numbers,
quaternions are an extension of the complex numbers.
These extensions come at a cost:
when going from $\R$ to $\C$, you have to give up the ordering.
When going from $\C$ to $\H$, you have to give up commutativity.
Apart from their rich structure that is interesting in its own right,
quaternions have many useful applications.
By considering $S^3$ as a subset of $\H$,
it inherits a group structure.
Theorem~\ref{thm:s3-surj-so3r} tells us that this group is in a sense
twice $\SOR$: every rotation of $\R^3$ is represented by two antipodal quaternions.
When traversing a great circle through $1$ in $S^3\!$,
the points $1$ and $-1$ both correspond to the identity in $\SOR$.
This path in $S^3$ corresponds to a $4\pi$ rotation of $\R^3\!$,
and after a $2\pi$ rotation we will have moved from $1 \in S^3$ to $-1 \in S^3\!$.
This property is reminiscent of \emph{spinors},
and indeed proposition~\ref{prop:s3-isom-su2c} links the unit quaternions to the Pauli spin matrices.


\chapter{Filtering}
\label{sec:filtering}
\section{Curvelet Coefficient Thresholding}
We now apply our digital transforms for
removing noise from image data. The methodology is standard and is
outlined mainly for the sake of clarity and self-containedness.

Suppose that one is given noisy data of the form
\[
x_{i,j} = f(i,j) + \sigma z_{i,j},
\]
where $f$ is the image to be recovered and $z$ is white noise, i.e.
$z_{i,j} \stackrel{i.i.d.}{\sim} N(0,1)$. Unlike FFTs or FWTs, our
discrete ridgelet (resp. curvelet) transform is not norm-preserving
and, therefore, the variance of the noisy ridgelet (resp. curvelet)
coefficients will depend on the ridgelet (resp. curvelet) index
$\lambda$.  For instance, letting $F$ denote the discrete curvelet
transform matrix, we have $F z \stackrel{i.i.d.}{\sim} N(0,F F^T)$
where $T$ denotes transpose.
Because the computation of $F F^T$ is prohibitively expensive, we
calculated an approximate value $\tilde{\sigma}^2_\lambda$ of the
individual variances using Monte Carlo simulations where the diagonal
elements of $F F^T$ are simply estimated by evaluating the curvelet
transforms of a few standard white noise images.

Let $y_\lambda$ be the noisy curvelet coefficients
($y = F x$). We use the following hard-thresholding rule
for estimating the unknown curvelet coefficients:
\begin{eqnarray}
\hat y_\lambda = y_\lambda  & \mbox{ if }  &
|y_\lambda|/\sigma \geq k \tilde{\sigma}_\lambda\\
\hat y_\lambda = 0 &  \mbox{ if } & |y_\lambda|/\sigma
<  k \tilde{\sigma}_\lambda.
\end{eqnarray}
In our experiments, we actually chose a scale-dependent value for $k$;
we have $k = 4$ for the first scale $(j = 1)$ while $k = 3$
for the others $(j > 1)$.

\subsection*{Poisson Observations}
Assume now that we have Poisson data $x_{i,j}$ with unknown mean
$f(i,j)$.  The Anscombe transformation \cite{rest:anscombe48}
\begin{equation}
\tilde{x} = 2\sqrt{x + \frac{3}{8}}
\end{equation}
stabilizes the variance and we have $\tilde{x} = 2 \sqrt{f} +
\epsilon$ where $\epsilon$ is a vector with independent and
approximately standard normal components. In practice, this is a good
approximation whenever the number of counts is large enough, greater
than 30 per pixel, say.

For a small number of counts, a possibility is to compute the Radon
transform of the image, and then to apply the Anscombe transformation
to the Radon data. The rationale is that, roughly speaking, the
Radon transform corresponds to a summation of pixel values over lines
and  the sum of independent Poisson random variables is a Poisson
random variable with intensity equal to the sum of the individual
intensities. Hence, the intensity of the sum may be quite large (hence
validating the Gaussian approximation) even though the individual
intensities may be small. This might be viewed as an interesting
feature  since,  unlike wavelet transforms, the ridgelet and curvelet
transforms tend to average data over elongated and rather large
neighborhoods.

\section{Filtering Experiments}

\subsubsection{Recovery of Linear Features}

\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=fig_line.ps,bbllx=1cm,bblly=12cm,bburx=15cm,bbury=26cm,width=7.5cm,height=7.5cm,clip=}
\psfig{figure=fig_line_g0p5.ps,bbllx=1cm,bblly=12cm,bburx=15cm,bbury=26cm,width=7.5cm,height=7.5cm,clip=}
}
\hbox{
\psfig{figure=fig_f24_line_g0p5.ps,bbllx=1cm,bblly=12cm,bburx=15cm,bbury=26cm,width=7.5cm,height=7.5cm,clip=}
\psfig{figure=fig_cur_line_g0p5.ps,bbllx=1cm,bblly=12cm,bburx=15cm,bbury=26cm,width=7.5cm,height=7.5cm,clip=}
}}}
\caption{The top panels display a geometric image and that same image
      contaminated with Gaussian white noise. The bottom left and right
      panels display the restored images using the undecimated wavelet
      transform and the curvelet transform, respectively.}
\label{fig_cur_line}
\end{figure}
The next experiment (Figure~\ref{fig_cur_line}) consists of an
artificial image containing a few bars, lines and a square.  The
intensity is constant along each individual bar; from left to right,
the intensities of the ten vertical bars (these are in fact thin
rectangles which are 4 pixels wide and 170 pixels long) are equal to
${32 \over 2^i}$, $i = 0, \ldots 9$.  The intensity along all the
other lines is equal to 1, and the noise standard deviation is $1/2$.
Displayed images have been log-transformed in order to better see
the results at low signal to noise ratio.

The curvelet reconstruction of the nonvertical lines is obviously
sharper than that obtained using wavelets.  The curvelet transform
also seems to go one step further as far as the reconstruction of the
vertical lines is concerned. Roughly speaking, for those templates,
the wavelet transform stops detecting signal at a SNR equal to 1
(we defined here the SNR as the intensity level of the pixels
on the line, divided by the noise standard deviation of the noise)
while the cut-off value equals 0.5 for the curvelet approach.  It is
important to note that the horizontal and vertical lines correspond to
privileged directions for the wavelet transform, because the underlying
basis functions are direct products of functions varying solely in
the horizontal
and vertical directions.  Wavelet methods will
give even poorer results on lines
of the same intensity but tilting substantially
away from the Cartesian axes.
Cf.\ the reconstructions of the faint diagonal
lines in the image.

\subsubsection{Recovery of Curves}
In this experiment (Figure~\ref{fig_cur_picasso}), we have added
Gaussian noise to ``War and Peace,'' a drawing from Picasso which
contains many curved features.  Figure \ref{fig_cur_picasso} bottom left
and right show respectively the restored images by the undecimated
wavelet transform and the curvelet transform. Curves are more sharply
recovered with the curvelet transform.

The authors are working on new methods (some of which will be based on
the curvelet transform) to extract and recover curves from noisy data
with greater accuracy and, therefore, this example is merely to be
taken for illustrative purposes.

\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=fig_picasso.ps,bbllx=2cm,bblly=13cm,bburx=18cm,bbury=25cm,width=8cm,height=6cm,clip=}
\psfig{figure=fig_picasso_g50.ps,bbllx=2cm,bblly=13cm,bburx=18cm,bbury=25cm,width=8cm,height=6cm,clip=}
}
\hbox{
\psfig{figure=fig_picasso_wt.ps,bbllx=2cm,bblly=13cm,bburx=18cm,bbury=25cm,width=8cm,height=6cm,clip=}
\psfig{figure=fig_picasso_cur.ps,bbllx=2cm,bblly=13cm,bburx=18cm,bbury=25cm,width=8cm,height=6cm,clip=}
}}}
\caption{The top panels display a Picasso picture (War and Peace)
      and that same image
      contaminated with Gaussian white noise. The bottom left and right
      panels display the restored images using the undecimated wavelet
      transform and the curvelet transform respectively.}
\label{fig_cur_picasso}
\end{figure}

\subsubsection{Denoising of a Color Image}

\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=fig_cmp_lena_rgb.ps,bbllx=2cm,bblly=13cm,bburx=20cm,bbury=25cm,width=6cm,height=4.cm,clip=}
\psfig{figure=fig_cmp_pepper_rgb.ps,bbllx=2cm,bblly=13cm,bburx=20cm,bbury=25cm,width=6cm,height=4.cm,clip=}
}
\hbox{
\psfig{figure=fig_cmp_mandrill_rgb.ps,bbllx=2cm,bblly=13cm,bburx=20cm,bbury=25cm,width=6cm,height=4.cm,clip=}
\psfig{figure=fig_cmp_barbara_rgb.ps,bbllx=2cm,bblly=13cm,bburx=20cm,bbury=25cm,width=6cm,height=4.cm,clip=}
}
}}
\caption{PSNR versus noise standard deviation using different filtering
      methods. YUV and curvelet, YUV and undecimated wavelet, and YUV and
      decimated wavelet transforms are represented respectively with a
      continuous, dashed, and dotted line.  The upper left panel
      corresponds to {\tt Lena} (RGB), the upper right to {\tt pepper}
      (RGB), the bottom left to {\tt Baboon} (RGB), and the bottom right
      to {\tt Barbara} (RGB).}
\label{fig_exp_rgb_curv}
\end{figure}

In a wavelet based denoising scenario, color RGB images are generally
mapped into the YUV space, and each YUV band is then filtered
independently from the others. The goal here is to see whether the
curvelet transform would give improved results. We used four of the
classical color images, namely {\tt Lena}, {\tt Peppers}, {\tt
      Baboon}, and {\tt Barbara} (all images except perhaps {\tt Barbara}
are available from the USC-SIPI Image Database \cite{SIPI}).
We performed
a series of experiments and
summarized our findings in Figure~\ref{fig_exp_rgb_curv} which again
displays the PSNR (peak signal-to-noise ratio)
versus the noise standard deviation for the four
images.

In all cases, the curvelet transform outperforms the wavelet
transforms in terms of PSNR -- at least for moderate and large values
of the noise level.  In addition, the curvelet transform outputs
images that are visually more pleasant.

\subsubsection{Saturn Rings}

\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=SATURN/fig_sat_sub_g20.ps,bbllx=1.8cm,bblly=12.7cm,bburx=14.5cm,bbury=25.4cm,width=7cm,height=7cm,clip=}
\psfig{figure=SATURN/fig_sat_sub_fil_owt.ps,bbllx=1.8cm,bblly=12.7cm,bburx=14.5cm,bbury=25.4cm,width=7cm,height=7cm,clip=}
}
\hbox{
\psfig{figure=SATURN/fig_sat_sub_fil_atrou.ps,bbllx=1.8cm,bblly=12.7cm,bburx=14.5cm,bbury=25.4cm,width=7cm,height=7cm,clip=}
\psfig{figure=SATURN/fig_sat_sub_fil_cur.ps,bbllx=1.8cm,bblly=12.7cm,bburx=14.5cm,bbury=25.4cm,width=7cm,height=7cm,clip=}
}}
}
\caption{Top left, part of Saturn image with Gaussian noise. Top right,
filtered image using the undecimated bi-orthogonal wavelet transform.
Bottom left and right, filtered image by the \`a trous wavelet
transform algorithm and the curvelet transform.}
\label{fig_sub_saturn_cur_filter}
\end{figure}

Gaussian white noise with a standard deviation
fixed to 20 was added to the {\tt Saturn} image.
We employed several methods to filter the noisy image:
\begin{enumerate}
\item Thresholding of the Curvelet transform.
\item Bi-orthogonal undecimated wavelet de-noising methods using  the
      Dau\-che\-chies-An\-to\-ni\-ni 7/9 fil\-ters (FWT-7/9) and hard thresholding.
\item A trous wavelet transform algorithm and hard thresholding.
\end{enumerate}
Our experiments are reported in Figure~\ref{fig_sub_saturn_cur_filter}.  The
curvelet reconstruction does not contain the quantity of disturbing artifacts
along edges that one sees in wavelet reconstructions. An examination
of the details of the restored images % (Figure~\ref{fig_sub_saturn_cur_filter})
is instructive. One notices that the decimated wavelet transform
exhibits distortions of the boundaries and suffers substantial loss of
important detail.  The \`a trous  wavelet transform gives better
boundaries, but completely omits to reconstruct certain ridges.
In addition, it exhibits numerous small-scale embedded
blemishes; setting higher thresholds to avoid these blemishes would
cause even more of the intrinsic structure to be missed.

% Further results are visible at the following URL: {\tt
%  http://www-stat.stanford.edu/$\sim$jstarck}.

\subsubsection{Supernova with Poisson noise}

\begin{figure}[htb]
\centerline{
\hbox{
\psfig{figure=fig_kepler1604.ps,bbllx=1.8cm,bblly=12.7cm,bburx=14.5cm,bbury=25.4cm,width=7cm,height=7cm,clip=}
\psfig{figure=fig_kepler1604_fil_rid.ps,bbllx=1.8cm,bblly=12.7cm,bburx=14.5cm,bbury=25.4cm,width=7cm,height=7cm,clip=}
}}
\caption{Left, XMM/Newton image of the Kepler SN1604 supernova.
Right, ridgelet filtered image.}
\label{fig_sn1604}
\end{figure}
Figure~\ref{fig_sn1604} shows an example of an X-ray image filtering by the
ridgelet transform using such an approach. Figure~\ref{fig_sn1604} left
and right
show respectively the  XMM/Newton image of the Kepler SN1604 supernova and
the ridgelet filtered image (using a five sigma hard thresholding).

% \clearpage
% \newpage