\chapter{\projmr3 \ Multi-Channel Data Set}
\index{multi-channel}
\section{Introduction}

Modern image processing applications often involve multispectral data.
This can be color images in R, G and B coordinates, observations of the same
area but at different times, etc. Much effort in the compression application
domain
has been expended in recent years in order to compress efficiently such
data sets. The challenge is to have a data representation which takes
into account at the same time both the spatial and the spectral
(or temporal) correlation. A three-dimensional transform-based coding
technique has been proposed in \cite{mc:saghri95}, consisting of
a one-dimensional spectral Karhunen-Lo\`eve transform
\cite{ima:karhunen47} (KLT)
and a two-dimensional spatial discrete cosine transform (DCT).
The KLT is used to decorrelate the spectral domain and the
DCT is used to decorrelate the spatial domain.
All images are first decomposed into blocks, and each block uses its own
Karhunen-Lo\`eve transform instead of one single transform
for the whole image.
Lee \cite{mc:lee99} has improved on this approach by introducing a varying
block size. The block size is defined using a quadtree, followed by
bit allocation
for each block. The DCT transform can also be replaced by a wavelet
transform \cite{mc:epstein92,mc:tretter95} (WT).

We present in this chapter how a Wavelet-KLT transform can be used for
noise removal. Decorrelating first the data in the spatial domain using
the WT and following that in the
spectral domain, by the KLT, has the advantage of
providing us with robust noise modeling in the WT-KLT space, and hence to
be able
to filter the transformed data in an efficient and effective
way. We show also that
the correlation matrix can be computed by different methods based on
the noise modeling. These methods are evaluated with a set of different
images.

We first present the Wavelet-KLT transform and how to model the
noise in this space. We then describe three different methods to
filter the coefficients, and finally illustrate these approaches with
 a set of experiments.

% We are interested in having an entropy measurement
% of such data sets, knowing that the correlation between two images
% can be relatively high. A 3-dimensional wavelet transform is not
% appropriate because the third dimension is generally completely different
% from the two first. The standard case is the data set where we have
% 2-dimensional spatial information versus a frequency band.

\section{The Wavelet-Karhunen-Lo\`eve transform}
\subsection{Definition}
The Karhunen-Lo\`eve transform, also often referred to as eigenvector,
Hotelling transform, or Principal Component Analysis (PCA)
 \cite {ima:karhunen47,ima:loeve48,ima:hotelling33}
allows us to transform  discrete signals into a sequence of uncorrelated
coefficients.
Considering a vector $D={d_1,..., d_L}$
of $L$ signals or images  of dimension N
(i.e., $N$ pixels per image),
we denote $M = \left\{m_1, ..., m_L\right\}$ the
mean vector of the population ($m_i$ is the mean of the ith
signal $d_i$). The covariance matrix $C$ of $D$ is defined by
$C = (D-M)(D-M)^t$, and is of order $L \times L$.
Each element $c_{i,i}$ of $C$ is
the variance of $d_i$, and each element $c_{i,j}$ is the
covariance between $d_i$ and
$d_j$. The KLT method consists of
applying the following transform to all
vectors $x_i = \left\{d_1(i),...,d_L(i)\right\}$ ($i=1..N$):
\begin{eqnarray}
y_i = \Lambda^{-\frac{1}{2}}A(x_i-M)
\label{eqn_trans}
\end{eqnarray}
where $\Lambda$ is the diagonal matrix of eigenvalues of the covariance
matrix $C$, and $A$ is a matrix whose rows are formed
from the eigenvectors
of $C$ \cite{ima:gonzalez93}, ordered following
decreasing order of eigenvalues.

Because the rows of $A$ are orthonormal vectors, $A^{-1} = A^t$, and
any vector $x_i$ can be recovered from its corresponding $y_i$ by:
\begin{eqnarray}
x_i = \Lambda^{\frac{1}{2}}A^t y_i + M
\end{eqnarray}

The $\Lambda$ matrix multiplication can be seen as a normalization.
Building $A$ from the correlation matrix instead of the covariance matrix
leads to another kind of normalization,
and the $\Lambda$ matrix can be suppressed
($y_i = A(x_i-M)$ and $x_i = A^t y_i + M$). Then the norm of $y$ will
be equal to the norm of $x$.

\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=fig_wtpca.ps,bbllx=1cm,bblly=11cm,bburx=19cm,bbury=27cm,width=12cm,height=10cm,clip=}
}
}}
\caption{WT-KLT transform flowchart. Each frame of the input data set is first wavelet
transformed, and a principal component analysis is applied at each resolution
level.}
\label{fig_wtpca}
\end{figure}

We suppose now that we have $L$ observations of the same view, e.g.\
at different wavelengths (or at different epochs, etc.), and denote
as $d_l$ one
observation, $W^{(l)}$ its wavelet transform, and $w_{l,j,k}$ one wavelet
coefficient at scale $j$ and at position $k$.
The standard approach would be to use an orthogonal wavelet transform, and
to calculate the correlation matrix $C$ from the wavelet coefficients instead
of the pixel values:
\begin{eqnarray}
C_{m,n} =
 \frac{\sum_{j=1}^{J} \sum_{k=1}^{N_j} w_{m,j,k} w_{n,j,k}}
                      {\sqrt{\sum_{j=1}^{J} \sum_{k=1}^{N_j} w^2_{m,j,k}} \sqrt{\sum_{j=1}^{J} \sum_{k=1}^{N_j} w^2_{n,j,k}}}
\label{Correlation_matrix1}
\end{eqnarray}
where $J$ is the number of bands, and $N_j$ is the number of coefficients
in the band $j$.
In \cite{mc:lee99}, a more complex approach has been
proposed, which  is to decompose the images into $N_b$ blocks and
apply a KLT o each block separately.
We investigate here different approaches for data restoration.

\subsection{Correlation matrix and noise modeling}
We introduce a noise model into our calculation of the correlation matrix.
Indeed, if the input sequence $D$ contains noise, then the
wavelet coefficient
are noisy too. Eigenvalues at the high scales are computed with
noisy WT coefficients and
we may lose the true underlying  correlation that exists
between the input images $d_l$.
The expression of the correlation matrix has to be modified in order
to allow us to take into account the noise.
We add a weighting term to each wavelet coefficient  which
depends on the signal to noise ratio.
The correlation matrix is calculated by:
\begin{eqnarray}
C_{m,n} =
 {\frac{ \sum_{j=1}^{J} \sum_{k=1}^{N_j}  p_j(w_{m,j,k}) w_{m,j,k} p_j(w_{n,j,k}) w_{n,j,k}}
                      {\sqrt{  \sum_{j=1}^{J} \sum_{k=1}^{N_j}  p^2_j(w_{m,j,k}) w^2_{m,j,k}} \sqrt{\sum_{j=1}^{J} \sum_{k=1}^{N_j} p^2_j(w_{n,j,k}) w^2_{n,j,k}}}}
\label{Correlation_matrix2}
\end{eqnarray}
where $p_j$ is a weighting function. The standard approach corresponds to the
specific case where $p_j(w_m) = 1$ (no weighting). By considering that only
wavelet coefficients with high signal to noise ratio should be used for
the correlation matrix calculation, $p_j$ can be defined by:
\begin{eqnarray}
p_j(w) = \left\{
  \begin{array}{ll}
  \mbox{ 1 } & \mbox{ if }  w \mbox{ is significant} \\
  \mbox{ 0 } & \mbox{ if }  w \mbox{ is not significant}
  \end{array}
  \right.
\end{eqnarray}
and a wavelet coefficient $w$ is said to be ``significant''
if its probability of being
due to noise is smaller than a given $\epsilon$ value. In the case of
Gaussian noise, it suffices to compare  the wavelet coefficients $w$ to
a threshold level $t_j$. $t_j$ is generally taken as $\lambda \sigma_j$, where
$\sigma_j$ is the noise standard deviation at scale $j$, and $\lambda$ is
chosen
between 3 and 5. The value of $\lambda=3$ corresponds to a probability of
false
detection of $0.27$\%, for Gaussian statistics.

This hard weighting scheme may lead to problems if only a few coefficient are
significant, and can be replaced by a soft weighting one, by
defining $p_j(w)$ by:
\begin{eqnarray}
p_j(w) =  1 - \mathrm{Prob}(W > \mid w \mid)
\end{eqnarray}
where $\mathrm{Prob}(W > \mid w \mid)$ is the probability that a wavelet
coefficient is larger than $w$ due to the noise. For Gaussian noise, we have:
\begin{eqnarray}
   p_j(w)  & =  & 1 - \frac{2}{\sqrt{2 \pi}
\sigma_j} \int_{\mid w  \mid}^{+\infty} \exp(-W^2/2\sigma^2_j) dW \nonumber \\
 & = & \mbox{erf}(\frac{\mid w  \mid }{\sqrt{2}\sigma_j})
\end{eqnarray}

\subsection{Scale and Karhunen-Lo\`eve transform}

We can also analyze separately each band of the
wavelet transform, and then apply one KLT per resolution level. This
implies calculating a correlation matrix $C^{(j)}$ for each band $j$.

\begin{eqnarray}
C^{(j)}_{m,n} =
 {\frac{\sum_{k=1}^{N_j} p_j(w_{m,j,k}) w_{m,j,k} p_j(w_{n,j,k}) w_{n,j,k}}
                      {\sqrt{\sum_{k=1}^{N_j}  p^2_j(w_{m,j,k}) w^2_{m,j,k}}  \sqrt{\sum_{k=1}^{N_j} p^2_j(w_{n,j,k}) w^2_{n,j,k}}}}
\label{Correlation_matrix3}
\end{eqnarray}

This has the advantage to take into account more complex behavior of the
signal. Indeed, structures of different sizes may have a different
spectral behavior (for example, stars and galaxies in astronomical images),
and a band-by-band independent analysis allows us to better represent
this kind of data.

\subsection{The WT-KLT transform}
The final WT-KLT algorithm has these steps:
\begin{enumerate}
\item Estimate the noise standard deviation $\sigma^{(l)}$ of each input
data set $d_l$.
\item Calculate the wavelet transform $W^{(l)}$ of its each input
data set $d_l$.
\item For each band $j$ of the wavelet transform, calculate
the correlation matrix $C^{(j)}$ relative to the vector
$x_j = \left\{ W^{(1)}_j, W^{(2)}_j, ..., W^{(L)}_j \right\}$, where
$W^{(l)}_j$ represents the band $j$ of the wavelet transform $W^{(l)}$ of $d_l$.
\item For each band $j$, we diagonalize the matrix $C^{(j)}$ and build the
transform matrix $A_j$ from the eigenvectors of $C^{(j)}$.
\item For each band $j$ and each position $k$, we apply the matrix $A_j$
to the vector
$x_{j,k} = \left\{ w_{1,j,k}, w_{2,j,k}, ..., w_{L,j,k} \right\}$:
\begin{eqnarray}
  y_{j,k} = A_j x_{j,k}
\end{eqnarray}
\item The WT-KLT coefficients $c_{l,j,k}$ are derived from $y_{j,k}$ by
$c_{l,j,k} = y_{j,k}(l)$. The $l$ index in the transformed coefficients
no longer represent the observation number, but instead the
eigenvector number.
$l=1$ indicates the main eigenvector while $l=L$  indicates  the last one.
\end{enumerate}
The mean vector $M$ disappears in this algorithm because the wavelet
coefficients are zero mean.

Figre~\ref{fig_wtpca} shows the flowchart of the WT-KLT transform.

\subsection{The WT-KLT reconstruction algorithms}

The reconstruction algorithm has these steps:
\begin{enumerate}
\item For each band $j$ and each position $k$, we apply the matrix $A_j^t$
to the vector
$y_{j,k} = \left\{ c_{1,j,k}, c_{2,j,k}, ..., c_{L,j,k} \right\}$
\begin{eqnarray}
  x_{j,k} = A_j^t y_{j,k}
\end{eqnarray}
\item The wavelet coefficients $w_{l,j,k}$ are derived from $x_{j,k}$ by
 $ w_{l,j,k} = x_{j,k}(l)$.
\item An inverse wavelet transform of $W^{(l)}$ furnishes $d_l$.
\end{enumerate}

\subsection{WT-KLT Transform of 1D Multichannel Data: wk1d\_trans}
\index{wk1d\_trans}
\label{sect_mc1d_trans}
The program
{\em wk1d\_trans} computes the WT-KLT  transform of
a multichannel 1D signal (2D data set).
The wavelet transform (\`a trous algorithm)
is applied on each individual 1D signal, and the KL transform is applied for
each resolution level, exept the last (smooth) scale.
So we have a set of eigenvectors for
each resolution level $j$ $\{e_{j,1}, \dots, e_{j,N}\}$.
The output is a set
$N$ multiresolution transform files in FITS format (suffix ``.fits'').
Eigenvectors
are stored by increasing importance. The ith multiresolution file
($i = 1 \dots N$)
contains the eigenvectors $\{e_{1,i}, \dots, e_{J-1,i}\}$, where $J$ is the
maximum resolution level (last smooth array).
{\bf
\begin{center}
 USAGE: wk1d\_trans option data\_in prefix\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
 \item {\bf [-n number\_of\_scales]} \\
 Number of scales used in the multiresolution transform.
 Default is 4.
\item {\bf [-C]} \\
Write the correlation matrix to disk.
The file name is ``Correl\_Matrix''. Default is not to do this.
\item {\bf [-x CorrelMat\_Method]}
{\small
\begin{itemize}
\item{0:} One correlation matrix per band.
\item{1:} One correlation matrix for all bands.
\item{2:} One correlation matrix for all band, except the last scale.
\item{3:} Import correlation matrix.
\end{itemize}
Default is one correlation matrix per band.
}
\item {\bf [-O Input\_CorrelMat\_FileName]} \\
Input correlation matrix file name.
Only used when option ``-x 3'' is set. Default is not to avail of this.
\end{itemize}

\subsubsection*{Examples}
\begin{itemize}
\item wk1d\_trans input.fits wk \\
Calculate the WT-KLT of the input data and store the eigenvectors
in the files ``wk\_ev\_1.fits, ..., wk\_ev\_N.fits''. Each file number $i$
contains a 2D data set. A given row $j$  represents eigenvector
number $i$ at resolution level $j$.
\item wk1d\_trans -C input.fits wk \\
Ditto, but write the calculated correlation matrix to disk.
\end{itemize}

\subsection{WT-KLT Reconstruction of 1D Multichannel Data: wk1d\_trec}
\index{wk1d\_trec}
The program
{\em wk1d\_trec} computes the WT-KLT  transform of
a multichannel 1D signal (2D data set), and reconstructs the data from
a subset of eigenvectors. By default, only the first eigenvector is
used at each resolution level.
{\bf
\begin{center}
 USAGE: wk1d\_trec option data\_in data\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item {\bf [-n number\_of\_scales]} \\
 Number of scales used in the multiresolution transform.
 Default is 4.
\item {\bf [-C]} \\
Write the correlation matrix to disk.
The file name is ``Correl\_Matrix''. Default is not to do this.
\item {\bf [-x CorrelMat\_Method]}
{\small
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item{0:} One correlation matrix per band.
\item{1:} One correlation matrix for all bands.
\item{2:} One correlation matrix for all band, except the last scale.
\item{3:} Import correlation matrix.
\end{itemize}
Default is one correlation matrix per band.
}
\item {\bf [-O Input\_CorrelMat\_FileName]} \\
Input correlation matrix file name.
Only used when option ``-x 3'' is set. Default is not to use this.
\item {\bf [-F NbrEigenVect]} \\
Number of eigenvectors used for the reconstruction.
Default is set to the number of images.
\item {\bf [-K EigenVect\_Number]} \\
Eigenvector number which will not be used for the reconstruction.
\end{itemize}
\subsubsection*{Examples}
\begin{itemize}
\item wk1d\_trec input.fits output.fits \\
Computes the WT-KLT  transform and reconstructs the data from
the first eigenvector of each resolution level.
\item wk1d\_trec -F 3  input.fits output.fits \\
Use the three first eigenvectors
\item wk1d\_trec -F 7 -K 2 -K 3 -K 4 trans.fits output.fits \\
Use four eigenvectors (i.e., eigenvectors 1,5,6,7).
\end{itemize}

\subsection{WT-KLT Transform of 2D Multichannel Data: wk\_trans}
\index{wk\_trans}
\label{sect_mc_trans}
The program
{\em wk\_trans} computes the WT-KLT  transform of
a multichannel image (3D data set) containing $N$ frames
($N < 500$). The wavelet transform
is applied to each frame, and the KL transform is applied to each
resolution level, exept the last (smooth) scale.
So we have a set of eigenvectors for
each resolution level $j$ $\{e_{j,1}, \dots, e_{j,N}\}$.
The output is a set
$N$ multiresolution transform files (suffice ``.mr''). Eigenvectors
are stored by increasing importance. The ith multiresolution file (i=1..N)
contains the eigenvectors $\{e_{1,i}, \dots, e_{J-1,i}\}$, where $J$ is the
maximum resolution level (last smooth array).
If the output file name
given by the user does not contain the ``.mr'' suffix, it is automatically
added. The ``.mr'' file is a FITS format file, and can be manipulated by
any package dealing with FITS format.
{\bf
\begin{center}
 USAGE: wk\_trans option image\_in wtklt\_transform\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
\item {\bf [-t type\_of\_multiresolution\_transform]}
\itemsep=0.1truecm
{\small
\begin{enumerate}
\baselineskip=0.4truecm
\item linear wavelet transform: \`a trous algorithm
\item B-spline wavelet transform: \`a trous algorithm
\item wavelet transform in Fourier space
\item morphological median transform
\item morphological minmax transform
\item pyramidal linear wavelet transform
\item pyramidal B-spline wavelet transform
\item pyramidal wavelet transform in Fourier space:
                     wavelet =  between two resolutions
\item  pyramidal wavelet transform in Fourier space:
                     wavelet = difference between the square of two resolutions
\item  pyramidal median transform
\item  pyramidal Laplacian
\item  morphological pyramidal minmax transform
\item  decomposition on scaling function
\item  (bi-) orthogonal wavelet transform. \\
Antonini 7/9 filters ~\cite{wave:antonini92} are used by default, with an
$L_1$ normalization. The filters can be changed using the ``-T'' option, and
an $L_2$ normalization is obtained by ``-L'' option.
\item  Feauveau wavelet transform
\item  Feauveau wavelet transform without undersampling
\item  G transform (non-redundant morphological min-max algorithm)
\item Haar wavelet transform (L2 normalization).
\item Half-pyramidal wavelet transform (HPWT)
\item Mixed HPWT and Median method
\item dyadic wavelet transform
\item Mixed WT and PMT method (WT-PMT)
\item Undecimated Haar transform: \`a trous algorithm
\item Undecimated (bi-) orthogonal wavelet transform. \\
Antonini 7/9 filters ~\cite{wave:antonini92} are used by default, with an
$L_1$ normalization. The filters can be changed using the ``-T'' option, and
an $L_2$ normalization is obtained by ``-L'' option.
\end{enumerate}}
Default is 2.
\item {\bf [-T type\_of\_filters]}
{\small
\begin{enumerate}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item Antonini 7/9 filters.
\item Daubechies filter 4.
\item Biorthogonal 2/6 Haar filters.
\item Biorthogonal 2/10 Haar filters.
\item Odegard 7/9 filters.
\item User's filters.
\end{enumerate}}
Default is Antonini 7/9 filters. \\
 This option is only available if the chosen transform method is
 the (bi-) orthogonal transform (-t 14 or -t 24).
\item {\bf [-L]} \\
Use an $L_2$ normalization. Default is $L_1$.
% \item {\bf [-u]} \\
% Number of undecimated scales used in the Undecimated Wavelet Transform
% Default is all scales.
 \item {\bf [-n number\_of\_scales]} \\
 Number of scales used in the multiresolution transform.
 Default is 4.
\item {\bf [-C]} \\
Write the correlation matrix to disk.
The file name is ``Correl\_Matrix''. Default is not to do this.
% \item {\bf [-p]} \\
% Normalize the correlation matrix. Values on the diagonal are 1.
% Default is no.
\item {\bf [-x CorrelMat\_Method]}
{\small
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item{0:} One correlation matrix per band.
\item{1:} One correlation matrix for all bands.
\item{2:} One correlation matrix for all band, except the last scale.
\item{3:} Import correlation matrix.
\end{itemize}
Default is one correlation matrix per band.
}
\item {\bf [-O Input\_CorrelMat\_FileName]} \\
Input correlation matrix file name.
Only used when option ``-x 3'' is set. Default is not to do this.
\end{itemize}

\subsubsection*{Examples}
\begin{itemize}
\item wk\_trans cube.fits wk \\
Calculate the WK-KLT of a 3D data set. If the cube contains $N$ frames,
$N$ files will be created (wk\_ev\_1.mr, ..., wk\_ev\_N.mr).
\item mr\_extract -b 2 wk\_ev\_1.mr b2\_1  \\
Extract the second band from the multiresolution file ``wk\_ev\_1.mr''.
The file ``b2\_1.fits'' contains an image which
corresponds to the first eigenvector at the second resolution level.
\end{itemize}

\subsection{WT-KLT Reconstruction of 2D Multichannel Data: wk\_trec}
\index{wk\_trec}
The program
{\em wk\_trec} computes the WT-KLT  transform of
a multichannel image (3D data set), and reconstructs the data from
a subset of eigenvectors.
The output file contains the reconstruction. Most of the options are the
same as those described in section~\ref{sect_mc_trans}.
{\bf
\begin{center}
 USAGE: wk\_trans option image\_in wtklt\_transform\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm

\item {\bf [-t type\_of\_multiresolution\_transform]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-T type\_of\_filters]}
see section~\ref{sect_mc_trans}.
\item {\bf [-L]}
see section~\ref{sect_mc_trans}.
\item {\bf [-n number\_of\_scales]}
\item {\bf [-w]} \\
 Write to  disk the eigenvectors multiresolution files.
File names are: ``wk\_ev\_x.mr'',  where x is the eigenvector number.
Default is not to do this.
\item {\bf [-C]}
Write the correlation matrix to disk.
The file name is ``Correl\_Matrix''. Default is not to do this.
\item {\bf [-x]}
{\small
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item{0:} One correlation matrix per band.
\item{1:} One correlation matrix for all bands.
\item{2:} One correlation matrix for all band, except the last scale.
\item{3:} Import correlation matrix.
\end{itemize}
Default is one correlation matrix per band.
}
\item {\bf [-O Input\_CorrelMat\_FileName]} \\
Input correlation matrix file name.
Only used when option ``-x 3'' is set. Default is not to do this.
\item {\bf [-F NbrEigenVect]} \\
Number of eigenvectors used for the reconstruction.
Default is set to the number of images.
\item {\bf [-K EigenVect\_Number]} \\
Eigenvector number which will not be used for the reconstruction.
\end{itemize}

\subsubsection*{Examples}
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item wk\_trec in\_cube.fits out\_cube \\
Calculate the WT-KLT transform, and reconstruct only from the
first eigenvector of each scale.
\item wk\_trec -F 5 in\_cube.fits out\_cube  \\
Calculate the WT-KLT transform, and reconstruct from the
first five eigenvectors of each scale.
\item wk\_trec -F 5 -K 2 in\_cube.fits out\_cube  \\
Ditto, but do not use eigenvector number 2. Only 4 eigenvectors are
used.
\item wk\_trec -F 5 -K 2 -t24  trans.fits output.fits \\
Ditto, but use the undecimated wavelet transform instead of the
\`a trous algorithm.
\item wk\_trec -F 5 -K 1 -K 3 -t24  trans.fits output.fits \\
Only three eigenvectors (i.e., eigenvectors numbered 2,4,5) are used.
\end{itemize}

\section{Noise Modeling in the WT-KLT Space}
Since a WT-KLT coefficient $c$ is obtained by two successive linear
transforms, robust noise modeling can be derived in order to know
 the noise standard deviation associated with the $c$ value.

\subsection{Non-Gaussian noise}
If the noise in the data $D$ is Poisson, the Anscombe transformation
\cite{rest:anscombe48}
\begin{eqnarray}
t(D) = 2\sqrt{D + \frac{3}{8}}
\end{eqnarray}
acts as if the data arose from a
Gaussian white noise model, with $\sigma = 1$, under the
assumption that the mean value of $I$ is sufficiently large.
The arrival of photons, and their expression by electron counts, on CCD
detectors may be modeled by a Poisson distribution.  In addition, there is
additive Gaussian read-out noise. The Anscombe
transformation has been extended to take this combined noise into
account.  The  generalization of the variance stabilizing
Anscombe formula is derived as \cite{starck:book98}:
\begin{eqnarray}
t(D) = \frac{2}{g} \sqrt{g D + \frac{3}{8} g^2 + \sigma^2 - g m}
\end{eqnarray}
where $g$ is the electronic gain of the detector, $\sigma$ and $m$ the standard deviation
and the mean of the read-out noise.

This implies that for the filtering of an image with Poisson noise or
a mixture of Poisson and Gaussian noise, we will first pre-transform
the data $D$ into another one $t(D)$ with Gaussian noise. Then $t(D)$
will be filtered, and the filtered data will be inverse-transformed.

For other kinds of noise, modeling must be performed in order to
define the noise probability distribution of the wavelet coefficients
\cite{starck:book98}.
In the following, we will consider only stationary Gaussian noise.

\subsection{Noise level on WT-KLT coefficients}
Assuming a Gaussian noise standard deviation $\sigma_l$ for each signal
or image $d_l$, the noise in the wavelet space follows a Gaussian distribution
$\sigma_{l,j}$, $j$ being the scale index. For a bi-orthogonal wavelet
transform with a $L^2$ normalization, $\sigma_{l,j} = \sigma_{l}$ for all $j$.
Since the WT-KLT coefficients are obtained from a linear transform, we can
easily derive the noise standard deviation relative to a WT-KLT coefficient
from the noise standard deviation relative to the wavelet coefficients.
Considering the noise standard deviation vector
$s = \left\{\sigma_{1}, ..., \sigma_{L} \right\}$, we apply the
following transformation:
\begin{eqnarray}
y_j = A_j^2 s^2
\end{eqnarray}
and the noise standard deviation relative to a WT-KLT coefficient
$ c_{l}(j,k)$ is $\sqrt{y_j(l)}$.

\section{Multichannel Data Filtering}
\index{filtering}
\subsection{Introduction}
KLT based filtering methods have been proposed
in the past \cite{mc:andrews76,mc:lee91,mc:constant97} for single images.
The proposed idea was to decompose the image $I$ of $M \times N$ pixels
into non-overlapping blocks $B_s$ of size $N_b \times N_b$. Typically, $N_b$ takes
values from 4 to 16. Let $\lambda_1,\lambda_2,...\lambda_n$ be the
singular values of the matrix $I$ in decreasing order, and
assuming that the matrix $I$ is noisy, the rank $r$ of $I$
has been defined as \cite{mc:constant88}
\begin{eqnarray}
\lambda_r \ge \epsilon_1 > \lambda_{r+1}
\end{eqnarray}
where $\epsilon_1$ is the norm of the noise. In case of
a Gaussian distribution of zero mean, an upper bound is $\sqrt{MN}\sigma$
\cite{mc:constant97}.
A filtered version of $I$ can be obtained by reconstructing each block
only from its $r$ first eigenvalues.
An original approach has been developed in \cite{mc:natarajan95,mc:constant97}
in order to find the optimal
$\epsilon$ value, based on the $\epsilon$-compression ratio curve, using
a lossless compression method like JPEG. It has been found that the maximum
of the second derivative of the curve furnishes the
optimal $\epsilon$ value \cite{mc:natarajan95}.

In the case of multichannel data filtering, several different approaches
may be considered based on noise modeling. They are
presented in this section, and evaluated in the next one.

\subsection{Reconstruction from a subset of eigenvectors}
The WT-KLT transform of a data set $D=\left\{d_1, ..., d_L\right\}$
consists of applying a KLT on the wavelet scales. Hence the vector
$W_j = \left\{ W_j^{(1)}, ..., W_j^{(L)}\right\}$ of the scales
$j$ of the wavelet
transforms $W^{l}$ can be decomposed uniquely as:
\begin{eqnarray}
 W_j = U_j \Lambda_j^{1 \over 2}  V_j^{-1} = \sum_{i=1}^L
 \sqrt{\lambda_{j,i}}  {u}_{j,i}
 {v}_{j,i}^t
\end{eqnarray}
where $\Lambda_j$ is the diagonal matrix of eigenvalues of the correlation
matrix $C_j$,  $U_j$ and $V_j$ are orthogonal matrices with column vectors
$u_{j,i}$ and $v_{j,i}$ which are respectively
 the eigenvectors of $W_j W_j^t$ and $W_j^t W_j$.

% \overrightarrow{u}_{j,i}
The filtered wavelet coefficients of the band $j$ can be obtained by:
\begin{eqnarray}
\tilde W_j =  \sum_{i=1}^r \sqrt{\lambda_{j,i}}  {u}_{j,i} {v}_{j,i}^t
\end{eqnarray}
where $r$ is the rank of the matrix.

\subsection{WT-KLT Coefficient Thresholding}
Hard thresholding can  be applied to the  WT-KLT coefficients
in a fashion analogous to the thresholding of wavelet coefficients.

\subsubsection*{Example: astronomical source detection}

Figure~\ref{fig_exp_trans1} shows a simulation. We created
a dataset of 18 frames, each of them containing a source (i.e., a point
source, or idealized star) at the
same position,
but at different intensity levels. The source is a small Gaussian, and
the source SNR is defined as the ratio between the maximum of the source and
the noise standard deviation. Figure~\ref{fig_exp_trans1} (top) shows the
evolution of the SNR in the 18 frames. Frames two and ten
are shown in Figure~\ref{fig_exp_trans1}, middle
left and right. The source
SNR ratio is respectively three and one. Figure~\ref{fig_exp_trans1}, bottom
left and right, shows respectively frame ten filtered by the
 wavelet transform and the WT-KLT. The WT detects only noise, while the
WT-KLT clearly identifies the source.

\begin{figure}[htb]
\centerline{
\vbox{
\hbox{
\psfig{figure=transp1.ps,bbllx=0.1cm,bblly=18cm,bburx=11cm,bbury=25cm,width=8cm,height=5.2cm,clip=}
}
\hbox{
\psfig{figure=transp1.ps,bbllx=0.5cm,bblly=1cm,bburx=22cm,bbury=18cm,width=13.6cm,height=11.2cm,clip=}
}
}}
\caption{Simulation: the dataset is composed of 18 frames.
Each of them contains a source (small Gaussian) at the same position,
but at different intensity levels. Top, plot of the source maximum value versus
the frame number. Middle, frames 2 and 10, and bottom, filtered version
of the frame 10 by the wavelet transform and wavelet Karhunen-Lo\`eve transform.}
\label{fig_exp_trans1}
\end{figure}

\subsection{Multiscale Entropy}
The multiscale entropy relative to a set of observations $D(1..M)$ can
be written as:
\begin{eqnarray}
H(D) = \sum_{l=1}^{L} \sum_{j=1}^{J}  \sum_{k=1}^{N_j} h(c_{l,j,k})
\end{eqnarray}
where $J$ is the number of scales used in the wavelet transform
decomposition, $L$ the number of observations, $k$ a pixel position,
 $c$ WT-PCA coefficients, and $l$ denotes the eigenvector number.

The last scale of the wavelet transform is not used, as previously, so
this entropy measurement is background independent, which is really
important because the background can vary from one wavelength to another.

As for a wavelet coefficients in the case of mono-channel data,
we know the noise standard deviation relative to a coefficient,
and coefficients are of zero mean. Therefore, we can apply the same
filtering method. The filtered WT-PCA coefficients are found by minimizing
for each  $c_{l,j,k}$:
\begin{eqnarray}
j(\tilde c_{l,j,k}) = h_s(c_{l,j,k} -\tilde c_{l,j,k})
                      + \alpha h_n(\tilde c_{l,j,k})
\label{eqn_func_mc}
\end{eqnarray}

\subsubsection*{Example}

\begin{figure}[htb]
\centerline{
\hbox{
\psfig{figure=fig_simu20_noi12.ps,bbllx=3cm,bblly=2.8cm,bburx=18cm,bbury=17.4cm,width=7.5cm,height=8cm,clip=}
}}
\caption{Simulation: Root Mean Square Error versus the  noise standard deviation.
See text.}
\label{fig_simu20_noi12}
\end{figure}

Figure~\ref{fig_simu20_noi12} relates to a simulation using the same dataset
as in the previous example (i.e.,
a dataset of 18 frames).
Additive noise was used, and the data were filtered.
We  calculated the Root Mean Square Error
(RMSE) on each individual frame on a $5 \times
5$ square centered on the source.
Hence, the RMSE reflects well the photometric errors, and the addition
of the 18 RMSE values, which we call IRMSE (Integrated RMSE),
furnishes us a reliable measurement of the filtering quality.
The simulation was repeated with 12 noise levels, and four
different filtering methods were compared. Figure~\ref{fig_simu20_noi12}
shows the IRMSE versus the noise standard deviation plot. The four
methods are (i) multiscale entropy aaplied to
the WT-KLT coefficients (diamond),
(ii) reconstruction from a subset of eigenvectors of the KLT (triangle),
(iii) multiscale entropy applied to each frame independently (square),
and (iv) thresholding applied to the wavelet transform of each frame (star).
This simulation shows clearly that the approach proposed here, multiscale
entropy applied to the WT-KLT coefficients, outperforms all other methods.

\begin{figure}[htb]
\centerline{
\hbox{
\psfig{figure=fig_planck_simu8_noi12.ps,bbllx=2.5cm,bblly=2.8cm,bburx=18cm,bbury=17.4cm,width=7.5cm,height=8cm,clip=}
}}
\caption{Planck Simulation: Root Mean Square Error versus the  noise standard deviation.
See text.}
\label{fig_planck_simu8_noi12}
\end{figure}

The same experiments were performed using a simulated Planck data set.
Planck is an upcoming European Space Agency mission to carry out measurements
of the cosmic microwave background.
The data set contains ten images, each one being a linear combination of
6 sky component images (CMB, SZ, free-free, etc.). As in the previous
simulation, noise was added, and the data were filtered
by the four methods. The only difference is that the RMSE is calculated
on the full frames. Figure~\ref{fig_planck_simu8_noi12} shows
IRMSE versus the noise standard deviation plot. Diamonds, triangles,
squares and stars represent the same methods as before.
Again, the multiscale entropy applied to the WT-KLT coefficients outperforms
the other methods.

\subsection{WT-KLT Filtering of 1D Multichannel Data: wk1d\_filter}
\label{sect_mc1d_filter}
\index{wk1d\_filter}
The program
{\em wk1d\_filter} filters
a multichannel signal (2D data set).
{\bf
\begin{center}
 USAGE: wk1d\_filter option data\_in data\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
 \itemsep=0.1truecm

\item {\bf [-n number\_of\_scales]}\\
see section~\ref{sect_mc_trans}.
\item {\bf [-g sigma]}\\
 The image contains Gaussian noise, and the standard deviation is
given by {\em sigma}. This option should be set only if the user
knows the standard deviation of the noise.
\item {\bf [-x]} \\
see section~\ref{sect_mc_trans}.
% \item {\bf [-p]} \\
% Normalize the correlation matrix. Values on the diagonal are 1.
% Default is no.
\item {\bf [-O Input\_CorrelMat\_FileName]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-y NoiseCorrelationType]}
{\small
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item{0:} Compute the correlation matrix with all wavelet coefficients.
\item{1:} Compute the correlation matrix only with significant wavelet coefficients.
\item{2:} Compute the correlation matrix with weighted wavelet coefficients.
\end{itemize}
}
Default is 1.
\item {\bf [-s NSigma]} \\
Wavelet coefficients are significant when their absolute values are larger
than nsigma * SigmaNoise. This option has an effect when
the ``-y'' option is set to 1 or 2.
Default {\em NSigma} value is  3.
\item {\bf [-S NSigma]} \\
The WT-KLT coefficients are  at {\em NSigma}*SigmaNoise.
Default is  3.
\item {\bf [-P]}\\
Positivity constraint. Default is not to enforce this.
\item {\bf [-F NbrEigenVect]} \\
Number of eigenvectors used for the reconstruction.
Default is set to the number of images.
\item {\bf [-K EigenVect\_Number]} \\
Eigenvector number which will not be used for the reconstruction.
\item {\bf [-C]}\\
see section~\ref{sect_mc_trans}.
\end{itemize}

\subsubsection*{Examples}
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item wk1d\_filter input.fits output.fits \\
Filter the 1D multi-channel data set using the WT-KLT transform.
\item wk1d\_filter -S 5 input.fits output.fits \\
Ditto, but smooth the WT-KLT at 5 sigma instead of 3.
\end{itemize}

\subsection{WT-KLT Filtering of 2D Multichannel Data: wk\_filter}
\index{wk\_filter}
\label{sect_mc_filter}
The program
{\em wk\_filter} filters
a multichannel image (3D data set).
{\bf
\begin{center}
 USAGE: wk\_filter option data\_in data\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item {\bf [-t type\_of\_multiresolution\_transform]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-T type\_of\_filters]}  \\
see section~\ref{sect_mc_trans}.
\item {\bf [-L]} \\
see section~\ref{sect_mc_trans}.
% \item {\bf [-u]} \\
% see section~\ref{sect_mc_trans}.
\item {\bf [-n number\_of\_scales]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-g sigma]}\\
Noise standard deviation. Default is automatically calculated.
\item {\bf [-x]} \\
see section~\ref{sect_mc_trans}.
% \item {\bf [-p]} \\
% Normalize the correlation matrix. Values on the diagonal are 1.
% Default is no.
\item {\bf [-O Input\_CorrelMat\_FileName]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-y NoiseCorrelationType]}
{\small
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item{0:} Compute the correlation matrix with all wavelet coefficients.
\item{1:} Compute the correlation matrix only with significant wavelet coefficients.
\item{2:} Compute the correlation matrix with weighted wavelet coefficients.
\end{itemize}
}
Default is 1.
\item {\bf [-s NSigma]} \\
Wavelet coefficients are significant when their absolute values are larger
than nsigma * SigmaNoise. This option has an effect when
the ``-y'' option is set to 1 or 2.
Default {\em NSigma} value is  3.
\item {\bf [-S NSigma]} \\
The WT-KLT coefficients are  at {\em NSigma}*SigmaNoise.
Default is  3.
\item {\bf [-P]}\\
Positivity constraint. Default is not to enforce this.
\item {\bf [-b]}\\
Maximum image constraint at 255.  Default is not to use this.
\item {\bf [-w]}\\
see section~\ref{sect_mc_trans}.
\item {\bf [-F NbrEigenVect]} \\
Number of eigenvectors used for the reconstruction.
Default is set to the number of images.
\item {\bf [-K EigenVect\_Number]} \\
Eigenvector number which will not be used for the reconstruction.
\item {\bf [-C]}\\
see section~\ref{sect_mc_trans}.
\end{itemize}
\subsubsection*{Examples}
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item wk\_filter in\_cube.fits out\_cube.fits \\
Filter the cube using all default options.
\item wk\_filter -S 5 in\_cube.fits out\_cube.fits\\
Threshold the WT-KLT coefficients at 5 sigma, instead of 3 sigma.
\item wk\_filter -S 5 -t24 in\_cube.fits out\_cube.fits\\
Ditto, but use an undecimated wavelet transform.
\end{itemize}

\subsection{WT-KLT Filtering of 2D Multichannel Data by the Multiscale
Entropy Method: wk\_memfilter}
\index{wk\_memfilter}
The program
{\em wk\_memfilter} filters
a multichannel image (3D data set) by the multiscale entropy method.
{\bf
\begin{center}
 USAGE: wk\_memfilter option cube\_in cube\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item {\bf [-t type\_of\_multiresolution\_transform]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-T type\_of\_filters]}  \\
see section~\ref{sect_mc_trans}.
\item {\bf [-L]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-n number\_of\_scales]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-g sigma]}\\
see section~\ref{sect_mc1d_filter}.
\item {\bf [-x]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-O]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-y NoiseCorrelationType]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-s nsigma]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-P]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-b]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-w]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-F NbrEigenVect]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-K EigenVect\_Number]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-C CvgParam]} \\
Convergence parameter. Default is $1e-2$.
\item {\bf [-G RegulParam]}
 Regularization parameter. Default is 1.
\item {\bf [-U Type\_of\_Regularization]}
{\small
\begin{enumerate}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item Use a fixed user Alpha value.
\item Estimate the optimal Alpha.
\item Estimate one Alpha value per band.
\end{enumerate}}
Default is 1.
\item {\bf [-D]} \\
Alpha is modified using the data SNR. Default is no.
\item {\bf [-i MaxIter]} \\
 Maximum number of iterations. Default is 20.
\end{itemize}
\subsubsection*{Examples}
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item wk\_memfilter -U 3 in\_cube.fits out\_cube.fits \\
Filtering using one regularization parameter per resolution level.
\item wk\_memfilter -U 3 -D in\_cube.fits out\_cube.fits \\
Ditto, but protect wavelet coefficients with high SNR from the regularization.
\item wk\_memfilter -t24 -U 3 -D in\_cube.fits out\_cube.fits \\
Ditto, but use the undecimated wavelet transform.
\end{itemize}

\section{Filtering using the Haar-Multichannel Transform}
\subsection{Definition}
We have seen in \proj \ that the Haar transform
presents some advantages, especially when the data contains Poisson noise.

In order to decorrelate the information both spatially and in wavelength (or in
time), a 2D-wavelet transform
must first be performed on each frame of the date cube $D(x,y,z)$.
We denote the result $w_{j,k_x,k_y,z}$,
where $j$ is the scale index ($j \in [1,J]$, and
where $J$ is the number of scales), $k_x,k_y$ the
spatial position in the scale ($k_x \in [0,N_x-1]$, and $k_y \in [0,N_y-1]$),
 and $z$ is the frame number.
This set must again be transformed in order to
decorrelate the information in the third dimension. We apply a 1D-transform
to each vector $w_{j,k_x,k_y,z}$,
and we get a new set of data $u_{j,j^{'},k_x,k_y,k_z}$,
where $j^{'}$ and $k_z$ are respectively the scale index and position
in the third dimension.

Using the unnormalized Haar transform,
a coefficient $u_{j+1,j^{'}+1,k,l,t}$ can be written
as:
\begin{eqnarray}
u_{j+1,j^{'}+1,k_x,k_y,k_z}
   = \sum_{i=2^{j^{'}+1}k_z}^{2^{j^{'}}k_z+2^{j^{'}}-1} w_{j+1,k_x,k_y,i} -
     \sum_{i=2^{j^{'}+1}k_z+2^{j^{'}}}^{2^{j^{'}+1}(k_z+1)-1} w_{j+1,k_x,k_y,i}
\end{eqnarray}
We assume now a constant background rate $\lambda_i$ for each frame $i$
of the data cube.
Each Haar wavelet coefficient $w_{j+1,k_x,k_y,i}$ is the difference between two
random variables $X_i$ and $Y_i$, which follows a Poisson distribution of
parameter $\lambda_{j,i}$, where $\lambda_{j,i}$ represents the rate per
pixel over $2^{j+1}$ pixels of the $i$th frame, and is equal to $2^{2j}\lambda-i$.
Then $u_{j+1,j^{'}+1,k_x,k_y,k_z}$ is the difference of
 two variables, $X=\sum_i X_i$ and $Y=\sum_i Y_i$, and both follow a Poisson
 distribution of parameter $\sum_i \lambda_{j,i}$.
The thresholding method described in \proj
can therefore be used, using the correct $\lambda$ value.

\subsubsection*{Filtering}

The multichannel filtering algorithm is:
\begin{enumerate}
\item For each frame $D(*,*,z)$, apply the 2D-Haar transform: we obtain
$w_{j,k,l}(z)$.
\item For each scale $j$, and at each position $k_x,k_y$,
extract the 1D vector
 $w_{j,k_x,k_y,*}$ and compute its wavelet transform. We get a new data set.
\item Threshold the coefficients using the correct $\lambda value$.
\item Inverse 1D transform.
\item Inverse 2D transform.
\end{enumerate}

For non-constant background, a coarse to fine approach, as explained in
\proj, can be used.

\subsection{WT-KLT Filtering of 2D Multichannel Data by the Multiscale
Haar Transform: ww\_filter}
\index{ww\_filter}
The program
{\em ww\_filter} filters
a multichannel image (3D data set) by applying a 2D  WT following
by a 1D WT in the third direction.
{\bf
\begin{center}
 USAGE: ww\_filter option data\_in data\_out
\end{center}}
where options are:
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item {\bf [-t type\_of\_multiresolution\_transform]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-T type\_of\_filters]}  \\
see section~\ref{sect_mc_trans}.
\item {\bf [-L]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-n number\_of\_scales]} \\
see section~\ref{sect_mc_trans}.
\item {\bf [-g sigma]}\\
see section~\ref{sect_mc1d_filter}.
\item {\bf [-s nsigma]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-S nsigma]} \\
see section~\ref{sect_mc_filter}.
\item {\bf [-P]} \\
see section~\ref{sect_mc_filter}.
\end{itemize}
\subsubsection*{Examples}
\begin{itemize}
\baselineskip=0.4truecm
\itemsep=0.1truecm
\item ww\_filter -t 18 -T 3 input.fits output.fits \\
Filtering using the Haar  wavelet transform.
\item ww\_filter -t 24 -T 3 input.fits output.fits \\
Ditto, but use an undecimated wavelet transform.
\item ww\_filter  -t 24 -T 3 -S 5 input.fits output.fits \\
Increase the thresholding level.
\end{itemize}

\newpage

\section{Independent Component Analysis}
\index{independent component analysis}
\index{blind source separation}
\index{ICA}

The idea of independent component analysis (ICA) originates from
the problem of {\em blind source separation} (BSS), and consists of
recovering unobserved signals or ``sources'' from several observed mixtures
\cite{mc:cardoso98}. Assuming that $n$ statistically independent
signals $s_1(t), ..., s_n(t)$ are mixed by an unknown $n \times n$ mixing
matrix $A=[a_{ij}]$, we have:
\begin{eqnarray}
X(t) = A S(t)
\end{eqnarray}
where $S(t) = [ s_1(t), \dots, s_n(t) ]^t$, and
 $X(t) = [ x_1(t), \dots, x_n(t) ]^t$ represents the observed signals, with
 $x_i(t) = \sum_{j=1}^n a_{ij} s_j(t)$.
The challenge is to find how to achieve separation using as our
only assumption that
the source signals are statistically independent. The solution
consists of finding
an $n \times n$ separating matrix $B$, $Y(t) = B S(t)$, such that
$Y$ is an estimate of S.
This is achieved by minimizing contrast functions $\Phi$, which are defined
in terms of the Kullback-Leibler divergence  $K$:
\index{Kullback-Leibler}
\begin{eqnarray}
\Phi(Y) = \int p_Y(u) \log{ {p_Y(u) \over \prod p_{Y_i}(u_i) }} du
\end{eqnarray}

The mutual information, expressed by the  Kullback-Leibler divergence,
vanishes if and only if the variables
$Y_i$ are mutually independent, and is strictly positive otherwise.

ICA has been used in astronomy to analyze multispectral
images \cite{mc:bijaoui00} of the galaxy 3C 120 and to separate
\index{data!\texttt{3}C120}
the Cosmic Microwave Background
\index{Cosmic Microwave Background}
from other sky components \cite{mc:bacci00}.
As for PCA, it has been shown \cite{mc:zibu99,mc:zibu01} that applying
ICA to  wavelet transformed signals leads to better quality results,
especially in the presence of noise.

\subsection*{Example}
Fig.~\ref{fig_ica1} shows three simulated signals. Six observed data sets
were derived from the three sources, by a linear combination
(see Fig.~\ref{fig_ica2}). Without any knowledge of the mixing matrix,
the JADE-ICA \cite{mc:cardoso98} method was applied and the
three signals were
reconstructed from the six observed data sets (see Fig.~\ref{fig_ica3}).

\begin{figure}[htb]
\centerline{
\hbox{
\psfig{figure=fig_ica_source.ps,bbllx=2cm,bblly=12.5cm,bburx=20cm,bbury=25.5cm,width=9cm,height=7.5cm,clip=}
}}
\caption{Example of three simulated sources.}
\label{fig_ica1}
\end{figure}

\begin{figure}[htb]
\centerline{
\hbox{
\psfig{figure=fig_ica_obs_signal.ps,bbllx=2cm,bblly=12.5cm,bburx=20cm,bbury=25.5cm,width=9cm,height=7.5cm,clip=}
}}
\caption{Mixed sources. Each of these six signals is a linear combination
of the three simulated sources.}
\label{fig_ica2}
\end{figure}

\begin{figure}[htb]
\centerline{
\hbox{
\psfig{figure=fig_ica_demixing.ps,bbllx=2cm,bblly=12.5cm,bburx=20cm,bbury=25.5cm,width=9cm,height=7.5cm,clip=}
}}
\caption{Reconstructed sources from the mixed signals.}
\label{fig_ica3}
\end{figure}

\subsection{JADE-ICA IDL Programs}

\subsubsection{JADE}
\label{jade}
\index{JADE}
 Apply the Independant Componant Method to a set of vectors using
the JADE method.
{\bf
\begin{center}
 USAGE:  jade, ObservSig, NbSource, DeMixingMat, Process, Verbose=Verbose
\end{center}}
where
\begin{itemize}
\item {\em  ObservSig:} Input Data (ObservSig = $A$ \# original signal).
ObservSig$(i,*)$ = $i$th vector.
\item {\em  NbSource:}  Input number of sources in the input signal.
\item {\em DeMixingMat:} Output demixing matrix.
\item {\em Process:} Output Reconstructed process = DeMixingMat \# ObservSig.
Process$(i, *)$ = $i$th Process, with $i = 0.. {\mathrm NbSource} - 1$.
\end{itemize}

\subsubsection{JADE1D}
Apply the Independant Componant Method to a set of vectors using
the JADE method. The routine is identical to jade routine, except
that the input-output vectors are ordered differently ($V(*,i)$ for
the $i$th vector, instead of $V(i,*)$).
{\bf
\begin{center}
 USAGE:  jade1d, ObservSig, NbSource, DeMixingMat, Process, Verbose=Verbose
\end{center}}
where
\begin{itemize}
\item {\em  ObservSig:} Input Data (ObservSig = $A$ \# original signal).
;                    ObservSig$(*,i)$ = $i$th vector.
\item {\em  NbSource:}  Input number of sources in the input signal.
\item {\em DeMixingMat:} Output demixing matrix.
\item {\em Process:} Output Reconstructed process = DeMixingMat \# ObservSig.
Process$(*, i)$ = $i$th Process, with $i = 0.. {\mathrm NbSource} - 1$.
\end{itemize}

\subsubsection{JADE2D}
Apply the Independant Componant Method to a set of images using
the JADE method. The routine is an extension to 2d of jade1d routine.
The input set of images must be given with the following syntax:
ObservSig$(*, *, i)$ = $i$th image. If the ``/wave'' keyword is set,
a 2D wavelet transform is first perform on each image, JADE is applied
on the wavelet transformed images, and an inverse wavelet transform
is applied on the reconstructed process. The wavelet transform is
performed by calling the program {\em mr\_transform}.
{\bf
\begin{center}
 USAGE: jade2d, ObservSig, NbSource, DeMixingMat, Process, wave=wave, optw=optw, Verbose=Verbose
\end{center}}
where
\begin{itemize}
\item {\em  ObservSig:} Input Data (ObservSig = $A$ \# original signal).
ObservSig$(*,*,i)$ = $i$th vector.
\item {\em  NbSource:}  Input Number of sources in the input signal.
\item {\em DeMixingMat:} Output demixing matrix.
\item {\em Process:} Output Reconstructed process = DeMixingMat \# ObservSig.
Process$(*, *, i)$ = $i$th Process, with $i = 0.. {\mathrm NbSource} - 1$.
\item {\em wave::} if set, an bi-orthogonal wavelet transform is applied
on each image, and the component separation is performed
on the wavelet transformed images.
The inverse wavelet transform is then applied on the
reconstructed process.
\item {optw:} string = options for the wavelet transformation
\end{itemize}

\subsubsection*{Examples:}
\begin{itemize}
\item jade2d,  ObservIma, NbSource, DeMixingMat, RecIma \\
Apply the JADE method to {\em ObservIma}.
\item jade2d, ObservIma, NbSource, DeMixingMat, RecIma, /wave, optw='-n5'\\
Ditto, but applied a wavelet transform with five scales on each image.
\end{itemize}


\chapter{Introduction to Spreadsheets}

For many real-world problems, spreadsheets are the perfect
tool. In this chapter, you will be introduced to how to use a
spreadsheet. There are numerous spreadsheet programs: Google Sheets,
Microsoft Excel, Apple Numbers, OpenOffice Calc, etc.  All of them are
very similar. I will be using Google Sheets, but if you are using one
of the others, you should be able to follow along.

The first spreadsheet program (VisiCalc) was introduced in 1979 as a
tool for finance people to play ``what if'' games.  For example, a
company might make a spreadsheet that told them how much more profit
they would make if they changed from using an expensive metal to using
a cheaper alloy.\index{Spreadsheet}

In honor of its history, let's start by studying a business question:
I have a friend who dreams of quitting her job to become a cooper. (A
cooper makes barrels that are used for aging wine and whiskey.)  She
says:
\begin{itemize}
\item It costs \$45 dollars in materials to build one barrel.
\item A barrel sells for \$100 dollars.
\item The workshop/warehouse she wants to rent costs \$2000 per month.
\item Taxes would take about 20\% of her profits
\item She needs to make \$4000 after taxes to live
\end{itemize}

She has asked you, ``How many barrels do I need to make each month to survive?''

\section{Solving It Symbolically}

Many problems can be solved two ways: symbolically or
numerically.\index{symbolic vs. numeric solutions} To solve this
problem symbolically, you would write out the facts as equations or
inequalities and then do symbol manipulations until you ended up with
an answer. In this case, you would let $b$ be the number of barrells
and create the following inequality:

$$(1.0 - 0.2)\left(b(100 - 45) - 2000\right) \geq 4000$$

You would simplify it:

$$(0.8)\left(55 b - 2000\right) \geq 4000$$

And simplify it more:

$$44b - 1600 \geq 4000$$

If that is true, then:

$$44b \geq 5600$$

And if that is true, then:

$$b \geq \frac{1400}{11}$$

$1400/11$ is about 127.27, so she needs to make and sell 128 barrels
each month.

That is a perfect answer, and we didn't need a spreadsheet at all. Two things:
\begin{itemize}
\item As problems get larger and more realistic, it gets much more difficult to solve them symbolically.
\item As soon as you say ``Yes, you need to make and sell 128 barrels
  each month.'' Your friend will ask ``What if I make and sell 200
  barrels? How much money will I make then?''
\end{itemize}

So we will use a spreadsheet to solve the problem numerically.

\section{Solving It Numerically (with a spreadsheet)}

Let's get back to our example. Put labels in the A column:
\begin{itemize}
\item{Barrels produced (per month)}
\item{Materials cost (per barrel)}
\item{Sale price (per barrel)}
\item{Pre-tax earnings (per month)}
\item{Taxes (per month)}
\item{Take home pay (per month)}
\end{itemize}

Format them any way you like. It should look something like this:

\includegraphics[width=0.4\textwidth]{BarrelLabels.png}

In the B column, the first four cells are values (not formulas):
\begin{itemize}
\item{115 formatted as a number with no decimal point}
\item{45 formatted as currency}
\item{100 formatted as currency}
\item{2000 formatted as currency}
\end{itemize}

It should look something like this:

\includegraphics[width=0.5\textwidth]{BarrelValues.png}

The next three cells in the B column will have formulas:
\begin{itemize}
\item{B1 * (B3 - B2) - B4}
\item{0.2 * B5}
\item{B5 - B6}
\end{itemize}

It should look something like this:

\includegraphics[width=0.5\textwidth]{BarrelFormulas.png}

Now you can share this spreadsheet with your friend and she can put
different values into the cells for what-if games.  Like ``If I can
get my materials cost down to \$42 per barrel, what happens to my take
home pay?''

Sometimes it is nice to show a range of values for a variable or two.
In this case, it might be nice to show your friend what the numbers
look like if she produces 115, 120, 125, 130, 135, or 140 barrels per
month.

We have one column, and now we need six. How do we duplicate cells?
\begin{enumerate}
\item Click B1 to select it and then shift click on B7 to select all seven cells.
\item Copy them. (There is probably a menu item for this.)
\item Click C1 to select it
\item Paste them.
\end{enumerate}

\includegraphics[width=0.5\textwidth]{BarrelCopyPaste.png}

We want the first cell in the new column to be 120. You could just
type in 120, but lets do something more clever.  Put a formula into that
cell: = B1 + 5.  Now the cell should show 120.

Why did we put in a formula? When we duplicate this column, this cell
will always have 5 more barrels than the cell to its left.

Now lets duplicate the second column a few times. The easy way to do
this is to select the cells as you did before and drag the lower-right
corner to the right until column G is in the selection. When you end
the drag, the copies will appear:

\includegraphics[width=0.8\textwidth]{BarrelDragPaste.png}

Nice, right? Now your friend can easily see how many barrels
corresponds to how much take-home pay. You know what she really wants? A graph.

\section{Graphing}

Graphing is a little different on every different platform.  Here is what you want the graph to look like.

\includegraphics[width=0.8\textwidth]{BarrelGraph.png}

On Google Sheets:\index{spreadsheet!graphs}

\begin{enumerate}
\item Select cells B7 through G7.
\item Choose the menu item Insert -> Chart.
\item Choose the chart type (Line)
\item Add the X-axis to be B1 through G1.
\item Under the Customize tab, Set the label for the X-axis to be ``Barrels Made and Sold''.
\item Delete the chart title (which is the same as the Y-axis label).
\end{enumerate}

\section{Other Things You Should Know About Spreadsheets}

Your spreadsheet document can have several ``Sheets''.  Each has its
own grid of cells.  The sheet has a name; usually you call it
something like ``Salaries''.  When you need to use a value from the
``Salaries'' sheet in another sheet, you can specify ``Salaries!A2''
-- that is, cell A2 on sheet ``Salaries''.  To flip between the sheets
there is usually a tab for each at the bottom of the document.

\includegraphics[width=0.5\textwidth]{Sheets.png}

By default, the cell references are relative.  That is, when you write
a formula in cell H5 that references the value in cell G4, the cell
remembers ``The cell that is one up and one to the left of me.''
Thus, if you copy that formula into B9, now that formula reads the
value from A8.

If you want an absolute reference, you use \$.  If H5 references
\$G\$4, G4 will be used no matter where on the sheet the formula is
copied to.

You can use the \$ on the row or colum.  In \$A4, the column is
absolute and the row is relative.  In A\$4, the row is absolute and
the column is relative.

\section{Challenge: Make a spreadsheet}

You have a company that bids on painting jobs. Make a
spreadsheet to help you do bids. Here are the parameters:
\begin{itemize}
\item The client will tell you how many square meters of wall need to be painted.
\item Paint costs \$0.02 per square meter of wall to cover
\item On average, a square meter of wall takes 0.02 hours to paint.
\item You can hire painters at \$15 per hour.
\item You add 20\% to your estimated costs for a margin of error and profit.
\end{itemize}

Make a spreadsheet such that when you type in the square meters to be
painted, the spreadsheet tells you how much you will spend on paint
and labor.  It also tells you what your bid should be.


\chapter{Conclusion}
In conclusion, different MIP exact model have been developed to resolve the TSP problem and \texttt{subtour\_callback\_general} has definitely obtain the best results in term of times. However no exact algorithm can resolve a problem with more than 1000 nodes in less than 600s. This must be considered in particular when the execution time is limited. In this case heuristics can be successfully applied to obtain a good solution (not the best) about 10 times faster. This result are confirmed with the performance profile in fig \ref{fig:pp_Lbest}.\\
For the heuristics, \texttt{best\_two\_opt} obtain really good results in term of execution time and cost optimization, indeed the best heuristic are the constructive ones combined with the iteration of \texttt{best\_two\_opt}.
The constructive \texttt{n\_greedy} achieves the best performances as a heuristic alone but the use of the \texttt{n\_grasp} as a warm start brings a more variability.
\begin{figure}[!h]
	\centering
	\begin{subfigure}{.49\textwidth}
		\centering
		\includegraphics[width=\columnwidth]{../res/Lbest_time.png}
		\caption{Performance profile in time domain.}
		\label{fig:Lbest_time}
	\end{subfigure}
\hfill
	\begin{subfigure}{.49\textwidth}
		\centering
		\includegraphics[width=\columnwidth]{../res/Lbest_lb.png}
		\caption{Performance profile in solution cost domain.}
		\label{fig:Lbest_lb}
	\end{subfigure}
	\caption{Performance profile of the best models executed in the union of \textit{data light} and \textit{data average}.}
	\label{fig:pp_Lbest}
\end{figure}


\documentclass[letterpaper,12pt]{article}
% \usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{ifpdf}

% \usepackage[spanish,mexico]{babel}
\usepackage{multicol}
\usepackage{tikz}

\usepackage{amssymb}
\usepackage{amsmath}

%opening
\title{Lecture 2}
\author{Isaac Ayala Lozano}

\begin{document}

% \maketitle

\section{Lecture 2 - Exercise 8}

Calculate the velocity, speed and acceleration for each
position vector.

% --------------------------

\subsection{$\vec{r}(t)= (\cos (\omega t), \mathrm{e}^{\omega t})$}

The first derivative of the vector gives us

\begin{equation*}
 \vec{v}(t) = (- \omega \sin (\omega t), \omega \mathrm{e}^{\omega t})
 \end{equation*}

The norm of the velocity vector gives us the speed.

 \begin{equation*}
  \left \| \vec{v}(t) \right \| = \sqrt{ \omega^2 \sin^2 (\omega t) + \omega ^2
\mathrm{e}^{2 \omega t}}
 \end{equation*}

 The second derivative results in

 \begin{equation*}
  \vec{a}(t) = (-\omega ^2 \cos (\omega t) , \omega ^2
\mathrm{e}^{\omega t})
 \end{equation*}

Figure \ref{fig: vector 1} shows the plots for all three vectors with
conditions $\omega = 1$, and $t \in \left [ 0, 2 \pi \right ] $.

\begin{figure}[h]
 \centering

\includegraphics[scale=0.65,keepaspectratio=true]{./img/graph01.png}
 \caption{Position, Velocity and Acceleration plots for
$\vec{r}(t)= (\cos (\omega t), \mathrm{e}^{\omega t})$
}
 \label{fig: vector 1}
\end{figure}

% -----------------------

\subsection{$\vec{r}(t) = ( \cos (\omega t - \phi), \sin (\omega t - \phi))$}

Using the following trigonometric identities

\begin{equation*}
 \begin{split}
  \sin (\alpha \pm \beta) & = \sin \alpha \cos \beta \pm \cos \alpha \sin \beta
\\
\cos (\alpha \pm \beta & = \cos \alpha \cos \beta \mp \sin \alpha \sin \beta
 \end{split}
\end{equation*}

the position vector can be rewritten as follows

\begin{equation*}
 \vec{r} = (
 \cos \omega t \cos \phi + \sin \omega t \sin \phi,
 \sin \omega t \cos \phi - \cos \omega t \sin \phi
 )
\end{equation*}

The velocity, speed and acceleration are as follows

 \begin{equation*}
 \begin{split}
 \vec{v}(t) & = ( - \omega \cos \phi \sin \omega t + \omega \sin \phi \cos
\omega t, \omega \cos \phi \cos \omega t + \omega \sin \phi \sin \omega t) \\
& = (- \omega \sin (\omega t + \phi), \omega \cos (\omega t + \phi))
 \end{split}
 \end{equation*}

\begin{equation*}
 \left \| \vec{v}(t) \right \| = \sqrt{ \omega ^2 \sin ^2 (\omega t + \phi) +
\omega ^ 2 \cos ^2 (\omega t + \phi)}
\end{equation*}

 \begin{equation*}
 \begin{split}
  \vec{a}(t) & =  ( - \omega ^2 \cos \phi \cos \omega t - \omega ^2 \sin \phi
\sin \omega t, - \omega ^2 \cos \phi \sin \omega t + \omega ^2 \sin \phi \cos
\omega t)\\
& = ( - \omega ^2 \cos (\omega t - \phi), -\omega ^ 2 \sin (\omega t - \phi))
 \end{split}
 \end{equation*}

 Figure \ref{fig: vector 2} shows the plotted movement equations for the
system. Notice that all three curves overlap when evaluated with $\omega = 1$,
 $t \in \left [ 0, 2 \pi \right ] $ and $\phi = \frac{\pi}{4}$.

 \begin{figure}[h]
 \centering
\includegraphics[scale=0.4,keepaspectratio=true]{./img/graph02.png}
 \caption{Position, Velocity and Acceleration plots for
$\vec{r}(t) = ( \cos (\omega t - \phi), \sin (\omega t - \phi))$
}
 \label{fig: vector 2}
\end{figure}

% ----------------------

\subsection{$\vec{r}(t) = ( c \cos ^3 (t), c \sin ^3 (t))$}

Apply the chain rule to obtain the derivatives of $\vec{r}(t)$.

 \begin{equation*}
  \vec{v}(t) = (- 3 c \cos ^2 (t) \sin (t), 3 c \sin ^ 2 (t) \cos (t))
 \end{equation*}

\begin{equation*}
 \left \| \vec{v}(t) \right \| = \sqrt{ 9 c^2 \cos ^4 (t) \sin^2 (t) + 9 c^2
\sin ^4 (t) \cos ^2 (t)}
\end{equation*}

 \begin{equation*}
  \vec{a}(t) = (- 3 c \cos ^3 (t) - 6 c \sin ^2 (t) \cos (t), -3 c \sin ^3 (t)
+ 6 c \cos ^2 (t) \sin (t))
 \end{equation*}

 All three vectors are plotted in figure \ref{fig: vector 3} with $t \in
\left [ 0, 2 \pi \right ] $, and $c = 1$.

 \begin{figure}[h]
 \centering

\includegraphics[scale=0.4,keepaspectratio=true]{./img/graph03.png}
 \caption{Position, Velocity and Acceleration plots for
$\vec{r}(t) = ( c \cos ^3 (t), and c \sin ^3 (t))$.
}
 \label{fig: vector 3}
\end{figure}

% ---------------------------

\subsection{$\vec{r}(t) = ( c (t - \sin t), c (1 - \cos t))$}

  \begin{equation*}
  \vec{v}(t) = (c (1 - \cos (t) ), c \sin (t))
 \end{equation*}

\begin{equation*}
 \left \| \vec{v}(t) \right \| = \sqrt{ c^2 - c^2 \cos ^2 (t) + c^2 \sin ^2 (t)}
\end{equation*}

 \begin{equation*}
  \vec{a}(t) = (c \sin (t), c \cos (t))
 \end{equation*}

 Figure \ref{fig: vector 4} shows the plots for all three equations evaluated
with $c = 1$ and $t \in
\left [ 0, 2 \pi \right ] $.

 \begin{figure}[h]
 \centering

\includegraphics[scale=0.4,keepaspectratio=true]{./img/graph04.png}
 \caption{Position, Velocity and Acceleration plots for
$\vec{r}(t) = ( c (t - \sin t), c (1 - \cos t))$
}
 \label{fig: vector 4}
\end{figure}

\end{document}


% This LaTeX was auto-generated from an M-file by MATLAB.
% To make changes, update the M-file and republish this document.

\subsection*{maha.m}

\begin{par}
\textbf{Summary:} Point-wise squared Mahalanobis distance (a-b)*Q*(a-b)'. Vectors are row-vectors
\end{par} \vspace{1em}

\begin{verbatim}  function K = maha(a, b, Q)\end{verbatim}
    \begin{par}
\textbf{Input arguments:}
\end{par} \vspace{1em}
\begin{verbatim}a   matrix containing n row vectors                                 [n x D]
b   matrix containing n row vectors                                 [n x D]
Q   weight matrix. Default: eye(D)                                  [D x D]\end{verbatim}
\begin{par}
\textbf{Output arguments:}  K    point-wise squared distances                                    [n x n]
\end{par} \vspace{1em}
\begin{par}
Copyright (C) 2008-2013 by Marc Deisenroth, Andrew McHutchon, Joe Hall, and Carl Edward Rasmussen.
\end{par} \vspace{1em}
\begin{par}
Last modified: 2013-03-21
\end{par} \vspace{1em}

\begin{lstlisting}
function K = maha(a, b, Q)
\end{lstlisting}

\subsection*{Code}

\begin{lstlisting}
if nargin == 2                                                  % assume unit Q
  K = bsxfun(@plus,sum(a.*a,2),sum(b.*b,2)')-2*a*b';
else
  aQ = a*Q; K = bsxfun(@plus,sum(aQ.*a,2),sum(b*Q.*b,2)')-2*aQ*b';
end
\end{lstlisting}


\chapter{Homomorphisms and quotient groups}
\section{Generators and group presentations}
\prototype{$D_{2n} = \left< r,s \mid r^n=s^2=1\right>$}

Let $G$ be a group.
Recall that for some element $x \in G$,
we could consider the subgroup
\[ \left\{ \dots, x^{-2}, x^{-1}, 1, x, x^2, \dots \right\} \]
of $G$.
Here's a more pictorial version of what we did:
\textbf{put $x$ in a box, seal it tightly, and shake vigorously}.
Using just the element $x$,
we get a pretty explosion that produces the subgroup above.

What happens if we put two elements $x$, $y$ in the box?
Among the elements that get produced are things like
\[ xyxyx, \quad x^2y^9x^{-5}y^3, \quad y^{-2015}, \quad \dots\]
Essentially, I can create any finite product of $x$, $y$, $x\inv$, $y\inv$.
This leads us to define:
\begin{definition}
	Let $S$ be a subset of $G$.
	The subgroup \vocab{generated} by $S$,
	denoted $\left<S\right>$,
	is the set of elements which can be written as a
	finite product of elements in $S$ (and their inverses).
	If $\left<S\right> = G$ then we say $S$ is a set of
	\vocab{generators} for $G$,
	as the elements of $S$ together create all of $G$.
\end{definition}
\begin{exercise}
	Why is the condition ``and their inverses''
	not necessary if $G$ is  a finite group?
	(As usual, assume Lagrange's theorem.)
\end{exercise}

\begin{example}[$\ZZ$ is the infinite cyclic group]
	Consider $1$ as an element of $\ZZ = (\ZZ, +)$.
	We see $\left<1\right> = \ZZ$, meaning $\{1\}$ generates $\ZZ$.
	It's important that $-1$, the inverse of $1$ is also allowed:
	we need it to write all integers as the sum of $1$ and $-1$.
\end{example}

This gives us an idea for a way to try and express groups compactly.
Why not just write down a list of generators for the groups?
For example, we could write
\[ \ZZ \cong \left< a \right> \]
meaning that $\ZZ$ is just the group generated by one element.

There's one issue: the generators usually satisfy certain properties.
For example, consider $\Zc{100}$.
It's also generated by a single element $x$,
but this $x$ has the additional property that $x^{100} = 1$.
This motivates us to write
\[ \Zc{100} = \left< x \mid x^{100} = 1 \right>. \]
I'm sure you can see where this is going.
All we have to do is specify a set of generators and
\vocab{relations} between the generators,
and say that two elements are equal if and only if
you can get from one to the other using relations.
Such an expression is appropriately called a \vocab{group presentation}.

\begin{example}[Dihedral group]
	The dihedral group of order $2n$ has a presentation
	\[ D_{2n} = \left< r, s
		\mid r^n = s^2 = 1, rs = sr\inv \right>. \]
	Thus each element of $D_{2n}$ can be written uniquely in the form $r^\alpha$
	or $sr^\alpha$, where $\alpha = 0, 1, \dots, n-1$.
\end{example}

\begin{example}[Klein four group]
	The \vocab{Klein four group},
	isomorphic to $\Zc2 \times \Zc2$, is given by the presentation
	\[ \left< a,b \mid a^2=b^2=1, ab=ba \right>. \]
\end{example}

\begin{example}
	[Free group]
	The \vocab{free group on $n$ elements} is the group
	whose presentation has $n$ generators and no relations at all.
	It is denoted $F_n$, so
	\[
		F_n = \left< x_1, x_2, \dots, x_n \right>.
	\]
	In other words, $F_2 = \left<a,b\right>$ is the set of strings
	formed by appending finitely many copies of $a$, $b$, $a\inv$, $b\inv$ together.
\end{example}
\begin{ques}
	Notice that $F_1 \cong \ZZ$.
\end{ques}
\begin{abuse}
	One might unfortunately notice that ``subgroup generated by $a$ and $b$''
	has exactly the same notation as the free group $\left<a,b\right>$.
	We'll try to be clear based on context which one we mean.
\end{abuse}

Presentations are nice because they provide a compact way to write down groups.
They do have some shortcomings, though.\footnote{%
Actually, determining whether two elements of a presentation are equal is undecidable.
In fact, it is undecidable to even determine if a group is finite from its presentation.}

\begin{example}
	[Presentations can look very different]
	The same group can have very different presentations.
	For instance consider
	\[ D_{2n} = \left< x,y \mid x^2=y^2=1, (xy)^n=1. \right>. \]
	(To see why this is equivalent, set $x=s$, $y=rs$.)
\end{example}

\section{Homomorphisms}
\prototype{The ``mod out by $100$'' map, $\ZZ \to \Zc{100}$.}

How can groups talk to each other?

Two groups are ``the same'' if we can write an isomorphism between them.
And as we saw, two metric spaces are ``the same''
if we can write a homeomorphism between them.
But what's the group analogy of a continuous map?
We simply drop the ``bijection'' condition.

\begin{definition}
	Let $G = (G, \star)$ and $H = (H, \ast)$ be groups.
	A \vocab{group homomorphism} is a map $\phi : G \to H$
	such that for any $g_1, g_2 \in G$ we have
	\[ \phi(g_1 \star g_2) = \phi(g_1) \ast \phi(g_2). \]
\end{definition}

\begin{example}
	[Examples of homomorphisms]
	Let $G$ and $H$ be groups.
	\begin{enumerate}[(a)]
		\ii Any isomorphism $G \to H$ is a homomorphism.
		In particular, the identity map $G \to G$ is a homomorphism.
		\ii The \vocab{trivial homomorphism} $G \to H$ sends
		everything to $1_H$.
		\ii There is a homomorphism from $\ZZ$ to $\Zc{100}$ by
		sending each integer to its residue modulo $100$.
		\ii There is a homomorphism from $\ZZ$ to itself by $x \mapsto 10x$
		which is injective but not surjective.
		\ii There is a homomorphism from $S_n$ to $S_{n+1}$ by ``embedding'':
		every permutation on $\{1,\dots,n\}$ can be thought of as a permutation
		on $\{1,\dots,n+1\}$ if we simply let $n+1$ be a fixed point.
		\ii A homomorphism $\phi: D_{12} \to D_6$
		is given by $s_{12} \mapsto s_6$
		and $r_{12} \mapsto r_6$.
		\ii Specifying a homomorphism $\ZZ \to G$ is the same as
		specifying just the image of the element $1 \in \ZZ$. Why?
	\end{enumerate}
\end{example}
The last two examples illustrates something: suppose we have a presentation of $G$.
To specify a homomorphism $G \to H$, we only have to specify where each generator of $G$ goes, in such a way that the relations are all satisfied.

Important remark:
the right way to think about an isomorphism is as a ``bijective homomorphism''.
To be explicit,
\begin{exercise}
	Show that $G \cong H$ if and only if there exist
	homomorphisms $\phi \colon G \to H$ and $\psi \colon H \to G$
	such that $\phi \circ \psi = \id_H$ and $\psi \circ \phi = \id_G$.
\end{exercise}
So the definitions of homeomorphism of metric spaces
and isomorphism of groups are not too different.

Some obvious properties of homomorphisms follow.
\begin{fact}
	Let $\phi \colon G \to H$ be a homomorphism.
	Then $\phi(1_G) = 1_H$ and $\phi(g\inv) = \phi(g)\inv$.
\end{fact}
\begin{proof}
	Boring, and I'm sure you could do it yourself if you wanted to.
\end{proof}

Now let me define a very important property of a homomorphism.
\begin{definition}
	The \vocab{kernel} of a homomorphism $\phi \colon G \to H$ is defined by
	\[ \ker \phi \defeq
		\left\{ g \in G : \phi(g) = 1_H \right\}.
		\]
	It is a \emph{subgroup} of $G$
	(in particular, $1_G \in \ker \phi$ for obvious reasons).
\end{definition}
\begin{ques}
	Verify that $\ker\phi$ is in fact a subgroup of $G$.
\end{ques}
We also have the following important fact, which we also encourage the reader to verify.
\begin{proposition}
	[Kernel determines injectivity]
	The map $\phi$ is injective if and only if $\ker\phi = \{1_G\}$.
\end{proposition}

To make this concrete, let's compute the kernel of each of our examples.
\begin{example}
	[Examples of kernels]
	\listhack
	\begin{enumerate}[(a)]
		\ii The kernel of any isomorphism $G \to H$ is trivial,
		since an isomorphism is injective.
		In particular, the kernel of the identity map $G \to G$ is $\{1_G\}$.
		\ii The kernel of the trivial homomorphism $G \to H$
		(by $g \mapsto 1_H$) is all of $G$.
		\ii The kernel of the homomorphism $\ZZ \to \Zc{100}$ by $n \mapsto \ol n$
		is precisely \[ 100\ZZ = \{\dots, -200, -100, 0, 100, 200, \dots \}. \]
		\ii The kernel of the map $\ZZ \to \ZZ$ by $x \mapsto 10x$ is trivial: $\{0\}$.
		\ii There is a homomorphism from $S_n$ to $S_{n+1}$ by ``embedding'',
		but it also has trivial kernel because it is injective.
		\ii A homomorphism $\phi\colon D_{12} \to D_6$
		is given by $s_{12} \mapsto s_6$ and $r_{12} \mapsto r_6$.
		You can check that
		\[ \ker \phi = \left\{ 1, r_{12}^{3} \right\} \cong \Zc2. \]
		\ii Exercise below.
	\end{enumerate}
\end{example}
\begin{exercise}
	Fix any $g \in G$.
	Suppose we have a homomorphism $\ZZ \to G$ by $n \mapsto g^n$.
	What is the kernel?
\end{exercise}

\begin{ques}
	Show that for any homomorphism $\phi: G \to H$,
	the image $\phi\im(G)$ is a subgroup of $H$.
	Hence, we'll be especially interested in the case where $\phi$ is surjective.
\end{ques}

\section{Cosets and modding out}
\prototype{Modding out by $n$: $\ZZ / (n \cdot \ZZ) \cong \Zc n$.}
\emph{The next few sections are a bit dense.
If this exposition doesn't work for you, try \cite{ref:gowers}.}

Let $G$ and $Q$ be groups, and suppose there exists
a \emph{surjective} homomorphism \[ \phi : G \surjto Q. \]
In other words, if $\phi$ is injective then $\phi : G \to Q$ is a bijection,
and hence an isomorphism.
But suppose we're not so lucky and $\ker\phi$ is bigger than just $\{1_G\}$.
What is the correct interpretation of a more general homomorphism?

Let's look at the special case where $\phi : \ZZ \to \Zc{100}$ is ``modding out by $100$''.
We already saw that the kernel of this map is
\[
	\ker \phi = 100\ZZ = \left\{ \dots, -200, -100, 0, 100, 200, \dots  \right\}.
\]
Recall now that $\ker \phi$ is a subgroup of $G$.
What this means is that \textbf{$\phi$ is indifferent to the subgroup $100\ZZ$ of $\ZZ$}:
\[ \phi(15) = \phi(2000 + 15) = \phi(-300 + 15) = \phi(700 + 15) = \dots. \]
So $\Zc{100}$ is what we get when we ``mod out by $100$''. Cool.

In other words, let $G$ be a group and $\phi : G \surjto Q$
be a surjective homomorphism with kernel $N \subseteq G$.
\begin{moral}
	We claim that $Q$ should be thought of as the quotient of $G$ by $N$.
\end{moral}
To formalize this, we will define a so-called
\vocab{quotient group} $G/N$
in terms of $G$ and $N$ only (without referencing $Q$)
which will be naturally isomorphic to $Q$.

For motivation, let's give a concrete description of $Q$ using just $\phi$ and $G$.
Continuing our previous example, let $N = 100\ZZ$ be our subgroup of $G$.
Consider the sets
\begin{align*}
	N &= \left\{ \dots, -200, -100, 0, 100, 200, \dots \right\} \\
	1+N &= \left\{ \dots, -199, -99, 1, 101, 201, \dots \right\} \\
	2+N &= \left\{ \dots, -198, -98, 2, 102, 202, \dots \right\} \\
	&\vdots \\
	99+N &= \left\{ \dots, -101, -1, 99, 199, 299, \dots \right\}.
\end{align*}
The elements of each set all have the same image when we apply $\phi$,
and moreover any two elements in different sets have different images.
Then the main idea is to notice that
\begin{moral}
	We can think of $Q$ as the group
	whose \emph{elements} are the \emph{sets} above.
\end{moral}

Thus, given $\phi$ we define an equivalence relation $\sim_N$
on $G$ by saying $x \sim_N y$ for $\phi(x) = \phi(y)$.
This $\sim_N$ divides $G$ into several equivalence classes in $G$
which are in obvious bijection with $Q$, as above.
Now we claim that we can write these equivalence classes very explicitly.

% Also, for each $g \in G$ define $\ol g$ to be the equivalence class of $g$ under $\sim_\phi$.
\begin{exercise}
	Show that $x \sim_N y$ if and only if $x = yn$ for some $n \in N$
	(in the mod $100$ example, this means they ``differ by some multiple of $100$'').
	Thus for any $g \in G$, the equivalence class of $\sim_N$ which contains $g$
	is given explicitly by \[ gN \defeq \left\{ gn \mid n \in N \right\}. \]
\end{exercise}
%Note that
%\[ \phi(x) = \phi(y)
%	\implies 1_Q = \phi(x)\phi(y)\inv = \phi(xy\inv)
%	\implies xy\inv \in \ker\phi. \]
%Hence another way to describe $\ol x$ is
%\[ \ol g = gN = \left\{ gn \mid n \in N \right\}. \]

Here's the word that describes the types of sets we're running into now.
\begin{definition}
	Let $H$ be any subgroup of $G$ (not necessarily the kernel of some homomorphism).
	A set of the form $gH$ is called a \vocab{left coset} of $H$.
\end{definition}
\begin{remark}
	Although the notation might not suggest it,
	keep in mind that $g_1N$ is often equal to $g_2N$ even if $g_1 \neq g_2$.
	In the ``mod $100$'' example, $3+N = 103+N$.
	In other words, these cosets are \emph{sets}.

	This means that if I write ``let $gH$ be a coset'' without telling you what $g$ is,
	you can't figure out which $g$ I chose from just the coset itself.
	If you don't believe me, here's an example of what I mean:
	\[ x+100\ZZ = \left\{ \dots, -97, 3, 103, 203, \dots \right\} \implies x = {?}. \]
	There's no reason to think I picked $x=3$. (I actually picked $x=-13597$.)
	\label{remark:coset_warning}
\end{remark}
\begin{remark}
	Given cosets $g_1H$ and $g_2H$,
	you can check that the map $x \mapsto g_2g_1\inv x$ is a bijection between them.
	So actually, all cosets have the same cardinality.
\end{remark}

So, long story short,
\begin{moral}
	Elements of the group $Q$ are naturally identified with left cosets of $N$.
\end{moral}
In practice, people often still prefer to picture elements of $Q$ as single points
(for example it's easier to think of $\Zc2$ as $\{0,1\}$
rather than $\big\{ \{\dots,-2,0,2,\dots\}, \{\dots,-1,1,3,\dots\} \big\}$).
If you like this picture,
then you might then draw $G$ as a bunch of equally tall fibers (the cosets),
which are then ``collapsed'' onto $Q$.

\begin{center}
	\begin{asy}
		size(9cm);
		for (int i=1; i<=6; ++i) {
			draw( (i,0)--(i,3.4) );
			dot( (i,0.6) );
			dot( (i,1) );
			dot( (i,1.7) );
			dot( (i,2.4) );
			dot( (i,2.8) );
			dot( (i,3) );
			label(rotate(90)*scale(1.4)*"$\Longleftarrow$", (i+0.02,-0.9), dir(90));
			dot( (i,-1) );
		}
		label("$G$", (0.5, 3));
		label("$Q$", (0.5,-1));
	\end{asy}
\end{center}

Now that we've done this, we can give an \emph{intrinsic}
definition for the quotient group we alluded to earlier.
\begin{definition}
	A subgroup $N$ of $G$ is called \vocab{normal} if it is the
	kernel of some homomorphism.
	We write this as $N \normalin G$.
\end{definition}
\begin{definition}
	Let $N \normalin G$.
	Then the \vocab{quotient group}, denoted $G/N$
	(and read ``$G$ mod $N$''),
	is the group defined as follows.
	\begin{itemize}
		\ii The elements of $G/N$ will be the left cosets of $N$.
		\ii We want to define the product of two cosets $C_1$ and $C_2$ in $G/N$.
		Recall that the cosets are in bijection with elements of $Q$.
		So let $q_1$ be the value associated to the coset $C_1$,
		and $q_2$ the one for $C_2$.
		Then we can take the product to be the coset corresponding to $q_1q_2$.

		Quite importantly,
		\textbf{we can also do this in terms of representatives of the cosets}.
		Let $g_1 \in C_1$ and $g_2 \in C_2$,
		so $C_1 = g_1N$ and $C_2 = g_2N$.
		Then $C_1 \cdot C_2$ should be the coset which contains $g_1g_2$.
		This is the same as the above definition since
		$\phi(g_1g_2) = \phi(g_1)\phi(g_2) = q_1q_2$;
		all we've done is define the product in terms of elements of $G$,
		rather than values in $H$.

		Using the $gN$ notation,
		and with \Cref{remark:coset_warning} in mind,
		we can write this even more succinctly:
		\[ (g_1N) \cdot (g_2N) \defeq (g_1g_2)N. \]
	\end{itemize}
\end{definition}
And now you know why the integers modulo $n$ are often written $\ZZ/n\ZZ$!
\begin{ques}
	Take a moment to digest the above definition.
\end{ques}
By the way we've built it, the resulting group $G/N$ is isomorphic to $Q$.
In a sense we think of $G/N$ as ``$G$ modulo the condition that $n=1$
for all $n \in N$''.

\section{(Optional) Proof of Lagrange's theorem}
As an aside, with the language of cosets
we can now show Lagrange's theorem in the general case.
\begin{theorem}
	[Lagrange's theorem]
	\label{thm:lagrange_grp}
	Let $G$ be a finite group, and let $H$ be any subgroup.
	Then $\left\lvert H \right\rvert$ divides $\left\lvert G \right\rvert$.
\end{theorem}

The proof is very simple: note that the cosets of $H$
all have the same size and form a partition of $G$
(even when $H$ is not necessarily normal).
Hence if $n$ is the number of cosets,
then $n \cdot \left\lvert H \right\rvert = \left\lvert G \right\rvert$.

\begin{ques}
	Conclude that $x^{\left\lvert G \right\rvert}=1$
	by taking $H = \left<x\right> \subseteq G$.
\end{ques}

\begin{remark}
	It should be mentioned at this point that
	in general, if $G$ is a finite group and $N$ is normal,
	then $|G/N| = |G| / |N|$.
\end{remark}

\section{Eliminating the homomorphism}
\prototype{Again $\ZZ/n\ZZ \cong \Zc n$.}
Let's look at the last definition of $G/N$ we provided.
The short version is:
\begin{itemize}
	\ii The elements of $G/N$ are cosets $gN$, which you can think
	of as equivalence classes of a relation $\sim_N$
	(where $g_1 \sim_N g_2$ if $g_1 = g_2n$ for some $n \in N$).
	\ii Given cosets $g_1N$ and $g_2N$ the group operation is
	\[ g_1N \cdot g_2N \defeq (g_1g_2)N. \]
\end{itemize}
Question: where do we actually use the fact that $N$ is normal?
We don't talk about $\phi$ or $Q$ anywhere in this definition.

The answer is in \Cref{remark:coset_warning}.
The group operation takes in two cosets,
so it doesn't know what $g_1$ and $g_2$ are.
But behind the scenes,
\textbf{the normal condition guarantees that the group operation can pick
any $g_1$ and $g_2$ it wants and still end up with the same coset.}
If we didn't have this property, then it would be hard to define the
product of two cosets $C_1$ and $C_2$ because it might make a difference
which $g_1 \in C_1$ and $g_2 \in C_2$ we picked.
The fact that $N$ came from a homomorphism meant we could pick any representatives
$g_1$ and $g_2$ of the cosets we wanted, because they all had the same $\phi$-value.

We want some conditions which force this to be true without referencing $\phi$ at all.
Suppose $\phi \colon G \to K$ is a homomorphism of groups with $H = \ker\phi$.
Aside from the fact $H$ is a group, we can get an ``obvious'' property:
\begin{ques}
	Show that if $h \in H$, $g \in G$,
	then $ghg\inv \in H$.
	(Check $\phi(ghg\inv) = 1_K$.)
\end{ques}
\begin{example}[Example of a non-normal subgroup]
	\label{ex:dihedral_normal_subgroup}
	Let $D_{12} = \left<r,s \mid r^6=s^2=1, rs=sr\inv\right>$.
	Consider the subgroup of order two $H = \{1,s\}$
	and notice that \[ rsr\inv = r(sr\inv)= r(rs) = r^2s \notin H. \]
	Hence $H$ is not normal, and cannot be the kernel of any homomorphism.
\end{example}
Well, duh -- so what?
Amazingly it turns out that that this is the \emph{sufficient} condition we want.
Specifically, it makes the nice ``coset multiplication'' we wanted work out.
\begin{remark}
	[For math contest enthusiasts]
	This coincidence is really a lot like functional equations at the IMO.
	We all know that normal subgroups $H$ satisfy $ghg\inv \in H$;
	the surprise is that from the latter seemingly weaker condition,
	we can deduce $H$ is normal.
\end{remark}

Thus we have a new criterion for ``normal'' subgroups which does not
make any external references to $\phi$.
\begin{theorem}[Algebraic condition for normal subgroups]
	Let $H$ be a subgroup of $G$.
	Then the following are equivalent:
	\begin{itemize}
		\ii $H \normalin G$.
		\ii For every $g \in G$ and $h \in H$, $ghg\inv \in H$.
	\end{itemize}
\end{theorem}
\begin{proof}
	We already showed one direction.

	For the other direction, we need to build a homomorphism with kernel $H$.
	So we simply \emph{define} the group $G/H$ as the cosets.
	To put a group operation, we need to verify:
	\begin{claim}
		If $g_1' \sim_H g_1$ and $g_2' \sim_H g_2$ then $g_1'g_2' \sim_H g_1g_2$.
	\end{claim}
	\begin{subproof}
		Boring algebraic manipulation (again functional equation style).
		Let $g_1' = g_1h_1$ and $g_2' = g_2h_2$, so we want to show that
		$g_1h_1g_2h_2 \sim_H g_1g_2$.
		Since $H$ has the property, $g_2\inv h_1g_2$ is some element of $H$, say $h_3$.
		Thus $h_1 g_2 = g_2 h_3$, and the left-hand side becomes $g_1g_2(h_3h_2)$,
		which is fine since $h_3h_2 \in H$.
	\end{subproof}
	With that settled we can just \emph{define} the
	product of two cosets (of normal subgroups) by \[ (g_1H) \cdot (g_2H) = (g_1g_2)H. \]

	Thus the claim above shows that this multiplication is well-defined
	(this verification is the ``content'' of the theorem).
	So $G/H$ is indeed a group!
	Moreover there is an obvious ``projection'' homomorphism
	$G \to G/H$ (with kernel $H$), by $g \mapsto gH$.
\end{proof}
%Another way to write the condition is
%\[ H = gHg\inv \defeq \left\{ ghg\inv \mid h \in H \right\}. \]
%You should take a moment to check that these definitions are equivalent.

\begin{example}[Modding out in the product group]
	Consider again the product group $G \times H$.
	Earlier we identified a subgroup
	\[ G' =  \left\{ (g, 1_H) \mid g \in G \right\} \cong G. \]
	You can easily see that $G' \normalin G \times H$.
	(Easy calculation.)

	Moreover, just as the notation would imply, you can check that
	\[ (G \times H) / (G') \cong H. \]
	Indeed, we have $(g, h) \sim_{G'} (1_G, h)$ for all $g \in G$ and $h \in H$.
\end{example}
\begin{example}[Another explicit computation]
	Let $\phi: D_8 \to \Zc 4$ be defined by \[ r \mapsto \ol 2, \quad s \mapsto \ol 2. \]
	The kernel of this map is $N = \{1,r^2,sr,sr^3\}$.

	We can do a quick computation of all the elements of $D_8$ to get
	\[ \phi(1) = \phi(r^2) = \phi(sr) = \phi(sr^3) = \ol 0
	\text{ and }
	 \phi(r) = \phi(r^3) = \phi(s) = \phi(sr^2) = \ol 2. \]
	The two relevant fibers are \[ \phi\pre(\ol 0) = 1N = r^2N = srN = sr^3N = \{1,r^2,sr,sr^3\} \] and
	\[ \phi\pre(\ol 2) = rN = r^3N = sN = sr^2N = \{r,r^3,s,sr^2\}. \]
	So we see that $|D_8/N| = 2$ is a group of order two, or $\Zc 2$.
	Indeed, the image of $\phi$ is \[ \left\{ \ol 0, \ol 2 \right\} \cong \Zc 2. \]
\end{example}

\begin{ques}
	Suppose $G$ is abelian.
	Why does it follow that any subgroup of $G$ is normal?
\end{ques}

Finally here's some food for thought:
suppose one has a group presentation for a group $G$
that uses $n$ generators.
Can you write it as a quotient of the form $F_n / N$,
where $N$ is a normal subgroup of $F_n$?

\section{(Digression) The first isomorphism theorem}
One quick word about what other sources usually say.

Most textbooks actually \emph{define} normal using the $ghg\inv \in H$ property.
Then they define $G/H$ for normal $H$ in the way I did above,
using the coset definition
\[ (g_1H) \cdot (g_2H) = g_1g_2H. \]
Using purely algebraic manipulations (like I did) this is well-defined,
and so now you have this group $G/H$ or something.
The underlying homomorphism isn't mentioned at all,
or is just mentioned in passing.

I think this is incredibly dumb.
The normal condition looks like it gets pulled out of thin air
and no one has any clue what's going on,
because no one has any clue what a normal subgroup actually should look like.

Other sources like to also write the so-called first isomorphism theorem.\footnote{
	There is a second and third isomorphism theorem.
	But four years after learning about them,
	I \emph{still} don't know what they are.
	So I'm guessing they weren't very important.}
It goes like this.
\begin{theorem}
	[First isomorphism theorem]
	Let $\phi : G \to H$ be a homomorphism.
	Then $G / \ker \phi$ is isomorphic to $\phi\im(G)$.
\end{theorem}
To me, this is just a clumsier way of stating the same idea.

About the only merit this claim has is that if $\phi$ is injective,
then the image $\phi\im(G)$ is an \emph{isomorphic copy}
of $G$ inside the group $H$.
(Try to see this directly!)
This is a pattern we'll often see in other branches of mathematics:
whenever we have an \emph{injective structure-preserving map},
often the image of this map will be some ``copy'' of $G$.
(Here ``structure'' refers to the group multiplication,
but we'll see some more other examples of ``types of objects'' later!)

In that sense an injective homomorphism $\phi : G \injto H$
is an \emph{embedding} of $G$ into $H$.

\section\problemhead
\begin{problem}
	[18.701 at MIT]
	Determine all groups $G$ for which the map $\phi : G \to G$ defined by
	\[ \phi(g) = g^2 \] is a homomorphism.
	\begin{hint}
		Write it out: $\phi(ab) = \phi(a)\phi(b)$.
	\end{hint}
	\begin{sol}
		Abelian groups: $abab =a^2b^2 \iff ab= ba$.
	\end{sol}
\end{problem}

\begin{problem}
	Consider the dihedral group $G = D_{10}$.
	\begin{enumerate}[(a)]
		\ii Is $H = \left< r \right>$ a normal subgroup of $G$?
		If so, compute $G/H$ up to isomorphism.
		\ii Is $H = \left< s \right>$ a normal subgroup of $G$?
		If so, compute $G/H$ up to isomorphism.
	\end{enumerate}
	\begin{hint}
		Yes, no.
	\end{hint}
	\begin{sol}
		Yes to (a): you can check this directly
		from the $ghg\inv$ definition.
		For example, for (a)
		it is enough to compute $(r^a s) r^n (r^a s)\inv = r^{-n} \in H$.
		The quotient group is $\Zc2$.

		The answer is no for (b) by following
		\Cref{ex:dihedral_normal_subgroup}.
	\end{sol}
\end{problem}

\begin{problem}
	Does $S_4$ have a normal subgroup of order $3$?
	\begin{hint}
		No.
	\end{hint}
	\begin{sol}
		A subgroup of order $3$ must be generated by
		an element of order $3$, since $3$ is prime.
		So we may assume WLOG that $H = \left< (1\; 2 \; 3) \right>$
		(by renaming elements appropriately).
		But then let $g = (3 \; 4)$; one can check $gHg\inv \ne H$.
	\end{sol}
\end{problem}

\begin{problem}
	Let $G$ and $H$ be finite groups, where $\left\lvert G \right\rvert = 1000$
	and $\left\lvert H \right\rvert = 999$.
	Show that a homomorphism $G \to H$ must be trivial.
	\begin{hint}
		$\gcd(1000,999)=1$.
	\end{hint}
	\begin{sol}
		$G/\ker G$ is isomorphic to a subgroup of $H$.
		The order of the former divides $1000$;
		the order of the latter divides $999$.
		This can only occur if $G / \ker G = \{1\}$
		so $\ker G = G$.
	\end{sol}
\end{problem}

\begin{problem}
	Let $\CC^\times$ denote the nonzero complex numbers under multiplication.
	Show that there are five homomorphisms $\Zc5 \to \CC^\times$
	but only two homomorphisms $D_{10} \to \CC^\times$,
	even though $\Zc5$ is a subgroup of $D_{10}$.
\end{problem}

\begin{problem}
	\gim
	Find a non-abelian group $G$
	such that every subgroup of $G$ is normal.
	(These groups are called \vocab{Hamiltonian}.)
	\begin{hint}
		Find an example of order $8$.
	\end{hint}
	\begin{sol}
		Quaternion group.
	\end{sol}
\end{problem}

\begin{problem}
	[PRIMES entrance exam, 2017]
	\gim
	Let $G$ be a group with presentation given by
	\[ G = \left< a,b,c \mid
		ab = c^2a^4, \; bc = ca^6, \; ac = ca^8, \;
		c^{2018} = b^{2019}
		\right>. \]
	Determine the order of $G$.
	\begin{hint}
		Try to show $G$ is the dihedral group of order $18$.
		There is not much group theory content here --- just manipulation.
	\end{hint}
	\begin{sol}
		The answer is $|G| = 18$.

		First, observe that by induction we have
		\[ a^n c = ca^{8n} \]
		for all $n \ge 1$.
		We then note that
		\begin{align*}
			a(bc) &= (ab)c \\
			a \cdot ca^6 &= c^2 a^4 \cdot c \\
			c a^8 \cdot a^6 &= c^2 a^4 \cdot c \\
			a^{14} &= c(a^4c) = c^2 a^{32}.
		\end{align*}
		Hence we conclude $c^2 = a^{-18}$.
		Then $ab = c^2a^4 \implies b = a^{-15}$.

		In that case, if $c^{2018} = b^{2019}$,
		we conclude $1 = a^{2018 \cdot 18 - 2019 \cdot 15} = a^{6039}$.
		Finally,
		\begin{align*}
			bc &= ca^6 \\
			a^{-15} c &= ca^6 \\
			a^{-15} c^2 &= c(a^6c) = c^2 a^{48} \\
			a^{-33} &= a^{30} \\
			\implies a^{63} &= 1.
		\end{align*}
		Since $\gcd(6039, 63) = 9$, we find $a^9 = 1$, hence finally $c^2 = 1$.
		So the presentation above simplifies to
		\[ G = \left< a,c \mid a^9=c^2=1, \; ac = ca^{-1} \right> \]
		which is the presentation of the dihedral group of order $18$.
		% a = r, r^9 = 1
		% b = r^3
		% c = s, s^2 = 2
		This completes the proof.
	\end{sol}
\end{problem}

\begin{problem}
	[Homophony group]
	\gim
    The homophony group (of English) is the group
	with $26$ generators $a$, $b$, \dots, $z$
	and one relation for every pair of English words
	which sound the same.
	For example $knight = night$ (and hence $k=1$).
	Prove that the group is trivial.
	\begin{hint}
		Get yourself a list of English homophones, I guess.
		Don't try too hard.
		Letter $v$ is the worst; maybe $felt = veldt$?
	\end{hint}
	\begin{sol}
		You can find many solutions by searching ``homophone group'';
		one is \url{https://math.stackexchange.com/q/843966/229197}.
	\end{sol}
\end{problem}