\section{Point Group Theory}
\index{Groups|(} \index{Point-group!theory} \index{Symmetry!in group theory}
This Section is based on the original work of Peter Bischof in the UMNDO
program, and made available to me by Dr David Danovich.

Some point-group theory has been added to MOPAC.  The main functionalities
added are: \index{AUTOSYM}\index{Bischof@{\bf Bischof, Peter}}
\index{UMNDO}\index{Danovich@{\bf Danovich, David}}

\begin{itemize}
\item `Normal' symmetry relationships are now automatically recognized if
\comp{AUTOSYM} is specified.

\item The symmetry of the system is printed both at the start of the run and at
the end.  If the point-group changes, the change will be shown in the different
point-group symbols.

\item Molecular orbitals will be characterized by Irreducible Representation
(I.R.).

\item Normal coordinates generated in the vibrational calculation will be
characterized by I.R.

\item State functions will be characterized by I.R.

\item All rotation groups up to order 8, except D$_{8d}$, are available.

\item The cubic groups T, T$_h$, T$_d$, O, O$_h$, I, and I$_h$ are available.

\item The infinite groups C$_{\infty v}$, D$_{\infty h}$ and R$_3$ are
available.

\item In \comp{FORCE} or vibrational frequency calculations, symmetry will be
used to accelerate the calculation, thus a calculation of benzene would involve
two atoms, a C and a H atom, to be calculated, rather than the normal 12 atoms.

\item In vibrational frequency calculations, the Hessian or force matrix
will be symmetrized.  \index{sym\_force} \label{sym_force}
$$
F_{ij}=\frac{1}{h}\sum_hR(h)^TF_{ij}'R(h)
$$

This eliminates the normal small deviations from exact symmetry
\begin{htmlonly}
(\htmlref{a qualification appears
elsewhere}{fc})
\end{htmlonly}
\begin{latexonly}
(see also p.~\pageref{fc}
for a qualification)
\end{latexonly}.

\end{itemize}

\subsection*{Limitations}
\begin{itemize}
\index{Point Group D$_{8d}$ missing}\index{D$_{8d}$ missing}
\item Group D$_{8d}$ is missing.  This group is characterized by the presence
of a 16-fold S$_n$ axis.  Only S$_n$ operations up to S$_{12}$  are  checked
for.  As a  result, D$_{8d}$ would not be recognized.  However, this is a rare
point-group, and its loss should not be important.

\item Some systems which are insufficiently near to a given point group will be
assigned to the nearest sub-group.  For example, if SF$_6$ is distorted so that
two opposite F atoms are at a different distance to the other four, the system
might be classified as O$_h$ or D$_{4h}$, depending on the degree of
distortion.  This shows up mainly in methyl groups, e.g.\ neopentane, in which
optimization normally stops before the angles of the hydrogens are fully
optimized.
\end{itemize}

\subsection{Representation of Point Groups}
The 57 groups recognized in MOPAC are given in Table~\ref{pgs}.
\begin{table}
\caption{\label{pgs} Point Groups available within Symmetry Code}
\begin{center}
\begin{tabular}{lllllllll} \hline
 C$_1$&C$_{s} $ & C$_{i }$   &       &      &      &     &  O             \\
 C$_2$&C$_{2v}$ & C$_{2h}$   & D$_2$ & D$_{2d}$  & D$_{2h}$  &     &  T   \\
 C$_3$&C$_{3v}$ & C$_{3h}$   & D$_3$ & D$_{3d}$  & D$_{3h}$  &     &  T$_d$   \\
 C$_4$&C$_{4v}$ & C$_{4h}$   & D$_4$ & D$_{4d}$  & D$_{4h}$  & S$_4$  &  T$_h$   \\
 C$_5$&C$_{5v}$ & C$_{5h}$   & D$_5$ & D$_{5d}$  & D$_{5h}$  &     &  O$_h$   \\
 C$_6$&C$_{6v}$ & C$_{6h}$   & D$_6$ & D$_{6d}$  & D$_{6h}$  & S$_6$  &  I & I$_h$   \\
 C$_7$&C$_{7v}$ & C$_{7h}$   & D$_7$ & D$_{7d}$  & D$_{7h}$  &  &   C$_{\infty v}$
& D$_{\infty h}$ \\
 C$_8$&C$_{8v}$ & C$_{8h}$   & D$_8$ &      & D$_{8h}$ & S$_8$  &   R$_3$   \\ \hline
\end{tabular}
\end{center}
\end{table}

Each point group is represented by a subset of the associated point-group
table. For example, the group D$_{2h}$ is represented by the subset shown in
Table~\ref{d2h}. The operations selected for the subgroup are the identity, E,
and that minimum set of operations which is sufficient to allow all the
operations to be generated as products of these operations.  Thus, for the
highest finite point group, I$_h$, the generating operations are: E, I, C$_3$,
and C$_5$.  Although it is not obvious, all 120 operations of the group can be
generated as products of these four operations.

\begin{table}
\caption{\label{d2h}Subset of Group D$_{2h}$}
\begin{center}
\begin{tabular}{lrrrr} \\ \hline
  $\Gamma$ &  E  &  C$_{2y}$ & C$_{2z}$& I \\ \hline
A$_g$   \\
B$_{1g}$ &1 &  1  &-1  &  1  \\
B$_{2g}$ &1 & -1  & 1  &  1  \\
B$_{3g}$ &1 & -1  &-1  &  1  \\
A$_{u} $ &1 &  1  & 1  & -1  \\
B$_{1u}$ &1 &  1  &-1  & -1  \\
B$_{2u}$ &1 & -1  & 1  & -1  \\
B$_{3u}$ &1 & -1  &-1  & -1  \\ \hline
\end{tabular}\end{center}
\end{table}

Each point-group is assumed to contain the totally symmetric representation,
here \index{Euler matrices} A$_{1g}$.  Operations are represented as $3\times3$
Euler matrices, thus C$_{2x}$, C$_{2y}$  and C$_{2z}$  would be represented as
in Figure~\ref{c2op} \ All operations not given can be generated as products of
operations already known, thus C$_{2x}$ = C$_{2y}$ $\times$ C$_{2z}$.

% 9 lines, including this line
\begin{figure}
\begin{makeimage}
\end{makeimage}
\begin{center}\hfil
C$_{2x}$: \begin{tabular}{|rrr|}1&0&0\\0&-1&0\\0&0&-1\end{tabular}\hfil
C$_{2y}$: \begin{tabular}{|rrr|}-1&0&0\\0&1&0\\0&0&-1\end{tabular}\hfil
C$_{2z}$: \begin{tabular}{|rrr|}-1&0&0\\0&-1&0\\0&0&1\end{tabular}\hfil
\end{center}
\caption{\label{c2op} Representation of Symmetry Operations}
\end{figure}

In order to minimize storage, the characters in  character tables are stored
separately from the point groups.  This allows, e.g., C$_{2v}$, C$_{2h}$, and
D$_{2}$ to use the same character table.

\subsection{Identification of Point-Groups}
\index{Infinite groups}
\subsubsection*{Infinite Groups}
In order to identify the molecular point-group the system must be oriented in a
specific way.  Four families of point-groups are checked for: (1) the infinite
groups, (2) the cubic groups, (3)  groups with one high-symmetry axis, and (4)
the Abelian groups. Each family is treated differently.  First, the moments of
inertia are calculated.  If all are zero, the system is a single atom, and the
associated group is R$_3$.  If two moments are zero, the system is  either
C$_{\infty v}$ or D$_{\infty h}$; the presence of a horizontal plane of
symmetry distinguishes between them.

\index{Cubic Groups}
\subsubsection*{Cubic Groups}
Having eliminated the infinite groups, the three moments of inertia are checked
to see if they are all the same.  If they are, then the system is cubic.  Cubic
systems are oriented by identifying atoms of the set nearest to the center of
symmetry.  If there are 4, 6, 8, 12, or 20 of these, and the number of
equidistant nearest neighbors  is 3, 4, 3, 5, or 3, respectively, then the
atoms are
%probably
at the vertices of one of the Platonic solids (tetrahedron, octahedron, cube,
icosahedron, dodecahedron),
%Platonic solids (tetrahedron, octahedron, cube, icosahedron, dodecahedron).
%There are a few cases in which these two conditions are true, but the
%solid is nevertheless not platonic.
%To exclude thses rare case, use is made of the ratio of the edge distance
%to the (vertex to center) distance.  This ratio is unique for each of the
%platonic solids, if the
%computed value is correct the solid is unambiguously platonic,
and therefore all atoms of the set lie on high-symmetry axes.  The first  atom
is selected and used to define the $z$ axis.

\index{Platonic solids} \index{Buckminsterfullerene} If the number of atoms in
the set does not correspond to any of the Platonic solids, then the set is
checked for the existence of a equilateral triangle, a square, or a regular
pentagon.  When one of these is found, the center of the polygon is used to
define the $z$ axis.  An example of this type of system is C$_{60}$,
Buckminsterfullerene, which has a five-fold axis going through the center of a
pentagonal face.

Once the $z$ axis is identified, the system is checked for C$_n$ axes, $n$=3 to
$n$=8.  To complete the orientation, the system is rotated about the $z$ axis
so that two atoms, having equal $z$ coordinates, have equal $y$ coordinates.
The existence of rotation axes which are not coincidental with the $z$ axis and
the presence or absence of a center of inversion are then used to identify
which cubic group the system belongs to.

\index{Degenerate groups}
\subsubsection*{Other Degenerate Groups}
If the system has still not been identified, then the two equal moments of
inertia indicate a degenerate point group.  As with the cubic groups, the $y$ and
$z$ axes (and, by implication, the $x$ axis) are identified. The system is
oriented, and the C$_n$ and S$_n$ axes identified.

The degenerate groups, C$_n$, C$_{nv}$, C$_{nh}$, D$_n$, D$_{nd}$, D$_{nh}$,
S$_n$, are distinguished by the existence or absence of C$_2$ axes
perpendicular to the $z$ axis, and by planes of symmetry.

\index{Abelian groups}\index{Orientation!for symmetry}
\index{Symmetry!orientation of molecules}

\subsubsection*{Abelian Groups}
All that remains are the Abelian groups, C$_1$, C$_2$, C$_i$, C$_s$,  C$_{2v}$,
C$_{2h}$, D$_2$,  and D$_{2h}$.  After orienting the molecule, the axes are
swapped around so that the normal convention for orienting Abelian systems is
obeyed.  For groups C$_1$, C$_2$, C$_i$, and C$_s$, there is no possibility for
ambiguity.  For C$_{2v}$ and D$_2$, however, the orientation of the system
affects the labels of the irreducible representations.  To prevent ambiguity,
the convention for orienting Abelian molecules is:
\begin{itemize}
\item The axis with the largest number of atoms is the $z$  axis.
\item The plane with the largest number of atoms that includes the $z$ axis is
the $yz$  plane.
\end{itemize}
Thus for ethylene, the $\pi$ orbitals point along the $x$  axis.

\subsubsection*{Tolerance}
Normally, molecular geometries do not exactly correspond to the idealized
point-group.  Thus benzene might have slightly different bond-lengths and
angles.  Of course, symmetry could be used to prevent this, but in the
discussion here we assume that the symmetry of the system is unknown.  To allow
for these slight distortions, a small tolerance is built in to the tests for
symmetry elements. This starts off at 0.1\AA\ , but may be tightened
automatically if ambiguities are detected.  An example of such an ambiguity is
found in tropylium, C$_7$H$_7^+$ ion, where the C-C distance is 1.4 \AA .
Rotating the ring by 45 degrees (a C$_8$ operation) would place the atoms at a
distance of only 0.18\AA\  from equivalent positions.  C$_7$ and C$_8$ would
thus  give almost identical results.  To resolve such ambiguities, when they
arise, the tolerance is reduced, and the test re-run.

Even with this feature, some systems still resist classification.   A distorted
geometry might have some, but not all, elements of a high point group.  Perhaps
a distorted benzene has a C$_2$(z) and a C$_3$(z), but  not a C$_6$(z),
impossible in a real system.  As such it would appear to be different from all
real point groups.  To accommodate such defects a descent in symmetry is
carried out.  This consists of checking each point-group in turn, in order of
decreasing symmetry.  Once all of the elements of a point group are satisfied,
the system is assigned to that point group, even if the system contains more
symmetry than the point group.

By these two devices, a variable tolerance and the descent in symmetry, most
systems should be identified correctly, or at least as a sub-group of the full
point group.

\subsection{Orientation of the Abelian groups C$_{2v}$ and D$_{2h}$}
Unlike all other groups, two of the Abelian groups, C$_{2v}$ and  D$_{2h}$,
present novel problems in assigning the irreducible representations.  For most
groups, the symmetry axis is obvious, or of there are several axes, the
principal axis is obvious.  For C$_{2v}$ and D$_{2h}$ an ambiguity exists.
Consider, for example, ethylene, a system of point-group D$_{2h}$.  Should the
$z$ axis be perpendicular to the plane of the molecule---that is the unique
direction, or should it go through the two carbon atoms---that is also a unique
direction, but for a different reason, or should it be the third orthogonal
direction---which is also unique.  The choice of $z$ axis is important in order
to correctly assign the B$_{1g}$ and B$_{1u}$ of point-group D$_{2h}$.  For
both C$_{2v}$ and D$_{2h}$ the $x$ and $y$ axes must also be unambiguously
defined in order to distinguish  between B$_{2g}$ and B$_{3g}$ and between
B$_{2u}$ and B$_{3u}$ of  D$_{2h}$, and between B$_{1}$ and B$_{2}$ of
C$_{2v}$. Clearly a convention has to be decided upon, otherwise one persons
B$_{1g}$ might be a second persons B$_{2g}$ and a third persons B$_{3g}$.

The convention used in MOPAC is the following:

If there are three C$_2$ axes, the one with the largest number of atoms unmoved
by a C$_2$ operation is $z$.  If there is only one C$_2$ axis, that is $z$.

Once $z$ is defined, the $y$ axis is defined as the axis of the remaining two
axes which has the larger number of atoms unmoved by  the $\sigma$ symmetry
operations.

The $x$ axis is the remaining axis.

To see how this works, consider ethylene, with the C--C axis being along the
$x$ direction, and the plane of the system being $xy$. Under the eight
operations of D$_{2h}$, E, C$_{2z}$, C$_{2y}$, C$_{2x}$, $\sigma_{xy}$,
$\sigma_{xz}$, $\sigma _{yz}$, and $i$, the number of atoms unmoved are 6, 0,
0, 2, 6, 2, 0, and 0 respectively.

From this it follows that the old $x$ axis is now re-defined as the $z$ axis.
The new $y$ axis has to be chosen based on the number of atoms unmoved under
the $\sigma_{xy}$ and $\sigma_{xz}$ operations (6 and 2). The new $y$ axis is
defined as being the old $y$ axis.  The remaining new axis $x$ therefore is the
old $z$ axis.

The overall result is that the symmetry axes in ethylene are defined as: $z$ -
along the C--C bond; $y$ - in the molecular plane, perpendicular to the C--C
bond, and $x$ - out-of-plane.

In MOPAC the orientation of the molecule is defined by the user, therefore the
assignment of the symmetry axes might be confusing.  If the irreducible
representations of ethylene are assigned, and the atoms are defined  using
internal coordinates in the order C, C, H, H, H, H, then the $p$ orbitals will
reflect the orientation used in the previous discussion, but the
representations will be correct according to the conventions just defined.

\subsection{Molecular Orbitals}
Each M.O.\ is subjected to the operation
$$
\psi' = |R|\psi>
$$
from which the expectation value
$$
\chi=<\psi'|\psi>
$$
can readily be calculated.

\index{Degenerate M.O.s}
All $\chi$'s within a given degenerate manifold are summed:
$$
\chi_i = \sum_j\chi_j^{(i)}
$$
where $j$ runs over all components of the degenerate manifold $i$.

This results in a set of characters which can be compared to those stored in
the character tables.

Since molecular orbitals involve single electrons, the irreducible
representations \index{Irreducible representations} are rendered into lower
case before printing.

\subsection{Normal Coordinates}
\index{States!vibration} \index{Normal coordinates}\index{Coordinates!normal}
Analysis of normal coordinates is a little simpler than that of molecular
orbitals in that every atom contributes precisely three components to each
normal coordinate, an $x$, $y$, and $z$ component.  These transform in the same
way as the p$_x$, p$_y$, and p$_z$ atomic orbitals.

Because normal coordinates are states, the first letter of the irreducible
representation is capitalized.

\subsection{States}
\index{States!electronic}\index{States!symmetry of}
Calculating the characters for electronic states is much more complicated than
that for M.O.s or normal coordinates.

Consider the effect of an operation, $R$, on a state, $\Phi_a$.  The
character of the operation is given by
$$
\chi_{R,a} = <\Phi_a|R|\Phi_a>.
$$
A state function can be expressed as a linear combination of microstates:
$$
\Phi_a = \sum_jC_{ja}\Psi_j,
$$
so the character of the operation on the state function can be written in
terms of microstates as
$$
\chi_{R,a} =\sum_i\sum_jC_{ia}C_{ja}<\Psi_i|R|\Psi_j>.
$$
Each microstate, $\Psi_j$, can be represented by a \mi{Slater determinant}
of $N$ molecular orbitals
:
$$
\Psi_j = \frac{1}{\sqrt{N!}}\sum_{P=1}^{N!}(-1)^PP(\prod_{k=1}^N\psi_k^j)
$$
\index{Microstates} where the molecular orbitals in the microstate consist of a
selection of the M.O.s in the  active space\index{Active space!in C.I.}.
Before we continue, let us examine this idea:

Consider a full set of M.O.s:
$$
\psi_1\psi_2\psi_3\psi_4 \psi_5 \psi_6 \psi_7 \psi_8 \psi_9
\psi_{10}\psi_{11}\psi_{12}\psi_{13}\psi_{14}\psi_{15}\psi_{16}.
$$
Let the active space be the M.O.s  from 8 to 11.  Then microstates containing
two electrons would be:
\begin{center}
\hfil
$\psi_8\psi_9$
\hfil
$\psi_8\psi_{10}$
\hfil
$\psi_8\psi_{11}$
\hfil
$\psi_9\psi_{10}$
\hfil
$\psi_9\psi_{11}$
\hfil
$\psi_{10}\psi_{11}$.
\hfil
\end{center}
These microstates could be represented by M.O.\ orbital occupancies.
\begin{center}
\hfil
1100
\hfil
1010
\hfil
1001
\hfil
0110
\hfil
0101
\hfil
0011.
\hfil
\end{center}
Remember that the M.O.s here can be of either $\alpha$ or $\beta$ spin.

To continue, we need to evaluate $<\Psi_i|R|\Psi_j>$.  This can be expressed
in terms of M.O.s as:
$$
<\Psi_i|R|\Psi_j> = \frac{1}{N!}\sum_{P=1}^{N!}(-1)^PP(<\prod_{k=1}^N\psi_k^i)|
R|
\sum_{Q=1}^{N!}(-1)^QQ(\prod_{l=1}^N\psi_l^j)>.
$$

For convenience, we will represent the integral $<\psi_k^i|R|\psi_l^j>$ by
$\chi_{kl}^{ij}$.  This integral can be described as ``The integral over M.O.\
$\psi_k$ in microstate $\Psi_i$ with the result of operator $R$ acting
on M.O.\ $\psi_l$ in microstate $\Psi_j$.''

Using this abbreviation, $<\Psi_i|R|\Psi_j>$ can be written as:
$$
<\Psi_i|R|\Psi_j> =\frac{1}{N!}\sum_{P=1}^{N!} \sum_{Q=1}^{N!}(-1)^P(-1)^Q
P\prod_{k=1}^{N!}Q\prod_{l=1}^{N!}\chi_{kl}^{ij}.
$$
Although it is not immediately obvious, the right-hand term is a determinant,
of order $N$:
$$
<\Psi_i|R|\Psi_j> =\left|
\begin{array}{cccc}
\chi_{11}^{ij} & \chi_{21}^{ij} & \chi_{31}^{ij} & \ldots \\
\chi_{12}^{ij} & \chi_{22}^{ij} & \chi_{32}^{ij} & \ldots \\
\chi_{13}^{ij} & \chi_{23}^{ij} & \chi_{33}^{ij} & \ldots \\
 \ldots           &  \ldots           &   \ldots          & \ldots
\end{array}
\right|.
$$

For our purposes, solution of the determinant is best done explicitly. To see
why, note that the number of M.O.s involved in the C.I.\ (the active space) is
very small.  Because of this, the number of electrons, $N$, in the Slater
determinants is also small; $N$ has a maximum value of  20.  Next, use can be
made of the fact that no point-group operation can mix $\alpha$ and $\beta$
electrons.  This allows the integral to be split into two parts, each of which
has a maximum value  of $N$=10. Finally, remember that $N$ is the number of
electrons, not M.O.s, used in the active space.  A system of $N$ electrons has
the same symmetry as a system in which all the M.O.s which were occupied were
replaced with all the M.O.s which were not occupied (the positron
equivalent)\index{Positron equivalent}. (This assumes that if every M.O.\ were
occupied, then the state of the system would be totally symmetric.)   Using
this fact, we can replace the $N$ occupied M.O.s with $N'$ unoccupied M.O.s, if
$N' < N$.

When these three points are considered, we see that $N$ has a maximum value  of
5 (for a system of 10 M.O.s).  Each case can be considered separately.
\begin{description}
\item{For $N$ = 1:}
$$
<\Psi_i|R|\Psi_i> = \frac{1}{1}\sum_{P=1}^{1} \sum_{Q=1}^{1}(-1)^P(-1)^Q
P\prod_{k=1}^{1}Q\prod_{l=1}^{1}\chi_{kl}^{ii}
$$
or
$$
<\Psi_i|R|\Psi_i> = <\psi_1^i|R|\psi_1^i> = \chi_{11}^{ii}.
$$
\item{For $N$=2:}
$$
<\Psi_i|R|\Psi_j> = \frac{1}{2!}\sum_{P=1}^{2!} \sum_{Q=1}^{2!}(-1)^P(-1)^Q
P\prod_{k=1}^{2}Q\prod_{l=1}^{2}\chi_{kl}^{ij}
$$
or
$$
<\Psi_i|R|\Psi_j> = <\psi_1^i|R|\psi_1^j>-<\psi_2^i|R|\psi_2^j>
<\psi_1^i|R|\psi_2^j><\psi_1^i|R|\psi_2^j>
$$
or
$$
<\Psi_a|R|\Psi_a> = \chi_{11}^{ij}\chi_{22}^{ij}-\chi_{12}^{ij}\chi_{21}^{ij}.
$$
\item{For $N$=3:}
$$
<\Psi_i|R|\Psi_j> = \frac{1}{3!}\sum_{P=1}^{3!} \sum_{Q=1}^{3!}(-1)^P(-1)^Q
$$
\begin{eqnarray}
<\Psi_i|R|\Psi_j>& = &\ <\psi_1^i|R|\psi_1^j><\psi_2^i|R|\psi_2^j><\psi_3^i|R|\psi_3^j> \nonumber   \\
&&-<\psi_1^i|R|\psi_1^j><\psi_2^i|R|\psi_3^j><\psi_3^i|R|\psi_2^j> \nonumber  \\
&&-<\psi_1^i|R|\psi_2^j><\psi_2^i|R|\psi_1^j><\psi_3^i|R|\psi_3^j> \nonumber  \\
&&+<\psi_1^i|R|\psi_2^j><\psi_2^i|R|\psi_3^j><\psi_3^i|R|\psi_1^j> \nonumber  \\
&&+<\psi_1^i|R|\psi_3^j><\psi_2^i|R|\psi_1^j><\psi_3^i|R|\psi_2^j> \nonumber  \\
&&-<\psi_1^i|R|\psi_3^j><\psi_2^i|R|\psi_2^j><\psi_3^i|R|\psi_1^j>  \nonumber
\end{eqnarray}
or
\begin{eqnarray}
<\Psi_i|R|\Psi_j>&=&\chi_{11}^{ij}\chi_{22}^{ij}\chi_{33}^{ij}+
                            \chi_{12}^{ij}\chi_{23}^{ij}\chi_{31}^{ij}+
                            \chi_{13}^{ij}\chi_{21}^{ij}\chi_{32}^{ij} \nonumber \\
&&-\chi_{11}^{ij}\chi_{23}^{ij}\chi_{32}^{ij}
-\chi_{12}^{ij}\chi_{21}^{ij}\chi_{33}^{ij}
-\chi_{13}^{ij}\chi_{22}^{ij}\chi_{31}^{ij}). \nonumber
\end{eqnarray}
\end{description}

For higher numbers of electrons, the associated determinant is solved using
standard methods.

The total character, $<\Psi_a|R|\Psi_a>$, is obtained by multiplying the
characters for the $\alpha$ and $\beta$ parts together:
$$
<\Psi_a|R|\Psi_a> = <\Psi_a^{\alpha}|R|\Psi_a^{\alpha}>
<\Psi_a^{\beta}|R|\Psi_a^{\beta}>.
$$
If the positron equivalent is taken for only one set of electrons, e.g.\ either
the $\alpha$ or the $\beta$ set, but not both, then the  character has  to be
multiplied by the determinant of the M.O.\ transform.

These expressions can then be used in
$$
\chi_{R,a} =\sum_i\sum_jC_{ia}C_{ja}<\Psi_i|R|\Psi_j>.
$$
to give the expectation value for the state. Finally, if the state is
degenerate, the character is given by summing the components of the state.

For the atom, the \mi{Russell-Saunders} coupling scheme can be reproduced.
States allowed are $S$, $P$, $D$, $F$, $G$, $H$, $I$, $K$, $L$, and $M$. This
set is more than sufficient to allow all possible Russell-Saunders states
spanned by a basis set of $s$, $p$, and $d$ orbitals to be represented. The
highest angular momentum achievable with such a basis set is 8, i.e. $L$. For
simpler atoms (ones with only a $s-p$ basis set) the allowed states are
$p^0,p^6$: $^1S_g$,  $p^1,p^5$: $^2P_u$, $p^2,p^4$: $^1S_g + ^3\!\!P_g +
^1\!\!D_g$,  $p^3$: $^4\!S_u +^2\!\!P_u + ^2\!\!D_u$.

For the axial infinite groups, allowed states are: $\Sigma$, $\Pi$, $\Delta$,
$\Phi$, and $\Gamma$.  Even quite simple systems can achieve quite high angular
momentum, \index{Angular momentum} thus acetylene, with a \comp{C.I.=4}  (the
HOMO $\pi$ and LUMO $\pi^*$) will contain a $^1\Gamma_g$ state,  i.e., the
angular momentum will be 4.

At present J-J coupling is not supported.
\index{Groups|)}


\chapter[The Module System]{The Module System\label{chapter:Modules}}

The module system extends the Calculus of Inductive Constructions
providing a convenient way to structure large developments as well as
a means of massive abstraction.
%It is described in details in Judicael's thesis and Jacek's thesis

\section{Modules and module types}

\paragraph{Access path.} It is denoted by $p$, it can be either a module
variable $X$ or, if $p'$ is an access path and $id$ an identifier, then
$p'.id$ is an access path.

\paragraph{Structure element.} It is denoted by \elem\ and is either a
definition of a constant, an assumption, a definition of an inductive,
 a definition of a module, an alias of module or a module type abbreviation.

\paragraph{Structure expression.} It is denoted by $S$ and can be:
\begin{itemize}
\item an access path $p$
\item a plain structure $\struct{\nelist{\elem}{;}}$
\item a functor $\functor{X}{S}{S'}$, where $X$ is a module variable,
  $S$ and $S'$ are structure expression
\item an application $S\,p$, where $S$ is a structure expression and $p$
an access path
\item a refined structure $\with{S}{p}{p'}$ or $\with{S}{p}{t:T}$ where $S$
is a structure expression, $p$ and $p'$ are access paths, $t$ is a term
and $T$ is the type of $t$.
\end{itemize}

\paragraph{Module definition,} is written $\Mod{X}{S}{S'}$ and
 consists of a module variable $X$, a module type
$S$ which can be any structure expression and optionally a module implementation $S'$
 which can be any structure expression except a refined structure.

\paragraph{Module alias,} is written $\ModA{X}{p}$ and
 consists of a module variable $X$ and a module path $p$.

\paragraph{Module type abbreviation,} is written $\ModType{Y}{S}$, where
$Y$ is an identifier and $S$ is any structure expression .

\section{Typing Modules}

In order to introduce the typing system we first slightly extend
the syntactic class of terms and environments given in
section~\ref{Terms}. The environments, apart from definitions of
constants and inductive types now also hold any other structure elements.
Terms, apart from variables, constants and complex terms,
include also access paths.

We also need additional typing judgments:
\begin{itemize}
\item \WFT{E}{S}, denoting that a structure $S$ is well-formed,

\item \WTM{E}{p}{S}, denoting that the module pointed by $p$ has type $S$ in
environment $E$.

\item \WEV{E}{S}{\overline{S}}, denoting that a structure $S$ is evaluated to
a structure $\overline{S}$ in weak head normal form.

\item \WS{E}{S_1}{S_2}, denoting that a structure $S_1$ is a subtype of a
structure $S_2$.

\item \WS{E}{\elem_1}{\elem_2}, denoting that a structure element
  $\elem_1$ is more precise that a structure element $\elem_2$.
\end{itemize}
The rules for forming structures are the following:
\begin{description}
\item[WF-STR]
\inference{%
  \frac{
    \WF{E;E'}{}
  }{%%%%%%%%%%%%%%%%%%%%%
    \WFT{E}{\struct{E'}}
  }
}
\item[WF-FUN]
\inference{%
  \frac{
    \WFT{E;\ModS{X}{S}}{\overline{S'}}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WFT{E}{\functor{X}{S}{S'}}
  }
}
\end{description}
Evaluation of structures to weak head normal form:
\begin{description}
\item[WEVAL-APP]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{S}{\functor{X}{S_1}{S_2}}~~~~~\WEV{E}{S_1}{\overline{S_1}}\\
    \WTM{E}{p}{S_3}\qquad \WS{E}{S_3}{\overline{S_1}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WEV{E}{S\,p}{S_2\{p/X,t_1/p_1.c_1,\ldots,t_n/p_n.c_n\}}
  }
}
\end{description}
In the last rule, $\{t_1/p_1.c_1,\ldots,t_n/p_n.c_n\}$ is the resulting
 substitution from the inlining mechanism. We substitute in $S$ the
 inlined fields $p_i.c_i$ form $\ModS{X}{S_1}$ by the corresponding delta-reduced term $t_i$ in $p$.
\begin{description}
\item[WEVAL-WITH-MOD]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{S}{\structe{\ModS{X}{S_1}}}~~~~~\WEV{E;\elem_1;\ldots;\elem_i}{S_1}{\overline{S_1}}\\
    \WTM{E}{p}{S_2}\qquad \WS{E;\elem_1;\ldots;\elem_i}{S_2}{\overline{S_1}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \begin{array}{c}
    \WEVT{E}{\with{S}{x}{p}}{\structes{\ModA{X}{p}}{p/X}}
    \end{array}
  }
}
\item[WEVAL-WITH-MOD-REC]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{S}{\structe{\ModS{X_1}{S_1}}}\\
    \WEV{E;\elem_1;\ldots;\elem_i}{\with{S_1}{p}{p_1}}{\overline{S_2}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \begin{array}{c}
    \WEVT{E}{\with{S}{X_1.p}{p_1}}{\structes{\ModS{X}{\overline{S_2}}}{p_1/X_1.p}}
    \end{array}
  }
}
\item[WEVAL-WITH-DEF]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{S}{\structe{\Assum{}{c}{T_1}}}\\
    \WS{E;\elem_1;\ldots;\elem_i}{\Def{}{c}{t}{T}}{\Assum{}{c}{T_1}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \begin{array}{c}
    \WEVT{E}{\with{S}{c}{t:T}}{\structe{\Def{}{c}{t}{T}}}
    \end{array}
  }
}
\item[WEVAL-WITH-DEF-REC]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{S}{\structe{\ModS{X_1}{S_1}}}\\
    \WEV{E;\elem_1;\ldots;\elem_i}{\with{S_1}{p}{p_1}}{\overline{S_2}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \begin{array}{c}
    \WEVT{E}{\with{S}{X_1.p}{t:T}}{\structe{\ModS{X}{\overline{S_2}}}}
    \end{array}
  }
}

\item[WEVAL-PATH-MOD]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{p}{\structe{ \Mod{X}{S}{S_1}}}\\
    \WEV{E;\elem_1;\ldots;\elem_i}{S}{\overline{S}}
   \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WEV{E}{p.X}{\overline{S}}
  }
}
\inference{%
  \frac{
    \begin{array}{c}
    \WF{E}{}~~~~~~\Mod{X}{S}{S_1}\in E\\
    \WEV{E}{S}{\overline{S}}
   \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WEV{E}{X}{\overline{S}}
  }
}
\item[WEVAL-PATH-ALIAS]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{p}{\structe{\ModA{X}{p_1}}}\\
    \WEV{E;\elem_1;\ldots;\elem_i}{p_1}{\overline{S}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WEV{E}{p.X}{\overline{S}}
  }
}
\inference{%
  \frac{
    \begin{array}{c}
      \WF{E}{}~~~~~~~\ModA{X}{p_1}\in E\\
      \WEV{E}{p_1}{\overline{S}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WEV{E}{X}{\overline{S}}
  }
}
\item[WEVAL-PATH-TYPE]
\inference{%
  \frac{
    \begin{array}{c}
    \WEV{E}{p}{\structe{\ModType{Y}{S}}}\\
    \WEV{E;\elem_1;\ldots;\elem_i}{S}{\overline{S}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WEV{E}{p.Y}{\overline{S}}
  }
}
\item[WEVAL-PATH-TYPE]
\inference{%
  \frac{
    \begin{array}{c}
    \WF{E}{}~~~~~~~\ModType{Y}{S}\in E\\
    \WEV{E}{S}{\overline{S}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WEV{E}{Y}{\overline{S}}
  }
}
\end{description}
 Rules for typing module:
\begin{description}
\item[MT-EVAL]
\inference{%
  \frac{
 \WEV{E}{p}{\overline{S}}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WTM{E}{p}{\overline{S}}
  }
}
\item[MT-STR]
\inference{%
  \frac{
    \WTM{E}{p}{S}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WTM{E}{p}{S/p}
  }
}
\end{description}
The last rule, called strengthening is used to make all module fields
manifestly equal to themselves. The notation $S/p$ has the following
meaning:
\begin{itemize}
\item if $S\lra\struct{\elem_1;\dots;\elem_n}$ then
  $S/p=\struct{\elem_1/p;\dots;\elem_n/p}$ where $\elem/p$ is defined as
  follows:
  \begin{itemize}
  \item $\Def{}{c}{t}{T}/p\footnote{Opaque definitions are processed as assumptions.} ~=~ \Def{}{c}{t}{T}$
  \item $\Assum{}{c}{U}/p ~=~ \Def{}{c}{p.c}{U}$
  \item $\ModS{X}{S}/p ~=~ \ModA{X}{p.X}$
  \item $\ModA{X}{p'}/p ~=~ \ModA{X}{p'}$
  \item $\Ind{}{\Gamma_P}{\Gamma_C}{\Gamma_I}/p ~=~ \Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p}$
  \item $\Indpstr{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p'}{p} ~=~ \Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p'}$
  \end{itemize}
\item if $S\lra\functor{X}{S'}{S''}$ then $S/p=S$
\end{itemize}
The notation $\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p}$ denotes an
inductive definition that is definitionally equal to the inductive
definition in the module denoted by the path $p$. All rules which have
$\Ind{}{\Gamma_P}{\Gamma_C}{\Gamma_I}$ as premises are also valid for
$\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p}$. We give the formation rule
for $\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p}$ below as well as
the equality rules on inductive types and constructors. \\

The module subtyping rules:
\begin{description}
\item[MSUB-STR]
\inference{%
  \frac{
    \begin{array}{c}
      \WS{E;\elem_1;\dots;\elem_n}{\elem_{\sigma(i)}}{\elem'_i}
                                  \textrm{ \ for } i=1..m \\
      \sigma : \{1\dots m\} \ra \{1\dots n\} \textrm{ \ injective}
    \end{array}
  }{
    \WS{E}{\struct{\elem_1;\dots;\elem_n}}{\struct{\elem'_1;\dots;\elem'_m}}
  }
}
\item[MSUB-FUN]
\inference{%       T_1 -> T_2 <: T_1' -> T_2'
  \frac{
    \WS{E}{\overline{S_1'}}{\overline{S_1}}~~~~~~~~~~\WS{E;\ModS{X}{S_1'}}{\overline{S_2}}{\overline{S_2'}}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WS{E}{\functor{X}{S_1}{S_2}}{\functor{X}{S_1'}{S_2'}}
  }
}
% these are derived rules
% \item[MSUB-EQ]
% \inference{%
%   \frac{
%     \WS{E}{T_1}{T_2}~~~~~~~~~~\WTERED{}{T_1}{=}{T_1'}~~~~~~~~~~\WTERED{}{T_2}{=}{T_2'}
%   }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%     \WS{E}{T_1'}{T_2'}
%   }
% }
% \item[MSUB-REFL]
% \inference{%
%   \frac{
%     \WFT{E}{T}
%   }{
%     \WS{E}{T}{T}
%   }
% }
\end{description}
Structure element subtyping rules:
\begin{description}
\item[ASSUM-ASSUM]
\inference{%
  \frac{
    \WTELECONV{}{T_1}{T_2}
  }{
    \WSE{\Assum{}{c}{T_1}}{\Assum{}{c}{T_2}}
  }
}
\item[DEF-ASSUM]
\inference{%
  \frac{
    \WTELECONV{}{T_1}{T_2}
  }{
    \WSE{\Def{}{c}{t}{T_1}}{\Assum{}{c}{T_2}}
  }
}
\item[ASSUM-DEF]
\inference{%
  \frac{
    \WTELECONV{}{T_1}{T_2}~~~~~~~~\WTECONV{}{c}{t_2}
  }{
    \WSE{\Assum{}{c}{T_1}}{\Def{}{c}{t_2}{T_2}}
  }
}
\item[DEF-DEF]
\inference{%
  \frac{
    \WTELECONV{}{T_1}{T_2}~~~~~~~~\WTECONV{}{t_1}{t_2}
  }{
    \WSE{\Def{}{c}{t_1}{T_1}}{\Def{}{c}{t_2}{T_2}}
  }
}
\item[IND-IND]
\inference{%
  \frac{
    \WTECONV{}{\Gamma_P}{\Gamma_P'}%
    ~~~~~~~~\WTECONV{\Gamma_P}{\Gamma_C}{\Gamma_C'}%
    ~~~~~~~~\WTECONV{\Gamma_P;\Gamma_C}{\Gamma_I}{\Gamma_I'}%
  }{
    \WSE{\Ind{}{\Gamma_P}{\Gamma_C}{\Gamma_I}}%
        {\Ind{}{\Gamma_P'}{\Gamma_C'}{\Gamma_I'}}
  }
}
\item[INDP-IND]
\inference{%
  \frac{
    \WTECONV{}{\Gamma_P}{\Gamma_P'}%
    ~~~~~~~~\WTECONV{\Gamma_P}{\Gamma_C}{\Gamma_C'}%
    ~~~~~~~~\WTECONV{\Gamma_P;\Gamma_C}{\Gamma_I}{\Gamma_I'}%
  }{
    \WSE{\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p}}%
        {\Ind{}{\Gamma_P'}{\Gamma_C'}{\Gamma_I'}}
  }
}
\item[INDP-INDP]
\inference{%
  \frac{
    \WTECONV{}{\Gamma_P}{\Gamma_P'}%
    ~~~~~~\WTECONV{\Gamma_P}{\Gamma_C}{\Gamma_C'}%
    ~~~~~~\WTECONV{\Gamma_P;\Gamma_C}{\Gamma_I}{\Gamma_I'}%
    ~~~~~~\WTECONV{}{p}{p'}
  }{
    \WSE{\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p}}%
        {\Indp{}{\Gamma_P'}{\Gamma_C'}{\Gamma_I'}{p'}}
  }
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\item[MOD-MOD]
\inference{%
  \frac{
    \WSE{S_1}{S_2}
  }{
    \WSE{\ModS{X}{S_1}}{\ModS{X}{S_2}}
  }
}
\item[ALIAS-MOD]
\inference{%
  \frac{
    \WTM{E}{p}{S_1}~~~~~~~~\WSE{S_1}{S_2}
  }{
    \WSE{\ModA{X}{p}}{\ModS{X}{S_2}}
  }
}
\item[MOD-ALIAS]
\inference{%
  \frac{
      \WTM{E}{p}{S_2}~~~~~~~~
      \WSE{S_1}{S_2}~~~~~~~~\WTECONV{}{X}{p}
  }{
    \WSE{\ModS{X}{S_1}}{\ModA{X}{p}}
  }
}
\item[ALIAS-ALIAS]
\inference{%
  \frac{
    \WTECONV{}{p_1}{p_2}
  }{
    \WSE{\ModA{X}{p_1}}{\ModA{X}{p_2}}
  }
}
\item[MODTYPE-MODTYPE]
\inference{%
  \frac{
    \WSE{S_1}{S_2}~~~~~~~~\WSE{S_2}{S_1}
  }{
    \WSE{\ModType{Y}{S_1}}{\ModType{Y}{S_2}}
  }
}
\end{description}
New environment formation rules
\begin{description}
\item[WF-MOD]
\inference{%
  \frac{
    \WF{E}{}~~~~~~~~\WFT{E}{S}
  }{
    \WF{E;\ModS{X}{S}}{}
  }
}
\item[WF-MOD]
\inference{%
  \frac{
\begin{array}{c}
  \WS{E}{S_2}{S_1}\\
  \WF{E}{}~~~~~\WFT{E}{S_1}~~~~~\WFT{E}{S_2}
\end{array}
  }{
    \WF{E;\Mod{X}{S_1}{S_2}}{}
  }
}

\item[WF-ALIAS]
\inference{%
  \frac{
    \WF{E}{}~~~~~~~~~~~\WTE{}{p}{S}
  }{
    \WF{E,\ModA{X}{p}}{}
  }
}
\item[WF-MODTYPE]
\inference{%
  \frac{
    \WF{E}{}~~~~~~~~~~~\WFT{E}{S}
  }{
    \WF{E,\ModType{Y}{S}}{}
  }
}
\item[WF-IND]
\inference{%
  \frac{
    \begin{array}{c}
      \WF{E;\Ind{}{\Gamma_P}{\Gamma_C}{\Gamma_I}}{}\\
      \WT{E}{}{p:\struct{\elem_1;\dots;\elem_n;\Ind{}{\Gamma_P'}{\Gamma_C'}{\Gamma_I'};\dots}}\\
      \WS{E}{\Ind{}{\Gamma_P'}{\Gamma_C'}{\Gamma_I'}}{\Ind{}{\Gamma_P}{\Gamma_C}{\Gamma_I}}
    \end{array}
  }{%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    \WF{E;\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p}}{}
  }
}
\end{description}
Component access rules
\begin{description}
\item[ACC-TYPE]
\inference{%
  \frac{
    \WTEG{p}{\struct{\elem_1;\dots;\elem_i;\Assum{}{c}{T};\dots}}
  }{
    \WTEG{p.c}{T}
  }
}
\\
\inference{%
  \frac{
    \WTEG{p}{\struct{\elem_1;\dots;\elem_i;\Def{}{c}{t}{T};\dots}}
  }{
    \WTEG{p.c}{T}
  }
}
\item[ACC-DELTA]
Notice that the following rule extends the delta rule defined in
section~\ref{delta}
\inference{%
  \frac{
    \WTEG{p}{\struct{\elem_1;\dots;\elem_i;\Def{}{c}{t}{U};\dots}}
  }{
    \WTEGRED{p.c}{\triangleright_\delta}{t}
  }
}
\\
In the rules below we assume $\Gamma_P$ is $[p_1:P_1;\ldots;p_r:P_r]$,
  $\Gamma_I$ is $[I_1:A_1;\ldots;I_k:A_k]$, and $\Gamma_C$ is
  $[c_1:C_1;\ldots;c_n:C_n]$
\item[ACC-IND]
\inference{%
  \frac{
    \WTEG{p}{\struct{\elem_1;\dots;\elem_i;\Ind{}{\Gamma_P}{\Gamma_C}{\Gamma_I};\dots}}
  }{
    \WTEG{p.I_j}{(p_1:P_1)\ldots(p_r:P_r)A_j}
  }
}
\inference{%
  \frac{
    \WTEG{p}{\struct{\elem_1;\dots;\elem_i;\Ind{}{\Gamma_P}{\Gamma_C}{\Gamma_I};\dots}}
  }{
    \WTEG{p.c_m}{(p_1:P_1)\ldots(p_r:P_r){C_m}{I_j}{(I_j~p_1\ldots
       p_r)}_{j=1\ldots k}}
  }
}
\item[ACC-INDP]
\inference{%
  \frac{
    \WT{E}{}{p}{\struct{\elem_1;\dots;\elem_i;\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p'};\dots}}
  }{
    \WTRED{E}{}{p.I_i}{\triangleright_\delta}{p'.I_i}
  }
}
\inference{%
  \frac{
    \WT{E}{}{p}{\struct{\elem_1;\dots;\elem_i;\Indp{}{\Gamma_P}{\Gamma_C}{\Gamma_I}{p'};\dots}}
  }{
    \WTRED{E}{}{p.c_i}{\triangleright_\delta}{p'.c_i}
  }
}

\end{description}

% %%% replaced by \triangle_\delta
% Module path equality is a transitive and reflexive closure of the
% relation generated by ACC-MODEQ and ENV-MODEQ.
% \begin{itemize}
% \item []MP-EQ-REFL
% \inference{%
%   \frac{
%     \WTEG{p}{T}
%   }{
%     \WTEG{p}{p}
%   }
% }
% \item []MP-EQ-TRANS
% \inference{%
%   \frac{
%     \WTEGRED{p}{=}{p'}~~~~~~\WTEGRED{p'}{=}{p''}
%   }{
%     \WTEGRED{p'}{=}{p''}
%   }
% }

% \end{itemize}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "Reference-Manual"
%%% End:


\documentclass{article}
%\usepackage[a4paper,margin=2.2cm,footskip=.5cm]{geometry}
\usepackage{fullpage}
%-----------------Hyperlink Packages--------------------
\usepackage{hyperref}
\hypersetup{
	 colorlinks   = true,
     citecolor    = black,
     linkcolor    = black,
     urlcolor     = black
}
%-----------------Figure Packages--------------------
\usepackage{graphicx}                       % For figures
\usepackage{stfloats}
\usepackage{caption}
\usepackage[export]{adjustbox}
\usepackage[tight,footnotesize]{subfigure}  % Create subfigures, ie 1A, 1B
%\usepackage{epsfig} % for postscript graphics files
%------------------Math Packages------------------------
\usepackage{amssymb,amsmath}
\usepackage{textcomp}
\usepackage{mdwmath}
\usepackage{mdwtab}
\usepackage{eqparbox}
%------------------Table Packages-----------------------
\usepackage{rotating}                     % Used to rotate tables
\usepackage{array}                        % Fixed column widths for tables
%-----------------Algorithm Packages--------------------
\usepackage{listings}                     % Source code
\usepackage{algorithm}                    % Pseudo Code
\usepackage[noend]{algpseudocode}
%---------------------------------------------------------

\begin{document}

\title {
	DIP Final Project \\
	Haze Removal
}

\author {
	Qiuyi Zhang 12330402 \\
	\href{mailto:joyeec9h3@gmail.com}{joyeec9h3@gmail.com}
}

\date {
	\today
}

\maketitle
\tableofcontents

\section{Introduction}

\subsection{Haze Removal Using Dark Channel Prior}
In computer vision and computer graphics, the formation of a hazy image is usually described by the following model:

$$
\mathbf{I}(\mathbf{x}) = \mathbf{J}(\mathbf{x})t(\mathbf{x}) + \mathbf{A}(1 - t(\mathbf{x}))
$$

where $\mathbf{I}$ is the observed intensity, $\mathbf{J}$ is the scene radiance, $\mathbf{A}$ is the global atmospheric light, and $t$ is the medium transmission. With this model, the goal of haze removal is to recover $\mathbf{J}$, $\mathbf{A}$, and $t$ when $\mathbf{I}$ is given.

In \cite{he2009single}, He proposed an approach to use the dark channel prior for haze removal. For an image $\mathbf{J}$, the dark channel of $\mathbf{J}$ is defined as

$$
J^{dark}(\mathbf{x}) = min_{c \in \{r,g,b\}}( min_{\mathbf{y} \in \Omega(\mathbf{x})}(J^c(\mathbf{y})))
$$

When $\mathbf{J}$ is a haze-free outdoor image, the intensity of $J^{dark}$ in the non-sky region is close to zero. Using this prior, the transmission can be estimated by:

$$
\tilde{t}(\mathbf{x}) = 1 - \omega min_{c}( min_{\mathbf{y} \in \Omega(\mathbf{x})}(\frac{I^c(\mathbf{y})}{A^c}))
$$

where $\omega \in (0, 1]$ is a constant parameter for keeping a certain amount of haze to make the output more natural Here $min_{c}( min_{\mathbf{y} \in \Omega(\mathbf{x})}(\frac{I^c(\mathbf{y})}{A^c}))$ is actually the dark channel of the normalized haze image $\frac{I^c(\mathbf{y})}{A^c}$.

To automatically obtain $\mathbf{A}$ from the input image, we can first pick the top $p\%$ brightest pixels in the dark channel, then, for each color channel, we chose the one with the highest intensity in the input image $\mathbf{I}$ as its atmospheric light.

Now that we have $\tilde{t}(\mathbf{x})$, $\mathbf{A}$ and $\mathbf{I}$, we can recover the haze-free image with:

$$
\mathbf{J}(\mathbf{x}) = \frac{\mathbf{I}(\mathbf{x}) - \mathbf{A}}{max(\tilde{t}(\mathbf{x}),t_0)} + \mathbf{A}
$$

where $t_0$ is a lower bound for $t$ to preserve a certain amount of haze in dense haze regions.

\subsection{Guided filter}

Since a transmission map is just an alpha map, we can apply soft matting to refine the estimated transmission. An efficient approach, as proposed in \cite{he2010guided}, is to use the input image $\mathbf{I}$ to guide the filter $\tilde{t}(\mathbf{x})$.

Define $q$ as a linear transform of a color image $\mathbf{I}$ in a window $\omega_k$ with radius $r$ centered at the pixel $k$:

$$
q_i = \mathbf{a}_k^T\mathbf{I}_i + b_k, \forall i \in \omega_k
$$

where $\mathbf{I}_k$ is a $3 \times 1$ color vector, $\mathbf{a}_k$ is a $3 \times 1$ coefficient vector, $b_k$ is a scalar coefficient. For a filter $p$, the coefficients $\mathbf{a}$ and $b$ can be computed with:

\begin{align*}
\mathbf{a}_k &= (\Sigma_k + \epsilon U)^{-1}(\frac{1}{|\omega|}\sum_{i \in \omega_k}\mathbf{I}_i p_i - \mu_k \bar{p}_k) \\
b_k &= \bar{p}_k - \mathbf{a}_k^T\mu_k \\
q_i &= \bar{\mathbf{a}}_i^T\mathbf{I}_i + \bar{b}_i
\end{align*}

where $\Sigma_k$ is the covariance matrix of $\mathbf{I}$ in $\omega_k$, $\epsilon$ is regularization parameter preventing $\mathbf{a}_k$ from being too large, $U$ is the $3 \times 3$ identity matrix, $\mu_k$ is the mean of $\mathbf{I}$ in $\omega_k$, $|\omega|$ is the number of pixels in $\omega_k$, and $\bar{p}_k$ is the mean of $p$ in $\omega_k$. Then, the resulting $q$ is the desired filter under the guidance of $\mathbf{I}$.

\section{Implementation}

Algorithm~\ref{alg:hr} describes how to remove the haze from the image using the dark channel prior. To refine the transmission map, we implement Algorithm~\ref{alg:gf}.

\begin{algorithm}[H]
\centering
\caption{Haze Removal Using Dark Channel Prior}
\label{alg:hr}
  \begin{algorithmic}[1]
    \Function{Dehaze}{$\mathbf{I}$, $t_{min}$, $A_{max}$, $w$, $p$, $\omega$, $r$, $\epsilon$}
        \Comment{$\mathbf{I}$ is the hazy image}
        \State Compute $\mathbf{I}^{dark}$ with $\mathbf{I}^{dark}(\mathbf{x}) = min_{c \in \{r,g,b\}}( min_{\mathbf{y} \in \Omega(\mathbf{x})}(I^c(\mathbf{y})))$
        \State Find the indexes $\mathbf{D}$ for highest $p\%$  pixels in $\mathbf{I}^{dark}$
        \State Compute atmosphere light $\mathbf{A}$ with $\mathbf{A}_c = max_{\mathbf{d} \in  \mathbf{D}}(\mathbf{I}_c(\mathbf{d}))$
        \State Threshold $\mathbf{A}$ with $A_{max}$
        \State Estimate transmission $\tilde{t}$ with $\tilde{t}(\mathbf{x}) \gets 1 - \omega min_{c}( min_{\mathbf{y} \in \Omega(\mathbf{x})}(\frac{I^c(\mathbf{y})}{A^c}))$
        \State $\tilde{t} \gets$ \Call{Guided Filter}{$\mathbf{I}$, $\tilde{t}$, $r$, $\epsilon$}
        \State Recover $\mathbf{J}$ with $\mathbf{J}(\mathbf{x}) = \frac{\mathbf{I}(\mathbf{x}) - \mathbf{A}}{max(\tilde{t}(\mathbf{x}),t_0)} + \mathbf{A}$
      \State \Return $\mathbf{J}$
    \EndFunction
  \end{algorithmic}
\end{algorithm}

\begin{algorithm}[H]
\centering
\caption{Guided Filter}
\label{alg:gf}
  \begin{algorithmic}[1]
    \Function{Guided Filter}{$\mathbf{I}$, $p$, $r$, $\epsilon$}
        \For{each pixel $k$ in $p$}
	        \State $\omega_k$ is the window with radius $r$ for pixel $k$
	        \State Compute $\Sigma_k$, the convariance matrix of $\mathbf{I}$ in $\omega_k$
	        \State Compute $|\omega|$, the number of pixels in $omega_k$
	        \State Compute $\mu_k$, the mean of $\mathbf{I}$ in $\omega_k$
	        \State Compute $\bar{p}_k$, the mean of $p$ in $\omega_k$
	        \State $\mathbf{a}_k = (\Sigma_k + \epsilon U)^{-1}(\frac{1}{|\omega|}\sum_{i \in \omega_k}\mathbf{I}_i p_i - \mu_k \bar{p}_k)$
	        \State $b_k = \bar{p}_k - \mathbf{a}_k^T\mu_k$
	        \State $q_k = \bar{\mathbf{a}}_k^T\mathbf{I}_k + \bar{b}_k$
        \EndFor
      \Return $\mathbf{q}$
    \EndFunction
  \end{algorithmic}
\end{algorithm}

\section{Results and Analysis}

The results in Figure~\ref{fig:result1} -~\ref{fig:result4} are generated with $t_{min} = 0.2, A_{max} = 220, w = 15, p=0.1, \omega=0.95,r=40, \epsilon=10^{-3}$. From top to bottom are the input hazy image, the dark channel, the estimated transmission map, the refined transmission map, the image dehazed with raw transmission map, and the image dehazed with refined transmission map.

From the results, it is easy to see that this approach can remove most of the haze in the image. The refined transmission map, however, can lead to much better results than the raw estimates.

Since generating the dark channel is somewhat similar to applying a min filter to the image, areas with low intensity in the dark channel will greatly affect its neighboring pixels. Therefore, when there are drastic transitions in the dark channel (e.g. between the sky/fog and salient objects), the area with low intensities will ``expand''. For example, in Figure~\ref{fig:result1}, the twigs in the dark channel of the forest are much thicker than in the original image. The leaves with good visibility in the front have low values in the dark channel, resulting in those small squares (usually with the same size as the window) in the dark channel map. Since the raw transmission map is actually calculated with the dark channel of the normalized image, this effect directly leads to the translucent borders around the twigs in the image dehazed with this raw estimate. Other images with this kind of drastic transitions in the dark channel also have the same problems. For example, the distant cones in the second image and the leaves in the last image of Figure~\ref{fig:result1}, the tall buildings in the first three images of Figure~\ref{fig:result2}, the roofs in the first image and the trunk in the second image of Figure~\ref{fig:result3}, etc. all have this kind of borders around them.

From the results it is obvious to see that after guiding the transmission map with the original image, the refined transmission map preserves much more detail than the raw estimate. Most objects become distinguishable in this refined map, and the edges in the tranmission map are closer to those in the original image. Therefore, the images dehazed with this refined transmission map no longer have the weird borders as in those dehazed with the raw estimate.

Since the haze will usually increase the brightness of the image (because they are usually white or gray-ish), the images with a large amount of haze will become relatively darker after dehazing, e.g. the first image in Figure~\ref{fig:result1}, the first two images in Figure~\ref{fig:result2},  the last image in Figure~\ref{fig:result4}. The images with just a thin layer of haze will have a better contrast after dehazing, e.g. the fourth image in Figure~\ref{fig:result1}, the fourth image in Figure~\ref{fig:result2}, the second and the third image in Figure~\ref{fig:result3}, and the first image in Figure~\ref{fig:result4}

During the experiment, we discover that the choice of parameters is crutial in the quality of haze removal. When there are sky regions in the image, $\mathbf{A}$ tends to be large. In this case, we need a higher $t_{min}$ or lower $A_{min}$ to threshold the computation so as to restrict the recovered $\mathbf{J}$ in a reasonable range. To make the guidance more accurate, $r$ should be larger than $w/2$. Larger $r$ can lead to better results, but it will also slow down the process.

\begin{figure}[H]
    \centering
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../img/canon7.jpg}
        \includegraphics[height=0.18\linewidth]{../img/cones.jpg}
        \includegraphics[height=0.18\linewidth]{../img/forest.jpg}
        \includegraphics[height=0.18\linewidth]{../img/flag.jpg}
        \includegraphics[height=0.18\linewidth]{../img/house-input.png}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/canon7-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/cones-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/forest-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/flag-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/house-input-dark.png}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/canon7-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/cones-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/forest-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/flag-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/house-input-rawt.png}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/canon7-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/cones-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/forest-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/flag-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/house-input-refinedt.png}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/canon7-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/cones-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/forest-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/flag-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/house-input-radiance-rawt.png}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/canon7-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/cones-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/forest-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/flag-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/house-input-radiance-refinedt.png}
    \end{minipage}
    \caption{From top to bottom: input hazy image, dark channel, estimated transmission map, refined transmission map, image dehazed with raw estimate, image dehazed with refined estimate}
    \label{fig:result1}
\end{figure}

\begin{figure}[H]
    \centering
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../img/IMG_8763.jpg}
        \includegraphics[height=0.18\linewidth]{../img/IMG_8766.jpg}
        \includegraphics[height=0.18\linewidth]{../img/ny17_photo.jpg}
        \includegraphics[height=0.18\linewidth]{../img/pumpkins.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/IMG_8763-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/IMG_8766-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny17_photo-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/pumpkins-dark.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/IMG_8763-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/IMG_8766-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny17_photo-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/pumpkins-rawt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/IMG_8763-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/IMG_8766-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny17_photo-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/pumpkins-refinedt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/IMG_8763-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/IMG_8766-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny17_photo-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/pumpkins-radiance-rawt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/IMG_8763-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/IMG_8766-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny17_photo-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/pumpkins-radiance-refinedt.jpg}
    \end{minipage}
    \caption{From top to bottom: input hazy image, dark channel, estimated transmission map, refined transmission map, image dehazed with raw estimate, image dehazed with refined estimate}
    \label{fig:result2}
\end{figure}

\begin{figure}[H]
    \centering
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../img/tiananmen1.png}
        \includegraphics[height=0.18\linewidth]{../img/swan.jpg}
        \includegraphics[height=0.18\linewidth]{../img/toys.jpg}
        \includegraphics[height=0.18\linewidth]{../img/ny12_photo.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/tiananmen1-dark.png}
        \includegraphics[height=0.18\linewidth]{../result/swan-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/toys-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny12_photo-dark.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/tiananmen1-rawt.png}
        \includegraphics[height=0.18\linewidth]{../result/swan-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/toys-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny12_photo-rawt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/tiananmen1-refinedt.png}
        \includegraphics[height=0.18\linewidth]{../result/swan-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/toys-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny12_photo-refinedt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/tiananmen1-radiance-rawt.png}
        \includegraphics[height=0.18\linewidth]{../result/swan-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/toys-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny12_photo-radiance-rawt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/tiananmen1-radiance-refinedt.png}
        \includegraphics[height=0.18\linewidth]{../result/swan-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/toys-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/ny12_photo-radiance-refinedt.jpg}
    \end{minipage}
    \caption{From top to bottom: input hazy image, dark channel, estimated transmission map, refined transmission map, image dehazed with raw estimate, image dehazed with refined estimate}
    \label{fig:result3}

\end{figure}

\begin{figure}[H]
    \centering
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../img/stadium1.jpg}
        \includegraphics[height=0.18\linewidth]{../img/yellowmountain.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/stadium1-dark.jpg}
        \includegraphics[height=0.18\linewidth]{../result/yellowmountain-dark.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/stadium1-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/yellowmountain-rawt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/stadium1-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/yellowmountain-refinedt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/stadium1-radiance-rawt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/yellowmountain-radiance-rawt.jpg}
    \end{minipage}
    \begin{minipage}[b]{\linewidth}
        \centering
        \includegraphics[height=0.18\linewidth]{../result/stadium1-radiance-refinedt.jpg}
        \includegraphics[height=0.18\linewidth]{../result/yellowmountain-radiance-refinedt.jpg}
    \end{minipage}
    \caption{From top to bottom: input hazy image, dark channel, estimated transmission map, refined transmission map, image dehazed with raw estimate, image dehazed with refined estimate}
    \label{fig:result4}
\end{figure}

\section{Discussion}

In the sky region, $\tilde{t} \to 0$ and $A^c \to L - 1$. Therefore the equation:

$$
\mathbf{J}(\mathbf{x}) = \frac{\mathbf{I}(\mathbf{x}) - \mathbf{A}}{max(\tilde{t}(\mathbf{x}),t_0)} + \mathbf{A}
$$

is prone to error in images with sky regions. One possible solution is to set a higher $t_0$ than the one ($0.1$) suggested in \cite{he2009single}. Another solution is to set a lower $A_max$ (which is not mentioned in \cite{he2009single}). For different images, we need different $t_0$ and $A_max$ to produce ideal results. Some results with different configuration of  $t_0$ and $A_max$ are listed in Figure~\ref{fig:sp1} -\ref{fig:sp4}.

\begin{figure}[htp]
    \centering
    \begin{minipage}[b]{0.319\linewidth}
        \centering
        \includegraphics[width=\linewidth]{../img/IMG_8763.jpg}
    \end{minipage}
    \begin{minipage}[b]{0.66\linewidth}
       \centering
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-refinedt.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-refinedt-20-170-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-refinedt-50-190-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-refinedt-50-220-15-40.jpg}
        \\
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-radiance-refinedt.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-radiance-refinedt-20-170-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-radiance-refinedt-50-190-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8763-radiance-refinedt-50-220-15-40.jpg}
    \end{minipage}

    \captionsetup{singlelinecheck=off}
    \caption{Leftmost image: the input hazy image. On the right side, the top row are tranmission maps with
        $t_{min} = 0.2, A_{max} = 220$;
        $t_{min} = 0.2, A_{max} = 170$;
        $t_{min} = 0.5, A_{max} = 190$;
        $t_{min} = 0.5, A_{max} = 220$;
        the bottom row are corresponding dehazed images.
    }
    \label{fig:sp1}
\end{figure}

\begin{figure}[h]
    \centering
    \begin{minipage}[b]{0.319\linewidth}
        \centering
        \includegraphics[width=\linewidth]{../img/IMG_8766.jpg}
    \end{minipage}
    \begin{minipage}[b]{0.66\linewidth}
       \centering
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-refinedt.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-refinedt-20-170-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-refinedt-50-190-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-refinedt-50-220-15-40.jpg}
        \\
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-radiance-refinedt.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-radiance-refinedt-20-170-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-radiance-refinedt-50-190-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/IMG_8766-radiance-refinedt-50-220-15-40.jpg}
    \end{minipage}

    \captionsetup{singlelinecheck=off}
    \caption{Leftmost image: the input hazy image. On the right side, the top row are tranmission maps with
        $t_{min} = 0.2, A_{max} = 220$;
        $t_{min} = 0.2, A_{max} = 170$;
        $t_{min} = 0.5, A_{max} = 190$;
        $t_{min} = 0.5, A_{max} = 220$;
        the bottom row are corresponding dehazed images.
    }
    \label{fig:sp2}
\end{figure}

\begin{figure}[H]
    \centering
    \begin{minipage}[b]{0.319\linewidth}
        \centering
        \includegraphics[width=\linewidth]{../img/ny17_photo.jpg}
    \end{minipage}
    \begin{minipage}[b]{0.66\linewidth}
       \centering
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-refinedt.jpg}
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-refinedt-20-170-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-refinedt-50-190-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-refinedt-50-220-15-40.jpg}
        \\
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-radiance-refinedt.jpg}
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-radiance-refinedt-20-170-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-radiance-refinedt-50-190-15-40.jpg}
        \includegraphics[width=0.24\linewidth]{../result/ny17_photo-radiance-refinedt-50-220-15-40.jpg}
    \end{minipage}

    \captionsetup{singlelinecheck=off}
    \caption{Leftmost image: the input hazy image. On the right side, the top row are tranmission maps with
        $t_{min} = 0.2, A_{max} = 220$;
        $t_{min} = 0.2, A_{max} = 170$;
        $t_{min} = 0.5, A_{max} = 190$;
        $t_{min} = 0.5, A_{max} = 220$;
        the bottom row are corresponding dehazed images.
    }
    \label{fig:sp3}
\end{figure}

\begin{figure}[H]
    \centering
    \begin{minipage}[b]{0.319\linewidth}
        \centering
        \includegraphics[width=\linewidth]{../img/tiananmen1.png}
    \end{minipage}
    \begin{minipage}[b]{0.66\linewidth}
       \centering
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-refinedt.png}
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-refinedt-20-170-15-40.png}
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-refinedt-50-190-15-40.png}
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-refinedt-50-220-15-40.png}
        \\
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-radiance-refinedt.png}
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-radiance-refinedt-20-170-15-40.png}
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-radiance-refinedt-50-190-15-40.png}
        \includegraphics[width=0.24\linewidth]{../result/tiananmen1-radiance-refinedt-50-220-15-40.png}
    \end{minipage}

    \captionsetup{singlelinecheck=off}
    \caption{Leftmost image: the input hazy image. On the right side, the top row are tranmission maps with
        $t_{min} = 0.2, A_{max} = 220$;
        $t_{min} = 0.2, A_{max} = 170$;
        $t_{min} = 0.5, A_{max} = 190$;
        $t_{min} = 0.5, A_{max} = 220$;
        the bottom row are corresponding dehazed images.
    }
     \label{fig:sp4}
\end{figure}
\bibliography{dehaze}

\bibliographystyle{acm}

\end{document}