% !Mode:: "TeX:UTF-8"
% !TEX program  = xelatex
\newgeometry{margin=2.1in, marginpar=1.9in}

\增加{Not from the original}{Since I'm using Linux OS, there are some compatibility problems with my LibreOffice viewer, which require a lot of time to solve, I change the method and re-type the article by \LaTeX{}, therefore I can use ``changes'' package to achieve the same function. By the way, I check this article through the \href{https://docs.qq.com}{QQ Docs}.}

\注释{Some words of the section title needs to be capitalized, say ``Introduction''.}
\注释{Normally the ﬁrst line of a paragraph is indented.}
\注释{Please pay attention to the articles and the verb forms.}

\section{What is \增加{An article is missing.}{a} mathematical article?}
Mathematical article including\删除{The colon is unnecessary.}{:} 1. research paper, 2. survey article or review paper, 3. referee report, 4. research proposal, 5. book review, 6. paper review, 7. reading notes and so on. Normally, a mathematical article consists of \textbf{title, abstract, introduction, main body of the article, conclusions, acknowledgments, references and appendix}. The following things we need to know, before writing an article:
\begin{itemize}
	\item \textbf{First of all}, we should \textbf{think \替换{}{about who}{who} will read} the article to make sure our readers understand the article.
	\item \textbf{Secondly}, we ought to \textbf{\替换{}{search for}{search} writing materials about what we are going to write, which may help us understand} academic terminology.
	\item \textbf{Thirdly}, we need to write \增加{}{an} article \textbf{briefly, precisely and logically}.
	\item \textbf{Fourthly}, we should \textbf{revise} our article repeatedly.
	\item \textbf{Fifthly}, we need to \textbf{take \替换{}{advice}{advise}} from people who have experience in article writing.
\end{itemize}

\section{Title}
A good title is not only \textbf{brief and precise} to tell the \textbf{main contributions or innovations} of the article but also \textbf{interesting} enough to attract readers who would read the article.
\begin{itemize}
	\item Firstly, avoid meaningless words or \增加{}{avoid} to talk in general.
	\item Secondly, be specific to reflect works in the article.
	\item Thirdly, using \增加{An article is missing.}{an} interesting title if we writing a review or expository article.
\end{itemize}

\section{Abstract}
An abstract is the heart of the article, it not only is written in \textbf{brief and precise} words\删除{The comma appears to be unnecessary.}{,} but also tell readers the \textbf{purpose, method and result of the article and make sure most of \增加{}{the} readers understand the abstract}. We should write the abstract after finishing the first edition of the article, avoid using sentences from \增加{}{the} introduction and conclusions. Normally, an abstract only contains 100 to 300 words.

\section{introduction}
An introduction should give \替换{}{an}{the} accurate background of the study problem objectively, including \textbf{history of the study, motivation and purpose} of the article and the \textbf{results} of the article.

\textbf{In the beginning}, we should mention the problem we meet in the study.

\textbf{In the middle}, we should define the problem and explain what attempts to do, not only summarize the results, progress achieved and problem left, but also mention our plan for the unsolved problem.

\textbf{In the end}, we give \替换{}{an}{the} outline of the article structure.

\section{The main body of the article}
If \增加{}{the} article focuses on theoretic math, the main body consists of \textbf{preliminaries, main result, the outline of proofs and extensions}. In \增加{}{the} preliminaries part, we talk about notation and lemma. In main results part, conclusion is mentioned and discussed.

If \增加{}{the} article focuses on computational math or applied math, the main body consists of \textbf{the problem or governing equations, the numerical method or experimental method, theoretical analysis, numerical results and conclusions}.

When we doing numerical \替换{}{experiments}{experiment}, we should \textbf{explain details} such as parameters and \替换{}{methods}{method} we use firstly. Next, using \textbf{moderate figures and tables} to contract may explain the results better. Thirdly, using \textbf{good examples}, which is widely used, may be persuasive. At last, \textbf{explanations about innovations} of the numerical results is necessary.

\section{Conclusions}
In the conclusions part, the following information we should concern.
\begin{enumerate}
	\item Provide a \textbf{summary of the main contribution} briefly, emphasize the \替换{}{importance}{important} of the results and try not to use repeated words.
	\item Explain the questions mentioned in introduction.
	\item Discuss the \textbf{general problems} which not mentioned in the article.
	\item Explain the \textbf{implications} of our research.
	\item Identify the next step or look to \增加{}{the} future.
\end{enumerate}

\section{Acknowledgments}
Acknowledgments \替换{}{are}{is} the part where we thank \删除{}{to} \textbf{people and institutions} who helped us to finish the article. We also need to mention \textbf{reviewers} if they give us significant advice which \替换{}{helps}{help} us revise the article.

\section{Reference}
Reference is the part where we put the \textbf{papers or books} which are used in the article. The paper in reference contains the following information: authors, article title, journal title, volume number, issue number, publication year and page numbers. The book in reference contains the following information: author, book title, publisher, edition information, publication city. Conference paper in reference also \替换{}{needs}{need} to mention editors. Dissertation in reference \替换{}{has}{have} to show the institution information.

\section{Appendix}
Appendix is the part where we put \textbf{long and complex mathematical formulas of proof} and some \textbf{relevant lemmas} but not concerned in the paper.

\section{Others}
\begin{enumerate}
	\item Name: take the responsibility of the article
	\item Date: get the data when we revise our article
	\item Keywords and subject classification: make your article easy to get found.
	\item Chapter title: interested readers and make the article logical.
\end{enumerate}

\section{Conclusion}
We not only acquire the basic knowledge of writing but also need to \textbf{read western mathematician papers} to enlarge our English mathematical vocabulary.


\chapter*{Abstract}

The in-medium similarity renormalization group (IMSRG)
is an \abinitio{} many-body method used to great success
to solve the time-independent Schr\"{o}dinger equation in medium-mass nuclear systems.
Its computational cost scales polynomially in the size of the truncated model space,
and its formalism is highly flexible,
leading to multiple variants that have been developed to extend its original closed-shell
formulation to open-shell systems.

The current state-of-the-art implementations truncate the IMSRG equations
at the normal-ordered two-body level,
the first non-trivial order in the expansion.
In this work, we seek to systematically study the effects
of extending this truncation to the normal-ordered three-body level,
the so-called IMSRG(3) approximation.
Exploitation of symmetries is essential to making IMSRG(3) calculations tractable.
We present the reduced $J$-scheme IMSRG(3) working equations,
which we arrive at by applying angular-momentum reduction to the IMSRG(3) for spherical systems.

We use our implementation of the $J$-scheme IMSRG(3)
to investigate three-body contributions
that first appear in the IMSRG(3)
in light and medium-mass nuclei.
We introduce approximate IMSRG(3) truncations
that leave out the most expensive parts of the IMSRG(3).
We find that in ${}^{4}\text{He}$
and ${}^{16}\text{O}$
in a restricted $\emax=2$ model space,
these approximate IMSRG(3) truncation schemes
deliver small, sub-percent corrections to the ground-state energies
and larger corrections to radii.
Further, by investigating the behavior
under the removal or inclusion of certain terms,
we see that the organization by computational cost
used to set up our approximate truncation schemes
is poorly motivated
and some computationally more expensive terms
provide larger corrections to ground-state energies
than the cheaper terms in the truncation.
This work is a key step towards
high-precision many-body calculations of medium-mass nuclei
in the IMSRG.


\section{Introduction}
\label{s:intro}
%
In many cases, establishing the correctness of a program can be
formulated as proving that the program has a particular dependent
type.
%
Different classes of dependent types provide trade-offs in terms of
program properties that they can express versus the degree to which
checking that a program has a given type, or inferring its type, can
be automated.

% refinement types: context:
Previous work has demonstrated that the class of \emph{refinement
  types} often occupy a sweet spot: they can often express program
properties of interest, but can also by inferred with some degree of
optimization.
%
In particular, various engines for inferring refinement types have
also been proposed that themselves provide tradeoffs between
expressiveness and automation.
% HORS model checking:
Engines based on model checking \emph{higher-order recursion schemes
  (HORS)} are completely automatic when applied to programs that
operate on only scalar data~\cite{kobayashi09}.
%
Other approaches for inferring dependent types from
counterexamples~\cite{kobayashi11} can only infer a restricted class
of predicates over recursive datatypes, such as length functions.
% Liquid Types:
Engines that infer \emph{liquid types} require a user to provide
predicates from which the refinement types of expressions are
constructed, but are complete for inferring such
types~\cite{kawaguchi09,rondon08,vazou14}.
% heavyweight stuff:
Other techniques attempt to infer refinement types completely
automatically, using either techniques for solving constraint-logic
programs (such as interpolation) or inference problems in machine
learning~\cite{terauchi10}.

% problem: all refinement types have to be given:
While multiple approaches have been proposed that address
refinement-type inference, all such approaches, to some extent,
require a programmer to provide definitions of refinement types of
interest.
%
I.e., many existing type inference engines synthesize refinements of
base and function types, and infer that program expressions have such
types.
%
However, even the most automated approaches~\cite{unno09} require that
a programmer provide complete definitions of recursive types, such as
lists or trees, which are chosen using conventional type-inference
heuristics~\cite{pierce98}.
%
While a programmer typically must provide at least a few such types
explicitly as components of the program's specification, requiring the
programmer to provide complete definitions of types of values
constructed during intermediate steps of a computation is an
unfortunate burden.

% our contribution: can synthesize refinement types
In this paper, we propose a type-inference engine, named \sys, that
both synthesizes complete definitions of recursive refinement types
and infers the types of program expressions.
%
\sys does not place any restriction on the interaction of core
features of a language: it can potentially infer types for programs
that make arbitrary use of higher-order functions and recursive data
types.
%
\sys can synthesize recursive types that are abstracted over both
base, function, and recursive values;
%
such types are particularly useful because they can be applied to
variables in the context of an expression to be typed.

% key feature: reduction to logic programming
The key feature of \sys is that, given program $P$ and expected type
$\tau$, it performs a syntax-directed translation to a system of
\emph{Constrained Horn Clauses (CHCs)} $\mathcal{C}$, such that each
valid typing $P$ as having $\tau$ corresponds to a solution to
$\mathcal{C}$.
%
In particular, each solution of $\mathcal{C}$ defines, for each
subexpression $e$ of $P$, both recursive refinement types that express
requirements of all recursive values bound in a context of $e$, and,
if all requirements are satisfied, a guaranteed recursive refinement
type of the value of $e$.
%
In principle, $\mathcal{C}$ can be solved by any solver for recursive
CHCs that finds solutions in combinations of theories used to model
program base values (i.e., linear arithmetic) and uninterpreted
functions with equality~\cite{bjorner13}.
%
In practice, \BH{??}

% discuss as framework that can be tuned to give liquid types
A key feature of our translation is that it can serve as a framework
that can be instantiated to simulate several previously-developed
type-inference engines, depending on desired trade-offs between
effectiveness and automation.
%
In particular, an engine that satisfies the convergence properties of
Liquid Types, but can potentially infer a strictly richer class of
refinement types, can be simulated by rewriting clauses generated by
the syntax-directed translation to use given predicates.

% outline rest of paper
The rest of this paper is organized as follows.
%
\autoref{s:overview} introduces \sys by example.
%
\autoref{s:background} reviews technical foundations on which \sys is
built.
%
\autoref{s:approach} describes \sys in technical detail.
%
\autoref{s:eval} describes our evaluation of \sys.
%
\autoref{s:relwk} compares \sys to related work, and %
\autoref{s:conclusion} concludes.


\chapter{FOB Formalism}
\label{chap:FOB}
The effect of the nuclei on the electrons is normally handled via the Hamiltonian. This is dependent on nuclear positions and is in turn used in the Schr\"odinger equation to propagate the electron dynamics. Often the construction of the Hamiltonian is carried out using density function theory (DFT). However, for large, dynamic systems this becomes too computationally expensive and a different technique should be used. In this work I will rely on an Analytical Overlap Method (AOM) \cite{gajdos_ultrafast_2014} to calculate the off-diagonal elements of the Hamiltonian. The diagonal elements will be calculated via a forcefield.
\section{AOM}
\begin{wrapfigure}{r}{0.45\textwidth}
  \includegraphics[width=0.45\textwidth]{./img/SOMO.png}
  \caption{\label{fig:SOMO}A depiction of a singly occupied molecular orbital (SOMO) on an ethylene molecule. Adapted from \cite{spencer_fob-sh:_2016}.}
\end{wrapfigure}
AOM assumes that the off-diagonal elements of the Hamiltonian are proportional to the off-diagonal elements of the overlap matrix between 2 singly occupied molecular orbitals (SOMO) see figure \ref{fig:SOMO}. This is show in equation \eqref{eq:H-CS}.
\begin{equation}
    H_{kl} = C S_{kl}
    \label{eq:H-CS}
\end{equation}
This approximation was originally by Longuet-Higgins and Roberts \cite{Prezhdo1997Oct} and its validity has been tested in our group previously in the small overlap regime \cite{gajdos_ultrafast_2014}.  Many of the presently most studied and promising organic semiconductors, such as Rubrene \cite{Braga2009Feb,Baris2014Aug} and Pentacene \cite{Tiwari2010May, Kim2015Dec} have pi-conjugation. In these systems it is often sufficient to include only 1 optimised p-orbital per atom
\cite{spencer_fob-sh:_2016}. This gives us an equation for the overlap of the SOMOs ($\varphi_{k}$) as:
\begin{equation}
    S_{kl}^{'} = \langle \varphi_{k} | \varphi_{l} \rangle = \sum_{i \in k}^{atoms} \sum_{j \in l}^{atoms} c_{p\pi, i}^{*} c_{p\pi, j} \langle p_{\pi, i}|p_{\pi, j}\rangle
    \label{eq:P-orbital-overlap}
\end{equation}
Where $\langle p_{\pi, i} |$ represents the optimised Slater p-orbital and the $c_{p\pi, i}$ terms are the AOM expansion coefficients. The gradient of the Hamiltonian can also be expressed in terms of the gradient of the overlap $\nabla_{\nu} H_{kl} = C \nabla_{\nu} S_{kl}$ (where $\nu$ represents the atom index). Further efficiencies can be made by expressing the gradient of the overlap in terms of (diabatic) NACVs e.g:
\begin{equation}
  \nabla_{\nu} H_{kl}^{'} = C \ \nabla_{\nu} S_{kl}^{'} = C \ \nabla_{\nu} \langle \varphi_{k} | \varphi_{l} \rangle = C ( \textbf{d}_{\nu, lk}^{'*} + \textbf{d}_{\nu, kl}^{'} )
\end{equation}
Where we have defined the non-adiabatic coupling vectors (in a non-orthogonal diabtic basis) as $\textbf{d}^{'}_{\nu, lk} = \langle \varphi_{l}  | \nabla_{\nu} \varphi_{k}\rangle$. The $H^{'}_{kl}$ term is the Hamiltonian in the non-orthogonal diabatic basis. The NACVs are calculated via a finite difference method.

\section{Different Bases}
The definitions in the previous section giving the key points in the AOM method were all expressed in a non-orthogonal diabatic basis. However, the electronic propagation in the FOB method is done in an orthogonal diabatic basis denoted $\phi_{l}$. Further the CTMQC equations have been derived and are expressed in the adiabtic basis, $\psi_{l}$. Therefore, it is important to define the transformations between the 3 bases used in the code.
\subsubsection{Basis Expansion}
The time-dependent electronic wavefunction can be expressed as a linear combination of basis functions with expansion coefficients determining the size of their contribution. For example:
\begin{equation}
  \Phi_{\textbf{R}}(\textbf{r}, t) = \sum_{n}^{N_{states}}C_{n}(\textbf{R}, t) \psi_{\textbf{R}, n}(\textbf{r}) = \sum_{l}^{N_{states}} u_{l}(\textbf{R}, t) \phi_{\textbf{R}, l}(\textbf{r})
\end{equation}
Where $\Phi_{\textbf{R}}(\textbf{r}, t)$ is the electronic wavefunction in the exact factorisation formalism. The subscript $\textbf{R}$ denotes a parametric dependence on nuclear coordinates, the subscript $n$ denote adiabatic state n similarly $l$ denotes the l$^{th}$ (orthogonal) diabatic state. Throughout this document I will use the following naming convention: $\textbf{R}$ denotes nuclear coordinates and $\textbf{r}$ for the electronic coordinates. The $C_{l}$ is the l$^{th}$ adiabatic state. The squared modulus of this gives the population of electrons on that state. Expressing the electronic wavefunction in this way allows us to rewrite the propagation equations in terms of the expansion coefficients $C_{l}$ or $u_{l}$.
\\\\
As the propagation equations are written in terms of the expansion coefficients it is sensible to give the transformation equation in terms of them. To transform from the adiabatic to diabatic basis we can use the following unitary transformation:
\begin{equation}
  \vec{C}(\textbf{R}, t) = \mathbb{U}(\textbf{R}, t) \  \vec{u}(\textbf{R}, t)
  \label{eq:ad_to_di}
\end{equation}
Here we have expressed the expansion coefficients as a vector e.g. $\vec{C} = \left( \begin{array}{c}
C_1 \\
\vdots \\
C_n
\end{array}\right)$
The unitary transformation matrix is given as the overlap between diabatic and adiabatic states, $U_{ln} = \langle \phi_{l} | \psi_{n} \rangle$, and is a square matrix of size $N_{states}^2$. This is obtained by diagonalising the Hamiltonian in the code. The fact that this matrix is unitary means transforming from the diabatic to adiabatic is no more computationally expensive. The transformation can be obtained by pre-multiplying both sides of eq \eqref{eq:ad_to_di} by $\mathbb{U}^{\dagger}$ e.g.
\begin{equation}
  \mathbb{U}^{\dagger}(\textbf{R}, t) \vec{C}(\textbf{R}, t) = \vec{u}(\textbf{R}, t)
  \label{eq:diab_to_adiab}
\end{equation}
\\\\
To transform from the non-orthogonal to orthogonal basis we need a new matrix. This is given by:
\begin{equation}
  \vec{\phi} = \mathbb{T} \vec{\varphi}
  \label{eq:orth_to_nonorth}
\end{equation}
This transformation is needed when transforming the non-orthogonal diabatic Hamiltonian, $H^{'}$, and non-adiabatic coupling terms to the orthogonal diabatic basis. For example, the orthogonal diabatic Hamiltonian is calculated as $H_{kl} = \left[\mathbb{T}^{-1} \mathbb{H}^{'} \mathbb{T} \right]_{kl}$. This matrix is equal to the inverse square of the overlap between SOMOs i.e. $T_{ml} = [\mathbb{S}^{-\frac{1}{2}}]_{ml}$.
\\\\
Replacing the full charge transfer determinant with parametrised singly occupied molecular orbitals (SOMOs) makes the calculation of the relevant terms in the propagation of the system very efficient. The approximation has been scrutinised before in the group
\cite{giannini_crossover_2018, carof_detailed_2017, gajdos_ultrafast_2014, Gajdos2013Mar, spencer_confronting_2016, spencer_fob-sh:_2016} and its first implementation within the surface hopping framework has been shown to reproduce known phenomena such as the crossover from band-like to hopping-like transport \cite{giannini_crossover_2018}. In this way the interaction of the nuclei on the electrons is accounted for, as the overlap between SOMOs (and therefore the coupling, $H_{ab}$) depends on the nuclear geometry.
\\\\
Thankfully, previous members of the group (see Spencer et al \cite{spencer_fob-sh:_2016}) have implemented the FOB method within the surface hopping framework in CP2K -an ab-initio molecular dynamics package. I will therefore be able to reuse/adapt a large number of the subroutines in my implementation of the FOB-CTMQC method within CP2K.


\documentclass{article}
\author{Scott Cotton\\
IRI France SAS\\
wsc@iri-labs.com}
\title{Windowing The Hilbert Transform}
\begin{document}
\maketitle
\abstract{
The Hilbert transform provides a method for rotating a signal
$90^{\circ}$ in all frequencies.  This transform is useful for
envelope tracking and decomposition of a signal into a single
amplitude and frequency modulated wave.  Hilbert transforms are
simple to implement via the Fourier transform, with one exception:
the edge effects.

In this document, we present a simple windowing method for constructing
a Hilbert transformed signal smoothly from a series of fast Fourier transforms
FFTs.  Traditional windowing methods modulate the edge effects, spreading them
out (albeit smoothly).  The windowing method presented here gives better results than
FIR filtering and traditional windowing, and moreover provides a solid basis
for signal representation by instantaneous frequency/amplitude modulation, as
phase differences and amplitude envelope are by construction coherent accross block
boundaries.
}

\section{Introduction}
The Hilbert transform is a Fourier based transform which can be seen as a
filter of "negative" frequencies or, alternatively, a method of producing
an all-pass, phase altered signal in which each frequency in the frequency
domain is rotated by $\frac{\pi}{2}$, a quarter cycle.  Such a rotation
puts a signal in {\em quadrature}, making it easy to represent in terms of
amplitude and phase.  Hilbert transforms are usually applied in narrow band
contexts because of analytic difficulties of understanding the meaning of
instaneous frequency in a wide band context.  However, Hilbert transforms also
provide a nice way of defining the envelope of any signal, be it narrow or
wide band.

In wide band contexts, the problem of applying a Hilbert transform is made
particularly difficult because of edge effects.  A narrow band Hilbert transform
can be much more readily constructed as either a FIR or IIR filter.

\section{Eye based Windowing}
The idea presented here is remarkably simple and effective.  We forego any
idea of windowing functions, such as the Hamming or Blackman windows, and
instead we simply implement a Hilbert transform by
\begin{enumerate}
	\item Using a sufficiently large FFT, let's call this $L$.
	\item Using a hop size $H$ small enough ($H < L$)so that centering
		$H$ in $L$ removes edge effects.
	\item If necessary, padding the input signal so that advancing it
	by $H$ samples at a time will center $H$ samples in $L$.
\end{enumerate}

What is perhaps unusual here, is that we do not need, nor do we {\em want}
to restrict the FFT to the data being analyzed.  Instead, in the context
of using a Hilbert transform, we desire continuity of the amplitude envelope
and phase differences, which an only really be achieved for a segment of a
signal with both preceding and succeeding context.

\section{}
\end{document}


\section{Math formulas}

\section{Inserting math formulas in a HTML page}
\begin{verbatim}
<!DOCTYPE html>
<html>
<head>
<title>MathJax TeX Test Page</title>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});
</script>
<script type="text/javascript"
  src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
</head>
<body>
When $a \ne 0$, there are two solutions to \(ax^2 + bx + c = 0\) and they are
$$x = {-b \pm \sqrt{b^2-4ac} \over 2a}.$$
</body>
</html>
\end{verbatim}


\chapter{Plan for the Work}

This work aspires to systematically organize the programming of
machine learning.  The work is meant to be open and provide tools so
that a variety of authors may contribute to the project.  The
inspiration for this work is Euclid's Elements, The Elements of
Programming, and Homotopy Type Theory.  The former served as the basis
for the development of axiomatic approaches to mathematics.  An
important observation is that the propositions of Euclid were
accompanied with constructive proofs that may be performed by the
reader with a physical compass and straight-edge.  The mathematics is
a model for real objects and operations that may be performed on them.  Another way to think about it
is that the proofs themselves could be compiled into programs of geometric construction.

Such a relationship to the physical has been broken for nearly a
century due to the set-focused approach to mathematical foundations
adopted in the early twentieth century.  With such an approach
mathematical propositions are compiled into purely abstract
entities.  The constructive work of Euclid became quite divorced
from the foundations common to the working mathematician.

This has tremendous consequences for programming.  Dijkstra observed that it is a mathematical discipline
even if the mathematics differ considerably from those of interest to the mathematician.  Stepanov and McJones
observe that programming is about bits and operations on them that can be realized in physical machines.  Indeed, that is the
basis for programming introduced in Donald Knuths ``Art of Computer Programming''.

Programs are data, and data are programs was an essential insight of Turing.  The insight of Homotopy Type Theory
is that programs are proofs (indeed this is the essence of the Curry-Howard Correspondence)
and types are propositions.

Most work in this area is of interest to logicians and people working with functional programming languages.
We argue for a return to bit-focused programs and programs that do not make a great deal of use of recursion so that they
are simple programs that are easy to reason about and also easy to implement on a real machine in real hardware.  So, we are
returning back to the Euclidean notion of proofs as programs: i.e. concrete procedures that may be applied to concrete entities.

  combined the concrete and abstract notions of
geometry together into a long-running work presented with a
magnificent style for a reader to enjoy.  The work gives a
correspondence between straight-edge compass constructions and
abstract geometry.  Insight into straight-edge compass constructions
is derived through the use of these correspondences.  The present work
is an attempt to perform an analogous work between bits and
mathematical propositions--Leibniz's dream.  This task has been
considered by many previous works and a notable instance is the
Curry-Howard Correspondence.  That is concerned, however, with
functions as the primitive objects belonging to mathematical type
theory.

This work is concerned with programming defined as manipulating bits
as is given to us from Leibniz.  The direct ancestor of this work is
the {\it Elements of Programming} \cite{stepanov2009elements} which
develops ideas of generic programming in several contexts.  We wish to
continue that, but with more explicit recognition of Curry-Howard
correspondence by writing code throughout the text which will serve as
proof mechanisms.  We will not shy away from purely abstract proofs
either, but our principal concern is about {\it objects} which is a
representation of a {\it value} in memory, i.e. a {\it typed datum}.
The object {\it type} is a way of storing and modifying values in
memory.

The Curry-Howard correspondence, however, is most often used to
understand proofs as functions so that functions are the object of
interest.  We focus on bits and the proof correspondence to bit
manipulations thus is more closely tied to Niklaus Wirth with {\it
  Systematic Programming}.  The innovation beyond Wirth's and
Dijkstra's {\it structured programming } is that we will construct
programs that are, in fact, proofs (although they will need extensive
comments to verify this).  Dijkstra says to construct a proof
simulataneous with the program but we more closely tie the two
operations specifically with our bit-based focus.  The idea is not new
and is purely a transplant of the literate program.  Indeed, we take
Knuth's idea of organizing a central repository of organized knowledge
and then apply the literate programming idea to represent that
knowledge as a computer program, and we prove things with it.

One final addition to this book is to specifically focus on
statistical problems.  The author does not recognize any meaningful
difference between reasoning in statistics or machine learning so
terms from both disciplines will be used interchangeably.  The
inclusion of statistics is particularly apt since there have been few
attempts to formalize notions of relating the concrete with the
abstract in statistical settings with the exception of an insightful
paper of Mcullagh. Statistics and Machine Learning are also rarely
formalized and this book takes a constructive view so that the discussion
of those topics is tightly interwoven to computer programs that can be run. The
programs essential to them are those from optimization, statistical tests,
 and feature construction.  These are the programs that characterize work in
statistics and machine learning.

We also harkon back to older work in statistics that tied the subject more closely to economics.
Indeed we will find that decision theory and statistics may be written together
and we can connect these with standard ideas in machine learning.  The mathematics we use
are going to be derived from the type theoretic perspective.  Our interest in these types is so that we can
relate formal mathematics with actual bit representations.  Thus, we are performaing an instance of
synthetic mathematics in the spirit of Euclid.

The purpose of this approach is to produce generic machine
learning/statistical programs.  Develop the underlying algorithms that
make these systems operate and define constructions between them.  We
want a reusable system.  The main purpose of genericity is because
generic code can be reasoned about analytically more easily.
Genericity puts many synthetic notions into a unifed representation
that can then be formally reasoned about using analytic methods.  This
enables us to prove things about the code for the human code reviewer
to understand.  Furthermore, this means that it should be easier to write
new machine learning programs applied to new datasets and new problems
after working through this book and implementing such systems.

The first topics we consider are related to the author's areas of
research and work: speaker recognition, bayesian fusion of systems,
and deep neural networks.  To get to these topics, however we must
excavate and systematically organize programs that prove things about
data and get the reasoning just right.


\subsection{Weighted least squares}


\usepackage{beamerarticle}

\chapter{Approximation de problèmes mixtes}
\label{cha:appr-de-probl-1}

\section{Model Problems}
\label{sec:model-problems}

  We consider now model problems as systems of PDEs where several functions are
  unknowns and which don't play the same roles \emph{mathematically} and
  \emph{physically}.
  \begin{problem}{Stokes}
    \begin{equation}
      \label{eq:chmixte:98}
      \left\{\begin{array}[c]{rl}
          -\Delta u + \nabla p & = f\ \mbox{ in } \Omega\\
          \nabla \cdot u & = 0\ \mbox{ in } \Omega
        \end{array}\right.
    \end{equation}
    where $u: \Omega \mapsto \R{d}$ is a velocity and $p: \Omega \mapsto \R{}$
    is a pressure.
  \end{problem}
  \begin{problem}{Darcy}
    \begin{equation}
      \label{eq:chmixte:99}
      \left\{\begin{array}[c]{rl}
          \sigma + \nabla u & = f\ \mbox{ in } \Omega\\
          \nabla \cdot \sigma & = g\ \mbox{ in } \Omega
        \end{array}\right.
    \end{equation}
    where $\sigma: \Omega \mapsto \R{d}$ is a velocity and $u: \Omega \mapsto \R{}$
    is a hydraulic charge(pressure).

  \end{problem}

\section{Applications}

  We shall focus on Stokes, but the abstract setting of the next section is
  the same for Stokes and Darcy.

\begin{block}{Stokes and incompressible Navier-Stokes for Newtonian fluids}
  The Stokes model is the basis for fluid mechanics models and is a
  simplication of the Navier-Stokes equations where the viscous effects/terms
  are much bigger than the convective ones
  \begin{equation}
    \label{eq:chmixte:3}
    \left\{\begin{array}[c]{rl}
           \rho( \frac{\partial u}{\partial t} + u \cdot \nabla u) - \nu \Delta u + \nabla p & = f\ \mbox{ in } \Omega\\
           \nabla \cdot u & = 0\ \mbox{ in } \Omega
         \end{array}\right.
     \end{equation}
     The first equation results from the conservation of momentum and the second
     from the conservation of mass.
   \end{block}

  \begin{remark}{}
    The well-posedness of these problems results from a so-called \alert{inf-suf
    condition} which is not automatically transfered at the \emph{discrete level}.
  \end{remark}

  \begin{remark}{In practice}
    In order to ensure that the finite element approximation is well-posed, we
    will need to choose approximation spaces that satisfy a \emph{compatibility
    condition} that ensures that a \emph{discrete inf-sup condition} is satisfied.
  \end{remark}

\section{Saddle point problems}
\label{sec:saddle-point-probl}

\subsection{Abstract Continuous Setting}
  Denote
  \begin{itemize}
  \item $X$ and $M$ two Hilbert spaces\footnote{An euclidian space which is
  complete for the norm induced by the scalar product}
  \item two linear forms $f \in X'=\mathcal{L}(X, \R{})$ and $g \in
    M'=\mathcal{L}(M, \R{})$
  \item $a \in \mathcal{L}(X\times X, \R{})$ and $b \in \mathcal{L}(X\times M,
    \R{})$ two bilinear forms
  \end{itemize}
  We are interested in the following abstract problem:
  \begin{problem}[Abstract formulation]
    \label{prob:chmixte:1}
    Look for $(u,p) \in X \times M$ such that
    \begin{equation}
      \label{eq:chmixte:4}
      \left\{
        \begin{array}[c]{rl}
          a(u,v) + b(v,p) & = f(v), \quad \forall v \in X\\
          b(u,q) & = g(q), \quad \forall q \in M
        \end{array}
      \right.
    \end{equation}
  \end{problem}

\subsection{Definition of a saddle point problem}
  \begin{definition}[Saddle point problem]
    \label{def:chmixte:1}
    If the bilinear form $a$ is \emph{symmetric} and positive on $X\times X$, we
    say that problem~\ref{prob:chmixte:1} is a \emph{saddle point problem}.
  \end{definition}

  \begin{remark}{Structure of the problem}
    \begin{itemize}
    \item the space of solution is the same of the test space
    \item the unknown $p$ does not appear in the second equation
    \item the unknown functions $u$ and $p$ are coupled via the same bilinear
      form $b$ is the first and second equation.
    \end{itemize}
    The next question is : \alert{is the problem~\ref{prob:chmixte:1} well-posed?}
  \end{remark}

\section{Well posed problem}
\label{sec:well-posed-problemframe}

\subsection{Reformulation}
  Let's rewrite Problem~\ref{prob:chmixte:1}. Denote $V=X\times M$ and introduce $c
  \in \mathcal{L}(V\times V, \R{})$ such that
  \begin{equation}
    \label{eq:chmixte:5}
    c((u,p),(v,q)) = a(u,v)+b(v,p)+b(u,q)
  \end{equation}
  and $h\in \mathcal{L}(V,\R{})$ such that
  \begin{equation}
    \label{eq:chmixte:6}
    h(v,q) = f(v)+g(q)
  \end{equation}
  then problem~\ref{prob:chmixte:1} reads
  \begin{problem}
    \label{prob:chmixte:2}
    Look for $(u,p) \in V$ such that
    \begin{equation}
      \label{eq:chmixte:7}
        \begin{array}[c]{rl}
          c((u,p), (v,q)) & = h(v,q), \quad \forall (v,q) \in V
        \end{array}
      \end{equation}
  \end{problem}

  \begin{theorem}[Well posedness of the Saddle-Point problem]
    \label{thr:chmixte:1}
    We suppose that $a$ is coercive over $X$, the problem~\ref{prob:chmixte:2} is
    \emph{well-posed} if and only if the bilinear form $b$ satisfies the
    following inf-sup condition: there exists $\beta > 0$ such that
    \begin{equation}
      \label{eq:chmixte:8}
      \inf_{q \in M} \sup_{v \in X} \frac{b(v,q)}{||v||_X ||q||_M} \geq \beta
    \end{equation}
  \end{theorem}
  \begin{remark}
    \begin{itemize}
    \item Lax-Milgram provides only a sufficient condition for well-posedness
    \item the form $c$ in Problem~\ref{prob:chmixte:2} does not satisfy Lax-Milgram.
    \end{itemize}
  \end{remark}

  Let's introduce the so-called \emph{Lagrangian} $l \in \mathcal{L}(X\times M,
  \R{})$ defined by
  \begin{equation}
    \label{eq:chmixte:9}
    l(v,q) =  \frac{1}{2} a(v,v) + b(v,q) - f(v) - g(q)
  \end{equation}
  \begin{definition}[Saddle point]
    \label{def:chmixte:2}
    We say that the point $(u,p)\in X\times M$ is a saddle point of $l$ if
    \begin{equation}
      \label{eq:chmixte:10}
      \forall (v,q) \in X\times M, \quad l(u,q) \leq l(u,p) \leq l(v,p)
    \end{equation}
  \end{definition}
  \begin{proposition}[Unicity of the saddle point]
    \label{prop:chmixte:1}
    Under the hypothesys of theorem~\ref{thr:chmixte:1}, the Lagrangian $l$ defined
    by~\eqref{eq:chmixte:9} has a \emph{unique} saddle point. Moreover this saddle point
    is the the unique solution of problem~\ref{prob:chmixte:1}.
  \end{proposition}

\section{Finite element approximation}
\label{sec:finite-elem-appr}

\subsection{Abstract Discrete Problem}
  We now turn to the approximation of the problem~\ref{prob:chmixte:1} by a \emph{standard
  Galerkin method} in a \emph{conforming} way.

  Denote the two spaces $X_h \subset X$ and $M_h \subset M$, we consider the
  following problem:
  \begin{problem}
    \label{prob:chmixte:3}
    Look for $(u_h,p_h) \in X_h \times M_h$ such that
    \begin{equation}
      \label{eq:chmixte:11}
      \left\{
        \begin{array}[c]{rl}
          a(u_h,v_h) + b(v_h,p_h) & = f(v_h), \quad \forall v_h \in X_h\\
          b(u_h,q_h) & = g(q_h), \quad \forall q_h \in M_h
        \end{array}
      \right.
    \end{equation}
  \end{problem}

  \begin{theorem}[Well-posedness of the discrete problem]
    \label{thr:chmixte:2}
    We suppose that $a$ is coercive over $X$ and that $X_h \subset X$ and $M_h
    \subset M$. Then the problem~\ref{prob:chmixte:3} is \emph{well-posed} if and only
    if the following \emph{discrete inf-sup condition}  is satisfied: there
    exists $\beta_h  > 0$ such that
    \begin{equation}
      \label{eq:chmixte:12}
      \inf_{q_h \in M_h} \sup_{v_h \in X_h} \frac{b(v_h,q_h)}{||v_h||_{X_h} ||q_h||_{M_h}} \geq \beta_h
    \end{equation}
  \end{theorem}

  \begin{remark}{Compatibility condition}
    Problem~\ref{prob:chmixte:3} to be well posed, requires that the spaces $X_h$ and
    $M_h$ satisfy the condition~\eqref{eq:chmixte:12}. This is known as the
    Babuska-Brezzi (BB) or Ladyhenskaya-Babuska-Brezzi (LBB).
  \end{remark}

  Regarding error analysis, we have the following lemma
  \begin{lemma}
    \label{lem:1}
      Thanks to the Lemma of Céa applied to Saddle-Point Problems, the unique
      solution $(u,p)$ of problem~\ref{prob:chmixte:3}  satisfy
      \begin{equation}
        \label{eq:chmixte:13}
        \begin{array}[c]{rl}
          ||u-u_h||_X & \leq c_{1h} \inf_{v_h \in X_h}  ||u-v_h||_X + c_{2}
          \inf_{q_h \in M_h}  ||q-q_h||_M\\
          ||p-p_h||_X & \leq c_{3h} \inf_{v_h \in X_h}  ||u-v_h||_X + c_{4h} \inf_{q_h \in M_h}  ||q-q_h||_M
        \end{array}
      \end{equation}
      where
      \begin{itemize}
      \item $c_{1h} =
        (1+\frac{||a||_{X,X}}{\alpha})(1+\frac{||b||_{X,M}}{\beta_h})$ where
        $\alpha$ is the coercivity constant of $a$ over X.
      \item $c_{2} = \frac{||b||_{X,M}}{\alpha}$
      \item $c_{3h} = c_{1h} \frac{||a||_{X,X}}{\beta_h}$, $c_{4h} = 1+ \frac{||b||_{X,M}}{\beta_h}+\frac{||a||_{X,X}}{\beta_h}$
      \end{itemize}
  \end{lemma}
  \begin{remark}
    The constants $c_{1h}, c_{3h}, c_{4h}$ are as large as $\beta_h$ is small.
  \end{remark}

\subsection{Linear system associated}
\label{sec:linear-system}

The discretisation process leads to a linear system. We denote
  \begin{itemize}
  \item $N_u = \dim {X_h}$
  \item $N_p = \dim {M_h}$
  \item $\{\phi_i\}_{i=1,...,N_u}$ a basis of $X_h$
  \item $\{\psi_k\}_{k=1,...,N_p}$ a basis of $M_h$
  \item for all $u_h = \sum_{i=1}^{N_u} u_i \phi_i$, we associate $U \in
    \R{N_u}$, $U=(u_1,\ldots,u_{N_u})^T$, the component vector of $u_h$ is
    $\{\phi_i\}_{i=1,\ldots,N_u}$
  \item for all $p_h = \sum_{k=1}^{N_p} u_k \psi_k$, we associate $P \in
    \R{N_p}$, $P=(p_1,\ldots,p_{N_p})^T$, the component vector of $p_h$ is $\{\psi_k\}_{k=1,\ldots,N_p}$
  \end{itemize}

  \begin{block}{}
    The matricial form of problem ~\ref{prob:chmixte:3} reads
    \begin{equation}
      \label{eq:chmixte:15}
      \begin{bmatrix}
        \mathcal{A} & \mathcal{B}^T\\
        \mathcal{B} & 0
      \end{bmatrix}
      \begin{bmatrix}
        U \\
        P
      \end{bmatrix}
      =
      \begin{bmatrix}
        F\\
        G
      \end{bmatrix}
    \end{equation}
    where the matrix $\mathcal{A} \in \R{N_u,N_u}$ and $\mathcal{B} \in
    \R{N_p,N_u}$ have the coefficients
    \begin{equation}
      \label{eq:chmixte:16}
      \mathcal{A}_{ij} = a(\phi_j,\phi_i), \quad \mathcal{B}_{ki} = b(\phi_i,\psi_k)
    \end{equation}
    and the vectors $\mathcal{F} \in \R{N_u}$ and $\mathcal{G} \in \R{N_p}$
    have the coefficients
    \begin{itemize}
    \item $F_i=f(\phi_i)$
    \item $G_k=g(\psi_k)$
    \end{itemize}
  \end{block}

  \begin{remark}[Remarks on the linear system]
    \label{rem:2}
    \begin{enumerate}
    \item Since $a$ is symmetric and coercive, $\mathcal{A}$ is \emph{symmetric
       positive definite}
     \item The matrix of the system \eqref{eq:chmixte:15} is symmetric but not positive
     \item The inf-sup condition~\eqref{eq:chmixte:12} is equivalent to the fact that
       $\mathcal{B}$ is of \emph{maximum rank}, \emph{i.e.} $\ker(\mathcal{B}^T)
       = \{0 \}$.
     \item From theorem~\ref{thr:chmixte:2}, the matrix of the system~\eqref{eq:chmixte:15} is
       invertible
    \end{enumerate}
  \end{remark}

  \begin{block}{When the Inf-sup condition is not satisfied}
    The counter examples when the inf-sup condition~\eqref{eq:chmixte:12} is not
    satisfied(e.g. $\mathcal{B}$ is not maximum rank ) occur usually in two
    cases:
    \begin{enumerate}
    \item $\dim {M_h} > \dim {X_h}$: the space of pressure is too large for the
      matrix $\mathcal{B}$ to be maximum rank. In that case $\mathcal{B}$ is
      injective ($\ker(\mathcal{B}) = \{0\})$. We call this \emph{locking}.
    \item there exists a vector $Q^* \neq 0$ in $\ker(\mathcal{B}^T)$. The
      discrete field $q^*_h$ in $M_h$, $q^*_h=\sum_{k=1}^{N_p} Q^*_k \psi_k$,
      associated is called a \emph{spurious mode}. $q^*_H$ is such that
      \begin{equation}
        \label{eq:chmixte:14}
        b(v_h,q^*_h)=0.
      \end{equation}

    \end{enumerate}
  \end{block}

We now introduce the Uzawa matrix.
\begin{block}{Uzawa}
  The matrix
    \begin{equation}
      \label{eq:chmixte:17}
      \mathcal{U} = \mathcal{B} \mathcal{A}^{-1} \mathcal{B}^T
    \end{equation}
    is called the \emph{Uzawa matrix}. It is \emph{symmetric positive definite}
    from the properties of $\mathcal{A}$, $\mathcal{B}$
  \end{block}
  \begin{block}{Applications}
    The Uzawa matrix occurs when eliminating the velocity in
    system~\eqref{eq:chmixte:15} and get a linear system on $P$:
    \begin{equation}
      \label{eq:chmixte:18}
      \mathcal{U} P = \mathcal{B} \mathcal{A}^{-1} F - G
    \end{equation}
    then one application is to solve \eqref{eq:chmixte:15} by solving \eqref{eq:chmixte:18}
    iteratively and compute the velocity afterwards.
  \end{block}

\section{Mixed finite element for Stokes}
\label{sec:mixed-finite-element}

\subsection{Variational formulation }
\label{sec:vari-form-}

We start with the Well-posedness at the continuous level
  \begin{itemize}
  \item   We consider the model problem~\eqref{eq:chmixte:1} with homogeneous Dirichlet
  condition on velocity $u = 0$ on $\partial \Omega$
  \item   We suppose the $f \in [L^2(\Omega)]^d$ and $g \in L^2(\Omega)$ with
  \begin{equation}
    \label{eq:chmixte:20}
    \int_\Omega g = 0
  \end{equation}
    Introduce
  \begin{equation}
    \label{eq:chmixte:19}
    L^2_0(\Omega) = \Big\{ q \in L^2(\Omega): \int_\Omega q = 0 \Big\}
  \end{equation}

  \end{itemize}

  \begin{remark}{The condition~\eqref{eq:chmixte:20}}
    The condition~\eqref{eq:chmixte:20} comes from the divergence theorem applied to the
    divergence equation and the fact that $u=0$ on the boundary
    \begin{equation}
      \label{eq:chmixte:21}
      \int_\Omega g = \int_\Omega \nabla \cdot u = \int_{\partial \Omega} u
      \cdot n = 0
    \end{equation}
    This is a \emph{necessary} condition for the existence of a solution $(u,p)$
    for the Stokes equations with these boundary conditions.
  \end{remark}

We turn now to the variational formulation.  The Stokes problem reads
    \begin{problem}[Stokes Variational Formulation]
      \label{prob:chmixte:4}
      Look for $(u,p) \in [H^1_0(\Omega)]^d \times L^2_0(\Omega)$ such that
      \begin{equation}
        \label{eq:chmixte:25}
      \left\{
        \begin{array}[c]{rl}
          \int_\Omega \nabla u : \nabla v -\int_\Omega p \nabla \cdot v  & =
          \int_\Omega f \cdot v, \quad \forall v \in [H^1_0(\Omega)]^d\\
          \int_\Omega q \nabla \cdot u & = - \int_\Omega g q, \quad \forall q \in L^2_0(\Omega)
        \end{array}
      \right.
    \end{equation}
    \end{problem}
    We retrieve the problem~\ref{prob:chmixte:1} with $X=[H^1_0(\Omega)]^d$ and
    $M=L^2_0(\Omega)$ and
    \begin{equation}
      \label{eq:chmixte:22}
      \begin{array}[c]{rlrl}
      a(u,v) &= \int_\Omega \nabla u : \nabla v,& \quad b(v,p) &= -\int_\Omega p
      \nabla \cdot v,\\
      \quad f(v) &=  \int_\Omega f \cdot v,& \quad g(q) &= - \int_\Omega g q
      \end{array}
    \end{equation}
    \begin{remark}{Pressure up to a constant}
      The pressure is known up to a constant, that's why we look for them in
      $L^2_0(\Omega)$ to ensure uniqueness.
    \end{remark}

% \begin{frame}
%   \frametitle{Well-posedness}

%   \begin{lemma}
%     \label{lem:2}
%     Denote $\Omega$ a domain of $\R{d}, d \geq 2$, Then the operator
%     \begin{equation}
%       \label{eq:chmixte:23}
%       \nabla \cdot : [H^1_0(\Omega)]^d \mapsto L^2_0(\Omega)
%     \end{equation}
%     is surjective.
%   \end{lemma}

%   This lemma and the theorem
%

\subsection{Finite element approximation}
\label{sec:finite-elem-appr-1}

  Denote $X_h \subset [H^1_0(\Omega)]^d$ and $M_h \subset L^2_0(\Omega)$

  \begin{problem}[Discrete Stokes formulation]
    \label{prob:chmixte:5}
    Look for $(u_h,p_h) \in X_h \times M_h$ such that
    \begin{equation}
      \label{eq:chmixte:24}
      \left\{
        \begin{array}[c]{rl}
          \int_\Omega \nabla u_h : \nabla v_h + \int_\Omega p_h \nabla \cdot v_h
          & = \int_\Omega f \cdot v_h, \quad \forall v_h \in X_h\\
          \int_\Omega q_h \nabla \cdot u_h & = -\int_\Omega g q_h, \quad \forall q_h \in M_h
        \end{array}
      \right.
    \end{equation}
  \end{problem}
  \begin{remark}[Well-posedness]
    \label{rem:1}
    This problem, thanks to theorem~\ref{thr:chmixte:2} is well-posed if and only if
    $X_h$ and $M_h$ are such that there exists $\beta_h > 0$
    \begin{equation}
      \label{eq:chmixte:26}
      \inf_{q_h \in M_h} \sup_{v_h \in X_h} \frac{\int_\Omega q_h \nabla \cdot v_h}{||v_h||_{X_h} ||q_h||_{M_h}} \geq \beta_h
    \end{equation}
  \end{remark}

% \lstinputlisting{mystokes.cpp}

\subsection{Some counter examples: bad finite element for Stokes}
\label{sec:counter-examples}

In this section we present two classical bad finite element approximations.

\subsubsection{Finite element $\poly{P}_1/\poly{P}_0$: locking}
    thanks to the Euler relations

    \begin{equation}
      \label{eq:chmixte:28}
      \begin{array}[c]{rl}
        N_{\mathrm{cells}} - N_{\mathrm{edges}} + N_{vertices}  &= 1-I\\
      N^\partial_{\mathrm{vertices}} - N^\partial_{\mathrm{edges}} &= 0
      \end{array}
    \end{equation}
    where $I$ is the number of holes in $\Omega$.  We have that $\dim {M_h} =
    N_{\mathrm{cells}}$, $\dim {X_h} = 2 N^i_{\mathrm{vertices}}$ and so
    \begin{equation}
      \label{eq:chmixte:29}
      \dim {M_h} - \dim {X_h} = N_{\mathrm{cells}} - 2 N^i_{\mathrm{vertices}} =
      N^\partial_{\mathrm{edges}} - 2 > 0
    \end{equation}
    so $M_h$ is too rich for the condition \eqref{eq:chmixte:26} and we have
    $\ker(\mathcal{B}) = \{0\}$ such that the \emph{only} discrete $u_h^*$,
    with components $U^*$, satisfying $\mathcal{B} U^*$ is the nul field, $U^*=0$.

\subsubsection{Finite element $\poly{Q}_1/\poly{P}_0$: spurious mode}

    We can construct in that case a function $q_h^*$ on a uniform grid which is
    equal alternatively -1, +1 (chessboard) in the cells of  the mesh, then
    \begin{equation}
      \label{eq:chmixte:27}
      \forall v_h \in [Q^1_{c,h}]^d, \quad \int_\Omega q^*_h \nabla \cdot v_h = 0
    \end{equation}
    and thus the associated $X_h$, $M_h$ do not satisfy  the condition~\eqref{eq:chmixte:26}.

\subsubsection{Finite element $\poly{P}_1/\poly{P}_1$: spurious mode}
    We can construct in that case a function $q_h^*$ on a uniform grid which is
    equal alternatively -1, 0, +1 at the vertices of the mesh, then
    \begin{equation}
      \label{eq:chmixte:27}
      \forall v_h \in [P^1_{c,h}]^d, \quad \int_\Omega q^*_h \nabla \cdot v_h = 0
    \end{equation}
    and thus the associated $X_h$, $M_h$ do not satisfy  the condition~\eqref{eq:chmixte:26}.

\subsection{Mini-Element}
\label{sec:finite-elem-stok}

  The problem with the $\poly{P}_1/\poly{P}_1$ mixed finite element is that the
  velocity is not \emph{rich} enough. A cure is to add a function $v_h^*$ in the
  velocity approximation space to ensure that
  \begin{equation}
    \label{eq:chmixte:30}
    \int_\Omega q^*_h \nabla \cdot v_h^* \neq 0
  \end{equation}
  where $q_h^*$ is the spurious mode. To do that we add the bubble function to
  the $\poly{P}_1$ velocity space.
  \begin{definition}[Bubble function]
    \label{def:chmixte:3}
    Recall the construction of finite elements on a reference convex $\hat{K}$.
    We say that $\hat{b}: \hat{K} \mapsto \R{}$ is a bubble function if:
    \begin{itemize}
    \item $\hat{b} \in H^1_0(\hat{K})
$
    \item $0 \leq \hat{b}(\hat{x}) \leq 1, \quad \forall \hat{x} \in \hat{K}$
    \item $\hat{b}(\hat{C}) = 1, \quad \mbox{where} \hat{C}$ is the barycenter of $\hat{K}$
    \end{itemize}
  \end{definition}

  \begin{exampleblock}{Example}
    The function
    \begin{equation}
      \label{eq:chmixte:31}
      \hat{b} = (d+1)^{d+1} \Pi_{i=0}^d\ \hat{\lambda}_i
    \end{equation}
    where $(\hat{\lambda}_0, \ldots, \hat{\lambda}_d)$ denote the barycentric
    coordinates on $\hat{K}$
  \end{exampleblock}

  Denote $\hat{b}$ a bubble fonction on $\hat{K}$, we set
  \begin{equation}
    \label{eq:chmixte:32}
    \hat{P} = [\poly{P}_1(\hat{K}) \oplus \vect{} (\hat{b})]^d,
  \end{equation}
  and introduce
  \begin{eqnarray}
    \label{eq:chmixte:33}
    X_h &=& \Big\{ v_h \in [C^0(\bar{\Omega})]^d : \forall K \in \mathcal{T}_h, v_h
    \circ T_K \in \hat{P}; v_{h_|{\partial \Omega}} = 0 \Big\}\\
    M_h &=& P^1_{c,h}
  \end{eqnarray}

  \begin{lemma}[Compatibility condition]
    \label{lem:3}
    The spaces $X_h$ and $M_h \cap L^2_0(\Omega)$ satisfy the compatibility
    condition~\eqref{eq:chmixte:26} uniformly in $h$.
  \end{lemma}

  \begin{theorem}[Error estimation for the mini-element]
    \label{thr:chmixte:3}
    Suppose that $(u,p)$, solution of problem~\ref{prob:chmixte:1}, is smooth enough,
    ie. $u \in [H^2(\Omega)]^d \cap [H^1_0(\Omega)]^d$ and $p\in H^1(\Omega)
    \cap L^2_0(\Omega)$. Then there exists a constant $c$ such that for all $h
    >0$
    \begin{equation}
      \label{eq:chmixte:34}
      \| u- u_h \|_{1,\Omega} + \|p-p_h\|_{0,\Omega} \leq c h (\|u\|_{2,\Omega}
      +\|p\|_{1,\Omega})
    \end{equation}
    and if the Stokes problem is stabilizing then
    \begin{equation}
      \label{eq:chmixte:35}
      \|u-u_h\|_{0,\Omega} \leq c h^2 ( \|u\|_{2,\Omega} +\|p\|_{1,\Omega}).
    \end{equation}
  \end{theorem}
  \begin{definition}[Stabilizing Stokes problem]
    \label{def:chmixte:4}
    We say that the Stokes problem is stabilizing if there exists a constant
    $c_S$ such that for all $f \in [L^2(\Omega)]^d$, the unique solution $(u,p)$
    of \eqref{eq:chmixte:25} with $g=0$ is such that:
    \begin{equation}
      \label{eq:chmixte:36}
      \|u\|_{2,\Omega} + \|p\|_{1,\Omega} \leq c_S \|f\|_{0,\Omega}
    \end{equation}
    A sufficient condition for stabilizing Stokes problem is that the $\Omega$
    is a polygonal convex in 2D or of class $C^1$  in $\R{d}, d=2,3$.
  \end{definition}

\subsection{Taylor-Hood Element}
\label{sec:taylor-hood-element}

  The mini-element solved the compatibility condition problem, but the error
  estimation in equation~\eqref{eq:chmixte:35} is not optimal in the sense that
  \begin{enumerate}
  \item the pressure space is sufficiently rich to enable a $h^2$ convergence in
    the pressure error,
  \item but the velocity space is not rich enough to ensure a $h^2$ convergence
    in the velocity error.
  \end{enumerate}

  \begin{block}{Taylor-Hood element}
    The idea of the Taylor-Hood element is to enrich even more the velocity
    space to ensure optimal convergence in $h$. Here we will take $[\poly{P}_2]^d$ for
    the velocity and $\poly{P}_1$ for the pressure.
  \end{block}

  Introduce
  \begin{eqnarray}
    \label{eq:chmixte:39}
    X_h &=&  [P^2_{c,h}]^d\\
    M_h &=& P^1_{c,h}
  \end{eqnarray}

  \begin{lemma}[Compatibility condition]
    \label{lem:3}
    The spaces $X_h$ and $M_h \cap L^2_0(\Omega)$ satisfy the compatibility
    condition~\eqref{eq:chmixte:26} uniformly in $h$.
  \end{lemma}

  \begin{theorem}[Error estimation for the Taylor-Hood element]
    \label{thr:chmixte:3}
    Suppose that $(u,p)$, solution of problem~\ref{prob:chmixte:1}, is smooth enough,
    ie. $u \in [H^3(\Omega)]^d \cap [H^1_0(\Omega)]^d$ and $p\in H^2(\Omega)
    \cap L^2_0(\Omega)$. Then there exists a constant $c$ such that for all $h
    >0$
    \begin{equation}
      \label{eq:chmixte:40}
      \| u- u_h \|_{1,\Omega} + \|p-p_h\|_{0,\Omega} \leq c h^2 (\|u\|_{3,\Omega}
      +\|p\|_{2,\Omega})
    \end{equation}
    and if the Stokes problem is stabilizing then
    \begin{equation}
      \label{eq:chmixte:41}
      \|u-u_h\|_{0,\Omega} \leq c h^3 ( \|u\|_{3,\Omega} +\|p\|_{2,\Omega}).
    \end{equation}
  \end{theorem}

  \begin{block}{Generalized Taylor-Hood element}
    We consider the mixed finite elements $\poly{P}_k/\poly{P}_{k-1}$ and
    $\poly{Q}_k/\poly{Q}_{k-1}$ which allows to approximate the velocity and
    pressure respectively with, on Simplices
    \begin{eqnarray}
        \label{eq:chmixte:42}
        X_h &=&  [P^{k}_{c,h}]^d\\
        M_h &=& P^{k-1}_{c,h}
      \end{eqnarray}
      On Hypercubes
      \begin{eqnarray}
        \label{eq:chmixte:43}
        X_h &=&  [Q^{k}_{c,h}]^d\\
        M_h &=& Q^{k-1}_{c,h}
      \end{eqnarray}

      We then have
      \begin{equation}
      \label{eq:chmixte:40}
      \|u-u_h\|_{0,\Omega} + h ( \| u- u_h \|_{1,\Omega} + \|p-p_h\|_{0,\Omega} ) \leq c h^{k+1} (\|u\|_{k+1,\Omega}
      +\|p\|_{k,\Omega})
    \end{equation}
  \end{block}

\begin{block}{Other Discretization spaces}
  \begin{itemize}
   \item Discrete inf-sup condition: dictates the choice of spaces
   \item Inf-sup stables spaces:
	\begin{itemize}
	  \item $\mathbb Q_k$-$\mathbb Q_{k-2}$, $\mathbb Q_k$-$\mathbb Q^{disc}_{k-2}$
	  \item $\mathbb P_k$-$\mathbb P_{k-1}$, $\mathbb P_k$-$\mathbb P_{k-2}$, $\mathbb P_k$-$\mathbb P^{disc}_{k-2}$
	  \item Discrete inf-sup constant independent of $h$, but dependent on $k$
	\end{itemize}
  \end{itemize}
 \end{block}

\subsection{Numerical validation: Test case}

  We consider the Kovasznay solution of the steady Stokes equations, see Kovasznay \cite{kovasznay_test}. The exact solution is
\begin{equation}\label{kovasznay_problem}
\begin{array}{r c l}
 \displaystyle \mathbf{u}(x,y) & = & \displaystyle \prect{1 - e^{\lambda x } \cos (2 \pi y), \frac{\lambda}{2 \pi} e^{\lambda x } \sin (2 \pi y)}^T \\
 \displaystyle p(x,y) & = & \displaystyle -\frac{e^{2 \lambda x}}{2} \\
 \displaystyle \lambda & = & \displaystyle \frac{1}{2 \nu} - \sqrt{\frac{1}{4\nu^2} + 4\pi^2}.
\end{array}
\end{equation}
The domain is defined as  $\domain = (-0.5,1) \times (-0.5,1.5)$ and $\nu = 0.035$. The forcing term for the momentum equation is obtained from the solution and is
\begin{equation}
 \mathbf{f} = \left( e^{\lambda x}  \left( \left( \lambda^2 - 4\pi^2 \right) \nu \cos (2\pi y) - \lambda e^{\lambda x} \right), e^{\lambda x} \nu \sin (2 \pi y) (-\lambda^2 + 4 \pi^2)           \right)^T
\end{equation}
Dirichlet boundary conditions are derived from the exact solution.

%%% Local Variables:
%%% coding: utf-8
%%% mode: latex
%%% TeX-PDF-mode: t
%%% TeX-parse-self: t
%%% TeX-auto-save: t
%%% x-symbol-8bits: nil
%%% TeX-auto-regexp-list: TeX-auto-full-regexp-list
%%% TeX-master: "mef-intro"
%%% ispell-local-dictionary: "american"
%%% End:


\vfill \eject
\section{{\tt allInOne.c} -- A Serial $LU$ Driver Program}
\label{section:LU-MT-driver}

\begin{verbatim}
/*  allInOneMT.c  */

#include "../spoolesMT.h"
#include "../../misc.h"
#include "../../FrontMtx.h"
#include "../../SymbFac.h"

/*--------------------------------------------------------------------*/
int
main ( int argc, char *argv[] ) {
/*
   ------------------------------------------------------------------
   all-in-one program to solve A X = Y
   using a multithreaded factorization and solve

   (1) read in matrix entries for A and form InpMtx object
   (2) read in right hand side for Y entries and form DenseMtx object
   (3) form Graph object, order matrix and form front tree
   (4) get the permutation, permute the front tree, matrix
       and right hand side and get the symbolic factorization
   (5) initialize the front matrix object to hold the factor matrices
   (6) get the domain-decomposition map from fronts to threads
   (7) compute the numeric factorization
   (8) post-process the factor matrices
   (9) get the map for the parallel solve
   (10) compute the solution
   (11) permute the solution into the original ordering

   created -- 98jun04, cca
   ------------------------------------------------------------------
*/
/*--------------------------------------------------------------------*/
char            *matrixFileName, *rhsFileName ;
DenseMtx        *mtxY, *mtxX ;
Chv             *rootchv ;
ChvManager      *chvmanager ;
double          droptol = 0.0, tau = 100. ;
double          cpus[10] ;
DV              *cumopsDV ;
ETree           *frontETree ;
FrontMtx        *frontmtx ;
FILE            *inputFile, *msgFile ;
Graph           *graph ;
InpMtx          *mtxA ;
int             error, ient, irow, jcol, jrhs, jrow, lookahead, msglvl,
                ncol, nedges, nent, neqns, nfront, nrhs, nrow,
                nthread, pivotingflag, seed, symmetryflag, type ;
int             *newToOld, *oldToNew ;
int             stats[20] ;
IV              *newToOldIV, *oldToNewIV, *ownersIV ;
IVL             *adjIVL, *symbfacIVL ;
SolveMap        *solvemap ;
SubMtxManager   *mtxmanager  ;
/*--------------------------------------------------------------------*/
/*
   --------------------
   get input parameters
   --------------------
*/
if ( argc != 10 ) {
   fprintf(stdout, "\n"
      "\n usage: %s msglvl msgFile type symmetryflag pivotingflag"
      "\n        matrixFileName rhsFileName seed"
      "\n    msglvl -- message level"
      "\n    msgFile -- message file"
      "\n    type    -- type of entries"
      "\n      1 (SPOOLES_REAL)    -- real entries"
      "\n      2 (SPOOLES_COMPLEX) -- complex entries"
      "\n    symmetryflag -- type of matrix"
      "\n      0 (SPOOLES_SYMMETRIC)    -- symmetric entries"
      "\n      1 (SPOOLES_HERMITIAN)    -- Hermitian entries"
      "\n      2 (SPOOLES_NONSYMMETRIC) -- nonsymmetric entries"
      "\n    pivotingflag -- type of pivoting"
      "\n      0 (SPOOLES_NO_PIVOTING) -- no pivoting used"
      "\n      1 (SPOOLES_PIVOTING)    -- pivoting used"
      "\n    matrixFileName -- matrix file name, format"
      "\n       nrow ncol nent"
      "\n       irow jcol entry"
      "\n        ..."
      "\n        note: indices are zero based"
      "\n    rhsFileName -- right hand side file name, format"
      "\n       nrow nrhs "
      "\n       ..."
      "\n       jrow entry(jrow,0) ... entry(jrow,nrhs-1)"
      "\n       ..."
      "\n    seed    -- random number seed, used for ordering"
      "\n    nthread -- number of threads"
      "\n", argv[0]) ;
   return(0) ;
}
msglvl = atoi(argv[1]) ;
if ( strcmp(argv[2], "stdout") == 0 ) {
   msgFile = stdout ;
} else if ( (msgFile = fopen(argv[2], "a")) == NULL ) {
   fprintf(stderr, "\n fatal error in %s"
           "\n unable to open file %s\n",
           argv[0], argv[2]) ;
   return(-1) ;
}
type           = atoi(argv[3]) ;
symmetryflag   = atoi(argv[4]) ;
pivotingflag   = atoi(argv[5]) ;
matrixFileName = argv[6] ;
rhsFileName    = argv[7] ;
seed           = atoi(argv[8]) ;
nthread        = atoi(argv[9]) ;
/*--------------------------------------------------------------------*/
/*
   --------------------------------------------
   STEP 1: read the entries from the input file
           and create the InpMtx object
   --------------------------------------------
*/
inputFile = fopen(matrixFileName, "r") ;
fscanf(inputFile, "%d %d %d", &nrow, &ncol, &nent) ;
neqns = nrow ;
mtxA = InpMtx_new() ;
InpMtx_init(mtxA, INPMTX_BY_ROWS, type, nent, 0) ;
if ( type == SPOOLES_REAL ) {
   double   value ;
   for ( ient = 0 ; ient < nent ; ient++ ) {
      fscanf(inputFile, "%d %d %le", &irow, &jcol, &value) ;
      InpMtx_inputRealEntry(mtxA, irow, jcol, value) ;
   }
} else {
   double   imag, real ;
   for ( ient = 0 ; ient < nent ; ient++ ) {
      fscanf(inputFile, "%d %d %le %le", &irow, &jcol, &real, &imag) ;
      InpMtx_inputComplexEntry(mtxA, irow, jcol, real, imag) ;
   }
}
fclose(inputFile) ;
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n input matrix") ;
   InpMtx_writeForHumanEye(mtxA, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -----------------------------------------
   STEP 2: read the right hand side matrix Y
   -----------------------------------------
*/
inputFile = fopen(rhsFileName, "r") ;
fscanf(inputFile, "%d %d", &nrow, &nrhs) ;
mtxY = DenseMtx_new() ;
DenseMtx_init(mtxY, type, 0, 0, neqns, nrhs, 1, neqns) ;
DenseMtx_zero(mtxY) ;
if ( type == SPOOLES_REAL ) {
   double   value ;
   for ( irow = 0 ; irow < nrow ; irow++ ) {
      fscanf(inputFile, "%d", &jrow) ;
      for ( jrhs = 0 ; jrhs < nrhs ; jrhs++ ) {
         fscanf(inputFile, "%le", &value) ;
         DenseMtx_setRealEntry(mtxY, jrow, jrhs, value) ;
      }
   }
} else {
   double   imag, real ;
   for ( irow = 0 ; irow < nrow ; irow++ ) {
      fscanf(inputFile, "%d", &jrow) ;
      for ( jrhs = 0 ; jrhs < nrhs ; jrhs++ ) {
         fscanf(inputFile, "%le %le", &real, &imag) ;
         DenseMtx_setComplexEntry(mtxY, jrow, jrhs, real, imag) ;
      }
   }
}
fclose(inputFile) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n rhs matrix in original ordering") ;
   DenseMtx_writeForHumanEye(mtxY, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -------------------------------------------------
   STEP 3 : find a low-fill ordering
   (1) create the Graph object
   (2) order the graph using multiple minimum degree
   -------------------------------------------------
*/
graph = Graph_new() ;
adjIVL = InpMtx_fullAdjacency(mtxA) ;
nedges = IVL_tsize(adjIVL) ;
Graph_init2(graph, 0, neqns, 0, nedges, neqns, nedges, adjIVL,
            NULL, NULL) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n graph of the input matrix") ;
   Graph_writeForHumanEye(graph, msgFile) ;
   fflush(msgFile) ;
}
frontETree = orderViaMMD(graph, seed, msglvl, msgFile) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n front tree from ordering") ;
   ETree_writeForHumanEye(frontETree, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ----------------------------------------------------
   STEP 4: get the permutation, permute the front tree,
           permute the matrix and right hand side, and
           get the symbolic factorization
   ----------------------------------------------------
*/
oldToNewIV = ETree_oldToNewVtxPerm(frontETree) ;
oldToNew   = IV_entries(oldToNewIV) ;
newToOldIV = ETree_newToOldVtxPerm(frontETree) ;
newToOld   = IV_entries(newToOldIV) ;
ETree_permuteVertices(frontETree, oldToNewIV) ;
InpMtx_permute(mtxA, oldToNew, oldToNew) ;
if (  symmetryflag == SPOOLES_SYMMETRIC
   || symmetryflag == SPOOLES_HERMITIAN ) {
   InpMtx_mapToUpperTriangle(mtxA) ;
}
InpMtx_changeCoordType(mtxA, INPMTX_BY_CHEVRONS) ;
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
DenseMtx_permuteRows(mtxY, oldToNewIV) ;
symbfacIVL = SymbFac_initFromInpMtx(frontETree, mtxA) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n old-to-new permutation vector") ;
   IV_writeForHumanEye(oldToNewIV, msgFile) ;
   fprintf(msgFile, "\n\n new-to-old permutation vector") ;
   IV_writeForHumanEye(newToOldIV, msgFile) ;
   fprintf(msgFile, "\n\n front tree after permutation") ;
   ETree_writeForHumanEye(frontETree, msgFile) ;
   fprintf(msgFile, "\n\n input matrix after permutation") ;
   InpMtx_writeForHumanEye(mtxA, msgFile) ;
   fprintf(msgFile, "\n\n right hand side matrix after permutation") ;
   DenseMtx_writeForHumanEye(mtxY, msgFile) ;
   fprintf(msgFile, "\n\n symbolic factorization") ;
   IVL_writeForHumanEye(symbfacIVL, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ------------------------------------------
   STEP 5: setup the domain decomposition map
   ------------------------------------------
*/
if ( nthread > (nfront = FrontMtx_nfront(frontmtx)) ) {
   nthread = nfront ;
}
cumopsDV = DV_new() ;
DV_init(cumopsDV, nthread, NULL) ;
ownersIV = ETree_ddMap(frontETree, type, symmetryflag,
                       cumopsDV, 1./(2.*nthread)) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n map from fronts to threads") ;
   IV_writeForHumanEye(ownersIV, msgFile) ;
   fprintf(msgFile, "\n\n factor operations for each front") ;
   DV_writeForHumanEye(cumopsDV, msgFile) ;
   fflush(msgFile) ;
}
DV_free(cumopsDV) ;
/*--------------------------------------------------------------------*/
/*
   ------------------------------------------
   STEP 6: initialize the front matrix object
   ------------------------------------------
*/
frontmtx   = FrontMtx_new() ;
mtxmanager = SubMtxManager_new() ;
SubMtxManager_init(mtxmanager, LOCK_IN_PROCESS, 0) ;
FrontMtx_init(frontmtx, frontETree, symbfacIVL, type, symmetryflag,
              FRONTMTX_DENSE_FRONTS, pivotingflag, LOCK_IN_PROCESS,
              0, NULL, mtxmanager, msglvl, msgFile) ;
/*--------------------------------------------------------------------*/
/*
   -----------------------------------------------------
   STEP 7: compute the numeric factorization in parallel
   -----------------------------------------------------
*/
chvmanager = ChvManager_new() ;
ChvManager_init(chvmanager, LOCK_IN_PROCESS, 1) ;
DVfill(10, cpus, 0.0) ;
IVfill(20, stats, 0) ;
rootchv = FrontMtx_MT_factorInpMtx(frontmtx, mtxA, tau, droptol,
                                 chvmanager, ownersIV, lookahead,
                                 &error, cpus, stats, msglvl, msgFile) ;
ChvManager_free(chvmanager) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n factor matrix") ;
   FrontMtx_writeForHumanEye(frontmtx, msgFile) ;
   fflush(msgFile) ;
}
if ( rootchv != NULL ) {
   fprintf(msgFile, "\n\n matrix found to be singular\n") ;
   exit(-1) ;
}
if ( error >= 0 ) {
   fprintf(msgFile, "\n\n fatal error at front %d", error) ;
   exit(-1) ;
}
/*--------------------------------------------------------------------*/
/*
   --------------------------------------
   STEP 8: post-process the factorization
   --------------------------------------
*/
FrontMtx_postProcess(frontmtx, msglvl, msgFile) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n factor matrix after post-processing") ;
   FrontMtx_writeForHumanEye(frontmtx, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -------------------------------------------------------
   STEP 9: get the solve map object for the parallel solve
   -------------------------------------------------------
*/
solvemap = SolveMap_new() ;
SolveMap_ddMap(solvemap, symmetryflag, FrontMtx_upperBlockIVL(frontmtx),
               FrontMtx_lowerBlockIVL(frontmtx), nthread, ownersIV,
               FrontMtx_frontTree(frontmtx), seed, msglvl, msgFile) ;
/*--------------------------------------------------------------------*/
/*
   --------------------------------------------
   STEP 10: solve the linear system in parallel
   --------------------------------------------
*/
mtxX = DenseMtx_new() ;
DenseMtx_init(mtxX, type, 0, 0, neqns, nrhs, 1, neqns) ;
DenseMtx_zero(mtxX) ;
FrontMtx_MT_solve(frontmtx, mtxX, mtxY, mtxmanager, solvemap,
                  cpus, msglvl, msgFile) ;
if ( msglvl > 1 ) {
   fprintf(msgFile, "\n\n solution matrix in new ordering") ;
   DenseMtx_writeForHumanEye(mtxX, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   --------------------------------------------------------
   STEP 11: permute the solution into the original ordering
   --------------------------------------------------------
*/
DenseMtx_permuteRows(mtxX, newToOldIV) ;
if ( msglvl > 0 ) {
   fprintf(msgFile, "\n\n solution matrix in original ordering") ;
   DenseMtx_writeForHumanEye(mtxX, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -----------
   free memory
   -----------
*/
FrontMtx_free(frontmtx) ;
DenseMtx_free(mtxX) ;
DenseMtx_free(mtxY) ;
IV_free(newToOldIV) ;
IV_free(oldToNewIV) ;
InpMtx_free(mtxA) ;
ETree_free(frontETree) ;
IVL_free(symbfacIVL) ;
SubMtxManager_free(mtxmanager) ;
Graph_free(graph) ;
SolveMap_free(solvemap) ;
IV_free(ownersIV) ;
/*--------------------------------------------------------------------*/
return(1) ; }
/*--------------------------------------------------------------------*/
\end{verbatim}


% LaTeX solution template for Math 5707 (Owen Levin, Darij Grinberg)
% ------------------------------------------------------------------

% Like most advanced LaTeX files, this one begins with a lot of
% boilerplate. You don't need to understand (or even read) most of it.
% All you need to do is fill in your name, UMN ID, email address,
% and the number of the pset. (Search for "METADATA" to find the place
% for this.) Then, you can go straight to the "EXERCISE 1"
% section and start writing your solutions.
% The "VARIOUS USEFUL COMMANDS" section is probably worth taking a
% look at at some point.

%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------
\documentclass[paper=a4, fontsize=12pt]{scrartcl} % A4 paper and 12pt font size
\usepackage[T1]{fontenc} % Use 8-bit encoding that has 256 glyphs
\usepackage[english]{babel} % English language/hyphenation
\usepackage{amsmath,amsfonts,amsthm,amssymb} % Math packages
\usepackage{mathrsfs}    % More math packages
\usepackage{sectsty}  % Allows customizing section commands
\allsectionsfont{\centering \normalfont\scshape} % Make all section titles centered, the default font and small caps %remove this to left align section tites
\usepackage{hyperref} % Turns cross-references into hyperlinks,
                      % and defines \url and \href commands.
\usepackage{graphicx} % For embedding graphics files.
\usepackage{framed}   % For the "leftbar" environment used below.
\usepackage{ifthen}   % Used for the \powset command below.
\usepackage{lastpage} % for counting the number of pages
\usepackage[headsepline,footsepline,manualmark]{scrlayer-scrpage}
\usepackage[height=10in,a4paper,hmargin={1in,0.8in}]{geometry}
\usepackage[usenames,dvipsnames]{xcolor}
\usepackage{tikz}     % This is a powerful tool to draw vector
                      % graphics inside LaTeX. In particular, you can
                      % use it to draw graphs.
\usepackage{verbatim} % For the "verbatim" environment, in which
                      % special symbols can be used freely without
                      % confusing the compiler. (And it's typeset in
                      % a constant-width font.)
                      % Useful, e.g., for quoting code (or ASCII art).

%\numberwithin{table}{section} % Number tables within sections (i.e. 1.1, 1.2, 2.1, 2.2 instead of 1, 2, 3, 4)

\setlength\parindent{20pt} % Makes indentation for paragraphs longer.
                           % This makes paragraphs stand out more.

%----------------------------------------------------------------------------------------
%	VARIOUS USEFUL COMMANDS
%----------------------------------------------------------------------------------------
% The commands below might be convenient. For example, you probably
% prefer to write $\powset[2]{V}$ for the set of $2$-element subsets
% of $V$, rather than writing $\mathcal{P}_2(V)$.
% Notice that you can easily define your own commands like this.
% Caveat: Some of these commands need to be properly "guarded" when
% they occur in subscripts or superscripts. So you should not write
% $K_\CC$, but rather $K_{\CC}$.
\newcommand{\CC}{\mathbb{C}} % complex numbers
\newcommand{\RR}{\mathbb{R}} % real numbers
\newcommand{\QQ}{\mathbb{Q}} % rational numbers
\newcommand{\NN}{\mathbb{N}} % nonnegative integers
\newcommand{\Z}[1]{\mathbb{Z}/#1\mathbb{Z}} % integers modulo k
                                            % (syntax: "\Z{k}")
\newcommand{\ZZ}{\mathbb{Z}} % integers
\newcommand{\id}{\operatorname{id}} % identity map
\newcommand{\lcm}{\operatorname{lcm}}
% Lowest common multiple. For historical reasons, LaTeX has a \gcd
% command built in, but not an \lcm command. The preceding line
% rectifies that.
\newcommand{\rev}{\operatorname{rev}} % reversal of a walk
\newcommand{\powset}[2][]{\ifthenelse{\equal{#2}{}}{\mathcal{P}\left(#1\right)}{\mathcal{P}_{#1}\left(#2\right)}}
% $\powset[k]{S}$ stands for the set of all $k$-element subsets of
% $S$. The argument $k$ is optional, and if not provided, the result
% is the whole powerset of $S$.
\newcommand{\set}[1]{\left\{ #1 \right\}}
% $\set{...}$ compiles to {...} (set-brackets).
\newcommand{\abs}[1]{\left| #1 \right|}
% $\abs{...}$ compiles to |...| (absolute value, or size of a set).
\newcommand{\tup}[1]{\left( #1 \right)}
% $\tup{...}$ compiles to (...) (parentheses, or tuple-brackets).
\newcommand{\ive}[1]{\left[ #1 \right]}
% $\ive{...}$ compiles to [...] (Iverson bracket, aka truth value).
\newcommand{\verts}[1]{\operatorname{V}\left( #1 \right)}
% $\verts{...}$ compiles to V(...) (vertex set of a graph/digraph).
\newcommand{\edges}[1]{\operatorname{E}\left( #1 \right)}
% $\edges{...}$ compiles to E(...) (edge set of a graph).
\newcommand{\arcs}[1]{\operatorname{A}\left( #1 \right)}
% $\arcs{...}$ compiles to A(...) (arc set of a digraph).
\newcommand{\underbrack}[2]{\underbrace{#1}_{\substack{#2}}}
% $\underbrack{...1}{...2}$ yields
% $\underbrace{...1}_{\substack{...2}}$. This is useful for doing
% local rewriting transformations on mathematical expressions with
% justifications. For example, try this out:
% $ \underbrack{(a+b)^2}{= a^2 + 2ab + b^2 \\ \text{(by the binomial formula)}} $
\newcommand{\horrule}[1]{\rule{\linewidth}{#1}} % Create horizontal rule command with 1 argument of height

%----------------------------------------------------------------------------------------
%	MAKING SUMMATION SIGNS ALWAYS PUT THEIR BOUNDS ABOVE AND BELOW
%	THE SIGN
%----------------------------------------------------------------------------------------
% The following are hacks to ensure that sums (such as
% $\sum_{k=1}^n k$) always put their bounds (i.e., the $k=1$ and the
% $n$) underneath and above the sign, as opposed to on its right.
% Same for products (\prod), set unions (\bigcup) and set
% intersections (\bigcap). Remove the 8 lines below if you do not want
% this behavior.
\let\sumnonlimits\sum
\let\prodnonlimits\prod
\let\cupnonlimits\bigcup
\let\capnonlimits\bigcap
\renewcommand{\sum}{\sumnonlimits\limits}
\renewcommand{\prod}{\prodnonlimits\limits}
\renewcommand{\bigcup}{\cupnonlimits\limits}
\renewcommand{\bigcap}{\capnonlimits\limits}

%----------------------------------------------------------------------------------------
%	ENVIRONMENTS
%----------------------------------------------------------------------------------------
% The incantations below define how theorem environments
% (\begin{theorem} ... \end{theorem}) and their likes will look like.
\newtheoremstyle{plainsl}% <name>
  {8pt plus 2pt minus 4pt}% <Space above>
  {8pt plus 2pt minus 4pt}% <Space below>
  {\slshape}% <Body font>
  {0pt}% <Indent amount>
  {\bfseries}% <Theorem head font>
  {.}% <Punctuation after theorem head>
  {5pt plus 1pt minus 1pt}% <Space after theorem headi>
  {}% <Theorem head spec (can be left empty, meaning `normal')>

% Environments which make the text inside them slanted:
\theoremstyle{plainsl}
  \newtheorem{theorem}{Theorem}[section]
  \newtheorem{proposition}[theorem]{Proposition}
  \newtheorem{lemma}[theorem]{Lemma}
  \newtheorem{corollary}[theorem]{Corollary}
  \newtheorem{conjecture}[theorem]{Conjecture}
% Environments that don't:
\theoremstyle{definition}
  \newtheorem{definition}[theorem]{Definition}
  \newtheorem{example}[theorem]{Example}
  \newtheorem{exercise}[theorem]{Exercise}
  \newtheorem{examples}[theorem]{Examples}
  \newtheorem{algorithm}[theorem]{Algorithm}
  \newtheorem{question}[theorem]{Question}
 \theoremstyle{remark}
  \newtheorem{remark}[theorem]{Remark}
\newenvironment{statement}{\begin{quote}}{\end{quote}}

%----------------------------------------------------------------------------------------
%	METADATA
%----------------------------------------------------------------------------------------
\newcommand{\myname}{Darij Grinberg} % ENTER YOUR NAME HERE
\newcommand{\myid}{00000000} % ENTER YOUR UMN ID HERE
\newcommand{\mymail}{dgrinber@umn.edu} % ENTER YOUR EMAIL HERE
\newcommand{\psetnumber}{0} % ENTER THE NUMBER OF THIS PSET HERE

%----------------------------------------------------------------------------------------
%	HEADER AND FOOTER
%----------------------------------------------------------------------------------------
\ihead{Solutions to homework set \#\psetnumber} % Page header left
\ohead{page \thepage\ of \pageref{LastPage}} % Page header right
\ifoot{\myname, \myid} % left footer
\ofoot{\mymail} % right footer

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------
\title{
\normalfont \normalsize
\textsc{University of Minnesota, School of Mathematics} \\ [25pt] % Your university, school and/or department name(s)
\horrule{0.5pt} \\[0.4cm] % Thin top horizontal rule
\huge Math 5707: Graph Theory, Spring 2017\\
Homework \psetnumber\\% The assignment title
\horrule{2pt} \\[0.5cm] % Thick bottom horizontal rule
}
\author{\myname}

\begin{document}

\maketitle % Print the title

%----------------------------------------------------------------------------------------
%	EXERCISE 1
%----------------------------------------------------------------------------------------
\horrule{0.3pt} \\[0.4cm]

\section{Exercise 1}

\subsection{Problem}

Let $G = \tup{V, E}$ be a simple graph.
% The \tup{...} command puts the ... in parentheses.
If $k \in \NN$,
% \NN is an abbreviation for \mathbb{N}, and stands for the
% nonnegative integers.
then a \textit{$k$-coloring} of $G$ means a map
$f : V \to \set{1, 2, \ldots, k}$.
% \ldots is the neatest way to get a "..." in LaTeX. But no one will
% mind if you just write "...".
If $f$ is a $k$-coloring of $G$ for some $k \in \NN$, then the value
$f \tup{v}$ of $f$ at a vertex $v \in V$ is called the
\textit{color} of $v$ in the $k$-coloring $f$.
(It is customary to visualize a $k$-coloring by pretending that the
numbers $1, 2, \ldots, k$ are colors, and so a $k$-coloring assigns to
each vertex a color; i.e., it ``colors'' the vertices.)

Prove that there exists a $2$-coloring of $G$ having the following
property: For each vertex $v \in V$, at most $\dfrac{1}{2} \deg v$
among the neighbors of $v$ have the same color as $v$.

\begin{remark} \label{rmk.exe1.r}
This problem is often restated as follows: You are given a (finite)
set of politicians; some politicians are mutual enemies. (No one is
their own enemy. If $u$ is an enemy of $v$, then $v$ is an enemy of
$u$. An enemy of an enemy is not necessarily a friend. So this is just
a simple graph.) Prove that it is possible to subdivide this set into
two (disjoint) parties such that no politician has more than half of
his enemies in his own party.
\end{remark}

\subsection{Solution}
% The solution below is written in more detail than necessary, to
% illustrate both the use of LaTeX and the concept of an algorithmic
% proof.
The idea of the solution is fairly simple, particularly using the
convenient language of Remark \ref{rmk.exe1.r}:
% The \ref command provides a cross-reference to a theorem,
% remark, definition, section or whatever. Of course, you need to
% specify which theorem etc. you want to reference; this is achieved
% by defining a label (in our case, "rmk.exe1.r"), which is done using
% the \label command.
% Note that you need to compile a LaTeX file *twice* in order for the
% cross-references to appear! During the first compilation, LaTeX
% collects all the labels and stores them in an .aux file; during the
% second compilation, it then actually writes the cross-references.
% Hence, you also need to recompile a LaTeX file twice if you have
% added new theorems/definitions/whatever (or removed some), because
% during the first compilation the .aux file is still unaware of the
% new theorems and so the numbers will be out-of-sync.
We subdivide our set of
politicians into two parties in some arbitrary way (e.g., by throwing
them all into one party and keeping the other party empty), and then
we improve the situation step by step by picking a politician who has
more than half his enemies in his own party, and moving him to the
opposite party. However, will this procedure necessarily terminate, or
will it result in an eternal process of politicians being kicked
around back and forth between the two parties? Fortunately, it will
terminate, but this needs to be proven.

Let us first make this procedure rigorous:

\begin{algorithm} \label{alg.exe1.1}
\begin{leftbar}
% The "leftbar" environment places a vertical bar on the left of
% whatever is inside it. I find it quite convenient here to clarify
% where this algorithm ends. But handle with care: For example,
% footnotes don't work if you are in a leftbar environment...

\textbf{Input:} a simple graph $G = \tup{V, E}$.

\textbf{Output:} a $2$-coloring of $G$ having the following
property: For each vertex $v \in V$, at most $\dfrac{1}{2} \deg v$
among the neighbors of $v$ have the same color as $v$.

\begin{enumerate}
% The "enumerate" environment creates a numbered list. If you want
% custom numbers (as opposed to just 1, 2, 3, ...), you can use
% \item[X] instead of plain \item.
\item Define a $2$-coloring $f : V \to \set{1, 2}$ arbitrarily (for
      example, by setting $f \tup{v} = 1$ for all $v \in V$).
\item \textbf{While} there exists some $v \in V$ such that
      more than $\dfrac{1}{2} \deg v$ among the neighbors of $v$ have
      the same color as $v$, \textbf{do} the following:
      \begin{itemize}
      % The "itemize" environment creates a bullet-point (unnumbered)
      % list. You can nest such environments inside them.
      \item Flip the color $f \tup{v}$ of this $v$ (that is, change
            this color to $2$ if it is $1$, and change it to $1$ if it
            is $2$).
      \end{itemize}
\item Output the $2$-coloring $f$.
\end{enumerate}
\end{leftbar}
\end{algorithm}

It is clear that if step 3 of this algorithm is ever reached, then the
$2$-coloring $f$ outputted by the algorithm
does have the property that for each vertex
$v \in V$, at most $\dfrac{1}{2} \deg v$ among the neighbors of $v$
have the same color as $v$. (Indeed, this property is saying
precisely that we have fallen out of the while-loop in step 2.) But
we need to show that step 3 will eventually be reached. This is not
obvious, since step 2 contains a while-loop, and a while-loop may go
on indefinitely. We need to prove that this while-loop must eventually
come to an end.

The most common way to prove such a claim is by exhibiting a
\textit{loop monovariant}. In our situation, this means a nonnegative
integer defined for each $2$-coloring $f$ and which has the property
that in every iteration of the while-loop, this integer decreases
(strictly). If we can define such an integer, then we can immediately
conclude that the while-loop must eventually come to an end (because
a nonnegative integer cannot keep decreasing indefinitely while
remaining a nonnegative integer).

The role of this nonnegative integer (the loop monovariant) will be
played by what I call the ``enmity'' of a $2$-coloring $f$:

If $f$ is a $2$-coloring of $G$, then I define the \textit{enmity} of
$f$ to be the number of $f$-monochromatic edges of $G$. Here, an edge
$e$ of $G$ is said to be \textit{$f$-monochromatic} if the two
endpoints of $e$ have the same color in the $2$-coloring $f$.

For each $2$-coloring $f$ of $G$, the enmity of $f$ is clearly a
nonnegative integer. Now, I claim the following:
\begin{statement}
\textit{Claim 1:} Let $f$ be a $2$-coloring of $G$. Let $v \in V$ be
a vertex such that more than $\dfrac{1}{2} \deg v$ among the neighbors
of $v$ have the same color as $v$. If we flip the color $f \tup{v}$ of
$v$, then the enmity of $f$ \textbf{decreases}.
\end{statement}

\begin{proof}[Proof of Claim 1.]
% The "proof" environment automatically puts a square at the end of
% the proof.
Let $a$ be the number of neighbors of $v$ that have the same color as
$v$. Then, $a > \dfrac{1}{2} \deg v$ (since more than
$\dfrac{1}{2} \deg v$ among the neighbors of $v$ have the same color
as $v$). Hence, $2a > \deg v$, so that $a > \deg v - a$.

Clearly, an edge $e$ containing $v$ is monochromatic\footnote{We
abbreviate the word ``$f$-monochromatic'' as ``monochromatic''.}
if and only if the neighbor of $v$ contained in $e$ has the same color
as $v$.
Thus, the monochromatic edges containing $v$ are in one-to-one
correspondence with the neighbors of $v$ that have the same color as
$v$. Since the number of the latter neighbors is $a$, we thus conclude
that the number of the former edges is also $a$. In other words, among
the $\deg v$ edges containing $v$, exactly $a$ are monochromatic.
If we flip the color $f \tup{v}$ of $v$, then these $a$ monochromatic
edges become no longer monochromatic (because they now connect
vertices of different color), whereas the remaining $\deg v - a$ edges
containing $v$ become monochromatic. As for the edges that do not
contain $v$, their status does not change (i.e., if they are
monochromatic before the flipping, then they remain so, and
if they are not monochromatic before the flipping, then they do not
become monochromatic), because none of their vertices changes its
color.
Hence, by flipping the color $f \tup{v}$ of $v$, we lose $a$
monochromatic edges but, at the same time, gain $\deg v - a$ new
monochromatic edges. Since $a > \deg v - a$, we thus lose
more monochromatic edges than we gain. Therefore, the number of
monochromatic edges of $G$ decreases. In other words, the enmity
of $f$ decreases (since the enmity of $f$ is defined as the number
of monochromatic edges of $G$). This proves Claim 1.
\end{proof}

Claim 1 shows that the enmity of $f$ decreases in each iteration of
the while-loop in Algorithm \ref{alg.exe1.1}. Hence, this while-loop
cannot go on indefinitely (since the enmity of $f$ is a nonnegative
integer, and thus cannot keep decreasing forever). Thus,
Algorithm \ref{alg.exe1.1} must eventually terminate (since the
while-loop in step 2 is the only part in which the algorithm might
get stuck). As we already
know, the $2$-coloring $f$ outputted by the algorithm
does have the property that for each vertex
$v \in V$, at most $\dfrac{1}{2} \deg v$ among the neighbors of $v$
have the same color as $v$. As a consequence, such a $2$-coloring
exists. This solves the exercise.

\subsection{Postscriptum}

\begin{remark}
In
\href{http://www.cip.ifi.lmu.de/~grinberg/t/17s/nogra.pdf}{the lecture notes},
% This is an example of a hyperlink created by the \href command.
% The general syntax is \href{URL}{text}. If you want just the URL
% itself to appear, you can also use \url{URL}.
I have stated the fact that
if $G = \tup{V, E}$ is a simple graph that has no isolated vertices,
then there exist two disjoint dominating subsets $A$ and $B$
of $V$ such that $A \cup B = V$. Do you see why this follows from
the above exercise?
\end{remark}

%Copy or imitate the above for each exercise you solve.

\end{document}


\section*{DATA}\label{data}
The data used in this study is described in
Tab.\ref{fig:data_description}.
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/data_description.png}\end{center}
\vspace{-0.7cm}
\caption{Spring-mass-damper system}
\label{fig:data_description}
\end{figure}
\subsection*{Exploratory data analysis}\label{exploratory-data-analysis}
The ship speed $V$, ship draughts $T_{aft}$ and $T_{fwd}$ were all
negative in the raw data file. This was imidiatelly corrected, to be
more in line with what would be expected from a more general sign
convention. The data seems to have been collected in time cronological
order, giving a time series of data. For a time series, measurements
close to each other in time have a high correlation, as they are
experiencing similar envinronmental conditions etc. This is confirmed by
looking at the autocorrelation plot in
Fig.\ref{fig:power_autocorrelation}. Dead reckoning (using ship
speed and heading) has been used to atempt to describe motion of the
ship as seen in Fig.\ref{fig:dead_reckoning}. The positions are
given in an unknown longintude and latitude scale, as the time step
between measurements is unknown. The speed of the ship is also indicated
as a color gradient in this figure.
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/power_autocorrelation.pdf}\end{center}
\vspace{-0.7cm}
\caption{Autocorrelation plot of the Power data}
\label{fig:power_autocorrelation}
\end{figure}
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/dead_reckoning.pdf}\end{center}
\vspace{-0.7cm}
\caption{Dead reckoning of the position of the ship}
\label{fig:dead_reckoning}
\end{figure}
Fig.\ref{fig:heat_map_raw_data} shows a heat map of the absolute
linear correlation coefficient between all of the features in the raw
data. It can be seen that $T_{aft}$ and $T_{fwd}$ have the highest
correlation with the $Power$. It can also be seen that the correlation
between $T_{aft}$ and $T_{fwd}$ is also very high (approximately 1)
implying a very high multicollinearity which is generally something that
should be avoided in regression problems. These two features are instead
replaced with the two features: mean draught $T$ and $trim$. The
correspondig heat map with the new features is shown in
Fig.\ref{fig:heat_map_data}. The mean draught $T$ now seems to
be a very important feature in this regression as it has the highest
linear correlation with the $Power$. This can also be seen in
Fig.\ref{fig:power_draught}, where the $Power$ has been
plotted together with the corresponding negative draught.
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/heat_map_raw_data.pdf}\end{center}
\vspace{-0.7cm}
\caption{Heat map showing absolute value of correlation coefficient between features in raw data}
\label{fig:heat_map_raw_data}
\end{figure}
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/heat_map_data.pdf}\end{center}
\vspace{-0.7cm}
\caption{Heat map showing absolute value of correlation coefficient between features in transformed data}
\label{fig:heat_map_data}
\end{figure}
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/power_draught.pdf}\end{center}
\vspace{-0.7cm}
\caption{The power is highly correlated with the draught}
\label{fig:power_draught}
\end{figure}
\subsection*{Extend data}\label{extend-data}
The data has also been extended by adding some additional features,
describing the wind speed and wave direction in a ship fixed coordinate
system:$u_{wind}$, $v_{wind}$, $wave_{direction}$. The new
features have very low linear correlation with $Power$ as seen in
Fig.\ref{fig:heat_map_extended_data}. They are however kept in
the data, to let the regression model decide if they are useable or not.
\begin{figure}[H]
\begin{center}\includegraphics[width = 0.95\textwidth]{figures/heat_map_extended_data.pdf}\end{center}
\vspace{-0.7cm}
\caption{Heat map of data with additional features.}
\label{fig:heat_map_extended_data}
\end{figure}