\documentclass[11pt, letterpaper]{article}

\input{packages.tex}

\author{Thomas Alexander, Zachary Schoenfeld}

\title{Project: Optimal Control Theory with Qiskit Pulse}

% Full command definitions
\input{imports/quantum_info.tex}

\begin{document}
\maketitle
\tableofcontents

\section{Introduction}
Optimal control theory studies how to optimally (given an objective) manipulate a quantum system
to guide a state through a desired evolution given available classical
stimulus sources. Qiskit Pulse exposes control of quantum computing
stimulus sources through a standardized API, making lab-level experiments
possible through the cloud. In Qiskit a quantum circuit is lowered to a pulse
schedule through a scheduling procedure using predefined gate to pulse schedule
calibrations.

The aim of this project is to explore incorporating optimal control theory
into the Qiskit compilation pipeline with the aim of enabling on the fly
gate design of arbitrary unitary evolutions for applications in Qiskit Aqua.
Specifically you will examine implementing GRAPE \cite{khanejaOptimalControlCoupled2005c} on
a single-qubit IBM Quantum device.
This idea was explored in the work of Shi et. al. \cite{shiOptimizedCompilationAggregated2019b}
(see Figure \ref{fig:compilationpipeline}) with their focus on dynamic gate
aggregation and design.

\begin{figure}[hbt!]
 \centering
 \includegraphics[width=\linewidth]{../figures/yunong_pipeline.png}
 \caption{Compilation pipeline proposed by Shi et. al., reproduced from \cite{shiOptimizedCompilationAggregated2019b}.}
 \label{fig:compilationpipeline}
\end{figure}

We will begin to build out this functionality into Qiskit.
You will take the first exploratory steps of performing optimal control on actual qubits with Qiskit
Pulse and then come up with a proposal for how to add this functionality
to the Qiskit compilation pipeline in production.

\subsection{Overall Goals}
\begin{itemize}
\item Design and implement an optimal pulse control pipeline for constructing single qubit quantum gates
\item Develop your physics and software skills by applying them to optimal control theory
\item Gain exposure to the agile development processes and working with a full stack team across software and hardware
\item Have some fun!
\end{itemize}

\subsubsection{Technical Goals}
\begin{itemize}
\item Understand in a broad sense the engineering of superconducting qubits \cite{blaisCavityQuantumElectrodynamics2004c, krantzQuantumEngineerGuide2019a}.
\item Bootstrap control of a qubit with Qiskit Pulse. See \cite{CalibratingQubitsQiskit}.
\item Model the Hamiltonian of your system and simulate it.
\item Understand the goal of optimal control and how GRAPE works \cite{khanejaOptimalControlCoupled2005c}.
\item Select a Python GRAPE package for incorporation into Qiskit. Examples include \cite{QuantumOptimalControl, SchusterLabQuantumoptimalcontrol2020, abdelhafezGradientbasedOptimalControl2019, QuantumUtilsMathematica}.
\item Estimate the Hamiltonian of a single qubit with Qiskit Pulse \cite{alexanderNotebooksDataQiskitPulse2020b, hincksHamiltonianLearningOnline2018, sheldonProcedureSystematicallyTuning2016b}.
\item Design arbitrary single qubit gates with Grape and Qiskit Pulse and characterize the gates with quantum process tomography \cite{alexanderNotebooksDataQiskitPulse2020b}.
\item Hack automated gate design into the Qiskit compilation pipeline.
\item Write a report reviewing the relevant underlying theory, your methodology and findings performing optimal control theory with Qiskit pulse.
Include a design section proposing how to implement automated gate aggregation \cite{shiOptimizedCompilationAggregated2019b} and compilation within Qiskit and any improvements that might be required.
\item Stretch goals (in no particular order other than what would be most fun for me)

\begin{itemize}
\item Estimate the Hamiltonian for a two-qubit system and design a two-qubit gate with Qiskit Pulse and GRAPE.
\item Reduce the runtime of an Aqua application with automated gate design.
\item Design a gate aggregation pass in the Qiskit transpiler to work with your pulse designer.
\item Suggest how to improve the usability of the Qiskit Pulse simulator.
\end{itemize}
\end{itemize}

\subsection{Workflow}
Due to the nature of the situation with Covid-19 the internship will be remote.
Fortunately, this project is focused on Qiskit Pulse, which gives hardware access
through the cloud. To help facilitate work both Thomas Alexander and
Zachary Schoenfeld will be providing supervision. Ben will partcipate in the
pulse teams agile sprints, demos and board. We will hold short, daily video
calls to synchronize our work. There will also be many other remote activities
setup for interns to network and socialize through the IBM Quantum Internship
program.\\
%
\\
%
IBM Quantum intern hub: \url{https://w3.ibm.com/w3publisher/quantum-interns}.

\subsection{Approximate Timeline (Subject to Change)}

Dates: 6/5/20-8/7/20 (8 weeks) \\
\\
%
Weeks 1, 2
\begin{itemize}
\item Onboard
	\begin{itemize}
		\item Get onto w3, Slack, join standups and agile
	\end{itemize}
\item Gain familiarity with IBM's quantum systems and pulse-level control (ask for more papers!)
\item Use Qiskit pulse to calibrate a qubit on a live device
\item \textit{Suggestion}: Draft notes as you go. This will make writing the final report easier!
\end{itemize}
Weeks 3, 4
\begin{itemize}
\item Model a 1q Hamiltonian with arbitrary drive pulse and simulate its evolution
	\begin{itemize}
		\item It may also be interesting to learn how to analytically solve this Hamiltonian for simple pulses shapes (RWA, etc)
	\end{itemize}
\item Learn GRAPE and explore possible open source packages for implementation into Qiskit
\end{itemize}
Weeks 5, 6
\begin{itemize}
\item Use chosen GRAPE package and Qiskit pulse to construct arbitrary 1q gates in an optimal fashion
\item Characterize these gates with Quantum Process Tomography
\item Begin to integrate GRAPE into the Qiskit compilation pipeline
\end{itemize}
Weeks 7, 8
\begin{itemize}
\item As time permits, think about how to add gate aggregation passes \cite{shiOptimizedCompilationAggregated2019b} to your work (design is more important; can implement if time)
\item Write up a report detailing the theory you learned about optimal pulse control, GRAPE and how you implemented it in Qiskit. More details provided in the \textbf{Goals} section.
\item As time permits, work on the stretch goals outlined in the \textbf{Goals} section
\item Offboard
\end{itemize}

\bibliographystyle{unsrt}
\bibliography{bibliography}

\end{document}


\documentclass[11pt]{extarticle}
\usepackage[a4paper, margin=1in]{geometry}

\usepackage[utf8]{inputenc}
\usepackage{amsmath,mathtools}

\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{tcolorbox}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{xspace}
\usepackage{hyperref}

\usepackage{tabularx}

\newtheorem{theorem}{Theorem}[section]
\newtheorem{claim}[theorem]{Claim}
\newtheorem{definition}{Definition}

\hypersetup{
    colorlinks=true,
    citecolor=blue,
    linkcolor=blue,
    filecolor=magenta,
    urlcolor=blue,
}
\usepackage{xcolor, soul}
\newcommand{\note}[1]{\textcolor{purple}{\small {#1}}}
\newcommand{\tool}{{\sc SynGuar}\xspace}
\newcommand{\toolvsa}{{\sc SynGuar-PROSE}\xspace}
\newcommand{\toolstun}{{\sc SynGuar-STUN}\xspace}
\newcommand{\parf}{g}

\newcommand{\distrib}{D}
\newcommand{\programsp}{H_S}
\newcommand{\samples}[2]{m_\programsp({#1}, {#2})}
\newcommand{\synthalgo}{\mathcal{A}(S)}
\newcommand{\etal}{et al.\xspace}
\newcommand{\concept}{\mathcal{C}}
\renewcommand{\vec}[1]{\mathbf{#1}}

\newcommand{\prb}{\mathop{\text{Pr}}}
\newcommand{\prbh}{\mathop{\text{Pr}}_{h_A\in H}}
\newcommand{\ep}{\mathop{\mathbb{E}}}
\newcommand{\var}{\textsf{Var}}
\newcommand{\cov}{\textsf{Cov}}
\newcommand{\ind}{\mathop{\mathbb{I}}}
\newcommand{\sothat}{\text{ s.t. }}

\usepackage[numbers]{natbib}

\title{Mathematical Proofs for \tool}

\begin{document}

\maketitle

\section{Preliminaries}

\paragraph{Problem Setup.} We are given an oracle to sample i.i.d. I/O examples from some unknown distribution $\distrib$, a synthesizer with bounded hypothesis space and the ability to soundly upper bound number of programs consistent with I/O examples, and user-specified $(\epsilon,\delta)$ parameters that capture the desired generalization guarantee. The synthesis algorithm queries the oracle for as many I/O examples as it needs and terminates with either $None$ or a synthesized function $f$. We assume that $f$ will satisfy all given I/O examples. We use $S$ to represent all the I/O examples queried by the synthesis algorithm until it terminates. The probability that the synthesizer returns a function $f$ that might not generalize should be under the given small $\delta$, and the randomness is on I/O examples.

Formally, the goal is for a synthesis algorithm to achieve a PAC-style $(\epsilon,\delta)$ generalization guarantee. We define $(\epsilon,\delta)$-synthesizer as the following:

\begin{definition}[$(\epsilon,\delta)$-synthesizer]
\label{def:eps-delta}
  A synthesis algorithm $\mathcal{A}$ with hypothesis space $H$ is an $(\epsilon, \delta)$-synthesizer with respect to a target class of
  functions $\concept$ iff for any input distributions $\distrib$, for all $t
  \in \concept$, $\epsilon \in (0, 0.5)$, $\delta \in (0, 0.5)$, if $\synthalgo$ outputs a program $f \in H$ on a set of samples $S$ drawn i.i.d from the $\distrib$, then:
%  \bo{The event and condition of Pr seems inverted}
  \begin{align*}
    Pr\lbrack \synthalgo~\text{outputs}~ f \text{ such that } error(f) > \epsilon \rbrack < \delta
  \end{align*}

\end{definition}

\paragraph{A starting point.} The number of examples provably sufficient to achieve the $(\epsilon, \delta)$-generalization is given by Blumer \etal~\cite{blumer1987occam}. We restate this result, which computes sample complexity as a function of $(\epsilon,\delta)$ and the capacity (or size) of any given hypothesis space $H$.

\begin{theorem}[Sample Complexity for $(\epsilon, \delta)$-synthesis]
  \label{thm:static-union-bound}
For all $\epsilon \in (0,\frac{1}{2})$, $\delta\in
(0,\frac{1}{2})$, hypothesis space $H$ and any target program $t$, a synthesis algorithm $\synthalgo$ which outputs functions consistent with $n$ i.i.d samples is an $(\epsilon, \delta)$-synthesizer, if
\begin{align*}
  n > \frac{1}{\epsilon}(\ln |H| + \ln
  \frac{1}{\delta})
\end{align*}

\end{theorem}

\begin{proof}
Assume the true concept (may or may not in the hypothesis space) is $t$ and all I/O examples are consistent with $t$.
Now consider a single hypothesis $f\in H$ first.
Let $error(f) = L_{(\mathcal{D},t)}(f)$ be the true error (expectation of 0-1 loss) $L_{(\mathcal{D},t)}(f)=\mathop{\mathbb{E}}_{x\sim\mathcal{D}}\mathbb{I}[t(x)\neq f(x)]$,
and let $L_{(\mathcal{S},t)}(f)$ be the empirical error (empirical average of 0-1 loss) $L_{(\mathcal{S},t)}(f)=\frac{1}{|S|}\sum_{x\in S}\mathbb{I}[t(x)\neq f(x)]$,
The following holds:
\begin{align*}
\prb_{x\in \mathcal{D}} [\mathbb{I}[t(x)\neq f(x)] = 0 ~|~ L_{(\mathcal{D},t)}(f) \geq \epsilon] \leq (1-\epsilon)
\end{align*}
Now consider a sample $S$ with size $n$, the following holds:
\begin{align*}
\prb_{S\in \mathcal{D}^{n}} [L_{(\mathcal{S},t)}(f) = 0 ~|~ L_{(\mathcal{D},t)}(f) \geq \epsilon] \leq (1-\epsilon)^{n}
\end{align*}

Then take union bound on all hypothesis in $H$:
\begin{align*}
    &\prb_{S\in \mathcal{D}^{n}} [\exists f\in H, ~L_{(\mathcal{S},t)}(f) = 0 ~\land~ L_{(\mathcal{D},t)}(f) \geq \epsilon] \leq |H|(1-\epsilon)^{n} < \delta\\
    \Rightarrow & ~n > \frac{1}{\epsilon}(\ln |H| + \ln
    \frac{1}{\delta}) \text{ suffices}
\end{align*}

So with sample size larger than $\frac{1}{\epsilon}(\ln |H| + \ln
\frac{1}{\delta})$, with probability at least $1-\delta$, all $f\in H$ that
have true error larger than $\epsilon$ have a non-zero empirical loss and will be ruled out
and any hypothesis in $H$ that is still consistent with the sample has
true error smaller than $\epsilon$.

\end{proof}

\section{Analysis of \tool}

\begin{algorithm}[t]
    \caption{\tool Synthesis returns a program with error smaller than $\epsilon$ with probability higher than $1-\delta$}
    \label{alg:main}
    \begin{algorithmic}[1]
      \Procedure{\tool}{$\epsilon, \delta$}
      \State $k = 1$ // tunable parameter
      \State $g \gets \Call{PickStoppingCond}{}$ \label{alg:pick-cond}
      \State $S' \gets \varnothing, s \gets 0$
      \State $size_{H} \gets \Call{ComputeSize}{H}$ \label{alg:call-compute}
      \State $n \gets g(size_{H})$
      %\While {$|S'| \leq n$}
      \While {$s \leq n$}
      \State $S' \gets S' \cup \Call{sample}{k}$
      \State $H_{S'} \gets $ \Call{UpdateHypothesis}{$S'$} \label{alg:call-update}
      %\State $H_{S'}$ = synthesize($S'$)
      \State $size_{H_{S'}} \gets \Call{ComputeSize}{H_{S'}}$ \label{alg:call-compute}
      \State $s \gets s + k$
      \State $n \gets min(n, s+g(size_{H_{S'}}))$ \label{alg:min-thresh}
      \EndWhile
      \State $m_{H_{S'}}=\frac{1}{\epsilon}(\ln{size_{H_{S'}}} + ln{\frac{1}{\delta}})$
      \State $T \gets \Call{sample}{m_{H_{S'}}(\epsilon, \delta)}$
      \State $S \gets S'\cup T$
      \State \Return $f$ program in $H_S$
      \EndProcedure
    \end{algorithmic}
\end{algorithm}

\tool's design is motivated by being able to give a formal generalization guarantee and a bounded sample complexity. For this purpose, we state and prove the following properties:
% Here, we detail and prove these properties.

\noindent\textbf{(P1: Termination)} \tool always terminates for a finite $|H|$.

\noindent\textbf{(P2: $(\epsilon, \delta)$ guarantees)} If \tool returns an $f$ then $f$ is $\epsilon$-far with probability $<\delta$.

\noindent\textbf{(P3: Sample complexity)} \tool's sample complexity is always within 2$\times$ of the optimal.

\begin{theorem}[P1]
\label{thm:p1}
\tool always terminates for a finite $|H|$.
\end{theorem}
\begin{proof}
It suffices to prove that the sampling phase (lines $7-13$) of \tool terminates in order to show that \tool terminates.
%The phase can be concisely written as follows:
In each iteration of the sampling phase,  let $S_i$ be the queue storing the user-provided examples after each iteration, $z_t$ be the $t^\text{th}$ example, $S_{i+1} = S_{i}\cup \{z_{ik+1},...z_{ik+k}\}$ and $S_0=\varnothing$. For each $S_i$, $H_{S_i}$ determines the set of consistent hypothesis that satisfy $S_i$. Let $N_i$ be the limit of  the number of I/O examples $n$ for the sampling phase after iteration $i$.
For iterations $i$ and $j$ where $i<j$ and $\forall g:\mathbb{N} \rightarrow \mathbb{Z}$ such that $g$ is monotonically non-decreasing, the following holds:
\begin{align*}
  &S_i\subset S_j \Rightarrow |H_{S_j}|\leq |H_{S_i}| \Rightarrow g(|H_{S_j}|) \leq g(|H_{S_i}|)\\
  &N_j \leq \min\{N_i, |S_j| + g(|H_{S_j}|)\} \leq N_i \text{ (see line $13$ in Alg.~\ref{alg:main})}
\end{align*}
Therefore, if $N_0 \leq g(|H|)$ then the loop will terminate at some iteration $p$ such that $N_p < |S_p| \leq N_p + k \leq N_0 + k$.
\end{proof}

\begin{theorem}[P2]
\label{thm:p2}
If \tool($\epsilon, \delta$) returns the synthesized program $f$ then $f$ is $\epsilon$-far with probability $<\delta$.
\end{theorem}
\begin{proof}
By Theorem~\ref{thm:p1}, we know that the sampling phase terminates with $S'$ samples (see line $14$). In lines $14-16$ \tool samples an additional number of I/O examples required to generalize and then synthesizes a program after seeing the additional samples. Therefore, Theorem~\ref{thm:p2} follows from Theorem~\ref{thm:static-union-bound}.
\end{proof}
% ====================================================

% \begin{theorem}{Sample Complexity}
%     \tool's sample complexity is within
% $2\times$ of the optimal.
% \end{theorem}

% % The last property (P3) of \tool is that its sample complexity is within
% % $2\times$ of the optimal. The optimal number of samples is determined by the
% % shortest length prefix that guarantees that the algorithm returns with
% % $(\epsilon, \delta)$ guarantees.
% \begin{proof}

% \end{proof}
In order to prove the last property, we define a new quantity $\omega(Q)$. It is the smallest sample size taken by \tool($\epsilon, \delta$) for any non-decreasing $g$ used for a sequence of I/O examples $Q$.

\begin{definition}[Smallest dynamic sample size]
    For any infinite sampled sequence of examples $Q$, let $\Call{Prefix}{Q,g}$ be the prefix of $Q$ at which \tool($\epsilon, \delta$) terminates. Then,
    % In other word, it is like that the best  stopping condition that result in smallest sample size is known before seeing the examples.
    $$
    \omega(Q) = \inf \{ |m_g|~:~\forall g,\text{ } m_g = \Call{Prefix}{Q,g}\}
    $$
\end{definition}

% Note that if we modify \tool to always take $\omega(Q)$ examples when the sampled sequence is $Q$, it is not an $(\epsilon,\delta)$-synthesizer.

\begin{theorem}[P3]
\label{thm:p3}
\tool uses no more than $2\omega(Q)$ examples on any $Q$ when the result is not None with $\parf(x) = \parf_{0}(x)=\max\{0, ~\frac{1}{\epsilon}(\ln(x)-\ln(\frac{1}{\delta}))\}$ and $k\leq \frac{1}{2\epsilon}\ln\frac{1}{\delta}$.
\end{theorem}

\begin{proof}
Let $S'\subset Q$ be the samples in sampling phase. Let $P$ be the samples when $\omega(Q)=\Call{Prefix}{Q,\parf}$ for some $\parf$ and let us call this the best stopping point for \tool's sampling phase on $Q$. Then $\omega(Q)=|P|+\frac{1}{\epsilon}(\ln{|H_P|} + ln{\frac{1}{\delta}})$.
Let $\parf_0(x)=\max\{0, ~\frac{1}{\epsilon}(\ln(x)-\ln(1/\delta))\}$, $\gamma(Q) = \Call{Prefix}{Q,\parf_0}$ and $S'\subset Q$ be the samples in sampling phase.

If $P$ is the samples for the best stopping point, then

$$
\omega(Q)=|P|+\frac{1}{\epsilon}(\ln{|H_P|} + ln{\frac{1}{\delta}})
$$

\noindent Case 1: \tool using $\parf_0$ is stopping earlier in phrase 1 than the best possible stopping point, $|S'| \leq |P|$
\begin{align*}
    \gamma(Q) - 2\cdot\omega(Q) &= |S'|- 2\cdot|P| +  \frac{1}{\epsilon}\cdot(\ln{|H_{S'}|} - 2\cdot \ln{|H_{P}|})\\
    &- \frac{1}{\epsilon}\cdot\ln{\frac{1}{\delta}}\\
    & \leq -|P| - \frac{1}{\epsilon}\cdot\ln{\frac{1}{\delta}} + \frac{1}{\epsilon}\cdot(\ln{|H_{S'}|})\\
    &\text{since }|S'| \leq |P|
\end{align*}
Observe that, for $\parf_0(x)=\max\{0, ~\frac{1}{\epsilon}(\ln(x)-\ln(1/\delta))\} $ the
$|S'| \geq \frac{1}{\epsilon}\cdot(\ln{|H_{S'}|}-\ln{\frac{1}{\delta}})$ (see, step $12$ in Algorithm $2$)
\begin{align*}
    \gamma(Q) - 2\cdot\omega(Q) \leq -|P| + |S'| \leq 0
\end{align*}

\noindent Case 2: \tool using $\parf_0$ is stopping after the best possible stopping point in phrase 1,  $|S'| > |P|$

In this case, $|H_{P}|\ge|H_{S'}|$. Observe that $|S'| \le \omega(Q)$. Because for \tool using $\parf_0$, after seeing $P$, it will take no more than $\parf_0(|H_{P}|) + 2k =\max\{0, \frac{1}{\epsilon}(\ln{|H_{P}|} - ln{\frac{1}{\delta}})\} + 2k \leq \frac{1}{\epsilon}(\ln{|H_P|} + ln{\frac{1}{\delta}})$ examples in phase 1.
\begin{align*}
    |S'| &\le |P| + 2k + \parf_0(|H_{P}|)\\
    \omega(Q) &= |P| + \frac{1}{\epsilon}(\ln{|H_{P}|} + ln{\frac{1}{\delta}})\\
    &\implies |S'| \le \omega(Q)
\end{align*}
Now,
\begin{align*}
    \gamma(Q) - 2\cdot\omega(Q) &= |S'| + \frac{1}{\epsilon}\cdot(\ln{|H_{S'}|} + \ln{\frac{1}{\delta}}) - 2\cdot\omega(Q)\\
    &\leq \frac{1}{\epsilon}\cdot(\ln{|H_{S'}|} + \ln{\frac{1}{\delta}}) - \omega(Q) \quad(\text{by } |S'| \le \omega(Q))\\
    &\leq -|P| +\frac{1}{\epsilon}\cdot(\ln{|H_{S'}|} - \ln{|H_{P}|})\\
    &\leq 0 \quad (\text{since, }|H_{P}|\ge|H_{S'}|)
\end{align*}
\end{proof}

\bibliographystyle{plainnat}
\bibliography{proofs}

\end{document}


\documentclass[11pt,answers]{exam}

% Preamble % (fold)

\usepackage[paper=letterpaper,margin=.75in,twoside=false,includehead]{geometry}

\usepackage{amsfonts,amsmath,amsthm,amssymb}
\usepackage{enumerate}
\usepackage{graphicx}

\usepackage{paralist}
\usepackage{multicol}
\usepackage{bm}

\let\svthefootnote\thefootnote

\newtheorem*{thm}{Theorem}

\newcommand{\Z}{\mathbb{Z}}
\newcommand{\E}{\mathbb{E}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\cP}{\mathcal{P}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\e}{\varepsilon}

%\printsolutions

\title{Communicating in Mathematics (MTH 210) Exam 1}
\date{February 12, 2020}

\begin{document}

\maketitle

\section*{Instructions}

\noindent Write your answers in the spaces provided. If you run out of space, continue your work on the back of the page and indicate that you have done so. If you still need additional space then additional sheets are available. If you use scrap paper you must turn in your scrap paper with the exam. \\

\begin{center}
\textbf{If you aren't sure what to do, take a deep breath and just show me what you know.}
\end{center}
\vspace{1in}

\begin{center}

\begin{tabular}{|c|c|c|}
\hline
Section &Score\\
\hline
Definitions and Notation &\\
&\\
\hline
Logic &\\
&\\
\hline
Proofs and Counterexamples &\\
&\\
\hline
Proof Section &\\
&\\
\hline
\end{tabular}
\end{center}

\vspace{1in}

\begin{center}
Name (print): \underline{\hspace{3in}}
\end{center}

\pagebreak

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Definitions and Notation}

\begin{questions}

\question Complete the following definition:
\begin{center}
A nonzero integer $a$ \emph{divides} an integer $b$, denoted $a\mid b$ provided that...
\end{center}
\vfill

\question Using the definition of divides, explain why, for any nonzero $a\in\Z$ we have that $a\mid 0$.
\vfill

\question Complete the following definition:
\begin{center}
Let $n\in\N$ and $a,b\in\Z$. Then $a\equiv b \pmod{n}$ provided that...
\end{center}

\vfill

%\question Give two integers $a$ and $b$ that are congruent modulo $7$. Briefly explain.
%\vfill

\question Let $A = \{a \in\Z \mid a\equiv 2 \pmod{3}\}$. Write $A$ in roster notation. Include at least 4 numbers in your roster notation. Include a negative number if it makes sense to.
\vfill

\question Fill in the blank with one of the symbols $\subseteq, =, \in, \not\subseteq, \not\in$:
	\begin{itemize}
	\item $1 \underline{\hspace{.5in}} \{1,2\}$\\

	\item $\{2\} \underline{\hspace{.5in}} \{1,2\}$
	\end{itemize}
\end{questions}
\newpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Logic}

\begin{questions}

\question For this question consider the following conditional statement:
	\begin{center}
	If $a$ is an irrational number and $b$ is an irrational number then $a\cdot b$ is an irrational number.
	\end{center}
	\begin{parts}
	\part State the converse of the statement above. If the original statement is true, must the converse be true?
	\vfill

	\part State the contrapositive of the statement above. If the original statement is true, must the contrapositive be true?
	\vfill

	\part State the negation of the statement above. (Don't worry about starting a sentence with a math symbol here.) If the original statement is true, what can you say about the truth value of the negation?
	\vfill

	\end{parts}

\question Suppose that Sally made the following claim:
	\begin{center}
	If it snows then I will go skiing.
	\end{center}

You find out that it did not snow, but Sally still went skiing. Is Sally's statement true or false? Justify your answer.
\vfill
\end{questions}

\newpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Proofs and Counterexamples}

\begin{questions}
\question Consider the following theorem:
	\begin{thm}
	For every integer $z$, if $8\mid z-4$ then $8\nmid z^2 - 1$.
	\end{thm}

	\begin{parts}
	\part If you were going to do a direct proof to prove the given statement is true, what would you assume? What would you try to show? \emph{Do not actually attempt to prove the statement.}

	Assume:
	\vfill

	Show:
	\vfill

	\part If you were going to do a proof by contrapositive to prove the given statement is true, shat would you assume? What would you try to show?

	Assume:
	\vfill

	Show:
	\vfill

	\part What would you have to demonstrate in order to prove the statement is false? Be specific, but \emph{do not actually attempt to prove the statement is false.}
	\vfill

		\end{parts}

\end{questions}

\newpage

\section{Proofs}

IMPORTANT DIRECTIONS: You need to do both of the following proofs. Each proof needs to be written according to our writing guidelines.  There next page is for the first proof (which you can just attach if you already have it) and the page after is for the second proof. You may use this page (and the backs of pages) for scratch work.

\begin{enumerate}
\item Prove the following theorem. If you did this proof before class you may just attach your work. The proof needs to be written according to our writing guidelines.

\begin{center}
For all integers $m$, if $m\equiv 1 \pmod{3}$ then $3m^2+7m+12\equiv 1 \pmod{3}$.
\end{center}

\item Prove the following theorem. You should write your proof according to our writing guidelines.

\begin{center}
For all integers $n$, $n^2$ is odd if and only if $n$ is odd.
\end{center}

\emph{Use the rest of this page for scratch work.}

\end{enumerate}

\newpage

\emph{Write your proof for 1 on this page. Make sure to include a theorem statement.}

\newpage

\emph{Write your proof for 1 on this page. Make sure to include a theorem statement.}

\end{document}


\chapter{Background}
\index{Background\emph{Background}}%

Mathematical Expressions MEs form an essential part of scientific
and technical documents. Mathematical Expressions can be typeset or
handwritten which uses two dimensional arrangements of symbols to
transmit information. Recognizing both form of mathematical
expressions are challenging. A variation to handwritten ME is
cursive handwriting. Unconstrained cursive property of such
handwritten expressions poses a major challenge to its recognition.

Generally speaking understanding and recognizing mathematical
expression, whether typeset or handwritten, involves three
activities: Expression localization, symbol recognition and
symbol-arrangement analysis. ME localization involves finding and
extracting mathematical expression from the document. Symbol
recognition converts the extracted expression image into a set of
symbols and symbol arrangement analyzes the spatial arrangement of
set of symbols to recover the information content of the given
mathematical notations.

Now based on the recognition process, symbol recognition activity
can further subdivided as 1) preprocessing - noise reduction,
deskewing, slant correction etc, 2) segmentation to isolate symbols
3) and finally, recognition. Similarly depending upon the
symbol-arrangement algorithm, symbol arrangement analysis can be
further subdivided into a) identification of spatial relationships
among symbols b) identification of logical relationships among
symbols 3) construction of meaning. These processes can be executed
in series or in parallel with latter processes providing contextual
feedback for the earlier processes. The order of these recognition
activities can vary somewhat, for example, partial identification of
spatial and logical relationships can be performed prior to symbol
recognition.

\section{Preprocessing}
\index{Preprocessing@\emph{Preprocessing}}%

Preprocessing is required to eliminate irregularities and noise from
the image, especially in handwritten character recognition. Certain
preprocessing method requirements may depend upon the techniques
used for recognition. \cite{gyeonghwan1997lda} uses chain code
method for handwritten image representation. Preprocessing involves
slant angle correction in which global slant angle from different
vertical lines is estimated and tangent of the estimated global
slant angle is used to correct for slant. Smoothing of image
involves elimination of small blobs (noise) on the contour. A
sliding 3-component one dimensional window is applied overall
components during which components are removed or added based on the
orientation of components. Average stroke width is estimated by
dividing chain code contours horizontally and by tracing left to
right various distances between outer and inner contour.
\cite{jcai1999issi} performs size normalization to reduce variation
in character size. To avoid significant deformation due to directly
scaling of all images to identical size, a holistic approach is used
for scaling in which if width/height ratio is less than 0.8 then
scale is identical horizontally and vertically otherwise the scale
factor is set to 0.8 to prevent large variation in image width.

\section{Character segmentation}
\index{Character segmentation@\emph{Character segmentation}}%

Character segmentation, next step in ME recognition, has long been a
critical area of OCR process. Depending upon the requirement,
character segmentation techniques is divided into four major
headings \cite{CaseyLecolinet1996}. Classical approach of
segmentation also called dissection technique consists of
partitioning the input image into sub-images based on their inherent
features, which are then classified. Another approach to
segmentation is a group of techniques that avoids dissection and
segments to image either explicitly by classification of
pre-specified windows, or implicitly by classification of subsets of
spatial features collected from the image as a whole. Another
approach is a hybrid approach employing dissection but using
classification to select from admissible segmentation possibility.
Finally holistic approach avoids segmentation process itself and
performs recognition entire character strings.

Various techniques have been used for segmentation that involves
dissection. White spaces between the characters are used to detect
segmentation points. Pitch which is the number of characters per
unit of horizontal distance provides a basis for estimating
segmentation points. The segmentation points obtained for a given
line should be approximately equally spaced at the distance that
corresponds to pitch \cite{CaseyLecolinet1996}.

Inter-character boundaries can be obtained if most segmentation
takes place by finding columns of white. Now all segmentation points
that do not lie near these boundaries can be rejected as caused due
to broken characters. Similarly we can estimate missed points due to
merged characters. Hoffman and McCullough gave a framework for
segmentation that involves three steps i.e. 1) Detection of the
start of the character, 2) A decision to begin testing for the end
of a character called sectioning, 3) Detection of end-of-character.
Sectioning is done by weighted analysis of horizontal black runs
completed versus run still incomplete.  Once sectioning determines
the regions of segmentation, rules were invoked to segment based on
either an increase in bit density or the use of special features
designed to detect end-of-character.

In \cite{arica2002ocr}, segmentation in cursive handwritten
characters is performed in the binary word image by using the
contour of the writing. Determination of segmentation regions is
done in three steps. In first step a straight line is drawn in the
slant angle direction from each local maximum until the top of the
word image. While going upward in the slant direction, if any
contour pixel is hit, this contour is followed until the slope of
the contour changes to the opposite direction. An abrupt change in
the slope of the contour indicates an end point. A line is drawn
from the maximum to the end point and path continues to go upward in
slant direction until the top of the word image. In step 2, a path
in the slant direction from each maximum to the lower baseline, is
drawn. Step 3 follows the same process as in step 1 in order to
determine the path from lower baseline to the bottom of the word
image. Combining all the three steps gives the segmentation regions.
In \cite{gyeonghwan1997lda} segmentation involves detecting
ligatures as segmentation points in cursive scripts. Alternatively,
concavity features in the upper contour and convexities in the lower
contour are used in conjunction with ligatures to reduce the number
of potentials segmentation points.

Another dissection technique that applies to non-cursive characters
is bounding box technique \cite{CaseyLecolinet1996}. In this
analysis, the adjacency relationships between characters are tested
to perform merging or their size or aspect ratios are calculated to
trigger splitting mechanisms.  Another involves splitting of
connected components. Connected components are merged or split
according to rules based on height and width of the bounding boxes.
Intersection of two characters can give rise to special image
features and different dissection methods have been developed to
detect these features and to use them in splitting a character
string images into sub-images.

\cite{chen2000sso} focuses on segmentation of single and multiple
touching character segmentation. \cite{chen2000sso} proposes a new
technique that links the feature points on the foreground and
background alternately to get the possible segmentation path.
Mixture Gaussian probability function is determined and used to rank
all the possible segmentation paths. Segmentation paths construction
is performed separately for single touching characters and for
multiple touching characters. All the paths from to two analysis are
collectively processed to remove useless strokes and then mixture
Gaussian probability function is applied to decide which on is the
best segmentation path.

Another kind of approach to character segmentation is recognition
based approach. In these segmentation processes letter segmentation
is a by-product of letter recognition. The basic principle is use a
mobile window of variable width to provide sequences of tentative
segmentation which are confirmed (or not) by character recognition.
A technique called Shortest Path Segmentation selects the optimal
combination of cuts from the predefined set of candidate cuts that
construct all possible legal segments through combination. A graph
whose nodes represent acceptable segments is the created. The paths
of these graphs represent all legal segmentations of the word. Each
node of the graph is then assigned a distance obtained by the neural
net recognizer. The shortest path though the graph thus corresponds
to the best recognition and segmentation of the word. An alternative
method attempts to match subgraphs of features with predefined
character prototypes. Different alternative are represented by a
directed network whose nodes correspond to the matched subgraphs.
Word recognition is done by searching for the path that gives the
best interpretation of the word features.

\section{Symbol-Arrangement Analysis}
\index{Symbol-Arrangement Analysis@\emph{Symbol-Arrangement Analysis}}%

One approach to symbol-arrangement analysis is syntactic approach.
Syntactic approach makes use of two dimensional grammar rules to
define the correct grouping of math symbols. Co-ordinate grammar for
recognition is presented by Anderson. The grammar specifies
syntactic rules that subdivide the set of symbols into several
subsets, each with its own syntactic subgoal. The final
interpretation result is given by the m attribute of the grammar
start's symbol where m represents ASCII encoding of the meaning of
symbol-set. Although coordinate grammar provides a clear and well
structured recognition approach, its slow parsing speed and
difficulty to handle errors are its major drawbacks. In [8], a
syntactic approach is adopted in which a system consisting of
hierarchy of parsers for the interpretation of 2-D mathematical
formulas is described. The ME interpreter consists of two syntactic
parser top-down and bottom-up. It starts with a priority operator in
the expression to be analyzed and tries to divide it into
sub-expressions or operands which are then analyzed in the same way
and so on. The bottom-up parser chooses from the starting character
and from the neighboring sub-expressions the corresponding rule in
the grammar. This rule gives instructions to the top-down parser to
delimit the zones of neighboring operands and operators.

Garain and Chaudhari in \cite{garain2004roh}, proposes a two pass
approach to determine arrangement of symbols. The first pass is a
scanning or lexicon analysis that performs micro-level examination
of the symbols to determine the symbol groups and to determine their
categories or descriptors. The second pass is parsing or syntax
analysis that processes the descriptors synthesized in the first
pass to determine the syntactical structure of the expression. A set
of predefined rules guides the activities in both the passes.

Another symbol-arrangement analysis approach is projection profile
cutting. It involves recursive projection-profile cutting. Cutting
by the vertical projection profile is attempted first, followed by
horizontal cuts for each resulting regions. The process repeats
until no further cutting is possible. The resulting spatial
relationships are represented by a tree structure.  Although the
method looks simple and efficient technique, it is still under study
and also involves additional processing for symbols like square
root, subscripts and superscripts as these can be handled by
projection profile cut.

Another approach discussed is the Graph Rewriting. Graph rewriting
involves information represented as an attributed graph and the
graph get updated through the application of graph-rewriting rules.
An initial graph contains one node to represent each symbol, with
nodes attributes recording the spatial coordinates of the symbol.
Graph rewriting rules are applied to add edges representing
meaningful spatial relationships. Rules are further applied to prune
or modify these edges identifying logical relationships from the
spatial relationships. In [7], Ann Grbavec and Dorothea Blostein
proposed a novel-graph rewriting techniques that addresses the
recursive structure of mathematical notations, the critical
dependence of the meaning upon operator precedence and the presence
of ambiguities that depends upon global context. The recognition
system proposed called EXPRESSO, is based on
Build-Constrain-Rank-Incorporate model where the Build phase
constructs edges to represent potentially meaningful spatial
relation- ships between symbols. The Constrain phase applies
information about the notational conventions of mathematics to
remove contra- dictions and resolve ambiguities. The Rank phase uses
information about the operator precedence to group symbols into
sub-expressions and the Incorporate phase interprets
sub-expressions.

Twaakyondo and Okamoto \cite{twaakyondo1995saa} discuss two basic
strategies to decide the layout of structure of the given
expression. One strategy is to check the local structures of the
sub-expressions using a bottom-up method (specific structure
processing). It is used to analyze nested structures like
subscripts, superscripts and root expressions. The other strategy is
to check the global structure of the whole expression by a top-down
method (fundamental structure processing). It is used to analyze the
horizontal and vertical relations between sub-expressions. The
structure of the expression is represented as a tree structure.

Chou in [11] proposed a two-dimensional stochastic context-free
grammar for recognition of printed mathematical expressions. The
recognized symbols are parsed with the grammar in which each
production rule has an associated probability. The main task of the
process is to find the most probable parse tree for the input
expression. The overall probability of a parse tree is computed by
multiplying together the probabilities for all the production rules
used in a successful parse.

\section{Conclusion}
\index{Conclusion@\emph{Conclusion}}%

As we saw through the survey, there have been tremendous advances in
the field of character recognition from so many years of research.
Some experiment tried to focus on one activity of recognition
process while other tried to build a complete system for character
recognition. Some researchers assumed complete well recognized
symbols are given and they focus on the symbol-arrangement
(structural) analysis of the recognized symbols. This survey
concentrated mainly on the two activities of character recognition
i.e. segmentation of symbols and symbol-arrangement of recognized
symbols.

Segmentation processes discussed have some limitations such as some
are restricted to be applied to cursive handwriting while other
focuses on non-cursive handwriting. Some researchers focus on
certain subset of mathematical symbols because of large mathematical
symbol set. Some concentrate on single touching characters some on
multiple touching characters. Certain approaches of segmentation
like holistic approach that recognizes entire word as a unit have
drawback of being restricted to predefined lexicons. Hence more
efficient and robust segmentation process is required as further
analysis of ME recognition depends on segmentation and recognition
of symbols.

Symbol-arrangement analysis discussed shows wide variations in
approaches. Some approach exploits the operator precedence property
of mathematical expression while some performs different level of
analysis (lexicon and syntax) to first group symbols into different
categories and then perform structural analysis using predefined
rules. Some using graph rewriting technique in which mathematical
symbols are linked to each other through graph rewriting rules. Some
use stochastic grammar rules to represent to the relationship
between symbols while some intelligently looks for local structures
of the expression to determine the features like nested, above or
below followed by global analysis to check for the correctness of
the expression as a whole and rectify wrong arrangements of symbols.
Symbol arrangement analysis may be not so crucial for problems that
involve only standard English character but problems like
recognition of mathematical expressions where the actual position
and location of symbols is important and there are many implicit
meaning to symbols which depends on their arrangement, it is
absolutely important to perform symbol arrangement analysis for
better recognition result.


\vfill \eject
\section{{\tt allInOne.c} -- A Serial $LU$ Driver Program}
\label{section:LU-MPI-driver}

\begin{verbatim}
/*  allInOneMPI.c  */

#include "../spoolesMPI.h"
#include "../../timings.h"

/*--------------------------------------------------------------------*/
int
main ( int argc, char *argv[] ) {
/*
   ------------------------------------------------------------
   all-in-one MPI program for each process

   order, factor and solve A X = Y

   ( 1) read in matrix entries and form InpMtx object for A
   ( 2) order the system using minimum degree
   ( 3) permute the front tree
   ( 4) create the owners map IV object
   ( 5) permute the matrix A and redistribute
   ( 6) compute the symbolic factorization
   ( 7) compute the numeric factorization
   ( 8) split the factors into submatrices
   ( 9) create the submatrix map and redistribute
   (10) read in right hand side entries
        and form dense matrix DenseMtx object for Y
   (11) permute and redistribute Y
   (12) solve the linear system
   (13) gather X on processor 0

   created -- 98jun13, cca
   ------------------------------------------------------------
*/
/*--------------------------------------------------------------------*/
char            buffer[20] ;
Chv             *rootchv ;
ChvManager      *chvmanager ;
DenseMtx        *mtxX, *mtxY, *newY ;
SubMtxManager   *mtxmanager, *solvemanager ;
FrontMtx        *frontmtx ;
InpMtx          *mtxA, *newA ;
double          cutoff, droptol = 0.0, minops, tau = 100. ;
double          cpus[20] ;
double          *opcounts ;
DV              *cumopsDV ;
ETree           *frontETree ;
FILE            *inputFile, *msgFile ;
Graph           *graph ;
int             error, firsttag, ient, irow, jcol, lookahead = 0,
                msglvl, myid, nedges, nent, neqns, nmycol, nproc, nrhs,
                nrow, pivotingflag, root, seed, symmetryflag, type ;
int             stats[20] ;
int             *rowind ;
IV              *oldToNewIV, *ownedColumnsIV, *ownersIV,
                *newToOldIV, *vtxmapIV ;
IVL             *adjIVL, *symbfacIVL ;
SolveMap        *solvemap ;
/*--------------------------------------------------------------------*/
/*
   ---------------------------------------------------------------
   find out the identity of this process and the number of process
   ---------------------------------------------------------------
*/
MPI_Init(&argc, &argv) ;
MPI_Comm_rank(MPI_COMM_WORLD, &myid) ;
MPI_Comm_size(MPI_COMM_WORLD, &nproc) ;
/*--------------------------------------------------------------------*/
/*
   --------------------
   get input parameters
   --------------------
*/
if ( argc != 7 ) {
   fprintf(stdout,
      "\n usage: %s msglvl msgFile type symmetryflag pivotingflag seed"
      "\n    msglvl -- message level"
      "\n    msgFile -- message file"
      "\n    type    -- type of entries"
      "\n      1 (SPOOLES_REAL)    -- real entries"
      "\n      2 (SPOOLES_COMPLEX) -- complex entries"
      "\n    symmetryflag -- type of matrix"
      "\n      0 (SPOOLES_SYMMETRIC)    -- symmetric entries"
      "\n      1 (SPOOLES_HERMITIAN)    -- Hermitian entries"
      "\n      2 (SPOOLES_NONSYMMETRIC) -- nonsymmetric entries"
      "\n    pivotingflag -- type of pivoting"
      "\n      0 (SPOOLES_NO_PIVOTING) -- no pivoting used"
      "\n      1 (SPOOLES_PIVOTING)    -- pivoting used"
      "\n    seed -- random number seed"
      "\n    "
      "\n   note: matrix entries are read in from matrix.k.input"
      "\n         where k is the process number"
      "\n   note: rhs entries are read in from rhs.k.input"
      "\n         where k is the process number"
      "\n", argv[0]) ;
   return(0) ;
}
msglvl = atoi(argv[1]) ;
if ( strcmp(argv[2], "stdout") == 0 ) {
   msgFile = stdout ;
} else {
   sprintf(buffer, "res.%d", myid) ;
   if ( (msgFile = fopen(buffer, "w")) == NULL ) {
      fprintf(stderr, "\n fatal error in %s"
              "\n unable to open file %s\n",
              argv[0], buffer) ;
      return(-1) ;
   }
}
type         = atoi(argv[3]) ;
symmetryflag = atoi(argv[4]) ;
pivotingflag = atoi(argv[5]) ;
seed         = atoi(argv[6]) ;
IVzero(20, stats) ;
DVzero(20, cpus) ;
/*--------------------------------------------------------------------*/
/*
   --------------------------------------------
   STEP 1: read the entries from the input file
           and create the InpMtx object
   --------------------------------------------
*/
sprintf(buffer, "matrix.%d.input", myid) ;
inputFile = fopen(buffer, "r") ;
fscanf(inputFile, "%d %d %d", &neqns, &neqns, &nent) ;
mtxA = InpMtx_new() ;
InpMtx_init(mtxA, INPMTX_BY_ROWS, type, nent, 0) ;
if ( type == SPOOLES_REAL ) {
   double   value ;
   for ( ient = 0 ; ient < nent ; ient++ ) {
      fscanf(inputFile, "%d %d %le", &irow, &jcol, &value) ;
      InpMtx_inputRealEntry(mtxA, irow, jcol, value) ;
   }
} else if ( type == SPOOLES_COMPLEX ) {
   double   imag, real ;
   for ( ient = 0 ; ient < nent ; ient++ ) {
      fscanf(inputFile, "%d %d %le %le", &irow, &jcol, &real, &imag) ;
      InpMtx_inputComplexEntry(mtxA, irow, jcol, real, imag) ;
   }
}
fclose(inputFile) ;
InpMtx_sortAndCompress(mtxA) ;
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n input matrix") ;
   InpMtx_writeForHumanEye(mtxA, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ----------------------------------------------------
   STEP 2: read the rhs entries from the rhs input file
   and create the DenseMtx object for Y
   ----------------------------------------------------
*/
sprintf(buffer, "rhs.%d.input", myid) ;
inputFile = fopen(buffer, "r") ;
fscanf(inputFile, "%d %d", &nrow, &nrhs) ;
mtxY = DenseMtx_new() ;
DenseMtx_init(mtxY, type, 0, 0, nrow, nrhs, 1, nrow) ;
DenseMtx_rowIndices(mtxY, &nrow, &rowind) ;
if ( type == SPOOLES_REAL ) {
   double   value ;
   for ( irow = 0 ; irow < nrow ; irow++ ) {
      fscanf(inputFile, "%d", rowind + irow) ;
      for ( jcol = 0 ; jcol < nrhs ; jcol++ ) {
         fscanf(inputFile, "%le", &value) ;
         DenseMtx_setRealEntry(mtxY, irow, jcol, value) ;
      }
   }
} if ( type == SPOOLES_COMPLEX ) {
   double   imag, real ;
   for ( irow = 0 ; irow < nrow ; irow++ ) {
      fscanf(inputFile, "%d", rowind + irow) ;
      for ( jcol = 0 ; jcol < nrhs ; jcol++ ) {
         fscanf(inputFile, "%le %le", &real, &imag) ;
         DenseMtx_setComplexEntry(mtxY, irow, jcol, real, imag) ;
      }
   }
}
fclose(inputFile) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n rhs matrix in original ordering") ;
   DenseMtx_writeForHumanEye(mtxY, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -------------------------------------------------------
   STEP 2 : find a low-fill ordering
   (1) create the Graph object
   (2) order the graph using multiple minimum degree
   (3) find out who has the best ordering w.r.t. op count,
       and broadcast that front tree object
   -------------------------------------------------------
*/
graph = Graph_new() ;
adjIVL = InpMtx_MPI_fullAdjacency(mtxA, stats,
                                  msglvl, msgFile, MPI_COMM_WORLD) ;
nedges = IVL_tsize(adjIVL) ;
Graph_init2(graph, 0, neqns, 0, nedges, neqns, nedges, adjIVL,
            NULL, NULL) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n graph of the input matrix") ;
   Graph_writeForHumanEye(graph, msgFile) ;
   fflush(msgFile) ;
}
frontETree = orderViaMMD(graph, seed + myid, msglvl, msgFile) ;
Graph_free(graph) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n front tree from ordering") ;
   ETree_writeForHumanEye(frontETree, msgFile) ;
   fflush(msgFile) ;
}
opcounts = DVinit(nproc, 0.0) ;
opcounts[myid] = ETree_nFactorOps(frontETree, type, symmetryflag) ;
MPI_Allgather((void *) &opcounts[myid], 1, MPI_DOUBLE,
              (void *) opcounts, 1, MPI_DOUBLE, MPI_COMM_WORLD) ;
minops = DVmin(nproc, opcounts, &root) ;
DVfree(opcounts) ;
frontETree = ETree_MPI_Bcast(frontETree, root,
                             msglvl, msgFile, MPI_COMM_WORLD) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n best front tree") ;
   ETree_writeForHumanEye(frontETree, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -------------------------------------------------------
   STEP 3: get the permutations, permute the front tree,
           permute the matrix and right hand side.
   -------------------------------------------------------
*/
oldToNewIV = ETree_oldToNewVtxPerm(frontETree) ;
newToOldIV = ETree_newToOldVtxPerm(frontETree) ;
ETree_permuteVertices(frontETree, oldToNewIV) ;
InpMtx_permute(mtxA, IV_entries(oldToNewIV), IV_entries(oldToNewIV)) ;
if (  symmetryflag == SPOOLES_SYMMETRIC
   || symmetryflag == SPOOLES_HERMITIAN ) {
   InpMtx_mapToUpperTriangle(mtxA) ;
}
InpMtx_changeCoordType(mtxA, INPMTX_BY_CHEVRONS) ;
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
DenseMtx_permuteRows(mtxY, oldToNewIV) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n rhs matrix in new ordering") ;
   DenseMtx_writeForHumanEye(mtxY, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -------------------------------------------
   STEP 4: generate the owners map IV object
           and the map from vertices to owners
   -------------------------------------------
*/
cutoff   = 1./(2*nproc) ;
cumopsDV = DV_new() ;
DV_init(cumopsDV, nproc, NULL) ;
ownersIV = ETree_ddMap(frontETree,
                       type, symmetryflag, cumopsDV, cutoff) ;
DV_free(cumopsDV) ;
vtxmapIV = IV_new() ;
IV_init(vtxmapIV, neqns, NULL) ;
IVgather(neqns, IV_entries(vtxmapIV),
         IV_entries(ownersIV), ETree_vtxToFront(frontETree)) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n map from fronts to owning processes") ;
   IV_writeForHumanEye(ownersIV, msgFile) ;
   fprintf(msgFile, "\n\n map from vertices to owning processes") ;
   IV_writeForHumanEye(vtxmapIV, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ---------------------------------------------------
   STEP 5: redistribute the matrix and right hand side
   ---------------------------------------------------
*/
firsttag = 0 ;
newA = InpMtx_MPI_split(mtxA, vtxmapIV, stats,
                        msglvl, msgFile, firsttag, MPI_COMM_WORLD) ;
firsttag++ ;
InpMtx_free(mtxA) ;
mtxA = newA ;
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n split InpMtx") ;
   InpMtx_writeForHumanEye(mtxA, msgFile) ;
   fflush(msgFile) ;
}
newY = DenseMtx_MPI_splitByRows(mtxY, vtxmapIV, stats, msglvl,
                                msgFile, firsttag, MPI_COMM_WORLD) ;
DenseMtx_free(mtxY) ;
mtxY = newY ;
firsttag += nproc ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n split DenseMtx Y") ;
   DenseMtx_writeForHumanEye(mtxY, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ------------------------------------------
   STEP 6: compute the symbolic factorization
   ------------------------------------------
*/
symbfacIVL = SymbFac_MPI_initFromInpMtx(frontETree, ownersIV, mtxA,
                     stats, msglvl, msgFile, firsttag, MPI_COMM_WORLD) ;
firsttag += frontETree->nfront ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n local symbolic factorization") ;
   IVL_writeForHumanEye(symbfacIVL, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -----------------------------------
   STEP 7: initialize the front matrix
   -----------------------------------
*/
mtxmanager = SubMtxManager_new() ;
SubMtxManager_init(mtxmanager, NO_LOCK, 0) ;
frontmtx = FrontMtx_new() ;
FrontMtx_init(frontmtx, frontETree, symbfacIVL, type, symmetryflag,
              FRONTMTX_DENSE_FRONTS, pivotingflag, NO_LOCK, myid,
              ownersIV, mtxmanager, msglvl, msgFile) ;
/*--------------------------------------------------------------------*/
/*
   ---------------------------------
   STEP 8: compute the factorization
   ---------------------------------
*/
chvmanager = ChvManager_new() ;
ChvManager_init(chvmanager, NO_LOCK, 0) ;
rootchv = FrontMtx_MPI_factorInpMtx(frontmtx, mtxA, tau, droptol,
                     chvmanager, ownersIV, lookahead, &error, cpus,
                     stats, msglvl, msgFile, firsttag, MPI_COMM_WORLD) ;
ChvManager_free(chvmanager) ;
firsttag += 3*frontETree->nfront + 2 ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n numeric factorization") ;
   FrontMtx_writeForHumanEye(frontmtx, msgFile) ;
   fflush(msgFile) ;
}
if ( error >= 0 ) {
   fprintf(stderr,
          "\n proc %d : factorization error at front %d", myid, error) ;
   MPI_Finalize() ;
   exit(-1) ;
}
/*--------------------------------------------------------------------*/
/*
   ------------------------------------------------
   STEP 9: post-process the factorization and split
   the factor matrices into submatrices
   ------------------------------------------------
*/
FrontMtx_MPI_postProcess(frontmtx, ownersIV, stats, msglvl,
                         msgFile, firsttag, MPI_COMM_WORLD) ;
firsttag += 5*nproc ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n numeric factorization after post-processing");
   FrontMtx_writeForHumanEye(frontmtx, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   -----------------------------------
   STEP 10: create the solve map object
   -----------------------------------
*/
solvemap = SolveMap_new() ;
SolveMap_ddMap(solvemap, symmetryflag,
               FrontMtx_upperBlockIVL(frontmtx),
               FrontMtx_lowerBlockIVL(frontmtx),
               nproc, ownersIV, FrontMtx_frontTree(frontmtx),
               seed, msglvl, msgFile);
if ( msglvl > 3 ) {
   SolveMap_writeForHumanEye(solvemap, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ----------------------------------------------------
   STEP 11: redistribute the submatrices of the factors
   ----------------------------------------------------
*/
FrontMtx_MPI_split(frontmtx, solvemap,
                   stats, msglvl, msgFile, firsttag, MPI_COMM_WORLD) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n numeric factorization after split") ;
   FrontMtx_writeForHumanEye(frontmtx, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ------------------------------------------------
   STEP 13: permute and redistribute Y if necessary
   ------------------------------------------------
*/
if ( FRONTMTX_IS_PIVOTING(frontmtx) ) {
   IV   *rowmapIV ;
/*
   ----------------------------------------------------------
   pivoting has taken place, redistribute the right hand side
   to match the final rows and columns in the fronts
   ----------------------------------------------------------
*/
   rowmapIV = FrontMtx_MPI_rowmapIV(frontmtx, ownersIV, msglvl,
                                    msgFile, MPI_COMM_WORLD) ;
   newY = DenseMtx_MPI_splitByRows(mtxY, rowmapIV, stats, msglvl,
                                   msgFile, firsttag, MPI_COMM_WORLD) ;
   DenseMtx_free(mtxY) ;
   mtxY = newY ;
   IV_free(rowmapIV) ;
}
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n rhs matrix after split") ;
   DenseMtx_writeForHumanEye(mtxY, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   ------------------------------------------
   STEP 14: create a solution DenseMtx object
   ------------------------------------------
*/
ownedColumnsIV = FrontMtx_ownedColumnsIV(frontmtx, myid, ownersIV,
                                         msglvl, msgFile) ;
nmycol = IV_size(ownedColumnsIV) ;
mtxX = DenseMtx_new() ;
if ( nmycol > 0 ) {
   DenseMtx_init(mtxX, type, 0, 0, nmycol, nrhs, 1, nmycol) ;
   DenseMtx_rowIndices(mtxX, &nrow, &rowind) ;
   IVcopy(nmycol, rowind, IV_entries(ownedColumnsIV)) ;
}
/*--------------------------------------------------------------------*/
/*
   --------------------------------
   STEP 15: solve the linear system
   --------------------------------
*/
solvemanager = SubMtxManager_new() ;
SubMtxManager_init(solvemanager, NO_LOCK, 0) ;
FrontMtx_MPI_solve(frontmtx, mtxX, mtxY, solvemanager, solvemap, cpus,
                   stats, msglvl, msgFile, firsttag, MPI_COMM_WORLD) ;
SubMtxManager_free(solvemanager) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n solution in new ordering") ;
   DenseMtx_writeForHumanEye(mtxX, msgFile) ;
}
/*--------------------------------------------------------------------*/
/*
   --------------------------------------------------------
   STEP 15: permute the solution into the original ordering
            and assemble the solution onto processor zero
   --------------------------------------------------------
*/
DenseMtx_permuteRows(mtxX, newToOldIV) ;
if ( msglvl > 2 ) {
   fprintf(msgFile, "\n\n solution in old ordering") ;
   DenseMtx_writeForHumanEye(mtxX, msgFile) ;
   fflush(msgFile) ;
}
IV_fill(vtxmapIV, 0) ;
firsttag++ ;
mtxX = DenseMtx_MPI_splitByRows(mtxX, vtxmapIV, stats, msglvl, msgFile,
                                firsttag, MPI_COMM_WORLD) ;
if ( myid == 0 && msglvl > 0 ) {
   fprintf(msgFile, "\n\n complete solution in old ordering") ;
   DenseMtx_writeForHumanEye(mtxX, msgFile) ;
   fflush(msgFile) ;
}
/*--------------------------------------------------------------------*/
MPI_Finalize() ;

return(1) ; }
/*--------------------------------------------------------------------*/
\end{verbatim}


%%^^A%% um-doc-legacyfontdimen.tex -- part of UNICODE-MATH <wspr.io/unicode-math>

\section{Legacy \TeX\ font dimensions}

\centerline{%
\begin{tabular}[t]{@{}lp{4cm}@{}}
\toprule
\multicolumn{2}{@{}c@{}}{Text fonts} \\
\midrule
$\phi_1$ & slant per pt                \\
$\phi_2$ & interword space             \\
$\phi_3$ & interword stretch           \\
$\phi_4$ & interword shrink            \\
$\phi_5$ & x-height                    \\
$\phi_6$ & quad width                  \\
$\phi_7$ & extra space                 \\
$\phi_8$ & cap height (\XeTeX\ only)   \\
\bottomrule
\end{tabular}
\quad
\begin{tabular}[t]{@{}lp{4cm}@{}}
\toprule
\multicolumn{2}{@{}c@{}}{Maths font, \cs{fam}2} \\
\midrule
$\sigma_5$    & x height                    \\
$\sigma_6$    & quad                        \\
$\sigma_8$    & num1                        \\
$\sigma_9$    & num2                        \\
$\sigma_{10}$ & num3                        \\
$\sigma_{11}$ & denom1                      \\
$\sigma_{12}$ & denom2                      \\
$\sigma_{13}$ & sup1                        \\
$\sigma_{14}$ & sup2                        \\
$\sigma_{15}$ & sup3                        \\
$\sigma_{16}$ & sub1                        \\
$\sigma_{17}$ & sub2                        \\
$\sigma_{18}$ & sup drop                    \\
$\sigma_{19}$ & sub drop                    \\
$\sigma_{20}$ & delim1                      \\
$\sigma_{21}$ & delim2                      \\
$\sigma_{22}$ & axis height                 \\
\bottomrule
\end{tabular}
\quad
\begin{tabular}[t]{@{}lp{4cm}@{}}
\toprule
\multicolumn{2}{@{}c@{}}{Maths font, \cs{fam}3} \\
\midrule
$\xi_8$    & default rule thickness      \\
$\xi_9$    & big op spacing1             \\
$\xi_{10}$ & big op spacing2             \\
$\xi_{11}$ & big op spacing3             \\
$\xi_{12}$ & big op spacing4             \\
$\xi_{13}$ & big op spacing5             \\
\bottomrule
\end{tabular}
}

\endinput

% /©
%
% ------------------------------------------------
% The UNICODE-MATH package  <wspr.io/unicode-math>
% ------------------------------------------------
% This package is free software and may be redistributed and/or modified under
% the conditions of the LaTeX Project Public License, version 1.3c or higher
% (your choice): <http://www.latex-project.org/lppl/>.
% ------------------------------------------------
% Copyright 2006-2019  Will Robertson, LPPL "maintainer"
% Copyright 2010-2017  Philipp Stephani
% Copyright 2011-2017  Joseph Wright
% Copyright 2012-2015  Khaled Hosny
% ------------------------------------------------
%
% ©/


\documentclass[]{scrartcl}

\usepackage{amsmath}
\usepackage{bussproofs}
\usepackage{color}
\usepackage{listings}
\usepackage{hyperref}
\usepackage{url}

\EnableBpAbbreviations

\newcommand{\WHILE}[2]{\ensuremath{\mathbf{while}\;#1\;\mathbf{do}\;\{#2\}}}
\newcommand{\IWHILE}[3]{\ensuremath{\mathbf{inv}\;#1\;\mathbf{while}\;#2\;\mathbf{do}\;\{#3\}}}
\newcommand{\IF}[3]{\ensuremath{\mathbf{if}\;#1\;\mathbf{then}\;\{#2\}\;\mathbf{else}\;\{#3\}}}
\newcommand{\SKIP}{\ensuremath{\mathbf{skip};}}
\newcommand{\ASSUME}[1]{\ensuremath{\mathbf{assume}\;#1;}}
\newcommand{\ASSERT}[1]{\ensuremath{\mathbf{assert}\;#1;}}
\newcommand{\WLP}[2]{\ensuremath{\mathbf{wlp}\;#1\;#2}}
\newcommand{\REPBY}[2]{\ensuremath{#1\;\mathbf{repby}\;#2}}
\newcommand{\UNROLLSQ}[3]{\ensuremath{[\mathit{while}]^{#1}(#2, #3)}}
\newcommand{\UNROLLDI}[3]{\ensuremath{\langle\mathit{while}\rangle^{#1}(#2, #3)}}
\lstset{
  frame=none,
  xleftmargin=2pt,
  stepnumber=1,
  numbers=left,
  numbersep=5pt,
  numberstyle=\ttfamily\tiny\color[gray]{0.3},
  belowcaptionskip=\bigskipamount,
  captionpos=b,
  escapeinside={*'}{'*},
  language=haskell,
  tabsize=2,
  emphstyle={\bf},
  commentstyle=\it,
  stringstyle=\mdseries\rmfamily,
  showspaces=false,
  keywordstyle=\bfseries\rmfamily,
  columns=flexible,
  basicstyle=\small\sffamily,
  showstringspaces=false,
  morecomment=[l]\%,
}

%opening
\title{A Program Verification Engine for Imperative Languages}
\subtitle{Course: Program Verification}
\author{Giovanni Garufi, Fabian Thorand}
\date{\today}

\newcommand{\blue}[1]{\textcolor{blue}{#1}}

\newcommand{\HT}[3]{\blue{\{#1\}} #2 \blue{\{#3\}}}

\begin{document}

\maketitle

\tableofcontents

%==================================================================================================
\section{Introduction}
%==================================================================================================

In this report we present our tool for formally verifying programs written in
imperative languages. While testing already significantly decreases the chances
of having bugs in a program, it is often hard or event impossible to write test
cases that cover every execution path.

One way of making sure a program works as intended is to formally prove that its
behavior adheres to the specification.
Such a specification usually consists of a pre- and a post-condition.
The intention is, that the results of a program satisfy the post-condition,
given that the inputs to the program fulfil the pre-condition.
The Hoare calculus provides one mathematical foundation for conducting such
proofs based on pre- and post-conditions.
But of course, doing proofs by hand is error-prone and tedious even for
simple programs.

Ideally, there would be a tool automatically proving properties about programs.
Unfortunately, it is -- in general -- undecidable to check whether a given
program (represented as a Turing machine) satisfies a given property, due to the
Halting problem\footnote{as shown by Rice's theorem}.
The power of turing machines is usually introduced in programming languages
by some form of loops or general recursion.

Fortunately, this can be migitated by requiring the programmer to annotate loops
with invariants, guiding the verification tool in the right direction.
For some specific cases it might even be possible to automatically infer those
invariants.

Using a technique called \emph{Predicate Transformers} it is possible to infer
the precondition required for a program to satisfy its post-condition.

Upon that technique we built our verification engine that we describe in the
following sections.

%==================================================================================================
\section{Project Structure}
%==================================================================================================

We implemented our project in the Haskell programming language making use of
Monads to achieve abstraction.
For invoking the solver we rely on the \texttt{sbv}\footnote{\url{https://hackage.haskell.org/package/sbv}}
library available on Hackage.
It provides a common interface to several solver backends via the SMT-LIB v2 standard,
although some features are only implemented in some solvers.
For that reason, we only use the Microsoft Z3 backend for now.

SBV provides a typed and untyped (w.r.t. expressions) interface to the prover.
As our internal AST representation is untyped and we rely on the type correctness of
the user-provided programs anyway, we use the untyped API.

Our core implementation is devided in four parts, the AST, the monadic DSL, the WLP transformer
and the adapter to the prover backend.
Furthermore, we have implemented a test suite asserting the expected results for the examples
in the project assignment description.

On top of the basic requirements, we also support array assignments, invariant inference and
program calls. The following sections describe our implementation in greater detail.

%==================================================================================================
\section{Monadic DSL for the Guarded Common Language}
%==================================================================================================

Our abstract syntax tree representing programs consists of three levels.
The root node stores the name of the program, the input and output arguments and a body statement.
The statement and expression trees closely follow the grammar outlined in the assignment description,
with one notable difference. Instead of sequencing statements pairwisely, we have an explicit \emph{block}
statement wrapping a list of statements.
Our expression AST is untyped, therefore we require the user to only construct type correct programs.
Variables in the AST are qualified, besides the literal name they also consist of a unqiue identifier.
At certain points in our code we assume that all variables in the supplied AST are indeed unique.

In order to facilitate dealing with variables we also provide a monadic DSL for constructing programs.
The DSL makes use of unqualified names, but performs name resolution and fails if undeclared variables
are used. All variable declarations in the resulting AST are unique.

We also make use of some language features to facilitate writing programs in the DSL.
It is possible to use Haskell string literals to be directly as variables by means of the
\emph{OverloadedStrings} GHC extension and to directly use integer literals and common mathematical
operations by making use the \emph{Num} type class.

It also takes care of some implementation details. If-expressions are actually implemented in terms of
non-deterministic branching in GCL. Furthermore, assignments to arrays like
\lstinline|a[i] := e| are translated to the corresponding \lstinline|a := a(i repby e)|.

Listing \ref{fig:prog:example} shows a program sorting a range $[0\ldots N)$ of an array $a$.
Program calls like \lstinline|call "minind" ["a", "i" + 1, "N"] ["m"]| mean $m := \mathit{minind}(a, i+1, N)$,
that is the first list represents the argument expressions and the second list the return variables.

\begin{lstlisting}[caption=Selection sort implemented in our DSL, label=fig:prog:example]
s = program "sort" [ "a" `as` array int, "N" `as` int] ["ret" `as` array int] $ do
  assume $ 0 .<= "N"
  var ["i" `as` int, "m" `as` int, "min" `as` int] $ do
    "i" $= 0
    let iv = ("N" .== 0 \/ "i" .<= "N" - 1)
            /\ forall ("k" `as` int) (0 .<= "k" /\ "k" .< "i" - 1 ==>
                 "a" ! "k" .<= "a" ! ("k" + 1))
            /\ forall ("k" `as` int) (0 .<= "k" /\ "k" .< "i" ==>
                 forall ("l" `as` int) ("i" .<= "l" /\ "l" .< "N" ==> "a" ! "k" .<= "a" ! "l"))
    invWhile (Just iv) ("i" .< "N" - 1) $ do
      call "minind" ["a", "i" + 1, "N"] ["m"]
      if_ ("a" ! "m" .< "a" ! "i")
        (call "swap" ["a", "i", "m"] ["a"])
        skip
      "i" $= "i" + 1
    "ret" $= "a"
    assert $ forall ("k" `as` int) (0 .<= "k" /\ "k" .< ("N" - 1) ==>
               "ret" ! "k" .<= "ret" ! ("k" + 1))
\end{lstlisting}

%==================================================================================================
\section{Implementing Predicate Transformer}
%==================================================================================================

The WLP transformer is implemented in a custom MonadProver monad we defined.
The MonadProver provides two basic actions: prove and trace, that respectively
either prove or provied a counterexample to a supplied predicate, allowing the
\textbf{wlp} to unfold, and allows us trace debug statements during the whole operation.

As we will see in later sections the there are different ways a WLP computation
could be carried on; these differences are abstracted over by a configuration
that must be provided to the transformer allowing different extensions to the
WLP behaviour.

The transformer operates under the partial correctness interpretation, as such
termination is assumed to be guaranteed.

\subsection{Basic Statements}

Dealing with most of the basic statements in the GCL language turns out to
be pretty straightforward. The implementation deals with these basic statements
exactly as presented in the lectures.
As a matter of fact: since we used the same datatype both for expressions in GCL
and the predicates built by the \textbf{wlp}, we managed to reuse the combinators defined
in the DSL to specify the predicates in the WLP. The effect of this on the code
is that it writes out almost exactly as in the equations presented in the lectures.

Listing \ref{lst:wlp:basic} shows two simple cases from our WLP implementation.
As we need to be able to run the prover on the fly, we run the WLP in a monadic context.
In the case for a block of statements we simply monadicly fold the WLP over all statements.
The implementation for assertions also follows closely the formal specification of the
WLP transformer.

\begin{lstlisting}[caption=WLP implementation example, label=lst:wlp:basic]
  go (AST.Block stmts) q = foldrM go q stmts
  go (AST.Assert e) q =  return (e /\ q)
\end{lstlisting}

\subsection{Loops}

In the case the user provides his loops with an annotated invariant the first
thing that must be done is to verify the validity of said invariant.
This can be stated as
\begin{align}
  I \land \neg g \Rightarrow Q \\
  I \land g \Rightarrow \WLP{S}{I}
\end{align}
The SBV solver has to be invoked to verify that these conditions hold, and that
the invariant supplied by the user is in fact a valid invariant.
If the invariant turns out to be invalid, it can still work out if the loop
is never entered. This gives us the following forumula for the \textbf{wlp} of an annotated
loop
\begin{align}
  \WLP{(\IWHILE{I}{g}{S})}{Q} &=
    \begin{cases}
      I, \qquad  \mathrm{if \; the \; provided \; invariant \; is \; valid}\\
      \neg g \land q, \mathrm{otherwise}
    \end{cases}
\end{align}
We will see in a subsequent section how to deal with unannotated loops.

Listing \ref{lst:wlp:loop} show an excerpt from the WLP function implementing the aforementioned case.
The behavior in the case where the invariant is invalid is configurable. We can either try to prove
that the loop is never entered, or we try to infer a correct invariant.
The invariant check itself can be turned off as well, then the invariant is always assumed correct.
That might be useful when the correctness proof is not feasible with the backend prover.

\begin{lstlisting}[caption=WLP loop case, label=lst:wlp:loop]
go (AST.InvWhile (Just iv) cnd s) q
  | not alwaysInferInvariant && checkInvariantAnnotation = do
      preInv <- go s iv
      let preserveInv = prepare $ iv /\ cnd ==> preInv
          postcnd     = prepare $ iv /\ neg cnd ==> q
      trace "trying to prove invariant is preserved"
      preserved <- prove preserveInv
      trace "trying to prove post condition"
      postValid <- prove postcnd
      if preserved && postValid
        then do
          trace "invariant valid: choosing invariant as precondition"
          return iv
        else case invalidInvariantBehavior of
          NeverExecute -> do
            trace "invariant invalid: requiring that loop is never executed"
            return (neg cnd /\ q)
          Infer -> do
            trace "invariant invalid: trying to infer an invariant"
            inferInv (Just iv) cnd s q
  | not alwaysInferInvariant && not checkInvariantAnnotation = do
      trace "assuming user-supplied invariant is correct"
      return iv
\end{lstlisting}

\subsection{Arrays}

To deal with array assignment we introduce the following translation
\begin{align}
  a[i] := e \qquad \equiv \qquad a := a ( \REPBY{i}{e} )
\end{align}
Where the newly introduced expression represents an array which is identical to
\emph{a}, except at position \emph{i} where it has value \emph{e}.

This allows us to define the corrisponding wlp as if it were a normal assignment
\begin{align}
  \WLP{(a[i] := e)}{Q} &= Q[a( \REPBY{i}{e} ) / a]
\end{align}

\subsection{Program Calls}

We can extend the \textbf{wlp} to verify programs which contain calls to other programs.
We can think of a program call like a statement with two associated sets:
the expressions passed to input argument and the output variables.
External programs can be specified in two different ways: by a complete program or
just by a specification of the preconditions and postconditions associated with
the call.
In the latter case we provide a function that builds the program body by asserting
its pre-conditions and assuming the post-conditions.
This allows us to treat both cases uniformly in the \textbf{wlp} function.
The environment is extended with associations between program names and the
tuple of input/output parameters and program body that completely specify a program
When a program call is made the body is simply inlined at call site taking care
of assigning argument expressions to input parameters and output parameters to the result variables.
Once this program fragment is built the \textbf{wlp} can simply proceed on that.

%==================================================================================================
\section{Invariant Inference}
%==================================================================================================

In the case where the user has not annotated the invariant of while-loops,
it might still be possible to infer the invariant using fixpoint-iteration
or finite unrolling.

\subsection{Fixpoint Iteration}

When we have a loop \lstinline|while g do S;|, it is equivalent to
\begin{lstlisting}
if g then { S; while g do S; }
     else skip;
\end{lstlisting}
effectively unrolling the loop once.

If $W$ is \textbf{wlp} of that loop, we have $W = \WLP{(\WHILE{g}{S})}{Q}$ as well as
$W = (g \land \WLP{S}{(\WLP{(\WHILE{g}{S})}{Q})}) \lor (\neg g \land Q)$ according to the unrolling above.
Therefore, we can conclude that $W = (g \land \WLP{S}{W}) \lor (\neg g \land Q)$.

We can find the greatest fixpoint of this equation by iterating, starting from the weakest invariant \emph{true},
following from the \emph{Knaster-Tarski} fixpoint-theorem.
\begin{align}
  \label{eq:fp:base} W_0 &= \mathit{true} \\
  \label{eq:fp:iter} W_{i+1} &= (g \land W_i) \lor (\neg g \land Q)
\end{align}

If a fixpoint exists, there is an $i$ for which $W_{i+1} = W_i$ holds.
We implemented the iteration according to equations \ref{eq:fp:base} and \ref{eq:fp:iter}.
After each iteration, we invoke the backend-prover to check whether $W_{i+1} \iff W_i$ holds.
In that case we have found the fixpoint $W_i$ and return it to the \textbf{wlp} function as
invariant.
We additionally implemented an optional limit to the number of iterations.
If that limit is exceeded without finding an invariant, we return a precondition that requires
that the loop will never be executed and the post-condition already holds.

\subsection{Loop Unrolling}

As an alternative to fixpoint-iteration we can also unroll a loop a finite number of times and
compute the \textbf{wlp} of the unrolling. The last iteration then either asserts that the
loop guard will not hold, or just assume that it doesn't hold anymore.
Formally, those unrollings, named  are defined as follows.
\begin{align}
\UNROLLSQ{0\phantom{+1}}{g}{S} &= \ASSERT{\neg g} \\
\UNROLLSQ{k+1}{g}{S} &= \IF{g}{S; \UNROLLSQ{k}{g}{S}}{\SKIP}
\end{align}

\begin{align}
\UNROLLDI{0\phantom{+1}}{g}{S} &= \ASSUME{\neg g} \\
\UNROLLDI{k+1}{g}{S} &= \IF{g}{S; \UNROLLDI{k}{g}{S}}{\SKIP}
\end{align}

Our implementation first computes (lazily) the unrolled loop and then applies
the \textbf{wlp} transformer to the unrolling.

%==================================================================================================
\section{Solver Backend}
%==================================================================================================

As mentioned before we rely on the \texttt{sbv} Haskell library for interacting with a theorem prover.
While it supports multiple provers, we only tested our implementation with Microsoft's Z3 being the
one with most features.
The WLP implementation itself is not bound to a specific prover backend and it should be easy to
add others if necessary.

\subsection{Free Monad Interface}

The WLP transformer is implemented as a free monad over WLP computations.
This allows us to specify different interpreters for the same abstract WLP tree.
We ended up implementing a handful of different interpreters along the ``main''
one, which relies on the SBV interface to carry out the proofs in predicate logic.
Other interpreters that were implemented allowed us to: \textit{pretty-printer} of WLP trees,
allowing us to inspect the different branching points and in general providing
a very useful debugging tool, an \textit{interactive} interpreter which asks the user
to carry out the requested proofs and a \textit{speculative} interpreter which, at branching
points, immediately proceeds resolving the \textbf{wlp} with all possible outcomes and
at a later point prunes the excessive branches when a proof is provided.

\subsection{Handling Quantifiers}

Our GCL DSL allows arbitrary nesting of universal and existential quantification,
but the solver library we use requires all quantified variables to be defined in
the beginning.
Fortunately, for every formula in classical logic there is an equivalent formula
in prenex normal form, where all quantifiers occur in the beginning.
The conversion to prenex normal form requires that every quantified variable is
unique to avoid accidental capturing when pulling the quantifiers outside.
Additionally, all negations have to be pushed inwards, swapping universal and
existential quantification on the go.
Skolemization is then perfomed by the prover backend.
An additional benefit from the conversion to prenex normal form is, that it allows
us to return a model for all existential quantifications in a SAT proof, as only
the values of the outermost existentials are returned by the library.

\subsection{Array Theory}

While the conversion and meaning of integer arithmetic and boolean logic should be
straightforward, arrays require are more subtle treatment.
As explained above, array assignments are translated to the $a(\REPBY{i}{e})$ syntax.
Fortunately, Z3 provides a built-in array theory with two functions \emph{select} and
\emph{store} directly mapping to array indexing and \emph{repby}.
The downside is, that the \texttt{sbv} library we use imposes some restrictions on the
use of arrays and it was not clear to us from the documentation whether these restrictions
stem directly from Z3 or from the wrapper library.
Most notable, arrays can only be used existentially during SAT proofs (i.e. only universally
during validity proofs).
This didn't mean any restriction for our examples though, because all variable declarations
result in universal quantifications.

\section{Conclusion}

Using Haskell allowed us to write the WLP implementation in a succinct declarative
style, closely following the underlying formal definitions. It also enabled us
to easily define a custom DSL for the GCL language which made working on the GCL
programs a relatively pain-free experience.

\end{document}


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[margin=1in]{geometry} % handle page geometry

\input{../header2.tex}

\title{Phys 220A -- Classical Mechanics -- Lec10}
\author{UCLA, Fall 2014}
\date{\formatdate{6}{11}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\setlength{\unitlength}{1mm}
\maketitle

\section{Symplectic Geometry}

If we want to write the Poisson bracket for some arbitrary functions, i.e.
\begin{eqn}
C(q,p) = \cbr{A(q,p), B(q,p)},
\end{eqn}
we can expand in $q, p$ and we find that the fundamental bracket is really the ``canonical'' coordinate brackets,
\begin{eqn}
\cbr{q_i, q_j} = 0, \qquad
\cbr{p_i, p_j} = 0, \qquad
\cbr{q_i, p_j} = \delta_{ij}.
\end{eqn}
We see that (as expected from quantum-classical correspondence) these are basically the same as in quantum mechanics just without the factor of $i\hbar$.

If we write points in phase space using aggregate coordinates
\begin{eqn}
\xi_I = (q_1, \dots, q_n, p_1, \dots, p_n)^\top,
\end{eqn}
we can reformulate Hamilton's equations as a single differential equation
\begin{eqn}
\dot \xi_I = \sum_J J_{IJ} \pd{H}{\xi_J}, \qquad
J = \pmat{0 & 1_n \\ -1_n & 0}.
\end{eqn}
Suppose we have a coordinate transformation
\begin{eqn}
Q = Q(q,p), \qquad P = P(q,p).
\end{eqn}
If this transformation leaves the structure of Hamilton's equations unchanged, i.e. it leaves the symplectic geometry invariant, we call it a canonical transformation. We can characterize canonical transformations in a few equivalent ways as follows.
\begin{enumerate}
\item Structure of Hamilton's equations is unchanged:
\begin{eqn}
H'(P,Q) = H(q(Q,P), p(Q,P))
\end{eqn}
we will have
\begin{eqn}
\dot P = -\pd{H'}{Q}, \qquad
\dot Q = \pd{H'}{P}.
\end{eqn}
\item Poisson brackets are invariant:
\begin{eqn}
\cbr{A'(Q,P), B'(Q,P)}_{QP} = \cbr{A(q,p), B(q,p)}_{qp}.
\end{eqn}
\item Under the transformation $Y_I = Y_I(X_I)$, the matrix $J_{IJ}$ can be written in terms of the Jacobian $j_{IJ}$ as
\begin{eqn}
J = j J j^\top, \qquad
J_{IJ} = \sum_{KL} \pd{Y_I}{X_K} J_{KL} \pd{Y_J}{X_L}.
\end{eqn}
\item Most importantly: the Poisson brackets of the new coordinates are canonical,
\begin{eqn}
\cbr{Q_i, Q_j}_{qp} = 0, \qquad
\cbr{P_i, P_j}_{qp} = 0, \qquad
\cbr{Q_i, P_j}_{qp} = \delta_{ij}.
\end{eqn}
\end{enumerate}

We can show that that canonical transformations reproduce Hamilton's equations [which characterization?]. We know
\begin{eqn}
\dot X_I = \sum_J J_{IJ} \pd{H}{X_J} = \sum_J \pd{X_I}{Y_J} \dot Y_J,
\end{eqn}
and also
\begin{eqn}
\pd{H}{X_J} = \sum_K \pd{H'}{Y_K} \pd{Y_K}{X_J}.
\end{eqn}
Then we find that
\begin{eqn}
\sum_J \dot Y_J \pd{X_I}{Y_J} = \sum_{J,K} J_{IJ} \pd{H'}{Y_K} \pd{Y_K}{X_J}
\end{eqn}
and using
\begin{eqn}
\sum_I \pd{X_I}{Y_J} \pd{Y_K}{X_I} = \delta_{KJ}
\end{eqn}
we multiply both sides by $\pd{Y_M}{X_I}$ and sum over $I$ to find
\begin{eqn}
\sum_J \dot Y_J \delta_{MJ}
	= \sum_{IJK} \pd{Y_M}{X_I} J_{IJ} \pd{Y_K}{X_J} \pd{H'}{Y_K}
	= \sum_K J_{MK} \pd{H'}{Y_K}.
\end{eqn}

\subsection{Some Examples of Canonical Transformations}

\begin{example}
Suppose we have the transformation $Q = p$, $P = -q$. Then
\begin{eqn}
\cbr{Q,P}_{qp} = - \cbr{p,q} = +1.
\end{eqn}
So this is a very simple example of a canonical transformation that shows how the $q$s and $p$s are really somewhat interchangeable. For example if we have a relativistic harmonic oscillator
\begin{eqn}
H = \sqrt{p^2 + m^2} + \frac{1}{2} m \omega^2 q^2,
\end{eqn}
under the transformation this is just
\begin{eqn}
H' = \sqrt{Q^2 + m^2} + \frac{1}{2} m \omega^2 P^2.
\end{eqn}
\end{example}

\begin{example}
Consider the transformation [this was on the final last year]
\begin{eqn}
Q = \frac{1}{2} p^2, \qquad
P = -\frac{p}{q},
\end{eqn}
so that
\begin{eqn}
p = \sqrt{2Q}, \qquad
q = -\sqrt{2Q} P.
\end{eqn}
Then we have
\begin{eqn}
\cbr{Q,P} = \frac{1}{2} \left( \pd{(p^2)}{q} \pd{(-q/p)}{p} - \pd{(p^2)}{p} \pd{(-q/p)}{q} \right) = p \frac{1}{p} = 1.
\end{eqn}
Then
\begin{eqn}
H = p^2 + q^2, \qquad
H' = 2Q + 2QP^2.
\end{eqn}
\end{example}

This approach can be somewhat messy but there it turns out that there is a nicer, more algorithmic way of checking for canonical transformations. This is the method of generating functions.

\subsection{Method of generating functions}

In this method, we will pick out functions of one old coordinate and one new coordinate so that, for example for some function $F_1(q,Q,t)$
\begin{eqn}
p = \eval{\pd{F_1}{q}}_Q \qquad \implies \qquad Q = Q(q,p),
\end{eqn}
and
\begin{eqn}
P = -\eval{\pd{F_1}{Q}}_q \qquad \implies \qquad P = P(q,p),
\end{eqn}
since
\begin{eqn}
\eval{\pd{P}{p}}_q = \eval{\pd{P}{Q}}_q \eval{\pd{Q}{p}}_q.
\end{eqn}
We will need to do this for four functions $F_1, F_2, F_3, F_4$ with parameters as all four combinations of $q,p$ and $Q,P$.

\begin{proof}
Recall that we can write the action
\begin{eqn}
S = \int \dif{t} (p \dot q - H),
\end{eqn}
so for some transformed coordinates we can write
\begin{eqn}
S = \int \dif t (P \dot Q - K)
\end{eqn}
for some function $K(Q,P)$ which will just be $H'(Q,P)$ for a canonical transformation. This is sometimes referred to as the "Cameltoenian". So we want
\begin{align}
p \dot q - H &= P \dot Q - K(Q,P) + \dod{}{t} F_1(q,Q) \\
	&= P \dot Q - K(Q, P) + \dpd{F_1}{t} + \dpd{F_1}{q} \dot q + \dpd{F_1}{Q} \dot Q.
\end{align}
So we have what we want if
\begin{eqn}
p = \pd{F_1}{q}, \qquad
P = -\pd{F_1}{Q}, \qquad
K = H + \pd{F_1}{t}.
\end{eqn}
\end{proof}

So for the three other functions we will have
\begin{eqn}
F_2(q,P), \qquad p = \pd{F_2}{q}, \qquad Q = \pd{F_2}{p},
\end{eqn} \begin{eqn}
F_3(p,Q), \qquad q = -\pd{F_3}{p}, \qquad P = -\pd{F_3}{Q},
\end{eqn} \begin{eqn}
F_4(p,P), \qquad q = -\pd{F_4}{p}, \qquad Q = \pd{F_4}{P}.
\end{eqn}

\setcounter{example}{0}
\begin{example}
Let $F_1 = qQ$, then
\begin{eqn}
p = \pd{F_1}{q} = Q, \qquad P = -\pd{F_1}{Q} = -q, \qquad H = K.
\end{eqn}
\end{example}

\begin{example}
Given Hamiltonian
\begin{eqn}
H = \frac{p^2}{2m} + \frac{1}{2} m \omega^2 q^2,
\end{eqn}
using
\begin{eqn}
F_1 = \frac{m \omega}{2} q^2 \cot Q,
\end{eqn}
we find
\begin{eqn}
p = m \omega q \cot Q, \qquad
P = \frac{m \omega q^2}{2 \sin^2 Q}.
\end{eqn}
so that
\begin{eqn}
q = \sqrt{\frac{2P}{m\omega}} \sin Q, \qquad
p = \sqrt{2 m \omega P} \cos Q,
\end{eqn}
and
\begin{eqn}
K = \omega P (\sin^2 Q + \cos^2 Q) = \omega P.
\end{eqn}
This one in particular is useful if $Q$ is cyclic and $P$ is constant, in which case $Q = Q_0 + \omega t$.
\end{example}

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[margin=1in]{geometry}
\usepackage{amsmath}
\usepackage{graphicx}
\setlength{\parindent}{0em}
\setlength{\parskip}{0.5em}

\title{CTA200 2020 Assignment 2}
\author{Henri Lamarre}
\date{}

\begin{document}

\maketitle

\section{Question 1}
\subsection{Methods}
In this section, we initialize 100000 sampled points in the complex square of width 4 centered at $(0,0)$. We then iterate for 20 steps with the rule $z_{i+1}=z_i+c$. We then assign the color red to the points that end up outside the box at the end of the 20 iterations. We assign the color blue to the points that are still in the box after the 20 iterations. Then, we zoom on the square of width $0.2$ centered at $(0,0)$ and redo the same process but with only 2500 sampled points.

\begin{figure}[h]
\includegraphics[width = 0.5\columnwidth]{question1_1.png}
\includegraphics[width = 0.5\columnwidth]{question1_2.png}
\caption{[Left] Scatter plot of the positions of points for each iteration step in the box of width 4. [Right] Scatter plot of the positions of points for each iteration step in the box of width 0.2.}
\end{figure}

\subsection{Analysis}
We note that points that converge are mostly points which $|z|$ value is smaller than $1$. We then zoom on the square to make sure that that restriction is respected and indeed, all the points converge.

\section{Question 2}
\subsection{Methods}
We solve the set of first degree ODEs using scipy.integrate.odeint(). We supply the following initial conditions: 999 susceptibles, 1 infected and 0 recovered. Then, we modify one of the ODE and add a new one by introducing deaths in the parameter $\alpha$. We then use scipy again to solve these four ODEs.

\begin{figure}[h]
\includegraphics[width = 0.5\columnwidth]{question2_1.png}
\includegraphics[width = 0.5\columnwidth]{question2_2.png}
\includegraphics[width = 0.5\columnwidth]{question2_3.png}
\includegraphics[width = 0.5\columnwidth]{question2_4.png}
\caption{Evolution of the number of people in the groups: Infected, Susceptible (Suspected), Recovered and Dead. The parameter $\alpha$ is 0 in the first three plots.}
\end{figure}

\subsection{Analysis}
By increasing $\beta$, the rate of propagation is substancially lowered. We hypothesise that $\beta$ is inversly proportionnal to the rate of propagation of the virus. Then, by increasing $\gamma$, we notice that more people get infected. We hypothesise that $\gamma$ is related to the chances of contracting the virus by being in contact with it. Then, by adding the death parameter, we notice that the number of infected decreases faster which was expected.

\end{document}