\documentclass{slnotes}
\newcommand*{\slbx}[1]{{\ifmmode\smash{\boxed{#1\vphantom{X}}}\else\smash{\fbox{#1\vphantom{X}}}\fi}}
\newcommand*{\ve}[1]{\mathbf{#1}}
\newcommand*\cve[3][]{\begin{pmatrix}\ifx\relax#1\relax\else#1\\\fi#2\\#3\end{pmatrix}}
\newcommand*{\im}{\mathrm{i}}
\DeclareMathOperator*{\Prb}{P}
\DeclareMathOperator*{\Pois}{Po}
\DeclareMathOperator*{\Binom}{B}
\DeclareMathOperator*{\Norm}{N}
\DeclareMathOperator*{\Exp}{E}
\DeclareMathOperator*{\Var}{Var}
\begin{document}
\chapter{Functions}
A relation \(f \colon X \to Y\) where \(X, Y \subseteq \mathbb R\) is a function if, \(\mathrel\forall a \in X\), the vertical line \(x = a\) cuts the graph of \(f\) at exactly one point.

A relation \(f\) can be proven to be not a function if there is some \(a \in X\) for which the vertical line \(x = a\) cuts the graph of \(f\) zero times or more than once.

A function \(f\) is one-one if, \(\mathrel\forall a \in \mathbb R\), the horizontal line \(y = a\) cuts the graph of \(f\) at most once.

A function \(f\) can be proven to be not one-one if there is some \(a \in \mathbb R\) for which the horizontal line \(y = a\) cuts the graph of \(f\) more than once.
\chapter{APGP}
For a series \(S_n\), the \(n\)th term of the corresponding progression \[T_n = S_n - S_{n-1}\]

For an arithmetic progression \(T_n\) and an arithmetic series \(S_n\) the \(n\)th term \begin{align*}T_n &= a + (n-1)d\\S_n &= \frac{n}{2}(a + l)\\&= \frac{n}{2}(2a+(n-1)d)\end{align*} where \(a\) is the first term, \(l\) is the last term and \(d\) is the common difference. To prove that a series \(S_n\) is an arithmetic series, \[T_n - T_{n-1}\text{ is a constant} \iff S_n\text{ is an arithmetic series}\]

For a geometric progression \(T_n\) and a geometric series \(S_n\) the \(n\)th term \begin{align*}T_n &= ar^{n-1}\\S_n &= a\frac{r^n-1}{r-1}\end{align*} where \(a\) is the first term and \(r\) is the common ratio. To prove that a series \(S_n\) is a geometric series, \[\frac{T_n}{T_{n-1}}\text{ is a constant} \iff S_n\text{ is a geometric series}\]

For a geometric progression with \(|r| < 1\) the sum to infinity \[S_\infty = \frac{a}{1-r}\]
\chapter{Induction format}
Let \(P_n\) be the statement \(\slbx{\text{LHS}} = \slbx{\text{RHS}}\text{, }n \in \slbx{X}\).

\begin{tabbing}
When \(n = \slbx{n}\), \=\(\text{LHS} = \slbx{\text{LHS}} = \slbx{\text{LHS value}}\).\\
\>\(\text{RHS} = \slbx{\text{RHS}} = \slbx{\text{RHS value}} = \text{LHS}\).\\
\>\(\mathrel\therefore P_\slbx{n}\text{ is true.}\)\\
\end{tabbing}

Assume \(P_k\) is true for some \(k \in \slbx{X}\) i.e. \(\slbx{\text{LHS}} = \slbx{\text{RHS}}\).

To prove \(P_{k+1}\) is also true i.e. \(\slbx{\text{LHS}} = \slbx{\text{RHS}}\) \begin{align*}\text{LHS} &= \slbx{\text{LHS}} = \cdots\\&= \slbx{\text{RHS}} = \text{RHS}\end{align*} \(\mathrel\therefore P_k\text{ is true} \implies P_{k+1}\text{ is true}\).

Since \(P_\slbx{n}\) is true, and \(P_k\text{ is true} \implies P_{k+1}\text{ is true}\), by mathematical induction, \(P_n\text{ is true}\mathrel\forall n \in \slbx{\mathrm X}\).
\chapter{Vectors}
Let there be two points \(A\) and \(B\) which have position vectors \(\ve a\) and \(\ve b\).

For the vectors \(\ve a\) and \(\ve b\), their dot product where \(\theta\) is the angle between them \begin{align*}\ve a \cdot \ve b &= |\ve a||\ve b|\cos\theta\\&=\cve[x_{\ve a}]{y_{\ve a}}{\vdots} \cdot \cve[x_{\ve b}]{y_{\ve b}}{\vdots} = x_{\ve a}x_{\ve b} + y_{\ve a}y_{\ve b} + \cdots\end{align*}

Their cross product where \(\theta\) is the angle between them and \(\ve{\hat n}\) is a unit vector perpendicular to both in the direction of a right-handed screw turned from \(\ve a\) to \(\ve b\) \begin{align*}\ve a \times \ve b &= \ve{\hat n}|\ve a||\ve b|\sin\theta\\&= \cve[x_{\ve a}]{y_{\ve a}}{z_{\ve a}} \times \cve[x_{\ve b}]{y_{\ve b}}{z_{\ve b}} = \cve[y_{\ve a}z_{\ve b} - y_{\ve b}z_{\ve a}]{z_{\ve a}x_{\ve b} - z_{\ve b}x_{\ve a}}{x_{\ve a}y_{\ve b} - x_{\ve b}y_{\ve a}}\end{align*}

From the above definitions \begin{align*}\ve a \mathrel\bot \ve b &\iff \ve a \cdot \ve b = 0\\\ve a \parallel \ve b &\iff \mathrel\exists \lambda \in (\mathbb{R} \setminus \{0\}): \ve b = \lambda\ve a\end{align*}

The angle between the vectors \[\theta = \cos^{-1}\frac{\ve a \cdot \ve b}{|\ve a||\ve b|}\]

The length of projection and the projection vector of \(\ve a\) onto \(\ve b\) where \(F\) is the foot of the perpendicular from \(A\) to \(OB\)\begin{align*}OF &= |\ve a \cdot \ve{\hat b}|\\\overrightarrow{OF} &= (\ve a \cdot \ve{\hat b})\ve{\hat b}\end{align*} The perpendicular distance from \(A\) to \(OB\) \[AF = |\ve a \times \ve{\hat b}|\]

A vector normal to \(\ve a\) and \(\ve b\) is simply \(\ve a \times \ve b\).

If \(ABCD\) is a parallelogram then \begin{align*}\text{area of }ABD &= \frac{1}{2}\left|\overrightarrow{AB} \times \overrightarrow{AD}\right|\\\text{area of }ABCD &=  \left|\overrightarrow{AB} \times \overrightarrow{AD}\right|\end{align*}

For three points \(A\), \(B\) and \(C\), {\jot0pt\begin{multline*}A\text{, }B\text{ and }C\text{ are collinear}\\\iff \mathrel\exists\lambda\in(\mathbb R \setminus \{0\}): \overrightarrow{AB} = \lambda\overrightarrow{AC}\end{multline*}}

The general equation of a line is \[\ve r = \ve a + \lambda\ve d\quad\lambda\in\mathbb R\] where \(\ve d\) is the direction vector of the line.

The parametric equation of a plane is \[\ve r = \ve a + \lambda\ve m_1 + \mu\ve m_2\quad\lambda,\mu\in\mathbb R\] The vector equation of the same plane is \[\ve r \cdot \ve n = D\quad D = \ve a \cdot \ve n\] The cartesian equation of the plane is \[\ve r = \cve[x]{y}{z} \wedge \ve a = \cve[a]{b}{c} \implies ax+by+cz = D\]
\chapter{Complex Numbers}
For a complex number \begin{align*}z &= x + \im y\\&= |z|(\cos(\arg z) + \im\sin(\arg z)) \\&= |z|e^{\im(\arg z)}\end{align*} its conjugate and magnitude \begin{align*}z^* &= x - \im y\\zz^* &= |z|^2 = x^2 + y^2 \implies |z| = \sqrt{x^2+y^2}\end{align*} and its argument \[\arg z = \begin{cases}\tan^{-1} \frac{y}{x} & x > 0\\\pi + \tan^{-1} \frac{y}{x} & x < 0 \wedge y \geq 0\\-\pi + \tan^{-1} \frac{y}{x} & x < 0 \wedge y < 0\end{cases}\]

The properties of the argument \(\arg z\) are identical to the properties of the logarithm.

If a polynomial with real coefficients has complex root \(\alpha\), then \(\alpha^*\) is also a root.

The fundamental theorem of algebra states that every polynomial equation of degree \(n\) has \(n\) roots that may not be distinct.
\chapter{Probability}
Some useful results are \begin{gather*}\Prb(A') = 1-\Prb(A)\\\Prb(A\cup B) = \Prb(A) + \Prb(B) - \Prb(A \cap B)\\\Prb(A | B) = \frac{\Prb(A \cap B)}{\Prb(B)}\end{gather*}

If \(A\) and \(B\) are independent events then \[\Prb(A|B) = \Prb(A) \wedge \Prb(B|A) = \Prb(B) \wedge \Prb(A\cap B) = \Prb(A)\Prb(B)\]

If \(A\) and \(B\) are mutually exclusive events then \[\Prb(A\cap B) = 0 \wedge \Prb(A \cup B) = \Prb(A) + \Prb(B)\]
\chapter{Distributions}
For an random variable to be modelled by a binomial distribution, it must have \begin{slinenum}
\item a fixed number of trials
\item independent trials
\item identical trials
\item the same probability of success for each trial
\item two possible outcomes.
\end{slinenum}

For an event to be modelled by a Poisson distribution, it must \begin{slinenum}
\item occur at a constant average rate
\item occur singly
\item have independent occurrences.
\end{slinenum}

For an event \(X\) \begin{gather*}X \sim \Norm(\mu, \sigma^2) \implies Z = \frac{X-\mu}{\sigma} \sim\Norm(0, 1)\\\Prb(X \leq x) = \Prb(Z \leq \frac{x- \mu}{\sigma})\end{gather*}
\chapter{Expectation and Variance}
The expected value of a random variable is the prob\-ability-weighted average of all possible values: \[\Exp(X) = \sum x \Prb(X = x) = \int_{-\infty}^\infty xf(x)\slid x\] where \(f(x)\) is the probability density function of weighing \(X\), if it is a continuous random variable..

If \(X\) and \(Y\) are independent random variables, \(X_1\) and \(X_2\) are independent observations of \(X\), and \(a\) and \(b\) are constants then \begin{align*}\Exp(a) &= a\\\Exp(aX + bY) &= a\Exp(X) + b\Exp(Y)\\\Var(a) &= 0\\\Var(aX + bY) &= a^2\Var(X) + b^2\Var(Y)\\\Var(X_1 + X_2) &= \Var(X_1) + \Var(X_2) = 2\Var(X)\end{align*}
\chapter{Approximations}
If \(X\) is a random discrete variable such that \[X \sim \Binom(n, p)\] then if \(n\) is large and \(p\) is small such that \(np < 5\), \[X \sim \Pois(np)\text{ approximately}\]

If \(X\) is a random discrete variable such that \[X \sim \Binom(n, p)\] then if \(n\) is large such that \(np > 5 \wedge nq > 5\), \[X \sim \Norm(np,npq)\text{ approximately}\]

If \(X\) is a random discrete variable such that \[X \sim \Pois(\lambda)\] then if \(\lambda > 10\), \[X \sim \Norm(\lambda,\lambda)\text{ approximately}\]

When a normal distribution is used to approximate a discrete distribution, continuity correction must be done.
\chapter{Sampling}
If \(X\) is a random variable with unknown or non-normal distribution where \[\Exp(X) = \mu\text{ and }\Var(X) = \sigma^2\] central limit theorem states that if sample size \(n\) is large, \[\overline{X} \sim \Norm(\mu,\frac{\sigma^2}{n})\text{ approximately}\]

Unbiased estimates of \(\mu\) and \(\sigma^2\) are sample mean \(\overline{x}\) and \(s^2\) respectively. \[s^2 = \frac{n}{n-1}(\text{sample variance})\]
\chapter{Hypothesis Testing}
\section{Format}
Let \(X\) be the \slbx{something} of a randomly chosen \slbx{thing}.

Let \(\mu\) be the population mean \slbx{something} of \slbx{things}.

{\fboxsep=3pt\fbox{\begin{varwidth}{0.5\textwidth}\begin{tabbing}
\textbf{One of} \=Given \(X \sim \Norm(\mu, \sigma^2)\),\\
\textbf{or}\>Assume \(X \sim \Norm(\mu, \sigma^2)\),\\
\textbf{or}\>Since \(n = \slbx{n}\), by central limit theorem,\end{tabbing}\end{varwidth}}}
\[\overline{X} \sim \Norm\left(\mu, \frac{\sigma^2}{n}\right)\ \slbx{\text{approximately}}\]

\textbf{H}\textsubscript0: \(\mu = \mu_0\); \textbf{H}\textsubscript1: \slbx{\(\mu < \mu_0\) \textbf{or} \(\mu > \mu_0\) \textbf{or} \(\mu \neq \mu_0\)}

Test statistic:

{\fboxsep=3pt\fbox{\begin{varwidth}{0.5\textwidth}\begin{tabbing}
\textbf{One of} \=\(T = \dfrac{\overline{X}-\mu}{S/\sqrt{n}} \sim t_{n-1}\)\\\\
\textbf{or}\>\(Z = \dfrac{\overline{X}-\mu}{\sigma/\sqrt{n}}\)\\\\
\textbf{or}\>\(Z = \dfrac{\overline{X}-\mu}{S/\sqrt{n}}\)\end{tabbing}\end{varwidth}}}

Level of significance: \slbx{\(100\alpha\%\)}; reject H\textsubscript0 if p-value < \slbx{\(\alpha\)}.

Under H\textsubscript0, p-value = \slbx{p-value}.

Since p-value = \slbx{p-value} \slbx{\(<\) \textbf{or} \(>\)} \(\alpha\), we \slbx{do not} reject H\textsubscript0 and conclude that there is \slbx{in}sufficient evidence, at \slbx{\(100\alpha\%\)} level, that \slbx{H\textsubscript1}.
\section{Definitions}
If the level of significance is \(100\alpha\%\), there is a probability of \(\alpha\) of concluding that H\textsubscript1 is true when in fact, H\textsubscript0 is true.

The p-value is the probability of obtaining a sample mean \textbf(less than or equal to \textbf{or} more than or equal to \textbf{or} as extreme or more extreme than\textbf) \(\overline{x}\), assuming H\textsubscript0 is true.
\chapter{Sampling Methods}
A population is a collection of individuals or objects from which we may collect data.

A random sample is a small representative of the population in which every member in the population has an equal probability of being selected.

We often take a sample instead of collecting data from the entire population as \begin{slinenumor}
\item it may be too costly or time consuming to collect the information from the whole population
\item the population may be infinite or too large.
\end{slinenumor}
\section{Quota sampling}
To take a quota sample, we \begin{slinenum}
\item divide the population into mutually exclusive subgroups called strata, namely \slbx{group} and \slbx{group}
\item select \slbx{number} \slbx{group} and \slbx{number} \slbx{group} to form the sample. The sample from each stratum is non-random.
\end{slinenum}

Quota sampling is advantageous as \begin{slinenum}
\item it is easy to select the sample and administer the survey
\item no sampling frame needed
\item it incurs low cost.
\end{slinenum}

However, it is bad in that \begin{slinenum}
\item the sample obtained is non-random
\item it is likely to result in selection bias as interviewer may select those who are easier to interview.
\end{slinenum}
\section{Simple random sampling}
To take a simple random sample, we \begin{slinenum}
\item obtain the list of \slbx{samplees} and number all \slbx{samplees} from 1 to \slbx{number}
\item use a random number generator to randomly select \slbx{number} numbers
\item select the \slbx{samplees} corresponding to the numbers to form the sample.
\end{slinenum}

Simple random sampling is advantageous as \begin{slinenum}
\item the data collected generally free from bias
\item analysis of data is relatively easy.
\end{slinenum}

However, it is bad in that \begin{slinenum}
\item it may be difficult to draw up the sampling frame as it is difficult to identify every member of the population
\item it may be difficult to get access to members who have been chosen for the sample
\end{slinenum}
\section{Systematic sampling}
To take a systematic sample, we \begin{slinenum}
\item obtain the list of \slbx{samplees} and number all \slbx{samplees} from 1 to \slbx{number}
\item compute the sampling interval \(k = \slbx{\text{calculation}}\)
\item randomly select an integer from 1 to \slbx{number} as the start
\item select every \slbx{\(k\)}th member of the population thereafter until the sample of \slbx{number} is obtained.
\end{slinenum}

Systematic sampling is advantageous as \begin{slinenum}
\item the data collected is generally free from bias
\item analysis of data is relatively easy.
\end{slinenum}

However, it is bad in that \begin{slinenum}
\item it may be difficult to draw up the sampling frame as it is difficult to identify every member of the population
\item it may be difficult to get access to members who have been chosen for the sample
\item there may be selection bias if the members of the population is arranged in a periodic or cyclic pattern.
\end{slinenum}
\section{Stratified sampling}
To take a stratified sample, we \begin{slinenum}
\item obtain the list of \slbx{samplees} and divide the population into mutually exclusive subgroups called strata, namely \slbx{group} and \slbx{group}
\item calculate the sample size that should be taken for each stratum, proportional to their size i.e. \(\slbx{\text{calculation}} = \slbx{\text{number}}\) for \slbx{group}, and \slbx{number} for \slbx{group}
\item select the sample from each stratum using simple random sampling.
\end{slinenum}

Stratified sampling is advantageous as \begin{slinenum}
\item it is more likely to give a good representative sample of the population
\item when there are clear strata present, it usually gives more reliable estimates of the population parameters than random or systematic sampling.
\end{slinenum}

However, it is bad in that \begin{slinenum}
\item it may be difficult to draw up the sampling frame as it is difficult to identify every member of the population
\item it is more difficult to conduct than random sampling
\item it may be difficult to identify appropriate strata
\item strata may not be clearly defined.
\end{slinenum}
\end{document}


\subsection{Spline basis sets}
\label{sec:spo_spline}
In this section we describe the use of spline basis sets to expand the \texttt{sposet}.
Spline basis sets are designed to work seamlessly with plane wave DFT code, e.g.\ Quantum ESPRESSO as a trial wavefunction generator.

In QMC algorithms, all the SPOs $\{\phi(\vec{r})\}$ need to be updated every time a single electron moves.
Evaluating SPOs takes very large portion of computation time.
In principle, PW basis set can be used to express SPOs directly in QMC like in DFT.
but it introduces an unfavorable scaling due to the fact
that the basis set size increases linearly as the system size.
For this reason, it is efficient to use a localized basis with compact
support and a good transferability from plane wave basis.

In particular, 3D tricubic B-splines provide a basis in which only
64 elements are nonzero at any given point in space~\cite{blips4QMC}.
The one-dimensional cubic B-spline is given by,
\begin{equation}
f(x) = \sum_{i'=i-1}^{i+2} b^{i'\!,3}(x)\,\,  p_{i'},
\label{eq:SplineFunc}
\end{equation}
where $b^{i}(x)$ are the piecewise cubic polynomial basis functions
and $i = \text{floor}(\Delta^{-1} x)$ is the index of
the first grid point $\le x$.  Constructing a tensor product in each Cartesian
direction, we can represent a 3D orbital as
\begin{equation}
  \phi_n(x,y,z) =
  \!\!\!\!\sum_{i'=i-1}^{i+2} \!\! b_x^{i'\!,3}(x)
  \!\!\!\!\sum_{j'=j-1}^{j+2} \!\! b_y^{j'\!,3}(y)
  \!\!\!\!\sum_{k'=k-1}^{k+2} \!\! b_z^{k'\!,3}(z) \,\, p_{i', j', k',n}.
\label{eq:TricubicValue}
\end{equation}
This allows the rapid evaluation of each orbital in constant time.
Furthermore, this basis is systematically improvable with a single spacing
parameter, so that accuracy is not compromised compared with plane wave basis.

The use of 3D tricubic B-splines greatly improves the computational efficiency.
The gain in computation time from plane wave basis set to an equivalent B-spline basis set
becomes increasingly large as the system size grows.
On the downside, this computational efficiency comes at
the expense of increased memory use, which is easily overcome by the large
aggregate memory available per node through OpenMP/MPI hybrid QMC.

The input xml block for the spline SPOs is give in Listing~\ref{listing:splineSPOs}. A list of options is given in
Table~\ref{table:splineSPOs}. \texttt{QMCPACK} has a very useful command line option \texttt{--save\_wfs} which allows to dump
the real space B-spline coefficient table into a h5 file on the disk.
When the orbital transformation from k space to B-spline requires more than available amount of scratch memory on the compute nodes,
users can perform this step on fat nodes and transfer back the h5 file for QMC calculations.

\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l l }
\hline
\multicolumn{6}{l}{\texttt{determinantset} element} \\
\hline
\multicolumn{2}{l}{parent elements:} & \multicolumn{4}{l}{\texttt{wavefunction}}\\
\multicolumn{2}{l}{child  elements:} & \multicolumn{4}{l}{\texttt{slaterdeterminant}}\\
\multicolumn{2}{l}{attribute      :} & \multicolumn{4}{l}{}\\
   &   \bfseries name                   & \bfseries datatype & \bfseries values & \bfseries default & \bfseries description \\
   &   \texttt{type}                    &  text              &   bspline        &                   &  Type of \texttt{sposet}. \\
   &   \texttt{href}                    &  text              &                  &                   &  Path to the h5 file made by pw2qmcpack.x. \\
   &   \texttt{tilematrix}              &  9 integers        &                  &                   &  Tiling matrix used to expand supercell. \\
   &   \texttt{twistnum}                &  integer           &                  &                   &  Index of the super twist. \\
   &   \texttt{twist}                   &  3 floats          &                  &                   &  Super twist. \\
   &   \texttt{meshfactor}              &  float             &  $\le 1.0$       &                   &  Grid spacing ratio. \\
   &   \texttt{precision}               &  text              &  single/double   &                   &  Precision of spline coefficients. \\
   &   \texttt{gpu}                     &  text              &  yes/no          &                   &  GPU switch. \\
   &   \texttt{Spline\_Size\_Limit\_MB} &  integer           &                  &                   &  Limit B-spline table size on GPU. \\
   &   \texttt{check\_orb\_norm}        &  text              &  yes/no          &  yes              &  Check norms of orbitals from h5 file. \\
   &   \texttt{source}                  &  text              &  \textit{any}    &  ion0             &  Particle set with atomic positions. \\
  \hline
\end{tabularx}
\end{center}
\caption{Options for the \texttt{determinantset} xml-block associated with B-spline single particle orbital sets.}
\label{table:splineSPOs}
\end{table}

%%\begin{lstlisting}[caption=.]
%%<sposet_builder type="bspline" href="pwscf.h5" tilematrix="2 0 0 0 2 0 0 0 2" twistnum="0"
%%                 source="i" meshfactor="1.0" precision="float" truncate="no">
%%   <sposet type="bspline" name="spo_ud" size="208" spindataset="0"/>
%%</sposet_builder>
%%<determinantset>
%%   <slaterdeterminant>
%%      <determinant id="updet" group="u" sposet="spo_ud" size="208"/>
%%      <determinant id="downdet" group="d" sposet="spo_ud" size="208"/>
%%   </slaterdeterminant>
%%</determinantset>
%%\end{lstlisting}

\begin{lstlisting}[caption=Determinant set XML element.\label{listing:splineSPOs}]
<determinantset type="bspline" source="i" href="pwscf.h5"
                tilematrix="1 1 3 1 2 -1 -2 1 0" twistnum="-1" gpu="yes" meshfactor="0.8"
                twist="0  0  0" precision="double">
  <slaterdeterminant>
    <determinant id="updet" size="208">
      <occupation mode="ground" spindataset="0">
      </occupation>
    </determinant>
    <determinant id="downdet" size="208">
      <occupation mode="ground" spindataset="0">
      </occupation>
    </determinant>
  </slaterdeterminant>
</determinantset>
\end{lstlisting}

Additional information:
\begin{itemize}
\item \texttt{precision}. Only effective on CPU version without mixed precision, `single' is always imposed with mixed precision. Using single precision not only saves memory usage but also speeds up the B-spline evaluation. It is recommended to use single precision since we saw little chance of really compromising the accuracy of calculation.
\item \texttt{meshfactor}. It is the ratio of actual grid spacing of B-splines used in QMC calculation with respect to the original one calculated from h5. Smaller meshfactor saves memory usage but reduces accuracy. The effects are similar to reducing plane wave cutoff in DFT calculation. Use with caution!
\item \texttt{twistnum}. If positive, it is the index. It is recommended not to take this way since the indexing may show some uncertainty. If negative, the super twist is referred by \texttt{twist}.
\item \texttt{Spline\_Size\_Limit\_MB}. Allows to distribute the B-spline coefficient table between the host and GPU memory. The compute kernels access host memory via zero-copy. Though the performance penaty introduced by it is significant but allows large calculations to go.
\end{itemize}


\hypertarget{lab-02-variables-arrays-and-scripts}{%
\section{Lab 02: Variables, Arrays, and
Scripts}\label{lab-02-variables-arrays-and-scripts}}

\hypertarget{variables}{%
\subsection{Variables}\label{variables}}

\begin{frame}{}
\protect\hypertarget{section}{}
Variables help us represent quantities or expressions in order to make
their use and re-use more convenient.
\end{frame}

\begin{frame}[fragile]{Naming Variables}
\protect\hypertarget{naming-variables}{}
\begin{itemize}[<+->]
\tightlist
\item
  Must start with a letter.
\item
  Followed by letters (a-z, A-Z) or numbers (0-9) or underscores (\_).
\item
  Maximum 65 characters (excluding the .m extension).
\item
  Must not be the same as any MATLAB reserved word.
\item
  Space is not permitted.
\item
  Case sensitive, i.e., \texttt{a\ \textasciitilde{}=\ A}.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Naming Variables}
\protect\hypertarget{naming-variables-1}{}
\begin{itemize}[<+->]
\tightlist
\item
  Be as descriptive as possible with your variable names.
\item
  Avoid built-in function/variable names (reserved keywords) such as
  \texttt{pi}, \texttt{sin}, \texttt{exp}, etc.
\item
  Check if a name is already in use: \texttt{which\ variableName} or
  \texttt{exist\ variableName}.
\end{itemize}
\end{frame}

\begin{frame}{Naming Conventions}
\protect\hypertarget{naming-conventions}{}
\begin{itemize}[<+->]
\tightlist
\item
  snake\_case: writing compound words or phrases in which the elements
  are separated with one underscore character (\_) and no spaces,
  e.g.~``foo\_bar''.
\item
  camelCase: writing compound words or phrases such that each word or
  abbreviation in the middle of the phrase begins with a capital letter,
  with no intervening spaces or punctuation, e.g.~``fooBar''
\item
  Other conventions: Hungarian notation, positional notation, etc.
\item
  Reference:
  \url{https://en.wikipedia.org/wiki/Naming_convention_(programming)}
\end{itemize}
\end{frame}

\begin{frame}{Default Variable Definitions}
\protect\hypertarget{default-variable-definitions}{}
\begin{table}[!hbtp]
    \begin{tabular}{rl}
        Command & Description \\
        \hline
        \texttt{pi} & variable defining $\pi$ \\
        \texttt{i} or \texttt{1i} & imaginary number $i = \sqrt{-1}$ \\
        \texttt{j} or \texttt{1j} & imaginary number $j = \sqrt{-1}$
    \end{tabular}
\end{table}
\end{frame}

\hypertarget{arrays}{%
\subsection{Arrays}\label{arrays}}

\begin{frame}[fragile]{Array, Vector, and Matrix}
\protect\hypertarget{array-vector-and-matrix}{}
\begin{itemize}[<+->]
\tightlist
\item
  An array is a data form that can hold several values, all of one type.
\item
  A vector is a \(1\)-D array: we can define row vectors, column
  vectors.
\item
  A matrix is a \(2\)-D array.
\item
  Also, we can define \(N\)-D array.
\item
  The general notation for a vector or matrix is a list of values
  enclosed in square brackets \texttt{{[}{]}} separated by commas
  (space) or semi-colons (or the combination).
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Vector: \texttt{{[}{]}}}
\protect\hypertarget{vector}{}
\begin{itemize}[<+->]
\item
  Row vector: \(x = \begin{bmatrix} 1 & 2 & 3 & 4 \end{bmatrix}\)

\begin{verbatim}
x = [1,2,3,4]
x = [1 2 3 4]
\end{verbatim}
\item
  Column vector: \(y = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix}\)
  or \(y = \begin{bmatrix} 1 & 2 & 3 & 4\end{bmatrix}^{\top}\) or
  \(y = x^{\top}\).

\begin{verbatim}
y = [1;2;3;4]
y = transpose([1 2 3 4])
y = [1 2 3 4]'
y = x'
y = x(:)
\end{verbatim}

  Note: \texttt{\textquotesingle{}} and \texttt{.\textquotesingle{}} are
  the infix notation for \texttt{ctrasnpose}, \texttt{transpose}
  operation.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Vector: \texttt{linspace} vs.~\texttt{colon}}
\protect\hypertarget{vector-linspace-vs.-colon}{}
\begin{itemize}[<+->]
\item
  \texttt{linspace(from,\ to,\ n)} generates \texttt{n} points between
  \texttt{from} (inclusive) and \texttt{to} (inclusive). For example,

  \texttt{a\ =\ linspace(2,\ 6,\ 5)\ \ \%\ same\ as\ a\ =\ {[}2\ 3\ 4\ 5\ 6{]}}
\item
  \texttt{colon(from,\ step,\ upper\_bound)} generates points between
  \texttt{from} (inclusive) and \texttt{upper\_bound} (may not be
  inclusive) with spacing \texttt{step}. For example,

\begin{verbatim}
a = colon(2, 1, 6)  % same as a = [2 3 4 5 6]
a = colon(2, 2, 6)  % same as a = [2 4 6]
a = colon(2, 1, 7)  % same as a = [2 3 4 5 6 7]
a = colon(2, 2, 7)  % same as a = [2 4 6]
\end{verbatim}
\item
  \texttt{from:step:upper\_bound} is same as
  \texttt{colon(from,\ step,\ upper\_bound)}.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Vector: \texttt{linspace} vs.~\texttt{colon}}
\protect\hypertarget{vector-linspace-vs.-colon-1}{}
\begin{itemize}[<+->]
\tightlist
\item
  \texttt{linspace(from,\ to,\ n)} is equivalent to
  \texttt{colon(from,\ (to\ -\ from)\ /\ (n\ -\ 1),\ to)}
\item
  \texttt{colon(from,\ step,\ upper\_bound)} is equivalent to
  \texttt{linspace(from,\ floor((upper\_bound\ -\ from)\ /\ step)\ *\ step\ +\ from,\ floor((upper\_bound\ -\ from)\ /\ step))}
\item
  Use \texttt{linspace} when the number of points is given.
\item
  Use \texttt{colon} when the spacing/step size is given.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Matrix: \texttt{{[}{]}}}
\protect\hypertarget{matrix}{}
Define a \(2 \times 3\) matrix
\(A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}\)

\begin{verbatim}
A = [1,2,3;4,5,6]
\end{verbatim}

or

\begin{verbatim}
row1 = [1,2,3]
row2 = [4,5,6]
A = [row1;row2]
\end{verbatim}

or

\begin{verbatim}
col1 = [1;4]
col2 = [2;5]
col3 = [3;6]
A = [col1,col2,col3]
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Matrix: \texttt{zeros}, \texttt{ones},
\texttt{eye}, \texttt{rand}, \texttt{randn}, \texttt{magic}}
\protect\hypertarget{matrix-zeros-ones-eye-rand-randn-magic}{}
\begin{itemize}[<+->]
\item
  \texttt{zeros(m,\ n)}: define a \texttt{m}-by-\texttt{n} matrix with
  zeros.

\begin{verbatim}
zeroRowVec = zeros(5, 1)
zeroColVec = zeros(1, 5)
zeroMatrix = zeros(5, 5)
zeroMatrix = zeros(5)
\end{verbatim}
\item
  \texttt{ones(m,\ n)}: define a \texttt{m}-by-\texttt{n} matrix with
  ones.
\item
  \texttt{eye(m,\ n)}: define a \texttt{m}-by-\texttt{n} matrix with
  diagonals being ones.
\item
  \texttt{rand(m,\ n)}: define a \texttt{m}-by-\texttt{n} matrix with
  uniformly distributed numbers.
\item
  \texttt{randn(m,\ n)}: define a \texttt{m}-by-\texttt{n} matrix with
  normally distributed numbers.
\item
  \texttt{magic(n)}: define a \texttt{n}-by-\texttt{n} magic square with
  row sum, column sum and diagonal sum being equal.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Dimension: \texttt{size}, \texttt{length},
\texttt{reshape}}
\protect\hypertarget{dimension-size-length-reshape}{}
\begin{itemize}[<+->]
\item
  \texttt{size(array)}: size of \texttt{array}. If \texttt{array} is
  \texttt{n}-dimensional, \texttt{size} will return a vector of length
  \texttt{n}.
\item
  \texttt{size(array,\ 1)}: number of rows of \texttt{array}.
\item
  \texttt{size(array,\ 2)}: number of columns of \texttt{array}.
\item
  \texttt{length(vec)}: length of vector \texttt{vec}, equivalent to
  \texttt{max(size(vec))}.
\item
  \texttt{reshape(array,\ dim1,\ dim2,\ dim3,\ ...)}.

\begin{verbatim}
rowVec = 1:8
matrix = reshape(rowVec, 2, 4)
% same as matrix = [1,3,5,7;2,4,6,8]
\end{verbatim}
\item
  \texttt{reshape(array,\ prod(size(array)),\ 1)} is same as
  \texttt{array(:)}.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{\(N\)-D array: \texttt{reshape}}
\protect\hypertarget{n-d-array-reshape}{}
Define 3-D array:

\begin{verbatim}
rowVec = 1:8
array = reshape(rowVec, 2, 2, 2);
length(size(array))  % check the dimension
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{1-D Array: Slicing}
\protect\hypertarget{d-array-slicing}{}
\begin{itemize}[<+->]
\item
  Define a row vector \texttt{rowVec}:

\begin{verbatim}
rowVec = [2,4,6,8,10]
rowVec = linspace(2,10,5)
rowVec = colon(2,2,10)    % or rowVec = 2:2:10
\end{verbatim}
\item
  \texttt{array(i)}: the \texttt{i}-th entry of \texttt{array}, where
  \texttt{i} is called the index:

  \begin{table}[!hbtp]
    \centering
    \begin{tabular}{cccccc}
      \toprule
      \verb|i|         & 1 & 2 & 3 & 4 & 5 \\
      \midrule
      \verb|rowVec(i)| & 2 & 4 & 6 & 8 & 10 \\
      \bottomrule
    \end{tabular}
  \end{table}
\end{itemize}
\end{frame}

\begin{frame}[fragile]{1-D Array: Slicing}
\protect\hypertarget{d-array-slicing-1}{}
\begin{table}[!hbtp]
  \centering
  \begin{tabular}{cccccc}
    \toprule
    \verb|i|         & 1 & 2 & 3 & 4 & 5 \\
    \midrule
    \verb|rowVec(i)| & 2 & 4 & 6 & 8 & 10 \\
    \bottomrule
  \end{tabular}
\end{table}

\begin{itemize}[<+->]
\item
  Extract one entry from a vector: For example, to extract \texttt{6}
  from \texttt{rowVec} and assign it to \texttt{x}:

  \texttt{x\ =\ rowVec(3)}
\item
  Extract multiple entries from a vector: For example, to extract
  \texttt{2}, \texttt{6}, \texttt{8} from \texttt{rowVec} and assign it
  to \texttt{x}:

  \texttt{x\ =\ rowVec({[}1,3,4{]})}
\item
  Extract multiple continguous entries from a vector: For example, to
  extract \texttt{4}, \texttt{6}, \texttt{8} from \texttt{rowVec} and
  assign it to \texttt{x}:

\begin{verbatim}
x = rowVec([2,3,4])
x = rowVec(2:4)
\end{verbatim}
\end{itemize}
\end{frame}

\begin{frame}[fragile]{2-D array: Slicing}
\protect\hypertarget{d-array-slicing-2}{}
\begin{itemize}[<+->]
\item
  Define a matrix \texttt{mat}

\begin{verbatim}
mat = reshape(1:8, 2, 4)
\end{verbatim}
\item
  \texttt{array(i,\ j)}: the entry of \texttt{array} at row \texttt{i}
  and column \texttt{j}, where \texttt{i} is colled row index,
  \texttt{j} is called column index:

  \begin{table}[!hbtp]
    \centering
    \begin{tabular}{c|cccc}
      \toprule
      \diagbox{\texttt{i}}{\texttt{mat(i, j)}}{\texttt{j}} & 1 & 2 & 3 & 4 \\
      \hline
      1 & 1 & 3 & 5 & 7 \\
      2 & 2 & 4 & 6 & 8 \\
      \bottomrule
    \end{tabular}
  \end{table}
\end{itemize}
\end{frame}

\begin{frame}[fragile]{2-D array: Slicing}
\protect\hypertarget{d-array-slicing-3}{}
\begin{table}[!hbtp]
  \centering
  \begin{tabular}{c|cccc}
    \toprule
    \diagbox{\texttt{i}}{\texttt{mat(i, j)}}{\texttt{j}} & 1 & 2 & 3 & 4 \\
    \hline
    1 & 1 & 3 & 5 & 7 \\
    2 & 2 & 4 & 6 & 8 \\
    \bottomrule
  \end{tabular}
\end{table}

Extract multiple rows and multiple columns from \texttt{mat}: For
example, to extract entries at row \texttt{1}, row \texttt{2}, and
column \texttt{2}, column \texttt{4}:

\begin{verbatim}
A = mat([1,2], [2,4])
A = mat(1:2, [2,4])
A = mat(1:end, [2,4])
A = mat(:, [2,4])
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Generate a \(3\)-D Array using Slicing}
\protect\hypertarget{generate-a-3-d-array-using-slicing}{}
\begin{verbatim}
slice1 = [1,2;3,4]
slice2 = [5,6;7,8]
C(:,:,1) = slice1
C(:,:,2) = slice2
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Concatenate Arrays}
\protect\hypertarget{concatenate-arrays}{}
\begin{verbatim}
row1 = [1,2,3]
row2 = [4,5,6]
rowVec = [row1,row2]  % rowVec = [1,2,3,4,5,6]
matrix = [row1;row2]  % matrix = [1,2,3;4,5,6]
matrix1 = [matrix1;matrix2]
% same as matrix1 = [1,2,3;4,5,6;1,2,3;4,5,6]
matrix2 = [matrix1,matrix2]
% same as matrix2 = [1,2,3,1,2,3;4,5,6,4,5,6]
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{1-D Array: Append/Delete Element}
\protect\hypertarget{d-array-appenddelete-element}{}
\begin{verbatim}
% 1-D array
rowVec = 1:5
rowVec(end + 1) = 6  % append 6 to rowVec
rowVec = [rowVec,7]  % append 7 to rowVec
rowVec(5) = []       % delete 5 from rowVec
rowVec(2:4) = []     % delete 2, 3, 4 from rowVec
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{2-D Array: Append/Delete Element}
\protect\hypertarget{d-array-appenddelete-element-1}{}
\begin{verbatim}
% 2-D array
matrix = magic(5)
matrix(:, end + 1) = 1:5   % append a column vector
matrix = [matrix,[6:10]']  % append a column vector
matrix(end + 1, :) = 1:7   % append a row vector
matrix = [matrix;8:14]     % append a row vector
matrix(:,6) = []           % delete column 2
matrix(:,3:5) = []         % delete column 3, 4, 5
matrix(2:4,:) = []         % delete row 2, 3, 4
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Char Array vs.~String Array}
\protect\hypertarget{char-array-vs.-string-array}{}
\begin{verbatim}
str = "abc"
arrayOfChars1 = 'abc'
arrayOfChars2 = ['a','b','c']
arrayOfChars1 == arrayOfChars2 % return logical 1 (true)
arrayOfChars1 == str           % return logical 1 (true)
class(str)                     % string
class(arrayOfChars1)           % char
[arrayOfChars1,arrayOfChars2]  % return 'abcabc'
[arrayOfChars1;arrayOfChars2]  % return ['abc';'abc']
[str,str]                      % return ["abc","abc"]
[str;str]                      % return ["abc";"abc"]
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Cell Array: array of elements of different types}
\protect\hypertarget{cell-array-array-of-elements-of-different-types}{}
\begin{itemize}[<+->]
\item
  \texttt{cell(n)}: create 1-D cell array of length \texttt{n}
\item
  \texttt{cell(m,n)}: create 2-D cell array of size \texttt{m} by
  \texttt{n}
\item
  Create a cell array of types \texttt{char}, \texttt{string},
  \texttt{double}:

\begin{verbatim}
cellArray = {[1,2,3], "abc", 'def'}
cellArray{1}          % return [1,2,3]
cellArray{2}          % return "abc"
cellArray{3}          % return 'def'
cellArray{4} = 'ghi'
cellArray{4}          % return 'ghi'
\end{verbatim}
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Array Operations: 1-D Array}
\protect\hypertarget{array-operations-1-d-array}{}
\begin{itemize}[<+->]
\tightlist
\item
  \texttt{sum(vec)}/\texttt{prod(vec)}: sum/product of all elements of
  \texttt{vec}.
\item
  \texttt{max(vec)}/\texttt{min(vec)}: maximum/minimum of \texttt{vec}.
\item
  \texttt{rowVec\ =\ rowVec1\ .*\ rowVec2}: elementwise multiplication,
  where \texttt{rowVec(i)\ =\ rowVec1(i)\ *\ rowVec2(i)}.
\item
  \texttt{rowVec\ .*\ colVec}: Kronecker product. If \texttt{rowVec} has
  length \texttt{m} and \texttt{colVec} has length \texttt{n}, then the
  resulting matrix is \texttt{m}-by-\texttt{n}.
\item
  \texttt{dot(vec1,\ vec2)}: dot product of \texttt{vec1} and
  \texttt{vec2}, \texttt{vec1} and \texttt{vec2} must be of the same
  length.
\item
  \texttt{sum(rowVec1\ .*\ rowVec2)}: \texttt{dot(rowVec1,\ rowVec2)}.
\item
  \texttt{rowVec1\ *\ rowVec2\textquotesingle{}}:
  \texttt{dot(rowVec1,\ rowVec2)}.
\item
  \texttt{indices\ =\ find(vec\ \textgreater{}\ n)}: find indices of
  elements greater than \texttt{n} in \texttt{vec}. Note:
  \texttt{\textgreater{}} can also be \texttt{\textless{}}, \texttt{==}.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Array Operations: 2-D Array}
\protect\hypertarget{array-operations-2-d-array}{}
\begin{itemize}[<+->]
\tightlist
\item
  \texttt{mat\ =\ mat1\ .*\ mat2}: elementwise multiplication, where
  \texttt{mat(i,\ j)\ =\ mat1(i,\ j)\ *\ mat2(i,\ j)}.
\item
  \texttt{mat\ =\ mat1\ *\ mat2}: matrix multiplication, where
  \texttt{mat1} is \texttt{m}-by-\texttt{p}, \texttt{mat2} is
  \texttt{p}-by-\texttt{n}, and \texttt{mat} is
  \texttt{m}-by-\texttt{n}.
\item
  \texttt{sum/prod(mat,\ \textquotesingle{}all\textquotesingle{})}:
  sum/product of all elements of \texttt{mat}.
\item
  \texttt{sum/prod(mat,\ 1)}: column sums/products.
\item
  \texttt{sum/prod(mat,\ 2)}: row sums/products.
\item
  \texttt{max/min(mat,\ {[}{]},\ \textquotesingle{}all\textquotesingle{})}:
  maximum/minimum of \texttt{mat}.
\item
  \texttt{max/min(mat,\ {[}{]},\ 1)}: column maximums/minimums.
\item
  \texttt{max/min(mat,\ {[}{]},\ 2)}: row maximums/minimums.
\item
  \texttt{{[}row,\ col{]}\ =\ find(mat\ \textgreater{}\ n)}: find
  indices of elements greater than \texttt{n} in \texttt{mat},
  \texttt{row}/\texttt{col} stores row/column indices.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Array Operations: 2-D Array}
\protect\hypertarget{array-operations-2-d-array-1}{}
\begin{itemize}[<+->]
\tightlist
\item
  \texttt{{[}V,\ D{]}\ =\ eig(mat)}: \texttt{V(:,\ i)} and
  \texttt{D(i,\ i)} are the \texttt{i}-th eigenvector and eigenvalue of
  \texttt{mat}.
\item
  \texttt{d\ =\ diag(mat,\ k)}: extract \texttt{k}-th diagonal elements
  that is above (\texttt{k\ \textgreater{}\ 0}) / below
  (\texttt{k\ \textless{}\ 0}) the main diagonal.
\item
  \texttt{mat\ =\ diag(d,\ k)}: construct a matrix with \texttt{k}-th
  diagonal elements being \texttt{d}.
\item
  \texttt{mat\ =\ diag(diag(mat,\ k))}: set elements to zero except the
  \texttt{k}-th diagonal elements.
\item
  \texttt{fliplr(mat)}: flip \texttt{mat} in left/right direction.
\item
  \texttt{flipud(mat)}: flip \texttt{mat} in up/down direction.
\item
  \texttt{rot90(mat,\ k)}: rotate \texttt{mat} \texttt{k\ *\ 90}
  degrees.
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Application: Image Processing}
\protect\hypertarget{application-image-processing}{}
\begin{itemize}[<+->]
\item
  A grayscale image is a 2-D array of pixels, each pixel has a integer
  value that represent depth of color.
\item
  A colored image is a 3-D array of pixels with RGB channels, each
  channel is a 2-D array.
\item
  \texttt{img\ =\ imread(filename)}: read image from graphics file
  \texttt{filename} and assign it \texttt{img}.
\item
  \texttt{imshow(img)}: display image \texttt{img} in handle graphics
  figure.
\item
  \texttt{imwrite(img,\ filename)}: write image \texttt{img} to graphics
  file named \texttt{filename}.

\begin{verbatim}
uw = imread('UW.png');
uwFlipud = flipud(uw);
imshow(uwFlipud);
imwrite(uwFlipud, 'UW_flipud.png');
\end{verbatim}
\end{itemize}
\end{frame}

\begin{frame}{Summary}
\protect\hypertarget{summary}{}
\begin{table}[!hbtp]
  \begin{tabular}{rl}
    Command                          & Description \\
    \hline
    \texttt{transpose} or \texttt{'} & Non-conjugate transpose of a vector \\
    \texttt{linspace}                & Linearly spaced vector \\
    \texttt{logspace}                & Logarithmically spaced vector \\
    \texttt{colon} or \texttt{:}     & Colon \\
    \texttt{zeros}                   & Zeros array \\
    \texttt{ones}                    & Ones array \\
    \texttt{eye}                     & Identity matrix \\
    \texttt{rand}                    & Uniformly distributed pseudorandom numbers \\
    \texttt{randn}                   & Normally distributed pseudorandom numbers \\
    \texttt{magic}                   & Magic square \\
    \texttt{size}                    & Size of array \\
    \texttt{length}                  & Length of vector \\
    \texttt{reshape}                 & Reshape array \\
  \end{tabular}
\end{table}
\end{frame}

\begin{frame}{Summary}
\protect\hypertarget{summary-1}{}
\begin{table}[!hbtp]
  \begin{tabular}{rl}
    Command                          & Description \\
    \hline
    \texttt{diag}                    & Diagonal matrices and diagonals of a matrix \\
    \texttt{cell}                    & Create cell array \\
    \texttt{sum}/\texttt{prod}       & Sum/Product of elements \\
    \texttt{min}/\texttt{max}        & Minimum/Maximum of elements \\
    \texttt{dot}                     & Vector dot product \\
    \texttt{find}                    & Find indices of nonzero elements \\
    \texttt{eig}                     & Find eigenvalues and eigenvectors \\
    \texttt{diag}                    & Diagonal matrices and diagonals of a matrix \\
    \texttt{fliplr}/\texttt{flipud}  & Flip an array \\
    \texttt{rot90}                   & Rotate an array 90 degrees \\
    \texttt{imread}/\texttt{imwrite} & Read/Write image from graphics file \\
    \texttt{imshow}                  & display image in Handle Graphics figure \\
    \texttt{uint8}                   & Convert to unsigned 8-bit integer \\
  \end{tabular}
\end{table}
\end{frame}

\begin{frame}{Additional Commands}
\protect\hypertarget{additional-commands}{}
\begin{table}[!hbtp]
  \begin{tabular}{rl}
    Command            & Description \\
    \hline
    \texttt{iskeyword} & Check if input is a keyword \\
    \texttt{who}       & List current variables \\
    \texttt{whos}      & List current variables, long form \\
    \texttt{which}     & Locate functions and files \\
    \texttt{clear}     & Clear variables and functions from memory \\
    \texttt{clc}       & Clear command window \\
    \texttt{clf}       & Clear current figure \\
    \texttt{close}     & Close figure \\
    \texttt{exist}     & Check existence of variable/script/function/folder/class \\
    \texttt{disp}      & Display array \\
  \end{tabular}
\end{table}
\end{frame}

\hypertarget{script-files}{%
\subsection{Script Files}\label{script-files}}

\begin{frame}{}
\protect\hypertarget{section-1}{}
A script file is simply a file that contains a chain of commands that
you edit in a separate window, then execute with a single mouse click or
command. This is where we can define variables, perform calculations and
leave comments to remind us what the file calculates.
\end{frame}

\begin{frame}{File Naming Conventions}
\protect\hypertarget{file-naming-conventions}{}
\begin{itemize}[<+->]
\tightlist
\item
  Start with a letter, followed by letters or numbers or underscore,
  maximum 63 characters (excluding the .m extension), and must not be
  the same as any MATLAB reserved word.
\item
  None of the conventions matter to MATLAB itself: they only matter to
  the people writing the code, and the people maintaining the code
  (usually a much harder task), and to the people paying for the code
  (you'd be amazed how much gets written into contract specifications.)
\item
  Reference:
  \url{https://www.mathworks.com/matlabcentral/answers/30223-what-are-the-rules-for-naming-script-files}
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Put Comments to Your Script File}
\protect\hypertarget{put-comments-to-your-script-file}{}
\begin{verbatim}
% MATH 3341, Semester Year
% Lab 02: Variables, Arrays, and Scripts
% Author: first_name last_name
% Date: mm/dd/yyyy
\end{verbatim}
\end{frame}

\begin{frame}{Useful MATLAB Shortcuts}
\protect\hypertarget{useful-matlab-shortcuts}{}
\begin{itemize}[<+->]
\tightlist
\item
  Windows shortcuts

  \begin{itemize}[<+->]
  \tightlist
  \item
    Press \fbox{\texttt{Ctrl}} + \fbox{\texttt{A}} to select all
  \item
    Press \fbox{\texttt{Ctrl}} + \fbox{\texttt{I}} to adjust indentation
  \item
    Press \fbox{\texttt{Ctrl}} + \fbox{\texttt{R}} to comment
  \item
    Press \fbox{\texttt{Ctrl}} + \fbox{\texttt{T}} to uncomment
  \end{itemize}
\item
  macOS shortcuts

  \begin{itemize}[<+->]
  \tightlist
  \item
    Press \fbox{\texttt{command}} + \fbox{\texttt{A}} to select all
  \item
    Press \fbox{\texttt{command}} + \fbox{\texttt{I}} to adjust
    indentation
  \item
    Press \fbox{\texttt{command}} + \fbox{\texttt{/}} to comment
  \item
    Press \fbox{\texttt{command}} + \fbox{\texttt{T}} to uncomment
  \end{itemize}
\end{itemize}
\end{frame}

\hypertarget{primer}{%
\subsection{\texorpdfstring{\LaTeX~Primer}{~Primer}}\label{primer}}

\begin{frame}[fragile]{\texttt{table} Environment}
\protect\hypertarget{table-environment}{}
\begin{verbatim}
\begin{table}[!hbtp]
  \caption{This is a table}
  \begin{tabular}{rcl}
  \toprule
  Column 1 & Column 2 & Column 3 \\
  \midrule
  1        & 1        & 1        \\
  12       & 12       & 12       \\
  123      & 123      & 123      \\
  \bottomrule
  \end{tabular}
\end{table}
\end{verbatim}
\end{frame}

\begin{frame}{\texttt{table} Environment}
\protect\hypertarget{table-environment-1}{}
\begin{table}[!hbtp]
  \caption{This is a table}
  \begin{tabular}{rcl}
  \toprule
  Column 1 & Column 2 & Column 3 \\
  \midrule
  1        & 1        & 1        \\
  12       & 12       & 12       \\
  123      & 123      & 123      \\
  \bottomrule
  \end{tabular}
\end{table}
\end{frame}

\begin{frame}[fragile]{\texttt{figure} Environment}
\protect\hypertarget{figure-environment}{}
\begin{verbatim}
\begin{figure}[!hbtp]
  \centering
  \includegraphics[height=0.3\textheight]{figure.pdf}
  \caption{Plot of $\sin{x}$}
  \label{fig:sin}
\end{figure}
\end{verbatim}

generates

\begin{figure}[!hbtp]
  \centering
  \includegraphics[height=0.3\textheight]{figure.pdf}
  \caption{Plot of $\sin{x}$}
  \label{fig:sin}
\end{figure}
\end{frame}

\begin{frame}[fragile]{\texttt{\textbackslash{}left} and
\texttt{\textbackslash{}right} vs.~\texttt{\textbackslash{}big},
\texttt{\textbackslash{}Big}, \texttt{\textbackslash{}Bigg}}
\protect\hypertarget{left-and-right-vs.-big-big-bigg}{}
\begin{verbatim}
\begin{align*}
\|x\|_2 & = \big(\sum_{i = 1}^{n} x_i^2 \big)^{1/2},
\|x\|_2 = \Big(\sum_{i = 1}^{n} x_i^2 \Big)^{1/2}, \\
\|x\|_2 & = \Bigg(\sum_{i = 1}^{n} x_i^2 \Bigg)^{1/2},
\|x\|_2 = \left(\sum_{i = 1}^{n} x_i^2 \right)^{1/2}.
\end{align*}
\end{verbatim}

generates \begin{align*}
\|x\|_2 & = \big(\sum_{i = 1}^{n} x_i^2 \big)^{1/2},
\|x\|_2 = \Big(\sum_{i = 1}^{n} x_i^2 \Big)^{1/2}, \\
\|x\|_2 & = \Bigg(\sum_{i = 1}^{n} x_i^2 \Bigg)^{1/2},
\|x\|_2 = \left(\sum_{i = 1}^{n} x_i^2 \right)^{1/2}.
\end{align*}
\end{frame}

\begin{frame}[fragile]{Links}
\protect\hypertarget{links}{}
\begin{verbatim}
\href{https://www.google.com}{Google}
\end{verbatim}

\href{https://www.google.com}{Google}

Or simply

\begin{verbatim}
\url{https://www.google.com}
\end{verbatim}

\url{https://www.google.com}
\end{frame}

\begin{frame}[fragile]{\texttt{case} Environment}
\protect\hypertarget{case-environment}{}
\begin{verbatim}
$$
f(x) =
\begin{cases}
5 x + 4   & \text{if~} x \leq 1, \\
3 x^2 + 6 & \text{if~} x > 1
\end{cases}
$$
\end{verbatim}

generates \[
f(x) =
\begin{cases}
5 x + 4   & \text{if~} x \leq 1, \\
3 x^2 + 6 & \text{if~} x > 1
\end{cases}
\]
\end{frame}

\begin{frame}[fragile]{Cross-Reference}
\protect\hypertarget{cross-reference}{}
\begin{verbatim}
\begin{equation}
\label{eq:ls}
A \mathbf{x} = \mathbf{b}.
\end{equation}

The expression \eqref{eq:ls} is a linear system.
\end{verbatim}

generates

\begin{equation}
\label{eq:ls}
A \mathbf{x} = \mathbf{b}.
\end{equation}

The expression \eqref{eq:ls} is a linear system.
\end{frame}

\begin{frame}[fragile]{Cross-Reference}
\protect\hypertarget{cross-reference-1}{}
\begin{verbatim}
\begin{table}[!hbtp]
\caption{$y = 2x$}
\label{tab:xy}
  \begin{tabular}{cc}
  \toprule
  $x$ & $y$ \\
  \midrule
  $6$ & $12$ \\
  $7$ & $14$ \\
  $8$ & $16$ \\
  \bottomrule
  \end{tabular}
\end{table}
Table \ref{tab:xy} gives the result of $y = 2x$.
\end{verbatim}
\end{frame}

\begin{frame}{Cross-Reference}
\protect\hypertarget{cross-reference-2}{}
\begin{table}[!hbtp]
\caption{$y = 2x$}
\label{tab:xy}
  \begin{tabular}{cc}
  \toprule
  $x$ & $y$ \\
  \midrule
  $6$ & $12$ \\
  $7$ & $14$ \\
  $8$ & $16$ \\
  \bottomrule
  \end{tabular}
\end{table}

Table \ref{tab:xy} gives the result of \(y = 2x\).
\end{frame}


\documentclass[english]{article}
\usepackage[T1]{fontenc}
\usepackage[latin1]{inputenc}
\usepackage{geometry}
\geometry{verbose,letterpaper,tmargin=1in,bmargin=1in,lmargin=1in,rmargin=1in}
\usepackage{amsmath}

\makeatletter
\usepackage{babel}
\makeatother
\begin{document}

\section{FEM model}

\begin{quote}
The problem is

\begin{align*}
\Delta\bar{p}-\frac{1}{c_{0}^{2}}\frac{\partial^{2}\bar{p}}{\partial^{2}t} & =f\end{align*}

with initial condition

\begin{eqnarray*}
\bar{p}(0,\mathbf{r}) & = & b(r)\end{eqnarray*}

Let

\begin{eqnarray*}
v & = & \frac{\partial\bar{p}}{\partial t}\end{eqnarray*}

then we have

\begin{alignat*}{1}
\bar{p}_{t}-v & =0\\
\Delta\bar{p}-\frac{1}{c_{0}^{2}}\, v_{t} & =f\end{alignat*}

and absorbing boundary condition

\begin{eqnarray*}
\frac{\partial\bar{p}}{\partial\mathbf{n}} & =- & \frac{1}{c_{0}}\frac{\partial\bar{p}}{\partial t}\end{eqnarray*}

$\frac{\partial\bar{p}}{\partial\mathbf{n}}$ is the normal derivative
at the boundary. This is a the time-varying FEM model. by discretizing
according to $t$, we have

\begin{eqnarray*}
(\frac{\bar{p}^{n}-\bar{p}^{n-1}}{\delta t},\phi)_{\Omega}-\,(\theta v^{n}+(1-\theta)v^{n-1},\phi)_{\Omega} & = & 0\\
-(\Delta((\theta\bar{p}^{n}+(1-\theta)\bar{p}^{n-1}),\bigtriangledown\phi)_{\Omega}-\frac{1}{c_{0}}(\frac{\bar{p}^{n}-\bar{p}^{n-1}}{\delta t},\phi)_{\partial\Omega}-\frac{1}{c_{0}^{2}}(\frac{v^{n}-v^{n-1}}{\delta t},\phi)_{\Omega} & = & (\theta f^{n}+(1-\theta)f^{n-1},\phi)_{\Omega}\end{eqnarray*}

we obtain

\begin{eqnarray*}
M\bar{p}^{n}-(\delta t\,\theta)Mv^{n} & = & M\bar{p}^{n-1}+\delta t\,(1-\theta)\, M\, v^{n-1}\\
(-c_{0}^{2}\,\delta t\,\theta A-c_{0}\, B)\bar{p}^{n}-Mv^{n} & = & (c_{0}^{2}\,\delta t\,(1-\theta)A-c_{0}B)\bar{p}^{n-1}-M\, v^{n-1}+c_{0}^{2}\delta t(\theta F^{n}+(1-\theta)F^{n-1})\end{eqnarray*}

Write the above two equations as a matrix form

\begin{eqnarray*}
\left(\begin{array}{cc}
M & -(\delta t\,\theta)M\\
c_{0}^{2}\,\delta t\,\theta A+c_{0}\, B & M\end{array}\right)\left(\begin{array}{c}
\bar{p}^{n}\\
v^{n}\end{array}\right) & = & \left(\begin{array}{c}
G_{1}\\
G_{2}\end{array}\right)\end{eqnarray*}

where

\begin{center}$\left(\begin{array}{c}
G_{1}\\
G_{2}\end{array}\right)=\left(\begin{array}{c}
M\bar{p}^{n-1}+\delta t\,(1-\theta)Mv^{n-1}\\
(-c_{0}^{2}\,\delta t\,(1-\theta)A+c_{0}B)\bar{p}^{n-1}+M\, v^{n-1}-c_{0}^{2}\delta t(\theta F^{n}+(1-\theta)F^{n-1})\end{array}\right)$\end{center}

From the above matrix, we can obtain

\begin{eqnarray*}
(M+(\delta t\,\theta\, c_{0})^{2}A+c_{0}\,\delta t\,\theta\, B)\bar{p}^{n} & = & G_{1}+(\delta t\,\theta)G_{2}\\
Mv^{n} & = & -(c_{0}^{2}\,\delta t\,\theta\, A+c_{0}B)\bar{p}^{n}+G_{2}\end{eqnarray*}

\end{quote}

\end{document}


\documentclass{article}
\usepackage{amsfonts}
\usepackage{mathtools}
\usepackage{amsthm}
\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\usepackage{proof}
\usepackage{cite}
\usepackage{hyperref}

\begin{document}

\title{CS292C Final Project proof of type soundness}
\author{Daniel Zhang}
\maketitle

\section{Introduction}

For this project I've decided to prove type soundness for and implement the Calculus of Constructions. Extensions of the Calculus of Constructions are used in Coq and Agda \cite{Casinghino10}, so understanding it will help with understanding those systems. I will not be describing extensions such as inductive types or universes.

The Calculus of Constructions is a Pure Type System. Pure Type Systems have a set of sorts. The Calculus of Constructions has three sorts: terms, types and kinds (the type of types).

$\star$ is a constant of sort kind. Note that it is not the only kind. $\star\rightarrow\star$ is a different kind. This corresponds to the type of a type constructor: a function that takes in a type and outputs a type.

The notation $\lambda x:A.B$ is used both for a function that takes in a term and a function that takes in a type.
$\Pi x:A.B$ is a dependent product. If $x$ does not occur in $B$, it is the same as $A\rightarrow B$. When $A$ is $\star$, it is equivalent to a universal type. \cite{Jones97}

An example of a term would be $\lambda T:*.(\lambda x:T.x)$. Types include expressions such as $\lambda x:\star.x$ and $\Pi x:\star.x$. An example of a kind would be $\Pi x:\star.\star$.

The Calculus of Constructions is the most powerful system in the Lambda Cube. Both types and terms can depend on types and terms.
The Calculus of Constructions is known to be strongly-normalizing, but the proof is considered difficult \cite{Casinghino10}.

\section{Syntax}

Expressions are considered equivalent up to $\alpha$-conversion.
Variables names are assumed to all be distinct (implementation uses de Bruijn indices, so substitution doesn't cause issues).

\[k\in \text{Const} \rightarrow \star \mid \square\]
\[e\in \text{Exp}\rightarrow k\in \text{Const} \mid x\in \text{Variable} \mid A B \mid \lambda x:A.B \mid \Pi x:A.B\]
This is the syntax for both terms and types.

\[v\in \text{Values}\rightarrow k\in \text{Const} \mid x\in \text{Variable} \mid \lambda x:A.B \mid \Pi x: A. B\]
where $A,B\in\text{Values}$.
\section{Semantics}

These rules are from \cite{Casinghino10}, modifed to be deterministic.

\[\infer[(LAM1)]{(\lambda x:A.B)\rightarrow (\lambda x:A'.B)}{A\rightarrow A'}\]
\[\infer[\text{if $A\in\text{Values}$}(LAM2)]{(\lambda x:A.B)\rightarrow (\lambda x:A.B')}{B\rightarrow B'}\]
\[\infer[(PI1)]{(\Pi x:A.B)\rightarrow (\Pi x:A'.B)}{A\rightarrow A'}\]
\[\infer[\text{if $A\in\text{Values}$}(PI2)]{(\Pi x:A.B)\rightarrow (\Pi x:A.B')}{B\rightarrow B'}\]
\[\infer[\text{if APP3 cannot apply}(APP1)]{A B\rightarrow A' B}{A\rightarrow A'}\]
\[\infer[\text{if $A\in \text{Values}$ and APP3 cannot apply}(APP2)]{A B\rightarrow A B'}{B\rightarrow B'}\]
\[\infer[(APP3)]{(\lambda x:A.B) C\rightarrow B[x\mapsto C]}{}\]

\section{Substitution}

\[k[x\mapsto D]=k\]
\[
y[x\mapsto D]=
\begin{cases}
  D & \text{if $x=y$}\\
  y & \text{if $x\ne y$}
\end{cases}
\]
\[(A B)[x\mapsto D]=A[x\mapsto D] B[x\mapsto D]\]
\[(\lambda y:A.B)[x\mapsto D]=\lambda y:A[x\mapsto D].B[x\mapsto D]\]
\[(\Pi y:A.B)[x\mapsto D]=\Pi y:A[x\mapsto D].B[x\mapsto D]\]

\[(\Gamma,y:B)[x\mapsto D]=\Gamma[x\mapsto D],y:B[x\mapsto D]\]

\section{Typing rules}

\[\infer[(STAR)]{\vdash\star:\square}{}\]
\[\infer[(VAR)]{\Gamma,x:A\vdash x:A}{\Gamma\vdash A:s}\]
\[\infer[(WEAK)]{\Gamma,x:A\vdash B:C}{\Gamma\vdash B:C & \Gamma\vdash A:s}\]
\[\infer[(APP)]{\Gamma\vdash f a:B[x\mapsto a]}{\Gamma\vdash f:(\Pi x:A.B) & \Gamma\vdash a:A}\]
\[\infer[(LAM)]{\Gamma\vdash(\lambda x:A.b):(\Pi x:A.B)}{\Gamma,x:A\vdash b:B & \Gamma\vdash(\Pi x:A.B):t}\]
\[\infer[(PI)]{\Gamma\vdash(\Pi x:A.B):t}{\Gamma\vdash A:s & \Gamma,x:A\vdash B:t & s,t\in\{\star,\square\}}\]
\[\infer[(CONV)]{\Gamma\vdash a:B}{\Gamma\vdash a:A & \Gamma\vdash B:s & A=_\beta B}\]

\[\infer[(=_\beta LAM)]{(\lambda x:A.B)C=_\beta B[x\mapsto C]}{}\]
and its symmetric transitive closure.

These rules are taken from \cite{Jones97} with some modifications. The notation for substitution was changed to match what was used in this class. The version in the paper had a relation on $s$ and $t$ in rule (PI) to restrict to other type systems in the Lambda Cube. The eight systems of the Lambda Cube correspond to all choices of pairs of valid $(s,t)$ in rule PI, where $(\star,\star)$ must be taken. The Calculus of Constructions allows all pairs of sorts on $\{\star,\square\}$.

\section{Proof of progress}

\begin{lemma}
  Let $e$ be a value.
  If $\Gamma\vdash e:\Pi x:A.B$, then $e=(\lambda x:C.D)$.
\end{lemma}

\begin{proof}
  Since $e$ is a value, it must either be of the form $\lambda x:C.D$ or $\Pi x:C.D$. Let $T$ the last type before trailing applications of CONV or WEAK: $\Gamma'\vdash e:T$ and $T=_\beta \Pi x.A:B$. If $e=\Pi x:C.D$, then $T\in\{\star,\square\}$ since $PI$ is the only rule that could be applied based on the form of $e$. But since neither of $\star$ or $\square$ is $\beta$-equivalent to $\Pi x:C.D$, this is a contradiction. Therefore, $e=\lambda x:C.D$.

\end{proof}

\begin{theorem}
  $(\forall e \in Exp)(\vdash e:\tau)\Rightarrow(e \in Values\vee(\exists e'\in Exp)(e\rightarrow e'))$
\end{theorem}

\begin{proof}
  Use structural induction on $e$:

  Case $e\in Const$: $e\in Values$

  Case $e\in Variable$: A variable can only have a type if it is in the context. Since we are in the empty context, a variable cannot be well-typed. Thus, this case cannot happen.

  Case $e=A B$:
  \begin{itemize}
  \item If $A$ is not a value, then $A\rightarrow A'$ by the inductive hypothesis. $e\rightarrow e'$ where $e'=A' B$ (rule APP1).
  \item If $A$ is a value but $B$ is not a value, then $B\rightarrow B'$ by the inductive hypothesis. Thus, $e\rightarrow e'$ where $e'=A B'$ (rule APP2).
  \item If $A$ and $B$ are both values: After removing trailing application of CONV from the typing derivation for $\tau$, we get $A B=\tau'$ where $\tau=_\beta \tau'$. The last rule used in the typing derivation for $\tau'$ is not CONV, so it must be APP. Thus, $A:\Pi x: C.D$. Then by Lemma 1, $A=(\lambda x:\tau.e_3)$, so $e=(\lambda x:\tau.e_3)B$. Finally, $e\rightarrow e'$ where $e'=e_3[x\mapsto B]$ (rule APP3).
  \end{itemize}

  Case $e=(\lambda x:A.B)$: If $A$ is not a value, then $A\rightarrow A'$ by the inductive hypothesis. $e\rightarrow e'$ where $e'=(\lambda x:A'.B)$. Otherwise, if $B$ is not a value, then $B\rightarrow B'$ by the inductive hypothesis. $e\rightarrow e'$ where $e'=(\lambda x:A.B')$. If both $A$ and $B$ are values, then $e\in Values$.

  Case $e=(\Pi x:A.B)$: If $A$ is not a value, then $A\rightarrow A'$ by the inductive hypothesis. $e\rightarrow e'$ where $e'=(\Pi x:A'.B)$. Otherwise, if $B$ is not a value, then $B\rightarrow B'$ by the inductive hypothesis. $e\rightarrow e'$ where $e'=(\Pi x:A.B')$. If both $A$ and $B$ are values, then $e\in Values$.

  This exhausts all the cases, so the theorem is true by induction.

\end{proof}

\section{Proof of preservation}

Compared to simply-typed lambda calculus, a major differences in the proving the Substitution Lemma is the type of a variable can depend on the value of another variable. This means order of variables in the context matters. Also, we must use the version of substitution that allows substituting arbitrary terms because rule APP requires it.

\begin{lemma}[Substitution Lemma for Terms]
  If $y$ is fresh in $D$,
  \[e[x\mapsto D][y\mapsto C[x\mapsto D]]=e[y\mapsto C][x\mapsto D]\]
\end{lemma}
\begin{proof}
  The proof is by induction on the structure of $e$.

  Case $e=k$: $k[x\mapsto D][y\mapsto C[x\mapsto D]]=k=k[y\mapsto C][x\mapsto D]$, as desired.

  Case $e=A B$:
  \begin{align*}
    (A B)[x\mapsto D][y\mapsto C[x\mapsto D]]&=A[x\mapsto D][y\mapsto C[x\mapsto D]]B[x\mapsto D][y\mapsto C[x\mapsto D]]\\
    &=A[y\mapsto C][x\mapsto D]B[y\mapsto C][x\mapsto D]\\
    &=(A B)[y\mapsto C][x\mapsto D]
  \end{align*}
  Using the inductive hypothesis on $A$ and $B$.

  Case $e=(\lambda x:A.B)$: identical to $A B$.

  Case $e=(\Pi x:A.B)$: identical to $A B$.

  Case $e=x$:
  \begin{align*}
    x[x\mapsto D][y\mapsto C[x\mapsto D]]&=D[y\mapsto C[x\mapsto D]]\\
    &=D\\
    &=x[x\mapsto D]\\
    &=x[y\mapsto C][x\mapsto D]
  \end{align*}

  Case $e=y$ where $y\ne x$:
  \begin{align*}
    y[x\mapsto D][y\mapsto C[x\mapsto D]]&=y[y\mapsto C[x\mapsto D]]\\
    &=C[x\mapsto D]\\
    &=y[y\mapsto C][x\mapsto D]
  \end{align*}

\end{proof}

\begin{lemma}[Substitution preserves $\beta$-equivalence]
  \[\text{If $A=_\beta B$, then $A[x\mapsto D]=_\beta B[x\mapsto D]$}\]
\end{lemma}
\begin{proof}
  It suffices to show that $((\lambda y:A.B)C)[x\mapsto D]=_\beta(B[y\mapsto C])[x\mapsto D]$

  \begin{align*}
  ((\lambda y:A.B)C)[x\mapsto D]&=((\lambda y:A.B)[x\mapsto D]C[x\mapsto D])\\
    &=((\lambda y:A[x\mapsto D].B[x\mapsto D])C[x\mapsto D])\\
    &=_\beta B[x\mapsto D][y\mapsto C[x\mapsto D]]\\
    &=B[y\mapsto C][x\mapsto D]
  \end{align*}
  since $y$ is fresh in $D$ (Substitution Lemma for Terms).
\end{proof}

\begin{lemma}[Substitution Lemma]
  \[\text{If $\Gamma,x:A,\Delta\vdash B:C$ and $\Gamma\vdash D:A$, then $\Gamma,\Delta[x\mapsto D]\vdash B[x\mapsto D]:C[x\mapsto D]$}\]
\end{lemma}
This is Lemma 5.2.1 in \cite{Barendregt92}.
\begin{proof}
  Use induction on the length of the derivation of $\Gamma,x:A,\Delta,\vdash B:C$.

  Abbreviate $M[x\mapsto D]$ as $M^*$. Note if $x$ is fresh in $M$, then $M=M^*$.

  Look at the last rule used in the derivation of $\Gamma,x:A,\Delta,\vdash B:C$.

  Case STAR: This case cannot apply because STAR only applies in the empty context, but the context contains $x$.

  Case WEAK: If $\Delta=<>$, the last rule was
  \[\infer{\Gamma,x:A\vdash B:C}{\Gamma\vdash B:C & \Gamma\vdash A:s}\]
  Since $x$ is fresh in $B$ and $C$, $B=B^*$, and $C=C^*$. Thus,
  \[\Gamma\vdash B^*:C^*\]
  as desired.
  On the other hand, if $\Delta=\Delta',y:E$, the last rule was
  \[\infer{\Gamma,x:A,\Delta',y:E\vdash B:C}{\Gamma,x:A,\Delta'\vdash B:C & \Gamma\vdash E:s}\]

  By the inductive hypothesis, $\Gamma,\Delta'^*\vdash B^*:C^*$. Also, since $x$ is fresh in $E$, $E=E^*$. By rule WEAK,

  \[\infer{\Gamma,\Delta'^*,y:E^*\vdash B^*:C^*}{\Gamma,\Delta'^*\vdash B^*:C^* & \Gamma\vdash E^*:s}\]

  Since $\Delta^*=\Delta'^*,y:E^*$,
  \[\Gamma,\Delta^*\vdash B^*:C^*\]
  as desired.

  Case VAR: If $\Delta=<>$, the last rule was
  \[\infer{\Gamma,x:A\vdash x:A}{\Gamma\vdash A:s}\]
  Since $\Gamma\vdash D:A$, $x^*=D$, and $A=A^*$,
  \[\Gamma\vdash x^*:A^*\]
  as desired.
  On the other hand, if $\Delta=\Delta',y:E$, the last rule was
  \[\infer{\Gamma,x:A,\Delta',y:E\vdash y:E}{\Gamma,x:A,\Delta'\vdash E:s}\]
  By the inductive hypothesis,
  \[\Gamma,\Delta'^*\vdash E^*,s^*\]
  Now apply rule VAR:
  \[\infer{\Gamma,\Delta'^*,y:E^*\vdash y:E^*}{\Gamma,\Delta'^*\vdash E^*,s^*}\]
  Since $\Delta^*=\Delta'^*,y:E^*$ and $y=y^*$,
  \[\Gamma,\Delta^*\vdash y^*:E^*\]

  as desired.

  Case PI: The last rule was
  \[\infer{\Gamma,x:A,\Delta\vdash(\Pi y:E.F):t}{\Gamma,x:A,\Delta\vdash A:s & \Gamma,x:A,\Delta,y:E\vdash F:t}\]
  By the inductive hypothesis, $\Gamma,\Delta^*\vdash A^*:s^*$ and $\Gamma,\Delta^*,y:E^*\vdash F^*:t^*$
  Applying rule PI, we get
  \[\infer{\Gamma,\Delta^*\vdash (\Pi y:E^*:F^*):t^*}{\Gamma,\Delta^*\vdash A^*:s^* & \Gamma,\Delta^*,y:E^*\vdash F^*:t^*}\]
  as desired

  Case LAM: The last rule was
  \[\infer{\Gamma,x:A,\Delta\vdash(\lambda y:E.f):(\Pi y:E.F)}{\Gamma,x:A,\Delta,y:E\vdash f:F & \Gamma,x:A,\Delta\vdash(\Pi y:E.F):t}\]
  By the inductive hypothesis, $\Gamma,\Delta^*,y:E^*\vdash f^*:F^*$ and $\Gamma,\Delta^*\vdash(\Pi y:E^*.F^*):t^*$
  Applying rule LAM,
  \[\infer{\Gamma,\Delta^*\vdash(\lambda y:E^*,f^*):(\Pi y:E^*:F^*)}{\Gamma,\Delta^*,y:E^*\vdash f^*:F^* & \Gamma,\Delta^*\vdash(\Pi y:E^*.F^*):t^*}\]
  as desired.

  Case CONV: The last rule was
  \[\infer{\Gamma,x:A,\Delta\vdash e:F}{\Gamma,x:A,\Delta\vdash e:E & \Gamma,x:A,\Delta\vdash F:s & E=_\beta F}\]
  By the inductive hypothesis, $\Gamma,\Delta^*\vdash e^*:E^*$ and $\Gamma,\Delta^*\vdash F^*:s^*$. Also, $E^*=_\beta F^*$ since substitution preserves $\beta$-equivalence. By rule LAM,
  \[\infer{\Gamma,\Delta^*\vdash e^*:F^*}{\Gamma,\Delta^*\vdash e^*:E^* & \Gamma,\Delta^*\vdash F^*:s^* & E^*=_\beta F^*}\]

  This exhausts all the cases, so the lemma is true by induction.
\end{proof}

\begin{theorem}[Preservation]
  $(\Gamma\vdash A:B)\wedge(A\rightarrow A')\Rightarrow(\Gamma\vdash A':B)$
\end{theorem}

This is based on Theorem 5.2.15 in \cite{Barendregt92}.

\begin{proof}
  Because the types of variables can contains other variables, we cannot just prove this by induction. Instead, we need to prove two statements simultaneously by induction on generation of $\Gamma\vdash A:B$:
  \[(\Gamma\vdash e:\tau)\wedge(e\rightarrow e')\Rightarrow(\Gamma\vdash e':\tau)\]
  \[(\Gamma\vdash e:\tau)\wedge(\Gamma\rightarrow \Gamma')\Rightarrow(\Gamma'\vdash e:\tau)\]
  where $\Gamma\rightarrow\Gamma'$ if $\Gamma=x_1:A_1,\ldots x_n:A_n$ and $\Gamma'=x_1:A_1'\ldots x_n:A_n'$ where $A_i\rightarrow A_i'$ for exactly one $i$ and for all $j\ne i$ $A_j=_\beta A_j'$. In other words, the type of exactly one variable in the context takes a step.

  Case STAR: This case cannot occur because neither the context nor the term may take a step.

  Case VAR:

  \[\infer{\Gamma,x:A\vdash x:A}{\Gamma\vdash A:s}\]

  Since $x$ cannot take a step, we only need to handle $(\Gamma,x:A)\rightarrow(\Gamma,x:A)'$
  This can happen in two ways:

  If $\Gamma\rightarrow\Gamma'$: By the inductive hypothesis, $\Gamma'\vdash A:s$. Thus,
  \[\infer{\Gamma',x:A\vdash x:A}{\Gamma'\vdash A:s}\]
  as desired.

  If $A \rightarrow A'$: By the inductive hypothesis, $\Gamma\vdash A':s$. Thus,
  \[\infer{\Gamma,x:A'\vdash x:A'}{\Gamma\vdash A':s}\]
  as desired.

  Case WEAK:

  \[\infer{\Gamma,x:A\vdash B:C}{\Gamma\vdash B:C & \Gamma\vdash A:s}\]

  First, consider the case where the term takes a step: $B\rightarrow B'$.
  By the inductive hypothesis, $\Gamma\vdash B':C$. By WEAK, $\Gamma,x:A\vdash B':C$, as desired.
  Now consider if the context takes a step. There are two cases. If $\Gamma\rightarrow\Gamma'$, then by the inductive hypothesis, $\Gamma'\vdash B:C$ and $\Gamma'\vdash A:s$. By WEAK, $\Gamma'\vdash B:C$.
  If $A\rightarrow A'$, then by the inductive hypothesis, $\Gamma\vdash A':s$. By WEAK, $\Gamma,x:A'\vdash B:C$, as desired.

  Case APP:
  \[\infer{\Gamma\vdash f a:B[x\mapsto a]}{\Gamma\vdash f:(\Pi:A.B) & \Gamma\vdash a:A}\]

  If $\Gamma\rightarrow \Gamma'$, by the inductive hypothesis, $\Gamma'\vdash f:(\Pi:A.B)$ and $\Gamma'\vdash a:A$. Thus,
  \[\Gamma\vdash f a:B[x\mapsto a]\]
  as desired.

  There are three ways for the term to take a step.

  If $e'=f' a$ where $f\rightarrow f'$ (APP1), then by the inductive hypothesis, $\Gamma\vdash f':(\Pi:A.B)$, so by APP, $\Gamma\vdash f' a:B[x\mapsto a]$, as desired.
  If $e'=f a'$ where $a\rightarrow a'$ (APP2), then by the inductive hypothesis, $\Gamma\vdash a':A$, so by APP, $\Gamma\vdash f a':B[x\mapsto a']$, as desired.

  If $e'=b[x\mapsto a]$ where $f=(\lambda x:A.b)$ (APP3), then by rules APP and LAM (and possible WEAK and CONV), we have

  \[\infer{\Gamma\vdash(\lambda x:A.b) a:B[x\mapsto a]}{\infer{\Gamma\vdash (\lambda x:A.b):(\Pi x:A.B)}{\infer{\cdots}{\infer{\Delta\vdash (\lambda x:A.b):(\Pi x:A.B)}{\Delta,x:A\vdash b:B & \Delta\vdash (\Pi x:A.B):t}}} & \Gamma\vdash x:A}\]

  We can add back variables to $\Delta$ to get $\Gamma, x:A\vdash b:B$. Also, $\Gamma\vdash a:A$. By the Substitution Lemma. $\Gamma\vdash b[x\mapsto a]:B[x\mapsto a]$, as desired.

  Case LAM: By rules LAM and PI (and possible WEAK and CONV).
  \[\infer{\Gamma\vdash(\lambda x:A.b):(\Pi x:A.B)}{\Gamma,x:A\vdash b:B & \infer{\Gamma\vdash (\Pi x:A.B):t}{\infer{\cdots}{\infer{\Delta\vdash (\Pi x:A.B):u}{\Delta\vdash A:s & \Delta,x:A\vdash B:u}}}}\]

  There are two ways for the term to take a step. If $e'=(\lambda x:A'.b)$ where $A\rightarrow A'$ (LAM1), then by the inductive hypothesis, $\Gamma,x:A'\vdash b:B$, $\Delta\vdash A':s$ and $\Delta,x:A'\vdash B:t$. Thus,

  \[\infer{\Gamma\vdash(\lambda x:A'.b):(\Pi x:A'.B)}{\Gamma,x:A'\vdash b:B & \infer{\Gamma\vdash (\Pi x:A'.B):t}{\infer{\cdots}{\infer{\Delta\vdash (\Pi x:A'.B):u}{\Delta\vdash A':s & \Delta,x:A'\vdash B:u}}}}\]
  as desired.

  If $e'=(\lambda x:A.b')$ where $b\rightarrow b'$ (LAM2), then by the inductive hypothesis, $\Gamma,x:A\vdash b':B$, Thus,

  \[\infer{\Gamma\vdash(\lambda x:A.b'):(\Pi x:A.B)}{\Gamma,x:A\vdash b':B & \Gamma\vdash (\Pi x:A.B):t}\]
    as desired.

    If $\Gamma\rightarrow\Gamma'$, by the inductive hypothesis, $\Gamma',x:A\vdash b:B$ and $\Gamma'\vdash (\Pi x:A.B):t$.

    Thus,
    \[\infer{\Gamma'\vdash(\lambda x:A.b):(\Pi x:A.B)}{\Gamma',x:A\vdash b:B & \Gamma'\vdash (\Pi x:A.B):t}\]
    as desired.

    Case PI: $e=(\Pi x:A_1.A_2)$.

  \[\infer{\Gamma\vdash(\Pi x:A_1.A_2):t}{\Gamma\vdash A_1:s & \Gamma,x:A_1\vdash A_2:t}\]

  Suppose $e\rightarrow e'$. This can happen in two ways.

  If $e'=A_1' A_2$ where $A_1\rightarrow A_1'$ (rule PI1): By the inductive hypothesis, $\Gamma\vdash A_1':s$, $\Gamma,x:A_1'\vdash A_2:t$. Thus, by rule PI,

  \[\infer{\Gamma\vdash(\Pi x:A_1'.A_2):t}{\Gamma\vdash A_1':s & \Gamma,x:A_1'\vdash A_2:t}\]
  as desired.

  If $e'=A_1 A_2'$ where $A_2\rightarrow A_2'$ (rule PI2): By the inductive hypothesis,  $\Gamma,x:A_1\vdash A_2':t$. Thus, by rule PI,
  \[\infer{\Gamma\vdash(\Pi x:A_1.A_2'):t}{\Gamma\vdash A_1:s & \Gamma,x:A_1\vdash A_2':t}\]
  as desired.

  Suppose $\Gamma\rightarrow\Gamma'$. By the inductive hypothesis, $\Gamma'\vdash A_1:s$, $\Gamma',x:A_1\vdash A_2:t$. Thus, by rule PI,

  \[\infer{\Gamma'\vdash(\Pi x:A_1.A_2):t}{\Gamma'\vdash A_1:s & \Gamma',x:A_1\vdash A_2:t}\]
  as desired.

  Case CONV:
  \[\infer{\Gamma\vdash a:B}{\Gamma\vdash a:A & \Gamma\vdash B:s & A=_\beta B}\]

  Suppose $a\rightarrow a'$.

  By the inductive hypothesis, $\Gamma\vdash a':A$, so by rule CONV,
  \[\infer{\Gamma\vdash a':B}{\Gamma\vdash a':A & \Gamma\vdash B:s & A=_\beta B}\]
  as desired.

  Suppose $\Gamma\rightarrow\Gamma'$.
  By the inductive hypothesis, $\Gamma'\vdash a:A$ and $\Gamma'\vdash B:s$, so by rule CONV,
  \[\infer{\Gamma\vdash a':B}{\Gamma\vdash a':A & \Gamma\vdash B:s & A=_\beta B}\]
  as desired.
\end{proof}

\bibliography{citations}{}
\bibliographystyle{plain}
\end{document}


\subsection{Evaluation Metrics}

\subsubsection{Multiclass classification}
For the multiclass dataset we use accuracy and macro-averaged F1 score.

\subsubsection{Accuracy} Accuracy is the percentage of labels for which the class is correctly predicted. Note that for multiclass classification the micro-averaged F1 score is equal to the accuracy.

\subsubsection{Macro-averaged F1 score}
Macro-averaged F1 score is computed by first computing the F1 score for each class independently and then take an averaging all the F1 scores. This metric treats all the classes as equal, independent of their frequency in the test set.

\subsubsection{Multilabel classification}
For multilabel classification we use three metrics - hamming loss, micro-averaged F1 score and macro-averaged F1 score.

\subsubsection{Hamming loss} Hamming loss is the proportion of mis-classified examples in the dataset.
\subsubsection{Micro-averaged F1 score} F-measure averaging on the prediction matrix.

\subsubsection{Macro-averaged F1 score} Macro-averaged F1 score is calculated by computing the F1 score for each of the labels, then averaging the label wise F1 scores.