\documentclass[11pt,letterpaper]{article}
\usepackage[top=1in,bottom=1in,left=1in,right=1in]{geometry}
\usepackage{natbib}      % http://merkel.zoneo.net/Latex/natbib.php
\usepackage{palatino}
\bibpunct{(}{)}{;}{a}{,}{,}
\usepackage{chngpage}
\usepackage{stmaryrd}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{graphicx}
\usepackage{lscape}
\usepackage{subfigure}
\usepackage[usenames,dvipsnames]{color}
\definecolor{myblue}{rgb}{0,0.1,0.6}
\definecolor{mygreen}{rgb}{0,0.3,0.1}
\usepackage[colorlinks=true,linkcolor=black,citecolor=mygreen,urlcolor=myblue]{hyperref}
\newcommand{\bocomment}[1]{\textcolor{Bittersweet}{[#1 -BTO]}}
\newenvironment{itemizesquish}{\begin{list}{\labelitemi}{\setlength{\itemsep}{0em}\setlength{\labelwidth}{2em}\setlength{\leftmargin}{\labelwidth}\addtolength{\leftmargin}{\labelsep}}}{\end{list}}
\newcommand{\ignore}[1]{}
\newcommand{\transpose}{^\mathsf{T}}
\newcommand{\inner}[1]{\langle #1 \rangle}
\newcommand{\smallsec}[1]{\noindent \textbf{#1\ }}

\newcommand{\til}[1]{\widetilde{#1}}

\newcommand{\solution}[1]{{\color{Blue}[\textbf{Solution:} #1]}}
% \newcommand{\solution}[1]{}
\theoremstyle{definition}
\newtheorem{question}{Question}[section]
% \newtheorem{question}{Question}

\title{agglomerative topic model
}

\author{
Brendan O'Connor
}

%\date{December 9, 2009}

\begin{document}
\maketitle

Goal: Find a model for which agglomerative word-level merging is doing inference for that model.

Rationale: agglomerative word-level merging is great!  Finding a model to justify it would be nice.

\section{One topic per word topic model}

Here's a topic model, in the sense it's a low-dimensional mixed-membership model for multiple multinomial groups of discrete data.  Unlike LDA it has no dirichlet priors or any priors.  For documents $d$ and token indexes $t$ and words $w \in V$ for vocabulary $V$ and topics $k \in K$,

\[ z_t \sim Disc(\theta_{d(t)}), w_t \sim \phi_{z_t} \]

using indexing notation $d(t)$ being the document at token $t$ (where token positions are unique globally in the corpus).  $\theta_d$ is one doc-topic distribution and $\phi_k$ is one topic-word distribution.  Note we use $V$ to mean either a set or that set's cardinality; same notation abuse for $K$.

The model has a one topic per word constraint: every word belongs to one topic, and it cannot appear in any other.  There exists a many-to-one map for all words, $k(w)$, representing the topic that a wordtype belongs to; this means that $P(w|z=k(w))>0$ and for all other topics $j \neq k(w)$, $P(w|z=j)=0$.
This mapping can also be interpreted as a hard clustering (a partition) over the wordtypes, into $K$ clusters.

Given a mapping, inference is trivial.  $P(z_t|w_t)$ is nonzero for only when $z_t=k(w_t)$.  There are no priors on either $\theta$ or $\phi$, so their estimates are simple maximum-likelihood relative frequency calculations.  For example, $\theta_{dk}$ is the number of $z_t=k$ within $t \in d(t)$, divided by the number of tokens in $d$.

We'll use some shortcut notation.  $w_t$ is one token at position $t$.  $w_d$ are the tokens in document $d$.  $w_k$ is the ``token text'',\footnote{that Heinrich ?2007 calls it} meaning all tokens in the corpus that have topic $k$.  Let $n$ denote numbers of tokens, like $n_{kv}$ the number of tokens with topic $k$ and wordtype $v$.

Total likelihood of the corpus, given a mapping, can be expressed in terms of documents

\[ \log P(w) = \sum_d \log P(w_d) = \sum_d \sum_v n_{dv} \log \frac{n_{kv}}{n_k} \]

You could also write it in terms of topic-texts

\[ \log P(w) = \sum_k \log P(w_k) = \sum_k \sum_v n_{kv} \log \frac{n_{kv}}{n_k} \]

As it stands, you can't actually learn anything interesting with this model.  There's no reason for co-occurring words to want to belong to the same topic.
In LDA, the Dirichlet prior on $\theta$ encourages this: it causes documents to tend to include only a subset of topics, and thus topics need to include co-occurring words in order to explain the data.

Let's abstract this into a document sparsity function, $S(\theta_d)$, which is higher if more of the probability mass of the topics in a document tend to be in a subset of topics, or tend to have a peaked distribution.  One way to define this is a Dirichlet unnorm-log-density function, as LDA uses, where you can set all $\alpha_k<1$ to prefer sparsity,
\[ S(\theta_d; \alpha) = \sum_k \left(\alpha_k - 1\right) \log \theta_{dk} \]

Of course, if we use this, $\theta_d$ is not allowed to have zeros.  We'd like to analyze this case, and we could do it by saying instead that $\theta_d$ is a latent variable that gets a posterior distribution compromising between the uniform prior $\alpha/(\sum\alpha_k)$ versus the empirical doc-topic distribution $[n_{dk}/n_d]_k$, and define the sparsity measurement function instead as $S(z_d;\alpha)$; this can totally be done with a collapsed dirichlet-multinomial, but tbd how to work it out in terms of agglomerative merging.

Here's another approach.  Let's keep with the trivial MLE estimators for $\theta$, namely that $\theta_d=n_{dk}/n_d$.  To promote doc-topic sparsity, impose some sort of made-up function $S$ on it, where higher $S$ indicates more sparsity.  For example, the Shannon negentropy,
\[ S_1(\theta_d) = \sum_k \theta_k \log \theta_{dk} \]
or the Renyi-0 negentropy, which just cares about the number of nonzero components,
\[ S_0(\theta_d) = -\log \sum_k 1\{\theta_{dk}>0\} = -\log ||\theta_d||_0\]
or the Renyi-2 negentropy (``collision''/``gini''), which is the chance a pair of tokens have the same topic,
\[ S_2(\theta_d) = \log \sum_k \theta_{dk}^2 = \log ||\theta_d||^2_2 \]
etc.  These all handle zeros just fine (shannon requires $0\log 0\equiv 0$).

Then the overall optimization objective for learning a mapping $k(.)$ is
\[ F = \lambda S(\theta) + \log P(w) \]
where $S(\theta) = \sum_d S(\theta_d)$, and $\lambda$ is a hyperparameter to control the amount of doc-topic sparsity.

\section{Merging algorithm for optimization}

Initialize $K=V$ topics, with each word as its own topic.  Greedily merge pairs of topics to improve the objective.  Can the objective improvement be calculated rapidly?

Say the candidate merge pair is topics $k$ and $j$, and we're considering the improvement
\[F_{new}-F_{old} = S_{new}-S_{old} + \log P_{new}-\log P_{old}\]

\subsection{Change to topic-text likelihood}
First analyze $\log P_{new}-\log P_{old}$ in terms of topic texts. The only parts that don't cancel are topic texts for the merge: $w_j$ and $w_k$.

Let $w(k)$ be the reverse map of $k(w)$, where $w(k)$ is the set of wordtypes with nonzero probability under topic $k$.
The old topic-text logprob for $k$ is
 \[ \ell_k = \sum_{v \in w(k)} n_v \log \frac{n_v}{n_k}
  = \sum_{v\in w(k)} n_v \log n_v - n_v \log n_k
  = \left(\sum_{v\in w(k)} n_v \log n_v\right) - n_k \log n_k
  \]
and similarly for $j$.  The new topic-text logprob is
  \[\ell_{j \cup k} = \sum_{v \in w(k) \cup w(j)} n_v \log \frac{n_v}{n_j+n_k} =
    \left(\sum_{v \in w(k) \cup w(j)} n_v \log  n_v\right) - (n_j+n_k) \log (n_j+n_k)\]
Then
\begin{align}
  \log P_{new}-\log P_{old}
  &= \ell_{j \cup k} - \ell_k - \ell_j
=
  n_k \log \frac{n_k}{n_j+n_k} + n_j \log \frac{n_j}{n_j+n_k} \label{e:pmerge}
\end{align}
The lexical terms $n_v \log n_v$ all cancel.  It doesn't matter what the words are, or their distributions within the topics.  All that matters is the relative sizes of the two topics.  One way to think of this equation is as the size of the new topic, times the negative entropy of the binary variable of whether $k$ or $j$ was the original topic text.  Thus it prefers the merging topics with very different sizes, or topics where both are small.  Here's a plot of equation \ref{e:pmerge}:
\begin{center}
\includegraphics[width=3in]{contours.pdf}
\end{center}
This value is always negative---you're asking to blunt the two distributions into a worse, coarser one, so it's a question of picking a not-too-bad merge.

\subsection{Change to sparsity}

By contrast, co-occurrence information is relevant to the sparsity term $S(\theta)=\sum_d S(\theta_d)$.  The merge can be thought of as moving from a model with $K$ topics to one with $K-1$ topics, so every vector $\theta_d$ shrinks.  For every document, if neither topic $j$ nor $k$ occurs in the document, $\theta_d$ doesn't change (beyond losing a zero entry).  If one occurs but not the other, again, $\theta_d$ doesn't change (beyond losing a zero entry and shuffling some indexes).  Only for documents having both, does it change: $\theta_d$ loses two entries, which are replaced by one entry that's the sum of them from before.

In fact, in the axiomatic definition of entropy,\footnote{\url{http://www.stat.cmu.edu/~cshalizi/350/2008/lectures/06a/lecture-06a.pdf}} the first two cases are defined to not change entropy; the third decreases entropy, i.e., increases our sparsity function $S$.

Let $D$ be the set of documents where both $j$ and $k$ occur.  Then the improvement for Renyi-0 negentropy is
  \begin{align}
  S^{new}_0-S^{old}_0
  &= \sum_{d\in D} -\log ||\theta^{new}_d||_0 + \log ||\theta^{old}_d||_0
  \\
  &= \sum_{d\in D} \log \frac{x_d}{x_d-1}
  \end{align}
where $x_d$ is the number of non-zero topics in the old model.  The more documents the topics co-occur in, the more they want to merge; and documents with already a low number of topics give a greater gain.

Or we could just use the 0-norms directly (exponentiated renyi-0)
  \begin{align}
  S^{new}_{e0}-S^{old}_{e0}
  &= \sum_{d\in D} -||\theta^{new}_d||_0 + ||\theta^{old}_d||_0
  = |D|
  \end{align}

For Shannon entropy it is
  \begin{align}
  S^{new}_1-S^{old}_1
  &= \sum_{d \in D} (\theta_j+\theta_k) \log(\theta_j+\theta_k)
      -\theta_j\log\theta_j-\theta_k\log\theta_k
      \\
  &= -\sum_{d \in D}
        \theta_j \log \frac{\theta_j}{\theta_j+\theta_k}
        +
        \theta_k \log \frac{\theta_k}{\theta_j+\theta_k}
      \\
  &= -\sum_{d \in D} (\theta_j+\theta_k)
      \left[
      \frac{\theta_j}{\theta_j+\theta_k}\log\frac{\theta_j}{\theta_j+\theta_k}
      +
      \frac{\theta_k}{\theta_j+\theta_k}\log\frac{\theta_k}{\theta_j+\theta_k}
      \right]
  \end{align}
For each document, this is the entropy for the choice between the two merged topics, weighted by their combined mass in the document; so this favors topics that have large and similar prevalences, when they co-occur.

\subsection{Computation}
The topic-text log-lik change is pretty simple to compute.

The sparsity change is the harder one.  There are probably several ways of doing it.  Maybe track topic-pair statistics, each of which is the sum of doc-level topic-pair stats.  To initialize, iterate through all documents and increment all topic-pair stats for pairs of topics in the same document (at this point, all topics are single words, so this is like computing all word co-occurrences and doing a little more with them).  When merging, update the stats in the documents in the intersection of the topic supports; all other ones don't have to change, at least not on a doc level?

\subsection{Relationship to pairwise associations}
Is this related to topic merging with PMI or other associations?
It has to come through the sparsity-promoting term.

The ``joint-occur'' ePMI is
\[ \frac{N^2 \sum_d n_{dj} n_{dk}}{n_j n_k \sum_d n_d^2} \]

the thing this looks closest to is the Renyi-2 forms, especially exponentiated renyi-2, just the difference in the squared L2 norms.

\begin{align} S^{new}-S^{old} &=
\sum_d ||\theta^{new}_d||^2 - ||\theta^{old}_d||^2
\\
&= \sum_d (\theta_j+\theta_k)^2 - \theta_j^2 - \theta_k^2
= \sum_d 2\theta_j\theta_k = 2 \sum_d \frac{n_{dj} n_{dk}}{n_d^2}
\end{align}

Here's the closest I can get their forms.  Ignore all $n_d$ terms---this is legitimate if all documents are the same size, then these terms can be collected and moved out of sums as constants.  Then the PMI looks like
\[ PMI = \log \sum_d n_{dj} n_{dk}
     - \underbrace{(\log n_j + \log n_k)}_{\text{normalization factor}} \]
and the $F$ delta takes the form, assuming we're using the $\log(S^{new}-S^{old})$ from the above,
\begin{align} F_{change} &=
        \log\sum_d n_{dj} n_{dk}  + \left[
        n_j \log \frac{n_j}{n_j+n_k} +
        n_k \log \frac{n_k}{n_j+n_k}
         \right]
\\
&=      \log\sum_d n_{dj} n_{dk}  -
            \underbrace{(n_j+n_k) H(\text{$J$ vs $K$})}_{\text{normalization factor}}
\end{align}

Roughly speaking, both equations reward high co-occurrence counts, while penalizing large topics, but the normalization happens in rather different ways.  PMI penalizes the log topic sizes.  F penalizes the topic size, but weighted by their relative sizes.   First off, note that entropy is always non-negative and the right term is a negative entropy, for the random variable ``is this topic $j$ or $k$?'', conditional on being either.  So it's negative, multiplied by the combined topic size $(n_j+n_k)$; thus it has some normalization for the topic sizes.  The preference against large topics is stronger if the entropy is high, i.e., if the topics are similar sizes.  But if the topics are very skewed, the entropy is low so the penalization is weaker.

\subsection{todo: dirichlet derivation}

chong wang's split-merge for hdp -- might be relevant?  jain and neal have lots of factorial stuff for the DPMM case.  i think this is for all the log-gamma functions which manifest your sparsity control under collapsed dir-mult inference.

\section{Justification for one topic per word}

The idea is: mixed membership models are good, but single membership models are faster.

We want to compromise LDA to make it faster.
Where do you want single membership to happen: in the doc-topic associations, or in the topic-word associations?

topic-word associations, when viewed from the word-to-topic angle, are just naturally the sparsest of those combinations.  therefore do that.

% more justification: for LDA, you can fit it to text and infer the Dirichlet concentration parameters for both $\theta$ and $\phi$.  What happens is most documents use only a few of the topics, and most topics use only a small proportion of words; and you can see this no matter how you define ``use only a few'', such as, say $x_d =$ how many topics make up the top-99\% of prob mass of a $\theta_d$; typically you get like median $x_d=4$ or something for say $K=20$.  Call this the sparsity of the topic model.

% There are two causes of this sparsity.  One is that the empirical doc-word distributions are just sparse.  So if you want a lower dimensional simplex to encompass them, you naturally get sparsity again.  Another is that the Dirichlet priors encourage it, or somehow allow it.  If you fixed your $\alpha$ this would definitely be what's happening; but it still happens when $\alpha$ is learned.  I'm still puzzled why this is so.  But it's neat.

% LDA is called a mixed-membership model because each document can belong to multiple latent components.  It can't belong to all of them --- you couldn't really learn anything interesting if that was the case --- but it belongs to a smaller subset of them.  The most extreme case is where each document only belongs to a single topic.  You can fit this model just fine---it's called a single-membership model, and sometimes called unsupervised multinomial Naive Bayes.  But it tends to work poorly; the topic-word distributions don't seem to make as much sense.

\bibliographystyle{plainnat}
\bibliography{brenocon}
\end{document}


\section{Future Work}
\label{section:futurework}
As with all machine learning techniques, many hours or even days could be spent fine-tuning the data normalization,
tokenizing, and parameterization of the naive bayes classifier implemented here.

Some key things that should be investigated to improve this classifier include generalizing the tokenizer concept
to support regression feature (such as ``avg word length = 4.5"), normalize the token counts across classes (this is
what likely had the largest impact on the success of this report's results), and investigate new ways to tokenize.


\subsection{Lotteries}

A choice may not have a certain outcome.

For example an action could have have 50% of one of two outcomes.


\section{Static Semantics for the Core}
\label{statcor.sec}
Our\index{20.1} first task in presenting the semantics -- whether for Core or Modules,
static or dynamic -- is to define the objects concerned. In addition
to the class of {\em syntactic} objects, which we have already defined,
there are classes of so-called {\em semantic} objects used to describe
the meaning of the syntactic objects. Some classes contain {\em simple}
semantic objects; such objects are usually identifiers or names of some
kind. Other classes contain {\em compound} semantic objects, such as
types or environments, which are constructed from component objects.

\subsection{Simple Objects}
%\ replacement{\thenostrsharing}{All semantic objects in the static semantics of the entire
%language are built from identifiers and two further kinds of simple objects:
%type constructor names and structure names.}{All semantic objects in
%the static semantics of the entire
%language are built from identifiers and one further kind of simple objects:
%type constructor names.}
\replacement{\theidstatus}{All semantic objects in the static semantics of the entire
language are built from identifiers and two further kinds of simple objects:
type constructor names and structure names.}{All semantic objects in
the static semantics of the entire
language are built from identifiers and two further kinds of simple objects:
type constructor names and identifier status descriptors.}
Type constructor names are the values taken by type constructors; we shall
usually refer to them briefly as type names, but they are to be clearly
distinguished from type variables and type constructors.
\deletion{\thenostrsharing}{Structure names play an active role only in
the Modules semantics; they enter the Core semantics only because
they appear in structure environments, which (in turn) are needed in the Core
semantics only to determine the values of long identifiers.} The simple object
classes, and the variables ranging over them, are shown in
Figure~\ref{simple-objects}. We have included $\TyVar$ in the table to
make visible the use of $\alpha$ in the semantics to range over $\TyVar$.\index{20.2}

%\vspace{-7mm}
%\vspace{-8mm}
\begin{figure}[h]
\vspace{2pt}
% \ adhocreplacementl{\thenostrsharing}{1cm}{
% \begin{displaymath}
% \begin{array}{rclr}
% \alpha\ {\rm or}\ \tyvar & \in   & \TyVar       & \mbox{type variables}\\
% \t               & \in   & \TyNames     & \mbox{type names}\\
% \m              & \in   & \StrNames     & \mbox{structure names}
% \end{array}
% \end{displaymath}}{\begin{displaymath}
% \begin{array}{rclr}
% \alpha\ {\rm or}\ \tyvar & \in   & \TyVar       & \mbox{type variables}\\
% \t               & \in   & \TyNames     & \mbox{type names}
% \end{array}
% \end{displaymath}}
\adhocreplacementl{\theidstatus}{1cm}{
\begin{displaymath}
\begin{array}{rclr}
\alpha\ {\rm or}\ \tyvar & \in   & \TyVar       & \mbox{type variables}\\
\t               & \in   & \TyNames     & \mbox{type names}\\
\m              & \in   & \StrNames     & \mbox{structure names}
\end{array}
\end{displaymath}}{\begin{displaymath}
\begin{array}{rcll}
\alpha\ {\rm or}\ \tyvar & \in   & \TyVar       & \mbox{type variables}\\
\t               & \in   & \TyNames     & \mbox{type names}\\
\is              & \in   & \IdStatus = \{\isc,\ise,\isv\}    & \mbox{identifier status descriptors}
\end{array}
\end{displaymath}}
\caption{Simple Semantic Objects}
\label{simple-objects}
%\vspace{3pt}
\end{figure}

Each\index{20.3} $\alpha \in\TyVar$ possesses a boolean {\sl equality} attribute,
which determines whether or not it {\sl admits equality}, i.e. whether
it is a member of EtyVar (defined on page~\pageref{etyvar-lab}).
%-- in which case we
%also say that it is an {\sl equality} type variable.
%poly
\deletion{\thenoimptypes}{Independently hereof, each $\alpha$ possesses a boolean attribute,
the {\sl imperative} attribute, which determines whether it is imperative,
i.e. whether it is a member of $\ImpTyVar$ (defined on page~\pageref{etyvar-lab})
or not.}

Each $\t\in\TyNames$ has
an arity $k\geq 0$, and also possesses an equality attribute.
We denote the class of type names with arity $k$ by $\TyNamesk$.

With\index{20.35} each special constant {\scon} we associate a type
name $\scontype(\scon)$ which is either \replacement{\thescon}{{\INT}, {\REAL}
 or {\STRING}}{{\INT}, {\REAL}, {\WORD}, {\CHAR}
 or {\STRING}}
as indicated by Section~\ref{cr:speccon}.
\insertion{\thelibrary}{(However, see Appendix~\ref{overload.sec}
concerning types of overloaded special constants.)}

\subsection{Compound Objects}
When\index{20.4} $A$ and $B$ are sets $\Fin A$ denotes the set of finite subsets of $A$,
and $\finfun{A}{B}$ denotes the set of {\sl finite maps} (partial functions
with finite domain) from $A$ to $B$.
The domain\index{21.1} and range of a finite map, $f$, are denoted $\Dom f$ and
$\Ran f$.
A finite map will often be written explicitly in the form $\kmap{a}{b},
\ k\geq 0$;
in particular the empty map is $\emptymap$.
We shall use the form $\{x\mapsto e\  ;\  \phi\}$ -- a form of set
comprehension -- to stand for the finite map $f$ whose domain
is the set of values $x$ which satisfy the condition $\phi$, and
whose value on this domain is given by $f(x)=e$.

When $f$ and $g$ are finite maps the map $\plusmap{f}{g}$, called
$f$ {\sl modified} by $g$, is the finite map with domain
$\Dom f \cup \Dom g$ and values
\[(\plusmap{f}{g})(a) = \mbox{if $a\in\Dom g$ then $g(a)$ else $f(a)$.}
\]

The compound objects for the static semantics of the Core Language are
shown in Figure~\ref{compound-objects}.
We take $\cup$ to mean disjoint union over
semantic object classes. We also understand all the defined object
classes to be disjoint.

\begin{figure}[h]
%\vspace{2pt}
\begin{displaymath}
\begin{array}{rcl}
        \tau    &\in    &\Type = \TyVar\cup\adhocreplacementl{\theidstatus}{6cm}{\RecType}{\RowType}\cup\FunType
                                 \cup\ConsType\\
 \longtauk\ {\rm or}\ \tauk
                & \in   & \Type^k\\
 \longalphak\ {\rm or}\ \alphak
                & \in   & \TyVar^k\\
 \varrho        & \in   & \adhocreplacementl{\theidstatus}{4cm}{\RecType}{\RowType} = \finfun{\Lab}{\Type} \\
 \tau\rightarrow\tau'
                & \in   & \FunType = \Type\times\Type \\
                &       & \ConsType = \cup_{k\geq 0}\ConsType^{(k)}\\
        \tauk\t & \in   & \ConsType^{(k)} = \Type^k\times\TyNamesk  \\
\typefcn\ {\rm or}\ \typefcnk
                & \in   & \TypeFcn = \cup_{k\geq 0}\TyVar^k\times\Type\\
\tych\ {\rm or}\ \longtych
                & \in   & \TypeScheme = \cup_{k\geq 0}\TyVar^k\times\Type\\
\adhocdeletion{\thenostrsharing}{4cm}{\S\ {\rm or}\ (\m,\E)
                & \in   & \Str = \StrNames\times\Env  \\ }
(\theta,\adhocreplacementl{\thece}{2cm}{\CE}{\VE})    & \in   & \adhocreplacementl{\thece}{-8cm}{\TyStr = \TypeFcn\times\ConEnv}{\TyStr = \TypeFcn\times\ValEnv}\\
\SE             & \in   & \adhocreplacementl{\thenostrsharing}{3cm}{\StrEnv = \finfun{\StrId}{\Str}}{\StrEnv = \finfun{\StrId}{\Env}}\\
\TE             & \in   & \TyEnv = \finfun{\TyCon}{\TyStr}\\
\adhocdeletion{\thece}{4cm}{\CE             & \in   & \ConEnv = \finfun{\Con}{\TypeScheme}\\ }\VE             & \in   & \adhocreplacementl{\theidstatus}{3cm}{\VarEnv = \finfun{(\Var\cup\Con\cup\Exn)}{\TypeScheme}}{\ValEnv = \finfun{\VId}{\TypeScheme\times\IdStatus}}\\
\adhocdeletion{\theidstatus}{5mm}{\EE             & \in   & \ExnEnv = \finfun{\Exn}{\Type}\\ }\E\ {\rm or}\ \adhocreplacementl{\theidstatus}{3cm}{\longE{}}{\newlongE{}}
                & \in   & \adhocreplacementl{\theidstatus}{-9cm}{\Env = \StrEnv\times\TyEnv\times\VarEnv\times\ExnEnv}{\Env = \StrEnv\times\TyEnv\times\ValEnv}\\
\T              & \in   & \TyNameSets = \Fin(\TyNames)\\
\U              & \in   & \TyVarSet = \Fin(\TyVar)\\
\C\ {\rm or}\ \T,\U,\E   & \in   & \Context = \TyNameSets\times\TyVarSet\times\Env
\end{array}
\end{displaymath}
\caption{Compound Semantic Objects\index{21.2}}
\label{compound-objects}
%\vspace{3pt}
\end{figure}

Note that $\Lambda$\index{21.3} and $\forall$ bind type variables.  For any semantic object
$A$, $\TyNamesFcn A$ and $\TyVarsFcn A$ denote respectively the set of
type names and the set of type variables occurring free in $A$.
\deletion{\thenoimptypes}{Moreover, $\imptyvars A$ and $\apptyvars A$ denote respectively the set
of imperative type variables and the set of applicative
type variables occurring free in $A$.}\index{21.4}
\insertion{\theidstatus}{\par Also note that a value environment maps
value identifiers to a pair of a type scheme and an identifier status.
If $\VE(\vid) = (\sigma,\is)$, we say that $\vid$ {\sl has status $\is$
in $\VE$}. An occurrence of a value identifier which is elaborated
in $\VE$ is referred to as a {\sl value variable}, a {\sl value constructor}
or an {\sl exception constructor}, depending on whether its status in $\VE$
is $\isv$, $\isc$ or $\ise$, respectively. }

\subsection{Projection, Injection and Modification}
\label{stat-proj}\index{22.1}
{\bf Projection}: We often need to select components of tuples -- for example,
the \replacement{\theidstatus}{variable-environment}{value-environment} component of a context. In such cases we
rely on \replacement{\theidstatus}{variable}{metavariable} names to indicate which component
is selected. For instance ``$\of{\VE}{\E}$'' means ``the \replacement{\theidstatus}{variable-environment}{value-environment}
component
of $\E$''\deletion{\thenostrsharing}{ and ``$\of{\m}{\S}$'' means ``the structure name of $\S$''}.

Moreover, when a tuple contains a finite map we shall ``apply'' the
tuple to an argument, relying on the syntactic class of the argument to
determine the relevant function. \replacement{\theidstatus}{For instance $\C(\tycon)$ means
$(\of{\TE}{\C})\tycon$.

A particular case needs mention:  $\C(\con)$ is taken to stand for
$(\of{\VE}{\C})\con$; similarly, $\C(\exn)$ is taken to stand for
$(\of{\VE}{\C})\exn$.
  The type scheme of a value constructor is
held in $\VE$ as well as in $\TE$ (where it will be recorded within
a $\CE$); similarly, the type of an exception constructor is held in
$\VE$ as well as in $\EE$.
Thus the re-binding of a constructor of either kind is given proper
effect by accessing it in $\VE$, rather than in $\TE$ or in $\EE$.}{For
instance $\C(\tycon)$ means
$(\of{\TE}{\C})\tycon$ and $\C(\vid)$ means $(\of{\VE}{(\of{E}{\C})})(\vid)$.}

Finally, environments may be applied to long identifiers.
\replacement{\theidstatus}{For instance if $\longcon = \strid_1.\cdots.\strid_k.\con$ then
$\E(\longcon)$ means
\[ (\of{\VE}
       {(\of{\SE}
            {\cdots(\of{\SE}
                       {(\of{\SE}{\E})\strid_1}
                   )\strid_2\cdots}
        )\strid_k}
    )\con.
\]
}{For instance if $\longvid = \strid_1.\cdots.\strid_k.\vid$ then
$\E(\longvid)$ means
\[ (\of{\VE}
       {(\of{\SE}
            {\cdots(\of{\SE}
                       {(\of{\SE}{\E})\strid_1}
                   )\strid_2\cdots}
        )\strid_k}
    )\vid.
\]
}

{\bf Injection}: Components may be injected into tuple classes; for example,\linebreak
``$\VE\ \In\ \Env$'' means the environment
\replacement{\theidstatus}{$(\emptymap,\emptymap,\VE,\emptymap)$.}{$(\emptymap,\emptymap,\VE)$.}

{\bf Modification}: The modification of one map $f$ by another map $g$,
written $f+g$, has already been mentioned.  It is commonly used for
environment modification, for example $\E+\E'$.  Often, empty components
will be left implicit in a modification; for example $\E+\VE$ means
\replacement{\theidstatus}{$\E+(\emptymap,\emptymap,\VE,\emptymap)$.}{$\E+(\emptymap,\emptymap,\VE)$.}  For set components, modification
means union, so that $\C+(\T,\VE)$ means
\[ (\ (\of{\T}{\C})\cup\T,\ \of{\U}{\C},\ (\of{\E}{\C})+\VE\ ) \]
Finally, we frequently need to modify a context $\C$ by an environment $\E$
(or a type environment $\TE$ say),
at the same time extending $\of{\T}{\C}$ to include the type names of $\E$
(or of $\TE$ say).
We therefore define $\C\oplus\TE$,\index{22.2} for example, to mean
$\C+(\TyNamesFcn\TE,\TE)$.
%\vspace*{12pt}

\subsection{Types and Type functions}
\label{tyfun-sec}
A type $\tau$ is an {\sl equality type},\index{22.3} or {\sl admits equality}, if it is
of one of the forms
\begin{itemize}
\item $\alpha$, where $\alpha$ admits equality;
\item $\{\lab_1\mapsto\tau_1,\ \cdots,\ \lab_n\mapsto\tau_n\}$,
      where each $\tau_i$ admits equality;
\item $\tauk\t$, where $t$ and all members of $\tauk$ admit equality;
\item $(\tau')\REF$.\index{23.1}
\end{itemize}
\label{tyfcn-lab}
A type function $\theta=\Lambda\alphak.\tau$\index{23.2}
 has arity $k$; \deletion{\theidstatus}{it must be
{\sl closed} -- i.e.
$\TyVarFcn(\tau)\subseteq\alphak$ -- and} the bound variables must
be distinct. Two type functions are considered equal
if they only differ in their choice of bound variables (alpha-conversion).
In particular, the equality attribute has no significance in a
bound variable of a type function; for example, $\Lambda\alpha.\alpha\to
\alpha$ and $\Lambda\beta.\beta\to\beta$ are equal type functions
even if $\alpha$ admits equality but $\beta$ does not.
%poly
\deletion{\thenoimptypes}{Similarly, the imperative attribute has no significance
in the bound variable of a type function.}
If $t$ has arity $k$, then we write $t$ to mean $\Lambda\alphak.\alphak\t$
(eta-conversion); thus $\TyNames\subseteq\TypeFcn$. $\theta=\Lambda\alphak.\tau$
is an {\sl equality} type function, or {\sl admits equality}, if when the
type variables $\alphak$ are chosen to admit equality then $\tau$ also admits
equality.

We write the application of a type function $\theta$ to a vector
$\tauk$ of types as $\tauk\theta$.
If $\theta=\Lambda\alphak.\tau$ we set $\tauk\theta=\tau\{\tauk/\alphak\}$
(beta-conversion).

We write $\tau\{\thetak/\tk\}$ for the result of substituting type
functions $\thetak$ for type names $\tk$ in $\tau$.
We assume that all beta-conversions
are carried out after substitution, so that for example
\[(\tauk\t)\{\Lambda\alphak.\tau/\t\}=\tau\{\tauk/\alphak\}.\]
%poly
\label{imp-ty-lab}
\deletion{\thenoimptypes}{A type is {\sl imperative} if all type variables occurring in it are
imperative.}
\subsection{Type Schemes}
\label{type-scheme-sec}
A type scheme $\tych=\forall\alphak.\tau$\index{23.3}
 {\sl generalises} a type $\tau'$,
written $\tych \succ\tau'$,
\replacement{\thenoimptypes}{if $\tau'=\tau\{\tauk/\alphak\}$ for some $\tauk$, where each member $\tau_i$
of $\tauk$ admits equality if $\alpha_i$ does,
%poly
and $\tau_i$ is imperative if $\alpha_i$ is imperative.}{if $\tau'=\tau\{\tauk/\alphak\}$ for some $\tauk$, where each member $\tau_i$
of $\tauk$ admits equality if $\alpha_i$ does.}
If $\tych'=\forall\beta^{(l)}.\tau'$ then $\tych$ {\sl generalises} $\tych'$,
written $\tych\succ\tych'$, if $\tych\succ\tau'$ and $\beta^{(l)}$ contains
no free type variable of $\tych$.
It can be shown that $\tych\succ\tych'$ iff, for all $\tau''$, whenever
$\tych'\succ\tau''$ then also $\tych\succ\tau''$.

Two type schemes $\tych$ and $\tych'$ are considered equal
if they can be obtained from each other by
renaming and reordering of bound type variables, and deleting type
variables from the prefix which do not occur in the body.
Here, in contrast to the case for type functions, the equality attribute
must be preserved in renaming; for example $\forall\alpha.\alpha\to\alpha$
and $\forall\beta.\beta\to\beta$ are only equal if either both $\alpha$
and $\beta$ admit equality, or neither does.
%poly
\deletion{\thenoimptypes}{Similarly, the imperative attribute of a bound type variable of a
type scheme {\sl is} significant.}
It can be shown that $\tych=\tych'$ iff $\tych\succ\tych'$ and
$\tych'\succ\tych$.

We consider a type $\tau$ to be a type scheme, identifying it with
$\forall().\tau$.
\oldpagebreak

\subsection{Scope of Explicit Type Variables}
\label{scope-sec}

In\index{23.10} the Core language, a type or datatype binding can
explicitly introduce type variables whose scope is that binding.
\insertion{\theexplicittyvars}{Moreover, in a value declaration
{\tt val $\tyvarseq$ $\valbind$}, the sequence $\tyvarseq$ binds
type variables: a type variable occurs free in
{\tt val $\tyvarseq$ $\valbind$} iff it occurs free in $\valbind$
and is not in the sequence $\tyvarseq$.}
\deletion{\theexplicittyvars}{
In the modules, a description of a value, type, or datatype
may contain explicit type variables whose scope is that
description.} However, \insertion{\theexplicittyvars}{explicit binding of type
variables at {\tt val} is optional, so} we\index{23.11} still have to account for the
scope of an explicit type variable occurring in the ``\ml{:}~$\ty$''
of a typed expression or pattern
or in the ``\ml{of} $\ty$'' of an exception binding. For the rest
of this section, we consider such \insertion{\theexplicittyvars}{free} occurrences of type variables only.

Every occurrence of a value declaration is said to
{\sl scope} a set of explicit type variables determined as follows.

%Every explicit type variable $\alpha$ is {\sl scoped at} a value binding
%which is determined as follows.

\replacement{\theexplicittyvars}{First, an occurrence of $\alpha$ in a value declaration $\valdec$ is said
to be {\sl unguarded} if the occurrence is not part of a smaller value
declaration within $\valbind$.}{First, a free occurrence of $\alpha$ in a value declaration
$\explicitvaldec$ is said
to be {\sl unguarded} if the occurrence is not part of a smaller value
declaration within $\valbind$.}
In this case we say that $\alpha$ {\sl occurs unguarded} in the
value declaration.

\replacement{\theexplicittyvars}{Then we say that $\alpha$ is {\sl scoped at}
a particular occurrence
$O$ of $\valdec$ in a program if}{Then we say that $\alpha$ is {\sl implicitly scoped at} a particular value declaration
{\tt val $\tyvarseq$ $\valbind$} in a program if}
(1) $\alpha$ occurs unguarded in this value declaration, and
(2) $\alpha$ does not occur unguarded in any larger value declaration
containing the \replacement{\theexplicittyvars}{occurrence $O$.}{given one.}\label{scope-def-lab}

\deletion{\theexplicittyvars}{
Hence, associated with every occurrence of a value declaration there is
a set $\U$ of the explicit type variables that are
scoped at that
occurrence. One may think of each occurrence of $\VAL$ as being implicitly
decorated with such a  set, for instance:

\vspace*{3mm}
\halign{\indent$#$&$#$&$#$\cr
\mbox{$\VAL_{\{\}}$ \ml{x = }}&\mbox{\ml{(}}&
\mbox{\ml{let $\VAL_{\{\mbox{\ml{'a}}\}}$ Id1:'a->'a = fn z=>z in Id1 Id1 end,}}\cr
& &\mbox{\ml{let $\VAL_{\{\mbox{\ml{'a}}\}}$ Id2:'a->'a = fn z=>z in Id2 Id2 end)}}\cr
\noalign{\vspace*{3mm}}
\mbox{$\VAL_{\{\mbox{\ml{'a}}\}}$ \ml{x = }}&\mbox{\ml{(}}&
\mbox{\ml{let $\VAL_{\{\}}$ Id:'a->'a = fn z=>z in Id Id end,}}\cr
& &\mbox{\ml{fn z=> z:'a)}}\cr}
}
\insertion{\thenoimptypes}{Henceforth, we assume that for every
value declaration $\boxml{val}\,\tyvarseq\cdots$ occurring in the
program, every explicit type variable implicitly scoped at the {\tt val}
has been added to $\tyvarseq$ (subject to the syntactic constraint in Section~\ref{synres.sec}). Thus for example, in the two declarations
\begin{tabbing}
\indent\=\tt  val x =  let val id:'a->'a = fn z=>z in id id end\\
       \>\tt  val x = (let val id:'a->'a = fn z=>z in id id end; fn z=>z:'a)
\end{tabbing}
the type variable \boxml{'a} is scoped differently; they become respectively
\begin{tabbing}
\indent\=\tt val x =  let val 'a id:'a->'a = fn z=>z in id id end\\
       \>\tt val 'a x = (let val id:'a->'a = fn z=>z in id id end; fn z=>z:'a)
\end{tabbing}
}

\replacement{\theidstatus}{According}{Then, according} to the
inference rules in Section~\ref{stat-cor-inf-rules}
the first example can be elaborated, but the second cannot since \ml{'a}
is bound at the outer value declaration leaving no possibility of two
different instantiations of the type of \replacement{\theidstatus}{\ml{Id}}{\ml{id}} in the application
\replacement{\theidstatus}{\ml{Id Id}}{\ml{id id}}.

\deletion{\thenoimptypes}{
\subsection{Non-expansive Expressions}
\label{expansive-sec}
In\index{23.4} order to treat polymorphic references and exceptions,
the set Exp of expressions is partitioned into two classes, the {\sl
expansive} and the {\sl non-expansive} expressions.
Any variable,
constructor and $\FN$ expression, possibly constrained by one or more
type expressions, is non-expansive; all other expressions are said to
be expansive.  The idea is that the dynamic evaluation of a
non-expansive expression will neither generate an exception nor extend
the domain of the memory, while the evaluation of an expansive
expression might.}

\insertion{\thenoimptypes}{
\subsection{Non-expansive Expressions}
\label{expansive-sec}
In\index{23.4} order to treat polymorphic references and exceptions,
the set Exp of expressions is partitioned into two classes, the {\sl
expansive} and the {\sl non-expansive} expressions.
An expression
     is {\sl non-expansive in context $\C$} if, after replacing infixed forms
     by their equivalent prefixed forms, and derived forms by their equivalent
     forms, it can be generated  by the following grammar from the
     non-terminal $\nexp$:
\medskip

\halign{&\indent\hfil$#$\ &\ $#$\hfil\ &\ $#$\hfil\cr
\nexp&::=&\scon & \hskip20mm\nexprow&::=&\boxml{$\lab$ = $\nexp\langle$, $\nexprow\rangle$}\cr
&&\langle\OP\rangle\longvid\cr
&&\ttlbrace\langle\nexprow\rangle\ttrbrace&\conexp&::=&\boxml{($\conexp\langle$:$\ty\rangle$)}\cr
&&\boxml{($\nexp$)}&&&\hbox{$\langle\OP\rangle\longvid$}\cr
&&\boxml{$\conexp\;\nexp$}\cr
&&\nexp \boxml{:} \ty\cr
%&&\boxml{$\nexp$ handle $\match$}\cr
&&\boxml{fn $\match$}\cr\noalign{\vskip6pt}}
\medskip

\hangindent=\parindent\hangafter=0\noindent
{\sl Restriction:}\/ Within a $\conexp$, we require $\longvid\neq\REF$ and
$\of{\is\,}{\,\C(\longvid)}\in\{\isc,\ise\}$.\medskip

\noindent
All other expressions are said to be {\sl expansive (in $C$)}.
The idea is that the dynamic evaluation of a
non-expansive expression will neither generate an exception nor extend
the domain of the memory, while the evaluation of an expansive
expression might.
}
\oldpagebreak
\subsection{Closure}
\label{closure-sec}
Let\index{24.2} $\tau$ be a type and $A$ a semantic object. Then $\cl{A}{(\tau)}$,
the {\sl closure} of $\tau$ with respect to $A$, is the type scheme
$\forall\alphak.\tau$, where $\alphak=\TyVarFcn(\tau)\setminus\TyVarFcn A$.
Commonly, $A$ will be a context $\C$.
We abbreviate the {\sl total} closure $\cl{\emptymap}{(\tau)}$ to
$\cl{}{(\tau)}$.
If the range of a \replacement{\theidstatus}{variable environment}{value
environment} $\VE$ contains only types (rather than
arbitrary type schemes) we set
\replacement{\theidstatus}{\[\cl{A}{\VE}=\{\id\mapsto\cl{A}{(\tau)}\ ;\ \VE(\id)=\tau\}\]}{\[\cl{A}{\VE}=\{\vid\mapsto(\cl{A}{(\tau)},\is)\ ;\ \VE(\vid)=(\tau,\is)\}\]}\deletion{\theidstatus}{with a similar definition for $\cl{A}{\CE}$.}

\label{clos-def-lab}
Closing\index{24.3} a \replacement{\theidstatus}{variable environment}{value environment} $\VE$ that stems from
the elaboration of a value binding $\valbind$ requires extra
care to ensure type security of references and exceptions and correct
scoping of explicit type variables.
Recall that $\valbind$ is not allowed to bind the
same variable twice. \replacement{\theidstatus}{Thus, for each $\var\in\Dom\VE$
there is a unique \mbox{\pat\ \ml{=} \exp}
in $\valbind$ which binds $\var$.}{Thus, for each $\vid\in\Dom\VE$
there is a unique \mbox{\pat\ \ml{=} \exp}
in $\valbind$ which binds $\vid$.}
If \replacement{\theidstatus}{$\VE(\var)=\tau$}{$\VE(\vid)=(\tau,\is)$}, let
\replacement{\theidstatus}{$\cl{\C,\valbind}{\VE(\var)}=\longtych$}{$\cl{\C,\valbind}{\VE(\vid)}=(\longtych,\is)$},
where
\replacement{\thenoimptypes}{\[\alphak=\cases{\TyVarFcn\tau\setminus\TyVarFcn\C,&if $\exp$
                                                    is non-expansive;\cr
                 \apptyvars\tau\setminus\TyVarFcn\C,&if $\exp$ is expansive.}
\]
}{
\[\alphak=\cases{\TyVarFcn\tau\setminus\TyVarFcn\C,&if $\exp$
                                                    is non-expansive in $\C$;\cr
                 (),&if $\exp$ is expansive in $\C$.}
\]}

\deletion{\thenoimptypes}{Notice that the form of $\valbind$ does not affect the binding of
applicative type variables, only the binding of imperative
type variables.}

\subsection{Type Structures and Type Environments}
\label{typeenv-wf-sec}
A type\index{24.4} structure
\replacement{\thece}{$(\theta,\CE)$}{$(\theta,\VE)$}\
is {\sl well-formed} if either
\replacement{\thece}{$\CE=\emptymap$}{$\VE=\emptymap$}, or $\theta$ is a type name $t$.
(The latter case arises, with \replacement{\thece}{$\CE\neq\emptymap$}{$\VE\neq\emptymap$}, in $\DATATYPE$
declarations.)
\insertion{\thenostrsharing}{An object or assembly $A$ of semantic objects is {\sl well-formed} if every type structure
occurring in $A$ is well-formed.}
\deletion{\thenostrsharing}{All type structures occurring in elaborations are
assumed to
be well-formed.}

A type structure \replacement{\thece}{$(\t,\CE)$}{$(\t,\VE)$}\ is said to
{\sl respect equality} if, whenever $\t$ admits equality, then
either $\t=\REF$ (see Appendix~\ref{init-stat-bas-app}) or,
for each \replacement{\theidstatus}{$\CE(\con)$}{$\VE(\vid)$} of the form
\replacement{\thece}{$\forall\alphak.(\tau\rightarrow\alphak\t)$,}{$(\forall\alphak.(\tau\rightarrow\alphak\t), \is)$,}
the type function $\Lambda\alphak.\tau$ also admits equality.
(This ensures that the equality
predicate ~{\tt =}~ will be applicable to a constructed value
\replacement{\theidstatus}{$(\con,v)$}{$(\vid,v)$} of
type $\tauk\t$ only when it is applicable to the value $v$ itself,
whose type is $\tau\{\tauk/\alphak\}$.)
A type environment $\TE$ {\sl respects equality} if all its type
structures do so.

Let $\TE$ be a type environment, and let $T$ be the set of type names
$\t$ such that \replacement{\thece}{$(\t,\CE)$ }{$(\t,\VE)$ } occurs in $\TE$ for some
\replacement{\thece}{$\CE\neq\emptymap$}{$\VE\neq\emptymap$}.
Then $\TE$ is said to {\sl maximise equality}
if (a) $\TE$ respects equality, and also (b) if any larger subset of
$T$ were to admit equality (without any change in the equality
attribute of any type names not in $T$) then $\TE$ would cease to
respect equality.

For any $\TE$ of the form
\replacement{\thece}{\[\TE=\{\tycon_i\mapsto(t_i,\CE_i)\ ;\ 1\leq i\leq k\},\]}{\[\TE=\{\tycon_i\mapsto(t_i,\VE_i)\ ;\ 1\leq i\leq k\},\]}
where no \replacement{\thece}{$\CE_i$}{$\VE_i$}\
is the empty map, and for any $\E$ we define
$\Abs(\TE,\E)$ to\index{25.1} be the environment obtained from
$\E$ and $\TE$ as
follows. First, let $\Abs(\TE)$ be the type environment
$\{\tycon_i\mapsto(t_i,\emptymap)\ ;\ 1\leq i\leq k\}$
in which all \replacement{\thece}{constructor}{value}
environments \adhocreplacementl{\thece}{-6cm}{$\CE_i$}{$\VE_i$}
have been replaced by the empty map.
Let $t_1',\cdots,t_k'$ be new distinct type names none of which
admit equality. Then $\Abs(\TE,\E)$ is the result of simultaneously
substituting
$t_i'$ for $t_i$, $1\leq i\leq k$,  throughout $\Abs(\TE)+\E$.
(The effect of the latter substitution is to ensure that the use of
equality on  an $\ABSTYPE$ is restricted to the $\WITH$ part.)
\label{abs-lab}
%\clearpage

\subsection{Inference Rules}
\label{stat-cor-inf-rules}
Each rule\index{26.1} of the semantics allows inferences among sentences of the form
\[A\ts{\it phrase}\ra A'\]
where
$A$ is usually \deletion{\theidstatus}{an environment or }a context, {\it phrase} is a phrase of
the Core, and $A'$ is a semantic object -- usually a type or an
environment.  It may be pronounced ``{\it phrase} elaborates to $A'$ in
(context\deletion{\theidstatus}{ or environment}) $A$''.  Some rules have extra hypotheses not of
this form; they are called {\sl side conditions}.

In the presentation of the rules, phrases within single
angle brackets ~$\langle\ \rangle$~ are called {\sl
first options}, and those within double
angle brackets ~$\langle\langle\ \rangle\rangle$~ are called {\sl
second options}.  To reduce the number of rules, we have adopted the
following convention:
\begin{quote} In each instance of a rule, the
first options must be either all present or all absent;
similarly the second options must be either all present or all absent.
\end{quote}

Although not assumed in our definitions, it is intended that every
context $\C=\T,\U,\E$ has the property that $\TyNamesFcn\E\subseteq\T$.
Thus $\T$ may be thought of, loosely, as containing all type names
which ``have been generated''. It is necessary to include $\T$ as a
separate component in a context, since $\TyNamesFcn\E$ may not contain
all the type names which have been generated; one reason is that a
context $\T,\emptyset,\E$ is a projection of the basis
\replacement{\thenostrsharing}{$\B=(\M,\T),\F,\G,\E$}{$\B=\T,\F,\G,\E$}
whose other components $\F$ and $\G$
could contain other such names -- recorded in $\T$ but not present in
$\E$.  Of course, remarks about what ``has been generated'' are not
precise in terms of the semantic rules. But the following precise result
may easily be demonstrated:
\begin{quote}
Let S be a sentence
~$\T,\U,\E\ts{\it phrase}\ra A$~ such that
$\TyNamesFcn\E\subseteq\T$,
and let S$'$ be a sentence
~$\T',\U',\E'\ts{\it phrase}'\ra A'$~
occurring in a proof of S; then also
$\TyNamesFcn\E'\subseteq\T'$.
\end{quote}

%                       Atomic Expressions
%
\rulesec{Atomic Expressions\index{26.2}}{\C\vdash\atexp\ra\tau}
%\begin{figure}[h]

\begin{equation}        % special constant
\label{sconexp-rule}
\frac{}
     {\C\ts\scon\ra\scontype(\scon)}\index{26.3}
\end{equation}

\replacement{\theidstatus}{\begin{equation}        % value variable
\label{varexp-rule}
\frac{\C(\longvar)\succ\tau}
     {\C\ts\longvar\ra\tau}
\end{equation}}{\begin{equation}        % value variable
\label{varexp-rule}
\frac{\C(\longvid) = (\sigma,\is)\qquad\sigma\succ\tau}
     {\C\ts\longvid\ra\tau}
\end{equation}}

\deletion{\theidstatus}{\begin{equation}        % value constructor
\label{conexp-rule}
\frac{\C(\longcon)\succ\tau}
     {\C\ts\longcon\ra\tau}
\end{equation}

\begin{equation}      % exception constant
%\label{exconexp-rule}
\frac{\C(\longexn)=\tau}
     {\C\ts\longexn\ra\tau}
\end{equation}}
\oldpagebreak

\begin{equation}        % record expression
%\label{recexp-rule}
\frac{\langle\C\ts\labexps\ra\varrho\rangle}
     {\C\ts\ttlbrace\ \recexp\ \ttrbrace\ra\emptymap\langle +\ \varrho\rangle{\rm\ in\ \Type}}\index{27.0}
\end{equation}
\vskip6pt

\replacement{\thesafelet}{\begin{equation}        % local declaration
\label{let-rule}
\frac{\C\ts\dec\ra\E\qquad\C\oplus\E\ts\exp\ra\tau}
     {\C\ts\letexp\ra\tau}\index{27.1}
\end{equation}}{\begin{equation}        % local declaration
\label{let-rule}
\frac{\C\ts\dec\ra\E\qquad\C\oplus\E\ts\exp\ra\tau\qquad\TyNamesFcn\tau\subseteq\of{\T}{\C}}
     {\C\ts\letexp\ra\tau}\index{27.1}
\end{equation}}

\begin{equation}        % paren expression
%\label{parexp-rule}
\frac{\C\ts\exp\ra\tau}
     {\C\ts\parexp\ra\tau}
\end{equation}
\comments
\begin{description}
\replacement{\theidstatus}{\item{(\ref{varexp-rule}),(\ref{conexp-rule})}
The instantiation of
type schemes allows different occurrences of a single $\longvar$
or $\longcon$ to assume different types.}{\item{(\ref{varexp-rule})}
The instantiation of
type schemes allows different occurrences of a single $\longvid$
to assume different types. Note that the identifier status is not
used in this rule.}
\item{(\ref{let-rule})}
The use of $\oplus$, here and elsewhere, ensures that
type names generated by the first sub-phrase are different from
type names generated by the second sub-phrase.\insertion{\thefixtypos}{The side condition
prevents type names generated by $\dec$ from escaping outside the local declaration.}
\end{description}

\rulesec{Expression Rows}{\C\ts\labexps\ra\varrho}
\begin{equation}        % expression rows
%\label{labexps-rule}
\frac{\C\ts\exp\ra\tau\qquad\langle\C\ts\labexps\ra\varrho\rangle}
     {\C\ts\longlabexps\ra\{\lab\mapsto\tau\}\langle +\ \varrho\rangle}\index{27.2}
\end{equation}
%                        Expressions
%
\rulesec{Expressions}{\C\ts\exp\ra\tau}
%\vspace{6pt}
%\fbox{$\C\ts\exp\ra\tau$}
\begin{equation}        % atomic
\label{atexp-rule}
\frac{\C\ts\atexp\ra\tau}
     {\C\ts\atexp\ra\tau}\index{27.3}
\end{equation}

\begin{equation}        % application
%\label{app-rule}
\frac{\C\ts\exp\ra\tau'\rightarrow\tau\qquad\C\ts\atexp\ra\tau'}
     {\C\ts\appexp\ra\tau}
\end{equation}

\begin{equation}        % typed
\label{typedexp-rule}
\frac{\C\ts\exp\ra\tau\qquad\C\ts\ty\ra\tau}
     {\C\ts\typedexp\ra\tau}
\end{equation}

\begin{equation}        % handle exception
%\label{handlexp-rule}
\frac{\C\ts\exp\ra\tau\qquad\C\ts\match\ra\EXCN\rightarrow\tau}
     {\C\ts\handlexp\ra\tau}
\end{equation}

\begin{equation}        % raise exception
\label{raiseexp-rule}
\frac{\C\ts\exp\ra\EXCN}
     {\C\ts\raisexp\ra\tau}
\end{equation}

\begin{equation}        % function
%\label{fnexp-rule}
\frac{\C\ts\match\ra\tau}
     {\C\ts\fnexp\ra\tau}
\end{equation}
\comments
\begin{description}
\item{(\ref{atexp-rule})}
The relational symbol $\ts$ is overloaded for all syntactic classes (here
atomic expressions and expressions).
\item{(\ref{typedexp-rule})}
Here $\tau$ is determined by $\C$ and $\ty$. Notice that type variables
in $\ty$ cannot be instantiated in obtaining $\tau$; thus the expression
\verb+1:'a+ will not elaborate successfully, nor will the expression
\verb+(fn x=>x):'a->'b+.
The effect of type variables in an explicitly typed expression is
to indicate exactly the degree of polymorphism present in the expression.\index{27.4}
\item{(\ref{raiseexp-rule})}
Note that $\tau$ does not occur in the premise; thus a $\RAISE$
expression has ``arbitrary'' type.
\end{description}
%                        Matches
%
\rulesec{Matches}{\C\ts\match\ra\tau}
\begin{equation}        % match
%\label{match-rule}
\frac{\C\ts\mrule\ra\tau\qquad\langle\C\ts\match\ra\tau\rangle}
     {\C\ts\longmatch\ra\tau}\index{28.1}
\end{equation}
\rulesec{Match Rules}{\C\ts\mrule\ra\tau}
\replacement{\thefixtypos}{
\begin{equation}        % mrule
%\label{mrule-rule}
\frac{\C\ts\pat\ra(\VE,\tau)\qquad\C+\VE\ts\exp\ra\tau'}
     {\C\ts\longmrule\ \ra\tau\rightarrow\tau'}
\end{equation}}{\begin{equation}        % mrule
\label{mrule-rule}
\frac{\C\ts\pat\ra(\VE,\tau)\qquad\C+\VE\ts\exp\ra\tau'\qquad\TyNamesFcn\VE\subseteq\of{\T}{\C}}
     {\C\ts\longmrule\ \ra\tau\rightarrow\tau'}
\end{equation}}
\comment  This rule allows new free type variables to enter
the context. These new type variables will be chosen, in effect, during
the elaboration of $\pat$ (i.e., in the inference of the first
hypothesis). In particular, their choice may have to be made to
agree with type variables present in any explicit type expression
occurring within $\exp$ (see rule~\ref{typedexp-rule}).

%
%                        Declarations
%
\rulesec{Declarations}{\C\ts\dec\ra\E}
%poly
\replacement{\theexplicittyvars}{
\begin{equation}        % value declaration
\label{valdec-rule}
\frac{\plusmap{\C}{\U}\ts\valbind\ra\VE\qquad
      \VE'=\cl{\C,\valbind}{\VE}\qquad
      \U\cap\TyVarFcn\VE'=\emptyset}
     {\C\ts\valdecS\ra\VE'\ \In\ \Env}\index{28.2}
\end{equation}}{\begin{equation}        % value declaration
\label{valdec-rule}
\frac{\begin{array}{l}
      U = \TyVarsFcn (\tyvarseq)\\
     \plusmap{\C}{\U}\ts\valbind\ra\VE\qquad
      \VE'=\cl{\C,\valbind}{\VE}\qquad
      \U\cap\TyVarFcn\VE'=\emptyset
      \end{array}}
     {\C\ts\boxml{val $\tyvarseq$ $\valbind$}\ra\VE'\ \In\ \Env}\index{28.2}
\end{equation}}

\begin{equation}        % type declaration
%\label{typedec-rule}
\frac{\C\ts\typbind\ra\TE}
     {\C\ts\typedec\ra\TE\ \In\ \Env}
\end{equation}

\begin{equation}        % datatype declaration
\label{datatypedec-rule}
\frac{\begin{array}{c}
\C\oplus\TE\ts\datbind\ra\VE,\TE\qquad
      \forall(\t,\adhocreplacementl{\thece}{9cm}{\CE}{\VE'})\in\Ran\TE,\ \t\notin(\of{\T}{\C}) \\
     \mbox{$\TE$ maximises equality}
     \end{array}
     }
     {\C\ts\datatypedec\ra(\VE,\TE)\ \In\ \Env}
\end{equation}

\insertion{\thedatatyperepl}{\begin{equation}        % datatype replication
\label{datatyperepldec-rule}
\frac{\C(\longtycon) = (\typefcn,\VE)\qquad
      \TE=\{\tycon\mapsto(\typefcn,\VE)\}
     }
     {\C\ts\datatyperepldec\ra(\VE,\TE)\ \In\ \Env}
\end{equation}}

\begin{equation}        % abstype declaration
\label{abstypedec-rule}
\frac{\begin{array}{rl}
      \C\oplus\TE\ts\datbind\ra\VE,\TE\qquad &
      \forall(\t,\adhocreplacementl{\thece}{9cm}{\CE}{\VE'})\in\Ran\TE,\ \t\notin(\of{\T}{\C})\\
      \C\oplus(\VE,\TE)\ts\dec\ra\E\qquad &
     \mbox{$\TE$ maximises equality}
      \end{array}
     }
     {\C\ts\abstypedec\ra\Abs(\TE,\E)}
\end{equation}
\vskip6pt

\replacement{\theidstatus}{\begin{equation}        % exception declaration
\label{exceptiondec-rule}
\frac{\C\ts\exnbind\ra\EE\quad\VE=\EE }
     {\C\ts\exceptiondec\ra(\VE,\EE)\ \In\ \Env }
\end{equation}}{\begin{equation}        % exception declaration
\label{exceptiondec-rule}
\frac{\C\ts\exnbind\ra\VE}
     {\C\ts\exceptiondec\ra\VE\ \In\ \Env }
\end{equation}}

\oldpagebreak
\begin{equation}        % local declaration
%\label{localdec-rule}
\frac{\C\ts\dec_1\ra\E_1\qquad\C\oplus\E_1\ts\dec_2\ra\E_2}
     {\C\ts\localdec\ra\E_2}\index{28.3}
\end{equation}
\vskip6pt

\replacement{\thenostrsharing}{\begin{equation}                % open declaration
%\label{open-dec-rule}
\frac{ \C(\longstrid_1)=(\m_1,\E_1)
            \quad\cdots\quad
       \C(\longstrid_n)=(\m_n,\E_n) }
     { \C\ts\openstrdec\ra \E_1 + \cdots + \E_n }
\end{equation}}{\begin{equation}                % open declaration
%\label{open-dec-rule}
\frac{ \C(\longstrid_1)= \E_1
            \quad\cdots\quad
       \C(\longstrid_n)= \E_n  }
     { \C\ts\openstrdec\ra \E_1 + \cdots + \E_n }
\end{equation}}
\vskip-4pt

\begin{equation}        % empty declaration
%\label{emptydec-rule}
\frac{}
     {\C\ts\emptydec\ra\emptymap\ \In\ \Env}
\end{equation}
\vskip4pt

\begin{equation}        % sequential declaration
%\label{seqdec-rule}
\frac{\C\ts\dec_1\ra\E_1\qquad\C\oplus\E_1\ts\dec_2\ra\E_2}
     {\C\ts\seqdec\ra\plusmap{E_1}{E_2}}
\end{equation}
\comments
\begin{description}
\item{(\ref{valdec-rule})}
Here $\VE$ will contain types rather than general
type schemes. \replacement{\theidstatus}{The closure of $\VE$ is exactly what allows variables to
be used polymorphically, via rule~\ref{varexp-rule}.}{The closure of $\VE$
allows value identifiers  to
be used polymorphically, via rule~\ref{varexp-rule}.}

\replacement{\theexplicittyvars}{Moreover, $\U$ is the set of explicit type variables scoped at this particular
occurrence of $\valdec$, cf. Section~\ref{scope-sec},
page~\pageref{scope-def-lab}. The side-condition on $\U$
ensures that these explicit type variables are bound by the closure
operation.}{The side-condition on $\U$
ensures that the type variables in $\tyvarseq$  are bound
by the closure operation,
if they occur free in the range of $\VE$.}

\replacement{\theidstatus}{On the other hand, no {\sl other} explicit type variable occurring
free in $\VE$ will become bound, since it must be in $\of{\U}{\C}$, and
is therefore excluded from closure by the definition of the closure operation
(Section~\ref{closure-sec}, page~\pageref{clos-def-lab})
since $\of{\U}{\C}\subseteq\TyVarFcn\C$.}{On the other hand,
if the phrase $\boxml{val}\,\tyvarseq\,\valbind$ occurs inside
some larger value binding $\boxml{val}\,\tyvarseq'\,\valbind'$
then no type variable $\alpha$ listed in $\tyvarseq'$ will become
bound by the $\cl{\C,\valbind}{\VE}$ operation; for $\alpha$ must
be in $\of{\U}{\C}$ and hence excluded from closure by the definition of the closure operation
(Section~\ref{closure-sec}, page~\pageref{clos-def-lab})
since $\of{\U}{\C}\subseteq\TyVarFcn\C$.}
\item{(\ref{datatypedec-rule}),(\ref{abstypedec-rule})}
The side conditions
express that the elaboration of each datatype binding
generates new type names and that as many of these new names
as possible admit equality.  Adding $\TE$ to the context on the left
of the $\ts$ captures the recursive nature of the binding.
%The side condition is
%the formal way of expressing that the elaboration of each datatype binding
%generates new type names.  Adding $\TE$ to the context on the left
%of the $\ts$ captures the recursive nature of the binding. Recall that $\TE$
%is assumed well-formed (as defined in Section~\ref{typeenv-wf-sec}). If
%$\TyNamesFcn(\of{\E}{\C})\subseteq\of{\T}{\C}$ and the side condition is
%satisfied then $\C\oplus\TE$ is well-formed.
\insertion{\thedatatyperepl}{\item{(\ref{datatyperepldec-rule})}
Note that no new type name is generated (i.e., datatype replication is
not generative). }
\item{(\ref{abstypedec-rule})}
The $\Abs$ operation was defined in Section~\ref{typeenv-wf-sec}, page~\pageref{abs-lab}.
\item{(\ref{exceptiondec-rule})}
\replacement{\theidstatus}{
No closure operation is used here, since $\EE$ maps exception
names to types rather than  to general type schemes.}{No closure operation is used here, as this would make the type system unsound.
Example: {\tt exception E of 'a; val it = (raise E 5) handle E f => f(2)}~.}
\deletion{\theidstatus}{
Note that $\EE$ is also recorded in the {\VarEnv} component of
the resulting environment (see Section~\ref{stat-proj}, page~\pageref{stat-proj}).\index{29.0}}
\end{description}

%                        Bindings
%
\rulesec{Value Bindings}{\C\ts\valbind\ra\VE}
%\vspace{6pt}
\begin{equation}        % value binding
\label{valbind-rule}
\frac{\C\ts\pat\ra(\VE,\tau)\qquad\C\ts\exp\ra\tau\qquad
      \langle\C\ts\valbind\ra\VE'\rangle }
     {\C\ts\longvalbind\ra\VE\ \langle +\ \VE'\rangle}\index{29.1}
\end{equation}

\replacement{\thefixtypos}{
\begin{equation}        % recursive value binding
\label{recvalbind-rule}
\frac{\C+\VE\ts\valbind\ra\VE}
     {\C\ts\recvalbind\ra\VE}
\end{equation}}{\begin{equation}        % recursive value binding
\label{recvalbind-rule}
\frac{\C+\VE\ts\valbind\ra\VE\qquad\TyNamesFcn\VE\subseteq\of{\T}{\C}}
     {\C\ts\recvalbind\ra\VE}
\end{equation}}
\comments
\begin{description}
\item{(\ref{valbind-rule})}
When the option is present we have $\Dom\VE\cap
\Dom\VE' = \emptyset$ by the syntactic restrictions.\index{29.2}
\oldpagebreak
\item{(\ref{recvalbind-rule})}
Modifying $\C$ by $\VE$ on the left captures the
recursive nature of the binding. From rule~\ref{valbind-rule} we see that any
type scheme occurring in $\VE$ will have to be a type. Thus each use of a
recursive function in its own body must be \replacement{\thenostrsharing}{ascribed}{assigned} the same type.
\insertion{\theidstatus}{Also note that $\C+\VE$ may overwrite
identifier status. For example, the program
    {\tt datatype t = f; val rec f = fn x => x;}~~  is legal.}
\end{description}

\rulesec{Type Bindings}{\C\ts\typbind\ra\TE}
%\fbox{$\C\ts\typbind\ra\TE$}
\begin{equation}        % type binding
%\label{typbind-rule}
\frac{\tyvarseq=\alphak\qquad\C\ts\ty\ra\tau\qquad
      \langle\C\ts\typbind\ra\TE\rangle}
     {\begin{array}{c}
      \C\ts\longtypbind\ra\\
      \qquad\qquad\qquad
      \{\tycon\mapsto(\typefcnk,\emptymap)\}\ \langle +\ \TE\rangle
      \end{array}
     }\index{29.3}
\end{equation}
\comment The syntactic restrictions ensure that the type function
$\typefcnk$ satisfies the well-formedness \replacement{\theidstatus}{constraints }{constraint }of
Section~\ref{tyfun-sec} and they ensure $tycon\notin\Dom\TE$.

\rulesec{Datatype Bindings}{\C\ts\datbind\ra\VE,\TE}
%\fbox{$\C\ts\datbind\ra\VE,\TE$}
\replacement{\theidstatus}{\begin{equation}        % datatype binding
\frac{\begin{array}{c}
        \tyvarseq=\alphak\qquad\C,\alphakt\ts\constrs\ra\CE\\
        \langle\C\ts\datbind\ra\VE,\TE\qquad
        \forall(\t',\CE)\in\Ran\TE, \t\neq\t'\rangle
      \end{array}
     }
     {\begin{array}{c}
        \C\ts\longdatbind\ra\\
        \qquad\qquad\qquad\cl{}{\CE}\langle +\ \VE\rangle,\
        \{\tycon\mapsto(\t,\cl{}{\CE})\}\ \langle +\ \TE\rangle
      \end{array}
     }\index{30.1}
\end{equation}}{\begin{equation}        % datatype binding
\frac{\begin{array}{c}
        \tyvarseq=\alphak\qquad\C,\alphakt\ts\constrs\ra\VE\qquad\arity\t=k\\
        \langle\C\ts\datbind'\ra\VE',\TE'\qquad
        \forall(\t',\VE'')\in\Ran\TE', \t\neq\t'\rangle
      \end{array}
     }
     {\begin{array}{l}
        \C\ts\newlongdatbind\ra\\
        \qquad(\cl{}{\VE}\langle +\ \VE'\rangle,\
        \{\tycon\mapsto(\t,\cl{}{\VE})\}\ \langle +\ \TE'\rangle
      \end{array}
     }\index{30.1}
\end{equation}}
\comment \replacement{\thece}{The syntactic restrictions ensure $\Dom\VE\cap\Dom\CE = \emptyset$
and $\tycon\notin\Dom\TE$.}{The syntactic restrictions ensure $\Dom\VE\cap\Dom\VE' = \emptyset$
and $\tycon\notin\Dom\TE'$.}

\replacement{\thece}{\rulesec{Constructor Bindings}{\C,\tau\ts\constrs\ra\CE}}{\rulesec{Constructor Bindings}{\C,\tau\ts\constrs\ra\VE}}
%\fbox{$\C,\tau\ts\constrs\ra\CE$}
\replacement{\theidstatus}{\begin{equation}        % data constructors
%\label{constrs-rule}
\frac{\langle\C\ts\ty\ra\tau'\rangle\qquad
      \langle\langle\C,\tau\ts\constrs\ra\CE\rangle\rangle }
     {\begin{array}{c}
      \C,\tau\ts\longerconstrs\ra\\
      \qquad\qquad\qquad\{\con\mapsto\tau\}\
     \langle +\ \{\con\mapsto\tau'\to\tau\}\ \rangle\
      \langle\langle +\ \CE\rangle\rangle
      \end{array}
     }\index{30.2}
\end{equation}}{\begin{equation}        % data constructors
%\label{constrs-rule}
\frac{\langle\C\ts\ty\ra\tau'\rangle\qquad
      \langle\langle\C,\tau\ts\constrs\ra\VE\rangle\rangle }
     {\begin{array}{c}
      \C,\tau\ts\longervidconstrs\ra\\
      \qquad\qquad\qquad\{\vid\mapsto(\tau,\isc)\}\
     \langle +\ \{\vid\mapsto(\tau'\to\tau,\isc)\}\ \rangle\
      \langle\langle +\ \VE\rangle\rangle
      \end{array}
     }\index{30.2}
\end{equation}}
\comment By the syntactic restrictions \replacement{\theidstatus}{$\con\notin\Dom\CE$.}{$\vid\notin\Dom\VE$.}

\replacement{\theidstatus}{
\rulesec{Exception Bindings}{\C\ts\exnbind\ra\EE}}{
\rulesec{Exception Bindings}{\C\ts\exnbind\ra\VE}}
%poly with polymorphic exceptions:
% \ replacement{\thenoimptypes}{\begin{equation}        % exception binding
%  \label{exnbind1-rule}
%  \frac{\langle\C\ts\ty\ra\tau\quad\mbox{$\tau$ is imperative}\rangle\qquad
%        \langle\langle\C\ts\exnbind\ra\EE\rangle\rangle }
%       {\begin{array}{c}
%        \C\ts\longexnbinda\ra\\
%        \qquad\qquad\qquad\{\exn\mapsto\EXCN\}\
%        \langle +\ \{\exn\mapsto\tau\rightarrow\EXCN\}\ \rangle\
%        \langle\langle +\ \EE\rangle\rangle
%        \end{array}
%       }\index{30.3}
%  \end{equation}}{\begin{equation}        % exception binding
%  \label{exnbind1-rule}
%  \frac{\langle\C\ts\ty\ra\tau\rangle\qquad
%        \langle\langle\C\ts\exnbind\ra\EE\rangle\rangle }
%       {\begin{array}{c}
%        \C\ts\longexnbinda\ra\\
%        \qquad\qquad\qquad\{\exn\mapsto\EXCN\}\
%        \langle +\ \{\exn\mapsto\tau\rightarrow\EXCN\}\ \rangle\
%        \langle\langle +\ \EE\rangle\rangle
%        \end{array}
%       }\index{30.3}
%  \end{equation}}
\replacement{\theidstatus}{\begin{equation}        % exception binding
\label{exnbind1-rule}
\frac{\langle\C\ts\ty\ra\tau\quad\mbox{$\tau$ is imperative}\rangle\qquad
      \langle\langle\C\ts\exnbind\ra\EE\rangle\rangle }
     {\begin{array}{c}
      \C\ts\longexnbinda\ra\\
      \qquad\qquad\qquad\{\exn\mapsto\EXCN\}\
      \langle +\ \{\exn\mapsto\tau\rightarrow\EXCN\}\ \rangle\
      \langle\langle +\ \EE\rangle\rangle
      \end{array}
     }\index{30.3}
\end{equation}}{\begin{equation}        % exception binding
\label{exnbind1-rule}
\frac{\langle\C\ts\ty\ra\tau\rangle\qquad
      \langle\langle\C\ts\exnbind\ra\VE\rangle\rangle }
     {\begin{array}{l}
      \C\ts\longvidexnbinda\ra\\
      \qquad\{\vid\mapsto(\EXCN,\ise)\}\
      \langle +\ \{\vid\mapsto(\tau\rightarrow\EXCN,\ise)\}\ \rangle\
      \langle\langle +\ \VE\rangle\rangle
      \end{array}
     }\index{30.3}
\end{equation}}
\vskip-4pt

%\vspace*{4mm}
\replacement{\theidstatus}{\begin{equation}        % exception binding
\label{exnbind2-rule}
\frac{\C(\longexn)=\tau\qquad
      \langle\C\ts\exnbind\ra\EE\rangle }
      {\C\ts\longexnbindb\ra\{\exn\mapsto\tau\}\ \langle +\ \EE\rangle}
\end{equation}}{\begin{equation}        % exception binding
\label{exnbind2-rule}
\frac{\C(\longvid)=(\tau,\ise)\qquad
      \langle\C\ts\exnbind\ra\VE\rangle }
      {\C\ts\longvidexnbindb\ra\{\vid\mapsto(\tau,\ise)\}\ \langle +\ \VE\rangle}
\end{equation}}
\comments
\begin{description}
\replacement{\thenoimptypes}{\item{(\ref{exnbind1-rule})} Notice that $\tau$ must not contain
any applicative type variables.}{\item{(\ref{exnbind1-rule})} Notice that $\tau$ may contain
type variables.}\index{30.35}
%with monotyped exceptions:
%\item{(\ref{exnbind1-rule})} Notice that $\tau$ must be a monotype
%(see also restriction~\ref{monotypes-res} in
%Section~\ref{further-restrictions-sec}).
\oldpagebreak
\item{(\ref{exnbind1-rule}),(\ref{exnbind2-rule})}
\replacement{\theidstatus}{
There is a unique $\EE$, for each $\C$ and $\exnbind$,
such that $\C\ts\exnbind\ra\EE$.}{For each $\C$ and $\exnbind$,
there is at most one $\VE$ satisfying $\C\ts\exnbind\ra\VE$.}
\end{description}

%\caption{Rules for Bindings}
%\end{figure}

%                        Atomic Patterns
%
\rulesec{Atomic Patterns}{\C\ts\atpat\ra(\VE,\tau)}
%\vspace{6pt}
%\fbox{$\C\ts\atpat\ra(\VE,\tau)$}
\begin{equation}        % wildcard pattern
%\label{wildcard-rule}
\frac{}
     {\C\ts\wildpat\ra (\emptymap,\tau)}\index{30.4}
\end{equation}

\begin{equation}        % special constant in pattern
\frac{}
     {\C\ts\scon\ra (\emptymap,\scontype(\scon))}\index{30.5}
\end{equation}

\replacement{\theidstatus}{
\begin{equation}        % variable pattern
\label{varpat-rule}
\frac{}
     {\C\ts\var\ra (\{\var\mapsto\tau\},\tau) }
\end{equation}}{\begin{equation}        % variable pattern
\label{varpat-rule}
\frac{\hbox{$\vid\notin\Dom(\C)$ or $\of{\is}{C(\vid)} = \isv$}}
     {\C\ts\vid\ra (\{\vid\mapsto(\tau,\isv)\},\tau) }
\end{equation}}

\replacement{\theidstatus}{
\begin{equation}        % constant pattern
%\label{constpat-rule}
\frac{\C(\longcon)\succ\tauk\t }
     {\C\ts\longcon\ra (\emptymap,\tauk\t)}
\end{equation}}{\begin{equation}        % constant pattern
\label{constpat-rule}
\frac{\C(\longvid)=(\sigma,\is)\qquad\is\neq\isv\qquad\sigma\succ\tauk\t }
     {\C\ts\longvid\ra (\emptymap,\tauk\t)}
\end{equation}}

\deletion{\theidstatus}{
\begin{equation}       % exception constant
%\label{exconapat-rule}
\frac{\C(\longexn)=\EXCN}
     {\C\ts\longexn\ra (\emptymap,\EXCN)}
\end{equation}}

\begin{equation}        % record pattern
%\label{recpat-rule}
\frac{\langle\C\ts\labpats\ra(\VE,\varrho)\rangle}
     {\C\ts\lttbrace\ \recpat\ \rttbrace\ra(\ \emptymap\langle +\ \VE\rangle,\ \emptymap
      \langle +\ \varrho\rangle\ \In\ \Type\ ) }\index{31.1}
\end{equation}

\begin{equation}        % parenthesised pattern
%\label{parpat-rule}
\frac{\C\ts\pat\ra(\VE,\tau)}
     {\C\ts\parpat\ra(\VE,\tau)}
\end{equation}
\comments
\begin{description}
\replacement{\theidstatus}{\item{(\ref{varpat-rule})}
Note that $\var$ can assume a type, not a general type scheme.}{\item{(\ref{varpat-rule}), (\ref{constpat-rule})} The context $\C$ determines which of these
two rules applies. In rule~\ref{varpat-rule}, note that
$\vid$ can assume a type, not a general type scheme.}
\end{description}

\rulesec{Pattern Rows}{\C\ts\labpats\ra(\VE,\varrho)}
%\fbox{$\C\ts\labpats\ra(\VE,\varrho)$}
\begin{equation}        % wildcard record
%\label{wildrec-rule}
\frac{}
     {\C\ts\wildrec\ra(\emptymap,\varrho)}\index{31.2}
\end{equation}

\replacement{\theidstatus}{\begin{equation}        % record component
\label{longlab-rule}
\frac{\C\ts\pat\ra(\VE,\tau)\qquad
      \langle\C\ts\labpats\ra(\VE',\varrho)\qquad\lab\notin\Dom\varrho\rangle
}
     {\C\ts\longlabpats\ra
     (\VE\langle +\ \VE'\rangle,\
      \{\lab\mapsto\tau\}\langle +\ \varrho\rangle) }
\end{equation}}{\begin{equation}        % record component
\label{longlab-rule}
\frac{\begin{array}{c}\C\ts\pat\ra(\VE,\tau)\\
      \langle\C\ts\labpats\ra(\VE',\varrho)\qquad\Dom\VE\cap\Dom\VE' = \emptyset\rangle
      \end{array}
}
     {\C\ts\longlabpats\ra
     (\VE\langle +\ \VE'\rangle,\
      \{\lab\mapsto\tau\}\langle +\ \varrho\rangle) }
\end{equation}}
\deletion{\theidstatus}{\comment
\begin{description}
\item{(\ref{longlab-rule})}
 By the syntactic restrictions, $\Dom\VE\cap\Dom\VE' = \emptyset$.
\end{description}}
\insertion{\theidstatus}{\comment
\begin{description}
\item{(\ref{longlab-rule})}
 The syntactic restrictions ensure $\lab\notin\Dom\varrho$.
\end{description}}
%                        Patterns
%
%\begin{figure}[h]
%\label{pat-rules}
\oldpagebreak
\rulesec{Patterns}{\C\ts\pat\ra(\VE,\tau)}
\begin{equation}        % atomic pattern
%\label{atpat-rule}
\frac{\C\ts\atpat\ra (\VE,\tau)}
     {\C\ts\atpat\ra (\VE,\tau)}\index{31.3}
\end{equation}

\replacement{\theidstatus}{
\begin{equation}        % construction pattern
%\label{conpat-rule}
\frac{\C(\longcon)\succ\tau'\to\tau\qquad\C\ts\atpat\ra(\VE,\tau')}
     {\C\ts\conpat\ra (\VE,\tau)}
\end{equation}}{\begin{equation}        % construction pattern
%\label{conpat-rule}
\frac{\C(\longvid) = (\sigma, \is)\qquad\is\neq\isv\qquad \sigma\succ\tau'\to\tau\qquad\C\ts\atpat\ra(\VE,\tau')}
     {\C\ts\vidpat\ra (\VE,\tau)}
\end{equation}}

\deletion{\theidstatus}{
\begin{equation}       %  exception construction pattern
%\label{exconpat-rule}
\frac{\C(\longexn)=\tau\rightarrow\EXCN\qquad
      \C\ts\atpat\ra(\VE,\tau)}
     {\C\ts\exconpat\ra(\VE,\EXCN)}
\end{equation}}

\begin{equation}        % typed pattern
%\label{typedpat-rule}
\frac{\C\ts\pat\ra(\VE,\tau)\qquad\C\ts\ty\ra\tau}
     {\C\ts\typedpat\ra (\VE,\tau)}
\end{equation}

\replacement{\theidstatus}{
\begin{equation}        % layered pattern
\label{layeredpat-rule}
\frac{\begin{array}{c}
      \C\ts\var\ra(\VE,\tau)\qquad\langle\C\ts\ty\ra\tau\rangle\\
      \C\ts\pat\ra(\VE',\tau)
      \end{array}
     }
     {\C\ts\layeredpat\ra(\plusmap{\VE}{\VE'},\tau)}
\end{equation}}{\begin{equation}        % layered pattern
\label{layeredpat-rule}
\frac{\begin{array}{c}
     \hbox{$\vid\notin\Dom(\C)$ or $\of{\is}{C(\vid)} = \isv$}\\
      \langle\C\ts\ty\ra\tau\rangle\qquad
      \C\ts\pat\ra(\VE,\tau)\qquad \vid\notin\Dom\VE
      \end{array}
     }
     {\C\ts\layeredvidpat\ra(\plusmap{\{\vid\mapsto(\tau,\isv)\}}{\VE},\tau)}
\end{equation}}
\deletion{\theidstatus}{\comments
\begin{description}
\item{(\ref{layeredpat-rule})}
By the syntactic restrictions, $\Dom\VE\cap\Dom\VE' = \emptyset$.
\end{description}}

%                        Type Expressions
\rulesec{Type Expressions}{\C\ts\ty\ra\tau}
\begin{equation}        % atype variable
%\label{tyvar-rule}
\frac{\tyvar=\alpha}
     {\C\ts\tyvar\ra\alpha}\index{32.1}
\end{equation}

\begin{equation}        % record type
%\label{rectype-rule}
\frac{\langle\C\ts\labtys\ra\varrho\rangle}
     {\C\ts\lttbrace\ \rectype\ \rttbrace\ra\emptymap\langle +\ \varrho\rangle\ \In\ \Type}
\end{equation}

\begin{equation}        % constructed type
\label{constype-rule}
\frac{\begin{array}{c}
      \tyseq=\ty_1\cdots\ty_k\qquad\C\ts\ty_i\ra\tau_i\ (1\leq i\leq k)\\
      \C(\longtycon)=(\theta,\adhocreplacementl{\thece}{8cm}{\CE}{\VE})
      \end{array}
     }
     {\C\ts\constype\ra\tauk\theta}
\end{equation}

\begin{equation}        % function type
%\label{funtype-rule}
\frac{\C\ts\ty\ra\tau\qquad\C\ts\ty\,'\ra\tau'}
     {\C\ts\funtype\ra\tau\to\tau'}
\end{equation}

\begin{equation}        % parenthesised type
%\label{partype-rule}
\frac{\C\ts\ty\ra\tau}
     {\C\ts\partype\ra\tau}
\end{equation}
\comments
\begin{tabbing}
(\ref{constype-rule}) \= Recall that for $\tauk\theta$ to be defined, $\theta$
must have arity $k$.
\end{tabbing}

\oldpagebreak
\rulesec{Type-expression Rows}{\C\ts\labtys\ra\varrho}
%\fbox{$\C\ts\labtys\ra\varrho$}
\begin{equation}        % record type components
%\label{longlabtys-rule}
\frac{\C\ts\ty\ra\tau\qquad\langle\C\ts\labtys\ra\varrho\rangle}
     {\C\ts\longlabtys\ra\{\lab\mapsto\tau\}\langle +\ \varrho\rangle}\index{32.15}
\end{equation}
\comment The syntactic constraints ensure $\lab\notin\Dom\varrho$.
%\caption{Rules for Types}
%\end{figure}

\subsection{Further Restrictions}
\label{further-restrictions-sec}
There\index{32.2} are a few restrictions on programs which should be enforced by a
compiler, but are better expressed apart from the preceding
Inference Rules.  They are:
\begin{enumerate}
%poly
%The following restriction has been removed from Version 1 because
%of polymorphic references and exceptions
%\item \label{monotypes-res}
%The reference value constructor ~{\tt ref}~ may occur in patterns
%with polymorphic type, but in an expression it must always elaborate to
%a monotype, i.e. a type containing no type variables.
%Moreover, exception constructors can have monotypes only.
%These restrictions
%will be relaxed in future Versions, but some restrictions will always be
%necessary to ensure soundness of the type discipline.
\item For each occurrence of a record pattern containing a record wildcard,
i.e. of the form
%\begin{quote}
\verb+{+${\it lab}_1$\ml{=}$\pat_1$\ml{,}$\cdots$\ml{,}${\it lab}_m$\ml{=}$\pat_m$\ml{,}\verb+...}+
%\end{quote}
the program context must determine uniquely the domain
$\{{\it lab}_1,\cdots,{\it lab}_n\}$
of its \replacement{\theidstatus}{record}{row} type, where $m\leq n$; thus, the context must
determine the labels $\{{\it lab}_{m+1},\cdots,{\it lab}_n\}$ of the fields
to be matched by the wildcard. For this purpose, an explicit type
constraint may be needed.  \deletion{\thenoimptypes}{This restriction is necessary to ensure the
existence of principal type schemes.}
\item In a match of the form ${\it pat}_1$ \ml{=>} $\exp_1$ \ml{|}$\;\cdots\;$
\ml{|} ${\tt pat}_n$ \ml{=>} $\exp_n$
the pattern sequence $\pat_1,\ldots,\pat_n$ should be {\sl irredundant};
that is, each $\pat_j$ must match some value
(of the right type) which is not matched by $\pat_i$ for any $i<j$.
In the context {\fnexp}, the $\match$ must also be {\sl exhaustive}; that is,
every value (of the right type) must be matched by some $\pat_i$.
The compiler must give warning on violation of these restrictions,
but should still compile the match.
The restrictions are inherited by derived forms; in particular,
this means that in the function\insertion{\thefixtypos}{-value} binding\index{33.1}
\replacement{\theidstatus}{$\var\ \atpat_1\ \cdots\ \atpat_n\langle : \ty\rangle$\ \ml{=}\ $\exp$}{$\vid\ \atpat_1\ \cdots\ \atpat_n\langle : \ty\rangle$\ \ml{=}\ $\exp$}
(consisting of one clause only), each separate $\atpat_i$ should be
exhaustive by itself.
%must be {\sl irredundant} and {\sl exhaustive}.  That is, each ${\it pat}_j$
%must match {\sl some} value (of the right type) which is not matched by
%${\it pat}_i$ for any $i <j$, and {\sl every} value (of the right type) must be
%matched by some ${\it pat}_i$. The compiler must give a warning on violation
%of this restriction, but should still compile the match.
\insertion{\theidstatus}{\item
        For each value binding   $\pat\ \mbox{\ml{=}}\ \exp$
        the compiler must issue a report (but
        still compile) if   $\pat$  is not exhaustive.
        This will detect a mistaken
        declaration like  $\VAL\ \ml{nil}\ \mbox{\ml{=}}\ \exp$
        in which the user expects to declare
        a new variable \ml{nil}
        (whereas the language dictates that \ml{nil} is here a
        constant pattern, so no variable gets declared).
        However, this warning
        should not be given when the binding is a component
        of a top-level declaration
        $\valdec$; e.g.
        $\VAL\ \mbox{\ml{x::l = }}\exp_1\ \mbox{\ml{\AND\ y = }}\exp_2$
        is not faulted by the
        compiler at top level, but may of course generate a \ml{Bind}
        exception (see Section~\ref{bas-exc}).}
\end{enumerate}

\deletion{\theprincipalenv}{
\subsection{Principal Environments}
\label{principal-env-sec}
The\index{33.15} notion of {\sl enrichment}, $\E\succ\E'$, between environments
$\E=(\SE,\TE,\VE,\EE)$ and $\E'=(\SE',\TE',\VE',\EE')$ is defined
in Section~\ref{enrichment-sec}. For the present section,  $\E\succ\E'$
may be taken to mean $\SE=\SE'=\emptymap$, $\TE=\TE'$,
$\EE=\EE'$, $\Dom\VE=\Dom\VE'$ and, for each $\id\in\Dom\VE$,
$\VE(\id)\succ\VE'(\id)$.

Let\index{33.2} $\C$ be a context, and suppose that $\C\ts\dec\ra\E$
according to the preceding Inference Rules. Then $E$ is {\em principal}
(for $\dec$ in the context $\C$) if, for all $\E'$ for
which $\C\ts\dec\ra\E'$, we have $\E\succ\E'$. We claim that if
$\dec$ elaborates to any environment in $\C$ then it elaborates to
a principal environment in $\C$. Strictly, we must allow for the
possibility that type names and imperative
 type variables
which do not occur in $\C$ are chosen
differently for $\E$ and $\E'$.
The stated claim is therefore made up to such variation.
}


\documentclass{article}
\usepackage[utf8]{inputenc}

\title{5760 Project 1}
\author{Scott Carnahan}
\date{February 2019}

\usepackage{natbib}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{float}
\usepackage[]{pdfpages}
\usepackage{siunitx}
\begin{document}

\maketitle
\tableofcontents

\section{Introduction}
For this project, I added a spherical reflective grating in place of the previous detector as well as a matched cylindrical detector along the grating's Rowland's Circle.  In the image below you'll see I made the detector exceedingly large in order ot be able to capture multiple diffraction orders.

\begin{figure}[H]
    \centering
    \includegraphics[width=10cm]{figures/lab_view.png}
    \caption{A 3d view of the ray trace with an impossibly large detector}
    \label{}

\end{figure}

\section{Diffraction Math}
First, we were asked to set $\beta$ to $0$. To do this, I focused on the center of the grating where I can treat it just as a flat grating with equal spacing. In that case, the grating equation reduces to:

\begin{equation}
    \sin{\alpha} = \frac{m \lambda}{d}
\end{equation}
where d for us is $1 / 3600$. I am assuming we are interested in the first order diffraction, as well. By inverting the sin, I got alpha. Now, with $\beta = 0$, a paraxial ray hitting the center of the grating would diffract along the normal of the spherical surface.  What I have to do is just rotate the sphere off-axis by $\alpha$.

The grating is placed with its vertex at $R\cos{\alpha}$ from the cassegrain focus. Then, the cylinder is placed along the Rowland's Circle with a diameter equal to the grating radius.

\subsection{Grating Math}
In order to keep my code vectorized, I followed the math by Spencer and Murty in \textit{General Ray Tracing Procedure}, 1961.  This treats the grating as the intersection of some virtual surfaces with the grating surface. If $\hat{p} = <u, v, w>$ is the normal of the virtual surfaces, then
\begin{equation}
    u = \frac{1}{\sqrt{1 + \frac{K^2}{L^2 + M^2}}}
\end{equation}

where $\hat{r} = <K, L, M>$ is the surface normal of the sphere at a point. Then, the local line spacing $d$ is the parallel line space $d_p$, divided by $u$:

\begin{equation}
    \label{eq:local}
    d = d_p / u
\end{equation}

\subsection{Diffraction Direction}
The governing equation is
\begin{equation}
    \label{eq:fun}
    S^\prime = \mu S - \Lambda p + \Gamma r
\end{equation}
where $S$ is the direction vector in and $S^\prime$ is the direction out. $\Lambda$ is defined as $\frac{n\lambda}{N^\prime d}$ where the addition of $N^\prime$ has to do with refraction through non-similar mediums. $\Gamma$ is defined by a quadrati equation which the authors insist on iterating to a solution for a reason I don't understand. It incorporates the surface normal, beam direction, and grating direction.

\section{Spectrogram}
In order to contain my enthusiasm, I wrote the grating to only handle one wavelength and one order at a time. So, I looped over the wavelengths requested. Orders 0, 1, and 2 are shown on the graph. It's a bit large, but clear if zoomed in. The wavelengths travel angularly around the cylinder while the aberrations travel axially on the cylinder. The x-axis is in [radians] but this is equivalent to [meters] of circumference for this case. \bf{The image is included at the end of the file due to its size.}

\section{Limiting Resolving Power}
The limiting resolving power is given as
\begin{equation}
    \label{eq:resolve}
    R = \frac{\lambda}{\Delta \lambda}
\end{equation}
where $\lambda$ is the center wavelength and $\Delta \lambda$ is the minimum distinguishable wavelength different in the image.  For this, we can just assume that two wavelengths would need to be $100\%$ separated to achieve resolution, so we proceed:
\begin{equation}
    \label{eq:Amm}
    \frac{\si{\angstrom}}{mm} = \frac{10^7\cos{\beta}}{ml\frac{1}{d}}
\end{equation} which reduces to
\begin{equation}
    \label{eq:Ammred}
    \frac{\si{\angstrom}}{mm} = \frac{10^7 * 1}{1 * 1000 * 3600}
\end{equation} for our case and gives about 2.7. This is confirmed by measuring the mm between $100 \si{\angstrom}$ on the graph. Then, we multiply the resolution by the spread of a single wavelength line to estimate the resolving power.
\begin{equation}
    \label{eq:resolvingPower}
    R = \frac{\si{\angstrom}}{mm} \Delta \lambda \approx
\end{equation} The results are:
\begin{center}
    \input "../resolvingTable.txt"
\end{center}
Notably, the resolving power is probably better than this. This measure includes the width of the entire wavelegnth blob including the curve. This curve wouldn't actually run into other blobs because they too are curved. I would eyeball that the actual resolving power may be 5x better than this at the central wavelength, so maybe 30,000 - as long as diffraction didn't get in the way.

\eject \pdfpagewidth=22.5in \pdfpageheight=7.5in
\includegraphics{figures/spectrum.png}

% \bibliograph\sin{\alpha} = \frac{m \lambda}{d}ystyle{plain}
% \bibliography{references}
\end{document}


\subsection{Solving sudokus with hybrid data-structures}

This section presents an implementation of a
backtracking Sudoku solver using a safe API for persistent arrays that
supports both mutable updates and immutable versioning.
The implementation showcases safe mixing of resource allocation and
deallocation in the presence of exclusive (mutable) and immutable
borrows. It also demonstrates two new aspects: the interaction between
closures and borrows and the notion of reborrowing.
%% A function that closes over a borrow must not escape the borrow's region.

Recently introduced persistent data structures
permit transient mutations where
non-linear uses lead to degraded performance
\cite{DBLP:conf/ml/ConchonF07} or to
dynamic and static checks \cite{DBLP:journals/pacmpl/Puente17}.
In particular, persistent Hash-Array-Mapped-Tries (HAMT) have been used with similar
APIs in several non-pure functional languages (OCaml, Clojure, \dots).
Affine types help formalize the performance contract between the programmer
and the library, while borrows avoid the need to thread state explicitly,
as usually required by an API for immutable data types.
%

Our implementation of a backtracking Sudoku solver abstracts this scenario.
% show how \lang allows us to define
The solver maintains a two-dimensional array to represent the state of
the game and uses backtracking when there are several choices to proceed.
As choice points may be revisited several times, it seems advantageous
to select a persistent data structure for the array.
However, local changes between choice points may be implemented as
cheap in-place mutations.

\cref{sig:hybarray} contains an API @HYBARRAY@ along with an
implementation @CowArray@
that enables using mutable and immutable modifications to
the board through affine types and borrows.
The signature differs slightly from the @Array@
signature. As our application requires the @get@ function, the array
elements must be unrestricted, but
the structure itself remains linear so as to be implemented in terms of @Array@.
The in-place mutation function @set_mut@ with type
@&!('a t) * int * 'a -> unit@ works on an exclusive borrow  whereas the persistent
@set@ operation has type @&('a t) * int * 'a -> 'a t@. It
takes a shared borrow because it only reads from the
argument array, but returns a fresh,  modified structure.
The module @CowArray@, also in \cref{ex:cow}, contains a very simple
implementation of
@HYBARRAY@ that represents hybrid arrays
as regular arrays and uses copy-on-write for persistent
modifications. The function
@mapi: (int * &'a -> 'b) * &('a t) -> 'b t@
is a simple variation on @Array.map@ where the mapping function also
takes the position of the element. Recall that @Array.map@ always
creates a new array for the result.
% A more elaborate implementation could use persistent arrays
% \cite{DBLP:conf/ml/ConchonF07}.
%
In the example code, we make use of a two-dimensional version
@Matrix@ of the @CowArray@ data\-structure. The only difference is
that the API functions @get@, @set@, and @set_mut@ now take two index
parameters of type @int@ instead of one. The internal working is exactly the same.

\begin{figure}[tp]
  \centering
  \begin{subfigure}[t]{0.45\linewidth}
\begin{lstlisting}
module type HYBARRAY = sig
  include ARRAY
  val set : &('a t) * int * 'a -> 'a t
  val set_mut : &!('a t) * int * 'a -> unit
end
\end{lstlisting}
  \end{subfigure}\hfill
  \begin{subfigure}[t]{0.55\linewidth}
\begin{lstlisting}
module CowArray : HYBARRAY = struct
  include Array
  let set (a, i0, x0) =
    Array.mapi ((fun (i, x) -> if i = i0 then x0 else x), a)
  let set_mut = Array.set
end
\end{lstlisting}
  \end{subfigure}
  \vspace{-15pt}
  \caption{Signature and Implementation of hybrid arrays}
  \label{sig:hybarray}
  \label{ex:cow}

  \begin{subfigure}[t]{0.45\linewidth}
  \lstinputlisting[linerange=1-13]{code/sukodu.affe}
  \end{subfigure}\hfill
  \begin{subfigure}[t]{0.55\linewidth}
  \lstinputlisting[linerange=14-31,firstnumber=last]{code/sukodu.affe}
  \end{subfigure}
  % \lstinputlisting[linerange=1-31,multicols=2]{code/sukodu.affe}
  \vspace{-10pt}
\begin{lstlisting}
val propagate : int -> int -> &!Matrix.t -> int -> unit
val solve : int -> int -> Matrix.t -> unit
\end{lstlisting}
  \vspace{-10pt}
  \caption{Excerpt of the Sudoku solver}
  \label{ex:sudoku}
\end{figure}

% Our implementation of a Sudoku solver (\cref{ex:sudoku}) performs
% modifications that correspond to choice points using @set@,
% which makes it trivial to come back to the previous version
% of the array, while other modifications use @set_mut@, which
% cannot be reverted.

Our implementation of a Sudoku solver (\cref{ex:sudoku}) represents
the board as a 2D-matrix (\cref{line:boardtype}).
Each cell contains an integer set of type @IntSet.t@ that represents
admissible solutions so far. This type is immutable, i.e.,
@IntSet.remove@ produces a new value.
%
The main functions are @solve@ and @propagate@ with the typings shown
on top of \cref{ex:sudoku}. The types of @propagate_line@ etc are the
same as for @propagate@. From the types, we can see that @solve@ takes
ownership of the board, whereas @propagate@ only takes a mutable
borrow. Hence, the @propagate@ functions can only modify or read the board,
whereas @solve@ has full control.

The Sudoku solver @solve@ iterates over the cells and tries each possible
solution (\cref{line:try_solution}).
When a value is picked for the current cell, it creates a choice point
in @new_board@, where the current cell is updated with an immutable modification (\cref{line:immutset}), and propagate
the changes with the @propagate@ function.
The @propagate@ function uses direct mutation through an
exclusive borrow of the matrix as it need not preserve the previous
version of the board.
The implementation of @propagate@ is split into three parts
for lines, columns, and square, which are all very similar to function
@propagate_lines@ (\cref{line:propline}).

The @board@ parameter of the @propagate@ function is an exclusive
borrow, it should be handled
in an affine manner. To pass it safely to the three helper functions,
the body of @propagate@ \emph{reborrows} (i.e., it takes a borrow of a
borrow) the board three times in
\crefrange{line:reborrow1}{line:reborrow3}.
The function @propagate_line@ also contains two reborrows of the
exclusive borrow @board@, an
immutable one (\cref{line:reborrow_i}) and an exclusive one
(\cref{line:reborrow_m}). It demonstrates the facility
to take immutable borrows from exclusive ones.

The typing ensures that the mutations do not compromise the state at
the choice point, because they operate on a new state @new_board@ created for one
particular branch of the choice.
As the @set@ function only requires an unrestricted shared borrow,
the closure @try_solution@ remains unrestricted even though
it captures the borrow @&board@.
The price is that @try_solution@ cannot escape from
@&board@'s region. In this example, the inferred region corresponds to the
@begin@/@end@ scope.
Hence, @try_solution@ can be used in
the iteration in \cref{line:iter}.
% As @&g@ is unrestricted, we could even use a parallel iteration
% instead of a sequential one.
As @board@ is linear we must free it outside of the region
before returning (\cref{line:free:g}).

While presented for copy-on-write arrays, the API
can easily be adapted to other persistent data structures with
transient mutability such as Relaxed-Radix Balance Vectors (RRB) \cite{DBLP:journals/pacmpl/Puente17}
or persistent HAMTs \cite{bagwell2001ideal,clojurehamt} to provide  a
convenient programming style without compromising performance.

% \subsection{Type abstraction and linearity violations}

% While linearity allows to verify many properties, it also prevents
% the implementation of functions that are nevertheless safe. Rust addresses this
% issue through the @unsafe@ construct, which allows to locally
% ignore linearity constraints. This is essential to build basic
% functions such as splitting strings.
% %
% \lang allows to partially emulate this behavior using type abstraction
% as provided by ML modules.
% In particular, data types can be internally defined as unrestricted but exposed
% as affine.

% \TODO{Show example with arrays}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


%!TEX root = vorlage.tex

\subsection{Fully Convolutional Networks}\label{sec:ap5}

\subsubsection{Architecture}
In the following, the mean value of the image net dataset was subtracted from
each image channel (mean blue: 103.939, mean green: 116.779, mean red: 123.68).

For the following experiment, the VGG-16 architecture as introduced
in~\cite{simonyan2014very} was used. Not only was the architecture used, but
the network was also pre-trained on image net and used as described
in~\cite{Fossard2016}. The VGG-16 architecture consists of 20~layers:

\begin{enumerate}
    \item \verb+conv1_1+, \verb+conv1_2+, \verb+pool1+: Two convolutional
          layers with 64~filters of size $3 \times 3$ each, followed by max
          pooling with a stride of $s = (2, 2)$ and a pooling region of
          $p = (2, 2)$.
    \item \verb+conv2_1+, \verb+conv2_2+, \verb+pool2+: Two convolutional
          layers with 128~filters of size $3 \times 3$ each, followed by max
          pooling with a stride of $s = (2, 2)$ and a pooling region of
          $p = (2, 2)$.
    \item \verb+conv3_1+, \verb+conv3_2+, \verb+conv3_3+, \verb+pool3+: Three
          convolutional layers with 256~filters of size $3 \times 3$ each,
          followed by max pooling with a stride of $s = (2, 2)$ and a pooling
          region of $p = (2, 2)$.
    \item \verb+conv4_1+, \verb+conv4_2+, \verb+conv4_3+, \verb+pool4+: Three
          convolutional layers with 512~filters of size $3 \times 3$ each,
          followed by max pooling with a stride of $s = (2, 2)$ and a pooling
          region of $p = (2, 2)$.
    \item \verb+conv5_1+, \verb+conv5_2+, \verb+conv5_3+, \verb+pool5+: Three
          convolutional layers with 512~filters of size $3 \times 3$ each,
          followed by max pooling with a stride of $s = (2, 2)$ and a pooling
          region of $p = (2, 2)$.
    \item \verb+fc6+: A fully connected layer with 4096~nodes.
    \item \verb+fc7+: A fully connected layer with 1000~nodes.
\end{enumerate}

The network was trained on image net data. The idea is to use the general
image recognition features learned in the first layers. Thus the network is a
very sophisticated preprocessing algorithm.
% The layer \verb+fc7+ is discarded
% and the feature vector produced by \verb+fc6+ is used.

After \verb+fc7+, a convolutional layer \verb+score_fr+ with $k$ filters of the
shape $1 \times 1 \times 1000$ is added where $k$ is the number of classes.
Then an upscore layer \verb+up+ is added to scale the coarse output of
\verb+score_fr+ to the same size as the input image.

The $k=2$ upsampling filters of size $64 \times 64$ were initialized as a
bilinear upsampling. Those filters make the result so consistent. As they
mainly interpolate between neighboring pixels, they will never give
unrealistically small segments.

The FCN was trained end-to-end with the mean softmax cross entropy
between logits and labels as training objective:

\[- \frac{1}{|D|} \sum_{(\mathbf{x}, \mathbf{y}) \in D} \sum_{y_i \in \mathbf{y}} \left [ y_i \ln a_i + (1-y_i) \ln(1 - a_i)\right ]\]

where $D$ is the set of training data, $x$ is a feature vector, $y$ is a one-hot
encoded target and $a_i$ is the prediction for the $i$-th class of the classifier.

Each weight was regularized with L2 loss. This means the original objective
function $f_\text{training data}(\mathbf{w})$ which gets minimized was modified
to

\[\tilde{f}_\text{training data}(\mathbf{w}) = f_\text{training data}(\mathbf{w}) + \sum_{w \in \mathbf{w} w^2}\]
\clearpage
Not only the given training data was used, but it was also heavily augmented by
the following methods:

\begin{itemize}
    \item Horizontal flipping
    \item Vertical flipping
    \item Horizontal and vertical flipping
    \item Random brightness adjustments
    \item Random crops were taken
\end{itemize}

\subsubsection{Results}
The FCN evaluation results are shown in \cref{table:challenge-results}. Our
fully convolutional network is noticeably better than the other semantic
segmentation systems.

A training run with 5000 epochs took about 5.5 hours. Running the network on a
single image takes in average $\SI{1.3}{\second}$.

\subsubsection{Analysis of data augmentation}

Data augmentation is commonly done to train the network to become invariant to
changes in the data which should not matter. For example, in computer vision
this is an image-wide change in brightness.

Data augmentation becomes especially useful if very little data is present to
artificially increase the available training material. However, it is unclear
how much data is enough data and which data augmentation methods have which
effect on the quality of the semantic segmentation system.

We were interested in three properties: The quality of the segmentation system
at the end of \num{10000} epochs (11 hours) of training, the stability of the
quality during training and the speed of convergence.

To evaluate the quality of the training, four metrics were used and evaluated
on the test set after each 250~epochs:

\begin{itemize}
    \item Accuracy: $\frac{TP + TN}{TP + TN + FP + FN} \in [0, 1]$
    \item Precision: $\frac{TP}{TP + FP} \in [0, 1]$
    \item Recall: $\frac{TP}{TP + FP} \in [0, 1]$
    \item F1-Score: $2 \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}} \in [0, 1]$
\end{itemize}

Other metrics besides the accuracy were used as over \SI{90}{\percent} of the
data was background. Precision and recall are typical choices to solve this
problem. We thought about using the F1 score as a single number is always
simpler to compare than two numbers. However, for our experiments with FCNs on
this dataset the Person correlation coefficient of the accuracy and the F1
score is 0.96. The F1 score has lower values and a higher variance, but it
shows the same characteristics for FCNs on this datasets.

\Cref{fig:graph-accuracy-all} suggests that data augmentation has little to no
effect on the overall accuracy. Only random crops seem to have a negative
effect on accuracy.

\Cref{fig:graph-precision-all,fig:graph-recall-all} show that the precision
and recall have a lot of variance for all data augmentation methods. Random
crops seem to have a negative effect on precision and recall as well.

All training methods suffer from enormous outliers in the quality even during
late stages in the training.

We did not notice any effect of any data augmentation method on the stability
during the training.