% !TeX program = pdflatex
% !BIB program = biber

\section{Results}
\label{sec:Results}

\blindtext
With this, we can test our hypotheses.

\subsection{Test of \autoref{hy:UNBAL}}

Our first result supports \autoref{hy:UNBAL}. \blindtext The analysis we conducted to obtain \autoref{r:main} is described in detail in \autoref{tmain:focus}. Let's reference a~section, a~subsection, and a~figure from the appendices: \autoref{sec:app:example_tables}, \autoref{sec:app:salience}, \autoref{fig:concentrated_cl}.

\begin{result} \label{r:main}
	\blindtext
\end{result}

\begin{table}[t]
	\caption{An Example Table}
	\label{tmain:focus}
	\begin{tabularx}{\textwidth}[t]{L T{1.3}}
		\toprule
		Dependent variable & {$\hat{d}$} \\
		\midrule
		Estimate      &       0.123{***} \\
		&     (0.011) \\
		\midrule
		Observations & {750} \\
		%clustered by subjects & \\
		Subjects      &         {250} \\
		\bottomrule
	\end{tabularx}
	\tablenotes{%
		Standard errors in parentheses, clustered on the subject level.
		{*} $p < 0.10$, {**} $p < 0.05$, {***} $p<0.01$.
	}
\end{table}

\blindtext

\subsection{Test of \autoref{hy:UNBAL_2}}
\blindtext We thereby test \autoref{hy:UNBAL_2}.

\begin{result} \label{r:main2}
	\blindtext
\end{result}

Our second result provides evidence in support of \autoref{hy:UNBAL_2}. \blindmathfalse\blindtext\blindmathtrue

\subsection{Heterogeneity}
\blindmathpaper


\documentclass[11pt]{article}

\usepackage{tikz}
\usepackage{amsmath}

\begin{document}

\title{Applying the Hidden Markov Model to Labelling CTI Event Sequences }
\author{Chris Davey}
\date{Dec 2014}

\maketitle

\newpage

\tableofcontents

\newpage

\section{Overview}

The Markov Model provides a framework for the determination of state of the current set of observed evidence variables \cite{mai}. It models all potential states and the probability of transition into those states given the evidence observed so far. This summary illustrates the usage of the concept applied to the arrival of CTI events (computer telephony interface) in order to superimpose a higher level determination of state labels upon the low level sequence of events (the evidence variables) allowing a mapping between CTI event sequences and State labels to be defined by use of a Hidden Markov Model (HMM) as opposed to the hard coding of control flow within a programming language. This document provides a motivation for the technique and presents a summary of the Hidden Markov Model encoding, and provides an example of methods for training such a model.

\section{Motivation}

In writing procedural object oriented applications in an imperative language to deal with event streams issued from CTI systems, it has been common practice to encode the sequence of events as a series of imperative branching statements or as a deterministic state machine. These branching statements depend upon state data or object, which may be used to convey a model of the call state and are often intermixed with control statements or triggering of events which enact upon the state data, based upon hard coded assumptions about the CTI event stream. This method of writing a program encodes the interpretation of a time dependent input signal (the series of CTI events) as a series of sequential statements that describes variation in a static manner. These are the logical procedures written within the program that deal with the current state data and the action performed in relation to reacting to the current interpretation of the current CTI event.\\

When faced with variation in arrival sequence of the CTI events due to the high load placed upon telephony systems under peak load, or performance testing. It is likely that CTI events will not necessarily arrive in the order previously encoded in the procedural sequences encoded in the program. The way to deal with this variation is to add further logical conditions to allow the program to encode the different variations in the order of CTI events, this may involve collecting events in a temporary buffer and delaying them until subsequences of dependent events have arrived, before the application can update the call state data based on the interpretation of the dependent event sequences. This adds additional complexity to the program, causing maintenance and potentially leaky abstraction in relation to the programs initial design and structural integrity.\\
Such transitions between states have also been encoded as state machines, with transitions and explicit entry conditions for each state corresponding to the predetermined rules produced in the assumptions made about the sequence of CTI events. The Hidden Markov Model lends itself to the state machine representation, and is essentially a finite state machine with probabilities assigned to the transitions between states instead of hard coded entry conditions \cite{pc}.

\section{Concepts}

This document proposes the use of a Hidden Markov Model that is trained on data sets of labelled CTI event sequences in order to provide a classifier that is capable of emitting "state" labels for time sequences of CTI event streams. Instead of encoding logical branching within the program to describe the relationship between an event stream and an abstracted call state, the HMM is trained to emit state labels in response to the incoming CTI events at each stage of the call. Specifically, the abstract call state is concerned with the modelling of call recording of the calling party (not the answering party). Additionally the answering party (a call centre agent) may potentially transfer the call amongst other call centre agents. The call recording state will need to be started, paused, stopped or restarted depending on the evidence provided by the incoming CTI stream. The abstract call recording states are illustrated below (note these do not reflect any specific implementation, but serve as an illustrative example only in the context of call recording for capturing audio in order to enrol a caller within a voice collection system). \\

\usetikzlibrary{positioning}
\begin{tikzpicture}[->, xscale=6,yscale=8,>=stealth, shorten >=1pt, auto, node distance=3cm, thick, draw=blue!100]
\tikzstyle{v}=[circle,minimum size=1mm, draw, thick]
\tikzstyle{bluen}=[v,draw=blue!75,fill=blue!20]
\tikzstyle{redn}=[v,draw=red!75,fill=red!20]

\node[bluen] (a) {$Initial$};
\node[bluen] (b) [below=of a] {$OnCall$};
\node[bluen] (c) [right=of b]{$OnHold$};
\node[bluen] (d) [right=of c]{$Enrol$};
\node[bluen] (e) [below=of b]{$Ended$};
\node[bluen] (f) [below=of d]{$Consulted$};

\path[every node/.style={font=\sffamily\small}]
(a) edge node [left] {} (b)
    edge [bend right] node {} (e)
    (b) edge node [left] {} (c)
        edge node[left] {} (e)
        edge [bend left] node {} (d)
        edge [left] node {} (f)
    (c) edge node[left] {} (e)
        edge node[left] {} (f)
    (d) edge node[left] {} (e)
    (f) edge node[left] {} (e);

\end{tikzpicture}

The above call recording states are for illustrative purposes, but demonstrate an example state space without the evidence variables for CTI events, these variables represent the non-observable state variables (labels). The additional evidence variables (combining CTI events and their associated data) are the observable variables that are time dependent and the assocations that define the resulting evidence/state space must be learnt from collected data. For example one such sequence may be.

\usetikzlibrary{positioning}
\begin{tikzpicture}[->, xscale=6,yscale=8,>=stealth, shorten >=1pt, auto, node distance=3cm, thick, draw=blue!100]
\tikzstyle{v}=[circle,minimum size=1mm, draw, thick]
\tikzstyle{bluen}=[v,draw=blue!75,fill=blue!20]
\tikzstyle{redn}=[v,draw=red!75,fill=red!20]
\tikzstyle{v}=[circle,minimum size=1mm, draw, thick]

\node[redn] (x1) {$Ringing_{t=1}$};
\node[redn] (x2) [below=of x1] {$Established_{t=2}$};
\node[redn] (x3) [below=of x2] {$UserEvent=Start_{t=3}$};
\node[bluen] (b1) [right=of x3] {$OnCall$};
\node[redn] (x4) [below=of x3] {$t_4...t_{n-k}$};

\path[every node/.style={font=\sffamily\small}]
    (x1) edge node [left] {} (x2)
    (x2) edge node [left] {} (x3)
    (x3) edge node[left] {$emits$} (b1)
    edge node[left] {} (x4);
\end{tikzpicture}

The diagram above illustrates that at each time $t$ an evidence variable (shown in red) is observed, and at points in the sequence a state variable is emitted (shown in blue) from the model (note those points where a state is not emitted we can call a "Null" state). The state variable identifies the current state of the call given the stream of sequences that have occured so far. The transitions between sequences represent two discrete conditional probability distributions The first relating each state variable $X$ to each of the other state variables $x$ and the second relating each state variable $X$ each evidence variable $e$ given the stream of previous events in time $1:t$, the next predicted state at time $t+k+1$ and prior states in time $t+k$ \cite{mai}.
$$
P(X_{t+k+1}|e_{1:t}) = \sum_{ x_{t+k} } P(X_{t+k+1} | x_{t+k}) P(x_{t+k}|e_{1:t})
$$
Hence the model in the discrete case consists of the state transition probability distribution. From our example the states consist of the set: $$\{Null, Initial,OnCall,OnHold, Enrol, Ended, Consulted\}$$
Note the addition of the $Null$ state which represents a state that has no action associated with it, this state indicates that the model is waiting for further evidence before it can determine the next appropriate state (to allow the model to emit a state at every incoming CTI event). The state transition table would have the following structure.\\
\\
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline
$t+1 \sim t$ & Null & Initial & OnCall & OnHold & Enrol & Ended & Consulted \\
\hline
Null & & & & & & & \\
\hline
Initial & & & & & & & \\
\hline
OnCall & & & & & & & \\
\hline
OnHold & & & & & & & \\
\hline
Enrol & & & & & & & \\
\hline
Ended & & & & & & & \\
\hline
Consulted & & & & & & & \\
\hline
\end{tabular}
\\
\\
And is essentially a tabular representation (an $M \times M$ matrix) of the graphical form presented earlier from this table we can determine the prior conditional distribution of $P(X_{t+1} | x_{t})$. Duda et al, refer to the transitions in this table as $a_{ij}$ \cite{pc}, the matrix in Bishop is termed $A$ \cite{mlp} so we have the matrix $A$.

\begin{equation*}
    \mathbf{A} = \left[
    \begin{array}{cccc}
    a_{11} & a_{12} & \ldots & a_{1j} \\
    \vdots & \vdots & \vdots & \vdots \\
    a_{i1} & a_{i2} & \ldots & a_{ij}

    \end{array}
    \right]
\end{equation*}

The prediction process described briefly above will update the priors given the time $t$, the task of learning the state transition table will be to determine the priors from the training data.\\
The second set of priors is the conditional distribution of the  evidence variable at time $t=1$ and the state variable. For the purpose of the example, the set of evidence variables are listed below, however in actual fact the evidence variables will be a much larger set, note that the evidence variables are a combination of event name, and some type attributes contained within the event, a larger model will also include the "party" that originates the event in terms of "party A", "party B", "party C" for instance. For the purpose of illustration let the CTI evidence variables be the following:
\\
\begin{itemize}
\item Ringing(inbound)
\item Dialing(consult)
\item Establised()
\item AttachedDataChanged()
\item UserEvent(Started)
\item UserEvent(Stopped)
\item OffHook()
\item Held()
\item Retrieve()
\end{itemize}

Note that the set of evidence variables above are a simplified example, in practice, additional details such as parties may be encoded in the evidence variables.\\
The resulting joint distribution would have the following structure.
\\\\
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline
$t=1$ & Null & Initial & OnCall & OnHold & Enrol & Ended & Consulted \\
\hline
Ringing(inbound) & & & & & & & \\
\hline
Dialing(consult) & & & & & & & \\
\hline
Establised() & & & & & & & \\
\hline
AttachedDataChanged() & & & & & & & \\
\hline
UserEvent(Started) & & & & & & & \\
\hline
UserEvent(Stopped) & & & & & & & \\
\hline
OffHook() & & & & & & & \\
\hline
Held() & & & & & & & \\
\hline
Retrieve() & & & & & & & \\
\hline
\end{tabular}
\\\\
The format is an $N \times M$ matrix indicating the conditional distribution $P(x_{t=1} | e_{t=1})$ denoted as a matrix $B$ where $B_{ij} = P(x_i | e_j )$ at time $t$ the state $x_t$ will be emitted given the evidence (or visible) variable $e_j$. \cite{pc}
\begin{equation*}
    \mathbf{B} = \left[
    \begin{array}{cccc}
    b_{11} & b_{12} & \ldots & b_{1j} \\
    \vdots & \vdots & \vdots & \vdots \\
    b_{i1} & b_{i2} & \ldots & b_{ij}

    \end{array}
    \right]
\end{equation*}

The task of the learning algorithm will be to learn these distributions over $t$. During the task of prediction, each table represents a prior distribution at time $t$, and is updated after evaluating the priors.

\section{Learning}

The goal of learning is to produce a hidden markov model containing the following components \cite{rab}.
\begin{itemize}

\item $N$ the total number of distinct states.

\item $M$ the total number of distinct evidence variables.

\item $\pi$ the prior state distribution of $P(x_i)$ at $t = 0$.

\item $A$ the transition matrix where $a_{ij} = P(x_i|x_j)$ with $i = t, j=t+1$ and $1 \le i,j \le N$.

\item $B$ the state evidence transition matrix where $b_{ij} = P(x_j|e_i)$ with $i = j = t$ and $1 \le i \le M, 1 \le j \le N$

\end{itemize}

The HMM is represented as the tuple below \cite{rab}.
$$
HMM = \lambda = (A, B, \pi)
$$

The matrices $A$, $B$ and the vector $\pi$ are constructed from the initial training data set. Then the learning algorithm is applied until convergence in order to build the updated model.

The initial step in learning are the state transition distribution and evidence and state distributions as described above, however as $t$ increases the probability distribution becomes a distribution of $P(x_{t+k}|e_{1:t+k})$ for each of the evidence variables the prior needs to be computed in terms of the  ${1:t+k}$ sequences. \\
The method of learning is performed through the ``forward backward algorithm'' which splits the learning process into two parts. Firstly the backward variable represents the probability that the model is in state $x_i(t)$ at time $t$ and will generate the remainder of the given target sequence from $t+1 \rightarrow T$ \cite{pc}. It can be described as \cite{rab}:
$$
\beta_i(t) = P(e_{t+1}, e_{t+2},...,e_T|x_i,\lambda)
$$
The conditionality includes the $\lambda$ model generated so far. $\beta$ is initialised as $\beta_T(i) = 1$ and is calculated as \cite{pc}
\begin{equation*}
\beta_i(t) = \left\{
    \begin{array}{rl}
    0 &\text{if }x_i \neq x_0 \text{ and } t = T,\\
    1 &\text{if }x_i = x_0 \text{ and } t = T,\\
    \sum_j^N a_{ij}b_j(e_{t+1})\beta_j(t+1) & \text{otherwise}.
    \end{array}
\right.
\end{equation*}

Given the recursive nature of the equation, dynamic programming is used to generate and cache results. The algorithm is evaluated tail recursively. The usage of $b_j(e_{t+1})$ represents the value from $B$ for $b_{ij}$ where $i$ is the index of the $e_{t+1}$ evidence variable in the current model $\lambda$.\\
The second part of the algorithm is defined by the forward variable, which represents the probability that the model is in state $x_i$ given the sequence of evidence variables observed so far $e_{1:t}$ \cite{pc}.
$$
\alpha_j(t) = P(e_1,e_2,...,e_t, x_i | \lambda) \text{ \cite{rab}}
$$
The $\alpha$ equation defined as follows \cite{pc}.
\begin{equation*}
\alpha_j(t) = \left\{
    \begin{array}{rl}
    0 & t = 0 \text{ and } x_j \neq x_0,\\
    1 & t = 0 \text{ and } x_j = x_0, \\
    \left[ \sum_i^N \alpha_i(t-1)a_{ij}\right]b_j(e_t) & \text{otherwise}
    \end{array}
\right.
\end{equation*}
The technique will also require a dynamic programming table to cache the recursive calls when computing $\alpha_i$. The term $b_j(e_t)$ represents the value from $B$ for $b_{ij}$ where $i$ is the index of the $e_t$ evidence variable from the current model $\lambda$.\\
The process perform an update for the model $A$ iteractly by estimating the probability of transitioning from state $x_i$ to state $x_j$ at time $t$ and $t+1$ given the model and observations $V^{t+1} = e_1,e_2, e_3,...,e_{t+1}$ \cite{rab} the matrix $\Xi$ is used to contain the new estimates for the transitions between time $t$ and $t+1$.
\begin{equation*}
 \mathbf{\Xi} = \left[
    \begin{array}{cccc}
    \xi_{11} & \xi_{12} & \ldots & \xi_{1j} \\
    \vdots & \vdots & \vdots & \vdots \\
    \xi_{i1} & \xi_{i2} & \ldots & \xi_{ij}
    \end{array}
    \right]
\end{equation*}
The equation for $\xi_{ij}$ is given as follows \cite{rab}.
$$
\xi_{ij} = P(x_i(t), x_j(t+1) | V^{t+1}, \lambda)
$$\\
$$
= \frac{\alpha_i(t)b_j(e_{t+1})\beta_j(t+1)}{P(V^{t+1}|\lambda)}
$$\\
$$
= \frac{\alpha_i(t)b_j(e_{t+1})\beta_j(t+1)}{\sum_{i=1}^N \sum_{j=1}^N \alpha_i(t) a_{ij}b_j(e_{t+1})\beta_j(t+1)}
$$
The expected number of times that state $x_i$ transitions to state $x_j$ is determined by summing over the values of $\xi_{ij}$ \cite{pc} \cite{rab}.
$$
\gamma_i(t) = \sum_{j=1}^T \xi_t(i,j)
$$
And the total number of expected times a state $x_i$ is visited is given by summing over $\gamma_i$ for all time \cite{pc} \cite{rab}.
$$
\sum_{t=1}^T \gamma_i(t)
$$
The matrix $A$ can be updated with new estimates for the state transitions as follows \cite{pc}.
$$
\hat{A} = \hat{a}_{ij} = \frac{\gamma_i(t) }{\sum_{t=1}^T \gamma_i(t)}
$$
And the matrix $B$ can be updated with new estimates for the evidence variables and states by calculating the ratio between the frequency that a particular evidence variable $e_k$ is emitted and any evidence variable is emitted \cite{pc}.
$$
\hat{B} = \hat{b}_{ij} = \frac{\sum_{t=1, e_k}^T \gamma_k(t)}{\sum_{t=1}^T \gamma_i(t)}
$$

The learning process repeats until convergence, that is the changes in previous values of $a_{ij}$ and $b_{ij}$ decreases below a threshold $\theta$ \cite{pc}.

\section{Prediction}

The product of the two conditional distributions at time $t+k$ predicts the state at $t+k+1$ as stated earlier using the following sum product.
$$
P(X_{t+k+1}|e_{1:t}) = \sum_{ x_{t+k} } P(X_{t+k+1} | x_{t+k}) P(x_{t+k}|e_{1:t})
$$
Similar to the learning process, prediction will iteratively generate estimates for both $\alpha$ and $\beta$ and use these to successively update the estimates for the model $A$. A new matrix
\begin{equation*}
 \mathbf{\Gamma} = \left[
    \begin{array}{cccc}
    \gamma_{11} & \gamma_{12} & \ldots & \gamma_{1j} \\
    \vdots & \vdots & \vdots & \vdots \\
    \gamma_{i1} & \gamma_{i2} & \ldots & \gamma_{ij}
    \end{array}
    \right]
\end{equation*}
Is formed by using the estimates to determine the new probability of being in state $x_i$ at time $t$ given the sequence of evidence variable observations $V^t = e_1,e_2,...,e_t$ and the current model $\lambda$ \cite{rab}.
$$
\gamma_i(t) = P(x_t | V^t, \lambda)
$$
The equation to generate $\gamma_i$ is given in \cite{rab} and \cite{pc} as follows.
$$
\gamma_i(t) = \frac{\alpha_i(t)\beta_j(t)}{P(x_t | V^t, \lambda)} = \frac{\alpha_i(t)\beta_j(t)}{\sum_{i=1}^N \alpha_j(t)\beta_i(t)}
$$

\section{Evaluation}

TODO: talk about model evaluation using test data and measurement of performance.

\section{Related Methods}

TODO: talk about relation to the finite state machine, reinforcement learning as a potential means of dynamically learning the policy (it shares a similar structure as the $\alpha$ and $\beta$  but performs updates in place over time), there are also other models (eg RNN) which can be applied here.

\section{Example Implementation}

TODO: implementation

\newpage

\begin{thebibliography}{99}

\bibitem{mai} Norvig, Peter. Russell, Stuart. (2003) $Artificial\ Intelligence\ A\ Modern\ Approach.\ Second\ Edition.$ USA: Pearson Education, Inc. pp537-583, 724-733.
\bibitem{dm} Han, Jiawei. Kamber, Micheline. (2006) $Data\ Mining:\ Concepts\ and\ Techniques.\ Second\ Edition.$ USA: Morgan Kaufman Publishers. pp518-526.
\bibitem{mlp} Bishop, Christopher M. (2006) $Pattern\ Recognition\ and\ Machine\ Learning$ USA: Springer Science + Business Media, LLC. pp607-635.

\bibitem{pc} Duda, Richard O. Hart, Peter E. Stork, David G. (2001) $Pattern\ Classification.\ Second\ Edition.$ Canada: Wiley and Sons Inc. pp128-139.

\bibitem{rab} Rabiner, Lawrence R. (1989) ``A Tutorial on Hidden Markov Models and Seleted Applications in Speech Recognition''. USA: Proceedings of the IEEE, VOL 77, NO.2 pp257-286

\end{thebibliography}

\end{document}


\section{Overview}

This section starts with a discussion on DK's declarative type system.
Then it introduces several techniques that have been used in algorithmic
formulations, and which have influenced our own algorithmic design.
Finally we introduce the novelties of our new algorithm.
In particular the support for inference judgments in
worklist judgments, and a new form of worklist judgment
that unifies \emph{ordered contexts} and the worklists themselves.

% DK's declarative system TODO point to background

\subsection{DK's Declarative System}

\begin{figure}[t]
	\[
	\begin{array}{l@{\qquad}lcl}
	\text{Type variables}\qquad&a, b
	\\
	\text{Types}\qquad&A, B, C &::=&\quad 1 \mid a \mid \all A \mid A\to B\\
	\text{Monotypes}\qquad&\tau,\sigma &::=&\quad 1 \mid a \mid \tau\to \sigma
	\\
	\text{Expressions}\qquad&e &::=&\quad x \mid () \mid \lam e \mid e_1~e_2 \mid (e:A)
	\\
	\text{Contexts}\qquad&\Psi &::=&\quad \nil \mid \Psi, a \mid \Psi, x:A
	\end{array}
	\]
	\Description{Syntax of Declarative System}
	\caption{Syntax of Declarative System (Extends Figure~\ref{fig:ITP:decl:syntax})}
\end{figure}

\begin{figure}[t]
\centering \framebox{$\Psi \vdash A$} Well-formed declarative type
\begin{gather*}
\inferrule*[right=$\mathtt{wf_d unit}$]
    {~}{\Psi\vdash 1}
\qquad
\inferrule*[right=$\mathtt{wf_d var}$]
    {a\in\Psi}{\Psi\vdash a}
\qquad
\inferrule*[right=$\mathtt{wf_d{\to}}$]
    {\Psi\vdash A\quad \Psi\vdash B}
    {\Psi\vdash A\to B}
\qquad
\inferrule*[right=$\mathtt{wf_d\forall}$]
    {\Psi, a\vdash A}
    {\Psi\vdash \forall a. A}
\end{gather*}

\centering \framebox{$\Psi \vdash e$} Well-formed declarative expression
\begin{gather*}
\inferrule*[right=$\mathtt{wf_d tmvar}$]
    {x:A\in\Psi}{\Psi\vdash x}
\qquad
\inferrule*[right=$\mathtt{wf_d tmunit}$]
    {~}{\Psi\vdash ()}
\qquad
\inferrule*[right=$\mathtt{wf_d abs}$]
    {\Psi,x:A\vdash e}
    {\Psi\vdash \lam e}
\\
\inferrule*[right=$\mathtt{wf_d app}$]
    {\Psi\vdash e_1 \quad \Psi\vdash e_2}
    {\Psi\vdash e_1~e_2}
\qquad
\inferrule*[right=$\mathtt{wf_d anno}$]
    {\Psi\vdash A \quad \Psi\vdash e}
    {\Psi\vdash (e:A)}
\end{gather*}

\centering \framebox{$\Psi \vdash A \le B$} Declarative subtyping
\begin{gather*}
\inferrule*[right=$\mathtt{{\le}Var}$]
    {a\in\Psi}{\Psi\vdash a\le a}
\qquad
\inferrule*[right=$\mathtt{{\le}Unit}$]
    {~}{\Psi \vdash 1 \le 1}
\qquad
\inferrule*[right=$\mathtt{{\le}{\to}}$]
    {\Psi \vdash B_1 \le A_1 \quad \Psi \vdash A_2 \le B_2}
    {\Psi\vdash A_1\to A_2 \le B_1\to B_2}
\\
\inferrule*[right=$\mathtt{{\le}\forall L}$]
    {\Psi\vdash \tau \quad \Psi\vdash [\tau/a] A \le B}
    {\Psi\vdash \all A \le B}
\qquad
\inferrule*[right=$\mathtt{{\le}\forall R}$]
    {\Psi, b\vdash A\le B}
    {\Psi\vdash A \le \all[b]B}
\end{gather*}
\Description{Declarative Well-formedness and Subtyping}
\caption{%Well-formedness of Declarative Types and
Declarative Well-formedness and Subtyping}
\end{figure}

\begin{figure}[t]
\begin{tabular}{rl}
    \framebox{$\Psi \vdash e \Lto A$} & $e$ checks against input type $A$.\\[0.5mm]
    \framebox{$\Psi \vdash e \To A$} & $e$ synthesizes output type $A$.\\[0.5mm]
    \framebox{$\Psi \vdash \appInf{A}{e}{C}$} & Applying a function of type $A$ to $e$ synthesizes type $C$.
\end{tabular}
\begin{gather*}
\inferrule*[right=$\mathtt{DeclVar}$]
    {(x:A)\in\Psi}{\Psi\vdash x\To A}
\qquad
\inferrule*[right=$\mathtt{DeclSub}$]
%e \neq \lam e' \quad B \neq \all B' \quad
    {\Psi\vdash e\To A \quad \Psi\vdash A\le B}
    {\Psi \vdash e\Lto B}
\\
\inferrule*[right=$\mathtt{DeclAnno}$]
    {\Psi \vdash A \quad \Psi\vdash e\Lto A}
    {\Psi\vdash (e:A)\To A}
\qquad
\inferrule*[right=$\mathtt{Decl1I}$]
    {~}{\Psi\vdash () \Lto 1}
\qquad
\inferrule*[right=$\mathtt{Decl1I{\To}}$]
    {~}{\Psi\vdash () \To 1}
\\
\inferrule*[right=$\mathtt{Decl\forall I}$]
    {\Psi,a \vdash e \Lto A}
    {\Psi\vdash e\Lto \all A}
\qquad
\inferrule*[right=$\mathtt{Decl\forall App}$]
    {\Psi \vdash \tau \quad \Psi\vdash \appInf{[\tau/a]A}{e}{C} }
    {\Psi\vdash \appInf{\all A}{e}{C}}
\\
\inferrule*[right=$\mathtt{Decl{\to}I}$]
    {\Psi,x:A \vdash e\Lto B}
    {\Psi\vdash \lam e \Lto A \to B}
\qquad
\inferrule*[right=$\mathtt{Decl{\to}I{\To}}$]
    {\Psi\vdash \sigma\to\tau \quad \Psi,x:\sigma \vdash e\Lto \tau}
    {\Psi\vdash \lam e \To \sigma\to\tau}
\\
\inferrule*[right=$\mathtt{Decl{\to} E}$]
    {\Psi\vdash e_1\To A \quad \Psi\vdash \appInf{A}{e_2}{C}}
    {\Psi\vdash e_1~e_2 \To C}
\qquad
\inferrule*[right=$\mathtt{Decl{\to}App}$]
    {\Psi\vdash e \Lto A}
    {\Psi\vdash \appInf{A \to C}{e}{C}}
\end{gather*}
\Description{Declarative Typing}
\caption{Declarative Typing}
\end{figure}

Subsection~\ref{subsec:dk:decl} introduces DK's
declarative subtyping and typing systems.
We also duplicate the rules here for the convenience of the reader.

\paragraph{Overlapping Rules}
A problem that we found in the declarative system is that
some of the rules overlap with each other.
Declarative subtyping rules $\mathtt{{\le}\forall L}$ and $\mathtt{{\le}\forall R}$
both match the conclusion $\Psi\vdash \all A \le \all B$.
In such a case, choosing $\mathtt{{\le}\forall R}$ first is always better,
since we introduce the type variable $a$ to the context earlier,
which gives more flexibility on the choice of $\tau$.
The declarative typing rule $\mathtt{DeclSub}$ overlaps with
both $\mathtt{Decl\forall I}$ and $\mathtt{Decl{\to}I}$.
However, we argue that more specific rules are always the best choices,
i.e. $\mathtt{Decl\forall I}$ and $\mathtt{Decl{\to}I}$ should have
higher priority than $\mathtt{DeclSub}$.

For example, $\Psi\vdash \lam x \Lto \all a\to a$ succeeds if derived from
Rule $\mathtt{Decl\forall I}$:
$$
\inferrule*[right={$\mathtt{Decl\forall I}$}]
	{\inferrule*[Right={$\mathtt{Decl{\to}I}$}]
		{\Psi,a,x:a \vdash x \Lto a}
		{\Psi,a \vdash \lam x \Lto a \to a}
	}
	{\Psi\vdash \lam x \Lto \all a \to a},
$$
but fails when applied to $\mathtt{DeclSub}$:
$$
\inferrule*[right={$\mathtt{DeclSub}$}]
	{
		\inferrule*[right=$\mathtt{Decl{\to}I{\To}}$]
			{\Psi \vdash \blue\sigma\to \blue\tau \quad \Psi,x:\blue\sigma\vdash e \Lto \blue\tau}
			{\Psi \vdash \lam x \To \blue\sigma\to \blue\tau}\quad
		\inferrule*[Right=$\mathtt{{\le}\forall R}$]
			{\inferrule*[Right=$\mathtt{{\le}{\to}}$]
				{
					\inferrule*[Right=$?$]
						{\textit{Impossible!} \\\\ a \notin \text{FV}(\blue\sigma)}
						{\Psi,a \vdash a \le \blue\sigma}
					\quad \Psi,a \vdash \blue\tau \le a
				}
				{\Psi,a \vdash \blue\sigma\to \blue\tau \le a \to a}
			}
			{\Psi\vdash \blue\sigma\to \blue\tau\le \all a \to a}
	}
{\Psi\vdash \lam x \Lto \all a \to a}.
$$

Rule $\mathtt{Decl{\to}I}$ is also better at handling higher-order types.
When the lambda-expression to be inferred has a polymorphic input type,
such as $\all a \to a$,
$\mathtt{DeclSub}$ may not derive some judgments.
For example, $\Psi,id:\all a\to a \vdash \lam[f] f~id~(f~()) \Lto (\all a\to a) \to 1$
requires the argument of the lambda-expression to be a polymorphic type,
otherwise it could not be applied to both $id$ and $()$.
If Rule $\mathtt{DeclSub}$ was chosen for derivation,
the type of its argument is restricted by Rule $\mathtt{Decl{\to}I{\To}}$,
which is not a polymorphic type.
By contrast,
Rule $\mathtt{Decl{\to}I}$ keeps the polymorphic argument type $\all a\to a$,
and will successfully derive the judgment.

We will come back to this
topic in Section~\ref{sec:metatheory:non-overlapping}
and formally derive a system without overlapping rules.

%-------------------------------------------------------------------------------
\subsection{DK's Algorithm}\label{ssec:DK_Algorithm}

DK's algorithm version revolves around their notion of \emph{algorithmic context}.
\[
\begin{array}{l@{\qquad}lcl}
\text{Algorithmic Contexts}\qquad&\Gamma,\Delta,\Theta &::=&\quad \nil \mid
\Gamma, a \mid \Gamma, x:A \mid \Gamma, \al \mid \Gamma, \al = \tau \mid
\Gamma, \blacktriangleright_{\al}
\end{array}
\]
In addition to the regular (universally quantified) type variables $a$, the
algorithmic context also contains \emph{existential} type variables
$\al$. These are placeholders for monotypes $\tau$ that are still to
be determined by the inference algorithm. When the existential variable is
``solved'', its entry in the context is replaced by the assignment
$\al = \tau$.
A context application on a type, denoted by $[\Gamma]A$,
substitutes all solved existential type variables in $\Gamma$
with their solutions on type $A$.

All algorithmic judgments thread an algorithmic context. They have the form
$\Gamma \vdash \ldots \dashv \Delta$, where $\Gamma$ is the input context and
$\Delta$ is the output context:
$\Gamma \vdash A \le B \dashv \Delta$  for subtyping,
$\Gamma \vdash e \Leftarrow A \dashv \Delta$  for type checking, and so on.
The output context is a functional update of the input context that records newly
introduced existentials and solutions.

Solutions are incrementally propagated by applying the algorithmic output
context of a previous task as substitutions to the next task. This can be seen
in the subsumption rule:
\[
\inferrule*[right=$\mathtt{DK\_Sub}$]
  {\Gamma \vdash e \Rightarrow A \dashv \Theta \\
   \Theta \vdash [\Theta]A \le [\Theta]B \dashv \Delta
  }
  { \Gamma \vdash e \Leftarrow B \dashv \Delta}
\]
The inference task yields an output context $\Theta$ which is applied as a substitution
to the types $A$ and $B$ before performing the subtyping check to propagate any solutions
of existential variables that appear in $A$ and $B$.

\paragraph{Markers for scoping.}
The sequential order of entries in the algorithmic context, in combination with
the threading of contexts,  does not perfectly capture the scoping of all
existential variables. For this reason the DK algorithm uses scope markers
$\blacktriangleright_{\al}$ in a few places. An example is given in the following
rule:
\[
\inferrule*[right=$\mathtt{DK\_{\le}\forall L}$]
  {\Gamma,\blacktriangleright_{\al}, \al \vdash [\al/a]A \le B \dashv \Delta,\blacktriangleright_{\al},\Theta}
  {\Gamma \vdash \all A \le B \dashv \Delta}
\]
To indicate that the scope of $\al$ is local to the subtyping check
$[\al/a]A \le B$, the marker is pushed onto its input stack and popped
from the output stack together with the subsequent part $\Theta$, which may
refer to $\al$.
(Remember that later entries may refer to earlier
ones, but not vice versa.) This way $\al$ does not escape its scope.
\begin{comment}
A type variable may have a similar functionality to the scoping markers.
An example rule that checks an expression against a polymorphic type is as follows:
$$
\inferrule*
{\Gm, a \vdash e \Lto A \dashv \Delta, a, \Theta}
{\Gm \vdash e \Lto \all A \dashv \Delta}
$$
\end{comment}

One may suggest that the marker $\blacktriangleright_{\al}$ is somewhat redundant,
since $\al$ already declares the scope.
However, in the following rule,
\[
\inferrule*[right=$\mathtt{DK\_InstLArr}$]
  {\Gamma[\al[2], \al[1], \al = \al[1] \to \al[2]] \vdash A_1 \le \al[1] \dashv \Theta \\
    \Theta \vdash \al[2] \le [\Theta]A_2 \dashv \Delta
  }
  {\Gamma[\al] \vdash \al \le A_1 \to A_2 \dashv \Delta}
\]
the algorithm introduces new existential variables right before $\al$.
In such case, the marker $\blacktriangleright_{\al}$ still appears
to the left of them.
Without the marker, it will be difficult to recycle the new existential variables
$\al[1]$ and $\al[2]$ properly, which should have the same scope of $\al$
and thus should be recycled together with $\al$.

%Due to the complication of its scoping control method,
%it is hard to argue the preciseness of scoping for each variable when designing algorithms.

At first sight, the DK algorithm would seem to be a good basis for mechanization. After all,
 it comes with a careful description and extensive manual proofs.
Unfortunately, we ran into several obstacles that have prompted us to formulate
a different, more mechanization-friendly algorithm.

%- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
\paragraph{Broken Metatheory} % Talk about the lemmas that are false, show counterexample.
While going through the manual proofs of DK's algorithm, we found several
problems.  Indeed, two proofs of lemmas---Lemma 19 (Extension Equality
Preservation) and Lemma 14 (Subsumption)--- wrongly apply induction hypotheses
in several cases. Fortunately, we have found simple workarounds that fix these
proofs without affecting the appeals to these lemmas.

% The proof of Lemma 19 (Extension Equality Preservation) applies a wrong
% induction hypothesis for the ${\longrightarrow}\mathtt{Uvar}$ case.
% Fortunately the lemma could be proven without the well-formedness conditions
% $\Gm\vdash A$ and $\Gm\vdash B$.
% Similar problem happens for Lemma 14 (Subsumption), where many applications of
% induction hypotheses are not correct.
% For manual proofs, induction hypotheses are not automatically generated by programs,
% thus are easily misused.

More seriously, we have also found a lemma that simply does not hold:
Lemma 29 (Parallel Admissibility)\footnote{Ningning Xie found the issue with Lemma 29 in 2016 on an earlier attempt to mechanically
  formalize DK's algorithm. The authors acknowledged the problem after we contacted them through email.
  Although they briefly mentioned that it should be possible to use a weaker lemma instead they did
  not go into details.}.
%
This lemma is used to relate the algorithmic system and declarative system
before and after the instantiation procedure.
We believe that the general idea of the lemma is correct,
but the statement may fail when the sensitive ordering of variables
breaks the ``parallelism'' in some corner cases.
%
This lemma is a cornerstone of the two metatheoretical results
of the algorithm, soundness, and completeness with respect to the declarative system.
In particular, both instantiation soundness (i.e. a part of subtyping
soundness) and typing completeness directly require the broken lemma.
Moreover, Lemma 54 (Typing Extension) also requires the broken lemma and is
itself used 13 times in the proof of typing soundness and completeness.
Unfortunately, we have not yet found a way to fix this problem.

In what follows, we briefly discuss the problem through counterexamples.
False lemmas are found in the manual proofs of DK' two papers
\citep{dunfield2013complete} and \citep{DunfieldIndexed}.
\begin{itemize}
    \item
        In the first paper, Lemma 29 on page 9 of its appendix says:
        \begin{lemma}[Parallel Admissibility of~\citep{dunfield2013complete}]~\\
        If $\Gamma_L \longrightarrow \Delta_L$ and
        $\Gamma_L, \Gamma_R \longrightarrow \Delta_L, \Delta_R$ then:
        \begin{enumerate}
            \item $\Gamma_L,\al,\Gamma_R \longrightarrow \Delta_L, \al, \Delta_R$
            \item If $\Delta_L \vdash \tau'$ then
                $\Gamma_L,\al,\Gamma_R \longrightarrow \Delta_L, \al = \tau', \Delta_R$.
            \item If $\Gamma_L \vdash \tau$ and $\Delta_L \vdash \tau'$ and
                $[\Delta_L]\tau = [\Delta_L]\tau'$, then
                $\Gamma_L, \al = \tau, \Gamma_R \longrightarrow \Delta_L, \al = \tau', \Delta_R$.
        \end{enumerate}
        \end{lemma}

        We give a counter-example to this lemma:\\
        Pick $\Gm_L = \nil, \Gm_R = \bt, \Delta_L = \bt, \Delta_R = \nil$, then both conditions
        $\nil\longrightarrow \bt$ and $\bt\longrightarrow \bt$ hold, but the first conclusion
        $\al,\bt \longrightarrow \bt,\al$ does not hold.
    \item
        In the second paper, as an extended work to the first paper, Lemma 26 on page 22 of its appendix says:
        \begin{lemma}[Parallel Admissibility of~\citep{DunfieldIndexed}]~\\
        If $\Gamma_L \longrightarrow \Delta_L$ and
        $\Gamma_L, \Gamma_R \longrightarrow \Delta_L, \Delta_R$ then:
        \begin{enumerate}
            \item $\Gamma_L,\al:\kappa,\Gamma_R \longrightarrow \Delta_L, \al:\kappa, \Delta_R$
            \item If $\Delta_L \vdash \tau' : \kappa$ then
                $\Gamma_L,\al:\kappa,\Gamma_R \longrightarrow \Delta_L, \al:\kappa = \tau', \Delta_R$.
            \item If $\Gamma_L \vdash \tau : \kappa$ and $\Delta_L \vdash \tau' types$ and
                $[\Delta_L]\tau = [\Delta_L]\tau'$, then
                $\Gamma_L, \al:\kappa = \tau, \Gamma_R \longrightarrow \Delta_L, \al:\kappa = \tau', \Delta_R$.
        \end{enumerate}
        \end{lemma}

        A similar counter-example is given:\\
        Pick $\Gm_L = \nil, \Gm_R = \bt:\star, \Delta_L = \bt:\star, \Delta_R = \nil$, then both conditions
        $\nil\longrightarrow \bt:\star$ and $\bt:\star\longrightarrow \bt:\star$ hold, but the first conclusion
        $\al:\kappa,\bt:\star \longrightarrow \bt:\star,\al:\kappa$ does not hold.
\end{itemize}

%- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
\paragraph{Complex Scoping and Propagation}

Besides the problematic lemmas in DK's metatheory, there are other reasons to
look for an alternative algorithmic formulation of the type system that is more
amenable to mechanization. Indeed, two aspects that are particularly
challenging to mechanize are the scoping of universal and existential type
variables, and the propagation of the instantiation of existential type
variables across judgments.
DK is already quite disciplined on these accounts, in particular compared to
traditional constraint-based type-inference algorithms like Algorithm $\mathcal{W}$~\citep{milner1978theory} which
features an implicit global scope for all type variables. Indeed, DK uses its
explicit and ordered context $\Gamma$ that tracks the relative scope of universal and
existential variables and it is careful to only instantiate existential
variables in a well-scoped manner.

Moreover, DK's algorithm carefully propagates instantiations by recording them
into the context and threading this context through all judgments.
% two This means that every judgments takes two contexts---an input and an output
% context---rather than the conventional single context. The output context
% records any new variable instantations; to propagate these instantiations to
% the remaining judgments, their predecessor's output context---which is their
% input context---is applied to them as a substitution.
While this works well on paper, this approach is still fairly involved and thus
hard to reason about in a mechanized setting. Indeed, the instantiations have
to be recorded in the context and are applied incrementally to each remaining
judgment in turn, rather than immediately to all remaining judgments at once.
Also, as we have mentioned above, the stack discipline of the ordered contexts
does not mesh well with the use of output contexts; explicit marker entries are
needed in two places to demarcate the end of an existential variable's scope.
Actually, we found a scoping issue related to the subsumption rule $\mathtt{DK\_Sub}$,
which might cause existential variables to leak across judgments.
In Section~\ref{sec:discussion:scoping} we give a detailed discussion.
%A detailed discussion on this issue with a possible fix is .

The complications of scoping and propagation are compelling reasons
to look for another algorithm that is easier to
reason about in a mechanized setting.

%-------------------------------------------------------------------------------
\subsection{Judgment Lists}\label{sec:overview:list}
To avoid the problem of incrementally applying a substitution to remaining
tasks, we can find inspiration in the formulation of constraint solving
algorithms. For instance, the well-known unification
algorithm by \citet{unification} decomposes the problem of unifying two terms $s \stackrel{.}{=} t$ into a number
of related unification problems between pairs of terms $s_i \stackrel{.}{=} t_i$. These smaller
problems are not tackled independently, but kept together in a set $S$.
The algorithm itself is typically formulated as a small-step-style state
transition system $S \rightarrowtail S'$ that rewrites the set of unification
problems until it is in solved form or until a contradiction has been found.
For instance, the variable elimination rule is written as:
\[
   x \stackrel{.}{=} t, S  ~~\rightarrowtail~~  x \stackrel{.}{=} t, [t/x]S   \qquad\qquad{if}~x \not\in t~\text{and}~{x \in S}
\]
Because the whole set is explicitly available, the variable $x$ can be
simultaneously substituted.

In the above unification problem, all variables are implicitly bound in the same
global scope.
Some constraint solving algorithms for Hindley-Milner type
inference use similar ideas, but are more careful tracking the
scopes of variables~\citep{remy-attapl}.
% However they have separate phases for constraint generation and solving.
Recent unification algorithms for dependently-typed languages
are also more explicit about scopes. For instance, \citet{Reed2009} represents a unification
problem as $\Delta \vdash P$ where $P$ is a set of equations to be solved and $\Delta$ is
a (modal) context. \citet{Abel2011higher} even use multiple contexts within a unification problem.
Such a problem is denoted $\Delta \Vdash \mathcal{K}$ where the meta-context
$\Delta$ contains all the typings of meta-variables in the constraint set
$\mathcal{K}$. The latter consists of constraints like $\Psi \vdash M = N : C$
that are equipped with their individual context $\Psi$. While accurately tracking
the scoping of regular and meta-variables, this approach is not ideal because it
repeatedly copies contexts when decomposing a unification problem, and
later it has to
substitute solutions into these copies.

% %-------------------------------------------------------------------------------
% \subsection{Small-Step Unification}
% % Mention some algorithms for unification for
% % dependent types that use a small-step approach. Credit them later for some ideas that
% % we also employ.
% There are literals that make use of small-step unification for dependently typed
% inference and reconstruction algorithms~\citep{Reed2009,Abel2011higher}.
% These approaches collect a list of constraints and process one at a time.
% Similar to DK's algorithm, unification variables are used to represent the types to guess.
% In order to solve unification variables to correctly scoped types
% (typically constrained by the context when the variable is introduced),
% the context information should be kept with the variables.
% Judgments that represent unifications of terms might also require such a context
% to keep the constraint collection well-formed.
%
% As an example, Abel et al. defines their constraint $K$ by
% \begin{gather*}
% \begin{aligned}
% K &::= {\top} \mid {\bot} &&\text{Trivial constraint and inconsistency.}\\
%     & \ \mid \  \Psi\vdash M = N : C &&\text{Unify term $M$ with $N$.}\\
%     & \ \mid \  \Psi\mid R:A \vdash E = E' &&\text{Unify evaluation context $E$ with $E'$.}\\
%     & \ \mid \  \Psi\vdash u\leftarrow M: C &&\text{Solution for $u$ (metavariable) found.}
% \end{aligned}
% \end{gather*}
% where the unification constraints and metavariable solutions are bound by a proper context.
% A unification problem in their system is described by $\Delta \Vdash \overline{K}$,
% where $\Delta$ is a collection of metavariables with their defined scopes.
% This approach clearly states the scroping of each unification problem,
% therefore rules out invalid instantiations to metavariables.
%
% However, such ``duplicated contexts'' are not ideal for our formalization.
% Following DK's algorithm, an existential variable $\al$ defined in the context
% could be decomposed into a function type $\al[1] \to \al[2]$,
% so the declaration of $\al$ should be replaced by two declarations $\al[1], \al[2]$.
% Such operation causes all the context that contains $\al$ to change,
% which brings difficulty on the synchronization of multiple contexts,
% as variable declarations are not centralized.
%
% We find a way of encoding multiple contexts by ``compressing'' them into a single worklist.
% As our type inference rules applied to a judgment,
% we typically keep the context or add variable declarations to the old context
% before analysing its sub-judgment(s).
% This enables us to write the worklist judgment in such a form:
% $$\Gm, \{\text{variable declarations}\}, \{\text{judgment}\},
% \{\text{variable declarations}\}, \{\text{judgment}\}, \ldots .$$
% Typically, we simplify the right-most judgment and push back smaller tasks,
% where the old ``context'' automatically gets inherited.
% When some rules solve or partially solve an existential variable,
% we could easily propagate the solution to all the judgments,
% and safely modify its declaration, such as remove from the worklist.
% New judgment(s) may also introduce new \emph{local} variables by declaring
% that variable right before the new judgment(s).
% After the new judgments are satisfied, these local variables are properly recycled.
%
% % \jimmy{TODO explain the duplicated context with some examples (their judgment form)}
% % Some issues to point out: Duplicated contexts (rather than shared contexts), which make
% % the formalization harder since requires ``synchronizing'' things ...
%
%-------------------------------------------------------------------------------
\subsection{Single-Context Worklist Algorithm for Subtyping}

As we have seen in Chapter~\ref{chap:ITP},
an algorithm based on \emph{worklist judgments} is mechanized
and shown to be correct with respect to DK's declarative
subtyping judgment.
This approach overcomes some problems in DK's algorithmic formulation
by using a worklist-based judgment of the form $$\Gamma \vdash \Omega$$
where $\Omega$ is a worklist (or sequence) of subtyping problems of the
form $A \leq B$.  The judgment is defined by case analysis on the first
element of $\Omega$ and recursively processes the worklist until it is empty.
Using both a single common ordered context $\Gamma$ and a worklist $\Omega$ greatly
simplifies propagating the instantiation of type variables in one
subtyping problem to the remaining ones.

Unfortunately, this work does not solve all problems. In particular, it has two
major limitations that prevent it from scaling to the whole DK system.

\paragraph{Scoping Garbage} Firstly, the single common ordered context
$\Gamma$ does not accurately reflect the type and unification variables
currently in scope. Instead, it is an overapproximation that steadily accrues
variables, and only drops those unification variables that are instantiated.
In other words, $\Gamma$ contains ``garbage'' that is no longer in scope.
This complicates establishing the relation with the declarative system.

\paragraph{No Inference Judgments}
Secondly, and more importantly, the approach only deals with a judgment for
\emph{checking} whether one type is the subtype of another. While this may not
be so difficult to adapt to the \emph{checking} mode of term typing $\Gamma
\vdash e \Leftarrow A$, it clearly lacks the functionality to support the
\emph{inference} mode of term typing $\Gamma \vdash e \Rightarrow A$. Indeed,
the latter requires not only the communication of unification variable
instantiations from one typing problem to another, but also an inferred type.

%-------------------------------------------------------------------------------
\subsection{Algorithmic Type Inference for Higher-Ranked Types: Key Ideas}

Our new algorithmic type system builds on the work above, but
addresses the open problems.

\paragraph{Scope Tracking}
We avoid scoping garbage by blending the ordered context and the
worklist into a single syntactic sort $\Gamma$, our algorithmic worklist. This
algorithmic worklist interleaves (type and term) variables with \emph{work}
like checking or inferring types of expressions. The interleaving keeps track
of the variable scopes in the usual, natural way: each variable is in scope of
anything in front of it in the worklist. If there is nothing in front, the
variable is no longer needed and can be popped from the worklist. This way, no
garbage (i.e. variables out-of-scope) builds up.

%\jimmy{Comment A is not satisfied with the term "continuation passing style" for our definition.}

$$\begin{aligned}
\text{Algorithmic judgment chain}\qquad&\jg &::=&\quad A \le B \mid e\Lto A \mid e\To_{a} \jg \mid \appInfAlg{A}{e}\\
\text{Algorithmic worklist}\qquad&\Gm &::=&\quad \nil \mid \Gm, a \mid \Gm, \al \mid \Gm, x: A \mid \Gm \Vdash \jg\\
\end{aligned}$$

For example, suppose that the top judgment of the following worklist
checks the identity function against $\all a \to a$:
$$\Gm \Vdash \lam x \Lto \all a \to a$$
To proceed, two auxiliary variables $a$ and $x$ are introduced to help the type checking:
$$\Gm, a, x:a \Vdash x \Lto a$$
which will be satisfied after several steps, and the worklist becomes
$$\Gm, a, x:a$$
Since the variable declarations $a, x:a$ are only used for a judgment already processed,
they can be safely removed, leaving the remaining worklist $\Gm$ to be further reduced.

Our worklist can be seen as an all-in-one stack,
containing variable declarations and subtyping/ typing judgments.
The stack is an enriched form of ordered context,
and it has a similar variable scoping scheme.
%The well-formedness condition, shown in Figure~\ref{fig:alg:syntax},
%states that every variable must be declared prior to it usage.
%Therefore, if any variable declaration is the top element,
%then no usage should appear in the rest of the worklist, so it can be removed.

% $$\Gm \Vdash \all a \to a \le \all a \to a$$
% $$\Gm, a, \al \Vdash \al \to \al \le a \to a$$

\paragraph{Inference Judgments}
To express the DK's inference judgments, we use a novel form of work entries in
the worklist: our algorithmic judgment chains $\omega$. In its simplest form,
such a judgment chain is just a check, like a subtyping check $A \leq B$ or a
term typecheck $e \Leftarrow A$.
However, the non-trivial forms of chains capture an
inference task together with the work that depends on its outcome. For
instance, a type inference task takes the form $e \Rightarrow_a \omega$.
This form expresses that we need to infer the type, say $A$, for expression $e$ and use it
in the chain $\omega$ by substituting it for the placeholder type variable $a$.
Notice that such $a$ binds a fresh type variable for the inner chain $\jg$,
which behaves similarly to the variable declarations in the context.
%\bruno{Removed CPS mention here. But if we remove it here should we remove it
%everywhere? Tom, your input would be good regarding this.}

Take the following worklist as an example
$$\al \Vdash \underline{(\lam x)~() \To_a a \le \al} , x:\al, \bt \Vdash \underline{\al \le \bt} \Vdash \underline{\bt \le 1}$$
There are three (underlined) judgment chains in the worklist,
where the first and second judgment chains (from the right) are two subtyping judgments,
and the third judgment chain, $(\lam x)~() \To_a a \le \al$,
is a sequence of an inference judgment followed by a subtyping judgment.

The algorithm first analyses the two subtyping judgments and
will find the best solutions $\al = \bt = 1$
(please refer to Figure~\ref{fig:alg} for detailed derivations).
Then we substitute every instance of $\al$ and $\bt$ with $1$,
so the variable declarations can be safely removed from the worklist.
Now we reduce the worklist to the following state
$$\nil \Vdash \underline{(\lam x)~() \To_a a \le 1}, x:1$$
which has a term variable declaration as the top element.
%Because no judgments before the declaration refer to $x$,
%as a basic property of an ordered context
%(Figure~\ref{fig:alg:syntax} shows its well-formedness criteria),
%we remove all declarations before the first judgment chain to collect unused variables.
%As a result all garbage variables are collected
%once the judgment chains refering to them are processed,
%just as what we have mentioned above about the scope tracking.
After removing the garbage term variable declaration from the worklist, we process the last remaining
inference judgment $(\lam x)~()\To~?$, with the unit type $1$ as its result.
Finally, the last judgment becomes $1 \le 1$, a trivial base case.

% Our algorithm borrows some ideas from previous work, while adding new ones.
% A small-step style processing worklists; \emph{Judgment Chains}; others.

% Importantly we now deal with inference judgment.


% !TEX root = scombinatorics.tex
\documentclass[scombinatorics.tex]{subfiles}
\begin{document}

\chapter{Szemer\'edi regularity lemma}

\label{szemeredi}

\def\medrel#1{\parbox[t]{6ex}{$\displaystyle\hfil #1$}}
\def\ceq#1#2#3{\parbox[t]{25ex}{$\displaystyle #1$}\medrel{#2}{$\displaystyle #3$}}

\section{The regularity lemma}

% For $B\subseteq\V$ we write $\sim_B$ for the equivalence relation on $\U$ of defined by

% \ceq{\hfill\emph{$a\sim_Ba'$}}
% {\IMP}
% {\phi(a\,;B)=\phi(a'\,;B)}

% A similar notation is used in the dual case, i.e.\@ to any $A\subseteq\U$ we associate a equivalence relation $\sim_A$ on $\V$.

% For every $\epsilon$ there is an $N=N(\epsilon)$ and two sets $A\subseteq\U$ $B\subseteq\V$ such that

% A multiset $A:\U\to\RR$ is \emph{simple\/} if for some finite $B$

% \ceq{\hfill a\sim_Ba'}
% {\IMP}
% {A(a)=A(a').}

% We may also say \emph{$B$-simple.}

% that are constant on $\sim_\V$ equivalence classes.

% For $B\subseteq\V$ the equivalence is defined

% is $\phi(\U\,;b)_{b\in\V}$-compatible if

% \ceq{\hfill \phi(a\,;\V)=\phi(a'\,;)}
% {\IMP}
% {A(a)=A(a')}

% For every $b\in B$ the multiset $\phi(A\,b)$ is two valued fun

% Let $A\subseteq\U$ then $\Indicator_A$ is constant on $B$ if and only if

% \begin{proposition}
%   For every multiset $A$, every $\epsilon>0$ and every finite $B\subseteq\U$ there is a multiset $A'$ constant on $B$ such that

% \end{proposition}

\section{Stable regularity lemma}

\section{Distal regularity lemma}
\end{document}

% \begin{proposition}
%   The following are equivalent
%   \begin{itemize}
%     \item[1.] every $b\in B$ the multiset $\phi(A\,;b)$ is constant on its support;
%     \item[2.] $\phi(a\,;B)=\phi(a';B)\ \IMP\ A(a)=A(a')$\quad for every $a,a'\in\U$.
%   \end{itemize}
% \end{proposition}

% \begin{proof}
%   \ssf{1}$\IMP$\ssf{2} Suppose $\phi(a\,;B)=\phi(a'\,;B)$ and $A(a)\neq A(a')$ for some  $a,a'\in\U$.

% \end{proof}


% !TEX TS-program = pdflatex
% !TEX encoding = UTF-8 Unicode

\documentclass{beamer}

\mode<presentation> { }

\usepackage[english]{babel}

\usepackage[utf8]{inputenc}
\usepackage{xcolor}
\usepackage{listings}
\usepackage{lsthaskell}
\usepackage{times}
\usepackage[T1]{fontenc}
\usepackage{verbatim}
\usepackage{mathrsfs}
\usepackage{mathtools}
\usepackage{proof}
\usepackage{amscd}
\setbeamertemplate{frametitle}
  {\begin{centering}\smallskip
   \insertframetitle\par
   \smallskip\end{centering}}
\setbeamertemplate{itemize item}{$\bullet$}
\setbeamertemplate{navigation symbols}{}
\setbeamertemplate{footline}[text line]{%
    \hfill\strut{%
        \scriptsize\sf\color{black!60}%
        \quad\insertframenumber
    }%
    \hfill
}

\definecolor{ltblue}{rgb}{0,0.4,0.4}
\definecolor{dkblue}{rgb}{0,0.05,0.4}
\definecolor{dkgreen}{rgb}{0,0.35,0}
\definecolor{dkviolet}{rgb}{0.3,0,0.5}
\definecolor{dkred}{rgb}{0.5,0,0}
\definecolor{grey}{rgb}{0.5,0.5,0.6}
\usepackage{listings}
\lstset{
basicstyle=\footnotesize,
columns=fullflexible,
language=Haskell}

%settings
\everymath{\displaystyle}

%--------

\title[]{Different recursion schemas and their properties}

\author{I.Zhirkov}

\date{2015}
\subject{Theoretical Computer Science}

\newcommand{\hin}[2]{\lstinputlisting[firstline=#1, lastline=#2]{listings/algebra.hs}}

% Delete this, if you do not want the table of contents to pop up at
% the beginning of each subsection:
\AtBeginSubsection[]
{
  \begin{frame}<beamer>{Outline}
    \tableofcontents[currentsection,currentsubsection]
  \end{frame}
}

% If you wish to uncover everything in a step-wise fashion, uncomment
% the following command:

%\beamerdefaultoverlayspecification{<+->}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

%\begin{frame}{Outline}
%  \tableofcontents
%\end{frame}

\begin{frame}
\frametitle{Morphisms}
\begin{itemize}
    \item Catamorphisms (destruction)
    \item Anamorphisms (construction)
    \item Hylomorphisms (combination of two)
    \item Paramorphisms (saves more information than hylomorphisms)
\end{itemize}
\end{frame}

\section{Catamorphisms}

\begin{frame}{Catamorphism}
\begin{itemize}
\item ``generalized fold''

$b::B$

\begin{align*}
h::&\  \text{List}\ A \rightarrow B\\
\\
h\ Nil &= b\\
h\ (Cons \ a\ as) &=  f\ a\ (h\ as)
\end{align*}

~

\item Notation: $h = (\!|b, f|\!)$
\item Arises from algebra ($f\ a \rightarrow a$).
\end{itemize}

\end{frame}

\begin{frame}{Anamorphism}
\begin{itemize}
\item ``generalized unfold''

$p::\ B\rightarrow Bool$

$g:: B \rightarrow (A,B)$

~

\begin{align*}
h::&\  B \rightarrow List\ A\\
h\ b &= Nil, & p \ b \\
h\ b &=  Cons a\ (h\ b'),& otherwise
\end{align*}

where $(a,b') = g\ b $

\item Notation $h = [\!(g,p)\!]$
\item Arises from coalgebra ($a \rightarrow f\ a$).
\end{itemize}

\end{frame}

\begin{frame}{Hylomorphism}
\begin{itemize}
\item ``call-tree looks like data structure''

$c::\ C$

$f:: B\rightarrow C \rightarrow C$

$g:: A \rightarrow (B,A)$

$p:\ A\rightarrow Bool$

~

\begin{align*}
h::&\   A \rightarrow C\\
h\ a &= c, & p \ a \\
 &= f\ b (h\ a'), & otherwise
\end{align*}

where $(b, a') = g\ a$

\item Notation $h = [\![(c,f), (g,p)]\!]$
\end{itemize}

\end{frame}

\begin{frame}{Hylomorphism-2}
\begin{itemize}

\item Is a composition of anamorphism and catamorphism

$[\![(c,f), (g,p)]\!] = (\!|c, f|\!) \circ [\!(g, p)\!]$

\item \textit{Look at the whiteboard for a fancy call-tree image}
\item Example: factorial is $[\![(1, \times), (g,p)]\!]$, where:

\begin{align*}
p\ n &= n == 0 \\
g\ n &= (n, n-1)
\end{align*}
\end{itemize}
\end{frame}

\begin{frame}[fragile]{Paramorphism}

\begin{itemize}
\item Hylomorphism for \textit{fac }is not inductively defined on \textit{nat}.
\item A nat paramorphism example:

\begin{align*}
h\ 0&= b\\
h\ (\text{Succ} \ n ) &= f\ n\ (h\ n)
\end{align*}

\item A list example:

\begin{align*}
h\ \text{Nil}& = b\\
h\ (\text{Cons}\ x\ xs ) &= f\ x\ xs\ (h\ xs)
\end{align*}

\item Notation: $\langle\![ b, f ]\!\rangle$
\end{itemize}

\end{frame}

\begin{frame}
We will provide some explanations about the algebra and coalgebra nature.

\end{frame}

\begin{frame}{Category}
$C = (obj, hom, \circ)$

Objects and morphisms between objects with their compositions.

Laws:
\begin{itemize}
\item Identity morphisms for each object
\item Composition is associative
\item Path equality
\end{itemize}

``Point'' is synonymous to ``Object''
\end{frame}

\begin{frame}{Categories: $Set$}
\begin{itemize}
\item $ob(Set)$ -- all sets
\item $hom(E,F)$ -- functions between sets $E$ and $F$
\item $\circ$ -- composition
\end{itemize}

\end{frame}

\begin{frame}{Categories: $Set$}
\begin{itemize}
\item $ob(Set)$ -- all sets
\item $hom(E,F)$ -- functions between sets $E$ and $F$
\item $\circ$ -- composition
\item Note: $ob(Set)$ is not a set itself (is a class).
\end{itemize}

\end{frame}

\begin{frame}{Categories}
\begin{itemize}
\item $Mon$: (monoids, morphisms, composition)
\item $Grp$: (groups, morphisms, composition)
\item $Hask$: (haskell types, functions, \hasqel{(.)})
\item ...
\end{itemize}

\end{frame}

\begin{frame}{Functor}
\begin{itemize}
\item Functor is a morphism between categories (preserves structures)
\end{itemize}

\end{frame}
\begin{frame}{Functor in FP}
\begin{itemize}
\item an (endo-)functor is an operation from types to types (from Hask to Hask)
\item preserves identity and composition.
\item functions can be 'mapped over' functors
\item Basic ones: identity, product, sum (tagged), arrow \dots

\end{itemize}

\end{frame}

\begin{frame}[fragile]{A usual recursive datatype}
\hin{9}{11}

is same as:
\hin{16}{20}
\end{frame}

\begin{frame}[fragile]{Test}
\hin{23}{24}
\end{frame}

\begin{frame}[fragile]{It is a functor!}

\hin{29}{32}

It is possible to construct an algebra on top of any functor.

\hin{55}{55}

We expect to be able to 'evaluate' children of \hasqel{Expr}.
Example:

\hin{37}{40}

\end{frame}

\begin{frame}[fragile]{What is an algebra?}

$(C, F, A, m)$
\begin{itemize}
\item Category $C$ ($Hask$, points are types of Haskell)
\item Endofunctor $F$ (maps Hask points into other Hask points)
\item Point $A$ (\textbf{carrier}) of category $C$ (some type).
\item Function $m$ mapping $F\ A \rightarrow A$
\end{itemize}

Hence our definition:

\hin{55}{55}

$C$ is implied ($Hask$), $F$ and $A$ are type-level, the rest is $m$, the function itself (of type \hasqel{Algebra f a}).

\textit{Look at the whiteboard for a fancy image!}

\end{frame}

\begin{frame}[fragile]{Initial algebra}
\begin{itemize}
\item There are infinitely many algebras based on same functor

\item One is particular \textbf{Initial algebra},

\item Does not 'forget' anything, preserves all information about input.

\item $\exists$ a homomorphism from initial algebra to any other algebra.

\item Assume: carrier is \hasqel{Fix f}, algebra is \hasqel{Fx} (its ctor).

\hin{55}{55}
\hin{63}{65}

\end{itemize}

\end{frame}

\begin{frame}[fragile]

Functor $f$ is fixed. Let $a$ be the carrier object for a new algebra.

~

\begin{tabular}{r | c | c}
 ~ & Carrier & Evaluator\\\hline
Initial algebra&   Fix f & Fx\\\hline
Some algebra &     a & alg \\
\end{tabular}

\end{frame}

\begin{frame}[fragile]{Constructing any algebra-1}

We want to get a homomorphism from initial algebra to some other algebra. Carrier mapper:

\hasqel{g:: Fix f -> a}

Remember: \hin{16}{16}

Thanks to the fact that $f$ is a functor, fmap is at our disposal to map :

\hasqel{fmap g :: f (Fix f) ->  f a}

\begin{center}
$
\begin{CD}
f (\text{Fix} \ f) @>\text{fmap} \ g>> f\ a\\
@VVFxV @VValgV\\
\text{Fix} \ f @>g>> a
\end{CD}
$

\end{center}
\end{frame}

\begin{frame}[fragile]{Constructing any algebra-2}

\hasqel{Fx} is lossless, thus invertible.

\hin{106}{107}

$
\begin{CD}
f (\text{Fix} \ f) @>\text{fmap} \ g>> f\ a\\
@VVFxV @VValgV\\
\text{Fix} \ f @>g>> a
\end{CD}
$
~~~~~becomes
$
\begin{CD}
f (\text{Fix} \ f) @>\text{fmap} \ g>> f\ a\\
@AAunFixA @VValgV\\
\text{Fix} \ f @>g>> a
\end{CD}
$

~

~

$g$ can be defined with unfix, fmap and evaluator:

\hin{110}{110}
\end{frame}

\begin{frame}[fragile]{Meet catamorphism}

\hin{110}{110}

\hin{115}{116}

Quick check:

\hin{43}{46}

\begin{verbatim}
*Main> :t cata alg
cata alg :: Fix ExprF -> String
\end{verbatim}
\end{frame}

\begin{frame}[fragile]{Coalgebra}

Algebra: $f\ a \rightarrow a$

Coalgebra: $a \rightarrow f\ a$

~

$
\begin{CD}
f (\text{Fix} \ f) @>\text{fmap} \ g>> f\ a\\
@VVFxV @VValgV\\
\text{Fix} \ f @>g>> a
\end{CD}
$
~~~~~becomes
$
\begin{CD}
f (\text{Fix} \ f) @<\text{fmap} \ g<< f\ a\\
@AAunFixA @AAcoalgA\\
\text{Fix} \ f @<g<< a
\end{CD}
$

~

~

~
The same reasoning about initial coalgebra applies.

\end{frame}

\begin{frame}{Coalgebra-2}
$\mu$ combines Fx and unFix.

\hin{141}{141}

\hin{142}{142}
\hin{55}{55}
\hin{145}{149}
\end{frame}

\begin{frame}{Coalgebra-3}

As $out$ is an initial algebra, $in$ is a terminal coalgebra (there exists a unique homomorphism from any coalgebra to $in$).
\begin{center}

$
\begin{CD}
f (\mu \ f) @<\text{fmap} \ g<< f\ a\\
@VVinV @AAcoalgA\\
\mu \ f @<g<< a
\end{CD}
$

~

~

~
\end{center}

\textit{Look at the whiteboard for a combined image}.

\end{frame}

\begin{frame}{Using fixed point}
We can define catamorphism and anamorphism in a non-recursive way.

\hin{168}{175}

\end{frame}

\begin{frame}{A note on notation}

Authors use this rather unusual notation:

$(f \xleftarrow{F} g) h = f \circ h_F \circ g$

$h_F$ is lifted inside the functor $F$.

Thus: $(f \xleftarrow{F} g)  = \lambda h. f \circ h_F \circ g$

Also:

$(f\ \Delta\  g) (x,y) = (f\  x, g \ y)$
\end{frame}

\begin{frame}{Notation summary}

\begin{itemize}
\item Catamorphism:

$(\!|\phi|\!)_F = \mu(\phi \xleftarrow{F} out)$

\hin{171}{171}

\item Anamorphism:

$[\!(\psi)\!]_F = \mu(in \xleftarrow{F} \psi )$

\hin{174}{174}

\item Hylomorphism:

$[\![\phi, \psi]\!]_F = \mu(\phi \xleftarrow{F} \psi )$

\item Paramorphism:

$\langle\![ \mathcal{E} ]\!\rangle = \mu ( \lambda f. \mathcal{E} \circ (id\ \Delta\ f)_F \circ out) $
\end{itemize}

\textit{Remember:} $(f \xleftarrow{F} g)  = \lambda h. f \circ h_F \circ g$

\end{frame}

\begin{frame}
For each morphism type we give:
\begin{itemize}
\item Evaluation rule
\item Uniqueness property
\item Fusion law (class-preserving functions mapping over morphism)
\end{itemize}
\end{frame}

\begin{frame}{Utility: fixed point fusion}

$f ( \bot ) = \bot \land f \circ g = h \circ f \Rightarrow f (\mu \ g) = \mu \ h $ (without proof)

\end{frame}

\begin{frame}[fragile]{Catamorphism}
\begin{itemize}
\item Evaluation rule:

$$(\!| \phi |\!)\circ in = \phi \circ (\!| \phi |\!)$$

Apply recursively, then $\phi$ again. Compare with fixed point:

$$x = \mu f \Rightarrow x = f \ x$$

\item \textbf{Uniqueness Property} (prove functions equality without explicit induction)

$f = (\!| \phi |\!)_F \equiv  f \circ \bot = ( \!| \phi |\!)_F \circ \bot \land f \circ in = \phi \circ f_{\mu\ F}$

Remember, \hasqel{In} is also a type ctor, applying isomorphism between $a$ and $F a$
\end{itemize}

\end{frame}

\begin{frame}{Catamorphisms - Fusion law}

Blend $cata\circ f$ into a single catamorphism.

$$f \circ (\!|\phi|\!) =  (\!|\psi|\!)  \Leftarrow f \ \bot = \bot \land f \circ \phi = \psi \circ f_{\mu F}$$

\textit{Look at the whiteboard for a fancy diagram}

~

\textbf{Useful variation}: $f$ is strict, not ``like $(\!|\psi|\!)$'':

$$f \circ (\!|\phi|\!) =  (\!|\psi|\!)  \Leftarrow f \circ \bot = (\!|\psi|\!)  \circ \bot \land f \circ \phi = \psi \circ f_{\mu F}$$

\end{frame}

\begin{frame}{Injective functions are catamorphisms}

$f: A \rightarrow B$\\
$\phi: A_F \rightarrow A$

Then:

$$f \circ (\!|\phi|\!) = (\!|f \circ \phi \circ g_F |\!) \Leftarrow \begin{cases} f \bot = \bot \\ g \circ f = id\end{cases}$$

Take $(\!|\phi|\!) = in$ :

$$f = (\!|f \circ in \circ g_F |\!) \Leftarrow \begin{cases} f \bot = \bot \\ g \circ f = id\end{cases}$$
\end{frame}

\begin{frame}{Catamorphisms preserve strictness}

A useful lemma for some proofs:

$$\mu F \circ \bot = \bot \Leftarrow \forall f : F f  \circ \bot = \bot$$

For cata:

$(\!|\phi |\!) _{F} \circ \bot = \bot \equiv \phi \circ \bot = \bot $

\end{frame}

\begin{frame}{Example: fold-unfold}
Whiteboard
\end{frame}

\begin{frame}{Anamorphisms}
Evaluation rule:

$$out \circ [\!(\psi )\!] = [\!(\psi )\!] _F \cdot \psi$$

Apply $[\!(\psi )\!] $, then apply $[\!(\psi )\!]_F$ to the result.

$$f = [\!(\psi )\!]  \equiv out \circ f = f_F \cdot \psi$$

\end{frame}

\begin{frame}{Fusion law}

$$[\!(\phi )\!] \circ f = [\!(\psi )\!]  \Leftarrow \phi \circ f = f_F \circ \psi$$

Proof by fixed point fusion theorem with $f := (\circ f)$, $g := in \leftarrow^F \phi $, $h :=in \leftarrow^L \psi$

\end{frame}
\begin{frame}{Surjective function is an anamorphism}

\end{frame}
\begin{frame}{Examples}

$iterate f = [\!(firstOfSum \circ id \Delta f  )\!] $

$takewhile\ p = [\!(secondOfSum\ 'map'\  (VOID  | id \circ ( not\ p \circ second)? ) \circ out \!] $

$f 'map' g \circ p?$ models if-then-else

\end{frame}
\begin{frame}

``Actually, even in Haskell recursion is not completely first class because the compiler does a terrible job of optimizing recursive code. This is why F-algebras and F-coalgebras are pervasive in high-performance Haskell libraries like vector, because they transform recursive code to non-recursive code, and the compiler does an amazing job of optimizing non-recursive code.'' (Gabriel Gonzales)

\end{frame}
\begin{frame}
Further reading:
\begin{itemize}
\item Control.Functor.Algebra
\end{itemize}
\end{frame}

\end{document}


\documentclass[10pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage[export]{adjustbox}
\graphicspath{ {./images/} }
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{mhchem}
\usepackage{stmaryrd}
\usepackage{bbold}

\title{Jinchao Xu }

\author{Deep Learning Algorithms and\\
Analysis}
\date{}

\begin{document}
\maketitle

Summer 2020

\includegraphics[max width=\textwidth]{2022_01_05_e65f0d6bf0db4974ee45g-02}

\section{Contents}
\includegraphics[max width=\textwidth]{2022_01_05_e65f0d6bf0db4974ee45g-03}

1.1 Motivation: from finite element to neural network $\ldots \ldots \ldots \ldots \ldots .5$

$1.2$ Why we need deep neural networks via composition $\ldots \ldots \ldots \ldots .7$

1.2.1 FEM ans DNN $_{1}$ in $1 \mathrm{D} \ldots \ldots . . . . . . . . . . . . . . . . . .$

1.2.2 Linear finite element cannot be recovered by DNN $_{1}$ for $d \geq 2 \quad 8$

$1.3$ Definition of neural network space $\ldots \ldots \ldots \ldots \ldots \ldots \ldots \ldots 11$

\includegraphics[max width=\textwidth]{2022_01_05_e65f0d6bf0db4974ee45g-04}

\section{FEM and DNN}
\subsection{Motivation: from finite element to neural network}
In this chapter, we will introduce the so-called shallow neural network (deep neural network with one hidden layer) from the viewpoint of finite element method.

Let us first consider the linear finite element functions on the unit interval $\bar{\Omega}=$ $[0,1]$ in $1 \mathrm{D}$. We then consider a set of equidistant girds $\Omega_{\ell}$ of level $\ell$ on the unit interval $\bar{\Omega}=[0,1]$ and mesh length $h_{\ell}=2^{-\ell}$. The grid points $x_{\ell, i}$ are given by
$$
x_{\ell, i}:=i h_{\ell}, \quad 0 \leq i \leq 2^{\ell} .
$$
For $\ell=1$, we denote the special hat function:
$$
\varphi(x)= \begin{cases}2 x & x \in\left[0, \frac{1}{2}\right] \\ 2(1-x) & x \in\left[\frac{1}{2}, 1\right] \\ 0, & \text { others }\end{cases}
$$
The next diagram shows this basis function:

\includegraphics[max width=\textwidth]{2022_01_05_e65f0d6bf0db4974ee45g-05}

Fig. 1.1. Diagram of $\varphi(x)$ Then, for any nodal basis function $\varphi_{\ell, i}$ as below:

\includegraphics[max width=\textwidth]{2022_01_05_e65f0d6bf0db4974ee45g-06}

Fig. 1.2. Diagram of $\varphi_{\ell, i}(x)$

in a fine grid $\mathcal{T}_{\ell}$ can be written as
$$
\varphi_{\ell, i}=\varphi\left(\frac{x-x_{\ell, i-1}}{2 h_{\ell}}\right)=\varphi\left(w_{\ell} x+b_{\ell, i}\right)
$$
That is to say, any $\varphi_{\ell, i}(x)$ can be obtained from $\varphi(x)$ ba scaling (dilation) and translation with
$$
w_{\ell}=2^{\ell-1}, \quad b_{\ell, i}=\frac{-(i-1)}{2}
$$
in $\varphi_{\ell, i}=\varphi\left(w_{\ell} x+b_{\ell, i}\right)$.

Let recall the finite element interpolation as
$$
u(x) \approx u_{\ell}(x):=\sum_{0 \leq i \leq 2^{\ell}} u\left(x_{\ell, i}\right) \varphi_{\ell, i}(x)
$$
for any smooth function $u(x)$ on $(0,1)$. The above interpolation will converge as $\ell \rightarrow \infty$, which show that
$$
\operatorname{span}\left\{\varphi\left(w_{\ell} x+b_{\ell, i}\right)\right\} \quad \text { is dense in } \quad H^{1}(0,1)
$$
Thus, we may have the next concise relation:

FE space $=\operatorname{span}\left\{\varphi\left(w_{\ell} x+b_{\ell, i}\right) \mid 0 \leq i \leq 2^{\ell}, \ell=1,2, \cdots\right\} \subset \operatorname{span}\{\varphi(w x+b) \mid w, b \in \mathbb{R}\} .$

In other words, the finite element space can be understood as the linear combination of $\varphi(w x+b)$ with certain special choice of $w$ and $b$.

Here, we need to point that this $\operatorname{span}\{\varphi(w x+b) \mid w, b \in \mathbb{R}\}$ is exact the deep neural networks with one hidden layer (shallow neural networks) with activation function $\varphi(x)$. More precisely,
$$
f \in \operatorname{span}\{\varphi(w x+b) \mid w, b \in \mathbb{R}\}
$$
means there exist positive integer $N$ and $w_{j}, b_{j} \in \mathbb{R}$ such that
$$
f=\sum_{j=1}^{N} a_{j} \varphi\left(w_{j} x+b_{j}\right)
$$
The above function is also called one hidden neural network function with $N$ neurons.

Remark $1 . \quad$ 1. By making $w_{\ell}$ and $b_{\ell, i}$ arbitrary, we get a much larger class of function which is exact a special neural network with activation function $\varphi(x)$.

  \begin{enumerate}
    \setcounter{enumii}{2}
    \item Generalizations:
  \end{enumerate}
a) $\varphi$ can be different, such as $\operatorname{ReLU}(x)=\max \{0, x\}$.

b) There is a natural extension for hight dimension $d$ as
$$
\{\varphi(w \cdot x+b)\}
$$
where $w \in \mathbb{R}^{d}, b \in \mathbb{R}$ and $w \cdot x=\sum_{i=1}^{d} w_{i} x_{i}$. This is called "deep" neural network with one hidden layer.

\subsection{Why we need deep neural networks via composition}
\subsubsection{FEM ans DNN $_{1}$ in 1D}
Thanks to the connection between $\varphi(x)$ and $\operatorname{ReLU}(x)=\{0, x\}$
$$
\varphi(x)=2 \operatorname{ReLU}(x)-4 \operatorname{ReLu}\left(x-\frac{1}{2}\right)+2 \operatorname{ReLU}(x-1)
$$
It suffices to show that each basis function $\varphi_{\ell, i}$ can be represented by a ReLU DNN. We first note that the basis function $\varphi_{i}$ has the support in $\left[x_{i-1}, x_{i+1}\right]$ can be easily written as
$$
\varphi_{\ell, i}(x)=\frac{1}{h_{\ell}} \operatorname{ReLU}\left(x-x_{\ell, i-1}\right)-\left(\frac{2}{h_{\ell}}\right) \operatorname{ReLU}\left(x-x_{\ell, i}\right)+\frac{1}{h_{\ell}} \operatorname{ReLU}\left(x-x_{\ell, i+1}\right)
$$
More generally, if function $\varphi_{i}$ is not on the uniform grid but has support in $\left[x_{i-1}, x_{i+1}\right]$ can be easily written as
$$
\varphi_{i}(x)=\frac{1}{h_{i-1}} \operatorname{ReLU}\left(x-x_{i-1}\right)-\left(\frac{1}{h_{i-1}}+\frac{1}{h_{i}}\right) \operatorname{ReLU}\left(x-x_{i}\right)+\frac{1}{h_{i}} \operatorname{ReLU}\left(x-x_{i+1}\right)
$$
where $h_{i}=x_{i+1}-x_{i}$.

Thus is to say, we have the next theorem.

Theorem 1. For $d=1$, and $\Omega \subset \mathbb{R}^{d}$ is a bounded interval, then DNN $_{1}$ can be used to cover all linear finite element function in on $\Omega$.

\subsubsection{Linear finite element cannot be recovered by DNN $_{1}$ for $d \geq 2$}
In view of Theorem 1 and the fact that $\mathrm{DNN}_{\mathrm{J}} \subseteq \mathrm{DNN}_{\mathrm{J}+1}$, it is natural to ask that how many layers are needed at least to recover all linear finite element functions in $\mathbb{R}^{d}$ for $d \geq 2$. In this section, we will show that
$$
J_{d} \geq 2, \quad \text { if } \quad d \geq 2
$$
where $J_{d}$ is the minimal $J$ such that all linear finite element functions in $\mathbb{R}^{d}$ can be recovered by DNN $_{J}$

In particular, we will show the following theorem.

Theorem 2. If $\Omega \subset \mathbb{R}^{d}$ is either a bounded domain or $\Omega=\mathbb{R}^{d}, \mathrm{DNN}_{1}$ can not be used to recover all linear finite element functions on $\Omega$.

Proof. We prove it by contradiction. Let us assume that for any continuous piecewise linear function $f: \Omega \rightarrow \mathbb{R}$, we can find finite $N \in \mathbb{N}, w_{i} \in \mathbb{R}^{1, d}$ as row vector and $\alpha_{i}, b_{i}, \beta \in \mathbb{R}$ such that
$$
f=\sum_{i=1}^{N} \alpha_{i} \operatorname{ReLU}\left(w_{i} x+b_{i}\right)+\beta
$$
with $f_{i}=\alpha_{i} \operatorname{ReLU}\left(w_{i} x+b_{i}\right), \alpha_{i} \neq 0$ and $w_{i} \neq 0 .$ Consider the finite element functions, if this one hidden layer ReLU DNN can recover any basis function of FEM, then it can recover the finite element space. Thus let us assume $f$ is a locally supported basis function for FEM. Furthermore, if $\Omega$ is a bounded domain, we assume that
$$
d(\operatorname{supp}(f), \partial \Omega)>0
$$
with
$$
d(A, B)=\inf _{x \in A, y \in B}\|x-y\|,
$$
as the distance of two closed sets.

A more important observation is that $\nabla f: \Omega \rightarrow \mathbb{R}^{d}$ is a piecewise constant vector function. The key point is to consider the discontinuous points for $g:=\nabla f=$ $\sum_{i=1}^{N} \nabla f_{i} .$

For more general case, we can define the set of discontinuous points of a function by
$$
D_{g}:=\{x \in \Omega \mid x \text { is a discontinuous point of } g\}
$$
Because of the property that
$$
D_{f+g} \supseteq D_{f} \cup D_{g} \backslash\left(D_{f} \cap D_{g}\right),
$$
we have
$$
D_{\sum_{i=1}^{N} g_{i}} \supseteq \bigcup_{i=1}^{N} D_{g_{i}} \backslash \bigcup_{i \neq j}\left(D_{g_{i}} \cap D_{g_{j}}\right)
$$
Note that
$$
g_{i}=\nabla f_{i}(x)=\nabla\left(\alpha_{i} \operatorname{ReLU}\left(w_{i} x+b_{i}\right)\right)=\left(\alpha_{i} H\left(w_{i} x+b_{i}\right)\right) w_{i} \in \mathbb{R}^{d},
$$
for $i=1: N$ with $H$ be the Heaviside function defined as:
$$
H(x)= \begin{cases}0 & \text { if } x \leq 0 \\ 1 & \text { if } x>0\end{cases}
$$
This means that
$$
D_{g_{i}}=\left\{x \mid w_{i} x+b_{i}=0\right\}
$$
is a $d-1$ dimensional affine space in $\mathbb{R}^{d}$

Without loss of generality, we can assume that
$$
D_{g_{i}} \neq D_{g_{j}}
$$
When the other case occurs, i.e. $D_{g_{6}}=D_{g_{2}}=\cdots=D_{g_{\ell_{2}}}$, by the definition of $g_{i}$ in (1.18) and $D_{g_{i}}$ in (1.19), this happens if and only if there is a row vector $(w, b)$ such that
$$
c_{\ell_{i}}(w b)=\left(w_{\ell_{i}} b_{\ell_{i}}\right)
$$
with some $c_{\ell_{i}} \neq 0$ for $i=1: k$. We combine those $g_{\ell_{i}}$ as
$$
\begin{aligned}
\tilde{g}_{\ell} &=\sum_{i=1}^{k} g_{\ell_{i}}=\sum_{i=1}^{k} \alpha_{\ell_{i}} H\left(w_{\ell_{i}} x+b_{\ell_{i}}\right) w_{\ell_{i}}, \\
&=\sum_{i=1}^{k}\left(c_{\ell_{i}} \alpha_{\ell_{i}} H\left(c_{\ell_{i}}(w x+b)\right)\right) w, \\
&=\left\{\begin{array}{lll}
\left(\sum_{i=1}^{k} c_{\ell_{i}} \alpha_{\ell_{i}} H\left(c_{\ell_{i}}\right)\right) w & \text { if } & w x+b>0, \\
\left(\sum_{i=1}^{k} c_{\ell_{i}} \alpha_{\ell_{i}} H\left(-c_{\ell_{i}}\right)\right) w & \text { if } & w x+b \leq 0 .
\end{array}\right.
\end{aligned}
$$
Thus, if
$$
\left(\sum_{i=1}^{k} c_{\ell_{i}} \alpha_{\ell_{i}} H\left(c_{\ell_{i}}\right)\right)=\left(\sum_{i=1}^{k} c_{\ell_{i}} \alpha_{\ell_{i}} H\left(-c_{\ell_{i}}\right)\right)
$$
$\tilde{g}_{\ell}$ is a constant vector function, that is to say $D_{\sum_{i=1}^{k} g_{\ell_{i}}}=D_{\tilde{g}_{\ell}}=\emptyset .$ Otherwise, $\tilde{g}_{\ell}$ is a piecewise constant vector function with the property that
$$
D_{\sum_{i=1}^{k} g_{\ell_{i}}}=D_{\tilde{g}_{\ell}}=D_{g_{\ell_{i}}}=\{x \mid w x+b=0\} .
$$
This means that we can use condition (1.21) as an equivalence relation and split $\left\{g_{i}\right\}_{i=1}^{N}$ into some groups, and we can combine those $g_{\ell_{i}}$ in each group as what we do above. After that, we have
$$
\sum_{i=1}^{N} g_{i}=\sum_{\ell=1}^{\bar{N}} \tilde{g}_{\ell}
$$
with $D_{\tilde{g}_{s}} \neq D_{\bar{g}_{t}}$. Finally, we can have that $D_{\tilde{g}_{s}} \cap D_{\bar{g}_{t}}$ is an empty set or a $d-2$ dimensional affine space in $\mathbb{R}^{d} .$ Since $N \leq N$ is a finite number,
$$
D:=\bigcup_{i=1}^{N} D_{\tilde{g}_{\ell}} \backslash \bigcup_{s \neq t}\left(D_{\tilde{g}_{s}} \cap D_{\tilde{g}_{t}}\right)
$$
is an unbounded set.

  \begin{itemize}
    \item If $\Omega=\mathbb{R}^{d}$,
  \end{itemize}
$$
\operatorname{supp}(\mathrm{f}) \supseteq D_{g}=D_{\sum_{i=1}^{N} g_{i}}=D_{\sum_{t=1}^{N} \tilde{g}_{\ell}} \supseteq D
$$
is contradictory to the assumption that $f$ is locally supported.

  \begin{itemize}
    \item If $\Omega$ is a bounded domain,
  \end{itemize}
$$
d(D, \partial \Omega)= \begin{cases}s>0 & \text { if } D_{\tilde{g}_{i}} \cap \Omega=\emptyset, \forall i \\ 0 & \text { otherwise. }\end{cases}
$$
Note again that all $D_{\tilde{g}_{i}}$ 's are $d-1$ dimensional affine spaces, while $D_{\tilde{g}_{i}} \cap D_{\tilde{g}_{j}}$ is either an empty set or a d-2 dimensional affine space. If $d(D, \partial \Omega)>0$, this implies that $\nabla f$ is continuous in $\Omega$, which contradicts the assumption that $f$ is a basis function in FEM. If $d(D, \partial \Omega)=0$, this contradicts the previous assumption in (1.15).

Hence DNN $_{1}$ cannot recover any piecewise linear function in $\Omega$ for $d \geq 2 .$

Following the proof above, we have the following theorem

Theorem 3. $\left\{\operatorname{ReLU}\left(w_{i} x+b_{i}\right)\right\}_{i=1}^{m}$ are linearly independent if $\left(w_{i}, b_{i}\right)$ and $\left(w_{j}, b_{j}\right)$ are linearly independent in $\mathbb{R}^{1 \times(d+1)}$ for any $i \neq j .$

\subsection{Definition of neural network space}
  \begin{enumerate}
    \item Primary variables $n_{0}=d$
  \end{enumerate}
$$
x^{0}=x=\left(\begin{array}{c}
x_{1} \\
x_{2} \\
\vdots \\
x_{n}
\end{array}\right)
$$

  \begin{enumerate}
    \setcounter{enumii}{2}
    \item $n_{1}$ hyperplanes
  \end{enumerate}
$$
W^{1} x+b^{1}=\left(\begin{array}{c}
w_{1}^{1} x+b_{1}^{1} \\
w_{2}^{1} x+b_{2}^{1} \\
\vdots \\
w_{n}^{1} x+b_{n}^{1}
\end{array}\right)
$$

  \begin{enumerate}
    \setcounter{enumii}{3}
    \item $n_{1}$-neurons:
  \end{enumerate}
$$
x^{1}=\sigma\left(W^{1} x+b^{1}\right)=\left(\begin{array}{c}
\sigma\left(w_{1}^{1} x+b_{1}^{1}\right) \\
\sigma\left(w_{2}^{1} x+b_{2}^{1}\right) \\
\vdots \\
\sigma\left(w_{n}^{1} x+b_{n}^{1}\right)
\end{array}\right)
$$

  \begin{enumerate}
    \setcounter{enumii}{4}
    \item $n_{2}$-hyperplanes
  \end{enumerate}
$$
W^{2} x^{1}+b^{2}=\left(\begin{array}{c}
w_{1}^{2} x^{1}+b_{2}^{2} \\
w_{2}^{2} x^{1}+b_{2}^{2} \\
\vdots \\
w_{n}^{2} x^{1}+b_{n}^{2}
\end{array}\right)
$$
Shallow neural network functions:

${ }_{n} \mathrm{~N}\left(n_{1}, n_{2}\right)={ }_{n} \mathrm{~N}\left(\sigma ; n_{1}, n_{2}\right)=\left\{W^{2} x^{1}+b^{2}, x^{1}=\sigma\left(W^{1} x+b^{1}\right)\right.$ with $\left.W^{\ell} \in \mathbb{R}^{n_{\ell} \times n_{\ell-1}}, b^{\ell} \in \mathbb{R}^{n_{\ell}}, \ell=1,2, n_{0}=d\right\}$

${ }_{n} \mathrm{~N}\left(\sigma ; n_{1}, n_{2}, \ldots, n_{L}\right)=\left\{W^{L} x^{L-1}+b^{L}, x^{\ell}=\sigma\left(W^{\ell} x^{\ell-1}+b^{\ell}\right)\right.$ with $\left.W^{\ell} \in \mathbb{R}^{n_{\ell} \times n_{\ell-1}}, b^{\ell} \in \mathbb{R}^{n_{\ell}}, \ell=1: L, n_{0}=d, x^{0}=x\right\}$

First, let us first define the so-called 1-hidden layer (shallow) neural network.

The 1 -hidden layer (shallow) neural network is defined as:
$$
{ }_{n} \mathrm{~N}={ }_{n} \mathrm{~N}(\sigma)={ }_{n} \mathrm{~N}^{1}(\sigma)=\bigcup_{n_{1} \geq 1} \mathrm{~N}\left(\sigma ; n_{1}, 1\right)
$$
The 2-hidden layer (shallow) neural network is defined as:
$$
{ }_{n} \mathrm{~N}^{2}(\sigma)=\bigcup_{n_{1}, n_{2} \geq 1} \mathrm{~N}\left(\sigma ; n_{1}, n_{2}, 1\right)
$$

\end{document}


\documentclass[../main]{subfiles}
\pagestyle{fancy}

\begin{document}

\chapter{Core model theory}
\label{apx.core-model-theory}
\thispagestyle{fancy}

As we will be utilising the \textit{core model} at various points throughout this thesis, we give here an idea of what we mean by the \textit{core model}. A convenient feature of core model theory is that most of the technical details regarding the construction is not needed for applications; it suffices to know only its abstract properties. That being said, we \textit{will} provide a glimpse of the construction at the end of this section. To see the full construction we refer the interested reader to \cite{MSc}, \cite{Zeman} and \cite{JensenSteel}.

\section{The core model $K$}

The \textbf{core model}\footnote{$K$ is short for \textit{Kern}, meaning \textit{core} in German.} $K$ of a universe is the roughly speaking the subuniverse that strikes a balance between retaining the complexity of the universe while being as simple as possible. The problem is then making all of this precise. Some aspects of the definition is agreed upon by most researchers:
\begin{enumerate}
  \item We choose to define the \textit{complexity} of a universe by its large cardinal structure. This is based on the empirical fact that large cardinals seem to capture the strength of every ``naturally defined'' hypothesis, and gives us a convenient yard stick. For instance, a universe containing a measurable cardinal is more complex than $L$, as Scott's Theorem \ref{theo.scott} shows that $L$ cannot contain any measurable cardinals (or any large cardinals stronger than measurables);
  \item We further postulate that $L$ is the simplest universe there is, and the simplicity of a universe should therefore be measured in terms of how much it resembles $L$. We will be more precise about what it means to ``resemble $L$'' below, but with this intuitive notion is should at least be clear that, say, $L$ is simpler than $L[\mu]$.\\
\end{enumerate}

Even though $(i)$ captures what we mean by complexity, it leaves much to be desired. For instance, as the structure of the large cardinal hierarchy can only be verified empirically, we might end up in an unfortunate situation where we simply do not know whether a given universe is more complex than another one\footnote{It might also be the case that the large cardinal hierarchy is not linear at all.}. The famous example of this is the current situation with the superstrong- and strongly compact cardinals, that we simply do not know which one is stronger\footnote{Although the general consensus is that the strongly compact cardinals should be equiconsistent with the supercompacts, making them stronger than the superstrongs.}. Thus, given a universe whose strength corresponds to that of a strongly compact and another one at the level of superstrongs, we would not be able to say which one is more complex.

\qquad To remedy this unfortunate situation, we choose instead to define the complexity of a universe in terms of an intermediate property. A universe satisfying this property should then entail that it inherits the large cardinal structure of its surrounding universe. All the intermediate properties currently being used are all instances of a general phenomenon called \textit{covering}. The intuitive idea is that every set in the universe can be ``approximated'' by a set in the subuniverse, and arose from a seminal theorem of Jensen, see \cite[Theorem 11.56]{SchindlerBook}, stating that $0^\sharp$ exists if and only if \textit{strong covering} fails for $L$, defined as follows.

\defi[Jensen]{
  We say that \textbf{strong covering} holds for universes $\U\subset\V$ if to every $\alpha<o(\V)$ and $X\in\p^{\V}(\alpha)$ there exists $A\in\U$ such that $X\subset A$ and $\Card^{\V}(X) = \Card^{\V}(A)$.
}

We can then interpret Jensen's result as saying that, if the complexity of the surrounding universe $\V$ is below the strength of $0^\sharp$ then $L$ is a good candidate for $K$. In a complex universe we would therefore be looking for the core model among subuniverses more complex than $L$, and it turns out that also requiring strong covering to hold in such models is too much to ask; the current definition of covering has thus been weakened to the following.

\defi[][defi.weak-covering]{
  We say that \textbf{(weak) covering} holds for universes $\U\subset\V$ if $\cof^{\V}(\alpha^{+\U}) = \Card^{\V}(\alpha^{+\U})$ holds for any ordinal $\alpha$ with $\alpha^{+\U}\geq\aleph_2^{\V}$.
}

This statement might seem very distant from the strong version, but one can think of weak covering as saying that $\U$ ``knows'' the true cofinality of its successor cardinals $\kappa\geq\aleph_2^{\V}$ within the error margin $\varepsilon:=\kappa^{+\U}-\Card^{\V}(\kappa)$. More concretely, we could equivalently define weak covering as $\U$ containing all cofinal maps $f\colon\gamma\to\kappa$ in $\V$ for every $\gamma\in\Card^{\V}(\kappa)$, making it closer in spirit to the strong covering property.

\figx{weak-covering.jpg}{Weak covering property}

\qquad When it comes to $(ii)$ we have to define what we mean by ``resembling $L$''. Ultimately this boils down to the current working definition of a \textit{mouse} and is still work in progress. If our universe is no more complex than the strength of a Woodin cardinal however, then we know what the correct definition of a mouse is, and hence also what ``resembling $L$'' would mean in this context. The definition of mice along with the assumption of covering then turns out to imply that the core model will indeed inherit the large cardinal strength of the universe\footnote{To show this one first uses covering to show that $K$ is \textit{universal}, i.e. that it wins every coiteration. With universality at hand, a comparison argument with any $L[\vec E]$-model containing a large cardinal will then show that $K$ will have an inner model with the large cardinal in question.}.

\qquad To construct the core model one could then take a bottom-up approach, starting with $L$ and then carefully include the complexity of the universe while remaining similar to $L$\footnote{This is the strategy undertaken by Steel and Sargsyan.}. Alternatively, a top-down approach would be to define a structure which has \textit{all} the complexity of the universe, and then showing that this structure indeed exhibits these $L$-like properties\footnote{Woodin is pursuing this path.}.

\section{Constructing $K$}
The standard construction of $K$ takes the bottom-up approach. The first step towards this is the construction of $K^c$\footnote{The ``c'' stands for \textit{certified}, as the extenders we put on the sequence was historically called \textit{certified extenders}.}, which we build by recursion on the ordinals. We start with $K^c_0 := \emptyset$ and at every successor ordinal $\alpha$ we do one of two things:
\begin{enumerate}
  \item If there exists a ``nice'' extender indexed at $\alpha$ then we put it onto the extender sequence of $\core(K^c_\alpha$), where $\core(X)$ is the transitive collapse of a certain hull of $X$\footnote{Think of $\core(X)$ as ``removing the noise of $X$''.};
  \item Otherwise we let $K^c_\alpha := \J(\core(K^c_{\alpha-1}))$, with $\J(x):=\text{rud}(\trcl(x\cup\{x\}))$ being the usual operator we use to build $L$ with Jensen's hierarchy.\\
\end{enumerate}

In other words, we are essentially building $L$ with extenders attached onto it in a canonical fashion. Taking cores at every step will ensure that the initial segments will be \textit{sound}, which ultimately is what guarantees iterability of $K^c$. The fact that we put on all the relevant extenders from $V$ is what will ensure the covering property of the model. It turns out that $K^c$ isn't exactly what we want however, as it relies \textit{too much} on the surrounding universe, in contrast with $L$ whose construction procedure builds the exact same model in every universe. To attain this \textit{canonicity} we are again taking certain ``thick'' hulls of $K^c$ (again, think of it as removing the noise). The resulting construction \textit{almost} gives us what we want and is dubbed \textit{pseudo-K}. The problem with this is that the technicalities of the construction uses certain properties of a fixed cardinal $\Omega$, so to build the true core model we ``glue'' these pseudo-K's together.

\qquad The takeaway here is that whenever we're working with an initial segment of $K$ then that segment will be build using the recursive steps $(i)$ and $(ii)$ above, carefully including extenders from $V$.

\section{Properties of $K$}

In terms of applications of core model theory, the properties of $K$ is usually what matters. We touched on the weak covering property above, but for completeness we state most of the properties usually employed when working with $K$, here. In \cite{JensenSteel} they isolate a set of properties of $K$ which leads them to \textit{define} $K$ as the structure satisfying the conjunction of these properties. These are as follows.\footnote{See REF for definitions of premice, iteration trees and iteration strategies.}

\begin{enumerate}
  \item $K$ is a transitive proper class premouse satisfying \zfc;
  \item $K$ is $\Sigma_2$-definable;
  \item $K$ has a $\Sigma_2$-definable iteration strategy $\Sigma$;
  \item $K$ is \textbf{generically absolute}, meaning that $K^V=K^{V[g]}$ and $\Sigma^{V[g]}\restr V=\Sigma^V$ for any $V$-generic filter $g\subset\mathbb P$ for a set-sized forcing notion $\mathbb P$;
  \item $K$ is \textbf{inductively defined}, meaning that $K\l\omega_1^V$ is $\Sigma_1$-definable over $J_{\omega_1}(\mathbb R)$;
  \item $K$ satisfies \textbf{weak covering} as in Definition \ref{defi.weak-covering}.\\
\end{enumerate}

On top of these properties, we will also employ the following property, which is proven in Lemmata 7.3.7--7.3.9 and 8.3.4 in \cite{Zeman}.\todo{Check if Zeman is the author}

\qtheo[Zeman][theo.beaver]{
  Assume $0^\pistol$ does not exist. If $\mu$ is a countably complete weakly amenable $K$-measure then $\mu\in K$.
}

\end{document}


%% LyX 2.2.3 created this file.  For more info, see http://www.lyx.org/.
%% Do not edit unless you really know what you are doing.
\documentclass[12pt,english]{article}
\usepackage{mathptmx}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{geometry}
\geometry{verbose,tmargin=1in,bmargin=1in,lmargin=1in,rmargin=1in}
\usepackage{babel}
\usepackage[authoryear]{natbib}
\usepackage[unicode=true,pdfusetitle,
 bookmarks=true,bookmarksnumbered=false,bookmarksopen=false,
 breaklinks=false,pdfborder={0 0 0},pdfborderstyle={},backref=false,colorlinks=false]
 {hyperref}
\usepackage{breakurl}

\makeatletter

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% LyX specific LaTeX commands.
%% Special footnote code from the package 'stblftnt.sty'
%% Author: Robin Fairbairns -- Last revised Dec 13 1996
\let\SF@@footnote\footnote
\def\footnote{\ifx\protect\@typeset@protect
    \expandafter\SF@@footnote
  \else
    \expandafter\SF@gobble@opt
  \fi
}
\expandafter\def\csname SF@gobble@opt \endcsname{\@ifnextchar[%]
  \SF@gobble@twobracket
  \@gobble
}
\edef\SF@gobble@opt{\noexpand\protect
  \expandafter\noexpand\csname SF@gobble@opt \endcsname}
\def\SF@gobble@twobracket[#1]#2{}

\makeatother

\begin{document}

\title{Notes on Optimization Algorithms}
\maketitle

\subsection*{Nelder-Mead Simplex Algorithm}

The algorithm iterates on the following steps:\footnote{For more details, see \url{http://www.mathworks.com/help/techdoc/math/bsotu2d.html\#bsgpq6p-11}}
\begin{enumerate}
\item Create a simplex of dimension $K+1$ (where there are $K$ parameters
to be estimated) around the starting value $x_{0}$.
\item Modify the simplex based on how the function looks at different trial
modifications. In particular, ``order the points in the simplex from
lowest function value $f(x(1))$ to highest $f(x(K+1))$. At each
step in the iteration, the algorithm discards the current worst point
$x(K+1)$, and accepts another point into the simplex'' (Matlab website).
Possible simplex modifications are:

\begin{enumerate}
\item Reflect
\item Expand
\item Contract outside
\item Contract inside
\item Shrink
\end{enumerate}
\item Repeat steps 1-2 until the diameter of the simplex is smaller than
the tolerance (default is $10^{-4}$)
\end{enumerate}

\subsection*{BFGS Medium-Scale algorithm: line search \textemdash{} No
user-supplied gradient or hessian}

The medium-scale algorithm iterates on the following four steps for
the $k$th iteration:
\begin{enumerate}
\item compute the search direction $p_{k}$ by solving
\begin{eqnarray*}
\hat{H}_{k}p_{k} & = & -\nabla f\left(x_{k}\right),\,\,\mbox{or}\\
p_{k} & = & -\hat{H}_{k}^{-1}\nabla f\left(x_{k}\right)
\end{eqnarray*}
where $\hat{H}_{k}$ is an approximation of the hessian.
\item Choose the step size $\alpha_{k}$ by solving
\[
\min_{\alpha}f\left(x_{k}+\alpha_{k}p_{k}\right)
\]
\item Update $x_{k+1}=x_{k}+\alpha_{k}p_{k}$
\item Update $\hat{H}_{k+1}$, letting $s_{k}=\alpha_{k}p_{k}$ and $y_{k}=\nabla f\left(x_{k+1}\right)-\nabla f\left(x_{k}\right)$:
\[
\hat{H}_{k+1}=\hat{H}_{k}+\frac{y_{k}y_{k}^{\prime}}{y_{k}^{\prime}s_{k}}-\frac{\hat{H}_{k}s_{k}s_{k}^{\prime}\hat{H}_{k}}{s_{k}^{\prime}\hat{H}_{k}s_{k}}
\]
where $^{\prime}$ refers to matrix transpose, not differentiation.
\item Continue steps 1-4 until $\left\Vert \nabla f\left(x_{k}\right)\right\Vert <\mbox{tolerance}$,
where $\mbox{tolerance}=10^{-6}$ by default.
\end{enumerate}

\subsection*{BFGS Large-Scale algorithm: trust region method \textemdash{} user-supplied
gradient and/or hessian}

The trust region method operates by dividing the optimization into
a series of 2-dimensional trust region subproblems. ``The basic idea
is to approximate $f$ with a simpler function $q$, which reasonably
reflects the behavior of function $f$ in a neighborhood $N$ around
the point $x$. This neighborhood is the \textbf{trust region}''
(Matlab website, emphasis added). The algorithm iterates on the following
steps:
\begin{enumerate}
\item Divide the problem into a series of 2-dimensional trust-region subproblems
\item Find the optimal step size $s$ by solving the following equation:
\[
\min_{s}\frac{1}{2}s^{\prime}Hs+s^{\prime}g\,\,\mbox{subject to}\,\,\left\Vert Ds\right\Vert <\Delta
\]
where $H$ is the hessian, $g$ is the gradient, $D$ is a diagonal
scaling matrix, and $\Delta$ is a positive scalar. $\left\Vert \cdot\right\Vert $
is the 2-norm.
\item If $f\left(x+s\right)<f\left(x\right)$ then $x_{k+1}=x_{k}+s$. If
not, then adjust $\Delta$.
\item Continue steps 1-3 until $\left\Vert \nabla f\left(x_{k}\right)\right\Vert <\mbox{tolerance}$,
or $f\left(x_{k+1}\right)-f\left(x_{k}\right)<\mbox{tolerance}$,
or $s<\mbox{tolerance}$(each $10^{-6}$ by default).
\end{enumerate}

\end{document}