Let us start by presenting a subset of the functional core of \OCaml
(previously known as \emph{Objective Caml}). Contrary to previous
chapters, the following introduction will be more formal, relying on a
systematic classification of the syntactic constructs and their
semantics. The reader is advised to keep in mind that what follows is
a very concise and limited introduction to \OCaml, with the aim to
show what a semi\hyp{}formal definition of a programming language
looks like. Afterwards, we will show how \OCaml can be used to parse
other languages, presenting along the way the basic theoretical
concepts borrowed from compiler construction and formal language
theory.

A \emph{sentence}, or global definition, is defined by the following
cases, where~\(e\) denotes an expression, \(x\)~and~\(f\) are
variables and \Xlet and~\Xrec are keywords:

\bigskip

\noindent
\begin{tabular}{rll}
    $\bullet$
  & \emph{global definition}
  & \phrase{$\Xlet \,\, x = e$}\\
    $\bullet$
  & \emph{global recursive definition}
  & \phrase{$\Xlet \,\, \Xrec \,\, f = e$}
\end{tabular}

\bigskip

Global definitions can be optionally followed by two semicolons: these
are necessary when inputting the sentences in the toplevel loop
(prompted after running the command \texttt{ocaml} from a Unix
shell). In contrast with \Erlang, note that the keyword \Xrec is
necessary to enable recursion. A program is a sequence of
sentences. When we write that \(e\)~is an expression, we mean that
\(e\)~denotes a part of a sentence which is classified as an
expression according to its syntax. We say that \(e\)~is a
\emph{metavariable} because, being a name, it is a variable, but that
variable does not belong to the language being described (\OCaml) and,
instead, exists only in the descriptive language. In other words, it
is not an \OCaml program but a notation to refer to fragments of
\OCaml programs, perhaps an infinity. For instance, the metavariable
\(x\)~denotes a set of \OCaml variables and we must not confuse it
with the \OCaml variable~\ident{x}. Similarly, the metavariable
\(e_1\)~denotes an infinity of expressions.

\paragraph{Expressions}

An expression \(e\)~is recursively defined by the following cases:

\bigskip

\noindent
\begin{tabular}{rll}
    $\bullet$
  & \emph{variable}
  & $f, g, h$ (function); $x, y, z$ (other). \\
    $\bullet$
  & \emph{function} (or \emph{abstraction})
  & $\Xfun \,\, x \rightarrow e$\\
    $\bullet$
  & \emph{call} (or \emph{application})
  & $e_1 \,\, e_2$ \\
    $\bullet$
  & \emph{arithmetic operator}
  & \lpar\texttt{+}\rpar \ \lpar\texttt{-}\rpar \ \lpar\texttt{/}\rpar
    \ \lpar\texttt{*}\rpar\\
    $\bullet$
  & \emph{arithmetic operation}
  & $e_1$ \texttt{+} $e_2$ or $e_1$ \texttt{-} $e_2$
    or $e_1$ \texttt{/} $e_2$ or $e_1$ \texttt{*} $e_2$\\
    $\bullet$
  & \emph{unit or integer constant}
  & \unit \ or \textsf{0} or \textsf{1} or \textsf{2} etc.\\
    $\bullet$
  & \emph{parentheses}
  & $\lpar{e}\rpar$\\
    $\bullet$
  & \emph{local definition}
  & $\Xlet \,\, x = e_1 \,\, \Xin \,\, e_2$
\end{tabular}

\bigskip

Note that what we print nicely as `\(\rightarrow\)' is actually
written \texttt{->} in the source code. Here is an example of a
program, where the keywords are set in bold for greater clarity:
\begin{alltt}
\textbf{let} x = 0
\textbf{let} id = \textbf{fun} x \(\rightarrow\) x
\textbf{let} y = 2 \textbf{in} id y
\textbf{let} x = (\textbf{fun} x \(\rightarrow\) \textbf{fun} y \(\rightarrow\) x + y) 1 2
\textbf{let} z = x+1
\end{alltt}
\noindent Let us remark the following:
\begin{itemize}

  \item Variables must start with a lowercase character.

  \item The arrow is right-associative, so the expression
    \begin{equation*}
      \Xfun \,\, x_1 \rightarrow \Xfun \,\, x_2 \rightarrow \ldots
      \rightarrow \Xfun \,\, x_n \rightarrow e
    \end{equation*}
    is equivalent to
     \(\Xfun \,\, x_1 \rightarrow (\Xfun \,\, x_2 \rightarrow
     (\ldots \rightarrow (\Xfun \,\, x_n \rightarrow e)) \ldots)\).

  \item Function calls are left\hyp{}associative, so the
  expression \(e_1 \, e_2 \, e_3 \, \ldots \, e_n\) is equivalent
  to \((((\ldots (e_1 \, e_2) \, e_3) \ldots) \, e_n)\).

  \item Function calls have higher priority than operator calls, for
  example, \(f \, 3 + 4\) is equivalent to \((f \, 3) + 4\).

  \item Operator calls have higher syntactic priority than
  abstractions, for instance, \(\Xfun \, x \rightarrow x + y \) is
  equivalent to \(\Xfun \, x \rightarrow (x + y)\).

\end{itemize}
A program, that is, a series of global definitions, can always be
rewritten into a single sentence by means of nested local
definitions. The previous program is thus equivalent to
\begin{alltt}
\textbf{let} x = 0 \textbf{in}
  \textbf{let} id = \textbf{fun} x \(\rightarrow\) x \textbf{in}
  \textbf{let} _ = \textbf{let} y = 2 \textbf{in} id y \textbf{in}
  \textbf{let} x = (\textbf{fun} x \(\rightarrow\) \textbf{fun} y \(\rightarrow\) x + y) 1 2
\textbf{in} x+1
\end{alltt}
The symbol `\texttt{\_}' denotes a variable whose name is globally
unique, that is, it is different from any other, but purposefully
omitted. Without loss of generality, we will consider programs reduced
to a single expression. The meaning of a program is
the \emph{evaluation} of \texttt{x+1}.

\section*{Abstract syntax}

As with any programming language, before considering the evaluation of
\OCaml, we must specify the notion of \emph{scope of variables}, that
is to say, what is denoted by a given variable at a particular
location in the source file. With this aim, a graphical representation
of programs (let us remember that expressions are enough) as trees
proves very handy.
\begin{center}
\begin{tabular}{l|c}
\toprule
  \multicolumn{1}{c}{Expression}
& \multicolumn{1}{c}{Tree}\\
\toprule
    $x$
  & $x$\\
    $\Xfun \,\, x \rightarrow e$
  & \includegraphics[bb=72 697 93 721]{fun_tree}\\
    $e_1 \,\, e_2$
  & \includegraphics[bb=72 695 101 720]{app_tree}\\
\bottomrule
\end{tabular}
\end{center}
Intuitively, the method for constructing abstract syntax trees
consists in first parenthesising fully the expression making up the
program. Each parenthesis corresponds to a subexpression and each
subexpression corresponds to a subtree. The tree is built from the
root, down to the leaves, by traversing subexpressions from the
outermost to the innermost, that is, the most embedded ones. For
instance, consider
\fig~\vref{fig:abs_trees}.
\begin{figure}
\centering
\subfloat[\texttt{(1+2)*(5/1)}]{\includegraphics{arith_tree1}}
\qquad
\subfloat[\texttt{let x = 1 in (1+2)*(5/1)}]{%
  \includegraphics{arith_tree2}}
\qquad
\subfloat[\texttt{let x = 1 in ((let x = 2 in x) + x)}]{
  \includegraphics[bb=71 658 160 721]{arith_tree3}
}
\caption{Concrete and abstract syntax\label{fig:abs_trees}}
\end{figure}
Again, note that $x$~and~\texttt{x} are different: the former is a
metavariable, standing for \emph{any} \OCaml variable, whilst the
latter is a particular \OCaml variable.

\paragraph{Static scoping and environment}

A sentence associates an expression~\(e\) to a variable~\(x\): we
speak of \emph{binding}, which we write \(x \mapsto e\). A subprogram
then defines a set of bindings called an \emph{environment}. A binding
is \emph{static} if we can determine at compile\hyp{}time, that is, by
examining the source code, what expression a given variable refers
to. For instance, in
\begin{alltt}
\textbf{let} x = 0 \textbf{in}
  \textbf{let} id = \textbf{fun} x \(\rightarrow\) x \textbf{in}
  \textbf{let} y = id x \textbf{in}
  \textbf{let} x = (\textbf{fun} x \(\rightarrow\) \textbf{fun} y \(\rightarrow\) x + y) 1 2
\textbf{in} x+1
\end{alltt}
the variable~\texttt{x} in the expression \texttt{x+1} denotes the
expression bound to the variable~\texttt{x} in the previous line, not
the first. Bindings are ordered in the environment \emph{by the order
of their definitions}. Thus
\begin{enumerate}

  \item the environment is initially empty: \{\};

  \item with \texttt{\textbf{let} x = 0 \textbf{in} \ldots}, it
    becomes $\{\ident{x} \mapsto 0\}$;

  \item with \texttt{\textbf{let} id = \textbf{fun} x $\rightarrow$ x
    \textbf{in} \ldots}, it is
    $\{\ident{id} \mapsto \Xfun \,\, \ident{x} \rightarrow \ident{x}; \ident{x} \mapsto
    0\}$;

  \item \texttt{\textbf{let} y = id x \textbf{in} \ldots} yields
    $\{\ident{y} \mapsto \ident{id} \, \ident{x}; \ident{id} \mapsto \Xfun \,\, \ident{x} \rightarrow \ident{x}; \ident{x} \mapsto
    0\}$;

  \item with \texttt{\textbf{let} x = \ldots}, it is
      $\{\underline{\ident{x} \mapsto \ldots}; \ident{y} \mapsto \ident{id} \, \ident{x}; \ident{id} \mapsto \Xfun \,\, \ident{x} \rightarrow \ident{x}; \underline{\ident{x} \mapsto
      0}\}$.

\end{enumerate}
The binding $\ident{x} \mapsto 0$ is therefore hidden, or out of
scope, in \texttt{x+1}. We will write~\(\rho(x)\) the first binding
of~\(x\) in the environment~\(\rho\), if any.

\paragraph{Free variables}

The local definition \Xlet $x$ = $e_1$ \Xin $e_2$ binds $e_1$~to~$x$,
which we write $x \mapsto e_1$, in~$e_2$. It may be the case that,
in~\(e_2\), another local definition binds the same variable. To
clearly understand that situation, we proceed as follows on the
abstract syntax tree of the program. From each variable occurrence, we
move up, towards the root. If we encounter a \Xlet binding that
variable, we create an oriented edge between the variable location and
that \Xlet, and we stop. Otherwise, if we reach the root (no such
\Xlet has been found), and the variable is said to be \emph{free} in the
expression. We note \(\mathcal{F}(e)\) the set of the variables free
in~\(e\). See for instance \fig~\vref{fig:free_var1}.
\begin{figure}
\centering
\subfloat[\texttt{\textbf{let} x = 1 \textbf{in} (\textbf{let} x = 2
  \textbf{in} x) + x}\label{fig:free_var1}]{
\includegraphics[bb=71 658 160 721]{free_var1}}
\qquad\qquad
\subfloat[\texttt{\textbf{fun} y \(\rightarrow\) x + (\textbf{fun} x \(\rightarrow\) x)
  y}\label{fig:free_var2}]{
  \includegraphics{free_var2}}
\caption{Graphical representation of bindings}
\end{figure}
A similar situation arises with functions. Indeed, in the body~\(e\)
of \Xfun $x \rightarrow e$, the parameter~\(x\) may shadow (hide)
another variable~\(x\) bound at an upper level in the tree. We must
then consider that \Xfun is a binder, just like \Xlet. See
\fig~\vref{fig:free_var2} for an example.

\paragraph{Closed expressions}

A \emph{closed} expression is an expression without free
variables. Only a closed program can be evaluated (executed). Indeed,
what would be the value of the program reduced to the simple
expression~\texttt{x}? That is why the first static analysis performed
by compilers consists in determining the variables which are free in
expressions. If the program is not closed, it is rejected. In the case
of~\texttt{x}, the \OCaml compiler prints
\begin{center}
\texttt{Unbound value x}
\end{center}
and stops. It is useful that this open expression is rejected at
compile\hyp{}time and does not cause a run\hyp{}time error.

\section{Evaluation}

The evaluation of an expression, therefore of a program, is a partial
function from expressions to \emph{values}. A partial function models
the fact that an evaluation may not terminate or can be interrupted
because of an error. The values~\(v\) of \OCaml are almost a strict
subset of the expressions, defined recursively by the following cases:

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{unit or integer constant}
  & \unit \ or \textsf{0} or \textsf{1} or \textsf{2} etc.\\
    $\bullet$
  & \emph{closure}
  & $\langle\Xfun \,\, x \rightarrow e, \rho\rangle$,\\
  && where $\rho$ is an environment.\\
  && For the operators: $\langle\lpar{\texttt{+}}\rpar, \rho\rangle$
  etc.
\end{tabular}

\bigskip

A \emph{closure} is a pair made of a function and an environment. This
means in particular that functions are values, that is, they can be
the result of the evaluation of an \OCaml program --~indeed, this is
the characteristic feature of a functional language.

In order to define the evaluation, we must modify the concept of
binding: a binding will now associate a variable to a \emph{value}
--~and not an expression anymore. Evaluation, and perhaps the
resulting value, will depend on the environment at the occurrence of
the expression. Let us note \((x \mapsto v) \oplus \rho\) the
environment resulting from extending the environment~\(\rho\) with the
binding \(x \mapsto v\), shadowing any other binding for~\(x\) that
might exist in~\(\rho\).

For each expression~\(e\), let us define an evaluation rule in an
environment~\(\rho\), yielding the value~\(v\), as follows:

\bigskip

\begin{tabular}{@{}rll@{}}
    $\bullet$
  & $x$
  & Look up the \emph{first} value bound to~\(x\) in~\(\rho\).\\[2mm]
    $\bullet$
  & $\Xfun \,\, x \rightarrow e$
  & The value is $\langle\Xfun \,\, x \rightarrow e, \rho\rangle$.\\[2mm]
    $\bullet$
  & \texttt{+} \ \texttt{-} \ \texttt{/} \ \texttt{*}
  & The value is $\langle\lpar\texttt{+}\rpar, \rho\rangle$ etc.\\[2mm]
    $\bullet$
  & $e_1$ \texttt{+} $e_2$ \ etc.
  & Evaluate $e_1$ and $e_2$ in $\rho$, and sum, etc.\\[2mm]
    $\bullet$
  & \unit \ or \textsf{0} or \textsf{1} or \textsf{2} etc.
  & The value is \unit or \textsf{0} or \textsf{1} etc.\\[2mm]
    $\bullet$
  & $\lpar{e}\rpar$
  & Evaluate $e$ into $v$ in $\rho$.\\[2mm]
    $\bullet$
  & $\Xlet \,\, x = e_1 \,\, \Xin \,\, e_2$
  & Evaluate $e_1$ into $v_1$ in $\rho$;\\
  &
  & evaluate $e_2$ into $v$ in $(x \mapsto v_1) \oplus \rho$.\\[2mm]
    $\bullet$
  & $e_1 \,\, e_2$
  & Evaluate $e_1$ and $e_2$ into $v_1$ and $v_2$ in $\rho$\\
  &
  & (order left unspecified);\\
  &
  & $v_1$ must be of the form $\langle\Xfun \,\, x \rightarrow
    e,\rho'\rangle$;\\
  &
  & evaluate $e$ in $(x \mapsto v_2) \oplus \rho'$:\\
  && the value is $v$.
\end{tabular}

\bigskip

\paragraph{Example}

The environment is initially empty: \(\rho = \{\}\). Let us evaluate
\begin{alltt}
\textbf{let} x = 0 \textbf{in}
  \textbf{let} id = \textbf{fun} x \(\rightarrow\) x \textbf{in}
  \textbf{let} y = id x \textbf{in}
  \textbf{let} x = (\textbf{fun} x \(\rightarrow\) \textbf{fun} y \(\rightarrow\) x + y) 1 2
\textbf{in} x+1
\end{alltt}
\begin{itemize}

\item The evaluation of \texttt{\textbf{let} x = 0 \textbf{in} \ldots}
  imposes the prior evaluation of~\texttt{0}, which, obviously,
  yields~\(0\). Then, we create the binding $\ident{x} \mapsto 0$, we
  add it to~\(\rho\), which results in $\{\ident{x} \mapsto 0\}$, and
  we evaluate the subexpression elided with this new environment.

\item The evaluation of \texttt{\textbf{let} id = \textbf{fun} x \(\rightarrow\) x
    \textbf{in} \ldots} is performed within the environment
  $\{\ident{x} \mapsto 0\}$. The value~\(v\) is then the closure
  $\langle\Xfun \,\, \ident{x} \rightarrow \ident{x}, \{\ident{x} \mapsto
  0\}\rangle$. We extend the current environment with
  $\ident{id} \mapsto v$ and we evaluate the subexpression with it.

\item The evaluation of \texttt{\textbf{let} y = id x \textbf{in}
    \ldots} is done within the environment $\{\ident{id} \mapsto
  \langle\Xfun \,\, \ident{x} \rightarrow \ident{x}, \{\ident{x}
  \mapsto 0\}\rangle; \ident{x} \mapsto 0\}$.
    \begin{itemize}

      \item We evaluate first (\texttt{id x}) in the current
        environment (we used parentheses for better readability). In
        order to do so, we evaluate~\texttt{id} and~\texttt{x}
        separately. These are both variables, thus we look them up in
        the environment to retrieve the first corresponding
        binding: \(\ident{id} \mapsto \langle\Xfun \,\, \ident{x} \rightarrow \ident{x}, \{\ident{x} \mapsto
        0\}\rangle\), and \(\texttt{x} \mapsto 0\). We need to
        evaluate~\texttt{x} in the environment $(\ident{x} \mapsto
        0) \oplus \{\ident{x} \mapsto 0\}$, which yields~\(0\).

      \item We create the binding $\ident{y} \mapsto 0$, we add it to
        the current environment and we evaluate the elided
        subexpression with the new environment.

    \end{itemize}

  \item The evaluation of \texttt{\textbf{let} x = (\textbf{fun} x \(\rightarrow\)
      \textbf{fun} y \(\rightarrow\) x + y) 1 2 \textbf{in} \ldots}
      uses the environment $\{\ident{y} \mapsto 0; \ident{id} \mapsto
    \langle\Xfun \,\, \ident{x} \rightarrow \ident{x}, \{\ident{x}
    \mapsto 0\}\rangle; \ident{x} \mapsto 0\}$.

\end{itemize}

\paragraph{Formal definition}

If we note \(\evalf{e}{\rho}\) the value obtained by evaluating the
expression~\(e\) in the environment~\(\rho\), then we can resume the
evaluation of \OCaml programs as follows:
\begin{align*}
\evalf{\overline{n}}{\rho} &= \dot{n}, \;\, \text{with
  $\overline{n}$ an \OCaml integer and $\dot{n} \in \mathbb{N}$};\\
\evalf{e_1 \texttt{+} \, e_2}{\rho} &= \evalf{e_1}{\rho} +
\evalf{e_2}{\rho},\;\, \text{etc.};\\
\evalf{\lpar{e}\rpar}{\rho} &= \evalf{e}{\rho};\\
\evalf{x}{\rho} &= \rho(x), \;\, \text{(the first binding of~$x$
  in~$\rho$)};\\
\evalf{\Xfun \,\, x \rightarrow e}{\rho} &= \langle\Xfun \,\, x
\rightarrow e, \rho\rangle;\\
\evalf{\Xlet \,\, x = e_1 \,\, \Xin \,\, e_2}{\rho} &= \evalf{e_2}{((x
  \mapsto \evalf{e_1}{\rho}) \oplus \rho)};\\
\evalf{e_1 \,\, e_2}{\rho} &= \evalf{e}{((x \mapsto \evalf{e_2}{\rho})
  \oplus \rho')},\\
&\phantom{=\llbracket} \textnormal{where} \ \ \evalf{e_1}{\rho} =
\langle\Xfun \,\, x \rightarrow e,\rho'\rangle.
\end{align*}
The evaluation consists in applying the equations from left to right,
until termination or an error occurs.

We establish directly that $\evalf{\lpar\Xfun \; x \rightarrow
  e_2\rpar \; e_1}{\rho} = \evalf{\Xlet \,\, x = e_1 \,\, \Xin \,\,
  e_2}{\rho}$, that is, the local binding is not necessary, at least
  as long as there is no type system.

\paragraph{Applications}

If closures are values, they can be the value of a function call. For
instance,
\begin{alltt}
\textbf{let} add = \textbf{fun} x \(\rightarrow\) \textbf{fun} y \(\rightarrow\) x + y \textbf{in}
  \textbf{let} incr = add 1
\textbf{in} incr 5
\end{alltt}
The call (\texttt{add 1}) is a \emph{partial application}, as opposed
to a \emph{complete application} like (\texttt{add 1 5}): a partial
application results in a closure. Like all applications, operations
can also be partially evaluated:
\begin{alltt}
\textbf{let} incr = (+) 1 \textbf{in} incr 5
\end{alltt}
where the parentheses in~\texttt{(+)} mean that the operator has to be
considered in prefix position, not infix, as usual.

\paragraph{Termination}

We can already compute with our tiny subset of \OCaml everything that
is computable with the underlying computer. For instance, we can write
the following non\hyp{}terminating program:
\begin{center}
\texttt{\textbf{let} omega = \textbf{fun} f \(\rightarrow\) f f \textbf{in} omega omega}
\end{center}
We have the evaluation
\begin{gather*}
\evalf{\texttt{\textbf{let} omega = \textbf{fun} f \(\rightarrow\) f f \textbf{in} omega omega}}{\rho}\\
\begin{align*}
&= \evalf{\texttt{omega omega}}{((\texttt{omega} \mapsto
\evalf{\texttt{\textbf{fun} f \(\rightarrow\) f f}}{\rho}) \oplus \rho)}\\
&= \evalf{\texttt{f f}}{((\texttt{\small f} \mapsto
    \langle\texttt{\textbf{fun} f \(\rightarrow\) f f},\rho\rangle) \oplus \rho)}\\
&= \text{\emph{idem.}}
\end{align*}
\end{gather*}

\paragraph{Recursion}

To demonstrate the expressive power of \OCaml, let us see how we can
simulate recursive function by means of the auto\hyp{}applicative
function \texttt{omega}. First, let us define a function \texttt{fix},
traditionally called the \emph{Y combinator}:
\begin{alltt}
\textbf{let} omega = \textbf{fun} f \(\rightarrow\) f f
\textbf{let} fix = \textbf{fun} g \(\rightarrow\) omega (\textbf{fun} h \(\rightarrow\) \textbf{fun} x \(\rightarrow\) g (h h) x)
\end{alltt}

\noindent It is tedious to show that, for all function~\(f\) and
variable~\(x\),
\begin{equation*}
\evalf{\lpar\ident{fix} \; f\rpar{} \, x}{\rho} = \evalf{f \;
  \lpar\ident{fix} \; f\rpar{} \, x}{\rho}.
\end{equation*}
In other words, we have $\lpar\ident{fix} \; f\rpar{} \equiv f \;
\lpar\ident{fix} \; f\rpar$. Moreover, by definition, the fixed
point~\(p\) of a function~\(f\) satisfies the equation \(p =
f(p)\). Therefore, the fixed point of a function~\ident{f}, \emph{if
  it exists}, is the value of \lpar\ident{fix} \ident{f}\rpar{}.

It is possible to define the semantics (the evaluation) of a family of
fixed\hyp{}point operators (and not just one as previously) by
declaring that such operators must satisfy
\begin{equation*}
\evalf{\ident{fix} \,\, e}{\rho} = \evalf{e_1}{(f \mapsto
  \evalf{\ident{fix} \, \lpar\Xfun \, f \rightarrow e_1\rpar}{\rho}
  \oplus \rho')},
\end{equation*}
where $\evalf{e}{\rho} = \langle\Xfun \, f \rightarrow
e_1,\rho'\rangle$.

Let us consider the following definitions:
\begin{alltt}
\textbf{let} pre\_fact =
  \textbf{fun} f \(\rightarrow\) \textbf{fun} n \(\rightarrow\) \textbf{if} n=1 \textbf{then} 1 \textbf{else} n * f(n-1)
\textbf{let} fact = fix pre\_fact
\end{alltt}
We see that \texttt{fact} is the fixed point of \texttt{pre\_fact}, if
it exists, that is,
\begin{align*}
\evalf{\texttt{fact}}{\rho} &= \evalf{\texttt{pre\_fact fact}}{\rho}\\
&= \evalf{\texttt{\textbf{fun} n \(\rightarrow\) \textbf{if} n=1 \textbf{then} 1 \textbf{else} n * fact(n-1)}}{\rho}
\end{align*}
Therefore, \texttt{fact} is the factorial function. We can then
predefine the fixed\hyp{}point operator \texttt{fix} (not necessarily
the one we used above) and allow the programmer to use it directly,
but, instead, we are going to extend our description of the subset of
\OCaml with a native recursive binder:
\begin{equation*}
\evalf{\Xlet \; \Xrec \; f = e_1 \, \Xin \, e_2}{\rho} =
\evalf{\Xlet \; f = \textsf{fix} \, \lpar\Xfun \, f \rightarrow
e_1\rpar \, \Xin \, e_2}{\rho}.
\end{equation*}

\section*{Extensions}

Let us add the following expressions:

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{Boolean constants}
  & \Xtrue{} or \Xfalse\\
    $\bullet$
  & \emph{Boolean operators}
  & \texttt{(\&\&)} or \texttt{(||)} or \textsf{\textbf{not}}\\
    $\bullet$
  & \emph{$n$-tuple}
  & $e_1, \ldots, e_n$\\
    $\bullet$
  & \emph{conditional}
  & \Xif{} $e_0$ \Xthen{} $e_1$ \Xelse{} $e_2$\\
    $\bullet$
  & \emph{local recursive binding}
  & \Xlet{} \Xrec{} $f$ = $e_1$ \Xin{} $e_2$
\end{tabular}

\bigskip

\noindent Note that parentheses are recommended around tuples.

\paragraph{Irrefutable patterns}

Let us distinguish the variables occurring after \Xlet{} and \Xfun{},
because they are \emph{irrefutable patterns}, noted $\ipat{p}$:

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{function}
  & $\Xfun \,\, \ipat{p} \rightarrow e$\\
    $\bullet$
  & \emph{local definition}
  & \(\Xlet \,\, \textrm{[}\Xrec\textrm{]} \,\, \ipat{p} = e_1 \,\,
    \Xin \,\, e_2\)
\end{tabular}

\bigskip

\noindent An irrefutable pattern $\ipat{p}$ is defined recursively by
the following cases:

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{variable}
  & $f, g, h$ (functions) and $x, y, z$ (others). \\
    $\bullet$
  & \emph{unit}
  & \unit\\
    $\bullet$
  & \emph{$n$-tuple}
  & $\ipat{p}_1, \ldots, \ipat{p}_n$\\
    $\bullet$
  & \emph{parentheses}
  & $\lpar\ipat{p}\rpar$\\
    $\bullet$
  & \emph{underscore}
  & {\Large \_}
\end{tabular}

\bigskip

Note that, from the syntactic point of view, irrefutable patterns are
special cases of expressions, except the underscore, which is a
special case avoiding the creation of a binding.

\paragraph{Additional rules for expressions}

The comma takes priority over the arrow: $(\Xfun \,\, x \rightarrow x,
y)$ is equivalent to $\Xfun \,\, x \rightarrow (x,y)$. In order to
alleviate the notation
$\Xfun \,\, \ipat{p}_1 \rightarrow \ldots \rightarrow \Xfun \,\, \ipat{p}_n \rightarrow
e$, we define the equivalent constructs:

    \medskip

    \begin{tabular}{rll}
      $\bullet$
      & \phrase{$\Xlet \,\,
        \textrm{[}\Xrec\textrm{]} \,\, f
        = \Xfun \,\, \ipat{p}_1 \; \ipat{p}_2 \; \ldots \; \ipat{p}_n \rightarrow e$}
      & (new expression)\\
        $\bullet$
      & \phrase{$\Xlet \,\,
        \textrm{[}\Xrec\textrm{]} \,\, f
        \,\, \ipat{p}_1 \; \ipat{p}_2 \; \ldots \; \ipat{p}_n = e$}
      & (new sentence)
    \end{tabular}

    \medskip

\noindent For example,  we would write
\begin{alltt}
\textbf{let} mult = \textbf{fun} x y \(\rightarrow\) x * y \textbf{in}
  \textbf{let} eq x y = (x = y) \textbf{in}
  \textbf{let rec} fact n = \textbf{if} eq n 1 \textbf{then} 1 \textbf{else} mult n (fact(n-1))
\textbf{in} fact 9
\end{alltt}
We also extend the syntax to alleviate certain expressions. By
definition,
\begin{equation*}
\Xlet \,\, \ipat{p}_1 = e_1 \,\,
    \Xand \,\, \ipat{p}_2 = e_2 \, \ldots \,\,
    \Xand \,\, \ipat{p}_n = e_n \,\, \Xin \,\, e
\end{equation*}
is equivalent to
\begin{equation*}
\Xlet \,\, \ipat{p}_1, \ldots, \ipat{p}_n = e_1, \ldots, e_n \,\, \Xin \,\, e.
\end{equation*}
Similarly, let us introduce mutually recursive definitions:
\begin{equation*}
\Xlet \,\, \Xrec \,\, \ipat{p}_1 = e_1 \,\,
    \Xand \,\, \ipat{p}_2 = e_2 \, \ldots \,\, \Xand \,\, \ipat{p}_n =
      e_n \,\, \Xin \,\, e.
\end{equation*}
Furthermore, the sentence~\(e\) is equivalent to \(\Xlet \,\, \_ =
e\). Let us consider the case where the irrefutable patterns are
variables:
\begin{equation*}
\Xlet \,\, x = e_1 \,\, \Xand \,\, y = e_2 \,\, \Xin \,\,
e, \; \text{where} \, x \neq y.
\end{equation*}
If~\(x\) is free in~\(e_2\), that is, $x \in \mathcal{F} (e_2)$, then
the previous construct is defined as being equivalent to
\begin{center}
\begin{minipage}{0.3\linewidth}
\begin{tabbing}
\underline{\Xlet} \= \underline{$z$ \equal{} $x$ \Xin}\\
\> \Xlet{} $x$ \equal{} $e_1$ \Xin\\
\> \Xlet{} $y$ \equal{} \underline{\Xlet{} $x$ \equal{} $z$ \Xin} $e_2$\\
\Xin $e$
\end{tabbing}
\end{minipage}
\end{center}
where \(z \not\in \mathcal{F}(e_1) \cup \mathcal{F}(e_2) \cup
\mathcal{F}(e)\), in order to avoid being captured by~\(e_1\),
\(e_2\)~or~\(e\). It is thus not an elementary construct.

The multiple \Xlet{} \Xrec{} (with \Xand) can always be reduced to a
simple \Xlet{} \Xrec{} (with \Xin) by parameterising one of the
definitions by the other. Consider the following sentence
\begin{equation*}
\Xlet \; \Xrec \; x \; \equal \; e_1 \; \Xand \; y \; \equal \; e_2 \;
\Xin \; e
\end{equation*}
where $x \neq y$. It is equivalent, by definition, to
\begin{center}
\begin{minipage}{0.3\linewidth}
\begin{tabbing}
\Xlet{} \= \Xrec{} $x$ \equal{} \underline{\Xfun{} $y$
  $\rightarrow$} $e_1$ \Xin\\
\> \Xlet{} \Xrec{} $y$ \equal{} \underline{\Xlet{} $x$ \equal{} $x \,
y$ \Xin} $e_2$ \Xin\\
\> \underline{\Xlet{} $x$ \equal{} $x \, y$}\\
\underline{\Xin} $e$
\end{tabbing}
\end{minipage}
\end{center}
Therefore, it is not an elementary construct either.

The addition of new expressions to the language forces us to extend
the values as well, which are now defined by

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{unit or constants}
  & \unit \ or \textsf{0} or \Xtrue{} etc.\\
    $\bullet$
  & \emph{closure}
  & $\langle\Xfun \,\, x \rightarrow e, \rho\rangle$, where~$\rho$ is
    an environment.\\
  &
  & For the operators: $\langle\lpar{\texttt{+}}\rpar, \rho\rangle$
    etc.\\
    $\bullet$
  & \emph{$n$-tuple}
  & $v_1, \ldots, v_n$
\end{tabular}

\bigskip

\paragraph{Curryfied functions}

A function is \emph{curryfied} --~after the name of the logician
Curry~-- if it returns a closure, thus enabling partial
applications. By the way, let us not forget that an \OCaml function
always takes exactly one argument: if we wish to simultaneously pass
multiple values, we have to employ a data structure, for example, a
tuple. In the following, the sentences are input one by one into
the \OCaml toplevel, which runs on top of a virtual machine, like
the \Erlang shell does, therefore they are terminated with a special
token, two semicolons:

\bigskip

\noindent\topin{\textbf{let} add x y = x + y}

\noindent\topout{\textbf{val} add~:~int $\rightarrow$ int $\rightarrow$ int}

\noindent\topin{\textbf{let} add' (x,y) = x + y}

\noindent\topout{\textbf{val} add'~:~int $\times$ int $\rightarrow$ int}

\bigskip

\noindent The function \texttt{add} is curryfied and \texttt{add'} is
not.

Earlier, we saw a static analysis which yields the variables that are
free in a given expression. Moreover, the evaluation of a closed
expression cannot fail for lack of a necessary binding. Most compilers
reject open programs, but, by doing so, they sometimes reject innocent
programs, like \Xif{} \Xtrue{} \Xthen{} \num{1} \Xelse{} \ident{x},
where \ident{x} is unbound. In order to accept this kind of example,
the compiler should be able to predict the control flow, here, which
branch of the conditional is used for all possible evaluations. In the
case above, this is trivial, but, in general, the problem is
undecidable.

\section{Types}

Types are terms associated to expressions, either by the programmer or
the compiler, and they summarise some invariant properties which can
be automatically composed and checked. A type~$t$ is a term defined
recursively by the following cases:

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{simple}
  & \textsf{char}, \textsf{bool}, \textsf{int}, \textsf{string},
    \textsf{float}, \textsf{unit}\\
    $\bullet$
  & \emph{Cartesian product}
  & $t_1 \times \ldots \times t_n$\\
    $\bullet$
  & \emph{functional}
  & $t_1 \rightarrow t_2$\\
    $\bullet$
  & \emph{parenthesised}
  & $(t)$\\
    $\bullet$
  & \emph{free variable}
  & $\alpha$, $\beta$, $\gamma$ etc. \\
    $\bullet$
  & \emph{parameterised}
  & $\alpha$ \textsf{list}
\end{tabular}

\bigskip

Until now, we have not encountered values of the type \textsf{float},
\textsf{char} or \textsf{string}, but they are fairly obvious.

We write $\times$, $\alpha$, $\beta$ etc. what is
written \textsf{*}, \textsf{'a}, \textsf{'b} etc. in \textsc{ascii}.

The Cartesian product takes multiple arguments, instead of being
binary like in mathematics, because $(\times)$~is not associative
in \OCaml: $t_1 \times t_2 \times t_3 \neq (t_1 \times t_2) \times
t_3 \neq t_1 \times (t_2 \times t_3)$.

The arrow is used in types too, where it is also
right\hyp{}associative: $t_1 \rightarrow
t_2 \rightarrow \ldots \rightarrow t_n$ is equivalent to
$t_1 \rightarrow (t_2 \rightarrow (\ldots (t_{n-1} \rightarrow
t_n)) \ldots)$.

The Cartesian product has priority over the arrow: $t_1 \times
t_2 \rightarrow t_3$ is equivalent to $(t_1 \times t_2) \rightarrow
t_3$.

The \OCaml compiler associates a type to each expression in a program:
this is called \emph{static type inference}. For the simple constants,
we have
\begin{center}
\begin{tabular}{l|l|c}
\toprule
\multicolumn{1}{c}{Type} & \multicolumn{1}{c}{Values}
& \multicolumn{1}{c}{Some functions}\\
\midrule
    \textsf{unit}
  & \unit
  & \\
    \textsf{bool}
  & \Xtrue \ \Xfalse
  & \texttt{\&\&} \ \texttt{||} \ \texttt{not} \\
    \textsf{int}
  & \textsf{1} \, \textsf{2} \ \textsf{max\_int} \ etc.
  & \texttt{+} \ \texttt{-} \ \texttt{*} \ \texttt{/} \ etc.\\
    \textsf{float}
  & \textsf{1.0} \, \textsf{2.} \ \textsf{1e4} \ etc.
  & \texttt{+.} \ \texttt{-.} \ \texttt{*.} \ \texttt{/.} \
    \texttt{cos} \ etc.\\
    \textsf{char}
  & \textsf{'a'} \ \textsf{'}\verb+\+\textsf{n'} \
    \textsf{'}\verb+\+\textsf{097'} \ etc.
  & \textsf{Char.code} \, \textsf{Char.chr} \ etc.\\
    \textsf{string}
  & \textsf{"a}\verb+\+\textsf{tb}\verb+\+\textsf{010c}\verb+\+\textsf{n"}
    \ etc.
  & \verb+(^)+ \, \textsf{s.[i]} \ etc.\\
\bottomrule
\end{tabular}
\end{center}
Operations on floating point numbers are written differently from
their equivalent counterparts on integers. Moreover, we extend the
syntax of the sentences to allow the binding of a type to a name,
or \emph{type aliasing}, just as we have aliasing on expressions:

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{type binding}
  & \(\textsf{type} \,\, q = t\)\\
    $\bullet$
  & \emph{recursive types}
  & $\textsf{type} \,\, q_1 = t_1 \,\, \textrm{[}\Xand
     \,\, q_2 = t_2 \,\, \ldots\textrm{]}$\\
    $\bullet$
  & \emph{type variable}
  & $q$
\end{tabular}

\bigskip

\noindent We can now write, for instance, the following:
\begin{alltt}
\textbf{type} abscissa = float
\textbf{type} ordinate = float
\textbf{type} point = abscissa \(\times\) ordinate
\end{alltt}

\paragraph{Type inference}

Tuples contain data of the same type and their arity is part of their
type, for instance, the pair \textsf{(1,2)} has type \textsf{int}
$\times$ \textsf{int}, and the triplet \textsf{(1,2,3)} has type
\textsf{int} $\times$ \textsf{int} $\times$ \textsf{int}, thus
are incompatible. Consider also the following different types:

\bigskip

\noindent\topin{\textbf{let} middle x y = (x+y)/2}

\noindent\topout{\textbf{val} middle~:~int $\rightarrow$ int $\rightarrow$ int}

\noindent\topin{\textbf{let} middle (x,y) = (x+y)/2}

\noindent\topout{\textbf{val} middle~:~int $\times$ int $\rightarrow$ int}

\paragraph{Polymorphism}

Projections are polymorphic on tuples of same arity, so \Xfun $(x, y,
z) \rightarrow x$ has type $(\alpha \times \beta \times \gamma)
\rightarrow \alpha$. Consider the session:

\bigskip

\noindent\topin{\textbf{let} compose f g = \textbf{fun} x \(\rightarrow\) f (g x)}

\noindent\topout{\textbf{val} compose~: $(\alpha \rightarrow \beta) \rightarrow
  (\gamma \rightarrow \alpha) \rightarrow \gamma \rightarrow \beta =
  \topclos$}
\begin{alltt}
\textbf{\# let rec} power f n = \textbf{if} n \(\leqslant\) 0 \textbf{then fun} x \(\rightarrow\) x
                                \textbf{else} compose f (power f (n-1));;
\end{alltt}
\vspace{-4mm}
\noindent\topout{\textbf{val} power~:~($\alpha \rightarrow \alpha$)
  $\rightarrow$ int $\rightarrow$ ($\alpha \rightarrow \alpha$) =
  $\topclos$}

\bigskip

\noindent The type of the function \textsf{compose} is inferred as follows:
\begin{itemize}

  \item the first argument~\textsf{f} is an arbitrary function, hence
    of type \(\alpha \rightarrow \beta\);

  \item the second argument~\textsf{g} is a function whose result is
    the argument of~\textsf{f}, therefore of type~\(\alpha\);

  \item the domain of~\textsf{g} is arbitrary, hence \textsf{g}~has
    type \(\gamma \rightarrow \alpha\);

  \item the function \textsf{compose} has an argument~\textsf{x} which
    is passed to~\textsf{g}, hence has type~\(\gamma\); finally, the
    result of \textsf{compose} is returned by~\textsf{f}, hence of
    type~\(\beta\).

\end{itemize}
\noindent The equality operator in \OCaml is polymorphic and thus is
built in:

\noindent\topin{(=)}

\noindent\topout{- : $\alpha \rightarrow \alpha \rightarrow \texttt{bool} = \topclos$}

\bigskip

\noindent We must be very careful that it coincides with our
intended notion of equality, because the compiler will always accept
it on non\hyp{}functional values. It consists in the syntactical
equality: two values are equal if they have the same structure and if
their respective parts are equal, recursively. In particular, it will
not apply to functions --~this is different from
\Erlang, where the abstract syntax trees of the functions are compared
at run\hyp{}time.

\bigskip

\noindent\topin{1 = 1 \&\& [1;2] = [1;2] \&\& \str{yes} = \str{yes}}

\noindent\topout{- : bool = true}

\noindent\topin{(\textbf{fun} x \(\rightarrow\) x) = (\textbf{fun} x \(\rightarrow\) x)}

\error{Exception: Invalid\_argument "equal: functional value".}

%We note \texttt{<>} the negation of equality.

\section{Pattern matching}

Let us extend further the expressions with \emph{matchings} against
\emph{patterns}:
\begin{center}
  \Xmatch{} $e$ \Xwith{} $p_1 \rightarrow e_1 \mid \ldots \mid
    p_n \rightarrow e_n$, where the~$p_i$ are patterns.
\end{center}

\noindent Patterns are recursively defined by the following cases:

\bigskip

\begin{tabular}{rll}
    $\bullet$
  & \emph{variable}
  & $f, g, h$ (functions) and $x, y, z$ (others). \\
    $\bullet$
  & \emph{unit or constant}
  & \unit \ or \textsf{0} or \Xtrue{} etc.\\
    $\bullet$
  & \emph{$n$-tuple}
  & $p_1, \ldots, p_n$\\
    $\bullet$
  & \emph{parentheses}
  & $\lpar{p}\rpar$\\
    $\bullet$
  & \emph{underscore}
  & {\Large \_}
\end{tabular}

\bigskip

\noindent Note that irrefutable patterns are patterns. Typically,
pattern matching is used to define functions case by case, as commonly
done in mathematics. For example,
\begin{alltt}
\textbf{let rec} fib n = \textbf{match} n \textbf{with} 0 \(\rightarrow\) 1
                           | 1 \(\rightarrow\) 1
                           | \_ \(\rightarrow\) fib(n-1) + fib(n-2)
\end{alltt}
As in mathematics, the cases are ordered by the order of writing and
the previous definition is read as follows: `If the value
of~\texttt{n} has the shape~\texttt{0}, then \texttt{fib(n)} has the
value~\texttt{1}; else, if the value of~\texttt{b} has the
shape~\texttt{1}, then \texttt{fib(n)} has the value~\texttt{1};
otherwise, \texttt{fib(n)} has the value of \texttt{fib(n-1) +
fib(n-2)}.' What is the meaning of the relation `\(v\)~has the
shape~\(p\)'?  Or, equivalently, `\(p\)~matches~\(v\)'?
\begin{itemize}

  \item A constant, including \unit, has the shape of itself in a
    pattern;

  \item a \(n\)-tuple has the shape of a \(n\)-tuple in a pattern;

  \item any value has the shape of a variable in a pattern or
    of the underscore `{\Large \_}'.

\end{itemize}
Note that patterns do not match closures, which means that the~\(e\)
in `\Xmatch{} $e$ \Xwith{}' must not have a closure as a value. In the
case of constants, including \unit, matching coincides with equality.

Matching is the evaluation of a \Xmatch construct. Informally, the
evaluation of `\Xmatch{} $e$ \Xwith{} $p_1 \rightarrow e_1 \mid
\ldots \mid p_n \rightarrow e_n$' begins with that of~$e$
in~$v$. Next, \(v\)~is compared to the patterns~\(p_i\) in the order
of writing. If~\(p_i\) is the first pattern to match~\(v\), then the
value of the matching is the value of~\(e_i\). For instance, here is
the curryfied definition of the logical disjunction:
\begin{alltt}
\textbf{let} disj = \textbf{fun} (x,y) \(\rightarrow\) \textbf{match} (x,y) \textbf{with} (\textbf{false}, \textbf{false}) \(\rightarrow\) \textbf{false}
                                       | \_ \(\rightarrow\) \textbf{true}
\end{alltt}

\paragraph{Variant types}

Variant types are a generalisation of enumerations. For instance,
Boolean values can be (re)defined as
\begin{alltt}
\textbf{type} boolean = True | False
\textbf{let} t = True \textbf{and} f = False
\end{alltt}
Note that data constructors, like \texttt{True} or
\texttt{False}, must begin with an uppercase letter. Pattern matching
enables the examination of the values of a variant type:
\begin{alltt}
\textbf{let} int\_of\_boolean b = \textbf{match} b \textbf{with} True  \(\rightarrow\) 1 | False \(\rightarrow\) 0
\end{alltt}
Data constructors can also carry information beyond their mere name,
for instance, a pack of playing cards can be defined as
\begin{alltt}
\textbf{type} card = Card \textbf{of} ordinary | Joker
\textbf{and} ordinary = suit \(\times\) face
\textbf{and} suit = Heart | Diamond | Spade | Club
\textbf{and} face = Ace | King | Queen | Jack | Simple \textbf{of} int
\end{alltt}

\noindent Let us define the cards and functions creating them:

\bigskip

\noindent\topin{\textbf{let} jack\_of\_spade = Card (Spade, Jack)}

\noindent\topout{\textbf{val} jack\_of\_spade : card = Card (Spade, Jack)}

\medskip

\noindent\topin{\textbf{let} card f c = Card (c,f)}

\noindent\topout{\textbf{val} card : face $\rightarrow$ suit $\rightarrow$ card
  = $\topclos$}

\medskip

\noindent\topin{\textbf{let} king = card King}

\noindent\topout{\textbf{val} king : suit $\rightarrow$ card = $\topclos$}

\begin{alltt}
\textbf{let} value c = \textbf{match} c \textbf{with} Card Ace        \(\rightarrow\) 14
                         | Card King       \(\rightarrow\) 13
                         | Card Queen      \(\rightarrow\) 12
                         | Card Jack       \(\rightarrow\) 11
                         | Card (Simple k) \(\rightarrow\) k
                         | Joker           \(\rightarrow\) 0
\end{alltt}
A pattern can capture several cases at once:
\begin{alltt}
\textbf{let} is\_smaller c = \textbf{match} c \textbf{with} Card (Simple \_) \(\rightarrow\) \textbf{true}
                              | \_ \(\rightarrow\) \textbf{false}
\end{alltt}

\paragraph{Incomplete patterns}

We have already seen that, when a value~\(v\) is matched with \(p_1
\rightarrow e_1 \mid \ldots \mid p_n \rightarrow e_n\), the
expression~\(e_i\) of the first pattern~\(p_i\) which matches~\(v\) is
evaluated in the current environment extended with the bindings
introduced by~\(p_i\), if any. The pattern matching is incomplete if
there exists at least a value which cannot be matched by any
pattern. In that case, a warning is printed at compile\hyp{}time
because it is strongly advised to avoid incomplete pattern matchings.

\bigskip

\noindent\topin{\textbf{let} simple c = \textbf{match} c \textbf{with} Card (\_, Simple k) $\rightarrow$ k}

\noindent\error{Characters 15-51}

\noindent\error{Warning: this pattern-matching is not exhaustive.}

\noindent\error{Here is an example of a value that is not matched:
  Joker}

\noindent\topout{\textbf{val} simple : card $\rightarrow$ int = $\topclos$}

\bigskip

\paragraph{Linearity}

Contrary to \Erlang, a variable cannot be bound twice or more in the
same pattern, because this would imply a default equality to be used
during pattern matching. Such a pattern is said to be
\emph{non\hyp{}linear}.

\medskip

\noindent\topin{\textbf{fun} (x,y) $\rightarrow$ \textbf{match} (x,y) \textbf{with} (Card z, z) $\rightarrow$ \textbf{true}}

\noindent\error{Characters 40-41}

\noindent\error{This variable is bound several times in this
  matching.}

\paragraph{Lists}

Lists can be defined as a polymorphic, recursive variant type:
\begin{alltt}
\textbf{type} \(\alpha\) mylist = Nil | Cons \textbf{of} \(\alpha\) \(\times\) \(\alpha\) mylist
\end{alltt}
The data constructors \texttt{Nil} and \texttt{Cons} are traditional
in the community of functional languages. The former denotes the empty
list; the latter represents a non\hyp{}empty list. A non\hyp{}empty
list is then modelled as a pair whose first component is an element
(of type~\(\alpha\)) of the list and the second component is the
remainder of the list, or a sublist (of
type \(\alpha\)~\texttt{mylist}). For instance:
\begin{alltt}
\textbf{let} empty= Nil
\textbf{let} singleton1 = Cons ('a', empty)
\textbf{let} singleton2 = Cons (7, empty)
\textbf{let} long = Cons (1, Cons (2, Cons (3, singleton2)))
\end{alltt}
By default, the \OCaml system predefines a
type \(\alpha\)~\texttt{list}, whose constructor for the empty list
is~\texttt{[]}, and those of the non\hyp{}empty list is~\texttt{::},
used in infix position. The function catenating two lists is also
predefined and noted~\texttt{(@)}. Note that it is \emph{not} a
constructor. Here are equivalent lists:
\begin{alltt}
\textbf{let} l = 1::(2::3::(4::[]))
\textbf{let} l = 1::2::3::4::[]
\textbf{let} l = [1;2;3;4]
\end{alltt}
The standard library supplies a module \textsf{List} with useful
functions on lists. For example, the reversal of a list:
\begin{alltt}
\textbf{let} \textbf{rec} reverse l = \textbf{match} l \textbf{with}
                      [] \(\rightarrow\) []
                    | h::l \(\rightarrow\) (reverse l) @ [h]
\end{alltt}
or, more efficiently, using an accumulator:
\begin{alltt}
\textbf{let} reverse l =
  \textbf{let rec} rev\_aux acc l = \textbf{match} l \textbf{with}
                            [] \(\rightarrow\) acc
                          | h::t \(\rightarrow\) rev\_aux (h::acc) t
\textbf{in} rev\_aux [] l
\end{alltt}

\section{Exceptions}

Exceptions are values of the same type \textsf{exn}. They behave in
the following manner.
\begin{itemize*}

  \item The predefined type \textsf{exn} is an \emph{open} variant
    type, that is, it is extensible;

  \item throwing an exception stops the current evaluation and returns
    an exceptional value of type \textsf{exn};

  \item an exception can be matched if the expression where it was
    raised is embedded in a construct `\textsf{\textbf{try}
    $e$ \textbf{with} $m$}', or if any pending function call was
    embedded in such a way.
    \begin{itemize*}

      \item If the evaluation of~\(e\) results in a normal value, that
        value is the value of the \textsf{\textbf{try}} block, without
        considering the patterns in~\(m\);

      \item otherwise, the exception is matched against the patterns
        in~\(m\); if one of those patterns, say~\(p_i\), matches the
        exception, then \(e_i\)~is evaluated, else the exception is
        propagated backwards in the control flow (call graph) until a
        handler matches it or the execution stops with an error. Note
        that the patterns for exceptions need not be complete, whence
        a source of potential problems because the programmer has to
        keep track of the set of exceptions potentially raised by a
        given expression. (Contrary to \Java, exceptions in \OCaml are
        no part of the type of functions, because this would be deemed
        cumbersome in the presence of higher\hyp{}order functions,
        like \textsf{List.map}.)

    \end{itemize*}

  \item It is possible to observe an exception, that is, to match it
    and reraise it: \texttt{\textbf{try} f x \textbf{with} Failure
    s \textbf{as} x $\rightarrow$ prerr\_string s; raise x}.

\end{itemize*}
Let us consider an example, the linear search in a list:
\begin{alltt}
\textbf{exception} Not_found

\textbf{let rec} find key l =
  \textbf{match} l \textbf{with}
    (h,v)::t \(\rightarrow\) \textbf{if} h = key \textbf{then} v \textbf{else} find key t
  |       [] \(\rightarrow\) raise Not_found

\textbf{let} k = \textbf{try}
          find "Louis" [("Georges",14); ("Louis",5)]
        \textbf{with} Not_found \(\rightarrow\) 10
\end{alltt}

\noindent Formally, we write

\bigskip

\begin{tabular}{ll}
\toprule
Definition (sentence) & \phrase{\textbf{exception} $C$ [\textbf{of} $t$]}\\
\midrule
Raising (expression) & \phrase{raise $e$}\\
Matching (expression) & \phrase{\textbf{try} $e$ \textbf{with}
$p_1 \rightarrow e_1 \mid \ldots \mid p_n \rightarrow e_n$}\\
\bottomrule
\end{tabular}

\bigskip

\noindent Notice the analogy with the pattern matching on
  non\hyp{}exceptional values.


\newpage\section{Tricks and Lemmas}

	\theo{https://en.wikipedia.org/wiki/Minkowski's_theorem}{Minkowski's theorem}{Any convex set in $\R^n$, which is symmetric with respect to the origin and with volume greater than $2^n d(L)$ contains a non-zero lattice point.}

	\subsection{Ad Hocs}

		\begin{enumerate}

			\item $x^2+1 = (x+i)(x-i)$ [USAMO 2014 P1]
			\item Add. Everything. Up.
			\item Send SHIT to the infinity.

		\end{enumerate}


\subsection{Decision Trees}
\label{decision_trees}

\noindent{\bf Description}
Decision trees (for classification) is a classifier that is considered
more interpretable than other statistical classifiers. This implementation
is well-suited to handle large-scale data and builds a (binary) decision
tree in parallel.\\

\noindent{\bf Usage}
\begin{tabbing}
\texttt{-f} \textit{path}/\texttt{decision-tree.dml -nvargs}
\=\texttt{X=}\textit{path}/\textit{file}
  \texttt{Y=}\textit{path}/\textit{file}
  \texttt{types=}\textit{path}/\textit{file}\\
\>\texttt{model=}\textit{path}/\textit{file}
  \texttt{bins=}\textit{int}
  \texttt{depth=}\textit{int}\\
\>\texttt{num\_leaf=}\textit{int}
  \texttt{num\_samples=}\textit{int}\\
\>\texttt{Log=}\textit{path}/\textit{file}
  \texttt{fmt=}\textit{csv}$\vert$\textit{text}
\end{tabbing}

\begin{tabbing}
\texttt{-f} \textit{path}/\texttt{decision-tree-predict.dml -nvargs}
\=\texttt{X=}\textit{path}/\textit{file}
  \texttt{Y=}\textit{path}/\textit{file}
  \texttt{model=}\textit{path}/\textit{file}\\
\>\texttt{fmt=}\textit{csv}$\vert$\textit{text}
  \texttt{accuracy=}\textit{path}/\textit{file}\\
\>\texttt{confusion=}\textit{path}/\textit{file}
  \texttt{predictions=}\textit{path}/\textit{file}
\end{tabbing}

\noindent{\bf Arguments}

\begin{itemize}
\item X: Location (on HDFS) to read the matrix of feature vectors;
each row constitutes one feature vector.
\item Y: Location (on HDFS) to read the one-column matrix of (categorical)
labels that correspond to feature vectors in X. Classes are assumed to be
contiguously labeled beginning from 1. Note that, this argument is optional
for prediction.
\item model: Location (on HDFS) that contains the learnt decision tree.
\item types: Location (on HDFS) that contains each feature's type. 1 denotes
continuous-valued (scale) and 2 denotes categorical.
\item bins (default: {\tt 50}): Number of thresholds to choose for each
continuous-valued feature (deterimined by equi-height binning).
\item depth (default: {\tt 10}): Maximum depth of the learnt decision tree.
\item num\_leaf (default: {\tt 1}): Parameter that controls pruning. The tree
is not expanded if a node receives less than num\_leaf training examples.
\item num\_samples (default: {\tt 10}): Parameter that decides when to switch
to in-memory building of subtrees. If a node $v$ receives less than num\_samples
training examples then this implementation switches to an in-memory subtree
building procedure to build the subtree under $v$ in its entirety.
\item Log: Location (on HDFS) that collects various useful metrics that indicate
training progress.
\item predictions: Location (on HDFS) to store predictions for a held-out test set.
Note that, this is an optional argument.
\item fmt (default: {\tt text}): Specifies the output format. Choice of
comma-separated values (csv) or as a sparse-matrix (text).
\item accuracy: Location (on HDFS) to store the testing accuracy from a
held-out test set during prediction. Note that, this is an optional argument.
\item confusion: Location (on HDFS) to store the confusion matrix
computed using a held-out test set. Note that, this is an optional
argument.
\end{itemize}

\noindent{\bf Details}

Decision trees (Breiman et al, 1984) are simple models of
classification that,  due to their structure,  are easy to
interpret. Given an example feature vector, each node in the learnt
tree runs a simple test on it. Based on the result of the test, the
example is either diverted to the left subtree or to the right
subtree. Once the example reaches a leaf, then the label stored at the
leaf is returned as the prediction for the example.

\par

Building a decision tree from a fully labeled training set entails
choosing appropriate tests for each internal node in the tree and this
is usually performed in a top-down manner. Choosing a test requires
first choosing a feature $j$ and depending on the type of $j$, either
a threshold $\sigma$, in case $j$ is continuous-valued, or a subset of
values $S \subseteq \text{Dom}(j)$ where $\text{Dom}(j)$ denotes
domain of $j$, in case it is categorical. For continuous-valued
features the test is thus of form $x_j < \sigma$ and for categorical
features it is of form $x_j \in S$, where $x_j$ denotes the $j^{th}$
feature value of feature vector $x$. One way to determine which test
to include, is to compare impurities of the subtrees induced by the
test and this implementation uses {\it Gini impurity}.

\par

The current implementation allows the user to specify the maximum
depth of the learnt tree using the argument {\it depth}. It also
allows for some automated pruning via the argument {\it num\_leaf}. If
a node receives $\leq$ {\it num\_leaf} training examples, then a leaf
is built in its place. Furthermore, for a continuous-valued feature
$j$ the number of candidate thresholds $\sigma$ to choose from is of
the order of the number of examples present in the training set. Since
for large-scale data this can result in a large number of candidate
thresholds, the user can limit this number via the arguments {\it
  bins} which controls the number of candidate thresholds considered
for each continuous-valued feature. For each continuous-valued
feature, the implementation computes an equi-height histogram to
generate one candidate threshold per equi-height bin. To determine the
best value subset to split on in the case of categorical features,
this implementation greedily includes values from the feature's domain
until the sum of the impurities of the subtrees induced stops
improving.

\par

Learning a decision tree on large-scale data has received  some
attention in the literature. The current implementation includes logic
for choosing tests for multiple nodes that belong to the same level in
the decision tree in parallel (breadth-first expansion) and for
building entire subtrees under multiple nodes in parallel (depth-first
subtree building). Empirically it has been demonstrated that it is
advantageous to perform breadth-first expansion for the nodes
belonging to the top levels of the tree and to perform depth-first
subtree building for nodes belonging to the lower levels of the tree
(Panda et al, 2009). The parameter {\it num\_samples} controls when we
switch to  depth-first subtree building. Any node in the decision tree
that receives $\leq$ {\it num\_samples} training examples, the subtree
under it is built in its entirety in one shot.

\par

{\it Description of the model}: The learnt decision tree is represented using a matrix that
contains at least 3 rows. Each column in the matrix contains the parameters relevant
to a single node in the tree. The $i^{th}$ column denotes the parameters for the $i^{th}$ node
whose left child is stored in the $2i^{th}$ column and right child is stored in the $2i+1^{th}$
column. Here is a brief description of what each row in the matrix contains:
\begin{itemize}
\item $1^{st}$ row: Contains the feature index of the feature that this node looks at if
the node is an internal node, otherwise -1.
\item $2^{nd}$ row: Contains the type of the feature that this node looks at if the node is
an internal node, otherwise the label this leaf node is supposed to predict.
1 denotes continuous-valued feature and 2 denotes categorical.
\item $3^{rd}$: Only applicable for internal nodes. Contains the threshold the example's
feature value is compared to if the feature chosen for this node is a continuous-valued feature.
If on the other hand, the feature chosen for this node is categorical then the size of the
subset of values is stored here.
\item $4^{th}$ row onwards: Only applicable in the case of internal nodes where the feature
chosen is a categorical feature. Rows $4, 5 \ldots$ depict the value subset
chosen for this node.
\end{itemize}
As an example, Figure \ref{dtree} shows a decision tree with $5$ nodes and its matrix
representation.

\begin{figure}
\begin{minipage}{0.3\linewidth}
\begin{center}
\begin{tikzpicture}
\node (labelleft) [draw,shape=circle,minimum size=16pt] at (0,0) {$2$};
\node (labelright) [draw,shape=circle,minimum size=16pt] at (1,0) {$1$};
\node (rootleft) [draw,shape=rectangle,minimum size=16pt] at (0.5,1) {$x_5 \in \{2,3\}$};
\node (rootlabel) [draw,shape=circle,minimum size=16pt] at (2.5,1) {$1$};
\node (root) [draw,shape=rectangle,minimum size=16pt] at (1.75,2) {$x_3 < 0.45$};

\draw[-latex] (root) -- (rootleft);
\draw[-latex] (root) -- (rootlabel);
\draw[-latex] (rootleft) -- (labelleft);
\draw[-latex] (rootleft) -- (labelright);

\end{tikzpicture}
\end{center}
\begin{center}
(a)
\end{center}
\end{minipage}
\hfill
\begin{minipage}{0.65\linewidth}
\begin{center}
\begin{tabular}{c|c|c|c|c|c|}
& Col 1 & Col 2 & Col 3 & Col 4 & Col 5\\
\hline
Row 1 & 3 & 5 & -1 & -1 & -1\\
\hline
Row 2 & 1 & 2 & 1 & 2 & 1\\
\hline
Row 3 & 0.45 & 2 &  &  & \\
\hline
Row 4 &  & 2 &  &  & \\
\hline
Row 5 &  & 3 &  &  & \\
\hline
\end{tabular}
\end{center}
\begin{center}
(b)
\end{center}
\end{minipage}
\caption{(a) An example tree and its (b) matrix representation. $x$ denotes an example and $x_j$ the $j^{th}$ feature's value in it.}
\label{dtree}
\end{figure}

\vspace{16pt}

\noindent{\bf Returns}

The matrix corresponding to the learnt model is written to a file in the format requested. See
details where the structure of the model matrix is
described. Depending on what arguments are provided during
invocation, decision-tree-predict.dml may compute one or more of
predictions,  accuracy and confusion matrix in the requested output format.
\\

\noindent{\bf Examples}
\begin{verbatim}
hadoop jar SystemML.jar -f decision-tree.dml -nvargs
                           X=/user/biadmin/X.mtx
                           Y=/user/biadmin/y.mtx
                           types=/user/biadmin/types.mtx
                           model=/user/biadmin/model.mtx
                           bins=50 depth=10 num_leaf=1
                           num_samples=250 fmt=csv
                           Log=/user/biadmin/accuracy.csv
\end{verbatim}

\begin{verbatim}
hadoop jar SystemML.jar -f decision-tree-predict.dml -nvargs
                           X=/user/biadmin/X.mtx
                           Y=/user/biadmin/y.mtx
                           model=/user/biadmin/model.mtx
                           fmt=csv
                           predictions=/user/biadmin/probabilities.csv
                           accuracy=/user/biadmin/accuracy.csv
                           confusion=/user/biadmin/confusion.csv
\end{verbatim}

\noindent{\bf References}

\begin{itemize}
\item B. Panda, J. Herbach, S. Basu, and R. Bayardo. \newblock{PLANET: massively parallel learning of tree ensembles with MapReduce}. In Proceedings of the VLDB Endowment, 2009.
\item L. Breiman, J. Friedman, R. Olshen, and C. Stone. \newblock{Classification and Regression Trees}. Wadsworth and Brooks, 1984.
\end{itemize}


\section{Let's Get the Atmosphere Moving}
In its current state, CLaUDE has a static planet. This means that the planet remains in place and does not move. However we know that planets move in orbit and more importantly, spin around
themselves. But before we start adding layers, let's talk about a term you will hear more often: numerical instability.

Numerical instability occurs when you first run the model. This is due to the nature of the equations. Nearly all equations are continuous, which means that they are always at work. However
when you start the model, the equations were not at work yet. It is as if you suddenly give a random meteor an atmosphere, place it in orbit around a star and don't touch i for a bit. You will
see that the whole system oscilates wildly as it adjusts to the sudden changes and eventually it will stabilise. Another term you might encounter is blow up, this occurs when when the model
suddenly no longer behaves like it should. This is most likely caused by mistakes in the code or incorrect paramter initialisation. Be wary of the existence of both factors, and do not dismiss
a model if it behaves weirdly as it has just started up.

\subsection{Equation of State and the Incompressible Atmosphere}
The equation of state relates one or more variables in a dynamical system (like the atmosphee) to another. The most common equation of state in the atmosphere is the ideal gas equation as
described by \autoref{eq:ideal gas} \cite{idealGas}. The symbols in that equation represent:

\begin{itemize}
    \item $p$: The gas pressure ($Pa$).
    \item $V$: The volume of the gas ($m^3$).
    \item $n$: The amount of moles\footnote{Mole is the amount of particles ($6.02214076 \cdot 10^{23}$) in a substance, where the average weight of one mole of particles in grams is about the
    same as the weight of one particle in atomic mass units ($u$)\cite{mole}} in the gas.
    \item $R$: The Gas constant, $8.3144621$ ($J(mol)^{-1}K$) \cite{idealGas}.
    \item $T$: The temperature opf the gas ($K$).
\end{itemize}

If we divide everything in \autoref{eq:ideal gas} by $V$ and set it to be unit (in this case, set it to be exactly $1 m^3$) we can add in the molar mass in both the top and bottom parts of the
division as show in \autoref{eq:gas unit}. We can then replace $\frac{nm}{V}$ by $\rho$ the density of the gas ($kgm^{-3}$) and $\frac{R}{m}$ by $R_s$ the specific gas constant (gas constant that varies per
gas in $J(mol)^{-1}K$) as shown in \autoref{eq:state gas}. the resulting equation is the equation of state that you get that most atmospheric physicists use when talking about the atmosphere \cite{simon}.

\begin{subequations}
    \begin{equation}
        pV = nRT
        \label{eq:ideal gas}
    \end{equation}
    \begin{equation}
        p = \frac{nR}{V}T = \frac{nmR}{Vm}T
        \label{eq:gas unit}
    \end{equation}
    \begin{equation}
        p = \rho R_sT
        \label{eq:state gas}
    \end{equation}
\end{subequations}

The pressure is quite important, as air moves from a high pressure point to a low pressure point. So if we know the density and the temperature, then we know the pressure and we can work out
where the air will be moving to (i.e. how the wind will blow). In our current model, we know the atmospheric temperature but we do not know the density. For simplicities sake, we will now assume
that the atmosphere is Incompressible, meaning that we have a constant density. Obviously we know that air can be compressed and hence our atmosphere can be compressed too but that is not
important enough to account for yet, especially considering the current complexity of our model.

The code that corresponds to this is quite simple, the only change that we need to make in \autoref{eq:state gas} is that we need to replace $T$ by $T_a$, the temperature of the atmosphere. As
$T_a$ is a matrix (known to programmers as a double array), $p$ will be a matrix as well. Now we only need to fill in some values. $\rho = 1.2$\cite{densityAir}, $R_s = 287$\cite{specificGasConstantAir}.

\subsection{The Primitive Equations and Geostrophy}
The primitive equations (also known as the momentum equations) is what makes the air move. It is actually kind of an injoke between physicists as they are called the primitive equations but
actually look quite complicated (and it says $fu$ at the end! \cite{simon}). The primitive equations are a set of equations dictating the direction in the $u$ and $v$ directions as shown in
\autoref{eq:primitive u} and \autoref{eq:primitive v}. We can make the equations simpler by using and approximation called geostrophy which means that we have no vertical motion, such that the
terms with $\omega$ in \autoref{eq:primitive u} and \autoref{eq:primitive v} become 0. We also assume that we are in a steady state, i.e. there is no acceleration which in turn means that the
whole middle part of the equations are $0$. Hence we are left with \autoref{eq:primitive u final} and \autoref{eq:primitive v final}.

\begin{subequations}
    \begin{equation}
        \frac{du}{dt} = \frac{\delta u}{\delta t} + u\frac{\delta u}{ \delta x} + v\frac{\delta u}{\delta v} + \omega\frac{\delta u}{\delta p} = -\frac{\delta \Phi}{\delta x} + fv
        \label{eq:primitive u}
    \end{equation}
    \begin{equation}
        \frac{dv}{dt} = \frac{\delta v}{\delta t} + u\frac{\delta v}{ \delta x} + v\frac{\delta v}{\delta v} + \omega\frac{\delta v}{\delta p} = -\frac{\delta \Phi}{\delta y} - fu
        \label{eq:primitive v}
    \end{equation}

    \begin{equation}
        0 = -\frac{\delta \Phi}{\delta x} + fv
        \label{eq:primitive u final}
    \end{equation}
    \begin{equation}
        0 = -\frac{\delta \Phi}{\delta y} - fu
        \label{eq:primitive v final}
    \end{equation}
\end{subequations}

\autoref{eq:primitive u final} can be split up into to parts, the $\frac{\delta \Phi}{\delta x}$ part (the gradient force) and the $fv$ part (the coriolis force). The same applies to
\autoref{eq:primitive v final}. Effectively we have a balance between the gradient and the coriolis force as shown in \autoref{eq:pu simple} and \autoref{eq:pv simple}. The symbols in both of
these equations are:

\begin{itemize}
    \item $\Phi$: The geopotential, potential (more explanation in \autoref{sec:potential}) of the planet's gravity field ($Jkg^{-1}$).
    \item $x$: The change in the East direction along the planet surface ($m$).
    \item $y$: The change in the North direction along the planet surface ($m$).
    \item $f$: The coriolis parameter as described by \autoref{eq:coriolis}, where $\Omega$ is the rotation rate of the planet (for Earth $7.2921 \cdot 10^{-5}$) ($rad \ s^{-1}$) and $\theta$ is the
    latitude \cite{coriolis}.
    \item $u$: The velocity in the latitude ($ms^{-1}$).
    \item $v$: The velocity in the longitude ($ms^{-1}$).
\end{itemize}

\begin{subequations}
    \begin{equation}
        f = 2\Omega\sin(\theta)
        \label{eq:coriolis}
    \end{equation}
    \begin{equation}
        \frac{\delta \Phi}{\delta x} = fv
        \label{eq:pu simple}
    \end{equation}
    \begin{equation}
        \frac{\delta \Phi}{\delta y} = -fu
        \label{eq:pv simple}
    \end{equation}
    \begin{equation}
        \frac{\delta p}{\rho \delta x} = fv
        \label{eq:pu simple final}
    \end{equation}
    \begin{equation}
        \frac{\delta p}{\rho \delta y} = -fu
        \label{eq:pv simple final}
    \end{equation}
\end{subequations}

Since we want to know how the atmosphere moves, we want to get the v and u components of the velocity vector (since $v$ and $u$ are the veolicites in longitude and latitude, if we combine them in a
vector we get the direction of the overall velocity). So it is time to start coding and calculating! If we look back at \autoref{alg:stream1v2}, we can see that we already have a double for loop.
In computer science, having multiple loops is generally considered a bad coding practice as you usually can just reuse the indices of the already existing loop, so you do not need to create a new
one. However this is a special case, since we are calculating new temperatures in the double for loop. If we then also would start to calculate the velocities then we would use new information
and old information at the same time. Since at index $i - 1$ the new temperature has already been calculated, but at the index $i + 1$ the old one is still there. So in order to fix that we need
a second double for loop to ensure that we always use the new temperatures. We display this specific loop in \autoref{alg:stream2}. Do note that everything in \autoref{alg:stream1v2} is still
defined and can still be used, but since we want to focus on the new code, we leave out the old code to keep it concise and to prevent clutter.

\begin{algorithm}[hbt]
    \SetAlgoLined

    \While{\texttt{TRUE}}{
        \For{$lat \in [-nlat, nlat]$}{
            \For{$lon \in [0, nlon]$}{
                $u[lat, lon] \leftarrow -\frac{p[lat + 1, lon] - p[lat - 1, lon]}{\delta y} \cdot \frac{1}{f[lat]\rho}$ \;
                $v[lat, lon] \leftarrow \frac{p[lat, lon + 1] - p[lat, lon - 1]}{\delta x[lat]} \cdot \frac{1}{f[lat]\rho}$ \;
            }
        }
    }
    \caption{The main loop of the velocity of the atmosphere calculations}
    \label{alg:stream2}
\end{algorithm}

The gradient calculation is done in \autoref{alg:gradient}. For this to work, we need the circumference of the planet. Herefore we need to assume that the planet is a sphere. While that is not
technically true, it makes little difference in practice and is good enough for our model. The equation for the circumference can be found in \autoref{eq:circumference} \cite{circumference},
where $r$ is the radius of the planet. Here we also use the f-plane approximation, where the coriolis paramter has one value for the northern hemisphere and one value for the southern hemisphere
\cite{fplane}.

\begin{equation}
    2 \pi r
    \label{eq:circumference}
\end{equation}

\begin{algorithm}
    \SetAlgoLined
    $C \leftarrow 2\pi R$ \;
    $\delta y \leftarrow \frac{C}{nlat}$ \;

    \For{$lat \in [-nlat, nlat]$}{
        $\delta x[lat] \leftarrow \delta y \cos(lat \cdot \frac{\pi}{180})$ \;

        \eIf{$lat < 0$}{
            $f[lat] \leftarrow -10^{-4}$ \;
        }{
            $f[lat] \leftarrow 10^{-4}$ \;
        }
    }
    \caption{Calculating the gradient $\delta x$}
    \label{alg:gradient}
\end{algorithm}

Because of the geometry of the planet and the construction of the longitude latitude grid, we run into some problems when calculating the gradient. Since the planet is not flat ("controversial
I know"\cite{simon}) whenever we reach the end of the longitude we need to loop around to get to the right spot to calculate the gradients (as the planet does not stop at the end of the
longitude line but loops around). So to fix that we use the modulus (mod) function which does the looping for us if we exceed the grid's boundaries. We do haveanother problem though, the poles.
As the latitude grows closer to the poles, they are converging on the center point of the pole. Looping around there is much more difficult so to fix it, we just do not consider that center
point in the main loop. The changed algorithm can be found in \autoref{alg:stream2v2}

\begin{algorithm}[hbt]
    \SetAlgoLined

    \While{\texttt{TRUE}}{
        \For{$lat \in [-nlat + 1, nlat - 1]$}{
            \For{$lon \in [0, nlon]$}{
                $u[lat, lon] \leftarrow -\frac{p[(lat + 1) \text{ mod } nlat, lon] - p[(lat -1) \text{ mod } nlat, lon]}{\delta y} \cdot \frac{1}{f[lat]\rho}$ \;
                $v[lat, lon] \leftarrow \frac{p[lat, (lon + 1) \text{ mod } nlon] - p[lat, (lon -1) \text{ mod } nlon]}{\delta x[lat]} \cdot \frac{1}{f[lat]\rho}$ \;
            }
        }
    }
    \caption{The main loop of the velocity of the atmosphere calculations}
    \label{alg:stream2v2}
\end{algorithm}

Do note that the pressure calculation is done between the temperature calculation in \autoref{alg:stream1v2} and the $u, v$ calculations in \autoref{alg:stream2v2}. At this point our model shows
a symmetric vortex around the sun that moves with the sun. This is not very realistic as you usually have convection and air flowing from warm to cold, but we do not have that complexity yet
(due to our single layer atmosphere).

\subsection{Introducing an Ocean}
Now we want to introduce an ocean, because most of the Earth is covered by oceans it plays quite an important role in atmospheric physics. To do this we need a new concept called albedo. Albedo
is basically the reflectiveness of a material (in our case the planet's surface) \cite{albedo}. The average albedo of the Earth is about 0.3. Now to add an ocean to the grid, we define a few
areas where the albedo differs. Where you do this does not really matter for the current complexity. Defining the oceans is as easy as hardcoding (what we computer scientists refer to when
setting parts of an array to be a specific value, where if you want to change the value you need to change it everywhere instead of doing it in a variable) the albedo value for the specific
regions as we do in \autoref{alg:albedo}. Water also takes longer to warm up, so let us change the specific heat capacity ($C_p$ in \autoref{alg:stream1v2}) from a constant to an array. The new
$C_p$ can also be found in \autoref{alg:albedo}, where we have made the specific heat capacity of water one order of magnitude (i.e. $10$ times) larger.

\begin{algorithm}[hbt]
    $a \leftarrow 0.5$ \;
    $a[5-55, 9-20] \leftarrow 0.2$ \;
    $a[23-50, 45-70] \leftarrow 0.2$ \;
    $a[2-30, 85-110] \leftarrow 0.2$ \;

    $C_p \leftarrow 10^7$ \;
    $C_p[5-55, 9-20] \leftarrow 10^8$ \;
    $C_p[23-50, 45-70] \leftarrow 10^8$ \;
    $C_p[2-30, 85-110] \leftarrow 10^8$ \;
    \caption{Defining the oceans}
    \label{alg:albedo}
\end{algorithm}

Now that we have that defined, we need to adjust the main loop of the program (\autoref{alg:stream1v2}). For clarity, all the defined constants have been left out. We need to add albedo into the
equation and change $C_p$ from a constant to an array. The algorithm after these changes can be found in \autoref{alg:stream2v3}. We multiply by $1 - a$ since albedo represents how much energy is
reflected instead of absorbed, where we need the amount that is absorbed which is exactly equal to $1$ minus the amount that is reflected.

\begin{algorithm}[hbt]
    \SetAlgoLined

    \While{\texttt{TRUE}}{
        \For{$lat \in [-nlat, nlat]$}{
            \For{$lon \in [0, nlot]$}{
                $T_p[lat, lon] \leftarrow T_p[lat, lon] + \frac{\delta t ((1 - a[lat, lon])S + 4\epsilon \sigma (T_a[lat, lon])^4 - 4\sigma (T_p[lat, lon])^4)}{C_p[lat, lon]}$ \;
                $T_a[lat, lon] \leftarrow T_a[lat, lon] + \frac{\delta t (\sigma (T_p[lat, lon])^4 - 2\epsilon\sigma (T_a[lat, lon])^4)}{C_a}$ \;
                $t \leftarrow t + \delta t$ \;
            }
        }
    }
    \caption{The main loop of the temperature calculations}
    \label{alg:stream2v3}
\end{algorithm}


\documentclass[main.tex]{subfiles}
\begin{document}

\subsection{Stationary SSD emission}

\marginpar{Tuesday\\ 2020-11-17, \\ compiled \\ \today}

Let us start by stating some assumption and definitions:
%
\begin{enumerate}
    \item \(\rho = \Sigma / H\);
    \item \(H = c_s (R^3 / GM)^{1/2} = R c_s / v_\phi \);
    \item \(c_s^2 = P / \rho \);
    \item \(P = P _{\text{gas}} + P _{\text{rad}} = k_B \rho T / (\mu m_p) + a T^{4} / 3\).
\end{enumerate}

From the center of the disk radiation can travel outward, however it will be optically thick: \(\tau = \int_{0}^{r} \alpha \dd{s} >1\).
We can approximate it as \(\tau \approx \kappa _R \rho H = \kappa _R \Sigma > 1\), where \(\kappa _R\) is the Rosseland mean opacity.

Radiative transport in a slab can be treated analytically in the diffusion approximation to yield the radiative flux
%
\begin{align}
F(z) = \frac{16 \sigma T^3}{3 \kappa _R \rho } \pdv{T}{z} = - \frac{4}{3} \frac{\sigma}{\kappa _R \rho } \pdv{(T^{4})}{z}
\,.
\end{align}

We can define the surface as the height at which \(\tau =1\).

The flux crossing the \(z = 0\) surface is
%
\begin{align}
F(0) \approx \frac{4}{3} \frac{\sigma }{\kappa _R \rho _c} \frac{T_c^{4}}{H} \approx \frac{4}{3} \frac{\sigma T_c^{4}}{\tau _c}
\,.
\end{align}

% \todo[inline]{not zero?}

The flux at the surface, on the other hand, is
%
\begin{align}
F(s) \approx \frac{4}{3} \frac{\sigma}{\tau _s} T_s^{4} \approx \frac{4}{3} \sigma T_s^{4}
\,.
\end{align}

So, their ratio is
%
\begin{align}
\frac{F(s)}{F(0)} = \qty(\frac{T_s}{T_c})^{4} \tau _c
\,.
\end{align}

We expect \(T_s < T_c\), as is natural if flux is going from inside to outside. Then, \((T_s / T_c)^{4} \ll 1\), so unless \(\tau _c\) is extremely large (and, as we will see, it is not) we get \(F(s) < F(0)\).
The difference \(F(0) - F(s) \approx F(0)\) corresponds to the produced energy \(D(R)\), so
%
\begin{align}
\frac{4}{3} \sigma \frac{T_c^{4}}{\tau _c} \approx D(R) = \frac{3 GM \dot{M}}{8 \pi R^3} \qty[1 - \qty(\frac{R _{\text{in}}}{R})^{1/2}]
\,.
\end{align}

This will be another assumption for us.
Also, we will use the relations
%
\begin{align}
\tau &= \kappa _R \Sigma  \\
\nu \Sigma &= \frac{\dot{M}}{3 \pi } \qty[1 - \qty(\frac{R _{\text{in}}}{R})^{1/2}]  \\
v_R &= - \frac{3 \nu }{2R} \qty[1 - \qty(\frac{R _{\text{in}}}{R})^{1/2}]^{-1}  \\
\kappa _R &= \kappa _R (\rho , T, \dots)  \\
\nu &= \nu  (\rho , T, \dots) = \alpha H c_s
\,.
\end{align}

Since we want to see what the emission will look like, we seek an explicit expression for the surface temperature:
%
\begin{align}
\sigma T_s^{4} &= D(R) = \frac{3 GM \dot{M}}{8 \pi R^3} \qty[1 - \qty(\frac{R _{\text{in}}}{R})^{1/2}]  \\
T_s &= \qty(\frac{3 GM \dot{M}}{8 \pi \sigma })^{1/4} R^{-3/4} \qty[1 - \qty(\frac{R _{\text{in}}}{R})^{1/2}]^{1/4}  \\
T_s &\approx \qty(\frac{3 GM \dot{M}}{8 \pi \sigma R _{\text{in}}^3 })^{1/4} \qty( \frac{R _{\text{in}}}{R})^{3/4}
\marginnote{If  \(R \gg R _{\text{in}}\).}
\label{eq:temperature-accretion-disk}
\,.
\end{align}

Then, we can see that the temperature decreases as a function of \(R\).

Let us define a typical surface temperature \(T _s^{\text{typical}}\) as
%
\begin{align}
T _s^{\text{typical}}
=
\qty(\frac{3 GM \dot{M}}{8 \pi \sigma R _{\text{in}}^3 })^{1/4}
\approx \SI{e7}{K} \qty(\frac{\dot{M}}{\SI{e17}{g /s}})^{1/4}
\qty(\frac{M}{M_{\odot}})^{1/4} \qty(\frac{R _{\text{in}}}{\SI{e6}{cm}})^{-3/4}
\,,
\end{align}
%
which yields soft \(X\)-rays, at around \SI{1}{keV}.
Using the complete formula, we find that the temperature is in the form
%
\begin{align}
T^{4} \propto \frac{1}{x^{3/4}} \qty(1 - \frac{1}{x^{1/2}})
\,,
\end{align}
%
where \(x = R / R _{\text{in}}\).
We can maximize this, and we find that the maximum temperature of the disk is found to be \(T _{\text{max}} \approx \num{.5} T^{\text{typical}}_s\).

There are several radiative processes taking place in an accretion disk:
\begin{enumerate}
    \item electron scattering;
    \item thermal free-free emission (bremsstrahlung).
\end{enumerate}

The latter is dominant, and the characteristic Rosseland mean opacity is
%
\begin{align}
\kappa _R^{\text{bremss}} \approx \num{6.6e22} \rho T^{-7/2} \SI{}{cm^2 g^{-1}}
\,.
\end{align}

For convenience, let us define
%
\begin{align}
f = 1 - \sqrt{ \frac{R _{\text{in}}}{R}}
\,.
\end{align}

We find an algebraic system with all the equations, and finally we get
%
\begin{align}
H = \SI{1.7e8}{cm} \times \alpha^{-1/10} (\dot{M}_{16})^{3/20} M^{-3/8} R_{10}^{9/8} f^{3/5}
\,,
\end{align}
%
from which we can confirm that \(R \gg H\).
Here, if a quantity has a subscript such as \(\dot{M}_{16}\), it means that we are normalizing the number to \SI{e16}{g/s}, or the appropriate cgs unit in general.
We can also calculate the central optical depth:
%
\begin{align}
\tau _c = 33 \alpha^{-4/5} \dot{M}_{16}^{1/5} f^{4/5}
\,.
\end{align}

Also, we find that \(T_s = T_c / 2\).

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Carlos Santos             %
% ECE 351-51                %
% Lab 10                    %
% 04/14/2020                %
%                           %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\documentclass[12pt]{article}

% Language and font encoding
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{caption}
\usepackage{float}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{rotating}
\usepackage{setspace}

% Sets page size and margins
\usepackage[a4paper,top=3cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}

% Useful packages
\usepackage[colorinlistoftodos]{todonotes}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}
\usepackage{listings}
\usepackage{gensymb}
\usepackage{graphicx} %package to manage images
\usepackage{listings}
\usepackage{color}

\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=Python,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

%Line Spacing
\setstretch{1.5}

%Info for Title Page
\title{ECE 351 Lab 10 Report \\ Section 52}
\date{April 14, 2020}
\author{Carlos Santos}
\begin{document}

%Make a Title Page
\vspace{\fill}
\maketitle
\vspace{\fill}
\clearpage

\maketitle
\tableofcontents

%Introduction
\section{Introduction}
This lab's purpose was focused on frequency tools and bode plots using Python.
%Equations
\section{Equations}

Transfer Function:
\begin{equation}
    H(s) = \frac{\frac{1}{RC}s}{s^2+\frac{1}{RC}s+\frac{1}{LC}}
\end{equation}

Magnitude:
\begin{equation}
    H|j\omega| = \frac{\frac{1}{RC}\omega}{\sqrt{\omega ^ 4 + [\frac{1}{RC}^2 - \frac{2}{LC}]\omega ^ 2}+\frac{1}{LC}^2}
\end{equation}

Phase:
\begin{equation}
    <H(J\omega) = \frac{\pi}{2} - tan^{-1}(\frac{\frac{1}{RC}\omega}{-\omega ^ 2 + \frac{1}{LC}})
\end{equation}

\begin{equation}
    x(t) = cos(2\pi*100t) + cos(2\pi*3024t) + sin(2\pi * 50000t)
\end{equation}

\begin{figure}[H]
\caption{RLC Circuit}
\centering
\includegraphics[width=.8\textwidth]{RLC_Circuit.png}
\end{figure}

%Methodology
\section{Methodology}
\begin{enumerate}
    \item First find the magnitude and phase equations for the transfer function equation.
    \item Turn the magnitude and phase equations into Python equations.
    \item Plot the magnitude and phase from $10^3 rad/s \leq \omega \leq 10^6 rad/s$
    \item Use scipy.signal.bode to plot the magnitude and phase on a Bode plot.
    \item Plot equation 4 from $0\leq t \leq 0.01s$
    \item Convert equation 4 into the Z-domian using scipy.signal.bilinear().
    \item Pass equation 4 through the filer using scipy.signal.lfilter().
    \item Lasly. plot the output signal y(t)
\end{enumerate}

%Results
\section{Results}

\begin{figure}[H]
\caption{Task 3.3.1}
\centering
\includegraphics[width=.8\textwidth]{hand_solved.png}
\end{figure}

\begin{figure}[H]
\caption{Task 3.3.2}
\centering
\includegraphics[width=.8\textwidth]{sig_bode.png}
\end{figure}

\begin{figure}[H]
\caption{Task 3.3.3}
\centering
\includegraphics[width=.8\textwidth]{log_plot.png}
\end{figure}

\begin{figure}[H]
\caption{Task 4.3.4}
\centering
\includegraphics[width=.8\textwidth]{signal_filtering.png}
\end{figure}

%Questions
\section{Questions}
\begin{enumerate}
    \item Explain how the filter and filtered output in Part 2 makes sense given the Bode plots from
Part 1. Discuss how the filter modifies specific frequency bands, in Hz.
    \begin{enumerate}
        \item By looking at the Bode plot it seems like our filter is a center-band filter. This means it keeps all the frequencies near the middle and discards the rest. This makes sense by looking at the filtered signal plot versus the unfiltered signal plot. It kept all the signals between -1 and 1. Any high frequency signals are going to be clipped.
    \end{enumerate}

    \item Discuss the purpose and workings of
scipy.signal.bilinear() and scipy.signal.lfilter().
    \begin{enumerate}
        \item The "bilinear()" function converts our equation over to the Z-domain. It returns two items which are the numerator and denominator. The "lfilter()" passes the signal through the defined filter and provides the output signal which is filtered.
    \end{enumerate}
    \item What happens if you use a different sampling frequency in scipy.signal.bilinear() than
you used for the time-domain signal?
    \begin{enumerate}
        \item If sampling frequency is smaller we will get less points on the graph. Likewise the higher the sampling frequency the more points on the graph.
    \end{enumerate}
    \item Leave any feedback on the clarity of lab tasks, expectations, and deliverables.
\end{enumerate}
    \begin{enumerate}
        \item N/A
    \end{enumerate}

%Conclusion
\section{Conclusion}
In conclusion Bode plots helps us visualize the characteristics of a filter. Generating those plots in Python proved to be a simple task given the proper libraries and functions. For Bode plots we need sig.bode(), con.TransferFunction(), and con.bode().

\end{document}


\documentclass[11pt]{article}

\usepackage{graphicx}
\usepackage{tcolorbox}

\title{\vspace{-5.0cm}DD2421 Machine Learning - Lab 1: Decision Trees \\ Python version}
\author{\"Orjan Ekeberg\\ Updated 2017 by Martin Hjelm \& Nils Bore }

\begin{document}
\maketitle

\section{Preparations}

In this lab you will use a set of predefined Python functions to
build and manipulate decision trees.  In order to run this, you
need to have Python installed.  We also use the Qt graphics library,
PyQt, for plotting. PyQt comes in different versions 4 or 5. For
Mac users version 5 is recommended.

For different Windows versions consult the Internet. For Mac use Homebrew
to install Python and PyQt5. For Ubuntu Python and PyQt are available in the
debian package repository. Python and PyQt are also installed on the Unix computers in
the computer halls.

\textbf{Note:} It is possible to do the lab without using the
plotting functions found in PyQt, but then you will not be able to see the generated
decision trees.

\section{MONK datasets}

This lab uses the artificial MONK dataset from the UC Irvine repository.
The MONK's problems are a collection of three binary classification
problems MONK-1, MONK-2 and MONK-3 over a six-attribute discrete domain.
The attributes \(a_1, a_2, a_3, a_4, a_5, a_6\) may take the following values:

\begin{center}
  \begin{tabular}{lll}
    \(a_1 \in \{1, 2, 3\}\) &
    \(a_2 \in \{1, 2, 3\}\) &
    \(a_3 \in \{1, 2\}\)\\
    \(a_4 \in \{1, 2, 3\}\) &
    \(a_5 \in \{1, 2, 3, 4\}\) &
    \(a_6 \in \{1, 2\}\)\\
  \end{tabular}
\end{center}
Consequently, there are 432 possible combinations of attribute values.
The \emph{true} concepts underlying each MONK's problem are given by
table \ref{tab:truemonk}.

\begin{tcolorbox}
\textbf{Assignment 0:}
Each one of the datasets has properties which makes them hard to learn.
Motivate which of the three problems is most difficult for a decision
tree algorithm to learn.
\end{tcolorbox}

\begin{table}
  \caption{True concepts behind the MONK datasets \label{tab:truemonk}}
  \begin{center}
    \begin{tabular}{|l|l|}
      \hline
      MONK-1 & \((a_1=a_2)\vee(a_5=1)\)\\
      % (attribute\_1 = attribute\_2) or (attribute\_5 = 1)
      \hline
      MONK-2 & \(a_i=1\) for exacly two \(i \in \{1, 2, \ldots, 6\}\)\\
      % (attribute\_n = 1) for EXACTLY TWO choices of n $\in \{1,2,...,6\}$
      \hline
      MONK-3 & \((a_5=1 \wedge a_4=1) \vee (a_5\ne 4 \wedge a_2\ne 3)\)\\
      %(attribute\_5  = 3 and attribute\_4  = 1) or\\
      %(attribute\_5 != 4 and attribute\_2 != 3)\\
      \hline
    \end{tabular}
  \end{center}
  MONK-3 has 5\% additional noise (misclassification) in the training set.
\end{table}

\begin{table}
  \caption{Characteristics of the three MONK datasets \label{tab:monk}}
  \begin{center}
    \begin{tabular}{|c|c|c|c|c|}\hline
      Name & \# train & \# test & \# attributes & \# classes\\ \hline \hline
      MONK-1 & 124 & 432 & 6 & 2\\ \hline
      MONK-2 & 169 & 432 & 6 & 2\\ \hline
      MONK-3 & 122 & 432 & 6 & 2\\ \hline
    \end{tabular}
  \end{center}
\end{table}

The data consists of three separate datasets MONK-1, MONK-2 and
MONK-3.  Each dataset is further divided into a training and test set,
where the first one is used for learning the decision tree, and the
second one to evaluate its classification accuracy (see table
\ref{tab:monk}).  The datasets are available in
the file \verb!monkdata.py!.  In particular, six variables are defined
which contain the datasets:
\verb!monk1!, \verb!monk1test!, \verb!monk2!,
\verb!monk2test!, \verb!monk3! and \verb!monk3test!.
Each dataset is a sequence (more precisely, a \emph{tuple}) of instances
of the class \texttt{Sample}, defined in the same file.

You can access the data in your own Python scripts by importing
the \texttt{monkdata.py} file as a module like this:
\begin{verbatim}
import monkdata as m
\end{verbatim}

This makes the variable \texttt{m} a shorthand for the module so that
you can access the datasets by writing \verb!m.monk1!, etc.

\section{Entropy}

In order to decide on which attribute to split, decision tree learning
algorithms such as ID3 and C4.5 use a statistical property called
\emph{information gain}.  It measures how well a particular attribute
distinguishes among different target classifications.  Information
gain is measured in terms of the expected reduction in the
\emph{entropy} or impurity of the data.  The entropy of an arbitrary
collection of examples is measured by
\begin{equation}
\textrm{Entropy}(S) = - \sum_i p_i \log_2 p_i
\label{eq:entropy}
\end{equation}
in which $p_i$ denotes the proportion of examples of class $i$ in $S$.
The monk dataset is a binary classification problem (class 0 or 1) and
therefore equation (\ref{eq:entropy}) simplifies to
\begin{equation}
\textrm{Entropy}(S) = - p_0 \log_2 p_0 - p_1 \log_2 p_1
\end{equation}
where $p_0$ and $p_1=1-p_0$ are the proportions of examples belonging to class
$0$ and $1$.\\[2ex]

\begin{tcolorbox}
\textbf{Assignment 1:} The file \verb!dtree.py! defines a function
\texttt{entropy} which calculates the entropy of a dataset.  Import
this file along with the monks datasets and use it to calculate the
entropy of the \emph{training} datasets.
\end{tcolorbox}

\begin{center}
  \begin{tabular*}{0.9\textwidth}{|c|c@{\extracolsep{\fill}}c|}
    \hline
    Dataset & Entropy & \\
    \hline\hline
    MONK-1 & & \\
    \hline
    MONK-2 & & \\
    \hline
    MONK-3 & & \\
    \hline
  \end{tabular*}
\end{center}

\begin{tcolorbox}
\textbf{Assignment 2:}
Explain entropy for a uniform distribution
%and a Gaussian distribution with high and low variance.
and a non-uniform distribution, present some example distributions with high and low entropy.
\end{tcolorbox}

\section{Information Gain}

The information gain measures the expected reduction in impurity
caused by partitioning the examples according to an attribute.
It thereby indicates the effectiveness of an attribute in classifying the
training data. The information gain of an attribute $A$, relative to
a collection of examples $S$ is defined as
\begin{equation}
\textrm{Gain}(S,A) = \textrm{Entropy}(S) -
 \sum_{k \in \textrm{values}(A)} \frac{|S_k|}{|S|} \textrm{Entropy}(S_k)
\end{equation}
where $S_k$ is the subset of examples in $S$ for the attribute $A$ has the value $k$.

\begin{tcolorbox}
\textbf{Assignment 3:} Use the function \texttt{averageGain} (defined
in \verb!dtree.py!)  to calculate the expected information gain
corresponding to each of the six attributes.  Note that the attributes
are represented as instances of the class Attribute (defined in
\verb!monkdata.py!) which you can access via \verb!m.attributes[0]!,
..., \verb!m.attributes[5]!. Based on the results, which attribute
should be used for splitting the examples at the root node?
\end{tcolorbox}

\begin{center}
  Information Gain\\[0.5ex]
  \begin{tabular*}{\textwidth}{|c@{\extracolsep{\fill}}|c|c|c|c|c|c|}
    \hline
    Dataset & $a_1$ & $a_2$ & $a_3$ & $a_4$ & $a_5$ & $a_6$ \\
    \hline
    \verb!MONK-1 ! & & & & & & \\
    \hline
    \verb!MONK-2 ! & & & & & & \\
    \hline
    \verb!MONK-3 ! & & & & & & \\
    \hline
  \end{tabular*}
\end{center}

\begin{tcolorbox}
\textbf{Assignment 4:}
For splitting we choose the attribute that maximizes the information gain, Eq.3.
Looking at Eq.3 how does the entropy of the subsets, $S_k$, look like when the
information gain is maximized? How can we motivate using the information gain
as a heuristic for picking an attribute for splitting? Think about reduction
in entropy after the split and what the entropy implies.
\end{tcolorbox}

\section{Building Decision Trees}

Split the \texttt{monk1} data into subsets according to the selected
attribute using the function \texttt{select} (again, defined in
\verb!dtree.py!)  and compute the information gains for the nodes on
the next level of the tree.  Which attributes should be tested for
these nodes?

For the \texttt{monk1} data draw the decision tree up to the first two
levels and assign the majority class of the subsets that resulted from
the two splits to the leaf nodes.  You can use the predefined function
\texttt{mostCommon} (in \verb!dtree.py!) to obtain the majority class
for a dataset.

Now compare your results with that of a predefined routine for ID3.
Use the function \verb!buildTree(data, m.attributes)! to build the
decision tree.  If you pass a third, optional, parameter to
\texttt{buildTree}, you can limit the depth of the generated tree.

You can use \texttt{print} to print the resulting tree in text form,
or use the function \texttt{drawTree} from the file \verb!drawtree_qt4.py!
or \verb!drawtree_qt5.py!, depending on your PyQt version, to draw a graphical
representation.

\begin{tcolorbox}
\textbf{Assignment 5:}
Build the full decision trees for all three Monk datasets using
\texttt{buildTree}.  Then, use the function \texttt{check} to measure the performance
of the decision tree on both the training and test datasets.

For example to built a tree for \texttt{monk1} and compute the performance on the test data
you could use
\begin{verbatim}
import monkdata as m
import dtree as d

t=d.buildTree(m.monk1, m.attributes);
print(d.check(t, m.monk1test))
\end{verbatim}

Compute the train and test set errors for the three Monk datasets for
the full trees. Were your assumptions about the datasets correct? Explain the
results you get for the training and test datasets.
\end{tcolorbox}

\begin{center}
  \begin{tabular*}{0.7\textwidth}{|c|@{\extracolsep{\fill}}c|c|}
    \hline
    & $E_\textrm{train}$ & $E_\textrm{test}$ \\
    \hline\hline
    \verb#MONK-1# & & \\
    \hline
    \verb#MONK-2# & & \\
    \hline
    \verb#MONK-3# & & \\
    \hline
  \end{tabular*}
\end{center}

\section{Pruning}
The idea of \emph{reduced error pruning} is to consider each node in
the tree as a candidate for removal.  A node is removed if the
resulting pruned tree performs at least as well as the original tree
over a separate \emph{validation dataset}, i.e. a dataset not used
during training.  When a node is removed, the subtree rooted at that
node is replaced by a leaf node, to which the majority classification
of examples in that node is assigned.

For the purpose of pruning, we have to split our original training
data into one training set for building the tree and one validation
set for pruning.  Notice, that using the test set for validation would
be cheating because we would then no longer be able to use the test
set for independently estimating the true error of our pruned decision
tree.  Instead, we will randomly partition the original training set into
training and validation set.  This can be done by defining a function
which randomly reorders the data samples and returns the first and second
parts separately:
\begin{verbatim}
import random

def partition(data, fraction):
    ldata = list(data)
    random.shuffle(ldata)
    breakPoint = int(len(ldata) * fraction)
    return ldata[:breakPoint], ldata[breakPoint:]

monk1train, monk1val = partition(m.monk1, 0.6)
\end{verbatim}

In the file \verb!dtree.py! there is a utility function \texttt{allPruned}
which returns a sequence of all possible ways a given tree can be pruned.

Write code which performs the complete pruning by repeatedly calling
\texttt{allPruned} and picking the tree which gives the best
classification performance on the validation dataset.  You should stop
pruning when all the pruned trees perform worse than the current
candidate.

\begin{tcolorbox}
\textbf{Assignment 6:}
Explain pruning from a bias variance trade-off perspective.
\end{tcolorbox}

\begin{tcolorbox}
\textbf{Assignment 7:} Evaluate the effect pruning has on the test
error for the \texttt{monk1} and \texttt{monk3} datasets, in
particular determine the optimal partition into training and pruning
by optimizing the parameter \texttt{fraction}.  Plot the
classification error on the test sets as a function of the parameter
\texttt{fraction} $\in \{0.3,0.4,0.5,0.6,0.7,0.8\}$. \\

Note that the split of the data is random. We therefore need to compute
the statistics over several runs of the split to be able to draw any
conclusions. Reasonable statistics includes mean and a measure of the spread.
Do remember to print axes labels, legends and data points as you will not pass without them.
\end{tcolorbox}

\end{document}


\documentclass[12pt]{article}
\usepackage[usenames]{color} %used for font color
\usepackage{amsmath, amssymb, amsthm}
\usepackage{wasysym}
\usepackage[utf8]{inputenc} %useful to type directly diacritic characters
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{float}
\usepackage{mathtools}
\usepackage [english]{babel}
\usepackage [autostyle, english = american]{csquotes}
\MakeOuterQuote{"}
\graphicspath{ {./} }
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\prob}{\mathbb{P}}
\newcommand{\degrees}{^{\circ}}
\DeclarePairedDelimiter\ceil{\lceil}{\rceil}
\DeclarePairedDelimiter\floor{\lfloor}{\rfloor}

\author{Tianshuang (Ethan) Qiu}
\begin{document}
\title{Math 74, Week 14}
\maketitle

\section{Mon Lec, 3a}
Let $a$ be the length of this rectangle that is opposite the wall, and $b$ be the length of the other side. So we have $a+2b \leq 36$, and we try to maximize $ab$. We can simplify the first equation to $a = 36 - 2b$
\newline
By AM-GM, we have that $\frac{2a+b}{2} \geq \sqrt{2ab}$, now since we know that $a+2b \leq 36$, we can subsitute that in.
$$18 \geq \frac{2a+b}{2} \geq \sqrt{2ab}$$
Thus the maximum the area $ab$ can be is $18^2/2 = 162$. Now we try to find an $a,b$ such that $ab = 162$.
Let $a = 9, b = 18$, and $ab=162$, achieving the maximum.

\section{Mon Lec, 5a}
We manipulate $2 \sqrt x > 3 - \frac{1}{x}$, and it is equivalent to showing
$$2 \sqrt{x} + \frac{1}{x} \geq 3$$
Then apply AM-GM to see that $\frac{\sqrt{x}+\sqrt{x}+1/x}{3} \geq
\sqrt[3]{\sqrt{x}\sqrt{x}1/x} = 1$. Rearranging this gives
$$2 \sqrt{x} + \frac{1}{x} \geq 3$$
Thus we have proven the statement.

\section{Mon Lec, 6}
We say two inequalities are equivalent when they are true and false at the same time.
\newline
In the first equation $(x-a)^2+1>0$, $(x-a)^2$ is non-negative, so the statement is always true.
$$4ax^2 + 4x + 1 > 0$$
$$(4a-4)x^2 + (2x+1)^2 > 0$$
This inequality holds true when $(4a-4)>0$, so the two inequalities are equivalent when $a > 1$
\newpage

\section{Mon Dis, 1a}
By AM-GM, $\frac{a+b}{2} \geq \sqrt{ab}$, $\frac{b+c}{2} \geq \sqrt{bc}$, etc.
\newline
We can multiply these equations to get
$$\frac{(a+b)(b+c)...(e+a)}{2^5} \geq \sqrt{a^2b^2c^2d^2e^2}$$
$$(a+b)(b+c)(c+d)(d+e)(e+a) \geq 32abcde$$

\section{Mon Dis, 1b}
By AM-CM, $(\sum_{n=1}^{2021} n) /2021 \geq \sqrt[2021]{2021!}$. We can apply the arithmetic series sum to the lhs:
$$\frac{(2021+1)2021}{2 \times 2021} \geq \sqrt[2021]{2021!}$$
$$(\frac{2022}{2}) ^ {2021} \geq 2021!$$
Thus it is proven.

\section{Mon Dis, 3c}
Let our box intersect the ellipsoid in octant 1 at $(x,y,z)$. Since it is on the ellipsoid we have $\frac{x^2}{a^2}+\frac{y^2}{b^2}+\frac{z^2}{c^2}=1$.
\newline
By AM-GM inequality we have $$(\frac{x^2}{a^2}+\frac{y^2}{b^2}+\frac{z^2}{c^2})/3 \geq \sqrt[3]{\frac{x^2}{a^2}\frac{y^2}{b^2}\frac{z^2}{c^2}}$$
$$\frac{1}{3} \geq \sqrt[3]{\frac{x^2y^2z^2}{a^2b^2c^2}}$$
$$\frac{1}{27} \geq \frac{x^2y^2z^2}{a^2b^2c^2}$$
Now the volume of our cube is simply $8xyz$ since each side is double the intersection point in octant 1.
$$\frac{1}{27a^2b^2c^2} \geq x^2y^2z^2$$
$$xyz \leq abc\sqrt{\frac{1}{27}}$$
$$8xyz \leq 8abc\sqrt{\frac{1}{27}}$$
Thus we have found the maximum value.
\end{document}