\title{Enumeration of the Building Game}
\author{
        Daniel Johnson \& Govind Menon\\
                Division of Applied Mathematics\\
        Brown University
}
\date{\today}

\documentclass[12pt]{article}

\usepackage{graphicx,amsmath,mathtools,bbm,amsthm,enumerate}
\usepackage{mathrsfs}
%\usepackage[subnum]{cases}

%\usepackage[titletoc,toc,title]{appendix}

\newtheorem{mythm}{Theorem}
\newtheorem{mylem}{Lemma}
\newtheorem{mycor}{Corollary}
\newtheorem{mydef}{Definition}

\newcommand{\colorA}{white}
\newcommand{\colorB}{black}
\newcommand{\colorAsm}{w}
\newcommand{\colorBsm}{b}
\newcommand{\poly}{$\mathscr{P}$}
\newcommand{\faceset}{F\left(\mathscr{P}\right)}
\newcommand{\spc}{ }
\newcommand{\xj}{$x^j$}
\newcommand{\xk}{$x^k$}
\newcommand{\Sjk}{$S_{jk}$}
\newcommand{\Skj}{$S_{kj}$}

\DeclareMathOperator{\diag}{diag}

\begin{document}
\maketitle

\begin{abstract}
The Building Game is a sequential coloring process on polyhedra. We enumerate the Building Game state space for all polyhedra in the Platonic, Archimedean, and Catalan solids classes of up to 30 faces. By putting a probability distribution on each step of the Building Game process, a distribution is induced on the entire state space. With the help of a finite group theoretic identity, we find the explicit form of these distributions. Finally, we examine the properties of the resulting distributions.
\end{abstract}

\section{Introduction}
The Building Game (BG) was first considered by Zlotnick~\cite{Zlotnick1994} as a model for the assembly of polyhedral viral capsids. We formalize the idea as a sequential coloring process that progresses from a polyhedron \poly\spc with each face colored \colorA, through a number of intermediate states each having a mix of \colorA\spc and \colorB\spc faces, and ending with all of the faces colored \colorB.
\begin{mydef}
A Building Game \textbf{intermediate} $x$ is a function from the faces of \poly\spc, $F\left(\mathscr{P}\right)$, to a color in $\left\{\colorA,\colorB\right\}$ such that the set $\left\{f_m \in F\left(\mathscr{P}\right) : x\left(f_m\right) = \colorB\right\}$ is edge connected along with the equivalence relation $x \sim x'$ if there is an element $g$ of \poly's rotation group $G$ that satisfies $x(f_m) = x'(g.f_m)$ for every $f_m \in F(\mathscr{P})$.
\end{mydef}
For ease of exposition, we use the notational shorthand $\left(x\right)_m$ for $x\left(f_m\right)$ and $x = g.x'$ when $x(f_m) = x'(g.f_m)$ for every $f_m \in F(\mathscr{P})$. Additionally, we denote the intermediate satisfying $\left(x\right)_m = \colorA$ for all $f_m \in \faceset$ as $x^\colorAsm$ and similarly $x^\colorBsm$ is the intermediate with $\left(x\right)_m = \colorB$ for all $f_m \in \faceset$. The function counting the number of \colorB\spc faces an intermediate has is denoted $h\left(x\right) \doteq |\left\{f_m \in \faceset : \left(x\right)_m = \colorB\right\}|$.

\begin{mydef}
Two intermediates $x^j$ and $x^k$ are \textbf{connected} ($x^j \leftrightarrow x^k$) if $\left(x^j\right)_m = \left(x^k\right)_m$ for all $f_m \in \faceset$ except for exactly one face $f_n$ that has  $\left(x^j\right)_n \neq \left(x^k\right)_n$.
\end{mydef}
\begin{mydef}
A Building Game \textbf{pathway} is a sequence of intermediates $x^{p_0}, x^{p_1}, x^{p_2}, \dots, x^{p_N}$ such that $x^{p_0} = x^\colorAsm$, $x^{p_N} = x^\colorBsm$, $x^{p_i}$ is connected to $x^{p_{i+1}}$ and $h\left(x^{p_i}\right) = i$.
\end{mydef}

In this way it is useful to think of intermediates as connected if it is possible to color one face of the first intermediate to get the second and a pathway as a sequence of these connections between $x^\colorAsm$ and $x^\colorBsm$. Figure~\ref{fig:DodecBG} shows a Building Game pathway for the dodecahedron using Schlegel diagrams. The pathways has 13 intermediates since there must be exactly one intermediate $x^{p_i}$ satisfying $h\left(x^{p_i}\right) = i$ for each $i = 0,1,2,\dots,12$.

\begin{figure}[ht]
\caption{One Building Game pathway on the dodecahedron.}
\label{fig:DodecBG}
\end{figure}

With many pairs of connected intermediates, we organize these relations in a graph.

\begin{mydef}
The Building Game \textbf{state space} for a polyhedron \poly\spc is a graph in which the nodes are \poly's intermediates and a graph edge exists between two intermediates if and only if they are connected.
\end{mydef}

When the intermediates are partitioned by their value of $h$, it is natural to arrange the state space as a tiered graph according to this partition. Figure~\ref{fig:CubeSS} shows the Building Game state space for the cube. As seen, each tier has intermediates with the same number of \colorB faces and connections thus exist with intermediates that are either in the tier directly above or below them. We can also see that there are three distinct pathways contained in the state space.

\begin{figure}[ht]
\caption{The Building Game state space of the cube.}
\label{fig:CubeSS}
\end{figure}

Interestingly, it is not the case that the recoloring of each face of \xj\spc results in a distinct intermediate.
\begin{mydef}
The number of different faces $\left|\left\{f_m \in \faceset : x^j + e^m \in \left[x^k\right]\right\}\right|$ of \xj\spc that can be colored to form \xk\spc is called the \textbf{degeneracy number} \Sjk.
\end{mydef}
It is important to note that in general the degeneracy number is not symmetric, i.e. \Sjk$\neq$\Skj\spc for some connections \xj\spc$\leftrightarrow$\spc\xk\spc in the state space. Both figures~\ref{fig:DodecBG} and ~\ref{fig:DodecBG} show the forward and backward degeneracy numbers for each connection.

\subsection{Related Work}
--Like polyominos on polyhedra

\subsection{Applications}
--Viral capsid assembly

--Self-assembly of molecular cages

--Self assembly for manufacturing purposes
%~\cite{Endres2005}
\subsection{Paper Overview}
-- Summary of subsequent sections

\section{Enumerative Results}

As we consider polyhedra with more and more faces, there is a combinatorial explosion in the number intermediates in state space. While the 6-faced cube state space has only 8 nodes and 9 nodes, the 20-faced icosahedron state space has 2,649 nodes and 17,241 nodes and the 26-faced truncated cuboctahedron state space has 1,525,605 nodes and 17,672,377. Figure \ref{fig:bgtable} details state space sizes of all polyhedra in the Platonic, Archimedean, and Catalan solid classes of up to 26 faces.

Also something about pathway statistics.

\begin{figure}[ht]
\scalebox{0.6}{
%{\footnotesize
\begin{tabular}{ l | c | c | c | c || r | r | r}
Polyhedra Name & Class & F$\left(\mathscr{P}\right)$ & E$\left(\mathscr{P}\right)$ & V$\left(\mathscr{P}\right)$ & Intermediates & Connections & Pathways \\
  \hline
Tetrahedron                     & P & 4 & 6 & 4         & 5     	& 4             & 1\\
Cube                            & P & 6 & 12 & 8        & 9     	& 10    	& 3\\
Octahedron                      & P & 8 & 12 & 6        & 15    	& 22    	& 14\\
Dodecahedron                    & P & 12 & 30 & 20      & 74    	& 264   	& 17,696 \\
Icosahedron                     & P & 20 & 30 & 12      & 2,650 	& 17,242        & 57,396,146,640\\
Truncated Tetrahedron           & A & 8 & 18 & 12       & 29    	& 65            & 402\\
Cuboctahedron                   & A & 14 & 24 & 12 	& 341   	& 1,636         & 10,170,968\\
Truncated Cube                  & A & 14 & 36 & 24 	& 500   	& 2,731         & 101,443,338 \\
Truncated Octahedron            & A & 14 & 36 & 24 	& 556           & 3,071         & 68,106,377\\
Rhombicuboctahedron             & A & 26 & 48 & 24 	& 638,851       & 6,459,804     & 16,494,392,631,838,879,380\\
Truncated Cuboctahedron         & A & 26 & 72 & 48 	& 1,525,605     & 17,672,377    & ? \\
Icosidodecahedron               & A & 32 & 60 & 30 	& ?             & ?             & ?\\
Truncated Dodecahedron          & A & 32 & 90 & 60 	& ?             & ? 	        & ? \\
Truncated Icosahedron           & A & 32 & 90 & 60 	& ?             & ? 	        & ?\\
Triakis Tetrahedron             & C & 12 & 18 & 8       & 99            & 319           & 38,938\\
Rhombic Dodecahedron            & C & 12 & 24 & 14 	& 128           & 494           & 76,936\\
Triakis Octahedron              & C & 24 & 36 & 14 	& 12,749        & 81,297        & 169,402,670,046,670\\
Tetrakis Hexahedron             & C & 24 & 36 & 14 	& 50,768        & 394,278       & 4,253,948,297,210,346\\
Deltoidal Icositetrahedron      & C & 24 & 48 & 26 	& 209,676       & 1,989,549     & ? \\
Pentagonal Icositetrahedron     & C & 24 & 60 & 38 	& 345,939       & 3,544,988     & 2,828,128,000,716,774,492\\
Rhombic Triacontahedron         & C & 30 & 60 & 32 	& ?             & ?             & 5,266,831,101,345,821,968\\
  \hline
\end{tabular}
}
\caption{Table of polyhedra in the Platonic (P), Archimedean (A), and Catalan (C) solid classes of up to 32 faces and their Building Game state space statistics.}
\label{fig:bgtable}
\end{figure}

\begin{figure}[ht]
\scalebox{0.6}{
%{\footnotesize
\begin{tabular}{ l | c | c | c | c || r | r | r}
Polyhedra Name & Class & F$\left(\mathscr{P}\right)$ & E$\left(\mathscr{P}\right)$ & V$\left(\mathscr{P}\right)$ & Intermediates & Connections & Pathways \\
  \hline
Tetrahedron                     & P & 4 & 6 & 4         & 5     & 4 & 1\\
Cube                            & P & 6 & 12 & 8        & 8     & 8 & 2\\
Octahedron                      & P & 8 & 12 & 6        & 12    & 12 & 14 \\
Dodecahedron                    & P & 12 & 30 & 20      & 53    & 156 & 2166\\
Icosahedron                     & P & 20 & 30 & 12      & 468   & 1984 & 105999738\\
Truncated Tetrahedron           & A & 8 & 18 & 12       & 22	& 42 & 174\\
Cuboctahedron                   & A & 14 & 24 & 12 	& 137	& 470 & 477776\\
Truncated Cube                  & A & 14 & 36 & 24 	& 248	& 1002 & 5232294\\
Truncated Octahedron            & A & 14 & 36 & 24 	& 343	& 1466 & 5704138\\
Rhombicuboctahedron             & A & 26 & 48 & 24 	& 70836	& 462149 &  48399693494788840\\
Truncated Cuboctahedron         & A & 26 & 72 & 48 	& ?	& ? & ?\\
Icosidodecahedron               & A & 32 & 60 & 30 	& ?	& ? & ?\\
Truncated Dodecahedron          & A & 32 & 90 & 60 	& ?	& ? & ?\\
Truncated Icosahedron           & A & 32 & 90 & 60 	& ?	& ? & ?\\
Triakis Tetrahedron             & C & 12 & 18 & 8       & 49	& 116 & 5012\\
Rhombic Dodecahedron            & C & 12 & 24 & 14 	& 68	& 196 & 6258\\
Triakis Octahedron              & C & 24 & 36 & 14 	& 667	& 2383 & 15255459\\
Tetrakis Hexahedron             & C & 24 & 36 & 14 	& 4220	& 21079 & 5854799360107\\
Deltoidal Icositetrahedron      & C & 24 & 48 & 26 	& ?	& ? & ?\\
Pentagonal Icositetrahedron     & C & 24 & 60 & 38 	& 95127	& 654537 & 5607231936129109\\
Rhombic Triacontahedron         & C & 30 & 60 & 32 	& 97368	& 697623 & 6889989896241902854\\
  \hline
\end{tabular}
}
\caption{Table of polyhedra in the Platonic (P), Archimedean (A), and Catalan (C) solid classes of up to 32 faces and their Building Game state space shellability statistics.}
\label{fig:bgtable_shell}
\end{figure}

\subsection{Bounds and Asymptotics}
Have upper, but what about lower?
\subsection{Methods}
\section{The Building Game as a Stochastic Process}
\label{sec:Prob}
Since the Building Game is a sequential process with several choices at each step, it is natural to consider it as a stochastic process. By putting a distribution on all possible faces that can be colored \colorB\spc at each step of the Building game, a distribution on the space of pathways is implicitly defined. Thus, for a choice of this transition rule, we can ask questions about the likelihood of the different pathways.

--Math and graphical results about putting a distribution on pathways

\subsection{Forward and Backward Transitions}

If we allow faces be changed both from \colorA\spc to \colorB\spc and from \colorB\spc to \colorA, the process consists of transitions from intermediate to intermediate along state space connections. By specifying a distribution on these transitions, it will induce a stationary measure on the state space.

We define the Markov process $X_t$ by the transition rate matrix $Q$, with the heuristic that the rate of transition to an intermediate \xk from an intermediate \xk should be proportional to the number of faces of \xj that can be colored to reach the intermediate \xk. For this reason, we include the degeneracy number \Sjk\spc as a factor in the transition rate matrix. Furthermore, we model the process after and energetic model in which each intermediate has an energy and to transtion between intermediates, an energy barrier $E_{jk} = E_{kj}$ must be overcome.
%\begin{align}
%\label{eq:TransitionProbability}
% P_{jk} = \frac{1}{z_j}S_{jk}\rh
%\end{align}
\begin{align}
\label{eq:TransitionRate}
Q_{jk} &= S_{jk}e^{-\beta\left(E_{jk} - E_j\right)} \\
Q_{jj} &= -z_j \\
\end{align}
Here, $z_j \doteq \sum_{\ell: \ell \neq j} S_{j\ell}e^{-\beta\left(E_{j\ell} - E_j\right)}$ is the rate at which the process leaves \xj.

\begin{mythm}
\label{thm:StatDist}
If the transition rate matrix $Q$ can be decomposed as $Q = DC$ where $D$ is diagonal with each entry of the diagonal positive and $C$ is a non-negative symmetric matrix with $C_{jk} > 0$ if and only if $x^j$ and $x^k$ are connected, then $X_t$ has the unique stationary distribution $\pi = \diag\left(D^{-1}\right)$.
\end{mythm}
\begin{proof}
First, we show $Q$ and $\pi$ satisfy detailed balance.
\begin{align}
\pi_jQ_{jk} &= \left(\frac{1}{D_{jj}}\right)\left(D_{jj}C_{jk}\right) \\
&= C_{jk} \\
&= C_{kj} \\
&= \left(\frac{1}{D_{kk}}\right)\left(D_{kk}C_{kj}\right) \\
                    &= \pi_kQ_{kj}
\end{align}

-- Prove aperiodicity
-- Prove positive reccurence

\end{proof}

In order to use theorem~\ref{thm:StatDist} to find the stationary distribution for the transition rule~\ref{eq:TransitionProbability}, we must be able to decompose the degeneracy number \Sjk\spc to fit the template of $\mathbf{C}$ and $\mathbf{D}$. In the following section we derive group theoretic identities to show that this is possible.

\subsection{Hitting Times}

\begin{align}
	\tau^{A}_{j} &\doteq \inf\left\{t \geq 0 : X_t \in A, X_0 = x^j\right\}
\end{align}

\begin{align}
	\nu^{A}_{j} &\doteq \inf\left\{n \geq 0 : Y_n \in A, Y_0 = x^j\right\}
\end{align}

For $j \not\in A$.
\begin{align}
	E\left[\tau^{A}_{j}\right] &= E\left[E\left[\tau^{A}_{j} | Y_1 \right]\right] \\
        &= E\left[ Exp\left(z_j\right) + \tau^{A}_{Y_1} \right] \\
        &=  \frac{1}{z_j} + E\left[\sum_{k}\tau^{A}_{Y_1}\mathbbm{1}_{Y_1 = k}\right] \\
        &=  \frac{1}{z_j} + \sum_{k: k\neq j}E\left[\tau^{A}_{k}\right] P\left(Y_1 = k\right) \\
        &=  \frac{1}{z_j}\left(1 + \sum_{k: k\neq j}q_{jk}E\left[\tau^{A}_{k}\right]\right)     \\
  \sum_{k}q_{jk}E\left[\tau^{A}_{k}\right] &= 1 \\
\end{align}

For $j \in A$.
\begin{align}
	E\left[\tau^{A}_{j}\right] &= 0 \\
\end{align}

As a linear system:
\begin{align}
	\left(\diag\left(\mathbbm{1}_A\right) - \diag\left(\mathbbm{1}_{A^c}\right)Q\right)E\left[\tau^{A}\right] =\mathbbm{1}_{A^c}\\
\end{align}

\begin{align}
\psi_j^A\left(t\right) &\doteq P\left(\tau^A_j \leq t\right) \\
\psi_j^A\left(0\right) &= \mathbbm{1}_{j\in A} \\
\psi_j^A\left(t\right) &= 0 \forall j \in A \\
\end{align}

For $j \not\in A$.

\begin{align}
\psi_j^A\left(t\right) &\doteq P\left(\tau^A_j \leq t\right) \\
                       &= \sum_k P\left(\tau^A_j \leq t | Y_1 = x^k\right) P\left(Y_1 = x^k\right) \\
                       &= \frac{1}{z_j}\sum_{k: k \neq j} q_{jk} P\left(Exp\left(z_j\right)\tau^A_j \leq t\right)  \\
                       &= \frac{1}{z_j}\sum_{k: k \neq j} q_{jk} \int^t_0 P\left(\tau^A_j \leq t - s\right) z_j e^{-z_j s} ds  \\
                       &= \sum_{k: k \neq j} q_{jk} \int^t_0\psi^A_k\left(t-s\right)e^{-z_j s} ds  \\
                       &= \sum_{k: k \neq j} q_{jk} \int^t_0\psi^A_k\left(r\right)e^{-z_j\left(t-r\right)} dr  \\
e^{z_jt}\psi^A_j\left(t\right) &= \sum_{k: k \neq j} q_{jk} \int^t_0 e^{z_jr}\psi^A_k\left(r\right) dr  \\
e^{z_jt}\frac{d\psi^A_j}{dt} + z_j e^{z_j t} \psi^A_j\left(t\right) &= \sum_{k: k \neq j} q_{jk} e^{z_jt}\psi^A_k\left(t\right)  \\
\frac{d\psi^A_j}{dt} &= \sum_{k} q_{jk} \psi^A_k\left(t\right)
\end{align}

Combining both cases, we get the linear system and solution.

\begin{align}
        \frac{d\psi^A}{dt} &= \diag\left(\mathbbm{1}_{A^c}\right)Q\psi^A \\
        \psi^A\left(0\right) &= \mathbbm{1}_{A} \\
        \psi^A\left(t\right) &= e^{\diag\left(\mathbbm{1}_{A^c}\right)Qt} \mathbbm{1}_{A} \\
\end{align}

This is the solution for the CDF of the stopping time $\tau^A$, but we can also compute the PDF explicitly for $t > 0$.

\begin{align}
        p\left(\tau^A = t\right) &= \frac{d\psi^A}{dt} \\
        &= \diag\left(\mathbbm{1}_{A^c}\right)Q\psi^A
\end{align}

\section{A Finite Geometric Result}

Since we define Building Game intermediates as rotationally unique from each other, it is useful to think about the problem in the context of $\mathscr{P}$'s rotational symmetry group $G \doteq G\left(\mathscr{P}\right)$ and group actions. For an intermediate $x^j$, the number of symmetries $r_j$ is the order of the stabilizer subgroup $G_{x^j} \doteq \left\{g \in G : g.x^j = x^j\right\}$ of $G$ that fixes $x^j$. Suppose $x^j$ and $x^k$ are connected in the state space and $\varphi$ is one of the $S_{jk}$ faces that, when added to $x^j$, forms $x^k$. We say $x^j + \varphi = x^k$. The degeneracy number $S_{jk}$ can then be expressed as the order of the orbit $\left(G_{x^j}\right).\varphi$ of $\varphi$ with respect to $x^j$'s stabilizer subgroup. Analogously, we define the reverse degeneracy number as $S_{kj} \doteq \left|\left(G_{x^k}\right).\varphi\right|$

\begin{mylem}
\label{lem:I}
For Building Game intermediates $x^j$ and $x^k$ connected in the state space and a face $f_m \in \faceset$ satisfying $x^j + e^m = x^k$, the stabilizer subgroup $G_{x^j,e^m}$ that fixes both $x^j$ and $e^m$ is the same stabilizer subgroup $G_{x^k,e^m}$ that fixes $x^k$ and $e^m$.
\end{mylem}
\begin{proof}
\begin{align}
G_{x^j,e^m} &\doteq \left\{g \in G | g.x^j = x^j, g.e^m = e^m \right\} \\
                &= \left\{g \in G | g.\left(x^k - e^m\right) = x^k - e^m, g.e^m = e^m \right\} \\
                &= \left\{g \in G | g.x^k = x^k, g.e^m = e^m \right\} \\
                &\doteq G_{x^k,e^m}
\end{align}
\end{proof}

\begin{mythm}
\label{thm:J}
For two Building Game intermediates $x^j$ and $x^k$ are connected in the BG state space, $r_kS_{jk} = r_jS_{kj}$.
\end{mythm}
\begin{proof}
Let $e^m$ be a face such that $x^k = x^j + e^m$. Then, by the orbit-stabilizer theorem, Lagrange's Theorem and lemma~\ref{lem:I} we have the following~\cite{Rotman1995}.
\begin{align}
\frac{r_j}{S_{jk}} &\doteq \frac{\left|G_{x^j}\right|}{\left|\left(G_{x^j}\right).e^m\right|} \\
                   &= \left[G_{x^j} : \left(G_{x^j}\right).e^m \right] \\
                   &= \left|G_{x^j,e^m}\right| \\
                   &= \left|G_{x^k,e^m}\right| \\
                   &= \left[G_{x^k} : \left(G_{x^k}\right).e^m \right] \\
                   &= \frac{\left|G_{x^k}\right|}{\left|\left(G_{x^k}\right).e^m\right|} \\
                   &\doteq \frac{r_k}{S_{kj}}
\end{align}
The result $r_kS_{jk} = r_jS_{kj}$ follows.
\end{proof}

\section{Stationarity}

\begin{mythm}
\label{thm:E}
The Markov process $X_t$ defined by the transition rate matrix $Q$ in equation~\ref{eq:TransitionRate} admits the unique stationary distribution $\frac{1}{zr_j}e^{-\beta E_j}$ where $z \doteq \sum_\ell \frac{1}{r_\ell}e^{-\beta E_\ell}$ is the partition function.
\end{mythm}
\begin{proof}
We take $C_{jk} \doteq \frac{S_{jk}}{zr_j}e^{-\beta E_{jk}}$ and notice that it is symmetric by theorem~\ref{thm:J}. With $D_{jj} \doteq zr_je^{\beta E_j}$ we have our partition.
\begin{align}
Q_{jk} &= S_{jk}e^{-\beta\left(E_{jk} - E_j\right)} \\
       &= \left(zr_je^{\beta E_j}\right) \left(\frac{S_{jk}}{zr_j}e^{-\beta E_{jk}}\right) \\
       &= D_{jj}C_{jk}
\end{align}
Thus, by theorem~\ref{thm:StatDist},  $\pi_j = \frac{1}{D_{jj}} = \frac{1}{zr_j}e^{-\beta E_j}$.
\end{proof}

\section{Discussion}
\subsection{Nonenumerative Approaches}

\section*{Acknowledgments}\label{ackowledgements}
        Supported by NSF grants DMS 07-48482 and EFRI 10-22638

\section{Potential Citations (temp)}
~\cite{Coxeter1963}
~\cite{Grunbaum2003}
~\cite{Cromwell1997}
~\cite{Ziegler1995}
~\cite{Gidas1995}
~\cite{Eden1961}
~\cite{Grayson2012}

\bibliographystyle{plain}
%\nocite{*}
\bibliography{Master}

%\appendix
%\section{Proof of Theorem ???}
%\section{Proof of Theorem ???}

\end{document}

%%%%%%%%%%%%%%%%%%%% CCCCUUUUTTTT %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{A Curious Observation}

As part of the exploration of the nature of the degeneracy numbers $S_{jk}$ to find a stationary distribution, an interesting phenomena was noticed. Given a closed path $x^{k_0} \to x^{k_1} \to \cdots \to x^{k_n}$ through the state space, starting at $x^{k_0}$ and ending back at $x^{k_n} \doteq x^{k_0}$, the the identity given in equation~\ref{eq:DegenIdent} was observed to hold independent of the particular closed path.
\begin{align}
\prod_{i=1}^n \frac{S_{k_i,k_{i-1}}}{S_{k_{i-1},k_i}} = 1
\label{eq:DegenIdent}
\end{align}
At first this relation between ratios of degeneracy numbers along the path was noticed in the cube's state space and it can easily be seen by examining figure~\ref{fig:CubeSS}. To test this conjecture more fully, it was subsequently verified on our library of state spaces for other polyhedra. This exploration lead to the discovery of a more general relation between intermediates that it described below.

In an attempt to get the the heart of the relation in equation~\ref{eq:DegenIdent}, we compiled a list of different geometric and combinatorial statistics for each state space connection $x_j \leftrightarrow x_k$. These statistics included number of faces in $x_j$ and $x_k$, the order of $x_j$ and $x_k$'s rotation groups $r_j$ and $r_k$, and a few others. With the ansatz that there may be a multiplicative relation between the degeneracy number $S_{jk}$ and some of these $d$ statistics $Z_{jk}^{\left(i\right)} \in \mathbbm{Z}^d$ for each connection, the following linear regression model was used.
\begin{align}
\log\left(S_{jk}\right) = \sum_{i=1}^d \beta^{\left(i\right)}\log\left(Z_{jk}^{\left(i\right)}\right)
\label{eq:LinReg}
\end{align}
The regression coefficients $\boldsymbol{\beta} \in \mathbbm{R}^d$ that minimize the least-squares residual were solved for and it was found that a perfect relation was found in the sense that equation~\ref{eq:LinReg} held exactly for all connections tested. The three non-zero entries in $\boldsymbol{\beta}$ corresponded to $r_j$, $r_k$ and $S_{kj}$. This result meant that $\log\left(S_{jk}\right) = \log\left(S_{kj}\right) + \log\left(r_{j}\right) - \log\left(r_{k}\right)$, or
\begin{align}
S_{jk} = \frac{r_jS_{kj}}{r_k}
\label{eq:RSConjecture}
\end{align}
for all connections we tested. This leads to the obvious conjecture that equation~\ref{eq:RSConjecture} holds for any connection in any Building Game state space.

In fact, if this is the case, it would explain why equation~\ref{eq:DegenIdent} holds.
\begin{align}
        \prod_{i=1}^n \frac{S_{k_i,k_{i-1}}}{S_{k_{i-1},k_i}} &= \prod_{i=1}^n \frac{r_{k_i}}{r_{k_{i-1}}} \\
        &= \frac{r_{k_n}}{r_{k_{0}}} \\
        &= \frac{r_{k_0}}{r_{k_{0}}} \\
        &= 1
\end{align}

\begin{figure}[ht]
\scalebox{0.6}{
%{\footnotesize
\begin{tabular}{ l | c | c | c | c || r | r | r}
Polyhedra Name & Class & F$\left(\mathscr{P}\right)$ & E$\left(\mathscr{P}\right)$ & V$\left(\mathscr{P}\right)$ & Intermediates & Connections & Pathways \\
  \hline
Tetrahedron & P & 4 & 6 & 4 & 5 & 4 & 1\\
Cube & P & 6 & 12 & 8 & 9 & 10 & 3\\
Octahedron & P & 8 & 12 & 6 & 15 & 22 & 14\\
Dodecahedron & P & 12 & 30 & 20 & 74 & 264 & 17,696 \\
Icosahedron & P & 20 & 30 & 12 & 2,650 & 17,242 & 57,396,146,640\\
Truncated Tetrahedron & A & 8 & 18 & 12 & 29 & 65 & 402\\
Cuboctahedron & A & 14 & 24 & 12 & 341 & 1,636 & 10,170,968\\
Truncated Cube & A & 14 & 36 & 24 & 500 & 2,731 & 101,443,338 \\
Truncated Octahedron & A & 14 & 36 & 24 & 556 & 3,071 & 68,106,377\\
Rhombicuboctahedron & A & 26 & 48 & 24 & 638,851 & 6,459,804 & 16,494,392,631,838,879,380\\
Truncated Cuboctahedron & A & 26 & 72 & 48 & 1,525,605 & 17,672,377 & ? \\
Icosidodecahedron & A & 32 & 60 & 30 & ? & ? & ?\\
Truncated Dodecahedron & A & 32 & 90 & 60 & ? & ? & ? \\
Truncated Icosahedron & A & 32 & 90 & 60 & ? & ? & ?\\
Triakis Tetrahedron & C & 12 & 18 & 8 & 99 & 319 & 38,938\\
Rhombic Dodecahedron & C & 12 & 24 & 14 & 128 & 494 & 76,936\\
Triakis Octahedron & C & 24 & 36 & 14 & 12,749 & 81,297 & 169,402,670,046,670\\
Tetrakis Hexahedron & C & 24 & 36 & 14 & 50,768 & 394,278 & 4,253,948,297,210,346\\
Deltoidal Icositetrahedron & C & 24 & 48 & 26 & 209,676 & 1,989,549 & ? \\
Pentagonal Icositetrahedron & C & 24 & 60 & 38 & 345,939 & 3,544,988 & 2,828,128,000,716,774,492\\
Rhombic Triacontahedron & C & 30 & 60 & 32 & ? & ? & 5,266,831,101,345,821,968\\
  \hline
\end{tabular}
}
\caption{Table of polyhedra in the Platonic (P), Archimedean (A), and Catalan (C) solid classes of up to 32 faces and their Building Game state space statistics.}
\label{fig:bgtable}
\end{figure}

%%%%%%%

The process begins with each face of a polyhedron \poly\spc all colored the same color, say \colorA. A face is then chosen and its color is changed to a second color, \colorB. From there, at each step a \colorA\spc face that is edge-adjacent to a \colorB\spc face is chosen and recolored \colorB. The process continues until all of \poly's faces are \colorB. We show one instance of the Building Game process for the dodecahedron in figure~\ref{fig:DodecBG}.

Each possible coloring of the polyhedron from the Building Game is referred to as an \textbf{intermediate}. Since we assume each face of the same color is otherwise indistinguishable, there is a rotational equivalence class on intermediate such that two intermediates are equivalent if the first is a rotation of the second.

\begin{figure}[ht]
\caption{One instance of the Building Game on the dodecahedron.}
\label{fig:DodecBG}
\end{figure}

We define the \textbf{state space} of the Building Game to be a graph in which the nodes are all of the possible intermediates allowed by the BG. Connections exist between two intermediates \xj\spc and \xk\spc if it is possible to color a single face of \xj\spc to form \xk. Interestingly, it is not the case that the recoloring of each face of \xj\spc maps to a distinct intermediate. Thus, the number of different faces of \xj\spc that can be colored to form \xk\spc is called the \textbf{degeneracy number} \Sjk. It is important to note that in general the degeneracy number is not symmetric, i.e. \Sjk$\neq$\Skj\spc for some connections \xj\spc$\leftrightarrow$\spc\xk\spc in the state space.

Since the state space can be partitioned by the number of \colorB\spc faces each intermediate has, it is natural to view the state space as a tiered graph in which intermediates in each tier have the same number of \colorB\spc faces. Organized this way, intermediates can only connect to those in the tier above or below them. A \textbf{pathway} in the state space is a sequence of intermediates $x^{k_0} \to x^{k_1}\to \cdots \to x^{k_F}$ connected in the state space such that $x^{k_0}$ is the intermediate with all \colorA\spc faces, $x^{k_F}$ is the intermediate with all \colorB\spc faces, and $x^{k_m}$ has $m$ \colorB\spc faces.

%%%%%%%%

To find the stationary measure of this process, we check if it is possible for our transition measure to satisfy the detailed balance equation $\pi_jP_{jk} = \pi_kP_{kj}$. As seen in equation~\ref{eq:DBTry}, for detailed balance to be satisfied, we must find a way to separate the ratio $\frac{S_{jk}}{S_{kj}}$ into a part depending only on $j$ terms and a part depending only on $k$ terms.
\begin{align}
        \frac{\pi_k}{\pi_j} &= \frac{P_{jk}}{P_{kj}} \\
        &= \frac{S_{jk}\rho_{jk}z_k}{S_{kj}\rho_{kj}z_j} \\
        &= \frac{z_k}{z_j}\frac{S_{jk}}{S_{kj}} \label{eq:DBTry}
\end{align}
While detailed balance is not a necessary condition for the existence of a stationary distribution, along with positive recurrence of the process, it is sufficient and would provide the exact form of the necessarily unique stationary distribution.

%%%%%%%%%%

As stated in section~\ref{sec:Prob}, if we can find a distribution $\pi$ such that the detailed balance equation $\pi_jP_{j,k} = \pi_kP_{k,j}$ is satisfied, we know that $\pi$ is the unique stationary distribution under $P_{jk}$. With theorem~\ref{thm:J} we find that $\frac{S_{jk}}{S_{kj}} = \frac{r_j}{r_k}$ is indeed separable as conjectured in equation~\ref{eq:DBTry}. This allows us to find the stationary distribution.

\begin{mythm}
\label{thm:E}
The Markov chain $X_t$ defined by the transition rule $P_{jk}$ in equation~\ref{eq:TransitionProbability} admits the unique stationary distribution $\pi_j = \frac{1}{z}\left(\frac{z_j}{r_j}\right)$ where $z \doteq \sum_i \frac{z_i}{r_i}$ is the partition function.
\end{mythm}
\begin{proof}
It suffices to show detailed balance and that each state is positively recurrent. By our definitions of $\pi$ and $P$ along with Theorem~\ref{thm:J}, we find detailed balance.
\begin{align}
        \pi_jP_{jk} &= \frac{z_j}{z r_j}\frac{1}{z_j}S_{jk}\rho_{jk} \\
                  &= \frac{1}{z}\left(\frac{S_{jk}}{r_{j}}\right)\rho_{jk} \\
                  &= \frac{1}{z}\left(\frac{S_{kj}}{r_k}\right)\rho_{kj} \\
                   &= \frac{z_k}{z r_k}\frac{1}{z_k}S_{kj}\rho_{kj} \\
                    &= \pi_kP_{kj}
\end{align}
Clearly $X_t$ is positive recurrent since it is a finite irreducible Markov chain on a connected state space in which there exist forward and backward transitions between each connected node with positive probabilities.
\end{proof}


\documentclass{memoir}
\usepackage{notestemplate}

%\logo{~/School-Work/Auxiliary-Files/resources/png/logo.png}
%\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
%\department{Department of Mathematics}
%\title{Class Notes}
%\subtitle{Based on MATH xxx}
%\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
%\context{Well, I was bored...}
%\date{\today}

%\makeindex

\begin{document}

% \maketitle

% Notes taken on 06/09/21

Let \(V\) be a finite dimensional vector space over \(F\) with dimension \(n\), and \(T\) a fixed linear transformation of \(V\). Recall that we can view \(V\) as an \(F[x]\)-module where \(x\) acts on \(V\) as the linear transformation \(T\).\\

Because \(V\) has finite dimension over \(F\), it must be a torsion \(F[x]\)-module. Hence \(V\) is isomorphic as an \(F[x]\)-module to the direct sum of cyclic, torsion \(F[x]\)-modules.\\

When we decompose \(V\) into the invariant factor decomposition basis, we obtain the rational canonical form for the matrix for \(T\). When we use the elementary divisor decomposition, we obtain the Jordan canonical form.

\begin{defn}[Minimal Polynomial]
 The \textbf{minimal polynomial of \(T\)} is the unique monic polynomial \(m_T(x) \in F[x]\) that generates the ideal \(\textrm{Ann}(V)\) in \(F[x]\).\\

 Let \(A\) be a matrix. The \textbf{minimal polynomial of \(T\)} is the unique monic polynomial of smallest degree \(m_A(x)\) that yields the zero matrix when evaluated at \(A\).
\end{defn}

The \textbf{rational canonical form} of the linear transformation \(T\) is the isomorphism
\begin{align*}
	V \cong F[x] / (a_1(x)) \oplus F[x] / (a_2(x)) \oplus \ldots \oplus F[x] / (a_m(x))
\end{align*}
where \(a_1(x),a_2(x),\ldots,a_m(x)\) are polynomials in \(F[x]\) of positive degree such that
 \begin{align*}
	 a_1(x) \mid a_2(x) \mid \ldots\mid a_m(x)
\end{align*}
Of course, the annihilator of \(V\) is the ideal \((a_m(x))\), and so we obtain:
\begin{prop}
	The minimal polynomial \(m_T(x)\) is the largest invariant factor of \(V\).
\end{prop}

Observe that we can get a basis for the vector space \(F[x] / (a(x))\) for a fixed
\begin{align*}
	a(x) = x^{k}+ b_{k-1}x^{k-1} + \ldots + b_1 x + b_0
\end{align*}
by defining \(\overline{x}^{k}= (x \pmod{a(x)})^{k}\). The basis is then \(\left\{ 1,\overline{x},\overline{x}^2,\ldots,\overline{x}^{k-1} \right\} \) which has an action under multiplication by \(x\) given by:
\begin{align*}
	1 \mapsto \overline{x}\\
	\overline{x} \mapsto \overline{x}^2\\
	\vdots\\
	\overline{x}^{k-2}\mapsto \overline{x}^{k-1}\\
	\overline{x}^{k-1}\mapsto \overline{x}^{k} = -b_0 - b_1\overline{x} - \ldots - b_{k-1}\overline{x}^{k-1}
\end{align*}
which follows because
\begin{align*}
	\overline{x}^{k} + b_{k-1}\overline{x}^{k-1} + \ldots + b_1\overline{x}+b_0 = 0
\end{align*}
This gives us a matrix for multiplication by \(x\):
\begin{defn}[Companion Matrix]
	Let \(a(x) = x^{k}+ b_{k-1}x^{k-1}+ \ldots + b_1x + b_0\) be a monic polynomial in \(F[x]\). The \textbf{companion matrix} of \(a(x)\) is the \(k\times k\) matrix representing the matrix for multiplication by \(x\), and is of the form
	\begin{align*}
		\begin{pmatrix}
			0 & 0 & \ldots & \ldots & -b_0\\
			1 & 0 & \ldots & \ldots & -b_1 \\
			0 & 1 & \ldots & \ldots & -b_2\\
			\vdots & \vdots &   & \ddots & \vdots\\
			0 & 0 & \ldots & 1 & -b_{k-1}
	\end{pmatrix}
	\end{align*}
	We denote the companion matrix of \(a(x)\) by \(\mathcal{C}_{a(x)}\).
\end{defn}

Now we will apply this to each of the cyclic modules in the rational canonical form of \(V\). Let \(\mathcal{B}_i\) be the set of basis elements for each cyclic factor \(F[x] / (a_i(x))\). The linear transformation \(T\) acts on \(\mathcal{B}_i\) by the companion matrix for \(a_i(x)\), and hence the union \(\mathcal{B} = \cup_{i} \mathcal{B}_i\) and the matrix of the transformation on \(V\) is the direct sum of the companion matrices.

\begin{defn}[Rational Canonical Form]
	A matrix is said to be in \textbf{rational canonical form} if it is the direct sum of companion matrices for monic polynomials \(a_1(x),\ldots,a_m(x)\) with
	\begin{align*}
		a_1(x) \mid a_2(x) \mid \ldots \mid a_m(x).
	\end{align*}
	The matrix is then of the form
	\begin{align*}
		\begin{pmatrix}
			\mathcal{C}_{a_1(x)} & & & \\
			& \mathcal{C}_{a_2(x)} & & \\
			& & \ddots & \\
			& & & \mathcal{C}_{a_m(x)}
		\end{pmatrix}.
	\end{align*}
	The polynomials are called the \textbf{invariant factors} of the matrix, and the matrix is said to be a \textbf{block diagonal matrix} with the blocks being the companion matrices for \(a_i(x)\).\\

	A \textbf{rational canonical form} for a linear transformation \(T\) is a matrix representing \(T\) in rational canonical form.
\end{defn}
One can check that every linear transformation \(T\) has a rational canonical form that is unique.

\begin{thm}[Rational Canonical Form for Linear Transformation]
	Let \(V\) be a finite dimensional vector space over the field \(F\) and let \(T\) be a linear transformation of \(V\). Thne there is a basis for \(V\) and the matrix of \(T\) is in rational canonical form with respect to this basis. Furthermore, the rational canonical form for \(T\) is unique.
\end{thm}
We will see that this exists for every \(T\), but Jordan canonical form may not.\\

For linear transformations \(S\) and \(T\), the following are equivalent:
\begin{itemize}
	\item \(S\) and \(T\) are similar linear transformations
	\item The \(F[x]\)-modules obtained from \(V\) via \(S,T\) are isomorphic
	\item \(S\) and \(T\) have the same rational canonical form
\end{itemize}
Of course, any matrix can be translated into a rational canonical form, as each matrix corresponds to a linear transformation. The \textbf{invariant factors} of an \(n\times n\) matrix over a field \(F\) are the invariant factors of its rational canonical form.

\begin{lemma}
	Let \(a(x) \in F[x]\) be any monic polynomial. The characteristic polynomial of the companion matrix of \(a(x)\) is \(a(x)\), and if  \(M\) is given by
	\begin{align*}
		M =
		\begin{pmatrix}
			A_1 & 0 & \ldots & 0 \\
			0 & A_2 & \ldots & 0\\
			\vdots & \vdots & \ddots & \vdots \\
			0 & 0 & \ldots & A_k
		\end{pmatrix}
	\end{align*}
	then the characteristic polynomial of \(M\) is the product of the characteristic polynomials of \(A_1,A_2,\ldots,A_k\).
\end{lemma}

\begin{thm}[Cayley-Hamilton Theorem]
	Let \(A\) be an \(n\times n\) matrix over the field \(F\). The minimal polynomial of \(A\) divides the characteristic polynomial of \(A\).
\end{thm}
In fact, the characteristic polynomial of \(A\) divides some power of the minimal polynomial of \(A\).

\begin{thm}
	Let \(A\) be an \(n\times n\) matrix over the field \(F\). The \(n\times n\) matrix \(xI - A\) can be put into the diagonal form
	\begin{align*}
		\begin{pmatrix}
			1 & & & & &\\
			  & \ddots & & & &\\
			  & & 1 & & &\\
			  & & & a_1(x) & &\\
			  & & & & \ddots & \\
			  & & & & & a_m(x)
		\end{pmatrix}
	\end{align*}
	with monic nonzero elements
	\begin{align*}
		a_1(x) \mid a_2(x) \mid \ldots \mid a_m(x).
	\end{align*}
	via the operations
	\begin{itemize}
		\item interchanging two rows or columns
		\item adding a multiple of one row or column to another
		\item multiplying any column or row by a unit in \(F[x]\)
	\end{itemize}
	The elements \(a_1(x),\ldots,a_m(x)\) are the invariant factors of \(A\).
\end{thm}

\subsection{Invariant Factor Decomposition Algorithm}
\label{sub:invariant_factor_decomposition_algorithm}

%% Fill in

\subsection{Converting an \(n\times n\) Matrix to Rational Canonical Form}
\label{sub:converting_an_n_by_n_matrix_to_rational_canonical_form}

% \printindex
\end{document}


\chapter{Appendix Probability}

\section*{Probability Basics}
Let $\mu$ be a non-negative countably additive set function over a sigma algebra $\Omega$ of sets from a sample space $S$. In probability theory, $\Omega$ is the set of possible events $E$. Sets of measure zero denote impossible outcomes. An important feature of measures $\nu$ and $\mu$ that agree on sets of measure zero is the ability to define a derivative $\frac{d \nu}{d \mu}$, the Radon-Nikodym derivative.  When $\nu (E)=0 \fall E \in \Omega \;| \; \mu(e)=0$  Alternatively, given a measure $\mu$ and a nonnegative measurable function $f$, a new measure can be defined by $ \nu(E) = \int\limits{E \in \Omega}{} f d \mu$. A random variable is a real valued function on a sample space into a metric space, $X : S \rightarrow \dblr^{1} $. Associated with a random variable is it's probability density function $f_{X}(x)=P(\{s \in S | X(s) = x\})$ operating on an algebra of sets generated by the sample space $S$. By definition, $f_{X}(x)$ is the sum of probabilities of the events in $S$ that get mapped to $x \in \dblr$ by $X$.  Let $\script{B}(S)$ be the Borel sets on $S$, then $X$ has density $f$ if $P(X \in A) = \int_{A \in \script{B}(S)} f(x) dx $ and distribution function $F(x) = P(X<x)=\int\limits_{-\infty}^x f(y) dy$ so $F'(x)=f(x) \;\; a.e.$.  $E(X)=\int x f(x) dx$ is the expectation of $X$.  The characteristic function $\phi(x) = E(e^{itX} )$ determines the distribution and is used in the proof of the CLT theorem, testing for symmetry, and conditional independence.  We denote samples with lower case in this section.  $x_1 \ldots x_n$ is a random sample of size $n$  In $\dblr^n$ the random variate $X$ has distribution function $F(x_i, \ldots , x_n) = P(X_1<x_1, \ldots , X_n<x_n )$ and density $f(x_1, \ldots , x_n )$.

For parametric distributions, one is interested in the question of what value of a parameter best describes the data at hand.  This obviously requires the assumption that the data derives from a family of distributions parameterized by one or more variables $\theta_k$.  If $x_1 \ldots x_n$ is a random sample from $X$ with a distribution given by $p(x;\theta_1 \ldots \theta_k)$, we can think of the joint pdf of of the sample $L(x_i, \ldots,x_n) = \prod\limits_{i=1}^{n} p(x_i;\theta_1 \ldots \theta_k)$ as being explained by the parameters.  $L$ is the likelihood of the data given the parameters.  The maximum likelihood estimate is obtained by solving the set of equations;
\begin{eqnarray} \nonumber
  \frac{\partial L(\theta_1 \ldots \theta_k)}{\partial
  \theta_1}=0 \\ \nonumber
  \vdots \\ \nonumber
   \frac{\partial L(\theta_1 \ldots \theta_k)}{\partial
   \theta_k}=0 \\ \nonumber
\end{eqnarray}

%$ P(X), P(X|Y), P(X,Y) p(x) p_i x_i \mathbf{X} \mathbf{\beta}$

The statistical moments of a random variable $X$ are defined $\mu_n = E [ X^n] = \int_\Omega X^n P(X)dX$.  The characteristic function is the fourier transform of $P(X)$ \[\Phi(\omega)=\digamma \mathcal{F} [P(X)] (\omega) = \int\limits_{\infty}^{\infty} e^{i \omega X} P(X) dX.\]  Taking the logarithm and expanding in a MacLauren series, we can relate the statistical moments to the coefficients. Statistical moments are central or raw.

Common sample descriptive statistics relating to location, scale, tail size,  and peakedness;
\[ mean = \hat{\mu_1} = \frac{1}{n}\sum x_i  \]
\[ variance = sdd^2 =  \hat{\mu_2} = \frac{1}{n-1} \sum (x_i- \hat{\mu_1}\]
\[ skewness = \frac{ \hat{\mu_3}}{\hat{\mu_2}^{3/2}} \]
\[ kurtosis = \frac{\hat{\mu_4}}{\hat{\mu_2}^2} \]
The linear association between $X_i$ and $X_j$ is measured by the covariance \[ Cov_{ij} = \sigma_{ij} = \frac{1}{n-1} \sum_k (x_ki - \hat{\mu_1}_i)(x_{kj} - \hat{\mu_1}_j). \]  For a measure without dependence on units the scaled covariance is the correlation \[C_{ij} =\frac{\sigma_{ij}}{\sqrt{\sigma_{ii}}\sqrt{\sigma_{jj}}}\]

\section*{Univariate Probability Distributions}
This section covers the properties of common univariate
probability distributions.

\subsection{Uniform,  $U(\alpha,\beta)$}
$X=_d U(\alpha,\beta)$ if $p(x; \alpha, \beta)=
\frac{\chi_{[\alpha,\beta]}}{\beta-\alpha}.$

\subsection{Exponential Class of Distributions}
The exponential class of distributions are characterized by the
functional for of the pdf; \[ p(x;\theta) =
exp(\alpha(x)\beta(\theta)+\gamma(\theta)+\delta(x) ) \].  This
class of distributions form the basis of Generalized Linear
Model Theory that is discussed below.

There is an alternate parametrization of the exponential family
that explicitly includes a dispersion parameter $\phi$.  This
is useful for count data where $E[X]=E[X^2]=\theta$  In general
if $E[X^2]>E[X]$ we say the process  or data is over dispersed.
The parameter $\phi$ is usually fixed in practice.  If we write
\[ p(x; \theta, \phi)=exp(\frac{x \theta-
\beta(\theta)}{\alpha(\phi)}+\gamma(x,\phi))\], the dispersion
parameter $\phi$ for some common distributions;
\[
\begin{array}{cc}
p(x;\theta, \phi) & \phi \\ \hline
N(\mu,\sigma) & \sigma^2 \\
IG(\mu,\sigma) & \sigma^2 \\
Gamma(\theta,\phi) & \frac{1}{\phi} \\
Poisson(\theta) & 1 \\
Binomial(\theta) & 1 \\
Negative Binomial(\theta,r) & r \\
\end{array}
\]

\subsubsection{Normal/Gaussian, $N(\mu,\sigma)$}
$X=_d N(\mu,\sigma)$ if \[p(x; \mu, \sigma) = \frac{1}{\sqrt{2
\pi \sigma^2}} exp ( \frac{(x-\mu)^2}{2 \sigma^2}\].  This can
be re-written in exponential form \[ p(x;\theta) = exp( \frac{x
\theta}{2 \sigma^2} - \frac{\theta^2}{2 \sigma^2 }-
\frac{1}{2}log(2 \pi \sigma^2) - \frac{x^2}{2 \sigma^2 } ) \]

\subsubsection{Binomial}
Setting \[\alpha(x)=x \;\;,
\beta(\theta)=log(\frac{\theta}{1-\theta}) \;\;,
\gamma(\theta)=n log(1-\theta) \;\;, \delta(x)=log ( \biggl(
\begin{array}{c}  n \\  y \\ \end{array}  \biggr) )
\] gives us the binomial distribution $p(x;\theta) = \binomial{n}{x} \theta^x
(1-\theta)^{(n-x)}$

\subsubsection{Negative Binomial}
\[p(x;\theta,r)=\binomial{x+r-1}{r-1} \theta^r (1-\theta)^x\]

\subsubsection{Poisson}
$X$ is Poisson distributed if $p(x;\theta)=\frac{\theta^x
e^{-\theta}}{x!}$  Setting $\beta(\theta)=log(\theta) \;\;,
\gamma(\theta)=-\theta \;\;, \delta(x)=-log(x!)$ we get the
exponential form of the Poisson distribution,
\[p(x;\theta)= exp(x log(\theta) - \theta- log(x!))\]

The expected value and the variance of a Poisson distributed random variable is equal to $\theta$. The higher moments of the Poisson distribution are the Touchard polynomials in $\theta$.  There is a combinatorial interpretation.  When $E[X]=1$ for a Poisson random variate then the i-th moment of $X$ is equal to the number of partitions of a set of size n $\frac{1}{e} \sum\limits_{n=0}^{\infty} \frac{n^i}{n!}$ via Dobinski.  The normal distribution with mean $\theta$ and variance $\theta$ is a good approximation to the Poisson distribution for large $\theta$.

\subsubsection{Pareto}
$X$ is Pareto distributed if \[p(x;\theta)=\theta
x^{-\theta}.\]

\subsubsection{Gamma}
$X$ is Gamma distributed if \[p(x;\theta,
\phi)=\frac{x^{\phi^{-1}}\theta^\phi e^{-x \theta}} {
\Gamma(\phi)}.\]

\subsubsection{Weibull}
$X$ follows the Weibull distribution if \[p(x; \theta , \lambda
) = \frac{\lambda x^{\lambda-1}}{\theta^\lambda}
e^{(\frac{x}{\theta} )^\lambda}.\]

\subsubsection{Inverse Gaussian/ Wald Distribution}
$X$ follows the Inverse Gaussian distribution if
\[p(x;\theta)= \sqrt{\frac{\theta}{2 \pi x^3 \sigma}}exp(-
\frac{\lambda(x-\theta)^2}{2 x \theta^2 \sigma}) \]

\subsection{Generalized Extreme Value Distribution $GEV(\theta,\phi,\xi)$}
This class of distributions includes the three limiting extreme
value distributions of \cite{Fisher Tippet (1928)} and
\cite{Gnedenko (1943)}.  $X =_d GEV(\theta,\phi,\xi)$ if
\[p(x;\theta,\phi,\xi)= exp( -  max\biggl(\bigl(1 + \xi
\frac{x-\theta}{\phi} \bigr)^{- \frac{1}{\xi} }, \;\; 0 \biggr)
\]. Where

\subsection{Multinomial}%bbcrevisit explanation
Let \[\Omega=\mathcal{B}(\prod\limits_{i=0}^{i=\infty}
\dblz(K))\] be the Borel Algebra generated by $\prod {1, 2,
\hdots , K}$. This is the sample space of all realizations of
experiments with $K$ categorical outcomes. Equip $\dblz(K)= {i
\in {1, \hdots, K}}$ with a measure $P(i)=\theta_i$. Let
${X_i}$ be n iid copies of $X =_d p(i ; \pi_1, \hdots ,
\pi_K)$.  Now map $\mbf{X}=(X_1, \hdots, X_n) \in \Omega
\rightarrow \mbf{Y} \in \dbln^K$  Then $Y_i$ are counts of the
number of elements of category $i$ in the experiment with n
observations.  The multinomial distribution is given by
\[p(\mbf{Y};n)=\frac{n!}{y_1! \hdots y_K!} (\theta_1)^{y_1} \hdots
(\theta_K)^{y_K} \].  This in not a member of the exponential
family, but we can show that the multinomial distribution is
the joint distribution of ${Y_i =_d Poission(\theta'_i)}_{i=1,
\hdots K}$ random variables conditional to their sum.
\[p(\mbf{Y}; \theta'_1, \hdots , \theta'_K) =
\prod\limits_{i=1}^{K} \frac{ (\theta'_i)^{y_i} \;
e^{-\theta'_i}}{y_i!}\], set $n=Y_1 + \hdots + Y_K$. Writing
$p(\mbf{Y} | n) =p(\mbf{Y}; \theta'_1, \hdots ,
\theta'_K) / p(n)$ and noting that $n=_d Poisson(\sum\limits_{i=1}^{K} \theta'_i)$, %bbcrevisit this is an exercise
we recover the multinomial distribution by simplifying and
setting $\theta_i=\frac{\theta'_i}{\sum\limits_{i=1}^{K}
\theta'_i}$

\subsection{$\chi^2(n)$}
If ${X_i}$ iid $N(0,1)$ and $Y_i=X_i^2$ then
$Y=\sum\limits_{i=1}^{n} Y_i =_d \chi^2(n).$  $E[Y]=n$ and
$Var(Y)=E[(Y-\mu_Y)^2]=E[(Y-E[Y])^2]=2n$. More generally, if
$Y_i=X_i+\mu_i$ then \[Y=\sum\limits_{i=1}^{n} (Y_i)^2 =
\sum\limits_{i=1}^{n} X_i^2 + 2 \sum\limits_{i=1}^{n} X_i \mu_i
+ \sum\limits_{i=1}^{n} \mu_i^2 =_d \chi^2(n,\lambda)\].  Where
$\lambda=\sum\limits_{i=1}^{n} \mu_i$ is non-centrality
parameter.

See the section on multivariate probability distributions for
further information, but it is worth noting that if $X =_d
N(\mu,\sigma)$ is multivariate and the variance covariance
matrix $s\sigma$ is non-singular, then $(y-\mu)^T  \sigma^{-1}
(y-\mu) =_d \chi^2(n)$ and setting $\lambda= \mu^T \sigma^{-1}
\mu$ we have $ y^T \sigma^{-1} y =_d \chi^s(n,\lambda)$

\subsection{Student-t $t(\nu)$}
$X =_d \frac{\Gamma(\frac{\nu+1}{2})}{\sqrt{\pi} \Gamma(\nu
/2)}(1+x^2)^{- \frac{\nu+1}{2}}$

%\cite{C. C. Heyde and N. N. Leonenko 2005}

\subsection{Generalized Inverse Gaussian $GIG(\lambda,\alpha,\beta )$}
The GIG distributions are characterized by\[p(x; \lambda,
\theta,\sigma)=\bigl(\frac{\theta}{\sigma}\bigr)^{\frac{\lambda}{2}}
x^{\lambda-1}\: \frac{1}{2 K_\lambda (\sqrt{\theta \sigma})} \:
exp(-\frac{1}{2} ( \theta x^{-1} + \sigma x) \].

Note,  $GIG(-1/2,\theta,\sigma)=IG(\theta,\sigma)$

The GIG family members arise as first passage time
distributions of ordinary Brownian diffusions to a constant
boundary.

\subsection{ Normalized Inverse Gaussian $NIG(\mu,\alpha,\beta,\delta)$}
$X =_d NIG(\mu,\alpha,\beta,\delta)$ if \[p(x;\mu,
\beta,\alpha,\delta)= \frac{\delta \alpha}{\pi} exp \bigl(
\delta \sqrt{\alpha^2 - \beta^2}+\beta(x-\mu) \bigr)
\frac{K_1(\alpha \; s_\delta(x-\mu))}{s_\delta(x-\mu)}\]

where $x \in \dblr \;\; \mu \in \dblr \;\; \delta>0 \;\; 0 \leq
|\beta| \leq \alpha$ and $s_\delta(x)=\sqrt{\delta^2+x^2}$ and
$K_1(x)=\frac{x}{4} \int\limits_{0}^{\infty} exp - \bigl (
y+\frac{x^2}{4 y} \bigr ) y^{-2} \;dy$ is the modified Bessel
function of the third kind.  This family of distributions is
infinitely divisible, $\exts \;\; X_t$, a Levy process, $ X_{t+
\Delta t} - X_t =_d X_{\Delta t} =_d
NIG(\mu,\alpha,\beta,\delta)$. $X_t$ is a pure jump process,
and
\[p(t; \alpha,\beta,\delta)= \biggl( \frac{\delta \alpha}{\pi |t|}
\biggr) e^{\beta t} K_1(\alpha  |t|)\]. See Eberlein and Keller
(1995) and  Barndorff-Nielsen (1998)

\subsection{Generalized Hyperbolic $GH(\lambda,\alpha,\beta,\delta,\mu)$}
The parameters $\lambda,\alpha,\beta,\delta,\mu$ have the
respective interpretation of tail heaviness, kurtosis,
skewness, and scale, and location.  The distribution includes
the important classes GIG, NIG, IG, and can be characterized as
a Normal variance-mean mixture NVMM parameterized by a GIG
distribution. Formally set $U=_d GIG()$, then $X=_d NVMM( )$ if
$P(X|U=u) =_d N(\mu+ \beta, u \Delta)$.  This gives a
stochastic representation $X= \mu+\beta Z + \sqrt{Z} Y$ where
$Y =_d N(0,1)$ and $Z =_d GIG( )$.

\section*{Limit Theorems}
Limit Theorems use the notion of a basin of attraction for pdf's in some functional space $\mathcal{H}$. In $L^2(\Omega)$, we have $N(\mu,\sigma) \subset L^2(\Omega, \nu)$ is the basin of attraction for all pdf's satisfying the conditions of the CLT.

The CLT says that the series $\frac{\sum\limits_{i=1}^{n} X_i}{n}$ converges in in probability to the mean of $x_i$. Cramer's theorem gives a bound on the probability of large deviation away from the mean in the series $\frac{\sum\limits_{i=1}^{n} X_i}{n}$.  The probability decays exponentially with a rate given by the Legendre transform of the cumulant generating function for $X_i$
\begin{thm}[Cramer's Theorem]
Let $X_1, X_2, \hdots $ be iid $E(X_i)=0$, $E(X_i^2)= \sigma^2$, and $F_n(x) = P(\frac{1}{\sigma n^{\frac{1}{2}}}  \sum\limits_{i=1}^{n} X_i < x)$, then if $x>1$ and $x=O(\sqrt{n})$ as  $n \rightarrow \infty$
we have
\begin{eqnarray*}
  \frac{1-F_n(x)}{1-\Phi(x)} = exp ( \frac{x^3}{\sqrt{n}} \lambda( \frac{x}{\sqrt{n}} )  ) [ 1 + O(\frac{x}{\sqrt{n}}) ]
\end{eqnarray*}

$\lambda(x) = \sum\limits_{i=0}^{\infty} c_i x_i$   where the $c_i$ depend on the moments of $X_i$.
$\Phi(x)$ is the distribution\ function of $N(0,1)$.
\end{thm}

\begin{thm}[Law of Large Numbers]
\end{thm}

The CLT tells us that the pdf of the scaled mean of a sample approaches the normal distribution and the Berry–Esseen theorem specifies the rate at which that happens.  The CLT requires $X_i$ to be iid, and with finite second moment and the Berry-Esseen theorem additionally requires a finite third moment.
\begin{thm}[Berry–Esseen]
Let ${X_i}$ be iid, $E(X_i^2)=\sigma$, $E(X_i^3)=\rho$, $Y_n = \frac{X_1 + \ldots + X_n}{n}$, and $F_n = \int \frac{Y_n \sqrt{n} }{\sigma}$ and $\Phi$ the CDF of $N(0,1)$, then
\begin{equation}
\abs{F_n(x) - \Phi(x)} \leq \frac{C \rho}{ \sigma^3 \sqrt{n} }
\end{equation}

\end{thm}

\section*{Multivariate Probability Distributions}

Let $S_n$ be the unit sphere in $\dblr^n$ A random variate $X$
is uniformly distributed on $S_n$ when $X$ is radially
symmetric and $||X||_{L^2} = 1 a.s.$  The pdf of a radially
symmetric random variable is necessarily of the form
$f(x_1,...,x_n)=g(||x||)$ for some $g \in [0,\infty) \ni
\int\limits_{0}^{\infty} n V_n r^{n-1} g(r) dr =1$ where
$V_n=\frac{\pi^{\frac{d}{2}}}{\Gamma(\frac{d}{2}+1) }$ is the
volume of $S_n$. If $X$ is radially symmetric , then
$\frac{X}{|||X||}$ is uniformly distributed on $S_n$.  If $X$
is uniformly distributed on $S_n$ then $(X_1^2, \ldots ,
X_{n}^{2}) =_{dist} (\frac{Y_1}{\kappa} , \ldots ,
\frac{Y_n}{\kappa} )$ where $Y_i$ iid $\Gamma(\frac{1}{2})$
with sum $\kappa$.  If $N_1, \ldots , N_n$ iid normal, then
$(N_1, \ldots , N_n)$ is radially symmetric with density $g(r)
= \frac{1}{(2 \pi)^{\frac{n}{2} } } e^{\frac{ - r^2}{2}}$  This
leads us to an algorithm for generating pseudo random variants
on uniformly distributed on $S_n$ ;
\begin{itemize}
    \item Generate $n$ iid $N(0,1)$
    \item Compute $ \kappa = ( \sum\limits_i=1^n N_i^2
        )^\frac{1}{2}$
    \item Return $(\frac{N_1}{\kappa} , \ldots ,
        \frac{N_n}{\kappa} )$
\end{itemize}
\cite{Devorye}

With a little linear algebra, the above can be generalized to a
generator for $N(\mu, \Sigma ) \in \dblr^n$.  Consider $f(x)
=\frac{1}{(2 \pi)^{\frac{n}{2} } } e^{ - \frac{1}{2} x^T \dot x
} x \in \dblr_n$, $f$ has density of $n$ iid $N(0,1)$ rv if

\subsection{Multivariate Normal $N(\mbf{\mu},\mbf{\Sigma})$ }
$N(\mbf{\mu},\mbf{\Sigma})$ is arguably the most important and tractable multivariate probability distribution. The Gaussian distribution is separable via rotation.  Precisely the rotation induced by PCA.

\subsection{Wishart Distribution}
The Wishart distribution $W(n)$ is the multivariate generalization of the $\chi^2(n)$ distribution.  If $X_{(i)} \sim N(\mbf{\mu},\mbf{\Sigma})$, then
$X X^t =S \sim W(N)$.

\subsection{Elliptic $E(\mbf{\mu},\mbf{\Sigma})$}
Elliptical distributions $E(\mbf{\mu},\mbf{\Sigma})$ extend the multivariate normal $N(\mbf{\mu},\mbf{\Sigma})$.  They can be characterized as affine maps of spherical distributions. The density functions
are defined by $p(x) = c g(  (x-\mu)' \Sigma^{-1} (x-\mu) )$  Where $g : \dblr^{+} \longrightarrow \dblr^{+}$ and $ \Sigma \succ 0$ is positive definite.
Many of the properties of the multivariate normal distribution
are shared by the elliptical distributions. Linear combinations, marginal distributions and conditional distributions of elliptical random variables can largely be determined by linear algebra using knowledge of covariance
matrix, mean and generator.

\section*{Statistical Dependence}
Linear correlation is a natural dependence measure for
multivariate normally and elliptically distributed random variables.  Other dependence concepts include rank correlation, comonotonicity, and Brownian covariance.

\section*{Distance measures for probability distribution functions.}
A number of distance measures for probability distance
functions exist.  Kullbak Lieber divergence:\[ J_D  =
\int\limits_{\text{x}} {[p({\text{x}}\mid\omega_1 ) -
p({\text{x}}\mid\omega_2 )]}
     \log \frac{{p(x\mid\omega_1 )}} {{p(x\mid\omega_2 )}}{\text{dx}}\]
which simplifies to:\[ J_D  = \tfrac{1} {2}\left( {\mu _2  -
\mu _1 } \right)^T  \left( {\Sigma _1^{^{ - 1} }  + \Sigma
_2^{^{ - 1} } } \right)\left( {\mu _2  - \mu _1 } \right) +
\tfrac{1} {2}{\text{tr}}\left\{ {\Sigma _1^{^{ - 1} } \Sigma _2
+ \Sigma _2^{^{ - 1} } \Sigma _1  - 2I} \right\}\] when \[X_1
=_d N(\mu_1,\Sigma_1) \;\; , \;\; X_2 =_d N(\mu_2,\Sigma_2).\]

The Bhattacharyya distance :
\[J_B  =  - \log \int {\left[ {p(\xi \left| {\omega _1 }
\right.)p(\xi \left| {\omega _2 } \right.)} \right]} ^{{1/2}}
{\text{d}}\xi
\] which simplifies to: \[
J_B  = \tfrac{1} {8}\left( {\mu _2  - \mu _1 } \right)^T \left(
{\frac{{\Sigma _1  + \Sigma _2 }} {2}} \right)^{ - 1} \left(
{\mu _2  - \mu _1 } \right) + \tfrac{1}
{2}{\text{log}}\frac{{\left| {\tfrac{1} {2}(\Sigma _1  + \Sigma
_2 )} \right|}} {{( {\left| {\Sigma _1 } \right|\left| {\Sigma
_2 } \right|} )^{1/2} }}
\] when \[X_1 =_d  N(\mu_1,\Sigma_1) \;\; , \;\; X_2 =_d
N(\mu_2,\Sigma_2).\]

The Matusita distance:
\[J_T  = \left\{ {\int {\left[ {\sqrt {p(\xi \left| {\omega _1 }
\right.)}  - \sqrt {p(\xi \left| {\omega _2 } \right.)} } \right]}
^2 {\text{d}}\xi } \right\}^{{1 / 2}} \] which simplifies to: \[
J_T = \left\{ {2\left[ {1 - \exp ( - J_B )} \right]}
\right\}^{{1/2 }} \] where $J_B$ is the Bhattacharyya distance,
when \[X_1 =_d  N(\mu_1,\Sigma_1) \;\; , \;\; X_2 =_d
N(\mu_2,\Sigma_2).\]

The Patrick-Fisher distance:
\[J_P  = \left\{ {\int {\left[ {p(\xi \left| {\omega _1 }
\right.)P_1  - p(\xi \left| {\omega _2 } \right.)P_2 } \right]} ^2
{\text{d}}\xi } \right\}^{{1/2}}\] which simplifies to:\[J_P =
\begin{array}{cc} {(2\pi )^d \left| {2\Sigma _1 } \right|} )^{ -
{1/2}} + ( {(2\pi )^d \left| {2\Sigma _2 } \right|} )^{ - {1/2}} -
\\ 2( {(2\pi )^d \left| {\Sigma _1  + \Sigma _2 } \right|}
)^{-{1/2}} \exp \left\{ { - \tfrac{1} {2}(\mu _2  - \mu _1
)^T(\Sigma _1  + \Sigma _2 )^{ - 1} (\mu _2  - \mu _1 )}
\right\}\end{array}\] when \[X_1 =_d N(\mu_1,\Sigma_1) \;\;  ,
\;\; X_2 =_d N(\mu_2,\Sigma_2).\]
 Reference: pp257, et sqq., Devijver, P.A.
\& Kittler, J (1982) "Pattern Recognition: A Statistical
Approach", Prentice Hall International, Englewood Cliffs, NJ.

\section*{$S_\alpha(\sigma,\beta,\mu)$ Stable Random Variates}

A Levy process is a stochastic process with a drift, a diffusion, and a jump component.
The Lévy–Khinchine representation of a Levy process $X_t$ with parameters $(a,\sigma^2,W)$
is given by

The Levy-Ito decompositon of a Levy process $X_t$ is a decomposition of $X_t$ into
singular, absolutely continuous, and discrete processes
\begin{eqnarray*}
 X_{ac} : X_{ac} \ll X \\
 X_s : X_s \perp X \\
 X_d : card (supp X_d) = \aleph_0
\end{eqnarray*}
via Lebesgue's decomposition theorem.

\subsection{4 definitions of stable}
\begin{itemize}
    \item If \[\exts C, D \fall A,B s.t. A X_1+BX_2=_dCX+D
        \fall X_1,X_2\] independent copies of $X$, then $X
        \in S_\alpha(\sigma,\beta,\mu)$. Furthermore $\exts
        \alpha \in (0,2] s.t. C$ satisfies
        $C^\alpha=A^\alpha+B^\alpha$, for any stable RV and
        $\fall A,B$
    \item Stable RV's satisfy a general CLT. If \[\fall n
        \geq 2 \exts C_n>0 D_n \in \dblr s.s. X_i+X_@
        \hdots X_n=_D C_n X+D_n\] where ${X_i}$ iid, then
        $X \in S_\alpha(\sigma,\beta,\mu)$
    \item If $\exts \; iid \; RV \;{Y_i}$ and \[{d_n},
        {a_n} \in BBCREVISIT^n \dblr^n s.t
        \frac{\sum\limits_{1}^{n} Y_i}{d_n}+a_n=_d X\] then
        $X \in S_\alpha(\sigma,\beta,\mu)$.
    \item If $\exts \alpha \in (0,2], \; \sigma \geq0 \;
        \beta \in [-1,1], \; \mu \in \dblr$ such that
    \[ E[e^{i \theta X}]=\int_\Omega e^{i \theta X} dX = exp\bigr(
    -\sigma^\alpha |\theta|^\alpha (1-i \beta \;sgn(\theta)\; tan
    \bigl(\frac{\pi \alpha}{2} \bigr) + i \mu \theta)\bigl) \] when $\alpha \neq 1$
    and when $\alpha =1$ we have \[E[e^{i \theta X}]=exp (
    -\sigma |\theta|(1+i \frac{2}{\pi} \beta \; sgn(\theta)
    \; ln|\theta| + i \mu \theta) )\]
\end{itemize}

\subsection{Variance Gamma Process}
$X_{VG}(t;\sigma,\nu,\theta)\theta  \gamma(t;\nu)+\sigma
W_{Y(t;\nu)}$ where $\gamma(t;\nu)$is a $\Gamma$ process.
\[P_{\gamma(t;\nu)}(x)=\frac{  x^{\frac{t}{\nu-1}} e^-\frac{x}{\nu}
} {\nu^{t/ \nu} \Gamma( t/ \nu)}\]  \[\Phi_{VG}(\omega)=E(e^{i
\omega X_{VG}}) = \frac{1}{(1-i \omega \nu \theta + \sigma^2
\nu \mu^2 / 2)^{t / \nu}}\]  We can show that the Variance
Gamma process is the difference of two independent Gamma
processes $ X_{VG}=\gamma_p - \gamma_n$ to obtain a new pdf
\[P_{\gamma(t;\nu)} =\biggl\{\begin{array}{cc} \frac{1}{\nu |x|} e^{-|x| / \eta_p} \;\;\;  x<0 \\
\frac{1}{\nu |x|} e^{-|x| / \eta_n} \;\;\;  x>0
\end{array} \].
%
%\section*{Martingales}
%Martingales are sequences of random variables $\{X_i\}$where
%the conditional expectation of $X_i$ given all the previous
%values is $X_{i-1}$.  \[ E( X_i | X_{i-1} \ldots X_0) = X_{i-1}
%\]  An ergodic stochastic process $X_t$is one where the sample
%moments $m_r = \frac{1}{N} \sum\limits_{i=1}^{N}(x_i- m)^r$
%converge to the population moments $E[X_t^r] = \int X_t^r dP$

\section*{Maximum Entropy}
Entropy in the context of information theory is expressed in units of bits, the amount of uncertainty in a yes or no question. Formally, for a sequence $ \{X_i\} \ni p_i  $ is a priori/posteriori probability of observing $X_i$ we define $H=- \sum\limits_{i} p_i log_2( p_i) $.  We can define the entropy of a probability distribution by $H=\int\limits_{\-infty}^{\infty} p(x) log( p(x) ) dx $.  We see the uniform distribution maximizes the entropy; if $p_i=\alpha \fall i$ then $\frac{\partial H}{\partial } = log p +1/p=0 $

\section*{Gaussian Processes}
$X_t$ is a Gaussian process (GP) if $\forall (t_1,...t_n)$ we have that $(X_{t_1}, ... ,X_{t_n}) \approx N(\mu, \Sigma)$. $\Sigma \sim K(x,x')$ the covariance functions.  Gaussian processes are defined by second order statistics (BBCREVISIT). $K(x,x')$ specifies a distribution on functions (in the distribution sense of functional analysis). Every covariance function is a scalar product of vectors.  Orenstein-Uhlenbeck (OU) and Brownian Motion (BM) are Gaussian processes. If a GP depends on $|x-x'|$ it is stationary.  The OU process is a stationary GP and BM is not. The covariance function of the OU process is not differentiable.  If a Gaussian process depends only on $|x-x'|$ it's called isotropic, these form an important class of
Here we list the common covariance functions (kernels) constant, linear, Gaussian noise, square exponential, and OU.
\begin{center}\begin{eqnarray*}
K_c(x,x')= const \\
K_\script{l}(x,x') = <x,x'> \\
K_{GN}(x,x') = \sigma^2 \delta_{x,x'} \\
K_{SE}(x,x') = e^{ \frac{-|d|^2}{2 \script{l}} }\\
K_{OU}(x,x') = e^{ \frac{-|d|}{\script{l}} }
\end{eqnarray*}\end{center}
Here $\script{l}$ is the length scale and $d = |x-x'|$

\section*{Discretizing Continuous Variables}

R discretization library implements Fayyad and Irani

Python https://github.com/navicto/Discretization-MDLPC

Most methods used for discretizing a continuous variable use its relationship
to another variable to determine the partitions. This is often found in
classification procedures, such as decision trees6 ] 10 and in naive Bayesian
classifiers.11,12
An entropy based method, proposed by Fayyad and Irani,6 chooses the
partitioning point s in a sorted set of continuous values to minimize the joint Ž .
entropy Ž Ž .. H X, Y of the continuous variable and the classification variable.
This is applied to the creation of decision tree structures for classification by
66 CLARKE AND BARTON
recursively finding more partitioning points top-down discretization . The Ž .
method is expanded to minimize a MDL metric to choose the partitioning
points.
Another MDL based method for discretization is described by Pfahringer.7
A set of the best partitioning points is determined by recursively partitioning the
Ž D sorted variable values to a depth D 2 y 1 partitions in a binary tree. Then .
the MDL metric is used in a best first search in this set to determine the best
partitions for decision tree classification.
A method that merges adjacent partitions of sorted variable values, according
to the x2 statistical test, is described by Liu and Setiono.8 The variable
values are sorted and initially partitioned into, at most, N intervals. The
intervals are first recursively merged according to the lowest x2 value until a
significance level of 0.05 is reached for each partition. The intervals are further
merged until a preset error rate with the classification variable is reached. If
there is only one resulting interval, the variable is not relevant to the classification
problem and is dropped. This method combines the discretization of
continuous variables with feature selection for classification.
Dougherty, Kohavi, and Sahami9 compared several discretization techniques
with decision trees and with naive Bayesian classifiers. They found that a
MDL metric, similar to that used by Fayyad and Irani,6 provided slightly better
classifications in both methods.
A metric for discretization, based upon a variable’s classification in relation
to other variables, is described by Hong.10 This metric is based upon a K
nearest neighbor clustering technique and is used to generate decision trees. An
interesting feature of this method is that it returns an optimal number of
partitions according to the metric. This is done by finding the ‘‘knee’’ of the
plotted curve of the score as a function of the number of partitions. The plot is a
concave function of the metric; the knee is the point on the plot where the
changes in the number of variable values Ž . X axis become greater than
the changes in the metric value Ž . Y axis . The concavity of a plot is exploited in
the information theoretic discretization methods developed in Section IV.
Subramonian, Venkata, and Chen,13 describe a visual framework for interactive
discretization for decision tree classification. A user can choose between
several algorithms and metrics for a classification problem, instead of being
limited to one method and metric. The choice of metrics includes cross-entropy
and the L y 1 norm between two distributions.
Pazzani11 describes a technique for iterative discretization of continuous
variables for naive Bayesian classifiers. Each continuous variable is initially
divided into five partitions. For each variable, two partitions are then merged or
a partition is divided into two partitions to find a lower classification error. This
procedure is repeated for each continuous variable until the error rate can no
longer be reduced.
Another method for constructing naive Bayesian classifiers using a MDL
metric is presented by Friedman and Goldszmidt.12 This method begins by
finding the best initial partition of a continuous variable by dividing the range
BAYESIAN BELIEF NETWORKS 67
into two partitions and then iterating the partitioning until there is no further
improvement in the MDL score top-down partitioning . The MDL metric Ž .
includes all of the variables used for classification and is repeated for each
continuous variable. Given a BBN structure, this method discretizes each
continuous variable in the Markov blanket of each classification variable. This
procedure is iterated until there is no improvement in each local MDL score.
Friedman and Goldszmidt propose that this method can be adopted to
learning BBN structure by starting with some initial discretization of each
continuous variable, learning an initial structure, and then rediscretizing as
described above. While this technique optimizes the conditional probabilities, it
depends upon the initial approximate discretization of continuous variables to
learn the correct network structure.
Extensions to current discretization techniques for classification methods
have been proposed for BBNs, but none have currently been published. Methods
for both static done in data preprocessing and dynamic done during BBN Ž .Ž
construction discretization of continuous variables are presented in Sections IV .
and V.
Often, a continuous variable has a normal, or Gaussian, frequency distribution.
The characteristics of the normal distribution are well understood and a
unified heuristic method for finding a BBN structure with both discrete variables
and continuous variables with a Gaussian distribution is described by Heckerman
and Geiger.14 This method requires the mean and variance of each
continuous variable’s values to parameterize the distribution


\documentclass[10pt,a4paper,final]{article}
\usepackage[utf8]{inputenc}
\usepackage[fixlanguage]{babelbib}
\usepackage[english]{babel}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm]{geometry}
\usepackage[round,sort,nonamebreak]{natbib} % citação

\usepackage[usenames,svgnames,dvipsnames]{xcolor}
\usepackage[pdftex]{graphicx}           % usamos arquivos pdf/png como figuras
\usepackage{float}%To make tables and figures stay where we want
\graphicspath{{./figures/}}

\title{Numerical Inverse Laplace Transform Integration on Ellipses}
\author{Pedro S. Peixoto}

\begin{document}
\maketitle

\section{Numerical Inverse Laplace Transform Integration on a Circle}

Inverse Laplace Transform (via Bromwich integral),
\begin{equation}
{ \mathfrak{ L } }^{ -1 }\left\{ \widehat { f }  \right\} =\frac { 1 }{ 2\pi i } \int _{ \gamma -i\infty  }^{ \gamma +i\infty  }{ { e }^{ st }\widehat { f } \left( s \right) ds } .
\end{equation}
From the Residual Theorem we can replace it with
\begin{equation}
{ \mathfrak{ L } }^{ -1 }\left\{ \widehat { f }  \right\}=\frac{1}{2\pi i}\int _{ { C }^{  } }{ { e }^{ st }\widehat { f } \left( s \right) ds },
\end{equation}
as long as all the poles of $\widehat{f}$ lay inside a closed contour ($C$). Following \citet{cclancy11} and \citet{clancy:PhD} one can choose a circle centered at the origin and of radius $\gamma$.
\begin{figure}[!h]
  \centering
  \hspace{1cm}
  \includegraphics[width=.5\textwidth]{Cstar}
  \caption[Closed contour $C $]{Closed contour $C$}
  \label{fig:Cstar}
\end{figure}

The integral over the contour $C $ can be solved using the parametrized circle $s(\theta)=\gamma e^{i\theta}$, $\theta \in [0,2\pi]$.
\begin{equation}
{ \mathfrak{ L } }^{ -1 }\left\{ \widehat { f }  \right\}=\frac{1}{2\pi i}\int _{ { C }^{  } }{ { e }^{ st }\widehat { f } \left( s \right) ds }=\frac{1}{2\pi i}\int _{0}^{2\pi}{ { e }^{ s(\theta)t }\widehat { f } \left( s(\theta) \right)s'(\theta) ds },
\end{equation}
where $s'(\theta)=is(\theta)$. This integral can be approximated via numerical quadrature, using Trapezoidal Rule, on the $N$  points $\theta_n=2\pi n/N$, that results in
\begin{equation}
{ \mathfrak{ L } }^{ * }_C\left\{ \widehat { f }  \right\}=\frac { 1 }{ Ni} \sum _{ n=1 }^{ N }{ { e }^{ { s }_{ n }t }\widehat { f } \left( { s }_{ n } \right) { s }'_{ n } }
\end{equation}
where,
\begin{equation}
s_{n}=\gamma e^{{i\theta_n}}, \quad s'_{n}=\gamma i e^{{i\theta_n}}=is_n.
\end{equation}

\section{Numerical Inverse Laplace Transform Integration on an Ellipse}

Now we wish to use an ellipse $E $ instead of a circle $C $ as closed contour. This ellipse will be chosen to satisfy the following properties:\\
(i) On the imaginary axis, it will be bounded by the interval $[-\gamma, \gamma]$, for $\gamma >0$.\\
(ii) On the real axis it will be bounded by the interval $[-\delta, \delta]$, where $\delta < 1$.

The first requirement ensures we can solve hyperbolic with purely imaginary eigenvalues up to the frequency defined by $\gamma$. The second requirement ensures that having larger timestep sizes in the numerical time integration will not lead to evaluations of complex functions with large real parts, avoiding floating point instabilities.

\subsection{Parameterized ellipse}

To attend the above requirements, the ellipse may be defined via specialized Joukowsky transforms. Let $\theta \in [0,2\pi]$ and $z=re^{i\theta}$ define a circle of radius $r$ on the complex plane. An ellipse, following the above requirements, maybe built as function of $\theta$ with the Joukowsky transform
\begin{equation}
w(\theta)=\frac{\gamma i}{d}\left(re^{i\theta}+\frac{e^{-i\theta}}{r}\right),
\end{equation}
where $d>0$ is yet to be determined. The $i\gamma/d$ term in the Joukowsky is responsible for a few things: $i$ shifts rotates the ellipse, so that its larger axis is placed in the imaginary axis; $\gamma/d$ is a normalization to allow that request (i) is satisfied.

The first condition (i) imposes that
\begin{equation}
|\text{Im}(w(\theta))|= \frac{\gamma }{d}\left(r+\frac{1}{r}\right)|\cos(\theta)| \leq \gamma, \quad \forall \theta \in [0, 2\pi].
\end{equation}
In the Joukowsky transform, when $r=1$, the ellipse degenerates to a line segment, in this case the interval $[-2\gamma i/d, 2\gamma i/d]$. Choosing
\begin{equation}
d=\left(r+\frac{1}{r}\right),
\end{equation}
is enough to fulfill condition (i).

To fulfill condition (ii), $r$ needs to be chosen such that
\begin{equation}
|\text{Re}(w(\theta))|=\frac{\gamma}{d}|(r^{-1}-r)\sin(\theta)|\leq\delta <1 , \quad \forall \theta \in [0, 2\pi].
\end{equation}
A condition for this to happen, already using that $d=r^{-1}+r$, is that
\begin{equation}
\frac{|r^{-1}-r|}{r^{-1}+r}<\delta /\gamma
\end{equation}
which, for $r>0$, is illustrated in the figure below.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.7]{rlimits1}
\caption{$\frac{|r^{-1}-r|}{r^{-1}+r}$}
\end{figure}

Therefore, condition (ii) is satisfied if
\begin{equation}
\sqrt{\frac{1-\delta/\gamma}{1+\delta/\gamma}} <r<\sqrt{\frac{1+\delta/\gamma}{1-\delta/\gamma}}.
\end{equation}

To avoid degenerate ellipses, we need $r\neq 1$. The symmetry around $r=1$ of the Joukowsky transform allows us to freely chose either $r>1$ or $r<1$. Here, we choose
\begin{equation}
r=\sqrt{\frac{1-\delta/\gamma}{1+\delta/\gamma}},
\end{equation}
which is adequate to ensure the imposed conditions.

With $\gamma=2$, $\delta=1$ (in practice $\delta$ should be strictly smaller than $1$, but this is just an illustration) the ellipse generated would look like the figure bellow.
\begin{figure}[h!]
\centering
\includegraphics[scale=0.6]{ellipse.png}
\label{fig:ellip2_1}
\end{figure}

\subsection{Inverse Laplace Transform}
Lets now go back to the Inverse Laplace Transform, but now with the contour defined via the ellipse contour.

Using the properties of contour integration, the Inverse Laplace transform on the ellipse maybe written as
\begin{equation}\label{MILT_ellipse}
{ \mathfrak{ L } }^{ -1 }\left\{ \widehat { f }  \right\}=\frac{1}{2\pi i}\int _{ { E }^{  } }{ { e }^{ st }\widehat { f } \left( s \right) ds }=\frac{-1}{2\pi i}\int_{0}^{2\pi} e^{s(\theta)t}\widehat { f } \left( s(\theta) \right) s'(\theta)\,d\theta,
\end{equation}
where
\begin{equation}
s(\theta)=\frac{\gamma i}{d}\left(re^{i\theta}+\frac{e^{-i\theta}}{r}\right),
\end{equation}
and
\begin{equation}
s'(\theta)=-\frac{\gamma}{d}\left(re^{i\theta}-\frac{e^{-i\theta}}{r}\right),
\end{equation}
and the minus sign in the integration ensures the contour is swept in a counter-clockwise way.

Uniformly partitioning the $[0,2\pi]$ interval on $N$ points with $\theta_n=2\pi n/N$, we may define the complex quadrature points as
\begin{equation}
s_n=\frac{\gamma i}{d}\left(re^{i\theta_n}+\frac{e^{-i\theta_n}}{r}\right), \quad n=1,2,..,N,
\end{equation}
and respectively the metric terms as
\begin{equation}
s'_n=-\frac{\gamma}{d}\left(re^{i\theta_n}-\frac{e^{-i\theta_n}}{r}\right),  \quad n=1,2,..,N.
\end{equation}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{ellipse-quadpoints64}
\caption{64 quadrature points on an ellipse with $\gamma=2$ and $\delta=1$.}
\end{figure}

The quadrature points are not uniformly (equidistantly) distributed on the ellipse contour. However, the $\theta_n$ points are equidistantly distributed on the $[0,\pi]$ interval, ensuring exponential convergence of the Trapezoidal Rule, for instance, of the following estimate for the inverse Laplace transform

\begin{equation}\label{milt_ellipse}
{ \mathfrak{ L } }^{ * }_E\left\{ \widehat { f }  \right\}=-\frac { 1 }{  N i} \sum _{ n=1 }^{ N }{ { e }^{ { s }_{ n }t }\widehat { f } \left( { s }_{ n } \right) { s }'_{ n } }
\end{equation}

\section{Constant reconstruction}

Let $\widehat{f}(s)=1/s$, such that $f(t)=1$. Then the following equalities should hold (approximately):

\begin{equation}
{ \mathfrak{ L } }^{ * }_C\left\{ 1/s \right\}=\frac { 1 }{ Ni} \sum _{ n=1 }^{ N }{ { e }^{ { s }_{ n }t }{ s }'_{ n } /s_n} \approx 1,
\end{equation}

\begin{equation}\label{milt_ellipse}
{ \mathfrak{ L } }^{ * }_E\left\{ 1/s  \right\}=-\frac { 1 }{  Ni} \sum _{ n=1 }^{ N }{ { e }^{ { s }_{ n }t }{ s }'_{ n }/s_n }\approx 1.
\end{equation}
where $s_n$ and $s'_n$ are defined differently on the circle and ellipse, as defined in the previous sections.

This reconstruction should be independent on $t$ (which will be our $dt$ in the ODE integration scheme).

We have set the a circle with radius $10$ and an ellipse with $\gamma=10$ and $\delta=0.5$, as the figure bellow.
\begin{figure}[h!]
\centering
\includegraphics[scale=0.5]{ellip_circ}
\label{fig:quad_elip_circ}
\caption{Quadrature points for circle ($\gamma=10$) and ellipse ($\gamma=10$, $\delta=0.5$) with $N=32$}
\end{figure}

The results bellow indicate how the circle provides an accurate reconstruction for small $dt$, but how it breaks down due to floating point errors when large $dt$ are used. The ellipse has larger errors, but allows larger $dt$.

\begin{figure}
\centering

\includegraphics[scale=0.5]{const_recon}
\label{fig:const}
\caption{Absolute error in the numerical inversion of the function $\widehat{f}=1/s$, which should result in $f(t)=1$ for any $t$ ($dt$). }
\end{figure}

\clearpage
\section{Exponential truncation}

\citet{cclancy11} and \citet{clancy:PhD} use a truncated Taylor series in the calculation of the exponential term,
\begin{equation}
{e }^{ { s }_{ n }t }=\sum_{j=0}^{N}\frac{(s_nt)^j}{j!},
\end{equation}
with $N$ matching the number of quadrature points. For the contour integration  on a circle, \citet{clancy:PhD} showed that the truncated series, with $N$ even, leads to exact reconstructions of polynomials, that is,
\begin{equation}
\mathfrak{ L }^{ * }_C\left(\frac{1}{s^{k+1}}\right)=\frac{t^k}{k!}, \quad 0\leq k \leq N.
\end{equation}

We explored numerically the effects of having the truncated exponential and the results are shown in  Figures \ref{fig:const_truncN} and \ref{fig:const_trunc16}, that show the errors when the exponential series is truncated with $N$ and $16$ terms respectively. The exponential series is highly sensitive to numerical floating point precision, therefore, for large $dt$, it creates numerical instabilities in both the Circle and Ellipse contours. Reducing the number of terms in the exponential series expansion (to 16) alleviates the problem of numerical stability on the circle, but deteriorates the results for the ellipse.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.5]{const_recon_truncN}
\label{fig:const_truncN}
\caption{Absolute error in the numerical inversion of the function $\widehat{f}=1/s$ using truncated exponential with $N$ terms, which should result in $f(t)=1$ for any $t$ ($dt$). }
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.5]{const_recon_trunc16}
\label{fig:const_trunc16}
\caption{Absolute error in the numerical inversion of the function $\widehat{f}=1/s$ using truncated exponential with 16 terms, which should result in $f(t)=1$ for any $t$ ($dt$). }
\end{figure}

\clearpage
\section{Purely imaginary poles}

Let $f(t)=e^{i\alpha t}$, $\alpha \in \mathbb{R}$,  so that $\widehat{f}(s)=\frac{1}{s-i\alpha}$. We now apply the numerical inverse Laplace transform (NILT), on both circle and ellipse, with $\alpha$ values approaching the limit of the contour ($\gamma=10$). See results bellow. The same contours were used as before, with an ellipse with $\delta=0.5$ (see Fig \ref{fig:quad_elip_circ}).

\begin{figure}[h!]
\centering
\includegraphics[scale=0.7]{nilt_expix_error}
\label{fig:const_trunc16}
\caption{Absolute error in the numerical inversion of the function $\widehat{f}(s)=1/(s-i\alpha)$, which should result in $f(t)=e^{i\alpha t}$ for varying $t$ ($dt$) and number of quadrature points. }
\end{figure}

\section{Relation to REXI }

The discussion above relies on the knowledge of the Laplace transform of the function one desires to invert. However, an analogous construction maybe done directly from the Cauchy Integration theorem:
\begin{equation}
f(x)=\frac{1}{2\pi i}\ointop_{C}\frac{f(z)}{z-x}dz,
\end{equation}
for $f$ holomorphic on $\mathbb{C}$, and the contour curve $C$ assumed to be closed and should be integrated in a counter-clockwise way, as before.

We can again chose $C$ to be the ellipse contour
\begin{equation}
s_n=\frac{\gamma i}{d}\left(re^{i\theta_n}+\frac{e^{-i\theta_n}}{r}\right), \quad n=1,2,..,N,
\end{equation}
and respectively the metric terms as
\begin{equation}
s'_n=-\frac{\gamma}{d}\left(re^{i\theta_n}-\frac{e^{-i\theta_n}}{r}\right),  \quad n=1,2,..,N,
\end{equation}
where $\theta_n=2\pi n/N$.

Therefore, the function $f$ can be approximated via,
\begin{equation}
f(x)=\frac{-1}{2\pi i} \int_{0}^{2\pi}\frac{f(s(\theta))}{s-x} s'(\theta) d\theta \approx
\frac{-1}{2\pi i} \sum_{n=1}^{N}\frac{f(s_n)}{s_n-x} s'_ n \frac{2\pi}{ N},
\end{equation}
Simplifying,
\begin{equation}
f(x)\approx
\frac{1}{ iN} \sum_{n=1}^{N}\frac{f(s_n)}{x-s_n} s'_ n =\frac{1}{N}\sum_{n=1}^{N}\frac{\beta_n}{x+\alpha_n},
\end{equation}
where
\begin{equation}
\beta_n=-if(s_n)s'_ n
\end{equation}
and
\begin{equation}
\alpha_n=-s_n
\end{equation}
and also,
\begin{equation}
r=\sqrt{\frac{1-\delta/\gamma}{1+\delta/\gamma}},
\end{equation}
\begin{equation}
d=r^{-1}+r,
\end{equation}
and $\gamma$ is the largest attainable imaginary value and $\delta$ is the largest attainable real value of the ellipse, as discussed before.

\clearpage

\bibliographystyle{unsrtnat}
%\selectbiblanguage{english}

\bibliography{bibliography}

\end{document}


\documentclass[10pt,a4paper]{article}
\usepackage[latin1]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\author{Daniel Frederico Lins Leite}
\title{Fibonnaci}
\newcommand{\qed}{\hfill\blacksquare}
\begin{document}
	\maketitle
\begin{align*}
	F_n >= 2^{n/2} &&\text{for $n >= 6$}
\end{align*}
\paragraph{Base}
\paragraph{TODO. CALCULATE 6 AND 7 MANUALLY}
\paragraph{Induction Step}
\begin{align*}
	F_n &>= 2^{n/2}\\
	F_n &= F_{n-1}+F_{n-2}\\
	&>= 2^{(n-1)/2} + 2^{(n-2)/2}\\
	& &= (X+2^{(n-2)/2}) + 2^{(n-2)/2} &&\text{$X>0$}\\
	& &= 2*2^{(n-2)/2}\\
	&>= 2*2^{(n-2)/2}\\
	& &=2^{\frac{n-2}{2}+1}\\
	& &=2^{\frac{n-2}{2}+\frac{2}{2}}\\
	& &=2^{\frac{n}{2}}\\
	&>= 2^{n/2}\\
	\qed
\end{align*}
\end{document}


\chapter{\textsc{Abstract Data Types}}
\section{Elements and tuples}
In Computer Science, \emph{datum} (data in plural) is a reference with a representation (syntax)  which is stored in a memory. Datum $a$ may be considered (semantically) as an element or object in some universe of discourse, i.e., universal set $U$, we write $a \in U$. When $A \subseteq U$ and $a \in U$, then $a$ carries some \emph{information} determined by $A$. The set $A$ is understood as a \emph{concept}.

\begin{example}
Consider a datum $a$ which semantically is an element in a set of digital cars (which is a universal set). One might say that $a$ is an \emph{instance of an object class ``digital car''}. However, the digital car $a$ might be considered as an object whose properties are stored in computer memory in very many memory blocks. Naturally, each memory block should be referred somehow (by variables!) and in this sense there are datum semantically forming this digital car $a$.
\end{example}

\begin{example}
An element 5 may be interpreted in many different ways: $5 \in r$ or $5 \in \mathbb{N}$.
\end{example}

\begin{example}
When data is processed, it is usually obligatory to know in which universal set each datum belongs. For example, the term (expression) $5+4$ is not reasonable if $4,5 \in \mathbf{S}$ ($\mathbf{S}$ is the set of arabic digits) and ``$+$'' is an operator on natural numbers. The operator ``$+$'' and $4,5$ are not compatible.
\end{example}

Sometimes a datum may be a \emph{tuple}. A tuple (or n-tuple) is an element of a product set, thus, $\left(a_1, \dots ,a_n \right) \in U_1 \times\cdots\times U_n$.

\begin{example}
If $V$ is a set of color names, and $M$ is a set of men's names, then $a=\left(75,\text{gray},\text{Jari} \right) \in \mathbb{N} \times V \times W$. One might say that tuple is a multidimensional datum.
\end{example}

\section{Memory and variables}
In program languages, data is stored in computer memory and data is processed using symbols called \emph{variables}. From a programmer point of view, a variable is a container, label, or frame. A variable (only a symbol) refers to a datum.\par
Program language compiler should transmit variable (symbol) to the operation system which then reserves on or more memory blocks to this symbol. Variable then refers to contents in computer memory block(s) having certain address(s), in which a datum is stored.\par
The use of variables in some context then serves as \emph{access} to corresponding data. As we have seen, datum is always of some type in the sense that the applicable processing methods variate.

\begin{example}
We have two variables $x$ and $y$, which refers to computer memory and there are datum. Remember that in computer memory blocks ONLY BITS ARE STORED. How the compiler of the current language ``knows'' that a program clause $z:=x+y$ (or cannot) be processed in a reasonable (in the sense of current program) way?
\end{example}

In program languages, the variables must be \emph{typed}. There are weakly and strongly typed program languages. \note Operators demand data types and they know themselves the type.

\section{Variables and data types}
The first clauses in a program are often declaring of variables and their types, like \verb|x::integer| or \verb|y::float|. After that, variable assignment can be processed; in this presentation, values of variables (variable assignment) are given by \verb|:=| symbol, for example \verb|x:=0|.\par
A \emph{data type s} is a symbolic name, which has a set $A$ with operations $O$ on that set as its semantics. The operations usually satisfy some laws, so, semantics of $s$ is some specified algebra $\mathcal{A}=\left(A,O \right)$. A variable $x$ of type $s$, written \verb|x::s|, will be assigned to an element $a \in A$ interpreted as \emph{a point in the algebra $\mathcal{A}$}, so we write \verb|x:=| $x$.

\begin{example}
clauses \verb|y::float| and \verb|y:=5| together stores 5 in some specified memory block, but 5 is understood now as a real number. So, the bits in this memory block are handled as a floating point number.
\end{example}

\begin{example}
The clauses \verb|z::char| and \verb|z:='c'| together could store the ASCII-code of the letter c.
\end{example}

\begin{example}
If an operation $\omega$ takes values of data types $s_1, \ldots ,s_n$ and produces values of data type $t$ then this is often written as $\omega : s_1 \times\ldots\times s_2 rightarrow t$. In this case, data types may have a many-sorted algebra as their semantics.
\end{example}

\note Some program languages (e.g. Python/Sage) process data as OBJECTS (object-oriented languages). Then, variables are often allowed to have \emph{aliases} by default.

\begin{example}
In a clause \verb|x:=5| the symbol \verb|x| refers to a memory block containing object 5 (must be created); this object has a ``nametag'' \verb|x|, i.e. this object is allowed to be referred by other variables also without copying the datum. This referencing method differs from symbol \verb|x| being a ``label of box'' which contains 5.
\end{example}

%\section{Advantages of typing}
%Programmers can focus on program language syntax and create the program syntactically using variables of reasonable data types and their operations. Unnecessary restrictions will be removed.\par
%Program languages have primitive data types (built-in data types) which define a kind of user interface to memory blocks. Which restrictions the hardware determines?\par When data types are used effectively then \emph{type erros} are avoided. For example,  leet \verb|x::interger| and \verb|y,z::char|. Then the clause \verb|z:=x+y| is reasonable if the operation ``+'' is of the form $+ \, : \text{integer} \times \text{char} \rightarrow \text{char}$.

\section{Data structures}
\subsection{What is data structure?}
Usually programs handle a large collection of different kinds of data from data type point of view. Some data may form an entity of its own, a datum, which may be called as \emph{data aggregate}. Data aggregate is referred by one variable.\par
The data itself is naturally stored in different memory blocks, but it is now reasonable to assume that there must be some relation between elements, because the data aggregate is a datum itself.\par
\emph{Data structure} is a way of organizing data such that data aggregate may be stored in and accessed from the memory efficiently and in a reasonable way from application point of view. The structure may be expressed as a relation between elements.
\subsection{Tuple}
One of the simplest data structure is \emph{tuple}. Tuples are elements of a product set and in this sense it is a data aggregate. The elements in the tuple are not changeable (tuples are immutable) and the relation between elements is the identity relation.\par
\note In a tuple each element may be of different data types. It might be possible to declare $x:: s_1, \dots ,s_n$ if $x$ would create a new type from the existing ones (type constructor).
\subsection{Array}
\emph{Array} is a data structure, where the elements have position numbers. The size of an array is fixed, but elements are changeable (arrays are mutable). There are no other specific methods to manipulate arrays.\par
In a more detail, an array $f$ with the size of $n$ position is a function, such that $f(i)$ is an element at the position number $i$, thus, $f:\lbrace 0,1,\ldots ,n-1 rbrace rightarrow S$, when the stored elements are members in $S$.\par
The structure is now given as a function producing a linear order between the elements. Quite often arrays are declared like \verb|x::array[0..n]| of \verb|s|, where \verb|s| is some data type. For example, \verb|x:=[3,4,5]| would assign a 3-placed array to \verb|x| declared by \verb|x::array[0..2]| of \verb|s|, where \verb|s| represents integer data type.\par
For \emph{many-dimensional arrays}, one uses many different indices to position the elements. Notice that some modern program language do not demand array declarations, but the form of writing declares the array themselves.\par
If the array is declared, however, to be the size of $n$ positions then the operating system reserves $n$ memory blocks from the memory, even if the program would not need them all.

\section{Abstract data type}
Primitive data types, like integer or float, are sets with operations such that they are defined by hardware, operating system or compilers. These kinds of data types are called ``built-in'' such that the corresponding algebras are implemented in hardware, operating system, or compiler.\par
An \emph{abstract data type} $s$ is a symbolic name, which has a collection of \emph{mathematically isomorphic algebras} as its semantics, i.e. the operations defined for these algebras behave in a similar manner and the underlying sets are bijective.

\begin{example}
An abstract data type \emph{bool} has a collection of all 2-element Boolean algebras (mathematically isomorphic algebras) as its semantics. For example, \emph{bookl} might have algebras like $B_1=\lbrace 0,1 rbrace$ and $B_2=\lbrace \text{True},\text{False} rbrace$ with the operations on these sets behaving in the same way (like connectives in Propositional Logic).
\end{example}

\begin{example}
Consider a program language with an abstract data type \emph{bool}. The  collection of all 2-element Boolean algebras is isomorphic to $\mathcal{B}=\left( B_2,\ast , +, ',F,T \right)$ as its semantics. In this program language, the syntactic operations for bool-type are $\ast , +, ',F,T$ where $F$ is the zero element and $T$ is the unit element. We have:
\begin{verbatim}
x,y,z::bool
z = x * y
WHILE z == T
...
ENDWHILE
\end{verbatim}
These program clauses are independent from the actual implementation: it does not matter which 2-element Boolean algebra there is implemented.
\end{example}


%---------------------------Warpage-----------------------------
\section{Warpage}

Warpage is defined as
\[
q =
  1 - \min \left\{
    \left( \hat n_0 \cdot \hat n_2  \right)^3,
    \left( \hat n_1 \cdot \hat n_3  \right)^3
  \right\}
\]
which is the cosine of the minimum dihedral angle formed by
planes intersecting in diagonals (to the fourth power).

Note that if $\normvec{N_k} < DBL\_MIN$ for any $k$, we set $q = DBL\_MAX$.

\quadmetrictable{warpage}%
{$1$}%                                      Dimension
{$[0,0.7]$}%                                Acceptable range
{$[0,2]$}%                                  Normal range
{$[0,DBL\_MAX]$}%                           Full range
{$0$}%                                      Unit square
{--}%                                       Citation
{v\_quad\_warpage}%                         Verdict function name