\chapter{Hartree-Fock Theory}

\minitoc

The goal of electronic structure theory is to solve the ``clamped-nuclei'' Schr\"odinger equation
\begin{align}
\label{eq:electronic-schrodinger-equation}
  \op{H}\Y_k
=&\
  E_k\Y_k
\end{align}
where
\begin{align}
  \op{H}
=&\
  V_{\mr{nuc}}
+
  \op{H}_e
=
  \sum_{a<b}^{\text{nuc.}}
  \fr{Z_aZ_b}{|\bo{R}_a-\bo{R}_b|}
-
  \fr{1}{2}
  \sum_i^{\text{elec.}}
  \nabla_i^2
-
  \sum_a^{\text{nuc.}}
  \sum_i^{\text{elec.}}
  \fr{Z_a}{|\bo{R}_a-\bo{r}_i|}
+
  \sum_{i<j}^{\text{elec.}}
  \fr{1}{|\bo{r}_i-\bo{r}_j|}
\end{align}
with an optimal balance of accuracy and efficiency for the problem of interest.

The most accurate solution possible for a given atomic orbital (AO) basis set\footnote{cc-pVXZ, 6-31G, ANO1, etc.} results from expanding the wavefunction
\begin{align}
\label{eq:full-ci-wavefunction-expansion}
  \Y_k
=&\
  \sum_\mu
  \F_\mu c_{\mu k}
\end{align}
in terms of all possible Slater determinants \index{Slater determinant} $\F_\mu$ that can be formed from an orthonormal one-electron basis of spin-orbitals, $\{\y_p\}$.
The expansion coefficients $(\bo{c})_k=c_{\mu k}$ are eigenvectors of the matrix $(\bo{H})_{\mu\nu}=\ip{\F_\mu|\op{H}|\F_\nu}$, which is the matrix representation of the Hamiltonian in the determinant basis.
This is called the \emph{full configuration-interaction} (FCI) solution.

Any one-electron basis spans the same ``function space'' as the AO basis set itself, and the full $n$-electron basis $\{\F_\mu\}$ spans the same space of $n$-electron functions regardless of how one forms spin orbitals from the AO basis set.
As a result, one obtains the same FCI solution for any choice of spin-orbitals.
In general, however, FCI solutions are completely unfeasible for basis sets of sufficient size to approach the complete basis set limit.
One can think of this as a simple counting problem: if there are $m$ functions in the AO basis, then there are $2m$ spin-orbitals in the one-electron basis,\footnote{$m$ $\a$-orbitals and $m$ $\b$-orbitals.} and there are ``$2m$ choose $n$''\footnote{The number of unique sets of $n$ marbles that can be drawn from a bag of $2m$ marbles. See \url{http://en.wikipedia.org/wiki/Combination}}
\begin{align*}
{2m \choose n}\equiv\fr{(2m)!}{n!(2m-n)!}
\end{align*}
unique Slater determinants in the $n$-electron basis that can be formed from the spin MOs.
The upshot is that we usually have to omit some Slater determinants in order to get an answer in a reasonable amount of time.

As soon as we truncate our determinant expansion (\ref{eq:full-ci-wavefunction-expansion}), our choice of spin MOs makes a significant difference in the quality of our results.
In particular, we need to choose our set of one-electron functions to minimize the number of Slater determinants it takes to ``get close to'' the exact wavefunction.

\section{The Hartree-Fock optimization problem}

It can be shown that optimizing $\ip{\Y|\op{H}_e|\Y}$ by varying $\Y$ subject to the normalization constraint $\ip{\Y|\Y}=1$ is equivalent to solving the Schr\"odinger equation.
When we further constrain the form of $\Y$ this is no longer true, but it \textit{does} generally allow us to get the best approximation to $\Y$ for a given approach (or ``Ansatz'').

In order to make the wavefunction expansion converge with a relatively small number of $\F_\mu$s, we wish to find the best single-determinant approximation to $\Y$.
That is, we wish to optimize
\begin{align}
  \ip{\F|\op{H}_e|\F}
  \F(1,\ld,n)
=
\fr{1}{\sqrt{n!}}
\left|\ar{%
  \y_1(1)&\y_2(1)&\cd&\y_n(1)\\
  \y_1(2)&\y_2(2)&\cd&\y_n(2)\\
  \vd    &\vd    &\dd&\vd    \\
  \y_1(n)&\y_2(n)&\cd&\y_n(n)}\right|
\end{align}
with respect to variation of the orbitals $\{\y_p\}$, enforcing the normalization constraint by keeping the spin orbitals orthonormal.
Note that here the function argument $(i)$ is shorthand for $(\bo{r}_i,s_i)$ where $\bo{r}_i$ denotes the position of the $i\eth$ electron and $s_i$ denotes its spin.
This optimization problem is the idea behind \textit{Hartree-Fock theory}.

Once we have solved for the Hartree-Fock optimization problem, the expectation value $\ip{\F|\op{H}_e|\F}$ is itself a good first approximation to the electronic energy.
More importantly, however, when we use this new set of Hartree-Fock spin-orbitals, $\{\y_p\}$, the FCI expansion tends to converge much more quickly to the true wavefunction.
Specifically, when we rewrite equation~\ref{eq:full-ci-wavefunction-expansion} in terms of single $\{\F_i^a\}$, double $\{\F_{ij}^{ab}\}$, triple $\{\F_{ijk}^{abc}\}$, etc.\ replacements\footnote{It is typical to use dummy indices $i,j,k,l$ to count over the orbitals in the reference determinant $\F$ -- the ``occupied orbitals'' -- and to use $a,b,c,d,$ to count over the orbitals not contained in $\F$ -- the ``unoccupied'' or ``virtual orbitals.''  Dummy indices $p,q,r,s$ are generally used to count over the full set of spin-orbitals, whether occupied or not.} of the orbitals in the Hartree-Fock determinant $\F$ with the remaining orbitals in the basis
\begin{align}
  \Y
=
  \F
+
  \sum_{\substack{a\\i}}
  \F_i^ac_a^i
+
  \sum_{\substack{a<b\\i<j}}
  \F_{ij}^{ab}c_{ab}^{ij}
+
  \sum_{\substack{a<b<c\\i<j<k}}
  \F_{ijk}^{abc}c_{abc}^{ijk}
+\ld
\end{align}
the coefficients tend to be very small, and are often virtually negligible for higher than quadruple replacements.

\section{The Hartree-Fock equations}

The electronic Hamiltonian $\op{H}_e$ contains one- and two-electron operators.
\begin{align}
  \op{H}_e
=&\
  \sum_i
  \op{h}(i)
+
  \sum_{i<j}
  \op{g}(i,j)
&
  \op{h}(i)
&\equiv
-
  \fr{1}{2}
  \nabla_i^2
+
  \sum_a
  \fr{Z_a}{|\bo{r}_i-\bo{R}_a|}
&
  \op{g}(i,j)
&\equiv
  \fr{1}{|\bo{r}_i-\bo{r}_j|}
\end{align}
Its expectation value with respect to a single determinant $\F$ is given by the first Slater rule
\begin{align}
\label{eq:slater-rule-1}
  \ip{\F|\op{H}_e|\F}
=
\sum_{i}^n
  \ip{\y_i|\op{h}|\y_i}
+\fr{1}{2}\sum_{ij}^n
  \ip{\y_i\y_j||\y_i\y_j}
&&
  \ip{\y_p\y_q||\y_r\y_s}
\equiv
  \ip{\y_p\y_q|\y_r\y_s}
-
  \ip{\y_p\y_q|\y_s\y_r}
\end{align}
where the one- and two-electron integrals are defined as follows.
\begin{align}
\label{eq:one-and-two-electron-integrals}
  \ip{\y_p|\op{h}|\y_q}
\equiv&\
  \int
  d(1)
  \y_p^*(1)
  \op{h}(1)
  \y_q(1)
&
  \ip{\y_p\y_q|\y_r\y_s}
\equiv&\
  \int
  d(1)d(2)
  \y_p^*(1)\y_q^*(2)
  \op{g}(1,2)
  \y_r(1)\y_s(2)
\end{align}
We wish to optimize equation~\ref{eq:slater-rule-1} while constraining the orbitals to be normalized and orthogonal.\footnote{The $\overset{!}=$ sign means ``must equal'' -- these are conditions to be satisfied.}
\begin{align}
  \ip{\y_i|\y_j}
\overset{!}{=}
  \delta_{ij}
\end{align}
The corresponding Lagrangian functional (see \cref{app:constrained-optimization}) is
\begin{align}\label{eq:hartree-fock-lagrangian}
  \mc{L}[\{\y_i\},\{\y_i^*\},\{\ev_{ij}\}]
=
  \sum_{i=1}^n
  \ip{\y_i|\op{h}|\y_i}
+
  \fr{1}{2}
  \sum_{i,j=1}^n
  \ip{\y_i\y_j||\y_i\y_j}
-
  \sum_{i,j=1}^n
  \ev_{ij}(\ip{\y_i|\y_j}-\delta_{ij})
\end{align}
where $\{\ev_{ij}\}$ are our Lagrangian multipliers for the orthonormality constraint.
% Note that the complex conjugates of the orbitals $\{\y_i^*\}$ are included as separate arguments of $\mc{L}$, since the real and imaginary components of $\y_i$ can be varied independently.\footnote{One can explicitly show that for complex variables $\dpd{z}{z^*}=0$.  See \url{https://en.wikipedia.org/wiki/Wirtinger_derivatives#Functions_of_one_complex_variable}.}

The stationarity conditions for the Hartree-Fock Lagrangian are (see \cref{app:functional-derivatives})
\begin{align}\label{eq:hartree-fock-lagrangian-stationarity}
\left.
  \fr{d\mc{L}[\y_k^*+\e \h^*]}{d\e}
\right|_{\e=0}
\overset{!}=
  0
\hspace{10pt}
  \text{and}
\hspace{10pt}
\left.
  \fr{d\mc{L}[\y_k+\e \h]}{d\e}
\right|_{\e=0}
\overset{!}=
  0
\hspace{10pt}
  \text{for all $\h$}
&&
  k=1,\ld,n
\end{align}
which can be stated in words as follows: For each orbital $\y_1,\ld,\y_n$ in the determinant $\F$, mixing in a little bit of an arbitrary function $\eta=\eta(\bo{r},s)$ doesn't change the Lagrangian.

Separating out the terms in \cref{eq:hartree-fock-lagrangian} involving a particular orbital $\y_k$, we can write
\begin{align*}
  \mc{L}
=&\
  \ip{\y_k|\op{h}|\y_k}
+
  \sum_i
  \ip{\y_k\y_i||\y_k\y_i}
-
  \sum_i
  \ev_{ki}(\ip{\y_k|\y_i}-\d_{ki})
-
  \sum_{i\neq k}
  \ev_{ik}(\ip{\y_i|\y_k}-\d_{ik})
\\&\
+
  \sum_{i\neq k}
  \ip{\y_i|\op{h}|\y_i}
+
  \fr{1}{2}
  \sum_{i\neq k,j\neq k}
  \ip{\y_i\y_j||\y_i\y_j}
-
  \sum_{i\neq k,j\neq k}
  \ev_{ij}(\ip{\y_i|\y_j}-\d_{ij})
\end{align*}
using $\ip{\y_k\y_i||\y_k\y_i}=\ip{\y_i\y_k||\y_i\y_k}$, which follows from exchanging integration variables in \cref{eq:one-and-two-electron-integrals}.
The functional directional derivative for varying $\y_k^*$ along $\h^*$ is then
\begin{align*}
\left.
  \fr{d\mc{L}[\y_k^*+\e \eta^*]}{d\e}
\right|_{\e=0}
=
\left.
\fr{d}{d\e}
\pr{
  \ip{\y_k+\e\eta|\op{h}|\y_k}
+
  \sum_i
  \ip{(\y_k+\e\eta)\y_i||\y_k\y_i}
-
  \sum_i
  \ev_{ki}\ip{\y_k+\e\eta|\y_i}
}
\right|_{\e=0}
\end{align*}
where we have dropped $\e$-independent terms since their derivatives vanish.
Evaluating the right-hand side gives
\begin{align*}
\left.
  \fr{d\mc{L}[\y_k^*+\e \h^*]}{d\e}
\right|_{\e=0}
=&\
  \ip{\eta|\op{h}|\y_k}
+
  \sum_i
  \ip{\eta\y_i||\y_k\y_i}
-
  \sum_i
  \ev_{ki}
  \ip{\eta|\y_i}
\end{align*}
where the first two terms can be written as follows.\footnote{Defining $\ip{\y_p(2)|\op{g}(1,2)|\y_q(2)}\equiv\int d(2) \y_p^*(2)\op{g}(1,2)\y_q(2)$}
\begin{align*}
  \ip{\eta|\op{h}|\y_k}
+
  \sum_i
  \ip{\eta\y_i||\y_k\y_i}
=
\int d(1)
  \eta^*(1)
  \pr{
    \op{h}(1)
  +
    \sum_i
    \ip{\y_i(2)|\op{g}(1,2)(1-\op{P}(1,2))|\y_i(2)}
  }
  \y_k(1)
\end{align*}
Here, the \textit{coordinate exchange operator} $\op{P}(1,2)$ allows us to write $(1-\op{P}(1,2))\y_i(2)\y_k(1)$ as a shorthand for $\y_i(2)\y_k(1)-\y_i(1)\y_k(2)$.
The expression in parentheses constitutes the \textit{Fock operator}.\footnote{
You may see this written as
$\op{f}(1)=\op{h}(1)+\sum_i(\op{J}_i(1)-\op{K}_i(1))$
where $\op{J}_i(1)\equiv\ip{\y_i(2)|\op{g}(1,2)|\y_i(2)}$ and $\op{K}_i\equiv\ip{\y_i(2)|\op{g}(1,2)\op{P}(1,2)|\y_i(2)}$ are the \textit{Coloumb} and \textit{exchange operators}.
}
\begin{align}
\label{fock}
  \op{f}(1)
\equiv
  \op{h}(1)
+\sum_i
  \ip{\y_i(2)|\op{g}(1,2)(1-\op{P}(1,2))|\y_i(2)}
\end{align}
Note that it implicitly depends on orbital set, $\op{f}=\op{f}[\{\y_i\}]$.

Using a similar procedure for the variation of $\y_k$ along $\h$, \cref{eq:hartree-fock-lagrangian-stationarity} evaluates to
\begin{align*}
  \int
  d(1)
  \eta^*(1)
  \pr{
    \op{f}(1)\y_k(1)
  -
    \sum_i
    \ev_{ki}
    \y_i(1)
  }
\overset{!}=0
\hspace{10pt}
  \text{and}
\hspace{10pt}
  \int
  d(1)
  \pr{
    \y_k^*(1)
    \op{f}(1)
  -
    \sum_i
    \y_i^*(1)
    \ev_{ik}
  }
  \eta(1)
\overset{!}=0
\hspace{10pt}
  \text{for all $\h$}
\end{align*}
which, by the Fundamental Lemma of Calculus of Variations (\cref{app:fundamental-lemma-of-calculus-of-variations}), is equivalent to the following
\begin{align}\label{eq:hartree-fock-penultimate-stationarity-condition}
  \op{f}(1)
  \y_k(1)
\overset{!}{=}
  \sum_i
  \ev_{ki}
  \y_i(1)
\hspace{20pt}
  \text{and}
\hspace{20pt}
  \op{f}(1)
  \y_k^*(1)
\overset{!}{=}
  \sum_i
  \ev_{ik}
  \y_i^*(1)
\end{align}
using the Hermitian-ness of the Fock operator, $\ip{\y_k|\op{f}\eta}=\ip{\op{f}\dg\y_k|\eta}=\ip{\op{f}\y_k|\eta}$.
Subtracting the complex conjugate of the right equation from the left gives
\begin{align*}
  \sum_i
  (\ev_{ki}-\ev_{ik}^*)\y_i(1)
\overset{!}=
  0
\end{align*}
which, since the orbitals are linearly independent,\footnote{\url{http://en.wikipedia.org/wiki/Linear_independence\#Definition}}
implies that $\bm{\ev}=[\ev_{ij}]$ forms a Hermitian matrix.
\begin{align}
  \ev_{k1}-\ev_{1k}^*=\cd=\ev_{kn}-\ev_{nk}^*=0
\end{align}
Requiring the multiplier matrix to be Hermitian makes the second condition in \cref{eq:hartree-fock-penultimate-stationarity-condition} redundant, so that the final \textit{Hartree-Fock equations} can be expressed as follows.
\begin{align}\label{eq:hartree-fock-noncanonical-stationarity-condition}
  \op{f}\y_i
\overset{!}=&\
  \sum_j
  \ev_{ij}
  \y_j
\hspace{10pt}
  \text{and}
\hspace{10pt}
  \bm{\ev}
\overset{!}=
  \bm{\ev}\dg
\end{align}
To review, these conditions define orbitals which optimize $\ip{\F|\op{H}_e|\F}$ subject to the constraint $\ip{\y_i|\y_j}=\d_{ij}$.

\subsection{The canonical Hartree-Fock equations}

\Cref{app:hartree-fock-orbital-invariance} shows that the Hartree-Fock energy and the orthogonality relations are invariant to unitary mixing of the orbitals in $\F$.
This implies that the solution to the Hartree-Fock optimization problem is not unique, because any unitary transformation of the orbitals in $\F$ is also a solution.
In this section we show how to use this freedom to our advantage, by choosing orbitals which diagonalize the Lagrange multiplier matrix, partially decoupling \cref{eq:hartree-fock-noncanonical-stationarity-condition}.
These orbitals are known as \textit{canonical Hartree-Fock orbitals}.

In matrix notation, the Hartree-Fock equations can be written as follows.
\begin{align}\label{eq:hartree-fock-noncanonical-stationarity-condition-matrix-form}
  \op{f}\bm\y
\overset{!}=
  \bm{\ev}\bm{\y}
\hspace{10pt}
  \text{and}
\hspace{10pt}
  \bm{\ev}=\bm{\ev}\dg
&&
  \bm{\ev}
=
  \ma{
    \ev_{11}&\cd&\ev_{1n}\\
    \vd&\dd&\vd\\
    \ev_{n1}&\cd&\ev_{nn}
  },
\hspace{10pt}
  \bm\y
=
  \ma{\y_1\\\vd\\\y_n}
\end{align}
Since the matrix $\bm{\ev}$ is Hermitian, it can be diagonalized by a unitary transformation $\bo{U}$.
\begin{align}
  \bm{\ev}
=
  \bo{U}\tl{\bm{\ev}}\bo{U}\dg
&&
  \tl{\bm{\ev}}
=
  \ma{
    \ev_1& 0 & \cd & 0\\
    0 & \ev_2 &\cd & 0\\
    \vd & \vd &\dd & \vd \\
    0 & 0 & \cd & \ev_n
  }
\end{align}
Inserting this decomposition into \cref{eq:hartree-fock-noncanonical-stationarity-condition-matrix-form} and multiplying both sides from the left by $\bo{U}\dg$, we get
\begin{align*}
  \op{f}(\bo{U}\dg\bm\y)
=
  \tl{\bm{\ev}}(\bo{U}\dg\bm\y)
\end{align*}
which shows that the problem can be decoupled by using a new set of orbitals  $\tl\y_1,\ld,\tl\y_n$, defined as follows.\footnote{In matrix notation this reads $\tl{\bm\y}=\bo{U}\dg\bm\y$.}
\begin{align}
  \tl\y_i
=
  \sum_{j=1}^n U_{ji}^*\y_j
\end{align}
It can be shown that the Fock operator $\op{f}$ is invariant to this type of transformation (see \cref{app:hartree-fock-orbital-invariance}).

Substituting the new orbitals into \cref{eq:hartree-fock-noncanonical-stationarity-condition-matrix-form} and dropping tildes yields the \textit{canonical Hartree-Fock equations}.
\begin{align*}
  \op{f}\y_i
=
  \ev_i\y_i
\sp
  i=1,\ld,n
\end{align*}
Since $\bm{\ev}$ is Hermitian, the Lagrangian eigenvalues are real.
Note that these equations are not fully decoupled, since $\op{f}$ still depends on the full orbital set $\{\y_i\}$.
Solving them amounts to solving for the \emph{self-consistent field}
\begin{align}
  \op{v}(1)
\equiv
\sum_i
  \ip{\y_i(2)|\op{g}(1,2)(1-\op{P}(1,2))|\y_i(2)}
=
\sum_i
  (\op{J}_i(1)-\op{K}_i(1))
\end{align}
in $\op{f}=\op{h}+\op{v}$ that allows all $n$ equations to hold true simultaneously.


%----------WORK EXPERIENCE
\section{Work Experience}
  \resumeSubHeadingListStart
    \resumeSubheading
      {Research Technologist}{Mar. 2017 -- Present}
      {Los Alamos National Laboratory -- Computational Earth Science group (EES-16)}{Los Alamos, NM}
      \resumeItemListStart
        \resumeItem{
          Project lead and primary developer on \textit{TINerator}, an open-source \textbf{Python} module for generating multi-scale geologic \textbf{3D polygonal meshes} from \textbf{GIS} \& \textbf{geospatial} data for use in flow and transport simulation codes -- widely used across U.S. D.O.E. national laboratories and presented in numerous conferences, lectures, and journals
        }
        \resumeItem{
          Project lead and primary developer on \textit{VORONOI}, an open-source \textbf{Fortran90} application for \textbf{MPI}-based ``embarrassingly parallel'' generation of Voronoi tessellations from polygonal meshes, formatted for use in various multi-physics numerical models -- presented at several conferences and has national \& international users
        }
        \resumeItem{
          Reduced computation time by $>$100x by leading development on an internal \textbf{Julia} module containing a \textbf{novel algorithm} for parallel computation of fire spread behavior from geospatial data
        }
        \resumeItem{
          Reduced mesh generation time by $\sim$40\% by designing high-performing \textbf{algorithms} in \textbf{C, C++, and Fortran} for polygonal mesh attribute interpolation \& dynamic sub-mesh querying and extraction
        }
        \resumeItem{
          Reduced model setup and analysis time by $\sim$70\% by developing an \textbf{award-winning program} for interactive \textbf{2D} \& \textbf{3D visualization} + \textbf{analysis} of geospatial data
        }
        \resumeItem{
          Dramatically increased the portability and stability of a mesh generation codebase by writing \textbf{C/FORTRAN bindings}, adding support for new compilers, improving \textbf{unit} and \textbf{regression} testing, and migrating the build system from Make to \textbf{CMake}
        }
        \resumeItem{
          Designed and deployed \textbf{static websites}, \textbf{API documentation}, \textbf{unit tests}, and \textbf{CI/CD} for over five internal codebases
        }
        \resumeItem{
          Responsible for the end-to-end procurement, setup \& deployment of an \textbf{Ubuntu Linux} web server hosting an internal group wiki
        }
        \resumeItemListEnd
    \resumeSubheading
      {Patent Analyst}{May 2016 -- Jan. 2017}
      {Global Patent Solutions, LLC}{Scottsdale, AZ}
      \resumeItemListStart
        \resumeItem{
          Performed extensive database research on technical literature related to the patentability of client invention concepts in the domains of nanotechnology, semiconductors, and electromagnetic elements
        }
        \resumeItem{
          Prepared and submitted comprehensive reports on patentability \& technical research findings to clients
        }
      \resumeItemListEnd
    \resumeSubheading
      {Graduate Research Assistant}{Jun. 2015 -- Dec. 2015}
      {Arizona State University -- School of Earth \& Space Exploration (SESE)}{Tempe, AZ}
      \resumeItemListStart
        \resumeItem{
          Developed an \textbf{Artificial Neural Network} which, when deployed on a cluster of drones acting as a ``mesh network'', finds and directs the drones to the optimal spatial configuration for maximum network coverage
        }
        \resumeItem{
          Reduced neural network training \& execution time by $\sim$70\% by implementing \textbf{shared-memory parallelization} with \textbf{OpenMP}
        }
        \resumeItem{
          Further reduced execution time by $\sim$50\%, while also decreasing technical debt and code complexity, by \textbf{profiling} runtime characteristics with Valgrind/Callgrind and \textbf{refactoring} high-cost functions
        }
        \resumeItem{
          Developed a \textbf{Python} script to visualize neural network behavior by interfacing with \textbf{Blender} to render drone movement in 3D
        }
      \resumeItemListEnd
  \resumeSubHeadingListEnd


\section{The \acrlong{nrpa}}%
\label{sec:the_nrpa}

The \gls{nrpa}\cite{nrpa} is a \gls{mcts} algorithm that has been used to solve optimization problem with state of the art results like the \gls{tsptwm}\cite{nrpa_tsptw}.
Howether, it is not a traditional \gls{mcts} algorithm as it does not follow the classic steps of these algorithms: selection, expansion, simulation and backpropagation (see section~\ref{sub:key_components_of_a_mcts}).
Even the tree procedure is very different from other \gls{mcts} algorithm.
The \gls{nrpa} learns a playout policy used during the rollouts.
Actions are associated to weights and sampled during rollout according to the exponential of this same weight.
Doing so, it find the best sequence of actions it can and modifiy its policy to adapt it to the best known sequence.
In the end, it returns the best sequence found.

This algorithm is very effective to solve non stochastic optimization problem and has found new records for game like Morpion Solitaire.
In this section, we will explain the \gls{nrpa} in three parts: the playout, the adaptation and the core function.

\subimport{./subs/}{playout.tex}
\subimport{./subs/}{adapt.tex}
\subimport{./subs/}{core.tex}
\subimport{./subs/}{code.tex}


\section{Gibb's sampling}


\section{The auxiliary functions \heapparent and \heapchild}
\label{sec:auxiliary-heap-functions}
\label{sec:heapparent}
\label{sec:heapchild}

This section features the two auxiliary heap functions
We start with the function \specref{heapparent}
which is in principle the \isoc~counterpart of the \acsl function \logicref{HeapParent}.
We say \emph{in principle} because our definition avoids
the border case of the parent node of~0.

\input{Listings/heap_parent.h.tex}

Neither do we provide exact \isoc-counterparts for
the logic functions \logicref{HeapLeft} and \logicref{HeapRight}.
In fact, we have encountered only one situation (in the implementation of \implref{popheap}),
where such functions would have been useful.
However, what we really need in \popheap is to determine
for a given index~\inl{p} a child index~\inl{c} where the maximum
of the respective values \inl{a[HeapLeft(p)]} and\\
\inl{a[HeapRight(p)]} resides.
This computation is performed by the function \specref{heapchild}.

\input{Listings/heap_child.h.tex}

\clearpage

Note that in the implementation of \implref{heapchild}
we explicitly handle the case that the computation of child indices
could overflow. If this occurs, the function \heapchild returns~\inl{n}.

\input{Listings/heap_child.c.tex}


\section{Stoke's theorem}


%\ifthenelse{\value{noa} = 1}
%%...................then
%{\chapter*{APPENDIX: THIS IS THE FIRST APPENDIX}
%\addcontentsline{toc}{chapter}{APPENDIX: THIS IS THE FIRST APPENDIX}
%\chaptermark{Appendix}
%\markboth{Appendix}{Appendix}
%\setcounter{chapter}{1}}
%%...................else
{\chapter{THIS IS THE FIRST APPENDIX}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%ADD LABEL

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Proof of Proposition 1.

(1) Since B(T,T)=1, Equation (8) yields

$$B(t,T)=e^{\theta (T-t)}\frac{E((\delta(T))^{\alpha-1}|\mathfrak{F}_{t})}{(\delta(t))^{\alpha-1}}$$

Using the fact that

$$(\frac{\delta(T)}{\delta(t)})^{\alpha-1}=exp{(\alpha-1)(\mu_{1}-\frac{1}{2}\sigma_{1}^{2})(T-t)
+\sigma_{1}(\alpha-1)(W_{1}(T)-W_{1}(t))}\prod_{i=N(t)+1}^{N(T)}\widetilde{V}_{i}^{\alpha-1}$$

$$E(\prod_{i=N(t)+1}^{N(t)}\widetilde{V}_{i}^{\alpha-1})=\sum_{j=0}{\infty}e^{-\lambda(T-t)}\frac{[\lambda(T-t)]^{j}}{j!}{\zeta_{1}^{(\alpha-1)}+1}^{j}$$
$$=exp{\lambda\zeta_{1}^{(\alpha-1)}(T-t)}$$

First equation yields

$$B(t,T)=exp[-(T-t){\theta -(\alpha-1)(\mu_{1}-\frac{1}{2}\sigma_{1}^{2})-\frac{1}{2}\sigma_{1}^{2}(\alpha-1)^{2}-\lambda\zeta_{1}^{(\alpha-1)}}]$$

Note that it implies

$$e^{-r(T-t)}=E(U_{c}(\delta(T),T)/U_{c}(\delta(t),t)|\mathfrak{F}_{t})$$

which shows that Z(t) is a martingale under P. Furthermore, it leads to

$$Z(t)=(\delta(0))^{\alpha-1}e^{(r-\theta)t}exp{(\alpha-1)(\mu_{1}-\frac{1}{2}\sigma_{1}^{2})t +\sigma_{1}(\alpha-1)(W_{1}(t))}\prod_{i=1}^{N(t)}\widetilde{V}_{i}^{\alpha-1}$$
$$=(\delta(0))^{\alpha-1}exp{-\frac{1}{2}\sigma_{1}^{2}(\alpha-1)^{2}-\lambda\zeta_{1}^{(\alpha-1)}}t
+\sigma_{1}(\alpha-1)(W_{1}(t))\prod_{i=1}^{N(t)}\widetilde{V}_{i}^{\alpha-1}$$

Now

$$\psi_{s}(t)=\frac{E(U_{c}(\delta(T),T)\psi_{s}(T)|\mathfrak{F}_{t})}{(U_{c}(\delta(t),t))}=e^{-rT}E\{\frac{Z(T)}{Z(t)}\psi_{s}(T)|\mathfrak{F}_{t}\}$$
$$=e^{-rT}E^{*}(\psi_{s}(T)|\mathfrak{F}_{t})$$

Proof of Theorem 1. The Girsanov theorem for jump-diffusion processes tells us that under $P^{*}$, $W_{1}^{\prime}(t)= W_{1}(t)-\sigma_{1}(\alpha-1)t$ is a new Brownian motion and under $P^{*}$ the jump rate of N(t) is $\lambda^{*}=\lambda E(\widetilde{V}_{i}^{\alpha-1})=\lambda (\zeta_{1}^{(\alpha-1)}+1)$ and
$\widetilde{V}_{i}$ has a new density $f_{\widetilde{V}}^{*}(x)=(1/(\zeta_{1}^{(\alpha-1)}+1))x^{\alpha-1}f_{\widetilde{V}}(x)$. Therefore, dynamics of S(t) is given by

$$\frac{dS(t)}{S(t-)}=\mu dt+\sigma\{ \rho dW_{1}(t)+\sqrt{1-\rho^{2}} dW_{2}(t)\} + \Delta (\sum_{i=1}^{N(t)}(V_{i}^{\beta}-1))$$
$$=\{\mu + \sigma_{1}\sigma\rho (\alpha-1)dt + \sigma\{\rho dW_{1}^{\prime}(t)+\sqrt{1-\rho^{2}} dW_{2}(t)\}+ \Delta (\sum_{i=1}^{N(t)}(V_{i}^{\beta}-1))$$

Because

$$E^{*}(\widetilde{V}_{i}^{\beta})=\int_{0}^{\infty}x^{\beta}\frac{1}{\zeta_{1}^{(\alpha-1)}+1}x^{(\alpha-1)}f_{\widetilde{V}}(x)dx$$
$$=\frac{1}{\zeta_{1}^{(\alpha-1)}+1}E(\widetilde{V}^{\alpha+\beta-1})=\frac{\zeta_{1}^{\alpha+\beta-1}+1}{\zeta_{1}^{\alpha-1}+1}$$

we have $$\lambda^{*}\{E^{*}(\widetilde{V}^{\beta})-1\}=\lambda(\zeta_{1}^{\alpha+\beta-1}-\zeta_{1}^{\alpha-1})$$.

Therefore

$$\frac{dS(t)}{S(t-)}=\{\mu + \sigma_{1}\sigma\rho (\alpha-1)+ \lambda(\zeta_{\alpha+\beta-1}-\zeta_{\alpha-1})\}dt$$
$$-\lambda^{*}\{E^{*}(\widetilde{V}^{\beta})-1\}dt+\sigma\{\rho dW_{1}^{\prime}(t)+\sqrt{1-\rho^{2}} dW_{2}(t)\}+ \Delta (\sum_{i=1}^{N(t)}(V_{i}^{\beta}-1))$$

Hence to satisfy the rational equilibrium requirement $S(t)=e^{-r(T-t)}E^{*}(S(T)|\mathfrak{F})$ we must have $\mu + \sigma_{1}\sigma\rho (\alpha-1)+ \lambda(\zeta_{\alpha+\beta-1}-\zeta_{\alpha-1})=r$

So, under the measure $P^{*}$, the dynamics of S(t) is given by

$$\frac{dS(t)}{S(t-)}=rdt-\lambda^{*}\{E^{*}(\widetilde{V}^{\beta})-1\}dt+\sigma\{\rho dW_{1}^{\prime}(t)+\sqrt{1-\rho^{2}} dW_{2}(t)\}+ \Delta (\sum_{i=1}^{N(t)}(V_{i}^{\beta}-1))$$


%% Dexter Barrows, 2016
%% dbarrows.github.io

\section{IF2 Parametric Bootstrapping Function}

    The parametric bootstrapping machinery used to produce forecasts.

    \lstinputlisting[style=Rsty]{../../code/stochastic-comparison/paraboot-parallel/if2-paraboot.r}

\section{RStan Forward Simulator}

    The code used to reconstruct the state estimates, then project the trajectory forward past data.

    \lstinputlisting[style=Rsty]{../../code/sir-functions/StocSIRstan.r}


%!TEX root=../Selex.tex
\section*{Methods} % (fold)
\label{sec:methods}

Simulated data were generated from an age-structured simulation model largely based on the 2010 Pacific hake assessment.  Simulated data were based on three alternative selectivity scenarios: (1) constant over time, (2) selectivity changes at four specific time-periods (blocks), and (3) selectivity changes continuously over time where the commercial fishery targets the most abundant cohort in each year.  First, we describe the model structure used to simulate data and estimate model parameters, followed by a description of the MSY-based reference points, and lastly the detailed description of the various scenario combinations explored.

\subsection*{Model description} % (fold)
\label{sub:model_description}

A statistical catch-age model was used to both generate simulated data sets and estimate model parameters based on simulated data. These simulation-estimation experiments were based on data from the Pacific hake fishery from 1977 to 2009, using the historical catch time series from US and Canada combined and the empirical weight-at-age data from this fishery available at the time \citep{Martell2009}.  The model was written in AD Model Builder \citep{fournier2011ad} and all model code and data are available from a code repository (see CAPAM branch at \url{https://github.com/smartell/iSCAM}).

Input data for the model consist of fishery removals along with age-composition information and empirical weight-at-age data from the commercial fishery.  In addition to the commercial data, a fisheries independent survey also exists and includes a relative index of abundance and age-composition information.  The actual acoustic survey for Pacific hake historically occurred every three years prior to 2001, then every two years, and since 2011 has occurred every year. For the simulation-estimation experiments we assume that fishery-independent abundance and age-composition information exist for all years.

Parameters for the simulation-estimation experiments were based on the maximum likelihood estimates of the initial numbers-at-age and annual recruitment deviations from the 2010 assessment  \citep{Martell2009}. The annual relative abundance data was assumed to be proportional to the available biomass and to have log-normal measurement errors:
\begin{equation}\label{eq:surveyIndex}
	I_t = q e^{\sigma_1\epsilon_t - 0.5\sigma_1^2} \sum_a \nu_a N_{a,t}  W_a
\end{equation}
where the random deviate is $\epsilon \sim N(0,1)$, $\sigma$ is the standard error, $\nu_a$ is the age-specific proportion that this selected by the acoustic sampling gear, $N_{a,t}$ is the numbers-at-age, and $W_a$ is the average weight-at-age.  For simplicity, the scaling parameter was fixed at $q=1$.

Age-composition data for both commercial and survey samples were randomly drawn from a multivariate distribution with a probability of $p_{a,t}$ of sampling an age-$a$ fish in a given year $t$.  The age-proportion samples must sum to 1 in each year, and random samples were based on the the following:

\begin{align}
	x_{a,t} &= \ln(\hat{p}_{a,t}) + \sigma_2 \epsilon_{a,t} - \frac{1}{A}
	\left[\sum_a \ln(\hat{p}_{a,t}) + \sigma_2 \epsilon_{a,t} \right],\nonumber \\
	p_{a,t} &= \dfrac{e^{x_{a,t}}}{\sum_{a} e^{x_{a,t}} } \label{eq:ageProportion}
\end{align}

where $\epsilon_{a,t}$ is a standard random normal deviate, $\sigma_2$ is the standard error, $\hat{p}$ is the expectation of the proportion-at-age in year $t$ in the sampled catch.

True parameter values used in the simulation model are listed in Table~\ref{table:simulationpars}.  Annual fishing mortality rates were conditioned on the observed catch from the Pacific hake fishery and it was assumed that both natural mortality and fishing mortality occur simultaneously.  Simulated age-specific fishing mortality rates were based on the annual age-specific selectivity which differs among three alternative simulation scenarios (see description in the Scenarios subsection).

\begin{table}[!tbh]
	\caption{Parameters used for simulation model in the integrated statistical catch-age model.}
	\label{table:simulationpars}
	\begin{center}
		\begin{tabular}{l|cl}
		\hline

		\hline
		\textbf{Description} & \textbf{Symbol} & \textbf{Value} \\
		\hline
			 Unfished age-1 recruits 			& $R_o$ 	 & 3.353 \\
			 Steepness (Beverton-Holt) 			& $h$ 		 & 0.727 \\
			 Natural mortality rate 			& $M$		 & 0.230 \\
			 Average age-1 recruitment			& $\bar{R}$	 & 1.300 \\
			 Initial recruitment 				& $\dot{R}$  & 0.428 \\
			 Survey standard deviation 	   		& $\sigma_1$ & 0.300 \\
			 Standard deviation in recruitment	& $\sigma_R$ & 1.120 \\
			 Age at 50\% selectivity in survey  & $\hat{a}$  & 2.500 \\
			 Std dev. in 50\% selectivity in survey  & $\hat{g}$  & 0.500 \\
			 Std dev. in age-sampling error          & $\sigma_2$ & 0.300\\
		\hline

		\hline
		\end{tabular}
	\end{center}
\end{table}

% subsection model_description (end)

\subsubsection*{Parameter estimation} % (fold)
\label{ssub:parameter_estimation}
Model parameters were estimated using maximum likelihood methods where the objective function includes additional penalties to constrain the shape of the selectivity curve and how much it is allowed to vary over time (Table \ref{tab:likelihoods}). There are 6 major components to the objective function that is being minimized: (1) the likelihood of the observed catch \eqref{eq:L1.1}, (2) the likelihood of the relative abundance index \eqref{eq:L1.2}, (3) the likelihood of the age-composition information \eqref{eq:L1.3}, (4) the likelihood of the stock-recruitment estimates given the values of steepness and unfished age-1 recruits \eqref{eq:L1.4}, (5) prior densities in negative log space for estimated model parameters\eqref{eq:L1.5}, and (6) penalties and constraints for selectivity coefficients \eqref{eq:L1.6}.
\begin{table}[!tbh]
	\tableEq
	\caption{Calculations for the various components of the objective function ($f(\Theta)$) that is being minimized in the integrated statistical catch age model.}\label{tab:likelihoods}
	\begin{align}
		\hline \nonumber\\
		&\mbox{Residuals}\nonumber\\
		w_t&= \ln(\hat{C}_t) - \ln(C_t)\label{eq:T1.1}\\
		z_t&= \ln(I_t) - \ln(B_t) - \frac{1}{I}\sum_{t\in I}\left[\ln(I_t) - \ln(B_t)\right]\label{eq:T1.2}\\
		\eta_{t,a}&= \ln(\hat{p}_{a,t}) - \ln(p_{a,t})
				 - \frac{1}{A}\sum_{a=1}^A [\ln(\hat{p}_{a,t}) - \ln(p_{a,t})]
				 \label{eq:T1.3}\\
		%
		\delta_t &= \ln(N_{1,t}) - \ln(f(R_o,h,B_{t-1})) \quad
		\mbox{for $t > 1$}\label{eq:T1.4}\\[1ex]
		%
		&\mbox{Negative loglikelihoods}\nonumber\\
		\ell(C)& = T [\ln(\sigma_C) + 0.5\ln(2 \pi) ] +\sum_{t=1}^T
		\frac{w_t^2}{2\sigma_C^2} \label{eq:L1.1}\\
		%
		\ell(I)& = I[\ln(\sigma_1) + 0.5\ln(2 \pi) ] + \sum_{t \in I}
		\frac{z_t^2}{2\sigma_C^2} \label{eq:L1.2} \\
		%
		\ell(P)& = (A-1)T \ln\left(\frac{1}{(A-1)T}\sum_{a\in p_{a,t}}
		\sum_{t\in p_{a,t}} \eta_{t,a}^2 \right)\label{eq:L1.3} \\
		%
		\ell(R)& = (T-1)[\ln(\sigma_R) + 0.5\ln(2 \pi) ] + \sum_{t =2}^T
		\frac{\delta_t^2}{2\sigma_R^2} \label{eq:L1.4} \\
		%
		{p}(\Theta) & = R_o \propto U(-5,15) + h\propto \beta(3,2)
			+ \bar{R} \propto U(-5,15) +\dot{R} \propto U(-5,15)
			\label{eq:L1.5}\\
		%
		\mathrm{P} & = \phantom{+} \lambda^{(1)}_k \sum_{a=3}^{A-1}(v_{a,t}-2v_{a-1,t} + v_{a-2,t})^2
		\nonumber\\
		\phantom{P} & \phantom{=}+ \lambda^{(2)}\sum_{A=1}^{A-1}
		\begin{cases}
			(v_{a,t}-v_{a+1,t})^2\quad \mbox{if}\quad v_{a,t} > v_{a+1,t}\\
			0\phantom{ (v_{a,t}-v_{a+1,t})} \quad \mbox{if} \quad v_{a,t} \leq v_{a+1,t}
		\end{cases}\nonumber\\
		\phantom{P} & \phantom{=}+ \lambda_k^{(3)} \sum_{t-3}^T
		(v_{a,t}-2v_{a,t-1} + v_{a,t-2})^2
		 \label{eq:L1.6}\\[1ex]
		%
		&\mbox{Objective function}\nonumber\\
		f(\Theta)& = \ell(C) + \ell(I) + \ell(P) + \ell(R) +{p}(\Theta) + \mathrm{P}\\
		\hline \nonumber
	\end{align}
	\normalEq
\end{table}

The observed catch data are assumed to have a lognormal error structure and the standard deviation in the residuals between observed and predicted log catch is fixed at 0.0707 for all years. The likelihood for the relative abundance data is assumed to have lognormal errors and the variance of the residuals is an estimated parameter.  Note that the conditional maximum likelihood estimate for $q$ is used in the likelihood calculation \citep{walters1994calculation}, and we use a weak informative prior of $\ln(q) \sim N(0,0.75)$ for the derived value of $q$ in our simulation studies to stabilize the scaling parameters in Monte Carlo trials.  Tests with this weak informative prior and a uniform prior on the true Pacific hake data yielded very similar MLE estimates.

The likelihood for age-composition information collected from commercial fisheries and the fisheries independent survey was assumed to come from a  multivariate logistic distribution, and these data were weighted by the conditional maximum likelihood of the variance.  Residual difference between observed ($\hat{p}_{a,t}$) and predicted ($p_{a,t}$) age-proportions were calculated using \eqref{eq:T1.3} with the constraint that $\sum_a \eta_{a,t} = 0$.  The advantage of this approach over a multinomial likelihood with a fixed effective sample size, is that the age-composition data are scaled by assigning all additional lack of fit to process error, and therefore weighted consistent with the model structure \citep{schnute1995influence}. An important point to note about the calculation of the age-composition residuals in \eqref{eq:T1.3} is that the function is undefined if $\hat{p}_{a,t}=0$.  The addition of a small constant to both the observed and predicted proportions seems like a reasonable solution; however, in cases where year-classes are extremely weak and only partially selected by the fishing gear, the assumed value of the constant can influence the overall result.  To avoid this problem, we alter the definition of an age-class in years where the observed proportion-at-age is 0 and pool this cohort into the adjacent age-class.  In our simulation testing, this grouping of age classes was much more robust for parameter estimation and did not appear to produce any significant biases in comparison to methods that add a small constant.  Similar results were also obtained by \cite{richards1997visualizing}, but in their simulation studies they explored pooling adjacent year classes if the observed proportion was less than 2\%.  For the case of Pacific hake, recruitment variation is sufficiently large that a weak year class following a strong year class could artificially be treated as two strong year classes.

Annual age-1 recruitment was estimated via a mean recruitment value and a vector of deviates that were constrained to sum to 0.  The integrated statistical catch age-model also jointly estimates the parameters of the resulting stock recruitment relationship given estimates of annual age-1 recruits and the  spawning stock biomass.  Residual deviations between annual recruitment and recruitment based on a Beverton-Holt stock recruitment model were calculated using \eqref{eq:T1.4}, and the unfished age-1 recruits ($R_o$) and steepness parameters ($h$) were jointly estimated based on the negative log likelihood \eqref{eq:L1.4}.  The variance parameter for recruitment deviations $\sigma_R^2$ was based on the 2007 stock assessment \citep{helser2007stock}.

Uniform priors were assumed for all the estimated parameters with the exception of an informative beta distribution for the steepness parameter in the interval 0.2--1.0, and informative priors for the variance parameters.  The expected value for the steepness prior was set at 0.6 with a standard deviation of 0.161.  For the variance parameters, we adopted a variance partitioning approach for estimating observation and process error variance. The estimated quantities consist of the total precision $\varphi^2$ and the proportion of the total variance that is associated with the observation errors $\rho$ and the total variance is partitioned as:

\begin{align}
	\sigma^2_1 &= \rho/\varphi^2 \nonumber \\
	\sigma^2_R   &= (1-\rho)/\varphi^2 \label{eq:variance}
\end{align}
The advantage of this approach over estimating $\sigma_1^2$ and $\sigma_R^2$ directly is increased numerical stability.  For the total precision an informative Gamma distribution was used as the prior $\varphi^2 \sim \Gamma(14.87,20.0)$ and a beta prior for the variance ratio $\rho \sim \beta(5.76,80.34)$.

A likelihood penalty for the selectivity parameters is defined in \eqref{eq:L1.6}.  Note that in \eqref{eq:L1.6} there are three terms, the first of which is a penalty on the second differences between the age-specific coefficients to ensure a smooth pattern.  The second term is a penalty on the amount of dome-shaped selectivity (often necessary when jointly estimating natural mortality rates).  The third term is a second difference penalty on how age-specific coefficients vary over time.  In each case, the user must specify the relative weights ($\lambda$) for each of these penalties.   For example, and infinitely large value of $\lambda^{(3)}$ would imply that selectivity coefficients are invariant over time.  For the simulation-estimation experiment, values for $\lambda^{(1)}$ and $\lambda^{(2)}$ were set at 12.5, which corresponds to a coefficient of variation of roughly 0.20.  The value of $\lambda^{(3)}$ was set at 1.0, which is equivalent to a CV of 0.71 and allows for a lot of variability in selectivity at a given age over time.

% 	1.6         -5.0    15       3       1       1.6    0.50    #log_ro/msy
% 	0.65        0.2     1.0      4       3       3       2      #steepness/fmsy
%    -1.469676   -5.0    0.0     -2       1       -1.469  0.015   #log.m
% 	1.2         -5.0    15       1       0       -5.0    15     #log_avgrec
% 	1.40        -5.0    15       1       0       -5.0    15     #log_recinit
% 	0.03090235  0.001   0.999   -3       3       5.00    20.0   #rho
% #	0.75        0.01    10.00    5       4       7.725587    10.0   #kappa (precision)
% 	0.7725587   0.01    10.00   -5       4       15.45117    20.0   #kappa (precision)

% subsubsection parameter_estimation (end)

\subsection*{Reference points} % (fold)
\label{sub:reference_points}
Reference points based on long-term maximum sustainable yield (MSY-based reference points) were calculated assuming steady-state conditions.  It was assumed that removals from the fishery independent survey were negligible.  The fishing mortality rate that produced the maximum sustainable yield was determined by setting the derivative of the catch equation to 0 and solving for $F_{\rm{MSY}}$.  MSY was subsequently determined by calculating the steady-state catch using $F_{\rm{MSY}}$.  Similarly, $B_{\rm{MSY}}$ was determined by calculating the steady-state spawning biomass under a fishing mortality rate of $F_{\rm{MSY}}$. Detailed descriptions of the steady state calculations for MSY-based reference points can be found in \cite{Martell2008pam}.

 All MSY-based reference points were based on the estimated selectivity value in the terminal year of the assessment.  In cases where selectivity is assumed to remain constant over time, the estimated MSY-based reference points vary with minor updates to population parameters as the time series increases in length.  However, in cases where selectivity is assumed to vary over time, MSY-based reference points become highly uncertain as the uncertainty in selectivity in the terminal year is a function of how much selectivity is allowed to vary.

% subsection reference_points (end)

\subsection*{Scenarios} % (fold)
\label{sub:scenarios}
% Some stuff from earlier:
% For each simulated data set, four alternative assessment models using different assumptions about selectivity were taking into consideration.  Initial numbers-at-age and annual recruitment deviates were fixed at the maximum likelihood estimates in all simulations; the only difference between simulations are the observation errors.

Three alternative datasets were generated with the simulation model using: (1) fixed length-based selectivity based on an asymptotic logistic function and observed length-at-age data from commercial samples, (2) four discrete time blocks where the same asymptotic length-based function changes in 1986, 1999, and 2001, and (3) continuous changes in selectivity each year where the fishery targets cohorts based on Ideal Free Distribution (IFD).  We refer to these as scenario (1), scenario (2) and scenario (3), throughout the text.  For simulations (1) and (3) the length-at-50\% vulnerability for the logistic function was set at 40 cm and a standard deviation of 1.5.  For the discrete time blocks, the length-at-50\% vulnerability ($L_{50}$) was set at 45, 40, 50 and 40 cm for each of the four blocks, respectively.  The standard deviations in $L_{50}$ were fixed at 2.5, 1.9, 2.5, and 3.0 cm, respectively.  For the IFD selectivity model, the age-specific selectivity coefficients were based on age-specific biomass that is vulnerable to the fishing gear.  Given a vector of selectivity coefficients $v_a$ (based on the same length-based logistic function used in the other scenarios), the age-specific selectivity coefficients each year were based on the relative biomass-at-age $b_a$ in a given year, and rescaled such that the mean of the vector is equal to 0 in log-space:
\begin{equation}\label{eq:ifsSelex}
	\omega_a = \ln(v_a) + 0.25\ln(b_a) -
	\frac{1}{A}\sum_{a=1}^A \left[ \ln(v_a) + 0.25\ln(b_a)\right]
\end{equation}
The coefficient of 0.25 is an arbitrary scaling of the biomass-at-age that would relate price premiums to larger size fish.  The larger the price premium the less dome-shaped the selectivity curve would be because there is a financial incentive to target larger more valuable fish that are less abundant.  In any case, the unique feature of \eqref{eq:ifsSelex} is that it allows for modal and multi-modal selectivity curves based on the relative abundance of each cohort (Figure \ref{fig:simSelex}).

\begin{figure}[!tbh]
	\begin{center}
		\includegraphics[width=0.37\textwidth]{../FIGS/fig:simSelex1.png}
		\hspace{-1.25cm}
		\includegraphics[width=0.37\textwidth]{../FIGS/fig:simSelex2.png}
		\hspace{-1.25cm}
		\includegraphics[width=0.37\textwidth]{../FIGS/fig:simSelex3.png}
	\end{center}
	\caption{True selectivity curves used to generate simulated data sets for scenario 1 (left), scenario 2 (middle), scenario 3 (right).}
	\label{fig:simSelex}
\end{figure}

Four alternative selectivity states were assumed in the assessment model (denoted by the letters a-d).  For the fixed scenario, model (a), the probability of catching an individual of a given length was assumed to be constant over time.  Length-based selectivity was based on estimating seven equally spaced selectivity coefficients (or knots) starting at the length at age-1 and ending at the length of age-15.  A cubic spline function was then used to interpolate between these knots to get the corresponding age-specific selectivity values based on the empirical length-at-age from fisheries samples.  An additional penalty was added to the objective function to ensure a smooth function and penalize  dome-shape selectivity curve; values for $\lambda^{(1)}$ and $\lambda^{(2)}$ were both set at 12.5 which is roughly equivalent to a 20\% coefficient of variation.  The resulting simulated age-based selectivities change with changes in the observed length-at-age data over time.

The same selectivity penalty weights were also applied to model (b), which estimates the same seven spline knots for four time blocks, one of which is only two years in length (1999 and 2000).  In this case there are a total of 28 selectivity coefficients being estimated.  Here, we assume the timing of the discrete changes in selectivity corresponds to some known event and the years in which selectivity changes were correctly specified.

For model (c) the same penalized seven spline knots are estimated for each year based on age (not length), resulting in a total of 231 estimated age-based selectivity parameters representing 495 age-year combinations.  An additional penalty weight of $\lambda^{(3)}=1.0$ (CV=0.707) was added to the time varying selectivity option to constrain how rapidly age-specific selectivity coefficients can change over time.  Near identical results were  obtained with $\lambda^{(3)}=0.01$, but estimation convergence in Monte Carlo trials was less likely with no constraint in annual changes in selectivity due to potential confounding in the terminal year selectivity and terminal year fishing mortality rate.  Note that as $\lambda^{(3)}\rightarrow 0$, the model is analogous to a Virtual Population Analysis (VPA) where the catch-at-age data is assumed known without error.

An alternative to the annual time-varying selectivity is to interpolate in 2-dimensions where a series of knots for both age and year are the estimated parameters, and the age-year selectivity coefficients are interpolated using a 2-dimensional bicubic spline.  For comparison with model (c), model (d) is based on estimating seven knots for the age-dimension and 12 knots for the time-dimension, for a total of 84 estimated selectivity coefficients.  In this case the penalty weights for smoothing, dome-shaped, and time-varying changes in age-based selectivity were the same as that used for model (c).

The model permutations and combinations are summarized in Table \ref{tab:scenarios}, along with the total number of estimated and implied parameters for each of the four assessment models.  The additional 67 implied parameters correspond to the use of conditional maximum likelihood estimates for the survey scaling parameter $q$ and 66 additional variance terms for the measurement error in the survey and commercial age-composition data.  Unless otherwise noted, figures and additional tables maintain the same layout as Table \ref{tab:scenarios} where the true states of selectivity are in the rows, and assumed states in the assessment models are represented in the columns.

\begin{table}[!tbh]
	\caption{List of model scenarios and labels associated with each scenario explored.  For example, scenario 2a is based on simulated data with a fixed selectivity curve, but assumes 3 discrete time blocks in the assessment model.}
	\label{tab:scenarios}
	\begin{center}
		\begin{tabular}{r|cccc}
		\hline

		\hline

		&\multicolumn{4}{c}{\textbf{\underline{Assumed selectivity states}}}\\
		\textbf{\textbf{\underline{True states}}}
		&\textbf{Fixed (a)} & \textbf{Discrete (b)} & \textbf{Continuous (c)} & \textbf{Bicubic spline (d)} \\
		\hline
		 \textbf{Fixed (1)}      & 1a & 1b & 1c & 1d\\
		 \textbf{Discrete (2)}   & 2a & 2b & 2c & 2d\\
		 \textbf{Continuous (3)} & 3a & 3b & 3c & 3d\\
		\hline

		\hline
		No. of parameters & 95 & 116 & 319 & 172\\
		Implied parameters & 67 &67 &67 & 67 \\
		\hline
		Total  & 162 & 183 &  386 & 239 \\
		\hline

		\hline
		\end{tabular}
	\end{center}
\end{table}

Taking into consideration that the appropriate structural assumption about selectivity may not be known, we examine three criteria for choosing the appropriate selectivity parameterization.  The first criterion compares how well each model configuration explains simulated data (statistical fit) based on the effective number of parameters and the  Deviance Information Criterion (DIC).  The second criterion is based on retrospective performance based on Monte Carlo trials. Finally, we examine the precision and bias of estimated reference points based on the same Monte Carlo trials.

To summarize the retrospective results from Monte Carlo trials, the mean bias and mean absolute bias are calculated based on the following:
\begin{align}
    \mu_1   &= \frac{1}{4} \sum_{t=2005}^{2009} \frac{B_t^y-B_t^{2010}}{B_t^{2010}}\label{eq:5}\\
    \mu_2 &= \frac{1}{4} \sum_{t=2005}^{2009}\left| \frac{B_t^y-B_t^{2010}}{B_t^{2010}}\right|, \label{eq:6}
\end{align}
and we use $\Omega$ to summarize both statistics as the relative distance between the mean bias (precision) and absolute bias (variance):
\begin{equation} \label{eq:7}
    \Omega = \sqrt{\mu_1^2 + \mu_2^2 }.
\end{equation}
In the Monte Carlo trials, the model with the lowest average $\Omega$ would correspond to the least variable and least biased with respect to retrospective performance.  We also examine the mean absolute deviations of the distribution of $\mu_2$ statistics for each model run as a measure of variability in retrospective bias. % \citep{schweigert2009stock}.

% subsection scenarios (end)

% section methods (end)


\section{Theory}
\label{sec:theory}

\newtheorem{prop}{Proposition}[section]

%Here we investigate the correctness and expressiveness properties of \sysname. We are mainly concerned with answering the following questions: (1) does the distributed, compiled policy faithfully implement the user's policy regardless of failures? (2) Are the resulting BGP configurations stable?, and (3) what polcies are or are not expressible in \sysname?

If \sysname compiles a user policy to a distributed implementation, then that distributed implementation faithfully meets the centralized policy's semantics under any failure scenario. We are primarily concerned with the steady state: that is, what happens after (if) the routing protocol converges.  We argue that \sysname is correct by breaking down the claim into two distinct parts:

\begin{prop}
A distributed BGP configuration is \textit{Sound} with respect to a \sysname policy, if the any route traffic takes in the network, is a valid route specified in the policy.
\end{prop}

The Soundness claim follows directly from compilation of the product graph. Since every path through the product graph (from start to end) must match the user's policy, and since the compiled configurations use communities to ensure only valid routes through the product graph are used, the resulting BGP configurations are Sound.

\begin{prop}
A distributed BGP configuration is \textit{Complete} with respect to a \sysname policy, if the distributed implementation always obtains a most preferred route between two nodes when the corresponding path exists in the network.
\end{prop}

%The argument for completeness is as follows: Assume the most preferred path that exists in the network is between two nodes $X$ and $Y$.  This means there is a valid path in the product graph corresponding to this path, and which is available in the network.
%Assume we are unable to achieve this route in the network. To not achieve this path means that some router along the advertisement route from $Y$ to $X$ preferred to accept an advertisement from another peer instead. This implies that router that makes the ``wrong'' choice appears as a separate node in the product graph. However, from our failure safety check, we know that the other, more-preferred node must have a superset of the paths to accepting states as the less preferred node (from which it stole the route) for at least as good of a final preference. The argument then proceeds by induction on the path length - there are only a finite number of such stealings that can occur, each of which will ultimately still result in a route advertisement reaching $X$ along a most preferred path.

Any \sysname-compiled policy will be \textit{Complete} as a result of the failure safety check.
%
The combination of Soundness and Completeness ensures that the distributed implementation will always achieve the best routes possible, yet will never also use any unwanted routes.

\begin{prop}
The internal routers in a distributed BGP configuration produced by \sysname will be stable.
\end{prop}

The reduction is via the well-known No-Dispute-Wheel condition~\ref{bib:todo} for stable BGP. In particular, any policy with a dispute wheel will not pass the stronger failure safety check by \sysname.

%\begin{figure}[t!]
%\includegraphics[width=\columnwidth]{figures/dispute-wheel}
%\label{fig:dispute-wheel}
%\caption{Product graph for a dispute wheel.}
%\end{figure}

%The reduction is via the well-known No-Dispute-Wheel condition~\ref{bib:todo}. If a set of BGP configurations do not have a dispute wheel, then they will converge to their best routes. Any BGP policy with a dispute wheel will not satisfy the failure safety condition in Section~\ref{sec:compilation}. Assume that the preferences in the policy describe a dispute wheel, and we will show that our failure check will reject the policy. Informally, a dispute wheel occurs when there are some number of nodes $u_1, \dots, u_n$ attempting to get a path to a destination node $d$. Each node $u_i$ prefers to go through its neighbor $u_{i+1}$ over route $R_i$ than directly to $d$ with route $Q_i$. Thus the nodes form a wheel of preferences. The product graph for a policy that contains a dispute wheel will look like Figure~\ref{fig:dispute-wheel}, which shows a dispute wheel of size 3. In order to pass the failure safety check for the compiler, the more-preferred node for $u_2$ will need to have a superset of the paths starting from the less-preferred node for $u_2$. In turn, this means that the maximum length path after visiting the more preferred $u_2$ will be $max(Z) = 1 + max(Y)$, similarly, if we look at $u_1$, we will determine that $max(Y) = 1 + max(X)$, and so on. The resulting equations define a system of unsatisfiable equations. Therefore, it is not possible for \sysname to compile a BGP policy that contains a dispute wheel. Since no-dispute-wheel is a sufficient condition for BGP stability, any compiled \sysname policy will be stable.

An important distinction is that BGP might still be instable with respect to other external ASes, since \sysname has no way of configuring their routing policy and can only rely on AS path filters to see what advertisements are observed.


% Created 2021-10-07 Thu 15:58
% Intended LaTeX compiler: pdflatex
\documentclass[presentation,aspectratio=169, usenames, dvipsnames]{beamer}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage{khpreamble}
\usepackage{amssymb}
\usepgfplotslibrary{groupplots}
\newcommand*{\shift}{\operatorname{q}}
\definecolor{ppc}{rgb}{0.1,0.1,0.6}
\definecolor{iic}{rgb}{0.6,0.1,0.1}
\definecolor{ddc}{rgb}{0.1,0.6,0.1}
\def\ucolor{blue!80!black}
\def\ycolor{green!60!black}
\newcommand*{\incolor}[1]{\textcolor{\ucolor}{#1}}
\newcommand*{\outcolor}[1]{\textcolor{\ycolor}{#1}}
\usetheme{default}
\author{Kjartan Halvorsen}
\date{\today}
\title{Anti-windup}
\hypersetup{
 pdfauthor={Kjartan Halvorsen},
 pdftitle={Anti-windup},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.3 (Org mode 9.4.6)},
 pdflang={English}}
\begin{document}

\maketitle

\section{PID tuning - (Smith and Corripio) Ziegler Nichols}
\label{sec:orgf8a8844}
\begin{frame}[label={sec:orge8e09d0}]{The PID - practical form}
\definecolor{ppc}{rgb}{0.1,0.1,0.6}
\definecolor{iic}{rgb}{0.6,0.1,0.1}
\definecolor{ddc}{rgb}{0.1,0.5,0.1}

\begin{center}
  \begin{tikzpicture}[node distance=22mm, block/.style={rectangle, draw, minimum width=15mm}, sumnode/.style={circle, draw, inner sep=2pt}]

    \node[coordinate] (input) {};
    \node[sumnode, right of=input, node distance=16mm] (sum) {\tiny $\Sigma$};
    \node[color=iic,block, right of=sum, node distance=28mm] (ii)  {$\frac{1}{\tau_is}$};
    \node[color=ppc, coordinate, above of=ii, node distance=10mm] (pp)  {};
    \node[color=ddc,block, below of=ii, node distance=13mm] (dd)  {$\frac{-\tau_ds}{\frac{\tau_d}{N}s + 1}$};
    \node[sumnode, right of=ii, node distance=20mm] (sum2) {\tiny $\Sigma$};
    \node[block, right of=sum2, node distance=20mm] (gain)  {$k_c$};
    \node[coordinate, below of=sum, node distance=18mm] (feedback) {};
    \node[coordinate, right of=gain, node distance=20mm] (output) {};

    \draw[->] (input) -- node[above, pos=0.3] {$r(t)$} (sum);
    \draw[->] (sum) -- node[above, pos=0.2] {$e(t)$} node[coordinate] (mm) {}  (ii);
    \draw[->] (gain) -- node[above, near end] {$u(t)$} (output);
    \draw[->] (feedback) -- node[left, near start] {$y(t)$} node[right, pos=0.95] {-} (sum);
    \draw[->, color=ppc] (mm) |- (pp) -| node[right,] {$u_P(t)$} (sum2);
    \draw[->, color=ddc] (feedback |- dd) -- node[above, pos=0.95] {} (dd);
    \draw[->, color=ddc] (dd) -| node[right,] {$u_D(t)$} (sum2)  ;
    \draw[->, color=iic] (ii)  -- node[above,] {$u_I(t)$} (sum2);
    \draw[->] (sum2) -- node[above, near end] {} (gain);

  \end{tikzpicture}
\end{center}
\end{frame}

\section{Windup}
\label{sec:org9d89f2b}

\begin{frame}[label={sec:orgfa6a877}]{Integral windup}
\href{https://tecdemonterrey.instructuremedia.com/embed/6fb197b1-38a6-4938-b622-b25d4a45efcc}{Video by Tomás Alejandro Lugo Salinas (MTY)}
\end{frame}

\section{Anti-windup}
\label{sec:org2f751ae}

\begin{frame}[label={sec:org4a22b68}]{Anti-windup using back-calculation}
   \begin{center}
 	  \begin{tikzpicture}[every node/.append style={transform shape}, scale = 0.7, font=\footnotesize,
	block/.style={rectangle, draw, minimum width=12mm, minimum height=10mm, inner sep=4pt},
	amp/.style = {regular polygon, regular polygon sides=3,
              draw, fill=white, text width=1em,
              inner sep=1pt, outer sep=0mm,
              shape border rotate=-90},
	      summ/.style = {circle, draw, inner sep = 1pt},]
	 \node[block, align=center] (motor) at (0,0) {Cart and \\pendulum};
	 \node[coordinate, right of=motor, node distance = 26mm,] (output) {};
	 \node[block, left of=motor, node distance=26mm] (saturation) {};

	 \node[block, left of=saturation, node distance=26mm] (gain)  {$k_{c}$};

	 \node[summ, left of=gain, node distance=16mm] (sum2) {};
	 \node[color=iic,block, left of=sum2, node distance=18mm] (int)  {$\frac{1}{s}$};
	 \node[summ, left of=int, node distance=16mm] (sumint) {};
	 \node[color=iic,block, left of=sumint, node distance=18mm] (ii)  {$\frac{1}{\tau_{i}}$};
	 \node[color=ppc, coordinate, above of=ii, node distance=10mm] (pp)  {};
	 \node[summ, left of=ii, node distance=16mm] (sumsp) {};
	 \node[coordinate, left of=sumsp, node distance = 10mm,] (setpoint) {};

       \draw[->] (sumsp) -- node[above, pos=0.2] {$e(t)$} node[coordinate] (mm) {}  (ii);
       \draw[->] (gain) -- node[coordinate] (idmeas) {} node[above, ] {$u(t)$} (saturation);
       \draw[->, color=ppc] (mm) |- (pp) -| node[right,] {$u_P(t)$} (sum2);
       \draw[->, color=iic] (int)  -- node[above,] {$u_I(t)$} (sum2);
       \draw[->] (sum2) -- node[above, near end] {} (gain);

	 \draw[->] (setpoint) -- node[above, very near start ] {$x_{ref}$} (sumsp);
	 \draw[->] (saturation) -- node[coordinate, ] (satmeas) {} node[above,] {$u_s(t)$} (motor);
	 \draw[->] (motor) -- node[coordinate] (meas) {} node[above, very near end] {$x$} (output);
	 \draw[->] (meas) -- node[right] {} ++(0,-28mm) -| node[pos=0.95, left] {$-$} (sumsp);

	 % Anti-windup
	 \draw ($ (saturation.south west) + (2mm, 2mm) $) -- ++(3mm, 0) -- ++(3mm, 5mm) -- ++(3mm, 0);
	 \node[block, below of=sumint, node distance=16mm] (back) {$\frac{1}{k_{c}\tau_t}$};
	 \node[summ, below of=saturation, node distance=12mm] (sumsat) {};
	 \draw[->] (satmeas) |- node[above, pos=0.8] {} (sumsat);
	 \draw[->] (idmeas) |- node[above, pos=0.8] {$-$} (sumsat);
	 \draw[->] (sumsat) |- (back);
	 \draw[->] (back) -- (sumint);
	 \draw[->] (ii) -- (sumint);
	 \draw[->] (sumint) -- (int);

	 \node[coordinate, right of=back, node distance=2cm] (sat) {};

\end{tikzpicture}
\end{center}

\alert{Activity} Assume \(e(t)=0\) and determine the transfer function from \(u_s(t)\) to \(u(t)\).
\end{frame}
\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Carlos Santos             %
% ECE 351-51                %
% Lab 8                     %
% 03/31/2020                %
%                           %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\documentclass[12pt]{article}

% Language and font encoding
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{caption}
\usepackage{float}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{rotating}
\usepackage{setspace}

% Sets page size and margins
\usepackage[a4paper,top=3cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}

% Useful packages
\usepackage[colorinlistoftodos]{todonotes}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}
\usepackage{listings}
\usepackage{gensymb}
\usepackage{graphicx} %package to manage images
\usepackage{listings}
\usepackage{color}

\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=Python,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

%Line Spacing
\setstretch{1.5}

%Info for Title Page
\title{ECE 351 Lab 8 Report \\ Section 52}
\date{March 31, 2020}
\author{Carlos Santos}
\begin{document}

%Make a Title Page
\vspace{\fill}
\maketitle
\vspace{\fill}
\clearpage

\maketitle
\tableofcontents

%Introduction
\section{Introduction}
The purpose of this lab was to use Fourier Series to approximate time-domain signals.

%Equations
\section{Equations}

\begin{equation}
    x(t) = \frac{1}{2}a_0 + \sum_{n=1}^{\infty} a_k cos(kw_0 t) + b_k sin(kw_0 t)
\end{equation}

Where the following is know:
\begin{equation}
     a_k = \frac{2}{T} \int_{0}^{T} x(t)cos(k\omega _0 t) dt
\end{equation}

\begin{equation}
    b_k = \frac{2}{T} \int_{0}^{T} x(t)sin(k\omega _0 t) dt
\end{equation}

and

\begin{equation}
    \omega _0 = \frac{2 \pi}{T}
\end{equation}

\begin{figure}[H]
\caption{Square Wave}
\centering
\includegraphics[width=.8\textwidth]{Square_Wave.png}
\end{figure}

%Methodology
\section{Methodology}
This lab was a pretty straight forward lab. I took the following steps for this lab:
\begin{enumerate}
    \item First, I turned equations 2 and 3 into a python function.
    \item Then I turned equation 1 into a python function.
    \item I then used the function I just defined to find ${a_0,a_1,b_1,b_2,b_3}$
    \item My next step was to plot the Fourier Series Approximation for a set a values where N=1,3,15,50,150,1500.
    \item Lastly, I made sure that my plots had two subplots of three items each.
\end{enumerate}

%Results
\section{Results}

\begin{figure}[H]
\caption{Fourier Series Approximation}
\centering
\includegraphics[width=.8\textwidth]{F_Ser_Approx_1.png}
\end{figure}

\begin{figure}[H]
\caption{Fourier Series Approximation}
\centering
\includegraphics[width=.8\textwidth]{F_Ser_Approx_2.png}
\end{figure}

This is the code I used to calculate the Fourier Series Approximations. This function takes in three parameters.  Those parameters are: n is the number of times for the summation, t is the time, T is the integral limits.
\begin{lstlisting}
def Fourier_approx(n,t,T):
    x=0
    for i in range(1,n+1):
        x = x + b(i)*np.sin(i*(2*np.pi/T)*t)
    return x
\end{lstlisting}

One small challenge faced was the divide by zero error. Python spits an error out that says, "Invalid value encountered in true\textunderscore divide". I wasn't able to fix it, but the program still ran and plotted what was expected.

%Questions
\section{Questions}
\begin{enumerate}
    \item Is x(t) an even or an odd function? Explain why?
    \begin{enumerate}
        \item x(t) is an even function because it is symmetric about the y-axis. This means we can fold the graph in half and it would overlap perfectly.
    \end{enumerate}
    \item Based on your results from Task 1, what do you expect the values of a2, a3, . . . , an to be?
Why?
    \begin{enumerate}
        \item I expect the rest of the values to be 0 as well a\textunderscore k does not contribute the the approximation. The only one that does is b\textunderscore k.
    \end{enumerate}
    \item How does the approximation of the square wave change as the value of N increases? In what
way does the Fourier series struggle to approximate the square wave?
    \begin{enumerate}
        \item As N is higher it starts to resemble the square wave more. We can see that by looking at the plots. The Fourier series struggles to capture the top and bottom parts of the square wave. The top being 1 and the bottom being -1.
    \end{enumerate}
    \item What is occurring mathematically in the Fourier series summation as the value of N increases?
    \begin{enumerate}
        \item As the value of N increases the Fourier series is adding more and more cosine and sine waves together to reach an approximation of a square wave. That means that when N is 1500 it has added together a total of 1500 cosine and sine terms together.
    \end{enumerate}
    \item Leave any feedback on the clarity of lab tasks, expectations, and deliverables.
    \begin{enumerate}
        \item N/A
    \end{enumerate}
\end{enumerate}

%Conclusion
\section{Conclusion}
In summary, this was a straight forward lab. It helped me visualize what we tend to do by hand in class. The python code executed and provided the results I needed. For future labs, I'll need to avoid the divide by zero error.

%Appendix
\section{Appendix}
Attached you'll find the console output.
\end{document}


\documentclass[serif,xcolor=pdftex,dvipsnames,table,hyperref={bookmarks=false,breaklinks}]{beamer}

\input{../config.tex}

\settitlecard{5}{NumPy 1}

\begin{document}
\lstset{style=custompython}
\maketitlepage

% \section{Preliminaries}
% \subsection{Foo}
%
% \begin{frame}[t]{Announcements}
% 	\begin{itemize}
% 		\item Assignment 2 due Thursday
% 		\item New OH: Thursdays 11:00am-Noon in CS 207
% 		\item Assignment 1 grades now on Moodle. Regrade requests accepted until 09/26.
% 		\begin{itemize}
% 			\item No points back for type mismatch.
% 		\end{itemize}
% 	\end{itemize}
% \end{frame}

\section{Numpy and Arrays}
\subsection{Foo}

\begin{frame}[t,fragile]{What is NumPy?}
	% import convention
	Numerical Python (NumPy) is the backbone of the Python scientific programming stack:
	\begin{itemize}[<+->]
		\item Provides multidimensional arrays (tensors).
		\item Most functions written in C (more efficient).
		\item Many useful functions for speeding up loops over array elements (vectorization).
	\end{itemize}

	\pause
	Typical import convention:
		\begin{tcolorbox}
			\begin{verbatim}
				import numpy as np
			\end{verbatim}
		\end{tcolorbox}
\end{frame}

\begin{frame}[t,fragile]{The Numpy Array}
	The core of NumPy is the Array type:

	\pause
	\begin{lstlisting}
		>>> import numpy as np
		>>> a = np.array([0, 1, 2, 3])
		>>> a
		array([0, 1, 2, 3])
	\end{lstlisting}

	\pause
	\begin{itemize}[<+->]
		\item Flexible structure good for storing typical scientific data:
		\begin{itemize}[<+->]
			\item Numerical data matrix
			\item Image pixel values
			\item 3-d images such as MRI scan data
		\end{itemize}
		\item Mutable contents, but not size.
		\item In general, no mixed types.
	\end{itemize}
\end{frame}

\begin{frame}[t,fragile]{Constructing Arrays}
	% Manualy
	Arrays can be created manually:

	\pause
	\begin{lstlisting}
		>>> a = np.array([1, 2]) # 1-D
		>>> a
		array([1, 2])
		>>> b = np.array([[1,2],[3,4]]) # 2-D
		>>> b
		array([[1, 2],
		       [3, 4]])
		>>> c = np.array([[[1,2],[3,4]],[[5,6],[7,8]]]) # 3-D
		>>> c
		array([[[1, 2],
		        [3, 4]],

		       [[5, 6],
		        [7, 8]]])
	\end{lstlisting}

\end{frame}

\begin{frame}[t,fragile]{Constructing Arrays}
	% arange, linspace, ones, eye, diag, empty
	Many functions exist to create NumPy Arrays:

	\pause
	\begin{lstlisting}
		# Create a range
		>>> np.arange(5)
		array([0, 1, 2, 3, 4])
		>>> np.arange(0,10,2) # start, end, step size
		array([0, 2, 4, 6, 8])

		# Or specify a number of points
		>>> np.linspace(0,1,5) # start, end, number of points
		array([ 0.  ,  0.25,  0.5 ,  0.75,  1.  ])
	\end{lstlisting}
\end{frame}

\begin{frame}[t,fragile]{Constructing Arrays}
	% arange, linspace, ones, eye, diag, empty
	Many functions exist to create NumPy Arrays:

 	\begin{lstlisting}
		# Create an array filled with ones
		>>> np.ones(5)
		array([ 1.,  1.,  1.,  1.,  1.])

		# Or zeros
		# Specify multiple dimensions as a tuple
		>>> np.zeros((2,2))
		array([[ 0.,  0.],
		       [ 0.,  0.]])
	\end{lstlisting}
\end{frame}

\begin{frame}[t,fragile]{Numpy Data Types}
	% float, int, bool, complex, string
	\begin{itemize}[<+->]
		\item Numpy supports Python numerical data types as well as strings.
		\item You can also control the number of bits used to store floats and ints using the data types: np.int16, np.int32, np.int64, np.float32, np.float64, ...
	\end{itemize}

	\pause
	\begin{lstlisting}
		# Create an array filled with ones
		>>> np.array([1,2,3], dtype=np.int32)
		array([1, 2, 3], dtype=int32)

		>>> np.array([1,2,3], dtype=np.float64)
		array([ 1.,  2.,  3.])

		>>> np.array([1,0,1], dtype=bool)
		array([ True, False,  True], dtype=bool)
	\end{lstlisting}

\end{frame}

\begin{frame}[t,fragile]{Interactive NumPy Demo}
	\centering
	\Huge{Interactive Demo}
	\normalsize
	\begin{enumerate}
		\item What do the attributes \verb|size|, \verb|shape|, \verb|ndim|, and \verb|dtype| store?
		\item NumPy arrays can also be created using the following three methods. What does each one do?
		\begin{itemize}
			\item \verb|eye|
			\item \verb|diag|
			\item \verb|empty|
		\end{itemize}
		\item What are the default number of bits for numpy floats and ints?
	\end{enumerate}
\end{frame}

\begin{frame}[t,fragile]{Interactive NumPy Demo}
	\begin{enumerate}[<+->]
		\item What do the attributes \verb|size|, \verb|shape|, \verb|ndim|, and \verb|dtype| store?
		\begin{itemize}[<+->]
			\item \verb|size|: The number of items in the array
			\item \verb|shape|: The dimensions of the array in a tuple
			\item \verb|ndim|: The number of dimensions
			\item \verb|dtype|: The type of the contents of the array
		\end{itemize}
		\item NumPy arrays can also be created using the following three methods. What does each one do?
		\begin{itemize}[<+->]
			\item \verb|eye|: Identity
			\item \verb|diag|: If the input is 1-D, returns a diagonal matrix with diagonal equal to the input. If the input is 2-D, returns the diagonal of the input.
			\item \verb|empty|: Returns an uninitialized array. Use with caution.
		\end{itemize}
		\item What are the default number of bits for numpy floats and ints?
		\begin{itemize}[<+->]
			\item \verb|float64| and \verb|int64|
		\end{itemize}
	\end{enumerate}
\end{frame}

\section{Element-wise Operations and Indexing}
\subsection{Foo}

\begin{frame}[t,fragile]{Mathematical Operators}
	% Scalar
	NumPy supports standard arithmetic operations between scalars and arrays:

	\pause
	\begin{lstlisting}
		>>> np.ones(3) + 2
		array([ 3.,  3.,  3.])

		>>> np.ones(3) - 2
		array([-1., -1., -1.])

		>>> np.ones(3) * 2
		array([ 2.,  2.,  2.])

		>>> np.ones(3) / 2
		array([ 0.5,  0.5,  0.5])
	\end{lstlisting}
\end{frame}

\begin{frame}[t,fragile]{Mathematical Operators}
	% Elementwise
	It also supports arithmetic operations between arrays which work on an element-wise basis:

	\pause
	\begin{lstlisting}
		>>> np.arange(1,4) + np.arange(1,4)
		array([2, 4, 6])

		>>> np.arange(1,4) - np.arange(1,4)
		array([0, 0, 0])

		# Not matrix multiplication!
		>>> np.arange(1,4) * np.arange(1,4)
		array([1, 4, 9])

		>>> np.arange(1,4) / np.arange(1,4)
		array([ 1.,  1.,  1.])
	\end{lstlisting}

	\pause
	Note: Python has a special function for matrix multiplication.
\end{frame}

% \begin{frame}[t]{Mathematical Operators}
% 	% WARNING about type conversion
% \end{frame}

\begin{frame}[t,fragile]{Logical Operators}
	% Elementwise
	Logical operators work in a similar fashion:

	\pause
	\begin{lstlisting}
		>>> np.arange(4) == 2
		array([False, False,  True, False], dtype=bool)

		>>> (np.array([[0,1],[2,3]]) % 2) == 0
		array([[ True, False],
		       [ True, False]], dtype=bool)

		>>> np.array([1,2,3,4]) < np.array([0,9,9,0])
		array([False,  True,  True, False], dtype=bool)
	\end{lstlisting}
\end{frame}

% \begin{frame}[t,fragile]{Logical Operators}
% 	% Elementwise
% 	Logical operators work in a similar fashion:
%
% 	\pause
% 	\begin{lstlisting}
% 		>>> np.arange(4) == 2
% 		array([False, False,  True, False], dtype=bool)
%
% 		>>> (np.array([[0,1],[2,3]]) % 2) == 0
% 		array([[ True, False],
% 		       [ True, False]], dtype=bool)
%
% 		>>> np.array([1,2,3,4]) < np.array([0,9,9,0])
% 		array([False,  True,  True, False], dtype=bool)
% 	\end{lstlisting}
% \end{frame}

\begin{frame}[t,fragile]{Indexing}
	% Basic indexing and slicing
	Access a single element by providing an index for each dimension separated by commas:
	\pause
	\begin{lstlisting}
		>>> a = np.arange(6)
		>>> a
		array([0, 1, 2, 3, 4, 5])
		>>> a[5]
		5

		>>> b = np.diag(np.arange(3))
		>>> b
		array([[0, 0, 0],
		       [0, 1, 0],
		       [0, 0, 2]])
		>>> b[2,2]
		2
	\end{lstlisting}
\end{frame}

\begin{frame}[t,fragile]{Slicing}
	Access multiple entries along a dimension using ":" notation. This is called \emph{slicing}.
	\pause
	\begin{lstlisting}
		>>> a = np.arange(10)
		>>> a
		array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
		>>> a[1:9:2] # start, end, step size
		array([1, 3, 5, 7])

		>>> b = np.array([[1,2,3],[4,5,6],[7,8,9]])
		>>> b
		array([[1, 2, 3],
		       [4, 5, 6],
		       [7, 8, 9]])
		>>> b[1,:]
		array([4, 5, 6])
	\end{lstlisting}
\end{frame}

\begin{frame}[t]{N-D Slicing}
	\centering
	\includegraphics[width=4.5in]{{../Figures/array_slicing/Slide1}.png}
\end{frame}

\begin{frame}[t]{N-D Slicing}
	\centering
	\includegraphics[width=4.5in]{{../Figures/array_slicing/Slide2}.png}
\end{frame}

\begin{frame}[t]{N-D Slicing}
	\centering
	\includegraphics[width=4.5in]{{../Figures/array_slicing/Slide3}.png}
\end{frame}

\begin{frame}[t]{N-D Slicing}
	\centering
	\includegraphics[width=4.5in]{{../Figures/array_slicing/Slide4}.png}
\end{frame}

\begin{frame}[t]{N-D Slicing}
	\centering
	\includegraphics[width=4.5in]{{../Figures/array_slicing/Slide5}.png}
\end{frame}

\begin{frame}[t,fragile]{N-D Slicing}
	If a single index is passed for a dimension, the result will have one fewer dimensions than the original array.

	\begin{lstlisting}
		>>> a = np.ones((2,3,4))
		>>> a[:,1,:].shape
		(2,4)
	\end{lstlisting}
\end{frame}

\begin{frame}[t,fragile]{Entry/Slice Assignment}
	As with lists, we can assign to individual array entries or slices.

	\begin{lstlisting}
		>>> a = np.ones((3,3))
		>>> a[2,1] = 9
		>>> a
		array([[ 1.,  1.,  1.],
		       [ 1.,  1.,  1.],
		       [ 1.,  9.,  1.]])
		>>> a[:,0] = 5
		>>> a
		array([[ 5.,  1.,  1.],
		       [ 5.,  1.,  1.],
		       [ 5.,  9.,  1.]])
		>>> a[0,:] = np.arange(3)
		>>> a
		array([[ 0.,  1.,  2.],
		       [ 5.,  1.,  1.],
		       [ 5.,  9.,  1.]])
	\end{lstlisting}
\end{frame}

\begin{frame}[t,fragile]{Interactive Demo}
	\centering
	\Huge{Interactive Demo}
	\normalsize
	\begin{enumerate}
		\item Create an the following array:
		\begin{itemize}
			\item $a_j = 2^{3j} - j,\,\,j = 1,...,10$
		\end{itemize}
		\item \verb|np.all| and \verb|np.any| are NumPy functions that operate on boolean arrays. What do these functions do?
		\item How do you extract matrix $B$ from matrix $A$ with only indexing?
	\end{enumerate}

	$$ A =
		\begin{bmatrix}
			1 & 2 & 3 & 4 & 5\\
			6 & 7 & 8 & 9 & 10\\
			11 & 12 & 13 & 14 & 15\\
			16 & 17 & 18 & 19 & 20\\
			21 & 22 & 23 & 24 & 25\\
		\end{bmatrix}\,\,\,\, B =
		\begin{bmatrix}
			25 & 23 & 21\\
			15 & 13 & 11\\
			5 & 3 & 1\\
		\end{bmatrix}$$
	%
\end{frame}

\begin{frame}[t,fragile]{Interactive Demo}
	\centering
	\Huge{Interactive Demo}
	\normalsize
	\begin{enumerate}
		\item Create an the following array:
		\begin{itemize}
			\item $a_j = 2^{3j} - j,\,\,j = 1,...,10$
			\item \verb|a = 2**(3*np.arange(1,11))-np.arange(1,11)|
		\end{itemize}
		\item \verb|np.all| and \verb|np.any| are NumPy functions that operate on boolean arrays. What do these functions do?
		\begin{itemize}
			\item \verb|np.all| returns \verb|True| if all elements of the array are true.
			\item \verb|np.any| returns \verb|True| if any elements of the array are true.
		\end{itemize}
		\item How do you extract matrix $B$ from matrix $A$ with only indexing?
		\begin{itemize}
			\item \verb|B = A[::-2,::-2]|
		\end{itemize}
	\end{enumerate}

	% $$ A =
	% 	\begin{bmatrix}
	% 		1 & 2 & 3 & 4 & 5\\
	% 		6 & 7 & 8 & 9 & 10\\
	% 		11 & 12 & 13 & 14 & 15\\
	% 		16 & 17 & 18 & 19 & 20\\
	% 		21 & 22 & 23 & 24 & 25\\
	% 	\end{bmatrix}\,\,\,\, B =
	% 	\begin{bmatrix}
	% 		25 & 23 & 21\\
	% 		15 & 13 & 11\\
	% 		5 & 3 & 1\\
	% 	\end{bmatrix}$$
	%
\end{frame}

\end{document}


\documentclass[11pt,fancychapters]{article}
\usepackage[a4paper, total={6in, 8in}]{geometry}
\usepackage{cite}
\usepackage{color}
\usepackage{xcolor}
\usepackage{empheq}
\usepackage{enumitem}
\usepackage{setspace}
\usepackage{hyperref}
\usepackage{minted}
\usepackage{acro}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{multirow}
\usepackage{graphicx}
\usepackage{geometry}
\usepackage{subcaption}
\usepackage{cancel}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{tcolorbox}
\usepackage{hyperref}
\usepackage{cleveref}
\usepackage{parskip}
\usepackage{tcolorbox}
\usepackage{float}
\usepackage{bm}
\usepackage{mathtools}
\usepackage{pgfplots}
 \geometry{
 a4paper,
 total={170mm,257mm},
 left=20mm,
 top=20mm,
 }
\pgfplotsset{width=8cm,compat=1.9}
\newcommand{\dbar}{{d\mkern-7mu\mathchar'26\mkern-2mu}}
\newcommand{\boxedeq}[2]{\begin{empheq}[box={\fboxsep=6pt\fbox}]{align}\label{#1}#2\end{empheq}}
\newcommand{\approptoinn}[2]{\mathrel{\vcenter{
  \offinterlineskip\halign{\hfil$##$\cr
    #1\propto\cr\noalign{\kern2pt}#1\sim\cr\noalign{\kern-2pt}}}}}
\newcommand{\appropto}{\mathpalette\approptoinn\relax}
\renewcommand*{\thesection}{\arabic{section}.}
\def\*#1{\mathbf{#1}}
\def\ab{ab}
\usepackage{tikz}
\usetikzlibrary{calc,trees,patterns,positioning,arrows,chains,shapes.geometric,%
    decorations.pathreplacing,decorations.pathmorphing,shapes,%
    matrix,shapes.symbols}
\usepgfplotslibrary{fillbetween}
\geometry{top=1.3in,bottom=1.3in}

\begin{document}
\centerline{\huge{SUTD 2021 50.007 Homework 4}}

\begin{table}[ht]
\centering
\footnotesize
 \begin{tabular}{c c}
James Raphael Tiovalen & 1004555
 \end{tabular}
\end{table}

\section*{Hidden Markov Model}

\subsection*{Question 1}

The transition probabilities are estimated as:

\begin{equation*}
	a_{u,v} = \frac{\text{Count}(u, v)}{\text{Count}(u)}
\end{equation*}

\begin{table}[h!]
	\centering
	\begin{tabular}{| c | c | c | c | c |}
		\hline
		$u \backslash v$ & X & Y & Z & STOP \\
		\hline
		START & 2/4 & 0 & 2/4 & 0 \\
		\hline
		X & 0 & 2/5 & 2/5 & 1/5 \\
		\hline
		Y & 1/5 & 0 & 1/5 & 3/5 \\
		\hline
		Z & 2/5 & 3/5 & 0 & 0 \\ [1ex]
		\hline
	\end{tabular}
\end{table}

The emission probabilities are estimated as:

\begin{equation*}
	b_{u}(o) = \frac{\text{Count}(u \rightarrow o)}{\text{Count}(u)}
\end{equation*}

\begin{table}[h!]
	\centering
	\begin{tabular}{| c | c | c | c |}
		\hline
		$u \backslash o$ & $\textbf{a}$ & $\textbf{b}$ & $\textbf{c}$ \\
		\hline
		X & 2/5 & 3/5 & 0 \\
		\hline
		Y & 1/5 & 0 & 4/5 \\
		\hline
		Z & 1/5 & 3/5 & 1/5 \\ [1ex]
		\hline
	\end{tabular}
\end{table}

\subsection*{Question 2}

Base case:

\begin{equation*}
	\pi(0, \text{START}) = 1, ~ \text{otherwise} ~ \pi(0, v) = 0 ~ \text{if} ~ v \neq \text{START}
\end{equation*}

Moving forward recursively:

\begin{itemize}
	\item[] $k = 1$
	\begin{align*}
		\pi(1, X) &= \pi(0, \text{START}) \times a_{\text{START}, X} \times b_X(\textbf{b}) = 2/4 \times 3/5 = 3/10 \\
		\pi(1, Y) &= \pi(0, \text{START}) \times a_{\text{START}, Y} \times b_Y(\textbf{b}) = 0 \\
		\pi(1, Z) &= \pi(0, \text{START}) \times a_{\text{START}, Z} \times b_Z(\textbf{b}) = 2/4 \times 3/5 = 3/10
	\end{align*}

	\item[] $k = 2$
	\begin{align*}
		\pi(2, X) &= \max_{u \in \mathcal{T}} \{ \pi(1, u) \times a_{u, X} \times b_X(\textbf{c}) \} \\
		&= \max \{ 3/10 \times 0 \times 0, 0, 3/10 \times 2/5 \times 0 \} \\
		&= 0 \\
		\pi(2, Y) &= \max_{u \in \mathcal{T}} \{ \pi(1, u) \times a_{u, Y} \times b_Y(\textbf{c}) \} \\
		&= \max \{ 3/10 \times 2/5 \times 4/5, 0, 3/10 \times 3/5 \times 4/5 \} \\
		&= 18/125 \\
		\pi(2, Z) &= \max_{u \in \mathcal{T}} \{ \pi(1, u) \times a_{u, Z} \times b_Z(\textbf{c}) \} \\
		&= \max \{ 3/10 \times 2/5 \times 1/5, 0, 3/10 \times 0 \times 1/5 \} \\
		&= 3/125
	\end{align*}

	\item[] $k = 3$
	\begin{align*}
		\pi(3, \text{STOP}) &= \max_{u \in \mathcal{T}} \{ \pi(2, u) \times a_{u, \text{STOP}} \} \\
		&= \max \{ 0 \times 1/5, 18/125 \times 3/5, 3/125 \times 0 \} \\
		&= 54/625
	\end{align*}
\end{itemize}

Backtracking:

\begin{align*}
	y_2^* &= \arg \max_{v \in \mathcal{T}} \{ \pi(2, v) \times a_{v, \text{STOP}} \} = Y \\
	y_1^* &= \arg \max_{v \in \mathcal{T}} \{ \pi(1, v) \times a_{v, Y} \} = Z
\end{align*}

Therefore, the optimal sequence is: $Z, Y$.

\subsection*{Question 3}

The set of possible states is $\{\text{START}, X, Y, Z, \text{STOP}\}$. In an HMM, for a sequence of length $n$, we have

\begin{align*}
	&P(x_1, \ldots, x_{i-1}, y_i = u, x_i, \ldots, x_n) \\
	&= P(x_1, \ldots, x_{i-1}, y_i = u) \times P(x_i, \ldots, x_n | y_i = u) \\
	&= \alpha_u(i) \beta_u(i),
\end{align*}

where:

\begin{align*}
	\alpha_u(i) &= P(x_1, \ldots, x_{i-1}, y_i = u), \\
	\beta_u(i) &= P(x_i, \ldots, x_n | y_i = u)
\end{align*}

As shown in class:

\begin{equation*}
	P(s_i = u | o_1 = \textbf{b}, o_2 = \textbf{c}) = \frac{\alpha_u(i) \beta_u(i)}{\sum_{v} \alpha_v(j) \beta_v(j)},
\end{equation*}

with any choice of $j \in \{1, 2\}$.

Hence, to calculate the marginal distribution probability term, we will use the Forward-Backward Algorithm.

\subsubsection*{Forward}

\begin{itemize}
	\item Base case:
	\begin{align*}
		\alpha_X(1) &= a_{\text{START}, X} = 2/4 \\
		\alpha_Y(1) &= a_{\text{START}, Y} = 0 \\
		\alpha_Z(1) &= a_{\text{START}, Z} = 2/4
	\end{align*}

	\item Moving forward recursively:
	\begin{align*}
		\alpha_X(2) &= \sum_v \alpha_v(1) \times a_{v, X} \times b_v(\textbf{b}) \\
		&= (2/4 \times 0 \times 3/5) + 0 + (2/4 \times 2/5 \times 3/5) = 3/25 \\
		\alpha_Y(2) &= \sum_v \alpha_v(1) \times a_{v, Y} \times b_v(\textbf{b}) \\
		& = (2/4 \times 2/5 \times 3/5) + 0 + (2/4 \times 3/5 \times 3/5) = 3/10 \\
		\alpha_Z(2) &= \sum_v \alpha_v(1) \times a_{v, Z} \times b_v(\textbf{b}) \\
		& = (2/4 \times 2/5 \times 3/5) + 0 + (2/4 \times 0 \times 3/5) = 3/25
	\end{align*}
\end{itemize}

\subsubsection*{Backward}

\begin{itemize}
	\item Base case:
	\begin{align*}
		\beta_X(2) &= a_{X, \text{STOP}} \times b_X(\textbf{c}) = 1/5 \times 0 = 0 \\
		\beta_Y(2) &= a_{Y, \text{STOP}} \times b_Y(\textbf{c}) = 3/5 \times 4/5 = 12/25 \\
		\beta_Z(2) &= a_{Z, \text{STOP}} \times b_Z(\textbf{c}) = 0 \times 1/5 = 0
	\end{align*}

	\item Moving backward recursively:
	\begin{align*}
		\beta_X(1) &= \sum_v a_{X, v} \times b_X(\textbf{b}) \times \beta_v(2) \\
		&= 0 + (2/5 \times 3/5 \times 12/25) + 0 = 72/625 \\
		\beta_Y(1) &= \sum_v a_{Y, v} \times b_Y(\textbf{b}) \times \beta_v(2) \\
		&= 0 + (0 \times 0 \times 12/25) + 0 = 0 \\
		\beta_Z(1) &= \sum_v a_{Z, v} \times b_Z(\textbf{b}) \times \beta_v(2) \\
		&= 0 + (3/5 \times 3/5 \times 12/25) + 0 = 108/625
	\end{align*}
\end{itemize}

Thus:

\begin{equation*}
	\sum_{v} \alpha_v(j) \beta_v(j) = (2/4 \times 72/625) + (2/4 \times 108/625) = (3/10 \times 12/25) = 18/125
\end{equation*}

Hence, the marginal distribution table would be:

\begin{table}[h!]
	\centering
	\begin{tabular}{|c | c | c |}
		\hline
		$u \backslash i$ & $1$ & $2$ \\
		\hline
		X & 0.0576/0.144 = 2/5 & 0 \\
		\hline
		Y & 0 & 0.144/0.144 = 1 \\
		\hline
		Z & 0.0864/0.144 = 3/5 & 0 \\ [1ex]
		\hline
	\end{tabular}
\end{table}

Taking the maximum marginal probability value for each column $i$ from the table:

\begin{align*}
	s_1^* &= \arg \max_{s_1}P(s_1 | o_1 = \textbf{b}, o_2 = \textbf{c}) = \arg \max_{s_1}P(s_1, o_1 = \textbf{b}, o_2 = \textbf{c}) = Z \\
	s_2^* &= \arg \max_{s_2}P(s_2 | o_1 = \textbf{b}, o_2 = \textbf{c}) = \arg \max_{s_2}P(s_2, o_1 = \textbf{b}, o_2 = \textbf{c}) = Y
\end{align*}

Therefore, we would also consistently obtain and arrive at the optimal sequence of: $Z, Y$.

\subsection*{Question 4}

Assume we have a set of possible states $\{ 0, 1, \ldots, N - 1, N \}$, where $0 = \text{START}$ and $N = \text{STOP}$.

\begin{align*}
	&P(x_1, \ldots, x_{i-1}, y_1, \ldots, y_{i-1}, z_i = u, x_i, \ldots, x_n, y_i, \ldots, y_n; \theta) \\
	&= P(x_1, \ldots, x_{i-1}, y_1, \ldots, y_{i-1}, z_i = u; \theta) \times P(x_i, \ldots, x_n, y_i, \ldots, y_n | z_i = u; \theta) \\
	&= \alpha_u(i) \beta_u(i),
\end{align*}

where

\begin{align*}
	\alpha_u(i) &= P(x_1, \ldots, x_{i-1}, y_1, \ldots, y_{i-1}, z_i = u; \theta), \\
	\beta_u(i) &= P(x_i, \ldots, x_n, y_i, \ldots, y_n | z_i = u; \theta)
\end{align*}

\subsubsection*{Forward}

\begin{itemize}
	\item Base case:
	\begin{equation*}
		\alpha_u(1) = a_{\text{START}, u}, ~ \forall ~ u \in \{ 1, \ldots, N - 1 \}
	\end{equation*}

	\item Moving forward recursively, for $i = 2, \ldots, n$:
	\begin{equation*}
		\alpha_u(i + 1) = \sum_v \alpha_v(i) \times a_{v, u} \times b_v(x_i) \times c_{x_i}(y_i),
	\end{equation*}
	where
	\begin{align*}
		b_v(x) &= P(x | v), \\
		c_x(y) &= P(y | x)
	\end{align*}
\end{itemize}

\subsubsection*{Backward}

\begin{itemize}
	\item Base case:
	\begin{equation*}
		\beta_u(n) = a_{u, \text{STOP}} \times b_u(x_n) \times c_{x_n}(y_n), ~ \forall ~ u \in \{ 1, \ldots, N - 1 \}
	\end{equation*}

	\item Moving backward recursively, for $i = n - 1, \ldots, 1$:
	\begin{equation*}
		\beta_u(i) = \sum_v a_{u, v} \times b_u(x_i) \times c_{x_i}(y_i) \times \beta_v(i + 1)
	\end{equation*}
\end{itemize}

At each time step/position, there are $N$ forward ($\alpha$) and $N$ backward ($\beta$) terms to compute. Hence, to compute each term, there are $O(N)$ operations. Thus, at each time step/position, there are $O(N^2)$ operations. Therefore, assuming that the length of the sentence is $n$, which is the number of different time steps/positions, the total time complexity for this algorithm is $O(nN^2)$.

\end{document}


\subsection{Approximating the Step Response}

So far, two methods for characterising step response functions were illustrated,
namely the method proposed  by  P.  Hudzovic (equation \ref{eq:tu_tg}) where you
calculate  $T_u$  and  $T_g$,  and  the  method  proposed by L.  Sani  (equation
\ref{eq:t10_t50_t90})   where  you  calculate  $t_{10}$,   $t_{50}$,   $t_{90}$.

Furthermore, two similar methods for constructing a transfer function $G_n(s,r)$
(equation \ref{eq:ptn}) were shown, again, one proposed by P. Hudzovic (equation
\ref{eq:hudzovic})  and  one  proposed by L. Sani (equation  \ref{eq:sani})  for
calculating the individual time constants $T_1 \ldots T_n$ based  on  the  input
parameters $r$ and $T$.

By  combining  the  various  approaches  with each other, 4 different  ways  for
determining $G_n(s,r)$ exist.

It  is  not  possible to \textit{directly} calculate appropriate values for $n$,
$T$  and $r$ when given an unknown step response  function,  however,  by  using
equations \ref{eq:ptn},  \ref{eq:hudzovic}  and \ref{eq:sani}, it is possible to
construct  a  lookup table by calculating a (theoretically) infinite  number  of
step responses in function of $n$  and  $r$,  characterise  their  step response
using  equations \ref{eq:tu_tg} or \ref{eq:t10_t50_t90}  (i.e.  determine  their
``complexity''), and perform a  reverse  lookup  on  those  results  to find the
parameters $n$, $r$.  This method of reverse lookup works because -- as you will
soon  see  --  the   lookup   curves  are  monotonically  increasing/decreasing.

In practice, it  is  sufficient  to  calculate  about 50 step responses for each
order  $n$  and  interpolate  between  those values when performing the  lookup.

As mentioned earlier, a remarkable  observation  is  that the parameters $r$ and
$n$ are independent of time and amplitude (and offset); that is,  the normalised
step response does not change its  shape when the parameter $T$ is changed. This
is fantastic,  because  it  allows  us  to eliminate a dimension from the lookup
table.

If $T=1$, $K_s=1$  and  $y_0=0$,  equations  \ref{eq:ptn}, \ref{eq:hudzovic} and
\ref{eq:sani}  can  be   used  to  calculate  a  series  of  transfer  functions
$G_n(s,r)$, their time domain step responses $g_{r,n}(t)$ can be calculated, and
they  can  be  characterised  by  calculating $g_{T_u/T_g}$  and  $g_{\lambda}$.

These two values alone aren't yet enough. The  result will still be dependent on
$T_g$ or $t_{50}$. In order to fully ``denormalise'' the result, $g_{1/T_g}$ and
$g_{1/t_{50}}$ must also be calculated for each step response.

\clearpage

\begin{figure}[t]
    \includegraphics[width=\linewidth]{images/hudzovic_curves_tu_tg}
    \caption{P. Hudzovic lookup curves. Top: Tu/Tg, bottom:1/Tg}
    \label{fig:hudzovic_tu_tg}
\end{figure}
\begin{figure}[t]
    \includegraphics[width=\linewidth]{images/hudzovic_curves_t10_t50_t90}
    \caption{P. Hudzovic lookup curves. Top: (t90-t10/t50), bottom: 1/t50}
    \label{fig:hudzovic_t3}
\end{figure}

\clearpage

\begin{figure}[t]
    \includegraphics[width=\linewidth]{images/sani_curves_tu_tg}
    \caption{L. Sani lookup curves. Top: Tu/Tg, bottom: 1/Tg lookup}
    \label{fig:sani_tu_tg}
\end{figure}
\begin{figure}[t]
    \includegraphics[width=\linewidth]{images/sani_curves_t10_t50_t90}
    \caption{L. Sani lookup curves. Top: (t90-t10)/t50, bottom: 1/t50}
    \label{fig:sani_t3}
\end{figure}

\clearpage

The various characterisations of the step response  function  $g(t)$  can all be
expressed as functions of $r$ and $n$:

\begin{align}
    g_{T_u/T_g}  &= g_{T_u/T_g}(r, n) \\
    g_{1/T_g}    &= g_{1/T_g}(r,n) \\
    g_{\lambda}  &= g_{\lambda}(r,n) \\
    g_{1/t_{50}} &= g_{1/t_{50}}(r,n)
\end{align}

Using P.  Hudzovic's  approach from equations \ref{eq:ptn} and \ref{eq:hudzovic}
the four functions $g_{T_u/T_g}(r, n)$, $g_{1/T_g}(r,n)$, $g_{\lambda}(r,n)$ and
$g_{1/t_{50}}(r,n)$  are  evaluated.  The figures  \ref{fig:hudzovic_tu_tg}  and
\ref{fig:hudzovic_t3} are obtained.

Similarly,  using   L.   Sani's   approach   from   equations  \ref{eq:ptn}  and
\ref{eq:sani} we  again  evaluate the same four functions and obtain the figures
\ref{fig:sani_tu_tg} and \ref{fig:sani_t3}.

What these plots show beautifully is that the  higher the order $n$, the steeper
-- or more ``complex'' -- the step response becomes (smaller  values of $T_g$ or
$t_{90}-t_{10}$ mean faster rise times of the step responses). Another important
thing  to observe is how lower orders of $G_n(s,r)$ aren't able to rise as  fast
as higher orders are able to, \textbf{regardless} of $r$ and $T$.  This can also
be seen in all four figures: Lower orders  cannot  reach  ratios of $T_u/T_g$ or
$\lambda$ that higher orders can.

\subsubsection*{Calculating T, r, n, by using Lookup Curves}

As already mentioned, determining the  complexity of a step response is directly
related  to  the  required  order $n$ of the model. Finding $n$ is now a  simple
matter of computing $\textrm{plant}_{T_u/T_g}$ or $\textrm{plant}_{\lambda}$ and
iterating     through    the    different    curves     either     in     figure
\ref{fig:hudzovic_tu_tg},   \ref{fig:sani_tu_tg},    \ref{fig:hudzovic_t3},   or
\ref{fig:sani_t3}  until  a value of $n$ is found that satisfies one  of  either
conditions:

\begin{align}
    g_{T_u/T_g}(r, n)\lvert_{r=max} \hspace{2mm} &\le \hspace{2mm} plant_{T_u/T_g} \\
    g_{\lambda}(r, n)\lvert_{r=max} \hspace{2mm} &\le \hspace{2mm} plant_{\lambda} \label{eq:find_n}
\end{align}

Ideally, $n$ should be as small as possible.

With the parameter $n$ defined, the next step is to find the  intersection point
of the horizontal line that passes through either  $\textrm{plant}_{T_u/T_g}$ or
$\textrm{plant}_{\lambda}$   and   $g_{T_u/T_g}(r,n)$   or   $g_{\lambda}(r,n)$,
respectively.  This will yield parameter $r$. This can be achieved by solving  a
simple  line  intersection  equation  and plugging in the locations of  the  two
horizontal lines.

The   last   parameter,   $T$,   can  finally  be   determined   by   evaluating
$\textrm{plant}_{g_{1/T_g}}(r,    n)$    or    $\textrm{plant}_{t_{50}}    \cdot
g_{1/t_{50}}$ Graphically, this equates to finding the intersection point of the
vertical line going through $r$ and the function  $g_{1/T_g}(r,  n)$  in  figure
\ref{fig:hudzovic_tu_tg} and multiplying the result by $T_g$.

Each    of   the   figures    \ref{fig:hudzovic_tu_tg},    \ref{fig:sani_tu_tg},
\ref{fig:hudzovic_t3}, and \ref{fig:sani_t3}  contain  a  dashed  line, which is
supposed to demonstrate  an  example  lookup,  to  help  visualise  the process.

\subsubsection*{Calculating T, r, n, by using Interpolation Formulae}

Instead   of   having   to   generate  the  lookup   curves   seen   in   figure
\ref{fig:sani_t3},  Sani\cite{ref:sani} included two formulae that very  closely
approximate these curves. They are:

\begin{equation}
    \frac{t_{50}}{T} = \log(2)-1+\frac{1-r^{n}}{1-r}
    \label{eq:T_t50}
\end{equation}
\begin{equation}
    \frac{t_{90}-t_{10}}{T} = 1.315 \sqrt{3.8\frac{1-r^{2n}}{1-r^2} - 1}
\end{equation}

Using  the  formula for calculating $\lambda$ in equation  \ref{eq:t10_t50_t90},
the interpolation formulae can be made independent of $T$:

\begin{equation}
    \frac{t_{90}-t_{10}}{t_{50}} = \frac{1.315 \sqrt{3.8\frac{1-r^{2n}}{1-r^2} - 1}}{\log(2)-1+\frac{1-r^{n}}{1-r}}
    \label{eq:sani_interpolation}
\end{equation}

Plotting  the  functions $\frac{t_{90}-t_{10}}{t_{50}}$  and  $\frac{1}{t_{50}}$
yields  nearly  identical   curves   as   seen   in   figure  \ref{fig:sani_t3}.

The benefit of using these interpolation formulae over a lookup table is ease of
implementation, lower memory footprint and  flexibility.  Orders  of  $n$ can be
chosen  arbitrarily,  whereas  with lookup tables an entry must exist for  every
order that should be supported.

Finding the parameter $n$ is identical to the previous  example  using  equation
\ref{eq:find_n}, except that instead of  using  a  lookup table, the function in
equation \ref{eq:sani_interpolation} is evaluated.

Finding the parameter $r$ can be achieved by performing a binary search  on  the
function  in  equation  \ref{eq:sani_interpolation}.  This   works  because  the
function is monotonically decreasing.

Finding $T$ is  a  simple  matter  of  solving  equation  \ref{eq:T_t50} for the
parameter $T$ and  evaluating  it using the found parameters $n$, $r$ as well as
the plant's $t_{50}$ value $\textrm{plant}_{t_{50}}$.