% !TeX root = ../libro.tex
% !TeX encoding = utf8
%
%*******************************************************
% Summary
%*******************************************************

\selectlanguage{english}
\chapter*{Summary}\label{ch:summary}
%\addcontentsline{toc}{chapter}{Summar}

Nowadays experimental research in Neural Networks is more advanced than theoretical
results.
From this we aim to establish a solid mathematical theory so as to optimize the current neural network models.

As a result of our study, we have proposed a novel neural
network model, and adapted and optimized
evaluation and learning methods to it.
Moreover, we have discovered some theorems that prove the
equivalence among some activation functions, and hence we propose a new
 algorithm to initialize the weights of neural networks. Thanks to the
first result, we obtain a criterion to choose the most
suitable activation function to maintain accuracy and reduce computational costs.
 Thanks to the second one, we might accelerate
learning convergence methods.

In addition, the models, methods and algorithms have been
implemented in Julia, resulting in the \textit{OptimizedNeuralNetwork.jl} library.

All the theory development, designs, decisions and results are
written in this memory, which has the following structure:
\begin{itemize}
 \item \textbf{Chapter \ref{ch00:methodology}: Description of the methodology followed.} We have organized our project according to an agile philosophy  based on personas methodology, user stories, milestones and tests. This method has conducted and linked mathematical and technical results and implementations, giving them coherence and validation methods.

 \item \textbf{Chapter \ref{chapter:Introduction-neuronal-networks}: Description of the learning problem.} We defined the characteristic and type of machine learning problems. We will focus on supervised learning ones.

 \item \textbf{Chapter \ref{ch03:teoria-aproximar}:  Approximation theory.} In order to establish a solid theory, we will start our work trying to solve machine learning problems by traditional approximation methods.  The main result we prove is the Stone Weierstrass's theorem. As a conclusion of this chapter we will achieve knowledge of the virtues and faults of traditional methods and understanding the necessity of new methods and structures such as neural networks.

 \item \textbf{Chapter \ref{chapter4:redes-neuronales-aproximador-universal}: Neural networks are universal approximators.}  In this chapter we introduce our neural
 network model and compare it with the conventional ones. In order to show it is well
 defined, we will prove the universal convergence of our model to any measurable
 function. In addition, we will give some results about how our model solves
 classification and regression problems as its number of neurons rises. Finally, we
 will argue if all of those math results can actually solve real life problems. The
 idea behind the debate is the computability representation of real numbers.

 \item \textbf{Chapter  \ref{chapter:construir-redes-neuronales}: The design and implementation of neural networks.} We will carefully  describe the design and
 implementation of our model of neural network. Thanks to that we will obtain some
 mathematical results about bias and classification function. This will be useful to
 compare our model with the conventional ones and justify
our selection. Moreover, we will explain, justify and design  learning and evaluation
methods to our model. These methods are optimized versions of Forward Propagation and
Backpropagation.

\item \textbf{Chapter \ref{funciones-activacion-democraticas-mas-demoscraticas}: Democratization of activation functions.}
We will explain in this chapter if there are better activation functions. In this
direction we will prove two original results which show that there are families of
activation functions that under the same conditions will solve problems with the same
accuracy. As a result, if we compare the computational cost of the members of those
families and choose the faster one, we will obtain a method to optimize evaluation and
learning of neural networks without loss of accuracy. We have used the Wilcoxon
signed-rank test as a statistical hypothesis test so as to give a rigorous study of
our criteria.

\item \textbf{Chapter \ref{section:inicializar_pesos}: Weight initializing algorithm.}
Since Backpropagation and other iterative  methods are sensitive to the initial
value of a neural network, we will show an original method to initialize its weights
from training data. This process not only will produce a better initial step but also
has lower computational cost than Backpropagation.  To test the potential of this
method we will use the Wilcoxon signed-rank test again and also, from the experiment's
requirements we will design and create our OptimizedNeuralNetwork.jl library. In this chapter we
will also explain every decision done during the design and implementation of the
library in order to be as efficient as possible.

\item \textbf{Chapter \ref{ch08:genetic-selection}: Use of genetic algorithm in the selection of activation function.}
In this chapter we will explain a future work. Given a fixed number of neurons, the
selection of its activation function may be crucial to reduce the train and test
error.  However, adding more free params to the search space increases its complexity
and at same time the cost of finding a solution.  Nevertheless, the result obtained in
chapter \ref{funciones-activacion-democraticas-mas-demoscraticas} and a property of our neural model will reduce the space complexity.

\item \textbf{Chapter \ref{ch09:conclusion}: Conclusions.}
\end{itemize}

\paragraph{KEYWORDS:}
\begin{itemize*}[label=,itemsep=1em,itemjoin=\hspace{1em}]
  \item neural networks
  \item optimization
  \item activation functions
  \item weights initializing
  \item machine learning library
\end{itemize*}

% Al finalizar el resumen en inglés, volvemos a seleccionar el idioma español para el documento
\selectlanguage{spanish}
\endinput


\documentclass{homework}
\course{Math 5522H}
\author{Jim Fowler}
\input{preamble}
\DeclareMathOperator{\Res}{Res}

\begin{document}
\maketitle

\begin{inspiration} % https://frinkiac.com/caption/S01E04/240324
  As far as anybody knows, we're a nice normal family.
\byline{Homer Simpson in S01E04}
\end{inspiration}

\section{Terminology}

\begin{problem}
  What is a \textbf{normal family} of holomorphic functions on an open set $U$?
\end{problem}

\begin{problem}\label{terminology-infinite-product}What does it mean to say that the infinite product
  \(
    \prod_{n=1}^\infty \left( 1 + a_n \right)
  \)
  converges?
\end{problem}

\section{Numericals}

\begin{problem}
  For which $z$ does the series
  \[
    \sum_{n=0}^\infty \frac{\cos \left( nz \right)}{n!}
  \]
  converge?

  Does it converge to a holomorphic function you recognize?
\end{problem}

\begin{problem}
  For which $z$ does the series
  \[
    \sum_{0 \neq n \in \Z} \left( \frac{1}{z+n} - \frac{1}{n} \right)
  \]
  converge?

  Does it converge to a meromorphic function you recognize?
  \textit{Hint:} \ref{residues-all-one}.
\end{problem}

\section{Exploration}

\begin{problem}\label{poisson-summation}Describe conditions on $f$ so
  that, for a suitable $\gamma_1$ and $\gamma_2$,
  \[
    \sum_{n=-\infty}^\infty f(n) = \int_{\gamma_1} \frac{f(z)}{e^{2\pi i z} - 1} \, dz - \int_{\gamma_2} \frac{f(z)}{e^{2\pi i z} - 1} \, dz.
  \]
  Then expand $1/(e^{2\pi i z} - 1)$ as a geometric series to deduce the \textbf{Poisson summation formula}
  \[
    \sum_{n=-\infty}^\infty f(n) = \sum_{n=-\infty}^\infty \hat{f}(n)
  \]
  where $\hat{f}$ is the Fourier transform, i.e.,
  \[
    {\hat {f}}(\xi ) := \int _{-\infty }^{\infty} f(x) \, e^{-2\pi ix \xi} \,dx.
  \]
\end{problem}

\begin{problem}\label{modularity}For $a > 0$ define
  $\vartheta(a) = \sum_{n=-\infty}^\infty e^{-a \pi z^2}$.  Recalling
  \ref{fourier-transform-itself} shows that if $f(z) = e^{-a \pi z^2}$
  we can compute $\hat{f}(z)$.  Use $\hat{f}(z)$ and
  \ref{poisson-summation} to verify
  \[
    \vartheta(a) = \vartheta(1/a) / \sqrt{a}.
  \]
\end{problem}

\begin{problem}
  Having just celebrated $\pi$-day, some computer calculations revealed
\begin{align*}
  \vartheta(1/(4\pi)) = \sum_{n=-\infty}^\infty e^{-n^2/4} &=
3.544907701811032\textbf{10533931955126186}\ldots \\
  2\sqrt{\pi} &=
3.544907701811032\textbf{05459633496668229}\ldots
\end{align*}
Is my computer broken?  (For more, see \texttt{https://arxiv.org/abs/1809.10907}.)
\end{problem}

\begin{problem}
  Define
  \[
    \wp(z) = \frac{1}{z^2} + \sum_{0 \neq \lambda \in \Z[i]} \left( \frac{1}{(z - \lambda)^2} - \frac{1}{\lambda^2} \right)
  \]
  which is \textbf{Weierstrass' elliptic function} on the square lattice.  Use \ref{sum-one-over-gaussian-integers} to verify that $\wp(z)$ converges.
\end{problem}

\begin{problem}\label{elliptic-derivative-periodic}Compute $\wp'(z)$ by differentiating term-by-term to show that $\wp'(z + \lambda) = \wp'(z)$ for $\lambda \in \Z[i]$.
\end{problem}

\begin{problem}
  Compare $\wp(z)$ and $\wp(-z)$.  Use this to relate $\wp(1/2)$ and $\wp(-1/2)$ and to relate $\wp(i/2)$ and $\wp(-i/2)$ and with \ref{elliptic-derivative-periodic}, conclude that $\wp(z + \lambda) = \wp(z)$ for $\lambda \in \Z[i]$.
\end{problem}

\section{Prove or Disprove and Salvage if Possible}

\begin{problem}\label{differentiating-taylor-series}If
  $f(z) = \sum_{n=0}^\infty a_n z^n$ has radius of convergence $r$,
  then the series
  \[
    g(z) = \sum_{n=1}^\infty n a_n z^{n-1}
  \]
  has radius of convergence $r$ and if $|z| < r$ then $f'(z) = g(z)$.
\end{problem}

\begin{problem}\label{sum-one-over-gaussian-integers}For $n > 2$, the
  series \( \displaystyle\sum_{0 \neq \lambda \in \Z[i]} \frac{1}{|\lambda|^n} \).
  converges.
\end{problem}

\begin{problem}\label{elliptic-more-than-two-poles}There is no meromorphic function $f : \C \to \hat{\C}$ having simple poles at $a+bi \in \Z[i]$ and satisfying $f(z) = f(z+a+bi)$ for all $z \in \C$ and $a+bi \in \Z[i]$.
\end{problem}

\begin{problem}\label{infinite-product-sine}For all $z \in \C$, we have
  \(
    \sin \left( \pi z \right) = \pi \displaystyle\prod_{n=1}^\infty \left( 1 - \frac{z^2}{n^2} \right)
  \). % missing factor of z
\end{problem}

\begin{problem}\label{normal-family-example}Define
  $f_w : B_1(0) \to \C$ by $f_w(z) = z/(z-w)$.  The family of
  functions $\mathcal{F} := \{ f_w \mid w \in \C \}$ is a normal
  family.
\end{problem}

\begin{problem}\label{derivatives-normal-then-not-normal}Suppose $\mathcal{F}$ is a family of functions defined on the domain $B_1(0)$.  If $\mathcal{F}' := \{ f' \mid f \in \mathcal{F} \}$, the
  family consisting of derivatives of functions in the family
  $\mathcal{F}$, is normal, then the original family $\mathcal{F}$ is
  normal. % assume that \{ f(0) \mid f \in \mathcal{F} \} is bounded
\end{problem}

\end{document}


\input{Setup.tex}
\begin{document}

\mainmatter

\section{Hebbian Learning}
\vspace{1mm}
\begin{quote}
\textsl{When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes places in one or both cells such that A's efficiency, as one of the cells firing B, is increased.}\cite{Hebb1949}
\end{quote}

\section{Spike Timing Dependant Plasticity}
Write about STDP, see the 'Postulates.pdf'. \\

With spike timing dependant plasticity we want to study how two neurons change their synaptic strength based on the time delay between spikes. When $\theta(t)_j > \pi$ we say that the neuron $\theta_j$ spikes at time $t$.
Let us say that $\theta_i$ spikes at time $t_i$ and $\theta_j$ spikes at $t_j$. Taking the time difference $\Delta t_{ij}$ as $t_j - t_i$, we can say that when $\Delta t_{ij} > 0$ the spikes are correlated (there exists a temporally causal relation), and we can model an increase in synaptic strength of the connection $K_{ij}$ from $\theta_i$ to $\theta_j$. In the same fashion we can decrease $K_{ji}$ when $\Delta t_{ij} < 0$ as there is no causal relation. \\
The functions $W(t)$ that relate $\Delta t_{ij}$ to $\Delta K_{ij}$ are called \textsl{learning windows},  as they define a range in which $K_{ij}$ is able to adapt. When signals between neurons show a very large time difference (negative or positive) we do not expect them to be correlated. Because the learning windows are generally not symmetrical we can also expect an asymmetrical adjacency matrix.\\
Another characteristic is the integral over the learning window. A window with a negative integral directs synaptic strengths mostly towards inhibitory behaviour, and vice versa with a positive integral. An integral of zero would mean that both inhibitory and excitatory synapses are stimulated equally.\\

Now that we have a feeling of how \STDP works, we need to formulate the behaviour exactly. One by one, we will denote our ideas into mathematics.

\subsection{Biphasic learning windows}

\subsubsection{Kempter1999}
Following the notation in \cite{Kempter1999}, we will denote the spike train coming from each neuron $\theta_i$ as $S_i^{\rm out}(t) = \sum_{n} \delta (t-t_{i}^{n})$, where $t_{i}^{n}$ is the time that $\theta_i$ has fired. Similarly, we will denote the spike train coming into each neuron $\theta_i$ as $S_i^{\rm in}(t) = \sum_{f} \delta (t-t_{i}^{f})$. Now we can say that the synaptic strengths are adjusted as:
\begin{align}
\Delta K_{ij} &= \int_{t}^{t+\mathcal{T}} w^{\rm{out}} S_i^{\rm out}(\tau) + w^{\rm{in}} S_{j}^{\rm {in}}(\tau) \mathrm{d}\tau
+ \iint_{t}^{t+\mathcal{T}} W( \tau^\prime - \tau) S_{i}^{\rm out}(\tau) S_{j}^{\rm in}( \tau^\prime) \mathrm{d} \tau \mathrm{d} \tau^\prime
\label{eq:KempterSTDPFormulation1} \\
&= \sum_{t_i^{n}\in \mathcal{T}} w^{\mathrm{out}} + \sum_{t_{j}^{f} \in \mathcal{T}} w^{\mathrm{in}} + \sum_{t_{j}^{f}, t_i^{n} \in \mathcal{T}} W (t_{j}^{f}-t_i^{n} ) \label{eq:KempterSTDPFormulation2}
\end{align}
where \cite{Kempter1999} proposes the following learning window:
\begin{align}
W(t)_K = \alpha
\begin{cases}
\left[A_{p}\left(1-\frac{t}{\tilde{\tau}_{p}}\right)+A_{n}\left(1-\frac{t}{\tilde{\tau}_{n}}\right)\right] \cdot \exp \left( \frac{t}{\tau_{\rm syn}} \right) & \text{for } t \leq 0 \\
A_{p} \cdot \exp \left(-\frac{t}{\tau_{p}}\right) + A_{n} \cdot \exp \left(-\frac{t}{\tau_{n}} \right) & \text{for } t > 0
\end{cases} \label{eq:learningwindowKempter1999}
\end{align}
Here $t$ is the delay between presynaptic spike arrival and postsynaptic firing, $\alpha$ is a small learning parameter and all $\tau$ are time constants. Numerical values are usually  $\alpha = 0.05$, $\tau_{\rm syn} = 5$ ms, $\tau_{p} = 1$ ms, $\tau_{n} = 20$ ms and $A_p = 1$ and $A_{n} = -1$. $\tilde{\tau}_{p} \equiv \tau_{\rm syn} \tau_{p} / (\tau_{\rm syn} + \tau_{p})$ and $\tilde{\tau}_{n} \equiv \tau_{\rm syn} \tau_{n} / (\tau_{\rm syn} + \tau_{n})$. \\
$\int W(s)_K \mathrm{d}s = 2.56 \times 10^{-4}$.

\subsubsection{Song 2000}
The first formulation of \STDP as a mathematical model was in \cite{Song2000}. It is postulated without being concerned about the biological aspect too much. The synaptic strengths are simply updated as:
\begin{align}
\Delta K_{ij} &= K_{ij} \sum_{t_{j}^{f}, t_i^{n} \in \mathcal{T}} W (t_{j}^{f}-t_i^{n} ) \label{eq:SongSTDPFormulation}
\end{align}
The learning window is defined as a discontinuous function:
\begin{align}
W(t)_S =
\begin{cases}
A_{p} \cdot \exp \left(\frac{-t}{\tau_p}\right) & \text{for } s > 0 \\
A_{n} \cdot \exp \left(\frac{t}{\tau_n}\right)  & \text{for } s \leq 0
\end{cases} \label{eq:learningwindowKempter1999}
\end{align}
where we will use $A_p = 0.005$, $A_n = -0.00525$ and $\tau_p = \tau_n = 20$ ms. $\int W(s)_S \mathrm{d}s = -3.70 \times 10^{-4}$ so we expect the weights to be suppressed towards a negative value.

\begin{figure}[H]
\centering
\includegraphics[width = \textwidth]{../Figures/LearningWindowsBiphasic.png}
\caption{Two different biphasic learning windows. We can see how in $W(t)_s$ a larger weight is put on the anti-Hebbian learning.}
\label{fig:LearningWindowsBiphasic}
\end{figure}

\subsection{Triphasic Learning windows}
Based off of neuronal actiity in the hippocmpus, biologically inspired triphasic windows have been developed. The papers describing the learning process are not specifying exactly how the weights are updated, as we have seen before there are a few options. We assume the change is simply:
\begin{align}
\Delta K_{ij} =  \sum_{t_{j}^{f}, t_i^{n} \in \mathcal{T}} W (t_{j}^{f}-t_i^{n} ) \label{eq:WaddingtonSTDPFormulation}
\end{align}
as there is no mention of other parameters.

\subsubsection{Chrol-Cannon 2012}
\begin{align}
W(t)_C = A_{p} \cdot \exp \left(\frac{-\left(t - 15 \right)^{2}}{ \tau_{p}}\right) - A_{n} \cdot \exp \left(\frac{-\left(t - 20\right)^{2}}{ \tau_{n}}\right)  \label{eq:learningwindowChrolCannon2012}
\end{align}
where $A_{p}=0.23$, $A_{n}=0.15$, $\tau_{p}=200$ and $\tau_n = 2000$. $\int W(s)_C \mathrm{d}s = -60.0 \times 10^{-4}$.

\subsubsection{Waddington 2014}
The learning window is then defined as:
\begin{align}
W(t)_W =  A \left[1-\frac{\left(t-\alpha\right)^{2}}{\alpha^{2}}\right] \cdot \exp \left(\frac{-\left|t - \alpha\right|}{\alpha}\right) \label{eq:learningwindowWaddington2014}
\end{align}
We will use $A = 0.1$ and $\alpha = 4.0$ ms . $\int W(s)_W \mathrm{d}s = -8.0 \times 10^{-4}$.

\begin{figure}[H]
\centering
\includegraphics[width = \textwidth]{../Figures/LearningWindowsTriphasic.png}
\caption{The mean degree $\kmean$ evolves over time as a function of the spiking behaviour, just like we expect.}
\label{fig:LearningWindowsTriphasic}
\end{figure}

\subsection{Interpretation}
The learning windows generally have $W(t^{\ast}) = 0$ for $t^{\ast} > 0$. This means that no learning will take place when the delay between neuron spikes is exactly $t^{\ast} > 0$. The triphasic windows show two of those points.

\subsection{Results}
\begin{figure}[H]
\centering
\includegraphics[width = \textwidth, trim={60 0 60 0}]{../Figures/LearningWithoutScaling.png}
\caption{Two different triphasic learning windows. We can see how in $W(t)_C$  a much higher penalty is given to signals that arrive too early or too late. The moments at which signals should peak also differ by about 10 ms.}
\label{fig:LearningWithoutScaling}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width = 0.2\textheight]{../Figures/STDPbeforeIP.png}
\caption{Results from \STDP.}
\label{fig:STDPbeforeIP}
\end{figure}

\subsection{Synaptic scaling}
There is no upper or lower bound on the synapse strength. Also generally, connection strengths are nonzero. This means that the notion of a \textsl{network} is lost.\\
One technique we can apply to keep the strengths within a definitive range is to scale homeostatically - a method where increases in synaptic strength will balance out any decreases by scaling:
\begin{align}
K_{ij}^s = K_{ij} \frac{\frac{1}{N} \sum_{i,j} K_{ij}}{\sum_{i} K_{ij}}
\end{align}
In this way, the out-degrees will remain constant. Using this approach, something has to remain constant, whether that is $\kmean$, or $\kmean^2$ or any other property of the adjacency matrix.

\subsection{Intrinsic plasticity}
Instead of scaling the weights to preserve a certain quantity in the network, we can allow the neurons to adjust the sensitivity to incoming signals. So when some synaptic strengths are increased, we can reduce the excitability. In \cite{Song2017} such a method is introduced in detail. We can simply update $\eta_{i} \rightarrow \eta_{i} + \eta_{\max } \cdot \phi_{i}$ where
\begin{align}
\phi_{i} (t) =\left\{\begin{array}{l}-\alpha \cdot \exp \left(\frac{T_{\min }-t}{T_{\rm min }}\right) \text { if } t<T_{ \rm min } \\ \alpha \cdot \exp \left(\frac{t-T_{\rm max }}{T_{\max }}\right) \text { if } t >T_{\rm max } \\ 0, \text { otherwise }\end{array}\right.
\end{align}

\begin{figure}[H]
\centering
\includegraphics[width = 0.5\textwidth]{../Figures/IPlearningFunction.png}
\caption{The learning behaviour for the intrinsic plastiity.}
\label{fig:IPlearningFunction}
\end{figure}

\subsection{Custom learning windows}
Some of the behaviour we want to observe is currently not accounted for: synaptic strengths are unbounded and are generally nonzero. How can we model the behaviour where synaptic strengths can also settle on being zero? Perhaps a better idea than updating the synaptic strength by adding a new value, we can scale it.
\begin{itemize}
\item When $\Delta t_{ij}$ is very large (both positive and negative) we expect no change in the synaptic strength: $W(-\infty) = W(-\infty) = 0$.
\item We expect a specific positive time delay to yield the most amount of synaptic strengthening: $\Delta t_{\rm best} = \underset{x}{\arg \max}  W(t)$
\item We expect small connections that have been reducing in size to quickly become zero,
\item A better scaling system would be that not the out- or in-degree is constant, but
\end{itemize}

\bibliographystyle{utphys}
\small{\bibliography{references}}

\label{LastPage}~

\end{document}


% Written by Anders Sjoqvist and Ulf Lundstrom, 2009
% The main sources are: tinyKACTL, Beta and Wikipedia

\chapter{Mathematics}

\section{Recurrences}
If $a_n = c_1 a_{n-1} + \dots + c_k a_{n-k}$, and $r_1, \dots, r_k$ are distinct roots of $x^k + c_1 x^{k-1} + \dots + c_k$, there are $d_1, \dots, d_k$ s.t.
\[a_n = d_1r_1^n + \dots + d_kr_k^n. \]
Non-distinct roots $r$ become polynomial factors, e.g. $a_n = (d_1n + d_2)r^n$.

\section{Trigonometry}
\begin{align*}
\sin(v+w)&{}=\sin v\cos w+\cos v\sin w\\
cos(v+w)&{}=\cos v\cos w-\sin v\sin w\\
\end{align*}
\begin{align*}
\tan(v+w)&{}=\dfrac{\tan v+\tan w}{1-\tan v\tan w}\\
\sin v+\sin w&{}=2\sin\dfrac{v+w}{2}\cos\dfrac{v-w}{2}\\
\cos v+\cos w&{}=2\cos\dfrac{v+w}{2}\cos\dfrac{v-w}{2}
\end{align*}
\[ (V+W)\tan(v-w)/2{}=(V-W)\tan(v+w)/2 \]
where $V, W$ are lengths of sides opposite angles $v, w$.
\begin{align*}
	a\cos x+b\sin x&=r\cos(x-\phi)\\
	a\sin x+b\cos x&=r\sin(x+\phi)
\end{align*}
where $r=\sqrt{a^2+b^2}, \phi=\operatorname{atan2}(b,a)$.

\section{Geometry}

\subsection{Triangles}
Side lengths: $a,b,c$\\
Semiperimeter: $p=\dfrac{a+b+c}{2}$\\
Area: $A=\sqrt{p(p-a)(p-b)(p-c)}$\\
Circumradius: $R=\dfrac{abc}{4A}$\\
Inradius: $r=\dfrac{A}{p}$\\
Length of median (divides triangle into two equal-area triangles): $m_a=\tfrac{1}{2}\sqrt{2b^2+2c^2-a^2}$\\
Length of bisector (divides angles in two): $s_a=\sqrt{bc\left[1-\left(\dfrac{a}{b+c}\right)^2\right]}$\\
Law of sines: $\dfrac{\sin\alpha}{a}=\dfrac{\sin\beta}{b}=\dfrac{\sin\gamma}{c}=\dfrac{1}{2R}$\\
Law of cosines: $a^2=b^2+c^2-2bc\cos\alpha$\\
Law of tangents: $\dfrac{a+b}{a-b}=\dfrac{\tan\dfrac{\alpha+\beta}{2}}{\tan\dfrac{\alpha-\beta}{2}}$\\

\subsection{Spherical coordinates}
\centerline{\includegraphics[width=25mm]{../content/math/sphericalCoordinates}}
\[\begin{array}{cc}
x = r\sin\theta\cos\phi & r = \sqrt{x^2+y^2+z^2}\\
y = r\sin\theta\sin\phi & \theta = \textrm{acos}(z/\sqrt{x^2+y^2+z^2})\\
z = r\cos\theta & \phi = \textrm{atan2}(y,x)
\end{array}\]

\section{Derivatives/Integrals}
\begin{align*}
	\dfrac{d}{dx}\arcsin x = \dfrac{1}{\sqrt{1-x^2}} &&& \dfrac{d}{dx}\arccos x = -\dfrac{1}{\sqrt{1-x^2}} \\
	\dfrac{d}{dx}\tan x = 1+\tan^2 x &&& \dfrac{d}{dx}\arctan x = \dfrac{1}{1+x^2} \\
	\int\tan ax = -\dfrac{\ln|\cos ax|}{a} &&& \int x\sin ax = \dfrac{\sin ax-ax \cos ax}{a^2} \\
	\int e^{-x^2} = \frac{\sqrt \pi}{2} \text{erf}(x) &&& \int xe^{ax}dx = \frac{e^{ax}}{a^2}(ax-1)
\end{align*}

Integration by parts:
\[\int_a^bf(x)g(x)dx = [F(x)g(x)]_a^b-\int_a^bF(x)g'(x)dx\]

\section{Sums}
\[ c^a + c^{a+1} + \dots + c^{b} = \frac{c^{b+1} - c^a}{c-1}, c \neq 1 \]
\begin{align*}
	1 + 2 + 3 + \dots + n &= \frac{n(n+1)}{2} \\
	1^2 + 2^2 + 3^2 + \dots + n^2 &= \frac{n(2n+1)(n+1)}{6} \\
	1^3 + 2^3 + 3^3 + \dots + n^3 &= \frac{n^2(n+1)^2}{4} \\
	1^4 + 2^4 + 3^4 + \dots + n^4 &= \frac{n(n+1)(2n+1)(3n^2 + 3n - 1)}{30} \\
\end{align*}

\section{Series}
$$e^x = 1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\dots,\,(-\infty<x<\infty)$$
$$\ln(1+x) = x-\frac{x^2}{2}+\frac{x^3}{3}-\frac{x^4}{4}+\dots,\,(-1<x\leq1)$$
$$\sqrt{1+x} = 1+\frac{x}{2}-\frac{x^2}{8}+\frac{2x^3}{32}-\frac{5x^4}{128}+\dots,\,(-1\leq x\leq1)$$
$$\sin x = x-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}+\dots,\,(-\infty<x<\infty)$$
$$\cos x = 1-\frac{x^2}{2!}+\frac{x^4}{4!}-\frac{x^6}{6!}+\dots,\,(-\infty<x<\infty)$$


\section{Measurement Model}

The measurement model serves the goal of allowing us to retrieve the equivalent measurement values for a given state. This is necessary for calculating the Innovation, and thus the Kalman Gain, which allows us to update the state in the best way possible by taking both the predictions and the measurements into account. This use implies the necessity of being able to retrieve the measurements from the state, which is why we include the wheel velocities $v_l$ and $v_r$ in the state despite not actually using these values for navigation; without including them directly, it is hard to recover them from the remaining state variables.

The desired behavior is
\begin{equation}
    \boldsymbol{z}_{\textrm{equiv}} = \boldsymbol{H} \cdot \boldsymbol{\hat{x}}_{n,n-1}
\end{equation}

In words, we multiply $\boldsymbol{H}$ by the prediction for the current state made in the previous timestep to recover an equivalent measurement. Since our measurement vector is simply a subset of the state, our measurement model is
\begin{equation}
    \boldsymbol{H} =
    \begin{pmatrix}
    1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
    0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\
    0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 \\
    0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \\
    0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\
    0 & 0 & 0 & 0 & 0 & 0 & 0 & 1
    \end{pmatrix}
\end{equation}

We can then calculate our innovation,
\begin{equation}
    \boldsymbol{y}_{n} = \boldsymbol{z}_{n} - \boldsymbol{z}_{\textrm{equiv}}
\end{equation}


The previous chapter on finite state machines provided an introduction into high-level robot behavior specification. Another important topic along this thread is \textit{sequential decision making}\cite{Bertsekas2019}, where the robot must make a series of decisions to accomplish an objective in an optimal way.
This chapter provides an introduction to fundamental topics in decision making, including for problems where there is some uncertainty (e.g. uncertainty about the robot's state or about the environment).

\notessection{Sequential Decision Making}
In addition to the motion planning and control problems discussed in earlier chapters (which focus on low-level tasks), there exist a broad range of situations where higher-level autonomous decision making is required. For example when deciding whether it is time for a self-driving car to cross an intersection, or whether a robot should first complete task A or task B.
Two of the fundamental challenges associated with robotic decision making are that \textit{sequences} of decisions must be made (which requires reasoning about future actions and observations) and that uncertainty may exist in the operating environment. This chapter presents a modeling framework for addressing decision making problems and will also introduce \textit{dynamic programming}, a fundamental approach for solving these problems .

\subsection{Deterministic Decision Making Problem}
The standard mathematical formulation for decision making problems includes several components: a model of the robot's behavior, a set of admissible controls, and a cost function. This set of components is quite similar to the components used in trajectory optimization problems discussed in previous chapters, however decision making problems are generally represented in \textit{discrete-time} rather than in \textit{continuous-time}\footnote{There is a continuous-time formulation, known as the Hamilton–Jacobi–Bellman formulation.}.

In the deterministic decision making problem, the model of the robot is expressed in \textit{discrete-time} as:
\begin{equation} \label{eq:detmodel}
\x_{k+1}= f_k(\x_k, \bu_k), \quad k = 0, \dots, N-1,
\end{equation}
where $\x$ is the robot's state, $\bu$ is the control, $f_k$ defines how the robot's state changes at time step $k$, and $N$ is an integer that defines a finite planning horizon for the decision making problem. There are generally no restrictions on how the functions $f_k$ are defined, they could come from a physics-based dynamics/kinematics model or even a higher-level state transition model similar to the finite state machine from the previous chapter.

It is also generally assumed that only some control actions are admissible at a given state, which denoted by the set $\mathcal{U}(\x_k)$. For example a car may only have an option to turn left or right when it is at an intersection.
Therefore the control constraints for the robot at time step $k$ are given by:
\begin{equation} \label{eq:SDMPconstraints}
\bu_k \in \mathcal{U}(\x_k).
\end{equation}
Again, there are generally no restrictions on how the set of admissible control is defined. For example $\mathcal{U}(\x_k)$ could be a finite set of actions, it could be a convex region of allowable inputs, etc.

The cost function is assumed to be \textit{additive}, and is defined as:
\begin{equation} \label{eq:detcost}
J(\x_0, \bu_0, \dots, \bu_{N-1}) = g_N(\x_N) + \sum_{k=0}^{N-1} g_k(\x_k, \bu_k),
\end{equation}
where $g_N$ is a terminal state cost function and $g_k$ for $k=0,\dots,N-1$ are stage cost functions. These individual cost functions are also not restricted to a particular form (e.g. convex, differentiable, etc.).

\begin{definition}[Deterministic Decision Making Problem]
The deterministic decision making problem can be expressed for the system model \eqref{eq:detmodel}, control constraints \eqref{eq:SDMPconstraints}, and cost function \eqref{eq:detcost} as:
\begin{equation} \label{eq:dproblem}
J^*(\x_0) = \underset{\bu_k \in \mathcal{U}(\x_k), \:\:k=0,\dots,N-1}{\min} J(\x_0, \bu_0, \dots, \bu_{N-1}).
\end{equation}
\end{definition}

Notice that this problem is used to compute an \textit{open-loop} control sequence $\{\bu_0, \dots, \bu_{N-1}\}$ given an initial condition $\x_0$, which is similar to the trajectory optimization problems seen in earlier chapters.
However, this problem is generally quite hard to solve since there is no guarantee that the model \eqref{eq:detmodel} and cost function \eqref{eq:detcost} have any particular structure that can be leveraged to make the optimization problem amenable to numerical optimization algorithms. While it is theoretically possible to solve the problem through a brute force search over all possible combinations of sequences $\{\bu_0, \dots, \bu_{N-1}\}$, this leads to a combinatorial explosion of options and is therefore not possible in practical settings (except of course for very small problems).

\subsubsection{Principle of Optimality (Deterministic)}
Fortunately, there is in fact an underlying structure to the deterministic decision making problem that can be leveraged to make the problem easier to solve. This structure is commonly referred to as the \textit{principle of optimality}.
\begin{figure}[ht]
    \centering
    \includegraphics[width=0.5\textwidth]{tex/figs/ch20_figs/princopt1.png}
    \caption{Starting from point $a$, let the red path $a-b-e$ be the optimal path from $a$ to $e$, with a total cost of $J^*_{ae} = J_{ab}+J_{be}$. The principle of optimality in this case says that the path $b-e$ must therefore be the optimal path when starting from point $b$. This can be proven by contradiction, since if the path $b-c-e$ had a lower cost than path $b-e$ (i.e. $J_{bce} < J_{be}$), then the original path $a-b-e$ cannot be optimal!}
    \label{fig:princopt1}
\end{figure}

The principle of optimality for deterministic systems is that for a sequence of optimal decisions, the \textit{tail} of the optimal sequence is also optimal for a \textit{tail subproblem}. For a concrete example see Figure \ref{fig:princopt1}. This can greatly simplify the overall problem, since you can ``reuse'' optimal paths for different scenarios. More formally, the principle of optimality is given by the following theorem:
\begin{theorem}[Principle of Optimality (Deterministic)]
Let $\{\bu_0^*, \bu_1^* \dots, \bu_{N-1}^*\}$ be an optimal control sequence to the deterministic decision making problem \eqref{eq:dproblem} with a given initial condition $\x^*_0$, such that the resulting optimal state sequence is $\{\x_0^*, \x_1^* \dots, \x_{N}^*\}$. Then, the tail sequence $\{\bu_k^*, \dots, \bu_{N-1}^*\}$ is an optimal control sequence when starting from $\x^*_k$ and minimizing the cost from time $k$ to time $N$
\begin{equation*}
J_{\text{tail}}(\x_k, \bu_k, \dots, \bu_{N-1}) = g_N(\x_N) + \sum_{m=k}^{N-1} g_m(\x_m, \bu_m).
\end{equation*}
\end{theorem}

To see how the principle of optimality can be applied to simplify the decision making problem, consider the scenario in Figure \ref{fig:princopt2}. In this case it is desired to find an optimal path from point $b$ to point $f$, and it is assumed that optimal paths from $c$, $d$, and $e$ to $f$ are already known. A brute force search over all possible paths in this problem would require nine paths to be evaluated:
\begin{equation*}
\begin{split}
\{b-c-f, \quad b-c-d-f, \quad b-c-d-e-f, \quad b-d-c-f, \quad b-d-f, \\
\quad b-d-e-f, \quad b-e-d-c-f, \quad b-e-d-f, \quad b-e-f \}.
\end{split}
\end{equation*}
However, by leveraging the principle of optimality the number of candidate paths is reduced to three:
\begin{equation*}
b-c-f, \quad b-d-f, \quad b-e-f.
\end{equation*}
In other words, the principle of optimality allows the search to be performed over \textit{immediate} decisions by also concatenating the optimal tail decisions! This procedure is generally implemented backward in time, for example in Figure \ref{fig:princopt2} the point $f$ (the goal) is first evaluated, then the points $c$, $d$, and $e$, and then finally the point $b$.
\begin{figure}[ht]
    \centering
    \includegraphics[width=0.65\textwidth]{tex/figs/ch20_figs/princopt2.png}
    \caption{Suppose the optimal paths from points $c$, $d$ and $e$ to $f$ are known (shown in red). By using the principle of optimality, an optimal path from point $b$ to $f$ can be found by \textit{only} searching over paths from $b$ to $c$, $d$, and $e$, and determining the lowest cost from the candidates $\{J_{bc} + J^*_{cf}, J_{bd} + J^*_{df}, J_{be} + J^*_{ef}\}$. In other words, the \textit{optimal tails} can be leveraged to reduce the total number of paths that need to be considered when finding an optimal path from $b$ to $f$!}
    \label{fig:princopt2}
\end{figure}

\subsubsection{Dynamic Programming (Deterministic)}
The dynamic programming (DP) algorithm \textit{globally} solves the deterministic decision making problem \eqref{eq:dproblem} by leveraging the principle of optimality\footnote[][4\baselineskip]{Note that the principle of optimality is a fundamental property that is actually utilized in almost all decision making algorithms, including reinforcement learning.}. The dynamic programming algorithm is given in Algorithm \ref{alg:dDP}, where it can be seen that a backward-in-time recursion is used and at each step a \textit{local} optimization is performed (this local optimization is referred to as the \textit{Bellman} equation), leveraging the optimal \textit{tail} costs from the previous iteration.
\begin{algorithm}[ht]
 $J^*_N(\x_N) = g_n(\x_N),$ for all $\x_N$\\
 \For{$k=N-1$ \KwTo $0$}{
  $J^*_k(\x_k) = \underset{\bu_k \in \mathcal{U}(\x_k)}{\min}g_k(\x_k,\bu_k) + J^*_{k+1}(f_k(\x_k, \bu_k)),$ for all $\x_k$\\
 }
 \Return $J^*_0(\cdot),\dots,J^*_N(\cdot)$
 \caption{Dynamic Programming (Deterministic)}
 \label{alg:dDP}
\end{algorithm}
The output of the dynamic programming algorithm is a set of costs $J^*_k(\x_k)$ for each time step $k = 0, \dots,N$ and states $\x_k$, which provide the optimal \textit{tail} cost for the \textit{tail} subproblem.

Given an initial condition $\x_0$, the optimal control sequence $\{\bu^*_0,\dots,\bu^*_{N-1}\}$ that solves the deterministic decision making problem can be computed with a ``forward pass'', where:
\begin{equation*}
\bu_0^* = \underset{\bu_0 \in \mathcal{U}(\x_0)}{\arg \min}\:\:g_0(\x_0,\bu_0) + J^*_{1}(f_0(\x_0, \bu_0)).
\end{equation*}
The next state is then computed as $\x^*_1 = f_0(\x_0, \bu^*_0)$, and the process is repeated:
\begin{equation*}
\bu_1^* = \underset{\bu_1 \in \mathcal{U}(\x^*_1)}{\arg \min}\:\:g_1(\x^*_1,\bu_1) + J^*_{2}(f_1(\x^*_1, \bu_1)),
\end{equation*}
until the full trajectory and optimal control is specified.

Note that in practice the DP algorithm is not practical for continuously values states $\x$, since an infinite number of states would have to be iterated over. Therefore one possible modification to handle continuously valued states is to \textit{quantize} the state space into a finite set of states (other approaches, such as interpolation, are also possible). Also, it is interesting to note that the addition of control constraints can actually simplify the procedure, since it restricts the number of possible options that need to be considered!

\begin{example}[Deterministic Dynamic Programming] \label{ex:detDP}
\theoremstyle{definition}
Consider the environment shown in Figure \ref{fig:detDPprob}, where the goal is to start at point $a$ and reach point $h$ while incurring the smallest cost. In this problem the state is represented as the current location (i.e. $a$, $b$, etc.), and the control constraints are encoded by the arrows indicating possible directions of travel (e.g. at point $c$ it is possible to either go right or up, but not down or left). The cost of traversing between two points is also denoted in Figure \ref{fig:detDPprob}.
\begin{figure}[ht]
    \centering
    \includegraphics[width=0.8\textwidth]{tex/figs/ch20_figs/dpexample.png}
    \caption{A deterministic decision making problem where the goal is to move from point $a$ to point $h$ while incurring the minimal amount of cost. The red path indicates the optimal path. This problem is solved by dynamic programming in Example \ref{ex:detDP}.}
    \label{fig:detDPprob}
\end{figure}

To implement the DP algorithm, the final point $h$ is chosen as $\x_N$, and the DP recursion begins with:
\begin{equation*}
    J^*_N(h) = 0,
\end{equation*}
since there is no cost to stay at point $h$.
Moving backward in time, it can be seen that the possible states $\x_{N-1}$ that can transition to $\x_N = h$ are the points $h$, $e$, and $g$ (assuming it is possible to stay at $h$ with no cost). Therefore in the first step of the DP recursion:
\begin{equation*}
\begin{split}
J^*_{N-1}(h) &= 0 + J^*_N(h) = 0, \quad u^*_{N-1}(h) = \text{stay}. \\
J^*_{N-1}(e) &= 8 + J^*_N(h) = 8, \quad u^*_{N-1}(e) = \text{right}, \\
J^*_{N-1}(g) &= 2 + J^*_N(h) = 2, \quad u^*_{N-1}(g) = \text{up}, \\
\end{split}
\end{equation*}
Note that $J^*_k(h) = 0$ for all $k \leq N$, and therefore it will not be explicitly included in the following steps. In the next step:
\begin{equation*}
\begin{split}
J^*_{N-2}(e) &= 8 + J^*_{N-1}(h) = 8, \quad u^*_{N-2}(e) = \text{right}, \\
J^*_{N-2}(g) &= 2, \quad u^*_{N-2}(g) = \text{up}, \\
J^*_{N-2}(d) &= 3 + J^*_{N-1}(e) = 11, \quad u^*_{N-2}(d) = \text{right}, \\
J^*_{N-2}(f) &= 3 + J^*_{N-1}(g) = 5, \quad u^*_{N-2}(f) = \text{right}, \\
\end{split}
\end{equation*}
At this point, these optimal tail costs can be considered to be the optimal costs associated with control actions that lead from $e$, $g$, $d$, or $f$ to the end point $h$ in \textit{two} steps! Continuing on:
\begin{equation*}
\begin{split}
J^*_{N-3}(e) &= \min \{8 + J^*_{N-2}(h), \:2 + J^*_{N-2}(f)\} = 7, \quad u^*_{N-3}(e) = \text{down}, \\
J^*_{N-3}(g) &= 2, \quad u^*_{N-3}(g) = \text{up},  \\
J^*_{N-3}(d) &= 3 + J^*_{N-2}(e) = 11, \quad u^*_{N-3}(d) = \text{right}, \\
J^*_{N-3}(f) & = 5, \quad u^*_{N-3}(f) = \text{right}, \\
J^*_{N-3}(a) &= 8 + J^*_{N-2}(d) = 19, \quad u^*_{N-3}(a) = \text{right}, \\
J^*_{N-3}(c) &= \min \{5 + J^*_{N-2}(d), \:3 + J^*_{N-2}(f)\} = 8, \quad u^*_{N-3}(c) = \text{right}. \\
\end{split}
\end{equation*}
Interestingly, it can be seen that it is now possible to accomplish the objective (i.e. go from point $a$ to $h$) in 3 time steps (i.e. on path $a-d-e-f$) and incur an optimal cost of 19. However it turns out that an even lower cost is achievable if the number of time steps is increased further! Continuing the DP recursion:
\begin{equation*}
\begin{split}
J^*_{N-4}(e) & = 7, \quad u^*_{N-4}(e) = \text{down}, \\
J^*_{N-4}(g) &= 2, \quad u^*_{N-4}(g) = \text{up}, \\
J^*_{N-4}(d) &= 3 + J^*_{N-3}(e) = 10, \quad u^*_{N-4}(d) = \text{right}, \\
J^*_{N-4}(f) &= 5,  \quad u^*_{N-4}(f) = \text{right},\\
J^*_{N-4}(a) &= 8 + J^*_{N-3}(d) = 19, \quad u^*_{N-4}(a) = \text{right} \\
J^*_{N-4}(c) &= \min \{5 + J^*_{N-3}(d), \:3 + J^*_{N-3}(f)\} = 8, \quad u^*_{N-4}(c) = \text{right}, \\
J^*_{N-4}(b) &= 9 + J^*_{N-3}(c) = 17, \quad u^*_{N-4}(b) = \text{right},\\
\end{split}
\end{equation*}
and finally with one more iteration:
\begin{equation*}
\begin{split}
J^*_{N-5}(e) & = 7, \quad u^*_{N-5}(e) = \text{down}, \\
J^*_{N-5}(g) &= 2, \quad u^*_{N-5}(g) = \text{up}, \\
J^*_{N-5}(d) &= 10, \quad u^*_{N-5}(d) = \text{right}, \\
J^*_{N-5}(f) &= 5, \quad u^*_{N-5}(f) = \text{right}, \\
J^*_{N-5}(a) &= \min \{8 + J^*_{N-4}(d), \:5 + J^*_{N-4}(b)\} = 18, \quad u^*_{N-5}(a) = \text{right} \\
J^*_{N-5}(c) &= \min \{5 + J^*_{N-4}(d), \:3 + J^*_{N-4}(f)\} = 8, \quad u^*_{N-5}(c) = \text{right}, \\
J^*_{N-5}(b) &= 9 + J^*_{N-4}(c) = 17, \quad u^*_{N-5}(b) = \text{right}. \\
\end{split}
\end{equation*}
Additional iterations are not included in this example because the costs and optimal decisions will no longer change with longer horizons (see for yourself!). Therefore it can be seen that with a sufficiently long horizon ($N \geq 5$), the optimal path from $a$ to $h$ is $a-d-e-f-g-h$ and incurs a cost of $18$. Not this process has actually given a lot more information than what was original asked for. In particular, given \textit{any} starting point and \textit{any} horizon it is straightforward to generate an optimal control sequence! For example, if you wanted to start at point $c$ and get to $h$ in $N=3$ steps you could immediately see that the optimal path is $c-f-g-h$ and the optimal cost is $8$.
\end{example}

\subsection{Stochastic Decision Making Problem}
In the stochastic decision making problem it is assumed that there is some \textit{uncertainty} in the robot's behavior or in the environment. This uncertainty is captured in the stochastic discrete-time robot model:
\begin{equation} \label{eq:smodel}
\x_{k+1}= f_k(\x_k, \bu_k, \w_k), \quad k = 0, \dots, N-1,
\end{equation}
where $\w_k$ represents a stochastic disturbance term. Additionally, it is assumed that this disturbance has a known conditional probability distribution $P_k(\w_k \mid \x_k, \bu_k)$. Note that it is assumed that the disturbance is only dependent on the current state $\x_k$ and control $\bu_k$, and not states from earlier in the robot's history.
This is another example of the Markov assumption, which was similarly used to develop the algorithms for localization and filtering in previous chapters.

Another main difference between the stochastic decision making problem and the deterministic problem is that a control \textit{policy} is computed in the stochastic case. A control policy, usually denoted $\bu = \pi(\x)$, is a function that maps the state $\x$ to a control $\bu$, and therefore defines a closed-loop controller (whereas in the deterministic setting an open-loop sequence was computed). Generally speaking, the search for control \textit{policies} makes the problem more difficult to solve, but is typically required in stochastic settings because uncertainty would lead to undesirable behavior under open-loop control plans. Specifically, in the stochastic decision making problem the policies $\pi = \{\pi_0, \dots, \pi_{N-1}\}$ are computed, which define the controls by $\bu_k = \pi_k(\x_k)$.

Of course the cost function is also modified to handle the uncertainty. In particular, a \textit{risk-neutral} formulation is used (i.e. minimize the cost \textit{on average}), where the cost is defined by the \textit{expected} value:
\begin{equation} \label{eq:scost}
J_\pi(\x_0) = E_{\w} \big[g_N(\x_N) + \sum_{k=0}^{N-1} g_k(\x_k, \pi(\x_k), \w_k)\big],
\end{equation}
where the expectation is over the stochastic variables $\w$. The stochastic decision making problem can now be stated as:
\begin{definition}[Stochastic Decision Making Problem]
The stochastic decision making problem can be expressed for the system model \eqref{eq:smodel}, control constraints \eqref{eq:SDMPconstraints}, and cost function \eqref{eq:scost} as:
\begin{equation} \label{eq:sproblem}
J^*(\x_0) = \underset{\pi}{\min} \:\:J_{\pi}(\x_0).
\end{equation}
\end{definition}

\subsubsection{Principle of Optimality (Stochastic)}
The principle of optimality can again be applied in the stochastic setting, and the intuition is identical to the deterministic case (however the proof is slightly different because the reasoning is in terms of probability distributions). The principle of optimality in the stochastic setting is stated formally as:
\begin{theorem}[Principle of Optimality (Stochastic)]
Let $\pi^* = \{\pi_0^*, \pi_1^* \dots, \pi_{N-1}^*\}$ be an optimal policy for the stochastic decision making problem \eqref{eq:sproblem}, and assume the state $\x_k$ is reachable. Then, the tail policy sequence $\{\pi_k^*, \dots, \pi_{N-1}^*\}$ is an optimal policy sequence when starting from $\x_k$ to minimize the cost from time $k$ to time $N$.
\end{theorem}
Again, by leveraging the principle of optimality the decision making problem can be simplified to making immediate decisions by concatenating optimal tail policies.

\subsubsection{Dynamic Programming (Stochastic)}
The dynamic programming algorithm for the stochastic setting is also quite similar to DP for deterministic problems, and is given in Algorithm \ref{alg:sDP}.
\begin{algorithm}[ht]
 $J_N(\x_N) = g_n(\x_N),$ for all $\x_N$\\
 \For{$k=N-1$ \KwTo $0$}{
  $J_k(\x_k) = \underset{\bu_k \in \mathcal{U}(\x_k)}{\min}E_{\w_k}\big[g_k(\x_k,\bu_k,\w_k) + J_{k+1}(f_k(\x_k, \bu_k, \w_k))\big],$ for all $\x_k$\\
 }
 \Return $J_0(\cdot),\dots,J_N(\cdot)$
 \caption{Dynamic Programming (Stochastic)}
 \label{alg:sDP}
\end{algorithm}
Once Algorithm \ref{alg:sDP} is run, the optimal policy is defined by:
\begin{equation*}
\pi^*_k(\x_k) = \underset{\bu_k \in \mathcal{U}(\x_k)}{\arg \min}E_{\w_k}\big[g_k(\x_k,\bu_k,\w_k) + J_{k+1}(f_k(\x_k, \bu_k, \w_k))\big].
\end{equation*}

\begin{example}[Stochastic Dynamic Programming] \label{ex:stoDP}
\theoremstyle{definition}
Consider an inventory control problem, where the available stock of a particular item is the state $x_k \in \mathbb{N}$, the ability to add to the inventory is the control $u_k \in \mathbb{N}$, and the demand for the item is a stochastic variable $w_k \in \mathbb{N}$. The dynamics of the available stock is modeled as:
\begin{equation*}
x_{k+1} = \max \{0, x_k + u_k - w_k\},
\end{equation*}
which models the fact that demand reduces available stock but can also never be negative. Additionally, consider the control constraints:
\begin{equation*}
x_k + u_k \leq 2,
\end{equation*}
which limits the amount of additional inventory that can be added based on the current available stock to ensure that $x_k \leq 2$.
The demand $w_k$ is assumed to be modeled probabilistically with a distribution:
\begin{equation*}
p(w_k=0) = 0.1, \quad p(w_k = 1) = 0.7, \quad p(w_k = 2) = 0.2.
\end{equation*}
Finally, the cost is given for a horizon of $N=3$ as:
\begin{equation*}
E\big[\sum_{k=0}^2 u_k + (x_k + u_k - w_k)^2 \big],
\end{equation*}
which penalizes ordering new stock at each time step and also having available stock at the next time step (i.e. having to store stock).

The dynamic programming algorithm can then be applied, starting with the end costs:
\begin{equation*}
J_3(x_3) = 0,
\end{equation*}
and then recursively computing:
\begin{equation*}
\begin{split}
J_2(0) &= \underset{u_2 \in \{0,1,2\}}{\min} E\big[u_2 + (u_2 - w_2)^2  \big] = \underset{u_2 \in \{0,1,2\}}{\min} u_2 + 0.1u_2^2 + 0.7(u_2-1)^2 + 0.2(u_2 - 2)^2 = 1.3, \\
J_2(1) &= \underset{u_2 \in \{0,1\}}{\min} E\big[u_2 + (1 + u_2 - w_2)^2  \big] = 0.3,\\
J_2(2) &= E\big[(2 - w_2)^2  \big] = 1.1, \\
\end{split}
\end{equation*}
where the last cost is easily evaluated since the constraint makes $u_2 = 0$ the only feasible choice.
The optimal stage policies associated with this step are:
\begin{equation*}
\begin{split}
\pi^*_2(0) &= 1, \\
\pi^*_2(1) &= 0, \\
\pi^*_2(2) &= 0. \\
\end{split}
\end{equation*}
In the next step:
\begin{equation*}
\begin{split}
J_1(0) &= \underset{u_1 \in \{0,1,2\}}{\min} E\big[u_1 + (u_1 - w_1)^2 + J_2(\max\{0, u_1 - w_1\}) \big] = 2.5, \\
J_1(1) &= \underset{u_1 \in \{0,1,\}}{\min} E\big[u_1 + (1 + u_1 - w_1)^2 + J_2(\max\{0, 1 + u_1 - w_1\})  \big] = 1.5,\\
J_1(2) &= E\big[(2 - w_1)^2  + J_2(\max\{0, 2 - w_1\}) \big] = 1.68,\\
\end{split}
\end{equation*}
with optimal stage policies:
\begin{equation*}
\begin{split}
\pi^*_1(0) &= 1, \\
\pi^*_1(1) &= 0, \\
\pi^*_1(2) &= 0. \\
\end{split}
\end{equation*}
Finally, in the last step:
\begin{equation*}
\begin{split}
J_0(0) &= \underset{u_0 \in \{0,1,2\}}{\min} E\big[u_0 + (u_0 - w_0)^2 + J_1(\max\{0, u_0 - w_0\}) \big] = 3.7, \\
J_0(1) &= \underset{u_0 \in \{0,1,\}}{\min} E\big[u_0 + (1 + u_0 - w_0)^2 + J_1(\max\{0, 1 + u_0 - w_0\})  \big] = 2.7,\\
J_0(2) &= E\big[(2 - w_0)^2  + J_1(\max\{0, 2 - w_0\}) \big] = 2.818,\\
\end{split}
\end{equation*}
with optimal stage policies:
\begin{equation*}
\begin{split}
\pi^*_0(0) &= 1, \\
\pi^*_0(1) &= 0, \\
\pi^*_0(2) &= 0. \\
\end{split}
\end{equation*}
Interestingly, the best scenario occurs with an initial stock of one, rather than have no stock or too much stock. Also, the policy ends up being the same at all time steps: if you have no stock you add one item, otherwise you do nothing.
\end{example}

\subsection{Challenges and Extensions of Dynamic Programming}
Dynamic programming is a powerful algorithm, but suffers from several practical considerations: the ``curse of dimensionality'', the ``curse of modeling'', and the ``curse of time''. The curse of dimensionality arises because of a exponential growth of the computational and storage requirements based on the dimension of the state. For example if the state has dimension one (i.e. $\x \in \R$) and can take on 100 different values, then at each step of the algorithm the Bellman equation must be solved 100 times. While this may be possible from a practical perspective, if $\x \in \R^3$ this would lead to $100^3$ solves of the Bellman equation! Additionally, extensions to the problems presented in this chapter where the full state is not \textit{known} (e.g. because you can only measure some parts of the state), the problem also become intractable. The curse of modeling results from the complexity of modeling stochastic systems. In particular, it can be very hard to obtain expressions for transition probabilities for real world systems! Lastly, the curse of time is that the data of the problem may not be known ahead of time (such that the DP algorithm can be run offline). Therefore it may be required to solve the DP algorithm online when the data becomes available, or when the data changes and the problem needs to be resolved.

\subsubsection{Reinforcement Learning}
The practical challenges related to dynamic programming motivated the development of \textit{suboptimal} dynamic programming approaches, which more commonly are referred to as \textit{reinforcement learning} approaches. The goal of these approaches is to make \textit{approximations} to the original problem that make it more practical for specific settings, such as with high-dimensional states, when the model is not known, and more. Broadly speaking, there are two main categories of approximations. The first category includes approximations in the value space (i.e. where the optimal cost function is approximated). The second category includes approximations in the policy space (i.e. where the policy is approximated by a neural network whose weights are optimized over).


\chapter{Characters}
Characters are basically the best thing ever.
To every representation $V$ of $A$ we will attach a
so-called character $\chi_V : A \to k$.
It will turn out that the characters of irreps of $V$
will determine the representation $V$ completely.
Thus an irrep is just specified by a set of $\dim A$ numbers.

\section{Definitions}
\begin{definition}
	Let $V = (V, \rho)$ be a finite-dimensional representation of $A$.
	The \vocab{character} $\chi_V : A \to k$ attached to
	$A$ is defined $\chi_V = \Tr \circ \rho$, i.e.\
	\[ \chi_V(a) \defeq \Tr\left( \rho(a) : V \to V \right). \]
\end{definition}
Since $\Tr$ and $\rho$ are additive, this is a $k$-linear map
(but it is not multiplicative).
Note also that $\chi_{V \oplus W} = \chi_V + \chi_W$
for any representations $V$ and $W$.

We are especially interested in the case $A = k[G]$, of course.
As usual, we just have to specify $\chi_V(g)$ for each
$g \in S_3$ to get the whole map $k[G] \to k$.
Thus we often think of $\chi_V$ as a function $G \to k$,
called a character of the group $G$.
Here is the case $G = S_3$:
\begin{example}
	[Character table of $S_3$]
	Let's consider the three irreps of $G = S_3$ from before.
	For $\CC_{\text{triv}}$ all traces are $1$;
	for $\CC_{\text{sign}}$ the traces are $\pm 1$ depending on sign
	(obviously, for one-dimensional maps $k \to k$ the trace ``is''
	just the map itself).
	For $\refl_0$ we take a basis $(1,0,-1)$ and $(0,1,-1)$, say,
	and compute the traces directly in this basis.
	\[
		\begin{array}{|r|rrrrrr|}
			\hline
			\chi_V(g) & \id & (1\;2) & (2\;3) & (3\;1)
				& (1\;2\;3) & (3\;2\;1)  \\ \hline
			\Ctriv & 1 & 1 & 1 & 1 & 1 & 1 \\
			\CC_{\mathrm{sign}} & 1 & -1 & -1 & -1 & 1 & 1 \\
			\refl_0 & 2 & 0 & 0 & 0 & -1 & -1 \\ \hline
		\end{array}
	\]
\end{example}
The above table is called the \vocab{character table} of the group $G$.
The table above has certain mysterious properties,
which we will prove as the chapter progresses.
\begin{enumerate}[(I)]
	\ii The value of $\chi_V(g)$ only depends on the conjugacy class of $g$.
	\ii The number of rows equals the number of conjugacy classes.
	\ii The sum of the squares of any row is $6$ again!
	\ii The ``dot product'' of any two rows is zero.
\end{enumerate}

\begin{abuse}
	The name ``character'' for $\chi_V : G \to k$ is a bit of a misnomer.
	This $\chi_V$ is not multiplicative in any way,
	as the above example shows: one can almost think of it as
	an element of $k^{\oplus |G|}$.
\end{abuse}

\begin{ques}
	Show that $\chi_V(1_A) = \dim V$,
	so one can read the dimensions of the representations
	from the leftmost column of a character table.
\end{ques}

\section{The dual space modulo the commutator}
For any algebra, we first observe that since $\Tr(TS) = \Tr(ST)$,
we have for any $V$ that
\[ \chi_V(ab) = \chi_V(ba). \]
This explains observation (I) from earlier:
\begin{ques}
	Deduce that if $g$ and $h$ are in the same conjugacy class of a
	group $G$, and $V$ is a representation of $\CC[G]$,
	then $\chi(g) = \chi(h)$.
\end{ques}
Now, given our algebra $A$ we define the \vocab{commutator} $[A,A]$
to be the (two-sided) ideal\footnote{%
	This means the ideal consists of sums elements of the form
	$a(xy-yx)b$ for $a,b \in A$.
}
generated by elements of the form $xy-yx$.
Thus $[A,A]$ is contained in the kernel of each $\chi_V$.
\begin{definition}
	The space $A / [A,A]$ is called the \vocab{abelianization} of $A$;
	for brevity we denote it as $A\ab$.
	We think of this as ``$A$ modulo the relation $ab=ba$ for each $a,b \in A$.''
\end{definition}
So we can think of each character $\chi_V$ as an element of $(A\ab)^\vee$.

\begin{example}
	[Examples of abelianizations]
	\listhack
	\begin{enumerate}[(a)]
		\ii If $A$ is commutative, then $[A,A] = \{0\}$
		and $A\ab = A$.
		\ii If $A = \Mat_k(d)$, then $[A,A]$ consists exactly
		of the $d \times d$ matrices of trace zero.
		(Proof: harmless exercise.)
		Consequently, $A\ab$ is one-dimensional.
		\ii Suppose $A = k[G]$.  We claim that $\dim A\ab$ is equal to the
		number of conjugacy classes of $A$.
		Indeed, an element of $A$ can be thought of as just
		an arbitrary function $\xi : G \to k$.
		So an element of $A\ab$ is a function $\xi: G \to k$ such that
		$\xi(gh) = \xi(hg)$ for every $g,h \in G$.
		This is equivalent to functions from conjugacy classes of $G$ to $k$.
	\end{enumerate}
\end{example}

\begin{theorem}
	[Character of representations of algebras]
	Let $A$ be an algebra over an algebraically closed field. Then
	\begin{enumerate}[(a)]
		\ii Characters of pairwise non-isomorphic irreps are
		linearly independent as elements of $A\ab$.
		\ii If $A$ is finite-dimensional and semisimple,
		then the characters attached to irreps
		form a basis of $A\ab$.
	\end{enumerate}
	In particular, in (b) the number of irreps of $A$ equals $\dim A\ab$.
\end{theorem}
\begin{proof}
	Part (a) is more or less obvious by the density theorem.
	Suppose there is a linear dependence, so that for every $a$ we have
	\[ c_1 \chi_{V_1}(a) + c_2 \chi_{V_2}(a) + \dots + c_r \chi_{V_r} (a) = 0\]
	for some integer $r$.
	\begin{ques}
		Deduce that $c_1 = \dots = c_r = 0$ from the density theorem.
	\end{ques}
	For part (b), assume there are $r$ irreps
	we may assume that \[ A = \bigoplus_{i=1}^r \Mat(V_i) \]
	where $V_1$, \dots, $V_r$ are the irreps of $A$.
	Since we have already showed the characters are linearly independent
	we need only show that $\dim ( A / [A,A] ) = r$,
	which follows from the observation earlier that each $\Mat(V_i)$
	has a one-dimensional abelianization.
\end{proof}
Since $G$ has $\dim \CC[G]\ab$ conjugacy classes,
this completes the proof of (II).

\section{Orthogonality of characters}
Now we specialize to the case of finite groups $G$, represented over $\CC$.
\begin{definition}
	Let $\Classes(G)$ denote the set conjugacy classes of $G$.
\end{definition}
If $G$ has $r$ conjugacy classes, then it has $r$ irreps.
Each (finite-dimensional) representation $V$, irreducible or not, gives a
character $\chi_V$.
\begin{abuse}
	From now on, we will often regard $\chi_V$ as a function $G \to \CC$
	or as a function $\Classes(G) \to \CC$.
	So for example, we will write both $\chi_V(g)$ (for $g \in G$)
	and $\chi_V(C)$ (for a conjugacy class $C$);
	the latter just means $\chi_V(g_C)$ for any representative $g_C \in C$.
\end{abuse}
\begin{definition}
	Let $\FunCl(G)$ denote the set of functions $\Classes(G) \to \CC$
	viewed as a vector space over $\CC$.
	We endow it with the inner form
	\[
		\left< f_1, f_2 \right> =
		\frac{1}{|G|}
		\sum_{g \in G} f_1(g) \ol{f_2(g)}.
	\]
\end{definition}
This is the same ``dot product'' that we mentioned at the beginning,
when we looked at the character table of $S_3$.
We now aim to prove the following orthogonality theorem,
which will imply (III) and (IV) from earlier.
\begin{theorem}[Orthogonality]
	For any finite-dimensional complex representations $V$ and $W$
	of $G$ we have
	\[ \left< \chi_V, \chi_W \right> = \dim \Homrep(W, V). \]
	In particular, if $V$ and $W$ are irreps then
	\[ \left< \chi_V, \chi_W \right>
		=
		\begin{cases}
			1 & V  \cong W \\
			0 & \text{otherwise}.
		\end{cases}
	\]
\end{theorem}
\begin{corollary}[Irreps give an orthonormal basis]
	The characters associated to irreps
	form an \emph{orthonormal} basis of $\FunCl(G)$.
\end{corollary}

In order to prove this theorem, we have to define
the dual representation and the tensor representation,
which give a natural way to deal with the quantity $\chi_V(g)\ol{\chi_W(g)}$.
\begin{definition}
	Let $V = (V, \rho)$ be a representation of $G$.
	The \vocab{dual representation} $V^\vee$ is the representation on $V^\vee$
	with the action of $G$ given as follows: for each $\xi \in V^\vee$,
	the action of $g$ gives a $g \cdot \xi \in V^\vee$ specified by
	\[ v \xmapsto{g \cdot \xi} \xi\left( \rho(g\inv)(v) \right). \]
\end{definition}
\begin{definition}
	Let $V = (V, \rho_V)$ and $W = (W, \rho_W)$
	be \emph{group} representations of $G$.
	The \vocab{tensor product} of $V$ and $W$ is the group representation
	on $V \otimes W$ with the action of $G$ given on pure tensors by
	\[
		g \cdot (v \otimes w)
		=
		(\rho_V(g)(v)) \otimes (\rho_W(g)(w)) \]
	which extends linearly to define the action of $G$ on all of $V \otimes W$.
\end{definition}
\begin{remark}
	Warning: the definition for tensors does \emph{not} extend to algebras.
	We might hope that $a \cdot (v \otimes w) = (a \cdot v) \otimes (a \cdot w)$
	would work, but this is not even linear in $a \in A$
	(what happens if we take $a=2$, for example?).
\end{remark}

\begin{theorem}
	[Character traces]
	If $V$ and $W$ are finite-dimensional representations of $G$,
	then for any $g \in G$.
	\begin{enumerate}[(a)]
		\ii $\chi_{V \oplus W}(g) = \chi_V(g) + \chi_W(g)$.
		\ii $\chi_{V \otimes W}(g) = \chi_V(g) \cdot \chi_W(g)$.
		\ii $\chi_{V^\vee}(g) = \ol{\chi_V(g)}$.
	\end{enumerate}
\end{theorem}
\begin{proof}
	Parts (a) and (b) follow from the identities
	$\Tr(S \oplus T) = \Tr(S) + \Tr(T)$
	and $\Tr(S \otimes T) = \Tr(S) \Tr(T)$.
	However, part (c) is trickier.
	As $(\rho(g))^{|G|} = \rho(g^{|G|}) = \rho(1_G) = \id_V$
	by Lagrange's theorem, we can diagonalize $\rho(g)$,
	say with eigenvalues $\lambda_1$, \dots, $\lambda_n$
	which are $|G|$th roots of unity,
	corresponding to eigenvectors $e_1$, \dots, $e_n$.
	Then we see that in the basis $e_1^\vee$, \dots, $e_n^\vee$,
	the action of $g$ on $V^\vee$ has eigenvalues
	$\lambda_1\inv$, $\lambda_2\inv$, \dots, $\lambda_n\inv$.
	So
	\[
		\chi_V(g) = \sum_{i=1}^n \lambda_i \quad\text{and}\quad
		\chi_{V^\vee}(g) = \sum_{i=1}^n \lambda_i\inv = \sum_{i=1}^n \ol{\lambda_i}
	\]
	where the last step follows from the identity $|z|=1 \iff z\inv = \ol z$.
\end{proof}
\begin{remark}
	[Warning]
	The identities (b) and (c) do not extend linearly to $\CC[G]$,
	i.e.\ it is not true for example that $\chi_V(a) = \ol{\chi_V(a)}$
	if we think of $\chi_V$ as a map $\CC[G] \to \CC$.
\end{remark}
\begin{proof}
	[Proof of orthogonality relation]
	The key point is that we can now reduce
	the sums of products to just a single character by
	\[ \chi_V(g) \ol{\chi_W(g)} = \chi_{V \otimes W^\vee} (g). \]
	So we can rewrite the sum in question as just
	\[
		\left< \chi_V, \chi_W \right>
		= \frac{1}{|G|} \sum_{g \in G} \chi_{V \otimes W^\vee} (g)
		= \chi_{V \otimes W^\vee}
		\left( \frac{1}{|G|} \sum_{g \in G} g \right).
	\]
	Let $P : V \otimes W^\vee \to V \otimes W^\vee$ be the
	action of $\frac{1}{|G|} \sum_{g \in G} g$,
	so we wish to find $\Tr P$.
	\begin{exercise}
		Show that $P$ is idempotent.
		(Compute $P \circ P$ directly.)
	\end{exercise}
	Hence $V \otimes W^\vee = \ker P \oplus \img P$ (by \Cref{prob:idempotent})
	and $\img P$ is the subspace of elements which are fixed under $G$.
	From this we deduce that
	\[ \Tr P = \dim \img P =
		\dim \left\{ x \in V \otimes W^\vee
		\mid g \cdot x = x \; \forall g \in G  \right\}.
		\]
	Now, consider the natural isomorphism $V \otimes W^\vee \to \Hom(W, V)$.
	\begin{exercise}
		Let $g \in G$.
		Show that under this isomorphism, $T \in \Hom(W, V)$
		satisfies $g \cdot T = T$ if and only if
		$T(g \cdot w) = g \cdot T(w)$ for each $w \in W$.
		(This is just unwinding three or four definitions.)
	\end{exercise}
	Consequently, $\chi_{V \otimes W^\vee}(P) = \Tr P = \dim \Homrep(W,V)$
	as desired.
\end{proof}

The orthogonality relation gives us a fast and mechanical way to check
whether a finite-dimensional representation $V$ is irreducible.
Namely, compute the traces $\chi_V(g)$ for each $g \in G$,
and then check whether $\left< \chi_V, \chi_V \right> = 1$.
So, for example, we could have seen the three representations of
$S_3$ that we found were irreps directly from the character table.
Thus, we can now efficiently verify any time we have
a complete set of irreps.

\section{Examples of character tables}
\begin{example}
	[Dihedral group on $10$ elements]
	Let $D_{10} = \left< r,s \mid r^5 = s^2 = 1, rs = sr\inv \right>$.
	Let $\omega = \exp(\frac{2\pi i}{5})$.
	We write four representations of $D_{10}$:
	\begin{itemize}
		\ii $\Ctriv$, all elements of $D_{10}$ act as the identity.
		\ii $\Csign$, $r$ acts as the identity while $s$ acts by negation.
		\ii $V_1$, which is two-dimensional and given by
		$r \mapsto \begin{bmatrix} \omega & 0 \\ 0 & \omega^4 \end{bmatrix}$
		and $s \mapsto \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$.
		\ii $V_2$, which is two-dimensional and given by
		$r \mapsto \begin{bmatrix} \omega^2 & 0 \\ 0 & \omega^3 \end{bmatrix}$
		and $s \mapsto \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$.
	\end{itemize}
	We claim that these four representations are irreducible
	and pairwise non-isomorphic.
	We do so by writing the character table:
	\[
		\begin{array}{|c|rccr|}
			\hline
			D_{10} & 1 & r, r^4 & r^2, r^3 & sr^k \\ \hline
			\Ctriv & 1 & 1 & 1 & 1 \\
			\Csign & 1 & 1 & 1 & -1 \\
			V_1 & 2 & \omega+\omega^4 & \omega^2+\omega^3 & 0 \\
			V_2 & 2 & \omega^2+\omega^3 & \omega+\omega^4 & 0 \\ \hline
		\end{array}
	\]
	Then a direct computation shows the orthogonality relations,
	hence we indeed have an orthonormal basis.
	For example, $\left< \Ctriv, \Csign \right> = 1 + 2 \cdot 1 + 2 \cdot 1 + 5 \cdot (-1) = 0$.
\end{example}

\begin{example}
	[Character table of $S_4$]
	We now have enough machinery to to compute the character
	table of $S_4$, which has five conjugacy classes
	(corresponding to cycle types $\id$, $2$, $3$, $4$ and $2+2$).
	First of all, we note that it has two one-dimensional representations,
	$\Ctriv$ and $\Csign$, and these are the only ones
	(because there are only two homomorphisms $S_4 \to \CC^\times$).
	So thus far we have the table
	\[
		\begin{array}{|c|rrrrr|}
			\hline
			S_4 & 1 & (\bullet\;\bullet) & (\bullet\;\bullet\;\bullet)
				& (\bullet\;\bullet\;\bullet\;\bullet)
				& (\bullet\;\bullet)(\bullet\;\bullet)
				\\ \hline
			\Ctriv & 1 & 1 & 1 & 1 & 1 \\
			\Csign & 1 & -1 & 1 & -1 & 1 \\
			\vdots & \multicolumn{5}{|c|}{\vdots}
		\end{array}
	\]
	Note the columns represent $1+6+8+6+3=24$ elements.

	Now, the remaining three representations have dimensions
	$d_1$, $d_2$, $d_3$ with
	\[ d_1^2 + d_2^2 + d_3^2 = 4! - 2 = 22 \]
	which has only $(d_1, d_2, d_3) = (2,3,3)$ and permutations.
	Now, we can take the $\refl_0$ representation
	\[ \left\{ (w,x,y,z) \mid w+x+y+z=0 \right\} \]
	with basis $(1,0,0,-1)$, $(0,1,0,-1)$ and $(0,0,1,-1)$.
	This can be geometrically checked to be irreducible,
	but we can also do this numerically by computing the
	character directly (this is tedious):
	it comes out to have $3$, $1$, $0$, $-1$, $-1$
	which indeed gives norm
	\[
		\left< \chi_{\refl_0}, \chi_{\refl_0} \right>
		=
		\frac{1}{4!}
		\left(
			\underbrace{3^2}_{\id}
			+ \underbrace{6\cdot(1)^2}_{(\bullet\;\bullet)}
			+ \underbrace{8\cdot(0)^2}_{(\bullet\;\bullet\;\bullet)}
			+ \underbrace{6\cdot(-1)^2}_{(\bullet\;\bullet\;\bullet\;\bullet)}
			+ \underbrace{3\cdot(-1)^2}_{(\bullet\;\bullet)(\bullet\;\bullet)}
		\right)
		= 1.
	\]
	Note that we can also tensor this with the sign representation,
	to get another irreducible representation
	(since $\Csign$ has all traces $\pm 1$, the norm doesn't change).
	Finally, we recover the final row using orthogonality
	(which we name $\CC^2$, for lack of a better name);
	hence the completed table is as follows.
	\[
		\begin{array}{|c|rrrrr|}
			\hline
			S_4 & 1 & (\bullet\;\bullet) & (\bullet\;\bullet\;\bullet)
				& (\bullet\;\bullet\;\bullet\;\bullet)
				& (\bullet\;\bullet)(\bullet\;\bullet)
				\\ \hline
			\Ctriv & 1 & 1 & 1 & 1 & 1 \\
			\Csign & 1 & -1 & 1 & -1 & 1 \\
			\CC^2 & 2 & 0 & -1 & 0 & 2 \\
			\refl_0 & 3 & 1 & 0 & -1 & -1 \\
			\refl_0 \otimes \Csign & 3 & -1 & 0 & 1 & -1 \\\hline
		\end{array}
	\]
\end{example}

\section\problemhead

\begin{dproblem}
	[Reading decompositions from characters]
	Let $W$ be a complex representation of a finite group $G$.
	Let $V_1$, \dots, $V_r$ be the complex irreps of $G$
	and set $n_i = \left< \chi_W, \chi_{V_i} \right>$.
	Prove that each $n_i$ is a non-negative integer and
	\[ W = \bigoplus_{i=1}^r V_i^{\oplus n_i}. \]
	\begin{hint}
		Obvious.
		Let $W = \bigoplus V_i^{m_i}$ (possible since $\CC[G]$ semisimple)
		thus $\chi_W = \sum_i m_i \chi_{V_i}$.
	\end{hint}
\end{dproblem}

\begin{problem}
	Consider complex representations of $G = S_4$.
	The representation $\refl_0 \otimes \refl_0$
	is $9$-dimensional, so it is clearly reducible.
	Compute its decomposition in terms of the five
	irreducible representations.
	\begin{hint}
		Use the previous problem, with $\chi_W = \chi_{\refl_0}^2$.
	\end{hint}
	\begin{sol}
		$\Csign \oplus \CC^2 \oplus \refl_0 \oplus (\refl_0\otimes\Csign)$.
	\end{sol}
\end{problem}

\begin{problem}
	[Tensoring by one-dimensional irreps]
	Let $V$ and $W$ be irreps of $G$, with $\dim W = 1$.
	Show that $V \otimes W$ is irreducible.
	\begin{hint}
		Characters. Note that $|\chi_W| = 1$ everywhere.
	\end{hint}
	\begin{sol}
		First, observe that $|\chi_W(g)|=1$ for all $g \in G$.
		\begin{align*}
			\left< \chi_{V \otimes W}, \chi_{V \otimes W} \right>
			&= \left< \chi_V \chi_W, \chi_V \chi_W \right> \\
			&= \frac{1}{|G|} \sum_{g \in G}
			\left\lvert \chi_V(g) \right\rvert^2
			\left\lvert \chi_W(g) \right\rvert^2 \\
			&= \frac{1}{|G|} \sum_{g \in G}
			\left\lvert \chi_V(g) \right\rvert^2 \\
			&= \left< \chi_V, \chi_V \right> = 1.
		\end{align*}
	\end{sol}
\end{problem}

\begin{problem}
	[Quaternions]
	Compute the character table of the quaternion group $Q_8$.
	\begin{hint}
		There are five conjugacy classes, $1$, $-1$
		and $\pm i$, $\pm j$, $\pm k$.
		Given four of the representations, orthogonality
		can give you the fifth one.
	\end{hint}
	\begin{sol}
		The table is given by
		\[
			\begin{array}{|c|rrrrr|}
				\hline
				Q_8 & 1 & -1 & \pm i & \pm j & \pm k \\ \hline
				\Ctriv & 1 & 1 & 1 & 1 & 1 \\
				\CC_i & 1 & 1 & 1 & -1 & -1 \\
				\CC_j & 1 & 1 & -1 & 1 & -1 \\
				\CC_k & 1 & 1 & -1 & -1 & 1 \\
				\CC^2 & 2 & -2 & 0 & 0 & 0 \\\hline
			\end{array}
		\]
		The one-dimensional representations (first four rows)
		follows by considering the homomorphism $Q_8 \to \CC^\times$.
		The last row is two-dimensional and can be recovered
		by using the orthogonality formula.
	\end{sol}
\end{problem}

\begin{sproblem}
	[Second orthogonality formula]
	\label{prob:second_orthog}
	\gim
	Let $g$ and $h$ be elements of a finite group $G$,
	and let $V_1$, \dots, $V_r$ be the irreps of $G$.
	Prove that
	\[
		\sum_{i = 1}^r \chi_{V_i}(g) \ol{\chi_{V_i}(h)}
		=
		\begin{cases}
			|C_G(g)| & \text{if $g$ and $h$ are conjugates} \\
			0 & \text{otherwise}.
		\end{cases}
	\]
	Here, $C_G(g) = \left\{ x \in G : xg = gx \right\}$
	is the centralizer of $g$.
	\begin{hint}
		Write as
		\[ \sum_{i=1}^r \chi_{V_i \otimes V_i^\vee} (gh\inv)
			= \chi_{\bigoplus_i V_i \otimes V_i^\vee}(gh\inv)
			= \chi_{\CC[G]}(gh\inv).
		\]
		Now look at the usual basis for $\CC[G]$.
	\end{hint}
\end{sproblem}


\documentclass[10pt]{article}
\usepackage{amsmath}

\title{Some Informal Notes for Radiative Transfer Code}
\author{Mark Baum}

\begin{document}

\small

\maketitle

\section*{Pressure Coordinates}

Get pressure coordinates by using the hydrostatic relation,
\begin{equation}
	\frac{dP}{dz} = -\rho g \, ,
\end{equation}
to replace $dz$ wherever it appears,
\begin{equation}
	dz = -\frac{dP}{\rho g} \, .
\end{equation}
Consider optical depth
\begin{equation}
	\tau	 = \int \sigma N dz \, ,
\end{equation}
where $\sigma$ is the absorption cross-section (m$^2$/molecule) and $N$ is the number density of the absorber (molecules/m$^3$). Switching to pressure coordinates yields
\begin{equation}
	\tau	 = \int \sigma N \frac{1}{\rho g} dP \, .
	\label{tau}
\end{equation}
Then the ideal gas law can be used to substitute for both $N$ and $\rho$.
\begin{align}
	N &= \frac{P}{k_B T} \\[1ex]
	\rho &= \frac{P \mu}{k_B T N_A}
\end{align}
where $\mu$ is the molar mass in kg/mole and $N_A$ is Avogadro's number. Because we have $N/\rho$ above in equation (\ref{tau}) for optical depth, lots of things cancel.
\begin{equation}
	\frac{N}{\rho} = \frac{P}{k_B T} \frac{k_B T N_A}{P \mu} = \frac{N_A}{\mu}
\end{equation}
Putting this into equation (\ref{tau}),
\begin{equation}
	\tau = \int \sigma \frac{N_A}{\mu g} dP \, .
\end{equation}
The negative sign in the original $dz=-dP/\rho g$ has been dropped for convenience. Obviously, the optical depth must be positive, so the coordinate switch probably brings in another negative sign, perhaps when integral limits are switched because low pressure is high altitude and vice versa.

The same steps apply to the closely related Schwarzschild equation,
\begin{equation}
	dI = \sigma N [B_{\nu}(T) - I] dz \, ,
\end{equation}
expressed here in differential form. We have the same $N dz$ factor that, through the hydrostatic relation and ideal gas law, gets replaced by $N_A/\mu g$.
\begin{equation}
	dI = \sigma	\frac{N_A}{\mu g} [B_{\nu}(T) - I] dP \, .
\end{equation}
Important to remember that the molar mass here has units of kg/mole.

\section*{log Pressure Coordinates}

Sometimes there is a case for very high resolution in the upper atmosphere, per unit mass. To get that, log pressure coordinates can be used, which is essentially the same as using altitude directly but lets you keep the equations in terms of $P$. The same steps from above are applied, with one more. Simply use
\begin{equation}
	\frac{1}{P} dP = d\ln P
\end{equation}
or, rearranged
\begin{equation}
	dP = P d\ln P \, .
\end{equation}
Putting this into the optical depth and Schwarzschild equations,
\begin{align}
	\tau	 &= \int \sigma N \frac{1}{\rho g} P d\ln P \\[1em]
	dI &= \sigma	\frac{N_A}{\mu g} [B_{\nu}(T) - I] P d\ln P \, .
\end{align}
It's the same as pressure coordinates, but multiplied by $P$. Then, of course, these equations must be integrated with the appropriate limits, $\ln(P_{\textrm{s}})$ and $\ln(P_{\textrm{toa}})$. Its important to consistently use the natural log, not log$_{10}$.

\section*{Integrating Multiple Streams}

In the absence of scattering, the total flux up or down in the atmosphere must account for irradiance at all angles in the hemisphere pointing up/down.
\begin{equation}
	F = \int_0^{2\pi} \int_0^{\pi/2} I(\theta	,\phi) \cos(\theta) \sin(\theta) d\theta d\phi \, ,
\end{equation}
where $\theta$ is the azimuthal angle (zero is straight up/down), $\phi$ is the ``latitude" angle going around the whole hemisphere. The cosine factor accounts for the angle of $I$ with respect to the horizontal planar surface that the flux is passing through. The sine factor simply comes out of the spherical integration, accounting for area attenuation near the very top of the hemisphere.

Assuming streams are identical with respect to the latitude angle $\phi$, that dimension is trivial to integrate.
\begin{equation}
	F = 2\pi	 \int_0^{\pi/2} I(\theta) \cos(\theta) \sin(\theta) d\theta
\end{equation}
The azimuthal dimension generally must be integrated numerically. $I$ should be quite smooth with respect to $\theta$, so gaussian integration is a good way to do it. The flux is then
\begin{equation}
	F = 2\pi \sum_{i=0}^N I(\theta_i) w_i \cos(\theta_i) \sin(\theta_i)
\end{equation}
where $\theta_i$ are the gaussian quadrature nodes mapped to $[0,\pi/2]$ and $w_i$ are the appropriately scaled weights. For a given number of streams $N$, one can precompute the factors $2\pi w_i \cos(\theta_i) \sin(\theta_i)$.

\end{document}


\section{Jinchao's inputs}

\subsection{Classification versus regression}
Classification is a special form of regression.   Basically, it tries to find the (continuous) boundary of domains (that contain different clusters of data)

\subsection{Overfitting}

It seems to be related to the Runge phenomenon in polynomial interpolation for non-analytic function.  It would be interesting to study, theoretically, whether deep neural network also has Runge phenomenon.

\subsection{CNN versus multigrid}
$$
Au=f
$$

\paragraph{Multigrid}

We know $f$, then try to find $u$.

We have multilevel

We also use average, namely convolution, which is smoother:

\begin{itemize}
\item filter out the high frequencies
\end{itemize}

\paragraph{CNN}
We know $u$, then try to find $f$.  In order to find $f$, we have to find $A$ first.

We use ``finite difference'', namely convolution, which is ``rougher''.
\begin{itemize}
\item filter out the low frequencies
\end{itemize}

\subsection{Role of probability theory and tool}

The role of probability is important for sure, but it is not essential ....

In the idealistic situation, namely all the data are good, no noise etc, we can approach deep learning problem without relying on probability theory and tools.

To understand the essence of deep learning, can we first skip probability tool?

If we skip probability, how much do we miss?   We would not miss very much for the understanding how and why DNN works.

\begin{itemize}
\item High school probability is enough.
\end{itemize}


\documentclass{notes}

  \title{Computability}
  \author{ian.mcloughlin@gmit.ie}
  \date{\today}

\begin{document}

    \section*{Natural numbers}
    \begin{align*}
      \mathbb{N}    &= \{ 1, 2, 3, 4, \ldots \} \\
      \mathbb{N}_0  &= \{ 0, 1, 2, 3, \ldots \} \\
      2\mathbb{N}_0 &= \{ 0, 2, 4, 6, \ldots \} \\
      3\mathbb{N}_0 &= \{ 0, 3, 6, 9, \ldots \} \\
      \mathbb{Z}    &= \{ \ldots, -2, -1, 0, 1, 2, \ldots \} \\
    \end{align*}

    \section*{Real numbers}

    \begin{center}
    \begin{tikzpicture}
        % https://tex.stackexchange.com/a/148253
        \draw[latex-latex] (-1.5,0) -- (3.5,0) ; %edit here for the axis

        \draw[shift={(0,0)},color=black]    (0pt,3pt) -- (0pt,-3pt);
        \draw[shift={(1,0)},color=black]    (0pt,3pt) -- (0pt,-3pt);
        \draw[shift={(1.41,0)},color=black] (0pt,3pt) -- (0pt,-3pt);
        \draw[shift={(3.12,0)},color=black] (0pt,3pt) -- (0pt,-3pt);

        \draw[shift={(0,0)},color=black]    (0pt,-3pt) -- (0pt,0pt) node[above] {$0$};
        \draw[shift={(1,0)},color=black]    (0pt,-3pt) -- (0pt,0pt) node[above] {$1$};
        \draw[shift={(1.41,0)},color=black] (0pt,0pt) -- (0pt,-3pt) node[below] {$\sqrt{2}$};
        \draw[shift={(3.14,0)},color=black] (0pt,0pt) -- (0pt,-3pt) node[below] {$\pi$};
        %\draw[very thick] (,0) -- (1.92,0);
    \end{tikzpicture}
    \end{center}

    \noindent Real numbers are difficult to define:
    \begin{enumerate}
        \item Draw an infinite
        straight line.
        \item Mark two distinct points.
        \item Call the left-most 0, the right-most 1.
        \item One unit is the length between 0 and 1.
        \item Mark any point on the line and measure its length $x$ in units from 0.
    \end{enumerate}
    Then $x$ is a real number.
    The set $\mathbb{R}$ is the set of all such $x$'s.

    \section*{Bijections}

    \[ f: \mathbb{N}_0 \rightarrow 2\mathbb{N}_0 : n \rightarrow 2n  \]

    \begin{align*}
        f(0) &= 0 \\
        f(1) &= 2 \\
        f(2) &= 4 \\
        f(3) &= 6 \\
        \vdots \\
    \end{align*}

    \noindent
    Every element of $\mathbb{N}_0$ is mapped to a different element of $2\mathbb{N}_0$.
    For instance, $1$ is the only number mapped to $2$.

    Also, every element of $2 \mathbb{N}_0$ is mapped to from some element of $\mathbb{N}_0$.
    There's no element of $2 \mathbb{N}_0$ that isn't the image of some element of $\mathbb{N}_0$.

    There are lots of functions like $f$ to and from each of the sets $\mathbb{N}, \mathbb{N}_0, \mathbb{Z}$ and so on, but there is none to $\mathbb{R}$.

    \section*{Diagonilsation}

    Suppose there was a map $f: \mathbb{N}_0 \rightarrow \mathbb{R}$.
    List out each element $n$ of $\mathbb{N}_0$ with $f(n)$ beside it with all of the decimal places displayed.
    For example, suppose the first few are as follows.

    \begin{alignat*}{2}
        0 & \quad \rightarrow \quad &   0&.16346234234234 \ldots \\
        1 & \quad \rightarrow \quad &  10&.56775344747474 \ldots \\
        2 & \quad \rightarrow \quad & 214&.99999999999999 \ldots \\
        3 & \quad \rightarrow \quad &  -1&.33333333333333 \ldots \\
        4 & \quad \rightarrow \quad &   0&.00000000000000 \ldots \\
          & \quad \;\; \vdots    \quad &    &                   \\
    \end{alignat*}

    While the list contains all of the natural numbers $\mathbb{N}_0$, we can show that it can't possibly contain all of the real numbers $\mathbb{R}$.
    We'll use Cantor's diagonal argument to construct a real number not in the list.

    Start by constructing a new number $0.$ and add decimal places as follows.
    Take the digit in the first decimal place of the first real number in the list.
    The first real number is $0.1634\ldots$ so the first decimal place contains a $1$.
    Add $1$ to this number, giving $2$, and use this as the digit in the first decimal place of our new number: $0.2$.

    Next add 1 to the digit in the second decimal place of the second real number.
    That gives 6, so the digit in the second decimal place of our new number is $6$, giving $0.26$ so far.
    Keep going down the list, adding one to each successive decimal place in each successive number.
    Any time you encounter $9$, adding $1$ would give $10$, so replace the $9$ with $0$ instead.
    That happens for the third real number in the list, so our new number becomes $0.260$ at that point.

    The new number differs from the first real number in the list in the first decimal place, the second real number in the second decimal place and so on.
    Therefore, the new number is not in the list --- so the list can't contain all real numbers.
    We have to conclude that the real numbers were not and can not be paired up with the natural numbers.

    \section*{Computable numbers}
    Cantor's argument has profound implications for what a computer can do.
    Each file on a computer is stored in binary -- a string over the alphabet $\{ 0, 1 \}$.
    The possible files are then $\{ 0 , 1 \}^* = \{ \epsilon, 0, 1, 00, 01, 10, 11, 000, \ldots \}$.
    We can pair $\mathbb{N}_0$ up with $\{ 0 , 1 \}^*$ using that listing: $0 \rightarrow \epsilon$, $1 \rightarrow 0$, $2 \rightarrow 1$, $3 \rightarrow 00$, and so on.

    Executables are just files.
    Consider an executable that prints a real number to the screen.
    For instance, we could write a program to print $0.0000\ldots$ to the screen where it just keeps printing $0$'s.
    It's printing the real number $0$.
    For any given real number, is there a program to print it out?
    There must be real numbers for which no such program exists -- because there are less possible programs than real numbers.

    Computers can't even print all numbers out.

  \bibliography{bibliography}
\end{document}


\documentclass{article}

\usepackage{mathrsfs,amsmath}
\usepackage{xcolor}
\usepackage{titlesec}
\usepackage{listings}
\usepackage{syntax}
\usepackage{pythonhighlighting}
\usepackage{fancyvrb}

\usepackage{graphicx}

\graphicspath{ {./assets/} }

\usepackage[margin=1.4in]{geometry}

\title{Homework \#3 | Fall 2021}
\author{Jared Dyreson\\
        California State University, Fullerton}

\DeclareRobustCommand{\bowtie}{%
  \mathrel\triangleright\joinrel\mathrel\triangleleft}

\usepackage [english]{babel}
\usepackage [autostyle, english = american]{csquotes}
\MakeOuterQuote{"}

\titlespacing*{\section}
{0pt}{5.5ex plus 1ex minus .2ex}{4.3ex plus .2ex}
\titlespacing*{\subsection}
{0pt}{5.5ex plus 1ex minus .2ex}{4.3ex plus .2ex}

\usepackage{hyperref}
\hypersetup{
    colorlinks,
    citecolor=black,
    filecolor=black,
    linkcolor=black,
    urlcolor=black
}

\begin{document}

\maketitle
\tableofcontents

\newpage

\section{Questions}

\subsection{Offline Ski Rental}

\subsubsection{Problem Description}

\begin{table}[!h]
\begin{tabular}{|l|l|}
\hline
\textbf{input} & a daily ski rental price $r > 0$, purchase price $p > 0$, and number of days $d > 0$ \\ \hline
\textbf{output} & \begin{tabular}[c]{@{}l@{}}True if it is cheaper to rent skis for d days at r dollars per day, or False if it is\\ cheaper to buy skis for p dollars\end{tabular} \\ \hline
\end{tabular}
\end{table}

\subsubsection{Pseudocode}

\begin{verbatim}
function ski(int rate, int purchasePrice, int days):
    rental_total = rate * days
    return rental_total < purchasePrice
\end{verbatim}

\subsubsection{Time Complexity}

This function is quite simple, as we are given all the information up front.
Each operation we have will complete in a time complexity of $O(1)$.
Therefore, the best/worst case of this is $O(1)$.

\subsubsection{Implementation}

\begin{python}

def solution(rental_price: int,
             purchase_price: int, days: int) -> bool:

    return (rental_price * days) < purchase_price
\end{python}

\newpage

\subsection{List Reversal}

\subsubsection{Problem Description}

\begin{table}[!h]
\begin{tabular}{|l|l|}
\hline
\textbf{input} & a list L of n elements \\ \hline
\textbf{output} & a list containing the elements of L but in reversed order \\ \hline
\end{tabular}
\end{table}

\subsubsection{Pseudocode}

\begin{verbatim}
function reverse(container):
    last_index = container.length # O(1)
    for element in container: # O(n)
        swap(element, container[last_index]) # O(1)
        last_index = last_index - 1 # O(1)
    return container # O(1)
\end{verbatim}

\subsubsection{Time Complexity}

This function will modify the container in place and does not create a copy.
Other variants can leave the original container intact and return a reversed copy.
The benefit of this would be data integrity, however, that is at the expense of having to have duplicate data.
We need to iterate over the entire container of "n" elements and we assume the swap function is of constant time.
Therefore, time complexity of this function is linear, or $O(n)$.
\subsubsection{Implementation}

\begin{python}
def solution(container: list) -> list:
    end = len(container) - 1
    for x, _ in enumerate(container):
        if x < end:
            container[x], container[end] = container[end], container[x]
            end -= 1
    return container
\end{python}

\newpage

\subsection{Pythagorean Triple}

\subsubsection{Problem Description}
\begin{table}[!h]
\begin{tabular}{|l|l|}
\hline
\textbf{input} & two positive integers; $a$ and $b$ with $a < b$ \\ \hline
\textbf{output} & \begin{tabular}[c]{@{}l@{}}a Pythagorean triple (x, y, z), such that $x$, $y$, and $z$ are positive integers,\\ $a \le x \le y \le z \le b$ and $x^{2} + y^{2} = z^{2}$, or None if no such triples exist\end{tabular}\\ \hline
\end{tabular}
\end{table}

\subsubsection{Pseudocode}

\begin{verbatim}
function validator(tuple[int, int, int] triplet):
    x, y, z = triplet # O(3), assignment and unpacking for three elements
    return z == square_root(x ** 2 + y ** 2) and x <= y <= z
    # O(n) <- for the squaring of x and y. This is building off the previous
    # assumption made in Project 2 for the Fibonacci sequence

function pythagorean_triples(int a /* lower bound */, int b /* upper bound */):
    candidates = [] # O(1)
    for x in range(a, b + 1): # get range of values, bounded by a and b # O(n)
        candidates.append(x) # O(1)
    # create all permutations
    # description does not specifically instruct to create an algorithm
    # to create these permutations, therefore we use the best case
    # of O(n^r), where r represents the length of the tuple -> O(n^3)
    permutations = generate_permutations(iterable=candidates, r=3) # O(n^3)
    # this is the function signature of itertools.combinations
    valid_triplets = [] # O(1)

    # this for loop block runs in O(n^2)
    for permuation in permutations: # O(n)
        if(validator(permuation)): # O(n)
            valid_triplets.append(permutation) # O(1)
    return valid_triplets # O(1)
\end{verbatim}

\subsubsection{Time Complexity}

The main portion of this algorithm comes solely from creating all possible combinations the range presents.
Here, we are tasked with creating triplets, given a range of numbers $a$ and $b$.
The creation of all possible combinations will take $O(n^{r})$, where $r$ is the length of the container you're producing.
Therefore in our case it is $O(n^{3})$ and more information regarding this can be found in \href{https://stackoverflow.com/a/20765011}{\underline{this StackOverflow comment thread}}.
All of these triplets need to then be processed by our validator function, which was defined in the problem description.
These validations will occur in $O(n)$ time, because we are assuming raising a number to a given power will take that time complexity.
It is disputed in implementations in Python, either $O(1)$ or $O(n)$ and since a previous homework assignment uses the latter, we will go with this assumption.
Overall, the time complexity will be of $O(n^{3})$, as it the leading term in the polynomial $O(n^{3}) + O(n^{2})$.

\newpage

\subsubsection{Implementation}

\begin{python}
import itertools
import math

def validator(container: tuple[int, int, int]) -> bool:
    x, y, z = container  # raise ValueError if container len != 3
    # x ^ 2 + y ^ 2 = z ^ 2
    # where x, y and z are perfect squares
    return z == math.sqrt(x ** 2 + y ** 2) and x <= y <= z

def solution(integers: tuple[int, int]) -> list[tuple[int, int, int]]:
    a, b = integers  # a is lower bound and b is upper bound
    if not(isinstance(a, int)
           and isinstance(b, int)
           and a < b):
        raise ValueError(
            f'[ERROR] Expected integers for `a` ({a}) and `b` ({b}), where a < b')
    candidates = [_ for _ in range(a, b+1)]
    """
    Given a = 1 and b = 4
    - (1, 2, 3)
    - (2, 3, 4)
    are all valid combinations
    """

    return [candidate for candidate in
            itertools.combinations(candidates, 3) if validator(candidate)]
\end{python}

\newpage

\subsection{Missing Integer}

\subsubsection{Problem Description}

\begin{table}[!h]
\begin{tabular}{|l|l|}
\hline
\textbf{input} & \begin{tabular}[c]{@{}l@{}}an array $A$ of size $N-1$ that contains all integers in the range of $1$ to $N$,\\ except one missing integer. This list contains no duplicates.\end{tabular} \\ \hline
\textbf{output} & the missing number from the list \\ \hline
\end{tabular}
\end{table}

\subsubsection{Pseudocode}

\begin{verbatim}
function missing_element(container: list[int]):
    N = len(container) + 1
    intended_sum = N * (N + 1) / 2 # Gaussian Sum
    return intended_sum - sum(container)
    # the missing element is the difference between each container
\end{verbatim}

\subsubsection{Time Complexity}

This function is quite efficient in finding the missing integer, as it only needs to loop through the container once.
Also, the algorithm does not require the list to be in sorted order and can be extended to tuples, sets and other Python data containers.
First, we need to compute the Gaussian Sum of the container as if it was intact.
The formula for this is:
$$\frac{N \times (N + 1)}{2}$$
and assumes the sequence is monotonically increasing, and contains all integers from $1$ to $N$ inclusive.
Here, we have access to a key characteristic of the container without having to reconstruct it.
Then, we translate the current container in terms of an integer and we apply the sum function $O(n)$.
Subtracting the difference between the two will give you the missing integer.
This will  run in $O(n)$ time, as the only linear aspect is to find the sum of the current container.
\subsubsection{Implementation}

\begin{python}
def missing_integer(container: typing.List[int], dialation: int = 1) -> int:
    """
    Find the missing integer in a monotonically increasing sequence
    """

    intended_sum = gaussian_sum(len(container) + 1)

    return intended_sum - sum(container)

def gaussian_sum(N: int, dialation: int = 1) -> int:
    """
    This is the sum of a contiguous sequence of integers
    Dialation is applied when a constant is applied to all elements in the sequence
    """

    return int(N * (N + 1) / 2) * dialation
\end{python}

\newpage

\section{Extra Credit}

\subsection{Street Map}

\begin{figure}[!h]
\centering
\includegraphics[width=10cm]{StreetMap}
\caption{Street View of Troy High School backroads}
\end{figure}

Each node can contain the following information, and applying this to node \#1 we get:

\begin{itemize}
\item Lateral intersection | Dorthy Lane (East/West)
\item Medial intersection | Acacia Avenue (South/North)
\item Longitude | SomeValue
\item Latitude | SomeValue
\item Dead end | False
\end{itemize}

Each edge on the graph represents the street connecting each intersection and each edge is bidirectional (forgot to make it so in the diagram).

\newpage

\subsection{Course Requirements}

\begin{figure}[!h]
\centering
\includegraphics[width=10cm]{CourseRequirements}
\caption{Example set of courses to be taken}
\end{figure}

Each class should be taken in sequential order and must have it's prerequisites met before.
This is called a topological graph and is applied in different areas such as package managers and neuroscience.
As shown here in this example, some sections or "forests" can exist independently of one another.
These do not have interlaced dependencies and can be completed in their own continuum.
Others are connected on a basis where the course  must be taken in conjunction with the another class.
This can be seen in CS 121 and Math 150A, however this can also be applied to lab courses and their associated lecture (PHYS 225 and PHYS 225L).

\newpage

\subsection{Social Media Network}

\begin{figure}[!h]
\centering
\includegraphics[width=13cm]{SocialMediaNetwork}
\caption{Example Social Media Network}
\end{figure}

Given the list of people and their associated friends, we can construct a graph showing their relationships.
\begin{itemize}
\item Sussy $\rightarrow$ Alex, Jo, Albert
\item Alex $\rightarrow$ Jacob, Sussy, Jon
\item Bobby $\rightarrow$ Nick, Nicole, Sheldon
\end{itemize}

Singly directed edges can be read as "Person 1 is following Person 2 but Person 2 is not following Person 1".
Bidirectional edges are when both parties are following each other.
Each "forest" is a collection of  nodes where they are all connected in some fashion.
The separation occurs when there are not mutual connections between every element in the network

\end{document}