\documentclass[8pt]{article}

\usepackage{geometry}
\geometry{landscape,margin=.2in}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[hidelinks]{hyperref}
% \hypersetup{
%     colorlinks=false,
%     linkcolor=white,
%     filecolor=magenta,
%     urlcolor=white,
% }

\bibliographystyle{abbrv}
\usepackage{multicol}
\usepackage{parskip}

\DeclareMathOperator*{\argmin}{argmin}

% ------------------------------------------------------------------------
\begin{document}

% ------------------------------------------------------------------------
% Overall intro
\section*{The math of reinforcement learning}
All the math for learning and choosing, in a common place and notation.

% ------------------------------------------------------------------------
% Start the sheet...
\begin{multicols}{4}

The aim of all reinforcement learning it to maximize reward recieved. Call this $\rho$.

\subsection*{Some formality}
In all reinforcement leaning problems there is a set of $\textbf{S}$ states, $\textbf{A}$ actions, and $\textbf{R}$ rewards.

A model always start of in some state $s_0 \in \textbf{S}$. Using policy $\pi$ it takes action $a \in \textbf{A}$. Each action leads to a new state $s' \in \textbf{S}$, and sometimes a reward $r \in \textbf{R}$. Eventually, a terminal state is found.

We don't know to find $\rho$ so it is estimated incrementally by an \emph{expected value} $V$. Said another way, $V \approx \rho$.

As we will see there are many ways to define $V$. Many of the proofs in reinforcement learning want to show that for some way of calculating $V$, $V \rightarrow \rho$ as $t \rightarrow \infty$.

Here the expected value at $s$ is $V$. The expected value at $s'$ is $V'$.

The intial value $V_0$ is arbitrary, but is important by Bellman's optimality principle \footnote{\url{https://en.wikipedia.org/wiki/Bellman_equation}}.

The size of each set is denoted by $k = |\textbf{A}|$, $m = |\textbf{S}|$, and $o = |\textbf{R}|$.

\subsection*{A recursive notion of time}
To keep the notation simple, we don't mantion time explicitly. Learning happens recursively, with changes denoted by the `$\leftarrow$'.

For example, $V \leftarrow V + r$ is equivilant to $V(t+\delta t) = V(t) + r(t)$.

\subsection*{Other simplifications}

Commonly $V$ is denoted as function of $s$, $a$, and $t$. That is as $V(s,a,t)$, or even as $V(s,a, s', a', t)$.

To keep the notation compact--at the cost of precision--this kind of thing is left implicit. Only when really needed is the complete notation used.

% ------------------------------------------------------------------------
% Begin denoting all the models
% \vfill\null
% \columnbreak
\section*{The models}
\subsection*{The minimal}

The simplest possible reinforcement learning is:

\begin{eqnarray}
    V \leftarrow V + r \\
    \pi \leftarrow \frac{V}{\sum_\textbf{A}{V}}
\end{eqnarray}

Where the intial value is free $V_0 = \mathbb{R}$, and $\sum_\textbf{A}{V}$ is the sum of the values for all actions $A$. The difference in size between $V$ and $r$ determines the learning rate.

Which is equivilant to:

\begin{eqnarray}
    V \leftarrow V + \alpha r
\end{eqnarray}

Where the learning rate is explicit as, $(0 < \alpha \leq 1)$ and $V_0 = 0$.

\subsubsection*{Other policies}

Different policies can be calculated with the same values. The linear version above is not common. Two common alternatives are softmax,

\begin{eqnarray}
    \pi \leftarrow \frac{e^{V}}{\sum_\textbf{A}{e^{V}}}
\end{eqnarray}

and $\epsilon$-greedy

\begin{eqnarray}
    TODO
\end{eqnarray}

\textbf{Note:} these, or any other policy, can be mixed and matched arbitrarily with any learning rule.

\subsection*{The minimal, discounted}
The effect of recent values can be modulated with:

\begin{eqnarray}
    V \leftarrow \gamma V + \alpha r
\end{eqnarray}

Where $(0 < \gamma \leq 1)$, $(0 < \alpha \leq 1)$, and $V_0 = 0$.

Once the ratio $\alpha/\gamma > 0.5$ current rewards begin to matter more than past values.

This general method for controlling recent values carries on to more complex models, and longer time horizons. Use it as needed.

\subsection*{The discount}
Often values or summed rewards will be discounted as time passes. A common form is:

\begin{eqnarray}
    \gamma \leftarrow \gamma ^ t
\end{eqnarray}

Where $0 < \gamma \leq 1$ and $t \in \mathbb{N}_1$ is an integer code for the elapsed discrete time.

Another form is exponential decay \cite{Francois-Lavet2015}:
\begin{eqnarray}
    \gamma \leftarrow \gamma - \gamma \tau
\end{eqnarray}

Where $\tau \in \mathbb{R}_+$. Typically $\tau \approx 0.02$.

\textbf{Note}: anywhere $\gamma$ is used here assume a discounted form can--and often is--substituted in.

% \vfill\null
% \columnbreak
\subsection*{The temporal difference}
If the reward is delayed, the minimal model can't give credit to past states.

One solution is to take the difference between the current $V$ next, $V'$, as in the \emph{SARSA} rule:

\begin{eqnarray}
    V \leftarrow V + \alpha (r  + \gamma V' - V)
\end{eqnarray}

Where $(0 < \gamma \leq 1)$, $(0 < \alpha \leq 1)$, and $V_0 = 0$. Unlike the minimal model here $\gamma$ wieghs the influence of the future, not the past.

\subsection*{The maximum difference}
\emph{Q learning} takes the difference between the current value $V$ and the maximum value, $\argmin_A \textbf{V}$.

\begin{eqnarray}
    V \leftarrow V + \alpha (r + \gamma \argmin_A \textbf{V} - V)
\end{eqnarray}

Where $(0 < \gamma \leq 1)$, $(0 < \alpha \leq 1)$, and $V_0 = 0$.

\textbf{Note:} as policies are define by their target action $a$, \emph{Q learning} is often called an \emph{off-policy} learning rule, while the temporal difference is \emph{on-policy}.

\subsection*{The average difference}
\emph{Avantage learning} works with relative changes in value, comparing $V$ to its average $\bar{V}$.
\begin{eqnarray}
    V \leftarrow V + \alpha (r + \gamma \bar{V} - V) \\
    \bar{V} = \frac{1}{k} \sum_\textbf{A}{V}
\end{eqnarray}

In practice $\bar{V}$ is approximated online, often with a discounting scheme.

\textbf{Note}: advantages play an important role in policy gradients.

\subsection*{The regretable difference}

TODO....

\vfill\null
\columnbreak
\subsection*{The policy gradient}
Policy gradients learn to map states to directly to actions. They are most useful when problems are continious in action spaace and time, or when promises of local optima are needed.

Policy gradients tend however to be sample inefficient, and by definition they are also \emph{on policy} methods, which can also slow learning down.

A policy is the probability of taking an action $a$, given a state $s$ and some generic parameters $\theta$.

\begin{eqnarray}
\pi = P(a, s, \theta)
\end{eqnarray}

To learn a good policy $\pi$, the parameter gradient should follow the reward gradient.

\begin{eqnarray}
\theta \leftarrow \theta + \alpha \frac{\partial \rho}{\partial \theta}
\end{eqnarray}

As with other reinforcement leaning methods, finding a suitable way to estimate $\rho$ becomes a major concern, and our problem becomes:

\begin{eqnarray}
\theta \leftarrow \theta + \alpha \frac{\partial V}{\partial \theta}
\label{eq:grad}
\end{eqnarray}

\subsection*{The average (again)}
In a gradient setting \emph{avantage learning} increases in $V$ are by definition better than average, $\bar{V}$.

\begin{eqnarray}
    V \leftarrow V + \alpha (r + \gamma \bar{V} - V) \\
    \bar{V} = \frac{1}{k} \sum_\textbf{A}{V}
\end{eqnarray}

 Gradients driven by advatage updates ensure learning is always better than average \cite{Sutton}. By eq. \ref{eq:grad}, purturbations to $\theta \leftarrow \theta + \delta \theta$ which increase $V$ must, by definition, be better than average.

\subsection*{The generalized advantage}
Discounting values can introduce a bias to the final estimate. The generalized advantage scheme takes this into account \cite{Schulman2015a}.

\subsection*{Actor-critic}
Value learning and policy can be joined into a single \emph{actor-critic architecture}.

TODO\ldots

% ------------------------------------------------------------------------
\vfill\null
\columnbreak
\section*{The planning models}
Planning means the learner has, or creates, an \emph{explicit map} of the state-space. This map can come from outside the system, from some useful oracle, or can be learned.

\subsection*{The DYNA}

\subsection*{The prioritized sweep}

\subsection*{The successor}

\end{multicols}

\newpage
\bibliography{library}
\end{document}


\newpage \subsection{Recurrent Sequences}

	\begin{myitemize}
		\item \href{https://ufile.io/ywbniil2}{WOOT 2010-11 Recursion}
	\end{myitemize}

	\begin{BoxedTheorem}{Sum of Geometric Sequences}{}\label{theorem:Sum of Geometric Sequences}
		Every recurrent sequence can be written as a sum of some geometric sequences. Given a recurrent sequence,
		\[x_n = a_1x_{n-1} + a_2x_{n-2}	+\dots + a_kx_{n-k}\]
		Then $ x_n $ can be written as
		\[x_n = c_1r_1^n + c_2r_2^n +\dots + c_lr_l^n\]
		For all $ c_i $ if $ r_i $ are the roots of the \emph{characteristic polynomial} of the recursion. Which is:
		\[\tcboxmath[colback=white, colframe=white]{x^k - a_1x^{k-1} - a_2x^{k-2} \dots - a_k = 0}\]
		If there are double roots, say $ r_1 = r_2 = r_3 $, then we instead have,
		\[x_n = \tcboxmath[colback=white, colframe=white]{c_1r_1^n + c_2n\ r_2^n + c_3n^2\ r^n} \dots + c_lr_l^n\]

		Reversely, we can say that a sequence defined by a sum of geometric recurrent series is a recursion.
	\end{BoxedTheorem}

	\lem{}{Let $ F_n $ be the $ n $th Fibonacci number. Then the following holds:
		\[F_{n}^2 + F_{n+1}^2 = F_{2n+1}\]}

	\proof{Expanding the general form of the terms, and showing that $ a_n = F_{n}^2 + F_{n+1}^2 - F_{2n+1} $ is a recursion by \autoref{theorem:Sum of Geometric Sequences}.}

	\begin{BoxedTheorem}[title=Repertoire Method]{}{}
		Given a recurrent function defined by
		\[f(n) = A(n)\a + B(n)\beta + C(n)\gamma\]
		We plug in different values for $ f(n) $, for example, $ f(n) = 1, n, 2n $ etc. for which the values are known from the recursion, and then solve for $ A, B, C $.
	\end{BoxedTheorem}


\chapter{Generative Convolutional Neural Networks}

In this chapter we develop the idea of a generative convolutional neural
network.  Doing so will allow us to use a natural generative pre-training
algorithm so that we do not need the same quantity of labeled data
that deep neural networks normally require.

\section{Fixed-length speech samples}

We begin our discussion with fixed-length vectors of speech signals
$\mathbf{X}\in\mathbb{R}^{N\times D}$ which denotes $N$ vectors
of dimension $D$.  We use Numpy-style slicing syntax so that
$\mathbf{X}(n_1:n_2,d_1:d_2)\in \mathbb{R}^{(n_2-n_1)\times(d_2-d_1)}$
refers to the matrix
\begin{equation}
\begin{bmatrix}
  \mathbf{X}(n_1,d_1) & \mathbf{X}(n_1,d_1+1) & \cdots & \mathbf{X}(n_1,d_2-1)\\
  \mathbf{X}(n_1+1,d_1) & \mathbf{X}(n_1+1,d_1+1) & \cdots & \mathbf{X}(n_1+1,d_2-1)\\
  \vdots & \vdots & \ddots & \vdots \\
  \mathbf{X}(n_2-1,d_1) & \mathbf{X}(n_2-1,d_1+1) & \cdots & \mathbf{X}(n_2-1,d_2-1)
\end{bmatrix}
\end{equation}
and we denote the index set for a vector of length $D$ as $[D]=\{0,1,\ldots,D-1\}$. We will consider a sequence of models for these data.  Initially we will begin with
a mixture model and progressively we will develop towards the convolutional neural network model.

\subsection{Circulant Gaussian Models}

We may model these vectors using circulant Gaussians $\mathcal{N}_D(\mathbf{0},\mathbf{\Sigma})$ where
$\tilde{\boldsymbol{\Sigma}}\in\Circ_D$ where $\Circ_D$ is the set of circulant matrices over $\mathbb{R}^{D\times D}$.
Since $\boldsymbol{\Sigma}$ is circulant we may write it as $F_D^*\diag(\tilde{\boldsymbol{\sigma}})F_D$ for some
real vector $\tilde{\boldsymbol{\sigma}}\in\mathbb{R}^D$. Since $\boldsymbol{\Sigma}$ is a real symmetric
 positive definite circulant matrix we may write it as $\boldsymbol{\Sigma}=\Circ(\boldsymbol{\sigma})$
where $\boldsymbol{\sigma}$ is the first column of $\boldsymbol{\Sigma}$.  Due to the symmetry and circularity of
$\boldsymbol{\Sigma}$ this means that $\boldsymbol{\sigma}$ is symmetric so that for $1 < d < D$ we have
$\boldsymbol{\sigma}(d)=\boldsymbol{\sigma}(D-d)$ and the same holds for $\tilde{\boldsymbol{\sigma}}=\mathbf{F}\boldsymbol{\sigma}$.

The circulant model is given by a single vector specifying the spectral distribution


\documentclass[11pt,twoside,a4paper]{article}
\usepackage{amsmath}
\usepackage{commath}
\usepackage{graphicx}
\usepackage{tikz}
\usetikzlibrary{bayesnet}

\begin{document}
\section{Uivariate Gaussian distribution}
  Let $\{x_i\}= \{x_1, x_2, \cdots,x_n\}$ denote $n$ random variates drawn from
  a univariate Normal distribution parametrised by the unknown variance $\omega$
  and known mean $\mu = 0$. Let us assume that the a Gaussian noise with known
  variannce $\zeta$ has been added to the variates yeilding new variates
  $\{y_i\}= \{y_1, y_2, \cdots,y_n\}$. Using statistical notations we can write
  \begin{align}
    \label{eqn_posteriors_signal_and_noise}
    \Pr \left(\{x_i\} | \omega \right) &= \prod_{i=1}^n
      \frac{1}{\sqrt{2 \pi \omega}} \exp \left(-\frac{x_i^2}{2 \omega} \right)\\
    \Pr \left(\{x_i\} | \{y_i\}, \zeta \right) &= \prod_{i=1}^n
      \frac{1}{\sqrt{2 \pi \zeta}} \exp
      \left(-\frac{\left(y_i - x_i \right)^2}{2 \zeta} \right)
  \end{align}
  Using Bayes theorem the inference about the variance $\omega$ can be written
  as
  \begin{align}
    \label{eqn_posterior_omega_given_noisy_random_variates}
    \Pr \left(\omega | \{y_i\} \right) \propto
      \Pr \left(\{y_i\} | \omega \right) \Pr(\omega).
  \end{align}
  Assuming a prior distribution $\Pr(\omega) = 1$, we can transfom the problem
  to the computation of the conditional distribution which is given by
  \begin{align}
    \Pr \left(\{y_i\} | \omega \right) &= \prod_{i=1}^n
      \frac{1}{\sqrt{2 \pi (\zeta + \omega)}} \exp
      \left(-\frac{y_i^2}{2(\zeta + \omega)} \right), \nonumber \\
    &= (2 \pi)^{-\frac{n}{2}} \left(\zeta + \omega \right)^{-\frac{n}{2}}
      \exp \left(-\frac{\sum_{i=1}^n y_i^2}{2 (\zeta + \omega)} \right) .
  \end{align}
  This equation represents an inverse-Gamma distribution:
  \begin{align}
    f(x | \alpha, \beta) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{\alpha - 1}
        \exp \left(-\frac{\beta}{x} \right).
  \end{align}
  The statistical properties of inverse-Gamma distribution is well known and
  are shown in the table below.
  \begin{table}
    \caption{Statistical properties of inverse-Gamma distribution}
    \label{tab_statistical_properties_of_inverse_gamma}
    \begin{center}
      \begin{tabular}{|c|c|c|}
        \hline
        mode & mean $(\alpha > 1)$ & variance $(\alpha > 2)$ \\
        \hline
        $\frac{\beta}{\alpha + 1}$ & $\frac{\beta}{\alpha - 1}$ &
          $\frac{\beta^2}{(\alpha - 1)^2 (\alpha - 2)}$ \\
        \hline
      \end{tabular}
    \end{center}
  \end{table}
  We can find the values of $n$ for which the variance of the distribution is
  defined as shown below:
  \begin{align}
    \alpha > 2 \\
    \frac{n}{2} + 1 > 2 \\
    \frac{n}{2} > 1 \\
    \boxed{n > 2}
  \end{align}
  This implies that we need at least 2 samples to determine the variance of the
  estimated and at least one sample to determine the mean.

  The logarithm of the posterior distribution can be written as
  \begin{align}
    \psi &= \log \left(\Pr \left(\{y_i\} | \omega \right) \right) \\
    &= -\frac{n}{2}\log(2 \pi) - \frac{n}{2}\log(\zeta + \omega)
      -\frac{1}{2} \frac{\sum_{i=1}^n y_i^2}{(\zeta + \omega)}.
  \end{align}
  The gradient of the log-posterior with respect to $\omega$ is given by
  \begin{align}
    \frac{\partial \psi}{\partial \omega} = -\frac{n}{2 (\zeta + \omega)}
      - \frac{\sum_{i=1}^n y_i^2}{2 (\zeta + \omega)^2}.
  \end{align}
  The mode of the distribution can be found by setting gradients equal to zero
  and solving the resulting equation:
  \begin{align}
    \frac{n}{2 (\zeta + \omega)}
      + \frac{\sum_{i=1}^n y_i^2}{2 (\zeta + \omega)^2} = 0, \\
    \frac{n}{2} + \frac{\sum_{i=1}^n y_i^2}{2 (\zeta + \omega)} = 0 \\
    n + \frac{\sum_{i=1}^n y_i^2}{(\zeta + \omega)} = 0 \\
    n (\zeta + \omega) + \sum_{i=1}^n y_i^2 = 0
    %\zeta + \omega + \frac{\sum_{i=1}^n y_i^2}{n} = 0 \\
    %\omega = \frac{\sum_{i=1}^n y_i^2}{n} - \zeta \\
    %\omega = \bar{y} - \zeta
  \end{align}
  Thus mode of the distribution is given by
  \begin{align}
    \boxed{\bar{\omega} = \bar{y} - \zeta.}
  \end{align}
  As we can see, the mode may have chance of being negative or zero which
  corresponds to non-physical variance. Hence we cannot treat this as point
  estimate of the distribution. The question then is what suffices as a
  ``good'' point estimate?

  From a Bayesian prespective, we haven't really assed the situaton carefully.
  By choosing uniform prior for $\omega$ we have allowed non-positive values
  for the variance. However, a sensible prior for the variance should disregard
  any values less than or equal to zero. According to Jeffery's a reasonable
  prior distribution for a variable taking only positive values should be
  a uniform prior on the logarithm of the variable. Let $\eta = \log(\omega)$
  and hence we can write,
  \begin{align}
    \Pr(\eta) = 1.
  \end{align}
  The rule of chang of variables says that
  \begin{align}
    \Pr(\omega) \dif \omega = \Pr(\eta) \dif \eta.
  \end{align}
  Therefore the Jeffreys prior on $\omega$ can be derived as
  \begin{align}
    \Pr(\omega) &= \Pr(\eta) \frac {\dif \eta}{\dif \omega}, \\
    &= \frac{1}{\omega}.
  \end{align}

  Applying the Jeffreys prior, the logarithm of the posterior distribution
  becomes
  \begin{align}
    \psi  = - \frac{n}{2} \log (2 \pi) - \frac{n}{2} \log (\zeta + \omega)
      - \frac{1}{2} \frac{\sum_{i=1}^n y_i^2}{(\zeta + \omega)} - \log(\omega).
  \end{align}
  We compare the two posterior distributions in
  Figure~\ref{fig_log_post_uniform_jeffreys}
  \begin{figure}
    \includegraphics[scale=0.8]{images/log_post_uniform_jeffreys.pdf}
    \caption{Comparing the log-posterior distributions of $\omega$ with uniform and Jeffreys prior.}
    \label{fig_log_post_uniform_jeffreys}
  \end{figure}
  As we can see the resultng posterior distribution has spike near zero. Can
  there be a probabilisitc explanation for this characterstic? Since we are
  estimating the variance in the presence of noice, we have have to think in
  terms of a hierarchical way.

\subsection{Hierarchical modelling}
  In hierarchical model, one considers a prior of the form
  \begin{align}
    \Pr(\zeta + \omega) = \frac{1}{\zeta + \omega}.
  \end{align}
  The logaritm of the posterior now has the form
  \begin{align}
    \psi  = - \frac{n}{2} \log (2 \pi) - \frac{n}{2} \log (\zeta + \omega)
      - \frac{1}{2} \frac{\sum_{i=1}^n y_i^2}{(\zeta + \omega)}
      - \log(\zeta + \omega).
  \end{align}

\end{document}


\chapter{Equation Conventions}
\label{ch:conventions}

\textit{In this manual we will} be adhering to the following conventions

\section{List of Symbols}

\begin{itemize}
	\item {\boldmath{$i$}} Receiver identification or {\boldmath{$r$}}
	\item {\boldmath{$j$}} Satellite identification or {\boldmath{$s$}}
	\item {\boldmath{$k$}} Epoch number or {\boldmath{$t$}}
	\item {\boldmath{$q$}} GNSS type (GPS,GALILEO,GLONASS,QZSS)
	\item {\boldmath{$c$}} Speed of light [m/s]
	\item {\boldmath{$x$}} Vector of parameters to be estimated
	\item {\boldmath{$y$}} Vector of observations
	\item {\boldmath{$v$}} Vector of residuals
	\item {\boldmath{$H$}} Design matrix
	\item {\boldmath{$\sigma$}} Standard deviation of observable
	\item {\boldmath{$\Delta$}} Increment to a priori values [m]
	\item {\boldmath{$\lambda$}} Wavelength or {\boldmath{$\lambda_1,\lambda_2,\lambda_5$}}
	\item {\boldmath{$f_1,f_2,f_5$}} frequency
	\item {\boldmath{$\alpha$}} Ambguity or {\boldmath{$N$}} Real valued ambguity and {\boldmath{$\bar{N}$}} Integer part of real valued ambguity
	\item {\boldmath{$\alpha$}} level of significance

	\item {\boldmath{$\beta$}} Biases
	\item {\boldmath{$\zeta$}} Clock offsets
	\item {\boldmath{$\delta t$}} Clock error [s]
	\item {\boldmath{$\kappa$}} Correction - relativity
	\item {\boldmath{$\iota$}} Ionosphere or  {\boldmath{$I$}}
	\item {\boldmath{$\tau$}} Troposphere or {\boldmath{$T,T_h,T_w$}}
	\item {\boldmath{$M$}} elevation dependent mapping function for the troposphere wet delay
	\item {\boldmath{$\xi$}} Phase wind-up error
	\item {\boldmath{$\epsilon$}} Error in observations and unmodelled effects [m]
	\item {\boldmath{$\phi_i^j$}} Carrier phase observable (times c) [m]
	\item {\boldmath{$P_i^j$}} Pseudo range observable [m]

\end{itemize}

Lets try this for example:
For an undifference, uncombined float solution, the linearized observation equations for pseudorange and phase observations from satellite $s$ to receiver $r$ can be described as:

\begin{math}
\Delta P_{r,f}^{q,s} = u_r^{q,s} . \Delta x + c . (\delta t_r^q - \delta t^{q,s}) + M_r^{q,s} . T_r + \gamma_f^q . I_{r,1}^{q,s} + d_{r,f}^q - d_f^{q,s} + \epsilon_{P,f}^q
\end{math}\\
\begin{math}
\Delta\phi_{r,f}^{q,s} = u_r^{q,s} . \Delta x + c . (\delta t_r^q - \delta t^{q,s}) + M_r^{q,s} . T_r - \gamma_f^q . I_{r,1}^{q,s} + \lambda_f^q . N_{r,f}^{q,s} + b_{r,f}^q - b_f^{q,s} + \epsilon_{L,f}^q
\end{math}

where$\Delta P_{r,f}^{q,s}$ and $\Delta\phi_{r,f}^{q,s}$ are the respective pseudorange and phase measurements on the frequency $f$(f=1,2), from which the computed values are removed;
$u_r^{q,s}$ is the receiver-to-satellite unit vector;
$\Delta x$ is the vector of the receiver position corrections to its preliminary position;
$\delta t_r^q$ and $\delta t^{q,s}$ are the receiver and satellite clock errors respectively;
$c$ is the speed of light in a vaccum
$M_r^{q,s}$ is the elevation dependent mapping function for the troposphere wet delay from the corresponding zenith one $T_r$;
$I_{r,1}^{q,s}$ is the ionosphere delay along the line-of-sight from a receiver to a satellite at the first frequency and $\gamma_f^q = (\lambda_f^q / \lambda_1^q)^2$;
$\lambda_f^q$ is the wavelength for the frequency $f$ of a GNSS $q$;
$N_{r,f}^{q,s}$is the phase ambguity
$d_{r,f}^q$ and $b_{r,f}^q$ are the receiver hardware delays of code and phase observations respectively;
$d_f^{q,s}$ and $b_f^{q,s}$ are the satellite hardware delays of code and phase observations, respectively;
$\epsilon_{P,f}$ and $\epsilon_{L,f}$ are the code and phase measurement noises respectively.


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{momftfr}
\section*{\hspace*{-1.6cm} momftfr}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Frequency moments (order 1 and 2) of a time-frequency representation.
\end{minipage}
\vspace*{.35cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
[tm,T2] = momftfr(tfr)
[tm,T2] = momftfr(tfr,tmin)
[tm,T2] = momftfr(tfr,tmin,tmax)
[tm,T2] = momftfr(tfr,tmin,tmax,time)
\end{verbatim}
\end{minipage}
\vspace*{.35cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty momftfr} computes the frequeny moments of order 1 and 2 of a
        time-frequency representation\,:
\[t_m(f) = \frac{1}{E}\ \int_{-\infty}^{+\infty} t\ \mbox{tfr}(t,f)\ dt\ \
;\ \ T^2(f) = \frac{1}{E}\ \int_{-\infty}^{+\infty} t^2\ \mbox{tfr}(t,f)\
dt - t_m(f)^2.\]

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c} Name &
Description & Default value\\ \hline {\ty tfr} & time-frequency
representation (size {\ty (N,M)}). \\ {\ty tmin} & smallest column
element of {\ty tfr} taken into account & {\ty 1} \\ {\ty tmax} & highest
column element of {\ty tfr} taken into account & {\ty M}\\ {\ty time} &
true time instants & {\ty (1:M)}\\ \hline {\ty tm} & averaged time (first order
moment)\\ {\ty T2} & squared time duration (second order moment)\\
\hline
\end{tabular*}

\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Example}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
         sig=fmlin(200,0.1,0.4); [tfr,t,f]=tfrwv(sig);
         [tm,T2]=momftfr(tfr);
         subplot(211); plot(f,tm); subplot(212); plot(f,T2);
\end{verbatim}
The first order moment represents an estimation of the group delay, and the
second order moment the variance of this estimator. We can see that the
estimation is better around the time center position than at the edges of
the observation interval.
\end{minipage}
\vspace*{.35cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
momttfr, margtfr.
\end{verbatim}
\end{minipage}


\section{The \variant{\childrentransitions} problem}
\label{sec:simplenodes}

In this section, we provide the formal problem definition of the
{\variant{\childrentransitions}}
and describe a greedy polynomial-time algorithm for solving it.

\begin{problem}[{\variant{\nodeitems}}]
\label{problem:simplenodes-variant}
Given $G=(V,E)$, transition matrix $\transition$,
initial distribution of items to nodes
$\initial$ and integer $k$, find
$S\subseteq V$ such that $|S|=k$ such that
$\objective_{_\shortchildrentransitions}\left(S\right)$ is minimized.
\label{problem:simple-nodes-variant}
\end{problem}

As we can see from the definition of
$\objective_\shortchildrentransitions\left(S\right)$
in \eqref{eq:shortchildrentransitions},
the contribution of each node $u\in {V-S}$
to the objective function is equal to
\begin{equation}
\label{eq:parent-contribution}
\objective_{_\shortchildrentransitions}(\{u\}) =
	\initial(u)\sum_{v\in V}\transition(u,v)\left(1-\transition(u,v)\right)
\end{equation}
and independent of any other node in $V-S$.
It directly follows, then, that Problem~\ref{problem:simplenodes-variant}
is solved by a simple greedy algorithm
that places in $S$ the {\bf top k} nodes $u$ in terms
of contribution $\objective_{_\shortchildrentransitions}(u)$.


\chapter{Graphical Models}
\label{chap:Graphical Models}

\section{Introduction}

\section{Directed Graph}

\section{Undirected Graph}

\section{Inference in Graphical Models}
\subsection{Factor graphs}
The joint distribution over a set of variables in the form of a product of factors
\begin{align}
p(\vec{x}) = \prod_s f_s(\vec{x}_s)
\end{align}
where $\vec{x}_s$ denotes a set of the variables.We shall denote individual variable by $x_i$,which can comprise groups of variables(such as vectors and matrices).Each factor $f_s$ is a function of a corresponding set of variable $\vec{x}_s$.
Directed graphs have factors $f_s(\vec{x}_s)$ as local conditional distributions.Undirected graphs' factors are potential function over the maximal cliques.

\subsection{The Sum-product algorithm}
\textbf{sum-product} algorithm solves the problem of evaluating the \textbf{local marginals} over nodes or subsets of nodes.And \textbf{max-sum} algorithm find the \textbf{most probable state},evaluating the maximal joint distribution.

The marginal is obtained by
\begin{align}
p(x) = \sum_{\vec{x}\backslash x}{p(\vec{x})}
\end{align}
where $\backslash x$ denotes the set of variables in $\vec{x}$ omitting $x$.

Join distribution
\begin{align}
p(\vec{x}) =\prod_{s\in \text{ne}(x)}{F_s(x,\vec{X}_s)}
\end{align}

Then interchanging the sums and products,we obtain
\begin{align}
\text{objective} &= p(x)=\sum_{\vec{x}\backslash x}{p(\vec{x})} \\
&= \prod_{s\in\text{ne}(x)}{\left[\sum_{X_s}F_s(x,X_s)\right]} \\
&= \prod_{s\in\text{ne}(x)}\mu_{f_s\rightarrow x}(x)
\end{align}
View \textbf{messages} from the factor nodes $f_s$ to the variable node $x$ as
\begin{align}
\mu_{f_s\rightarrow x}(x) &\equiv \sum_{X_s}F_s(x,X_s) \\
F_s(x,X_s) &= f_s(x,x_1,\ldots,x_M)G_1(x_1,X_{s1})\ldots G_M(x_M,X_{sM})
\end{align}

Recursive inference in the sub-graph and interchanging sums and products leads to
\begin{align}
\mu_{f_s\rightarrow x}(x) &= \sum_{x_1}\ldots\sum_{x_M} f_s(x,x_1,\ldots,x_M)\prod_{m\in \text{ne}(f_s)\backslash x} \mu_{x_m\rightarrow f_s}(x_m)
\end{align}
where $\text{ne}(x)$ denotes the set of neighbor variables.
And define \textbf{messages} from variable nodes to factor nodes
\begin{align}
\mu_{x_m\rightarrow f_s}(x_m) &\equiv \sum_{X_{sm}}G_m(x_m,X_{sm})
\end{align}

Again,making use of (sub)-graph factorization,we have
\begin{align}
\mu_{x_m\rightarrow f_s}(x_m) &= \prod_{l\in \text{ne}(x_m)\backslash f_s}\left[\sum_{X_{ml}}F_l(x_m,X_{ml}) \right] \\
&=\prod_{l\in \text{ne}(x_m)\backslash f_s}\mu_{f_l\rightarrow x_m}(x_m)
\end{align}

\subsection{Max-sum algorithm}
The \textbf{sum-product} algorithm allows us to take a joint distribution $p(\vec(x))$ expressed as a factor graph and efficiently find marginal component variables.\textbf{Max-sum} find a setting of variables that has the largest probability,which is an application of \textbf{dynamic programming}.
Maximize the joint distribution
\begin{align}
\vec{x}^{max} = \arg\max_{\vec{x}}p(\vec{x})
\end{align}
Joint distribution
\begin{align}
p(\vec{x}) &= \prod_{s\in \text{ne}(x)}F_s(x,X_s)
\end{align}
Making use of \textbf{distributive law} for multiplication with max operator,similarly to add operator
\begin{align}
\text{objective} &= \max\ln p(\vec{x}) =\max\sum\ln F_s(x,X_s) \\
&=\sum\max\ln F_s(x,X_s)
\end{align}
Making use of recursion,factorize $F_s(x,X_s)$ with sub-graph,we can obtain
\begin{align}
\mu_{f\rightarrow x} &= \max_{x_1,\ldots,x_M}\left[\ln f(x,x_1,\ldots,x_M) + \sum_{m\in \text{ne}(f_s)\backslash x} {\mu_{x_m\rightarrow f}(x_m)} \right] \\
\mu_{x\rightarrow f}(x) &= \sum_{l\in \text{ne}(x)\backslash f}{\mu_{f_l\rightarrow x}}(x).
\end{align}
where $\mu_{f\rightarrow x} \equiv \max\ln F_s(x,X_s)$


\section{FourierTransform: Fast Fourier Transforms}
[Warren Smith prepared the analysis and modules.]

This is an {\em in place} FFT routine with array length:  $N = 2^n$.
\begin{equation}
 a_{output,k} = \sum_{m=0}^{N-1} a_{input,m}
        e^{(2 \pi i \mbox{direction} m k/N)}
\end{equation}

where direction $= \pm 1$.
You have to do any scaling (by $1/N$ or $1/\sqrt{N}$) or zeroing of a[] by
yourself.

Uses $2N \ln N + 2 N + O(\ln N)$ [real] multiplications,
and $ 3N \ln N + 2 N + O(\ln N)$ [real] additions.
(And hopefully the compiler optimizes the subscripting in the inner loop so
that only $2 N \ln N$ total subscripting ops are needed.)

The test routine shown yields
error after 1 forward and one backward use
at least 3 decimal places better than
slow FT has after only one use!

The basic FFT idea (for N a power of 2) is from J.Cooley \& J.Tukey:
An algorithm for the Machine Calculation of Complex Fourier Series,
MOC 19 (1965) 297-301. (Knuth, Henrici, and Dahlquist/Bjorck also discuss
the FFT.) The basic idea is that to calculate the DFT
\begin{eqnarray}
    A[k]  & = & \sum_{j=0}^{N-1}a_j*W^{kj}  \, \mbox{for k=0..N-1}\\
    N & = & \mbox{a power of 2,}\, N>1\\
    W & = & \mbox{a principal Nth root of unity}\\
   \mbox{e.g., } W & = & e^{2 i \pi/N}
\end{eqnarray}
We may write
\begin{eqnarray}
    A_k & = & g_k + h_k * W^k \, \mbox{ where}\\
    g_k & = & \sum_{0<=2j<N} a_{2j}   * (W^2)^{k j}\\
    h_k & = & \sum_{0<=2j<N} a_{2j+1} * (W^2)^{k j}
\end{eqnarray}
are FFTs of half the size (on the even and odd indexed a[]'s, respectively)
note that $g_k$ and $h_k$ are periodic in k with period $N/2$,
since $W^{N/2} = -1$,
so that if $T(N)=\mbox{time to calculate FFT of size N}$, then
$T(N)=2T(N/2)+O(N)$ and so $T(N)=O(N \ln N)$. The inverse transform is:
\begin{equation}
    a_k  =  \mbox{Ninverse}  \sum_{j=0}^{N-1} a_j
              \mbox{Winverse}^{k j} \mbox{  for k=0..N-1}
\end{equation}
also an FFT and is calculable by the same method. (Ninverse*N=1. Winverse*W=1.)
Cooley-Tukey also works for general highly composite N.
A short and elegant implementation is:
[Warren E.Ferguson: A simple derivation of glassman's general N FFT,
Computers \& Math. with Applics. 8 (1982) 401-411].

For an efficient implementation of the FFT, further refinements are desired.
These include:
\begin{enumerate}
\item Removal of the recursion by "reverse binary permuting" the
original data;
\item "In place" implementation with no extra storage requirement;
\item Calculation of the $W^k$'s (k=0,1..) by efficient and stable recurrences
(thus avoiding the need for transcendental functions);
\item Possibly use "fast complex multiplication";
\item In specific but common applications,
some further savings may be possible. Thus when doing a fast convolution
of two arrays of N reals using an FFT, the two FFTs may be
accomplished by ONE N point complex FFT,
followed by linear time uncombining/termwise
multiplication step, followed by a N/2 point reverse FFT.
The reverse binary
permuting stages may be avoided completely.
\end{enumerate}

The basic symmetry here is that
the FFT of a real array is "conjugate even", e.g.
\begin{equation}
    A_k = \mbox{CompConjugate}(A_{N-k}) \mbox{ if a[0..n-1] is real}
\end{equation}

R.C.Singleton [e.g. see his algorithm for fast circular convolutions,
CACM 12,3 (March 1969) 179; his FFT2 algorithm, CACM 11 (Nov 1968) 773-779;
and his article CACM 10 (Oct 1967) 647-654] has suggested a second
difference
method [which keeps the real and imaginary parts of exp(ick) uncoupled]
for evaluating exp(ick):
\begin{eqnarray}
    e^{ic(k+1)} & = & e^{ick} + I_{k+1}\\
    I_{k+1} & = & -4 \sin^2(c/2) e^{ikc} + I_k\\
    I_0 & = & 2 i \sin(c/2) e^{-ic/2}
\end{eqnarray}
which is both faster and experimentally far more stable (typically
yielding 500 times smaller error for 128 point transform+inverse transform)
than the straightforward "multiply by exp(ic)" method. (It's more stable
because of the small multiplier $-4 \sin^2(c/2)$, as opposed to 1.)

However a better idea which I recently thought of is to use this recurrence:
\begin{equation}
    e^{ic(k+1)} = e^{ic(k-1)} + 2 i \sin(c) e^{ick}
\end{equation}
which requires only 2*,2+ per complex exponential (Singleton is 2*,4+;
naive method is 4*,2+ and isn't stable) and is also stable since it involves
the small multiplier $2 i \sin(c)$. (In fact, this method runs faster than
Singleton, is easier to program, and even yielded slightly better accuracy
in the test program below, too!) Therefore this modification of Singleton
is the method I've used.

Incidentally, Press et al. in their NR book use the Singleton algorithm
but neglect to credit Singleton!

By a trivial modification of my code, one could generate the sines and
cosines by repeated application of the bisection identities:
\begin{eqnarray}
    \cos(t/2) & = & \sqrt{0.5 (1.0 + \cos(t))}\\
    \sin(t/2) & = & 0.5 \sin(t)/\cos(t/2)
\end{eqnarray}
starting from the values with $t=\pi$ and $t=\pi/2$ as special cases, thus
avoiding trig subroutine calls entirely.
OK, I've now done this; now using precomputed table.

The time savings introduced by either of these is small, however.
Finally, "radix 8" transforms are experimentally the most efficient,
typically 20\% faster than "radix 2" routines like this one, although far
more complicated; and anyway I suspect the advantage is $<20\%$
in the modern cache-memory system world, since I suspect the radix 2 algorithm
has better cache locality. However, that has not been tested.

Also you could take advantage of 1's and 0's to save a little time at
the expense of considerably more space.

Another idea which I have chosen not to implement is the fact that two
complex numbers may be multiplied in 3 real multiplications:
Thus  $E+iF = (a+bi)*(c+di)$  may be accomplished via the instructions
\begin{eqnarray}
  bpa & = & b+a; \\
  bma & = & b-a; \\
  E & = & a*(c+d);\\
  F & = & bma*c+E; \\
  E & -= & bpa*d;
\end{eqnarray}
and if bpa and bma are precomputed, this is a 3*,3+ method for a complex
multiplication. This idea may be used to reduce (?) the box score from
the present  $2N \ln N+2N+O(\ln N)$ mults,
$3N \ln N+2N+O(\ln N)$ adds  to
$1.5N \ln N+2N+O(\ln N)$ mults  and  $3.5N \ln N+4N+O(\ln N)$ adds.
If a floating point
multiplication is M times slower than a floating point addition,
this idea pays iff $ln N>4/(M-1)$. On PDP-11/44 C, however, rough timing
has shown that $M=1.08$ (but with considerable standard deviation.
It does
about $5*10^4$ additions/sec.) so this idea is not worth it unless N is
enormous.

Some similar, but worse, ideas have been suggested by Buneman:
\begin{verbatim}
 If c=cos(m),s=sin(m), then precompute
    t1 = (1-c)/s = s/(1+c)= tan(m/2)  and
    t2 = (1+s)/c = c/(1-s).
\end{verbatim}
Then $X+iY = (a+bi)*(c+is)$ may be found in 3*,3+ by:
\begin{verbatim}
  if(|t1|<|t2|) then
     X = a-t1*b; Y = b+s*X; X -= t1*Y;
  else
     X = b+t2*a; Y = c*X-a; X -= t2*Y;
\end{verbatim}
tans may be updated by
$\tan(x+y)-\tan(x-y) = 2*\tan(y)/(1-(\tan(x)*\tan(y))^2)$,
but the extra overhead seems not to be worth it].

There is also a symmetric 3*,5+ (4+ with precomputation) formula for
$E+iF = (a+bi)*(c+di)$:
\begin{eqnarray}
  E & = & a*c-b*d\\
  F & = & (a+b)*(c+d)-a*c-b*d
\end{eqnarray}

Other FFT algorithms: Winograd has shown how to design FFTs with N prime
(as opposed to the Cooley-Tukey approach which works for N highly composite)
that run in $O(N \ln N)$ time and even with only O(N) multiplications;
the latter figure is optimal. [S.Winograd: Math. of Comput. 32 (1978) 175-179;
Advs in Math 32 (1979) 83-117].

Winograd's approach is based on a theorem that allows him (by a permutation
of the original and transformed variables) to express FFTs for N prime
in terms of a circular convolution of N-1 elements, plus some additions.
He then shows how circular convolutions of k elements (for certain small k)
may be computed in a small number of arithmetic operations, (for k=2..6,
the number of multiplications Winograd uses is 2,4,5,10,8; for k prime,
Winograd shows that a (2k-2)* scheme for CC(k) always exists) and further, how
CC(n1) and CC(n2) algorithms may be composed to make a CC(n1*n2) algorithm, IF
n1 and n2 are relatively prime, that uses mult(n1)*mult(n2) multiplications.
Also, he shows how FFT(n1*n2) may be computed via FFT(n1) and FFT(n2)
in mult(n1)*mult(n2) multiplications, IF n1,n2 relatively prime, and
also gives methods for FFT(prime power). He gives two appendices
containing optimized CC(2..6) and FFT(2..9) algorithms. Winograd's methods
don't appear suitable for general N, but if N is specified in advance, they
make it possible to do considerable fine tuning at the expense of
large algorithm complexity.

Meanwhile, Nussbaumer [H.J.Nussbaumer: Fast Polynomial Transform algorithms
for digital convolutions, IEEE Transactions on Audio, Speech, Signal Processing
28,2 (April 1980) 205-215 (this article has many references to other FFT
schemes); see also Knuth 2: 503, 652-653] has found another way to do circular
convolutions of arrays of (N=a power of 2) reals without any NTTs, FFTs, trig,
or complex numbers. His approach is based on viewing circular convolutions
as polynomial multiplications modulo certain simple polynomials, factoring the
modular polynomials, divide and conquer, chinese remainder thm.
His approach uses roughly NlgN *,
$N \ln N \ln \ln N$ +, is fairly complicated to program, and requires extra space.

[Nussbaumer \& Quandalle: IBM JResDev 22 (1978) 134-144] show how some
particularly efficient CC and FFT schemes for N in the range 10-3000
may be constructed; their approach is based on some novel ways to combine
efficient small CC schemes that is rather like the NTT (Number theoretic
transform) only in rings of polynomials rather than in the integers.

However even the best known arithmetic op count methods only improve on my
method by perhaps 30\%, and at the cost of considerable complexity.
C.H.Papadimitriou [Optimality of the FFT, JACM 26 (1979) 95-102 and its refs]
has shown that in some models of computation $O(N\ln N)$ is
optimal for the FFT, while Patterson et al have shown a lower
bound of $O(N\ln N/\ln \ln N)$ for integer multiplication on multitape
Turing machines, see Knuth 2.

FFTs in a finite field (if the ring ZmodK, called "number theoretic
transforms") are discussed in Aho,Hopcoft,Ullman: The Design and Analysis
of Computer algorithms, Addison-Wesley 1974. They recommend using W=2,
N=a power of 2, do all arithmetic in the ring of integers
modulo $2^{N/2}+1$
(in which W is an Nth root of unity, and in which the convolution theorem
\begin{equation}
    C_i=A_iB_i  \Longleftrightarrow
         c_i = \sum_{j=0}{N-1} a_i*b_{i-j \bmod N}
\end{equation}
[Which makes possible the calculation of discrete convolutions in $N\ln N$ time]
still holds). [See also R.Agrawal\&C.Burrus: NTTs to implement fast digital
convolutions, ProcIEEE 63 (1975) 550; articles by H.Nussbaumer on "Fermatrix"
and "Mersenne" transforms, IBMJR\&D 21 (1976) 282 and 498.]

These FFFFTs are of use in applications where it is desirable
to completely eliminate roundoff error and floating point operations,
e.g. all-integer convolutions. However, as you can see, NTTs have severe
word length and transform length limitations; the need for high precision
modular arithmetic can be a major stumbling block. On the other hand,
multiplications by W=2 are easy, while modular arithmetic modulo a
Fermatrix number is not that hard. Thus using N=16, modulo 65537 arithmetic,
W=2 [left shift and modulo], Winverse=32769 [right shift; modular
addition correction if inexact], and all numbers in 0..89 allows
computation of CC(16) in 16*, many bit shifts and additions.

Rabiner,Schafer,Rader: The Chirp-Z transform and its Applications,
BSTJ 48,3 (1969) 1249-1292 show how DFTs (for any N) may be calculated in
$N\ln N$ time by using fast convolutions; the method also works for an extension
of FFTs (to W=any complex number, not just the principal Nth root of unity):
\begin{equation}
    A_k  =  \sum_{j=0}^{N-1}a_j*W^{k*j}  \mbox{ for k=0..N-1}
\end{equation}
may be calculated in $N\ln N$ time by a fast convolution by the "Chirp-Z
transform" identity
\begin{equation}
    A_k  = W^{k*k/2} \sum_{j=0}^{N-1} W^{-((j-k)^2)/2} W^{j*j/2}b_j
\end{equation}

Aho,Stieglitz,Ullman: Evaluating Polynomials at fixed sets of points,
SIAMJComp 4,4 (Dec 1975) 533-539, demonstrate that a polynomial and all its
derivatives at one point (or equivalently, an origin shift of an Nth
degree polynomial) may be calculated in $N\ln N$ time by a fast convolution
via the (binomial theorem) identity
\begin{eqnarray}
    \sum_{j=0}^{N-1} c_j*(x+q)^j  & = & \sum_{r=0}^{N-1} x^r * d_r/r!
  \mbox {  where}\\
    d_r & = & \sum_{j=r}^{N-1} c_j*j! * q^{j-r}/(j-r)!
\end{eqnarray}

Some other applications of FFTs are:

Fast multiplication and division of N digit integers may be done in
(roughly) $N\ln N$ time by using fast convolutions followed by a carry step.
(See Aho-Hopcoft-Ullman: Design and Analysis of Computer algorithms, for
further discussion.)

Base conversion of an N digit number may be done in $N(\ln N)^2$ time by
divide and conquer (convert the left and right half of the number recursively,
then do a fast multiplication and addition to combine them).

Fast polynomial multiplication and division by fast convolutions in $N\ln N$
time are also discussed in AHU. (This may also be done for Chebyshev
series...) Given the N roots of a polynomial, you can find its
coefficients (as Chebyshev or as regular) in $N(\ln N)^2$ time by fast
polynomial multiplications on a binary tree. On the other hand,
you can perform a "root squaring" transformation on a polynomial
in (Chebyshev or power form)
\begin{equation}
   P(y) = -P(x)*P(-x)  \mbox{ where }y = x^2
\end{equation}
in $N\ln N$ time by a fast multiplication, or alternatively can implement
a Henrici-Gargantini or Korsak-Pease (or other) simultaneous all root iteration
step, in $N(\ln N)^2$ time by a fast multipoint evaluator, see below.

All shifted correlations of vectors (and/or autocorrelations) may be
calculated in $N\ln N$ time by fast convolutions; this has application in
signal processing, 1D pattern recognition, Electrical engineering.

Fast polynomial multiplication/division/remaindering and a divide and
conquering of the Lagrange interpolation formula may be used to do fast
Nth degree polynomial interpolation and N-point evaluation, as was
shown by Borodin\&Moenck [JCompSystSci 1974]. I have extended B\&M's
results to Chebyshev polynomials and less successfully to other
polynomials.

Fast algorithms exist for power-series to continued fraction interconversion;
these may also be generalized to Chebyshev series.

Fast polynomial evaluation/interpolation at special point sets
(e.g. $Z^k$ for some Z) may be accomplished in $N\ln N$ time by the Chirp-Z and
FF transforms; this also carries over to Chebyshev. Thus fast Taylor
and Chebyshev series calculations.

Fast composition of Taylor series -
$O((N\ln N)^{3/2})$ is also possible, as was discovered by Brent\&Kung,
via a "block Horner" approach. This may also be extended to Chebyshev.

Fast Elliptic linear PDE solvers (by finite differneces or spectrally):
there are many schemes based on FFTs that run in $N\ln N$ time, N=size of
output.

A complete list of FFT applications is far too huge to discuss here...


\section{Modelization of a DC-DC converter}
In the system we want to simulate, as in many real-life systems, is present a DC-DC converter, which will have its non-idealities too, resulting in energy losses.

This means, while in an ideal DC-DC converter the input and output power will be equal ($P_{in}=P_{out}$), in a real one the output power will be scaled by a factor $\eta$, and thus it will be $\eta P_{in}=P_{out}$. $\eta$ will be called \emph{efficiency} of the DC-DC converter. in order to simulate the blocks into SimuLink, we wanted to find a good expression approximating the power loss due to the non-unary efficiency ($P_{loss}$). The chosen analytical expression to fit on the $P_{loss}$ data has been a quadratic one:
$$
P_{loss}=c_1I_{out}^2+c_2I_{out}+c_3
$$

As in the previous part, the first work has consisted in extrapolating data from a graph present into the datasheet of the converter (\emph{LTC3789}), in order to obtain a current-efficiency curve. It has been important to notice that the data were in logarithmic scale on the X axis, and so have had to be interpolated as exponents and transformed into linear values post-audit.

After extracting data from the graph and linearly interpolating it, in order to fit the analytical onto the $P_{loss}$ data it has been needed to use the relationship (which is valid by definition of $\eta$) between $\eta$ and $P_{loss}$:
\begin{gather*}
\eta=\frac{P_{out}}{P_{in}} \\
\Downarrow \\
\eta=\frac{P_{out}}{P_{out}+P_{loss}}
\end{gather*}

Instead of reverting this equation and doing a two-step process to fit the curve, again the \emph{fmins} has been used to evaluate fitting between the available $\eta$ data and the new $P_{loss}$ data. The handle passed to \emph{fmins} calls a function which evaluates the expected $\eta$ vector starting from a sample $P_{loss}$ vector, compares it with the actual $\eta$ vector and returns the squared difference (i.e. the error) between the two. Minimizing this cost function is equivalent to fitting the $P_{loss}$ curve to the desired one.

Again, the result has been satisfactory enough.

\begin{center}
\begin{tabular}{|c| c|}

    \hline
    Parameter name & Parameter value \\
    \hline
    $c_{1}$ & 0.0309 \\
    $c_{2}$ & 0.1166 \\
    $c_{3}$ & 0.4535 \\
    \hline
\end{tabular}

\begin{figure}[h]
  \centering
\includegraphics[width=2.5in]{fitted_eta}
\caption{Current-Efficiency curve as computed after fitting $P_{loss}$}
\end{figure}
\begin{figure}[h]
  \centering
\includegraphics[width=2.5in]{ploss}
\caption{Obtained Current-Power loss curve for the converter}
\end{figure}

\end{center}


% $Id$
%
% Author: David Fournier
% Copyright (c) 2008 Regents of the University of California
%

\section{The necessity for adjoint code}

When you write code for variable objects in \ADM, all the
derivatives are calculated for you in a transparent manner.
To accomplish this, \ADM\ must save certain information for
later use. We shall refer to this as derivative information.
Each arithmetic operation generates about 32~bytes of
derivative information.  If you have some simple function that
has 20 arithmetic operations, it will therefore
generate 640 bytes of derivative information every time
it is called. The purpose of writing adjoint code is to
reduce the amount of derivative information that must
be calculated. For a function that is called many times,
this can greatly reduce the amount of derivative information that
must be stored.

In this chapter, we investigate how to write and debug
adjoint code. To begin, we investigate how to write adjoint code for a
simple function that takes 1--4 independent variables and returns 1~dependent variable.
The adjoint code for such functions is
simpler to write than that for a general function---such as the
singular value decomposition of a matrix, which we will consider later.

\section{Writing adjoint code: a simple case}

Consider a simple function $f$ which takes 1~independent variable $x$
and returns a dependent variable $y$, i.e.,
$$y=f(x)$$
where $f(x)=\exp(-x^2/2)$. The code for this example can be written like
\begin{lstlisting}
  dvariable errf(const prevariable& x)
  {
    return exp(-0.5*square(x));
  }
\end{lstlisting}
There are three arithmetic operations here: square, multiplication, and exponentiation,
so 96 bytes of derivative information will be generated. (Actually, the \texttt{return} operation
also generates 32 bytes of derivative information, but we will ignore that for now.)
A less efficient way to write the code (but more useful for
showing adjoint code for this simple example) would be:
\begin{lstlisting}
  dvariable errf(const prevariable& x)
  {
    dvariable y;
    y=exp(-0.5*square(x));
    return y;
  }
\end{lstlisting}
Here is the same code with the derivative calculated by the one line of adjoint
code \texttt{double dfx=-value(x)*value(y);}:
\begin{lstlisting}
  dvariable errf(const prevariable& x)
  {
    dvariable y;
    value(y)=exp(-0.5*square(value(x)));
    double dfx=-value(x)*value(y);
    AD_SET_DERIVATIVES1(y,x,dfx);  // 1 dependent variable
    return y;
  }
\end{lstlisting}
So what is going on here? Consider the line
\begin{lstlisting}
  value(y)=exp(-0.5*square(value(x)));
\end{lstlisting}
The \texttt{value} function returns a constant type, that is, a double,
that has the same value as the corresponding \texttt{dvariable} or prevariable.
In fact, it is the {\it same} object.  That is, it shares the same address,
but the type has been changed to double. So, the above line of
code assigns the value \texttt{exp(-0.5*square(value(x)))} to \texttt{y},
but without generating any derivative code. Similarly, since
the calculations are made on \texttt{value(x)}, these calculations
will not generate any derivative code. So, it is the responsibility of the
programmer to calculate the derivative code and store it where it
can be used later. The line
\begin{lstlisting}
  double dfx=-value(x)*value(y);
\end{lstlisting}
calculates the derivative $f^\prime(x)$
of $y$ with respect to $x$ and stores it with the line of code
\begin{lstlisting}
  AD_SET_DERIVATIVES1(y,x,dfx);
\end{lstlisting}
This code will only generate 32 bytes of derivative information.

\section{Debugging adjoint code: a simple case}

The simplest way to debug the adjoint code is to put your new
function into an \ADM\ template file and use the \texttt{-dd 1}
command line option to call the derivative checker.
\XX{\fontindexentry{tt}{-dd}}{command line argument}
\XX{command line arguments}{\fontindexentry{tt}{-dd}}
\begin{lstlisting}
DATA_SECTION
PARAMETER_SECTION
  init_number x
 !! x=2;
  objective_function_value f;
PROCEDURE_SECTION
  f=square(errf(x));

GLOBALS_SECTION
  #include <admodel.h>
  dvariable errf(const prevariable& x)
  {
    dvariable y;
    value(y)=exp(-0.5*square(value(x)));
    double dfx=-value(x)*value(y);
    AD_SET_DERIVATIVES1(y,x,dfx);  // 1 dependent variable
    return y;
  }
\end{lstlisting}

\section{Adjoint code for more than\br one independent variable}

The following code shows how to write the adjoint code for a function with two
independent variables:
\begin{lstlisting}
DATA_SECTION
  vector lengths(1,10)
  vector ages(1,10)
  !! lengths.fill_seqadd(1,1);
  !! ages.fill_seqadd(1,1);
  !! lengths=sqrt(lengths);

PARAMETER_SECTION
  init_bounded_number linf(0,10)
  init_bounded_number rho(0,1)
  objective_function_value f;
PROCEDURE_SECTION
  for (int i=1;i<=10;i++)
    f+=square(lengths(i)-vb_growth(linf,rho,ages(i)));

GLOBALS_SECTION
  #include <admodel.h>

  dvariable vb_growth(const prevariable& linf, const prevariable& rho,
    double t)
  {
    double clinf=value(linf);
    double crho=value(rho);
    dvariable len;
    value(len)=clinf*(1-pow(crho,t));
    double dflinf=1-pow(crho,t);
    double dfrho=-clinf*t*pow(crho,t-1);
    AD_SET_DERIVATIVES2(len,rho,dfrho,linf,dflinf);  // 2 dependent variable
    return len;
  }
\end{lstlisting}
This approach to writing adjoint code has been implemented for functions of
up to four independent variables.
\begin{lstlisting}
DATA_SECTION
  vector lengths(1,10)
  vector ages(1,10)
  !! lengths.fill_seqadd(1,1);
  !! ages.fill_seqadd(1,1);
  !! lengths=sqrt(lengths);

PARAMETER_SECTION
  init_bounded_number linf(0,10)
  init_bounded_number rho(0,1)
  init_number t0
  init_bounded_number gamma(.1,1.9)
  objective_function_value f;
PROCEDURE_SECTION
  for (int i=1;i<=10;i++)
    f+=square(lengths(i)-vb_growth(linf,rho,t0,gamma,ages(i)));

GLOBALS_SECTION
  #include <admodel.h>

  dvariable vb_growth(const prevariable& linf, const prevariable& rho,
    const prevariable& t0, const prevariable gamma,double t)
  {
    double clinf=value(linf);
    double ct0=value(t0);
    double crho=value(rho);
    double cgamma=value(gamma);
    dvariable len;
    value(len)=pow(clinf*(1-pow(crho,t-ct0)),cgamma);
    double tmp=cgamma*pow(clinf*(1-pow(crho,t-ct0)),cgamma-1);
    double dflinf=tmp*(1-pow(crho,t-ct0));
    double dft0=tmp*(clinf*log(crho)*pow(crho,t-ct0));
    double dfrho=-tmp*clinf*(t-ct0)*pow(crho,t-ct0-1);
    double dfgamma=value(len)*log(clinf*(1-pow(crho,t-ct0)));
    AD_SET_DERIVATIVES4(len,t0,dft0,rho,dfrho,linf,dflinf,gamma,dfgamma);
       // 4 dependent variable
    return len;
  }
\end{lstlisting}

\section{Structured calculation of derivatives in adjoint code}

Until now, we have deliberately calculated the derivatives with respect to the independent
variables in an {\it ad-hoc} fashion. While this approach works for simple functions,
it rapidly becomes untenable when the function is more complicated. In the following example,
we have calculated the derivatives in a more structured fashion. Notice that to calculate the
derivatives, every line of code in the function is repeated in the opposite
order (commented out, of course) and the corresponding derivatives are calculated.
\begin{lstlisting}
DATA_SECTION
  vector lengths(1,10)
  vector ages(1,10)
  !! lengths.fill_seqadd(1,1);
  !! ages.fill_seqadd(1,1);
  !! lengths=sqrt(lengths);

PARAMETER_SECTION
  init_bounded_number linf(0,10)
  init_bounded_number rho(0,1)
  init_number t0
  init_bounded_number gamma(.1,1.9)
  objective_function_value f;
PROCEDURE_SECTION
  for (int i=1;i<=10;i++)
    f+=square(lengths(i)-vb_growth(linf,rho,t0,gamma,ages(i)));

GLOBALS_SECTION
  #include <admodel.h>

  dvariable vb_growth(const prevariable& linf, const prevariable& rho,
    const prevariable& t0, const prevariable gamma,double t)
  {
    double clinf=value(linf);
    double ct0=value(t0);
    double crho=value(rho);
    double cgamma=value(gamma);
    dvariable len;
    double u1=pow(crho,t-ct0);
    double u2=clinf*(1-u1);
    value(len)=pow(u2,cgamma);
    double dflen=1.0;
    //value(len)=pow(u2,cgamma);
    double dfu2=dflen*cgamma*pow(u2,cgamma-1.0);
    double dfgamma=dflen*value(len)*log(u2);
    //double u2=clinf*(1-u1);
    double dflinf=dfu2*(1-u1);
    double dfu1=-dfu2*clinf;
    //double u1=pow(crho,t-ct0);
    double dfrho=dfu1*(t-ct0)*pow(crho,t-ct0-1.0);
    double dft0=-dfu1*u1*log(crho);

    AD_SET_DERIVATIVES4(len,t0,dft0,rho,dfrho,linf,dflinf,gamma,dfgamma);
      // 4 dependent variable
    return len;
  }
\end{lstlisting}

\section{General adjoint code}

So far, the adjoint code has been for a simple function which has from~1 to~4
independent variables and returns 1~dependent variable.
Now we consider the general case where the function can take
any number of dependent variables and return any number of
dependent variables, and these variables can be in the form of numbers,
vectors, or matrices.
\begin{lstlisting}
 dvar_vector  operator * (_CONST dvar_matrix& m,_CONST dvar_vector& x )
 {
   if (x.indexmin() != m.colmin() || x.indexmax() != m.colmax())
   {
     cerr << " Incompatible array bounds in dvar_vector  operator * "
            << "(_CONST dvar_matrix& m,_CONST dvar_vector& x)\n";
     ad_exit(21);
   }

   dvar_vector tmp(m.rowmin(),m.rowmax());
   double sum;

   for (int i=m.rowmin(); i<=m.rowmax(); i++)
   {
     sum=0.0;
     for (int j=x.indexmin(); j<=x.indexmax(); j++)
     {
       sum+=(m.elem(i)).elem_value(j)*x.elem_value(j);
     }
     tmp.elem_value(i)=sum;
   }
  save_identifier_string("PLACE4");
  x.save_dvar_vector_value();
  x.save_dvar_vector_position();
  save_identifier_string("PLACE3");
  m.save_dvar_matrix_value();
  m.save_dvar_matrix_position();
  save_identifier_string("PLACE2");
  tmp.save_dvar_vector_position();
  save_identifier_string("PLACE1");

  ADJOINT_CODE(dmdv_prod);
   return(tmp);
 }
\end{lstlisting}
To calculate the adjoint code, it will be necessary to have the
values of the matrix \texttt{m} and the vector \texttt{x}.
This is accomplished with the instructions
\begin{lstlisting}
  x.save_dvar_vector_value();
  m.save_dvar_matrix_value();
\end{lstlisting}
Also, to calculate the derivatives, it will be necessary to know where the
derivatives with respect to the independent and dependent variables are
located. This information is saved with the instructions
\begin{lstlisting}
  x.save_dvar_vector_position();
  m.save_dvar_matrix_position();
  tmp.save_dvar_vector_position();
\end{lstlisting}
Finally, we need to save the name of the routine that calculates
the adjoint code, so that it can be called at the appropriate time. To write the
code for the adjoint calculations, keep in mind that everything must be recovered
from the stack in the reverse order from which it was put on the stack.
This process can be a bit confusing and if you don't do it properly, the
stack will become corrupted and nothing will work.  To help diagnose
problems, function \texttt{save\_identifier\_string} can be
used to put a string on the stack. This string value can be checked in the
adjoint code with the \texttt{verify\_identifier\_string} function. At least two of
these functions should be left in any adjoint code, so that stack
integrity can be monitored if problems show up later. For optimized code,
they are not used, and so contribute almost nothing to the overhead.

The adjoint code begins by reading the information that was saved on the stack.
An object of type \texttt{dvar\_vector\_position} contains both the size and
address information associated with a \texttt{dvar\_vector}---which are needed to
recover or store derivative values, or to build a \texttt{dvector} with the
same shape as the {\t dvar\_vector}.
The function \texttt{restore\_dvar\_vector\_der\-ivatives} gets the values of the
derivatives with respect to the dependent variables, so they can be used in the
adjoint code. The functions
\begin{lstlisting}
   dfx.save_dvector_derivatives(x_pos);
   dfm.save_dmatrix_derivatives(m_pos);
\end{lstlisting}
use the position information to save the derivatives with respect to the
independent variables in the appropriate places.
\begin{lstlisting}
void dmdv_prod(void)
{
  verify_identifier_string("PLACE1");
  dvar_vector_position tmp_pos=restore_dvar_vector_position();
  verify_identifier_string("PLACE2");
  dvar_matrix_position m_pos=restore_dvar_matrix_position();
  dmatrix m=restore_dvar_matrix_value(m_pos);
  verify_identifier_string("PLACE3");
  dvar_vector_position x_pos=restore_dvar_vector_position();
  dvector x=restore_dvar_vector_value(x_pos);
  verify_identifier_string("PLACE4");
  dvector dftmp=restore_dvar_vector_derivatives(tmp_pos);

  dmatrix dfm(m_pos);
  dvector dfx(x_pos.indexmin(),x_pos.indexmax());
  dfm.initialize();
  dfx.initialize();

   double dfsum;
   for (int i=m.rowmax(); i>=m.rowmin(); i--)
   {
     // tmp.elem_value(i)=sum;
     dfsum=dftmp.elem(i);
     for (int j=x.indexmax(); j>=x.indexmin(); j--)
     {
       //sum+=(m.elem(i)).elem_value(j)*x.elem_value(j);
       dfm.elem(i,j)+=dfsum*x.elem(j);
       dfx.elem(j)+=dfsum*m.elem(i,j);
     }
     //sum=0.0;
     dfsum=0.0;
   }
   dfx.save_dvector_derivatives(x_pos);
   dfm.save_dmatrix_derivatives(m_pos);
}
\end{lstlisting}


\section{Extension to Higher Dimensions}\label{sec:mud-higher-dimensions}
We now show that when the dimension of $\pspace$ is higher, the benefit from maximizing the dimension of the QoI map is even more considerable than what we observed in the two-dimensional cases, as long as we aggregate data into components of the QoI map in a purposeful manner.
In the first example, we demonstrate the impact of increasing the volume of the parameter space while fixing all other attributes of the inverse problem.
In the second example, we illustrate how we could utilize the solution to the SIP for the two-dimensional case from \ref{subsec:pde-example} to improve the MUD estimate in five dimensions.

\subsection{A SIP with a Naive Initial Density}
Consider a naive implementation in five dimensions, where the same prior knowledge on the bounds of $g$ inform the choice of a parameter space.
Suppose $\pspace$ is given by $[-4, 0]^5$, induced by five regularly-spaced knot points in the interior of $x_2 \in (0,1)$ for which a uniform density is assumed in each component.
In Figure~\ref{fig:pde-highd-initial-5d}, we show what one-thousand such initial functions look like, and note that many of them appear to exhibit fluctuating behavior which may require a finer mesh than the one being used to solve the problem ($36\times36$), in order to reduce numerical errors.
This choice of initial density induces a lot of ``conceptual'' noise into the SIP, since ``common sense'' from a modeler's perspective could rule out functions which zig-zag excessively.

\begin{figure}
\centering
  \includegraphics[width=0.475\linewidth]{figures/pde-highd/pde-highd_init_D5.png}
\caption{
One thousand initial parameter samples (our model evaluation ``bugdet'') were used to estimate $g$, constructed by taking independent uniform samples from $[-4, 0]$ for each direction are shown in purple.
}
\label{fig:pde-highd-initial-5d}
\end{figure}

We solve the SIP for both scalar-- and vector--valued QoI maps.
The latter is constructed with horizontal bands\---which correspond to the partitioning of $\Omega$ into components of the QoI map\---shown in the left of Figure \ref{fig:pde-highd-5d-example}.
We refer to this map as $\qoi_\text{5D}$, and $\qoi_\text{1D}$ will again represent the scalar--valued map.
In the right half of Figure~\ref{fig:pde-highd-5d-example}, the MUD solutions for $\qoi_\text{5D}$ and $\qoi_\text{1D}$ are shown in parameter space for a representative SIP.
The scalar-valued MUD misidentifies the location of $g$'s minimum value, but the vector-valued QoI is able to resolve the general qualitative behavior.

\begin{figure}
\centering
  \includegraphics[width=0.45\linewidth]{figures/pde-highd/pde-highd_sensors_D5}
  \includegraphics[width=0.45\linewidth]{figures/pde-highd/pde-highd_comp_exmud_D5_m100}
\caption{
(Left): Layout for 5-D vector--valued map and comparison of the two MUD solutions in parameter space.
(Right): Example MUD solutions for $\qoi_\text{5D}$ and $\qoi_\text{1D}$.
}
\label{fig:pde-highd-5d-example}
\end{figure}

In Figure~\ref{fig:pde-highd-5d-mud} we plot the results from twenty repeated trials (perturbations of noise) when using all $100$ measurements.
We observe, as before, the same difference in going from scalar-- to vector--valued solutions that we saw in two dimensions in Fig.~\ref{fig:pde-MUD}.
In the left half of Figure~\ref{fig:pde-highd-5d-mud}, the scalar-valued QoI is unable to differentiate between resolving residual discrepancies in different locations in $\Omega$.
By contrast, the vector-valued QoI shown to the right is constructed with respect to the flow of information in the system, and so many more of the twenty trials land closer to the true minimum value of $g$.
The solutions for the vector-valued approach instead explore the available knots (at $x_2=1/6$ and $1/3$), nearest the actual minimum value of $2/7$ instead, a much more valuable area of $\Lambda$ to explore.

\begin{figure}
\centering
  \includegraphics[width=0.45\linewidth]{figures/pde-highd/pde-highd_pair_D5-1_m100}
  \includegraphics[width=0.45\linewidth]{figures/pde-highd/pde-highd_pair_D5-5_m100}
\caption{ SIP solutions using $\qoi_\text{5D}$ and $\qoi_\text{1D}$ for twenty realizations of noise polluting the hundred measurements used to construct the map.
(Left): Scalar-valued solutions.
(Right): Vector-valued solutions.
}
\label{fig:pde-highd-5d-mud}
\end{figure}

Recall from \ref{subsec:pde-example} that we previously solved a two-dimensional version of this problem.
We leverage the effort involved in solving this first problem in order to better refine the approximation of $g$ in \ref{sec:mud-pde-sequence} below.
By using the former SIP solution results to define a much smaller region of $\pspace$ to explore, the results from the second SIP can improve considerably.


\section{Definition of a battery model}

In order to define a good model for Li-Po battery, comprehending also a good approximation of all its non-idealities, a two-branches model has been used.
In the first part, current SoC is computed, by taking as a reference electrical model a current generator and a capacitor. The discharge current $I_{b}$ charges the capacitor, of capacity $C_{m}$ (representing the total capacity of the battery), producing an output voltage $V_{SOC}$ which represents the SoC.

The output equation of the circuit is thus $V_{SOC} = V_{SOC}(0)+\int\!\frac{I_{b}}{C_{m}}dt$, which is equivalent to the SoC expression for the battery.

The second part of the model is composed by a voltage generator and a series resistance, as in a usual models of voltage sources. The big difference from standard models is that both the voltage and the series resistance of the battery depend on the current state of charge, and are thus functions of $V_{SOC}$.

The output of this second circuit is the true battery voltage as seen by the load.

\begin{figure}[h]
  \centering
  \includegraphics[width=3in]{circuit}
  \caption{Equivalent circuit of the Li-Po battery}
\end{figure}

While the first part of the circuit is well-defined by the initial and total capacity of the battery, in the second part the two functions $V(SoC)$ and $R(SoC)$ has to be extracted from available battery data. In this case, the available source of information contained into the battery's datasheet has been the graph showing output voltage ($V_{B}$) as a function of both discharge capacity - which is, after fixing a maximum capacity value, easily convertible to SoC - and load current. The latter dependance is due to the series resistance of the battery.

Points from these curves have been extracted using a figure digitizer tool into MatLab and have been converted to different Soc-voltage tables. In order to find a good value for R two curves have been necessary (as R affects only the $V_{B}$-$I_{load}$ relationship). To minimize sampling errors, the highest and lowest currents' curves have been digitalized. Finally, the saved data have been interpolated in a linear fashion to obtain a continous sequence of equally-spaced points on the graph.

From these data, $V(Soc)$ and $R(Soc)$ values can be easily obtained by solving the second circuit. Although at this point a model of the two circuits has been made, having the values stored as a lookup table is not the best choice, both because of the space occupation which would not scale for bigger graphs, and because of the missing smoothness of the obtained relationship, which is typical of a real-life system. In order to obtain a smoother and simpler relationship, the data have been fitted using MatLab $fmins$ function, in order to have an analytical representation for them:
\begin{align*}
V(SoC)&=b_1e^{(b_2SoC)}+b_3SoC^4+b_4SoC^3+b_5SoC^2+b_6SoC+b_7 \\
R(SoC)&=b_8e^{(b_9SoC)}+b_{10}
\end{align*}

The curve fitting produced a satisfactory result:

\begin{center}
\begin{tabular}{|c| c|}

    \hline
    Parameter name & Parameter value \\
    \hline
    $b_{1}$ & 1.3518 \\
    $b_{2}$ & -0.7752 \\
    $b_{3}$ & 0.4669 \\
    $b_{4}$ & -1.0723 \\
    $b_{5}$ & 0.8729 \\
    $b_{6}$ & -0.0031 \\
    $b_{7}$ & 2.8434 \\
    $b_{8}$ & 0.0000 \\
    $b_{9}$ & 9.7875 \\
    $b_{1}$ & 0.1909 \\
    \hline
\end{tabular}

\begin{figure}[h]
  \centering
\includegraphics[width=3in]{fitted_r_curve}
\caption{$SoC-R$ curve after fitting}
\end{figure}

\begin{figure}[h]
  \centering
\includegraphics[width=4in]{fitted_vsoc_curve}
\caption{$SoC-V$ curve after fitting}
\end{figure}

\end{center}


\chapter{R Background}
This chapter describes the R programming techniques that we will need in order
to do Bayesian statistics both within R and also by using JAGS.

\section{Vectors}
We will use vectors frequently throughout the course. A vector can be thought
of as a list of numbers. In R, you can create a vector using the function
{\tt c}, which stands for ``concatenate''.
\begin{verbatim}
my_vector = c(12.4, -6.2, 4.04)
\end{verbatim}
You can then examine the contents of the vector by typing its name.
\begin{verbatim}
> my_vector
[1] 12.40 -6.20  4.04
\end{verbatim}
There are various helpful things you can do with vectors. You can get the length
of a vector (the number of elements in it) like so:
\begin{verbatim}
> length(my_vector)
[1] 3
\end{verbatim}
You can do arithmetic
with vectors too, if they're the same length. For example:
\begin{verbatim}
> c(1,2,3)*c(2,5,3)
[1]  2 10 9
\end{verbatim}
If you wanted, you could assign the output to a new variable:
\begin{verbatim}
> x = c(1,2,3)
> y = c(2,5,3)
> z = x*y
> z
[1]  2 10 9
\end{verbatim}
It is important to understand how to access subsets (or ``slices'') of vectors
based on which elements satisfy a certain condition. Here is an example:
\begin{verbatim}
> x = c(1, 2, 3, 0, 10)
> test = x > 3
> test
[1] FALSE FALSE FALSE FALSE  TRUE
> y = x[x <= 2]
> y
[1] 1 2 0
> z = sum(x[x <= 2])
> z
[1] 3
\end{verbatim}
This kind of thing will be used frequently in the computational parts of the
course.

\section{Lists}
Lists are a bit like vectors, in that they can contain a lot of information in
a single variable. But instead of just being numbers, lists can contain all
sorts of things. For example, suppose I want a variable/object in an R program
to represent a person. A person can have a name and an age. Here is how to make
a list:
\begin{verbatim}
a_person = list(name="Nicole", age=21)
\end{verbatim}
If you needed to extract certain elements from a list, you can do it using the
\$ operator in R. For example suppose I wanted to extract the {\tt name} variable
from within {\tt a\_person}. I could do this:
\begin{verbatim}
> a_person$name
[1] "Nicole"
\end{verbatim}
and voila. When we use JAGS, a data set will be represented using a list.
So will our JAGS output.

If you have ever learned C, C++, or Matlab, a list in R is basically the same thing as
a ``struct'' in these languages.
If you have learned C++, Java, or Python, a list is
like a ``class'' but with just variables and no functions.
In Python, ``dictionaries''
are also very similar to R lists.

\section{Functions}
In this course you'll need to be able to read and understand simple R
functions, and perhaps write a few. A function is like a machine that takes
an input, does something, and returns an output. You have probably used many
built-in R functions already, like {\tt sum()}.

\begin{verbatim}
# Defining a function called my_function
my_function = function(x)
{
  # Do some stuff
  result = 3*x + 0.5
  return(result)
}
\end{verbatim}

\section{For Loops}
For loops are mostly used to repeat an action many times. Here is an example.
\begin{verbatim}
N = 100
for(i in 1:N)
{
  print(i)
}
\end{verbatim}

\section{Useful Probability Distributions}
Since R is a statistics program, it knows about a lot of probability
distributions already. So, if I wanted to use the probability density function
of a normal distribution, instead of having to code something like this:
\begin{verbatim}
f = exp(-0.5*((x - mu)/sigma)**2)/(sigma*sqrt(2*pi))
\end{verbatim}
I can just use the built-in function {\tt dnorm}.
\begin{verbatim}
f = dnorm(x, mean=mu, sd=sigma)
\end{verbatim}
Much easier! If, instead of wanting to evaluate the PDF, I wanted to generate
random samples from a normal distribution, I could use {\tt rnorm}.
\begin{verbatim}
# Generate 1000 samples
samples = rnorm(1000, mean=50., sd=10.)
\end{verbatim}


\documentclass{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{epstopdf}
\usepackage{inputenc}
\usepackage{geometry}
\usepackage{cancel}
\geometry{left=2.5cm,right=2.5cm,top=2.5cm,bottom=2.5cm}

\begin{document}

	\title{MITx 15.455x Mathematical Methods for Quantitative Finance
	 \\
	\begin{large}
		Recitation 3
	\end{large} }

	\maketitle

	\section*{Module 3: Forecasting}

Let's talk about forecasting.
We saw in lecture that forecasting
is an application of conditional probability to time series
models.
So we think of the time series as generating the data.
And we ask at a given point in time,
given all the observations that exist up
through that point in time, what can we say about the future?
We might want to know what happens one time step ahead,
two time steps ahead, or in limit
as the number of times that goes to infinity,
but the key thing is the break between past and future.
So we take the present time to be just after we've
made our final observation.
And we're first forecast period is one step ahead.
And, as I said, the key tool is in addition to
our previous ones with expectations,
linearity, algebraic substitution, recursion.
We had a new one, which is conditional probability.
In this setting, conditional probability simply
means that for any variable that was
a random variable in the past that has been realized,
it's no longer a random variable.
Now it's a scalar.
Now it's a number.
So those things that show up in the recursion
that we previously thought of as random variables,
in this setting, when we take conditional expectations
of our defining equations or equations of evolution,
then they take a different character,
because they're already known.
They're no longer random variables.
So we looked at an example in lecture.
Let's just do a couple of examples together.
It's the same structure.
They're just one or two things it might be worth
paying attention to as you look at the blind forecast
of your own.
Some of the things that might have look special for the case
of the AR1 are quite general.
So let's take a look at a model.

\subsection*{ARMA(1,1)}

Let's look at, say, an ARMA(1,1)  which is this form:

$$ x_t = c_o + c_1 x_{t-1} \sigma z_t + \phi_1 z_{t-1} $$

So I'm going to keep the notation $\sigma z_t$ for our shock for innovation in the period.
You could give it another name for its coefficient.
Remember that $\sigma$, in this case,
is not the standard deviation of $x$.
It's just a scaling parameter for the random shock $z$.

So the first thing we'd like to do
is compute the mean value for $x$ and simplify our expression.
So we use our usual trick.
$\mu$ is the name will give the expectation of $x_t$.
And then we take expectations on the right hand side
and apply linearity.
So we have this is going to be:

$$ \mu = E[x_t] = c_0 + c_1 E[x_{t-1}] + 0 + 0 $$

$$ \mu = \frac{c_0}{1-c_1} $$

And you should recognize that expression from our AR 1 model.
It's exactly the same expression.
And therefore, we can substitute and rewrite our equation
as a bunch of things that are grouped together in such a way
that they all have 0 mean.
That is to say, if I substitute for $c_0$
and say it's equal to $\mu (1 - c_1)$,
then I can write this as:

$$ x_t = \mu (1 - c1) + c_1 x_{t-1} + \sigma z_t + \phi_1 z_{t-1} $$
$$ x_t - \mu =   c_1 (x_{t-1} - \mu)+ \sigma z_t + \phi_1 z_{t-1} $$

Now the next thing we want to do is put things
in forecasting form.
So what I'm going to do is I'm going to shift $t$ to $t+1$.
Everywhere where I see a $t$, I'm going to write $t+1$.
And that puts the future values on the left
and the present and past values or the known
values on the right hand side.
So for one-time step ahead, this makes exactly the split
that we want to have:

$$ x_{t+1}  =  \mu + c_1 (x_{t} - \mu)+ \sigma z_{t+1} + \phi_1 z_{t} $$

So what's our forecast?
So our definition for our forecast
is going to be the conditional probability.
So our first forecast is going to be:

$$f_{t,1} = E[x_{t+1}|x_t, z_t, \dots] $$

That is the forecast made at time t for one-time step
ahead in the future.
And there are different notations for this.
So what you really should rely on is not the notation,
but reduce everything to expectations.
So this is the expectation of $t+1$ given everything
up through that point, which includes $x_t$, $z_t$,
and any previous failures, but those
don't show up in this equation on the right-hand side.
So let's compute the expectation.
And what we noticed is that the only random variable
on the right hand side is $z_{t+1}$.
That's the only thing that's random.
So $x_t$ is known.
This $z_t$ is known.
There's only one random variable, i.e. $z_{t+1}$ .
And we take expectations, the expectation of $z_{t+1}$
is going to vanish.
So what do we get?
We're going to get that the forecast is equal to:

$$f_{t,1} =\mu + c_1 (x_{t} - \mu) + \phi_1 z_{t}  $$

So we just do the calculation.
It's exactly the same as the expression
above, except for this $\sigma z_{t+1}$, this random term
that dropped out, because it has zero expectation.
So let's take a look at the forecast error.
So the first forecast error for one-time step
ahead is going to be:

$$e_{t+1} = x_{t+1} - f_{t,1} $$

That is the forecast error as defined
as the difference between what we predicted
and what actually happened.
What we'd like to do and the way in which
we have this definition for the forecast being optimal
is we're going to minimize the mean squared forecast error.
That is going to minimize $e^2$
in expectation of all possible things that would happen.
Now we could use other kinds of loss functions.
Those will depend on the settings,
depending on the economic values.
So it's just a question of doing and minimization,
but in this case, we have this basic result
that this will be our expectation,
but let's compute the forecast error.
So the idea is that what we really want, in general, if you
want to drive this and check, what we really want
is we want a predictor for $f$, that some function of all
the previous observations, in this case, just $x_t$ and $z_t$,
we'd like to be a linear function.
And when we find the linear function that
minimizes the mean squared error,
we find that it's this conditional expectation.
So let's compute this quantity.
And we'll see that there's a really nice structure that
shows up.
$x_{t+1} - f_{t,1}$, you
notice that most of the terms are common.

And therefore, the mean squared forecast error
is:
$$E[e_{t+1}^2] = E[ x_{t+1} - f_{t,1} ] = E[(\sigma z_{t+1})^2] =\sigma^2$$

Now what about looking at multiple forecasts?
So we'd like to go for future horizons.
One-step was easy.
So the general rule is that when we
want to do it two-step ahead forecast,
we're going to shift everything.
We're going to add one more.
So we write an expression for $x_{t+2}$.

So here's the general procedure.
Keep doing the recursion.
Keep substituting in back until all of the $X$'s on the right
hand side of the equation have a time index of $t$ or earlier.
The $z$'s can be later.
That's OK, because we're going to take their expectations,
and they're going to vanish.
So we might have some possibly unknown $z$'s.
But the basic idea is once we go beyond one-time step,
how do you know what to do?
It's easy.
You're doing $x_t+h$ for horizon $h$
on the left hand side.
And on the right hand side, you do probably $h$ or $h-1$
recursive substitutions of the defining equation
until we can express $x_t$ plus $h$ in terms of $x$, $x_{t-1}$
minus 1, and so on.
Those are all known quantities.
Then we take conditional expectations.
We have our forecast.
We compute the forecast errors.
We take expectations of their squares.
That's easy as well, because it's just the same quantities.
Now we have the known and the unknown ones
clearly delineated.
And we get our results for our mean squared forecast error.
And there are two really important properties
to keep in mind for our solution.
One of them is the forecast error has zero expectation.
And the forecast error is orthogonal
to the other variables, to the other  to the other predictor variables.

\subsection*{ARMA(2,2)}

So let's take a look for another example
Let's take a look at an ARMA(2,2), shall we?
So now when you're given numbers for the forecasts
or particular horizons, you can either plug them in right away
or you can leave the parameters general.
That generally makes it easier to check your math
and find any sign errors that might be there,
but either way, whether you substitute indefinite numbers
for the parameters before or after shouldn't
make any difference.
When you're doing forecast, you do want to pay attention though
to the initial conditions if you have to bootstrap your process,
because if you're asked for a forecast one or three
or seven steps ahead, you'll need enough data to get
the process started before you can generate the recursion.
The recursive techniques that we have
that I'm writing down here for the forecast
are technically appropriate for the case, where we go
infinitely far into the past.
Even if the series did exist an infinitely long time,
we wouldn't have an infinite amount of data.
So we do need to make sure we have the initial conditions set
to get numerical answers, but the basic rules are compute
conditional expectations after writing our defining
equations with variable to be forecast on the left hand side
and only known observations on the right hand side.

So let's do this for ARMA(2,2).
it's little bit more complicated at least initially.
So it looks like:

$$x_t = c_0+c_1 x_{t-1} + c_2 x_{t-2} + \sigma z_t + \phi_1 z_{t-1} + \phi_2 z_{t-2} $$

And you can see how it would go for a general ARMA(p,q).
I'm going to write it in concrete form
so we don't have summations running around.
And when we shift, it gets a little messy
keeping track of the indices, but you can do it.
And you can take a look at the literature as well for that.
So what's our expectation in this case?

$$ \mu = E[x_t] =  c_0+c_1 \mu + c_2 x_{t-2} + \mu  $$
$$ \mu = \frac{c_0}{1-c_1 - c_2}    $$

And you can guess how this generalizes
for the general ARMA(p,q) model, we
get the generalization of this expression.
And then we can write our equation
as:

$$ x_t - \mu = c_1(x_{t-1}-\mu) + c_2(x_{t-2}-\mu) + \sigma z_t + \phi_1 z_{t-1} + \phi_2 z_{t-2} $$

Let's do some forecasting, shall we?

You should notice that each of the terms that I've
written down is a way that it has 0 mean.
That just makes it a bit easier to see what's going on,
to see what the dynamics are, and to do some calculations.
You can also, if you want to clean it up,
you can find a new variable $y$ to be $x_t - \mu$ and shift things
back.
So you can do any rescaling things you like.
The end results are going to be the same if they're just
redefinitions of the parameters and rescalings
of the variables.

So let's take a look for time $t+1$:

$$ x_{t+1} - \mu = c_1(x_{t}-\mu) + c_2(x_{t-1}-\mu) + \sigma z_{t+1} + \phi_1 z_{t} + \phi_2 z_{t-1} $$

So that one is pretty easy.
It looks like the expression that we just
did before for the ARMA(1,1).
That is everything on the right hand side has an index of
all of the $x$'s are taken at time $t$ or $t-1$.
And the $z$'s are mostly in the past.
We have these two with coefficients
of $\phi$'s both in the past.
And this is the only one that's still a random variable,
but at time 2, things change.
So let's look one more time step ahead.
Let's contrast a little bit.
So if we write this as:

$$ x_{t+2} - \mu = c_1(x_{t+1}-\mu) + c_2(x_{t}-\mu) + \sigma z_{t+2} + \phi_1 z_{t+1} + \phi_2 z_{t} $$

So all I've done is I shifted $t$ to $t+1$ again.

So I have two of the terms on the right hand side
involve the present or the past.
And that's OK.
Those are known quantities, but I have these other expressions.
So I can't just directly compute the expectation
on the left hand side.
Now I could take the previous work I did before
and substitute that would be fine.
But if I want to see the general approach for doing it
at a particular time step, what I'm going to say
is that the expressions  $\sigma z_{t+2}$ and $\phi_1 z_{t+1}$ , I can leave alone,
because those are both fine.
These just involves $z$'s.
And I know their expectations, but I
don't want the recursive structure for $x_t+1$, because remember this is saying that the value two
days from now depends on the value tomorrow, which
I haven't yet observed.
So the value today, the value yesterday, those are all known,
but rather than expressing it this way,
I'd like to do my recursions, get it out.
And then I can take clean ordinary simple expectations
and get an answer.
So let's do that.
And we just need to substitute one more time and put that in.
So which term are we going to substitute--
we're going to substitute in?
We'll have :

$$ x_{t+2} = \mu + c_1\Big[  c_1(x_{t}-\mu) + c_2(x_{t-1}-\mu) + \sigma z_{t+1} + \phi_1 z_{t} + \phi_2 z_{t-1} \Big] + c_2(x_{t}-\mu) + \sigma z_{t+2} + \phi_1 z_{t+1} + \phi_2 z_{t} $$

So what have we got?
So I now have an expression for $x_{t+2}$, where on the right hand side,
everything depends on known quantities.
So we put it in.
We turn the crank.
We compute the expectation two-time steps ahead.
That will give us our forecast today for two-time steps ahead.
When tomorrow is realized, that forecast
for what will then be one more day ahead is going to change.
And how will it change?
It will be updated by the new observation.
So if today is Monday and I'd like
to know about Wednesday's weather,
I would make a forecast today.
Tomorrow I'll have a new forecast for Wednesday.
And that will change.
It will change by my knowledge of what happened on Tuesday.
But from a mathematical point of view,
we separated things into the form that we want.
We can first take the expectation
with the appropriate conditions.
And second, we can compute forecast error in advance.
And we can compute the mean squared forecast
two steps ahead.
And so that's looking ahead for what
our expected forecast error.
The reason that that's important is
that we want to think not only about distributions.
What's the exact value?
Because that exact value probably won't be realized.
It gives us characteristics of the full distribution
of the actual outcome.
And it lets us have a sense as to how we
should evaluate our forecasts.
So that's an expectation.
Once Wednesday is realized, then we'd like to go back.
And then our Xt plus 2 will be a known quantity.
We can compare it with our forecast.
And, of course, this will differ by some amount,
but if we'd like to improve our forecasting techniques,
our forecasting methodology, and our forecasting quality,
what we do is we look over time.
We take a large collection of forecasts and outcomes.
And we study the statistics of the forecast errors.
So we use forecast errors both in expectation going forward
and to assess forecast quality and look
for improvements going backward once we've
collected the relevant data.

\end{document}


\chapter{Combinatorial}

\section{Permutations}
	\subsection{Factorial}
		\import{factorial.tex}
		\kactlimport{IntPerm.h}

	\subsection{Cycles}
		Let $g_S(n)$ be the number of $n$-permutations whose cycle lengths all belong to the set $S$. Then
		$$\sum_{n=0} ^\infty g_S(n) \frac{x^n}{n!} = \exp\left(\sum_{n\in S} \frac{x^n} {n} \right)$$

	\subsection{Derangements}
		Permutations of a set such that none of the elements appear in their original position.
		\[ \mkern-2mu D(n) = (n-1)(D(n-1)+D(n-2)) = n D(n-1)+(-1)^n = \left\lfloor\frac{n!}{e}\right\rceil \]

	\subsection{Burnside's lemma}
		Given a group $G$ of symmetries and a set $X$, the number of elements of $X$ \emph{up to symmetry} equals
		 \[ {\frac {1}{|G|}}\sum _{{g\in G}}|X^{g}|, \]
		 where $X^{g}$ are the elements fixed by $g$ ($g.x = x$).

		 If $f(n)$ counts ``configurations'' (of some sort) of length $n$, we can ignore rotational symmetry using $G = \mathbb Z_n$ to get
		 \[ g(n) = \frac 1 n \sum_{k=0}^{n-1}{f(\text{gcd}(n, k))} = \frac 1 n \sum_{k|n}{f(k)\phi(n/k)}. \]

\section{Partitions and subsets}
	\subsection{Partition function}
		Number of ways of writing $n$ as a sum of positive integers, disregarding the order of the summands.
		\[ p(0) = 1,\ p(n) = \sum_{k \in \mathbb Z \setminus \{0\}}{(-1)^{k+1} p(n - k(3k-1) / 2)} \]
		\[ p(n) \sim 0.145 / n \cdot \exp(2.56 \sqrt{n}) \]

		\begin{center}
		\begin{tabular}{c|c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c@{\ }c}
			$n$    & 0 & 1 & 2 & 3 & 4 & 5 & 6  & 7  & 8  & 9  & 20  & 50  & 100 \\ \hline
			$p(n)$ & 1 & 1 & 2 & 3 & 5 & 7 & 11 & 15 & 22 & 30 & 627 & $\mathtt{\sim}$2e5 & $\mathtt{\sim}$2e8 \\
		\end{tabular}
		\end{center}

	\subsection{Lucas' Theorem}
		Let $n,m$ be non-negative integers and $p$ a prime. Write $n=n_kp^k+...+n_1p+n_0$ and $m=m_kp^k+...+m_1p+m_0$. Then $\binom{n}{m} \equiv \prod_{i=0}^k\binom{n_i}{m_i} \pmod{p}$.

	\subsection{Binomials}
		\kactlimport{multinomial.h}

\section{General purpose numbers}
	\subsection{Bernoulli numbers}
		EGF of Bernoulli numbers is $B(t)=\frac{t}{e^t-1}$ (FFT-able).
		$B[0,\ldots] = [1, -\frac{1}{2}, \frac{1}{6}, 0, -\frac{1}{30}, 0, \frac{1}{42}, \ldots]$

		Sums of powers:
		\small
		\[ \sum_{i=1}^n n^m = \frac{1}{m+1} \sum_{k=0}^m \binom{m+1}{k} B_k \cdot (n+1)^{m+1-k} \]
		\normalsize

		Euler-Maclaurin formula for infinite sums:
		\small
		\[ \sum_{i=m}^{\infty} f(i) = \int_m^\infty f(x) dx - \sum_{k=1}^\infty \frac{B_k}{k!}f^{(k-1)}(m) \]
		\[ \approx \int_{m}^\infty f(x)dx + \frac{f(m)}{2} - \frac{f'(m)}{12} + \frac{f'''(m)}{720} + O(f^{(5)}(m)) \]
		\normalsize

	\subsection{Stirling numbers of the first kind}
		Number of permutations on $n$ items with $k$ cycles.
		\begin{align*}
			&c(n,k) = c(n-1,k-1) + (n-1) c(n-1,k),\ c(0,0) = 1 \\
			&\textstyle \sum_{k=0}^n c(n,k)x^k = x(x+1) \dots (x+n-1)
		\end{align*}
		$c(8,k) = 8, 0, 5040, 13068, 13132, 6769, 1960, 322, 28, 1$ \\
		$c(n,2) = 0, 0, 1, 3, 11, 50, 274, 1764, 13068, 109584, \dots$

	\subsection{Eulerian numbers}
		Number of permutations $\pi \in S_n$ in which exactly $k$ elements are greater than the previous element. $k$ $j$:s s.t. $\pi(j)>\pi(j+1)$, $k+1$ $j$:s s.t. $\pi(j)\geq j$, $k$ $j$:s s.t. $\pi(j)>j$.
		$$E(n,k) = (n-k)E(n-1,k-1) + (k+1)E(n-1,k)$$
		$$E(n,0) = E(n,n-1) = 1$$
		$$E(n,k) = \sum_{j=0}^k(-1)^j\binom{n+1}{j}(k+1-j)^n$$

	\subsection{Stirling numbers of the second kind}
		Partitions of $n$ distinct elements into exactly $k$ groups.
		$$S(n,k) = S(n-1,k-1) + k S(n-1,k)$$
		$$S(n,1) = S(n,n) = 1$$
		$$S(n,k) = \frac{1}{k!}\sum_{j=0}^k (-1)^{k-j}\binom{k}{j}j^n$$

	\subsection{Bell numbers}
		Total number of partitions of $n$ distinct elements. $B(n) =$
		$1, 1, 2, 5, 15, 52, 203, 877, 4140, 21147, \dots$. For $p$ prime,
		\[ B(p^m+n)\equiv mB(n)+B(n+1) \pmod{p} \]

	\subsection{Labeled unrooted trees}
		\# on $n$ vertices: $n^{n-2}$ \\
		\# on $k$ existing trees of size $n_i$: $n_1n_2\cdots n_k n^{k-2}$ \\
		\# with degrees $d_i$: $(n-2)! / ((d_1-1)! \cdots (d_n-1)!)$

	\subsection{Catalan numbers}
		\[ C_n=\frac{1}{n+1}\binom{2n}{n}= \binom{2n}{n}-\binom{2n}{n+1} = \frac{(2n)!}{(n+1)!n!} \]
		\[ C_0=1,\ C_{n+1} = \frac{2(2n+1)}{n+2}C_n,\ C_{n+1}=\sum C_iC_{n-i} \]
		${C_n = 1, 1, 2, 5, 14, 42, 132, 429, 1430, 4862, 16796, 58786, \dots}$
		\begin{itemize}[noitemsep]
			\item sub-diagonal monotone paths in an $n\times n$ grid.
			\item strings with $n$ pairs of parenthesis, correctly nested.
			\item binary trees with with $n+1$ leaves (0 or 2 children).
			\item ordered trees with $n+1$ vertices.
			\item ways a convex polygon with $n+2$ sides can be cut into triangles by connecting vertices with straight lines.
			\item permutations of $[n]$ with no 3-term increasing subseq.
		\end{itemize}


%% theorem/proof/lemma/corollary/example blocks
\section{Prime Numbers}
\subsection{Definition}
\begin{frame}{What Are Prime Numbers?}
  \begin{definition}
    A \alert{prime number} is a number that has exactly two divisors.
  \end{definition}
  \begin{example}
    \begin{itemize}
      \item 2 is prime (two divisors: 1 and 2).
      \item 3 is prime (two divisors: 1 and 3).
      \item 4 is not prime (\alert{three} divisors: 1, 2, and 4).
    \end{itemize}
  \end{example}
\end{frame}

\lipsum[1]


\documentclass[main]{subfiles}
\begin{document}
\section{Poisson Regression}
\label{sec:poisson_regression}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{}

Transform the poisson distribution function
\begin{align*}
	p(y;\lambda)
		&=
			e^{-\lambda}
			\frac {(\lambda)^y} {y!} 			\\
		&=
			\frac {1} {y!}
			\exp (
				y \log \lambda - \lambda
			)
\end{align*}

Compare it to the general equation that defines the exponential family
\begin{center}
\begin{math}
	p(y;\lambda) =
		b(y) \exp ( \eta \, y - a(\eta) )
\end{math}
\end{center}

We get
\begin{align*}
	a(\eta) 	&=
			\lambda = e^{\eta} 					\\
	b(y)		&=
			\frac {1} {y!}						\\
	\eta 		&=
			\log \lambda						\\
	T(y)		&=
			y
\end{align*}

\subsection{}

The mean value of a random variable that abides by the Poisson distribution is
$\lambda$. Thus the \textbf{canonical response function} is
\begin{center}
\begin{math}
	g(\eta) = E[y;\lambda] = \lambda = e^{\eta}
\end{math}
\end{center}

\subsection{}
skipped

\subsection{}
Given $T(y) = y$, the general equation of exponential distributions is
simplified to
\begin{align*}
	p(y|\eta) =
		b(y) \exp ( \eta \, y - a(\eta) )
\end{align*}

With the course assumption $\eta = \v{\theta}\cdot\v{x}$, the log-likelihood
function can be written as
\begin{align*}
	l(\theta)
		&=
			\log p(y^{(i)}|x^{(i)};\theta) 		\\
		&=
			\log b(y) +
			(\v{\theta}\cdot\v{x}^{(i)}) y - a(\v{\theta}\cdot\v{x}^{(i)})
\end{align*}

Given a data point $(x^{(i)}, y^{(i)})$, the gradient of the log-likelihood with
respect to $\theta$ is
\begin{align*}
	\pd{l(\theta)}{\theta_j}
		=
			x^{(i)}_j (
				y - a^{\p}(\eta)
			)
\end{align*}

By observation, it concludes this proof if
\begin{align*}
	a^{\p}(\eta) = g(\eta)
\end{align*}

It can be proved. First let's define a function of $\eta$
\begin{align*}
	I(\eta)
		&=
			\int
			p(y;\eta)
			dy 									\\
		&=
			\int
			b(y) \exp ( \eta \, y - a(\eta) )
			dy
\end{align*}

$I(\eta)$ is just the sum of the probabilities of all possible states, which
must be one, i.e.
\begin{center}
\begin{math}
	I(\eta) = 1
\end{math}
\end{center}

Take the derivative with respect to $\eta$
\begin{align*}
	I^{\p}(\eta)
		&=
			\int
			\pd {} {\eta}
			b(y) \exp ( \eta \, y - a(\eta) )
			dy 									\\
		&=
			\int
			y \, b(y) \exp ( \eta \, y - a(\eta) )
			dy 									\\
		&-
			a^{\p}(\eta)
			\int
			b(y) \exp ( \eta \, y - a(\eta) )
			dy 									\\
		&=
			E[y]
			- a^{\p}(\eta) I(\eta)				\\
		&=
			g(\eta)
			- a^{\p}(\eta) 						\\
		&=
			0
\end{align*}

The last step used the fact that the derivative of a constant number is zero.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}