%\nomenclature[]{MLE}{Maximum Likelihood Estimate}
%\note{Make sure it's clear whether using random variables or not}
%\note{Might need to make clear Markov assumptions }
As discussed, HMMs and DBNs are useful tools for modelling complex random dynamic processes. Typically, models are used to provide some kind of inference about a process, to gain insight into its inner workings. By inference, we mean calculating the probability distributions over variables of interest. Some algorithms will be outlined here that will demonstrate how to calculate both exact and approximate inferences, using general structures for HMMs and DBNs. \par

First, inference algorithms commonly used with HMMs are discussed, as their representation is more rigid, meaning that it is easier to exploit their structure generally to perform inference. As previously stated, HMMs and DBNs are routinely used to model systems that have state variables that cannot be directly observed. In most cases, we are interested in inferring what the state of the system might be, given the observed evidence variables received. This is the first and arguably the most useful inference challenge that we inspect.

The actual quantity that we want to compute is
\[P(X_t | e_1, e_2, ..., e_t) = P(X_t | e_{1:t})\]
that is, the probability distribution of the hidden state variable, given all previously observed evidence variables. The notation $e_{1:t}$ denotes the joint distribution of $e_1, e_2, ..., e_t$. We are interested in computing this value, rather than the unconditional state distribution, $P(X_t)$, because we cannot observe the state directly, but we can observe the evidence variables. The conditional distribution $P(X_t | e_{1:t})$ is frequently referred to as the \textit{belief state} and the process of calculation of this distribution is frequently referred to as \textit{state estimation} or \textit{filtering} \cite{AIAMA}, \cite{Thrun:2005:ProbabilisticRobotics}, \cite{KollerPGM}. The \textit{forward algorithm} can be used to calculate the value of $P(X_t | e_{1:t})$ \cite[p.~572]{AIAMA}. It is a recursive algorithm, and takes advantage of the fact that the underlying process is Markovian. The forward algorithm for HMMs is shown in Algorithm \ref{alg:forwardAlgorithmHMMs} and is based on the algorithm outlined in \cite[p.~27]{Thrun:2005:ProbabilisticRobotics}. The derivation is useful to follow as an exercise, so we present it in the subsequent sub-section.
\subsubsection{HMM Filtering Algorithm Derivation}
\label{section:HMMFiltering}
\note{Might be better to put this in an appendix.}
Two well-known probability identities are used in the derivation:
\note{Fix the formatting here}
\begin{center}
%- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
\end{center}
%quad adds space
\[(a) \quad p(A | B, C) = \frac{p(B | A, C) p(A | C)}{p(B | C)} \quad \text{and} \quad (b) \quad p(A | B) = \int\limits_{C}P(A | B, C) P(C | B)dC\]

\begin{center}
%- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
\end{center}
The derivation is as follows:
\begin{enumerate}
\item {$ p(X_t | e_{1:t}) = p(X_t | e_{1:t-1}, e_t) $ }

\item{$ \text{applying (a) and letting} \quad \eta = \frac{1}{p(e_t | e_{1:t-1})} \quad = \quad \eta p(e_t | e_{1:t-1}, X_t)p(X_t|e_{1:t-1}) $ }

\item{$ \text{by the Markov property} \quad = \quad \eta p(e_t | X_t)p(X_t|e_{1:t-1})$}

\item{$\text{applying (b)} \quad =  \quad \eta p(e_t | X_t)\int_{X_{t-1}}p(X_t|e_{1:t-1}, X_{t-1}) p(X_{t-1}|e_{1:t-1})dX_{t-1}$}

\item{$ \text{by the Markov property} \quad = \quad \eta p(e_t | x_t)\int_{X_{t-1}}p(X_t|X_{t-1}) p(X_{t-1}|e_{1:t-1})dX_{t-1} $}

\end{enumerate}
Note that $\eta$ is a normalizing constant that ensures that the probability distribution integrates to 1, and that the probabilities that need to be calculated can be done so from the parameters specified in the HMM; namely $p(e_t | X_t)$, specified by the sensor model, and $p(X_t | x_{t-1})$, specified by the transition model.

\begin{algorithm}{}
\caption{Forward Algorithm for HMMs}
\label{alg:forwardAlgorithmHMMs}

\begin{algorithmic}[1]
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}
%Input
\REQUIRE $\newline P(x_{t-1} | e_{1:t-1})=bel(x_{t-1}): \quad \text{The belief distribution as far as the previous time-step}
\newline e_t: \quad \text{The most recent observation}
\newline hmm: \quad \text{A Hidden Markov Model specifying the transition and observation probabilities,} \newline p(X_t | x_{t-1}) \text{ and } p(E_t | x_t)$
%Output
\ENSURE  $\newline P(X_{t} | e_{1:t}) = bel(X_{t})$

\hfill\pagebreak

\FOR{all $x_t$}
\STATE $\overline{bel}(x_t) = \int p(x_t | x_{t-1}) p(x_{t-1} | e_{1:t-1}) d x_{t-1}$
\STATE $bel(x_t) = p(e_t | x_t) \overline{bel}(x_t)$
\ENDFOR
\STATE $ \eta = 1 / \int_{x_t}{bel(x_t)}dx_t$
 \FOR{all $x_t$ do:}
\STATE $bel(x_t) = \eta{bel}(x_t)$
\ENDFOR

%\ENDWHILE
\RETURN $bel(X_t)$
\end{algorithmic}
\end{algorithm}

\note{Don't forget to mention: Sufficient statistics, forward algorithm}

Algorithm \ref{alg:forwardAlgorithmHMMs} uses integrals to account for the fact that the distribution of the hidden state, $X_t$, may be continuous. The work done in this thesis only uses discrete distributions, and so summations are used instead of integrals for the subsequent discussion. A consequence of using discrete distributions is that it is possible to formulate the forward algorithm using vector and matrix notation. As outlined in Section \ref{subsec:BGHMM}, the transition model $T=p(X_t | X_{t-1})$ for a HMM can be written down in matrix form, as can the observation model $O_t=p(E_t | X_T)$. This allows for a highly compact notation: denoting $p(x_t | e_{1:t})$ as $f_{1:t}$, we can write $f_{1:t} = \eta O_{t} T^{T} f_{1:t-1}$, where $f_{1:t}$ contains the vector of values of $p(x_t | e_{1:t})$ for every possible $x_t$ \cite[p.~579]{AIAMA}. Note that the observation model, $O_t$ is time dependent, as opposed to the transition model, which is assumed to be stationary. \par

Some insight can be gained from studying Algorithm \ref{alg:forwardAlgorithmHMMs}. There are three main steps to this algorithm, and the intuition behind them is explained based on explanations in \cite{AIAMA}, \cite{Thrun:2005:ProbabilisticRobotics} and \cite{Murphy1994DynamicLearning}:
\note{These are highly verbose, edit to make less wordy}
\begin{enumerate}

    \item The first step is often referred to as the prediction step:
    \begin{center}
    $\overline{bel}(x_t) = \sum_{x_{t-1}} p(x_t | x_{t-1}) p(x_{t-1} | e_{1:t-1}) d x_{t-1}$
    \end{center}

    This step carries out the first part of the calculation of the belief value for a given state $x_t$, by marginalizing over all possible hidden states ($x_{t-1}$) that could have preceded the current one ($x_t$). This reflects intuition: the probability of being in state $x_t$ should depend on the probability of transitioning from all possible previous states to $x_t$.
    %\note{this sentence doesn't seem necessary}
    Think "\textit{the probability that I am in location x is the sum of probabilities of starting at all other possible locations and subsequently ending up in location x, weighted against my belief that I began in each other possible location}".

    %product of our most recent belief of being in state $x_{t-1}$, given by $p(x_{t-1} | e_{1:t-1})$ and the probability of transitioning from that state to state $x_{t}$, for all previous possible states that we could have been in.
    $\overline{bel}(x_t)$ effectively projects our current belief to the next time step by using the transition probabilities specified in the HMM.

    \item The second step is often referred to as the correction step, or the measurement update step:
    \begin{center}
    $bel(x_t) = p(e_t | x_t) \overline{bel}(x_t)$
    \end{center}
    This step takes the projected belief in step 1 and multiplies it by the probability of observing the data that we did in the given hidden state, $x_t$. This can be thought of as a correction because the use of the measurement data narrows the probabilities of the possible states that the system could have transitioned to.
    \note{Need to re-word this to make more concise}
    %For example, if the value of a transition probability from state x to state y is low, and there is a low probability that the system was previously in state x, a high probability of observing the observed sensor reading given state y can still result in a relatively high posterior probability of the system being in state y given the new evidence.

    \item The final step is to ensure that the distribution sums to 1. This is done by multiplying by a normalizing constant, $\eta$.

\end{enumerate}
A key strength of this algorithm is the fact it allows updates to be performed online. The reason for this is that the information-state vector, $p(x_t | e_{1:t})$ is a sufficient statistic for the past history of observations. For a full proof of this property, the reader is referred to Appendix A of \cite{Smallwood1973TheHorizon}, which also provides insight into the update rule itself. In relation to the update rule, note that an initial distribution of the estimated state must be provided to this algorithm before the first update can be performed. The complexity of the forward algorithm is $\theta (nm^2)$, where m is the size of the joint distribution of the hidden variables, x, and n is the number of time-steps that have occurred. This is clear to see when viewing the update as the matrix-vector multiplication $f_{1:t} = \eta O_{t} T^{T} f_{1:t-1}$, as the size of T is $m^2$.

\subsubsection{Evidence Likelihood Algorithm Using a HMM}\label{subsubsec:EvLikelihood}
\note{Will talk about this briefly for SPRT}
The second useful quantity that we would like to calculate is the probability of observing the evidence that we did, often referred to the likelihood function of the data, as identified in Section \ref{subsec:BGHMM}. In simpler terms, this is the calculation of the probability of observing the data, given a fixed parameterised model of the data (the HMM). This value is useful as it can reveal insights into whether one model is more likely than another model, with applications including \textbf{M}aximum \textbf{a} \textbf{P}osteriori (MAP) parameter estimates and hypothesis testing. It is used in this thesis in Section \ref{subsubsec:SeachTerminationMethodology}. We would like to compute the value of
\[P(e_1, e_2, ..., e_t) = P(e_{1:t})\]
given the fixed set of parameters provided by the HMM. A brief outline of the derivation is shown:\note{might be best to put the derivation in an appendix}

\subsubsection{HMM Evidence Likelihood Algorithm Derivation}\label{subsubsec:BGEvidenceLikelihood}

Analogous to the filtering algorithm derivation, two well-known probability identities are used in the derivation:
\begin{center}
%- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
\end{center}
%quad adds space
\[(a) \quad p(A, B, C) = p(A | B, C) p(B, C) \quad \text{and} \quad (b) \quad p(A, B) = \sum_{C}{p(A | B, C)p(B, C)}\]

\begin{center}
%- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
\end{center}

The derivation is as follows:
\begin{enumerate}

\item {$p(e_{1:t}) = \sum_{x_t}p(e_{1:t}, x_t) \quad = \quad \sum_{x_t}p({e_{1:t-1}, e_t, x_t}) $
}
\item {$\text{applying (a) } \quad= \quad
\sum_{x_t}{p(e_t | e_{1:t-1}, x_t)p(x_t, e_{1:t-1})}$
}
\item{$\text{by the Markov property}
\quad = \quad \sum_{x_t}p(e_t | x_t)p(x_t, e_{1:t-1})$
}

\item{$\text{applying (b)} \quad =  \quad
\sum_{x_t}p(e_t | x_t) \sum_{x_{t-1}}p(x_t|e_{1:t-1}, x_{t-1}) p(x_{t-1},e_{1:t-1})$
}

\item{$\text{by the Markov property} \quad = \quad
\sum_{x_t}{p(e_t | x_t)\sum_{x_{t-1}}p(x_t|x_{t-1}) p(x_{t-1},e_{1:t-1})}$
}

\end{enumerate}

The algorithm for calculating likelihoods uses the forward algorithm, $\alpha$, with the replacement of  $p(x_{t-1}|e_{1:t-1})$ with $p(x_{t-1}, e_{1:t-1})$ and one final summation. This means that it is possible to use the forward algorithm to calculate likelihoods: $p(e_{1:t}) = \sum_{i=1}^{n} \alpha(x_i, e_i)$, where $\alpha(x, e)$ is the result of applying the forward algorithm to the sequence of evidence variables $(e_1, ..., e_t)$.

%No point in writing up likelihood algorithm as it's very similar to forward algorithm.

%\subsubsection{HMM Parameter Learning Using Expectation Maximization}\label{subsubsec:EMAlgo}
%\note{Can talk about this briefly for battery model}
%Since a HMM is a parameterised model of a stochastic process, a natural question to ask is whether it is possible to determine a \textit{"best"} set of parameters to describe the process, by using observations of the process. This is a more complex version of the well-known process of finding the parameters that maximize the likelihood function of fully observable data, known as the \textbf{M}aximum \textbf{L}ikelihood \textbf{E}stimate (MLE). Finding the most likely set of parameters which provided the observed values can be described mathematically as the value of \note{fix argmax notation}$\argmax_{\lambda} P(E_{1:t} | \lambda)$. Note that this is a fundamentally different problem to that discussed in the preceding section; here we do not assume a fixed set of parameters for the HMM ($\lambda$).

%This is a well-studied problem for HMMs and the solution is given by the \textbf{E}xpectation \textbf{M}aximisation (EM) algorithm, proposed by \citeauthor{Dempster1977MaximumAlgorithm}. This algorithm is reasonably complex and highly detailed overviews can be found in many texts that deal with parameter estimation for statistical models. A quick outline of how the algorithm works is provided here.\par

%The reason why the standard methods for finding the MLE for a parameterised model do not work with HMMs is due to the fact that there is no way to directly express the quantity  $P(E_{1:t} | \lambda)$ directly. This can be seen in the derivation of the evidence likelihood in <reference whichever appendix it ends up in>. Instead, it is necessary to marginalize over states: $ P(e_{1:t} | \lambda) = \sum_{x_t}{p(x_t, e_{e:t} | \lambda)}$. The standard approach to finding the parameters that maximize a model is to take a log transform of the likelihood function, which preserves the solution but turns the problem into one of finding the parameters that maximize a sum rather than a product. Using standard methods of calculus, this can usually be done easily. The issue with HMMs is that when a logarithmic transformation is applied to the likelihood function, the summation sign remains within the log function, which means finding the maximum is still a difficult problem. The EM algorithm, taking the approach of many iterative techniques, relaxes the constraints on the problem in order to find a lower bound of the log-likelihood and then iteratively improves this lower bound with increasing likelihood values.

%The Baum-Welch algorithm is a special case of the EM algorithm and is frequently used to solve the problem of estimating the parameters of a HMM. The algorithm is described fully in appendix <reference appendix>.

\subsubsection{Filtering Algorithm for DBNs}\label{subsubsec:filteringDBN}
\note{Extend HMM to add control variable}
\begin{wrapfigure}{r}{0.68\textwidth}
    \centering
    \includegraphics[width = 0.68\textwidth]{Chapters/BackgroundKnowledgeAndRelatedWork/MultiAgentTargetDetectionBackground/Figs/HMMs/HMMWithControl.png}
    \caption{A DBN with Hidden State variables (grey), Observation variables (orange) and Control variables (green).}
    \label{fig:HMMWithControlVariablesExample}
\end{wrapfigure}
The forward algorithm for state estimation was presented in Section \ref{section:HMMFiltering}. A slightly modified version of this algorithm is now described, which is very commonly used in stochastic systems that can be influenced by actions that an agent may take. Such systems have \textit{control variables} as well as hidden state variables and observation variables. These control variables (variables which represent the actions that an agent may perform) have an influence on the transition probabilities between states. The graphical model in Figure \ref{fig:HMMWithControlVariablesExample} describes the basic case. The arrows describe the conditional independence assumptions: at time t, the hidden state ($x_t$) depends on the previous state ($x_{t-1}$), and the previous control action taken ($u_t$). As with the HMM, the observation $e_t$ is conditionally dependent on the state $x_t$. This causes a slight change in the filtering algorithm: on line 2 of algorithm \ref{alg:forwardAlgorithmHMMs}: $\overline{bel}(x_t) = \sum_{x_{t-1}} p(x_t | x_{t-1}) p(x_{t-1} | e_{1:t-1}) $ is replaced with $\overline{bel}(x_t) = \sum_{x_{t-1}} p(x_t | u_t, x_{t-1}) p(x_{t-1} | e_{1:t-1}, u_{1:t-1})$, reflecting that transition probabilities now depend on the most recent control action taken, as well as the previous state.

%DBNs have already been shown to be a powerful tool while modelling stochastic systems, since the hidden variables can be described by conditional independences, rather than

%This algorithm can be further generalized - if multiple control variables, hidden state variables and observation variables are needed to describe the system fully, it is possible to make minor modifications which reflect the conditional independences stated by the underlying DBN by .

It is worth noting that there are many filtering algorithms that deal with data that come from certain distributions and satisfy constraints related to their transition and observation models, for example the Kalman Filter \cite[p.~43]{Thrun:2005:ProbabilisticRobotics}.

%\subsubsection{Discrete Bayes Filter}
%\note{This is what is used in the work that I did, so it is outlined}

\subsubsection{Approximate State Estimation}
\note{Thought about using Particle filter to avoid the dimensionality problem when attempting to maintain estimated state}
The state estimation algorithms mentioned so far maintain the exact values of the distribution $p(x_t | e_{1:t}, u_{1:t})$, but we will quickly mention an approximate method that is also frequently used. The computational complexity of the filtering algorithms is determined by how well the joint distribution of hidden state variables, observation variables and control action variables can be factored, but in the worst case the complexity is given by the case where the joint distribution of the hidden state variables cannot be factored at all. The forward algorithm, described in Algorithm \ref{alg:forwardAlgorithmHMMs} for a HMM performs the update from each time state in O($n^2$), where $n$ is the dimension of the hidden state variable \cite{Smyth1997ProbabilisticModels}.
%$O(n^2ut)$, where n are the number of hidden states, u is the number of available control actions and t is the number of time steps.
In some cases, this can become intractable if the state space is large enough. % and the transition model cannot be factored.
In this case, approximate techniques are used. An example is the \textit{particle filter}. Details of the particle filter can be found in \cite[p.~96]{Thrun:2005:ProbabilisticRobotics} and \cite[p.~665]{KollerPGM}.


\documentclass[12pt, letterpaper]{article}
\include{eu}

\newcommand{\escape}{\mathrm{escape}}

\begin{document}

\section*{\textsl{Einstein's Universe} Problem Set 6}

This problem set is not to be handed in for credit. But it is due
before \textbf{Thursday December 9}, when some of these problems
will appear in part, near-verbatim, on Term Exam 6.

\begin{problem}
The formula for escape velocity $v_{\escape}$ from the surface of the Earth
is
\begin{equation}
\frac{1}{2}\,v_{\escape}^2 - \frac{G\,M}{R} = 0
\quad,
\end{equation}
where $G$ is Newton's constant, $M$ is the mass of the Earth, and $R$
is the radius of the Earth.
Now imagine that the Earth was compressed into a sphere so small that
this escape velocity became the speed of light! That's a proposal for what
makes a black hole. What would be the radius of that small sphere?
Now look up the true formula for the radius of a black hole and compare
your answer to the true answer.

If you make $M$ the mass of the Sun, and $R$ the radius (1\,AU) of
Earth's orbit around the Sun, then the velocity you compute is the
escape velocity from the Earth's orbit to outside the Solar System!
This is the velocity at which the {\small NASA} \textsl{Voyager} probes had to
be launched.
How massive would the Sun have to be for the escape velocity at 1\,AU
to be the speed of light? You can use either the equation above, or
else the true formula for a black hole.
Give your answer in kg, and also in Solar masses.
\end{problem}

\begin{problem}
The \textsl{\small LIGO} discovery of the neutron-star-merger system
{\small GW170817} gave an opportunity to measure the speed of
gravitational waves (GWs), because the discovered event had both a
light signal (a flash) and a GW signal (a chirp).
Look up the distance to this event.
Now imagine that GWs travel more slowly than the speed of light.
How much slower than the speed of light would GWs have to travel for
the two signals to arrive at Earth separated by 10 seconds?
Express your answer as a dimensionless fraction:
That is, what is the ratio of 10 seconds to the travel time of light from
the event to us?
\end{problem}

\begin{problem}
At the center of the Milky Way, there is a star called {\small S2}
orbiting a supermassive black hole (called Sag A$\ast$).
The star orbits the black hole with a period of 16\,yr, on an orbit
that has a mean radius (semi-major axis) of 1000\,AU.
Plug these numbers into the classic Kepler--Newton formula for orbital
period $T$ given semi-major axis $a$ to estimate the mass of the black
hole.
\begin{equation}
T^2 = \frac{4\,\pi^2}{G\,M}\,a^3
\end{equation}
You might have to do some unit conversions!
Give your answer in kg, and also in Solar masses.

Also compute the mean orbital velocity $v$ of the star using
\begin{equation}
v \approx \frac{2\,\pi\,a}{T}
\end{equation}
Do you think the non-relativistic Kepler formula is very wrong in this
case? Why or why not?
\end{problem}

\begin{problem}
When a black hole of mass $M_1$ merges with a black hole of mass $M_2$,
it creates a new black hole of mass $M_3$. Do you expect $M_3$ to be
equal to, less than, or more than, the sum $M_1 + M_2$? Recall mass--energy
equivalence and the fact that there is gravitational radiation emitted as
they merge.

Now look up the first \textsl{\small LIGO} discovery {\small GW150914} and
look up $M_1, M_2, M_3$ for this discovery. Are you right?
\end{problem}

\end{document}


\section{Custom commands}

\begin{frame}[fragile]{Simple macros}
  Used to simplify repetitive and/or complex formatting.

  Usually specified in the preamble
  \begin{lstlisting}
\newcommand{\name}{definition}
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]{Simple macros: an example}
  \begin{lstlisting}
\newcommand{\R}{\mathbb{R}}

The set of real numbers are usually represented
by a blackboard capital r: $\R$.
  \end{lstlisting}

  The set of real numbers are usually represented by a blackboard capital r:
  $\R$.
\end{frame}

\begin{frame}[fragile]{Macros with parameters}
  Macros can also be constructed to accept parameters:
  \begin{lstlisting}

\newcommand{\name}[# params]{definition}
  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]{Macros with parameters: an example}
  \begin{lstlisting}
\newcommand{\bb}[1]{\mathbb{#1}}

Other numerical systems have similar notations.
The complex numbers $\bb{C}$, the rational
numbers $\bb{Q}$ and the integer numbers
$\bb{Z}$.

  \end{lstlisting}
  Other numerical systems have similar notations. The complex numbers $\bb{C}$,
  the rational numbers $\bb{Q}$ and the integer numbers $\bb{Z}$.
\end{frame}

\begin{frame}[fragile]{Macros with default parameters}
  It is also possible to define macros which take default parameters:
  \begin{lstlisting}
\newcommand{\name}[# params][default #1]{def.}

  \end{lstlisting}
\end{frame}

\begin{frame}[fragile]{Macros with default parameters}
  \begin{lstlisting}
\newcommand{\plusbinomial}[3][2]{(#2 + #3)^#1}

We make a new command to save time writing
expressions of the form $\plusbinomial{x}{y}$
and $\plusbinomial[4]{a}{b}$.

  \end{lstlisting}
  We make a new command to save time writing expressions of the form
  $\plusbinomial{x}{y}$ and $\plusbinomial[4]{a}{b}$.
\end{frame}

\begin{frame}[standout]
  \href{https://jwalton.info/assets/teaching/latex/exercise_2.pdf}%
  {\color{white}Exercise 2}
\end{frame}


%%%%%%%%%%%%%%%%
%% Background %%
%%

\section{Introduction}

Seed based functional connectivity, variance decomposition, and clustering analyses of resting state and task fMRI data have shown that that brain activity can be broken down into a non-overlapping networks that are consistently found across individuals. Early analyses have extracted these networks from longer duration data (5 minutes or more) but shorter-scale analyses are showing quite a bit of temporal dynamics in these networks, which appear to break down into temporal functional modes (TFM). These TFMs represient transient states of network coherence which occur several times during long blocks of brain activity. Although many of the networks can be attributed behaviors from their appearance in task analyses, the meaning of other networks, and the complex interactions between networks is poorly understood. We have developed a real-time fMRI based method for tracking network activity that can be used to develop temprally adabtive experiments for better probing these networks.

Based on preliminary work in functional and effective connectivity, the activity of brain networks can be modeled as a weighted average of activity in the brain areas involved in the network. If the number of brain areas is small compared to the number of observations, then the weights in the average can be solved directly using well established linear algebra techniques. But, in the case of neuroimaging, were the number of brain areas tends to significantly outnumber observations, alternative methods need to be used to solve the equations. There are a variety of different methods in the statistics and machine learning literature for doing this and one method is support vector regression.

Once the weights in the linear equation are solved for, they can be applied to newly acquired fMRI data to decode a measure of network activity TR-by-TR.

\subsection{MRI Acquisition}

Data was acquired on a 3 T Siemens Magnetom TIM Trio scanner (Siemens Medical Solutions USA: Malvern PA, USA) using a 12-channel head coil. Anatomic images were acquired at $1 \times 1 \times 1$ mm$^3$ resolution with a 3D T1-weighted magnetization-prepared rapid acquisition gradient-echo (MPRAGE) sequence \cite{Mugler1990} in 192 sagittal partitions each with a $256 \times 256$ field of view (FOV), 2600 ms repetition time (TR), 3.02 ms echo time (TE), 900 ms inversion time (TI), 8$^\circ$ flip angle (FA), and generalized auto-calibrating partially parallel acquisition (GRAPPA) \cite{Griswold2002} acceleration factor of 2 with 32 reference lines. The sMRI data were acquired immediately after a fast localizer scan and preceded the collection of the functional data.

A gradient echo field map sequence was acquired with the following parameters: TR 500 ms, TE1/TE2 2.72 ms/5.18 ms, FA 55$^\circ$, 64 x 64 matrix, with a 220 mm FOV, 30 3.6 mm thick interleaved, oblique slices, and in plane resolution of $3.4 \times 3.4$ mm$^2$. All functional data were collected with a blood oxygenation level dependent (BOLD) contrast weighted gradient-recalled echo-planar-imaging sequence (EPI) that was modified to export images, as they were acquired, to AFNI over a network interface \cite{Cox1995,LaConte2007}. FMRI acquisition consisted of 30 3.6mm thick interleaved, oblique slices with a \%10 slice gap, TR 2000 ms, TE 30 ms, FA 90$^\circ$, $64 \times 64$ matrix, with 220 mm FOV, and in plane resolution of $3.4 \times 3.4$ mm$^2$. Functional MRI scanning included a three volume “mask” scan and a six-minute resting state scan followed by three task scans (described later) whose order was counterbalanced across subjects.

During all scanning, galvanic skin response (GSR), pulse and respiration waveforms were measured using MRI compatible non-invasive physiological monitoring equipment (Biopac Systems, Inc.). Rate and depth of breathing were measured with a pneumatic abdominal circumference belt. Pulse was monitored with a standard infrared pulse oximeter placed on the tip of the index finger of the non-dominant hand. Skin conductance was measured with disposable passive electrodes that were non-magnetic and non-metallic, and collected on the hand. The physiological recordings were synchronized with the imaging data using a timing signal output from the scanner. Visual stimuli were presented to the participants on a projection screen that they could see through a mirror affixed to the head coil. Audio stimuli were presented through headphones using an Avotec Silent-Scan® pneumatic system (Avotec, Inc.: Stuart FL, USA).

\subsection{MRI acquisition order and online Processing}

The real time fMRI neurofeedback experiment utilizes a classifier based approach for extracting DMN activity levels from fMRI data TR-by-TR, similar to \cite{Craddock2012}. The classifier is trained from resting state fMRI data using a time course of DMN activity extracted from the data using spatial regression to a publicly available template derived from a meta-analysis of task and resting state datasets \cite{Smith2009,fmrib_RSNS}. Several stages of online processing are necessary to perform this classifier training, as well as, denoising of fMRI data in real-time. These stages include calculating transforms required to convert the DMN template from MNI space to subject space, creating white matter (WM) and cerebrospinal fluid (CSF) masks for extracting nuisance signals, and training a support vector regression (SVR) model for extracting DMN activity. The MRI session was optimized to collect the data required for these various processing steps, and to perform the processing, while minimizing delays in the experiment.

After acquiring a localizer, the scanning protocol began with the acquisition of a T1 weighted anatomical image used for calculating transforms to MNI space and white matter and CSF masks. Once the image was acquired it was transferred to a DICOM server on the real-time analysis computer (RTAC), which triggered initialization of online processing. The processing script started AFNI in real-time mode, configured it for fMRI acquisition, and began structural image processing. Structural processing included reorienting the structural image to RPI voxel order, skull-stripping using AFNI’s 3dSkullStrip \cite{Cox1996}, resampling the image to isotropic 2-mm voxels (to reduce computational cost of subsequent operations), segmentation into grey matter (GM), white matter (WM), and cerebrospinal fluid (CSF) using FSL’s FAST \cite{Zhang2001}, and normalization into MNI space using FSL’s FLIRT \cite{Jenkinson2002,Jenkinson2001}. WM and CSF probability maps were binarized using a 90\% threshold. The CSF mask was constrained to the lateral ventricles using an ROI from the AAL atlas to avoid overlap with GM.

In parallel with the structural processing a FieldMap was collected but not used in the online processing. Subsequently, a three volume “mask” EPI scan was acquired and transferred to the RTAC. The three images were averaged, reoriented to RPI, and used to create a mask to differentiate brain signal from background (using AFNI’s 3dAutomask \cite{Cox1996}). The mean image was coregistered to the anatomical image using FSL’s boundary based registration (BBR) \cite{Greve2009}. The resulting linear transform was inverted and applied to the WM and CSF masks to bring them into alignment with the fMRI data. Additionally, the transform was combined with the inverted anatomical-to-MNI transform and applied to the canonical map of the DMN (from \cite{Smith2009}).

Next, a 6-minute resting state scan (182 volumes) was collected and used as training data to create the support vector regression (SVR) model. This procedure involved motion correction followed by a nuisance regression procedure to orthogonalize the data to six head motion parameters, mean WM, mean CSF, and global signal \cite{Friston1996,Fox2005,Lund2006}. A SVR model of the DMN was trained using a modified dual regression procedure in which a spatial regression to the unthresholded DMN template was performed to extract a time course of DMN activity. The Z-transformed DMN time course was then used as labels (independent variable), with the preprocessed resting state data as features (dependent variables), for SVR training (C=1.0, $\epsilon = 0.01$) using AFNI’s 3dsvm tool \cite{LaConte2005}. The result was a DMN map tailored to the individual participant based on preexisting expectations about DMN anatomy and function. After SVR training was completed (generally took less than 2 minutes), the MSIT (198 volumes), Moral Dilemma (144 volumes), and Neurofeedback test (412 volumes) scans were run. The order of the task based functional scans was counterbalanced and stratified for age and sex across participants.

\subsection{Functional MRI Tasks}

\subsubsection{Resting state scan}

Participants were instructed to keep their eyes open during the scan and fixate on a white plus (+) sign centered on a black background. They were asked to let their mind wander freely and if they noticed themselves focusing on any particular train of thought, to let their mind wander away from it.


\section{Mod $\cC$ Hurewicz, Whitehead, cohomology spectral
sequence}\label{mod-c-hurewicz}
We had $\cC_{fg}$ and $\cC_{tors}$, and
$$
\cC_\cP = \{A|\ell:A\xrightarrow{\simeq} A,\ell\not\in\cP\},\quad \cC_p = \cC_{\{p\}},\quad \cC_{p^\prime} = \cC_{\text{not }p}
$$
Another one is $\cC_{p^\prime}\cap \cC_{tors}$, which consists of torsion groups such that $p$ is an isomorphism on $A$.
There is therefore no $p$-torsion, and it has only prime-to-$p$ torsion.
This is the same thing as saying that $A\otimes\Z_{(p)} = 0$.
\begin{theorem}[Mod $\cC$ Hurewicz]
    Let $X$ be simply connected and $\cC$ a Serre class such that $A,B\in\cC$ implies that $A\otimes B,\Tor_1(A,B)\in \cC$ (this is axiom 2).
    Assume also that if $A\in\cC$, then $H_j(K(A,1)) = H_j(BA)\in\cC$ for all $j>0$.
    (This is valid for all our examples, and is what is called Axiom 3.)

    Let $n\geq 1$.
    Then $\pi_i(X)\in\cC$ for any $1<i<n$ if and only if $H_i(X)\in\cC$ for any $1<i<n$,
    and $\pi_n(X)\to H_n(X)$ is a mod $\cC$ isomorphism.
\end{theorem}
\begin{example}
    For $1<i<n$, the group $\widetilde{H}_i(X)$ is:
    \begin{enumerate}
	\item torsion;
	\item finitely generated;
	\item finite;
	\item $-\otimes \Z_{(p)} = 0$
    \end{enumerate}
    if and only if $\pi_i(X)$ for $1<i<n$.
\end{example}
\begin{proof}
    Look at $\Omega X\to PX \to X$.
    Then $\pi_1 \Omega X\in\cC$.
    Look at Davis+Kirk.
\end{proof}
There's a Whitehead theorem that comes out of this, that I want to state for you.
\begin{theorem}[Mod $\cC$ Whitehead theorem]
    Let $\cC$ be a Serre class satisfying axioms 1, 2, 3, and:
    \begin{enumerate}
	\item[($2^\prime$)] $A\in\cC$ implies that $A\otimes B\in\cC$ for any $B$.
    \end{enumerate}
    This is satisfied for all our examples except $\cC_{fg}$.

    Suppose I have $f:X\to Y$ where $X,Y$ are simply connected.
    Suppose $\pi_2(X)\to \pi_2(Y)$ is onto.
    Let $n\geq 2$.
    Then $\pi_i(X)\to \pi_i(Y)$ is a $\cC$-isomorphism for $2\leq i\leq n$ and is a $\cC$-epimorphism for $i=n$,
    with the same statement for $H_i$.
\end{theorem}
These kind of theorems help us work locally at a prime, and that's super.
You'll see this in the next assignment, which is mostly up on the web.
You'll also see this in calculations which we'll start doing in a day or two.

Change of subject here.
Today I'm going to say a lot of things for which I won't give a proof.
I want to talk about cohomology sseq.
\subsection{Cohomology sseq}
We're building up this powerful tool using spectral sequences.
We saw how powerful the cup product was, and that is what cohomology is good for.
In cohomology, things get turned upside down:
\begin{definition}
    A \emph{decreasing filtration} of an object $A$ is
    $$A\supseteq\cdots\supseteq F^{-1} A\supseteq F^0 A \supseteq F^1 A\supseteq F^2 A\supseteq \cdots\supseteq 0$$
    This is called ``bounded above'' if $F^0 A = A$.
    Write $\gr^s A = F^sA/F^{s+1}A$.
\end{definition}
\begin{example}
    Suppose $X$ is a filtered space.
    So there's an increasing filtration $\emptyset=F_{-1}X\subseteq F_0X\subseteq\cdots$.
    Let $R$ be a commutative ring of coefficients.
    Then I have $S^\ast(X)$, where the differential goes up one degree.
    Define
    $$F^s S^\ast(X) = \ker(S^\ast(X)\to S^\ast(F_{s-1}X))$$
    For instance, $F^0 S^\ast(X) = S^\ast(X)$.
    Thus this is a bounded above decreasing filtration \todo{My computer will run out of juice soon, \TeX this up later!}.
\end{example}
\begin{example}
    Let $X=E\xrightarrow{\pi}B = \text{ CW-complex}$ with $\pi_1(B)$ acting trivially on $H_t(F)$.
    Then $F_s E = \pi^{-1}(\mathrm{sk}_s B)$.
    Thus I get a filtration on $S^\ast(E)$, and
    $$
    F^s H^\ast(X) = \ker(H^\ast(X)\to H^\ast(F_{s-1}X))
    $$
\end{example}
Doing everything the same as before, we get a \emph{cohomology spectral sequence}.
Here are some facts.
\begin{enumerate}
    \item First, you have $E_r^{s,t}$ (note that indices got reversed).
	There's a differential $d_r:E^{s,t}_r \to E_r^{s+r,t-r+1}$, so that the total degree of the differential is $1$.
    \item You discover that
	$$
	E^{s,t}_2 \simeq H^s(B;H^t(F))
	$$
    \item and $E^{s,t}_\infty \simeq \gr^s H^{s+t}(E)$.
    \item
\end{enumerate}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{enumitem}

\newcommand{\problem}[2]{\subsection*{#1 - Problem #2}}


\documentclass[aps,pra,notitlepage,amsmath,amssymb,letterpaper,12pt]{revtex4-1}
\usepackage{amsthm}
\usepackage{graphicx}

\newenvironment{problem}[2][Problem]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}\hskip \labelsep {\bfseries #2.}]}{\end{trivlist}}
\newenvironment{solution}{\begin{proof}[Solution]}{\end{proof}}

\begin{document}

\title{Classwork 13: George and Moana but In LaTeX}
\author{Morgan Holve}
\affiliation{MATH 220, Schmid College of Science and Technology, Chapman University}
\date{\today}

\maketitle

\section{Introduction}
The goal of this is to basically put everything from Classwork 12 and Homework 12 into LaTeX. Allow me to restate everything from those Jupyter notebooks into this:

\vspace{5mm}
We are considering a ball of mass $m$ rolling in a double-well potential (whatever that is) of $V(x) = x^4/4 - x^2/2$. This potential is unaptly called the "sombrero potential" despite its likeness toward stingrays.

\vspace{5mm}
Taking physics things into account and also remembering that the ball must follow Newton's Second Law, we obtain the equation:
$$m\ddot{x} = f_{\text{hat}}(x) + f_{\text{drag}}(\dot{x}) + f_{\text{drive}}(t) = x - x^3 - \nu \dot{x} + F\cos(\omega t)$$
Which can be split into the system of differential equations:
$$\dot{x}(t) = y(t)$$
$$m\dot{y}(t) = -\nu y(t) + x(t) - x^3(t) + F\cos(\omega t)$$
Here, we will solve these using the Runge-Kutta 4th Order method outlined in the past two classworks, taking $m = 1$, $\omega = 1$ and $\nu = .25$. We'll graph our results and then see what everything means. \par
\noindent
Reminder of the method:
$u_{k+1} = u_k + (K_1 + 2K_2 + 2K_3 + K_4)/6$,

   with

   $K_1 = \Delta t\,f[t_k,u_k]$,

   $K_2 = \Delta t\, f[t_k + \Delta t/2, u_k + K_1/2]$,

   $K_3 = \Delta t\, f[t_k + \Delta t/2, u_k + K_2/2]$,

   $K_4 = \Delta t\,f[t_k + \Delta t, u_k + K_3]$

\section{Implementation}
We solved the differential equation each for different $F$ values and initial conditions.

\subsection{Varying Intial Conditions}
When our code was implemented, the following figures were produced for the different initial conditions.

\begin{figure}[h!]
  \includegraphics[width=0.4\textwidth]{frame02.jpg}
  \caption{Initial Conditions: $F = 0.18$, $t\in[0,2\pi\, 50]$, with $x(0) = 0.9$ and $y(0) = 0$; Scatter Plot}
  \label{fig:figlabel}
\end{figure}

\begin{figure}[h!] % h forces the figure to be placed here, in the text
  \includegraphics[width=0.4\textwidth]{frame04.jpg}  % if pdflatex is used, jpg, pdf, and png are permitted
  \caption{Initial Conditions: $F = 0.25$, $t\in[0,2\pi\, 50]$, with $x(0) = 0.2$ and $y(0) = 0.1$}
  \label{fig:figlabel}
\end{figure}

\section{Analysis}
Now, what do these graphs mean? To do this, we look at the relative density of the lines on each side of the graph. For Figure 1, we see that both the left and right sides of the figure are balanced, meaning the ball moved uniformly and evenly in between the wells. However, in Figure 2, we see that the right side is obviously favored over the left, meaning the ball spend more time in the rightmost well rather than the left. \par
\noindent
For your viewing pleasure, here is the Moana graph because I'm a little obsessed with it.
\begin{figure}[h!] % h forces the figure to be placed here, in the text
  \includegraphics[width=0.4\textwidth]{frame03.jpg}  % if pdflatex is used, jpg, pdf, and png are permitted
  \caption{See the light as it shines on the sea, it calls me. And no one knows how far it goes. If the wind in my sail on the sea stays behind me, I know one day I'll find the way.}
  \label{fig:figlabel}
\end{figure}
\end{document}


% !TEX root = main.tex

\chapter{Bivariate analysis}\label{chap:bivariate}

\input{11A_two_sample_tests}
\input{11B_chi_squared}
\input{11C_anova}
\input{11D_regression}


\documentclass[11pt,twoside]{article}

\usepackage{paperlighter}

% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\graphicspath{ {figure/} }

\usepackage{subfigure}
\usepackage{booktabs} % for professional tables

% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Todonotes is useful during development; simply uncomment the next line
%    and comment out the line below the next line to turn off comments
%\usepackage[disable,textsize=tiny]{todonotes}
\usepackage[textsize=tiny]{todonotes}
\usepackage{wrapfig}

\slimtitle{Force on a Wire Lab Report}
\slimauthor{Tanish Tyagi}

\begin{document}

\lightertitle{Force on a Wire Lab Report}
\lighterauthor{Tanish Tyagi}

\section{Experiment Results and Analysis}

% Part 1 Graph and Analysis
\subsection{Part 1 Results and Analysis}
\begin{wrapfigure}{l}{0.7\textwidth}
    \centering
    \includegraphics[width=0.7\textwidth]{part_1_graph.png}
    \caption{Graph of Force vs. Current}
\end{wrapfigure}

The equation of best fit for this graph was $Force (N) = -0.002254 (\frac{N}{amp}) \cdot I (amp)$. I made the equation go through the origin, as when $0$ amps of current is passed through the wire and plastic board, no additional force will be exerted by the wire onto the magnet and therefore the scale. Therefore, the point $(0,0)$ can serve as an additional data point. The data also suggests that force is proportional to current.

% Part 2 Graph and Analysis
\subsection{Part 2 Results and Analysis}
\begin{wrapfigure}{l}{0.7\textwidth}
    \centering
    \includegraphics[width=0.7\textwidth]{part_2_graph.png}
    \caption{Graph of Force vs. Length of Wire}
\end{wrapfigure}

The equation of best fit for this graph was $Force (N) = -0.1354 (\frac{N}{m}) \cdot L (m)$. I made the equation go through the origin, as when the length of the wire is $0$ amps of current is passed through the wire and plastic board, there is no opportunity for current to go through the wire and create a downward force that increases the balance on the scale. \\ \\ Therefore, the point $(0,0)$ can serve as an additional data point. The data also suggests that force is proportional to the length of the wire.

% Part 3 Graph and Analysis
\subsection{Part 3 Results and Analysis}
\begin{wrapfigure}{l}{0.7\textwidth}
    \centering
    \includegraphics[width=0.7\textwidth]{part_3_graph.png}
    \caption{Graph of Force vs. Number of Magnets}
\end{wrapfigure}

The equation of best fit for this graph was $Force (N) = -0.0007808 (N) \cdot M$. I made the equation go through the origin, as it was of the recorded data points. This makes sense, as when $0$ magnets are present, there are no objects exerting a force onto the wire, which means that the wire will not exert a downwards force onto the objects / scale by Newton's 3rd Law. This data also suggests that force is proportional to the number of magnets. \\

\textbf{General Note about all three graphs: You will notice that all my slopes and force values are negative. This is because the orientation of the horseshoe was flipped during the experiment. However, all that needs to be considered is the magnitudes of the aforementioned values.}

\section{Creating a Single Equation that Combines the Relationships between Force and Current, Length of Wire, and Number of Magnets}

In the above section we learned three things:

\begin{enumerate}
\item $F \propto I$, $I$ = current
\item $F \propto L$, $L$ = length of wire
\item $F \propto N$, $N$ = number of magnets
\end{enumerate}

We can combine these relationships to get a general equation $F = K \cdot I \cdot L \cdot N$. The units of $K$ will be $\frac{N}{amp \cdot m} = \frac{N}{C \cdot \frac{m}{s}}$, which is equivalent to a Tesla. This makes sense, as $K \cdot N$ describes the strength of $B$, which also has the unit of Tesla.

To find the numerical value of $K$, I utilized two methods:

\begin{enumerate}
    \item I extracted three separate $K$ values from the Force vs Current, Length of Wire, and Number of Magnets graphs. For example, take the Force vs. Current graph.

    The line of best for this graph can be written as $F = M \cdot I$. We know that $M = K \cdot L \cdot N$, and $L = 0.0322 m$ and $N = 6$ magnets. We can now solve for $K$.

    I performed a similar process for the other two graphs as well to ensure the rigor of my $K$ value. The $K$ values for all three values are below:

    \begin{enumerate}
        \item Force vs. Current - $K \approx 0.01167$
        \item Force vs. Length of Wire - $K \approx 0.01128$
        \item Force vs. Number of Magnets - $K \approx 0.01214$
    \end{enumerate}

    When these $K$ values are used on graphs that contain different data from the one used to compute the particular $K$ value, the correlation is on average $0.995$, showing that these values are generalizable.

    \item I graphed $F$ vs. $N \cdot I \cdot L$ for all the data I collected and found the slope of the best fit line for this data.

\end{enumerate}
\begin{wrapfigure}{l}{0.7\textwidth}
    \centering
    \caption{Graph of $F vs. N \cdot I \cdot L$}
    \includegraphics[width=0.7\textwidth]{k_graph.png}
\end{wrapfigure}

The equation of best fit for this graph was $F = 0.01162 \frac{N}{amp \cdot m} \cdot NIL (Amp \cdot m)$. Using this method, $K = 0.01162 \frac{N}{amp \cdot m}$. If we compare all four $K$ values, we see that they are all extremely similar, with variations starting to develop only around the 3rd significant figure. This $K$ value is also generalizable to the other graphs, as it achieved correlation values of $0.996$. \\

$K$ is not a universal constant; it would primarily depend on the strength of the magnetic field generated. The strength of the magnetic field also depends on how close to the wire the magnet. The closer the magnet is to the wire, the larger the magnitude of the magnetic field and the larger the force exerted by the wire onto the magnet are.

\section{Additional Questions}

\begin{enumerate}
    \item Using the slap rule, we can find the direction of the force at various points in the wire. Figure 5 depicts this.

    \parbox{\linewidth}{\centering
    \begin{wrapfigure}{l}{0.5\textwidth}
        \centering
        \caption{}
        \includegraphics[width=0.4\textwidth]{force_on_wire_additional_question_2.png}
    \end{wrapfigure}

    As we can see, at the vertical points in the wire, the direction of the force is horizontal. This means that these forces will not affect the scale reading as they are not oriented up or down. Secondly, the forces created by the vertical parts of the wire will always cancel out, as the amount of current and strength of the magnetic field will be the same at corresponding points. This means that should there be minor set-up errors in which the plastic board is not fully vertical, any component with a direction up or down will be irrelevant. This is the motivation behind solely measuring the horizontal portion of the wire.
    }
    \linebreak % \linebreak \linebreak \linebreak %\linebreak \linebreak \linebreak

    \item Newton's 3rd law states that $F_A = -F_B$, where $A$ and $B$ are the magnet and wire, respectively. Using this, we can conclude that in order for there to be a downward force on the magnet exerted by the wire, there needs to be an upward force exerted by the magnet onto the wire. The wire will exert a downward force onto the magnet in response. Using our magnetism rules, we know that the magnetic force can only point upwards if the current is going towards the right ($\rightarrow$).

    \item The amount of force that needs to be exerted upwards onto the wire by the magnet needs to be equivalent to 6.7 grams in Newtons. Using $F = ma$, we get 6.7 grams = $\approx 0.066$ Newtons. In prior parts, we used to our data to calculate $K$. I will be using $K = 0.01162 \frac{N}{amp \cdot m}$, but since the values of $K$ calculated are so similar, either one is fine.

    Using $F = K \cdot I \cdot L \cdot N$, we can solve for $I$ to get $I = \frac{F}{K \cdot L \cdot N}$. Using our value of $K$ and the givens from the question, we get $I = \frac{0.066}{0.01140 \cdot 0.048 \cdot 4} = \approx 30$ Newtons.

\end{enumerate}

\end{document}


\documentclass[12pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{lmodern}
\usepackage[left=2cm,right=2cm,top=2cm,bottom=2cm]{geometry}
\begin{document}
\title{Conjecture about tautologies, contingencies
and contradictions}
\author{Günter Hofer}
\maketitle
\newpage
\paragraph{Introduction}
The satisfiability problem is not easy. With n variables there are $2^n$ possibilities in propositional logic. But when you only need to know if a given Boolean formula
is satisfiable, there is a simple method.

\[A \land B := A \times B\]
\[A \lor B 		:= A + B - A \times B\]
\[\lnot A := 1 - A\]
\[A \supset B := 1 - (A \times (1 - B))\]
\[A \equiv B := (A \supset B) \times (B \supset A) \]
\[A \oplus B := 1 - (A \equiv B)\]
\[A \bar{\land} B := 1- A \times B\]
\[A \bar{\lor}B := 1 - (A + B - A \times B)\]
\paragraph{}
A and B are real numbers in [0,1] .
\paragraph{}
Here is a J program.
\begin{verbatim}
NB. logic functions as arithmetic formulas
not =: 3 : '1-y'
and =: 4 : 'x*y'
or =: 4 : 'x+y-x*y'
imp =: 4 : '1-(x*(1-y))'
eqv =: 4 : '(x imp y) * (y imp x)'
xor =: 4 : '1 - (x eqv y)'
nand =: 4 : '1-x*y'
nor =: 4 : '1-x+y-x*y'
\end{verbatim}

\paragraph{Tautology Conjecture}
If and only if all results, whereby at least one variable has at least 2 different values, are above 0.5 it can be a tautology.

\paragraph{Contradiction Conjecture}
If and only if all results, whereby at least one variable has at least 2 different values, are under 0.5 it can be a contradiction.

\paragraph{Contingency Conjecture}
If neither the Tautology Conjecture nor the Contradictions Conjecture applies, then it is satisfiable but not a tautology.
\newpage
\paragraph{Examples}

It is assumed that the above J program is loaded.
\begin{verbatim}
  a=.b=.c=.d=.0.5
   ((a or b) and (b imp c)) or (c imp d) or (c imp b)
0.972656
   a=.0 1
   ((a or b) and (b imp c)) or (c imp d) or (c imp b)
0.960938 0.984375
   a=.a,a
   b=.0 0 1 1
   a
0 1 0 1
   b
0 0 1 1
   ((a or b) and (b imp c)) or (c imp d) or (c imp b)
0.875 1 1 1
   a=.0
   b=.0
   c=.0 1
   ((a or b) and (b imp c)) or (c imp d) or (c imp b)
1 0.5
NB. Because of the 0.5 you know it is a contingency
   c=.1
   d=. 0 1
   ((a or b) and (b imp c)) or (c imp d) or (c imp b)
0 1
\end{verbatim}

\begin{verbatim}
   a=.b=.0.5
   (a imp (b imp a))
0.875
   a=. 0 1
   (a imp (b imp a))
1 1

   a=.b=.0.5
   (a imp (b imp a))
0.875
   b=.0 1
   (a imp (b imp a))
1 0.75
   b=.1
   a=.0 1
   (a imp (b imp a))
1 1

\end{verbatim}
\end{document}


\documentclass{memoir}
\usepackage{notestemplate}

%\logo{./resources/pdf/logo.pdf}
%\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
%\department{Department of Mathematics}
%\title{Class Notes}
%\subtitle{Based on MATH xxx}
%\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
%\context{Well, I was bored...}
%\date{\today}

\begin{document}

% \maketitle

% Notes taken on 01/29/21

For posterity we restate the definition of a polynomial ring.

\begin{defn}[Polynomial Ring]
	Let \(R\) be a commutative ring. We define a \textbf{polynomial} in \(x\) to be the formal sum
	\begin{align*}
		a_n x^{n} + a_{n-1}x^{n-1} + \ldots + a_1 x + a_0
	\end{align*}
	where \(n\geq 0\) and \(a_i \in R\). If \(a_n \neq 0\), then the polynomial is of \textbf{degree \(n\)}, and \(a_nx^{n}\) is te \textbf{leading term} (\(a_n\) is the \textbf{leading coefficient}). Furthermore, we say the polnyomial is \textbf{monic} if \(a_n=1\).\\

	The set of all such polynomials is called the \textbf{ring of polynomials in \(\R\)} and will be denoted \(R[x]\). We define addition and multiplication by the standard version from algebra:
	\begin{align*}
		(a_nx^{n} + \ldots + a_1x + a_0) + (b_nx^{n} + \ldots + b_1x + b_0) = (a_n+b_n)x^{n} + \ldots + (a_1+b_1)x + (a_0 + b_0)\\
		(a_0+a_1x + a_2x^2+\ldots) \times (b_0 + b_1x + b_2x^2 + \ldots) = a_0b_0 + (a_0b_1 + a_1b_0)x + (a_0b_2 + a_1b_1 + a_2b_0)x^2 + \ldots
	\end{align*}
	That is, the coefficient in the product of \(x^{k}\) is \(\sum_{i=0}^{k} a_i b_{k-i}\).
\end{defn}
Recall that \(\textrm{deg}(p(x)q(x)) = \textrm{deg}(p(x)) + \textrm{deg}(q(x))\) if \(p,q \neq 0\). Furthermore, the units of \(R[x]\) are the units of \(R\), and \(R[x]\) is an integral domain.

\begin{prop}
	Let \(I\triangleleft R\) be an ideal and let \(\left( I \right) = I[x]\) denote the ideal in \(R[x]\) generated by \(I\). Then
	\begin{align*}
		R[x] / (I) \cong (R / I)[x]
	\end{align*}
	and hence if \(I\) is a prime ideal of \(R\), \((I)\) is a prime ideal of \(R[x]\).
\end{prop}
This does not hold for maximal ideals, but \((I,x)\) is maximal in \(R[x]\) if \(I\) is maximal in \(R\).

\begin{defn}[Polynomial Rings over Multiple Variables]
	We inductively define the \textbf{polynomial ring in the variables} \(x_1,x_2,\ldots,x_n\) with coefficients in \(R\) to be
	\begin{align*}
		R[x_1,x_2,\ldots,x_n] := R[x_1,x_2,\ldots,x_n][x_n]
	\end{align*}
	Hence, we can view polynomial rings of multiple variables as polynomial rings on a single variable, with polynomials of \(n-1\) variables as coefficients.
\end{defn}
We say a polynomial is \textbf{homogeneous} if all its terms have the same degree. If \(f\) is a nonzero polynomial in \(n\) variables, the sum of all monomial terms in \(f\) of degree \(k\) is called the \textbf{homogeneous component of \(f\) of degree \(k\)}.

\section{Polynomial Rings over Fields}
\label{sec:polynomial_rings_over_fields}

Let \(R = F\) be a field. We can define a natural norm on \(F[x]\) by
\begin{align*}
	N(p(x)) = \textrm{deg}(p(x)).
\end{align*}
\begin{thm}
	Let \(F\) be a field. The polynomial ring \(F[x]\) is a Euclidean Domain. This implies that if \(a(x),b(x) \in F[x]\) with \(b(x)\) nonzero, then
	\begin{align*}
		a(x) = q(x)b(x) + r(x)
	\end{align*}
	where \(q(x),r(x) \in F[x]\) are unique polynomials, and \(r(x) = 0\) or \(\textrm{deg}(r(x)) < \textrm{deg}(b(x))\).
\end{thm}

\begin{proof}

\end{proof}
Of course, this tells us that \(F[x]\) is a PID and a UID.\\

In fact, the quotient and remainder in the division algorithm are \textit{independent of field extensions}. That is, if \(F\subset E\) is a field extension, then \(b(x)\mid a(x)\) in \(E[x]\) if and only if \(b(x)\mid a(x)\) in \(F[x]\), and \(\textrm{gcd}(a(x),b(x))\) is the same in both fields.

\begin{prop}
	The maximal ideals in \(F[x]\) are the ideals \((f(x))\) generated by irreducible polynomials \(f(x)\). In particular, \(F[x] / (f(x))\) is a field if and only if \(f(x)\) is irreducible.
\end{prop}

\begin{prop}
	Let \(g(x) \in F[x]\) be nonconstant and let
	\begin{align*}
		g(x) = f_1(x)^{n_1}f_2(x)^{n_2}\ldots f_k(x)^{n_k}
	\end{align*}
	be its factorization into irreducibles, where the \(f_i(x)\) are distinct. Then
	\begin{align*}
		F[x] / (g(x)) \cong F[x] / (f_1(x)^{n_1}) \times F[x] / (f_2(x)^{n_2}) \times  \ldots \times F[x] / (f_k(x)^{n_k}).
	\end{align*}
\end{prop}

Notice that if \(f(x)\) has roots \(\alpha_1,\alpha_2,\ldots,\alpha_k\) in \(F\), then \(f(x)\) has \((x-\alpha_1)\ldots(x-\alpha_k)\) as a factor. In other words, a polynomial of degree \(n\) over a field has at most \(n\) roots in \(F\).

\begin{prop}
	A finite subgroup of the multiplicative group of a field is cyclic. In particular, if \(F\) is a finite field, then \(F^{\times }\) is a cyclic group.
\end{prop}
\begin{proof}

\end{proof}

\begin{cor}
	Let \(p\) be a prime. Then \((\Z / p\Z)^{\times }\) of nonzero residue classes \(\pmod{p}\) is cyclic.
\end{cor}

\begin{cor}
	Let \(n\geq 2\) be an integer with factorization
	\begin{align*}
		n = p_1^{\alpha_1}p_2^{\alpha_2}\ldots p_r^{\alpha_r}
	\end{align*}
	with \(p_1,\ldots,p_r\) are distinct. Then
	\begin{align*}
		(\Z / n\Z)^{\times } \cong ( \Z/ p_1^{\alpha_1} \Z)^{\times } \times (\Z / p_2^{\alpha_2}\Z)^{\times } \times \ldots \times (\Z / p_r^{\alpha_r}\Z)^{\times },
	\end{align*}
	in particular \((\Z / 2^{\alpha }\Z)^{\times }\) is the direct product of a cyclic group of order 2 and a cyclic group of order \(2^{\alpha -2}\) for all \(\alpha \geq 2\).\\

	Finally, \((\Z / p^{\alpha }\Z)^{\times }\) is a cyclic group of order \(p^{\alpha-1}(p-1)\) for all odd primes \(p\).
\end{cor}
These describe the group theory structure of the automorphism group of the cyclic group \(\Z_n\), as \(\textrm{Aut}(\Z_n) \cong (\Z / n\Z)^{\times }\).

\begin{proof}

\end{proof}

\section{Polynomial Rings and UFDs}
\label{sec:polynomial_rings_and_ufds}

\begin{defn}[Primitive]
	A polynomial \(f(x) \in R[x]\) is \textbf{primitive} if \( \textrm{gcd}(\left\{ \text{coeff of }f(x) \right\} ) = 1_R\)
\end{defn}
Recall that since \(R\) is an integral domain, one can form its field of fractions by
\begin{align*}
	F:= \textrm{Frac}(R) = \left\{ \frac{r}{s} \mid r,s \in R, \; s \neq 0 \right\}
\end{align*}
\begin{lemma}[Gauss' Lemma]
	Let \(R\) be a UFD with \(F = \textrm{Frac}(R)\).
	\begin{itemize}
		\item If \(f(x),g(x) \in R[x]\) are primitive, then so is \(f(x)\cdot g(x)\).
		\item Take \(f(x) \in R[x]\). Then \(f(x) = \varphi(x) \psi(x) \in F[x]\) with \( \textrm{deg}(\varphi) \textrm{deg}(\psi)\geq 1 \iff f(x) = \psi(x) \varphi(x)\) in \(R[x]\).
	\end{itemize}
\end{lemma}
The elements of the ring \(R\) become units in the UFD \(F[x]\).

\begin{cor}
	Let \(R\) be a UFD. The irreducible elements of \(R[x]\) are of two types:
	\begin{itemize}
		\item nonzero scalar polynomials that are irreducible as elements of \(R\)
		\item primitive polynomials in \(R[x]\) that are irreducible in \(F[x]\)
	\end{itemize}
\end{cor}
Essentially, a polynomial \(p(x)\) with \(\textrm{deg}(p(x))\geq 1\) is irreducible in \(R[x]\) if and only if it is irreducible in \(F[x]\).

\begin{thm}
	\(R\) is a Unique Factorization Domain if and only if \(R[x]\) is a Unique Factorization Domain.
\end{thm}
\begin{proof}

\end{proof}
By induction, it holds that \(R[x_1,x_2,\ldots,x_n]\) is a UFD if and only if \(R\) is a UFD.

\end{document}

Abstract Algebra - Introductory/Algebra II/Notes/source/Lecture3 - GaussLemma_IntroPolyRings.tex


\section{Statistical Mechanics}
This is a brief summary of statistical mechanics for the PGRE and covers the bare essentials of this topic.
Most of the subtopics here are frequently tested, but the section over solids is mostly to get a qualitative understanding of energy levels and so on.
The most important topic, I feel, is the canonical ensemble and its partition function.
Again, it's a good idea to flesh out this section with your own graphs and diagrams.

\subsection{Theorem of Equipartition of Energy}
Each degree of freedom which contributes a quadratic term to the total energy has an average energy \(\frac{1}{2}k_BT\) and contributes \(\frac{k_B}{2}\) to the heat capacity (at constant volume, \(C_V\))\\\\*
For diatomic molecule:\\*
3 translational degrees of freedom: \(H_{trans}=\frac{1}{2}m\left(\dot{x}^2+\dot{y}^2+\dot{z}^2\right)\)\\*
2 vibrational degrees of freedom: \(H_{vib}=\frac{1}{2}m\left(\dot{x}^2+\omega^2x^2\right)\)\\*
2 rotational degrees of freedom: \(H_{rot}=\frac{1}{2}I\left(\omega_1^2+\omega_2^2\right)\) (other rotational axis has extremely small moment of inertia)\\*
Total degrees of freedom: 7

\subsection{Gases}

\subsubsection{Maxwell-Boltzmann Speed Distribution Function}
\(\displaystyle N_v=4\pi N\left(\frac{mv^2}{2\pi k_BT}\right)^{3/2}e^{-mv^2/2k_BT}\)

\subsubsection{Velocities}
Root-Mean Square velocity:\\*
\(\displaystyle K_{trans}=\frac{1}{2}mv_{rms}^2=\frac{3}{2}k_BT\rightarrow v_{rms}=\sqrt{\frac{3k_BT}{m}}\)\\\\*
Average Speed:\\*
\(\displaystyle \bar{v}=\sqrt{\frac{8k_BT}{\pi m}}\)\\\\*
Most Probable Speed:\\*
\(\displaystyle v_{mp}=\sqrt{\frac{2k_BT}{m}}\)\\\\*
Relationship Between Speeds:\\*
\(v_{rms}>\bar{v}>v_{mp}\)

\subsection{Radiation}
\subsubsection{Stefan's Law}
\(P=\sigma A e T^4\), where \(P\) is power,  \(A\) is the surface area, and \(e\) is the emissivity (the fraction of incoming radiation that the surface absorbs)\\*
If the surroundings are at \(T_0\) then \(P=\sigma A e (T^4-T_0^4)\)

\subsubsection{Blackbody Radiation}
\(e=1 \rightarrow P=\sigma A T^4\)\\*
Wien's displacement law: \(\lambda_{max} T\approx.003(mK)\), where \(\lambda_{max}\) is the maximum wavelength of light emitted from a blackbody at temperature \(T\)

\subsubsection{Ideal Reflector}
\(e=0\rightarrow P=0\)

\subsection{The Canonical \& Grand Canonical Ensembles}

\subsubsection{The Canonical Ensemble}
Partition function: \(\displaystyle Z=\sum_i{g_ie^{-E_i/k_BT}}\), where \(g_i\) is the degeneracy of state \(i\)\\*
Probability of system to be in state \(i\): \(\displaystyle p_i=\frac{g_ie^{-E_i/k_BT}}{Z}\)\\*
Ratio of probabilities: \(\displaystyle \frac{p_i}{p_j}=\frac{g_i}{g_j}e^{-(E_i-E_j)/k_BT}\)\\*
Entropy: \(\displaystyle S=-k_B\sum_i{p_i\ln{p_i}}\)\\*
Helmholtz free energy: \(F=-k_BT\ln{Z}\)\\*
Entropy from free energy: \(\displaystyle S=-\left(\frac{\partial F}{\partial T}\right)_V=k_B\ln{Z}+k_BT\frac{\partial\ln{Z}}{\partial T}\)\\*
Average internal energy: \(\displaystyle \bar{U}=\sum_i{p_iE_i}=k_BT^2\left(\frac{\partial\ln{Z}}{\partial T}\right)_V=k_B\frac{T^2}{Z}\left(\frac{\partial Z}{\partial T}\right)_V\)

\subsubsection{The Grand Canonical Ensemble}
Grand partition function: \(\displaystyle \Xi=\sum_i{g_ie^{-(E_i-\mu N_i)/k_BT}}\)\\*
Probability of system to be in state \(i\): \(\displaystyle p_i=\frac{g_ie^{-(\epsilon_i-\mu)/k_BT}}{\Xi}\)\\*
Grand potential: \(\Phi_G=-k_BT\ln{\Xi}=\bar{U}-\mu\bar{N}-TS\)\\*
Thermodynamic quantities:\\*
\(\displaystyle S=-\left(\frac{\partial\Phi_G}{\partial T}\right)_{V,\mu}\), \(\displaystyle P=-\left(\frac{\partial\Phi_G}{\partial V}\right)_{T,\mu}\), and \(\displaystyle \bar{N}=-\left(\frac{\partial\Phi_G}{\partial \mu}\right)_{V,T}\)

\subsection{Number Density}
\(\displaystyle n_V(E)=n_0e^{\frac{-E}{k_BT}}\)

\subsection{Symmetry and Statistics}
If a system starts in a symmetric/anti-symmetric state it must stay in a symmetric/anti-symmetric state.

\subsubsection{Symmetric State}
\(\psi(x_1,x_2)=\psi(x_2,x_1)\)\\\\*
Bosons are symmetric (photons, mesons, \(^4\)He).\\*
e.g.: \(\psi_{Bose}(x_1,x_2)=\phi_i(x_1)\phi_j(x_2)+\phi_i(x_2)\phi_j(x_1)\)\\\\*
%
Bose-Einstein distribution function:\\*
Distribution function: \(\displaystyle f(k)=\frac{1}{e^{(\epsilon(k)-\mu)/k_BT}-1}\), where \(\epsilon(k)-\mu>0\)

\subsubsection{Anti-Symmetric State}
\(\psi(x_1,x_2)=-\psi(x_2,x_1)\)\\\\*
Fermions are anti-symmetric (electrons, neutrinos, protons, \(^3\)He).\\*
e.g.: \(\psi_{Fermi}(x_1,x_2)=\phi_i(x_1)\phi_j(x_2)-\phi_i(x_2)\phi_j(x_1)\)\\\\*
%
Fermi-Dirac distribution function:\\*
\(\displaystyle\epsilon(k)=\frac{\hbar^2k^2}{2m}\)\\*
Distribution function: \(\displaystyle n(k)=\frac{1}{e^{(\epsilon(k)-\mu)/k_BT}+1}\)\\*
At high temperatures this reverts back to a Boltzmann distribution: \(\displaystyle n(k)\rightarrow e^{-(\epsilon(k)-\mu)/k_BT}\)\\*
At low temperatures \(n(k)\rightarrow1\)
%
\subsubsection{Fermi Gas}
\(n=\frac{\bar{N}}{V}\)\\*
Fermi wave number: \(k_F=\left(3\pi^2n\right)^{1/3}\)\\*
Fermi energy: \(\displaystyle E_F=\frac{\hbar^2k_F^2}{2m}\)\\*
Fermi temperature: \(\displaystyle T_F=\frac{E_F}{k_B}\)\\*
Fermi velocity: \(\displaystyle v_F=\frac{\hbar k_F}{m}\)\\\\*
%
In the high temperature limit: \(P=nk_BT\)\\\\*
When \(T>T_F\), \(\displaystyle n(k)\rightarrow e^{-(\epsilon(k)-\mu)/k_BT}\)\\*
When \(T\ll T_F\), one can assume the system is in its ground state; all electrons have energies less than or equal to the Fermi energy\\*
At \(T=0\): \(\displaystyle P=\frac{2nE_F}{5}\) and \(\displaystyle \bar{U}=\frac{3E_F}{5}\)\\*
At low \(T\): \(\displaystyle C_V=\frac{Nk_BT^2}{2}\left(\frac{k_BT}{E_F}\right)\)\\\\*
%
Review what these different distributions look like and their relationships between each other.

\subsection{Statistical Models of Solids}

\subsubsection{Basics}
Energy levels in a solid form a band structure.\\*
Review energy level diagrams for metals, insulators, and semiconductors\\\\*
\(n\)-type semiconductors: impurity atoms are \emph{donors} of electrons\\*
\(p\)-type semiconductors: impurity atoms are \emph{acceptors} of electrons\\*
Review energy level diagrams for \(n\) and \(p\)-type semiconductors\\\\*
Effective electron mass: \(\displaystyle m^*=\frac{\hbar^2}{\left(\frac{\mathrm{d}^2E}{\mathrm{d}k^2}\right)}\)

\subsubsection{Einstein's Model of Vibrations in a Solid}
Atoms are treated as SHO's.\\*
Every atom oscillates at the same frequency (Einstein frequency \(\omega_E\)).\\\\*
\(\displaystyle C_V=3Nk_B\left(\frac{\hbar\omega_E}{k_BT}\right)^2\frac{e^{\hbar\omega_E/k_BT}}{\left(e^{\hbar\omega_E/k_BT}-1\right)^2}\)\\\\*
when \(k_BT\gg\hbar\omega_E\rightarrow C_V=3Nk_B\)

\subsubsection{Debye's Model}
Atoms are treated as SHO's.\\*
Atoms oscillate within a range frequencies.\\\\*
Developed by considering the speed of sound in a material: \(\displaystyle \frac{3}{\bar{s}^3}=\frac{1}{\bar{s}_L^3}+\frac{2}{\bar{s}_T^3}\)\\*
\(\bar{s}\) is the average speed of sound, \(L\) means longitudinal, and \(T\) means traverse\\*
Debye frequency: \(\displaystyle\omega_D=\bar{s}\left(\frac{6\pi^2N}{V}\right)^{1/3}\)\\*
Debye energy: \(E_D=\hbar\omega_D\)\\\\*
\(\displaystyle C_V=\frac{2\pi^2k_B^4T^3V}{5\hbar^3\bar{s}^3}\)


\XtoCBlock{TF2}
\label{block:TF2}
\begin{figure}[H]\includegraphics{TF2}\end{figure}

\begin{XtoCtabular}{Inports}
In & Input In(k)\tabularnewline
\hline
\end{XtoCtabular}

\begin{XtoCtabular}{Outports}
Out & Output Out(k)\tabularnewline
\hline
\end{XtoCtabular}

\begin{XtoCtabular}{Mask Parameters}
b2 & b2\tabularnewline
\hline
b1 & b1\tabularnewline
\hline
b0 & b0\tabularnewline
\hline
a1 & a1\tabularnewline
\hline
a0 & a0\tabularnewline
\hline
ts\_fact & Multiplication factor of base sampling time (in integer format)\tabularnewline
\hline
\end{XtoCtabular}

\subsubsection*{Description:}
Second order transfer function:

    G(z) = (b2.z\textsuperscript{2} + b1.z + b0) / (z\textsuperscript{2} + a1.z + a0)

\subsubsection*{Implementations:}
\begin{tabular}{l l}
\textbf{FiP16} & 16 Bit Fixed Point Implementation\tabularnewline
\textbf{FiP8} & 8 Bit Fixed Point Implementation\tabularnewline
\textbf{FiP32} & 32 Bit Fixed Point Implementation\tabularnewline
\textbf{Float32} & 32 Bit Floating Point Implementation\tabularnewline
\textbf{Float64} & 64 Bit Floating Point Implementation\tabularnewline
\end{tabular}

\XtoCImplementation{FiP16}
\index{Block ID!3297}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP16 \tabularnewline
\textbf{ID} & 3297 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & TF2\_FiP16.c \tabularnewline
\textbf{H filename} & TF2\_FiP16.h \tabularnewline
\end{tabular}
\vspace{1ex}

16 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & \tabularnewline
\hline
b1 & \tabularnewline
\hline
b2 & \tabularnewline
\hline
a0 & \tabularnewline
\hline
a1 & \tabularnewline
\hline
sfrb & \tabularnewline
\hline
sfra & \tabularnewline
\hline
in\_old & In(k-1)\tabularnewline
\hline
in\_veryold & In(k-2)\tabularnewline
\hline
out\_old & Out(k-1)\tabularnewline
\hline
out\_veryold & Out(k-2)\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int16         *In;
     int16         Out;
     int16         b0;
     int16         b1;
     int16         b2;
     int16         a0;
     int16         a1;
     int8          sfrb;
     int8          sfra;
     int16         in_old;
     int16         in_veryold;
     int16         out_old;
     int16         out_veryold;
} TF2_FIP16;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_TF2_FiP16.tex}{}{}
\fi
\XtoCImplementation{FiP8}
\index{Block ID!3296}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP8 \tabularnewline
\textbf{ID} & 3296 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & TF2\_FiP8.c \tabularnewline
\textbf{H filename} & TF2\_FiP8.h \tabularnewline
\end{tabular}
\vspace{1ex}

8 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & \tabularnewline
\hline
b1 & \tabularnewline
\hline
b2 & \tabularnewline
\hline
a0 & \tabularnewline
\hline
a1 & \tabularnewline
\hline
sfrb & \tabularnewline
\hline
sfra & \tabularnewline
\hline
in\_old & In(k-1)\tabularnewline
\hline
in\_veryold & In(k-2)\tabularnewline
\hline
out\_old & Out(k-1)\tabularnewline
\hline
out\_veryold & Out(k-2)\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int8          *In;
     int8          Out;
     int8          b0;
     int8          b1;
     int8          b2;
     int8          a0;
     int8          a1;
     int8          sfrb;
     int8          sfra;
     int8          in_old;
     int8          in_veryold;
     int8          out_old;
     int8          out_veryold;
} TF2_FIP8;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_TF2_FiP8.tex}{}{}
\fi
\XtoCImplementation{FiP32}
\index{Block ID!3298}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & FiP32 \tabularnewline
\textbf{ID} & 3298 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & TF2\_FiP32.c \tabularnewline
\textbf{H filename} & TF2\_FiP32.h \tabularnewline
\end{tabular}
\vspace{1ex}

32 Bit Fixed Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & \tabularnewline
\hline
b1 & \tabularnewline
\hline
b2 & \tabularnewline
\hline
a0 & \tabularnewline
\hline
a1 & \tabularnewline
\hline
sfrb & \tabularnewline
\hline
sfra & \tabularnewline
\hline
in\_old & In(k-1)\tabularnewline
\hline
in\_veryold & In(k-2)\tabularnewline
\hline
out\_old & Out(k-1)\tabularnewline
\hline
out\_veryold & Out(k-2)\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     int32         *In;
     int32         Out;
     int32         b0;
     int32         b1;
     int32         b2;
     int32         a0;
     int32         a1;
     int8          sfrb;
     int8          sfra;
     int32         in_old;
     int32         in_veryold;
     int32         out_old;
     int32         out_veryold;
} TF2_FIP32;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_TF2_FiP32.tex}{}{}
\fi
\XtoCImplementation{Float32}
\index{Block ID!3299}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & Float32 \tabularnewline
\textbf{ID} & 3299 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & TF2\_Float32.c \tabularnewline
\textbf{H filename} & TF2\_Float32.h \tabularnewline
\end{tabular}
\vspace{1ex}

32 Bit Floating Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & \tabularnewline
\hline
b1 & \tabularnewline
\hline
b2 & \tabularnewline
\hline
a0 & \tabularnewline
\hline
a1 & \tabularnewline
\hline
in\_old & In(k-1)\tabularnewline
\hline
in\_veryold & In(k-2)\tabularnewline
\hline
out\_old & Out(k-1)\tabularnewline
\hline
out\_veryold & Out(k-2)\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     float32       *In;
     float32       Out;
     float32       b0;
     float32       b1;
     float32       b2;
     float32       a0;
     float32       a1;
     float32       in_old;
     float32       in_veryold;
     float32       out_old;
     float32       out_veryold;
} TF2_FLOAT32;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_TF2_Float32.tex}{}{}
\fi
\XtoCImplementation{Float64}
\index{Block ID!3300}
\nopagebreak[0]
% Implementation details
\begin{tabular}{l l}
\textbf{Name} & Float64 \tabularnewline
\textbf{ID} & 3300 \tabularnewline
\textbf{Revision} & 0.1 \tabularnewline
\textbf{C filename} & TF2\_Float64.c \tabularnewline
\textbf{H filename} & TF2\_Float64.h \tabularnewline
\end{tabular}
\vspace{1ex}

64 Bit Floating Point Implementation

\begin{XtoCtabular}{Controller Parameters}
b0 & \tabularnewline
\hline
b1 & \tabularnewline
\hline
b2 & \tabularnewline
\hline
a0 & \tabularnewline
\hline
a1 & \tabularnewline
\hline
in\_old & In(k-1)\tabularnewline
\hline
in\_veryold & In(k-2)\tabularnewline
\hline
out\_old & Out(k-1)\tabularnewline
\hline
out\_veryold & Out(k-2)\tabularnewline
\hline
\end{XtoCtabular}

% Implementation data structure
\XtoCDataStruct{Data Structure:}
\begin{lstlisting}
typedef struct {
     uint16        ID;
     float64       *In;
     float64       Out;
     float64       b0;
     float64       b1;
     float64       b2;
     float64       a0;
     float64       a1;
     float64       in_old;
     float64       in_veryold;
     float64       out_old;
     float64       out_veryold;
} TF2_FLOAT64;
\end{lstlisting}

\ifdefined \AddTestReports
\InputIfFileExists{\XcHomePath/Library/Control/Doc/Test_TF2_Float64.tex}{}{}
\fi


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{tfrideal}
\section*{\hspace*{-1.6cm} tfrideal}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Ideal TF-representation for given instantaneous frequency laws.
\end{minipage}
\vspace*{.3cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
[tfr,t,f] = tfrideal(iflaws)
[tfr,t,f] = tfrideal(iflaws,t)
[tfr,t,f] = tfrideal(iflaws,t,N)
[tfr,t,f] = tfrideal(iflaws,t,N,trace)
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty tfrideal} generates the ideal time-frequency representation
        corresponding to the instantaneous frequency laws of the components
        of a signal.\\

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
        {\ty iflaws} & {\ty (M,P)}-matrix where each column corresponds to
                 the instantaneous frequency law of an {\ty (M,1)}-signal.
                 These {\ty P} signals do not need to be present at the same time
		 instants. The values of {\ty iflaws} must be
		 between 0 and 0.5\\
        {\ty t}      & time instant(s)      & {\ty (1:M)}\\
        {\ty N}      & number of frequency bins & {\ty M}\\
        {\ty trace}  & if nonzero, the progression of the algorithm is shown
                                          & {\ty 0}\\
     \hline {\ty tfr}    & output time-frequency matrix, of size {\ty (N,length(t))}\\
        {\ty f}      & vector of normalized frequencies\\

\hline
\end{tabular*}
\vspace*{.2cm}

When called without output arguments, a contour plot of {\ty tfr} is
automatically displayed on the screen.
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Example}
\begin{verbatim}
         N=140; t=0:N-1; [x1,if1]=fmlin(N,0.05,0.3);
         [x2,if2]=fmsin(70,0.35,0.45,60);
         if2=[zeros(35,1)*NaN;if2;zeros(35,1)*NaN];
         tfrideal([if1 if2]);
\end{verbatim}
\vspace*{.5cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
{\ty plotifl, plotsid} and all the {\ty tfr*} functions.
\end{minipage}


\documentclass[../main.tex]{subfiles}

\begin{document}
\subsection{Understanding the baseline matrix multiply}

The baseline code does not perform very well. The following performance estimates will be taken as reference for further optimizations, see subsection \ref{1-b}
\begin{table}[H]
	\centering
	\begin{tabular}{ccccc}
		\multicolumn{2}{c}{Latency} & \multicolumn{2}{c}{Interval} & Pipeline\\
		\hline
		min  &   max  &   min  &   max  &   Type  \\
		230331&  230331&  230332&  230332&   none
	\end{tabular}
	\caption{Performance: baseline matrix multiplication algorithm}
	\label{1-a-perf-table}
\end{table}

\begin{table}[H]
	\centering
	\begin{tabular}{lcccc}
		Name      & BRAM\_18K& DSP48E&   FF   &  LUT  \\
		\hline
		DSP              &        -&      -&       -&      -\\
		Expression       &        -&      -&       0&    538\\
		FIFO             &        -&      -&       -&      -\\
		Instance         &        0&      5&     384&    752\\
		Memory           &       16&      -&       0&      1\\
		Multiplexer      &        -&      -&       -&    559\\
		Register         &        -&      -&     779&      -\\
		\hline
		Total            &       16&      5&    1163&   1847\\
		Available        &      280&    220&  106400&  53201\\
		\hline
		Utilization ($\%$)  &        5&      2&       1&      3
	\end{tabular}
	\caption{Resource utilization of the baseline code}
	\label{1-a-resources}
\end{table}

\begin{table}[H]
	\centering
	\begin{tabular}{lcccc}
		Module & Number of instances \\
		\hline
		floating point adder & 1 \\
		floating point multiplier & 1
	\end{tabular}
	\caption{Utilization of multipliers and adders of the baseline code}
	\label{1-a-resources-arithmetic}
\end{table}

\subsection{Pipelining in HLS}
\label{1-b}
\begin{enumerate}
	\item Optimization attempt: pipeline the most inner loop \\
		$\Rightarrow$ speedup of $\approx 2.13$ regarding the max latency compared to the baseline code
	\begin{table}[H]
		\centering
		\begin{tabular}{ccccc}
			\multicolumn{2}{c}{Latency} & \multicolumn{2}{c}{Interval} & Pipeline\\
			\hline
			min  &   max  &   min  &   max  &   Type  \\
			107995&  107995&  107996&  107996&   none
		\end{tabular}
		\caption{Performance: most inner loop pipelined}
		\label{1-b-perf-table-1}
	\end{table}

	\begin{table}[H]
	\centering
	\begin{tabular}{lcccc}
		Name      & BRAM\_18K& DSP48E&   FF   &  LUT  \\
		\hline
		DSP              &        -&      -&       -&      -\\
		Expression       &        -&      -&       0&    557\\
		FIFO             &        -&      -&       -&      -\\
		Instance         &        0&      5&     384&    751\\
		Memory           &       16&      -&       0&      0\\
		Multiplexer      &        -&      -&       -&    579\\
		Register         &        -&      -&     945&     64\\
		\hline
		Total            &       16&      5&    1329&   1951\\
		Available        &      280&    220&  106400&  53200\\
		\hline
		Utilization ($\%$)  &        5&      2&       1&      3
	\end{tabular}
	\caption{Resource utilization: most inner loop pipelined}
	\label{1-b-resources-1}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{cc}
			Module & Number of instances \\
			\hline
			floating point adder & 1 \\
			floating point multiplier & 1
		\end{tabular}
		\caption{Utilization of multipliers and adders: most inner loop pipelined}
		\label{1-a-resources-arithmetic-1}
	\end{table}

	\item Optimization attempt: pipeline the first inner loop \\
		\label{attempt2}
		$\Rightarrow$ speedup of $\approx 14.22 $ regarding the max latency compared to the non optimizeed code
	\begin{table}[H]
		\centering
		\begin{tabular}{ccccc}
			\multicolumn{2}{c}{Latency} & \multicolumn{2}{c}{Interval} & Pipeline\\
			\hline
			min  &   max  &   min  &   max  &   Type  \\
			16194&  16194&  16195&  16195&   none
		\end{tabular}
		\caption{Performance: first inner loop pipelined}
		\label{1-b-perf-table-2}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{lcccc}
			Name      & BRAM\_18K& DSP48E&   FF   &  LUT  \\
			\hline
			DSP              &        -&      -&       -&      -\\
			Expression       &        -&      -&       0&  26003\\
			FIFO             &        -&      -&       -&      -\\
			Instance         &        0&     10&     732&   1462\\
			Memory           &       16&      -&       0&      0\\
			Multiplexer      &        -&      -&       -&   4725\\
			Register         &        -&      -&   24650&   6496\\
			\hline
			Total            &       16&     10&   25382&  38686\\
			Available        &      280&    220&  106400&  53200\\
			\hline
			Utilization ($\%$)  &        5&      4&      23&     72
		\end{tabular}
		\caption{Resource utilization: first inner loop pipelined}
		\label{1-b-resources-2}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{cc}
			Module & Number of instances \\
			\hline
			floating point adder & 2 \\
			floating point multiplier & 2
		\end{tabular}
		\caption{Utilization of multipliers and adders: first inner loop pipelined}
		\label{1-b-resources-arithmetic-2}
	\end{table}

	The \texttt{HLS pipeline} directive for the first inner loop does not completely unroll the most inner loop. The most inner loop can only be parallelized by two units, because the input buffer is stored into two separate BRAM blocks and therefore only provides two separate access channels. The hls tool gives a warning, that there are problems with scheduling the input buffer accesses which is related to this isssue. Optimization of memory accesses will be addressed in subsection \ref{1-c}

	\item Optimization attempt: Attempt \ref{attempt2} and pipelining of all input functions \\
		\label{attempt3}
		$\Rightarrow$ speedup of $\approx 16.64 $ regarding the max latency compared to the baseline code
	\begin{table}[H]
		\centering
		\begin{tabular}{ccccc}
			\multicolumn{2}{c}{Latency} & \multicolumn{2}{c}{Interval} & Pipeline\\
			\hline
			min  &   max  &   min  &   max  &   Type  \\
			13840&  13840&  13841&  13841&   none
		\end{tabular}
		\caption{Performance: first inner loop and input functions pipelined}
		\label{1-b-perf-table-3}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{lcccc}
			Name      & BRAM\_18K& DSP48E&   FF   &  LUT  \\
			\hline
			DSP              &        -&      -&       -&      -    \\
			Expression       &        -&      -&       0&  26037    \\
			FIFO             &        -&      -&       -&      -    \\
			Instance         &        0&     10&     732&   1462    \\
			Memory           &       16&      -&       0&      0    \\
			Multiplexer      &        -&      -&       -&   4793    \\
			Register         &        -&      -&   24611&   6496    \\
			\hline
			Total            &       16&     10&   25343&  38788    \\
			Available        &      280&    220&  106400&  53200    \\
			\hline
			Utilization ($\%$)  &        5&      4&      23&     72
		\end{tabular}
		\caption{Resource utilization: first inner loop and input functions pipelined}
		\label{1-b-resources-3}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{cc}
			Module & Number of instances \\
			\hline
			floating point adder & 2 \\
			floating point multiplier & 2
		\end{tabular}
		\caption{Utilization of multipliers and adders: first inner loop and input functions pipelined}
		\label{1-b-resources-arithmetic-3}
	\end{table}

\end{enumerate}

\subsection{Increasing Pipeline Parallelism by Repartitioning Memories}
\label{1-c}
This optimization step introduces repartitioning of the memories \texttt{in\_buf} and \texttt{weight\_buf}. We group adjacent columns to a total of factor$=8$ blocks. This increases parallelism by a factor 8. Therefore the \texttt{vivado\_hls} tool instantiates eight times more adders and multipliers, which also increases hardware usage.

$\Rightarrow$ speedup of $\approx 46.14 $ regarding the max latency compared to the baseline code
	\begin{table}[H]
		\centering
		\begin{tabular}{ccccc}
			\multicolumn{2}{c}{Latency} & \multicolumn{2}{c}{Interval} & Pipeline\\
			\hline
			min  &   max  &   min  &   max  &   Type  \\
			4992&  4992&  4993&  4993&   none
		\end{tabular}
		\caption{Performance:  optimizations as in \ref{1-b} \ref{attempt3} and memory repartitioning}
		\label{1-c-perf-table}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{lcccc}
			Name      & BRAM\_18K& DSP48E&   FF   &  LUT  \\
			\hline
			DSP              &        -&      -&       -&      -\\
			Expression       &        -&      -&       0&   3309\\
			FIFO             &        -&      -&       -&      -\\
			Instance         &        0&     80&    5604&  11416\\
			Memory           &       36&      -&       0&      0\\
			Multiplexer      &        -&      -&       -&   6324\\
			Register         &        -&      -&   32355&  14129\\
			\hline
			Total            &       36&     80&   37959&  35178\\
			Available        &      280&    220&  106400&  53200\\
			\hline
			Utilization ($\%$)  &       12&     36&      35&     66
		\end{tabular}
		\caption{Resource utilization: optimizations as in \ref{1-b} \ref{attempt3} and memory repartitioning}
		\label{1-c-resources}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{cc}
			Module & Number of instances \\
			\hline
			floating point adder & 16 \\
			floating point multiplier & 16
		\end{tabular}
		\caption{Utilization of multipliers and adders: optimizations as in \ref{1-b} \ref{attempt3} and memory repartitioning}
		\label{1-c-resources-arithmetic}
	\end{table}

	As we can see, the hardware usage does not exeed any limits in this configuration. The next step for memory optimization would be a repartitioning into $16$ groups.
	This allows higher parallelism, but also increases the amount of multipliers and adders by factor $4$. In this configuration, the usage of flipsflops would exeed the maximum by $5\%$. This overhead might be solved by more detailed optimizations during hardware implementation, but here it does not meet our requirements.
	Intermediate partition steps between $8-16$ result in a significant increase of latency, most likely caused by the misaligned memory groupings, which causes additional control hardware.

	\subsection{Amortizing Iteration Latency with Batching}
	The optimization step exploits that initialization overhead does not increase proportionally with increasing batch size. The batch size is now $256$, the largest possible value, as power to two, for given hardware resources.

$\Rightarrow$ speedup of $\approx 92.84 $ regarding the max latency compared to the baseline code and normalized to batch size
	\begin{table}[H]
		\centering
		\begin{tabular}{ccccc}
			\multicolumn{2}{c}{Latency} & \multicolumn{2}{c}{Interval} & Pipeline\\
			\hline
			min  &   max  &   min  &   max  &   Type  \\
			79392&  79392&  79393&  79393&   none
		\end{tabular}
		\caption{Performance:  batch size increased to $256$ and optimizations as in \ref{1-c}}
		\label{1-d-perf-table}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{lcccc}
			Name      & BRAM\_18K& DSP48E&   FF   &  LUT  \\
			\hline
			DSP              &        -&      -&       -&      -\\
			Expression       &        -&      -&       0&   3704\\
			FIFO             &        -&      -&       -&      -\\
			Instance         &        0&     80&    5604&  11416\\
			Memory           &      154&      -&       0&      0\\
			Multiplexer      &        -&      -&       -&   6324\\
			Register         &        -&      -&   32591&  14129\\
			\hline
			Total            &      154&     80&   38195&  35573\\
			Available        &      280&    220&  106400&  53200\\
			\hline
			Utilization ($\%$)  &       55&     36&      35&     66
		\end{tabular}
		\caption{Resource utilization: batch size increased to $256$ and optimizations as in \ref{1-c}}
		\label{1-d-resources}
	\end{table}

\subsection{Extending Batch Size with Tiling}
	Now the batch size increased to $2048$, but the batch is separated into tiles of size $128$. The usage of \texttt{BRAM\_18K} Blocks is now lower, whereas the normalized speedup increased a lot.

	$\Rightarrow$ speedup of $\approx 11507.26 $ regarding the max latency compared to the baseline code and normalized to batch size
	\begin{table}[H]
		\centering
		\begin{tabular}{ccccc}
			\multicolumn{2}{c}{Latency} & \multicolumn{2}{c}{Interval} & Pipeline\\
			\hline
			min  &   max  &   min  &   max  &   Type  \\
			655889&  655889&  655890&  655890&   none
		\end{tabular}
		\caption{Performance: tiling and optimizations as in \ref{1-c}}
		\label{1-e-perf-table}
	\end{table}

	\begin{table}[H]
		\centering
		\begin{tabular}{lcccc}
			Name      & BRAM\_18K& DSP48E&   FF   &  LUT  \\
			\hline
			DSP              &        -&      -&       -&      -\\
			Expression       &        -&      -&       0&   3632\\
			FIFO             &        -&      -&       -&      -\\
			Instance         &        0&     80&    5604&  11416\\
			Memory           &       86&      -&       0&      0\\
			Multiplexer      &        -&      -&       -&   6319\\
			Register         &        -&      -&   32542&  14129\\
			\hline                                              \\
			Total            &       86&     80&   38146&  35496\\
			Available        &      280&    220&  106400&  53200\\
			\hline                                              \\
			Utilization ($\%$)  &       30&     36&      35&     66
		\end{tabular}
		\caption{Resource utilization:tiling and optimizations as in \ref{1-c}}
		\label{1-e-resources}
	\end{table}

	\subsection{Hardware compilation and FPGA testing on the PYNQ}
	Uploading and execution of the Jupyther notebook unfortunately gives the following error:
	\begin{lstlisting}[basicstyle=\tiny]
	TimeoutError                              Traceback (most recent call last)
<ipython-input-3-28b2e4095493> in <module>()
      7 start_t = time()
      8 dma1.transfer((CLASSES+CLASSES*FEAT+BATCH*FEAT)*4, direction=0)
----> 9 dma2.wait()
     10 fpga_time = time()-start_t
     11

/opt/python3.6/lib/python3.6/site-packages/pynq/drivers/dma.py in wait(self, wait_timeout)
    439         with timeout(seconds = wait_timeout, error_message = Error):
    440             while True:
--> 441                 if libdma.XAxiDma_Busy(self.DMAengine,self.direction) == 0:
    442                     break
    443

/opt/python3.6/lib/python3.6/site-packages/pynq/drivers/dma.py in handle_timeout(self, signum, frame)
    173
    174     def handle_timeout(self, signum, frame):
--> 175         raise TimeoutError(self.error_message)
    176
    177     def __enter__(self):

TimeoutError: DMA wait timed out.
	\end{lstlisting}

	\paragraph{\textit{UPDATE: }} This problem is now fixed. The FPGA was programmed in order to receive the offset values and the full weight matrix in each iteration, whereas this is only required once in the beginning.
	The implementation on the FPGA gives a speedup of $6.74$. The validation error $14.79\%$ is as good as on the CPU.

\end{document}


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[margin=1in]{geometry} % handle page geometry

\input{../header2.tex}

\title{Phys 220A -- Classical Mechanics -- Lec07}
\author{UCLA, Fall 2014}
\date{\formatdate{23}{10}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\setlength{\unitlength}{1mm}
\maketitle

%\section{Lagrange points}
%Placed in Lec06 material

\section{Parametric resonance}
Question: If a swing has a stable equilibrium how can you make it swing higher and higher?

Let's look at a simple example where we set the mass equal to 1
\begin{equation}
L = \frac{1}{2} \dot{q}^2 - \frac{1}{2} \omega(t)^2 q^2
\end{equation}
where $\omega(t + T) = \omega(t)$, i.e. is a periodic function. The simplest example of this is
\begin{equation}
\omega(t)^2 = \omega_0^2 \left( 1 + \epsilon \cos\left(\frac{2\pi t}{T}\right)\right)
\end{equation}
where $\epsilon$ is small. Let's now assume that $T = 2\pi$

Our equation of motion is thus
\begin{equation}
\ddot{q} + \omega^2(t) q = 0
\end{equation}
We can rewrite this as a first order system
\begin{equation}
x_1 = q
\end{equation}
\begin{equation}
x_2 = \dot{q}
\end{equation}
and
\begin{equation}
\dot{x_1} = x_2
\end{equation}
\begin{equation}
\dot{x_2} = -\omega^2(t) x_1
\end{equation}
with
\begin{equation}
\v{x} = \left( \begin{array}{c} x_1 \\ x_2\end{array}\right)
\end{equation}
Then our equation becomes
\begin{equation}
\dot{\v{x}} = M(t) \v{x}
\end{equation}
where $M(t)$ is also a periodic function. The solution is a matrix integral
\begin{equation}
\v{x(t)} = P \int_0^z dt M(t) \v{x}(0)
\end{equation}
But we know that the solution is periodic
\begin{equation}
x(t + 2\pi) = A\v{x}(t)
\end{equation}
The long time development goes as
\begin{equation}
x(t + 2\pi n) = A^n\v{x}(t)
\end{equation}
This is a generalized 2 by 2 matrix
\begin{equation}
\lambda^2 - Tr(A) \lambda + Det(A) = 0
\end{equation}
Can show that
\begin{equation}
\lambda_\pm = \frac{\hbar}{2} \pm \sqrt{ \frac{(\hbar A)^2}{4}  - 1}
\end{equation}
This gives us a stability condition of
\begin{equation}
\hbar A < 2
\end{equation}
For a general $\omega$ the analysis is difficult, but we can consider a small perturbation and look at the unperturbed case
\begin{equation}
\omega^2(t) = \omega_0^2
\end{equation}
\begin{equation}
\dot{x_1} = x_2
\end{equation}
\begin{equation}
\dot{x_2} = -\omega^2 x_1
\begin{equation}
\begin{equation}
M = \left( \begin{array}{cc} 0 & 1 \\ \omega_0^2 & 0  \end{array} \right)
\end{equation}
\begin{equation}
A(t) = exp(Mt) = \left( \begin{array}{cc} \cos\omega_0 & \frac{\sin\omega t}{\omega_0} \\ -\omega_0^2\sin\omega_0 t & \cos\omega_0t  \end{array} \right)
\end{equation}
\begin{equation}
A(t=2\pi) = \left( \begin{array}{cc} \cos2\pi\omega_0 & \frac{\sin2\pi\omega}{\omega_0} \\ -\omega_0^2\sin2\pi\omega_0  & \cos2\pi  \end{array} \right)
\end{equation}
We get
\begin{equation}
\omega_0 = 0, \frac{1}{2}, 1, \frac{3}{2}
\end{equation}
When one is away from the $\omega_0$ then the motion is stable even for small $\epsilon$. The $\omega_0$ are the frequencies of parametric resonance.

\section{Legendre transform (for Hamiltonian mechanics)}

Given a function $f(x)$, we say that it is convex if $f''(x) > 0$. If it is, then we will have a Legendre transform
\begin{eqn}
\mathcal{L} : f(x) \mapsto (\mathcal{L}f)(p)
\end{eqn}
taking $f(x)$ from position space to a function $(\mathcal{L}f)(p)$ in momentum space. Define
\begin{eqn}
F(p,x) = px - f(x),
\end{eqn}
which we picture as the difference between a line of slope $p$ and the convex function $f(x)$. Then the Legendre transformed function is defined as
\begin{eqn}
(\mathcal{L}f)(p) = \sup_{x \in \reals} F(p,x).
\end{eqn}
Put another way, given $p$ there exists some $x(p)$ which maximizes $F(p,x)$, i.e.
\begin{eqn}
\eval[3]{\pd{F}{x}(p,x)}_{x = x(p)} = 0,
\end{eqn}
Then we have
\begin{eqn}
\mathcal{L}f(p) = F(p,x(p)).
\end{eqn}

[missed examples here: $f(x) = ax^2$ and $f(x) = x^a / a$]

An important property of Legendre transforms is that they are \textit{involutions}, that is the Legendre transform squares to the identity. In other words,
\begin{eqn}
\mathcal{L}^2 f(x) = \mathcal{L}(\mathcal{L}f)(x) = f(x).
\end{eqn}
Another useful property is \textit{Young's inequality},
\begin{eqn}
px - f(x) \leq \mathcal{L}f(p).
\end{eqn}

\end{document}