\chapter{Results}
\label{chap:results}
We expect that the \gls{wdt} method will improve the statistics of
Serpent 2 calculations. With normal delta-tracking, virtual collisions
provide no statistical benefit, and are therefore an inefficient use
of computational resources. Virtual collisions that result in
absorption events do contribute to statistics when using
\gls{wdt}. Therefore, we expect an improvement in the results of a
Serpent 2 simulation. In this section, we will describe how
performance in Serpent 2 is quantified, describe the three test cases,
and the results of those test cases.

\section{Performance metrics}
\label{sec:fom}

\subsection{Figure of Merit}
\label{sec:fom}

Monte Carlo codes such as Serpent 2 run many iterations of a single
simulation, calculating the mean of values of interest $\hat{x}$
across many runs. All calculations done in this study were criticality
source calculations. For these types of calculations in Serpent 2,
source neutrons are run in cycles. Each cycle contains the same number
of source neutrons, and the source distribution is determined by the
previous cycle~\cite{jaakko2015}.
% does serpent use batches if fission isn't involved?
% maybe define a batch?
% is this across many batches, or many simulated particles?
By the central limit theorem, we know that the means across these many
simulations will form a normal distribution with variance
$\sigma^2(\hat{x})$. We are interested in reducing the variance of the
final value, which will provide more confidence in the simulation
results. In general, running the simulation more times with more
particle histories will reduce this variance at the cost of requiring
more computation time. The variance depends on the number of
particle histories, which is proportional to the number of cycles $n$:
% Is this cycles, or number of particles? How is that related to batches?
\begin{equation}
\label{eq:variance}
  \sigma^2(\hat{x}) \propto \frac{1}{n}\:.
\end{equation}
Cycles will be used in this report in the place of particle histories
because this value is easily drawn from the Serpent 2 output files.

Serpent 2 reports the standard deviation of the mean,
$\sigma(\hat{x})$, which is proportional to $n^{-1/2}$. The standard deviation
plotted against cycles is shown in Fig.~\ref{fig:error_convergence} on
a log-log plot. As expected, we see a linear evolution with a slope of
1/2 once enough cycles have been executed to give good statistical calculations.
\begin{figure}[hbtp]
  \centering
  \includegraphics[scale=0.5]{images/error_convergence}
  \caption[Variance in $\phi_{\infty}$ for the fast-neutron group as a
  function of Serpent 2 cycle.]{Variance in the infinite flux
    $\phi_{\infty}$ for the fast-neutron group as a function of
    Serpent 2 cycle, with \gls{wdt} threshold of 0.2.}
  \label{fig:error_convergence}
\end{figure}
% you haven't defined \phi_{\infty}
Per the central limit theorem and Eq.~\eqref{eq:variance}, we can
always continue running more cycles to improve the variance. A more
efficient simulation should provide better statistics, resulting in a
lower variance in less time. The variance is therefore dependent on
two things: the number of cycles we run, and the efficiency of our
algorithm.

% longer, or with more particles/cycles?
% I think this section needs a cleanup on precision of word choice about all of this.

For this study, we are interested in the efficiency of the \gls{wdt}
algorithm. We therefore need to establish a standard measuring device,
a \gls{fom}, that is independent of the number of cycles, $n$. This
will then allow us to utilize the variance to directly measure of the
efficiency of our algorithm. The standard \gls{fom} used by most
simulations, described by Lewis and Miller~\cite{lewis1993}, is shown
in Eq.~\eqref{eq:fom}:
\begin{equation}
  \label{eq:fom}
  \mathrm{FOM} = \frac{1}{\sigma(\hat{x})^2T}\:,
\end{equation}
where $T$ is the runtime of the simulation and is proportional to the number
of cycles $T \propto n$. Plugging this and
Eq.~\eqref{eq:variance} into Eq.~\eqref{eq:fom} yields:
\begin{equation*}
  \mathrm{FOM} = \frac{1}{\sigma(\hat{x})^2T} =
  \frac{n}{C_1}\cdot\frac{1}{C_2\cdot n} = C_3\:,
\end{equation*}
where $C_1,C_2,C_3$ are constants.

As we can see, we expect the
\gls{fom} to be a constant value, independent of the number of cycles
$n$.  A higher \gls{fom} indicates higher accuracy per computation
time, and therefore a more efficient algorithm. Serpent 2 collects
scores for each cycle (or batch of cycles) and outputs the sample mean
and standard deviation~\cite{VTT-R-00371-14}. This enables comparison of Serpent 2
running with and without \gls{wdt} to determine the efficiency of the
algorithm.

\subsection{Convergence}
\label{sec:convergence}

As discussed in the previous section, the \gls{fom} should be
independent of cycle number $n$. In reality, it takes enough cycles
for good statistics to develop for the \gls{fom} to converge to its
final value. As seen in Fig.~\ref{fig:fom_convergence}, the \gls{fom}
begins to converge around $1\times 10^4$ cycles in this example.
% why are we inverting and squaring the fom? how is that related to
% this plot?
% Sentence removed, it explained nothing and just confused things-JSR
It may take many cycles for the
\gls{fom} to converge to a single value. Therefore, we calculate an
average value using a selected number of the final data points.
\begin{figure}[hbtp]
  \centering
  \includegraphics[scale=0.5]{images/fom_convergence_example}
  \caption[Figure of merit in $\phi_{\infty}$ for the fast-neutron group as a
    function of Serpent 2 cycle.]{Figure of merit in $\phi_{\infty}$ for the fast-neutron group as a
    function of Serpent 2 cycle, with \gls{wdt} threshold of 0.2.}
  \label{fig:fom_convergence}
\end{figure}

\section{Analysis Tools}
\label{sec:analysis}

Serpent 2 generates a single output file as the simulation runs. The
parameters of interest are overwritten as their mean value and
standard deviation are updated. This is sufficient to calculate the
final value of the \gls{fom}, which gives an indication of the quality
of the simulation. In addition to this, figures such as those shown in
Sec.~\ref{sec:fom} are useful to verify that the \gls{fom} has in fact
converged, and that the variance is reducing as expecting. Therefore,
in addition to changing the Serpent 2 source code to include the
\gls{wdt} method, it was modified to generate uniquely named output
files at the end of each batch. Processing these files enables us to
generate variance and FOM convergence graphs.

As seen in Fig.~\ref{fig:fom_convergence}, the final \gls{fom} is not
a single value, but may oscillate for a large number of
cycles. Therefore, we calculated an average \gls{fom} value for each
run, to allow comparison of runs without requiring a visual inspection
of each graph. To support processing the large amount of data, we
developed a python package, \verb|WDT_Analysis|.

\subsection{The \texttt{WDT\_Analysis} package}
\label{sec:wdt_analysis}

As described above, we modified the Serpent 2 source code to generate
uniquely named output files at various cycle values. The
\verb|WDT_Analysis| package\footnote{Available on GitHub
  \url{https://github.com/jsrehak/WDT_Analysis}} leverages python's
object oriented
% I'd maybe include a footnote with a link to the github, or something like that
% Whatever is the best way for someone to be able to find the package documentation.
programming structure to enable easy analysis of the data. We utilize
a module, \verb|fom|, to analyze the convergence of \gls{fom} across
many Serpent 2 output files. A summary
of each of the major objects used by the \verb|fom| module are outlined in
Table.~\ref{tab:wdt_analysis}.
\begin{table}[hbtp]
  \centering
  \caption{Object structure of the \texttt{WDT\_Analysis} package.\label{tab:wdt_analysis}}
  \begin{tabular}{ll} \toprule
   \textbf{Object} & \textbf{Description} \\ \midrule
    \verb|DataFile| & Contains all the data from a single Serpent 2
                      output file. \\
    \verb|Analyzer| & Contains all the \verb|DataFile| objects
                       from a run of Serpent 2. \\
    \verb|Comparator| & Compares the \gls{fom} convergence for one or
                        more \verb|Analyzer| objects. \\
    \bottomrule
  \end{tabular}
\end{table}
Detailed use of the package is provided in the package
documentation. Use of the package makes analysis of the \gls{fom}
convergence simple:
\begin{enumerate}
\item Run desired Serpent 2 simulations.
\item Place all output files from each desired run in separate directories
\item Initialize a \verb|Comparator| object, providing the directories
  and names for the datasets. While initializing, the
  \verb|Comparator| will:
  \begin{enumerate}
  \item Create a \verb|DataFile| object for each serpent output file.
  \item Collect the \verb|DataFile| objects from a single run into a single
    \verb|Analyzer|.
  \item Collect these \verb|Analyzer| objects into the
    \verb|Comparator|.
  \end{enumerate}
\item Use the \verb|compare| and \verb|plot| functions of the
  \verb|Comparator| object to observe the \gls{fom} convergence.
\end{enumerate}
We used this process to compare the \gls{fom} convergence for
different values of the \gls{wdt} threshold for three test cases.

\section{Parameters of Study}
\label{sec:parameters}
Each Serpent 2 simulation generates hundreds of output parameters that
describe all processes that occur during particle propagation. Finding
values where the \gls{wdt} method improves the \gls{fom} is a daunting
task. To focus our search, we identified three  quantities that
will be the focus of this research study. Further study into other
parameters may reveal further benefit or issues with the \gls{wdt}
method. The three parameters we selected are flux,
total cross-section, and the scattering matrices for the
zeroth scattering moment. Each of the three test-cases have
reflective boundary conditions, so we will examine the infinite values for these
parameters.

The neutron flux quantifies the total distance traveled by all
neutrons in a volume per unit time. This is an important quantity, as
it is representative of the neutron population. A low variance is
desired in neutron flux, as it indicates that the uncertainty in the
neutron population in each energy group is well
understood.

The total cross-section quantifies the total interaction probability
for neutrons of that energy range. The zeroth scattering moment is a
matrix that quantifies scattering between the energy groups in the
problem, both upscattering and downscattering. A low variance is
desired in these parameters, as the goal of implementing this method
is to improve cross-section development for a highly scattering
medium, \gls{treat}.

A Serpent 2 simulation was run for each of the three test cases using
values of the \gls{wdt} threshold ($t_\mathrm{wdt}$) from 0.1 to unity in increments of
0.1, maintaining a value of $(1-c) = 0.1$, as described in
Fig.~\ref{fig:ray_wdt}. To determine the effectiveness of \gls{wdt},
we compared the final average \gls{fom} for each value of
$t_{\mathrm{wdt}}$ to the final average \gls{fom} with no \gls{wdt}:
\begin{equation*}
  R(t_{\mathrm{wdt}}) =
  \frac{\overline{\mathrm{FOM}}(t_\mathrm{wdt})}{\overline{\mathrm{FOM}}_0}\:,
\end{equation*}
where $\overline{\mathrm{FOM}}_0 =
\overline{\mathrm{FOM}}(0.1)$ is
the average \gls{fom} with no \gls{wdt}. Increasing values of
$t_{\mathrm{wdt}}$ leads to increasing amount of \gls{wdt} over
regular delta-tracking. The parameters for the rouletting scheme are
kept constant, with a weight cutoff of $w = 0.1$ and probability of
rouletting $P_{\mathrm{kill}} = 0.5$.

\subsection{Test Cases}
\label{sec:test_cases}

Three test cases were selected to test the \gls{fom} values for the
\gls{wdt} method. These are a \gls{pwr} pin cell, a fast reactor pin
cell, and a homogenous fuel element based on the composition of the
\gls{treat} fuel elements at the \gls{inl}. All test cases were run on
the Abacus server at UC Berkeley, the specifications of the server are
shown in Tab.~\ref{tab:abacus}.
\begin{table}[hbtp]
  \centering
  \caption{Abacus server specifications}
  \begin{tabular}{ll}
    \toprule
    \textrm{Parameter} & \textrm{Specification} \\ \midrule
    Processor & 2 x TenCore Intel Xeon Processor E52687W v3 3.10GHz
                 25MB Cache \\
    RAM & 16 x 16GB PC417000 2133MHz DDR4 \\
    Hard drive & 2 x 800GB Intel SATA 6.0Gb/s Solid State Drive
          \\ \bottomrule
  \end{tabular}
  \label{tab:abacus}
\end{table}

\section{\Acrlong{pwr} pin cell}
\label{sec:pwr}
The \gls{pwr} pin cell was chosen from the Serpent 2 validation input
files provided on the \gls{vtt} Serpent
webpage~\cite{serpent_web}. The geometry and physical parameters are
shown in Fig.~\ref{fig:pwr}. The fuel is a 2.68 w/o enriched UO$_2$
mixture, with Zircalloy cladding, light water moderation, and
reflective boundary conditions. We ran the simulation with two energy
groups with the group boundary at 0.625 eV.
\begin{figure}[hbtp]
  \centering
  \begin{tabular}{cc}
  \includegraphics[scale=0.5]{images/results/pwr_geom1} \\
\begin{tabular}{@{}lr@{}}
\toprule
\textbf{Parameter}     & \textbf{Value (cm)} \\ \midrule
Fuel radius     & 0.412          \\
Cladding radius & 0.475          \\
Pitch              & 1.33          \\ \bottomrule
\end{tabular}
  \end{tabular}
  \caption[Pressurized water reactor geometry and physical
  parameters.]{Pressurized water reactor geometry and physical
    parameters. The fuel is shown in light blue, the cladding in
    purple and the coolant is green.}
  \label{fig:pwr}
\end{figure}
\subsection{Infinite Flux}
A summary of the \gls{fom} ratio and
$\overline{\mathrm{FOM}}(t_{\mathrm{wdt}})$, for the \gls{pwr} pin
% I thought the ratio was R, and that
% \overline{\mathrm{FOM}}(t_{\mathrm{wdt}}) was an average, not a
% ratio?
% Changed it to "and", was referencing the two things shown on the
% table -jsr
cell for infinite flux ($\phi_{\infty}$) is shown in Tab.~\ref{tab:pwr_inf_flx} and
shown graphically in Fig.~\ref{fig:pwr_inf_flx}.
\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
    the \acrshort{pwr} pin cell infinite flux.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the \gls{pwr} pin cell \textit{infinite flux}.}
  \begin{tabular}{cc} Fast group ($E > 0.0625$) & Thermal group \\
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ & Ratio\\
\midrule
 0.1 & 1.240281$\times 10^{6}$ & 1.000000 \\
 0.2 & 1.257046$\times 10^{6}$ & 1.013517 \\
 0.3 & 1.262320$\times 10^{6}$ & 1.017769 \\
 0.4 & 1.302585$\times 10^{6}$ & 1.050233 \\
 0.5 & 1.268738$\times 10^{6}$ & 1.022944 \\
 0.6 & 1.304854$\times 10^{6}$ & 1.052063 \\
 0.7 & 1.311428$\times 10^{6}$ & 1.057364 \\
 0.8 & 1.296513$\times 10^{6}$ & 1.045338 \\
 0.9 & 1.306470$\times 10^{6}$ & 1.053366 \\
 1.0 & 1.302047$\times 10^{6}$ & 1.049800 \\
\bottomrule
\end{tabular} &
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ &
                                                                   Ratio\\
\midrule
 0.1 & 1.283594$\times 10^{6}$ & 1.000000 \\
 0.2 & 1.305878$\times 10^{6}$ & 1.017360 \\
 0.3 & 1.335006$\times 10^{6}$ & 1.040053 \\
 0.4 & 1.309221$\times 10^{6}$ & 1.019965 \\
 0.5 & 1.322382$\times 10^{6}$ & 1.030218 \\
 0.6 & 1.321435$\times 10^{6}$ & 1.029481 \\
 0.7 & 1.283419$\times 10^{6}$ & 0.999863 \\
 0.8 & 1.304767$\times 10^{6}$ & 1.016495 \\
 0.9 & 1.339323$\times 10^{6}$ & 1.043416 \\
 1.0 & 1.360193$\times 10^{6}$ & 1.059675 \\
\bottomrule
\end{tabular}
  \end{tabular}

  \label{tab:pwr_inf_flx}
\end{table}
\begin{figure}[hbtp]
  \centering
  \begin{tabular}{c}
  \includegraphics[scale=0.9]{images/results/pwr_inf_flx_grp_1} \\
    (a) Group 1\\
  \includegraphics[scale=0.9]{images/results/pwr_inf_flx_grp_2} \\
    (b) Group 2
  \end{tabular}
  \caption[Figure of merit ratio of the \textit{infinite flux} for the
  PWR]{Figure of merit ratio of the \textit{infinite flux} for the PWR vs the
    \gls{wdt} threshold for the (a) fast and (b) thermal energy
    groups. The group boundary is at $0.0625$ eV. \gls{fom} is
    calculated using the last 20 data points.}
  \label{fig:pwr_inf_flx}
\end{figure}
%
These ratios were calculated using the final 20 data points for each
series, then compared to the base case. For the fast group, \gls{wdt} threshold values
of 0.6 and higher return a consistently higher \gls{fom} ratio. This
behavior is not observed in the thermal group \gls{fom} ratio.

\subsection{Infinite Total Cross-section}
\label{sec:pwr_inf_total}
A summary of the \gls{fom} ratio and
$\overline{\mathrm{FOM}}(t_{\mathrm{wdt}})$, for the \gls{pwr} pin
cell for infinite total cross-section ($\Sigma_{t,
  \infty}$) is shown in Tab.~\ref{tab:pwr_inf_tot} and shown
graphically in Fig.~\ref{fig:pwr_inf_tot}. The \gls{fom} ratio for the
fast group when using \gls{wdt} is always less than unity. The FOM ratio behavior for this quantity is inverted compared to the fast group infinite flux \gls{fom}
ratio. The thermal group shows consistent improvement with increased \gls{wdt}, but
with no meaningful trend. The peak improvement occurs with full
\gls{wdt}, $t_{\mathrm{wdt}} = 1.0$, as with the infinite flux.

\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
  the \acrshort{pwr} pin cell \textit{infinite total cross-section}.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the \gls{pwr} pin cell \textit{infinite total cross-section}.}
  \begin{tabular}{cc} Fast group ($E > 0.0625$) & thermal group \\
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ & Ratio\\
\midrule
 0.1 & 1.467242$\times 10^{7}$ & 1.000000 \\
 0.2 & 1.407526$\times 10^{7}$ & 0.959300 \\
 0.3 & 1.404113$\times 10^{7}$ & 0.956974 \\
 0.4 & 1.419632$\times 10^{7}$ & 0.967552 \\
 0.5 & 1.155054$\times 10^{7}$ & 0.787228 \\
 0.6 & 1.181301$\times 10^{7}$ & 0.805117 \\
 0.7 & 1.191133$\times 10^{7}$ & 0.811818 \\
 0.8 & 1.188149$\times 10^{7}$ & 0.809784 \\
 0.9 & 1.212526$\times 10^{7}$ & 0.826398 \\
 1.0 & 1.199368$\times 10^{7}$ & 0.817430 \\
\bottomrule
\end{tabular} &
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ &
                                                                   Ratio\\
\midrule
 0.1 & 8.065829$\times 10^{6}$ & 1.000000 \\
 0.2 & 8.154616$\times 10^{6}$ & 1.011008 \\
 0.3 & 8.231395$\times 10^{6}$ & 1.020527 \\
 0.4 & 8.172302$\times 10^{6}$ & 1.013200 \\
 0.5 & 8.221818$\times 10^{6}$ & 1.019339 \\
 0.6 & 8.270724$\times 10^{6}$ & 1.025403 \\
 0.7 & 8.280600$\times 10^{6}$ & 1.026627 \\
 0.8 & 8.273215$\times 10^{6}$ & 1.025712 \\
 0.9 & 8.114104$\times 10^{6}$ & 1.005985 \\
 1.0 & 8.670363$\times 10^{6}$ & 1.074950 \\
\bottomrule
\end{tabular}
  \end{tabular}
  \label{tab:pwr_inf_tot}
\end{table}
\begin{figure}[hbtp]
  \centering
  \begin{tabular}{c}
  \includegraphics[scale=0.9]{images/results/pwr_inf_tot_grp_1} \\
    (a) \\
  \includegraphics[scale=0.9]{images/results/pwr_inf_tot_grp_2} \\
    (b)
  \end{tabular}
  \caption[Figure of merit ratio of the \textit{infinite total cross-section} for the
  PWR]{Figure of merit ratio of the \textit{infinite total cross-section} for the PWR vs the
    \gls{wdt} threshold for the (a) fast and (b) thermal energy
    groups. The group boundary is at $0.0625$ eV. \gls{fom} is
    calculated using the last 20 data points.}
  \label{fig:pwr_inf_tot}
\end{figure}

\subsection{Scattering Matrices}
\label{sec:pwr_inf_total}

For the two-group case, the scattering matrices for the zeroth
moment, $S_0$, is $2 \times 2$. A summary of the \gls{fom} ratio
for $S_0$ is shown in Table~\ref{tab:pwr_inf_sp0}. The
tables are for the entry in row $g$ and column $g'$, which represents
scattering from group $g'$ into group $g$, indicated as $g' \to
g$. The \gls{fom} ratios are shown graphically in Fig.~\ref{fig:pwr_inf_sp0}.
% reference to figure?

The \gls{wdt} method under-performs in most cases, except the
inscattering cross-section for the thermal group.

\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
  the \acrshort{pwr} pin cell \textit{infinite $S_0$ scattering matrix}.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the \gls{pwr} pin cell \textit{infinite $S_0$ scattering matrix}}

  \begin{tabular}{cc} $\Sigma_{1\to 1}$ & $\Sigma_{2 \to 1}$ \\
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ & Ratio\\
\midrule
 0.1 & 1.422641$\times 10^{7}$ & 1.000000 \\
 0.2 & 1.358990$\times 10^{7}$ & 0.955258 \\
 0.3 & 1.359954$\times 10^{7}$ & 0.955936 \\
 0.4 & 1.378010$\times 10^{7}$ & 0.968628 \\
 0.5 & 1.135270$\times 10^{7}$ & 0.798002 \\
 0.6 & 1.174407$\times 10^{7}$ & 0.825511 \\
 0.7 & 1.193568$\times 10^{7}$ & 0.838981 \\
 0.8 & 1.191534$\times 10^{7}$ & 0.837551 \\
 0.9 & 1.220204$\times 10^{7}$ & 0.857703 \\
 1.0 & 1.211019$\times 10^{7}$ & 0.851247 \\
\bottomrule
\end{tabular} &
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ &
                                                                   Ratio\\
\midrule
 0.1 & 1.395679$\times 10^{6}$ & 1.000000 \\
 0.2 & 1.366134$\times 10^{6}$ & 0.978831 \\
 0.3 & 1.401562$\times 10^{6}$ & 1.004215 \\
 0.4 & 1.402313$\times 10^{6}$ & 1.004753 \\
 0.5 & 1.190960$\times 10^{6}$ & 0.853319 \\
 0.6 & 1.199284$\times 10^{6}$ & 0.859284 \\
 0.7 & 1.199826$\times 10^{6}$ & 0.859672 \\
 0.8 & 1.190166$\times 10^{6}$ & 0.852750 \\
 0.9 & 1.189501$\times 10^{6}$ & 0.852274 \\
 1.0 & 1.180054$\times 10^{6}$ & 0.845505 \\
\bottomrule
\end{tabular} \\

$\Sigma_{1\to 2}$ & $\Sigma_{2 \to 2}$ \\
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ & Ratio\\
\midrule
 0.1 & 9.700866$\times 10^{3}$ & 1.000000 \\
 0.2 & 9.397108$\times 10^{3}$ & 0.968688 \\
 0.3 & 9.525420$\times 10^{3}$ & 0.981914 \\
 0.4 & 9.411723$\times 10^{3}$ & 0.970194 \\
 0.5 & 7.684674$\times 10^{3}$ & 0.792164 \\
 0.6 & 7.966642$\times 10^{3}$ & 0.821230 \\
 0.7 & 7.659128$\times 10^{3}$ & 0.789530 \\
 0.8 & 7.866229$\times 10^{3}$ & 0.810879 \\
 0.9 & 7.728249$\times 10^{3}$ & 0.796656 \\
 1.0 & 7.551567$\times 10^{3}$ & 0.778443 \\
\bottomrule
\end{tabular} &
\begin{tabular}{lrr}
\toprule
$t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ &
                                                                   Ratio\\
\midrule
 0.1 & 1.395679$\times 10^{6}$ & 1.000000 \\
 0.2 & 1.366134$\times 10^{6}$ & 0.978831 \\
 0.3 & 1.401562$\times 10^{6}$ & 1.004215 \\
 0.4 & 1.402313$\times 10^{6}$ & 1.004753 \\
 0.5 & 1.190960$\times 10^{6}$ & 0.853319 \\
 0.6 & 1.199284$\times 10^{6}$ & 0.859284 \\
 0.7 & 1.199826$\times 10^{6}$ & 0.859672 \\
 0.8 & 1.190166$\times 10^{6}$ & 0.852750 \\
 0.9 & 1.189501$\times 10^{6}$ & 0.852274 \\
 1.0 & 1.180054$\times 10^{6}$ & 0.845505 \\
\bottomrule
\end{tabular}
  \end{tabular}
  \label{tab:pwr_inf_sp0}
\end{table}

% % ==================== PWR S_1 Table

% \begin{table}[hbtp]
%   \centering
%   \caption[$\overline{\mathrm{FOM}}$ and ratio for
%   the \acrshort{pwr} pin cell infinite $S_1$ scattering matrix.]{$\overline{\mathrm{FOM}}$ as a function of
%     $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
%     the \gls{pwr} pin cell infinite $S_1$ scattering matrix}
%   \begin{tabular}{cc} $\Sigma_{1\to 1}$ & $\Sigma_{2 \to 1}$ \\
% \begin{tabular}{lrr}
% \toprule
% $t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ & Ratio\\
% \midrule
%  0.1 & 6.050967$\times 10^{6}$ & 1.000000 \\
%  0.2 & 5.751850$\times 10^{6}$ & 0.950567 \\
%  0.3 & 5.622341$\times 10^{6}$ & 0.929164 \\
%  0.4 & 5.716568$\times 10^{6}$ & 0.944736 \\
%  0.5 & 4.676774$\times 10^{6}$ & 0.772897 \\
%  0.6 & 4.697950$\times 10^{6}$ & 0.776397 \\
%  0.7 & 4.732003$\times 10^{6}$ & 0.782024 \\
%  0.8 & 4.790409$\times 10^{6}$ & 0.791677 \\
%  0.9 & 4.763818$\times 10^{6}$ & 0.787282 \\
%  1.0 & 4.675605$\times 10^{6}$ & 0.772704 \\
% \bottomrule
% \end{tabular} &
% \begin{tabular}{lrr}
% \toprule
% $t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ &
%                                                                    Ratio\\
% \midrule
%  0.1 & 2.116372$\times 10^{5}$ & 1.000000 \\
%  0.2 & 2.139720$\times 10^{5}$ & 1.011032 \\
%  0.3 & 2.070176$\times 10^{5}$ & 0.978172 \\
%  0.4 & 2.105112$\times 10^{5}$ & 0.994680 \\
%  0.5 & 1.720552$\times 10^{5}$ & 0.812972 \\
%  0.6 & 1.761463$\times 10^{5}$ & 0.832303 \\
%  0.7 & 1.684928$\times 10^{5}$ & 0.796140 \\
%  0.8 & 1.711624$\times 10^{5}$ & 0.808754 \\
%  0.9 & 1.676021$\times 10^{5}$ & 0.791931 \\
%  1.0 & 1.649123$\times 10^{5}$ & 0.779222 \\
% \bottomrule
% \end{tabular} \\

% $\Sigma_{1\to 2}$ & $\Sigma_{2 \to 2}$ \\
% \begin{tabular}{lrr}
% \toprule
% $t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ & Ratio\\
% \midrule
%  0.1 & 3.486377$\times 10^{3}$ & 1.000000 \\
%  0.2 & 3.399501$\times 10^{3}$ & 0.975081 \\
%  0.3 & 3.316465$\times 10^{3}$ & 0.951264 \\
%  0.4 & 3.356495$\times 10^{3}$ & 0.962746 \\
%  0.5 & 2.834145$\times 10^{3}$ & 0.812920 \\
%  0.6 & 2.878089$\times 10^{3}$ & 0.825524 \\
%  0.7 & 2.859532$\times 10^{3}$ & 0.820202 \\
%  0.8 & 2.854791$\times 10^{3}$ & 0.818842 \\
%  0.9 & 2.803923$\times 10^{3}$ & 0.804251 \\
%  1.0 & 2.841147$\times 10^{3}$ & 0.814928 \\
% \bottomrule
% \end{tabular} &
% \begin{tabular}{lrr}
% \toprule
% $t_{\mathrm{wdt}}$ &                   $\overline{\mathrm{FOM}}$ &
%                                                                    Ratio\\
% \midrule
%  0.1 & 1.583134$\times 10^{6}$ & 1.000000 \\
%  0.2 & 1.627093$\times 10^{6}$ & 1.027767 \\
%  0.3 & 1.526534$\times 10^{6}$ & 0.964248 \\
%  0.4 & 1.671871$\times 10^{6}$ & 1.056052 \\
%  0.5 & 1.621064$\times 10^{6}$ & 1.023958 \\
%  0.6 & 1.610514$\times 10^{6}$ & 1.017295 \\
%  0.7 & 1.582685$\times 10^{6}$ & 0.999716 \\
%  0.8 & 1.579132$\times 10^{6}$ & 0.997472 \\
%  0.9 & 1.605180$\times 10^{6}$ & 1.013925 \\
%  1.0 & 1.631345$\times 10^{6}$ & 1.030453 \\
% \bottomrule
% \end{tabular}
%   \end{tabular}
%   \label{tab:pwr_inf_sp1}
% \end{table}

% ==================== End SP1 table ================================

\begin{figure}[hbtp]
  \centering
  \begin{tabular}{cc}
  \includegraphics[scale=0.6]{images/results/pwr_inf_sp0_grp_1} &
  \includegraphics[scale=0.6]{images/results/pwr_inf_sp0_grp_2} \\
    (a) & (b) \\
  \includegraphics[scale=0.6]{images/results/pwr_inf_sp0_grp_3} &
  \includegraphics[scale=0.6]{images/results/pwr_inf_sp0_grp_4} \\
    (c) & (d)
  \end{tabular}
  \caption[Figure of merit ratio of the \textit{infinite zeroth-moment of the
  scattering cross-section} for the PWR]{Figure of merit ratio of the
    \textit{infinite zeroth-moment of the scattering cross-section} $S_0$ for
    the PWR vs the \gls{wdt} threshold for the (a) $\Sigma_{1 \to
      1}$, (b) $\Sigma_{2 \to 1}$, (c) $\Sigma_{1 \to 2}$ and (d)
    $\Sigma_{2 \to 2}$ entries. The group boundary is at $0.0625$ eV. \gls{fom} is
    calculated using the last 20 data points.}
  \label{fig:pwr_inf_sp0}
\end{figure}
\newpage
% I think italicizing which thing you're talking about in the captions
% helps make it easier to see what is what--please do for remainder if
% you agree.
%
% Agreed -jsr
\subsection{Discussion}
\label{sec:pwr_discussion}

There is no clear value of the \gls{wdt} threshold that improves
\gls{fom} for all cases. Depending on the desired output parameters of
the simulation, using \gls{wdt} may be beneficial. We observe an
improvement or parity in infinite flux for both groups across all values of
$t_\mathrm{wdt}$. Therefore, we should look to other parameters to
determine a useful value of $t_{\mathrm{wdt}}$.

The infinite total cross-section for the thermal group shows a similar
improvement to the infinite flux
% similarly to infinite flux? unclear
but the fast group data show a consistent under performance
as shown in Fig.~\ref{fig:pwr_inf_tot} and
Tab.~\ref{tab:pwr_inf_tot}. Values of $t_{\mathrm{wdt}} \geq 0.5$
result in the largest reduction, these values have a mean \gls{fom}
ratio of 0.81. Lower values $0.1 \leq t_{\mathrm{wdt}} \leq 0.4$ have
a mean \gls{fom} ratio of 0.92.

We see a similar pattern in the entries in the scattering matrix, as
shown in Fig.~\ref{fig:pwr_inf_sp0} and
Tab.~\ref{tab:pwr_inf_sp0}. The thermal-group inscattering cross-section
has a consistent improvement, while the fast group underperforms. For
the fast group scattering into either group, as with infinite total
cross-section, there are two domains. When
$0.1 \leq t_{\mathrm{wdt}} \leq 0.4$, the ratio is between 0.95 and
unity, and is much lower otherwise. These data, therefore, suggest the
same range of \gls{wdt} threshold as the infinite total
cross-section. For the upscattering cross-section $\Sigma_{2 \to 1}$,
two values within this range return an improvement in \gls{fom} ratio,
$t_\mathrm{wdt} = 0.3, 0.4$.

Overall, the ratio data indicate that a \gls{wdt} threshold of
$t_{\mathrm{wdt}} = 0.3$ or $t_{\mathrm{wdt}} = 0.4$ provides an
improvement in the statistics for thermal neutrons, without much loss
of statistics elsewhere. The final \gls{fom} ratios for these values
are summarized in Tab.~\ref{tab:pwr_results}. If the thermal group
infinite parameters discussed here for a \gls{pwr} are desired with
high accuracy, the \gls{wdt} method provides a means to improve these
statistics without causing a significant impact on the fast group.

\begin{table}[hbtp]
  \centering
  \caption[Summary of \acrshort{fom} ratios for the PWR for select
    cases.]{Summary of \acrshort{fom} ratios for the PWR for select
    cases of \gls{wdt} threshold.}
  \begin{tabular}{cc} $t_{\mathrm{wdt}}=0.3$ &  $t_{\mathrm{wdt}}=0.4$ \\
  \begin{tabular}{lr}\toprule
    \textbf{Parameter}& $\overline{\mathbf{FOM}}$ \textbf{Ratio} \\ \midrule
    $\phi_{1, \infty}$ & 1.017769 \\
    $\phi_{2, \infty}$ & 1.040053 \\
    $\Sigma_{1,t}$ & 0.956974 \\
    $\Sigma_{2,t}$ & 1.020527 \\
    $\Sigma_{1 \to 1}$ & 0.955936 \\
    $\Sigma_{2 \to 1}$ & 1.004215 \\
    $\Sigma_{1 \to 2}$ & 0.981914 \\
    $\Sigma_{2 \to 2}$ & 1.004215 \\ \bottomrule
  \end{tabular}&
  \begin{tabular}{lr}\toprule
    \textbf{Parameter}& $\overline{\mathbf{FOM}}$ \textbf{Ratio} \\ \midrule
    $\phi_{1, \infty}$ & 1.050233 \\
    $\phi_{2, \infty}$ & 1.019965 \\
    $\Sigma_{1,t}$ & 0.967552 \\
    $\Sigma_{2,t}$ & 1.013200 \\
    $\Sigma_{1 \to 1}$ & 0.968628 \\
    $\Sigma_{2 \to 1}$ & 1.004753 \\
    $\Sigma_{1 \to 2}$ & 0.970194 \\
    $\Sigma_{2 \to 2}$ & 1.004753 \\ \bottomrule
  \end{tabular}
  \end{tabular}
  \label{tab:pwr_results}
\end{table}

\newpage
\section{Fast reactor pin cell}
\label{sec:fast_pin_cell}

We adapted the fast reactor pin cell from another example provided in
the Serpent validation files~\cite{serpent_web}. This is a lead cooled
pin cell with \gls{mox} fuel containing uranium, plutonium and a small
amount of americium. The relative isotope amounts are shown in Tab.~\ref{tab:fast_fuel}.
% how much? the enrichment and relatively amounts are probably needed here.
The cladding is stainless steel. The lattice is
hexagonal and has reflective boundary conditions. As with the
\gls{pwr} pin cell, we ran the simulation with two energy
groups with the group boundary at 0.625 eV.

\begin{table}[hbtp]
  \caption{Relative atomic density of the fast reactor pin cell \acrshort{mox}
    fuel.}
  \centering
  \begin{tabular}{cr}\toprule
    Isotope & Relative Atomic Density \\ \midrule
    \ele{U}{238} & 1.00 \\
    \ele{Pu}{239} & 0.16 \\
    \ele{Pu}{240} & 7.40 $\times 10^{-2}$ \\
    \ele{Pu}{242} & 2.09 $\times 10^{-2}$ \\
    \ele{Pu}{238} & 6.45 $\times 10^{-3}$ \\
    \ele{U}{235} & 4.11 $\times 10^{-3}$ \\
    \ele{Am}{241} & 3.57 $\times 10^{-3}$ \\
    \ele{U}{236} & 1.00 $\times 10^{-4}$ \\
    \ele{U}{234} & 3.01 $\times 10^{-5}$ \\ \bottomrule
  \end{tabular}
  \label{tab:fast_fuel}
\end{table}

\begin{figure}[hbtp]
  \centering
  \begin{tabular}{cc}
  \includegraphics[scale=0.5]{images/results/fast_pin_geom1} \\
\begin{tabular}{@{}lr@{}}
\toprule
\textbf{Parameter}     & \textbf{Value (cm)} \\ \midrule
Void radius     & 0.1          \\
Pellet radius & 0.45          \\
Inner cladding radius & 0.465          \\
Outer cladding radius & 0.525          \\
Pitch              & 1.789          \\ \bottomrule
\end{tabular}
  \end{tabular}
  \caption[Fast pin cell geometry and physical parameters.]{Fast pin
    cell geometry and physical parameters. The pellet is shown in
    blue, the cladding green, and the coolant orange. Black regions
    are voids.}
  \label{fig:fast}
\end{figure}

\subsection{Infinite flux}
\label{sec:fast_inf_flx}
A summary of the \gls{fom} ratio and
$\overline{\mathrm{FOM}}(t_{\mathrm{wdt}})$, for the fast reactor pin
cell for infinite flux ($\phi_{\infty}$) is shown in Tab.~\ref{tab:fast_inf_flx} and
shown graphically in Fig.~\ref{fig:fast_inf_flx}. The \gls{fom} ratio
for group 1 is consistently higher; for group 2 the \gls{fom} values are
very low.
% is the ratio not useful, or the method?
% Both, removed, as I think it is more clearly explained by the
% additions in the next section.
Although there appears to
be a high ratio (3.140) at the threshold value $t_{\mathrm{wdt}}
=0.4$, this merely reflects the poor statistics. As seen in
Fig.~\ref{fig:fast_inf_flx_example}, the \gls{fom} is not converging
to a single value. We expect the thermal group to be underpopulated in
a fast reactor, resulting in poor statistics. The \gls{wdt} method
does not improve this, but the statistics of the thermal group is of
minimal interest in a fast reactor.
% what is this shown earlier referring to?
% may also want to comment that in a fast system it (a) isn't surprising that
% the thermal group is underpopulated (poor statistics) and (b) that we probably
% don't care about that.
\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
    the fast pin cell \textit{infinite flux}.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the fast pin cell \textit{infinite flux}.}
  \begin{tabular}{cc} Group 1 ($E > 0.0625$) & Group 2 \\
    \input{include/tables/fast_inf_flx_grp_1}
 &
   \input{include/tables/fast_inf_flx_grp_2}
  \end{tabular}
\label{tab:fast_inf_flx}
\end{table}
\begin{figure}[hbtp]
  \centering
  \begin{tabular}{c}
  \includegraphics[scale=0.9]{images/results/fast_inf_flx_grp_1} \\
    (a) \\
  \includegraphics[scale=0.9]{images/results/fast_inf_flx_grp_2} \\
    (b)
  \end{tabular}
% Just noticed this: number if significant digits on the FOMs and ratios provided, are they all meaningful?
  \caption[Figure of merit ratio of the \textit{infinte flux} for the
  PWR]{Figure of merit ratio of the \textit{infinte flux} for the fast pin cell vs the
    \gls{wdt} threshold for the (a) fast and (b) thermal energy
    groups. The group boundary is at $0.0625$ eV. \gls{fom} is
    calculated using the last 20 data points.}
  \label{fig:fast_inf_flx}
\end{figure}
\begin{figure}[hbtp]
  \centering
  \includegraphics[scale=0.5]{images/results/fast_inf_flx_example}
  \caption[\Acrshort{fom} convergence for \textit{infinite flux} for the base
  and $t_{\mathrm{wdt}} = 0.4$ case for group 2.]{\Acrshort{fom} convergence for
    \textit{infinite flux} the base and $t_{\mathrm{wdt}} = 0.4$ case for
    group 2 in the fast pin cell.}
  \label{fig:fast_inf_flx_example}
\end{figure}

\subsection{Infinite total cross-section}
\label{sec:fast_inf_tot_cross_section}
A summary of the \gls{fom} ratio and
$\overline{\mathrm{FOM}}(t_{\mathrm{wdt}})$, for the fast reactor pin
cell for infinite total cross-section ($\Sigma_{t,
  \infty}$) is shown in Tab.~\ref{tab:fast_inf_tot} and shown
graphically in Fig.~\ref{fig:fast_inf_tot}. The \gls{fom} ratio for
group 1 is higher for all values of $t_{\mathrm{wdt}}$, with the
exception of 0.2. For group 2, as seen in Sec.~\ref{sec:fast_inf_flx}, the
\gls{fom} values are very low, and one ratio is artificially inflated
when $t_{\mathrm{wdt}} = 0.6$
due to the lack of convergence. This case is shown in
Fig.~\ref{fig:fast_inf_tot_example}; there are not enough statistics
for the low energy group to cause the \gls{fom} to converge to a high
enough value to be significant.
\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
  the fast reactor pin cell \textit{infinite total cross-section}.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the fast reactor pin cell \textit{infinite total cross-section}.}
  \begin{tabular}{cc} Group 1 ($E > 0.0625$) & Group 2 \\
    \input{include/tables/fast_inf_tot_grp_1}
 &
   \input{include/tables/fast_inf_tot_grp_2}
  \end{tabular}
\label{tab:fast_inf_tot}
\end{table}
\begin{figure}[hbtp]
  \centering
  \begin{tabular}{c}
  \includegraphics[scale=0.9]{images/results/fast_inf_tot_grp_1} \\
    (a) \\
  \includegraphics[scale=0.9]{images/results/fast_inf_tot_grp_2} \\
    (b)
  \end{tabular}
  \caption[Figure of merit ratio of the  \textit{infinite total cross-section} for the
  fast reactor pin cell]{Figure of merit ratio of
     \textit{infinite total cross-section} for the fast reactor pin cell vs the \gls{wdt}
    threshold for (a) fast and (b) thermal groups. The group
    boundary is at $0.0625$ eV. \gls{fom} is calculated using the last
    20 data points.}
  \label{fig:fast_inf_tot}
\end{figure}
\begin{figure}[hbtp]
  \centering
  \includegraphics[scale=0.5]{images/results/fast_inf_tot_example}
  \caption[\Acrshort{fom} convergence for  \textit{infinite total cross-section} the base
  and $t_{\mathrm{wdt}} = 0.6$ case in the fast pin cell.]{\Acrshort{fom} convergence for
     \textit{infinite total cross-section} the base and $t_{\mathrm{wdt}} = 0.6$ case in the fast pin cell.}
  \label{fig:fast_inf_tot_example}
\end{figure}

\subsection{Scattering Matrices}
\label{sec:fast_inf_sp0}

For the two-group case, the scattering matrices for $S_0$ is again $2 \times 2$. A summary of the \gls{fom} ratio
for the zeroth moment $S_0$ is shown in Table~\ref{tab:fast_inf_sp0}. The
tables are for the entry in row $g$ and column $g'$, which represents
scattering from group $g'$ into group $g$, indicated as $g' \to g$.

The group 1 inscattering matrix displays good statistics, the other
three entries have an extremely low \gls{fom}, indicating that there
are not enough statistics. The group 1 inscattering \gls{fom} ratio is
plotted in Fig.~\ref{fig:fast_inf_sp0}. \gls{wdt} returns a higher
\gls{fom} for all values of $t_{\mathrm{wdt}} \geq 0.3$, and close to
parity for $t_{\mathrm{wdt}} = 0.2$.

\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
  the fast reactor pin cell \textit{infinite zeroth-moment of the scattering matrix}.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the fast reactor pin cell \textit{infinite zeroth-moment of the scattering matrix}}
  \begin{tabular}{cc} $\Sigma_{1\to 1}$ & $\Sigma_{2 \to 1}$ \\
\input{include/tables/fast_inf_sp0_grp_1} &
\input{include/tables/fast_inf_sp0_grp_2} \\
$\Sigma_{1\to 2}$ & $\Sigma_{2 \to 2}$ \\
\input{include/tables/fast_inf_sp0_grp_3} &
\input{include/tables/fast_inf_sp0_grp_4}
  \end{tabular}
  \label{tab:fast_inf_sp0}
\end{table}
\begin{figure}[hbtp]
  \centering
  \includegraphics[scale=0.9]{images/results/fast_inf_sp0_grp_1}
  \caption[Figure of merit ratio $\Sigma_{1 \to 1}$ for the fast reactor pin cell]{Figure of
    merit ratio of $\Sigma_{1 \to 1}$ for
    the fast reactor pin cell. The group boundary is at $0.0625$ eV. \gls{fom} is calculated
    using the last 20 data points.}
  \label{fig:fast_inf_sp0}
\end{figure}

\newpage

\subsection{Discussion}
\label{sec:fast_discussion}

There is very little thermal flux in a fast reactor. This results in
poor statistics for that group, a fact that \gls{wdt} does not
remedy. As evidenced by the results presented in this section, all
values of $t_{\mathrm{wdt}}$ continued to result in group 2 \gls{fom} orders
of magnitude smaller than both group 1 for this problem, and either
group for the \gls{pwr}.

For the fast group, there is consistent improvement in the \gls{fom}
for all values of $t_{\mathrm{wdt}}$. The \gls{fom} ratio peaks at
$t_{\mathrm{wdt}} = 0.6$ for the infinite flux, total cross-section
and group 1 inscattering cross-section. This value represents the
highest improvement in group 1, and the values are summarized in
Tab.~\ref{tab:fast_discussion}. The values for group 2 are omitted as
this value, like all others, did not result in meaningful improvement.
% I know you can't turn this on for only one group. Do you have a sense of whether
% the improvement in group 1 is more valuable than the detriment in group 2?

\begin{table}[hbtp]
  \centering
  \caption{Summary of \acrshort{fom} ratios for the fast reactor pin
  cell for $t_{\mathrm{wdt}}$ = 0.6.}
  \begin{tabular}{lr}\toprule
    \textbf{Parameter}& $\overline{\mathbf{FOM}}$ \textbf{Ratio} \\ \midrule
    $\phi_{1, \infty}$ & 1.360716 \\
    $\Sigma_{1,t}$ & 1.216532 \\
    $\Sigma_{1 \to 1}$ & 1.341268 \\ \bottomrule
  \end{tabular}
  \label{tab:fast_discussion}
\end{table}

\newpage
\section{Homogenous fuel element}
\label{sec:homog}

The third test case is derived from the \gls{treat} reactor at
\gls{inl}. Work at \gls{inl} focuses on using deterministic solvers to
model the core; the Serpent 2 Monte Carlo code is used to generate the
cross-sections for these solvers. We used a homogenized fuel element
that replicates the material but not geometry of the \gls{treat}
fuel. The simulation uses eleven energy groups, which replicates the
groups used by the deterministic solvers. This group structure is
summarized in Table.~\ref{tab:homog_energy}.
\begin{table}[hbtp]
  \centering
  \caption{Energy group structure for the homogenous fuel element.}
  \begin{tabular}{cc}
  \begin{tabular}{rr} \toprule
    \textbf{Group}& \textbf{Upper energy limit (MeV)} \\ \midrule
    1 & 20.0000  \\
    2 & 3.32870  \\
    3 & 1.15620 $\times 10^{-1}$ \\
    4 & 3.48110 $\times 10^{-3}$ \\
    5 & 1.32700 $\times 10^{-4}$ \\
    6 & 8.10003 $\times 10^{-6}$ \\ \bottomrule
  \end{tabular} &
  \begin{tabular}{rr} \toprule
    \textbf{Group}& \textbf{Upper energy limit (MeV)} \\ \midrule
    7 & 6.25000 $\times 10^{-7}$ \\
    8 & 2.09610 $\times 10^{-7}$ \\
    9 & 7.64970 $\times 10^{-8}$ \\
    10 & 4.73020 $\times 10^{-8}$ \\
    11 & 2.00100 $\times 10^{-8}$ \\ \\\bottomrule
  \end{tabular}
  \end{tabular}
  \label{tab:homog_energy}
\end{table}

To simplify the analysis, and to allow comparison between the other
two test cases, these groups can be collapsed. The values we are
considering, $\phi_{\infty}$, $\Sigma_{t,\infty}$, and the zeroth
scattering matrix are all independent measurements with regard to
energy. We can sum all the $\phi_{\infty}$ for each energy group to
get a total $\phi_{\infty}$ for the entire energy range. As there is
no covariance, the variance of the sum will be the sum of the
variance~\cite{taylor1997}. The variance for summing energy groups $g$ to $g'$ is therefore:
\begin{equation*}
  \mathrm{Var}\left( \sum_{j=g}^{g'}\phi_{g,\infty}\right) =   \sum_{j=g}^{g'}\mathrm{Var}\left(\phi_{g,\infty}\right)\:.
\end{equation*}
This allows us to collapse groups one through six into a single group,
hereafter denoted by an $f$ for fast, and groups seven through eleven as
 a thermal group, $t$. The boundary of group seven aligns with the separation
of the two groups in the previous test problems, so we can compare the
results in a more natural
fashion. Fig.~\ref{fig:homog_fom_convergence_example} shows the
\gls{fom} convergence for the combined fast group. As expected, the
combined group shows the same convergence characteristic as the single
groups.

\begin{figure}[hbtp]
  \centering
  \includegraphics[scale=0.5]{images/homog_fom_convergence_example}
  \caption{Convergence of \textit{infinite flux} for the combined fast group
    for the homogenous fuel element for the base case
    $t_{\mathrm{wdt}} = 0.1$.}
  \label{fig:homog_fom_convergence_example}
\end{figure}

\subsection{Infinite flux}
\label{sec:homog_inf_flx}

A summary of the \gls{fom} ratio and
$\overline{\mathrm{FOM}}(t_{\mathrm{wdt}})$, for the homogenous fuel
element for infinite flux ($\phi_{\infty}$) is shown in
Tab.~\ref{tab:homog_inf_flx} and shown graphically in
Fig.~\ref{fig:homog_inf_flx}. The Tables for all eleven of the
original energy groups are included in
App.~\ref{sec:homog_inf_flx_data}.

As shown in both the table and plot, we observe an improvement
in \gls{fom} for \gls{wdt} threshold values
$0.2 \leq t_{\mathrm{wdt}} \leq 0.6$ and $t_{\mathrm{wdt}} = 0.8, 1.0$ for
the fast group. The improvement is moderate for this
group, with a maximum improvement of 1.07 times the base case. The
improvements are even slighter for the thermal group, where we observe a
maximum of 1.014 times the base case for $0.4 \leq t_{\mathrm{wdt}} 0.6$ and $t_{\mathrm{wdt}} = 1.0$.

\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
    the homogenous fuel element \textit{infinte flux}.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the homogenous fuel element \textit{infinte flux}.}
  \begin{tabular}{cc} Fast Group ($E > 0.0625$) & Thermal Group \\
    \input{include/tables/homog_inf_flx_grp_comb1}
 &
   \input{include/tables/homog_inf_flx_grp_comb2}
  \end{tabular}
\label{tab:homog_inf_flx}
\end{table}
\begin{figure}[hbtp]
  \centering
  \begin{tabular}{c}
  \includegraphics[scale=0.9]{images/results/homog_inf_flx_grp_comb1} \\
    (a) \\
  \includegraphics[scale=0.9]{images/results/homog_inf_flx_grp_comb2} \\
    (b)
  \end{tabular}
  \caption[Figure of merit ratio of the \textit{infinte flux} for the
  homogeneous fuel element]{Figure of merit ratio of the \textit{infinte flux}
    for the homogeneous fuel element vs the
    \gls{wdt} threshold for the (a) fast and (b) thermal
    groups. The group boundary is at $0.0625$ eV. \gls{fom} is
    calculated using the last 20 data points.}
  \label{fig:homog_inf_flx}
\end{figure}

\subsection{Infinite total cross-section}
\label{sec:homog_inf_total}

A summary of the \gls{fom} ratio and
$\overline{\mathrm{FOM}}(t_{\mathrm{wdt}})$, for the homogeneous fuel element
 infinite total cross-section is shown in Tab.~\ref{tab:homog_inf_tot} and shown
graphically in Fig.~\ref{fig:homog_inf_tot}. The full data for all
eleven groups is included in App.~\ref{sec:homog_inf_tot_data}.

For both groups, the \gls{wdt} method underperforms the base case in
all but a few select cases. Where it performs better, we observe a very
moderate improvement in \gls{fom}, with the only shared improvement in
both groups at $t_{\mathrm{wdt}} = 0.7$.

\begin{table}[hbtp]
  \centering
  \caption[$\overline{\mathrm{FOM}}$ and ratio for
    the homogeneous fuel element \textit{infinite total cross-section}.]{$\overline{\mathrm{FOM}}$ as a function of
    $t_{\mathrm{wdt}}$ and the ratio of values to the base case for
    the homogeneous fuel element \textit{infinite total cross-section}.}
  \begin{tabular}{cc} Fast group ($E > 0.0625$) & Thermal group \\
    \input{include/tables/homog_inf_tot_grp_comb1}
 &
   \input{include/tables/homog_inf_tot_grp_comb2}
  \end{tabular}
\label{tab:homog_inf_tot}
\end{table}
\begin{figure}[hbtp]
  \centering
  \begin{tabular}{c}
  \includegraphics[scale=0.9]{images/results/homog_inf_tot_grp_comb1} \\
    (a) \\
  \includegraphics[scale=0.9]{images/results/homog_inf_tot_grp_comb2} \\
    (b)
  \end{tabular}
  \caption[Figure of merit ratio of the  \textit{infinite total cross-section} for the
  homogeneous fuel element]{Figure of merit ratio of
     \textit{infinite total cross-section} for the homogeneous fuel element vs the \gls{wdt}
    threshold for (a) fast group and (b) thermal group. The group
    boundary is at $0.0625$ eV. \gls{fom} is calculated using the last
    20 data points.}
  \label{fig:homog_inf_tot}
\end{figure}

\subsection{Scattering Matrices}
\label{sec:homog_inf_sp0}

For the eleven-group case, the scattering matrices for the zeroth
moment is $11 \times 11$. All the tables showing the \gls{fom}
ratios for each value of the \gls{wdt} threshold are shown in
App.~\ref{sec:homog_inf_sp0_raw}. The tables are for the entry in row $g$
and column $g'$, which represents scattering from group $g'$ into
group $g$, indicated as $g' \to g$. Collapsing the groups for
comparison can not be done in this case, as the \gls{fom} is so low in
many of the matrix entries that it dominates the value of the
\gls{fom}. Instead, we have created matrix maps that show the
\gls{fom} ratios when \gls{wdt} is used. These maps are included in
App.~\ref{sec:homog_inf_sp0_mat}.

A characteristic map is shown in Fig.~\ref{fig:homog_inf_sp0_map}, for
$t_{\mathrm{wdt}} = 0.2$. The trend we observe in these images is a
slight deviation from unity in most entries, indicating a moderate
improvement or reduction in the \gls{fom}. In only three cases did the
use of \gls{wdt} populate an entry in the matrix where the base case
did not. In each of these cases, the \gls{fom} remains very low, on
the order of $10^{-4}$. There is some improvement in the scattering
entries for group five into higher energy groups,
$\Sigma_{5 \to 1}, \Sigma_{5 \to 2},$ and $\Sigma_{5 \to 1}$. This
improvement is higher in lower values of $t_{\mathrm{wdt}}$.
\begin{figure}[hbtp]
  \centering
    \includegraphics[scale=0.75]{images/results/matshows/homog_sp0_matshow_1}
  \caption{Matrix image for the \textit{infinite zeroth-moment of the
      scattering cross-section} for the
    homogeneous fuel element for $t_{\mathrm{wdt}} = 0.2$.}
  \label{fig:homog_inf_sp0_map}
\end{figure}
\newpage
\subsection{Discussion}
\label{sec:homog_discussion}

There is no clear \gls{wdt} threshold value that creates an overall
improvement in the parameters we studied. The results of the infinite
flux show an improvement in most cases for the fast flux, and a
moderate improvement for thermal flux, for $0.4 \leq t_{\mathrm{wdt}} \leq
0.6$ and unity. For the infinite total cross-section, improvement in
the thermal group only occurs for two values, $t_{\mathrm{wdt}} = 0.7,
1.0$. We also observe improvement in the fast group total
cross-section for $t_{\mathrm{wdt}} = 0.7$. The scattering matrices
show similar patterns for all values of the \gls{wdt} threshold. The
original use of the homogeneous fuel element by \gls{inl} is for the
development of cross-sections. Based on this intent, a threshold value
of $t_{\mathrm{wdt}} = 0.7$ will return improvements in both total
cross-section and the zeroth order scattering matrix, and near parity
with the \gls{fom} of the base case for flux. The ratios for this
parameter are summarized in Fig.~\ref{fig:homog_summary}.

\begin{figure}[hbtp]
  \centering
  \begin{tabular}{c} $t_{\mathrm{wdt}}=0.7$\\
  \begin{tabular}{lr}\toprule
    \textbf{Parameter}& $\overline{\mathbf{FOM}}$ \textbf{Ratio} \\ \midrule
    $\phi_{f, \infty}$ & 0.990465\\
    $\phi_{s, \infty}$ & 0.988462 \\
    $\Sigma_{f,t}$ & 1.010220 \\
    $\Sigma_{s,t}$ & 1.052628 \\
    \bottomrule
  \end{tabular}\\
    \includegraphics[scale=0.75]{images/results/matshows/homog_sp0_matshow_6}
\end{tabular}
  \caption{Summary of the data for the homogeneous fuel element for $t_{\mathrm{wdt}} = 0.7$.}
\label{fig:homog_summary}
\end{figure}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "../masters_report"
%%% End:


\documentclass[letterpaper,10pt]{article}
\usepackage[margin=2cm]{geometry}

\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage[colorlinks]{hyperref}

\newcommand{\panhline}{\begin{center}\rule{\textwidth}{1pt}\end{center}}

\title{\textbf{Naive Bayes Classifier}}
\author{Aarti Singh}

\begin{document}

\maketitle

\panhline
\href{../index.html}{Back to Index}

\panhline
\tableofcontents

\section*{Resources}

\begin{itemize}
	\item \href{../../Lectures/06_NaiveBayes.pdf}{Lecture}
\end{itemize}

\panhline

Optimal Classification (Bayes classifier)

$$f^* = \arg\min_f P(f(X)\neq Y)$$

Even the optimal classifier makes mistakes $R(f^*)=P(f^*)>0$

Bayes rule $P(Y|X) \propto P(X|Y)P(Y)$
\begin{itemize}
	\item $P(X|Y)$: class conditional distribution
	\item $P(Y)$: class probability
\end{itemize}

\end{document}


\documentclass[usenames,dvipsnames]{beamer}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc}    % use 8-bit T1 fonts
\usepackage{hyperref}       % hyperlinks
\usepackage{url}            % simple URL typesetting
\usepackage{booktabs}       % professional-quality tables
\usepackage{amsfonts}       % blackboard math symbols
\usepackage{nicefrac}       % compact symbols for 1/2, etc.
\usepackage{microtype}      % microtypography
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{tikz}
\usetikzlibrary{calc}
\usetikzlibrary{bayesnet}
\usetikzlibrary{arrows}
\usepackage{color}
\usepackage{array}
\usepackage{dsfont}
\usepackage{multirow, graphicx}
 \usepackage{float}
\newcolumntype{C}[1]{>{\centering\arraybackslash}p{#1}}
\newcolumntype{R}[1]{>{\raggedleft\arraybackslash}p{#1}}
\newcolumntype{L}[1]{>{\raggedright\arraybackslash}p{#1}}
\usepackage{caption}
\usepackage{subfig}
\usepackage{pifont}
\usepackage{xcolor}
\usepackage{algorithm,algorithmic}
% \floatname{algorithm}{Procedure}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}
\newcommand{\cmark}{\textcolor{green!80!black}{\ding{51}}}
\newcommand{\xmark}{\textcolor{red}{\ding{55}}}
\DeclareMathOperator*{\argmin}{argmin}
\urlstyle{same}
\usepackage{listings}
\usepackage[export]{adjustbox}
\usepackage{inconsolata}
% \usetheme{Boadilla}

\title{Attention is All You Need}
% \subtitle{Using Beamer}
\author{Bill Watson}
\institute{S\&P Global}
\date{\today}

\newenvironment{nospaceflalign*}
 {\setlength{\abovedisplayskip}{0pt}\setlength{\belowdisplayskip}{0pt}%
  \csname flalign*\endcsname}
 {\csname endflalign*\endcsname\ignorespacesafterend}

\AtBeginSection[]{
  \begin{frame}
  \vfill
  \centering
  \begin{beamercolorbox}[sep=8pt,center,shadow=true,rounded=true]{title}
    \usebeamerfont{title}\insertsectionhead\par%
  \end{beamercolorbox}
  \vfill
  \end{frame}
}

\begin{document}

\begin{frame}
\titlepage
\end{frame}

\begin{frame}
\frametitle{Recap: Encoder-Decoder Models}
\begin{figure}
  \centering
  \includegraphics[width=10cm, valign=c]{assets/enc-dec}
\end{figure}
\begin{itemize}
  \item \textbf{Encoders:} Give the source sentence meaning
  \item \textbf{Decoders:} Emit a variable-length sequence
  \item \textbf{Seq2Seq} models rely on propagating the hidden state for generation
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{How can we improve this model?}
\begin{itemize}
  \item Add an \textbf{alignment model} to force the model to 'attend' and 'read' different parts of the input
  \item Gives the decoder direct access to the encodings to make generations
  \item Known as a \textbf{context vector}
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{What are Context Vectors?}
\begin{equation*}
  c_i = \sum_{j} \alpha_{ij} \cdot h_j
\end{equation*}
\begin{itemize}
  \item A \textbf{context vector} $c_i$ is the weighted sum of the encodings for the $i$-th iteration of the decoder
  \item $s_{i-1}$ is the previous hidden state of the decoder
  \item $h_j$ is the encoding for the $j$-th input word
  \item $\alpha_{ij}$ is the weight associated for the $j$-th input word for the $i$-th deocding iteration
\end{itemize}
\end{frame}

\begin{frame}
  \frametitle{How to use Context Vectors?}
  \begin{itemize}
    \item No general rule
    \item Usually, \textbf{concat} the token embedding with context vector $c_i$
    \item Pass this as the input to the RNN, along with the previous decoder state $s_{i-1}$ as the hidden state
  \end{itemize}

\end{frame}

\begin{frame}
  \frametitle{Visual}
  \begin{figure}
      \centering
      \begin{tikzpicture}[shorten >=1pt,node distance=1.5cm,on grid,auto, every node/.style={scale=0.75}]
        \node[] (x) {$x_{i-1}$};

        \node[draw,diamond] (h0) [above=of x] {$h_k$};
        \node[] (h1) [above=of h0, yshift=-0.75cm] {$\vdots$};
        \node[draw,diamond] (h2) [above=of h1, yshift=-1cm] {$h_j$};
        \node[] (h3) [above=of h2, yshift=-0.75cm] {$\vdots$};
        \node[draw,diamond] (h4) [above=of h3, yshift=-1cm] {$h_0$};

        \node[] (c) [right=of h2] {$c_i$};

        \node[]  (emb) [right=of x, xshift=0.75cm] {Embedding};
        \node[]  (concat) [right=of c, xshift=1cm] {Concat};
        \node[] (rnn) [right=of concat, xshift=0.5cm] {RNN};

        \node[] (xn) [right=of rnn] {$x_{i}$};

        \node[] (s) [above=of rnn] {$s_{i-1}$};

        \node[] (sn) [below=of rnn] {$s_i$};

        \path [->]
          (x) edge [] node {} (emb)
          (emb) edge [] node {} (concat)
          (c) edge [] node {} (concat)

          (h0) edge [bend right=10] node {} (c)
          (h2) edge [] node {} (c)
          (h4) edge [bend left=10] node {} (c)

          (concat) edge [] node {} (rnn)
          (s) edge [] node {} (rnn)
          (rnn) edge [] node {} (sn)
          (rnn) edge [] node {} (xn)
          (s) edge [bend right=15] node {} (c);

      \end{tikzpicture}
    \end{figure}
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{(Simple) Attention Mechanisms}

\begin{frame}
\frametitle{Mean Attention}
\begin{equation*}
  \begin{split}
  \alpha_{ij} &= \frac{1}{S} \\
  c_i &= \frac{1}{S} \sum_j^S h_j = \sum_{j} \alpha_{ij} \cdot h_j
  \end{split}
\end{equation*}
\begin{itemize}
  \item Very simple and naive approach is to apply uniform weights to each encoding $h_j$
  \item Fancy way of saying average
  \item $S$ is the length of the source sentence
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Location Based: Laplace}
\begin{equation*}
  \begin{split}
  \alpha_{ij} &= f \left(j \mid i, b \right) = \frac{1}{2b} \exp \left( - \frac{\lvert j - i \rvert}{b} \right) \\
  c_i &= \sum_{j} \alpha_{ij} \cdot h_j
  \end{split}
\end{equation*}
\begin{itemize}
  \item Weighted by location in sequence
  \begin{itemize}
    \item Center a Laplace at $\mu=i$, scale $b$
    \item Weight is the probability of position $j$ relative to $i$
  \end{itemize}
  \item Penalizes elements away from the diagonal
  \item Heavier tail than a Gaussian
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Location Based: Gaussian}
\begin{equation*}
  \begin{split}
  \alpha_{ij} &= f \left(j \mid i, \sigma^2 \right) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp \left( - \frac{\left( j - i \right)^2}{2\sigma^2} \right) \\
  c_i &= \sum_{j} \alpha_{ij} \cdot h_j
  \end{split}
\end{equation*}
\begin{itemize}
  \item Weighted by location in sequence
  \begin{itemize}
    \item Center a Gaussian at $\mu=i$, scale $\sigma$
    \item Weight is the probability of position $j$ relative to $i$
  \end{itemize}
  \item Smoother distribution closer to the diagonal
  \item Smaller tail than Laplace, so outliers are penalized more
\end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Review: Laplace and Gaussian}
  \begin{figure}
    \includegraphics[width=\textwidth]{assets/dist}
  \end{figure}
\end{frame}

\begin{frame}
\frametitle{Visualizing Location-Based Distributions}
\begin{columns}
  \begin{column}{0.45\textwidth}
    \begin{figure}
      \centering
      \includegraphics[width=4.75cm, valign=c]{assets/laplace}
      \caption{Laplace}
    \end{figure}
  \end{column}
  \begin{column}{0.45\textwidth}
    \begin{figure}
      \centering
      \includegraphics[width=4.75cm, valign=c]{assets/normal}
      \caption{Gaussian}
    \end{figure}
  \end{column}

\end{columns}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{(Parameter-based) Attention Mechanisms}

\begin{frame}
\frametitle{Parameter-based Attention}
% rework
\begin{itemize}
  \item Why not use learnable weights to effectively mix the encodings?
  \item Let deep learning figure out the alignment
  \item We now define a scoring function conditioned on the previous decoder state $s_{i-1}$ and input encodings $h_j$
\end{itemize}
\begin{equation*}
  a\left( s_{i-1}, h_j \right) = \;?
\end{equation*}
\begin{itemize}
  \item In order to constrain the weights to a valid distribution, we softmax
\end{itemize}
\begin{equation*}
  \begin{split}
    \alpha_{ij} &= \frac{\exp\left( a\left( s_{i-1}, h_j \right) \right)}{\sum_k \exp\left( a\left( s_{i-1}, h_k \right) \right)}\\
    c_i &= \sum_{j} \alpha_{ij} \cdot h_j
  \end{split}
\end{equation*}
\end{frame}

\begin{frame}
  \frametitle{Attention Computation Graph}
  \begin{figure}
      \centering
      \begin{tikzpicture}[shorten >=1pt,node distance=1.25cm,on grid,auto, every node/.style={scale=0.75}]

        \node[draw,diamond]  (h2)  {$h_j$};
        \node[]  (h1) [above=of h2] {$\vdots$};
        \node[]  (h3) [below=of h2] {$\vdots$};
        \node[draw,diamond] [above=of h1]  (h0)  {$h_0$};
        \node[draw,diamond]  (h4) [below=of h3] {$h_{k}$};

        \node[] (s) [left=of h2] {$s_{i-1}$};

        \node[]   (a0) [right=of h0, xshift=0.75cm] {$a\left( s_{i-1}, h_0 \right)$};
        \node[]   (a1) [right=of h1, xshift=0.75cm] {$\vdots$};
        \node[]   (a2) [right=of h2, xshift=0.75cm] {$a\left( s_{i-1}, h_j \right)$};
        \node[]   (a3) [right=of h3, xshift=0.75cm] {$\vdots$};
        \node[]   (a4) [right=of h4, xshift=0.75cm] {$a\left( s_{i-1}, h_k \right)$};

        \node[] (softmax) [right=of a2, xshift=1.5cm] {$\frac{\exp\left( a\left( s_{i-1}, h_j \right) \right)}{\sum_k \exp\left( a\left( s_{i-1}, h_k \right) \right)}$};

        \node[] (mult0) [right=of a0, xshift=4.5cm] {$\alpha_{i0} \cdot h_0$};
        \node[] (mult1) [right=of a1, xshift=4.5cm] {$\vdots$};
        \node[] (mult2) [right=of a2, xshift=4.5cm] {$\alpha_{ij} \cdot h_j$};
        \node[] (mult3) [right=of a3, xshift=4.5cm] {$\vdots$};
        \node[] (mult4) [right=of a4, xshift=4.5cm] {$\alpha_{ik} \cdot h_k$};

        \node[] (sum) [right=of mult2, xshift=0.25cm] {$\sum_j$};

        \node[] (c) [right=of sum, xshift=-0.25cm] {$c_i$};

        \path [->]
          (h0) edge [] node[] {} (a0)
          (h2) edge [] node[] {} (a2)
          (h4) edge [] node[] {} (a4)

          (a0) edge [bend right=10] node[] {} (softmax)
          (a2) edge [] node[] {} (softmax)
          (a4) edge [bend left=10] node[] {} (softmax)

          (softmax) edge [bend right=10] node[] {} (mult0)
          (softmax) edge [] node[] {} (mult2)
          (softmax) edge [bend left=10] node[] {} (mult4)

          (mult0) edge [bend right=10] node[] {} (sum)
          (mult2) edge [] node[] {} (sum)
          (mult4) edge [bend left=10] node[] {} (sum)

          (sum) edge [] node[] {} (c)

          (h0) edge [bend left=30] node {} (mult0)
          (h2) edge [bend left=40] node {} (mult2)
          (h4) edge [bend right=30] node {} (mult4)

          (s) edge [bend left=20] node {} (a0)
          (s) edge [bend right=40] node {} (a2)
          (s) edge [bend right=20] node {} (a4);

      \end{tikzpicture}
    \end{figure}
\end{frame}

\begin{frame}
\frametitle{Bahdanau Attention}
\begin{equation*}
  a\left( s_{i-1}, h_j \right) = v_a^T \tanh \left( W_a s_{i-1} + U_a h_j + b \right)
\end{equation*}
\begin{itemize}
  \item $W_a$, $U_a$, are learnable matrices
  \item $v_a$, $b$ are learnable vectors
  \item Output is a scalar value
  \item AKA: Additive Attention, Concat Attention
\end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Bahdanau Attention Visual}
  \begin{figure}
    \includegraphics[height=7.25cm]{assets/concat1}
  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Bahdanau Attention Visual}
  \begin{figure}
    \includegraphics[height=7.25cm]{assets/concat2}
  \end{figure}
\end{frame}

\begin{frame}
\frametitle{Luong Attention: Dot Product}
\begin{equation*}
  a\left( s_{i-1}, h_j \right) = s_{i-1}^T h_j
\end{equation*}
\begin{itemize}
  \item Notice: No Learnable Parameters
  \item Simple dot product between decoder state and encoding
  \item Output is a scalar value
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Luong Attention: Scaled Dot Product}
\begin{equation*}
  a\left( s_{i-1}, h_j \right) = \frac{1}{\sqrt{\lvert h_j \rvert}} s_{i-1}^T h_j
\end{equation*}
\begin{itemize}
  \item Rescale the dot product accoridng to the dimensionality of the vectors
  \item \textbf{Why?} Because for large dimensionality, the dot-product is large in magnitude, resulting in small gradients from the softmax
  \item Output is a scalar value
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Luong Attention: General}
\begin{equation*}
  a\left( s_{i-1}, h_j \right) = s_{i-1}^T W_a h_j
\end{equation*}
\begin{itemize}
  \item Allow a learnable matrix $W_a$ inbetween the states
  \item AKA a Bilinear Layer: $x_0^T A x_1$
  \item Output is a scalar value
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Luong Attention: Location Attention}
\begin{equation*}
  a\left( s_{i-1}, h_j \right) = W_a s_{i-1}
\end{equation*}
\begin{itemize}
  \item An approach to reweight exclusively on the decoder state
  \item Single learnable matrix $W_a$
  \item Output is vector of fixed-length
  \item \textbf{Footnote:} For short sentences, only use the top part of the vector. For long sentences, ignore words near the end.
  \item \textbf{Note:} Implies that this only works for fixed-length sentences!
\end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Luong Attention Visual}
  \begin{columns}
  \begin{column}{0.45\textwidth}
  \begin{figure}
    \includegraphics[width=5cm]{assets/global}
    \caption{Luong Attention}
  \end{figure}
\end{column}
\begin{column}{0.45\textwidth}
  \begin{figure}
    \includegraphics[width=5cm]{assets/gold}
    \caption{Gold Alignments}
  \end{figure}
\end{column}
  \end{columns}
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{(Advanced) Attention Mechanisms}

\begin{frame}
  \frametitle{Luong Attention: Local Attention}
  \begin{itemize}
    \item What if we focused on a window instead of all the vectors?
    \item This approach is known as \textit{local} attention
    \item Before, we were using \textit{global} attention
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Luong Attention: Local Attention}
  \begin{itemize}
    \item Define a window $\mathcal{W}$ of size $2D$ at alignment point $p_t$:
  \end{itemize}
  \begin{equation*}
    \mathcal{W} = \left[ p_t - D, p_t + D \right]
  \end{equation*}
  \begin{itemize}
    \item Our context vector is now an average over this window
  \end{itemize}
  \begin{equation*}
    c_i = \sum_{j \in \mathcal{W}} \alpha_{ij} \cdot h_j
  \end{equation*}
\end{frame}

\begin{frame}
  \frametitle{Luong Attention: Local Attention}
  \begin{itemize}
    \item How do we select $p_t$, the alignment position?
    \item \textbf{Monotonic} alignment - set $p_t = t$
    \begin{itemize}
      \item Assumes \texttt{src} and \texttt{trg} are roughly monotonically aligned
    \end{itemize}
    \item \textbf{Predictive} alignment - predict the aligned position
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Luong Attention: Local Predictive Attention}
  \begin{equation*}
    p_t = S \cdot \sigma \left( v_p^T \tanh \left( W_p \cdot s_{i-1} \right) \right)
  \end{equation*}
  \begin{itemize}
    \item \textbf{Predictive} alignment allows for learnable positions
    \item $S$ is the length of the source sentence
    \item $W_p$ and $v_p$ are learnable
    \item Sigmoid enforces the constraint $p_t \in \left[ 0, S \right]$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Luong Attention: Local Predictive Attention}
  \begin{equation*}
    a_{p} \left( s_{i-1}, h_j \right) = a\left( s_{i-1}, h_j \right) \cdot \exp \left( -\frac{ \left(j - p_t\right)^{2} }{2\sigma^2} \right)
  \end{equation*}
  \begin{itemize}
    \item Use a gaussian penalty to favor words near $p_t$
    \item Note: $p_t \in \mathbb{R}$, $j \in \mathbb{Z}$
    \item $j$ is the \textit{integer} position of the encoding within the window
    \item Empirically set $\sigma = \frac{D}{2}$
    \item $a\left( s_{i-1}, h_j \right)$ is any previous attention mechanism
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Luong Attention Visuals}
  \begin{columns}
    \begin{column}{0.45\textwidth}
      \vspace{-5mm}
    \begin{figure}
      \includegraphics[width=0.95\textwidth]{assets/global}
      \caption{Global Attention}
    \end{figure}
    \vspace{-7mm}
    \begin{figure}
      \includegraphics[width=0.95\textwidth]{assets/local_p}
      \caption{Local-P}
    \end{figure}
    \end{column}

    \begin{column}{0.45\textwidth}
      \vspace{-5mm}
    \begin{figure}
      \includegraphics[width=0.95\textwidth]{assets/local_m}
      \caption{Local-M}
    \end{figure}
    \vspace{-7mm}
    \begin{figure}
      \includegraphics[width=0.95\textwidth]{assets/gold}
      \caption{Gold Alignments}
    \end{figure}
    \end{column}

  \end{columns}
\end{frame}

\begin{frame}
\frametitle{Fine-Grained Attention}
\begin{equation*}
  a^k \left( s_{i-1}, h_j \right) = \texttt{FF}^k(s_{i-1}, h_j)
\end{equation*}
\begin{equation*}
  \alpha_{ij}^d = \frac{\exp\left(a^d\left(s_{i-1}, h_j\right)\right)}{\sum_k \exp \left( a^d \left(s_{i-1}, h_k\right) \right)}
\end{equation*}
\begin{equation*}
  c_i = \sum_j \alpha_{ij} \odot h_j
\end{equation*}
\begin{itemize}
  \item What's different?
  \item Instead of a scalar scoring function, have it output a weight \textit{per} dimension
  \item Now a weighted average on the feature dimension \textit{instead} of the entire encoding
\end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Fine-Grained Attention}
  \begin{equation*}
    c_i = \textcolor{BrickRed}{0.64} \cdot \begin{bmatrix*}[r] 0.17 \\ 0.53 \\ -0.22 \end{bmatrix*} + \textcolor{BrickRed}{0.12} \cdot \begin{bmatrix*}[r] 1.34 \\ 0.39 \\ 0.45 \end{bmatrix*} + \textcolor{BrickRed}{0.24} \cdot \begin{bmatrix*}[r] -0.08 \\ -1.39 \\ 1.21 \end{bmatrix*}
  \end{equation*}
  \vspace{1.5cm}
  \begin{equation*}
    c_i = \begin{bmatrix} \textcolor{WildStrawberry}{0.59} \\ \textcolor{Fuchsia}{0.18} \\ \textcolor{ForestGreen}{0.16} \end{bmatrix} \odot \begin{bmatrix*}[r] 0.17 \\ 0.53 \\ -0.22 \end{bmatrix*} + \begin{bmatrix} \textcolor{WildStrawberry}{0.09} \\ \textcolor{Fuchsia}{0.36} \\ \textcolor{ForestGreen}{0.74} \end{bmatrix} \odot \begin{bmatrix*}[r] 1.34 \\ 0.39 \\ 0.45 \end{bmatrix*} + \begin{bmatrix} \textcolor{WildStrawberry}{0.32} \\ \textcolor{Fuchsia}{0.46} \\ \textcolor{ForestGreen}{0.10} \end{bmatrix} \odot \begin{bmatrix*}[r] -0.08 \\ -1.39 \\ 1.21 \end{bmatrix*}
  \end{equation*}
\end{frame}

\begin{frame}
\frametitle{Multi-Headed Attention}
\begin{equation*}
  \alpha_{ij}^k = \frac{\exp\left(a^k\left(s_{i-1}, h_j\right)\right)}{\sum_k \exp \left( a^k \left(s_{i-1}, h_k\right) \right)}
\end{equation*}
\begin{itemize}
  \item Why not have $K$ attention weights?
\end{itemize}
\begin{equation*}
  c_i^{k} = \sum_j \alpha_{ij}^k \cdot h_j
\end{equation*}
\begin{equation*}
  c_i =\frac{1}{K} \sum_k c_i^k
\end{equation*}
\begin{itemize}
  \item \textit{Multi-Headed Attention} is just aggregating several attended representations of the encodings
  \item Fancy way of ensembling attention mechanisms
\end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Multi-Headed Attention}
  \begin{equation*}
    \alpha_{ij}^d = \frac{\exp\left(a^d\left(s_{i-1}, h_j\right)\right)}{\sum_k \exp \left( a^d \left(s_{i-1}, h_k\right) \right)}
  \end{equation*}
  \begin{equation*}
    c_i^{k} = \sum_j \alpha_{ij}^k \cdot h_j
  \end{equation*}
  \begin{equation*}
    c_i = \begin{bmatrix*} c^0_{i} \\ \vdots \\ c^k_{i} \\ \vdots \\ c^K_{i} \end{bmatrix*} W_a
  \end{equation*}
  \begin{itemize}
    \item \textbf{Alternative:} Concat each context vector and use a linear layer to reduce the dimensionality
    \item Weighted Average of $K$ 'attended' context vectors!
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Multi-Headed Attention}
  \begin{figure}
      \centering
      \begin{tikzpicture}[shorten >=1pt,node distance=1.25cm,on grid,auto, every node/.style={scale=0.75},
        el/.style={inner sep=2pt, align=left, sloped}]

        \node[draw,diamond]  (h2)  {$h_j$};
        \node[]  (h1) [above=of h2] {$\vdots$};
        \node[]  (h3) [below=of h2] {$\vdots$};
        \node[draw,diamond] [above=of h1]  (h0)  {$h_0$};
        \node[draw,diamond]  (h4) [below=of h3] {$h_{k}$};

        \node[] (a0) [right=of h2, xshift=4cm] {$\texttt{Attention}$};
        \node[] (tmp1) [above=of a0, yshift=-6mm] {$\vdots$};
        \node[] (a1) [above=of tmp1, yshift=-6mm] {$\texttt{Attention}$};
        \node[] (tmp2) [below=of a0, yshift=6mm] {$\vdots$};
        \node[] (a2) [below=of tmp2, yshift=6mm] {$\texttt{Attention}$};

        \node[] (tmp3) [above=of a1, yshift=-6mm] {$\vdots$};
        \node[] (tmp4) [below=of a2, yshift=6mm] {$\vdots$};
        \node[] (a3) [above=of tmp3, yshift=-6mm] {$\texttt{Attention}$};
        \node[] (a4) [below=of tmp4, yshift=6mm] {$\texttt{Attention}$};

        \node[] (s) [left=of a3, xshift=-2.5cm, yshift=8mm] {$s_{i-1}$};

        \node[] (agg) [right=of a0, xshift=2.5cm] {$\texttt{Aggregate}$};

        \node[] (c) [right=of agg, xshift=0.5cm] {$c_i$};

        \path[->]
          (h0) edge [bend right=10] node[] {} (a0)
          (h2) edge [] node[] {} (a0)
          (h4) edge [bend left=10] node[] {} (a0)
          (h0) edge [bend right=10] node[] {} (a1)
          (h2) edge [] node[] {} (a1)
          (h4) edge [bend left=10] node[] {} (a1)
          (h0) edge [bend right=10] node[] {} (a2)
          (h2) edge [] node[] {} (a2)
          (h4) edge [bend left=10] node[] {} (a2)
          (h0) edge [bend right=10] node[] {} (a3)
          (h2) edge [] node[] {} (a3)
          (h4) edge [bend left=10] node[] {} (a3)
          (h0) edge [bend right=10] node[] {} (a4)
          (h2) edge [] node[] {} (a4)
          (h4) edge [bend left=10] node[] {} (a4)

          (s) edge [bend right=15] node[] {} (a0)
          (s) edge [bend left=15] node[] {} (a1)
          (s) edge [bend right=15] node[] {} (a2)
          (s) edge [bend left=15] node[] {} (a3)
          (s) edge [bend right=15] node[] {} (a4)

          (a0) edge [] node[el] {$c_{i}^k$} (agg)
          (a1) edge [bend right=10] node[] {} (agg)
          (a2) edge [bend left=10] node[] {} (agg)
          (a3) edge [bend right=10] node[el, above] {$c_{i}^0$} (agg)
          (a4) edge [bend left=10] node[el, below] {$c_{i}^K$} (agg)

          (agg) edge [] node[] {} (c);

      \end{tikzpicture}
    \end{figure}
\end{frame}

% \begin{frame}
%   \frametitle{MHA Visual from the Paper}
%
% \end{frame}

\begin{frame}
  \frametitle{Self-Attention: Generalized}
  \begin{equation*}
    \texttt{Attention}\left(Q, K, V \right) = \texttt{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V
  \end{equation*}
  \begin{itemize}
    \item Query $Q$, Key $K$, and Value $V$
    \item $d_k$ is the dimensionality of the query and keys
    \item \textbf{Note} that this implies that we can re-mix the encodings via linear layers before attending
    \item \textbf{Note} the similiarity to Luong's Scaled Dot Attention
  \end{itemize}
\end{frame}

% MESSY
\begin{frame}
  \frametitle{Self-Attention: What are Queries, Keys, and Values?}
  % \begin{itemize}
  %   \item \textbf{Encoders}
  %   \begin{itemize}
  %     \item Source Embeddings, optionally re-mixed with linear layers
  %   \end{itemize}
  % \end{itemize}
  % \begin{itemize}
  %   \item \textbf{Decoders}
  %   \begin{itemize}
  %     \item More complicated...
  %     \item Technically 2 Self-Attentions
  %     \item One that is the decoded embeddings
  %     \item Then use that as the value for a second attention
  %   \end{itemize}
  % \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Self-Attention: Packing the Encodings}
  \begin{itemize}
    \item To create Queries, Keys, and Values, we must pack the encodings
    \item Create a matrix $H \in \mathbb{R}^{S \times d}$
    \item $S$ is sequence length, $d$ is the dimensionality of the vectors
  \end{itemize}
  \begin{equation*}
    H = \begin{bmatrix}
    \longleftarrow h_0 \longrightarrow \\
    \vdots \\
    \longleftarrow h_j \longrightarrow  \\
    \vdots \\
    \longleftarrow h_k \longrightarrow
\end{bmatrix}
  \end{equation*}
\end{frame}

\begin{frame}
  \frametitle{Self-Attention: Computation Graph}
  \begin{figure}
      \centering
      \begin{tikzpicture}[shorten >=1pt,node distance=1cm,on grid,auto, every node/.style={scale=0.75},
        el/.style={inner sep=2pt, align=left, sloped}]

        \node[draw,diamond]  (H)  {$H$};

        \node[] (WK) [below=of H] {$\texttt{linear}$};
        \node[] (WQ) [right=of WK, xshift=1cm] {$\texttt{linear}$};
        \node[] (WV) [left=of WK, xshift=-1cm] {$\texttt{linear}$};

        \node[] (Q) [below=of WQ] {$Q$};
        \node[] (K) [below=of WK] {$K$};
        \node[] (V) [below=of WV] {$V$};

        \node[] (matmul) [below=of Q, xshift=-1.15cm] {$\texttt{matmul}$};
        \node[] (scale) [below=of matmul] {$\texttt{scale}$};
        \node[] (softmax) [below=of scale] {$\texttt{softmax}$};
        \node[] (matmul2) [below=of softmax, xshift=-1.2cm] {$\texttt{matmul}$};

        \node[] (c) [below=of matmul2] {$\texttt{output}$};

        \path[->]
        (H) edge [bend left=20] node[] {} (WQ)
        (H) edge [] node[] {} (WK)
        (H) edge [bend right=20] node[] {} (WV)

        (WQ) edge [] node[] {} (Q)
        (WK) edge [] node[] {} (K)
        (WV) edge [] node[] {} (V)

        (Q) edge [bend left=20] node[] {} (matmul)
        (K) edge [bend right=20] node[] {} (matmul)
        (matmul) edge [] node[] {} (scale)
        (scale) edge [] node[] {} (softmax)
        (softmax) edge [bend left=20] node[] {} (matmul2)
        (V) edge [bend right=20] node[] {} (matmul2)
        (matmul2) edge [] node[] {} (c);

      \end{tikzpicture}
    \end{figure}
\end{frame}

\section{Practical Considerations}
% Rework
\begin{frame}
\frametitle{Masking}

\end{frame}

\begin{frame}
\frametitle{Which one to use?}
\begin{itemize}
  \item Trend seems to be moving towards \textbf{Transformers}
  \item AKA a series of Self-Attentions inside Multi-headed Attentions
  \item Ideally, experiment and go crazy
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Mixing and Matching}
  \begin{itemize}
    \item Remember! You can always mix and match!
    \item Different attention mechanisms under MHA
    \item Etc, etc
  \end{itemize}
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Tools, References, and Further Reading}

\begin{frame}
\frametitle{Papers}
  \begin{itemize}
    \item \href{https://arxiv.org/pdf/1409.0473}{Bahdanau et al., Neural Machine Translation by Jointly Learning to Align and Translate}
    \item \href{https://arxiv.org/pdf/1508.04025}{Luong et al.m Effective Approaches to Attention-based Neural Machine Translation}
    \item \href{https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf}{Vaswani et al., Attention is All You Need}
    \item \href{http://www.aclweb.org/anthology/N13-1073}{Dyer et al., A Simple, Fast, and Effective Reparameterization of IBM Model 2}
  \end{itemize}
\end{frame}

% Refs, ideas, etc

\end{document}


\subsection{The circle as a topology}


% Lecture for ph2a Caltech 2017: Vibrations and Waves
\documentclass[pdf,hideothersubsections]{beamer}
\usepackage{beamerthemeshadow}
\mode<presentation>
  {
    \usefonttheme{structuresmallcapsserif}
    \usetheme{CambridgeUS}
    \usecolortheme{seahorse}
    %\useinnertheme{circles}
%    \useoutertheme{tree}
  }

\usepackage{svg}
\usepackage{xmpmulti}

\usepackage{hyperref}
\hypersetup{
    pdffitwindow=true,     % window fit to page when opened
    colorlinks=true,       % false: boxed links; true: colored links
    linkcolor=orange,          % color of internal links (change box color with linkbordercolor)
    citecolor=green,        % color of links to bibliography
    filecolor=magenta,      % color of file links
    urlcolor=blue,           % color of external links
   pdfstartview={Fit}
}

% Fonts/encoding
\renewcommand{\UrlFont}{\tiny}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
%\usepackage[sc,medium,raggedright]{titlesec}
\usepackage{newtxmath}
%\usepackage{libertine}
\usepackage[osf]{ebgaramond}

\graphicspath{{Figures/}}

\begin{document}
\title{Driven Oscillations and Resonance}
\author{Caltech: ph2a}
\date{5 - Oct - 2017}
%\logo{\includegraphics[height=0.5cm]{../caltech_logo.png}}

\frame{\titlepage}

\frame{\frametitle{Table of contents}\tableofcontents}

\setbeamerfont{footnote}{size=\tiny}

\section{Previous Summary}
\begin{frame}
\frametitle{Recently}
\begin{itemize}
\item Q is a measure of the 'Quality' of an oscillator; how many cycles it
  takes to ring down: $Q = \omega_0 \tau / 2$
\pause
\item Air track demo shows how this effect works for a driven oscillator
\pause
\item High Q oscillators have a narrow 'bandwidth'. The resonance has
  a sharp peak in the frequency domain.
\pause
\item Undamped SHO energy $= \frac{1}{2}m \dot{x}^2 + \frac{1}{2}k x^2$
\end{itemize}
\end{frame}

\section{Driven Oscillations}
\begin{frame}
\frametitle{Overview}
\pause
\begin{enumerate}
\item For free vibrations, $m \ddot{x} + b \dot{x} + k x = 0$. For a
  driven system the R.H.S. is \emph{not} equal to zero.
\pause
\item For general driven systems, the RHS can be anything: a series of impulses, a few
  cycles of a sine wave, the sum of many sine waves, shark fin wave,
  random noise, etc. Arbitrary forces can be applied to objects.
\pause
\item As we shall see in Lecture \#8, we can usually represent
  arbitrary forces as a sum of sinusoidal forces of differing
  frequencies, phases, and amplitudes.
\pause
\item As a warm up, we consider the case of a harmonically driven,
  damped, simple harmonic oscillator. This simple analysis actually
  solves everything due to a beautiful symmetry between time and frequency.
\end{enumerate}
\end{frame}

\subsection{Solving the Harmonically driven SHO}
\begin{frame}
\frametitle{Setting up the problem}
In the case of a (sinusoidally) driven SHO, the $F$, in $F = m a$, now
has an additonal term:
\pause
\begin{centering}
\begin{align*}
\action<+->{m \ddot{x} &= -k x - b \dot{x} + F(t) \\}
\action<+->{-m \omega^2 x + i \omega b x + k x &= F_0 e^{i (\omega t)} \\}
\action<+->{\frac{A}{F_0} &= \frac{1/m}{\omega_0^2 + i \omega \frac{b}{m} - \omega^2}}
\end{align*}
\end{centering}
\pause
where $A$ is the amplitude of the oscillation ($\tilde{x} = A(\omega) e^{i(\omega t)}$).

\end{frame}

\begin{frame}
\frametitle{The Bode plot: Mag \& Phase vs. Frequency}
\begin{columns}
  \begin{column}{0.5\textwidth}
      \includegraphics[height=0.8\textheight,keepaspectratio]{DrivenSHO.pdf}
  \end{column}
  \begin{column}{0.5\textwidth}
    \begin{enumerate}
      \pause
      \item The magnitude of the response is the absolute value of the
        ratio of the displacement to the force (independent of phase
        lag).
      \pause
      \item To get the \emph{relative} phase between the force and the
        displacement, we need to use Euler's formula: $e^{i \theta} =
        cos{\theta} + i \sin{\theta}$
        \pause
      \item $\theta$ is in units of \emph{radians}, so we multiple by
        $\frac{180}{\pi}$ to get to \emph{degrees} for this plot
    \end{enumerate}
  \end{column}

\end{columns}
\end{frame}

\subsection{Quality Factor}
\begin{frame}
\frametitle{The Quality Factor: I}
\pause
\begin{block}{Definition I:}
$Q \equiv 2 \pi \frac{\rm energy~stored}{\rm energy~dissipated~per~cycle}$
\end{block}
\pause
\begin{block}{Definition II:}
$Q \simeq \omega_0 \tau / 2$ \\
-- accurate for high Q\footnotemark systems
\end{block}

\footnotetext[1]{Not to be confused with 'Q' as a variable indicating charge! }

\end{frame}

\begin{frame}
\frametitle{Decaying Oscillation}

\centering
\includegraphics[width=0.75\textwidth]{damped_sine_wave.pdf}

\end{frame}

\begin{frame}
\frametitle{The Quality Factor: II}

\begin{columns}
\begin{column}{0.4\textwidth}
\pause
\begin{block}{Definition II:}
\centering
$Q \equiv \frac{\omega_R}{\Delta \omega}$
\end{block}
\pause
where $\Delta \omega$ is defined as the width of the peak at the
points at which: \\
\centering
$|A(\omega)|^2 = \frac{1}{2} |A_{max}|^2$

\end{column}

\pause
\begin{column}{0.6\textwidth}
\centering
\includegraphics[width=0.8\textwidth]{DrivenSHO.pdf}

\end{column}
\end{columns}
%\pause
%DEMO: Air Track with magnets and \emph{lasers}.
\end{frame}

\begin{frame}
\frametitle{The Quality Factor: Estimation Test}

Fill in the missing parameters:
\begin{table}
\centering
    \resizebox{\columnwidth}{!}{%
    \begin{tabular}{|l|c|r|}
      \hline
    Item Name                     & Frequency (Hz)  & Quality Factor \\ \hline
    Rubber Stopper                       & 10       & 0.5      \\
    Church Bell (Cologne)                & 330      & 1500     \\
   Tuning Fork on Wood                   &         &          \\
   Mass on Track                         &         &          \\
   Mass on Track (w/ magnetic damping)   &         &          \\
  Suspension Bridges (e.g. Golden Gate)  &         &           \\
  Ionosphere (Schumann resonances)        & 8-30   & 3-5       \\
    Stop Sign (hit with a rubber mallet) & 200      & 1000     \\
    ${}_0S_0$ Dilational Mode of the Earth     & 0.0008   & 3000     \\
    75 mm silicon wafer                  & 800      & 50,000   \\
    Quartz Crystal Resonator             & 10$^6$   & $10^4$\,--\,$10^6$ \\
    Large, High Purity Glass             & 3000     & 10$^8$   \\ \hline
    \end{tabular}}
\end{table}

\end{frame}

\subsection{Resonance}
\begin{frame}
\frametitle{Examples of Resonance}
\begin{itemize}
\pause
\item Wine glass: thin walls, high Q (can be shattered by roosters ?)
\pause
\item Bridges: \href{https://en.wikipedia.org/wiki/Tacoma_Narrows_Bridge_(1940)}{Tacoma
    Narrows} ? (see video)
\pause
\item Ionospheric Resonances (driven by lightning and solar activity)
\pause
\item others?
\end{itemize}
\end{frame}

\begin{frame}
\frametitle{Schumann Resonances}
\includegraphics[width=\textwidth]{Schumann.jpg}
\end{frame}

\section{Energy of Oscillators}
\begin{frame}
\frametitle{Kinetic and Potential Energy}
\pause
\begin{enumerate}
\item total energy, $E =  \frac{1}{2} m v^2 + \frac{1}{2}k x^2$
\pause
\item for an undamped oscillator, the energy is
  conserved. $\frac{1}{2} k A^2$, where $A$ is the amplitude of the
  sinusoidal motion.
\end{enumerate}

\end{frame}

\begin{frame}
\frametitle{Resonant Ampification}
How much amplification do we get from a resonant system?
\pause
\begin{enumerate}
\item For a usual free mass, driven sinusoidally, $F = m a \implies x = - (F_0/m) / \omega^2$
\pause
\item For a resonant mass-spring, we have instead: $A =
  \frac{F_0/m}{\omega_0^2 + i \omega \frac{b}{m} - \omega^2}$
\pause
\item At the resonant frequency, $\omega = \omega_0$, so $A =
  \frac{F_0/m}{i \omega \frac{b}{m}}$ or, substituting our expression
  for Q,...
\pause
\item we see the amplification factor is just $Q$:  $A = -i Q \frac{F_0/m}{w_0^2}$

\end{enumerate}

\end{frame}

%\section{The Drag Force as Damping}
%\begin{frame}
%\frametitle{A Model for Drag Force}
%\end{frame}

\section{Summary}
\begin{frame}
\frametitle{Summary}
\begin{enumerate}
\pause
\item For linear systems, the resultant of two vibrations is just their sum.
\pause
\item Sum of two oscillations with different frequencies leads to \emph{beats}.
\pause
\item In a conservative system, the energy relations determine the
  motions.
%\item Drag force
\end{enumerate}
\end{frame}

\end{document}


%------------------------------------------------------------------------------
% Beginning of journal.tex
%------------------------------------------------------------------------------
%
% AMS-LaTeX version 2 sample file for journals, based on amsart.cls.
%
%        ***     DO NOT USE THIS FILE AS A STARTER.      ***
%        ***  USE THE JOURNAL-SPECIFIC *.TEMPLATE FILE.  ***
%
% Replace amsart by the documentclass for the target journal, e.g., tran-l.
%
\documentclass{amsart}
\input{newmacros}
%\input{algorithm_labels.aux}
%     If your article includes graphics, uncomment this command.
\usepackage{graphicx}
\usepackage[backend=bibtex,style=numeric,natbib=true]{biblatex}
% Use the bibtex backend with the authoryear citation style (which resembles APA)
\usepackage{url}
\usepackage{citesort}
\usepackage{rotating}
\usepackage{geometry}
\usepackage{float}
\geometry{left=30mm, top=20mm}
\usepackage{hyperref}
\addbibresource{mainbib.bib} % The filename of the bibliography

\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{corollary}[theorem]{Corollary}

\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\newtheorem{xca}[theorem]{Exercise}

\theoremstyle{remark}
\newtheorem{remark}[theorem]{Remark}
\numberwithin{equation}{section}

\newcommand*{\defeq}{\mathrel{\vcenter{\baselineskip0.5ex \lineskiplimit0pt
                     \hbox{\scriptsize.}\hbox{\scriptsize.}}}%
                     =}

\newcommand{\la}{\langle}
\newcommand{\ra}{\rangle}

%    Absolute value notation
\newcommand{\abs}[1]{\lvert#1\rvert}

%    Blank box placeholder for figures (to avoid requiring any
%    particular graphics capabilities for printing this document).
\newcommand{\blankbox}[2]{%
  \parbox{\columnwidth}{\centering
%    Set fboxsep to 0 so that the actual size of the box will match the
%    given measurements more closely.
    \setlength{\fboxsep}{0pt}%
    \fbox{\raisebox{0pt}[#2]{\hspace{#1}}}%
  }%
}

\begin{document}

\title{Probabilistic algorithms for constructing \\ approximate matrix decompositions}

%    Information for first author
\author{Alex Nowak}
%    Address of record for the research reported here
\address{Master "Math\'ematiques, Vision et Apprentissage" \\
\'Ecole Normale Sup\'erieure de Cachan}
%    Current address
% \curraddr{Department of Mathematics and Statistics,
% Case Western Reserve University, Cleveland, Ohio 43403}
\email{alexnowakvila@gmail.com}
%    \thanks will become a 1st page footnote.
\thanks{As a project of the course: \textit{Sparsity and Compressed Sensing}
at ENS Cachan given by \href{http://www.gpeyre.com/}{Pr. Gabriel Peyr\'e}}

% %    Information for second author
% \author{Author Two}
% \address{Mathematical Research Section, School of Mathematical Sciences,
% Australian National University, Canberra ACT 2601, Australia}
% \email{two@maths.univ.edu.au}
% \thanks{Support information for the second author.}

%    General info
% \subjclass[2000]{Primary 54C40, 14E20; Secondary 46E25, 20C20}

% \date{January 1, 2001 and, in revised form, June 22, 2001.}

% \dedicatory{This paper is dedicated to our advisors.}

% \keywords{Differential geometry, algebraic geometry}

\begin{abstract}
Low-rank matrix approximations are obliquous in many areas ranging
from data analysis to scientific computing. From a data science point of
view, probably the most important application is due to Principal
Component Analysis (PCA), which aims to reveal hidden linear structure in
massive datasets through a low-rank matrix decomposition.
Consequently, the complexity of the algorithm plays a central role in the
applicability of the algorithms to big data. The most common
approximative factorization
is the so-called truncated singular value decomposition (k-SVD) which can be
computed in $\mathcal{O}(mnk)$ floating-point operations, where $k$ is the target rank
of the decomposition and $m$ and $n$ are the corresponding dimensions of the
matrix.
In this review, we introduce to the reader randomized algorithms
that can achieve the aforementioned task with numerous advantadges compared
to the classical algorithms. These randomized methods are based on the fact that the
image of a low-rank matrix can be approximated by the action
of the matrix to a reasonable amount of random vectors from the input space.
Starting from this point, it is possible to develop
algorithms that achieve a complexity of $\mathcal{O}(mn\log(k))$ for
dense-matrices, matches the flop count of classical
Krylov subspace methods for sparse matrices with a gain in robustness, and
for large matrices that can not be stored in memory (RAM), they
achieve a constant number of passes compared to the
$\mathcal{O}(k)$ for classical algorithms.
\end{abstract}

\maketitle

\input{sections/introduction}
\input{sections/algorithms}
\input{sections/theory}
\input{sections/experiments}

\section{Conclusion}
Randomization schemes for approximate matrix factorization are a powerful and
versatile tool that \textbf{every data scientist should know about}.
These methods are more robust and scalable than classical algorithms,
and despite not achieving the same level of accuracy, the error gap can be
perfectly controlled with the \textit{oversampling parameter} $p$ and
the number of power iterations $q$.

Moreover, these methods rely on theoretical bounds resulting from theorems
on concentration of measure. We have seen in Experiment \ref{fig:exp1-1}
that the bounds on the
expectation can be very loose for certain matrices. One interesting direction
of research can be the sharpening of bounds like \ref{thm:avg-frob-error-gauss}
under additional hypothesis on the matrix $\mtx{A}$.

There are several variants in this set of methods that
are designed for matrices with different properties, see Section \ref{sec:full-algos}.
One drawback is that sometimes they can be not that easy to verify before
applying the algorithm, for e.g, the decaying
rate of the spectrum.

From a more personal point of view, I have personally appreciated both the
theoretical and experimental section of this project and how they complement each other.
I think it is extremely important to have nice experimental results
always grounded by theory.

\printbibliography[heading=bibintoc]
% \bibliographystyle{siam}

% \bibliographystyle{amsplain}
\end{document}

%------------------------------------------------------------------------------
% End of journal.tex
%------------------------------------------------------------------------------
\grid


\section{Project Summary}

Quantum computing is a discipline that has sparked the interest of both physicist and computer scientists for its exciting performance implications. While the field has witnessed rapid progress in the development of new software, hardware technologies and efficient algorithms, the core principles at play have been quite stable. In almost every physical or abstract quantum computer, information encoding and processing occurs via unitary transformations while the result is written to a classical information storage via projective measurements. In this project we aim to investigate \emph{measurement-based quantum computing}, which is a different formulation for quantum computation that provides several advantages over quantum circuits. First, we will start with a review of \emph{graph states}, a general framework for studying the entanglement properties of qubits via graphs. Next, we are going to introduce the notion of a \emph{cluster state} which is a lattice like graph state that we will use to describe computation with. Cluster states are especially useful for this purpose since a careful arrangement of measurements being performed along a direction in the cluster can simulate the logic of any quantum program described by a quantum circuit. After describing the general computational framework, we are going to describe how common quantum computation constructs like Hadamard or phase operators can be realized by this schema. Finally, we are going to use these constructs to effectively describe a well-known quantum algorithm in terms of a measurement-based model. This description will allow us to compare different features of the gate-based model and measurement-based and gain further insight on the possible use cases.