\documentclass{article}

\usepackage{arxiv}

\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc}    % use 8-bit T1 fonts
\usepackage{hyperref}       % hyperlinks
\usepackage{url}            % simple URL typesetting
\usepackage{booktabs}       % professional-quality tables
\usepackage{amsfonts}       % blackboard math symbols
\usepackage{nicefrac}       % compact symbols for 1/2, etc.
\usepackage{microtype}      % microtypography
\usepackage{lipsum}
\usepackage{graphicx}
\usepackage{amsmath,amssymb}
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\graphicspath{ {./images/} }
\usepackage{fixmath}
\usepackage{graphicx}
\usepackage{subcaption}

\DeclareMathOperator{\EX}{\mathbb{E}}% expected value

\title{FREE vs FAST adversarial training}
\author{
	Giuseppe Capaldi\\
	University of Rome "La Sapienza"\\
	\texttt{capaldi.1699498@studenti.uniroma1.it}\\
	\And
	Gianluca Capozzi\\
	University of Rome "La Sapienza"\\
	\texttt{capozzi.1693255@studenti.uniroma1.it}\\
}

\begin{document}

\maketitle

\begin{abstract}
In this work we propose a comparison between two methods for adversarial training: Free and Fast. The code is available at \url{https://github.com/not-a-genius/neuralNetworkExam}.
\end{abstract}

\section{Introduction}
Adversarial training is a method for training robust deep neural networks, training on adversarial examples as a defense against adversarial attacks. This is typically considered to be more expensive than standard training due to the necessity of constructing adversarial examples via a first-order method like projected gradient descent (PGD). The papers presented show that it is possible to train empirically robust models using methods no more costly than standard training in practice.
These methods show results comparable to train a model against PGD but at lower cost in terms of computational time.
The goal of adversarial training is to learn a model which is not only accurate on the data (accuracy on train and test data) but also accurate on adversarially perturbed versions of the data (i.e. accuracy on images perturbed using the PGD attack).
This leads to particular attention at the results we obtained after adversarial training experiments, aware of the presence of an acceptable compromise in accuracy, given a large increase in robustness due to the tradeoff between robustness and generalization \cite{TsiprasEtAl, ZhangEtAl2019a, ShafahiEtAl2019a}.

\subsection{Related Works}
Our work is based on “Adversarial training for free!” paper \cite{ShafahiEtAl2019b}, that presented an algorithm able to eliminate the overhead cost of generating adversarial examples by recycling the gradient information computed when updating the model parameters. While researching related work in the field we discovered a more recent paper, called "Fast is better than free: Revisiting adversarial training" \cite{WongEtAl2020} presenting an improvement of an already known method called FGSM, previously considered ineffective due to what the paper calls “catastrophic overfitting”. This failure condition is preventable with the use of random initialization points. Moreover the same paper reported a further acceleration in training even for the Free algorithm, thanks to standard techniques for efficient training, including cyclic learning rate and mixed-precision arithmetic. This caught our interest and we decided to try to replicate the reported results and compare them with our implementation, in order to confirm or deny these thesis.

\subsection{Dataset and architecture}

We decided to choose CIFAR-10 as the dataset on which our experiments have been conducted. All of them are run on a single NVIDIA RTX 2070-Super GPU.\\
The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.\\
We conducted adversarial training examples using as models ResNet50 and PreActResNet18.
Both ResNet50 and PreActResnet18 results have been reported and compared between them, under the same conditions.

\section{Adversarial Machine Learning}
Adversarial machine learning is a machine learning technique that attempts to fool models by supplying deceptive input (called also adversarial example). Adversarial examples exploit the way artificial intelligence algorithms work to disrupt the behavior of such algorithms. In the past few years, adversarial machine learning has become an active area of research as the role of AI continues to grow in many of the applications we use. There’s growing concern that vulnerabilities in machine learning systems can be exploited for malicious purposes. The main idea to dealt with adversarial examples is represented by adversarial training; it can be traced back to \cite{GoodfellowEtAl2015}, in which models were hardened by producing adversarial examples and injecting them into training data \cite{ShafahiEtAl2019b}.\\
This technique can be summarized as follows: given a network $f_{\theta}$ parametrized by $\theta$, a dataset ($x_i$, $y_i$), a loss function $l$ and a threat model $\Delta$, the learning problem consists in the following optimization problem:
\begin{center}
	$min_{\theta}{\sum_{i}l(f_{\theta}(x_i + \delta), y_i)}$
\end{center}
A typical choice for the adversarial model is to take $\Delta = \{\delta : ||\delta||_{\infty} \le \epsilon\}$ for some $\epsilon > 0$.

\section{PGD attack}
The PGD attack is a white-box attack, which means that the attacker has access to the model's weights, giving to the attacker much more power than a black-box attack (in which the model's weights are not known), as he can specifically craft the attack to fool the chosen model without having to rely on transfer attacks, which often result in human-visible perturbations.
The key to understanding the PGD attack is to frame finding an adversarial example as a contrained optimisation problem. PGD attempts to find the perturbation that maximises the loss of a model on a particular input while keeping the size of the perturbation smaller than a specified amount referred to as epsilon. This constraint is usually expressed as the $l_2$ or $l_{\infty}$ norm of the perturbation and it is added so the content of the adversarial example is the same as the unperturbed sample.
The PGD algorithm can be summarised with the 4 steps below (although the attacker is free to apply any optimisation improvements such as momentum, Adam, multiple restarts, etc...):
\begin{enumerate}
	\item Start from a random perturbation in the $l_p$ ball around a sample;
	\item Take a gradient step in the direction of the greatest loss;
	\item Project perturbation back into the $l_p$ ball if necessary;
	\item Repeat steps 2-3 until convergence.
\end{enumerate}
In our case, adversarial training consists simply of putting the PGD attack inside the training loop, applying a kind of "data augmentation"; in this case, instead of performing random transformations as a preprocessing step, we create specific perturbations that best fool the model and indeed adversarially trained models do exhibit less overfitting when trained on small datasets.\\
The PGD attack is based on the simplest version of what is called Fast Gradient Sign Method, used to approximate the inner maximization of Delta. This could be seen as a relatively inaccurate approximation of the inner maximization for $l_{\infty}$ perturbations, and has the following closed form (as shown in \cite{GoodfellowEtAl2014}):
\begin{center}
	$\delta^{\star} = \epsilon \cdot sign(\nabla_{x}l(f(x), y))$.
\end{center}
A better approximation of the inner maximization is to take multiple, smaller FGSM steps of size alpha. When the iteration leaves the threat model, it is projected back to the set $\Delta$. Multiple restarts within the threat model $\Delta$ typically improve the approximation of the inner maximization even further. The combination of all these techniques is the PGD attack. The problem of such technique is that the number of gradient computations here is proportional to O(MK) in a single epoch, where M is the size of the dataset and K is the number of steps taken by the PGD adversary. This is K times greater than standard training so adversarial training is typically K times slower than standard training. The pseudo-code of the PGD attack is reported in Algorithm 1.
\begin{algorithm}[H]
	\caption{PGD adversarial training for T epochs, given some radius $\epsilon$, adversarial step size $\alpha$ and $K$ PGD steps and a a dataset of size $M$ for a network $f_{\theta}$}
	\begin{algorithmic}[1]
		\For{$t = 1...T$}
		\For{$i = 1...M$}
		\State// Perform PGD adversarial attack
		\State $\delta = 0$ // or randomly initialized
		\For{$j = 1 ... K$}
		\State $\delta = \delta + \alpha\cdot sign(\nabla_{\delta}l(f_{\delta}(x_i + \delta), y_i))$
		\State $\delta = max(min(\delta, \epsilon), -\epsilon)$
		\EndFor
		\State$\theta = \theta - \nabla_{\theta}l(f_{\theta}(x_i + \delta), y_i)$ // Update model weights with some optimizer, e.g. SGD
		\EndFor
		\EndFor
	\end{algorithmic}
\end{algorithm}
The execution of K-PGD in \cite{MadryEtAl2017} took four days on a Titan X (with model WideResNet and dataset CIFAR-10). This led us to consider the reported results \cite{ShafahiEtAl2019b},\cite{WongEtAl2020}, obtained using K-PGD adversarial training, without running them directly.

\section{Free Adversarial training for free!}

Free adversarial training \cite{ShafahiEtAl2019b} claims to reduce the overhead compared to natural training. The idea behind the improved algorithm is the computation of ascent step by reusing the backward pass needed for the descent step. So the gradient with respect to the network parameters is computed on the backward pass and later on this same backward pass the gradient of the loss with respect to the input image is also computed.\\
In order to allow multiple adversarial updates on the same image the training is made on the same minibatch \textit{m} times in a row. Every minibatch is repeated m times before switching to the next minibatch (“minibatch replays”), so the number of epochs can be considered as the number of epochs divided by m, such that the overall number of training iterations remains constant. Note that perturbations are not reset between mini-batches. The value of this parameter is shown to be crucial, since increasing m comes with the risk of increasing generalization error revealing  the presence of a trade-off: robustness is increased at the cost of validation accuracy on natural images.\\
The pseudocode of Free is shown in Algorithm 2.

\begin{algorithm}[H]
	\caption{"Free" Adversarial Training (Free-m)}
	\begin{algorithmic}[1]
		\Require Training samples X, perturbation bound $\epsilon$, learning rate $\tau$, hop steps $m$
		\State Initialize $\theta$
		\State $\delta$ $\leftarrow$ 0
		\For{$epoch = 1...N_{ep}/m$}
		\For{$minibatch \; B \in X$}
		\For{$i = 1...m$}
		\State // Update $\theta$ with stochastic gradient descent
		\State $g_{\theta} \leftarrow \EX_{(x,y) \in B} \nabla_{\theta} [l(x+\delta, y, \theta)]$
		\State $g_{adv} \leftarrow \nabla_{x} l(x+\delta, y, \theta)$
		\State $\theta \leftarrow \theta - \tau g_{\theta}$
		\State // Use gradients calculated for the minimization step to update $\delta$
		\State $\delta \leftarrow + \epsilon \cdot sign(g_{adv})$
		\State $clip(\delta, -\epsilon, \epsilon)$
		\EndFor
		\EndFor
		\EndFor
	\end{algorithmic}
\end{algorithm}
In the choice of m one problem may arise:
it's possible that catastrophic forgetting happens, e.g. when all the “informative” images of one class are in the first few mini-batches. In this extreme case, we do not see useful examples for most of the epoch, and forgetting may occur.
How much does mini-batch replay hurt generalization?
To answer the question we followed \cite{ShafahiEtAl2019b} approach trying different values of \textit{m}, reaching the same results: dropoff in accuracy for small \textit{m}, while we see an inversion in PGD and test accuracy for high values of it. On CIFAR-10, robustness can be reached with values of $m \leq 10$; our best results have been obtained with $m = 8$.\\
It is known that adversarially trained classifiers, e.g. classifiers trained against PGD,  have generative behaviours. When an image is perturbed from one class to another, it adopts features that make it “look” like its adopted class, to a human eye.
Free trained models exhibit this benefit and also smooth and flattened loss surface.
Indeed, standard adversarially trainined classifiers show the property to have a flatten and smooth loss landscape while some defenses work by “masking” the gradients. This techniques makes it difficult to identify
adversarial examples using gradient methods, even though adversarial examples remain present.
Reference \cite{EngstromEtAl2018} argues that gradient masking adds little security, free training does not operate by masking gradients and uses a rough loss surface.

\subsection{Experiments results}
In this section, we present the results of our experiments conducted using Free method and varying the number of minibatch replays. We can notice that the train time increase as \textit{m} increases as expected but in the case of $m=10$ we got values of accuracies lower than $m=8$.\\
All experiments using Free adversarial training in this section are carried out with the same number of epochs and epsilon value $\epsilon = 8/255$. We are interested evaluating the accuracy of the model against the natural images and the accuracy against images perturbed using the PGD attack. In particular, we run PGD using three different values for the number \textit{K} of iterations, which are {10, 20, 50}. The number of random restarts used for PGD is 1; all the other hyper-parameters are the same used by [Shafahi et al.] in \cite{ShafahiEtAl2019b}. The implementation of Free provided by [Wong et al.] in \cite{WongEtAl2020} includes also the optimization of cyclic learning rate and mixed-precision.\\
The following table shows the results when the used model is PreactResNet18:
\begin{table}[hbt!]
\begin{tabular}{|c|l|l|l|l|l|l|p{2cm}|}
\hline
\multicolumn{1}{|l|}{{ }}        & \multicolumn{4}{c|}{{ \textbf{Accuracy using PreActResNet18}}}                                                                                & \multicolumn{3}{l|}{{ }}                                                                                \\ \hline
{ \textbf{Minibatch-replay (m)}} & \multicolumn{1}{p{1.5cm}|}{{ \textbf{Natural Images}}} & \multicolumn{1}{c|}{{ \textbf{PGD-10}}} & \multicolumn{1}{c|}{{ \textbf{PGD-20}}} & \multicolumn{1}{c|}{{ \textbf{PGD-50}}} & \multicolumn{1}{c|}{{ \textbf{Epoch}}} & \multicolumn{1}{p{1.5cm}|}{{ \textbf{Train Time (min)}}} & \multicolumn{1}{p{1.5cm}|}{{ \textbf{Avg Epoch Time (sec)}}} \\ \hline
{ \textit{m} = 2}                & { 0.895}                                        & { 0.324}                                & { 0.297}                                & { 0.290}                                & { 45}                                  & { 22 }                               & { 29.4 }                                 \\ \hline
{ \textit{m} = 4}                & { 0.867}                                        & { 0.471}                                & { 0.457}                                & { 0.451}                                & { 45}                                  & { 44 }                               & { 58.2 }                                 \\ \hline
{ \textit{m} = 8}                & { 0.839}                                        & { 0.498}                                & { 0.486}                                & { 0.483}                                & { 45}                                  & { 88 }                               & { 117 }                                  \\ \hline
{ \textit{m} = 10}               & { 0.821}                                        & { 0.490}                                & { 0.482}                                & { 0.479}                                & { 45}                                  & { 108 }                              & { 143.4 }                                \\ \hline
\end{tabular}
\end{table}

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FreePre/free2.png}
    \caption{ Train and validation accuracies results.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FreePre/free5.png}
    \caption{PGD validation accuracy}
  \end{subfigure}
  \caption{Free Adversarial training using PreActResnet18 (epsilon = 8/255, m = 8, batch size = 128,  epochs = 45)}
  \label{fig:coffee}
\end{figure}

The following results are related to the training and evaluation based on ResNet50 model instead of PreActResnet18. The number of epochs has been increased to 90 to see the impact on the accuracy in the long run.

%Figure \ref{fig:boat1} shows a test and validation accuracy.

\begin{table}[hbt!]
\centering
\begin{tabular}{|c|c|l|l|l|l|l|}
\hline
\multicolumn{1}{|l|}{}        & \multicolumn{2}{c|}{\textbf{Accuracy using ResNet50}}                               & \multicolumn{3}{l|}{\textbf{}}                                                                                & \textbf{}                                \\ \hline
\textbf{Minibatch-replay (m)} & \textbf{Natural Images}                      & \multicolumn{1}{c|}{\textbf{PGD-50}} & \multicolumn{1}{c|}{\textbf{Epoch}} & \multicolumn{1}{p{1.5cm}|}{\textbf{Train Time (min)}} & \multicolumn{1}{p{1.5cm}|}{\textbf{Avg Epoch Time (sec)}} & \multicolumn{1}{c|}{\textbf{Batch size}} \\ \hline
m = 8       & \multicolumn{1}{l|}{0.735} & { 0.404}         & { 90}           & { 118}                     & { 78.6}                        & { 128}               \\ \hline
\end{tabular}
\end{table}

\newpage

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FreeResnet/Figure_2.png}
    \caption{ Train and validation accuracies results.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FreeResnet/Figure_5.png}
    \caption{PGD validation accuracy}
  \end{subfigure}
  \caption{Free Adversarial training using ResNet50 (epsilon = 8/255, m = 8, batch size = 128,  epochs = 90)}
  \label{fig:coffee}
\end{figure}

\section{Fast is better than free}

[Wong et al.] in \cite{WongEtAl2020} consider it possible to have shorter adversarial training times than the Free method proposed by [Shafai et al.] in \cite{ShafahiEtAl2019b} using a modified version of the simple FGSM method, called Fast; if we compare the data from the two original papers [Shafahi et al.] \cite{ShafahiEtAl2019b} and [Wong et al.] \cite{WongEtAl2020}, we can see that the Free method requires 785 minutes for achieving a good robustness on CIFAR-10 against PGD while the Fast method, in order to achieve the same robustness, requires 12.17 minutes. Similar results about the speed-up in adversarial training time can been observed if the chosen dataset is Imagenet.\\
A key difference between Fast/FGSM and Free adversarial training, is that in the latter the perturbation from the previous minibatch is a reasonable starting point for the next minibatch, while FGSM can be used as it is, just starting from a non-zero initial perturbation.\\
So, “Fast” adversarial training is just standard FGSM with random non-zero initialization for the perturbation and this is claimed to be as effective as PGD adversarial training. This idea of non-zero random initial perturbation was already introduced by [Tramèr et al.] in \cite{TramerEtAl2017} but, the key difference between this implementation and Fast is that, the first one uses a more restricted random initialization and step size, which doesn't give a robust model against PGD attack.  So, for this adversarial training, one crucial parameter is the step size $\alpha$ that shows results on par with “Free” ones only choosing values in a particular interval. Indeed, two failure modes arise:

\begin{itemize}

\item a step size too small ($\alpha$ =$\epsilon$) would mean a zero-initialized perturbation and lead to a too weak defense
\item a step size too large ($\alpha$ = $2\epsilon$) will lead to a model overfitted, restricted to specific threats (“catastrophic overfitting”).
\end{itemize}

A good compromise for these parameters is shown to be: $\alpha$ = 1.25, $\epsilon$ = 10/255.\\
Another key difference is that FGSM does not need to repeat mini-batches, but needs two backward passes to compute gradients separately from the perturbation and the model weights. From a computational complexity point of view this means that an epoch of FGSM is equivalent to two epochs of standard training.

\begin{algorithm}[H]
	\caption{FGSM adversarial training for T epochs}
	\begin{algorithmic}[1]
		\Require Dataset of size M, radius $\epsilon$, K PGS steps, step size $\alpha$
		\For{$t = 1...T$}
		\For{$i = 1...M$}
		\State // Perform FGSM adversarial attack
		\State $\delta = Uniform(-\epsilon, \epsilon)$
		\State $\delta = \delta + \alpha \cdot sign(\nabla_{\delta} l(f_{\theta}(x_i + \delta), y_i))$
		\State $\delta = max(min(\delta, \epsilon), -\epsilon)$
		\State // Update model weights with some optimizer, e.g. SGD
		\State $\theta = \theta - \nabla_{\theta} l(f_{\theta}(x_i + \delta), y_i)$
		\EndFor
		\EndFor
	\end{algorithmic}
\end{algorithm}

\subsection{Experiments results}

All experiments using FGSM adversarial training in this section are carried out with the same number of epochs, epsilon value $\epsilon = 8/255$. All PGD adversaries used at
evaluation are run with 1 random restarts for 10,20,50 iterations (with some of the same hyperparameters as those
used by Wong et al. (2019) but further strengthened with random restarts). Speedup with mixed-precision was incorporated with the Apex amp package at the O2 without loss scaling.

\begin{center}
\begin{table}[hbt!]
\begin{tabular}{|c|l|l|l|l|l|l|l|l|}
\hline
\multicolumn{1}{|l|}{{ }} & \multicolumn{4}{c|}{{ \textbf{Accuracy using PreActResNet18}}}                                                                                & \multicolumn{4}{l|}{{ }}                                                                                \\ \hline
{ \textbf{Step-size}}     & \multicolumn{1}{p{1.5cm}|}{{ \textbf{Natural Images}}} & \multicolumn{1}{c|}{{ \textbf{PGD-10}}} & \multicolumn{1}{c|}{{ \textbf{PGD-20}}} & \multicolumn{1}{c|}{{ \textbf{PGD-50}}} & \multicolumn{1}{c|}{{ \textbf{Epoch}}} & \multicolumn{1}{p{1.5cm}|}{{ \textbf{Train Time (min)}}} & \multicolumn{1}{p{1.5cm}|}{{ \textbf{Avg Epoch Time (sec)}}} & \multicolumn{1}{c|}{{ \textbf{Batch size}}} \\ \hline
{ $\alpha = 10$}       & { 0.847}                                        & { 0.480}                                & { 0.463}                                & { 0.450}                                & { 45}                                  & { 21 }                               & { 28.8 }                                & { 128}                                      \\ \hline
{ $\alpha = 10$}       & { 0.850}                                        & { 0.454}                                & { 0.438}                                & { 0.434}                                & { 45}                                  & { 20 }                               & { 25.8 }                                 & { 256}                                      \\ \hline
{ $\alpha = 16$}       & { 0.856}                                        & { 0}                                    & { 0}                                    & { 0}                                    & { 45}                                  & { 21 }                               & { 28.2 }                                 & { 128}                                      \\ \hline
{ $\alpha = 16$}       & { 0.87}                                         & { 0}                                    & { 0}                                    & { 0}                                    & { 45}                                  & { 20 }                               & { 26.4 }                                 & { 256}                                      \\ \hline
\end{tabular}
\end{table}
\end{center}

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FastPre/fast2.png}
    \caption{ Train and validation accuracies results.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FastPre/fast5.png}
    \caption{PGD validation accuracy}
  \end{subfigure}
  \caption{FGSM Adversarial training using Resnet50 (epsilon = 8/255, m = 8, batch size = 128, epochs = 45)}
  \label{fig:coffee}
\end{figure}

The following results are related to the training and evaluation based on ResNet50 model instead of PreActResnet18. The number of epochs has been increased to 90 to see the impact on the accuracy in the long run.

\begin{table}[hbt!]
\begin{tabular}{|c|c|l|l|l|l|l|}
\hline
\multicolumn{1}{|l|}{}    & \multicolumn{2}{c|}{\textbf{Accuracy using Resnet50}}                               & \multicolumn{4}{l|}{\textbf{}}                                                                                \\ \hline
\textbf{Step-size}        & \textbf{Natural Images}                      & \multicolumn{1}{c|}{\textbf{PGD-50}} & \multicolumn{1}{c|}{\textbf{Epoch}} & \multicolumn{1}{c|}{\textbf{Train Time (min)}} & \multicolumn{1}{c|}{\textbf{Avg Epoch Time (sec)}} & \multicolumn{1}{c|}{\textbf{Batch size}} \\ \hline
$\alpha$ = 10 & \multicolumn{1}{l|}{0.766} & { 0.391}         & { 90}           & { 28.18}                   & { 18.78}                       & { 128}               \\ \hline
\end{tabular}
\end{table}

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FastResnet/Figure_2.png}
    \caption{ Train and validation accuracies results.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/FastResnet/Figure_5.png}
    \caption{PGD validation accuracy}
  \end{subfigure}
  \caption{FGSM Adversarial training using Resnet50 (epsilon = 8/255, m = 8, batch size = 128, epochs = 90)}
  \label{fig:coffee}
\end{figure}
\newpage

\section{Final Experiments}
\subsection{Free vs Fast using PreActResnet18}
In this section we compare the best results we obtained from the precedent experiments,
then showing the evolution of validation accuracy in these two methods.
\\
The following table contains the best m and batch size values for free method, obtained from the different free experiments compared with the best step size and batch size for FGSM.
\begin{table}[hbt!]
\begin{tabular}{|l|p{2.1cm}|c|l|l|l|l|l|}
\hline
              & \multicolumn{1}{l|}{}         & \multicolumn{2}{c|}{\textbf{Accuracy using PreactResnet18}}                         & \multicolumn{3}{l|}{\textbf{}}                                                                                & \textbf{}                                \\ \hline
 & \textbf{Minibatch-replays (m) / step size ($\alpha$)} & \textbf{Natural Images}                      & \multicolumn{1}{c|}{\textbf{PGD-50}} & \multicolumn{1}{c|}{\textbf{Epoch}} & \multicolumn{1}{p{1cm}|}{\textbf{Train Time (min)}} & \multicolumn{1}{p{1cm}|}{\textbf{Avg Epoch Time (sec)}} & \multicolumn{1}{c|}{\textbf{Batch size}} \\ \hline
\textbf{FREE} & m = 8 & \multicolumn{1}{l|}{0.845} & { 0.485}         & { 90}           & { 176}                     & { 117}                         & { 128}               \\ \hline
\textbf{FAST} & $\alpha = 10/255$       & \multicolumn{1}{l|}{0.836} & { 0.434}         & { 90}           & { 43}                     & { 28}                         & { 128}               \\ \hline
\end{tabular}
\end{table}

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/preComp/Figure_1.png}
    \caption{ Free vs Fast, Validation accuracies.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/preComp/Figure_2.png}
    \caption{Free vs Fast, PGD validation accuracies.}
  \end{subfigure}
  \caption{Comparing results for Adversarial training using PreActResNet18 in Free vs Fast.}
  \label{fig:coffee}
\end{figure}

\subsection{Free vs Fast using ResNet50}

\begin{table}[hbt!]
\begin{tabular}{|l|p{2.1cm}|c|l|l|l|l|l|}
\hline
              & \multicolumn{1}{l|}{}         & \multicolumn{2}{c|}{\textbf{Accuracy using ResNet50}}                         & \multicolumn{3}{l|}{\textbf{}}                                                                                & \textbf{}                                \\ \hline
 & \textbf{Minibatch-replays (m) / step size ($\alpha$)} & \textbf{Natural Images}                      & \multicolumn{1}{c|}{\textbf{PGD-50}} & \multicolumn{1}{c|}{\textbf{Epoch}} & \multicolumn{1}{p{1cm}|}{\textbf{Train Time (min)}} & \multicolumn{1}{p{1cm}|}{\textbf{Avg Epoch Time (sec)}} & \multicolumn{1}{c|}{\textbf{Batch size}} \\ \hline
\textbf{FREE} & m = 8 & \multicolumn{1}{l|}{0.735} & { 0.404}         & { 90}           & { 176}                     & { 117}                         & { 128}               \\ \hline
\textbf{FAST} & $\alpha = 10/255$       & \multicolumn{1}{l|}{0.766} & { 0.391}         & { 90}           & { 29}                     & { 19}                         & { 128}               \\ \hline
\end{tabular}
\end{table}

\newpage

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/resComp/Figure_1.png}
    \caption{ Free vs Fast, Validation accuracies.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/resComp/Figure_2.png}
    \caption{Free vs Fast, PGD validation accuracies.}
  \end{subfigure}
  \caption{Comparing results for Adversarial training using Resnet50 in Free vs Fast.}
  \label{fig:coffee}
\end{figure}

\subsection{PreAct18 vs ResNet50}
In the following two charts we compare the evolution of Free adv.learning using the two different models ResNet50 and PreActResnet18.

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/freeComp/Figure_1.png}
    \caption{Preact18 vs Resnet50  Validation accuracies.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/freeComp/Figure_2.png}
    \caption{Preact18 vs Resnet50 PGD validation accuracies.}
  \end{subfigure}
  \caption{Free Adversarial training (epsilon = 8/255, m = 8, batch size = 128,  epochs = 90)}
  \label{fig:coffee}
\end{figure}

\begin{figure}[hbt!]
  \centering
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/fastComp/Figure_1.png}
    \caption{ Preact18 vs Resnet50 Validation accuracies.}
  \end{subfigure}
  \begin{subfigure}[b]{0.4\linewidth}
    \includegraphics[width=\linewidth]{images/fastComp/Figure_2.png}
    \caption{Preact18 vs Resnet50 PGD validation accuracies.}
  \end{subfigure}
  \caption{Fast Adversarial training (epsilon = 8/255, $\alpha$ = 10/255, batch size = 128,  epochs = 90)}
  \label{fig:coffee}
\end{figure}

\newpage
\section{Conclusions}
<<<<<<< Updated upstream
Wong et al.'s paper \cite{WongEtAl2020} was useful to reconsider FGSM as a feasible method for adversarial training once step size parameter is tuned correctly, and how simple optimization operations on learning rate adaptation (Cyclic Learning Rate) or based on hardware compatibility with mixed precision computation (Mixed-precision arithmetic) can decrease computation time, with small effort.
FGSM shows results similar to the ones obtained from the "Free" method, considering the same number of epochs, but the total training time suggests FGSM as a faster and better method to speed up adversarial training compared to free, based on our experiments on CIFAR-10.
=======
Wong et al.'s paper was useful to reconsider FGSM as a feasible method for adversarial training once step size parameter is tuned correctly, and how simple optimization operations on learning rate adaptation (Cyclic Learning Rate) or based on hardware compatibility with mixed precision computation (Mixed-precision arithmetic) can decrease computation time, with small effort.
FGSM shows results similar to the ones obtained from the "Free" method, considering the same number of epochs, but the total training time suggests FGSM as a faster and better method to speed up adversarial training compared to free, based on our experiments on CIFAR10.
>>>>>>> Stashed changes

\begin{thebibliography}{100}
	\bibitem{TsiprasEtAl} {Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. ICLR, 1050:11, 2018}

	\bibitem{ZhangEtAl2019a} {Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. ICML, 2019a.}
	\bibitem{ShafahiEtAl2019a} {Ali Shafahi, W Ronny Huang, Christoph Studer, Soheil Feizi, and Tom Goldstein. Are adversarial examples inevitable? ICLR, 2019a.}

	\bibitem{ShafahiEtAl2019b} {Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S
		Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free!
		arXiv preprint
		arXiv:1904.12843, 2019b.}

	\bibitem{WongEtAl2020} {Eric Wong and Leslie Rice and J. Zico Kolter. Fast is better than free: revisiting adversarial learning, arxiv:2001.03994, 2020.}

	\bibitem{GoodfellowEtAl2015} {Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. ICLR, 2015.}

	\bibitem{GoodfellowEtAl2014} {Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.}

	\bibitem{MadryEtAl2017} {Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ICLR, 2017.}

	\bibitem{EngstromEtAl2018} {Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv:1807.10272, 2018.}

	\bibitem{TramerEtAl2017} {Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc-
		Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204,
		2017.}

\end{thebibliography}

\end{document}


% ----------------------------------------------------------------------
\lecture{Advanced modeling}{encoding}
% ----------------------------------------------------------------------
\part{Advanced modeling}
% ----------------------------------------------------------------------
\input{encoding/declarativity}
% ----------------------------------------------------------------------
\section{Queens reloaded}
% ------------------------------
\input{modeling/queens-problem}
\input{encoding/queens}
%
% \input{encoding/methodology}
% \input{hitori}
% ----------------------------------------------------------------------
\section{...}
% ----------------------------------------------------------------------
\section{Summary}
% ------------------------------
\input{encoding/summary}
% ----------------------------------------------------------------------
%
%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


The \ttH cross section measurement is extracted by performing a maximum likelihood fit of the signal and background models to the diphoton invariant mass distribution (\mgg) observed in data.
This fit, described in full detail in Sec.~\ref{sec:tth_results}, relies on the construction of reliable models of the signal and background processes, described in this section.

\subsection{Signal Models} \label{sec:tth_sig_models}
Models of signal (\ttH) and other standard model Higgs boson production modes (which are considered as backgrounds for the \ttH cross section measurement) are built as a function of \mgg, using a Double Crystal Ball plus Gaussian function.
A separate fit is performed for each signal region in each channel.
Additionally, the fits are performed independently for each year of data-taking: 2016, 2017, and 2018, with the final signal model taken as the sum of the signal models for each of the three years, scaling the normalization of the signal model for each year by the appropriate luminosity.
Signal fits are performed separately by year in order to capture the changes in \mgg resolution in each year, due to the evolving CMS ECAL.
As the mass of the Higgs boson, \mH, is not precisely known, the fit parameters of the signal models are modeled as linear functions of \mH.
The \mH dependence is determined by fitting signal models with simulation samples corresponding to three different values of \mH: 120, 125, and 130 GeV.
With three years, two channels and four signal regions per channel, this results in 24 signal models per Higgs boson production mode.
Some representative signal models are shown for \ttH in Fig.~\ref{fig:tth_sig_model}.
\begin{figure} [h!]
    \centering
    \begin{tabular}{c c}
        \includegraphics[width=0.48\linewidth]{figures/tth/TTHLeptonicTag_2016_1.pdf} &
        \includegraphics[width=0.48\linewidth]{figures/tth/TTHHadronicTag_2018_0.pdf}
    \end{tabular}
    \caption{Fitted signal models for simulation of \ttH production, shown for leptonic tag 1 in 2016 (left) and hadronic tag 0 in 2018 (right).}
    \label{fig:tth_sig_model}
\end{figure}

\subsection{Background Models} \label{sec:tth_bkg_models}
The background model in each category represents the smoothly falling spectrum of events in \mgg arising from processes other than Higgs boson production.
The exact shape of this spectrum is not known, so a variety of functional forms are used to fit the \mgg distribution.
Moreover, different choices for the functional form will generally result in different predictions for the background yield under the \mH peak.
For this reason, the choice of functional form used to describe the smoothly falling background is treated as a discrete nuisance parameter.
This strategy is known as the ``discrete profiling method'', first decribed in Ref.~\cite{envelope}.

There are four families of functions considered for the background fits:
\begin{enumerate}
    \item Exponential
    \begin{equation}
        f_N(x) = \sum_{i=0}^N a_i \exp{(-b_i x)}
    \end{equation}
    \item Power Law
    \begin{equation}
        f_N(x) = \sum_{i=0}^N a_i x^{-b_i}
    \end{equation}
    \item Bernstein polynomial
    \begin{equation}
        f_N(x) = \sum_{i=0}^N a_i \binom{N}{i} x^i (1-x)^{N-i}
    \end{equation}
    \item Laurent series
    \begin{equation}
        f_N(x) = \sum_{i=0}^N a_i x^{-4 + \sum_{j=0}^i (-1)^j j}
    \end{equation}
\end{enumerate}
The $a_i$ and $b_i$ are the parameters to be fitted in each case.
In general, as the order $N$ of each family of function is increased, the function gains more tunable parameters and can better fit any arbitrary distribution.
In order to determine the optimal order $N$ of each function that is considered, an F-test~\cite{fisher_1922} is employed to assess the improvement in goodness-of-fit brought by using a higher-order function in the context of the increase in function complexity; a higher-order function is selected only if the improvement is greater than some threshold, chosen to penalize more complex functions.
The final set of functions and their respective orders considered for the background models are shown for a few representative signal regions in Fig.~\ref{fig:tth_bkg_functions}.
\begin{figure} [h!]
    \centering
    \begin{tabular}{c c}
        \includegraphics[width=0.48\linewidth]{figures/tth/multipdf_TTHLeptonicTag_1.pdf}
        \includegraphics[width=0.48\linewidth]{figures/tth/multipdf_TTHHadronicTag_2.pdf}
    \end{tabular}
    \caption{Families of functions considered for the background model, shown for leptonic tag 1 (left) and hadronic tag 2 (right).}
    \label{fig:tth_bkg_functions}
\end{figure}
Unlike the fits for the signal models, the fits for the background models are performed inclusively for all three years of data-taking.
The best-fit function for each signal region is taken as the nominal value of the background.
The final background models, along with uncertainties, are shown in Fig.~\ref{fig:tth_bkg_models} for the same signal regions as shown in Fig.~\ref{fig:tth_bkg_functions}.
\begin{figure} [h!]
    \centering
    \begin{tabular}{c c}
        \includegraphics[width=0.48\linewidth]{figures/tth/bkgplot_TTHLeptonicTag_1.pdf}
        \includegraphics[width=0.48\linewidth]{figures/tth/bkgplot_TTHHadronicTag_2.pdf}
    \end{tabular}
    \caption{Families of functions considered for the background model, shown for leptonic tag 1 (left) and hadronic tag 2 (right).}
    \label{fig:tth_bkg_models}
\end{figure}


\documentstyle[12pt,epsf]{article}
\setlength{\textwidth}{150mm} \setlength{\textheight}{200mm}
\setlength{\topmargin}{5mm} \setlength{\oddsidemargin}{5mm}
\setlength{\evensidemargin}{5mm} \setlength{\parskip}{1mm}
\setlength{\parindent}{10mm} \setlength{\baselineskip}{50mm}
\def\refname{References}
\linespread{1.6}
\begin{document}

\title{ Hamilton-Jacobi theory for one dimensional autonomous systems using
parametric transformations }
\author{ G. Gonz\'alez \thanks{%
E-mail: gabriel${}_{-}$glez@yahoo.com}, \\
{\it Departamento de Matem\'aticas y F{\'\i}sica} \\
{\it I.T.E.S.O.}\\
{\it Perif\'erico Sur \# 8585 C.P. 45090} \\
{\it Guadalajara, Jalisco, M\'exico.}}
\date{}
\maketitle

\begin{abstract}
A necessary and sufficient condition for a parameter
transformation that leaves invariant the energy of a one
dimensional autonomous system is obtained. Using a parameter
transformation the Hamilton-Jacobi equation is solved by a
quadrature. An example of this approach is given.\\
{\em Keywords:} Parameter transformation, Lagrangian, Hamiltonian,
Hamilton-Jacobi equation, autonomous system.\\
{\em PACS:}45.20.Jj
\end{abstract}

\newpage

\section{Introduction}

It is well known that the problem of integrating Hamilton's
canonical equations of motion, which are expressed as a system of
ordinary differential equations of the first order, can be
replaced by an equivalent problem of finding a complete solution
of a single non linear partial differential equation of the first
order, called the Hamilton-Jacobi equation \cite{B1}. If we
succeed in finding a complete solution of the Hamilton-Jacobi
partial differential equation, the motion of the dynamical system
can be easily found directly from this complete solution by using
only elementary algebraic operations \cite{Vu}. \\
The Hamilton-Jacobi equation can be considered the
most elegant and powerful method known for finding the general
solution to the mechanical equations of motion and gives an
important physical example of the deep connection between first
order partial differential equations and first order ordinary
differential equations \cite{Vi}. \\
Each complete integral of the Hamilton-Jacobi equation gives rise
to a family of solutions of Hamilton's equations and is the
generating function of the canonical transformation which maps the
dynamical system to a trivial one with vanishing Hamiltonian. \\
Apart from its practical aspects, the Hamilton-Jacobi theory leads
to a geometric picture of dynamics relating the dynamics to wave
motion and has been the starting point for Schr\"{o}dinger to
state the wave equation in quantum mechanics \cite{Co}. \\
The applicability of using the Hamilton-Jacobi procedure depends
in our ability to solve a first order non linear partial
differential equation which can be very complicated to solve in
some cases \cite{GLo}. The main purpose of this article is to show
that, under certain conditions, it is possible to introduce a
parameter transformation which allow us to obtain a complete
integral for the Hamilton-Jacobi equation for one dimensional
autonomous systems without having to solve the partial
differential equation given by the Hamilton-Jacobi method.

\section{Parameter transformations and the Hamilton-Jacobi equation}

Newton's equation of motion for one dimensional autonomous systems can be
written as the following dynamical system
\begin{equation}
\frac{dx}{dt}=v,\qquad \frac{dv}{dt}=F(x,v),  \label{eq1}
\end{equation}%
where $x$ is the position of the particle, $v$ is the velocity and
$F(x,v)$ is the force divided by the mass of the particle. It is
well known that the Lagrangian $L(x,v)$ for a one dimensional
system always exist \cite{D} thus we may obtain (\ref{eq1}) from
the Euler-Lagrange equation \cite{B1}
\begin{equation}
\frac{d}{dt}\left( \frac{\partial L}{\partial v}\right) =\frac{\partial L}{%
\partial x},  \label{eq2}
\end{equation}%
and the associated constant of motion for (\ref{eq1}) from the Legendre
transformation \cite{B1}
\begin{equation}
K(x,v)=v\frac{\partial L}{\partial v}-L,  \label{eq3}
\end{equation}%
the numerical value of (\ref{eq3}) will be referred as the energy
of the system. Knowing the constant of motion of (\ref{eq1}) we
can obtain the associated Hamiltonian by expressing the constant
of motion in terms of the canonical variables $H(x,p)=K(x,v(x,p))$
and using this Hamiltonian we obtain the Hamitlon-Jacobi equation
\begin{equation}
H\left(x,\frac{\partial S}{\partial x}\right)+\frac{\partial
S}{\partial t}=0, \label{eq3a}
\end{equation}
where $S$ is known as Hamitlon's principal function \cite{B1}.\\
Let us now consider the following parameter transformation of the
form
\begin{equation}
\tau =\tau (t),\qquad \tilde{x}=x(t),  \label{eq4}
\end{equation}
which does not affect the position variable and it will be assumed
that the functions $\tau (t)$ are of class $C^{2}$ such that
$d\tau /dt>0$. The problem is to find a parameter transformation
which leaves invariant the energy of the system when subject to
this type of transformation. To do this let $L(x,v)$ and
$\tilde{L}(\tilde{x},\tilde{v})$ be the Lagrangians for
(\ref{eq1}) before and after the parameter transformation
respectively. Assume that $\partial ^{2}L/\partial v^{2}$ and
$\partial ^{2}\tilde{L}/\partial \tilde{v}^{2}$ do not vanish or
become infinite in some region $R$ of the dynamical space then we
have the following

\newtheorem{ein}{Theorem}
\begin{ein}
A necessary and sufficient condition for a parameter
transformation to leave invariant the energy of a one dimensional
autonomous system is that \\
\[ \frac{\partial^{2}L}{\partial v^2}\frac{dv}{dt}=\frac{\partial^{2}\tilde{L}}
{\partial \tilde{v}^2}\frac{d\tilde{v}}{d\tau}. \]
\end{ein}

\newtheorem{proof}{Proof}
\begin{proof}

The Euler-Lagrange equation for $L(x,v)$ may be written as
\[
\frac{\partial^{2}L}{\partial x\partial
v}v+\frac{\partial^{2}L}{\partial v^2}\frac{dv}{dt}=\frac{\partial
L }{\partial x}
\]
similarly the Euler-Lagrange equation for
$\tilde{L}(\tilde{x},\tilde{v})$ is given by
\[
\frac{\partial^{2}\tilde{L}}{\partial \tilde{x}\partial
\tilde{v}}\tilde{v}+\frac{\partial^{2}\tilde{L}}{\partial
\tilde{v}^2}\frac{d\tilde{v}}{d\tau}=\frac{\partial \tilde{L}
}{\partial \tilde{x}},
\]
by assumption of the theorem we have
\[
\frac{\partial L }{\partial x}-\frac{\partial^{2}L}{\partial
x\partial v}v=\frac{\partial \tilde{L} }{\partial
\tilde{x}}-\frac{\partial^{2}\tilde{L}}{\partial \tilde{x}\partial
\tilde{v}}\tilde{v}
\]
therefore
\[
\frac{\partial}{\partial x}\left(L-v\frac{\partial L }{\partial v}
\right)= \frac{\partial}{\partial
\tilde{x}}\left(\tilde{L}-\tilde{v}\frac{\partial \tilde{L}
}{\partial \tilde{v}} \right),
\]
which implies
\[
\frac{\partial K(x,v)}{\partial
x}=\frac{\partial\tilde{K}(\tilde{x},\tilde{v})}{\partial
\tilde{x}}.
\]
Taking the total time derivative of
$\tilde{K}(\tilde{x},\tilde{v})$ we obtain
\[
\frac{d\tilde{K}}{dt}=\frac{\partial\tilde{K}}{\partial
\tilde{x}}\frac{d\tilde{x}}{d\tau}\frac{d\tau}{dt}+\frac{\partial\tilde{K}}{\partial
\tilde{v}}\frac{d\tilde{v}}{d\tau}\frac{d\tau}{dt},
\]
using the fact that $\partial
\tilde{K}/\partial\tilde{v}=(\partial ^{2}\tilde{L}/\partial
\tilde{v}^{2})\tilde{v}$ then
\[
\frac{d\tilde{K}}{dt}=\frac{d\tilde{x}}{d\tau}\frac{d\tau}{dt}\left(\frac{\partial
\tilde{K}}{\partial\tilde{x}}+\frac{\partial^{2}\tilde{L}}
{\partial \tilde{v}^2}\frac{d\tilde{v}}{d\tau} \right)
\]
but $dx/dt=(d\tilde{x}/d\tau)(d\tau/dt)$, $\partial K/\partial
x=\partial\tilde{K}/\partial \tilde{x}$ and $\partial
^{2}L/\partial v^{2}(dv/dt)=\partial ^{2}\tilde{L}/\partial
\tilde{v}^{2}(d\tilde{v}/d\tau)$ then
\[
\frac{d\tilde{K}}{dt}=\frac{dK}{dt}
\]
which implies that $K(x,v)=\tilde{K}(\tilde{x},\tilde{v})$ and
completes the proof of the theorem.

\end{proof}

The main advantage of using this type of parameter transformation
is that neither the position or the energy of the system are
affected, using this fact and taking into account that we are
dealing with an autonomous system we can express the solution to
the Hamilton-Jacobi equation as \cite{B1}
\begin{equation}
S=W(x)-Et, \qquad \tilde{S}=\tilde{W}(x)-E\tau, \label{eq5}
\end{equation}

where $S$ and $\tilde{S}$ are Hamilton's principal functions for
(\ref{eq1}) before and after the parameter transformation
respectively and $W$ and $\tilde{W}$ represents the solution to a
nonlinear first order partial differential equation given by
\cite{B1}
\begin{equation}
H\left(x,\frac{dW}{dx}\right)=E, \qquad
\tilde{H}\left(x,\frac{d\tilde{W}}{dx}\right)=E, \label{eq6}
\end{equation}
which is usually known as Hamilton's characteristic function. If
we know how the generalized linear momentum transforms under
(\ref{eq4}) and recalling that
\begin{equation}
\frac{dW}{dx}=p, \qquad \frac{d\tilde{W}}{dx}=\tilde{p},
\label{eq7}
\end{equation}
we may express Hamilton's principal function for (\ref{eq1}) as
\begin{equation}
S=\int p(x,\tilde{p}(x,E))\,dx-Et. \label{eq8}
\end{equation}
Therefore, we obtain the solution for the Hamilton-Jacobi equation
by a quadrature. This way of solving the Hamilton-Jacobi equation
is useful to solve complicated problems as it would be shown in
the following example.
\section{Example}
Consider a relativistic particle of mass at rest $m$ under the
action of a constant force $\lambda>0$ and immersed in a medium
that exerts some type of friction which is proportional to the
square of the velocity. The classical equation of motion for this
system is given by
\begin{equation}
m\frac{dv}{dt}=(\lambda-\gamma v^2)(1-v^2/c^2)^{3/2}, \label{eq9}
\end{equation}
where $\gamma$ is a positive real parameter and $c$ represents the
speed of light. Writing (\ref{eq9}) at first order of
approximation in $v^2/c^2$ we have
\begin{equation}
m\frac{dv}{dt}=(\lambda-\gamma v^2)(1-\alpha^2 v^2), \label{eq10}
\end{equation}
where $\alpha^2=\frac{3}{2c^2}$ .The Lagrangian associated to this
system is \cite{G1}
\begin{equation}
L(x,v)=\frac{mv\tanh^{-1}(\alpha v)}{2\alpha}
e^{-2x(\lambda\alpha^2-\gamma)/m}-\frac{m\lambda}{2(\lambda\alpha^2-\gamma)}\left(
e^{-2x(\lambda\alpha^2-\gamma)/m} - 1\right), \label{eq11}
\end{equation}
and the Hamiltonian of this system is given by \cite{G1}
\begin{equation}
H(x,p)= \frac{m}{2}
e^{-2x(\lambda\alpha^2-\gamma)/m}\sum_{n=0}^{\infty}\alpha^{2n}
v^{2n+2}(x,p) +\frac{m\lambda}{2(\lambda\alpha^2-\gamma)}\left(
e^{-2x(\lambda\alpha^2-\gamma)/m} - 1\right), \label{eq12}
\end{equation}
where $v^{2n+2}(x,p)$ is given by
\begin{equation}
v^{2n+2}(x,p)= \left(\frac{p}{m}e^{-2\gamma
x/m}\frac{2n+1}{(n+1)!}\left(\frac{ 2\lambda
x}{m}\right)^{n}\right)^{(2n+2)/(2n+1)}. \label{eq13}
\end{equation}
The Hamiltonian (\ref{eq12}) is valid for the case $|\alpha v|<1$
and has physical meaning only when $p>0$ and $x>0$. Once knowing
the Hamiltonian we can obtain the Hamilton-Jacobi equation for the
system
\begin{eqnarray}
\frac{m}{2}
e^{-2x(\lambda\alpha^2-\gamma)/m}\sum_{n=0}^{\infty}\alpha^{2n}\left(\frac{\partial
S }{\partial x}e^{-2\gamma x/m}\frac{2n+1}{m(n+1)!}\left(\frac{
2\lambda x}{m}\right)^{n}\right)^{(2n+2)/(2n+1)} + \nonumber\\
\frac{m\lambda}{2(\lambda\alpha^2-\gamma)}\left(
e^{-2x(\lambda\alpha^2-\gamma)/m} - 1\right) +\frac{\partial S
}{\partial t}=0,\label{eq14}
\end{eqnarray}
therefore one has to solve (\ref{eq14}) to obtain Hamilton's
principal function, which at first sight may seem like a
formidable task, but it can be done applying the approach
described in the last section.\\
Consider the following parameter transformation
\begin{equation}
\tau=\int_{0}^{t} \sqrt{1-\alpha^{2}v^{2}}\,dt, \qquad
\tilde{x}=x, \label{eq15}
\end{equation}
therefore equation (\ref{eq10}) transforms into
\begin{equation}
m\frac{d\tilde{v}}{d\tau}=\lambda+\tilde{\gamma}\tilde{v}^2,
\label{eq16}
\end{equation}
where $\tilde{\gamma}=\lambda\alpha^2-\gamma$ and the
transformation equations between one set of dynamical variables
$(x,v)$ to the other set of dynamical variables
$(\tilde{x},\tilde{v})$ are given by
\begin{eqnarray}
{x=\tilde{x} \atop
v=\frac{\tilde{v}}{\sqrt{1+\alpha^2\tilde{v}^{2}}}} \qquad
{\tilde{x}=x \atop \tilde{v}=\frac{v}{\sqrt{1-\alpha^2v^{2}}}}
\label{eq17}
\end{eqnarray}
The Lagrangian and the Hamiltonian for (\ref{eq16}) can be
obtained, and are given by \cite{GL2}
\begin{equation}
\tilde{L}(x,\tilde{v})=\frac{m}{2}\tilde{v}^2
e^{-2\tilde{\gamma}x/m}-\frac{m\lambda}{2\tilde{\gamma}}\left(
e^{-2\tilde{\gamma}x/m} - 1\right), \label{eq18}
\end{equation}
\begin{equation}
\tilde{H}(x,\tilde{p})=\frac{\tilde{p}^2}{2m}e^{2\tilde{\gamma}x/m}+
\frac{m\lambda}{2\tilde{\gamma}}\left(e^{-2\tilde{\gamma}x/m} -
1\right),  \label{eq19}
\end{equation}
where we have dropped the tilde for the position variable for the
sake of simplicity. \\
Using (\ref{eq9}),(\ref{eq11}), (\ref{eq16}) and (\ref{eq18}) it
is easy to convince oneself that theorem $1$ is fulfill, therefore
the energy is invariant under transformation (\ref{eq15}). What we
need now is to find how the generalized linear momentum transforms
under (\ref{eq15}), to do that we express
$\tilde{p}=m\tilde{v}exp(-2\tilde{\gamma}x/m)$ in terms of $v$
using (\ref{eq17}), which gives
\begin{equation}
\tilde{p}=m\tilde{v}e^{-2\tilde{\gamma}x/m}=\frac{mve^{-2\tilde{\gamma}x/m}}
{\sqrt{1-\alpha^2v^{2}}}=me^{-2\tilde{\gamma}x/m}\sum_{n=0}^\infty
\frac{(2n)!\alpha^{2n}v^{2n+1}}{2^{2n}(n!)^2}, \label{eq20}
\end{equation}
where we have used the fact that $|\alpha v|<1$ in the last step.
Substituting (\ref{eq13}) into (\ref{eq20}) we have the way the
generalized linear momentum transforms under (\ref{eq15})
\begin{equation}
\tilde{p}=pe^{-2\lambda\alpha^2 x/m}\sum_{n=0}^\infty
\frac{(2n+1)(2n)!}{2^{n}(n+1)!(n!)^2}\left(\frac{\lambda\alpha^2
x}{m} \right)^{n}=pe^{-\lambda\alpha^2 x/m}\left(
I_{0}(\lambda\alpha^2 x/m)+I_{1}(\lambda\alpha^2 x/m)
\right),\label{eq21}
\end{equation}
where $I_{n}(z)$ is the modified Bessel function of the first kind
\cite{A} . Using the fact that
\begin{equation}
\tilde{p}=e^{-\tilde{\gamma}x/m}\sqrt{2mE-\frac{m^2\lambda}{\tilde{\gamma}}
\left(e^{-2\tilde{\gamma}x/m}-1\right)}, \label{eq22}
\end{equation}
and substituting (\ref{eq22}) into (\ref{eq21}) we have
\begin{equation}
p(x,\tilde{p}(x,E))= \frac{e^{\gamma
x/m}\sqrt{2mE-\frac{m^2\lambda}{\lambda\alpha^2-\gamma}
\left(e^{-2(\lambda\alpha^2-\gamma)x/m}-1\right)}}{I_{0}(\lambda\alpha^2
x/m)+I_{1}(\lambda\alpha^2 x/m)},\label{eq23}
\end{equation}
substituting (\ref{eq23}) into (\ref{eq8}) we obtain Hamilton's
principal function for $x>0$
\begin{equation}
S=\int \frac{e^{\gamma
x/m}\sqrt{2mE-\frac{m^2\lambda}{\lambda\alpha^2-\gamma}
\left(e^{-2(\lambda\alpha^2-\gamma)x/m}-1\right)}}{I_{0}(\lambda\alpha^2
x/m)+I_{1}(\lambda\alpha^2 x/m)}\,dx-Et.\label{eq24}
\end{equation}
All the expressions derived in this paper have the right limit when
$\gamma\rightarrow 0$ and $\alpha \rightarrow 0$.  \\

\section{Conclusions}
A necessary and sufficient condition for a parameter
transformation that leaves invariant the energy of a one
dimensional autonomous system was obtained. A method for solving
the Hamilton-Jacobi equation using a parameter transformation was
deduced. All the expressions obtained in this paper converge to
the conservative case
when the dissipation parameter goes to zero. \\

~~~~~\\

\newpage

%\hspace{1cm}

\begin{thebibliography}{99}

\bibitem{A} Andrews C. Larry, (1998). {\it Special functions of mathematics for
engineers}, Oxford Univeristy Press.

\bibitem{Co} Cook B. David, (1988). {\it Schr\"{o}dinger's
Mechanics}, World Scientific Lecture Notes in Physics, Vol. 28.

\bibitem {D} Darboux,  G.  (1984). {\it Lecons sur la th\'eorie g\'en\'eral des
surfaces et les applications g\'eometriques du calcul
infinit\'esimal}, Gauthier-Villars, Paris, IVi\'eme partie.

\bibitem {B1} Goldstein H., (1980). {\it Classical Mechanics},
Addison-Wesley.

\bibitem {G1} Gonz\'alez, G. (2004). To appear in the {\it Internation Journal of Theoretical
Physics}.

\bibitem{GL2} L\'opez, G., Gonz\'alez, G.  (2003). {\it Il Nuovo Cimento B} {\bf 118}, 1.

\bibitem{GLo} L\'opez, Gustavo, (1999). {\it Partial Differential
Equations of First Order and Their Applications to Physics}, World
Scientific.

\bibitem {Vi} Vilasi, Gaetano,(2001).
 {\it Hamiltonian Dynamics}, World Scientific.

\bibitem {Vu} Vujanovic, B. D., Jones, S. E. (1989).
 {\it Variational methods in nonconservative
phenomena}, Academic Press, Inc.

\end{thebibliography}

\end{document}


\appendixchapter{IDT Construction From Captures of Target Materials}{i}

If the digital camera system spectral responsivities cannot be obtained from the camera system manufacturer, and the equipment and environment for spectral sensitivity characterization are unavailable, an alternative approach may provide enough data to construct a digital camera system IDT, albeit one with less flexibility and potentially lower quality than direct use of spectral responsivities would provide. With this alternative approach, target patches are captured by both a reference measurement device (such as, for example, a telespectroradiometer) and by the digital camera system being characterized. The RICD's documented spectral responsivities are combined with the patches' spectral characteristics to produce the RICD's captured values for those patches, and optimization of those two sets of patches—after both have been transformed to some error minimization space—yields the 3x3 matrix at the core of the IDT.

This process can be divided into steps:
\begin{enumerate}
	\item Spectral or colorimetric measurement of target patches and the target illumination source
	\item Calculation of ACES RGB relative exposure values from spectral or colorimetric measurements
	\item Digital camera system capture of the target patches under the target illumination source
	\item Reconstruction of linear camera exposure values from digital camera system OECF and captured patch values
	\item Conversion of reconstructed linear camera exposure and ACES RGB relative exposure values to an error minimization space
	\item Linear regression of reconstructed linear camera exposure values and ACES RGB relative exposure values, as converted to the error minimization space, to produce initial values of 3x3 IDT matrix
	\item Optimization of the 3x3 IDT matrix
	\item Embedding of the optimized 3x3 IDT matrix in an IDT transform
\end{enumerate}

Those steps are discussed in the following sections:

\subsection*{Spectral measurement of target patches}
Telespectroradiometric measurements are preferable to combined spectral reflectance and illumination measurements, especially if the target exhibits fluorescence.

The measurement geometry—the relative position of measurement device or digital camera, sample patch and (for reflective target patches) any illumination source(s)—should be rigidly established and maintained. It is critical that this measurement geometry be carried over to target patch capture with the digital camera. If characterization is done on a lab bench, the measurement device and the digital camera should be on the target normal at target center, and the target illumination should be from two opposing directions at 45$^\circ$ to the target normal. The illumination sources must have identical spectral characteristics. The illumination sources should be sufficiently distant from the target so as not to cause a combined illumination variation exceeding $\pm$1\% across the target area. Two omni-directional light sources positioned as above, and at a height exceeding four times the largest target dimension, results in a variation within $\pm$1\%. (If characterization is done on–set in daylight conditions then the requirement for opposing directions is dropped.) Neither the target's visibility to the capture device nor the illumination sources' visibility to the target should be in any way obstructed.

The acceptance angle of the measuring instrument, the instrument's position relative to the target patch and the size of the target patch should be such that the edges of the target patch are not being measured: if the patch is recessed in its support, the edges may be shadowed, and moreover the patch center may be more planar than the patch edges. Typical acceptance angles for measuring instruments, when used for this type of capture, range between ${{}^1/_2}^\circ$ and 2$^\circ$.

Ideally the measurement device or digital camera position would be fixed, the illumination source position and spectral power distribution would be fixed, target patches would be measured individually with the target moved under a mask made from a black material, with mask and target held in a plane perpendicular to the measurement device or digital camera. As with recessed patches, however, this risks shadows being cast into the sampled target area by the target patch mask.

\subsection*{Calculation of ACES RGB relative exposure values from measurements}
The ACES R, G and B relative exposure values are obtained by a two-step process.

\begin{enumerate}
	\item Compute the dot products of the measured spectral radiance of the target patch (either emitted radiance, or reflected illumination radiance) and the corresponding area-normalized RICD spectral sensitivities given in Annex C of the ACES document.
	\item Augment the computed values with the amount of camera flare associated with the RICD, as specified in the ACES document.
\end{enumerate}

\subsection*{Digital camera capture of the target patches}
The considerations of in situ capture, measurement geometry, avoidance of patch edges and use of target patch mask have been covered in a previous section. Digital camera capture requires several other factors be considered:

\begin{itemize}
	\item The digital camera system data should not be irrevocably modified as part of any color rendering process. Digital camera system output that has been color rendered, typically encoded as video files, often has been internally altered in some spatially and perhaps temporally variant way to suit aesthetic preferences and has lost its relationship to scene radiometry.
	\item Capture should be at a single exposure level; automatic exposure determination should be disabled. The aperture and exposure time should be established such that clipping of the chart patches is avoided. Usually this can be accomplished when a gray card is captured with `normal' exposure.
	\item Capture should be with the lens and filter(s) which will be used in production, including filtration for white balance, for reduction of UV, IR or unpolarized light, or for overall reduction of exposure with ND filters.
	\item Digital camera image compression should be `lossless', or disabled entirely.
	\item No aesthetic corrections or color-balancing operations should be done between capture and analysis.
	\item The captured data should be radiometrically linear, whether that linearity be native to the captured image format, or the result of inversion of the native camera system image encoding. If natively linear, there should be at least 10 bits of linear data per RGB component. If linear is the result of an inversion of a log or power function used by the native camera system image encoding data metric, then the data prior to inversion should have at least 8 bits of encoded data per RGB component. The preservation of radiometric linearity can be verified by measurement and capture of a series of spectrally neutral targets.
	\item If multiple target patches are acquired in a single exposure, the target should occupy the center of the frame and its image on the camera sensor should not extend into regions where lens vignetting or other factors influence transmitted energy relative to that transmitted along the common camera and lens axis.
	\item The black level should be captured. This allows for estimating and removing the dark current. The black level should be captured with the same exposure settings as used to capture the patches, for example by covering the lens with a lens cap. The black level should be captured immediately after capturing the spatial illumination, as this tends to reduce the black current in defect pixels to levels commonly found in actual production.
	\item The spatial distribution of the illumination should also be captured. This allows for compensation of uneven illumination and vignetting. This should be captured by replacing the target with a white card, gray card, or similar object having a uniform color and non-specular Binary Reflectance Distribution Function (BRDF). The replacement object must have the same placement as the target, so as not to change the reflected light.
	\item The captured data for the target patch should be spatially (and if multiple captures are possible, temporally) averaged prior to use. As mentioned above, the averaged pixels should be from target patch center; target patch edges may be in shadow or nonplanar and should be avoided. At least 1,000 pixels, preferably 10,000 pixels, should be included in the computation of the average. The standard error of mean should not exceed 0.1\% of the average value.
\end{itemize}

\subsection*{Reconstruction of linear camera exposure values from digital camera OECF and captured patch values}
The linear data from the prior step describe the response of the camera to light present at the focal plane (that is, at the sensor). These data should be obtained by

\begin{enumerate}
	\item Averaging any sequences of image data representing multiple captures of a single target patch.
	\item Applying the inverse of the Camera system optoelectronic conversion function (OECF) to the camera system values.\footnote{Determination of the Camera OECF should follow the procedures given in ISO 14524, Photography -- Electronic still-picture cameras -- Methods for measuring opto-electronic conversion functions (OECFs).}
	\item Correcting for the estimated irregularities in the spatial distribution of the illumination.
	\item Subtracting the estimated dark current.
\end{enumerate}

\subsection*{Conversion of reconstructed linear camera system exposure and ACES RGB relative exposure values to an error minimization space}
The reconstructed linear camera system exposure and ACES RGB relative exposure values should be converted to a common error minimization space.

If this common error minimization space depends on the choice of some white tristimulus value, the same CIE XYZ value for white should be used for both sets of conversions.

\subsection*{Regression of linear camera system exposure values and ACES RGB relative exposure values to produce an optimized 3x3 IDT matrix}

Regression of the 3x3 IDT matrix values proceeds in a manner identical to that described in Sections \ref{sec:compmat}, \ref{sec:compcostfn}, and \ref{sec:compend}.

\subsection*{Embedding of the optimized 3x3 IDT matrix in an IDT transform}
The optimized 3x3 IDT matrix is embedded in a CTL transform in a manner identically to that described in \autoref{sec:application}.


\documentclass{article}

\usepackage{url}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{cancel}
\usepackage{subcaption}
\usepackage{graphicx}

\begin{document}

\title{Sigvisa RJMCMC moves}
\author{Dave Moore}
\maketitle

%The SIGVISA state is a special case of the open-world problem described above, in which there are two kinds of unknown objects: unassociated templates, and events. Each unassociated template is specific to an observed waveform (at a particular station, channel, and frequency band), and described by a set of shape parameters, currently including arrival time, peak offset time, height, and coda decay rate. By contrast, each event exists globally, but is described by a large set of parameters, describing both the event itself (lon/lat/depth/time/mb/type) as well as the shape and wiggles of the phase arrivals for each observed waveform. So if we have 100 observed waveforms, and we want to model only P arrivals, with four parameters per arrival, ignoring wiggles, then each event will have 400 parameters, plus the six intrinsic event parameters -- though really only five since event type is a discrete param -- for a total of 405 parameters per event.

\section{Template}

TODO

\subsection{Birth/Death}

The birth move proposes a new template. The {\em peak time} of this template (note that peak time = arrival time + peak\_offset) is sampled proportionally to the height of the current unexplained signal, i.e., the observed signal minus the current hypothesized arrivals. All other template parameters are sampled from the prior. {\em Wiggle} parameters are not currently treated specially (i.e., they are also sampled from the prior, leading to weird results in many cases), but ({\bf TODO}) should in principle also be sampled signal-conditionally, i.e. by running an FFT or similar on the unexplained signal.

The death move kills an unassociated template at random. The template to kill is selected uniformly from all current unassociated templates. In principle, we could be smarter about this, and propose to kill templates with probability inversely proportional to their support in the signal (i.e., the ratio $p(S)/(p_U(T)p_T(S))$), so as to better focus our attention on weak templates. But that would require computing probabilities for each template during every move, effectively requiring $O(n_u)$ work as opposed to $O(1)$ work under the uniform death proposal.

\subsection{Split/Merge}

The merge and split moves allow the combination of two adjacent templates, or the splitting of a single template into two adjacent templates. Note that since we don't in general expect there to be two adjacent templates in the actual signal, the merge move is mostly useful to allow us to combine such templates that have been created by a too-enthusiastic birth move. For example, the first template proposed by a birth move doesn't have a long enough decay time to fully explain the spike in the signal, and before it can adjust, the birth move proposes a second template in the same spot. Now the two templates together explain the signal, but neither one will easily adjust by itself to explain the whole signal, and so neither one can be easily killed. The solution is to merge them together. Since we want a merge move, detailed balance requires a corresponding split move, although I can't think of an example in which this is anything other than a necessary evil.

The merge move begins by sorting all arrivals at a station (including both event and unassociated arrivals) by arrival time. It then proposes merging two adjacent templates $k, k+1$ with probability proportional to
\[p_k \propto \begin{cases}
   0 & \text{if $k$ and $k+1$ are both events}\\
   t_{k+1} - t_k & \text{otherwise}.
\end{cases}\]
The merge operation consists of combining the amplitude of both templates into the shape of just one of them. First, we compute a new log-amplitude $a$ for the merged template, generated by summing the amplitudes of both templates:
\[a_M = \log\left(\exp(a_k) + \exp(a_{k+1})\right).\]
In this way the total amplitude is preserved during the merge (imagine we have two copies of the exact same template at the same time; in merging them, we now have one template with the same height as the sum of the two original templates). Then, we choose one of the two templates, $T'$, to throw out, and we keep the other, $T$. If one of the templates is an event template, we always keep that one, otherwise we make the choice uniformly at random. Finally, we assign the new amplitude $a$ to the merged template $T$, which we now refer to as $M$.

By contrast, in the split move we propose to split an arrival chosen uniformly at random from the set of all arrivals, both events and unassociated. The arrival $j$ that we split from is unchanged, except that it loses an amount of its amplitude, chosen uniformly from $a'_j \sim [0, a_j]$, to a new template $j'$ that we will create. The arrival time of $j'$ is sampled uniformly from between the two adjacent arrivals $[t_{j-1}, t_{j+1}]$; note that this is necessary since the merge move can only merge adjacent arrivals. The other shape parameters for $j'$ are sampled from the prior on unassociated templates (note that $j'$ must be an unassociated template, since the merge operation only throws away unassociated templates).

Note the merge move requires no randomness beyond the choice of move, but the split move requires us to sample an arrival time, amplitude, and other parameters for the new template. All are sampled directly, so their randomness is accounted for in $g'(u')$, except for the amplitude which will contribute to the Jacobian determinant as follows. Let the combined amplitude be $a_M$, and suppose we sample $u'_\text{amp} \sim U(0,1)$. Then we let
\begin{align*}
h'(a_M, u'_\text{amp}) &= [\log( u'_\text{amp} \exp(a_M)), \log((1-u'_\text{amp}) \exp(a_M)) ] \\
&=  [\log( u'_\text{amp}) + a_M, \log(1-u'_\text{amp}) + a_M)]\end{align*}
giving a Jacobian of
\[\left|\frac{\partial(x, u)}{\partial(x',u')}\right| = \left|\begin{array}{cc} 1 & 1/u'_\text{amp}\\1 & -1/(1-u'_\text{amp})\end{array}\right| = \frac{1}{u'_\text{amp}} + \frac{1}{1-u'_\text{amp}} = 1/\left((1-u'\text{amp})\cdot u'_\text{amp} \right)\]

Now an acceptance ratio of
\[\frac{\pi(x')g'(u')j_m(x')}{\pi(x)g(u)j_m(x)}\left|\frac{\partial(x', u')}{\partial(x,u)}\right|\]
\[\frac{p_T(a_M)p_S(M)}{p_T(a_T)p_U(T')p_S(T, T')} \cdot \frac{1/(t_{j+1} - t_{j-1}) \cdot p_U(T') } {1} \cdot \frac{1/(n+1)}{p_k} \cdot (1-u'_\text{amp})\cdot u'_\text{amp}\]

\section{Event}

\subsection{Birth/Death}

Event moves are somewhat complex, because they're not pure birth/death moves in the usual sense. When we birth an event, we create templates at every station in the network, for the arrivals of all modeled phases of that event (formally speaking, these templates are all just a part of the event object). However, if an arrival is actually supported by a significant spike in the signal, we expect that there will already be an unassociated template in the place we'd want to create the new phase arrival. Rather than creating a second template in the same spot, which would have the immediate effect of {\em decreasing} the signal probability (assuming the unassociated template was already a good fit), we want to leverage the work that's already gone into the unassociated template by {\em associating} it with the event. Formally this can be viewed as killing the unassociated template and using its parameters to instantiate the new event template. In this way, an event ``birth move'' is actually responsible for the deaths of many unassociated templates, and conversely an event ``death move'' will birth many new templates (note that this is what allows us to kill an event while still maintaining some explanation for the signal spikes that we had previously thought were caused by the event; without this, all event deaths would pay a huge probability penalty and be summarily rejected).

\begin{figure}
        \centering
        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_AKBB_final}
                \caption{AKASG}
        \end{subfigure}%
        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_YKR8_final}
                \caption{YKA}
        \end{subfigure}%
        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_JNU_final}
                \caption{YKA}
        \end{subfigure}%

        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_IL31_final}
                \caption{ILAR}
        \end{subfigure}%
        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_WR1_final}
                \caption{WRA}
        \end{subfigure}%
        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_FIA0_final}
                \caption{FINES}
        \end{subfigure}%

        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_AS12_final}
                \caption{ASAR}
        \end{subfigure}%
        \begin{subfigure}[b]{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{unass_NV01_final}
                \caption{NVAR}
        \end{subfigure}%
        %\begin{subfigure}[b]{0.3\textwidth}
        %        \centering
        %        \includegraphics[width=\textwidth]{unass_STKA_final}
        %        \caption{STKA}
        %\end{subfigure}%
        \caption{Unassociated templates from 120 steps of MCMC at several stations recording the 2009 DPRK event. Purple indicates individual templates, green shows the sum of all templates. Note that sometimes several templates are generated to explain the same event; this is likely due to the lack of a wiggle model in the current implementation.}\label{fig:templates}
\end{figure}

\begin{figure}
\includegraphics[width=\textwidth]{hough_9stas}
\caption{Hough transform of the unassociated templates in Figure \ref{fig:templates}.}
\label{fig:hough}

\end{figure}

\subsubsection{Global Component}

{\bf Birth proposal:} The first consideration of an event birth move is the proposed event itself: location, depth, time, magnitude, and source type (natural or explosion). For simplicity, all events are proposed with natural sources of magnitude 3.5 at depth 0 (actually, this can't be strictly true, since we need our proposal distribution to be supported everywhere, in order not to screw up the death moves. but it'll be something like this...). For the location and time components, we use a Hough transform to aggregate information from unassociated templates across the network.

We use a three-dimensional Hough accumulator array, corresponding to two spatial dimensions (latitude and longitude) and a time dimension. The width of the time bins is chosen to be roughly equivalent to the time required for a P wave to travel one side length of the spatial bins, so that detections from two events occurring at the same time in the same spatial bin are likely to invert to the same or adjacent time bins.

In the first stage, we compute a separate accumulator array for each station. For each unassociated template $i$ at station $s$, we estimate a probability that the template corresponds to a ``true'' detection: this is taken to be a logistic function of the signal-to-noise ratio for that template, so that templates with high SNR contribute fully, while those near the noise floor are given less weight.
\[w_{s,i} = \frac{1}{1 + \exp(-\alpha \cdot (\text{SNR}(s,i) - \beta))}\]

Currently we have arbitrarily set $\alpha = 2, \beta=4$. Also for each template, for each spatial bin $(x,y)$ in the accumulator we compute the expected origin time under the assumption that the template was generated by an event occurring in the center of the spatial bin:
\[t_{s,i}(x,y) = \text{arrival\_time}(s, i) - E[\text{travel\_time}((x,y), s)].\]
This inverted origin time $t_{s,i}(x,y)$ will fall within some time bin $T$. Let $b$ denote the distance in half-time-bin-widths from $t_{s,i}(x,y)$ to the center of $T$, i.e., we will have $b=0$ if $t_{s,i}(x,y)$ is exactly at the center of $T$, and $b=1$ if it is exactly on the border with an adjacent bin $T'$. For robustness, we spread the weight of the template across the two adjacent bins, according to the (arbitrary) formula
\[w_{s,i}(x, y, T) = w_{s,i} \cdot b^3/2\]
\[w_{s,i}(x, y, T') = w_{s,i} \cdot (1 - b^3/2)\]
which splits the weight evenly when the inverted time is exactly on the border, but with a cubic falloff as the time is centered more closely within the bin. (One might imagine we'd want an exponential falloff, since the travel-time model errors are roughly Laplacian, but we also need to account for the error in having assumed the origin location is the center of the spatial bin, when in fact it might have equally well been any of the corners, so we are a bit more forgiving).

At the level of the station accumulator, we take the weight of a space-time bin to be the {\em maximum} of all templates falling in that bin,
\[w_{s}(x,y,T) = max(\delta, \max_i w_{s,i}(x,y,T)),\]
along with an additional parameter $\delta$, interpreted as the probability of an ``missed detection'', i.e., the probability that an event could have happened in that bin even if there are no corresponding templates. The overall maximization is roughly analogous to computing the most likely association between a hypothetical event in that bin and the templates at station $s$.

Finally, at the global level, we compute the {\em product} of the station weights, essentially treating each station weight as an independent estimate of the probability that an event occurred in a particular bin (of course, this computation is actually implemented in log space). Normalizing, we obtain a global probability distribution on space-time bins. Since our event proposal distribution needs to be continuous (in particular, we'd like it to be supported over the entire earth, since MH will only allow us to kill events in locations where a birth move would be supported), we assume a uniform distribution within each space-time bin.

Figure \ref{fig:hough} shows the distribution resulting from a Hough transform on unassociated templates corresponding to the 2009 DPRK event (shown in Figure \ref{fig:templates}).

{\bf Death proposal:} For the death move, we sample an event to kill with probability proportional to $p_U(T)/p_E(T)$, where $T$ denotes the set of templates generated by arrivals from event $E$, $p_U(T)$ is the probability density of those templates considered under the unassociated template models of their various stations, and $p_E(T)$ is the density of the same templates under the distribution generated by the event $E$.

\subsubsection{Station-specific components}

Here we consider the effect of the birth/death move at individual stations. Since each arriving phase is treated independently in the birth proposal, we focus on a single arriving phase $P$; sometimes referring to the event-phase pair as $(E,P)$.

{\bf Birth proposals:} At each station, we can associate any of the unassociated templates with $(E,P)$, or we can create a new template. Let $p_{E,P}(\cdot)$ denote the distribution on a template at this station generated by phase $P$ of event $E$, and $p_U(\cdot)$ denote the distribution on unassociated templates at this station. Also let $N_U$ be the random variable corresponding to the number of unassociated templates at this station (following, probably, a Poisson distribution), with current value (at the time of the proposal) denoted by $n_U$. Then we propose to associate existing template $i$ with probability

\[a_i \propto \frac{p_{E, P}(T_i)}{p_U(T_i)}\]

versus creating a new template with probability

\[b \propto \frac{p(N_U = n_U)}{p(N_U = n_U - 1)}\]

giving a normalization constant of

\[Z_\text{birth} = \frac{p(N_U = n_U)}{p(N_U = n_U - 1)} + \sum_j  \frac{p_{E, P}(T_j)}{p_U(T_j)}.\]

For another interpretation of these probabilities, let $\alpha_i$ denote the joint distribution over all templates which are currently unassociated at this station, but associating template $i$ with the arrival $(E,P)$,
\[\alpha_i = \left(p_{E, P}(T_i)\prod_{k \ne i} p_U(T_k) \right)p(N_U = n_U - 1),\]
and let $\beta$ denote the joint probability of those same templates left entirely unassociated:
\[\beta = p(N_U = n_U)\prod_k p_U(T_k)\]

It's then straightforward to show that

\[a_i = \frac{\frac{p_{E, P}(T_i)}{p_U(T_i)}}{Z_\text{birth}} = \frac{\alpha_i}{\beta + \sum_j \alpha_j} \propto \alpha_i\]
\[b = \frac{\frac{p(N_U = n_U)}{p(N_U = n_U - 1)}}{Z_\text{birth}} = \frac{\beta}{\beta + \sum_j \alpha_j} \propto \beta,\]

So we can think of the proposal in terms of the Roman letters, corresponding to odds ratios, or Greek letters, corresponding to joint probabilities.

One might imagine that a better proposal would also take into account the signal at the station: if there is a large unexplained spike in the signal at exactly the predicted moment of arrival, maybe that should make us more likely to propose creating a new template there, as opposed to associating some existing template. But that's more complicated to figure out, and in any case, any seriously large spikes should already have an unassociated template fit to them.

%\begin{align*}
%a_i &= \frac{\frac{p_{E, P}(T_i)}{p_U(T_i)}}{\frac{p(N_U = n_U)}{p(N_U = n_U - 1)} + \sum_j  \frac{p_{E, P}(T_j)}{p_U(T_j)}}\\
%&= \frac{\frac{p_{E, P}(T_i)}{p_U(T_i)}}{\frac{p(N_U = n_U)}{p(N_U = n_U - 1)} + \frac{\sum_j   p_{E, P}(T_j)\prod_{k\ne j} p_U(T_k)}{\prod_k p_U(T_k)}}\\
%&= \frac{\left(p_{E, P}(T_i)\prod_{k \ne i} p_U(T_k) \right)p(N_U = n_U - 1) }{p(N_U = n_U)\prod_k p_U(T_k) + \sum_j \left(\left(p_{E, P}(T_j)\prod_{k\ne j} p_U(T_k)\right) p(N_U = n_U - 1)\right)}\\
%&= \frac{\alpha_i }{\beta + \sum_j \alpha_j}\\
%\end{align*}

{\bf Death proposals}

In an event death move, we have only two choices for each arriving phase template: we can convert the template to an unassociated template, or we can delete it entirely. We propose de-association with probability proportional to

\[\frac{p(N_U = n_U + 1)p_U(T)}{p(N_U = n_U)}\]

%\[p_T(S)p_U(T)p(N_U = n_U + 1)\]

and deletion with probability proportional to

\[\frac{p(S)}{p_T(S)}\]

%\[p(S) p(N_U = n_U),\]

with normalizing constant

\begin{align*}
Z_\text{death} &= \frac{p(N_U = n_U + 1)p_U(T)}{p(N_U = n_U)} + \frac{p(S)}{p_T(S)}
%\\ &= p_T(S)p_U(T) p(N_U = n_U + 1) + p(S) p(N_U = n_U)
\end{align*}

Similarly to the birth move, the death move also has an alternate interpretation in terms of probabilities (as opposed to odds ratios).

{\bf Overall}

Suppose we propose associating template $i$. Then our acceptance ratio is
\begin{align*}
\frac{\pi(x')g'(u')j_m(x')}{\pi(x)g(u)j_m(x)} &= \frac{p_{E,P}(T_i)p(N_U = n_U-1)}{p_U(T_i)p(N_U = n_U)} \cdot \frac{1}{1} \cdot \frac{\frac{p(N_U = n_U)p_U(T_i)}{p(N_U = n_U-1)} / Z_\text{death}}{\frac{p_{E, P}(T_i)}{p_U(T_i)} / Z_\text{birth}}\\
&= \frac{Z_\text{birth}}{Z_\text{death} / p_U(T_i)}\\
&=  \frac{\frac{p(N_U = n_U)}{p(N_U = n_U - 1)} + \sum_j  \frac{p_{E, P}(T_j)}{p_U(T_j)}}{\frac{p(N_U = n_U)}{p(N_U = n_U-1)} + \frac{p(S)}{p_{T_i}(S)p_U(T_i)}}
\end{align*}

Note that $\sum_j \frac{p_{E, P}(T_j)}{p_U(T_j)}$ is the total odds that {\em some} unassociated template at this station is better explained as an $(E, P)$ phase arrival. Since we are proposing to associate an unassociated template, this is (hopefully) large. Meanwhile, $\frac{p(S)}{p_{T_i}(S)p_U(T_i)}$ is the odds of explaining the signal {\em without} the template we proposed associating, relative to with it. If this is a sizeable template, this will be quite small.

Now consider the case where we propose creating a new template $T$. Our acceptance ratio is

\begin{align*}
\frac{\pi(x')g'(u')j_m(x')}{\pi(x)g(u)j_m(x)} &= \frac{p_{E,P}(T)p_{T}(S)}{p(S)} \cdot \frac{1}{Q(T)} \cdot \frac{\frac{p(S)}{p_{T}(S)} / Z_\text{death}}{\frac{p(N_U = n_U)}{p(N_U = n_U - 1)} / Z_\text{birth}}\\
&= \frac{p_{E,P}(T)}{Q(T)}  \cdot \frac{Z_\text{birth}}{Z_\text{death}}\cdot \frac{p(N_U = n_U -1)}{p(N_U = n_U)}\\
&= \frac{p_{E,P}(T)}{Q(T)}  \cdot \frac{1 + \sum_j  \frac{p_{E, P}(T_j)p(N_U = n_U-1)}{p_U(T_j)p(N_U = n_U)}}{\frac{p(N_U = n_U+1)p_U(T)}{p(N_U = n_U)} + \frac{p(S)}{p_{T}(S)}}
\end{align*}

which isn't obvious to me how to simplify or interpret.

\end{document}