\section{The random effects estimator}


\hypertarget{field}{%
\section{Field}\label{field}}

Fields are used to store information, including numbers or matrices,
associated with the elements of a \texttt{Mesh} object.

You can create a \texttt{Field} by applying a function to each of the
vertices,

\begin{lstlisting}
var f = Field(mesh, fn (x, y, z) x+y+z)
\end{lstlisting}

or by supplying a single constant value,

\begin{lstlisting}
var f = Field(mesh, Matrix([1,0,0]))
\end{lstlisting}

Fields can then be added and subtracted using the \texttt{+} and
\texttt{-} operators.

To access elements of a \texttt{Field}, use index notation:

\begin{lstlisting}
print f[grade, element, index]
\end{lstlisting}

where * \texttt{grade} is the grade to select * \texttt{element} is the
element id * \texttt{index} is the element index

As a shorthand, it's possible to omit the grade and index; these are
then both assumed to be \texttt{0}:

\begin{lstlisting}
print f[2]
\end{lstlisting}

\hypertarget{grade}{%
\section{Grade}\label{grade}}

To create fields that include grades other than just vertices, use the
\texttt{grade} option to \texttt{Field}. This can be just a grade index,

\begin{lstlisting}
var f = Field(mesh, 0, grade=2)
\end{lstlisting}

which creates an empty field with \texttt{0} for each of the facets of
the mesh \texttt{mesh}.

You can store more than one item per element by supplying a list to the
\texttt{grade} option indicating how many items you want to store on
each grade. For example,

\begin{lstlisting}
var f = Field(mesh, 1.0, grade=[0,2,1])
\end{lstlisting}

stores two numbers on the line (grade 1) elements and one number on the
facets (grade 2) elements. Each number in the field is initialized to
the value \texttt{1.0}.

\hypertarget{shape}{%
\section{Shape}\label{shape}}

The \texttt{shape} method returns a list indicating the number of items
stored on each element of a particular grade. This has the same format
as the list you supply to the \texttt{grade} option of the
\texttt{Field} constructor. For example,

\begin{lstlisting}
[1,0,2]
\end{lstlisting}

would indicate one item stored on each vertex and two items stored on
each facet.

\hypertarget{op}{%
\section{Op}\label{op}}

The \texttt{op} method applies a function to every item stored in a
\texttt{Field}, returning the result as elements of a new \texttt{Field}
object. For example,

\begin{lstlisting}
f.op(fn (x) x.norm())
\end{lstlisting}

calls the \texttt{norm} method on each element stored in \texttt{f}.

Additional \texttt{Field} objects may be supplied as extra arguments to
\texttt{op}. These must have the same shape (the same number of items
stored on each grade). The function supplied to \texttt{op} will now be
called with the corresponding element from each field as arguments. For
example,

\begin{lstlisting}
f.op(fn (x,y) x.inner(y), g)
\end{lstlisting}

calculates an elementwise inner product between the elements of Fields
\texttt{f} and \texttt{g}.


% Created 2016-05-02 mån 18:07
% Intended LaTeX compiler: pdflatex
\documentclass{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage{khpreamble}
\newcommand{\tustin}{\frac{2}{h}\frac{z-1}{z+1}}
\author{Kjartan Halvorsen}
\date{2016-04-29}
\title{Computerized control - final exam (dummy)}
\hypersetup{
 pdfauthor={Kjartan Halvorsen},
 pdftitle={Computerized control - final exam (dummy)},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 24.5.1 (Org mode 8.3.4)},
 pdflang={English}}
\begin{document}

\maketitle

\section*{Problem 1}
\label{sec:orgheadline1}
The figure below shows the poles of a continuous-time transfer function representing the dynamics of a system.
\begin{center}
\includegraphics[width=0.3\linewidth]{imaginary-plane-ct-poles}
\end{center}
Choose a reasonable sampling period \(h\), and plot the poles of the discrete-time system obtained by zero-order-hold sampling
\begin{center}
\includegraphics[width=0.3\linewidth]{imaginary-plane-empty}
\end{center}

\section*{Problem 2}
\label{sec:orgheadline5}
Circle the correct answer to each question. Motivate your answer briefly with 1-2 sentences.

\subsection*{(a)}
\label{sec:orgheadline2}
\begin{center}
\includegraphics[width=0.6\linewidth]{../../homework/2dof-block-complete}
\end{center}
The figure above shows a block-diagram of two-degrees-of-freedom control system. Which of the following pulse transfer operators describes the \textbf{closed-loop response}, \(y(kh)\), to the measurement noise sequence, \(n(kh)\).
\begin{enumerate}
\item \(H_n(q) = \frac{1}{1 + G(q)F_b(q)}\)
\item \(H_n(q) = -\frac{G(q)F_f(q)}{1 + G(q)F_f(q)}\)
\item \(H_n(q) = -\frac{G(q)F_b(q)}{1 + G(q)F_b(q)}\)
\item \(H_n(q) = \frac{G(q)F_b(q)}{1 + G(q)F_b(q)}\)
\end{enumerate}

\subsection*{(b)}
\label{sec:orgheadline3}
The continuous-time harmonic oscillator has transfer function
\[ G(s) = \frac{\omega^2}{s^2 + \omega^2}. \]
Zero-order hold sampling of the system gives the pulse transfer function
\begin{enumerate}
\item \(H(z) = \frac{(1-\cos\omega h)(z+1)}{(z-\cos \omega h)^2 + \sin^2 \omega h}\)
\item \(H(z) = \frac{(1-\cos\omega h)(z+1)}{(z-1)^2 + \sin^2 \omega h}\)
\item \(H(z) = \frac{(1-\cos\omega h)(z+1)}{(z+\cos \omega h)^2 + \sin^2 \omega h}\)
\end{enumerate}

\subsection*{(c)}
\label{sec:orgheadline4}
The figure below shows the step-response of a closed-loop system with a discretized PID controller for some values of the controller parameters. In continuous-time the controller has the form
 \[ F(s) = K\Big(1 + \frac{1}{T_i s} + \frac{T_ds}{1 + T_ds/N}\Big). \]
\begin{center}
\includegraphics[width=0.4\linewidth]{tuned-response}
\end{center}
How should the controller be modified if we want the response of the closed-loop system to be \textbf{faster with less overshoot}?
\begin{enumerate}
\item Increase \(N\) and decrease \(T_i\).
\item Increase \(K\) and \(T_d\).
\item Decrease \(K\) and increase \(T_d\).
\item Increase \(K\) and decrease \(T_i\).
\end{enumerate}

\section*{Problem 3}
\label{sec:orgheadline8}
Consider the discrete-time double integrator
\[ H(z) = \frac{h^2(z+1)}{2(z-1)^2}. \]
\subsection*{(a)}
\label{sec:orgheadline6}
Write the system on state-space form, using the controllable canonical form
\begin{equation*}
  \begin{split}
  x(k+1) &= \underbrace{\bbm -a_1 & -a_2 & \cdots & - a_{n-1} & -a_n\\
                  1  &   0   &  \cdots & 0        &   0\\
                  0  &   1   &  \cdots & 0        &   0\\
                  \vdots &\vdots & \ddots & \vdots&   \vdots\\
                  0  &   0   &  \cdots & 1        &   0
             \ebm}_{\Phi}
	     x(k) + \underbrace{\bbm 1\\0\\0\\\vdots\0\ebm}_{\Gamma} u(k), \\
  y(k) &= \bbm b_1 & b_2 & \cdots & b_n \ebm x(k)
  \end{split}
 \end{equation*}
where
\[ H(z) = \frac{b_1z^{n-1} + b_2z^{n-2} + \cdots + b_n}{z^n + a_1z^{n-1} + \cdots + a_n}. \]

\subsection*{(b)}
\label{sec:orgheadline7}
Determine a linear state feedback
\[ u(k) = -Lx(k) + u_c(k) \]
such that the closed-loop system has poles in \(\pm i0.4\)

\section*{Solutions}
\label{sec:orgheadline17}
\subsection*{Problem 1}
\label{sec:orgheadline9}
The system has two stable, complex-conjugated poles with distance \(\omega_0 = \sqrt{2}\) from the origin, and one unstable pole in \(s=0.5\). The two stable poles are faster than the unstable pole, since they are farther from the origin. We can use the rule-of-thumb
\[ \omega_0 h \approx 0.2 - 0.6, \]
but we should be cautious and choose a sampling period in the shorter end of the range. The reason is that zero-order-hold implies a time-delay of approximately \(h/2\), and time-delays in unstable systems are problematic.

With \(\omega_0h=0.2\) we get the discrete time poles
\begin{center}
\includegraphics[width=0.3\linewidth]{imaginary-plane-dt-poles}
\end{center}

\subsection*{Problem 2}
\label{sec:orgheadline13}

\subsubsection*{(a)}
\label{sec:orgheadline10}
The correct answer is: 3. \(H_n(q) = -\frac{G(q)F_b(q)}{1 + G(q)F_b(q)}\). We can calculate the transfer function to find this answer, or argue as follows. There is a minus sign (negation) in the path from \(n\) to \(y\), so the pulse transfer operator must be negative. Only alternatives 2 and 3 are negative. Of these two, alternative 2 includes the pulse transfer operator \(F_f(q)\), but \(F_f(q)\) is outside the signal path from \(n\) to \(y\).

\subsubsection*{(b)}
\label{sec:orgheadline11}
The correct answer is: 1. \(H(z) = \frac{(1-\cos\omega h)(z+1)}{(z-\cos \omega h)^2 + \sin^2 \omega h}.\)
The harmonic oscillator has poles on the imaginary axis in the continuous-time case, and on the unit circle in the discrete-time case. Both alternative 1 and 3 have poles on the unit circle. However, the discrete-time poles are obtained from the continuous-time poles \(\pm i\omega\) according to the mapping
\[ p = \mexp{\pm i\omega h} = \cos\omega h \pm i\sin\omega h, \]
where the last equality is the famous Euler's formula. Alternative 1 has indeed poles in
\[ \cos\omega h \pm i \sin\omega h, \]
whereas Alternative 3 has poles in
\[ -\cos\omega h \pm i \sin\omega h. \]

\subsubsection*{(c)}
\label{sec:orgheadline12}
The correct answer is:  2. Increase \(K\) and \(T_d\). To make the response faster, we must increase the gain of the controller \(K\). Only alternative 2 and 4 suggest this. To make the overshoot smaller, we must increase the damping. This is done by increasing \(T_d\).

\subsection*{Problem 3}
\label{sec:orgheadline16}

\subsubsection*{(a)}
\label{sec:orgheadline14}
The harmonic oscillator can be written
\[ H(z) = \frac{h^2(z+1)}{2(z-1)^2} = \frac{h^2/2(z+1)}{z^2-2z + 1}, \]
which  on controllable canonical form is
   \begin{equation*}
 \begin{split}
  x(k+1) &= \bbm 2 & -1\\1 & 0 \ebm x(k) + \bbm 1\\0\ebm u(k)\\
  y(k) &= \bbm h^2/2 & h^2/2 \ebm .
 \end{split}
\end{equation*}

\subsubsection*{(b)}
\label{sec:orgheadline15}
Linear feedback control of a system on controllable canonical form is particularly easy, since the resulting system is also on controllable canonical form. The closed loop system with
\[ u(k) = -Lx(k) + u_c(k) \]
has pulse transfer function
\[ H_c(z) = \frac{h^2/2(z+1)}{z^2 (-2+l_1)z + 1+l_2} \]
and the desired denominator is
\[ (z-i0.4)(z+i0.4) = z^2 + 0.4^2, \]
Equating the coefficients gives the feedback gains
\begin{align*}
-2+l_1 &= 0 \quad \Rightarrow \quad l_1 = 2\\
1+l_2 &= 0.16 \quad \Rightarrow \quad l_2 = -0.84
\end{align*}

A step-response with \(h=1\) and a step in \(u_c\) occurring at \(t=1\) is shown below.
\begin{center}
\includegraphics[width=0.6\linewidth]{problem3_dummy_step_response-crop}
\end{center}
\end{document}


\section{Introduction}
\label{sec:introduction}

\begin{figure}[t]
	\centering
	\vspace*{-0.2cm}
	\hspace*{-0.25cm}
	\includegraphics[width=0.5\textwidth]{plots_contributions_arxiv}
	\vspace*{-18px}
	\caption{\textbf{Robust Generalization and Flatness:} Robust loss (\RCE, lower is more robust, y-axis), \ie, cross-entropy loss on PGD adversarial examples \cite{MadryICLR2018}, against our \emph{average-case flatness} measure of \RCE in weight space (lower is ``flatter'', x-axis).
	Popular AT variants improving adversarial robustness on \CifarT, \eg, TRADES \cite{ZhangICML2019}, AT-AWP \cite{WuNIPS2020}, MART \cite{WangICLR2020} or AT with self-supervision \cite{HendrycksNIPS2019}/unlabeled examples \cite{CarmonNIPS2019}, also correspond to flatter minima. Vice-versa, regularization explicitly improving flatness, \eg, Entropy-SGD \cite{ChaudhariICLR2017}, weight decay or weight clipping \cite{StutzMLSYS2021}, also improve robustness.
	Across all models, there is a \textbf{clear relationship between good robust generalization and flatness in \RCE.}
	{\LARGE$\bullet$},\raisebox{0.5mm}{$\mathbin{\blacklozenge}$} Our models, \underline{w/o} early stopping.
	{\large$\blacktriangle$} RobustBench \cite{CroceARXIV2020b} models \emph{w/} early stopping.}
	\label{fig:introduction}
	\vspace*{-6px}
\end{figure}

In order to obtain robustness against adversarial examples \cite{SzegedyICLR2014}, \emph{adversarial training (AT)} \cite{MadryICLR2018} augments training with adversarial examples that are generated on-the-fly. While many different variants have been proposed, AT is known to require more training data \cite{KhouryARXIV2018,SchmidtNIPS2018}, generally leading to generalization problems \cite{FarniaICLR2019}. In fact, \emph{robust overfitting} \cite{RiceICML2020} has been identified as the main problem in AT:  adversarial robustness on test examples eventually starts to decrease, while robustness on training examples continues to increase (\cf \figref{fig:main-overfitting}). This is typically observed as increasing \emph{robust loss (\RCE)} or \emph{robust test error (\RTE)}, \ie, (cross-entropy) loss and test error on adversarial examples. As a result, the \emph{robust generalization gap}, \ie, the difference between test and training robustness, tends to be very large. In \cite{RiceICML2020}, early stopping is used as a simple and effective strategy to avoid robust overfitting. However, despite recent work tackling robust overfitting \cite{SinglaARXIV2021,WuNIPS2020,HwangARXIV2020}, it remains an open and poorly understood problem.

In \emph{``clean''} generalization (\ie, on natural examples), overfitting is well-studied and commonly tied to flatness of the loss landscape in weight space, both visually \cite{LiNIPS2018} and empirically \cite{NeyshaburNIPS2017,KeskarICLR2017,JiangICLR2020}.
In general, the optimal weights on test examples do not coincide with the minimum found on training examples. Flatness ensures that the loss does \emph{not} increase significantly in a neighborhood around the found minimum. Therefore, flatness leads to good generalization because the loss on test examples does not increase significantly (\ie, small generalization gap, \cf \figref{fig:main-illustration}, right).
\cite{LiNIPS2018} showed that \emph{visually} flatter minima correspond to better generalization. \cite{NeyshaburNIPS2017} and \cite{KeskarICLR2017} formalize this idea by measuring the change in loss within a local neighborhood around the minimum considering random \cite{NeyshaburNIPS2017} or ``adversarial'' weight perturbations \cite{KeskarICLR2017}.
These measures are shown to be effective in predicting generalization in a recent large-scale empirical study \cite{JiangICLR2020} and explicitly encouraging flatness during training has been shown to be successful in practice \cite{ZhengARXIV2020c,CicekICCVWOR2019,TinICLR2020,ChaudhariICLR2017,IzmailovUAI2018}.

Recently, \cite{WuNIPS2020} applied the idea of flat minima to AT: through \emph{adversarial weight perturbations}, AT is regularized to find flatter minima of the \emph{robust} loss landscape. This reduces the impact of robust overfitting and improves robust generalization, but does not \emph{avoid} robust overfitting. As result, early stopping is still necessary. Furthermore, flatness is only assessed \emph{visually} and it remains unclear whether flatness does actually improve in these adversarial weight directions.
Similarly, \cite{GowalARXIV2020} shows that weight averaging \cite{IzmailovUAI2018} can improve robust generalization, indicating that flatness might be beneficial in general. This raises the question whether other ``tricks'' \cite{PangARXIV2020b,GowalARXIV2020}, \eg, different activation functions \cite{SinglaARXIV2021} or label smoothing \cite{SzegedyCVPR2016}, or approaches such as AT with self-supervision \cite{HendrycksNIPS2019}/unlabeled examples \cite{CarmonNIPS2019} are successful \emph{because of} finding flatter minima.

\begin{figure}[t]
	\centering
	\vspace*{-0.2cm}
	\includegraphics[width=0.225\textwidth]{plots_short_introduction_overfitting1}
	\includegraphics[width=0.225\textwidth]{plots_short_introduction_overfitting2}
	\vspace*{-8px}
	\caption{\textbf{Robust Overfitting:} Robust (cross-entropy) loss (\RCE) and robust error (\RTE) over epochs (normalized by $150$ epochs) for AT, \red{using a ResNet-18 on \CifarT (\cf \secref{sec:experiments})}, to illustrate \emph{robust} overfitting. \textbf{Left:} Training \RCE ({\color{plot0}light blue}) reduces continuously throughout training, while test \RCE ({\color{plot1}dark blue}) eventually increases again.
	We also highlight that robust overfitting is \emph{not} limited to incorrectly classified examples ({\color{plot4}green}), but also affects correctly classified ones ({\color{plot2}rose}). \textbf{Right:} Similar behavior, but less pronounced, can be observed considering \RTE. We also show \RTE obtained through early stopping ({\color{plot5}red}).}
	\label{fig:main-overfitting}
	\vspace*{-6px}
\end{figure}

\textbf{Contributions:} In this paper, we study \textbf{whether flatness of the robust loss (\RCE) in weight space improves robust generalization}. To this end,
we propose both average- and worst-case flatness measures for the \emph{robust} case, \red{thereby addressing challenges such as scale-invariance \cite{DinhICML2017}, estimation of \RCE on top or jointly with weight perturbations, and the discrepancy between \RCE and \RTE}. We show that \textbf{robust generalization generally improves alongside flatness} and vice-versa: \figref{fig:introduction} plots \RCE (lower is more robust, y-axis) against our average-case flatness in \RCE (lower is flatter, x-axis), showing a clear relationship.
\red{In contrast to \cite{WuNIPS2020}, not providing empirical flatness measures, our results show that this relationship is stronger for average-case flatness.}
This trend covers a wide range of AT variants on \CifarT, \eg,  AT-AWP \cite{WuNIPS2020}, TRADES \cite{ZhangICML2019}, MART \cite{WangICLR2020}, AT with self-supervision \cite{HendrycksNIPS2019} or additional unlabeled examples \cite{CarmonNIPS2019,UesatoNIPS2019}, as well as various regularization schemes, including AutoAugment \cite{CubukARXIV2018}, label smoothing \cite{SzegedyCVPR2016} and noise or weight clipping \cite{StutzMLSYS2021}. Furthermore, we consider hyper-parameters, \eg, learning rate schedule, weight decay, batch size, or different activation functions \cite{ElfwingNN2018,MisraBMVC2020,HendrycksARXIV2016}, and methods explicitly improving flatness, \eg, Entropy-SGD \cite{ChaudhariICLR2017} or weight averaging \cite{IzmailovUAI2018}.