\section{Polar Coordinates}\label{sec:PolarCoordinates}
Coordinate systems are tools that let us use algebraic methods to
understand geometry. While the \dfont{rectangular} (also called
\dfont{Cartesian}) coordinates that we
have been using are the most common, some problems are easier to
analyze in alternate coordinate systems. \index{Cartesian coordinates} \index{rectangular coordinates}

A coordinate system is a scheme that allows us to identify any point
in the plane or in three-dimensional space by a set of numbers. In
rectangular coordinates these numbers are interpreted, roughly
speaking, as the lengths of the sides of a rectangle.
In \dfont{polar coordinates}\index{polar coordinates} a
point in the plane is identified by a pair of numbers $(r,\theta)$.
The number $\theta$ measures the angle between the positive
$x$-axis and a ray that goes through the point,
as shown in Figure~\ref{fig:polar coordinates}; the number
$r$ measures the distance from the origin to the
point (the radius). Figure~\ref{fig:polar coordinates} shows the point with
rectangular coordinates $\ds (1,\sqrt3)$ and polar coordinates
$(2,\pi/3)$, 2 units from the origin and $\pi/3$ radians from the
positive $x$-axis.

\figure[H]
\centerline{\vbox{\beginpicture
\normalgraphs
%%\ninepoint
\setcoordinatesystem units <15truemm,15truemm>
\setplotarea x from 0 to 1.25, y from 0 to 1.9
\axis left ticks length <2pt> withvalues
  {1.728} / at 1.728 / /
\axis bottom ticks length <2pt> withvalues {$1$} / at 1 / /
\arrow <4pt> [0.35, 1] from 0 0 to 1.2 2.078
\put {$(2,\pi/3)$} [l] <3pt,0pt> at 1 1.73
\put {$\bullet$} at 1 1.728
\circulararc 60 degrees from 0.2 0 center at 0 0
\endpicture}}
\caption{Polar coordinates of the point $\ds (1,\sqrt3)$. \label{fig:polar coordinates}}
\endfigure

Just as we describe curves in the plane using equations involving $x$
and $y$, so can we describe curves using equations involving $r$ and
$\theta$. Most common are equations of the form $r=f(\theta)$.

\begin{example}{Circle in Polar Coordinates}{circleinpolar}
 Graph the curve given by $r=2$.
\end{example}

\begin{solution}
All points with $r=2$ are at
distance 2 from the origin, so $r=2$ describes the circle of radius 2
with center at the origin.
\end{solution}

\begin{example}{Cardioid}{cardiodexample}
 Graph the curve given by $r=1+\cos\theta$.
\end{example}

\begin{solution}
We first consider
$y=1+\cos x$, as in figure~\ref{fig:cardioid}. As $\theta$ goes
through the values in $[0,2\pi]$, the value of $r$ tracks the value of
$y$, forming the ``cardioid''
shape of  figure~\ref{fig:cardioid}.
For example, when $\theta=\pi/2$, $r=1+\cos(\pi/2)=1$, so we graph the
point at distance 1 from the origin along the positive $y$-axis, which
is at an angle of $\pi/2$ from the positive $x$-axis. When
$\theta=7\pi/4$, $\ds r=1+\cos(7\pi/4)=1+\sqrt2/2\approx 1.71$, and the
corresponding point appears in the fourth quadrant. If we look at the curve, we should explain the indentation which occurs at $\theta=\pi$. It should be obvious that $\cos x<0$ on $(\pi/2,3\pi/2)$, thus $r=1+\cos\theta<1$. Specifically, when $\theta=\pi$, $\cos\pi=-1$, so $r=0$, and we get the resulting indentation. This illustrates
one of the potential benefits of using polar coordinates: the equation
for this curve in rectangular coordinates would be \emph{quite} complicated.
\end{solution}

\figure[H]
\centerline{\vbox{\beginpicture
\normalgraphs
%%\ninepoint
\setcoordinatesystem units <12truemm,12truemm>
\setplotarea x from 0 to 6.2832, y from 0 to 2.2
\axis left ticks length <2pt> numbered from 0 to 2 by 1 /
\axis bottom ticks length <2pt> withvalues
  {$\pi/2$} {$\pi$} {$3\pi/2$} {$2\pi$} / at 1.57 3.14 4.7 6.28 / /
\multiput {$\bullet$} at 1.57 1 5.5 1.71 /
\setquadratic
\plot
0.000 2.000 0.105 1.995 0.209 1.978 0.314 1.951 0.419 1.914
0.524 1.866 0.628 1.809 0.733 1.743 0.838 1.669 0.942 1.588
1.047 1.500 1.152 1.407 1.257 1.309 1.361 1.208 1.466 1.105
1.571 1.000 1.676 0.895 1.780 0.792 1.885 0.691 1.990 0.593
2.094 0.500 2.199 0.412 2.304 0.331 2.409 0.257 2.513 0.191
2.618 0.134 2.723 0.086 2.827 0.049 2.932 0.022 3.037 0.005
3.142 0.000 3.246 0.005 3.351 0.022 3.456 0.049 3.560 0.086
3.665 0.134 3.770 0.191 3.875 0.257 3.979 0.331 4.084 0.412
4.189 0.500 4.294 0.593 4.398 0.691 4.503 0.792 4.608 0.895
4.712 1.000 4.817 1.105 4.922 1.208 5.027 1.309 5.131 1.407
5.236 1.500 5.341 1.588 5.445 1.669 5.550 1.743 5.655 1.809
5.760 1.866 5.864 1.914 5.969 1.951 6.074 1.978 6.178 1.995
6.283 2.000 /
\setcoordinatesystem units <12truemm,12truemm> point at -8 -1
\setplotarea x from -1 to 2.5, y from -1.5 to 1.5
\axis left shiftedto x=0 /
\axis bottom shiftedto y=0 /
\put {$\bullet$} at 0 1
\put {$\bullet$} at 1.207 -1.207
\setquadratic
\plot 2.000 0.000 1.984 0.208 1.935 0.411 1.856 0.603 1.748 0.778
1.616 0.933 1.464 1.063 1.295 1.166 1.117 1.240 0.933 1.285
0.750 1.299 0.572 1.285 0.405 1.245 0.251 1.182 0.115 1.098
-0.000 1.000 -0.094 0.891 -0.165 0.775 -0.214 0.657 -0.241 0.542
-0.250 0.433 -0.242 0.333 -0.221 0.246 -0.191 0.172 -0.155 0.112
-0.116 0.067 -0.079 0.035 -0.047 0.015 -0.021 0.005 -0.005 0.001
0.000 0.000 -0.005 -0.001 -0.021 -0.005 -0.047 -0.015 -0.079 -0.035
-0.116 -0.067 -0.155 -0.112 -0.191 -0.172 -0.221 -0.246 -0.242 -0.333
-0.250 -0.433 -0.241 -0.542 -0.214 -0.657 -0.165 -0.775 -0.094 -0.891
0.000 -1.000 0.115 -1.098 0.251 -1.182 0.405 -1.245 0.572 -1.285
0.750 -1.299 0.933 -1.285 1.117 -1.240 1.295 -1.166 1.464 -1.063
1.616 -0.933 1.748 -0.778 1.856 -0.603 1.935 -0.411 1.984 -0.208
2.000 0.000 /
\setdashes
\arrow <4pt> [0.35, 1] from 0 0 to 1.207 -1.207
\endpicture}}
\caption{A cardioid: $y=1+\cos x$ on the left, $r=1+\cos\theta$ on the right.\label{fig:cardioid}}
\endfigure

Each point in the plane is associated with exactly one pair of numbers
in the rectangular coordinate system; each point is associated with an
infinite number of pairs in polar coordinates. In the cardioid example,
we considered only the range $0\le \theta\le2\pi$, and
already there was a duplicate: $(2,0)$ and $(2,2\pi)$ are the same
point. Indeed, every value of $\theta$ outside the interval $[0,2\pi)$
duplicates a point on the curve $r=1+\cos\theta$ when
$0\le\theta<2\pi$. We can even make sense of polar coordinates like
$(-2,\pi/4)$: Go to the direction $\pi/4$ and then move a distance 2
in the opposite direction; see figure~\ref{fig:negative r
coordinate}. As usual, a negative angle $\theta$ means an angle
measured clockwise from the positive $x$-axis. The point in
figure~\ref{fig:negative r coordinate} also has coordinates
$(2,5\pi/4)$ and $(2,-3\pi/4)$.

\figure[H]
\centerline{\vbox{\beginpicture
\normalgraphs
%\eightpoint
\setcoordinatesystem units <10truemm,10truemm>
\setplotarea x from -2 to 2, y from -2 to 2
\axis left shiftedto x=0 ticks length <2pt>
  withvalues $-2$ $-1$ {} $1$ $2$ / quantity 5 /
\axis bottom shiftedto y=0 ticks length <2pt>
  withvalues $-2$ $-1$ {} $1$ $2$ / quantity 5 /
\put {$\pi/4$} [bl] <3pt,-4pt> at 0.4 0.3
\put {$\bullet$} at -1.4142 -1.4142
\circulararc 45 degrees from 0.4 0 center at 0 0
\setdashes\arrow <4pt> [0.35, 1] from 1 1 to -1.4142 -1.4142
\endpicture}}
\caption{The point $(-2,\pi/4)=(2,5\pi/4)=(2,-3\pi/4)$ in polar coordinates.\label{fig:negative r coordinate}}
\endfigure

The relationship between rectangular and polar coordinates is quite easy to
understand. The point with polar coordinates $(r,\theta)$ has
rectangular coordinates $x=r\cos\theta$ and $y=r\sin\theta$; this
follows immediately from the definition of the sine and cosine
functions. Using figure~\ref{fig:negative r coordinate} as an example, the point shown has rectangular coordinates
$\ds x=(-2)\cos(\pi/4)=-\sqrt2\approx 1.4142$ and
$\ds y=(-2)\sin(\pi/4)=-\sqrt2$.  This makes it very easy to convert
equations from rectangular to polar coordinates. \index{polar coordinates!convert to rectangular}

\begin{example}{Straight Line in Polar Coordinates}{StraightLineinPolar}
 Find the equation of the line $y=3x+2$ in polar
coordinates.
\end{example}

\begin{solution}
We merely substitute: $r\sin\theta=3r\cos\theta+2$, or
$\ds r= {2\over \sin\theta-3\cos\theta}$.
\end{solution}

\begin{example}{Equation of a Circle}{equationofcircleinpolar}
 Find the equation of the circle $\ds (x-1/2)^2+y^2=1/4$ in polar
coordinates.
\end{example}

\begin{solution}
Again substituting:
$\ds (r\cos\theta-1/2)^2+r^2\sin^2\theta=1/4$. A bit of algebra turns this
into $r=\cos(t)$. You should try plotting a few $(r,\theta)$ values to
convince yourself that this makes sense.
\end{solution}

\begin{example}{Spiral of Archimedes}{spiralofArchimedes}
 Graph the polar equation $r=\theta$.
\end{example}

\begin{solution}
Here the distance from
the origin exactly matches the angle, so a bit of thought makes it
clear that when $\theta\ge0$ we get the spiral of
Archimedes in figure~\ref{fig:spiral of Archimedes}. When $\theta<0$, $r$ is also
negative, and so the full graph is the right hand picture in the figure.
\end{solution}

\figure[H]
\centerline{\vbox{\beginpicture
\normalgraphs
%\eightpoint
\setcoordinatesystem units <5truemm,5truemm>
\setplotarea x from -3.4 to 6.5, y from -4.8 to 2
\axis left shiftedto x=0 /
\axis bottom shiftedto y=0 /
\multiput {$\bullet$} at 0.54 0.84 0 1.57 -3.14 0 6.28 0 /
\put {$(2\pi,2\pi)$} [b] <0pt,3pt> at 6.28 0
\put {$(\pi,\pi)$} [br] <-3pt,0pt> at -3.14 0
\put {$(\pi/2,\pi/2)$} [br] <-2pt,4pt> at 0 1.57
\put {$(1,1)$} [bl] <3pt,0pt> at 0.54 0.84
\plot 0.000 0.000 0.104 0.011 0.205 0.044 0.299 0.097 0.383 0.170
0.453 0.262 0.508 0.369 0.545 0.490 0.561 0.623 0.554 0.762
0.524 0.907 0.469 1.052 0.388 1.195 0.283 1.332 0.153 1.458
0.000 1.571 -0.175 1.666 -0.370 1.741 -0.582 1.793 -0.809 1.818
-1.047 1.814 -1.293 1.779 -1.542 1.712 -1.790 1.612 -2.033 1.477
-2.267 1.309 -2.487 1.107 -2.689 0.874 -2.868 0.610 -3.020 0.317
-3.142 0.000 -3.229 -0.339 -3.278 -0.697 -3.287 -1.068 -3.253 -1.448
-3.174 -1.833 -3.050 -2.216 -2.879 -2.593 -2.663 -2.957 -2.401 -3.304
-2.094 -3.628 -1.746 -3.922 -1.359 -4.183 -0.936 -4.405 -0.482 -4.582
0.000 -4.712 0.504 -4.791 1.023 -4.814 1.553 -4.781 2.087 -4.688
2.618 -4.534 3.139 -4.321 3.644 -4.047 4.125 -3.714 4.575 -3.324
4.988 -2.880 5.357 -2.385 5.677 -1.845 5.941 -1.263 6.145 -0.646
6.283 0.000 /
\setcoordinatesystem units <5truemm,5truemm> point at -18 0
\setplotarea x from -6.5 to 6.5, y from -4.8 to 2
\axis left shiftedto x=0 /
\axis bottom shiftedto y=0 /
\multiput {$\bullet$} at -0.54 0.84 0 1.57 3.14 0 -6.28 0 /
\put {$(-2\pi,-2\pi)$} [b] <0pt,3pt> at -6.28 0
\put {$(-\pi,-\pi)$} [bl] <3pt,2pt> at 3.14 0
\put {$(-\pi/2,-\pi/2)$} [br] <-2pt,4pt> at 0 1.57
\put {$(-1,-1)$} [t] <0pt,0pt> at -1.5 -1
\plot -6.283 0.000 -6.145 -0.646 -5.941 -1.263 -5.677 -1.845 -5.357 -2.385
-4.988 -2.880 -4.575 -3.324 -4.125 -3.714 -3.644 -4.047 -3.139 -4.321
-2.618 -4.534 -2.087 -4.688 -1.553 -4.781 -1.023 -4.814 -0.504 -4.791
0.000 -4.712 0.482 -4.582 0.936 -4.405 1.359 -4.183 1.746 -3.922
2.094 -3.628 2.401 -3.304 2.663 -2.957 2.879 -2.593 3.050 -2.216
3.174 -1.833 3.253 -1.448 3.287 -1.068 3.278 -0.697 3.229 -0.339
3.142 0.000 3.020 0.317 2.868 0.610 2.689 0.874 2.487 1.107
2.267 1.309 2.033 1.477 1.790 1.612 1.542 1.712 1.293 1.779
1.047 1.814 0.809 1.818 0.582 1.793 0.370 1.741 0.175 1.666
0.000 1.571 -0.153 1.458 -0.283 1.332 -0.388 1.195 -0.469 1.052
-0.524 0.907 -0.554 0.762 -0.561 0.623 -0.545 0.490 -0.508 0.369
-0.453 0.262 -0.383 0.170 -0.299 0.097 -0.205 0.044 -0.104 0.011
0.000 0.000 0.104 0.011 0.205 0.044 0.299 0.097 0.383 0.170
0.453 0.262 0.508 0.369 0.545 0.490 0.561 0.623 0.554 0.762
0.524 0.907 0.469 1.052 0.388 1.195 0.283 1.332 0.153 1.458
0.000 1.571 -0.175 1.666 -0.370 1.741 -0.582 1.793 -0.809 1.818
-1.047 1.814 -1.293 1.779 -1.542 1.712 -1.790 1.612 -2.033 1.477
-2.267 1.309 -2.487 1.107 -2.689 0.874 -2.868 0.610 -3.020 0.317
-3.142 0.000 -3.229 -0.339 -3.278 -0.697 -3.287 -1.068 -3.253 -1.448
-3.174 -1.833 -3.050 -2.216 -2.879 -2.593 -2.663 -2.957 -2.401 -3.304
-2.094 -3.628 -1.746 -3.922 -1.359 -4.183 -0.936 -4.405 -0.482 -4.582
0.000 -4.712 0.504 -4.791 1.023 -4.814 1.553 -4.781 2.087 -4.688
2.618 -4.534 3.139 -4.321 3.644 -4.047 4.125 -3.714 4.575 -3.324
4.988 -2.880 5.357 -2.385 5.677 -1.845 5.941 -1.263 6.145 -0.646
6.283 0.000 /
\setdashes <2pt>
\setlinear
\plot -2 -0.9 -0.54 0.84 /
%\altarrow <4pt,5pt> [0.35, 1] from -2 -0.9 to -0.54 0.84
\endpicture}}
\caption{The spiral of Archimedes and the full graph of $r=\theta$.\label{fig:spiral of Archimedes}}
\endfigure

Converting polar equations to rectangular equations can be somewhat
trickier, and graphing polar equations directly is also not always easy.

\begin{example}{Graphing Polar Equations}{graphingpolarequations}
 Graph $r=2\sin\theta$.
\end{example}

\begin{solution}
Given that the sine is periodic, we know
that we will get the entire curve for values of $\theta$ in
$[0,2\pi)$. As $\theta$ runs from 0 to $\pi/2$, $r$ increases from 0
to 2. Then as $\theta$ continues to $\pi$, $r$ decreases again to
0. When $\theta$ runs from $\pi$ to $2\pi$, $r$ is negative, and it
is not hard to see that the first part of the curve is simply traced
out again, so in fact we get the whole curve for values of $\theta$
in $[0,\pi)$. Thus, the curve looks something like
figure~\ref{fig:circle from polar equation}. Now, this suggests
that the curve could possibly be a circle, and if it is, it would
have to be the circle $\ds x^2+(y-1)^2=1$. Having made this guess, we
can easily check it. First we substitute for $x$ and $y$ to get
$\ds (r\cos\theta)^2+(r\sin\theta-1)^2=1$; expanding and simplifying
does indeed turn this into $r=2\sin\theta$.
\end{solution}

\figure[H]
\centerline{\vbox{\beginpicture
\normalgraphs
%%\ninepoint
\setcoordinatesystem units <15truemm,15truemm>
\setplotarea x from -1 to 1, y from 0 to 2.2
\axis left shiftedto x=0 ticks length <2pt> numbered from 1 to 1 by 1 /
\axis bottom ticks length <2pt> numbered from -1 to 1 by 1 /
\circulararc 360 degrees from 1 1 center at 0 1
\endpicture}}
\caption{Graph of $r=2\sin\theta$.\label{fig:circle from polar equation}}
\endfigure

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:PolarCoordinates}}

\begin{enumialphparenastyle}

%%%%%%%%%%
\begin{ex}
Plot these polar coordinate points on one graph:
$(2,\pi/3)$, $(-3,\pi/2)$, $(-2,-\pi/4)$, $(1/2,\pi)$, $(1,4\pi/3)$,
$(0,3\pi/2)$.
\end{ex}

%%%%%%%%%%
\begin{ex}
Find an equation in polar coordinates that has the same
graph as the given equation in rectangular coordinates.
\begin{multicols}{3}
\begin{enumerate}
	\item	$\ds y=3x$
	\item	$\ds y=-4$
	\item	$\ds xy^2=1$
	\item	$\ds x^2+y^2=5$
	\item	$\ds y=x^3$
	\item	$\ds y=\sin x$
	\item	$\ds y=5x+2$
	\item	$\ds x=2$
	\item	$\ds y=x^2+1$
\end{enumerate}
\end{multicols}
\begin{sol}
\begin{multicols}{3}
\begin{enumerate}
	\item	$\ds \theta=\arctan(3)$
	\item	$\ds r=-4\csc\theta$
	\item	$\ds r=\sec\theta\csc^2\theta$
	\item	$\ds r=\sqrt{5}$
	\item	$\ds r^2=\sin\theta\sec^3\theta$
	\item	$\ds r\sin\theta=\sin(r\cos\theta)$
	\item	$\ds r=2/(\sin\theta-5\cos\theta)$
	\item	$\ds r=2\sec\theta$
	\item	$\ds 0=r^2\cos^2\theta-r\sin\theta+1$
\end{enumerate}
\end{multicols}
\end{sol}
\end{ex}

%%%%%%%%%%
\begin{ex}
Sketch the following curves:
\begin{multicols}{2}
\begin{enumerate}
	\item	$\ds r=\cos\theta$
	\item	$\ds r=1+\theta^1/\pi^2$
	\item	$\ds r=\sin(\theta+\pi/4)$
	\item	$\ds r=-\sec\theta$
	\item	$\ds r=\theta/2$, $\theta\ge0$
	\item	$\ds r=\cot\theta\csc\theta$
	\item	$\ds r={1\over\sin\theta+\cos\theta}$
	\item	$\ds r^2=-2\sec\theta\csc\theta$
\end{enumerate}
\end{multicols}
\end{ex}

%%%%%%%%%%
\begin{ex}
Find an equation in rectangular coordinates that has the same
graph as the given equation in polar coordinates.
\begin{multicols}{2}
\begin{enumerate}
	\item	$\ds r=\sin(3\theta)$
	\item	$\ds r=\sin^2\theta$
	\item	$\ds r=\sec\theta\csc\theta$
	\item	$\ds r=\tan\theta$
\end{enumerate}
\end{multicols}
\begin{sol}
\begin{multicols}{2}
\begin{enumerate}
	\item	$\ds (x^2+y^2)^2=4x^2y-(x^2+y^2)y$
	\item	$\ds (x^2+y^2)^{3/2}=y^2$
	\item	$\ds x^2+y^2=x^2y^2$
	\item	$\ds x^4+x^2y^2=y^2$
\end{enumerate}
\end{multicols}
\end{sol}
\end{ex}

\end{enumialphparenastyle}


% $Id$
%
% Author: David Fournier
% Copyright (c) 2008 Regents of the University of California
%

\section{The Kalman filter}

The Kalman filter is a device for estimating parameters in
a class of ``time-series''-like models that are put into state-space
form. We have used the
notation from \cite{harvey1990}, Chapter~3.
\X{state-space form}
\X{Kalman filter}
The general state-space form is an  multivariate time series
$$y_t=Z_t\alpha_t+d_t+\epsilon_t$$
where $Z_t$ is an $N\times m$ matrix, $d_t$ is an $N$-dimensional
vector, $y_t$ is an $N$-dimensional vector, and $\epsilon_t$ is
a set of serially uncorrelated $N$-dimensional random vectors
with mean $0$ and correlation $H_t$.
The elements of $\alpha_t$ are not observable, but are
assumed to be generated by a first-order Markov process
$$\alpha_t=T_t\alpha_{t-1}+c_t+R_t\eta_t$$
where $T_t$ is an $m\times m$ matrix, $c_t$ is an $m\times 1$
vector, $R_t$ is an $m\times g$ matrix and $\eta_t$
is a $g\times 1$ vector of serially uncorrelated
random vectors with mean $0$ and covariance matrix $H_t$.
The specification of the state-space system is completed by
two further assumptions:
\begin{enumerate}
  \item The initial state vector $\alpha_0$ has a mean of $a_0$ and a
variance of~$P_0$.
  \item The random vectors $\epsilon_t$ and $\eta_t$ are uncorrelated
with each other, and uncorrelated with the initial state.
\end{enumerate}

In applications of the model, many of the
parameters $Z_t$, $d_t$, $H_t$, $T_t$, $c_t$, $R_t$, and $Q_t$
may be independent of $t$, in which case we will write them
without the subscript. Also, $R$ may be the identity matrix,
in which case we will omit it.

As a simple example of such a model, consider the (2-dimensional) random walk observed with error: %fixed.
 \begin{align}
  \nonumber \alpha_t =&\alpha_{t-1}+\eta_t\\
   y_t =&\alpha_t+\epsilon_t
 \end{align}
For this model, the following parameters are fixed
$$ T=\begin{pmatrix}1 & 0\\
                    0 & 1 \end{pmatrix}
  \quad
   Z=\begin{pmatrix}1 & 0\cr
                    0 & 1 \end{pmatrix}
  \quad
   R=\begin{pmatrix}1 & 0\cr
                    0 & 1\end{pmatrix}
  \quad
  d=(0,0)
  \quad
  c=(0,0)
$$
while the covariance matrices $Q$ and $H$ are estimated. The true values of $Q$ and $H$ used in the simulation were %fixed
$$
   Q=\begin{pmatrix}1 & 0.8\cr
                   0.8 & 1 \end{pmatrix}
  \quad
   H=\begin{pmatrix}3 & -2.5\cr
                   -2.5 & 3 \end{pmatrix}
$$
and the initial value of $a$ is $(0,0)$.%fixed

\section{Equations for the Kalman filter}

For a moment, go back to the general state-space model.
Given $a_0$ and $P_0$, we recursively calculate the a number of quantities
via the relationships
 \begin{align}
   \nonumber  \aone &= T_t\, a_{t-1}+c_t\\[3pt]
    \nonumber   \Pone &=T_t\, P_{t-1}\, T^\prime_t +R_t\, Q_t\, R^\prime_t\\[3pt]
   \nonumber    v_t &=y_t-Z_t\, \aone-d_t\\[3pt]
    \nonumber   F_t &=Z_t\, \Pone\, Z_t^\prime+H_t\\[3pt]
    \nonumber   a_t &=\aone-\Pone\, Z_t^\prime\, F_t^{-1}\, v_t\\[3pt]
      P_t &= \Pone-\Pone\, Z_t^\prime\, F_t^{-1}\, Z_t\, \Pone
 \end{align}
The log-likelihood function for the models parameters is
given by:
$$\log L = -\tfrac{NT}{ 2}\log 2\pi -0.5 \sum_{t=1}^T \log|F_t|
        -0.5 \sum_{t=1}^T v_t F_t^{-1} v_t
$$

The \textsc{tpl} file for the random walk Kalman filter code
follows:
\begin{lstlisting}
DATA_SECTION
  init_int nt
  init_int N
  init_int m
  int m1
  init_matrix Y(1,nt,1,N)
  matrix P0(1,m,1,m)
 !! P0.initialize();
 !! m1=m*(m+1)/2;
PARAMETER_SECTION
  init_bounded_vector Qcoff(1,m1,-10.,10.1)
  init_bounded_vector Hcoff(1,m1,-10.,10.1)
  init_vector a0(1,m)
  matrix T(1,m,1,m)
  matrix TT(1,m,1,m)
  vector d(1,N)
  vector c(1,m)
  matrix chQ(1,m,1,m)
  sdreport_matrix Q(1,m,1,m)
  matrix chH(1,N,1,N)
  sdreport_matrix H(1,N,1,N)
  matrix Z(1,N,1,m)
  matrix TZ(1,m,1,N)
  objective_function_value f
 LOCAL_CALCS
   d.initialize();
   c.initialize();
   Z.initialize();
   Z(1,1)=1; Z(2,2)=1;
   T.initialize();
   T(1,1)=1; T(2,2)=1;
   TZ=trans(Z);
   TT=trans(T);
PROCEDURE_SECTION
  setup_Q();
  setup_H();
  f+=kalman_filter();
  cout << " f = " << f << endl;

FUNCTION setup_Q
  chQ.initialize();
  int ii=1;
  for (int i=1;i<=m;i++)
    for (int j=1;j<=i;j++)
      chQ(i,j)=Qcoff(ii++);
  Q=chQ*trans(chQ);
FUNCTION setup_H
  chH.initialize();
  int ii=1;
  for (int i=1;i<=N;i++)
    for (int j=1;j<=i;j++)
      chH(i,j)=Hcoff(ii++);
  H=chH*trans(chH);

FUNCTION dvariable kalman_filter(void)
  dvar3_array P(0,nt,1,m,1,m);
  dvar3_array P1(1,nt,1,m,1,m);
  dvar3_array F(1,nt,1,N,1,N);
  dvar3_array Finv(1,nt,1,N,1,N);
  dvar_matrix Ptemp(1,m,1,m);
  dvar_matrix a(0,nt,1,m);
  dvar_matrix a1(1,nt,1,m);
  dvar_matrix v(1,nt,1,N);
  a(0)=a0;
  P(0)=P0;
  // This is the Kalman filter recursion. The objects tmp1
  // and tmp2 hold common calculations to optimize a bit
  int t;
  for (t=1;t<=nt;t++)
  {
    a1(t)=T*a(t-1)+c;
    P1(t)=T*P(t-1)*TT+Q;
    dvar_vector pred_y=Z*a1(t)+d;
    v(t)=Y(t)-pred_y;
    dvar_matrix tmp1=P1(t)*TZ;
    F(t)=Z*tmp1+H;
    Finv(t)=inv(F(t));
    dvar_matrix tmp2= tmp1*Finv(t);
    P(t)=P1(t)-tmp2*Z*P1(t);
    a(t)=a1(t)+tmp2*v(t);
  }
  int sgn=0;
  dvariable f=0.0;
  for (t=1;t<=nt;t++)
    f+=ln_det(F(t),sgn)+v(t)*Finv(t)*v(t);
  return f;
TOP_OF_MAIN_SECTION
  arrmblsize=20000000;
  gradient_structure::set_CMPDIF_BUFFER_SIZE(3000000);
  gradient_structure::set_GRADSTACK_BUFFER_SIZE(1000000);
\end{lstlisting}
This example was deliberately not optimized as much as it could be,
in order to retain the flavor of the more general state-space
problem. For example, since $T$ is the identity matrix and $c$ is the
zero vector, the line of code
\begin{lstlisting}
  a1(t)=T*a(t-1)+c;
\end{lstlisting}
reduces to
\begin{lstlisting}
  a1(t)=a(t-1);
\end{lstlisting}
The parameters being estimated are  $a_0$, $Q$, and~$H$.

To parameterize the covariance matrices, the Choleski decomposition
parameterization was used. This ensures that the covariance matrices are
positive (semi-) definite. The technique can be seen in the function
\texttt{setup\_Q}.
The lower triangular matrix \texttt{ch\_Q} is filled with parameters
from a bounded vector:
\begin{lstlisting}
FUNCTION setup_Q
  chQ.initialize();
  int ii=1;
  for (int i=1;i<=m;i++)
    for (int j=1;j<=i;j++)
      chQ(i,j)=Qcoff(ii++);
  Q=chQ*trans(chQ);   // chQ is the choleski decomposition of Q
\end{lstlisting}
Notice that the bounded vector \texttt{Qcoff} has slightly asymmetric
bounds. This is a simple way to ensure that its initial value is not
identically zero, which would lead to a singular covariance matrix.
\begin{lstlisting}
  init_bounded_vector Qcoff(1,m1,-10.,10.1)
\end{lstlisting}

The model parameters, standard deviations, and
correlations are reproduced from the
standard \textsc{admb} report.

\begin{smallcode}
 index   name    value      std.dev    7      8      9     10     11     12     13     14     15      16
    7   a0    -1.1682e+00 9.0191e-01  1.000
    8   a0     1.2218e+00 8.6442e-01  0.352  1.000
    9   Q      9.9468e-01 1.0862e-01  0.059 -0.006  1.000
   10   Q      7.8808e-01 7.8737e-02  0.038  0.028  0.683  1.000
   11   Q      7.8808e-01 7.8737e-02  0.038  0.028  0.683  1.000  1.000
   12   Q      8.7279e-01 9.6118e-02 -0.018  0.069  0.185  0.721  0.721  1.000
   13   H      3.1352e+00 1.8123e-01 -0.015 -0.007 -0.305 -0.136 -0.136 -0.018  1.000
   14   H     -2.7119e+00 1.4922e-01 -0.021  0.001 -0.102 -0.238 -0.238 -0.139 -0.692  1.000
   15   H     -2.7119e+00 1.4922e-01 -0.021  0.001 -0.102 -0.238 -0.238 -0.139 -0.692  1.000  1.000
   16   H      3.2264e+00 1.7936e-01  0.015 -0.029 -0.031 -0.121 -0.121 -0.249  0.370 -0.698 -0.698  1.000
\end{smallcode}

\section{Parameterizing the covariance\br matrix parameterizations}

The Choleski decomposition parameterization merely ensures that the
matrix is positive semi-definite. By adding a small positive number to the
diagonal elements, one can ensure that the covariance matrix is
positive definite, and can speed up and improve the stability of the
estimation. Of course, what is meant by ``small'' will depend on the
particular problem being considered.
A modified form of the routine \texttt{setup\_Q}
follows:
\begin{lstlisting}
FUNCTION setup_Q
  int i;
  chQ.initialize();
  int ii=1;
  for (i=1;i<=m;i++)
    for (int j=1;j<=i;j++)
      chQ(i,j)=Qcoff(ii++);
  Q=chQ*trans(chQ);   // chQ is the choleski decomposition of Q
  for (i=1;i<=m;i++)
    Q(i,i)+=0.1;   // make Q positive definite
\end{lstlisting}
Performing this modification for the present model for both $Q$ and
$H$ causes the program to converge about twice as fast.


\subsection{Time Series Decomposition}
\label{decomp}

Concerning the time table in Figure \ref{f:timetable}, a seasonal demand
    pattern is inherent to both horizontal and vertical time series.
First, the weekday influences if people eat out or order in with our partner
    receiving more orders on Thursday through Saturday than the other four
    days.
This pattern is part of both types of time series.
Second, on any given day, demand peaks occur around lunch and dinner times.
This only regards vertical series.
Statistical analyses show that horizontally sliced time series indeed exhibit
    a periodicity of $k=7$, and vertically sliced series only yield a seasonal
    component with a regular pattern if the periodicity is set to the product
    of the number of weekdays and the daily time steps indicating a distinct
    intra-day pattern per weekday.

Figure \ref{f:stl} shows three exemplary STL decompositions for a
    $1~\text{km}^2$ pixel and a vertical time series with 60-minute time steps
    (on the x-axis) covering four weeks:
With the noisy raw data $y_t$ on the left, the seasonal and trend components,
    $s_t$ and $t_t$, are depicted in light and dark gray for increasing $ns$
    parameters.
The plots include (seasonal) na\"{i}ve forecasts for the subsequent test day
    as dotted lines.
The remainder components $r_t$ are not shown for conciseness.
The periodicity is set to $k = 7 * 12 = 84$ as our industry partner has $12$
    opening hours per day.

\begin{center}
\captionof{figure}{STL decompositions for a medium-demand pixel with hourly
                   time steps and periodicity $k=84$}
\label{f:stl}
\includegraphics[width=.95\linewidth]{static/stl_gray.png}
\end{center}

As described in Sub-section \ref{stl}, with $k$ being implied by the
    application, at the very least, the length of the seasonal smoothing
    window, represented by the $ns$ parameter, must be calibrated by the
    forecaster:
It controls how many past observations go into each smoothened $s_t$.
Many practitioners, however, skip this step and set $ns$ to a big number, for
    example, $999$, then referred to as "periodic."
For the other parameters, it is common to use the default values as
    specified in \cite{cleveland1990}.
The goal is to find a decomposition with a regular pattern in $s_t$.
In Figure \ref{f:stl}, this is not true for $ns=7$ where, for
    example, the four largest bars corresponding to the same time of day a
    week apart cannot be connected by an approximately straight line.
On the contrary, a regular pattern in the most extreme way exists for
    $ns=999$, where the same four largest bars are of the same height.
This observation holds for each time step of the day.
For $ns=11$, $s_t$ exhibits a regular pattern whose bars adapt over time:
The pattern is regular as bars corresponding to the same time of day can be
    connected by approximately straight lines, and it is adaptive as these
    lines are not horizontal.
The trade-off between small and large values for $ns$ can thus be interpreted
    as allowing the average demand during peak times to change over time:
If demand is intermittent at non-peak times, it is reasonable to expect the
    bars to change over time as only the relative differences between peak and
    non-peak times impact the bars' heights with the seasonal component being
    centered around $0$.
To confirm the goodness of a decomposition statistically, one way is to verify
    that $r_t$ can be modeled as a typical error process like white noise
    $\epsilon_t$.

However, we suggest an alternative way of calibrating the STL method in an
    automated fashion based on our unified CV approach.
As hinted at in Figure \ref{f:stl}, we interpret an STL decomposition as a
    forecasting method on its own by just adding the (seasonal) na\"{i}ve
    forecasts for $s_t$ and $t_t$ and predicting $0$ for $r_t$.
Then, the $ns$ parameter is tuned just like a parameter for an ML model.
To the best of our knowledge, this has not yet been proposed before.
Conceptually, forecasting with the STL method can be viewed as a na\"{i}ve
    method with built-in smoothing, and it outperformed all other
    benchmark methods in all cases.


%!TEX root = main.tex
We demonstrate the performance and capability of Bempp-Exafmm via electrostatic simulations, including computing the solvation energy of a Zika virus.
This section presents five types of results.
The first result explains the behavior of two variants of the mathematical formulation, from the conditioning point of view.
Second, we show solution verification through two grid-convergence studies: with a spherical molecule (having an analytical solution), and with a real biomolecule (using Richardson extrapolation).
Third, we compare our results with those obtained with the APBS package using 8 other molecules.
The fourth type of result looks at performance with problem sizes between 8,000 and 2 million elements, including timings, breakdowns, and computational complexity.
Our final result is a demonstration using a structure with about 1.6 million atoms, the Zika virus, discretized with about 10 million boundary elements.

We ran all experiments on a single CPU node of \textit{Pegasus}, a Linux cluster at the George Washington University.
Each node is equipped with two 20-core Intel Xeon Gold 6148 CPUs (base frequency at 2.4 GHz, max turbo frequency at 3.7 GHz) and 192GB RAM.
All runs are based on Bempp-cl version 0.2.2 and Exafmm-t version 0.1.0.
We compiled Exafmm with Intel compiler (version 19.0.5.281) and enabled \texttt{-xHost} option for vectorization.
We used the full GMRES from the SciPy library as our linear solver.

In all of the following test cases, the relative permittivity is $\epsilon_1 = 4$ in the solute region and $\epsilon_2 = 80$ in the solvent region, and the inverse of Debye length in the solvent region is $\kappa = 1/8\ {\si{\angstrom}}^{-1}$.
We downloaded the molecule structures from the Protein Data Bank (PDB), parameterized them with \texttt{pdb2pqr}~\cite{DolinskyETal2004}, and generated meshes on the solvent-excluded surface (SES) using \texttt{Nanoshaper} with a probe radius of $1.4 \si{\angstrom}$.

\paragraph{Matrix conditioning of two derivative formulations} \label{result_conditioning}
Section \ref{s:formulation} presents the two formulations to solve the integral equations  \eqref{eq:volume_potential} derived from the Poisson-Boltzmann model of biomolecular electrostatics:
the \emph{direct}~\cite{YoonLenhoff1990}  and \emph{derivative}~\cite{JufferETal1991} formulations.
The latter is well-known to lead to a better-conditioned matrix.
Its most common solution method finds the potential and its derivative in the interior of the boundary via equations \eqref{eq:juffer}, but an alternative is to solve for the exterior fields via \eqref{eq:lu}.
Bempp unfetters the user to experiment with these variants of the boundary element solution method, editing just a few lines of Python.
We could easily try both the interior and exterior versions of the derivative formulation, whereas previous publications opted for one method and used it throughout.
The programming effort required to implement a second formulation would have been a good reason.
In our experiments, the exterior version used by Lu and coworkers~\cite{LuETal2006,LuETal2009,ZhangETal2019} took about half as many iterations to converge than the interior version---a sizable advantage.
This led us to study the properties of the two variants of the derivative formulation in more detail.
The results in this section aim to give a simple explanation for the different numerical behavior of the two methods.
Our ability to explore and explain this issue showcases the power of interactive computing with a high-productivity software platform, like that provided by Bempp-Exafmm.

GMRES methods have an intricate convergence behavior \cite{mark1999a}.
Heuristically, if the eigenvalues are clustered with the cluster being sufficiently far away from the origin, we expect fast convergence of GMRES to the desired solution.
Figures \ref{fig:derivative_interior_eig} and \ref{fig:derivative_exterior_eig} show the eigenvalues of the interior and exterior derivative formulations, respectively, on the complex plane.
With the interior formulation, eigenvalues cluster around two points, while eigenvalues cluster around only one point with the exterior formulation.

The difference is due to the diagonal of the corresponding system of integral equations.
In the case of the interior formulation, the associated left-hand side operator takes the form
$$
\begin{bmatrix}\frac{1}{2}(1 + \frac{\epsilon_2}{\epsilon_1})I & 0 \\ 0 & \frac{1}{2}(1 + \frac{\epsilon_1}{\epsilon_2})I
\end{bmatrix} + \mathcal{C}_{int},
$$
where $\mathcal{C}_{int}$ is a compact operator on sufficiently smooth domains. (On smooth domains the single-layer, double-layer and adjoint double-layer operators are compact operators.
Furthermore, the difference of the hypersingular operators is compact \cite{Hiptmair2006-om}.)
The eigenvalues of the interior derivative operator hence accumulate at the points $\frac{1}{2}(1 + \frac{\epsilon_2}{\epsilon_1})$ and $\frac{1}{2}(1 + \frac{\epsilon_1}{\epsilon_2})$.
In contrast, the exterior derivative operator has the form
$$
\begin{bmatrix}\frac{1}{2}(1 + \frac{\epsilon_1}{\epsilon_2})I & 0 \\ 0 & \frac{1}{2}(1 + \frac{\epsilon_1}{\epsilon_2})I
\end{bmatrix} + \mathcal{C}_{ext},
$$
where $\mathcal{C}_{ext}$ is again a compact operator.
We now only have one accumulation point, namely $\frac{1}{2}(1 + \frac{\epsilon_1}{\epsilon_2})$.
Unless $\epsilon_1\approx \epsilon_2$ we therefore expect the eigenvalues to be much closer together than in the interior derivative case and therefore the GMRES convergence to be faster for the exterior derivative formulation.

We want to emphasize that the above argument is valid for the continuous operators.
Under discretization, the resulting eigenvalue problem is of the form $A\mathbf{x}=\lambda M\mathbf{x}$ (or equivalently $M^{-1}A\mathbf{x}=\lambda \mathbf{x}$, where $A$ is the $2\times 2$ block operator associated with the Galerkin discretization of the integral operator system and $M = \text{diag}(\hat{M}, \hat{M})$ is a block diagonal mass matrix, where the matrix $\hat{M}$ is the matrix containing the surface inner products $\int_{\Gamma}\psi_i(\mathbf{r})\phi_j(\mathbf{r})ds(\mathbf{r})$ for test functions $\psi_j$ and trial functions $\phi_i$ (which are both chosen as continuous, piecewise linear basis functions for the derivative formulation in this paper).
The mass-matrix preconditioned linear system of equations to solve has the form $M^{-1}A\mathbf{x} = M^{-1}\mathbf{b}$ for vector of unknowns $\mathbf{x}$ and right-hand side $\mathbf{b}$ and Figures \ref{fig:derivative_interior_eig} and \ref{fig:derivative_exterior_eig} show the eigenvalues of $M^{-1}A$ for the interior and exterior derivative formulation.
In practice, the action of $M^{-1}$ can be computed through a sparse LU decomposition of $M$.
However, for problems with millions of unknowns this is becoming expensive.
For practical electrostatic computations we have therefore chosen a simple mass lumping approach, in which we substitute $M$ by a diagonal matrix, where each diagonal entry is the sum of the corresponding row values of $M$.
This diagonal matrix can then be trivially inverted.
In our experiments the mass lumping only led to a modest increase in the number of iterations compared to using the LU decomposition of the mass matrix $M$.
We stress that mass matrix preconditioning is necessary to solve the interior and exterior formulations in a reasonable number of iterations.
The approximation of the inverse of the mass matrix $M$ through mass lumping only affects the preconditioner and not the solution of the underlying linear system of equations itself.

In each of the following studies, we present two sets of results: one from using the exterior derivative formulation with piecewise linear elements, preconditioned by a mass lumping matrix, and the other from using the direct formulation with piecewise constant elements, preconditioned by a block-diagonal matrix presented by Altman and co-workers \cite{AltmanBardhanWhiteTidor2009}.

\begin{figure*}
    \begin{center}
        \subfloat[][]{\includegraphics[width=0.4 \textwidth]{derivative_interior_eig.pdf}
        \label{fig:derivative_interior_eig}}\qquad
        \subfloat{\includegraphics[width=0.4 \textwidth]{derivative_exterior_eig.pdf}
        \label{fig:derivative_exterior_eig}}
    \end{center}
    \caption{Eigenvalues of the system matrix of the derivative formulation for interior field (\textbf{a}) and for exterior field (\textbf{b}).
    }
\end{figure*}

\paragraph{Mesh refinement study using a spherical molecule} \label{result_convergence_sphere}

As a form of solution verification with the Bempp-Exafmm software, we completed two mesh-refinement studies.
The first used a spherical molecule with an off-center charge, for which we have an analytical solution.
In the next sub-section, we present a mesh-refinement study with a real molecule of biological relevance.
Figure \ref{fig:sketch_sphere_convergence} depicts the problem setup for the current case:
a spherical molecule of radius 4 \AA\ and relative permittivity $\epsilon_1 = 4$, with a unit charge located at $(1,1,1)$.
The solvent region has the relative permittivity of water ($\epsilon_2 = 80$), and a salt concentration of $150$mM $(\kappa = 1/8\ {\si{\angstrom}}^{-1})$.
Other simulation parameters are listed in Table \ref{tab:convergence}.
With an expansion order of 10, our \fmm achieved 9 digits of accuracy.
We computed the solvation energy of this molecule using $5$ different meshes, obtained using a constant refinement factor of 4.

\begin{table}[]
    \centering
    \begin{tabular}{lc}
    \hline
    \gmres tolerance          & $10^{-7}$ \\
    \# regular quadrature points  & 6    \\
    \fmm expansion order      & 10   \\
    \fmm $\ncrit$             & 500  \\
    \hline  \vspace{0.3 cm}
    \end{tabular}

    \begin{tabular}{cc}
    number of elements & mesh density ($\#/{\si{\angstrom}}^2$) \\
    \hline
    3032               & 1                                       \\
    6196               & 2                                       \\
    12512              & 4                                       \\
    25204              & 8                                       \\
    50596              & 16
    \end{tabular}
    \caption{Simulation parameters for the grid-refinement studies (top); mesh sizes/densities used in the grid refinement study on 5PTI. Mesh densities measured as number of elements per square Angstrom.}
    \label{tab:convergence}
\end{table}

Kirkwood's derivation \cite{kirkwood1934theory} allows us to compute the analytical solution for the solvation energy in this case, to compare with the numerical result: $-12.258363$ [kcal/mol].
Figure \ref{fig:sphere_convergence} shows the error in the solvation energy, converging at the expected rate of $1/N$ for both formulations.
The observed order of convergence is 1.001 for the direct formulation and 0.999 for the derivative formulation, using the middle three values.

\paragraph{Mesh refinement study using 5PTI} \label{result_convergence_5PTI}

Next, we tested our software using a real biomolecule: bovine pancreatic trypsin inhibitor (PDB code 5PTI), whose structure \cite{wlodawer1984structure} is shown in Figure \ref{fig:5PTI_structure}.
We parameterized the molecule with \texttt{pdb2pqr} and the \texttt{charmm} force field, and then computed the solvation energy using 5 meshes with the element density ranging from 1 to 16 (Table \ref{tab:convergence}).
This test used the same parameters listed in Table \ref{tab:convergence}, which are fine enough to reveal the discretization error.
Since an analytical solution is not available for this geometry, we obtained the reference values for error estimation via Richardson extrapolation.
The estimated relative error with the finest mesh is 1.2\% with the direct formulation, and 1.5\% with the derivative formulation.

Figure \ref{fig:5PTI_convergence} shows that the error of the computed solvation energy for 5PTI converges linearly with respect to $N$.
The observed order of convergence is 1.156 for the direct formulation and 1.038 for the derivative formulation, using the middle three values.
Both convergence results provide solution verification, and are evidence that our software solves the mathematical model correctly.

\begin{figure*}
        \centering
     \subfloat[][Sphere with an off-center unit charge at $(1,1,1)$.]{\includegraphics[width=0.4 \textwidth]{sketch_sphere_convergence.pdf}
        \label{fig:sketch_sphere_convergence}}\quad
     \subfloat[][Structure of 5PTI.]{\includegraphics[width=0.35 \textwidth]{5PTI.png}
        \label{fig:5PTI_structure}}\\
     \subfloat[][]{\includegraphics[width=0.4 \textwidth]{sphere_convergence.pdf}
        \label{fig:sphere_convergence}}
     \subfloat[][]{\includegraphics[width=0.4 \textwidth]{5PTI_convergence.pdf}
        \label{fig:5PTI_convergence}}
    \caption{Mesh refinement studies using a spherical molecule and a real biomolecule: bovine pancreatic trypsin inhibitor (PDB code 5PTI).
    \textbf{c}, Mesh convergence study on a spherical molecule with an off-center charge, using both direct formulation and derivative formulation. The error on the solvation energy with respect to the analytical solution is plotted for five meshes:
    the sphere discretized with 512, 2048, 8192, 32768 and 131072 boundary elements.
    \textbf{d}, Mesh convergence study of the solvation energy of bovine pancreatic trypsin inhibitor (PDB code 5PTI), using both direct formulation and derivative formulation.
    The error is with respect to the extrapolated solution using Richardson extrapolation.
    }
\end{figure*}

% add comparison with results from APBS
\paragraph{Comparison with trusted community software} \label{comparison}
We build more confidence on Bempp-Exafmm by comparing our results with a well-established finite-difference code APBS (version 1.5), using 5PTI and 8 other molecules.
We prepared the molecular structures using the \texttt{charmm} force field.
For each molecule, we created three successive meshes to study the convergence with each software.
For Bempp-Exafmm, we used \texttt{Nanoshaper} to generate three surface meshes with the element density set to 4, 8, 16 ${\si{\angstrom}}^{-2}$, respectively.
In APBS, we controlled the multigrid parameters to ensure that three grid spacings are refined with a constant factor of 2.
All mesh sizes are listed in Appendix \ref{sec:apbs_mesh}.
Other parameters are the same as our previous grid-convergence study, as listed in Table \ref{tab:convergence}.

Table \ref{tab:APBS_result} presents the convergence results of both codes using a set of molecules.
Bempp used the derivative formulation throughout.
In each case, we obtained an extrapolated value of the solvation energy of each molecule, which serves as the reference value to estimate the discretization error with each software.
As expected, the solvation energy computed from Bempp converges at the rate of $\mathcal{O}(N^{-1})$, where $N$ is the number of elements.
With APBS, the average observed order of convergence over all cases is 1.25, with respect to the grid spacing $h$; other studies \cite{CooperBardhanBarba2014,GengKrasny2013} have reported a similar rate.
Our $\mathcal{O}(N^{-1})$ convergence corresponds to $\mathcal{O}(h^{-2})$ convergence in grid size, but for boundary elements (surface triangles) it is more natural to look at convergence with respect to $N$.
The difference in the extrapolated solvation energies obtained with the two codes varies between 0.8\% and 1.8\% across all cases.
A discrepancy is expected because Bempp-Exafmm and APBS use different geometrical representations for the molecular surface, while the atomic charges in the molecule are mapped to grid points through interpolation in APBS.
(We also compared our results with MIBPB \cite{chen2011mibpb} using only the finest mesh: the details can be found in Appendix \ref{sec:comp_mibpb}.)

\begin{table*}[]
    \centering
    \resizebox{\textwidth}{!}{%
    \begin{tabular}{cc|ccc|cc|ccc|cc}
      &   & \multicolumn{5}{c|}{APBS}                                                                    & \multicolumn{5}{c}{Bempp}                                                                   \\
      &   & \multicolumn{3}{c|}{Error}     & \multirow{2}{*}{$\Delta G_{solv}$} & \multirow{2}{*}{Order$(1/h)$} & \multicolumn{3}{c|}{Error}     & \multirow{2}{*}{$\Delta G_{solv}$} & \multirow{2}{*}{Order$(1/N)$} \\
    ID   & $N_{atoms}$ & coarse   & medium   & fine     &                                    &                        & coarse   & medium   & fine     &                                    &                        \\ \hline
    1AJJ & 513 & 4.40e-02 & 1.33e-02 & 4.00e-03 & -266.15                            & 1.73                   & 2.75e-02 & 1.12e-02 & 4.59e-03 & -268.67                            & 1.29                   \\
    1VJW & 826 & 3.09e-02 & 1.45e-02 & 6.77e-03 & -297.06                            & 1.09                   & 4.92e-02 & 2.23e-02 & 1.01e-02 & -302.50                            & 1.14                   \\
    5PTI & 892 & 3.68e-02 & 1.38e-02 & 5.14e-03 & -311.69                            & 1.42                   & 5.12e-02 & 2.23e-02 & 9.67e-03 & -314.34                            & 1.20                   \\
    1R69 & 997 & 4.31e-02 & 2.10e-02 & 1.02e-02 & -261.02                            & 1.04                   & 5.09e-02 & 2.34e-02 & 1.08e-02 & -265.02                            & 1.12                   \\
    1A2S & 1272 & 5.25e-02 & 2.40e-02 & 1.10e-02 & -456.56                            & 1.13                   & 4.21e-02 & 1.93e-02 & 8.86e-03 & -461.25                            & 1.12                   \\
    1SVR & 1433 & 5.28e-02 & 2.21e-02 & 9.28e-03 & -393.45                            & 1.25                   & 6.48e-02 & 2.89e-02 & 1.29e-02 & -398.83                            & 1.16                   \\
    1A63 & 2065 & 4.51e-02 & 1.88e-02 & 7.82e-03 & -559.39                            & 1.26                   & 5.82e-02 & 2.60e-02 & 1.16e-02 & -567.24                            & 1.16                   \\
    1A7M & 2804 & 4.13e-02 & 1.88e-02 & 8.53e-03 & -524.29                            & 1.14                   & 4.93e-02 & 2.24e-02 & 1.02e-02 & -531.48                            & 1.14                   \\
    1F6W & 8247 & 6.45e-02 & 2.78e-02 & 1.20e-02 & -1277.51                           & 1.21                   & 4.18e-02 & 1.80e-02 & 7.76e-03 & -1301.08                           & 1.22
    \end{tabular}
    }
    \caption{Convergence results of the solvation energy of 9 molecules using APBS and Bempp with derivative formulation.
    The error is with respect to an extrapolated value of the solvation energy using Richardson extrapolation.
    The solvation energy $\Delta G_{solv}$ is in units of kcal/mol.
    The observed order of convergence is with respect to the grid spacing $h$ for APBS (volumetric-based solver) and with respect to the number of elements $N$ for Bempp (boundary-element solver).}
    \label{tab:APBS_result}
\end{table*}

\paragraph{Performance study with direct and derivative formulations} \label{result_performance}

In this sub-section, we investigate the computational performance of Bempp-Exafmm using a spherical molecule.
The sphere has a radius of 1 \AA\, and 100 charges are placed randomly inside, representing the atoms in the solute.
We used the same dielectric constants and salt concentration as in previous grid-convergence studies.
Other simulation parameters are listed in Table \ref{tab:sim_params_performance}.

\begin{table}[]
    \centering
    \begin{tabular}{lc}
    \hline
    \gmres tolerance          & $10^{-4}$ \\
    \# regular quadrature points  & 6    \\
    \fmm expansion order      & 5   \\
    \fmm $\ncrit$             & 500  \\
    \hline
    \end{tabular}
    \caption{Simulation parameters used in the performance study for a spherical molecule.}
    \label{tab:sim_params_performance}
\end{table}

To cover a wide range of problem sizes, we used five surface discretizations, with the number of elements ranging from 8 thousand to 2 million.
Table \ref{tab:sphere_time} presents the assembly time, the solution time and the number of iterations to converge in each case for both formulations.
The assembly time includes the time to pre-compute the \fmm invariant matrices, create sparse and singular assemblers and calculate preconditioners.
The algebraic convergence shows that the condition number grows as the problem size increases with the direct formulation, while it remains at the same level with the derivative formulation.

\begin{table*}[]
    \centering
    \begin{tabular}{c|cccc|cccc}
                                                                 & \multicolumn{4}{c|}{direct}                                                                                & \multicolumn{4}{c}{derivative}                                                                                \\ \hline
    \begin{tabular}[c]{@{}c@{}}number of\\ elements\end{tabular} & \begin{tabular}[c]{@{}c@{}}total\\ time (s)\end{tabular} & \begin{tabular}[c]{@{}c@{}}assembly\\ time (s)\end{tabular} & \begin{tabular}[c]{@{}c@{}}GMRES\\ time (s)\end{tabular} & \# iterations & \begin{tabular}[c]{@{}c@{}}total\\ time (s)\end{tabular} & \begin{tabular}[c]{@{}c@{}}assembly\\ time (s)\end{tabular} & \begin{tabular}[c]{@{}c@{}}GMRES\\ time (s)\end{tabular} & \# iterations \\ \hline
    8192                                                         & 14.0                                                     & 5.4                                                         & 8.6                                                      & 20            & 16.1                                                     & 9.6                                                         & 6.5                                                      & 5            \\
    32768                                                        & 35.1                                                     & 11.7                                                        & 23.4                                                     & 24            & 35.5                                                     & 22.2                                                        & 13.3                                                     & 4            \\
    131072                                                       & 144.2                                                    & 32.8                                                        & 111.4                                                    & 34            & 114.7                                                    & 67.2                                                        & 47.5                                                     & 4            \\
    524288                                                       & 675.8                                                    & 121.6                                                       & 554.2                                                    & 51            & 421.3                                                    & 256.3                                                       & 165.0                                                    & 4            \\
    2097152                                                      & 3159.8                                                   & 483.3                                                       & 2676.5                                                   & 70            & 1592.1                                                   & 1011.6                                                      & 580.5                                                    & 4
    \end{tabular}
    \caption{Assembly and solution times of calculating the solvation energy of a spherical molecule with 100 random charges inside, using the direct and derivative (exterior) formulations.
    6 regular quadrature points were used per element and the \fmm expansion order was set to 5.}
    \label{tab:sphere_time}
\end{table*}

In our implementation, each iteration in direct formulation requires $8$ \fmm evaluations, whereas each iteration in the derivative formulation requires 19, making it more than twice as expensive.
That explains why the direct formulation leads to a shorter solution time (\gmres time), despite a slower convergence, in the two smaller cases.
For larger problem sizes, faster convergence in the derivative formulation offsets the larger cost per iteration.

As for the assembly time, the derivative formulation is about $2\times$ slower, since it needs to construct twice as many operators as the direct formulation.
In addition, the two hypersingular operators make it even more involved.
Figure \ref{fig:sphere_assembly_time} shows the linear scaling of the assembly time with respect to $N$.

Next, we confirm that the time complexity of mat-vecs in \gmres is also $\mathcal{O}(N)$.
The Poisson equation requires \fmm with a Laplace kernel and the linearized Poisson-Boltzmann equation requires \fmm with a Yukawa kernel.
As mentioned, each iteration involves multiple \fmm evaluations: 4 Laplace {\fmm}s and 4 Yukawa {\fmm}s for the direct formulation, 8 and 11 for the derivative formulation.
We averaged the time spent on 1 Laplace \fmm and 1 Yukawa \fmm respectively using direct formulation, and plotted it with respect to $N$ in Figure \ref{fig:sphere_fmm}.
Using an \fmm order of 5, we achieve about 5 digits of accuracy in each mat-vec.
The timings and linear scaling substantiate the efficiency of our \fmm implementation.
In the largest case with over 12 million quadrature points, one Laplace \fmm costs $2.1$s and one Yukawa \fmm costs $5.4$s to compute.

In Bempp, the matrix-vector product has the shape $A\mathbf{x} = P_2^T (G - C)P_1 \mathbf{x} + S \mathbf{x}$ (see Equation \ref{eq:bempp_fmm_matvec}), where the dominant costs are the \fmm evaluation of the Green's function matrix $G$ and the on-the-fly evaluation of the singular correction matrix $C$. Moreover, as stated above for the full $2\times 2$ block system a number of \fmm passes together with corresponding singular corrections need to be performed.
Therefore, the \gmres time reported here consists of \fmm time, singular correction time, as well as the time spent on other steps in the \gmres algorithm.
Figure \ref{fig:sphere_gmres_direct} and \ref{fig:sphere_gmres_derivative} show the time breakdown of \gmres in percentages.
As problem size increases, \fmm evaluations dominate the solution time.

We also measured the peak memory usage using the GNU time command \texttt{/usr/bin/time -v}. (Some Linux distributions do not ship with GNU time.)
We observed a linear space complexity as shown in Figure \ref{fig:sphere_memory}.
The largest case, with more than $2$ million elements, requires $36$GB for the direct formulation and $43$GB for the derivative formulation.

\begin{figure*}[t]
\centering
    \subfloat[][Assembly time]{\includegraphics[width=0.33\textwidth]{sphere_assembly_time.pdf}
        \label{fig:sphere_assembly_time}}
   \subfloat[][Average evaluation time.]{\includegraphics[width=0.33\textwidth]{sphere_fmm.pdf}
        \label{fig:sphere_fmm}}
    \subfloat[][Overall memory consumption in GB.]{\includegraphics[width=0.33\textwidth]{sphere_memory.pdf}
        \label{fig:sphere_memory}}\\
    \subfloat[][]{\includegraphics[width=0.4\textwidth]{sphere_gmres_direct.pdf}
        \label{fig:sphere_gmres_direct}}
  \subfloat[][]{\includegraphics[width=0.4\textwidth]{sphere_gmres_derivative.pdf}
        \label{fig:sphere_gmres_derivative}}
    \caption{Performance on a spherical molecule with 100 random charges inside;
    6 regular quadrature points per element; \fmm expansion order set to 5 to achieve 5 digits of accuracy. Problem size represented by number of elements, $N$. Evaluation time (b) is an average for 1 Laplace \fmm evaluation and 1 Yukawa \fmm evaluation across all iterations in GMRES using direct formulation.
    \textbf{c},\textbf{d}, Time breakdown of \gmres in percentage using direct formulation (\textbf{c}) and derivative formulation (\textbf{d}).}
\end{figure*}

\paragraph{Solvation energy of a Zika virus} \label{result_zika}

Finally, we present a more challenging problem that studies the solvation energy of the Zika virus (PDB code 6CO8), whose structure \cite{sevvana2018refinement} is shown in Figure \ref{fig:6CO8_assembly}.
We parameterized the molecular structure with the \texttt{amber} force field, and generated a mesh on the SES using \texttt{Nanoshaper}. (Mesh generation took less than 5 min.)
The prepared structure contains about 1.6 million atoms and our mesh has around 10 million boundary elements and a total surface area of $3.14\times 10^6 {\si{\angstrom}}^{2}$, corresponding to an element density of $3.16 {\si{\angstrom}}^{-2}$.
In this experiment, 3 quadrature points were used for regular Galerkin integrals over disjoint elements.
The \fmm expansion order was set to 4 obtain 4 digits of accuracy in mat-vecs and the tolerance of \gmres was $10^{-4}$.

\begin{table*}[]
    \centering
    \begin{tabular}{c|c|ccccc}
                 & \begin{tabular}[c]{@{}c@{}}$\Delta G_{\mathrm{solv}}$\\ (kcal/mol)\end{tabular} & \begin{tabular}[c]{@{}c@{}}total \\ time (s)\end{tabular} & \begin{tabular}[c]{@{}c@{}}assembly \\ time (s)\end{tabular} & \begin{tabular}[c]{@{}c@{}}GMRES \\ time (s)\end{tabular} & \begin{tabular}[c]{@{}l@{}}memory\\ (GB)\end{tabular} & \# iterations \\ \hline
    direct     & -116587.5                                               & 11005.4                                                   & 1534.5                                                       & 9470.9                                                    & 109.7                                                 & 105           \\
    derivative & -116254.9                                               & 8370.3                                                    & 3553.9                                                       & 4816.4                                                    & 152.0                                                 & 18
    \end{tabular}
    \caption{Results of computing the solvation energy of a Zika virus with Bempp using both the direct and derivative formulations.
    3 regular quadrature points were used per element and the \fmm expansion order was set to 4.
    The mesh generation time is minor and not included in the total time.
    To verify our result, we ran the same case with \pygbe using the direct formulation on a workstation with a 24-core CPU.
    The solvation energy computed from \pygbe is -117261.1 [kcal/mol].
    }
    \label{tab:6CO8_result}
\end{table*}

\begin{figure*}
    \subfloat[][]{\includegraphics[width=0.25\textwidth]{6CO8_assembly.png}
        \label{fig:6CO8_assembly}}
\subfloat[][]{\includegraphics[width=0.75\textwidth]{6CO8_potential.png}
        \label{fig:6CO8_potential}}
    \caption{\textbf{a}, Structure of Zika virus (PDB code 6CO8) in assembly view. Each color indicates a polymer chain.
    \textbf{b}, Surface electrostatic potential of a Zika virus.
    The color bar is in units of [kcal/mol$\cdot$e].
    Visualization generated using ParaView.
    The starfish pattern seen in the polymer-chain colorization of Figure \ref{fig:6CO8_assembly} is faintly visible in the potential.
    }
\end{figure*}

Table \ref{tab:6CO8_result} summarizes the results and performance.
Again, we confirmed that the derivative formulation yields a well-conditioned system, which converged in 18 iterations and took less than 1.5 hours to solve.
By contrast, the direct formulation took almost twice as long to converge.

In this case, we also verified against \pygbe \cite{cooper2016pygbe}, a Python \bem library for biomolecular electrostatics.
Based on the solvation energy computed from \pygbe: $-117261.1$ [kcal/mol], the relative difference of our result is 0.6\% with the direct formulation and 0.9\% with the derivative formulation.
Figure \ref{fig:6CO8_potential} shows the computed electrostatic potential on the surface.

\paragraph{Reproducibility package}

Besides releasing all our software with an open-source license, we spared no effort to maximize the reproducibility of this work, compiling a ``repro-pack" in a version-control repository.
It contains all raw data from the experiments and a small Python package---\texttt{bempp\_pbs}, to facilitate running PB simulations with Bempp.
\texttt{bempp\_pbs} comprises a collection of scripts, including all post-processing scripts that are necessary to produce every result presented in this section, and driver and utility scripts for different formulations, with which readers can run these cases on their own hardware.
In addition, the ``repro-pack" also provides a Jupyter notebook for each study to generate secondary data and results.
The notebook corresponding to section \ref{result_conditioning} showcases how we could run a simple case of a spherical molecule using two formulations with just a few lines of code, and then compare their conditioning quantitatively with the help of other scientific Python tools.
These supplementary materials are included in our paper's GitHub repository at \href{https://github.com/barbagroup/bempp\_exafmm\_paper/}{https://github.com/barbagroup/bempp\_exafmm\_paper/}, which is also archived on Zenodo at \href{http://doi.org/10.5281/zenodo.4568951}{doi:10.5281/zenodo.4568951}.
We made separate archival deposits of input data (meshes and \texttt{pqr} files ) on the Zenodo service.
The deposit can be downloaded from \href{http://doi.org/10.5281/zenodo.4568768}{doi:10.5281/zenodo.4568768}.


%Chapter "Elemental Matrices"
%
\chapter{Explicit Dynamic Analysis}

\section{Statement of the Problem}
Consider the discrete dynamic equilibrium equations at time $t$

\begin{equation}
M^{t}A+C^{t}V+K^{t}U=^{t}F
\label{equil}
\end{equation}

where $M$, $C$, $K$ are the assembled mass, damping and stiffness matrix respectively and similarly $^{t}A$, $^{t}V$, $^{t}U$, ${t}F$ are the nodal accelerations, velocities, displacements and external loads vectors at time $t$. In terms of forces \eqref{equil} can be written like;

\begin{equation}
^{t}F^I+^{t}F^D+^{t}F^s=^{t}F
\label{force equil}
\end{equation}

where $^{t}F^I$, $^{t}F^D$ and $^{t}F^s$ are inertial, damping and elastic components respectively.

Expanding the acceleration and velocity terms at time $t$ in a consistent finite central differences scheme we have;

\begin{equation}
\begin{aligned}
^{t}A&=\dfrac{1}{\Delta t^2}\left(^{t-\Delta t}U-2^{t}U+^{t+\Delta t}U\right)\\
^{t}V&=\dfrac{1}{2\Delta t}\left(-^{t-\Delta t}U+^{t+\Delta t}U\right)
\end{aligned}
\label{finitediff}
\end{equation}

Using \eqref{finitediff} in \eqref{force equil} yields;

\begin{equation}
\left(\dfrac{1}{\Delta t^2}M+\dfrac{1}{2\Delta t}C\right) ^{t+\Delta t}U=^{t}F-\left(K-\dfrac{2}{\Delta t^2}M\right) ^{t}U-\left(\dfrac{1}{\Delta t^2}M-\dfrac{1}{2\Delta t^2}C\right)^{t-\Delta t}U
\label{resequil}
\end{equation}

Redefine forces as follows
\begin{equation}
\begin{aligned}
^{j}F^I&=\dfrac{1}{\Delta t^2}M ^{j}U\\
^{j}F^D&=\dfrac{1}{2 \Delta t}C ^{j}U\\
^{j}F^S&=K ^{j}U
\end{aligned}
\label{redefine}
\end{equation}

and write \eqref{resequil} as;

\begin{equation}
^{t+\Delta t}F^I+^{t+\Delta t}F^D=^{t}F-^{t}F^s+2 ^{t}F^I-^{t-\Delta t}F^I+^{t-\Delta t}F^D
\label{force equil 2}
\end{equation}

\begin{itemize}
\item \eqref{force equil 2} is an equilibrium equation at time $t=t$ allowing to predict the displacements at time $t=t+\Delta t$ in terms of previously known values at times $t$ and $t=t-\Delta t$.

\item The equation is exact within the error introduced by the expansion used in \eqref{finitediff}.

\begin{figure}[h]
\centering
\includegraphics[width=12cm]{img/figure7_0.pdf}
\caption{Definition of the general iteration}
\label{fig:time iteration}
\end{figure}

\item The first predicted solution is at $t=\Delta t$ and we require data at $t=-\Delta t$ and at $t=0$.

\begin{figure}[h]
\centering
\includegraphics[width=12cm]{img/figure7_1.pdf}
\caption{Definition of the general iteration}
\label{fig:initial time iteration}
\end{figure}

\end{itemize}

\subsection{Damping Assumptions}

\begin{itemize}
\item[1] Use Rayleigh Damping and retain the velocity expansion used in \eqref{finitediff}. That is;

\begin{equation}
C=\alpha M+\beta K
\end{equation}

then we have (in terms of forces);

\begin{equation}
(1+\beta \Delta t^2) ^{t+\Delta t}F^I+\dfrac{\alpha}{2\Delta t} ^{t+\Delta t}F^S=^{t}\hat{F}
\label{Rayleigh}
\end{equation}

where;

\[
^{t}\hat{F}=^{t}R-^{t}F^S+2 ^{t}F^I-^{t-\Delta t}F^I+^{t-\Delta t}F^D
\]

Solution in equation \eqref{Rayleigh} requires the full assembly and factorization of an effective stiffness matrix.

\item[2] Neglect damping (This is however inconvenient for finite domains)

\begin{equation}
^{t+\Delta t}F^I=^{t}F-^{t}F^S+2 ^{t}F^I-^{t-\Delta t}F^I
\label{Nodamping}
\end{equation}

\item[3] Use Rayleigh damping but modify the velocity expansion introduced in \eqref{finitediff}. Using

\begin{equation}
^{t}V=\dfrac{1}{\Delta t}(^{t}U-^{t-\Delta t}U)
\label{velocity}
\end{equation}

yielding;

\begin{equation}
^{t+\Delta t}F^I=^{t}F-^{t}F^S+2 ^{t}F^{I}-^{t}F^D+^{t-\Delta t}F^D-^{t-\Delta t}F^I
\end{equation}

Defining a set of forces associated to the initial conditions like;

\[
^{t-\Delta t}F^{IC}=^{t-\Delta t}F^I-^{t-\Delta t}F^D
\]

we have

\begin{equation}
^{t+\Delta t}F^I=^{t}F-^{t}F^S+2 ^{t}F^{I}-^{t}F^D-^{t-\Delta t}F^{IC}
\label{modvelocity}
\end{equation}

\end{itemize}

\subsection{Algorithm corresponding to the damping assumption 3}

\begin{figure}[h]
\centering
\includegraphics[width=12cm]{img/figure5.pdf}
\caption{Definition of the general iteration}
\label{fig:general iteration}
\end{figure}

Let us write \eqref{modvelocity} like

\begin{equation}
^{j+1}F^I=^{j}F-^{j}F^S+2 ^{j}F^{I}-^{j}F^D-^{j-1}F^{IC}
\label{modveliter}
\end{equation}

where the initialization process corresponds to;

\begin{figure}[h]
\centering
\includegraphics[width=12cm]{img/figure6.pdf}
\caption{Definition of the initial iteration}
\label{fig:initial iteration}
\end{figure}

Applying \eqref{modveliter} for $t=0$ we have;

\[
^{0}F^I=^{0}F-^{0}F^S+2 ^{0}F^{I}-^{0}F^D-^{-\Delta t}F^{IC}
\]

from which it is clear that we require $-^{\Delta t}U$. Applying the central difference expansion at $t=0$ and solving for $-^{\Delta t}U$ yields;

\begin{equation}
^{-\Delta t}U=^{0}F-\Delta t ^{0}V+\dfrac{\Delta t^2}{2} ^{0}A
\label{initialU}
\end{equation}

Using \eqref{initialU} in \eqref{modveliter} allows us to start up the algorithm.

\subsubsection{Particulars}
In what follows we concentrate on this last algorithm and in order to study some details we return to its standard displacements form. Writing \eqref{modveliter} in terms of displacements and re-arranging yields;

\begin{equation}
\dfrac{1}{\Delta t^2}M ^{t+\Delta t}U=^{t}F-(1+\dfrac{\beta}{\Delta t})K ^{t}U+(\dfrac{2}{\Delta t^2}-\dfrac{\alpha}{\Delta t})M ^{t}U-(\dfrac{1}{\Delta t^2} - \dfrac{\alpha}{\Delta t})M ^{t-\Delta t}U+(\dfrac{\beta}{\Delta t}) K ^{t-\Delta t}U
\label{disequil}
\end{equation}

Let;

\[
\begin{aligned}
a_1&=1+\dfrac{\beta}{\Delta t}\\
a_2&=\dfrac{2}{\Delta t^2}-\dfrac{\alpha}{\Delta t}\\
a_3&=\dfrac{1}{\Delta t^2} - \dfrac{\alpha}{\Delta t}\\
a_4&=\dfrac{\beta}{\Delta t}
\end{aligned}
\]

\begin{equation}
^{t+\Delta t}F^I=^{t}F-a_1K ^{t}U+a_2M ^{t}U-a_3M ^{t-\Delta t}U+a_4K ^{t-\Delta t}U
\label{equliassum3}
\end{equation}

\subsection{Decoupling}
Consider the equation for the $i$-th d.o.f;

\begin{equation}
\dfrac{1}{\Delta t^2}M_{ij} ^{t+ \Delta t}U_j=^{t}F_i-a_1K_{ij} ^{t}U_j+a_2M_{ij}^{t}U_j-a_3M_{ij} ^{t-\Delta t}U_j+a_4K_{ij}^{t-\Delta t}U_j
\label{equildecoupled1}
\end{equation}

where we keep $i$ fixed in \eqref{equildecoupled1}. For a lumped mass matrix we can write;

\[
M_{ij}=m_I\delta_{ij}
\]

then \eqref{equildecoupled1} becomes;

\begin{equation}
\dfrac{1}{\Delta t^2}m_I ^{t+ \Delta t}U_i= ^{t}F_I-a_1 K_{ij} ^{t}U_j+a_2 m_I ^{t}U_i-a_3 m_I ^{t-\Delta t} U_i+a_4 K_{ij} ^{t-\Delta t} U_j
\label{equildecoupled2}
\end{equation}

Let;

\begin{equation}
\begin{aligned}
^{t+\Delta t} F_i^I&=\dfrac{1}{\Delta t^2} m_{I} ^{t+\Delta t}U_i\\
^{t} \hat{F}^S_i&=a_1 K_{ij} ^{t}U_j\\
^{t}\hat{F}_i^I&=a_2m_{I} ^{t}U_i\\
^{t-\Delta t}\hat{F}_i^I&=a_3m_{I} ^{t-\Delta t}U_i\\
^{t-\Delta t}\hat{F}_i^S&=a_4K_{ij}^{t-\Delta t}U_j
\end{aligned}
\label{forces2}
\end{equation}

so the recursive equation takes the form;

\begin{equation}
^{t+\Delta t} F_i^I=^{t}F_i-^{t} \hat{F}^S_i+^{t}\hat{F}_i^I-^{t-\Delta t}\hat{F}_i^I+^{t-\Delta t}\hat{F}_i^S
\label{forces3}
\end{equation}

and the algorithm then reduces to;

\begin{algorithm}[H]
\SetAlgoLined
\KwData{Time span, Geometry, Material Paramters}
\KwResult{Displacements, Velocity and Acceleration time histories }
Compute $^{t+\Delta t} F_i^I$\\
Solve for $^{t+\Delta t}U_i=\left(\dfrac{\Delta t^2}{m_I}\right) ^{t+\Delta t}F_{i}^I$\\
Update $^{t}V_i$, $^{t}A_i$
\caption{Summarized Algorithm}
\end{algorithm}

To initialize the algorithm we apply the FD's equations at $t=0$

\[
\dfrac{1}{\Delta t^2}m_I ^{\Delta t}U_i=^{0}F_i-a_1 K_{ij} ^{0}U_j+a_2 m_I ^{0}U_i-a_3 m_I ^{-\Delta t} U_i+a_4 K_{ij} ^{-\Delta t} U_j
\]

where $^{-\Delta t} U_i$ is obtained from \eqref{initialU}

\begin{equation}
^{-\Delta t}U_i=^{0}U_i-\Delta t ^{0}V_i+\dfrac{\Delta t^2}{2} ^{0}A_i
\label{initialU2}
\end{equation}

The initial acceleration is obtained after assuming homogeneous IC's;

\[
m_{I} ^{0}A_i+C_{ij} ^{0}V_j +K_{ij} ^{0}U_j=^{0}F_i
\]

therefore

\[
\begin{aligned}
m_{I} ^{0}A_i=\dfrac{^{0}F_i}{m_I}\\
^{-\Delta t} U_i=\dfrac{\Delta t^2}{2m_I}^{0}F_i
\end{aligned}
\]

Moreover, neglecting the damping effects on the prediction of $^{\Delta t}U_i$ yields;

\[
^{\Delta t} U_i=\dfrac{\Delta t^2}{2m_I}^{0}F_i
\]

\begin{algorithm}[H]
 \SetAlgoLined
 \KwData{Time span, Geometry, Material Paramters}
 \KwResult{Displacements, Velocity and Acceleration time histories }
 Initialize solution vectors ($j=1$)\;
 $^{0}U_i\longrightarrow ^{1}U_i=0$, $^{0}V_i=0$, $^{1}A_i=\dfrac{^{1}R_i}{m_I}$ \;
 Select $\Delta t$ and integration constants $a_1$,$a_2$, $a_3$, $a_4$\;
 Fix 1-st predicted value (let $j=2$)\;
 \[
^{\Delta t} U_i \longleftarrow \dfrac{\Delta t^2}{2m_I} ^{0}F_i\longleftrightarrow \left[^{2}U_i \longleftarrow\dfrac{\Delta t^2}{2m_I} ^{1}F_i \right]
 \]
Time Integration Phase\;
\While{$j \leq N$}{
\[
\begin{aligned}
^{j+1}F^I_i\longleftarrow& ^{j}F_i-a_1 K_{ij} ^{j}U_j+a_2 m_I ^{j}U_j-a_3 m_I ^{j-1} U_i+a_4 K_{ij} ^{j-1} U_j\\
 ^{j+1}U_i\longleftarrow&\dfrac{\Delta t^2}{2m_I} ^{0}F^I_i\\
 ^{j}A_i\longleftarrow&\dfrac{1}{\Delta t^2}\left(^{j-1}U_i-2^{j}U_i+^{j+1}U_i\right)\\
 ^{j}V_i\longleftarrow&\dfrac{1}{2\Delta t}\left(-^{j-1}U_i+^{j+1}U_i\right)\\
 j\longleftarrow&j+1
\end{aligned}
\]
}
\caption{Full Algorithm}
\end{algorithm}

\subsubsection{Nodal Assembler}
In the uncoupled explicit finite element formulation the equation solving process proceeds one degree of freedom at a time. This implies a different assembly process to the one used in an implicit algorithm where a formal coefficient matrix is assembled and inverted. Now the mass, damping and stiffness elemental matrices are used to obtain effective nodal forces at each degree of freedom. In summary the mesh is not covered in an element by element basis, but in a node by node basis. In the following algorithm we discuss this nodal assembly process where in order to solve the displacement at a given degree of freedom prior knowledge of the element contributing to the given node is necessary. In the nodal assembler algorithm the following arrays are needed.

\noindent
ILIST(): Stores the elements connected to the current node.\\
LPLIST():Stores the local position of the current node in each one of the elements of ILIST().\\
NIEL():Number of elements at the current node. This array is used to access ILIST().\\

\begin{algorithm}[H]
\SetAlgoLined
\KwData{Number of nodal points, number of elements, Model}
\KwResult{Displacements, Velocity and Acceleration time histories }
\While {$i \leq NUMNP$}
{
$K\leftarrow NIEL(i)$
%\while {$ik \leq 2$}{
%$jK \leftarrow ID(ik,i)$
%}
}
\caption{Nodal Assembler}
\end{algorithm}