% body
\section{Comparison with \cite{YBZ}\label{app:results-compare}}
%We turn our attention to \cite{YBZ}, a previously proposed high-order, kernel-independent singular quadrature method in \threed for complex geometries.
%Since \qbkix shares these characteristics, we will to compare the two approaches in this section.

To understand the performance of \cite{YBZ} and \qbkix and see the implications of this complexity difference in practice, we now compare the performance of \qbkix with that of \cite{YBZ} on several concrete numerical examples.
The metric we are interested is \textit{cost for a given relative error}.
Assuming the surface discretization is $O(N)$, we measure the \textit{cost} of a method as its total wall time during execution $T$ divided by the total wall time of an \fmm evaluation on the same $O(N)$ discretization, $T_\lbl{FMM}$.
By normalizing by the \fmm evaluation cost, we minimize the dependence of the cost on machine- and implementation-dependent parameters, such as clock speed, cache size, performance optimizations, etc.
We run the tests in this section on the sphere geometry shown in \cite[Figure 8-left]{morse2020robust}  and continue to focus on the singular quadrature scheme of \cite{YBZ} as described in \cite[Section 6.2]{morse2020robust}.
%This means that the cost of \qbkix will quickly overtake the near-singular scheme of \cite{YBZ} as the target accuracy increases.
%Moreover, using \qbkix for singular quadrature is adversarial in the sense of accuracy, since the extrapolation error is maximized for points on the surface.
%This means we are comparing the worst-case error of \qbkix with the average-case error of \cite{YBZ}.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% Cut starting here
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Complexity comparison}

The algorithm of \cite{YBZ} substantially differs from \qbkix in two main ways.
First, on-surface singular integral evaluation is computed in \cite{YBZ} by subtracting the inaccurate part of the \fmm-accelerated smooth quadrature rule using a partition-of-unity (\pou) function, then adding an accurately computed part singular integral close to singularity via polar quadrature.
Second, \cite{YBZ} sets more algorithms parameters \emph{a priori} rather than determining them adaptively.
Specific choices used in  \cite{YBZ} may be considered  optimal for the uniform volume point distribution described in \cref{sec:complexity_matvec}, but need to be adjusted based on additional analysis for other distribution types. Additionally, \cite{YBZ} has a trade-off between accuracy and complexity proportional to the \pou radius $d_P$, which \qbkix does not have.

The intermediate and far zone complexity estimates are similar for both \qbkix and \cite{YBZ}.
The near-zone complexity for the algorithm of \cite{YBZ} has an additional term of the form  $O(N d_P^2/\Lmx^2)$, where $d_P$ is the radius of the \pou function.
For simplicity, we use $\Lmx$ as a measure of surface sampling density as in \cref{sec:complexity_admissibility,sec:complexity_upsampling}, since $\Lmx$ and the $h$ from \cite{YBZ} differ by a constant.

The error of \cite{YBZ}'s singular evaluation is $O( d_P^{-2q-1}\Lmx^{2q})$, for an optimally chosen local quadrature rule.
We note that the factor $d_P^{-2q-1}$ is entirely an artifact of using a compactly supported \pou function to localize the singular integral computation.
As observed in \cite{YBZ}, to achieve optimal convergence  as the surface is refined, $d_P$ needs to decrease slower than $\Lmx$, i.e., slower than $N^{-1/2}$, under the assumptions on point distribution in $\Omega$ from \cref{sec:complexity_matvec}.
In \cite{YBZ}, $d_P = O(N^{-1/2(1+\gamma)})$ is suggested.
As a result, the overall complexity is $O(N^{1+\gamma})$ and grows faster than $N$.

By choosing $\gamma = \frac{1}{2}$, \cite{YBZ}'s final complexity
becomes $O(N^{3/2})$ in order to produce an error proportional to $N^{(-2q+1)/4}$.
In other words, the work needed for an error $\eps$ is proportional to $\eps^{-6/(2q-1)}$, which is asymptotically higher than \qbkix (with $\eps$ from \cref{sec:complexity_upsampling}).
On the other hand, our method has the disadvantage of requiring $p$ check point evaluations for every sample point in $\Nn$.
This requires an \fmm call that is $(m+p)$-times larger than \cite{YBZ}.
In common use cases, such as solving \cite[Equation 5]{morse2020robust} via \gmres, repeated \qbkix evaluations through a more expensive \fmm can require more work in practice for lower accuracy than \cite{YBZ}.

\subsection{Experimental comparison.}

To understand the performance of these two methods and see the implications of this complexity difference in practice, we now compare the performance of \qbkix with that of \cite{YBZ} on several concrete numerical examples.
The metric we are interested is \textit{cost for a given relative error}.
Assuming the surface discretization is $O(N)$, we measure the \textit{cost} of a method as its total wall time during execution $T$ divided by the total wall time of an \fmm evaluation on the same $O(N)$ discretization, $T_\lbl{FMM}$.
By normalizing by the \fmm evaluation cost, we minimize the dependence of the cost on machine- and implementation-dependent machine-dependent parameters, such as clock speed, cache size, performance optimizations, etc.

\paragraph{Comparison on $C^\infty$ surface of \cite{ying2004simple}}
An important contribution of \cite{YBZ} was the use of a $C^\infty$ surface representation, first introduced in \cite{ying2004simple}, allowing for exponential accuracy via the trapezoidal rule, and easy resampling for singular quadrature.
To fairly compare the two quadrature methods, we have implemented a modified version of \qbkix on the surface representation of \cite{ying2004simple}.
The algorithm of \cite[Section 3.1]{morse2020robust} has the following modifications:
(i) we discretize the vertex-centered patches of \cite{ying2004simple} with the tensor-product trapezoidal rule for compactly supported functions with spacing parameter $h$, as in \cite{YBZ}; (ii) the upsampled quadrature rule uses a trapezoidal rule with spacing $h/4$; (iii) density interpolation is computed with \fft's, as in \cite{YBZ}; the rest of the algorithm proceeds unchanged.
This essentially matches \cite[Section 3.1]{morse2020robust} but uses the discretization scheme of \cite{YBZ} instead of Clenshaw-Curtis.

For each of the tests in this section, we choose some initial spacing parameter $h_0$ to discretize the surface of \cite{ying2004simple} as in \cite{YBZ} and use the same $16\times$ upsampled grid to evaluate both \qbkix and \cite{YBZ}.
We apply the modified \qbkix algorithm and the scheme of \cite{YBZ} with spacing $h_0$ and compute the relative error and collect timing statistics.
We repeat this test with $h_0/2^i$ for $i=1,\hdots 4$ and plot the results.
This ensures that the smooth quadrature rule used by both methods have the same resolution.

We choose the floating partition of unity size in \cite{YBZ} to be $\sqrt{h}$ as in the original work.
As in the previous section, we choose the parameters $r$ and $R$ of \qbkix to be $O(\sqrt{h})$ to observe standard convergence behavior.
For both quadrature methods, we use a multipole order of $16$ for \pvfmm with at most 250 points in each leaf box and with the same initial spacing.

In \cref{fig:compare-const-density,fig:compare-solve}, we summarize our results for two test cases.
In \cref{fig:compare-const-density}, we evaluate \cite[Equation 8]{morse2020robust} using one-sided \qbkix and the singular quadrature method of \cite{YBZ} with the density $\phi=1$, in order to demonstrate their behavior without interaction with \gmres.
In \cref{fig:compare-solve}, we construct a boundary condition using \cite[Equation 25]{morse2020robust} with random charge values and solve \cite[Equation 5]{morse2020robust} using two-sided \qbkix and with the singular quadrature method of \cite{YBZ} inside of \gmres.
We then evaluate the singular integral at a finer discretization of the surface using either one-sided \qbkix or \cite{YBZ}, respectively.
From left to right, each plot details the total cost of each scheme, the cost of each subroutine for \qbkix (denoted \abbrev{HH}) and the singular quadrature scheme of \cite{YBZ} (denoted \abbrev{POU}), and the relative error as a function of $h$.
Each data point in the plots, from right to left, is the result of running the method on a discretization with spacing $h_0/2^i$ for $i=0,\hdots,4$.
We plot the cost of both schemes the cost of each algorithmic step as a function of their computed relative error. % computed by this discretization.
In each figure, we present results for a Laplace problem (top) and an elasticity problem (bottom), to highlight the difference in performance between scalar and vector kernels.

\begin{figure}[!htb]
  \centering
    \makebox[\textwidth][c]{ \includegraphics[width=1.\textwidth]{figs/comparison_cube_laplace_timing.pdf} }

    \makebox[\textwidth][c]{ \includegraphics[width=1.\textwidth]{figs/comparison_cube_const_den_navier_timing.pdf} }
%  \begin{minipage}{\textwidth}
%    \includegraphics[width=1.5\textwidth]{figs/comparison_cube_laplace_timing.pdf}
%  \end{minipage}\hfill
%  \begin{minipage}{\textwidth}
%    \includegraphics[width=\linewidth]{figs/comparison_cube_const_den_navier_timing.pdf}
%  \end{minipage}\hfill
  \mcaption{fig:compare-const-density}{ Comparison of \qbkix (HH) versus \cite{YBZ}  (POU) on the surface representation of \cite{ying2004simple} evaluating double-layer potential with $\phi=1$}{
      Laplace (top) and elasticity (bottom) problems solved on the sphere shown in \cite[Figure 8-left]{morse2020robust}.
From left to right, we plot the total cost of each scheme, the cost of each subroutine for \qbkix (blue) and the singular quadrature scheme of \cite{YBZ} (red), and the relative error as a function of $h$.
  The plots show the cost and relative error for $h_0 = .3$ representing the right-most data point and each point to the left corresponding to a spacing of $h_i = h_0/2^i$.
For the Laplace problem, we choose $r=.186\sqrt{h}$, $R=1.12\sqrt{h}$ and $p=6$ for \qbkix parameters; for the elasticity problem, we choose $r=.133\sqrt{h}$, $R=.8\sqrt{h}$ and $p=6$.
The initial spacing parameter is $h_0=.3$.
}
\end{figure}

\begin{figure}[!htb]
  \centering
  \hfill
    \makebox[\textwidth][c]{ \includegraphics[width=1.\textwidth]{figs/comparison_cube_solve_laplace_timing.pdf} }
    \makebox[\textwidth][c]{ \includegraphics[width=1.\textwidth]{figs/comparison_cube_navier_timing.pdf} }
%  \begin{minipage}{\textwidth}
%    \includegraphics[width=\linewidth]{figs/comparison_cube_solve_laplace_timing.pdf}
%  \end{minipage}\hfill
%  \begin{minipage}{\textwidth}
%    \includegraphics[width=\linewidth]{figs/comparison_cube_navier_timing.pdf}
%  \end{minipage}\hfill
  \mcaption{fig:compare-solve}{Comparison of \qbkix versus \cite{YBZ} on the surface representation of \cite{ying2004simple} solving via \gmres for $u_c$}{
    This figure's format is similar to \cref{fig:compare-const-density}.
    For the Laplace problem, we choose $r=.028\sqrt{h}$, $R=.172\sqrt{h}$ and $p=6$ for \qbkix parameters; for the elasticity problem, we choose $r=.042\sqrt{h}$, $R=.253\sqrt{h}$ and $p=6$.
The initial spacing parameter is $h_0=.3$.
  }
\end{figure}

As expected, the \qbkix total cost curves lie somewhere between 1 and 10, since the required \fmm evaluation is $(m+p)$-times larger than $N$.
This step is the dominant cost: the next most expensive step is density interpolation, which is two orders of magnitude faster.
Initially, the main cost of \cite{YBZ} is \fmm evaluation time, but eventually the local correction cost begins to dominant.
Note that the \qbkix and \cite{YBZ}-\fmm curves are not quite flat, due to the initial quadratic complexity of a shallow \fmm tree.

From \cref{fig:compare-const-density,fig:compare-solve}, we observe a higher convergence rate for \qbkix than \cite{YBZ}, except for the elasticity solve in \cref{fig:compare-solve}-bottom, where the methods perform about equally.
This allows the cost of \qbkix to decrease below \cite{YBZ} for errors less than $10^{-7}$ for Laplace problems.
More importantly, however, \cite{YBZ} outperforms \qbkix for elasticity problems for all tested discretizations, and also for low and moderate accuracy Laplace problems.
This is due to the greater cost of a vector \fmm evaluation compared to a scalar one: the $m+p$ factor saved in the \fmm evaluation of \cite{YBZ} can be accelerated more efficiently with the method's small dense linear algebra computations.
This means that a local singular quadrature method of \textit{worse} complexity can beat a global method, simply by virtue of reducing the \fmm size.
Moreover, our implementation of \cite{YBZ} is not highly optimized, so we can expect a well-engineered \pou singular quadrature implementation such as \cite{malhotra2019taylor} to widen this gap.
By noting the large difference between the \qbkix \fmm cost and the \qbkix density interpolation, we can reasonably infer that a local \qbkix scheme should narrow this gap and outperform \cite{YBZ}, assuming that this transition does not dramatically affect error convergence.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% Cut ending here
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\iffalse
%\paragraph{Comparison on approximate polynomial surfaces}
To compare the full \qbkix method with \cite{YBZ}, we fit polynomial patches to the $C^\infty$ surface of \cite{ying2004simple}, denoted $\Gamma_b$, to produce $\Gammah$ during the first step of \cref{sec:admissible_algo}.
We then apply the remaining geometry preprocessing algorithms of \cref{sec:admissible_algo} to $\Gammah$ to produce $\Pcoarse$.
After producing $\Pfine$ with two levels of uniform upsampling, we solve \cref{eq:int-eq} with two-sided \qbkix on $\Gammah$ and evaluate the solution on the boundary with one-sided \qbkix.
We then solve for the solution to \cref{eq:int-eq} on $\Gamma_b$ using \cite{YBZ}.

For each of the tests in this section, we choose some initial spacing parameter $h_0$ to discretize the surface of \cite{ying2004simple} as in \cite{YBZ} and use the same $16\times$ upsampled grid and floating partition of unity size of size $O(\sqrt{h})$ as in \cite{YBZ}.
We apply \qbkix to $\Gammah$ and the scheme of \cite{YBZ} to $\Gamma_b$ with spacing $h_0$ and compute the relative error and collect timing statistics for the important components of each method.
We repeat this test with $h_0/2^i$ for \cite{YBZ} and $i$ levels of subdivision in $\Pcoarse$ for $i=1,\hdots 4$ and plot the results.

As in the previous section, we choose the parameters $r$ and $R$ of \qbkix to be $O(\sqrt{L})$ to observe standard convergence behavior.
For both quadrature methods, we use a multipole order of $16$ for \pvfmm with at most 250 points in each leaf box.
\begin{figure}[!htb]
  \centering
    \makebox[\textwidth][c]{ \includegraphics[width=1.\textwidth]{figs/comparison_face_map_vs_blendsurf_cube_laplace_timing.pdf} }
    \makebox[\textwidth][c]{ \includegraphics[width=1.\textwidth]{figs/comparison_face_map_vs_blendsurf_cube_navier_timing.pdf} }
%  \hfill
%  \begin{minipage}{\textwidth}
%    \includegraphics[width=\linewidth]{figs/comparison_face_map_vs_blendsurf_cube_laplace_timing.pdf}
%  \end{minipage}\hfill
%  \begin{minipage}{\textwidth}
%    \includegraphics[width=\linewidth]{figs/comparison_face_map_vs_blendsurf_cube_navier_timing.pdf}
%  \end{minipage}\hfill
    \mcaption{fig:compare-solve-surface}{Comparison of \qbkix on polynomial patches (HH) versus \cite{YBZ} on the surface representation of \cite{ying2004simple} (POU) solving via \gmres for $u_c$}{
    Laplace (top) and elasticity (bottom) problems solved on the sphere shown in \cref{fig:greens-id-test-cases}.
From left to right, we plot the total cost of each scheme, the cost of each subroutine for \qbkix (blue) and the singular quadrature scheme of \cite{YBZ} (red), and the relative error as a function of $h$.
    We plot error convergence of \cite{YBZ} as a function of $h$ and \qbkix as a function of $L$, due to the distinct discretizations.
    For \qbkix parameters, we choose $r=.013\sqrt{L}$, $R=.075\sqrt{L}$ for the Laplace problem; for the elasticity problem, we choose $r=.013\sqrt{L}$, $R=.08\sqrt{L}$. We choose $p=6$ and $q=15$ for both problems.
    For \cite{YBZ} the spacing is $h_0=.35$.
  }
\end{figure}
The results are shown in \cref{fig:compare-solve-surface}.
From left to right, each plot details the total cost of each scheme, the cost of each subroutine for \qbkix (denoted \abbrev{HH}) and the singular quadrature scheme of \cite{YBZ} (denoted \abbrev{POU}), and the relative error as a function of $h$ and $L$, respectively.
Each data point in the plots, from right to left, is the result of $4\times $ finer sampling of the surfaces.
We plot the cost of both schemes the cost of each algorithmic step as a function of their computed relative error.
In each figure, we present results for a Laplace problem (top) and an elasticity problem (bottom), to highlight the difference in performance between scalar and vector kernels.

From \cref{fig:compare-solve-surface}, we observe a slightly higher convergence rate for \qbkix than \cite{YBZ}, which is consistent with original results.
More importantly, however, \cite{YBZ} outperforms \qbkix for elasticity problems for all tested discretizations, and also for low and moderate accuracy Laplace problems.
This is due to the greater cost of a vector \fmm evaluation compared to a scalar one: the $m+p$ factor saved in the \fmm evaluation of \cite{YBZ} can be accelerated more efficiently with the method's small dense linear algebra computations.
This means that a local singular quadrature method of \textit{worse} complexity can beat a global method, simply by virtue of reducing the \fmm size; we observe that the \fmm evaluation in \cref{fig:compare-solve-surface} accounts for at least 95\% of the \qbkix cost.
Moreover, our implementation of \cite{YBZ} is not highly optimized, so we can expect a well-engineered \pou singular quadrature implementation such as \cite{malhotra2019taylor} to widen this gap.
By noting the large difference between the \qbkix \fmm cost and the \qbkix density interpolation, we can reasonably infer that a local \qbkix scheme should narrow this gap and outperform \cite{YBZ}, assuming that this transition does not dramatically affect error convergence.

%\qbkix is more efficient in the high-accuracy regime for Laplace problems, but \cite{YBZ} is more efficient for low-accuracy Laplace and elasticity problems.
%However, main difference from the previous section is that the crossover point in performance appears to be larger; \qbkix becomes more efficient than \cite{YBZ} around $10^{-5}$ and the gap between \qbkix and \cite{YBZ} for elasticity is less dramatic at $10^{-8}$.
%We attribute this improvement to the more efficient Clenshaw-Curtis discretization of \qbkix compared to the overlapping trapezoidal discretization of \cite{YBZ}.
%This is further supporting evidence that a local \qbkix implementation should surpass \cite{YBZ}.
\subsection{Requested target precision vs. computed accuracy}
\begin{figure}[!htb]
\begin{tikzpicture}
    \begin{loglogaxis}[
        xlabel=$\err{target}$,
        ylabel= $\infty$-norm relative error,
        ylabel near ticks,
        xlabel near ticks,
        label style={font=\scriptsize},
        tick label style={font=\scriptsize},
        width=.333\linewidth%7cm,height=3cm
    ]
    \addplot[smooth,mark=*,blue] plot coordinates {
        (1e-4,0.0003143683435301948)
        (1e-5,6.031924968205908e-05)
        (1e-6,4.103514547039301e-06)
        (1e-7,6.614064613404895e-07)
        (1e-8,1.122384348756594e-07)
    };
        \addplot[domain=1e-8:1e-4,dotted,thick]{x};

    \end{loglogaxis}
    \end{tikzpicture}
    \begin{tikzpicture}
    \begin{semilogxaxis}[
        xlabel=$\err{target}$,
        ylabel= Target points/second/core,
        ylabel near ticks,
        xlabel near ticks,
        label style={font=\scriptsize},
        tick label style={font=\scriptsize},
        width=.333\linewidth%7cm,height=3cm
    ]
    \addplot[smooth,mark=*,blue] plot coordinates {
        (1e-4,2077)
        (1e-5,1620)
        (1e-6,772)
        (1e-7,400)
        (1e-8,222)
    };
    \end{semilogxaxis}
    \end{tikzpicture}
    \begin{tikzpicture}
    \begin{loglogaxis}[
        xlabel=$\err{target}$,
        ylabel= Number of patches,
        ylabel near ticks,
        xlabel near ticks,
        label style={font=\scriptsize},
        tick label style={font=\scriptsize},
        legend style={font=\tiny},
         legend image post style={scale=0.5},
        width=.333\linewidth%7cm,height=3cm
    ]
    \addplot[smooth,mark=*,blue] plot coordinates {
        (1e-4,128)
        (1e-5,128)
        (1e-6,128)
        (1e-7,128)
        (1e-8,128)
    };
    \addlegendentry{$|\Pcoarse|$}
    \addplot[smooth,mark=*,red] plot coordinates {
        (1e-4,5408)
        (1e-5,8288)
        (1e-6,28256)
        (1e-7,60032)
        (1e-8,113936)
    };
    \addlegendentry{$|\Pfine|$}
    \end{loglogaxis}
    \end{tikzpicture}
        \mcaption{fig:full-algo-perf}{Performance of full algorithm}{
            Left: $\infty$-norm relative error in singular integral vs requested target accuracy (blue).
        The dotted line is the ideal behavior $y=x$.
        Middle: Performance in terms of target points evaluated per second per core with \qbkix.
        Right: Number of patches in $\Pcoarse$ and $\Pfine$ computed by the preprocessing algorithms.}
\end{figure}

In this section, we study the performance of the full algorithm outlined in \cref{sec:algo}.
We test \qbkix on the torus domain shown in \cref{fig:greens-id-test-cases}-right.
We choose a reference solution of the form of \cref{eq:point-charge-solution} with a single point charge located at the origin, in the middle of the hole of the torus.
We solve the integral equation with two-sided \qbkix and evaluate the singular integral on a distinct discretization with one-sided \qbkix.
We choose $q=20$, $p=6$ and $a=b/6$.
We select various values for $\err{target}$ using the plot in \cref{fig:extrap-err-p6} to choose $b$ to ensure sufficiently accurate extrapolation.
We plot the results of our tests in \cref{fig:full-algo-perf}.

We see in \cref{fig:full-algo-perf}-left that we are consistently close to the requested target precision.
We see a decline in target points per second per core as accuracy increases in \cref{fig:full-algo-perf}-middle.
This is explained by \cref{fig:full-algo-perf}-right, which shows an increase in the size $\Pfine$ as $\Pcoarse$ remains a fixed size.
The initial 128 patches in $\Pcoarse$ are enough to resolve the boundary condition and $\Gamma$, but we need greater quadrature accuracy for lower values of $\err{target}$ .
Decreasing the number of points in passed to the \fmm, i.e., decreasing the size of $\Pfine$, is the main way to improve performance of our method.
This is further indication that a local version of \qbkix will outperform a global approach.

\subsection{Full algorithm on interlocking torii\label{sec:results-torii}}
We now demonstrate the full algorithm pipeline on an exterior Laplace problem, whose boundary is defined by four interlocking torii shown in \cref{fig:torii}.
The domain boundary is contained in the box $[-3.8, 2.4] \times [-1.1, 1.1] \times [-1,1]$.
The shortest distance between two adjacent torii is less than 10\% of a polynomial patch length defining the boundary.
We again use a boundary condition of the form \cref{eq:point-charge-solution} with a single point charge located at $(0,.03,.875)$, inside the upper half of the second torus from the right in \cref{fig:torii}.
This problem is challenging due to the nearly touching geometry of the torii, along with the singularity placed close to the boundary.
We run the admissibility and adaptive upsampling algorithms outlined in \cref{sec:algo}, solve \cref{eq:int-eq} using two-sided \qbkix, and evaluate the solution on the boundary using one-sided \qbkix.
The absolute error in the $\infty$-norm of the singular evaluation is plotted on the boundary surface.

\begin{figure}[!htb]
  \centering
  \hfill
  \begin{minipage}{.65\textwidth}
    \includegraphics[width=\linewidth]{figs/interlocking_torus_error.png}
  \end{minipage}\hfill
  \begin{minipage}{.35\textwidth}
    \includegraphics[width=\linewidth]{figs/interlocking_torus_error_cross_section.png}
  \end{minipage}\hfill
  \mcaption{fig:torii}{Absolute error of \gmres solve via \qbkix on interlocking torii}{Left: The admissible set of 1128 patches in $\Pcoarse$ used to solve \cref{eq:int-eq} is shown (black lines denote patch boundaries).
  The point charge generated the boundary condition is located within the second torus from the right.
    Right: a cross-section of the torii geometry through the $xz$-plane, showing the second torus from the right and the location of the singularity (green point).
}
\end{figure}
Using $a=.1$, $b=.025$, $p=6$ and $q=20$, we achieve a maximum pointwise error of $1.29\times 10^{-5}$.
\gmres was able to reduce the residual by a factor of $10^{-13}$ over 109 iterations.
There are 288768 quadrature points in the coarse discretization, 18235392 quadrature points in the fine discretization, and 3465216 check points used in the two-sided \qbkix evaluation inside \gmres.
We evaluate the solved density at 451200 points on the boundary with one-sided \qbkix to produce the render in \cref{fig:torii}.
On a machine with two Intel Xeon E-2690v2 3.0GHz \abbrev{CPU}'s, each with 10 cores, and 100 \abbrev{GB} of \abbrev{RAM},
the \gmres solve and interior evaluation required 5.7 hours and can evaluate the singular integral at a rate of 1709 target points per second per core.

\subsection{Solution on complex geometry\label{sec:results-blood-vessel}}

\begin{figure}[!htb]
  \centering
  \setlength\figureheight{1.9in}
  \setlength\figurewidth{2.1in}
  \hfill
  \begin{minipage}{.7\textwidth}
    \includegraphics[width=\linewidth]{figs/largest_vessel_section.png}
  \end{minipage}\hfill
  \mcaption{fig:vessel}{Absolute error of \gmres solve via \qbkix on complex blood vessel geometry used in \cite{lu2019scalable}}{The blood vessel uses 40,960 8th order polynomial patches (black edges denote patch boundaries). The geometry is admissible by construction.
    The point charge is located on left side of the figure (green)}
\end{figure}

We have demonstrated in \cite{lu2019scalable} a parallel implementation of \cref{sec:singular-eval}, applied to simulating red blood cell flows.
The surface geometry of the blood vessel shown in \cref{fig:vessel} is complex, with rapidly varying curvatures and geometric distortions due to singular vertices in the surface mesh.
Since the surface is admissible, we are able to apply parallel \qbkix directly without geometric preprocessing to solve an interior Dirichlet Stokes problem.
We use $a=.125$, $b=.125$, $p=6$ and $q=16$ as simulation parameters.

Using 32 machines each with twenty 2.6 Ghz cores with 100\abbrev{GB} of \abbrev{RAM}, we achieve a maximum pointwise error of $3\times 10^{-6}$ when solving a Stokes problem with constant density.
We then place a random vector point charge two patch lengths away (relative to the patches in $\Pcoarse$) from the domain boundary (on the left side of \cref{fig:vessel}, solve \cref{eq:int-eq} using two-sided \qbkix, and evaluate the solution on the boundary using one-sided \qbkix.
The absolute error in the $\infty$-norm of the singular evaluation is plotted on the boundary surface.
There are 10,485,760 quadrature points in the coarse discretization, 167,772,160 quadrature points in the fine discretization, and 125,829,120 check points used in the two-sided \qbkix evaluation inside \gmres.
We evaluate the solved density at 209,715,200 points on the boundary with one-sided \qbkix to produce the render in \cref{fig:torii}.
We achieve a maximum pointwise error of $1.8\times 10^{-2}$ and can evaluate the singular integral at rate of 3529 target points per second per core.
%The \gmres solve with two-sided \qbkix converged to \note[MJM]{X} in \note[MJM]{Y} iterations in \note[MJM]{Z} hours.
\fi


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[english]{babel} % for hyphenation dictionary
%\setdefaultlanguage{english} % polyglossia command for use with XeTeX / LuaTeX
\usepackage[margin=1in]{geometry} % handle page geometry

% import packages and commands
\input{../header2.tex}

% title information
\title{Phys 221A -- Quantum Mechanics -- Lec11}
\author{UCLA, Fall 2014}
\date{\formatdate{10}{11}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\maketitle

\section{More on Path Integrals}

Aside: Turns out Yaroslav was wrong about Feynman and Schwinger being perfect prodigies. Feynman wanted to go to Columbia but couldn't get in, went to MIT instead. But they both ended up sharing the Nobel prize, which is funny? Anyway...

So the propagator $K(\v r, t; \v r', t')$ basically tells us all the information we want to know about the system, since
\begin{eqn}
\psi(\v r, t) = \int \dif^3{r'} K(\v r, t; \v r, t') \psi(\v r', t').
\end{eqn}
Last time we started breaking up the propagator into multiple integrals
\begin{eqn}
K(\v r, t; \v r', t') = \int \dif^3{r''} \cdots \dif^3{r^{(n)}} \braket{\v r,t}{\v r^{(n)},t^{(n)}} \cdots \braket{\v r'', t''}{\v r', t'}
\end{eqn}
for $t' < t'' < \dots < t^{(n)} < t$. We will take
\begin{eqn}
t^{(\ell+1)} - t^{(\ell)} = \frac{t-t'}{n} \equiv \delta,
\end{eqn}
where for convenience we denote $t^{(n+1)} = t$. If we write
\begin{eqn}
K_\ell = \braket{\v r^{(\ell+1)}, t^{(\ell+1)}}{\v r^{(\ell)}, t^{(\ell)}}
\end{eqn}
then we can think of the propagator as an integral
\begin{eqn}
K(\v r, t; \v r', t') = \int \mathcal{D}[\v r] I[\v r], \qquad
I = K_1 K_2 \cdots K_n.
\end{eqn}
Furthermore note that when $n$ is large, we can make the approximation
\begin{eqn}
\vd r_\ell = \frac{\v r^{(\ell+1)} - \v r^{(\ell)}}{\delta}.
\end{eqn}

Recall the propagator for a free particle, which we derived last time (in 1D, here in 3D),
\begin{eqn}
K_0(\v r, t; \v r', t') = \left( \frac{m}{2\pi i \hbar (t-t')} \right)^{3/2} \exp \left[ \frac{im(\v r - \v r')^2}{2\hbar (t-t')} \right].
\end{eqn}
We will use this to derive a general expression for the path integral. For the general propagator, we have
\begin{eqn}
K(\v r, t; \v r', t') = \matrixel{\v r}{U(t,t')}{\v r'},
\end{eqn}
where the time evolution operator is given by
\begin{eqn}
U(t,t') = e^{-iH(t-t') / \hbar} = \exp \cbr{ -\frac{i}{\hbar} \left[ \frac{p^2}{2m} (t-t') + V(\v r) (t-t') \right]}.
\end{eqn}
Making the appropriate approximations (skipping a few steps here), we find
\begin{eqn}
K(\v r, t; \v r', t') \approx K_0(\v r, t; \v r', t') e^{-\frac{i}{\hbar} V(\v r) (t-t')}.
\end{eqn}
In the limit $n \rightarrow \infty$ we have
\begin{eqn}
K_\ell \approx \left( \frac{m}{2\pi i \hbar \delta} \right)^{3/2} \exp \cbr{ \frac{i}{\hbar} \left[ \frac{1}{2} m \vd r_\ell^2 - V(\bar{\v r}_\ell, \bar t_\ell) \right] } \delta
	= \left( \frac{m}{2\pi i \hbar \delta} \right)^{3/2} e^{\frac{i}{\hbar} S_\ell}.
\end{eqn}
where
\begin{eqn}
\bar{\v r}_\ell = \frac{1}{2} (\v r^{(\ell+1)} + \v r^{(\ell)}), \qquad
\bar t_\ell = \frac{1}{2} (t^{(\ell+1)} + t^{\ell}).
\end{eqn}
Then in the limit $n \rightarrow \infty$ the full propagator becomes
\begin{align}
K(\v r, t; \v r', t') &= \lim_{n \rightarrow \infty} \int \dif^3{r''} \cdot \dif^3{r^{(n)}} \, K_1 \cdots K_n \\
	&= \lim_{n \rightarrow \infty} \int \left( \frac{m}{2\pi i \hbar \delta} \right)^{3n/2} \dif^3{r''} \cdots \dif^3{r^{(n)}} \, e^{(i / \hbar) (S_1 + \dots + S_n)} \\
	&= \int_{\v r(t)}^{\v r'(t')} \mathcal{D} [\v r(t)] e^{i S[\v r(t)] / \hbar},
\end{align}
where our new measure is
\begin{eqn}
\mathcal{D}[\v r(t)] = \lim_{n \rightarrow \infty} \left( \frac{m}{2\pi i \hbar \delta} \right)^{3n/2} \dif^3{r''} \cdots \dif^3{r^{(n)}}
\end{eqn}
and the full action is just
\begin{eqn}
S[\v r(t)] = \sum_{n=0}^\infty S_n = \int_t^{t'} \dif{t''} L(\v r, \vd r, t''), \qquad
L(\v r, \vd r, t) = \frac{1}{2} m \vd r^2 - V(\v r, t).
\end{eqn}

Classically, the principle of least action (really extremal action), also called Hamilton's principle, tells us that the classical trajectory is the one which extremizes the action. Quantum mechanically, we find a similar principle holds, only instead we need to sum over all paths, and we find that paths nearest the classical trajectory hold the greatest weight in this sum. Since $\hbar$ is so darn small, for macroscopic events we find that the nearly-classical paths interfere constructively and we just reproduce Hamilton's principle. In fact, one can show that if we take $\hbar \rightarrow 0$, we precisely reproduce classical mechanics because all non-classical paths interfere destructively. It's only for very small-scale events that non-classical paths becomes important.

When $\hbar$ is small, we can employ the saddle-point approximation or SPA as follows. Suppose we have an integral
\begin{eqn}
I(h) = \int_{-\infty}^\infty \dif{x} \, e^{i x^2 / h} f(x)
\end{eqn}
with some function $f(x)$. When $h \rightarrow 0$ we can use the limit
\begin{eqn}
\lim_{h \rightarrow 0} \frac{e^{i x^2 / h}}{\sqrt{i\pi \hbar}} = \delta(x)
\end{eqn}
to show that
\begin{eqn}
\lim_{h \rightarrow 0} I(h) = f(0).
\end{eqn}

\end{document}


\section{Simulations}

A comprehensive set of subroutines required to compute and evaluate the transfer
functions    outlined    in    this    report    can     be     obtained    from
GitHub\cite{ref:TheComet93}.

The subroutines are located in the folder \textit{matlab/mfunctions}, along with
various utility functions. The folder must be appended to MATLAB's path like so:

\begin{lstlisting}
addpath([pwd,'/mfunctions']);
\end{lstlisting}

This section provides a short overview  of  what  each  subroutine does and how.

\subsubsection*{Pre-processing}

Typically, the  input  data is never perfect. It might contain noise or it might
not have equispaced time values. The function \textit{preprocess\_curve} returns
equispaced time values and smooths the signal.

\begin{lstlisting}
% Smooth input data and generate equispaced
% time vector
[x, y] = preprocess_curve(xr, yr);
\end{lstlisting}

This is achieved  by  using a combination of MATLAB's \textit{smooth()} function
and a custom sliding average  function  to  smooth the beginning and ends of the
signal.

\subsubsection*{Characterising the curve}

With the data prepared (preprocessed), it is now possible to calculate $T_u/T_g$
or $t_{10}$, $t_{50}$, $t_{90}$, depending on  what  you  wish  to  do next. The
function \textit{characterise\_curve()}  can handle both cases for you, like so:

\begin{lstlisting}
% Characterises the curve, either using
% Hudzovic's method or Sani's method.
[Tu, Tg] = characterise_curve(x, y);
[t10, t50, t90] = characterise_curve(x, y);
\end{lstlisting}

Determining $T_u$, $T_g$ is achieved by calculating  the  derivative of the data
to  find  the  point  of  inflection.  The maximum and minimum of the signal  is
determined  by  taking  the  value  of  the  first  and  last  element  in  $y$.

Determining   $t_{10}$,  $t_{50}$,  $t_{90}$  is  achieved  by  using   MATLAB's
\textit{spline()}  function for higher accuracy and  intersecting  the  data  at
10\%, 50\% and 90\% amplitude.  In  the  case of noisy input data, if the spline
fails, the fallback method is to simply find the nearest point. The minimum  and
maximum of the  input  signal  is  again  determined by using the first and last
element in $y$.

Because L. Sani's approach heavily relies  on  accurately  determining the start
and  end  values  of  the input step response function, and it  was  found  that
noisier input  signals  would  significantly  throw  off  correctly  determining
$t_{10}$,  $t_{50}$,  $t_{90}$, it is possible to override the  default  min/max
values by passing in a third argument:

\begin{lstlisting}
% Override Sani's min/max.
ymax = 37;  % Step function stops at 37
ymin = 15;  % Step function starts at 15
[t10, t50, t90] = characterise_curve(x, y, [ymin, ymax]);
\end{lstlisting}

The third argument is only valid for L. Sani's method.

In the case  of  noisy  signals, it is usually better to determine the beginning
and end values of the step response function manually.

\subsubsection*{Calculating T, r and n}

With  $T_u$,  $T_g$  or  $t_{10}$,  $t_{50}$,  $t_{90}$  determined,  the  three
constants $T$, $r$ and $n$ can be calculated using  either  P.  Hudzovic's or L.
Sani's  transfer function (equations \ref{eq:hudzovic}  or  \ref{eq:sani}).  For
this, the two functions  \textit{sani\_lookup()} and \textit{hudzovic\_lookup()}
may be used.

Both  of these functions accept either $T_u/T_g$ or $t_{10}$, $t_{50}$, $t_{90}$
as parameters. Depending  on  which  one  you  choose  to use, the function will
perform a different lookup:

\begin{lstlisting}
% Hudzovic method, Sani method, and their permutations
[T, r, n] = hudzovic_lookup(Tu, Tg);
[T, r, n] = hudzovic_lookup(t10, t50, t90);
[T, r, n] = sani_lookup(t10, t50, t90);
[T, r, n] = sani_lookup(Tu, Tg);
\end{lstlisting}

Note how it's  also  possible  to  use the $t_{10}$, $t_{50}$, $t_{90}$ approach
with  Hudzovic's method,  and  similarly,  use  $T_u/T_g$  with  Sani's  method.

When  calling  these  functions  for  the  first time, they will spend some time
generating  the  lookup  curves  discussed in the theory section. This can  take
about a minute. They are saved to  disk  and  loaded  again if available, so all
subsequent calls will be fast.

The  \textit{sani\_lookup()}  function  will  use  the   interpolation  formulae
(equation  \ref{eq:sani_interpolation})   when   passing   $t_{10}$,   $t_{50}$,
$t_{90}$.   All  other  combinations   will   have   to   use   lookup   curves.

\subsubsection*{Calculating the Transfer Function}

Once    $T$,    $r$    and    $n$    are    obtained,    the    two    functions
\textit{hudzovic\_transfer\_function()}  and \textit{sani\_transfer\_function()}
will help  convert  those  three  constants into a continuous transfer function:

\begin{lstlisting}
% Calculating the transfer functions
G_hudzovic = hudzovic_transfer_function(T, r, n);
G_sani = sani_transfer_function(T, r, n);
\end{lstlisting}

Of  course,  if  the  parameters  $T$,  $r$   and   $n$  were  determined  using
\textit{sani\_lookup()} then  the  function  \textit{sani\_transfer\_function()}
must be used. The same is true for Hudzovic.

Once the transfer function is obtained,  one can view the step response by using
\textit{step()}:

\begin{lstlisting}
% Plot step response
[g, t] = step(G_hudzovic);
plot(t, g);
\end{lstlisting}

\subsubsection*{Fitting}

So  far,  the  typical  work-flow  for  calculating  the transfer function looks
somewhat like the following code:

\begin{lstlisting}
% Hudzovic, Tu/Tg
[Tu, Tg] = characterise_curve(x, y);
[T, r, order] = hudzovic_lookup(Tu, Tg);
G = hudzovic_transfer_function(T, r, order);
\end{lstlisting}

To  further  refine  the result, it is possible to perform a least squares curve
fit  on  a  result  obtained  by  either  method  using   the  \textbf{original}
(non-smoothed)   data.    This    can    be    achieved   with   the   functions
\textit{hudzovic\_fit()} and \textit{sani\_fit()}.

\begin{lstlisting}
% Hudzovic, Tu/Tg, with fitting
[Tu, Tg] = characterise_curve(x, y);
[T, r, n] = hudzovic_lookup(Tu, Tg);
[T, r] = hudzovic_fit(T, r, n, xr, yr);
G = hudzovic_transfer_function(T, r, order);
\end{lstlisting}

Here,  \textit{xr},  \textit{yr}  contain  the  ``raw''  data,  and  \textit{x},
\textit{y} contain the pre-processed (smoothed) data.

Similarly, it is  possible  to  further  refine  a  result obtained by L. Sani's
method using the function \textit{sani\_fit()}:

\begin{lstlisting}
% Hudzovic, Tu/Tg, with fitting
[t10, t50, t90] = characterise_curve(x, y);
[T, r, n] = sani_lookup(t10, t50, t90);
[T, r] = sani_fit(T, r, n, xr, yr);
G = sani_transfer_function(T, r, order);
\end{lstlisting}

The fit is currently not  able  to determine the required order by itself, so it
is necessary to first call \textit{characterise\_curve()} on  the  smoothed data
to retrieve the order. A nice  side  effect  of  doing this is it also gives you
good starting values for $T$ and $r$. This  avoids  the  possibility  of falling
into a local minimum while fitting the data.

\input{sections/simulations/image_import}


\section{Analysis on request textual content}
In this section, we perform analysis on the two text-based attributes in our dataset, i.e. \textit{request\_title} and \textit{request\_text\_edit\_aware}. These attributes contain the title and textual body of each request. Most of decision will be based upon the contents of these two attributes.

\subsection{Textual content preprocessing}

In order to perform analysis, we need to do preprocessing steps on these two attributes. First, we observed that the title of a request somewhat summaries the content in the request text, thus, analyze these two text attributes separately it is a good idea to combine title and the request text in to a single text document. Next, we tokenized a document into words using simple regular expression to find words: \verb|[a-z][a-z'\-_]+[a-z])||\verb|([a-z]+)|. In language, there are several stop words which are words that have meaning too general e.g.\texttt{you, me}, etc. These words ofter do not have predictive power because they occur in both successful and unsuccessful requests. Thus, after tokenizing words, we removed stop words from documents. After stop words removal step, there are 12930 distinct words occur in the corpus, 6358 distinct words only occur in successful requests, and 10823 distinct words only occur in unsuccessful requests.

\subsection{Empirical analysis on words}

When looking at request text, we are often interested in keywords or topics that are associated with successful requests and unsuccessful requests. These keywords can give some insights about which kind of requests often receive pizza and which kind of requests do not. We defined ``positive'' keywords as the words that often occur in successful request but rarely occur in the unsuccessful requests, and ``negative'' keywords as the words that often occur in unsuccessful request but rarely occur in the successful requests. To find top ``positive'' words, we ranked all words based on \textit{constract score}:

\begin{equation}
	\phi = \frac{p}{n}(p-n)
\end{equation}

where $p$ and $n$ are correspondingly the counts of successful and unsuccessful requests containing the word. After compute \textit{constract score}, we can select top-$k$ words based on the ranking of scores. The process of finding top ``negative'' words as similar but in the \textit{constract score} formula we exchange $n$ and $p$. Figure \ref{top} shows top-30 ``positive'' words and ``negative'' words. From the figures, there are several interesting points we can see. Top ``positive'' words contains several words related to children topic like \texttt{\{disney, daughter's, babies\}}, location topic like \texttt{\{toledo, wellington, asian\}}. This suggests that requests mentioned about those topics tend to be successful. Top ``negative'' words contains several words related to money topic like \texttt{\{money, penny, penniless\}}, relationship topic like \texttt{\{love, friend, girlfriends, coworkers\}}, bad activities like \texttt{\{frunk, smoking, broke, dumped\}}, and locations like \texttt{\{island, alaska\}}. This suggests that requests mentioned about those topics tend to be unsuccessful.

\begin{figure}
	\centering
	\includegraphics[width=0.40\textwidth]{data/positive}
	\includegraphics[width=0.40\textwidth]{data/negative}
	\caption{Top ``positive'' words and top ``negative'' words}
	\label{top}
\end{figure}

\subsection{Outcome prediction}
In second analysis, we perform a classification task using text attributes. For the prediction model, we built the same logistic regression model (maximum entropy classifier) as the model for requester information attributes. We also used the same cross validation settings and folds as the previous classification experiment. Table \ref{confusion2} shows a confusion matrix for the same fold as the previous experiment but now using text attributes. In the result, the overall $F$-measure are \textbf{0.31}. The result now is much better than using requester information attributes. The classifier now can classify both successful and unsuccessful request.

 \begin{table}[]
	\centering
	\caption{The confusion matrix for a fold in the cross validation (using text attributes)}
	\label{confusion2}
	\begin{tabular}{|c|c|c|c|}
		\hline
		\multicolumn{2}{|c|}{\multirow{2}{*}{}}                & \multicolumn{2}{c|}{Actual}                            \\ \cline{3-4}
		\multicolumn{2}{|c|}{}                                 & \multicolumn{1}{c|}{TRUE} & \multicolumn{1}{c|}{FALSE} \\ \hline
		\multicolumn{1}{|c|}{\multirow{2}{*}{Predict}} & TRUE  & 27                         & 79                          \\ \cline{2-4}
		\multicolumn{1}{|c|}{}                         & FALSE & 33                        & 265                        \\ \hline
	\end{tabular}
\end{table}


%  qd.tex
%
%  This work was supported by the Director, Office of Science, Division
%  of Mathematical, Information, and Computational Sciences of the
%  U.S. Department of Energy under contract number DE-AC03-76SF00098.
%
%  (C) September 2000-2004
\documentclass[11pt]{article}
\usepackage{graphicx}
\usepackage{amsthm}

\pagestyle{plain}

\setlength{\textwidth}{6.5in}
\setlength{\textheight}{8.5in}
\setlength{\topmargin}{-0.2in}
\setlength{\oddsidemargin}{0.0in}
\setlength{\evensidemargin}{0.0in}

% All thm/def/lem/etc... have same numbering
\newtheorem{thm}{Theorem}
\newtheorem{defn}[thm]{Definition}
\newtheorem{lem}[thm]{Lemma}
\newtheorem{prop}[thm]{Proposition}

\theoremstyle{definition}
\newtheorem{alg}[thm]{Algorithm}

\newcommand{\round}{\mathrm{round}}
\newcommand{\fl}{\mathrm{fl}}
\newcommand{\err}{\mathrm{err}}
\newcommand{\ulp}{\mathrm{ulp}}
\newcommand{\hi}{\mathrm{hi}}
\newcommand{\lo}{\mathrm{lo}}
\newcommand{\eps}{\varepsilon}
\newcommand{\epsqd}{\varepsilon_\mathrm{qd}}

\title{Library for Double-Double and Quad-Double Arithmetic\footnotemark[1]}
\author{Yozo Hida\footnotemark[2] \and Xiaoye S. Li\footnotemark[3]
  \and David H. Bailey\footnotemark[3]}
\date{\today}

\begin{document}
\maketitle

% change foot note symbols
\renewcommand{\thefootnote}{\fnsymbol{footnote}}

\footnotetext[1]{This research was supported by the Director,
  Office of Science, Division of Mathematical, Information, and
  Computational Sciences of the U.S. Department of Energy under
  contract number DE-AC03-76SF00098.}
\footnotetext[2]{Computer Science Division, University of California,
  Berkeley, CA 94720 ({\tt yozo@cs.berkeley.edu}).}
\footnotetext[3]{NERSC, Lawrence Berkeley National Laboratory, 1 Cycloton Rd,
  Berkeley, CA 94720 ({\tt xiaoye@nersc.gov}, {\tt dhbailey@lbl.gov}).}

% change footnote symbols back to numbers
\renewcommand{\thefootnote}{\arabic{footnote}}

\vspace{1cm}
\begin{abstract}
  A double-double number is an unevaluated sum of two IEEE double
  precision numbers, capable of representing at least 106 bits of
  significand.   Similarly, a quad-double number is an unevaluated sum of
  four IEEE double precision numbers, capable of representing at least
  212 bits of significand.  Algorithms for various arithmetic operations
  (including the four basic operations and various algebraic and
  transcendental operations) are presented. A C++ implementation of
  these algorithms is also described, along with its C and Fortran
  interfaces.  Performance of the library is also discussed.
\end{abstract}

\newpage
\tableofcontents
\newpage

\section{Introduction} \label{sec:intro}
Multiprecision computation has a variety of application areas, such as
pure mathematics, study of mathematical constants, cryptography,
and computational geometry. Because of this, many arbitrary precision
algorithms and libraries have been developed using only the fixed
precision arithmetic. They can be divided into two groups based on
the way precision numbers are represented. Some libraries store numbers in a
{\em multiple-digit} format, with a sequence of digits coupled with a
single exponent, such as the symbolic computation package
Mathematica, Bailey's MPFUN~\cite{bai-mp}, Brent's MP~\cite{brent} and
GNU MP~\cite{gnu-mp}.  An alternative approach is to store numbers
in a {\em multiple-component} format, where a number is expressed
as unevaluated sums of ordinary floating-point words, each with its own
significand and exponent.  Examples of this format
include~\cite{dek71,pri92,she97}.  The multiple-digit approach can
represent a much larger range of numbers, whereas the multiple-component
approach has the advantage in speed.

We note that many applications would get full benefit from using merely
a small multiple of (such as twice or quadruple) the working precision,
without the need for arbitrary precision.
The algorithms for this kind of ``fixed'' precision can be made
significantly faster than those for arbitrary precision.
Bailey~\cite{bai-dd} and Briggs~\cite{kbriggs97} have developed
algorithms and software for ``double-double'' precision, twice the
double precision. They used the multiple-component format, where a
double-double number is represented as an unevaluated
sum of a leading double and a trailing double.

In this paper we present the algorithms used in the {\tt qd} library,
which implements both double-double and quad-double arithmetic.
A quad-double number is an unevaluated sum of four IEEE doubles.
The quad-double number $(a_0, a_1, a_2, a_3)$ represents the exact
sum $a = a_0 + a_1 + a_2 + a_3$, where $a_0$ is the most sigficant component.
We have designed and implemented algorithms for basic arithmetic
operations, as well as some algebraic and transcendental functions.
We have performed extensive correctness tests and compared the results
with arbitrary precision package MPFUN.
Our quad-precision library is available at
{\tt http://www.nersc.gov/\~{ }dhbailey/mpdist/mpdist.html}.
Our quad-double library has been successfully integrated into a parallel
vortex roll-up simulation code; this is briefly described in
\cite{hida00}.

The rest of the paper is organized as follows.
Section~\ref{sec:prelim} describes some basic properties of IEEE
floating point arithmetic and building blocks for our quad-double
algorithms. In Section~\ref{sec:basic} we present the quad-double
algorithms for basic operations, including renormization, addition,
multiplication and division.
Section~\ref{sec:algebraic} and~\ref{sec:transcendental}
present the algorithms for some algebraic operations and transcendental
functions. Section~\ref{sec:misc} describes some auxiliary functions.
Section~\ref{sec:implement} briefly describes our C++ library
implementing the above algorithms.
Section~\ref{sec:performance} presents the timing results of the
kernel operations on different architectures.
Section~\ref{sec:future} discusses future work.

\section{Preliminaries} \label{sec:prelim}
In this section, we present some basic properties and algorithms of IEEE
floating point arithmetic used in quad-double arithmetic.
These results are based on Dekker \cite{dek71}, Knuth \cite{knu81},
Priest \cite{pri92}, Shewchuk \cite{she97}, and others.  In fact,
many of the algorithms and diagrams are directly taken from, or based on,
Shewchuk's paper.

All basic arithmetics are assumed to be performed in IEEE double format,
with round-to-even rounding on ties.  For any binary operator
$\cdot \in \{+, -, \times, /\}$, we use $\fl(a \cdot b) = a \odot b$ to denote
the floating point result of $a \cdot b$, and define $\err(a \cdot b)$
as $a \cdot b = \fl(a \cdot b) + \err(a \cdot b)$.
Throughout this paper, $\eps = 2^{-53}$ is the machine epsilon for
IEEE double precision numbers, and $\epsqd = 2^{-211}$ is
the precision one expects for quad-double numbers.

\begin{lem} {\rm \cite[p. 310]{she97}}
  Let $a$ and $b$ be two $p$-bit floating point numbers such that
  $|a| \ge |b|$.  Then $|\err(a+b)| \le |b| \le |a|$.
\end{lem}

\begin{lem} {\rm \cite[p. 311]{she97}}
  Let $a$ and $b$ be two $p$-bit floating point numbers.  Then
  $\err(a+b) = (a + b) - \fl(a+b)$ is representable as a $p$-bit
  floating point number.
\end{lem}

\begin{alg} \cite[p. 312]{she97}
  The following algorithm computes $s = \fl(a+b)$ and $e = \err(a+b)$,
  assuming $|a| \ge |b|$.

  \vspace{0.1in}
  \hfill
  \begin{minipage}[t]{4in}
    {\sc Quick-Two-Sum}($a, b$) \\
    \begin{tabular}{rl}
      1. & $s \leftarrow a \oplus b$ \\
      2. & $e \leftarrow b \ominus (s \ominus a)$ \\
      3. & {\bf return} $(s, e)$
    \end{tabular}
  \end{minipage}
\end{alg}

\begin{alg} \cite[p. 314]{she97}
  The following algorithm computes $s = \fl(a+b)$ and $e = \err(a+b)$.
  This algorithm uses three more floating point operations instead of
  a branch.

  \vspace{0.1in}
  \hfill
  \begin{minipage}[t]{4in}
    {\sc Two-Sum}($a, b$) \\
    \begin{tabular}{rl}
      1. & $s \leftarrow a \oplus b$ \\
      2. & $v \leftarrow s \ominus a$ \\
      3. & $e \leftarrow (a \ominus (s \ominus v)) \oplus (b \ominus v)$ \\
      4. & {\bf return} $(s, e)$
    \end{tabular}
  \end{minipage}
\end{alg}

\begin{alg} \cite[p. 325]{she97}
  The following algorithm splits a 53-bit IEEE double precision
  floating point number into $a_{\hi}$ and $a_{\lo}$, each with 26 bits of
  significand, such that $a = a_\hi + a_\lo$.  $a_\hi$ will contain
  the first $26$ bits, while $a_\lo$ will contain the lower $26$ bits.

  \vspace{0.1in} \hfill
  \begin{minipage}[t]{4in}
    {\sc Split}($a$) \\
    \begin{tabular}{rl}
      1. & $t \leftarrow (2^{27}+1) \otimes a$ \\
      2. & $a_\hi \leftarrow t \ominus (t \ominus a)$ \\
      3. & $a_\lo \leftarrow a \ominus a_\hi$ \\
      4. & {\bf return} $(a_\hi, a_\lo)$
    \end{tabular}
  \end{minipage}
\end{alg}

\begin{alg} \cite[p. 326]{she97}
  The following algorithm computes $p = \fl(a \times b)$ and
  $e = \err(a \times b)$.

  \vspace{0.1in} \hfill
  \begin{minipage}[t]{5in}
    {\sc Two-Prod}($a, b$) \\
    \begin{tabular}{rl}
      1. & $p \leftarrow a \otimes b$ \\
      2. & $(a_\hi, a_\lo) \leftarrow$ {\sc Split}($a$) \\
      3. & $(b_\hi, b_\lo) \leftarrow$ {\sc Split}($b$) \\
      4. & $e \leftarrow ((a_\hi \otimes b_\hi \ominus p) \oplus
             a_\hi \otimes b_\lo \oplus a_\lo \otimes b_\hi) \oplus
             a_\lo \otimes b_\lo$  \\
      5. & {\bf return} $(p, e)$
    \end{tabular}
  \end{minipage}
\end{alg}

Some machines have a fused multiply-add instruction (FMA) that
can evaluate expression such as $a \times b \pm c$ with a single
rounding error.  We can take advantage of this instruction to compute
exact product of two floating point numbers much faster.  These
machines include IBM Power series (including the PowerPC), on which
this simplification is tested.

\begin{alg}
  The following algorithm computes $p = \fl(a \times b)$ and
  $e = \err(a \times b)$ on a machine with a FMA instruction.
  Note that some compilers emit FMA instructions for
  $a \times b + c$ but not for $a \times b - c$; in this case, some
  sign adjustments must be made.

  \vspace{0.1in} \hfill
  \begin{minipage}[t]{4in}
    {\sc Two-Prod-FMA}($a, b$) \\
    \begin{tabular}{rl}
      1. & $p \leftarrow a \otimes b$ \\
      2. & $e \leftarrow \fl(a \times b - p)$ \\
      3. & {\bf return} ($p, e$)
    \end{tabular}
  \end{minipage}
\end{alg}

The algorithms presented are the basic building blocks of quad-double
arithmetic, and are represented in Figures \ref{quick_two_sum_fig},
\ref{two_sum_fig}, and \ref{two_prod_fig}.  Symbols for normal
double precision sum and product are in Figure~\ref{normal_sum_prod_fig}.

\begin{figure}
  \hfill
  \begin{minipage}[t]{2in}
    \begin{center}
      \includegraphics{quick-two-sum.eps}
      \caption{\label{quick_two_sum_fig}{\sc Quick-Two-Sum}}
    \end{center}
  \end{minipage}
  \begin{minipage}[t]{2in}
    \begin{center}
      \includegraphics{two-sum.eps}
      \caption{\label{two_sum_fig}{\sc Two-Sum}}
    \end{center}
  \end{minipage}
  \begin{minipage}[t]{2in}
    \begin{center}
      \includegraphics{two-prod.eps}
      \caption{\label{two_prod_fig}{\sc Two-Prod}}
    \end{center}
  \end{minipage}
\end{figure}

\begin{figure}
  \begin{center}
    \includegraphics{normal_sum_prod.eps}
    \caption{\label{normal_sum_prod_fig}Normal IEEE double precision sum and product}
  \end{center}
\end{figure}

\section{Basic Operations} \label{sec:basic}
\subsection{Renormalization}
A quad-double number is an unevaluated sum of four IEEE double numbers.
The quad-double number $(a_0, a_1, a_2, a_3)$ represents the exact
sum $a = a_0 + a_1 + a_2 + a_3$.  Note that for any given representable
number $x$, there can be many representations as an unevaluated sum of
four doubles.  Hence we require that the quadruple $(a_0, a_1, a_2, a_3)$
to satisfy
\begin{displaymath}
  |a_{i+1}| \le \frac{1}{2}\ulp(a_i)
\end{displaymath}
for $i = 0, 1, 2$, with equality occurring only if $a_i = 0$ or
the last bit of $a_i$ is $0$ (that is, round-to-even is used in case
of ties).  Note that the first double $a_0$ is a double-precision
approximation to the quad-double number $a$, accurate to almost
half an ulp.

\begin{lem}
  For any quad-double number $a = (a_0, a_1, a_2, a_3)$, the normalized
  representation is unique.
\end{lem}

Most of the algorithms described here produce an expansion that
is not of canonical form -- often having overlapping bits.
Therefore, a five-term expansion is produced, and then renormalized
to four components.

\begin{alg}
  \label{renorm_alg}
  This renormalization procedure is a variant of Priest's renormalization
  method \cite[p. 116]{pri92}.  The input is a five-term expansion with
  limited overlapping bits, with $a_0$ being the most significant component.

  \vspace{0.1in} \hfill
  \begin{minipage}[t]{5in}
    {\sc Renormalize}($a_0, a_1, a_2, a_3, a_4$) \\
    \begin{tabular}{rl}
      1.  & $(s, t_4) \leftarrow$ {\sc Quick-Two-Sum}($a_3, a_4$) \\
      2.  & $(s, t_3) \leftarrow$ {\sc Quick-Two-Sum}($a_2, s$)   \\
      3.  & $(s, t_2) \leftarrow$ {\sc Quick-Two-Sum}($a_1, s$)   \\
      4.  & $(t_0, t_1) \leftarrow$ {\sc Quick-Two-Sum}($a_0, s$) \\
      \\
      5.  & $s \leftarrow t_0$ \\
      6.  & $k \leftarrow 0$   \\
      7.  & {\bf for\footnotemark[1]} $i \leftarrow 1, 2, 3, 4$ \\
      8.  & \quad $(s, e) \leftarrow$ {\sc Quick-Two-Sum}($s, t_i$) \\
      9.  & \quad {\bf if} $e \ne 0$             \\
      10. & \quad \quad $b_k \leftarrow s$       \\
      11. & \quad \quad $s \leftarrow e$         \\
      12. & \quad \quad $k \leftarrow k + 1$     \\
      13. & \quad {\bf end if}  \\
      14. & {\bf end for} \\
      15. & {\bf return} $(b_0, b_1, b_2, b_3)$
    \end{tabular}
  \end{minipage}
\end{alg}

\footnotetext[1]{In the implementation, this loop is unrolled to several
  {\tt if} statements.}

Necessary conditions for this renormalization algorithm to work correctly
are, unfortunately, not known.  Priest proves that if the input expansion
does not overlap by more than 51 bits, then the algorithm works correctly.
However, this condition is by no means necessary; that the renormalization
algorithm (Algorithm \ref{renorm_alg})
works on all the expansions produced by the algorithms below
remains to be shown.

\subsection{Addition}
\subsubsection{Quad-Double + Double}
The addition of a double precision number to a quad-double number
is similar to Shewchuk's {\sc Grow-Expansion} \cite[p. 316]{she97}, but the double
precision number $b$ is added to a quad-double number $a$ from
most significant component first (rather than from least significant).
This produces a five-term expansion which is the exact result,
which is then renormalized.  See Figure~\ref{qd_add_qd_d_fig}.

\begin{figure}
  \begin{center}
    \includegraphics{qd_add_qd_d.eps}
    \caption{\label{qd_add_qd_d_fig}Quad-Double + Double}
  \end{center}
\end{figure}

Since the exact result is computed, then normalized to four components,
this addition is accurate to at least the first 212 bits of the result.

\subsubsection{Quad-Double + Quad-Double}
We have implemented two algorithms for addition.  The first one
is faster, but only satisfies the weaker (Cray-style) error bound
$a \oplus b = (1 + \delta_1)a + (1 + \delta_2)b$ where the magnitude
of $\delta_1$ and $\delta_2$ is bounded by $\epsqd = 2^{-211}$.

\begin{figure}
  \begin{center}
    \includegraphics{qd_add.eps}
    \caption{\label{qd_add_fig}Quad-Double + Quad-Double}
  \end{center}
\end{figure}

Figure \ref{qd_add_fig} best describes the first addition algorithm of two
quad-double numbers.  In the diagram, there are three large boxes
with three inputs to them.  These are various {\sc Three-Sum} boxes,
and their internals are shown in Figure \ref{three_sum_fig}.

\begin{figure}
  \begin{center}
    \includegraphics{three-sum.eps} \includegraphics{three-sum-2.eps}
    \includegraphics{three-sum-3.eps}
    \caption{\label{three_sum_fig}{\sc Three-Sum}s}
  \end{center}
\end{figure}

Now for a few more lemmas.
\begin{lem}
  \label{two_sum_bound}
  Let $a$ and $b$ be two double precision floating point numbers.
  Let $M = \max(|a|, |b|)$.  Then $|\fl(a+b)| \le 2M$, and consequently,
  $|\err(a+b)| \le \frac{1}{2}\ulp(2M) \le 2 \eps M$.
\end{lem}

\begin{lem}
  \label{three_sum_bound}
  Let $x, y,$ and $z$ be inputs to {\sc Three-Sum}.
  Let $u, v, w, r_0, r_1,$ and $r_2$ be as indicated in Figure
  \ref{three_sum_fig}.  Let $M = \max (|x|, |y|, |z|)$.  Then
  $|r_0| \le 4M$, $|r_1| \le 8 \eps M$, and $|r_2| \le 8 \eps^2 M$.
\end{lem}
\begin{proof} This follows from applying Lemma \ref{two_sum_bound}
to each of the three {\sc Two-Sum} boxes.
First {\sc Two-Sum} gives $|u| \le 2M$ and
$|v| \le 2\eps M$.  Next {\sc Two-Sum} (adding $u$ and $z$) gives
$|r_0| \le 4M$ and $|w| \le 4\eps M$.  Finally, the last {\sc Two-Sum}
gives the desired result.
\end{proof}

Note that the two other {\sc Three-Sum}s shown are simplification
of the first {\sc Three-Sum}, where it only computes one or two components,
instead of three; thus the same bounds apply.

The above bound is not at all tight; $|r_0|$ is bounded closer
to $3M$ (or even $|x| + |y| + |z|$), and this makes the bounds for $r_1$
and $r_2$ correspondingly smaller.  However, this suffices for the
following lemma.

\begin{lem}\label{sum_lemma}
  The five-term expansion before the renormalization step in the
  quad-double addition algorithm shown in Figure \ref{qd_add_fig} errs
  from the true result by less than $\epsqd M$, where $M = \max(|a|, |b|)$.
\end{lem}
\begin{proof}
This can be shown by
judiciously applying Lemmas \ref{two_sum_bound} and \ref{three_sum_bound}
to all the {\sc Two-Sum}s and {\sc Three-Sum}s in Figure \ref{qd_add_fig}.
See Appendix \ref{proof_appendix} for detailed proof.
\end{proof}

Assuming that the renormalization
step works (this remains to be proven), we can then obtain the error bound
\begin{displaymath}
  \fl(a+b) = (1 + \delta_1) a + (1 + \delta_2) b \qquad \textrm{with} \quad
  |\delta_1|, |\delta_2| \le \epsqd.
\end{displaymath}

Note that the above algorithm for addition is particularly suited to
modern processors with instruction level parallelism, since the first
four {\sc Two-Sum}s can be evaluated in parallel.  Lack of branches
before the renormalization step also helps to keep the pipelines full.

Note that the above algorithm does not satisfy the IEEE-style error
bound
\begin{displaymath}
  \fl(a+b) = (1+\delta)(a+b) \qquad \textrm{with} \quad
  |\delta| \le 2\epsqd \textrm{ or so.}
\end{displaymath}
To see this, let $a = (u, v, w, x)$ and $b = (-u, -v, y, z)$,
where none of $w, x, y, z$ overlaps and $|w| > |x| > |y| > |z|$.
Then the above algorithm produces $c = (w, x, y, 0)$ instead
of $c = (w, x, y, z)$ required by the stricter bound.

The second algorithm, due to J. Shewchuk and S. Boldo, computes the first
four components of the {\em result} correctly.  Thus it satisfies
more strict error bound
\begin{displaymath}
  \fl(a+b) = (1+\delta)(a+b) \qquad \textrm{with} \quad
  |\delta| \le 2\epsqd \textrm{ or so.}
\end{displaymath}
However, it has a corresponding speed penalty; it runs significantly slower
(factor of $2$--$3.5$ slower).

The algorithm is similar to Shewchuk's {\sc Fast-Expansion-Sum}
\cite[p. 320]{she97},
where it merge-sorts the two expansions.  To prevent components with
only a few significant bits to be produced, a double-length accumulator
is used so that a component is output only if the inputs gets small
enough to not affect it.

\begin{alg}
  \label{double_accum_alg}
  Assuming that $u, v$ is a two-term expansion, the following algorithm
  computes the sum $(u, v) + x$, and outputs the significant component
  $s$ if the remaining components contain more than one double worth
  of significand. $u$ and $v$ are modified to represent the other
  two components in the sum.

  \vspace{0.1in} \hfill
  \begin{minipage}[t]{5in}
    {\sc Double-Accumulate}($u, v, x$) \\
    \begin{tabular}{rl}
      1.  & $(s, v) \leftarrow$ {\sc Two-Sum}($v, x$)  \\
      2.  & $(s, u) \leftarrow$ {\sc Two-Sum}($u, s$)  \\
      3.  & {\bf if} $u = 0$ \\
      4.  & \quad $u \leftarrow s$ \\
      5.  & \quad $s \leftarrow 0$ \\
      6.  & {\bf end if} \\
      7.  & {\bf if} $v = 0$ \\
      8.  & \quad $v \leftarrow u$ \\
      9.  & \quad $u \leftarrow s$ \\
      10. & \quad $s \leftarrow 0$ \\
      11. & {\bf end if}   \\
      12. & {\bf return} ($s, u, v$) \\
    \end{tabular}
  \end{minipage}
\end{alg}

The accurate addition scheme is given by the following algorithm.
\begin{alg}
  \label{accurate_add_alg}
  This algorithm computes the sum of two quad-double numbers
  $a = (a_0, a_1, a_2, a_3)$ and $b = (b_0, b_1, b_2, b_3)$.
  Basically it merge-sorts the eight doubles, and performs
  {\sc Double-Accumulate} until four components are obtained.

  \vspace{0.1in} \hfill
  \begin{minipage}[t]{5in}
    {\sc QD-Add-Accurate}($a, b$) \\
    \begin{tabular}{rl}
      1.  & $(x_0, x_1, \ldots, x_7) \leftarrow$
          {\sc Merge-Sort}$(a_0, a_1, a_2, a_3, b_0, b_1, b_2, b_3)$ \\
      2.  & $u \leftarrow 0$ \\
      3.  & $v \leftarrow 0$ \\
      4.  & $k \leftarrow 0$ \\
      5.  & $i \leftarrow 0$ \\
      6.  & {\bf while} $k < 4$ {\bf and} $i < 8$ {\bf do} \\
      7.  & \quad $(s,u,v)\leftarrow$ {\sc Double-Accumulate}$(u, v, x_i)$ \\
      8.  & \quad {\bf if} $s \ne 0$ \\
      9.  & \quad \quad $c_k \leftarrow s$ \\
     10.  & \quad \quad $k \leftarrow k + 1$ \\
     11.  & \quad {\bf end if} \\
     12.  & \quad $i \leftarrow i + 1$ \\
     13.  & {\bf end while} \\
     14.  & {\bf if} $k < 2$ {\bf then} $c_{k+1} \leftarrow v$ \\
     15.  & {\bf if} $k < 3$ {\bf then} $c_k \leftarrow u$ \\
     16.  & {\bf return} {\sc Renormalize}$(c_0, c_1, c_2, c_3)$
    \end{tabular}
  \end{minipage}
\end{alg}

\subsection{Subtraction}
Subtraction $a - b$ is implemented as the addition $a + (-b)$,
so it has the same algorithm and properties as that of addition.
To negate a quad-double number, we can just simply negate each component.
On a modern C++ compiler with inlining, the overhead is
noticeable but not prohibitive (say $5\%$ or so).

\subsection{Multiplication}
Multiplication is basically done in a straightforward way, multiplying
term by term and accumulating.  Note that unlike addition, there
are no possibilities of massive cancellation in multiplication, so
the following algorithms satisfy the IEEE style error bound
$a \otimes b = (1 + \delta)(a \times b)$ where $\delta$ is bounded
by $\epsqd$.

\subsubsection{Quad-Double $\times$ Double}
Let $a = (a_0, a_1, a_2, a_3)$ be a quad-double number, and let $b$ be
a double precision number.  Then the product is the sum of four terms,
$a_0 b + a_1 b + a_2 b + a_3 b$.  Note that $|a_3| \le \eps^3 |a_0|$,
so $|a_3 b| \le \eps^3 |a_0 b|$, and thus only the first 53 bits of
the product $a_3 b$ need to be computed.  The first three terms are
computed exactly using {\sc Two-Prod} (or {\sc Two-Prod-FMA}).
All the terms are then accumulated in a similar fashion as addition.
See Figure \ref{qd_mul_qd_d_fig}.

\begin{figure}
  \begin{center}
    \includegraphics{qd_mul_qd_d.eps}
    \caption{\label{qd_mul_qd_d_fig}Quad-Double $\times$ Double}
  \end{center}
\end{figure}

\subsubsection{Quad-Double $\times$ Quad-Double}
Multiplication of two quad-double numbers becomes a bit complicated,
but nevertheless follows the same idea.  Let $a = (a_0, a_1, a_2, a_3)$
and $b = (b_0, b_1, b_2, b_3)$ be two quad-double numbers.  Assume
(without loss of generality) that $a$ and $b$ are order 1.
After multiplication, we need to accumulate $13$ terms of order $O(\eps^4)$
or higher.
\begin{displaymath}
  \begin{array}{rcl@{\qquad}l}
    a \times b &\approx& a_0 b_0 & O(1) \textrm{ term} \\
    & & +\; a_0 b_1 + a_1 b_0 & O(\eps) \textrm{ terms}\\
    & & +\;a_0 b_2 + a_1 b_1 + a_2 b_0 & O(\eps^2) \textrm{ terms}\\
    & & +\;a_0 b_3 + a_1 b_2 + a_2 b_1 + a_3 b_0 & O(\eps^3) \textrm{ terms}\\
    & & +\;a_1 b_3 + a_2 b_2 + a_3 b_1 & O(\eps^4) \textrm{ terms}
  \end{array}
\end{displaymath}
Note that smaller order terms (such as $a_2 b_3$, which is $O(\eps^5)$)
are not even computed, since they are not needed to get the first
212 bits.  The $O(\eps^4)$ terms are computed using normal double
precision arithmetic, as only their first few bits are needed.

For $i+j \le 3$, let $(p_{ij}, q_{ij}) = ${\sc Two-Prod}($a_i$, $b_j$).
Then $p_{ij} = O(\eps^{i+j})$ and $q_{ij} = O(\eps^{i+j+1})$.
Now there are one term ($p_{00}$) of order $O(1)$, three
($p_{01}$, $p_{10}$, $q_{00}$) of order $O(\eps)$, five
($p_{02}$, $p_{11}$, $p_{20}$, $q_{01}$, $q_{10}$)
of order $O(\eps^2)$, seven of order $O(\eps^3)$,
and seven of order $O(\eps^4)$.
Now we can start accumulating all the terms by their order,
starting with $O(\eps)$ terms (see Figure \ref{qd_mul_accum_fig}).

\begin{figure}
  \begin{center}
    \includegraphics{qd_mul_accum.eps}
    \caption{\label{qd_mul_accum_fig}Quad-Double $\times$ Quad-Double accumulation phase}
  \end{center}
\end{figure}

In the diagram, there are four different summation boxes.
The first (topmost) one is {\sc Three-Sum}, same as the one in
addition.  The next three are, respectively, {\sc Six-Three-Sum}
(sums six doubles and outputs the first three components),
{\sc Nine-Two-Sum} (sums nine doubles and outputs the first two components),
and {\sc Nine-One-Sum} (just adds nine doubles using normal arithmetic).

{\sc Six-Three-Sum} computes the sum of six doubles to three double
worth of accuracy (i.e., to relative error of $O(\eps^3)$).  This is
done by dividing the inputs into two groups of three, and
performing {\sc Three-Sum} on each group.  Then the two sums
are added together, in a manner similar to quad-double addition.
See Figure \ref{six_three_sum_fig}.

{\sc Nine-Two-Sum} computes the sum of nine doubles to double-double
accuracy.  This is done by pairing the inputs to create four double-double
numbers and a single double precision number, and performing
addition of two double-double numbers recursively until one arrives
at a double-double output.  The double-double addition (the large square
box in the diagram) is the same as David Bailey's algorithm \cite{bai-dd}.
See Figure \ref{nine_two_sum_fig}.

\begin{figure}
  \begin{center}
    \includegraphics{six-three-sum.eps}
    \caption{\label{six_three_sum_fig}{\sc Six-Three-Sum}}
  \end{center}
\end{figure}

\begin{figure}
  \begin{center}
    \includegraphics{nine-two-sum.eps}
    \caption{\label{nine_two_sum_fig}{\sc Nine-Two-Sum}}
  \end{center}
\end{figure}

If one wishes to trade few bits of accuracy for speed, we don't
even need to compute the $O(\eps^4)$ terms; they can affect the
first 212 bits only by carries during accumulation.
In this case, we can compute the
$O(\eps^3)$ terms using normal double precision arithmetic,
thereby speeding up multiplication considerably.

Squaring a quad-double number can be done significantly
faster since the number of terms that needs to be accumulated
can be reduced due to symmetry.

\subsection{Division}
Division is done by the familiar long division algorithm.
Let $a = (a_0, a_1, a_2, a_3)$ and $b = (b_0, b_1, b_2, b_3)$ be
quad-double numbers.  We can first compute
an approximate quotient $q_0 = a_0 / b_0$.  We then compute
the remainder $r = a - q_0 \times b$, and compute the
correction term $q_1 = r_0 / b_0$.  We can continue this process
to obtain five terms, $q_0,\; q_1,\; q_2,\; q_3$, and $q_4$.
(only four are needed if few bits of accuracy is not important).

Note that at each step, full quad-double multiplication and
subtraction must be done since most of the bits will be canceled
when computing $q_3$ and $q_4$.  The five-term (or four-term)
expansion is then renormalized to obtain the quad-double quotient.

\section{Algebraic Operations} \label{sec:algebraic}
\subsection{$N$-th Power}
$N$-th Power computes $a^n$, given a quad-double number $a$ and an
integer $n$.  This is simply done by repeated squaring, borrowed
from David Bailey \cite{bai-dd}.

\subsection{Square Root}
Square root computes $\sqrt{a}$ given a quad-double number $a$.
This is done with Newton iteration on the function
\begin{displaymath}
  f(x) = \frac{1}{x^2} - a
\end{displaymath}
which has the roots $\pm a^{-1/2}$. This gives rise to the iteration
\begin{displaymath}
  x_{i+1} = x_i + \frac{x_i (1 - ax_i^2)}{2}.
\end{displaymath}
Note that the iteration does not require division of quad-double
numbers.  (Multiplication by $1/2$ can be done component-wise.)
Since Newton's iteration is locally quadratically convergent,
only about two iterations are required if one starts out with
double precision approximation $x_0 = \sqrt{a_0}$.  (In the
implementation it is done three times.)  After $x = a^{-1/2}$ is
computed, we perform a multiplication to obtain $\sqrt{a} = ax$.

\subsection{$N$-th Root}
$N$-th Root computes $\sqrt[n]{a}$ given a quad-double number $a$
and an integer $n$.  This is done again by Newton's iteration on
the function
\begin{displaymath}
  f(x) = \frac{1}{x^n} - a
\end{displaymath}
which has the roots $a^{-1/n}$. This gives rise to the iteration
\begin{displaymath}
  x_{i+1} = x_i + \frac{x_i (1 - ax_i^n)}{n}.
\end{displaymath}
Three iterations are performed, although twice is almost sufficient.
After $x = a^{-1/n}$ is computed, we can invert to obtain $a^{1/n} = 1/x$.

\section{Transcendental Operations} \label{sec:transcendental}
\subsection{Exponential}
The classic Taylor-Maclaurin series is used to evaluate $e^x$.  Before
using the Taylor series, the argument is reduced by noting that
\begin{displaymath}
  e^{kr + m \log 2} = 2^m (e^r)^k,
\end{displaymath}
where the integer $m$ is chosen so that $m \log 2$ is closest to $x$.
This way, we can make $|kr| \le \frac{1}{2} \log 2 \approx 0.34657$.
Using $k = 256$, we have $|r| \le \frac{1}{512} \log 2 \approx 0.001354$.
Now $e^r$ can be evaluated using familiar Taylor series.  The argument
reduction substantially speeds up the convergence of the series, as
at most 18 terms are need to be added in the Taylor series.

\subsection{Logarithm}
Since the Taylor series for logarithm converges much more slowly than
the series for exponential, instead we use Newton's iteration to
find the zero of the function $f(x) = e^x - a$.  This leads to
the iteration
\begin{displaymath}
  x_{i+1} = x_i + ae^{-x_i} - 1,
\end{displaymath}
which is repeated three times.

\subsection{Trigonometrics}
Sine and cosine are computed using Taylor series after argument reduction.
To compute $\sin x$ and $\cos x$, the argument $x$ is first reduced modulo
$2 \pi$, so that $|x| \le \pi$.  Now noting that $\sin (y + k \pi /2)$
and $\cos (y + k \pi / 2)$ are of the form $\pm \sin y$ or $\pm \cos y$
for all integers $k$, we can reduce the argument modulo $\pi/2$ so that
we only need to compute $\sin y$ and $\cos y$ with $|y| \le \pi / 4$.

Finally, write $y = z + m (\pi / 1024)$ where the integer $m$ is chosen
so that $|z| \le \pi / 2048 \approx 0.001534$.  Since $|y| \le \pi / 4$,
we can assume that $|m| \le 256$.
By using a precomputed table
of $\sin (m \pi / 1024)$ and $\cos (m \pi / 1024)$, we note that
\begin{displaymath}
  \sin (z + m \pi / 1024) = \sin z \cos (m \pi / 1024) +
  \cos z \sin (m \pi / 1024)
\end{displaymath}
and similarly for $\cos (z + m \pi / 1024)$.  Using this argument
reduction significantly increases the convergence rate of sine,
as at most 10 terms need be added.

Note that if both cosine and sine are needed, then one can compute
the cosine using the formula
\begin{displaymath}
  \cos x = \sqrt{1 - \sin^2 x}.
\end{displaymath}

The values of $\sin (m \pi / 1024)$ and $\cos (m \pi / 1024)$ are
precomputed by using arbitrary precision package such as MPFUN \cite{bai-mp}
using the formula
\begin{displaymath}
    \sin \left( \frac{\theta}{2} \right) =
    \frac{1}{2} \sqrt{2 - 2 \cos \theta}
\end{displaymath}
\begin{displaymath}
    \cos \left( \frac{\theta}{2} \right) =
    \frac{1}{2} \sqrt{2 + 2 \cos \theta}
\end{displaymath}
Starting with $\cos \pi = -1$, we can recursively use the above formula
to obtain $\sin (m \pi / 1024)$ and $\cos (m \pi / 1024)$.

\subsection{Inverse Trigonometrics}
Inverse trigonometric function $\arctan$ is computed using Newton
iteration on the function $f(x) = \sin x - a$.

\subsection{Hyperbolic Functions}
Hyperbolic sine and cosine are computed using
\begin{displaymath}
  \sinh x = \frac{e^x - e^{-x}}{2} \qquad \cosh x = \frac{e^x + e^{-x}}{2}
\end{displaymath}
However, when $x$ is small (say $|x| \le 0.01$), the above formula for
$\sinh$ becomes unstable, and the Taylor series is used instead.

\section{Miscellaneous Routines} \label{sec:misc}
\subsection{Input / Output}
  Binary to decimal conversion of quad-double number $x$ is done by
determining the integer $k$ such that $1 \le |x 10^{-k}| < 10$,
and repeatedly extracting digits and multiplying by 10.  To minimize
error accumulation, a table of accurately precomputed powers of 10
is used.  This table is also used in decimal to binary conversion.

\subsection{Comparisons}
Since quad-double numbers are fully renormalized after each
operation, comparing two quad-double number for equality can
be done component-wise.  Comparing the size can be done from
most significant component first, similar to dictionary ordering
of English words.  Comparison to zero can be done just by checking
the most significant word.

\subsection{Random Number Generator}
The quad-double random number generator produces a quad-double
number in the range $[0, 1)$, uniformly distributed.  This is
done by choosing the first 212 bits randomly.  A 31-bit system-supplied
random number generator is used to generate 31 bits at a time,
this is repeated $\lceil 212 / 31 \rceil = 7$ times to get
all 212 bits.

\section{C++ Implementation} \label{sec:implement}
  The quad-double library is implemented in ANSI C++, taking full
advantage of operator / function overloading and user-defined data
structures.  The library should compile fine with ANSI Standard compliant
C++ compilers.  Some of the test codes may not work with compilers
lacking full support for templates.  Please see the files {\tt README}
and {\tt INSTALL} for more details on the implementation and build
instructions.

  Full C++ implementation of double-double library is included as
a part of the quad-double library, including full support for mixing
three types: double, double-double, and quad-double.  In order to use
the library, one must include the header file {\tt qd.h}
and link the code with the library {\tt libqd.a}.
Quad-double variables are declared as {\tt qd\_real}, while double-double
variables are declared as {\tt dd\_real}.

A sample C++ program is given below.

\begin{tt}\begin{verbatim}
  #include <iostream>
  #include <qd/qd.h>

  using std::cout;
  using std::endl;

  int main() {
    int oldcw;
    fpu_fix_start(&oldcw);     // see notes on x86 machines below.

    qd_real a = "3.141592653589793238462643383279502884197169399375105820";
    dd_real b = "2.249775724709369995957";
    dd_real r;

    r = a + b;
    cout << "pi + e = " << r << endl;

    r = sqrt(r + 1.0);

    fpu_fix_end(&oldcw);       // see notes on x86 machines below.
    return 0;
  }
\end{verbatim}\end{tt}

Note that strings must be used to assign to a quad-double (or double-double)
numbers; otherwise the double precision approximation is assigned.
For example, {\tt a = 0.1} does not assign quad-double precision 0.1,
but rather a double precision number 0.1.  Instead, use {\tt a = "0.1"}.

Common constants such as $\pi$, $\pi/2$, $\pi/4$, $e$, $\log 2$ are
 provided as {\tt qd\_real::\_pi}, {\tt qd\_real::\_pi2},
{\tt qd\_real::\_pi4}, {\tt qd\_real::\_e}, and {\tt qd\_real::\_log2}.
These were computed using an arbitrary precision package (MPFUN++
\cite{cha98}), and therefore are accurate to the last bit.

\noindent{\bf Note on Intel x86 Processors}.
The algorithms in this library assume IEEE double precision floating
point arithmetic.  Since Intel x86 processors have extended (80-bit)
floating point registers, the round-to-double flag must be enabled in
the control word of the FPU for this library to function properly
under x86 processors.  The function {\tt fpu\_fix\_start} turns
on the round-to-double bit in the FPU control word, while
{\tt fpu\_fix\_end} will restore the original state.

\section{Performance} \label{sec:performance}
Performance of various operations on quad-double numbers on a variety
of machines are presented in Table \ref{timing_table}.  The tested
machines are
\begin{itemize}
\item Intel Pentium II, 400 MHz, Linux 2.2.16, g++ 2.95.2 compiler,
  with {\tt -O3 -funroll-loops -finline-functions -mcpu=i686 -march=i686}
  optimizations.
\item Sun UltraSparc 333 MHz, SunOS 5.7, Sun CC 5.0 compiler,
  with {\tt -xO5 -native} optimizations.
\item PowerPC 750 (Apple G3), 266 MHz, Linux 2.2.15, g++ 2.95.2 compiler,
  with {\tt -O3 -funroll-\\loops -finline-functions} optimizations.
\item IBM RS/6000 Power3, 200 MHz, AIX 3.4, IBM xlC compiler,
  with {\tt -O3 -qarch=pwr3 -qtune\\=pwr3 -qstrict} optimizations.
\end{itemize}

\noindent {\bf Note}: For some reason, GNU C++ compiler ({\tt g++}) has
a terrible time optimizing the code for multiplication; it runs more than
15 times slower than the code compiled by Sun's CC compiler.

\begin{table}[t]
  \begin{center}
  \label{timing_table}
  \begin{tabular}{|l|c|c|c|c|} \hline
    Operation &
    \begin{tabular}{c}Pentium II \\ 400MHz \\ Linux 2.2.16 \end{tabular} &
    \begin{tabular}{c}UltraSparc \\ 333 MHz \\ SunOS 5.7 \end{tabular} &
    \begin{tabular}{c}PowerPC 750 \\ 266 MHz \\ Linux 2.2.15 \end{tabular} &
    \begin{tabular}{c}Power3 \\  200 MHz \\ AIX 3.4 \end{tabular} \\
    \hline
    \multicolumn{5}{|l|}{{\em Quad-double}} \\
    add          &  0.583 &  0.580 &  0.868 &  0.710 \\
    accurate add &  1.280 &  2.464 &  2.468 &  1.551 \\
    mul          &  1.965 &  1.153 &  1.744 &  1.131 \\
    sloppy mul   &  1.016 &  0.860 &  1.177 &  0.875 \\
    div          &  5.267 &  6.440 &  8.210 &  6.699 \\
    sloppy div   &  4.080 &  4.163 &  6.200 &  4.979 \\
    sqrt         & 23.646 & 15.003 & 21.415 & 16.174 \\ \hline
    \multicolumn{5}{|l|}{{\em MPFUN}}\\
    add		 &  5.729 &  5.362 &  ---   &  4.651 \\
    mul		 &  7.624 &  7.630 &  ---   &  5.837 \\
    div     	 & 10.102 & 10.164 &  ---   &  9.180 \\ \hline
  \end{tabular}
  \caption{Performance of some Quad-Double algorithms on several machines.
    All measurements are in microseconds. We include the performance of
    MPFUN~\cite{bai-mp} as a comparison. Note, we do not have the MPFUN
    measurements on the PowerPC, because we do not have a Fortran-90 compiler.}
  \end{center}
\end{table}

Most of the routines runs noticeably faster if implemented in C,
it seems that C++ operator overloading has some overhead associated
with it -- most notably excessive copying of quad-double numbers.
This occurs because operator overloading does not account for
where the result is going to be placed.  For example,
for the code
\begin{quote}\begin{tt}c = a + b;\end{tt}\end{quote}
the C++ compiler often emits the code equivalent to
\begin{quote}\begin{tt}
\begin{verbatim}
qd_real temp;
temp = operator+(a, b);     // Addition
operator=(c, temp);         // Copy result to c
\end{verbatim}
\end{tt}\end{quote}
In C, this copying does not happen, as one would just write
\begin{quote}\begin{tt}\begin{verbatim}
c_qd_add(a, b, c);          // Put (a+b) into c
\end{verbatim}\end{tt}\end{quote}
where the addition routine knows where to put the result directly.
This problem is somewhat alleviated by inlining, but not completely
eliminated.  There are techniques to avoid these kinds of copying~\cite{c++},
but they have
their own overheads associated with them and is not practical for
quad-double with only 32 bytes of data\footnote{These techniques
are feasible, for larger data structures, such as for much higher
precision arithmetics, where copying of data becomes time consuming.}.

\section{Future Work} \label{sec:future}
Currently, the basic routines do not have a full correctness proof.
The correctness of these routines rely on the fact that renormalization
step works; Priest proves that it does work if the input does not overlap
by 51 bits and no three components overlap at a single bit.  Whether
such overlap can occur in any of these algorithm needs to be proved.

There are improvements due in the remainder operator, which computes
$a - \round(a/b) \times b$, given quad-double numbers $a$ and $b$.
Currently, the library does the na\"\i ve method of just divide, round,
multiply, and subtract.  This leads to loss of accuracy when $a$ is large
compared to $b$.  Since this routine is used in argument reduction for
exponentials, logarithms and trigonometrics, a fix is needed.

A natural extention of this work is to extend the precision beyond
quad-double.  Algorithms for quad-double additions and multiplication
can be extended to higher precisions, however, with more components,
asymptotically faster algorithm due to S. Boldo and J. Shewchuk may
be preferrable (i.e. Algorithm~\ref{accurate_add_alg}).
One limitation these higher precision expansions have
is the limited exponent range -- same as that of double.  Hence
the maximum precision is about 2000 bits (39 components),
and this occurs only if the first component is near overflow and the
last near underflow.

\section{Acknowledgements}
We thank Jonathan Shewchuk,
Sylvie Boldo, and James Demmel for constructive discussions on
various basic algorithms.  In particular, the accurate version of
addition algorithm is due to S. Boldo and J. Shewchuk.  Problems with
remainder was pointed out by J.Demmel.

\appendix
\newpage
\section{Proof of Quad-Double Addition Error Bound (Lemma \ref{sum_lemma})}
\label{proof_appendix}

\noindent
{\bf Lemma \ref{sum_lemma}}.
\emph{The five-term expansion before the renormalization step in the
quad-double addition algorithm shown in Figure \ref{qd_add_fig} errs
from the true result by less than $\epsqd M$, where $M = \max(|a|, |b|)$.}

\begin{proof}
  The proof is done by applying Lemmas \ref{two_sum_bound} and
\ref{three_sum_bound} to each of {\sc Two-Sum}s and {\sc Three-Sum}s.
Let $e_0, e_1, e_2, e_3, t_1, t_2, t_3, x_0, x_1, x_2, x_3, x_4, u, v,
w, z, f_1, f_2$, and $f_3$ be as shown in Figure \ref{qd_add_proof_fig}.

We need to show
that the five-term expansion $(x_0, x_1, x_2, x_3, x_4)$ errs from the
true result by less than $\epsqd M$, where $M = \max(|a_0|, |b_0|)$.
Note that the only place that any error is introduced is in {\sc Three-Sum} 7
and {\sc Three-Sum} 8, where lower order terms $f_1, f_2,$ and $f_3$ are discarded.
Hence it suffices to show $|f_1| + |f_2| + |f_3| \le \epsqd M$.

\begin{figure}[h]
  \begin{center}
    \includegraphics{qd_add_proof.eps}
    \caption{\label{qd_add_proof_fig}Quad-Double + Quad-Double}
  \end{center}
\end{figure}

First note that $|a_1| \le \eps M$, $|a_2| \le \eps^2 M$,and
$|a_3| \le \eps^3 M$, since the input expansions are assumed to be
normalized.  Similar inequalities applies for expansion $b$.
Applying Lemma \ref{two_sum_bound} to {\sc Two-Sum}s 1, 2, 3, 4, we obtain
\begin{displaymath}
  \begin{array}{rcl@{\qquad}rcl}
    |x_0| &\le& 2M & |e_0| &\le& 2\eps M\\
    |t_1| &\le& 2\eps M   & |e_1| &\le& 2\eps^2 M\\
    |t_2| &\le& 2\eps^2 M & |e_2| &\le& 2\eps^3 M\\
    |t_3| &\le& 2\eps^3 M & |e_3| &\le& 2\eps^4 M.
  \end{array}
\end{displaymath}

Now we can apply Lemma \ref{two_sum_bound} to {\sc Two-Sum} 5, to obtain
$|x_1| \le 4\eps M$ and $|u| \le 4 \eps^2 M$.  Then we apply Lemma
\ref{three_sum_bound} to {\sc Three-Sum} 6 to obtain
\begin{displaymath}
  \begin{array}{r@{\;\le\;}l}
    |x_2| & 16\eps^2 M \\
    |w| & 32 \eps^3 M  \\
    |v| & 32 \eps^4 M.
  \end{array}
\end{displaymath}

Applying Lemma \ref{three_sum_bound} to {\sc Three-Sum} 7, we have
\begin{displaymath}
  \begin{array}{r@{\;\le\;}l}
    |x_3| & 128\eps^3 M \\
    |z| & 256 \eps^4 M  \\
    |f_1| & 256 \eps^5 M.
  \end{array}
\end{displaymath}

Finally we apply Lemma \ref{three_sum_bound} again to {\sc Three-Sum} 8
to get
\begin{displaymath}
  \begin{array}{r@{\;\le\;}l}
    |x_4| & 1024\eps^4 M \\
    |f_2| & 2048 \eps^5 M  \\
    |f_3| & 2048 \eps^6 M.
  \end{array}
\end{displaymath}

Thus we have
\begin{displaymath}
  |f_1| + |f_2| + |f_3| \le 256 \eps^5 M + 2048 \eps^5 M + 2048 \eps^6 M
  \le 2305 \eps^5 M \le \epsqd M
\end{displaymath}
as claimed.
\end{proof}

\newpage
\bibliographystyle{plain}
\bibliography{qd}

\end{document}