\section{Algorithms\label{sec:algo}}

We now detail a set of algorithms to solve the integral equation in \cref{eq:int-eq} and evaluate the solution via the double layer integral in \cref{eq:double_layer} at a given target point $\vx \in \Omega$.
As described in the previous section, both solving \cref{eq:int-eq} and evaluating \cref{eq:double_layer} require accurate evaluation of singular/near-singular integrals of functions defined on the surface $\Gammah$.
We first outline our unified singular/near-singular integration scheme, \qbkix, its relation to existing approximation-based quadrature methods and geometric problems that can impede accurate solution evaluation.
We then describe two geometry preprocessing algorithms, \textit{admissibility refinement} and \textit{adaptive upsampling}, that address these issues to obtain the sets of patches $\Pcoarse$ and $\Pfine$ used by \qbkix.

\subsection{Singular and Near-Singular Evaluation \label{sec:singular-eval}}
%We start with describing our (near-) singular integration algorithm, identifying conditions that the patches need to satisfy for the algorithm to achieve a target accuracy $\etrg$.

%We start with the informal idea of the algorithm. As in all QBX-style algorithms, we take advantage of the fact that while the integrand may be (near-)singular, the solution of the PDE given by the integral is not, and can be extrapolated from points where it can be computed reliably to points close to the surface or on the surface.
We begin with an outline of the algorithm.
%As with all \qbx-style algorithms, we observe that while the integrand may be singular/near-singular for a particular choice of $\vx$, the solution of the PDE given by \cref{eq:double_layer} is well-defined.
%This allows us to extrapolate the solution to $\vx$ from nearby points where the integrand is smooth and standard quadrature rules are accurate.
%Specifically, for a quadrature sample $x_j$ on a patch $P$ from $\Pcoarse$, where we need to evaluate the singular integral  to solve \cref{eq:int-eq}, we compute the values for extrapolation at points $c_{j,s}$ (check points) sampled along the normal to the surface at $\vy_j$.  Suppose we fix the placement of points $c_{j,s}$ in the interior of $\Omega$.  We can always refine $\Pfine$  so for evaluation of integrals at these points $c_{j,s}$ the smooth quadrature rules using denser quadrature points $\tilde{x}_j$  on $\Pfine$ are sufficiently accurate. Then extrapolation to $\vy_j$ can be applied to these accurate values.  In practice, we perform refinement to obtain  $\Pcoarse$,  not just $\Pfine$ (admissibility refinement). For $\Pcoarse$, we use a heuristic error estimate to place check points for every $x_j$,  and refine patches to ensure that they are away from patches other than $P$. In this way, the amount of refinement needed for $\Pfine$ is reduced.
For a point $\vsx \in \Gammah$ on a patch $\vP$ from $\Pcoarse$ that is closest to $\vx$, we first upsample the density $\phi$ from $\Pcoarse$ to $\Pfine$ and compute the solution at a set of points $\vc_s$, $s = 1, \hdots p$ called \emph{check points}, sampled along the surface normal at $\vsx$ away from $\Gammah$.
We use \cref{eq:double_layer_disc} to approximate the solution at the check points.
We then extrapolate the solution to $\vx$.
%The placement of check points relative to $\Pcoarse$ is critical to ensuring the accuracy of the overall method.
%In \cref{sec:admissible}, we list the geometric criteria that $\Pcoarse$ must satisfy in order to solve \cref{eq:int-eq} accurately; we enforce these criteria by a sequence of quadrisection algorithms called \emph{admissibility refinement}.
%The resulting set of patches determines a set of check points $\{\vc_{I,s}\}$ in $\Omega$, which are used to extrapolate the solution to each of the quadrature samples $\vy_I$ in the discretization of $\Pcoarse$.
%Once we fix a set of check points, we further refine $\Pcoarse$ to produce a set of patches $\Pfine$ such that \cref{eq:double_layer_disc} can be evaluated accurately at each $\vc_{I,s}$.
%The algorithm to construct $\Pfine$ from $\Pcoarse$ is called \emph{adaptive upsampling}.
%We use empirical heuristics to place check points in admissibility refinement and trigger refinement in adaptive upsampling to reduce the overall amount number of quadrature patches in $\Pfine$.

For a given surface or quadrature patch $\vP: \I^2 \rightarrow \mathbb{R}^3$,  we define the \textit{characteristic length} $L(P)$ as the square root of the surface area of $\vP$, i.e., $L(\vP) = \sqrt{\int_{\vP}d\vy_{\vP}}$.
We use  $L = L(\vP)$ or $L_\vy$ for $\vy \in \vP(D)$ to denote the characteristic length when $\vP$ is clear from context.
For a point $\vx \in \Omega$, we assume that there is a single closest point $\vsx \in \Gammah$ to $\vx$; all points to which the algorithm is applied will have this property by construction.
Note that $\vn(\vsx)$, the vector normal to $\Gammah$ at $\vsx$, is chosen to point outside of $\Omega$.
%We assume that two sets of patches are defined:  $\Pcoarse$, which serves as the discretization of $\Gamma$, and $\Pfine$, which is obtained from $\Pcoarse$ by refinement.
%We will provide a detailed discussion regarding this refinement in \cref{sec:adaptive_upsampling}.

%Expanding \cref{eq:double_layer_disc}, we obtain the following approximation of $u$ with respect
%to discretization on a collection of patches $\qP$:
%\begin{equation}
%  \hat{u}(\vx; \qP) = \sum_{P \in \qP}\sum_{\ell \in I(P)} \frac{\partial G(\vx,\vy_\ell)}{\partial n}\cdot \phi_\ell w_\ell.
%\label{eq:double_layer_disc}
%\end{equation}
%We want to compute $\hat{u}$ such that $\|u(\vx) - \hat{u}(\vx)\|_2 \leq \etrg$.

%Recall \cref{eq:double_layer_disc}, in which we defined $\hat{u}(\vx;\Pcoarse)$ as the discretization of \cref{eq:double_layer} with a quadrature rule for smooth funcitons.
%\edittxt{We define three zones in $\Omega$, in terms of \cref{eq:double_layer_disc}, for which \cref{eq:double_layer} is evaluated differently.}{We define three zones in $\Omega$ for which \cref{eq:double_layer} is evaluated differently in terms of \cref{eq:double_layer_disc} and the desired solution accuracy $\etrg$ .}
We define three zones in $\Omega$ for which \cref{eq:double_layer} is evaluated differently in terms of \cref{eq:double_layer_disc} and the desired solution accuracy $\etrg$ .
The \emph{far field}  $\Omega_F = \{\vx \in \Omega \,|\, \|u(\vx) - \hat{u}(\vx;\Pcoarse)\|_2 \leq \etrg\}$, where the quadrature rule corresponding to $\Pcoarse$ is sufficiently accurate, and the \emph{intermediate field}  $\Omega_I= \{\vx \in \Omega \,|\, \|u(\vx) - \hat{u}(\vx;\Pfine)\|_2 \leq \etrg\}$, where quadrature over $\Pfine$ is sufficiently accurate.
The remainder of $\Omega$ is the \emph{near field} $\Omega_N = \Omega \setminus \Omega_I$.

%In a large portion of $\Omega$, computing $\hat{u}(\vx, \Pcoarse)$ or $\hat{u}(\vx, \Pfine)$ with \cref{eq:double_layer_disc} directly is sufficient.

%We'll refer to $\Omega_F$ as the \textit{far field} of $\Pcoarse$ and $\Omega_I$ as the \textit{intermediate field} of $\Pcoarse$.

%When $\vx$ is not in $\Omega_I$, special treatment is required to approximate the potential.

\paragraph*{Non-singular integration}
To compute the solution at points $\vx$ in $\Omega_F$, \cref{eq:double_layer_disc} is accurate to $\etrg$, so we can simply compute $\hat{u}(\vx, \Pcoarse)$ directly.
Similarly for points in $\Omega_I \setminus \Omega_F$, we know by definition that $\hat{u}(\vx, \Pfine)$ is sufficiently accurate, so it can also be applied directly.

\paragraph*{Singular/near-singular integration algorithm}

\begin{figure}[!htb]
  %\setlength\figureheight{1.9in}
  %\setlength\figurewidth{2.1in}
  \begin{minipage}{\textwidth}
  \centering
      \includegraphics[width=.85\linewidth]{figs/qbkix_schematic.pdf}
  \end{minipage}\hfill
  \mcaption{fig:qbkix-schematic}{Schematic of singular/near-singular
  evaluation}{A small piece of a boundary $\Gammah$ is shown, along with the set of
  patches $\Pcoarse$ (patch boundaries are drawn in black). The target point $\vx$, in this
  case on $\Gammah$, is shown in green. The solution is evaluated
  at the check points $\vc_s$ (gray points off-surface) using the fine
  discretization $\Pfine$ (small dots on-surface). The distance from the first
  check point $\vc_0$ to $\Gammah$ is $R$ and the distance between consecutive
  check points $\vc_i$ and $\vc_{i+1}$ is $r$. In this example, $\Pfine$ is computed
  from $\Pcoarse$ with two levels of uniform quadrisection, producing 16 times
  more patches. The patch length $L$ is roughly proportional to the average edge
  length of the patch.}
\end{figure}

For the remaining points in $\Omega_N$, we need an alternative means of evaluating the solution.
In the spirit of the near-singular evaluation method of \cite{YBZ},  we construct a set of \textit{check points} $\vc_0, \hdots, \vc_p$ in $\Omega_I$ along a line intersecting $\vx$ to approximate the solution near $\vx$.
However, instead of interpolating the solution as in \cite{YBZ}, we instead extrapolate the solution from the check points to $\vx$.
%We first place check points in $\Omega_I$, compute $\hat{u}(\vc_i, \Pfine)$ for each $i$, then extrapolate the approximate values at the check points to $\vx$.  \note[MJM]{kill sentence}
We define two distances relative to $\vsx$: $R(\vsx) =b L_\vsx = \|\vc_0 - \vsx\|_2$, the distance from the first check point $\vc_0$ to $\Gammah$,  and $r(\vsx) =a L_\vsx = \|\vc_i - \vc_{i+1}\|_2$, the distance between consecutive check points.
We assume  $0<a,b <1$.
%The points are placed along the surface normal $\vn(\vsx)$.

The overall algorithm for the unified singular/near-singular evaluation scheme is as follows.
A schematic for \qbkix is depicted in \Cref{fig:qbkix-schematic}.

\begin{enumerate}
  \item Find the closest point $\vsx$ on $\Gammah$ to $\vx$.
  \item Given values $a$ and $b$, generate check points $C = \{\vc_0, \hdots, \vc_{p}\}$
    %distance $R(\vy)$ away from the $\Gamma$ and equispaced in $\vr(\vy)$ along the inward-pointing normal:
    \begin{equation}
      \vc_s = \vsx -(R(\vsx) +s r(\vsx)) \vn(\vsx), \quad s=0, \hdots, p
      \label{eq:check}
    \end{equation}
    The center of mass of these check points $\vhc$ is called the \textit{check center} for $\vx$.
    Note that $\Pfine$ must satisfy the condition that $\vc_s$ are in $\Omega_I$ for a given choice of $a$ and $b$.
\item Upsample $\phi$.
  We interpolate the density values $\phi_I$ at $x_I$ on patches in $\Pcoarse$ to quadrature points $\tilde{x}_J$ on patches in $\Pfine$
  with global indices $I$ and $J$ on $\Pcoarse$ and $\Pfine$ respectively.
  If a patch $\vP_i$ in $\Pcoarse$ is split into $m_i$ patches in $\Pfine$, we are interpolating from $q^2$ points to $m_iq^2$ points.
  %Let $\wfine$ and $\phifine$ be the vector quadrature weights and interpolated density values at $\tilde{x}_J$ of $\Pfine$.
  \item Evaluate the potential at check points via smooth quadrature with the upsampled density, i.e. evaluate $\hat{u}(\vc_s) = \hat{u}(\vc_s, \Pfine)$ for $s=0,\hdots, p$.
  \item Compute a Lagrange interpolant $\tilde{u}$ through the check points $\vc_0,\hdots, \vc_p$ and values \linebreak $\hat{u}(\vc_0), \hdots, \hat{u}(\vc_{p})$ and evaluate at the interpolant at $\vx$:
      \begin{equation}
          \tilde{u}(\vx) = \sum_{s=0}^p \hat{u}(\vc_s)\ell_s(t_\vx),
      \end{equation}
        where $\ell_s(\vx)$ is the $s$th Lagrange basis function through the points $\vc_0,\hdots, \vc_p$, and $t_\vx\in \mathbb{R}$ is such that $\vx = \vsx - t_\vx\vn(\vsx)$ (see \cref{fig:extrap-err-setup} for a schematic of the check points).
    Since $\vx$ lies between $\vc_0$ and $\Gammah$, we are extrapolating when computing $\tilde{u}(\vx)$.
\end{enumerate}

%\begin{algorithm}
%    \KwData{A set of surface patches $\Pcoarse$, a set of quadrature patches $\qPfine$, a target point $\vx$, extrapolation order $p$, quadrature order $q$, }
%  \KwResult{The closest point $\vsx$ on $\qP$ to $\vx$}
%
%  \DontPrintSemicolon
%  Construct an AABB tree $T_T$ from a fine triangle mesh of the quadrature patches of $\qP$\;
%  Construct an AABB tree $T_B$ from bounding boxes of quadrature patches in $\qP$.\;
%    $\tau_0 = $ closest triangle to $\vx$ computed with $T_T$ \;
%
%    $P_{i_0} = $ patch corresponding to $\tau_0$\;
%    Find the closest point $\vector{s}_{\vector{\vx},0}$ on $P_{i_0}$ to $\vx$ with \cref{app:closest_point}.\;
%    $d_{i_0} = \|\vx - \vector{s}_{\vector{\vx},0}\|_2$\;
%    $B_{d_{i_0}}(\vx)=$ a box centered a $\vx$ with edge length $2d_{i_0}$\;
%    Find the boxes $B_{i_1}, \hdots B_{i_k}$ in $T_B$ that intersect $B_{d_{i_0}}(\vx)$\;
%
%    \For{$B_{i_j} \in B_{i_1}, \hdots B_{i_k}$}{
%      $P_{i_j} =$ quadrature patch corresponding to $B_{i_j}$ \;
%      Find the closest point $\vector{s}_{\vector{\vx},j}$ on $P_{i_j}$ to $\vx$ with \cref{app:closest_point} to precision $\err{opt}$.\;
%      $d_{i_j} = \|\vx - \vector{s}_{\vector{\vx},j}\|_2$\;
%    }
%    $j^* = \mathrm{argmin}_j\{d_{i_j}\}$ \;
%    \Return{$\vector{s}_{\vector{x},j^*}$}
%  \mcaption{alg:singular_eval}{Evaluate the singular/near-singular layer potential at $\vx$}{}
%\end{algorithm}
%The parameters involved in this scheme are the number of check points $p$ and the relative spacing parameters of the check points $a$ and $b$.
%A critical aspect of the scheme is ensuring that the check points are in the intermediate field, i.e., $\Pfine$ is chosen to satisfy this condition.
%We use the error discussion in \cref{sec:error} and the algorithms of \cref{sec:adaptive_upsampling} to compute values of $a$, $b$ and  $\Pfine$ for a given $\etrg$.

\paragraph*{Ill-conditioning of the discrete integral operator}
%This scheme is used to compute singular integrals needed in the iterative solver for the solution of \cref{eq:int-eq}.
This evaluation scheme can be used directly to extrapolate all the way to the surface and obtain the
values of the singular integral in \cref{eq:int-eq}.
However, in practice, due to a distorted eigenspectrum of this approximate operator, \gmres tends to stagnate at a level of error corresponding to the accuracy of \qbkix when it is used to compute the matrix-vector product.
This is a well-known phenomenon of approximation-based singular quadrature schemes; \cite[Section 3.5]{KBGN}\cite[Section 4.2]{RBZ} present a more detailed study.
To address this, we average the interior and exterior limits of the solution at the quadrature nodes, computed via \qbkix, to compute the on-surface potential and add $\frac{1}{2}I$ to produce the interior limit.
This shifts the clustering of eigenvalues from around zero to around $\frac{1}{2}$, which is ideal from the perspective of \gmres.
We call this \textit{two-sided} \qbkix, while the standard version described above is called \textit{one-sided} \qbkix.
We observe stable and consistent convergence of \gmres when two-sided \qbkix is used to evaluate the matrix-vector multiply to solve \cref{eq:linear_system}.
In light of this, we always use two-sided \qbkix within \gmres and set the stopping tolerance for \gmres to $\err{\gmres}=10^{-12}$, regardless of the geometry, boundary condition or  quadrature order.

\subsection{Geometric criteria for accurate quadrature\label{sec:geom_criteria}}
The accuracy of the method outlined above is controlled by two competing error terms: \textit{quadrature error} incurred from approximating the layer potential \cref{eq:double_layer} with \cref{eq:double_layer_disc} in Step 4 and \textit{extrapolation error} due to approximating the singular integral with an extratpolated value in Step 5.
Both errors are determined by the location of check points relative to the patches in $\Pcoarse$ and $\Pfine$ (see \Cref{heuristic:error_quad_high_order,thm:extrap_error}).

\begin{figure}[!htb]
  \centering
  %\setlength\figureheight{1.9in}
  %\setlength\figurewidth{2.1in}
  \begin{minipage}{.33\textwidth}
      \includegraphics[width=\linewidth]{figs/admissibility_motivation2.pdf}
  \end{minipage}\hfill
  \begin{minipage}{.33\textwidth}
    \includegraphics[width=\linewidth]{figs/admissibility_motivation1.pdf}
  \end{minipage}\hfill
  \begin{minipage}{.33\textwidth}
    \includegraphics[width=\linewidth]{figs/admissibility_motivation3.pdf}
  \end{minipage}\hfill
  \mcaption{fig:admissibility-motivation}{Possible check point configurations}{A \twod  example depicting three choices of $a$ and $b$ in \cref{eq:check}.
 Shown is the boundary $\Gammah$, with black tick marks denoting patch boundaries of $\Pcoarse$, green tick marks denoting patch boundaries of $\Pfine$, the target point (red dots), its check points (blue dots) along the normal closest to the target point, and the medial axis of $\Gammah$ (gray dotted line).
Large (left) and small (middle) values of $a$ and $b$ can cause clustering of check points near to $\Gammah$, which requires large amounts of upsampling to compute the potential accurately. Using the medial axis as a heuristic to for admissibility (right), we can minimize the amount of adaptive upsampling required.}
\end{figure}
In \Cref{fig:admissibility-motivation}, we show three examples of different choices of check point locations to evaluate the potential at a point with \qbkix.
%Suppose that we have chosen extrapolation parameters $a$ and $b$ such that the extrapolation accuracy is less then $\etrg$, following the discussions in \cref{sec:extrap_error,sec:parameter-selection}. From an accuracy perspective, each choice of parameters is are equally valid.
%However, each choice will require a different set $\Pfine$ in order to ensure accurate integration in \cref{eq:double_layer_disc}.
In \cref{fig:admissibility-motivation}-left, $\vc_0$ is placed close to the target point, while in \cref{fig:admissibility-motivation}-middle, $\vc_0$ is far from the target point, but $\vc_p$ is close to a non-local piece of $\Gammah$.
Both cases will require excessive refinement of $\Pcoarse$ in order to resolve \cref{eq:double_layer_disc} accurately with $\Pfine$.
On the other hand, in \cref{fig:admissibility-motivation}-right, we can either perform one refinement step on $\Pcoarse$ or adjust $a$ and $b$, which will result in fewer patches in $\Pfine$, and therefore provide a faster integral evaluation, while maintaining accuracy.

In an attempt to strike this balance between speed and accuracy, we need certain constraints on the geometry of $\Gammah$ to ensure the efficient and accurate application of \qbkix, which we impose on the patch sets $\Pcoarse$ and $\Pfine$.
We will first outline our constraints on the quadrature patch sets $\Pcoarse$ and $\Pfine$ which allow for accurate evaluation with \qbkix.

\edit{}{We note here that this is largely a consequence of the global nature of our method.
We compute accurate potentials at each check point with a single large quadrature evaluation over the boundary.
Alternatively, one could use a \textit{local} approach which performs a local correction to an inaccurate \fmm evaluation of a singular/near-singular integral.
The net effect of this is a faster \fmm evaluation without much concern for where check points are placed, but many small quadratic computations on individual patches.
We explore this tradeoff by comparing \qbkix with a competing approach in \cref{sec:results-compare}.
}
\subsubsection{Admissibility criteria\label{sec:admissible}}
A set of patches $\qP$ is \textit{admissibile} if the following statements are satisfied on each quadrature patch in $\qP$:
\begin{criteria}
  \item The error of a surface patch $\vP_i$ approximating an embedding $\gamma_r$ is below some absolute target accuracy $\err{g}$ \label{criteria:1}
  \item The interpolation error of the boundary condition $f$ is below some absolute target accuracy $\err{f}$ \label{criteria:2}
  \item For each check center $\vhc_j$ corresponding to the quadrature point $\vy_j$ on the surface, the closest point on $\hat{\Gamma}$ to $\vhc_j$ is $\vy_j$. \label{criteria:3}
%  \item Each patch has characteristic length $L \geq L_\lbl{min}$.\label{criteria:4}
\end{criteria}

\Cref{criteria:1} is required to ensure that $\Gammah$ approximates $\Gamma$ with sufficient accuracy to solve the integral equation.
We discuss how to choose $\err{g}$ in \cite[Section 6]{morse2020bsupplementary}; for the tests in this paper, we simply choose $\err{g} < \err{target}$.
\Cref{criteria:2} guarantees that $f$ can be represented at least as accurately as the desired solution accuracy.
We therefore similarly choose $\err{f} < \etrg$.
%The parameters $a$ and $b$ in \cref{eq:check} are chosen to place check points to balance the extrapolation error, which grows as $a$ and $b$ increase, and smooth quadrature error, which grows as $a$ and $b$ decrease, while attempting to minimize cost.
\Cref{criteria:3}  balances the competing geometric constraints of cost and accuracy by flexibly placing check points as far as possible from $\Gammah$ without causing too much upsampling on other patches.
%Rather than checking all check point locations individually, we use the check center $\vhc$ as a proxy.
If a check point $\vc$ constructed from a surface patch $\vP$ is too close to another surface patch $\vP'$, \Cref{criteria:3} will indicate that $\vP$ is inadmissible.
If $\vP$ is subdivided into its children, new check points $\vc^\prime$ generated from these children of $\vP$ will be closer to $\vP$ and further from $\vP'$.
Since check points are placed at distances proportional to $L(\vP)$, repeated refinement of $\vP$ will eventually satisfy \Cref{criteria:3}.

\subsubsection{Upsampling criteria\label{sec:adaptive_upsampling}}
Once we have a set of admissible surface patches satisfying \Cref{criteria:1,criteria:2,criteria:3}, we need to determine the upsampled quadrature patches $\Pfine$ that ensure that the check points generated from $\Pcoarse$ are in $\Omega_I$, i.e., $\|u(\vc) - \hat{u}(\vc, \Pfine)\| < \etrg$.
%Once the set of admissible surface patches $\Pcoarse$ is computed, we need to determine the upsampled quadrature patches $\Pfine$ that ensure that the check points generated from $\Pcoarse$ are in $\Omega_I$, i.e., $\|u(\vc) - \hat{u}(\vc, \Pfine)\| < \etrg$.
To achieve this, we need a criterion to determine which patches are ``too close'' to a given check point for the error to be below $\err{target}$.
We make the following assumption about the accuracy of our smooth quadrature rule: \textit{\cref{eq:double_layer_disc} is accurate to $\err{target}$ at points further than  $L(\vP)$ from $\vP$, for $\err{target} > 10^{-12}$}.
This is motivated by \cite{aT2,barnett2014evaluation}, which demonstrate the rapid convergence of the layer potential quadrature error with respect to $\|\vx - \vsx\|_2$.
For sufficiently high quadrature orders, such as $q=20$, this assumption seems to hold in practice.
We say that a point $\vx$ is \textit{near} to $\vP$ if the distance from $\vx$ to $\vP$ is less than $L(\vP)$; otherwise, $\vx$ is \textit{far} from $\vP$.
We would like all check points required for the singular/near-singular evaluation of the discretization of \cref{eq:double_layer} using \qbkix to be far from all patches in $\Pfine$.
If this is satisfied, then we know that the Clenshaw-Curtis quadrature rule will be accurate to $10^{-12}$ at each check point.

\subsection{Refinement algorithm preliminaries}
Computing the distance from a check point to a given patch is a fundamental step in verifying the constraints on $\Pcoarse$ and $\Pfine$ from \cref{sec:admissible,sec:adaptive_upsampling}.
Before detailing our refinement algorithms to enforce these criteria, we introduce several geometric algorithms and data structures that will be used to compute the closest point on piecewise polynomial surfaces.

\subsubsection{\aabb trees\label{sec:aabb_trees}}
In order to implement our algorithms to enforce admissibility efficiently, we use a fast spatial data structure to find the patches that are close to a query point $\vx$.
In \cite{RKO, wala20193d}, the quadtree and octree within an \fmm is extended to support the geometric queries needed for a fast \qbx  algorithm.
In this work, we use an axis-aligned bounding box (\aabb) tree, which is a type of bounding volume hierarchy \cite{samet2006foundations}, implemented in \texttt{geogram} \cite{geogram}.
An \aabb is a tree with nodes corresponding to bounding boxes and leaves corresponding to bounding boxes containing single objects.
A bounding box $B_0$ is a child of another box $B_1$ if $B_0 \subset B_1$; the root node is a bounding box of the entire domain of interest.
Operations supported by \aabb trees include: (i) finding all bounding boxes containing a query point, (ii) finding all bounding boxes that intersect another query box, (iii) finding the closest triangle to a query point (because triangles have trivial bounding boxes).
By decoupling geometric queries from fast summation, the individual algorithms can be more thoroughly optimized, in exchange for the additional memory overhead of maintaining two distinct data structures.
The query algorithm presented in \cite{lu2019scalable} likely has better parallel scalability, but \aabb trees are faster for small to medium problem sizes on a single machine due to less redundant computation.

To define an \aabb tree for our patch-based surface $\Gammah$, we make use of the following fact: the control points of a B\'ezier surface ($\vector{a}_{\ell m}$'s from \cref{eq:tensor-product}) form a convex hull around the surface that they define \cite{F}.
As a result, we can compute a bounding box of a surface or quadrature patch $\vP$ directly from the B\'ezier coefficients simply by computing the maximum and minimum values of each component of the $\vector{a}_{\ell m}$'s, as shown in \cref{fig:patch-coeffs-bbox}-middle.
This bounding box can then be inserted into the \aabb tree as a proxy for a surface or quadrature patch.
\begin{figure}[!htb]
  \centering
  %\setlength\figureheight{1.9in}
  %\setlength\figurewidth{2.1in}
  \begin{minipage}{.33\textwidth}
      \includegraphics[width=\linewidth]{figs/patch_with_coeffs.png}
  \end{minipage}\hfill
  \begin{minipage}{.33\textwidth}
    \includegraphics[width=\linewidth]{figs/patch_with_coeffs_bbox.png}
  \end{minipage}\hfill
  \begin{minipage}{.33\textwidth}
    \includegraphics[width=\linewidth]{figs/patch_with_coeffs_near_bbox.png}
  \end{minipage}\hfill
  \mcaption{fig:patch-coeffs-bbox}{Relationship between control points and
  bounding boxes}{Left: a patch in the tensor product B\'ezier basis, with control
      points ($\vector{a}_{\ell m}$'s from \cref{eq:tensor-product}) plotted. The convex hull of the control points of a patch are guaranteed
      to contain the patch. Center: The patch bounding box, computed from the control
      points. Right: The near-zone bounding box of the patch from
      \cref{sec:adaptive_upsampling_algo} computed by
      inflating the bounding box by $L(\vP)$.
}
\end{figure}

\subsubsection{Computing the closest point to a patch\label{sec:closest_point_algo}}

To find a candidate closest patch $\vP_{i_0}$ to $\vx$, we construct a fine triangle mesh and bounding boxes of each patch in $\Pcoarse$ and insert them into an \aabb tree.
We can query the \aabb tree for the nearest triangle to $\vx$ with the \aabb tree, which corresponds to $\vP_{i_0}$.
We then compute the accurate true distance $d_{i_0}$ to $\vP_{i_0}$ using a constrained Newton method, presented in detail in \cref{app:closest_point}.

However, there may be other patches whose distance to $\vx$ is less than $d_{i_0}$, as shown in \cref{fig:candidate-near-patch}.
To handle this case, we then query the \aabb tree for all patches $\vP_{i_1}, \hdots, \vP_{i_k}$ that are distance at most $d_{i_0}$ from $\vx$.
This is achieved by forming a query box centered at $\vx$ with edge length $2d_{i_0}$ and querying the \aabb tree for all intersection bounding boxes.
The precise distance is then computed for each patch  $\vP_{i_1}, \hdots, \vP_{i_k}$ with \cref{app:closest_point} and the smallest distance is chosen.
We summarize this process in \cref{alg:closest_point}.

\begin{algorithm}[!htp]
    \KwData{A set of quadrature patches $\qP$, a query point $\vx$, Newton method tolerance $\err{opt}$}
  \KwResult{The closest point $\vsx$ on $\qP$ to $\vx$}

  \DontPrintSemicolon
  Construct an AABB tree $T_T$ from a fine triangle mesh of the quadrature patches of $\qP$\;
  Construct an AABB tree $T_B$ from bounding boxes of quadrature patches in $\qP$.\;
    $\tau_0 = $ closest triangle to $\vx$ computed with $T_T$ \;

    $\vP_{i_0} = $ patch corresponding to $\tau_0$\;
    Find the closest point $\vector{s}_{\vector{\vx},0}$ on $\vP_{i_0}$ to $\vx$ with \cite[Section 2]{morse2020bsupplementary}.\;
    $d_{i_0} = \|\vx - \vector{s}_{\vector{\vx},0}\|_2$\;
    $B_{d_{i_0}}(\vx)=$ a box centered a $\vx$ with edge length $2d_{i_0}$\;
    Find the boxes $B_{i_1}, \hdots B_{i_k}$ in $T_B$ that intersect $B_{d_{i_0}}(\vx)$\;

    \For{$B_{i_j} \in B_{i_1}, \hdots B_{i_k}$}{
      $\vP_{i_j} =$ quadrature patch corresponding to $B_{i_j}$ \;
      Find the closest point $\vector{s}_{\vector{\vx},j}$ on $\vP_{i_j}$ to $\vx$ with \cite[Section 2]{morse2020bsupplementary} to precision $\err{opt}$.\;
      $d_{i_j} = \|\vx - \vector{s}_{\vector{\vx},j}\|_2$\;
    }
    $j^* = \mathrm{argmin}_j\{d_{i_j}\}$ \;
    \Return{$\vector{s}_{\vector{x},j^*}$}
  \mcaption{alg:closest_point}{Compute the closest point to $\vx$}{}
\end{algorithm}

\begin{figure}[!htb]
  \centering
  %\setlength\figureheight{1.9in}
  %\setlength\figurewidth{2.1in}
  \hfill
  \begin{minipage}{.3\textwidth}
      \includegraphics[width=\linewidth]{figs/candidate_near_patch_trimesh.pdf}
  \end{minipage}\hfill
  \mcaption{fig:candidate-near-patch}{A \twod schematic of near-patch candidate selection}{
      A visual depiction of the quantities defined in lines 3-7 of \cref{alg:closest_point} (shown here in \twod for simplicity), with notation matching \cref{alg:point_marking}.
      The triangle-mesh proxy is drawn in as black lines and patches are drawn as gray curves.
      We have found an initial closest triangle $\tau_0$ to $\vx$ corresponding to patch $\vP_{i_0}$ and computed $d(\vx, \vP_{i_0}) = d_{i_0}$.
      We then query the \aabb tree for all patches that intersect box $B_{d_{i_0}}$ with edge length $2d_{i_0}$, shown in blue.
      There is clearly a patch that is closer to $\vx$ than $\vP_{i_0}$ that will be returned from the query, which will be distance $d_\lbl{min}$ from $\vx$.
}
\end{figure}

\subsection{Admissibility algorithm\label{sec:admissible_algo}}
Our algorithm to enforce \cref{criteria:1,criteria:2,criteria:3} proceeds as follows:
\begin{itemize}
    \item To enforce \Cref{criteria:1}, we adaptively fit a set of surface patches to the embeddings $\gamma_r$ representing $\Gamma$.
We construct a bidegree $(n,n)$ piecewise polynomial least-squares approximation $\vP_i$ in the form of \cref{eq:tensor-product} to $\gamma_r$ on $I^2$.
        If $\vP_i$'s domain $\mathcal{D}_i$ is obtained by refinement of $E_r$, we fit $\vP_i \circ \eta_i$ to $\gamma_r$ on $\I^2$, using $4n \times 4n$ samples on $\I^2$.
        If the pointwise error of $\vP_i$ and its partial derivatives is greater than $\err{g}$, then it is quadrisected and the process is repeated.

\item  Once the embeddings are resolved, we resolve $f$ on each surface patch produced from the previous step in a similar fashion to enforce \Cref{criteria:2}.
  However, rather than a least-squares approximation in this stage, we use piecewise polynomial interpolation.

\item To enforce \Cref{criteria:3}, we construct the set of check centers $\vhc_I$ which correspond to the check points required to evaluate the solution at the quadrature nodes $\vy_I$.
    For each check center $\vhc_I$, we find the closest point $\vector{s}_{\vhc_I} \in \Gammah$.
    If $ \|\vector{s}_{\vhc_I} - \vy_I\| \geq \err{opt}$, we split the quadrature patch $\vP$ containing $\vy_I$.
        The tolerance $\err{opt}$ is used in the Newton's method in \cite[Section 2]{morse2020bsupplementary}; we usually choose $\err{opt}=10^{-14}$.
Since $d(\vhc_I,\Gammah)$ is proportional to $L_{\vy_I}$, the new centers $\vhc_I$ for the refined patches  will be closer to the surface.
    We use \cref{alg:closest_point} to compute $\vector{s}_{\vhc_I}$.
        However, in the case of check points, we can skip lines 1-6 to compute $d_{i_0}$,  since $\vhc_I$ is $R + r(p+1)/2$ away from $\vy_I \in \vP(D)$ by construction.
        We can apply lines 7-14 of \cref{alg:closest_point} with $d_{i_0} = R + r(p+1)/2$ to compute $\vector{s}_{\vhc_I}$.
\end{itemize}

We summarize the algorithm to enforce \Cref{criteria:3} in \cref{alg:admissibility}.
At each refinement iteration, the offending patches are decreased by quadrisection, which reduces the distance from the quadrature point $\vy_I$ to its checkpoints.
This eventually satisfies \Cref{criteria:3} and the algorithm terminates.

\begin{algorithm}[!ht]
  \KwData{A set of quadrature patches $\qP$, optimization tolerance $\err{opt}$}
  \KwResult{An admissible set of quadrature patches $\qP$}

  \DontPrintSemicolon
  $\qP = \Pcoarse$\;
  Mark all patches in $\qP$ as inadmissible.\;

  \While{any patch in $\qP$ is inadmissible}{
    Construct an AABB tree $T$ as described in \cref{sec:closest_point_algo} from $\qP$\;
    \For{$\vP \in \qP$}{
        \If{$\vP$ is inadmissible}{
            Construct a set of check centers $C_\vP$ for each $\vy_J \in \vP(D)$ \;

            \For{$\vhc \in C_\vP$}{
                $d_{i_0} = R + r(p+1)/2$\;
                Compute $\vector{s}_\vhc$ with lines 7-14 of \cref{alg:closest_point} with precision $\err{opt}$ and $d_{i_0}$.\;
                %Construct a box $B(\vhc)$ with edge length $2R + r(p+1)$ centered at $\vhc$.\;
                %$B_{i_1}, \hdots B_{i_k}$ = \texttt{query\_bbox\_intersections}($T$, $B(\vhc)$\texttt)\;
                %$P_{i_1}, \hdots P_{i_k}$ =  patches corresponding to $B_{i_1}, \hdots B_{i_k}$\;
                %\eIf{$P \in \{P_{i_1}, \hdots P_{i_k}\}$}{
                %    Compute candidate closest points $\vector{s}_\vhc^{(1)}, \hdots \vector{s}_\vhc^{(k)}$ on $P_{i_1}, \hdots P_{i_k}$ to $\vhc$ with \cref{app:closest_point} to accuracy $\err{opt}.$\;
                %    $\vector{s}_\vhc = \mathrm{argmin}_i \|\vector{s}_\vhc^{(i)}, - \vhc\|_2$\;
                \eIf{$\|\vector{s}_\vhc -\vy_J\|_2 < \err{opt}$}{
                    Mark $\vP$ as admissible.\;
                } {
                    Mark $\vP$ as inadmissible.\;
                    break   \tcp{only need one bad check center to mark $\vP$ for refinement}
                }
            }
        }
    }
    \For{$\vP \in \qP$}{
      \If{$\vP$ is inadmissible}{
        Split $\vP$ into its four child patches, mark each as inadmissible, and replace $\vP$ with its children in $\qP$.
      }
    }
  }
  \Return{$\qP$}
  \mcaption{alg:admissibility}{Enforce admissibility \Cref{criteria:3} on a set of quadrature patches}{}
\end{algorithm}

\subsection{Adaptive upsampling algorithm \label{sec:adaptive_upsampling_algo}}
%Simply applying the point marking algorithm detailed \cref{app:point_marking} to each check point is not sufficient.
Before detailing our upsampling algorithm to satisfy the criteria outlined in \cref{sec:adaptive_upsampling}, we must define the notion of a \textit{near-zone bounding box} of a quadrature patch $\vP$, denoted $B_\lbl{near}(\vP)$.
The near-zone bounding box of $\vP$ is computed as described in \cref{sec:aabb_trees}, but then is inflated by $2L(\vP)$, as shown in \cref{fig:patch-coeffs-bbox}-right.
This inflation guarantees that any point $\vx$ that is near $\vP$ is contained in $B_\lbl{near}(\vP)$ and, for an admissible set of quadrature patches $\Pcoarse$, that any $\vx\in \Omega_N$ must be contained in some quadrature patch's near-zone bounding box.
This means that by forming $B_\lbl{near}(\vP)$ for each quadrature patch in $\Pfine$, a check point is in $\Omega_I$ if it is not contained in any near-zone bounding boxes.

To compute the upsampled patch set from $\Pcoarse$, we initially set $\Pfine = \Pcoarse$, compute the near-zone bounding boxes of each patch in $\Pfine$ and insert them into an \aabb tree.
We also construct the set of check points $C$ required to evaluate our discretized layer-potential with \qbkix (\cref{sec:singular-eval}).
For each check point $\vc \in C$, we query the \aabb tree for all near-zone bounding boxes that contain $\vc$.
If there are no such boxes, we know $\vc$ is far from all quadrature patches and can continue.
If, however, there are near-zone bounding boxes $B_{i_0},\hdots, B_{i_k}$ containing $\vc$, we compute the distances $d_{i_k}$ from $\vc$ to $\vP_{i_1},\hdots,\vP_{i_k}$ using \cref{app:closest_point}.
If $d_{i_k} < L(\vP_{i_k})$, we replace $\vP_{i_k}$ in $\Pfine$ with its four children produced by quadrisection.

To improve the performance of this refinement procedure, we allow for the option to skip the Newton method in \cref{alg:closest_point} and immediately refine all patches $\vP_{i_0},\hdots \vP_{i_k}$.
This is advantageous in the early iterations of the algorithm, when most check points are near to patches by design.
We allow for a parameter $n_\lbl{skip}$ to indicate the number of iterations to skip the Newton optimization and trigger refinement immediately.
We typically set $n_\lbl{skip}=2$.
We summarize our algorithm in \cref{alg:adaptive_upsampling}.
%We first use the error estimate of \cref{sec:quad_error_heuristic} to determine the distance from the intermediate field $\Omega_I$ to the patch $P$,  which we will call $d_\lbl{near}(P)$.
%We compute a bounding box of $P$ as described in \cref{sec:mark_near}.
%We compute a bounding box of each patch $P$ in $\Pfine$ as described in \cref{sec:aabb_trees}.
%We then inflate the box size by $2d_\lbl{near}$ to produce the \textit{near-zone bounding box} of $P$, denoted $B_\lbl{near}(P)$, as shown in \cref{fig:patch-coeffs-bbox}-right.
%We then inflate the each boxes' size by $2L(P)$ to produce the \textit{near-zone bounding box} of $P$, denoted $B_\lbl{near}(P)$, as shown in \cref{fig:patch-coeffs-bbox}-right.

%However, we need to still determine if a point $\vc$ in $B_\lbl{near}(P)$ is actually near to $P$, which requires computing the distance from $\vc$ to $\Gammah$.

%To check this efficiently, we insert all the near-zone bounding boxes into an \aabb tree.
%Let $C$ be the set of all check points required to evaluate our discretized layer potential.
%For each check point $\vc \in C$, we query the tree for all boxes containing $\vc$.
%The set of quadrature patches corresponding to the returned set of boxes are candidate patches for upsampling.
%We can now check the distance from $\vc$ to each of these quadrature patches using \cref{app:closest_point} and trigger refinement if the distance between $\vc$ and a given patch is less than $L$.
%Alternatively, one can also simply trigger refinement on all of the patches returned by the \aabb tree query without explicitly checking the distance.
%This is a cheaper operation that avoids the Newton iterations of \cref{app:closest_point}, but is less accurate and can cause over-refinement.
%One can either trigger upsampling on all of these patches or explicitly compute the distance from $\vc$ to each quadrature patch using .
%The former is a cheaper condition to check but less accurate; the latter is more precise and triggers less upsampling overall, but more expensive.
%This bounding box proxy is an over-approximation of $\Omega_N$; there are many cases in which a check point is contained in a near-zone bounding box, but strictly in $\Omega_I$.
%In these cases, the true distance from $\vc$ to each quadrature patch is more accurate.
%We strike a balance by triggering upsampling for all returned quadrature patches for the first iteration or two of upsampling and explicitly check distance to patches for the remaining iterations.

%The set $C$ is determined by $\Pcoarse$ and therefore fixed.
%The average size of near-zone bounding boxes decreases after each step of refinement until the algorithm terminates; eventually no near-zone bounding boxes will contain check points.
%We summarize in \cref{alg:adaptive_upsampling}.

\begin{algorithm}[!htp]
    \KwData{An admissible patch set $\qP$, number of iterations $n_\lbl{skip}$ before using \cite[Section 2]{morse2020bsupplementary}}
  \KwResult{An upsampled set of quadrature patches}

  \DontPrintSemicolon
  Compute inflated near-zone bounding boxes $B_1, \hdots,  B_N$ of each $\vP \in \qP$.\;
  Construct an AABB tree $T$ from the near-zone bounding boxes.\;
  Construct all check points $C$ required to evaluate the \cref{eq:int-eq} on $\qP$.\;

  $\qP_\lbl{fine} = \qP$\;
  Mark all check points in $C$ as near.\;
  $i=0$

  \While{any $\vc \in C$ is marked near}{
    \For{$\vc \in C$}{
        \If{$\vc$ is marked near}{
            Query $T$ for all bounding boxes $B_{i_1}, \hdots B_{i_k}$ containing $\vc$.\;
            $\vP_{i_1}, \hdots \vP_{i_k} = $ patches corresponding to boxes $B_{i_1}, \hdots B_{i_k}$\;
            Mark $\vc$ as far\;
            \For{$\vP \in \vP_{i_1}, \hdots \vP_{i_k}$}{
                \eIf{$i > n_\lbl{skip}$}{
                    Find the closest point $\vector{s}_{\vc}$ on $\vP$ to $\vc$ with \cref{alg:closest_point}.\;
                    %\If{The error estimate in \cref{sec:quad_error_heuristic} is greater than $\eps$ with $d = \|\vy - \vc\|_2$}{
                    \If{ $\|\vector{s}_{\vc} - \vc\|_2 < L(\vP)$}{
                        Split $\vP$ and replace it in $\qP_\lbl{fine}$ with its children. \;
                        Mark $\vc$ as near\;
                    }
                } {
                    Split $\vP$ and replace it in $\qP_\lbl{fine}$ with its children.\;
                    Mark $\vc$ as near\;
                }

            }
       }
    }
    $i=i+1$\;

  }
    \mcaption{alg:adaptive_upsampling}{Adaptively upsample to accurately evaluate \cref{eq:double_layer_disc} at check points}{}
\end{algorithm}

%In \cite{RKO}, the panel size of the quadrature rule was tied to the distance of the \qbx expansion center from the boundary.
%This design choice produces an artificial dependence of the quadrature and expansion error, since the global upsampling factor for the fine discretization is determined by the closest expansion to the boundary.
%\cite{wala20193d} first presented an approach that decouples the coarse and fine discretizations; we follow a similar approach here.
%However, their approach does not explicitly depend on surface curvature, which has a dramatic impact on quadrature accuracy, which can lead to unnecessary upsampling.
%By taking advantage of the error heuristic in \cref{sec:error}, we are able to accurately determine quadrature accuracy at a given check point.

\subsection{Marking target points for evaluation\label{app:point_marking}}

Once we have solved \cref{eq:linear_system} for $\phi$ on $\Gammah$, we need the ability to evaluate \cref{eq:double_layer} at an arbitrary set of points in the domain.
For a target point $\vx$, in order apply the algorithm in \cref{sec:singular-eval}, we need to determine whether or not $\vx \in \Omega$ and, if so, whether $\vx $ is in $\Omega_N, \Omega_I$ or $\Omega_F$.
Both of these questions can be answered by computing the closest point $\vsx$ on $\Gammah$ to $\vx$. % with \cref{app:closest_point_opt}.
If $\vn(\vsx)\cdot(\vx - \vsx) < 0$, then $\vx \in \Omega$.
As we have seen in \cref{sec:adaptive_upsampling}, the distance $\|\vx - \vsx\|$ determines whether $\vx \in \Omega_N,\Omega_I$ or $\Omega_F$.
However, for large numbers of target points, a brute force calculation of closest points on $\Gammah$ to all target points is prohibitively expensive.
We present an accelerated algorithm combining \cref{alg:closest_point} and an \fmm evaluation to require only constant work per target point.

\subsubsection{Marking and culling far points \label{sec:mark_far}}
A severe shortcoming of \cref{alg:closest_point} is that its performance deteriorates as the distance from $\vx$ to $\Gammah$ increases.
Consider the case where $\Gammah$ is a sphere with radius $r$ with $\vx$ at its center.
The first stage of \cref{alg:closest_point} returns a single quadrature patch that is distance $r$ from $\vx$; the next stage will return all quadrature patches.
This will take $O(N)$ time to check the distance to each patch.
Even on more typical geometries, we observe poor performance of \cref{alg:closest_point} when $\vx$ is far from $\Gammah$.

%It's important that we can mark points in $\Omega_F$ and mark points as inside or outside $\Omega$ without using \cref{sec:mark_near}.
To address this, we use an additional \fmm-based acceleration step to mark most points far from $\Gammah$ before using applying \cref{alg:closest_point}.
Our approach is based on computing the generalized winding number \cite{jacobson2013robust} of $\Gammah$ at the evaluation points.
For closed curves in $\mathbb{R}^2$, the \textit{winding number} at a point counts the number of times the curve travels around that point.
The \textit{generalized winding number} of a surface $\Gammah$ at a point $\vx \in \mathbb{R}^3$ can be written as
%\begin{equation}
%\omega_{\Gammah}(\vx) = %\frac{1}{4\pi} W_S(\vx) =
%  \frac{1}{4\pi}\iint_{\Gammah} sin(\phi)d\phi d\theta, \quad (\text{when }\vx =0).
%  \label{eq:gen_winding}
%\end{equation}
%%where $W(\vx)$ is the solid angle subtended by $S$.
%%This is the projection of $S$ onto the unit sphere centered at $\vx$.
%The integrand can be interpreted as the signed differential solid angle subtended by an infinitesimal surface patch centered at $\vx$.
%
%In our case, $\Gammah$ is composed of a collection of surface patches with independent parametrizations, and can be computed patch by patch.
%%It's clear that the solid angle of $\Gammah$ with respect to $\vx$ is the sum of the solid angles of the surface patches:
%\begin{equation}
%  \omega_{\Gammah}(\vx) = \frac{1}{4\pi}\sum_i \iint_{\gamma_i} sin(\phi)d\phi d\theta.
%  \label{eq:gen_winding_patch}
%\end{equation}
%By a change of variables to Cartesian coordinates, we can rewrite \cref{eq:gen_winding} as

\begin{equation}
  \omega_{\Gammah}(\vx) = -\frac{1}{4\pi}\int_{\Gammah} \frac{(\vx - \vy) \cdot \vn}{\|\vx - \vy\|^3} d\vy_{\Gammah}
  \label{eq:gen_winding2}
\end{equation}
We recognize this integral as the double-layer potential in \cref{eq:double_layer} for a Laplace problem with $\phi = 1$.
Its values in $\mathbb{R}^3$ are  \cite{K}:
\begin{equation}
  \omega_{\Gammah}(x) =
  \begin{cases}
    1 & \vx \in \Omega \setminus \Gammah\\
    1/2 & \vx \in \Gammah\\
    0 &  \vx \in \mathbb{R}^3 \setminus \overline{\Omega}
  \end{cases}
  \label{eq:const-density}
\end{equation}
\cref{eq:gen_winding2} can be evaluated using the same surface quadrature in \cref{eq:double_layer_disc} using an \fmm in $O(N)$ time.
While the quadrature rule is inaccurate close to the surface, $\Omega_F$ is defined precisely as the zone where the quadrature rule is sufficiently accurate.
%The rule is accurate far from $\Gamma$, which is exactly where we want to mark target points.
For this reason, we use
\begin{equation}
  |\omega_{\Gammah}(\vx) -1| < \etrg
  \label{eq:marking-far}
\end{equation}
to mark points $\vx \in \Omega_F \subset \Omega$ and a similar relation
\begin{equation}
  |\omega_{\Gammah}(\vx)| < \etrg
  \label{eq:marking-out}
\end{equation}
to mark points $\vx \not \in \Omega$.
This approach is similar in spirit to the spectrally accurate collision detection scheme of \cite[Section 3.5] {QB}.
Unlike \cite{QB}, however, we do \textit{not} use singular integration to mark all points.
This isn't possible since at this stage since we do not yet know which target points require singular integration.
We use the \fmm evaluation purely as a culling mechanism before applying the full marking algorithm.

\noindent\textbf{Remark:} Since the quadrature rule may be highly inaccurate for points close to the surface, due the near-singular nature of the integrand, $\omega_{\Gammah}(\vx)$ may happen to be close to one or zero.
We highlight that it is possible that points outside $\Omega_F$ may be mismarked, although we have not observed this in practice.

%We use \cref{eq:const-density} simply as a filter for far target points and mark the remaining points as described in \cref{sec:mark_near}.

\subsubsection{Full marking algorithm}
We combine the algorithms of the previous two sections into a single marking pipeline for a general set of target points in $\mathbb{R}^3$, by first applying the algorithm of \cref{sec:mark_far} to mark all points satisfying  \cref{eq:marking-far} then passing the remaining points to \cref{alg:closest_point}.
The full marking algorithm is summarized as \cref{alg:point_marking}.

\begin{algorithm}
  \KwData{An admissible set of quadrature patches $\qP, \etrg$, target points $\vX$}
  \KwResult{A marked set of target points $\vX$}

  \DontPrintSemicolon
  $\phi_0 = 1$\;
  $\omega_{\Gammah} =$ \texttt{Laplace\_FMM}($\qP$, $\vX$, $\phi_0$)\;

  \For{$\vx \in \vX$}{
    \uIf{$|\omega_{\Gammah}(\vx) - 1| < \etrg$}{
      Mark $\vx$ as inside $\Omega$.\;
      Mark $\vx$ as in $\Omega_\lbl{F}$.\;
    }
    \uElseIf{$|\omega_{\Gammah}(\vx)| < \etrg$}{
      Mark $\vx$ as outside $\Omega$.\;
      %Mark $\vx$ in $\Omega_\lbl{F}$.\;
    }
  }
  \For{$\vx \in \vX$}{
    \If{$\vx$ is unmarked}{
      %Find the closest triangle $\tau_0$ to $\vx$ using $T_T$. \;
      %$P_{i_0} = $ patch corresponding to $\tau_0$\;
      %Compute the distance $d_{i_0}$ from $\vx$ to $P_{i_0}$.\;
      %$B_{d_{i_0}}(\vx)=$ a box centered a $\vx$ with edge length $2d_{i_0}$\;
      %Find the boxes $B_{i_1}, \hdots B_{i_k}$ in $T_B$ that intersect $B_{d_{i_0}}(\vx)$\;

      %\For{$B_{i_j} \in B_{i_1}, \hdots B_{i_k}$}{
      %  $P_{i_j} =$ quadrature patch corresponding to $B_{i_j}$ \;
      %  Compute the distance $d_{i_j}$ from $\vx$ to $P_{i_j}$.
      %}
      %$d_\lbl{min} = \min_j\{d_{i_j}\}$ \;
        Compute the closest point $\vsx$ to $\vx$ with \cref{alg:closest_point}\;
        $d_\lbl{min} = \|\vsx -\vx\|_2$\;
        \eIf{$d_\lbl{min} \leq L_{\vsx}$}{
            Mark $\vx$ as in $\Omega_N$\;
        }{
            Mark $\vx$ as in $\Omega_I$\;
        }
        \If{$\vn(\vsx)\cdot(\vx-\vsx) < 0$}{
            Mark $\vx$ as inside $\Omega$\;
        }{
            Mark $\vx$ as outside $\Omega$\;
        }
    }
  }
  \mcaption{alg:point_marking} {Mark points in regions $\Omega_F$, $\Omega_I$ and $\Omega_N$}{}

\end{algorithm}

%\subsection{Comparison with \cite{wala20193d,wala2019optimization} \label{app:comp_wala}}
%\note[MJM]{move to aux doc}
%Our work most closely resembles the advancements presented in \cite{wala20193d, wala2019optimization}.
%We have presented a \textit{global} singular/near-singular quadrature method, i.e., the potential values at the check points are computed with a quadrature rule from the entire boundary.
%\cite{wala20193d} proposed a global \qbx method that computes \qbx expansion coefficients via \fmm translation operators from within an \fmm tree.
%Our method is \textit{target-specific} as in \cite{ST}, creating one set of check points for each target point.
%\cite{wala20193d} was further refined to include target-specific \qbx expansions in \cite{wala2019optimization}.
%
%Our admissibility algorithm is similar to the Stage-1 refinement of \cite{wala20193d}.
%Both approaches first resolve the boundary data and input geometry, then enforce a criteria that will guarantee accurate smooth quadrature rules at prescribed point locations.
%The improvement in our approach is the decoupling of the spatial data structure for the required geometry queries to enforce admissibility and the data structure for \fmm acceleration.
%This allows for less memory overhead and faster spatial queries and \fmm evaluations by leveraging existing software packages.
%Additionally, our algorithm is formulated in terms of patches and bounding boxes rather than in terms of quadrature point locations.
%This allows us to perform fewer spatial queries on a smaller data structure to enforce our criteria and make guarantees about the proximity of a patch to a check point that is independent of the quadrature order.
%As in \cite{wala20193d}, we also fix the check point location before upsampling, which decouples the coarse and upsampled discretization.
%We both compute upsampled discretizations based on empirical heuristics to approximate quadrature error behavior.
%
%However, the primary improvement of \qbkix over \cite{wala20193d} is \textit{algorithmic simplicity}.
%Our only requirement is a standard point \fmm without modifications.
%This allows us to utilize existing optimized algorithms for spatial queries and fast summation, which have been extensively optimized.
%Most importantly, it prevents the \qbx-\fmm error coupling handled carefully in \cite{wala20193d,wala2019optimization}. %with the target confinement rule, which constrains a \qbx expansion to reside within an appropriately sized \fmm box to prevent error accumulation due to numerical differentiation.
%The price we must pay for this simplicity is a larger point \fmm evaluation, since we are using the discretization of $\Pfine$ as source points.
%Since we are using the kernel-independent \fmm, we must use a higher multipole order to counteract the accumulation of translation operator error inherent in this approach \cite{ying2004kernel}.
%A standard \fmm method would not have this downside, but we believe that \pvfmm's impressive performance optimizations make this is reasonable trade-off.
%


\subsection{Threat Log Comparison}\label{sec:threatlog}

Another motivating application is referred to as threat log comparison where multiple organizations share data about current attacks on their computer networks. The goal of sharing this data is to allow the participating parties to identify and stop threats in a more timely manner. Facebook has a service called ThreatExchange\cite{threat} which provides this functionality. One drawback of the Facebook approach is that all of the data is collected on their servers and is often viewable by the other participants. This architecture inherently relies on trusting Facebook with this data.

We propose using our distributed protocol to provide a similar functionality while reducing the amount of trust in any single party, e.g. Facebook. In this setting we consider a moderate number of parties each holding a dataset containing the suspicious events on their network along with possible meta data on that event, e.g. how many times that event occurred. All of the parties input these sets into our join framework where the occurrences  of each event type are counted. An example of such an event is the IP address that makes a suspicious request.

There are at least two ways to securely compute the occurrences of these events. One method is to perform a full join of all the events where the counts are added together during each join. The resulting table would contain all of the events and the number of times that each event occurred. The drawback of this approach is that each full joins require performing a left join followed by a union, twice the overhead compared to other join operations.

Now consider a different strategy for this problem.  First, the parties can compute and reveal the union of the events. Given this information the parties can locally compute the number of times this event occurred on their network and secret share this information between the parties. The parties then add together this vector of secret shared counts and reveal it.

One shortcoming of this approach is no ability to limit which events are revealed. For example, it can be desirable to only reveal an event if it happens on $k$ out of the $n$ networks. This can be achieved by having the parties compute and reveal the randomized encodings for all of the items in the union, instead of the items themselves. Under the same encoding key, each party holding a set employs the three server parties to compute the randomized encodings for the items in their set. These encodings are revealed to the party holding the set. For each encoding in the union, the parties use the MPC protocol to compute the number of occurrences that event had and conditionally reveal the value. For example, if at least $k$ of the networks observed the event. Other computation on meta data can also be performed as this stage.


\section{Methods and Implementation}
\label{sec:methods}
\fred implements a minimal user interface, Figure \ref{fig:surgery_fred}, to demonstrate registration of a pre-operative image to intra-operative space. At startup a target is placed at a random location within the pre-operative image, shown as a red circle. The standard deviation of the \gls{FLE} is randomly sampled from a uniform distribution
between 0.5 and 5.0 pixels \footnote{All units are in pixels, as the concepts being explored do not depend on the units used. Figure
\ref{fig:surgery_fred} shows a brain MRI 458x512 pixels, by design SciKit-SurgeryFRED should work with other images
of arbitrary dimensions.}. By default the \gls{FLE} is modelled as an isotropic (in 3 dimensions), normally distributed, and independent random variable, though this can be easily changed.
The variance of the \gls{FLE} is shown to the right of the intra-operative image (as expected value), along with the number of fiducial markers.

\begin{figure}
	\begin{center}
	\includegraphics[width=\linewidth]{scikit-surgeryfred_gui.eps}
		\caption{\label{fig:surgery_fred}SciKit-SurgeryFRED graphical user interface after 6 fiducial markers placed. The red circle in the pre-operative image (left) represents a clinical target, which is located in the
		intra-operative space (middle) by fiducial based registration, using the fiducial markers (also red). FLE is added to each marker in the intra-operative image, this is more clearly visible on the zoomed in image at the right. The resulting registration results in a TRE, shown by the misalignment of the red circle and crosshair
		on the enlarged image at right. TRE and other statistics are shown in boxes to the right of the intra-operative image.}
	\end{center}
\end{figure}

Clicking either image adds a fiducial marker to both images. By default the marker is added to the pre-operative image
with no \gls{FLE}. \gls{FLE} is added to the marker location in the intra-operative image, visualised as the misalignment
between the red circle centre and the cross-hair. Once sufficient markers are placed ($>2$) the two sets of markers are registered using least squares fitting as described by Arun et al.\cite{Arun1987}. The expected values (variance) of the \gls{FRE} and \gls{TRE} are calculated
using equations described by Fitzpatrick et al.\cite{Fitzpatrick1998}. The student can use this interface together with an
online tutorial
to explore the relationships between the various statistics and error measures. The
user can keep adding as many fiducial markers as they like for a given target.
Pressing the ``New Target'' button will place a new target at a random position
within the pre-operative image and randomly sample a new standard deviation for the
\gls{FLE}.

It is straightforward to explore how both \gls{TRE} and \gls{FRE} change in response to
both the number and geometry of the fiducial markers as is well established in the literature\cite{1295074, Fitzpatrick1998}.
It is
also simple and instructive to create degenerate marker geometries, for example
a linear arrangement can produce low \gls{FRE} but extreme values of \gls{TRE}.

During use the registration results are
stored in the browser. At any time the user can press the ``Plot Results'' button to quickly create a set of
plots showing the relationship between the actual \gls{TRE} and the various statistics.
An example plot is shown in Figure \ref{fig:correlation}.
The student is usually able to see first hand that, as expected, \gls{FRE} is uncorrelated with \gls{TRE}.

\begin{figure}
	\begin{center}
	\includegraphics[width=0.9\linewidth]{images/default.eps}
		\caption{\label{fig:correlation}Plots of TRE against various error measures, generated by
		205 user registrations.}
	\end{center}
\end{figure}

\fred does not implement tests of statistical significance to avoid excessive software dependencies, so it is often
useful for the students to download the registration results and perform some statistical analysis. This is facilitated by the
``Download Results'' button which allows the user to get the results as a file in comma separated variable format.
For example the results in Figure \ref{fig:correlation} may prompt some students to ask if there is in fact a correlation
between \gls{FRE} and \gls{TRE}, as there is an apparent though slight increase in \gls{TRE} with \gls{FRE}. This can quickly
be dispelled by asking the student to download the results and perform a test of significance on the data.

\subsection{Game Based Registration Study}
\label{sec:game_method}
Once the students have an understanding of what statistics can be used to estimate \gls{TRE}, we wanted to test
how knowledge of a particular statistic affects optimal treatment planning. We designed a serious game to
do this. During this game the students are asked to perform a registration with a maximum of
6 fiducial markers, then set a treatment margin and ablate the target. The goal is
to treat 100\% of the target with minimal ablation of surrounding tissue.
A score of 1000 was awarded for complete ablation and 0
for anything less. From this starting score 10 times the percentage volume of any
surrounding tissues ablated is subtracted.
Successful treatments (100\% ablation) should therefore score between 0 and 1000, with a larger margin giving a lower score.
The optimum (minimum successful) treatment
margin is the actual \gls{TRE} for a given registration.
Any treatment that fails to treat 100\% of the target will receive a negative score.

Each student participated in the game, performing 20 simulated ablations. For the first four ablations they were told the
actual \gls{TRE} for training and to supply baseline data. After that they performed 16 more ablations and were shown one of four randomly selected statistics on which to base their decision,
the expected values of the \gls{TRE}/\gls{FRE}, the actual \gls{FRE}, the expected value of the \gls{FLE}. These statistics
were chosen as these are either known or can be estimated for a clinical procedure. Each statistic was shown 4 times, though
the order was randomised for each participant. The scores for each ablation were recorded,  yielding 20 data points for
each participant.

\subsection{Software Implementation}
\fred is part of the \sksurgery\cite{PMID:32436132} family of libraries. In common with \sksurgery the majority of \fred is implemented in Python. Python was chosen as it combines sufficient features for clinical
applications such as the SmartLiver system\cite{PMID:32780240}, whilst remaining easy enough for students to learn and contribute to.
In contrast, although platforms built using {C\raisebox{0.5ex}{\tiny\textbf{++}}} provide power and
flexibility, the choice of language creates a barrier to learning the key concepts of image guided surgery \cite{surgineering}. A key design goal of
\sksurgery is to keep individual libraries compact and orthogonal\cite{pragmaticprog}, simplifying dependency structures. Based on
analysis using cloc\footnote{\href{https://github.com/AlDanial/cloc}{https://github.com/AlDanial/cloc} [v1.82]} \fred consists of 1479 lines of Python
code. The user interface is implemented in HTML5 and JavaScript, enabling multiple simple deployment
options. Again, using cloc, there are 957 lines of HTML5 and JavaScript. These numbers are similar to the other \sksurgery libraries which typically have around 2000 lines of code \cite{PMID:32436132}.
%In comparison, this paper consists of 431 lines of Latex code.

Figure \ref{fig:dependencies} shows the direct dependencies of \fredns. The key functional dependency is
\core\cite{matt_clarkson_2020_3965731}. \core implements matched point based registration \cite{Arun1987} together
with the calculation of expected \gls{FLE} and \gls{TRE}
(equations 10 and 31 from Fitzpatrick et al.\cite{Fitzpatrick1998}).
{NumPy} \cite{2020NumPy-Array} is used for array handling.
Flask\footnote{\href{https://palletsprojects.com/p/flask/}{https://palletsprojects.com/p/flask/}  [v1.1.2]} provides the web application framework
to enable the browser based user interface to communicate with the Python based back end. The user
interface communicates with the back end with a series of {POST} requests. All state information is stored in the
browser front end, allowing the back end to remain stateless, simplifying deployment.
Including the Google Cloud FireStore API\footnote{\href{https://pypi.org/project/google-cloud-firestore/}{https://pypi.org/project/google-cloud-firestore/} [v2.0.1]}
enables the optional storage of results in a remotely hosted database. Plotting functionality is implemented
using Chart.js\footnote{\href{https://www.chartjs.org/}{https://www.chartjs.org/} [v2.9.4]}

\begin{figure}
	\begin{center}
	\includegraphics[width=0.7\linewidth]{dependency_graph.eps}
		\caption{\label{fig:dependencies}Software dependencies of SciKit-SurgeryFRED. The registration algorithms and statistical measures of {TRE} are imported from SciKit-SurgeryCore and are shared with clinical applications built on SciKit-Surgery. The user interface is a Flask based web application, using Chart.js for plotting functionality.}
	\end{center}
\end{figure}

In common with the rest of \sksurgery \fred utilises extensive testing and software process \cite{1398621} to
ensure the application is robust, reusable, and sustainable\cite{VENTERS2018174}. Change control and issue tracking is managed
on GitHub\footnote{\href{https://github.com/UCL/scikit-surgeryfred}{https://github.com/UCL/scikit-surgeryfred}} and continuous integration is managed using
GitHub Actions.

\subsection{Availability and Usage}
For those who want to quickly try \fredns, we currently maintain a running instance hosted at \href{https://scikit-surgeryfred.ew.r.appspot.com/}{https://scikit-surgeryfred.ew.r.appspot.com/}. This should be accessible from most modern web browsers.

If you want to run a locally hosted instance, or modify the code for your needs, you can download the source code. \fred is entirely open source software and is tested on Linux, MacOS, and Windows. The latest version can be obtained from Github. Alternatively, archived releases can be retrieved via Zenodo\cite{stephen_thompson_2021_4462897}.

\begin{lstlisting}[language=bash]
	git clone https://github.com/UCL/scikit-surgeryfred
\end{lstlisting}

Once installed the dependencies can be installed on a local virtual machine using tox. The application can then be run with as follows. This should output a web address that you can open in a browser to run the software.

\begin{lstlisting}[language=bash]
	cd scikit-surgeryfred
	tox
	source .tox/py37/bin/activate
	python main.py
\end{lstlisting}

\subsection{Extension to Anisotropic Errors}
\label{sec:anis_method}
In many cases {FLE} cannot be properly modelled as an isotropic, independent, random variable. In the case of optical tracking systems for example \cite{10.1117/12.536128} the errors normal to the camera plane are approximately 3 times those parallel to the camera plane. It is straightforward to implement such an anisotropic \gls{FLE} in SciKit-SurgeryFRED and test the effect on registration outcomes.
The following code snippet is taken from  main.py. Line 66 defines the ratio of \gls{FLE} in three directions. By default they are all equal.

\begin{lstlisting}[language=python, firstnumber = 57]
@app.route('/getfle', methods=['POST'])
def getfle():
    """
    Returns values for fiducial localisation errors
    Values are randomly selected from a uniform
    distribution from 0.5 to 5.0 pixels
    """
    fle_sd = np.random.uniform(low=0.5, high=5.0)
    #change fle_ratio if you want anisotropic fle
    fle_ratio = np.array([1.0, 1.0, 1.0], dtype=np.float64)
    anis_scale = math.sqrt(3.0 / (np.linalg.norm(fle_ratio) ** 2))
    fixed_fle = fle_ratio * fle_sd * anis_scale

    moving_fle = np.array([0., 0., 0.], dtype=np.float64)
    fixed_fle_eavs = expected_absolute_value(fixed_fle)
    moving_fle_eavs = expected_absolute_value(moving_fle)

    returnjson = jsonify({
            'fixed_fle_sd': fixed_fle.tolist(),
            'moving_fle_sd': moving_fle.tolist(),
            'fixed_fle_eav': fixed_fle_eavs.tolist(),
            'moving_fle_eav': moving_fle_eavs.tolist()
            })
    return returnjson
\end{lstlisting}

Changing line 66 to;
\begin{lstlisting}[language=python, firstnumber=66]
    fle_ratio = np.array([3.0, 1.0, 1.0], dtype=np.float64)
\end{lstlisting}
sets the error in the x direction to 3 times that in the y and z directions.

\subsection{Addition of Systematic Errors}
\label{sec:sys_method}
A second significant source of error that is usually overlooked is the presence of systematic errors. For example when using a tracked pointer for fiducial localisation,
any pointer calibration error will be be added to all fiducial markers.
Similarly some optical tracking systems can introduce a systematic error on
the tracking markers \cite{6294449}. \fred allows systematic error to
be added to each fiducial. The fiducial localisation error is set within
the JavaScript function init{\textunderscore}fles()
(defined in static/main.js) each time a new target is set.
\begin{lstlisting}[language=java, firstnumber = 496]
/**
 * Sets the global fiducial localisation error (FLE)
 */
function init_fles() {
  fetch("/getfle", {
      method: "POST",
    })
    .then(resp => {
      if (resp.ok)
        resp.json().then(data => {

        let preOpFLEStdDev = data.moving_fle_sd;
        let intraOpFLEStdDev = data.fixed_fle_sd;
        let preOpFLEEAV = data.moving_fle_eav;
        let intraOpFLEEAV = data.fixed_fle_eav;

        let preOpSysError = [0.0, 0.0, 0.0];
        let intraOpSysError = [0.0, 0.0, 0.0];

        FLE = { preOpFLEStdDev, intraOpFLEStdDev,
                preOpFLEEAV, intraOpFLEEAV,
                preOpSysError, intraOpSysError };

        expectedFLEText.innerHTML = Math.round(Math.sqrt(FLE.intraOpFLEEAV)*100)/100;
      });
    })
    .catch(err => {
      console.log("An error occured setting fles", err.message);
    });
}
\end{lstlisting}
By default there is no systematic error, (lines 512 and 513). We can
add a systematic interoperative error at line 513 as;

\begin{lstlisting}[language=java, firstnumber = 513]
        let intraOpSysError = [1.0 * (Math.random() - 0.5),
                               1.0 * (Math.random() - 0.5),
                               1.0 * (Math.random() - 0.5)];
\end{lstlisting}

In this case the error is an isotropic uniform random variable, in the range
-0.5 to 0.5. This error will be applied to all fiducial markers for a
given registration.


\section{Future Work}
\label{sec:Future}

Many studies focused on how to minimize the amount of computation in the
encoding process, such as changing the recursion definition formula.  We
can use the transformation of several register state instead of the
memory access.  These methods are used off-line encoding. In the case of
the maximum value of the suffix,  our approach still does not apply all
the transformation in registers.  If we can change the mathematical
definition to reduce memory access, the performance will be more
significantly improved.


\documentclass{article}
\usepackage[utf8]{inputenc}

\title{homework 4, task 2}
\author{Alexander Smirnov}
\date{March 2019}

\usepackage{natbib}
\usepackage{graphicx}

\begin{document}

\maketitle

\section{What to do}
Prove that $S\:K\:K = I: $

$$(\lambda x\:y\:z.x\:z\:(y\:z))\:(\lambda x\:y.x)\:(\lambda x\:y.x)=\lambda x.x$$

\section{Solution}

$$(\lambda x\:y\:z.x\:z\:(y\:z))\:(\lambda x\:y.x)\:(\lambda x\:y.x)$$

$$(\lambda x.\lambda y\:z.x\:z\:(y\:z))\:(\lambda x\:t.x)\:(\lambda x\:y.x)$$

$$(\lambda y\:z.(\lambda x\:t.x)\:z\:(y\:z))\:(\lambda x\:y.x)$$

$$(\lambda y\:z.z)\:(\lambda x\:y.x)$$

$$(\lambda y.\lambda z.z)\:(\lambda x\:y.x)$$

$$\lambda z.z$$

\end{document}


\subsection{Build RAVEN Input: \xmlNode{MultiRun}}
\label{sub:tutorialMultiRun}
The \textbf{MultiRun} step allows the user to assemble the calculation flow of an analysis that requires multiple
``runs'' of the same model. This step is used, for example, when the input (space) of the model needs to be
perturbed by a particular sampling strategy. In the \xmlNode{MultiRun} input block, the user needs to specify the
objects that need to be used for the different allowable roles. This step accepts the following roles:

\begin{itemize}
  \item \textit{Input}
  \item \textit{Model}
  \item \textit{Output}
  \item \textit{Sampler}
  \item \textit{Optimizer}
  \item \textit{SolutionExport}
\end{itemize}

\textbf{MultiRun} is intended to handle calculations that involve multiple runs of a driven code (sampling strategies).
Firstly, the RAVEN input file associates the variables to a set of PDFs and to a sampling strategy. The ``multi-run''
step is used to perform several runs in a block of a model (e.g. in a MC sampling).

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\textwidth]  {pics/MultiRunCalculationFlow.png}
  \caption{Calculation flow for a multi-run sampling}
  \label{fig:multiRun}
\end{figure}

As shown in Figure~\ref{fig:multiRun}, at the beginning of each sub sequential run, the sampler provides the new values of the variables to be perturbed.
The code API places those values in the input file. At this point, the code API generates the run command and asks
to be queued by the job handler. The job handler manages the parallel execution of as many runs as possible within
a user prescribed range and communicates with the step controller when a new set of output files are ready to be
processed. The code API receives the new input files and collects the data in the RAVEN internal format. The sampler
is queried to assess if the sequence of runs is ended, if not, the step controller asks for a new set of values
from the sampler and the sequence is restarted.
The job handler is currently capable to run different run instances of the code in parallel and can also handle
codes that are multi-threaded or using any form of parallel implementation.
RAVEN also has the capability to plot the simulation outcomes while the set of sampling is performed and to store
the data for later recovery.

In this section, we will show the user how to set up the \textit{Sampler}, and employ \textbf{MultiRun} to execute
all perturbed models. The use of \textit{Optimizer} and \textit{SolutionExport} will be introduced in another section.
The \textbf{Samplers}  entity is the container of all the algorithms designed to perform the perturbation of the
input space. The Samplers can be categorized into three main classes:

\begin{itemize}
  \item  \textit{Forward}. Sampling strategies that do not leverage the information coming from already evaluated
    realizations in the input space. For example, Monte-Carlo, Stratified (LHS), Grid, Response Surface, Factorial Design,
    Sparse Grid, etc.
  \item  \textit{Adaptive}. Sampling strategies that take advantages of the information coming from already evaluated
    realizations of the input space, adapting the sampling strategies to key figures of merits. For example, Limit Surface
    search, Adaptive sparse grid, etc.
  \item \textit{Dynamic Event Tree}. Sampling strategies that perform the exploration of the input space based on the
    dynamic evolution of the system, employing branching techniques. For example, Dynamic Event Tree, Hybrid
    Dynamic Event Tree, etc.
\end{itemize}

The sampler is probably the most important entity in the RAVEN framework. It provides many different sampling
strategies that can be used in almost all RAVEN related applications. In this section, we will only illustrate
the simplest forward sampler, i.e. \textit{Monte-Carlo}, to familarize the user with the use of sampler. Monte-Carlo method
is one of the most-used methodologies in several mathematic disciplines. The theory of this method can be found in
the RAVEN theory manual. In addition, we will continue to use the \textbf{AnalyticBateman} to illustrate the setup
of \textbf{MultiRun} and \textbf{Samplers}. In order to accomplish these tasks, the following precedures or RAVEN
entities are needed:

\begin{enumerate}
  \item Set up the running environment: \xmlNode{RunInfo}
  \item Provide the required files: \xmlNode{Files}
  \item Link between RAVEN and dirven code: \xmlNode{Models}
  \item Define probability distribution functions for inputs: \xmlNode{Distributions}
  \item Set up a simple Monte-Carlo sampling for perturbing the input space: \xmlNode{Samplers}
  \item Store the input and output data: \xmlNode{DataObjects}
  \item Print and plot input and output data: \xmlNode{OutStreams}
  \item Control multiple executions: \xmlNode{Steps}
\end{enumerate}

In section~\ref{sub:singleRun}, we have already discussed the use of \xmlNode{RunInfo}, \xmlNode{Files}, \xmlNode{Models},
\xmlNode{DataObjects}, \xmlNode{OutStreams}. For practice, the user can try to build these RAVEN entities by themselves,
and refer to the complete input file located at
\textit{raven/tests/framework/user\_guide/ ravenTutorial/MonteCarlo.xml}.
In this section, we'd like to show the users how to set up \xmlNode{Distributions}, \xmlNode{Samplers} and \textbf{MultiRun}
of \xmlNode{Steps}. RAVEN employs \xmlNode{Distributions} to define many different probability distribution
functions (PDFs) that can be used to characterize the input parameters. One can consider the \xmlNode{Distributions}
entity to be a container of all the stochastic representation of random variables. Currently, RAVEN supports:
\begin{itemize}
  \item \textit{1-Dimensional} continuous and discrete distributions, such as Normal, Weibull, Binomial, etc.
  \item \textit{N-Dimensional} distributions, such as Multivariate Normal, user-inputted N-Dimensional distributions.
\end{itemize}
For the \textbf{AnalyticBateman} example, two 1-D uniform distributions are defined:
\begin{itemize}
  \item $sigma \sim \mathbb{U}(1,10)$, used to model the uncertainties associated with the \textit{sigma} Model variables;
  \item $decayConstant \sim \mathbb{U}(0.5e-8,1e-8)$,  used to model the uncertainties associated with the Model variable
    \textit{decay constants}.  Note that the same distribution can be re-used for multiple input variables, while
    still keeping those variables independent.
\end{itemize}
The following is the definition of \xmlNode{Distributions} block that is used for the \textbf{AnalyticBateman} problem:
\xmlExample{framework/user_guide/ravenTutorial/MonteCarlo.xml}{Distributions}
For uniform distributions, only \xmlNode{lowerBound} and \xmlNode{upperBound} are required. For other distributions,
please refer to the RAVEN user manual.

As we already mentioned, we will employ Monte-Carlo sampling strategy to demonstrate \textbf{MultiRun}.
To employ the Monte-Carlo sampling strategy, a \xmlNode{MonteCarlo} node needs to be defined.
The user also needs to specify the variables that need to be sampled using \xmlNode{variable}. In addition,
the setting for this sampler need to be specified in the \xmlNode{samplerInit} block. The only required sub-node
\xmlNode{limit} is used to specify the number of Monte Carlo samples. The user can also use other optional sub-node
to characterize their samplers. For this example, the \xmlNode{Samplers} block is:
\xmlExample{framework/user_guide/ravenTutorial/MonteCarlo.xml}{Samplers}
In this case, the Monte-Carlo method is employed on $two$ model variables, each of which are listed by name and
are associated with a distribution. Note that the \textit{decay-\*} and \textit{sigma-\*} variables are
associated with the distributions $decayConstant$ and $sigma$, respectively. These variables and their values are
passed to the model via the generic code interface. This requires the users to make some changes in their input files
in order to accept these variables. For example, the input file of \textbf{AnalyticBateman} becomes:
\xmlExample{framework/user_guide/ravenTutorial/commonFiles/referenceInput_generic_CI.xml}{nuclides}
As shown in this example, the values of nodes \xmlNode{sigma} and \xmlNode{decayConstant} are replaced with variables
\xmlString{\$RAVEN-decay-A|10\$} and \xmlString{\$RAVEN-sigma-A|10\$}, respectively. \nb we use prefix \xmlString{RAVEN-}
$+$ \xmlString{variable names defined inside RAVEN input files} within \xmlString{\$ \$} to define the
RAVEN-editable input parameters. In other words, the RAVEN-editable input parameters is used to transfer the sampled values
of RAVEN variables to input parameters of given code. This is the only way to connect the input parameters of code
with variables of RAVEN if \textit{Generic code interface} is employed. In addition, we use the wild-cards \texttt{|}
to define the format of the value of the RAVEN-editable input parameters. In this case, the value that is going to
be replaced by the generic code interface will be left-justified with a string length of 10 (e.g. ``\texttt{|}10''). Other
formatting options can be found in the RAVEN user manual.

As we already mentioned, the \textit{Generic code interface} requires that the codes need to return a CSV file with
the input parameters and output parameters. The filename of the CSV file should includes:
\begin{itemize}
  \item prefix: ``out$\mathtt{\sim}$''
  \item filename: the base of input filename without extension
  \item extension: ``.csv''
\end{itemize}
For this case, the input filename is ``referenceInput\_generic\_CI.xml'', thus the output CSV filename should be
``out$\mathtt{\sim}$referenceInput\_generic\_CI.csv''.

Once all the other entities are defined in the RAVEN input file, they must be combined in the \xmlNode{Steps} block,
which dictates the workflow of RAVEN.  For this case, two \xmlNode{Steps} are defined:
\begin{itemize}
  \item \xmlNode{MultiRun} ``sample'', used to run the multiple instances of the driven code and  collect the outputs
    in the two \textit{DataObjects}. As it can be seen, the \xmlNode{Sampler} is specified to communicate to the
    \textit{Step} that the driven code needs to be perturbed through the Monte-Carlo sampling.
  \item  \xmlNode{IOStep} named ``writeHistories'', used to 1) dump the ``histories'' and ``samples'' \textit{DataObjects}
    \textbf{Entity} to a CSV file and 2) plot the data in the EPS file.
\end{itemize}

Figures~\ref{fig:historiesMCPlotLine_A} and ~\ref{fig:samplesMCPlotLine_A} show the report generated by RAVEN of the
evolution of the variable $A$ and its final values, respectively.
\xmlExample{framework/user_guide/ravenTutorial/MonteCarlo.xml}{Steps}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%figure histories
\begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ravenTutorial/gold/MonteCarlo/1-history_A_line.png}
  \caption{Plot of the histories generated by the Monte Carlo sampling for variable $A$.}
  \label{fig:historiesMCPlotLine_A}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%figure samples
\begin{figure}[h!]
  \centering
  \includegraphics[scale=0.7]{../../tests/framework/user_guide/ravenTutorial/gold/MonteCarlo/1-samplesPlot_A_scatter.png}
  \caption{Plot of the samples generated by the MC sampling for variable $A$.}
  \label{fig:samplesMCPlotLine_A}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\clearpage