%=================================================================
\section{GPU Testbed and Selected CUDA kernels} \label{sec:methodChar}
Several different benchmarks have been proposed in the literature to measure and assess existing GPGPU heterogeneous architectures, such as Rodinia~\citep{Rodinia:Che:2009}, Parboil~\citep{Stratton:2012:Parboil} and SHOC~\citep{Danalis:2010:SHOC}. Rodinia benchmark was devised for heterogeneous parallel computing research, it has had a high level of acceptance in the community and its applications represent different high-level domains or behaviours,  called the Berkeley dwarfs~\cite{asanovic2009dwarfs}. In this work, we have selected  a set of GPU Applications from the Rodinia Benchmark suite and  other classical algorithms of linear algebra. First to all, we present in the next subsection the different GPUs that we used for our experiments. After in Subsection~\ref{ssec:useCases}, we characterize each one of the kernel that we used for our experiments.

%For each application we iterated over the values of one or two parameters, with some applications invoking the same kernel multiple times on each execution. During our evaluation, all applications were executed using the CUDA profiling tool \textit{nvprof}. Each experiment is presented as the average of ten executions, with a confidence interval of 95\%. In this section we describe the GPU applications, GPU testbed used to do this work.

%---------------------------------------------------------------------------

\subsection{GPU Testbed}\label{ssec:GPUTestbed}
For our experiments in the Chapter~\ref{chap:BSPmodel} and Chapter~\ref{Chap:ML} we used 9 different GPUs, described in Table~\ref{tab:GPUs}, with 5 belonging to Kepler architecture (Compute Capability 3.X), 3 to Maxwell (C.C. 5.X) and 1 to Pascal (C.C. 6.x). More information about these GPUs are presented in Table~\ref{tab:CC} and we have described the main changes of hardware and software between architectures in Section~\ref{ssec:GPUroadmap}.

%Experiments with the Tesla Pascal P-100 were performed accessing the Free OpenPOWER Cloud by Unicamp, running in an IBM OpenPOWER Server configured with 16 physical cores, up to 128 simultaneous treads or vcpus, and 2 x NVIDIA GPU Tesla-P100, connected via NVLINK 1.0.

% \todoRaph{You described where the Pascal experiments were performed. And the other boards? \textbf{Corrected - That was question of acknowledge}}

\begin{table}[htpb]
    \centering
    \scalebox{0.75}{
    \begin{tabular}{lccccccc}
        \toprule
        \textbf{Model}&\textbf{C.C.}&\textbf{Memory}&\textbf{Bus}&\textbf{Bandwidth}&\textbf{L2}&\textbf{Cores/SM}&\textbf{Clock} \\ \bottomrule
        GTX-680&3.0&2 GB&256-bit&192.2 GB/s&0.5 M&1536/8&1058 Mhz \\ \midrule
        Tesla-K40&3.5&12 GB&384-bit&276.5 GB/s&1.5 MB&2880/15&745 Mhz \\ \midrule
        Tesla-K20&3.5&4 GB&320-bit&200 GB/s&1 MB&2496/13&706 MHz\\ \midrule
        %Titan Black&3.5&6 GB&384-bit&336 GB/s&1.5 MB&2880/15 &980 Mhz \\ \midrule
        Titan&3.5&6 GB&384-bit&288.4 GB/s&1.5 MB&2688/14&876 Mhz\\ \midrule
        Quadro K5200&3.5&8 GB&256-bit&192.2 Gb/s&1 MB&2304/12&771 Mhz \\ \midrule
        Titan X&5.2&12 GB&384-bit&336.5 GB/s&3 MB&3072/24&1076 Mhz \\ \midrule
        GTX-970&5.2&4 GB&256-bit&224.3 GB/s&1.75 MB&1664/13&1279 Mhz\\ \midrule
        GTX-980&5.2&4 GB&256-bit&224.3 GB/s&2 MB&2048/16&1216 Mhz \\ \midrule
        Pascal-P100&6.0 GB&16 GB&4096-bit&732 GB/s&4 MB&3584/56&1328 Mhz\\ \midrule
    \end{tabular}}
    \caption{Hardware specifications of the GPUs in the testbed}
    \label{tab:GPUs}
\end{table}

\subsection{Selected CUDA kernels}\label{ssec:useCases}
The  source code for all the use cases, experiments and results are available\footnote{Hosted at GitHub: \texttt{\scriptsize https://github.com/marcosamaris/gpuperfpredict} [Accessed on March 2018]} under Creative Commons Public License for the sake of reproducibility.

Our benchmark contains 4 different strategies for \emph{matrix multiplication}~\citep{CUDAGuide}, 2 algorithms for \emph{matrix addition}, 1 dot product algorithm, 1 vector addition algorithm and 1 \emph{maximum sub-array problem} algorithm~\citep{Cleber:Thesis}, and 11 CUDA kernel functions belonging to 6 applications from Rodinia benchmarking suite (see Table~\ref{tab:Rodinia}). The remainder of this section discusses some details of these algorithms, and introduces a code of letters for each application, used in Chapter~\ref{chap:BSPmodel} and Chapter~\ref{Chap:ML}.

\subsubsection{Matrix Multiplication}
Matrix multiplication is the core of many scientific areas. This operation is highly used in areas like deep learning, visual computing and digital images processing, among others. The analysis and modeling of these algorithms brings a better understanding and help to researcher and developer of Job Management Systems to deal with this mathematical operations.

In this research, we used four different versions of matrix multiplication, these versions differ in memory access optimizations: global memory with non-coalesced accesses (MMGU); global memory with coalesced accesses (MMGC); shared memory with non-coalesced accesses (MMSU); and shared memory with coalesced accesses (MMSC). This algorithm has a high utilization of the Streaming Processors and it obtains a high throughput of communication in the different levels of memory, L2 cache and L1 cache for small matrices. We have adopted \texttt{block\_size}$^2$ threads per block and defined the number of blocks to be square of \texttt{(N + block\_size-1)/block\_size}, dynamically devised from the size of the problem ($N$) and the block size. Aiming to take advance of coalesced accesses the value of \texttt{block\_size} is equal to 16 in our experiments. In Section~\ref{sec:characterization} is presented an communication analysis of this application with different sizes of threads per block.

The asymptotic computational complexity of a square matrix multiplication of size $N$ is $O(N^3)$ in a sequential algorithm, in a CUDA algorithm this complexity is $O(N)$ using $N\times{}N$ threads. In this algorithm each thread requests $N$ elements from both matrices and perform a dot product with these arrays. This CUDA kernel performs $N$ reads from global memory for each matrix and $N$ arithmetic operations. A single write instruction is performed. shared memory is not used in two versions, MMGU and MMGC. In Figure~\ref{fig:matMulGMUN} is shown the source code of the kernel MMGU, we only changed the data access pattern, to permit coalesced accesses to data in global memory. In Figure \ref{fig:matMulGMUN} line 7 is changed to \texttt{Pvalue += A\_d[i * N + k] * B\_d[k * N + j];} and line 10 is changed to \texttt{C\_d[i * Width + j] = Pvalue;}.

\lstset{emph={[2]__global__}, emphstyle={[2]\color{red}\bf },language=C++, keywordstyle=\color{blue}, numbers=left, showspaces=false,
    showstringspaces=false, tabsize=1, breaklines=false,stringstyle=\color{red}, commentstyle=\color{red}, morecomment=[l][\color{magenta}]{\#}}

\begin{figure}[htpb]
\centering
{\scriptsize
\begin{lstlisting}
__global__ void matMul(float* C_d, float* A_d, float* B_d, int N) {
  float Pvalue = 0.0;
  int j = blockIdx.x * blockDim.x + threadIdx.x;
  int i = blockIdx.y * blockDim.y + threadIdx.y;

  for (int k = 0; k < N; ++k)
    Pvalue += A_d[j * N + k] * B_d[k * N + i];

  C_d[j * N + i] = Pvalue;
}

\end{lstlisting}}
\caption{Kernel in CUDA of matrix multiplication only with global memory and uncoalesced accesses (MMGU).}
\label{fig:matMulGMUN}
\end{figure}

The version MMSU and MMSC use shared memory to load data from global memory and to process them with a lower latency of communication. As shared memory is limited in GPU architectures, the implementations of matrix multiplication with shared memory must be tiled. The concept of tiling in shared memory is graphically described with Figure~\ref{fig:TilingMMSU}, with the matrix multiplication. Tiling is a common strategy to partition data into subset called tiles such that each tile fits into the shared memory. This technique splits our problem domain into phases. The tiled process is executed to guarantee that each thread can access data in shared memory to perform its part of the matrix multiplication.

In process shown in Figure~\ref{fig:TilingMMSU}, a tile process charges a subset from matrix A and Matrix B in the shared memory of the GPU. A barrier synchronization is used to guarantee that all data was loaded in the shared memory, after the calculations are performed, the barrier synchronization guarantees that the shared memory can be safely overwritten. In this application \texttt{tile\_size=block\_size}, consequently the sizes in bytes of both subsets of the matrices will be \texttt{(tile\_size)$^2 \times$FP\_Bytes}, where \texttt{FP\_Bytes} is the size of the single precision used for the application.

\begin{figure}[htpb]
	\centering
    \includegraphics[scale=.3]{images/square-tiling.png}
    \caption{Tiling technique of the Matrix multiplication using shared memory}
    \label{fig:TilingMMSU}
\end{figure}

\subsubsection{Matrix Addition}
For the Matrix addition algorithm, we used two different memory access optimizations: global memory with non-coalesced accesses (MAU); and global memory with coalesced accesses (MAC);  The run-time complexity for a sequential matrix addition algorithm using two matrices of size $N\times{}N$ is $O(N^2)$. In a CUDA implementation, the run-time complexity of the matrix addition is $O(1)$ using $N^2$ threads. In this algorithm each thread request $1$ element from each one of the matrix elements and perform a single addition. The implementation of this kernel is similar than matrix multiplication. In Figure \ref{fig:matMulGMUN} the loop of the line 6-7 is deleted and the statement with the addition is added \texttt{C[tid] = A[tid] + B[tid];}, where \texttt{tid} = \texttt{int tid = i*N + j;}. To take advances of coalesced accesses, \texttt{tid} is changed to \texttt{int tid = j*N + i;}.

Matrix addition has the same threads hierarchy than matrix multiplication. The blocks are bi-dimensional with \texttt{block\_size}$^2$ threads per block. The size of the grid is square of \texttt{N/block\_size}, dynamically devised from the size of the problem \texttt{N} and \texttt{block\_size}.

\subsubsection{Vector Addition Algorithm (vAdd)}
For two vectors $A$ and $B$, the Vector Addition $C = A + B$ is obtained by adding the corresponding components. In a CUDA implementation, the run-time complexity of the vector addition algorithm is $O(1)$ using N threads, so each threads perform an addition of a position of the vectors $A$ and $B$ and stores the result in the vector $C$. This algorithm is a simplified version of the matrix addition. This application has an uni-dimensional block, the block size used for the experiments is $256$ and the size of the grid is also uni-dimensional and dynamically devised from the size of the problem  \texttt{N} and \texttt{block\_size}. The source code of this kernel is similar than matrix addition, only change the dimension indexes of each threads, i.e. \texttt{int tid = blockDim.x * blockIdx.x + threadIdx.x;}, using only the dimension $x$ of \texttt{blockDim}.

\subsubsection{Dot Product Algorithm (dotP)}
For two vectors $A$ and $B$, the dot product $C = A \cdot B$ is obtained by adding the multiplication of corresponding components of the input, the result of this operation is a scalar. Unlike vector addition, dot product is a reduction from vectors to a scalar. In a GPU algorithm, each thread performs a multiplication of a position of the vectors $A$ and $B$ and stores the result shared variable. Then a reduction using the shared memory is performed and finally a vector with size equal to \texttt{N/block\_size} is transferred to the CPU memory for later processing. The block size used for the experiments is $256$ and the size of the grid is also uni-dimensional and dynamically devised from the size of the problem  \texttt{N} and \texttt{block\_size}.

\subsubsection{Maximum Sub-Array Problem (MSA)}
Let $X$ be a sequence of $N$ integer numbers $(x_1, ... , x_N)$. The Maximum Sub-Array Problem (MSA) consists of finding the contiguous sub-array within $X$ which has the largest sum of elements. The solution for this problem is frequently used in computational biology for gene identification, analysis of sequence of protein and DNAs, identification of hydrophobic regions, among others. The implementation used in this paper creates a kernel with 4096 threads, divided in 32 blocks with 128 threads~\citep{Cleber:Thesis}. The $N$ elements are divided in intervals of $N/t$, and each block receives a portion of the array. The blocks use the shared memory for storing segments, which are read from the global memory using coalesced accesses. Each interval is reduced to a set of 5 integer variables, which are stored in vector of size $5 \times t$ in global memory. This vector is then transferred to the CPU memory for later processing.

A summary of each one of these applications is shown in Table~\ref{tab:useCases}, in this table is shown the thread hierarchy and the request shared memory per block in each kernel. In column \texttt{dimGrid} and \texttt{dimBlock} is shown how each solution is associated to the dimension of the problem. Column Shared Mem shows the size of shared memory that each kernel needs per block.

\begin{table}[htpb]
    \centering
    \scalebox{.9}{
        \begin{tabular}{cccccc}
            \midrule\midrule
            \textbf{Application} & \textbf{Param} & \textbf{Kernel}& \textbf{\texttt{dimGrid}}&  \textbf{\texttt{dimBlock}}&\textbf{Shared Mem}\\\midrule\midrule
            \multirow{4}{*}{Matrix Mul} & \multirow{4}{*}{1} &MMGU &\multirow{4}{*}{(GS, GS, 1)}&\multirow{4}{*}{ (BS, BS, 1)}&\multirow{4}{*}{($BS^2\times{}2\times{}4B$)}\\
            &  &MMGC & & &\\
            &  &MMSU & & &\\
            &  &MMSC & & &\\\midrule
            \multirow{2}{*}{Matrix Add} & \multirow{2}{*}{1} &MAU &\multirow{2}{*}{(GS, GS, 1)}&\multirow{2}{*}{ (BS, BS, 1)}&\multirow{2}{*}{0}\\
            &  &MAC & & & \\\midrule
            Vector Add&1  &VAdd &(GS, 1, 1) &(BS, 1, 1)& 0 \\\midrule
            Dot Product&1  &dotP &(GS, 1, 1) &(BS, 1, 1) &($BS\times{}4B$)\\\midrule
            Max. Sub Array&1  &MSA & (48, 1, 1) &(128, 1, 1)&($4096\times{}4B$)\\\midrule
        \end{tabular}}
    \caption{Key linear algebra applications used in the experiments}
    \label{tab:useCases}
\end{table}

\subsubsection{Rodinia Benchmark Suite}
We selected 11 CUDA kernel functions belonging to 6 applications from Rodinia for the benchmarks (Table~\ref{tab:Rodinia}). Some applications invoke the same kernel multiple times on each execution, resulting in the number of collected samples shown in the last column. For each kernel we iterated over the values of one or two parameters, with the number of iterations indicated inside the brackets. For instance, the Hot Spot application has 2 parameters, with the first iterated among 5 values and the second 4 values, for a total of 20 executions in each machine. %In those applications only some part of them can be compute concurrently, above we will explain the performance behavior of each one of the selected Rodinia benchmarks kernels.

\begin{table*}[htpb]
    \centering
    \scalebox{.8}{
        \begin{tabular}{cccccc}
            \midrule\midrule
            \textbf{Application} & \textbf{Berkeley Dwarf} & \textbf{Domain}& \textbf{Param.}& \textbf{Kernels} & \textbf{Samples}\\\midrule\midrule
            \multirow{2}{*}{Back Propagation (BCK)} & \multirow{2}{*}{Unstructured Grid} & \multirow{2}{*}{Pattern Recognition} &\multirow{2}{*}{1 - [57]}&layerforward&\multirow{2}{*}{57}\\
            &  &  & &adjust-weights  \\\midrule
            \multirow{2}{*}{Gaussian Elimination (GAU)} & \multirow{2}{*}{Dense Linear Algebra} & \multirow{2}{*}{Linear Algebra} &\multirow{2}{*}{1 - [32]}& Fan1&\multirow{2}{*}{34800}  \\
            &  &  & & Fan2  \\\midrule
            Heart Wall (HWL) & Structured Grid & Medical Imaging &1 - [84]& heartWall&5270 \\\midrule
            Hot Spot (HOT) & Structured Grid & Physics Simulation &2 - [5,4]& calculate-temp&396288 \\\midrule
            % Hot Spot 3D (H3D) & Structured Grid & Physics Simulation &2 - [3,10]& hotspotOpt1&150150 \\\midrule
            %LavaMD (LMD) & N-Body & Molecular Dynamics &1 - [50]& LavaMD&50\\\midrule
            \multirow{3}{*}{LU Decomposition (LUD)} & \multirow{3}{*}{Dense Linear Algebra} & \multirow{3}{*}{Linear Algebra} &\multirow{3}{*}{1 - [32]}&diagonal&8448 \\
            & & &  & perimeter&8416 \\
        &  &  & & internal&8416 \\\midrule
            \multirow{2}{*}{Needleman-Wunsch (NDL)} & \multirow{2}{*}{Dynamic Programming} & \multirow{2}{*}{Bioinformatics} &\multirow{2}{*}{2 - [16,10]}& needle-1&21760 \\
             &  &  & & needle-2&21600 \\\midrule

        \end{tabular}}
    \caption{Rodinia applications used in the experiments}
    \label{tab:Rodinia}
\end{table*}

\begin{itemize}
    \item {\bf Back Propagation (BCK):}
BCK trains a layered neural network. The application is comprised of two kernels: Forward Phase (BCK-K1), in which the activation are propagated from the input to the output layer, and Backward Phase (BCK-K2), in which the error between the observed and requested values in the output layer is propagated backwards to adjust the weights and bias values. The time complexity of a back-propagation neural network algorithm on a single processor is of $O(W^3)$; where $W$ is the count of weights in the network. In this CUDA implementation both kernels have a complexity $O(log(BS))$ where $BS=16$. Each block has \texttt{BLOCK\_SIZE}$^2$ number of threads. The number of blocks in the grid is \texttt{(N/block\_size} nad it is dynamically calculated from the layer size ($N$) and the block size.

In kernel (BCK-K1), only one thread (with id 0) in the each block load an element of the input layer on the shared memory, after each thread load an element of the weight matrix on the shared memory. Then, the weight matrix is updated with the values of the input layer. Finally a loop reduction is done with a loop of size log2(block\_size). Inside this loop, log2(block\_size) power instructions are done and log2(block\_size) additions over data in the shared memory. Each interval is reduced to a set of \texttt{grid\_size}$\times$\texttt{block\_size} integer variables. This vector is then transferred to the main memory of the host for later processing. In kernel (BCK-K2), shared memory is not used. According to the source code of (BCK-K2), each thread performs O(1) reads and write in the global memory and does different computations over this data.

\item{\bf Gaussian Elimination (GAU):}
GAU solves a linear system $Ax = b$, the application analyzes an $n\times{}n$ matrix and an associated $1 x n$ vector to solve a
set of equations with $n$ variables and $n$ unknowns. Gaussian Elimination algorithms has a complexity $O(n^3)$. This application compute the results row by row. The algorithm synchronizes between iterations, but the values calculated in each iteration is computed in parallel, see Figure~\ref{fig:GauCode}.

, where $n$ is the size of the matrix elements in both dimensions, i.e. the matrix is $n\times{}n$.

The application has two different kernels Fan1 and Fan2, which we call (GAU-K1) and (GAU-K2) respectively. (GAU-1) calculate multiplier matrix and (GAU-2) modify the matrix A into LUD. In the experiments, we varied the size of the matrix.

In this implementation, First kernel GAU-K1, has a uni-dimensionmal block, each block \texttt{block\_size = 512} threads and the number of block is dynamic and computed with the next expression \texttt{(N/block\_size)}. Second kernel has a bi-dimensional block, its dimension is \texttt{block\_size}$^2$ and \texttt{block\_size = 4} in this implementation. The size of the grid also is dynamic depending to size of the problem $N$and the $BS$ (Block size), and it is computed as \texttt{(N/block\_size)}. Both kernel are iterative, it means that the same kernels are invoked multiple times on a single execution of the whole application. Any of these kernel use the shared memory of the streaming processors.

\lstset{emph={[2]__global__}, emphstyle={[2]\color{red}\bf },language=C++, keywordstyle=\color{blue}, numbers=left, showspaces=false,
    showstringspaces=false, tabsize=1, breaklines=false,stringstyle=\color{red}, commentstyle=\color{red}, morecomment=[l][\color{magenta}]{\#}}

\begin{figure}[htpb]
\centering
{\scriptsize
\begin{lstlisting}
for (t=0; t<(Size-1); t++) {
		Fan1<<<dimGrid1,dimBlock1>>>(m_cuda,a_cuda,Size,t);
		cudaThreadSynchronize();
		Fan2<<<dimGrid2,dimBlock2>>>(m_cuda,a_cuda,b_cuda,Size,t);
		cudaThreadSynchronize();
	}
\end{lstlisting}}
\caption{Kernel in CUDA of matrix multiplication only with global memory and uncoalesced accesses (MMGU).}
\label{fig:GauCode}
\end{figure}

\item {\bf Heart Wall (HWL):}
HWL tracks the movement of a mouse heart over a sequence of 104 609x590 ultrasound images to record response to the stimulus. In the experiments, we varied the number of frames to process. This kernel has two different stages, in the first stage the kernel performs operations on the first frame to detect initial, partial shapes of inner and outer heart walls in the second stage the kernel presents multiple nested loops that process batches of 10 frames and 51 points in each image. This kernel is very complicated to analyze, i.e. this kernel has 1300 codes lines more or less, it uses 8 variables in the shared memory and 44 variables to store all its computations in each streaming multiprocessor, obtaining a low latency in this process. it runs multiple times depending the number of frame to compute. As the number of threads in a block as the number of blocks in a thread ar uni-dimensionals. For all the experiments, this kernels has a \texttt{block\_size}$= 256$ and a \texttt{grid\_size}$ = 51$. For a total of 13056 threads in each one of the executions. This application modeled a set of  ordinary differential equations (ODEs) that are determined by more than 200 parameters. HWL requires the inclusion of some non-parallel computation into the kernel, leading to a slight warp under-utilization.

% \todoRaph{Describe something about the computations performed in the HWL kernel/application.}

\item {\bf HotSpot (HOT):}
HOT is a tool to estimate processor temperature based on an architectural floor plan and simulated power measurements. This application includes the 2D transient thermal simulation kernel which iteratively solves a series of differential equations to determine block temperatures. This kernel has a bi-dimensional grid and block. The dimension of the block is square of \texttt{block\_size} and \texttt{block\_size}$ = 16$. The size of the grid is the square of \texttt{N/block\_size} and it is dynamic depending the size of the problem and the size of the thread block. In the experiments, we varied two parameters: size of the problem and number of iterations. This application has a single kernel. It uses the shared memory.

\item {\bf LU Decomposition (LUD):}
this is a factorization algoritm, where ''LU'' means \emph{lower upper}, LUD is an algorithm to calculate the solutions of a set of linear equations. The LUD kernel decomposes a matrix as the product of a lower triangular matrix and an upper triangular matrix. This benchmark present tree different kernels (LUD-K1, LUD-K2 and LUD-K3). Similarly to the Gaussian Elimination, the matrix size of this experiment was also iterated. First kernel LUD-K1 (named diagonal) has a static size of threads, the size of threads of this kernels always 16. The second kernel LUD-K2 (named perimeter) has a uni-dimensional block the size of thread per block always is \texttt{block\_size}$*2$, the number of block in the grid also is uni-dimensional and it is computed dynamically with the next expression \texttt{(matrix\_dim-i)/block\_size-1}. The third kernel LUD-K3 (named internal) has a block bi-dimensional, this size is the square of \texttt{block\_size} and \texttt{block\_size}$ = 16$, the grid also is bi-dimensional and it is computed with the same expression than the second kernel, \texttt{(matrix\_dim-i)/block\_size-1}.

\item {\bf Needleman-Wunsch (NDL):}
is a nonlinear global optimization method for DNA sequence alignments. The potential pairs of sequences are organized in a 2D matrix. In the first step, the algorithm fills the matrix from top left to bottom right, step-by-step. In the second step, the maximum path is traced backward to deduce the optimal alignment. Needleman-Wunsch has two CUDA kernels, NDL-K1 and NDL-K2. We iterated over the matrix dimension and penalty positive integer parameters. In both kernels each block use $2180$ KB of shared memory. This implementation use a static number of threads per block of 16 and the grid is dynamically computed with this expression $(N-1)/16$. Where $N$ is the size of the input matrix.

% \lstset{language=C++, keywordstyle=\color{blue}, numbers=left, showspaces=false,
%     showstringspaces=false, tabsize=1, breaklines=false,stringstyle=\color{red}, commentstyle=\color{red}, morecomment=[l][\color{magenta}]{\#}}

% \begin{figure}[htpb]
% 	\centering
%     \lstset{emph={[2]dim3},
%             emphstyle={[2]\color{red}\bf},
%             emph={[3]dimGrid, dimBlock},
%             emphstyle={[3]\color{blue}\bf},
%             emphstyle={\bf}, basicstyle={\ttfamily \scriptsize},
%             commentstyle={\em},
%             keywordstyle={\color{blue}\bf},
%             numbers=left,
%             captionpos=b,
%             xleftmargin=1cm}
% 	\begin{lstlisting}[frame=trBL,language=C,linewidth=.92\linewidth]
% for( int i = 1 ; i <= block_width ; i++){
% 		needle_cuda_shared_1<<<dimGrid, dimBlock>>>(referrence_cuda, matrix_cuda
% 		                                      ,max_cols, penalty, i, block_width);
% 	}
% 	for( int i = block_width - 1  ; i >= 1 ; i--){
% 		dimGrid.x = i;
% 		dimGrid.y = 1;
% 		needle_cuda_shared_2<<<dimGrid, dimBlock>>>(referrence_cuda, matrix_cuda
% 		                                      ,max_cols, penalty, i, block_width);
% 	}

% 	\end{lstlisting}
% \caption{Launched kernel with its respective dimension variables dimBlock and dimGrid}
% \label{fig:dimGridBlock}
% \end{figure}

\end{itemize}


\documentclass[main.tex]{subfiles}
\begin{document}

\chapter{Beyond QED}

\section{Decay of a heavy vector boson}

\marginpar{Friday\\ 2020-6-19, \\ compiled \\ \today}

We consider a \textbf{massive real vector field}. The generalization to a complex one is simple.
The Lagrangian, describing the massive vector \(V^{\mu }\) and the fermion \(\psi \), is:
%
\begin{align}
\mathscr{L} &= - \frac{1}{4} V^{\mu \nu } V_{\mu \nu }
+ \frac{1}{2} M^2 V^{\mu } V_{\mu }
+ \overline{\psi} \qty(i \slashed{\partial} - m) \psi
+ \mathscr{L} _{\text{int}} \\
\mathscr{L} _{\text{int}} &=
\frac{g}{c_{w}} \overline{\psi} \qty(
    c_L \gamma^{\mu } p_L +
    c_R \gamma^{\mu } p_R
) \psi V_{\mu }
\,,
\end{align}
%
where as usual the field strength is defined as \(V^{\mu \nu } = 2 \partial^{[\mu } V^{\nu ]}\), while \(\theta_{w}\) is the Weinberg angle, whose cosine is denoted \(c_w = \cos \theta_{w} \approx \num{.88}\).

It is experimentally measured that \(\sin^2\theta \approx \num{.22}\).
On the other hand, the coupling \(g\) is \(g = e / \sin \theta \approx \num{.64} \) in natural units.

The couplings \(c_{L, R}\) depend on the specific particle considered.

The terms \(P_L\) and \(P_R\) are the projection operators defined in \eqref{eq:chirality-projection-operators}: they project onto the left- or right-handed subspaces.

\subsection{Massive vector diagrams}

If we have a free field term for the massive vector in which it is incoming we get a factor \(\epsilon^{\mu }_{\lambda } (p)\), if the massive vector is outgoing we have its conjugate, \(\epsilon^{\mu *}_{\lambda } (p)\).

The propagator (in momentum space) is given by
%
\begin{align}
\widetilde{D}^{\mu \nu }_F (q) &=-
\frac{i}{q^2 - M^2}
\qty(
    \eta^{\mu \nu }
    - \frac{p^{\mu }p^{\nu }}{M^2}
)
\,,
\end{align}
%
while for a vertex in which the massive vector interacts with a fermion with \(\mathscr{L} _{\text{int}}\) we get a term:
%
\begin{align}
-i \frac{g}{c_w} \qty(c_L \gamma^{\mu }_{L} + c_R \gamma^{\mu }_{R})
\,,
\end{align}
%
where the matrices \(\gamma^{\mu }_{L, R}\) are defined as \(\gamma^{\mu} P_{L, R}\).

A useful relation is this pseudo-completeness one:
%
\begin{align}
\sum _{\lambda = 1}^{3} \epsilon^{\mu }_{\lambda }(p) \epsilon^{\nu }_{\lambda } (p) &= - \eta^{\mu \nu } + \frac{p^{\mu } p^{\nu }}{M^2}
\,.
\end{align}

Notice the fact that we have three independent polarizations now, as opposed to the massless vector which has two; the polarizations of the massive vector span the whole spacelike section of spacetime orthogonal to \(p^{\mu }\), whereas the polarizations of the massless vector only span the two-dimensional region which is spacelike \emph{and} orthogonal to the 3D wavevector \(\vec{k}\).
This is also relevant when averaging over polarizations: we will need to divide by \(3\).

Now, since we have the Feynman rules we can quickly calculate the Feynman amplitude for the decay \(V \to \overline{f} f\), which now is permitted:
%
\begin{align}
\mathcal{M}_{V \to \overline{f}f}
= -i \frac{g}{c_w} \overline{u}_{r} (k)
\qty(c_L \gamma^{\mu }_{L} + c_R \gamma^{\mu }_{R})
u_{r'} (k') \epsilon_{\mu }^{\lambda} (p)
\,,
\end{align}
%
and from it we can sum and average to get the unpolarized square amplitude:
%
\begin{align}
\abs{\mathcal{\overline{M}}}^2 &=
\frac{g^2}{c_w^2} \frac{1}{3}
\sum _{\lambda } \epsilon_{\mu }^{\lambda }(p) \epsilon_{\nu }^{\lambda *}(p)
\sum _{r r'} (\overline{u}_r (\dots)^{\mu } v_{r'}  )
( \overline{v}_{r'} (\dots)^{\nu } u_r )  \\
&= \frac{1}{3} \frac{g^2}{c_w^2}
\qty(- \eta_{\mu\nu } \frac{p_{\mu }p_{\nu}}{M^2})
\Tr[(\slashed{k} - m)\qty(c_L \gamma^{\mu }_{L} + c_R \gamma^{\mu }_{R})
 (\slashed{k}' + m)\qty(c_L \gamma^{\nu }_{L} + c_R \gamma^{\nu }_{R})]
\,.
\end{align}

\begin{claim}
The trace can be explicitly calculated to give
%
\begin{align}
\begin{split}
&\Tr[(\slashed{k} - m)\qty(c_L \gamma^{\mu }_{L} + c_R \gamma^{\mu }_{R})
(\slashed{k}' + m)\qty(c_L \gamma^{\nu }_{L} + c_R \gamma^{\nu }_{R})] = \\
&\phantom{=}\ = 2 (c_L^2 + c_R^2) \qty(k^{\prime \nu } k^{\mu } - \eta^{\mu \nu } \qty(k \cdot k') + k^{\prime \mu } k^{\nu })
- 4 m^2 c_L c_R \eta^{\mu \nu }
\end{split}
\,.
\end{align}
\end{claim}

\begin{proof}
\todo[inline]{To do.}
\end{proof}

Using this result, we find
%
\begin{align}
\abs{\mathcal{\overline{M}}}^2 &=
\frac{1}{3} \frac{g^2}{c_w^2} \qty(2 (c_L^2 + c_R^2)
\qty(k \cdot k' + 2 \frac{(p \cdot k) (p \cdot k')}{M^2})
+ 12 m^2 c_L c_R
)
\,.
\end{align}

Now, we can specify the kinematics to the rest frame of the massive vector. Then, we can see that the ``Mandelstam variables'' are given by \(s = p^2 = M^2\), \(t = u = (p-k)^2 = (k')^2 = m^2\), and that
%
\begin{align}
k \cdot k' = \frac{M^2}{2} - m^2
&&
p \cdot k = p \cdot k' = \frac{M^2}{2}
\,.
\end{align}

So the amplitude reads:
%
\begin{align}
\abs{\mathcal{\overline{M}}}^2 &=
\frac{1}{3} \frac{g^2}{c_w^2} \qty(2 (c_L^2 + c_R^2)
\qty( \frac{M^2}{2} - m^2 + 2 \frac{M^2}{4}) + 12 m^2 c_L c_R )  \\
&= \frac{g^2}{c_w^2} \frac{M^2}{3}
\qty(2 (c_L^2 + c_R^2) \qty(1 - \frac{m^2}{M^2}) + 12 c_L c_R \frac{m^2}{M^2})
\,.
\end{align}

If the vector is very massive compared to the fermion (\(m \ll M\)) then this simplifies heavily:
%
\begin{align}
\abs{\mathcal{\overline{M}}}^2
\overset{m \ll M}{\approx}
\frac{g^2}{c_w^2} \frac{M^2}{3} 2 (c_L^2 + c_R^2)
\,,
\end{align}
%
however we will not make this assumption.

We can apply the general formula for the \textbf{differential decay rate} (in the lab frame):
%
\begin{align}
\eval{\dv{\overline{\Gamma} }{\Omega }}_{\text{lab}}
= \frac{1}{64 \pi^2} \frac{1}{M}
\sqrt{1- \frac{4m^2}{M^2}} \abs{\mathcal{\overline{M}}}^2_{\text{lab}}
\,,
\end{align}
%
but this does not depend on the angles (as it should: this is the decay of a particle in its frame, it better be isotropic!) we can integrate over the angles, which gives a factor \(4 \pi \), so we get
%
\begin{align}
\eval{\overline{\Gamma}}_{\text{lab}}=
\frac{g^2}{c_w^2}
\frac{M}{48 \pi } \sqrt{1 - \frac{4m^2}{M^2}}
\qty(2 (c_L^2 + c_R^2) \qty(1 - \frac{m^2}{M^2}) + 12 c_L c_R \frac{m^2}{M^2})
\,,
\end{align}
%
which, in the limit \(m \ll M\), becomes
%
\begin{align}
\eval{\overline{\Gamma}}_{\text{lab}}
\overset{m \ll M}{\approx}
\frac{g^2}{c_w^2}
\frac{M}{24 \pi }
(c_L^2 + c_R^2)
\,.
\end{align}

For example, in the decay \(Z \to \nu \overline{\nu}\) we have \(c_L = 1/2\), \(c_R = 0\) so the result is
%
\begin{align}
\overline{\Gamma}_{\text{lab}}
= \frac{g^2}{c_w^2} \frac{M}{96 \pi }
\,.
\end{align}

This is known as the \emph{invisible decay width} of the \(Z\) boson, since it is extremely difficult to detect the neutrino pair generated by the decay, so at a collider one would see a fraction of the \(Z\) bosons effectively disappear without a trace.

\section{Spontaneous Symmetry Breaking} \label{sec:ssb}

\subsection{SSB for a discrete symmetry}

Consider the following Lagrangian for a real scalar field \(\varphi \):
%
\begin{align}
\mathscr{L} &= \underbrace{\frac{1}{2} \qty( \partial_{\mu } \varphi ) \qty(\partial^{\mu } \varphi )}_{\text{kinetic term}}
\underbrace{- \frac{1}{2} m^2 \varphi^2 - \frac{\lambda}{4!} \varphi^{4} }_{V(\varphi )}
\,.
\end{align}

The potential \(V(\varphi )\) can also be written as
%
\begin{align}
V(\varphi ) = \frac{\lambda}{4!} \qty(\varphi^2 + \mu^2)^2 + C
\,,
\end{align}
%
where \(\mu^2 = 6 m^2 / \lambda \) and \(C\) is an inessential constant (which does not affect the EOM).

The Lagrangian has a discrete symmetry: \(\varphi \to -\varphi \).

The Hamiltonian for the theory is
%
\begin{align}
\mathscr{H} = \frac{1}{2} \qty(\partial_0 \varphi )^2 + \frac{1}{2} \qty(\nabla \varphi )^2 + V(\varphi )
\,,
\end{align}
%
and it is positive definite as long as \(\lambda > 0\), while \(\lambda = 0\) returns us to the free theory. The parameter \(\mu \), instead, is unconstrained.

We are interested in the study of the \textbf{ground state} \(\varphi_0 \) of the theory: the field configuration which has the minimum energy.
What are the requirements we must ask in order for it to actually be the ground state?

\begin{enumerate}
    \item It must be stationary: \(\partial_{\mu } \varphi_0 = 0\). This corresponds to the kinetic part of the Lagrangian vanishing, so that the kinetic energy of the configuration is zero.
    \item The value of the constant \(\varphi_0 \) is determined by minimizing the potential \(V (\varphi_0 )\).
\end{enumerate}

Let us consider two separate cases.

\subsubsection{Positive mass: \(\mu^2> 0\)}

In this case the state \(\varphi_0  = 0\) is a stable minimum: we have \(V' (\varphi  = 0) = 0\) and \(V'' (\varphi = 0) > 0\).

This is the same vacuum we have in the free theory.

\subsubsection{Negative mass: \(\mu^2 <0\)}

Now we have \(V'(\varphi =0) = 0\) and \(V'' (\varphi=0) < 0\): the configuration \(\varphi_0 = 0\) is an \emph{unstable} maximum.

Now, however, we have two new minima at \(\varphi = \pm \abs{\mu }^2\): these are stable.

Now, a perturbation of the ground state will look like
%
\begin{align}
\varphi (x) = \sigma (x) + \abs{\mu }
\,,
\end{align}
%
where we have arbitrarily chosen the right vacuum: in order to perturb around the vacuum we lose the parity symmetry.

Then, we can express the Lagrangian with respect to this fluctuation:
%
\begin{align}
\mathscr{L} = \frac{1}{2} \qty(\partial_{\mu }\sigma ) \qty(\partial^{\mu }\sigma ) - \frac{1}{2} \qty(\frac{\lambda }{3} \abs{\mu }^2)\sigma^2 - \frac{\lambda}{3!} \abs{\mu } \sigma^3 + \frac{\lambda}{4!} \sigma^{4}
\,,
\end{align}
%
which appears to have no parity symmetry anymore!
In fact the symmetry is still there, but it will have to be expressed in a complicated way in terms of the powers of \(\sigma \).

\subsection{SSB for a continuous symmetry}

Now we consider the situation in which a continuous global symmetry is spontaneously broken. Consider the following Lagrangian:
%
\begin{align}
\mathscr{L} = \qty(\partial_{\mu }\varphi )^{*} \qty(\partial_{\mu } \varphi ) \underbrace{- \lambda \qty(\varphi^{*} \varphi + \frac{\mu^2}{2})^2}_{V(\varphi )}
\,,
\end{align}
%
which is, up to constants, the most general renormalizable theory for a complex scalar.
Notice that this Lagrangian has a global \(U(1)\) symmetry, \(\varphi \to e^{i \alpha } \varphi \).

Just like before, we assume \(\lambda > 0\), which is needed in order for the Hamiltonian to be positive definite (or, equivalently, to have a global minimum for the energy) and discuss separately the cases \(\mu^2 \lessgtr 0\).

\subsubsection{\(\mu^2 > 0\): positive mass}

In this case we have \(\partial_{\varphi } \eval{V}_{\varphi =0} = 0\), so the field configuration \(\varphi_0 = 0 \) is stable.

The Hessian is positive definite in this case, so this is a minimum. In fact, the potential is globally convex, so it is the global minimum.

In this case, perturbing around the vacuum as \(\varphi (x) = \varphi_0 + \sigma (x)\) does not change the Lagrangian, since \(\varphi_0 = 0\). The symmetry is preserved.

\subsubsection{\(\mu^2 < 0\): negative mass}

In this case, solving \(V' (\varphi ) = 0\) yields two classes of solutions: one is \(\varphi_0 = 0\), the other instead is \(\varphi^{*}_{0} \varphi_0 = v^2 / 2\), where we define \(v^2 = - \mu^2 > 0 \). Notice that this is not a single point, but a whole circle: the condition on \(\varphi^{*}_{0} \varphi_0 \) has a \(U(1)\) symmetry!

The point \(\varphi_0 = 0\) is a local maximum (the Hessian is negative definite), while the other stationary point can be parametrized as
%
\begin{align}
\varphi_0 = e^{i \alpha } \frac{v}{\sqrt{2}}
\,,
\end{align}
%
where \(\alpha \) is a real parameter.

Now, it is convenient to express the perturbation around this vacuum as
%
\begin{align}
\varphi (x) = \qty(\frac{\sigma (x) + v}{\sqrt{2}}) e^{i \pi (x) / v}
\,,
\end{align}
%
where \(\sigma (x)\) and \(\pi (x)\) are both real fields.
This means that we are choosing a specific vacuum in the continuum of the circle (that at \(\alpha = 0\)) and moving from it; changing \(\sigma \) moves us in the direction of increasing potential, while changing \(\pi \) moves us around the circle.

Substituting this in, we get
%
\begin{align}
\mathscr{L} = \frac{1}{2} \qty(\partial \sigma )^2 - \frac{1}{2} \qty(2 \lambda v^2) \sigma^2 - \lambda v \sigma^3 - \frac{\lambda}{4} \sigma^{4}
+ \frac{1}{2} \qty(\partial \pi )^2 + \qty(v \sigma + \frac{\sigma^2}{2}) \qty(\partial \pi )^2
\,.
\end{align}

In terms of \(\sigma \) and \(\pi \) there is no \(U(1)\) symmetry anymore.

In the unbroken \(\mu^2 > 0\) theory we have one complex scalar of mass \(m\), which is equivalent to two real scalars of mass \(m\): if we set \(\varphi = (\varphi_1 + i \varphi_2 ) / \sqrt{2}\) we have
%
\begin{align}
\mathscr{L} = \sum _{i=1}^{2}\qty( \frac{1}{2} (\partial \varphi_{i} )^2 - \frac{1}{2} m \varphi_{i}^2 ) - \frac{\lambda }{4} \qty(\varphi_1^2 + \varphi_2^2)^2
\,.
\end{align}

On the other hand, in the spontaneously broken symmetry we get two real scalars: the mass of \(\sigma \) is \(m_\sigma^2 =  2 \lambda v^2\); the mass of \(\pi \) is \(m_\pi^2 = 0\)!

The field \(\pi \) is called the \textbf{(Nambu) - Goldstone boson} of the \(U(1)\) theory which is spontaneously broken to \(\phi \).

\todo[inline]{What is \(\phi \)?}

The physics of the SB theory look different, however there are some important facts which are maintained:
\begin{enumerate}
    \item the coupling terms are those of the original theory, \(\lambda \) and \(v^2 = - \mu^2\);
    \item the Goldstone boson can only have \textbf{derivative couplings}.
\end{enumerate}

What we have done can be stated with more generality: if we have a theory with a global symmetry group \(G\) which is spontaneously broken by a choice of vacuum to a group \(H\), then there will be \(\dim G - \dim H\) massless real Goldstone bosons.
This is the \textbf{Goldstone theorem}.

In our case we had \(G = U(1)\) and \(H = \mathbb{1}\), since the vacuum of our theory was a single point.

The Higgs mechanism is the spontaneous breaking of a local symmetry.

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}

\title{RFC stats}
\author{Chloe Yugawa}
\date{November 2017}

\begin{document}

\maketitle

\section{Introduction}
Signals that collide are problematic with the current proposed solution to this problem. In this section, we will explore the probability that signals will collide in a few different cases. Assumptions for this section are as follows:
\begin{itemize}
\item The number of crabs transmitting in the entire study is 500
\item The target listening radius is $50$ meters with an area of $7853.98m^2$
\item The estimated density of tagged crabs is 0.00012 tagged crabs/m2 (that's 1 tagged crab per 8500 square meters)
\item The conservative estimate of tagged crabs is 0.001 crabs/m2 (1 tagged crab per 1000 square meters).  The average distance between crabs will be about 32 meters if they're all evenly spaced.
\item For the sake of simplicity, a signal is defined as the total time between the beginning of a signal and the end of the signal, including the encoded silence. The time for this is estimated to be 11ms to 212ms, or 0.0011s to .212s.
\item The signal space is defined as the time between the end of one signal and the start of the next. This will be between 10 and 15 seconds.
\item The total signal time will then be between 10.0011 and 15.212 seconds (signal + signal space). On average, that's 5 signals per minute.
\item A conservative estimate for the percentage of crabs tagged is $.5\% $
\item The probability of finding more than one crab within a meter (tagged or untagged) during the season when research will occur is $3-5\%$

\end{itemize}

The number of crabs within the detection radius is shown in table 1. The low estimate is assuming 1 crab per $8500m^2$, while the high estimate is 1 crab per $1000m^2$.

Looking at periods of 0.5 seconds as a conservative estimate and dividing 60 seconds into 5 12 second sections, assume that each signal is contained within one of those 0.5 second periods. Within a 12 second section, each crab will on average signal once. Dividing that section into 24 0.5 second parts, by the pidgin hole principle, at least two crabs must signal within the same 0.5 second period when the number of signaling crabs is greater than 24. See table 3 for probability of collisions. Table 2 shows the same reasoning for a period of 0.2 seconds and 60 intervals.

\section{Conclusion}
If average signal is 4 per minute, 9 crabs gives a 30\% chance of collision for 0.2 second periods. When determining factors such as detection radius and length of signal space, it is advised to look at 3 different tagged crab densities: the conservative 1 crab per 1000$m^2$; the more likely 1 crab per 8500$m^2$; and the midpoint of the two estimates of 1 crab per 4750$m^2$. For each estimate, calculate the number of crabs within the detection radius. Calculate probabilities based on signal time and space. If the number of crabs in the detection radius for the middle estimate has a probability of collision over 30\%, variables should be adjusted.
\newpage
\section{Tables}

\begin{table}[ht]
\centering
\caption{Number of tagged crabs based on detection radius}
%\label{my-label}
\begin{tabular}{|c|c|c|c|}
\hline
\textbf{Radius (meters)} & \textbf{Area (meters squared)} & \textbf{\# of crabs - low} & \textbf{\# of crabs - high} \\ \hline
50                       & 7854                           & 1                          & 8                           \\ \hline
100                      & 31,416                         & 4                          & 31                          \\ \hline
150                      & 70,686                         & 8                          & 71                          \\ \hline
200                      & 125,664                        & 15                         & 126                         \\ \hline
250                      & 196,350                        & 23                         & 196                         \\ \hline
\end{tabular}
\end{table}

\begin{table}[ht]
\centering
\caption{Crab Collision Probability with 0.2 second periods}
%\label{my-label}
\begin{tabular}{|c|c|}
\hline
Number of crabs & Probability of collision \\ \hline
1               & 0                        \\ \hline
2               & .017                     \\ \hline
3               & .05                      \\ \hline
4               & .096                      \\ \hline
5               & .15                      \\ \hline
6               & .22                      \\ \hline
7               & .30                      \\ \hline
8               & .38                      \\ \hline
9               & .45                      \\ \hline
10              & .53                      \\ \hline
11              & .60                      \\ \hline
12              & .67                        \\ \hline
\end{tabular}
\end{table}
\begin{table}[ht]
\centering
\caption{Crab Collision Probability with 0.5 second periods}
%\label{my-label}
\begin{tabular}{|c|c|}
\hline
Number of crabs & Probability of collision \\ \hline
1               & 0                        \\ \hline
2               & .042                     \\ \hline
3               & .12                      \\ \hline
4               & .23                      \\ \hline
5               & .35                      \\ \hline
6               & .47                      \\ \hline
7               & .59                      \\ \hline
8               & .70                      \\ \hline
9               & .78                      \\ \hline
10              & .85                      \\ \hline
11              & .90                      \\ \hline
12              & .94                        \\ \hline
\end{tabular}
\end{table}

\end{document}


\section{Syntax and Semantics}\label{sec:abstractrefinements:check}

Next, we present a core calculus \corelan that formalizes the notion
of abstract refinements. We start with the syntax (\S~\ref{sec:syntax}),
present the typing rules (\S~\ref{sec:abstractrefinements:typing}), show soundness
via a reduction to contract calculi \cite{Knowles10,Greenberg11}
(\S~\ref{sec:soundness}), and inference via Liquid types (\S~\ref{sec:abstractrefinements:infer}).

\subsection{Syntax}\label{sec:syntax}

Figure~\ref{fig:abstractrefinements:syntax} summarizes the syntax of our core
calculus \corelan which is a polymorphic $\lambda$-calculus
extended with abstract refinements.
%
We write
$b$,
$\tref{b}{\reft}$ and
$\tpp{b}{\areft}$
to abbreviate
$\tpref{b}{\true}{\true}$,
$\tpref{b}{\true}{\reft}$, and
$\tpref{b}{\areft}{\true}$ respectively.
We say a type or schema is \emph{non-refined} if all the
refinements in it are $\true$. We write $\overline{z}$
to abbreviate a sequence $z_1 \ldots z_n$.

\mypara{Expressions}
\corelan\ expressions include the standard variables $x$,
primitive constants $c$, $\lambda$-abstraction $\efunt{x}{\tau}{e}$,
application $\eapp{e}{e}$, type abstraction $\etabs{\alpha}{e}$,
and type application $\etapp{e}{\tau}$. The parameter $\tau$ in
the type application is a \emph{refinement type}, as described shortly.
The two new additions to \corelan are the refinement abstraction
$\epabs{\rvar}{\tau}{e}$, which introduces a refinement variable
$\rvar$ (together with its type $\tau$), which can appear in refinements
inside $e$, and the corresponding refinement application $\epapp{e}{e}$.
%
%where the argument, is of the form $\efun{\bar{x}{e}}$ which is
%an abbreviation for $\efun{x_1 \ldots x_n}{e}$.
%which is of the form $\ptype{\bar{\tau}}$ which is an
%abbreviation for $\ptype{tau_1 \rightarrow \ldots \tau_n}$,
%where each $\tau_i$ is a simple (non-function) type.

\mypara{Refinements}
A \emph{concrete refinement} \reft is a boolean valued expression \reft
drawn from a strict subset of the language of expressions which
includes only terms that
(a)~neither diverge nor crash and
(b)~can be embedded into an SMT decidable refinement logic including
the theory of linear arithmetic and uninterpreted functions.
%
An \emph{abstract refinement} \areft is a conjunction of refinement
variable applications of the form $\rvapp{\pi}{e}$.

\mypara{Types and Schemas}
The basic types of \corelan include the base types $\tbint$ and $\tbbool$
and type variables $\alpha$. An \emph{abstract refinement type} $\tau$ is
either a basic type refined with an abstract and concrete refinements,
$\tpref{b}{\areft}{\reft}$, or
a dependent function type where the parameter $x$ can appear in the
refinements of the output type.
We include refinements for functions, as refined type variables can be
replaced by function types. However, typechecking ensures these refinements
are trivially true.
%
%type application
%Type application consists of a type constructor, its type arguments
%and its refinement arguments
%that are used to describe properties between its elements.
%
Finally, types can be quantified over refinement variables and type
variables to yield abstract refinement schemas.

\begin{figure}[t!]
\centering
\captionsetup{justification=centering}
$$
\begin{array}{rrcl}
\emphbf{Expressions} \quad
  & e
  & ::=
  &      x
  \spmid c
  \spmid \efunt{x}{\tau}{e}
  \spmid \eapp{e}{e} \\
  &&\spmid &\etabs{\alpha}{e}
  \spmid \etapp{e}{\tau}
  \spmid \epabs{\rvar}{\tau}{e}
  \spmid \epapp{e}{e}
  \\[0.05in]

\emphbf{Abstract Refinements} \quad
  & \areft
  & ::=
  &      \true
  \spmid \areft \land \rvapp{\rvar}{e}
  \\[0.05in]

\emphbf{Basic Types} \quad
  & b
  & ::=
  &      \tbint
  \spmid \tbbool
  \spmid \alpha
  \\[0.05in]

\emphbf{Abstract Refinement Types} \quad
  & \tau
  & ::=
  &      \tpref{b}{\areft}{\reft}
  \spmid \trfun{x}{\tau}{\tau}{\reft}
  \\[0.05in]

\emphbf{Abstract Refinement Schemas} \quad
  & \sigma
  & ::=
  &      \tau
  \spmid \ttabs{\alpha}{\sigma}
  \spmid \tpabs{\rvar}{\tau}{\sigma}
  \\[0.05in]
\end{array}
$$
\caption[Syntax of \corelan.]{\textbf{Syntax of Expressions, Refinements, Types and Schemas of \corelan.}}
\label{fig:abstractrefinements:syntax}
\end{figure}

\subsection{Static Semantics}\label{sec:abstractrefinements:typing}

\input{text/abstractrefinements/rules}

Next, we describe the static semantics of \corelan by describing the typing
judgments and derivation rules. Most of the rules are
standard~\cite{Ou2004,LiquidPLDI08,Knowles10,GordonTOPLAS2011}; we
discuss only those pertaining to abstract refinements.
%

\mypara{Judgments}
A type environment $\Gamma$ is a sequence of type bindings $x:\sigma$.
We use environments to define three kinds of typing judgments.

\mypara{Wellformedness judgments (\isWellFormed{\Gamma}{\sigma})}
state that a type schema $\sigma$ is well-formed under environment
$\Gamma$, that is, the refinements in $\sigma$ are boolean
expressions in the environment $\Gamma$.
%
The wellformedness rules check that the concrete and abstract
refinements are indeed $\tbbool$-valued expressions in the
appropriate environment.
The key rule is \wtBase, which checks, as usual, that the (concrete)
refinement $\reft$ is boolean and additionally, that the abstract
refinement $\areft$ applied to the value $\vref$ is also boolean.
This latter fact is established by \wtRVApp which checks that
each refinement variable application $\rvapp{\rvar}{e}\ \vref$
is also of type \tbbool in the given environment.

\mypara{Subtyping judgments}
(\isSubType{\Gamma}{\sigma_1}{\sigma_2})
state that the type schema $\sigma_1$ is a subtype of the type schema
$\sigma_2$ under environment $\Gamma$, that is, when the free variables
of $\sigma_1$ and $\sigma_2$
are bound to values described by $\Gamma$, the set of values described
by $\sigma_1$ is contained in the set of values described by $\sigma_2$.
%
The rules are standard except for \tsubVar, which encodes the base types'
abstract refinements $\areft_1$ and $\areft_2$ with conjunctions of
\emph{uninterpreted predicates}
$\inter{\areft_1\ \vref}$ and $\inter{\areft_2\ \vref}$ in the
refinement logic as follows:
\begin{align*}
\inter{\true\ \vref} & \defeq \true\\
\inter{(\areft \land \rvapp{\rvar}{e})\ \vref} & \defeq \inter{\areft\
\vref} \land \rvar(\inter{e_1},\ldots,\inter{e_n},\vref)
\end{align*}
where $\rvar(\overline{e})$ is a term in the refinement logic corresponding
to the application of the uninterpreted predicate symbol $\rvar$ to the
arguments $\overline{e}$.
% $\text{Valid}(p)$ (\tsubBase) holds if an SMT determines the formula $p$
% is \emph{valid}~\cite{Nelson81}.

\mypara{Typing judgments}
(\hastype{\Gamma}{e}{\sigma}) state that
the expression $e$ has the type schema $\sigma$ under environment $\Gamma$,
that is, when the free variables in $e$ are bound to values described by
$\Gamma$, the expression $e$ will evaluate to a value described by $\sigma$.
%
The type checking rules are standard except for \tpgen and \tpinst, which
pertain to abstraction and instantiation of abstract refinements.
%
The rule \tpgen is the same as \tfunction: we simply check the body
$e$ in the environment extended with a binding for the refinement
variable $\rvar$.
%
The rule \tpinst checks that the concrete refinement is of the appropriate
(unrefined) type $\tau$, and then replaces all (abstract) applications of
$\rvar$ inside $\sigma$ with the appropriate (concrete) refinement $\reft'$
with the parameters $\overline{x}$ replaced with arguments at that application.
Formally, this is represented as $\rpinst{\sigma}{\rvar}{\efunbar{x:\tau}{\reft'}}$
which is $\sigma$ with each base type transformed as
\begin{align*}
\rpinst{\tpref{b}{\areft}{\reft}}{\rvar}{z}
  & \defeq \tpref{b}{\areft''}{\reft \land \reft''} \\
\mbox{where} \quad (\areft'', \reft'')
  & \defeq \rpapply{\areft}{\rvar}{z}{\true}{\true}
\intertext{$\mathsf{Apply}$ replaces each application of $\rvar$ in
$\areft$ with the corresponding conjunct in $\reft''$, as}
\rpapply{\true}{\cdot}{\cdot}{\areft'}{\reft'}
  & \defeq (\areft', \reft') \\
\rpapply{\areft \wedge \rvapp{\rvar'}{e}}{\rvar}{z}{\areft'}{\reft'}
  & \defeq \rpapply{\areft}{\rvar}{z}{\areft' \land \rvapp{\rvar'}{e}}{\reft'} \\
\rpapply{\areft \wedge \rvapp{\rvar}{e}}{\rvar}{\efunbar{x:\tau}{\reft''}}{\areft'}{\reft'}
  & \defeq
  \rpapply{\areft}{\rvar}{\efunbar{x:\tau}{\reft''}}{\areft'}{\reft' \wedge \SUBST{\reft''}{\overline{x}}{\overline{e},\vref}}
\end{align*}
In other words, the instantiation can be viewed as two symbolic
reduction steps: first replacing the refinement variable with the
concrete refinement, and then ``beta-reducing" concrete refinement
with the refinement variable's arguments. For example,
$$\rpinst{\tpref{\tbint}{\rvar\ y}{\vref > 10}}
       {\rvar}
       {\efunt{x_1}{\tau_1}{\efunt{x_2}{\tau_2}{x_1 < x_2}}}
\defeq \tref{\tbint}{\vref > 10 \land y < \vref}$$
%%rp(x:tx->t , \rvar, z) = x:tx' -> t'
%%  where tx'      = rp(tx, \rvar, z)
%%        t'       = rp(t , \rvar, z)
%%
%%rp(\a.t, \rvar, z) = \a.t'
%%  where t'       = rp(t, \rvar, z)
%%
%%rp(\p.t, \rvar, z) = \p.t'
%%  where t'       = rp(t, \rvar, z)

%%The other rule that handles abstract refinements is \tcase.
%%This rule initially checks that the expression to be analyzed
%%has a type application type $T = \tcon{\chi}{e_\chi}{\listOf{T}}{\listOf{e}}$.
%%Then for all cases, the case expression is typechecked in the initial environment,
%%extended with case binders \listOf{x_i} and the initial expression binder $x$.
%%The types of these binders are gained after unfolding data constructor's type \tc{K_i}.
%%The unfolding is done by replacing its type variables with actual type arguments
%%of $T$, ie. \listOf{T}
%%its abstract refinements with actual inferred refinements \listOf{e},
%%and its binders with actual binders \listOf{x_i}.

\subsection{Soundness}\label{sec:abstractrefinements:soundness}

As hinted by the discussion about refinement variable instantiation,
we can intuitively think of abstract refinement variables as
\emph{ghost} program variables whose values are boolean-valued
functions. Hence, abstract refinements are a special case of
higher-order contracts, that can be statically verified using
uninterpreted functions. (Since we focus on static checking,
we don't care about the issue of blame.)
We formalize this notion by translating \corelan programs into
the contract calculus \conlan of \cite{Greenberg11} and use this
translation to define the dynamic semantics and establish soundness.

\mypara{Translation}
We translate \corelan schemes $\sigma$ to \conlan schemes $\tx{\sigma}$
as by translating abstract refinements into contracts,
and refinement abstraction into function types:

\[\arraycolsep=0.5pt
\begin{array}{rclcrcl}
\tx{\true\ \vref} & \defeq
  & \true
  & \quad \quad &

\tx{\tpabs{\rvar}{\tau}{\sigma}} & \defeq
  & \tfun{\rvar}{\tx{\tau}}{\tx{\sigma}} \\

\tx{(\areft \land \rvapp{\rvar}{e})\ \vref} & \defeq
  & \tx{\areft\ \vref} \land \eapp{\eapp{\rvar}{\overline{e}}}{\vref}
  & \quad \quad &

\tx{\ttabs{\alpha}{\sigma}} & \defeq
  & \ttabs{\alpha}{\tx{\sigma}} \\

\tx{\tpref{b}{\areft}{\reft}} & \defeq
  & \tref{b}{\reft \land \tx{\areft\ \vref}}
  & \quad \quad &

\tx{\tfun{x}{\tau_1}{\tau_2}} & \defeq
  & \tfun{x}{\tx{\tau_1}}{\tx{\tau_2}}
%\tx{\trfun{x}{\tau_1}{\tau_2}{\reft}} \defeq
%  & \trfun{x}{\tx{\tau_1}}{\tx{\tau_2}}{\tx{\reft}} \\
\end{array}\]
Similarly, we translate \corelan terms $e$ to \conlan
terms $\tx{e}$ by converting refinement abstraction and application
to $\lambda$-abstraction and application
\[\arraycolsep=0.5pt
\begin{array}{rclcrcl}
\tx{x} & \defeq & x & \quad \quad \quad & \tx{c} & \defeq & c \\
\tx{\efunt{x}{\tau}{e}} & \defeq & \efunt{x}{\tx{\tau}}{\tx{e}} & \quad & \tx{\eapp{e_1}{e_2}} & \defeq & \eapp{\tx{e_1}}{\tx{e_2}} \\
\tx{\etabs{\alpha}{e}} & \defeq & \etabs{\alpha}{\tx{e}} & \quad & \tx{\etapp{e}{\tau}} & \defeq & \eapp{\tx{e}}{\tx{\tau}} \\
\tx{\epabs{\rvar}{\tau}{e}} &\defeq & \efunt{\rvar}{\tx{\tau}}{\tx{e}} & \quad & \tx{\epapp{e_1}{e_2}} &\defeq & \eapp{\tx{e_1}}{\tx{e_2}}
\end{array}\]

%%\begin{align*}
%%\tx{\true\ \vref} \defeq
%%  & \true\\
%%\tx{(\areft \land \rvapp{\rvar}{e})\ \vref} \defeq
%%  & \tx{\areft\ \vref} \land \eapp{\eapp{\rvar}{\overline{e}}}{\vref}\\
%%\tx{\tpref{b}{\areft}{\reft}} \defeq
%%  & \tref{b}{\reft \land \tx{\areft\ \vref}} \\
%%\tx{\tfun{x}{\tau_1}{\tau_2}} \defeq
%%  & \tfun{x}{\tx{\tau_1}}{\tx{\tau_2}} \\
%%%\tx{\trfun{x}{\tau_1}{\tau_2}{\reft}} \defeq
%%%  & \trfun{x}{\tx{\tau_1}}{\tx{\tau_2}}{\tx{\reft}} \\
%%\tx{\ttabs{\alpha}{\sigma}} \defeq
%%  & \ttabs{\alpha}{\tx{\sigma}} \\
%%\tx{\tpabs{\rvar}{\tau}{\sigma}} \defeq
%%  & \tfun{\rvar}{\tx{\tau}}{\tx{\sigma}}
%%\end{align*}
%%\tx{x} \defeq & x \\
%%\tx{c} \defeq & c \\
%%\tx{\efunt{x}{\tau}{e}} \defeq & \efunt{x}{\tx{\tau}}{\tx{e}} \\
%%\tx{\eapp{e_1}{e_2}} \defeq & \eapp{\tx{e_1}}{\tx{e_2}} \\
%%\tx{\etabs{\alpha}{e}} \defeq & \etabs{\alpha}{\tx{e}} \\
%%\tx{\etapp{e}{\tau}} \defeq & \eapp{\tx{e}}{\tx{\tau}} \\
%%\tx{\epabs{\rvar}{\tau}{e}} \defeq & \efunt{\rvar}{\tx{\tau}}{\tx{e}} \\
%%\tx{\epapp{e_1}{e_2}} \defeq & \eapp{\tx{e_1}}{\tx{e_2}}

\mypara{Translation Properties}
We can show by induction on the derivations that the
type derivation rules of \corelan \emph{conservatively approximate}
those of \conlan. Formally,

\begin{itemize}
\item If $\isWellFormed{\Gamma}{\tau}$ then $\isWellFormedH{\Gamma}{\tau}$,
\item If $\isSubType{\Gamma}{\tau_1}{\tau_2}$ then $\isSubTypeH{\Gamma}{\tau_1}{\tau_2}$,
\item If $\hastype{\Gamma}{e}{\tau}$ then
$\hastypeH{\tx{\Gamma}}{\tx{e}}{\tx{\tau}}$.
\end{itemize}

\mypara{Soundness} Thus rather than re-prove preservation and progress
for \corelan, we simply use the fact that the type derivations are
conservative to derive the following preservation and progress
corollaries from \cite{Greenberg11}:
%
\begin{itemize}
\item{\textbf{Preservation: }}
  If $\hastype{\emptyset}{e}{\tau}$
  and $\tx{e} \longrightarrow e'$
  then $\hastypeH{\emptyset}{e'}{\tx{\tau}}$

\item{\textbf{Progress: }}
  If $\hastype{\emptyset}{e}{\tau}$, then either
  $\tx{e} \longrightarrow e'$ or
  $\tx{e}$ is a value.
\end{itemize}
%
Note that, in a contract calculus like \conlan, subsumption is encoded
as a \emph{upcast}. However, if subtyping relation can be statically
guaranteed (as is done by our conservative SMT based subtyping)
then the upcast is equivalent to the identity function and can
be eliminated. Hence, \conlan terms $\tx{e}$ translated from well-typed
\corelan terms $e$ have no casts.

\subsection{Refinement Inference}\label{sec:abstractrefinements:infer}

Our design of abstract refinements makes it particularly easy to
perform type inference via Liquid typing, which is crucial for
making the system usable by eliminating the tedium of instantiating
refinement parameters all over the code. (With value-dependent
refinements, one cannot simply use, say, unification to determine
the appropriate instantations, as is done for classical type systems).
We briefly recall how Liquid types work, and sketch how they are
extended to infer refinement instantiations.

\mypara{Liquid Types}
The Liquid Types method infers refinements in three steps.
%
First, we create refinement \emph{templates} for the unknown,
to-be-inferred refinement types. The \emph{shape} of the template
is determined by the underlying (non-refined) type it corresponds to,
which can be determined from the language's underlying (non-refined)
type system.
The template is just the shape refined with fresh refinement variables
$\kappa$ denoting the unknown refinements at each type position.
For example, from a type ${\tfun{x}{\tbint}{\tbint}}$ we create
the template ${\tfun{x}{\tref{\tbint}{\kappa_x}}{\tref{\tbint}{\kappa}}}$.
%
Second, we perform type checking using the templates (in place of the
unknown types). Each wellformedness check becomes a wellformedness
constraint over the templates, and hence over the individual $\kvar$,
constraining which variables can appear in $\kvar$.
Each subsumption check becomes a subtyping constraint
between the templates, which can be further simplified, via syntactic
subtyping rules, to a logical implication query between the variables
$\kappa$.
%
Third, we solve the resulting system of logical implication constraints
(which can be cyclic) via abstract interpretation --- in particular,
monomial predicate abstraction over a set of logical qualifiers
\cite{Houdini,LiquidPLDI08}. The solution is a map from $\kvar$ to
conjunctions of qualifiers, which, when plugged back into the templates,
yields the inferred refinement types.

\mypara{Inferring Refinement Instantiations}
The key to making abstract refinements practical is a means of
synthesizing the appropriate arguments $\reft'$ for each refinement
application $\epapp{e}{\reft'}$.
Note that for such applications, we can, from $e$, determine the
non-refined type of $\reft'$, which is of the form
${\tau_1 \rightarrow \ldots \rightarrow \tau_n \rightarrow \tbbool}$.
Thus, $\reft'$ has the template
${\efunt{x_1}{\tau_1}{\ldots \efunt{x_n}{\tau_n}{\kvar}}}$
where $\kvar$ is a fresh, unknown refinement variable that
must be solved to a boolean valued expression over $x_1,\ldots,x_n$.
Thus, we generate a \emph{wellformedness} constraint
${\isWellFormed{x_1:\tau_1, \ldots, x_n:\tau_n}{\kvar}}$
and carry out typechecking with template, which, as before, yields
implication constraints over $\kvar$, which can, as before, be
solved via predicate abstraction.
Finally, in each refinement template, we replace each $\kvar$ with its
solution $e_\kvar$ to get the inferred refinement instantiations.

%%To infer appropabstract refinements we used the liquid type variables
%%$\kappa$ with explicit arguments to avoid inferring function
%%expressions:
%%To infer an expression to replace a predicate of type
%%$\listOf{x_i:\tau_i; \tau}$ in an environment $\Gamma$,
%%we create a fresh liquid variable $\kappa$ on $\tau$ which is
%%wellformed in the environment $\Gamma$ extended with the bindings
%%\listOf{x_i:\tau_i}.
%%When $\kvar$ is solved via predicate abstraction to an expression
%%$e_\kvar$, we simply replace $\kvar$ with
%%the set as the inferred expression
%%$e =\efun{x_1}{\dots\efun{x_n}{\efun{v}{e_\kappa}}}$.
%add a refinement variable in place of expressions.
%\mypara{Constants}\jhala{constant-refinements and soundness guarantees?}
%%TODO:
%%[SKIP] add paragraph on constants to opsem
%%[SKIP] redefine \reft to r? (to emphasize not arbitrary expression?)


\section{Method}
\label{sec:method}
\label{sec:algorithm}

\paragraph{Assumption}

The key assumption about textures is locality under contextual control \cite{Wei:2009:STTS}.
In particular, each texture sample (e.g. pixel of an image texture or vertex of a mesh texture) relates to only those that are in its spatial vicinity (locality) \cite{Efros:1999:TSN}, but its property is conditioned upon global control such as environment variables (context) \cite{Lu:2007:CT}.
Taking these two together, a texture consists of repetitive patterns that are local but potentially globally varying.

\paragraph{Idea}

Our method is adapted from
k-coherence \cite{Tong:2002:SBT} and randomized patch-match \cite{Barnes:2009:PRC} for quality\nothing{efficacy}, efficiency, flexibility, and simplicity.
The basic idea stems from the coherence observation: regardless of the exact synthesis algorithms used, as long as the output has sufficient quality, it is usually composed of coherent patches from the input exemplar.
In other words, if we record the source position $\position(\inputex, \sample)$ within the input exemplar $\inputex$ from which each output pixel $\sample \in \outputex$ comes from, the positions usually form coherent patches as visualized in \cite{Wei:2002:TSF}.
This implies that to determine the color value $\range(\sample)$ of $\sample$, we can look at each one of its already synthesized neighbor $\sample' \in \outputex$ with input position $\position(\inputex, \sample')$, and use their relative input and output position offsets to compute a good potential candidate:
\begin{align}
\position(\inputex, \sample' \rightarrow \sample) = \position(\outputex, \sample) - \position(\outputex, \sample') + \position(\inputex, \sample')
\label{eqn:coherence_offset}
\end{align}
, where $\position(\inputex, \sample' \rightarrow \sample)$ indicates a potentially good source position of $\sample$ at the input $\inputex$ as voted from an already computed output pixel $\sample'$.
We can collect a set of such $\sample'$ in the (coherence) neighborhood $\corneighborhood$ of $\sample$, and see which one has the best matched (texture) neighborhood $\texneighborhood$:
\begin{align}
\range(\outputex, \sample) \leftarrow \range\left(\argmin_{\sample'} \left\{\left|\texneighborhood(\outputex, \sample) - \texneighborhood(\outputex, \sample')\right|^2 \forall \sample' \in \corneighborhood(\outputex, \sample) \right\} \right)
\label{eqn:coherence_synthesis}
\end{align}
The coherence and texture neighborhoods $\corneighborhood$ and $\texneighborhood$ are usually small regions with specific shapes and sizes, e.g. circles or squares.

\paragraph{Variations}

Both k-coherence \cite{Tong:2002:SBT} and patch-match \cite{Barnes:2009:PRC} are generalizations of the main idea expressed in \Cref{eqn:coherence_synthesis}.
In particular, \cite{Tong:2002:SBT} pre-computes a set of $\knumber$ candidates with similar texture neighborhoods to each input pixel $\sample \in \inputex$, so that each $\sample$ can provide $\knumber$ instead of just $1$ votes in \Cref{eqn:coherence_offset}.
Empirical evidence indicates that this can indeed improve quality (with sufficiently small $\knumber$ in the range of 4 to 8), but this pre-computation can be too expensive for large input exemplars.
Patch-match \cite{Barnes:2009:PRC} also bypasses this pre-computation, but instead of adding $\knumber$ random candidates to \Cref{eqn:coherence_synthesis}, which works surprisingly well.

Optimization \cite{Kwatra:2005:TOE,Han:2006:TDO} can be imposed as an orthogonal component, but greedy assignment usually suffices.

\paragraph{Constraint}

Constraints can be easily added by simply presetting the colors of certain output pixels in \Cref{eqn:coherence_synthesis}, and let the synthesis process assign neighboring output to have consistent texture patterns.
Good output quality can never be fully guaranteed as it depends on several factors, including whether the constraints leave enough degrees of freedom for optimization.

\paragraph{Multi-resolution}

The basic ideas above can be directly extended for multi-resolution synthesis \cite{Wei:2000:FTS} via the following basic steps:
\begin{enumerate}
\item
Compute a Gaussian pyramid from the input with number of levels specified by the user.

\item
Synthesize the output from lower to higher levels, with each level generated from the corresponding input level.
Each level computation is very similar to the single-resolution algorithm above, except that we can use multi-resolution neighborhoods (for both similarity measure and coherence).

\item
Write out the highest output level.
\end{enumerate}

Hard constraints are currently handled directly as invariant values inside the output pyramid during multi-resolution synthesis.
This is not theoretically optimal, as hard constraints should be on the highest level only.
But it seems to work well in practice.
For better theoretical approach, we will need to pursue a joint optimization schemes such as \cite{Kwatra:2005:TOE,Han:2006:TDO} or a multi-grid solver.