% ---------------------------------------------------------------------
\documentclass{article}

\usepackage{nberpreamble}
\usepackage{tikz}
\usetikzlibrary{
  arrows,
  patterns,
  positioning,
  calc,
  fit,
  intersections,
  decorations.text,
  decorations.markings,
  decorations.pathmorphing,
  shadows.blur
}

\title{\bfseries Notes on Parametric Power \\ for Clustered Randomization}
\author{\sffamily Prepared by Mauricio C\'aceres}
\date{\sffamily \today}

\renewcommand{\displayoptions}{
  \maketitle
  \pagenumbering{arabic}
}

% ---------------------------------------------------------------------
\begin{document}
\displayoptions

%----------------------------------------------------------------------
\section{MDE and Sample Size with a Cluster Design}
\label{sec:mde_and_sample_size_with_a_cluster_design}

Recall \Cref{eq:cluster_reg}; without controls we have
\begin{equation}
\label{eq:cluster_reg}
Y_{ij} = \alpha + \beta T_j + \varepsilon_{ij},
\end{equation}

with $E (\varepsilon_{ij} \varepsilon_{i^\prime j}) \ne 0$ and the randomization $T_j$ at the cluster level ($j = 1, \ldots, J$ clusters and $n_j$ individuals per cluster, $\sum^{J}_{j = 1} n_j = N$). Following \citet[p. 3921-2]{DufloGlennersterKremer2007}, suppose we can additively decompose the error term $\varepsilon_{ij}$ as
\begin{equation}
\label{eq:cluster_reg_additive}
Y_{ij} = \alpha + \beta T_j + v_j + u_{ij},
\end{equation}

with $v_j \stackrel{iid}{\sim} (0, \sigma_v^2), u_{ij} \stackrel{iid}{\sim} (0, \sigma_u^2)$. \citeauthor{DufloGlennersterKremer2007} provide a formula for the case when we have equal cluster sizes, but not the case when cluster sizes vary. Let us further assume $n_j \stackrel{iid}{\sim} (\mu_n, \sigma^2_n)$ and $\rho \equiv \sigma_v^2 / \sigma_\varepsilon^2$ so that $E (\varepsilon_{ij} \varepsilon_{i^\prime j}) = \rho \sigma_\varepsilon^2$. For significance level $\alpha$, power $\kappa$, and proportion randomized $P$ we have
\begin{equation}
  \begin{array}{rl}
    J   & = \left(\dfrac{t_{1 - \kappa} + t_{\alpha / 2}}{MDE}\right)^2 \dfrac{DE \cdot \sigma_\varepsilon^2}{\mu_n P(1 - P)}
          = N_0 \dfrac{DE}{\mu_n} \\[9pt]
    MDE & = \Fabs{t_{1 - \kappa} + t_{\alpha / 2}} \sqrt{\dfrac{DE \cdot \sigma_\varepsilon^2}{J \mu_n P(1 - P)}}
          = MDE_0 \cdot \sqrt{DE},
  \end{array}
\end{equation}

where $MDE_0, N_0$ are the MDE and sample size required if the model used individual data and $DE$ is the so-called design effect (DE) or variance inflation factor (VIF):
\begin{equation}
  DE =
  \begin{cases}
    1 + \rho (\mu_n - 1) & \sigma^2_n = 0 \\
    1 + \rho ((\sigma^2_n / \mu_n^2 + 1) \mu_n - 1) & \sigma^2_n > 0
  \end{cases}.
\end{equation}

The formulas above are a slight modification of equations (1) and (3) in \citet{manatunga_sample_2001} to account for the case when the proportion randomized $P \ne 0.5$. Note we're leveraging the fact that testing the significance of $\widehat{\beta}_{OLS}$ is equivalent to a paired $t$-test in this case. If $Y_{ij}$ is also binary then we need to adjust the variance. Equation (3) in \citet{kong_sample_2003} gives
\begin{equation}
J = \left(\dfrac{t_{1 - \kappa} + t_{\alpha / 2}}{MDE}\right)^2 \dfrac{DE}{\mu_n P(1 - P)}
    \Big(\mu_T (1 - \mu_T) (1 - P) + \mu_C (1 - \mu_C) P\Big).
\end{equation}

Again, we modify the formula slightly so we account for $P \ne 0.5$. Note that the variance of $\widehat{\beta}_{OLS}$ is
\begin{equation}
\begin{array}{rl}
    V_{\widehat{\beta}} & = \widehat{V}_T + \widehat{V}_C \\
    V_{k} & = \dfrac{\sum^{J}_{j = 1} n_{ik} \left(1 + (n_{jk} - 1) \rho\right)}{\left(\sum^{J}_{j = 1} n_{jk}\right)^2} \sigma^2_{\varepsilon}
        \quad\quad k = T, C.
  \end{array}
\end{equation}

$J V_{\widehat{\beta}} \xrightarrow{P} \sigma^2_{\varepsilon} DE / \mu_n$ as $J \to \infty$, meaning we can estimate $DE$ if the cluster sizes are known, given an estimate of $\rho$. \citeauthor{kong_sample_2003} suggests using an ANOVA-based estimate. Intuitively, from \cref{eq:cluster_reg_additive} we see that a random effects model $Y_{ij} = \alpha + v_j + u_{ij}$ is the true model under the null of $H_0: \beta = 0$; then $\rho = \sigma_v^2 / (\sigma_u^2 + \sigma_v^2)$.

%----------------------------------------------------------------------
\section{The Effect of Covariates}
\label{sec:the_effect_of_covariates}

Adding controls has the effect of absorbing some of the variation in the error terms, thereby reducing $\sigma_\varepsilon^2$ and $\rho$ and improving the precision of our estimates. The proportion of the unexplained variation absorbed by the covariates will be $R^2$, meaning we can adjust our parametric estimates to account for covariates by multiplying the variance by $1 - R^2$.

For the intra-cluster correlation $\rho$ there is no trivial way of accounting for an arbitrary number of covariates---however, it is possible to make a simple adjustment for any one covariate. We follow the approach outlined in \citet{stanish_estimation_1983} and adjust the ANOVA-based estimate for $\rho$ to account for the lag of the outcome variable. Taking $MSB$ and $MSW$ as they are usually defined, we have
\begin{align*}
  n & = \dfrac{1}{J - 1} \left[N - \sum^{}_{j} n_j^2 / N \right] \\
  k & = \dfrac{1}{J - 1} \left[\dfrac{\sum^{}_{j} n_j^2 (\overline{x}_j - \overline{x})^2}{\sum^{}_{j} \sum^{}_{i} (x_{ij} - \overline{x})^2}\right] \\
  \widehat{\rho} & = \dfrac{MSB - MSW}{MSB + (n - k - 1) MSW},
\end{align*}

where the unadjusted $\widehat{\rho}$ would simply use $k = 0$.

%----------------------------------------------------------------------
% \clearpage

\bibliographystyle{apalike}
\bibliography{power.bib}

%----------------------------------------------------------------------
\end{document}


\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[osf]{libertine}
\usepackage[scaled=0.8]{beramono}
\usepackage[margin=1.5in]{geometry}
\usepackage{url}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{nicefrac}
\usepackage{microtype}
\usepackage{subcaption}
\usepackage{bm}

\usepackage{sectsty}
\sectionfont{\large}
\subsectionfont{\normalsize}

\usepackage{titlesec}
\titlespacing{\section}{0pt}{10pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}
\titlespacing{\subsection}{0pt}{5pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}

\usepackage{pgfplots}
\pgfplotsset{
  compat=newest,
  plot coordinates/math parser=false,
  tick label style={font=\footnotesize, /pgf/number format/fixed},
  label style={font=\small},
  legend style={font=\small},
  every axis/.append style={
    tick align=outside,
    clip mode=individual,
    scaled ticks=false,
    thick,
    tick style={semithick, black}
  }
}

\pgfkeys{/pgf/number format/.cd, set thousands separator={\,}}

\usepgfplotslibrary{external}
\tikzexternalize[prefix=tikz/]

\newlength\figurewidth
\newlength\figureheight

\setlength{\figurewidth}{12cm}
\setlength{\figureheight}{6cm}

\newlength\squarefigurewidth
\newlength\squarefigureheight

\setlength{\squarefigurewidth}{4cm}
\setlength{\squarefigureheight}{4cm}

\newlength\smallsquarefigurewidth
\newlength\smallsquarefigureheight

\setlength{\smallsquarefigurewidth}{3.25cm}
\setlength{\smallsquarefigureheight}{3.25cm}

\newlength\smallfigurewidth
\newlength\smallfigureheight

\setlength{\smallfigurewidth}{6.25cm}
\setlength{\smallfigureheight}{4cm}

\setlength{\parindent}{0pt}
\setlength{\parskip}{1ex}

\newcommand{\acro}[1]{\textsc{\MakeLowercase{#1}}}
\newcommand{\given}{\mid}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\data}{\mc{D}}
\newcommand{\intd}[1]{\,\mathrm{d}{#1}}
\newcommand{\inv}{^{-1}}
\newcommand{\trans}{^\top}
\newcommand{\mat}[1]{\bm{\mathrm{#1}}}
\renewcommand{\vec}[1]{\bm{\mathrm{#1}}}
\newcommand{\R}{\mathbb{R}}
\renewcommand{\epsilon}{\varepsilon}

\DeclareMathOperator{\var}{var}
\DeclareMathOperator{\cov}{cov}
\DeclareMathOperator{\diag}{diag}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}

\begin{document}

\section*{The Kernel Trick}

Consider the assumption of linear regression with an explicit feature
expansion $\phi\colon \vec{x} \mapsto \phi(\vec{x})$:
\begin{equation*}
  y(\vec{x}) = \phi(\vec{x})\trans \vec{w} + \epsilon(\vec{x}).
\end{equation*}
Given training data $\data = \bigl\{ (\vec{x}, y) \bigr\} = (\mat{X},
\vec{y})$, recall we modeled the residuals $\epsilon(\vec{x})$ as
zero-mean independent, identically distributed Gaussians with variance
$\sigma^2$:
\begin{equation*}
  p(\vec{\epsilon})
  =
  \mc{N}(\vec{\epsilon}; \vec{0}, \sigma^2 \mat{I}),
\end{equation*}
giving rise to the following likelihood:
\begin{equation*}
  p(\vec{y} \given \mat{X}, \vec{w}, \sigma^2)
  =
  \mc{N}(\vec{y};\mat{\Phi}\trans \vec{w}, \sigma^2 \mat{I}).
\end{equation*}
where we have defined $\mat{\Phi} = \phi(\mat{X})$.

In Bayesian linear regression, we further chose a multivariate
Gaussian prior for $\vec{w}$:
\begin{equation*}
  p(\vec{w}) = \mc{N}(\vec{w}; \vec{\mu}, \mat{\Sigma}).
\end{equation*}
For simplicity, in the below we assume the prior mean for $\vec{w}$ is
$\vec{\mu} = \vec{0}$.

Given these assumptions, we were able to derive the posterior
distribution of $\vec{w}$ given the data $\data$:
\begin{equation*}
  p(\vec{w} \given \data, \sigma^2)
  =
  \mc{N}(\vec{w};
  \vec{\mu}_{\vec{w}\given\data},
  \mat{\Sigma}_{\vec{w}\given\data}
  ),
\end{equation*}
where
\begin{align*}
  \vec{\mu}_{\vec{w}\given\data}
  &=
  \mat{\Sigma}
  \mat{\Phi}\trans
  (\mat{\Phi}\mat{\Sigma}\mat{\Phi}\trans + \sigma^2 \mat{I})\inv
  \vec{y};
  \\
  \mat{\Sigma}_{\vec{w}\given\data}
  &=
  \mat{\Sigma}
  -
  \mat{\Sigma}
  \mat{\Phi}\trans
  (\mat{\Phi}\mat{\Sigma}\mat{\Phi}\trans + \sigma^2 \mat{I})\inv
  \mat{\Phi}
  \mat{\Sigma},
\end{align*}

If we wish to use our model to predict the outputs $\vec{y}_\ast$
associated with a set of inputs $\mat{X}_\ast$, we previously derived:
\begin{equation*}
  p(\vec{y}_\ast \given \mat{X}_\ast, \data, \sigma^2)
  =
  \mc{N}(
  \vec{y}_\ast;
  \mat{\Phi}_\ast \vec{\mu}_{\vec{w}\given\data},
  \mat{\Phi}_\ast \mat{\Sigma}_{\vec{w}\given\data} \mat{\Phi}_\ast\trans + \sigma^2 \mat{I}).
\end{equation*}
Examining the forms of these expressions, we see that the feature
expansion $\phi$ always appears in one of the following expressions:
\begin{equation*}
  \mat{\Phi}\mat{\Sigma}\mat{\Phi}\trans
  \qquad
  \mat{\Phi}_\ast\mat{\Sigma}\mat{\Phi}\trans
  \qquad
  \mat{\Phi}\mat{\Sigma}\mat{\Phi}_\ast\trans
  \qquad
  \mat{\Phi}_\ast\mat{\Sigma}\mat{\Phi}_\ast\trans.
\end{equation*}
The entries of these matrices are always of the form
$\phi(\vec{x})\trans \mat{\Sigma} \phi(\vec{x}')$, where $\vec{x}$ and
$\vec{x}'$ are two arbitrary inputs.  To simply our expressions, we
define a function
\begin{equation*}
  K(\vec{x}, \vec{x}')
  =
  \phi(\vec{x})\trans\mat{\Sigma}\phi(\vec{x}').
\end{equation*}
Because $\mat{\Sigma}$ is positive definite, it has a ``matrix square
root,'' $\mat{\Sigma}^{\nicefrac{1}{2}}$ with the property
$(\mat{\Sigma}^{\nicefrac{1}{2}})^2 = \mat{\Sigma}$.\footnote{You can
  prove this via the singular value decomposition (\acro{SVD}): write
  $\mat{\Sigma} = \mat{U}\mat{D}\mat{U}\trans$, where $\mat{U}$ is
  unitary and $\mat{D}$ is diagonal with positive entries (because
  $\mat{\Sigma}$ is positive definite), then define
  $\mat{\Sigma}^{\nicefrac{1}{2}} =
  \mat{U}\mat{D}^{\nicefrac{1}{2}}\mat{U}\trans$.} If we define the
function $\psi(\vec{x}) =
\mat{\Sigma}^{\nicefrac{1}{2}}\phi(\vec{x})$, we can see that
$K$ is simply an inner product:
\begin{equation*}
  K(\vec{x}, \vec{x}')
  =
  \psi(\vec{x})\trans \psi(\vec{x}').
\end{equation*}
Such a function $K$ is guaranteed to always produce positive-definite
Gram matrices (a \emph{Gram matrix} is a square matrix of inner
products between pairs of elements), and is called a \emph{kernel} or
\emph{covariance function.}

Sometimes it is possible to specify a covariance function $K$ directly
without ever computing the feature map explicitly.  With such a
function, we could perform efficient Bayesian linear regression even
with a high-dimensional (or even infinite dimensional!) feature
expansion $\phi$ \emph{implicitly.} This idea of computing inner
products in a feature space directly is called the \emph{kernel trick}
and has been the basis of a large amount of work in the
machine-learning community.  Effectively, any algorithm that operates
purely in terms of inner products between input vectors can be made
nonlinear by replacing normal inner products with the evaluation of a
kernel.

With this definition, we may rewrite the predictive distribution
for $\vec{y}_\ast$:
\begin{equation*}
  p(\vec{y}_\ast \given \mat{X}_\ast, \data, \sigma^2)
  =
  \mc{N}(
  \vec{y}_\ast;
  \mu_{\vec{y}_\ast \given \data},
  K_{\vec{y}_\ast \given \data}),
\end{equation*}
where
\begin{align*}
  \vec{\mu}_{\vec{y}_\ast\given\data}
  &=
  K(\mat{X}_\ast, \mat{X})
  \bigl(K(\mat{X}, \mat{X}) + \sigma^2 \mat{I}\bigr)\inv
  \vec{y};
  \\
  K_{\vec{y}_\ast\given\data}
  &=
  K(\vec{X}_\ast, \vec{X}_\ast)
  -
  K(\mat{X}_\ast, \mat{X})
  \bigl(K(\mat{X}, \mat{X}) + \sigma^2 \mat{I}\bigr)\inv
  K(\mat{X}, \mat{X}_\ast).
\end{align*}

\subsection*{Examples}

Perhaps the most-commonly used kernel is the \emph{squared exponential
  covariance function:}
\begin{equation*}
  K(\vec{x}, \vec{x}'; \lambda, \ell)
  =
  \lambda^2
  \exp\biggl(-\frac{\lVert \vec{x} - \vec{x}' \rVert^2}{2\ell^2}\biggr),
\end{equation*}
where $\lambda$ and $\ell$ are parameters.  The former is simply a
multiplicative scaling constant (you can think of this as an implicit
scalar multiplication in the implicit feature map $\phi$).  The latter
takes the role of a \emph{length scale;} vectors separated by more
than a couple length scales will have a kernel value near zero.

An example of Bayesian linear regression using this kernel function is
shown in Figure \ref{kernel_example}.  We see that the use of this
kernel function allowed us to achieve nice nonlinear regression
without computing explicit basis expansions.  In fact, you can show
that the squared exponential kernel corresponds to an
\emph{infinite-dimensional} basis expansion, where we use a Gaussian
basis function \emph{centered on every point.}  Such a feature
expansion would be impossible to use if we attempted to use explicit
feature computation.

\begin{figure}
  \centering
  \input{figures/kernel_example.tex}
  \caption{Example of Bayesian linear regression using the squared
    exponential covariance function.  The true function is $f =
    \sin(x)$.  The kernel parameters are $\lambda = \ell = 1$, and the
    noise variance was set to $\sigma^2 = 0.1^2$.}
  \label{kernel_example}
\end{figure}

Sometimes thinking in terms of the kernel can help even when you have
an explicit feature expansion on hand.  As an example, imagine our
inputs are binary vectors of length $n$ (so each input $\vec{x}$ is a
subset, a member of the power set $\mc{P}(n)$).  One rather expensive
feature expansion we could try would be to enumerate every member of
$\mc{P}(n)$ and define $\phi(\vec{x})_i = \vec{s}_i \subset \vec{x}$,
where $\vec{s}_i$ is the $i$th element of the power set.  So we
represent our set $\vec{x}$ by a feature vector of length $2^n$
indicating every subset of $\vec{x}$.  This is a very expensive
feature expansion, requiring exponential space to store for each
input.  However, if we take $\mat{\Sigma} = \mat{I}$, we can compute
the dot product as:
\begin{equation}
  K(\vec{x}, \vec{x}') = 2^{\lvert x \cap x' \rvert},
\end{equation}
which only requires time and space linear in $n$!

\end{document}


\documentclass[10pt]{article}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{mathptmx}
\begin{document}

\title{Supplementary Information for ``A Theoretically-Sufficient And Computationally-Practical Technique For Deterministic Frequency Seriation''}
\author{Carl P. Lipo, Mark E. Madsen, and Robert C. Dunnell}
\date{March 2015}
\maketitle

\section*{S1 Text:  Pseudo-code representation of the IDSS algorithm}

\begin{algorithm}[h]
\caption{Algorithm for IDSS seriation}\label{alg:idss}
\begin{algorithmic}[1]

 \Require  Input file $I$ format:
 \State Tab-delimited text, column 0 contains assemblage name
 \State Remaining columns contain type counts as integers

\Procedure{IDSS}{$I$}

   \State Read input file $I$
   \State Calculate relative frequency of each type
   \State Calculate max frequency difference between assemblage pairs
   \State Create list of assemblages $A$

    \ForAll{triplets of assemblages $T$}
        \If{using continuity threshold $t_c$}
            \If{max frequency difference $> t_c$ for pairs $\in T$}
            \State Skip triplet
            \EndIf
        \EndIf

        \If{triplet $T$ is valid given unimodality for all types}
            \State Store triplet in candidate solutions $\mathbf{C}$
        \EndIf

        \State $R \gets$ assemblages $\notin \mathbf{C}$
        \Comment Remaining assemblages

        \Repeat
            \ForAll{assemblages $ a \in R$}
                \If{using continuity threshold $t_c$}
                    \If{max freq difference $> t_c$ for $a$ and all $\mathbf{C}$}
                        \State Skip assemblage $a$ for this loop
                    \EndIf
                \EndIf

                \If{assemblage $a$ + candidate solution $c \in \mathbf{C}$}
                    \State Replace $c$ in $\mathbf{C}$ with $c+a$
                    \State Remove $a$ from $R$
                    \Comment Grow existing solutions
                \EndIf

            \EndFor

        \Until{ $R = \empty$ or loop repeats with no changes }

            \ForAll{candidate solutions $c \in \mathbf{C}$}
                \If{$c$ is strict subset of another solution in $\mathbf{C}$}
                    \State Remove $c$ from $\mathbf{C}$
                \EndIf
            \EndFor

    \EndFor
    \Comment $\mathbf{C}$ now contains the set of solutions

    \State Output $\mathbf{C}$ in various formats

\EndProcedure
\end{algorithmic}
\end{algorithm}

\end{document}


\chapter{Setting up the linear system}
\label{section:setup-linear-system}
\par
Our typical user is interested in solving $A X = Y$,
where $A$ is square, large and sparse,
and $X$ and $Y$ are dense matrices with one or more columns.
{\bf SPOOLES} is a very large sophisticated library with
a commensurate learning curve to master its functionality.
But what is the bare minimum a user has to know to obtain a
solution to their linear system?
\begin{itemize}
\item
They need to construct an {\tt InpMtx} object that holds
the entries of $A$. ({\tt InpMtx} stands for
{\tt Inp}ut {\tt m}a{\tt t}ri{\tt x},
for it is an easy to use object that one uses to input,
assemble, sort and manipulate entries in a sparse matrix.)
\item
They need to construct a {\tt DenseMtx} object that holds
the entries of $Y$.
\item
They need to construct a {\tt DenseMtx} object to hold
the entries of $X$.
\end{itemize}
These two objects encapsulate the minimal interface to the
{\bf SPOOLES} library.
the application program needs to know how to construct
the {\tt InpMtx} and {\tt DenseMtx} objects, either directly inside
an application program, or by reading in a custom matrix file.
This is what we now describe.
\par
\section{Constructing an {\tt InpMtx} object}
\label{subsection:construct-InpMtx}
\par
The {\tt InpMtx} object is more of an ``Input'' object
than a ``Matrix'' object.
It descended from an out-of-core assembly code that assembled and
sorted entries of a sparse matrix.
Simplicity and functionality are its goals, at some expense of
efficiency in storage and computation.
{\it Note: all indices are zero-based as in C, not 1-based as in
FORTRAN.}
\par
The {\tt InpMtx} object is simplest understood as a ``bag'' of
triples $\langle r(i,j),c(i,j),a_{i,j}\rangle$,
where $r()$ and $c()$ are some
functions that define the first and second coordinates.
Each {\tt InpMtx} object has a ``coordinate type'', one of
\begin{itemize}
\item
{\tt INPMTX\_BY\_ROWS}, where $r(i,j) = i$, $c(i,j) = j$.
\item
{\tt INPMTX\_BY\_COLUMNS}, where $r(i,j) = j$, $c(i,j) = i$.
\item
{\tt INPMTX\_BY\_CHEVRONS},
where $r(i,j) = \min(i,j)$, $c(i,j) = j - i$.
\end{itemize}
Rows and columns are self-explanatory, the first coordinate
$r(i,j)$ is either the row or column of $a_{i,j}$.
The $j$-th ``chevron'' is composed of the diagonal entry $a_{j,j}$,
entries in the $j$-th row of the upper triangle,
and entries in the $j$-th column of the lower triangle.
It is the natural data structure for the assembly of the matrix
entries into the ``fronts'' used to factor the matrix.
\par
% ``Entries'' of the {\tt InpMtx} object can be one of three types.
The {\tt InpMtx} object can hold one of three types of entries as
``indices only'' (no entries are present),
real entries, or complex entries.
The type is specified by the {\tt inputMode}
parameter to the {\tt InpMtx\_init()} method.
\begin{itemize}
\item
{\tt INPMTX\_INDICES\_ONLY} where the triples
$langle r(i,j),c(i,j),-\rangle$ are really only pairs,
i.e., no numerical values are present.
This mode is useful for assembling graphs.
\item
{\tt SPOOLES\_REAL} where $a_{i,j}$ is a real number,
a {\tt double} value.
\item
{\tt SPOOLES\_COMPLEX} where $a_{i,j}$ is a complex number,
really two consecutive {\tt double} values.
\end{itemize}
``Coodinate type'' and ``input mode'' (equivalently, the type of
entries) are the two parameters that must be specified when
initializing an {\tt InpMtx} object.
\begin{verbatim}
InpMtx   *mtxA = InpMtx_new() ;
InpMtx_init(mtxA, coordType, inputMode, 0, 0) ;
\end{verbatim}
Every object in the {\bf SPOOLES} library is initialized
via an {\tt {\it ObjectName}\_new()} method, which allocates space
for the object and sets its fields to default values.
If you wish to use an {\it automatic} variable, then one must
explicitly set the default fields, as follows.
\begin{verbatim}
InpMtx   mtxA ;
InpMtx_setDefaultFields(&mtxA) ;
InpMtx_init(&mtxA, coordType, inputMode, 0, 0) ;
\end{verbatim}
Only the coordinate type and input mode are necessary.
The fourth and fifth arguments are upper bounds on the number of
entries and vectors for the object. (More on vectors in just a
moment.) The user does not need to know values for the number of
entries or vectors, for the object resizes itself as necessary
as information is placed into it.
\par
``Vectors'' is one way that the entries can be stored.
There are actually three ways, specified by the
{\tt storageMode} field of the {\tt InpMtx} object.
\begin{itemize}
\item
{\tt INPMTX\_RAW\_DATA}, where the pairs or triples are stored in
unordered form.
\item
{\tt INPMTX\_SORTED}, where the pairs or triples are stored in
ascending lexicographic order of the first two coordinates.
\item
{\tt INPMTX\_BY\_VECTORS}, where the pairs or triples are sorted
and stored in vectors defined by their first coordinate.
\end{itemize}
The storage mode can be changed via a call to
{\tt InpMtx\_changeStorageMode()}.
\par
The user does not really need to know about this
``storage mode''. Fill the {\tt InpMtx} object with data in any way
at all (we will describe this shortly).
The wrapper method will check that the data is in the form it needs.
If is isn't, the object will be transformed as necessary.
The ``sort'' operation is really ``sort-and-compress'', the pairs
or triples are sorted into ascending order, and then the list is
scanned duplicates are ``merged'' together, i.e., if real or
complex entries are present, they are added together.
(This allows us to assemble finite element matrices.)
The knowledgeable user can change the storage mode as necessary,
and thus avoiding expensive sorts when possible.
For example, after reading in the matrix data from the CSAR-Nastran
file,
the entries are already in sorted form, and the explicit sort
can be avoided.
\par
Now let us see how we ``input'' information into the {\tt InpMtx}
object.
There are several input methods, e.g., single entries, rows,
columns, and submatrices, and each input method has three types of
input, e.g, indices only, real entries, or complex entries.
Here are the prototypes below.
\begin{itemize}
\item Input methods for ``indices only'' mode.
\begin{verbatim}
void InpMtx_inputEntry ( InpMtx *mtxA, int row, int col ) ;
void InpMtx_inputRow ( InpMtx *mtxA, int row, int rowsize, int rowind[] ) ;
void InpMtx_inputColumn ( InpMtx *mtxA, int col, int colsize, int colind[] ) ;
void InpMtx_inputMatrix ( InpMtx *mtxA, int nrow, int ncol, int rowstride,
                          int colstride, int rowind[], colind[] ) ;
\end{verbatim}
\item Input methods for real entries.
\begin{verbatim}
void InpMtx_inputRealEntry ( InpMtx *mtxA, int row, int col, double value ) ;
void InpMtx_inputRealRow ( InpMtx *mtxA, int row, int rowsize,
                           int rowind[], double rowent[] ) ;
void InpMtx_inputRealColumn ( InpMtx *mtxA, int col, int colsize,
                              int colind[], double colent[] ) ;
void InpMtx_inputRealMatrix ( InpMtx *mtxA, int nrow, int ncol, int rowstride,
                    int colstride, int rowind[], colind[], double mtxent[] ) ;
\end{verbatim}
\item Input methods for complex entries.
\begin{verbatim}
void InpMtx_inputComplexEntry ( InpMtx *mtxA, int row, int col,
                                double real, double imag ) ;
void InpMtx_inputComplexRow ( InpMtx *mtxA, int row, int rowsize,
                               int rowind[], double rowent[] ) ;
void InpMtx_inputComplexColumn ( InpMtx *mtxA, int col, int colsize,
                                 int colind[], double colent[] ) ;
void InpMtx_inputComplexMatrix ( InpMtx *mtxA, int nrow, int ncol, int rowstride,
              int colstride, int rowind[], colind[], double mtxent[] ) ;
\end{verbatim}
\end{itemize}
The {\tt rowind[]} row indices and {\tt colind[]} column indices
are precisely that.
Don't worry about what coordinate type the {\tt InpMtx} object has,
the translation from row and column indices into the particular
coordinate is done inside the input methods.
\par
Let us look at a particular example, where we have a
${\tt n1} \times {\tt n2}$ grid and we want to have a
$\left \lbrack \begin{array}{ccc}
   & -1 &    \\
-1 &  4 & -1 \\
   & -1 &
\end{array} \right \rbrack$
5-point operator at each grid point.
Note, this matrix is symmetric, so we need input only the upper
triangle (or the lower triangle) of the matrix.
\begin{verbatim}
mtxA = InpMtx_new() ;
InpMtx_init(mtxA, INPMTX_BY_ROWS, SPOOLES_REAL, 0, 0) ;
for ( ii = 0 ; ii < n1 ; ii++ ) {
   for ( jj = 0 ; jj < n2 ; jj++ ) {
      ij = ii + jj*n1 ;
      indices[0] = ij ;
      entries[0] = 4.0 ;
      count = 1 ;
      if ( ii < n1 ) {
         indices[count] = ij + 1 ;
         entries[count] = -1.0 ;
         count++ ;
      }
      if ( jj < n2 ) {
         indices[count] = ij + n1 ;
         entries[count] = -1.0 ;
         count++ ;
      }
      InpMtx_inputRealRow(mtxA, ij, count, indices, entries) ;
   }
}
InpMtx_changeStorageMode(mtxA, INPMTX_BY_VECTORS) ;
\end{verbatim}
The process begins by allocating an {\tt InpMtx} object {\tt mtxA}
using the {\tt InpMtx\_new()} method,
initializing it with the {\tt InpMtx\_init()} method,
and filling it with matrix entries with the {\tt
InpMtx\_inputRealRow()} method.
The last method, {\tt InpMtx\_changeStorageMode()},
``assembles'' the data (not really necessary because the entries
are disjoint,
``sorts'' the data (again not necessary since the entries were
input in ascending order,
and creates a vector structure inside the {\tt InpMtx} object that
allows easy access to each individual row.
\par
We could have input all the entries and treated it as a
nonsymmetric matrix, but that would not be efficient with respect
to storage or factorization cost.
Alternatively, we could have input all the entries and called the
{\tt InpMtx\_dropLowerTriangle()} method to drop the lower
triangular entries.
\par
\section{Constructing an {\tt DenseMtx} object}
\label{subsection:construct-DenseMtx}
\par
The {\tt DenseMtx} stores a real or complex dense matrix.
It is not just an array of numbers, it also has row indices and
column indices.
This allows it to exist in a distributed MPI environment where each
processors has only a submatrix of the matrix.
Here is how to initialize a {\tt DenseMtx} object.
\begin{verbatim}
int        type, rowid, colid, nrow, ncol, inc1, inc2 ;
DenseMtx   *mtx = DenseMtx_new() ;
DenseMtx_init(mtx, type, rowid, colid, nrow, ncol, inc1, inc2) ;
\end{verbatim}
\begin{itemize}
\item
The {\tt type} is either {\tt SPOOLES\_REAL}
or {\tt SPOOLES\_COMPLEX}.
\item
The {\tt rowid} and {\tt colid} values are used to identify a
{\tt DenseMtx} as a submatrix of a larger matrix.
Any values are suitable.
\item
{\tt nrow} and {\tt ncol} are the number of rows and columns in the
matrix, respectively.
\item
The entries of the matrix can be stored in either row major or
column major form.
For row major, use {\tt inc1 = ncol} and {\tt inc2 = 1}.
For column major, use {\tt inc1 = 1} and {\tt inc2 = nrow}.
Note, all solve and matrix-matrix multiply methods require that the
{\tt DenseMtx} object be column major.
\end{itemize}
For example, here is the call to initialize a {\tt DenseMtx} object
to have real entries, 100 rows and 5 columns, entries column major.
\begin{verbatim}
DenseMtx_init(mtx, SPOOLES_REAL, 0, 0, 100, 5, 1, 100) ;
\end{verbatim}
During the initialization,
the row indices are set to $0, 1, \dots, \mathtt{nrow - 1}$
and the column indices are set to $0, 1, \dots, \mathtt{ncol - 1}$.
The entries are {\bf not} initialized.
Zero the entries with a call to {\tt DenseMtx\_zero()}.
(This is crucial when loading a sparse right hand side into
the {\tt DenseMtx} object.)
\par
Once we have the {\tt DenseMtx} object initialized, we want to be
able to access the row indices, the column indices and the entries.
We do this through instance methods.
\begin{verbatim}
void DenseMtx_rowIndices ( DenseMtx *mtx, int *pnrow, int *prowind ) ;
void DenseMtx_columnIndices ( DenseMtx *mtx, int *pncol, int *pcolind ) ;
double * DenseMtx_entries ( DenseMtx *mtx ) ;
\end{verbatim}
We would use them as follows.
\begin{verbatim}
double   *entries ;
int      ncol, nrow, *colind, *rowind ;

DenseMtx_rowIndices(mtx, &nrow, &rowind) ;
DenseMtx_columnIndices(mtx, &ncol, &colind) ;
entries = DenseMtx_entries(mtx) ;
\end{verbatim}
We can now fill the indices or the entries.
The location of the {\tt (irow,jcol)} entry is found
at {\tt offset = irow*inc1 + jcol*inc2}.
The row and column increments can be found as follows.
\begin{verbatim}
int inc1 = DenseMtx_rowIncrement(mtx) ;
int inc2 = DenseMtx_columnIncrement(mtx) ;
\end{verbatim}
\par
To avoid dealing with row and column increments,
we can retrieve and set values of a particular entry.
\begin{verbatim}
double   value, real, imag ;
int      irow, jcol ;

DenseMtx_realEntry(mtx, irow, jcol, &value) ;
DenseMtx_complexEntry(mtx, irow, jcol, &real, &imag) ;
DenseMtx_setRealEntry(mtx, irow, jcol, value + 10.) ;
DenseMtx_setComplexEntry(mtx, irow, jcol, real + 1., imag + 2.) ;
\end{verbatim}
As a real example, consider the ${\tt n1} \times {\tt n2}$ grid
from the previous subsection, where we assembled a finite
difference matrix.
Assume that the right hand side is zero except for points where
{\tt (n1-1,0:n2-1)}, where a unit load is applied.
Here is the code to generate the {\tt DenseMtx} object.
\begin{verbatim}
mtxY = DenseMtx_new();
DenseMtx_init(mtxY, SPOOLES_REAL, 0, 0, n1*n2, 1, 1, n1*n2) ;
DenseMtx_zero(mtxY) ;
ii = n1 - 1 ;
for ( jj = 0 ; jj < n2 ; jj++ ) {
   ij = ii + jj*n1 ;
   DenseMtx_setRealEntry(mtxY, ij, 1, 1.0) ;
}
\end{verbatim}
Do not forget to zero the entries in {\tt mtxY} before setting any
entries.
\par
\section{IO for the {\tt InpMtx} and {\tt DenseMtx} objects}
\label{subsection:IO}
\par
The three driver programs that we describe in the next sections
read $A$ and $Y$ from files and write $X$ to a file.
So the first thing we know is that the {\tt InpMtx} and {\tt
DenseMtx} objects can read and write themselves from and to files.
This convention is supported by most of the objects in the
{\bf SPOOLES} library.
In fact, there is a common {\it protocol} that is followed.
Let us take a look at the common IO methods for the {\tt InpMtx}.
\begin{itemize}
\item
{\tt int InpMtx\_readFromFile ( InpMtx *obj, char *filename ) ;}
\item
{\tt int InpMtx\_readFromFormattedFile ( InpMtx *obj, FILE *fp ) ;}
\item
{\tt int InpMtx\_readFromBinaryFile ( InpMtx *obj, FILE *fp ) ;}
\item
{\tt int InpMtx\_writeToFile ( InpMtx *obj, char *filename ) ;}
\item
{\tt int InpMtx\_writeToFormattedFile ( InpMtx *obj, FILE *fp ) ;}
\item
{\tt int InpMtx\_writeToBinaryFile ( InpMtx *obj, FILE *fp ) ;}
\item
{\tt int InpMtx\_writeForHumanEye ( InpMtx *obj, FILE *fp ) ;}
\end{itemize}
There are corresponding methods for the {\tt DenseMtx} object,
just replace ``{\tt Inp}'' by ``{\tt Dense}'' in the above prototypes.
\par
Two methods take as input {\tt char *} file names. Each object can
be archived in its own file with a particular suffix.
For example, {\tt InpMtx} objects can be read from and written to
files of the form {\tt *.inpmtxf} for a formatted file and
{\tt *.inpmtxb} for a binary file.
For a {\tt DenseMtx} object, the file names are
{\tt *.densemtxf} and {\tt *.densemtxb}.
The {\tt InpMtx\_readFromFile()} method looks at the {\tt filename}
argument, and calls the binary or formatted read methods, depending on
the suffix of {\tt filename}.
A normal return code is {\tt 1}.
If the suffix does not match either {\tt *.inpmtxf} or {\tt *.inpmtxb},
an error message is printed and the return code is {\tt 0}.
Something similar works for writing an {\tt InpMtx} object to a
file using {\tt InpMtx\_writeToFile()}, except if {\tt filename}'s
suffix does not match, the {\tt InpMtx\_writeForHumanEye()} method
is called.
\par
Here are three approaches to link $A$ and $Y$ from an application
code to the {\tt InpMtx} and {\tt DenseMtx} objects demanded by the
{\bf SPOOLES} application.
\begin{itemize}
\item
An application could take the simple approach of creating an {\tt
InpMtx} and {\tt DenseMtx} object to hold $A$ and $Y$, write them
to a file, and then call a totally separate code that functions
much like our drivers, reading in $A$ and $Y$, computing $X$ and
writing $X$ to a file, which is then read in by the application code.
\item
A second approach, one that was taken during the first integration
of the {\bf SPOOLES} library into CSAR-Nastran, was to have the
CSAR-Nastran code generate two files for $A$ and $Y$ in CSAR-Nastran
format.
(This way CSAR-Nastran did not need to know any of the {\bf SPOOLES}
interface.)
Two custom routines were written to read in the entries of $A$ and
$Y$ from the CSAR-Nastran files and construct {\tt InpMtx} and {\tt
DenseMtx} objects.
The wrapper routines we describe in the next three chapters were
called to solve for $X$ which was then written to a CSAR-Nastran file.
\item
A third approach would be to generate the {\tt InpMtx} and {\tt
DenseMtx} objects in the application program, and then call the
wrapper methods to solve for $X$, i.e., no IO would be necessary.
\end{itemize}


\section{Homogenization and the Two-Scale Approach}
\label{sec:61homogenization}

\minitoc{70mm}{4}

\noindent
We roughly follow the presentation given in
\multicite{%
  Huebner14Mehrdimensionale,%
  Valentin14Hierarchische,%
  Valentin16Hierarchical%
}.
The necessary notation is summarized in
\cref{tbl:glossaryTopologyOptimization}.

\begin{table}
  \setnumberoftableheaderrows{0}%
  \newcommand*{\pnst}[1]{\printnotationsymbol{#1}&\printnotationtext{#1}}%
  \begin{tabular}{%
    >{\kern\tabcolsep}=l<{\kern-1.5mm}+l<{\kern2.9mm}+l<{\kern-1.5mm}+l%
    <{\kern2.9mm}+l<{\kern-1.25mm}+l<{\kern\tabcolsep}%
  }
    \toprulec
    \pnst{\objdomain}&       \pnst{\force}&        \pnst{\densglobal}\\
    \pnst{\dimobjdomain}&    \pnst{\displacement}& \pnst{\denscell}\\
    $d$&\#micro-cell param.& \pnst{\compliance}&   \pnst{\densub}\\
    \pnst{\mcp}&             \pnst{\vol}&          \pnst{\etensor}\\
    &&                       \pnst{\voldens}&      \pnst{\cholfactor}\\
    \bottomrulec
  \end{tabular}%
  \caption[Glossary for topology optimization]{%
    Glossary of the notation for topology optimization.%
  }%
  \label{tbl:glossaryTopologyOptimization}%
\end{table}

\subsection{Homogenization}
\label{sec:611homogenization}

\paragraph{Density function}

Let $\objdomain \subset \real^{\dimobjdomain}$ be the object domain.%
\footnote{%
  We use tildes to denote variables and quantities
  that correspond to the object domain $\objdomain$
  (e.g., $\tilde{\*x}$ is a point in $\objdomain$).
  In contrast, variables without a tilde will correspond
  to the sparse grid domain $\clint{\*0, \*1} = \clint{0, 1}^d$
  (e.g., $\gp{\*l,\*i} \in \clint{\*0, \*1}$ will be a sparse grid point).%
}
Usually, we assume $\dimobjdomain = 2$ or $\dimobjdomain = 3$,
although the method can be generalized to
arbitrary dimensionalities $\dimobjdomain \in \nat$.
Shapes and topologies are described by \term{density functions}
$\densglobal\colon \objdomain \to \clint{0, 1}$.
The function values $\densglobal(\tilde{\*x}) \in \clint{0, 1}$
tell if $\tilde{\*x}$ is contained in the object (value of one) or
not (value of zero).
The \term{homogenization} approach also allows values between
zero and one, giving the physical density of the material in $\tilde{\*x}$.

\paragraph{Optimization of compliance values}

Furthermore, for every density function $\densglobal$,
let $\compliance(\densglobal)$ be an objective function value.
In our setting, which is shown in \cref{fig:topoOptExample},
we exert a force $\force$ on the object,
measure the resulting deformation, and
compute the \term{compliance} (i.e., the inverse of the stiffness) as
the objective function value~$\compliance(\densglobal)$:
\begin{equation}
  \compliance(\densglobal)
  = \int_{\objdomain} \tr{\force} \displacement_{\densglobal}(\tilde{\*x})
  \diff\tilde{\*x},
\end{equation}
where the \term{displacement function}
$\displacement_{\densglobal}\colon \objdomain \to \real^{\dimobjdomain}$
depends on the density \cite{Huebner14Mehrdimensionale}.
We want to find the density function
with the minimal compliance value:
\begin{equation}
  \label{eq:topoOptProblemContinuous}
  \min_{\densglobal}\, \compliance(\densglobal).
\end{equation}
If we do not impose additional conditions,
then there are often uninteresting trivial solutions.
For example, choosing $\densglobal :\equiv 1$
(i.e., filling the entire domain $\objdomain$ with material)
usually leads to the topology with the
highest stiffness and, thus, the smallest displacement and compliance value.
Therefore, we introduce the following volume constraint:
\begin{equation}
  \frac{\voldens{\densglobal}{\objdomain}}{\vol{\objdomain}} \le \densub,\quad
  \voldens{\densglobal}{\objdomain}
  \ceq \int_{\objdomain} \densglobal(\tilde{\*x}) \diff\tilde{\*x},\quad
  \vol{\objdomain}
  \ceq \voldens{1}{\objdomain},
\end{equation}
where $\vol{\objdomain} = \int_{\objdomain} 1 \diff\tilde{\*x}$
is the volume of the object domain and
$\densub \in \clint{0, 1}$ is an upper bound on the volume fraction.

\begin{SCfigure}
  \includegraphics{topoOptExample_1}%
  \caption[%
    Example scenario for topology optimization%
  ]{%
    Example scenario for topology optimization.
    An object \emph{\textcolor{hellblau}{(light blue)}}
    is fixed on the left side
    of the object domain $\objdomain$
    \emph{\textcolor{mittelblau!50}{(darker blue)}}
    and deformed by a force $\force$, resulting in a displaced object
    \emph{(dashed).}
    The density function $\densglobal(\tilde{\*x})$ is one inside the object
    and zero outside.%
  }%
  \label{fig:topoOptExample}%
\end{SCfigure}

\subsection{Two-Scale Approach}
\label{sec:612twoScale}

\paragraph{Discretization and two-scale approach}

Of course, we cannot solve the problem \eqref{eq:topoOptProblemContinuous}
numerically,
as there are infinitely many density functions $\densglobal$.
For simplicity, we assume that $\objdomain$ is some hyper-rectangle
$\clint{\tilde{\*a}, \tilde{\*b}}
= \clint{\tilde{a}_1, \tilde{b}_1} \times \dotsb \times
\clint{\tilde{a}_{\dimobjdomain}, \tilde{b}_{\dimobjdomain}}$;
if it is not, we replace $\objdomain$ with its bounding box.
The object domain $\objdomain$ can then be split into
$M_1 \times \dotsb \times M_{\dimobjdomain}$
equally-sized and axis-aligned sub-hyper-rectangles,
which we call \term{macro-cells}
(where $M_1, \dotsc, M_{\dimobjdomain} \in \nat$).

In the \term{two-scale approach,}
we assume the material of the macro-cells to be
repetitions of infinitesimally small periodic structures
(i.e., identical for each macro-cell),
called \term{micro-cells.}
These micro-cells have a specific shape, which is parameterized by
$d$ \term{micro-cell parameters} $x_1, \dotsc, x_d$,
normalized to values in the unit interval $\clint{0, 1}$.
For instance, in two dimensions,
this shape may be an axis-aligned cross
with thicknesses $x_1$ and $x_2$, as shown in \cref{fig:twoScale}.
The choice of a suitable \term{micro-cell model}
(parametrization of the micro-cells)
depends on the optimization scenario and has to be done a priori.

\begin{figure}
  \includegraphics{twoScale_1}%
  \caption[%
    Two-scale approach for topology optimization%
  ]{%
    Two-scale approach to discretize the homogenized topology
    optimization problem in two dimensions ($\dimobjdomain = 2$).
    \emph{Left:} The object domain $\objdomain$ is
    subdivided into $M_1 \times M_2$ macro-cells,
    each with its own density \emph{(gray squares).}
    \emph{Center:} Every macro-cell is the repetition of infinitesimally small
    periodic micro-cells.
    \emph{Right:} The shape of the structure in every micro-cell is
    described by a micro-cell model with $d$ parameters $x_1, \dotsc, x_d$.
    Here, the micro-cell model is a cross with two parameters
    that represent the thickness of each crossbar.%
  }%
  \label{fig:twoScale}%
\end{figure}

\paragraph{Elasticity tensors}

Note that while the shape of all micro-cells in one macro-cell is identical,
the micro-cell parameters corresponding to different macro-cells differ
in general.
This enables varying densities in different regions of $\objdomain$.
We denote the micro-cell parameters corresponding to the $q$-th macro-cell
with $\mcp{q} = (\mcpentry{1}{q}, \dotsc, \mcpentry{d}{q}) \in
\clint{\*0, \*1} = \clint{0, 1}^d$,
where $q = 1, \dotsc, M$ and
$M \ceq M_1 \dotsm M_{\dimobjdomain}$ is the number of macro-cells.
With linear elasticity,
one can compute so-called \term{elasticity tensors} $\etensor^{(q)}$,
which encode information about the material properties
of the different macro-cells.
The elasticity tensors can be written as symmetric matrices
in $\real^{3 \times 3}$ (for $\dimobjdomain = 2$) or
in $\real^{6 \times 6}$ (for $\dimobjdomain = 3$).%
\footnote{%
  In general, the elasticity tensor is a fourth-order tensor in
  $\real^{\dimobjdomain \times \dimobjdomain \times \dimobjdomain \times \dimobjdomain}$.
  One can reduce the size of the tensor by exploiting various symmetries
  \cite{Huebner14Mehrdimensionale}
  to obtain $6$ or $21$ stiffness coefficients
  in two or three dimensions, respectively.
  These coefficients can then be expressed as a symmetric matrix.%
}
To simplify the following considerations,
we assume that $\dimobjdomain = 3$, i.e.,
$\etensor^{(q)} \in \real^{6 \times 6}$.
The elasticity tensors are usually computed as the solution of a \fem problem
\term{(micro-problem).}
Once all $\etensor^{(q)}$ are known,
we can compute the compliance value
by solving another \fem problem \term{(macro-problem),}
see \cite{Allaire04Topology} and \cite{Huebner14Mehrdimensionale}.

\vspace{-0.5em}

\paragraph{Discretized optimization problem}

The new optimization problem emerging from the
two-scale discretization process has the form
\begin{subequations}
  \label{eq:topoOptProblemDiscrete}
  \setlength{\belowdisplayskip}{5pt}%
  \begin{gather}
    \min J(\mcp{1}, \dotsc, \mcp{M}),\quad
    \mcp{1}, \dotsc, \mcp{M} \in \clint{\*0, \*1}
    \quad\text{s.t.}\quad
    \densmean(\mcp{1}, \dotsc, \mcp{M}) \le \densub,\\
    \densmean(\mcp{1}, \dotsc, \mcp{M})
    \ceq \frac{1}{M} \sum_{q=1}^M \denscell^{(q)}(\mcp{q}).
  \end{gather}
\end{subequations}
Here, $\denscell^{(q)}(\mcp{q}) \in \clint{0, 1}$ is the
density of the $q$-th macro-cell with micro-cell parameter $\mcp{q}$
(i.e., the fraction of material volume of one micro-cell
with respect to its total volume)
and $\densmean(\mcp{1}, \dotsc, \mcp{M}) \in \clint{0, 1}$
is the resulting total mean density.
This discretized optimization problem can now be implemented and
solved numerically.


%
% personal commentary:
%        DRAFT DRAFT DRAFT
%        - KFALL
% RNG section reviewed and revised 11-Sep-98, johnh.
%
% RNG section revised 28-Aug-02 by Tim Buchheim to include material
% from Michelle Weigle which describes the new implementation of RNG

\chapter{Mathematical Support}
\label{chap:math}

The simulator includes a small collection of mathematical
functions used to implement random variate generation and integration.
This area of the simulator is currently undergoing some
changes.

The procedures and functions described in this chapter can be found in
\nsf{tools/rng.\{cc, h\}},
\nsf{tools/random.\{cc, h\}},
\nsf{tools/ranvar.\{cc, h\}},
\nsf{tools/pareto.\{cc, h\}},
\nsf{tools/expoo.\{cc, h\}},
\nsf{tools/integrator.\{cc, h\}}, and
\nsf{tcl/lib/ns-random.tcl}.

\section{Random Number Generation}
\label{sec:random}

The RNG class contains an implementation of the combined multiple
recursive generator MRG32k3a proposed by L'Ecuyer
\cite{lecuyer99}. The C++ code was adapted from \cite{lecuyer01}.
This replaces the previous implementation of \code{RNG}, which used
the minimal standard multiplicative linear congruential generator of
Park and Miller~\cite{Park88:Random}.  The newer (MRG32k3a) \code{RNG} is
used in ns versions 2.1b9 and later.

The MRG32k3a generator provides $1.8$x$10^{19}$ independent
streams of random numbers, each of which consists of
$2.3$x$10^{15}$ substreams. Each substream has a period
(\emph{i.e.}, the number of random numbers before overlap) of
$7.6$x$10^{22}$. The period of the entire generator is
$3.1$x$10^{57}$. Figure \ref{streams} provides a graphical idea of
how the streams and substreams fit together.
\begin{figure}[ht]
\centering
\includegraphics[angle=270,width=6 in]{rng-streams.eps}
\caption{Overall arrangement of streams and substreams.
\cite{lecuyer01}} \label{streams}
\end{figure}

A default RNG ({\tt defaultRNG}), created at simulator initialization
time, is provided. If multiple random variables are used in a
simulation, each random variable should use a separate RNG object.
When a new RNG object is created, it is automatically seeded to
the beginning of the next independent stream of random numbers.
Used in this manner, the implementation allows for a maximum of
$1.8$x$10^{19}$ random variables.

Often, multiple independent replications of a simulation are
needed (\emph{i.e.}, to perform statistical analysis given
multiple runs with fixed parameters).  For each replication, a
different substream should be used to ensure that the random
number streams are independent. (This process is given as an OTcl
example later.) This implementation allows for a maximum of
$2.3$x$10^{15}$ independent replications. Each random variable in
a single replication can produce up to $7.6$x$10^{22}$ random
numbers before overlapping.

{\bf Note:} Only the most common functions are described here. For
more information, see \cite{lecuyer01} and the source code found
in {\tt tools/rng.h} and {\tt tools/rng.cc}.  For a comparison of this RNG to
the more common LCG16807 RNG (and why LCG16807 is not a good RNG),
see \cite{lecuyer-wsc}.
\subsection{Seeding The RNG}
Due to the nature of the RNG and its implementation, it is not
necessary to set a seed (the default is 12345).  If you wish to
change the seed, functions are available. You should only set the
seed of the default RNG.  Any other RNGs you create are
automatically seeded such that they produce independent streams.
The range of valid seeds is 1 to {\tt MAXINT}.

To get non-deterministic behavior, set the seed of the default RNG
to 0.  This will set the seed based on the current time of day and
a counter.  {\bf This method should not be used to set seeds for
independent replications.}  There is no guarantee that the streams
produced by two random seeds will not overlap.  The only way to
guarantee that two streams do not overlap is to use the substream
capability provided by the RNG implementation.

\subsubsection{Example}
\begin{verbatim}
 # Usage: ns rng-test.tcl [replication number]

 if {$argc > 1} {
    puts "Usage: ns rng-test.tcl \[replication number\]"
    exit
 }
 set run 1
 if {$argc == 1} {
    set run [lindex $argv 0]
 }
 if {$run < 1} {
    set run 1
 }

 # seed the default RNG
 global defaultRNG
 $defaultRNG seed 9999

 # create the RNGs and set them to the correct substream
 set arrivalRNG [new RNG]
 set sizeRNG [new RNG]
 for {set j 1} {$j < $run} {incr j} {
    $arrivalRNG next-substream
    $sizeRNG next-substream
 }

 # arrival_ is a exponential random variable describing the time
 # between consecutive packet arrivals
 set arrival_ [new RandomVariable/Exponential]
 $arrival_ set avg_ 5
 $arrival_ use-rng $arrivalRNG

 # size_ is a uniform random variable describing packet sizes
 set size_ [new RandomVariable/Uniform]
 $size_ set min_ 100
 $size_ set max_ 5000
 $size_ use-rng $sizeRNG

 # print the first 5 arrival times and sizes
 for {set j 0} {$j < 5} {incr j} {
    puts [format "%-8.3f  %-4d" [$arrival_ value] \
            [expr round([$size_ value])]]
 }
\end{verbatim}

\subsubsection{Output}

\begin{verbatim}
 % ns rng-test.tcl 1
 6.358     4783
 5.828     1732
 1.469     2188
 0.732     3076
 4.002     626

 % ns rng-test.tcl 5
 0.691     1187
 0.204     4924
 8.849     857
 2.111     4505
 3.200     1143
\end{verbatim}

\subsection{OTcl Support}

\subsubsection{Commands}

The following commands on the RNG class can be accessed from OTcl
and are found in {\tt tools/rng.cc}:
\begin{description}
    \item[{\tt seed $n$}] -- seed the RNG to $n$, if $n == 0$, the seed
    is set according to the current time and a counter
    \item[{\tt next-random}] -- return the next random number
    \item[{\tt seed}] -- return the current value of the seed
    \item[{\tt next-substream}] -- advance to the next substream
    \item[{\tt reset-start-substream}] -- reset the stream to the beginning
    of the current substream
    \item[{\tt normal $avg$ $std$}] -- return a number sampled from a normal
    distribution with the given average and standard deviation
    \item[{\tt lognormal $avg$ $std$}] -- return a number sampled from a
      lognormal distribution with the given average and standard deviation
\end{description}

The following commands on the RNG class can be accessed from OTcl
and are found in {\tt tcl/lib/ns-random.tcl}:
\begin{description}
    \item[{\tt exponential $mu$}] -- return a number sampled from an
      exponential distribution with mean $mu$
    \item[{\tt uniform $min$ $max$}] -- return an integer sampled from a
    uniform distribution on [$min$, $max$]
    \item[{\tt integer $k$}] -- return an integer sampled from a uniform
    distribution on [0, $k$-1]
\end{description}

\subsubsection{Example}

\begin{verbatim}
 # Usage: ns rng-test2.tcl [replication number]

 if {$argc > 1} {
    puts "Usage: ns rng-test2.tcl \[replication number\]"
    exit
 }
 set run 1
 if {$argc == 1} {
    set run [lindex $argv 0]
 }
 if {$run < 1} {
    set run 1
 }

 # the default RNG is seeded with 12345

 # create the RNGs and set them to the correct substream
 set arrivalRNG [new RNG]
 set sizeRNG [new RNG]
 for {set j 1} {$j < $run} {incr j} {
    $arrivalRNG next-substream
    $sizeRNG next-substream
 }

 # print the first 5 arrival times and sizes
 for {set j 0} {$j < 5} {incr j} {
    puts [format "%-8.3f  %-4d" [$arrivalRNG lognormal 5 0.1] \
            [expr round([$sizeRNG normal 5000 100])]]
 }
\end{verbatim}

\subsubsection{Output}

\begin{verbatim}
 % ns rng-test2.tcl 1
 142.776   5038
 174.365   5024
 147.160   4984
 169.693   4981
 187.972   4982

 % ns rng-test2.tcl 5
 160.993   4907
 119.895   4956
 149.468   5131
 137.678   4985
 158.936   4871
\end{verbatim}

\subsection{C++ Support}

\subsubsection{Member Functions}
The random number generator is implemented by the RNG class and is
defined in {\tt tools/rng.h}.

{\bf Note:} The Random class in {\tt tools/random.h} is an older
interface to the standard random number stream.

Member functions provide the following operations:

\begin{description}
    \item[{\tt void set\_seed (long seed)}] -- set the seed of the RNG, if
    $seed == 0$, the seed is set according to the current time and a
    counter
    \item[{\tt long seed (void)}] -- return the current seed
    \item[{\tt long next (void)}] -- return the next random number as an
    integer on [0, {\tt MAXINT}]
    \item[{\tt double next\_double (void)}] -- return the next random number
    on [0, 1]
    \item[{\tt void reset\_start\_substream (void)}] -- reset the stream
    to the beginning of the current substream
    \item[{\tt void reset\_next\_substream (void)}] -- advance to the next
    substream
    \item[{\tt int uniform (int k)}] -- return an integer sampled from a
      uniform distribution on [0, k-1]
    \item[{\tt double uniform (double r)}] -- return a number sampled from a
      uniform distribution on [0, r]
    \item[{\tt double uniform (double a, double b)}] -- return a number
      sampled from a uniform distribution on [a, b]
    \item[{\tt double exponential (void)}] -- return a number sampled from an
    exponential distribution with mean 1.0
    \item[{\tt double exponential (double k)}] -- return a number sampled from
      an exponential distribution with mean k
    \item[{\tt double normal (double avg, double std)}] -- return a number
      sampled from a normal distribution with the given average and standard
    deviation
    \item[{\tt double lognormal (double avg, double std)}] -- return a number
      sampled from a lognormal distribution with the given average and
      standard deviation
\end{description}

\subsubsection{Example}

\begin{verbatim}
 /* create new RNGs */
 RNG arrival (23456);
 RNG size;

 /* set the RNGs to the appropriate substream */
 for (int i = 1; i < 3; i++) {
   arrival.reset_next_substream();
   size.reset_next_substream();
 }

 /* print the first 5 arrival times and sizes */
 for (int j = 0; j < 5; j++) {
   printf ("%-8.3f  %-4d\n", arrival.lognormal(5, 0.1),
             int(size.normal(500, 10)));
 }
\end{verbatim}

\subsubsection{Output}
\begin{verbatim}
 161.826   506
 160.591   503
 157.145   509
 137.715   507
 118.573   496
\end{verbatim}

\section{Random Variables}
\label{sec:ranvar}

The \clsref{RandomVariable}{../ns-2/ranvar.h}
provides a thin layer of functionality on top
of the base random number generator and the default random number stream.
It is defined in \nsf{ranvar.h}:

\begin{program}
  class RandomVariable : public TclObject \{
  public:
        virtual double value() = 0;
        int command(int argc, const char*const* argv);
        RandomVariable();
  protected:
        RNG* rng_;
  \};
\end{program}

Classes derived from this abstract class implement specific
distributions.  Each distribution is parameterized with the values of
appropriate parameters.  The value method is used to return a value
from the distribution.

The currently defined distributions, and their associated parameters are:

\begin{tabular}{rl}
\clsref{UniformRandomVariable}{tools/ranvar.h} & \code{min_}, \code{max_} \\
\clsref{ExponentialRandomVariable}{tools/ranvar.h} & \code{avg_} \\
\clsref{ParetoRandomVariable}{tools/ranvar.h} & \code{avg_}, \code{shape_}\\
\clsref{ParetoIIRandomVariable}{tools/ranvar.h} & \code{avg_}, \code{shape_}\\
\clsref{ConstantRandomVariable}{tools/ranvar.h} & \code{val_}\\
\clsref{HyperExponentialRandomVariable}{tools/ranvar.h} & \code{avg_}, \code{cov_}\\
\clsref{NormalRandomVariable}{tools/ranvar.h} & \code{avg_}, \code{std_}\\
\clsref{LogNormalRandomVariable}{tools/ranvar.h} & \code{avg_}, \code{std_}\\
\end{tabular}

The RandomVariable class is available in OTcl.  For instance, to
create a random variable that generates number uniformly on [10, 20]:
\begin{program}
        set u [new RandomVariable/Uniform]
        $u set min_ 10
        $u set max_ 20
        $u value
\end{program}
By default, RandomVariable objects use the default random number
generator described in the previous section.  The use-rng method can
be used to associate a RandomVariable with a non-default RNG:
\begin{program}
        set rng [new RNG]
        $rng seed 0

        set e [new RandomVariable/Exponential]
        $e use-rng $rng
\end{program}

\section{Integrals}
\label{sec:integral}

The  \clsref{Integrator}{../ns-2/integrator.h}
supports the approximation of (continuous) integration by (discrete)
sums; it is defined in \nsf{integrator.h} as
\begin{program}
{\rm From integrator.h:}
        class Integrator : public TclObject \{
        public:
                Integrator();
                void set(double x, double y);
                void newPoint(double x, double y);
                int command(int argc, const char*const* argv);
        protected:
                double lastx_;
                double lasty_;
                double sum_;
        \};

{\rm From integrator.cc:}
        Integrator::Integrator() : lastx_(0.), lasty_(0.), sum_(0.)
        \{
                bind("lastx_", &lastx_);
                bind("lasty_", &lasty_);
                bind("sum_", &sum_);
        \}

        void Integrator::set(double x, double y)
        \{
                lastx_ = x;
                lasty_ = y;
                sum_ = 0.;
        \}

        void Integrator::newPoint(double x, double y)
        \{
                sum_ += (x - lastx_) * lasty_;
                lastx_ = x;
                lasty_ = y;
        \}

        int Integrator::command(int argc, const char*const* argv)
        \{
                if (argc == 4) \{
                        if (strcmp(argv[1], "newpoint") == 0) \{
                                double x = atof(argv[2]);
                                double y = atof(argv[3]);
                                newPoint(x, y);
                                return (TCL_OK);
                        \}
                \}
                return (TclObject::command(argc, argv));
        \}
\end{program}
This class provides a base class used by other classes such
as \code{QueueMonitor} that keep running sums.
Each new element of the running sum is added by
the \fcn[x, y]{newPoint} function.
After the $k$th execution of \code{newPoint}, the running sum
is equal to $\sum_{i=1}^{k}y_{i-1}(x_i - x_{i-1})$ where
$x_0 = y_0 = 0$ unless \code{lastx\_}, \code{lasty\_}, or \code{sum\_}
are reset via OTcl.
Note that a new point in the sum can be added either by the
C++ member \code{newPoint} or the OTcl member \code{newpoint}.
The use of integrals to compute certain types of averages
(e.g. mean queue lengths) is given in (pp. 429--430, \cite{Jain91:Art}).

\section{\code{ns-random}}

{\bf \code{ns-random} is an obsolete way to generate random numbers.
This information is provided only for backward compatibility.}

\code{ns-random} is implemented in \nsf{misc.\{cc,h\}}.
When called with no argument, it generates a random number with
uniform distribution between 0 and \code{MAXINT}.
When an integer argument is provided, it seeds the random generater
with the given number.
A special case is when \code{ns-random 0} is called, it randomly seeds
the generator based on current time.
This feature is useful to produce non-deterministic results across
runs.

\section{Some mathematical-support related objects}
\label{sec:mathobjects}

\textsc{Integrator objects}
Integrator Objects support the approximate computation of continuous
integrals using discrete sums. The running sum(integral) is computed as:
\code{sum_ += [lasty_ * (x lastx_)]} where (x, y) is the last element
entered and
(lastx\_, lasty\_) was the element previous to that added to the sum.
lastx\_ and lasty\_ are updated as new elements are added. The first
sample point defaults to (0,0) that can be changed by changing the values
of (lastx\_,lasty\_).
\code{$integrator newpoint <x> <y>}\\ %$
Add the point (x,y) to the sum. Note that it does not make sense for x to
be less than lastx\_.

There are no configuration parameters specific to this object. \\

State Variables are:
\begin{description}
\item[lastx\_]
x-coordinate of the last sample point.

\item[lasty\_]
y-coordinate of the last sample point.

\item[sum\_] Running sum (i.e. the integral) of the sample points.
\end{description}

\textsc{Samples Object}
Samples Objects support the computation of mean and variance statistics
for a given sample.

\code{$samples mean}\\
Returns mean of the sample.

\code{$samples variance}\\
Returns variance of the sample.

\code{$samples cnt}\\
Returns a count of the sample points considered.

\code{$samples reset }\\
Reset the Samples object to monitor a fresh set of samples.

There are no configuration parameters or state variables specific to this
object.

\section{Commands at a glance}
\label{sec:mathcommand}

Following is a list of mathematical support related commands commonly used
in simulation scripts:
\begin{flushleft}
\code{set rng [new RNG]}\\
This creates a new random number generator.

\code{$rng seed <0 or n>}\\
This command seeds the RNG. If 0 is specified, the RNG is seeded
heuristically. Otherwise the RNG is seeded with the value <n>.

\code{$rng next-random}\\
This returns the next random number from RNG.

\code{$rng uniform <a> <b>}\\
This returns a number uniformly distributed on <a> and <b>.

\code{$rng integer <k>}\\
This returns an integer uniformly distributed on 0 and k-1.

\code{$rng exponential}\\
This returns a number that has exponential distribution with average 1.

\code{set rv [new Randomvariable/<type of random-variable>]}\\
This creates an instance of a random variable object that generates random
variables with specific distribution. The different types of random
variables derived from the base class are:\\
RandomVariable/Uniform, RandomVariable/Exponential, RandomVariable/Pareto,
RandomVariable/Constant, RandomVariable/HyperExponential.
Each of these distribution types are parameterized with values of
appropriate parameters. For details see section \ref{sec:ranvar} of this
chapter.

\code{$rv use-rng <rng>}\\
This method is used to associated a random variable object with a
non-default RNG. Otherwise by default, the random variable object is
associated with the default random number generator.

\end{flushleft}

% LocalWords:  ranvar pareto expoo tcl lib ns cc integrator RNG rng TclObject
% LocalWords:  enum RNGSources PREDEF int MAXINT rX edv prob edp setbit iph ECN
% LocalWords:  ip ecn eed OURCE OTcl RandomVariable argc const argv min avg val
% LocalWords:  UniformRandomVariable ExponentialRandomVariable cov newPoint pp
% LocalWords:  ParetoRandomVariable ConstantRandomVariable lastx lasty strcmp
% LocalWords:  HyperExponentialRandomVariable newpoint atof QueueMonitor


\documentclass[11pt,twoside,a5paper]{article}

\usepackage[a4paper,top=2.5cm,bottom=2.5cm,left=2.2cm,right=2.2cm]{geometry}
\usepackage[T1]{fontenc}
\usepackage{verbatim}
\usepackage[normalem]{ulem} % for striking out with \sout
\usepackage{amsfonts,amsmath,amssymb} % for more math support
\usepackage{times}
\usepackage[italic]{mathastext} % use normal text font (times) in equations
\usepackage{enumitem}
%\usepackage[draft]{hyperref}
\usepackage[colorlinks,linkcolor=blue,citecolor=blue,urlcolor=blue]{hyperref}
\usepackage{color}
\usepackage{listings}

\newcommand{\dc}{\mathcal{D} }

\begin{document}
\title{3x2 Covariance emulator}
%\author{Tom McClintock}
\maketitle

\section{Emulator in the big picture}
Our covariance emulator will be quite simple: it will only take in a cosmology and then spit out a matrix to the user. The function signature for making predictions will look something like
\begin{lstlisting}[language=Python]
def predict_matrix(cosmology):
    """
    Inputs:
    cosmology - a container of the cosmological parameters

    Outputs:
    C_inverse - an NxN array containing the inverse covariance matrix.
    The dimensions will be specified by the training data,
    which we have to assume will never change, unless we
    want to get real fancy.
    """
    #do things
    return C_inverse.
\end{lstlisting}
Internally, the algorithm used to make a prediction with the emulator will look like the following:
\begin{enumerate}
\item Predict some small number of important quantities, $X$, that depend on cosmology $\Omega$ such as PCA components of some matrix.
\item Use $X$ to reconstruct the diagonal components $D$ and the off-diagonal components $T$ of the GCD components.
\item Reassemble the GCD components into the inverse covariance matrix $C^{-1}$.
\end{enumerate}
This algorithm is thus the reverse of section 3 from Morrison \& Schneider (2013), where they describe how the emulator is designed. The prediction could be made with Gaussian processes, but doesn't have to be. If, for instance, we observed that some PCA components depended very simply on a certain cosmological parameter then we could model that directly.

Thus, to train our emulator we would apply this prediction code in reverse: take all the input inverse covariance matrices, reduce them to only a few numbers, find a good interpolation of those numbers as a function of cosmology. The following sections will describe how the emulator should be trained.

\section{Matrix decomposition: GCD}
Following Morrison \& Schneider, we should probably use GCD, but this isn't a necessity. The inverse covariance matrix, which is $N\times N$ gets decomposed like (I'm using slightly different notation than in their paper, for convenience)
\begin{equation}
  \label{eq:gcd}
  C^{-1} = L^TDL
\end{equation}
where $L$ is a lower triangular matrix (and therefore has $N(N-1)/2$ indepenent elements) and $D$ is a diagonal semi-positive definite matrix (with $N$ independent elements). Emulating $N(N-1)/2+N$ independent elements isn't feasible, so we have to reduce this further with PCA.

Before moving on, I should note that elements of $L$ can take on any numerical value, but since $D$ is semi-positive then we will work with $\ln D = d$ instead. In other words, we will work with the logarithm of the elements of $D$. This is important because the Gaussian process wouldn't have knowledge that $D$ is semi-positive, and could yield unphysical predictions.

We also subtract of the mean of all $d$ values and rescale the values of $d$ so that they have variance 1. This looks like
\begin{equation}
  \label{eq:rescale}
  d \rightarrow \frac{d - \bar{d}}/\sigma_d.
\end{equation}
This makes PCA described next slightly cleaner. When making the prediction, once we have $d$ from the Gaussian processes, we can transform it back to its original form by multiplying on $\sigma_d$ and adding $\bar{d}$. The same treatment goes for the avlues of $L$.

\section{PCA on the GCD parts}
Let's pretend that we have run $M$ different simulations (so for us, $M\sim16$). Let's think just about $d$ first. If we take all the $d$s obtained for each $M$ simulation and ``stack'' them on top of each other into a new matrix $\dc$ which is now $N\times M$ (row/column order doesn't matter). Performing PCA on this gives us
\begin{equation}
  \label{eq:PCA}
  \dc = UBV^T
\end{equation}
where if $p=\min(N,M)$ then $U$ is a $N\times p$ matrix with $U^TU=I_p$, $V$ is a $M\times p$ with $V^TV=I_p$ and $VV^T=I_M$, and $B$ is $p\times p$ diagonal matrix of singular values. Basis vectors are formed by $\Phi = \frac{1}{\sqrt{M}}UB$ which is $N\times p$, with weights $w=\sqrt{M}V^T$ normalized by $\frac{1}{\sqrt{M}}w^Tw=I_M$. The columns of $\Phi$, which are of length $N$, represent descreasingly important principle components of the original matrix $\dc$. This means we only need to use the first few of them. In the Coyote papers they found that using the first five was sufficient. This will be something to test.

The $i$th column of $\Phi$ corresponds to the $i$th row of $w$, or $w_i$. This $w_i$ has mean zero, and (according to the papers...) contains all of the cosmologial dependence, and is also a function of the domain.

In the coyote papers, the domain was $k$, whereas in our paper we have this very strange behaviour where the domain is not monotonic, since the ``full'' data vector is a combination of a bunch of data vectors. In other words, looking at Figure 5 of Elizabeth's paper, the data vectors are $\xi_+$, $\xi_-$, $\gamma_t$, and $w$. The domain is not monotonically changing along the edge, which isn't viable for design of the emulator. We will have to think of something, like using just bin number or something. The downside of doing so would mean that the number of bins would have to remain fixed forever.

\section{Gaussian Process}
In this design if we keep five columns then we would utilize five interpolators. We can do this because the weights are all independent of each other, so the prediction by one interpolator is independent of any other interpolator. The challenge we have is to figure out the best way to interpolate $w_i$ across cosmologies and domain. Gaussian processes (GPs) were used in the past because they provide an estimate of the expectation value $\langle w_i(\Omega,x)\rangle$ as well as an estimate of the variance ${\rm Var}(w_i)$. Note that we could achieve the same thing if we defined a model (a line, a quadratic, something else) for $w_i$ and knew the posterior distributions of the parameters in our model. Using GPs or such a model means that we can propogate the uncertainty in our estimate forward in a Bayesian way. On the other hand this could be computational expensive and end up being not adding much to a cosomological analysis.

I'll fill the rest of this out later, but the important thing that we care about is picking kernels with hyperparameters, writing a likelihood, and then finding the best hyperparameters.

\end{document}