\chapter{Introduction}
In this paper the nonhomogenous Poisson process is used to generate data and conditional samples from sufficient statistics. The nonhomogenous Poisson process is a combination of the power law case and log-linear case. To generate the samples a Monte Carlo approach is used. The approach is a Gibbs sampler with a Metropolis-Hastings step.
\\
\\
The hypothesis test theory is used to check if the samples comes from the right distribution. This is the null hypothesis. To check the null hypothesis several condtional tests are used. The conditional tests are generated by simulation of conditional samples. These samples are conditional on minimum sufficent statistics.
\\
\\
The motivation for this study comes from the papers \cite{lindqvist2011monte} and \cite{lockhart2007use}. In the first paper two types of NHPP were studied. The first one was a case of log-linear rate function, and the second case was the log-linear case. The paper proposed to look at a NHPP where these cases were combined. The second paper is about conditional sampling from a gamma distribution\cite{stacy1962generalization}. In this paper the same sufficient statistics were used.
\\
\\
First the paper is going to introduce the nonhomogenous Poisson process. Further necessary theory and the sampling algorithm is introduced. At the end the results and analysis of the simulations are presented.


\title{Overlapping Group Lasso Via ADMM in Python}
\author{Daniel Kessler}

\documentclass{article}

% \usepackage{parskip}
\usepackage{geometry}
\usepackage{amsmath,amssymb,physics,framed,mathtools}
\usepackage{algorithm,algpseudocode}
\usepackage{graphicx}
\usepackage[utf8]{inputenc}
\usepackage[style=numeric,citestyle=numeric]{biblatex}
\addbibresource{refs.bib}

\AtEveryBibitem{
    \clearfield{urlyear}
    \clearfield{urlmonth}
}

\usepackage{hyperref}
\begin{document}

\maketitle

\begin{abstract}
  In this report, we present a brief summary of the overlapping group lasso, showing how it can be motivated as an extension first of the regular lasso to a group setting, and then as a further generalization thereof.
  We then provide some background on the Alternating Direction Method of Multipliers (ADMM) algorithm, and show why it is a reasonable choice for solving overlapping group lasso problems.
  Next, we derive the ADMM algorithm for overlapping group lasso, and present a software implementation in python that implements this algorithm.
  Finally, we show some experimental results on synthetic data across a variety of parameter settings.
\end{abstract}

\section{Notation}
\label{sec:notation}

First, we fix notation.
Since much of our derivations are based on \cite{boyd_distributed_2011}, we will use different notation than is typically deployed in more statistics-oriented treatments of the lasso.
Let $A \in \mathbb{R}^{m \times n}$ be a (fixed) design matrix, with $m$ observations and $n$ covariates, and $b \in \mathbb{R}^m$ a vector of observations.
We assume that our $b$ follows $b = A x^{\star} + \epsilon$, where $x^{\star} \in \mathbb{R}^p$ is an unknown weight vector, and $\epsilon$ are independent and identically distributed errors (for a simple case, we can take them to follow $\mathcal{N}(0,\sigma^2)$ for some fixed, but unknown, $\sigma^2$).
When norms are not otherwise specified, they are taken to be the 2-norm, i.e., $\lVert \cdot \rVert \overset{\Delta}{=} \lVert \cdot \rVert_2$.
We will generally be interested in minimizing the least squares loss, i.e., finding $\hat{x} \in \operatorname{argmin}_x \left\| Ax - b \right\|$.
Note: A great deal of the treatment below, including that of the lasso, group lasso, overlapping group lasso, and ADMM is taken from \cite{boyd_distributed_2011}.
We explicitly cite this text at key points, but we do not cite \emph{every} claim which is based on \cite{boyd_distributed_2011} to avoid cluttering the text.

\section{Background: Overlapping Group Lasso}
\label{sec:backgr-group-lasso}
In order to introduce the overlapping group lasso, we will first discuss the regular lasso and then show how it can be extended to the (non-overlapping) group lasso setting.
The lasso is a highly popular method that is especially useful in high dimensional settings, i.e., where $n \gg m$.
In this setting (presuming $A$ is of full rank), the OLS estimate is no longer uniquely determined, as there exist infinitely many candidate $\hat{x}$ that yield zero loss.
Instead, one can instead minimize a \emph{regularized problem} in order to obtain a \emph{sparse solution}.
While various (essentially equivalent) formulations of the lasso objective exist, for our purposes we will define the primal lasso problem as
\begin{equation}
  \label{eq:1}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \lVert x \rVert_1,
\end{equation}
where $\lambda > 0$ determines the amount of regularization.
As $\lambda \to 0$, we see that \eqref{eq:1} becomes the OLS problem, and as $\lambda \to \infty$, $x \to 0$.
One advantage of the lasso is that it will typically recover a sparse solution, i.e., a solution where the minimizing $\hat{x}$ has many entries that are identically 0.

In the setting where covariates can be organized into groups, as may be natural in many applied settings (e.g., where the covariates are gene expression levels, and genes can be organized based on chromosome or location), we may wish not to simply select useful covariates, but instead to select useful \emph{groups} of covariates.
This motivates the use of \emph{group lasso} \cite{yuan_model_2006}, where we replace the objective in \eqref{eq:1} with
\begin{equation}
  \label{eq:2}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_i \rVert_2,
\end{equation}
where $x_i$ is a sub-vector of $x$ containing only the coefficients corresponding to the $i$'th group, with $i \in [N]$.
Note that when extending to the group lasso, the penalty term no longer involves the 1-norm but instead has the 2-norm.
Although this may seem surprising, the 1-norm is separable, i.e., $\left\| \begin{pmatrix} a_1 \\ a_2 \end{pmatrix} \right\|_1 = \lVert a_1 \rVert_1 + \lVert a_2 \rVert_1$, and this would devolve back to the original lasso.
Critically, the 2-norm in the regularizer is not squared, which yields an analogous geometry to the lasso, with singularities corresponding to solutions that are group-sparse (see \cite{yuan_model_2006}, Fig 1 for a helpful illustration of this phenomena).
Note that for a singleton vector, $\lVert a \rVert_1 = \lVert a \rVert_2$, so when $N = n$, i.e., each feature is alone in its own atomic group, we can rewrite \eqref{eq:2} as
\begin{align*}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_i \rVert_2
  &= \operatorname*{min}_x \frac{1}{2}  \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_i \rVert_1 \\
  &= \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda  \lVert x \rVert_1    ,
\end{align*}
and we can recover the original lasso formulation in \eqref{eq:1} as a special case of the group lasso.

Finally, the \emph{group lasso} can be further extended to accommodate \emph{overlapping groups} in the ``overlapping group lasso'' \cite{zhao_composite_2009,mairal_network_2010}.
In this setting, rather than partitioning $x$ into disjoint sub-vectors, we let $G_i, i = [N]$ be an index set holding the indices of coefficients corresponding to the $i$'th group, i.e., $x_{G_i}$ is a vector of coefficients for group $i$, $x_{G_j}$ is a vector of coefficients for group $j$, and it may be the case that $G_i \cap G_j \neq \emptyset$.
As a toy example, suppose $n=3$, and in this simple setting we have two groups, with $G_1 = \left\{ 1, 2 \right\}, G_2 = \left\{ 2, 3 \right\}$, such that $x_2$ is common to both $x_{G_1}$ and $x_{G_2}$, i.e., $G_1 \cap G_2 = \left\{ 2 \right\}$.
In this setting, the overlapping group lasso objective is given by
\begin{equation}
  \label{eq:3}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_{G_i} \rVert_2.
\end{equation}
Of course, the (non-overlapping) group lasso discussed above is a special case of the overlapping group lasso, and by transitivity, since lasso is a special case of the group lasso, it is also a special case of the overlapping group lasso.

The geometry of this problem is rather complicated, and there is some work (e.g., \cite{jacob_group_2009}) that proposes addressing the overlapping group lasso through latent variables, in essence, performing variable duplication to render the problem non-overlapping, and then using the standard group lasso formulation in \eqref{eq:2} to solve the problem.
However, in the present work we will focus on directly optimizing the objective given in \eqref{eq:3}, although as we shall see in Section \ref{sec:group-lasso-via}, our algorithmic approach will involve a sort of variable duplication, but with an update step that pulls our duplicated variables back toward one another.

\section{Background: ADMM}
\label{sec:background:-admm}
The Alternating Direction Method of Multipliers (ADMM) is an algorithmic approach to optimization well suited to solving problems that can be decomposed as the sum of two problems in distinct variables, subject to linear constraints.
The background we provide here will closely follow \cite{boyd_distributed_2011}, as this was our primary resource when endeavoring to learn the material.
Our development here will be terse and limited, and we refer the reader to \cite{boyd_distributed_2011}, as the exposition given below chiefly consists of key highlights from this very useful text.

ADMM is formulated to solve problems structured as
\begin{equation}
  \begin{aligned}
    \label{eq:4}
    \operatorname*{min}_{x,z} \qquad &f(x) + g(z) \\
    \text{subject to} \qquad&Ax + Bz = c
  \end{aligned}.
\end{equation}
It is closely related to the method of multipliers (a brief background is given in \cite{boyd_distributed_2011}) and proceeds by first constructing an augmented Lagrangian
\begin{equation}
  \label{eq:5}
  L_{\rho}(x,z,y) = f(x) + g(z) + y^T (Ax + Bz - c) + \frac{\rho}{2} \lVert Ax + Bz - c \rVert_2^2,
\end{equation}
where $y$ is a dual variable, and the last term is the ``augmenting'' piece.
Although augmenting may seem unnatural at first, we note that when the linear constraints are satisfied, this last term is identically 0 and thus inconsequential for the objective function at the optimum, and its inclusion makes the use of the \texttt{prox} operator natural during the optimization.
ADMM is an iterative procedure, which given some initial values for $x,z,y$, proceeds as
\begin{align}
  \label{eq:6}
  x^{k+1} &\leftarrow \operatorname*{argmin}_x L_{\rho}(x,z^k,y^k) \\
  \label{eq:7}
  z^{k+1} &\leftarrow \operatorname*{argmin}_x L_{\rho}(x^{k+1},z,y^k) \\
  \label{eq:8}
  y^{k+1} &\leftarrow y^k + \rho(Ax^{k+1} + Bz^{k+1} -c).
\end{align}
Of course, the rub lies in actually solving the sub-problems given in \eqref{eq:6}\eqref{eq:7}, but using the augmented Lagrangian $L_{\rho}$ makes this tractable for certain problems.

\section{Overlapping Group LASSO Via ADMM}
\label{sec:group-lasso-via}
There is treatment of lasso, group lasso, and (very brief) mention of overlapping group lasso in \S 6.4 of \cite{boyd_distributed_2011}.
We noted in Section \ref{sec:backgr-group-lasso} that the group lasso, and in turn lasso, can be recovered as special cases of the overlapping group lasso.
However, \cite{boyd_distributed_2011} attempts to make (both overlapping and non-overlapping) group lasso notationally consistent with their later section on consensus learning, which actually yields an algorithm that does not permit recovery of these simpler forms as special cases, since the roles of $x$ and $z$ are reversed.
In our treatment here, we have eschewed this notational change and describe an approach that connects more naturally to lasso.

Our goal is to rewrite \eqref{eq:3} in a form amenable to application of ADMM but that is still equivalent to the original optimization problem.
In a similar spirit to the ``latent'' approach to overlapping group lasso \cite{jacob_group_2009,obozinski_group_2011}, we will create many \emph{new} variables which in a strict sense do not overlap, but then use linear equality constraints to impose the requirement that their relevant components coincide via a common ``anchoring'' global variable, which will indirectly enforce equality in the shared components.
In particular, we rewrite \eqref{eq:3} as,
\begin{equation}
  \begin{aligned}
  \label{eq:9}
  \operatorname*{min}_{x,z_i, i \in [N]} \quad &\frac{1}{2} \lVert A x - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert z_i \rVert_2 \\
  \text{such that} \quad & \tilde{x}_i - z_i = 0, \quad \forall i \in [N]
  \end{aligned},
\end{equation}
where $\tilde{x}_i \overset{\Delta}{=} x_{G_i}$, i.e., the components of $x$ corresponding to group $i$ for $i \in [N]$, and the constraint ensures that the solution satisfies $z_i = x_{G_i}, z_i \in \mathbb{R}^{\left| G_i \right|}$
This is now in a form compatible with ADMM, where the second term is analogous to $g(z)$, where $z$ is the concatenation of all the $z_i$'s, and $g$ very naturally decomposes across the $z_i$, which permits the update of the $z_i$ to be done in parallel.

Now, our task is to give explicit forms for steps \eqref{eq:6}\eqref{eq:7}.
Our update can be performed as
\begin{align}
  \label{eq:11}
  x^{k+1} &\leftarrow (A^TA + \rho I)^{-1}(A^T b + \rho(\bar{z}^{k+1} - \bar{u}^k)) \\
  \label{eq:10}
  z_i^{k+1} &\leftarrow \operatorname*{argmin}_{z_i} (\lambda \lVert z_i \rVert_2 + \left( u_i^k \right)^T(z_i - \tilde{x}_i^k) + \frac{\rho}{2} \lVert z_i - \tilde{x}_i^k \rVert_2^2) \\
  \label{eq:12}
  u_i^{k+1} &\leftarrow u_i^k +  \tilde{x}_i^{k+1} - z_i^{k+1},
\end{align}
where $\bar{z}, \bar{u}$ are obtained by averaging over $z_i, u_i$ at the relevant components.
The operation at \eqref{eq:10} is precisely the proximal operator, which for group lasso is the vector soft thresholding operation, as given in \S 6.4.2 of \cite{boyd_distributed_2011}, i.e.,
\begin{equation}
  \label{eq:13}
  z_i^{k+1} = (1 - \frac{\lambda}{\rho} \lVert \tilde{x}_i^{k+1} + u^k \rVert_2^{-1})_+ (\tilde{x}_i^{k+1} + u^k)
\end{equation}

We summarize the approach in Algorithm \ref{alg:grp-lasso-admm}.
\begin{algorithm}
    \caption{Overlapping Group LASSO Via ADMM}
    \label{alg:grp-lasso-admm}
    \begin{algorithmic}[1]
      \Require{$A, b, x^0 \in \mathbb{R}^n, \rho, \lambda, G$}
      \State $Q \gets (A^TA + \rho I)^{-1}$
      \For{$i \in [N]$}
      \State $z_i \gets x^0_{G_i}$
      \State $u_i \gets \left\{ 0 \right\}^{\lvert G_i \rvert}$
      \EndFor
      \State $k \gets 0$
      \While{not converged}
      \State $x^{k+1} \gets Q \left( A^Tb + \rho(\bar{z}^{k+1} - \bar{u} \right)$
      \For{$i \in [N]$}
      \State $z_i^{k+1} \gets S_{\lambda/\rho}(\tilde{x}_i^{k+1} + u_i)$
      \EndFor
      \For{$i \in [N]$}
      \State $u_i^{k+1} \gets u_i^k + \tilde{x}_i^{k+1} - z_i^{k+1} $
      \EndFor
      \State $k \gets k+1$
      \EndWhile
  \end{algorithmic}
\end{algorithm}

\section{Software Implementation}
\label{sec:implementation}
We implemented the procedure of Algorithm \ref{alg:grp-lasso-admm} in python 3.7.
The implementation, along with this report, is available on github at \url{http://github.com/dankessler/608a-project}.
The dependency environment can be reconstructed using \href{https://pipenv.readthedocs.io/en/latest/}{\texttt{pipenv}} using the \texttt{Pipfile} and \texttt{Pipfile.lock} found at the root of the repository.

\section{Experimental Results}
\label{sec:experimental-results}
We conducted numerical experiments in simulated data.
A script named \texttt{simulations.py} reproducing these results is available on github in the root directory of our repository \url{http://github.com/dankessler/608a-project}.
We generate synthetic data as $y = A x^{\star} + \epsilon$, where $\epsilon \sim \mathcal{N}_m(0,I_m), A \in \mathbb{R}^{m \times n}$ is a random matrix with iid standard normal entries, and for all simulations we fix $m = 50, n = 100$.
Our three experiments below are under different settings for $x^{\star}$.
In each case, we run our implementation of Algorithm \ref{alg:grp-lasso-admm} on the synthetic data with $\rho = 1$ and vary $\lambda$ over a logarithmic grid.
Each evaluation yields a regularized estimate $\hat{x}$, and with this we evaluate
\begin{enumerate}
\item The prediction error: $
  \left\|
    A x^{\star} - A \hat{x}
  \right\|_2$
\item The accuracy of the estimated $x$: $
  \left\|
    x^{\star} - \hat{x}
  \right\|_2$
\item The precision of the recovery of the support: $\frac{
    \left|
      \left\{
        i: \hat{x}_i \neq 0
      \right\}
      \cap
      \left\{
        j: x^{\star}_j \neq 0
      \right\}
    \right|}{
    \left\|
      \hat{x}
    \right\|_0}$
\item The recall of the recovery of the support: $\frac{
    \left|
      \left\{
        i: \hat{x}_i \neq 0
      \right\}
      \cap
      \left\{
        j: x^{\star}_j \neq 0
      \right\}
    \right|}{
    \left\|
      x^{\star}
    \right\|_0}$
\end{enumerate}
Because the $x$ iterates in our algorithm never become \emph{truly} sparse, as a heuristic after returning $\hat{x}$ for a fixed number of iterations (here one thousand) we set to 0 any entries of $\hat{x}$ with magnitude smaller than $\epsilon = .01$.
This enables more meaningful interpretation for the latter two metrics (since otherwise the set of components of $\hat{x}$ that are nonzero will be all $n$.
These results are depicted graphically as a function of $\lambda$ in Figures \ref{fig:lasso}, \ref{fig:glasso}, and \ref{fig:oglasso}.

\subsection{Lasso as a Special Case}
\label{sec:lasso-as-special}
As discussed earlier, if we place each covariate into its own atomic group, our problem devolves to the classic lasso.
We set $x_i^{\star} = 10$ for $i \in [25]$, and $x_i^{\star} = 0$ for $i \in [50] \setminus [25]$.
We conduct the simulation approach as described above, and display our results in Figure \ref{fig:lasso}.

\begin{figure}[h]
  \centering
  \includegraphics[width=\textwidth]{../lasso}
  \caption{Performance metrics for simulations under lasso. Larger values of $\lambda$ generally encourage a sparser solution, which seems to yield worse prediction error, estimation error, and recall in recovery of support. Precision initially improves, but this is likely due to a shrinking denominator.}
  \label{fig:lasso}
\end{figure}

\subsection{Non-overlapping Group Lasso as a Special Case}
\label{sec:non-overl-group}
Our set up for (non-overlapping) group lasso is reasonably straightforward.
We construct two disjoint groups as $G_1 = [25], G_2 = [50] \setminus G_1$.
We set $x^{\star}_i = 10$ for $i \in G_1$, and $x^{\star}_i = 0$ for $i \in G_2$.
Note that this is the same $x^{\star}$ as in lasso above, but the penalization is different.
The results are presented in Figure \ref{fig:glasso}.

\begin{figure}[h]
  \centering
  \includegraphics[width=\textwidth]{../glasso}
  \caption{Performance metrics for simulations under (non-overlapping) group lasso. These plots are reasonably similar to those in Figure \ref{fig:lasso}, which is unsurprising since the problem set up is very similar.}
  \label{fig:glasso}
\end{figure}

\subsection{Overlapping Group Lasso}
\label{sec:overl-group-lasso}
For overlapping group lasso, we construct 9 groups of equal size that overlap by the same amount.
In particular, $G_i = \{5(i-1) + j : j \in [10]\}, i \in [9]$.
For example, $G_1 =
\left\{
  1, 2, 3, 4, 5, 6, 7, 8, 9, 10
\right\}$, $G_2 =
\left\{
  6, 7, 8, 9, 10, 11, 12, 13, 14, 15
\right\}$, so $G_1 \cup G_2 =
\left\{
  6, 7, 8, 9, 10
\right\}$.
As was discussed briefly earlier, our formulation of the overlapping group lasso, unlike that in \cite{jacob_group_2009,obozinski_group_2011}, has singularities corresponding to the complement of unions of groups.
For this reason, we set $x^{\star}_i = 10$ for $i \in
\left\{
  1, 2, 3, 4, 5, 16, 17, 18, 19, 20
\right\}$ and 0 otherwise.
This corresponds to the complement of the union of groups $G_2, G_3, G_5, G_6, G_7, G_8$,  and $G_9$.
Results are presented in Figure \ref{fig:oglasso}.

\begin{figure}[h]
  \centering
  \includegraphics[width=\textwidth]{../oglasso}
  \caption{Performance metrics for simulations under overlapping group lasso.
As with all our results, increasing $\lambda$ hurts prediction error. Also, we can infer that we require high values of $\lambda$  before we obtain solutions that are at all sparse, and this is why recall is initially so high. In terms of estimation error, extreme values of $\lambda$ (too big or too small) give poor performance, but things are quite good in the middle.}
  \label{fig:oglasso}
\end{figure}

\section{Conclusion and Future Directions}
\label{sec:concl-future-direct}
In this report, we provided a brief introduction to overlapping group lasso, reviewed requisite background materials on ADMM, and then showed how ADMM could be applied to the overlapping group lasso problem.
We then conducted numerical experiments on simulated data under three different settings: lasso as a special case of overlapping group lasso, (non-overlapping) group lasso as a special case of overlapping group lasso, and finally a non-trivial overlapping group lasso problem.
One important note is that our approach is in contrast to one favored by some statisticians \cite{obozinski_group_2011,jacob_group_2009}.
The reason for this is that the explicit formulation of the overlapping group lasso, as we have implemented,
is well positioned for recovery in settings where the true support is the complement of a union of groups.
Conceptually, this corresponds to ``deactivating'' a set of groups, and zeroing out all of their corresponding features, which in the case of overlaps, will result in some ``active'' groups only being active in components that are not shared with a deactivated group.
In some applied settings this may be a reasonably accurate regime, but in settings where it is more natural to countenance the support being the union of \emph{active} groups, an alternative formulation, described in \cite{obozinski_group_2011,jacob_group_2009}, proposes that variables be duplicated to render the problem non-overlapping, but does \emph{not} involve the introduction of linear constraints that require that corresponding entries perfectly match.
This effectively renders the problem non-overlapping.
For some of the applied settings in neuroscience where we propose to use our newly developed overlapping group lasso package in python, this latter formulation is more natural.
However, the good news is that since we have demonstrated that our approach can solve \emph{both} of these problems, as the latter is a special case of the former, the software that we have developed in this project can be used directly (with some wrapper functionality to handle the bookkeeping involved in variable duplication).

As a next step, we plan to validate the performance of our implementation against a reference implementation.
While there exist many implementations of lasso and a few implementations of non-overlapping group lasso, we have yet to find a overlapping group lasso implementation that seems to give sensible results.
Nonetheless, if we can validate our implementation against the simpler cases, we may be able to use our package as a benchmark against this other implementations to better understand their shortcomings.

\clearpage
\printbibliography

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:

%  LocalWords:  ADMM covariates argmin OLS eq th regularizer Bz prox
%  LocalWords:  notationally notational thresholding github pipenv py
%  LocalWords:  Pipfile iid covariate neuroscience


\documentclass[a4paper]{siamart190516}

\usepackage{damacros}

% PACKAGES added by TB
% PACKAGES FOR ALGORITHMS (PSEUDO-CODE)
\usepackage{algorithm}
\usepackage{algorithmic}

% Sets running headers as well as PDF title and authors
\headers{Coupling intra-cellular and multi-cellular dynamics in spatially-extended models of root-hair initiation}{D. Avitabile, S. Perotto, N. Ferro, T. Babini}

% Title. If the supplement option is on, then "Supplementary Material"
% is automatically inserted before the title.
\title{Coupling intra-cellular and multi-cellular dynamics in spatially-extended models of root-hair initiation}

% Authors: full names plus addresses.
\author{%
  Daniele Avitabile%
  \thanks{%
    Vrije Universiteit Amsterdam,
    Department of Mathematics,
    Faculteit der Exacte Wetenschappen,
    De Boelelaan 1081a,
    1081 HV Amsterdam, The Netherlands.
  \protect\\
    Inria Sophia Antipolis M\'editerran\'ee Research Centre,
    MathNeuro Team,
    2004 route des Lucioles-Boîte Postale 93 06902,
    Sophia Antipolis, Cedex, France.
  \protect\\
    (\email{d.avitabile@vu.nl}, \url{writemywebpage}).
  }
 %  .... mancano altri
  % \and
%   Paul T. Frank \thanks{Department of Applied Mathematics, Fictional University, Boise, ID
% (\email{ptfrank@fictional.edu}, \email{jesmith@fictional.edu}).}
% \and Jane E. Smith\footnotemark[3]
}

\begin{document}

\maketitle

\begin{abstract}
This thesis deals with novel models and numerical approximations of spatially-extended multi-cellular models of Rho Of Plants (ROPs), that is, a family of proteins responsible for root-hair initiation in the plant cell Arabidopsis thaliana. The study of this dynamical system is of great relevance in the so-called agriculture 4.0, since it is instrumental to optimise plant uptake. In particular, ascertaining how intra-cellular protein distributions and extra-cellular coupling influence root-hair initiation is a challenging but pressing problem.

Current studies have focussed on two separate model types: on the one hand, ROPs dynamics is studied in single-cell models, which resolve patterns at sub cellular level; on the other, multi- cellular models with realistic geometries neglect intra-cellular patterning. In this thesis we make progress on coupling these two
model descriptions.

We initially focus on a well-established single-cell, nonlinear reaction-diffusion model, here approximated for the first time with a finite-element scheme. In addition, we present a new model which couples multiple cells through ROP flux at the interface. We present numerical evidence that such coupling has a bearing on the patterns supported by the model. It is shown that, under variations of auxin gradients, the model robustly forms ROP hotspots from ROP stripes, and that spots are later advected downstream.

Finally, we consider a novel model in which the auxin dynamics are not prescribed, but derive from the interaction between this hormone and other membranals proteins (PIN). We show that self-sustained auxin oscillations influence ROP intracellular patterning.
\end{abstract}

\section{Introduction (Daniele+Teresa+Simona)}

mettere il modello Capitolo 4 (solo equazioni) +  posizionamento in
letteratura + novelty paper

% from chapter 4 physical model sec.

the dynamics of auxin and carriers proteins PIN in each cell $\Omega_i$ with $i \in \{ 1, ..., N \}$ can be written as the following system of coupled ordinary differential equations:
\begin{equation}\begin{aligned}
\begin{cases}
  {\displaystyle d a_i\over\displaystyle d t} & = {\displaystyle 1 \over \displaystyle V_i} \sum_{j=1}^{N} A_{ij} \Phi_{ij} +k - \delta a_i \\[8pt]
  {\displaystyle d \Tilde{P}_{ij}\over\displaystyle d t} & = h\left(\Phi_{ij}\right) + \rho_0 - \mu \Tilde{P}_{ij},
\end{cases}
\end{aligned} \end{equation}

% ...

The original system of equations can be rescaled and simplified. In particular, under the assumptions of cells having same volume $V =V_i$ and exchange surface areas $A = A_{ij}$, rescaling properly the diffusion coefficient $D_a$ and variables $P_{ij}$, a new system is obtained. Thus, the final system of equations we work on is:
\begin{equation}\label{eq:Sys_auxPIN}\begin{aligned}
\begin{cases}
  {\displaystyle d a_i\over\displaystyle d t} & = \sum_{j \in \mathcal{N}_i} \displaystyle \Phi_{ji} +k - \delta a_i \\[8pt]
  {\displaystyle d P_{ij}\over\displaystyle d t} & = h\left(\Phi_{ij}\right) - \mu P_{ij},
\end{cases}
\end{aligned} \end{equation}

% ... from chapter 2 sec physical model
The dimentional reaction-diffusion model summarizing binding process, autocatalytic activation and catalysis of ROPs proteins described in Section \ref{sec:intromodel} is formulated as follows:
% As mentioned before, the dimentional reaction-diffusion model is:
\begin{equation} \label{eq:FM}\begin{aligned}
\left\lbrace
\begin{matrix}
\partial_t u = & D_1 \Delta_s u + k_{20} \alpha(x,y)u^2 v - \left(c+r\right) u + k_1 v  & \ \text{in} \ \Omega\\
    \partial_t v = & D_2 \Delta_s v - k_{20} \alpha(x,y) u^2 v - k_1 v + c \ u + b   & \ \text{in} \ \Omega.
\end{matrix}
\right.
\end{aligned}\end{equation}

% ...
The RD system is rewritten to explicit its mathematical formulation and methods in order to solve it in the most general way, including both the case of the RD system with the original parameters and the RD system with the rescaled ones.

As a consequence, we define a unified system of equations comprehensive of both \eqref{eq:FM} and \eqref{eq:adim} systems:
\begin{equation} \label{eq:final}
\left\lbrace
\begin{matrix}
\partial_t u = & \Tilde{D_1} \Delta_s u + \Tilde{a_1} u + \Tilde{b_1} v + \Tilde{c_1} u^2 v \ \ \text{in} \ \Omega\\
\partial_t v = & \Tilde{D_2} \Delta_s v + \Tilde{a_2} v + \Tilde{b_2} u + \Tilde{c_2} u^2 v + f_2 \ \ \text{in} \ \Omega,
\end{matrix}
\right.
\end{equation}
with tilde parameters defined differently in the two cases under study

\section{A starting modeling  (Teresa+Daniele)}

Modello di Capitolo 3

\textbf{Physical model - Sec.3.1}
We consider the root-hair cell projection onto a 2D rectangular domain, neglecting axial dimension.
% as in Chapter \ref{cap:2}
A system of four cells is schematically presented in Figure \ref{fig:2cell}. We can see that each cell has longitudinal and transverse boundaries in common with close cells.
We recall the single cellular model, namely:
\begin{equation} \label{eq:singModel}
\left\lbrace
\begin{matrix}
  \begin{aligned}
    & \partial_t u = \Tilde{D_1} \Delta_s u + \Tilde{a_1} u + \Tilde{b_1} v + \Tilde{c_1} u^2 v & \ \text{in} \ \Omega\\
    & \partial_t v = \Tilde{D_2} \Delta_s v + \Tilde{a_2} v + \Tilde{b_2} u + \Tilde{c_2} u^2 v + f_2 & \ \text{in} \ \Omega \\
    & \Tilde{D_1} \nabla_s u \cdot \mathbf{n} = 0 & \ \text{on} \ \partial \Omega \\
    & \Tilde{D_2} \nabla_s v \cdot \mathbf{n} = 0 & \ \text{on} \ \partial \Omega.
  \end{aligned}
\end{matrix}
\right.
\end{equation}
No-flux on $\partial \Omega$, namely Neumann homogeneous boundary conditions, characterizes the system behaviour along the cell boundary.
% prima spieghiamo il significato fisico, poi come rappresentarlo matematicamente
In the multi-cellular model, communication between cells is represented by allowed flux of ROPs, active and inactive, through localized channels along boundaries between neighboring cells.

We define as neighbor of cell $\Omega_i$ the set of cells with index in $\mathcal{N}_i = \{ j : \partial \Omega_j  \cap \partial \Omega_i \neq \emptyset \}$. The flux of concentration of active and inactive ROPs $(u_i, v_i)$ is proportional to the difference of concentration $(u_j, v_j)$ in neighbouring cells for $j \ \in \ \mathcal{N}_i$.

We formulate the new model still focusing on one single cell domain $\Omega_i$, taking into account the new flux generated from the discrepancy of concentrations with the neighboring cells. The new flux results in adding a non-homogeneous Neumann boundary condition on the common interfaces, as follows:
\begin{equation} \label{eq:pluriModel}
\left\lbrace
\begin{matrix}
  \begin{aligned}
    & \partial_t u_i = \Tilde{D_1} \Delta_s u_i + \Tilde{a_1} u_i + \Tilde{b_1} v_i + \Tilde{c_1} (u_i)^2 v_i & \ \text{in} \ \Omega_i\\[6pt]
    & \partial_t v_i = \Tilde{D_2} \Delta_s v_i + \Tilde{a_2} v_i + \Tilde{b_2} u_i + \Tilde{c_2} (u_i)^2 v_i + f_2 & \ \text{in} \ \Omega_i \\[6pt]
    & \Tilde{D_1} \nabla_s u_i \cdot \mathbf{n} = 0 & \ on \ \partial \Omega_i \backslash \cup_{j \in \mathcal{N}_i} \Gamma_{j,i} \\[6pt]
    & \Tilde{D_2} \nabla_s v_i \cdot \mathbf{n} = 0 & \ on \ \partial \Omega_i \backslash \cup_{j \in \mathcal{N}_i} \Gamma_{j,i} \\[6pt]
    & \Tilde{D_1} \nabla_s u_i \cdot \mathbf{n} = \beta_{uRR} \ \alpha_{uRR} \left(u_j - u_i \right) & \ on \ \Gamma_{j,i} \ \forall j \in  \mathcal{N}_i \\[6pt]
    & \Tilde{D_2} \nabla_s v_i \cdot \mathbf{n} = \beta_{vRR} \ \alpha_{vRR} \left(v_j - v_i \right) & \ on \ \Gamma_{j,i}\ \forall j \in  \mathcal{N}_i ,
  \end{aligned}
\end{matrix}
\right.
\end{equation}
where we define as $(u_i, v_i)$ the concentrations of active and inactive ROPs restricted to cell $\Omega_i$: $(u_i, v_i): \Omega_i \times \left(0, T_{max} \right) \longrightarrow \RSet^2$ and $\Gamma_{j,i}$ represents the common side between cell $\Omega_i$ and cell $\Omega_j \ \in \mathcal{N}_i $, therefore defined as: $\Gamma_{j,i} = \partial \Omega_i \cap \partial \Omega_j$.

Each of the neighboring cells follows the same model for hair formation, meaning that system in \eqref{eq:pluriModel} holds $\forall \ i$ cells composing the pluricellular system. As a consequence, the newly defined boundary conditions is coupled with the solutions $(u_j, v_j)$ with $j \ \in \mathcal{N}_i$. Therefore, the pluricellular system requires a proper iterative method for setting correctly boundary conditions depending on solutions in the neighboring cells.

Not communicating with other RH cells boundaries have as before no-flux. The new boundary conditions are characterized by a function and a coefficient for both active active ROPs $u$ and inactive ROPs $v$, having the same meaning:
\begin{itemize}
  \item $\beta_{u/v RR} \ [\frac{1}{\mu m^2}]$ are indicator functions defined on boundaries of cells, equal to $1$ where the communicating channels are open and $0$ where  no-flux is assumed;
  \item $\alpha_{u/v RR} \ [\frac{1}{\mu m}]$ are transport efficiency coefficients, representing a sort of flux quantity allowed through channels.
  \end{itemize}
These channel parameters aim at representing the average active transport along the sides of confining cells, set equal to the flux of proteins from one cell to the neighbouring ones.

We have no physical insight on previously cited functions modeling open channels for ROPs. A whole set of simulations for the proper tuning of parameters is required, in order to find a sufficiently plausible setting of the system.

\section{A new dd-wise coupling approach (Simona+Daniele+Nicola+Teresa)}

DD sul modello del Capitolo 3 + parte discreta

\textbf{Numerical treatment - Sec.3.2}
% brutto mettere gli stessi titoli?
% -> prima questione: trattarlo con uno schema iterativo simile robin robin algorithm of domain decomposition, in modo da accoppiare le soluzioni; quindi descrivere bene lo schema (dalla strong formulation); già partendo dalo schema semi implicit nel tempo
The communication between cells requires a proper iterative algorithm in order to deal with the mutual interplay between confining cells.

Every subdomain $\Omega_i$ of the pluricellular system $\Omega$ represents the single cell and the original system of equations in \eqref{eq:final} is solved in $\Omega_i$ for all $i = 1, ..., N$. We solve such systems by means of the semi-implicit method described in Section \ref{sec:SI method}. Let us consider the weak formulation restricted to $\Omega_i$, defining the functional space $V_i = \{ w_i \in \ H^1\left(\Omega_i\right)\}$, the finite element subspace $V_{i,h} \subset V_i $ and the time interval discretization used in Section \ref{sec:SI method}. In particular, we divide the time interval $\left[0, T_{max}\right]$ in $N_{max}$ time steps such that $t^n = n \Delta t$ with $\Delta t = T_{max} / N_{max}  $. We rewrite the full discretized formulation, identifying $u_{i,h}$ with $u_h|_{\Omega_i}$, as:

given the initial state $(u_{i,h}^0, v_{i,h}^0) $, find $(u_{i,h}^{n+1}, v_{i,h}^{n+1}) \ \in V_{i,h} \times V_{i,h}$ such that
\begin{equation} \label{eq:fullGalerkin}
\left\lbrace
\begin{matrix}
\begin{aligned}
  a_{i,u}(u_{i,h}^{n+1}, w_{i,h}) + b_{i,u}(v_{i,h}^{n+1}, w_{i,h}) + c_{i,u}(v_{i,h}^{n+1}, w_{i,h}) = f_{i,u}(w_{i,h}) \ \forall \ w_{i,h} \ \in V_{i,h} \\[6pt]
 a_{i,v}(v_{i,h}^{n+1}, w_{i,h}) + b_{i,v}(u_{i,h}^{n+1}, w_{i,h}) + c_{i,v}(v_{i,h}^{n+1}, w_{i,h}) = f_{i,v}(w_{i,h}) \ \forall \ w_{i,h} \ \in V_{i,h},
\end{aligned}
\end{matrix}
\right.
\end{equation}
$\forall n = 0, ... N_{max}$, where
\begin{subequations} \label{eq:Gvarfmono}
\begin{align}
    a_{i,u}(u_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left( \frac{1}{\Delta t} u_{i,h}^{n+1} w_{i,h} + \Tilde{D}_1 \nabla_s u_{i,h}^{n+1} \cdot w_{i,h} - \Tilde{a}_1 u_i^{n+1} w_{i,h} \right) \label{Gmono:au}\\ - & \int_{\partial \Omega_i}\left(\Tilde{D}_1 \nabla_s u_{i,h}^{n+1} \cdot \mathbf{n} w_{i,h}\right)  \nonumber\\
    b_{i,u}(v_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left(- \Tilde{b}_1 v_{i,h} w_{i,h} \right)  \label{Gmono:bu} \\
    c_{i,u}(v_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left(- \Tilde{c}_1 (u_{i,h}^{n})^2  v_{i,h}^{n+1} w_{i,h} \right) \label{Gmono:cu} \\[6pt]
    a_{i,v}(v_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left(\frac{1}{\Delta t} v_{i,h}^{n+1} w_{i,h} + \Tilde{D}_2 \nabla_s v_{i,h}^{n+1} \cdot w_{i,h} - \Tilde{a}_2 v_i^{n+1} w_{i,h} \right) \label{Gmono:av} \\ - &  \int_{\partial \Omega_i} \left( \Tilde{D}_1 \nabla_s u_{i,h}^{n+1} \cdot \mathbf{n} w_{i,h} \right) \nonumber\\
    b_{i,v}(v_{i,h}^{n+1}, w_{i,h}) = &\int_{\Omega_i} \left( - \Tilde{b}_2 u_{i,h} w_{i,h} \right) \label{Gmono:bv}\\
    c_{i,v}(v_{i,h}^{n+1}, w_{i,h}) =& \int_{\Omega_i} \left( - \Tilde{c}_2 (u_{i,h}^{n})^2  v_{i,h}^{n+1} w_{i,h} \right) \label{Gmono:cv}\\[6pt]
    f_{i,u}(w_{i,h}) = & \int_{\Omega_i} \left( \frac{1}{\Delta t} u_{i,h}^n \ w_{i,h} \right) \label{Gmono:fu}\\
    f_{i,v}(w_{i,h}) = & \int_{\Omega} \left( \frac{1}{\Delta t} v_{i,h}^n \ w_{i,h} + f_2 w_{i,h} \right). \label{Gmono:fv}
\end{align}
\end{subequations}

The introduction of different boundary conditions will require to modify the bilinear forms \eqref{Gmono:au} and \eqref{Gmono:av} and to add contributions in the right hand sides \eqref{Gmono:fu} and \eqref{Gmono:fv}.

To this aim, we synthetically rewrite the model problem \eqref{eq:fullGalerkin}, assuming generic boundary conditions, through a linear operator $\mathcal{L}$ in the following way:

Given the initial state $(u_i^0, v_i^0)$, find $(u_i^{n+1}, v_i^{n+1}) \ \in \Omega_i$ such that:
\begin{equation}\label{eq:modelpb}
% \begin{cases}
\mathcal{L}^n (u_i^{n+1}, v_i^{n+1}) = \mathbf{f}^n \ \text{in} \ \Omega_i
% \Tilde{D_1} \nabla_s u_i^{n+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_i \\
% \Tilde{D_2} \nabla_s v_i^{n+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_i
% \end{cases}
\end{equation}
$\forall n = 0, ... N_{max}$.

\textbf{A new iterative modeling algorithm - Sec 3.2.3 }
The model that we propose to make cells communicate can be regarded as a simplification of the classical domain decomposition scheme with Robin boundary conditions. We start for simplicity from a two cells problem and rewrite the common interface boundary conditions in \eqref{eq:RR} to recover the modelled open channels in \eqref{eq:pluriModel}. In the spirit of a block-Gauss-Seidel algorithm, we solve in sequence:
\begin{equation} \label{eq:RR_final}
\begin{aligned}
& \begin{cases}
\mathcal{L}^n (u_1^{k+1}, v_1^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_1 \\
\Tilde{D_1} \nabla_s u_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_1} \displaystyle{\partial u_1^{k+1}\over\partial \mathbf{n}} = \alpha_{uRR} u_2^{k} - \alpha_{uRR} u_1^{k+1} \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_2} \displaystyle{\partial v_1^{k+1}\over\partial \mathbf{n}} = \alpha_{vRR} v_2^{k} -\alpha_{vRR} v_1^{k+1} \ \text{on} \ \Gamma
\end{cases}
\\[6pt]
& \begin{cases}
\mathcal{L}^n (u_2^{k+1}, v_2^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_2 \\
\Tilde{D_1} \nabla_s u_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \setminus \Gamma \\
\Tilde{D_1} \displaystyle{\partial u_2^{k+1}\over\partial \mathbf{n}} = \alpha_{uRR} u_1^{k+1} - \alpha_{uRR} u_2^{k+1} \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \setminus \Gamma\\
\Tilde{D_2} \displaystyle{\partial v_2^{k+1}\over\partial \mathbf{n}} = \alpha_{vRR} v_1^{k+1} -\alpha_{vRR} v_2^{k+1} \ \text{on} \ \Gamma.
\end{cases}
\end{aligned}\end{equation}
The flux imposed depends on the difference of the neighbouring solutions. As a consequence, we are imposing a not necessarily null Neumann boundary condition. Equation \eqref{eq:RR_final} defines a RR iterative method applied to two cells using proper parameters $\beta_{u/vRR}$ and $\alpha_{u/v RR}$ from the model formulated in Section \ref{sec:PluriMod}:

starting from $(u_2^{k = 0}, v_2^{k = 0}) = (u_2^n, v_2^n)$, find $(u_1^{k+1}, v_1^{k+1}) \ \in V_{1}$ and $(u_2^{k+1}, v_2^{k+1}) \ \in V_{2}$:
\begin{equation}\label{eq::RRmod}
\begin{aligned}
& \begin{cases}
\mathcal{L}^n (u_1^{k+1}, v_1^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_1 \\
\Tilde{D_1} \nabla_s u_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_1} \displaystyle{\partial u_1^{k+1}\over\partial \mathbf{n}} = \beta_{uRR} \alpha_{uRR} \left( u_2^{k} - u_1^{k+1} \right) \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_2} \displaystyle{\partial v_1^{k+1}\over\partial \mathbf{n}} = \beta_{vRR} \alpha_{vRR} \left( v_2^{k} - v_1^{k+1} \right) \ \text{on} \ \Gamma
\end{cases}
\\[6pt]
& \begin{cases}
\mathcal{L}^n (u_2^{k+1}, v_2^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_2 \\
\Tilde{D_1} \nabla_s u_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \\
\Tilde{D_1} \displaystyle{\partial u_2^{k+1}\over\partial \mathbf{n}} = \beta_{uRR} \alpha_{uRR} \left( u_1^{k+1} - u_2^{k+1} \right) \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \setminus \Gamma\\
\Tilde{D_2} \displaystyle{\partial v_2^{k+1}\over\partial \mathbf{n}} = \beta_{vRR} \alpha_{vRR} \left( v_1^{k+1} - v_2^{k+1} \right) \ \text{on} \ \Gamma.
\end{cases}
\end{aligned}\end{equation}
for $k \geq 0$ up to convergence.

We remark that in \eqref{eq::RRmod} the coefficients $\beta_{u/vRR}$ and $\alpha_{u/v RR}$ have physical meaning since they come from the model \eqref{eq:pluriModel}. This is in contrast with the model and method presented in Section \ref{sec:RRclassic}, where the Robin coefficients are arbitrary.

Let $V_{i,h}$ denote the finite dimensional subspace of $H^1\left(\Omega_i\right)$, wih $\Omega_i$ being the sub-domain of the pluricellular system $\Omega$ corresponding to cell. We find solutions $\left(u_{h}^{n+1}, v_{h}^{n+1}\right)|_{\Omega_i}$ identified with $\left(u_{i,h}, v_{i,h}\right) \ \in V_{i,h}$ for each time step $t^{n+1}$, solving up to convergence the iteration step, whose Galerkin formulation is:
\begin{equation}\begin{aligned}
    a_{i,u}^{RR}(u_{i,h}^{k+1}, w_{i,h}) + b_{i,u}(v_{i,h}^{k+1}, w_{i,h}) + c_{i,u}^n(v_{i,h}^{k+1}, w_{i,h}) = f_{i,u}^{RR}(w_{i,h}) \ \forall w_{i,h} \in V_{i,h} \\
    a_{i,v}^{RR}(v_{i,h}^{k+1}, w_{i,h}) + b_{i,v}(u_{i,h}^{k+1}, w_{i,h}) + c_{i,v}^n(v_{i,h}^{k+1}, w_{i,h}) = f_{i,v}^{RR}(w_{i,h}) \ \forall w_{i,h} \in V_{i,h}.
\end{aligned}\end{equation}

The bilinear forms used are equal to \eqref{eq:Gvarfmono} - \eqref{eq:au&avRR} for classic Robin-Robin algorithm. The only difference is in the right hand side \eqref{eq:fu&fvRR} in which has been neglected the weak normal derivative of the neighbour solutions, as follows:
\begin{align}
 f_{i,u}^{RR}(w_{i,h}) & = f_{i,u}(w_{i,h}) + \int_{\Gamma} \left(\beta_{uRR} \alpha_{uRR} \mathcal{I}_{i,j} u_{j,h}^{k} \mathcal{I}_{i,j}w_{j,h} \right), \\
f_{i,v}^{RR}(w_{i,h}) & = f_{i,v}(w_{i,h}) + \int_{\Gamma} \left(\beta_{vRR} \alpha_{vRR} \mathcal{I}_{i,j} v_{j,h}^{k} \mathcal{I}_{i,j} w_{j,h} \right).
\end{align}

Consequently, the algebraic formulation of the new iterative method used is formulated similarly as in \eqref{eq:LinSysRR}, with time-dependent block matrix that need to be reassembled at each time-step. The right-hand sides depend on the previous solution found for the neighbouring cells and their contributions need to be interpolated by means of a interpolation matrix as in Robin-Robin classic method. We here explicit the whole iterative method for a two cells composed system.

Starting from initial guess given by the previous time-step solution  $\begin{bmatrix} \mathbf{U}_2^{0} \\ \mathbf{V}_2^{0} \end{bmatrix} = \begin{bmatrix} \mathbf{U}_2^{n} \\ \mathbf{V}_2^{n} \end{bmatrix}$, solve problem for $i = 1$ to find $\begin{bmatrix} \mathbf{U}_1^{k+1} \\ \mathbf{V}_1^{k+1} \end{bmatrix}$:
\begin{equation*}
\begin{aligned}
 \begin{bmatrix}
    A_u^1 & B_u^1 + C_u^1\left(  \mathbf{U}_1^n\right) \\
    B_v^1 & A_v^1 + C_v^1\left(  \mathbf{U}_1^n\right)
    \end{bmatrix} \begin{bmatrix}
    \mathbf{U}_1^{k+1} \\ \mathbf{V}_1^{k+1} \end{bmatrix} = \begin{bmatrix} F^1_u \left(\mathbf{U}_2^k\right) \\ F^1_v \left(\mathbf{V}_2^k\right)
    \end{bmatrix}
        \end{aligned}
\end{equation*}
and then solve problem for $ i = 2$ to find $\begin{bmatrix} \mathbf{U}_2^{k+1} \\ \mathbf{V}_2^{k+1} \end{bmatrix}$:
\begin{equation*}
    \begin{aligned}
\begin{bmatrix}
    A_u^2 & B_u^2 + C_u^2\left(  \mathbf{U}_2^n\right) \\
    B_v^2 & A_v^2 + C_v^2\left(  \mathbf{U}_2^n\right)
    \end{bmatrix} \begin{bmatrix}
    \mathbf{U}_2^{k+1} \\ \mathbf{V}_2^{k+1} \end{bmatrix} = \begin{bmatrix} F^2_u \left(\mathbf{U}_1^k\right) \\ F^2_v \left(\mathbf{V}_2^k\right)
    \end{bmatrix}
\end{aligned}\end{equation*}
for $k \geq 0$ up to convergence.

Iterations end when the normalized residual of consecutive computed solutions is smaller than a proper tolerance or when a maximum number of iterations is performed and we update the new solution as:
 $$\begin{bmatrix} \mathbf{U}_1^{n+1} \\ \mathbf{V}_1^{n+1} \end{bmatrix} = \begin{bmatrix} \mathbf{U}_1^{k+1} \\ \mathbf{V}_1^{k+1} \end{bmatrix}, \ \ \ \begin{bmatrix} \mathbf{U}_2^{n+1} \\ \mathbf{V}_2^{n+1} \end{bmatrix} = \begin{bmatrix} \mathbf{U}_2^{k+1} \\ \mathbf{V}_2^{k+1} \end{bmatrix}$$

Matrices and vectors used are defined in the following way:
\begin{equation}
    \begin{aligned}
    & \left[ A_u^i\right]_{j,l} & = a_{i,u}^{RR}(\phi_l, \phi_j), \ \ \ \left[ A_v^i\right]_{j,l} & = a_{i,v}^{RR}(\phi_l, \phi_j) \\
    & \left[ B_u^i\right]_{j,l} & = b_{i,u}(\phi_l, \phi_j), \ \ \ \left[ B_v^i\right]_{j,l} & = b_{i,v}(\phi_l, \phi_j)\\
    & \left[ C_u^i\right]_{j,l} & = c_{i,u}(\phi_l, \phi_j),\ \ \ \left[ C_v^i\right]_{j,l} & = c_{i,v}(\phi_l, \phi_j)\\
    & \left[F_u^i\right]_{j} & = f_{i,u}^{RR}(\phi_j), \ \ \
    \left[F_v^i\right]_{j} & = f_{i,v}^{RR}(\phi_j),
    \end{aligned}
\end{equation}
being $\{\phi_l\}_{l = 1}^{N_h}$ the functional basis of $V_{i,h}$ finite dimensional space defined on each cell $\Omega_i$ with $i =1,2$.

A sketch of the procedure to be adopted to deal with a generic N cells pluricellular system using Robin-Robin modifed algorithm is schematically given in Algorithm \ref{alg:RRmod}; $r_i$ are different coefficients characterizing initial state of concentrations, necessary for having flux between communicating cells. For physical reasons, we choose same initial guesses in the direction of the auxin gradient.

We have implemented a solver for a system of four cells.

As expressed in \eqref{eq::RRmod}, the iterative procedure is formulated such that the pluricellular domain is solved sequentially, in the sense that the boundary conditions characterizing sub-domain $i$, depending on sub-domain solutions of $j \in \mathcal{N}_i$, are computed using the newly updated solutions. In view of a parallel implementation, the method can be reformulated such that the new boundary conditions are a function of the previous iteration solution.

\begin{algorithm}[t]
    \caption{Pluricellular system solver procedure: RR}
    \label{alg:RRmod}
    Given $N \geq 1$ cells, $r_i$
    \begin{algorithmic}[1]
    \STATE Initialization: $\forall i = 1, ..., N$
    \STATE \verb|[U0i, V0i]| $\gets [r_i u_0, r_i v_0]$
    \STATE \verb|[Uiprec, Viprec]| $\gets$  \verb|[U0i, V0i]|
    \WHILE{$t < T_{max}$}
    \STATE{\verb|assemble| matrix for $\forall i = 1,..., N$}
    \FOR{$iter < Niter$}
    \STATE{$\forall i =1, ..., N$}
    \STATE{compute BC contribute from $j \in \mathcal{N}_i$}
    \STATE{\verb|interpolate| on $i$}
    \STATE{update \verb|rhs|}
    \STATE{\verb|solve| $\Omega_i$ problem \eqref{eq:pluriModel}}
    \STATE{update residual, check tolerance, update $iter$}
    \STATE \verb|[Uiprec, Viprec]| $\gets$  \verb|[Ui, Vi]|
    \ENDFOR
    \STATE \verb|[U0i, V0i]| $\gets$  \verb|[Ui, Vi]|
    \ENDWHILE
    \end{algorithmic}
\end{algorithm}

\subsection{Convergence check (Simona+Nicola+Teresa+Daniele)}
\subsection{Reliability (Simona+Nicola+Teresa+Daniele)}
\subsection{Eventuali sensitivity (Simona+Nicola+Teresa+Daniele)}

\section{DD per il modello completo (Simona+Nicola+Teresa+Daniele)}

\tb{si intende solo i risultati o anche la discretizzazione?}

\section{Conclusions and future developments (Simona+Daniele)}

\textbf{Conclusions and future developments}%
In this thesis we developed a multi-cellular model accounting for a spatially-extended intra-cellular system for ROPs pattern formation. In the proposed framework, we solved ROPs pattern formation in a system composed by multiple cells, together with a transport model for hormone auxin. The RD model explains the auxin-mediated action of ROPs in an Arabidopsis root hair cell leading to formation of the localized patches of activated ROPs.

Our simulations support conclusions reached in other works regarding ROPs dynamics under a-priori defined auxin distribution \cite{phdthesis:victor, intra1_R, intra2}. We have analyzed various scenarios such that a stripe-like patch forms where auxin concentration is higher. Then, instability of stripe into spot-like states occurs and multiple spots align with auxin gradient or travel towards auxin minimum. Several results confirm that, for a transversally independent gradient, lateral stripes become unstable states.

Successively as a new contribution, we extended the intra-cellular dynamics for root-hair initiation model to a multi-cellular system, developing a new model taking into account communication between cells. In order to do so, we have defined a boundary value problem which assumes new boundary conditions between neighboring cells. Subsequently, we develop an iterative procedure to solve it. We take as reference scheme a Robin-Robin domain decomposition method. We impose fluxes of ROPs betweeen neighboring cells, depending on the difference of the ROPs concentrations, through localised open channels. Such connections are tuned in order to visualize considerably different results with respect to a configuration characterized by stagnant cells. In addition,we aim at preserving all previous analyses over important parameters characterizing the system. We numerically assessed the robustness of the proposed model in cooperating with auxin distribution in influencing ROPs pattern formation.

Having defined a reliable multi-cellular model, we have taken into account also auxin concentration dynamics. Hormone auxin is regulated by carriers PIN following a non-linear ODEs system. We implement a semi-implicit method to solve such as a system on a two cells setting. The simulations we carried out show oscillating values of auxin concentration for specific sets of parameters, as expected from previous studies. A further confirmation of the robustness of pattern formation according to the proposed new multi-cellular model, when considering channel communication between cells, is given in Chapter \ref{cap:4}. It is shown that even if the system is under steady homogeneous auxin concentrations, the model robustly forms hotspots when considering open channels under a sufficiently high overall auxin level.

The results show two ways spots of active ROPs can be generated. The first factor is the auxin gradient, which still guarantees and influence stripe to spot evolution. The second factor is the structural coupling between cells. Even if not characterized by variation in space but with a sufficiently high value of the auxin concentrations, the multi-cellular model leads the system to multiple spots.

In conclusion, this thesis provides a first attempt in modeling communication between root-hair cells in pattern formation. Future developments include to study the structural model or to deeply analyze channels characterization. Moreover, the iterative procedure implemented could be applied to a multi-cellular system composed by more than four cells and eventually try to increase the computational performance through parallel computing. Another possible road to follow is to approach the problem through homogenization techniques. The cited method may increase the efficiency in solving the model, particularly when the number of cells involved becomes large. Finally, one could assume other auxin transport models, less simplified or accounting for spatial dependence inside the cells.

\bibliographystyle{siamplain}
\bibliography{references}
\end{document}


\subsection{Gravitational Body Force Examples}
\label{sec:example:3dhex8:gravity}

PyLith features discussed in this example:
\begin{itemize}
\item Gravitational body forces
\item Initial stresses
\item Finite strain
\item Generalized Maxwell linear viscoelastic material
\end{itemize}

\subsubsection{Overview}

This set of examples describes a set of problems for PyLith involving
gravitational body forces. All of the examples are quasi-static and
run for a time period of 200 years. These examples also demonstrate
the use of a generalized Maxwell viscoelastic material, which is used
for the lower crust in all examples. The final example (step17)
demonstrates the usage of a finite strain formulation, which
automatically invokes the nonlinear solver. All of the examples are
contained in the directory \filename{examples/3d/hex8}, and the
corresponding \filename{cfg} files are \filename{step15.cfg},
\filename{step16.cfg}, and \filename{step17.cfg}.  Run the examples as follows:
\begin{shell}
# Step15
$ pylith step15.cfg

# Step16
$ pylith step16.cfg

# Step17
$ pylith step17.cfg
\end{shell}
This will cause PyLith to read the default parameters in
\filename{pylithapp.cfg}, and then override or augment them with the
additional parameters in the \filename{stepXX.cfg} file. Each
\filename{cfg} file is extensively documented, to provide detailed
information on the various parameters.

\subsubsection{Step15 - Gravitational Body Forces}

The \filename{step15.cfg} file defines a problem with extremely simple
Dirichlet boundary conditions. On the positive and negative x-faces,
the positive and negative y-faces, and the negative z-face, the
displacements normal to the face are set to zero. Because all of the
materials in the example have the same density, the elastic solution
for loading via gravitational body forces is
\begin{equation}
\sigma_{zz}=\rho gh;\:\sigma_{xx}=\sigma_{yy}=\frac{\nu\rho gh}{1-\nu}\:.\label{eq:1-1}
\end{equation}

We set the gravity field, which by default has values of 9.80655
$\unitfrac{m}{s^{2}}$ for acceleration and $\left[0,0,-1\right]$
for direction and time stepping implementation:
\begin{cfg}[Excerpt from \filename{Step15.cfg}]
<h>[pylithapp.timedependent]</h>
<f>gravity_field</f> = spatialdata.spatialdb.GravityField ; Set gravity field

<h>[pylithapp.timedependent.implicit]</h>
# Change time stepping algorithm from uniform time step, to adaptive
# time stepping.
<f>time_step</f> = pylith.problems.TimeStepAdapt

# Change the total simulation time to 200 years, and set the maximum time
# step size to 10 years.
<h>[pylithapp.timedependent.implicit.time_step]</h>

total_time
 = 200.0*year

max_dt
 = 10.0*year

stability_factor
 = 1.0 ; use time step equal to stable value from materials
\end{cfg}

We use a generalized Maxwell model for the lower crust (see Section
\vref{sec:materials:formulation:generalized:Maxwell}), and use a \object{SimpleDB} to
provide the properties. We also request the relevant properties and
state variables for output:
\begin{cfg}[Excerpt from \filename{Step15.cfg}]
# Change material type of lower crust to generalized Maxwell viscoelastic.
<h>[pylithapp.timedependent]</h>
<f>materials.lower_crust</f> = pylith.materials.GenMaxwellIsotropic3D
# Provide a spatial database from which to obtain property values.
# Since there are additional properties and state variables for the
# generalized Maxwell model, we explicitly request that they be output.
# Properties are named in cell\_info\_fields and state variables are named in
# cell\_data\_fields.
<h>[pylithapp.timedependent.materials.lower_crust]</h>

db_properties.iohandler.filename
 = spatialdb/mat\_genmaxwell.spatialdb

output.cell_info_fields
 = [density, mu, lambda, shear_ratio, maxwell_time]

output.cell_data_fields
 = [total_strain, stress, viscous_strain_1, viscous_strain_2, \\
  viscous_strain_3]
\end{cfg}
The boundary conditions for this example are trivial, so we are able
to use the default \object{ZeroDispDB} for all faces. When we have
run the simulation, the output VTK files will be contained in \filename{examples/3d/hex8/output}
(all with a prefix of \filename{step15}). Results using ParaView are
shown in Figure \vref{fig:example:3dhex8:step15:displacement}.

\begin{figure}
  \includegraphics[width=10cm]{examples/figs/3dhex8_step15-displ-t200}
  \caption{Displacement field for example step15 at t = 200 years visualized
    using ParaView. The z-component of the displacement field is shown
    with the color contours, and the vectors show the computed displacements.}
  \label{fig:example:3dhex8:step15:displacement}
\end{figure}

\subsubsection{Step16 - Gravitational Body Forces with Initial Stresses}

The \filename{step16.cfg} file defines a problem that is identical to
example step15, except that initial stresses are used to prevent the
initial large displacements due to 'turning on' gravity. Since all
normal stress components are given an initial stress of $\rho gh$,
the initial stress state is lithostatic, which is an appropriate condition
for many tectonic problems in the absence of tectonic stresses (e.g.,
McGarr \cite{McGarr:1988}). When compared to example step15, this
example should maintain a lithostatic state of stress for the entire
simulation, and displacements should remain essentially zero.

We set the gravity field, as in example step15, and we again use adaptive
time stepping with a generalized Maxwell rheology for the lower crust.
We provide values for the initial stress for both the upper and lower
crust. Since the materials have the same density, we are able to use
the same \object{SimpleDB} with a linear variation for both (see file
\filename{examples/3d/hex8/spatialdb/initial\_stress.spatialdb}):
\begin{cfg}[Excerpt from \filename{Step16.cfg}]
# We must specify initial stresses for each material.
# We provide a filename for the spatial database that gives the stresses,
# and we change the query_type from the default 'nearest' to 'linear'.
<h>[pylithapp.timedependent.materials.upper_crust]</h>
<f>db_initial_stress</f> = spatialdata.spatialdb.SimpleDB

db_initial_stress.iohandler.filename
 = spatialdb/initial_stress.spatialdb

db_initial_stress.query_type
 = linear

<h>[pylithapp.timedependent.materials.lower_crust]</h>
<f>db_initial_stress</f> = spatialdata.spatialdb.SimpleDB

db_initial_stress.iohandler.filename
 = spatialdb/initial_stress.spatialdb

db_initial_stress.query_type
 = linear
\end{cfg}
Note that we use a \texttt{linear} \property{query\_type} rather than
the default type of \texttt{nearest}, so that a linear interpolation
is performed along the z-direction. When we have run the simulation,
the output VTK files will be contained in \filename{examples/3d/hex8/output}
(all with a prefix of \filename{step16}). Results using ParaView are
shown in Figure \vref{fig:example:3dhex8:step16:stress}.

\begin{figure}
  \includegraphics[width=10cm]{examples/figs/3dhex8_step16-stress_xx-t200}
  \caption{Stress field (xx-component) for example step16 at t = 200 years visualized
    using ParaView. Note that for this example, Stress\_xx = Stress\_yy
    = Stress\_zz, and there is no vertical displacement throughout the
    simulation. Also note that the stresses appear as four layers since
    we have used \object{CellFilterAvg} for material output.}
  \label{fig:example:3dhex8:step16:stress}
\end{figure}

\subsubsection{Step17 - Gravitational Body Forces with Small Strain}

The \filename{step17.cfg} file defines a problem that is identical to
example step15, except that we now use a small strain formulation
(see Section \vref{sec:small:strain:formulation}). All of the problems
up to this point have assumed infinitesimal strain, meaning that the
change in shape of the domain during deformation is not taken into
account. In many problems it is important to consider the change in
shape of the domain. This is particularly important in many problems
involving gravitational body forces, since a change in shape of the
domain results in a different stress field. By examining the stress
and deformation fields for this example in comparison with those of
example step15, we can see what effect the infinitesimal strain approximation
has on our solution.

We set the gravity field, as in example step15 and again use adaptive
time stepping withs a generalized Maxwell rheology for the lower crust.
The only change is that we change the problem formulation from the
default \object{Implicit} to \object{ImplicitLgDeform}. Since the
large deformation formulation is nonlinear, PyLith automatically switches
the solver from the default \object{SolverLinear} to \object{SolverNonlinear}.
It is thus only necessary to change the formulation:
\begin{cfg}[Excerpt from \filename{Step17.cfg}]
<h>[pylithapp.timedependent]</h>
# Set the formulation for finite strain. The default solver will
# automatically be switched to the nonlinear solver.
<f>formulation</f> = pylith.problems.ImplicitLgDeform
\end{cfg}
When we have run the simulation, the output VTK files will be contained
in \filename{examples/3d/hex8/output} (all with a prefix of \filename{step17}).
Results using ParaView are shown in Figure \vref{fig:example:3dhex8:step17:displacement}.

\begin{figure}
  \includegraphics[width=10cm]{examples/figs/3dhex8_step17-displ-t200}
  \caption{Displacement field for example step17 at t = 200 years visualized
    using ParaView. The z-component of the displacement field is shown
    with the color contours, and the vectors show the computed displacements.
    Note the larger displacements compared with example step15.}
  \label{fig:example:3dhex8:step17:displacement}
\end{figure}


\documentclass[notheorems]{beamer}

\usepackage[english]{babel}

\usepackage{amsmath}
\usepackage{amssymb}

% Typesetting evaluation rules (http://cristal.inria.fr/~remy/latex/)
\usepackage{mathpartir}

\usepackage[T1]{fontenc}
\usepackage{ae,aecompl}

\usepackage{listings}

\usepackage{beamerthemesplit}

\setbeamertemplate{background canvas}[vertical shading][bottom=red!10,top=blue!10]
\setbeamertemplate{navigation symbols}{}
\setbeamertemplate{headline}{}
\usetheme{Warsaw}
\useinnertheme{rectangles}

\colorlet{darkred}{red!80!black}
\colorlet{darkblue}{blue!80!black}
\colorlet{darkgreen}{green!80!black}

\lstdefinelanguage{Coq}{
  mathescape=true,
  texcl=false,
  keywords={Require, Declare},
  morekeywords={forall, exists, with, match, let, in, if, then, else},
  morekeywords={
    destruct, change, Variable, Inductive, CoInductive, Fixpoint, CoFixpoint, Definition,
    Lemma, Theorem, Proof, Axiom, Local, Save, Grammar, Syntax, Intro, Trivial,
    Qed, Intros, Symmetry, Simpl, Rewrite, Apply, Elim, Assumption, Left,
    Cut, Case, Auto, Unfold, Exact, Right},
  morekeywords={Section, Module, End},
  %emph={[1]Type, Set, nat, bool}, emphstyle={[1]\textit},
  comment=[s]{(*}{*)},
  showstringspaces=false,
}%[keywords,comments,strings]%

\lstset{
  numbers=none,
  basicstyle=\footnotesize\ttfamily,
  frame=tb,
  language=Coq,
  captionpos=b,
  xleftmargin=0em,
  xrightmargin=0em,
  aboveskip=1em,
  belowskip=1em
}

\title{Names and Numbers in Binding}

\author{Martijn Vermaat}
\institute{mvermaat@cs.vu.nl\\
http://www.cs.vu.nl/\~{}mvermaat/}
\date{Literature Study\\
March 13, 2009}

\begin{document}

\frame{\titlepage}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents

}

\section{Mechanical Reasoning about Languages}

\frame{

  \frametitle{Mechanical Reasoning}

  Shift from on-paper reasoning to mechanical reasoning:

  \begin{itemize}
    \item History of on-paper proofs and ideas
    \item Informal mechanical implementations of ideas
    \item Add a scale increase and we have a gap
  \end{itemize}

  \uncover<2->{
    \begin{block}{Goal}
      Let's make rigorous mechanical reasoning possible.
    \end{block}
  }

}

\frame{

  \frametitle{Reasoning about Languages}

  \begin{block}{Reasoning about languages}
    Often not intrinsically hard, but cumbersome in a mechanical setting.
  \end{block}

  Why?
  \uncover<2->{
    \begin{itemize}
    \item Most languages have a notion of binding
    \item Bindings and bound variables are easy on paper, hard on a computer
    \end{itemize}
  }

}

\frame{

  \frametitle{Mechanical Reasoning about Languages}

  So we want to
  \begin{quote}
    Reason about terms with bindings in tools like Coq, in a way that is close
    to the on-paper way.
  \end{quote}

  \uncover<2->{
    We need a representation for binders and variables.
  }

}

\section{Representing Bindings}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents[currentsection]

}

\frame{

  \frametitle{Representing Bindings}

  Classical problems related to binders and variables:

  \begin{itemize}
    \item $\alpha$-equivalence
    \item $\alpha$-conversion (e.g. in substitution)
  \end{itemize}

  \uncover<2->{
    Let's look at some representations.
    \begin{block}{Running example}
      Substitution in untyped $\lambda$-calculus
    \end{block}
  }

}

\subsection{Named Variables}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents[currentsubsection]

}

\frame{

  \frametitle{Traditional Representation 1/2}

  Traditional representation with named variables:
  \begin{align*}
    M ::=             &\; x
    && \text{variable} \\
    \llap{\textbar\:} &\; \lambda x.M
    && \text{abstraction} \\
    \llap{\textbar\:} &\; M \; M
    && \text{application} \\
  \end{align*}
  Abstractions introduce names.

}

\frame{

  \frametitle{Traditional Representation 2/2}

  \begin{itemize}

    \item $\alpha$-equivalent terms are routinely identified

    \item Substitution $M[N/x]$:
      \begin{align*}
        x[N/x]              &= N\\
        y[N/x]              &= y                  && \text{$x \neq y$} \\
        (\lambda y.M')[N/x] &= \lambda y.M'[N/x]  && \text{$x \neq y$ and $y$ not free in $N$} \\ % TODO: fix layout
        (M_1 \; M_2)[N/x]   &= M_1[N/x] \; M_2[N/x]
      \end{align*}

    \item Now implement this
      % alpha-quivalence is very hard
      % side-conditions are declarative
      % (\x.M)[N/x] not even defined
      % use variable convention, or a more general substitution (next slide)

  \end{itemize}

}

\frame{

  \frametitle{Simple Substitution}

  Use $\alpha$-conversion to rename bound variables and define
  substituting $N$ for $x$ in $M$ inductively$^*$ on M:

  \begin{align*}
    x[N/x]              &= \text{$N$ if $x = y$, $y$ otherwise}\\
    (\lambda y.M')[N/x] &= \lambda z.M'[z/y][N/x]  && \text{$z$ not free in $N, M'$} \\ % TODO: fix layout
    (M_1 \; M_2)[N/x]   &= M_1[N/x] \; M_2[N/x]
  \end{align*}

  % Note that M'[z/y] is not a subterm of M

  \uncover<2->{
    Already difficult enough to read, but just what we would intuitively do.
    So on paper, we can get by with some handwaving.
  }

}

\begin{frame}[fragile]

  \frametitle{Named Variables in Coq}

  Term datatype:
  \begin{lstlisting}
Inductive term : Set :=
  | Var : name -> term
  | Abs : name -> term -> term
  | App : term -> term -> term.
  \end{lstlisting}

\end{frame}

\begin{frame}[fragile]

  \frametitle{Substitution in Coq}

  Simple substitution:
  \begin{lstlisting}
Fixpoint subst (t:term) (n:name) (t':term)
  {struct t'} : term :=
  match t' with
  | Var x =>
      if eq_name x n then t else t'
  | Abs x b =>
      let z := fresh_name
                 (n :: (free_vars t) ++ (free_vars b))
      in
      Abs z (subst t n (rename x z b))
  | App f a =>
      App (subst t n f) (subst t n a)
end.
  \end{lstlisting}
  \uncover<2->{
    \emph{But this is ill-defined.}
  }

\end{frame}

\begin{frame}[fragile]

  \frametitle{Recursion on Term Size in Coq 1/3}

  Using term size as a measure:
  \begin{lstlisting}
Fixpoint size (t:term) : nat :=
  match t with
  | Var _   => 0
  | Abs x b => S (size b)
  | App f a => 1 + (size f) + (size a)
end.

Lemma size_rename : forall (n n':name) (t:term),
  size (rename n n' t) = size t.
Proof.
unfold size.
unfold rename.
induction t;
  [ case (eq_name n0 n); intro; trivial  (* Var *)
  | congruence                           (* Abs *)
  | congruence ].                        (* App *)
Qed.
  \end{lstlisting}

\end{frame}

\begin{frame}[fragile]

  \frametitle{Recursion on Term Size in Coq 2/3}

  Substitution with term size as a recursion measure:
  \begin{lstlisting}
Function subst (t:term) (n:name) (t':term)
  {measure size t'} : term :=
  match t' with
  | Var x =>
      if eq_name x n then t else t'
  | Abs x b =>
      let z := fresh_name
                 (n :: (free_vars t) ++ (free_vars b))
      in
      Abs z (subst t n (rename x z b))
  | App f a =>
      App (subst t n f) (subst t n a)
end.
(* Leaves us with 3 obligations. *)
  \end{lstlisting}

\end{frame}

\begin{frame}[fragile]

  \frametitle{Recursion on Term Size in Coq 3/3}

  Proving termination of \texttt{subst}:
  \begin{lstlisting}
Proof.
intros.
rewrite size_rename.
auto.

intros.
unfold size.
inversion f; omega.

intros.
unfold size.
inversion a; omega.
Defined.
  \end{lstlisting}
  \uncover<2->{
    We really prefer structural recursion.
  }

\end{frame}

\frame{

  \frametitle{Simultaneous Substitution}

  Stoughton suggests the structurally recursive simultaneous substitution $M \sigma$:
  \begin{align*}
    x \sigma              &= \sigma x\\
    (\lambda x.M') \sigma &= \lambda y.(M' \; \sigma[y/x])  && \text{$y$ not free in $M', \sigma$}\\
    (M_1 \; M_2) \sigma   &= M_1 \sigma \; M_2 \sigma
  \end{align*}
  where
  \begin{equation*}
    \sigma[N/y] \; x =
    \begin{cases}
      N        & \text{if $x = y$,}\\
      \sigma x & \text{otherwise}
    \end{cases}
  \end{equation*}

  \uncover<2->{
    Substituting $N$ for $x$ in $M$ is now $M \; \iota [N/x]$ with $\iota$ the identity substitution.
  }

}

\begin{frame}[fragile]

  \frametitle{Simultaneous Substitution in Coq}

  \begin{lstlisting}
Fixpoint sim_subst (l:list (term*name)) (t:term)
  {struct t} : term :=
  match t with
  | Var x =>
      apply_subst l x
  | Abs x b =>
      let z := fresh_name
                  ((free_vars_sub l) ++ (free_vars b))
      in
      Abs z (sim_subst ((Var z, x)::l) b)
  | App f a =>
      App (sim_subst l f) (sim_subst l a)
end.

Definition subst' (t:term) (n:name) (t':term)
  : term := sim_subst ((t, n) :: nil) t'.
  \end{lstlisting}

\end{frame}

\subsection{de Bruijn Indices}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents[currentsubsection]

}

\frame{

  \frametitle{de Bruijn Indices 1/2}

  Represent variable occurrences by number of binders between itself and abstraction:
  \begin{align*}
    M ::=             &\; n
    && \text{variable} \\
    \llap{\textbar\:} &\; \lambda .M
    && \text{abstraction} \\
    \llap{\textbar\:} &\; M \; M
    && \text{application} \\
  \end{align*}

  \uncover<2->{
    \begin{itemize}
      \item Harder to read
      \item $\alpha$-equivalence is term equivalence
      \item Substitution is structurally recursive
      \item No renaming (but lifting)
      \item Mechanically less tedious
    \end{itemize}
  }

}

\begin{frame}

  \frametitle{de Bruijn Indices 2/2}

  Substitution can be defined as:
  \begin{align*}
    n[N/n]             &= N\\
    m[N/n]             &= m                    && \text{$m \neq n$} \\
    (\lambda .M')[N/n] &= \lambda .M'[\uparrow \! N / n\!+\!1]\\
    (M_1 \; M_2)[N/n]  &= M_1[N/n] \; M_2[N/n]
  \end{align*}
  where $\uparrow \! M$ is $M$ with all free variables incremented.

\end{frame}

\begin{frame}[fragile]

  \frametitle{de Bruijn Indices in Coq}

  Term datatype:
  \begin{lstlisting}
Inductive term : Set :=
  | Var : nat -> term
  | Abs : term -> term
  | App : term -> term -> term.
  \end{lstlisting}

\end{frame}

\begin{frame}[fragile]

  \frametitle{Substitution in Coq}

  Substitution:
  \begin{lstlisting}
Fixpoint lift (l:nat) (t:term) {struct t} : term :=
  match t with
  | Var n   => Var (if le_lt_dec l n then (S n)
                    else n)
  | Abs u   => Abs (lift (S l) u)
  | App u v => App (lift l u) (lift l v)
end.

Fixpoint subst (t:term) (n:nat) (t':term)
  {struct t'} : term :=
  match t' with
  | Var m   => if eq_nat_dec n m then t else t'
  | Abs u   => Abs (subst (lift 0 t) (S n) u)
  | App u v => App (subst t n u) (subst t n v)
end.
  \end{lstlisting}

\end{frame}

\subsection{Locally Nameless}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents[currentsubsection]

}

\begin{frame}

  \frametitle{Locally Nameless Representation 1/2}

  Combine names and numbers -- names for free variables, de Bruijn indices for bound variables:
  \begin{align*}
    M ::=             &\; x
    && \text{free variable} \\
    \llap{\textbar\:} &\; n
    && \text{bound variable} \\
    \llap{\textbar\:} &\; \lambda .M
    && \text{abstraction} \\
    \llap{\textbar\:} &\; M \; M
    && \text{application} \\
  \end{align*}

  \uncover<2->{
    \begin{itemize}
      \item $\alpha$-equivalence is term equivalence
      \item No renaming, no lifting (but freshening)
      \item Substitution is structurally recursive
    \end{itemize}
  }

\end{frame}

\begin{frame}

  \frametitle{Locally Nameless Representation 2/2}

  Two substitution operations:
  \begin{itemize}
  \item Substitute a term for a named variable
  \item Substitute a term for a de Bruijn index\\[2em]
  \end{itemize}

  \only<2>{
  Named variable substitution:
  \begin{align*}
    x[N/x]             &= N\\
    y[N/x]             &= y                 && \text{$x \neq y$} \\
    n[N/x]             &= n\\
    (\lambda .M')[N/x] &= \lambda .M'[N/x]\\
    (M_1 \; M_2)[N/x]  &= M_1[N/x] \; M_2[N/x]
  \end{align*}
  No renaming.
  }

  \only<3->{
  de Bruijn substitution:
  \begin{align*}
    x[N/n]             &= x\\
    n[N/n]             &= N\\
    m[N/n]             &= m                    && \text{$m \neq n$} \\
    (\lambda .M')[N/n] &= \lambda .M'[N / n\!+\!1]\\
    (M_1 \; M_2)[N/n]  &= M_1[N/n] \; M_2[N/n]
  \end{align*}
  No lifting.
  }

\end{frame}

\begin{frame}[fragile]

  \frametitle{Locally Nameless Representation in Coq}

  Term datatype:
  \begin{lstlisting}
Inductive term : Set :=
  | FreeVar  : name -> term
  | BoundVar : nat -> term
  | Abs      : term -> term
  | App      : term -> term -> term.
  \end{lstlisting}

\end{frame}

\begin{frame}[fragile]

  \frametitle{Named Variable Substitution in Coq}

  Substitute a term for a named variable:
  \begin{lstlisting}
Fixpoint subst (t:term) (x:name) (t':term)
  {struct t'} : term :=
  match t' with
  | FreeVar y  => if eq_name x y then t else t'
  | BoundVar n => t'
  | Abs b      => Abs (subst t x b)
  | App f a    => App (subst t x f) (subst t x a)
end.
  \end{lstlisting}

\end{frame}

\begin{frame}[fragile]

  \frametitle{de Bruijn Substitution in Coq}

  Substitute a term for a de Bruijn index:
  \begin{lstlisting}
Fixpoint subst (t:term) (n:nat) (t':term)
  {struct t'} : term :=
  match t' with
  | FreeVar x  => t'
  | BoundVar m => if eq_nat_dec m n then t else t'
  | Abs b      => Abs (subst t (S n) b)
  | App f a    => App (subst t n f) (subst t n a)
end.
  \end{lstlisting}

\end{frame}

\section{Implementations}

\subsection{The POPLmark Challenge}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents[currentsubsection]

}

\begin{frame}

  \frametitle{The POPLmark Challenge}

  Mechanized metatheory for the masses:
  \begin{quote}
    Your average POPL paper should include machine-checked proofs
  \end{quote}

  Set of benchmarks for measuring progress:
  \begin{itemize}
    \item Based on metatheory of System F$_\texttt{<\!:}$
    \item Binding issues are a central aspect
    \item 15 (partial) solutions
    \item Part 1a: transitivity of subtyping
  \end{itemize}

\end{frame}

\begin{frame}

  \frametitle{Syntax of System F$_\texttt{<\!:}$}

  Part 1a considers just the type language of F$_\texttt{<\!:}$.

  \begin{align*}
    T ::=             &\; X
    && \text{type variable} \\
    \llap{\textbar\:} &\; \texttt{Top}
    && \text{maximum type} \\
    \llap{\textbar\:} &\; T \rightarrow T
    && \text{type of functions} \\
    \llap{\textbar\:} &\; \forall X\texttt{\small<\!:}T.T
    && \text{universal type} \\[2em]
    \Gamma ::=        &\; \emptyset
    && \text{empty type environment} \\
    \llap{\textbar\:} &\; \Gamma, X \texttt{\small<\!:}T
    && \text{type variable binding} \\
  \end{align*}

\end{frame}

\begin{frame}

  \frametitle{Subtyping Rules of System F$_\texttt{<\!:}$}

  \begin{mathpar}
    \infer*[right=SA-Top]
           {\:}
           {\Gamma \vdash S \texttt{\large<\!:} \texttt{Top}}
    \and
    \infer*[right=SA-Refl-TVar]
           {\:}
           {\Gamma \vdash X \texttt{\large<\!:} X}
    \\
    \infer*[right=SA-Trans-TVar]
           {X \texttt{\small<\!:} U \in \Gamma
             \\ \Gamma \vdash U \texttt{\large<\!:} T}
           {\Gamma \vdash X \texttt{\large<\!:} T}
    \\
    \infer*[right=SA-Arrow]
           {\Gamma \vdash T_1 \texttt{\large<\!:} S_1
             \\ \Gamma \vdash S_2 \texttt{\large<\!:} T_2}
           {\Gamma \vdash S_1 \rightarrow S_2 \; \texttt{\large<\!:} \; T_1 \rightarrow T_2}
    \\
    \infer*[right=SA-All]
           {\Gamma \vdash T_1 \texttt{\large<\!:} S_1
             \\ \Gamma, X \texttt{\small<\!:} T_1 \vdash S_2 \texttt{\large<\!:} T_2}
           {\Gamma \vdash \forall X \texttt{\small<\!:} S_1.S_2 \; \texttt{\large<\!:} \; \forall X \texttt{\small<\!:} T_1.T_2}
  \end{mathpar}

  POPLmark 1a: transitivity of $\texttt{<\!:}$

\end{frame}

\begin{frame}

  \frametitle{Solutions in Coq}

  Named variables:
  \begin{itemize}
    \item \only<1>{Stump} \uncover<2->{{\bf Stump} (7641)}
  \end{itemize}

  de Bruijn indices:
  \begin{itemize}
    \item \only<1>{Vouillon} \uncover<2->{{\bf Vouillon} (5443)}
    \item Sallinens \uncover<2->{(unavailable)}
    \item Charg\'eraud \uncover<2->{(3727)}
  \end{itemize}

  Locally nameless:
  \begin{itemize}
    \item \only<1>{Leroy} \uncover<2->{{\bf Leroy} (1081+5414=6495)}
    \item Chlipala \uncover<2->{(2650+2400=5050)}
    \item Chargu\'eraud \uncover<2->{(803+3533=4336)}
  \end{itemize}

  Nested datatypes:
  \begin{itemize}
    \item Hirschowitz and Maggesi \uncover<2->{(2757)}
  \end{itemize}

\end{frame}

\begin{frame}

  \frametitle{Stump -- Named Variables}

  Two main techniques to avoid difficulties with named variables:
  \begin{enumerate}
    \item Free and bound variables are disjoint
      \begin{itemize}
        \item Substitution is just grafting
      \end{itemize}
    \item \uncover<2->{Use common bound variable in the bodies of the SA-All rule
      \begin{itemize}
        \item Avoid $\alpha$-equivalence issues
        \item Original rule:
          \begin{mathpar}
            \infer*[right=SA-All]
                   {\Gamma \vdash T_1 \texttt{\large<\!:} S_1
                     \\ \Gamma, X \texttt{\small<\!:} T_1 \vdash S_2 \texttt{\large<\!:} T_2}
                   {\Gamma \vdash \forall X \texttt{\small<\!:} S_1.S_2 \; \texttt{\large<\!:} \; \forall X \texttt{\small<\!:} T_1.T_2}
          \end{mathpar}
        \item \uncover<3->{Adapted rule:
          \begin{mathpar}
            \infer*[right=SA-All]
                   {\Gamma \vdash T_1 \texttt{\large<\!:} S_1
                     \\ \Gamma, X \texttt{\small<\!:} T_1 \vdash S_2[X/X_1] \; \texttt{\large<\!:} \; T_2[X/X_2]}
                   {\Gamma \vdash \forall X_1 \texttt{\small<\!:} S_1.S_2 \; \texttt{\large<\!:} \; \forall X_2 \texttt{\small<\!:} T_1.T_2}
          \end{mathpar}}}
      \end{itemize}
  \end{enumerate}

\end{frame}

\begin{frame}

  \frametitle{Vouillon -- de Bruijn Indices}

  \begin{itemize}
    \item Very clear implementation, even suggested as baseline by POPLmark team
    \item A lot of code deals with shifting (but straightforward)
    \item Proofs not by structural induction, but by induction on the size of types
    \item Narrowing and transitivity are proved separately
    \item Unfortunately no accompanying paper
  \end{itemize}

\end{frame}

\begin{frame}

  \frametitle{Leroy -- Locally Nameless Representation}

  \begin{itemize}
   \item Two substitution operations, no renaming or lifting
   \item Considering abstraction bodies, freshening is needed
   \item Proofs not by structural induction, but by induction on the size of types
   \item A lot of code deals with swaps (used for equivariance proofs)
   \item Room for improvements, some implemented by Chargu\'eraud
     \begin{itemize}
       \item Useless case in de Bruijn substitution
       \item Treat well-formed typing environments as sets
       \item Cofinite quantification of free variable in SA-All
       \item Proofs by induction on well-formdness derivation instead of size
     \end{itemize}
  \end{itemize}

\end{frame}

\subsection{Engineering Formal Metatheory}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents[currentsubsection]

}

\begin{frame}

  \frametitle{Engineering Formal Metatheory}

  Aydemir et al, POPL'08: {\em Engineering Formal Metatheory}\\[2em]

  Complete style for formalizing language metatheory:
  \begin{itemize}
    \item Building on experience from POPLmark solutions
    \item Locally nameless representation
    \item Cofinite quantification of free variables in inductive definitions of relations on terms\\[2em]
  \end{itemize}

  Implemented in this style:
  \begin{itemize}
    \item Parts of POPLmark challenge
    \item Type soundness for core ML
    \item Subject reduction for Calculus of Constructions
    \item Several small developments
  \end{itemize}

\end{frame}

\begin{frame}

  \frametitle{LNgen}

  Aydemir and Weirich, this Wednesday:
  \begin{quote}
    LNgen: Tool Support for Locally Nameless Representations\\[2em]
  \end{quote}

  Building on {\em Engineering Formal Metatheory}:
  \begin{itemize}
    \item Takes Ott-like specifications
    \item Generates locally nameless infrastructure for Coq
  \end{itemize}

\end{frame}

\section{Conclusions}

\frame{

  \frametitle{Names and Numbers in Binding}

  \tableofcontents[currentsection]

}

\begin{frame}

  \frametitle{Related Work}

  Other Representations

  \begin{block}{Nominal Representation}
    \begin{itemize}
    \item Based on nominal logic (Pitts, Gabbay)
    \item Names for variables
    \item Swapping as primitive
    \item Urban in Isabelle/HOL
    \end{itemize}
  \end{block}

  \begin{block}{Higher Order Abstract Syntax}
    \begin{itemize}
    \item Meta-variables for variables
    \item Meta-functions for functions
    \item $\alpha$-equivalence for free
    \item No renaming needed
    \item Leads to quite unusual formulations
    \end{itemize}
  \end{block}

\end{frame}

\frame{

  \frametitle{Conclusion 1/2}

  Named variables:
  \begin{itemize}
    \item Closest to paper, but too much trouble mechanically\\[1em]
  \end{itemize}

  \uncover<2->{
    de Bruijn indices:
    \begin{itemize}
    \item Good candidate for mechanical developments
    \item Involves a lot of `easy' work
    \item Not so easy to read\\[1em]
    \end{itemize}
  }

  \uncover<3->{
    Locally nameless representation:
    \begin{itemize}
    \item Improves on de Bruijn
    \item Still quite some boilerplate code
    \item Tools like LNgen might make it feasible\\[1em]
    \end{itemize}
  }

  \uncover<4->{
    Nominal approaches are promissing
  }

}

\begin{frame}

  \frametitle{Conclusion 2/2}

  Of course, it also depends on your goal:
  \begin{itemize}
    \item A language implementation might not need to be close to its concrete syntax
    \item Metatheory for many users on the other hand probably does\\[1em]
  \end{itemize}

  \ldots and on your environment:
  \begin{itemize}
    \item We focussed on Coq
    \item There are other tools
  \end{itemize}

\end{frame}

\begin{frame}

  \frametitle{Questions and Further Reading}

  Questions?\\[4em]

  \begin{block}{Literature}
    \begin{itemize}
      \item de Bruijn, 1972: {\em $\lambda$-calculus with nameless dummies}
      \item Stoughton, 1988: {\em Substitution revisited}
      \item McBride and McKinna, 2004: {\em I am not a number -- I am a free variable}
      \item Aydemir et al, 2005: {\em The POPLmark challenge}
      \item Pollack, 2006: {\em Reasoning about languages with binding}
      \item Aydemir et al, 2008: {\em Engineering formal metatheory}
    \end{itemize}
  \end{block}

\end{frame}

\end{document}


\section{Discussion} \label{sec:discuss}
\subsection{Forward Model} \label{sec:forward-model}
In the previous section, we demonstrated the accuracy of \sedflow~posteriors.
Nevertheless, a primary determining factor for the fidelity of \sedflow, or any
ML model, is the quality of the training data set and, thus, the forward model
used to construct it.
Below, we discuss the caveats and limitations of our forward model, which has
two components: the PROVABGS SPS model and noise model
(Section~\ref{sec:training}).

First, for our noise model, we assign uncertainties to noiseless photometric
fluxes based on an empirical estimate of $p(\sigma_X\given f_X)$ for each band
independently.
This is a simplicistic prescription and, as the bottom right panels of
Figure~\ref{fig:data} ($g - \sigma_r$ and $r - \sigma_r$) reveal, there are
discrepancies between the magnitude - uncertainty distributions of the training
data and observations.
Despite these discrepancies, \sedflow~provides excellent estimates of the true
posterior.
This is because we design our ANPE to include $\sigma_X$ as a conditional
variable (Section~\ref{sec:anpe_train}).
The $f_X-\sigma_X$ distribution of our training data does not impact the
accuracy of the posteriors as long as there are sufficient training data near
$\bfi{x}$ to train the NDE in that region.

A more accurate noise model will, in theory, improve the performance of
\sedflow~because the $\bfi{x}$-space of the training data will more efficiently
span the observations.
Fewer training data would be expended in regions of $\bfi{x}$-space that are
devoid of observations.
However, for our application, we do not find significantly
improved performance when we alter the noise model.
This suggests that even with our simplistic noise model, the $\bfi{x}$-space of
observations is covered sufficiently well by the training data.
We note that when we decrease $N_{\rm train}$ to below 500,000,
\sedflow~posteriors are significantly less accurate.
A more realistic forward model may reduce this $N_{\rm train}$ threshold for
accurate posteriors.
However, generating $N_{\rm train}{\sim}1,000,000$ training SEDs has a
negligible computational cost compared to MCMC SED modeling, so we do not
consider it necessary to explore this further.

Next, we consider limitations in the PROVABGS SPS model used in our forward
model.
Our SPS model uses a compact and flexible prescription for SFH and ZH that can
describe a broad range of SFHs and ZHs.
However, the prescription is derived from simulated Illustris galaxies, whose
SFHs and ZHs may be not reflect the full range of SFHs and ZHs of real
galaxies.
If certain subpopulations of observed galaxies have SFHs and ZHs that cannot be
well described by the PROVABGS prescription, they cannot be accurately modeled.
Even if the PROVABGS SFH and ZH prescriptions are sufficient, there are
limitations in our understanding of stellar evolution.

There is currently no consensus in the stellar evolution, stellar spectral
libraries, or IMF of galaxies~\citep[\emph{e.g.}][]{treu2010, vandokkum2010,
rosani2018, ge2019, sonnenfeld2019}.
The PROVABGS model uses MIST isochrones, \cite{chabrier2003} IMF, and the MILES
+ BaSeL spectral libraries.
These choices limit the range of SEDs that can be produced by the training
data.
For instance, if galaxies have significant variations in their IMF, assuming a
fixed IMF would falsely limit the range of our training data.
A more flexible SED model that includes uncertainties in SPS would broaden the
range of galaxy SEDs that can be modeled.
Data-driven approaches may also enable SED models to be more
descriptive~\citep[\emph{e.g.}][]{hogg2016, portillo2020}.
Improving  SED models, however, is beyond the scope of this work.
Our focus is on improving the Bayesian inference framework.
In that regard, the limitations of the SED model equally impacts conventional
approaches with MCMC.

We encounter the caveats above when we apply \sedflow~to the NSA catalog.
For a small fraction of NSA galaxies (588 out of 33,884), \sedflow~generates
posteriors that are outside of the prior volume.
This is because the photometry or uncertainties of these galaxies lie outside
of the support of the training data and where \sedflow~is well trained.
They either have higher photometric uncertainties, for a given magnitude, or
bluer photometric colors than the training data.
Some of these may be observational artifacts or problematic photometry.
Nevertheless, \sedflow~fails because we cannot construct training data near
them with our limited noise and SPS models.
Since this only affects a small fraction of the NSA galaxies, we flag them in
our catalog and, for completeness, infer their galaxy properties by applying
PROVABGS with MCMC sampling.
For more details, we refer readers to Appendix~\ref{sec:fail}.

To test for limitations of the forward model, we can construct additional tests
of posteriors derived from ANPE.
For instance, the $\chi^2$ of the best-fit parameter value from the
estimated posterior can be used to assess whether the best-fit model
accurately reproduces observations.
This would only require one additional model evaluation per galaxy.
One can also construct an Amortized Neural Likelihood Estimator (ANLE) using
the same training data.
Unlike the ANPE, which estimates $p(\btheta\given f_X, \sigma_X, z)$, the ANLE
would estimate $p(f_X \given \btheta, \sigma_X, z)$.
We can then further validate the posteriors by assessing whether the observed
photometry lies within the ANLE distribution.
Based on the overall high level of accuracy of \sedflow~posteriors, we do not
explore these additional tests; however, they can be used to further validate
any ANPE posteriors.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Advantages of \sedflow}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The primary advantage of \sedflow~is its computational speed.
This becomes even more pertinent if we want to add additional parameters to
address concerns about the choices in current SPS models, described above.
To relax these assumptions, SPS models would need to introduce additional
parameters that flexibly model these uncertainties~\citep{conroy2009,
conroy2010c}.
While the dimensionality of current SPS models already makes MCMC methods
computational infeasible, ANPE has been applied to higher dimensional
applications.
For instance, \cite{dax2021} constructed an accurate ANPE for a
15-dimensional model parameter and 128-dimensional conditional variable
spaces.
NDE is an actively developing field in ML and new methods are constantly
emerging~\citep[\eg][]{wu2020, dhariwal2021}.
Since ANPE can handle higher dimensionality, we can in the future include
additional parameters that model uncertainties in SPS.
This will not only improve our SED modeling, but also improve our understanding
of stellar evolution and the IMF.

In addition to enabling scalable SED modeling for the next generation galaxy
surveys, \sedflow~will also enable us to tackle other key challenges in SED
modeling.
For example, recent works have demonstrated that priors of SED models can
significantly impact the inferred galaxy properties~\citep{carnall2018,
leja2019, hahn2022}.
Even ``uniformative'' uniform priors on SED model parameters can impose
undesirable priors on derived galaxy properties such as $M_*$, SFR, SFH, or
ZH.
To avoid significant biases, galaxy studies must carefully select priors and
validate their results using multiple different choices.
With an MCMC approach, selecting a different prior means reevaluating every
posterior and repeating all the SED model evaluations in the MCMC sampling.
For an ANPE approach, the prior is set by the distribution of parameters in the
training data.
For a new prior, instead of reconstructing the training data, we can resample
it in such a way that the parameters follow the new prior.
Then, the ANPE model can be re-trained, re-validated on the test data, and
re-deployed on observations.
Each of these steps require substantially less computational resources than
generating a new set of training data or using MCMC methods.
Hence, the ANPE approach provides a way to efficiently vary the prior without
multiplying computational costs.