\chapter{Layer 3}

\section{Routed Network}

\subsection{Administrative Distance}

\begin{table}[]
    \centering
    \resizebox{\columnwidth}{!}{%
        \begin{tabular}{|l|l|}
            \hline
            \textbf{Routing Protocol} & \textbf{Administrative distance} \\ \hline
            Directly connected interface & 0 \\ \hline
            Static route out an interface & 1 \\ \hline
            Static route to next-hop address & 1 \\ \hline
            DMNR - Dynamic Mobile Network Routing & 3 \\ \hline
            EIGRP summary route & 5 \\ \hline
            External BGP & 20 \\ \hline
            Internal EIGRP & 90 \\ \hline
            IGRP & 100 \\ \hline
            OSPF & 110 \\ \hline
            IS-IS & 115 \\ \hline
            Routing Information Protocol (RIP) & 120 \\ \hline
            Exterior Gateway Protocol (EGP) & 140 \\ \hline
            On Demand Routing (ODR) & 160 \\ \hline
            External EIGRP & 170 \\ \hline
            Internal BGP & 200 \\ \hline
            Next Hop Resolution Protocol (NHRP) & 250 \\ \hline
            Floating Static Route (ex. DHCP-learned) & 254 \\ \hline
            Unknown (Others) & 255 \\ \hline
        \end{tabular}%
    }
    \caption{Cisco default administrative distances}
    \label{cisco-default-administrative-distances}
\end{table}

Always remember the following points for Cisco devices:\cite{wiki:Administrative_distance}
\begin{itemize}
    \item An administrative distance of 255 will cause the router to remove the route from the routing table and not use it.
    \item Since IOS 12.2, the administrative distance of a static route with an exit interface is 1. Prior to the release of 12.2 it was in fact 0.
    \item Only the interface itself has an administrative distance of 0, since a route cannot have a distance of less than 1.
    \item Directly connected routes have an administrative distance of 0.
\end{itemize}

\newpage

\section{OSPF}

\newpage

\section{IS-IS}

\newpage

\section{EIGRP}

\gls{eigrp} is Cisco's enhanced edition if \gls{igrp}. Dating back to 1993 and a
leg for Cisco over other vendors back in the early days of the Internet. (..
hmm. And remember Cisco's implementation of \gls{ospf} was known to be unstable
until the early 2000's.)

The change to \gls{eigrp} from \gls{igrp} was due to the support of classless
routing. (\gls{igrp} only supported classful routing of class A (/8), B (/16),
and C (/24) networks.)

Cisco converted \gls{eigrp} to an open standard back in 2013 with
\rfc{7868}.\cite{wiki:Enhanced_Interior_Gateway_Routing_Protocol}

\gls{eigrp} adds support for \gls{vlsm} and the \gls{dual} with improved routing
capabilities in comparison to \gls{igrp}. Overall \gls{eigrp} provides better
capabilities compared to it's predecessor.

\subsection[Math]{The Math behind}

\fig{math/eigrp-dual-long}{eigrp-dual-long}{\glspl{eigrp} \gls{dual} full
    formula}

By default $K_2$, and $K_4$ is set to zero. (The are user customizable!) And
$K_5$ is set to 0.

In effect the resulting shorter formula is this:

\fig{math/eigrp-dual-short}{eigrp-dual-short}{\glspl{eigrp} \gls{dual} short
    formula}

\subsection{How it actually works}

\gls{eigrp} does it routing on a \texttt{next-hop} basis. Meaning it only stores
information about a given routes next turn. And \textbf{not} about the
destination itself. (Like \gls{ospf} does)

\subsubsection{Defaults}

\gls{eigrp} runs on Cisco equipment with values of:
\begin{itemize}
    \item \itemhead[]{Administrative Distance (Defaults)}
    \begin{itemize}
        \item Internal: 90
        \item External: 180
        \item Summary: 5
    \end{itemize}
    \item \itemhead[]{Timers}
    \begin{itemize}
        \item Hello: 5s / 60s \footnote{The larger timer value applies at speeds <= T1 circuit bandwidth (a.k.a. lower-than-equal-to 1.544 Mbps)}
        \item Hold: 15s / 180s
    \end{itemize}
    \item \itemhead[]{Supported Protocols}
    \begin{itemize}
        \item \gls{ip}
        \item \gls{ipx}
        \item AppleTalk
    \end{itemize}
    \item \itemhead[]{Other}
    \begin{itemize}
        \item Type: Distance Vector
        \item Algorithm: \gls{dual}
        \item Transport: \gls{ip}/88
        \item Authentication: MD5
        \item Multicast \gls{ipv4}: 224.0.0.10
        \item Multicast \gls{ipv6}: <!-- some number -->
    \end{itemize}
\end{itemize}

\subsubsection{Tables}

\gls{eigrp} contains three tables for storing route information.

\begin{enumerate}
    \item \itemhead[]{Neighbor Table}
    \begin{itemize}
        \item \textit{Lists \textbf{all} directly connected neighbors}
        \item Next-Hop Router(s)
        \item Interface(s)
    \end{itemize}
    \item \itemhead[]{Topology Table}
    \begin{itemize}
        \item \textit{Lists \textbf{all} learned from \textbf{all} \gls{eigrp} neighbors}
        \item Destination
        \item Metric
    \end{itemize}
    \item \itemhead[]{Global Routing Table}
    \begin{itemize}
        \item \textit{Best routes from \gls{eigrp} topology tabel will be copied to the routing table}
    \end{itemize}
\end{enumerate}

\subsection{Stub Routing}

\gls{eigrp} features some different ways to configure routers for specific routing scenarios. \texttt{Stub} routing is one of them.

Stub routing effectively prevents the given stub site/router to be used for transit traffic.

If a router is configured as a \texttt{Stud Router}. It only carries information about \tsq{it}s locally connected routes and a default route back to the transit routers. \tsq{It}s also omitted from being queried about networks is does not advertise. {\small (I.e. The Routes Query Process)}s

\subsection{Updates Exchange}

\begin{enumerate}
    \item Routers always exchange full topology table information when neighbor-ship is established.
    \item Now refresh of data will be done. Unless topology updates occur.
    \item If the topology changes. A partial update about the specific prefix is sent to neighbors.\footnote{Changes includes metric components, link statuses, change in announced topology information.}
    \item If a neighbor-ship fail. Then a new adjacency is formed with a full topology exchange is done.
\end{enumerate}

Always remember \gls{eigrp} does split-horizon by default on all active links when exchanging topology updates.

Split-horizon is the \textit{famous} rule about not sending topology updates back to the router who sent the topology update in the first place.

\gls{eigrp} uses \gls{rtp} to send topology updates and confirmation receipts.

On \gls{p2p} interfaces. \gls{eigrp} simply send and ACK back to the sender.
On multi-access interface/segments. \gls{eigrp} sends updates to 224.0.0.10 and receivers reply with a unicast ACK message to the sender.

\subsection[SIA]{Stuck-in-Active}
\gls{eigrp} is known and feared for it being \texttt{Stuck-in-Active} mode when exchanging route updates between routers. The Stuck-in-Active state could cause problems on low end network gear with a low amount of resources available for the routing process. Which in cases could cause the network device to use all available resources when querying neighbor devices for updates.

\fig{eigrp/stuck-in-active}{stuck-in-active}{EIGRP Stuck in Active}

End result is neighbor ships between \gls{eigrp} routers is terminated and re-established again with a fresh set of route updates going out.

Imagine a large \gls{eigrp} topology (3-digit number of routers) in a single domain and beginning with 1 neighbor-ship gets re-established. And then again and again and again. Can result in quickly several routers going Stuck-in-Active because \gls{eigrp} queries are not getting answered within a short time-frame when all routers needs to query \tsq{it}s own neighbors and their neighbors and their neighbors and so forth. Until their are no more neighbors to be queried up and down the hierarchical network topology.

\fig{network/routing}{network-topology}{Medium sized network topology}

\subsection{Non-Cisco Implementations}

This is always a particular point of discussion around \gls{eigrp}. Cisco has released \textit{some} information on how \gls{eigrp} works.
Not \textit{all} information.
And only as \textit{informational} RFCs.
This ensures Cisco \textit{still} has control of the development of the protocol.
Because they have retained ownership by only publishing informational RFCs.

This has been a hindrance for the adoption of \gls{eigrp} in other vendors than Cisco them-self.

Known adoptions of EIGRP includes:
\begin{itemize}
    \item \gls{frr} targeted release 4\cite{Frr30-4047:online}
    \item Quagga has known works for trying to implement \gls{eigrp}. Still not officially implemented, thou.
\end{itemize}

\subsection{Filtering}

\gls{eigrp} routing updates can of course be filtered in- and outgoing direction on Cisco boxes by using prefix-lists and applying the prefix lists under the \texttt{router eigrp x} process.

\newpage

\section{RIP}

rip is the old routing protocol still existing using nukber-of-hops as the metric to calculate which route to take to the intended destination. This approach can work fine but has a nukber of weaknesses.
1. Scales like shit.
2. Does not take into account if 2 different paths with the same amount of hops has different transfer capacity.
3. Has no concept of load sharing traffic between links if the same amount of hops is present amongst the best paths available.
4. Limited to a miximum of 16 hops between source and destination.

\newpage

\subsection{RIPng}

The version of rip supporting ipv6. different to the standard rip in the ng version is using different multicast addresses to communicate on.

\newpage

\section{Babel}

Babel is built on the principles of 1) \gls{dsdv}, 2) \gls{aodv}, and 3)
\gls{eigrp} protocols.
Made for hybrid networks\footnote{network contains wired and wireless links}
and can account for a high level of instability on wireless links.

Babel has been reported to be running stable in unstable wireless networks with
a level of reliability and fast convergence.

\begin{itemize}
    \item Loop-avoidance protocol,
    \item Uses distributed Bellman-Ford algorithm,
    \item \rfc{6126} in 45 pages, \textit{(28 are normative)}
    \begin{enumerate}
        \item Updates by \rfc{7298} {\scriptsize (The Babel extension
        mechanism)}, and
        \item \rfc{7557} {\scriptsize (Babel \gls{hmac} Cryptographic
        Authentication)}.
    \end{enumerate}
    \item Highly extensible protocol,\cite{BabelDoe86:online}
    \item Supports \gls{ip4} and \gls{ip6}.
\end{itemize}

Babel on \underline{wired} networks uses by default hop-count. Can be
configured to include several values when computing the metrics.
On \underline{wireless} networks Babel should be configured to take into
account factors such as link latency, packet loss, hop-count, and radio
diversity.

\subsection{Protocol support}

Currently the following projects include support for Babel:
\cite{Babel-al30:online}

\begin{itemize}
    \item Bird {\footnotesize \url{http://bird.network.cz/}}
    \item FRR {\footnotesize \url{https://frrouting.org/}}
    \item Pybabel {\footnotesize \url{https://github.com/fingon/pybabel/}}
    \item Sbabeld {\footnotesize \url{http://github.com/jech/sbabeld/}}
\end{itemize}

\newpage

\section{Static}

\newpage

\section{BGP}

\wikicommons{BGP_FSM}

The protocol of the internet used since 1994.\cite{wiki:Border_Gateway_Protocol}
Currently based upon \rfc{4271} with updates following in \rfc{6286} \rfc{6608}, \rfc{6793}, \rfc{7606}, \rfc{7607}, \rfc{7705}.

\subsection{Properties}

\begin{itemize}
    \item Uses tcp/179 as \gls{dst} port
    \item Sends keep-alive message every 1 minute
    \item Keep-alive message is 19 byte long
\end{itemize}

Be ware if sessions are terminated immediately upon trying to establish connection. Try debugging following points.

\begin{itemize}
    \item tcp/179 is not open,
    \item random port 1023> is not open,
    \item incorrect peer-ip,
    \item incorrect peer-as.
\end{itemize}

\subsection{Route exchange}

Exchanging routes between routers is a reliant and tolerant manner is \glspl{bgp} 1-advantage over \gls{ospf}/\gls{isis}/\gls{rip}/\gls{eigrp}.

The sheer tuning and control mechanisms \gls{bgp} can offer is simply astounding. Route-maps is the key and access-lists just one option.

\subsubsection[Route-maps]{Route-maps mechanism}

Route-maps is used to target a select set of routes and either modify/add/remove attributes attached to the select route-set.

\begin{itemize}
    \item Routes can be aggregated between \glspl{as}.
    \item Properties can be changed on the fly by matching
    \begin{enumerate}[label={\alph*)}]
        \item \Gls{bgp} communities,
        \item \Gls{ip} prefix,
        \item \Gls{bgp} as-path,
    \end{enumerate}
\end{itemize}

An simple example of using route-maps is

\begin{cisco}
ip prefix-list 1 permit 172.16.0.0/16
ip prefix-list 2 permit 192.168.1.0/24
!
route-map RED permit 10
 match ip address prefix-list 1
 set ip next hop 10.1.1.1
 continue 20              ! Continues to apply rules normally only
                          ! applied to prefix-list 2. To apply to
                          ! prefix-list 1, too.
                          ! Any attributes set in '20' will
                          ! override any set during '10'.
route-map RED permit 20
 match ip address prefix-list 2
 set ip next hop 10.2.2.2 ! Last rule overrides previous rules from
                          ! previous '10' rule-set.
\end{cisco}

When rules from a rule-set is chained together as shown above. The last rule will override all previous set values regarding the attribute being applied. In this case \texttt{next-hop} from 'permit 10' is overridden in 'permit 20'.

\subsection[States]{BGP States}

The states is the way \gls{bgp} handles peer/neighbor connection establishing. The \underline{playbook} so to speak.

\begin{enumerate}
    \item Idle: \gls{bgp} while initializing refuses all incoming connections. Will initiate \gls{tcp} connection to peer.
    \item Connect: Waits for \gls{tcp} connection. If \gls{tcp} is established goes to state OpenSent. If \gls{tcp} is \textit{un}successful ConnectRetry timer is started and then goes to Active state.
    \item Active: When ConnectRetry counter reaches 0 goes to state Connect.
    \item OpenSent: Sends \gls{msg} to remote node. Waits for reply \gls{msg} before going to OpenConfirm.
    \item OpenConfirm: Nodes exchange keepalive \glspl{msg} and goes to Established state if successful.
    \item Established: Nodes can now exchange KeepAlive, Updates, and Notification \glspl{msg}.
\end{enumerate}

\subsection[iBGP]{Internal Border Gateway Protocol}

\gls{ibgp} is running \gls{bgp} within the same \gls{as} between routers. Much like running a general \gls{igrp} in the network.

Tradition one has to be fearful of creating \textit{routing loops} in the network. \glspl{bgp} mechanism for this is using either \begin{mylist} \item Full Mesh, or \item \glspl{rr} \end{mylist}.

Problems by running \textit{Full Mesh} is the formula of \[ iBGPsessions = n*(n-1)/2 \] \note{where $ n $ is the number \gls{ibgp} speakers} which results in scaling problems as \gls{ibgp} speakers are added to the \gls{as}.

\textit{\glspl{rr}} solves this problem by peering with all \gls{ibgp} speakers in the \gls{as}. All \gls{ibgp} speakers are then clients of the \glspl{rr}. This in turn helps maintainability by also advertising routes learnt from \gls{ibgp} clients to clients. Classic filtering/mathing route-maps/prefix-filters can be used to \textit{not} advertise all routes select group of clients from the \glspl{rr}.

\subsection[eBGP]{External Border Gateway Protocol}

\gls{ebgp} connections is inherently different from \gls{ibgp} connections. Some assumptions are made such as
\begin{enumerate}
    \item a \gls{ttl} of 1 is the default\footnote{Multi-hop \gls{ebgp} can thou be configured and therefore increase the max-\gls{ttl} value},
    \item distance is set to 20 compared to 200 for \gls{ibgp} routes,
    \item Next hop does \textit{not} change for \gls{ebgp} routes advertised to \gls{ibgp} neighbours \textit{by-default}\footnote{Often times it is necessary to tell a router to set itself as the next-hop before advertising to \gls{ibgp} neighbours}.
\end{enumerate}


\chapter{Equation Input} \label{chap:equation}

The expressions to be evaluated are entered by the user as equations.
The syntax is very similar to \caps{FORTRAN} equation syntax. The first
item is the variable to be assigned, followed by an ``\cmd{=}'', then
the expression to be evaluated. The expression consists of constants,
variables, arithmetic operators, and functions.

The equations must adhere to the following syntax rules.
\setlength{\itemsep}{\medskipamount} \begin{itemize}
\item
Blanks are treated as delimiters, but are otherwise ignored.
\item
Either lowercase or uppercase letters are acceptable, but lowercase
letters are converted to uppercase.
\item
A ``\verb|'|'' character in any equation starts a comment. The
``\verb|'|'' and any characters following it on the line are ignored.
\item
An equation may be continued over several lines with a ``\verb|>|''
character. The ``\verb|>|'' and any characters following it on the
current line are ignored and the next line is appended to the current
line.
\end{itemize}

\section{The Assigned Variable} \label{assvar}

The assigned variable name must start with a letter and can be up to
eight alphanumeric characters (A--Z, 0--9) long. A name that is longer
than eight characters is truncated with a warning. Blanks cannot be
embedded in a variable name.

All assigned variables (except temporary variables specified by a
\cmd{DELETE} command) will be written to the output database. The input
database variables are not written to the output database unless they
are assigned in an equation (such as \cmd{X~=~X}) or transferred with a
\cmd{SAVE} command.

The type of the assigned variable depends on the expression. There are
three types of ``quantities'' in an expression that are related to the
variable types:
\setlength{\itemsep}{\medskipamount} \begin{itemize}
\item
Global quantities include global variables and nodal or element
variables for specific nodes or elements.
\item
Nodal quantities include nodal variables and nodal coordinates, unless
the value is limited to a specific node.
\item
Element quantities include element variables, unless the value is
limited to a specific element.
\end{itemize}
Global quantities are referred to as ``single-value''
quantities. Nodal and element quantities are referred to as ``arrays''.

Each part of an expression yields a result of a particular type. The
types of constants and variables are defined above. The type of an
arithmetic operation is dependent on the types of its operands. If both
operands are global quantities, the operation yields a global
quantity. If either operand is an array, the operation type is
the array type. Thus a nodal quantity and an element quantity cannot
appear in the same operation. For array operations, the operator is
applied to each array element. The type of a function is dependent on
the types of its parameters. The rules for operand types also apply to
all function parameters. One special type of function yields a global
quantity regardless of the parameter type.

The assigned variable can be reassigned, but it must be assigned to the
same variable type (global, nodal or element).

The equations are evaluated in order. The assigned variables are grouped
by variable type, but are otherwise output in the order they are first
assigned by the equations.

If there are no timesteps on the input database, a timestep will be
added to the output database. The equation defining new variable(s)
can access constants and coordinates.

\section{Restricting the Nodes and/or Elements}

Nodes and/or elements may be deleted from the input database with the
\cmd{ZOOM}, \cmd{VISIBLE}, \cmd{FILTER}, or \cmd{REMOVE} commands. An
input variable is defined for all input nodes and/or elements. An
output variable is only defined for the nodes and/or elements to be
output.

Element variables may be undefined for certain element blocks. This may
be further restricted with the \cmd{BLOCKS} command. If two or more
element variables are combined with an operator or are function
parameters, the resulting variable is only defined for an element block
if all the variables involved are defined for that block.

When an operation or function is performed on an array variable, it is
only performed for the defined nodes/elements. This is done to prevent
problems with numerical errors such as divide by zero for undefined
values.

\section{Constants}

Constants can be entered in any legal \caps{FORTRAN} numeric format
(e.g., 5, 5.4 or 5.4E3). All integers are converted to real numbers. If
the constant is signed, parenthesis should surround the sign and
constant.

\section{Variables}

The variables that may be found in the expression to be evaluated are:
\setlength{\itemsep}{\smallskipamount} \begin{itemize}
\item any input database global, nodal or element variable,
\item the values for any coordinate,
\item a reference to a specific nodal or element quantity,
\item the time associated with each time step, and
\item any previously assigned variable.
\end{itemize}

If an embedded blank is included in an input database variable or
coordinate name, the blank must be deleted in references to the
variable. For example, variable ``\cmd{SIG X}'' must be entered as
``\cmd{SIGX}''.

The coordinates may be referenced in the expression by name. They are
treated as an input database nodal variable whose value remains constant
in all ``whole'' time steps.

If the value for a specific node or element is desired, a ``\cmd{\$}''
and the node or element number is appended to the variable name. For
example, \cmd{SIGR\$40} refers to the value for the \nth{40}{th} element
of variable \cmd{SIGR}. A specifier may be appended to the name of any
nodal or element quantity in an expression, including coordinates and
previously assigned variables. References to specific nodes and/or
elements can only be made if the variable is defined at that node and/or
element.

The value of a variable in the previous time step is referenced by
appending a ``\cmd{:}'' to the variable name. The value in the first
time step is referenced by appending a ``\cmd{:1}'' to the variable
name. If time steps are selected, the previous and first time steps
refer to the selected time steps, not the input time steps.

The name ``\cmd{TIME}'' is reserved for the time associated with each
time step. The output database times are copied from the input database
unless a value is assigned to the variable \cmd{TIME}. \cmd{TIME} may also
appear in the expression, referring to the input or assigned database
time.

The equations are evaluated in order. References to a variable name in
the expression refer to the last assigned value, or to the input
variable if the name has not been assigned. For example, if input global
variable \cmd{CONST} has a value of 4 and the following equations are
executed,
\cenlinesbegin
\cmd{X = CONST} \\
\cmd{CONST = 2$*$CONST} \\
\cmd{Y = CONST}
\cenlinesend
the result is \cmd{X} equals 4, \cmd{CONST} equals 8, and \cmd{Y} equals
8.

\section{Operators}

The legal operations are addition ($+$), subtraction ($-$),
multiplication ($*$), division ($/$), and exponentiation ($**$). The
operands may be either single-value or array quantities as explained in
Section~\ref{assvar}.

\caps{FORTRAN} operator precedence rules apply (e.g., multiplication is
performed before addition). Parenthesis may be used to change the order
of evaluation.

Two operators cannot be placed in succession. To precede a value with a
sign, enclose the sign and value in parenthesis. For example,
\cenlinesbegin
\cmd{A = $-$5 $*$ $-$SIN(0.5)}
\cenlinesend
should be written as
\cenlinesbegin
\cmd{A = ($-$5) $*$ ($-$SIN(0.5))}
\cenlinesend
where the parenthesis around the \cmd{$-$5} are optional.

\section{Functions}

Many of the standard \caps{FORTRAN} functions and several special
functions are implemented in \caps{\PROGRAM}.
These functions are summarized in Appendix~\ref{appx:function}.
The parameters for any
function may be expressions and all parameters must be supplied. The
parameters may be either single-value or array quantities as explained
in Section~\ref{assvar}.

A function in an equation is distinguished from a variable name by the
``\cmd{(}'' which follows the function name. This allows the user to
assign variable names which are the same as the function names and to
reference input database variables with the same names as the functions.

\subsection*{\caps{FORTRAN} Functions}

The standard \caps{FORTRAN} functions implemented are: \cmd{AINT},
\cmd{ANINT}, \cmd{ABS}, \cmd{MOD}, \cmd{SIGN}, \cmd{DIM}, \cmd{MAX},
\cmd{MIN}, \cmd{SQRT}, \cmd{EXP}, \cmd{LOG}, \cmd{LOG10}, \cmd{SIN},
\cmd{COS}, \cmd{TAN}, \cmd{ASIN}, \cmd{ACOS}, \cmd{ATAN}, \cmd{ATAN2},
\cmd{SINH}, \cmd{COSH}, and \cmd{TANH}. The use and result of these
functions is the same as in \caps{FORTRAN}, and the same restrictions
apply.

\subsection*{Tensor Principal Values and Magnitude Functions}

Functions \cmd{PMAX} and \cmd{PMIN} calculate the maximum and minimum
principal values of a symmetric tensor. For example, to obtain the
maximum principal values for a tensor $T$,
\cenlinesbegin
\cmd{SMAX = PMAX ($T_{11}$, $T_{22}$, $T_{33}$, $T_{12}$, $T_{23}$,
$T_{31}$)}.
\cenlinesend
For a two-dimensional tensor or a tensor using cylindrical coordinates
for an axisymmetric solution, \cmd{PMAX2} and \cmd{PMIN2} may be used:
\cenlinesbegin
\cmd{SMAX = PMAX2 ($T_{11}$, $T_{22}$, $T_{12}$)}.
\cenlinesend

The function \cmd{TMAG} calculates the magnitude of the deviatoric part
of a symmetric tensor. To calculate the magnitude of tensor $T$,
\cenlinesbegin
\cmd{SMAG = TMAG ($T_{11}$, $T_{22}$, $T_{33}$, $T_{12}$, $T_{23}$,
$T_{31}$)}
\cenlinesend
where the following calculation is made:
\cenlinesbegin
\cmd{SMAG = }$\sqrt{(T_{11} - T_{22})^{2} + (T_{22} - T_{33})^{2} +
(T_{33} - T_{11})^{2} + 6 * (T_{12}^{2} + T_{23}^{2} + T_{31}^{2})}$.
\cenlinesend

To obtain the von Mises stress, the value supplied by function
\cmd{TMAG} is multiplied by the constant $1/\sqrt{2}$. To calculate
effective strain, multiply by the constant $\sqrt{2.0}/3.0$.

\subsection*{IF Functions}

The functions \cmd{IFLZ}, \cmd{IFEZ}, and \cmd{IFGZ} provide a simple
\mbox{if-then-else} capability. Each function expects three parameters:
a condition, a true result, and a false result. Function \cmd{IFLZ}
returns the true result if the condition evaluates to less than zero;
otherwise the function returns the false result. Function \cmd{IFEZ}
checks for equal to zero and \cmd{IFGZ} checks for greater than zero.
For example, the equation
\cenlinesbegin
\cmd{\param{x} = IFLZ (\param{cond}, \param{rtrue}, \param{rfalse})}
\cenlinesend
with global parameters \param{cond}, \param{rtrue}, and \param{rfalse}
could be implemented in \caps{FORTRAN} by
\cenlinesbegin
\cmd{IF (\param{cond} .LT. 0.0) THEN} \\
\hspace*{\myindent} \cmd{\param{x} = \param{rtrue}} \\
\cmd{ELSE} \\
\hspace*{\myindent} \cmd{\param{x} = \param{rfalse}} \\
\cmd{END IF}
\cenlinesend
All the parameters are evaluated before the function, so both the true
result and the false result are evaluated even though only one is
needed.

\subsection*{Array $\Rightarrow$ Global Variable Functions}

The functions \cmd{SUM}, \cmd{SMAX}, and \cmd{SMIN} perform a
calculation on a nodal or element array parameter which produces a
global result. \cmd{SUM} sums all the array values. \cmd{SMAX} and
\cmd{SMIN} return the maximum and minimum of all the array values.

Values for specific nodes and/or elements are only included in the
function calculation if the variable is defined at that node and/or
element.

\subsection*{Envelope Functions}

An ``envelope'' function performs a calculation that is cumulative for
all previous time steps. The function \cmd{ENVMAX} results in an array
(assuming the parameter is an array) that is the maximum of each array
value for all previous selected time steps and the current time step. On
the last time step, \cmd{ENVMAX} contains the maximum of each array
value for all selected time steps. \cmd{ENVMIN} is the corresponding
minimum function.


\chapter{Introduction}
In this paper the nonhomogenous Poisson process is used to generate data and conditional samples from sufficient statistics. The nonhomogenous Poisson process is a combination of the power law case and log-linear case. To generate the samples a Monte Carlo approach is used. The approach is a Gibbs sampler with a Metropolis-Hastings step.
\\
\\
The hypothesis test theory is used to check if the samples comes from the right distribution. This is the null hypothesis. To check the null hypothesis several condtional tests are used. The conditional tests are generated by simulation of conditional samples. These samples are conditional on minimum sufficent statistics.
\\
\\
The motivation for this study comes from the papers \cite{lindqvist2011monte} and \cite{lockhart2007use}. In the first paper two types of NHPP were studied. The first one was a case of log-linear rate function, and the second case was the log-linear case. The paper proposed to look at a NHPP where these cases were combined. The second paper is about conditional sampling from a gamma distribution\cite{stacy1962generalization}. In this paper the same sufficient statistics were used.
\\
\\
First the paper is going to introduce the nonhomogenous Poisson process. Further necessary theory and the sampling algorithm is introduced. At the end the results and analysis of the simulations are presented.


\title{Overlapping Group Lasso Via ADMM in Python}
\author{Daniel Kessler}

\documentclass{article}

% \usepackage{parskip}
\usepackage{geometry}
\usepackage{amsmath,amssymb,physics,framed,mathtools}
\usepackage{algorithm,algpseudocode}
\usepackage{graphicx}
\usepackage[utf8]{inputenc}
\usepackage[style=numeric,citestyle=numeric]{biblatex}
\addbibresource{refs.bib}

\AtEveryBibitem{
    \clearfield{urlyear}
    \clearfield{urlmonth}
}

\usepackage{hyperref}
\begin{document}

\maketitle

\begin{abstract}
  In this report, we present a brief summary of the overlapping group lasso, showing how it can be motivated as an extension first of the regular lasso to a group setting, and then as a further generalization thereof.
  We then provide some background on the Alternating Direction Method of Multipliers (ADMM) algorithm, and show why it is a reasonable choice for solving overlapping group lasso problems.
  Next, we derive the ADMM algorithm for overlapping group lasso, and present a software implementation in python that implements this algorithm.
  Finally, we show some experimental results on synthetic data across a variety of parameter settings.
\end{abstract}

\section{Notation}
\label{sec:notation}

First, we fix notation.
Since much of our derivations are based on \cite{boyd_distributed_2011}, we will use different notation than is typically deployed in more statistics-oriented treatments of the lasso.
Let $A \in \mathbb{R}^{m \times n}$ be a (fixed) design matrix, with $m$ observations and $n$ covariates, and $b \in \mathbb{R}^m$ a vector of observations.
We assume that our $b$ follows $b = A x^{\star} + \epsilon$, where $x^{\star} \in \mathbb{R}^p$ is an unknown weight vector, and $\epsilon$ are independent and identically distributed errors (for a simple case, we can take them to follow $\mathcal{N}(0,\sigma^2)$ for some fixed, but unknown, $\sigma^2$).
When norms are not otherwise specified, they are taken to be the 2-norm, i.e., $\lVert \cdot \rVert \overset{\Delta}{=} \lVert \cdot \rVert_2$.
We will generally be interested in minimizing the least squares loss, i.e., finding $\hat{x} \in \operatorname{argmin}_x \left\| Ax - b \right\|$.
Note: A great deal of the treatment below, including that of the lasso, group lasso, overlapping group lasso, and ADMM is taken from \cite{boyd_distributed_2011}.
We explicitly cite this text at key points, but we do not cite \emph{every} claim which is based on \cite{boyd_distributed_2011} to avoid cluttering the text.

\section{Background: Overlapping Group Lasso}
\label{sec:backgr-group-lasso}
In order to introduce the overlapping group lasso, we will first discuss the regular lasso and then show how it can be extended to the (non-overlapping) group lasso setting.
The lasso is a highly popular method that is especially useful in high dimensional settings, i.e., where $n \gg m$.
In this setting (presuming $A$ is of full rank), the OLS estimate is no longer uniquely determined, as there exist infinitely many candidate $\hat{x}$ that yield zero loss.
Instead, one can instead minimize a \emph{regularized problem} in order to obtain a \emph{sparse solution}.
While various (essentially equivalent) formulations of the lasso objective exist, for our purposes we will define the primal lasso problem as
\begin{equation}
  \label{eq:1}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \lVert x \rVert_1,
\end{equation}
where $\lambda > 0$ determines the amount of regularization.
As $\lambda \to 0$, we see that \eqref{eq:1} becomes the OLS problem, and as $\lambda \to \infty$, $x \to 0$.
One advantage of the lasso is that it will typically recover a sparse solution, i.e., a solution where the minimizing $\hat{x}$ has many entries that are identically 0.

In the setting where covariates can be organized into groups, as may be natural in many applied settings (e.g., where the covariates are gene expression levels, and genes can be organized based on chromosome or location), we may wish not to simply select useful covariates, but instead to select useful \emph{groups} of covariates.
This motivates the use of \emph{group lasso} \cite{yuan_model_2006}, where we replace the objective in \eqref{eq:1} with
\begin{equation}
  \label{eq:2}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_i \rVert_2,
\end{equation}
where $x_i$ is a sub-vector of $x$ containing only the coefficients corresponding to the $i$'th group, with $i \in [N]$.
Note that when extending to the group lasso, the penalty term no longer involves the 1-norm but instead has the 2-norm.
Although this may seem surprising, the 1-norm is separable, i.e., $\left\| \begin{pmatrix} a_1 \\ a_2 \end{pmatrix} \right\|_1 = \lVert a_1 \rVert_1 + \lVert a_2 \rVert_1$, and this would devolve back to the original lasso.
Critically, the 2-norm in the regularizer is not squared, which yields an analogous geometry to the lasso, with singularities corresponding to solutions that are group-sparse (see \cite{yuan_model_2006}, Fig 1 for a helpful illustration of this phenomena).
Note that for a singleton vector, $\lVert a \rVert_1 = \lVert a \rVert_2$, so when $N = n$, i.e., each feature is alone in its own atomic group, we can rewrite \eqref{eq:2} as
\begin{align*}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_i \rVert_2
  &= \operatorname*{min}_x \frac{1}{2}  \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_i \rVert_1 \\
  &= \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda  \lVert x \rVert_1    ,
\end{align*}
and we can recover the original lasso formulation in \eqref{eq:1} as a special case of the group lasso.

Finally, the \emph{group lasso} can be further extended to accommodate \emph{overlapping groups} in the ``overlapping group lasso'' \cite{zhao_composite_2009,mairal_network_2010}.
In this setting, rather than partitioning $x$ into disjoint sub-vectors, we let $G_i, i = [N]$ be an index set holding the indices of coefficients corresponding to the $i$'th group, i.e., $x_{G_i}$ is a vector of coefficients for group $i$, $x_{G_j}$ is a vector of coefficients for group $j$, and it may be the case that $G_i \cap G_j \neq \emptyset$.
As a toy example, suppose $n=3$, and in this simple setting we have two groups, with $G_1 = \left\{ 1, 2 \right\}, G_2 = \left\{ 2, 3 \right\}$, such that $x_2$ is common to both $x_{G_1}$ and $x_{G_2}$, i.e., $G_1 \cap G_2 = \left\{ 2 \right\}$.
In this setting, the overlapping group lasso objective is given by
\begin{equation}
  \label{eq:3}
  \operatorname*{min}_x \frac{1}{2} \lVert Ax - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert x_{G_i} \rVert_2.
\end{equation}
Of course, the (non-overlapping) group lasso discussed above is a special case of the overlapping group lasso, and by transitivity, since lasso is a special case of the group lasso, it is also a special case of the overlapping group lasso.

The geometry of this problem is rather complicated, and there is some work (e.g., \cite{jacob_group_2009}) that proposes addressing the overlapping group lasso through latent variables, in essence, performing variable duplication to render the problem non-overlapping, and then using the standard group lasso formulation in \eqref{eq:2} to solve the problem.
However, in the present work we will focus on directly optimizing the objective given in \eqref{eq:3}, although as we shall see in Section \ref{sec:group-lasso-via}, our algorithmic approach will involve a sort of variable duplication, but with an update step that pulls our duplicated variables back toward one another.

\section{Background: ADMM}
\label{sec:background:-admm}
The Alternating Direction Method of Multipliers (ADMM) is an algorithmic approach to optimization well suited to solving problems that can be decomposed as the sum of two problems in distinct variables, subject to linear constraints.
The background we provide here will closely follow \cite{boyd_distributed_2011}, as this was our primary resource when endeavoring to learn the material.
Our development here will be terse and limited, and we refer the reader to \cite{boyd_distributed_2011}, as the exposition given below chiefly consists of key highlights from this very useful text.

ADMM is formulated to solve problems structured as
\begin{equation}
  \begin{aligned}
    \label{eq:4}
    \operatorname*{min}_{x,z} \qquad &f(x) + g(z) \\
    \text{subject to} \qquad&Ax + Bz = c
  \end{aligned}.
\end{equation}
It is closely related to the method of multipliers (a brief background is given in \cite{boyd_distributed_2011}) and proceeds by first constructing an augmented Lagrangian
\begin{equation}
  \label{eq:5}
  L_{\rho}(x,z,y) = f(x) + g(z) + y^T (Ax + Bz - c) + \frac{\rho}{2} \lVert Ax + Bz - c \rVert_2^2,
\end{equation}
where $y$ is a dual variable, and the last term is the ``augmenting'' piece.
Although augmenting may seem unnatural at first, we note that when the linear constraints are satisfied, this last term is identically 0 and thus inconsequential for the objective function at the optimum, and its inclusion makes the use of the \texttt{prox} operator natural during the optimization.
ADMM is an iterative procedure, which given some initial values for $x,z,y$, proceeds as
\begin{align}
  \label{eq:6}
  x^{k+1} &\leftarrow \operatorname*{argmin}_x L_{\rho}(x,z^k,y^k) \\
  \label{eq:7}
  z^{k+1} &\leftarrow \operatorname*{argmin}_x L_{\rho}(x^{k+1},z,y^k) \\
  \label{eq:8}
  y^{k+1} &\leftarrow y^k + \rho(Ax^{k+1} + Bz^{k+1} -c).
\end{align}
Of course, the rub lies in actually solving the sub-problems given in \eqref{eq:6}\eqref{eq:7}, but using the augmented Lagrangian $L_{\rho}$ makes this tractable for certain problems.

\section{Overlapping Group LASSO Via ADMM}
\label{sec:group-lasso-via}
There is treatment of lasso, group lasso, and (very brief) mention of overlapping group lasso in \S 6.4 of \cite{boyd_distributed_2011}.
We noted in Section \ref{sec:backgr-group-lasso} that the group lasso, and in turn lasso, can be recovered as special cases of the overlapping group lasso.
However, \cite{boyd_distributed_2011} attempts to make (both overlapping and non-overlapping) group lasso notationally consistent with their later section on consensus learning, which actually yields an algorithm that does not permit recovery of these simpler forms as special cases, since the roles of $x$ and $z$ are reversed.
In our treatment here, we have eschewed this notational change and describe an approach that connects more naturally to lasso.

Our goal is to rewrite \eqref{eq:3} in a form amenable to application of ADMM but that is still equivalent to the original optimization problem.
In a similar spirit to the ``latent'' approach to overlapping group lasso \cite{jacob_group_2009,obozinski_group_2011}, we will create many \emph{new} variables which in a strict sense do not overlap, but then use linear equality constraints to impose the requirement that their relevant components coincide via a common ``anchoring'' global variable, which will indirectly enforce equality in the shared components.
In particular, we rewrite \eqref{eq:3} as,
\begin{equation}
  \begin{aligned}
  \label{eq:9}
  \operatorname*{min}_{x,z_i, i \in [N]} \quad &\frac{1}{2} \lVert A x - b \rVert_2^2 + \lambda \sum_{i=1}^N \lVert z_i \rVert_2 \\
  \text{such that} \quad & \tilde{x}_i - z_i = 0, \quad \forall i \in [N]
  \end{aligned},
\end{equation}
where $\tilde{x}_i \overset{\Delta}{=} x_{G_i}$, i.e., the components of $x$ corresponding to group $i$ for $i \in [N]$, and the constraint ensures that the solution satisfies $z_i = x_{G_i}, z_i \in \mathbb{R}^{\left| G_i \right|}$
This is now in a form compatible with ADMM, where the second term is analogous to $g(z)$, where $z$ is the concatenation of all the $z_i$'s, and $g$ very naturally decomposes across the $z_i$, which permits the update of the $z_i$ to be done in parallel.

Now, our task is to give explicit forms for steps \eqref{eq:6}\eqref{eq:7}.
Our update can be performed as
\begin{align}
  \label{eq:11}
  x^{k+1} &\leftarrow (A^TA + \rho I)^{-1}(A^T b + \rho(\bar{z}^{k+1} - \bar{u}^k)) \\
  \label{eq:10}
  z_i^{k+1} &\leftarrow \operatorname*{argmin}_{z_i} (\lambda \lVert z_i \rVert_2 + \left( u_i^k \right)^T(z_i - \tilde{x}_i^k) + \frac{\rho}{2} \lVert z_i - \tilde{x}_i^k \rVert_2^2) \\
  \label{eq:12}
  u_i^{k+1} &\leftarrow u_i^k +  \tilde{x}_i^{k+1} - z_i^{k+1},
\end{align}
where $\bar{z}, \bar{u}$ are obtained by averaging over $z_i, u_i$ at the relevant components.
The operation at \eqref{eq:10} is precisely the proximal operator, which for group lasso is the vector soft thresholding operation, as given in \S 6.4.2 of \cite{boyd_distributed_2011}, i.e.,
\begin{equation}
  \label{eq:13}
  z_i^{k+1} = (1 - \frac{\lambda}{\rho} \lVert \tilde{x}_i^{k+1} + u^k \rVert_2^{-1})_+ (\tilde{x}_i^{k+1} + u^k)
\end{equation}

We summarize the approach in Algorithm \ref{alg:grp-lasso-admm}.
\begin{algorithm}
    \caption{Overlapping Group LASSO Via ADMM}
    \label{alg:grp-lasso-admm}
    \begin{algorithmic}[1]
      \Require{$A, b, x^0 \in \mathbb{R}^n, \rho, \lambda, G$}
      \State $Q \gets (A^TA + \rho I)^{-1}$
      \For{$i \in [N]$}
      \State $z_i \gets x^0_{G_i}$
      \State $u_i \gets \left\{ 0 \right\}^{\lvert G_i \rvert}$
      \EndFor
      \State $k \gets 0$
      \While{not converged}
      \State $x^{k+1} \gets Q \left( A^Tb + \rho(\bar{z}^{k+1} - \bar{u} \right)$
      \For{$i \in [N]$}
      \State $z_i^{k+1} \gets S_{\lambda/\rho}(\tilde{x}_i^{k+1} + u_i)$
      \EndFor
      \For{$i \in [N]$}
      \State $u_i^{k+1} \gets u_i^k + \tilde{x}_i^{k+1} - z_i^{k+1} $
      \EndFor
      \State $k \gets k+1$
      \EndWhile
  \end{algorithmic}
\end{algorithm}

\section{Software Implementation}
\label{sec:implementation}
We implemented the procedure of Algorithm \ref{alg:grp-lasso-admm} in python 3.7.
The implementation, along with this report, is available on github at \url{http://github.com/dankessler/608a-project}.
The dependency environment can be reconstructed using \href{https://pipenv.readthedocs.io/en/latest/}{\texttt{pipenv}} using the \texttt{Pipfile} and \texttt{Pipfile.lock} found at the root of the repository.

\section{Experimental Results}
\label{sec:experimental-results}
We conducted numerical experiments in simulated data.
A script named \texttt{simulations.py} reproducing these results is available on github in the root directory of our repository \url{http://github.com/dankessler/608a-project}.
We generate synthetic data as $y = A x^{\star} + \epsilon$, where $\epsilon \sim \mathcal{N}_m(0,I_m), A \in \mathbb{R}^{m \times n}$ is a random matrix with iid standard normal entries, and for all simulations we fix $m = 50, n = 100$.
Our three experiments below are under different settings for $x^{\star}$.
In each case, we run our implementation of Algorithm \ref{alg:grp-lasso-admm} on the synthetic data with $\rho = 1$ and vary $\lambda$ over a logarithmic grid.
Each evaluation yields a regularized estimate $\hat{x}$, and with this we evaluate
\begin{enumerate}
\item The prediction error: $
  \left\|
    A x^{\star} - A \hat{x}
  \right\|_2$
\item The accuracy of the estimated $x$: $
  \left\|
    x^{\star} - \hat{x}
  \right\|_2$
\item The precision of the recovery of the support: $\frac{
    \left|
      \left\{
        i: \hat{x}_i \neq 0
      \right\}
      \cap
      \left\{
        j: x^{\star}_j \neq 0
      \right\}
    \right|}{
    \left\|
      \hat{x}
    \right\|_0}$
\item The recall of the recovery of the support: $\frac{
    \left|
      \left\{
        i: \hat{x}_i \neq 0
      \right\}
      \cap
      \left\{
        j: x^{\star}_j \neq 0
      \right\}
    \right|}{
    \left\|
      x^{\star}
    \right\|_0}$
\end{enumerate}
Because the $x$ iterates in our algorithm never become \emph{truly} sparse, as a heuristic after returning $\hat{x}$ for a fixed number of iterations (here one thousand) we set to 0 any entries of $\hat{x}$ with magnitude smaller than $\epsilon = .01$.
This enables more meaningful interpretation for the latter two metrics (since otherwise the set of components of $\hat{x}$ that are nonzero will be all $n$.
These results are depicted graphically as a function of $\lambda$ in Figures \ref{fig:lasso}, \ref{fig:glasso}, and \ref{fig:oglasso}.

\subsection{Lasso as a Special Case}
\label{sec:lasso-as-special}
As discussed earlier, if we place each covariate into its own atomic group, our problem devolves to the classic lasso.
We set $x_i^{\star} = 10$ for $i \in [25]$, and $x_i^{\star} = 0$ for $i \in [50] \setminus [25]$.
We conduct the simulation approach as described above, and display our results in Figure \ref{fig:lasso}.

\begin{figure}[h]
  \centering
  \includegraphics[width=\textwidth]{../lasso}
  \caption{Performance metrics for simulations under lasso. Larger values of $\lambda$ generally encourage a sparser solution, which seems to yield worse prediction error, estimation error, and recall in recovery of support. Precision initially improves, but this is likely due to a shrinking denominator.}
  \label{fig:lasso}
\end{figure}

\subsection{Non-overlapping Group Lasso as a Special Case}
\label{sec:non-overl-group}
Our set up for (non-overlapping) group lasso is reasonably straightforward.
We construct two disjoint groups as $G_1 = [25], G_2 = [50] \setminus G_1$.
We set $x^{\star}_i = 10$ for $i \in G_1$, and $x^{\star}_i = 0$ for $i \in G_2$.
Note that this is the same $x^{\star}$ as in lasso above, but the penalization is different.
The results are presented in Figure \ref{fig:glasso}.

\begin{figure}[h]
  \centering
  \includegraphics[width=\textwidth]{../glasso}
  \caption{Performance metrics for simulations under (non-overlapping) group lasso. These plots are reasonably similar to those in Figure \ref{fig:lasso}, which is unsurprising since the problem set up is very similar.}
  \label{fig:glasso}
\end{figure}

\subsection{Overlapping Group Lasso}
\label{sec:overl-group-lasso}
For overlapping group lasso, we construct 9 groups of equal size that overlap by the same amount.
In particular, $G_i = \{5(i-1) + j : j \in [10]\}, i \in [9]$.
For example, $G_1 =
\left\{
  1, 2, 3, 4, 5, 6, 7, 8, 9, 10
\right\}$, $G_2 =
\left\{
  6, 7, 8, 9, 10, 11, 12, 13, 14, 15
\right\}$, so $G_1 \cup G_2 =
\left\{
  6, 7, 8, 9, 10
\right\}$.
As was discussed briefly earlier, our formulation of the overlapping group lasso, unlike that in \cite{jacob_group_2009,obozinski_group_2011}, has singularities corresponding to the complement of unions of groups.
For this reason, we set $x^{\star}_i = 10$ for $i \in
\left\{
  1, 2, 3, 4, 5, 16, 17, 18, 19, 20
\right\}$ and 0 otherwise.
This corresponds to the complement of the union of groups $G_2, G_3, G_5, G_6, G_7, G_8$,  and $G_9$.
Results are presented in Figure \ref{fig:oglasso}.

\begin{figure}[h]
  \centering
  \includegraphics[width=\textwidth]{../oglasso}
  \caption{Performance metrics for simulations under overlapping group lasso.
As with all our results, increasing $\lambda$ hurts prediction error. Also, we can infer that we require high values of $\lambda$  before we obtain solutions that are at all sparse, and this is why recall is initially so high. In terms of estimation error, extreme values of $\lambda$ (too big or too small) give poor performance, but things are quite good in the middle.}
  \label{fig:oglasso}
\end{figure}

\section{Conclusion and Future Directions}
\label{sec:concl-future-direct}
In this report, we provided a brief introduction to overlapping group lasso, reviewed requisite background materials on ADMM, and then showed how ADMM could be applied to the overlapping group lasso problem.
We then conducted numerical experiments on simulated data under three different settings: lasso as a special case of overlapping group lasso, (non-overlapping) group lasso as a special case of overlapping group lasso, and finally a non-trivial overlapping group lasso problem.
One important note is that our approach is in contrast to one favored by some statisticians \cite{obozinski_group_2011,jacob_group_2009}.
The reason for this is that the explicit formulation of the overlapping group lasso, as we have implemented,
is well positioned for recovery in settings where the true support is the complement of a union of groups.
Conceptually, this corresponds to ``deactivating'' a set of groups, and zeroing out all of their corresponding features, which in the case of overlaps, will result in some ``active'' groups only being active in components that are not shared with a deactivated group.
In some applied settings this may be a reasonably accurate regime, but in settings where it is more natural to countenance the support being the union of \emph{active} groups, an alternative formulation, described in \cite{obozinski_group_2011,jacob_group_2009}, proposes that variables be duplicated to render the problem non-overlapping, but does \emph{not} involve the introduction of linear constraints that require that corresponding entries perfectly match.
This effectively renders the problem non-overlapping.
For some of the applied settings in neuroscience where we propose to use our newly developed overlapping group lasso package in python, this latter formulation is more natural.
However, the good news is that since we have demonstrated that our approach can solve \emph{both} of these problems, as the latter is a special case of the former, the software that we have developed in this project can be used directly (with some wrapper functionality to handle the bookkeeping involved in variable duplication).

As a next step, we plan to validate the performance of our implementation against a reference implementation.
While there exist many implementations of lasso and a few implementations of non-overlapping group lasso, we have yet to find a overlapping group lasso implementation that seems to give sensible results.
Nonetheless, if we can validate our implementation against the simpler cases, we may be able to use our package as a benchmark against this other implementations to better understand their shortcomings.

\clearpage
\printbibliography

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:

%  LocalWords:  ADMM covariates argmin OLS eq th regularizer Bz prox
%  LocalWords:  notationally notational thresholding github pipenv py
%  LocalWords:  Pipfile iid covariate neuroscience


\documentclass[a4paper]{siamart190516}

\usepackage{damacros}

% PACKAGES added by TB
% PACKAGES FOR ALGORITHMS (PSEUDO-CODE)
\usepackage{algorithm}
\usepackage{algorithmic}

% Sets running headers as well as PDF title and authors
\headers{Coupling intra-cellular and multi-cellular dynamics in spatially-extended models of root-hair initiation}{D. Avitabile, S. Perotto, N. Ferro, T. Babini}

% Title. If the supplement option is on, then "Supplementary Material"
% is automatically inserted before the title.
\title{Coupling intra-cellular and multi-cellular dynamics in spatially-extended models of root-hair initiation}

% Authors: full names plus addresses.
\author{%
  Daniele Avitabile%
  \thanks{%
    Vrije Universiteit Amsterdam,
    Department of Mathematics,
    Faculteit der Exacte Wetenschappen,
    De Boelelaan 1081a,
    1081 HV Amsterdam, The Netherlands.
  \protect\\
    Inria Sophia Antipolis M\'editerran\'ee Research Centre,
    MathNeuro Team,
    2004 route des Lucioles-Boîte Postale 93 06902,
    Sophia Antipolis, Cedex, France.
  \protect\\
    (\email{d.avitabile@vu.nl}, \url{writemywebpage}).
  }
 %  .... mancano altri
  % \and
%   Paul T. Frank \thanks{Department of Applied Mathematics, Fictional University, Boise, ID
% (\email{ptfrank@fictional.edu}, \email{jesmith@fictional.edu}).}
% \and Jane E. Smith\footnotemark[3]
}

\begin{document}

\maketitle

\begin{abstract}
This thesis deals with novel models and numerical approximations of spatially-extended multi-cellular models of Rho Of Plants (ROPs), that is, a family of proteins responsible for root-hair initiation in the plant cell Arabidopsis thaliana. The study of this dynamical system is of great relevance in the so-called agriculture 4.0, since it is instrumental to optimise plant uptake. In particular, ascertaining how intra-cellular protein distributions and extra-cellular coupling influence root-hair initiation is a challenging but pressing problem.

Current studies have focussed on two separate model types: on the one hand, ROPs dynamics is studied in single-cell models, which resolve patterns at sub cellular level; on the other, multi- cellular models with realistic geometries neglect intra-cellular patterning. In this thesis we make progress on coupling these two
model descriptions.

We initially focus on a well-established single-cell, nonlinear reaction-diffusion model, here approximated for the first time with a finite-element scheme. In addition, we present a new model which couples multiple cells through ROP flux at the interface. We present numerical evidence that such coupling has a bearing on the patterns supported by the model. It is shown that, under variations of auxin gradients, the model robustly forms ROP hotspots from ROP stripes, and that spots are later advected downstream.

Finally, we consider a novel model in which the auxin dynamics are not prescribed, but derive from the interaction between this hormone and other membranals proteins (PIN). We show that self-sustained auxin oscillations influence ROP intracellular patterning.
\end{abstract}

\section{Introduction (Daniele+Teresa+Simona)}

mettere il modello Capitolo 4 (solo equazioni) +  posizionamento in
letteratura + novelty paper

% from chapter 4 physical model sec.

the dynamics of auxin and carriers proteins PIN in each cell $\Omega_i$ with $i \in \{ 1, ..., N \}$ can be written as the following system of coupled ordinary differential equations:
\begin{equation}\begin{aligned}
\begin{cases}
  {\displaystyle d a_i\over\displaystyle d t} & = {\displaystyle 1 \over \displaystyle V_i} \sum_{j=1}^{N} A_{ij} \Phi_{ij} +k - \delta a_i \\[8pt]
  {\displaystyle d \Tilde{P}_{ij}\over\displaystyle d t} & = h\left(\Phi_{ij}\right) + \rho_0 - \mu \Tilde{P}_{ij},
\end{cases}
\end{aligned} \end{equation}

% ...

The original system of equations can be rescaled and simplified. In particular, under the assumptions of cells having same volume $V =V_i$ and exchange surface areas $A = A_{ij}$, rescaling properly the diffusion coefficient $D_a$ and variables $P_{ij}$, a new system is obtained. Thus, the final system of equations we work on is:
\begin{equation}\label{eq:Sys_auxPIN}\begin{aligned}
\begin{cases}
  {\displaystyle d a_i\over\displaystyle d t} & = \sum_{j \in \mathcal{N}_i} \displaystyle \Phi_{ji} +k - \delta a_i \\[8pt]
  {\displaystyle d P_{ij}\over\displaystyle d t} & = h\left(\Phi_{ij}\right) - \mu P_{ij},
\end{cases}
\end{aligned} \end{equation}

% ... from chapter 2 sec physical model
The dimentional reaction-diffusion model summarizing binding process, autocatalytic activation and catalysis of ROPs proteins described in Section \ref{sec:intromodel} is formulated as follows:
% As mentioned before, the dimentional reaction-diffusion model is:
\begin{equation} \label{eq:FM}\begin{aligned}
\left\lbrace
\begin{matrix}
\partial_t u = & D_1 \Delta_s u + k_{20} \alpha(x,y)u^2 v - \left(c+r\right) u + k_1 v  & \ \text{in} \ \Omega\\
    \partial_t v = & D_2 \Delta_s v - k_{20} \alpha(x,y) u^2 v - k_1 v + c \ u + b   & \ \text{in} \ \Omega.
\end{matrix}
\right.
\end{aligned}\end{equation}

% ...
The RD system is rewritten to explicit its mathematical formulation and methods in order to solve it in the most general way, including both the case of the RD system with the original parameters and the RD system with the rescaled ones.

As a consequence, we define a unified system of equations comprehensive of both \eqref{eq:FM} and \eqref{eq:adim} systems:
\begin{equation} \label{eq:final}
\left\lbrace
\begin{matrix}
\partial_t u = & \Tilde{D_1} \Delta_s u + \Tilde{a_1} u + \Tilde{b_1} v + \Tilde{c_1} u^2 v \ \ \text{in} \ \Omega\\
\partial_t v = & \Tilde{D_2} \Delta_s v + \Tilde{a_2} v + \Tilde{b_2} u + \Tilde{c_2} u^2 v + f_2 \ \ \text{in} \ \Omega,
\end{matrix}
\right.
\end{equation}
with tilde parameters defined differently in the two cases under study

\section{A starting modeling  (Teresa+Daniele)}

Modello di Capitolo 3

\textbf{Physical model - Sec.3.1}
We consider the root-hair cell projection onto a 2D rectangular domain, neglecting axial dimension.
% as in Chapter \ref{cap:2}
A system of four cells is schematically presented in Figure \ref{fig:2cell}. We can see that each cell has longitudinal and transverse boundaries in common with close cells.
We recall the single cellular model, namely:
\begin{equation} \label{eq:singModel}
\left\lbrace
\begin{matrix}
  \begin{aligned}
    & \partial_t u = \Tilde{D_1} \Delta_s u + \Tilde{a_1} u + \Tilde{b_1} v + \Tilde{c_1} u^2 v & \ \text{in} \ \Omega\\
    & \partial_t v = \Tilde{D_2} \Delta_s v + \Tilde{a_2} v + \Tilde{b_2} u + \Tilde{c_2} u^2 v + f_2 & \ \text{in} \ \Omega \\
    & \Tilde{D_1} \nabla_s u \cdot \mathbf{n} = 0 & \ \text{on} \ \partial \Omega \\
    & \Tilde{D_2} \nabla_s v \cdot \mathbf{n} = 0 & \ \text{on} \ \partial \Omega.
  \end{aligned}
\end{matrix}
\right.
\end{equation}
No-flux on $\partial \Omega$, namely Neumann homogeneous boundary conditions, characterizes the system behaviour along the cell boundary.
% prima spieghiamo il significato fisico, poi come rappresentarlo matematicamente
In the multi-cellular model, communication between cells is represented by allowed flux of ROPs, active and inactive, through localized channels along boundaries between neighboring cells.

We define as neighbor of cell $\Omega_i$ the set of cells with index in $\mathcal{N}_i = \{ j : \partial \Omega_j  \cap \partial \Omega_i \neq \emptyset \}$. The flux of concentration of active and inactive ROPs $(u_i, v_i)$ is proportional to the difference of concentration $(u_j, v_j)$ in neighbouring cells for $j \ \in \ \mathcal{N}_i$.

We formulate the new model still focusing on one single cell domain $\Omega_i$, taking into account the new flux generated from the discrepancy of concentrations with the neighboring cells. The new flux results in adding a non-homogeneous Neumann boundary condition on the common interfaces, as follows:
\begin{equation} \label{eq:pluriModel}
\left\lbrace
\begin{matrix}
  \begin{aligned}
    & \partial_t u_i = \Tilde{D_1} \Delta_s u_i + \Tilde{a_1} u_i + \Tilde{b_1} v_i + \Tilde{c_1} (u_i)^2 v_i & \ \text{in} \ \Omega_i\\[6pt]
    & \partial_t v_i = \Tilde{D_2} \Delta_s v_i + \Tilde{a_2} v_i + \Tilde{b_2} u_i + \Tilde{c_2} (u_i)^2 v_i + f_2 & \ \text{in} \ \Omega_i \\[6pt]
    & \Tilde{D_1} \nabla_s u_i \cdot \mathbf{n} = 0 & \ on \ \partial \Omega_i \backslash \cup_{j \in \mathcal{N}_i} \Gamma_{j,i} \\[6pt]
    & \Tilde{D_2} \nabla_s v_i \cdot \mathbf{n} = 0 & \ on \ \partial \Omega_i \backslash \cup_{j \in \mathcal{N}_i} \Gamma_{j,i} \\[6pt]
    & \Tilde{D_1} \nabla_s u_i \cdot \mathbf{n} = \beta_{uRR} \ \alpha_{uRR} \left(u_j - u_i \right) & \ on \ \Gamma_{j,i} \ \forall j \in  \mathcal{N}_i \\[6pt]
    & \Tilde{D_2} \nabla_s v_i \cdot \mathbf{n} = \beta_{vRR} \ \alpha_{vRR} \left(v_j - v_i \right) & \ on \ \Gamma_{j,i}\ \forall j \in  \mathcal{N}_i ,
  \end{aligned}
\end{matrix}
\right.
\end{equation}
where we define as $(u_i, v_i)$ the concentrations of active and inactive ROPs restricted to cell $\Omega_i$: $(u_i, v_i): \Omega_i \times \left(0, T_{max} \right) \longrightarrow \RSet^2$ and $\Gamma_{j,i}$ represents the common side between cell $\Omega_i$ and cell $\Omega_j \ \in \mathcal{N}_i $, therefore defined as: $\Gamma_{j,i} = \partial \Omega_i \cap \partial \Omega_j$.

Each of the neighboring cells follows the same model for hair formation, meaning that system in \eqref{eq:pluriModel} holds $\forall \ i$ cells composing the pluricellular system. As a consequence, the newly defined boundary conditions is coupled with the solutions $(u_j, v_j)$ with $j \ \in \mathcal{N}_i$. Therefore, the pluricellular system requires a proper iterative method for setting correctly boundary conditions depending on solutions in the neighboring cells.

Not communicating with other RH cells boundaries have as before no-flux. The new boundary conditions are characterized by a function and a coefficient for both active active ROPs $u$ and inactive ROPs $v$, having the same meaning:
\begin{itemize}
  \item $\beta_{u/v RR} \ [\frac{1}{\mu m^2}]$ are indicator functions defined on boundaries of cells, equal to $1$ where the communicating channels are open and $0$ where  no-flux is assumed;
  \item $\alpha_{u/v RR} \ [\frac{1}{\mu m}]$ are transport efficiency coefficients, representing a sort of flux quantity allowed through channels.
  \end{itemize}
These channel parameters aim at representing the average active transport along the sides of confining cells, set equal to the flux of proteins from one cell to the neighbouring ones.

We have no physical insight on previously cited functions modeling open channels for ROPs. A whole set of simulations for the proper tuning of parameters is required, in order to find a sufficiently plausible setting of the system.

\section{A new dd-wise coupling approach (Simona+Daniele+Nicola+Teresa)}

DD sul modello del Capitolo 3 + parte discreta

\textbf{Numerical treatment - Sec.3.2}
% brutto mettere gli stessi titoli?
% -> prima questione: trattarlo con uno schema iterativo simile robin robin algorithm of domain decomposition, in modo da accoppiare le soluzioni; quindi descrivere bene lo schema (dalla strong formulation); già partendo dalo schema semi implicit nel tempo
The communication between cells requires a proper iterative algorithm in order to deal with the mutual interplay between confining cells.

Every subdomain $\Omega_i$ of the pluricellular system $\Omega$ represents the single cell and the original system of equations in \eqref{eq:final} is solved in $\Omega_i$ for all $i = 1, ..., N$. We solve such systems by means of the semi-implicit method described in Section \ref{sec:SI method}. Let us consider the weak formulation restricted to $\Omega_i$, defining the functional space $V_i = \{ w_i \in \ H^1\left(\Omega_i\right)\}$, the finite element subspace $V_{i,h} \subset V_i $ and the time interval discretization used in Section \ref{sec:SI method}. In particular, we divide the time interval $\left[0, T_{max}\right]$ in $N_{max}$ time steps such that $t^n = n \Delta t$ with $\Delta t = T_{max} / N_{max}  $. We rewrite the full discretized formulation, identifying $u_{i,h}$ with $u_h|_{\Omega_i}$, as:

given the initial state $(u_{i,h}^0, v_{i,h}^0) $, find $(u_{i,h}^{n+1}, v_{i,h}^{n+1}) \ \in V_{i,h} \times V_{i,h}$ such that
\begin{equation} \label{eq:fullGalerkin}
\left\lbrace
\begin{matrix}
\begin{aligned}
  a_{i,u}(u_{i,h}^{n+1}, w_{i,h}) + b_{i,u}(v_{i,h}^{n+1}, w_{i,h}) + c_{i,u}(v_{i,h}^{n+1}, w_{i,h}) = f_{i,u}(w_{i,h}) \ \forall \ w_{i,h} \ \in V_{i,h} \\[6pt]
 a_{i,v}(v_{i,h}^{n+1}, w_{i,h}) + b_{i,v}(u_{i,h}^{n+1}, w_{i,h}) + c_{i,v}(v_{i,h}^{n+1}, w_{i,h}) = f_{i,v}(w_{i,h}) \ \forall \ w_{i,h} \ \in V_{i,h},
\end{aligned}
\end{matrix}
\right.
\end{equation}
$\forall n = 0, ... N_{max}$, where
\begin{subequations} \label{eq:Gvarfmono}
\begin{align}
    a_{i,u}(u_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left( \frac{1}{\Delta t} u_{i,h}^{n+1} w_{i,h} + \Tilde{D}_1 \nabla_s u_{i,h}^{n+1} \cdot w_{i,h} - \Tilde{a}_1 u_i^{n+1} w_{i,h} \right) \label{Gmono:au}\\ - & \int_{\partial \Omega_i}\left(\Tilde{D}_1 \nabla_s u_{i,h}^{n+1} \cdot \mathbf{n} w_{i,h}\right)  \nonumber\\
    b_{i,u}(v_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left(- \Tilde{b}_1 v_{i,h} w_{i,h} \right)  \label{Gmono:bu} \\
    c_{i,u}(v_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left(- \Tilde{c}_1 (u_{i,h}^{n})^2  v_{i,h}^{n+1} w_{i,h} \right) \label{Gmono:cu} \\[6pt]
    a_{i,v}(v_{i,h}^{n+1}, w_{i,h}) = & \int_{\Omega_i} \left(\frac{1}{\Delta t} v_{i,h}^{n+1} w_{i,h} + \Tilde{D}_2 \nabla_s v_{i,h}^{n+1} \cdot w_{i,h} - \Tilde{a}_2 v_i^{n+1} w_{i,h} \right) \label{Gmono:av} \\ - &  \int_{\partial \Omega_i} \left( \Tilde{D}_1 \nabla_s u_{i,h}^{n+1} \cdot \mathbf{n} w_{i,h} \right) \nonumber\\
    b_{i,v}(v_{i,h}^{n+1}, w_{i,h}) = &\int_{\Omega_i} \left( - \Tilde{b}_2 u_{i,h} w_{i,h} \right) \label{Gmono:bv}\\
    c_{i,v}(v_{i,h}^{n+1}, w_{i,h}) =& \int_{\Omega_i} \left( - \Tilde{c}_2 (u_{i,h}^{n})^2  v_{i,h}^{n+1} w_{i,h} \right) \label{Gmono:cv}\\[6pt]
    f_{i,u}(w_{i,h}) = & \int_{\Omega_i} \left( \frac{1}{\Delta t} u_{i,h}^n \ w_{i,h} \right) \label{Gmono:fu}\\
    f_{i,v}(w_{i,h}) = & \int_{\Omega} \left( \frac{1}{\Delta t} v_{i,h}^n \ w_{i,h} + f_2 w_{i,h} \right). \label{Gmono:fv}
\end{align}
\end{subequations}

The introduction of different boundary conditions will require to modify the bilinear forms \eqref{Gmono:au} and \eqref{Gmono:av} and to add contributions in the right hand sides \eqref{Gmono:fu} and \eqref{Gmono:fv}.

To this aim, we synthetically rewrite the model problem \eqref{eq:fullGalerkin}, assuming generic boundary conditions, through a linear operator $\mathcal{L}$ in the following way:

Given the initial state $(u_i^0, v_i^0)$, find $(u_i^{n+1}, v_i^{n+1}) \ \in \Omega_i$ such that:
\begin{equation}\label{eq:modelpb}
% \begin{cases}
\mathcal{L}^n (u_i^{n+1}, v_i^{n+1}) = \mathbf{f}^n \ \text{in} \ \Omega_i
% \Tilde{D_1} \nabla_s u_i^{n+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_i \\
% \Tilde{D_2} \nabla_s v_i^{n+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_i
% \end{cases}
\end{equation}
$\forall n = 0, ... N_{max}$.

\textbf{A new iterative modeling algorithm - Sec 3.2.3 }
The model that we propose to make cells communicate can be regarded as a simplification of the classical domain decomposition scheme with Robin boundary conditions. We start for simplicity from a two cells problem and rewrite the common interface boundary conditions in \eqref{eq:RR} to recover the modelled open channels in \eqref{eq:pluriModel}. In the spirit of a block-Gauss-Seidel algorithm, we solve in sequence:
\begin{equation} \label{eq:RR_final}
\begin{aligned}
& \begin{cases}
\mathcal{L}^n (u_1^{k+1}, v_1^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_1 \\
\Tilde{D_1} \nabla_s u_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_1} \displaystyle{\partial u_1^{k+1}\over\partial \mathbf{n}} = \alpha_{uRR} u_2^{k} - \alpha_{uRR} u_1^{k+1} \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_2} \displaystyle{\partial v_1^{k+1}\over\partial \mathbf{n}} = \alpha_{vRR} v_2^{k} -\alpha_{vRR} v_1^{k+1} \ \text{on} \ \Gamma
\end{cases}
\\[6pt]
& \begin{cases}
\mathcal{L}^n (u_2^{k+1}, v_2^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_2 \\
\Tilde{D_1} \nabla_s u_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \setminus \Gamma \\
\Tilde{D_1} \displaystyle{\partial u_2^{k+1}\over\partial \mathbf{n}} = \alpha_{uRR} u_1^{k+1} - \alpha_{uRR} u_2^{k+1} \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \setminus \Gamma\\
\Tilde{D_2} \displaystyle{\partial v_2^{k+1}\over\partial \mathbf{n}} = \alpha_{vRR} v_1^{k+1} -\alpha_{vRR} v_2^{k+1} \ \text{on} \ \Gamma.
\end{cases}
\end{aligned}\end{equation}
The flux imposed depends on the difference of the neighbouring solutions. As a consequence, we are imposing a not necessarily null Neumann boundary condition. Equation \eqref{eq:RR_final} defines a RR iterative method applied to two cells using proper parameters $\beta_{u/vRR}$ and $\alpha_{u/v RR}$ from the model formulated in Section \ref{sec:PluriMod}:

starting from $(u_2^{k = 0}, v_2^{k = 0}) = (u_2^n, v_2^n)$, find $(u_1^{k+1}, v_1^{k+1}) \ \in V_{1}$ and $(u_2^{k+1}, v_2^{k+1}) \ \in V_{2}$:
\begin{equation}\label{eq::RRmod}
\begin{aligned}
& \begin{cases}
\mathcal{L}^n (u_1^{k+1}, v_1^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_1 \\
\Tilde{D_1} \nabla_s u_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_1} \displaystyle{\partial u_1^{k+1}\over\partial \mathbf{n}} = \beta_{uRR} \alpha_{uRR} \left( u_2^{k} - u_1^{k+1} \right) \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_1^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_1 \setminus \Gamma \\
\Tilde{D_2} \displaystyle{\partial v_1^{k+1}\over\partial \mathbf{n}} = \beta_{vRR} \alpha_{vRR} \left( v_2^{k} - v_1^{k+1} \right) \ \text{on} \ \Gamma
\end{cases}
\\[6pt]
& \begin{cases}
\mathcal{L}^n (u_2^{k+1}, v_2^{k+1}) = \mathbf{f}^n \ \text{in} \ \Omega_2 \\
\Tilde{D_1} \nabla_s u_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \\
\Tilde{D_1} \displaystyle{\partial u_2^{k+1}\over\partial \mathbf{n}} = \beta_{uRR} \alpha_{uRR} \left( u_1^{k+1} - u_2^{k+1} \right) \ \text{on} \ \Gamma \\
\Tilde{D_2} \nabla_s v_2^{k+1} \cdot \mathbf{n} = 0 \ \text{on} \ \partial \Omega_2 \setminus \Gamma\\
\Tilde{D_2} \displaystyle{\partial v_2^{k+1}\over\partial \mathbf{n}} = \beta_{vRR} \alpha_{vRR} \left( v_1^{k+1} - v_2^{k+1} \right) \ \text{on} \ \Gamma.
\end{cases}
\end{aligned}\end{equation}
for $k \geq 0$ up to convergence.

We remark that in \eqref{eq::RRmod} the coefficients $\beta_{u/vRR}$ and $\alpha_{u/v RR}$ have physical meaning since they come from the model \eqref{eq:pluriModel}. This is in contrast with the model and method presented in Section \ref{sec:RRclassic}, where the Robin coefficients are arbitrary.

Let $V_{i,h}$ denote the finite dimensional subspace of $H^1\left(\Omega_i\right)$, wih $\Omega_i$ being the sub-domain of the pluricellular system $\Omega$ corresponding to cell. We find solutions $\left(u_{h}^{n+1}, v_{h}^{n+1}\right)|_{\Omega_i}$ identified with $\left(u_{i,h}, v_{i,h}\right) \ \in V_{i,h}$ for each time step $t^{n+1}$, solving up to convergence the iteration step, whose Galerkin formulation is:
\begin{equation}\begin{aligned}
    a_{i,u}^{RR}(u_{i,h}^{k+1}, w_{i,h}) + b_{i,u}(v_{i,h}^{k+1}, w_{i,h}) + c_{i,u}^n(v_{i,h}^{k+1}, w_{i,h}) = f_{i,u}^{RR}(w_{i,h}) \ \forall w_{i,h} \in V_{i,h} \\
    a_{i,v}^{RR}(v_{i,h}^{k+1}, w_{i,h}) + b_{i,v}(u_{i,h}^{k+1}, w_{i,h}) + c_{i,v}^n(v_{i,h}^{k+1}, w_{i,h}) = f_{i,v}^{RR}(w_{i,h}) \ \forall w_{i,h} \in V_{i,h}.
\end{aligned}\end{equation}

The bilinear forms used are equal to \eqref{eq:Gvarfmono} - \eqref{eq:au&avRR} for classic Robin-Robin algorithm. The only difference is in the right hand side \eqref{eq:fu&fvRR} in which has been neglected the weak normal derivative of the neighbour solutions, as follows:
\begin{align}
 f_{i,u}^{RR}(w_{i,h}) & = f_{i,u}(w_{i,h}) + \int_{\Gamma} \left(\beta_{uRR} \alpha_{uRR} \mathcal{I}_{i,j} u_{j,h}^{k} \mathcal{I}_{i,j}w_{j,h} \right), \\
f_{i,v}^{RR}(w_{i,h}) & = f_{i,v}(w_{i,h}) + \int_{\Gamma} \left(\beta_{vRR} \alpha_{vRR} \mathcal{I}_{i,j} v_{j,h}^{k} \mathcal{I}_{i,j} w_{j,h} \right).
\end{align}

Consequently, the algebraic formulation of the new iterative method used is formulated similarly as in \eqref{eq:LinSysRR}, with time-dependent block matrix that need to be reassembled at each time-step. The right-hand sides depend on the previous solution found for the neighbouring cells and their contributions need to be interpolated by means of a interpolation matrix as in Robin-Robin classic method. We here explicit the whole iterative method for a two cells composed system.

Starting from initial guess given by the previous time-step solution  $\begin{bmatrix} \mathbf{U}_2^{0} \\ \mathbf{V}_2^{0} \end{bmatrix} = \begin{bmatrix} \mathbf{U}_2^{n} \\ \mathbf{V}_2^{n} \end{bmatrix}$, solve problem for $i = 1$ to find $\begin{bmatrix} \mathbf{U}_1^{k+1} \\ \mathbf{V}_1^{k+1} \end{bmatrix}$:
\begin{equation*}
\begin{aligned}
 \begin{bmatrix}
    A_u^1 & B_u^1 + C_u^1\left(  \mathbf{U}_1^n\right) \\
    B_v^1 & A_v^1 + C_v^1\left(  \mathbf{U}_1^n\right)
    \end{bmatrix} \begin{bmatrix}
    \mathbf{U}_1^{k+1} \\ \mathbf{V}_1^{k+1} \end{bmatrix} = \begin{bmatrix} F^1_u \left(\mathbf{U}_2^k\right) \\ F^1_v \left(\mathbf{V}_2^k\right)
    \end{bmatrix}
        \end{aligned}
\end{equation*}
and then solve problem for $ i = 2$ to find $\begin{bmatrix} \mathbf{U}_2^{k+1} \\ \mathbf{V}_2^{k+1} \end{bmatrix}$:
\begin{equation*}
    \begin{aligned}
\begin{bmatrix}
    A_u^2 & B_u^2 + C_u^2\left(  \mathbf{U}_2^n\right) \\
    B_v^2 & A_v^2 + C_v^2\left(  \mathbf{U}_2^n\right)
    \end{bmatrix} \begin{bmatrix}
    \mathbf{U}_2^{k+1} \\ \mathbf{V}_2^{k+1} \end{bmatrix} = \begin{bmatrix} F^2_u \left(\mathbf{U}_1^k\right) \\ F^2_v \left(\mathbf{V}_2^k\right)
    \end{bmatrix}
\end{aligned}\end{equation*}
for $k \geq 0$ up to convergence.

Iterations end when the normalized residual of consecutive computed solutions is smaller than a proper tolerance or when a maximum number of iterations is performed and we update the new solution as:
 $$\begin{bmatrix} \mathbf{U}_1^{n+1} \\ \mathbf{V}_1^{n+1} \end{bmatrix} = \begin{bmatrix} \mathbf{U}_1^{k+1} \\ \mathbf{V}_1^{k+1} \end{bmatrix}, \ \ \ \begin{bmatrix} \mathbf{U}_2^{n+1} \\ \mathbf{V}_2^{n+1} \end{bmatrix} = \begin{bmatrix} \mathbf{U}_2^{k+1} \\ \mathbf{V}_2^{k+1} \end{bmatrix}$$

Matrices and vectors used are defined in the following way:
\begin{equation}
    \begin{aligned}
    & \left[ A_u^i\right]_{j,l} & = a_{i,u}^{RR}(\phi_l, \phi_j), \ \ \ \left[ A_v^i\right]_{j,l} & = a_{i,v}^{RR}(\phi_l, \phi_j) \\
    & \left[ B_u^i\right]_{j,l} & = b_{i,u}(\phi_l, \phi_j), \ \ \ \left[ B_v^i\right]_{j,l} & = b_{i,v}(\phi_l, \phi_j)\\
    & \left[ C_u^i\right]_{j,l} & = c_{i,u}(\phi_l, \phi_j),\ \ \ \left[ C_v^i\right]_{j,l} & = c_{i,v}(\phi_l, \phi_j)\\
    & \left[F_u^i\right]_{j} & = f_{i,u}^{RR}(\phi_j), \ \ \
    \left[F_v^i\right]_{j} & = f_{i,v}^{RR}(\phi_j),
    \end{aligned}
\end{equation}
being $\{\phi_l\}_{l = 1}^{N_h}$ the functional basis of $V_{i,h}$ finite dimensional space defined on each cell $\Omega_i$ with $i =1,2$.

A sketch of the procedure to be adopted to deal with a generic N cells pluricellular system using Robin-Robin modifed algorithm is schematically given in Algorithm \ref{alg:RRmod}; $r_i$ are different coefficients characterizing initial state of concentrations, necessary for having flux between communicating cells. For physical reasons, we choose same initial guesses in the direction of the auxin gradient.

We have implemented a solver for a system of four cells.

As expressed in \eqref{eq::RRmod}, the iterative procedure is formulated such that the pluricellular domain is solved sequentially, in the sense that the boundary conditions characterizing sub-domain $i$, depending on sub-domain solutions of $j \in \mathcal{N}_i$, are computed using the newly updated solutions. In view of a parallel implementation, the method can be reformulated such that the new boundary conditions are a function of the previous iteration solution.

\begin{algorithm}[t]
    \caption{Pluricellular system solver procedure: RR}
    \label{alg:RRmod}
    Given $N \geq 1$ cells, $r_i$
    \begin{algorithmic}[1]
    \STATE Initialization: $\forall i = 1, ..., N$
    \STATE \verb|[U0i, V0i]| $\gets [r_i u_0, r_i v_0]$
    \STATE \verb|[Uiprec, Viprec]| $\gets$  \verb|[U0i, V0i]|
    \WHILE{$t < T_{max}$}
    \STATE{\verb|assemble| matrix for $\forall i = 1,..., N$}
    \FOR{$iter < Niter$}
    \STATE{$\forall i =1, ..., N$}
    \STATE{compute BC contribute from $j \in \mathcal{N}_i$}
    \STATE{\verb|interpolate| on $i$}
    \STATE{update \verb|rhs|}
    \STATE{\verb|solve| $\Omega_i$ problem \eqref{eq:pluriModel}}
    \STATE{update residual, check tolerance, update $iter$}
    \STATE \verb|[Uiprec, Viprec]| $\gets$  \verb|[Ui, Vi]|
    \ENDFOR
    \STATE \verb|[U0i, V0i]| $\gets$  \verb|[Ui, Vi]|
    \ENDWHILE
    \end{algorithmic}
\end{algorithm}

\subsection{Convergence check (Simona+Nicola+Teresa+Daniele)}
\subsection{Reliability (Simona+Nicola+Teresa+Daniele)}
\subsection{Eventuali sensitivity (Simona+Nicola+Teresa+Daniele)}

\section{DD per il modello completo (Simona+Nicola+Teresa+Daniele)}

\tb{si intende solo i risultati o anche la discretizzazione?}

\section{Conclusions and future developments (Simona+Daniele)}

\textbf{Conclusions and future developments}%
In this thesis we developed a multi-cellular model accounting for a spatially-extended intra-cellular system for ROPs pattern formation. In the proposed framework, we solved ROPs pattern formation in a system composed by multiple cells, together with a transport model for hormone auxin. The RD model explains the auxin-mediated action of ROPs in an Arabidopsis root hair cell leading to formation of the localized patches of activated ROPs.

Our simulations support conclusions reached in other works regarding ROPs dynamics under a-priori defined auxin distribution \cite{phdthesis:victor, intra1_R, intra2}. We have analyzed various scenarios such that a stripe-like patch forms where auxin concentration is higher. Then, instability of stripe into spot-like states occurs and multiple spots align with auxin gradient or travel towards auxin minimum. Several results confirm that, for a transversally independent gradient, lateral stripes become unstable states.

Successively as a new contribution, we extended the intra-cellular dynamics for root-hair initiation model to a multi-cellular system, developing a new model taking into account communication between cells. In order to do so, we have defined a boundary value problem which assumes new boundary conditions between neighboring cells. Subsequently, we develop an iterative procedure to solve it. We take as reference scheme a Robin-Robin domain decomposition method. We impose fluxes of ROPs betweeen neighboring cells, depending on the difference of the ROPs concentrations, through localised open channels. Such connections are tuned in order to visualize considerably different results with respect to a configuration characterized by stagnant cells. In addition,we aim at preserving all previous analyses over important parameters characterizing the system. We numerically assessed the robustness of the proposed model in cooperating with auxin distribution in influencing ROPs pattern formation.

Having defined a reliable multi-cellular model, we have taken into account also auxin concentration dynamics. Hormone auxin is regulated by carriers PIN following a non-linear ODEs system. We implement a semi-implicit method to solve such as a system on a two cells setting. The simulations we carried out show oscillating values of auxin concentration for specific sets of parameters, as expected from previous studies. A further confirmation of the robustness of pattern formation according to the proposed new multi-cellular model, when considering channel communication between cells, is given in Chapter \ref{cap:4}. It is shown that even if the system is under steady homogeneous auxin concentrations, the model robustly forms hotspots when considering open channels under a sufficiently high overall auxin level.

The results show two ways spots of active ROPs can be generated. The first factor is the auxin gradient, which still guarantees and influence stripe to spot evolution. The second factor is the structural coupling between cells. Even if not characterized by variation in space but with a sufficiently high value of the auxin concentrations, the multi-cellular model leads the system to multiple spots.

In conclusion, this thesis provides a first attempt in modeling communication between root-hair cells in pattern formation. Future developments include to study the structural model or to deeply analyze channels characterization. Moreover, the iterative procedure implemented could be applied to a multi-cellular system composed by more than four cells and eventually try to increase the computational performance through parallel computing. Another possible road to follow is to approach the problem through homogenization techniques. The cited method may increase the efficiency in solving the model, particularly when the number of cells involved becomes large. Finally, one could assume other auxin transport models, less simplified or accounting for spatial dependence inside the cells.

\bibliographystyle{siamplain}
\bibliography{references}
\end{document}


\section{sub1}\label{sec:sub1}

\begin{equation}
  f(x) = 42
  \label{eq:sub-is-working}
\end{equation}


\subsection{Gravitational Body Force Examples}
\label{sec:example:3dhex8:gravity}

PyLith features discussed in this example:
\begin{itemize}
\item Gravitational body forces
\item Initial stresses
\item Finite strain
\item Generalized Maxwell linear viscoelastic material
\end{itemize}

\subsubsection{Overview}

This set of examples describes a set of problems for PyLith involving
gravitational body forces. All of the examples are quasi-static and
run for a time period of 200 years. These examples also demonstrate
the use of a generalized Maxwell viscoelastic material, which is used
for the lower crust in all examples. The final example (step17)
demonstrates the usage of a finite strain formulation, which
automatically invokes the nonlinear solver. All of the examples are
contained in the directory \filename{examples/3d/hex8}, and the
corresponding \filename{cfg} files are \filename{step15.cfg},
\filename{step16.cfg}, and \filename{step17.cfg}.  Run the examples as follows:
\begin{shell}
# Step15
$ pylith step15.cfg

# Step16
$ pylith step16.cfg

# Step17
$ pylith step17.cfg
\end{shell}
This will cause PyLith to read the default parameters in
\filename{pylithapp.cfg}, and then override or augment them with the
additional parameters in the \filename{stepXX.cfg} file. Each
\filename{cfg} file is extensively documented, to provide detailed
information on the various parameters.

\subsubsection{Step15 - Gravitational Body Forces}

The \filename{step15.cfg} file defines a problem with extremely simple
Dirichlet boundary conditions. On the positive and negative x-faces,
the positive and negative y-faces, and the negative z-face, the
displacements normal to the face are set to zero. Because all of the
materials in the example have the same density, the elastic solution
for loading via gravitational body forces is
\begin{equation}
\sigma_{zz}=\rho gh;\:\sigma_{xx}=\sigma_{yy}=\frac{\nu\rho gh}{1-\nu}\:.\label{eq:1-1}
\end{equation}

We set the gravity field, which by default has values of 9.80655
$\unitfrac{m}{s^{2}}$ for acceleration and $\left[0,0,-1\right]$
for direction and time stepping implementation:
\begin{cfg}[Excerpt from \filename{Step15.cfg}]
<h>[pylithapp.timedependent]</h>
<f>gravity_field</f> = spatialdata.spatialdb.GravityField ; Set gravity field

<h>[pylithapp.timedependent.implicit]</h>
# Change time stepping algorithm from uniform time step, to adaptive
# time stepping.
<f>time_step</f> = pylith.problems.TimeStepAdapt

# Change the total simulation time to 200 years, and set the maximum time
# step size to 10 years.
<h>[pylithapp.timedependent.implicit.time_step]</h>

total_time
 = 200.0*year

max_dt
 = 10.0*year

stability_factor
 = 1.0 ; use time step equal to stable value from materials
\end{cfg}

We use a generalized Maxwell model for the lower crust (see Section
\vref{sec:materials:formulation:generalized:Maxwell}), and use a \object{SimpleDB} to
provide the properties. We also request the relevant properties and
state variables for output:
\begin{cfg}[Excerpt from \filename{Step15.cfg}]
# Change material type of lower crust to generalized Maxwell viscoelastic.
<h>[pylithapp.timedependent]</h>
<f>materials.lower_crust</f> = pylith.materials.GenMaxwellIsotropic3D
# Provide a spatial database from which to obtain property values.
# Since there are additional properties and state variables for the
# generalized Maxwell model, we explicitly request that they be output.
# Properties are named in cell\_info\_fields and state variables are named in
# cell\_data\_fields.
<h>[pylithapp.timedependent.materials.lower_crust]</h>

db_properties.iohandler.filename
 = spatialdb/mat\_genmaxwell.spatialdb

output.cell_info_fields
 = [density, mu, lambda, shear_ratio, maxwell_time]

output.cell_data_fields
 = [total_strain, stress, viscous_strain_1, viscous_strain_2, \\
  viscous_strain_3]
\end{cfg}
The boundary conditions for this example are trivial, so we are able
to use the default \object{ZeroDispDB} for all faces. When we have
run the simulation, the output VTK files will be contained in \filename{examples/3d/hex8/output}
(all with a prefix of \filename{step15}). Results using ParaView are
shown in Figure \vref{fig:example:3dhex8:step15:displacement}.

\begin{figure}
  \includegraphics[width=10cm]{examples/figs/3dhex8_step15-displ-t200}
  \caption{Displacement field for example step15 at t = 200 years visualized
    using ParaView. The z-component of the displacement field is shown
    with the color contours, and the vectors show the computed displacements.}
  \label{fig:example:3dhex8:step15:displacement}
\end{figure}

\subsubsection{Step16 - Gravitational Body Forces with Initial Stresses}

The \filename{step16.cfg} file defines a problem that is identical to
example step15, except that initial stresses are used to prevent the
initial large displacements due to 'turning on' gravity. Since all
normal stress components are given an initial stress of $\rho gh$,
the initial stress state is lithostatic, which is an appropriate condition
for many tectonic problems in the absence of tectonic stresses (e.g.,
McGarr \cite{McGarr:1988}). When compared to example step15, this
example should maintain a lithostatic state of stress for the entire
simulation, and displacements should remain essentially zero.

We set the gravity field, as in example step15, and we again use adaptive
time stepping with a generalized Maxwell rheology for the lower crust.
We provide values for the initial stress for both the upper and lower
crust. Since the materials have the same density, we are able to use
the same \object{SimpleDB} with a linear variation for both (see file
\filename{examples/3d/hex8/spatialdb/initial\_stress.spatialdb}):
\begin{cfg}[Excerpt from \filename{Step16.cfg}]
# We must specify initial stresses for each material.
# We provide a filename for the spatial database that gives the stresses,
# and we change the query_type from the default 'nearest' to 'linear'.
<h>[pylithapp.timedependent.materials.upper_crust]</h>
<f>db_initial_stress</f> = spatialdata.spatialdb.SimpleDB

db_initial_stress.iohandler.filename
 = spatialdb/initial_stress.spatialdb

db_initial_stress.query_type
 = linear

<h>[pylithapp.timedependent.materials.lower_crust]</h>
<f>db_initial_stress</f> = spatialdata.spatialdb.SimpleDB

db_initial_stress.iohandler.filename
 = spatialdb/initial_stress.spatialdb

db_initial_stress.query_type
 = linear
\end{cfg}
Note that we use a \texttt{linear} \property{query\_type} rather than
the default type of \texttt{nearest}, so that a linear interpolation
is performed along the z-direction. When we have run the simulation,
the output VTK files will be contained in \filename{examples/3d/hex8/output}
(all with a prefix of \filename{step16}). Results using ParaView are
shown in Figure \vref{fig:example:3dhex8:step16:stress}.

\begin{figure}
  \includegraphics[width=10cm]{examples/figs/3dhex8_step16-stress_xx-t200}
  \caption{Stress field (xx-component) for example step16 at t = 200 years visualized
    using ParaView. Note that for this example, Stress\_xx = Stress\_yy
    = Stress\_zz, and there is no vertical displacement throughout the
    simulation. Also note that the stresses appear as four layers since
    we have used \object{CellFilterAvg} for material output.}
  \label{fig:example:3dhex8:step16:stress}
\end{figure}

\subsubsection{Step17 - Gravitational Body Forces with Small Strain}

The \filename{step17.cfg} file defines a problem that is identical to
example step15, except that we now use a small strain formulation
(see Section \vref{sec:small:strain:formulation}). All of the problems
up to this point have assumed infinitesimal strain, meaning that the
change in shape of the domain during deformation is not taken into
account. In many problems it is important to consider the change in
shape of the domain. This is particularly important in many problems
involving gravitational body forces, since a change in shape of the
domain results in a different stress field. By examining the stress
and deformation fields for this example in comparison with those of
example step15, we can see what effect the infinitesimal strain approximation
has on our solution.

We set the gravity field, as in example step15 and again use adaptive
time stepping withs a generalized Maxwell rheology for the lower crust.
The only change is that we change the problem formulation from the
default \object{Implicit} to \object{ImplicitLgDeform}. Since the
large deformation formulation is nonlinear, PyLith automatically switches
the solver from the default \object{SolverLinear} to \object{SolverNonlinear}.
It is thus only necessary to change the formulation:
\begin{cfg}[Excerpt from \filename{Step17.cfg}]
<h>[pylithapp.timedependent]</h>
# Set the formulation for finite strain. The default solver will
# automatically be switched to the nonlinear solver.
<f>formulation</f> = pylith.problems.ImplicitLgDeform
\end{cfg}
When we have run the simulation, the output VTK files will be contained
in \filename{examples/3d/hex8/output} (all with a prefix of \filename{step17}).
Results using ParaView are shown in Figure \vref{fig:example:3dhex8:step17:displacement}.

\begin{figure}
  \includegraphics[width=10cm]{examples/figs/3dhex8_step17-displ-t200}
  \caption{Displacement field for example step17 at t = 200 years visualized
    using ParaView. The z-component of the displacement field is shown
    with the color contours, and the vectors show the computed displacements.
    Note the larger displacements compared with example step15.}
  \label{fig:example:3dhex8:step17:displacement}
\end{figure}