\chapter{Timestamping}
\label{chpr:timestamping}
Placing a certain date on some data is surprisingly useful \cite{Haber91howto, Bayer93improvingthe, Massias99designof, OTSannouncment}.
An inventor who had a patentable idea or a scientist who came to a relevant conclusion may want to create a verifiable proof that at a certain moment they discovered something. This can help them to protect their intellectual property, proving others their precedence over competing claims.
In a communication protocol having the possibility to attach a certain time to messages can improve the security of the transmission. However it is often difficult to come to an agreement on which is the correct time to use, so different security models yield to different practical schemes.
When storing documents in a third party cloud server, it could behave maliciously, for instance modifying their contents. Placing a certain and tamper resistant date on a document will make harder for the server provider to corrupt that file: the attacker should also be able to falsify the certificate stating the date.

Other practical applications of this technique are possible, however to have a complete understanding of the subject it is important to figure out which are its limits and which are the right choices to take to correctly put the concepts into practice. So we need to be a little more formal,
\begin{mydef}
	A timestamp is a proof that some data $d$ existed prior to time $t$.
\end{mydef}
To create such proof, $d$ has to cause an event that could not have been generated without the existence of $d$. Such event it is bound to time $t$ and can be observed by others, we call its record \textit{time attestation}. So a proof consists in the data $d$, the set of \textit{operations} that were applied to cause the event and the time attestation.

Proofs are useful if they are able to convince the verifiers. He must be able to check the correctness of the operations and must retain trustworthy the time attestation. Depending on the problem in exam one should properly choose which operations and attestations to use.

Let's consider the case of a sent letter. The data $d$ is the content of the letter, $d$ caused the palpable letter: if $d$ were different the letter would be different. When the letter went through the post office a postmark with the date $t$ was stamped on the letter, the postmark is the time attestation. If the post officer is trustworthy, a verifier who examines the letter may be convinced that the content of the letter existed prior to the time stated in the postmark. However a good counterfeiter could change the content of the letter or falsify the postmark placing a false date, thus for some cases such proof would not be appropriate.

In the case of a digital document new problems arises: it is not necessary to be a good counterfeiter to falsify a document without leaving any kind of tamper evidence, thus new solutions must be adopted. A timestamp for the data $d$ should guarantee that if even only a single bit of $d$ is modified the timestamp proof becomes invalid. To solve this problem cryptography tools are used \cite{Haber97securenames}. Digital data can also be shared easily and with little costs, this is among the features that enables the possibility to achieve distributed and decentralized consensus. Such an achievement would give user the chance to timestamp without any trust in a third party.

Resuming, at a certain moment $t_e$ some data $d$ exists, then at $t_c$ someone (or something) will have the necessity to prove the existence of $d$, so he implements the timestamp \textit{creation procedure} that results in a proof stating that $d$ existed priort to time $t$. Consequently at time $t_v$ a challenger implements the \textit{verification procedure} that ends in a binary result: true if he retains the proof correct, false otherwise. Naturally we have $t_e<t_c<t<t_v$.

To avoid common misunderstandings, it is important to clarify what a timestamp does not prove. The time $t$ is the first moment when $d$ went to existence, the creation procedure is not instantaneous and of course if the proof for $(d,t)$ is true then there exists a proof for $(d,t')$ which holds true for all $t'>t$. A timestamp is not necessarily linked to its creator and moreover it doesn't prove that who owns the timestamp (it can be owned by multiple entities) is the creator of the data $d$: it just proves that someone knew $d$. If an inventor comes up with a timestamp stating that he had a particularly smart idea prior to time $t$, it does not mean that he was the first one to have such idea, in fact he could have simply overheard the idea from a colleague and afterwards timestamped it. A timestamp does not prove that data $d' \neq d$ does not exist. Imagine someone stating that he knew the result of the elections prior to the vote, he provides a timestamp and claims that it proves he predicted the correct outcome. However he could be an imposter: he may had timestamped several different results and once he saw how the vote count ended he will provide only the proof which make him look as a visionary.
Although these limitations being able to timestamp is still useful. If stronger proofs are needed the used system must be endowed with other tools that actually provides what is asked.

In the following sections we analyse the two ingredients of timestamp proofs: operations and attestations.

\section{Commitment Operations}

The operations that compose a timestamp proof should be defined in a precise way, a verifier will check their correctness when evaluating the proof. An useful operation binds the data in a way that it is hard or impossible to modify the data after the creation of the timestamp. Being more general, such an operation commits the input to the output: the input cannot be changed without changing the output. The result of the operation is a commitment to the input, in the sense that it was caused by the input or, in other words, the input precede in time the output.
For instance in the case of a sent letter, the input is the content of the letter, the piece of paper that is sent is the output and physically writing the letter is the operation that commits the input to the output. We can say that a letter is a physical commitment to its content. If one wants to modify the content of the letter ex post it will change the letter itself. However, tamper evidence could be extremely hard to spot.

Digital document are easier to tamper, but they could be defined in a more precise way, representing each document with bits. To take full advantage of this, we need a formal definition,
\begin{mydef}
	A function $C:X \rightarrow Y$ is a commitment operation if given $x_1 \in X$ it is not feasible to compute $x_2 \in X$ s.t. $x_1 \neq x_2, C(x_1)=C(x_2)$.
\end{mydef}
The property required is sometimes referred as second pre-image resistance. For practical purposes $X$, $Y$ can be thought as bit string spaces, their element can be seen as bit strings or another of their representations, for instance in hexadecimal digits or using a conventional encoding.
The simplest examples are the append and prepend operations:

\begin{myexample}
	\textquotedblleft hello\textquotedblright, \textquotedblleft world\textquotedblright $\xrightarrow{\text{append()}}$ \textquotedblleft helloworld\textquotedblright.
\end{myexample}
It is impossible to change the inputs without changing the output: the output contains the inputs themselves.
In the example the function is a binary operation, however, by fixing one input, it is possible to turn these commitment operations into unary operations:
\begin{myexample}
	\textquotedblleft world\textquotedblright $\xrightarrow{\text{prepend(\textquotedblleft hello\textquotedblright)}}$ \textquotedblleft helloworld\textquotedblright.
\end{myexample}
Append and prepend have two problems: they reveal everything about the inputs and the size of the output is always greater than the size of the inputs.

Hiding the input of a commitment is often useful. For instance Robert Hooke \cite{Petroski96invention} in 1676 had formulated the spring law that will take his name. He wanted to prove that he knew that without revealing the law itself, so he published the latin anagram \textquotedblleft ceiiinosssttuv\textquotedblright. Later in 1678 he revealed the solution, \textquotedblleft ut tensio, sic vis\textquotedblright (\textquotedblleft as the extension, so the force\textquotedblright). However the anagram is not an optimal commitment operation: \textquotedblleft ut vis, sic tensio\textquotedblright is a solution too. For that time it was an acceptable solution, the verification procedure had minimal requirements and there was no real incentive in lying. Nowadays better solutions are available, hence using an anagram will make the verifier suspicious.

To address these problems cryptographic hash functions are used. Hash functions maps bit strings of arbitrary finite length into bit strings of fixed length \cite{Damgard:1989:DPH:118209.118248}.
\begin{mydef}
	$h : \{ 0, 1 \} ^* \rightarrow \{ 0, 1 \} ^n $ is a hash function if it is computable in polynomial time in the length of the input.
\end{mydef}
We will refer at the input of such functions as preimage, and the output as hash value. The codomain is strictly contained in the domain hence the presence of collisions (pairs of inputs with identical outputs) is unavoidable.
To give to these functions a practical use some of the following properties are required
\begin{mydef}
	\label{hash-prop}
	Let $h$ be a hash function, the following properties may hold:
	\begin{itemize}
		\item preimage resistance: given $h(x)$ it is not feasible to compute $x$;
		\item second-preimage resistance: given $x$ it is not feasible to compute $y$ s.t. $x \neq y$, $h(x)=h(y)$;
		\item collision resistance: it is not feasible to find $x, y$ s.t. $x \neq y$, $h(x)=h(y)$.
	\end{itemize}
\end{mydef}

A preimage resistant hash function hides the inputs, this property is sometimes referred as one-wayness.
A second-preimage resistant hash function is also a commitment operation, in addition it hides the input and create a fixed size fingerprint. A huge file can be mapped into a short bit string which is a commitment to it.
Collision resistance, although is a nice to have, is not necessary for timestamping purposes: when a collision ($x \neq y$ s.t $h(x)=h(y)$) is found, $x$ and $y$ are found at the same time \cite{SHA1}.

Such properties depend on the dimension of the codomain, which is given by the parameter $n$, precisely $2^n$. If it is too low it will be easy to produce examples that invalidates the properties, if it is too high the hash function is not a good tool for reducing the size of the input. A good compromise has to be found, it should be based on the current state of the art of cryptanalysis and computer science. However what is considered acceptable during a period may not be accepted in a subsequent one, in fact if new techniques that break an hash function are discovered, then its use will be considered insecure.
Several hashing functions has been proposed, we show a couple of examples:
\begin{myexample}
	SHA1 (Secure Hashing Algorithm) was designed by the National Security Agency (NSA), and released in 1995. It maps bit strings to a 160 bit space, it was considered preimage, second-preimage and collision resistant. In 2005 cryptanalysts discover a theoretical procedure to find collisions, later practical attacks were published \cite{cryptoeprint:2017:190}, hence it was declared insecure. However only collision resistance was broken, thus it can still be used for timestamping purposes.
	\begin{verbatim}
	SHA1(b'Hello World!\n') = a0b65939670bc2c010f4d5d6a0b3e4e4590fb
	                          92b
	SHA1(b'Hello World\n')  = 648a6a6ffffdaa0badb23b8baf90b6168dd16
	                          b3a
	\end{verbatim}
\end{myexample}

\begin{myexample}
	SHA256 belongs to the family SHA2, the generation of hash functions following SHA1\footnote{Currently, it is available SHA3, which contains, among others, KECCAK256.}. It outputs  strings 256 bit long and, at the moment, it is considered to satisfy all the properties in Definition \ref{hash-prop}.
	Several systems are built upon this assumption.
	In addition, as other hash functions, SHA256 could be modeled as a random oracle, a fixed input will provide always the same output, since hash functions are deterministic, but the outputs corresponding to new inputs will give results that are indistinguishable from a uniform distribution.
	Note how a little change in the input produces outputs very dissimilar in both hash functions, this feature helps to spot alterations in the inputs.
	\begin{verbatim}
	SHA256(b'Hello World!\n') = 03ba204e50d126e4674c005e04d82e84c21
	                            366780af1f43bd54a37816b6ab340
	SHA256(b'Hello World\n')  = d2a84f4b8b650937ec8f73cd8be2c74add5
	                            a911ba64df27458ed8229da804a26
	\end{verbatim}
	Since SHA256 is available, the use of SHA1 should be avoided unless there is a particular motivation.
\end{myexample}

More complex commitment operations are possible.
But they should be used only if there are some valid and shared motivations. Finding a badly motivated commitment operation in a proof will make verifiers suspicious.

Combining the examples we saw it is possible to implement a data structure that is useful to embed several commitments into a single hash value,
this structure is called \textit{Merkle tree} \cite{DBLP:conf/sp/Merkle80}. Let $h$ be a second-preimage resistant hash function, start from $k$ data to timestamp $\{d_i\}_{i=1}^k$, called \textit{leaves}, compute their hash values $\{h(d_i)\}_{i=1}^k=\{h_i\}_{i=1}^n$. The first step combines the couples of adjacent values (if there is one) by concatenating ($||$) the two values and computing the hash value of the concatenation: $h(h_i||h_{i+1})$ $\forall i \in [1,k]$ odd. The following steps proceed in an analogous manner, but starting from the results computed at the step before as described in Algorithm \ref{alg:merkle-tree}.

\begin{algorithm}
	\caption{Merkle tree construction}
	\label{alg:merkle-tree}
	\begin{algorithmic}[1]
		\Procedure{merklestep}{$\{h_i\}_{i=1}^k$}
		\State $S \gets \{\}$
		\For {$i\gets 1,k$ s.t. $i\equiv 1 (2)$}
		\State $S \gets S \cup \begin{cases}
		h(h_i||h_{i+1}) & i \neq k \\
		h(h_i) & i=k
		\end{cases}$
		\EndFor
		\State \textbf{return} $S$
		\EndProcedure
		\Statex
		\Procedure{merklelize}{$\{d_i\}_{i=1}^k$} \Comment{from the leaves to the Merkle tip}
		\State $S \gets \{\}$
		\For {$i\gets 1, k$}
		\State $S\gets S \cup h(d_i)$\Comment{hash the leaves}
		\EndFor
		\State $k\gets |S|$
		\While {$k \neq 1$}
		\State $S \gets$ \Call{merklestep}{S} \Comment{if necessary, store $S$}
		\State $k\gets |S|$
		\EndWhile
		\State \textbf{return} $S$ \Comment{Merkle tip}
		\EndProcedure
	\end{algorithmic}
\end{algorithm}

After at most $\lceil log(k) \rceil$ steps the algorithm ends and one hash value is returned, it is called \textit{Merkle tip} or \textit{Merkle root}.
Thanks to reiteration of $h$ the tip is a commitment to all the leaves $\{d_i\}_{i=1}^k$, moreover to prove that a element $d_i$ is committed it is not necessary to know all tree, in fact it's enough to show the \textit{Merkle path} starting from $d_i$ and ending in the tip.
For instance referring to Figure \ref{fig:merkle-tree} the minimal requirement to show the commitment of $data3$ in $MT$ is the ordered sequence of operations to apply is:
\begin{verbatim}
h, append(h4), h, prepend(A), h, append(E), h
\end{verbatim}
Merkle trees can aggregate several commitments into one hash value, giving a scalability solution: the length of a single proof grows logarithmically with the number of leaves.

\begin{figure}
	\begin{center}
		\includegraphics[width=\linewidth]{Images/merkle-tree.png}
		\caption[Merkle tree example]{Merkle tree example. The Merkle root is a commitment to the leaves $\{data_i\}_{i=1}^5$.}
		\label{fig:merkle-tree}
	\end{center}
\end{figure}

\section{Time Attestations}
Attestations are provided by a notary, which has the authority to state the time.
Notaries are sometimes referred as timestamp servers, who ask for creating or verifying that a timestamp is referred as client.
A time attestation binds some data $d$ to the time $t$, $d$ could be directly the data to timestamp or a commitment to them.
To properly design a timestamp scheme it is crucial to analyse which features an attestation should have and how to choose a notary that provides the desired security.

An attestation should be \textit{tamper resistant} or, even better, \textit{immutable}. Once created no one should be able to modify that, backdating the timestamp or changing the underlying data without making the attestation invalid.

In the case of a sent letter an important issue comes to the attention: the attestation (postmark) can be verified only if the letter is in our hands. This makes the task of a counterfeiter easier, few people will examine the postmark making the success of the attack more likely. A solution that mitigates this issue is \textit{widely publishing} the attestation, for instance inserting it in a newspaper. The attacker has an harder task, in most cases, to guarantee himself good chances of success he needs to modify several, or even all, attestations leading to a higher cost. Considering an attestation on a newspaper, an attacker could counterfeit only the exact copy the verifier is going to check. To prevent such situation the verifier should check multiple sources and should not expose any information on where he is retrieving the copies used to examine. Attestations should be \textit{easily accessible}, for instance a good solution is to publish them on the internet.

We call the service providing the attestations notary, it has the authority to state the time. To maintain such power it has to show itself as \textit{trustworthy}: it should place the correct time in the attestations, it should not trick the clients modifying the timestamp ex post and he should not collude with the creator of the timestamp to trick a verifier. Moreover a notary should be \textit{competent}: if it looses the information necessary to verify the timestamp, then its creator will be damaged.

A good notary does not make distinctions among clients or data to timestamp, in other words it does \textit{not censor}. An improvement would be if it is not aware of what it is timestamping, like the case of the commitments hiding their inputs, but even better if it is not aware of being used as a timestamp server, like the newspaper.

In the case of a sent letter, the notary is the post officer, he will place the postmark on the letter giving it a date.
Clients must trust such notary, which in theory can place a wrong date $t'$ instead of $t$, creating a false timestamp (if $t'<t$) or a deliberate weaker proof (if $t'>t$). Alternately he can decide not to timestamp a letter or modifying its content before timestamping. In addition the resulting timestamp is unique, thus if it gets damaged or lost the proof is gone forever.

A notary willing to show himself as more trustworthy can make its possibly dishonest behaviour harder to implement. A possible solution is to link all the timestamps so that changing one would implies changing all the subsequent ones. A timestamp proof $TS_i$ for the data $d_i$ at the time $t_i$ ($t_{i-1}<t_i<t_{i+1}$) would have the following structure:
\begin{equation}
	\label{singed-chain}
	\begin{cases}
		TS_0 =(d_0,t_0, \sigma_N((d_0,t_0))) &
		\\
		TS_i =(d_i,t_i,TS_{i-1}, \sigma_N((d_i,t_i,TS_{i-1}))) & i = 1, ..., i_{max}
	\end{cases}
\end{equation}
where $\sigma_N (x)$ is a commitment to the data $x$. Such commitment is created by the notary, who is (supposedly) the only one who is able to apply $\sigma_N$. We call this function \textit{signature}, it can be physical (as a postmark) or digital (with a public key cryptosystem). The signature $\sigma_N$ is a commitment operation, so if $TS_{i}$ changes then $\sigma_N((d_{i+1},t_{i+1},TS_i))$ changes, thus $TS_{i+1}$ changes resulting in $TS_j$ changing for all $j>i$. Starting from the \textit{genesis timestamp} $TS_0$, the linked timestamps $\{ TS_{i} \}_{i=0}^{i_{max}}$ form a \textit{chain}. A trusted notary which implement this scheme will be preferred by honest clients. Depending on each particular problem the designer of the system has to properly choose how to collect the timestamp requests, how to commit them into the chain, the time frequency, the signature procedure, how to publish the chain and how to distribute the timestamp proof to the clients.

Even though linking the timestamps is a great improvement with respect to the previous schemes, having a trusted notary still involves some issues. The notary may loose what he needs for signing (e.g. stamp, private key); if an attacker can sign in the place of the notary, it will no longer be considered authoritative; if no one can sign the clients will be damaged. In addition the notary can create more than one chain starting from the same genesis timestamp, then he can use different chains for different clients.
Anyhow nothing can guarantee that a trusted notary will behave honestly in the future, it can attack some clients in several ways and such bad behaviour can be hard to spot rapidly, a single client may realize that he has been tricked only when he needs the timestamp.

Trusted attestations has security issues hence, in some situations, they could be considered not enough appropriate, yet for years an efficient solution that does not involve a trusted third party was considered barely impossible.
In 2008 Satoshi Nakamoto proposed \cite{Nakamoto_bitcoin:a} which described a system to transfer value from one party to another without relying on the presence of a central authority. The system can be also used to timestamp arbitrary data without trusting any notary.


\lispy/ is statically typed. This means type safety (i.e.\ that there are no discrepancies between expected types of values and actual types of values) is verified at compile time by analysing the source code. The language has  a small set of types and a limited (but simple) type system. The language has type inference for everything except function parameters, meaning it can determine the type of a value at compile time without relying on being explicitly told the type by the programmer.

\subsection{int and float}
There are both integer and rational numbers. The former are \texttt{int}s and the latter are \texttt{float}s (a fixed-point representation). The syntax for them is a bit laxer than it is in LISP 1.5 (see~\nameref{sec:syntax}). However, octal number literals are unsupported. There are also \texttt{float} constants for \texttt{inf} (floating-point positive infinity) and \texttt{nan} (``not a number'').

The behaviour of precision and overflow is implementation-defined. For the reference implementation, these types are implemented using Python's \texttt{int} and \texttt{float} types, so they are subject to the same limitations in \lispy/ as they are in Python. That further depends on the Python interpreter used to run the compiled code.

\subsection{bool}
The \texttt{bool} type is represented by the literal atoms \texttt{true} and \texttt{false}. This is a replacement for the \texttt{T} and \texttt{NIL} used by predicates in LISP 1.5. Having a single Boolean type makes it simple from a type checking perspective to implement predicates in \lispy/.

% TODO: mention that recursion is possible.
\subsection{func} \label{subsec:func}
A \texttt{func} is a function as described in~\nameref{subsec:simpleforms}. The fundamental way to create a function is with a special form know as a \textit{lambda expression}. The resulting function is called a \textit{lambda function}. The distinction from just \textit{function} is that a lambda function is not bound to any name. Thus, to evaluate a lambda function, a composed form has to be used. In fact, the use of a lambda expression in place of a function name is what was being alluded to briefly in the description of~\nameref{subsec:composedforms}. As with function names, the lambda expression is evaluated first in the form, and then the arguments are evaluated.

\begin{figure}[htp]
    \centering
    \begin{cminted}[autogobble=true, escapeinside=??]{lisp}
        (?\tikzmark{exp_start}?(lambda ?\tikzmark{param_start}?((?\tikzmark{atom_start}?x?\tikzmark{atom_end}? int) (y ?\tikzmark{type_start}?int?\tikzmark{type_end}?))?\tikzmark{param_end}? ?\tikzmark{form_start}?(div x y)?\tikzmark{form_end}?)?\tikzmark{exp_end}? ?\tikzmark{arg_start}?1 2?\tikzmark{arg_end}?)
    \end{cminted}
    \begin{tikzpicture}[
        remember picture,
        overlay,
        thick,
        font=\scriptsize,
        every node/.style={pos=0.5, black}
    ]
        \draw[decorate, decoration={calligraphic brace, mirror, raise=0.25em, amplitude=1pt}]
            (pic cs:atom_start) -- (pic cs:atom_end)
            node[below=0.5em]{atomic literal};

        \draw[decorate, decoration={calligraphic brace, mirror, raise=0.25em, amplitude=3pt}]
            (pic cs:type_start) -- (pic cs:type_end)
            node[below=0.5em]{type};

        \draw[decorate, decoration={calligraphic brace, mirror, raise=1.75em, amplitude=3pt}]
            (pic cs:param_start) -- (pic cs:param_end)
            node[below=2em]{parameters};

        \draw[decorate, decoration={calligraphic brace, mirror, raise=0.25em, amplitude=3pt}]
            (pic cs:form_start) -- (pic cs:form_end)
            node[below=0.5em]{body};

        \draw[decorate, decoration={calligraphic brace, mirror, raise=0.25em, amplitude=3pt}]
            (pic cs:arg_start) -- (pic cs:arg_end)
            node[below=0.5em]{arguments};

        \draw[decorate, decoration={calligraphic brace, mirror, raise=3.25em, amplitude=5pt}]
            (pic cs:exp_start) -- (pic cs:exp_end)
            node[below=3.5em]{lambda expression};
    \end{tikzpicture}
    \captionsetup[figure]{font=small, skip=5em}
    \captionof{figure}{A lambda function being evaluated.}
\end{figure}

A function's evaluated value (the \textit{return value}) is the result of evaluating the function body with the function variables, which are bound to some values (the \textit{arguments}) prior to evaluation.

Function parameters are defined as pairs, of which there can be 0 or more. The first element of the pair is the name of the parameter. The second element is the type of that parameter. All parameters must be specified with their types; they are never type-inferred. Note that neither element of this pair is evaluated (this is possible because a lambda expression is a special form).

Unlike parameters, the return value and arguments \textit{will} be type inferred. In fact, there is no mechanism in the language for explicitly specifying the type of a return value or argument.

\subsubsection{First-class Functions}
Functions are \textit{first-class citizens} in the language. This means that, like other values, a function can be passed as an argument to another function, can be returned from other functions, and can be assigned to variables. As a consequence, \textit{higher-order functions} are supported, which are functions that takes a function as an argument and/or return a function.

\begin{figure}[htp]
    \centering
    \begin{cminted}[autogobble=true]{lisp}
        (lambda ((f (func (int int) int))
                 (i int))
          (f 3 i)
        )
    \end{cminted}
    \captionsetup[figure]{font=small}
    \captionof{figure}{A higher-order function that takes a function \texttt{f} as an argument. \texttt{f} has two \texttt{int} parameters and an \texttt{int} return value.}
\end{figure}

\paragraph{Special Forms}
Unlike regular functions, special forms are not first-class citizens. However, this can be worked around by defining a lambda function which ``wraps'' the special form.

\begin{figure}[htp]
    \centering
    \begin{cminted}[autogobble=true]{lisp}
        ((lambda ((f (func (int int) int))
                 (i int))
          (f 3 i))
        (lambda ((a int) (b int)) (prod a b)) 2)
    \end{cminted}
    \captionsetup[figure]{font=small}
    \captionof{figure}{A lambda function wrapping a special form and being passed to another function. This is effectively doing $3 \times 2 = 6$.}
\end{figure}

\subsubsection{Named Functions}
Lambda functions are not inherently bound to any variable. However, it can be quite useful to define a function so it can be re-used later. Since lambda functions are first-class citizens, it is valid to assign a lambda expression to a variable.

One way to do this has actually already been shown: rely on the binding mechanism of a lambda expression. A lambda can be bound to another lambda's parameters, and then the lambda will be accessible via a variable within the other lambda. However, this is tedious. A more convenient way to define functions with names is to use the \texttt{let} special form. This will be discussed in~\nameref{sec:bindingassign}.

\subsection{list}
The \texttt{list} type stores a homogenous (i.e.\ all of the same type) sequence of elements. Lists always have \texttt{nil} as the final element.

Lists can be created with the special form \texttt{cons}. For example, \texttt{(cons 1 (2 (3 nil)))} creates the list \texttt{(1 2 3)} (the presence of \texttt{nil} is implicit). Notice that the second argument must be a list which holds elements that have the same type as the first argument. Alternatively, the special form \texttt{list}, which takes an indefinite number of arguments, can create the same list: \texttt{(list 1 2 3)}.

The first element of a list can be retrieved using the special form \texttt{car}. For example, \texttt{(car (list 1 2 3))} returns the value \texttt{1}. Conversely, \texttt{cdr} can be used to retrieve the remaining list: \texttt{(cdr (list 1 2 3))} returns the value \texttt{(2 3)}. Both \texttt{car} and \texttt{cdr} only accept one argument, which must be a \texttt{list}.

\begin{figure}[htp]
    \centering
    \begin{cminted}[autogobble=true]{lisp}
        (lambda ((n int) (l (list int)))
          (prod n (car l)))
    \end{cminted}
    \captionsetup[figure]{font=small}
    \captionof{figure}{A lambda with a parameter that's a \texttt{list} of \texttt{int}s.}
\end{figure}

\subsubsection{nil}
The literal atom \texttt{nil} represents an empty list. Thus, it also has a type of \texttt{list}. In fact, an alternative way to represent \texttt{nil} is \texttt{()}. For \texttt{nil}, \texttt{car} and \texttt{cdr} are unsupported -- attempting to do so will result in a runtime error.

\texttt{nil} is a built-in ``global'' variable that's accessible everywhere. However, it's special as far as variables go because it cannot be assigned a new value.


\documentclass[letterpaper,final,12pt,reqno]{amsart}

\usepackage[total={6.3in,9.2in},top=1.1in,left=1.1in]{geometry}

\usepackage{times,bm,bbm,empheq,fancyvrb,graphicx,amsthm,amssymb}
\usepackage[dvipsnames]{xcolor}
\usepackage{longtable}
\usepackage{booktabs}

\usepackage{tikz}
\usetikzlibrary{decorations.pathreplacing}

\usepackage[kw]{pseudo}
\pseudoset{left-margin=15mm,topsep=5mm,idfont=\texttt}

\usepackage{float}

% hyperref should be the last package we load
\usepackage[pdftex,
colorlinks=true,
plainpages=false, % only if colorlinks=true
linkcolor=blue,   % ...
citecolor=Red,    % ...
urlcolor=black    % ...
]{hyperref}

\renewcommand{\baselinestretch}{1.05}

\allowdisplaybreaks[1]  % allow display breaks in align environments, if they avoid major underfulls

\newtheoremstyle{cstyle}% name
  {5pt}% space above
  {5pt}% space below
  {\itshape}% body font
  {}% indent amount
  {\itshape}% theorem head font
  {.}% punctuation after theorem head
  {.5em}% space after theorem head
  {\thmname{#1}\thmnumber{ #2}\thmnote{ (#3)}}% theorem head spec
\theoremstyle{cstyle}

\newtheorem{theorem}{Theorem}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{assumptions}[theorem]{Assumptions}

\newtheoremstyle{cstyle*}% name
  {5pt}% space above
  {5pt}% space below
  {\itshape}% body font
  {}% indent amount
  {\itshape}% theorem head font
  {.}% punctuation after theorem head
  {.5em}% space after theorem head
  {\thmname{#1}}% theorem head spec
\theoremstyle{cstyle*}
\newtheorem{assumptions*}{Assumptions}

\newtheoremstyle{dstyle}% name
  {5pt}% space above
  {5pt}% space below
  {}%{\itshape}% body font
  {}% indent amount
  {\itshape}% theorem head font
  {.}% punctuation after theorem head
  {.5em}% space after theorem head
  {\thmname{#1}\thmnumber{ #2}\thmnote{ (#3)}}% theorem head spec
\theoremstyle{dstyle}

\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}

% numbering
\numberwithin{equation}{section}
\numberwithin{figure}{section}
\numberwithin{table}{section}
\numberwithin{theorem}{section}

\newcommand{\eps}{\epsilon}
\newcommand{\RR}{\mathbb{R}}

\newcommand{\grad}{\nabla}
\newcommand{\Div}{\nabla\cdot}
\newcommand{\trace}{\operatorname{tr}}

\newcommand{\hbn}{\hat{\mathbf{n}}}

\newcommand{\bb}{\mathbf{b}}
\newcommand{\be}{\mathbf{e}}
\newcommand{\bbf}{\mathbf{f}}
\newcommand{\bg}{\mathbf{g}}
\newcommand{\bn}{\mathbf{n}}
\newcommand{\br}{\mathbf{r}}
\newcommand{\bu}{\mathbf{u}}
\newcommand{\bv}{\mathbf{v}}
\newcommand{\bw}{\mathbf{w}}
\newcommand{\bx}{\mathbf{x}}
\newcommand{\bF}{\mathbf{F}}
\newcommand{\bV}{\mathbf{V}}
\newcommand{\bX}{\mathbf{X}}
\newcommand{\bxi}{\bm{\xi}}
\newcommand{\bzero}{\bm{0}}

\newcommand{\cK}{\mathcal{K}}
\newcommand{\cV}{\mathcal{V}}

\newcommand{\rhoi}{\rho_{\text{i}}}

\newcommand{\ip}[2]{\left<#1,#2\right>}

\newcommand{\mR}{R^{\bm{\oplus}}}
\newcommand{\iR}{R^{\bullet}}

\newcommand{\nn}{{\text{n}}}
\newcommand{\pp}{{\text{p}}}
\newcommand{\qq}{{\text{q}}}
\newcommand{\rr}{{\text{r}}}

\newcommand{\supp}{\operatorname{supp}}
\newcommand{\Span}{\operatorname{span}}

\begin{document}
\title[On multilevel constraint decomposition methods]{On multilevel constraint decomposition methods \\ for nonlinear variational inequalities}

\author{Ed Bueler}

\date{\today}

\begin{abstract} FIXME
\end{abstract}

\maketitle

%\tableofcontents

\thispagestyle{empty}
%\bigskip

\newfloat{pseudofloat}{t}{xyz}[section]
\floatname{pseudofloat}{Algorithm}

\section{Introduction} \label{sec:intro}

The goal of this paper is to generalize the constraint decomposition (CD) method of X.-C.~Tai \cite{Tai2003} to nonlinear VI problems to which it has not been previously applied.  The convergence of this method, for several types of decompositions, was proven by Tai for coercive variational inequality (VI) problems which arise from minimization of a convex functional over a convex set.  In its multilevel form the method has been shown to have optimal complexity for elliptic, linear obstacle problems \cite[Subsection 5.4]{Tai2003}; see also \cite[Theorem 4.6 and Algorithm 4.7]{GraeserKornhuber2009}.  We construct a new proof of convergence which removes the assumption that the problem arises from (constrained) optimization.  Furthermore we address its finite element (FE) implementation in nonlinear cases, and we demonstrate the resulting performance.

The algorithms and theory presented in \cite{Tai2003} are extended in four particular directions:
\renewcommand{\labelenumi}{\emph{(\roman{enumi})}}
\begin{enumerate}
\item We do not assume that the continuum VI problem arises from optimization of a scalar objective.  Nonetheless we can prove convergence in $H^1$ norm at the same rate as the original method converges in energy (Section \ref{sec:convergence}); % HOPE
examples are explored in Sections \ref{sec:vi} and \ref{sec:results}.

\item We make the observation that, if implemented appropriately, multilevel ``up-smoothing'' is intrinsically more efficient than ``down-smoothing'' (Section \ref{sec:multilevel}).  This observation seems to be new; compare the comments on V(1,0) and V(1,1) cycles in \cite{GraeserKornhuber2009,Tai2003}.  A strong preference for up-smoothing is, apparently, special to multilevel CD methods and does not arise in corresponding unconstrained problems.

\item We make the multilevel CD algorithm more practical by addressing efficient storage of intermediate quantities.  In Section \ref{sec:results} we show results from a full approximation storage (FAS; see \cite{Brandt1977}) implementation for nonlinear operators, one which avoids global (Newton) linearization; compare \cite{GraeserKornhuber2009}.

\item For problems of porous-media type the nonlinear operator $f$ is not coercive (Appendix A), so the convergence theory in \emph{(i)} does not apply.  However, a full-cycle implementation of the multilevel CD method is demonstrated numerically.  By ``freezing' the solution-dependent coefficient, the operator is approximated by a coercive operator, nonlinear in general, for the duration of the V-cycle.  The resulting scheme is highly-effective for doubly-nonlinear diffusion operators (Section \ref{sec:results}).
\end{enumerate}

The iterates from the CD method are always admissible, and thus the operator need only be defined for admissible states; Section \ref{sec:results} includes a nontrivial example.  Admissible-iterate methods should permit direct solutions of certain VI problems, such as fluid-layer dynamics problems \cite{Bueler2021conservation,JouvetBueler2012}, for which non-admissible methods, such as semi-smooth methods \cite{BensonMunson2006}, require unnatural modifications of the operator formula.  The full-cycle scheme in \emph{(iv)} above, which applies to non-coercive VIs, is also designed for these problems, which are characterized by the solution of an auxiliary PDE on a domain determined inside the VI residual evaluation.

Especially for the classical obstacle problem with a Laplacian operator, certain multilevel techniques are known to improve performance relative to the multilevel CD method \cite{GraeserKornhuber2009}.  These improved methods either track the active set in the discretization or modify the nodal basis functions, and in this sense they are discrete algorithms.  By contrast the CD method applies at the level of the continuum problem (Sections \ref{sec:cd} and \ref{sec:convergence}), and thus it is a more fundamental method than the truncated monotone multigrid method \cite{Kornhuber1994}, for example.  In any case, acceleration of our nonlinear multilevel CD method via active-set and/or basis-level manipulations represents a potential extension of the method here, though as a topic for future research.

% A BRIDGE TOO FAR:  In one example at the end of this paper (Section \ref{sec:resultsnonlocal}) we consider a nonlocal residual functional, that is, one which is not a partial differential operator.  Each evaluation of this functional requires the solution of a Stokes problem for a layer of fluid \nocite{SayagWorster2013} on a substrate (which forms the obstacle), and thus the corresponding FE operator discretization is also not sparse.  In this case we cannot prove coercivity but we nonetheless succeed in demonstrating near-optimal complexity of the Section \ref{sec:multilevel} multilevel CD algorithm in practice.

\section{Coercive variational inequalities} \label{sec:vi}

Suppose $\cV$ is a real, reflexive Banach space with norm $\|\cdot\|$ and topological dual space $\cV'$.  Denote the dual pairing of $\phi \in \cV'$ and $v\in\cV$ by $\ip{\phi}{v} = \phi(v)$, and note that $\|\phi\|_{\cV'} = \sup_{\|v\|=1} |\ip{\phi}{v}|$ defines a (Banach space) norm on $\cV'$.

Let $\cK \subset \cV$ be a nonempty closed and convex subset, the \emph{constraint set}; elements of $\cK$ are said to be \emph{admissible}.  For a continuous, but generally nonlinear, operator $f:\cK \to \cV'$ and \emph{source} $g\in \cV'$ we consider the following \emph{variational inequality} (VI) for the (exact) solution $u^*\in \cK$, if it exists:
\begin{equation}
\ip{f(u^*)}{v-u^*} \ge \ip{g}{v-u^*} \qquad \text{for all } v\in \cK. \label{eq:vi}
\end{equation}
Because $f$ is a (generally) nonlinear map, $g$ is not strictly needed when posing this VI, but its presence is helpful to the algorithms of Section \ref{sec:multilevel}.  (Note that by redefining $f$ we may take $g=0$.)

VI \eqref{eq:vi} generalizes the nonlinear system of equations $f(u^*)=g$ from a vector space $\cV$ to the constrained case (over $\cK$).  Informally, if we conceptualize the dual pairing as an inner product then \eqref{eq:vi} says that the angle between $f(u^*)-g$ and any arbitrary vector $v-u$ pointing from $u$ into $\cK$ is at most $90^\circ$.  That is, \eqref{eq:vi} says that $f(u^*)-g$ may not be zero but it points directly into $\cK$.  In particular, if $u^* \in \cK^\circ$ (interior) then \eqref{eq:vi} implies $f(u^*)=g$.  The following definitions are also standard \cite{KinderlehrerStampacchia1980}.

\begin{definition} A map $f:\cK \to \cV'$ is \emph{monotone} if
\begin{equation}
\ip{f(u)-f(v)}{u-v} \ge 0 \qquad \text{for all } u,v \in \cK, \label{eq:monotone}
\end{equation}
\emph{strictly monotone} if equality in \eqref{eq:monotone} implies $u=v$, and \emph{coercive} if there exists $w \in \cK$ so that
\begin{equation}
\frac{\ip{f(u)-f(w)}{u-w}}{\|u-w\|} \to +\infty \qquad \text{as } \|u\|\to +\infty. \label{eq:coercive}
\end{equation}
We say VI \eqref{eq:vi} is \emph{monotone} if $f$ is monotone, and likewise for strictly monotone and coercive. \end{definition}

It is well-known that if $f:\cK \to \cV'$ is continuous, monotone, and coercive then VI \eqref{eq:vi} has a solution \cite[Corollary III.1.8]{KinderlehrerStampacchia1980}, and also that the solution $u^* \in \cK$ is unique when $f$ is strictly monotone.  As in the calculus of variations \cite{Evans2010}, coercivity permits a compactness argument for unbounded sets $\cK$; recall that the bounded, closed subsets of a reflexive Banach space are weakly compact.  The condition of continuity can be weakened to only apply on finite-dimensional subspaces \cite{KinderlehrerStampacchia1980}, but the stronger condition will apply in our examples.

The coercive VIs solved in this paper satisfy a stronger inequality than \eqref{eq:coercive}, and thus they are well-posed.

\begin{definition}  Let $p>1$.  The map $f:\cK \to \cV'$ is \emph{$p$-coercive} if there exists $\kappa>0$ such that
\begin{equation}
\ip{f(u)-f(v)}{u-v} \ge \kappa \|u-v\|^p \qquad \text{for all } u,v \in \cK. \label{eq:pcoercive}
\end{equation}
\end{definition}

Note \cite{Tai2003} uses ``coercive'' for $2$-coercive.  It is easy to see that if $f$ is $p$-coercive then it is monotone, strictly monotone, and coercive, and thus the following result holds.

\begin{theorem}  \label{thm:viwellposed}  If $f:\cK \to \cV'$ is continuous and $p$-coercive (for $p>1$) then there exists a unique $u\in \cK$ solving VI \eqref{eq:vi}.
\end{theorem}

When $f$ is monotone, VI \eqref{eq:vi} generalizes the problem of minimizing a convex function over $\cK$.  Suppose $F:\cK \to \RR$ is lower semi-continuous and (G\^ateau) differentiable with continuous derivative $F':\cK \to \cV'$.  Then $F$ is convex if and only if $F'$ is monotone \cite[Proposition I.5.5]{EkelandTemam1976}.  Furthermore, Proposition II.2.1 in \cite{EkelandTemam1976} shows that if $F$ is convex then \eqref{eq:vi} holds for $f=F'$ and $g=0$ if and only if
\begin{equation}
u^* = \operatorname{arg-min}_{v\in\cK} F(v). \label{eq:minimization}
\end{equation}
The CD methods of Tai \cite{Tai2003} address problem \eqref{eq:minimization} under the hypothesis that $F'$ is coercive.

From now on $\Omega \subset \RR^d$ denotes a bounded, open set with smooth or piecewise-smooth (e.g.~polygonal) boundary.  Also, Sobolev spaces \cite{Evans2010} are denoted by $W^{k,p}(\Omega)$, for integer $k$ and $1\le p \le \infty$, with $W^{k,2}$ denoted by $H^k$.

The following example includes the classical obstacle problem for the linear Laplacian \cite{GraeserKornhuber2009} and the obstacle problem for the $p$-Laplacian \cite{ChoeLewis1991}.

\begin{example}  \label{ex:plaplacian}  Suppose $a\in L^\infty(\Omega)$ such that $a(x)\ge a_0$ a.e.~for some constant $a_0>0$, and $p\ge 2$.  For $u,v \in \cV = W^{1,p}_0(\Omega)$ define $f:\cV \to \cV'$ by
\begin{equation}
\ip{f(u)}{v} = \int_\Omega a(x) |\grad u|^{p-2} \grad u \cdot \grad v\,dx. \label{eq:plaplacian}
\end{equation}
Now, if $x,y\in\RR^d$ then $(|x|^{p-2} x - |y|^{p-2} y)\cdot (x-y) \ge 2^{2-p} |x-y|^p$ \cite[see Appendix A and references therein]{Bueler2021conservation}.  Thus it follows from the Poincar\'e inequality that
    $$\ip{f(u) - f(v)}{u-v} \ge 2^{2-p} a_0 \|\grad u - \grad v\|_p^p \ge 2^{2-p} a_0 C \|u-v\|^p$$
for some $C>0$, and thus $f$ is $p$-coercive.  (The map in \eqref{eq:plaplacian} is coercive if $1<p<2$, but the proof is somewhat different \cite[Theorem 4.4]{Bueler2021conservation}.  In Section \ref{sec:results} we need only the $p\ge 2$ case.)  For $g\in\cV'$ define
    $$F(u) = \int_\Omega \frac{a(x)}{p} |\grad u|^p\,dx - g(u).$$
Then $F'(u) = f(u) - g$, $F$ is a convex functional (since $f$ is coercive), and, for any closed and convex $\cK\subset \cV$, VI problem \eqref{eq:vi} for is equivalent to optimization problem \eqref{eq:minimization}.
\end{example}

However, not all VI problems arise from optimization.  We give two such examples next, first a coercive and linear advection-diffusion problem, and then a nonlinear porous-medium-type problem.  Each of these examples is important in applications.  The first is preceded by a lemma.

\begin{lemma}  \label{lem:advectionskew}  \cite{Elmanetal2014}\,  Suppose $\bX :\Omega \to \RR^d$ is a smooth and bounded vector field on $\Omega$ with zero divergence ($\Div \bX=0$).  For $u,v \in H^1(\Omega)$ let $b(u,v) = \int_\Omega (\bX \cdot \grad u) v\,dx$.  Then $b(u,u) = \frac{1}{2} \int_{\partial \Omega} u^2 \bX\cdot \bn\,dx$ where $\bn$ is the outward normal on $\partial \Omega$.
\end{lemma}

\begin{proof}
Integration by parts gives $b(u,v) = - b(v,u) + \int_{\partial \Omega} uv \bX\cdot \bn\,dx$, so the result follows.
\end{proof}

\begin{example}  \label{ex:advectiondiffusion}  Suppose $\partial\Omega$ is partitioned into Dirichlet and Neumann portions, i.e.~$\partial\Omega = \partial_D\Omega \cup \partial_N\Omega$, with $\partial_D\Omega$ of positive measure.  Let $\cV = H_0^1(\Omega)$ be the space of functions with value zero on $\partial_D\Omega$.  (For simplicity we consider zero boundary data.)  Consider a smooth and bounded velocity field $\bX$ on $\Omega$ such that $\Div \bX=0$, but additionally assume that the flow is outward on the Neumann boundary, i.e.~$\bX \cdot \bn \ge 0$ at points of $\partial_N\Omega$.  For $u,v \in \cV = H_0^1(\Omega)$ and $\eps>0$ define
\begin{equation}
\ip{f(u)}{v} = \int_\Omega \eps \grad u \cdot \grad v - (\bX \cdot \grad u) v\,dx. \label{eq:advectiondiffusion}
\end{equation}
Consider VI \eqref{eq:vi} for any closed and convex $\cK \subset \cV$ and any $g\in\cV'$.  It is easy to see that $|\ip{f(u)}{v}| \le (\eps + \|\bX\|_\infty) \|u\| \|v\|$, thus that $f:\cK \to \cV'$ is continuous (and Lipschitz).  On the other hand, Lemma \ref{lem:advectionskew} says that $s(u,v)$ is skew-symmetric up to a nonnegative term.  However, by the outward flow assumption and the Poincar\'e inequality,
\begin{align*}
\ip{f(u)-f(v)}{u-v} &= \eps \int_\Omega |\grad u - \grad v|^2\,dx + b(u-v,u-v) \\
                    &= \eps \int_\Omega |\grad u - \grad v|^2\,dx + \frac{1}{2} \int_{\partial_N\Omega} (u-v)^2 \bX\cdot\bn \ge \eps C \|u-v\|^2.
\end{align*}
Thus $f$ is 2-coercive, and so VI problem \eqref{eq:vi} is well-posed for any closed and convex $\cK \subset \cV$ and $g\in \cV'$.
\end{example}

If $\bX \ne 0$ then VI \eqref{eq:vi} for $f$ in \eqref{eq:advectiondiffusion} is not a (constrained) minimization problem.  Indeed, the Jacobian of $f$, namely the bilinear form $J_f(u,v) = \int_\Omega \eps \grad u \cdot \grad v\,dx - b(u,v)$, is not symmetric in that case.\footnote{Assume $\bX \ne 0$ is continuous for simplicity.  For $u,v$ which are zero on $\partial \Omega$, $J_f(u,v) - J_f(v,u) = -2 b(u,v)$.  There exist $u,v$ so $b(u,v)$ is nonzero; construct $u,v$ locally near some point where $\bX$ is nonzero.}  Thus $f$ in \eqref{eq:advectiondiffusion} cannot be the Hessian of a scalar objective $F$.  References \cite{Bueler2021conservation,ChangNakshatrala2017} consider such advection-diffusion VI problems over $\cK = \{v\ge 0\}$.

\begin{example}  \label{ex:porous}  FIXME
\begin{equation}
\ip{f(u)}{v} = \int_\Omega \phi(u) \grad u \cdot \grad v\,dx \label{eq:porous}
\end{equation}
\end{example}

Numerical solver performance for Examples \ref{ex:plaplacian}, \ref{ex:advectiondiffusion}, and \ref{ex:porous} will be considered in Section \ref{sec:results}.  Further examples of nonlinear VI problems appear in ice sheet models \cite{Calvoetal2002,JouvetBueler2012} and other geophysical fluids \cite{Bueler2021conservation}.

\section{Constraint decomposition, the basic algorithm} \label{sec:cd}

Suppose there are $m<\infty$ closed subspaces $\cV_i \subset \cV$ so that the sum
\begin{equation}
\cV = \sum_{i=0}^{m-1} \cV_i \label{eq:subspacedecomp}
\end{equation}
holds in the sense that if $w \in \cV$ then there exist $w_i \in \cV_i$ so that $w = \sum_i w_i$; this is called a \emph{subspace decomposition} \cite{Xu1992}.  Suppose further that $\cK_i \subset \cV_i$ are nonempty, closed, and convex subsets such that
\begin{equation}
\cK = \sum_{i=0}^{m-1} \cK_i. \label{eq:constraintdecomp}
\end{equation}
The sum in \eqref{eq:constraintdecomp} is required to hold in two senses: \emph{(i)}~if $w \in \cK$ then there exist $w_i \in \cK_i$ so that $w = \sum_i w_i$, and \emph{(ii)}~if $z_i \in \cK_i$ for each $i$ then $\sum_i z_i \in \cK$.\footnote{Sense \emph{(ii)} for \eqref{eq:subspacedecomp} is automatic because the $\cV_i$ are subspaces.}

Note that neither decomposition \eqref{eq:subspacedecomp} or \eqref{eq:constraintdecomp} is required to be unique.  Also, $\cK_i \not\subset \cK$ in many applications; see the cartoon in Figure \ref{fig:cartoon}.

Finally, for each $\cK_i$ we assume that there are bounded, (generally) nonlinear restriction operators $R_i : \cK \to \cK_i$ such that if $v \in \cK$ then
\begin{equation}
v = \sum_{i=0}^{m-1} R_i v.  \label{eq:constraintrestrictionsum}
\end{equation}
A \emph{constraint decomposition} (CD) of $\cK$ is a choice of $\cV_i,\cK_i,R_i$ satisfying \eqref{eq:subspacedecomp}--\eqref{eq:constraintrestrictionsum} \cite{Tai2003}.

\begin{figure}[ht]
\includegraphics[width=0.55\textwidth]{genfigs/cartoon.pdf}
\caption{A constraint decomposition (CD) in a one-sided obstacle problem ($\mathcal{K}=\{v\ge \psi\}$) might look like this if $\mathcal{V}$ were 2-dimensional.}
\label{fig:cartoon}
\end{figure}

In Section \ref{sec:multilevel} we will introduce discretizations and describe practical algorithms, but the fundamental CD concept applies even at the level of the continuum problem.  The following two examples illustrate this for obstacle problems \cite{GraeserKornhuber2009}.  First we consider an overlapping domain decomposition.

\begin{example}  \label{ex:domaindecomposition}  Consider a bounded domain $\Omega \subset \RR^d$, let $\cV = W_0^{k,p}(\Omega)$ for $k\ge 0$ and $p\ge 1$, and suppose the obstacle $\psi \in W^{k,p}(\Omega)$ satisfies $\psi|_{\partial \Omega} \le 0$.  Let $\cK = \{v \ge \psi\} \subset \cV$.  Suppose further that $\{\phi_i\}_{i=0}^{m-1}$ is a smooth partition of unity on $\Omega$, satisfying $0 \le \phi_i\le 1$ and $\sum_i \phi_i = 1$, and let $\Omega_i$ be the support of $\phi_i$.  Let $\cV_i = \{w \in \cV:w|_{\Omega \setminus \Omega_i} =0 \}$, $\cK_i = \{v \in \cV_i: v \ge \phi_i \psi\}$, and $R_i(v) = \phi_i v$.  Then \eqref{eq:subspacedecomp}, \eqref{eq:constraintdecomp}, and \eqref{eq:constraintrestrictionsum} all hold.
\end{example}

Our second example is a disjoint frequency decomposition.  A multilevel FE CD, e.g.~the one proposed in Section \ref{sec:multilevel}, approximates such a frequency decomposition.

\begin{example}  \label{ex:frequencydecomposition}  For simplicity suppose $\Omega = (0,a)^d \subset \RR^d$ is a cube, and let $\cV = H_{\text{per}}^k(\Omega)$, $k\ge 0$, be the periodic functions.  Suppose $\psi \in H_{\text{per}}^k(\Omega)$ and let $\cK = \{v \ge \psi\} \subset \cV$.  Without using any detailed notation for Fourier representation, but noting that the frequencies are discrete, suppose $\{\cV_i\}$ are $m<\infty$ subspaces of $\cV$ defined by an (nonoverlapping) partition by frequency, thus satisfying \eqref{eq:subspacedecomp} as an orthogonal decomposition.  Suppose $P_i:\cV \to \cV_i$ are the corresponding orthogonal projections, satisfying $I = \sum_i P_i$.  Let $\cK_i = \{v \ge P_i \psi\} \subset \cV_i$ and $R_i = P_i$.  Then \eqref{eq:constraintdecomp} and \eqref{eq:constraintrestrictionsum} also hold.
\end{example}

Note that $\cK_i \not\subset \cK$ in most cases.  In Example \ref{ex:domaindecomposition}, if $\psi$ is positive over portions of $\Omega$ where the decomposition into overlapping subdomains $\Omega_i$ is nontrivial, then $\cK_i \not\subset \cK$, and similarly for Example \ref{ex:frequencydecomposition}.  Instead, the important inclusion is $\cK_i \subset \cV_i$.

Algorithm \ref{alg:basiccd} below states the basic CD algorithm as an iteration which solves smaller VI problems over each set $\cK_i$.  (By contrast, the algorithms in \cite{Tai2003} solve optimization problems over each $\cK_i$.)  The algorithm starts from a current iterate $u \in \cK$ and computes a new iterate $w\in\cK$, an improved approximation of the solution $u^* \in \cK$ of \eqref{eq:vi}, using a damping parameter $0<\alpha\le 1$.  There are parallel (additive) and successive (multiplicative) versions of the algorithm, generalizing the Jacobi and Gauss-Seidel iterations \cite{Greenbaum1997}, respectively.  For the parallel version the \textbf{for} loop can be computed in any order.

\begin{pseudofloat}[H]
\begin{pseudo*}
\pr{cditeration}(u\in\cK)\text{:} \\+
    if \pr{parallel}: \\+
        for $i \in \{0,\dots,m-1\}$: \\+
            $\hat w_i\in \cK_i$: \\+
                 $\boxed{\ip{f(u - R_i u + \hat w_i)}{v_i-\hat w_i} \ge g(v_i-\hat w_i)} \quad \forall v_i\in \cK_i$ \\---
    else: \\+
        for $i = 0,\dots,m-1$: \\+
            $\hat w_i\in \cK_i$: \\+
                $\displaystyle \boxed{\ip{f\Big(\sum_{j<i} w_j + \hat w_i + \sum_{j>i} R_j u\Big)}{v_i-\hat w_i} \ge g(v_i-\hat w_i)} \quad \forall v_i\in \cK_i$ \\-
            $w_i = (1-\alpha) R_i u + \alpha \hat w_i\in\cK_i$ \\--
    $\hat w = \sum_i \hat w_i\in\cK$ \\
    return $w=(1-\alpha) u + \alpha \hat w\in\cK$
\end{pseudo*}
\caption{The basic constraint decomposition (CD) algorithm for VI problem \eqref{eq:vi}.}
\label{alg:basiccd}
\end{pseudofloat}

The reader may confirm that, inside each boxed VI in Algorithm \ref{alg:basiccd}, the argument of $f$ is an element of $\cK$.  While the difference $v_i - \hat w_i \in \cV_i$ appears, note that by \eqref{eq:constraintdecomp} and \eqref{eq:constraintrestrictionsum} one may write it as a difference of admissible vectors (i.e.~from $\cK$), namely
\begin{align*}
[u - R_i u + v_i] - [u - R_i u + \hat w_i] &= v_i - \hat w_i, \label{eq:admissibledifference} \\
\left[\sum_{j<i} w_j + v_i + \sum_{j>i} R_j u\right] - \left[\sum_{j<i} w_j + \hat w_i + \sum_{j>i} R_j u\right] &= v_i - \hat w_i,  \notag
\end{align*}
for the two versions, respectively.

In other words, we solve VI \eqref{eq:vi} over perturbations from the sets $\cK_i$.  Define
\begin{equation}
e_i = \hat w_i - R_i u \in \cV_i \label{eq:ithupdate}
\end{equation}
as the $i$th update from a pass through either \textbf{for} loop in Algorithm \ref{alg:basiccd}.  Then $\hat w = u + \sum_{i} e_i$ and $w = u + \alpha \sum_i e_i$.  Easy identities $u - R_i u + \hat w_i = u + e_i$ and $\hat w = u^* + \sum_i \hat w_i - R_i u^*$ also hold.  We will use these identities in Section \ref{sec:convergence}.

The convergence results in Section \ref{sec:convergence} require substantial damping when applying the parallel algorithm, i.e.~$\alpha \le 1/m$, the same as in \cite{Tai2003}, but the successive version can be shown to converge without damping ($\alpha=1$).  In Section \ref{sec:results} we demonstrate practical convergence for a larger range of $\alpha$ than suggested by the theory.  % HOPE

In order for Algorithm \ref{alg:basiccd} to have a practical and efficient FE implementation (Section \ref{sec:multilevel}), we need additional notions when $f$ has any of the following properties: \emph{(i)} $f$ is defined only on $\cK$, or \emph{(ii)} $f$ is nonlinear, or \emph{(iii)} $f$ is non-local.  The issue of practical implementation for any such $f$ seems not to have been addressed in the CD literature, which is why we call Algorithm \ref{alg:basiccd} the ``basic'' algorithm.  In particular  \cite{GraeserKornhuber2009,Tai2003} only apply the CD method to the classical obstacle problem.  The following example explains the simplifications available in that easy case.

\begin{example}  \label{ex:fnice} Suppose $f:\cV \to \cV'$ is linear and defined on all of $\cV$.  Furthermore suppose $f$ is local in the sense that a basis $\{\phi_i\}$ of $\cV_i$ exists, with each support small in $\Omega$, such that $\ip{f(\phi_i)}{z}$ can be computed by an integral over the support of $\phi_i$.  (When $\cV$ is an FE space then this is the usual case for evaluating PDE weak forms over a basis of hat functions; see Section \ref{sec:multilevel}.  Constrast integral operators where $\ip{f(\phi_i)}{z}$ requires an integral over $\Omega$ even if the support of $\phi_i$ is small.)  Considering only the parallel version of Algorithm \ref{alg:basiccd} for simplicity, the boxed VI over $\cK_i$ can be written as
\begin{equation}
\ip{f(e_i)}{v_i-\hat w_i} \ge \tilde g(v_i-\hat w_i) \label{eq:linearlocalvi}
\end{equation}
where $\tilde g(z) = g(z) - \ip{f(u)}{z}$.  Noting $e_i = \hat w_i - R_i u \in \cV_i \notin \cK$, in general, VIs \eqref{eq:linearlocalvi} will only make sense because $f$ is defined over $\cV$.  Each problem \eqref{eq:linearlocalvi} can be solved using a stored residual $\ip{f(u)}{\cdot}$ (already included into the source term).  The VI \eqref{eq:vi} is then approximately solved in any incremental and efficient manner, by computations over the basis supports, during one application of \pr{cditeration}.
\end{example}

In other words, when $f$ has all the nice properties proposed in Example \ref{ex:fnice} then the decomposed VI problems can be computed from inexpensive data.  Note that the actual solution method for VI \eqref{eq:linearlocalvi} is not the concern here; we are observing that the data of problem \eqref{eq:linearlocalvi}, and the cost of residual evaluation, is small in practice.  A solver which would work for Example \ref{ex:fnice} can thus have implementation efficiencies which are unavailable in general.

Additional ideas, like those in Section \ref{sec:multilevel}, are needed for practical application of CD methods to more difficult problems.  In particular, we will extend the basic Algorithm \ref{alg:basiccd} to nonlinear $f$ by applying the full approximation storage (FAS) idea of Brandt \cite{Brandt1977}, in which case the decomposed problems only evaluate $f$ over (admissible) elements of $\mathcal{K}$, and we propose restrictions which allow inexpensive approximations of the decomposed problems.  However, extending the algorithm to non-local residual functionals is a topic for future research.

\section{Convergence of the basic algorithm} \label{sec:convergence}

We now seek a proof of the convergence of Algorithm \ref{alg:basiccd}.  This will be possible if we restrict to $2$-coercive operators and make certain assumptions as in \cite{Tai2003}.  First we define a new quantity related to VI problem \eqref{eq:vi}.

\begin{definition} Suppose $f:\cK \to \cV'$ and $g \in \cV'$.  For $u,v \in \cK$ let
\begin{equation}
  E(v,u) = \ip{f(v)}{v-u} - g(v-u).  \label{eq:normlikedefn}
\end{equation}
\end{definition}

If $u^*$ solves \eqref{eq:vi} then $E(v,u^*)$ is somewhat like an ``merit function,'' something used in the context of solving nonlinear equations \cite{NocedalWright2006} to replace an objective (scalar) functional.  (Note that $E(v,u^*)$ is \emph{not} a residual for \eqref{eq:vi}.)  The next lemma, which follows directly from $p$-coercivity, shows $E(v,u^*)$ is bounded below.

\begin{lemma} \label{lem:normlike}  Suppose $f:\mathcal{K} \to \mathcal{V}'$ is $p$-coercive and $u^* \in \mathcal{K}$ solves \eqref{eq:vi}.  For $v \in \mathcal{K}$,
\begin{equation}
  E(v,u^*) \ge \kappa \|v-u^*\|^p.  \label{eq:normlikebound}
\end{equation}
\end{lemma}

\begin{proof}
\begin{align*}
E(v,u^*) &= \ip{f(v)}{v-u^*} - \ip{f(u^*)}{v-u^*} + \ip{f(u^*)}{v-u^*} - g(v-u^*) \\
   &\ge \kappa \|v-u^*\|^p + 0.  \qedhere
\end{align*}
\end{proof}

Next we make two assumptions which are essentially the same as (7) and (8) in \cite{Tai2003}.

\begin{assumptions*}  There exists a constant $C_1>0$ so that
\begin{equation}
\left(\sum_{i=0}^{m-1} \|R_i u - R_i v\|^2\right)^{1/2} \le C_1 \|u-v\| \label{as:lipschitzrestrictions}
\end{equation}
for all $u,v\in\cK$.  Furthermore, there exists $C_2>0$ so that for all $v_i \in \cV_i$ and $y_j \in \cV_j$,
\begin{equation}
\sum_{i=0}^{m-1} \sum_{j=0}^{m-1} \left|\ip{f(w_{ij} + v_i) - f(w_{ij})}{y_j}\right| \le C_2 \left(\sum_{i=0}^{m-1} \|v_i\|^2\right)^{1/2} \left(\sum_{j=0}^{m-1} \|y_j\|^2\right)^{1/2} \label{as:lipschitzresidual}
\end{equation}
for all $w_{ij} \in \cK$ such that $w_{ij} + v_i \in \cK$ for all $i$ and $j$.
\end{assumptions*}

These inequalities might be described as ``totally Lipschitz'' requirements for the maps $R_i$ and $f$, respectively.  We make the following observations:
\begin{itemize}
\item Assumption \eqref{as:lipschitzrestrictions} addresses only the CD, and not the map $f$.
\item It is sometimes the case that \eqref{as:lipschitzresidual} addresses only a subspace decomposition property of $f$ (and not the CD itself).  For example, in the classical obstacle problem one may verify \eqref{as:lipschitzresidual} over all $w_{ij} \in \cV$.
\item The unconstrained versions of \eqref{as:lipschitzrestrictions} and \eqref{as:lipschitzresidual} are (13), (14) in \cite{TaiXu2002}, respectively.
\item If we assume $f$ itself is Lipschitz then the existence of $C_2$ is clear \cite{TaiXu2002}.  However, our bound for the convergence rate for Algorithm \ref{alg:basiccd} is improved when $C_1,C_2$ are made smaller.
\item Tai \cite{Tai2003} gives values of the constants $C_1$ in \eqref{as:lipschitzrestrictions} for obstacle problem constraint decompositions using $P_1$ FE spaces over shape-regular and quasi-uniform triangulations.  In particular, constants $C_1$ are known for overlapping domain decompositions (with or without an additional coarse mesh) and standard multilevel hierarchies.  Thus the cases we will demonstrate in Section \ref{sec:results} are already covered.
\item The constants $C_1,C_2$ generally depend on $m$.  However, regarding the convergence proof below, \cite{Tai2003} observes that if repeated application of Algorithm \ref{alg:basiccd} generates a bounded sequence of iterates in $\mathcal{K}$ then constants $C_1,C_2$ would be permitted to depend on $u,v,w_{ij},v_i,y_i$ as well.
\end{itemize}

From the above definitions and assumptions we have the following estimate for the parallel version of Algorithm \ref{alg:basiccd}.  Our proof follows \cite{Tai2003}.

\begin{lemma} \label{lem:core}  Suppose $f$ is $2$-coercive and $u^* \in \mathcal{K}$ solves \eqref{eq:vi}.  Recall $E(v,u^*)$ is defined in \eqref{eq:normlikedefn}.  Suppose $\mathcal{V}_i$, $\mathcal{K}_i$, $R_i$, and $f$ satisfy \eqref{as:lipschitzrestrictions} and \eqref{as:lipschitzresidual}.  Suppose $\hat w$ is computed from $u$ by the parallel version of Algorithm \ref{alg:basiccd}, and recall $e_i = \hat w_i - R_i u$.  Then
\begin{equation}
   E(\hat w,u^*) \le C_2 \sum_{i=0}^{m-1} \|e_i\|^2 + \kappa^{-1} C_1 C_2 \left(\sum_{i=0}^{m-1} \|e_i\|^2\right)^{1/2} E(u,u^*)^{1/2}. \label{eq:core}
\end{equation}
\end{lemma}

\begin{proof}  Recalling that $\hat w = u^* + \sum_i \hat w_i - R_i u^*$, by the (parallel version) boxed VI in Algorithm \ref{alg:basiccd} with $v_i = R_i u^*$ we get
\begin{align}
E(\hat w,u^*) &= \sum_{i=0}^{m-1} \ip{f(\hat w)}{\hat w_i - R_i u^*} - g(\hat w_i - R_i u^*) \label{eq:startcore} \\
    &\le \sum_{i=0}^{m-1} \ip{f(\hat w)}{\hat w_i - R_i u^*} + \ip{f(u + e_i)}{R_i u^* - \hat w_i} \notag \\
    &= \sum_{i=0}^{m-1} \ip{f(\hat w) - f(u + e_i)}{\hat w_i - R_i u^*}. \notag
\end{align}
For $i\in \{0,1,\dots,m-1\}$ use wrapped indices to define elements $\phi_{i,j}$:
\begin{align}
\phi_{i,0} &= u + e_i, \label{eq:gridcore} \\
\phi_{i,1} &= u + e_i + e_{i+1}, \notag \\
  &\vdots \notag \\
\phi_{i,m-1} &= u + e_i + e_{i+1} + \dots + e_{m-1} + e_0 + \dots + e_{i-1}, \notag
\end{align}
and $\phi_{i,j} \in \cK$ by \eqref{eq:constraintdecomp} and \eqref{eq:constraintrestrictionsum}.  Note that $\phi_{i,m-1} = u + \sum_i e_i = \hat w$.  From \eqref{eq:startcore}, apply a telescoping sum using the elements in \eqref{eq:gridcore}:
\begin{align}
E(\hat w,u^*) &\le \sum_{i=0}^{m-1} \sum_{j=1}^{m-1} \ip{f(\phi_{i,j}) - f(\phi_{i,j-1})}{\hat w_i - R_i u^*} \label{eq:nextcore} \\
  &\le \sum_{i=0}^{m-1} \sum_{j=0}^{m-1} \left|\ip{f(\phi_{i,j}) - f(\phi_{i,j-1})}{\hat w_i - R_i u^*}\right|. \notag
\end{align}
By an easy renumbering of the indices $i,j$ we may apply assumption \eqref{as:lipschitzresidual}, then the triangle inequality, and then assumption \eqref{as:lipschitzrestrictions}; we denote $\left(\sum_{i=0}^{m-1} \|e_i\|^2\right)^{1/2}$ by $Z$:
\begin{align}
E(\hat w,u^*) &\le C_2 Z \left(\sum_{i=0}^{m-1} \|\hat w_i - R_i u^*\|^2\right)^{1/2} = C_2 Z \left(\sum_{i=0}^{m-1} \|\hat w_i - R_i u + R_i u - R_i u^*\|^2\right)^{1/2} \label{eq:nextnextcore} \\
  &\le C_2 Z \left(Z + \left(\sum_{i=0}^{m-1} \|R_i u - R_i u^*\|^2\right)^{1/2}\right) \le C_2 Z \left(Z + C_1 \|u-u^*\|\right). \notag
\end{align}
Finally apply \eqref{eq:normlikebound} to give \eqref{eq:core}.
\end{proof}

FIXME NOW THE ENTIRE BATTLE IS TO GET A CONVEXITY RESULT $E(w,u) \ge C \sum_{j=0}^{m-1} \|e_j\|^2$ for some reasonable $C$

\section{Finite elements and multilevel constraint decomposition} \label{sec:multilevel}

In practice we will solve VI \eqref{eq:vi} over a finite-dimensional space $\cV$ based on a choice of a mesh over $\Omega$ and a finite element (FE) space.

FIXME triangulation

\begin{example}  FIXME if $\cV_i=\Span\{\phi_i\}$ for $\phi_i\in\cV$ are 1d spaces and if $\cK = \{v \ge \psi\} \subset \cV$ (obstacle problem), and if $R_i : \cK \to \cK_i$ for obstacle problem; note $\cK_i \not\subset \cK$ when $\psi>0$
\end{example}

FIXME state essentially Algorithm 4.7 \cite{GraeserKornhuber2009} but with $\text{V}(\nu_1,\nu_2)$ cycles which works for linear; observe that up-smoothing is more efficient; state the FAS version which has $O(m)$ residual evaluation complexity on each level

\section{Results for local variational inequalities} \label{sec:results}

FIXME

% A BRIDGE TOO FAR:  \section{Results for a nonlocal variational inequality} \label{sec:resultsnonlocal}

\small
\bibliography{mcd2}
\bibliographystyle{siam}

\normalsize
\appendix

\section{Porous-medium problems are never monotone}

Based on the examples in Section \ref{sec:vi}, the property of $p$-coercivity would seem to be a mild generalization of uniform ellipticity (including its quasi-linear meaning \cite[Section 8.3]{Evans2010}).  Indeed, weak-form operators with certain lower-order terms (Example \ref{ex:advectiondiffusion}) or degenerate quasi-linear type (Example \ref{ex:plaplacian}) can be monotone and coercive.  However, even the weaker property of monotonicity is ``fragile'' in a pointwise sense, as explained next, while ellipticity is more robust.  In particular, we show that no nontrivial porous-medium type functional is monotone, nor $p$-coercive.  We have not found any such explanation in the literature, but the observation of non-monotonicity in \cite{RocknerWang2008} is related.

Consider the expression $(Ax - By)\cdot (x-y)$ where $A,B \in \RR^{d\times d}$ are symmetric matrices and $x,y\in \RR^d$.  This expression is the quadratic form for a certain symmetric block matrix.  In fact, regarding $x,y$ as column vectors, we observe that
\begin{equation}
(Ax - By)\cdot (x-y) = \begin{bmatrix} x^\top & y^\top \end{bmatrix} \begin{bmatrix}
                        A & -\frac{1}{2}(A+B) \\
                        -\frac{1}{2}(A+B) & B \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix}. \label{eq:blockmatrix}
\end{equation}
Let $M_{A,B}$ denote the symmetric $2d \times 2d$ matrix in \eqref{eq:blockmatrix}.  Clearly the expression is only bounded below if $M_{A,B}$ is nonnegative definite.  Stated as the following Lemma, the important point here is that exact equality $A=B$ is required for nonnegativity of the expression; knowing $A$ is close to $B$ is not helpful.

\begin{lemma}  Suppose $A,B \in \RR^{d\times d}$ are symmetric matrices which commute.  Then $(Ax - By)\cdot (x-y)\ge 0$ for all $x,y\in \RR^d$ if and only if $A=B$ is nonnegative definite. \end{lemma}

\newcommand{\sbvec}[2]{\left[\begin{smallmatrix} #1 \\ #2 \end{smallmatrix}\right]}
\newcommand{\sbmat}[4]{\left[\begin{smallmatrix} #1 & #2 \\ #3 & #4 \end{smallmatrix}\right]}

\begin{proof}
If $A=B$ is nonnegative definite then nonnegativity of the expression follows immediately.  Conversely, since $A$ is symmetric there exists an orthogonal basis $\{x_i\}$ of eigenvectors, with $Ax_i = \lambda_i x$.  By commutativity these are also eigenvectors of $B$, with $Bx_i = \mu_i x_i$.  Let $\big\{\sbvec{x_1}{x_1}$, $\sbvec{x_1}{-x_1}$, $\dots$, $\sbvec{x_d}{x_d}$, $\sbvec{x_d}{-x_d}\big\}$ be an ordered basis of $\RR^{2d}$.  It is easy to confirm that in this basis $M_{A,B}$ is block diagonal with $2\times 2$ diagonal blocks $\sbmat{0}{-\frac{1}{2}(\lambda_i+\mu_i)}{-\frac{1}{2}(\lambda_i+\mu_i)}{\lambda_i+\mu_i}$.  Thus the eigenvalues of $M_{A,B}$ come in pairs $\frac{1}{2}(\lambda_i + \mu_i) \pm \sqrt{\frac{1}{2}(\lambda_i^2 + \mu_i^2)}$ for $i=1,\dots,d$.  By the strict concavity of the square root, $\sqrt{\frac{1}{2}(\lambda_i^2 + \mu_i^2)} > \frac{1}{2}(|\lambda_i| + |\mu_i|)$ if $\lambda_i\ne \mu_i$.  Thus if any eigenvalues of $A$ differ from those of $B$, i.e.~$\lambda_i\ne \mu_i$ for any $i$, then there exists a negative eigenvalue of $M_{A,B}$.
\end{proof}

It is not clear if the hypotheses of symmetry and commutativity can be removed.  However, when the matrices are scalar multiples of the identity then the Lemma asserts that $(ax-by)\cdot(x-y) \ge 0$ for all $x,y\in\RR^d$ if and only if $a=b\ge 0$.  This scalar case suffices for the following result.

\begin{theorem}
Suppose $\phi:[0,\infty) \to (0,\infty)$ is continuous.  For $\Omega \subset \RR^d$ open and nonempty, $\cV = H_0^1(\Omega)$, and $g\in \cV'$, consider the following functional on $\cK = \{v\ge 0\} \subset \cV$:
\begin{equation}
\ip{f(u)}{v} = \int_\Omega \phi(u) \grad u\cdot \grad v\,dx - g(v).  \label{eq:porousagain}
\end{equation}
If $\phi$ is not constant then $f$ is not monotone.
\end{theorem}

\begin{proof}
Suppose $\phi(\alpha)\ne \phi(\beta)$ for $\alpha > 0$ and $\beta \ge 0$.  (That is, suppose $\phi$ is not constant.)  For $x\in\Omega$, construct $u\in \cK$ smooth so that $u(x)=\alpha$ and $\xi = \grad u(x) \ne 0$.  Using the Lemma, choose $\eta \in \RR^d$ so that $(\phi(\alpha) \xi - \phi(\beta) \eta) \cdot (\xi - \eta) < 0$.  FIXME NOT SURE IF THIS IS GOING TO WORK
\end{proof}

The main idea is that even the slightest variation in the coefficient $\phi(u)$ violates monotonicity.  A strong and tight ellipticity hypothesis, e.g.~the existence of $c_0>0$ and $\eps>0$ so that $c_0 \le \phi(u) \le (1+\eps)c_0$, will not imply monotonicity.  Contrast the nontrivial coefficient dependence on $\grad u$ in the $p$-coercive $p$-Laplacian Example \ref{ex:plaplacian} (with $p>2)$.

No nontrivial functional \eqref{eq:porousagain} is monotone, nor is it ever $p$-coercive for any $p>1$.
\end{document}


%
% File acl2017.tex
%
%% Based on the style files for ACL-2015, with some improvements
%%  taken from the NAACL-2016 style
%% Based on the style files for ACL-2014, which were, in turn,
%% based on ACL-2013, ACL-2012, ACL-2011, ACL-2010, ACL-IJCNLP-2009,
%% EACL-2009, IJCNLP-2008...
%% Based on the style files for EACL 2006 by
%%e.agirre@ehu.es or Sergi.Balari@uab.es
%% and that of ACL 08 by Joakim Nivre and Noah Smith

\documentclass[11pt,a4paper]{article}
\usepackage[hyperref]{acl2017}
\usepackage{times}
\usepackage{latexsym}
\usepackage{bm}
\usepackage{graphicx}
\usepackage{amsmath}
\DeclareMathOperator*{\argmax}{argmax} % thin space, limits underneath in displays
\usepackage{url}
\usepackage[inline,shortlabels]{enumitem}
\usepackage{caption}% <-- added

\usepackage{tabularx,booktabs}
\newcolumntype{C}{>{\centering\arraybackslash\hsize=.5\hsize}X} % centered version of "X" type
\setlength{\extrarowheight}{1pt}

\usepackage{adjustbox}
\usepackage{float}

\aclfinalcopy % Uncomment this line for the final submission
%\def\aclpaperid{***} %  Enter the acl Paper ID here

%\setlength\titlebox{5cm}
% You can expand the titlebox if you need extra space
% to show all the authors. Please do not make the titlebox
% smaller than 5cm (the original size); we will check this
% in the camera-ready version and ask you to change it back.

\newcommand\BibTeX{B{\sc ib}\TeX}

\title{CS395T: Neural Networks for Sentiment Analysis Project Report}

\author{Zeyuan Hu \\
  Computer Science Department \\
  University of Texas at Austin \\
  Austin, Texas \\
  {\tt iamzeyuanhu@utexas.edu} \\
}

\date{}

\begin{document}
\maketitle

\begin{abstract}
In this project, I implement a Feedforward Neural Network based on
averaged word vectors over the input sentence and a convolutional neural
network (CNN) to perform the sentiment analysis on the sentences from
Rotten Tomatoes introduced by Pang and Lee~\shortcite{Pang:2005}.
\end{abstract}

\section{Collaborators}
Zhan Shi, Danlu Wang

\section{Introduction}

Sentiment analysis is an umbrella term for several closely-related tasks. One of them is to
classifiy the sentences as either negative or positive. Traditionally, Naive Bayes or SVM
bag-of-words models are well-suited for those tasks. However, neural network approaches
including convolutional networks \cite{Kim14f} and feedforward networks \cite{Iyyer2015}
can substitute those traditional models to achieve similar performance. In this project,
I implement a feedford network and a convolutional neural networks for the sentiment analysis.
In addition, I explore various architectural and parameter options to
tune the systems to gain good classification performance.

\section{Implementation Details}

\subsection{Feedforward Neural Networks}

The implementation of the feedforword neural network (FFNN) follows closely to
the architecture of the deep averaging network (DAN)
described by Iyyer et al.~\shortcite{Iyyer2015}. In the paper, the authors
break down the architecture into three simple steps: 1. take the average of
word embeddings of words in a given sentence 2. pass the average into a
FFNN with one or more hidden layers 3. perform classification
on the final output. I follow those three steps closely and my network contains
one hidden layer. Essentially, I calculate $P(\boldsymbol{y}|\boldsymbol{x})$ in the network.
\verb|one_best| contains the classifiction result of the sentences in the computataion graph \verb|graph|.
I implement batching and during each epoch, I shuffle
the training examples at the very beginning. For the optimizer, I experiment with both the Adam
and the stochastic gradient descent (SGD). The parameter tuning and the impact of the optimizer
to the performance will be disccued in details in the experiment section.

\subsection{CNN}

The implementation of CNN follows Zhang and Wallace~\shortcite{Zhang15}'s empiricial study of
one-layer CNN hyperparameter setting on sentence classification. In my implementation, I use
one convolution layer with multiple filter widths and feature maps and then I use 1-max pooling
suggested in the paper. The code structure is divided into five steps:
\begin {enumerate*}[1) ]
\item construct embedding layer by looking up pre-trained word embeddings,
\item create convolution layer and implement 1-max pooling strategy,
\item combine all the pooled features,
\item add dropout, and
\item make final predictions.
\end {enumerate*}
The rest of work is to focus on the hyperparameter setting, which
includes filter region size(s) (i.e., \verb|filter_sizes|),
the number of filters to use with each size (i.e., \verb|num_filters|),
the activation function(s) (i.e., \verb|activation_func|), and so on.

\section{Experiments}

The experiment is carried out in a similar way as Zhang and Wallace~\shortcite{Zhang15}'s.
For each exploration task, I report the mean, minimum, and maximum average
accuracy values observed over three replications.

\subsection{Feedforward Neural Networks}

For the FFNN, I explore the impact of the architecture and
the choice of the optimizer to the system performance. Without special note, the experiments
is based on the hyperparameter setting in table 1.

\begin{table}
\captionsetup{size=footnotesize}
\caption{FFNN Configuration} \label{tab:freq}
%\setlength\tabcolsep{0pt} % let LaTeX compute intercolumn whitespace
\footnotesize\centering
%This table provides the frequencies.

\smallskip
\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}cc}
\toprule
  Description  & Values  \\
\midrule
 batch size & 10      \\
 number of hidden layer & 1 \\
 number of hidden units & 150        \\
 number of epochs & 50        \\
 initial learning rate & 0.1        \\
 decay steps & 10 \\
 learning rate decay factor & 0.99 \\
 optimizer & SGD \\
\bottomrule
\end{tabular*}
\end{table}

\textbf{Optimizer:} I compare the performance between Adam and SGD, which is controlled by
the environment variable \verb|OPT| in the code. One surprising finding is that whether
supplying \verb|global_step| argument inside the function call \verb|apply_gradients|
can have impact on the performance of the system . I use tensorflow version 1.3.0 and the result
is shown in table 1.

\begin{table}[!htb]
\captionsetup{size=footnotesize}
\caption{Optimizer Performance with ``global\_step"} \label{tab:freq}
\setlength\tabcolsep{0pt} % let LaTeX compute intercolumn whitespace
\footnotesize\centering
%This table provides the frequencies.

\smallskip
\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}rccr}
\toprule
  Optimizer  & Arg & Accuracy (training) & Accuracy (dev) \\
\midrule
 SGD & Yes   & 76.53 (76.44, 76.59)    & 74.83 (74.67, 75.05)      \\
 ADAM & Yes   & 77.58 (76.72, 78.23)    & \textbf{74.98 (74.48, 75.33)}     \\
 SGD & No  & \textbf{81.72 (81.15, 82.70)}     & 74.42 (74.20, 74.77)      \\
 ADAM & No   & 50.06 (50.06, 50.06)     & 49.06 (49.06, 49.06)      \\
\bottomrule
\end{tabular*}
\end{table}

As shown in the table, \verb|global_step| does not have impact for SGD
measured by the accuracy on the dev set. Adam has similar performance
as SGD with \verb|global_step|. However, without \verb|global_step|,
Adam performs poorly and SGD has significant increase in training set
accuracy compared with \verb|global_step| unset case.

\textbf{Architecture:} The architecture of FFNN is determined by the number of
hidden layers. In this experiment, I explore the FFNN with zero and one
hidden layers. The result is shown in table 3.

\begin{table}[!htb]
\captionsetup{size=footnotesize}
\caption{FFNN architecture} \label{tab:freq}
\setlength\tabcolsep{0pt} % let LaTeX compute intercolumn whitespace
\footnotesize\centering
%This table provides the frequencies.

\smallskip
\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}rcr}
\toprule
  Hidden layers  &  Accuracy (training) & Accuracy (dev) \\
\midrule
 0   &  71.73 (66.66, 77.40)   &  69.48 (65.76, 73.92)     \\
 1    & \textbf{81.72 (81.15, 82.70)}    & \textbf{74.42 (74.20, 74.77)}     \\
 2   & 51.22 (49.94, 52.05)     & 51.88 (50.38, 53.00)     \\
\bottomrule
\end{tabular*}
\end{table}

As shown in the table, there is a significant performance gain
when we increase our hidden layer from zero to one. However, with
the given configuration, two layers architecture has notable performance
drop. This suggests that hyperparameter setting and architecture has
to be carefully picked to achieve good performance.

%
%\begin{table*}
%\caption{Impact of the parameters on the UAS score}
%\label{data-table}
%\begin{tabularx}{\textwidth}{@{}l*{8}{C}c@{}}
%\toprule
%Parser    & Optimizer & Learning Rate & Epoches & Sentences     & Batch Size     & UAS    \\
%\midrule
%Greedy    & SGD        & 0.2              & 10         & 5000          & 1         & 72.27  \\
%Greedy    & SGD        & 0.2            & 20      & 5000           & 1                & 74.67  \\
%Greedy    & SGD       & 0.2             & 30        & 5000          & 1               & 75.81  \\
%Greedy    & SGD       & 0.2              & 40         & 5000         & 1               & 75.74  \\
%\addlinespace
%Greedy    & SGD       & 0.2              & 30         & 6000          & 1              & 76.35  \\
%Greedy    & SGD       & 0.2              & 30      & 7000         & 1               & 76.43  \\
%Greedy    & SGD       & 0.2              & 30      & 39832          & 1                & \textbf{78.60}  \\
%Global    & Adagrad(0.0001, 5) & 5    & 10      & 39832          & 20               & \textbf{75.67}  \\
%\bottomrule
%\end{tabularx}
%\end{table*}

\subsection{CNN}

For the CNN, I explore the impact of the hyperparameter setting to the system performance.
Without special note, the experiments is based on the configuration in table 4. Figure 1
shows the performance of the CNN versus the number of epochs.

\begin{figure}[h]
\includegraphics[scale=0.5]{cnn-epoch.png}
\caption{Accuracy vs. number of epochs in dev set}
\end{figure}

\begin{table}
\captionsetup{size=footnotesize}
\caption{CNN Configuration}
%\setlength\tabcolsep{0pt} % let LaTeX compute intercolumn whitespace
\footnotesize\centering
%This table provides the frequencies.

\smallskip
\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}cc}
\toprule
  Description  & Values  \\
\midrule
input word vectors & GloVe-300      \\
filter region size & (3,4,5) \\
feature maps &    128    \\
activation functions & ReLU        \\
pooling & 1-max pooling        \\
dropout rate & 0.5 \\
$l2$ norm constant & 0 \\
optimizer & Adam \\
number of epochs & 20 \\
batch size & 64 \\
\bottomrule
\end{tabular*}
\end{table}

In the following experiments, I train the CNN for $5$ epochs.
\textbf{Filter region size(s):}
In this experiment, I examine whether the filter region sizes will have
the impact on the CNN performance. The choice of filter region sizes are
picked based on Zhang and Wallace~\shortcite{Zhang15}'s empirical study.
The result is shown in table 5.

\begin{table}[!htb]
\captionsetup{size=footnotesize}
\caption{Filter region sizes and CNN performance} \label{tab:freq}
\setlength\tabcolsep{0pt} % let LaTeX compute intercolumn whitespace
\footnotesize\centering
%This table provides the frequencies.

\smallskip
\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}cc}
\toprule
  Filter region sizes  &   Accuracy (dev) \\
\midrule
(3,4,5)    &  \textbf{77.52 (77.39, 77.77)}     \\
(7,7,7,7)  &  77.20 (76.92, 77.58)     \\
(7)        &  76.92 (76.08, 77.39)     \\
\bottomrule
\end{tabular*}
\end{table}

I try different region sizes and find out that the impact of the region sizes
to the performance is not significant. Even we have one region size $7$, there
is almost no performance drop.

\textbf{Feature maps:}
Zhang and Wallace~\shortcite{Zhang15} suggests that the number of feature maps
for each filter region size is ranged between $100$ and $600$. This is quite large
range. I experiment with three feature maps with one value between $100$ and $200$,
one value between $200$ and $400$, and one value between $400$ and $600$. The result is
shown in table 6.

\begin{table}[!htb]
\captionsetup{size=footnotesize}
\caption{Feature maps and CNN performance} \label{tab:freq}
\setlength\tabcolsep{0pt} % let LaTeX compute intercolumn whitespace
\footnotesize\centering
%This table provides the frequencies.

\smallskip
\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}cc}
\toprule
  Feature maps  &   Accuracy (dev) \\
\midrule
128    &  77.52 (77.39, 77.77)     \\
300  &    \textbf{77.86 (77.67, 78.05)} \\
600    &  77.61 (76.83, 78.52)    \\
\bottomrule
\end{tabular*}
\end{table}

\textbf{Activation functions:}
In this experiment, I experiment with two activation functions: \verb|tanh| and \verb|relu|.
The result is summarized in the table 7.

\begin{table}[!htb]
\captionsetup{size=footnotesize}
\caption{Activation functions and CNN performance} \label{tab:freq}
\setlength\tabcolsep{0pt} % let LaTeX compute intercolumn whitespace
\footnotesize\centering
%This table provides the frequencies.

\smallskip
\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}cc}
\toprule
  Activation function  &   Accuracy (dev) \\
\midrule
 \verb|tanh|   &   77.49 (77.30, 77.77)     \\
 \verb|relu|    &  \textbf{77.52 (77.39, 77.77)}     \\
\bottomrule
\end{tabular*}
\end{table}

As shown in the table 7, the choice of the activation function does not have impact on the
CNN performance.

\section{Conclusion and Future Work}

In this project, I implement a feedforward neural network and a CNN for sentiment analysis.
I explore the impact of architecture choice and hyperparameter setting on the performance of
the neural networks to the task. In the future, I can carry out similar empirical study towards
other NLP tasks with different type of neural networks (i.e., LSTM).

\bibliography{acl2017}
\bibliographystyle{acl_natbib}

\end{document}


\documentclass[a4paper,11pt,captions=tableheading,DIV=12]{scrartcl}
\pdfoutput=1

% ----------------------------------------------------------- Packages
\usepackage{amsmath,amssymb,url,cite,slashed,cancel,booktabs,graphicx,listings,subcaption}
\usepackage[colorlinks=true,urlcolor=blue,citecolor=magenta]{hyperref}
%%%UNUSED%%% \usepackage{feynmp,enumerate,multirow,wrapfig}
\renewcommand\citepunct{,\penalty1000\hskip.13emplus.1emminus.1em\relax} % no line-break in \cite

% MATH NOTATION
\newcommand\w[1]{_{\mathrm{#1}}}
\newcommand\vc[1]{{\boldsymbol{#1}}}
\newcommand\dd{\mathrm{d}}
\newcommand\ee{\mathrm{e}}
\newcommand\ii{\mathrm{i}}
\newcommand\pmat[1]{\begin{pmatrix}#1\end{pmatrix}}
\newcommand\Order{\mathop{\mathcal{O}}}
\newcommand\Br{\mathop{\mathrm{Br}}}
\newcommand{\dn}[3]{\frac{\dd^#1 #2}{\dd #3^#1}}
\newcommand{\pdn}[3]{\frac{\partial^#1 #2}{\partial #3^#1}}
\newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}}

\newcommand\unit[1]{\,\mathrm{#1}}
\newcommand\eV{\unit{eV}}
\newcommand\keV{\unit{keV}}
\newcommand\MeV{\unit{MeV}}
\newcommand\GeV{\unit{GeV}}
\newcommand\TeV{\unit{TeV}}
\newcommand\fb{\unit{fb}}
\newcommand\pb{\unit{pb}}
\newcommand\iab{\unit{ab^{-1}}}
\newcommand\ifb{\unit{fb^{-1}}}
\newcommand\ipb{\unit{pb^{-1}}}
\newcommand\fm{\unit{fm}}

\makeatletter
\def\EE{\@ifnextchar-{\@@EE}{\@EE}}
\def\@EE#1{\ifnum#1=1 \times\!10 \else \times\!10^{#1}\fi}
\def\@@EE#1#2{\times\!10^{-#2}}
\makeatother

\newcommand{\fracp}[3]{\ifnum1=#3\relax\biggl(\frac{#1}{#2}\biggr)\else\biggl(\frac{#1}{#2}\biggr)^{#3}\fi}
\newcommand{\neut}[1][]{{\tilde\chi^0_{#1}}}
\newcommand{\chrg}[1][]{{\tilde\chi^\pm_{#1}}}
\newcommand{\chrgP}[1][]{{\tilde\chi^+_{#1}}}
\newcommand{\chrgM}[1][]{{\tilde\chi^-_{#1}}}

\lstset{columns=[l]fullflexible,basicstyle=\small\ttfamily,xleftmargin=2em,frame=L,keepspaces=true}
\bibliographystyle{utphys27mod}

% ---------------------------------------------------- For Sho's Notes
\usepackage{scrlayer-scrpage,color,soul}
\usepackage[hhmmss]{datetime}
\newdateformat{mydate}{\THEDAY\;\shortmonthname.\;\THEYEAR}
\addtokomafont{pagehead}{\small\normalfont}
\rohead*{\texttt{[\jobname~@~\mydate\today~\currenttime]}}

\newcommand{\TODO}[1]{\begingroup\color{red}\textbf{$\clubsuit$#1$\clubsuit$}\endgroup}

\title{SLHA convention compared against References}
\author{Sho Iwamoto}
\date{}
\newcommand{\wL}{_{\mathrm L}}
\newcommand{\wR}{_{\mathrm R}}
\newcommand{\bU}{\bar U}
\newcommand{\bD}{\bar D}
\newcommand{\bE}{\bar E}
\newcommand{\tra}{^{\mathrm T}}
\begin{document}
\maketitle
%---------------------------------------------------------------------
\section{Comparison}
\subsection{Higgs potential}

\subsubsection{SLHA convention}
The SLHA~\cite{SLHA} is based on Gunion--Haber's notation~\cite{GH}:
\begin{align}
\begin{split}
  W &\supset-\epsilon_{ab}
 \mu H_1^aH_2^b
 \\&=-\mu(H_1^1H_2^2-H_1^2H_2^1)
 \\&\equiv \mu(-H_1^0H_2^0+H_1^-H_2^+),
\end{split}
 \tag{SLHA:3}\\
\begin{split}
  V_2 &\supset
  m_{H_{1}}^{2}|H_1|^2
 + m_{H_{2}}^{2}|H_2|^2
 - (m_3^2\epsilon_{ab}H_1^aH_2^b +\text{h.c.}),
\\
&=
  m_{H_{1}}^{2}|H_1|^2
 + m_{H_{2}}^{2}|H_2|^2
+ \left[m_3^2 (-H_1^0H_2^0+H_1^-H_2^+) +\text{h.c.}\right].
\end{split}
 \tag{SLHA:7}
\end{align}
where $\epsilon_{12}=\epsilon^{12}=+1$.
The parameter $m_A^2$, set by \texttt{EXTPAR 24}, is then defined as
\begin{equation}
 m_A^2 = \frac{2m_3^2}{\sin2\beta}.
\tag{SLHA:8}
\end{equation}
The Higgs vacuum expectation value is defined as
\begin{align}
 \langle{H_i^0}\rangle&=\frac{v_i}{\sqrt{2}};&
 m_Z^2 = \frac14(g'^2+g^2)(v_1^2+v_2^2)=\frac14(g'^2+g^2)v^2,
\end{align}
i.e., $v\simeq246\GeV$ is given in the \texttt{HMIX 3}.

\subsubsection{Comparison to GH convention}
In Gunion--Haber~\cite{GH}, the definitions are as follows, with their errata applied:
\begin{align}
 W &\supset -\epsilon_{ab}\mu H_1^aH_2^b,\tag{GH:3.3--3.4}
\\
V\w{soft}&\supset m_1^2|H_1|^2 + m_2^2 |H_2|^2 - (m_{12}^2\epsilon_{ab}H_1^a H_2^b+\text{h.c.}),
\tag{GH:3.9}
%
\end{align}
where $\bE\equiv\tilde R$ in their notation, with the same definition $\epsilon_{12}=1$ (found below Eq.~(3.2)).
So, with the identification $(m_1^2,m_2^2,m_{12}^2)\equiv(m_{H_1}^2,m_{H_2}^2,m_3^2)$, this is identical to SLHA convention.
Also we note the difference of the vacuum expectation values:
\begin{align}
 \langle H_1\rangle &= \pmat{\bar v_1\\0},&
 \langle H_2\rangle &= \pmat{0\\\bar v_2},
\tag{GH:3.7}
\end{align}
with $\bar v_1>0$, $\bar v_2>0$ (3.24), and $\tan\beta=\bar v_2/\bar v_1=v_2/v_1$ (2.8).
Here we use bars to denote $v$s under this definition.

We can then use their results, which say
\begin{align}
 &m_{H_1}^2 = -|\mu|^2 + 2\lambda_1 \bar v_2^2 - m_Z^2/2,
\tag{GH:3.21c}
\\
 &m_{H_2}^2 = -|\mu|^2 + 2\lambda_1 \bar v_1^2 - m_Z^2/2,
\tag{GH:3.21d}
\\
 &m_{H_1}^2 + m_{H_2}^2 + 2|\mu|^2
= m_{3}^2(\tan\beta+\cot\beta)
= \frac{2m_{3}^2}{\sin2\beta},
\tag{GH:3.22}
\end{align}
and, noting that
\begin{align}
 m_Z^2&=(g^2+g'^2)\bar v^2/2,&
 m_W^2&=g^2\bar v^2/2,
\end{align}
which are found below Eq.~(3.19),
\begin{equation}
 \lambda_1
= \frac{g^2+g'^2}{4} + \frac{m_3^2}{2\bar v_1\bar v_2}
= \frac{m_Z^2}{2(\bar v_1^2+\bar v_2^2)} + \frac{m_3^2}{2\bar v_1\bar v_2};
\end{equation}
combining them, we have
\begin{align}
 m_{H_1}^2&=-|\mu|^2-\frac{m_Z^2}{2}\cos2\beta + m_3^2\tan\beta,\\
 m_{H_2}^2&=-|\mu|^2+\frac{m_Z^2}{2}\cos2\beta + m_3^2\cot\beta.
\end{align}
Also the Higgs parameters are given by
\begin{align}
 m_{H^\pm}^2&=(4\lambda_1-g'^2)\bar v^2=m_W^2+m_A^2,\tag{GH:3.16}\\
 m_{H_3^0}^2&=m_{H^\pm}^2-m_W^2=m_A^2,\tag{GH:3.17}\\
 m_{H_1^0,H_2^0}^2 &=
 \frac12\left[m_{H_3^0}^2+m_Z^2\pm\sqrt{
 (m_{H_3^0}^2+m_Z^2)^2-4m_Z^2 m_{H_3^0}^2\cos^22\beta
}\right],\tag{GH:3.18}\\
\tan2\alpha&=\frac{m^2_{H_3^0}+m_Z^2}{m_{H_3^0}^2-m_Z^2}\tan2\beta\tag{GH:3.19}.
\end{align}

\subsubsection{Comparison to SUSY Primer convention}
We here compare the SLHA/GH notation with Martin's SUSY primer~\cite{Martin:1997ns}.
The potentials are given by
\begin{align}
 W\w{MSSM}
&\supset
 \mu H\w u H\w d
\equiv
\mu (H_2^+H_1^- - H_1^0H_2^0),\tag{SP:6.1.1--3}\\
%
\begin{split}
-\mathcal L\w{soft}&\supset
  m_{H\w d}^{2}|H\w d|^2
 + m_{H\w u}^{2}|H\w u|^2
 + (b\epsilon^{ab}H\w u^aH\w d^b +\text{h.c.}),
\end{split}\tag{SP:6.3.1}
\\
&\bar v_u = \langle{H\w u^0}\rangle,\quad \bar v_d = \langle{H\w d^0}\rangle,
\tag{SP:8.1.5}
%
% (|\mu|^2+{m^2_{H\w u}})|H_u|^2
%+(|\mu|^2+{m^2_{H\w d}})|H_d|^2
%+\left(b(H\w u^+H\w d^- - H\w u^0 H\w d^0)+\text{c.c.}\right),\tag{SP:8.1.1}
\end{align}
where $\epsilon^{12}=+1$ (2.13).
So the parameters are identified by the replacement
\begin{align}
\mu&=\mu,\\
m_3^2&=b,
\\
m_{H_1}^2 &= m_{H\w d}^2,
\\
m_{H_2}^2 &= m_{H\w u}^2,
\end{align}
where the LHS are the SLHA parameters and the RHS are those in SUSY primer.
With this identification, we can confirm that the above-shown formulae agree with SUSY Primer's equations (8.1.8)--(8.1.11) and (8.1.19)--(8.1.22).

\subsection{Interaction terms}
Here, to simplify the notation, we omit the SU(2) indices with assuming
\begin{equation*}
 AB=-BA\equiv A^1B^2-B^1A^2.
\end{equation*}
\subsubsection{SLHA convention}
The SLHA~\cite{SLHA} convention for the interection terms are
\begin{align}
 W &=
 - H_2 QY_U\bU
 + H_1 QY_D\bD
 + H_1 LY_E\bE,
\tag{SLHA:3}\\
 V_3 &=
 - H_2 \tilde Q T_U \tilde u^*
 + H_1 \tilde Q T_D \tilde d^*
 + H_1 \tilde L T_E \tilde e^*
\tag{SLHA:5}\\
 V_2 &=
  \tilde Q^*m_{Q}^{2} \tilde Q
 + \tilde L^*m_{L}^{2} \tilde L
 + \tilde u  m_{u}^{2} \tilde u^*
 + \tilde d  m_{d}^{2} \tilde d^*
 + \tilde e  m_{e}^{2} \tilde e^*
\tag{SLHA:7}\\
 \mathcal L_G &= \frac{M_1}{2}\tilde b\tilde b+\frac{M_2}{2}\tilde w\tilde w+\frac{M_3}{2}\tilde g\tilde g + \text{h.c.}
\tag{SLHA:9}
\end{align}
The neutralino and charginos are defined as
\begin{align}
 \tilde\psi^0&=(-\ii\tilde b,-\ii\tilde w^3,\tilde h_1,\tilde h_2)\tra,
&
\tilde\psi^+&=(-\ii\tilde w^+,\tilde h_2^+),
&
\tilde\psi^-&=(-\ii\tilde w^-,\tilde h_1^-).
\tag{SLHA:10+13}
\end{align}

\subsubsection{Comparison to SUSY Primer convention}
In SUSY primer~\cite{Martin:1997ns}, the interaction terms are defined by
\begin{align}
 W\w{MSSM}
&\supset \bar U y_u Q H\w u - \bar d y_d Q H\w d - \bar e y_e L H\w d
\tag{SP:6.1.1}\\
&=
 -H_2Q y_u\tra \bar U + H_1 Qy_d\tra\bar d  +H_1 Ly_e\tra\bar e
\notag\\
V_3 &=
 \tilde u^* a_u \tilde Q H\w u
-\tilde d^* a_d \tilde Q H\w d
-\tilde e^* a_e \tilde L H\w d
\notag\\
&=
-H_2 \tilde Q a_u\tra \tilde u^*
+H_1 \tilde Q a_d\tra \tilde d^*
+H_1 \tilde L a_e\tra \tilde e^*
\notag\\
V_2 &\supset
 \tilde Q^* m_Q^2 \tilde Q
+\tilde L^* m_L^2 \tilde L
+\tilde u^* m_u^2 \tilde u
+\tilde d^* m_d^2 \tilde d
+\tilde e^* m_e^2 \tilde e
\notag\\
&=
 \tilde Q^* m_Q^2 \tilde Q
+\tilde L^* m_L^2 \tilde L
+\tilde u (m_u^2)\tra \tilde u^*
+\tilde d (m_d^2)\tra \tilde d^*
+\tilde e (m_e^2)\tra \tilde e^*
\notag\\
\mathcal L&\supset
-\frac{M_1}{2}\tilde b\tilde b-\frac{M_2}{2}\tilde w\tilde w-\frac{M_3}{2}\tilde g\tilde g + \text{h.c.},\tag{SP:8.1.1}
\end{align}
and the neutralinos and charginos are
\begin{align}
 \tilde\psi^0&=(\tilde b,\tilde w^3,\tilde h_1,\tilde h_2)\tra,
&
\tilde\psi^+&=(\tilde w^+,\tilde h_2^+),
&
\tilde\psi^-&=(\tilde w^-,\tilde h_1^-).
\tag{SP:8.2.1+15}
\end{align}
So the notations can be matched with
\begin{align}
 Y_U, Y_D, Y_E &= y_u\tra, y_d\tra, y_e\tra\\
 T_U, T_D, T_E &= a_u\tra, a_d\tra, y_e\tra\\
 m_Q^2, m_L^2  &= m_Q^2, m_L^2\\
 m_u^2, m_d^2, m_e^2  &= (m_u^2)\tra,(m_d^2)\tra,(m_e^2)\tra,\\
 M_1, M_2, M_3 &= M_1, M_2, M_3,
\end{align}
where the LHS are the SLHA parameters and the RHS are those in SUSY primer.

\section{Reverting sfermion rotation}
In SLHA2 convention~\cite{SLHA2}, the sfermion mass basis $\tilde f_i$ and super-CKM/PMNS basis $\tilde{\vc f}$ are related as
\begin{equation}
 \tilde f_i = R_{iA}\tilde f_A,
\end{equation}
where $R$ is the rotation matrix (e.g.~\texttt{USQMIX}), capital (small) indices are used to denote super-CKM/PMNS (mass) eigenstates, and
the super-CKM/PMNS eigenstates are defined as, e.g.,
\begin{equation}
 \tilde {\vc u} = (
\tilde u\w L, \tilde c\w L, \tilde t\w L,
\tilde u\w R, \tilde c\w R, \tilde t\w R)\tra.
\end{equation}

If the model has no flavor (and CP) violation and the left--right mixings in the first and second generation are negligible, we can convert the SLHA2 output to SLHA1 output.
The SLHA1 mixing matrix $R'$ is obtained as
\begin{equation}
R' = \begin{pmatrix}
  R_{a3} & R_{a6}\\
  R_{b3} & R_{b6}
\end{pmatrix}
\end{equation}
with all elements are non-zero and $a<b$, and thus $\tilde f_a\equiv\tilde f^{\text{3rd}}_1$
and  $\tilde f_b\equiv\tilde f^{\text{3rd}}_2$.
The first and second generation fermions, $\tilde f_A$, are identified with $\tilde f_i$, where $|R_{iA}|>|R_{jA}|$ for all $j\neq i$.
\bibliography{bibliography}
\end{document}