\section{The Generic Solver}

Solving a parity game is done by a central module called the \emph{generic solver}.
It combines the universal optimisations described above with any of the implemented
algorithms of heuristics described above. This is realised by taking a solver, i.e.\
one of these algorithms or heuristics as a parameter. Then it roughly works as follows.
\begin{enumerate}
\item Self-cycles are eliminated from the game, and attractors of nodes for which the
      self-cycle is part of a winning strategy are computed and removed.
\item The entire game is decomposed into SCCs.
\item Terminal SCCs are solved as follows.
      \begin{enumerate}
        \item Priorities are compressed.
        \item The SCC is checked for being a special case of a game.
              \begin{itemize}
                 \item If it is, winning regions and strategies are constructed accordingly.
                 \item Otherwise, the solver given as the parameter is used to solve this
                       SCC.
              \end{itemize}
        \item Attractors of the computed winning regions are also computed, together with
              corresponding strategies which are added to the winning regions and strategies.
        \item The computed winning regions are removed from the game.
        \item All non-terminal SCCs which have lost some nodes in the removal of these
              attractors are again decomposed into more fine-grained SCCs.
      \end{enumerate}
\item Step 3 is repeated until the entire game is solved.
\end{enumerate}
Note that the removal of terminal SCCs will make other, previously non-terminal SCCs terminal.
Also, note that this scheme is sound -- the regions and strategies it computes are
in fact winning regions and winning strategies for the corresponding players on the given
game -- if the parameter solver is sound. Hence, it can also be used with sound heuristics.
Furthermore, it is complete if it is guaranteed that the parameter solver solves at least
one node of every SCC that it is given. Hence, the solvers used as backends need not be
complete for the generic solver to be complete. This is why this scheme can solve whole games
using heuristics that are incomplete themselves.

We also note that the features SCC decomposition, detection of special cases, and priority
compression can be switched off via command-line options. This may be useful when the performance
of an algorithm on its own is to be measured. In addition, it is possible to turn the feature
priority propagation on in which case it is done before priority compression. However, in general
this does not seem to be an optimisation since it slows down virtually any backend.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


\chapter{Engineering NOON states in one-dimensional quantum gases}
\label{ch:1d}

Until recently, simulating dynamic control protocols to engineer specific quantum states have been difficult to perform on GPU devices without full control of the software, itself.
This meant that researchers wishing to engineer specific quantum states by using GPU simulations would be required to have domain-specific knowledge in both software design and quantum mechanics, neither of which are trivial to understand.
This chapter serves as motivation for several methods to be discussed in Chapter~\ref{ch:gpu} to allow for the simulation of quantum control methods on GPU devices, with a particular focus on quantum optimal control~\cite{werschnik2007} and Shortcuts To Adiabaticity (STA)~\cite{guery2019}.
Both of these control protocols will be used in a physical example for the non-adiabatic generation of superposition states in a one-dimensional Tonks--Girardeau (TG) gas~\cite{schloss2016}.
Between the two methods, the drawbacks to STA methods are the strengths of quantum optimal control.
Where shortcuts can only be used on a specific subset of problems to evolve adiabatically and that are amenable to the analytical methods used, quantum optimal control is a more general tool for a wider variety of systems.
On the other hand, STA protocols are semi-analytical and if such protocols can be found, they greatly reduce the computational cost to engineering particular quantum states.
To start, I will discuss the field of optimization algorithms before moving to STA protocols and a physical system using both in practice.

The work in this chapter has been published in \textit{New Journal of Physics}~\cite{schloss2016}, and in this publication, I performed all calculations for all the figures generated and focused primarily on optimal control methods.
The STA protocol was devised by J\'er\'emie Gillet, and the research was supervised by Albert Benseny and Thomas Busch.

\section{Optimization methods}

Optimization algorithms have become essential to many areas of modern computing and focus on either minimizing or maximizing a cost function by modifying several control parameters~\cite{lewis2012}.
The number of control parameters create an $n$-dimensional space to traverse, and optimization algorithms are tasked at finding the global minimum or maximum of this domain.
For certain domains, it is difficult to find a global optimization strategy and many methods instead get caught in local minima while attempting to find the appropriate solution.
Because generalized optimization is such a fundamental problem, there are many known optimization methods, such as gradient descent~\cite{ruder2016}, the Nelder--Mead or simplex method~\cite{nelder1965}, genetic algorithms~\cite{koza1997}, and many more~\cite{lewis2012}.
Of these, gradient descent is often considered to be one of the easiest to implement with favorable complexity and convergence guarantees, and because of this, it has become ubiquitous in many areas such as machine learning, which is of particular interest for GPU engineering.
Even so, there are limitations to gradient descent, such as its dependence on calculating the gradient of the cost function's solution domain along with lengthy convergence times for high-precision solutions.

For this work, I am primarily interested in the area of quantum optimal control, which is a method typically used to determine the optimal time-dependent control parameters necessary to transform an initial state to a final, desired state~\cite{werschnik2007}.
This means that one will often be maximizing the fidelity between states, defined as $\mathcal{F} = |\braket{\psi|\phi}|^2$, where $\psi$ is the engineered quantum state and $\phi$ is the state one is attempting to replicate.
This problem can be re-framed as an attempt to find the maximum of a fidelity \textit{landscape}, where each point in the domain is a calculation of the fidelity.
This means that each point in the fidelity landscape necessarily involves solving the Schr\"odinger equation for chosen control parameters.
For this reason, I will be introducing a gradient-less (derivative-free) optimization algorithm, the Nelder--Mead (simplex) method, which is often used as a heuristic approach to this problem and there are several known optimizations for this method~\cite{nelder1965,kolda2003,lewis2007}.
The Nelder--Mead method is also the recommended optimization algorithm for the chosen quantum optimal control method of the physical example that will be discussed in Section~\ref{sec:CRAB}, Chopped RAndom Basis (CRAB) optimal control.

\subsection{Nelder--Mead}
\label{sec:NM}

The Nelder--Mead method is one of the most commonly implemented gradient-less optimization algorithms to-date and relies heavily on the concept of a simplex, which is a generalization of the three-dimensional tetrahedron to $n$-dimensions.
For example, a 0-simplex is a point, a 1-simplex is a line, a 2-simplex is a triangle, a 3-simplex is a tetrahedron, and so on.
If the Nelder--Mead method is attempting to optimize a cost function with $n$ control parameters, an $n+1$ point simplex will be created, and the points of this simplex will be manipulated until they have converged to a minimum or maximum in the domain.
For this section, I will focus on the method introduced in the original work by Nelder and Mead in 1965 while working at the National Vegetable Research Station in Warwick, England~\cite{nelder1965}.
For this method, one is attempting to find the minimum value in some $n$-dimensional space.
In Nelder and Mead's original notation, a ``height'' is defined as the value of the cost function with provided input parameters, often depicted as the an elevation out of the plane.

For $P_i \in i=\{0,...,n\}$ simplex points with heights of $y_i \in i=\{0,...,n\}$, the Nelder--Mead method is tasked at minimizing all points such that $\sqrt{\sum(y_i-\bar y)^2/n} < \eta$, where $\bar y$ denotes the height of the centroid location of the simplex, and $\eta$ is some pre-defined convergence threshold value.
This convergence criteria assumes that if all points have converged with this method, the final location must be the minimum of the optimization domain; however, as mentioned in the previous section, this method may become trapped in a local minimum.
At every step in the Nelder--Mead method, the points with the highest and lowest values are determined and denoted as $P_h$ and $P_l$, respectively.
In addition, the centroid location is found as $\bar P$.
This method then performs up to three basic operations on the simplex, itself:

\begin{description}

\item[Reflection] For this operation, $P_h$ is flipped across $\bar P$, such that the new location,
\begin{equation}
P^* = (1+\alpha)\bar P - \alpha P_h.
\end{equation}
\noindent Here, $\alpha > 0$ is a constant known as the reflection coefficient.
After this operation, the new height is compared to $y_l$.
If it is lower, the method proceeds to an expansion step.
Otherwise, the method checks whether the new height is lower than some other $y_i \in \{0,...,n\},n\neq \{l,h\}$, and if it is, the point is kept and the method continues to find the new simplex ordering.
If it is found that the reflected point, $P^*$, is higher in value than all other points, the method then keeps whichever point corresponds to the lowest height between the reflected and previous highest point and proceeds to the contraction step.

\item[Expansion] For this operation, an expansion is performed, such that the new location,
\begin{equation}
P^{**} = \gamma P^* + (1-\gamma)\bar P.
\end{equation}
\noindent Here, $\gamma > 1$ is a constant known as the expansion coefficient.
If the new height is less than the previously lowest point, the expanded point, $P^{**}$, is kept, otherwise the reflected point is kept.

\item[Contraction] For this operation, a contraction is performed, such that the new location,
\begin{equation}
P^{**} = \beta P_h + (1-\beta)\bar P.
\end{equation}
\noindent Here, $0 < \beta < 1$ is a constant known as the contraction coefficient.
If the contracted point, $P^{**}$, is lower than the previous highest point, the contracted point is kept, otherwise, the entire simplex is contracted closer to the lowest point with $P_i = (P_i + P_l)/2$.
\end{description}

Each step in the Nelder--Mead method begins with a proposed reflection of the least optimal point about its centroid position.
From there, the method follows the protocol described above.
The choice of $\alpha$, $\beta$, and $\gamma$ is somewhat arbitrary and should be optimized by-hand.
An example of the centroid locations for minimization using this method with the Rosenbrock banana function~\cite{pohlheim2007} can be seen in Figure~\ref{fig:minimize_NM}.

\begin{figure}
\center \includegraphics[width=0.75\textwidth]{data/1d/NM/NM.pdf}
\caption{Plot of the centroid locations (green dots connected with white line) for the Nelder--Mead method while optimizing the Rosenbrock banana function, $f(x,y)=(a-x)^2+b(y-x^2)^2$, with $a=1$ and $b=100$.
Here, centroid locations start at $(-0.851,-1.553)$ and end at the known minimum of $(1,1)$ in 19 iterations.
Here, $\alpha = 1$, $\beta = 0.5$, and $\gamma = 1.5$.}
\label{fig:minimize_NM}
\end{figure}

Even though the Nelder--Mead method is a heuristic approach and can become stuck in a local minimum, as long as a sufficient number of random simplexes are chosen at the start of the simulation, it can be used to find an adequately optimal solution.
Ultimately, any gradient-less optimization algorithm can be used to traverse the fidelity landscape for quantum optimal control, and in the next section, I will discuss a common method used in the field: the CRAB optimal control method.

\subsection{Chopped random basis optimal control}
\label{sec:CRAB}

The CRAB technique works by modifying a control parameter for a given system, $\Gamma$, with a multiplicative term as
\begin{equation}
\Gamma^{\text{CRAB}}(t) = \Gamma^0(t)\gamma(t),
\end{equation}

\noindent where $\Gamma^0(t)$ is an initial guess, and the function $\gamma(t)$ is written as a sum of $2J$ sinusoidal functions,
\begin{equation}
\gamma(t)=1+\frac{1}{\lambda(t)}\sum_{j=1}^J(A_j \sin(\nu_jt) + B_j\cos(\nu_jt)).
\end{equation}

\noindent Here, $\lambda(t)$ is usually defined by the system, such that $\Gamma^{\text{CRAB}}$ and $\Gamma^0$ coincide at initial and final times.
This means that $\lim_{t\rightarrow 0} \lambda(t) = \lim_{t\rightarrow T}\lambda(t) = \infty$, where $T$ is the final time of evolution.
As such, any smooth function may be chosen with these constraints.
For example, one might use
\begin{equation}
\lambda(t) = \frac{T^2}{4t(t-T)},
\end{equation}
\noindent which satisfies the provided conditions.
This then transforms the optimization problem into an optimization of the space spanning $\{A_j, B_j, \nu_j\}$, which can be done by using Nelder--Mead with a simplex of random initial points.
As an example, if $J = 10$, a thirty-dimensional space would be created and a thirty-one simplex would be formed to traverse this space.
It is important to remember that each new simplex and simplex-operation requires re-solving the Schr\"odinger equation for those values, and as such, this is a computational costly technique.
Even though higher $J$ values will produce a more accurate result, lower values should be chosen, if possible.

This method is a general-purpose computational tool for determining the optimal pulse to ensure the generated state is as close to the desired state as possible, and I will show an example of it being used later in this chapter.
Even so, it is sometimes worthwhile to attempt to devise analytical frameworks that serve a similar purpose, and for certain systems, this can be done with STA protocols.

\section{Shortcuts to adiabaticity}

STA protocols are semi-analytical methods that allow for quantum state generation while retaining the effects of adiabatic movement.
Here, adiabatic processes are defined as actions by which slow changes in the control parameters leave particular properties invariant, such as the quantum number~\cite{guery2019}.
The ultimate goal of STA protocols is to achieve adiabatic motion in sub-adiabatic time, and this can be done in a number of ways; however, in this section, I will introduce only the invariant-based inverse-engineering approach using Lewis--Riesenfeld invariants~\cite{torrontegui2013}.
In particular, I will focus on the specific methods necessary for the example to be introduced later in this chapter and much of this section will follow traditional derivations from various sources~\cite{torrontegui2013,guery2019, schloss2016}.

With the method of Lewis-Riesenfeld invariants, the theory for relating different eigenstates of a time-dependent, Hermitian invariant to the solutions to the Schr\"odinger equation~\cite{lewis1969} can be applied to systems with time-dependent Hamiltonians, such that
\begin{equation}
i\hbar \frac{\partial I(t)}{\partial t} - \left[\mathcal{\hat H},I(t)\right] = 0,
\end{equation}
\noindent where $I(t)$ is the invariant.
This ensures that the expectation values for the states driven by $\mathcal{\hat H}$ are constant in time.
It is possible to expand the state of the system $\ket{\Psi(t)}$ into the orthonormal basis of the invariant with,
\begin{equation}
\ket{\Psi(t)} =\sum_{n=1}^\infty c_n e^{i\alpha_n(t)} \ket{\phi_n(t)},
\end{equation}
\noindent where $c_n$ are time-independent amplitudes for each state, and $\ket{\phi_n(t)}$ are orthonormal eigenvectors of the invariant, such that
\begin{equation}
I(t) = \sum_n^\infty\ket{\phi_n(t)}\lambda_n\bra{\phi_n(t)}.
\end{equation}
\noindent Here, the $\lambda_n$ are real constants, and the phase is defined as ~\cite{lewis1969}
\begin{equation}
\alpha_n(t) = \frac{1}{\hbar}\int_0^t\braket{\phi_n(t')|i\hbar\frac{\partial}{\partial t'} - \mathcal{\hat H}(t')|\phi_n(t')}dt'.
\end{equation}

From here, inverse engineering can be used to create the desired time-dependent Hamiltonian, by imposing some dynamics on the system.
The phases, $\alpha_n(t)$ may be chosen as arbitrary functions to create a time-dependent, unitary evolution operator,
\begin{equation}
U = \sum_n^\infty e^{i\alpha_n(t)}\ket{\phi_n(t)}\bra{\phi_n(0)},
\end{equation}
\noindent that obeys $i\hbar \dot U = \mathcal{\hat H}(t)U$ and the dot is a time-derivative.
If one considers Hamiltonians of the Lewis and Leach variety~\cite{lewis1982},
\begin{equation}
\mathcal{\hat H} = \frac{p^2}{2m}  -F(t)x + \frac{m}{2}\omega^2(t)x^2 + \frac{1}{\rho(t)^2}U\left[\frac{x-x_c}{\rho(t)}\right] + f(t),
\label{eqn:HSTA}
\end{equation}
there will be an invariant that is quadratic in momentum,
\begin{equation}
I = \frac{1}{2m}[\rho(p-m\dot x_c)-m\dot \rho(x-x_c)]2 + \frac{1}{2}m\omega_0^2\left( \frac{x-x_c}{\rho} \right)^2 + U\left( \frac{x-x_c}{\rho}\right).
\end{equation}
\noindent These equations are valid so long as $\rho$, $x_c$, $\omega$, and $F$ satisfy
\begin{align}
\ddot \rho + \omega^2(t)\rho &= \frac{\omega_0^2}{\rho^3} \label{eqn:rho}\\
\ddot x_c + \omega^2(t)x_c &= F(t)/m \label{eqn:xc},
\end{align}
\noindent with $\omega_0$ as a constant whose physical interpretation depends on the system.
As in the case of quantum optimal control, additional constraints must be considered to ensure the Hamiltonian and its invariant commute at initial and final times $t_0$ and $T$.

Now that I have provided specific examples of methods used in quantum engineering, it is time to put them into practice with an example of creating large-scale superposition states non-adiabatically in the highly-correlated TG gas regime.

\section{Non-adiabatic generation of NOON states in a Tonks--Girardeau gas}

For this example application of quantum optimal control and STA protocols, I am interested in generating the maximally entangled $\ket{N,0} + \ket{0,N}$ (NOON) state, which is composed of two modes where all particles can be found exclusively in one or the other.
Recently, Hallwood \textit{et al.} proposed an experimentally realistic method to generate NOON states in a gas of strongly interacting, neutral bosons on a one-dimensional ring.
In this system, different rotational states can be coupled by breaking the rotational symmetry and it is possible to create superposition states with rotating and non-rotating components.
Because the atoms are considered to be in the strongly correlated TG gas regime, this process results in a macroscopically-entangled state.
It is worth discussing the TG gas in further detail before moving to the precise method of NOON state generation for this example.

\subsection{Tonks--Girardeau gas}

As mentioned in Chapter~\ref{ch:splitop}, the TG gas consists of a number of bosons that have the properties of spinless, non-interacting fermions.
This is a particular case of the one-dimensional Schr\"odinger equation where the repulsive interaction strength $g\rightarrow\infty$.
In this case, the bosons cannot be at the same location, which acts formally similar to the Pauli-exclusion principle for fermionic systems.
In this case, the bosonic Hamiltonian can be solved by the Bose--Fermi mapping theorem \cite{girardeau2001ground, girardeau2001measurement}, which replaces the interaction terms in the Hamiltonian with a boundary condition on the many-body bosonic wavefunction,
\begin{equation}
\Psi_B(x_1, x_2, \ldots, x_N) = 0,\qquad \mathrm{if}\qquad x_i - x_j = 0 \quad\textrm{with}\quad i \ne j,
\end{equation}

\noindent following the many-body Hamiltonian,
\begin{equation}
\mathcal{\hat H} = \sum_{n=1}^N\left(\frac{p_n^2}{2m} + V_n + b\delta(x_n)\right) + \sum_{j<k}V(|x_j - x_k|).
\end{equation}

\noindent Here, $p_n = -i\hbar\frac{\partial}{\partial x_n}$, $V_n = \frac{1}{2}m\omega^2x_n^2$, $b\delta(x_n)$ is a delta barrier with strength $b$, and $V$ is an interaction potential between bosonic particles.
This allows us to treat strongly interacting bosons as spinless, non-interacting fermions, for which the many-body wavefunction can be calculated using the Slater determinant~\cite{slater1929},
\begin{equation}
\Psi_F (x_1, x_2, \ldots, x_N) = \frac{1}{\sqrt{N}} \det\Big[\psi_n(x_j)\Big]_{n,j=1}^N,
\end{equation}
\noindent where $\psi_n(x_j)$ are the single-particle eigenstates of the trapping potential $V_n$.
Because the fermionic many-body wavefunction is anti-symmetric, it needs to be symmetrized for bosonic states as,
\begin{equation}
\Psi_B(x_1, x_2, \ldots, x_N) =
\prod_{i < j}
\mathrm{sgn}(x_i - x_j)\Psi_F(x_1, x_2, \ldots, x_N),
\end{equation}
\noindent which means that calculating the time evolution of a TG gas requires evolving single-particle states, governed by a much simpler Hamiltonian.

\subsection{NOON states in a TG gas}
\label{sec:controltro}

Similar to other ring systems introduced in the literature~\cite{das2002,girardeau2009}, the system suggested by Hallwood \textit{et al.} considers a gas of $N$ interacting bosons of mass $m$ on a one-dimensional ring with circumference $L$~\cite{hallwood2010}.
In addition, this system includes a potential barrier, modeled by a Dirac $\delta$-function that rotates with an angular frequency $\Omega$, as shown in Figure~\ref{fig:ring_scheme}.
In the rotating frame, the scaled Hamiltonian of the system in the rotating frame is given by \cite{hallwood2010}
\begin{equation}H^{(N)} = \sum_{n=1} ^{N} \left[{\frac{1}{2}\bigg(-i\frac{\partial}{\partial x_n}-\Omega}\bigg)^2 + b\delta(x_n) +g \sum_{j<k} ^{N} \delta (x_j - x_k )\right],
\end{equation}
\noindent where $b$ is the height of the barrier (in units of $\hbar^2/mL^2$), $x_n \in \left[-1/2,1/2\right]$ is the position of the $n$--th particle (in units of $L$) and $g$ (in units of $\hbar^2/mL^2$) is the effective interaction strength between the atoms.
As discussed in the previous section, the evolution of the full TG gas can be calculated from the evolution of single-particle states, and in the case of this system, the Hamiltonian in the laboratory frame becomes,

\begin{equation}
H = -\frac{1}{2} \frac{\partial^2}{\partial x} + b\delta \left[ x-x_0(t) \right],
\end{equation}
where $x_0$ is the position of the barrier at time $t$.

\begin{figure}
\center \includegraphics[width = 0.5\textwidth]{data/1d/scheme.pdf}
\caption{Schematic of the system.
Here, the density profile for five atoms in a TG gas is shown being stirred by a highly localized potential, indicated by the vertical line.}
\label{fig:ring_scheme}
\end{figure}

The energy spectrum of this system is shown in Figure~\ref{fig:avoid} as a function of the rotational frequency $\Omega \equiv \dot x_0 /L$ of the system.
In Figure~\ref{fig:avoid}(a) it is shown that in the absence of a barrier, when the eigenstates of $\mathcal{\hat H}$ are plane waves with quantized angular momentum in integer multiples of $2 \pi$, each angular momentum manifold exists separately such that the energy levels cross; however when $b>0$ (Figure~\ref{fig:avoid}(b)), the rotational symmetry is broken and avoided crossings appear in the energy spectrum.
This makes transitions between different manifolds possible~\cite{schenke2012}.

\begin{figure}

 \centering
 \subfigure{
 \centering
 \includegraphics[width = 0.4\linewidth]{data/1d/cross.png}}
 \subfigure{
 \centering
 \includegraphics[width = 0.4\linewidth]{data/1d/nocross.png}}

\caption{Single-particle energy spectrum as a function of $\Omega$ for a barrier height of (a) $b = 0$ and (b) $b=2$.
When a barrier is present in the system, avoided crossings appear in the energy spectrum which grow as the barrier strength increases.}
\label{fig:avoid}
\end{figure}

By adiabatically accelerating the barrier's rotational frequency from 0 to $\pi$, a particle will enter a superposition between two rotational states, and in the case of the TG gas, this will create a macroscopic NOON superposition state between successive values of angular momentum~\cite{hallwood2010}.
This means that the manifolds will have an angular momentum of 0, 1, 2, $\ldots$, $N$, where $N$ is the number of particles in the system.
Any non-adiabatic behavior around the rotational frequencies of the avoided crossings can lead to a transition to a higher energy state and destroy the NOON state.
For this reason, the condition for adiabaticity must depend on the gap size, which is dictated by the barrier strength~\cite{nunnenkamp2008}; however, for a constant delta barrier, the gap size stays constant to first-order approximation~\cite{hallwood2007}.

Because this system requires adiabatic movement to properly generate the NOON state, it is difficult to efficiently generate it experimentally.
For this reason, it is a perfect example of a system where quantum optimal control and STA protocols can be used to rapidly engineer the appropriate states.
For quantum optimal control in this system, a non-adiabatic rotational frequency $\Omega(t)$ must be found, for which I will use the CRAB technique with an initial condition of $\Omega = 0$ and final condition of $\Omega = \pi$.
For each simulation in the fidelity landscape, this pulse will be modified with procedurally generated sinusoidal functions, the fidelity will be calculated, and then the Nelder--Mead method will be used to optimize the result.
This will allow one to determine an optimal pulse that maximizes the fidelity of the generated state when compared to the expected NOON state in a pre-set amount of time.
For this system, I will show the optimal pulse for the cases where I manipulate the rotational velocity, the barrier height, and both.

For STA protocols, the acceleration process will be split into two, one that breaks the rotational symmetry and another that accelerates the atoms.
At the end of the protocol, the potential is lowered to restore rotational symmetry.
Here, it is worth mentioning that a FAst, QUasi-ADiabatic (FAQUAD) shortcut for the creation of superposition states in a TG gas has also been created with some similarities~\cite{garaot2015}.

For both of these methods, instead of calculating the fidelity I calculate the \textit{infidelity}, which is simply $1-\mathcal{F} = 1-|\braket{\Psi|\Phi}|^2$, as the function to minimize.
It is also worth mentioning that the fidelity between two many-particle states in a TG gas can be calculated by using the method of mode projections~\cite{campo2011,lelas2011},
\begin{eqnarray}
\braket{\Psi | \Phi} &=& \frac1{N!} \sum_{\eta, \mu \in P} \epsilon_\eta \epsilon_\mu \braket{\psi_{\eta_1}(x_1) | \phi_{\mu_1}(x_1) } \cdots \braket{ \psi_{\eta_N}(x_N) | \phi_{\mu_N}(x_N) } \nonumber \\
\label{eq:fid}
&=&
\det \Big[ \braket{\psi_i | \phi_j }\Big]_{i,j=1}^N
\end{eqnarray}
which follows directly from the form of the TG state~\cite{girardeau1960}
\begin{equation}
\Psi(x_1,x_2,\ldots, x_N)= \frac1{\sqrt{N!}} \prod_{i<j}\textnormal{sign}(x_i-x_j) \sum_{\eta \in P} \epsilon_\eta \psi_{\eta_1}(x_1)\cdots\psi_{\eta_N}(x_N).
\label{eq:TG}
\end{equation}
Here $P$ represents the set of all permutations of $N$ elements, $\epsilon_\eta$ represents the anti-symmetric tensor of the permutation $\eta$, and $\psi_i$ represent the orbitals.
Now I will discuss the findings with both quantum optimal control and STA protocols.

\subsection{Quantum optimal control protocols}

\begin{figure}
 \centering
 \subfigure{
 \centering
 \includegraphics[width=0.45\textwidth]{data/1d/figR0.pdf}}
 \subfigure{
 \centering
 \includegraphics[width=0.45\textwidth]{data/1d/figB0.pdf}}
 \caption{Accelerating a single particle from the ground state with $J=15$.
 (a) Optimal rotational velocity pulses for $T = 1$, 10, and 100 for fixed barrier height $b=1$.
 (b) Optimal barrier height for a linearly increasing rotational velocity, $\Omega = \pi t/T$ for $T=1$, 10, and 100.}
 \label{fig:pulses}
\end{figure}

First, I will focus on the acceleration of a single particle, initially in the ground state of the system.
Figure~\ref{fig:pulses} (a) shows the results of this simulation if the barrier height is kept constant and one assumes an initial unmodified pulse that corresponds to a linear ramp from $\Omega = 0$ to $\pi$ for a preset total time, $T$.
For longer evolution times, there are many local maxima for the fidelity, and
as such, longer evolution times effectively produce noisy signals and the Nelder--Mead method converges on one of many local minima.
For shorter evolution times, the pulse greatly affects the system and the shapes vary greatly from the initial linear ramp.
The infidelities for the linear guess pulse and its corresponding optimized pulse can be found in Figure~\ref{fig:lfid}, and one can see an improvement of several orders of magnitude.
Here, for longer evolution times, the initial linear pulse is a reasonable method to generate NOON states with an infidelity of $10^{-2}$ because it is already close to adiabatic; however, even in this case, the NOON state generation fidelity is better with optimization.
For all quantum optimal control results in this chapter, the CRAB method was run 100 times and the data with the highest fidelity was kept.

For optimizations of the barrier strength, a simple linear ramp for $\Omega$ and an initial and final height for the barrier of $b = 1$ were chosen.
The optimal pulses for the barrier height for $T=1$, 10, and 100 are shown in Figure~\ref{fig:pulses}(b) and shorter evolution times similarly produce larger deviations from the initial pulse.
Again these pulses lead to significant improvements in the fidelity shown in Figure~\ref{fig:lfid}.

\begin{figure}
\centering
 \subfigure{
 \centering
 \includegraphics[width=0.45\textwidth]{data/1d/figR1.pdf}
 }
 \subfigure{
 \centering
 \includegraphics[width=0.45\textwidth]{data/1d/figB1.pdf}
 }
 \caption{
 Optimal pulses to accelerate a single particle initially in the ground state of the trap
 for $T = 1$, 10, and 100 for the (a) rotational velocity and (b) barrier height when optimizing over both simultaneously. }
 \label{fig:pulses_pair}
\end{figure}

With the CRAB method, it is possible to optimize over as many control parameters as one would like, and as such, it is possible to optimize over both the barrier strength and rotational frequency.
The results can be seen in Figure~\ref{fig:pulses_pair}, where (a) is the modified rotational frequency and (b) is the barrier height.
When comparing to the previous cases, similar trends emerge.
In particular, shorter evolution times result in less noisy optimizations when compared to longer evolution.
Even so, all sets of pulses are radically different when compared to optimizations over a single variable.
When comparing the fidelities in Figure~\ref{fig:lfid}, it is clear that optimizations over rotation alone provide the simplest method to optimize the fidelity.
It is likely that each run of the CRAB method is stuck in a local minimum in the fidelity landscape at some point and that optimization over both variables can provide the same optimization as rotating, alone.

\begin{figure}
 \centering \includegraphics[width=0.5\textwidth]{data/1d/figlfid.pdf}
\caption{ Infidelities as a function of the overall process time for optimally controlled rotational acceleration, barrier height, or both.
Here, `linear' refers to an unoptimized linear acceleration from $\Omega = 0$ to $\pi$ while keeping the barrier height fixed at $b=1$.
}
\label{fig:lfid}
\end{figure}

\begin{figure}[t]
 \centering
 \subfigure{
 \centering
 \includegraphics[width=0.45\textwidth]{data/1d/figTG3.pdf}
 }
 \subfigure{
 \centering
 \includegraphics[width=0.45\textwidth]{data/1d/figTG5.pdf}
 }
 \caption{
Infidelities for the evolution of a TG gas with (a) $N=3$ and (b) $N=5$ particles using the CRAB optimal control technique.
The infidelities for optimized pulses with the particle at the Fermi edge is shown as blue crosses, and for the full TG gas as red circles.
Here, the green squares show the fidelity of a linear pulse for the atom closest to the Fermi edge.
A clear range where the CRAB algorithm is effective for generating NOON states with multiple particles can be clearly identified.}
 \label{fig:TGOC}
\end{figure}

As such, when discussing the dynamics of a TG gas with 3 and 5 particles, I have only optimized over rotation.
Because of the Bose--Fermi mapping theorem, the evolution of an $N$-particle TG gas can be calculated by evolving a gas of $N$ spinless fermions.
In the zero-temperature limit, the fermions in the initial and target state create a Fermi sea by filling the lowest $N$ energy levels.
In this case, only atoms near the Fermi edge can transition into empty states and it is thus crucial to optimize the dynamics of the overall gas with respect to the particle with highest energy~\cite{garaot2015}.
In Figure~\ref{fig:TGOC}, I show the fidelity for the particle closest to the Fermi edge and the entire TG gas for $N=3$ and 5.
In this figure, one can see that by performing the optimization for the atoms near the Fermi edge, one can increase the fidelity of the entire gas for certain regimes; however, in contrast to Figure~\ref{fig:lfid}, there seems to be no fidelity increase from a linear pulse for short evolution times.
One can also observe what appears to be a crossover regime where optimizations of the particle at the Fermi edge seem to fail, but evolution of the entire gas is still slightly better than the linear pulse.
It is clear that the CRAB method creates highly effective pulses; however, for very short and long evolution times, the fidelity increase from a linear pulse is not as drastic.

\subsection{Results with STA protocols}

\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{data/1d/STAscheme.png}
\caption{Scheme for the acceleration of a single atom using STA.
 In this example, the homogeneous ground state gets localized, accelerated and released at the angular velocity of $\Omega=\pi$ into the state
 $\left( \exp(i 2\pi x) +1 \right) /\sqrt 2$.
 The atomic density is indicated in blue and the potential is in red.}
\label{fig:STA-scheme}
\end{figure}

In this section, I will describe an STA protocol to generate NOON states in this system non-adiabatically, and though this was mentioned briefly in Section~\ref{sec:controltro}, it will be described more rigorously here.
In this case, I will start with rotational symmetry and break break this symmetry by introducing a time-dependent external potential at $t=0$ and removing it in the end.
For this, we do not choose a $\delta$ function, but instead a harmonic or sinusoidal potential along the ring.

The protocol consists of five steps:
\begin{enumerate}
\item Adiabatic raising of a weak harmonic or sinusoidal potential around the ring.
\item Fast tightening of this potential to localize the particles.
\item Accelerating the particles by moving the center of the potential.
\item Loosening the potential by reversing step 2.
\item Adiabatic lowering of the harmonic or sinusoidal potential.
\end{enumerate}
A schematic of this process is shown in Figure~\ref{fig:STA-scheme}.
For steps 2-4, pre-existing STA protocols can be used, and these will be discussed in this section.
The full protocol for the TG ring example will follow the STA methods outlined above with Lewis--Riesenfeld invariants and in this case, one needs to fulfill boundary conditions such that $\mathcal{\hat H}(t_0) = \mathcal{\hat H}(T)=p^2/2m$.
To be clear, the NOON states created with the STA protocol are slightly different those generated with quantum optimal control, as the STA variant does not rely on a $\delta$ barrier.

One of the two shortcuts explored for this system involves raising and lowering a harmonic potential~\cite{chen2010,chen20102}.
For this shortcut, a stationary harmonic potential is required and $F$, $x_c$, and $U$ from Equation~\eqref{eqn:HSTA} can all be set to zero, leading to
\begin{equation}
 \mathcal{\hat H}= -\frac{1}{2} \frac{\partial^2}{\partial x}+ \frac 1 2 \omega^2(t) x^2.
\end{equation}
\noindent To change the frequency while keeping the commutation relations and $\omega(t)$ continuous, one must impose the following conditions:
\begin{equation}
 \begin{array}{lcl}
\rho(t_0)=1, && \rho(t_f)=\gamma=\sqrt{\omega_0 / \omega_f},\\
\dot \rho(t_0)=0, && \dot \rho(t_f) =0, \\
\ddot \rho(t_0)=0, && \ddot \rho(t_f)=0.
\end{array} \label{eqn:squeeze}
\end{equation}

\noindent \noindent Which, together with Equations~\eqref{eqn:rho} and \eqref{eqn:squeeze} allow one to choose any form of $\rho$.
A good choice is the polynomial,
\begin{equation}
 \rho (s) = 6 \left(\gamma -1\right) s^5 -15 \left(\gamma-1\right) s^4 +10 \left(\gamma-1\right)s^3 + 1, \label{eq:rho_pol}
\end{equation}
\noindent where $s=(t-t_0)/(t_f-t_0)$ allows one to numerically find a solution for $\omega(t)$ that leads to the squeezing or expansion of the particle wavefunction with high fidelity in a short time.
As an important note, for small values of $\omega_0$, Equation~\eqref{eqn:rho} leads to purely imaginary values for $\omega(t)$, corresponding to repulsive potentials.
In order to avoid this and because the final states of this protocol require the external potential to be absent, the first and final steps in the protocol for this system involve adiabatically raising and lowering a potential to a suitable $\omega_0$ value.

Once the potential has been raised, the particles are then accelerated to the chosen frequency, and a shortcut for this process with a harmonic trap exists~\cite{masuda2009,torrontegui2011,masuda2012}.
In the rotational shortcut, the trapping frequency is held constant and the position of the potential is modified.
This means that $U=0$, $F=\omega_0^2 x_0(t)$, and
\begin{equation}
 H= -\frac{1}{2} \frac{\partial^2}{\partial x}+ \frac 1 2 \omega^2_0 (x-x_0(t))^2.
\end{equation}
\noindent Here, Equation~\eqref{eqn:xc} becomes the only relevant auxiliary equation,
\begin{equation}
 \ddot{x}_c+\omega^2_0 (x_c-x_0)=0,
\end{equation}
and the conditions that must be imposed on $x_c$, are such that
\begin{equation}
 \begin{array}{lcl}
x_c(t_0)=x_0(t_0), && x_c(t_f)=d,\\
\dot x_c(t_0)=0, && \dot x_c(t_f) =\Omega_f, \\
\ddot x_c(t_0)=0, && \ddot x_c(t_f)=0,
\end{array}
\end{equation}
\noindent where $d$ is the final position of the potential minimum and $\Omega_f$ is its final velocity.
For most applications of this shortcut, $d$ is important, and $\Omega_f$ is set to zero; however, this case is the opposite.

Like for the shortcut for raising the potential, the exact form of $x_c$ can be chosen somewhat arbitrarily, and a convenient choice is
\begin{equation}
 x_c(s)= (6 d -3 \Omega_f )s^5 - (15 d-7 \Omega_f )s^4+(10d-4 \Omega_f) s^3 + x_0(t_0),
\end{equation}
where, as above, $s$ is the normalized time.
The value of $\Omega_f$ can then be chosen to be odd multiples of $\pi$ to generate the desired NOON states based on the energy spectrum shown in Figure~\ref{fig:avoid}.

Unlike the shortcut to raise the potential, this shortcut is only approximate and works best when $\omega$ is large so that the particles are highly localized.
Both of these shortcuts rely on the presence of a harmonic potential of the form
\begin{equation}
 V_{H}(x,t)=\frac 1 2 \omega^2(t) \left( x-x_0(t)\right)^2,
\end{equation}
where  $\omega$ is the frequency of the trap (in units of $\hbar/mL^2$) and $x_0$ the position of its minimum.
In the case of the TG ring, the potential must be symmetric around $x_0$, such that it is continuous at $x=\pm 1/2$; therefore, the real form of $(x-x_0)$ must be $(x-x_0+1/2)(\mathrm{mod~} 1)-1/2$.
The potential $V_H$ is then continuous everywhere on the ring, but its derivative is discontinuous at $x=x_0+1/2$ because this position is diametrically opposite to $x_0$.
Though $V_H$ is easy to work with theoretically, it is not necessarily experimentally realistic, and for this reason, we also consider a sinusoidal potential of the form~\cite{phelan2013,masuda2014},
\begin{equation}
 V_{S}(x,t)= \frac{\omega^2(t)}{2 \pi^2} \sin^2 \left(\pi \left( x-x_0(t)\right) \right) ,
\end{equation}
where the notation is the same as before.
Here, prefactors are chosen such that $V_{H}$ is an approximation of $V_S$ around $x_0$.

\begin{figure}
\centering
\subfigure{
\centering
\includegraphics[width=0.48\linewidth]{data/1d/fig1.png}}
\subfigure{
\centering
\includegraphics[width=0.48\linewidth]{data/1d/fig2.png}}

\caption{ Energy eigenspectrum of the system with (a) a harmonic or (b) a sinusoidal potential as a function of $\omega$.
The eigenstates continuously change from angular momentum states of energy $E_k=2\pi^2 k^2$ (with $k=0,\pm1,\ldots$) at $\omega=0$, towards harmonic-oscillator states of energy $E_n=\omega(n+1/2)$ (with $n=0,1,\ldots$) for large $\omega$.
For comparison, the horizontal lines on the right vertical axis give the energy levels in a harmonic potential with $\omega=200$.}
\label{fig:spectrum}
\end{figure}

In Figure~\ref{fig:spectrum}, I show the difference between the two potentials by computing the energy spectra of both Hamiltonians.
Here, the eigenstates at $\omega=0$ are the angular momentum states $e^{i 2 \pi k x}$, with degenerate clockwise and counterclockwise momentum states of opposite quantum number $k$.
As $\omega$ is increased, the degeneracy ceases and the spectrum asymptotically approaches that of a harmonic oscillator.
For the sinusoidal case, the difference with the harmonic spectrum increases with the quantum number $n$.

\begin{figure}
\centering
\includegraphics[width=0.45\linewidth]{data/1d/fig5.png}
\includegraphics[width= 0.45\linewidth]{data/1d/fig6.png}
\caption{
(a) Plot of the parameters $\omega(t)$ and the angular velocity $\Omega(t)$ for the entire protocol.
The parameters are $\omega_0=2$, $\omega_f=100$, $d=100$, each step is executed in $t_f-t_0=10$, and $\Omega_f$ is picked depending on the desired output state (here, $\Omega_f=5 \times 2\pi$).
(b) Final infidelities for $\Omega_f=1,2,\ldots,10 \times 2\pi$ for $V_H$ (dotted blue line) and $V_S$ (solid red line).
The rest of parameters are as shown in (a).
}
\label{fig:final+param}
\end{figure}

Like the case of quantum optimal control, I will first show the single-particle results.
In Figure~\ref{fig:final+param}(a), the values for $\omega(t)$ and $\Omega(t)$ are shown, and
in Figure~\ref{fig:final+param}(b), the infidelities for the state preparation of plane waves $e^{i \Omega_f x}$ with $\Omega_f=1\ldots10 \times 2 \pi$ are shown.
Here, it is clear that even for a large amount of angular momentum, the fidelities remain high for both the harmonic and sinusoidal potentials.

For a multi-particle case in the TG regime, the initial states for the particles will be eigenstates of free space, which are simply plane waves $e^{i 2 \pi k x}$ with integer $k$.
Because the states with $\pm k$ are degenerate, it is equally valid to consider the initial eigenstates
\begin{align}
\phi^i_0(x)&=1,\\
\phi^i_{2l-1}(x)&=\frac{1}{\sqrt 2} \left( e^{i 2 \pi l x}-e^{-i 2 \pi l x} \right)= i \sqrt{2} \sin(2 l \pi x), \\
\phi^i_{2l}(x)&= \frac{1}{\sqrt 2}\left( e^{i 2 \pi l x}+e^{-i 2 \pi l x} \right) = \sqrt{2} \cos(2 l \pi x),
\end{align}
for $l=\{1,2,\ldots\}$.
These states have a total angular momentum of zero and are well-suited for the provided STA protocol because when an odd number of particles occupies the lower eigenstates, the sine and cosine pairs are guaranteed to be populated.

For $\Omega_f=\pi$, the plane wave of quantum numbers $k+1$ and $-k$ are degenerate and one can construct the target states
\begin{align}
\phi^t_{2l}(x)&=\frac{1}{\sqrt 2} \left( e^{i 2 \pi (l+1) x} + e^{-i 2 \pi l x} \right) = \sqrt{2} \cos[(2l+1) \pi x] e^{i \pi x} , \\
\phi^t_{2l+1}(x)&=\frac{1}{\sqrt 2} \left( e^{i 2 \pi (l+1) x}-e^{-i 2 \pi l x} \right) = i \sqrt{2} \sin[(2l+1) \pi x]e^{i \pi x} ,
\end{align}
for $l=\{0,1,2,\ldots\}$.
The states with total angular momentum $\pi$ are similar to NOON states.

\begin{figure}
\centering
\subfigure{
\centering
\includegraphics[width= 0.45\linewidth]{data/1d/fig7a.png} }
\subfigure{
\centering
\includegraphics[width= 0.45\linewidth]{data/1d/fig7b.png}}
\caption{ Final fidelities $F$ of TG states of increasing particle number for the protocol shown in Figure \ref{fig:final+param}(a) with $\Omega_f = \pi$ for $V_H$ (red circle) and $V_S$ (blue cross). Plot (a) shows the fidelity of the protocol with $\omega_f = 100$ and (b) with $\omega_f = 200$.}
\label{fig:TG-STA}
\end{figure}

Any initial state $\ket{\phi^i_l}$ can be brought to the target state $\ket{\phi^t_l}$ with high fidelity with the proposed protocol, and the process also works for TG gases.
In Figure~\ref{fig:TG-STA}(a), the harmonic potential fidelities are shown to remain high for $N \leq 11$ after which they decrease due to a finite maximum height of the potential enforced by periodic boundary conditions.
The fidelities can be improved by increasing the maximum trapping frequency $\omega_f$ as was demonstrated in Figure \ref{fig:TG-STA}(b) where the value of $\omega_f$ is doubled and the fidelities remain high until $N \leq 21$.
When using the sinusoidal potential, the fidelity drops for smaller particle numbers compared to the  harmonic potential (although it also increases with $\omega_f$), due to the lower height $V_S$ has compared to $V_H$.

\section{Outlook}

In this chapter, quantum optimal control and STA protocols were introduced to optimize quantum engineering tasks in cold atomic gases.
I have introduced a physical system to generate NOON states in a TG gas non-adiabatically with both methods, and they were shown to be highly effective.
Such dynamical evolution techniques require time-dependent control parameters, such as rotation frequency or barrier height, and allowing for these dynamic operations has hitherto been a difficult task on GPU hardware.
In the following chapter, I will discuss GPU hardware in-depth and also tackle this issue, along with several others noted in Chapter~\ref{ch:splitop}.


\section{Modal pomsets}
In order to perform a sharper analysis of dependency, we present an alternate
semantics using modal pomsets defined below.  Modal pomsets make a formal
distinction between strong order and weak order.
\begin{definition}
  A \emph{modal (memory model) pomset} is a tuple
  $(\Event, {\sle}, {\gtN},
  \labeling)$, such that
  \begin{itemize}
   \item $(\Event, {\gtN},
  \labeling)$ is a (memory model) pomset, and
\item ${\sle} \subseteq {\gtN}$ is a partial order.
  \end{itemize}
\end{definition}
We write $\bEv\slt\aEv$ when $\bEv\sle\aEv$ and $\bEv\neq\aEv$, and similarly for $\gtN$.   Thus, $(\sle \cup \reco)^{*} \subseteq {\gtN}$.

We list out a few observations to illustrate the relationship between \tvalpom s and pomsets.  We are given a  \tvalpom,
  $(\Event, {\sle}, {\gtN}, \labeling)$.  Then:
\begin{itemize}
\item $(\Event, {\gtN},\labeling)$ is a pomset with the same reads-from relation.

\item Let $\reco$ be the restriction of $\gtN$ to conflicting actions on the same location.  Then, $(\Event, {\sle}, (\sle \cup \reco)^{*}, \labeling)$ is a \tvalpom, and $(\sle \cup \reco)^{*} \subseteq {\gtN}$.
\end{itemize}

\paragraph*{Changes to definitions}
The definition of the semantics of programs using \tvalpom\ largely follows the one using pomsets.  We sketch the changes to definitions below.

\begin{itemize}
\item
  We say that $\bEv$ \emph{fulfills $\aEv$ on $\aLoc$} if $\bEv$ writes
  $\aVal$ to $\aLoc$, $\aEv$ reads $\aVal$ from $\aLoc$,
  \begin{itemize}
  \item $\bEv \slt \aEv$, and
  \item if an event $\cEv$ writes to $\aLoc$ then either $\cEv \gtN \bEv$ or $\aEv \gtN \cEv$.
  \end{itemize}

\item
Augmentation has to include ${\slt}$. i.e
$\aPS'$ is an \emph{augmentation} of $\aPS$ if $\Event'=\Event$,
  ${\labeling'}={\labeling}$, ${\sle'}\supseteq{\sle}$, and
  ${\gtN'}\supseteq{\gtN}$.

\item The definitions of substitution, restriction and the filtering
  operations  stay the same, with $\sle$ carried over unchanged.  For
  example, substitution is defined as follows:

Let $\aPSS\aSub$ be the set $\aPSS'$ where $\aPS'\in\aPSS'$ whenever
there is $\aPS\in\aPSS$ such that:
$\Event' = \Event$,
${\sle'} = {\sle}$,
${\gtN'} = {\gtN}$,
and
$\labeling'(\aEv) = (\bForm\aSub \mid \aAct)$ when $\labeling(\aEv) = (\bForm \mid \aAct)$.

\item In composition,we require ${\sle'}\supseteq{\sle^1}\cup{\sle^2}$

\item The changes to the definition \ref{def:prefix} of prefixing are as follows.  The key changes are that synchronization and dependency enforce $\slt$ whereas coherence only enforces $\gtN$.
\begin{itemize}
\item ${\sle'}\supseteq{\sle}$.

% \item Item 5b changes to: if $\aEv$ is a write then either $\cEv\slt'\aEv$
% or $\labelingForm'(\aEv)$ implies $\labelingForm(\aEv)$.

\item 5b changes to: if $\bEv$ and $\aEv$ are \external actions in conflict, then $\bEv \gtN' \aEv$,
% \item Item \ref{pre-coherence} changes to:
%  if $\aAct$ is a write that conflicts with $\labelingAct(\aEv)$
%     then $\cEv \gtN' \aEv$,

\item Item 5a, 5c, 5d, 5e 5c change to impose $\slt$ order: eg. if $\aAct$ is an acquire or $\labelingAct(\aEv)$ is a release then $\cEv \slt' \aEv$.
\end{itemize}
\end{itemize}
We use $\tsem{\aCmd}$ to stand for the \tvalpom\ semantics of $\aCmd$.

\subsection{Generators. } Modal pomsets provide a characterization of generators from section~\ref{sec:sc}.

Recall that \emph{generators} in the pomset semantics are pomsets that are minimal with respect to augmentation and implication.  These generators are induced by pomsets  that are minimal with respect to augmentation and implication in the \tvalpom\ semantics  in the following sense.

$(\Event, {\gtN},\labeling)$ is a generator for $\sem{\aCmd}$
if there exists  $(\Event, \slt, {\gtN},\labeling) \in \tsem{\aCmd}$ minimal w.r.t.~augmentation and implication, and  $\gtN = (\sle \cup \reco)^{*}$.

Furthermore, any strong order that is outside of program order must be induced by a reads-from.  In the two-thread case, we can state the latter
property as follows: suppose $\aEv$ and $\bEv$ are not related by program
order and $\aEv\slt\bEv$; then there exist $\bEv'$ that reads-from $\aEv'$
such that $\aEv\xpox\aEv'$, $\bEv'\xpox\bEv$ and
$\aEv \slt \aEv' \slt \bEv' \slt \bEv$.

\subsection{Closure properties}
The fine grain analysis of dependency in the modal semantics allows us to establish some closure properties of the semantics of programs.

We consider programs of the
form $\vec{\aLoc}\GETS\vec{0}\SEMI\FENCE\SEMI\aCmd$, where $\aCmd$ is
restriction-free.  Thus, all memory locations are initialized to $0$,
initialization happens-before the execution of any command,

We say that $\aPS' = \aPS\restrict{\Event'}$ when
 $\Event' \subseteq \Event$,
 ${\labeling'} = {\labeling}\restrict{\Event'}$,   and
 ${\le'} = {\le}\restrict{\Event'}$.
% ${\gtN'} = {\gtN}\restrict{\Event'}$.

\begin{definition}
Let $(\aPS \after \aEv) = {\{ \bEv\in\Event \mid \aEv \le \bEv
  \}}$ be the set of events that follow $\aEv$ in $\aPS$.
\end{definition}

The semantics of read is ``input''-enabled, since it permits the read of any visible value.   Thus, any racy read in a program can be replaced by a read of a earlier value (w.r.t.~$\reco$), even while  the races with existing independent writes are maintained.   A canonical example to keep in mind for this lemma is the program:
\begin{align*}
  (y\GETS 0 \SEMI \aReg \GETS y  \SEMI x \GETS 1)
  \PAR
  (x\GETS 0 \SEMI \bReg \GETS x \SEMI y \GETS 1)
\end{align*}
with both registers getting value $1$ via the execution:
\begin{tikzdisplay}[node distance=1em]
\event{wy0}{\DW{y}{0}}{}
\event{ry1}{\DR{y}{1}}{right=of wy0}
\event{wx1}{\DW{x}{1}}{right=of ry1}
\event{wx0}{\DW{x}{0}}{below=of wy0}
\event{rx1}{\DR{x}{1}}{right=of wx0}
\event{wy1}{\DW{y}{1}}{right=of rx1}
\rf{wx1}{rx1}
\rf{wy1}{ry1}
\wk{wx0}{rx1}
\wk{wy0}{ry1}
\end{tikzdisplay}
The lemma constructs the execution:
\begin{tikzdisplay}[node distance=1em]
\event{wy0}{\DW{y}{0}}{}
\event{ry1}{\DR{y}{0}}{right=of wy0}
\event{wx1}{\DW{x}{1}}{right=of ry1}
\event{wx0}{\DW{x}{0}}{below=of wy0}
\event{rx1}{\DR{x}{0}}{right=of wx0}
\event{wy1}{\DW{y}{1}}{right=of rx1}
\rf{wx0}{rx1}
\rf{wy0}{ry1}
\wk{rx1}{wx1}
\wk{ry1}{wy1}
\end{tikzdisplay}

\begin{lemma}\label{inputen}
%Let $\aCmd = \vec{\aLoc}\GETS\vec{0}\SEMI \FENCE\SEMI (\aCmd^1 \PAR \cdots \PAR \aCmd^n)$.
Let $\aPS \in \tsem{\aCmd}$ be a top level pomset.
Let $\aEv \in \aPS$ read from write event $\bEv$  on $\aLoc$,  $\neg(\bEv \xhb \aEv)$.
Then, there exists $\bPS \in \tsem{\aCmd}$ such that:
\begin{itemize}
%\item $(\exists \aEv' \in \Event_{\bPS})$ such that $
%\Event_{\bPS}$ is the disjoint union of  $\Event_{\aPS} \setminus
%(\aPS \after \aEv))$ and $(\bPS \after \aEv')$.
\item $\aEv'$ reads from $\aLoc$, with matching write event $\bEv'$, such that $\bEv' \xeco \bEv$ in $\bPS$
\item The restriction of $\sle$  in $\aPS$ to $\Event_{\aPS} \setminus  (\aPS \after \aEv)$ agrees with the restriction of $\sle$ in $\bPS$ to $\Event_{\bPS} \setminus  (\aPS \after \aEv)$  in  $\bPS$.
\item The restriction of $\le$  in $\aPS$ to $\Event_{\aPS} \setminus  (\aPS \after \aEv)$ agrees with the restriction of $\le$ in $\bPS$ to $\Event_{\bPS} \setminus  (\aPS \after \aEv)$  in  $\bPS$.
\end{itemize}
\end{lemma}
\begin{proof}
The form of $\aCmd$ ensures that there is always a write to $\aLoc$ that is related by $\xhb$ to any read.  Thus, there is at least one other write than can satisfy the read recorded as  $\aEv$.

The key observation behind the proof is that change in a  prefixing read action can only affect the events that are dependent, ie. in the $\slt$ order to the read action.
\end{proof}

In the following lemma,  invert the $\reco$ relationship between a read and a write.   A canonical example to keep in mind for this lemma is the program:
\begin{align*}
(y\GETS 0 \SEMI   x \GETS 1  \SEMI \aReg \GETS y)
\PAR (x\GETS 0 \SEMI  y \GETS 1  \SEMI  \bReg \GETS x)
\end{align*}
with both registers getting value $0$ via the execution:
\begin{tikzdisplay}[node distance=1em]
\event{wy0}{\DW{y}{0}}{}
\event{wx1}{\DW{x}{1}}{right=of wy0}
\event{ry0}{\DR{y}{0}}{right=of wx1}
\event{wx0}{\DW{x}{0}}{below=of wy0}
\event{wy1}{\DW{y}{1}}{right=of wx0}
\event{rx0}{\DR{x}{0}}{right=of wy1}
\rf[bend right]{wx0}{rx0}
\rf[bend left]{wy0}{ry0}
\wk{rx0}{wx1}
\wk{ry0}{wy1}
\wk{wx0}{wx1}
\wk{wy0}{wy1}
\end{tikzdisplay}
The lemma constructs the execution:
\begin{tikzdisplay}[node distance=1em]
\event{wy0}{\DW{y}{0}}{}
\event{wx1}{\DW{x}{1}}{right=of wy0}
\event{ry0}{\DR{y}{1}}{right=of wx1}
\event{wx0}{\DW{x}{0}}{below=of wy0}
\event{wy1}{\DW{y}{1}}{right=of wx0}
\event{rx0}{\DR{x}{1}}{right=of wy1}
\rf{wx1}{rx0}
\rf{wy1}{ry0}
\wk{wx0}{wx1}
\wk{wy0}{wy1}
\end{tikzdisplay}

\begin{lemma}\label{removerw}
Let $\aPS \in \tsem{\aCmd}$ be a top-level pomset.
Let $\bEv \in \aPS$ be a write on $\aLoc$.
Let $\aEv \in \aPS$ read from $\aLoc$ such that $\aEv \xeco \bEv$ and $\neg(\aEv \slt \bEv)$.  Then, there exists $\bPS \in \tsem{\aCmd}$ such that:
\begin{itemize}
\item $\aEv' \in \bPS \setminus \aPS$ reads from $\aLoc$, with matching write $\bEv$.
\item The restriction of $\sle$ in $\aPS$ to $\Event_{\aPS} \setminus (\aPS\ \after\ \aEv)$ agrees with the restriction of $\sle$ in $\bPS$ to $\Event_{\bPS} \setminus  (\aPS\ \after\ \aEv)$.
\end{itemize}
\end{lemma}
\begin{proof}
The proof proceeds similar to the above proof; in this case, replace the value read in $\aEv$ to come from $\bEv$.
\end{proof}
Any new  event $\bEv'$ in $\bPS \after \aEv'$ reading from $\aLoc$ cannot have a matching write event $\bEv'' \xeco \bEv$ since that  implies $\bEv' \xeco \bEv$ and a $\reco$ cycle $\bEv \slt \aEv \slt \aEv' \xeco \bEv$.  Thus, the above lemma can be iterated if the new pomset is has any further reads that precede $\bEv$ in $\reco$, so we can finally derive a pomset with no reads and writes satisfying the hypothesis of the lemma.

The $\reco$ order between writes that are not related by $\lt$ can be reversed.
A canonical example to keep in mind for this lemma is the program:
\begin{align*}
(x\GETS 1)
\PAR (x\GETS 0)
\end{align*}
\begin{tikzdisplay}[node distance=1em]
\event{wy0}{\DW{x}{1}}{}
\event{wx0}{\DW{x}{0}}{right=of wy0}
\wk{wy0}{wx0}
\end{tikzdisplay}
The lemma constructs the execution:
\begin{tikzdisplay}[node distance=1em]
\event{wy0}{\DW{x}{1}}{}
\event{wx0}{\DW{x}{0}}{right=of wy0}
\wk{wx0}{wy0}
\end{tikzdisplay}
\begin{lemma}\label{cohww}
Let $\aPS \in \tsem{\aCmd}$ be a top level pomset.  Let $\bEv, \aEv$ be a writes to $\aLoc$ such that:
\begin{itemize}
\item $\bEv\gtN \aEv$
\item for all writes $\cEv$ to $\aLoc$ such that  $ \bEv \gtN \cEv \gtN  \aEv$,  it is the case that  $ \neg(\cEv \slt \aEv)$ and $\neg(\cEv \xpox \aEv)$
\end{itemize}

Then, there exists $\bPS \in \tsem{\aCmd}$ such that $\Event_{\aPS} = \Event_{\bPS}$, $\sle_{\aPS} = \sle_{\bPS}$, and
$\aEv \gtN \bEv$ in $\bPS$.
\end{lemma}
\begin{proof}
We show how to interchange $\aEv, \bEv$ adjacent in $\gtN$, ie. we assume that  $\neg(\exists \cEv) \  \bEv \gtN \cEv \gtN \aEv$.  The full proof follows by induction.

Since  $\sem{\aCmd}$ is augmentation closed, it suffices to show that we can build $\bPS$ while satisfying the constraints between $\slt,\gtN$.  We list the changes below.
\begin{itemize}
\item $\aEv \gtN \bEv$ in $\bPS$
\item For all reads $\cEv$ matched to $\aEv$, change from $\bEv \gtN \cEv$ in $\aPS$ to $\cEv \gtN \bEv$ in $\bPS$
\item For all reads $\cEv$ matched to $\bEv$, change from $\cEv \gtN \aEv$ in $\aPS$ to $\aEv \gtN \cEv$ in $\bPS$
\popQED
\end{itemize}
\end{proof}

\section{Proof of DRF}\label{drfproof}

In this section of the appendix, we develop a proof of DRF for \tvalpom s.  By the results in the earlier section, it yields DRF for the pomset semantics, since the races are identical in both models.

In the rest of this section, we assume that $\aPS$ is a generator for
$\tsem{\aCmd}$.

We prove:
\begin{description}
\item[DRF1: ] If $\aPS$ does not have a race, $\aPS \in \tsemsc{\aCmd}$.
\item[DRF2: ] If $\aPS$ has a race, then there exists $\bPS\in \tsemClosed{\aCmd}$ such that $\bPS \in \tsemsc{\aCmd}$ and has a race.
\end{description}

\paragraph*{Proof of DRF1}
We first show that if $\aPS \in \tsem{\aCmd} \setminus \tsemsc{\aCmd}$, then $\aPS$ has a race.  By assumption, there is a cycle in  $\rpox \cup \slt \cup \xeco$.  Let this cycle be $\aEv_0, \aEv'_0, \aEv_1, \aEv'_1, \ldots, \aEv_n, \aEv'_n, \aEv_0$ where for all $i$, $\aEv_i \xpox \aEv'_i$ and $\aEv'_i  \not\xpox \aEv'_{i+1}$.
If for all $i$, $\aEv'_i  \xhb \aEv'_{i+1}$, then the above is a cycle in $\rhb$, which is a contradiction.
So, there is at least one $i$ such that $\aEv'_i  \not\xhb \aEv'_{i+1}$.  There are two cases to consider.
\begin{itemize}
\item $\aEv'_i  \xeco \aEv'_{i+1}$.   In this case, there is a race.
\item  $\aEv'_i  \slt \aEv'_{i+1}$.  In this case, $\aEv'_i$ is a write and $\aEv'_{i+1}$ is a conflicting read, so there is a race.
\end{itemize}

\paragraph*{Proof of DRF2}

We define a size $|\aPS|$ as follows: $\size(\aPS)$ is the number of events in $\aPS$.    Since we are considering loop free programs, there is an $\aPS \in \tsemsc{\aCmd}$ with maximum size, which we identify as $\size(\aCmd)$.

We prove by induction on $\size(\aCmd) - \size(\bPS)$ that given $(\aPS, \bPS)$ such that:
\begin{itemize}
\item $\bPS$ is a prefix of some $\aPS' \in \tsemsc{\aCmd}$
\item $\bPS$ is a prefix of $\aPS$ under all of $\xpox,\gtN,\lt$
\item $\aPS$ has a race
\end{itemize}
there exists $\bPS\in \tsem{\aCmd}$ that demonstrates the race.

The required theorem follows by setting $\bPS$ to be the empty pomset.

For the base case, $\bPS = |\aPS|$.  In this case, $\aPS$ is the required witness.

Otherwise, consider a maximal sequential prefix, extending $\bPS$, w.r.t.~all of  $\rpox,\reco,\slt$.  If it strictly contains $\bPS$, result follows from induction hypothesis.

If not, $\bPS$ is already maximal.  Consider the set of all events in $\aPS \setminus \bPS$ that are minimal w.r.t.~$\rhb$.  In particular, these events will also be minimal w.r.t.~$\rpox$.

If one of these events, say $\aEv$  is a write, we proceed as follows.   Using $\rhb$-minimality of $\aEv$, we deduce $\rpox$ minimality of $\aEv$.  Using the generator properties, we deduce that $\aEv$ is $\slt$-minimal .  Using lemma~\ref{removerw}, we build $\aPS_1$ from $\aPS$ without changing $\bPS$ to ensure that there are is no read $\bEv \in \aPS_1 \setminus \bPS$ such that $\bEv \xeco \aEv$.  Using lemma~\ref{cohww}, we build $\aPS_2$ from $\aPS_1$ without changing $\bPS$ to ensure that there are is no write $\bEv \in \aPS_2 \setminus \bPS$ such that $\bEv \xeco \aEv$.  Thus, $\aEv$ is $\reco$-minimal in $\aPS_2 \setminus \bPS$.  Result follows from induction hypothesis by considering $(\aPS_2,\bPS_1)$ where $\bPS_1$ is got from $\bPS$ by adding $\aEv$.

So, we can assume that  all events in $\aPS \setminus \bPS$, say $\aEv_0, \ldots, \aEv_n$  that are minimal w.r.t.~$\rhb$ are reads, and we have  events
$\aEv'_0, \aEv'_1, \ldots, \aEv'_n, \aEv_0$ such that:
\[
\begin{array}{lrl}
\aEv_i \xpox\ \aEv'_i \\
\aEv'_i \  (\reco\ \cup \slt)  \ \aEv_{(i+1)\mod n}
\end{array}
\]
Let $\bEv$ be the matching write for $\aEv_{(i+1)\mod n}$. If $\bEv_i \in \bPS$bEv , then by $\reco$ prefix closure of $\bPS$, $\bEv \xeco\ \aEv'_i$ and $\aEv_{(i+1)\mod n} \reco\ \aEv'_i$, which is a contradiction to $\reco$ being a partial order per location.  So, we can assume that $\aEv'_i \  \slt  \ \aEv_{(i+1)\mod n}$.

We proceed as follows.  We use lemma~\ref{inputen} on the  pomset $\aPS$ and read $\aEv_{(i+1)\mod n}$ and write $\aEv'_i$ to construct $\cPS$ that changes the value read in $\aEv_{j}$ to a value from $\bPS$.  $\dPS$  is derived adding the modified read yielded by lemma~\ref{inputen} to $\bPS$.  Result follows by induction hypothesis since $\dPS$ is a prefix of $\cPS$ under all of $\xpox,\lt, \reco$,  $\cPS$ has a race, and $\size(\dPS) = \size(\bPS) + 1$.

\endinput

\begin{comment}
Operation        Implementation
Relaxed read     ldr
Relaxed write    str
Acquiring read   ldar
Releasing write  stlr
Fence            dmb.sy
\end{comment}

\begin{comment}
ob does not contradict eco

ob does not contradict (co cap po):

Suppose that wx1 po wx2 then it cannot be that wx2 ob wx1.
We know that wx1 co wx2 by SC-PER-LOC

% Case 1. w1 is read externally, then we have
%   wx1 rfe r
% and
%   r fre w2
% so
%   wx1 obs+ wx2
% which contradicts EXTERNAL

% Case 2. wx1 is not read externally.
We show this by contradiction
Assume
  wx1 co wx2
and
  wx2 ob wx1

Note that
  po supseteq dob cup aob cup bob
So in order to get order into wx1, we must have
  wx2 (ob?; obs; ob?; obs; ob?) wx1

Note that we cannot have dob or bob into wx1 after obs, since then we would
also have it into wx2, creating a cycle in EXTERNAL.  This holds because both
dob and bob are closed on the right w.r.t. coi

So it must be that
  wx2 (ob?; obs; ob?; wx0; coe) wx1,
in which case we also have wx0 coe wx2, contradicting EXTERNAL
or
  wx2 (ob?; obs; ob?; rx0; fre) wx1
in which case we also have rx0 fre wx2, contradicting EXTERNAL

Internal reads do not need to respect ob:
Arm allows the following:

  Ra1 -ctrl-> Wx1 -rfi-> Rx1 ---> Wb1    if(a){x=1}; b=x
   |                               |
  Wa1 <-------------------------- Rb1    a=b

Suppose that wx1 po rx2 and rx2 is read externally.
Then it cannot be that rx2 ob wx1.

Case 1: if wx1 co wx2, then we have wx1 coe wx2 rfe rx2, contradicting EXTERNAL
Case 2: if wx2 co wx1, then we have rx2 fr wx1, contradicting SC-PER-LOC

Suppose that rx1 po wx2 and rx1 is read externally.
Then it cannot be that wx2 ob rx1.

Case 1: if wx2 co wx1, then wx2 co wx1 rf rx1 po wx2, contradicting SC-PER-LOC
Case 2: if wx1 co wx2, for a contradiction, suppose wx2 ob rx1.
then we need another thread involved to get order from wx2 to rx1.
To get order into the read, there are several options:
- use cross thread read, then dob; but dob does not include reads in it's domain.
  An attempt to do this is something like:

              Wx1                 x=1
               |
  Ra2 -ctrl-> Rx1 - - -> Wx2      if(a){r=x}; x=2
   |                      |
  Wa2 <----------------- Rx2      a=x

  But the ctrl dependency is not included in ob between reads.
- use cross thread read then barrier, but then you contradict EXTERNAL
- create and ob edge from Rx2 to Wx1.
  An attempt to do this is,

  Wx1 <-------------- Ra1
   |                   |        But cannot get Wx2 --> Wa1 without a barrier
  Rx1 - - -> Wx2 ---> Wa1

  Wx1 <----- Rx2
   |          |                 contradicts SC-PER-LOC
  Rx1 - - -> Wx2

Other examples to type in:
Allowed:
Rx1 -> Wy0  Wy1
Ry1 -> Wz0  Wz1
Rz1 -> Wx0  Wx1

Forbidden:
Rx1 -> Wy0 Wy1
Ry1 -> Wx0 Wx1

\end{comment}

\begin{comment}
\citet{DBLP:journals/pacmpl/PodkopaevLV19} define the \emph{Intermediate
  Memory Model (IMM)} and provide efficient implementations of the IMM into
several processor architectures, including TSO, ARMv8 and Power.

In this section, we show that any execution allowed by a sublanguage of the
IMM is also allowed by our semantics.  The sublanguage we consider bans
loops, read-modify-write (RMW) operations, and fences.  In addition, we take
the set of memory locations, $\Loc$, to be finite.  Syntactically, we drop
the superscript \textsf{rlx} on relaxed reads and writes; in addition, we use
structured conditionals rather than the more general \textsf{goto}.  We refer
to this sublanguage as $\muIMM$.

$\muIMM$ programs sit in the restriction-free fragment of our language, where
all memory locations are initialized to $0$ and parallel-composition occurs
only at top level.  In other words, $\muIMM$ programs have the form
\begin{displaymath}
  {\aLoc_1}\GETS{0}\SEMI
  \cdots\SEMI
  {\aLoc_m}\GETS{0}\SEMI
  (\aCmd^1 \PAR \cdots \PAR \aCmd^n)
\end{displaymath}
where $\aCmd^1$, \ldots, $\aCmd^n$ do not include either composition or
restriction.

Due to space limitations, we do not include a full description of the IMM.
The broad strokes of the argument given here should be clear, but interested
readers will need to refer to \citep{DBLP:journals/pacmpl/PodkopaevLV19} for
details.
\end{comment}

\endinput

\section{Proof of DRF}

For any $\aPS$, then $\closed(\aPS)$ is set enriched with useless reads
(preserving augmentation closure) and where we remove any event whose
precondition is not a tautology.

For top level programs:
\begin{displaymath}
  \semClosed{\VAR\vec{\aLoc}\SEMI
    \vec{\aLoc}\GETS\vec{0}\SEMI
    \vec{\bLoc}\GETS\vec{0}\SEMI
    \FENCE\SEMI
    (\aCmd^1 \PAR \cdots \PAR \aCmd^n)}
  =
  \VAR\vec{\aLoc}\SEMI
    \vec{\aLoc}\GETS\vec{0}\SEMI
    \vec{\bLoc}\GETS\vec{0}\SEMI
    \FENCE\SEMI
    (\semClosed{\aCmd^1} \PAR \cdots \PAR \semClosed{\aCmd^n})
\end{displaymath}

\begin{definition}
A thread: top level component of a parallel composition
\end{definition}

\begin{definition}
$\aPS$ is a generator of  $\semClosed{\aCmd}$ if for all $\bPS \in \semClosed{\aCmd}$ such that $\aPS$ augments $\bPS$, $\aPS = \bPS$.
\end{definition}

Since the program we consider are loop free, for any command $\aCmd$, the size of the pomsets in $\aCmd$ are bounded by a constant, that we denote by $\size(\aCmd)$.

\section{Generators for semantics of programs with parallel composition}
All generators $\aPS$  satisfy the following factorization of cross-thread $\lt$.

\begin{lemma}\label{pargen}
Consider the subset of pomsets of $\semClosed{\aCmd \PAR \bCmd}$ that are  $\aLoc$-closed for all $\aLoc$.

Let $\aPS$  be any generator.
%\begin{itemize}
% \item
 Let $\aEv\lt\bEv$ and $\aEv \in \semClosed{\aCmd}$ and  $\bEv \in \semClosed{\bCmd} $.

Then there is a write  $\aEv' \in \semClosed{\aCmd}$, and  a read $\bEv' \in \semClosed{\bCmd}$ such that  $\bEv'$ reads-from $\aEv'$ and $\aEv \lt \aEv' \lt \bEv' \lt \bEv$.

%\item $\aEv \gtN \bEv$ only if $ \aEv  [\lt \cup (\le; \reco;\le)^{\star}]
%\bEv$.

% \item If $\aEv\lt\bEv$ and $\aEv, \bEv \in \semClosed{\aCmd}$,
%then there exists

%There exists a release action $\aEv'$ in $\sem{\aCmd}$, a
%matching acquire action $\bEv'$ in $\sem{\bCmd}$ such that $
%\aEv \lt \aEv'$, $\bEv' \lt \bEv$ and $\aEv' \lt \bEv'$.

\end{lemma}

The proof of lemma~\ref{cohsat} yields the following two corollaries.
\begin{corollary}\label{cohrw}
Let $\aPS \in \sem{\aCmd}$ be a generator. Let
\begin{itemize}
\item $\bEv'$ be a read from $\aLoc$ with matching write $\bEv$.  \item $\aEv$ be a write to $\aLoc$ such that  $\bEv' \gtN \aEv$.   \item Forall writes $\cEv$ to $\aLoc$ such that  $ \bEv \gtN \cEv \gtN  \aEv$,  it is the case that  $ \neg(\bEv' \lt \cEv)$ and $\neg(\bEv \xpox \cEv) ]$
\end{itemize}

Then, there exists $\bPS \in \sem{\aCmd}$, also a generator, such that $\Event_{\aPS} = \Event_{\bPS}$, $\le_{\aPS} = \le_{\bPS}$, and $\aEv \gtN \bEv'$ in $\bPS$.
\end{corollary}
\begin{corollary}\label{cohwr}
Let $\aPS \in \sem{\aCmd}$ be a generator. Let
\begin{itemize}
\item $\aEv'$  read from $\aLoc$ with matching write $\aEv$.
\item $\bEv$ be a  write to $\aLoc$ such that  $\bEv \gtN \aEv'$.  \item Forall writes $\cEv$ to $\aLoc$ such that  $ \bEv \gtN \cEv \gtN  \aEv$ and $\cEv \not= \aEv$,  it is the case that  $ \neg(\cEv \lt \aEv')$ and $\neg(\cEv \xpox \aEv) ]$.
\end{itemize}

Then, there exists $\bPS \in \sem{\aCmd}$, also a generator, such that:
$\Event_{\aPS} = \Event_{\bPS}$, $\le_{\aPS} = \le_{\bPS}$, and
$\aEv' \gtN \bEv$ in $\bPS$.

\end{corollary}

===============good lemma. Not used. ==================

\begin{definition}
$ \aEv \xeco  \bEv$ if both $\aEv$ and $\bEv$ touch the same location, at least one is a write, and $\aEv \xird \bEv$  or $\aEv \xrb \bEv$ or $\aEv\xird \bEv$ or $\bEv \gtN \aEv$.
\end{definition}

By lemma~\ref{extendob}, if $\aEv \not=\aEv'$, we deduce $\aEv \xob \bEv'$, and thus $\aEv \xob \bEv$.  If $\bEv \not=\bEv'$, we deduce $\aEv' \xob \bEv$ and thus $\aEv \xob \bEv$.

Thus, if $\aEv \not=\aEv'$ or $\bEv \not=\bEv'$, then there is a cycle $\aEv \xob \bEv \xob \cEv \xob \cEv' \xob \aEv$.

So we can assume that  $\aEv' = \aEv$, $\bEv' = \bEv$ and
\[ \aEv  \xeco \bEv \xob \cEv \xob \cEv' \xeco \aEv \]
where all of $\aEv, \bEv, \cEv, \cEv'$ access the same location and at least one of $\aEv,\bEv$ is a write, at least one of $\aEv,\cEv'$ is a write, and at least one of $\bEv,\cEv$ is a write.

We reason by cases.
\begin{itemize}
\item If $\cEv'$ is a write or both $(\aEv, \bEv)$ are writes.

We deduce that $\bEv \xeco \cEv' \xeco \aEv$ and thus $\bEv \xeco \aEv$.

\item $\cEv'$ is a read.  $\aEv$ is a write.  $\bEv$ is a read.

In this case $\cEv$ is a write.  From $\cEv \xob \aEv$, we deduce $\cEv \xeco \aEv$. Combining with $\bEv \xeco \cEv$, we deduce that $\bEv \xeco \aEv$.

\end{itemize}
In either case, there is a contradiction $\aEv \xeco \bEv \xeco \aEv$.

 Consider the write $\cEv'$ fulfilling $\aEv$.

 $\cEv' (\xobi \cap \xeco) \aEv$.  Since $\aEv$ not $\rrfi$ event.
Also,

So, we can assume that $\aEv \xpox \bEv$, and the situation is:
\[ \cEv' \xobi \cEv (\xpox \cap \xobi) \aEv (\xeco \cap \xpox)  \bEv  (\xpox \cap \xobi) \cEv'' \xobi \cEv' \]

By lemma~\ref{extendob}, if $\cEv \not= \aEv$, $\cEv \xob \bEv$, and we have a cycle in $\xob$.
Similarly, if $\bEv \not= \cEv''$, $\aEv \xob \cEv''$, and we have a cycle in $\xob$.  So, the situation is:
\[ \cEv' \xobi \aEv (\xeco \cap \xpox)  \bEv \xobi \cEv' \]
$\cEv',\aEv,\bEv$ are events on same variable.  The above is a cycle in $\xeco$.


\documentclass{report}
\usepackage{graphicx}

\usepackage{amsmath}
\DeclareMathOperator*{\argmax}{argmax}

\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,
    urlcolor=cyan,
}

\begin{document}

\title{Banana World Project Report}
\author{Denis Sergeev}

\section*{Problem definition}

The goal of the project was to create a RL Agent which is able to train from scratch by interacting with the Banana World environment for a series of episodes. During the episode the agent receives the environment state. The agent is able to perform some actions which may change the environment state.

\subsection*{Environment description}
The \textbf{state space} has \(37\) dimensions each of which is a continuous variable. It includes the agent's velocity, along with ray-based perception of objects around the agent's forward direction. The \textbf{action space} contains the following \(4\) legal actions:
\begin{itemize}
  \item move forward \(0\)
  \item move backward \(1\)
  \item turn left \(2\)
  \item turn right \(3\)
\end{itemize}
A reward of \(+1\) is provided for collecting a yellow banana, and a reward of \(-1\)` is provided for collecting a blue banana. Thus, the goal of the agent is to collect as many yellow bananas as possible while avoiding blue bananas. The task is episodic, and in order to solve the environment, the agent must get an average score of \(+13\) over \(100\) consecutive episodes.

\section*{Solution}
\subsection*{Deep Q-Network Architecture}
\subsubsection*{Network}
Since the agent should act upon environment state provided as vector of size \(37\), I decided to build simple network with \(3\) fully connected layers, which should predict the Q-value for each action given the input state. The input has size 37, output 4. I tried 64x64, 64x32, 32x32, 32x16 hidden layers configuration. The best one was 32x32, since it was able to reach the training goal faster and it has less internal parameters then 64x64 and 64x32.

\subsection*{Training}
\subsubsection*{Loss function and optimizing method}

I tried 2 loss functions \ref{loss}: Simple DQN loss \ref{dqn-loss} and \href{https://arxiv.org/pdf/1509.06461.pdf}{Double DQN loss} \ref{ddqn-loss}
\begin{equation} \label{loss}
L \equiv (Y_{t} - Q(S_{t}, a; \theta_{t}))^{2}
\end{equation}
\begin{equation} \label{dqn-loss}
Y_t \equiv R_{t+1} + \gamma \max_a Q(S_t, a; \theta'_t)
\end{equation}
\begin{equation} \label{ddqn-loss}
Y_t \equiv R_{t+1} + \gamma Q(S_{t+1}, \argmax_a Q(S_t, a; \theta_t); \theta'_t)
\end{equation}
Where \(Q(S_t, a, \theta_t)\) is output (value of action \(a\)) of the network with internal parameters \(\theta_t\) given input \(S_t\). Both loss functions used 2 sets of network parameters: target \(\theta'_t\) and local \(\theta_t\). Target network parameters was updating through soft update \(\theta'_t = (1 - \tau) \theta'_t + \tau \theta_t\).
As optimizing method for \(\theta_t\) I used Adam with learning rate \(\alpha\).

\subsubsection*{Experience replay}

The input for training did not come directly from episodes. Instead I used experience replay technique (see \href{https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf}{Human-level control through deep reinforcement learning}).

\section*{Results}

I came up using following training hyper parameters:
\begin{itemize}
	\item \(\epsilon_{start}\): 1.0, \(\epsilon_{end}\): 0.01, \(\epsilon_{decay}\): 0.97
	\item Buffer size: 100000
	\item Batch size: 64
	\item \(\gamma\): 0.99
	\item \(\tau\): 0.005
	\item \(\alpha\): 0.0005
\end{itemize}
They were used across all training experiments below.

\subsection*{Training DDQN 32x16}

DQN architecture: fully connected with bias 37x32x16x4. Loss \ref{ddqn-loss}.

Training history:

Episode 100.	Average Score: 4.88.	Time elapsed: 4:35

Episode 200.	Average Score: 12.21.	Time elapsed: 9:11

Environment solved in 170 episodes!	Average Score: 13.12. Time elapsed: 12:31s. See \ref{fig:DDQN_32x16}.

\begin{figure}
	\includegraphics[width=0.9\linewidth]{res/DDQN_32x16/score.png}
	\caption{DDQN 32x16: Rewards per episode}
	\label{fig:DDQN_32x16}
\end{figure}

\subsection*{Training DDQN 32x32}

DQN architecture: fully connected with bias 37x32x32x4. Loss \ref{ddqn-loss}.

Training history:

Episode 100.	Average Score: 6.50.	Time elapsed: 3:26

Episode 200.	Average Score: 12.66.	Time elapsed: 6:51

Environment solved in 109 episodes!	Average Score: 13.08. Time elapsed: 7:09s. See \ref{fig:DDQN_32x32}.

\begin{figure}
\includegraphics[width=0.9\linewidth]{res/DDQN_32x32/score.png}
\caption{DDQN 32x32: Rewards per episode}
\label{fig:DDQN_32x32}
\end{figure}

\subsection*{Training DDQN 64x32}

DQN architecture: fully connected with bias 37x64x32x4. Loss \ref{ddqn-loss}.

Training history:

Episode 100.	Average Score: 5.69.	Time elapsed: 3:25

Episode 200.	Average Score: 12.38.	Time elapsed: 6:52

Environment solved in 118 episodes!	Average Score: 13.06. Time elapsed: 7:33s. See \ref{fig:DDQN_64x32}.

\begin{figure}
\includegraphics[width=0.9\linewidth]{res/DDQN_64x32/score.png}
\caption{DDQN 64x32: Rewards per episode}
\label{fig:DDQN_64x32}
\end{figure}

\subsection*{Training DQN 64x32}

DQN architecture: fully connected with bias 37x64x32x4. Loss \ref{dqn-loss}.

Training history:

Episode 100.	Average Score: 4.75.	Time elapsed: 3:10

Episode 200.	Average Score: 11.97.	Time elapsed: 6:22

Environment solved in 136 episodes!	Average Score: 13.02. Time elapsed: 7:31s. See \ref{fig:DQN_64x32}.

\begin{figure}
\includegraphics[width=0.9\linewidth]{res/DQN_64x32/score.png}
\caption{DQN 64x32: Rewards per episode}
\label{fig:DQN_64x32}
\end{figure}

\subsection*{Training DQN 64x64}

DQN architecture: fully connected with bias 37x64x64x4. Loss \ref{dqn-loss}.

Training history:

Episode 100.	Average Score: 4.75.	Time elapsed: 3:21

Episode 200.	Average Score: 10.90.	Time elapsed: 6:39

Environment solved in 130 episodes!	Average Score: 13.03. Time elapsed: 7:38s. See \ref{fig:DQN_64x64}.

\begin{figure}
\includegraphics[width=0.9\linewidth]{res/DQN_64x64/score.png}
\caption{DQN 64x64: Rewards per episode}
\label{fig:DQN_64x64}
\end{figure}

\subsection*{Conclusion}

From the above results we can summarize, that DDQN has slightly better training performance. Also 32x32 hidden layer configuration is the best. It was trained faster and has few internal parameters.

\section*{Areas of improvement}

During evaluation of the trained agent I noticed, that in some cases the agent hangs up. It begins alternating between 2 positions. You may see it at the end of the video I uploaded on \href{https://youtu.be/PPSJ2k9RBq0}{youtube}. This is caused by the fact, that the agent makes its decision based only on instant state. It does not take into account previous states and actions. Intuitively is clear that the agent needs some memory to store previous actions and states. This could be achieved by \href{http://cs229.stanford.edu/proj2016/report/ChenYingLaird-DeepQLearningWithRecurrentNeuralNetwords-report.pdf}{adding recurrent unit to the network}, like LSTM, GRU or just simple RNN.
Another improvement could be implementing \href{https://arxiv.org/abs/1511.05952}{prioritized experience replay}.

\end{document}


\section{Modifying and applying RISE (single pixel)}
\nblink{brats/06\_rise.ipynb}

The approach how to apply RISE on a segmentation task is the same as with Grad-CAM. Pixels in the output of a segmentation model are equivalent to classes of a classification model.
As a first version, we only analyze a single pixel. We chsose the first pixel in the tumor ground truth segment by iterating over the segment linearly until the first active pixel is found.

In the standard implementation, RISE analyzes every class that is returned by the network. Our segmentation model returns $ image\_height \cdot image\_width $ count classes. This produces too many classes and the algorithm crashes with an out of memory error. We modified the code to only analyze the output neuron for our selected pixel.

\subsection{Results}

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{chapters/04_segmentation/images/rise_single_pixel.png}
\caption{RISE saliency map analyzing the topmost pixel in the scan, overlaid on the binarized network output and the T1 contrast enhanced input modality.}
\label{rise_single_pixel_result}
\end{figure}

\subsection{Discussion}
The produced output in Figure \ref{rise_single_pixel_result} looks correct, the color blob is at the expected position. It clearly shows that the neural network looks at the correct location to generate the segmentation.
Apart from this basic correctness verification, no further insight is provided by the saliency map, because the resolution generated by RISE is too low.

\subsection{Conclusion}
The generated output is low resolution but still helpful. We therefore decided to build a version of RISE which works on all pixels of the segmentation.

\section{Modifying and applying RISE (multi pixel)}
\nblink{brats/17\_rise\_multipixel.ipynb}

As written in the previous section, RISE crashes when running on all the pixels in the output of the model. We already modified the algorithm to work on a single pixel. For this multi pixel variant, we extend the algorithm to work on all pixels in the segmentation ground truth. We batched the pixels into groups of 1000 pixels each, because depending on the size of the output segment the algorithm still crashed.

With this changed algorithm, we generate a saliency map like in Figure \ref{rise_single_pixel_result} for every single pixel in the output segment.

To have a single saliency map for all pixels, we use the max and mean functions of numpy on the whole set of saliency maps.

\subsection{Results}

\begin{figure}[H]
    \centering
    \begin{subfigure}[t]{.4\textwidth}
        \centering
        \includegraphics[width=\linewidth]{chapters/04_segmentation/images/rise_multipixel_max_1-0.png}
        \caption{Saliency map created by the max function}
    \end{subfigure}\hspace{1cm}%
    \begin{subfigure}[t]{.4\textwidth}
        \centering
        \includegraphics[width=\linewidth]{chapters/04_segmentation/images/rise_multipixel_max_1-1.png}
        \caption{Saliency map created by the max function, overlaid on the binarized network output}
    \end{subfigure}
    \caption{Single saliency map over all output pixels generated by applying the max function on the set of all saliency maps.}
    \label{rise_multipixel_max}
\end{figure}

\begin{figure}[H]
    \centering
    \begin{subfigure}[t]{.4\textwidth}
        \centering
        \includegraphics[width=\linewidth]{chapters/04_segmentation/images/rise_multipixel_mean_2-0.png}
        \caption{Saliency map created by the mean function}
    \end{subfigure}\hspace{1cm}%
    \begin{subfigure}[t]{.4\textwidth}
        \centering
        \includegraphics[width=\linewidth]{chapters/04_segmentation/images/rise_multipixel_mean_2-1.png}
        \caption{Saliency map created by the mean function, overlaid on the binarized network output}
    \end{subfigure}
    \caption{Single saliency map over all output pixels generated by applying the mean function on the set of all saliency maps.}
    \label{rise_multipixel_mean}
\end{figure}

\subsection{Discussion}
Figure \ref{rise_multipixel_max} shows a single saliency map matching quite well with the binarized network output. The map was generated with the numpy max function on all generated saliency maps. This means the highest pixel value in any of the saliency map is copied into the final saliency map. The resolution of the generated saliency map is quite low, and not many insights can be gathered from it.

Figure \ref{rise_multipixel_mean}, which was generated with the mean function, also shows a heat map matching the binarized network output, although with much less detail than the map generated with the max function.

\subsection{Conclusion}
The modified RISE method provides useful insight into the correctness of the model. It clearly marks the important regions the neural network uses to generate the segmentation output. However, the generated saliency map is of low resolution and does only provide a coarse look into the inner workins of the model. Increasing the quantity of used masks by RISE is possible, but generating a single RISE output already takes around 30 seconds on a high-end consumer GPU with 6000 masks. To generate a higher resolution saliency map, many more masks would be required.


% ******************************* Thesis Appendix B ********************************

\chapter{Supporting information: Chapter 5}

\graphicspath{{Appendix4/Figures/}}

\section{Supplementary figures and tables}
\newpage
\begin{figure}[H]
\flushleft{\textbf{Figure S5.1.} Positive pairwise species correlations to the environment derived from single-predictor LVMs for the full dataset. Connecting lines between species nodes denote positive mean posterior correlations with credible intervals excluding zero. Line colour and thickness indicates the strength of the positive correlation where darker and thicker lines are closer to 1. Species node labels combine the first letters of the genus and specific epiphet.}
\includegraphics[width=1.0\linewidth]{networkplots-positive-unshrink}\\
\end{figure}

\begin{figure}[H]
\flushleft{\textbf{Figure S5.2.} Abundance of each species (excl. \textit{Empodisma minus}) through time. Trend lines obtained with thin-plate splines in a GAMM framework (see main text for model fitting). Shaded region represents 95\% confidence intervals.}
\includegraphics[width=1.0\linewidth]{timeplots}\\
\end{figure}

\begin{table}[H]
  \centering
  \flushleft\textbf{Table S5.1.} Negative and positive pairwise environmental and residual correlations in each of the four plot clusters.
    \begin{tabular}{lllll}
    \toprule
          & \multicolumn{2}{l}{No. sig. env. correlations} & \multicolumn{2}{l}{No. sig. res. correlations} \\
    \midrule
          & -ve   & +ve   & -ve   & +ve \\
    deep soil-moisture & 6,0,1,0 & 12,2,4,13 & 8,11,7,0 & 10,14,30,41 \\
    shallow soil-moisture & 2,0,4,0 & 3,3,2,0 & 10,10,5,0 & 17,9,16,42 \\
    pH    & 0,0,3,0 & 1,1,11,0 & 7,10,6,0 & 11,13,23,41 \\
    total iron & 2,0,2,0 & 2,1,3,1 & 10,11,7,0 & 6,8,17,41 \\
    phosphorous & 4,0,9,3 & 3,5,7,3 & 12,11,4,0 & 7,10,26,34 \\
    organic carbon & 1,0,6,0 & 0,0,10,16 & 8,10,6,0 & 6,12,27,39 \\
    exchangeable aluminium & 2,0,2,6 & 4,8,2,12 & 10,16,5,0 & 15,20,25,41 \\
    exchangeable magnesium & 3,0,5,1 & 3,1,9,8 & 10,11,6,0 & 11,13,28,41 \\
    cation exchange capacity & 1,0,5,1 & 2,3,6,0 & 8,13,5,0 & 14,10,21,43 \\
    \bottomrule
    \end{tabular}%
  \label{tab:addlabel}%
\end{table}%

\includepdf[pages={-}, offset=0.4cm 0cm,  addtotoc= {1,section,1,LVM Primer,hlabel}]{Appendix4/LVM-primer.pdf}

\subsection{fitLVM-auxiliaryfunctions.R}

\begin{verbatim}
##########################################
## Auxilary functions for fitting LVMs  ##
##########################################

## Calculate the correlation between spp due to shared/diverging env responses
extract.env.cor <- function(fit.mod, X, y) {
	n.species <- ncol(y)
	fit.mcmcBase <- fit.mod$BUGSoutput
	mcmc.runs <- mcmc(fit.mcmcBase$sims.matrix, start = 1,
                    thin = fit.mcmcBase$n.thin)
	rm(fit.mcmcBase)

	all.shared.env.mat <- matrix(0,nrow(mcmc.runs),n.species^2)

	## For each MCMC sample, find the correlation between spp responses
	for(t in 1:nrow(mcmc.runs)) {
		cw.spp.coef <- matrix(mcmc.runs[t,grep("spp.coef\\[",colnames(mcmc.runs))],
                          n.species,byrow=F)
		eta.mat <- X%*%t(cw.spp.coef) ## On linear predictor scale
		all.shared.env.mat[t,] <- as.vector(cor(eta.mat))
		}

	## Average/Median over the MCMC samples
	env.mat.mean <- sig.env.mat.mean <- matrix(apply(all.shared.env.mat,2,mean),
                                             n.species,byrow=F)
	env.mat.median <- sig.env.mat.median <- matrix(apply(all.shared.env.mat,2,median),
                                                 n.species,byrow=F)

	get.cor.intervals <- HPDinterval(as.mcmc(all.shared.env.mat), prob = 0.95)
	id.sign.cors <- which(get.cor.intervals[,1] > 0 | get.cor.intervals[,2] < 0)
	sig.env.mat.mean[-id.sign.cors] <- 0
	sig.env.mat.median[-id.sign.cors] <- 0

	return(list(envcor.mean = env.mat.mean,
              envcor.median = env.mat.median,
              sig.envcor.mean = sig.env.mat.mean,
              sig.envcor.median = sig.env.mat.median))
	}

## Produce the residual correlation
extract.residual.cor <- function(fit.mod, X, y) {
	fit.mcmcBase <- fit.mod$BUGSoutput
	mcmc.runs <- mcmc(fit.mcmcBase$sims.matrix,
                    start = 1,
                    thin = fit.mcmcBase$n.thin)
	rm(fit.mcmcBase)

	n.species <- ncol(y)
	n.sites <- nrow(y)
	Tau.arr <- matrix(NA,nrow(mcmc.runs),n.species^2)
	for(t in 1:nrow(mcmc.runs)) {
		lvs <- matrix(mcmc.runs[t,grep("LV",colnames(mcmc.runs))],
                  n.sites,byrow=F)
		lv.coefs <- matrix(mcmc.runs[t,grep("lv.coef",colnames(mcmc.runs))],
                       n.species,byrow=F)
		Tau.arr[t,] <- as.vector(cor(lvs%*%t(lv.coefs))) }

	## Average/Median over the MCMC samples
	Tau.mat.mean <- sig.Tau.mat.mean <- matrix(apply(Tau.arr,2,mean),
                                             n.species,byrow=F)
	Tau.mat.median <- sig.Tau.mat.median <- matrix(apply(Tau.arr,2,median),
                                                 n.species,byrow=F)

	get.cor.intervals <- HPDinterval(as.mcmc(Tau.arr), prob = 0.95)
	id.sign.cors <- which(get.cor.intervals[,1] > 0 | get.cor.intervals[,2] < 0)
	sig.Tau.mat.mean[-id.sign.cors] <- 0
	sig.Tau.mat.median[-id.sign.cors] <- 0

	return(list(rescor.mean = Tau.mat.mean,
              rescor.median = Tau.mat.median,
              sig.rescor.mean = sig.Tau.mat.mean,
              sig.rescor.median = sig.Tau.mat.median))
	}

## Produce mean and median point estimates from fit
extract.params <- function(fit.mod) {
	fit.mcmcBase <- fit.mod$BUGSoutput
	mcmc.runs <- mcmc(fit.mcmcBase$sims.matrix, start = 1,
                    thin = fit.mcmcBase$n.thin)
	rm(fit.mcmcBase)
	n.species <-  length(grep("spp.int",colnames(mcmc.runs)))

	all.spp.coef <- mcmc.runs[,grep("spp.coef",colnames(mcmc.runs))]
	spp.coef.mean <- matrix(apply(all.spp.coef,2,mean),n.species,byrow=F)
	spp.coef.median <- matrix(apply(all.spp.coef,2,median),n.species,byrow=F)

	all.spp.int <- mcmc.runs[,grep("spp.int",colnames(mcmc.runs))]
	spp.int.mean <- apply(all.spp.int,2,mean)
  spp.int.median <- apply(all.spp.int,2,median)

	all.spp.phi <- mcmc.runs[,grep("spp.phi",colnames(mcmc.runs))]
	spp.phi.mean <- apply(all.spp.phi,2,mean)
  spp.phi.median <- apply(all.spp.phi,2,median)

	all.mu.beta <- mcmc.runs[,grep("mu.beta",colnames(mcmc.runs))]
	mu.beta.mean <- apply(all.mu.beta,2,mean)
  mu.beta.median <- apply(all.mu.beta,2,median)

	all.tau.beta <- mcmc.runs[,grep("tau.beta",colnames(mcmc.runs))]
	tau.beta.mean <- apply(all.tau.beta,2,mean)
  tau.beta.median <- apply(all.tau.beta,2,median)

	return(list(spp.coef.mean = spp.coef.mean, spp.coef.median = spp.coef.median,
              spp.int.mean = spp.int.mean, spp.int.median = spp.int.median,
              spp.phi.mean = spp.phi.mean, spp.phi.median = spp.phi.median,
              mu.beta.mean = mu.beta.mean, mu.beta.median = mu.beta.median,
              tau.beta.mean = tau.beta.mean, tau.beta.median = tau.beta.median))
	}
\end{verbatim}