\chapter{Cart-Pole \& Mountain-Car}
\section{Cart-Pole}
The cart-pole environment, which is an implementation of the famous cart-pole problem [7], is a popular AI benchmark that simulates a pole that needs to be balanced on the flat rectangle (cart) that it's attached to. The observation object of this environment includes the velocity and position of the cart, the velocity at the tip of the pole, and the angle of the pole. Initially, these parameters are assigned a random value in the range $[-0.05, 0.05]$, which results in the pole standing almost upright with a slight tilt. The environment only allows two discrete actions (left and right) which the agent uses to balance the pole before it falls over, by pushing the cart without causing it to go off-screen. The environment terminates if the agent fails in either of those tasks, or after the maximum number of time-steps (200) is reached. The agent receives $1$ reward on every time-step until one of these termination criteria is met. Finally, this environment is considered to be solved if the agent achieves an average reward of greater than or equal to $195.0$ over 100 consecutive trials. More details about the various environmental parameters can be found in the environment’s official GitHub Wiki page\footnote{\url{https://github.com/openai/gym/wiki/CartPole-v0}}.

\subsection{Experiment 1: Binary Programs}
The goal of the first experiment was to implement the simplest GP algorithm that performs better than a random agent. The simplest language that can encode the actions allowed in this environment is one of 0’s and 1’s, where 0’s represent ‘left’ and 1’s represent ‘right’. The program structure, therefore, included the terminal set \verb+T = {0, 1}+ and an empty function set. The intuition for this structure was that, in principle, there has to exist a string of left-right actions that manages to balance the pole on the cart most of the time.

This GP agent did not managed to achieve better-than-random performance: a random agent achieves an average reward of $\approx22.0$ and the best programs the GP algorithm generated achieved an average reward of $16.76$. The most likely reason for this is that these programs are static, which means they don’t use information about the state of the environment to compute the next action to take. This limitation would not be a problem if the initial state of the environment was always the same, because that would make the state transitions deterministic and, therefore, there would exist a static program that would reliably balance the pole. However, the initial state of the environment is randomised each time the environment starts running and the range of possible values for the various parameters of the environment is sufficiently large, such that no static strategy can perform well.

\subsection{Experiment 2: Known Strategy}
The main goal of the second experiment was to enable the agent to utilise the environment state to choose actions dynamically. At the same time, I wanted to test the capabilities of the GP algorithm, since it would be used to find increasingly complex programs as the environments proved to be more difficult to solve. To achieve both of these goals, a manual strategy was discovered that performed well by utilising the environment state. The program structure of the GP algorithm was also adjusted to allow it to discover the strategy automatically. The chosen strategy can be described as follows:

\begin{verbatim}
    if pole_angle <= 0 then push_left() else push_right()
\end{verbatim}

Intuitively, this strategy describes the behaviour "if the pole is leaning to the left, push the cart to the left; if the pole is leaning to the right, push the cart to the right". This should keep the pole balanced at all times. To test this intuition, I implemented an agent that follows this strategy and tested it on the environment. The agent achieved an average score of $42.0$, a major improvement over the previous agent. Then, I introduced a simple environment-specific language to replace the binary language of the previous experiment, which included variables to represent the environment state and the actions the agent can take, flow-control (if-statements) and comparison operators to allow the agent to make decisions dynamically, and constants for calculations. The function and terminal set used for this experiment were \verb+F = {IFLTE}+ and \verb+T = {pa, 0, L, R}+ respectively.

The function set included a single function, \verb+IFLTE+ (if-less-than-or-equal), which accepts 4 inputs; if the first input is less than or equal to the second input, it outputs the third input; otherwise, it outputs the fourth input. This functionality seemed sufficient to encode the strategy described above. The terminal set included four elements: the pole angle (\verb+pa+) used to encode the corresponding environment observation, the numerical constant \verb+0+ which was required for the comparison performed in the strategy, and the actions \verb+L+ and \verb+R+, left and right respectively. The program structure was quite restricted in this first version of the algorithm to make it easier to evaluate and troubleshoot before applying it to more difficult environments that would require a more complex program structure. Specifically, each program was restricted to consist of a single \verb+IFLTE+ function, which could only accept \verb+pa+ and \verb+0+ as its first two arguments, and \verb+L+ and \verb+R+ as its third and fourth arguments (figure \ref{fig:iflte_tree}). It was important to restrict the third and fourth arguments to actions since the output of the program had to be the action the agent would perform in the environment. Therefore, a simple type system was implemented to perform this check. Using this simple functional language, the manually-discovered strategy could be encoded as \verb+IFLTE(pa, 0, L, R)+.

\begin{figure}[ht]
    \centering
    \includegraphics[width=12cm]{images/simple_iflte_program.png}
    \caption{Tree representation of the program structure of the Known Strategy agent}
    \label{fig:iflte_tree}
\end{figure}

With these structural restrictions, it was easy for the GP algorithm to converge to the intended strategy (it only required a single generation, and mutation was not needed) because the program space was very small. The exact parameters used for the experiment are summarised in table \ref{tab:cartpole_exp2_params}.

\begin{table}[ht]
    \centering
    \begin{tabular}{|l|c|}
        \hline
        \textbf{Parameters} & \textbf{Values} \\
        \hline
        Population size     & 10  \\
        Max generations     & 5   \\
        Max program depth   & 1  \\
        Terminal fitness & 195.0 \\
        Number of runs      & 1   \\
        Number of episodes  & 100 \\
        Episode length      & 200  \\
        \hline
    \end{tabular}
    \caption{Experiment parameters (cart-pole experiment 2)}
    \label{tab:cartpole_exp2_params}
\end{table}

\subsection{Experiment 3: Exploration}
\subsubsection{Setup and Motivation}
The goal of the final experiment was to expand the program space to encourage exploration and allow the algorithm to discover better strategies than the one found manually. The terminal set was extended to include the second observation of the environment, pole velocity (\verb+pv+), to allow the algorithm to incorporate it in the solution. Additionally, I experimented with constants other than \verb+0+ and it turned out that the performance was very sensitive to even very small deviations from \verb+0+. Specifically, I discovered that including the constant \verb+0.025+ in the terminal set increased the performance of the discovered solutions significantly. Finally, the program structure was adapted to allow for the generation of programs that utilised higher-order functions (i.e. functions that accept functions as arguments) which can be represented as a trees of varying depth (see figure \ref{fig:deep_iflte_tree} for an example). The extended terminal set was \verb+T = {pa, pv, 0, 0.025, L, R}+. The parameters used for this experiment are summarised in table \ref{tab:cartpole_exp3_params}.

\begin{figure}[ht]
    \centering
    \includegraphics[width=12cm]{images/complex_iflte_program.png}
    \caption{Tree representation of the program structure of the Exploration agent}
    \label{fig:deep_iflte_tree}
\end{figure}

\begin{table}[ht]
    \centering
    \begin{tabular}{|l|c|}
        \hline
        \textbf{Parameters} & \textbf{Values} \\
        \hline
        Population size     & 100  \\
        Max generations     & 5  \\
        Max program depth   & 2  \\
        Terminal fitness & 195.0 \\
        Number of runs      & 1   \\
        Number of episodes  & 100 \\
        Episode length      & 200  \\
        \hline
    \end{tabular}
    \caption{Experiment parameters (cart-pole experiment 3)}
    \label{tab:cartpole_exp3_params}
\end{table}

\subsubsection{Results and Discussion}
The initial results of this experiment showed that the second observation, pole velocity, was more useful than the pole angle, so programs using \verb+pv+ instead of \verb+pa+ tended to dominate the population during the first few generations. The first run of GP converged on the program \verb+IFLTE(pv, 0, L, R)+, which achieved an average reward of $181.78$. This improvement could be attributed to the fact that the velocity of the pole provided the agent with more useful information than its angle. Using the first strategy (\verb+IFLTE(pa, 0, L, R)+), if the pole leaned towards, say, the right at the beginning of the run, the agent would begin to push the cart to the right to cause the pole to lean towards the left, bringing it back to the center. The problem with this strategy was that by the time the pole angle became negative (meaning the pole leaned towards the left) it had accumulated too much momentum for the agent to manage to push left to balance it on time. The agent was essentially over-correcting. Using the new strategy, the agent detected which direction the pole was accelerating towards (using the sign of its velocity) and pushed the cart towards that direction for just long enough to prevent the pole from falling over, without over-correcting. However, this strategy was still problematic (as is evident from the imperfect score it achieved) because the agent stopped pushing the cart as soon as the sign of the pole velocity was flipped, so the pole didn't have enough momentum to lean towards the other side, thereby letting it fall towards the same side again. The result of this was that the agent was now under-correcting, balancing the pole perfectly, but at an angle, causing the cart to move until it eventually exited the screen and the environment terminated. It was evident at this point that a solution would involve information about both the pole angle and its velocity.

Increasing the maximum number of generations from 5 to 10 allowed for more complex solutions to arise. Eventually, the algorithm consistently produced solutions that achieved the reward specified as the solution criterion for the environment. These solutions included programs using nested if statements and combining both observation variables. An example of such a program is:
\begin{verbatim}IFLTE(0.025, pa, IFLTE(pv, pv, R, L), IFLTE(0.025, pv, R, L))\end{verbatim}
This program achieves an average reward of $198.74$ over 100 consecutive trials. The following is an equivalent, but more readable, version of the strategy described by this program:

\begin{verbatim}
if pa > 0.025 then R else (if pv > 0.025 then R else L)
\end{verbatim}

Following this strategy, the agent pushed the cart to the right to balance the pole if it was leaning at a right angle greater than some threshold ($0.025$). If the pole was leaning at a left angle or a small right angle, the agent checked the pole's velocity to make its decision: if the pole was accelerating towards the right (\verb+pv > 0.025+) then the agent pushed right to counteract it; otherwise it pushed left for the same reason.

This extended version of the program structure allowed the agent to achieve the goal of utilising exploration to find new and better solutions by exploring a larger program space. At this stage, the GP algorithm can incorporate new language extensions as well as additional genetic operators, both of which will be required in the following environments. So, Cart Pole was a useful starting point in showcasing the potential of this approach and establishing a framework for applying it to other RL environments.

\section{Mountain-Car}
The second RL environment attempted, was mountain-car. This environment simulates a car that attempts to climb a steep hill (figure \ref{fig:mountain-car}). There are two versions of this environment, one in which the actions the agent can take are discrete (similar to cart-pole) and one in which they are continuous (the action is a number that represents the magnitude of the force the agent applies to the car). The former is, in principle, easier to solve because there are fewer possible actions and, therefore, fewer strategies to explore before discovering a solution. The discrete version of the environment was attempted first with the hope that a solution to the simpler environment could be used as a basis for the more complex one.

\subsection{Experiment 1: Discrete Actions}
For the discrete version of this environment, I used the same parameters as in cart-pole experiment 3, and a similar program structure, adapted to the details of this environment:

\begin{verbatim}
F = {IFLTE}
T = {position, velocity, 0.0, 0, 1, 2}
\end{verbatim}
The terminals \verb+position+ and \verb+velocity+ represent the corresponding environment observations, \verb+0.0+ is a numerical constant used for comparisons (similar to cart-pole), and \verb+0+, \verb+1+, and \verb+2+ encode the discrete action space of the environment (push left, no action, and push right respectively).

In fewer than $10$ generations, the GP algorithm found the simple and intuitive program \\\verb+IFLTE(0.0, velocity, 2, 0)+, which achieves a fitness score of approximately $-120$, which is very close to the requirement for a solution, $-110$. This program encodes the strategy “if the velocity of the car is positive, push the car to the right (i.e. increase its velocity), otherwise push it to the left (i.e. decrease its velocity).” The agent effectively swings the car back and forth, maximising its momentum, until it can reach a high enough velocity to climb the hill and reach the goal.

\subsection{Experiment 2: Continuous Actions}
The interpretable nature of the strategy found for the discrete version of this environment makes it reasonable to hypothesise that the same strategy should perform well on the continuous version of this environment. The same principle of applying a force proportional to the car’s velocity can be used, so the only required modification is changing the value of this applied force to a continuous value in the range $[0.0, 1.0]$, which defines the action space of this environment.

The problem then becomes a simple numerical optimisation problem with one variable. Evaluating the program using all values in $[0.0, 1.0]$ with a $0.001$ increment, reveals that the optimal force is $0.180$, which produces an average fitness score of $98.80$, which is greater than the requirement for a solution ($90$). The complete set of results is visualised in figure \ref{fig:mountain_car_cont}, which shows that values between $0.0$ and $0.179$ produce negative fitness scores (the car is unable to reach the goal), and values after $0.180$ linearly decrease performance. The reason for the negative values at the beginning of the range is that the force applied to the car is not great enough to eventually allow it to climb the hill, so it only receives negative rewards from the environment on every time step. The optimal value is the smallest amount of force necessary to eventually allow the car to climb the hill, which satisfies the requirement of the environment (reaching the goal with the minimum amount of effort). Every value greater than the optimal increases the amount of effort beyond what is necessary for the goal to be achieved and, therefore, reduces the overall fitness score.

\begin{figure}[ht]
    \centering
    \includegraphics[width=12cm]{images/mountaincarcont.png}
    \caption{The x-axis represents the magnitude of the for applied to the car (multiplied by 1000) and the y-axis represents the average reward achieved using that magnitude.}
    \label{fig:mountain_car_cont}
\end{figure}

The simple and intuitive nature of this solution is further confirmation that GP is a suitable approach for solving (at least simple) RL environments. Another interesting feature of this program is its similarity to the solution for cart-pole, despite the differences between the environments. This is an indication that environments that share common characteristics (e.g. classic control environments) might be solvable using similar solutions.


\documentclass[main.tex]{subfiles}
\begin{document}

\section{Initial Data Problem}

\marginpar{Tuesday\\ 2021-6-1, \\ compiled \\ \today}

It is perhaps the most difficult problem currently.

The problem is stated as
%
\begin{align}
C_0 &= R + K^2 - K_{ij} K^{ij} - 16 \pi E = 0  \\
C_i &= D_j K^{j}_{i} - D_i K - 8 \pi P_i = 0
\,,
\end{align}
%
and we want to calculate \(\gamma_{ij}\) and \(K_{ij} \) on \(\Sigma _0\) such that
\begin{enumerate}
    \item the constraints are satisfied;
    \item the solution is physically meaningful: it really describes a BH, BBH, \(n\)-BH, NS spacetime.
\end{enumerate}

We will assume for today that the matter fields are given or zero --- in general one would have to solve the hydrodynamic equations for them as well.

We have 4 equations for 12 unknowns!
We will need to prescribe 8 quantities.

The problem is split in two: a problem of constrained data (4)
and one of free data (8).

The choice of free data is guided by a few principles:
\begin{enumerate}
    \item we need to define our (astro)physical expectations for \(\Sigma_0 \) and verify that they are matched;
    \item we need some heuristic or intuition for the various fields;
    \item mathematical (/ computational?) necessity: the equations \(C_\alpha = 0\) should be as nice as possible --- linearity, decoupling, well-posedness are all desirable.
\end{enumerate}

The \textbf{Conformal Decomposition} allows us to implement these.
Within it, there are two main formalisms: one is the ``conformal/transverse traceless'' formalism (CTT), due to York in 1973; the other is the ``conformal/thin sandwich'' (CTS), also due to York in 1999.
Both are based off of early work by Lichnerowicz.

The CTS formalism is often used to generate binary systems --- it makes it easier to implement the quasi-symmetries there.

We start from the Lichnerowicz equation (the Hamiltonian constraint):
%
\begin{align}
C_0 &= \widetilde{D}_i \widetilde{D}^{i} \psi
- \frac{1}{8} \hat{A}_{ij} \hat{A}^{ij} \psi^{-7} +
\qty(- \frac{K^2}{12} + 2 \pi E) \psi^{5} &= 0
\,,
\end{align}
%
with \(p = -10\).
Let us solve it as a standalone equation: it is a Poisson-like one, since \(\widetilde{D}_i \widetilde{D}^{i}\) is similar to a Laplacian.
Then, this is some sort of elliptic operator applied to \(\psi \) equated to some powers of \(\psi \).

If \(K^2= 0\) the equation simplifies, and the BVP can be studied: there are several known results about the well-posedness of this equation under the hypothesis \(K = \const\).

These are known as Constant Mean Curvature (CMC) spacetimes.

If we take an Asymptotically Flat, CMC spacetime with \(K = 0\), and with \(E = 0\), then the BVP with the Lichnerowicz equation is \emph{solvable} for a ``large'' class of metrics \(\widetilde{\gamma}_{ij}\) (a ``positive Yamabi class'').

There is a ``prototype equation'' for the Lichnerowicz one if we set \(K = 0\):
%
\begin{align}
\triangle u = f^2 u^{p}
\,,
\end{align}
%
in flat spacetime, for some function \(f\).

These theorems are found in somewhat specialized PDE literature.
In the linear case \(p = 1\), and we take \(\eval{u}_{\partial \Omega } = 0\), then the solution, identically zero, is unique.

In the nonlinear case, the solution is unique iff \(p > 0\) (or specifically, the same sign of the coefficient of the \(f^2 u^{p}\) term).

So, what about the Lichnerowicz equation?
The easiest thing to do is to linearize the equation: we write it as
%
\begin{align}
\widetilde{\triangle} \psi + H (\psi ) = 0
\,.
\end{align}

We write \(\psi = \psi_0 + \epsilon \), and expand
%
\begin{align}
H(\psi ) = H(\psi_0 ) + \eval{\pdv{H}{\epsilon }}_{0} \epsilon + \order{\epsilon^2}
\,,
\end{align}
%
so we find an equation in the form \(\widetilde{\triangle} \epsilon = f \epsilon \) with:
%
\begin{align}
f = \frac{1}{8 } \widetilde{R} + \frac{7}{8} \hat{A}_{ij} \hat{A}^{ij} \psi_0^{-8} - 10 \pi E \psi_0^{4}
\,.
\end{align}

This immediately shows us the problem: if \(K =0 \) then \(\widetilde{R} > 0\), but we can check that \(f\) is not positive, because of the ``matter term''!

However, we insist on solving this equation, and we can do so as long as we \emph{rescale} the matter term.

If we define \(\widetilde{E} = \psi^{s} E\), then, the term becomes
%
\begin{align}
- 5 \times 2 \pi E \psi_0^{4} \to - (s \times 5) 2 \pi \widetilde{E} \psi^{4-s}
\,,
\end{align}
%
so this is \(> 0\) for any \(s > 5\).
This is \emph{not} a trick: in practice, these elliptic equations are solved by iteration --- specifying a guess, and then iterating.

Specifying the ``correct'' RHS therefore is key if we want to find a solution.
So, what do we use?
The best answer turns out to be 8: if we do this, we can express the dominant energy condition in conformal variables: if
%
\begin{align}
\widetilde{E} = \psi^{8} E
\qquad \text{and} \qquad
\widetilde{P}^{i} = \psi^{10} P^{i}
\,,
\end{align}
%
then \(\widetilde{E}^2 \geq \widetilde{P}^2\) implies \(E^2 \geq P^2\). In general this would read
%
\begin{align}
\psi^{2s} E^2 \geq \psi^{-4} \psi^{10} \psi^{10} P^2
\,.
\end{align}

\subsection{Maximal Slicing}

What does it mean to impose \(K = 0\)?
As we will see shortly, this is a gauge condition which extremizes the volume of \(\Sigma _t\).

We just need to remember the definition of the trace: it is an identity that
%
\begin{align}
K = \gamma^{ab} K_{ab} = - \frac{1}{2 \alpha } \gamma^{ij} \mathscr{L}_m \gamma_{ij} = - \frac{1}{2 \alpha } \mathscr{L}_m \log \gamma
\,,
\end{align}
%
which can be nicely written as
%
\begin{align}
K = - \frac{1}{\alpha } \mathscr{L}_m \log \sqrt{\gamma }
\,,
\end{align}
%
so we recover the volume element on \(\Sigma _t\), \(\sqrt{\gamma }\)!

The volume in a certain region is \(V = \int \sqrt{\gamma } \dd[3]{x}\), so if we perform a variation \(v^{a} = \delta t \qty(\alpha n^a + \beta^{a})\), so that \(\eval{v^{a}}_{S} = 0\) (the deformation is zero at the boundary), the volume changes as such:
%
\begin{align}
\fdv{V}{t} &= \int \partial_{t} \sqrt{\gamma } \dd[3]{x}
= \int \qty(- \alpha K + D_i \beta^{i}) \dd[3]{x}
\marginnote{Using the kinematic equation.}  \\
&= - \int_V \alpha K \sqrt{\gamma } \dd[3]{x} + \underbrace{\oint_S \beta^{i} S_i}_{= 0}
= - \int \alpha K \sqrt{\gamma } \dd[3]{x}
\,,
\end{align}
%
therefore \(K = 0\) yields \(\delta V = 0\): this is a maximum.

It is the exact same geometric problem as a film of soap.
If the soap is held at a ring, it stays flat.

The only difference from that case is that here we are in a Lorentzian geometry: in the Euclidean case we have a minimum, here we have a maximum.

This is an example of a singularity-avoiding gauge, used by many works.

\subsection{The Conformal Transverse Traceless formalism}

We use the \(p = -10\) scaling of \(K_{ij}\), and on top of the Lichnerowicz equation we derive an equation from the momentum constraint \(C_i = 0\), for a vector \(x^{i}\) which is obtained with a further decomposition of \(\hat{A}^{ij}\):
%
\begin{align}
\hat{A}^{ij} = \hat{A}^{ij}_L + \hat{A}^{ij}_{TT}
\,,
\end{align}
%
where the TT part is transverse and traceless: \(\widetilde{\gamma}_{ij} \hat{A}^{ij}_{TT} = 0\) and \(\widetilde{D}_j \hat{A}^{ij}_{TT} = 0\).

The longitudinal part on the other hand is expressed as follows:
%
\begin{align}
\qty(\widetilde{L} x)^{ij} = \widetilde{D}^{i} x^{j} + \widetilde{D}^{j} x^{i} - \frac{2}{3} \widetilde{D}_k x^{k} \widetilde{\gamma}^{ij}
\,.
\end{align}

This seems like a rather strange expression: \(\widetilde{L}\) is called the \textbf{Conformal Killing operator} on \(x^{i}\).
It is determined as follows:
%
\begin{align}
\widetilde{D}_j  \hat{A}^{ij} =
\widetilde{D}_j  \qty(\widetilde{L} x)^{ij}
= \widetilde{D}_j  \widetilde{D}^{j} x^{i}
+ \frac{1}{3} \widetilde{D}^{i} \widetilde{D}_j x^{i} + \widetilde{R}^{i}{}_j x^{i} = \widetilde{\triangle}_L x^{i}
\,,
\end{align}
%
where \(\widetilde{\triangle}_L x^{i}\) is the \textbf{conformal vector Laplacian operator}.

There exists a unique L + TT decomposition of \(\hat{A}^{ij}\) iff there exists a unique solution of the conformal Laplacian equation:
%
\begin{align}
\widetilde{\triangle}_L x^{i} = \widetilde{D}_j \hat{A}^{ij}
\,.
\end{align}

This is useful because there is a theorem by Cantor in 1979 which tells us that if \(\Sigma \) is asymptotically flat and we have \(\partial_{k} \partial_{l} \widetilde{\gamma}_{ij} = \order{r^{-3}}\),
then existence and uniqueness are guaranteed.

With this decomposition, we obtain
%
\begin{align}
\widetilde{\triangle}_L x^{i} - \frac{2}{3} \widetilde{D}^{i} K \psi^{6} - 8 \pi \widetilde{P}^{i} = 0
\,,
\end{align}
%
which can be solved together with the Lichnerowicz equation.
This allows us to constrain \(\psi\) and \(x^{i}\), as long as we determine the free data: \(\widetilde{\gamma}_{ij}\), \(\hat{A}^{ij}_{TT}\) and \(K\).

Under maximal slicing, \(K = 0\), the two equations decouple!
They also partially decouple if we take \(K = \const\), since we can solve one and then the other.

Also, it is nice that the free data are the conformal metric: \(\widetilde{\gamma}_{ij}\) and \(\hat{A}_{TT}^{ij}\) are useful in heuristically determining the GW content of \(\Sigma \).

Let us give an example of CTT data under the four hypotheses of conformal flatness, asymptotic flatness, and maximal slicing.

Asymptotic flatness is chosen because we want the solution to exist.
This is the simplest choice of free data: we take \(\hat{\gamma}_{ij} = f_{ij}\) (conformal flatness) and \(K = 0\) (maximal slicing).

Further, we consider \(E = 0 = P^{i}\) (vacuum).

Without justification we also assume that \(\hat{A}^{ij}_{TT} = 0\).

Under all these assumptions we have
%
\begin{align}
\widetilde{D}_i = D_i
\,,
\end{align}
%
therefore
%
\begin{align}
\widetilde{D}_i \widetilde{D}^{i} = D_i D^{i} = \triangle
\,
\end{align}
%
is simply the flat elliptic operator, also \(\widetilde{R} = 0\) and \(\widetilde{L} = L\) is the one calculated in the flat case.

So, the CTT equations become:
%
\begin{align}
\triangle \psi + \frac{1}{8} \qty(Lx )_{ij} \qty(Lx)^{ij} \psi^{-7} &= 0  \\
\triangle_L x^{i} = \triangle x^{i} + \frac{1}{3} D_j D^{i} x^{j} &= 0
\,,
\end{align}
%
which are decoupled, so they can be solved independently of one another.
They are ``easy'', since they use the flat Euclidean operators.
If we want a boundary value problem we also need to specify the boundaries: we specify asymptotic flatness: \(\psi = 1\) and \(x^{i} = 0\) at \(\iota_0\).

Optionally we can impose strong-field inner boundary conditions: for example we can fix the topology.

Our \textbf{case 1} is to take \(\Sigma_0 = \mathbb{R}^{3}\) (no inner boundary condition): then, the solution of the second CTT equation is \(x^{i}= 0\), and the first one reads \(\triangle \psi = 0\) together with \(\psi = 1\) at \(\iota_0 \): this implies \(\psi \equiv 1\).
As expected, we recover flat spacetime.

Can we also get a nontrivial solution?
For that, we need an inner boundary.

\end{document}


% BReviewofLinearAlgebra.tex
% Fund Science! & Help Ernest finish his Physics Research! : quantum super-A-polynomials - a thesis by Ernest Yeung
%
% http://igg.me/at/ernestyalumni2014
%
% Facebook     : ernestyalumni
% github       : ernestyalumni
% gmail        : ernestyalumni
% google       : ernestyalumni
% linkedin     : ernestyalumni
% tumblr       : ernestyalumni
% twitter      : ernestyalumni
% youtube      : ernestyalumni
% indiegogo    : ernestyalumni
%
% Ernest Yeung was supported by Mr. and Mrs. C.W. Yeung, Prof. Robert A. Rosenstone, Michael Drown, Arvid Kingl, Mr. and Mrs. Valerie Cheng, and the Foundation for Polish Sciences, Warsaw University.

\section{Review of Linear Algebra }

\subsection{Linear Maps}

\exercisehead{B.1}
\begin{enumerate}
  \item[(a)]
\item[(b)]
\item[(c)]
\item[(d)] \textbf{Want}: if $(v_1 \dots v_k)$ linearly dependent $k$-tuple in $V$, $v_1 \neq 0$, \\
then some $v_i = \sum_{j=1}^{i-1} c^j v_j$ \\
\begin{proof}
$(v_1 \dots v_k)$ linearly dependent, so if $\sum_{i=1}^k a^i v_i = 0$, $a^i$ not all equal to $0$.  \\

Suppose for fixed $i$, $2\leq i \leq k$, $a^{i+1} = \dots =a^k=0$. \\
\quad Indeed, suppose for $\sum_{i=1}^k a^i v_i = 0$, $a^2 = \dots = a^k = 0$.  $a^1 v_1 =0$, $v_1 \neq 0$, $a^1 =0$.  Then $(v_1 \dots v_k)$ linearly independent.  Contradiction.

if $i=2$, \\
\phantom{\quad} $a^1 v_1 + a^2 v_2 =0$ \\
\phantom{\quad \,} $\Longrightarrow v_2  = \frac{-a^1}{a^2} v_1$ \\

if $i=k$, $v_k = \frac{ -\sum_{i=1}^{k-1} a^i v_i }{ a^k}$  \\

So in general, $v_i = \frac{ -\sum_{j=1}^{i-1} a^j v_j }{ a^i}$
\end{proof}
\end{enumerate}

\exercisehead{B.9}
given $(E_1 \dots E_n)$ basis for $V$ \\
\phantom{ \quad } $\exists \, \lbrace i_1 \dots i_k \rbrace \subset \lbrace 1 \dots n \rbrace$ s.t. \\
\phantom{ \quad \, } $ \text{span}(E_{i_1} \dots E_{i_k} )$ is complement to $S$ \\

Hence $\forall \, $ subspace $S \subseteq V$, $\exists \, $ complementary subspace $T$ in $V$, so $V = S \oplus T$

\begin{proof}
$\forall \, $ subspace $S$ is itself a vector space, closed under addition and multiplication. \\
\phantom{ \quad } Hence $S$ has basis $(F_1 \dots F_m)$ with $\text{dim}S = m$ \\

Consider ordered $(m+n)$-tuple
\[
(F_1 \dots F_m, E_1 \dots E_n)
\]

$(F_1 \dots F_m, E_1 \dots E_n)$ linearly dependent in $V$, by linear algebra.

For $j_1 \in \lbrace 1 \dots n \rbrace$, $E_{j_1}$ linear combination of previous vectors (cf. Exercise B.1(d)) \\
\phantom{ \quad } eliminate $E_{j_1}$ : $(F_1 \dots F_m, E_1 \dots \widehat{E}_{j_1} \dots E_n )$. \\

Repeat, until there are $n-m$ $E$ basis vectors left, labeled $i_1 \dots i_{n-m}$ (hence \textbf{use Exercise B.1(d)} many and enough times, $m$ times)
\[
\Longrightarrow (F_1 \dots F_m, E_{i_1} \dots E_{i_{n-m}} )
\]
By linear algebra, $(F_1 \dots F_m, E_{i_1} \dots E_{i_{n-m}})$ a basis for $V$, linearly independent. The procedure wouldn't have ``overshot'' by a Thm. (see Apostol's \textbf{Calculus} Vol. 2, first few chapters, linear algebra part) \\

$\forall \, v\in V$, $v= a^iF_i  + b^{i_j} E_{i_j}$ with $a^i F_i \in S$.  Then $b^{i_j} E_{i_j} \in T$.  \\
\phantom{ \quad } since $V = S\oplus T$, $T$ complement to $S$

$\Longrightarrow $ given fixed basis of $V$, $(E_1 \dots E_n)$, subspace $S \subseteq V$, $S$ having basis $(F_1 \dots F_m)$, $S$ has complementary subspace in $V$, $T$, \\
\phantom{ \quad } s.t. basis of $T$ is $(E_{i_1} \dots E_{i_{n-m}})$ and $V = S\oplus T$

\end{proof}

\exercisehead{B.13}  Suppose $\exists \, $ linear $T: V \to W $ s.t. $T(E_i) = w_i$, \quad \, $i = 1 \dots n$ \\
Suppose $\exists \, $ linear $T':V \to W$ s.t. $T'(E_i)= w_i$, \quad \, $i = 1 \dots n$

Let $x\in V$, so $x  = x^i E_i$ (the key idea is that with a basis, the vector space is completely determined, vectors in the vector space are spanned by the basis elements)
\[
(T-T')(x) \equiv T(x) - T'(x)  = x^i w_i - x^i w_i = 0
\]
$T(x) = T'(x)$\quad \, $\forall \, x \in V$ \\
so $T=T'$.  $T$ unique.

$T$ exists by construction.
\hrulefill

\textbf{ affine subspace } of $V$ parallel to $S$, linear subspace $S \subseteq V$, $v + S = \lbrace v + w | w \in S \rbrace$, some fixed $v \in V$ \\

\textbf{affine map } $F: V \to W$ if $F(v) = w + Tv$ for some $T: V \to W$, some fixed $w\in W$ \\

\exercisehead{B.16}

Let $a,b \in \mathbb{C}$, $x, y \in F(V)$ \\

Now
\[
F(V) = \lbrace y | y = w + Tv = F(v), \, v \in V, \text{ fixed } w \in W , \text{ some } T \rbrace
\]

\subsubsection{Change of Basis}

\exercisehead{B.22} Suppose $V,W, X$ finite-dim. vector spaces \\
$S:V\to W$, \, $T:W \to X$

\begin{enumerate}
\item[(a)] $\text{rank}S \leq \text{dim}V$ \quad \, $\text{rank}S = \text{dim}V$ iff $S$ injective
\item[(b)] $\text{rank}S \leq \text{dim}W$ \quad \, $\text{rank}S = \text{dim}W$ iff $S$ surjective
\item[(c)] if $\text{dim}V = \text{dim}W$ and $S$ either injective or surjective, then $S$ isomorphism
\item[(d)] $\text{rank}TS \leq \text{rank}S$ \quad \, $\text{rank}TS = \text{rank}S$ iff $\text{im}S \bigcap \text{ker}T = 0$
\item[(e)] $\text{rank}TS \leq \text{rank}T$ \quad \, $\text{rank}TS = \text{rank}T$ iff $\text{im}S + \text{ker}T = W$
\item[(f)] if $S$ isomorphism, then $\text{rank}TS = \text{rank}T$
\item[(g)] if $T$ isomorphism, then $\text{rank}TS = \text{rank}S$
\end{enumerate}

EY : Exercise B.22(d) is useful for showing the chart and atlas of a Grassmannian manifold, found in the More examples, for smooth manifolds.

\begin{proof}
\begin{enumerate}
\item[(a)]
\item[(b)]
\item[(c)]
\item[(d)] Now
\[
\begin{aligned}
  & \text{dim}V = \text{rank}TS + \text{nullity}TS \\
  &  \text{dim}V = \text{rank}S + \text{nullity}S
\end{aligned}
\]
$\text{ker}S \subseteq \text{ker}TS$, clearly, so $\text{nullity}S \leq \text{nullity}TS$
\[
\Longrightarrow \boxed{ \text{rank}TS \leq \text{rank}S }
\]

If $\text{rank}TS = \text{rank}S$, \\
\phantom{ \quad } then $\text{nullity}S = \text{nullity}TS$ \\
\phantom{ \, } Suppose $w \in \text{Im}S \bigcap \text{ker}T$, $w \neq 0$ \\
\phantom{ \quad } Then $\exists \,  v\in S$, s.t. $w = S(v)$ and $T(w)=0$ \\
\phantom{ \quad \, } Then $T(w) = TS(v) =0$.  So $v\in \text{ker}TS$ \\
\phantom{ \quad \quad \, } $v\notin \text{ker}S$ since $w = S(v) \neq 0$ \\
\phantom{ \quad \quad \, } This implies $\text{nullity}TS > \text{nullity}S$.  Contradiction. \\
$\Longrightarrow \text{Im}S \bigcap \text{ker}T =0$ \\

If $\text{Im}S \bigcap \text{ker}T =0$, \\
\phantom{ \quad } Consider $v \in \text{ker}TS$.  Then $TS(v)=0$.  \\
\phantom{ \quad  Consider $v \in \text{ker}TS$}.  Then $S(v)  \in \text{ker}T$ \\
\phantom{ \quad  } $S(v) =0$; otherwise, $S(v) \in \text{Im}S$, contradicting given $\text{Im}S \bigcap \text{ker}T =0$ \\
\phantom{ \quad \quad } $v\in \text{ker}S$ \\

$\text{ker}TS \subseteq \text{ker}S$\\
$\Longrightarrow \text{ker}TS = \text{ker}S$ \\
So $\text{nullity}TS = \text{nullity}S$  \\
$\Longrightarrow \text{rank}TS = \text{rank}S$

\item[(e)]
\item[(f)]
\item[(g)]
\end{enumerate}
\end{proof}

\subsection*{Inner Products and Norms}

\subsubsection*{Norms}

If $V$ real vector space, \\
\phantom{\quad } norm on $V$, $v \mapsto |v| \in \mathbb{R}$ s.t.
\begin{enumerate}
\item[(i)] \textsc{Positivity} $|v| \geq 0$, \, $\forall \, v \in V$, $|v| =0$ iff $v=0$
\item[(ii)] \textsc{Homogeneity} $|cv| = |c| |v|$ \quad \, $\forall \, c \in \mathbb{R}$, $v\in V$
\item[(iii)] \textsc{Triangle Inequality} $|v+w | \leq |v| + |w|$, \, $\forall \, v,w \in V$
\end{enumerate}

2 norms $| \, \cdot \, |_1$, $| \, \cdot \, |_2$ on vector space $V$ equivalent if $\exists \, $ constants $c,C >0$ s.t.
\[
c|v|_1 \leq |v|_2 \leq C|v|_1 \quad \quad \, \forall \, v \in V
\]

\exercisehead{B.49} $\forall \, x \in V$, \\
Consider $B_r(x) = \{ y \in V | |y-x|_2 < r \}$ \\
\phantom{ \quad } Note $y-x \in V$ as $V$ is a vector space

\[
\begin{gathered}
  c|y-x|_1 \leq |y-x|_2 \leq C |y-x|_1 \\
 c|y-x|_1 \leq |y-x_2| < r \\
 |y-x|_1 < \frac{r}{c}
\end{gathered}
\]
Now suppose $S \subseteq V$ open in $| \, \cdot \, |_2$ \\
\phantom{ \quad } But $S$ also open in $| \, \cdot \, |_1$ as $\exists \, \frac{r}{c'} >0$ s.t. $B_{\frac{r}{c'}}(x) \subseteq S$ for the exact same pts. as $S$ \\
so $| \, \cdot \, |_1, | \, \cdot \, |_2$ equivalent norms yield the same metric topology.

EY : 20141220

\subsection*{Direct Products and Direct Sums}


\section{Introduction}
	The aim of this project is to familiarize with different methods to solve an Asymmetric Travelling Salesman Problem.
	\begin{itemize}
		\item Exact method that gives the optimal solution. This is implemented with the CPLEX API.
		\item Meta heuristic methods: local search and tabu search.
	\end{itemize}

	\subsection{Problem}
		The combinatorial optimization problem was:
		\begin{quote}
				A company produces boards with holes used to build electric frames.  Boards are positioned over a machines and a drill moves over the board, stops at the desired positions and makes the holes.  Once a board is drilled, a new board is positioned and the process is iterated many times.  Given the position of the holes on the board, the company asks us to determine the hole sequence that minimizes the total drilling time, taking into account that the time needed for making an hole is the same and constant for all the holes.
		\end{quote}

		This problem can be modelled as an Asymmetric Travelling Salesman Problem. The salesman is represent by the drill and cities by the holes in the board.

	\subsection{Attachments}
		I attach to this report the folders:
		\begin{itemize}
			\item \verb|RealInstances|: it contains the \verb|.dat| and the information pdf of the real instances tested;
			\item \verb|Results|: it contains the pdf of all results taken, the most interesting data is included in this report. It should not be necessary to consult it, I attached them for completeness;
			\item \verb|Programs|: it contains the source code and executables of the two programs require by the assignment.
		\end{itemize}

	\subsection{System of testing}
		All the computation are be made on the following system:
		\begin{itemize}
			\item OS: Ubuntu 16.04 LTS (64 bit);
			\item Hardware: RAM 8 GB, CPU Intel Core i5-7500 3.40 GHz.
		\end{itemize}

	\subsection{Document structure}
		The first part of reports show how the exact method, the local search and the tabu search were been implemented with C++ code. In the second part it describes the test done and it shows the results obtained. There are two test sections, one with instances originated from random generator and one with real instances generated from real gerber (the standard format for represent PBCs) files.


\section{Evaluation}\label{sec:evaluation:krap}

Evaluating \krap{} requires an understanding of how well it classifies the \spec{} of an \isol{}.
There are a few areas of focus that we have when interpreting the results of \krap{}:
\begin{itemize}
\item What size $k$ achieves the best results?
\item What size $\alpha$ achieves the best results?
\item Which metric resolution algorithm achieves the best results?
\end{itemize}
Indeed we can define ``best'' in many ways, but we choose to look at two metrics, recall and precision, and a combination of the two, the $F$-measure. The metrics look at the accuracy of the classification on the object and the object on the classification respectively, while \fmeasure{} hopes to represent a balance between the two.
We test \krap{} by performing cross validation with holdout.

\subsection{Cross Validation with Holdout}
\index{cross validation with holdout}
To gauge the effectiveness of \krap{} at classifying the \spec{} of an \isol{}, we cross-validated against the library by separately holding out each \isol{} in CPLOP from CPLOP, classifying it against CPLOP, and verifying whether it is correct.
Since each \isol{} in CPLOP has the correct \spec{}, we know whether a classification is correct or not.

\subsection{Recall}
In our study, recall tracks how well we are able to discover all isolates from a given category, i.e. with a given host species.
Given a category (\spec{} name), the recall for that host species is the percentage of isolates taken from this host species that have been properly identified.
For example, if our database had 100 cat isolates, and 74 of them were classified by our method as having come from a cat, the recall would be 74\%.
In this study, we compute both overall recall (what percentage of \isols{} were classified as their proper \spec{} label) as well as \spec{}-level recall (what percentage of isolates that came from dogs/humans/sheep/etc. were classified
as their proper label).

\subsection{Precision}
Precision tracks how well our method avoids misclassification errors.
Given a category and a list of isolates our method classified as belonging to it, the precision of the method on the
category is the percent of isolates from the list that has the correct label.
For example, if our method returned 100 isolates labelled ``Dog'' of which 77 isolates really did come from dogs, the precision of the method is 77\%. As with recall, we compute both overall precision, as well as the precision for each category/species label.

\subsection{\textit{F}-Measure}
The \fmeasure{}, $F_1$, is the \textit{harmonic mean} of the precision, $P$ and the recall, $R$:
\begin{equation*}
    F_1
    =
    \frac{2}{\frac{1}{P}
    +
    \frac{1}{R}}
    = 2\cdot
    \frac{P\cdot R}
    {P + R}
\end{equation*}
While we prefer maximizing this value, a value near 0.5 means we are doing well.


%-----------------------------------------------------------------------------%
%                                                                             %
%    K A P I T E L   3                                                      %
%                                                                             %
%-----------------------------------------------------------------------------%

\chapter{Contact Stability Constrained DDP}\label{c3}
This chapter presents a generic method for integrating contact stability constraints into DDP-like solvers. The key idea is to define inequality constraints for unilaterality, friction and the \gls{CoP} of each contact surface with the goal of generating inherently balanced motions.

\section{The Idea}\label{sec:StabilityIdea}
Stability of the contacts is an essential objective of motion planning since prevents the robot from sliding and falling down. In \cref{sec:TheoryStability} we have explored two different criteria for ensuring contact stability for dynamic systems, namely the \gls{ZMP} and \gls{CoP}. As outlined, the application of the \gls{ZMP} is limited due to the assumptions of sufficiently high friction and the existence of one planar contact surface. Since we want to provide a \textit{generic} method that can also be used for e.g. walking up stairs, these simplifying assumptions do not hold anymore.

Consequently, we decide to model a 6D surface contact, as introduced in \cref{eqn:EoMLeggedRobotSurfaceContact} with dedicated constraints for (i) unilaterality of the contact forces (ii) Coulomb friction on the resultant force, and (iii) \gls{CoP} inside the support area. For the sake of simplicity, we model a rectangular contact area. Nevertheless, this concept can be extended  to arbitrary feet designs. This approach can be compared to the concept of contact wrench cone \cite{caron2015stability}, without additionally enforcing the yaw torque constraint. These inequality constraints for surface contacts can compactly be summarized as
\begin{subequations}\label{eqn:contractWrenchConeReduced}
\begin{align}
f_i^z &> 0 \label{subeqn:stabilityUnilaterality},\\
\mid f_i^x\mid &\leq \mu f_i^z \label{subeqn:stabilityFrictionX},\\
\mid f_i^y\mid &\leq \mu f_i^z \label{subeqn:stabilityFrictionY},\\
\mid X\mid & \geq C_x \label{subeqn:stabilityCoPPitch},\\
\mid Y\mid & \geq C_y \label{subeqn:stabilityCoPRoll}.
\end{align}
\end{subequations}
%Original CoP constraints from Caron paper for horizontal floor
%\mid \tau_i^x\mid & \leq Yf_i^z \label{subeqn:stabilityCoPPitch},\\
%\mid \tau_i^y\mid & \leq Xf_i^z \label{subeqn:stabilityCoPRoll}.

Let us now detail each line of the approach.
The first inequality \cref{subeqn:stabilityUnilaterality} accounts for the unilaterality of the contact force. By nature, contact forces always have to be positive since the robot can only \textit{push} from the ground, not \textit{pull} to the ground (\cref{img:simple_contact}).
Inequality \cref{subeqn:stabilityFrictionX,subeqn:stabilityFrictionY} corresponds to the Coulomb friction, where $\mu$ denotes the static coefficient of friction. From a modeling perspective, this can be interpreted via the concept of spatial friction cones \cite{kao2016contact}. If, and only if the distributed contact forces lie inside their respective friction cones, these constraints are satisfied.
Finally, inequality \cref{subeqn:stabilityCoPPitch,subeqn:stabilityCoPRoll} constrain the \gls{CoP} to lie inside the rectangular contact area of each foot (see \cref{img:contact_surface}). $C_x$ and $C_y$ denote the x and y position of $\bp_{CoP}$, respectively. These \gls{CoP} constraints prevent the robot from tilting around the edges of the rectangular surface contact. In particular, \cref{subeqn:stabilityCoPPitch} corresponds to a constraint of tilting around the pitch axis and \cref{subeqn:stabilityCoPRoll} prevents tilting around the roll axis.

Both, the unilaterality of the contact forces and the friction cone constraints, are already implemented inside Crocoddyl. However, the central component, bounding the \gls{CoP} to lie inside the support area of each contact foot, is missing. Therefore, the rest of this chapter deals with the derivation of a set of implementable \gls{CoP} constraints and describes the integration of these constraints as a cost function into the Crocoddyl framework.
\begin{figure}[t]
	\begin{subfigure}{.5\textwidth}
		\centering
		\includegraphics[width=.95\linewidth]{img/simple_contact}
		\caption{}
		\label{img:simple_contact}
	\end{subfigure}%
	\begin{subfigure}{.5\textwidth}
		\centering
		\includegraphics[width=.77\linewidth]{img/contact_surface}
		\caption{}
		\label{img:contact_surface}
	\end{subfigure}
	\caption[Simplified contact situation and CoP notation]{(a) Visualization of acting forces on a simple rigid body and (b) notation used for the \gls{CoP} definition in the contact surface plane \cite{caron2015stability}.}
	\label{fig:natural2robot}
\end{figure}

\section{Center of Pressure (CoP) Constraints}\label{sec:StabilityCoP}
In this section we will derive a universal set of implementable constraints that bound the \gls{CoP} to lie inside the rectangular contact area of each foot.

\subsection{CoP Stability Conditions}
Recapitulate the constraints from inequality \crefrange{subeqn:stabilityCoPPitch}{subeqn:stabilityCoPRoll}. Instead of using the absolute value of $X$ and $Y$, one can also formulate the constraints as
\begin{align}
\begin{split}
-X &\leq C_x \leq X,\\
-Y &\leq C_y \leq Y.
\end{split}
\end{align}
Based on this formulation it becomes evident that the \gls{CoP} is constrained to lie inside the foot geometry visualized in \cref{img:contact_surface}. In fact, these conditions can be represented via four single inequality equations as
\begin{align}\label{eqn:CoPInequalities}
\begin{split}
X + C_x \geq 0, \\
X - C_x \geq 0, \\
Y + C_y \geq 0, \\
Y - C_y \geq 0. \\
\end{split}
\end{align}
These four inequality equations will be used in the following to formulate the \gls{CoP} constraints.

\subsection{CoP Computation}
Our goal is to determine explicit expressions for $C_x$ and $C_y$ for arbitrary floor orientations, including inclined ground. To this end, consider the computation routine for the \gls{CoP} from \cref{eqn:CoPComputation}
\begin{equation*}
\bp_{CoP}=\dfrac{\bn\times\myM{\btau_O^c}}{\bfun^c\cdot\bn}.
\end{equation*}
For arbitrary orientations of the contact normal vector $\bn$, we obtain
\begin{equation}\label{eqn:CoPComputationDetailed}
\bp_{CoP}=\dfrac{\bn\times\myM{\btau_O^c}}{\bfun^c\cdot\bn} = \dfrac{\begin{bmatrix} n_x \\ n_y \\ n_z \end{bmatrix} \times \begin{bmatrix} t_x \\ t_y \\ t_z \end{bmatrix}}{\begin{bmatrix} f_x \\ f_y \\ f_z \end{bmatrix} \cdot \begin{bmatrix} n_x \\ n_y \\ n_z \end{bmatrix}} =
\begin{bmatrix} n_yt_z - n_zt_y \\ n_zt_x-n_xt_z \\ n_xt_y-n_yt_x \end{bmatrix}\cdot \dfrac{1}{f_xn_x+f_yn_y+f_zn_y},
\end{equation}
and solve for the desired position $C_x$ and $C_y$ of the \gls{CoP} as
\begin{subequations}
\begin{align}
C_x&=\dfrac{n_yt_z - n_zt_y}{f_xn_x+f_yn_y+f_zn_y}, \label{subeqn:Cx}\\
C_y&=\dfrac{n_zt_x-n_xt_z}{f_xn_x+f_yn_y+f_zn_y} \label{subeqn:Cy}.
\end{align}
\end{subequations}

\subsection{CoP Inequality Constraints}
Now that we have found explicit expressions for computing the \gls{CoP} (\crefrange{subeqn:Cx}{subeqn:Cy}), we can insert them into \cref{eqn:CoPInequalities}, which gives a set of four \gls{CoP} constraints as
\begin{align}\label{eqn:CoPInequalityEqs}
\begin{split}
X + \dfrac{n_yt_z - n_zt_y}{f_xn_x+f_yn_y+f_zn_y} \geq 0, \\
X - \dfrac{n_yt_z - n_zt_y}{f_xn_x+f_yn_y+f_zn_y} \geq 0, \\
Y + \dfrac{n_zt_x-n_xt_z}{f_xn_x+f_yn_y+f_zn_y} \geq 0, \\
Y - \dfrac{n_zt_x-n_xt_z}{f_xn_x+f_yn_y+f_zn_y} \geq 0. \\
\end{split}
\end{align}
These conditions can be written in matrix form as:
\begin{equation}\label{eqn:CoPInequalityMatrix}
\begin{bmatrix}
Xn_0 & Xn_1 & Xn_2 & 0 & -n_2 & n_1 \\
Xn_0 & Xn_1 & Xn_2 & 0 & n_2 & -n_1 \\
Yn_0 & Yn_1 & Yn_2 & n_2 & 0 & -n_0 \\
Yn_0 & Yn_1 & Yn_2 & -n_2 & 0 & n_0 \\ \end{bmatrix}
\begin{bmatrix} f^x \\ f^y \\ f^z \\ \tau^x \\ \tau^y \\ \tau^z \end{bmatrix} \geq
\begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \end{bmatrix},
\end{equation}
and finally yield an implementable set of inequality equations for constraining the \gls{CoP} to lie inside the rectangular contact area of each foot.

\section{Integration Into the Crocoddyl Framework}\label{sec:StabilityIntegration}
This section presents the integration of the derived CoP inequality constraints from \cref{eqn:CoPInequalityMatrix} into the Crocoddyl framework.

\subsection{Inequality Constraints by Penalization}
In \cref{sec:TheoryConstrainedDDP} we have discussed possible ways of incorporating inequality constraints into DDP-like solvers. Crocoddyl handles inequality constraints, such as joint limits or friction cone, via penalization.
In numerical optimization, the goal is to minimize a given cost function. In Crocoddyl, an \textit{action model} combines dynamics and cost model for each knot of the discretized \gls{OC} problem from \cref{eqn:totalCost}. The cost function for an action model at knot $n$ can be written as:
\begin{equation}\label{eqn:costSum}
l_n=\sum_{c=1}^{C}\alpha_c\Phi_c(\bq,\dot{\bq}, \btau),
\end{equation}
where $C$ different costs $\Phi_c$ are weighted by a respective coefficient $\alpha_c\in R$. The goal of the following two parts is to demonstrate how the inequality \cref{eqn:CoPInequalityMatrix} is implemented inside a novel cost function into the framework.

\subsection{Computation of Residual and Cost}
%\subsection{Computation of the Residual}
In numerical analysis, the term \textit{residual} corresponds to the error of a result \cite{shewchuk1994introduction}. For the sake of compactness, we abbreviate \cref{eqn:CoPInequalityMatrix} as
\begin{equation} \label{eqn:costPositive}
\myM{A} \myM{w} \geq \myM{0},
\end{equation}
where $\myM{A}$ corresponds to a matrix of \gls{CoP} inequality constraints and $\myM{w}$ is the contact wrench acting on the according foot. The residual $\myM{r}\in \myM{R}_{4\times 1}$ of the cost is retrieved by a simple matrix-vector multiplication:
\begin{equation}\label{eqn:costResidual}
\myM{r} = \myM{A} \myM{w}.
\end{equation}

%\subsection{Computation of the Cost}
The residual vector depicted in \cref{eqn:costResidual} typically contains non-zero numbers. The resulting scalar \gls{CoP} cost value $\Phi_{CoP}$ is computed via a bounded quadratic activation as
\begin{equation}\label{eqn:CoPCostComputation}
\Phi_{CoP}=
\begin{cases}
\quad\dfrac{1}{2}\myM{r}^T\myM{r} &\mid \text{lb} > \myM{r} > \text{ub} \\[10pt]
\quad 0 &\mid \text{lb} \leq \myM{r} \leq \text{ub}.
\end{cases}
\end{equation}

In order to account for the positiveness of $\myM{r}$ (see \cref{eqn:costPositive}), the bounds are set to $\text{lb}=\myM{0}$ and $\text{ub}=\infty$, respectively. Finally, this bounded quadratic activation of the residual vector has the following implications:
\begin{itemize}
\item The \gls{CoP} cost is zero, whenever $\bp_{CoP}$ lies inside or on the border of the foot area spanned by $X$ and $Y$.
\item The \gls{CoP} cost increases in a quadratic manner, when $\bp_{CoP}$ exceeds the foot area spanned by $X$ and $Y$.
\end{itemize}
Besides this formulation of the cost function also other designs are conceivable for the future. For example, the Euclidean distance from the CoP to the coordinate origin could be considered when computing the residual (see  \cref{eqn:CoPCostComputation}).

\subsection{Basic Usage and Contributions}
%\subsection{Basic Usage of the CoP Cost}
The cost function is implemented in C++, but can be accessed via Python bindings for versatile and fast prototyping. In the following, a basic example is provided to demonstrate the interface of the \gls{CoP} cost function to the interested reader.
\begin{verbatim}
# 1. Creating the cost model container
costModel = crocoddyl.CostModelSum(state, actuation.nu)
# 2. Defining the CoP cost
footGeometry = np.array([0.2, 0.08]) # dim [m] of the foot area
CoPCost = crocoddyl.CostModelContactCoPPosition(state,
crocoddyl.FrameCoPSupport(footId, footGeometry), actuation.nu)
# 3. Adding the CoP cost term with assigned weight to the cost model
costModel.addCost("LF_CoPCost", CoPCost, 1e3)
\end{verbatim}

%\subsection{List of Contributions}
The contributions of this thesis to the open-source framework Crocoddyl are summarized in two main pull requests. The first one, \href{https://github.com/loco-3d/crocoddyl/pull/792}{\#792} contains the basic formulation of the \gls{CoP} cost function for contact dynamics action models (see \cref{app:ContactCoP}). With \href{https://github.com/loco-3d/crocoddyl/pull/830}{\#830}, an additional version of the cost function is implemented for impulse dynamics action models (see \cref{app:ImpulseCoP}).
A functional unit test that checks the cost against numerical differentiation as well as accessible python bindings can be found in the according directory of \cite{crocoddylweb}.

%\subsection{Backup: Friction Cone constraints}
%\begin{align}
%\begin{split}
%\mid\mid f^x\mid\mid &\leq \mu f^z \\
%\mid\mid f^y\mid\mid &\leq \mu f^z \\
%f^z &> 0
%\end{split}
%\end{align}
%For the case of four edges of the linear approximation of the friction cone, the equations become:
%\begin{equation}
%\begin{bmatrix} 1 & 0 & -\mu \\
%-1 & 0 & -\mu \\
%0 & 1 & -\mu \\
%0 & -1 & -\mu \\
%0 & 0 & -\mu \\ \end{bmatrix} \cdot
%\begin{bmatrix} f^x \\ f^y \\ f^z \end{bmatrix} \leq
%\begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}
%\end{equation}

%\subsection{CoP Constraints: Horizontal Floor}
%For the special case of horizontal floor, with the according normal vector $\myM{n}=[0,0,1]$, the \gls{CoP} can be computed with the help of \cref{eqn:CoPComputationDetailed} to
%\begin{equation}
%\myM{p}_{CoP} = \begin{bmatrix} -t_y/f_z \\ t_x/f_z \\ 0 \end{bmatrix},
%\end{equation}
%which in turn can be represented by four inequality conditions:
%\begin{align}
%\begin{split}
%X-\dfrac{\tau_y}{f_z} &\geq 0, \\
%X+\dfrac{\tau_y}{f_z} &\geq 0, \\
%Y+\dfrac{\tau_x}{f_z} &\geq 0, \\
%Y-\dfrac{\tau_x}{f_z} &\geq 0. \\
%\end{split}
%\end{align}
%These conditions can be transformed into matrix form for the purpose of implementation as
%\begin{equation}
%\begin{bmatrix}
%0 & 0 & X & 0 & -1 & 0 \\
%0 & 0 & X & 0 & 1 & 0 \\
%0 & 0 & Y & 1 & 0 & 0 \\
%0 & 0 & Y & -1 & 0 & 0 \end{bmatrix} \cdot
%\begin{bmatrix} f^x \\ f^y \\ f^z \\ \tau^x \\ \tau^y \\ \tau^z \end{bmatrix} \geq
%\begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}.
%\end{equation}


\documentclass[11pt]{article}
\newcommand\tab[1][1cm]{\hspace*{#1}}
\usepackage{graphicx}
\graphicspath{ {C:/Users/yedkk/Desktop/CS465/hw4} }
\begin{document}
\section{Homework 9}
Name: Kangdong Yuan

\subsection{problem1}
a).I did not work in a group.
\\b).I did not consult without anyone my group members
\\c).I did not consult any non-class materials.

\subsection{problem2}
We prove by contradiction.\\
Given $e^*$ is the heaviest edge in some cycle C of undirected graph G.\\
We assume the claim is Flase, so T is a minimal spanning forest and $e^* \in T$. Let $S, \ V-S$ be the two connected components in $T \setminus \{e\}$. Since C is a cycle, there are two edges in C that cut $S, \ V-S$. We define another edge is r. The $e^*$ has more cost than edge r, which is $w(e)>w(r)$. So, the new minimal spanning forest with edge r cost less than minimal spanning forest with edge $e^*$. It is a contradiction to the minimal cost principle of minimal spanning forest.\\
So, $e^*$ cannot in minimal spanning forest, which cannot appear in any MST of G.

\subsection{problem3}
a). $f_a>f_b \ and \ f_a>f_c$, so $f_a=10, f_b=5, f_c=5$\\
b). This encoding is not possible, because the code for $a$ is $(0)$, is a prefix of the code for $c$ is $(00)$. \\
c). This encoding is not optimal, because the $\{0,10,11\}$ cost less space. And, the Huffman tree of this encoding is not complete, which is not optimal.

\subsection{problem4}
we can construct Huffman tree to find prefix-free encoding of minimal total cost.
As a common convention, bit '0' represents following the left child and bit '1' represents following the right child.\\
\\
First, using a priority queue $Q$ to store all the words as key and $f_i*c_i$ as value. In Q, lowest value is given highest priority. \\
\\
1. Create a leaf node for each symbol and add it to the	Q.\\
2. While there is more than one node in the queue:\\
\tab Remove the two nodes from Q (lowest $f_i*c_i$)\\
\tab Create a new internal node with these two nodes as children and with value ($f_i*c_i$) equal to the sum of the two nodes' value ($f_i*c_i$).\\
\tab Add the new node to the queue.\\
3. The remaining node is the root node and the tree is complete.\\

Then the prefix-free encoding of each word is path of each node. for example\\
\includegraphics[scale=0.7]{huff}\\
Time complexity: $O(nlogn)$, because each iteration requires $O(logn)$ time to determine the lowest frequencies and insert the new word in priority queue. There are $O(n)$ iterations.

\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{geometry}
\usepackage[usenames,dvipsnames,svgnames,table]{xcolor}
%\usepackage{color}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{mathabx}
\usepackage{commath}

\geometry{textwidth=6.5in, textheight=9.0in,
    marginparsep=7pt, marginparwidth=.6in}
\setlength{\parindent}{0in}
\setlength{\parskip}{0.08in}

\newcommand{\red}[1]{\textcolor{red}{#1}}
\newcommand{\blue}[1]{\textcolor{blue}{#1}}
\newcommand*{\annot}[1]{\tag*{\footnotesize{\textcolor{gray}{#1}}}}

\let\Re\undefined
\DeclareMathOperator{\Re}{Re}
\let\Im\undefined
\DeclareMathOperator{\Im}{Im}

\title{Color Gradients Notes}
\author{Josh Meyers}
\date{October 2015}

\begin{document}

\section{Introduction}

The goal is to create a \textsc{GalSim} ChromaticObject with realistic color gradients using high
resolution multiband images (most likely from HST) as input.  This is done by modeling the
preconvolution chromatic surface brightness profile $f(\vec{x}, \lambda)$ as a sum of two or more
separable chromatic surface brightness profiles, each with a particular asserted SED.  I.e.,

\begin{equation}
    \label{eqn:sbprof}
    f(\vec{x}, \lambda) = \sum_j S_j(\lambda) a_j(\vec{x}),
\end{equation}

where $S_j(\lambda)$ is the $j$th SED asserted as part of the decomposition, and $a_j(\vec{x})$ is
the spatial component of the $j$th separable chromatic profile.

The required set of inputs is:

\begin{itemize}

\item Two or more HST images of the same galaxy in different filters, $I_i(\vec{x})$, where $i$
labels the different filters.

\item The 2D noise covariance function for each image, $\xi_i(\Delta\vec{x})$.  If the noise is
uncorrelated, then this could be simply the noise variance ($\xi_i(\Delta\vec{x}=0) = \sigma^2_i$, $\xi_i(\Delta\vec{x}\ne0)=0$).

\item The chromatic HST PSF, $\Pi(\vec{x}, \lambda)$.

\item The HST throughput for each image filter, $T_i(\lambda)$.

\item Two or more SEDs, $S_j(\lambda)$, to use in the decomposition in Equation \ref{eqn:sbprof}.

\end{itemize}

The model for the observed images is:

\begin{align}
    I_i(\vec{x})
    &= \int T_i(\lambda) \left[\Pi(\vec{x}, \lambda) \Asterisk f(\vec{x}, \lambda)\right] \dif{\lambda} + \eta_i(\vec{x}) \\
    &= \int T_i(\lambda) \sum_j S_j(\lambda) \left[\Pi(\vec{x}, \lambda) \Asterisk a_j(\vec{x})\right] \dif{\lambda} + \eta_i(\vec{x})
\end{align}

where $\eta_i(\vec{x})$ indicates (potentially spatially correlated) Gaussian noise in image $i$ and
the $\Asterisk$ symbol indicates (spatial) convolution.  The noise $\eta_i(\vec{x})$ is related to
the noise covariance function via $\langle\eta_i(\vec{x}_l) \eta_i(\vec{x}_m)\rangle =
\xi_i(\vec{x}_l - \vec{x}_m)$, where angle brackets indicate averaging over realizations of the
noise.  Note that we also assume that $\xi_i$ is even: $\xi_i(\Delta\vec{x}) =
\xi_i(-\Delta\vec{x})$.

The convolution is easier to work with in Fourier space where it becomes a mode-by-mode product.
Indicating the Fourier transform of the real-space quantity $g(\vec{x})$ with $\tilde{g}(\vec{k})$,
the model in Fourier space is

\begin{align}
    \tilde{I}_i(\vec{k})
    &= \int T_i(\lambda) \sum_j S_j(\lambda) \tilde{\Pi}(\vec{k}, \lambda) \tilde{a}_j(\vec{k}) \dif{\lambda} + \tilde{\eta}(\vec{k}) \\
    &= \sum_j \left[\int T_i(\lambda) S_j(\lambda) \tilde{\Pi}(\vec{k}, \lambda) \dif{\lambda}\right] \tilde{a}_j(\vec{k}) + \tilde{\eta}(\vec{k}) \\
    \label{eqn:solveme}
    &=  \sum_j \tilde{\Pi}^\mathrm{eff}_{ij}(\vec{k}) \tilde{a}_j(\vec{k}) + \tilde{\eta}(\vec{k}),
\end{align}

where the effective PSF for the $i$th filter and $j$th SED is

\begin{equation}
  \tilde{\Pi}^\mathrm{eff}_{ij}(\vec{k}) = \int T_i(\lambda) S_j(\lambda) \tilde{\Pi}(\vec{k}, \lambda) \dif{\lambda}.
\end{equation}

The crux of the problem is to solve for the (complex-valued) $\tilde{a}_j(\vec{k})$ and to propagate
the statistics of the noise.

\section{Solving for $\tilde{a}_j(\vec{k})$}

There are several possible ways to estimate the required $\tilde{a}_j(\vec{k})$.

\subsection{No noise}

We'll look first at the case where one ignores noise entirely ($\eta(\vec{x}) =
\tilde{\eta}(\vec{k})= 0$).

If the number of asserted SEDs $N_j$ is larger than the number of input images $N_i$, then the
system of equations represented by Equation \ref{eqn:solveme} is underdetermined.  While in
principle this could still be solvable using some prior constraints, for now, we'll just ignore this
possibility.

If $N_j = N_i$, then we can exactly solve Equation \ref{eqn:solveme} using matrix inversion for each
Fourier mode $\vec{k}$:

\begin{equation}
  \tilde{a}_j(\vec{k}) = \sum_i [(\tilde{\Pi}^\mathrm{eff}(\vec{k}))^{-1}]_{ij} \tilde{I}_i(\vec{k}).
\end{equation}

If there are more images than spectra ($N_i > N_j$), then the system of equations is overdetermined
and no longer has an exact solution.  Instead, there is a (usually) unique ``least-squares''
solution for each $\vec{k}$ obtained by minimizing

\begin{equation}
  \label{eqn:lstsq}
  \sum_i\left|\tilde{I}_i(\vec{k}) - \sum_j \tilde{\Pi}^\mathrm{eff}_{ij}(\vec{k}) \tilde{a}_j(\vec{k})\right|^2.
\end{equation}

\subsection{Uncorrelated noise}

Stationary, uncorrelated Gaussian pixel noise in the $i$th image is completely described by its
variance $\sigma^2_i$.  In Fourier space, the variance of each mode is independent of $\vec{k}$ and
proportional to $\sigma^2_i$, where the constant of proportionality depends on the particular
Fourier conventions employed.  We can therefore write down a likelihood for $\tilde{a}_j(\vec{k})$
as

\begin{equation}
    \label{eqn:like}
    \chi^2(\vec{k}) = -2 \log \mathcal{L}(\vec{k}) = \sum_i\frac{1}{\sigma_i^2}\left|\tilde{I}_i(\vec{k}) - \sum_j \tilde{\Pi}^\mathrm{eff}_{ij}(\vec{k}) \tilde{a}_j(\vec{k})\right|^2.
\end{equation}

This is a \textit{weighted} least-squares problem.  Note that the weights only matter for
determining the Fourier coefficients $\tilde{a}_j(\vec{k})$ if the number of input images is greater
than the number of asserted spectra.  When these quantities are equal, it is always possible to find
$\tilde{a}_j(\vec{k})$ such that $\tilde{I}_i(\vec{k}) = \sum_j
\tilde{\Pi}^\mathrm{eff}_{ij}(\vec{k}) \tilde{a}_j(\vec{k})$.  The weights do still matter in this
case if we're interested in propagating the noise, however.

\subsection{Correlated noise}

If the noise covariance function for a particular image $\xi_i(\vec{\Delta x})$ is non-zero away
from the origin, then the variance of different Fourier modes is not constant, but proportional to
the noise power spectrum, which is the Fourier transform of the noise covariance function:

\begin{equation}
  P_i(\vec{k}) = \int \xi_i(\Delta \vec{x}) e^{-2 \pi i \vec{k}\cdot\Delta\vec{x}}.
\end{equation}

Due to assumed translation invariance of the noise (though note that we do not assume isotropy), the
covariance of different Fourier modes vanishes.  More explicitly, if the noise in image $i$ and mode
$\vec{k_l}$ is $\tilde\eta_i(\vec{k_l})$, then

\begin{equation}
  \langle\tilde\eta_i^*(\vec{k_l})\tilde\eta_i(\vec{k_m})\rangle = \delta(\vec{k_l} - \vec{k_m})P_i(\vec{k_l}).
\end{equation}

Once the noise power spectrum has been computed, therefore, the only change to the likelihood in
Equation \ref{eqn:like} is that the mode variance now depends on the particular Fourier mode in
question:

\begin{equation}
    \label{eqn:likecorr}
    -2 \log \mathcal{L}(\vec{k}) = \sum_i\frac{1}{P_i(\vec{k})}\left|\tilde{I}_i(\vec{k}) - \sum_j \tilde{\Pi}^\mathrm{eff}_{ij}(\vec{k}) \tilde{a}_j(\vec{k})\right|^2.
\end{equation}

This is essentially the same weighted least squares problem as in the uncorrelated noise case -- the
only difference being that the weights now depend on $\vec{k}$.  Written in matrix notation, the
solution for a particular $\vec{k}$ mode is:

\begin{equation}
    \tilde{a} = \left(\tilde{\Pi}^{\mathrm{eff}, \dagger} W \tilde{\Pi}^\mathrm{eff} \right)^{-1} \tilde{\Pi}^{\mathrm{eff}, \dagger} W \tilde{I}
\end{equation}

where
\begin{equation}
    W_{ij} = \delta_{ij}/\sqrt{P_i}.
\end{equation}

The covariance matrix for the elements of $\tilde{a}$ is given by
\begin{equation}
    \Sigma = \left(\tilde{\Pi}^{\mathrm{eff}, \dagger} W \tilde{\Pi}^\mathrm{eff} \right)^{-1}.
\end{equation}

\section{Propagating the noise covariance}

At this point, we have the $\tilde{a}_j(\vec{k})$ necessary to represent a chromatic surface
brightness profile in Fourier space as

\begin{equation}
  \tilde{f}(\vec{k}, \lambda) = \sum_j S_j(\lambda) \tilde{a}_j(\vec{k}).
\end{equation}

The uncertainty in each $\tilde{a}_j(\vec{k})$ is uncorrelated from one $\vec{k}$ to the next (since
we defined independent likelihoods for each $\vec{k}$), but does, in general, possess correlations
between the different SED components $j$ and $j^\prime$ for a given $\vec{k}$.  Using
$\Sigma_{jj^\prime}(\vec{k})$ to represent these covariances, we have

\begin{equation}
    \langle\delta\tilde{a}^*_j(\vec{k}_l)\delta\tilde{a}_{j^\prime}(\vec{k}_m)\rangle = \delta(\vec{k}_l - \vec{k}_m)\Sigma_{jj^\prime}(\vec{k_l}).
\end{equation}

Fortunately, $\Sigma_{jj^\prime}(\vec{k})$ is analytically computable for weighted least squares
problems.

This $\Sigma_{jj^\prime}(\vec{k})$ can then be transformed alongside the $\tilde{a}_j(\vec{k})$ to
effect shears, rotations, dilations, etc., the same way that noise is transformed in existing
\textsc{GalSim} routines.

Finally, the model for creating a (Fourier-domain) output image $\tilde{I}^\mathrm{out}(\vec{k})$,
convolving by an output chromatic PSF $\tilde{\Pi}^\mathrm{out}(\vec{k}, \lambda)$ and drawing
through an output filter with transmission $T^\mathrm{out}(\lambda)$, is

\begin{align}
    \tilde{I}^\mathrm{out}(\vec{k})
    &= \int T^\mathrm{out}(\lambda) \tilde{\Pi}^\mathrm{out}(\vec{k}, \lambda) \tilde{f}(\vec{k}, \lambda) \dif{\lambda} \\
    &= \int T^\mathrm{out}(\lambda) \tilde{\Pi}^\mathrm{out}(\vec{k}, \lambda) \sum_j S_j(\lambda) \tilde{a}_j(\vec{k}) \dif{\lambda} \\
    &= \sum_j \tilde{\Pi}_j^\mathrm{out, eff}(\vec{k}) \tilde{a}_j(\vec{k})
\end{align}

where the effective PSF for the $j$th component of the output image is

\begin{equation}
  \tilde{\Pi}_j^\mathrm{out, eff}(\vec{k}) = \int \tilde{\Pi}^\mathrm{out}(\vec{k}, \lambda) T^\mathrm{out}(\lambda) S_j(\lambda)\dif{\lambda}.
\end{equation}

Propagating the (potentially transformed) covariance spectrum $\Sigma_{jj^\prime}(\vec{k})$ into the
final output noise power spectrum $P^\mathrm{out}(\vec{k})$ then follows the normal propagation of
errors formula:

\begin{equation}
  P^\mathrm{out}(\vec{k}) = \sum_{jj^\prime} \tilde{\Pi}_j^\mathrm{out, eff, *}(\vec{k}) \Sigma_{jj^\prime}(\vec{k}) \tilde{\Pi}_{j^\prime}^\mathrm{out, eff}(\vec{k}).
\end{equation}

This power spectrum can then be used in the existing correlated noise whitening and symmetrizing
GalSim routines.

\section{Implementation notes}

\textsc{GalSim} uses the methodology developed by Bernstein \& Gruen (2013) to interpolate
discretely sampled surface brightness profiles in real and Fourier space.  Here we investigate the
impact of this interpolation on the Wiener-Khinchin theorem, which relates the real-space
autocorrelation function to the Fourier-space power spectrum.

The discrete version of the Wiener-Khinchin theorem is derived as follows.  We are interested in
the (co)-variance of the Discrete Fourier Transform amplitudes of some discrete real space samples
with discrete auto-covariance function $\xi\left[r\right]$ (we follow the convention of
Rowe++14 in using square brackets to indicate the arguments of a discretly sampled function, and
reserve parentheses to indicate arguments of continuous objects).  Note that since we're using DFTs,
$\xi$ is implicitly periodic: $\xi\left[r\right] = \xi\left[r+N\right]$.

The 1D DFT of an N-point sampled function is:

\begin{equation}
    \tilde{b}_k = \sum_{j=-N/2}^{N/2-1} b_j e^{-2 \pi i j k / N}.
\end{equation}

% with inverse transform
%
% \begin{equation}
%     b_j = \frac{1}{N}\sum_{k=-N/2}^{N/2-1} \tilde{b}_k e^{2 \pi i j k / N}.
% \end{equation}

We are interested in the quantity $\left\langle\tilde{b}^*_k \tilde{b}_{k^\prime}\right\rangle$
where the angle brackets indicate averaging over noise realizations.

\begin{align}
    \left\langle\tilde{b}^*_k \tilde{b}_{k^\prime}\right\rangle
    \annot{Sub in FT definition}
    &= \left\langle\sum_{j=-N/2}^{N/2-1} b_j e^{+2 \pi i j k / N} \sum_{j^\prime=-N/2}^{N/2-1} b_{j^\prime} e^{-2 \pi i j^\prime k^\prime / N}\right\rangle \\
    \annot{Interchange expectation and summation}
    &= \sum_{j=-N/2}^{N/2-1} \sum_{j^\prime=-N/2}^{N/2-1} \left\langle b_j b_{j^\prime} \right\rangle e^{2 \pi i (jk - j^\prime k^\prime)/N} \\
    \annot{$\xi\left[r\right]$ definition}
    &= \sum_{j=-N/2}^{N/2-1} \sum_{j^\prime=-N/2}^{N/2-1} \xi\left[j-j^\prime\right] e^{2 \pi i (jk - j^\prime k^\prime)/N} \\
    \annot{change variables $r=j^\prime-j$}
    &= \sum_{j=-N/2}^{N/2-1} \sum_{r=-N/2-j}^{N/2-1-j} \xi\left[r\right] e^{2 \pi i (jk - (r+j) k^\prime)/N} \\
    \annot{simplify using periodicty of $\xi$ and $\exp$}
    &= \sum_{j=-N/2}^{N/2-1} \sum_{r=-N/2}^{N/2-1} \xi\left[r\right] e^{-2 \pi i r k^\prime/N} e^{2 \pi i j (k-k^\prime)/N} \\
    \annot{definition of $P\left[k\right]$}
    &= P[k^\prime] \sum_{j=-N/2}^{N/2-1} e^{2 \pi i j (k-k^\prime)/N} \\
    \annot{$e^{2 \pi i j (k-k^\prime)/N}$ evenly samples unit circle unless $k-k^\prime \in N\mathbb{Z}$...}
    &= N P[k^\prime] \sum_{N=0}^\infty\delta^{k+N}_{k^\prime} \\
    \annot{...but we really only care about $k \in \left[-N/2, N/2-1 \right]$ }
    &= N P[k^\prime] \delta^{k}_{k^\prime}.
\end{align}

Now we try the same thing using the Bernstein \& Gruen continuously interpolated Fourier transform.
Recall their result:

\begin{align}
    \tilde{F}(u)
    &= \int{F(x) e^{-2 \pi i u x} \dif{x}} \\
    &\approx \tilde{K}_x(u) \sum_{k=-N/2}^{N/2-1}\tilde{b}_k K_u(u-k/N)
\end{align}

where $K_x$ is an asserted real-space interpolant, $\tilde{K}_x$ is its Fourier transform,
and $K_u$ is an asserted Fourier-space interpolant.  Note that the final equality is exact if and
only if

\begin{equation}
    K_u(v) = e^{i \pi v} \frac{\mathrm{sinc}\, N v}{\mathrm{sinc}\, v}.
\end{equation}

We are interested in the (co)-variance of Fourier amplitudes:

\begin{align}
    \left\langle \tilde{F}^*(u) \tilde{F}(u^\prime)\right\rangle
    \annot{Sub in GalSim approximate FT definition}
    &= \left\langle \tilde{K}_x^*(u) \sum_{k=-N/2}^{N/2-1}\tilde{b}_k^* K_u^*(u-k/N) \tilde{K}_x(u^\prime) \sum_{k^\prime=-N/2}^{N/2-1}\tilde{b}_{k^\prime} K_u(u^\prime-k^\prime/N)  \right\rangle \\
    \annot{Rearrange}
    &= \tilde{K}_x^*(u) \tilde{K}_x(u^\prime) \sum_{k=-N/2}^{N/2-1} \sum_{k^\prime=-N/2}^{N/2-1} K_u^*(u-k/N) K_u(u^\prime-k^\prime/N) \left\langle \tilde{b}_k^* \tilde{b}_{k^\prime} \right\rangle \\
    \annot{definition of $P\left[k\right]$}
    &= \tilde{K}_x^*(u) \tilde{K}_x(u^\prime) \sum_{k=-N/2}^{N/2-1} \sum_{k^\prime=-N/2}^{N/2-1} K_u^*(u-k/N) K_u(u^\prime-k^\prime/N) N P[k^\prime]\delta_{k^\prime}^k \\
    \annot{sum over Kronecker $\delta$}
    &= N \tilde{K}_x^*(u) \tilde{K}_x(u^\prime) \sum_{k=-N/2}^{N/2-1} K_u^*(u-k/N) K_u(u^\prime-k/N) P[k]
\end{align}

This is as far as I've been able to push (ran out of $\delta$ functions!).  Assuming there are no
further simplifications, one implication is that Fourier amplitudes of an interpolated noise image
are not uncorrelated.  Equivalently, the continuous auto-covariance function of interpolated
discretely stationary noise is not itself stationary.  This makes some intuitive sense, I think.
For example, the variance of interpolated points seems like it should always be less than the
variance of the input samples, implying that the continuously regarded noise is not stationary.

\end{document}


\chapter{Creating Spacecraft}

Epoch, State, Ballistic and Mass Properties, Input Coordinate
System,

\noindent\begin{ScriptType} \noindent
Create Spacecraft Sat1\\
Sat1.Epoch.TAIGregorian  = 01 Jan 2000\\
12:00:00.000; \\
Sat1.SMA        = 8000; \\
Sat1.ECC        = .01;\\
Sat1.INC        = 28.5;\\
Sat1.AOP        = 0;\\
Sat1.RAAN = 90;\\
Sat1.TA         = 180;\\     %  Use M for mean anomaly, and E for eccentric anomaly
Sat1.Cd         = 2.0;\\
Sat1.Cr         = 1.4;\\
Sat1.DragArea   = 1;\\
Sat1.SRPArea    = 1;\\
Sat1.DryMass    = 100;
\end{ScriptType}

\section{Spacecraft Epoch}

Alternative ways of Defining a Spacecraft's Epoch

Assuming a reference reference epoch for modified JD as MJD = JD -
2430000, the following epochs are all equivalent.

\noindent \% TAI in Modified Julian Date Format\\
 \begin{ScriptType}
Sat1.Epoch.TAIModJulian = 21545.0003703704 \end{ScriptType}

\noindent \% TAI in Gregorian Date Format\\
\begin{ScriptType} Sat1.Epoch.TAIGregorian  = 01 Jan
2000 12:00:32.0000;\end{ScriptType}

\noindent \% UTC in Modified Julian Date Format\\
  \begin{ScriptType}
 Sat1.Epoch.UTCModJulian  =
21545.00000; \end{ScriptType}

\noindent \% UTC in Gregorian Date Format\\
\begin{ScriptType} Sat1.Epoch.UTCGregorian  = 01 Jan
2000 12:00:00.000;  \end{ScriptType}

\section{Spacecraft Orbit State}

\%  Example: Creating a spacecraft using  the
Cartesian State\\
\begin{ScriptType}
Create Spacecraft Sat1;\\\
Sat1.X = 7082.960306079;\\
Sat1.Y = 7.052885901083;\\
Sat1.Z = -48.94363747128;\\
Sat1.VX = 0.05237087208446;\\
Sat1.VY = -1.069925309259;\\
Sat1.VZ = 7.424767278548;
\end{ScriptType}

\%  Example: Cartesian State: Method 2 \\
\begin{ScriptType}
Create Spacecraft Sat1;\\
Sat1.StateType = Cartesian;\\
Sat1.Element1 = 7000; \\   % X
Sat1.Element2 = 0.0;  \\  % Y
Sat1.Element3 = 0.0;  \\   % Z
Sat1.Element4 = 0.0;  \\   % VX
Sat1.Element5 = 8.0;  \\   % VY
Sat1.Element6 = 0.0;  \\   % VZ
\end{ScriptType}

\noindent \%  Example: Creating a spacecraft using  the
Keplerian Elements\\
\begin{ScriptType}
Create Spacecraft Sat1;\\
Sat1.SMA        = 8000;\\
Sat1.ECC        = .01;\\
Sat1.INC        = 28.5;\\
Sat1.AOP = 0; \\
Sat1.RAAN = 90;\\
Sat1.TA         = 180; \\    %  Use M for mean anomaly, and E for eccentric anomaly
\end{ScriptType}

\noindent \%  Keplerian Elements: Method 2\\
\begin{ScriptType}
Create Spacecraft Sat1;\\
Sat1.StateType   = Keplerian;\\
Sat1.AnomalyType = TA; \\   %  Alternatives:  MA, EA
Sat1.Element1 = 8000;   \\  % SMA
Sat1.Element2 = .01;   \\  % ECC
Sat1.Element3 = 28.5;  \\   % INC
Sat1.Element4 = 0;     \\   % RAAN
Sat1.Element5 = 90;    \\   % AOP
Sat1.Element6 = 180;   \\   % Anomaly
\end{ScriptType}


\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}	% Para caracteres en español
\usepackage{amsmath,amsthm,amsfonts,amssymb,amscd}
\usepackage{multirow,booktabs}
\usepackage[table]{xcolor}
\usepackage{fullpage}
\usepackage{lastpage}
\usepackage{enumitem}
\usepackage{fancyhdr}
\usepackage{mathrsfs}
\usepackage{wrapfig}
\usepackage{setspace}
\usepackage{url}
\usepackage{calc}
\usepackage{multicol}
\usepackage{cancel}
\usepackage[retainorgcmds]{IEEEtrantools}
\usepackage[margin=3cm]{geometry}
\usepackage{amsmath}
\newlength{\tabcont}
\setlength{\parindent}{0.0in}
\setlength{\parskip}{0.05in}
\usepackage{empheq}
\usepackage{framed}
\usepackage[most]{tcolorbox}
\usepackage{xcolor}
\colorlet{shadecolor}{orange!15}
\parindent 0in
\parskip 12pt
\geometry{margin=1in, headsep=0.25in}
\theoremstyle{definition}
\newtheorem{defn}{Definition}
\newtheorem{reg}{Rule}
\newtheorem{exer}{Exercise}
\newtheorem{note}{Note}

%\newif\ifnotes
%\newcommand{\h}[1]{\ifnotes{\textcolor{red}{#1}}\fi}
\newcommand{\h}[1]{\textcolor{red}{#1}}

\begin{document}
\setcounter{section}{8}
\title{Chapter 9 Review Notes}

\thispagestyle{empty}

\begin{center}
{\LARGE \bf Convexity of Differentiable Function Approximations}\\
\end{center}
\section{Convexity Reference}

\subsection{Definitions}

There are several definitions and properties of convexity which are useful for determining where functions are convex.

\textbf{Definition.} A function $f(x)$ is \textit{convex} on interval $I$ if $\forall x_1, x_2 \in I, \forall \in [0, 1]: f(t x_1 + (1-t)x_2) \leq tf(x_1) + (1-t)f(x_2)$.

\textbf{First Derivative} Using the first derivative this becomes: $\forall x_1, x_2 \in I: f(x_1) \geq f(x_2) + f'(x_2)*(x_1 - x_2)$. In other words, the derivative cannot decrease on $I$.

\textbf{Second Derivative} This also means that $\forall x \in I: f''(x) \geq 0$

\subsection{Compositions}

\textbf{Summation and Multiplication} Sums of convex functions are convex. Multiplication by a constant is convex, provided the constant is greater than 0.

\textbf{Composition} If $f$ and $g$ are convex, $g \circ f$ is convex on intervals where $g' \geq 0$, (ie where $g$ is monotonically increasing).

\begin{figure}[h!]
  \centering
  %\includegraphics[width=\linewidth]{figs/mlp2.png}
  %\includegraphics[height=6cm]{figs/mlp2.png}
  \includegraphics[width=\linewidth]{figs/convex_examples.png}
  \caption{Some examples of convex and concave functions. (from \protect\url{http://www.math.cmu.edu/~lohp/docs/math/mop2013/convexity-soln.pdf})}
  %\label{fig:mlp2}
\end{figure}

\section{Convexity of Integer Operations}
\subsection{Arithmetic Operations}

\textbf{Addition}.
\begin{enumerate}
\item \textbf{Function:} $g(x) = x + c$ is trivially convex, since $g'(x) = 1$. ,
\item \textbf{Composition:}  $g \circ f$ is also convex given convex $f$, since $\forall x: g'(x) \geq 1$.
\item \textbf{Binary Composition:} For addition given two convex input functions $f_1$ and $f_2$: $g(f_1(x), f_2(x)) = f_1(x) + f_2(x)$. This is still convex, since $\frac{dg}{df_1} = \frac{dg}{df_2} = 1$.
\item \textbf{Step size:} Step size does not affect convexity for additon ($Ax + b$ is an affine tranformation).
\end{enumerate}

\textbf{Subtraction}.
\begin{enumerate}
\item \textbf{Function:} $g(x) = x - c$ is both trivially convex, since $g'(x) = 1$. ,
\item \textbf{Composition:}  $g \circ f$ is also convex given convex $f$, since $\forall x: g'(x) \geq 1$.
\item \textbf{Binary Composition:} This is still convex, since $\frac{dg}{df_1} = \frac{dg}{df_2} = 1$. For subtraction $g(f_1(x), f_2(x)) = f_1(x) - f_2(x)$, $g''(f_1(x), f_2(x)) = f_1''(x) - f_2''(x)$, this will be convex where $f_1''(x) \geq f_2''(x)$. For example, for $f_1(x) = x^2$, $f_2(x) = x^3$, $g(x)$ will be convex where the 2nd derivative $2 - 6x \geq 0$, ($x \leq \frac{1}{3}$).
\item \textbf{Binary Composition with bounds:} Given a particular interval $x \in I$ in which $f_1''(x) \in [d_{1low}, d_{1high}]$ and $f_2''(x) \in [d_{2low}, d_{2high}]$, $g(x)$ will be convex if $d_{1low} - d_{2high} \geq 0$, since this means $g''(x) \geq 0$ for $x \in I$.
\item \textbf{Step size:} Does not effect single function composition, for multiple function composition $x$ becomes $\frac{x}{s}$ (for step size $s$), which may effect the regions where $g(x)$ is convex depending on the input functions.
\end{enumerate}

\textbf{Multiplication}
\begin{enumerate}
\item \textbf{Function:} $g(x) = cx$  is trivially convex, since $g''(x) = 0$. ,
\item \textbf{Composition:}  $g \circ f$ is convex for convex $f$ when $c \geq 0$, since $g'(x) = c$. For concave $f$, the inverse holds. If $f''(x)$ is bounded, this simply means the lower bound must greater than 0 if $c>0$, or the upper bound must be less than 0 if $c<0$.
\item \textbf{Binary Composition:} For multiplication given two convex input functions $f_1$ and $f_2$: $g(f_1(x), f_2(x)) = f_1(x) * f_2(x)$. This is not convex in general, counterexample $g(-x, x) = -x^2$, but can be convex or convex in regions.
\item \textbf{Binary Composition with Bounds:} \h{Bounds don't seem to help much here, it seems you need more information about the functions. Example $g(x^2, -(x-2))$ is convex for $x<\frac{2}{3}$, how do you predict that from bounds on derivatives?}

\item \textbf{Step size:} $x = \frac{x}{s}$, does not effect single function composition.
\end{enumerate}

\textbf{Division}
\begin{enumerate}
\item \textbf{Function:} $g(x) = x/c$  is trivially convex, since $g''(x) = 0$. For $g(x) = c/x$, $g''(x) = 2cx^{-3}$, this is convex for $x \in [0, \inf), c \geq 0$, and $x \in (-\inf, 0), c < 0$. (note, may need to adjust this for integer division)
\item \textbf{Composition:} For $g(x) = x/c$, $g \circ f$ is convex when $c \geq 0$, since $g'(x) = c^{-1}$.  For $g(x) = c/x$, $g'(x) = -cx^{-2}$, then $\forall c < 0: g'(x) \geq 0$, and it is convex under composition for all $f(x)<0$.
\item \textbf{Binary Composition:} Given two convex input functions $f_1$ and $f_2$: $g(f_1(x), f_2(x)) = f_1(x) / f_2(x)$. This is not convex in general, counterexample $g(x, x^2) = 1/x$, inputs are convex but output is concave, however it can still be convex or convex in regions.
\item \textbf{Binary Composition with Bounds:} \h{Similar issues to multiplication here.}
\item \textbf{Step size:} $x = \frac{x}{s}$, does not effect single function composition.
\end{enumerate}

\textbf{Exponentiation}
\begin{enumerate}
\item \textbf{Function:} $g(x) = x^c$  is convex if $c \mod 2 = 0$, since $g''(x) = (c-1)(c-2)x^{c-2} \geq 0$ if $c$ is even. If $c$ is odd, $g''(x) \geq 0$ for $x > 0$, so the function will be convex for $x>0$. ($x^1$ is the exception is convex everywhere).  For $g(x) = c^x$, it is convex for $c>0$ since $g''(x) = c^x \log^2(x)$. For $c<0$, the sign alternates so the function is convex over every interval $(x, x+2)$.

\item \textbf{Composition:} For $g(x) = x^c$, $g \circ f$ has second derivative $g''(x) = c(c-1) (f(x))^{c-2}f'(x) + cf(x)^{c-1}f''(x)'$.
\item \textbf{Binary Composition:} For given two convex input functions $f_1$ and $f_2$: $g(f_1(x), f_2(x)) = f_1(x) ^ f_2(x)$. This does not preserve convexity in general, counterexample $g(-x, x) = -x^2$, inputs are convex but output is concave.
\item \textbf{Step size:}  $x = \frac{x}{s}$, does not effect single function composition, where $g(x) = f(x)^c$, but renders analysis for $g(x) = c^{f(x)}$ invalid.
\end{enumerate}

\subsection{Bitwise Operations}

%\textbf{And}
\subsubsection*{And}
\begin{enumerate}
\item \textbf{Function:} For $g(x) = x \& c$, $g(x)$ will be convex in the following regions: let $k$ denote the number of contiguous leading bits in $c$ that share the same value, either 0 or 1. Then $g(x)$ will be convex where $x \in [x_k, x_k + 2^k - 1], x_k \mod 2^k = 0$.

{\it Proof.}

{\it Lemma 1} Only the first $k$ bits of $x$ change in the region $x \in [x_k, x_k + 2^k - 1] x_k \mod 2^k = 0$. Pf. The first $k$ bits of $x_k$ are 0 b/c $x_k \mod 2^k = 0$. Since the maximum in the region $x \in [x_k, x_k + 2^k - 1] x_k \mod 2^k = 0$ from $x_k$ is $2^k-1$, which is also represented in $k$ 1s, all changes in the region from $x_k$ only effect the first $k$ bits.

There are two cases, either the first $k$ bits are 0s or they are 1s. If they are 0s, then $g(x)$ will not change in the region $[x_k, x_k = 2^k-1]$ because from \textbf{lemma 1} only the first $k$ bits will change and the first $k$ bits are being bitwise $\&$ with $k$ 0s. Then $g(x) = g(x_k)$ in the region, and the definition of convexity applies. If the first $k$ bits are 1s, then from \textbf{lemma 1} any change in $x$ is anded with 1s, and $g(x) = g(x_k) + x - x_k$, which is linear and therefore a convex function.

Also note that for leading 1s, in some places the convexity region can be extended to include $x_k-1$: $x \in [x_k - 1, x_k + 2^k - 1] x_k \mod 2^k = 0$, while for the leading 0s, in some places the upper bound could be extended to $x_k + 2^k$. However these cases do not apply to all regions.

\item \textbf{Composition:} For $g \circ f$, the previous analysis holds when the leading bits are 0s for $f(x) \in [f(x)_k, f(x)_k + 2^k - 1] f(x)_k \mod 2^k = 0$, since $g(x)$ will still be constant in those regions. If the leading bits are 1s, then $g(x)$ is equivalent to multiplication by $1$ of $-1$, and convexity will be preserved if $f''(x) > 0$, or inverted if $f''(x) < 0$.
\item \textbf{Output bounds:} For leading 1s, bounds are limited to bounds in the convex region defined by the bitwise operation, or could remain the same if evaluating the input 2nd derivative in the region for tighter bounds is not feasible. For leading 0s, the 2nd derivative is set to 0 within the flat (convex) intervals.
\item \textbf{Step size:} If $s < 2^k$, then step size shrinks the convexity bounds to $\frac{2^k}{s}$, for $s > 2^k$, the input will jump between convex regions on changes in $x$.
\end{enumerate}

\textbf{Composition} If $f$ and $g$ are convex, $g \circ f$ is convex on intervals where $g' \geq 0$, (ie where $g$ is monotonically increasing).

\begin{figure}[h!]
  \centering
  \includegraphics[width=\linewidth]{figs/and.png}
  \caption{Convex regions for bitwise \textbf{and}}
\end{figure}

%\textbf{Or}
\subsubsection*{Or}
\begin{enumerate}
\item \textbf{Function:}  For $g(x) = x | c$, $g(x)$ will be convex in the following regions: let $k$ denote the number of contiguous leading bits in $c$ that share the same value, either 0 or 1. Then $g(x)$ will be convex where $x \in [x_k, x_k + 2^k - 1], x_k \mod 2^k = 0$.

{\it Proof.} There are two cases, either the first $k$ bits are 0s or they are 1s. If they are 1s, then $g(x)$ will not change in the region $[x_k, x_k = 2^k-1]$ because from \textbf{lemma 1} only the first $k$ bits will change and the first $k$ bits are being bitwise $|$ with $k$ 1s, setting them all to 1. Then $g(x) = g(x_k)$ in the region, and the definition of convexity applies. If the first $k$ bits are 0s, then from \textbf{lemma 1} any change in $x$ is bitwise \textbf{or}ed with 1s, and $g(x) = g(x_k) + x - x_k$, which is linear and therefore a convex function.

\item \textbf{Composition:} For $g \circ f$, the previous analysis holds when the leading bits are 1s for $f(x) \in [f(x)_k, f(x)_k + 2^k - 1] f(x)_k \mod 2^k = 0$, since $g(x)$ will still be constant in those regions. If the leading bits are 0s, then $g(x)$ is equivalent to multiplication by $1$ of $-1$, and convexity will be preserved if $c > 0$, or inverted if $c < 0$.
\item \textbf{Step size:} If $s < 2^k$, then step size shrinks the convexity bounds to $\frac{2^k}{s}$, for $s > 2^k$, the input will jump between convex regions on changes in $x$.
\item \textbf{Output bounds:} For leading 0s, bounds are limited to bounds in the convex region defined by the bitwise operation, or could remain the same if evaluating the input 2nd derivative in the region for tighter bounds is not feasible. For leading 1s, the 2nd derivative is set to 0 within the flat (convex) intervals.
\end{enumerate}

\begin{figure}[h!]
  \centering
  \includegraphics[width=\linewidth]{figs/or.png}
  \caption{Convex regions for bitwise \textbf{or}}
\end{figure}

%\textbf{Not}
\subsubsection*{Not}
\begin{enumerate}
\item \textbf{Function:} For $g(x) = ~x$, this is equivalent to $g(x) = -x - 1$, and as a linear function is convex.
\item \textbf{Composition:} For $g \circ f$, $f$ will be inverted so that concave regions become convex.
\end{enumerate}

%\textbf{Xor}
\subsubsection*{Xor}
\begin{enumerate}
\item \textbf{Function:} For $g(x) = x \oplus c$, $g(x)$ will be convex in the following regions: let $k$ denote the number of contiguous leading bits in $c$ that share the same value, either 0 or 1. Then $g(x)$ will be convex where $x \in [x_k, x_k + 2^k - 1], x_k \mod 2^k = 0$.

{\it Proof.} There are two cases, either the first $k$ bits are 0s or they are 1s. If they are 0s, then $g(x) = g(x_k) + x - x_k$  in the region $[x_k, x_k = 2^k-1]$ because from \textbf{lemma 1} only the first $k$ bits will change and the first $k$ bits are being bitwise $\oplus$ with $k$ 0s, leaving them unchanged, and the definition of convexity applies. If the first $k$ bits are 1s, then from \textbf{lemma 1} any change in $x$ is bitwise $\oplus$ed with 1s, and $g(x) = g(x_k) - x + x_k$, which is also linear and therefore a convex function.
\item \textbf{Composition:} For $g \circ f$, $f$ will be unchanged in the convex regions if there are leading 0s, preserving convexity in these regions, or inverted if there are leading 1s, making any concave parts of the function convex.
\item \textbf{Step size:} If $s < 2^k$, then step size shrinks the convexity bounds to $\frac{2^k}{s}$, for $s > 2^k$, the input will jump between convex regions on changes in $x$.
\item \textbf{Output bounds:} For leading 1s, bounds are inverted in the convex region defined by the bitwise operation. For leading 1s, bounds remain the same, or could be tightened to the range within the convex region of the xor.
\end{enumerate}

\begin{figure}[h!]
  \centering
  \includegraphics[width=\linewidth]{figs/xor.png}
  \caption{Convex regions for bitwise \textbf{xor}}
\end{figure}
%\begin{figure}
  %\centering
%\begin{subfigure}{.5\textwidth}
  %\centering
  %%\includegraphics[width=\linewidth]{figs/mlp2.png}
  %\includegraphics[height=6cm]{figs/mlp2.png}
  %\caption{Multilayer Perceptron with 2 hidden layers.}
%\end{subfigure}%
%\begin{subfigure}{.5\textwidth}
  %\centering
  %\includegraphics[height=6.5cm]{figs/mlp3.png}
  %\caption{Multilayer Perceptron with 2 hidden layers and residual connections.}
%\end{subfigure}
  %\caption{Two Layer MLPs.}
  %\label{fig:mlp2}
%\end{figure}

\end{document}