\documentclass[simplex.tex]{subfiles}
% NO NEED TO INPUT PREAMBLES HERE
% packages are inherited; you can compile this on its own

\onlyinsubfile{
\title{NeuroData SIMPLEX Report: Subfile}
}

\begin{document}
\onlyinsubfile{
\maketitle
\thispagestyle{empty}

The following report documents the progress made by the labs of Randal~Burns and Joshua~T.~Vogelstein at Johns Hopkins University towards goals set by the DARPA SIMPLEX grant.

%%%% Table of Contents
\tableofcontents

%%%% Publications
\bibliographystyle{IEEEtran}
\begin{spacing}{0.5}
\section*{Publications, Presentations, and Talks}
%\vspace{-20pt}
\nocite{*}
{\footnotesize	\bibliography{simplex}}
\end{spacing}
%%%% End Publications
}

\subsection{Multiscale Generalized Correlation (MGC)}

MGC is the optimal local correlation between two datasets  $X$ and $Y$.
For any given global correlation (Pearson’s, rank, Mantel, distance
correlation, etc.), their respective local correlations can be
efficiently computed.  By choosing the optimal local correlation based
on maximizing testing powers, the Oracle MGC dominates the global
correlation.

We demonstrate that Oracle MGC is a consistent test statistic (power
converge to 1 as sample size increases) under standard regularity
conditions, is equivalently to the global correlation under linear
dependency (i.e., each observation $X_i$ is a linear transformation of
$Y_i$), and can be strictly better than the global correlation under
common nonlinear dependencies. Thus Oracle MGC dominates the global
correlation, and the sample MGC (i.e., choose the optimal scale by
p-value map approximation, as the testing power are not available in the
absence of the true model and training data) also empirically dominates
the global correlation.

Numerically, we showed that both Oracle and sample MGC significantly
improve over the global correlation and other existing state-of-the-art
methods for the dependence test. Moreover, the optimal scale helps
discovering the nature of the dependency, i.e., the global scale is
close to optimal in the power / p-value map if and only if the
underlying dependency is close to linear.

On real data, MGC helps identify useful relationships between brain
activity vs personality, brain hippocampus vs major depressive disorder,
which was confirmed by domain experts but not detected on raw data by
existing statistical methods.

\subsubsection{Testing Node Contribution via (MGC)} We continue the
development of the Multiscale Generalized Correlation toolbox, to handle
the node contribution task within the MGC testing framework.

MGC is the optimal local correlation between two datasets $X$ and $Y$.
For any given global correlation (Pearson’s, rank, Mantel, distance
correlation, etc.), their respective local correlations can be
efficiently computed. By choosing the optimal local correlation based on
maximizing testing powers, the Oracle MGC is a consistent test statistic
that dominates the global correlation.

An important question useful for subsampling and outlier detection, is
how important is each sample observation, regarding their relative
contribution to the underlying dependency. If this question can be
successfully and efficiently answered, those important samples can be
kept for later inference, while less important samples may be treated as
outliers.

By decomposing the optimal local correlation $C*$ into each sample
point, the MGC computation readily offers a weight statistic $w_i$ for
each pair of observations $(X_i,Y_i)$ where $\sum_i{w_i} = C*$.  Then
each sample can be ranked based on $W_i$.  This is a very efficient
algorithm, since computing the MGC statistic for all data automatically
yields $w_i$ for all sample pairs.

\begin{figure}[h!]
\begin{cframed}
\centering
\includegraphics[width=0.95\textwidth]{./figs/mgcPow.png}
\caption{
This figure considers a mixture model in the graph domain,
where half the nodes are dependent to their attributes, and the other
half nodes are independently generated from the attributes. We calculate
the MGC statistic and its power for dependence testing, as well as the
node contribution statistic and its power, i.e., the percentage of all
dependent nodes that are ranked among the first half by $w_i$.  Indeed,
the power plot not only shows that our node contribution algorithm can
successfully identifies all important nodes at mildly large sample size,
but also hints a tight relationship between $w_i$ and $C*$ that is
worthy of further investigation.
}
\label{fig:mgcP}
\end{cframed}
\end{figure}

\subsection{Synaptome Statistics}

We have continued to examine the Kristina15 synaptome dataset and have
now added the Weiler Chessboard dataset to our explorations.  Using
\verb+meda+, and other methods along the way, we have started to compare
the structure of these two datasets, see figure~\ref{fig:synClaw}.

\begin{figure}[h!]
\begin{cframed}
\centering
\includegraphics[width=\textwidth]{./figs/2dProjClaw.pdf}
\caption{
  Principal Component Analysis (PCA) has been run on the correlation
  matrices of both of these dataset respectively.  The first two
  principal components are plotted against each other.  The colors
  correspond to the type of marker, (``excitatory'' -- green,
  ``inhibitory'' -- red, ``other'' -- blue).  Notice that some markers are
  different between datasets, but there is a similar "claw" structure
  present.
}
\label{fig:synClaw}
\end{cframed}
\end{figure}

\end{document}


\documentclass[a4paper,12pt]{article}
%\documentclass[a4paper,12pt]{scrartcl}

\usepackage{xltxtra}

\input{../preamble.tex}

% \usepackage[spanish]{babel}

% \setromanfont[Mapping=tex-text]{Linux Libertine O}
% \setsansfont[Mapping=tex-text]{DejaVu Sans}
% \setmonofont[Mapping=tex-text]{DejaVu Sans Mono}

\title{Homework \#15}
\author{Isaac Ayala Lozano}
\date{\today}

\begin{document}
\maketitle

\begin{enumerate}
 \item Explain why $G_{xx}(f)$ is a constant for Example 6.3 in \cite{bendat2011random}.

 The function $G_{xx}(f)$ is a constant for the example because $G_{xx}(f)$ is the autospectral density function of the input.
 This function is defined as

 \begin{equation*}
  G_{xx} = 2 S_{xx}(f)
 \end{equation*}

 Recall that

\begin{equation*}
 S_{xx}(f) = 2 \int_0^\infty R_{xx}(\tau) \cos(2 \pi f \tau ) d\tau
\end{equation*}

 As such, $G_{xx}$ becomes

 \begin{equation*}
  G_{xx} = 4 \int_0^\infty R_{xx}(\tau) \cos(2 \pi f \tau ) d\tau
 \end{equation*}

 And $R_{xx}$ is defined as

\begin{equation*}
 R_{xx}(\tau) = E[x_k(t) x_k(t+\tau)]
\end{equation*}

The expected value of the input, white noise, is a constant $\mu$ that depends on the samples. Thus, $G_{xx}$ is a constant because the Fourier transformation of the original input results in a constant as well.

\newpage

\item Explain how the output spectral function $G_{yy}$ for Example 6.3 results in the output autocorrelation function $R_{yy}$ shown below.

\begin{align*}
 G_{yy}(f) &= \abs{H(f)}_{f-d}^2 G = \frac{G}{\abs{1-(f/f_n)^2}^2 + (2 \zeta f/f_n)^2 } \quad 0 \leq f < \infty \\
 R_{yy}(\tau) &= \frac{G \pi f_n \exp(-2\pi f_n \zeta \abs{\tau})}{4 \zeta} F(\tau, \zeta) \\
 F(\tau, \zeta) &=
  \cos(2 \pi f_n \sqrt{1-\zeta^2} \abs{\tau})
 +
 \frac{\zeta}{\sqrt{1-\zeta^2}}
 \sin(2 \pi f_n \sqrt{1-\zeta^2} \abs{\tau})
\end{align*}

Recall that the the autocorrelation function can be expressed as

\begin{equation*}
 R_{yy}(\tau) = \int_0^\infty G_{yy} (f) \cos(2 \pi f \tau) df
\end{equation*}

Substituting known values in the expression

\begin{align*}
 R_{yy}(\tau) &= \int_0^\infty \abs{H(f)}_{f-d}^2 G \cos(2 \pi f \tau) df \\
 &= G \int_0^\infty \abs{H(f)}_{f-d}^2 \cos(2 \pi f \tau) df \\
 &= G \int_0^\infty
 \frac{\cos(2 \pi f \tau)  }{\abs{1-(f/f_n)^2}^2 + (2 \zeta f/f_n)^2 }
 df
\end{align*}

Solving the integral, we obtain the following

\begin{align*}
 R_{yy}(\tau) &= \frac{G \pi f_n \exp(-2\pi f_n \zeta \abs{\tau})}{4 \zeta}
 \cos(2 \pi f_n \sqrt{1-\zeta^2} \abs{\tau}) \\
 & \qquad + \frac{G \pi f_n \exp(-2\pi f_n \zeta \abs{\tau})}{4 \zeta}
 \frac{\zeta}{\sqrt{1-\zeta^2}}
 \sin(2 \pi f_n \sqrt{1-\zeta^2} \abs{\tau})
\end{align*}

which can then be factorized to

\begin{align*}
  R_{yy}(\tau) &= \frac{G \pi f_n \exp(-2\pi f_n \zeta \abs{\tau})}{4 \zeta} F(\tau, \zeta) \\
 F(\tau, \zeta) &=
  \cos(2 \pi f_n \sqrt{1-\zeta^2} \abs{\tau})
 +
 \frac{\zeta}{\sqrt{1-\zeta^2}}
 \sin(2 \pi f_n \sqrt{1-\zeta^2} \abs{\tau})
\end{align*}

\newpage

\item Present plots for both $G_{yy}$ and $R_{yy}$

\begin{figure}[htb!]
\centering
\import{./img/}{hw15_comparison.tex}
\caption{Comparison.}
\label{fig: comparison}
\end{figure}

\begin{figure}[htb!]
\centering
\import{./img/}{hw15_Gyy.tex}
\caption{Output Spectral function.}
\label{fig: gyy}
\end{figure}

\begin{figure}[htb!]
\centering
\import{./img/}{hw15_Ryy.tex}
\caption{Output Autocorrelation function.}
\label{fig: ryy}
\end{figure}

\end{enumerate}

\newpage
\pagebreak

\printbibliography

\newpage
\pagebreak
\appendix
\section{Octave Code}
\lstinputlisting[language=Matlab]{hw15_plots.m}

\end{document}


\documentclass[12pt]{cdblatex}
\usepackage{fancyhdr}
\usepackage{footer}

\begin{document}

% =================================================================================================
% create checkpoint file

\bgroup
\CdbSetup{action=hide}
\begin{cadabra}
   import cdblib
   checkpoint_file = 'tests/semantic/output/dRabcd.json'
   cdblib.create (checkpoint_file)
   checkpoint = []
\end{cadabra}
\egroup

% =================================================================================================
\section*{Symmetrised partial derivatives of the Riemann tensor}

Here we compute the symmetrised partial derivatoves $R^{a}{}_{(b{\Dot c}d,\ue)}$ in terms of
the symmetrised covariant derivatives $R^{a}{}_{(b{\Dot c}d;\ue)}$. Note that the dot over an
index indicates that that index does not take part in the symmetrisation.

We will use the algorithm described in section (10.3) of my lcb09-03 paper. Here we will make
one small change of notation -- the symbol $D^a$ will replaced with $A^a$.

We have lots of space (and no annoying editors to appease with brevity) so I will take the
liberty to expand slightly on what I wrote in the lcb0-03 paper.

Our starting point is the simple identity
\begin{align}
   \label{eqn:submain}
   \left( R^{a}{}_{cdb} B^{b}{}_{a} A^{c} A^{d} \right)_{;e} A^{e}
   =
   \left( R^{a}{}_{cdb} B^{b}{}_{a} A^{c} A^{d} \right)_{,e} A^{e}
\end{align}
This is true in all frames since the quantity inside the brackets is a scalar. We are free to
make any choice we like for $A^{a}$ and $B^{a}{}_{b}$ so let's choose $A^{a}$ to be the tangent
vector to any geodesic through the origin and choose the $B^{a}{}_{b}$ to be constants (i.e,
all partial derivatives are zero). We will also use local Riemann normal coordinates and as a
consequence, the $A^{a}$ will also be constant along the integral curves of $A$ (the geodesics
in an RNC are always of the form $x^{a}(s) = s A^{a}$ for some affine parameter $s$ on the
geodesic). Let $df/ds$ be the directional derivative of the function $f$ along the geodesics
defined by $A^{a}$ and assume that $s$ is the proper length along the geodesic (although any
affine parameter would be sufficient).

Thus at the origin we have, by choice,
\begin{gather*}
   0 = B^{a}{}_{b,c} = B^{a}{}_{b,cd} = B^{a}{}_{b,cde} = \dots\\[5pt]
   0 = dA^{a}/ds = d^2A^{a}/ds^2 = d^3A^{a}/ds^3 = \dots\\[5pt]
   0 = A^{a}{}_{,b} A^{b} = \left(A^{a}{}_{,b} A^{b}\right)_{,c} A^{c} = \left(\left(A^{a}{}_{,b} A^{b}\right)_{,c} A^{c}\right)_{,d} A^{d}\\[5pt]
   0 = A^{a}{}_{;b} A^{b} = \left(A^{a}{}_{;b} A^{b}\right)_{;c} A^{c} = \left(\left(A^{a}{}_{;b} A^{b}\right)_{;c} A^{c}\right)_{;d} A^{d}\\[5pt]
   df/ds = f_{,a} A^{a} = f_{;a} A^{a}\\[5pt]
   d^2f/ds^2 = \left ( f_{,a} A^{a} \right)_{,b} A^{b} = \left ( f_{;a} A^{a} \right)_{;b} A^{b}\\[5pt]
   d^3f/ds^3 = \left(\left ( f_{,a} A^{a} \right)_{,b} A^{b}\right)_{,c} A^{c} = \left(\left ( f_{;a} A^{a} \right)_{;b} A^{b}\right)_{;c} A^{c}
\end{gather*}
I admit I've gone overboard here in writing out more than I need to but it's handy to have all
of these equations laid bare in one convenient place.

Now put $f = R^{p}{}_{abq} B^{q}{}_{p} A^{a} A^{b}$. Then upon taking successive derivatives,
while taking full advantage of the asummptions just noted, we can eaily see that
\begin{align}
   \label{eqn:main}
   \left( R^{a}{}_{cdb} B^{b}{}_{a} \right)_{;\ue} A^{c} A^{d} A^{\ue}
   =
   \left( R^{a}{}_{cdb} \right)_{,\ue} B^{b}{}_{a} A^{c} A^{d} A^{\ue}
\end{align}
This is the equation that will be computed by the following Cadabra code. All of the
compuations will be carried out on the left hand side (in the first version of the paper I
swapped the left and righ hand sides).

We will need the successive covariant derivatives of $B$. The first covariant derivative is
just \begin{align*} B^{a}{}_{b;c} A^c &= \Gamma^{a}{}_{dc}B^{d}{}_{b} A^c -
\Gamma^{d}{}_{bc}B^{a}{}_{d} A^c \end{align*} The quantities on the left hand side are the
components of a tensor so further covariant derivatives of the right hand side can be computed
(despite the presence of the $\Gamma$'s) by application of the usual rule for a covariant
derivative of a mixed tensor.

% =================================================================================================
\section*{Stage 1: Symmetrised partial derivatives of $R$}

The first stage involves the expansion of the left side of (\ref{eqn:main}). This leads to
expressions for the symmetrized partial derivatives of $R_{abcd}$ in terms of the symmetrized
covariant derivatives of $R_{abcd}$ and $B^{a}{}_{b}$.

\begin{dgroup*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,e} B^{b}{}_{a} A^{c} A^{d} A^{e}
                  = \cdb{dRabcd01.108} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,ef} B^{b}{}_{a} A^{c} A^{d} A^{e} A^{f}
                  = \cdb{dRabcd02.108} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,efg} B^{b}{}_{a} A^{c} A^{d} A^{e} A^{f} A^{g}
                  = \cdb{dRabcd03.108} \end{dmath*}
\end{dgroup*}

% =================================================================================================
\section*{Stage 2: Symmetrised covariant derivatives of $B$}

In this stage the symmetrized covariant derivatives of $B^{a}{}_{b}$ are computed in terms of
its partial derivatives (which by choice are all zero) and the connection and its partial
derivatives (which in general are not zero).

\begin{dgroup*}
   \begin{dmath*} A^{c}\nabla_{c}\left(B^{a}{}_{b}\right)
                  = \cdb{dBab01.209} \end{dmath*}
   \begin{dmath*} A^{d}A^{c}\nabla_{d}\left(\nabla_{c}\left(B^{a}{}_{b}\right)\right)
                  = \cdb{dBab02.209} \end{dmath*}
   \begin{dmath*} A^{e}A^{d}A^{c}\nabla_{e}\left(\nabla_{d}\left(\nabla_{c}\left(B^{a}{}_{b}\right)\right)\right)
                  = \cdb{dBab03.209} \end{dmath*}
\end{dgroup*}

% =================================================================================================
\section*{Stage 3: Impose the Riemann normal coordinate condition on covariant derivs of $B$}

Here we impose the RNC condition (that $\Gamma = 0$ while $\partial\Gamma\not=0$).

\begin{dgroup*}
   \begin{dmath*} A^{c}\nabla_{c}\left(B^{a}{}_{b}\right)
                  = \cdb{dBab01.301} \end{dmath*}
   \begin{dmath*} A^{d}A^{c}\nabla_{d}\left(\nabla_{c}\left(B^{a}{}_{b}\right)\right)
                  = \cdb{dBab02.301} \end{dmath*}
   \begin{dmath*} A^{e}A^{d}A^{c}\nabla_{e}\left(\nabla_{d}\left(\nabla_{c}\left(B^{a}{}_{b}\right)\right)\right)
                  = \cdb{dBab03.301} \end{dmath*}
\end{dgroup*}

% =================================================================================================
\section*{Stage 4: Replace covariant derivs of $B$ with partial derivs of $\Gamma$}

This stage uses the results from the second stage to eliminate the $\nabla B$ terms from the
results of the first stage. This produces expressions for the symmetrized partial derivatives
of $R_{abcd}$ in terms of the symmetrized covariant derivatives of $R_{abcd}$ and the partial
derivatives of the connection. In this stage we also set the $B^{a}{}_{b}$ to equal 1.

\begin{dgroup*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,e} A^{c} A^{d} A^{e}
                  = \cdb{dRabcd01.401} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,ef} A^{c} A^{d} A^{e} A^{e}
                  = \cdb{dRabcd02.401} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,efg} A^{c} A^{d} A^{e} A^{f} A^{g}
                  = \cdb{dRabcd03.401} \end{dmath*}
\end{dgroup*}

% =================================================================================================
\section*{Stage 5: Replace partial derivs of $\Gamma$ with partial derivs of $R$}

The fifth stage draws in results from {\tt dGamma.tex} to replace the partial derivatives of
$\Gamma$ with partial derivatives of $R_{abcd}$.

\begin{dgroup*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,e} A^{c} A^{d} A^{e}
                  = \cdb{dRabcd01.500} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,ef} A^{c} A^{d} A^{e} A^{f}
                  = \cdb{dRabcd02.505} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,efg} A^{c} A^{d} A^{e} A^{f} A^{g}
                  = \cdb{dRabcd03.507} \end{dmath*}
\end{dgroup*}

% =================================================================================================
\section*{Stage 6: Replace partial derivs of $R$ with covariant derivs of $R$}

The final stage is to eliminate the $\partial R$ by using earlier results. For example, in the
equation for $\partial^3 R$ we see terms involving $\partial R$. These first order partial
derivatives can be replaced with the expression previously computed for $\partial R$ in terms
of $\nabla R$.

\begin{dgroup*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,e} A^{c} A^{d} A^{e}
                  = \cdb{dRabcd01.702} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,ef} A^{c} A^{d} A^{e} A^{e}
                  = \cdb{dRabcd02.702} \end{dmath*}
   \begin{dmath*} \left( R^{a}{}_{cdb} \right)_{,efg} A^{c} A^{d} A^{e} A^{f} A^{g}
                  = \cdb{dRabcd03.702} \end{dmath*}
\end{dgroup*}

The end result are expressions for the symmetrized partial derivatives of $R_{abcd}$ solely in
terms of the symmetrized covariant derivatives of $R_{abcd}$.

\clearpage

% =================================================================================================
\section*{Shared properties}

\begin{cadabra}
   import time

   {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w#}::Indices(position=independent).

   \nabla{#}::Derivative.
   \partial{#}::PartialDerivative.

   g_{a b}::Metric.
   g^{a b}::InverseMetric.
   g_{a}^{b}::KroneckerDelta.
   g^{a}_{b}::KroneckerDelta.

   R_{a b c d}::RiemannTensor.
   R^{a}_{b c d}::RiemannTensor.

   \Gamma^{a}_{b c}::TableauSymmetry(shape={2}, indices={1,2}).

   g_{a b}::Depends(\partial{#}).
   R_{a b c d}::Depends(\partial{#}).
   R^{a}_{b c d}::Depends(\partial{#}).
   \Gamma^{a}_{b c}::Depends(\partial{#}).

   B^{a}_{b::Depends(\nabla{#}).
   R_{a b c d}::Depends(\nabla{#}).
   R^{a}_{b c d}::Depends(\nabla{#}).

\end{cadabra}

\clearpage

% =================================================================================================
\section*{Stage 1: Symmetrised partial derivatives of $R$}

\begin{cadabra}
   def flatten_Rabcd (obj):
       substitute (obj,$R^{a}_{b c d}   -> g^{a e} R_{e b c d}$)
       substitute (obj,$R_{a}^{b}_{c d} -> g^{b e} R_{a e c d}$)
       substitute (obj,$R_{a b}^{c}_{b} -> g^{c e} R_{a b e d}$)
       substitute (obj,$R_{a b c}^{d}   -> g^{d e} R_{a b c e}$)
       unwrap     (obj)
       sort_product   (obj)
       rename_dummies (obj)
       return obj

   # compute the symmetric covariant derivatives of R^{a}_{bcd} B^{d}_{a}

   beg_stage_1 = time.time()

   dRabcd00:=R^{a}_{b c d} B^{d}_{a} A^{b} A^{c}.        # cdb(dRabcd00.101,dRabcd00)

   dRabcd01:=A^{a}\nabla_{a}{ @(dRabcd00) }.             # cdb(dRabcd01.101,dRabcd01)
   distribute     (dRabcd01)                             # cdb(dRabcd01.102,dRabcd01)
   product_rule   (dRabcd01)                             # cdb(dRabcd01.103,dRabcd01)
   distribute     (dRabcd01)                             # cdb(dRabcd01.104,dRabcd01)
   substitute     (dRabcd01,$\nabla_{a}{A^{b}} -> 0$)    # cdb(dRabcd01.105,dRabcd01)
   substitute     (dRabcd01,$\nabla_{a}{g^{b c}} -> 0$)  # cdb(dRabcd01.106,dRabcd01)

   sort_product   (dRabcd01)
   rename_dummies (dRabcd01)
   canonicalise   (dRabcd01)                             # cdb(dRabcd01.107,dRabcd01)
   dRabcd01 = flatten_Rabcd (dRabcd01)                   # cdb(dRabcd01.108,dRabcd01)

   dRabcd02:=A^{a}\nabla_{a}{ @(dRabcd01) }.             # cdb(dRabcd02.101,dRabcd02)
   distribute     (dRabcd02)                             # cdb(dRabcd02.102,dRabcd02)
   product_rule   (dRabcd02)                             # cdb(dRabcd02.103,dRabcd02)
   distribute     (dRabcd02)                             # cdb(dRabcd02.104,dRabcd02)
   substitute     (dRabcd02,$\nabla_{a}{A^{b}} -> 0$)    # cdb(dRabcd02.105,dRabcd02)
   substitute     (dRabcd02,$\nabla_{a}{g^{b c}} -> 0$)  # cdb(dRabcd02.106,dRabcd02)

   sort_product   (dRabcd02)
   rename_dummies (dRabcd02)
   canonicalise   (dRabcd02)                             # cdb(dRabcd02.107,dRabcd02)
   dRabcd02 = flatten_Rabcd (dRabcd02)                   # cdb(dRabcd02.108,dRabcd02)

   dRabcd03:=A^{a}\nabla_{a}{ @(dRabcd02) }.             # cdb(dRabcd03.101,dRabcd03)
   distribute     (dRabcd03)                             # cdb(dRabcd03.102,dRabcd03)
   product_rule   (dRabcd03)                             # cdb(dRabcd03.103,dRabcd03)
   distribute     (dRabcd03)                             # cdb(dRabcd03.104,dRabcd03)
   substitute     (dRabcd03,$\nabla_{a}{A^{b}} -> 0$)    # cdb(dRabcd03.105,dRabcd03)
   substitute     (dRabcd03,$\nabla_{a}{g^{b c}} -> 0$)  # cdb(dRabcd03.106,dRabcd03)

   sort_product   (dRabcd03)
   rename_dummies (dRabcd03)
   canonicalise   (dRabcd03)                             # cdb(dRabcd03.107,dRabcd03)
   dRabcd03 = flatten_Rabcd (dRabcd03)                   # cdb(dRabcd03.108,dRabcd03)

   dRabcd04:=A^{a}\nabla_{a}{ @(dRabcd03) }.
   distribute     (dRabcd04)
   product_rule   (dRabcd04)
   distribute     (dRabcd04)
   substitute     (dRabcd04,$\nabla_{a}{A^{b}} -> 0$)
   substitute     (dRabcd04,$\nabla_{a}{g^{b c}} -> 0$)

   sort_product   (dRabcd04)
   rename_dummies (dRabcd04)
   canonicalise   (dRabcd04)
   dRabcd04 = flatten_Rabcd (dRabcd04)

   dRabcd05:=A^{a}\nabla_{a}{ @(dRabcd04) }.
   distribute     (dRabcd05)
   product_rule   (dRabcd05)
   distribute     (dRabcd05)
   substitute     (dRabcd05,$\nabla_{a}{A^{b}} -> 0$)
   substitute     (dRabcd05,$\nabla_{a}{g^{b c}} -> 0$)

   sort_product   (dRabcd05)
   rename_dummies (dRabcd05)
   canonicalise   (dRabcd05)
   dRabcd05 = flatten_Rabcd (dRabcd05)

   def combine_nabla (obj):
       substitute (obj,$\nabla_{p}{\nabla_{q}{\nabla_{r}{\nabla_{s}{\nabla_{t}{A??}}}}}->\nabla_{p q r s t}{A??}$,repeat=True)
       substitute (obj,$\nabla_{p}{\nabla_{q}{\nabla_{r}{\nabla_{s}{A??}}}}->\nabla_{p q r s}{A??}$,repeat=True)
       substitute (obj,$\nabla_{p}{\nabla_{q}{\nabla_{r}{A??}}}->\nabla_{p q r}{A??}$,repeat=True)
       substitute (obj,$\nabla_{p}{\nabla_{q}{A??}}->\nabla_{p q}{A??}$,repeat=True)
       return obj

   dRabcd01 = combine_nabla (dRabcd01)
   dRabcd02 = combine_nabla (dRabcd02)
   dRabcd03 = combine_nabla (dRabcd03)
   dRabcd04 = combine_nabla (dRabcd04)
   dRabcd05 = combine_nabla (dRabcd05)

   end_stage_1 = time.time()
\end{cadabra}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd00.101} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd01.101} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd01.102} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd01.103} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd01.104} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd01.105} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd01.106} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd01.107} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd01.108} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd02.101} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.102} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.103} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.104} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.105} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.106} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.107} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.108} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd03.101} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.102} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.103} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.104} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.105} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.106} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.107} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.108} \end{dmath*}
\end{dgroup*}

\clearpage

% =================================================================================================
\section*{Stage 2: Symmetrised covariant derivatives of $B$}

\begin{cadabra}
   # compute the covariant derivatives of B^{a}_{b}, note B^{a}_{b,c} is zero, by choice
   # this method of computing covariant derivatives does not use auxillary fields

   beg_stage_2 = time.time()

   dBab00:=B^{a}_{b}.      # cdb(dBab00.201,dBab00)

   dBab01:=A^{c}\partial_{c}{ @(dBab00) } + \Gamma^{a}_{p q} W^{p}_{b} A^{q}
                                          - \Gamma^{p}_{b q} W^{a}_{p} A^{q}.
                                                         # cdb(dBab01.201,dBab01)
   distribute   (dBab01)                                 # cdb(dBab01.202,dBab01)
   product_rule (dBab01)                                 # cdb(dBab01.203,dBab01)
   distribute   (dBab01)                                 # cdb(dBab01.204,dBab01)
   substitute   (dBab01,$\partial_{a}{A^{b}} -> 0$)      # cdb(dBab01.205,dBab01)
   substitute   (dBab01,$\partial_{a}{B^{b}_{c}} -> 0$)  # cdb(dBab01.206,dBab01)
   substitute   (dBab01,$W^{a}_{b} -> @(dBab00)$)        # cdb(dBab01.207,dBab01)
   distribute   (dBab01)                                 # cdb(dBab01.208,dBab01)
   canonicalise (dBab01)                                 # cdb(dBab01.209,dBab01)

   dBab02:=A^{c}\partial_{c}{ @(dBab01) } + \Gamma^{a}_{p q} W^{p}_{b} A^{q}
                                          - \Gamma^{p}_{b q} W^{a}_{p} A^{q}.
                                                         # cdb(dBab02.201,dBab02)
   distribute   (dBab02)                                 # cdb(dBab02.202,dBab02)
   product_rule (dBab02)                                 # cdb(dBab02.203,dBab02)
   distribute   (dBab02)                                 # cdb(dBab02.204,dBab02)
   substitute   (dBab02,$\partial_{a}{A^{b}} -> 0$)      # cdb(dBab02.205,dBab02)
   substitute   (dBab02,$\partial_{a}{B^{b}_{c}} -> 0$)  # cdb(dBab02.206,dBab02)
   substitute   (dBab02,$W^{a}_{b} -> @(dBab01)$)        # cdb(dBab02.207,dBab02)
   distribute   (dBab02)                                 # cdb(dBab02.208,dBab02)
   canonicalise (dBab02)                                 # cdb(dBab02.209,dBab02)

   dBab03:=A^{c}\partial_{c}{ @(dBab02) } + \Gamma^{a}_{p q} W^{p}_{b} A^{q}
                                          - \Gamma^{p}_{b q} W^{a}_{p} A^{q}.
                                                         # cdb(dBab03.201,dBab03)
   distribute   (dBab03)                                 # cdb(dBab03.202,dBab03)
   product_rule (dBab03)                                 # cdb(dBab03.203,dBab03)
   distribute   (dBab03)                                 # cdb(dBab03.204,dBab03)
   substitute   (dBab03,$\partial_{a}{A^{b}} -> 0$)      # cdb(dBab03.205,dBab03)
   substitute   (dBab03,$\partial_{a}{B^{b}_{c}} -> 0$)  # cdb(dBab03.206,dBab03)
   substitute   (dBab03,$W^{a}_{b} -> @(dBab02)$)        # cdb(dBab03.207,dBab03)
   distribute   (dBab03)                                 # cdb(dBab03.208,dBab03)
   canonicalise (dBab03)                                 # cdb(dBab03.209,dBab03)

   dBab04:=A^{c}\partial_{c}{ @(dBab03) } + \Gamma^{a}_{p q} W^{p}_{b} A^{q}
                                          - \Gamma^{p}_{b q} W^{a}_{p} A^{q}.
   distribute   (dBab04)
   product_rule (dBab04)
   distribute   (dBab04)
   substitute   (dBab04,$\partial_{a}{A^{b}} -> 0$)
   substitute   (dBab04,$\partial_{a}{B^{b}_{c}} -> 0$)
   substitute   (dBab04,$W^{a}_{b} -> @(dBab03)$)
   distribute   (dBab04)
   canonicalise (dBab04)

   dBab05:=A^{c}\partial_{c}{ @(dBab04) } + \Gamma^{a}_{p q} W^{p}_{b} A^{q}
                                          - \Gamma^{p}_{b q} W^{a}_{p} A^{q}.
   distribute   (dBab05)
   product_rule (dBab05)
   distribute   (dBab05)
   substitute   (dBab05,$\partial_{a}{A^{b}} -> 0$)
   substitute   (dBab05,$\partial_{a}{B^{b}_{c}} -> 0$)
   substitute   (dBab05,$W^{a}_{b} -> @(dBab04)$)
   distribute   (dBab05)
   canonicalise (dBab05)

   end_stage_2 = time.time()
\end{cadabra}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dBab00.201} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dBab01.201} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.202} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.203} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.204} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.205} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.206} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.207} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.208} \end{dmath*}
   \begin{dmath*} \cdb*{dBab01.209} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dBab02.201} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.202} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.203} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.204} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.205} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.206} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.207} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.208} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.209} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dBab03.201} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.202} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.203} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.204} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.205} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.206} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.207} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.208} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.209} \end{dmath*}
\end{dgroup*}

\clearpage

% =================================================================================================
\section*{Stage 3: Impose the Riemann normal coordinate condition on covariant derivs of $B$}

\begin{cadabra}
   def impose_rnc (obj):
       # hide the derivatives of Gamma
       substitute (obj,$\partial_{d}{\Gamma^{a}_{b c}} -> zzz_{d}^{a}_{b c}$,repeat=True)
       substitute (obj,$\partial_{d e}{\Gamma^{a}_{b c}} -> zzz_{d e}^{a}_{b c}$,repeat=True)
       substitute (obj,$\partial_{d e f}{\Gamma^{a}_{b c}} -> zzz_{d e f}^{a}_{b c}$,repeat=True)
       substitute (obj,$\partial_{d e f g}{\Gamma^{a}_{b c}} -> zzz_{d e f g}^{a}_{b c}$,repeat=True)
       substitute (obj,$\partial_{d e f g h}{\Gamma^{a}_{b c}} -> zzz_{d e f g h}^{a}_{b c}$,repeat=True)
       # set Gamma to zero
       substitute (obj,$\Gamma^{a}_{b c} -> 0$,repeat=True)
       # recover the derivatives Gamma
       substitute (obj,$zzz_{d}^{a}_{b c} -> \partial_{d}{\Gamma^{a}_{b c}}$,repeat=True)
       substitute (obj,$zzz_{d e}^{a}_{b c} -> \partial_{d e}{\Gamma^{a}_{b c}}$,repeat=True)
       substitute (obj,$zzz_{d e f}^{a}_{b c} -> \partial_{d e f}{\Gamma^{a}_{b c}}$,repeat=True)
       substitute (obj,$zzz_{d e f g}^{a}_{b c} -> \partial_{d e f g}{\Gamma^{a}_{b c}}$,repeat=True)
       substitute (obj,$zzz_{d e f g h}^{a}_{b c} -> \partial_{d e f g h}{\Gamma^{a}_{b c}}$,repeat=True)
       return obj

   # switch to RNC

   beg_stage_3 = time.time()

   dBab01 = impose_rnc (dBab01)   # cdb (dBab01.301,dBab01)
   dBab02 = impose_rnc (dBab02)   # cdb (dBab02.301,dBab02)
   dBab03 = impose_rnc (dBab03)   # cdb (dBab03.301,dBab03)
   dBab04 = impose_rnc (dBab04)   # cdb (dBab04.301,dBab04)
   dBab05 = impose_rnc (dBab05)   # cdb (dBab05.301,dBab05)

   end_stage_3 = time.time()
\end{cadabra}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dBab01.301} \end{dmath*}
   \begin{dmath*} \cdb*{dBab02.301} \end{dmath*}
   \begin{dmath*} \cdb*{dBab03.301} \end{dmath*}
   \begin{dmath*} \cdb*{dBab04.301} \end{dmath*}
   \begin{dmath*} \cdb*{dBab05.301} \end{dmath*}
\end{dgroup*}

\clearpage

% =================================================================================================
\section*{Stage 4: Replace covariant derivs of $B$ with partial derivs of $\Gamma$}

\begin{cadabra}
   # substitute covariant derivs of B^{a}_{b} into covariant derivs of R^{a}_{bcd}B^{d}_{a}
   # this produces expressions for the partial derivs of Rabcd its covariant derivs and partial derivs of Gamma
   # the partial derivs of Gamma will be eliminted later by using results imported from dGamma.json

   beg_stage_4 = time.time()

   substitute (dRabcd01,$A^{c}\nabla_{c}{B^{a}_{b}} -> @(dBab01)$,repeat=True);   distribute (dRabcd01)
   substitute (dRabcd02,$A^{c}\nabla_{c}{B^{a}_{b}} -> @(dBab01)$,repeat=True);   distribute (dRabcd02)
   substitute (dRabcd03,$A^{c}\nabla_{c}{B^{a}_{b}} -> @(dBab01)$,repeat=True);   distribute (dRabcd03)
   substitute (dRabcd04,$A^{c}\nabla_{c}{B^{a}_{b}} -> @(dBab01)$,repeat=True);   distribute (dRabcd04)
   substitute (dRabcd05,$A^{c}\nabla_{c}{B^{a}_{b}} -> @(dBab01)$,repeat=True);   distribute (dRabcd05)

   substitute (dRabcd02,$A^{c}A^{d}\nabla_{c d}{B^{a}_{b}} -> @(dBab02)$,repeat=True);   distribute (dRabcd02)
   substitute (dRabcd03,$A^{c}A^{d}\nabla_{c d}{B^{a}_{b}} -> @(dBab02)$,repeat=True);   distribute (dRabcd03)
   substitute (dRabcd04,$A^{c}A^{d}\nabla_{c d}{B^{a}_{b}} -> @(dBab02)$,repeat=True);   distribute (dRabcd04)
   substitute (dRabcd05,$A^{c}A^{d}\nabla_{c d}{B^{a}_{b}} -> @(dBab02)$,repeat=True);   distribute (dRabcd05)

   substitute (dRabcd03,$A^{c}A^{d}A^{e}\nabla_{c d e}{B^{a}_{b}} -> @(dBab03)$,repeat=True);   distribute (dRabcd03)
   substitute (dRabcd04,$A^{c}A^{d}A^{e}\nabla_{c d e}{B^{a}_{b}} -> @(dBab03)$,repeat=True);   distribute (dRabcd04)
   substitute (dRabcd05,$A^{c}A^{d}A^{e}\nabla_{c d e}{B^{a}_{b}} -> @(dBab03)$,repeat=True);   distribute (dRabcd05)

   substitute (dRabcd04,$A^{c}A^{d}A^{e}A^{f}\nabla_{c d e f}{B^{a}_{b}} -> @(dBab04)$,repeat=True); distribute (dRabcd04)
   substitute (dRabcd05,$A^{c}A^{d}A^{e}A^{f}\nabla_{c d e f}{B^{a}_{b}} -> @(dBab04)$,repeat=True); distribute (dRabcd05)

   substitute (dRabcd05,$A^{c}A^{d}A^{e}A^{f}A^{g}\nabla_{c d e f g}{B^{a}_{b}} -> @(dBab05)$,repeat=True); distribute (dRabcd05)

   # no longer need B, so let's get rid of it

   # two subtle tricks are used here
   # 1) rename A and B as A002 and A001 before sort_product,
   #    this ensures B will be to left of A after the sort
   # 2) indices on B changed from B^{a}_{b} to B_{b}^{a},
   #    this ensures that after factor_out B will have dummy indices B_{a}^{b}

   def remove_Bab (obj):
       foo := @(obj).
       substitute     (foo,$A^{a}->A002^{a},B^{a}_{b}->A001_{b}^{a}$)  # need this to sort B to the left of A
       sort_product   (foo)
       rename_dummies (foo)
       factor_out     (foo,$A001^{a?}_{b?},A002^{c?}$)
       substitute     (foo,$A001_{a}^{b}->1,A002^{a}->A^{a}$)  # recover A and set B = 1, free indices now ^{a}_{b}
       return foo

   dRabcd01 = remove_Bab (dRabcd01)   # cdb(dRabcd01.401,dRabcd01)
   dRabcd02 = remove_Bab (dRabcd02)   # cdb(dRabcd02.401,dRabcd02)
   dRabcd03 = remove_Bab (dRabcd03)   # cdb(dRabcd03.401,dRabcd03)
   dRabcd04 = remove_Bab (dRabcd04)   # cdb(dRabcd04.401,dRabcd04)
   dRabcd05 = remove_Bab (dRabcd05)   # cdb(dRabcd05.401,dRabcd05)

   end_stage_4 = time.time()
\end{cadabra}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd01.401} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.401} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.401} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.401} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd05.401} \end{dmath*}
\end{dgroup*}

\clearpage

% =================================================================================================
\section*{Stage 5: Replace partial derivs of $\Gamma$ with partial derivs of $R$}

\begin{cadabra}
   import cdblib

   beg_stage_5 = time.time()

   dGamma01 = cdblib.get ('dGamma01','dGamma.json')  # cdb(dGamma01.500,dGamma01)
   dGamma02 = cdblib.get ('dGamma02','dGamma.json')  # cdb(dGamma02.500,dGamma02)
   dGamma03 = cdblib.get ('dGamma03','dGamma.json')  # cdb(dGamma03.500,dGamma03)
   dGamma04 = cdblib.get ('dGamma04','dGamma.json')  # cdb(dGamma04.500,dGamma04)
   dGamma05 = cdblib.get ('dGamma05','dGamma.json')  # cdb(dGamma05.500,dGamma05)

   distribute (dRabcd01)   # cdb(dRabcd01.500,dRabcd01)
   distribute (dRabcd02)   # cdb(dRabcd02.500,dRabcd02)
   distribute (dRabcd03)   # cdb(dRabcd03.500,dRabcd03)
   distribute (dRabcd04)   # cdb(dRabcd04.500,dRabcd04)
   distribute (dRabcd05)   # cdb(dRabcd05.500,dRabcd05)

   # use dGamma to eliminate the partial derivs of Gamma
   # this will introduces some lower order partial dervis of Rabcd on the rhs
   # these extra partial derivs of Rabcd will be eliminated (later) by substiting lower order dRabcd into the higher order dRabcd

   substitute (dRabcd02,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{d b}} -> @(dGamma01)$,repeat=True)                # cdb(dRabcd02.501,dRabcd02)
   substitute (dRabcd02,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{b d}} -> @(dGamma01)$,repeat=True)                # cdb(dRabcd02.502,dRabcd02)
   distribute (dRabcd02)                                                                                # cdb(dRabcd02.503,dRabcd02)
   sort_product   (dRabcd02)                                                                                # cdb(dRabcd02.504,dRabcd02)
   rename_dummies (dRabcd02)                                                                                # cdb(dRabcd02.505,dRabcd02)

   substitute (dRabcd03,$A^{c}A^{b}A^{e}\partial_{c e}{\Gamma^{a}_{d b}} -> @(dGamma02)$,repeat=True)         # cdb(dRabcd03.501,dRabcd03)
   substitute (dRabcd03,$A^{c}A^{b}A^{e}\partial_{c e}{\Gamma^{a}_{b d}} -> @(dGamma02)$,repeat=True)         # cdb(dRabcd03.502,dRabcd03)
   substitute (dRabcd03,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{d b}} -> @(dGamma01)$,repeat=True)                # cdb(dRabcd03.503,dRabcd03)
   substitute (dRabcd03,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{b d}} -> @(dGamma01)$,repeat=True)                # cdb(dRabcd03.504,dRabcd03)
   distribute (dRabcd03)                                                                                # cdb(dRabcd03.505,dRabcd03)
   sort_product   (dRabcd03)                                                                                # cdb(dRabcd03.506,dRabcd03)
   rename_dummies (dRabcd03)                                                                                # cdb(dRabcd03.507,dRabcd03)

   substitute (dRabcd04,$A^{c}A^{b}A^{e}A^{f}\partial_{c e f}{\Gamma^{a}_{d b}} -> @(dGamma03)$,repeat=True)  # cdb(dRabcd04.501,dRabcd04)
   substitute (dRabcd04,$A^{c}A^{b}A^{e}A^{f}\partial_{c e f}{\Gamma^{a}_{b d}} -> @(dGamma03)$,repeat=True)  # cdb(dRabcd04.502,dRabcd04)
   substitute (dRabcd04,$A^{c}A^{b}A^{e}\partial_{c e}{\Gamma^{a}_{d b}} -> @(dGamma02)$,repeat=True)         # cdb(dRabcd04.503,dRabcd04)
   substitute (dRabcd04,$A^{c}A^{b}A^{e}\partial_{c e}{\Gamma^{a}_{b d}} -> @(dGamma02)$,repeat=True)         # cdb(dRabcd04.504,dRabcd04)
   substitute (dRabcd04,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{d b}} -> @(dGamma01)$,repeat=True)                # cdb(dRabcd04.505,dRabcd04)
   substitute (dRabcd04,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{b d}} -> @(dGamma01)$,repeat=True)                # cdb(dRabcd04.506,dRabcd04)
   distribute (dRabcd04)                                                                                # cdb(dRabcd04.507,dRabcd04)
   sort_product   (dRabcd04)                                                                                # cdb(dRabcd04.508,dRabcd04)
   rename_dummies (dRabcd04)                                                                                # cdb(dRabcd04.509,dRabcd04)

   substitute (dRabcd05,$A^{c}A^{b}A^{e}A^{f}A^{g}\partial_{c e f g}{\Gamma^{a}_{d b}} -> @(dGamma04)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{b}A^{e}A^{f}A^{g}\partial_{c e f g}{\Gamma^{a}_{b d}} -> @(dGamma04)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{b}A^{e}A^{f}\partial_{c e f}{\Gamma^{a}_{d b}} -> @(dGamma03)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{b}A^{e}A^{f}\partial_{c e f}{\Gamma^{a}_{b d}} -> @(dGamma03)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{b}A^{e}\partial_{c e}{\Gamma^{a}_{d b}} -> @(dGamma02)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{b}A^{e}\partial_{c e}{\Gamma^{a}_{b d}} -> @(dGamma02)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{d b}} -> @(dGamma01)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{b}\partial_{c}{\Gamma^{a}_{b d}} -> @(dGamma01)$,repeat=True)
   distribute (dRabcd05)
   sort_product   (dRabcd05)
   rename_dummies (dRabcd05)

   end_stage_5 = time.time()
\end{cadabra}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd01.500} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd02.500} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.501} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.502} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.503} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.504} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.505} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd03.500} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.501} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.502} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.503} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.504} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.505} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.506} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.507} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd04.500} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.501} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.502} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.503} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.504} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.505} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.506} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.507} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.508} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.509} \end{dmath*}
\end{dgroup*}

\clearpage

% =================================================================================================
\section*{Stage 6: Replace partial derivs of $R$ with covariant derivs of $R$}

\begin{cadabra}
   # now eliminate remaining partial derivs of Rabcd by substitution from the lower order dRabcd

   # note that
   #   dRabcd01 = R^a_{cdb,e} A^c A^d A^e
   #   dRabcd02 = R^a_{cdb,ef} A^c A^d A^e A^f
   #   dRabcd03 = R^a_{cdb,efg} A^c A^d A^e A^f A^g

   # thus we can use
   #   dRabcd01 to eliminate 1st partial derivs of R in dRabcd03, dRabcd04, etc.
   #   dRabcd02 to eliminate 2nd partial derivs of R in dRabcd04, dRabcd05, etc.
   #   dRabcd03 to eliminate 3rd partial derivs of R in dRabcd05, dRabcd06, etc.

   beg_stage_6 = time.time()

   substitute (dRabcd03,$A^{c}A^{d}A^{e}\partial_{e}{R^{a}_{c d b}} -> @(dRabcd01)$,repeat=True)         # cdb(dRabcd03.601,dRabcd03)
   distribute (dRabcd03)                                                                                # cdb(dRabcd03.602,dRabcd03)

   # note: dRabcd04 and dRabcd05 unused in this code (or any other code)

   substitute (dRabcd04,$A^{c}A^{d}A^{e}A^{f}\partial_{e f}{R^{a}_{c d b}} -> @(dRabcd02)$,repeat=True)  # cdb(dRabcd04.601,dRabcd04)
   substitute (dRabcd04,$A^{c}A^{d}A^{e}\partial_{e}{R^{a}_{c d b}} -> @(dRabcd01)$,repeat=True)         # cdb(dRabcd04.602,dRabcd04)
   distribute (dRabcd04)                                                                                # cdb(dRabcd04.603,dRabcd04)

   substitute (dRabcd05,$A^{c}A^{d}A^{e}A^{f}A^{g}\partial_{e f g}{R^{a}_{c d b}} -> @(dRabcd03)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{d}A^{e}A^{f}\partial_{e f}{R^{a}_{c d b}} -> @(dRabcd02)$,repeat=True)
   substitute (dRabcd05,$A^{c}A^{d}A^{e}\partial_{e}{R^{a}_{c d b}} -> @(dRabcd01)$,repeat=True)
   distribute (dRabcd05)

   end_stage_6 = time.time()
\end{cadabra}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd03.601} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.602} \end{dmath*}
\end{dgroup*}

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd04.601} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.602} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.603} \end{dmath*}
\end{dgroup*}

\clearpage

% =================================================================================================
\section*{Stage 7: Reformatting}

\begin{cadabra}
   beg_stage_7 = time.time()

   dRabcd01 = flatten_Rabcd (dRabcd01)  # cdb(dRabcd01.701,dRabcd01)
   dRabcd02 = flatten_Rabcd (dRabcd02)  # cdb(dRabcd02.701,dRabcd02)
   dRabcd03 = flatten_Rabcd (dRabcd03)  # cdb(dRabcd03.701,dRabcd03)
   dRabcd04 = flatten_Rabcd (dRabcd04)  # cdb(dRabcd04.701,dRabcd04)
   dRabcd05 = flatten_Rabcd (dRabcd05)  # cdb(dRabcd05.701,dRabcd05)

   canonicalise (dRabcd01)   # cdb(dRabcd01.702,dRabcd01)
   canonicalise (dRabcd02)   # cdb(dRabcd02.702,dRabcd02)
   canonicalise (dRabcd03)   # cdb(dRabcd03.702,dRabcd03)
   canonicalise (dRabcd04)   # cdb(dRabcd04.702,dRabcd04)
   canonicalise (dRabcd05)   # cdb(dRabcd05.702,dRabcd05)

   end_stage_7 = time.time()

   # cdbBeg (timing)
   print ("Stage 1: {:7.1f} secs\\hfill\\break".format(end_stage_1-beg_stage_1))
   print ("Stage 2: {:7.1f} secs\\hfill\\break".format(end_stage_2-beg_stage_2))
   print ("Stage 3: {:7.1f} secs\\hfill\\break".format(end_stage_3-beg_stage_3))
   print ("Stage 4: {:7.1f} secs\\hfill\\break".format(end_stage_4-beg_stage_4))
   print ("Stage 5: {:7.1f} secs\\hfill\\break".format(end_stage_5-beg_stage_5))
   print ("Stage 6: {:7.1f} secs\\hfill\\break".format(end_stage_6-beg_stage_6))
   print ("Stage 7: {:7.1f} secs".format(end_stage_7-beg_stage_7))
   # cdbEnd (timing)

\end{cadabra}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd01.701} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.701} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.701} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.701} \end{dmath*}
   % \begin{dmath*} \cdb*{dRabcd05.701} \end{dmath*}
\end{dgroup*}

\clearpage

\begin{dgroup*}
   \begin{dmath*} \cdb*{dRabcd01.702} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd02.702} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd03.702} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd04.702} \end{dmath*}
   \begin{dmath*} \cdb*{dRabcd05.702} \end{dmath*}
\end{dgroup*}

\clearpage

\begin{cadabra}
   cdblib.create ('dRabcd.json')

   cdblib.put ('dRabcd01',dRabcd01,'dRabcd.json')
   cdblib.put ('dRabcd02',dRabcd02,'dRabcd.json')
   cdblib.put ('dRabcd03',dRabcd03,'dRabcd.json')
   cdblib.put ('dRabcd04',dRabcd04,'dRabcd.json')
   cdblib.put ('dRabcd05',dRabcd05,'dRabcd.json')

\end{cadabra}

% =================================================================================================
% the remaining code is just for pretty printing

\clearpage

\begin{cadabra}
   # note: keeping numbering as is (out of order) to ensure R appears before \nabla R etc.
   def product_sort (obj):
       substitute (obj,$ A^{a}                            -> A001^{a}               $)
       substitute (obj,$ x^{a}                            -> A002^{a}               $)
       substitute (obj,$ g^{a b}                          -> A003^{a b}             $)
       substitute (obj,$ \nabla_{e f g h}{R_{a b c d}}    -> A008_{a b c d e f g h} $)
       substitute (obj,$ \nabla_{e f g}{R_{a b c d}}      -> A007_{a b c d e f g}   $)
       substitute (obj,$ \nabla_{e f}{R_{a b c d}}        -> A006_{a b c d e f}     $)
       substitute (obj,$ \nabla_{e}{R_{a b c d}}          -> A005_{a b c d e}       $)
       substitute (obj,$ R_{a b c d}                      -> A004_{a b c d}         $)
       sort_product   (obj)
       rename_dummies (obj)
       substitute (obj,$ A001^{a}                  -> A^{a}                         $)
       substitute (obj,$ A002^{a}                  -> x^{a}                         $)
       substitute (obj,$ A003^{a b}                -> g^{a b}                       $)
       substitute (obj,$ A004_{a b c d}            -> R_{a b c d}                   $)
       substitute (obj,$ A005_{a b c d e}          -> \nabla_{e}{R_{a b c d}}       $)
       substitute (obj,$ A006_{a b c d e f}        -> \nabla_{e f}{R_{a b c d}}     $)
       substitute (obj,$ A007_{a b c d e f g}      -> \nabla_{e f g}{R_{a b c d}}   $)
       substitute (obj,$ A008_{a b c d e f g h}    -> \nabla_{e f g h}{R_{a b c d}} $)

       return obj

   def reformat (obj,scale):
       foo  = Ex(str(scale))
       bah := @(foo) @(obj).
       distribute     (bah)
       bah = product_sort (bah)
       rename_dummies (bah)
       canonicalise   (bah)
       factor_out     (bah,$A^{a?}$)
       ans := @(bah).
       return ans

   scaled1 = reformat (dRabcd01, 1)    # cdb(scaled1.601,scaled1)
   scaled2 = reformat (dRabcd02, 1)    # cdb(scaled2.601,scaled2)
   scaled3 = reformat (dRabcd03,-2)    # cdb(scaled3.601,scaled3)
   scaled4 = reformat (dRabcd04,-5)    # cdb(scaled4.601,scaled4)
   scaled5 = reformat (dRabcd05,-3)    # cdb(scaled5.601,scaled5)

\end{cadabra}

\clearpage

% =================================================================================================
\section*{Symmetrised partial derivatives of $R^a{}_{bcd}$}
\begin{dgroup*}
   \begin{dmath*}    A^c A^d A^e R^a{}_{cdb,e} = \cdb{scaled1.601} \end{dmath*}
   \begin{dmath*}    A^c A^d A^e A^{f} R^a{}_{cdb,ef} = \cdb{scaled2.601} \end{dmath*}
   \begin{dmath*} -2 A^c A^d A^e A^{f} A^{g} R^a{}_{cdb,efg} = \cdb{scaled3.601} \end{dmath*}
   \begin{dmath*} -5 A^c A^d A^e A^{f} A^{g} A^{h} R^a{}_{cdb,efgh} = \cdb{scaled4.601} \end{dmath*}
   \begin{dmath*} -3 A^c A^d A^e A^{f} A^{g} A^{h} A^{i}R^a{}_{cdb,efghi} = \cdb{scaled5.601} \end{dmath*}
\end{dgroup*}

% LCB: only need dRabcd01,02,03. So I could save time by not
%      computing dRabcd04,05

% computing just dRabcd0n to n=4 takes about 1 min 40 sec.
% but going to n=5 takes about 7 min

\clearpage

% =================================================================================================
% export selected objects, these will later be imported into a library
% these are the objects that will appear in the paper

\begin{cadabra}
   substitute (scaled1,$A^{a}->1$)
   substitute (scaled2,$A^{a}->1$)
   substitute (scaled3,$A^{a}->1$)
   substitute (scaled4,$A^{a}->1$)
   substitute (scaled5,$A^{a}->1$)

   cdblib.create ('dRabcd.export')

   # 6th order dRabcd, scaled
   cdblib.put ('dRabcd61scaled',scaled1,'dRabcd.export')
   cdblib.put ('dRabcd62scaled',scaled2,'dRabcd.export')
   cdblib.put ('dRabcd63scaled',scaled3,'dRabcd.export')
   cdblib.put ('dRabcd64scaled',scaled4,'dRabcd.export')
   cdblib.put ('dRabcd65scaled',scaled5,'dRabcd.export')

   checkpoint.append (scaled1)
   checkpoint.append (scaled2)
   checkpoint.append (scaled3)
   checkpoint.append (scaled4)
   checkpoint.append (scaled5)

\end{cadabra}

\clearpage

% =================================================================================================
\section*{Timing}

\cdb{timing}

% =================================================================================================
% export checkpoints in json format

\bgroup
\CdbSetup{action=hide}
\begin{cadabra}
   for i in range( len(checkpoint) ):
      cdblib.put ('check{:03d}'.format(i),checkpoint[i],checkpoint_file)
\end{cadabra}
\egroup

\end{document}


% BEGIN LICENSE BLOCK
% Version: CMPL 1.1
%
% The contents of this file are subject to the Cisco-style Mozilla Public
% License Version 1.1 (the "License"); you may not use this file except
% in compliance with the License.  You may obtain a copy of the License
% at www.eclipse-clp.org/license.
%
% Software distributed under the License is distributed on an "AS IS"
% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied.  See
% the License for the specific language governing rights and limitations
% under the License.
%
% The Original Code is  The ECLiPSe Constraint Logic Programming System.
% The Initial Developer of the Original Code is  Cisco Systems, Inc.
% Portions created by the Initial Developer are
% Copyright (C) 1999 - 2006 Cisco Systems, Inc.  All Rights Reserved.
%
% Contributor(s): Joachim Schimpf, IC-Parc
%
% END LICENSE BLOCK
%
% $Id: umscalendar.tex,v 1.1 2006/09/23 01:50:40 snovello Exp $
%
% Joachim Schimpf, IC-Parc
%

\section{Calendar Library}
\label{chapcal}
This library contains a set of predicates to assist with the handling
of dates and times.  It is loaded using
\begin{quote}\begin{verbatim}
:- use_module(library(calendar)).
\end{verbatim}\end{quote}
The library represents time points as {\em Modified Julian Dates} (MJD).
Julian Dates (JD) and Modified Julian Dates (MJD) are a
consecutive day numbering scheme widely used in astronomy,
space travel etc.
That means that every day has a unique integer number, and consecutive
days have consecutive numbers.
Note that you can also use fractional MJDs to denote the time of day.
Then every time point has a unique floating point representation!
With this normalised representation, distances between times are
obviously trivial to compute, and so are weekdays (by simple mod(7)
operation).

The predicates provided are
\begin{description}
\item[date_to_mjd(+D/M/Y, -MJD)] converts a Day/Month/Year structure into
	its unique integer MJD number.
\item[mjd_to_date(+MJD, -D/M/Y)] converts an MJD
	(integer or float) into the corresponding Day/Month/Year.
\item[time_to_mjd(+H:M:S, -MJD)] returns a float MJD \lt 1.0 encoding the
	time of day (UTC/GMT). This can be added to an integral day number
	to obtain a full MJD.
\item[mjd_to_time(+MJD, -H:M:S)] returns the time of day (UTC/GMT)
	corresponding to the given MJD as Hour:Minute:Seconds structure,
	where Hour and Minute are integers and Seconds is a float.
\item[mjd_to_weekday(+MJD, -DayName)] returns the weekday of the
	specified MJD as atom monday, tuesday etc.
\item[mjd_to_dow(+MJD, -DoW)] returns the weekday of the
	specified MJD as an integer (1 for monday up to 7 for sunday).
\item[mjd_to_dow(+MJD, +FirstWeekday, -DoW)] as above, but allows to choose
	a different starting day for weeks, specified as atom monday,
	tuesday etc.
\item[mjd_to_dy(+MJD, -DoY/Y), dy_to_mjd(+DoY/Y, -MJD)] convert MJDs
	to or from a DayOfYear/Year representation, where DayOfYear is
	the relative day number starting with 1 on every January 1st.
\item[mjd_to_dwy(+MJD, -DoW/WoY/Y), dwy_to_mjd(+DoW/WoY/Y, -MJD)] convert
	MJDs to or from a DayOfWeek/WeekOfYear/Year representation, where
	DayOfWeek is the day number within the week (1 for monday up to
	7 for sunday), and WeekOfYear is the week number within the year
	(starting with 1 for the week that contains January 1st).
\item[mjd_to_dwy(+MJD, +FirstWeekday, -DoW/WoY/Y)]
\item[dwy_to_mjd(+DoW/WoY/Y, +FirstWeekday, -MJD)]
	As above, but allows to choose a different starting day for weeks,
	specified as atom monday, tuesday etc.
\item[unix_to_mjd(+UnixSec, -MJD)] convert the UNIX time representation
	into a (float) MJD.
\item[mjd_now(-MJD)] returns the current date/time as (float) MJD.
\item[jd_to_mjd(+JD, -MJD), mjd_to_jd(+MJD, -JD)] convert MJDs to or
	from JDs. The relationship is simply MJD = JD-2400000.5.
\end{description}
The library code is valid for dates between
	 1 Mar 0004 = MJD -677422 = JD 1722578.5
and
	22 Jan 3268 = MJD  514693 = JD 2914693.5.

\subsection{Examples}
What day of the week was the 29th of December 1959?
\begin{quote}\begin{verbatim}
[eclipse 1]: lib(calendar).
[eclipse 2]: date_to_mjd(29/12/1959, MJD), mjd_to_weekday(MJD,W).
MJD = 36931
W = tuesday
\end{verbatim}\end{quote}
What date and time is it now?
\begin{quote}\begin{verbatim}
[eclipse 3]: mjd_now(MJD), mjd_to_date(MJD,Date), mjd_to_time(MJD,Time).
Date = 19 / 5 / 1999
MJD = 51317.456238425926
Time = 10 : 56 : 59.000000017695129
\end{verbatim}\end{quote}
How many days are there in the 20th century?
\begin{quote}\begin{verbatim}
[eclipse 4]: N is date_to_mjd(1/1/2001) - date_to_mjd(1/1/1901).
N = 36525
\end{verbatim}\end{quote}
The library code does not detect invalid dates,
but this is easily done by converting a date to its MJD and back
and checking whether they match:
\begin{quote}\begin{verbatim}
[eclipse 5]: [user].
valid_date(Date) :-
        date_to_mjd(Date,MJD),
        mjd_to_date(MJD,Date).

[eclipse 6]: valid_date(29/2/1900).   % 1900 is not a leap year!
no (more) solution.
\end{verbatim}\end{quote}