\documentclass{article}
\usepackage{expsagetex}
\usepackage{amsmath}
\usepackage{parskip}
\usepackage{shortvrb}
\usepackage{csquotes}
\usepackage{expl3}
\usepackage[svgnames]{xcolor}
\usepackage{tikz}
\usepackage{xfp}
\usepackage{siunitx}
\sisetup{group-separator = \text{\,}}
\usepackage{hyperref}
\hypersetup{colorlinks=true, anchorcolor=Lime, linkcolor=RoyalBlue,
            urlcolor=Crimson}

\title{%
  \pkg{expsagetex} -- a fully expandable interface for SageTeX%
  \thanks{This file describes \pkg{expsagetex}~v0.1, last revised 2020-01-11.}%
}
\author{%
  Florent Rougon%
  \thanks{%
    E-mail: \href{mailto:f.rougon@free.fr}{mailto:f.rougon@free.fr}%
  }%
}

\newcommand*{\pkg}{\textsf}
\newcommand*{\meta}[1]{$\langle \textsl{#1} \rangle$}
\newcommand*{\tikzname}{Ti\emph{k}Z}

\setlength{\parindent}{0pt}     % will probably look better here

\begin{document}

\maketitle
\begin{abstract}
  This package is based on \texttt{sagetex.sty}, the style file for SageTeX.
  One deficiency of current \texttt{sagetex.sty} (version~3.3 from 2019/01/09)
  is that it doesn't provide any user-accessible way to retrieve a result
  computed by Sage in places where only expansion happens. This packages uses
  the low-level SageTeX machinery to implement a set of commands that can
  \enquote{record} results computed by Sage and a fully expandable command
  that can yield any such result wherever you need it, even in expansion-only
  contexts.
\end{abstract}

\MakeShortVerb{\|}

\tableofcontents
\clearpage

Besides the \nameref{sec:quick-start} section below, there is introductory
material to \pkg{expsagetex}
\href{https://tex.stackexchange.com/questions/521319/sagetex-1000sep-for-sage-calculated-number-siunitx/521389#521389}{on
  TeX.stackexchange.com}.

\section{Quick start}
\label{sec:quick-start}

Getting a result from Sage using \pkg{expsagetex} involves three steps in the
\LaTeX\ code:
\begin{enumerate}
\item Recording a Python expression to be written to the
  \texttt{.sagetex.sage} file. This step is done with one of the |\est*Record|
  functions.
\item Using the result where you need it. This is done with |\estGet| and
  works in many places, in particular in expansion-only contexts (inside
  |\edef|, |\message|, |\write|, etc.).
\item Declaring that you use the result, so that a warning can be displayed if
  the result isn't available yet (|\estGet| can't print the warning itself
  because it must work in expansion-only contexts). This is done with
  |\estRefUsed|.
\end{enumerate}

Step~3 can be done before or after step~2, but must come after step~1. When
you run |sage| on the \texttt{.sagetex.sage} file written by step~1, the
Python expression is evaluated, converted to a Python string and written to
the |.sagetex.sout| file. This file is in turn read by \texttt{sagetex.sty} at
startup, which makes the result available for |\estGet|.

Thus, a very simple example can be
\estRecordFormatted{\littleFermatExple}{(45**6) \percent 7}%
\estRefUsed{\littleFermatExple}%
$45^6 \equiv \estGet{\littleFermatExple} \pmod 7$. This was obtained with:
\begin{verbatim}
\estRecordFormatted{\littleFermatExple}{(45**6) \percent 7}%
\estRefUsed{\littleFermatExple}%
$45^6 \equiv \estGet{\littleFermatExple} \pmod 7$
\end{verbatim}

For compiling, use the same method as with \pkg{sagetex} (which
\pkg{expsagetex} relies on): you need one \LaTeX\ run, one |sage| run on the
\texttt{.sagetex.sage} file followed by at least one more \LaTeX\ run (if more
are needed, warnings are printed by \LaTeX\ as usual).

\section{Examples}

\subsection{Integers}

\subsubsection{Local assignments}

When using |\estRecordFormatted{|\meta{macro}|}{|\meta{expr}|}| with a Python
expression \meta{expr} that evaluates to an integer,
|\estGet[|\meta{fallback}|]{|\meta{macro}|}| should give (i.e., expand to) the
expected result. This is because calling |str()| on a Python integer doesn't
use any exponent notation, thus |latex()| should not add any formatting. Let's
execute the following Python statement using the |sageblock| environment:

\begin{sageblock}
x = 2**333
\end{sageblock}

Now, record the value of $x$ and associate it with macro |\mymacro|. This
makes |\mymacro| a kind of reference to the saved value, but to retrieve the
tokens associated to this reference, you have to use |\estGet| instead of
|\ref| (this can be done in an expansion-only context like inside |\edef|,
which is impossible with |\ref|). See section~\ref{sec:technical-details}
below for more details on this.
{%
  \estRecordFormatted{\mymacro}{x}%
  Recover the saved value and format it with the |\num| macro of
  \pkg{siunitx}:\\
  \num{\estGet[-1]{\mymacro}}.%
  \estRefUsed{\mymacro}\\
  This was achieved with the following calls:
\begin{verbatim}
\estRecordFormatted{\mymacro}{x}
\num{\estGet[-1]{\mymacro}}
\estRefUsed{\mymacro}
\end{verbatim}
}%
\ifx\mymacro\undefined\else
  \par Bug: |\mymacro| should be undefined here!
\fi

Another way to do the same without having Sage call |latex()| at all is to
use |\estRecordStr| to record a string representation of the integer we are
interested in:
\begin{verbatim}
\estRecordStr{\mymacro}{str(x)}
\num{\estGet[-1]{\mymacro}}
\estRefUsed{\mymacro}
\end{verbatim}

\pagebreak[3]

Notes:
\begin{itemize}
\item |\estGet| wraps its return value within |\unexpanded| (|\exp_not:n|),
  which implies that it won't expand further when used in |\edef| or an
  \pkg{expl3} |x|-type argument.
\item In order to use the result of evaluating a Python expression in the
  \LaTeX\ document, it is not necessary to store it in a Python variable.
  Example:%
  \estRecordFormatted{\twoToTheThenth}{2**10}%
  \estRefUsed{\twoToTheThenth}%
  \[ 2^{10} = \estGet{\twoToTheThenth} \]
\end{itemize}

Another example with variables:

\begin{sageblock}
x = 23 % 8         # That is, x = 7.
y = 2*x
\end{sageblock}

Save the value of $1000000\times (y-10)$ (as computed by Sage) and associate
it with macro |\numberBasedOnY|.
\estRecordFormatted{\numberBasedOnY}{1000000*(y-10)}%
% Use the value -1000000 when Sage hasn't written the result to the .sout file
% yet.
\[ 1000000\times (y - 10) = \num{\estGet[-1000000]{\numberBasedOnY}} \]
\estRefUsed{\numberBasedOnY}

\subsubsection{Global assignments}

While |\estRecordFormatted| assigns locally to the macro given by its
first argument, all |\est*Record| functions have a global variant that
performs a global assignment instead. The global variant of
|\estRecordFormatted| is |\estGRecordFormatted|, that of
|\estRecordStr| is |\estGRecordStr|, that of
|\estARecordFormatted| is |\estGARecordFormatted|, etc. Example:

\begin{sageblock}
x = 3+4
\end{sageblock}

Save the value of $x$ and globally assign macro |\Iamseven| to allow us
to retrieve the saved value.
{%
  \estGRecordFormatted{\Iamseven}{x}%
}%
Get it back from outside the group where this was done, and format it with
|\num|: the result is
\num{\estGet[-2]{\Iamseven}}.%
\estRefUsed{\Iamseven}

\subsection{Floating point numbers}

|\estRecordFormatted| works with floating point numbers, but it uses \LaTeX\
markup if the string representation of the result (in Python) uses exponent
notation. When this is not desirable---i.e., when you just want a numerical
result without any formatting---use |\estRecordFloat| or one of its variants.
Example:
\begin{sageblock}
x = 3.1415927
\end{sageblock}
\estRecordFloat{\PiRoundedByPython}{x}{.2f}%
\estRefUsed{\PiRoundedByPython}%
$x$ rounded to two decimal places is \estGet[0]{\PiRoundedByPython}. This was
rounded by Python.
\estRecordFloat{\PiRoundedBySiunitx}{x}{.10f}%
\estRefUsed{\PiRoundedBySiunitx}%
Thanks to the expandable nature of |\estGet|, one can also do this: $x$
rounded to four decimal places is
\num[round-mode=places, round-precision=4]{\estGet[0]{\PiRoundedBySiunitx}}.
This was rounded by \pkg{siunitx}.

Of course, one can also compute a float with Sage without storing it into a
Python variable. Here is an approximation of $\pi$ using an expression based
on Machin's formula and the power series expansion of $\arctan$:
\estRecordFloat{\PiViaMachinAndPowerSeries}
  {
    N(4*sum([ (-1)**n / (2*n+1) * (4*(1/5)**(2*n+1) - (1/239)**(2*n+1))
              for n in range(10) ]))
  }{.6f}%
\estRefUsed{\PiViaMachinAndPowerSeries}%
\begin{align*}
  \pi &= 4\sum_{n=0}^{+\infty} \frac{(-1)^n}{(2n+1)}
         \Biggl( 4 \, {\Bigl( \frac{1}{5} \Bigr)}^{2n+1} \! -
                 {\Bigl( \frac{1}{239} \Bigr)}^{2n+1} \Biggr)\\
      &\approx \estGet[0]{\PiViaMachinAndPowerSeries}
\end{align*}

\subsection{Strings}

Functions working on Python strings are the most general here. For instance,
|\estRecordFormatted| first evaluates |latex(...)| in Sage (Python), where
|...| is the Python expression you entered; this results in a string with
\LaTeX\ math-mode formatting commands (e.g., if evaluating |str(...)| in Python string yields something containing brackets or
exponents); then this string is written to the |.sout| file inside an
argument of |\newlabel|, which \pkg{sagetex} reads at startup. Similarly,
|\estRecordFloat| causes Python to convert a float to a string using the
user-specified format, then this string is written to the |.sout| file
inside an argument of |\newlabel|, just as with
|\estRecordFormatted|. This applies to global and array variants of the
\pkg{expsagetex} functions as well.

So, |\estRecordStr| and its variants can be very useful in case your
Python code returns a data type that isn't handled in a satisfactory way by
functions belonging to the families of |\estRecordFormatted| or
|\estRecordFloat|. All you have to do is to make your Python code format
your data as a string with a syntax that is convenient for the \LaTeX\
document; you record this string with |\estRecordStr| or one of its
variants, use |\estGet| and |\estRefUsed| as usual and voilà, your new data
type is nicely handled by \pkg{expsagetex}. We'll see an example of this
approach in
section~\ref{sec:plotting-with-tikzname-based-on-Sage-computations}, where
Python code generates a large amount of $(x_i, y_i)$ coordinates from two
lists of floats $[x_1, \dotsc, x_n]$ and $[y_1, \dotsc, y_n]$. The Python code
formats the list of $(x_i, y_i)$ coordinates in a form that is suitable for
the \tikzname\ \texttt{plot coordinates} operation, which allows one to plot
the data right away.

Here is a very simple example with |\estRecordStr| used to record a Sage
(Python) string:
\begin{sageblock}
 some_string = "abc  def"
\end{sageblock}
\estRecordStr{\somestring}{some_string}%
X\estGet{\somestring}Y%
\estRefUsed{\somestring}

The two spaces between |abc| and |def| are present in the
|.sout| file, but get coalesced into a single space token when
|\newlabel| tokenizes its second argument. If this is not desired, the
easiest way is probably to use another character instead of space (e.g.,
\verb|~|).

\subsection{Using array data}
\label{sec:using-array-data}

\pkg{expsagetex} offers variants of |\estRecordFormatted|,
|\estGRecordFormatted|, |\estRecordFloat|, |\estGRecordFloat|, |\estRecordStr|
and |\estGRecordStr| that allow one to easily use an index inside the macro
name given for the recording operation. These variants all have the letter
\texttt{A} (standing for ``array'') right before \texttt{Record}.

Where the non-array versions of the |\est*Record*| commands take one argument
specifying the destination macro name, the array versions accept two arguments
\meta{base} and \meta{index} and determine the destination macro name as the
result of expanding |\csname |\meta{base}@\meta{index}|\endcsname|. As a
consequence, the \meta{base} and \meta{index} arguments are recursively
expanded, which can be used to dynamically compute a numeric index using for
instance |\numexpr|, or whatever you want.

Here is an example that uses Python to evaluate the elements of a Vandermonde
matrix:
\ExplSyntaxOn
\int_new:N \l__my_row_int
\int_new:N \l__my_col_int
\int_new:N \g__my_row_counter_int
\clist_const:Nn \c__my_clist { 2, sqrt(2)/2, 2/7, 4/3, 1/5 }
\clist_const:Nn \c__my_formatted_clist
  { 2, \frac{\sqrt{2}}{2}, \frac{2}{7}, \frac{4}{3}, \frac{1}{5} }
\seq_new:N \l__my_matrix_rows_seq

\int_step_variable:nnNn { 1 } { 5 } \l__my_row_int
  {
    % One l3seq per matrix row
    \seq_clear_new:c { l__my_matrix_row \l__my_row_int _seq }
    % \use:c needs two expansion steps to do its work
    \exp_args:NNo \seq_put_right:No \l__my_matrix_rows_seq
      { \use:c { l__my_matrix_row \l__my_row_int _seq } }

    \int_step_variable:nnNn { 1 } { 5 } \l__my_col_int
      {
        \tl_set:Nx \l_tmpa_tl
          {
            ( \clist_item:Nn \c__my_clist { \l__my_row_int } ) **
                                                          ( \l__my_col_int - 1 )
          }
        \estARecordFormatted { Vandermonde } { \l__my_row_int - \l__my_col_int }
          { \tl_use:N \l_tmpa_tl }
        \seq_put_right:cx { l__my_matrix_row \l__my_row_int _seq }
          { \estAGet [0] { Vandermonde } { \l__my_row_int - \l__my_col_int } }
        \estARefUsed { Vandermonde } { \l__my_row_int - \l__my_col_int }
      }
  }

\[ \renewcommand{\arraystretch}{1.2}
  \mathcal{V} \Bigl( \clist_use:Nn \c__my_formatted_clist { , } \Bigr) =
   \begin{bmatrix}
     \int_gzero:N \g__my_row_counter_int
     \seq_map_inline:Nn \l__my_matrix_rows_seq
       {
         \int_gincr:N \g__my_row_counter_int
         \seq_use:Nn #1 { & }
         % Append \\ unless it is the last line
         \int_compare:nNnT
           { \g__my_row_counter_int } < { \seq_count:N \l__my_matrix_rows_seq }
           { \\ }
       }
   \end{bmatrix} \]
\ExplSyntaxOff

This technique can be used to plot curves with \tikzname\ based on coordinates
computed by Python, but this is rather wasteful. We'll see a better method
in section~\ref{sec:plotting-with-tikzname-based-on-Sage-computations}.

\subsection{Displaying lists}
\label{sec:lists}

|\estRecordFormatted| is fine for a list of integers:%
\estRecordFormatted{\listA}{list(range(2, 21, 3))}%
\begin{quote}
  \estRefUsed{\listA}%
  $\estGet[{[]}]{\listA}$
\end{quote}

It may do what you want for floats too:
\estRecordFormatted{\listB}{map(float, list(range(2, 21, 3)))}%
\begin{quote}
  \estRefUsed{\listB}%
  $\estGet[{[]}]{\listB}$
\end{quote}

But beware: if you use |\estRecordFormatted|, floats are formatted by Sage's
|latex()| function:
\estRecordFormatted{\listC}{[1e-10] + [5.0, 3e-14]}
\begin{quote}
  \small
  \estRefUsed{\listC}%
  $\estGet[{[]}]{\listC}$
\end{quote}

Here is a way to use a Python list of floats formatted with no exponent
and using a chosen number of decimal places:%
% We format each float individually as a string. We could as well omit the
% outer brackets: they are just string characters here (the final assembled
% expression is a Python string; it is *not* evaluated as a list by Python).
\estRecordStr{\listD}
  {
    '[' +
    ', '.join(map(lambda x: "\percent.14f" \percent (x,), [1e-10, 5.0, 3e-14])) +
    ']'
  }
\begin{quote}
  \estRefUsed{\listD}%
  $\estGet[{[]}]{\listD}$
\end{quote}

\subsection{Plotting with \tikzname\ based on Sage computations}
\label{sec:plotting-with-tikzname-based-on-Sage-computations}

Plotting functions based on coordinates computed by Sage can be done using
array-style functions of \pkg{expsagetex} (see
section~\ref{sec:using-array-data}), however such an approach is vastly
inefficient if the plot uses a significant number of points. A better approach
is to prepare one string on the Python side containing all needed coordinates
in a format that the \LaTeX\ plotting code can easily handle. This way, only
one reference---as in |\label| and |\ref|---is used to pass data for the whole
plot. As an example, here is a part of the Euler spiral where all coordinates
are computed by Sage and transferred in one go to the \LaTeX\ side (for the
full Euler Spiral, you need to make $t$ vary from $-\infty$ to $+\infty$).
%
\begin{sagesilent}
  x = var('x')
  start = -5*pi/2
  end = 5*pi/2
  nb_samples = 2000
  delta = float(end-start)/(nb_samples-1)
\end{sagesilent}
%
\estRecordStr{\computedCoordsRef}
  {
    " ".join([ "(\percent.6f, \percent.6f)" \percent
               (0.1*( numerical_integral(cos(x**2), 0, t)[0] ),
                0.1*( numerical_integral(sin(x**2), 0, t)[0] ))
               for t in ( start + _*delta for _ in range(nb_samples) ) ])
  }
\edef\computedCoords{\estGet[(0,0)]{\computedCoordsRef}}
\estRefUsed{\computedCoordsRef}
%
\begin{center}
\begin{tabular}{cc}
\raisebox{-0.5\height}{%
  \begin{tikzpicture}
    \draw[red!60!black, scale=20] plot coordinates { \computedCoords };
  \end{tikzpicture}%
}\hspace*{0.06\linewidth} &
\hspace*{0.06\linewidth}%
$ \newcommand{\diff}{\mathop{}\!\mathrm{d}}
  \left\{
    \begin{array}{>{\displaystyle}l}
      x(t) = \int_{0}^{t} \cos s^2 \diff s\\[10pt]
      y(t) = \int_{0}^{t} \sin s^2 \diff s
    \end{array}
  \right. ,\ t \in [-\frac{5\pi}{2}, \frac{5\pi}{2}]$
\end{tabular}
\end{center}

\section{Expandable commands work everywhere!}

Since |\estGet| is expandable and we have |\usepackage{xfp}| in the preamble,
we can do:
\begin{verbatim}
$\fpeval{3*\estGet[-1]{\Iamseven}}$
\end{verbatim}
This expands to $\fpeval{3*\estGet[-1]{\Iamseven}}$, since |\Iamseven|
contains the last value of $x$ computed by Sage, which is $7$ (if Sage hasn't
been run yet, it expands to $-3$ due to the fallback value \texttt{-1}
specified in the optional argument of |\estGet|).%
\estRefUsed{\Iamseven}%

{%
  Given that \TeX\ expands the right-hand side of integer, dimen and glue
  assignments until this yields the proper syntactic element, one can also use
  |\estGet| like this:
  \begin{verbatim}
  \count2=4
  \multiply \count2 by \estGet[-1]{\Iamseven}
  \the\count2
  \estRefUsed{\Iamseven}% just to be clean
  \end{verbatim}
  \count2=4
  \multiply \count2 by \estGet[-1]{\Iamseven}
  $\rightarrow$ \the\count2 % prints 28 (4*7)
  \estRefUsed{\Iamseven}
}

\bigskip
Same thing with a |\dimen| register:
\begin{verbatim}
\dimen0=1pt
\dimen0=\estGet[-1]{\Iamseven}\dimen0
\the\dimen0
\estRefUsed{\Iamseven}% just to be clean
\end{verbatim}
\dimen0=1pt
\dimen0=\estGet[-1]{\Iamseven}\dimen0
$\rightarrow$ \the\dimen0 % prints 7.0pt
\estRefUsed{\Iamseven}

\newcommand*{\grammarsymbol}[1]{\mbox{\meta{#1}}}%
And since \LaTeX\ lengths are |\skipdef| tokens and |\setlength| is
basically a glue assignment, one can have fun with
|\estGet[-1]{\Iamseven}| expanding to the \grammarsymbol{factor} of
a \grammarsymbol{normal dimen} (cf.~\TeX book p.~270):
\begin{verbatim}
\dimen0=2pt
\dimen2=3pt
\newlength{\mylength}
\setlength{\mylength}{%
       \estGet[-1]{\Iamseven}\dimen0
  plus \estGet[-1]{\Iamseven}\dimen2}%
\the\mylength
\estRefUsed{\Iamseven}% just to be clean
\end{verbatim}
\dimen0=2pt
\dimen2=3pt
\newlength{\mylength}
\setlength{\mylength}{%
       \estGet[-1]{\Iamseven}\dimen0
  plus \estGet[-1]{\Iamseven}\dimen2}%
$\rightarrow$ \the\mylength % prints 14.0pt plus 21.0pt
\estRefUsed{\Iamseven}

\section{Troubleshooting}

In case something goes wrong when using \pkg{sagetex} or \pkg{expsagetex}, you
may get stuck and unable to perform a full \LaTeX\ run. This can for instance
happen if one of your Python expressions is invalid. This kind of error is
similar to problems that may occur when something ``bad'' was written to the
|.aux| file. The cure is similar too: locate the error (inspecting the
\texttt{.sagetex.sage} and \texttt{.sagetex.sout} files can help; if the
problem is an invalid Python expression, the |sage| run will tell you),
regenerate the files containing invalid things after fixing their source, then
rerun the \LaTeX\ and |sage| commands as usual. Typically, you'll fix a Python
expression in your |.tex| file, remove the |.sagetex.sout| file and rerun
\LaTeX, |sage| and again \LaTeX. If you want to be \emph{really} sure to
restart from a clean state, remove the \texttt{.sagetex.sage},
\texttt{.sagetex.sout} and |.aux| files before redoing the \LaTeX\ \& |sage|
dance.

\section{Technical details}
\label{sec:technical-details}

Some comments about what a call such as |\estRecordStr{\mymacro}{|$s$|}| as
seen below really does ($s$ must evaluate in Python to a string). Informally,
one might be tempted to say that this saves the value of $s$ in the specified
macro, but that would be rather inaccurate. After the call to |\estRecordStr|,
the specified macro contains an integer which is used to form a reference
name---a label, if you wish---through which we can get the value of $s$ using
inner gears of the |\label| and |\ref| machinery. The \texttt{.sagetex.sout}
file written when running Sage on the \texttt{.sagetex.sage} file contains
|\newlabel| commands that define the associated text (tokens) for reference
names \texttt{@sageinline0}, \texttt{@sageinline1}, etc. The trailing number
is what is really stored by the above call to |\estRecordStr| as the
replacement text of |\mymacro|. \texttt{sagetex.sty} reads this
\texttt{.sagetex.sout} file at startup when it exists, which defines the
labels from the point of view of the \LaTeX\ kernel and allows us to retrieve
the Sage output for each recorded expression (after the |\newlabel| commands
have been executed, the Sage outputs are available, after some simple data
extraction, in macros |\r@@sageinline0|, |\r@@sageinline1|, etc.).

\end{document}


\nwfilename{_src/2019/day/01.nw}\nwbegindocs{0}\newpage% ===> this file was generated automatically by noweave --- better not edit it
\section{Day 1: The Tyranny of the Rocket Equation}
\todoo{Copy description}
\marginnote{\url{https://adventofcode.com/2019/day/1}}
\nwenddocs{}\nwfilename{_src/2019/gap/01.nw}\nwbegindocs{0}\subsection{GAP Solution}

\begin{marginfigure}
\[
 \text{fuel} := \text{mass} \backslash 3 - 2
\]
\end{marginfigure}
\nwenddocs{}\nwbegincode{1}\sublabel{NW3RAD1b-3gOu99-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-1}}}\moddef{Day01.g~{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-1}}}\endmoddef\nwstartdeflinemarkup\nwprevnextdefs{\relax}{NW3RAD1b-3gOu99-2}\nwenddeflinemarkup
FuelRequiredModule := function( mass )
    return Int( Float( mass / 3 ) ) - 2;
end;;

\nwalsodefined{\\{NW3RAD1b-3gOu99-2}\\{NW3RAD1b-3gOu99-3}\\{NW3RAD1b-3gOu99-4}}\nwnotused{Day01.g}\nwendcode{}\nwbegindocs{2}\nwdocspar

\nwenddocs{}\nwbegincode{3}\sublabel{NW3RAD1b-3gOu99-2}\nwmargintag{{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-2}}}\moddef{Day01.g~{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-1}}}\plusendmoddef\nwstartdeflinemarkup\nwprevnextdefs{NW3RAD1b-3gOu99-1}{NW3RAD1b-3gOu99-3}\nwenddeflinemarkup
PartOne := function( )
    local input, line, mass, sum;;
    sum := 0;
    input := InputTextFile ( "./input/day01.txt" );
    line := ReadLine( input );
    repeat
        mass := Int( Chomp( line ) );
        sum := sum + FuelRequiredModule( mass );
        line := ReadLine( input );
    until line = fail or IsEndOfStream( input );
    return sum;
end;;

\nwendcode{}\nwbegindocs{4}\nwdocspar

\nwenddocs{}\nwbegincode{5}\sublabel{NW3RAD1b-3gOu99-3}\nwmargintag{{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-3}}}\moddef{Day01.g~{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-1}}}\plusendmoddef\nwstartdeflinemarkup\nwprevnextdefs{NW3RAD1b-3gOu99-2}{NW3RAD1b-3gOu99-4}\nwenddeflinemarkup
TotalFuelRequiredModule := function( mass )
    local fuel;;
    fuel := FuelRequiredModule( mass );
    if IsPosInt( fuel ) then
        return fuel + TotalFuelRequiredModule( fuel );
    else
        return 0;
    fi;
end;;

\nwendcode{}\nwbegindocs{6}\nwdocspar

\nwenddocs{}\nwbegincode{7}\sublabel{NW3RAD1b-3gOu99-4}\nwmargintag{{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-4}}}\moddef{Day01.g~{\nwtagstyle{}\subpageref{NW3RAD1b-3gOu99-1}}}\plusendmoddef\nwstartdeflinemarkup\nwprevnextdefs{NW3RAD1b-3gOu99-3}{\relax}\nwenddeflinemarkup
PartTwo := function( )
    local input, line, mass, sum;;
    sum := 0;
    input := InputTextFile ( "./input/day01.txt" );
    line := ReadLine( input );
    repeat
        mass := Int( Chomp( line ) );
        sum := sum + TotalFuelRequiredModule( mass );
        line := ReadLine( input );
    until line = fail or IsEndOfStream( input );
    return sum;
end;;
\nwendcode{}\nwbegindocs{8}\nwdocspar
\nwenddocs{}\nwfilename{_src/2019/day/04.nw}\nwbegindocs{0}\newpage
\section{Day 4: Secure Container}
\todoo{Copy description}
\marginnote{\url{https://adventofcode.com/2019/day/4}}
\nwenddocs{}\nwfilename{_src/2019/haskell/04.nw}\nwbegindocs{0}\subsection{Haskell Solution}

\newthought{My puzzle input} was the range \text{236491-713787}, which I converted into a
list of lists of \hs{digits}.

\nwenddocs{}\nwbegincode{1}\sublabel{NW35miTa-1GvnV-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW35miTa-1GvnV-1}}}\moddef{Input~{\nwtagstyle{}\subpageref{NW35miTa-1GvnV-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}}\nwenddeflinemarkup
getInput :: IO [[Int]]
getInput = pure $ reverse . digits 10 <$> [236491 .. 713787]
\nwused{\\{NW35miTa-15Rjc8-1}}\nwendcode{}\nwbegindocs{2}\nwdocspar

\newthought{Spoiler:} Parts One and Two vary only in the strictness of the definition of a double, so a generic solver can be parameterized by the binary operation to compare the number of adjacent digits that are the same with \hs{2}. In both parts of the puzzle, it must also be the case that the digits never decrease, i.e. the password \hs{isSorted}.

\nwenddocs{}\nwbegincode{3}\sublabel{NW35miTa-2ApJRg-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW35miTa-2ApJRg-1}}}\moddef{Generic solver~{\nwtagstyle{}\subpageref{NW35miTa-2ApJRg-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}}\nwenddeflinemarkup
solve :: (Int -> Int -> Bool) -> [[Int]] -> Int
solve = count . (isSorted <&&>) . hasDouble
  where
    hasDouble cmp = any ((`cmp` 2) . length) . group
\nwused{\\{NW35miTa-15Rjc8-1}}\nwendcode{}\nwbegindocs{4}\nwdocspar

\newthought{For Part One,} there must be a double, i.e. at least two adjacent digits that are the same.

\nwenddocs{}\nwbegincode{5}\sublabel{NW35miTa-2iOjQS-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}}\moddef{Part One~{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwprevnextdefs{\relax}{NW3AzXtx-2iOjQS-1}\nwenddeflinemarkup
\nwlinkedidentc{partOne}{NW3AzXtx-2iOjQS-4} :: [[Int]] -> Int
\nwlinkedidentc{partOne}{NW3AzXtx-2iOjQS-4} = solve (>=)
\nwalsodefined{\\{NW3AzXtx-2iOjQS-1}\\{NW3AzXtx-2iOjQS-2}\\{NW3AzXtx-2iOjQS-3}\\{NW3AzXtx-2iOjQS-4}}\nwused{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwidentuses{\\{{\nwixident{partOne}}{partOne}}}\nwindexuse{\nwixident{partOne}}{partOne}{NW35miTa-2iOjQS-1}\nwendcode{}\nwbegindocs{6}\nwdocspar

\newthought{For Part Two,} the password must have a strict double.

\nwenddocs{}\nwbegincode{7}\sublabel{NW35miTa-4P9qKy-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW35miTa-4P9qKy-1}}}\moddef{Part Two~{\nwtagstyle{}\subpageref{NW35miTa-4P9qKy-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwprevnextdefs{\relax}{NW3AzXtx-4P9qKy-1}\nwenddeflinemarkup
\nwlinkedidentc{partTwo}{NW3AzXtx-4P9qKy-1} :: [[Int]] -> Int
\nwlinkedidentc{partTwo}{NW3AzXtx-4P9qKy-1} = solve (==)
\nwalsodefined{\\{NW3AzXtx-4P9qKy-1}}\nwused{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwidentuses{\\{{\nwixident{partTwo}}{partTwo}}}\nwindexuse{\nwixident{partTwo}}{partTwo}{NW35miTa-4P9qKy-1}\nwendcode{}\nwbegindocs{8}\nwdocspar

\newthought{Bring it} all together.

\nwenddocs{}\nwbegincode{9}\sublabel{NW35miTa-15Rjc8-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW35miTa-15Rjc8-1}}}\moddef{Day04.hs~{\nwtagstyle{}\subpageref{NW35miTa-15Rjc8-1}}}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
module AdventOfCode.Year2019.Day04 where

import AdventOfCode.TH (defaultMain)
import AdventOfCode.Util (count, (<&&>))
import Data.FastDigits (digits)
import Data.List (group)
import Data.List.Ordered (isSorted)

\nwlinkedidentc{main}{NW3AzXtx-19Srvv-1} :: IO ()
\nwlinkedidentc{main}{NW3AzXtx-19Srvv-1} = $(defaultMain)

\LA{}Input~{\nwtagstyle{}\subpageref{NW35miTa-1GvnV-1}}\RA{}

\LA{}Part One~{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}\RA{}

\LA{}Part Two~{\nwtagstyle{}\subpageref{NW35miTa-4P9qKy-1}}\RA{}

\LA{}Generic solver~{\nwtagstyle{}\subpageref{NW35miTa-2ApJRg-1}}\RA{}
\nwnotused{Day04.hs}\nwidentuses{\\{{\nwixident{main}}{main}}}\nwindexuse{\nwixident{main}}{main}{NW35miTa-15Rjc8-1}\nwendcode{}\nwbegindocs{10}\nwdocspar
\nwenddocs{}\nwfilename{_src/2019/day/08.nw}\nwbegindocs{0}\newpage
\section{Day 8: }\todor{Add missing title}
\todoo{Copy description}
\marginnote{\url{https://adventofcode.com/2019/day/8}}
\nwenddocs{}\nwfilename{_src/2019/haskell/08.nw}\nwbegindocs{0}\subsection{Haskell solution}

\newthought{A pixel} can be black, white, or transparent.

\nwenddocs{}\nwbegincode{1}\sublabel{NW3AzXtx-2M5oYw-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-2M5oYw-1}}}\moddef{Define a Pixel data type~{\nwtagstyle{}\subpageref{NW3AzXtx-2M5oYw-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3AzXtx-19Srvv-1}}\nwenddeflinemarkup
data \nwlinkedidentc{Pixel}{NW3AzXtx-2M5oYw-1}
  = \nwlinkedidentc{Black}{NW3AzXtx-2M5oYw-1}
  | \nwlinkedidentc{White}{NW3AzXtx-2M5oYw-1}
  | \nwlinkedidentc{Transparent}{NW3AzXtx-2M5oYw-1}
  deriving (Enum, Eq)
\nwindexdefn{\nwixident{Pixel}}{Pixel}{NW3AzXtx-2M5oYw-1}\eatline
\nwindexdefn{\nwixident{Black}}{Black}{NW3AzXtx-2M5oYw-1}\eatline
\nwindexdefn{\nwixident{White}}{White}{NW3AzXtx-2M5oYw-1}\eatline
\nwindexdefn{\nwixident{Transparent}}{Transparent}{NW3AzXtx-2M5oYw-1}\eatline
\nwused{\\{NW3AzXtx-19Srvv-1}}\nwidentdefs{\\{{\nwixident{Black}}{Black}}\\{{\nwixident{Pixel}}{Pixel}}\\{{\nwixident{Transparent}}{Transparent}}\\{{\nwixident{White}}{White}}}\nwendcode{}\nwbegindocs{2}\nwdocspar

Show black pixels as spaces, white ones as hashes, and transparent as dots.

\nwenddocs{}\nwbegincode{3}\sublabel{NW3AzXtx-QyGx2-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-QyGx2-1}}}\moddef{Implement \hs{Show} for \code{}Pixel\edoc{}~{\nwtagstyle{}\subpageref{NW3AzXtx-QyGx2-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3AzXtx-19Srvv-1}}\nwenddeflinemarkup
instance Show \nwlinkedidentc{Pixel}{NW3AzXtx-2M5oYw-1} where
  show \nwlinkedidentc{Black}{NW3AzXtx-2M5oYw-1} = " "
  show \nwlinkedidentc{White}{NW3AzXtx-2M5oYw-1} = "#"
  show \nwlinkedidentc{Transparent}{NW3AzXtx-2M5oYw-1} = "."
\nwused{\\{NW3AzXtx-19Srvv-1}}\nwidentuses{\\{{\nwixident{Black}}{Black}}\\{{\nwixident{Pixel}}{Pixel}}\\{{\nwixident{Transparent}}{Transparent}}\\{{\nwixident{White}}{White}}}\nwindexuse{\nwixident{Black}}{Black}{NW3AzXtx-QyGx2-1}\nwindexuse{\nwixident{Pixel}}{Pixel}{NW3AzXtx-QyGx2-1}\nwindexuse{\nwixident{Transparent}}{Transparent}{NW3AzXtx-QyGx2-1}\nwindexuse{\nwixident{White}}{White}{NW3AzXtx-QyGx2-1}\nwendcode{}\nwbegindocs{4}\nwdocspar

\newthought{Define a {\Tt{}\nwlinkedidentq{Layer}{NW3AzXtx-LSl4Q-1}\nwendquote}} as a list of {\Tt{}\nwlinkedidentq{Row}{NW3AzXtx-LSl4Q-1}\nwendquote}s, and a {\Tt{}\nwlinkedidentq{Row}{NW3AzXtx-LSl4Q-1}\nwendquote} as a list of {\Tt{}\nwlinkedidentq{Pixel}{NW3AzXtx-2M5oYw-1}\nwendquote}s.

\nwenddocs{}\nwbegincode{5}\sublabel{NW3AzXtx-LSl4Q-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-LSl4Q-1}}}\moddef{Define a few convenient type aliases~{\nwtagstyle{}\subpageref{NW3AzXtx-LSl4Q-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3AzXtx-19Srvv-1}}\nwenddeflinemarkup
type \nwlinkedidentc{Image}{NW3AzXtx-LSl4Q-1} = [\nwlinkedidentc{Layer}{NW3AzXtx-LSl4Q-1}]

type \nwlinkedidentc{Layer}{NW3AzXtx-LSl4Q-1} = [\nwlinkedidentc{Row}{NW3AzXtx-LSl4Q-1}]

type \nwlinkedidentc{Row}{NW3AzXtx-LSl4Q-1} = [\nwlinkedidentc{Pixel}{NW3AzXtx-2M5oYw-1}]
\nwindexdefn{\nwixident{Image}}{Image}{NW3AzXtx-LSl4Q-1}\eatline
\nwindexdefn{\nwixident{Layer}}{Layer}{NW3AzXtx-LSl4Q-1}\eatline
\nwindexdefn{\nwixident{Row}}{Row}{NW3AzXtx-LSl4Q-1}\eatline
\nwused{\\{NW3AzXtx-19Srvv-1}}\nwidentdefs{\\{{\nwixident{Image}}{Image}}\\{{\nwixident{Layer}}{Layer}}\\{{\nwixident{Row}}{Row}}}\nwidentuses{\\{{\nwixident{Pixel}}{Pixel}}}\nwindexuse{\nwixident{Pixel}}{Pixel}{NW3AzXtx-LSl4Q-1}\nwendcode{}\nwbegindocs{6}\nwdocspar

\newthought{Parse an {\Tt{}\nwlinkedidentq{Image}{NW3AzXtx-LSl4Q-1}\nwendquote},} i.e. one or more {\Tt{}\nwlinkedidentq{Layer}{NW3AzXtx-LSl4Q-1}\nwendquote}s comprised of \hs{height}
{\Tt{}\nwlinkedidentq{Row}{NW3AzXtx-LSl4Q-1}\nwendquote}s of \hs{width} {\Tt{}\nwlinkedidentq{Pixel}{NW3AzXtx-2M5oYw-1}\nwendquote}s.

\nwenddocs{}\nwbegincode{7}\sublabel{NW3AzXtx-4aeb4o-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-4aeb4o-1}}}\moddef{Parse an image~{\nwtagstyle{}\subpageref{NW3AzXtx-4aeb4o-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3AzXtx-19Srvv-1}}\nwenddeflinemarkup
\nwlinkedidentc{image}{NW3AzXtx-4aeb4o-1} :: Int -> Int -> Parser \nwlinkedidentc{Image}{NW3AzXtx-LSl4Q-1}
\nwlinkedidentc{image}{NW3AzXtx-4aeb4o-1} width height = some layer
  where
    layer :: Parser \nwlinkedidentc{Layer}{NW3AzXtx-LSl4Q-1}
    layer = count height row
    row :: Parser \nwlinkedidentc{Row}{NW3AzXtx-LSl4Q-1}
    row = count width pixel
\nwindexdefn{\nwixident{image}}{image}{NW3AzXtx-4aeb4o-1}\eatline
\nwused{\\{NW3AzXtx-19Srvv-1}}\nwidentdefs{\\{{\nwixident{image}}{image}}}\nwidentuses{\\{{\nwixident{Image}}{Image}}\\{{\nwixident{Layer}}{Layer}}\\{{\nwixident{Row}}{Row}}}\nwindexuse{\nwixident{Image}}{Image}{NW3AzXtx-4aeb4o-1}\nwindexuse{\nwixident{Layer}}{Layer}{NW3AzXtx-4aeb4o-1}\nwindexuse{\nwixident{Row}}{Row}{NW3AzXtx-4aeb4o-1}\nwendcode{}\nwbegindocs{8}\nwdocspar

Parse an encoded black, white, or transparent pixel.

\nwenddocs{}\nwbegincode{9}\sublabel{NW3AzXtx-1aCFXy-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-1aCFXy-1}}}\moddef{Parse a pixel~{\nwtagstyle{}\subpageref{NW3AzXtx-1aCFXy-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3AzXtx-19Srvv-1}}\nwenddeflinemarkup
pixel :: Parser \nwlinkedidentc{Pixel}{NW3AzXtx-2M5oYw-1}
pixel =
  (char '0' *> pure \nwlinkedidentc{Black}{NW3AzXtx-2M5oYw-1} <?> "A black pixel")
    <|> (char '1' *> pure \nwlinkedidentc{White}{NW3AzXtx-2M5oYw-1} <?> "A white pixel")
    <|> (char '2' *> pure \nwlinkedidentc{Transparent}{NW3AzXtx-2M5oYw-1} <?> "A transparent pixel")
\nwused{\\{NW3AzXtx-19Srvv-1}}\nwidentuses{\\{{\nwixident{Black}}{Black}}\\{{\nwixident{Pixel}}{Pixel}}\\{{\nwixident{Transparent}}{Transparent}}\\{{\nwixident{White}}{White}}}\nwindexuse{\nwixident{Black}}{Black}{NW3AzXtx-1aCFXy-1}\nwindexuse{\nwixident{Pixel}}{Pixel}{NW3AzXtx-1aCFXy-1}\nwindexuse{\nwixident{Transparent}}{Transparent}{NW3AzXtx-1aCFXy-1}\nwindexuse{\nwixident{White}}{White}{NW3AzXtx-1aCFXy-1}\nwendcode{}\nwbegindocs{10}\nwdocspar

\newthought{Solve} Part One.

\nwenddocs{}\nwbegincode{11}\sublabel{NW3AzXtx-2iOjQS-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-2iOjQS-1}}}\moddef{Part One~{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwprevnextdefs{NW35miTa-2iOjQS-1}{NW3AzXtx-2iOjQS-2}\nwenddeflinemarkup
\nwlinkedidentc{partOne}{NW3AzXtx-2iOjQS-4} :: \nwlinkedidentc{Image}{NW3AzXtx-LSl4Q-1} -> Int
\nwused{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwidentuses{\\{{\nwixident{Image}}{Image}}\\{{\nwixident{partOne}}{partOne}}}\nwindexuse{\nwixident{Image}}{Image}{NW3AzXtx-2iOjQS-1}\nwindexuse{\nwixident{partOne}}{partOne}{NW3AzXtx-2iOjQS-1}\nwendcode{}\nwbegindocs{12}\nwdocspar

Return the product of the number of ones ({\Tt{}\nwlinkedidentq{White}{NW3AzXtx-2M5oYw-1}\nwendquote} pixels) and the number of
twos ({\Tt{}\nwlinkedidentq{Transparent}{NW3AzXtx-2M5oYw-1}\nwendquote} pixels) in the \hs{layer} with the fewest {\Tt{}\nwlinkedidentq{Black}{NW3AzXtx-2M5oYw-1}\nwendquote} pixels.

\nwenddocs{}\nwbegincode{13}\sublabel{NW3AzXtx-2iOjQS-2}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-2iOjQS-2}}}\moddef{Part One~{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwprevnextdefs{NW3AzXtx-2iOjQS-1}{NW3AzXtx-2iOjQS-3}\nwenddeflinemarkup
\nwlinkedidentc{partOne}{NW3AzXtx-2iOjQS-4} layers = numberOf \nwlinkedidentc{White}{NW3AzXtx-2M5oYw-1} layer * numberOf \nwlinkedidentc{Transparent}{NW3AzXtx-2M5oYw-1} layer
  where
\nwused{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwidentuses{\\{{\nwixident{partOne}}{partOne}}\\{{\nwixident{Transparent}}{Transparent}}\\{{\nwixident{White}}{White}}}\nwindexuse{\nwixident{partOne}}{partOne}{NW3AzXtx-2iOjQS-2}\nwindexuse{\nwixident{Transparent}}{Transparent}{NW3AzXtx-2iOjQS-2}\nwindexuse{\nwixident{White}}{White}{NW3AzXtx-2iOjQS-2}\nwendcode{}\nwbegindocs{14}\nwdocspar

Find the \hs{layer} with the fewest zeros\todoo{sp?}, i.e. {\Tt{}\nwlinkedidentq{Black}{NW3AzXtx-2M5oYw-1}\nwendquote} pixels.

\nwenddocs{}\nwbegincode{15}\sublabel{NW3AzXtx-2iOjQS-3}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-2iOjQS-3}}}\moddef{Part One~{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwprevnextdefs{NW3AzXtx-2iOjQS-2}{NW3AzXtx-2iOjQS-4}\nwenddeflinemarkup
    layer = minimumBy (compare `on` numberOf \nwlinkedidentc{Black}{NW3AzXtx-2M5oYw-1}) layers
\nwused{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwidentuses{\\{{\nwixident{Black}}{Black}}}\nwindexuse{\nwixident{Black}}{Black}{NW3AzXtx-2iOjQS-3}\nwendcode{}\nwbegindocs{16}\nwdocspar

Return the number of elements equivalent to a given one, in a given list of
lists of elements of the same type. More specifically, return the number of
{\Tt{}\nwlinkedidentq{Pixel}{NW3AzXtx-2M5oYw-1}\nwendquote}s of a given color in a given {\Tt{}\nwlinkedidentq{Layer}{NW3AzXtx-LSl4Q-1}\nwendquote}.

\todoo{There's gotta be a Data.List function for this..}

\nwenddocs{}\nwbegincode{17}\sublabel{NW3AzXtx-2iOjQS-4}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-2iOjQS-4}}}\moddef{Part One~{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwprevnextdefs{NW3AzXtx-2iOjQS-3}{\relax}\nwenddeflinemarkup
    numberOf :: Eq a => a -> [[a]] -> Int
    numberOf x = sum . fmap (length . filter (== x))
\nwindexdefn{\nwixident{partOne}}{partOne}{NW3AzXtx-2iOjQS-4}\eatline
\nwused{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwidentdefs{\\{{\nwixident{partOne}}{partOne}}}\nwendcode{}\nwbegindocs{18}\nwdocspar

\newthought{Solve} Part Two.

\nwenddocs{}\nwbegincode{19}\sublabel{NW3AzXtx-4P9qKy-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-4P9qKy-1}}}\moddef{Part Two~{\nwtagstyle{}\subpageref{NW35miTa-4P9qKy-1}}}\plusendmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwprevnextdefs{NW35miTa-4P9qKy-1}{\relax}\nwenddeflinemarkup
\nwlinkedidentc{partTwo}{NW3AzXtx-4P9qKy-1} :: \nwlinkedidentc{Image}{NW3AzXtx-LSl4Q-1} -> String
\nwlinkedidentc{partTwo}{NW3AzXtx-4P9qKy-1} layers =
  unlines . map (concatMap show) $
    foldl decodeLayer (\nwlinkedidentc{transparentLayer}{NW3AzXtx-dIQyV-1} 25 6) layers
  where
    decodeLayer :: \nwlinkedidentc{Layer}{NW3AzXtx-LSl4Q-1} -> \nwlinkedidentc{Layer}{NW3AzXtx-LSl4Q-1} -> \nwlinkedidentc{Layer}{NW3AzXtx-LSl4Q-1}
    decodeLayer = zipWith (zipWith decodePixel)
    decodePixel :: \nwlinkedidentc{Pixel}{NW3AzXtx-2M5oYw-1} -> \nwlinkedidentc{Pixel}{NW3AzXtx-2M5oYw-1} -> \nwlinkedidentc{Pixel}{NW3AzXtx-2M5oYw-1}
    decodePixel \nwlinkedidentc{Transparent}{NW3AzXtx-2M5oYw-1} below = below
    decodePixel above _ = above
\nwindexdefn{\nwixident{partTwo}}{partTwo}{NW3AzXtx-4P9qKy-1}\eatline
\nwused{\\{NW35miTa-15Rjc8-1}\\{NW3AzXtx-19Srvv-1}}\nwidentdefs{\\{{\nwixident{partTwo}}{partTwo}}}\nwidentuses{\\{{\nwixident{Image}}{Image}}\\{{\nwixident{Layer}}{Layer}}\\{{\nwixident{Pixel}}{Pixel}}\\{{\nwixident{Transparent}}{Transparent}}\\{{\nwixident{transparentLayer}}{transparentLayer}}}\nwindexuse{\nwixident{Image}}{Image}{NW3AzXtx-4P9qKy-1}\nwindexuse{\nwixident{Layer}}{Layer}{NW3AzXtx-4P9qKy-1}\nwindexuse{\nwixident{Pixel}}{Pixel}{NW3AzXtx-4P9qKy-1}\nwindexuse{\nwixident{Transparent}}{Transparent}{NW3AzXtx-4P9qKy-1}\nwindexuse{\nwixident{transparentLayer}}{transparentLayer}{NW3AzXtx-4P9qKy-1}\nwendcode{}\nwbegindocs{20}\nwdocspar

\newthought{Define a helper function} to create a transparent layer.

\nwenddocs{}\nwbegincode{21}\sublabel{NW3AzXtx-dIQyV-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-dIQyV-1}}}\moddef{A transparent layer~{\nwtagstyle{}\subpageref{NW3AzXtx-dIQyV-1}}}\endmoddef\nwstartdeflinemarkup\nwusesondefline{\\{NW3AzXtx-19Srvv-1}}\nwenddeflinemarkup
\nwlinkedidentc{transparentLayer}{NW3AzXtx-dIQyV-1} :: Int -> Int -> \nwlinkedidentc{Layer}{NW3AzXtx-LSl4Q-1}
\nwlinkedidentc{transparentLayer}{NW3AzXtx-dIQyV-1} width height = replicate height (replicate width \nwlinkedidentc{Transparent}{NW3AzXtx-2M5oYw-1})
\nwindexdefn{\nwixident{transparentLayer}}{transparentLayer}{NW3AzXtx-dIQyV-1}\eatline
\nwused{\\{NW3AzXtx-19Srvv-1}}\nwidentdefs{\\{{\nwixident{transparentLayer}}{transparentLayer}}}\nwidentuses{\\{{\nwixident{Layer}}{Layer}}\\{{\nwixident{Transparent}}{Transparent}}}\nwindexuse{\nwixident{Layer}}{Layer}{NW3AzXtx-dIQyV-1}\nwindexuse{\nwixident{Transparent}}{Transparent}{NW3AzXtx-dIQyV-1}\nwendcode{}\nwbegindocs{22}\nwdocspar

\todo[inline]{Add some prose here.}

\nwenddocs{}\nwbegincode{23}\sublabel{NW3AzXtx-19Srvv-1}\nwmargintag{{\nwtagstyle{}\subpageref{NW3AzXtx-19Srvv-1}}}\moddef{Day08.hs~{\nwtagstyle{}\subpageref{NW3AzXtx-19Srvv-1}}}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
module AdventOfCode.Year2019.Day08 where

import AdventOfCode.Input (parseInput)
import AdventOfCode.TH (defaultMain, inputFilePath)
import Control.Applicative ((<|>))
import Data.Function (on)
import Data.List (minimumBy)
import Text.Trifecta (Parser, char, count, some, (<?>))

\LA{}Define a Pixel data type~{\nwtagstyle{}\subpageref{NW3AzXtx-2M5oYw-1}}\RA{}

\LA{}Implement \hs{Show} for \code{}Pixel\edoc{}~{\nwtagstyle{}\subpageref{NW3AzXtx-QyGx2-1}}\RA{}

\LA{}Define a few convenient type aliases~{\nwtagstyle{}\subpageref{NW3AzXtx-LSl4Q-1}}\RA{}

\nwlinkedidentc{main}{NW3AzXtx-19Srvv-1} :: IO ()
\nwlinkedidentc{main}{NW3AzXtx-19Srvv-1} = $(defaultMain)

getInput :: IO \nwlinkedidentc{Image}{NW3AzXtx-LSl4Q-1}
getInput = parseInput (\nwlinkedidentc{image}{NW3AzXtx-4aeb4o-1} 25 6) $(inputFilePath)

\LA{}Part One~{\nwtagstyle{}\subpageref{NW35miTa-2iOjQS-1}}\RA{}

\LA{}Part Two~{\nwtagstyle{}\subpageref{NW35miTa-4P9qKy-1}}\RA{}

\LA{}Parse an image~{\nwtagstyle{}\subpageref{NW3AzXtx-4aeb4o-1}}\RA{}

\LA{}Parse a pixel~{\nwtagstyle{}\subpageref{NW3AzXtx-1aCFXy-1}}\RA{}

\LA{}A transparent layer~{\nwtagstyle{}\subpageref{NW3AzXtx-dIQyV-1}}\RA{}
\nwindexdefn{\nwixident{main}}{main}{NW3AzXtx-19Srvv-1}\eatline
\nwnotused{Day08.hs}\nwidentdefs{\\{{\nwixident{main}}{main}}}\nwidentuses{\\{{\nwixident{Image}}{Image}}\\{{\nwixident{image}}{image}}}\nwindexuse{\nwixident{Image}}{Image}{NW3AzXtx-19Srvv-1}\nwindexuse{\nwixident{image}}{image}{NW3AzXtx-19Srvv-1}\nwendcode{}

\nwixlogsorted{c}{{A transparent layer}{NW3AzXtx-dIQyV-1}{\nwixd{NW3AzXtx-dIQyV-1}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Day01.g}{NW3RAD1b-3gOu99-1}{\nwixd{NW3RAD1b-3gOu99-1}\nwixd{NW3RAD1b-3gOu99-2}\nwixd{NW3RAD1b-3gOu99-3}\nwixd{NW3RAD1b-3gOu99-4}}}%
\nwixlogsorted{c}{{Day04.hs}{NW35miTa-15Rjc8-1}{\nwixd{NW35miTa-15Rjc8-1}}}%
\nwixlogsorted{c}{{Day08.hs}{NW3AzXtx-19Srvv-1}{\nwixd{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Define a few convenient type aliases}{NW3AzXtx-LSl4Q-1}{\nwixd{NW3AzXtx-LSl4Q-1}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Define a Pixel data type}{NW3AzXtx-2M5oYw-1}{\nwixd{NW3AzXtx-2M5oYw-1}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Generic solver}{NW35miTa-2ApJRg-1}{\nwixd{NW35miTa-2ApJRg-1}\nwixu{NW35miTa-15Rjc8-1}}}%
\nwixlogsorted{c}{{Implement \hs{Show} for \code{}Pixel\edoc{}}{NW3AzXtx-QyGx2-1}{\nwixd{NW3AzXtx-QyGx2-1}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Input}{NW35miTa-1GvnV-1}{\nwixd{NW35miTa-1GvnV-1}\nwixu{NW35miTa-15Rjc8-1}}}%
\nwixlogsorted{c}{{Parse a pixel}{NW3AzXtx-1aCFXy-1}{\nwixd{NW3AzXtx-1aCFXy-1}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Parse an image}{NW3AzXtx-4aeb4o-1}{\nwixd{NW3AzXtx-4aeb4o-1}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Part One}{NW35miTa-2iOjQS-1}{\nwixd{NW35miTa-2iOjQS-1}\nwixu{NW35miTa-15Rjc8-1}\nwixd{NW3AzXtx-2iOjQS-1}\nwixd{NW3AzXtx-2iOjQS-2}\nwixd{NW3AzXtx-2iOjQS-3}\nwixd{NW3AzXtx-2iOjQS-4}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{c}{{Part Two}{NW35miTa-4P9qKy-1}{\nwixd{NW35miTa-4P9qKy-1}\nwixu{NW35miTa-15Rjc8-1}\nwixd{NW3AzXtx-4P9qKy-1}\nwixu{NW3AzXtx-19Srvv-1}}}%
\nwixlogsorted{i}{{\nwixident{Black}}{Black}}%
\nwixlogsorted{i}{{\nwixident{Image}}{Image}}%
\nwixlogsorted{i}{{\nwixident{image}}{image}}%
\nwixlogsorted{i}{{\nwixident{Layer}}{Layer}}%
\nwixlogsorted{i}{{\nwixident{main}}{main}}%
\nwixlogsorted{i}{{\nwixident{partOne}}{partOne}}%
\nwixlogsorted{i}{{\nwixident{partTwo}}{partTwo}}%
\nwixlogsorted{i}{{\nwixident{Pixel}}{Pixel}}%
\nwixlogsorted{i}{{\nwixident{Row}}{Row}}%
\nwixlogsorted{i}{{\nwixident{Transparent}}{Transparent}}%
\nwixlogsorted{i}{{\nwixident{transparentLayer}}{transparentLayer}}%
\nwixlogsorted{i}{{\nwixident{White}}{White}}%


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}

\title{Marlin Diagram}
\date{July 2020}

\usepackage[x11names]{xcolor}
\usepackage[b4paper,margin=1.2in]{geometry}
\usepackage{tikz}
\usepackage{afterpage}

\newenvironment{rcases}
  {\left.\begin{aligned}}
  {\end{aligned}\right\rbrace}

\begin{document}

\newcommand{\cm}[1]{\ensuremath{\mathsf{cm}_{#1}}}
\newcommand{\vcm}[1]{\ensuremath{\mathsf{vcm}_{#1}}}
\newcommand{\s}{\ensuremath{\hat{s}}}
\newcommand{\w}{\ensuremath{\hat{w}}}
\newcommand{\x}{\ensuremath{\hat{x}}}
\newcommand{\z}{\ensuremath{\hat{z}}}
\newcommand{\za}{\ensuremath{\hat{z}_A}}
\newcommand{\zb}{\ensuremath{\hat{z}_B}}
\newcommand{\zc}{\ensuremath{\hat{z}_C
}}
\newcommand{\zm}{\ensuremath{\hat{z}_M}}

\newcommand{\val}{\ensuremath{\mathsf{val}}}
\newcommand{\row}{\ensuremath{\mathsf{row}}}
\newcommand{\col}{\ensuremath{\mathsf{col}}}
\newcommand{\rowcol}{\ensuremath{\mathsf{rowcol}}}

\newcommand{\hval}{\ensuremath{\widehat{\val}}}
\newcommand{\hrow}{\ensuremath{\widehat{\row}}}
\newcommand{\hcol}{\ensuremath{\widehat{\col}}}
\newcommand{\hrowcol}{\ensuremath{\widehat{\rowcol}}}

\newcommand{\bb}{\ensuremath{\mathsf{b}}}
\newcommand{\denom}{\ensuremath{\mathsf{denom}}}

\newcommand{\sumcheckinner}{\mathsf{inner}}
\newcommand{\sumcheckouter}{\mathsf{outer}}

\newcommand{\Prover}{\mathcal{P}}
\newcommand{\Verifier}{\mathcal{V}}

\newcommand{\F}{\mathbb{F}}

\newcommand{\DomainA}{H}
\newcommand{\DomainB}{K}

\newcommand{\vPoly}[1]{\ensuremath{v_{#1}}}

This diagram (on the following page) shows the interaction of the Marlin prover and verifier. It is similar to the diagrams in the paper (Figure 5 in Section 5 and Figure 7 in Appendix E, in the latest ePrint version), but with two changes: it shows not just the AHP but also the use of the polynomial commitments (the cryptography layer); and it aims to be fully up-to-date with the recent optimizations to the codebase. This diagram, together with the diagrams in the paper, can act as a ``bridge" between the codebase and the theory that the paper describes.

\section{Glossary of notation}
\begin{table*}[htbp]
  \centering
  \begin{tabular}{c|c}
    $\F$ & the finite field over which the R1CS instance is defined \\
     \hline
    $x$ & public input \\
     \hline
    $w$ & secret witness \\
     \hline
    $\DomainA$ & variable domain \\
     \hline
    $\DomainB$ & matrix domain \\
     \hline
    $X$ & domain sized for input (not including witness) \\
     \hline
    $v_D(X)$ & vanishing polynomial over domain $D$ \\
    \hline
    $u_D(X, Y)$ & bivariate derivative of vanishing polynomials over domain $D$\\
     \hline
    $A, B, C$ & R1CS instance matrices \\
    \hline
    $A^*, B^*, C^*$ &
    \begin{tabular}{@{}c@{}}shifted transpose of $A,B,C$ matries given by $M^*_{a,b} := M_{b,a} \cdot u_\DomainA(b,b) \; \forall a,b \in \DomainA$ \\ (optimization from Fractal, explained in Claim 6.7 of that paper) \end{tabular} \\
     \hline
    $\hrow, \hcol$ &
    \begin{tabular}{@{}c@{}} LDEs of (respectively) row positions and column positions of non-zero elements of any \\ linear combination of $A^*$, $B^*$, and $C^*$ (the choice of combination is irrelevant).\end{tabular} \\
    \hline
    ${\hrowcol}$ &
    \begin{tabular}{@{}c@{}} LDE of the element-wise product of $\row$ and $\col$, given separately for efficiency  \\ (namely to allow this product to be part of a \textit{linear} combination) \end{tabular} \\
    \hline
    $\hval_{\{A^*, B^*, C^*\}}$ &
    \begin{tabular}{@{}c@{}} preprocessed polynomials containing LDEs of \\ the values of non-zero elements of any linear combination of $A^*$, $B^*$, and $C^*$. \\ That is, if $\kappa$ is the $k$-th element of $\DomainB$, then $(\sum_M \eta_M \hval_{M^*})(\kappa)$ is the \\$k$-th non-zero entry of $\sum_M \eta_M M^*$, for arbitrary $\eta_{\{A, B, C\}} \in \F$.\end{tabular} \\
     \hline
    $\Prover$ & prover \\
     \hline
    $\Verifier$ & verifier \\
     \hline
    $\Verifier^{p}$ &
    	\begin{tabular}{@{}c@{}} $\Verifier$ with ``oracle" access to polynomial $p$ (via commitments provided \\ by the indexer, later opened as necessary by $\Prover$) \end{tabular}\\
    \hline
    $\bb$ & bound on the number of queries \\
    \hline
    $r_M(X, Y)$ & an intermediate polynomial defined by $r_M(X, Y) = M^*(Y,X)$\\
    \hline
  \end{tabular}
\end{table*}

\afterpage{%
\newgeometry{margin=0.2in}

\section{Diagram}

\centering
\begin{tikzpicture}[scale=0.95, every node/.style={scale=0.95}]

\tikzstyle{lalign} = [minimum width=3cm,align=left,anchor=west]
\tikzstyle{ralign} = [minimum width=3cm,align=right,anchor=east]

\node[lalign] (prover) at (-3,27.3) {%
$\Prover(\F, \DomainA, \DomainB, A, B, C, x, w)$
};

\node[ralign] (verifier) at (16.2,27.3) {%
$\Verifier^{\hrow, \hcol, \hrowcol, \hval_{A^*}, \hval_{B^*}, \hval_{C^*}}(\F, \DomainA, \DomainB, x)$
};

\draw [line width=1.0pt] (-3,27.0) -- (16,27.0);

\node[lalign] (prover1) at (-3,26.1) {%
$z := (x, w), z_A := Az, z_B := Bz$ \\
sample $\w(X) \in \F^{<|w|+\bb}[X]$ and $\za(X), \zb(X) \in \F^{<|\DomainA|+\bb}[X]$ \\
sample mask poly $\s(X) \in \F^{<3|\DomainA|+2\bb-2}[X]$ such that $\sum_{\kappa \in \DomainA}\s(\kappa) = 0$
};

\draw [->] (-2,24.8) -- node[midway,fill=white] {commitments $\cm{\w}, \cm{\za}, \cm{\zb}, \cm{\s}$} (15,24.8);

\node[ralign] (verifier1) at (16,24.0) {%
$\eta_A, \eta_B, \eta_C \gets \F$ \\
$\alpha \gets \F \setminus \DomainA$
};

\draw [->] (15,23.3) -- node[midway,fill=white] {$\eta_A, \eta_B, \eta_C, \alpha \in \F$} (-2,23.3);

\node[lalign] (prover2) at (-3,22.5) {%
compute $t(X) := \sum_M \eta_M r_M(\alpha, X)$
};

\draw (-2.9,22.0) rectangle (15.9,3.8);

\node (sc1label) at (6.5,21.7) {%
\textbf{sumcheck for} $\s(X) + u_H(\alpha, X) \left(\sum_M \eta_M \zm(X)\right) - t(X)\z(X)$ \textbf{ over } $\DomainA$
};

\node[lalign] (prover3) at (-2,20.7) {%
let $\zc(X) := \za(X) \cdot \zb(X)$ \\
find $g_1(X) \in \F^{|\DomainA|-1}[X]$ and $h_1(X)$ such that \\
$s(X)+u_H(\alpha, X)(\sum_M \eta_M \zm(X)) - t(X)\z(X) = h_1(X)\vPoly{\DomainA}(X) + Xg_1(X)$ \hspace{0.3cm} $(*)$
};

\draw [->] (-1,19.5) -- node[midway,fill=white] {commitments $\cm{t}, \cm{g_1}, \cm{h_1}$} (14,19.5);

\node[ralign] (verifier2) at (15.4,19.1) {%
$\beta \gets \F \setminus \DomainA$
};

\draw [->] (14,18.7) -- node[midway,fill=white] {$\beta \in \F$} (-1,18.7);

\draw (-0.85,18.2) rectangle (13.85,7.6);

\node (sc2label) at (6.5,17.6) {%
\textbf{sumcheck for } $\sum\limits_{M \in \{A, B, C\}} \eta_M \frac{\vPoly{\DomainA}(\beta) \vPoly{\DomainA}(\alpha)\hval_{M^*}(X)}{\color{purple}(\beta-\hrow(X))(\alpha-\hcol(X))} $ \textbf{ over } $\DomainB$
};

\node[align=center] (mid1) at (6.5, 15) {%
$\begin{aligned}
\text{let } {\color{purple} \denom(X)} &{}:= (\beta - \hrow(X)) (\alpha - \hcol(X)) \\
                                       &{}= {\color{gray}\alpha\beta} - {\color{gray}\alpha}\hrow(X) - {\color{gray}\beta}\hcol(X) + \hrowcol(X) \text{ (over $\DomainB$)}\\\\
    \text{ let } {\color{orange} a(X)} &{}:= {\color{gray} \vPoly{\DomainA}(\beta) \vPoly{\DomainA}(\alpha)} \sum\limits_{M \in \{A, B, C\}} \eta_M \hval_{M^*}(X)
\\\\
    \text{ let } {\color{Green4} b(X)} &{}:= {\color{purple} \denom(X)}\\\\
\end{aligned}$
};

\node[lalign] (prover4) at (-0.75,12.2) {%
find $g_2(X) \in \F^{|\DomainB|-1}[X]$ and $h_2(X)$ s.t. \\
$h_2(X)\vPoly{\DomainB}(X) = {\color{orange} a(X)} - {\color{Green4} b(X)} (Xg_2(X)+t(\beta)/|\DomainB|)$ \hspace{0.3cm} $(**)$
};

\draw [->] (0,11.2) -- node[midway,fill=white] {commitments $\cm{g_2}, \cm{h_2}$} (13,11.2);

\draw [->] (13,10.5) -- node[midway,fill=white] {$\gamma \in \F$} (0,10.5);

\node[ralign] (verifier3) at (14.5, 10.9) {%
$\gamma \gets \F$
};

\draw[dashed] (1.5,10.0) rectangle (11.5,7.8);

\node[align=center] (mid4) at (6.5, 8.9) {%
To verify $(**)$, $\Verifier$ will need to check the following: \\[10pt]
$ \underbrace{{\color{orange} a({\color{black} \gamma})} - {\color{Green4} b({\color{black} \gamma})} {\color{gray} (\gamma g_2(\gamma) + t(\beta) / |\DomainB|) - \vPoly{\DomainB}(\gamma)} h_2(\gamma)}_{\sumcheckinner(\gamma)} \stackrel{?}{=} 0 $
};

\node[ralign] (verifier3) at (15.4, 6.9) {%
Compute $\x(X) \in \F^{<|x|}[X]$ from input $x$
};

\draw[dashed] (-2.7,7.4) rectangle (15.7,4.2);

\node[align=center] (mid5) at (6.5, 5.3) {%
To verify $(*)$, $\Verifier$ will need to check the following: \\[10pt]
$ \underbrace{s(\beta) + {\color{gray} v_H(\alpha, \beta)} ({\color{gray} \eta_A} \za(\beta) + {\color{gray} \eta_C\zb(\beta)} \za(\beta) + {\color{gray} \eta_B\zb(\beta)}) - {\color{gray} t(\beta) \vPoly{X}(\beta)} \w(\beta) - {\color{gray} t(\beta) \x(\beta)} - {\color{gray} \vPoly{\DomainA}(\beta)} h_1(\beta) - {\color{gray} \beta g_1(\beta)}}_{\sumcheckouter(\beta)} \stackrel{?}{=} 0 $
};

\node[lalign] (prover5) at (-3,2.9) {%
$v_{g_2} := g_2(\gamma)$ \\[3pt]
$v_{g_1} := g_1(\beta), v_{\zb} := \zb(\beta), v_{t} := t(\beta)$
};

\draw [->] (-2,1.9) -- node[midway,fill=white] {$v_{g_2}, v_{g_1}, v_{\zb}, v_{t}$} (15,1.9);

\node[align=center] (mid7) at (6.5,0.8) {%
use index commitments $\hrow, \hcol, \hrowcol, \hval_{\{A^*, B^*, C^*\}}$, commitment $\cm{h_2}$, {\color{gray} and evaluations $g_2(\gamma),t(\beta)$} \\
to construct virtual commitment $\vcm{\sumcheckinner}$
};

\node[align=center] (mid8) at (6.5,-0.5) {%
use commitments $\cm{\s}, \cm{\za}, \cm{\w}, \cm{h_1}$ {\color{gray} and evaluations $\zb(\beta), t(\beta), g_1(\beta)$} \\
to construct virtual commitment $\vcm{\sumcheckouter}$
};

\node[ralign] (verifier4) at (16,-1.4) {%
$\xi_1, \dots, \xi_5 \gets F$
};

\draw [->] (15,-2.1) -- node[midway,fill=white] {$\xi_1, \dots, \xi_5$} (-2,-2.1);

\node[lalign] (prover6) at (-3,-3.6) {%
use $\mathsf{PC}.\mathsf{Prove}$ with randomness $\xi_1, \dots, \xi_5$ to \\
construct a batch opening proof $\pi$ of the following: \\
$(\cm{g_2}, {\color{red} \vcm{\sumcheckinner}})$ at $\gamma$ evaluate to $(v_{g_2}, {\color{red} 0})$ \hspace{0.3cm} ${\color{red} (**)}$ \\
$(\cm{g_1}, \cm{\zb}, \cm{t}, {\color{red} \vcm{\sumcheckouter}})$ at $\beta$ evaluate to $(v_{g_1}, v_{\zb}, v_{t}, {\color{red} 0})$ \hspace{0.3cm} ${\color{red} (*)}$ \\
};

\draw [->] (-2,-4.7) -- node[midway,fill=white] {$\pi$} (15,-4.7);

\node[ralign] (verifier5) at (16,-6.0) {%
verify $\pi$ with $\mathsf{PC}.\mathsf{Verify}$, using randomness $\xi_1, \dots, \xi_5$, \\
evaluations $v_{g_2}, v_{g_1}, v_{\zb}, v_{t}$, and \\
commitments $\cm{g_2},\vcm{\sumcheckinner}, \cm{g_1}, \cm{\zb}, \cm{t}, \vcm{\sumcheckinner}$
};

\end{tikzpicture}

\clearpage
\restoregeometry
}

\end{document}


% This document is part of the emcee3 project.
% Copyright 2015 Dan Foreman-Mackey
%
%  RULES OF THE GAME
%
%  * 80 characters
%  * line breaks at the ends of sentences
%  * eqnarrys ONLY
%

\documentclass[12pt,preprint]{aastex}

\pdfoutput=1

\usepackage{color,hyperref}
\definecolor{linkcolor}{rgb}{0,0,0.5}
\hypersetup{colorlinks=true,linkcolor=linkcolor,citecolor=linkcolor,
            filecolor=linkcolor,urlcolor=linkcolor}
\usepackage{url}
\usepackage{amssymb,amsmath}
\usepackage{subfigure}
\usepackage{booktabs}

\usepackage{natbib}
\bibliographystyle{apj}

% Typography
\newcommand{\project}[1]{\textsl{#1}}
\newcommand{\license}{MIT License}
\newcommand{\paper}{\textsl{Article}}
\newcommand{\foreign}[1]{\emph{#1}}
\newcommand{\etal}{\foreign{et\,al.}}
\newcommand{\etc}{\foreign{etc.}}

\newcommand{\figref}[1]{\ref{fig:#1}}
\newcommand{\Fig}[1]{\figurename~\figref{#1}}
\newcommand{\fig}[1]{\Fig{#1}}
\newcommand{\figlabel}[1]{\label{fig:#1}}
\newcommand{\Tab}[1]{Table~\ref{tab:#1}}
\newcommand{\tab}[1]{\Tab{#1}}
\newcommand{\tablabel}[1]{\label{tab:#1}}
\newcommand{\Eq}[1]{Equation~(\ref{eq:#1})}
\newcommand{\eq}[1]{\Eq{#1}}
\newcommand{\eqalt}[1]{Equation~\ref{eq:#1}}
\newcommand{\eqlabel}[1]{\label{eq:#1}}
\newcommand{\sectionname}{Section}
\newcommand{\Sect}[1]{\sectionname~\ref{sect:#1}}
\newcommand{\sect}[1]{\Sect{#1}}
\newcommand{\sectalt}[1]{\ref{sect:#1}}
\newcommand{\App}[1]{Appendix~\ref{sect:#1}}
\newcommand{\app}[1]{\App{#1}}
\newcommand{\sectlabel}[1]{\label{sect:#1}}

% Algorithms
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage[]{algpseudocode}
\newcommand*\Let[2]{\State #1 $\gets$ #2}
\newcommand{\Alg}[1]{Algorithm~\ref{alg:#1}}
\newcommand{\alg}[1]{\Alg{#1}}
\newcommand{\alglabel}[1]{\label{alg:#1}}

% To-do
\newcommand{\todo}[3]{{\color{#2}\emph{#1}: #3}}
\newcommand{\dfmtodo}[1]{\todo{DFM}{red}{#1}}

% Response to referee
\definecolor{mygreen}{rgb}{0, 0.50196, 0}
\newcommand{\response}[1]{#1}
% \newcommand{\response}[1]{{\color{mygreen} {\bf #1}}}

% Notation for this paper.
\newcommand{\T}{{\ensuremath{\mathrm{T}}}}
\newcommand{\bvec}[1]{{\ensuremath{\boldsymbol{#1}}}}
\newcommand{\lnprob}{{\ensuremath{\mathcal{L}}}}
\newcommand{\pos}{{\bvec{q}}}
\newcommand{\mom}{{\bvec{p}}}
\newcommand{\mass}{{\bvec{M}}}
\newcommand{\normal}[2]{{\ensuremath{\mathcal{N}(#1,\,#2)}}}

\begin{document}

\title{%
    Affine-invariant Hamiltonian Monte Carlo
}

\newcommand{\uw}{2}
\newcommand{\sagan}{3}
\author{%
    Daniel~Foreman-Mackey\altaffilmark{1,\uw,\sagan}
}
\altaffiltext{1}         {To whom correspondence should be addressed:
                          \url{danfm@uw.edu}}
\altaffiltext{\uw}       {Astronomy Department, University of Washington,
                          Seattle, WA 98195}
\altaffiltext{\sagan}    {Sagan Fellow}

\begin{abstract}

Hamiltonian Monte Carlo (HMC) sampling is an efficient method for drawing
samples from a probability density when the gradient of the probability with
respect to the parameters can be computed.
We present a simple but effective affine-invariant HMC method that uses an
ensemble of samplers to adaptively update the mass matrix.
We demonstrate the performance of this method on some simple test cases and
compare its computational cost on a real data analysis problem in exoplanet
astronomy.
A well-tested and efficient Python implementation is released alongside this
note.

\end{abstract}

\keywords{%
methods: data analysis
---
methods: statistical
}

\section{Introduction}

% Text. \citep{Foreman-Mackey:2013}
% Adaptive: \citet{Girolami:2011, Wang:2013, Hoffman:2014}
% \section{Hamiltonian Monte Carlo}

Pseudocode for the standard implementation of the Hamiltonian Monte Carlo
(HMC) algorithm \citep{Neal:2011} is shown in \alg{basic-hmc}.
In this implementation, there are $(D^2 + D) / 2 + 2$ tuning parameters, where
$D$ is the dimension of the problem.
Of these parameters, $(D^2 + D) / 2$ are the elements of the positive
definite mass matrix \bvec{M} and the other 2 are the step size $\epsilon$ and
the number of steps $L$.
Most practical applications of HMC fix the mass matrix to a constant (often
set to 1) times the identity and reduce the tuning to only the two parameters
$\epsilon$ and $L$.
Methods have been developed to automatically tune these parameters \citep[for
example][]{Hoffman:2014}.

The major problem with fixing the mass matrix to be diagonal is that the
\emph{units} of the input space can change the performance of the algorithm.
For example, sampling from a Gaussian with different variances in the
different dimensions or covariance between the parameters will be less
efficient than sampling from an isotropic Gaussian.
It has been demonstrated that samplers that satisfy affine invariance can be
very useful for real problems in science where the dynamic range of parameters
can vary by orders of magnitude \citep{Goodman:2010, Foreman-Mackey:2013}.
It turns out that HMC can be simply adapted to an affine-invariant algorithm.

The affine-invariant samplers proposed by \citet{Goodman:2010} sample the
target density by evolving an \emph{ensemble} of parallel MCMC chains (called
``walkers'') where the instantaneous proposal for one walker is conditioned on
the current locations of the other walkers, the \emph{complementary ensemble}.
If the move preserves the conditional distribution of the target walker given
the complementary ensemble, it will also preserve the joint distribution of
the ensemble.
The intuition from these proposals can be incorporated into HMC to derive an
affine-invariant algorithm.

\begin{algorithm}
    \caption{Standard implementation of a single HMC step \alglabel{basic-hmc}}
    \begin{algorithmic}
        \Function{HMCStep}{$\lnprob(\pos),\,\pos_t,\,\mass,\,\epsilon,\,L$}
        \State $\mom_t \sim \normal{\bvec{0}}{\mass}$
            \Comment{sample the initial momentum exactly}
        \Let{\pos}{$\pos_t$}
        \State
        \Let{\mom}{$\mom_t + \frac{\epsilon}{2}\,\nabla\lnprob(\pos)$}
            \Comment{run $L$ steps of leapfrog integration}
        \For{$l \gets 1 \textrm{ to } L$}
            \Let{\pos}{$\pos + \epsilon\,\mass^{-1}\,\mom$}
            \If{$l < L$}
                \Let{\mom}{$\mom + \epsilon\,\nabla\lnprob(\pos)$}
            \EndIf
        \EndFor
        \Let{\mom}{$\mom + \frac{\epsilon}{2}\,\nabla\lnprob(\pos)$}
            \Comment{synchronize the momentum and position}
        \State
        \State{$r \sim \mathcal{U}(0, 1)$}
        \If{$r < \exp\left[\lnprob(\pos) - \frac{1}{2}\mom^T\mass^{-1}\mom
            - \lnprob(\pos_t)+\frac{1}{2}{\mom_t}^T\mass^{-1}\mom_t \right]$}
            \State\Return{$\pos$}   \Comment{accept}
        \Else
            \State\Return{$\pos_t$} \Comment{reject}
        \EndIf
        \EndFunction
    \end{algorithmic}
\end{algorithm}

\begin{algorithm}
    \caption{Affine-invariant HMC \alglabel{ai-hmc}}
    \begin{algorithmic}
\Function{AIHMCStep}{$\lnprob(\pos),\,\{\pos_k\}_{k=1}^K,\,\epsilon,\,L$}
\For{$k \gets 1 \textrm{ to } K$}
    \Let{${\mass_k}^{-1}$}{$\mathrm{Cov}(\pos_{[k]})$}
        \Comment{estimate the empirical mass matrix}
    \State $\mom_k \sim \normal{\bvec{0}}{\mass_k}$
        \Comment{sample the initial momentum exactly}
    \State
    \Let{$\mom^\prime$}{$\mom_k$} \Comment{save the initial coordinates}
    \Let{$\pos^\prime$}{$\pos_k$}
    \State
    \Let{$\mom^\prime$}{$\mom^\prime +
                \frac{\epsilon}{2}\,\nabla\lnprob(\pos^\prime)$}
        \Comment{run $L$ steps of leapfrog integration}
    \For{$l \gets 1 \textrm{ to } L$}
        \Let{$\pos^\prime$}{$\pos^\prime +
                \epsilon\,{\mass_k}^{-1}\,\mom^\prime$}
        \If{$l < L$}
            \Let{$\mom^\prime$}{$\mom^\prime +
                \epsilon\,\nabla\lnprob(\pos^\prime)$}
        \EndIf
    \EndFor
    \Let{$\mom^\prime$}{$\mom^\prime +
                \frac{\epsilon}{2}\,\nabla\lnprob(\pos^\prime)$}
        \Comment{synchronize the momentum and position}
    \State
    \State{$r \sim \mathcal{U}(0, 1)$}
    \If{$r < \exp\left[
        \lnprob(\pos^\prime)
        - \frac{1}{2}{\mom^\prime}^T{\mass_k}^{-1}\mom^\prime
        - \lnprob(\pos_k)
        + \frac{1}{2}{\mom_k}^T{\mass_k}^{-1}\mom_k \right]$}
        \Let{$\pos_k$}{$\pos^\prime$}   \Comment{accept}
    \Else
        \Let{$\pos_k$}{$\pos_k$}   \Comment{reject}
    \EndIf
\EndFor
\State\Return{$\{ \pos_k \}_{k=1}^K$}
\EndFunction
    \end{algorithmic}
\end{algorithm}

\clearpage
\bibliography{emcee-hmc}
\clearpage

\end{document}


%auto-ignore
\providecommand{\MainFolder}{..}
\documentclass[\MainFolder/Text.tex]{subfiles}
\begin{document}
\section{Twisted IBL-infinity-structure for spheres
%The $\IBL$-structure on the homology of $\OPQ_{110}^\PMC$.
}
\label{Section:HomSphere}
Let $e_0$, $e_1$ be the basis of $\Harm(\Sph{n})[1]$ defined by
\begin{equation*} \label{Eq:BasisOfHarm}
 e_0 \coloneqq \NOne \coloneqq \SuspU 1, \quad e_1\coloneqq\NVol \coloneqq \frac{1}{V}\SuspU\Vol.
\end{equation*}
The degrees satisfy
$$ \Abs{\NOne} = -1, \quad \Abs{\NVol} = n-1. $$
The matrix of the pairing $\Pair$ with respect to the basis $e_0$, $e_1$ reads
$$ \Pair=\begin{pmatrix}
 0 & 1 \\
 (-1)^{n} & 0
\end{pmatrix}. $$
The dual basis $e^0$, $e^1$ to $e_0$, $e_1$ with respect to $\Pair$ is thus
$$ e^0= \NVol,\quad e^1 = (-1)^{n} \NOne. $$
It follows that the matrix $(T^{ij})$ from~\eqref{Eq:PropagatorT} satisfies
\begin{equation*}
%\label{Eq:KerIdMatrix}
 (T^{ij}) = - \begin{pmatrix}
0 & 1 \\
1 & 0
\end{pmatrix}.
\end{equation*}
%This data will be used below to write down the canonical operations $\OPQ_{210}$, $\OPQ_{120}$ (see Def.~\ref{Def:CanonicaldIBL}).

We clearly have
$$ \CRedDBCyc \Harm(\Sph{1}) = \Bigl\{ \sum_{k=1}^\infty c_k \NVol^{k*} \BigMid c_k \in \R \Bigr\}, $$
where $\NVol^{k*}$ is the dual to the cyclic word $\NVol^k = \NVol \dots \NVol$ of length $k$. Observe that the cyclic symmetry gives
\begin{equation*}
\NVol^i = (-1)^{(n-1)(i-1)} \NVol^i\quad\text{for all }i\ge 1.
\end{equation*}
Therefore, $\NVol^{i*} = 0$ holds if both $n$ and $i$ are even.

For $n\ge 2$, the vector space $\Harm(\Sph{n})$ is connected and simply-connected, and Proposition~\ref{Prop:SimplCon} implies that there are no long reduced cyclic cochains (i.e., we have only finite sums of $\NVol^{k*}$'s).

The product $\mu_2: \Harm[1]^{\otimes 2} \rightarrow \Harm[1]$ from \eqref{Eq:HarmProd} has the following matrix with respect to the basis $\NOne$, $\NVol$:
\begin{equation*}
%\label{Eq:BinaryOperatorSphere}
 \mu_2 = \begin{pmatrix} \NOne &  \NVol \\ (-1)^n \NVol & 0 \end{pmatrix}.
\end{equation*}
Because $\mu_2(\NVol, \NVol) = 0$, we get
\begin{equation*}
%\label{Eq:RedHIBLSn}
\HIBL^\MC(\RedCycC(\Harm(\Sph{n})))[1] = \begin{cases}
                        \langle \Susp \NVol^{i*} \mid i \ge 1 \rangle & \text{for }n\ge 3\text{ odd}, \\
                        \langle  \Susp \NVol^{2i-1*} \mid i\ge 1 \rangle & \text{for }n\text{ even},\\
 \bigl\{ \Susp\sum_{k=1}^\infty c_k \NVol^{k*} \mid c_k\in \R \bigr\} & \text{for }n=1.
\end{cases}
\end{equation*}
Because we are in the strictly unital and strictly augmented case, we obtain
\begin{equation} \label{Eq:HIBLSn}
\HIBL^\MC(\CycC)[1] = \begin{cases}
\langle \Susp \NVol^{i*}, \Susp \NOne^{2j-1*} \mid i, j \ge 1\rangle & \text{for }n\ge 3 \text{ odd}, \\
\langle \Susp \NVol^{2 i-1*}, \Susp \NOne^{2j-1*} \mid i, j \ge 1\rangle &\text{for }n\text{ even}, \\
 \bigl\langle \Susp\sum_{k=1}^\infty c_k \NVol^{k*}, \Susp \NOne^{2j-1*}\mid c_k\in \R, j \ge 1\bigr\rangle & \text{for }n=1.
\end{cases}
\end{equation}
%Note that the canonical Maurer-Cartan element $\MC$ satisfies
%$$ (-1)^n \MC_{10}(\Susp \NOne \NVol \NOne) = \MC_{10}(\Susp\NVol\NOne\NOne) = \MC_{10}(\Susp\NOne\NOne\NVol) = (-1)^{n-2}. $$
The canonical $\IBL$-operations can be written as
\begin{align*}
\OPQ_{210}(\Susp^2 \psi_1 \otimes \psi_2)(\Susp \omega) &= \begin{multlined}[t]-\sum \varepsilon(\omega\mapsto \omega^1 \omega^2)[(-1)^{(n-1)\Abs{\omega^1}} \psi_1(e_0 \omega^1) \\ \psi_2(e_1 \omega^2) + (-1)^{\Abs{\omega_1}}\psi_1(e_1 \omega^1) \psi_2(e_0 \omega^2)], \end{multlined} \\
\OPQ_{120}(\Susp \psi)(\Susp^2 \omega_1\otimes \omega_2) & = \begin{multlined}[t] - \frac{1}{2} \sum \varepsilon(\omega_1\mapsto \omega_{1}^{1}) \varepsilon(\omega_2\mapsto \omega_{2}^{1}) [(-1)^{(n-1)\Abs{\omega_{1}^{1}}} \\ \psi(e_0 \omega_{1}^{1} e_1 \omega_{2}^{1})  + (-1)^{\Abs{\omega_{1}^{1}}}\psi(e_1 \omega_{1}^{1} e_0 \omega_{2}^{1})] \end{multlined}
\end{align*}
for all $\psi$, $\psi_1$, $\psi_2 \in \CDBCyc\Harm$ and generating words $\omega$, $\omega_1$, $\omega_2\in \BCyc\Harm$. For all $k$, $k_1$, $k_2 \ge 1$, we have
$$ \OPQ_{210}((\Susp \NVol^{k_1*}) \cdot (\Susp \NVol^{k_2*})) = 0\quad\text{and}\quad \OPQ_{120}(\Susp \NVol^{k*}) = 0$$
because both $\OPQ_{210}$ and $\OPQ_{120}$ feed $\NOne$ into their inputs. For the \emph{canonically twisted reduced $\IBL$-algebra}, this implies the following:
$$ \IBL\bigl(\HIBL^\MC(\RedCycC)\bigr) = \bigl(\HIBL^\MC(\RedCycC), \OPQ_{210} \equiv 0, \OPQ_{120} \equiv 0 \bigr)\quad \text{for all }n\in \N.  $$
By Proposition~\ref{Prop:Ones}, the only possibly non-zero relation  of $\IBL(\HIBL^\MC(\CycC))$ is
$$\begin{aligned}
& \OPQ_{210}(\Susp \NOne^* \otimes \Susp \NVol^{k*}) \\[\jot]
&\qquad = (-1)^{n-2} \Susp (\NVol^{k*} \circ \iota_\NVol) \\
&\qquad = (-1)^{n-2}\bigl(\sum_{i=1}^{k-1} (-1)^{i \Abs{\NVol}}\bigr)\Susp\NVol^{k-1 *}
= \begin{cases}
   -(k-1) \Susp \NVol^{k-1*} & \text{for }n\text{ odd},\\
    0 & \text{for }n\text{ even}.
  \end{cases}\end{aligned}$$
The reason for $0$ for even $n$ is that either $k$ is odd, in which case $\sum_{i=1}^{k-1} (-1)^i = 0$, or $k$ is even, in which case $\NVol^{k*} = 0$. Therefore, for the \emph{canonically twisted $\IBL$-algebra}, we have
\begin{equation*}
%\label{Eq:CanonTwistIBL}
\IBL\bigl(\HIBL^\MC(\CycC)\bigr) = \bigl(\HIBL^\MC(\CycC), \OPQ_{210}, \OPQ_{120} \equiv 0 \bigr)\quad \text{for all }n\in \N,
\end{equation*}
where $\HIBL^\MC(\CycC)$ is given by \eqref{Eq:HIBLSn} and $\OPQ_{210}$ satisfies the following:
\begin{description}[font=\normalfont\itshape]
\item[($n$ even):] $\OPQ_{210} \equiv 0$.
\item[($n\ge 3$ odd):] The non-trivial relations are
$$ \OPQ_{210}(\Susp \NOne^* \otimes \Susp \NVol^{k*}) = \OPQ_{210}(\Susp \NVol^{k*} \otimes \Susp \NOne^*) = -(k-1) \NVol^{k-1*}\quad\text{for }k\ge 2.  $$
\item[($n=1$):]  The non-trivial relations are
$$ \OPQ_{210}\Bigl(\Susp \NOne^* \otimes \Susp\sum_{k=1}^\infty c_k \NVol^{k*}\Bigr) = - \Susp \sum_{k=1}^\infty k c_{k+1} \NVol^{k*} \quad\text{for }c_k\in \R. $$
\end{description}
Recall that the twist by $\MC$ does not produce any higher operation $\OPQ_{1lg}^\MC$.

We will now consider $\dIBL^\PMC(\CycC(\Harm(\Sph{n})))$. Recall that $\OPQ_{110}^\PMC = \OPQ_{210}\circ_1 \PMC_{10}$, $\OPQ_{210}^\PMC = \OPQ_{210}$ and $\OPQ_{120}^\PMC = \OPQ_{120} + \OPQ_{210}\circ_1 \PMC_{20}$. By Proposition~\ref{Proposition:MCSphere}, we have $\PMC_{10} = \MC_{10}$ for all $n\in \N$ and $\PMC_{20} = 0$ for all $n\ge 2$. It follows that $\OPQ_{110}^\PMC = \OPQ_{110}^\MC$ for all $n\in \N$ and that the only non-trivial twist may occur in $\OPQ_{120}^\PMC$ for $\Sph{1}$. Using~\eqref{Eq:Twistn2}, we get for all $\psi\in \CDBCyc \Harm(\Sph{n})$ and generating words $\omega_1$, $\omega_2 \in \BCyc \Harm(\Sph{n})$ the following:
\begin{equation}
\begin{aligned}
& (\OPQ_{210}\circ_1 \PMC_{20})(\Susp \psi)(\Susp \omega_1 \otimes \Susp \omega_2) \label{Eq:CoprodTwist}\\
& \quad = \begin{multlined}[t] (-1)^{n-2}\Bigl[ \sum \varepsilon(\omega_1 \mapsto \omega_1^1 \omega_1^2)\psi(\NOne \omega_1^1)\PMC_{20}(\Susp \NVol \omega_1^2 \otimes \Susp \omega_2) \\ {}+ (-1)^{(n-3+\Abs{\omega_1})(n-3+\Abs{\omega_2})} \sum \varepsilon(\omega_2 \mapsto \omega_2^1 \omega_2^2) \psi(\NOne \omega_2^1)  \\ \PMC_{20}(\Susp \NVol \omega_2^2 \otimes \Susp \omega_1)\Bigr].  \end{multlined}\end{aligned}
\end{equation}

In this paragraph, we suppose that $n=1$ and compute $\OPQ_{120}^\PMC$. Clearly, $(\OPQ_{210}\circ_1\PMC_{20})(\Susp \NVol^{k*}) = 0$ for all $k\ge 1$ since~$\NOne$ is fed into $\NVol^{k*}$. A non-zero evaluation of $(\OPQ_{210}\circ_1\PMC_{20})(\Susp\NOne^{k*})$ for some $k\ge 1$ odd is possible only on $\Susp \NOne^{k-1}\NVol^{k_1}\otimes \Susp\NVol^{k_2}$ for $k_1$, $k_2\ge 0$ (up to a transposition of arguments and their cyclic permutation). If $k>1$, only the first summand of~\eqref{Eq:CoprodTwist} contributes, and we get
%
\begin{equation*}
\begin{aligned}
&(\OPQ_{210}\circ_1\PMC_{20})(\Susp\NOne^{k*})(\Susp \NOne^{k-1}\NVol^{k_1}\otimes \Susp\NVol^{k_2})  \\
& \qquad= \begin{multlined}[t] (-1)^{n-2} \sum \varepsilon(\NOne^{k-1}\NVol^{k_{1}} \mapsto \omega_1 \omega_2) \NOne^{k*}(\NOne \omega_1) \PMC_{20}(\Susp \NVol \omega_2\otimes \Susp \NVol^{k_2}) \end{multlined} \\ & \qquad = (-1)^{n-2} \NOne^{k*}(\NOne\NOne^{k-1}) \PMC_{20}(\Susp \NVol\NVol^{k_{1}}\otimes \Susp \NVol^{k_2})  \\ & \qquad = - \PMC_{20}(\Susp \NVol^{k_{1}+1}\otimes \Susp\NVol^{k_2}).
\end{aligned}
\end{equation*}
%
According to Proposition~\ref{Proposition:MCSphere}, this is non-zero if and only if $k_{1}+k_2$ is odd. It follows that
$$ \OPQ_{120}^\PMC \neq \OPQ_{120}^\MC = \OPQ_{120}\quad\text{on the chain level for }\Sph{1}. $$
However, the chains $\Susp \NOne^{k-1}\NVol^{k_1}\otimes \Susp \NVol^{k_2}$ for $k>1$ do not survive to the homology (c.f., \eqref{Eq:HIBLSn}). The only possibility is thus $k=1$. In this case, both summands of~\eqref{Eq:CoprodTwist} contribute, and  using~\eqref{Eq:MC20}, we get for all $k_1$, $k_2 \ge 1$ the following:
%
\allowdisplaybreaks
\begin{align*}
&(\OPQ_{210}\circ_1\PMC_{20})(\Susp\NOne^*)(\Susp\NVol^{k_1} \otimes \Susp\NVol^{k_2}) \\ &\qquad = \begin{multlined}[t](-1)^{n-2}\Bigl[\sum \varepsilon(\NVol^{k_1} \mapsto \NVol^0 \NVol^{k_1}) \NOne^*(\NOne) \PMC_{20}(\Susp\NVol^{k_1+1}\otimes \Susp \NVol^{k_2}) \\ {}+ (-1)^{(n-3 + k_1(n-1))(n-3 + k_2(n-1))} \sum \varepsilon(\NVol^{k_2} \mapsto \NVol^0 \NVol^{k_2})  \NOne^*(\NOne)\\ \PMC_{20}(\Susp \NVol^{k_2 + 1} \otimes\Susp\NVol^{k_1})\Bigr] \end{multlined}
\\&\qquad = -  k_1 \PMC_{20}(\Susp\NVol^{k_1+1}\otimes\Susp\NVol^{k_2}) -  k_2 \PMC_{20}(\Susp\NVol^{k_2+1}\otimes\Susp\NVol^{k_1}) \\
&\qquad = \begin{multlined}[t] -\frac{1}{2}(k_1+k_2+1)!I(k_1+k_2+1)\Bigl[(-1)^{k_1} k_1 (k_1+1) \binom{k_1+k_2}{k_1+1} \\ {}+ (-1)^{k_2} k_2  (k_2+1) \binom{k_1+k_2}{k_2+1}\Bigr] \end{multlined} \\
&\qquad =  -\frac{1}{2}(k_1+k_2+1)! k_1 k_2 \binom{k_1+k_2}{k_1} \underbrace{I(k_1 + k_2 + 1) [(-1)^{k_1} + (-1)^{k_2}]}_{=:(*)}.
\end{align*}
%
Denoting $k\coloneqq k_1 + k_2 + 1$, we have that $(-1)^{k_1} + (-1)^{k_2} = 0$ for $k$ even and $I(k) = 0$ for~$k$ odd. Therefore, $(*) = 0$ for any $k_1$, $k_2\ge 1$.
This implies that
$$ \OPQ_{120}^\PMC = \OPQ_{120}^\MC = \OPQ_{120}\quad\text{on the homology for }\Sph{1}. $$
We conclude that the \emph{twisted $\IBL$-algebra} satisfies
$$ \IBL\bigl(\HIBL^\PMC(\CycC(\Harm(\Sph{n})))\bigr) = \IBL\bigl(\HIBL^\MC(\CycC(\Harm(\Sph{n})))\bigr) \quad\text{for all }n\in \N. $$

As for the \emph{higher twisted operations}, combining Propositions~\ref{Prop:dIBL} and~\ref{Proposition:MCSphere}, we see that for~$\Sph{n}$ with $n\in \N\backslash\{2\}$ all higher operations~$\OPQ_{1lg}^\PMC$ vanish already on the chain level. For $n=2$, we have that $\OPQ_{1l0}^\PMC = 0$ for all $l\ge 3$ and $\OPQ_{111}^\PMC = 0$ on the chain level. However, we did not prove that all higher operations vanish on the chain level. As for the operations induced on the homology, the graded vector space~$\HIBL^\PMC(\CycC(\Harm(\Sph{2})))$ is concentrated in even degrees and $\OPQ_{1lg}^\PMC$ are odd (see Definition~\ref{Def:IBLInfty}). Therefore, all higher operations vanish also on $\HIBL^\PMC(\CycC(\Harm(\Sph{2})))$.

The string topology $\StringH(\Sph{n})$ and the string operations $\StringOp_2$ and $\StringCoOp_2$ were computed in \cite{Basu2011} for all $n\in \N$.
%For $n\ge 2$, they used the method of minimal models to get $\StringCoH^*(\Sph{n}; \Q)$. For $n$ odd, $\StringOp_2$
%\eqref{Eq:Gysin} They obtained $\StringH_*(\Sph{1}; \Z)$ by topological considerations.
We review their results and basic ideas below:

We will consider \emph{even spheres} first. The minimal model for the Borel construction $\LoopBorel \Sph{2m}$ for $m\in \N$ is denoted by $\Lambda^{\Sph{1}}(2,m)$ --- it is the free graded commutative dga (=:cdga) over~$\R$ generated by homogenous vectors $x_1$, $y_1$, $x_2$, $y_2$, $u$ of degrees
$$ \Abs{x_1} = 2m,\quad \Abs{y_1} = 2m - 1, \quad\Abs{x_2} = 4m-1,\quad \Abs{y_2} = 2(2m - 1),\quad\Abs{u} = 2, $$
whose differential $\Dd$ satisfies
$$ \Dd y_1 = 0, \quad \Dd x_1 = y_1 u, \quad \Dd y_2 = - 2 x_1 y_1, \quad\Dd x_2 = x_1^2 + y_2 u. $$
The minimal model for the loop space $\Loop \Sph{2m}$ is the dga $\Lambda(2,m)$ which is obtained from $\Lambda^{\Sph{1}}(2,m)$ by setting $u = 0$. A computation (see \cite[Theorem 3.6]{Basu2011}) gives the following for all $m\in \N$:
\begin{equation}\label{Eq:EvenSphereString}
\begin{aligned}
\H^*(\Loop \Sph{2m}; \R) &\simeq \H(\Lambda(2,m), \Dd) =\langle y_2^i x_1 - 2 i y_1 x_2 y_2^{i-1}, y_1 y_2^j, 1 \mid i, j\in \N_0 \rangle,  \\
\StringCoH^*(\Loop \Sph{2m}; \R) &\simeq \H(\Lambda^{\Sph{1}}(2,m),\Dd) = \langle y_1 y_2^i, u^j \mid i, j\in \N_0\rangle,
\end{aligned}
\end{equation}
where $y_2^0 \coloneqq u^0 \coloneqq 1$ is the unit in $\Lambda^{\mathrlap{\Sph{1}}\hphantom{S}}(2,m)$ and $\langle \cdot \rangle$ denotes the linear span over~$\R$. Clearly, the cohomology groups are degree-wise finite-dimensional, and hence, using the universal coefficient theorem, they are isomorphic to the corresponding homology groups. We can thus identify $\H(\Loop \Sph{2m}; \R)$ and $\StringH(\Loop\Sph{2m}; \R)$ with the vector spaces on the right hand side of \eqref{Eq:EvenSphereString}. We have $\StringH_{2k}= \langle u^k \rangle$ for all $k\in \N_0$, and hence the multiplication with $u$ induces an isomorphism $\StringH_{2k} \simeq \StringH_{2k+2}$. This corresponds to the cap product with the Euler class in \eqref{Eq:Gysin}, and exactness of the sequence implies $\Mark(\StringH_{2k}) = \Erase(\StringH_{2k}) = 0$. Using this and degree considerations, we get $\StringOp_2=\StringCoOp_2 = 0$.

We will now consider \emph{odd spheres} with $n\ge 3$. The minimal model for $\LoopBorel \Sph{2m+1}$
for $m\in \N$ is denoted simply by $\Lambda(x,y,u)$ --- it is the free cdga on homogenous vectors $x$, $y$, $u$ of degrees
$$ \Abs{x} = 2m+1, \quad \Abs{y} = 2m,\quad \Abs{u} = 2, $$
such that
$$ \Dd x = y u, \quad \Dd y = \Dd u = 0. $$
We get immediately
$$\begin{aligned}
\H^*(\Loop \Sph{2m+1}; \R) & \simeq \langle x^i, y^j \mid i, j \in \N_0 \rangle,  \\
\StringCoH^*(\Loop \Sph{2m+1}; \R) &\simeq \langle y^i, u^j \mid i,j \in \N_0 \rangle, \end{aligned}$$
and we can again identify $\H$ and $\StringH$ with the vector spaces on the right hand side. Clearly, $\StringH_{2k-1} = 0$ for all $k\in \N$, and hence $\StringOp_2 = \StringCoOp_2 = 0$ for degree reasons (the operations are odd).

We will now consider \emph{the circle} $\Sph{1}$. For every $i\in \Z$, let $\alpha_i : \Sph{1} \rightarrow \Sph{1}$ and $\theta_i : \Sph{1} \rightarrow \Loop \Sph{1}$ be the maps defined by
$$ \alpha_i(z) \coloneqq z^i\quad\text{and}\quad\theta_i(w) \coloneqq w \alpha_i \quad \text{for all }w,z\in \Sph{1}\subset \C. $$
By examining the equivariant homology of connected components of $\Loop \Sph{1}$ containing~$\alpha_i$ separately as in \cite[Section 2.1.4]{Basu2011}, we get
$$\begin{aligned}
\H(\Loop \Sph{1}; \R) &=\langle \alpha_i, \theta_j \mid i,j\in \Z\rangle, \\
\StringH(\Loop \Sph{1}; \R) &= \langle u^i, \theta_0 u^j, \alpha_k \mid i, j \in \N_0, k\in \Z\backslash\{0\} \rangle,
\end{aligned}$$
where $u$ corresponds to the Euler class and
$$ \Abs{u} = 2, \quad \Abs{\theta_i} =1, \quad \Abs{\alpha_i} = 0 $$
are the degrees in the singular chain complex. On \cite[p. 21]{Basu2011} they show that the string cobracket $\StringCoOp_2$ is $0$ and that all non-trivial relations for the string bracket $\StringOp_2: \StringH(\Loop \Sph{1})[2]^{\otimes 2}\rightarrow \StringH(\Loop \Sph{1})[2]$ are the following:
\begin{equation*}
%\label{Eq:NontrivRelString}
\StringOp_2( \Susp \alpha_{k}, \Susp \alpha_{-k}) = k^2 \Susp\theta_0 \quad\forall k \in \N.
\end{equation*}

We will now compare the reduced $\IBL$-structures motivated by Conjecture~\ref{Conj:StringTopology}. The point-reduced versions $\RedStringH(\Loop \Sph{n})$ for $n\ge 2$ are obtained from $\StringH(\Loop \Sph{n})$ by deleting $u^i$. We have the following isomorphisms of graded vector spaces:
$$ \begin{aligned}
  \HIBL^{\PMC}(\RedCycC(\Harm(\Sph{n})))[1] &\longrightarrow  \RedStringH(\Loop \Sph{n})[3-n] && \\
       \Susp \NVol^i &\longmapsto \Susp y^i  &&\text{for }n> 1\text{ odd}, \\
    \Susp \NVol^{2i+1} &\longmapsto \Susp y_1 y_2^i  &&\text{for }n\text{ even}.
  \end{aligned}$$
Because all operations are trivial, it induces the isomorphism
$$ \IBL\bigl(\HIBL^{\PMC}(\RedCycC(\Harm(\Sph{n})))\bigr) \simeq \IBL\bigl(\RedStringH(\Loop \Sph{n})[2-n]\bigr) \quad\text{for }n\ge 2. $$
For $n = 1$, the reduced homology is seemingly different.

\begin{Remark}[Triviality for degree reasons]\label{Rem:DegRes}\Modify[caption={DONE Too dense text}]{Add paragraphs here --- too dense. Ans also add $\Sph{1}$!!}
%Generators of the reduced cyclic cohomology for $\Sph{2m-1}$ with $m\ge 2$ are $\Susp \NVol^{i*}$ of degrees $2n-4$, $3n-5$, $4n-6$, $\dots$, whereas the generators for $\Sph{2m}$ are $\Susp \NVol^{2i-1 *}$ of degrees $2n-4$, $4n-6$, $6n-8$, $\dots$ for $i=1$, $2$, $3$, $\dots$. We see that in both cases, the reduced homology concentrates in even degrees, and hence any $\IBLInfty$-structure on the reduced homology must be trivial for degree reasons.Prop:Ones
%$-2(n-3)(g-1) + n -4$
%
%The non-reduced \eqref{Prop:Ones}
%$\OPQ_{1lg}^\PMC(\Susp\NOne^*) = - \PMC_{lg} \circ \iota_\NVol$
%The non-reduced cyclic cohomology contains in addition $\Susp \NOne^{2i-1*}$ of degrees $n-4$, $n-6$, $n-8$, $\dots$ and the non-reduced string topology contains $\Susp u^i$ of degrees $n-1$, $n+1$, $n+3$, $\dots$ for $i=1$, $2$, $3$, $\dots$.
The graded vector spaces
$$ \StringH(\Loop \Sph{2m-1})[3-n]\quad\text{and}\quad\HIBL^\PMC(\CycC(\Harm(\Sph{2m})))[1] $$
are concentrated in even degrees, and so any $\IBLInfty$-structure must be trivial for degree reasons. On the other hand, the graded vector spaces
$$ \StringH(\Loop \Sph{2m})\quad\text{and}\quad\HIBL^\PMC(\CycC (\Harm(\Sph{2m-1})))[1] $$
have both even and odd degrees, and hence an additional argument is needed to prove vanishing of the $\IBL$-structure. This is not the case of the reduced homology, which is again concentrated in even degree.\qedhere
%Therefore, the integrals computed in Section~\ref{Section:MCSphere} were useful at least for $\Sph{2m-1}$.

\Add[caption={DONE Non-trivial degree}]{Add here the computation of which relations on homology are not implied automatically by degree reasons.}
\end{Remark}
\end{document}


\documentclass{report}

\usepackage{amsmath} % provides numberwithin (and lots more)
\usepackage{graphicx}
\usepackage{listings}
\usepackage[backend=bibtex]{biblatex}
\bibliography{qfrmTechnicalQuestionsDev}

\newtheorem{problem}{}
\numberwithin{problem}{chapter} % important bit
\let\oldroblem\problem
\renewcommand{\problem}{\oldroblem\normalfont}
\newcommand{\ds}{\displaystyle}

\begin{document}

\begin{titlepage}
\begin{center}
 {\huge\bfseries Fixed Income Primer\\}
 % ----------------------------------------------------------------
 \vspace{1.5cm}
 {\bfseries Pete Benson}\\[5pt]
 pbenson@umich.edu\\[14pt]
  % ----------------------------------------------------------------
 \vspace{10cm}
 % ----------------------------------------------------------------
\includegraphics{QFRM_rgb}\\[5pt]
{Department of Mathematics}\\[5pt]
{530 Church Street, 2082C East Hall}\\[5pt]
{Ann Arbor, MI 48109-1043,
 USA}\\
 \vfill

\end{center}
\end{titlepage}

\tableofcontents
\newpage

%----------------------
% review
%----------------------
\chapter{Introduction}
Discuss: this preps you for a fixed income interview, and includes practice exercises via Excel, python, etc.

\section{What are fixed income instruments?}
Fixed income instruments feature guaranteed payments. Examples include short term borrowing, bonds, swaps, MBS (mortgage-backed securities), CDOs (collateralized debt obligations), CDS (credit default swaps), and their many variations.

To model a fixed income instrument, you need to understand the underlying contract, which guides the timing and size of payments, and who will be paying. Timing and size of payments are used to value payments, and the payer affects the likelihood that payment will be made.

\subsection{Payers}
Broadly, you can divide payers into riskless payers (e.g. sovereign countries that owe money denominated in their own currency), and everyone else. Typical examples of a riskless payer would be the US Treasury, making payments on bills, notes, and bonds.  Note that this would not include a country that does not control its own currency, such as countries in the EU. Greece, for example, can issue EUR-denominated debt, but that does not guarantee they can meet their obligations. Even Germany could potentially default on EUR-denominated debt, but this is considered very unlikely. In rare instances, countries may even default bonds denominated in their own currency, such as in the 1998 Russian financial crisis.

The rest of the payers could be categorized as businesses, or pools of individuals, and there are securities for each.

\subsection{Zero coupon bond with riskless payer}
Discuss time value of money, present value, term structure of interest rates, term structure of discount prices.

\subsection{Fixed coupon bond with riskless payer}
Discuss simple pricing based off discount curve, also formulated in terms of interest rates, Par vs premium vs discount bonds. Compound interest continuous vs. periodic compounding, coupon schedules, accounting conventions, accrual (clean vs dirty), primary vs secondary markets, treasury auctions.

\subsection{Bootstrapping zero coupon curves}
Forward rate curve, mention variety of techniques, demonstrate piecewise flat forward curve.

\subsection{Bootstrapping a zero coupon curve}

%----------------------
%Bibliography
%----------------------
\printbibliography

\end{document}