\newcommand{\eos}{\hfill{}$\cdots\diamond\cdots$\hfill{}\vspace{5mm}}

\newcommand{\mathpredn}{\mathbin{-\!\!\!\mid\mid\!\rightarrow}}

\newcommand{\KC}{\con{K}}
\newcommand{\SC}{\con{S}}
\newcommand{\bk}{\char'134}

\chapter{Example: Combinatory Logic}
\label{chap:combin}

\section{Introduction}
\label{sec:Introduction}

This small case study is a formalisation of (variable-free)
combinatory logic.  This logic is of foundational importance in
theoretical computer science, and has a very rich theory.  The example
builds principally on a development done by Tom Melham.  The complete
script for the development is available as \texttt{clScript.sml} in
the \texttt{examples/ind\_def} directory of the distribution.  It is
self-contained and so includes the answers to the exercises set at the
end of this document.

The HOL sessions assume that the Unicode trace is \emph{on} (as it is
by default), meaning that even though the inputs may be written in
pure ASCII, the output still uses nice Unicode output (symbols such as
$\forall$ and $\Rightarrow$).  The Unicode symbols could also be used
in the input.

\section{The type of combinators}
\label{sec:Type-Combinators}

The first thing we need to do is define the type of
\emph{combinators}.  There are just two of these, \KC{} and \SC, but
we also need to be able to \emph{combine} them, and for this we need
to introduce the notion of application.  For lack of a better ASCII
symbol, we will use the hash (\#) to represent this in the logic:
\setcounter{sessioncount}{0}
\begin{session}
\begin{verbatim}
- Hol_datatype `cl = K | S | # of cl => cl`;
> val it = () : unit
\end{verbatim}
\end{session}
We also want the \# to be an infix, so we set its fixity to be a tight
left-associative infix:
\begin{session}
\begin{verbatim}
- set_fixity "#" (Infixl 1100);
> val it = () : unit
\end{verbatim}
\end{session}

% C = S (S (K S) (S (K K) S)) (K K)

\section{Combinator reductions}
\label{sec:Comb-Reduct}

Combinatory logic is the study of how values of this type can evolve
given various rules describing how they change.  Therefore, our next
step is to define the reductions that combinators can undergo.  There
are two basic rules:
\[\begin{array}{l@{\;\;\rightarrow\;\;}l}
\KC\;x\;y & x\\
\SC\;f\;g\;x & (f x)(g x)
\end{array}\]
Here, in our description outside of HOL, we use juxtaposition instead
of the \#.  Further, juxtaposition is also left-associative, so that
$\con{K}\;x\;y$ should be read as $\con{K}\;\#\;x\;\#\;y$ which is in
turn $(\con{K}\;\#\;x)\;\#\;y$.

Given a term in the logic, we want these reductions to be able to fire
at any point, not just at the top level, so we need two further
congruence rules:\[
\begin{array}{l}
\infer{x\;y\;\;\rightarrow\;\;x'\;y}{x\;\;\rightarrow\;\;x'}\\[5mm]
\infer{x\;y\;\;\rightarrow\;\;x\;y'}{y\;\;\rightarrow\;\;y'}
\end{array}\]
In HOL, we can capture this relation with an inductive definition.
First we need to set our arrow symbol up as an infix to make everything that
bit prettier
\begin{session}
\begin{verbatim}
- set_fixity "-->" (Infix(NONASSOC, 450));
> val it = () : unit
\end{verbatim}
\end{session}
We make our arrow symbol non-associative, thereby making it a
parse error to write \verb!x --> y --> z!. It would be nice to be able
to write this and have it mean \verb!x --> y /\ y --> z!, but this is
not presently possible with the HOL parser.

Our next step is to actually define the relation with the
\ml{xHol\_reln} function.  In addition to a quotation specifying the
rules for the new relation, it requires a name to use as the stem for
the theorems it proves.  We pick the string \ml{"redn"}.\footnote{The
  related function \ml{Hol\_reln} can be used if the system's choice
  of stem is acceptable.  In this case, \ml{Hol\_reln} can't cope with the
  non-alphanumeric characters in \holtxt{-->} and will raise an
  error.}
The \ml{xHol\_reln} function for
doing this returns three separate theorems, and we bind the first (the
``rules'' theorem) and
third (the ``cases'' theorem):
\begin{session}
\begin{alltt}
val (redn_rules, _, redn_cases) = xHol_reln "redn"
   `(!x y f. x --> y   ==>    f # x --> f # y) /\bs
    (!f g x. f --> g   ==>    f # x --> g # x) /\bs
    (!x y.   K # x # y --> x) /\bs
    (!f g x. S # f # g # x --> (f # x) # (g # x))`;
> val redn_rules =
    |- (\(\forall\)x y f. x --> y \(\Rightarrow\) f # x --> f # y) \(\land\)
       (\(\forall\)f g x. f --> g \(\Rightarrow\) f # x --> g # x) \(\land\)
       (\(\forall\)x y. K # x # y --> x) \(\land\)
       \(\forall\)f g x. S # f # g # x --> f # x # (g # x) : thm
  val redn_cases =
    |- \(\forall\)a0 a1.
         a0 --> a1 \(\Leftrightarrow\)
         (\(\exists\)x y f. (a0 = f # x) \(\land\) (a1 = f # y) \(\land\) x --> y) \(\lor\)
         (\(\exists\)f g x. (a0 = f # x) \(\land\) (a1 = g # x) \(\land\) f --> g) \(\lor\)
         (\(\exists\)y. a0 = K # a1 # y) \(\lor\)
         \(\exists\)f g x. (a0 = S # f # g # x) \(\land\) (a1 = f # x # (g # x))
    : thm
\end{alltt}
\end{session}

In addition to proving these three theorems for us, the inductive
definitions package has also saved them to disk.

Now, using our theorem \texttt{redn\_rules} we can demonstrate single
steps of our reduction relation:
\begin{session}
\begin{verbatim}
- PROVE [redn_rules] ``S # (K # x # x) --> S # x``;
Meson search level: ...
> val it = |- S # (K # x # x) --> S # x : thm
\end{verbatim}
\end{session}
The system we have just defined is as powerful as the
$\lambda$-calculus, Turing machines, and all the other standard models
of computation.

One useful result about the combinatory logic is that it is
\emph{confluent}.  Consider the term $\SC\;z\;(\KC\;\KC)\;(\KC\; y\;
x)$.  It can make two reductions, to $\SC\;z\;(\KC\;\KC)\;y$ and also
to $(z\;(\KC\;y\;x))\,(\KC\;\KC\;(\KC\;y\;x))$.  Do these two choices
of reduction mean that from this point on the terms have two
completely separate histories?  Roughly speaking, to be confluent
means that the answer to this question is \emph{no}.

\section{Transitive closure and confluence}
\label{sec:Transitive-Clos-Conf}

A notion crucial to that of confluence is that of \emph{transitive
  closure}.  We have defined a system that evolves by specifying how
an algebraic value can evolve into possible successor values in one
step.  The natural next question is to ask for a characterisation of
evolution over one or more steps of the $\rightarrow$ relation.

In fact, we will define a relation that holds between two values if
the second can be reached from the first in zero or more steps.  This
is the \emph{reflexive, transitive closure} of our original relation.
However, rather than tie our new definition to our original relation,
we will develop this notion independently and prove a variety of
results that are true of any system, not just our system of
combinatory logic.

So, we begin our abstract digression with another inductive
definition.  Our new constant is \con{RTC}, such that
$\con{RTC}\;R\;x\;y$ is true if it is possible to get from $x$ to $y$
with zero or more ``steps'' of the $R$ relation.  (The standard
notation for $\con{RTC}\;R$ is $R^*$.) We can express this idea with
just two rules.  The first \[ \infer{\con{RTC}\;R\;x\;x}{} \] says
that it's always possible to get from $x$ to $x$ in zero or more
steps.  The second \[
\infer{\con{RTC}\;R\;x\;z}{R\;x\;y\qquad\con{RTC}\;R\;y\;z}
\] says that if you can take a single step from $x$ to $y$, and then
take zero or more steps to get $y$ to $z$, then it's possible to take
zero or more steps to get between $x$ and $z$.  The realisation of
these rules in HOL is again straightforward.

(As it happens, \con{RTC} is already a defined constant in the context
we're working in (it is found in \texttt{relationTheory}), so we'll
hide it from view before we begin.  We thus avoid messages telling us
that we are inputting ambiguous terms.  The ambiguities would always
be resolved in the favour of more recent definition, but the warnings
are annoying.)
\begin{session}
\begin{alltt}
val _ = hide "RTC";

val (RTC_rules, _, RTC_cases) = Hol_reln `
    (!x.     RTC R x x) /\bs
    (!x y z. R x y /\bs RTC R y z ==> RTC R x z)`;
<<HOL message: inventing new type variable names: 'a>>
> val RTC_rules =
    |- \(\forall\)R. (\(\forall\)x. RTC R x x) \(\land\)
           \(\forall\)x y z. R x y \(\land\) RTC R y z \(\Rightarrow\) RTC R x z : thm
  val RTC_cases =
    |- \(\forall\)R a0 a1. RTC R a0 a1 \(\Leftrightarrow\) (a1 = a0) \(\lor\)
                                \(\exists\)y. R a0 y \(\land\) RTC R y a1 : thm
\end{alltt}
\end{session}
Now let us go back to the notion of confluence.  We want this to mean
something like: ``though a system may take different paths in the
short-term, those two paths can always end up in the same place''.
This suggests that we define confluent thus:
\begin{session}
\begin{verbatim}
- val confluent_def = Define
    `confluent R =
       !x y z. RTC R x y /\ RTC R x z ==>
               ?u. RTC R y u /\ RTC R z u`;
\end{verbatim}
\end{session}
This property states of $R$ that we can ``complete the diamond'';
if we have
\[\xymatrix @R=5mm @C=2.5mm {
 & x \ar[dl]_{*} \ar[dr]^{*} & \\
y & & z }\]
\[\xymatrix @R=5mm @C=2.5mm {
& x \ar[dl]_{*} \ar[dr]^{*} & \\
y \ar@{.>}[dr]_{*} & & z \ar@{.>}[dl]^{*} \\
& u}\]

One nice property of confluent relations is that from any one starting
point they produce no more than one \emph{normal form}, where a normal
form is a value from which no further steps can be taken.
\begin{session}
\begin{alltt}
- val normform_def = Define`normform R x = !y. ~(R x y)`;
<<HOL message: inventing new type variable names: 'a, 'b>>
Definition has been stored under "normform_def".
> val normform_def = |- \(\forall\)R x. normform R x \(\Leftrightarrow\) \(\forall\)y. \(\neg\)R x y : thm
\end{alltt}
\end{session}
In other words, a system has an $R$-normal form at $x$ if there are no
connections via $R$ to any other values.  (We could have written
\verb!~?y. R x y! as our RHS for the definition above.)

We can now prove the following:
\begin{session}
\begin{alltt}
- g `!R. confluent R ==>
         !x y z.
           RTC R x y /\bs normform R y /\bs
           RTC R x z /\bs normform R z ==> (y = z)`;
<<HOL message: inventing new type variable names: 'a>>
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         \(\forall\)R.
           confluent R \(\Rightarrow\)
           \(\forall\)x y z. RTC R x y \(\land\) normform R y \(\land\) RTC R x z \(\land\) normform R z \(\Rightarrow\)
                   (y = z)
\end{alltt}
\end{session}
We rewrite with the definition of confluence:
\begin{session}
\begin{alltt}
- e (RW_TAC std_ss [confluent_def]);
OK..
1 subgoal:
> val it =
    y = z
    ------------------------------------
      0.  \(\forall\)x y z. RTC R x y \(\land\) RTC R x z \(\Rightarrow\) \(\exists\)u. RTC R y u \(\land\) RTC R z u
      1.  RTC R x y
      2.  normform R y
      3.  RTC R x z
      4.  normform R z
\end{alltt}
\end{session}
    Our confluence property is now assumption 0, and we can use it to
    infer that there is a $u$ at the base of the diamond:
\begin{session}
\begin{alltt}
- e (`?u. RTC R y u /\bs RTC R z u` by PROVE_TAC []);
OK..
Meson search level: .........
1 subgoal:
> val it =
    y = z
    ------------------------------------
      0.  \(\forall\)x y z. RTC R x y \(\land\) RTC R x z \(\Rightarrow\) \(\exists\)u. RTC R y u \(\land\) RTC R z u
      1.  RTC R x y
      2.  normform R y
      3.  RTC R x z
      4.  normform R z
      5.  RTC R y u
      6.  RTC R z u
\end{alltt}
\end{session}
    So, from $y$ we can take zero or more steps to get to $u$ and
    similarly from $z$.  But, we also know that we're at an $R$-normal
    form at both $y$ and $z$.  We can't take any steps at all from
    these values.  We can conclude both that $u = y$ and $u = z$, and
    this in turn means that $y = z$, which is our goal.  So we can
    finish with
\begin{session}
\begin{alltt}
- e (PROVE_TAC [normform_def, RTC_cases]);
OK..
Meson search level: ..........

Goal proved. [...]
> val it =
    Initial goal proved.
    |- \(\forall\)R.
         confluent R \(\Rightarrow\)
         \(\forall\)x y z.
           RTC R x y \(\land\) normform R y \(\land\) RTC R x z \(\land\) normform R z \(\Rightarrow\)
           (y = z)
\end{alltt}
\end{session}
Packaged up so as to remove the sub-goal package commands, we can
prove and save the theorem for future use by:
\begin{session}
\begin{verbatim}
val confluent_normforms_unique = store_thm(
  "confluent_normforms_unique",
  ``!R. confluent R ==>
        !x y z. RTC R x y /\ normform R y /\
                RTC R x z /\ normform R z ==> (y = z)``,
  RW_TAC std_ss [confluent_def] THEN
  `?u. RTC R y u /\ RTC R z u` by PROVE_TAC [] THEN
  PROVE_TAC [normform_def, RTC_cases]);
\end{verbatim}
\end{session}
\eos{}

Clearly confluence is a nice property for a system to have.  The
question is how we might manage to prove it.  Let's start by defining
the diamond property that we used in the definition of confluence.
\begin{session}
\begin{alltt}
- val diamond_def = Define
    `diamond R = !x y z. R x y /\bs R x z ==> ?u. R y u /\bs R z u`;
<<HOL message: inventing new type variable names: 'a>>
Definition has been stored under "diamond_def".
> val diamond_def =
    |- \(\forall\)R. diamond R \(\Leftrightarrow\) \(\forall\)x y z. R x y \(\land\) R x z \(\Rightarrow\) \(\exists\)u. R y u \(\land\) R z u
     : thm
\end{alltt}
\end{session}
    Now we clearly have that confluence of a relation is equivalent to
    the reflexive, transitive closure of that relation having the
    diamond property.
\begin{session}
\begin{verbatim}
val confluent_diamond_RTC = store_thm(
  "confluent_diamond_RTC",
  ``!R. confluent R = diamond (RTC R)``,
  RW_TAC std_ss [confluent_def, diamond_def]);
\end{verbatim}
\end{session}
    So far so good.  How then do we show the diamond property for
    $\con{RTC}\;R$?  The answer that leaps to mind is to hope that if
    the original relation has the diamond property, then maybe the
    reflexive and transitive closure will too.  The theorem we want is
    \[ \con{diamond}\;R \supset \con{diamond}\,(\con{RTC}\;R)\] Graphically,
    this is hoping that from
    \[\xymatrix @R=5mm @C=2.5mm {
& x \ar[dl] \ar[dr] & \\
y \ar@{.>}[dr] & & z \ar@{.>}[dl] \\
& u}\]
 we will be able to conclude\[\xymatrix @R=4mm @C=2mm {
& & & x \ar[dl] \ar[dr] & \\
& & y \ar@{.>}[dr] \ar@{-->}[ddll] & & z \ar@{.>}[dl] \ar@{-->}[ddrr] \\
& & & u \\
p \ar@{.>}[dddrrr] & & & & & & q \ar@{.>}[dddlll] \\ \\ \\
& & & r}\] where the dashed lines indicate that these steps (from $x$ to $p$,
for example) are using $\con{RTC}\;R$.  The presence of two instances
of $\con{RTC}\;R$ is an indication that this proof will require two
inductions.  With the first we will prove
\[\xymatrix @R=4mm @C=2mm {
& & & x \ar[dl] \ar[dr] & \\
& & y \ar@{.>}[dr] \ar@{-->}[ddll] & & z \ar@{.>}[dl] \\
& & & u \ar@{.>}[ddll] \\
p \ar@{.>}[dr] \\
& r}\]
In other words, we want to show that if we take one step in one
direction (to $z$) and many steps in another (to $p$), then the
diamond property for $R$ will guarantee us the existence of $r$,
to which will we be able to take many steps from both $p$ and $z$.

We take some care to state the goal so that after stripping away the
outermost assumption (that $R$ has the diamond property), it will match the
induction principle for \con{RTC}.\footnote{In this and subsequent
  proofs using the sub-goal package, we will present the proof manager
  as if the goal to be proved is the first ever on this stack.  In
  other words, we have done a \texttt{dropn 1;} after every successful
  proof to remove the evidence of the old goal.  In practice, there is
  no harm in leaving these goals on the proof manager's stack.}
\begin{session}
\begin{alltt}
- g `!R. diamond R ==>
         !x p. RTC R x p ==>
               !z. R x z ==>
                   ?u. RTC R p u /\bs RTC R z u`;
<<HOL message: inventing new type variable names: 'a>>
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         \(\forall\)R.
           diamond R \(\Rightarrow\)
           \(\forall\)x p. RTC R x p \(\Rightarrow\) \(\forall\)z. R x z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u
\end{alltt}
\end{session}
First, we strip away the diamond property assumption (two things need to
be stripped: the outermost universal quantifier and the antecedent of
the implication):
\begin{session}
\begin{alltt}
- e (GEN_TAC THEN STRIP_TAC);
OK..
1 subgoal:
> val it =
    \(\forall\)x p. RTC R x p \(\Rightarrow\) \(\forall\)z. R x z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u
    ------------------------------------
      diamond R
\end{alltt}
\end{session}
Now we can use the induction principle for reflexive and transitive closure (alternatively, we perform a ``rule induction'').
To do this, we use the \ml{Induct\_on} command that is also used to do
structural induction on algebraic data types (such as numbers and
lists).
We provide the name of the constant whose induction principle we want
to use, and the tactic does the rest:
\begin{session}
\begin{alltt}
- e (Induct_on `RTC`);
OK..
1 subgoal:
> val it =
    (\(\forall\)x z. R x z \(\Rightarrow\) \(\exists\)u. RTC R x u \(\land\) RTC R z u) \(\land\)
    \(\forall\)x x' p.
      R x x' \(\land\) RTC R x' p \(\land\) (\(\forall\)z. R x' z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u) \(\Rightarrow\)
      \(\forall\)z. R x z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u
    ------------------------------------
      diamond R
\end{alltt}
\end{session}
Let's strip the goal as much as possible with the aim of making what
remains to be proved easier to see:
\begin{session}
\begin{alltt}
- e (REPEAT STRIP_TAC);
OK..
2 subgoals:
> val it =
    \(\exists\)u. RTC R p u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  R x x'
      2.  RTC R x' p
      3.  \(\forall\)z. R x' z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u
      4.  R x z

    \(\exists\)u. RTC R x u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  R x z
\end{alltt}
\end{session}
This first goal is easy.  It corresponds to the case where the many
steps from $x$ to $p$ are actually no steps at all, and $p$ and $x$
are actually the same place.  In the other direction, $x$ has taken
one step to $z$, and we need to find somewhere reachable in zero or
more steps from both $x$ and $z$.  Given what we know so far, the only
candidate is $z$ itself.  In fact, we don't even need to provide this
witness explicitly. \texttt{PROVE\_TAC} will find it for us, as long
as we tell it what the rules governing \con{RTC} are:
\begin{session}
\begin{alltt}
- e (PROVE_TAC [RTC_rules]);
OK..
Meson search level: .....

Goal proved. [..] |- \(\exists\)u. RTC R p u \(\land\) RTC R z u
Remaining subgoals:
> val it =
    \(\exists\)u. RTC R p u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  R x x'
      2.  RTC R x' p
      3.  \(\forall\)z. R x' z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u
      4.  R x z
\end{alltt}
\end{session}
    And what of this remaining goal?  Assumptions one and four
    between them are the top of an $R$-diamond.  Let's use the fact
    that we have the diamond property for $R$ and infer that there
    exists a $v$ to which $y$ and $z'$ can both take single steps:
\begin{session}
\begin{alltt}
- e (`?v. R x' v /\bs R z v` by PROVE_TAC [diamond_def]);
OK..
Meson search level: ............
1 subgoal:
> val it =
    \(\exists\)u. RTC R p u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  R x x'
      2.  RTC R x' p
      3.  \(\forall\)z. R x' z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u
      4.  R x z
      5.  R x' v
      6.  R z v
\end{alltt}
\end{session}
Now we can apply our induction hypothesis (assumption 3) to complete
the long, lop-sided strip of the diamond.  We will conclude that there
is a $u$ such that $\con{RTC}\;R\;p\;u$ and $\con{RTC}\;R\;v\;u$.  We
actually need a $u$ such that $\con{RTC}\;R\;z\;u$, but because there
is a single $R$-step between $z$ and $v$ we have that as well.  All
we need to provide \texttt{PROVE\_TAC} is the rules for \con{RTC}:
\begin{session}
\begin{alltt}
- e (PROVE_TAC [RTC_rules]);
OK..
Meson search level: .......

Goal proved. [...]
> val it =
    Initial goal proved.
    |- \(\forall\)R.
         diamond R \(\Rightarrow\)
         \(\forall\)x p. RTC R x p \(\Rightarrow\) \(\forall\)z. R x z \(\Rightarrow\) \(\exists\)u. RTC R p u \(\land\) RTC R z u
\end{alltt}
\end{session}
    Again we can (and should) package up the lemma, avoiding the
    sub-goal package commands:
\begin{session}
\begin{verbatim}
val R_RTC_diamond = store_thm(
  "R_RTC_diamond",
  ``!R. diamond R ==>
         !x p. RTC R x p ==>
               !z. R x z ==>
                   ?u. RTC R p u /\ RTC R z u``,
  GEN_TAC THEN STRIP_TAC THEN Induct_on `RTC` THEN
  REPEAT STRIP_TAC THENL [
    PROVE_TAC [RTC_rules],
    `?v. R x' v /\ R z v` by PROVE_TAC [diamond_def] THEN
    PROVE_TAC [RTC_rules]
  ]);
\end{verbatim}
\end{session}
\eos{}

Now we can move on to proving that if $R$ has the diamond property, so
too does $\con{RTC}\;R$.  We want to prove this by induction again.
It's very tempting to state the goal as the obvious \[
\con{diamond}\;R\supset\con{diamond}\,(\con{RTC}\;R)
\] but doing so will actually make it harder to apply the induction
principle when the time is right.  Better to start out with a
statement of the goal that is very near in form to the induction
princple.  So, we manually expand the meaning of \con{diamond} and state
our next goal thus:
\begin{session}
\begin{alltt}
- g `!R. diamond R ==> !x y. RTC R x y ==>
                             !z. RTC R x z ==>
                                 ?u. RTC R y u /\bs RTC R z u`;
<<HOL message: inventing new type variable names: 'a>>
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         \(\forall\)R.
           diamond R \(\Rightarrow\)
           \(\forall\)x y. RTC R x y \(\Rightarrow\) \(\forall\)z. RTC R x z \(\Rightarrow\) \(\exists\)u. RTC R y u \(\land\) RTC R z u
\end{alltt}
\end{session}
    Again we strip the diamond property assumption, apply the
    induction principle, and strip repeatedly:
\begin{session}
\begin{alltt}
- e (GEN_TAC THEN STRIP_TAC THEN Induct_on `RTC` THEN REPEAT STRIP_TAC);
OK..
2 subgoals:
> val it =
    \(\exists\)u. RTC R y u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  R x x'
      2.  RTC R x' y
      3.  \(\forall\)z. RTC R x' z \(\Rightarrow\) \(\exists\)u. RTC R y u \(\land\) RTC R z u
      4.  RTC R x z

    \(\exists\)u. RTC R x u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  RTC R x z
\end{alltt}
\end{session}
The first goal is again an easy one, corresponding to the case where
the trip from $x$ to $y$ has been one of no steps whatsoever.
\begin{session}
\begin{alltt}
- e (PROVE_TAC [RTC_rules]);
OK..
Meson search level: ...

Goal proved. [...]

Remaining subgoals:
> val it =
    \(\exists\)u. RTC R y u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  R x x'
      2.  RTC R x' y
      3.  \(\forall\)z. RTC R x' z \(\Rightarrow\) \(\exists\)u. RTC R y u \(\land\) RTC R z u
      4.  RTC R x z
\end{alltt}
\end{session}
This goal is very similar to the one we saw earlier.  We have the top
of a (``lop-sided'') diamond in assumptions 1 and 4, so we can infer
the existence of a common destination for $x'$ and $z$:
\begin{session}
\begin{alltt}
- e (`?v. RTC R x' v /\bs RTC R z v` by PROVE_TAC [R_RTC_diamond]);
OK..
Meson search level: ............
1 subgoal:
> val it =
    \(\exists\)u. RTC R y u \(\land\) RTC R z u
    ------------------------------------
      0.  diamond R
      1.  R x x'
      2.  RTC R x' y
      3.  \(\forall\)z. RTC R x' z \(\Rightarrow\) \(\exists\)u. RTC R y u \(\land\) RTC R z u
      4.  RTC R x z
      5.  RTC R x' v
      6.  RTC R z v
\end{alltt}
\end{session}
    At this point in the last proof we were able to finish it all off
    by just appealing to the rules for \con{RTC}.  This time it is not
    quite so straightforward.  When we use the induction hypothesis
    (assumption 3), we can conclude that there is a $u$ to which both
    $y$ and $v$ can connect in zero or more steps, but in order to
    show that this $u$ is reachable from $z$, we need to be able to
    conclude $\con{RTC}\;R\;z\;u$ when we know that
    $\con{RTC}\;R\;z\;v$ (assumption 6 above) and
    $\con{RTC}\;R\;v\;u$ (our consequence of the inductive
    hypothesis).  We leave the proof of this general result as an
    exercise, and here assume that it is already proved as the theorem
    \texttt{RTC\_RTC}.
\begin{session}
\begin{alltt}
- e (PROVE_TAC [RTC_rules, RTC_RTC]);
Meson search level: .......

Goal proved. [...]
> val it =
    Initial goal proved.
    |- \(\forall\)R.
         diamond R \(\Rightarrow\)
         \(\forall\)x y. RTC R x y \(\Rightarrow\) \(\forall\)z. RTC R x z \(\Rightarrow\) \(\exists\)u. RTC R y u \(\land\) RTC R z u
\end{alltt}
\end{session}
We can package this result up as a lemma and then prove the prettier
version directly:
\begin{session}
\begin{verbatim}
val diamond_RTC_lemma = prove(
  ``!R.
       diamond R ==>
       !x y. RTC R x y ==> !z. RTC R x z ==> ?u. RTC R y u /\ RTC R z u``,
  GEN_TAC THEN STRIP_TAC THEN Induct_on `RTC` THEN
  REPEAT STRIP_TAC THENL [
    PROVE_TAC [RTC_rules],
    `?v. RTC R x' v /\ RTC R z v` by PROVE_TAC [R_RTC_diamond] THEN
    PROVE_TAC [RTC_RTC, RTC_rules]
  ]);
val diamond_RTC = store_thm(
  "diamond_RTC",
  ``!R. diamond R ==> diamond (RTC R)``,
  PROVE_TAC [diamond_def,diamond_RTC_lemma]);
\end{verbatim}
\end{session}

\section{Back to combinators}
\label{sec:Return-to-Land}

Now, we are in a position to return to the real object of study and
prove confluence for combinatory logic.  We have done an abstract
development and established that\[
\begin{array}{ccccc}
\con{diamond}\;R & \supset & \con{diamond}\,(\con{RTC}\;R)\\
& & \land\\
& & \con{diamond}\,(\con{RTC}\;R) & \equiv & \con{confluent}\;R\\
\end{array}
\]  (We have also established a couple of other useful results along
the way.)

\newcommand{\topk}{\KC\;\SC\;(\KC\;\KC\;\KC)} Sadly, it just isn't the
case that $\rightarrow$, our one-step relation for combinators, has
the diamond property.  A counter-example is $\topk$.  Its possible evolution
can be described graphically: \[\xymatrix @R=5mm @C=2.5mm {
& \topk \ar[dl] \ar[dr] & \\
\SC & & \KC\;\SC\;\KC \ar[dl] \\
& \SC}\]
If we had the diamond property, it should be possible to find a common
destination for $\KC\;\SC\;\KC$ and $\SC$.  However, \SC{} doesn't
admit any reductions whatsoever, so there isn't a common
destination.\footnote{In fact our counter-example is more complicated
  than necessary.  The fact that $\KC\;\SC\;\KC$ has a
  reduction to the normal form $\SC$ also acts as a counter-example.
  Can you see why?}

This is a problem.  We are going to have to take another approach.
We will define another reduction strategy (\emph{parallel reduction}),
and prove that its reflexive, transitive closure is actually the same
relation as our original's reflexive and transitive closure.  Then we
will also show that parallel reduction has the diamond property.  This
will establish that its reflexive, transitive closure has it too.
Then, because they are the same relation, we will have that the
reflexive, transitive closure of our original relation has the diamond
property, and therefore, our original relation will be confluent.

\subsection{Parallel reduction}
\label{sec:Parallel-Reduction}

Our new relation allows for any number of reductions to occur in
parallel.  We use the \texttt{-||->} symbol to indicate parallel
reduction because of its own parallel lines:
\begin{session}
\begin{verbatim}
- set_fixity "-||->" (Infix(NONASSOC, 450));
> val it = () : unit
\end{verbatim}
\end{session}
    Then we can define parallel reduction itself.  The rules look very
    similar to those for $\rightarrow$.  The difference is that we
    allow the reflexive transition, and say that an application of
    $x\;u$ can be transformed to $y\;v$ if there are transformations
    taking $x$ to $y$ and $u$ to $v$.  This is why we must have
    reflexivity incidentally.  Without it, a term like
    $(\KC\;x\;y)\,\KC$ couldn't reduce because while the LHS of the
    application ($\KC\;x\;y$) can reduce, its RHS (\KC) can't.
\begin{session}
\begin{alltt}
- val (predn_rules, _, predn_cases) = xHol_reln "predn"
      `(!x. x -||-> x) /\bs
       (!x y u v. x -||-> y /\bs u -||-> v
                         ==>
                  x # u -||-> y # v) /\bs
       (!x y. K # x # y -||-> x) /\bs
       (!f g x. S # f # g # x -||-> (f # x) # (g # x))`;
> val predn_rules =
    |- (\(\forall\)x. x -||-> x) \(\land\)
       (\(\forall\)x y u v. x -||-> y \(\land\) u -||-> v \(\Rightarrow\) x # u -||-> y # v) \(\land\)
       (\(\forall\)x y. K # x # y -||-> x) \(\land\)
       \(\forall\)f g x. S # f # g # x -||-> f # x # (g # x) : thm
  val predn_cases =
    |- \(\forall\)a0 a1.
         a0 -||-> a1 \(\Leftrightarrow\)
         (a1 = a0) \(\lor\)
         (\(\exists\)x y u v. (a0 = x # u) \(\land\) (a1 = y # v) \(\land\)
                    x -||-> y \(\land\) u -||-> v) \(\lor\)
         (\(\exists\)y. a0 = K # a1 # y) \(\lor\)
         \(\exists\)f g x. (a0 = S # f # g # x) \(\land\) (a1 = f # x # (g # x))
    : thm
\end{alltt}
\end{session}

\subsection{Using \con{RTC}}
\label{sec:Using-RTC}

Now we can set up nice syntax for the reflexive and transitive
closures of our two relations.  We will use ASCII symbols for both
that consist of the original symbol followed by an asterisk.  Note
also how, in defining the two relations, we have to use the
\texttt{\$} character to ``escape'' the symbols' usual fixities.  This
is exactly analogous to the way in which ML's \texttt{op} keyword is
used.  First, we create the desired symbol for the concrete syntax,
and then we ``overload'' it so that the parser will expand it to the
desired form.
\begin{session}
\begin{verbatim}
- set_fixity "-->*" (Infix(NONASSOC, 450));
> val it = () : unit

- overload_on ("-->*", ``RTC $-->``);
> val it = () : unit
\end{verbatim}
\end{session}
We do exactly the same thing for the reflexive and transitive closure
of our parallel reduction.
\begin{session}
\begin{verbatim}
- set_fixity "-||->*" (Infix(NONASSOC, 450));
> val it = () : unit

- overload_on ("-||->*", ``RTC $-||->``);
> val it = () : unit
\end{verbatim}
\end{session}
Incidentally, in conjunction with \texttt{PROVE} we can now
automatically demonstrate relatively long chains of reductions:
\begin{session}
\begin{verbatim}
- PROVE [RTC_rules, redn_rules] ``S # K # K # x -->* x``;
Meson search level: ......
> val it = |- S # K # K # x -->* x : thm

- PROVE [RTC_rules, redn_rules]
    ``S # (S # (K # S) # K) # (S # K # K) # f # x -->*
      f # (f # x)``;
Meson search level: ...........................
> val it = |- S # (S # (K # S) # K) # (S # K # K) # f # x -->* f # (f # x)
           : thm
\end{verbatim}
\end{session}
(The latter sequence is seven reductions long.)

\subsection{Proving the \con{RTC}s are the same}
\label{sec:Proving-RTCs-same}

We start with the easier direction, and show that everything in
$\rightarrow^*$ is in $\mathpredn^*$.  Because
\con{RTC} is monotone (which fact is left to the reader to prove),
we can reduce this to showing that $x\rightarrow y\supset x\mathpredn y$.

Our goal:
\begin{session}
\begin{alltt}
- g `!x y. x -->* y ==> x -||->* y`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         \(\forall\)x y. x -->* y \(\Rightarrow\) x -||->* y
\end{alltt}
\end{session}
We back-chain using our monotonicity result:
\begin{session}
\begin{alltt}
- e (MATCH_MP_TAC RTC_monotone);
OK..
1 subgoal:
> val it =
    \(\forall\)x y. x --> y \(\Rightarrow\) x -||-> y
\end{alltt}
\end{session}
Now we can induct over the rules for $\rightarrow$:
\begin{session}
\begin{alltt}
- e (Induct_on `$-->`);
OK..
1 subgoal:
> val it =
    (\(\forall\)x y f. x --> y \(\land\) x -||-> y \(\Rightarrow\) f # x -||-> f # y) \(\land\)
    (\(\forall\)f g x. f --> g \(\land\) f -||-> g \(\Rightarrow\) f # x -||-> g # x) \(\land\)
    (\(\forall\)x y. K # x # y -||-> x) \(\land\)
    \(\forall\)f g x. S # f # g # x -||-> f # x # (g # x)
\end{alltt}
\end{session}
We could split the 4-way conjunction apart into four goals, but there
is no real need.  It is quite clear that each follows immediately from
the rules for parallel reduction.
\begin{session}
\begin{alltt}
- e (PROVE_TAC [predn_rules]);
OK..
Meson search level: ............

Goal proved. [...]
> val it =
    Initial goal proved.
    |- \(\forall\)x y. x -->* y \(\Rightarrow\) x -||->* y : goalstack
\end{alltt}
\end{session}
Packaged into a tidy little sub-goal-package-free parcel, our proof is
\begin{session}
\begin{verbatim}
val RTCredn_RTCpredn = store_thm(
  "RTCredn_RTCpredn",
  ``!x y. x -->* y   ==>   x -||->* y``,
  MATCH_MP_TAC RTC_monotone THEN
  Induct_on `$-->` THEN PROVE_TAC [predn_rules]);
\end{verbatim}
\end{session}
\eos{}

Our next proof is in the other direction.  It should be clear that we
will not just be able to appeal to the monotonicity of \con{RTC} this
time; one step of the parallel reduction relation can not be mirrored
with one step of the original reduction relation.  It's clear that
mirroring one step of the parallel reduction relation might take many
steps of the original relation.  Let's prove that then:
\begin{session}
\begin{alltt}
- g `!x y. x -||-> y   ==>   x -->* y`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         \(\forall\)x y. x -||-> y \(\Rightarrow\) x -->* y
\end{alltt}
\end{session}
This time our induction will be over the rules defining the parallel
reduction relation.
\begin{session}
\begin{alltt}
- e (Induct_on `$-||->`);
OK..
1 subgoal:
> val it =
    (\(\forall\)x. x -->* x) \(\land\)
    (\(\forall\)x y x' y'. x -||-> y \(\land\) x -->* y \(\land\) x' -||-> y' \(\land\) x' -->* y' \(\Rightarrow\)
                 x # x' -->* y # y') \(\land\)
    (\(\forall\)x y. K # x # y -->* x) \(\land\)
    \(\forall\)f g x. S # f # g # x -->* f # x # (g # x)
\end{alltt}
\end{session}
    There are four conjuncts here, and it should be clear that all but
    the second can be proved immediately by appeal to the rules for
    the transitive closure and for $\rightarrow$ itself.  We could
    split apart the conjunctions and enter a \texttt{THENL} branch.
    However, we'd need to repeat the same tactic three times to
    quickly close three of the four branches.  Instead, we use the
    \texttt{TRY} tactical to try applying the same tactic to all four
    branches.  If our tactic fails on branch \#2, as we expect,
    \texttt{TRY} will protect us against this failure and let us
    proceed.
\begin{session}
\begin{alltt}
e (REPEAT CONJ_TAC THEN
   TRY (PROVE_TAC [RTC_rules, redn_rules]));
OK..
Meson search level: ....
Meson search level: ....
Meson search level: ...............................
Meson search level: ..
1 subgoal:
> val it =
    \(\forall\)x y x' y'. x -||-> y \(\land\) x -->* y \(\land\) x' -||-> y' \(\land\) x' -->* y' \(\Rightarrow\)
                x # x' -->* y # y'
\end{alltt}
\end{session}
    Note that wrapping \texttt{TRY} around \texttt{PROVE\_TAC} is not
    always wise.  It can often take the \texttt{PROVE\_TAC} tactic an extremely
    long time to exhaust its search space, and then give up with a
    failure.  Here, ``we got lucky''.

    Anyway, what of this latest sub-goal?  If we look at it for long
    enough, we should see that it is another monotonicity fact.  More
    accurately, we need what is called a \emph{congruence} result for
    \holtxt{-->*}.  In this form, it's not quite right for easy proof.
    Let's go away and prove \texttt{RTCredn\_ap\_monotonic}
    separately. (Another exercise!)  Our new theorem should state
\begin{session}
\begin{verbatim}
val RTCredn_ap_congruence = store_thm(
  "RTCredn_ap_congruence",
  ``!x y. x -->* y ==> !z. x # z -->* y # z /\ z # x -->* z # y``,
  ...);
\end{verbatim}
\end{session}
    Now that we have this, our sub-goal is almost immediately
    provable.  Using it, we know that \[\begin{array}{c}
      x\;x' \rightarrow^* y\;x' \\
      y\;x' \rightarrow^* y\;y'
    \end{array}\]
    All we need to do is ``stitch together'' the two transitions above
    and go from $x\;x'$ to $y\;y'$.  We can do this by appealing to our
    earlier \texttt{RTC\_RTC} result.
\begin{session}
\begin{alltt}
e (PROVE_TAC [RTC_RTC, RTCredn_ap_congruence]);
OK..
Meson search level: .......

Goal proved. [...]
> val it =
    Initial goal proved.
    |- \(\forall\)x y. x -||-> y \(\Rightarrow\) x -->* y : goalstack
\end{alltt}
\end{session}
But given that we can finish off what we thought was an awkward branch
with just another application of \texttt{PROVE\_TAC}, we don't need to
use our fancy \texttt{TRY}-footwork at the stage before.  Instead, we
can just merge the theorem lists passed to both invocations, dispense
with the \texttt{REPEAT CONJ\_TAC} and have a very short tactic proof
indeed:
\begin{session}
\begin{verbatim}
val predn_RTCredn = store_thm(
  "predn_RTCredn",
  ``!x y. x -||-> y  ==>  x -->* y``,
  Induct_on `$-||->` THEN
  PROVE_TAC [RTC_rules, redn_rules, RTC_RTC, RTCredn_ap_congruence]);
\end{verbatim}
\end{session}
\eos{}

Now it's time to prove that if a number of parallel reduction steps
are chained together, then we can mirror this with some number of
steps using the original reduction relation.  Our goal:
\begin{session}
\begin{alltt}
- g `!x y. x -||->* y  ==> x -->* y`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         \(\forall\)x y. x -||->* y \(\Rightarrow\) x -->* y
\end{alltt}
\end{session}
We use the appropriate induction principle to get to:
\begin{session}
\begin{alltt}
- e (Induct_on `RTC`);
OK..
1 subgoal:
> val it =
    (\(\forall\)x. x -->* x) \(\land\)
    \(\forall\)x x' y. x -||-> x' \(\land\) x' -||-> y* \(\land\) x' -->* y \(\Rightarrow\) x -->* z
\end{alltt}
\end{session}
This we can finish off in one step.  The first conjunct is obvious,
and in the second the \verb!x -||-> y! and our last result combine to
tell us that \verb!x -->* y!.  Then this can be chained together with
the other assumption in the second conjunct and we're done.
\begin{session}
\begin{alltt}
- e (PROVE_TAC [RTC_rules, predn_RTCredn, RTC_RTC]);
OK..
Meson search level: .......

Goal proved.[...]
> val it =
    Initial goal proved.
    |- \(\forall\)x y. x -||->* y \(\Rightarrow\) x -->* y : proof
\end{alltt}
\end{session}
Packaged up, this proof is:
\begin{session}
\begin{verbatim}
val RTCpredn_RTCredn = store_thm(
  "RTCpredn_RTCredn",
  ``!x y. x -||->* y   ==>  x -->* y``,
  Induct_on `RTC` THEN PROVE_TAC [predn_RTCredn, RTC_RTC, RTC_rules]);
\end{verbatim}
\end{session}
\eos{}

Our final act is to use what we have so far to conclude that
$\rightarrow^*$ and $\mathpredn^*$ are equal.  We state our goal:
\begin{session}
\begin{verbatim}
- g `$-||->* = $-->*`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         $-||->* = $-->*
\end{verbatim}
\end{session}
We want to now appeal to extensionality.  The simplest way to do this
is to rewrite with the theorem \texttt{FUN\_EQ\_THM}:
\begin{session}
\begin{alltt}
- FUN_EQ_THM;
> val it = |- \(\forall\)f g. (f = g) \(\Leftrightarrow\) \(\forall\)x. f x = g x : thm
\end{alltt}
\end{session}
So, we rewrite:
\begin{session}
\begin{alltt}
- e (SIMP_TAC std_ss [FUN_EQ_THM]);
OK..
1 subgoal:
> val it =
    \(\forall\)x x'. x -||->* x' = x -->* x'
\end{alltt}
\end{session}

This goal is an easy consequence of our two earlier implications.
\begin{session}
\begin{verbatim}
- e (PROVE_TAC [RTCpredn_RTCredn, RTCredn_RTCpredn]);
OK..
Meson search level: ......

Goal proved. [...]
> val it =
    Initial goal proved.
    |- $-||->* = $-->* : goalstack
\end{verbatim}
\end{session}
Packaged, the proof is:
\begin{session}
\begin{verbatim}
val RTCpredn_EQ_RTCredn = store_thm(
  "RTCpredn_EQ_RTCredn",
  ``$-||->* = $-->*``,
  SIMP_TAC std_ss [FUN_EQ_THM] THEN
  PROVE_TAC [RTCpredn_RTCredn, RTCredn_RTCpredn]);
\end{verbatim}
\end{session}

\subsection{Proving a diamond property for parallel reduction}
\label{sec:predn-diamond}

Now we just have one substantial proof to go.  Before we can even
begin, there are a number of minor lemmas we will need to prove first.
These are basically specialisations of the theorem
\texttt{predn\_cases}.  We want exhaustive characterisations of the
possibilities when the following terms undergo a parallel reduction:
$x\;y$, \KC, \SC, $\KC\;x$, $\SC\;x$, $\KC\;x\;y$, $\SC\;x\;y$ and
$\SC\;x\;y\;z$.

To do this, we will write a little function that derives
characterisations automatically:
\begin{session}
\begin{verbatim}
- fun characterise t = SIMP_RULE (srw_ss()) [] (SPEC t predn_cases);
> val characterise = fn : term -> thm
\end{verbatim}
\end{session}
The \ml{characterise} function specialises the theorem
\ml{predn\_cases} with the input term, and then simplifies.  The
\ml{srw\_ss()} simpset includes information about the injectivity and
disjointness of constructors and eliminates obvious impossibilities.
For example,
\begin{session}
\begin{alltt}
- val K_predn = characterise ``K``;
<<HOL message: more than one resolution of overloading was possible>>
> val K_predn = |- \(\forall\)a1. K -||-> a1 = (a1 = K) : thm

- val S_predn = characterise ``S``;
<<HOL message: more than one resolution of overloading was possible>>
> val S_predn = |- \(\forall\)a1. S -||-> a1 = (a1 = S) : thm
\end{alltt}
\end{session}
Unfortunately, what we get back from other inputs is not so good:
\begin{session}
\begin{alltt}
- val Sx_predn0 = characterise ``S # x``;
> val Sx_predn0 =
    |- \(\forall\)a1.
         S # x -||-> a1 =
         (a1 = S # x) \(\lor\)
         \(\exists\)y v. (a1 = y # v) \(\land\) S -||-> y \(\land\) x -||-> v : thm
\end{alltt}
\end{session}
That first disjunct is redundant, as the following demonstrates:
\begin{session}
\begin{verbatim}
val Sx_predn = prove(
  ``!x y. S # x -||-> y = ?z. (y = S # z) /\ (x -||-> z)``,
  REPEAT GEN_TAC THEN EQ_TAC THEN
  RW_TAC std_ss [Sx_predn0, predn_rules, S_predn]);
\end{verbatim}
\end{session}
Our \texttt{characterise} function will just have to help us in the
proofs that follow.
\begin{session}
\begin{verbatim}
val Kx_predn = prove(
  ``!x y. K # x -||-> y = ?z. (y = K # z) /\ (x -||-> z)``,
  REPEAT GEN_TAC THEN EQ_TAC THEN
  RW_TAC std_ss [characterise ``K # x``, predn_rules, K_predn]);
\end{verbatim}
\end{session}
What of $\KC\;x\;y$?  A little thought demonstrates that there really
must be two cases this time.
\begin{session}
\begin{verbatim}
val Kxy_predn = prove(
  ``!x y z.
       K # x # y -||-> z =
       (?u v. (z = K # u # v) /\ (x -||-> u) /\ (y -||-> v)) \/
       (z = x)``,
  REPEAT GEN_TAC THEN EQ_TAC THEN
  RW_TAC std_ss [characterise ``K # x # y``, predn_rules,
                 Kx_predn]);
\end{verbatim}
\end{session}
By way of contrast, there is only one case for $\SC\;x\;y$ because it
is not yet a ``redex'' at the top-level.
\begin{session}
\begin{verbatim}
val Sxy_predn = prove(
  ``!x y z. S # x # y -||-> z =
            ?u v. (z = S # u # v) /\ (x -||-> u) /\ (y -||-> v)``,
  REPEAT GEN_TAC THEN EQ_TAC THEN
  RW_TAC std_ss [characterise ``S # x # y``, predn_rules,
                 Sx_predn]);
\end{verbatim}
\end{session}
Next, the characterisation for $\SC\;x\;y\;z$:
\begin{session}
\begin{verbatim}
val Sxyz_predn = prove(
  ``!w x y z. S # w # x # y -||-> z =
              (?p q r. (z = S # p # q # r) /\
                       w -||-> p /\ x -||-> q /\ y -||-> r) \/
              (z = (w # y) # (x # y))``,
  REPEAT GEN_TAC THEN EQ_TAC THEN
  RW_TAC std_ss [characterise ``S # w # x # y``, predn_rules,
                 Sxy_predn]);
\end{verbatim}
\end{session}
Last of all, we want a characterisation for $x\;y$.   What
\texttt{characterise} gives us this time can't be improved upon,
for all that we might look upon the four disjunctions and despair.
\begin{session}
\begin{alltt}
- val x_ap_y_predn = characterise ``x # y``;
> val x_ap_y_predn =
    |- \(\forall\)a1.
         x # y -||-> a1 =
         (a1 = x # y) \(\lor\)
         (\(\exists\)y' v. (a1 = y' # v) \(\land\) x -||-> y' \(\land\) y -||-> v) \(\lor\)
         (x = K # a1) \(\lor\)
         \(\exists\)f g. (x = S # f # g) \(\land\) (a1 = f # y # (g # y)) : thm
\end{alltt}
\end{session}
\eos{}

\noindent Now we are ready to prove the final goal.  It is
\begin{session}
\begin{alltt}
- g `!x y. x -||-> y ==>
           !z. x -||-> z ==> ?u. y -||-> u /\bs z -||-> u`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         \(\forall\)x y. x -||-> y \(\Rightarrow\) \(\forall\)z. x -||-> z \(\Rightarrow\) \(\exists\)u. y -||-> u \(\land\) z -||-> u
\end{alltt}
\end{session}
We now induct and split the goal into its individual conjuncts:
\begin{session}
\begin{alltt}
- e (Induct_on `$-||->` THEN REPEAT CONJ_TAC);
OK..
4 subgoals:
> val it =
    \(\forall\)f g x z. S # f # g # x -||-> z \(\Rightarrow\)
              \(\exists\)u. f # x # (g # x) -||-> u \(\land\) z -||-> u

    \(\forall\)x y z. K # x # y -||-> z \(\Rightarrow\) \(\exists\)u. x -||-> u \(\land\) z -||-> u

    \(\forall\)x y u v.
      x -||-> y \(\land\)
      (\(\forall\)z. x -||-> z \(\Rightarrow\) \(\exists\)u. y -||-> u \(\land\) z -||-> u) \(\land\)
      u -||-> v \(\land\)
      (\(\forall\)z. u -||-> z \(\Rightarrow\) \(\exists\)u. v -||-> u \(\land\) z -||-> u) \(\Rightarrow\)
      \(\forall\)z. x # u -||-> z \(\Rightarrow\) \(\exists\)u. y # v -||-> u \(\land\) z -||-> u

    \(\forall\)x z. x -||-> z \(\Rightarrow\) \(\exists\)u. x -||-> u \(\land\) z -||-> u
\end{alltt}
\end{session}
The first goal is easily disposed of.  The witness we would provide
for this case is simply \texttt{z}, but \texttt{PROVE\_TAC} will do
the work for us:
\begin{session}
\begin{verbatim}
- e (PROVE_TAC [predn_rules]);
OK..
Meson search level: ...

Goal proved. [...]
\end{verbatim}
\end{session}
    The next goal includes two instances of terms of the form
    \verb!x # y -||-> z!.  We can use our \verb!x_ap_y_predn!
    theorem here.  However, if we rewrite indiscriminately with it, we
    will really confuse the goal.  We want to rewrite just the
    assumption, not the instance underneath the existential
    quantifier.  Starting everything by repeatedly stripping can't
    lead us too far astray.
\begin{session}
\begin{alltt}
- e (REPEAT STRIP_TAC);
OK..
1 subgoal:
> val it =
    \(\exists\)u. y # v -||-> u \(\land\) z -||-> u
    ------------------------------------
      0.  x -||-> y
      1.  \(\forall\)z. x -||-> z \(\Rightarrow\) \(\exists\)u. y -||-> u \(\land\) z -||-> u
      2.  u -||-> v
      3.  \(\forall\)z. u -||-> z \(\Rightarrow\) \(\exists\)u. v -||-> u \(\land\) z -||-> u
      4.  x # u -||-> z
\end{alltt}
\end{session}
We need to split up assumption 4.  We can get it out of the assumption
list using the \texttt{Q.PAT\_ASSUM} theorem-tactical.  We will write
\begin{verbatim}
    Q.PAT_ASSUM `x # y -||-> z`
      (STRIP_ASSUME_TAC o SIMP_RULE std_ss [x_ap_y_predn])
\end{verbatim}
The quotation specifies the pattern that we want to match.  The second
argument specifies how we are going to transform the theorem.  Reading
the compositions from right to left, first we will simplify with the
\verb!x_ap_y_predn! theorem and then we will assume the result back
into the assumptions, stripping disjunctions and existentials as we
go.\footnote{An alternative to using \texttt{PAT\_ASSUM} is to use
  \texttt{by} instead: you would have to state the four-way
  disjunction yourself, but the proof would be more ``declarative'' in
  style, and though wordier, might be more maintainable.}

We already know that doing this is going to produce four new sub-goals
(there were four disjuncts in the \verb!x_ap_y_predn! theorem).  At
least one of these should be trivial because it will correspond to the
case when the parallel reduction is just a ``do nothing'' step.  Let's
try eliminating the simple cases with a ``speculative'' call to
\texttt{PROVE\_TAC} wrapped inside a \texttt{TRY}.  And before doing
that, we should do some rewriting to make sure that equalities in the
assumptions are eliminated.

So:
\begin{session}
\begin{alltt}
- e (Q.PAT_ASSUM `x # y -||-> z`
      (STRIP_ASSUME_TAC o SIMP_RULE std_ss [x_ap_y_predn]) THEN
     RW_TAC std_ss [] THEN
     TRY (PROVE_TAC [predn_rules]));
OK..
Meson search level: ...............................
Meson search level: ...............................
Meson search level: ..................
Meson search level: .....
2 subgoals:
> val it =
    \(\exists\)u'. y # v -||-> u' \(\land\) f # u # (g # u) -||-> u'
    ------------------------------------
      0.  S # f # g -||-> y
      1.  \(\forall\)z. S # f # g -||-> z \(\Rightarrow\) \(\exists\)u. y -||-> u \(\land\) z -||-> u
      2.  u -||-> v
      3.  \(\forall\)z. u -||-> z \(\Rightarrow\) \(\exists\)u. v -||-> u \(\land\) z -||-> u

    \(\exists\)u. y # v -||-> u \(\land\) z -||-> u
    ------------------------------------
      0.  K # z -||-> y
      1.  \(\forall\)z'. K # z -||-> z' \(\Rightarrow\) \(\exists\)u. y -||-> u \(\land\) z' -||-> u
      2.  u -||-> v
      3.  \(\forall\)z. u -||-> z \(\Rightarrow\) \(\exists\)u. v -||-> u \(\land\) z -||-> u
\end{alltt}
\end{session}
Brilliant!  We've eliminated two of the four disjuncts already.  Now
our next goal features a term \verb!K # z -||-> y! in the assumptions.
We have a theorem that pertains to just this situation.  But before
applying it willy-nilly, let us try to figure out exactly what the
situation is.  A diagram of the current situation might look like
\[\xymatrix @C=2cm {
*+<4pt>[F--]\txt{\fbox{\texttt{K\kern2mm \#\kern2mm z}} \texttt{\#} \fbox{\texttt{u}}} \ar[r] & \txt{\texttt{z}} \ar@{.>}[d] \\
*+<4pt>[F--]\txt{\fbox{\texttt{y}} \texttt{\#} \fbox{\texttt{v}}} \ar@{.>}[r] & \txt{\texttt{?u?}}
\ar "1,1"!<-12pt,0pt>;"2,1"!<-12pt,0pt>
\ar "1,1"!<28pt,0pt>;"2,1"!<11pt,0pt>
} \]
Our theorem tells us that \texttt{y} must actually be of the form
\verb!K # w! for some \texttt{w}, and that there must be an arrow
between \texttt{z} and \texttt{w}.  Thus:
\begin{session}
\begin{alltt}
- e (`?w. (y = K # w) /\bs (z -||-> w)` by PROVE_TAC [Kx_predn]);
OK..
Meson search level: ......
1 subgoal:
> val it =
    \(\exists\)u. y # v -||-> u \(\land\) z -||-> u
    ------------------------------------
      0.  K # z -||-> y
      1.  \(\forall\)z'. K # z -||-> z' \(\Rightarrow\) \(\exists\)u. y -||-> u \(\land\) z' -||-> u
      2.  u -||-> v
      3.  \(\forall\)z. u -||-> z \(\Rightarrow\) \(\exists\)u. v -||-> u \(\land\) z -||-> u
      4.  y = K # w
      5.  z -||-> w
\end{alltt}
\end{session}
    On inspection, it becomes clear that the \texttt{u} must be
    \texttt{w}.  The first conjunct requires \verb!K # w # v -||-> w!,
    which we have because this is what \KC{}s do, and the second
    conjunct is already in the assumption list.  Rewriting
    (eliminating that equality in the assumption list first will make
    \texttt{PROVE\_TAC}'s job that much easier), and then first order
    reasoning will solve this goal:
\begin{session}
\begin{alltt}
- e (RW_TAC std_ss [] THEN PROVE_TAC [predn_rules]);
OK..
Meson search level: ...

Goal proved. [...]
Remaining subgoals:
> val it =
    \(\exists\)u'. y # v -||-> u' \(\land\) f # u # (g # u) -||-> u'
    ------------------------------------
      0.  S # f # g -||-> y
      1.  \(\forall\)z. S # f # g -||-> z \(\Rightarrow\) \(\exists\)u. y -||-> u \(\land\) z -||-> u
      2.  u -||-> v
      3.  \(\forall\)z. u -||-> z \(\Rightarrow\) \(\exists\)u. v -||-> u \(\land\) z -||-> u
\end{alltt}
\end{session}
This case involving \SC{} is analogous.  Here's the tactic to apply:
\begin{session}
\begin{alltt}
- e (`?p q. (y = S # p # q) /\bs (f -||-> p) /\bs (g -||-> q)`
        by PROVE_TAC [Sxy_predn] THEN
     RW_TAC std_ss [] THEN PROVE_TAC [predn_rules]);
OK..
Meson search level: ........
Meson search level: ...........

Goal proved.[...]
Remaining subgoals:
> val it =
    \(\forall\)f g x z. S # f # g # x -||-> z \(\Rightarrow\)
              \(\exists\)u. f # x # (g # x) -||-> u \(\land\) z -||-> u

    \(\forall\)x y z. K # x # y -||-> z \(\Rightarrow\) \(\exists\)u. x -||-> u \(\land\) z -||-> u
\end{alltt}
\end{session}
This next goal features a \verb!K # x # y -||-> z! term that we have a
theorem for already.  And again, let's speculatively use a call to
\texttt{PROVE\_TAC} to eliminate the simple cases immediately
(\verb!Kxy_predn! is a disjunct so we'll get two sub-goals if we don't
eliminate anything).
\begin{session}
\begin{alltt}
- e (RW_TAC std_ss [Kxy_predn] THEN
     TRY (PROVE_TAC [predn_rules]));
OK..
Meson search level: ..
Meson search level: ...

Goal proved. [...]
Remaining subgoals:
> val it =
    \(\forall\)f g x z. S # f # g # x -||-> z \(\Rightarrow\)
              \(\exists\)u. f # x # (g # x) -||-> u \(\land\) z -||-> u
\end{alltt}
\end{session}
Better yet! We got both cases immediately, and have moved onto the
last case.  We can try the same strategy.
\begin{session}
\begin{alltt}
- e (RW_TAC std_ss [Sxyz_predn] THEN PROVE_TAC [predn_rules]);
OK..
Meson search level: ..
Meson search level: ...........

Goal proved.[...]
> val it =
    Initial goal proved.
    |- \(\forall\)x y. x -||-> y \(\Rightarrow\) \(\forall\)z. x -||-> z \(\Rightarrow\)
             \(\exists\)u. y -||-> u \(\land\) z -||-> u : goalstack
\end{alltt}
\end{session}
The final goal proof can be packaged into:
\begin{session}
\begin{verbatim}
val predn_diamond_lemma = prove(
  ``!x y. x -||-> y ==>
          !z. x -||-> z ==> ?u. y -||-> u /\ z -||-> u``,
  Induct_on `$-||->` THEN REPEAT CONJ_TAC THENL [
    PROVE_TAC [predn_rules],
    REPEAT STRIP_TAC THEN
    Q.PAT_ASSUM `x # y -||-> z`
      (STRIP_ASSUME_TAC o SIMP_RULE std_ss [x_ap_y_predn]) THEN
    RW_TAC std_ss [] THEN
    TRY (PROVE_TAC [predn_rules]) THENL [
      `?w. (y = K # w) /\ (z -||-> w)` by PROVE_TAC [Kx_predn] THEN
      RW_TAC std_ss [] THEN PROVE_TAC [predn_rules],
      `?p q. (y = S # p # q) /\ (f -||-> p) /\ (g -||-> q)` by
         PROVE_TAC [Sxy_predn] THEN
      RW_TAC std_ss [] THEN PROVE_TAC [predn_rules]
    ],
    RW_TAC std_ss [Kxy_predn] THEN PROVE_TAC [predn_rules],
    RW_TAC std_ss [Sxyz_predn] THEN PROVE_TAC [predn_rules]
  ]);
\end{verbatim}
\end{session}
\eos{}

We are on the home straight.  The lemma can be turned into a statement
involving the \con{diamond} constant directly:
\begin{session}
\begin{verbatim}
val predn_diamond = store_thm(
  "predn_diamond",
  ``diamond $-||->``,
  PROVE_TAC [diamond_def, predn_diamond_lemma]);
\end{verbatim}
\end{session}

And now we can prove that our original relation is confluent in
similar fashion:

\begin{session}
\begin{verbatim}
val confluent_redn = store_thm(
  "confluent_redn",
  ``confluent $-->``,
  PROVE_TAC [predn_diamond, confluent_diamond_RTC,
             RTCpredn_EQ_RTCredn, diamond_RTC]);
\end{verbatim}
\end{session}

\section{Exercises}

If necessary, answers to the first three exercises can be found by
examining the source file in \texttt{examples/ind\_def/clScript.sml}.

\begin{enumerate}
\item Prove that \[\con{RTC}\;R \;x\; y \;\;\land \;\;
  \con{RTC}\;R\;y\;z\;\;\;\supset\;\;\; \con{RTC}\;R\;x\;z
\] You will need to prove the goal by induction, and will probably
  need to massage it slightly first to get it to match the appropriate
  induction principle.  Store the theorem under the name
  \texttt{RTC\_RTC}.
\item Another induction.  Show that \[
  (\forall x\,y.\; R_1\;x\;y\supset R_2\;x\;y) \supset
  (\forall x\,y.\; \con{RTC}\;R_1\;x\;y \supset \con{RTC}\;R_2\;x\;y)
\] Call the resulting theorem \texttt{RTC\_monotone}.
\item Yet another \con{RTC} induction, but where $R$ is no longer
  abstract, and is instead the original reduction relation.  Prove
\[
x \rightarrow^* y \;\;\;\supset\;\;\;
\forall z.\;\; x\;z \rightarrow^* y \;z \land
z\;x \rightarrow^* z\;y
\] Call it \texttt{RTCredn\_ap\_congruence}.

\item Come up with a counter-example for the following property: \[
\begin{array}{c}
  \left(
    \begin{array}{ll}
      \forall x\,y\,z. &
      R\;x\;y\;\;\land\;\; R\;x\;z \;\;\;\supset\\
      & \exists u.\;\con{RTC}\;R\;y\;u \;\;\land\;\;\con{RTC}\;R\;z\;u
      \end{array}\right)\\
    \supset\\
    \con{diamond}\;(\con{RTC}\;R)
  \end{array}
  \]
\end{enumerate}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "tutorial"
%%% End:


\subsection{Algebraic proof (using Spectral Theorem)}

The proof in this section relies on the Spectral Theorem, which says
that we can diagnonalize a symmetric matrix. Instead of jumping right
away to the proof of SVD with such heavy machinery, we prefer a
gradual approach consisting of the following steps: \\

\begin{itemize}
\item Emphasize that the main task of SVD factorization, is to find a
  basis for \R{n} whose orthogonality is preserved under $A$.
\item Introduce Fundamental Theorem of Linear Algebra along the four
  subspaces related to an arbitrary matrix
  $A$.
\item Motivated by the discussion about the four subspaces, bring to the
  picture the symmetric matrix $\trans{A}A$ (aka the gramian), and
  state the properties we will need for the SVD proof.
\item The symmetric nature of $\trans{A} A$, will justify the
  usage of the Spectral Theorem; which we proceed to prove.
\item Finally, we  prove SVD \cref{thm:SVD} itself using the Spectral
  Theorem as the main tool, but we also use  the auxiliary theorems
  stated along the way.
\end{itemize}
\hfill

\input{svd-proof-spec-fact.tex}
\input{svd-proof-spec-fund.tex}
\input{svd-proof-spec-gram.tex}
\input{svd-proof-spec-spec.tex}

\subsubsection{The spectral proof of SVD}

We are now all set to prove the SVD theorem, and actually, we do not
need to prove the full version stated in \cref{thm:SVD},
because we have worked out the factorization part with theorems
\cref{thm:SVD1} and \cref{thm:SVD2}. Those theorems started from the
assumption, that there exist two orthonormal bases such that $A\vec{v_i} =
\sigma_i\vec{u_i}$; what remains to prove then, is the existence of
those bases. \\

\begin{theorem}[SVD Part 3: existence of the bases]
\label{thm:SVD3}
Let $A$ be a real matrix of $m \times n$ with rank $r$ $\implies$ there exist
orthonormal basis $\{\vec{v_1},\vec{v_2},\dots,\vec{v_n}\}$ and
$\{\vec{u_1},\vec{u_2},\dots,\vec{u_m}\}$, for \R{n} and \R{m}
respectively; along with positive real values $\sigma_1 \ge \sigma_2 \ge \cdots
\sigma_r$, such that:

\[
A\vec{v_i} = \sigma_i\vec{u_i}
\]
\end{theorem}
\hfill

\begin{proof}
We know from previous sections, that
the key is to find first the
basis for \R{n}, such that its orthogonality is preserved through
$A$ (this interesting approach, and most if this particular proof, is taken from
Kalman \cite{kalman96}). \\

Per the Fundamental
Theorem of Linear Algebra, the symmetric
matrix $\trans{A}A$ came to
the picture; and here comes the magical step: it turns out, that the
eigenvectors of such matrix (whose existence is guaranteed by the
Spectral Theorem we just proved), are precisely the orthonormal basis
$\{\vec{v_1},\vec{v_2},\dots,\vec{v_n}\}$ that we are looking for. Let
us verify that is actually the case, that is, that $A$ preserves the
orthogonality of the eigenvectors of $\trans{A}A$. \\

Let \vec{v_i} and \vec{v_j} be eigenvectors of $\trans{A}A$, and
$\lambda_j$ the eigenvalue of $\vec{v_j}$, then: \\

\[
(A\vec{v_i}) \cdot (A\vec{v_j}) =
\trans{(A\vec{v_i})}(A\vec{v_j}) =
\trans{\vec{v_i}} (\trans{A} A\vec{v_j}) =
\trans{\vec{v_i}} ( \lambda_j \vec{v_j} ) =
\lambda_j \vec{v_i} \cdot \vec{v_j}
\]
\hfill

The above derivation tells us that the orthogonality of the images of
the eigenvectors, named $A\vec{v_i}$ and $A\vec{v_j}$, totally depends of the
orthogonality of the pre-images \vec{v_i} and \vec{v_j}. Another way of
saying that, given that the two eigenvectors were picked arbitrarily,
is that the orthogonality of the eigenvectors of $\trans{A}A$ is
preserved through $A$. This is exactly the basis we were looking for!
\\

The real work is to find the the basis
$\{\vec{v_1},\vec{v_2},\dots,\vec{v_n}\}$ in \R{n}, as the basis in \R{m} is
simply calculated to meet the requirement that $A\vec{v_i} =
\sigma_i\vec{u_i}$. When proving that orthogonality of the
\vec{v}\apos{s} is preserved, we came up with the following identity: \\

\[
(A\vec{v_i}) \cdot (A\vec{v_j}) =  \lambda_j \vec{v_i} \cdot \vec{v_j}
\]
\hfill

The particular case of $i = j$, will give us the following
relationship between the eigenvalues of $\trans{A}A$ and the images
$A\vec{v_i}$ (let us recall that the Spectral Theorem guaranteed an
orthonormal basis, hence $\norm{\vec{v_i}}_2 = 1$): \\

\[
(A\vec{v_i}) \cdot (A\vec{v_i}) =  \lambda_i \vec{v_i} \cdot \vec{v_i}
\ds{\iff} \norm{A\vec{v_i}}_2^2 = \lambda_i \norm{\vec{v_i}}_2^2
\ds{\iff} \norm{A\vec{v_i}}_2 = \sqrt{\lambda_i}
\]
\hfill

Now we just define the vectors \vec{u}\apos{s} as the unitary version
of the images of vectors \vec{v}\apos{s}; and use the above
relationship to bring the eigenvalues of $\trans{A}A$ into the
picture: \\

\[
\vec{u_i} = \dfrac{A\vec{v_i}}{\norm{A\vec{v_i}}} =
\dfrac{1}{\sqrt{\lambda_i}} A\vec{v_i}; \ds{\forall} i=1 \dots r = \func{rank}(A)
\]
\hfill

Do we have enough singular values $\lambda_i$ in $\trans{A}A$ (we
need exactly $r$), and all of them are positive? (otherwise,
$\sqrt{\lambda_i}$ would not be real). We have properly prepared for
this moment, and the whole purpose of having mentioned
\cref{thm:grameig}, was precisely to give a positive answer to these
questions. We are safe in this regard then, and can proceed. \\

In general $\func{rank}(A) = r < m = \dim{\R{m}}$, so we must likely
need to extend the set $\{\vec{u_1},\vec{u_2},\dots,\vec{u_r}\}$ to an
orthonormal basis of \R{m} to complete the SVD
factorization. Fortunately, there is a
known theorem in Linear Algebra that guarantees that we can do that
indeed (see theorem $2.1.1$ from \cite{golub13}, for example). \\

Finally, by naming $\norm{A\vec{v_i}}_2 = \sqrt{\lambda_i}$ as $\sigma_i$
(for $1 \le i \le r$), we can finally achieve the long wanted property
of the two bases: \\

\[
A\vec{v_i} = \norm{A\vec{v_i}} \vec{u_i} =
\sqrt{\lambda_i} \vec{u_i} =
\sigma_i \vec{u_i} \ds{;} \ds{\forall} 1 \le i \le r = \func{rank}(A)
\]
\end{proof}
\hfill

The just proved \cref{thm:SVD3} is the precondition that we
need to apply \cref{thm:SVD1} and \cref{thm:SVD2} from previous
sections. The eigenvalues of the symmetric matrix $\trans{A}A$ may not
necessarily be in descending order, as the SVD theorem requires; but
once we have them, we can sort them in such way (which will implicitly
sort the $\vec{v}\apos{s}$ and $\vec{u}\apos{s}$ vectors in the
bases). All together can finally tackle the original SVD
\cref{thm:SVD}, stated at the beginning of this chapter. This
concludes our proof of the SVD factorization, using the Spectral
Theorem as the main tool. \\

It may had seen as an extremely detailed proof, even if not all the
auxiliary theorems were proved in this work (but most of them were at least
mentioned explicitly, some even formally). This unusual level of
detail may appear cumbersome for the professional mathematician, as all the
literature we consulted always presented quite compressed proofs which
skipped or simplified a lot steps. But we
considered that the approach taken here, could be useful for the occasional
reader and for people who are introducing themselves to the topic, and
want to have an almost self-contained proof of the SVD that requires
little previous context (at least much less than regular books and
articles).  Worth to say also, that this level of detail was needed
for the authors' own understanding as well.


\hypertarget{group__numpp__differentiation__finite__central}{}\section{Central Finite Differentiation}
\label{group__numpp__differentiation__finite__central}\index{Central Finite Differentiation@{Central Finite Differentiation}}
\subsection*{Functions}
\begin{DoxyCompactItemize}
\item
{\footnotesize template$<$typename Func , typename T , typename  = std\+::enable\+\_\+if\+\_\+t$<$std\+::is\+\_\+floating\+\_\+point$<$\+T$>$\+::value$>$$>$ }\\C\+O\+N\+S\+T\+E\+X\+PR auto \hyperlink{group__numpp__differentiation__finite__central_ga310a47693c9fd648ccb763f452a813d0}{numpp\+::differentiation\+::finite\+::central} (Func \&\&f, T x)
\end{DoxyCompactItemize}

\subsection{Detailed Description}
Calculates central finite derivative $\frac{f(x+ 0.5*h)-f(x-0.5*h)}{h}$

\begin{DoxyTemplParams}{Template Parameters}
{\em Func} & Function whose finite difference derivative should be calcuated \\
\hline
{\em T} & Argument type (e.\+g. double)\\
\hline
\end{DoxyTemplParams}

\begin{DoxyParams}{Parameters}
{\em f} & Functor or function whose derivative we want to obtain \\
\hline
{\em x} & Point where the derivative is calculated \\
\hline
{\em h} & Disturbtion to x argument, should be around e-\/7 usually\\
\hline
\end{DoxyParams}
\begin{DoxyWarning}{Warning}
If you are unsure about parameter h use function with automatically calculated overload!
\end{DoxyWarning}
Eventual error performed by this calculation is not provided.

For better accuracy and numerical stability see\+:

\hyperlink{group__numpp__differentiation__symbolic}{Symbolic Differentiation} \hyperlink{group__numpp__differentiation__automatic}{Automatic Differentiation}

\begin{DoxyReturn}{Returns}
Derivative at point x
\end{DoxyReturn}

\subsection{Function Documentation}
\mbox{\Hypertarget{group__numpp__differentiation__finite__central_ga310a47693c9fd648ccb763f452a813d0}\label{group__numpp__differentiation__finite__central_ga310a47693c9fd648ccb763f452a813d0}}
\index{Central Finite Differentiation@{Central Finite Differentiation}!central@{central}}
\index{central@{central}!Central Finite Differentiation@{Central Finite Differentiation}}
\subsubsection{\texorpdfstring{central()}{central()}}
{\footnotesize\ttfamily template$<$typename Func , typename T , typename  = std\+::enable\+\_\+if\+\_\+t$<$std\+::is\+\_\+floating\+\_\+point$<$\+T$>$\+::value$>$$>$ \\
C\+O\+N\+S\+T\+E\+X\+PR auto numpp\+::differentiation\+::finite\+::central (\begin{DoxyParamCaption}\item[{Func \&\&}]{f,  }\item[{T}]{x }\end{DoxyParamCaption})}

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Automatically calculated disturbtion h to argument x, taken from Numerical Recipees (add source)


\section{Introduction}
% problem statement
% motivation
% general approach
% (outlook on results)

%conventions: lower-scalar, lower,bold-vector, upper,bold-matrix

Finding periodic orbits in dynamic systems has various applications mainly in engineering. %cite
This project's goal is the implementation and evaluation of methods for this task.
This document aims at explaining the methods, techniques and ideas used in a concise and connected manner.

The general problem can be described as follows.
Given a system of differential equations $\frac{d \mathbf x}{dt} = \mathbf f(\mathbf x)$, find solutions $\mathbf y$ such that $\mathbf y(t+T) = \mathbf y(t)$ for all $t \in \R$.
Finding closed-form representations of these solutions is impossible in general.
We thus resort to numerical methods for the task.

The tool used for the task of judging whether a given solution is a correct periodic solution is \emph{Galerkin}'s method.
This is a form of collocation method, allowing to convert a continuous operator equation to a system of equations.
Solving this possibly non-linear system yields a periodic solution.

Given a periodic solution for a system smoothly depending on a parameter, if the parameter varies only slightly, so will the solution.
This can be used to track solutions over a broad range of the parameter.
Projecting the found periodic orbits to a simpler object (for example a set of scalars) allows to plot them against the parameter in a bifurcation diagram.
This allows to evaluate the global behavior of the system.
Being able to dynamically create bifurcation diagrams for a given system is a goal of this project.

\subsection{Notation}

Throughout this text, the following conventions are used
\begin{compactitem}
	\item Scalar values are represented by lowercase letters ($a$, $\tilde b$), vectors by lowercase, bold letters ($\mathbf v$), matrices and other higher dimensional constructs by capital, bold letters ($\mathbf M$).
	\item Subsets of the natural numbers up to an upper limit are denoted using subscript notation
		\[
			\N_n \coloneqq \{ x \ |\ x \in \N, x \le n\} \text.
		\]
	\item Sequences of natural numbers are notated in square brackets
		\[
			\text{for } a,b \in \N,\ a \le b,\ [a:b] \coloneqq (a+i-1)_{i \in \N_{b-a+1}} \text.
		\]
	\item Indexing of vectors is done using subscript notation, matrices are indexed using subscript for the row and superscript for the column.
	\item Vectors and matrices are build using subscript notation
		\[
			\mathbf M = (f(i,j))_{i,j \in \N_n} \equiv \mathbf M_i^j = f(i,j) \text{ for } i,j \in \N_n \text.
		\]
	\item Function application using curly brackets denotes element wise application of a function.
		That is, for sets $U$,$V$, $\mathbf u \in U^n$ for an $n \in \N$ and a function $f: U \to V$,
		\[
			f\{\mathbf u\} = \begin{pmatrix}
				f(\mathbf u_1) \\ \vdots \\ f(\mathbf u_n)
			\end{pmatrix} \text.
		\]
	\item The $\cdot||\cdot$ operator denotes concatenation.
	\item The $((\cdot))_N$ operator takes its operand modulo $N$.
	\item Symbols $\Re$, $\Im$ denote the real and imaginary part of its argument.
	\item The symbol $\delta$ denotes the Kronecker delta.
\end{compactitem}


\chapter{Number theory}

\section{Modular arithmetic}
	\kactlimport{ModularArithmetic.h}
	\kactlimport{ModInverse.h}
	\kactlimport{ModPow.h}
	\kactlimport{ModLog.h}
	\kactlimport{ModSum.h}
	\kactlimport{ModMulLL.h}
	\kactlimport{ModSqrt.h}

\section{Primality}
	\kactlimport{FastEratosthenes.h}
	\kactlimport{MillerRabin.h}
	\kactlimport{Factor.h}

\section{Divisibility}
	\kactlimport{euclid.h}
	\kactlimport{Euclid.java}
	\kactlimport{CRT.h}

	\subsection{Bézout's identity}
	For $a \neq $, $b \neq 0$, then $d=gcd(a,b)$ is the smallest positive integer for which there are integer solutions to
	$$ax+by=d$$
	If $(x,y)$ is one solution, then all solutions are given by
	$$\left(x+\frac{kb}{\gcd(a,b)}, y-\frac{ka}{\gcd(a,b)}\right), \quad k\in\mathbb{Z}$$

	\kactlimport{phiFunction.h}

\section{Fractions}
	\kactlimport{ContinuedFractions.h}
	\kactlimport{FracBinarySearch.h}

\section{Pythagorean Triples}
 The Pythagorean triples are uniquely generated by
 \[ a=k\cdot (m^{2}-n^{2}),\ \,b=k\cdot (2mn),\ \,c=k\cdot (m^{2}+n^{2}), \]
 with $m > n > 0$, $k > 0$, $m \bot n$, and either $m$ or $n$ even.

\section{Primes}
	$p=962592769$ is such that $2^{21} \mid p-1$, which may be useful. For hashing
	use 970592641 (31-bit number), 31443539979727 (45-bit), 3006703054056749
	(52-bit). There are 78498 primes less than 1\,000\,000.

	Primitive roots exist modulo any prime power $p^a$, except for $p = 2, a > 2$, and there are $\phi(\phi(p^a))$ many.
	For $p = 2, a > 2$, the group $\mathbb Z_{2^a}^\times$ is instead isomorphic to $\mathbb Z_2 \times \mathbb Z_{2^{a-2}}$.

\section{Estimates}
	$\sum_{d|n} d = O(n \log \log n)$.

	The number of divisors of $n$ is at most around 100 for $n < 5e4$, 500 for $n < 1e7$, 2000 for $n < 1e10$, 200\,000 for $n < 1e19$.

\section{Mobius Function}
\[
	\mu(n) = \begin{cases} 0 & n \textrm{ is not square free}\\ 1 & n \textrm{ has even number of prime factors}\\ -1 & n \textrm{ has odd number of prime factors}\\\end{cases}
\]
  Mobius Inversion:
  \[ g(n) = \sum_{d|n} f(d) \Leftrightarrow f(n) = \sum_{d|n} \mu(d)g(n/d) \]
  Other useful formulas/forms:

  $ \sum_{d | n} \mu(d) = [ n = 1] $ (very useful)

  $ g(n) = \sum_{n|d} f(d) \Leftrightarrow f(n) = \sum_{n|d} \mu(d/n)g(d)$

 $ g(n) = \sum_{1 \leq m \leq n} f(\left\lfloor\frac{n}{m}\right \rfloor ) \Leftrightarrow f(n) = \sum_{1\leq m\leq n} \mu(m)g(\left\lfloor\frac{n}{m}\right\rfloor)$


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{bm}
\usepackage{url}
\usepackage{algorithmic}
\usepackage[margin=2.5cm]{geometry}

%opening
\title{Multiplicative Inverse}
\author{Armando Faz\\\footnotesize{\url{armfazh@ic.unicamp.br}}}
\date{}
\begin{document}
\maketitle

\section{Itoh-Tsujii}
 In order to compute the multiplicative inverse
of an element $a\in \mathbb{F}_p^*$, the following identity is used: $a^{-1}\equiv
a^{p-2}\; (\textrm{mod }p)$; part of this exponentiation can be calculated using an addition chain as shown by Itoh-Tsujii.

Let $x,y\in \mathbb{Z}^+$ and $x\leq y$, define the term $\bm{\alpha}_{x} = a^{2^x-1}$ and the relation $\bm{\alpha}_x\rightarrow \bm{\alpha}_y$ as $\bm{\alpha}_{y}=(\bm{\alpha}_x)^{2^{y-x}}\bm{\alpha}_{y-x}$.

The cost for $\bm{\alpha}_x\rightarrow \bm{\alpha}_y$ is $y-x$ squares and 1 multiplication.

\subsection{$p=2^{384}-2^{128}-2^{96}+2^{32}-1.$}
This is the addition chain for NIST P-384 curve. We start with $
\bm{\alpha}_{1} \rightarrow \bm{\alpha}_{2}  \rightarrow \bm{\alpha}_{3}  \rightarrow
\bm{\alpha}_{6} \rightarrow \bm{\alpha}_{12} \rightarrow \bm{\alpha}_{15} \rightarrow
\bm{\alpha}_{30}$. After that, we pre-compute $T_1 = a^{2^{32}-1}$, $T_2 = a^{2^{32}-3}$, $T_3 = a^{2^{31}-1}$.
using the following:
\begin{eqnarray*}
 T_1 = a^{2^{32}-1} &=& (\bm{\alpha}_{30})^{2^2}a \\
 T_2 = a^{2^{32}-3} &=& (\bm{\alpha}_{30})^{2^2}\bm{\alpha}_2 \\
 T_3 = a^{2^{31}-1} &=& (\bm{\alpha}_{30})^{2}a
\end{eqnarray*}
Then, define  $L_{0},\dots,L_{353}= 0$ and
\begin{eqnarray*}
 a^{T_2} &=& L_{0  } \\
 a^{T_1} &=& L_{96 } =
 L_{129} =
 L_{161} =
 L_{193} =
 L_{225} =
 L_{257} =
 L_{289} =
 L_{321} \\
 a^{T_3} &=& L_{353}
\end{eqnarray*}
To obtain $c =a^{-1}$ compute:
\begin{algorithmic}
 \STATE $c = 1$
 \FOR{$i=353$ \TO 0}
	\STATE $c= c^2$
	\IF{$L_i \neq 0$}
		\STATE  $c = c\times L_i$
	\ENDIF
 \ENDFOR
 \RETURN $c$
\end{algorithmic}
Cost: 19 multiplications and 384 squares.

\subsection{$p=2^{255}-19.$}

It was given an addition chain for $\mathbb{F}_{2^{255}-19}$, starting with $a^9=(a^2)^{2^2}\cdot a$ and $\bm{\alpha}_5=a^{31}=(a^2\cdot a^9)^2\cdot (a^9)$  compute
$
\bm{\alpha}_{5} \rightarrow \bm{\alpha}_{10} \rightarrow \bm{\alpha}_{20} \rightarrow
\bm{\alpha}_{40} \rightarrow \bm{\alpha}_{50} \rightarrow \bm{\alpha}_{100} \rightarrow
\bm{\alpha}_{200} \rightarrow \bm{\alpha}_{250}
$, the multiplicative inverse is obtained as
\begin{eqnarray*}
a^{-1} &=& a^{2^{255}-21} \\
       &=& (\bm{\alpha}_{250})^{2^5}\cdot a^{11}
\end{eqnarray*}
Cost: 11 multiplications and 254 squares.

\subsection{$p=2^{448}-2^{224}-1.$} For $\mathbb{F}_{2^{448}-2^{224}-1}$, starting with
$
\bm{\alpha}_{1} \rightarrow   \bm{\alpha}_{2}  \rightarrow
\bm{\alpha}_{3}   \rightarrow \bm{\alpha}_{6}  \rightarrow \bm{\alpha}_{12} \rightarrow
\bm{\alpha}_{24}  \rightarrow \bm{\alpha}_{27} \rightarrow \bm{\alpha}_{54} \rightarrow
\bm{\alpha}_{108} \rightarrow \bm{\alpha}_{111}\rightarrow \bm{\alpha}_{222} \rightarrow
\bm{\alpha}_{223}
$, the multiplicative inverse is obtained as $a^{-1} = b^{2^2}\cdot a$:
\begin{eqnarray*}
b &=& (\bm{\alpha}_{223})^{2^{223}}\cdot (\bm{\alpha}_{222})\\
  &=& (a^{2^{223}}-1)^{2^{223}} (a^{2^{222}}-1)\\
  &=& a^{2^{446}-2^{223}+2^{222}-1}\\
  &=& a^{2^{446}-2^{222}-1}
\end{eqnarray*}
\begin{eqnarray*}
a^{-1} &=& b^{2^2}\cdot a\\
	   &=& (a^{2^{446}-2^{222}-1})^{2^2}a\\
	   &=& a^{2^{448}-2^{222}-4+1}\\
       &=& a^{2^{448}-2^{224}-3}\\
\end{eqnarray*}

Cost: 13 multiplications and 447 squares.

\paragraph{Bonus:}  The value $b=2^{446}-2^{222}-1$ is used for computing square-roots ($b = \frac{p-3}{4}$); thus, there is only one addition chain used for both purposes.

\end{document}