% \pagelayout{fullwidth}
% \includegraphics{./figures/test.pdf}
% \pagelayout{margin}

\chapter{Formalized Meta-Theory of \kl(tit){PCUIC}}
\label{chap:metacoq-general}

\margintoc

Before we can attempt to build a certified kernel, we need a thorough meta-theoretical study
of the type system. This is necessary in order to show that the invariants used by the kernel – typically, well-formation of the objects it manipulates –
are preserved during the type-checking algorithm.
The use of these invariants goes beyond correctness:
the cumulativity test used as a sub-routine by the \kl{kernel} needs to reduce terms, and,
since all functions in \kl{Coq} must be terminating,
this reduction is defined by well-founded induction on the \kl{normalization}
of well-typed terms.
Since evaluation is not normalizing on ill-typed terms, the mere \emph{definition} of the
cumulativity check relies on \kl{subject reduction} to be able to iterate reduction steps.

The properties under scrutiny in this chapter are not new,
and neither are the basic strategy of most proofs.
Indeed, the development roughly follows the architecture we already exposed in
\cref{sec:tech-properties}. The main difficulty is the scale: due to the
complexity of \kl{PCUIC}, even well-understood techniques are challenging to apply.
Moreover, subtleties that do not appear in a simpler setting become apparent –
typically pertaining to universe levels or general inductive types –,
demanding original ideas. Thus, rather than getting lost in
the gory details of the formalization which are best understood by looking at it
– and maybe replaying it –, we try and focus on describing these interesting subtleties.

In more details, we start with the main definitions : the syntax, cumulativity and typing
judgments (\cref{sec:meta-defs}). We follow with the basic
stability properties (\cref{sec:meta-stabilities}):
renaming, substitution, environment extension, etc.
Next comes the first important proof, that of confluence, and its multiple consequences
(\cref{sec:meta-confluence}). This leads to the properties pertaining to typing, culminating
with subject reduction (\cref{sec:meta-typing-prop}).
Finally, we discuss the place of normalization (\cref{sec:meta-normalization}).

\section{Setting up the Definitions: Terms, Cumulativity and Types}[Setting up the Definitions]
\label{sec:meta-defs}

\subsection{Terms}

First thing first: the syntax of terms, defined in \pcuicfile{Ast} and reproduced in \cref{fig:metacoq-ast}.

\begin{figure*}
  \coqfile{./code/PCUICAst.v}
  \caption{The Abstract Syntax Tree of terms in \kl{MetaCoq} (\pcuicline{Ast}{term}{194})}
  \label{fig:metacoq-ast}
\end{figure*}

It of course contains  the term formers introduced in \cref{chap:tech-intro}:
\coqe{tRel} for variables, \coqe{tAbs} for abstractions,
\coqe{tApp} for application, and \coqe{tProd} for dependent function types.
The syntax uses De Bruijn indices for binders – the integer argument of the \coqe{tRel}
term former –, but names are still recorded, mainly for printing purposes, directly in the
binders – the \coqe{aname} argument of \coqe{tProp} and \coqe{tAbs}.
There are also local definitions, in the form of the \coqe{tLetIn} constructor,
binding the term \coqe{b} of type \coqe{B} in \coqe{t}.

The \coqe{tSort} constructor is for sorts – what we have called universes earlier.
Its \coqe{Universe.t} argument represents its \kl{universe level}, which can be either $\Prop$
or an algebraic expression based on universe variables, in order to handle \kl{typical ambiguity}.

Next come \coqe{tVar} and \coqe{tEvar}, which
correspond respectively to named variables and existential variables.
These are ill-typed in the current notion of typing,
and thus ignored in most of the development. Still, they are kept to be as
faithful as possible to the representation of the \kl{Coq} kernel.
Indeed, the inductive \coqe{term} corresponds directly to the
\mintinline{OCaml}{constr} datatype used there.%
\sidenote{The only differences are that the latter uses
an n-ary application rather than a binary one, and casts that inform the \kl{kernel} as to
which cumulativity algorithm to use, but which is left out since we implement only one such
algorithm.}

Follow the three term formers \coqe{tConst}, \coqe{tInd} and
\coqe{tConstruct} all referring to previous definitions, stored in a \kl{global environment}.
The first corresponds to constants, that either have a body – definitions –
or do not – axioms. The next two are respectively for
inductive types and inductive constructors. Co-inductive and record types and
constructors are also represented by these term formers, the information contained in the
\coqe{inductive} argument is used to separate between them.
All of these can be \kl{universe polymorphic}, in which case they must be instantiated
with a list of universes – their \coqe{Instance.t} argument.

The two subsequent \coqe{tCase} and \coqe{tProj} are destructors for (co-)inductive types. The
latter is a \kl{projection}, used to destruct \kl{record types}. The former represents the
pattern-matching construction. Its main components are the \kl{predicate} \coqe{p}, the
\kl{scrutinee} \coqe{c} and the \kl{branches} \coqe{brs}. While it will appear more clearly
when giving the typing rule, let us note already that \coqe{p} and \coqe{brs} both contain not
only the body of the \kl{predicate}/\kl{scrutinee}, but also the context extension over which
they live, roughly corresponding to the variable bounds in the \kl{recursors} of \cref{sec:tech-cic}.
Thus, they represent a form of binding in the “primitive” way of a context extension, rather
than using Π-types or λ-abstraction. This is the new case representation alluded to
at the end of \cref{sec:bidir-pcuic-inductives}.

Finally, the two very similar \coqe{tFix} and \coqe{tCoFix} are for (co-)fixed-points.
These can be mutual: the \coqe{mfix} argument is a list of definitions,
that can refer to each other.

\subsection{Cumulativity}

The next important definition is that of \kl{cumulativity}, given in \pcuicfile{CumulativitySpec}.%
\sidenote{The “Spec” part comes from the fact that this is the \emph{specification} of cumulativity,
by contrast to the algorithmic version encountered later on.}
It is stated in the \kl[declarative conversion]{declarative} \kl(conv){untyped} fashion,
akin to how we defined \kl{declarative conversion} in
\cref{fig:cic-uconv-beta,fig:cic-uconv-equiv,fig:cic-uconv-cong}.
This time, however, it is done relatively to
both a \kl{global environment} \coqe{Σ} and a context \coqe{Γ},
as these contain definitions that \kl{cumulativity} can unfold.

\kl{Cumulativity} is defined mutually with \kl{conversion}, because for instance when two Π-types
are compared for \kl{cumulativity}, their codomains are recursively compared for \kl{cumulativity},
but their domains are compared for \kl{conversion} instead.%
\sidenote{As explained in \cref{sec:tech-pcuic},
this is a consequence of the models justifying cumulativity.}
Since the two relations are extremely similar, they are actually fused in a single
inductive relation, \coqe{Σ ;;; Γ ⊢ t ≤s[pb] u}.
\AP This relation is indexed by a \intro{conversion problem} \coqe{pb : conv_pb}, which can take
the two values \coqe{Conv} and \coqe{Cumul}, so that \kl{cumulativity}
is actually \coqe{≤s[Cumul]} – and \kl{conversion} is
\coqe{≤s[Conv]}. This has the advantage that a lot of definitions and proofs
can be factored using \coqe{conv_pb}.
Moreover, using a simple boolean allows for case reasoning
when needed, which would be more complex if the index was \eg a relation.%
\sidenote{This used to be the case prior to the uniform introduction of \coqe{conv_pb}: the
relation was the one to be used at leaves to compare universes, which differed between
\kl{conversion} and \kl{cumulativity}.}

\begin{figure}[h]
  \ContinuedFloat*
  \coqfile[firstline=3,lastline=12]{./code/PCUICCumulativitySpec.v}
  \caption{Pre-order rules (\pcuicline{CumulativitySpec}{cumulSpec0}{29})}
  \label{fig:meta-cumul-struct}
\end{figure}

\AP The first set of rules are the pre-order rules of \cref{fig:meta-cumul-struct}:
transitivity, symmetry and reflexivity. Note that symmetry restricts the \kl{conversion problem},
since only \kl{conversion} should be symmetric. Using this rule twice shows that
\kl{conversion} is included inside \kl{cumulativity}.
Another important thing to note is that transitivity is somewhat restricted:
the middle term is required to be \intro{well-scoped},%
\sidenote{Expressed by the \pcuicline[Syntax]{OnFreeVars}{is\_open\_term}{253} predicate.}
\ie all its variables refer correctly
to either a binder or to the context \coqe{Γ}.
This is key when proving the equivalence between this notion of \kl{cumulativity} and the
algorithmic version that appears later on. Indeed, this equivalence relies on \kl{confluence},
which is only true on \kl{well-scoped} terms in \kl{PCUIC}. Thus, we need to know that
\kl(conv){declarative} \kl{cumulativity} only ever goes through \kl{well-scoped} terms,
which is exactly what this condition enforces.

\begin{figure}[ht]
  \ContinuedFloat
  \coqfile[firstline=44,lastline=48]{./code/PCUICCumulativitySpec.v}
  \caption{Example of congruence rule (\pcuicline{CumulativitySpec}{cumulSpec0}{29})}
  \label{fig:meta-cumul-cong}
\end{figure}

\begin{figure}[h]
  \ContinuedFloat
  \coqfile[firstline=15,lastline=29]{./code/PCUICCumulativitySpec.v}
  \caption{Cumulativity rules (\pcuicline{CumulativitySpec}{cumulSpec0}{29})}
  \label{fig:meta-cumul-cumul}
\end{figure}

Next come the rules of congruence. There are actually two kinds of them. The first are the
“standard” ones, similar to those of \cref{fig:cic-uconv-cong}. An example
is given in \cref{fig:meta-cumul-cong}.
More interesting are the rules of
\cref{fig:meta-cumul-cumul}, which implement “real” cumulativity. Rule \coqe{cumul_Sort} directly
implements subtyping between universes, while rules \coqe{cumul_Ind} and \coqe{cumul_Construct}
implement cumulativity of inductive types \sidecite{Timany2017}. The latter two apply respectively
to \emph{fully applied} inductive types and inductive constructors, that can be considered equal
if their arguments are one-to-one convertible, and their universe levels are correctly related.
This means that \eg the \coqe{nil} constructor of polymorphic lists \emph{always} satisfies that
\coqe{nil@{u} A} is convertible to \coqe{nil@{u'} A}, irrespective of the universe levels \coqe{u}
and \coqe{u'}.

\begin{figure*}
  \ContinuedFloat
  \coqfile[firstline=87,lastline=96]{./code/PCUICCumulativitySpec.v}
  \caption{Computation rules for destructors (\pcuicline{CumulativitySpec}{cumulSpec0}{29})}
  \label{fig:meta-cumul-dest}
\end{figure*}

Last are the rules for computation.
The three rules of \cref{fig:meta-cumul-dest} are for destructors, \ie applied functions,
pattern-matching on a constructor, and projections. The β rule for functions directly uses
substitution: \coqe|b{0 := a}| denotes the substitution of \coqe{a} for the variable of De Bruijn
index \coqe{0} in \coqe{b}. Similarly, the
\coqe{iota_red} function computes the substitution of the branch \coqe{br} by the arguments
of the constructor \coqe{args}.
Finally, the rule for projections simply selects the right field of the record.

\begin{figure*}
  \ContinuedFloat
  \coqfile[firstline=99,lastline=106]{./code/PCUICCumulativitySpec.v}
  \caption{Computation rules for definitions (\pcuicline{CumulativitySpec}{cumulSpec0}{29})}
  \label{fig:meta-cumul-defs}
\end{figure*}

The next three rules deal with definitions (\cref{fig:meta-cumul-defs}).
Rule \coqe{cumul_zeta} directly reduces a
let-binder into a substitution, while definitions are unfolded using \coqe{cumul_rel} and \coqe{cumul_delta},
respectively those of the local context or the \kl{global environment}. In the latter
case, the definition must be instantiated with a universe instance,%
\sidenote{A list of universe levels, corresponding to its polymorphic universe variables.}
which is denoted \coqe{@[u]}.

\begin{figure}[h]
  \ContinuedFloat
  \coqfile[firstline=108,lastline=119]{./code/PCUICCumulativitySpec.v}
  \caption{Computation rules for fixed-points (\pcuicline{CumulativitySpec}{cumulSpec0}{29})}
  \label{fig:meta-cumul-fix}
\end{figure}

The last rules (\cref{fig:meta-cumul-fix}) pertain to the reduction of (co-)fixed-points.
In all cases, they are unfolded in a guarded fashion, in order to avoid a non-terminating behaviour.
On fixed-points, this guard is that they have to be applied to a constructor,
and dually co-fixed-points are unfolded when
they are forced, either by a pattern-matching or a projection. In both cases, this ensures that
the unfolded (co-)fixed-point can reduce further, either by consuming the constructor of
its recursive argument, or by producing a constructor to be consumed by the destructor
that forced the unfolding.

\FloatBarrier
\subsection{Typing}

With the \kl{cumulativity} relation defined, we can turn to typing, defined in \pcuicfile{Typing}.
Similarly to \kl{cumulativity}, \pcuicline{Typing}{typing}{188}
is an inductively defined relation \coqe{Σ ;;; Γ |- t : T},
relative to a global environment \coqe{Σ} and a local context \coqe{Γ}.
The rules correspond roughly to ones we already went over in \cref{chap:tech-intro}.

\begin{figure}
  \ContinuedFloat*
  \coqfile[firstline=2,lastline=22]{./code/PCUICTyping.v}
  \caption{Functional fragment (\pcuicline{Typing}{typing}{188})}
  \label{fig:meta-typing-ccw}
\end{figure}

The first set of typing rules, given in \cref{fig:meta-typing-ccw},
pertain to the purely functional fragment. There are not many differences
there with respect to \cref{fig:ccw-typing}. Rule \coqe{type_Rel} looks up for the type of a
variable in the context, and ensures that said context is well-formed:
the \pcuicline{Typing}{wf\_local}{284} predicate corresponds to $\vdash \Gamma$,
but here as everything else it is relative to a global environment.
Rule \coqe{type_Sort} uses the \coqe{super} function to compute
the successor of an algebraic universe, and similarly Rule \coqe{type_Prod} uses
\coqe{sort_of product} to compute the universe level of a Π-type.%
\sidenote{This not only includes computing the maximum of two algebraic universes expressions,
but also handling the \kl{impredicativity} of the sort $\Prop$.}
Context extension with an assumption, \ie a variable without a body,
is written \coqe{Γ ,, vass na A}, and used as expected in Rules \coqe{type_Prod}
and \coqe{type_Lambda}. Finally, \coqe{type_App} is for application. It contains an
assumption that the product is well-formed, which is not strictly speaking needed once we
prove \kl{validity}, but is useful in some cases to provide a needed induction hypothesis.

\begin{figure}
  \ContinuedFloat
  \coqfile[firstline=23,lastline=27]{./code/PCUICTyping.v}
  \caption{Local definitions (\pcuicline{Typing}{typing}{188})}
  \label{fig:meta-typing-letin}
\end{figure}

The rule for local definitions, given in \cref{fig:meta-typing-letin}, is similar to the one
for λ-abstractions, with the only difference that the body too is typed, and that the context
is extended with a definition, \ie a variable with a body, which is written \coqe{Γ,, vdef na b B}.

\begin{figure*}
  \ContinuedFloat
  \coqfile[firstline=28,lastline=42]{./code/PCUICTyping.v}
  \caption{Globally defined terms (\pcuicline{Typing}{typing}{188})}
  \label{fig:meta-typing-genv}
\end{figure*}

Next (\cref{fig:meta-typing-genv}) are the three rules performing look-ups in
the \kl{global environment}, respectively constants – \coqe{type_Const} –, inductive types –
\coqe{type_Ind} – and inductive constructors – \coqe{type_Construct}. In all cases the
term should be declared in the \kl{global environment}, the context well-formed
– since these are leaves of a term –,
and the universe instance given should respect the constraints coming from the
entry in the \kl{environment} – this is the \coqe{consistent\_instance\_ext} predicate.

\begin{figure*}
  \ContinuedFloat
  \coqfile[firstline=43,lastline=58]{./code/PCUICTyping.v}
  \caption{(Co-)inductive destructors (\pcuicline{Typing}{typing}{188})}
  \label{fig:meta-typing-idest}
\end{figure*}

The rules of \cref{fig:meta-typing-idest} are the ones for the destructors of (co-)inductive
types. Rule \coqe{type_Case} is somewhat similar to \ruleref{rule:bool-ind}. First,
the \kl{scrutinee} should be of an inductive type, declared in the global environment.
Next, the \kl{predicate} and \kl{branches} should all be well-typed
in the appropriate context – obtained by combining the information stored in
\coqe{p} or \coqe{brs}, with that retrieved from the entry in the environment corresponding to
the \kl{scrutinee}'s type.
Finally, \pcuicline{Typing}{case\_side\_conditions}{159} handles universe instances, checks
that the elimination is allowed%
\sidenote{This is where \kl{PCUIC} enforces computational irrelevance of proofs,
by imposing the so-called
“singleton elimination” criterion, which ensures that only inductive
types of a certain specific shape – sub-singletons – can be matched on to build proof relevant
content, so that that content cannot actually depend on the value of a proof.}…
The second rule, \coqe{type_Proj}, is somewhat similar, albeit a bit simpler: the \kl{scrutinee}
should still be some applied co-inductive/record type, and the type is constructed by
substitution from the projection information.

\begin{figure}
  \ContinuedFloat
  \coqfile[firstline=59,lastline=76]{./code/PCUICTyping.v}
  \caption{(Co-)fixed-points (\pcuicline{Typing}{typing}{188})}
  \label{fig:meta-typing-fix}
\end{figure}

The last typing rules for terms are those for (co-)fixed-points,
given in \cref{fig:meta-typing-fix}. They are almost identical,
the main part amounts to checking that the types and bodies of all
the mutual definitions are well-typed.
The \pcuicline{Typing}{wf\_fixpoint}{118} and \pcuicline{Typing}{wf\_cofixpoint}{145}
predicates both check that the definitions are on
the same block of mutually-defined (co-)inductive types, and that these are of the right kind –
inductive for \coqe{tFix} and co-inductive for \coqe{tCoFix}.
Finally, the \coqe{fix_guard} and \coqe{cofix_guard} predicates correspond to the guard condition,
ensuring that the definitions do not endanger \kl{normalization}. We come back to those in
\cref{sec:meta-normalization}.

\begin{figure}
  \ContinuedFloat
  \coqfile[firstline=77,lastline=81]{./code/PCUICTyping.v}
  \caption{Cumulativity (\pcuicline{Typing}{typing}{188})}
  \label{fig:meta-typing-cumul}
\end{figure}

The final rule (\cref{fig:meta-typing-cumul}) is that which uses \kl{cumulativity} as just defined
to change the type of the term, \eg the equivalent of \ruleref{rule:cic-cum}.

From the definition of \pcuicline{Typing}{typing}{188}, two more pervasively used definitions
follow. We have already encountered \pcuicline{PCUICTyping}{wf\_local}{284},
asserting that a local context is well-formed.
Its sibling predicate \pcuicline{PCUICTyping}{wf}{439}%
\sidenote{Often replaced by \pcuicline{PCUICTyping}{wf\_ext}{444},
an extension that in addition takes into account the universes of the current definition.}
asserts that the \kl{global environment} is well-formed. It ensures not only that all definitions
are properly typed, but also of the validity of various information
related to inductive types – in particular the positivity criterion which ensures that inductive
definitions are well-founded –, and universe polymorphism.

\section[Stabilities]{The Easy Properties: Stabilities}
\label{sec:meta-stabilities}

With the main definitions set up, we can turn to the properties that we collectively called
stabilities in \cref{sec:tech-properties}. These assert that \kl{cumulativity} and typing
as just defined are stable by various ubiquitous operations: extension of the local context%
\sidenote{\pcuicfile[Conversion]{WeakeningConv} and \pcuicfile[Typing]{WeakeningTyp}.}
and global environment,%
\sidenote{\pcuicfile[Conversion]{WeakeningEnvConv} and \pcuicfile[Typing]{WeakeningEnvTyp}.}
and substitution, not only for terms,%
\sidenote{\pcuicfile[Conversion]{InstConv} and \pcuicfile{Substitution}}
but also for universe variables.%
\sidenote{\pcuicfile[Conversion]{UnivSubstitutionConv} and \pcuicfile[Typing]{UnivSubstitutionTyp}}

\AP One last property falling in the section of low-hanging fruits as well
is the fact that well-typed terms are \kl{well-scoped}.%
\sidenote{\pcuicfile{ClosedTyp}.}
%, that is that if \coqe{Σ ;;; Γ |- t : T}
%then all free variables of \coqe{t} appear in \coqe{Γ}.
This \kl{well-scoping} conditions appears
in the transitivity rule for \kl{cumulativity}
(\cref{fig:meta-cumul-struct}) and is a hypothesis for many lemmas.

All of these are proven by induction on the typing derivation.
While the proof by themselves are not very surprising, the formalization of the definitions
deserves a few comments.

The first point to note is that while the weakening and substitution operations are defined
directly by induction on the syntax of terms, the proofs are not done directly
on those definitions. Rather, \kl{MetaCoq} uses notions of renaming and instantiation inspired
by the σ-calculus \sidecite{Abadi1991,Schaefer2015}, as functions from natural numbers
to natural numbers for renamings, and to terms for instantiations.
\kl{Weakening} and \kl{substitution} then correspond respectively to a specific form of
renaming and of instantiation, and the stability for the former those follows from
more general versions for the latter.
For instance, \kl{weakening} is but a consequence of general stability by
(unconditional) renaming, as presented in \cref{prop:stab-renaming}.
This approach makes it easier to handle the complex binding structures
present in the syntax of \kl{PCUIC}, which require parallel substitution or
lifting by a whole context at once,
operations that are easier to handle in the general framework of the σ-calculus.

More interestingly and novel, the same approach is taken also for \kl{well-scoping}:
\kl{MetaCoq} generalizes that predicate into a way of lifting
any boolean function on natural numbers – seen as a property of variables – to a boolean function
on terms.\sidenote{This is \pcuicline[Syntax]{OnFreeVars}{on\_free\_vars}{71}.}
This makes it easy to relate the σ-calculus
to \kl{well-scoping} assumptions, \eg to show that if two substitutions \coqe{σ} and \coqe{σ'}
agree on the free variables of a term \coqe{t}, then the application of \coqe{σ} and \coqe{σ'}
to \coqe{t} are equal.

\begin{figure}[h]
  \coqfile{./code/PCUICSubstitution.v}
  \caption{Well-formed substitution (\pcuicline{Substitution}{subslet}{44})}
  \label{fig:meta-subslet}
\end{figure}

A last point of interest is the definition of a well-formed substitution, a predicate called
\pcuicline{Substitution}{subslet}{44}.
Indeed, the usual typing judgment for substitutions
is of the form $\Delta \vdash \sigma \ty \Gamma$, meaning that
$\sigma$ maps each assumption $(x : A) \in \Gamma$ to a term $t$ such that
$\Delta \vdash t \ty \multisubs{T}{\sigma}$. But in our setting we must account for variables
that can be \emph{defined} in $\Delta$. This leads to the definition of a well-formed substitution as in \cref{fig:meta-subslet}.
A similar definition, called \pcuicline[Syntax]{InstDef}{well\_subst}{65}, is also available for
instantiations.

\section[Confluence]{Things Get Serious: Confluence}
\label{sec:meta-confluence}

\begin{figure*}[ht]
  \coqfile{./code/PCUICReduction.v}
  \caption{\kl{One-step reduction} and \kl{reduction}}
  \label{fig:meta-red}
\end{figure*}

As in \cref{sec:tech-properties}, the next step is to define \kl{reduction} and
establish its properties.
An excerpt of the definitions is given in \cref{fig:meta-red}:
\kl{reduction} is \pcuicline{Reduction}{red}{303},
defined as as the reflexive, transitive closure
of \kl{one-step reduction} \pcuicline{Reduction}{red1}{18}.
The former is written \coqe{Σ ;;; Γ |- t ~>* u}, and the latter
\coqe{Σ ;;; Γ |- t ~> u}.

\subsection{Parallel reduction}
\label{sec:metacoq-parred}

\AP The proof of \kl{confluence} follows the standard Tait-Martin-Löf approach as exposed by
\sidetextcite{Takahashi1995}. It relies on a notion of \intro{parallel reduction}
$\mathord{\intro*\parred}$, which can reduce multiple redexes present in a term $t$ in parallel.
As an example, the rules for application are given in \cref{fig:parred-app}.
\begin{marginfigure}
  \begin{mathpar}
  \inferrule
    {t \parred t' \\ u \parred u'}
    {t\ u \parred t'\ u'} \and
  \inferrule
    {t \parred t' \\ u \parred u'}
    {(\l x : A.\ t)\ u \parred \subs{t'}{x}{u'}}
  \end{mathpar}
  \caption{Parallel reduction for application}
  \label{fig:parred-app}
\end{marginfigure}
The generic congruence
rule allows reduction to happen in parallel in both the function and argument. Moreover,
if the function is an abstraction, a β step can also be fired simultaneously
with those. Note that this does \emph{not} allow reducing further redexes
that would be produced by the substitution. For instance, we do not have
\[(\l f : \Nat \to \Nat . f\ 0)\ (\l x : \Nat. x) \parred 0\]
because the redex $(\l x : \Nat. x)\ 0$ only appears after a first step of substitution.

\kl{Parallel reduction} has two interesting properties. First, it
is related to \kl[reduction]{standard reduction}.

\begin{lemma}[Parallel reduction and reduction]
  \label{lem:parred-red}
  We have $\ored \subset \mathord{\parred} \subset \mathord{\fred}$.

  This implies that if \kl{parallel reduction} is \kl{confluent}, then so is \kl{reduction}.
\end{lemma}

\AP But the interesting characteristic of \kl{parallel reduction}, which \kl{reduction}
does not satisfy, is the
\intro{diamond property}, a strong version of local confluence,%
\sidenote{
  Which says that if $t \ored t_1$ and $t \ored t_2$ then there exists $t'$ such that $t_1 \red t'$
  and $t_2 \red t'$.}
which contrarily to it implies \kl{confluence} even in the absence of \kl{normalization}.
Thanks to \cref{lem:parred-red} above, in order to establish \kl{confluence} of
\kl{reduction}, it suffices to show this \kl{diamond property}.

The proof idea goes as follows. First, show that \kl{parallel reduction} is substitutive, in
the following sense.
\begin{lemma}[Parallel reduction is substitutive]
  If $t \parred t'$ and $u \parred u'$ then
  $\subs{t}{x}{u} \parred \subs{t'}{x}{u'}$.
\end{lemma}
\AP This allows to define a \intro{best parallel reduct} $\intro*\bestredop$,
which reduces \emph{all} possible redexes in parallel,%
\sidenote{ For instance $\bestred{(\l x : A.\ t)\ u}$ is $\subs{\bestred{t}}{x}{\bestred{u}}$.
}
and to show that it is really a best reduct.
\begin{lemma}[Triangle property]
  Given a term $t$, if $t \parred t'$ then $t' \parred \bestred{t}$.
\end{lemma}

\begin{marginfigure}
  \centering
  \begin{tikzcd}
      t \arrow[triple,dr] \arrow[triple,d] & \\
      t' \arrow[triple,r] & \bestred{t}
  \end{tikzcd}

  \begin{tikzcd}
    & t \arrow[triple,dl] \arrow[triple,dr] & \\
    t_1 \arrow[triple, dr] && t_2 \arrow[triple,dl] \\
    & \bestred{t} &
  \end{tikzcd}
  \caption{The triangle and diamond properties, as diagrams}
\end{marginfigure}

This is enough to get the diamond property, because any two parallel reducts of a term $t$
both reduce to $\bestred{t}$ \emph{in one step}. This basically amounts to firing in both reducts
all the redexes that could have been triggered but have not been.

\begin{minipage}{\textwidth}
\begin{lemma}[Diamond property]
  Given a term $t$ and $t_1$, $t_2$ such that $t \parred t_1$ and $t \parred t_2$, there exists
  $t'$ such that $t_1 \parred t'$ and $t_2 \parred t'$.
\end{lemma}
\end{minipage}

From this, it follows by a bit of diagram chasing that \kl{parallel reduction} is confluent,
and thus that \kl{reduction} is.

\begin{minipage}{\textwidth}
\begin{theorem}[\kl{Confluence} of \kl{reduction}]
  \kl{Reduction} is \kl{confluent}, that is if $t \red t_1$ and $t \red t_2$ then there exists
  $t'$ such that $t_1 \red t'$ and $t_2 \red t'$.
\end{theorem}
\end{minipage}

Note that it can directly be shown without resorting to \kl{parallel reduction} that \kl{reduction}
is \emph{locally} confluent. However, in the absence of \kl{normalization}
to apply Newman’s lemma, this
is not enough to ensure \kl{confluence}. But, while we can hope that \kl{normalization} holds for
well-typed term, it is clearly false on untyped terms. Yet, it is crucial to get confluence on
those, as otherwise we would not be able to establish \kl{injectivity of type constructors} and thus
\kl{subject reduction} with our untyped notion of \kl{cumulativity}.
Thus, this detour through \kl{parallel reduction} is really unavoidable.

\subsection{Formalizing Takahashi’s proof}

The previous section sets down a quite precise plan, that we can almost directly follow in \kl{MetaCoq}. There is one important subtlety though: because of local definitions, \kl{reduction}
depends on contexts, so these must be taken into account in \kl{parallel reduction}. But
bodies of definitions should also be reduced by \kl{parallel reduction},
and so the actual relation is between pairs of a context and a term,
something like $\Gamma, t \parred \Gamma', t'$.

Apart from this difficulty%
\sidenote{And the technicality of defining $\bestredop$ in a terminating fashion,
which is done using the \kl{Equations} plugin \cite{Sozeau2019a}.}%
\margincite{Sozeau2019a}
the plan can be followed quite closely.
\kl{Parallel reduction} is defined as \pcuicline{ParallelReduction}{pred1}{241} in
\pcuicfile{ParallelReduction}. Then the diamond property is proven in \pcuicfile{ParallelReductionConfluence} – $\bestredop$ is \pcuicline{ParallelReductionConfluence}{rho}{697},
and the diamond property itself is \pcuicline{ParallelReductionConfluence}{pred1\_diamond}{4421}.
Finally, \pcuicfile{Confluence} goes back from this to properties of \kl{reduction}, concluding
with its \kl{confluence} (\pcuicline{Confluence}{red\_confluence}{3959}).

\section[The Road to Subject Reduction]{Reaping the Fruits: the Road to Subject Reduction}
\label{sec:meta-typing-prop}

With \kl{confluence} proven, it is time to reap the fruits. First,
\kl(conv){declarative} \kl{cumulativity},
as used to define typing, can be related to \kl(conv){algorithmic} \kl{cumulativity}. This entails
many useful consequences, including \kl{injectivity of type constructors}.
A series of important properties then follow, culminating with \kl{subject reduction}.

\subsection{Algorithmic cumulativity}

The first use of confluence is to relate \kl(conv){declarative} \kl{cumulativity}
as used to define typing to \kl(conv){algorithmic} \kl{cumulativity} –
\pcuicline{Cumulativity}{cumulAlgo}{31} –
defined as reduction to terms related by the \kl{α-pre-order} $\alphleq$.%
\sidenote{The generalization of \kl{α-equality} to handle \kl{cumulativity}.}
But in the setting of \kl{PCUIC} this relation, called \pcuicline{Equality}{leq\_term}{382} in
the formalization, is far from being trivial to define!
It needs to handle algebraic universe levels, but also polymorphic inductive types. Moreover,
it is parameterized over the relation used to compare universes, so that it can also be used
to express “pure” \kl{α-equality}%
\sidenote{Meaning that the terms can differ only on binder names, but that their universe
must be syntactically the same rather than related using the constraints present in the
environment.}
when instantiated with equality rather than universe comparison.

To show that \kl{algorithmic conversion} is equivalent to \kl{declarative conversion},
\kl{confluence} is the major ingredient, but it is not enough. Instead, we also need to show
that \kl{reduction} interacts well with this \kl{α-pre-order}.

\begin{lemma}[The \kl{α-pre-order} is a simulation
    (\pcuicline{Confluence}{red1\_eq\_term\_upto\_univ\_l}{838})]
  \begin{marginfigure}
    \centering
    \begin{tikzcd}
        t \arrow[d, "1" very near end] \arrow[r, phantom, "\alphleq" description] &
        t' \arrow[d, dashed, "1" very near end] \\
        u \arrow[r,phantom,"\alphleq" description] & u'
    \end{tikzcd}
    \caption{Simulation, as a diagram}
    \label{fig:alphleq-sim}
  \end{marginfigure}
  If $t \alphleq t'$ and $\Gamma \vdash t \ored u$ then there exists some $u'$ such that
  $\Gamma \vdash t' \ored u'$ and $u \alphleq u'$.
\end{lemma}

While the proof
is still relatively straightforward since $t$ and $t'$ have the same structure, it becomes
much more challenging if we wish to integrate extensionality rules such as η-equality or
strict propositions%
\sidenote{Two features present in \kl{Coq} but not yet in \kl{MetaCoq}, exactly due to this
kind of difficulties.}
into $\alphleq$.

Combining this simulation property with \kl{confluence},
transitivity of \kl(conv){algorithmic} \kl{cumulativity} follows,
and finally its equivalence with \kl(conv){declarative} \kl{cumulativity}.

\begin{theorem}[Equivalence of the presentations of \kl{cumulativity}]
  \kl(conv){Algorithmic} and \kl(conv){declarative} \kl{cumulativity} are equivalent.
\end{theorem}

This equivalence is the main theorem of \pcuicfile{Conversion} –
one direction is \pcuicline{Conversion}{cumulSpec\_cumulAlgo}{3680},
and the other is \pcuicline{Conversion}{cumulAlgo\_cumulSpec}{190}.
That central file also proves multiple variants of
\kl{injectivity of type constructors}.
For instance, \kl{injectivity of function types} is \pcuicline{Conversion}{ws\_cumul\_pb\_Prod\_Prod\_inv}{718},
and the stronger form used in \kl(bidir){completeness} of bidirectional typing
(\cref{lem:conv-red-tycons}) is \pcuicline{Conversion}{ws\_cumul\_pb\_Prod\_r\_inv}{686}.

\subsection{Reaping the fruits}

With injectivity settled, we can get to the main properties of typing.
The easiest is \pcuicline{Validity}{validity}{453},
asserting that if $\Gamma \vdash t \ty T$ then $T$ is a well-formed type in $\Gamma$.
The second is that typing is insensible to names (\pcuicline{Alpha}{typing\_alpha}{1009}):
if two terms differ only in variable names and one is typable, then so is the other.
And, finally, comes \pcuicline{SR}{subject\_reduction}{3072}.

While the main proofs of these theorems are far from simple,%
\sidenote{That for \kl{subject reduction} needs more than 1500 lines for the main induction!}
an important part of the proof effort is actually required ahead of them
in \pcuicfile{Inductives} and \pcuicfile{InductiveInversion},
in order to show that the various contexts, substitutions, applications, etc.
that appear in conversion and typing for inductive types behave as expected.

Regarding \kl{subject reduction}, a caveat applies. Because the “positive”
presentation of co-fixed-points does not preserve types, as explained in
\cref{sec:pcuic-records}, only the “negative” presentation based on projections is allowed.
In practice, this means
that the typing rule \pcuicline{Typing}{type\_case}{241} of \cref{fig:meta-typing-idest}
forbids the \kl{scrutinee} from being of a co-inductive type.%
\sidenote{This is part of the \pcuicline{Typing}{case\_side\_conditions}{159}.}
Hence, while \kl{reduction} and \kl{conversion} take this presentation into account,%
\sidenote{See the \pcuicline{CumulativitySpec}{cumul\_cofix\_case}{158}
constructor in \cref{fig:meta-cumul-fix}.}
thus showing at least that it is confluent, it can never appear in a well-typed term.

An alternative solution, which would allow an easier transition away from today’s positive
co-inductive types, is to see co-inductive scrutinees as effectful terms
and restricts predicates allowed
for dependent elimination to be linear, following \sidetextcite[0em]{Pedrot2017}.
This approach is not formalized in \kl{MetaCoq} – yet –, but the project
provides a natural setting to explore this kind of questions in all their gritty details
before working towards an actual implementation in \kl{Coq}.

\section[Normalization]{Gödel’s Thorn in the Side: Normalization}
\label{sec:meta-normalization}

One last important property remains: \kl{normalization}. In \kl{PCUIC}, the key constraint to
ensure it is the \kl{guard condition}, to which (co-)fixed-points are subject in order to
ensure that they are well-founded – see \cref{sec:pcuic-ind}.
However, due to Gödel’s second incompleteness theorem, if \kl{PCUIC} is \kl(log){consistent}
then it cannot prove its own \kl(log){consistency}.
But if \kl{normalization} were provable, then so would be \kl(log){consistency}.
There is therefore no hope to give a guard condition and prove that it entails \kl{normalization}.

\subsection{An abstract guard condition}

\begin{figure}[h]
  \coqfile{./code/PCUICGuard.v}
  \caption{The \pcuicline{Typing}{guard}{54} axiom}
  \label{fig:meta-guard}
\end{figure}

Instead, \kl{MetaCoq} takes a different approach. The existence of a \kl{guard condition} is
\emph{assumed} in an abstract, axiomatic fashion – see \cref{fig:meta-guard} –
and used in typing – see \cref{fig:meta-typing-fix}.
Similarly, \pcuicfile{GuardCondition} assumes properties of this axiomatic guard: it
should be stable by universe and term substitution, extension of the \kl{global environment},
cumulativity of the local context insensible to names, and, most importantly, stable by
reduction.

These abstract properties are enough to handle the whole development outlined above. In other
words, given any notion of \kl{guard condition} that satisfies the criteria of
\pcuicfile{GuardCondition}, typing satisfies \kl{injectivity of type constructors},
\kl{validity}, \kl{subject reduction}… Thus, our abstract approach provides a precise
characterization of the properties the \kl{guard condition} needs to satisfy in order
for typing to be well-behaved.

In particular, since the trivial guard condition
that is always true fulfils the requirements, none of the aforementioned
properties rely on \kl{normalization} – or \kl(log){consistency} of the theory.
Thus, \kl{PCUIC} is a \kl[safety]{safe} programming language, \emph{unconditionally}.

\subsection{The normalization axiom}

\begin{figure}[h]
  \coqfile{./code/PCUICSN.v}
  \caption{The \pcuicline{SN}{normalization}{36} axiom}
  \label{fig:meta-sn}
\end{figure}

But of course we need more. In particular, if we wish to define a convertibility check
inside \kl{Coq}, which only allows to define terminating functions,
we must know that reduction is terminating. Once again, we axiomatize the necessary
axiom of (strong) \kl{normalization}, as given in \cref{fig:meta-sn}.
Note that this axiom takes exactly the form we gave to \kl{normalization}
in \cref{prop:normalization}, as the accessibility of any well-typed term for
co-reduction. This is done under the assumption that the global context is well-formed,
otherwise it could contain \eg non-positive inductive types which
could be used to define non-terminating terms.

Using this axiom, it is possible to prove \kl(log){consistency} of \kl{PCUIC}, as shown in
\safefile{Consistency}. The rough idea of the proof is that given for \cref{prop:log-cons},
\ie to deduce \kl(log){consistency} from \kl{canonicity}.
However, instead of proving \kl{progress} directly we rely on a proven-complete function
computing \kl(red){weak-head} normal forms implemented as part of the type-checker.%
\sidenote{Which can be seen as a constructive witness of \kl{canonicity}!}
Assuming an inhabitant \coqe{t} of $\P b : \Prop.\ b$%
\sidenote{Corresponding to the \kl{MetaCoq} term
\coqe{tProd b (tSort Prop_univ) (tRel 0)}.}
in any axiom-free global environment \coqe{Σ}, the proof extends
\coqe{Σ} with the empty inductive type $\Empty$, use \coqe{t} to construct a
\kl(red){weak-head} normal inhabitant of that type, and from this finally derive a contradiction.


\subsection{View}
A view is basically an index set on a block. It provides an interface between a subset of data stored in a block and a function needing to calculate on that data.  In \cvl{} we say the blocks have a shape which is for \cvl{} a matrix, vector, or tensor but in general you should be able to define many types of views including for sparse data.
\\[6pt]
For \cvl{} a view must be bound to a block and the binding is immutable. I am not sure what the reason was to make the binding immutable in \cvl{} but in general some types of views may not bind to some types of blocks. \\
For \cvl{} the primary view creation routine is the \\*
\ttbf{\emph{ds}view\_\emph{p}* vsip\_\emph{ds}bind\_\emph{p}(length,hint)}\\*
routine.
\paragraph{Comment on view creation} The primary method to create a view is the bind operation. Necessary information to create a view (bind parameters) is the block object the view will index, and the shape information (offset, strides(s), lengths). For \cvl{} the \ttbf{bind} functionality is in the \vw sections for the particular \vw shape. In \pyjv{} I made \ttbf{bind} a method within the Block class. So to create a vector view instance you would do
\begin{minted}{python}
aView=aBlock.bind(aOffset,aStride,aLength)
\end{minted}
or to create a matrix view instance you would do
\begin{minted}{python}
aView=aBlock.bind(aOffset, aColStride, aColLength,aRowStride,aRowlength}
 \end{minted}
\paragraph{Comment on Sub View Types} Both C++ VSIPL and Mark Haymans data model seem to have some special View type for Subviews. I don't know what this is. For \cvl{} we have a subview function but it just returns a \vw{}. It has no special type. Ditto for rowview, colview, transview, etc.  They all just return \vw{}s. Perhaps somebody could explain what the subview type is about.  See, for instance, line 27 in figure 1\\*
\begin{minted}{cpp}
 vsip::Matrix<vsip::scalar_f>::subview_type  Auu(A(uu));
 \end{minted}


%-------------------------------------------------------
%    DOCUMENT CONFIGURATIONS
%-------------------------------------------------------

%-------------------------------------------------------
%    START OF PROOF OF ACTIVITY ANALYSE
%-------------------------------------------------------

\subsection{Proof of Activity}
The protocol from Bentov, Lee, Mizrahi and Rosenfeld  \cite{Bentov2013ProofStake} which is implemented into PeerCoin (and its clones), is considered as a hybrid of the PoW and PoS. The nodes are doing PoW work by mining blocks and at the same time with the PoS (meaning that the blockchain includes both types of blocks).

\paragraph{The procedure}
\begin{itemize}
\item The PoW miner mine.
\item Block is found, the network is notified and creates a template. (multiple templates are possible)
\item The block hash is used to find random owners by using its hash as numbers to determine owners (nodes from the network).
\item Turn by turn each chosen owners sign the key with the key of the block.
\begin{itemize}
\item If a chosen owner is unavailable, the process paused. (it is not a problem this concurrently miners are still mining and generating new templates with different owners)
\end{itemize}
\item At some point in time, blocks will be signed, and the reward will be given to the miner and the owners.
\end{itemize}

\paragraph{Continues data exchange}
To reduce the data traffic, each template does not include a transaction list during the signing process itself; it is the last owner (signer) that is adding it when creating the block.

%-------------------------------------------------------
%    END OF PROOF OF ACTIVITY ANALYSE
%-------------------------------------------------------


\chapter{Variational inference}


\chapter{Tools for the LHC}
\label{ch:LHCtools}
Including data from a wide range of LHC or collider sources into a global PDF determination provides several challenges, particularly in the context of the computationally intensive
NNPDF methodology. In this chapter we shall discuss some of the methods that have been developed in order to study the impact of collider data, and include their constraints into
PDF fits.

Firstly we shall describe the method of Bayesian reweighting of Monte Carlo error sets, along with the associated set of tools made available by a Bayesian study of PDF sets and their uncertainties.
Secondly the FastKernel method developed by the NNPDF collaboration for the fast evolution of PDFs will be introduced, along with its extension to the fast computation of experimental observables in the {\tt FK} method.
Finally we shall discuss the application of interpolation methods such as FastKernel to the automated calculation of cross sections at next-to-leading order accuracy in QCD. To this end
we shall perform a brief overview of such calculations in the context of general purpose event generators.

\section{Bayesian reweighting}
When examining the statistical properties of PDF fits it is important to note that in the Monte Carlo approach, not only the uncertainties on PDFs are provided, but a full representation of the probability distribution.
As described in Section~\ref{sec:errors}, the integral over the PDF probability distribution is approximated by a sum over replicas,
\ba \left<f\right>(x,Q^2) &=& \int f(x,Q^2) \mathcal{P}\left(f \middle| d\right) \mathcal{D}f \nonumber \\
&\approx& \frac{1}{N_{\text{rep}}}\sum_i^{N_{\text{rep}}}f_i(x,Q^2),
\ea
where the subscript $i$ here refers to the PDF replica in the Monte Carlo ensemble. This correspondence leaves PDFs in the Monte Carlo representation open to standard statistical analysis methods. One of the most important of which is the \emph{Bayesian reweighting} technique, first proposed by Giele and Keller
alongside the original Monte Carlo procedure~\cite{Giele:1998gw} and then subsequently developed by the NNPDF collaboration~\cite{Ball:2011gg,Ball:2010gb}. The problem that reweighting seeks to address is the rapid addition of experimental data into an existing parton determination. The method is particularly useful in cases where there are no fast implementations of a calculation, and allows for the fast assessment of experimental impact upon PDFs and their uncertainties.

Given a probability distribution for PDFs, Bayes' theorem suggests that we can update the experimental information in an existing PDF fit, here denoted $\mathcal{P}(f)$ by determining the conditional probability of the PDF given the new dataset ${y}$,
\begin{equation}
  {\mathcal P}(f|y) \mathcal Df  =  \frac{{\mathcal P}(y|f)}{{\mathcal P}(y)} \, {\mathcal P}(f) \mathcal Df\, .   \label{eq:bayes2}
\end{equation}
However it was noted in Ref.~\cite{Ball:2010gb} that the probability of a PDF given the new data is not strictly what a fitting procedure would obtain. Rather the fitting procedure aims to find the probability distribution of the PDFs given some measure of fit quality to the new data, e.g $\chi^2$. Therefore to obtain a distribution statistically equivalent to a refit, one should attempt to determine
\begin{equation}
  \label{eq:bayes3}
  {\mathcal P}(f|\chi) \mathcal Df =  \frac{{\mathcal P}(\chi |f)}{{\mathcal P}(\chi)} \, {\mathcal P}(f) \mathcal Df\, ,
\end{equation}
where $\mathcal{P}(\chi)$ may be marginalised over to obtain the correct normalisation for ${\mathcal P}(f|\chi)$. Armed with such a distribution, we may then compute our predictions for a general observable given the information contained in the new dataset,
\begin{eqnarray}
\left<\mathcal{O}\right>_{\text{new}}&=&\int \mathcal{O}[f] \,
\mathcal{P}(f|\chi)\,Df\nn\\ &=&\int \mathcal{O}[f]
\,\frac{\mathcal{P}(\chi|f)}{\mathcal{P}(\chi)}
\mathcal{P}(f)\,Df,\nn
\end{eqnarray}
where $\left<\mathcal{O}\right>_{\text{new}}$ is the central value prediction for the observable $\mathcal{O}$ provided by a PDF distribution updated with the new experimental data. Given this probability distribution we can form a Monte Carlo representation in terms of PDF replicas once again,
\begin{eqnarray}
\left<\mathcal{O}\right>_{\text{new}}&=&\frac{1}{N_{\text{rep}}}\sum_{i=1}^{N_{\text{rep}}}
\frac{\mathcal{P}(\chi|f_i)}{\mathcal{P}(\chi)}\,\mathcal{O}[f_i], \nn \\
&=& \frac{1}{N_{\text{rep}}}\sum_{i=1}^{N_{\text{rep}}} w_i\,\mathcal{O}[f_i].
\label{eq:avgnewx}
\end{eqnarray}
The weights $w_i$ for the individual replicas encoding the information from the new dataset, may be obtained from the $\chi^2$ goodness-of-fit measure to the new data
\begin{equation}
w_i = \frac{\mathcal{P}(\chi|f_i)}{\mathcal{P}(\chi)} \propto \chi_i^{n-1} e^{-\half\chi_i^2}.
\label{eq:weightscsq}
\end{equation}
Where $n$ denotes the number of new datapoints. The new data may therefore be included into an existing MC parton set by the simple calculation of a $\chi^2$ for each replica in the set. In comparison to a fitting procedure where many thousands of $\chi^2$ computations are required, this procedure is extremely fast. Furthermore, as a purely statistical exercise
this PDF reweighting does not suffer from any of the inherent vagaries of a fitting procedure.

The reweighting technique does however come at a cost in that it may reduce the overall efficiency of the Monte Carlo ensemble's representation of the underlying probability distribution. As can be seen from Eqn.~\ref{eq:weightscsq}, replicas in the prior distribution which do not provide a good description
of the new experimental data and therefore have a large $\chi^2$ value are penalised by small weights. For a sufficiently large or constraining dataset this can mean that many of the replicas are effectively switched out of the distribution, leaving a smaller number of \emph{effective} replicas. The efficiency of the
representation can be quantified by the Shannon entropy, which provides the number of effective replicas as
  \be N_{\textrm{eff}} \equiv \exp \left(\frac{1}{N_{\mathrm{rep}}}\sum_{i=1}^{N_{\mathrm{rep}}}w_i\ln(N_{\mathrm{rep}}/w_i)\right). \label{eq:shannonentropy}\ee
As the constraining power of the new dataset increases, so the Shannon entropy $N_{\text{eff}}$ decreases. Consequently a larger number of replicas sampling the prior distribution are required to maintain a fixed level of ensemble accuracy. Despite this limitation, reweighting can provide an extremely useful method
for analysis of a typical experimental dataset.

\subsection{Error rescaling parameter}
A Bayesian analysis of the Monte Carlo probability representation opens up other avenues of investigation. Of particular interest is the examination of an \emph{error rescaling parameter}.
When examining the impact of an experimental measurement, we can study how the constraints are modified under a global rescaling of the experimental error, i.e the $\chi^2$ values
\be \chi^2_k \to \chi^2_{k,\alpha} = \chi^2_k/\alpha^2, \ee
where $\alpha$ is the rescaling parameter. In our reweighting exercise the weights are subsequently given by
\be w_k(\alpha) \propto (\chi^2_{k,\alpha})^{(n-1)/2}\mathrm{e}^{-\chi^2_{k,\alpha}/2}.\ee
Our Bayesian expression for the updated probability density is now also a function of the rescaling parameter $\alpha$. A further application of Bayes' theorem inverts
this relationship, and allows us to form a probability density for the rescaling parameter itself.
\be \mathcal{P}(\alpha | \chi^2)\propto \smallfrac{1}{\alpha}\sum_{k=1}^N w_k(\alpha).\ee
This probability distribution provides an estimate as to whether the experimental errors in the new dataset may have been under or overestimated, based upon agreement with the prior distribution. An experimental result where the uncertainties have accurately estimated leads to a $\mathcal{P}(\alpha)$ distribution peaked at $\alpha=1$, whereby an over(under)-estimated set of uncertainties leads to a lower(higher) peak in the distribution.  This is a particularly
useful tool for analysing experimental uncertainties, and can provide some differentiation between inconsistent and constraining data in cases where $N_{\mathrm{eff}}$ is small.

\subsection{PDF unweighting}

While the PDF reweighing approach is a powerful method for the addition of new data to an existing set, a reweighted PDF set is unsuitable for general distribution. For use in typical calculational codes, a standard interface is required through packages such as LHAPDF. Therefore the provision of a PDF ensemble with an  associated set of weights would require the retooling of codes in which a reweighted calculation is desired. To alleviate this a method was developed in order to present a reweighted distribution as a standard MC replica ensemble~\cite{Ball:2011gg}.

This is done by representing the reweighted set upon a cumulative line of weights as in Figure~\ref{fig:unweighting}. Each line segment corresponds to the weight of an individual replica. The total cumulant line therefore being normalised to $N_{\text{rep}}$, the number of replicas in the reweighted distribution. Replicas in an `unweighted' set are then chosen by distributing evenly $N_{\text{rep}}^\prime$ replicas across this cumulant line. When one of these replicas falls into the weight segment of a corresponding reweighted replica, that PDF is selected for inclusion in the unweighted set. Importantly, the same reweighted replica may be selected more than once to appear in the unweighted set.

As an example, consider the case where there are four replicas in an initial distribution, with weights $w_i = \left\{ 1, 2, 3, 4 \right\} $. The cumulant line formed by these weighted replicas is shown on the left side of Figure~\ref{fig:unweighting}. This line is subdivided into $N_{\text{rep}}^\prime + 1$ intervals. With $N_{\text{rep}}^\prime = 20$ as shown in the Figure, two unweighted replicas fall in the first weighted segment, three in the second, six in the third and nine in the fourth. Therefore the unweighted ensemble is formed by duplicating the original weighted replicas with a frequency dictated by how many unweighted replicas fall in their respective line segment.

\begin{figure}[ht!]
\centering
\includegraphics[width=1\textwidth]{4-LHCtools/figs/unweighting.pdf}
\caption[Unweighting of a Bayesian reweighted Monte Carlo PDF set]{The unweighting of a Bayesian reweighted Monte Carlo PDF set. The left hand figure shows the weight cumulant segments for the original weighted set, with four replicas of weight $w_i = \left\{ 1, 2, 3, 4 \right\} $. The line is subdivided by $N_{\text{rep}}^\prime = 20$ lines. The right hand figure illustrates the \emph{unweighted} set in this case. Here each replica in the unweighted set has equal weight, with different line strokes denoting different replicas from the weighted distribution. }
\label{fig:unweighting}
\end{figure}

The weights of the original set are therefore approximately represented as replica multiplicities in the unweighted set, with low-weight replicas selected few times (if at all), and large weight replicas selected multiple times. In this way a conventional MC ensemble can be formed with the usual LHAPDF interface, this time including duplicate replicas for those with high weights and excluding replicas with weights that fall under the unweighted set's resolution. Therefore the unweighting procedure can provide an exact representation of the reweighted ensemble in the limit $N_{\text{rep}}^\prime \to \infty$.

However in practice a number of unweighted replicas of the order of the number of effective replicas $N_{\text{eff}}$ is typically sufficient for a good level of accuracy in the reproduction.

\subsection{Reweighting validation}
The Bayesian reweighting procedure has been extensively validated by the NNPDF collaboration in a number of highly non-trivial tests of the methodology. As the method has been designed to update a prior distribution with new information analogously to the approach used in an ideal fit, the first test is to ensure that a PDF set reweighted with a new dataset is statistically equivalent to a new set refitted from scratch utilising the new data. This was first performed in~\cite{Ball:2010gb} by reweighting an NNPDF 2.0 fit which included only DIS and Drell-Yan data with information from Tevatron inclusive jet measurements. The reweighted set was compared to the full NNPDF 2.0 fit including the data. As Figure~\ref{fig:rwvalid} demonstrates, the reweighted set is able to reproduce the refitted set up to the level of statistical fluctuation.
\clearpage
\begin{figure}[ht!]
\centering
\includegraphics[width=0.45\textwidth]{4-LHCtools/figs/jets-t0-xg_Q2_2_lin.pdf}
\includegraphics[width=0.45\textwidth]{4-LHCtools/figs/jets-t0-abserror-xg_Q2_2_lin.pdf}
\caption[Validation of Bayesian reweighting by the inclusion of Tevatron jet data]{The validation of Bayesian reweighting by the inclusion of Tevatron jet data. The left figure demonstrates the prior distribution along with the reweighted and refitted distributions upon the addition of Tevatron jet data. The right plot shows the absolute error upon the PDFs for the three sets. Figures are from~\cite{Ball:2010gb}.}
\label{fig:rwvalid}
\end{figure}

The development of the unweighting method as outlined in the previous section, allowed for further tests of the reweighting method. A series of tests were carried out in order to assess the behaviour of PDFs under successive reweighting operations.

When including multiple datasets into a PDF fit via reweighting, there are three possibilities. One can reweight with the combined $\chi^2$ values for the two experiments, or reweight first with one experiment, unweight the PDF ensemble, then reweight with the second. The resulting PDFs should be reasonably independent of the method chosen, and of the order in which the successive reweighting is performed. This requirement is a stringent test of the Monte Carlo PDF representation, as it determines whether or not the ensemble truly behaves as a probability distribution. More pragmatically, the test verifies whether the loss of ensemble efficiency in one reweighting operation is not so great as to prevent a further reweighing. This investigation was carried out in Ref.~\cite{Ball:2011gg} with a DIS only prior. The E605 Drell-Yan experiment and CDF/D0 inclusive jet measurements were included into this set by reweighting. As the E605 experiment provides global fits with rather stringent constraints compared to the moderate effect of the jet data, this is a rather asymmetrical and therefore effective test.

\begin{figure}[ht]
\centering
\includegraphics[width=0.48\textwidth]{4-LHCtools/figs/e605-jets_xg_lin_zoom_rel.pdf}
\includegraphics[width=0.48\textwidth]{4-LHCtools/figs/e605-jets_xV_lin_zoom_rel.pdf}
\caption[Test of Bayesian reweighting under a successive reweighting operation]{Test of Bayesian reweighting under a successive reweighting operation. Inclusive Jet and Drell-Yan data are added as a combined dataset, and as individual reweightings separated by an unweighting operation. The resulting distributions, the gluon PDF on the left and the valence distribution on the right, show excellent agreement between the different procedures. All curves are normalised to the prior, NNPDF 2.1 DIS result. The figures are taken from~\cite{Ball:2011gg}.}
\label{fig:SRWvalid}
\end{figure}

In Figure~\ref{fig:SRWvalid} these reweighting procedures are compared for the case of the gluon and valence distributions of the NNPDF2.0 DIS only fit. It is clear that while the impact of the data upon the prior is substantial, the three reweighting methods hardly differ in their results. There is therefore a strong confirmation of the statistical properties of both the Monte Carlo representation of PDFs, and of the reweighting method.

\section{The FastKernel method}
The method of Bayesian reweighting provides an extremely fast and efficient method of including new data into a determination. However as described previously, the method is
ill-suited to the addition of a large or very constraining dataset as the required size of the prior distribution in replicas rapidly becomes unmanageable. Therefore the standard fitting methodology
remains the most important procedure in the determination of parton distributions.

The primary issue in the standard methodology upon the addition of a large LHC dataset is the computational time required to perform the theoretical predictions for experimental data.
Not only must the standard double convolution over the two parton densities be performed, but also each PDF must be evolved from some initial fitting scale to the scale of the experimental data by yet another set of convolutions. We shall first describe the methods used for fast PDF evolution, before going on to discuss the extension to the calculation of physical observables at colliders.

\subsection{Fast PDF evolution}
While there are many methods for performing the evolution of parton distributions, the technique used in NNPDF fits must be particularly efficient due to the computational complexity of the NNPDF procedure. The evolution of a flavour basis PDF of flavour $i$ from an initial scale $Q^2_0$ to a target scale $Q^2_\tau$ can be expressed as
\be f_i(x_{\alpha}, Q^2_\tau) = \sum_j^{N_{f}} \int_{x_\alpha}^1 d\xi \; \Gamma_{ij}\left(\frac{x_\alpha}{\xi},\frac{Q^2_\tau}{Q_0^2}\right) f_j(\xi, Q_0^2), \label{eq:DGLAPconv} \ee
where the $\Gamma$ are found by solution of the DGLAP equation as shown in Eqn.~\ref{eq:DGLAP}. In order to take advantage of the sparse nature of the DGLAP evolution kernels, we work in the evolution basis defined in Section~\ref{sec:DGLAP},
\be N_i(x_{\alpha}, Q^2_\tau) = \sum_j^{N_{f}} \int_{x_\alpha}^1 d\xi \; \widetilde{\Gamma}_{ij}\left(\frac{x_\alpha}{\xi},\frac{Q^2_\tau}{Q_0^2}\right) N_j(\xi, Q_0^2), \ee
where here we have introduced the notation $N$ for the evolution basis PDFs; related to the flavour basis by a simple rotation
\be f_i(x,Q^2_\tau) =  \sum_j^{N_f}R_{ij}N_j(x,Q_\tau^2). \label{eq:LHA2EVLN}\ee
Having to perform many instances of the convolution integral in Eqn.~\ref{eq:DGLAPconv} would be prohibitively expensive in most fitting applications, and so an alternative approach must be used. In the NNPDF framework this is based upon the {\tt FastKernel} interpolation method introduced in Ref.~\cite{Ball:2010de}, and shares the general approach with other interpolating methods, while maintaining a hybrid $x$ and Mellin space solution. Here we shall outline the general method used in all interpolating tools.

The first step is to expand the initial-state PDFs upon some set of interpolating basis functions $\mathcal{I}$,
\be f_i(x,Q^2_0) \approx \sum_\beta^{N_{\mathrm{fn}}} c_i^{(\beta)} \;\mathcal{I}^{(\beta)}(x),\ee
with the coefficients of this expansion calculable through the usual overlap integral
\be c^{(\beta)}_i = \int_0^1 dx\;f_i(x,Q^2_0) \; \mathcal{I}^{(\beta)}(x). \ee
Substituting the interpolated version of the initial state PDF into the evolution equation and applying the inverse transformation of Eqn.~\ref{eq:LHA2EVLN} to work in the evolution basis we obtain
\be N_i(x_{\alpha}, Q^2_\tau) = \sum_{j,k}^{N_{f}} \sum_\beta^{N_{\mathrm{fn}}} \int_{x_\alpha}^1 d\xi \; \widetilde{\Gamma}_{ij}\left(\frac{x_\alpha}{\xi},\frac{Q^2_\tau}{Q_0^2}\right) R^{-1}_{jk}\;c_k^{(\beta)} \;\mathcal{I}^{(\beta)}(x). \ee
In this expression, we can actually factorise the PDF-dependent expansion coefficients $c$ from the integral, and perform the convolution over the interpolating functions
\be N_i(x_{\alpha}, Q^2_\tau) = \sum_{k}^{N_{f}} \sum_\beta^{N_{\mathrm{fn}}} E_{ik\alpha\beta}^\tau\;c_k^{(\beta)}, \ee
where the \emph{evolution tables} $E$ are given by
\be E_{ik\alpha\beta}^\tau =  \sum_{j}^{N_{f}} \int_{x_\alpha}^1 d\xi \; \widetilde{\Gamma}_{ij}\left(\frac{x_\alpha}{\xi},\frac{Q^2_\tau}{Q_0^2}\right) R^{-1}_{jk} \;\mathcal{I}^{(\beta)}(x). \label{eq:evolfactors} \ee
While it may seem as if we have simply moved the problem from the convolution with the DGLAP kernel to the overlap integral required to compute the coefficients $c$, this can be
avoided via a careful choice in the interpolating functions. A suitable choice of interpolating function yields the following identification for the coefficients
\be c_k^{(\beta)} = f_k(x_\beta,Q^2_0),\ee
that is, the interpolants effectively pick out the value of the PDF at some point $\beta$ in an $x$-grid. Providing the grid in $\beta$ is dense enough the interpolation accuracy can still be very high.
With such a choice of functional basis, the full evolution product becomes particularly simple
\ba
f_i(x_{\alpha},Q^2_\tau) &=&  \sum_{j,k}^{n_f}\sum_{\beta}^{N_x} R_{ij}E^\tau_{\alpha\beta j k}\; f_k(x_\beta,Q^2_0), \\
&=& \sum_{k}^{n_f} \sum_\beta^{N_x} A^\tau_{\alpha\beta ik}\; f_k(x_\beta,Q^2_0). \label{eq:fastPDFfinal}
\ea
The convolution required by the initial solution to the DGLAP equation has now been reduced via interpolation methods to a simple product over a rotated evolution table $A$.

\subsection{Fast calculation of collider observables}
Similar methods to what we have discussed for fast PDF evolution have also been applied to the calculation of collider observables. For a typical observable with two partons in the initial state,
a full calculation is given by a double convolution over two parton densities,
\be  \sigma_{pp\to X} = \left( \frac{\alpha_s(Q^2)}{2\pi} \right)^{p} \int dx_1\, dx_2\,  f_i(x_1,Q^2)\; d\hat{\sigma}_{ij\to X}\; f_j(x_2,Q^2) \, . \label{eq:hadconv} \ee
The double convolution can once again be avoided by inserting interpolated versions of the PDFs, and performing the convolution over the interpolating functions.
\be  \sigma_{pp\to X} = \left( \frac{\alpha_s(Q^2)}{2\pi} \right)^{p} \sum_{\alpha,\beta}^{N_X}  f_i(x_\alpha,Q^2)\; W_{\alpha\beta,ij}\; f_j(x_\beta,Q^2) \, . \label{eq:hadconv2} \ee
The weight grid $W$ is calculated analogously to the evolution tables in Eqn.~\ref{eq:evolfactors},
\be W_{\alpha\beta,ij} =  \int dx_1\, dx_2\,  \mathcal{I}^{(\alpha)}(x_1)\; d\hat{\sigma}_{ij\to X}\; \mathcal{I}^{(\beta)}(x_2) \, . \ee
Identical methods can be used to interpolate over the hard scale $Q^2$ in multi-scale processes. These techniques are used in publicly available tools such as { \tt APPLgrid}~\cite{Carli:2010rw} and { \tt FastNLO}~\cite{Kluge:2006xs}. In the { \tt APPLgrid} framework, the full product used to calculate a hadronic observable is
\be
\label{eq:applconv}
\sigma = \sum_p \sum_{s}^{N_{\mathrm{sub}}} \sum_{\alpha,\beta}^{N_x} \sum_{\tau}^{N_{Q}}
W_{\alpha\beta\tau}^{(p)(s)} \, \left( \frac{\alpha_s\left(Q^2_{\tau}\right)}{2\pi}\right)^{p}
F^{(s)}\left(x_{\alpha}, x_{\beta},  Q^2_{\tau}\right),
\ee
where the interpolation over a grid of points in hard scale runs over the index $\tau$, and the perturbative order of the contributions is separated by the index $p$. The initial state parton combinations have been grouped into the appropriate QCD subprocesses $s$, according to a table of coefficients $C$,

\be F^{(s)}\left(x_{\alpha}, x_{\beta},  Q^2_{\tau}\right) =  \sum_{i,j}^{13} C^{(s)}_{ij}  \left( f_i(x_{\alpha},Q^2_\tau)f_j(x_{\beta},Q^2_\tau) \right).  \label{eq:APPLsubproc}\ee
The resulting product in Eqn~\ref{eq:applconv} allows for the simple variation of PDFs, strong coupling and perturbative scales in a fast calculation; the product taking typically of order milliseconds rather than the hours to days required to obtain reliable statistics in an NLO code.

Despite the dramatic speed improvement, the { \tt APPLgrid}/{ \tt FastNLO} products represent a considerable computational expense when introducing a large dataset. The NNPDF methodology in particular is extremely sensitive to the convolution speed due to the nature of the genetic algorithm minimisation, orders of magnitude more convolutions are required than in competing approaches. Therefore in order to practically include a large collider dataset into an NNPDF fit more work must be done on improving the convolution algorithm.

\subsection{Combined evolution and observable calculation}
The { \tt APPLgrid}/{ \tt FastNLO} approach maintains a great deal of flexibility, in that scale, $\alpha_S$ and PDF variations are all possible within the same framework. In a PDF fit the only requirement is an efficient variation of input parton distributions. We can therefore try to improve the efficiency of the calculation at the cost of some of the flexibility available in the fast convolution tools. The {\tt FK} procedure and toolchain has therefore been developed, implementing a combined PDF evolution and collider observable calculation.

Recalling the fast PDF evolution method in Eqn.~\ref{eq:fastPDFfinal} with the suitable grids precomputed, PDF evolution can be performed simply as
\be
f_i(x_{\alpha},Q^2_\tau) = \sum_{k}^{n_f} \sum_\beta^{N_x} A^\tau_{\alpha\beta ik}\; f_k(x_\beta,Q^2_0). \label{eq:fastPDFfinal_recalled}
\ee
The evolution of the { \tt APPLgrid} subprocess in Eqn.~\ref{eq:APPLsubproc} from an initial state distribution is therefore
\ba F^{(s)}\left(x_{\alpha}, x_{\beta},  Q^2_{\tau}\right) &=&  \sum_{i,j}^{13} \sum_{k,l}^{n_f}  \sum_{\delta,\gamma}^{N_x} C^{(s)}_{ij}  \left[  A^\tau_{\alpha\delta ik}\; f_k(x_\delta,Q^2_0) A^\tau_{\beta\gamma jl}\; f_l(x_\gamma,Q^2_0) \right]\;\;\; \\
&=&   \sum_{k,l}^{n_f}\sum_{\delta,\gamma}^{N_x} \widetilde{C}^{(s),\tau}_{kl,\alpha\beta\gamma\delta} f_k(x_\delta,Q^2_0) f_l(x_\gamma,Q^2_0), \label{eq:FK1}
\ea
where the evolved subprocess coefficients are
\be \widetilde{C}^{(s),\tau}_{kl,\alpha\beta\gamma\delta} = \sum_{i,j}^{13} C^{(s)}_{ij} A^\tau_{\alpha\delta ik} A^\tau_{\beta\gamma jl}.\ee
Substituting the expression for the subprocess in terms of initial state PDFs, Eqn.~\ref{eq:FK1}, into the { \tt APPLgrid} expression for the full convolution shown in Eqn.~\ref{eq:applconv} we obtain
\be
\sigma = \sum_p \sum_{s}^{N_{\mathrm{sub}}} \sum_{\alpha,\beta}^{N_x^\prime} \sum_{\tau}^{N_{Q}}
W_{\alpha\beta\tau}^{(p)(s)} \, \left( \frac{\alpha_s\left(Q^2_{\tau}\right)}{2\pi}\right)^{p}
 \sum_{k,l}^{n_f}\sum_{\delta,\gamma}^{N_x} \widetilde{C}^{(s),\tau}_{kl,\alpha\beta\gamma\delta} f_k(x_\delta,Q^2_0) f_l(x_\gamma,Q^2_0).
\ee
where the number of points in the { \tt APPLgrid} $x$-grid is denoted $N_x^\prime$ to indicate that the grid is different to the input parton $x$-grid which runs over $\gamma,\delta$ up to $N_x$ points. Now that the PDF evolution has been factorised into the coefficients $\widetilde{C}$, much more of this sum may now be precomputed. Specifically we are now able to sum over the indices for subprocess $s$, perturbative order $p$, hard scale $\tau$, and the { \tt APPLgrid} $x-$grids $\alpha$ and $\beta$. The resulting expression for the combined evolution and observable calculation is therefore
\be
\label{eq:FK}
\sigma = \sum_{k,l}^{n_f}\sum_{\delta,\gamma}^{N_x}
\widetilde{W}_{kl\delta\gamma} \,f_k(x_\delta,Q^2_0) f_l(x_\gamma,Q^2_0),
\ee
with the combined grid, which may be precomputed and stored, given by
\be
\label{eq:FKTable}
\widetilde{W}_{kl\delta\gamma} = \sum_p \sum_{s}^{N_{\mathrm{sub}}} \sum_{\alpha,\beta}^{N_x^\prime} \sum_{\tau}^{N_{Q}}
W_{\alpha\beta\tau}^{(p)(s)} \, \left( \frac{\alpha_s\left(Q^2_{\tau}\right)}{2\pi}\right)^{p}\widetilde{C}^{(s),\tau}_{kl,\alpha\beta\gamma\delta} .
\ee

The quantity $\widetilde{W}_{kl\delta\gamma}$ is the {\tt FK} table for the observable $\sigma$ and encodes all of the theoretical treatment of the observable. The product in Eqn.~\ref{eq:FK} is therefore completely
agnostic with regards to all theory parameters such as process, scales, perturbative order and strong coupling value. This makes the {\tt FK} table particularly simple to implement in a fitting procedure, and allows a clean separation
of theory concerns from the calculation.

The {\tt FK} convolution also benefits from requiring considerably fewer floating point operations than a typical { \tt APPLgrid} convolution. This is particularly evident when studying multi-scale processes, where the sum over the scale grid is precomputed. The product over PDF flavours is now also limited to the $n_f$, typically seven, light partons rather than the general 13 parton basis. Of course in the {\tt FK} procedure the ability to vary scales and the strong coupling with a single grid is lost, and new {\tt FK} tables $\widetilde{W}$ must be generated for different theoretical treatments.

The procedure outlined above for generating {\tt FK} tables from { \tt APPLgrid} or { \tt FastNLO} files has been implemented in a {\tt C++} framework, alongside a comprehensive toolchain for performing {\tt FK} table {\tt I/O} and optimisation. The convolution in Eqn.~\ref{eq:FK} has been implemented for a general PDF input (for example Neural Network or LHAPDF) and extensively optimised. The optimisation ensures only the relevant parton sub channels and $x$-grid entries enter the product, which is performed as a memory-aligned scalar product with the use of SSE intrinsics~\cite{SSE}.   Table~\ref{tab:FKtimings} compares the relative speed improvement compared to the { \tt APPLgrid} calculation of the basic  {\tt FK} convolution and the optimised version, using PDFs obtained through the LHAPDF library.
\begin{table}[htp]
\begin{center}
\begin{tabular}{|c|c|c|c|c|}
\hline Observable &APPLgrid & FK & optimised FK \\
\hline Total $W^+$ xsec &1.03 ms & 0.41 ms (2.5X) & 0.32 ms (3.2X) \\
\hline Jet distribution &2.45 ms & 20.1 $\mu$s (120X) & 6.57 $\mu$s (370X) \\
\hline
\end{tabular}
\caption[Comparison of { \tt APPLgrid} and {\tt FK} convolution timings]{Typical timings per observable for several convolution methods. Two observables are presented, the total cross-section for $W^+$ production and the inclusive jet \pt  distribution. Values are given per datapoint. In brackets the relative speed-up compared to the native { \tt APPLgrid} convolution is shown. For this test, the timings were calculated with a 2.9 GHz Intel Core i7 processor.}
\end{center}
\label{tab:FKtimings}
\end{table}
\clearpage
In Table~\ref{tab:FKtimings} a reasonable speed improvement is evident for the example single-scaled process of $W^+$ production, and a very significant improvement on the multi-scale jet production observable. It is important to note that these figures were obtained via convolutions with LHAPDF parton densities, and in a PDF fit a considerably greater speed advantage is gained via the {\tt FK} procedure as no additional operation is required to evolve the PDFs.

While for most applications, the original { \tt APPLgrid} convolution speed is more than sufficient, these speed improvements make the inclusion of a large LHC dataset possible, rather than prohibitively expensive in the NNPDF methodology. For example, in a typical NNPDF fit of 20,000 genetic algorithm generations, including a 100 datapoint jet dataset via the { \tt APPLgrid} interface would add several days of additional computer time to each individual replica fit. With the {\tt FK} procedure this additional cost is reduced to minutes.

The speed improvement is achieved without any loss of accuracy, as the interpolation procedure used to perform the PDF evolution is required in both the { \tt APPLgrid} and {\tt FK} convolutions. The two methods were benchmarked in Ref.~\cite{Ball:2012cx}, with the results shown in Table~\ref{tab:fkbenchatlasjet}. The relative discrepancy $\epsilon$ noted in the table is largely due to the additional interpolation in hard scale $Q^2$ from LHAPDF required in the { \tt APPLgrid} convolution that is not present in the {\tt} FK method, as evolution is performed directly to the required scale.

\begin{table}[ht!]
\begin{center}
\footnotesize
\begin{tabular}{c|c|c|c||c|c|c|}
\cline{2-7}
   & \multicolumn{3}{|c||}{${\rm W}^{+}$ distribution [pb]} & \multicolumn{3}{|c|}{${\rm W}^{-}$ distribution [pb]}  \\
\hline
\multicolumn{1}{|c||}{$| \eta_{l} |$} & {\tt FK}	 & {\tt APPLgrid} &  $\epsilon_{\rm rel}$ & {\tt FK}  &  {\tt APPLgrid} &  $\epsilon_{\rm rel}$\\
\hline
\multicolumn{1}{|c||}{0.00--0.21} & 617.287 & 617.345 & 0.01\% & 456.540 & 456.819 & 0.06\% \\
\multicolumn{1}{|c||}{0.21--0.42} & 616.988 & 617.062 & 0.01\% & 453.045 & 453.315 & 0.06\%\\
\multicolumn{1}{|c||}{0.42--0.63} & 620.237 & 620.290 & 0.01\% & 448.902 & 449.172 & 0.06\%\\
\multicolumn{1}{|c||}{0.63--0.84} & 624.192 & 624.235 & 0.01\% & 441.789 & 442.045 & 0.06\%\\
\multicolumn{1}{|c||}{0.84--1.05} & 630.235 & 630.286 & 0.01\% & 432.206 & 432.435 & 0.05\%\\
\multicolumn{1}{|c||}{1.05--1.37} & 636.835 & 636.886 & 0.01\% & 419.027 & 419.222 & 0.05\%\\
\multicolumn{1}{|c||}{1.37--1.52} & 642.800 & 642.861 & 0.01\% & 403.908 & 404.084 & 0.04\%\\
\multicolumn{1}{|c||}{1.52--1.74} & 642.499 & 642.569 & 0.01\% & 390.564 & 390.724 & 0.04\%\\
\multicolumn{1}{|c||}{1.74--1.95} & 642.351 & 642.437 & 0.01\% & 377.328 & 377.473 & 0.04\%\\
\multicolumn{1}{|c||}{1.95--2.18} & 628.592 & 628.693 & 0.02\% & 359.373 & 359.498 & 0.03\%\\
\multicolumn{1}{|c||}{2.18--2.50} & 590.961 & 591.079 & 0.02\% & 337.255 & 337.366 & 0.03\% \\
\hline
 \end{tabular}

 \vspace{2mm}

 \begin{tabular}{c|c|c|c|}
 \cline{2-4}
   & \multicolumn{3}{|c|}{Z distribution [pb]}  \\
    \hline
\multicolumn{1}{|c||}{$|y|$} & {\tt FK}	  &    {\tt APPLgrid}  &   $\epsilon_{\rm rel}$\\
\hline
\multicolumn{1}{|c||}{0.0--0.4} & 124.634 & 124.633 & 0.001\%\\
\multicolumn{1}{|c||}{0.4--0.8} & 123.478 & 123.488 & 0.01\%\\
\multicolumn{1}{|c||}{0.8--1.2} & 121.079 & 121.108 & 0.02\%\\
\multicolumn{1}{|c||}{1.2--1.6} & 118.057 & 118.108 & 0.04\%\\
\multicolumn{1}{|c||}{1.6--2.0} & 113.512 & 113.549 & 0.03\%\\
\multicolumn{1}{|c||}{2.0--2.4} & 106.552 & 106.562 & 0.01\%\\
\multicolumn{1}{|c||}{2.4--2.8} & 93.7637 & 937.838 & 0.02\%\\
\multicolumn{1}{|c||}{2.8--3.6} & 55.8421 & 558.538 & 0.02\%\\
\hline
 \end{tabular}

  \vspace{2mm}

\begin{tabular}{c|c|c|c|}
\cline{2-4}
   & \multicolumn{3}{|c|}{ATLAS 2010 jets [pb]}  \\
    \hline
\multicolumn{1}{|c||}{$p_T$ (GeV)} & {\tt FK}	  &    {\tt APPLgrid}  &    $\epsilon_{\rm rel}$ \\
\hline
\multicolumn{1}{|c||}{20--30}   & $6.1078 \times 10^6$ & $6.1090 \times 10^6$ &	0.02\% \\
\multicolumn{1}{|c||}{30--45}   & 986285 	                     & 98654      & 0.03\% \\
\multicolumn{1}{|c||}{45--60}   & 190487 	                     & 190556    & 0.04\% \\
\multicolumn{1}{|c||}{60--80}   & 48008.7 	                     & 48029.7   & 0.04\% \\
\multicolumn{1}{|c||}{80--110} & 10706.6 	                     & 10710.4   & 0.03\% \\
\multicolumn{1}{|c||}{110--160} & 1822.62 			  & 1822.87   & 0.01\% \\
\multicolumn{1}{|c||}{160--210} & 303.34 			  & 303.443   & 0.03\% \\
\multicolumn{1}{|c||}{210--260} & 76.1127 			  & 76.1338   & 0.03\% \\
\hline
 \end{tabular}

 \end{center}
\caption[Benchmark of the {\tt FK} result for datasets with different underlying processes]{ Benchmark of the {\tt FK} result for datasets with different underlying processes, all generated according to ATLAS experimental kinematics and acceptances. The { \tt APPLgrid} and {\tt FK} results are presented along with the relative discrepancy between the two. Table from \cite{Ball:2012cx}.\label{tab:fkbenchatlasjet} }
\end{table}
%%%%%%%%%%%%%%%%%%%

%

%
%\be f_i(x_{\alpha},Q^2_\tau) =  \sum_j^{13}R_{ij}N_j(x_{\alpha},Q_\tau^2) =\sum_{j}^{13} \sum_{\gamma}^{N_x}  \sum_{k}^{N_{\mathrm{pdf}}} R_{ij}E^\tau_{\alpha\gamma j k}N^0_k(x_\gamma).\ee
%\be F^{(l)}\left(x_{\alpha}, x_{\beta},  Q^2_{\tau}\right) =   \sum_{i,j}^{13} \sum_{k,l}^{N_{\mathrm{pdf}}}  C^{(l)}_{ij}A^\tau_{\alpha\gamma i k}A^\tau_{\beta\delta j l}N^0_k(x_\gamma)N^0_l(x_\delta),
%\quad\quad A^\tau_{\alpha\gamma i k} =  \sum_j^{13} R_{ij}E^\tau_{\alpha\gamma j k}.\ee
%\be \sigma =  \sum_{i,j}^{N_{\mathrm{pdf}}} \sum_{\alpha,\beta}^{N_x} \widetilde{W}_{\alpha\beta i j} N_i^0(x_\alpha)N_j^0(x_\beta),\ee
%\be \widetilde{W}_{\alpha\beta i j} = \sum_p \sum_{l=0}^{N_{\mathrm{sub}}}\sum_{k,l}^{13} \sum_{\gamma,\delta}^{N_x} \sum_{\tau}^{N_{Q}}
%W_{\gamma\delta\tau}^{(p)(l)} \, \left( \frac{\alpha_s\left(Q^2_{\tau}\right)}{2\pi}\right)^{p} C^{(l)}_{kl}A^\tau_{\gamma\alpha k i}A^\tau_{\delta\beta l j}, \ee
%

\section{Interpolating tools for automated NLO}
Tools such as { \tt FastNLO}/{ \tt APPLgrid} and their extension for fast PDF fitting in the $\tt FK$ method, are invaluable in the analysis of collider data. Their usefulness is not limited to applications such as fitting, but can also be used to perform thorough QCD analysis with rigorous theory uncertainty estimation in situations where obtaining sufficient statistics with an NLO code or event generator would be extremely expensive computationally. Despite this, at the outset of LHC data taking the amount of codes interfaced to such interpolating tools was extremely limited. Additionally the need for separate interfaces to existing codes meant a great deal of duplication in terms of analysis tools and software.  The { \tt APPLgrid} group provided a direct interface to the NLO codes {\tt MCFM}~\cite{Campbell:2000bg} and {\tt nlojet++}~\cite{Nagy:2001fj,Nagy:2003tz}. { \tt FastNLO} provided a set of precomputed scenarios generated through a private interface to NLO codes. More recently, a public toolkit was released to allow for the interfacing of { \tt FastNLO} to external calculations.

A conspicuous absence was an interface to tools providing automated NLO calculations via computer algebra suitable one-loop methods\cite{Anastasiou:2006jv,Berger:2008sj,Denner:2005nn,Ellis:2007br,Giele:2008ve,Ossola:2006us} and their implementations in parton level Monte Carlo codes such as {\tt MadGraph}~\cite{Alwall:2011uj}, {\tt HELAC}~\cite{Bevilacqua:2011xh} and {\tt SHERPA}~\cite{Gleisberg:2003xi,Gleisberg:2008ta}. In this section we shall discuss the implementation of a fast interface to such codes, the {\tt MCgrid}~\cite{DelDebbio:2013kxa} package; developed with the aid of funding from the MCnet initial training network.

\subsection{Reweighting Monte Carlo calculations}
Recalling Eqn.~\ref{eq:hadconv}, a hadronic observable calculation proceeds via
\be  \sigma_{pp\to X} = \sum_p \left( \frac{\alpha_s(Q^2)}{2\pi} \right)^{p} \int dx_1\, dx_2\,  F_l(x_1,x_2,Q^2)\; d\hat{\sigma}_{l\to X}^{(p)}\; , \label{eq:doubleconv2} \ee
where the initial state PDFs have been grouped according to Eqn.~\ref{eq:APPLsubproc} and the sum over subprocesses is implicit. In an event generator this integral is performed via Monte Carlo integration. At leading order this is a relatively straightforward procedure,
\begin{align}
  \label{eq:basicMC}
  \sigma_{pp\to X}^{\text{LO}} &= \sum_{e=1} \tilde{w}_e(k_e) = \sum_{e=1}^{N_\evt} \left(  \frac{\alpha_s\left(k_e\right)}{2\pi} \right)^{p_{\text{LO}}} w_e(k_e) F_{l_e}(k_e)\, ,
\end{align}
where $\tilde{w}$ is the full event weight and the $w$ are the matrix element weights generated via importance sampling of the integrand of Eqn.~\ref{eq:doubleconv2}. The $F_{l_e}$ refer to the parton density of the event's subprocess $l_e$. Each event is generated according to a set of kinematics
\begin{align}
  k_e &= \left\{p_1, ..., p_n,\,x_1,\,x_2,\,\frac{\mu_F^2}{Q^2},\,\frac{\mu_R^2}{Q^2} \right\}.
\end{align}

As a full re-run of the event generator for every parameter variation is extremely expensive, the variation is typically performed via an event-by-event reweighting procedure. The full set of events is stored
in a common format such as { \tt HepMC}~\cite{Dobbs:2001ck} and re-processed by dividing out the appropriate factors of the old PDFs and $\alpha_S$ and multiplying in the desired new values.

\be \tilde{w}_e (k_e)\to \left(  \frac{\alpha_s^\prime\left(k_e\right)}{\alpha_s\left(k_e\right) } \right)^{p_{\text{LO}}}  \frac{F^\prime_{l_e}(k_e)}{F_{l_e}(k_e)}\; \tilde{w}_e(k_e), \ee
where the primed quantities denote the new, reweighted strong coupling and PDF choices. Having the full generated event sample stored also has the advantage of being able to rerun analysis software with varying parameters/selections without the need to rerun the potentially expensive event generation.

The reweighting situation in an NLO calculation is considerably more complicated. In order to be able to solve the integral numerically, a divergence-subtraction scheme e.g Catani-Seymour~\cite{Catani:1996vz}
or Frixione-Kunst-Signer (FKS)~\cite{Frixione:1995ms,Frixione:1997np} must be employed. These subtraction algorithms separate the calculation into distinct sections which are to be numerically evaluated individually. Here we shall discuss the implementation in terms of a Catani-Seymour dipole scheme. The four contributions to the total NLO cross section are

\begin{equation}
  \label{eq:NLOxsect}
  \sigma_{pp\to X}^{\text{NLO}}= \int d\hat{\sigma}^\mathrm{B}
  + \int d\hat{\sigma}^\mathrm{V}
  + \int d\hat{\sigma}^\mathrm{I}
  + \int d\hat{\sigma}^\mathrm{RS}
  \, .
\end{equation}
The terms $B$, $V$, $I$ and $RS$ refer to the Born (B), Virtual (V), Integrated subtraction (I) and Real Subtracted (RS) cross section elements respectively. Neglecting terms used in the variation of perturbative scales, the $B$, $V$ and $RS$ terms may be integrated via a Monte Carlo procedure equivalently to Eqn.~\ref{eq:basicMC}. The integrated dipole term however has a rather more complicated dependance upon the initial state PDFs, originating as it does through the splitting of an initial state parton. The Monte Carlo solution to the $I$ integral is given by

\def\varref#1{{\tt #1}}
\begin{eqnarray}
 \int d\hat{\sigma}^\mathrm{I}  &=&  \sum_{e=1}^{N_\evt}   \left(  \frac{\alpha_s(\mu_R^2)}{2\pi} \right)^{p_{\text{NLO}}} \Bigg\{  F_{l_e}(x_1,x_2,\mu^2_F)\,  w^{(0)}_e
\nonumber\\
&&+\sum_{s}^{N_{\text{sub}}} F_s(x_1/x'_1, x_2, \mu^2_F)\, \tilde{w}^{(1)}_{e,s} \label{eq:NTupleBreakdown} \\
&&+\sum_{s}^{N_{\text{sub}}} F_s(x_1, x_2/x'_2, \mu^2_F)\, \tilde{w}^{(2)}_{e,s}
 \Bigg\},
  \, \nonumber
\end{eqnarray}
in which the weight $w^{(0)}$ arises through the usual Born-like PDF dependance, and the weights $w^{(1/2)}$ arise from integration over parton-$x$ from the first or second parton in the initial state splitting. To reweight such events these weights must therefore be properly distinguished
in the event record. While is is not the case in the standard {\tt HepMC} layout, a format based upon {\tt ROOT NTuples} was designed by the {\tt BlackHat-Sherpa} group for the reweighting of NLO event weights~\cite{Bern:2013zja}. In the {\tt BlackHat NTuple} format the weights that must be distinguished for the accurate treatment of scale variations are also stored.  \\
While the event reweighting approach is considerably faster than an entire rerun of the Monte Carlo, a reweight of a full event sample can still take a considerable amount of computer time. The key issue being that the statistical accuracy of the calculation is limited by the number of events in the sample, and therefore for a more accurate calculation, more computational expense is incurred. This dependence on the event loop is not removed by the event reweighting procedure. The dependance can however be removed by applying the interpolation methods of the previous section, by providing an interface for event generators to interpolating packages such as { \tt APPLgrid}.
\subsection{An interpolation interface for automated NLO}
\label{sec:interp}
The {\tt MCgrid} project began as a direct interface for the {\tt SHERPA} event generator framework to the {\tt APPLgrid} interpolation package. The development of a new interface between an event generator and the {\tt APPLgrid} framework in principle requires the implementation of an analysis suite to provide the categorisation of event final states into appropriate observable bins. However the {\tt MCgrid} interface is built upon standard analysis tools and formats to provide a more general interface between standards-compliant Catani-Seymour event generators to the {\tt APPLgrid} package.

{\tt MCgrid} is written as a set of additional tools for the {\tt Rivet} MC analysis system. The {\tt Rivet} system implements a wide range of experimental analysis tools and provides the flexibility for the user to define their own selection criteria and processing tools to operate on an event final state. Writing the {\tt APPLgrid} interface as a {\tt Rivet} extension therefore removes the need to implement a separate toolchain, and allows a degree of generator agnosticism. As {\tt} Rivet operated upon events in the standard {\tt HepMC} format, any generator equipped to output events in this format may potentially be interfaced to {\tt APPLgrid} through {\tt MCgrid}.

The interface requires additional information over the standard data available in {\tt HepMC}, as the information on the weight breakdown as per Eqn.~\ref{eq:NTupleBreakdown} must be available. However this can be straightforwardly appended in the {\tt HepMC} user defined weights fields. The interface then provides the correct handling of initial state parton mappings from the PDF basis used in the Catani-Seymour process to the APPLgrid flavour basis.

With the appropriate mapping to initial state parton flavours performed, the weights must be converted to the appropriate subprocess basis. The minimal initial state PDF basis can be automatically determined by a set of packaged scripts.
General purpose Monte Carlo codes such as {\tt SHERPA} will typically generate events will the full initial parton flavours explicit rather then generating events based upon QCD subprocesses. Weights originating from different flavour basis channels are generated via importance sampling of the distribution in order to ensure an efficient description of the most important channels.  Accordingly there is a selection weight present in each event weight, given by
\begin{equation}
w_e(i,j,k_e) =\mathcal{N}_{ij}\;d \hat{\sigma}_{l_e \to X}(k_e) \Pi_{\text{ps}}(k_e) \Theta (k_e-k_{\text{cuts}}), \label{eq:mcweight}
\end{equation}
where the factor $\mathcal{N}_{ij}$ is approximately given by
\begin{equation}
\mathcal{N}_{ij} \sim \frac{N_{\text{tot}}}{N_{ij}}, \label{eq:selectionweight}
\end{equation}
where $N_{\text{tot}}$ denotes the total number of events in the sample, and $N_{ij}$ is the number of events initiated by partons of flavour $i$ and $j$. In Eqn.~\ref{eq:mcweight} we use $\Pi$ to represent the phase space weight associated with the kinematics $k_e$, and the $\Theta$ as a step function implementing the desired kinematic cuts in the analysis.
The selection weights $\mathcal{N}$ must be converted into the appropriate subprocess selection weight to prevent the statistical uncertainty in poorly-sampled distributions from overwhelming the subprocess. {\tt MCgrid} monitors the relative population of the channels and subprocesses in order to provide a statistically sound subprocess combination. The selection weight in Eqn.~\ref{eq:selectionweight} must be converted into the appropriate subprocess selection weight as
\be \mathcal{N}_{ij} \to \mathcal{N}_{l} = \frac{N_{\mathrm{tot}}}{N_{l}}, \ee
where $N_{l}$ is the number of events falling into the initial state subprocess $l$. Converting the selection weights to the appropriate subprocess selection weight is therefore a matter of multiplying each event weight by a factor $N_{ij}/N_{l}$.

In this way the fully exclusive predictions given in a typical Monte Carlo event generator may be effectively converted into the relevant subprocess basis. With this accomplished, and the correct weight conversion performed according to the exact PDF dependance of the Catani-Seymour counterterms, the weights may be filled directly into an APPLgrid type weight grid. Figure~\ref{fig:MCgridreps} demonstrates the application of the {\tt MCgrid} package when used in conjunction with {\tt Sherpa} and {\tt BlackHat} as a one-loop generator. The tools are applied to the test cases of Drell-Yan and inclusive jet production, with the resulting { \tt APPLgrid} applied to the estimation of scale and $\alpha_S$ uncertainties alongside standard PDF error, requiring a very large number of replicas.

The {\tt MCgrid} project is publicly available\footnote{The software is available at \emph{http://mcgrid.hepforge.org}.}, and allows for the first time calculations from automated NLO event generators to be interfaced to interpolation tools, for potential application in PDF fits, or indeed fast parameter variation studies in phenomenological applications.

\begin{figure}[h!]
\centering
\includegraphics[width=0.48\textwidth]{4-LHCtools/figs/100MDYReplicas_Scales.pdf}
\includegraphics[width=0.48\textwidth]{4-LHCtools/figs/JetsAlphas_0.pdf}
\caption[An example of the output of the {\tt MCgrid} package]{An example of the output of the {\tt MCgrid} package. A $Z$ boson rapidity distribution plot is shown on the left, with scale error estimation. The plot on the right demonstrates the grids applied to inclusive jet data, with $\alpha_S$ error estimation. Both plots are normalised to their central values, to demonstrate the level of uncertainty.}
\label{fig:MCgridreps}
\end{figure}