\section{A Universe of Inductive Datatypes}
\label{sec:universe-desc}

\begin{wstructure}
<- Why starting with simple datatypes
    <- For pedagogical purposes
        <- datatypes as we know them every day
        /> Target dependent types
    -> Cut down version of Induction Recursion
        -> Presentation evolves independently as we go to dependent types
\end{wstructure}

In this section, we describe an implementation of inductive types, as
we know them from ML-like languages. By working with familiar datatypes,
we hope to focus on the delivery mechanism, warming up gently to the
indexed datatypes we really want.  % Our encoding is inspired by
Dybjer and Setzer's closed formulation of
induction-recursion~\cite{dybjer:axiom-ir}, but without the
`-recursion'.  An impredicative Church-style encoding of datatypes is
not adequate for dependently typed programming, as although such
encodings present data as non-dependent eliminators, they do not
support dependent \emph{induction}~\cite{geuvers:induction-not-derivable}.
Whilst the \(\LAMBINDER\)-calculus captures all that data can \emph{do},
it cannot ultimately delimit all that data can \emph{be}.

\subsection{The power of $\Sigma$}

\begin{wstructure}
<- The duality of Sigma
    <- Sigma generalises sum over arbitrary arities
        -> \Sigma A B == \Sigma_{x : A} B x
    <- Sigma generalises product to have a dependant second component
        -> \Sigma A B == (x : A) \times (B x)
\end{wstructure}

In dependently typed languages, $\Sigma$-types can be interpreted as
two different generalisations. This duality is reflected in the
notation we can find in the literature. The notation
$\blue{\Sigma}_{\x : \M{A}} (\M{B}\: \x)$ stresses that
$\Sigma$-types are `dependent sums', generalising sums over
arbitrary arities, where simply typed languages have finite sums.

On the other hand, our choice, $\SIGMA{\x}{\M{A}}
(\M{B}\:\x)$, emphasises that $\Sigma$-types generalise products,
with the type of the second component depending on the value of the
first. Simply typed languages do not express such relative validity.

\begin{wstructure}
<- datatypes in the simply-typed world
    -> "sums-of-product"
        <- Sum of constructors
        <- Product of arguments
<- datatypes in the dependently-typed world
    -> "sigmas-of-sigmas"
    /> Need ability to manipulate these sigmas
        -> Define a Code for datatypes
        -> Together with a sigma-based Interpretation
\end{wstructure}

In ML-like languages, datatypes are presented as a
\emph{sum-of-products}. A datatype is defined by a finite sum of
constructors, each carrying a product of arguments. To embrace
these datatypes, we have to capture this grammar.
With dependent types, the notion of sum-of-products translates into
\emph{sigmas-of-sigmas}.

\subsection{The universe of descriptions}
\label{sec:desc-universe}

\begin{wstructure}
<- Introduction to Universe construction
    <- Define a Code
        -> Name objects
    <- Define an Interpretation of codes into the type theory
        -> Give a semantics to objects
    -> Ability to manipulate code
    -> Ability to compute with these objects
\end{wstructure}

While sigmas-of-sigmas can give a \emph{semantics} for the
sum-of-products structure in each node of the tree-like values in a
datatype, we need to account for the recursive structure which
ties these nodes together. We do this by
constructing a \emph{universe}~\cite{martin-lof:itt}. Universes
are ubiquitous in dependently typed
programming~\cite{benke:universe-generic-prog, oury:power-of-pi},
but here we take them as the foundation of our notion
of datatypes.

%The key idea behind universe construction is our ability to make names
%by defining new types. These names are called \emph{codes}. By
%defining a set of codes, we define the syntax of a language. To give
%it a semantics, we \emph{interpret} these codes back into the type
%theory. Hence, codes act as labels, while the type theory provides the
%computational machinery. Codes being mere labels, we can inspect them,
%hence regaining structural information. We can also compute over them:
%deriving new codes from others, or even functions on them.

\begin{wstructure}
<- Justification of the code
    <- [both figures]: cannot be read separately
    <- Mimic the standard grammar of datatypes description
        -> Just as we already know it
        <- '\Sigma for making sigmas-of-sigmas
        <- 'indx for exhibiting the functoriality
            -> For recursive arguments
        <- '1 for end of description
\end{wstructure}

To add inductive types to our type theory, we build a universe of
datatype \emph{descriptions} by implementing the signature presented
in Figure~\ref{fig:desc_universe}, with codes mimicking the grammar of
datatype declarations. We can read a description $\V{D}:\Desc^{n}$ as
a `pattern functor' on $\Set^{n}$, with $\SYMBdescop{\V{D}}$ its
action on an object, \(\V{X}\), soon to be instantiated
recursively. The superscripts indicate the $\Set$-levels at which we
\emph{expect} these objects in a stratified system. This is but an
informal notation, to give a flavour of the stratified
presentation. Note that the functors so described are \emph{strictly
  positive}, by construction.

Descriptions are sequential structures ending in $\DUnit$,
indicating the empty tuple. To build sigmas-of-sigmas, we provide a
$\SYMBDSigma$ code, interpreted as a $\Sigma$-type. To request a
recursive component, we have $\DIndx{\V{D}}$, where \(\V{D}\)
describes the rest of the node. These codes give us
sigmas-of-sigmas with recursive places. An equivalent, more algebraic presentation
could be given, as illustrated in Section~\ref{sec:indexing-desc}.

\begin{figure}
\[\stk{
\begin{array}{@{}ll}
\Desc^n
    &: \Set^{n+1}\\
\DUnit
    &: \Desc^{n}\\
\DSigma{(\Bhab{\V{S}}{\Set^{n}})}{(\Bhab{\V{D}}{\V{S} \To \Desc^{n}})}
    &: \Desc^{n}\\
\DIndx{(\Bhab{\V{D}}{\Desc^{n}})}
    &: \Desc^{n} \\
\end{array}
\smallskip\\
\begin{array}{l@{\V{X}\:\mapsto\:}l}
\multicolumn{2}{l}{\descop{\_\:}{} : \Desc^{n} \To \Set^{n} \To \Set^{n}} \\
 \descop{\DUnit}{} &  \Unit \\
 \descop{\DSigma{\V{S}}{\V{D}}}{} &
     \SIGMAS{\Bhab{\V{s}}{\V{S}}}{\descop{\V{D}\,\V{s}}{\!\V{X}}}  \\
\descop{\DIndx{\V{D}}}{}  &  \TIMES{\V{X}}{\descop{\V{D}}{\V{X}}}
\end{array}
}\]

\caption{Universe of Descriptions}
\label{fig:desc_universe}

\end{figure}

We admit to being a little coy,
writing of `implementing a signature' without clarifying how. A viable
approach would simply be to extend the theory with constants for the
constructors and an operator for \(\descop{\V{D}}\). In
Section~\ref{sec:desc-levitate}, you will see what we do instead.
Meanwhile, let us gain some intuition by developing examples.

\subsection{Examples}
\label{sec:desc-examples}

\begin{wstructure}
<- Nat
    <- Sum of zero, suc
    <- zero case: done
    <- suc case: leave open and done
    -> NatD Z = 1 + Z
\end{wstructure}

We begin with the natural numbers, now working in the high-level
expression language of Section~\ref{sec:type-propagation}, exploiting
type propagation.
%
\[\stk{
\NatD : \Desc^n \\
\NatD \mapsto \DSigma{\EnumT{\sqr{\NatZero\: \SYMBNatSuc }}}
                     {\sqr{ \DUnit \quad (\DIndx{\DUnit}) }}
}\]
%
Let us explain its construction. First, we use $\SYMBDSigma$ to
give a choice between the $\NatZero$ and $\SYMBNatSuc$ constructors.
What follows depends on this choice, so we write the function
computing the rest of the description in tuple notation.  In the
$\NatZero$ case, we reach the end of the description. In the
$\SYMBNatSuc$ case, we attach one recursive argument and close the
description. Translating the \(\Sigma\) to a binary sum, we have
effectively described the functor:
%
\[    \NatD\: \V{Z} \mapsto \Unit \mathop{\D{+}} \V{Z}    \]
Correspondingly, we can see the injections to the sum:
\[
\sqr{\NatZero}:\descop{\NatD}Z \qquad
\sqr{\NatSuc{(\Bhab{z}{Z})}}:\descop{\NatD}Z
\]

\begin{wstructure}
<- List
    <- Sum of nil, cons
    <- nil case: done
    <- cons case: product of X with leave open and done
    -> ListD X Z = 1 + X * Z
\end{wstructure}

The pattern functor
for lists needs but a small change:
%
\[\stk{
\ListD : \Set^n \To \Desc^n \\
\ListD \: \V{X} \mapsto
 \DSigma{\EnumT{\sqr{ \ListNil\: \SYMBListCons }}}
         {\sqr{ \DUnit \quad (\DSigma{\V{X}}{\LAM{\_} \DIndx{\DUnit}}) }}
}\]
%
The $\SYMBNatSuc$ constructor becomes
$\SYMBListCons$, taking an $\V{X}$ followed by a
recursive argument. This code describes the following functor:
%
\[    \ListD\: \V{X}\: \V{Z} \mapsto \Unit \mathop{\D{+}} \V{X} \D{\ensuremath{\times}} \V{Z}     \]

\begin{wstructure}
<- Tree
    <- sum of leaf, node
    <- leaf case: done
    <- node case: product of X with two leave open and done
    -> TreeD X Z = 1 + X * Z * Z
\end{wstructure}

Of course, we are not limited to one recursive argument. Here are
the node-labelled binary trees:
%
\[\stk{
\TreeD : \Set^n \To \Desc^n \\
\begin{array}{@{}l@{}l}
\TreeD \: \V{X} \mapsto
    \DSigma{ & \EnumT{\sqr{ \TreeLeaf\: \TreeNode }} \\}
           { & \sqr{ \DUnit \quad
                     (\DIndx{(\DSigma{\V{X}}{\LAM{\_}\DIndx{\DUnit}})})} }
\end{array}
}\]
%
Again, we are one evolutionary step away from $\ListD$. However,
instead of a single call to the induction code, we add another. The
interpretation of this code corresponds to the following functor:
%
\[    \TreeD\: \V{X}\: \V{Z} \mapsto \Unit \mathop{\D{+}}
          \V{Z} \Prod \V{X}  \Prod \V{Z}     \]

\begin{wstructure}
<- Tagged description
    <- Form TDesc = List (UId x Desc) [equation]
    <- Follow usual sums-of-product presentation of datatype
        <- Finite set of constructors
        <- Then whatever you want
    -> Any Desc datatype can be turned into this form
        -> No loss of expressive power
        /> Guarantee a ``constructor form''
\end{wstructure}

From the examples above, we observe that datatypes are defined by a
$\SYMBDSigma$ whose first argument enumerates the constructors. We
call codes fitting this pattern \emph{tagged} descriptions. Again,
this is a clear reminder of the sum-of-products style. Any
description can be forced into this style with a singleton constructor
set. We characterise tagged descriptions thus:
\[\stk{
 \TagDesc^n : \Set^{n+1} \\
 \TagDesc^n \mapsto \SIGMA{\V{E}}{\EnumU} (\spi{\V{E}}{\LAM{\_} \Desc^n})
\smallskip\\
\SYMBtoDesc : \TagDesc^n \To \Desc^n \\
\SYMBtoDesc \mapsto
\spl{\LAM{\V{E}}\LAM{\V{D}}
\DSigma{\EnumT{\V{E}}}{(\F{switch}\:\V{E}\:(\LAM{\_}\Desc^n)\:\V{D})}}
}\]
It is not such a stretch to expect that the familiar datatype
declaration might desugar to the \emph{definitions} of a tagged description.

\begin{wstructure}
<- Fictive object [figure 'data Desc']
    -> Must be read as a type signature
    -> See further for its actual implementation
        <- Subject to our levitation exercise
\end{wstructure}

\subsection{The least fixpoint}
\label{sec:desc-fix-point}

\begin{wstructure}
<- Build the fixpoint of functors
    <- See examples: need to build their initial algebra
    -> Extend the type theory with Mu/Con [figure]
        <- Straightforward definition of a fixpoint
            <- Interpret D with (Mu D) as sub-objects
\end{wstructure}

\newcommand{\mude}[1]{\D{\(\mu^{\!+}\)}\:#1}

So far, we have built pattern functors with our \(\Desc\) universe.
Being polynomial functors, they all admit a least fixpoint, which we
now construct by \emph{tying the knot}: the element type abstracted
by the functor is now instantiated recursively:
%
\[
\Rule{\Gamma \vdash \Bhab{\M{D}}{\Desc^n}}
     {\Gamma \vdash \Bhab{\Mu{\M{D}}}{\Set^n}} \qquad
\Rule{\Gamma \vdash \Bhab{\M{D}}{\Desc^n} \quad
      \Gamma \vdash \Bhab{\M{d}}{\descop{\M{D}}{(\Mu{\M{D}})}}}
     {\Gamma \vdash \Bhab{\Con{\M{d}}}{\Mu{\M{D}}}}
\]
Tagged descriptions are very common, so we abbreviate:
\[
  \mude{}:\TagDesc^n \To \Set^n \qquad \mude{\V{T}}\mapsto\Mu{(\toDesc{\V{T}})}
\]

\begin{wstructure}
<- Elimination on Mu
    <- We are used to foldD : \forall X. (desc D X -> X) -> mu D -> X
        /> Not dependent
        -> Cannot express some (which one again?) properties
    -> Develop a dependent induction
        <- Everywhere/All
        <- Induction
    -> *Generic*
    ???
\end{wstructure}

We can now build datatypes and their elements, e.g.:
\[\stk{
\Nat \mapsto \Bhab{\mude{\pair{\sqr{\NatZero\: \SYMBNatSuc }}
                               {\sqr{\DUnit \quad (\DIndx{\DUnit})}}{}}}
                  {\Set^n}
\\
\Con{\sqr{\NatZero}}:\Nat \qquad
\Con{\sqr{\NatSuc{(\Bhab{n}{\Nat})}}}:\Nat
}\]

But how shall we compute with our data?  We should expect an
elimination principle. Following a categorical intuition, we might
provide the `fold', or `iterator', or `catamorphism':
%
\[
\F{cata} : \PITEL{\V{D}}{\Desc^n}
           \PI{\V{T}}{\Set^n}
           (\descop{\V{D}}{\V{T}} \To \V{T}) \To
           \Mu{\V{D}} \To \V{T}
\]

\newcommand{\SYMBind}{\F{ind}}

However, iteration is inadequate for \emph{dependent} computation.
We need \emph{induction} to write functions whose type depends on inductive
data. Following \citet{benke:universe-generic-prog}, we adopt the following:
%
\[\stk{
\F{ind} : \stk{ \PITEL{\V{D}}{\Desc^n}
                \PI{\V{P}}{\Mu{\V{D}} \To \Set^k}         \\
               (\PI{\V{d}}{\descop{\V{D}}{(\Mu{\V{D}})}}
                \All{\V{D}}{(\Mu{\V{D}})}{\V{P}}{\V{d}} \To \V{P} (\Con{\V{d}})) \To \\
               \PI{\V{x}}{\Mu{\V{D}}} \V{P} \V{x}
} \\
\F{ind}\, \V{D}\, \V{P}\, \V{m}\, (\Con{\V{d}}) \mapsto
    \V{m}\, \V{d}\, (\all{\V{D}}{(\Mu{\V{D}})}{\V{P}}
                           {(\F{ind}\, \V{D}\, \V{P}\, \V{m})}
                           {\V{d}})
}\]
%
Here, $\All{\V{D}}{\V{X}}{\V{P}}{\V{d}}$ states that
$\Bhab{\V{P}}{\V{X} \To \Set^k}$ holds for every subobject $\V{x}:\V{X}$
in \(\V{D}\), and \(\all{\V{D}}{\V{X}}{\V{P}}{\V{p}}{\V{d}}\) is a
`dependent map', applying some
\(\Bhab{\V{p}}{\PIS{\Bhab{\x}{\V{X}}}\V{P}\,\V{x}}\) to each \(\x\)
contained in \(\V{d}\). The definition (including an extra case,
introduced soon) is in
Figure~\ref{fig:all-predicates}.\footnote{To pass the termination
  checker, we had to inline the definition of $\SYMBall$ into
  $\SYMBind$ in our Agda model. A simulation argument shows that the
  definition presented here terminates if the inlined version
  does. Hence, although not directly structural, this definition is
  indeed terminating.} So, $\SYMBind$ is our first
operation generic over descriptions, albeit hardwired.  Any
datatype we define comes with induction.

Note that the very same functors \(\SYMBdescop{\M{D}}\) also admit greatest
fixpoints: we have indeed implemented coinductive types this way, but
that is another story.

\begin{figure*}

\[
\begin{array}{ll}
%%
\stk{
\begin{array}{@{}ll}
\SYMBAll : & \PITEL{\V{D}}{\Desc^n}
             \PITEL{\V{X}}{\Set^n}
             \PITEL{\V{P}}{\V{X} \To \Set^k} \\
           & \PI{\V{xs}}{\descop{\V{D}}{\V{X}}}
             \Set^k
\end{array} \\
\begin{array}{@{}l@{}l@{\:\mapsto\:\:}l}
\All{\DUnit}{& \V{X}}{\V{P}}{\Void} &
    \Unit \\
\All{(\DSigma{\V{S}}{\V{D}})}{& \V{X}}{\V{P}}{\pair{\V{s}}{\V{d}}{}} &
    \All{(\V{D}\: \V{s})}{\V{X}}{\V{P}}{\V{d}} \\
\All{(\DIndx{\V{D}})}{& \V{X}}{\V{P}}{\pair{\V{x}}{\V{d}}{}} &
    \TIMES{\V{P}\: \V{x}}{\All{\V{D}}{\V{X}}{\V{P}}{\V{d}}} \\
\All{(\DHindx{\V{H}}{\V{D}})}{& \V{X}}{\V{P}}{\pair{\V{f}}{\V{d}}{}} &
    \TIMES{(\PI{\V{h}}{\V{H}} \V{P}\: (\V{f}\: \V{h}))}
          {\All{\V{D}}{\V{X}}{\V{P}}{\V{d}}}
\end{array}
}
&
%%
\stk{
\begin{array}{@{}ll}
\SYMBall : & \PITEL{\V{D}}{\Desc^n}
             \PITEL{\V{X}}{\Set^n}
             \PITEL{\V{P}}{\V{X} \To \Set^k} \\
           & \PITEL{\V{p}}{\PI{\V{x}}{\V{X}} \V{P}\: \V{x}}
             \PI{\V{xs}}{\descop{\V{D}}{\V{X}}}
             \All{\V{D}}{\V{X}}{\V{P}}{\V{xs}}
\end{array} \\
\begin{array}{@{}l@{}l@{\:\mapsto\:\:}l}
\all{\DUnit}{& \V{X}}{\V{P}}{\V{p}}{\Void} &
    \Void \\
\all{(\DSigma{\V{S}}{\V{D}})}{& \V{X}}{\V{P}}{\V{p}}{\pair{\V{s}}{\V{d}}{}} &
    \all{(\V{D}\: \V{s})}{\V{X}}{\V{P}}{\V{p}}{\V{d}} \\
\all{(\DIndx{\V{D}})}{& \V{X}}{\V{P}}{\V{p}}{\pair{\V{x}}{\V{d}}{}} &
    \pair{\V{p}\: \V{x}}
         {\all{\V{D}}{\V{X}}{\V{P}}{\V{p}}{\V{d}}}{} \\
\all{(\DHindx{\V{H}}{\V{D}})}{& \V{X}}{\V{P}}{\V{p}}{\pair{\V{f}}{\V{d}}{}} &
    \pair{\LAM{\V{h}} \V{p}\: (\V{f}\: \V{h})}
         {\all{\V{D}}{\V{X}}{\V{P}}{\V{p}}{\V{d}}}{}
\end{array}
\end{array}
}
\]

\caption{Defining and collecting inductive hypotheses}
\label{fig:all-predicates}

\end{figure*}

\subsection{Extending type propagation}

\begin{wstructure}
<- Extending type propagation
    <- datatype declaration turns into definitions
        -> Straightforward translation to Desc
        -> Creation of a variable referring to the structure
    <- Labelled Mu
        /> Just mention the possibility of labelling, no details required
        -> For the user, objects have names rather than Mu of codes
    <- Push Mu to an applied name [figure]
        -> Direct integration into the type propagation machinery
    -> Coded presentation is practical
        <- The user never see a code
\end{wstructure}

We now have low level machinery to build and manipulate inductive
types. Let us apply cosmetic surgery
to reduce the syntactic overhead. We extend type checking of
expressions:
%
\[
\Rule{\Gamma\Vdash\propag{\push{\etag{c}}{\EnumT{\M{E}}}}{n} \quad
\Gamma\Vdash
\propag{
  \push{\sqr{\vec{t}}}
   {\descop{\M{D}\: n}{(\Mu{(\DSigma{\EnumT{\M{E}}}{\M{D}})})}}}
        {\M{t'}}}
     {\Gamma\Vdash\propag{\push{\etag{c}\: \vec{\M{t}}}
                   {\Mu{(\DSigma{\EnumT{\M{E}}}{\M{D}})}}}
             {\Con{\pair{n}{\M{t'}}{}}}}
\]
%
Here $\etag{c}\:\vec{t}$ denotes a tag `applied' to a sequence
of arguments, and \(\sqr{\vec{t}}\) that sequence's repackaging as
a right-nested tuple. Now we can just write data directly.
\[
\NatZero:\Nat\qquad \NatSuc{(\Bhab{n}{\Nat})}:\Nat
\]
Once again, the type explains the legible presentation, as well as
the low-level representation.

We may also simplify appeals to induction by type propagation, as we
have done with functions from pairs and enumerations.
\[
\Rule{\Gamma\Vdash\stk{
\propag{\push{\M{f}}
             {\PI{\V{d}}{\descop{\M{D}}{(\Mu{\M{D}})}}
                \All{\M{D}}{(\Mu{\M{D}})}{(\PLAM{\x}{\Mu{\M{D}}}\M{P})}{\V{d}} \To \M{P}[\Con{\V{d}}/\x]\\}}
       {\M{f'}}}}
{\Gamma\Vdash
\propag{\push{\sind{\M{f}}}
             {\PIS{\Bhab{\x}{\Mu{\M{D}}}}\M{P} }}
       {\F{ind}\:\M{D}\:(\PLAM{\x}{\Mu{\M{D}}}\M{P})\:\M{f'}}}
\]
This abbreviation is no substitute for the dependent pattern
matching to which we are entitled in a high-level language built on top
of this theory~\cite{goguen:pattern-matching}. It does at least make
`assembly language' programming mercifully brief, albeit hieroglyphic.
\[\stk{
\F{plus}:\Nat\To\Nat\To\Nat \\
\F{plus}\mapsto\sind{\spl{\sqr{(\LAM{\_}\LAM{\_}\LAM{\V{y}}\V{y})
  \quad(\LAM{\_}\spl{\LAM{\V{h}}\LAM{\_}\LAM{\V{y}}
    \NatSuc{(\V{h}\:\V{y})}})}}}
}\]

This concludes our introduction to the universe of datatype descriptions.
We have encoded sum-of-products datatypes from the simply-typed world
as data and equipped them with computation. We have also made sure
to hide the details by type propagation.


%!TEX root = head-full.tex

\section{Related Work} \label{sec:relatedwork}

In this paper, we do not dig very deep into a specific field of the algorithms, but provide an overall view of the MCMC method.
There are brilliant seniors who have done many marvellous works in different specific field.

\para{Metropolis-Hastings algorithm}
Gilks\cite{gilks2005markov} and Chib et al.\cite{chib1995} did a significant job in giving a clear introduction and deeper explanation to the Metropolis-Hastings Algorithm. Beskos et al.\cite{beskos2009} made a great work in analysing the complexity of the Algorithm.

\para{Partition function estimation}
Hubbard\cite{hubbard1959calculation} is probably the earliest reseacher managing to reduce the complexity of calculating a partition function. Jarzynski\cite{nonequilibrium} systemetically analyzed the bias between the estimation and the real value of the partition functions in an sampling process.

Due to my limited knowledge, there might be some mistakes and flaws in this paper, please don't hesitate to contact and correct me.


%\special{papersize=8.5in,11in}
%\setlength{\parindent}{0pt}
\documentclass[12pt]{article}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\input{preambles}
%%%% Packages %%%%

% AMS--related
\usepackage{amssymb,amsmath,amsthm}

\usepackage{upgreek}
\usepackage{mathtools}

\usepackage{dsfont}
\usepackage{slashed}
\usepackage{cancel}
\usepackage{fullpage}
\setlength{\parskip}{1em}
\usepackage{graphicx}

\usepackage{mathrsfs}

\usepackage[font=small,labelfont=bf]{caption} %Font size of Figure Caption
\usepackage{float}

\usepackage{authblk} % multiple authors

\usepackage{braket}
\usepackage{color}
\usepackage{siunitx}
\usepackage{xcolor}
\usepackage{cancel}
\usepackage{systeme}
\usepackage{bm}% bold math symbol, \bm, \hm. for vectors

\usepackage{epstopdf}% use eps figures

\usepackage{natbib}
% \usepackage[style=numeric-comp,
% 			backend=biber,
% 			natbib=true,
% 			isbn=false,
% 			url=false,
% 			doi=false,
% 			eprint=true,
% 			hyperref=true,
% 			sorting=none]{biblatex}
%https://tex.stackexchange.com/questions/116088/sort-biblatex-bibliography-by-ap
%pearance-of-cites-in-the-document
%\addbibresource{./eft.bib}

\usepackage{hyperref}
\hypersetup{
     colorlinks   = true,
     citecolor    =  red
}
%\usepackage[noadjust]{cite}
\usepackage{cleveref}

\usepackage{amsmath,amssymb}

% ':=' as \coloneqq
\usepackage{mathtools}
% Physical bras and kets
\usepackage{braket}
% SI units
\usepackage{siunitx}
\sisetup{separate-uncertainty}

%\usepackage[colorinlistoftodos]{todonotes}

% Boxed equations. NEED TO BE LOADED BEFORE unicode-math!
\usepackage{empheq}
% Theorems

% Plot
\usepackage{tikz}
\usetikzlibrary{decorations.pathmorphing}
\usetikzlibrary{calc}
% save compiled tikz plots; enable --shell-escape
%\usetikzlibrary{external}
%\tikzexternalize[prefix=./tikz/]

\usepackage{subcaption}

%%%% Chemical--Mathematical--Physical Constants %%%%
\newcommand\mi{\mathrm{i}} % imaginary unit i
\newcommand\me{\mathrm{e}} % natural number e

\usepackage{upgreek}

\newcommand\pp{\uppi}

\newcommand\const{\mathrm{const}} % constant

%%%% Math Commands without Parameter %%%%

\newcommand\dif{\mathrm{d}}
\newcommand\Dif{\mathrm{D}}
\newcommand\dva{\updelta}

\DeclareMathOperator{\arcsinh}{arcsinh}
\DeclareMathOperator{\arccosh}{arccosh}
\DeclareMathOperator{\arctanh}{arctanh}
\DeclareMathOperator{\arccoth}{arccoth}
\DeclareMathOperator{\arcctgh}{arcctgh}
\DeclareMathOperator{\arcsech}{arcsech}
\DeclareMathOperator{\arccsch}{arccsch}

\DeclareMathOperator{\BesselJ}{J}
\DeclareMathOperator{\BesselY}{Y}
\DeclareMathOperator{\BesselF}{F}
\DeclareMathOperator{\BesselG}{G}
\DeclareMathOperator{\BesselI}{I}
\DeclareMathOperator{\BesselK}{K}
\DeclareMathOperator{\BesselL}{L}

\DeclareMathOperator{\sgn}{sgn}
\DeclareMathOperator{\grad}{grad}
\DeclareMathOperator{\curl}{curl}
\DeclareMathOperator{\rot}{rot}
\DeclareMathOperator{\opdiv}{div}
\DeclareMathOperator{\opdeg}{deg}

\DeclareMathOperator{\sech}{sech}
\DeclareMathOperator{\csch}{csch}

\DeclareMathOperator{\diag}{diag}
\DeclareMathOperator{\tr}{tr}
\DeclareMathOperator{\Tr}{Tr}
\DeclareMathOperator{\rank}{rank}

\DeclareMathOperator{\ad}{ad}

\DeclareMathOperator{\expi}{expi}

%%%% Math Commands with Parameters %%%%

% Bracket-like
\newcommand{\rbr}[1]{{\left(#1\right)}}
\newcommand{\sbr}[1]{{\left[#1\right]}}
\newcommand{\cbr}[1]{{\left\{#1\right\}}}
\newcommand{\abr}[1]{{\left<#1\right>}}
\newcommand{\vbr}[1]{{\left|#1\right|}}
\newcommand{\dvbr}[1]{{\left\|#1\right\|}}
\newcommand{\fat}[2]{{\left.#1\right|_{#2}}}
\newcommand*\abs[1]{\left|#1\right|}% \abs{}, absolute value bracket

% Functions; note the space between the name and the bracket!
\newcommand{\rfun}[2]{{#1}\mathopen{}\left(#2\right)\mathclose{}}
\newcommand{\sfun}[2]{{#1}\mathopen{}\left[#2\right]\mathclose{}}
\newcommand{\cfun}[2]{{#1}\mathopen{}\left\{#2\right\}\mathclose{}}
\newcommand{\afun}[2]{{#1}\mathopen{}\left<#2\right>\mathclose{}}
\newcommand{\vfun}[2]{{#1}\mathopen{}\left|#2\right|\mathclose{}}
% Fraction-like
\newcommand{\frde}[2]{{\frac{\dif{#1}}{\dif{#2}}}}
\newcommand{\frDe}[2]{{\frac{\Dif{#1}}{\Dif{#2}}}}
\newcommand{\frpa}[2]{{\frac{\partial{#1}}{\partial{#2}}}}
\newcommand{\frdva}[2]{{\frac{\dva{#1}}{\dva{#2}}}}

% overline-like marks
\newcommand{\ol}[1]{{\overline{{#1}}}}
\newcommand{\ul}[1]{{\underline{{#1}}}}
\newcommand{\tld}[1]{{\widetilde{{#1}}}}
\newcommand{\ora}[1]{{\overrightarrow{#1}}}
\newcommand{\ola}[1]{{\overleftarrow{#1}}}
\newcommand{\td}[1]{{\widetilde{#1}}}
\newcommand{\what}[1]{{\widehat{#1}}}
%\newcommand{\prm}{{\symbol{"2032}}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\DeclarePairedDelimiter\ceil{\lceil}{\rceil}
\DeclarePairedDelimiter\floor{\lfloor}{\rfloor}

\def\beq{\begin{equation}}
\def\eeq{\end{equation}}

\def\a{\alpha}
\def\kap{\varkappa}
\def\e{\varepsilon}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\newcommand*\ov[1]{\overset{(5)}{#1}}
\newcommand*\ovn[1]{\overset{(n)}{#1}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%roman number
\makeatletter
\newcommand*{\rom}[1]{\expandafter\@slowromancap\romannumeral #1@}
\makeatother
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\title{Effective action of scalar electrodynamics}

\date{}

%\author[1,4]{Alexander A. Andrianov\thanks{a.andrianov@spbu.ru} }
%\author[2]{Chen Lan\thanks{stlanchen@yandex.ru} }
%\author[1]{Oleg O. Novikov\thanks{o.novikov@spbu.ru}}
\author[3]{Yi-Fan Wang\thanks{yfwang@thp.uni-koeln.de}}
%\author[5]{Zhongyi Zhang\thanks{zhongyi@th.physik.uni-bonn.de}}

%\affil[1]{Saint-Petersburg State University, St. Petersburg 198504, Russia}
%\affil[2]{ELI-ALPS, ELI-Hu NKft, Dugonics t\'er 13, Szeged 6720, Hungary}
\affil[3]{Institut f\"ur Theoretische Physik, Universit\"at zu K\"oln,
Z\"ulpicher Stra\ss e 77, 50937 K\"oln, Germany}
%\affil[4]{Institut de Ci\`encies del Cosmos (ICCUB), Universitat de Barcelona,
%Spain}
%\affil[5]{Physikalisches Institut, Universit\"at Bonn, Nussallee 12, 53115
%Bonn, Germany}

\maketitle

%\input{./sections/0abstract}
%\abstract{}

%\tableofcontents

The idea of a critical field traces back to \cite{Sauter1931}, where Dirac
equation in a linear potential was solved.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Classical field}
\label{sec:cla-fie}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Complex Klein--Gordon action in flat space-time
\begin{align}
\sfun{S_\text{S}}{\phi,\phi^*}\coloneqq\int\dif^{d+1}x\,\cbr{-\eta^{\mu\nu}
\rbr{\partial_\mu\phi}^* \rbr{\partial_\nu\phi} - m^2\phi^*\phi}.
\end{align}
Interaction terms
\begin{align}
\sfun{S_\text{SM}}{A_\mu,\phi,\phi^*} \coloneqq \int\dif^{d+1}x\,
\eta^{\mu\nu}{}
\cbr{\mi e A_\mu \rbr{-\phi^*\partial_\nu\phi+\phi\partial_\nu\phi^*}
+e^2 A_\mu A_\nu \phi^* \phi}.
\end{align}
The total action for scalar electrodynamics reads
\begin{align}
\sfun{S}{A_\mu, \phi, \phi^*} &\coloneqq
S_\text{S} + S_\text{SM} + S_\text{Maxwell} \nonumber \\
&= \int\dif^{d+1}x\,\cbr{-\rbr{\nabla_{\!\mu} \phi}^* \rbr{\nabla^\mu\phi} -
m^2\phi^*\phi -\frac{1}{4}F^{\mu\nu}F_{\mu\nu}},
% \nonumber \\
% &= \int\dif^{d+1} x \cbr{\phi^*\rbr{-\nabla_\mu \nabla^\mu - m^2}\phi
% - \frac{1}{4} F^{\mu\nu} F_{\mu\nu} + \text{b.t.}},
\end{align}
where
\begin{align}
\nabla_\mu\phi \coloneqq \rbr{\partial_\mu+\mi e A_\mu}\phi.
%,\qquad\text{b.t.} = \partial_\mu \rbr{\phi^*\nabla^\mu \phi}.
\end{align}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Functional quantisation}
\label{sec:gen-fun}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%
\subsection{Wick rotation}

%%%%%%%%%%%%%%%%

Lorentzian generating functional

Wick rotation
\begin{align}
x_\text{E}^4 = \mi x^0,\quad A_4 = -\mi A_0,
\end{align}
so that
\begin{align}
\partial_{x^0} = \mi \partial_{x_\text{E}^4},\quad
F_{0i} = \mi F_{4i}.
\end{align}

The Euclidean action reads
\begin{align}
\sfun{S_\text{E}}{A_I,\phi,\phi^*} = \int \dif^{D} x_\text{E}\,\rbr{
	\frac{1}{4} F_{IJ} F^{IJ} + \rbr{\nabla_{\!I}\phi}^* \rbr{\nabla^I \phi}
	+ m^2 \phi^* \phi}.
\end{align}

Working with the Euclidean signature is easier than in the Lorentzian
signature.

%%%%%%%%%%%%%%%%
\subsection{Effective action}
\label{ssec:gen-fun-eff-act}
%%%%%%%%%%%%%%%%

\emph{Generating functional} and the \emph{connected generating functional}
(omitting subscript E for Euclidean systematically)
\begin{align}
\sfun{\mathcal{Z}}{j^I, J, J^*} &\coloneqq
\int\Dif A_I\,\Dif\phi^*\,\Dif\phi\,
	\cfun{\exp}{- S - \int\dif^{D} x\,\rbr{j^I A_I + J^* \phi+\phi^* J}}
\\
&\eqqcolon \cfun{\exp}{-\sfun{W}{j^I, J, J^*}}.
\end{align}
The expectation value of $A_I$, $\phi$ etc.
\begin{align}
	\sfun{A^\text{e}_I}{j^I, J, J^*} \coloneqq \frdva{W}{j^I},
	\\
	\sfun{\phi^\text{e}}{j^I, J, J^*} \coloneqq \frdva{W}{J^*}.
\end{align}

One would like to find \emph{an effective action for $A$}, defined by the
Legendre transformation
\begin{align}
\sfun{\varGamma_A}{A_I} + \int \dif^D x\,\overline{j}^I A_I
&\coloneqq \sfun{W}{\overline{j}^I, 0, 0}
\equiv -\ln \sfun{\mathcal{Z}}{\overline{j}^I, 0, 0},
\label{eq:gen-fun-eff-act-100}
\\
\fat{\frdva{\sfun{W}{j, 0, 0}}{j^I}}{j^I = \overline{j}^I}
	&\coloneqq A_I.
\end{align}

In order to understand $\sfun{\varGamma_A}{A_I}$, take $j^I$ as the fundamental
field, and take functional derivative with respect to it in
\cref{eq:gen-fun-eff-act-100}, yielding
\begin{align}
\frdva{\varGamma_A}{A_I}\frdva{A_I}{j^J} + A_J + j^I\frdva{A_I}{j^J} =
\frdva{W}{j^J} &= A_J,
\end{align}
or
\begin{empheq}[box=\fbox]{align}
\frdva{\varGamma_A}{A_I} + j^I = 0.
\label{eq:gen-fun-eff-act-gam-10}
\end{empheq}
At the leading order, \cref{eq:gen-fun-eff-act-gam-10} reduces to the
expression for classical action principle with source. One also has
\begin{align}
\sfun{\varGamma_A}{A^\text{e}_I} &\equiv
-\ln \int \Dif A_I\,\Dif \phi^*\,\Dif \phi\,
	\cfun{\exp}{-\sfun{S}{A^\text{c}_I+A_I, \phi, \phi^*}};
\end{align}
at the leading order, the external field $A^\text{e}_I$ reduces to the
classical field $A^\text{c}_I$. See \cite[ch.\ 34]{Schwartz2013}, and
better \cite[ch.\ 22]{Kleinert2015}.

The derivative term of $\phi$ can be rearranged
\begin{align}
\rbr{\nabla_{\! I} \phi}^* \rbr{\nabla^{I} \phi} = \partial_I\rbr{\phi^*
\nabla^{I}\phi} - \phi^* \nabla_{\! I} \nabla^{I} \phi.
\end{align}
Hence up to boundary terms,
\begin{align}
\sfun{S}{A_I, \phi, \phi^*} = \int \dif^D x\, \frac{1}{4} F_{IJ} F^{IJ} +
	\int \dif^D x\,\dif^D y\,
	\rfun{\phi^*}{x} M[A_I; x-y) \rfun{\phi}{y},
\end{align}
where
\begin{align}
M[A_I; x-y) \coloneqq \rbr{-\nabla_{\! x^I} \nabla^{x^I} + m^2}
\rfun{\delta^{(D)}}{x-y},
\end{align}
see e.g.\ \cite[ch.\ 6]{mosel2004} for details. Now the scalar field can
formally be integrated, giving the (Euclidean) Euler--Heisenberg ``effective
action''
\begin{align}
\sfun{\mathcal{Z}}{j^I, 0, 0} &\eqqcolon
\int \Dif A_I\,\cfun{\exp}{-\sfun{\varGamma_\text{EH}}{A_I}
+ \int \dif^D x\,j^I A_I},
\\
\sfun{\varGamma_\text{EH}}{A_I} - \int\dif^{D} x\,
\frac{1}{4} F_{IJ} F^{IJ}
&=
-\ln \int \Dif\phi^*\,\Dif\phi\,
	\cfun{\exp}{-\int\dif^{D} x\, \rbr{
	\rbr{\nabla_{\! I} \phi}^* \rbr{\nabla^{I} \phi} + m^2 \phi^* \phi}}
\nonumber \\
&= -\ln \frac{1}{\det M[A_I; x-y)} = \tr \ln M,
\label{eq:gen-fun-eff-act-150}
\end{align}
where normalisation is implicit in $\det M$. \Cref{eq:gen-fun-eff-act-150}
traces
back to \cite{heisenberg1936,weisskopf1936}.

Going back to \cref{eq:gen-fun-eff-act-100}, one now has
\begin{align}
\sfun{\mathcal{Z}}{j^I, 0, 0} &=
\int\Dif A\, \cfun{\exp}{-\int \dif^D x\,
\rbr{\frac{1}{4} F_{IJ} F^{IJ} - j^{I} A_{I}} - \tr\ln M}
\nonumber \\
&=
\sfun{\mathcal{Z}_{A}}{0}
	\Braket{j^I | \rfun{\exp}{-\tr\ln M} | j^I}
\equiv \sfun{\mathcal{Z}_{A}}{0}
	\Braket{A_I | \rfun{\exp}{-\tr\ln M} | A_I},
\label{eq:gen-fun-eff-act-200}
\end{align}
where the expectation is defined as
\begin{align}
\Braket{A_I | \mathcal{O} | A_I} &\coloneqq
	\sfun{\mathcal{Z}_{A}^{-1}}{0}
	\int\Dif A\, \mathcal{O}\,\cfun{\exp}{-\int \dif^D x\,
		\rbr{\frac{1}{4} F_{IJ} F^{IJ} - j^{I} A_{I}}},
\\
\sfun{\mathcal{Z}_{A}}{j^I} &\coloneqq
\int\Dif A\, \cfun{\exp}{-\int \dif^D x\,
\rbr{\frac{1}{4} F_{IJ} F^{IJ} - j^{I} A_{I}}}.
\end{align}

%%%%%%%%%%%%%%%%
\subsection{Certain limit}
\label{ssec:eucl-cert}
%%%%%%%%%%%%%%%%
In a certain limit (\textbf{which limit?}), the correlation between $\tr \ln M$
can be omitted, so that \cref{eq:gen-fun-eff-act-200} goes
\begin{align}
\sfun{\mathcal{Z}}{j^I, 0, 0} &\approx
\sfun{\mathcal{Z}_{A}}{0}
	\rfun{\exp}{-\Braket{A_I | \tr\ln M | A_I}},
\end{align}
and \cref{eq:gen-fun-eff-act-100} goes
\begin{align}
\sfun{\varGamma_A}{A_I} &\approx -\ln \sfun{\mathcal{Z}_{A}}{0}
	+ \Braket{A_I | \tr\ln M | A_I}.
\label{eq:eucl-cert-100}
\end{align}

%%%%%%%%%%%%%%%%
\subsection{World-line formalism}
\label{ssec:eucl-wlfm}
%%%%%%%%%%%%%%%%

In \cref{eq:gen-fun-eff-act-150,eq:gen-fun-eff-act-200,eq:eucl-cert-100},
$\tr\ln M$ is crucial.

Using the Schwinger integral representation \cite{Schwinger1951} (up to
normalisation, see \cref{sec:sch-int})
\begin{align}
\ln \alpha = -\int_0^{+\infty} \frac{\dif s}{s}\,\me^{-\alpha s},
\end{align}
one has
\begin{align}
- \tr\ln M = \int_0^{+\infty} \frac{\dif T}{T}\,
	\rfun{\exp}{-\frac{m^2 T}{2\widetilde{m}}}\,
	\tr \rfun{\exp}{-\frac{M}{2\widetilde{m}}},
\end{align}
where $T$ has the dimension of time, and $\widetilde{m}$ that of mass, which
will both be eliminated later. Introduce the Hamiltonian of a non-relativistic
point particle in $D$ spatial dimensions (note originally one works in $d =
D-1$ spatial dimensions) (\textbf{check sign!})
\begin{align}
H \coloneqq \frac{1}{2 \widetilde{m}} \rbr{P_I + e A_I}^2,
\end{align}
so that quantisation yields the following representation (\textbf{check sign!})
\begin{align}
\tr \rfun{\exp}{-\frac{M}{2\widetilde{m}}} &= \int_{-\infty}^{+\infty}\dif x\,
\Braket{x | \me^{-\widehat{H} T} | x}
\\
&= \oint \Dif x\,\cfun{\exp}{-\int_{0}^{T}\dif T'\,
	\rbr{\frac{\widetilde{m}}{2} \rbr{\frde{x^I}{T'}}^2 +
	\mi e A_I \frde{x^I}{T'}}}.
\label{eq:eucl-wlfm-050}
\end{align}
Rescaling $T' \eqqcolon \lambda T$ gives
\begin{align}
- \tr\ln M = \int_{0}^{+\infty} \frac{\dif T}{T}\,
	\rfun{\exp}{-\frac{m^2 T}{2 \widetilde{m}}}
	\oint \Dif x\,
		\rfun{\exp}{-\frac{\widetilde{m}}{2T} \int_{0}^{1} \dif \lambda\,
			\dot{x}_I^2 - \mi e \oint A_I\,\dif x^I}.
\label{eq:eucl-wlfm-100}
\end{align}

%%%%%%%%%%%%%%%%
\subsection{Euler--Heisenberg ``effective action''}
\label{ssec:eucl-ehel}
%%%%%%%%%%%%%%%%

If the instanton magnetic field in \cref{eq:eucl-wlfm-050} is constant, the
path integral can be performed exactly \cite{Feynman1965}, so that
\cref{eq:gen-fun-eff-act-150} can be expressed in terms of an integral of $T$.

It is difficult to obtain a classical solution for the motion of a point
particle in a more generic magnetic field. %, e.g.\ \cite{Kondo1964}.
Therefore the generalisation in this direction is limited.

%%%%%%%%%%%%%%%%
\subsection{World-line instanton approximations}
\label{ssec:eucl-wlia}
%%%%%%%%%%%%%%%%

In \cref{eq:eucl-wlfm-100}, one may also perform the $T$ integral first. Using
the integral expression and the asymptotic expansion for a modified Bessel
function
\begin{align}
\rfun{\BesselK_0}{x} &= \frac{1}{2} \int_0^{+\infty} \frac{\dif t}{t}\,
	\rfun{\exp}{-t-\frac{x^2}{4t}}
\\
&\approx \sqrt{\frac{\pp}{2x}}\,\me^{-x},\qquad
	x \gg 1,
\end{align}
one has
\begin{align}
- \tr\ln M &= 2 \oint \Dif x\,
	\rfun{\BesselK_0}{m\sqrt{\int_0^1 \dif\lambda\, \dot{x}_I^2}}
		\rfun{\exp}{- \mi e \oint A_I\,\dif x^I}
\\
&\approx \sqrt{\frac{2\pp}{m}} \oint \Dif x\,
	\rbr{\int_0^1 \dif\lambda\, \dot{x}_I^2}^{-1/4}
	\rfun{\exp}{-m\sqrt{\int_0^1 \dif\lambda\, \dot{x}_I^2}
		- \mi e \oint A_I\,\dif x^I},
\label{eq:eucl-wlia-100}
\end{align}
where \cref{eq:eucl-wlia-100} works for
\begin{align}
m \sqrt{\int_0^1 \dif\lambda\, \dot{x}_I^2} \gg 1
\qquad\text{or}\qquad
\int_0^1 \dif\lambda\, \dot{x}_I^2 \gg m^{-2}.
\end{align}
This idea traces back to \cite{Affleck1982}.

In such a limit, one may estimate $\tr\ln M$ by the classical trajectory of the
non-linear instanton action
\begin{align}
S_\text{i} = -\ln\rfun{\BesselK_0}{m\sqrt{\int_0^1\dif\lambda\, \dot{x}_I^2}}
+ \mi e \oint A_I\,\dif x^I.
\end{align}
The action principle gives an integro-differential equation
\begin{align}
0 = \frdva{S_\text{i}}{x_I} = -m
\frac{\rfun{\BesselK_1}{m\sqrt{\int_0^1\dif\lambda\, \dot{x}_I^2}}}%
{\rfun{\BesselK_0}{m\sqrt{\int_0^1\dif\lambda\, \dot{x}_I^2}}}
\frac{\ddot{x}^I}{\sqrt{\int_0^1\dif\lambda\, \dot{x}_I^2}} +
	\mi e F^{IJ} \dot{x}_J.
\label{eq:eucl-wlia-200}
\end{align}
Contracting \cref{eq:eucl-wlia-200} with $\dot{x}_I$ yields
\begin{align}
\frde{\dot{x}_I^2}{\lambda} = 0,\qquad
\dot{x}_I^2 \eqqcolon a^2,\quad a > 0.
\end{align}
Hence \cref{eq:eucl-wlia-200} turns into
\begin{align}
m \rfun{K}{a} \ddot{x}^I = \mi e F^{IJ} \dot{x}_J,\qquad
\rfun{K}{a} \coloneqq \frac{1}{a}
\frac{\rfun{\BesselK_1}{ma}}{\rfun{\BesselK_0}{ma}}.
\label{eq:eucl-wlia-10}
\end{align}

In Euclidean signature, \cref{eq:eucl-wlia-10} is intrinsically complex
\cite{Dumlu2011}.

%%%%%%%%%%%%%%%%
\subsection{Application of instanton approximations}
\label{ssec:eucl-appl}
%%%%%%%%%%%%%%%%

\cite{Dunne2005}

%%%%%%%%
\subsubsection{Constant electric field}
\label{ssec:eucl-appl-cons}
%%%%%%%%

%%%%%%%%%%%%%%%%
\subsection{Duality}
\label{ssec:eucl-dual}
%%%%%%%%%%%%%%%%

Idea \cite{Semenoff2011}; book \cite[ch.\ 5]{Casalderrey-Solana2009}; review
\cite{Kawai2015};

Neveu--Schwarz--Neveu--Schwarz two-form \cite{Becker2006}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Flat space-time (Lorentzian signature)}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The content below needs revise.

Generating functional
\begin{equation}
\sfun{\mathcal{Z}}{j^\mu, J, J^*} \coloneqq
\int\Dif A\,\Dif\phi\,\Dif\phi^*\,\cfun{\exp}{\mi\rbr{S_0
+\int\dif^{d+1} x\,\rbr{j^\mu A_\mu + J^* \phi+\phi^* J}}}.
\end{equation}

Effective action
\begin{equation}
\sfun{\mathcal{Z}}{j^\mu, 0, 0} \eqqcolon
\int\Dif A\,\cfun{\exp}{\mi\rbr{S_\text{Maxwell} +
\sfun{\varGamma_\text{W}}{A_\mu}
+\int\dif^{d+1} x \,j^\mu A_\mu}}.
\end{equation}
In other words,
\begin{align}
\cfun{\exp}{\mi\sfun{\varGamma_\text{W}}{A_\mu}} &\coloneqq
\int\Dif\phi\,\Dif\phi^*\,
\cfun{\exp}{\mi\rbr{S_\text{CKG}+S_\text{ICKGM}}}
\nonumber \\
&\equiv  \int\Dif\phi\,\Dif\phi^*\,
\cfun{\exp}{\mi\int\dif^{d+1}x\,\cbr{-\rbr{\nabla_\mu\phi}^*
\rbr{\nabla^\mu\phi} - m^2\phi^*\phi}}.
\label{eq:scalar-qed-to-be-manipulated}
%\nonumber \\
\end{align}
The integral in the exponent can be manipulated; only the first term is
essential
\begin{align}
&\quad\int\dif^{d+1}x\,\rbr{-\rbr{\nabla_\mu\phi}^* \rbr{\nabla^\mu\phi}}
\nonumber \\
&= \int\dif^{d+1}x\,\dif^{d+1}y\,\rbr{
-\rbr{\nabla_{x^\mu}\rfun{\phi}{x}}^* \rfun{\delta^{d+1}}{x-y}
\nabla^{y^\mu}\rfun{\phi}{y}},
\label{eq:scalar-qed-matrix-1}
\end{align}
where
\begin{align}
\rfun{\delta^{d+1}}{x-y} \nabla^{y^\mu}\rfun{\phi}{y}
&= \rfun{\delta^{d+1}}{x-y}
\cbr{\partial^{y^\mu}+\mi e \rfun{A^\mu}{y}}\rfun{\phi}{y} \nonumber \\
&= \cbr{-\rbr{\nabla^{y^\mu}}^*\rfun{\delta^{d+1}}{x-y}}\rfun{\phi}{y}
+\partial^{y^\mu}B,
\end{align}
in which
\begin{align}
B = \rfun{B}{x,y} \coloneqq \rfun{\delta^{d+1}}{x-y}\rfun{\phi}{y};
\end{align}
going back to \cref{eq:scalar-qed-matrix-1},
\begin{align}
&= \int\dif^{d+1}x\,\dif^{d+1}y\,\cbr{
-\cbr{\cbr{\partial_{x^\mu}+\mi e \rfun{A_\mu}{x}}\rfun{\phi}{x}}^*
\rfun{\delta^{d+1}}{x-y} \nabla^{y^\mu}\rfun{\phi}{y}}
\nonumber \\
&= \int\dif^{d+1}x\,\dif^{d+1}y\,\cbr{ -\partial_{x^\mu} C^\mu +
\rfun{\phi^*}{x}
\nabla_{x^\mu}
\rfun{\delta^{d+1}}{x-y} \nabla^{y^\mu}\rfun{\phi}{y}} \nonumber \\
&= \int\dif^{d+1}x\,\dif^{d+1}y\,\cbr{ -\partial_{x^\mu} C^\mu +
\rfun{\phi^*}{x}
\cbr{-\rbr{\nabla_{y^\mu}}\rbr{\nabla^{y^\mu}}^*\rfun{\delta^{d+1}}{x-y}}
\rfun{\phi}{y} + \partial^{y^\mu}\nabla_{x^\mu} B},
\end{align}
in which
\begin{equation}
C^\mu = \rfun{C^\mu}{x,y} \coloneqq \rfun{\phi^*}{x}
\rfun{\delta^{d+1}}{x-y} \nabla^{y^\mu}\rfun{\phi}{y}.
\end{equation}
Now \cref{eq:scalar-qed-to-be-manipulated} can be written as (dropping the
boundary terms)
\begin{align}
&= \int\Dif\phi\,\Dif\phi^*\,
\cfun{\exp}{-\mi\int\dif^{d+1} x\,\dif^{d+1} y\,
\rfun{\phi^*}{x} \rfun{D^{-1}}{x, y} \rfun{\phi}{y}}
\nonumber \\
&= \tilde{\mathcal{N}}\cbr{\sfun{\det}{\rfun{D^{-1}}{x, y}}}^{-1/2},
\end{align}
where
\begin{align}
\rfun{D^{-1}}{x, y} \coloneqq \mathcal{D}^{-1}_y \rfun{\delta^{d+1}}{x-y},\qquad
\mathcal{D}^{-1}_y \coloneqq +\rbr{\nabla_{y^\mu}}\rbr{\nabla^{y^\mu}}^* + m^2.
\end{align}

\begin{align}
\sfun{\varGamma_\text{W}}{A_\mu} &\equiv
-\mi\rbr{\ln\tilde{\mathcal{N}} - \frac{1}{2} \ln \det D^{-1}} \nonumber \\
&= \frac{\mi}{2} \tr_x \rfun{\ln}{\mathcal{N}^{-1} D^{-1}} \nonumber \\
&= \frac{\mi}{2} \int_0^{+\infty}\frac{\dif s}{s}
\int\dif^{d+1}x\,\dif^{d+1}y\,\rfun{\delta^{d+1}}{x-y} \nonumber \\
&\quad\cdot\cbr{
-\me^{\mi s\rbr{+\rbr{\nabla_{y^\mu}}\rbr{\nabla^{y^\mu}}^* + m^2 + \mi 0^+}
\rfun{\delta^{d+1}}{x-y}}
+\me^{\mi s\rbr{\mathcal{N}+\mi 0^+}}}.
\end{align}

%\begin{align}
%\sfun{\varGamma_\text{W}}{A_\mu} &\equiv
%-\mi\rbr{\ln\mathcal{N} + \ln \sfun{\det}{-\mi M}} \nonumber \\
%&= -\mi \rbr{\ln\mathcal{N} + \Tr \sfun{\ln}{-\mi \rfun{D^{-1}}{x, y}}}.
%\end{align}

\cite{weisskopf1936}

%\section*{Acknowledgements}
%\addcontentsline{toc}{section}{\protect\numberline{}Acknowledgements}%

\appendix

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Notions and conventions}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The metric convention is mostly positive, i.e.\
$\eta_{\mu\nu} \coloneqq \rfun{\diag}{-, +, +, \ldots}$

Pauli matrices
\begin{equation}
\sigma^1 \coloneqq \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix},\quad
\sigma^2 \coloneqq \begin{pmatrix} 0 & -\mi \\ +\mi & 0 \end{pmatrix},\quad
\sigma^3 \coloneqq \begin{pmatrix} +1 & 0 \\ 0 & -1 \end{pmatrix}.
\end{equation}

The $\gamma$-matrices satisfy \cite[sec.~5]{weinberg1995}
\begin{equation}
\sbr{\gamma^\mu, \gamma^\nu}_+ \coloneqq 2\eta^{\mu\nu} \mathbf{1}_4.
\end{equation}

\begin{equation}
\mathscr{J}^{\mu\nu} \coloneqq -\frac{\mi}{4}\sbr{\gamma^\mu, \gamma^\nu}_-
\end{equation}

\begin{equation}
\sigma^{\mu\nu} \coloneqq \frac{\mi}{2}\sbr{\gamma^\mu, \gamma^\nu}_-
\equiv -2 \mathscr{J}^{\mu\nu}.
\end{equation}

In $\rbr{3+1}$ dimensions, choose the chiral representation
\begin{equation}
\gamma^\mu = -\mi
\begin{bmatrix}0 & \sigma^\mu \\ \bar\sigma^\mu & 0\end{bmatrix},
\end{equation}
where
\begin{equation}
\sigma^\mu \coloneqq \rbr{1_2, +\vec{\sigma}},\qquad
\bar\sigma^\mu \coloneqq \rbr{1_2, -\vec{\sigma}}.
\end{equation}

\begin{align}
\sigma^{\mu\nu} &\equiv -\frac{\mi}{2}
\begin{bmatrix}
\sigma^\mu\bar\sigma^\nu-\sigma^\nu\bar\sigma^\mu & 0 \\
0 & \bar\sigma^\mu\sigma^\nu-\bar\sigma^\nu\sigma^\mu
\end{bmatrix} \nonumber \\
&=
\begin{cases}
0 & \mu = 0, \nu = 0; \\
\mi \begin{bmatrix}+\sigma^j & 0 \\ 0 & -\sigma^j\end{bmatrix}
& \mu = 0, \nu = j; \\
\mi \begin{bmatrix}-\sigma^i & 0 \\ 0 & +\sigma^i\end{bmatrix}
& \mu = i, \nu = 0; \\
\begin{bmatrix}
+\epsilon^{ij}{}_k \sigma^k & 0
\\ 0 & +\epsilon^{ij}{}_k \sigma^k\end{bmatrix}
& \mu = i, \nu = j.
\end{cases}
\end{align}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Fresnel functional integral}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\citep[ch.\ 10]{mosel2004}

\section{Algebra}
\begin{align}
\sbr{a\partial_1 \partial_2, x^1}_- = a\partial_2
\end{align}
central.

Baker--Campbell--Hausdorff formula

\begin{align}
\me^{+a\partial_1\partial_2} x^1 \me^{-a\partial_1\partial_2}
= x^1 + a \partial_2.
\end{align}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Schwinger integral}
\label{sec:sch-int}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{align}
-\int_{\epsilon}^{+\infty} \frac{\dif t}{t}\,\me^{-\alpha t} =
-\rfun{\Gamma}{0, \alpha \epsilon} = \upgamma_\text{E} + \ln\alpha +
\ln\epsilon + \rfun{O}{\epsilon},
\end{align}
where $\rfun{\Gamma}{a,z}$ is the incomplete Gamma function,
$\upgamma_\text{E}$ is Euler's constant.

\bibliographystyle{plainnat}
\bibliography{./eft}
%\printbibliography

\end{document}


% Created 2015-11-11 Wed 18:00
\documentclass{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{fixltx2e}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{float}
\usepackage{wrapfig}
\usepackage{soul}
\usepackage{textcomp}
\usepackage{marvosym}
\usepackage{wasysym}
\usepackage{latexsym}
\usepackage{amssymb}
\usepackage{hyperref}
\tolerance=1000
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{parskip}
\usepackage{standalone}
\usepackage{tikz,pgf,pgfplots}
\usetikzlibrary{decorations.pathmorphing,patterns}
\usetikzlibrary{arrows,snakes,backgrounds,patterns,matrix,shapes,fit,calc,shadows,plotmarks,decorations.markings,datavisualization,datavisualization.formats.functions,intersections,external}
\usetikzlibrary{decorations.pathmorphing,patterns}
\pgfplotsset{compat=1.9}
\newcommand*{\mexp}[1]{\ensuremath{\mathrm{e}^{#1}}}
\newcommand*{\laplace}[1]{\ensuremath{\mathcal{L} \{#1\}}}
\newcommand*{\laplaceinv}[1]{\ensuremath{\mathcal{L}^{-1} \{#1\}}}
\newcommand*{\realpart}[1]{\ensuremath{\operatorname{Re}(#1)}}
\newcommand*{\impart}[1]{\ensuremath{\operatorname{Im}(#1)}}
\newcommand*{\vsp}[1]{\rule{0pt}{#1}}
\newcommand*{\tderiv}[1]{\ensuremath{\frac{d^{#1}}{dt^{n}}}}
\newcommand*{\bbm}{\begin{bmatrix}}
\newcommand*{\ebm}{\end{bmatrix}}
\newcommand*{\obsmatrix}{\mathcal{O}}
\newcommand*{\contrmatrix}{\mathcal{C}}
\newcommand*{\cwh}{\ensuremath{\cos \omega h}}
\newcommand*{\swh}{\ensuremath{\sin \omega h}}
\providecommand{\alert}[1]{\textbf{#1}}

\title{Computerized control - homework 1}
\author{Kjartan Halvorsen}
\date{Due 2015-08-20}
\hypersetup{
  pdfkeywords={},
  pdfsubject={},
  pdfcreator={Emacs Org-mode version 7.9.3f}}

\begin{document}

\maketitle

\section{Exercises}
\label{sec-1}
\subsection{Why a theory of sampled-data systems?}
\label{sec-1-1}

   Explain briefly \textbf{three different arguments} for why is makes sense to work with sampled-data description of control systems!
\subsection{Sampling}
\label{sec-1-2}

The continuous-time system
\begin{align*}
\dot{x} &= \begin{bmatrix} 0 & \omega\\-\omega & 0 \end{bmatrix} x + \begin{bmatrix}1\\0\end{bmatrix} u\\
y &= \begin{bmatrix} 1 & 0 \end{bmatrix} x
\end{align*}
is sampled with sampling interval $h$.
\subsubsection{Determine the system matrices for the sampled system on state-space form.}
\label{sec-1-2-1}
\subsubsection{For which values of $h$ is the system observable?}
\label{sec-1-2-2}
\section{Solutions}
\label{sec-2}
\subsection{Why a theory of sampled systems}
\label{sec-2-1}

\begin{itemize}
\item Basically all modern control systems are digital and work in discrete time. A theory that does not take this into account can only be suboptimal.
\item A simple discretizaction of an analog control design does not take into account the properties of the discrete controller. Performance will only approach that of the analog design as the sampling interval $h$ goes to zero.
\item Discrete control can in some cases outperform analog control (e.g. deadbeat control).
\item Many systems are discrete-time by nature, for instance radar, biological systems (at small scales), internal combustion, particle accelerator. See Å\&W chapter 1.4.
\end{itemize}
\subsection{Sampling}
\label{sec-2-2}
\subsubsection{Sampling the state-space system}
\label{sec-2-2-1}

    Assume zero-hold sampling, hence inputs $u$ are constant between sampling instants. Solve the analog system using $x(kh) = x(k)$ as the initial state and $u(k)=u_k$ to be the constant input signal. We get
\begin{equation*}
x(k+1) = \Phi(h)x(k) + \Gamma B u_k = \mexp{Ah}x(k) + \int_0^h \mexp{As}ds B u_k.
\end{equation*}
There are several ways to compute the matrix exponential. One alternative is to use the Laplace transform. Note that the matrix exponential $\Phi(t)=\mexp{At}$ is the solution to the differential equation
\begin{equation*}
\frac{d}{dt} \Phi(t) = A\Phi(t),
\end{equation*}
with the initial condition $\Phi(0) = I$. Solving this differential equation using the Laplace transform gives
\begin{equation*}
s\laplace{\Phi} - \Phi(0) = A\laplace{\Phi} \quad \Rightarrow \quad \Phi = \laplaceinv{\big(sI - A\big)^{-1}}.
\end{equation*}
With the $A$-matrix given in the exercise we get
\begin{equation*}
\laplace{\Phi} = \bbm s & -\omega\\ \omega & s\ebm^{-1}
               = \frac{1}{s^2 + \omega^2} \bbm s & \omega\\ -\omega & s \ebm,
\end{equation*}
and taking the inverse laplace transform of each element we get (from standard table of the Laplace transform)
\begin{equation*}
\Phi(t) = \bbm \cos \omega t & \sin \omega t\\ -\sin \omega t & \cos \omega t \ebm.
\end{equation*}

To find the matrix $\Gamma$ we integrate to get
\begin{equation*}
\Gamma(h) = \int_0^h \mexp{As}ds\bbm 1\\0\ebm = \bbm \int_0^h \cos \omega s ds \\ -\int_0^h \sin \omega s ds \ebm  = \frac{1}{\omega} \bbm \sin \omega h \\ \cos \omega h - 1 \ebm.
\end{equation*}

To summarize, zero-order hold sampling gives the sampled system ($x(kh) = x(k)$)
\begin{align*}
x(k+1) &= \bbm \cos \omega h & \sin \omega h\\ -\sin \omega h & \cos \omega h \ebm x(k) +
          \frac{1}{\omega} \bbm \sin \omega h \\ \cos \omega h - 1 \ebm u(k), \\
y(k) &= \bbm 1 & 0 \ebm x(k).
\end{align*}
\subsubsection{Observability}
\label{sec-2-2-2}

The observability matrix for this second-order system is given by
\[
\obsmatrix = \bbm C \\ C\Phi(h) \ebm
           = \bbm 1 & 0\\ \cwh & \swh \ebm
\]
Observability is lost when the matrix becomes singular, which is for
\[
\det \obsmatrix = \swh = 0.
\]

Thus, the system is observable for
\[ 0 < h < \pi/\omega. \]

\end{document}


\subsubsection{The Arrow of Time}
One interesting property of a lot of physics equations is that they are invariant under time reversal; that is, equations like Newton's second law
\begin{align*}
\vec{F} = m\vec{a}
\end{align*}
can be applied even in cases where time was running in reverse\footnote{Assuming no friction, of course.}. However, it is clear in our immediate experience that a lot of things are not invariant under time reversal. If I were to show you a video of heat flowing from a cold bath of water to a hot bath of water, you would tell me immediately that the video must be playing in reverse. If the laws of physics are supposed to work the same way in either direction, what's going on here?

As we discussed previously, where we see phenomena that only proceed in one time direction, what we are seeing is something go from a low entropy state to a higher entropy state. After the process, by the second law of thermodynamics, we can't go back to the lower entropy state. But this explanation feels a bit unsatisfactory; sure, we've accepted the second law of thermodynamics as a law, but there's still this underlying feeling of "Well, why \textbf{can't} things go back to a lower entropy state?" The statistical definition will allow us to settle this confusion once and for all.

From a statistical viewpoint, what are we saying when a system goes from a low to a high entropy state? Well by the definition of $S = k_b\ln\mathbb{W}$ for the entropy of a macrostate, what we're really saying is that the system moves from a macrostate with a low number of microstates to a high number of microstates. Okay sure, but why? Well let's think back to the fundamental assumption of statistical mechanics that I just outlined a page ago. This assumption tells us that every microstate is equally as likely to appear, and each microstate persists for the same amount of time. Well, if every microstate is equally as likely, then simple notions of probability will tell us that the system at any given point will tend to a macrostate with more microstates.

Let's try to make this a little more concrete with an example. Say I was keeping tracks of 10 numbered cows in a field, separated into a left and right hand side, where the cows were allowed to walk around the field totally randomly. Then, a macrostate of the system could be defined by how many cows were on the left side of the field (where I have 11 microstates, from 0-10 cows on the left side). Now, suppose I start in a configuration where all 10 cows are on the left side of the field. There are a couple ways in which I could arrange the cows in this scenario. However, the macrostate corresponding to 5 cows on the right side and 5 cows on the left side of the field has a lot more microstates; there are a lot more ways I could arrange 5 and 5 cows on either side of the field compared to having all the cows on one side. With this in mind, let's imagine that the cows start all on one side, I walk away from the field, and come back some amount of time later after the cows were free to move around. Since every configuration of cows is equally likely, I would be most likely see a state where the cows were distributed equally on both sides of the field (as we would expect). Probabilistically, the system has evolved from an initial low entropy state (all cows on one side) to a higher entropy state (cows distributed on both sides).

Now, imagine instead of left and right side of a field, we have two containers of gas in thermal contact, which can freely exchange units of energy (which have replaced the cows in this scenario). Although now we have an extremely large number of energy units instead of just 10\footnote{Many 10s of orders of magnitude more...}, the same sort of logic applies; if I start in a macrostate with all of the energy units on one side (i.e. one gas container is very hot, and the other is not), with time, the system will tend to a macrostate of thermal equilibrium (approximately equal thermal energy on either side) as there are so many more microstates for the equilibrium temperature macrostate compared to a macrostate where one side has a lot more energy units than the other. Hence, we have come up with a statistical explanation for why heat tends to flow from hot objects to cold objects!

Note that this analysis reveals something interesting about the second law of thermodynamics; it's very much a statistical law, that tells us that a isolated system is (extremely) likely to tend to a high entropy state. Note that both in the cows and the heat example, there technically is a possibility that the system could temporarily return to a low entropy state (either with all the cows, or all the thermal energy units on one side); its just that this configuration is \textbf{extremely unlikely}. In fact, as we make our system larger and larger, the likelihood that the system could ever return to a low entropy state becomes more and more unlikely, and therefore in the limit of large/macroscopic systems, we can treat the second law as (basically) absolute.

This concludes our discussion of Science One Thermodynamics. If you enjoyed this part of the course, I would highly recommend taking PHYS 203 (Thermal Physics) next year, even if you decide not to go into a physics major\footnote{If you do go into physics, you won't have a choice!}. If you go into chemistry, you will also cover more thermodynamics material in the CHEM 205+304 sequence. Schroeder's "An Introduction to Thermal Physics" is a good text to consult for further reading on the subject (if you wanted to go beyond the material covered in your textbooks this year). We hope that even if you found this unit challenging at times, you can appreciate how we can learn so much about the workings of the universe from a couple (fairly simple) definitions and laws; if not, we hope that you take pride in the fact that you've made it through one of the most difficult units of Science One physics!


\documentclass[]{article}
\usepackage[margin=1.0in]{geometry}
\usepackage{amssymb}

%title material
\title{Astronomy 400B Homework \#4}
\author{Please Show Your Work for Full Credit}
\date{Due April 9, 2015 by 9:35am}

%include latex definitions
\input{astro400B_definitions.tex}

%begin the document
\begin{document}

%make the title, goes after document begins
\maketitle

\section{Sparke \& Gallagher Problem 7.1}

Suppose that gas atoms and galaxies in a group move at the same average random speed $\sigma$ along each direction. At temperature $T$, the average energy of gas particle is $3k_B T/2$, where $k_B$ is Boltzmann's constant.  If the gas is mainly ionized hydrogen, these particles are protons and electrons; show that, if the atom's kinetic energy $(3 m_p/2)\sigma^2$ is shared equally
between them, then
\begin{equation}
T \approx \frac{(m_p/2)\sigma^2}{k_B} \approx 5 \times 10^6 \left( \frac{\sigma}{300~\km~\s^{-1}}\right)^2~\K.
\end{equation}
\noindent
Hot gas in a group or cluster is usually close to this {\it virial temperature}.

\subsection{Solution}

If the atom's kinetic energy is shared between protons and electrons, then we should equate the thermal energy with half the kinetic energy.  This gives
\begin{eqnarray}
\frac{3}{2} k_B T &\approx& \frac{3}{4} m_p \sigma^2 \\
T &\approx& \frac{m_p}{2} \frac{\sigma^2}{k_B} = \frac{1.672622\times10^{-24}~\g}{2} \frac{\sigma^2}{1.380650\times10^{-16}~\erg~\K^{-1}}\\
T &\approx& 6.1\times 10^{-1} \frac{\sigma^2}{\cm^2~\s^{-2}}~\K \\
\therefore T &\approx& 5.5 \times 10^{6} \left(\frac{\sigma}{300~\km~\s^{-1}}\right)^2~\K
\end{eqnarray}

\section{Sparke \& Gallagher Problem 7.13}

If the lens $L$ is an object of mass $M_{\odot}$ at a distance $d_{\mathrm{Lens}}$ from
us, show that the Einstein radius for a star at distance $d_S = 2 d_{\mathrm{Lens}}$ is
\begin{equation}
\theta_{E} = \sqrt{\frac{R_s}{d_{\mathrm{Lens}}}} \approx 2 \times 10^{-3} \sqrt{\frac{1~\kpc}{d_{\mathrm{Lens}}}}~\mathrm{arcsec}.
\end{equation}

\subsection{Solution}

The Einstein radius is given by
\begin{equation}
\theta_{E} = \left( \frac{4 G M}{c^2} \frac{d_{LS}}{d_L d_S}\right)^{\frac{1}{2}} = \sqrt{ 2 R_s \frac{d_{LS}}{d_L d_S}},
\end{equation}
\noindent
where $R_s = 2 G M c^{-2}$ is the Schwarzschild radius (for the Sun, $R_s =  = 295343.16~\cm = 9.57\times10^{-17}~\kpc$. If $d_s = 2 d_L$, then $d_{LS} = d_L$. We then have
\begin{equation}
\theta_E = \sqrt{ 2 R_s \frac{d_{LS}}{d_L d_S}} = \sqrt{ 2 R_s \frac{d_L}{2 d_L^2}} = \sqrt{\frac{R_s}{d_L}} = \sqrt{9.57\times10^{-17} \frac{1~\kpc}{d_L}} = 9.78\times10^{-9}~\mathrm{rad}~\sqrt{\frac{1~\kpc}{d_L}}
\end{equation}
\begin{equation}
\therefore \theta_E  = 2.02\times10^{-3}~\mathrm{arcsec}~\sqrt{\frac{1~\kpc}{d_L}}.
\end{equation}

\section{Sparke \& Gallagher Problem 7.17}

If a lens at distance $d_{\mathrm{Lens}}$ bends the light of a much more
distant galaxy, so that $d_S$ and $d_{LS}\gg d_{\mathrm{Lens}}$, show that the
critical density is
\begin{equation}
\Sigma_{\mathrm{crit}} \approx 2 \times 10^4 \left( \frac{100~\Mpc}{d_{\mathrm{Lens}}}\right)~M_{\odot}~\pc^{-2},
\end{equation}
\noindent
and that the mass projected within angle $\theta_E$ of the center is
\begin{equation}
M(<\theta_E) \approx \left(\frac{d_{\mathrm{Lens}}}{100~\Mpc}\right)\left(\frac{\theta_E}{1~\mathrm{arcsec}}\right)^2 ~ 10^{10}~M_{\odot}.
\end{equation}

\subsection{Solution}
The critical surface density is
\begin{equation}
\Sigma_{\mathrm{crit}} = \frac{c^2}{4\pi G} \frac{d_S}{d_L d_{LS}}.
\end{equation}
\noindent
If $d_{LS}\gg d_L$, then $d_S \equiv d_{LS} + d_L \approx d_{LS}$.
Then we have
\begin{eqnarray}
\Sigma_{\mathrm{crit}} &\approx& \frac{c^2}{4\pi G} \frac{d_{LS}}{d_L d_{LS}}\\
\Sigma_{\mathrm{crit}} &\approx& \frac{c^2}{4\pi G} \frac{1}{d_L}\\
\Sigma_{\mathrm{crit}} &\approx& \frac{(3\times10^5~\km~\s)^2}{4\pi \left(4.301\times10^{-3}~\km^2~\s^2~\Msun^{-1}~\pc\right)} \frac{1}{d_L}\\
\Sigma_{\mathrm{crit}} &\approx& 1.67\times10^{12}\frac{1}{d_L}~\Msun~\pc^{-1}\\
\therefore \Sigma_{\mathrm{crit}} &\approx& 1.67\times10^{4}\left(\frac{100~\Mpc}{d_L}\right)~\Msun~\pc^{-2}.
\end{eqnarray}
\noindent
With the definition that $b_E = \theta_E d_L$, the projected mass is
\begin{eqnarray}
M(<\theta_E) = \pi b_E^2 \Sigma_{\mathrm{crit}}&=& \pi \left(\theta_E d_L\right)^2 \Sigma_{\mathrm{crit}}\\
M(<\theta_E) &=& \pi \left(\theta_E d_L\right)^2 \left[ 1.67\times10^{4}\left(\frac{100~\Mpc}{d_L}\right)~\Msun~\pc^{-2}\right]\\
M(<\theta_E) &=& \pi\times10^{16}~\pc^2 \theta_E^2 \left(\frac{d_L}{100~\Mpc}\right)^2 \left[ 1.67\times10^{4}\left(\frac{100~\Mpc}{d_L}\right)~\Msun~\pc^{-2}\right]\\
M(<\theta_E) &=& 5.25\times10^{10} \theta_E^2 \left(\frac{d_L}{100~\Mpc}\right)~10^{10}\Msun\\
M(<\theta_E) &=& 5.25\times10^{10} \left(4.85\times10^{-6}~\rad\right)^2 \left(\frac{\theta_E}{1~\mathrm{arcsec}}\right)^2 \left(\frac{d_L}{100~\Mpc}\right)~10^{10}\Msun\\
M(<\theta_E) &=& 1.23 \left(\frac{\theta_E}{1~\mathrm{arcsec}}\right)^2 \left(\frac{d_L}{100~\Mpc}\right)~10^{10}\Msun\\
\therefore M(<\theta_E) &\approx&  \left(\frac{\theta_E}{1~\mathrm{arcsec}}\right)^2 \left(\frac{d_L}{100~\Mpc}\right)~10^{10}\Msun\\
\end{eqnarray}

\end{document}


\section{Flow of Data}
\subsection{Characteristics}
\begin{itemize}
	\item Bandwidth: The volume of data per second, or bits per second. (eg. 40 Gbps capacity is 40 Gigabit per second)
	\item Latency: The time it takes for a particular bit or group of bits to travel through a network. Limited by speed of light.
	\item Error Rate: Can be caused by anything (cosmic rays, cable issues, etc.). Determines how much error checking and correction is justifiable.
\end{itemize}
\paragraph{Trade-Off Data}
Different applications have different requirements because we don't live in an ideal world. Voice programs can accept low bandwidth and high errors, but latency problems are a big no no. File transfer of operating system images is mostly concerned with reliability; you don't want a corrupted OS. The trade-offs are dependent on the application, so anything from speed, error or latency can be traded off for your desire purpose.
\paragraph{Circuit Switching}
Old phones were linked with physical wires, so the phone on one end was continuously connected to the speaker on other end. This was very inefficient; even if no audio was being transferred the line would be reserved. Multiplexing was very complicated and expensive using 1950s tech, so circuit switching was not a good method for networking. We can use packet switching in a circuit.
   \begin{figure}[!htb]
	\center{\includegraphics[width=9cm]
		{networks/packetswitching}}
	\caption{\label{fig:packetswitching} Packet switching in Time domain with 8 kbps lines into a 16 kbps line}
\end{figure}
\paragraph{Packet Switching}
Instead of having a continuous stream of data, we can divide it into smaller units called "packets". We can use these packets and switch them across the network to reach their destination. By splitting them into smaller units, you can multiplex the packets in the time domain, instead of the frequency domain. It is easier to build, but adds latency. You end up with a statistical gain in bandwidth because no resources are used while no data is being sent.
\paragraph{Virtual Circuits}
Virtual circuits aim to emulate physical circuits. Endpoints tell the network to establish a connection between them; this path is then given an identifier used throughout the network to route the emulated connections. Once A connection is no longer needed, it is "torn down" like a physical circuit. This method allows the network to know everything happening with the connection, and could be used to sort out packet issues. The finer details are removed from the end user with virtual circuits.
\paragraph{Datagrams}
Packets in the datagram method contain complete address information. There is no line connecting two endpoints, instead packets are sent off on a journey through the network space. All that happens is the network routes the packets using the data in the packet. All implementation and packet issues are dealt with at the endpoints. This is referred to as a datagram service.
\subsection{Layering}
A way to think of networks as having different layers of interactions. Each layer provides a service to those above it and makes use of services to those below it. There are two major models we looked at, OSI (a failure) and DOD (a success). We will focus on DoD as it is used everywhere.
   \begin{figure}[!htb]
	\center{\includegraphics[width=11cm]
		{networks/dodmodel}}
	\caption{\label{fig:dodmodel} End-to-End DoD Model}
\end{figure}
\paragraph{Process}
Processes or Applications are running code that do actual work, and the protocols they use. (eg. HTTP, ssh)
\paragraph{Host-to-Host}
Host-to-Host transport layer moves data between two end systems via a network whose topology and other properties the transport layer doesn't know much about. (TCP for streams, UDP for packets). This layer guarantees various properties; reliability, sequenced delivery, etc. This layer also deals with separating data between the appropriate applications.
\paragraph{IP}
This layer moves packets between end systems, but doesn't offer any guarantees. It handles choosing which link to get the data closer to its destination. This layer does not deal with separating data by applications. Uses IPv4/IPv6.
\paragraph{Link}
This layer moves data between one network element and the next element; concerned with packetisation, addressing, etc. Can have a variety of protocols in use (Ethernet primarily, ATM sometimes pops up)
\paragraph{Physical}
The physical layer is the actual encoding used to shift bits over a distance. (eg. Fibre Optics)
   \begin{figure}[!htb]
	\center{\includegraphics[width=13cm]
		{networks/packet}}
	\caption{\label{fig:packet} Layers encapsulated by packet structure}
\end{figure}

\subsection{Ethernet}
Network spaces can be divided as Local Area Networks (LAN), Wide Area Networks (WAN) WAN technology through most of its history was slow and unreliable. In terms of the lowest layer of networking, Ethernet is the most widely used. Ethernet was originally a bus; a single cable with computers connected throughout it. Packets were transferred through the cable; minimum size 64 bytes.
\paragraph{Ethernet}
The first set of bits on an ethernet cable is composed of alternating 1 and 0; this is because the clocks of computers can be horribly desynced (+- 50ppm), so the stream of 1 and 0 was used for clock syncing. There is also a method of calculating checksums continuously to find the end of a packet instead of just an end buffer. Ethernet's main issue is network collisions.
\paragraph{Collision Detection}
Ethernet is formally known as "CSMA CD" - Carrier Sense Multiple Access Collision Detection. Systems connected to Ethernet effectively share a line, each system reserving it while sending data. If more than one device is transmitting at a time, this results in packet collisions and causes packets to get corrupted. Collision is detected by the transmitted also listening on the cable; if there is a mismatch, collision detected. The system then sends a jamming signal alerting of a collision. The minimum packet size exists to ensure collisions are detected in the cable due to latency.
\paragraph{Recovery}
On the nth attempt of collision recovery, the system chooses a random number k from ${0..2^k}$ and delay k x 512 bit periods before trying again. After 10 attempts it simply gives up. Random generation isn't anything fancy, so long as its generation is different from other systems in the cluster.
\paragraph{Problems}
Collisions increase non-linearly with load, and the precise curve depends on the exact traffic mix. This sharp increase meant that adding more systems crippled performance. Latency was also unpredictable due to the collision system using random values, meaning it's impossible to be specific about latency.
\paragraph{Cables}
\begin{itemize}
	\item 10base5: 500m length cables with speed 10 MBps; heavy coax cables, expensive, difficult to work with
	\item 10base2: Thing coax, higher resistance but 185m length, instead of transceivers, simply T-piece
	\item 10baseT: Modern ethernet, modem internet, uses RJ45 connectors to a hub, originally Cat3 cables (think phone cables can now be used for networking)
\end{itemize}
   \begin{figure}[!htb]
	\center{\includegraphics[width=9cm]
		{networks/smartbridge}}
	\caption{\label{fig:smartbridge} Smart bridges increased network speed by allowing connections in different clusters to simultaneously function without interrupting each other}
\end{figure}
\paragraph{Ethernet Add-ons}
\begin{itemize}
	\item Repeater: An analogue amplifier, collisions can be seen on both sides
	\item (Smart) Bridge: Receives, buffers and transmits frames, so collisions are not propagated
	\item (Dumb) Bridge: Propagates everything, regardless of source and destination address being on one side
\end{itemize}


\chapter{Instrumental Variables}

05 - Causal Networks/chapters/09_instrumental_variables.tex


\subsection{Overview}

The traffic assignment component of this integrated modeling pipeline provides vehicles in the network specific paths to make trips between their origins and destinations (ODs). The origins, destinations, and numbers of vehicles making trips are received from ActivitySim and then assigned to specific network edges through traffic assignment. The resulting path assignment results in a total number of vehicles traveling on each edge in the transportation network, which is then used to compute a travel time on each edge. The travel time on an edge is a function of the number of vehicles using that edge. These travel times are then given back to UrbanSim and ActivitySim to be used for congested accessibility and skims.

\subsection{Inputs}

The primary inputs used by the traffic assignment model comprise the network (including the associated origin-destination travel demand) and BPR coefficients associated with each edge. These inputs were described in detail in Sections \ref{sec:activitysim} and \ref{sec:network} and reviewed briefly here.

\subsubsection{The network}

The network infrastructure is based on a set of nodes and edges in which connected edges share a node. Vehicles are allowed to travel from one edge to another if the edges are connected via a node. The ordered set of these edges (in which the ordering denotes a shared node between two edges) are called a path. In order for a vehicle to move from its origin to its destination, it must take a particular path. The possible paths between origins and destinations are defined by the topology of the network (i.e., the number of edges, the number of nodes, and the connections between these nodes and edges). The network described in Section \ref{sec:network} is used as the foundation of the traffic assignment model.

\subsubsection{BPR coefficients}

The BPR coefficients computed previously are used to assign a pseudo travel time to each edge as a function of the edge's load (i.e. number of vehicles on the edge). This is necessary in order to determine which paths vehicles should be assigned to (since vehicles are more likely to take a path which requires less time).

\subsection{How it works}

The currently implemented version of the traffic assignment model determines a static user equilibrium using the Frank-Wolfe algorithm. Static traffic assignment does not consider time varying parameters of any kind so there is no concept of flow dynamics but there is a well-defined equilibrium. Static user equilibrium, often called Wardrop's first principle in the transportation literature, is a well-defined state in which all vehicles take the shortest path from their origin to their destination. The resulting traffic assignment is based on shortest paths that are calculated while considering the travel times resulting from a loaded network in which each user is attempting to minimize their travel time.

Wardrop's first principle states that the actual travel time experienced by a user in the network is equal or less than the travel time that the same driver would experience on any other route \citep{wardrop1952road}. Static user equilibrium---which is equivalent to Nash equilibrium---is defined in Equation \ref{eq:static_user_equilibrium}:

\begin{equation}
    \label{eq:static_user_equilibrium}
    \begin{split}
    \min \sum_{e\in\ \textit{edges}}\int_{0}^{v_e} S_e(x) dx\\
    \text{subject to} \\
    v_e = \sum_i \sum_j \sum_r \alpha_{ij}^{er} x_{ij}^{r}\\
    \sum_r x_{ij}^{r} = T_{ij}\\
    v_e \geq 0 \\
    x_{ij}^{r} \geq 0
    \end{split}
\end{equation}

The minimization is over travel time on each edge as a function of traffic volume. $x_{ij}^{r}$ is the number of vehicles on path $r$ from node $i$ to node $j$. $\alpha_{ij}^{ar}$ is equal to 1 if edge $a$ is contained in path $r$ and 0 otherwise. To perform this static user equilibrium calculation we use the Frank-Wolfe algorithm \citep{frank1956algorithm}. The Frank-Wolfe algorithm is a first-order optimization algorithm that is used to solve convex problems such as the static user equilibrium problem defined above. The algorithm works as follows; consider a problem of the form:

\begin{align*}
    S \in \mathcal{R}^n \text{ a polyhedron and } f: \mathcal{R}^n \rightarrow \mathcal{R} C^1, \text{ solve:}
\end{align*}

\begin{align*}
    \text{min. } f(x) \text{ subject to } x \in S
\end{align*}

Frank-Wolfe algorithm:

\begin{enumerate}
    \item Initialize with $x_0 \in S$ and let $k= 0$
    \item Determine a search direction $d_k = y_k - x_k$ by solving the linear program: \\
    $y_k \in \text{argmin}_{y \in S} \{ \nabla f(x_k)^Ty \} $
    \item Determine a step length $\alpha_k \in [0.1]$ such that: \\
    $f(x_k + \alpha_k d_k) = f((1 - \alpha)x_k + \alpha y_k) \leq f(x_k)$
    \item Update $x_{k+1} = (1 - \alpha)x_k + \alpha y_k$ let $k = k+1$ and go to step 2
\end{enumerate}

\subsection{Outputs}

From the traffic assignment model, we obtain the number of vehicles on each edge in the network and the resulting travel time (calculated using the BPR coefficients) of each edge. Since these travel times are a result of a static traffic assignment, they are not interpretable as exact travel times and the units are effectively meaningless. Rather, they represent the congested travel time of each edge calculated using the BPR equations.

As described previously, these BPR function are empirical. These travel times are used to generate congested skims and accessibility, in which the congestion on each edge in the network is known in relative terms, and therefore areas of relatively high versus low congestion can be specified. UrbanSim and ActivitySim use these congested edge travel times to calculate accessibility and skims. For these calculations, the precise definition of travel time is not necessary; the comparison between edges is the important component.

\subsection{Calibration and validation}

In its current implementation, the traffic assignment model is static, meaning that back-propagation of congestion and other time dependent phenomena observed in real transportation networks cannot be well-modeled. However, the resulting travel times due to loading on each edge can be compared with real data to understand how well the static approximation matches with the observed travel times on a network.


% TEMPLATE FOR STANDARD QUIZ
% by laura

\documentclass[11pt,epsfig]{article}
\usepackage{amsmath}
\usepackage{mdframed}
\usepackage{graphicx}
\usepackage{multirow}
\def\infinity{\rotatebox{90}{8}}
\oddsidemargin=0in
\evensidemargin=0in
\textwidth=6.3in
\topmargin=-0.5in
\textheight=9in
\renewcommand{\arraystretch}{2}
\parindent=0in
\pagestyle{empty}

\input{notestpoints}

\begin{document}

%%%(change to appropriate class and semester)
\textbf{Notes for Chapter 7: CIS 2033 Fall 2015 Computational Prob and Stat}

%%%(change to appropriate quiz type and date)
\textbf{Shanshan Zhang {(tuf14438@temple.edu)} }

% problem
\section{Chapter 7: Expectation and Variance}
\subsection{Definition}
\begin{tabular}{|l|}
\hline
\textbf{Definition:}\\
$E[X]= \begin{cases} \sum_xxP(x), & X~ \text{is discrete}\\
\int_x xf(x)dx, &X～ \text{is continuous} \end{cases},Var[X]= E[(X-E[X])^2]$\\
\hline
\end{tabular}

\subsubsection{Important expectation and variance}
\begin{tabular}{ |l|l|l|l| }
%\caption{Important Expectation and Variance}
\hline
& \bf{Notation }& \bf{$E[X]$} & \bf{$Var[X]$}\\ \hline
\multirow{3}{*}{Discrete} & $X\sim Ber(p)$ & $p$ & $p(1-p)$\\ \cline{2-4}
 & $X\sim Bin(n,p)$ & $np$& $np(1-p)$\\  \cline{2-4}
 & $X\sim Geo(p)$ & $\frac{1}{p}$& $\frac{1-p}{p^2}$\\ \cline{2-4}
 & $X\sim Pois(\mu)$ & $\mu$ & $\mu$\\ \hline
\multirow{4}{*}{Continuous} & $X\sim Unif(a,b)$ & $\frac{a+b}{2}$& $\frac{(b-a)^2}{12}$\\ \cline{2-4}
 & $X\sim Exp(\lambda)$ & $\lambda^{-1}$& $\lambda^{-2}$\\ \cline{2-4}
 & $X\sim Par(\alpha)$ & $\frac{\alpha}{\alpha-1}$& $\frac{\alpha}{(\alpha-1)^2(\alpha-2)},for ~\alpha >2$\\ \cline{2-4}
& $X\sim N(\mu,\sigma^2)$&$\mu$&$\sigma^2$\\ \hline
\end{tabular}

\subsection{Change of variables}
Exercise 7.3, 7.4 \\
\begin{tabular}{|l|}
\hline
\textbf{Definition:} $E[g(X)] =\begin{cases} \sum_xg(x)P(X=x), & X~ \text{is discrete}\\ \int_x g(x)f(x)dx, &X～ \text{is continuous} \end{cases};Var[g(X)]= E[(g(X)-E[g(X)])^2]$\\
\textbf{Useful formulas:}: $Var[X]=E[X^2]-(E[X])^2;E[aX+b] = aE[X]+b; Var[aX+b] = a^2Var[X] $\\
\hline
\end{tabular}
\end{document}


\documentclass{article}
\usepackage[minionint,mathlf,textlf]{MinionPro} % To gussy up a bit
\usepackage[margin=1in]{geometry}
\usepackage{graphicx} % For .eps inclusion
%\usepackage{indentfirst} % Controls indentation
\usepackage[compact]{titlesec} % For regulating spacing before section titles
\usepackage{adjustbox} % For vertically-aligned side-by-side minipages
\usepackage{array, mathrsfs, mhchem, amsmath} % For centering of tabulars with text-wrapping columns
\usepackage{hyper ref}
\usepackage[autolinebreaks,framed,numbered]{mcode}
\pagenumbering{gobble}
\setlength\parindent{0 cm}
\begin{document}
\large

\section*{Introduction}
\begin{itemize}
\item In our last lecture, you learned how to compose a system's chemical master equation and exploit it to determine something about the state probabilities, including means, variances, and (estimated) distributions of the variables.
\item These properties of the probability distribution, however, do not provide us with sample trajectories. If there are multiple behaviors occurring, the probability distribution and its moments will only give us information about them in aggregate.
\item Today we will introduce the Gillespie algorithm, one of the most famous stochastic simulation techniques. Before beginning, however, it is necessary to introduce some statistics to motivate how the algorithm works.
\end{itemize}

\section*{Point processes}
A point process can be thought of as a ``random set of points" in either time or space: the number of points as well as their time/location vary between realizations. Point processes are appropriate descriptors of many biological phenomena with discrete events, including action potentials, births/deaths, and chemical reactions. Because of the variable number of ``points" in a realization, their probability distributions are described a bit differently from the usual. We must define a series of functions $P_s(\tau_1, \ldots, \tau_s)$, each of which represent the probability that $s$ events occurred \textit{and} they occurred at the times $\tau_i$. If the $\tau_i$ are ordered, then the normalized probability distribution may be written as:
\[ P_0 + \int_{-\infty}^{\infty} P_1(\tau_1) \, d\tau_1 +  \int_{-\infty}^{\infty}  \left[ \int_{\tau_1}^{\infty} P_2(\tau_1,\tau_2) \, d\tau_2 \right] \, d\tau_1 + \ldots = 1 \]
This normalization sums over all the possible numbers of events and all of the combinations of $\tau_i$ that can occur for a given number of events. Suppose we relax the restriction that the $\tau_i$s be ordered: then, for any set $\{ \tau_1, \ldots, \tau_s\}$, there are $s!$ permutations. If we require that each function $P_s$ is symmetric in its variables, then we can rewrite the normalization above in a handier form:
\begin{eqnarray}
P_0 + \sum_{s=1}^{\infty} \frac{1}{s!} \int_{\mathbb{R}^{s}} P_s(\tau_1,\ldots,\tau_s) \, d\tau_1 \, \ldots d\tau_s = 1 \label{eqn:normpoint}
\end{eqnarray}
Notice that we have accounted for the expansion of the limits of integration by dividing each term in the sum by the number of possible permutations of the $\tau_i$.\\

A natural first question is to ask of a point process is how to determine the average number of events that occur within an interval $(t_a,t_b)$.To address this, we introduct the indicator function
\[ f(t)= \left\{
     \begin{array}{lr}
       1 & : t \in (t_a,t_b)\\
       0 & :\textrm{otherwise}
     \end{array}
   \right. \]
Then if $s$ events occur at times $\tau_k$,  the number that occur within this interval is $N = \sum_{k=1}^s f(\tau_k)$. The average number of events within this interval is therefore:
\begin{eqnarray*}
\left< N \right> & = & \left< \sum_{k=1}^{s} f(\tau_k) \right>\\
& = & \sum_{s=1}^{\infty}  \frac{1}{s!} \int_{\mathbb{R}^{s}} \left( \sum_{k=1}^{s} f(\tau_k) \right) P_s(\tau_1,\ldots,\tau_s) \, d\tau_1 \, \cdots d\tau_s
\end{eqnarray*}

To simplify this expression, we note that we can rewrite it as follows:
\begin{eqnarray*}
\left< N \right> & = & \sum_{s=1}^{\infty}  \frac{1}{s!} \sum_{k=1}^{s} \int_{\mathbb{R}^{s}}  f(\tau_k) P_s(\tau_1,\ldots,\tau_s) \, d\tau_1 \, \cdots d\tau_s\\
& = & \sum_{s=1}^{\infty}   \frac{1}{s!}  \left[ \int_{\mathbb{R}^{s}}  f(\tau_1) P_s(\tau_1,\ldots,\tau_s) \, d\tau_1 \, \cdots d\tau_s + \int_{\mathbb{R}^{s}}  f(\tau_2) P_s(\tau_1,\ldots,\tau_s) \, d\tau_1 \, \cdots d\tau_s+ \ldots \right]
\end{eqnarray*}

Since we have required that the functions $P_s$ be symmetric in their variables, each of these integrals is equal:
\begin{eqnarray*}
\left< N \right>  & = & \sum_{s=1}^{\infty} \frac{1}{s!}  s \int_{-\infty}^{\infty} f(\tau_1) \left[  \int_{\mathbb{R}^{s-1}}  P_s(\tau_1,\ldots,\tau_s) \, d\tau_2 \, \cdots d\tau_s \right] \, d \tau_1\\
& = & \sum_{s=1}^{\infty} \frac{1}{(s-1)!}  \int_{t_a}^{t_b}  \left[  \int_{\mathbb{R}^{s-1}}  P_s(\tau_1,\ldots,\tau_s) \, d\tau_2 \, \cdots d\tau_s \right] \, d \tau_1\\
\end{eqnarray*}
In the second line, we have used the definition of the indicator function to change the limits of integration for one variable.

\section*{Poisson processes}

Consider the special case where events occur independently of one another, e.g., the fact that one event occurred at time $t=\tau_1$ does not influence the probability that another event occurred at time $t=\tau_2$. This idealization is not well-suited to action potentials in neurons owing to their refractory period, but is appropriate for the chemical reactions which we would like to simulate. In this special case, the $P_s$ can be factorized in the form
\[ P_s(\tau_1,\ldots, \tau_s) = e^{-v} P(\tau_1) \cdots P(\tau_s), \hspace{3 cm}  v = \int_{-\infty}^{\infty} P(\tau) \, d\tau \]
where the normalization constant follows from equation \ref{eqn:normpoint}. Plugging this into the formula above allows us to simplify somewhat:
\begin{eqnarray}
\left< N \right>  & = & e^{-v} \sum_{s=1}^{\infty} \frac{1}{(s-1)!}  \int_{t_a}^{t_b}  \left[  \int_{\mathbb{R}^{s-1}}  P(\tau_1)\cdots P(\tau_s) \, d\tau_2 \, \cdots d\tau_s \right] \, d \tau_1 \nonumber \\
& = & e^{-v} \sum_{s=1}^{\infty} \frac{1}{(s-1)!}  \int_{t_a}^{t_b}  P(\tau_1) \left[  \int_{\mathbb{R}^{s-1}}  P(\tau) \, d\tau \right]^{s-1} \, d \tau_1 \nonumber \\
& = & e^{-v} \int_{t_a}^{t_b} P(\tau_1) \, d \tau_1 \left( \sum_{s=1}^{\infty} \frac{1}{(s-1)!}   \left[  \int_{-\infty}^{\infty} P(\tau) \, d\tau \right]^{s-1} \right) = e^{-v} \int_{t_a}^{t_b} P(\tau_1) e^v \, d \tau_1 \nonumber \\
& = & \int_{t_a}^{t_b} P(\tau) \, d \tau \label{eqn:avgpoisson}
\end{eqnarray}
It is no harder to show by a similar method that for this type of point process,
\[ \sigma_N^2 = \left< N^2 \right> - \left< N \right>^2 = \left< N \right> \]
As you know, equivalence of the mean and variance are properties of the Poisson distribution. To learn more about the probability distribution of $N$, the number of events occurring in an interval $(t_a,t_b)$, we will find its characteristic function. A characteristic function $G(k)$ for a variable $x$ is defined as the average value of $e^{ikx}$:
\[ G(k) =\left< e^{ikx} \right> = \int_{-\infty}^{\infty} e^{ikx} P(x)\, dx \]
For a random variable $x$ that can only take on integer values,
\begin{eqnarray}
 P(x=n) = \sum_{n \in \mathbb{Z}} p_n \delta(x-n) \implies G(k) = \sum_{n \in \mathbb{Z}} p_n e^{ikn} \label{eqn:discretechar}
 \end{eqnarray}
Our approach will be to find an expression for the characteristic function of $N$ and compare the powers of $e^{ikn}$ to find an expression for $p_n$, the probability that $N=n$:
\begin{eqnarray}
\left< e^{ikN} \right> & = & \sum_{s=0}^{\infty} \frac{1}{s!} \int_{\mathbb{R}^{s}} e^{ikN} P_s(\tau_1,\ldots,\tau_s) \, d\tau_1 \, \ldots d\tau_s =  \sum_{s=0}^{\infty} \frac{1}{s!} \int_{\mathbb{R}^{s}} e^{ik\sum_{j=1}^s f(\tau_j)} P_s(\tau_1,\ldots,\tau_s) \, d\tau_1 \, \ldots d\tau_s \nonumber  \\
& = & e^{-v} \sum_{s=0}^{\infty} \frac{1}{s!} \int_{\mathbb{R}^{s}} e^{ik f(\tau_1)} P(\tau_1) \cdots e^{ik f(\tau_s)} P(\tau_s) \, d\tau_1 \, \ldots d\tau_s  = e^{-v} \sum_{s=0}^{\infty} \frac{1}{s!} \left[ \int_{-\infty}^{\infty} e^{ikf(\tau)} P(\tau) \, d\tau \right]^s \nonumber \\
& = & \exp \left[\int_{-\infty}^{\infty} e^{ikf(\tau)} P(\tau) \, d\tau  - \int_{-\infty}^{\infty} P(\tau) \, d\tau \right] = \exp \left[  \left( e^{ik} - 1 \right) \int_{t_a}^{t_b} P(\tau) \, d\tau \right] \nonumber \\
& = & \exp \left[  \left( e^{ik} - 1 \right)  \left< N \right> \right] = e^{-\left< N \right>} \sum_{k=0}^{\infty} \frac{\left< N \right>^k e^{ikN}}{k!} \label{eqn:charpp}
\end{eqnarray}
Comparing equations \ref{eqn:discretechar} and \ref{eqn:charpp}, we find that:
\[ p(N=n) =   \sum_{n=0}^{\infty} \frac{\left< N \right>^n e^{-\left< N \right>}}{n!}, \hspace{1cm} \textrm{ which is the Poisson probability density function with } \lambda = \left< N \right>\]
It is hopefully now clear why point processes with independent events are called Poisson processes.

\section*{Waiting times}
We now know enough about the statistics of chemical reactions to begin calculating quantities necessary for our simulations. In particular, we will want to know how much time passes between reactions, i.e., the waiting time. Since nothing of interest is happening during the waiting time, the chemical reaction rate has some constant value $k$ during this period. The average number of events we expect in some period $(0,t)$ over which $k$ is effectively unchanged will be given by equation \ref{eqn:avgpoisson}:
\[ \left< N \right> = \int_0^t k \, d\tau = kt \]
If we denote the time at which the first reaction occurs by $T$, then
\begin{eqnarray*}
P(T > t) & = & P(\textrm{zero reactions occur in the time interval [0,$t$]})\\
& = & \frac{(kt)^0 e^{-kt}}{0!} = e^{-kt}
\end{eqnarray*}
This gives us a formula for the cumulative density function of $T$:
\[ P(T <  t) = 1 - e^{-kt} \]
To find the probability density function, we take the derivative of the cumulative density function with respect to $t$:
\[ P(T = t) = k e^{-kt} \]
This means that $T$ follows an \textit{exponential distribution}. If we wanted to simulate when the next reaction would occur, we could just draw a random number from an exponential distribution (which MATLAB and any other self-respecting statistics package will allow us to do). Be wary, however, as some statistics packages -- including MATLAB -- define exponential distributions like this:
\[ P(X = x) = \frac{ e^{-t/\lambda}}{\lambda} \]
so double-check whether you need to take the reciprocal when supplying the parameter to the exponential random number generator.\\

To simulate systems with more than one reaction, we need to know when the next of many possible reactions will happen. (We also want to know which type of reaction it was, but as we will discuss later, we can worry about that problem separately.) In other words, if each reaction $i$ is occuring at a known rate $k_i$, the we want to know the distribution of times until the next reaction occurs, i.e.,  the distribution of $T_0 = \min_{i} T_i$, where each $T_i$ is exponentially-distributed with its own parameter $k_i$. Since all reactions are assumed to occur independently,
\[ P\left( T_0 > t \right) = P\left( T_1 > t \right)\cdots P\left( T_m > t \right) = \prod_i  \exp \left( -k_i t \right) = \exp \left( -t \sum_i k_i \right) \]

Following the same math we used above to convert this to a probability density function, we find that the time until the first reaction occurs ($T_0$) is therefore exponentially-distributed with parameter $k = \sum_i k_i$. In other words, once we have calculated the rates for all reactions in the system, we can simulate the time at which the next reaction will occur by drawing a number from an exponential distribution with parameter $\sum_i k_i$.\\

Now we return to the problem of deciding which reaction will occur next. Reaction $i$ is next if $T_i = T_0 < \min_{j\neq i} T_j$. By ansatz, I'll provide the solution:
\[ P(T_i = T_0) = \frac{k_i}{\sum_j k_j} \]
We'll prove this by induction, a method you'll use on problem 2b of problem set 7. The idea is to prove that the statement is true for a ``base case," then show that if it is true for a certain number of reactions $n-1$ then it is true for $n$ reactions as well. By extension the formula will hold for any number of reactions greater than the base case. For our base case, we consider a system with one reaction -- then:
\[ P(T_i = T_0) = \frac{k_1}{k_1} = 1 \]
That was the easy part. Next, we assume that the formula has been found to hold for a system with $n-1$ reactions. Let's define $Y_0 = \min_{i<n} T_i$ to be the minimum time until one of the first $n-1$ reactions occurs We know from the discussion above that $Y_0$ is exponentially-distributed with effective rate $k = \sum_{i=1}^{n-1} k_i$. We want to know the probability that the $n$th reaction occurs first, i.e.:
\begin{eqnarray*}
P(T_n = T_0) = P(T_n < Y_0) & = & \int_0^{\infty} \int_0^{Y_0}  k k_n e^{-k_n T_n}e^{-k Y_0} dT_n \, dY_0 \\
& = & k k_n  \int_0^{\infty} e^{-k Y_0} \left[ - \frac{e^{-k_n T_n}}{k_n} \right]_0^{Y_0} \, dY_0\\
& = & k  \int_0^{\infty} e^{-\left( k + k_n \right) Y_0} \, dY_0  - k_n \int_0^{\infty} e^{-k Y_0} dY_0\\
& = & 1 - \frac{k}{k + k_n} = \frac{k_n}{k + k_n} = \frac{k_n}{\sum_{i=1}^n k_i} \\
\end{eqnarray*}
This completes the induction step. To simulate which reaction will occur next, we normalize the rates to $k_i^*$ so that their total sum is one, then pick a random number uniformly-distributed between 0 and 1. This number will fall into a ``bin"  $\ell$ whose endpoints are $(\sum_{i=1}^{\ell - 1} k_{\ell}^*, \sum_{i=1}^{\ell} k_{\ell}^*)$: we will therefore say that reaction $\ell$ has occurred. In implementation, the bins can remain figurative, since we can call the MATLAB function \mcode{randsample()} with our array of $k_i$s to get a random reaction index.

\section*{Gillespie's Stochastic Simulation Algorithm (SSA)}

We now have all of the information we need to motivate the Gillespie SSA. In his original paper/presentation (1976), Gillespie proposed two alternative methods, then showed that their assumptions were equivalent (though one algorithm, the ``direct method," was more efficient). I will present first the more intuitive ``first reaction method".\\

For both methods, the simulation begins with specified initial numbers of all molecular species, an initial time (wlog, $t=0$), and a termination condition to prevent the algorithm from going on indefinitely. Most folks store e.g. the times and concentrations in a large array with e.g. one row per $n$ rounds: in this case it is wise to pre-allocate the array in memory and to set the termination condition to when the array will be full. If you write directly to disk you might easily choose a more intricate termination condition.\\

What happens in each round, however, differs between the two methods. Each round of the first reaction method simulation has four main parts:
\begin{enumerate}
\item After checking the termination condition, the rates for this time step are calculated: these typically depend on the current molecule numbers but may also be time-dependent (for example, we can cause production of a molecule to begin at time $t_0$).
\item For each reaction $i$, a time $T_i$ until its next reaction is drawn from an exponential distribution with rate $k_i$.
\item The smallest of the $T_i$s is identified: this $T_i$ is the time elapsed during the step, and the reaction corresponding to this $T_i$ is the one which occurs in this round.
\item The current time $t$ and the number of molecules of each species are updated accordingly. Commonly a stoichiometry matrix is used to facilitate the molecule count updates. The updated values are stored if desired. The remaining $T_i$s and the calculated rates $k_i$ are erased. (At least some of them will not apply in the next round since some molecule numbers will have changed.)
\end{enumerate}

Many people find this algorithm intuitive. But as I hinted above, we can find the time until the first of $n$ reactions occurs without having to generate all of the $T_i$ and calculate their minimum. Furthermore, the time until the next reaction occurs can be selected independently of selecting which reaction occurs next! To see why, consider the probability that the next reaction is $i$ \textit{and} this reaction occurs at time $t$. The first reaction method asserts that this probability is:
\begin{eqnarray*}
P(\textrm{reaction $i$ next, at time $t$}) & \propto & P(T_i = t)  \prod_{j\neq i} P(T_j > t)\\
& = & k_i e^{-k_i t}   \prod_{j\neq i} e^{-k_j t}\\
& = & k_i \exp \left(-t \sum_i k_i \right)
\end{eqnarray*}

The direct method, by contrast, assumes that the probabilities of the two events are independent and follow the distributions we spent time calculating earlier in this lecture:
\begin{eqnarray*}
P(\textrm{reaction $i$ next, at time $t$}) & \propto & P(\textrm{reaction $i$ next}) \times P(\textrm{next at time $t$})\\
& = & \left( \frac{k_i}{\sum_i k_i} \right) \times \left[ \sum_i k_i \right] \exp \left[ -t \sum_i k_i\right] \\
& = & k_i \exp \left(-t \sum_i k_i \right)
\end{eqnarray*}
The results, of course, are equivalent. We will see that the direct method, however, requires fewer random number generations and comparison operations. The four main steps in each round of the direct method are (omitting unnecessary detail given above):
\begin{enumerate}
\item After checking the termination condition, the rates for this time step are calculated.
\item A time until the next reaction occurs is randomly generated from an exponential distribution with parameter $\sum_i k_i$.
\item The next reaction to occur is chosen using \mcode{randsample()} (or similar) as described above.
\item The current time $t$ and the number of molecules of each species are updated accordingly.
\end{enumerate}

\section*{Supplemental: Next Reaction Method}
\begin{itemize}
\item One of the more time-consuming steps in the Gillespie algorithm is the update of rates with each time step. Typically most of the rates will not change at all, and the few that do change won't change by much, in between two time steps.
\item The \textit{next reaction method} is a modification of the first reaction method. However, it attempts to retain the reaction rates and randomly-generated time of the next reaction ($\tau_i$) wherever possible, i.e., whenever the rate of a reaction is not affected by the reaction that occurred last.
\item You should be wary of reusing random numbers in general, but for this particular method it is proven to be legitimate (Gibson and Bruck, 2000). The next reaction method uses absolute time rather than relative time.
\item To improve efficiency, the suggested implementation uses data structures you may not have encountered previously. These are:
\begin{itemize}
\item A dependency graph, indicating which reactions needs to have their rates/times updated if reaction $i$ occurs because the concentration of one or more reactants has changed (usually represented by a simple adjacency matrix), and
\item An indexed heap (priority queue) allows the smallest $\tau_i$ to be easily identified, and makes updating the $\tau_i$s rather fast.
\end{itemize}
\item The algorithm initiates by setting $t=0$, specifying the initial molecule numbers, determining the initial rates, and generating the $\tau_i$s.
\item In each step:
\begin{enumerate}
\item After checking the termination condition, the reaction with the smallest $\tau_i$ is picked off the heap and chosen to be the reaction that occurs next.
\item  $\mathbf{X}$ and $t$ are updated based on the reaction that occurs.
\item The dependency graph is consulted to determine which rates need to be recalculated and which $\tau_i$s need to be regenerated.
\end{enumerate}
\end{itemize}

\end{document}

lectures/Lecture 22 - Stochastic Simulation/lecture 22 notes.tex