Common Lagrangian in introductory courses of classical mechanics depends of $q$
and $\dot{q}$. We refer to them as \emph{first-order Lagrangian} because the
highest derivative term is $\dot{q}$, i.e.\ the first derivative of $q(t)$ with
respect to time. If in the Lagrangian appears also $\ddot{q}$ it will be a
\emph{second-order Lagrangian} and so on. \\

In the following, the mathematical machinery develop in Section~\ref{section:
lagrangin and hamiltonian formalism}, will be used to study two apparently
similar Lagrangian and introduce the core problem of HD systems.

\subsection{First-order vs Second-order Lagrangian}
The simplest non-trivial Lagrangians are those in polynomial form like
\begin{equation*}
  L_1(q, \dot{q}) = \frac{\dot{q}^2}{2} - \alpha(q) \qquad
  L_2(q, \ddot{q}) = \frac{\ddot{q}^2}{2} - \alpha(q)
\end{equation*}
where $\alpha$ is a smooth function of $q$ only~\cite{Chen13}. No Greek indices
appear on $q^{(i)}$ because we limit ourself to one-dimensional systems. $L_1$
is a first-order Lagrangian while $L_2$ it's a second-order one.

\paragraph{First-order Lagrangian} In order to get the equation of motion we
used the Euler-Lagrange equation~\eqref{eq:euler-lagrange}

\begin{equation} \label{eq: first-order_motion_eq_lagr}
  \frac{\partial L_1}{\partial q} -
  \frac{d}{dt}\frac{\partial L_1}{\partial \dot{q}} +
  \underbrace{
  \frac{d^2}{dt^2}\frac{\partial L_1}{\partial \ddot{q}} - \ldots }_0 = 0
  \qquad \Rightarrow \qquad
  \ddot{q} = - \frac{d\alpha(q)}{dq}
\end{equation}

If $q$ is the position coordinate of a unit mass point, the function $\alpha(q)$
can be interpreted as the potential energy and~\eqref{eq:
first-order_motion_eq_lagr} is nothing less than Newton's second law. To get the
Hamiltonian description apply the recipe in Section~\ref{subsection:
hamiltonian_formalism}

\begin{equation*}
  \begin{cases}
    Q_1 := q \\
    P_1 := \frac{\delta L_1}{\delta \dot{q}} = \dot{q}
  \end{cases}
  \qquad
  \tilde{H}_1(q, \dot{q}) :=
  \frac{\delta L_1}{\delta \dot{q}} \dot{q} - L_1(q, \dot{q})
\end{equation*}

\begin{equation} \label{eq: first-order_motion_eq_ham}
  H_1(Q_1, P_1) = \frac{P_1^2}{2} + \alpha(Q_1) \qquad
  \begin{cases}
    \dot{Q_1} =   \frac{\partial H_1}{\partial P_1} = P_1 \\
    \dot{P_1} = - \frac{\partial H_1}{\partial Q_1} = - \frac{d\alpha(Q_1)}{dQ_1}
  \end{cases}
\end{equation}

Since $L_1$ does not explicitly depend on time (only through $q$ and $\dot{q}$),
the Hamiltonian $H_1$ can be interpreted as the total energy of the system (sum
of kinetic and potential energy). Combining the two equation in~\eqref{eq:
first-order_motion_eq_ham} we restore~\eqref{eq: first-order_motion_eq_lagr}
proving the equivalence between Lagrangian and Hamiltonian mechanics for simple
systems describe by $L_1$.

\paragraph{Second-order Lagrangian} Proceeding as before, equation of motion
follow from Euler-Lagrange equation~\eqref{eq:euler-lagrange}

\begin{equation} \label{eq: second-order_motion_eq_lagr}
  \frac{\partial L_2}{\partial q} -
  \frac{d}{dt}\frac{\partial L_2}{\partial \dot{q}} +
  \frac{d^2}{dt^2}\frac{\partial L_2}{\partial \ddot{q}} -
  \underbrace{
    \frac{d^3}{dt^3}\frac{\partial L_2}{\partial q^{(3)}} + \ldots
  }_0 = 0
  \quad \Rightarrow \quad
  q^{(4)} = - \frac{d\alpha(q)}{dq}
\end{equation}

Equation~\eqref{eq: second-order_motion_eq_lagr} never appear in classical
mechanics, but can it describe some kind of system? The study of the Hamiltonian
can give us some clues about the total energy of the system.

\begin{equation*}
  \begin{cases}
    Q_1 := q \\
    P_1 := \frac{\delta L_2}{\delta \dot{q}}
         = \frac{\partial L_2}{\partial \dot{q}} -
           \frac{d}{dt} \left( \frac{\partial L_2}{\partial \ddot{q}} \right)
         = 0 - \frac{d}{dt} \left( \ddot{q} \right) = -q^{(3)}
  \end{cases}
  \begin{cases}
    Q_2 := \dot{q} \\
    P_2 := \frac{\delta L_2}{\delta \ddot{q}}
         = \frac{\partial L_2}{\partial \ddot{q}} = \ddot{q}
  \end{cases}
\end{equation*}

Therefore
\begin{equation}\label{eq:canonical_coordinates_second_order_lagrangian}
  \begin{cases}
    q        = Q_1 \\
    \dot{q}  = Q_2 \\
    \ddot{q} = P_2
  \end{cases}
  \qquad
  \begin{cases}
    \frac{\delta L_2}{\delta \dot{q}}  = P_1 \\
    \frac{\delta L_2}{\delta \ddot{q}} = P_2
  \end{cases}
\end{equation}

\begin{equation*}
  \tilde{H}_2(q, \dot{q}) :=
    \frac{\delta L_2}{\delta \dot{q}} \dot{q} +
    \frac{\delta L_2}{\delta \ddot{q}} \ddot{q} -
    L_2(q, \ddot{q})
    = \frac{\delta L_2}{\delta \dot{q}} \dot{q} +
    \frac{\delta L_2}{\delta \ddot{q}} \ddot{q} -
    \frac{\ddot{q}^2}{2} + \alpha(q)
\end{equation*}

Substituting relations~\eqref{eq:canonical_coordinates_second_order_lagrangian}
in the expression for $\tilde{H}_2$ we get the Hamiltonian

\begin{align} \label{eq: second-order_motion_eq_ham}
  H_2(Q_1, Q_2, P_1, P_2)
    &= P_1 Q_2 + P_2 P_2 - \frac{P_2^2}{2} + \alpha(Q_1) \notag \\
    &= P_1 Q_2 + \frac{P_2^2}{2} + \alpha(Q_1)
\end{align}

There is a significant difference between the two spectra of $H_1$ and $H_2$:
the first is bounded from below while the latter is not (and both are not
bounded from above).

\subsection{Linear Ostrogradskian instability}~\label{subsection:
linear_ostrogradskian_instability}
Consider an \emph{isolated} system describe by the Hamiltonian $H_2$; if the
energy is conserve even though the spectra is not bounded the energy stay
constant. Things start to going wrong when ones consider \emph{interacting}
system (e.g. $H_1$-system interacting with $H_2$-system). The phase space of
$H_2$-system infinitely extend where the Hamiltonian is negative. In the example
this is due to the \emph{linear} term $P_1$. When two systems interact the
$H_1$-system tends to occupy higher and higher energies states while the
$H_2$-system fall lower and lower in negative energies; this behavior entails
the conservation of energy~\cite{Kallosh08, Eliezer89}.
This is the so called \emph{linear Ostrogradskian instability}. \\

One can encounter the same kind of instability in quantum mechanics when try to
canonically quantized higher order Hamiltonian (e.g. $H_2$). This process lead
to \emph{negative norm states} (or \emph{negative energy states}) which are
often called \emph{``ghosts''}. Systems (even classical ones) involving this
type of ghosts are called \emph{ghost-like} or \emph{ghost-ridden} systems. \\

One may wonder when these ghosts are summoned; the answer is contains in the
Ostrogradsky theorem

\begin{theorem}[Ostrogradsky]\label{th:ostrogradsky_classical}
  If in the Lagrangian~\eqref{eq:general_lagrangian} $n \geq 2$ and the
  canonical momentum $\bm{P}_n$ does not vanish, the
  Hamiltonian~\eqref{eq:general_hamiltonian} may acquire an arbitrary real
  value.
\end{theorem}
\begin{proof}
  The proof is given for a one-dimensional system (i.e. $\mu=1$) but it is
  generalize straight to higher dimensions. Consider the
  Hamiltonian~\eqref{eq:general_hamiltonian} in terms of conjugate coordinates
  \begin{equation} \label{eq:general_hamiltonian_1dim}
    H = P_n h + P_{n-1} Q_n + \cdots + P_1 Q_2 - L(Q_1, Q_2, \ldots, h)
  \end{equation}
  The function $h$ express $q^{(n)}$ in terms of conjugate coordinates under
  the assumption of regular Lagrangian, i.e.
  \begin{equation*}
    q^{(n)} = h(Q_1, \ldots, Q_n, P_n)
  \end{equation*}
  Observe that $h$ does not depend on $P_1, \ldots, P_{n-1}$ so the only
  contribution of these momenta to the Hamiltonian is in the linear factor
  \begin{equation*}
    P_{n-1} Q_n + \cdots + P_1 Q_2
  \end{equation*}
  which is linear in momenta and hence can acquire any real value and so does
  the Hamiltonian~\eqref{eq:general_hamiltonian_1dim}.
\end{proof}

For many years, due to consequences of Theorem~\ref{th:ostrogradsky_classical},
HD theories are consider intrinsically sick and not worthy of further study.
However a deeper investigation reveals that Ostrogradsky instability can be cure
and HD theories should not be discard in the first instance. One way to
exorcising Ostrogradsky's ghost is through imposition of
constraints~\cite{Chen13}.

\subsection{Removing Ostrogradskian instability with constraints}

In Hamiltonian mechanics, \emph{constraints} are relation between coordinates
and conjugate momenta. Constraints are classify in two group: \emph{first class
constraints} and \emph{second class constraints}. Second class constraints can
be thought as ``physical'' (e.g.\ train on a railway, particle on a plane) so
that the solutions of equations of motion are different with or without them.
First-class are instead related to a gauge freedom: the solutions of equations
of motion differ by a function of time and hence represent the same system.
Moreover the Poisson Bracket of a first class constraint with all the other
constraints vanishes on the constraints surface in the phase space (i.e.
$\{\phi, \tilde{\phi}\} \approx 0$, see later for the notation used).

Another independent classification can be made for constraints (and not confused
with the previous one): \emph{primary constraints} and \emph{secondary
constrains}. Primary constraints are relation between $Q$s and $P$s which are
given at the beginning.

\begin{equation} \label{eq:constraint}
  \phi_1(Q, P) = 0
\end{equation}

As the name suggests, secondary constraints are derived from the primary ones
imposing the conservations of the constraints during the evolution of the
system. This generates a series of constraints where the constraints relations
are called \emph{consistency relations}.

\begin{equation} \label{eq:consistency_relations}
  \phi_1 \approx 0 \quad \Rightarrow \quad
  \left\{ \phi_1 , H \right\} =: \phi_2 \approx 0 \quad \Rightarrow \quad
  \left\{ \phi_2 , H \right\} =: \phi_3 \approx 0 \quad \Rightarrow \quad
  \ldots
\end{equation}

The weak equality symbol ``$\approx$'' has been used to highlight the fact that
these relations vanish only on the hypersurface where all constraints are
satisfied. When we will be said that a constrain $\phi_i$ \emph{vanishes} it is
to be intended in this weak sense.

\subsubsection{Lagrangian and Hamiltonian with $m$ auxiliary variables}
Constraints in Hamiltonian mechanics can be imposed with the help of auxiliary
variables $\lambda_i$ in the Lagrangian:

\begin{equation*}
  L = L(
  q, \dot{q}, \ddot{q}, \ldots, q^{(n)},
  \lambda_1, \lambda_2, \ldots, \lambda_m
  )
\end{equation*}

So the Lagrangian, besides the equations of motion~\eqref{eq:euler-lagrange},
need to satisfy the constraints equation
\begin{equation}
  \frac{\partial L}{\partial \lambda_i} = 0 \qquad i=1, 2, \ldots, m
\end{equation}

Canonical coordinates have to be given also for $\lambda_i$ and the following
choice can be done
\begin{equation} \label{eq:def_canonical_coordinates_lambda}
  \Lambda_{i}:= \lambda_{i}
  \quad \leftrightarrow \quad
  \Pi_{i} := \frac{\delta L}{\delta \dot{\lambda_{i}}} = 0
  \qquad i = 1, 2, \ldots, m
\end{equation}
so the primary constraints in~\eqref{eq:constraint} simply become
$\phi_{1, i}: \Pi_i = 0$. To the Hamiltonian~\eqref{eq:Ham_in_q} are now added
the auxiliary variables terms and it becomes
\begin{equation} \label{eq:Ham_constraints_in_q}
  \tilde{H} :=
  \sum_{j=1}^{n} \frac{\delta L}{\delta q^{(j)}} q^{(j)} +
  \sum_{i=1}^{m} \frac{\delta L}{\delta \dot{\lambda_i}} \dot{\lambda_i} -
  L(q, \ldots, q^{(n)}, \lambda_1, \ldots, \lambda_m)
\end{equation}

The $\dot{\lambda_i}$ can be obtain by the consistency
relations~\eqref{eq:consistency_relations} but there is no need to explicitly
calculate them. $\dot{\lambda_i}$ can be written as functions of canonical
coordinates: $\dot{\lambda_i} = u_i(Q_1, \ldots, Q_n,  P_1, \ldots, P_n)$.
Expressing~\eqref{eq:Ham_constraints_in_q} using canonical
coordinates we obtain the Hamiltonian
\begin{align} \label{eq:Ham_constraints}
  H =\ & P_n h + P_{n-1} Q_n + \cdots + P_1 Q_2 \nonumber \\
       & + \phi_m u_m + \phi_{m-1} u_{m-1} + \cdots + \phi_1 u_1 \nonumber \\
       & - L ( Q_1, Q_2, \ldots, h, \Lambda_1, \ldots, \Lambda_m)
\end{align}

After calculating the secondary constraints
with~\eqref{eq:consistency_relations}

\begin{equation*}
  \phi_{2, i} := \left\{ \phi_{1,i}, H \right\} =
  \frac{\partial \phi_{1,i}}{\partial \Pi_i}
  \frac{\partial H}{\partial \Lambda_i} =
  \frac{\partial L}{\partial \lambda_i} \Big|_{\lambda_i = \Lambda_i}
  \approx 0
\end{equation*}
two cases are possible:

\begin{enumerate}
  \item $\{\phi_{1,i}, \phi_{2,j}\} \not\approx 0$: $\phi_{1,i}$, $\phi_{2,j}$ are
    both second-class constraints thus there are no further constraints
    derivable from consistency relations.
  \item $\{\phi_{1,i}, \phi_{2,j}\} \approx 0$: $\phi_{1,i}$, $\phi_{2,j}$ are
    not second-class constraints then further constraints can be found employing
    the chain~\eqref{eq:consistency_relations} until either \emph{case 1} is
    reached or the new constraint can be expressed using those found previously.
\end{enumerate}

After all constraints have been found and the gauge symmetries are fixed, one
can use these equations to write the auxiliary canonical variables in terms of
the ``usual'' canonical variables
\begin{equation} \label{eq:constraints_in_terms_of_canonical_vars}
  \begin{cases}
    \Lambda_i =\ f_i(Q_1, \ldots, Q_n, P_n) \\
    \Pi_i =\ 0
  \end{cases}
  \qquad i = 1, 2, \ldots, m \\
\end{equation}
and then substitute in the Hamiltonian~\eqref{eq:Ham_constraints}
\begin{equation*}
  H =\ P_n h + P_{n-1} Q_n + \cdots + P_1 Q_2
     - L ( Q_1, Q_2, \ldots, h, f_1, \ldots, f_m)
\end{equation*}

To recap, we started form a phase space of dimension $2n$ (n$Q$ + n$P$), then we
enlarged it by adding $2m$ auxiliary variables (m$\Lambda$ + m$\Pi$) so the
total phase space dimension is $2(n+m)$. Then we associated with every pair of
canonical auxiliary variables two constraints (\emph{case 1}) or more
(\emph{case 2}) have been found. During the last substitution the dimension of
phase space is bring down to it's original dimensions (\emph{case 1}: $2(n+m) -
2m = 2n$) or further reduced (\emph{case 2}: e.g. $2(n+m) - 3m < 2n$).

\subsubsection{Example of non-stable system with constraints}
We introduce here a widely studied HD Lagrangian: the
\emph{Pais-Uhlenbeck oscillator} (PU).
\begin{equation} \label{eq:lagrangian_PU}
  L_{PU} = \frac{1}{2} \left[
    \ddot{q}^2 - (\omega_1^2 + \omega_2^2) \dot{q}^2 + \omega_1^2 \omega_2^2 q^2
  \right]
\end{equation}

According to Ostrogradsky Theorem~\ref{th:ostrogradsky_classical} this system
shows a problematic instability. Then we can try to cure that by introducing a
constraint such as $\ddot{q}^2 - \dot{q}^2 = 0$ and see if the Hamiltonian is
still two-side unbounded. Using the auxiliary variable $\lambda$ one can write
the constraint version of~\eqref{eq:lagrangian_PU}:
\begin{equation} \label{eq:lagrangian_PUC}
  L_{PUC} = L_{PU} + \frac{\lambda}{2} \left(\ddot{q}^2 - \dot{q}^2\right)
\end{equation}
and from it we derive the Hamiltonian following the given prescription.
\begin{align*}
  & \begin{cases}
    Q_1 := q \\
    P_1 := \frac{\delta L_{PUC}}{\delta \dot{q}} =
        - (\lambda + 1) q^{(3)} - \dot{\lambda}\ddot{q}
        - (\lambda + \omega_1^2 + \omega_2^2)\dot{q}
  \end{cases} \\
  & \begin{cases}
    Q_2 := \dot{q} \\
    P_2 := \frac{\delta L_{PUC}}{\delta \ddot{q}} =
        (\lambda + 1) \ddot{q}
        \quad  \Rightarrow \quad \ddot{q} = \frac{P_2}{(\Lambda + 1)}
  \end{cases} \\
  & \begin{cases}
    \Lambda := \lambda \\
    \Pi := \frac{\delta L_{PUC}}{\delta \lambda} = 0
        \quad  \Rightarrow \quad \phi_1: \Pi \approx 0
  \end{cases}
\end{align*}
\begin{equation} \label{eq:ham_PUC_with_aux_vars}
   H_{PUC} =\ P_1 Q_2 + \frac{P_2^2}{2(\Lambda + 1)} + \phi_1 u_1
  + \frac{Q_2^2}{2} \left(\Lambda + \omega_1^2 + \omega_2^2 \right)
  - \frac{Q_1^2}{2} \omega_1^2 \omega_2^2
\end{equation}

\begin{equation} \label{eq:secondary_constraint_PU}
  \phi_2 :=
  \left\{ \phi_1, H_{PUC} \right\} =
  \frac{1}{2} \left[ \frac{P_2}{\Lambda + 1} - Q_2 \right]
  \left[ \frac{P_2}{\Lambda + 1} + Q_2 \right]
  \approx 0
\end{equation}

The secondary constraint equation~\eqref{eq:secondary_constraint_PU}
vanish if one of the factors in square brackets vanish. The two square bracket
represent two hypersurfaces and one solution is chosen instead of the other
based on initial conditions. In the following, we choose the bracket with the
minus sign. Moreover note that $\phi_1$ and $\phi_2$ are already second-class
constraints, indeed
\begin{equation*}
  \begin{cases}
    \phi_1: \Pi = 0 \\
    \phi_2: \frac{P_2}{\Lambda + 1} - Q_2 \approx 0
  \end{cases}
  \quad \Rightarrow \quad
  \left\{ \phi_1, \phi_2 \right\} = -
  \frac{\partial\phi_1}{\partial\Pi} \frac{\partial\phi_2}{\partial\Lambda} =
  \frac{P_2}{{(\Lambda + 1)}^2} \not\approx 0
\end{equation*}
so no further constraints are needed (\emph{case 1}). The equation for
$\phi_1$ (trivial) and for $\phi_2$ can be locally
inverted~\eqref{eq:constraints_in_terms_of_canonical_vars} obtaining that
$\Pi = 0$ and $\Lambda = P_2/Q_2 - 1$. Substituting them
in~\eqref{eq:ham_PUC_with_aux_vars} we get the final Hamiltonian of the system
but unfortunately the Ostrogradsky instability persists.
\begin{equation} \label{eq:ham_PUC_instable}
   H_{PUC} =\ P_1 Q_2 + P_2 Q_2
  + \frac{Q_2^2}{2} \left(\omega_1^2 + \omega_2^2 - 1 \right)
  - \frac{Q_1^2}{2} \omega_1^2 \omega_2^2
\end{equation}

\subsubsection{Example of stable system with constraints}
Now consider the following constrained Pais-Uhlenbeck oscillator~\cite{Chen13}
\begin{equation}
  L_{PUC} = L_{PU} +
  4 \omega_1^2\omega_2^2 q^2 \lambda (1+ \lambda) +
  2 \sqrt{2} \omega_1\omega_2 \lambda q \ddot{q}
\end{equation}

Analogously to the previous example, canonical variable and hamiltonian
$H_{PUC}$ can be derived in similar fashion
\begin{align*}
  & \begin{cases}
    Q_1 := q \\
    P_1 := - q^{(3)} -
    \left( 2\sqrt{2} \omega_1\omega_2\lambda + \omega_1^2 + \omega_1^2 \right)
    \dot{q} - 2\sqrt{2}\omega_1\omega_2\dot{\lambda} q
  \end{cases} \\
  & \begin{cases}
    Q_2 := \dot{q} \\
    P_2 := \ddot{q} + 2\sqrt{2} \omega_1\omega_2\lambda q
    \quad \Rightarrow \quad
    \ddot{q} = P_2 - 2\sqrt{2} \omega_1\omega_2\Lambda Q_1
  \end{cases} \\
  & \begin{cases}
    \Lambda := \lambda \\
    \Pi := 0
    \quad  \Rightarrow \quad \phi_1: \Pi \approx 0
  \end{cases}
\end{align*}
\begin{align} \label{eq:ham_PUC_with_aux_vars_stable}
  H_{PUC} =\ & P_1 Q_2 + \frac{P_2^2}{2} -
  \omega_1^2\omega_2^2\frac{Q_1^2}{2} +
  \left( \omega_1^2 + \omega_2^2 \right) \frac{Q_2^2}{2} \\
  & - 4 \omega_1^2\omega_2^2\Lambda Q_1^2 -
  2\sqrt{2}\omega_1\omega_2\Lambda P_2 Q_1 + \phi_1 u_1
\end{align}

Secondary constraints relations carry useful information up to the forth order
\footnote{
  To obtain $\phi_4$, information from $\phi_1$ and $\phi_2$ have been used in
  order to write $\phi_4$ in terms of $\Lambda$.
}
\begin{align*}
  & \phi_1 : \Pi = 0 \\
  & \phi_2 := \{\phi_1, H_{PUC}\} =
    P_2 + \sqrt{2} \omega_1\omega_2 Q_1 \approx 0 \\
  & \phi_3 := \{\phi_2, H_{PUC}\} =
    P_1 + \left(\omega_1^2 + \omega_2^2 - \sqrt{2}\omega_1\omega_2 \right) Q_2
    \approx 0 \\
  & \phi_4 := \{\phi_3, H_{PUC}\} =
  \omega_1 \omega_2 (3+8\Lambda) - \sqrt{2}
  \left( \omega_1^2 + \omega_2^2 \right) \left(1 + 2\Lambda \right) \approx 0
\end{align*}

These consistency relations can be inverted so that $P_2$, $Q_2$ and $\Lambda$
are express in terms of $P_1$ and $Q_1$. Substituting them
into~\eqref{eq:ham_PUC_with_aux_vars_stable} obtaining

\begin{equation} \label{eq:ham_PUC_stable}
  H_{PUC} =\ \frac{\omega_1^2\omega_2^2}{2} Q_1^2 +
  \frac{\omega_1\omega_2}{\sqrt{2} {\left(\sqrt{2} \omega_1\omega_2 -
  \omega_1^2 - \omega_2^2 \right)}^2} P_1^2
\end{equation}

The Hamiltonian~\eqref{eq:ham_PUC_stable} is bounded from below (and not from
above, just like an harmonic oscillator) and hence this system does not suffer
of Ostrogradsky instability unlike the previous example. \\

The main difference between the two is the role played by constraints. In the
first example the dimensions of phase space of the unconstrained and constrained
Hamiltonian are both four (i.e.\ four variable appear in the reduced
Hamiltonian~\eqref{eq:ham_PUC_instable}: $Q_1$, $P_1$, $Q_2$ and $P_2$). In the
second example the constraints reduce the dimensionality of the phase space from
the original four to two (indeed just $Q_1$ and $P_1$ appear
in~\eqref{eq:ham_PUC_stable}). As shown in~\cite{Chen13} a reduction of the
phase space, due to imposed constraints, can cure the typical Ostrogradsky
instability of HD theories.


\gotosection{2}{3}
\subsection{Matrix inverses and elementary matrices}

\begin{exercise}{2}
\begin{enumerate}
\item $\xmat{1/6&5/54\\-1/6&1/54}$

\item Inverse doesn't exist, as $[A|I]$ is reduced to $\xmat{1&3&1&0\\0&0&-3&1}$.

\item $\xmat{2&-1&-3\\-2&2&3\\1&-1&-1}$

\item Inverse doesn't exist, as this is not a square matrix.

\item $\xmat{1/7 & -1/2 & 1/14 \\ 4/21 & 5/6 & -1/14 \\ -4/21 & 1/6 & 1/14}$

\item $\xmat{0&1&-1 \\ 1&-2& 1&-1&1}$

\item $\xmat{4 & -6 & 4 & 1 \\ -6 & 14 & -11 & 3 \\ 4 & -11 & 10 & -3 \\ -1 & 3 & -3 & 1}$
\end{enumerate}
\end{exercise}

\begin{exercise}{3}
\begin{enumerate}
\item Suppose an matrix $A \in M_{m,n}(\mathbb{R})$. If $A$ has an inverse matrix, then $T_A$ must be bijective, or for any $\Vect{b} \in \mathbb{R}^n$, $A\Vect{x}=\Vect{b}$ has a unique solution. If $m < n$, then $\widetilde{A}$ has at least one column without a pivotal 1, so there exist infinitely many solutions, namely $T_A$ is not injective. If $m > n$, there must exist a row-reduced vector $\widetilde{\mathbf{b}}$ that contains a pivotal 1---since row operations are invertible, we can always find a $\Vect{b}$ for any given $\widetilde{\mathbf{b}}$---there exists no solution, namely $T_A$ is not surjective. Only when $m = n$ could $A$ be reduced to an identity matrix and therefore has a unique solution for every possible $\widetilde{\mathbf{b}}$. Hence only squre matrices have inverses. \rQED

\item $A = \xmat{1&0&0\\0&1&0}, B = \xmat{1&0\\0&1\\0&0}$.
\end{enumerate}
\end{exercise}

\begin{exercise}{5}
Label them as $A, B, C$. $A+C$ and $B+2C$ gives $4x+4z=2, 4x+3y=3$, therefore, $x = (1-2z)/2$. Substituting $x$ with $z$ gives $y = (1+4z)/3$. Substituting $x$ and $y$ with $z$ into $A$ gives $z = 1/8$, then $x = 3/8$, $y = 1/2$.

\begin{enumerate}
\item Echelon form of this linear system is
$\xmat{1 & 0 & 0 & 3/8\\
       0 & 1 & 0 & 1/2\\
       0 & 0 & 1 & 1/8}$. Therefore, $x = 3/8$, $y = 1/2$, $z=1/8$.

\item $A^{-1} = \xmat{3/16 & 1/4 & -1/16\\
                      -1/4 & 0   & 3/4  \\
                      1/16 & -1/4 & 5/16}$. $A^{-1}\xmat{1\\1\\1} = \xmat{3/8\\1/2\\1/8}$.
\end{enumerate}
\end{exercise}

\begin{exercise}{8}
\begin{enumerate}
\item \begin{enumerate}
        \item Multiply the second row by 3.
        \item Exchange the second and the third row.
        \item Add a multiple 2 of the first row to the third row.
      \end{enumerate}

\item \begin{enumerate}
        \item $\xmat{1&0&-1\\6&3&3\\0&1&2}$.
        \item $\xmat{1&0&-1\\0&1&2\\2&1&1}$.
        \item $\xmat{1&0&-1\\2&1&1\\2&1&0}$.
      \end{enumerate}

\item \begin{enumerate}
        \item Multiply the second column by 3.
        \item Exchange the second and the third column.
        \item Add a multiple 2 of the first column to the third column.
      \end{enumerate}

\item \begin{enumerate}
        \item $\xmat{1&0&-1\\2&3&1\\0&3&2}$.
        \item $\xmat{1&-1&0\\2&1&1\\0&2&1}$.
        \item $\xmat{1&0&1\\2&1&5\\0&1&2}$.
      \end{enumerate}
\end{enumerate}
\end{exercise}

\begin{exercise}{11}
Column operations applied to a matrix $A$ can be seen as the transpose of the matrix generated by applying the row operations with the same parameters to $A^{\top}$. Here ``the same parameters" means the same index of row/column to multiply, the same multiplication constant, the same indexes of the two row/columns to exchange, or the same multiple of the same index of row/column to be added to another same index of row/column.

Given any elementary matrix $E$, $AE^{\top} = (EA^{\top})^{\top}$. Since $E^{\top}$ is also an elementary matrix, column operations can be achieved by multiplication on the right by elementary matrices. Note that if $E$ is of type 1 or 3, its transpose is the same as itself; but if $E = E_2(i,j,x)$ for some $i, j, x$, its transpose will be $E_2(j,i,x)$, which means that $AE_2(i,j,x)$ will add the multiple $x$ of the $i$th column of $A$ to its $j$th column, rather than adding that multiple of the $j$th column of $A$ to its $i$th column. \rQED
\end{exercise}

\begin{exercise}{12}
\begin{enumerate}
\item Show $E_1(i,x)E_1(i,1/x) = E_1(i,1/x)E_1(i,x) = I$.

\Proof\ For $E_1(i,x)E_1(i,1/x)$, the $i$th-row of $E_1(i,1/x)$ will be multiplied by $x$, then its $i,i$ entry will become $x \cdot (1/x) = 1$, therefore the result is an identity matrix. For $E_1(i,1/x)E_1(i,x)$, the $i$th-row of $E_1(i,x)$ will be multiplied by $1/x$, then its $i,i$ entry will become $(1/x) \cdot x = 1$, therefore the result is an identity matrix. This shows that $E_1(i,1/x)$ is the inverse matrix of $E_1(i,x)$, and that $E_1(i,x)$ is invertible. \rQED

\item Show $E_2(i,j,x)E_2(i,j,-x) = E_2(i,j,-x)E_2(i,j,x) = I$.

\Proof\ Consider $E_2(i,j,-x)E_2(i,j,x)I$, which can be seen as applying two row operations to the identity matrix. Let $r_k$ be the $k$-th row of $I$. After the first operation $E_2(i,j,x)$, the $i$th row becomes $xr_j + r_i$; after the second operation $E_2(i,j,-x)$, the $i$th row of the identity matrix becomes $xr_j + r_i - xr_j = r_i$. Since all other rows of the identity matrix keeps invariant during this two operations, after the two operations the identity itself keeps invariant, namely $E_2(i,j,-x)E_2(i,j,x)I = I$, or $E_2(i,j,-x)E_2(i,j,x)=I$. From what we just proved we can get $E_2(i,j,x)E_2(i,j,-x) = I$ by seeing $-x$ as $y$ and $x$ as $-y$. Then, $E_2(i,j,-x)$ is both the left and right inverse matrix of $E_2(i,j,x)$, namely $E_2(i,j,-x)$ is the inverse of $E_2(i,j,x)$ and the later is invertible. \rQED

\item Show $E3(i,j)E3(i,j) = I$.

\Proof\ Since $E3(i,j)I = E3(i,j)$, $E3(i,j)$ is just the identity matrix with its $i,j$ rows exchanged. $E3(i,j)E3(i,j)$ exchanges these two rows back, so the product matrix is just the identity matrix. This shows that $E3(i,j)E3(i,j) = I$ and that $E3(i,j)$ is the inverse of itself. \rQED
\end{enumerate}
\end{exercise}


\documentclass[../main]{subfiles}
\pagestyle{fancy}

\begin{document}

\chapter{Large Cardinals}
\label{apx.large-cardinals}
\thispagestyle{fancy}

Since large cardinals came into existence in the beginning of the 20th century, a vast zoo of different types of such have appeared. The aim of this appendix is to act as a reference for the definitions of these as well as the relations between them.

\section{Inaccessibles}

\defi{
  A cardinal $\kappa$ is \textbf{regular} if $\cof\kappa=\kappa$; i.e. that there are no $\gamma<\kappa$ with a cofinal function $f\colon\gamma\to\kappa$. $\kappa$ is a \textbf{strong limit} if $2^\lambda<\kappa$ for all cardinals $\lambda<\kappa$. If $\kappa$ is both regular and a strong limit then we say that it is (strongly) \textbf{inaccessible}.
}

\qprop[\cite{Kanamori} Proposition 1.2]{
  If $\kappa$ is inaccessible then $(V_\kappa, \in)\models\zfc$.
}

G\"odel's Second Incompleteness Theorem from \cite{godel-incompleteness} then immediately implies the following corollary.

\qcoro{
  \zfc\ can not prove the existence of any inaccessible cardinals. Indeed, not even the \textit{consistency} of the existence of any inaccessible cardinals can be proven in \zfc.
}

\section{Weakly compacts}

\defi{
  For any function $f\colon A\to B$, a subset $H\subset A$ is \textbf{homogeneous for $f$} if $f\restr H$ is a constant function.
}

\defi{
  Let $\kappa$ and $\lambda$ be infinite cardinals, $\gamma$ an ordinal and $n<\omega$. Then the partition relation $\kappa\to(\lambda)^n_\gamma$ holds if to every function $f\colon[\kappa]^n\to\gamma$ there exists a subset $H\subset[\kappa]^n$ of size $\lambda$ which is homogeneous for $f$. If $\gamma=2$ then we usually leave it out and simply write $\kappa\to(\lambda)^n$.
}

\defi{
  An uncountable cardinal $\kappa$ is \textbf{weakly compact} if $\kappa\to(\kappa)^2$.
}

\qtheo[\cite{Jech} Lemma 9.9]{
  Every weakly compact cardinal is a limit of inaccessible cardinals.
}

\section{Ineffables and completely ineffables}

\defi{
  An uncountable cardinal $\kappa$ is \textbf{ineffable} if to any function $f\colon[\kappa]^2\to 2$ there exists a \textit{stationary} $H\subset[\kappa]^2$ which is homogeneous for $f$.
}

Ineffable cardinals are weakly compact by definition, and the following theorem from \cite{Friedman} shows that they are strictly stronger.

\qtheo[Friedman]{
  Ineffable cardinals are weakly compact limits of weakly compacts.
}

A way of improving ineffability is to ``close under homogeneity'', in the sense that if $H$ is homogeneous for $f\colon[\kappa]^2\to 2$ and $g\colon[H]^2\to 2$ is any function, then there is a subset of $H$ which is homogeneous for $g$. To formalise this notion we use the concept of a \textit{stationary class}.

\xdefi{
  For $X$ any set, a collection $\R\subset\p(X)$ is a \textbf{stationary class} if
  \begin{itemize}
    \item $\R\neq\emptyset$;
    \item Every $A\in\R$ is a stationary subset of $X$;
    \item If $A\in\R$ and $B\supset A$ then $B\in\R$.$\hfill\circ$
  \end{itemize}
}

\defi{
  An uncountable cardinal $\kappa$ is \textbf{completely ineffable} if there is a stationary class $\R\subset\p(\kappa)$ such that for every $A\in\R$ and $f\colon[A]^2\to 2$ there exists a $H\in\R$ which is homogeneous for $f$.
}

As suspected, these completely ineffable cardinals are indeed strictly stronger than the ineffables, as the following theorem from \cite{Abramson} shows.

\qtheo[Abramson et al]{
  Completely ineffable cardinals are ineffable limits of ineffable cardinals.
}

\section{Measurables, strongs and supercompacts}

\defi{
  For two first-order structures $\M$ and $\N$ with underlying sets $M$ and $N$, an \textbf{elementary embedding} $j\colon\M\to\N$ between them is a function $j\colon M\to N$ such that, for any first-order formula $\varphi(v_1,\dots,v_n)$ and sets $x_1,\dots,x_n\in\M$ it holds that $\M\models\varphi[x_1,\dots,x_n]$ iff $\N\models\varphi[j(x_1),\dots,j(x_n)]$.
}

As elementary embeddings in particular preserve equality, they are always injective. Identity embeddings are of course always elementary, so we say that an elementary embedding is \textbf{non-trivial} if it is not the identity. The following then shows that in most situations these non-trivial embeddings can be associated to a unique ordinal.

\qprop[\cite{Kanamori} Propostion 5.1]{
  If $j\colon(\M,\in)\to(\N,\in)$ is an elementary embedding such that $\M$ is transitive and \textit{either} $\N\subset\M$ or $\M\models\zfc$, then there exists an ordinal $\alpha<o(\M)$ moved by $j$, i.e. that $j(\alpha)\neq\alpha$. We call the least such ordinal the \textbf{critical point} of $j$, and denote it by $\crit j$.
}

\defi[\gbc]{
  An uncountable cardinal $\kappa$ is \textbf{measurable} if there exists a transitive class $\M$ and an elementary embedding $j\colon (V,\in)\to(\M,\in)$ with critical point $\kappa$.
}

The measurable cardinals were the first large cardinals shown to ``transcend $L$''.

\qtheo[Scott's Theorem, \cite{Kanamori} Corollary 5.5][theo.scott]{
  $L$, G\"odel's constructible universe, has no measurable cardinals.
}

Given this result, it's not surprising that the measurables then exceed the strength of the previous large cardinals.

\prop{
  Measurable cardinals are completely ineffable limits of completely ineffable cardinals.
}
\proof{
  (Sketch) If $j\colon V\to\M$ is a non-trivial elementary embedding then the \textbf{derived ultrafilter} $\mu\subset\p(\kappa)$ on $\kappa:=\crit j$ is defined as $X\in\mu$ iff $\kappa\in j(X)$. Section 5 in \cite{Kanamori} shows that it is indeed an ultrafilter and that its ultrapower $\ult(V, \mu)$ is wellfounded. A reflection argument then shows that we can simply take $\R:=\mu$.
}

\defi[\gbc]{
  An uncountable cardinal $\kappa$ is \textbf{strong} if there to every cardinal $\theta>\kappa$ exists a transitive class $\M_\theta$ satisfying that $H_\theta\subset\M_\theta$, and an elementary $j_\theta\colon(V,\in)\to(\M_\theta,\in)$ with critical point $\kappa$. We say that $\kappa$ is \textbf{$\theta$-strong} if the property holds for a specific $\theta$.
}

\qprop[\cite{Kanamori} 26.6]{
  Strong cardinals are measurable limits of measurable cardinals.
}

\defi[\gbc]{
  An uncountable cardinal $\kappa$ is \textbf{supercompact} if there to every cardinal $\theta>\kappa$ exists a transitive class $\M_\theta$ satisfying that $^{<\theta}\M_\theta\subset\M_\theta$, and an elementary $j_\theta\colon(V,\in)\to(\M_\theta,\in)$ with critical point $\kappa$.
}

\prop{
  If $\kappa$ is supercompact then
  \eq{
    V_\kappa\models\godel{\text{There exists a proper class of strong cardinals}}.\tag*{$(1)$}
  }
}
\proof{
  (Sketch) By noting that the restrictions of the supercompact embedding is an element of the target model by supercompactness, $\kappa$ is strong in the target model, so that a reflection argument shows $(1)$.
}

\section{Woodins and Vop\v enkas}

\defi[][defi.A-strong]{
  Let $A$ be any set. An uncountable cardinal $\kappa$ is \textbf{$A$-strong} if there to every cardinal $\theta>\kappa$ exists a transitive class $\M_\theta$ satisfying that $H_\theta\subset\M_\theta$, and an elementary $j_\theta\colon(V,\in)\to(\M_\theta,\in)$ with critical point $\kappa$, such that $A\cap H_\theta = j(A)\cap H_\theta$.
}

\defi{
  An uncountable cardinal $\delta$ is a \textbf{Woodin cardinal} if there to every subset $A\subset H_\delta$ exists $\kappa<\delta$ such that $(H_\delta, \in, A)\models\godel{\text{$\kappa$ is $A$-strong}}$.
}

\theo[\cite{Kanamori} Theorem 26.14]{
  The following are equivalent for an uncountable cardinal $\kappa$.
  \begin{enumerate}
    \item $\kappa$ is a Woodin cardinal;
    \item For any $f\colon\kappa\to\kappa$ there exists $\alpha<\kappa$ such that $f[\alpha]\subset\alpha$, a transitive $\M$ with $V_{j(f)(\alpha)}\subset\M$ and an elementary embedding $j\colon(V,\in)\to(\M,\in)$ with $\crit j=\kappa$.
  \end{enumerate}
}

\defi[\gbc]{
  \textbf{Vop\v enka's Principle (\vp)} postulates that to any first-order language $\mathcal L$ and proper class $\C$ of $\mathcal L$-structures, there exist distinct $\M,\N\in\C$ and an elementary embedding $j\colon\M\to\N$.
}

\defi{
  An uncountable cardinal $\delta$ is \textbf{Vop\v enka} if $(V_\delta,\in;V_{\delta+1})\models\vp$.
}

The following theorem is from \cite{Perlmutter}.

\qtheo[Perlmutter]{
  Vop\v enka cardinals are equivalent to cardinals that are ``Woodin for supercompactness'', meaning a cardinal $\delta$ such that to any subset $A\subset H_\delta$ there is a cardinal $\kappa<\delta$ such that $(H_\delta,\in,A)\models\godel{\text{$\kappa$ is $A$-supercompact}}$.\footnote{Here $\kappa$ is, in analogy with Definition \ref{defi.A-strong}, \textbf{$A$-supercompact} if there to every cardinal $\theta>\kappa$ exists a transitive class $\M_\theta$, closed under ${<}\theta$-sequences, and an elementary $j_\theta\colon(V,\in)\to(\M_\theta,\in)$ with critical point $\kappa$, such that $A\cap H_\theta=j(A)\cap H_\theta$.}
}

\section{Reinhardts and Kunen inconsistency}

\defi[\gbc]{
  An uncountable cardinal $\kappa$ is a \textbf{Reinhardt cardinal} if there exists an elementary embedding $j\colon(V,\in)\to (V,\in)$ with $\crit j=\kappa$.
}

\theo[Kunen inconsistency, \gbc, \cite{Kanamori} Theorem 23.12]{
  There are no Reinhardt cardinals. Even more, there is no non-trivial elementary $j\colon(V_{\lambda+2},\in)\to(V_{\lambda+2}, \in)$ for any uncountable cardinal $\lambda$.
}

When we're dealing with the \textit{virtual} large cardinals in Chapter \ref{chapter.virtual-large-cardinals} we show that the property $j(\kappa)>\theta$ is a highly non-trivial assumption. However, when we're not in the virtual world then this is simply automatic.

\qprop[\cite{Kanamori} 26.7]{
  If $j\colon V\to\M_\theta$ witnesses that $\kappa:=\crit j$ is a $\theta$-strong cardinal then $j(\kappa)>\theta$.
}

Note that a crucial part of the proof of the above is Corollary 23.14 in \cite{Kanamori}, which relies \textit{heavily} on the Kunen inconsistency. There is also the following even stronger version of the Reinhardts.

\defi{
  An uncountable cardinal $\kappa$ is \textbf{super Reinhardt} if for all ordinals $\lambda$ there exists an elementary embedding $j\colon (V,\in)\to(V,\in)$ with $\crit j=\kappa$ and $j(\kappa)>\lambda$.
}

\section{Berkeleys}

\defi[\gb]{
  An uncountable cardinal $\delta$ is a \textbf{proto-Berkeley cardinal} if to every transitive \textit{set} $\M$ such that $\delta\subset\M$ there exists an elementary embedding $j\colon(\M,\in)\to(\M,\in)$ with $\crit j<\delta$.
}

Note that if $\kappa$ is a proto-Berkeley cardinal then every $\lambda>\kappa$ is also proto-Berkeley, which makes it quite an uninteresting notion. But we can isolate the interesting cases, leading to the definition of a Berkeley cardinal. The following is Theorem 2.1.14 in \cite{Cutolo}.

\qtheo[Cutolo]{
  If $\delta_0$ is the least proto-Berkeley cardinal then we can choose the critical point of the embedding to be arbitrarily large below $\delta_0$.
}

As this property is clearly not preserved upwards, this makes for a good candidate for the large cardinal notion.

\defi[\gb]{
  A proto-Berkeley cardinal $\delta$ is \textbf{Berkeley} if we can choose the critical point of the embedding to be arbitrarily large below $\delta$. If we furthermore can choose the critical point as an element of any club $C\subset\delta$ then we say that $\delta$ is \textbf{club Berkeley}.
}

In \cite{Cutolo}, they furthermore mention that, among the above-mentioned cardinals, the non-trivial relative consistency implications currently known are the following, being Theorem 2.2.1 and 2.2.2 in \cite{Cutolo}, respectively.

\qtheo[Cutolo]{
  Berkeley cardinals are consistency-wise strictly stronger than Reinhardt cardinals.
}

\qtheo[Cutolo]{
  Club Berkeley cardinals are consistency-wise strictly stronger than super Reinhardt cardinals.
}

\end{document}


\section{Bimatrix Games and Labels}
\label{labels-sect}

Let $(A,B)$ be bimatrix game, and let $X=\Delta^1$ and
$Y=\Delta^2$ be the mixed strategy
simplices of player 1 and 2, respectively, that is,
\begin{equation}
\arraycolsep.2em
\label{mixed-strat-simplices}
\begin{array}{rcll}
X&=&\{\, x\in\reals^m&\mid x\geq\0,\ \1\T x = 1 \},\\
Y&=&\{\, y\in\reals^n&\mid y\geq\0,\ \1\T y = 1 \}.
\end{array}
\end{equation}
A {\em labeling} of the sets $X$ and $Y$ assigns to each
$x\in X$ and $y\in Y$ a set of {\em labels}, which is a
subset of $[m+n]$, as follows:
\begin{enumerate}
\item the $m$ pure strategies of player 1 are denoted as $i\in [m]$;
\item the $n$ pure strategies of player 2 are denoted as $m+j$ with $j\in [n]$;
\item each mixed strategy $x\in X$ of player 1 has
    \begin{itemize}
    \item label $i$ for each $i\in [m]$ such that $x_i = 0$,
    \item label $m + j$ for each $j\in [n]$ such that the $j$-th pure strategy
    of player 2 is a best response to $x$;
    \end{itemize}
\pagebreak[2]
\item each mixed strategy $y\in Y$ of player 2 has
    \begin{itemize}
    \item label $m + j$ for each $j\in [n]$ such that $y_j = 0$,
    \item label $i$ for each $i\in [m]$ such that the $i$-th pure strategy
    of player 1 is a best response to $y$.
    \end{itemize}
\end{enumerate}
This labeling can be used to characterize the Nash equilibria of the game:
A pair $(x,y)$ is called {\em completely labeled} if each
possible label in $[m+n]$ is a label of $x$ or of~$y$.

\begin{theorem}{\rm (Shapley \cite{shapley})}\label{comp-label-thm}
Let $(x,y)\in X\times Y$. Then $(x,y)$ is a Nash equilibrium of the bimatrix
game $(A,B)$ if and only if $(x,y)$ is completely labeled.
\end{theorem}

\begin{proof}
The mixed strategy $x\in X$ has label $m + j$ for some $j\in [n]$ if and
only if the $j$-th pure strategy of player 2 is a best response to $x$. By
Proposition \ref{br-played-thm}, this is a necessary and sufficient condition
for
player 2 to play his $j$-th strategy at every equilibrium where player 1
chooses $x$. The analogous holds for the strategies $y\in Y$ and player 1
playing her $i$-th strategy in response to player 2 choosing $y$.
Therefore, at an equilibrium $(x,y)$ all labels $m + j$, with $j\in [n]$,
appear either as labels of $x$ or of $y$.
Conversely, if $(x,y)$ is not completely labeled, then some
label does not appear as a label of $x$ or $y$. This label
represents a pure strategy that is played with positive
probability but is not a best response, which contradicts
the equilibrium property.
\end{proof}

A useful graphical representation of labels on the simplices $X$ and $Y$
is done by labeling the outside of each simplex according to the player's own
pure strategies that are not played, and by subdividing its interior
into closed polyhedral sets, called {\em best response
regions}, that correspond to the other player's pure
best responses.
We give an example of this construction.

\begin{example}({\rm Savani and von Stengel \cite{uvg}})
\label{br-game-ex}
Consider the $3\times 3$ game $(A,B)$ with
\begin{equation}
\label{AB}
A = \left(\begin{matrix}1&0&0\\ 0&1&0\\
0&0&1\end{matrix}\right),
\qquad
B = \left(\begin{matrix}0&2&4\\ 3&2&0\\
0&2&0\end{matrix}\right).
\end{equation}
Figure~\ref{br-regions-fig} shows the mixed strategy simplices $X$ and $Y$:
the exterior facets are labeled with the pure strategy that is played
% In general, do not use a semicolon where period works.
at the opposite vertex of the simplex. The interior is covered
by the best response regions, labeled by the other player's pure best
response strategies.
For example, the best-response region in $Y$ with label~$1$
is the set of all the $(y_1,y_2,y_3)\in\reals^3$ such that $y_1\ge y_2$ and
$y_1\ge y_3$.
There is only one pair $(x,y)$
that is completely labeled, namely $x=(\frac13,\frac23,0)$
with labels $3,4,5$, and $y=(\frac12,\frac12,0)$ with labels
$1,2,6$. This is the only Nash equilibrium of the game.
\begin{figure}[hbt]
\strut\hfill
\includegraphics[width=30ex]{chapter-2/fig-labels/Xnondeg.pdf}%
\hfill
%\vrule height 52mm ~ \vrule height 27ex
\hfill
\includegraphics[width=30ex]{chapter-2/fig-labels/Yimit.pdf}%
\hfill\strut
\caption[The labeled mixed strategy simplices of a game]{%
The labeled best response regions of the mixed strategy simplices
of player 1 (left) and player 2 (right) in game (\ref{AB}).
}
\label{br-regions-fig}
\end{figure}
\end{example}

The representation of a game and its Nash equilibria in terms of best
response regions can be translated to an equivalent construction on
polytopes.
The first step is to notice that the best-response regions can be obtained as
projections on $X$ and $Y$ of the {\em best-response facets} of
the polyhedra
\begin{equation}
\label{br-polyhedron}
\arraycolsep.2em
\begin{array}{rcll}
\overline{P}&=&\{ (x,v)\in X\times\reals& \mid B\T x\leq\1 v \},\\
\overline{Q}&=&\{ (y,u)\in Y\times\reals& \mid A y\leq\1 u \}. %
\end{array}
\end{equation}
In $\overline{P}$, these facets are the points $(x,v)\in X\times\reals$
such that $(B\T x)_j = v$, which in turn correspond to the strategies
$x\in X$ of player 1 that give exactly payoff $v$ to player~2
when he plays strategy $j$; this payoff is the
best-response payoff by the definition of $P$.
The projection of the facet defined by $(B\T x)_j = v$ to
$X$ then has label~$j$.
Analogously, the facet of $\overline{Q}$ given by the points
$(y,u)\in Y\times\reals$ such that $(A y)_i = u$ projects to the
best-response region of $Y$ with label $i$.

\begin{example}({\rm Savani and von Stengel \cite{uvg}})
\label{br-polyhedra-ex}
In Example \ref{br-game-ex}, the inequalities $B\T x\le\1v$ are
\[
\arraycolsep.2em
\begin{array}{rcl}
3x_2 & \leq&v \\
2x_1+ 2x_2+ 2x_3 & \leq&v \\
4x_1 & \leq&v.
\end{array}
\]
Figure~\ref{br-polyhedron-fig} shows the best-response facets of
$\overline{P}$ and their projection to $X$ by ignoring the payoff
variable~$v$, which gives the subdivision of $X$ into best-response
regions of Figure \ref{br-regions-fig}.
\begin{figure}[hbt]
\strut\hfill
\includegraphics[width=40ex]{chapter-2/fig-labels/upenv.pdf}%
\hfill\strut
\caption[The best response polyhedron of a game]{%
The best response polyhedron of player 1 in game (\ref{AB}).}
\label{br-polyhedron-fig}
\end{figure}
\end{example}

Given the assumptions on non-negativity of $A$ and $B\T$, we can
change coordinates to $x_i / v$ and $y_j / u$ and replace $\overline{P}$ and
$\overline{Q}$ with the {\em best-response polytopes}
\begin{equation}
\label{br-polytopes}
\arraycolsep.2em
\begin{array}{rcll}
P&=&\{\, x\in\reals^m& \mid\,x\geq\0,\ B\T x\leq\1 \},\\
Q&=&\{\, y\in\reals^n& \mid\,A y\leq\1,~ y\geq\0  \}.
\end{array}
\end{equation}
In (\ref{br-polytopes}), both $P$ and $Q$ are defined by
$m+n$ inequalities that correspond, in the listed order, to the
pure strategies of the two players with labels in $[m+n]$.
The polytope $P$ is the intersection of the $m+n$ half-spaces
that correspond to either player 1 not playing her $i$-th pure
strategy or to a best response $j$ of player~2,
where $i\in [m]$ and $j\in [n]$.
The analogous statement holds for $Q$.
Formally, a point $x\in P$ has label $k$ if and only if either
$x_k = 0$ for $k\in [m]$ or $(B\T x)_j = 1$ for $k=m+j$ with $j\in [n]$, and
a point in $Q$ has label $k$ if and only if either
$(A y)_k=1$ for $k\in [m]$ or $y_j = 0$ for $k=m+j$ with $j\in [n]$.

Hence, a point $(x,y)\in P\times Q$ is completely labeled if and only if
it satisfies the {\em complementarity condition} that states
% the quantifiers should be outside to avoid confusion
that for all $i\in[m]$ and all $j\in[n]$,
\begin{equation}
%\label{}
\begin{array}{ccrll}
x_i = 0 & \text{ or }&(Ay)_i= 1~,\\
y_j = 0 & \text{ or }&(B\T x)_j=1~.
\end{array}
\end{equation}
Therefore, if $(x,y)\in P\times Q$ is completely labeled either the
corresponding point in $\overline{P}\times\overline{Q}$
is a Nash equilibrium or $(x,y)=(\0,\0)$; we refer to the latter case
as {\em artificial equilibrium}.

\begin{example}({\rm Savani and von Stengel \cite{uvg}})
Keeping on with Example \ref{br-game-ex} and \ref{br-polyhedra-ex},
the best response
polyhedron $\overline{P}$ of Figure \ref{br-polyhedron-fig} becomes
the best response polytope of Figure \ref{br-polytopes-fig}.
Notice that the vertex $(x,y)=(\0,\0)$ is completely labeled, since it is labeled
by the labels $1,2,3$ in $P$ and $4,5,6$ in $Q$.
\begin{figure}[htb]
\strut\hfill
\includegraphics[width=40ex]{chapter-2/fig-labels/polytope.pdf}%
\hfill\strut
\caption[The best response polytope of a game]{%
The best response polytope of player 1 in game (\ref{AB}).
}
\label{br-polytopes-fig}
\end{figure}
\end{example}

We now consider some special cases of games and how they are related to each
other in terms of computational complexity.
First of all, we note that any bimatrix game can be ``symmetrized''.
This result is due to Gale, Kuhn and Tucker \cite{gale-kuhn-tucker}
for zero-sum games, while its extension to non-zero-sum games is a folklore
result.

\begin{proposition}\label{symmetrize-c}
Let $(A,B)$ be a bimatrix game and let $(x,y)$ be one of its Nash equilibria.
Then $(z,z)$, where $z=(x\alpha,y\beta)$ for suitable
positive scalars $\alpha$ and $\beta$, is a Nash equilibrium of the symmetric game
$(C,C\T)$, where
\begin{equation}
C = \left(
    \begin{array}{cc}
    0 & A \\
    B\T & 0
    \end{array}
    \right).
\end{equation}
\end{proposition}

McLennan and Tourky \cite{mclennan-tourky} have proven a result in the
opposite direction of Proposition \ref{symmetrize-c}: any symmetric game
can be translated into a {\em imitiation game}, where the payoff matrix of
player 1 is the identity matrix $I$. In any Nash equilibrium of $(I,B)$,
the mixed strategy $x$
of player 1 corresponds exactly to the symmetric equilibrium $(x,x)$ in
the symmetric game defined by the payoff matrix of player 2.
Since it takes polynomial time in the size of a matrix to calculate its
transpose, an algorithm that finds a Nash equilibrium of a bimatrix game
can be used to find a symmetric Nash equilibrium of a symmetric game.

\begin{theorem}{\rm (McLennan and Tourky \cite{mclennan-tourky})}
\label{imitation-thm}
The pair $(x,x)$ is a symmetric Nash equilibrium of the symmetric bimatrix
game $(C,C\T)$ if and only if there is some $y$ such that $(x,y)$ is a
Nash equilibrium of the imitation game $(I,B)$ with $B=C\T$.
\end{theorem}

Notice that Theorem \ref{imitation-thm} applies to the symmetric
equilibria of the symmetric game but not to all its Nash equilibria; there
could be non-symmetric equilibria of $(C,C\T)$ that are not found
through the imitation game, as shown in the following example.

\begin{example}({\rm Savani and von Stengel \cite{uvg}})
As an example, consider the symmetric game $(C,C\T)$ with
\begin{equation}
\label{C}
C = \left(\begin{matrix}0&3&0\\ 2&2&2\\
4&0&0\end{matrix}\right),
\qquad
C\T = \left(\begin{matrix}0&2&4\\ 3&2&0\\
0&2&0\end{matrix}\right).
\end{equation}
The corresponding imitation game is $(I,C\T)=(A,B)$, seen in
Example \ref{br-game-ex}.
Figure~\ref{imit-fig} shows the labeled mixed-strategy simplices $X$ and $Y$
for the game~(\ref{C}); since the game is symmetric, only the labels
are different.
In addition to the symmetric equilibrium $(x,x)$ where
$x=(\frac13,\frac23,0)$, the game has two non-symmetric equilibria in $(a,b)$
and $(b,a)$ with $a=(\frac12,\frac12,0)$ and $b=(0,\frac23,\frac13)$.
The imitation game $(A,B)$, on the other hand, has only one equilibrium
$(x,y)$, corresponding to $(x,x)$, with $y=(\frac{1}{2},\frac{1}{2},0)$.
\begin{figure}[hbt]
\strut\hfill
\includegraphics[width=30ex]{chapter-2/fig-labels/Xnondeg.pdf}%
\hfill
\includegraphics[width=30ex]{chapter-2/fig-labels/Ynondeg.pdf}%
\hfill\strut
\caption[The best response regions of a symmetric game]{%
The best response regions of the symmetric game (\ref{C}).
}
\label{imit-fig}
\end{figure}
\end{example}

The characterization of Nash equilibria as completely labeled pairs $(x,y)$
holds for arbitrary bimatrix games. From now on, we impose a
further condition: all points in $P$ have at most $m$ labels,
and all points in $Q$ have at most $n$ labels. These games are called
{\em nondegenerate}.
This condition is required for the Lemke-Howson algorithm (see
section \ref{lh-sect}).
% This is a loss of generality
The algorithm can also be applied to degenerate games by
lexicographic perturbation, as shown in von Stengel \cite{vs-agt}.
In an equilibrium $(x,y)$ of a nondegenerate game each label appears
exactly once. This also means that the number of pure best response
strategies against a mixed strategy is never larger than the size of
the support of that mixed strategy.
Geometrically, this means that no point of the best response polytope $P$
lies on more than $m$ facets and no point of the best response polytope $Q$
lies on more than $n$ facets, so both $P$ and $Q$ are simple.
Furthermore, a point of
$P$ has exactly $m$ labels if and only if it is a vertex, and a point
of $Q$ has exactly $n$ labels if and only if it is a vertex. Therefore,
all completely labeled points $(x,y)$ are vertices of the best response
polytopes, and Nash equilibria are isolated points.

\begin{example}({\rm Savani and von Stengel \cite{uvg}})
An example of degenerate game is given by $(C,C\T)$ with
\begin{equation}
\label{Cdeg}
C = \left(\begin{matrix}0&4&0\\ 2&2&2\\
4&0&0\end{matrix}\right),
\qquad
C\T = \left(\begin{matrix}0&2&4\\ 4&2&0\\
0&2&0\end{matrix}\right).
\end{equation}
As shown in Figure \ref{deg-fig}, the mixed strategy
$x=(\frac12,\frac12,0)$, that also defines the
unique symmetric equilibrium $(x,x)$ of the game, has three pure best
responses. The Nash equilibria $(x,y)$ of the imitation game
$(I,C\T)$ are not unique, since any convex
combination of $(\frac12,\frac12,0)$ and $(\frac13,\frac13,\frac13)$
can be chosen for~$y$.
\begin{figure}[hbt]
\strut\hfill
\includegraphics[width=30ex]{chapter-2/fig-labels/Xsymmdeg.pdf}%
\hfill
\includegraphics[width=30ex]{chapter-2/fig-labels/Yimitdeg.pdf}%
\hfill\strut
\caption[A degenerate symmetric game]{%
Left: The best-response regions of the degenerate symmetric
game (\ref{Cdeg}). \\
Right: The best-response regions for player 2 in the corresponding
imitation game $(I,C\T)$.
}
\label{deg-fig}
\end{figure}
\end{example}

A generalization of imitation games is the class of
{\em unit vector games}, introduced by Balthasar \cite{balthasar}.
These are defined as bimatrix games of the form $(U,B)$ where the
columns of the matrix $U$ are unit vectors.
By the results above, finding a Nash equilibrium of a bimatrix game is
at least as hard as finding a Nash equilibrium of a unit vector game.
Savani and von Stengel \cite{uvg} have shown that
the problem of finding a completely labeled vertex of the product of the
best response polytopes $P\times Q$ can be simplified for unit vector games:
it is enough to find a completely labeled vertex of a single
polytope $P^l$, for which the last $n$ facets are labeled following a
the labeling in $[m]$ that also encodes the matrix $U$.

\begin{theorem}{\rm (Savani and von Stengel \cite{uvg})}
\label{unit-vector-thm}
Let $l:[n]\to [m]$ be a function, and let $(U,B)$ be the unit vector game with
$U=(e_{l(1)}\ \cdots\ e_{l(n)})$.

Let $N_i = \{ j\in [n]\ \mid\ l(j)=i \}$ for
$i\in [m]$, and define $P^l$ and $Q^l$ as
\begin{equation}\label{p-l}
\arraycolsep.2em
\begin{array}{rcll}
P^l&=&\{ x\in\reals^m& \mid\,x\geq\0,\ B\T x\leq\1 \}, \\
Q^l&=&\{ y\in\reals^n& \mid\,y\geq\0,\
\sum_{j\in N_i } y_j\leq 1 \hbox{ for } i\in [m] \}.
\end{array}
\end{equation}
Let $l_f$ be the labeling of the facets of $P^l$ with labels
in $[m]$ defined as follows:
\begin{equation}
\label{facet-labeling-unitv}
\begin{array}{rll}
x_i\geq 0 & \text{ has label }i & \text{ for }i\in [m], \\
(B\T x)_j \leq 1 & \text{ has label }l(j) & \text{ for }j\in [n].
\end{array}
\end{equation}
Then $x\in P^l$ is a completely labeled vertex of
$P^l\setminus\{\0\}$ (that is, has all labels in $[m]$)
if and only if there is some $y\in Q^l$ such that, after scaling,
the pair $(x,y)$ is a Nash equilibrium
of $(U,B)$.
\end{theorem}

\begin{proof}
Let $P$ and $Q$ be the best response polytopes of $(U,B)$ as
in (\ref{br-polytopes}),
and let $(x,y)\in P\times Q\setminus\{ (\0,\0) \}$ be a Nash equilibrium of
$(U,B)$. Then $(x,y)$ is completely labeled with labels in $[m + n]$. If
$x_i=0$, then $x$ has label $i\in m$.
If $x_i > 0$, then $y$ has label $i$, so $(Uy)_i = 1$.
Therefore for some $j\in [n]$ we have $y_j > 0$ and $U_j = e_i$; that is,
we have $y_j > 0$ and $l(j)=i$ for some $j\in [n]$.
Since $y_j > 0$, $x\in P$ has label $m+j$; then, $(B\T x)_j = 1$;
therefore $x\in P^l$ has label $l(j) = i$.
Hence, $x$ is a completely labeled vertex of $P^l$.

Conversely, let $x\in P^l\setminus \{ \0 \}$ be completely labeled.
If $x_i > 0$, then there is $j\in [n]$ such that $(B\T x) = j$ and
$l(j) = i$; that is, $j\in N_i$. For all $i\in [m]$ such that $x_i > 0$,
define $y$ as follows: $y_j = 1$, and
$y_h = 0$ for all $h\in N_i\setminus \{ j \}$.
Then $(x,y)\in P\times Q$ is completely labeled.
\end{proof}

\begin{example}({\rm Savani and von Stengel \cite{uvg}})
The game in Example \ref{br-game-ex} is a unit vector game with $l(i)=i$.
In the polytope $P^l$ of Figure~\ref{p-l-fig} the labels 4, 5 and 6 of the
best response polytope $P$ of Figure \ref{br-polytopes-fig}
are replaced by 1, 2 and 3, since the corresponding columns of $A$
are the unit vectors $e_1,e_2,e_3$.
The only completely labeled point of $P^l$ are the origin $\0$, corresponding
to the ``artificial'' equilibrium, and $x$, corresponding to the unique
Nash equilibrium of the unit vector game (\ref{AB}).
\begin{figure}[hbt]
\strut\hfill
\includegraphics[width=30ex]{chapter-2/fig-labels/p-l-color.pdf}%
\hfill\strut
\caption[The polytope $P^l$ of a unit vector game]{%
The polytope $P^l$ of the unit vector game (\ref{AB}).
}
\label{p-l-fig}
\end{figure}
\end{example}

We now move on the dual version of Theorem \ref{unit-vector-thm}
given by Balthasar \cite{balthasar}.
We can translate the polytope $P^l$ of (\ref{p-l}) to
$P = \{ x - \1\ |\ x\in P^l \}$,
possibly multiplying all payoffs in $B$ by a
constant so that $\0$ is in the interior of $P$, which holds
if $\1$ in the interior of $P^l$, that is, all columns $b_j$
of the matrix $B$ fulfill $\1\T b_j<1$, for $j\in[n]$. Then
\begin{align*}
P   & = \{ x + \1\geq\0,\ (x + \1)\T B\leq\1 \} = \\
    & = \{ x\in\reals^m\ |\ -x_i\leq 1\text{ for }i\in [m],\
    x\T (b_j/(1 - \1\T b_j)) \leq 1\text{ for }j\in [n] \}.
\end{align*}
The polar of $P$ is then
\begin{equation}\label{p-l-dual}
P^\Delta = \conv(\{-e_i\ |\ i\in [m] \}\ \cup\ \{ c_j\ |\ j\in [n] \})
\end{equation}
where $c_j=b_j/(1 - \1\T b_j)$.
Since $P$ and $P^{\Delta}$ have $\0$ in their interior,
$P^{\Delta\Delta}=P$. Furthermore,
$P^\Delta $ is simplicial and its facets correspond to the vertices of $P$
and vice versa.
We label the vertices
of $P^\Delta $ as the corresponding facets in $P^l$, so the completely
labeled facets of $P^\Delta $ correspond to the completely
labeled vertices of $P^l$.
In particular, the facet corresponding to $\0$ is
\begin{equation}
\label{f-0}
\arraycolsep.3em
\begin{array}{rcl}
F_0 & = & \{ x\in P^\Delta\ \mid\ -\1\T x = 1 \}\\
    & = & \conv\{-e_i \mid i\in [m] \}.
\end{array}
\end{equation}
Theorem \ref{unit-vector-thm} then translates to the following.

\begin{theorem}{\rm (Balthasar \cite{balthasar})}
\label{unit-vector-dual-thm}
Let $P^\Delta $ be a labeled $m$-dimensional simplicial polytope with \0 in
its interior and vertices $e_1,\ldots,e_m,c_1,\ldots,c_n$ such that
$F_0$ in $(\ref{f-0})$ is a facet of $P^\Delta $.

Let $(U,B)$ be a unit vector game, with
$U=(e_{l(1)}\ \cdots\ e_{l(n)})$
for a labeling $l:[n]\to [m]$ and $B = [\,b_1\,\cdots\,b_n\,]$, where
$b_j = c_j/(1 + \1\T c_j)$ for $j\in [n]$.

Let $l_v$ be the labeling
of the vertices of $P^\Delta $ given by
\begin{equation}
\label{vert-labeling-unitv}
\arraycolsep.2em
\begin{array}{rcll}
l_v(-e_i)&=&i\quad & \text{ for }i\in [m],\\
l_v(c_j)&=&l(j)\quad & \text{ for }j\in [n].
\end{array}
\end{equation}
Then a facet $F\neq F_0$ of $P^\Delta $ with normal vector $v$ is completely
labeled if and only if $(x,y)$ is a Nash equilibrium of $(U,B)$, where
$x = (v + \1) / (\1\T (v + \1))$, so that
$x_i = 0$ if and only if $−e_i\in F$ for $i\in [m]$
and the mixed strategy $y$ is the uniform distribution on the set of
the pure best replies to $x$, which in turn correspond to all $j\in [n]$
such that $c_j$ is a vertex of $F$.
\end{theorem}

Theorem \ref{unit-vector-thm} gives a correspondence between
completely labeled vertices of $P^l$ and equilibria of the unit vector game
$(U,B)$ with the ``artificial'' equilibrium corresponding to the vertex $\0$.
Theorem \ref{unit-vector-dual-thm} gives a correspondence between
completely labeled facets of $P^\Delta $
and equilibria of $(U,B)$ with the ``artificial'' equilibrium
corresponding to the facet $F_0$ in (\ref{f-0}).

% ... here you come back to the old Q!
Given a bimatrix game $(A,B)$, it takes polynomial time
to write and solve the linear equations defining its best response polyhedra
$\overline{P},\overline{Q}$ and its best response polytopes $P,Q$.
It also takes polynomial time to label $\overline{P},\overline{Q}$ and $P,Q$.
Analogously, given a unit vector game $(U,B)$, it takes polynomial time
to construct and label the polytope $P^l$ and its polar.
Therefore, Theorem \ref{unit-vector-thm} gives a polynomial time
reduction from the problem 2-{\sc Nash} to the
problem {\sc Another Completely Labeled Vertex} of Table \ref{aclv} and
Theorem \ref{unit-vector-dual-thm} gives a dual reduction to the problem
{\sc Another Completely Labeled Facet} of Table \ref{aclf}.

\begin{problem}
{Another Completely Labeled Vertex}
{An $m$-dimensional simple polytope $P$ with $m+n$ facets;
a labeling $l_f:[m+n]\to [n]$;
a vertex $v_0$ of $P$ that is completely labeled by~$l_f$.}
{A vertex $v\neq v_0$ of $P$ that is completely labeled by $l_f$.}
\label{aclv}
\end{problem}

\begin{problem}
{Another Completely Labeled Facet}
{A simplicial $m$-dimensional polytope $P^\Delta$ with $m+n$ vertices;
a labeling $l_v:[m+n]\to [n]$;
a facet $F_0$ of $P^\Delta$ that is completely labeled by $l_v$.}
{A facet $F\neq F_0$ of $P^\Delta$ that is completely labeled by $l_v$.}
\label{aclf}
\end{problem}

\begin{proposition}
\label{nash-to-acl}
{\sc 2-Nash} reduces in polynomial time to
{\sc Another Completely Labeled Vertex} and
{\sc Another Completely Labeled Facet}.
\end{proposition}


\subsection{Real series}\label{subsec:real_series}

\begin{proposition}\label{thm:almost_all_terms_positive_implies_absolute_convergent}
  If only finitely many coefficients in a real \hyperref[def:convergent_series]{convergent} series are negative, then the series converges absolutely.
\end{proposition}
\begin{proof}
  Let \( N \) be the index of the last negative coefficient in \eqref{def:convergent_series/series}. Then the series
  \begin{equation*}
    \sum_{k={N+1}}^\infty a_k
  \end{equation*}
  is absolutely convergent since every coefficient is positive. Then
  \begin{equation*}
    \sum_{k=0}^\infty \abs{a_k} = \sum_{k=0}^N \abs{a_k} + \sum_{k=N+1}^\infty \abs{a_k}
  \end{equation*}
  is convergent since the first term on the right side is a finite sum and the second is a convergent series. Hence, the series \eqref{def:convergent_series/series} converges absolutely.
\end{proof}

\begin{corollary}\label{thm:almost_all_terms_negative_implies_absolute_convergent}
  If only finitely many coefficients in a real \hyperref[def:convergent_series]{convergent} series are positive, then the series converges absolutely.
\end{corollary}

\begin{theorem}[Riemann's series permutation theorem]\label{thm:riemanns_series_permutation_theorem}\mcite[\textnumero 247]{Фихтенгольц1968Том2}
  If the real series
  \begin{equation*}
    \sum_{k=0}^\infty a_k
  \end{equation*}
  is \hyperref[def:convergent_series]{convergent}, but not absolutely convergent, then for any extended real number \( x \in \BbbR \cup \{ -\infty, +\infty \} \) there exists a \hyperref[def:symmetric_group]{permutation} \( p \) of the coefficients \( a_0, a_1, a_2 \)
  such that
  \begin{equation*}
    \sum_{k=0}^\infty p(a_k) = x.
  \end{equation*}
\end{theorem}
\begin{proof}
  If the series is not absolutely convergent, then there exist both infinitely many positive and infinitely many negative coefficients.

  First, assume that \( x \) is finite.

  Define the permuted series
  \begin{equation*}
    \sum_{k=0}^\infty b_k
  \end{equation*}
  as follows:
  \begin{thmenum}
    \thmitem{thm:riemanns_series_theorem/positive} Assign to \( b_n \) only nonnegative elements of the sequence \( \{ a_k \}_{k=0}^\infty \) until \( \sum_{k=0}^n b_k \geq x \). Then go to \ref{thm:riemanns_series_theorem/negative}.
    \thmitem{thm:riemanns_series_theorem/negative} Assign to \( b_n \) only negative elements of the sequence \( \{ a_k \}_{k=0}^\infty \) until \( \sum_{k=0}^n b_k \geq x \). Then go to \ref{thm:riemanns_series_theorem/positive}.
  \end{thmenum}

  This mutual recursion builds a series that converges to \( x \) because the coefficients \( \{ a_k \}_{k=0}^\infty \) get arbitrarily close to each other.

  If \( x = +\infty \), we can add positive coefficients until \( \sum_{k=0}^n b_k \geq 1 \), then add a single negative coefficient, then continue adding positive coefficients until \( \sum_{k=0}^n b_k \geq 2 \) and, so on.

  If \( x = -\infty \), we use the same process, but with milestones of \( -1, -2, -3, \ldots \).
\end{proof}

\begin{example}\label{ex:riemanns_series_permutation_theorem/alternating_harmonic_series}\cite[\textnumero 247]{Фихтенгольц1968Том2}
  Consider the alternating harmonic series \eqref{eq:ex:harmonic_series/alternating}. Denote its sum by \( a \).

  We can rearrange this series by repeating two negative terms and a single positive term as follows:
  \begin{equation}\label{ex:riemanns_series_permutation_theorem/alternating_harmonic_series/rearranged}
    1 - \frac 1 2 - \frac 1 4 + \frac 1 3 - \frac 1 6 - \frac 1 8 + \cdots
    =
    \sum_{m=1}^\infty \left( \frac 1 {2m - 1} - \frac 1 {4m - 2} - \frac 1 {4m} \right).
  \end{equation}

  Note that \fullref{ex:riemanns_series_permutation_theorem/alternating_harmonic_series/rearranged} is equivalent to
  \begin{equation*}
    \sum_{m=1}^\infty \left( \frac 1 {2m - 1} - \frac 1 {4m - 2} - \frac 1 {4m} \right)
    =
    \sum_{m=1}^\infty \left( \frac 1 {4m - 2} - \frac 1 {4m} \right)
    =
    \frac 1 2 \sum_{m=1}^\infty \left( \frac 1 {2m - 1} - \frac 1 {42} \right)
    =
    \frac a 2.
  \end{equation*}
\end{example}

\begin{proposition}\label{thm:positive_series_comparison}\mcite[\textnumero 237]{Фихтенгольц1968Том2}
  Fix two nonnegative series
  \begin{equation}\label{def:positive_series_comparison/a}
    \sum_{k=0}^\infty a_k
  \end{equation}
  and
  \begin{equation}\label{def:positive_series_comparison/b}
    \sum_{k=0}^\infty b_k
  \end{equation}
  that is, series with nonnegative real coefficients. Assume that there exists an index \( K \) such that
  \begin{equation*}
    a_k \leq b_k \quad\forall k \geq K.
  \end{equation*}

  We say that the series \fullref{def:positive_series_comparison/b} \term{dominates} the series \fullref{def:positive_series_comparison/a}.

  Then
  \begin{thmenum}
    \thmitem{thm:positive_series_comparison/b_converges} If \fullref{def:positive_series_comparison/b} converges, so does \fullref{def:positive_series_comparison/a}.

    \thmitem{thm:positive_series_comparison/a_diverges} If \fullref{def:positive_series_comparison/a} diverges, so does \fullref{def:positive_series_comparison/b}.
  \end{thmenum}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:positive_series_comparison/b_converges} Suppose that \fullref{def:positive_series_comparison/b} converges. Then by \fullref{thm:real_monotone_sequence_converges_iff_bounded}, the sequence of partial sums is bounded. Therefore, the sequence of partial sums of \fullref{def:positive_series_comparison/a} is also bounded and, by \fullref{thm:real_monotone_sequence_converges_iff_bounded} again, the series is convergent.

  \SubProofOf{thm:positive_series_comparison/a_diverges} Analogous to \fullref{thm:positive_series_comparison/b_converges}, but using the negation of \fullref{thm:real_monotone_sequence_converges_iff_bounded}.
\end{proof}

\begin{proposition}[Cauchy's root test]\label{thm:cauchys_root_test}\mcite[thm. 3.33]{Rudin1976Principles}
  Consider the nonnegative series \fullref{def:positive_series_comparison/a}. Put
  \begin{equation*}
    q \coloneqq \limsup_{k \to \infty} \sqrt[k]{a_k},
  \end{equation*}
  where \( q = \infty \) if the limit does not exist. Then
  \begin{itemize}
    \item If \( q < 1 \), the series converges.
    \item If \( q > 1 \), the series diverges.
    \item If the limit does not exist (e.g. if \( a_k = k^k \)), the series diverges.
    \item If \( q = 1 \), the series may either converge or diverge.
  \end{itemize}
\end{proposition}
\begin{proof}
  The case when the limit \( q \) does not exist is obvious by the contraposition to \fullref{thm:convergent_series_terms_vanish}.

  Suppose that the limit exists. Therefore, there exists an index \( K \) such that
  \begin{equation*}
    \sqrt[k]{a_k} \leq q \quad\forall k \geq K.
  \end{equation*}

  Thus, we have the inequality
  \begin{equation*}
    a_k \leq q^k \quad\forall k \geq K.
  \end{equation*}

  The statement of the theorem now follows from comparison (\fullref{thm:positive_series_comparison}) with the geometric series \eqref{eq:def:geometric_progression/series}.
\end{proof}

\begin{proposition}[d'Alambert's ratio test]\label{thm:dalamberts_ratio_test}\mcite[thm. 3.33]{Rudin1976Principles}
  Consider the nonnegative series \fullref{def:positive_series_comparison/a}. Put
  \begin{equation*}
    q \coloneqq \limsup_{k \to \infty} \frac {a_{k+1}} {a_k},
  \end{equation*}
  where \( q = \infty \) if the limit does not exist. Then
  \begin{itemize}
    \item If \( q < 1 \), the series converges.
    \item If there exists an index \( k_0 \) such that \( \frac {a_{k+1}} {a_k} \geq 1 \) for all \( k \geq k_0 \), the series diverges.
    \item If the limit does not exist (e.g. if \( a_k = k! \)), the series diverges.
  \end{itemize}
\end{proposition}
\begin{proof}
  All cases except for \( q < 1 \) are obvious by the contraposition to \fullref{thm:convergent_series_terms_vanish}.

  Suppose that the limit exists. Therefore, there exists an index \( k_0 \) such that
  \begin{equation*}
    a_{k+1} \leq q a_k \quad\forall k \geq k_0.
  \end{equation*}

  Thus,
  \begin{equation*}
    a_{k_0 + m} \leq q^m a_{k_0} \quad\forall m \geq \BbbZ^{\geq 0}.
  \end{equation*}

  Convergence now follows from comparison (\fullref{thm:positive_series_comparison}) of the geometric series \eqref{eq:def:geometric_progression/series} with the subseries of \fullref{def:positive_series_comparison/a} obtained by trimming the first \( k_0 \) elements.
\end{proof}

\begin{proposition}\label{rem:nonnegative_series_convergence_test_equivalence}
  The values of \( q \) in \fullref{thm:cauchys_root_test} and in \fullref{thm:dalamberts_ratio_test} are equal.
\end{proposition}
\begin{proof}
  If we assume that they are not equal, then the same series would have to be convergent and divergent simultaneously in some region.
\end{proof}

\begin{definition}\label{def:alternating_series}
  Series of the form
  \begin{equation}\label{def:alternating_series/series}
    \pm \sum_{k=0}^\infty (-1)^k a_k,
  \end{equation}
  where all \( a_k, k = 0, 1, \ldots \) are nonnegative, are called \term{alternating}.
\end{definition}

\begin{proposition}[Leibniz' alternating series test]\label{thm:leibniz_alternating_series_test}
  Consider the alternating series \fullref{def:alternating_series}. If the sequence of terms \( \{ a_k \}_{k=0}^\infty \) decreases monotonically and if \( \lim_{k \to \infty} a_k = 0 \), then the series converges.
\end{proposition}

\begin{theorem}\label{thm:weierstrass_series_criterion_nessessity}\mcite[\textnumero 268]{Фихтенгольц1968Том2}
  \Fullref{thm:weierstrass_series_criterion} is a necessary condition for nonnegative real functions.
\end{theorem}


\section{Mixed States}
In many situations in physics we don't know what the wave function is,
but we do know that there is some probability of the wave function being
something.
So to compute things we need to take averages.
There are two averages.
One is quantum mechanical, but one is classical.
Probabilities in quantum mechanics don't add,
they might subtract.
Amplitudes add,
but there are cross terms.
The second average is a classical average,
which is due to my ignorance.

If one is to describe it with QM,
instead of having kets,
I'm going to have an operator called the density operator.
It's a bit different,
because even the state is an operator.
If I don't know which state I'm in,
say there's a probability $p_n$ of being $\ket{n}$ ,
then the density operator is
\begin{align}
    \rho &= \sum_{n} p_n \ket{n}\bra{n}
\end{align}
Then the expectation value of $A$ is
\begin{align}
    \bar{A} = \Tr(\rho A)
\end{align}
There are special states though.
If I know 100\% that my system is in state $\ket{\psi}$,
then in that sum,
the probability is zero in every state except for $\ket{\psi}$,
in which case the density operator is
\begin{align}
    \rho = \ket{\psi}\bra{\psi}.
\end{align}
And so the expectation value is
\begin{align}
    \Tr(\rho A) &=
    \Tr(\ket{\psi}\bra{\psi} A)\\
    &= \sum_n \braket{n}{\psi} \bra{\psi}A\ket{n}\\
    &= \sum_n \bra{A}A\ket{n}\braket{n}{\psi}\\
    &= \bra{\psi}A\ket{\psi}\\
    &= \bar{A}
\end{align}
where we made a sandwich of whatever basis $\ket{n}$ we wanted
and used the resolution of the identity.
So this reduces to the usual rule for pure states.

But what happens when we don't have a pure state,
but a mixed state,
in which case the $\Tr(\rho A)$ rule applies more generally.

It's not obvious, but there is a way to test if a density operator is a pure
state.
In fact,
$\rho$
is a pure state if and only if
$\rho^2 = \rho$.

And we also required that it be positive definite
\begin{align}
    \bra{\phi}\rho \ket{\rho} \ge 0.
\end{align}
Also,
$\Tr \rho = 1$.

Example: Thermodynamic ensemble
Suppose we have a system in thermal equilibrium.
We don't know where the microscopic particles are,
so there are certain probabilities of certain wave functions.
Turns out if you take most systems and let them evolve,
the wave function keeps changing,
but everything behaves as if we were in a basis state.
I'm going to point out the following here.
If I wait some time,
and make macroscopic measurements of energy, pressure, etc.
Everything behaves as if it were a density operator of the following form.
\begin{align}
    \rho &= \frac{e^{-\beta H}}{\Tr e^{-\beta H}}
\end{align}
where $H$ is Hermitian so $e^{-\beta H} = 1 - \beta H + \cdots$
is also Hermitian.
You can check that $H^2$ is hermitian with
$(H^2)^\dagger = (HH)^\dagger = H^\dagger H^\dagger = H^2$.
Don't be confused if you have an $i$ factor instead,
where $e^{itH}$ is unitary,
but $e^{-\beta H}$ is hermitian.
We have to divide by the real number $\Tr e^{-\beta H}$
to make the trace 1,
which is obvious,
because the trace of the numerator cancels the denominator.

But is it positive definite?
It's not obvious,
you can work it out yourself,
but the eigenvalues of $H$ are real,
and when you exponentiate a real number,
you get a positive number,
so the eigenvalues of $\rho$ must be positive.

This is a perfectly good density matrix.
It's sometimes written in a slightly different way.
I could write $\rho$ in the eigenbasis of energy
with matrix elements $\bra{n}\rho\ket{m}$.
In other words $H\ket{n} = E_n \ket{n}$.
\begin{align}
    \bra{n}\rho\ket{m} &=
    \frac{\bra{n} e^{-\beta H}\ket{m}}{\Tr e^{-\beta H}}\\
    &= \frac{e^{-\beta E_m} \delta_{nm}}{
        \sum_n \bra{n} e^{-\beta E_n}\ket{n}
    }\\
    &= \frac{e^{-\beta E_m} \delta_{nm}}{
        \sum_n e^{-\beta E_n}
    }.
\end{align}
I'm sure you're familiar with the top
$e^{-\beta E_n}$
which is just the Boltzmann factor.
States with large energy are unlikely,
the states that are more likely have smaller energy.
Large and small compared to what?

Remember that this $\beta E = \frac{E}{k_B T}$.
So for the exponential to be small,
we need $E$ to be large.
So higher temperatures are more likely to have more likely higher energy states.
We'll see this a lot later on.

Example: Photo polarization
I can have light linearly polarized in $x$ and $y$ directions,
and various superpositions like circular, left, right, etc.
But I can also have mixed states.
\begin{itemize}
    \item 50\% linearly polarized along $\hat{x}$ direction and
        50\% linearly polarized along $\ket{y}$ direction
        can be written as the state
        \begin{align}
            \rho =
            \frac{1}{2}\ket{\hat{x}}\bra{\hat{x}}
            + \frac{1}{2}\ket{\hat{y}}\bra{\hat{y}}
        \end{align}
        and this is just a specific case of the more general formula
        \begin{align}
            \rho = \sum_n p_n \ket{n}\bra{n}
        \end{align}
    \item Now consider the state with
        50\% probability of being left circularly polarized
        and 50\% probability of being right circularly polarized.
        Let's write down the density matrix for that.
        \begin{align}
            \rho &=
            \frac{1}{2} \ket{L}\bra{L}
            + \frac{1}{2} \ket{R}\bra{R}\\
            &=
            \frac{1}{2}
            \underbrace{
                \frac{
                    \ket{\hat{x}} + i\ket{\hat{y}}
                }{\sqrt{2}}
            }_{\ket{L}}
            \underbrace{
                \frac{
                    \bra{\hat{x}} - i\bra{\hat{y}}
                }{\sqrt{2}}
            }_{\bra{L}}
            + \frac{1}{2}
            \underbrace{
                \frac{
                    \ket{\hat{x}} - i\ket{\hat{y}}
                }{\sqrt{2}}
            }_{\ket{R}}
            \underbrace{
                \frac{
                    \bra{\hat{x}} + i\bra{\hat{y}}
                }{\sqrt{2}}
            }_{\bra{R}}\\
            &= \frac{1}{2}\ket{\hat{x}}\bra{\hat{x}}
            + \frac{1}{2}\ket{\hat{y}}\bra{\hat{y}}
        \end{align}
        so although the description in words they sound different,
        they are in fact the same state,
        and there is no experiment in the world that will allow you to
        differentiate between the two.
\end{itemize}
So the description in words in different,
but in reality they are different.

\subsection{Entropy (max)minimization}
Let me point something about entropy because people talk about it so I should
point it out as soon as possible.

You can talk about the entropy of a mixed state.
There are two concepts of entropy closely related.
We can define entropy the following way.
Suppose your system is defined by a density matrix $\rho$
that might be a mixed state.
You can compute the following quantity
\begin{align}
    S &= - \Tr \left( \rho \ln \rho \right)
\end{align}
But first,
do you know how to define the log of an operator?
Taylor expansion is one way?
Inverse of the exponential operator?
However,
if you can diagonalize the operator,
you can take any function you want,
because you can take the function of any diagonal matrix,
you just take the function of each eigenvalue on the diagonal,
no different from taking the exponential of an operator.
Why should we call this entropy?
Entropy is something you learn in high school.

For that,
let me write this in a slightly different way.
I'm going to take a basis and I'm going to write
\begin{align}
    S &=
    - \sum_n
    \bra{n} \rho \ln\rho \ket{n}
\end{align}
where $\ket{n}$ is an eigenbasis of $\rho$.
To make things a little better,
I can insert a complete set of states here.
\begin{align}
    S &=
    - \sum_{n,n'}
    \bra{n} \rho \ket{n'}\bra{n'}\ln\rho \ket{n}
\end{align}
and so you notice that because $\rho$ is diagonal in this basis,
\begin{align}
    S &= \sum_{n,n'} p_{n'}\delta_{nn'} \delta_{nn'}\ln p_n\\
    &= - \sum_n p_n \ln p_n
\end{align}
This expression you may have seen before in thermodynamics.
Why is this important?
Imagine if $\rhO$ is a pure state.
It's a pure state when I write $\rho$ as a sum like
\begin{align}
    \rho &= \sum_n p_n \ket{n}\bra{n} = \ket{\psi}\bra{\psi}
\end{align}
so all the $p_n=0$ except for one $n=n_0$ where $p_{n_0}=1$.
So what does that mean?
There are two kinds of terms in the sum,
terms where $p_n=0$,
and only one term where $p_n=1$,
so then
\begin{align}
    S &= - 0 \ln 0 - 1\ln 1 = 0
\end{align}
So if I have a pure state
and I compute the entropy,
the entropy is zero.

\begin{question}
    What about superpositions?
    Do we get non-zero entropy?
\end{question}
No no no no!
Let's take the harmonic oscillator
and say we have an excited state like
\begin{align}
    \rho &=
    \underbrace{\left( \frac{3}{5}\ket{0} + \frac{4}{5}\ket{1} \right)}_{%
        \ket{\psi}
    }
    \underbrace{\left( \frac{3}{5}\bra{0} + \frac{4}{5}\ket{1} \right)}_{%
        \bra{\psi}
    }\\
    &\ne \left( \frac{3}{5} \right)^2 \ket{0}\bra{0}
    + \left( \frac{4}{5} \right)^2 \ket{1}\bra{1}
\end{align}
because of the cross terms.

Anyway,
let us consider the uniform distribution
$p_n=1/N$
and the dimension $N$ of the Hilbert space is finite.
Then the entropy is
\begin{align}
    S &= -\sum_n p_n\ln p_n
    = -\frac{N}{N}\ln \frac{1}{N} = \ln N
\end{align}
and it turns out that this is the maximum value the entropy can be.
It represents complete ignorance.

\begin{question}
    This is just the Shannon entropy of each hermitian operator viewed as a
    random variable?
\end{question}
Yes, I can't parse that precisely but sure.

\begin{question}
    Is $N$ the same thing as the number of basis states?
\end{question}
Yes, that's the definition of dimension.

\subsection{Subsystems and mixed states}
The idea is the following.
Imagine I have a system made up of two subsystems.
If the system is made up of two subsystems,
then the total space is the tensor product.
The generic state of a composite system can be written like this
\begin{align}
    \ket{\psi} &=
    \sum_{n,m} \psi_{n,m}
    \ket{n}\otimes\ket{m}
\end{align}
where $\ket{n}\otimes\ket{m}$ forms the basis for the composite system
$\mathcal{H}$,
$\ket{n}$ forms a basis for $\mathcal{H}_1$ and
$\ket{m}$ forms a basis for $\mathcal{H}_2$.

Now imagine if you have an operator $A$,
but I only want to measure one subsystem.
The operator $A\otimes \mathbf{1}$ does this.
\begin{align}
    A\otimes \mathbf{1} \ket{\phi}\otimes\ket{\chi}
    &=
    A \ket{\phi} \otimes \ket{\chi}
\end{align}
It does nothing to subsystem 2.

Suppose I want to find an expectation value like this.
But I'm going to assume the system is in what is called an entangled state,
something like this
\begin{align}
    \ket{\psi} =
    \frac{%
        \ket{\phi_1}\otimes\ket{\chi_1}
        + \ket{\phi_2}\otimes\ket{\chi_2}
    }{\sqrt{2}}
\end{align}
There is a correlation between particles 1 and 2.
This is called \emph{entanglement}.
It's a perfectly good state,
so let's see what happens.
\begin{align}
    \bra{\psi} A\otimes\mathbf{1} \ket{\psi} &=
    \frac{%
        \ket{\phi_1}\otimes\ket{\chi_1}
        + \ket{\phi_2}\otimes\ket{\chi_2}
    }{\sqrt{2}}
    A\otimes\mathbf{1}
    \frac{%
        \bra{\phi_1}\otimes\bra{\chi_1}
        + \bra{\phi_2}\otimes\bra{\chi_2}
    }{\sqrt{2}}\\
    &=
    \frac{1}{2}
    \bra{\phi_1}A\ket{\phi_1}
    \bra{\chi_1}1\ket{\chi_1}
    +
    \frac{1}{2}
    \bra{\phi_1}A\ket{\phi_2}
    \bra{\chi_1}1\ket{\chi_2}\\\nonumber
    &\qquad+
    \frac{1}{2}
    \bra{\phi_2}A\ket{\phi_1}
    \bra{\chi_2}1\ket{\chi_1}
    +
    \frac{1}{2}
    \bra{\phi_1}A\ket{\phi_1}
    \bra{\chi_2}1\ket{\chi_2}\\
    &=
    \frac{1}{2}\bra{\phi_1}A\ket{\phi_1}
    + \frac{1}{2}\bra{\phi_2}A\ket{\phi_2}
\end{align}
But notice that we have seen this before and it is the same as an incoherent sum
\begin{align}
    \frac{1}{2}\bra{\phi_1}A\ket{\phi_1}
    + \frac{1}{2}\bra{\phi_2}A\ket{\phi_2}
    =\Tr\left[
        \underbrace{\left(
            \frac{1}{2}\ket{\phi_1}\bra{\phi_1}
            + \frac{1}{2}\ket{\phi_2}\bra{\phi_2}
        \right)}_{\rho_1} A
    \right]
\end{align}
The situation with a pure state over 2 systems is indistinguishable from having
a classical mixed state in system 1.
It's like you destroyed the interference!
This density matrix $\rho_1$ you have here is only for subsystem 1
If I had no access to subsystem 2,
like if I moved it to another galaxy,
all I would see is a classical mixed state in subsystem 1.

[break]

Let me repeat this same calculation in a slightly more general setting so you
understand the general rule.
Suppose you want to calculate the expectation value of a pure state of a
composite system.
The most general composite state looks like this
\begin{align}
    \sum_{n'm'}\ket{n'}\otimes\ket{m'}
\end{align}
This is a little tricky,
so I'm going to do it slowly.
\begin{align}
    \bra{\psi}A\otimes\mathbf{1}\ket{\psi}
    &=
    \sum_{n',m',n,m}
    \bra{n}\otimes\bra{m}
    \psi_{nm}^*
    A\otimes\mathmbf{1}
    \psi_{n'm'}\ket{n'}\otimes\ket{m'}\\
    &=
    \sum_{n',m',n,m}
    \psi_{nm}^* \psi_{n'm'}
    \bra{n}A\ket{n'}
    \underbrace{\bra{m}\mathbf{1}\ket{m'}}_{\delta_{mm'}}\\
    &=
    \sum_{nn'}\bra{n}A\ket{n'}
    \sum_m\psi_{nm}^* \psi_{n'm}\\
    &= \Tr\left[
        \underbrace{\sum_{n,m,n'} \psi_{nm}^* \psi_{n'm}\ket{n}\bra{n'}}_{%
            \rho_1
        }
        A
    \right]
\end{align}
Note that the proper way to compute a trace is to sandwich a complete basis on
each side and sum over.
Using the cyclic property of the trace is not a valid way of doing it,
but it just happens to work in this case.

Tensor products look like a mouthful on paper,
but it's completely trivial to manipulate.

Anyway,
the point of this is that $\rho_1$ may be a mixed state of only subsystem 1.
That's the price you pay for not knowing about subsystem 2.

Let's doe something exciting.
Shooting particles through holes and interference.
People say that if you somehow have a way of looking which way the particle goes
through,
just by having the possibility of knowing,
the interference pattern gets destroyed.
It's a bit mysterious because of the following.
If you know the particle is left or right,
you make a measurement,
and you collapse the system.

If it was a superposition,
because of measurement,
you collapse it.
\begin{align}
    \frac{\ket{L} + \ket{R}}{\sqrt{2}}
    \to
    \ket{L}
\end{align}
But what does it even mean to make a measurement.
Suppose you had a machine that automatically writes left or right,
but nobody looks at that piece of paper,
is that enough to destroy the interference or not.
Let me not use the collapse of the wave function,
but using the knowledge we just learnt.

Suppose my state is a superposition
\begin{align}
    \ket{\psi_0} &=
    \frac{\ket{L} + \ket{R}}{\sqrt{2}}
\end{align}
but suppose I have a piece of equipment that spies on the particle.
But now I have both the particle and the spy
as a composite system.
My spy system $\ket{\text{spy}}$ lives in a Hilbert space
$\mathcal{H}_{\text{spy}}$,
which could be quite complicated
because it's a complicated device.

\begin{align}
    \ket{\psi_0} &=
    \frac{\ket{L} + \ket{R}}{\sqrt{2}}
    \otimes
    \ket{\text{spy}}
\end{align}
But then it evolves according to a Hamiltonian $H$ and we get
\begin{align}
    \ket{\psi} &=
    \frac{\ket{L} + \ket{R}}{\sqrt{2}}
    \otimes
    \ket{\text{spy}'}
\end{align}
But now it's evolved in a way completely independent of the particle,
but that's not useful because it's not spying!
So what we really want is for the spy to distinguish between left and right,
so we something like this.
\begin{align}
    \ket{\psi} &=
    \frac{1}{\sqrt{2}}
    \ket{L}
    \otimes
    \ket{\text{spy,left}}
    +
    \frac{1}{\sqrt{2}}
    \ket{R}
    \otimes
    \ket{\text{spy,right}}
\end{align}
That is a 100\% precise measurement by the spy device.
Now I'm going to ask the following question.
The spy device going left or right is writen on the piece of paper,
but no one is going to look at it.
It's not really a measurement.
Now suppose the paper is burnt,
destroyed.

The particles are still going to hit the screen,
so what I'm going to be interested in doing is measure things,
say $A$,
that refer only to the particle,
so the operator is $A\otimes\mathbf{1}$.
If I only measure things like this,
then I know everything that I can possibly measure about the system
is contained in a density matrix obtained the same as before.

Take the density matrix intially which is a pure state
\begin{align}
    \rho &= \ket{\psi}\bra{\psi}\\
    &=
    \sum_{n,m,n',m'} \psi_{n'm'}\ket{n'}\otimes\ket{m'}\\
    \bra{n}\otimes\bra{m}\psi_{nm}^*\\
    &\to
    \sum_{n,m,n'} \psi_{nm}^* \psi_{n'm}\ket{n}\bra{n'}
    = \rho_1
\end{align}
But how did I get there?
I had to take a trace over system 2.
\begin{align}
    \sum_{m''}\bra{m''}\rho\ket{m''} &=
    \sum_{m''}\bra{m''}\ket{\psi}\bra{\psi}\ket{m''}\\
    &=
    \sum_{m''}\bra{m''}\sum_{n,m,n',m'} \psi_{n'm'}\ket{n'}\otimes\ket{m'}
    \bra{n}\otimes\bra{m}\psi_{nm}^*\ket{m''}\\
    &\to
    \sum_{n,m,n'} \psi_{nm}^* \psi_{n'm}\ket{n}\bra{n'}
    = \rho_1
\end{align}

This procedure here,
there's a name of it,
it's called taking the trace of subsystem 2.
\begin{align}
    \Tr_2 \rho = \sum_{m''}\bra{m''} \rho \ket{m''}
\end{align}
where $\ket{m''}$ form a basis for subsystem 2.
You will be left with a possibly mixed state $\rho_1$ for only subsystem 1.
This is sometimes called the \emph{partial trace},
an operation that is very common
and you should know this.

Let's go back to the problem.
So we have the state for subsystem 1
\begin{align}
    \rho_1 &=
    \Tr_2 \rho\\
    &=
    \Tr_2
    \left(
        \frac{1}{\sqrt{2}}\ket{L}\otimes\ket{\text{spy,l}}
        + \frac{1}{\sqrt{2}}\ket{R}\otimes\ket{\text{spy,r}}
    \right)
    \left(
        \frac{1}{\sqrt{2}}\bra{L}\otimes\bra{\text{spy,l}}
        + \frac{1}{\sqrt{2}}\bra{R}\otimes\bra{\text{spy,r}}
    \right)\\
    &=
    \bra{\text{spy.l}}
    \left(
        \frac{1}{\sqrt{2}}\ket{L}\otimes\ket{\text{spy,l}}
        + \frac{1}{\sqrt{2}}\ket{R}\otimes\ket{\text{spy,r}}
    \right)
    \left(
        \frac{1}{\sqrt{2}}\bra{L}\otimes\bra{\text{spy,l}}
        + \frac{1}{\sqrt{2}}\bra{R}\otimes\bra{\text{spy,r}}
    \right)
    \ket{\text{spy.l}}\\\nonumber
    &\qquad+
    \bra{\text{spy.r}}
    \left(
        \frac{1}{\sqrt{2}}\ket{L}\otimes\ket{\text{spy,l}}
        + \frac{1}{\sqrt{2}}\ket{R}\otimes\ket{\text{spy,r}}
    \right)
    \left(
        \frac{1}{\sqrt{2}}\bra{L}\otimes\bra{\text{spy,l}}
        + \frac{1}{\sqrt{2}}\bra{R}\otimes\bra{\text{spy,r}}
    \right)
    \ket{\text{spy.r}}\\
    &= \frac{1}{2}\ket{L}\bra{L} + \frac{1}{2}\ket{R}\bra{R}
\end{align}
This gives the density matrix of the particle
and you've seen this before.
It describes the situation where you don't know where the particle is,
and it could left or right.
In particular,
If I try to compute expectation values of some of the particles,
I just get the usual thing of incoherent sum, no intereference.
\begin{align}
    \bar{A} &=
    \Tr \rho_A\\
    &= \frac{1}{2}\bra{L}A\ket{L}
    + \frac{1}{2}\bra{R}A\ket{R}
\end{align}
but I'm now going to look at this piece of paper and it doesn't matter.
The interference pattern is gone.
It works whether it's written by the piece of paper or not.
It could be a single photon going through the particle,
and that reveals the position of the particle,
because the future state of the photon depends where the particle is.
That entanglement means I end up with a situation indistinguishable from a
coherent sum.
A single photon going through the experiment
correlating with the particle destroys the interference.

What counts as a measurmenet?
one thing you can tell from our calculation.
It's enough to have anything entangling with the particle and you won't be able
to see the intereference.
it doesn't have to be a monkey, a grad student, or whatever,
a single photon correlation is enough.

\begin{question}
    You call it incoherent because it has no inteference terms?
\end{question}
Yes that's right, some lingo I introduced before.

\begin{question}
    What if we're not aware of the entanglement?
    Will it stil affect the system?
\end{question}
It doesn't matter what Paulo sees,
any system entangling with the particle will destroy interference.
Doesn't matter if I know about it or not.

Let me make a comment.
what this suggests is that to see intference effects is tricky in pratice.
A single straight photon will destroy it.
It's not hard with light.
But if you tried doint it with electrons,
it's hard.
Photons don't interact with each other,
but there's always light to interact with electrons.
So to do experiments with electrons is difficult.

There's great work recently with quantum devices of all kinds
Quantum computers being the most famous ones.
It's why quantum computers are so hard to make.
Your quantum coputer needs a hilbert space of some dimension,
but all the coherence will disappear if your qubits get entangled with the
outside world,
and that could be anything,
even the metal frame that's holding it.
You have to make sure the whole thing that is supporting your device doesn't get
entangled with it.
certainly you have to make sure it's dark,
because a little photon can interact with it and it can entangle,
and you get a mixed state.
It's a tremendous technological challange.
It's not hard to get this in the lab,
it's how they discovered QM.
But instead of an atom,
but a bigger system,
certainly a macroscopic thing,
it's hard.
There's awhole fued nowadays in physics,
quantum registers or other systems,
could they keep a superposition.
I won't be surprised if half of you spend half your life trying to do this
stuff.
Any other comments?

\begin{question}
    Future of quantum computing?
\end{question}
I cannot predict the future,
but I can tell you
I have not invested in quantum computing companies.
I can tell you I was on the ground on Amazon,
and I did not buy Amazon stock either.
I was like this is stupid,
who wants to buy books,
books are heavy.
I did not buy Amazon stock,
I'm not buying quantum computing stock.

Someone's comment is that the effect of cosmic rays in classical computers is a
thing.
It may happen, it does happen.
It happens a lot in satellites though.
Cosmic rays are like a hammer,
it cna destroy annything,
including classical informaiton.
What I'm saying is way worse than this.
Classical computers don't even use error correcting codes.
They're so precise they don't hvae to.
Quantum computers are a non-starter without quanutm error correction.
I'm not an experimentalist,
I don't know,
but I'm excitedly waiting for a quantum computer.
The difficulty is that you don't need a measurement to collapse a wave funciton,
all you need is a bit of entangelmetn.

There are many other things that can entangle.

We have 9 minutes left.
I have more stuff to talk about,
but about a little bit of a different topic.

\begin{question}
    Before yo urevisited the double slit expreiment,
    we were looking at the expectation of $A$.
    If we have the same $\psi$ on both sides,
    why do we need separate indices on both sides $m'$ and $n'$.
\end{question}
Every time there's an issue with sums and indices,
the best thing to do is do an example.
Let's say $n,m,\ldots, = 1,2$.
Say your state is
\begin{align}
    \ket{\psi} = \frac{\ket{1} + \ket{2}}{\sqrt{2}}
\end{align}
Then we have
\begin{align}
    \bra{\psi}A\ket{\psi} &=
    \frac{\bra{1} + \bra{2}}{\sqrt{2}} A
    \frac{\ket{1} + \ket{2}}{\sqrt{2}}\\
    &=
    \frac{1}{2} \bra{1} A\ket{1}
    + \frac{1}{2} \bra{2} A\ket{2}
    + \frac{1}{2} \bra{1} A\ket{2}
    + \frac{1}{2} \bra{2} A\ket{1}
\end{align}
Great, let's write things in terms of indices now?
Is this the same thing as
\begin{align}
    \sum_{n=1}2\bra{n}A\ket{m}?
\end{align}
Clearly not,
but it is the same as
\begin{align}
   \sum_{n,m=1}2\bra{n}A\ket{m}?
\end{align}
You can settle these issues by doing an example that's trivial,
and it's immediately completely obvious.

\begin{question}
    If I think of that line conceptually as no intference,
    we have a particle and it got entangled with another system,
    and we lost informaitno about the other system,
    and is that why we don't have inteference.
\end{question}
I agree with everything we said,
but it doesn't mean one expalins another.
I find it completely mind-boggling
that I start with a pure state,
and if I look at just a subsystem,
and I get a mixed state.
If somehow saying those words makes you happy,
then everything is alrgith.
You're smarter than me.
Yeah, I don't know what to say.
I don't find anything at all.
I just got used to it.

A week ago,
if I told you this,
you'd say oh yeah correlaing wiht another particle,
nitference should disapear.

You've heard the double slit experiment a million times,
and yo've been told if you look at it it will disappear.
Did you know that entangling is a measurment devices and they destroy
inteference.
When I was a sutdent this wasn't explained.
Everything you said was right,
that's the only thing I can tell you.

\begin{question}
    Does the uniform distirbution have maximum entropy?
    But what's the distribution that maximises entropy for the haronic
    osciollator?
\end{question}
Yes.
There is no upper bound for the entropy of an infinite dimensinoal system.
Entropy increases with temperature,
and there is no bound for temperature.
The entropy does have a maximum for finite-state systems.

Tentative new office hours Friday 1pm.