\section{Regular Languages and Finite State Automata}

% \subsection{Description}
In this section we will introduce the definitions of \textit{Regular Language} and \textit{Automaton} over an alphabet $\Sigma$ since these notions are crucial to understand the two algorithms L* and NL*.
% An automaton $\A$ is a computing device used in abstract computer science to understand regular languages seen as a set of words formed with the concatenation of zero or more letters \letter{} belonging to an alphabet noted \alphabet{}. We note $\Sigma^*$ as the set of all words over the alphabet $\Sigma$, including $\E$: the word of length 0.

\begin{theorem}[Kleene's theorem]
  \label{th:kleene}
  A language is called regular if it can be represented by a regular expression or equivalently by a finite state automaton.
\end{theorem}

Let $\Sigma^*$ be the set of all words over the alphabet $\Sigma$ (including the word of length $0$ noted $\E$).

A recursive definition of a regular language can be expressed by the following grammar:
\[ L = \varnothing \mid \E \mid \alpha \mid L \cup L \mid \overline{L} \mid L \cdot L \mid L^* \]
where:
\begin{itemize}
  \item $\varnothing$ is the empty language;
  \item $\E$ is the word of length 0;
  \item $\alpha$ is a symbol belonging to the alphabet $\Sigma$;
  \item $L_1 \cup L_2 = \{\omega \in \Sigma^* \mid \omega \in L_1 \vee \omega \in L_2\}$, which is the union of $L_1$ and $L_2$ ;
  \item $L_1 \cdot L_2 = \{\omega \in \Sigma \mid \exists w_1 \in L_1, \exists w_2 \in L_2 \text{ and } w_1 \cdot w_2 = \omega\footnote{Note $ w_1 \cdot w_2$ is the concatenation of the word $w_1$ followed by the word $w_2$}\}$, which is the concatenation of all words in $L_1$ with all words in $L_2$;
  \item $\overline{L} = \{\omega \in \Sigma^* \mid \omega \notin L\}$, called the complement of the language $L$;
  \item To define $L^*$ let $L^0 = {\E}$, $L^1 = L$ and $L^{i+1} = L^i \cdot L$, then $L^* = \bigcup\limits_{i=0}^{\infty} L_{i}$. This operation is called the \textit{Kleene's star}
\end{itemize}

Some properties are not listed since they can be derived from the previous definition, for example the intersection betweem languages ($L_1 \cap L_2 = \overline{\overline{L_1} \cup \overline{L_2}}$).

\begin{definition}[Automaton]
  An automaton $\A$ is represented by a 5-tuple $\left\langle \Sigma, \Q, \delta, q_I, F\right\rangle $ where \alphabet{} is the alphabet, \states{} is the set of states composing the automaton, $\delta$ is the set of transition, \qzero{} $\in \statesN{}$ is the initial state and $F \subseteq \Q$ is the set of accepting states. A transition $\delta$ is a mapping $Q \times \Sigma \rightarrow Q$.
\end{definition}

\begin{example}
  A transition like $\delta(q_i, \alpha) = q_j$ means that we can move from state $q_i$ to state $q_j$ when reading symbol $\alpha$.
\end{example}

A state $q_i$ is called successor of $q_j$ if it exists $\delta(q_i, \alpha) = q_j$ for some $\alpha \in \Sigma$. If $q_j$ is successor of $q_i$ then $q_i$ is the predecessor of $q_j$.

Following the construction of the automaton $\A$, and especially focusing on its transition function, it is possible to classify $\A$ into two different classes.

\begin{definition}[DFA and NFA]
  If for every state $q_i \in \statesN{}$ and for every letter $\alpha \in \Sigma$ there exists only one successor by $\alpha$ then the automaton is \textit{deterministic} and noted \textit{DFA}, otherwise the automaton is called \textit{non-deterministic} and noted \textit{NFA}.
\end{definition}

We can define a \textit{DFA} also with a logic formula:
\[\forall q_i \in \Q, \forall \alpha \in \Sigma \text{ if } \exists \delta_1, \delta_2 \in \delta \text{ such that } \delta_1(q_i, \alpha) = q_j' \text{ and } \delta_2(q_i, \alpha) = q_j'' \text{ then } q_j' = q_j'' \]

Since every regular language can be represented by an automaton, be it deterministic or not, the operations like intersection, union and complementation are also available on automata. Among these properties, the most difficult to compute is the complementation when the automaton is non-deterministic as it asks first to make the \textit{NFA} deterministic \footnote{This operation is called determinisation} which is a \textit{PSPACE-complete} problem.

However, \textit{NFA}s have a big advantage since they can be exponentially smaller than the corresponding \textit{DFA} and this may be interesting mainly if we care about space resources.

\begin{remark}
  \textit{NFA} and \textit{DFA} have the same expressive power meaning that \[\bigcup NFA = \bigcup DFA = \bigcup RegularLanguages\]
\end{remark}

\begin{definition}[Minimal automaton]
  \label{lemma:minimalAut}
  An automaton $\A$ is minimal if it is not possible to construct a smaller automaton $\A'$ for the same language having less states than $\A$. Moreover, a minimal \textit{DFA} (noted \textit{mDFA}), for a given regular language, is \textit{unique}. However there may exists several minimal \textit{NFA}, noted \textit{mNFA}, for a regular language $\U$.
\end{definition}

Finally, we define, Regular Expressions (or \textit{RegEx}) with the following grammar:
\[R = \omega \mid R + R \mid R \cdot R \mid R^* \mid (R)\]
where $\omega$ is a word belonging to $\Sigma^*$, the ``$+$'' symbol indicates the union operator, ``$\cdot$'' is the concatenation of two regular expression, the ``$*$'' is defined as for regular languages and finally parentheses allows to introduce priority over the other operators.

\begin{example}[DFA vs NFA]
  In \cref{fig:nfa_vs_dfa} we have an example of two minimal automata for the language $L = (a+b)^*a(a+b)$ where we can see that the NFA needs less states then the corresponding minimal DFA.
  \begin{figure}[h]
    \centering
    \begin{subfigure}{0.45\textwidth}
      \centering
      \begin{tikzpicture}[shorten >=1pt,node distance=2cm,on grid,auto]
        \node[state,initial, initial text=] (q_I) {$q_I$};
        \node[state] (q_1) [right=of q_I] {$q_1$};
        \node[state, accepting] (q_2) [right=of q_1] {$q_2$};
        \path[->]
        (q_I) edge [loop above] node {a,b} ()
        (q_I) edge  node {a} (q_1)
        (q_1) edge  node {a,b} (q_2);
      \end{tikzpicture}
      \caption{NFA}
      \label{subfig:nfa_x_star_ax}
    \end{subfigure}
    \begin{subfigure}{0.45\textwidth}
      \centering
      \begin{tikzpicture}[shorten >=1pt,node distance=2cm,on grid,auto]
        \node[state,initial, initial text=] (q_I) {$q_I$};
        \node[state] (q_1) [right=of q_I] {$q_1$};
        \node[state, accepting] (q_2) [above right=of q_1] {$q_2$};
        \node[state, accepting] (q_3) [below right=of q_1] {$q_3$};
        \path[->]
        (q_I) edge [loop above] node {b} ()
        (q_2) edge [loop above] node {a} ()
        (q_I) edge  node {a} (q_1)
        (q_1) edge  node {a} (q_2)
        (q_1) edge  node {a} (q_2)
        (q_2) edge  node {b} (q_3)
        (q_3) edge[bend left, below]  node {b} (q_I)
        (q_3) edge[bend left, below]  node {a} (q_1)
        (q_1) edge[bend left, above]  node {b} (q_3);
      \end{tikzpicture}
      \caption{DFA}
      \label{subfig:dfa_x_star_ax}
    \end{subfigure}
    \caption{Minimal NFA vs minimal DFA}
    \label{fig:nfa_vs_dfa}
  \end{figure}
\end{example}

\subsection{Some notation}
We will note an automaton $\A$, $L = \LA$ means that the regular language recognized by the automaton $\A$ is $L$. An expression of type $L = (a+b)$ indicates that $L$ is recognized by the regular expression $(a+b)$. If not specified we suppose $\Sigma$ to be equal to ${a, b}$ and $\omega$ be a word over $\Sigma^*$.


% !TeX spellcheck = en_EN
%\documentclass[12pt,a4paper]{article}
%\usepackage{import}
%\subimport{../}{uebung.tex}

%\toggletrue{showSolution}

%\title{\thetitle{}\\Tutorial 4}
%\date{\WeeksAfter{3}}

%\begin{document}

%\maketitle

\BoSSSopen{tutorial4/tutorial4}
\graphicspath{{tutorial4/tutorial4.texbatch/}}

\BoSSScmd{
///\section*{What's new:}
///\begin{itemize}
///    \item{generating .plt-files for visualization}
///    \item{implementing a numerical flux}
///    \item{definition and evaluation of the spatial operator}
///    \item{explicit time integration}
///\end{itemize}
///\section*{Prerequisites:}
///\begin{itemize}
///    \item{projection onto a DG-field}
///\end{itemize}
///Within this tutorial, we are going to implement the scalar transport equation via the definition of a spatial operator and an explicit time integrator. The implementation of the numerical flux is described on the basis of a upwinding scheme. For the visualization of the results, we are generating .plt-files, which can be opened by a viewer of your choice
///\section{Problem statement}
///We are considering the following definition of the scalar transport equation with
///\begin{equation}
///   \label{eq:divergenceTerm}
///   \frac{\partial c}{\partial t} + \nabla \cdot (\vec{u} c) = 0,
///\end{equation}
///where $c = c(x,y,t) \in \mathbb{R}$ is the unknown concentration and
///\begin{equation*}
///     \vec{u} = \begin{pmatrix}
///         y\\-x
///     \end{pmatrix}
///\end{equation*}
///is a given velocity field in $\domain = [-1, 1] \times [-1, 1]$. Furthermore, the exact solution is given by
///\begin{equation*}
///    c_\text{Exact}(x,y,t) = \cos(\cos(t) x - \sin(t) y) \quad \text{ for } (x,y) \in \domain
///\end{equation*}
///In this tutorial we will simulate the evolution of the concentration $c(x,y,t)$.
///\section{Solution within the \BoSSS{} framework}
///We start a new project
 }
\BoSSSexeSilent
\BoSSScmd{
restart
 }
\BoSSSexeSilent
\BoSSScmd{
using System.IO;
 }
\BoSSSexe
\BoSSScmd{
/// Delete old plots in the current directory if any
Directory.GetFiles(".", "*.plt").ForEach(file => File.Delete(file));
 }
\BoSSSexe
\BoSSScmd{
/// \subsection{Projection and visualization}
/// In this first section we get to know the plotting tool \code{Tecplot}, which generates .plt-files of our \code{DGFields}.
/// Previously, we define the exact solution $c_{Exact}(x,y,t)$ and the scalar components of the velocity field $\vec{u}$ as functions,
Func<double[], double, double> cExact =  \newline
\btab (X, t) => Math.Cos(Math.Cos(t)*X[0] - Math.Sin(t)*X[1]);
 }
\BoSSSexe
\BoSSScmd{
Func<double[], double> u = X => X[1];
 }
\BoSSSexe
\BoSSScmd{
Func<double[], double> v = X => -X[0];
 }
\BoSSSexe
\BoSSScmd{
/// Next, we need to construct the computational domain, i.e a unit square with one cell.
double[] nodes = GenericBlas.Linspace(-1.0, 1.0, 2); \newline
GridCommons grid = Grid2D.Cartesian2DGrid(nodes, nodes); \newline
GridData gridData = new GridData(grid);
 }
\BoSSSexe
\BoSSScmd{
/// We instantiate the \code{SinglePhaseField} \emph{ch} with a \code{Basis} of DG-degreee of 2. Then we can project the initial value $c(x,y,0.0)$ onto \emph{ch}.
int dgDegree = 2;  \newline
Basis basis = new Basis(gridData, dgDegree);  \newline
SinglePhaseField ch = new SinglePhaseField(basis, "ch");  \newline
ch.ProjectField(X => cExact(X, 0.0));
 }
\BoSSSexe
\BoSSScmd{
/// Now, we can export the initial projection in our \code{Tecplot} format.
using BoSSS.Solution.Tecplot;
 }
\BoSSSexe
\BoSSScmd{
/// One important parameter for visualization is \textit{superSampling}. It is essential for higher order methods since almost all
/// plotting tools work with piecewise linear interpolations of the data in the vertices. For our
/// case, the plot with \code{superSampling=0} would just show a constant value! By increasing the
/// rate of the \emph{superSampling}, we provide more sampling points for the plot tool.
/// \begin{itemize}
/// \item This has nothing to do with the computation! Only required for visualization!
/// \item The number of sampling points grows exponentially with the value of
///  \code{superSampling}. Never use a value above 5 or 6!
/// \end{itemize}
 }
\BoSSSexe
\BoSSScmd{
uint superSampling = 0;  \newline
Tecplot tecplot    = new Tecplot(gridData, superSampling);\newline
tecplot.PlotFields( \newline
\btab "plot\_tutorial4\_superSampling0", \newline
\btab 0.0, \newline
\btab ch);
 }
\BoSSSexe
\BoSSScmd{
superSampling = 3;  \newline
tecplot    = new Tecplot(gridData, superSampling); \newline
tecplot.PlotFields( \newline
\btab "plot\_tutorial4\_superSampling3", \newline
\btab 0.0, \newline
\btab ch);
 }
\BoSSSexe
\BoSSScmd{
/// There should now be two plot-files in your current directory. Those can be opened by any standard viewer for .plt-files.
 }
\BoSSSexe
\BoSSScmd{
/// \subsection{Implementation of the (numerical) flux}
/// Before we can define the spatial operator for the scalar transport equation, we need to implement
/// the flux for the given problem, i.e. the flux of the divergence operator. A flux defines the volume term
/// (the \emph{flux}) as well as the boundary terms (via the \emph{numerical flux}).
/// We derive such a flux from the class \code{NonlinearFlux} which simplifies the implementation of fluxes in
/// flux-based formulations.
using BoSSS.Platform.LinAlg;  \newline
class ScalarTransportFlux : NonlinearFlux \{  \newline
 \newline
    /// \leftskip=1cm \code{ArgumentOrdering} defines on which arguments this flux depends, and in which order.
    /// Here, we have just one argument (the concentration $c$). The name is arbitrary at this point, but has to be
    /// referenced when defining the spatial operator (see next section). Since this flux only depends on one
    /// argument, the parameters \code{U}, \code{Uin} and \code{Uout} will have a length of 1 and will contain the
    /// local values of $c$ in the first entry, i.e. \code{U[0]} in the method \code{Flux(...)}
\btab public override IList<string> ArgumentOrdering \{  \newline
\btab \btab get \{ return new string[] \{ "c" \}; \}  \newline
\btab \}  \newline
 \newline
    /// \code{Flux(...)} defines the volume term. The array \code{output} (whose length is determined by the
    /// spatial dimension of the problem) has to contain the evaluated \emph{flux} on exit.
\btab protected override void Flux(double time, double[] x, double[] U, double[] output) \{  \newline
\btab \btab output[0] = u(x) * U[0];  \newline
\btab \btab output[1] = v(x) * U[0];  \newline
\btab \}  \newline
 \newline
    /// \code{InnerEdgeFlux(...)} defines the \emph{numerical flux} between inner edges. The parameters
    /// \code{Uin} and \code{Uout} contain the value from the \emph{in} and \emph{out} side, respectively,
    /// where the normal vector \code{normal} points from \emph{in} to \emph{out}
\btab protected override double InnerEdgeFlux(double time, double[] x, double[] normal, \newline
\btab double[] Uin, double[] Uout, int jEdge) \{  \newline
\btab \btab Vector n              = new Vector(normal);  \newline
\btab \btab Vector velocityVector = new Vector(u(x), v(x));  \newline
 \newline
\btab \btab if (velocityVector * n > 0) \{  \newline
\btab \btab \btab return (velocityVector * Uin[0]) * n;  \newline
\btab \btab \} else \{  \newline
\btab \btab \btab return (velocityVector * Uout[0]) * n;  \newline
\btab \btab \}  \newline
\btab \}  \newline
 \newline
    /// \code{BorderEdgeFlux(...)} defines the \emph{numerical flux} at boundary edges, where only inner values
    /// (\code{Uin}) are given. Here, we reuse \code{InnerEdgeFlux(...)} and the exact solution \code{cExact}
    /// to define a suitable boundary condition.
\btab protected override double BorderEdgeFlux(double time, double[] x, double[] normal, \newline
\btab byte EdgeTag, double[] Uin, int jEdge) \{  \newline
\btab \btab double[] Uout = new double[] \{ cExact(x, time) \};  \newline
\btab \btab return InnerEdgeFlux(time, x, normal, Uin, Uout, jEdge);  \newline
\btab \}  \newline
\} \newline
/// \leftskip=0cm
 }
\BoSSSexe
\BoSSScmd{
/// \subsection{Definition of a spatial operator}
/// The next step is the definition of the spatial operator.
var spatialTerm = new SpatialOperator( \newline
\btab new string[] \{ "c" \},        // Domain variable \newline
\btab new string[] \{ "div" \},      // Co-domain variable\newline
\btab QuadOrderFunc.NonLinear(2)); // Order of integration\newline
/// The name of the \emph{domain variable} must be the same used in \code{ArgumentOrdering}
/// in the definition of the flux, i.e \code{ScalarTransportFlux}.
/// The name of the \emph{co-domain variable} is arbitrary and is used when the
/// fluxes are added. In our case, we only have one type of flux.
/// \code{QuadOrderFunc.NonLinear(int x)} computes the required integration order
/// for a non-linear flux. Here, the flux is given by $\vec{u} c$, where $\vec{u}$ is linear.
/// So, we have second order terms (flux times the ansatz functions), i.e the required
/// order is \code{2*dgDegree+1} \newline
 }
\BoSSSexe
\BoSSScmd{
/// We add the flux of the divergence operator to the corresponding equation component, identified by the
/// \emph{co-domain variable},
spatialTerm.EquationComponents["div"].Add(new ScalarTransportFlux());
 }
\BoSSSexe
\BoSSScmd{
/// and finalize the definition of the operator
spatialTerm.Commit();
 }
\BoSSSexe
\BoSSScmd{
/// \subsection{Time integration}
/// Finally, we have to specify a time stepping scheme to solve the time dependent
/// scalar transport equation \eqref{eq:divergenceTerm}. For simplification, we use
/// the \code{ExplicitEuler} scheme, which just needs the \code{SpatialOperator} and
/// the \code{DGField} as arguments.
using BoSSS.Solution.Timestepping;
 }
\BoSSSexe
\BoSSScmd{
ExplicitEuler timeStepper = new ExplicitEuler(spatialTerm, ch);
 }
\BoSSSexe
\BoSSScmd{
/// We want to perform a full revolution ($t \in [0, 2\pi]$) with 24 timesteps.
 }
\BoSSSexe
\BoSSScmd{
double endTime = 2.0 * Math.PI; \newline
int numberOfTimesteps = 24;
 }
\BoSSSexe
\BoSSScmd{
/// Plot the initial data
tecplot.PlotFields( \newline
\btab "plot\_tutorial4\_0", \newline
\btab 0.0, \newline
\btab ch); \newline
/// Now, we can start the simulation, where the \code{timestepper} performs in each iteration one
/// explicit euler timestep with the timestep size \code{dt}
double dt = endTime / numberOfTimesteps; \newline
for (int i = 1; i <= numberOfTimesteps; i++) \{ \newline
\btab timeStepper.Perform(dt); \newline
\btab tecplot.PlotFields(          // plot each timestep\newline
\btab \btab "plot\_tutorial4\_" + i, \newline
\btab \btab timeStepper.Time, \newline
\btab \btab ch); \newline
\}
 }
\BoSSSexe
\BoSSScmd{
/// Furthermore, we can postprocess our simulation data in various ways: For example,
/// we can compute the L2-Error at the end of the simulation
double error = ch.L2Error(X => cExact(X, timeStepper.Time)); \newline
error;
 }
\BoSSSexe
\BoSSScmd{
/// \section{Advanced topics}
/// So far we used the \code{timestepper} to evaluate the \code{SpatialOperator} in order to get
/// the current change rate for the explicit Euler scheme. In the next section we will evaluate the operator
/// in each iteration manually. But first we set the \code{DGField} back to the initial values and plot
ch.ProjectField(X => cExact(X, 0.0));\newline
tecplot.PlotFields( \newline
\btab "plot\_tutorial4\_advanced\_0", \newline
\btab 0.0, \newline
\btab ch);
 }
\BoSSSexe
\BoSSScmd{
/// \subsection{Evaluation of the spatial operator}
/// To evaluate the \code{SpatialOperator}, we have to provide a mapping of the DG-coordinates of \emph{ch}.
/// This describes a bijective mapping between \emph{local unique indices} and \emph{global unique indices}
var mapping = new CoordinateMapping(ch);
 }
\BoSSSexe
\BoSSScmd{
/// In other words, it maps the DG-coordinates into one long, one-dimensional \code{CoordinateVector}
var DGCoordinates = new CoordinateVector(mapping);
 }
\BoSSSexe
\BoSSScmd{
/// Now, we can create an evaluator of the \code{SpatialOperator} for the \code{DGField} \emph{ch}
var evaluator = spatialTerm.GetEvaluator(mapping.Fields, mapping);
 }
\BoSSSexe
\BoSSScmd{
/// In our case this evaluator computes the fluxes of the divergence operator evaluated for the \code{DGField} \emph{ch}
double[] flux = new double[ch.CoordinateVector.Count]; \newline
evaluator.time = 0.0;\newline
evaluator.Evaluate(1.0, 0.0, flux);\newline
/// After the evaluation the output \code{flux} is $\code{flux} = 0.0 \cdot \code{flux} + 1.0 \cdot \code{spatialTerm(time: 0.0)}$
 }
\BoSSSexe
\BoSSScmd{
/// Finally, we can use this kind of \code{spatial operator} evaluation to implement the explicit Euler scheme manually.
double physTime = 0.0;\newline
for (int i = 1; i <= numberOfTimesteps; i++) \{ \newline
\btab evaluator.time = physTime;\newline
\btab evaluator.Evaluate(1.0, 0.0, flux);\newline
\btab DGCoordinates.axpy<double[]>(flux, -dt);\newline
\btab physTime += dt;\newline
\btab tecplot.PlotFields(        \newline
\btab \btab "plot\_tutorial4\_advanced" + i, \newline
\btab \btab physTime, \newline
\btab \btab ch); \newline
\}
 }
\BoSSSexe
\BoSSScmd{
double error = ch.L2Error(X => cExact(X, physTime)); \newline
error;
 }
\BoSSSexe


\documentclass{article}

\usepackage[preprint]{./template/neurips_2020}

\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc}    % use 8-bit T1 fonts
\usepackage{hyperref}       % hyperlinks
\usepackage{url}            % simple URL typesetting
\usepackage{booktabs}       % professional-quality tables
\usepackage{amsfonts}       % blackboard math symbols
\usepackage{nicefrac}       % compact symbols for 1/2, etc.
\usepackage{microtype}      % microtypography
\usepackage{physics}
\usepackage{amsmath}

\title{Chapter 2: Introduction to Quantum Mechanics}

\author{
  John Martinez \\
  \texttt{john.r.martinez14@gmail.com} \\
  % examples of more authors
  % \And
  % Coauthor \\
  % Affiliation \\
  % Address \\
  % \texttt{email} \\
  % \AND
  % Coauthor \\
  % Affiliation \\
  % Address \\
  % \texttt{email} \\
  % \And
  % Coauthor \\
  % Affiliation \\
  % Address \\
  % \texttt{email} \\
  % \And
  % Coauthor \\
  % Affiliation \\
  % Address \\
  % \texttt{email} \\
}

\begin{document}
\maketitle

%\begin{abstract}
%  The abstract paragraph should be indented \nicefrac{1}{2}~inch (3~picas) on
%  both the left- and right-hand margins. Use 10~point type, with a vertical
%  spacing (leading) of 11~points.  The word \textbf{Abstract} must be centered,
%  bold, and in point size 12. Two line spaces precede the abstract. The abstract
%  must be limited to one paragraph.
%\end{abstract}
\section{Intro to Quantum Mechanics}
\subsection{Linear Algebra}
Vector space
\begin{center}
  $\begin{bmatrix}
    z_{1} \\
    \vdots \\
    z_{n}
  \end{bmatrix}$
\end{center}

%%% Section 1.1.1
\subsubsection{Bases and Linear Independence}
A \emph{spanning set} for a vector space is a set of vectors
$\ket{v_{1}},...,\ket{v_{n}}$ such that any vector $\ket{v}$ in the vector
space can be written as a linear combination
$\ket{v} = \displaystyle\sum_{i}a_{i}\ket{v_{i}}$ in that set. The spanning
set for the vector space $\mathbb{C}^{2}$ is the set
  \begin{center}
    $\ket{v_{1}} \equiv \begin{bmatrix} 1 \\ 0 \end{bmatrix};\mspace{10mu}
    \ket{v_{2}} \equiv \begin{bmatrix} 0 \\ 1 \end{bmatrix}$
  \end{center}

A second spanning set in $\mathbb{C}^{2}$ is
  \begin{center}
    $
    \ket{v_{1}} \equiv \frac{1}{\sqrt{2}}
      \begin{bmatrix} 1 \\ 1
      \end{bmatrix};\mspace{10mu}
    \ket{v_{2}} \equiv \frac{1}{\sqrt{2}}
      \begin{bmatrix} 1 \\ -1
      \end{bmatrix}$
  \end{center}

A set set of non-zero vectors $\ket{v_{1}},...,\ket{v_{n}}$ are
\emph{linear dependent} if there exists a set of complex numbers
$a_{1},...,a_{n}$ with $a_{i} \neq 0$ for at least one value of $i$, such that
\begin{center}
  $a_{1}\ket{v_{1}} + a_{2}\ket{v_{2}} + \hdots + a_{n}\ket{v_{n}} = 0$.
\end{center}

%%% Section 1.1.2
\subsubsection{Linear Operators and Matrices}
A \emph{linear operator} between vector spaces $V$ and $W$ is defined to be any
function $A : V \rightarrow W$ which is linear in its inputs
  \begin{center}
    $A\left(\displaystyle\sum_{i}a_{i}\ket{v_{i}}\right) =
    \displaystyle\sum_{i}a_{i}A(\ket{v_{i}})$
  \end{center}

%%% Section 1.1.3
\subsubsection{Pauli Matrices}
  \begin{center}
    $
    \sigma_{0} \equiv I \equiv
      \begin{bmatrix} 1 & 0 \\
      0 & 1 \end{bmatrix};\mspace{10mu}
    \sigma_{1} \equiv \sigma_{x} \equiv X \equiv
      \begin{bmatrix} 0 & 1 \\
      1 & 0 \end{bmatrix}
    $
  \end{center}

  \begin{center}
    $
    \sigma_{2} \equiv \sigma_{y} \equiv Y \equiv
      \begin{bmatrix} 0 & -i \\
      i & 0 \end{bmatrix};\mspace{10mu}
    \sigma_{3} \equiv \sigma_{z} \equiv Z \equiv
      \begin{bmatrix} 1 & 0 \\
      0 & -1 \end{bmatrix}
    $
  \end{center}

%%% Section 1.1.4
\subsubsection{Inner Products}
An \emph{inner product} is a function which takes as input two vectors
$\ket{v}$ and $\ket{w}$ form a vector space and produces a complex number
as output.

A function from $V \times V \rightarrow \mathbb{C}$ is an inner product if it
satisfies the requiments that:

(1) (., .) is linear in the second argument
  \begin{center}
    $
      \left(\ket{v}, \displaystyle\sum_{i}\lambda_{i}\ket{w_{i}}\right) =
      \displaystyle\sum_{i}\lambda_{i}(\ket{v}, \ket{w_{i}})
    $
  \end{center}

(2) $(\ket{v}, \ket{w}) = (\ket{w}, \ket{v})^{*}$

(3) $(\ket{v}, \ket{v}) \geq 0$ with equality if and only if $\ket{v} = 0$

For example, $\mathbb{C}^{n}$ has an inner product defined by
  \begin{center}
    $
      ((y_{1},...,y_{n}),(z_{1},...,z_{n})) \equiv
      \displaystyle\sum_{i}y_{i}^{*}z_{i} =
      \begin{bmatrix}
      y_{1}^{*} \hdots y_{n}^{*}
      \end{bmatrix}
      \begin{bmatrix}
      z_{1} \\
      \vdots \\
      z_{n}
      \end{bmatrix}
    $
  \end{center}

Vectors $\ket{v}$ and $\ket{v}$ are \emph{orthogonal} if their inner product
is zero. We define the \emph{norm} of a vector $\ket{v}$ by
  \begin{center}
    $\|\ket{v}\| \equiv \sqrt{\bra{v}\ket{v}}$
  \end{center}

A \emph{unit vector} is a vector $\ket{v}$ such that $\|\ket{v}\| = 1$.

Suppose $\ket{w_{1}},\hdots,\ket{w_{d}}$ is a basis set for some vector space
$V$ with an inner product. There is a useful method, the \emph{Gram-Schmidt}
procedure, which can be used to produce an orthonormal basis set
$\ket{v_{1}},\hdots,\ket{v_{d}}$ for a vector space $V$. Define
$\ket{v_{1}} \equiv \frac{\ket{w_{1}}}{\|\ket{w_{1}}\|}$ and for
$1 \leq k \leq d-1$ define $\ket{v_{k+1}}$ inductively by
  \begin{center}
  $
    \ket{v_{k+1}} \equiv
    \frac{\ket{w_{k+1}} - \sum_{i=1}^{k}\bra{v_{i}}\ket{w_{k+1}}\ket{v_{i}}}{
    \|\ket{w_{k+1}} - \sum_{i=1}^{k}\bra{v_{i}}\ket{w_{k+1}}\ket{v_{i}}\|}
  $
  \end{center}

%%% Section 1.1.5
\subsubsection{Eigenvectors and Eigenvalues}
An \emph{eigenvector} of a linear operator $A$ on a vector space is a non-zero
vector $\ket{v}$ such that $A\ket{v} = v\ket{v}$, where $v \in \mathbb{C}$ and
is known as the \emph{eigenvalue} of $A$ corresponding to $\ket{v}$. The
\emph{characteristic function} is defined as
$c(\lambda) \equiv det |A - \lambda I|$ where det is the \emph{determinant}
function of matrices. The solutions to the characteristic equation,
$c(\lambda) = 0$, are the eigenvalues of the operator $A$.

A \emph{diagonal representation} for an operator $A$ on a vector space $V$ is
a representation
  \begin{center}
    $A = \displaystyle\sum_{i}\lambda_{i}\ket{i}\bra{i}$
  \end{center}
where the vectors $\ket{i}$ form an orthonormal set of eigenvectors for $A$.
A diagonal representation of Pauli $Z$ matrix:
  \begin{center}
    $Z = \begin{bmatrix} 1 & 0 \\ 0 & -1\end{bmatrix} =
    \ket{0}\bra{0} - \ket{1}\bra{1}$
  \end{center}

%%% Section 1.1.6
\subsubsection{Adjoints and Hermitian Operators}
Suppose $A$ is any linear operator on a Hilbert space, $V$, then
$\exists A^{\dagger}$ on $V$ such that $\forall \ket{v}, \ket{w} \in V$,
  \begin{center}
    $(\ket{v}, \ket{w}) = (A^{\dagger}\ket{v}, \ket{w})$.
  \end{center}

This linear operator is know ad the \emph{adjoint} or \emph{Hermitian conjugate}
of the operator $A$.

Note: $(AB)^{\dagger} = B^{\dagger}A^{\dagger}$

($\dagger$) is called "dagger" and is equal to
$A^{\dagger} \equiv (A^{*})^{T}$, which is the transpose of the complex
conjugate. Example:
  \begin{center}
    $
    \begin{bmatrix}
      1 + 3i & 2i \\
      1 + i & 1 - 4i
    \end{bmatrix}^{\dagger} =
    \begin{bmatrix}
      1 - 3i & 1 - i \\
      -2i & 1 + 4i
    \end{bmatrix}
    $
  \end{center}

%%% Section 1.1.7
\subsubsection{Tensor Products}
The \emph{tensor product} if a way of putting vector spaces together to form
larger vector spaces. Suppose $V$ and $W$ are vector spaces of dimensions $m$
and $n$ respectively, then $V \otimes W$ is an $nm$ dimensional vector space.
The elements of $V \otimes W$ are linear combinations of tensor products
$\ket{v}\otimes\ket{w}$ of elements $\ket{v}$ of $V$ and $\ket{w}$ of $W$. A
more intuitive way to represent this is through the \emph{Kronecker product}.
Suppose A is an $m$ by $n$ matrix and B is a $p$ by $q$ matrix then:
  \begin{center}
  $ A \otimes B =
  \begin{bmatrix}
    A_{11}B & A_{12}B & \cdots & A_{1n}B \\
    A_{21}B & A_{22}B & \cdots & A_{2n}B \\
    \vdots  & \vdots  & \ddots & \vdots  \\
    A_{m1}B & A_{m2}B & \cdots & A_{mn}B
  \end{bmatrix}$
  \end{center}
produces an $nq \cross mp$ matrix.

Example:
  \begin{center}
    $
    \begin{bmatrix}
      1 \\ 2
    \end{bmatrix} \otimes
    \begin{bmatrix}
      2 \\ 3
    \end{bmatrix} =
    \begin{bmatrix}
      2 \\ 3 \\ 4 \\ 6
    \end{bmatrix}
    $.
  \end{center}

%%% Section 1.1.8
\subsubsection{Operator Functions}
Let $A = \sum_{a}a\ket{a}\bra{a}$ be a spectral decomposition for a normal
operator $A$. Define $f(A) \equiv \sum_{a}f(a)\ket{a}\bra{a}$.

Example:
  \begin{center}
  $e^{\theta Z} =
    \begin{bmatrix}
      e^{\theta} & 0 \\
      0 & e^{-\theta}
    \end{bmatrix}$
  \end{center}

Since $Z$ has eigenvectors $\ket{0}$ and $\ket{1}$.

The \emph{trace} of $A$ is defined to be the sum of its diagonal elements,
  \begin{center}
    $tr(A) \equiv \displaystyle\sum_{i} A_{ii}$.
  \end{center}
Some important properities:

The trace is \emph{cyclic}:
  \begin{center}$tr(AB) = tr(BA)$\end{center}

The trace is \emph{linear}:
  \begin{center}$tr(A + B) = tr(A) + tr(B)$ and $tr(zA) = z tr(A)$\end{center}

The trace is invariant under the \emph{unitary similarity transform}:
  \begin{center}$A \rightarrow UAU^{\dagger}$\end{center}
as
  \begin{center}$tr(UAU^{\dagger}) = tr(U^{\dagger}UA) = tr(A)$\end{center}

Suppose $\ket{\psi}$ is a unit vector and $A$ is an arbitrary operator. To
evaluate $tr(A\ket{\psi}\bra{\psi})$ use the Gram-Schmidt procedure to extend
$\ket{\psi}$ to an orthonormal basis $\ket{i}$ which includes $\ket{\psi}$ as
the first element. Then we have

  \begin{center}
    $
      tr(A\ket{\psi}\bra{\psi}) =
      \displaystyle\sum_{i}\bra{i}A\ket{\psi}\bra{\psi}\ket{i}
    $
  \end{center}
  \begin{center}
    $
      = \bra{\psi}A\ket{\psi}
    $
  \end{center}

%%% Section 1.1.9
\subsubsection{The Commutator and Anti-Commutator}
The \emph{commutator} between two operators $A$ and $B$ is defined to be
  \begin{center}
    $[A, B] \equiv AB - BA$.
  \end{center}

If $[A, B] = 0$, that is, $AB = BA$, then we say that $A$ \emph{commutes} with
$B$.  The \emph{anti-commutator} is defined as:
  \begin{center}
    $\{A, B\} \equiv AB + BA$
  \end{center}
And we say $A$ \emph{anti-commutes} with $B$ if $\{A, B\} = 0$.

%%% Section 1.1.10
\subsubsection{Polar and Singular Value Decomposition}
Let $A$ be a linear operator on a vector space $V$. $\exists$ a unitary $U$
and positive operators $J$ and $K$ such that $A = UJ = KU$, defined by
$J \equiv \sqrt{A^{\dagger}A}$ and $K \equiv \sqrt{AA^{\dagger}}$.

Let $A$ be a square matrix. Then $\exists$ unitary matrics $U$ and $V$, and a
diagonal matrix $D$ with non-negative entries such that $A = UDV$. The
diagonal elements $D$ are called the \emph{singular values} of $A$.

\subsection{Postulates of Quantum Mechanics}
\subsubsection{State Space}
\textbf{Postulate 1}: Assocaited to any isolated physical system is a complex
vector space with inner product (that is, Hilbert space) know as the
\emph{state space} of the system. The system is completely described by its
\emph{state vector}, which is a unit vector in the system's state space.

The simplest quantum mechanical system is the \emph{qubit}. An arbitrary
vector in the state space can be written $\ket{\psi} = a\ket{0} + b\ket{1}$
where $a, b \in \mathbb{C}$. The conditional that $\ket{\psi}$ be a unit
vector, $\bra{\psi}\ket{\psi} = 1$, is equivalent to
$|a|^{2} + |b|^{2} = 1$

\subsubsection{Evolution}
How does the state, $\ket{\psi}$, of a quantum mechanical system change with
time?

\textbf{Postulate 2.1}: The evolution of a \emph{closed} quantum system is
described by a \emph{unitary transformation}. That is, the state
$\ket{\psi}$ of the system at time $t_{1}$ is related to the state
$\ket{\psi'}$ of the system at time $t_{2}$ by a unitary operator
$U$ which depends only on the times $t_{1}$ and $t_{2}$:
  \begin{center}
    $\ket{\psi'} = U\ket{\psi}$.
  \end{center}

\textbf{Postulate 2.2}: The time evolution of the state of a closed quantum
system is described by the \emph{Schrödinger equation}.

  \begin{center}
    $i\hbar\frac{\partial \ket{\psi}}{\partial t} = H\ket{\psi}$
  \end{center}

\subsubsection{Quantum Measurement}
\textbf{Postulate 3}: Quantum measurements are describled by a collection
${M_{m}}$ of \emph{measurement operators}.

If the state of the quantum system is $\ket{\psi}$ immediately before the
measurement then the probability that the result $m$ occurs is give by
  \begin{center}
    $p(m) = \bra{\psi}M_{m}^{\dagger}M_{m}\ket{\psi}$
  \end{center}
and the state after the measurement is
  \begin{center}
    $\frac{M_{m}\ket{\psi}}{\sqrt{\bra{\psi}M_{m}^{\dagger}M_{m}\ket{\psi}}}$
  \end{center}

\subsubsection{Distinguishing Quantum States}
\end{document}


% \documentclass[draft,11pt]{article}
%\documentclass[11pt]{article}

\chapter{Introduction to Spectral Graph Theory}

%\allowdisplaybreaks

%%% for this lecture
%\newcommand\gap{\text{gap}}
%\newcommand{\Zhao}[1]{{\color{red} Zhao: #1}}

%\begin{document}
\sloppy
%\lecture{4 --- Wednesday, March 11th}
%{Spring 2020}{Rasmus Kyng}{Introduction to Spectral Graph Theory}

In this chapter, we will study graphs through linear algebra. This
approach is known as Spectral Graph Theory and turns out to be
surprisingly powerful.
An in-depth treatment of many topics in this area can be found in \cite{S19}.

\section{Recap: Incidence and Adjacency Matrices, the Laplacian
  Matrix and Electrical Energy}

In Chapter \ref{sec:intro}, we looked at undirected graphs and we introduce the
incidence matrix and the Laplacian of the graph. Let us recall these.

We consider an undirected weighted graph $G=(V, E, \ww)$, with
$n=\sizeof{V}$ vertices and $m = \sizeof{E}$ edges, where $\ww \in
\R_+^E$ assigns positive weight for every edge.
Let's assume $G$ is connected.

To introduce the \emph{edge-vertex incidence matrix} of the graph,
we first have to associate an arbitrary direction to every edge.
We then let
$\BB \in \R^{V \times E}$.
\[
  \BB(v,e) =
  \begin{cases}
    1 & \text{ if } e = (u,v) \\
    -1 &\text{ if } e = (v,u) \\
    0 &\text{ o.w.}
  \end{cases}
\]
The edge directions are only there to help us track the meaning of signs of
quantities defined on edges: The math we do should not depend on the
choice of sign.

Let $\WW \in \R^{E \times E}$ be the diagonal matrix given by $\WW =
\diag(\ww)$, i.e $\WW(e,e) = \ww(e)$.
We define the Laplacian of the graph as $\LL = \BB\WW\BB^{\trp}$.
Note that in the first lecture, we define the Laplacian as
$\BB\RR^{-1}\BB^{\trp}$,
where $\RR$ is the diagonal matrix with edge resistances on the
diagonal.
We want to think of high \emph{weight} on an edge as expressing that two
vertices are highly connected, whereas we think of high resistance on
an edge as expressing that the two vertices are poorly connected, so
we let $\ww(e) = 1/\RR(e,e)$.

The weighted adjacency matrix $\AA \in \R^{V \times V}$ of a graph is
given by
\[
  \AA(u,v) =
  \begin{cases}
    \ww(u,v) & \text{ if } \setof{u,v} \in E \\
    0  & \text{ otherwise. }
  \end{cases}
\]
Note that we treat the edges as undirected here, so $\AA^{\trp} = \AA$.
The weighted degree of a vertex is defined as $\dd(v) = \sum_{\setof{u,v}
  \in E} w(u,v)$. Again we treat the edges as undirected.
Let $\DD = \diag(\dd)$ be the diagonal matrix in $\R^{V \times V}$
with weighted degrees on the diagonal.

In Problem Set 1, you showed that $\LL = \DD - \AA$, and that for $\xx
\in \R^V$,
\[
\xx^{\trp} \LL \xx = \sum_{\setof{a,b} \in E} \ww(a,b) (\xx(a) - \xx(b))^2.
  \]

Now we can express the net flow constraint that $\ff$ routes $\dd$ by
\[
  \BB \ff = \dd
  .
\]
This is also called a conservation constraint. In our examples so far,
we have $\dd(s) = -1$, $\dd(t) = 1$ and $\dd(u) = 0$ for all $u\in V\setminus\setof{s,t}$.

If we let $\RR = \diag_{e \in E} \rr(e)$
then Ohm's law tells us that electrical voltages $\xx$ will induce an
electrical flow $\ff = \RR^{-1}\BB^{\trp} \xx$.
We defined the electrical energy of a flow $\ff \in \R^E$ to be
\[
  \energy(\ff) = \sum_e \rr(e) \ff(e)^2 = \ff^{\trp} \RR \ff.
\]
And, from Ohm's Law, we can then see that
\[
 \energy(\ff) = \ff^{\trp} \RR \ff = \xx^{\trp} \LL \xx.
\]
Hence, define the electrical energy associated with a set of voltages
to be
\[
 \energy(\xx) = \xx^{\trp} \LL \xx.
\]
\paragraph{The Courant-Fisher Theorem.} Let us also recall the
Courant-Fischer theorem, which we proved in Chapter 3 (Theorem \ref{thm:courant-fischer}).
\begin{theorem}[The Courant-Fischer Theorem]
  \label{thm:courant-fischer}
  Let $\AA$ be a symmetric matrix in $\R^{n\times n}$, with
  eigenvalues $\lambda_1\leq \lambda_2 \leq \ldots \leq \lambda_n$.
  Then
  \begin{enumerate}
  \item
  \label{thm:courant-fischer:minmax}
    \[
    \lambda_i = \min_{
      \substack{
        \mathrm{subspace~} W \subseteq \R^n
        \\
        \dim{W} = i
      }
    }
    \max_{
      \xx \in W, \xx \neq \veczero
    }
    \frac{\xx^\trp \AA\xx}{\xx^\trp\xx}
  \]
\item
  \label{thm:courant-fischer:maxmin}
    \[
    \lambda_i
    =
    \max_{
      \substack{
        \mathrm{subspace~} W \subseteq \R^n
        \\
        \dim{W} = n+1-i
      }
    }
    \min_{
      \xx \in W, \xx \neq \veczero
    }
    \frac{\xx^\trp \AA\xx}{\xx^\trp\xx}
    \]
  \end{enumerate}
\end{theorem}

In fact, from our proof of the Courant-Fischer theorem in Chapter 3,
we can also extract a slightly different statement:

\begin{theorem}[The Courant-Fischer Theorem, eigenbasis version]
  \label{thm:courant-fischer-eigvec}
  Let $\AA$ be a symmetric matrix in $\R^{n\times n}$, with
  eigenvalues $\lambda_1\leq \lambda_2 \leq \ldots \leq \lambda_n$,
  and corresponding eigenvectors $\xx_1, \xx_2, \ldots, \xx_n$ which
  form an othernormal basis.
  Then
  \begin{enumerate}
  \item
  \label{thm:courant-fischer-eigvec:minmax}
    \[
    \lambda_i =
    \min_{
      \substack{ \xx \perp \xx_1, \ldots \xx_{i-1}
        \\ \xx \neq \veczero}
    }
    \frac{\xx^\trp \AA\xx}{\xx^\trp\xx}
  \]
\item
  \label{thm:courant-fischer2:maxmin}
    \[
    \lambda_i =
    \max_{
      \substack{  \xx \perp \xx_{i+1}, \ldots \xx_{n}
        \\ \xx \neq \veczero}
    }
    \frac{\xx^\trp \AA\xx}{\xx^\trp\xx}
    \]
  \end{enumerate}
\end{theorem}

Of course, we also have $\lambda_i(\AA) = \frac{\xx_i ^\trp \AA \xx_i
}{\xx_i^\trp\xx_i}$.
\section{Understanding Eigenvalues of the Laplacian}
We would like to understand the eigenvalues of the Laplacian matrix of
a graph.

But first, why should we care? It turns out that Laplacian eigenvalues can help
us understand many properties of a graph.
But we are going to start off with simple motivating observation: Electrical
voltages $\xx \in \R^V$ consume electrical energy $\energy(\xx) =
\xx^{\trp} \LL \xx$.
This means that by the Courant-Fischer Theorem
\[
  \energy(\xx) =
  \xx^{\trp} \LL \xx \leq \lambda_n(L)  \xx^{\trp} \xx
\]
And, for any voltages $\xx \perp \vecone$,
\[
  \energy(\xx) =
  \xx^{\trp} \LL \xx \geq \lambda_2(L)  \xx^{\trp} \xx.
\]
Thus, we can use the eigenvalues to give upper and lower bounds on how
much electrical energy will be consumed by the flow induced by $\xx$,
in terms compared to $\xx^{\trp} \xx = \norm{\xx}_2^2$.

In a couple of chapters, we will also prove the following claim, which
shows that the Laplacian eigenvalues can directly tell us about the
electrical energy that is required to route a given demand.

\begin{claim}
  Given a demand vector
  $\dd \in \R^V$ such that $\dd \perp \vecone$,
  the electrical voltages $\xx$ that route $\dd$ satisfy $\LL \xx =
  \dd$ and the electrical energy of these voltages satifies
  \[
\frac{\norm{\dd}_2^2}{\lambda_n} \leq \energy(\xx) \leq \frac{\norm{\dd}_2^2}{\lambda_2}
    \]
\end{claim}

\paragraph{Eigenvalues of the Laplacian of a Complete Graph.}
To get a sense of how Laplacian eigenvalues behave, let us start by considering the $n$ vertex complete graph with unit weights,
which we denote by $K_n$.
The adjacency matrix of $K_n$ is $\AA = \vecone \vecone^\trp - \II$,
since it has ones everywhere, except for the diagonal, where entries
are zero.
The degree matrix $\DD = (n-1) \II$.
Thus the Laplacian is $\LL =  \DD - \AA = n\II - \vecone
\vecone^\trp$.

Thus for any $\yy \perp \vecone$, we have
$\yy^{\trp} \LL \yy = n \yy^{\trp} \yy - (\vecone^{\trp} \yy)^2 = n \yy^{\trp} \yy$.

From this, we can conclude that any $\yy \perp \vecone$ is an
eigenvector of eigenvalue $n$, and that all $\lambda_2 = \lambda_3 =
\ldots = \lambda_n = n$.

Next, let us try to understand $\lambda_2$ and $\lambda_n$ for
$P_n$, the $n$ vertex path graph with unit weight edges.
I.e. the graph has edges $E = \setof{ \setof{i,i+1}  \text{ for } i = 1 \text{ to
  } (n-1) }$.

This is in a sense the least well-connected unit weight graph on $n$
vertices, whereas $K_n$ is the most well-connected.

\subsection{Test Vector Bounds on $\lambda_2$ and $\lambda_n$}
We can use the eigenbasis version of the Courant-Fisher theorem to observe
that the second-smallest eigenvalue of the Laplacian is given by
\begin{align}
  \label{eq:lambda2bymin}
\lambda_2(\LL) = \min_{ \substack{ \xx \neq \veczero \\ \xx^\top \vecone = 0} } \frac{ \xx^\top \LL \xx}{ \xx^\top \xx}.
\end{align}

We can get a better understanding of this particular case through a couple of simple
observations. Suppose $\xx = \yy + \alpha \vecone$, where $\yy \perp
\vecone$.
Then $\xx^{\trp} \LL \xx = \yy^{\trp} \LL \yy$, and $\norm{\xx}_2^2 =
\norm{\yy}^2 + \alpha^2 \norm{\vecone}^2$.
So for any given vector, you can increase the value of $\frac{
  \xx^\top \LL \xx}{ \xx^\top \xx}$, by instead replacing $\xx$ with
the component orthogonal to $\xx$, which we denoted by $\yy$.

We can conclude from Equation~\eqref{eq:lambda2bymin} that for \emph{any} vector $\yy \perp \vecone$,
\begin{align*}
\lambda_2 \leq \frac{ \yy^\trp \LL \yy}{ \yy^\trp \yy}
\end{align*}
When we use a vector $\yy$ in this way to prove a bound on an eigenvalue, we call it a \emph{test vector}.

Now, we'll use a test vector to give an upper bound on $\lambda_2(\LL_{P_n})$.
Let $\xx \in \R^V$ be given by $\xx ( i ) = ( n + 1 ) - 2 i$, for $i
\in [n]$. This vector satisfies $\xx \bot \vecone$.
We picked this because we wanted a sequence of values growing linearly along
the path, while also making sure that the vector is orthogonal to
$\vecone$.
Now
\begin{align*}
\lambda_2(\LL_{P_n})
\leq & ~ \frac{ \sum_{i \in [n-1]} ( \xx(i) - \xx(i+1) )^2  }{ \sum_{i=1}^n \xx(i)^2 } \\
= & ~ \frac{ \sum_{i=1}^{n-1} 2^2 }{ \sum_{i=1}^n ( n + 1 - 2 i )^2 } \\
= & ~ \frac{ 4 ( n - 1 ) }{ ( n + 1 ) n ( n - 1 ) / 3 } \\
= & ~ \frac{ 12 } { n ( n + 1 ) } \leq \frac{ 12 } { n^2 }.
\end{align*}

Later, we will prove a lower bound that shows this value is right up
to a constant factor.
But the test vector approach based on the Courant-Fischer theorem
doesn't immediately work
when we want to prove lower bounds on $\lambda_2(\LL)$.

We can see from either version of the Courant-Fischer theorem that
\begin{align}
  \label{eq:lambda2bymin}
\lambda_n(\LL) = \max_{ \substack{ \vv \neq \veczero} } \frac{ \vv^\top \LL \vv}{ \vv^\top \vv}.
\end{align}
Thus for \emph{any} vector $\yy \neq 0$,
\begin{align*}
\lambda_n \geq \frac{ \yy^\trp \LL \yy}{ \yy^\trp \yy}.
\end{align*}
This means get a test vector-based lower bound on $\lambda_n$.
Let us apply this to the Laplacian of $P_n$.
We'll try the vector $\xx \in \R^V$ be given by $\xx ( 1 ) = -1$,
and $\xx(n) = 1$ and $\xx(i) = 0$ for $i \neq 0,1$.

Here we get
\begin{align*}
\lambda_n(\LL_{P_n}) \geq \frac{ \yy^\trp \LL \yy}{ \yy^\trp \yy} = \frac{2}{2} = 1.
\end{align*}

Again, it's not clear how to use the Courant-Fischer theorem to prove
an upper bound on $\lambda_n(\LL) $.
But, later we'll see how to prove an upper that shows that
for $P_n$, the lower bound we obtained is right up to constant factors.

% The Courant-Fischer theorem is not as helpful when we want to prove lower bounds on $\lambda_2$. To prove lower bounds, we need the form with a maximum on the outside, which gives
% \begin{align*}
% \lambda_2 \geq \max_{S : \dim{S} = n - 1 } \min_{ \vv\in S } \frac{ \vv^\top \LL \vv}{ \vv^\top \vv}
% \end{align*}
% This is not too helpful, as it is difficult to prove lower bounds on
% \begin{align*}
% \min_{ \vv\in S } \frac{ \vv^\top \LL \vv}{ \vv^\top \vv}
% \end{align*}
% over a space $S$ of large dimension. We need another technique.

% \subsection{Graphic Inequalities}
% I begin by recalling an extremely useful piece of notation that is used in the Optimization community. For a symmetric matrix ${\bf A}$, we write
% \begin{align*}
% {\bf A} \succeq 0
% \end{align*}
% if ${\bf A}$ is positive semidefinite. That is, if all of the eigenvalues of ${\bf A}$ are nonnegative, which is equivalent to
% \begin{align*}
% \vv^\top {\bf A} \vv\geq 0,
% \end{align*}
% for all $\vv$. We similarly write
% \begin{align*}
% {\bf A} \succeq {\bf B}
% \end{align*}
% if
% \begin{align*}
% {\bf A} - {\bf B} \succeq 0
% \end{align*}
% which is equivalent to
% \begin{align*}
% \vv^\top {\bf A} \vv\geq \vv^\top {\bf B} \vv
% \end{align*}
% for all $\vv$.

% The relation $\preceq$ is an example of a partial order. It applies to some pairs of symmetric matrices, while others are incomparable. But, for all pairs to which it does apply, it acts like an order. For example, we have
% \begin{align*}
% {\bf A} \succeq {\bf B}, \mathrm{~and~} {\bf B} \succeq {\bf C} \mathrm{~implies~} {\bf A} \succeq {\bf C},
% \end{align*}
% and
% \begin{align*}
% {\bf A} \succeq {\bf B} \mathrm{~implies~} {\bf A} + {\bf C} \succeq {\bf B} + {\bf C},
% \end{align*}
% for symmetric matrices ${\cal A}$, ${\cal B}$ and ${\cal C}$.

% I find it convenient to overload this notation by defining it for graphs as well. Thus, I'll write
% \begin{align*}
% G \succeq H
% \end{align*}
% if $\LL_{G} \succeq \LL_H$.

% For example, if $G = (V,E)$ is a graph and $H = (V,F)$ is a subgraph of $G$, then
% \begin{align*}
% \LL_G \succeq \LL_H.
% \end{align*}

% To see this, recall the Laplacian quadratic form:
% \begin{align*}
% \xx^\top \LL_G \xx = \sum_{ (u,v) \in E } w_{u,v} ( \xx(u) - \xx(v) )^2.
% \end{align*}
% It is clear that dropping edges can only decrease the value of the quadratic form. The same holds for decreasing the weights of edges.

% This notation is most powerful when we consider some multiple of a graph. Thus, I could write
% \begin{align*}
% G \succeq c \cdot H, \mathrm{~for~some~} c > 0.
% \end{align*}
% What is $c \cdot H$? It is the same graph as $H$, but the weight of every edge is multiplied by $c$.

% Using the Courant-Fischer Theorem, we can prove
% \begin{lemma}
% If $G$ and $H$ are graphs such that
% \begin{align*}
%  G \succeq c \cdot H,
% \end{align*}
% then
% \begin{align*}
% \lambda_k (G) \geq c \cdot \lambda_k(H), \mathrm{~for~all~} k.
% \end{align*}
% \end{lemma}
% \begin{proof}
% The Courant-Fischer Theorem tells us that
% \begin{align*}
% \lambda_k (G)
% = & ~ \min_{ S \subseteq \R^n, \dim{S} = k } \max_{ \xx \in S }
%  \frac{ \xx^\top \LL_G \xx }{ \xx^\top \xx } \\
% \geq & ~ c \dot \min_{ S \subseteq \R^n, \dim{S} = k } \max_{ \xx \in S } \frac{ \xx^\top L_H \xx }{ \xx^\top \xx } \\
% = & ~ c \cdot \lambda_k (H).
% \end{align*}
% \end{proof}

% \begin{corollary}
% Let $G$ be a graph and let $H$ be obtained by either adding an edge to $G$ or increasing the weight of an edge in $G$. Then, for all $i$,
% \begin{align*}
% \lambda_i (G) \leq \lambda_i (H).
% \end{align*}
% \end{corollary}

% \subsection{Approximations of Graphs}
% An idea that we will use in later lectures is that one graph approximations another if their Laplacian quadratic forms are similar. For example, we will say that $H$ is a $c$-approximation of $G$ if
% \begin{align*}
% c \cdot H \succeq G \succeq H /c.
% \end{align*}
% Surprising approximations exist.

% For example, expander graphs are very sparse approximations of the complete graph. For example, the following is known
% \begin{theorem}
% For every $\epsilon > 0$, there exists a $d > 0$ such that for all sufficiently large $n$ there is a $d$-regular graph $G_n$ that is $(1+\epsilon)$-approximation of $K_n$.
% \end{theorem}

% These graphs have many fewer edges than the complete graphs!
% In a latter lecture we will also prove that every graph can be well-approximated by a sparse graph.

% \subsection{The path inequality}

% By now you should be wondering, ``how do we prove that $G \succeq c \cdot H$ for some graph $G$ and $H$?'' Not too many ways are known. We'll do it by proving some inequalities of this form for some of the simplest graphs, and then extending them to more general graphs.

% For example, we will
% \begin{align*}
% (n-1) \cdot P_n \succeq G_{1,n},
% \end{align*}
% where $P_n$ is the path from vertex $1$ to vertex $n$, and $G_{1,n}$ is the graph with just the edge $(1,n)$. All of these edges are unweighted.

% The following very simple proof of this inequality was discovered by Sam Daitch.
% \begin{lemma}
% \begin{align*}
% (n-1) \cdot P_n \succeq G_{1,n}.
% \end{align*}
% \end{lemma}
% \begin{proof}

% We need to show that for every $x \in \in \R^n$,
% \begin{align*}
% (n-1) \cdot \sum_{i=1}^{n-1} ( x(i+1) - x(i) )^2 \geq ( x(n) - x(1) )^2.
% \end{align*}
% For $i \in [n-1]$, set
% \begin{align*}
% \Delta (i) = x(i+1) - x(i).
% \end{align*}
% The inequality we need to prove then becomes
% \begin{align*}
% (n-1) \sum_{i=1}^{n-1} ( \Delta(i) )^2 \geq \left( \sum_{i=1}^{n-1} \Delta (i)  \right)^2.
% \end{align*}
% But, this is just the Cauchy-Schwartz inequality. I'll remind you that Cauchy-Schwartz just follows from the fact that the inner product of two vectors is at most the product of their norms:
% \begin{align*}
% (n-1) \sum_{i=1}^{n-1} ( \Delta (i) )^2
% = & ~ \| \vecone_{n-1} \|^2 \cdot \| \Delta \|^2 \\
% = & ~ ( \| \vecone_{n-1} \| \cdot \| \Delta \|^2 )^2 \\
% \geq & ~  ( \vecone^\top_{n-1} \Delta )^2  \\
% = & ~ (  \sum_{i=1}^{n-1} \Delta(i) )^2
% \end{align*}

% \end{proof}

% \Zhao{We skip Lemma 4.6.2}

% \subsubsection{Bounding $\lambda_2$ of a Path Graph}

% I'll now demonstrate the power of Lemma 4.6.1 by using it to prove a lower bound on $\lambda_2 (P_n)$ that will be very close to the upper bound we obtained from the test vector.

% To prove a lower bound on $\lambda_2 (P_n)$, we will prove that some multiple of the path is at least the complete graph. To this end, write
% \begin{align*}
% L_{K_n} = \sum_{i < j} L_{G_{i,j}}
% \end{align*}
% and recall that
% \begin{align*}
% \lambda_2 (K_n) = n.
% \end{align*}

% For every edge $(i,j)$ in the complete graph, we apply the only inequality available in the path :
% \begin{align*}
% G_{i,j}
% \preceq & ~ (j-i) \sum_{k=i}^{j-1} G_{k,k+1} \\
% \preceq & ~ (j-i) P_n.
% \end{align*}
% This inequality says that $G_{i,j}$ is at most $(j-i)$ times the part of the path connecting $i$ to $j$, and that this part of the path is less than the whole.

% Summing inequality (4.3) over all edges $(i,j) \in K_n$ gives
% \begin{align*}
% K_n = \sum_{i < j} G_{i,j} \preceq \sum_{i,j} (j-i)P_n.
% \end{align*}
% To finish the proof, we compute
% \begin{align*}
% \sum_{1 \leq i < j \leq n} (j-i)
% = & ~ \sum_{k=1}^{n-1} k (n-k) \\
% = & ~ n (n+1) (n-1)/6.
% \end{align*}

% So
% \begin{align*}
% L_{K_n} \preceq \frac{ n(n+1) (n-1) }{6} \cdot L_{P_n}.
% \end{align*}

% Applying Lemma 4.4.1, we obtain
% \begin{align*}
% \frac{6}{ (n+1)(n-1) } \leq \lambda_2 (P_n).
% \end{align*}
% This only differs from the upper bound (4.1) by a factor of 2. \Zhao{will fix (4.1) later.}

% \subsection{The complete binary tree}

% Let's do the same analysis with the complete binary tree.

% One way of understanding the complete binary tree of depth $d+1$ is to identify the vertices of the tree strings over $\{0,1\}$ of length at most $d$. The root of the tree is the empty string. Every other node has one ancestor, which is obtained by removing the last character of its string, and two children, which are obtained by appending one character to its label.

% Alternatively, you can describe it as the graph on $n = 2^{d+1} - 1$ nodes with edges of the form $(i,2i)$ and $(i,2i+1)$ for $i < n$.
% We will name this graph $T_d$.

% \Zhao{Let's ignore the picture for now.}

% Let's first upper bound $\lambda_2 (T_d)$ by constructing a test vector $x$. Set $x(1) = 0$, $x(2) = 1$, and $x(3) = -1$. Then, for every vertex $u$ that we can reach from node $2$ without going through node $1$, we set $x(u) = 1$. For all the other nodes, we set $x(u) = -1$.

% We then have
% \begin{align*}
% \lambda_2
% \leq & ~ \frac{ \sum_{ (i,j) \in T_d } ( x_i - x_j )^2 } { \sum_i x_i^2 } \\
% = & ~ \frac{ (x_1 - x_2)^2 + (x_1 - x_3)^2  }{ n - 1 } \\
% = & ~ 2/ (n-1).
% \end{align*}

% We will again prove a lower bound comparing $T_d$ to the complete graph. For each edge $(i,j) \in K_n$, let $T_d^{i,j}$ denote the unique path in $T$ from $i$ to $j$. This path will have length at most $2d$. So, we have
% \begin{align*}
% K_n
% = & ~ \sum_{i < j} G_{i,j} \\
% \preceq & ~ \sum_{i < j} (2d) T_d^{i,j} \\
% \preceq & ~ \sum_{i < j} (2 \log_2 n) T_d \\
% = & ~ {n \choose 2} (2\log_2 n) T_d
% \end{align*}
% SO, we obtain the bound
% \begin{align*}
% {n \choose 2} \cdot (2 \log_2 n) \lambda_{2} (T_d) \geq n,
% \end{align*}
% which implies
% \begin{align*}
% \lambda_2 (T_d) \geq \frac{1}{ (n-1) \log_2 n }
% \end{align*}
% In the next problem set, I will ask you to improve this lower bound to $1/(cn)$ for some constant $c$.

% \section{Graded HW notes}

% \begin{itemize}
% \item I'd like an exercise about applying accelerated gradient
%   descent to
% \end{itemize}

%\FloatBarrier
%\bibliographystyle{alpha}
%\bibliography{refs}

%\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "agao21_script"
%%% TeX-engine: luatex
%%% End: