\documentclass{amsart}

\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{listings}
\usepackage{color}
\usepackage{courier}
\definecolor{mygreen}{RGB}{28,172,0} % color values Red, Green, Blue
\definecolor{mylilas}{RGB}{170,55,241}
\lstset{language=Matlab,%
    basicstyle=\footnotesize\ttfamily,
    breaklines=true,%
    morekeywords={matlab2tikz},
    keywordstyle=\color{blue},%
    morekeywords=[2]{1}, keywordstyle=[2]{\color{black}},
    identifierstyle=\color{black},%
    stringstyle=\color{mylilas},
    commentstyle=\color{mygreen},%
    showstringspaces=false,%without this there will be a symbol in the places where there is a space
    numbers=none,%
    numberstyle={\tiny \color{black}},% size of the numbers
    numbersep=9pt, % this defines how far the numbers are from the text
    emph=[1]{for,end,break},emphstyle=[1]\color{red}, %some words to emphasise
    %emph=[2]{word1,word2}, emphstyle=[2]{style},
}

\theoremstyle{definition}
\newtheorem{remark}{Remark}
\newtheorem{exercise}{\color{red}Exercise}
\newtheorem{example}{Example}

\title[Programming finite elements in Matlab]{Programming finite elements in high-level languages with a focus on Matlab}
\author{Tom Gustafsson}

\begin{document}

\maketitle

\tableofcontents

\section{Variational problems}

Finite element method for linear problems concerns the computer
solution of the following \emph{variational problem}: find $u
\in U$ such that
\begin{equation}
  a(u,v) = l(v),
\end{equation}
for every $v \in V$. Here $U$ and $V$ are Hilbert spaces with the
norms $\|\cdot\|_U$ and $\|\cdot\|_V$, $a \in \mathcal{L}(U \times V;
\mathbb{R})$ is a continuous bilinear form and $l \in
\mathcal{L}(V;\mathbb{R})$ is a continuous linear form.

Recall that if $U=V$ the existence and uniqueness of the solution
follows from the Lax--Milgram theorem if the bilinear form $a$ is
\emph{coercive} in $V$. If $a$ is not coercive or if $U\neq V$, a more
general theory is required, see Babu\v{s}ka--Aziz~\cite{BA72}.

\begin{remark}
    The uniqueness aspect has no effect on the practical programming
in the sense that using finite elements one can easily \emph{try} to
discretize and solve continuous problems that are not
well-posed. Usually such approaches lead to singular,
non-invertible, matrix systems.
\end{remark}

\section{Galerkin methods}

Suppose that there exist finite dimensional subspaces $U_N \subset U$
and $V_M \subset V$ with the dimensions $N$ and $M$, respectively. The
\emph{discrete variational problem} reads: find $u_N \in U_N$ such
that
\begin{equation}
  \label{eq:dweakf}
  a(u_N,v) = l(v),
\end{equation}
for every $v \in V_M$.

\begin{remark}
  If $U_N = V_M$, the resulting method is sometimes referred to as the
  \emph{Ritz--Galerkin method}. Otherwise, the term
  \emph{Petrov--Galerkin method} is used.
\end{remark}

Suppose that $U_N = \mathrm{span}\,\{\varphi_i\}_{i=1}^N$ and $V_N =
\mathrm{span}\,\{\xi_i\}_{i=1}^M$ for some basis functions $\varphi_i
\in U_N$, $i\in\{1,\cdots,N\}$ and $\xi_i \in V_M$,
$i\in\{1,\cdots,M\}$.
Then we may expand the solution as
\begin{equation}
  u_N = \sum_{j=1}^N u_j \varphi_j,
\end{equation}
where $u_j \in \mathbb{R}$, $j\in\{1,\cdots,N\}$, are the degrees of
freedom. Substituting this expanded solution to \eqref{eq:dweakf}
leads to the matrix system
\begin{equation}
  \label{eq:matrixsystem}
  A x = b,
\end{equation}
where $A \in \mathbb{R}^{M \times N}$, $A_{ij} = a(\varphi_j, \xi_i)$, $b \in \mathbb{R}^M$, $b_i = l(\xi_i)$ and $x \in \mathbb{R}^N$, $x_i = u_i$.

\begin{exercise}
  Derive \eqref{eq:matrixsystem} from \eqref{eq:dweakf}.
\end{exercise}

The programming aspect of the Galerkin method is focused on
understanding and implementing the \emph{assembly procedure} that
constructs the pair $(A,b)$ given a bilinear form $a$, linear form $l$
and a set of basis functions. It is now \emph{crucial} to understand
that given a basis and a bilinear form, we \emph{always} have a corresponding
matrix that approximates the continuous operator.

\begin{remark}
    In mathematical literature the finite dimensional spaces $U_N$ and
$V_M$ are sometimes defined by limiting the set of arbitrary functions
to, e.g., piecewise polynomial functions. In such cases the basis is
implicit in the sense that we must do additional work to find the
basis functions or evaluate them pointwise.  Thus, from a programming
point-of-view an explicit basis is usually more preferable.
\end{remark}

\subsection{On the choice of basis functions}

Let us briefly discuss through examples how the choice of the basis
affects to the properties of the matrix $A$. In the following examples
$U=V$ and $U_N = V_M$.

\begin{example}
  \label{ex:diagonal}
  Consider the bilinear and linear forms
  \begin{equation}
    a(w,v) = \int_0^1 w^\prime v^\prime\,\mathrm{d}x, \quad l(v) = \int_0^1 v\,\mathrm{d}x,
  \end{equation}
  and the basis $\{\sin (\pi (2n-1) x)\}_{n=1}^N$. In terms of Matlab
  code, we have
  \begin{lstlisting}
    syms x
    N=5;
    dphi=diff(sin(pi*(1:2:(2*N))*x),x);
    A=int(dphi'*dphi,x,0,1);
  \end{lstlisting}
  which outputs the matrix
  \begin{equation}
\left(\begin{array}{ccccc} \frac{{\pi}^2}{2} & 0 & 0 & 0 & 0\\ 0 & \frac{9\, {\pi}^2}{2} & 0 & 0 & 0\\ 0 & 0 & \frac{25\, {\pi}^2}{2} & 0 & 0\\ 0 & 0 & 0 & \frac{49\, {\pi}^2}{2} & 0\\ 0 & 0 & 0 & 0 & \frac{81\, {\pi}^2}{2} \end{array}\right).
  \end{equation}
  Notice that the matrix is diagonal. This means that the solution
will be quick.  The corresponding linear form can be assembled and
the system solved with
  \begin{lstlisting}
    b=int(sin(pi*(1:2:(2*N))*x),x,0,1)';
    x=A\b;
  \end{lstlisting}
\end{example}

\begin{exercise}
  Draw the solution given by the Galerkin method in the previous example.
\end{exercise}

\begin{remark}
  The usage of symbolic integration is not a good idea in general due
to a high computational cost in comparison to numerical integration.
\end{remark}

\begin{example}
  Consider the same bilinear form as in Example~\ref{ex:diagonal}
  and the polynomial basis $\{x^n(1-x)^n\}_{n=1}^N$. Now the matrix $A$ is
  \begin{equation}
    \left(\begin{array}{ccccc} \frac{1}{3} & \frac{1}{15} & \frac{1}{70} & \frac{1}{315} & \frac{1}{1386}\\ \frac{1}{15} & \frac{2}{105} & \frac{1}{210} & \frac{4}{3465} & \frac{5}{18018}\\ \frac{1}{70} & \frac{1}{210} & \frac{1}{770} & \frac{1}{3003} & \frac{1}{12012}\\ \frac{1}{315} & \frac{4}{3465} & \frac{1}{3003} & \frac{4}{45045} & \frac{1}{43758}\\ \frac{1}{1386} & \frac{5}{18018} & \frac{1}{12012} & \frac{1}{43758} & \frac{5}{831402} \end{array}\right).
  \end{equation}
  Notice that this time the matrix is full and hence the solution of
the linear system requires more effort.
\end{example}

Based on the examples, we can deduce that the choice of the basis (or,
equivalently, the subspace $V_N$) has an effect on
\begin{itemize}
  \item the \emph{sparsity
    pattern} of the resulting matrix, and
    \item the \emph{condition
number}\footnote{As given by the Matlab function \emph{cond}.} which for the
former matrix is 81 whereas the condition number of the latter matrix
is approximately $2.1 \cdot 10^8$.
\end{itemize}
Moreover, let $P_N : V \rightarrow V_N$ be an orthogonal
projection. Using the continuity and coercivity of $a$, we get
\begin{align*}
  \|u-u_N\|_V^2 &\leq C^\prime a(u-u_N,u-u_N) \\
                &= C^\prime a(u-u_N,u-P_N u) \\
  &\leq C \|u-u_N\|_V \|u-P_N u\|_V,
\end{align*}
for some $C, C^\prime >0$. This leads to
\begin{equation}
  \|u-u_N\|_V \leq C \|u-P_N u\|_V,
\end{equation}
which essentially means that the error (in the sense of $\|u-u_N\|_V$)
depends on \emph{how well the space $V_N$ can represent functions of
the space $V$}. In an ``ideal case'' the exact solution belongs to the
discrete space $V_N$ leading to zero error. In practice this is rarely
the case.

Everything we have discussed so far applies to all Galerkin methods.
Next we focus on a particular Galerkin method called the \emph{finite
element method}.

\section{Finite element method}

Finite element methods are special Galerkin methods where the basis
functions are defined using a \emph{mesh}. In the following we
consider two dimensional problems but many parts can be
straightforwardly generalized to three dimensional problems.

\subsection{Mesh}

Let $\Omega \subset \mathbb{R}^2$ a polygonal domain. We split the
domain $\Omega$ into non-overlapping triangles or \emph{elements} $T
\in \mathcal{T}$ that satisfy
\begin{equation}
  \overline{\Omega} = \bigcup_{T \in \mathcal{T}} \overline{T}.
\end{equation}
The collection of elements $\mathcal{T}$ is called a mesh.  One
frequently encounters the notion of a \emph{mesh parameter $h_T$ of a
triangle $T$} that describes the maximum edge length of $T$, i.e.,
\begin{equation}
  h_T = \max_{x,y \in T} \|x-y\|,
\end{equation}
where $\|\cdot\|$ is the Euclidean norm. The mesh parameter of a
mesh $\mathcal{T}$ is then
\begin{equation}
  h = \max_{T \in \mathcal{T}} h_T.
\end{equation}

Let $N$ be the number of vertices in the mesh and let $M$ be the
number of elements.  In high-level array-based languages such as
Matlab, a convenient way to represent triangular (and other) meshes is
to use two matrices $p \in \mathbb{R}^{2 \times N}$ and $t \in
\mathbb{R}^{3 \times M}$. In the former, each column corresponds to
$x$ and $y$ coordinates of a vertex and in the latter each column
corresponds to the three indices of the vertices of an element.

\begin{example}
  Using Matlab PDE Toolbox we can generate a triangular mesh for the
  unit square with $h\leq 0.1$ using the command
\begin{lstlisting}
    [p,~,t]=initmesh(decsg([3 4 0 1 1 0 0 0 1 1]'),'Hmax',0.1)
\end{lstlisting}
  The matrix $t$ given by \emph{initmesh} has one extra row which
  in case of multiple domains tells us the subdomain in which the
  triangle lies. We do not need this information and therefore we
  strip the last row as follows:
\begin{lstlisting}
    t=t(1:3,:);
\end{lstlisting}
  The mesh can now be visualized with
\begin{lstlisting}
    trimesh(t',p(1,:),p(2,:));
\end{lstlisting}
\end{example}

\begin{exercise}
  Visualize two meshes with different mesh parameters.
\end{exercise}

\subsection{Definition of the piecewise linear basis functions}

In this text we focus on the \emph{piecewise linear basis}.  Each
basis function is defined so that it has value 1 in one vertex of the
mesh and value 0 in all other vertices. Furthermore, the basis
function is linear inside each element.

\begin{example}
  Let us generate a mesh and visualize all the basis functions.
  \begin{lstlisting}
    [p,~,t]=initmesh(decsg([3 4 0 1 1 0 0 0 1 1]'),'Hmax',0.5);
    t=t(1:3,:);
    figure;
    % there is one basis function per node, loop over nodes
    for itr = 1:size(p,2)
        subplot(ceil(sqrt(size(p,2))),ceil(sqrt(size(p,2))),itr);
        % the basis function is zero everywhere else ...
        u=zeros(size(p,2),1);
        % ... but one node.
        u(itr)=1;
        % draw the resulting surface
        trisurf(t',p(1,:),p(2,:),u);
        colormap parula;
        shading interp;
        view(2);
    end
  \end{lstlisting}
  From the resulting figure you can probably see why these
  kind of basis functions are sometimes referred to as
  the \emph{hat functions}.
\end{example}

These kind of basis functions have some nice properties.
For example:
\begin{enumerate}
\item They are based on a mesh and hence any domain
  that can be meshed is amenable to discretization.
\item The basis functions are nonzero on a very small area.
  Therefore, most entries of the resulting system matrix are zero
  and the matrix will be sparse.
\item We can easily project an arbitrary function to the
  basis by evaluating it at the vertices of the mesh.
\end{enumerate}

\subsection{Assembly of finite elements}

Let us denote by $\varphi_i$, $i \in \{1, \cdots, N\}$ the set of
piecewise linear basis functions defined by the mesh $\mathcal{T}$
of the domain $\Omega$.
Consider the bilinear form
\begin{equation}
  a(w,v) = \int_\Omega \nabla w \cdot \nabla v\,\mathrm{d}x.
\end{equation}
Our goal is to construct the matrix $A \in \mathbb{R}^{N \times N}$
with the entries $A_{ij} = a(\varphi_j, \varphi_i)$.  Since $N$ can be
very large and a large portion of the values $A_{ij}$ are zero, it is
not a good idea to evaluate each $a(\varphi_j, \varphi_i)$
separately. Instead, we evaluate \emph{all nonzero terms} of the form
\begin{equation}
  \label{eq:integral}
    \int_T \nabla \varphi_j \cdot \nabla \varphi_i\,\mathrm{d}x, \quad T \in \mathcal{T}, \quad i,j \in \{1,\cdots,N\},
\end{equation}
and add them together appropriately to \emph{assemble} $A$.

\begin{exercise}
  Download the commented version of the Matlab class template with the command
  \begin{lstlisting}
    unzip('http://super.org/file.zip')
  \end{lstlisting}
  Read through \emph{LinearAssembler.m} and try to run the tests with
  \begin{lstlisting}
    run(LinearAssemblerTests)
  \end{lstlisting}
\end{exercise}

Next we start introducing necessary concepts to implement the methods
of the class \verb|LinearAssembly|. We first implement the static
methods, secondly we write the constructor which does some
precomputations and finally implement the methods that assemble
bilinear and linear forms.

\subsection{Reference element}

Evaluation of the integral \eqref{eq:integral} can be done
systematically by mapping it to the \emph{reference triangle}
$\widehat{T}$ defined as
\begin{equation}
  \widehat{T} = \{ (\widehat{x}, \widehat{y}) : \widehat{x} \geq 0,~ \widehat{y} \geq 0,~ \widehat{x}+\widehat{y}\leq 1 \}.
\end{equation}
We end up evaluating integrals of polynomials on the reference
triangle and for this we require \emph{quadrature rules}. The idea of
quadrature rules is to replace the integral with a sum, i.e., for
a given polynomial $p \in P_n(\widehat{T})$
and triplets $(\widehat{x}_k,\widehat{y}_k,w_k)$, $k \in \{1, \cdots, M\}$,
we write
\begin{equation}
  \int_{\widehat{T}} p(\widehat{x},\widehat{y}) \,\mathrm{d}\widehat{x}\,\mathrm{d}\widehat{y} \approx \sum_{k = 1}^{M} p(\widehat{x}_k,\widehat{y}_k) w_k.
\end{equation}
\begin{remark}
  For any given polynomial order $n$ the quadrature rule can be exact
  if $M$ and the triplets are chosen right. The construction of exact
  rules is, however, non-trivial. See, e.g., Dunavant~\cite{D85} where
  an exact rule for second order polynomials is given by the triplets
  \begin{equation}
    (\tfrac{1}{3},\tfrac{1}{3},\tfrac{1}{6}), \quad
    (\tfrac{2}{3},\tfrac{1}{3},\tfrac{1}{6}), \quad
    (\tfrac{1}{3},\tfrac{2}{3},\tfrac{1}{6}).
  \end{equation}
\end{remark}
\begin{exercise}
  Implement the method \verb|localQuadrature|.
\end{exercise}
Let $F_T : \widehat{T} \rightarrow T$ be an affine mapping of the form
\begin{equation}
  \begin{bmatrix} x \\ y \end{bmatrix} = F_T(\widehat{x},\widehat{y}) = B_T\begin{bmatrix} \widehat{x} \\ \widehat{y}\end{bmatrix}+c_T,
\end{equation}
where $B_T \in \mathbb{R}^{2 \times 2}$ and $c_T \in \mathbb{R}^{2 \times 1}$
depend on the location and the shape of the triangle $T$.
Notice that we denote by $(x,y)$ points in the global
coordinate system and by $(\widehat{x},\widehat{y})$ points in the
coordinate system of the reference element.
\begin{exercise}
  Find explicit expressions for $B_T$ and $c_T$ in terms of the
vertices of the element $T$ and implement the method \verb|affineMappings|.
\end{exercise}
\begin{exercise}
  Implement the method \verb|F| which maps points from the local coordinate
  system to the global coordinate system using $B_T$ and $c_T$.
\end{exercise}
\subsection{Transforming the local basis}
Let the local basis functions be defined as
\begin{equation}
  \widehat{\varphi}_1(\widehat{x},\widehat{y}) = 1 - \widehat{x} - \widehat{y}, \quad \widehat{\varphi}_2(\widehat{x},\widehat{y}) = \widehat{x}, \quad \widehat{\varphi}_3(\widehat{x},\widehat{y}) = \widehat{y}.
\end{equation}
Our goal is to describe the global basis functions $\varphi_i$,
$i\in\{1,\cdots,N\}$, using these local basis functions.
\begin{exercise}
  Implement the method \verb|localBasis|.
\end{exercise}
Let $m$ be the column corresponding to the triangle $T$ in the element
matrix $t$. The global basis functions that are nonzero in element $T$
can be now defined as
\begin{equation}
  \label{eq:phi}
  \varphi_{e_i^{\mathrm{T}}te_m}(F_T(\widehat{x},\widehat{y}))= \widehat{\varphi}_i(\widehat{x},\widehat{y}),
\end{equation}
for each $i \in \{1,2,3\}$. Using the chain rule, we deduce that the
derivatives transform as follows:
\begin{equation}
  \label{eq:dphi}
  \nabla \varphi_{e_i^{\mathrm{T}}te_m}(F_T(\widehat{x},\widehat{y})) = B_T^{-\mathrm{T}} \widehat{\nabla} \widehat{\varphi}_i(\widehat{x},\widehat{y}),
\end{equation}
\begin{exercise}
  Implement \verb|globalBasis| which maps the local basis to global
basis using \eqref{eq:phi} and \eqref{eq:dphi}.
\end{exercise}
Finally, the integral \eqref{eq:integral} is transformed as
\begin{equation}
  \label{eq:integraltransform}
    \int_T \nabla \varphi_{e_j^{\mathrm{T}}te_m} \cdot \nabla \varphi_{e_i^{\mathrm{T}}te_m}\,\mathrm{d}x = \int_{\widehat{T}} B_K^{-\mathrm{T}}\widehat{\nabla} \widehat{\varphi}_j \cdot B_K^{-\mathrm{T}}\widehat{\nabla}\widehat{\varphi}_i|\det B_K|\,\mathrm{d}\widehat{x}.
\end{equation}
\begin{exercise}
  Implement \verb|determinant| for computing the determinant in \eqref{eq:integraltransform}.
\end{exercise}

\begin{thebibliography}{1}
\bibitem{BA72} I.~Babu\v{s}ka, A.~K.~Aziz. The Mathematical Foundations of the Finite Element Method with Applications to Partial Differential Equations. Academic Press (1972).
\bibitem{D85} D.~A.~Dunavant. High degree efficient symmetrical Gaussian quadrature rules for the triangle (1985).
\end{thebibliography}

\end{document}


\documentclass{article}
\usepackage{ws_template}

\usepackage{amsmath}
\usepackage{listings}

\title{homework sheet 01}
\author{
	\name{Denys Sobchyshak}\\
	\imat{03636581}\\
	\email{denys.sobchyshak@gmail.com}
	\And
	\name{Sergey Zakharov}\\
	\imat{03636642}\\
	\email{ga39pad@mytum.de}
}

\begin{document}
	\maketitle

	\section{Problem}
	To find eigenvalues of the matrix A we have to solve $ det(A-\lambda I)=0 $ which cane be derived from an eigenvelue equation. \\
	\[
	det(A-\lambda I)= det
	\begin{pmatrix}
		2-\lambda & -1 & 0  \\
		-1 & 2-\lambda & -1 \\
		0 & -1 & 2-\lambda
	\end{pmatrix}
	=(2-\lambda)^3-2(2-\lambda)=(2-\lambda)((2-\lambda)^2-2)=0
	\]\\
	Thus we can see that the spectrum of the matrix is given by $\lambda(A)=\left\{2, 2-\sqrt{2},2+\sqrt{2}\right\}$. To find the corresponding eigenvectors we substitute our eigenvalues into the eigenvalue equation.\\
	\begin{enumerate}
		\item For $\lambda=2$:\\
		$
		A-2I=0 \Rightarrow
		\left(\begin{array}{ccc|c}
		2-2 & -1 & 0 & 0 \\
		-1 & 2-2 & -1 & 0\\
		0 & -1 & 2-2 & 0
		\end{array}\right)
		\Rightarrow \left\{
		\begin{array}{cc}
		x=-z\\
		y=0\\
		\end{array}\right.
		$\\
		Thus using $z=-1$ we get the next eigenvector $v_1=\begin{pmatrix}1 \\0\\ -1\end{pmatrix}$.

		\item For $\lambda=2-\sqrt{2}$:\\
		$
		A-2I=0 \Rightarrow
		\left(\begin{array}{ccc|c}
		2-2+\sqrt{2} & -1 & 0 & 0 \\
		0 & -1 & 2-2+\sqrt{2} & 0 \\
		-1 & 2-2+\sqrt{2} & -1 & 0
		\end{array}\right)
		\Rightarrow \left\{
		\begin{array}{cc}
		x=\frac{y}{\sqrt{2}}\\
		y=\sqrt{2}z\\
		z=\frac{y}{\sqrt{2}}\\
		\end{array}\right.
		$\\
		Thus using $z=1$ we get the next eigenvector $v_2=\begin{pmatrix}1 \\\sqrt{2}\\ 1\end{pmatrix}$.

		\item For $\lambda=2+\sqrt{2}$:\\
		$
		A-2I=0 \Rightarrow
		\left(\begin{array}{ccc|c}
		2-2-\sqrt{2} & -1 & 0 & 0 \\
		-1 & 2-2-\sqrt{2} & -1 & 0\\
		0 & -1 & 2-2-\sqrt{2} & 0
		\end{array}\right)
		\Rightarrow \left\{
		\begin{array}{cc}
		x=-\frac{y}{\sqrt{2}}\\
		y=-\sqrt{2}z\\
		z=-\frac{y}{\sqrt{2}}\\
		\end{array}\right.
		$\\
		Thus using $z=1$ we get the next eigenvector $v_3=\begin{pmatrix}1 \\-\sqrt{2} \\ 1\end{pmatrix}$.
	\end{enumerate}
	The corresponding python/numpy code looks as follows:\\
	\lstinputlisting[language=Python,firstline=21]{scripts/ws1-linear-algebra.py}

	\section{Problem}
	If $ x_1, x_2 ... x_n$ are linearly independent eigenvectors of $ B \in  R^{n \times n} $ and $ \lambda_i $ are its eigenvalues then the following equation is satisfied: $ Bx_i = \lambda_i x_i $.
	This equation can be rewritten as $ BU = UD $ from here we get matrix $ B = UDU^{-1} $ as follows:\\\\
	$
	BU=B\begin{bmatrix} X_{1} & X_{2} & \cdots & X_n \end{bmatrix}
	=\begin{bmatrix} BX_{1} & BX_{2} & \cdots & BX_n \end{bmatrix}
	=\begin{bmatrix} \lambda_1X_{1} & \lambda_2X_{2} & \cdots & \lambda_nX_n \end{bmatrix}\\
	=\begin{bmatrix}
	\lambda_1x_{1,1} & \lambda_2x_{2,1} & \cdots & \lambda_nx_{n,1} \\
	\lambda_1x_{1,2} & \lambda_2x_{2,2} & \cdots & \lambda_nx_{n,2} \\
	\vdots & \vdots & \ddots & \vdots \\
	\lambda_1x_{1,n} & \lambda_2x_{2,n} & \cdots & \lambda_nx_{n,n}
	\end{bmatrix}
	=\begin{bmatrix}
	x_{1,1} & x_{2,1} & \cdots & x_{n,1} \\
	x_{1,2} & x_{2,2} & \cdots & x_{n,2} \\
	\vdots & \vdots & \ddots & \vdots \\
	x_{1,n} & x_{2,n} & \cdots & x_{n,n}
	\end{bmatrix}
	\begin{bmatrix}
	\lambda_1 & 0 & \cdots & 0 \\
	0 & \lambda_2 & \cdots & 0 \\
	\vdots & \vdots & \ddots & \vdots \\
	0 & 0 & \cdots & \lambda_n
	\end{bmatrix}\\
	=UD \Leftrightarrow BUU^{-1}=B=UDU^{-1}
	$
	\section{Problem}
	\begin{itemize}
		\item $B \bar{x} = \lambda \bar{x} $. Suppose that $\lambda$ and $\bar{x}$ are possibly complex and  $x \not=0$  .
		We take conjugate transpose to get $\bar{\lambda} x^* = (\lambda x)^* = (Ax)^* =x^*A^* = x^*A$  (because A is real and symmetric). Then $\bar{\lambda}^*x = (x^*A)x = x^*(Ax) = \lambda x^*x$. $x^*x$ is nonzero and real. So we get $\bar{\lambda} = \lambda$, which means that $\lambda$ is real.
		\item Assume $Av =\lambda v$ and  $ Aw =\mu w$. Then  \[ \lambda(v, w) = (\lambda v, w) = (Av, w) = (v, A^T w) = (v, Aw) = (v, \mu w) = \mu (v, w) \] is possible only if $(v, w) = 0$.
	\end{itemize}

	\section{Problem}
	\begin{itemize}
		\item $ |B| =  |UDU^{-1}| = |DU^{-1}U| = |D| = \prod_i{\lambda_i} $
		\item $ tr(B) = tr(UDU^{-1}) = tr(DU^{-1}U) = tr(D) = \sum_i{\lambda_i} $
	\end{itemize}

	\section{Problem}
	\begin{itemize}
		\item $ h(x_0) = w_0 + w^T x_0 = 0 \Rightarrow w^T x_0 = -w_0 $
		\item $ h(x_1) = w_0 + w^T x_1 = 0 \\
		h(x_2) = w_0 + w^T x_2 = 0 \\
		h(x_1) - h(x_2) = w^T x_1 - w^T x_2 =  w^T (x_1 - x_2) = 0 $
		\item Normal vector is defined as $\nabla h(x) = w$. $ \frac{\nabla h(x)}{|| \nabla h(x)||} = \frac{w}{||w||}$ is a unit normal vector. So $ \hat{w} = \frac{w}{||w||}$ is a unit normal vector.
		\item Distance can be found by projecting $(x - x_0)$ onto normal vector $w$. \[ D = |proj_w(x-x_0)| =\frac{w^T (x - x_0)}{||w||} = \hat{w}(x-x_0) = \frac{w^T x - w^T x_0}{||w||}  \overset{w^T x_0 = -w_0}{= \joinrel = \joinrel = \joinrel = \joinrel = \joinrel = \joinrel =} \frac{w^T x + w_0}{||w||} \]
	\end{itemize}

\end{document}


\documentclass[12pt]{article}

\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}
\usepackage{graphicx}
\usepackage{setspace}
\usepackage{bm}
\usepackage{amsmath}
\usepackage{amssymb,amsmath}
\usepackage{listings}
\usepackage{color}
\usepackage{enumitem}
\usepackage{fancyvrb}
\usepackage{hyperref}
\usepackage{diagbox}
\usepackage{float}
\geometry{letterpaper}
\linespread{1.1}% \geometry{landscape} % rotated page geometry

\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}
\definecolor{outcolor}{rgb}{0.545, 0.0, 0.0}

\lstdefinestyle{mystyle}{
	backgroundcolor=\color{backcolour},
	commentstyle=\color{codegreen},
	keywordstyle=\color{magenta},
	numberstyle=\tiny\color{codegray},
	stringstyle=\color{codepurple},
	basicstyle=\footnotesize,
	breakatwhitespace=false,
	breaklines=true,
	captionpos=b,
	keepspaces=true,
	numbers=left,
	numbersep=5pt,
	showspaces=false,
	showstringspaces=false,
	showtabs=false,
	tabsize=2
}

\lstset{style=mystyle}

\title{HW 2: State Estimation in Oil \& Gas Well Drilling}
\date{16 Feb. 2018}
\author{Franklin Zhao \\ SID: 3033030808}

\begin{document}

	\maketitle
	\newcommand{\tabitem}{~~\llap{\textbullet}~~}
	\renewcommand\theequation{\arabic{equation}}
	\renewcommand{\figurename}{Fig.}
	\renewcommand\thesection{Problem \arabic{section}:}
	\renewcommand\thesubsection{(\alph{subsection})}
	\onehalfspacing

\section{Dynamic System Modeling}
\subsection{}
\textbf{Modeling objective:} The objective is to formulate a mathmetical model that estimates the drill bit velocity given the table torque $T(t)$.\\
\textbf{Controllable input:} Table torque $\bf{T(t)}$\\
\textbf{Uncontrollable input:} Frictional torque $\bf{T_f(t)}$\\
\textbf{Measured output:} Table velocity $\bf{\omega_T(t)}$\\
\textbf{Performance output:} Bit velocity $\bf{\omega_B(t)}$
\subsection{}
For the top/table:
\begin{equation}
J_T\ddot{\theta}_T(t)=-k(\theta_T(t)-\theta_B(t))-b\omega_T(t)+T(t)
\end{equation}
For the bottom/bit:
\begin{equation}
J_B\ddot{\theta}_B(t)=k(\theta_T(t)-\theta_B(t))-b\omega_B(t)-T_f(t)
\end{equation}
\subsection{}
\begin{equation}
\frac{\partial}{\partial t}\left[
\begin{array}{c}
\theta_T\\
\dot\theta_T\\
\theta_B\\
\dot\theta_B
\end{array}\right]=\underbrace{\left[
\begin{array}{cccc}
0&1&0&0\\
-\frac{k}{J_T}&-\frac{b}{J_T}&\frac{k}{J_T}&0\\
0&0&0&1\\
\frac{k}{J_B}&0&-\frac{k}{J_B}&-\frac{b}{J_B}
\end{array}\right]}_\text{\textbf{A}}\left[
\begin{array}{c}
\theta_T\\
\dot\theta_T\\
\theta_B\\
\dot\theta_B
\end{array}\right]+\underbrace{\left[
\begin{array}{c}
0\\
\frac{1}{J_T}\\
0\\
0
\end{array}
\right]}_\text{\textbf{B}}T+\left[
\begin{array}{c}
0\\
0\\
0\\
-\frac{1}{J_B}
\end{array}
\right]T_f
\end{equation}
\begin{equation}
\dot\theta_T=\underbrace{\left[
\begin{array}{cccc}
0&1&0&0
\end{array}
\right]}_\text{\textbf{C}}\left[
\begin{array}{c}
\theta_T\\
\dot\theta_T\\
\theta_B\\
\dot\theta_B
\end{array}
\right]
\end{equation}
\section{Observability Analysis}
\subsection{}
\lstinputlisting[language=Python]{2a.py}
\begin{Verbatim}
Rank of Observability Matrix for four-state system: 3
\end{Verbatim}
Since $rank(\mathcal{O})=3<4$, $(A,C)$ is not observable.
\subsection{}
\begin{equation}
\frac{\partial}{\partial t}\left[
\begin{array}{c}
\theta\\
\dot\theta_T\\
\dot\theta_B
\end{array}\right]=\underbrace{\left[
	\begin{array}{ccc}
	0&1&-1\\
	-\frac{k}{J_t}&-\frac{b}{J_t}&0\\
	\frac{k}{J_B}&0&-\frac{b}{J_B}
	\end{array}\right]}_\text{\textbf{A}}\left[
\begin{array}{c}
\theta\\
\dot\theta_T\\
\dot\theta_B
\end{array}\right]+\underbrace{\left[
	\begin{array}{c}
	0\\
	\frac{1}{J_T}\\
	0
	\end{array}
	\right]}_\text{\textbf{B}}T+\left[
\begin{array}{c}
0\\
0\\
-\frac{1}{J_B}
\end{array}
\right]T_f
\end{equation}
\begin{equation}
\dot\theta_T=\underbrace{\left[
	\begin{array}{cccc}
	0&1&0
	\end{array}
	\right]}_\text{\textbf{C}}\left[
\begin{array}{c}
\theta\\
\dot\theta_T\\
\dot\theta_B
\end{array}
\right]
\end{equation}
\subsection{}
\lstinputlisting[language=Python]{2c.py}
\begin{Verbatim}
Rank of Observability Matrix for three-state system: 3
\end{Verbatim}
Since $rank(\mathcal{O})=3$, $(A,C)$ is observable.
\section{Measurement Data}
\lstinputlisting[language=Python]{3.py}
The plots of table torque and measured table velocity versus time are shown in Figure~\ref{fig:3}.
\begin{figure}[H]
	\centering
	\includegraphics[width=\linewidth]{3.png}
	\caption{Table torque and measured table velocity versus time}
	\label{fig:3}
\end{figure}
\section{Luenberger Observer}
\subsection{}
Denote $\left[
\begin{array}{c}
\hat\theta\\
\hat{\dot\theta}_T\\
\hat{\dot\theta}_B
\end{array}
\right]$ by $\hat x(t)$, $\hat{\dot\theta}_T$ by $\hat y(t)$, $T(t)$ by $u(t)$, and the Luenberger observer equations will be:
\begin{equation}\label{eq:luen1}
\dot{\hat x}(t)=A\hat x(t)+Bu(t)+L[y(t)-\hat y(t)],\ \ \ \hat x(0)=\hat{x}_0
\end{equation}
\begin{equation}\label{eq:luen2}
\hat y(t)=C\hat x(t)
\end{equation}
\lstinputlisting[language=Python]{4a.py}
\begin{Verbatim}
Eigenvalues of open-loop system:
[-0.08338525+0.29860789j -0.08338525-0.29860789j -0.08322949+0.j        ]
\end{Verbatim}
\subsection{}
The eigenvalues I chose were $2.5\lambda(A)$. The choice was justified in the RMSE error to be computed and converge speed to be shown in part (d). Basically I chose \{5, 1, 2, 3, 2.5\} and did the validation. It turned out that $2.5\lambda(A)$ was the best choice.
\lstinputlisting[language=Python]{4b.py}
\begin{Verbatim}
L:
[[-25.125]
[  0.375]
[ -0.675]]
\end{Verbatim}
\subsection{}
Write Equation~(\ref{eq:luen1})(\ref{eq:luen2}) in LTI form, and we obtain $A_{lobs}$, $B_{lobs}$, $C_{lobs}$:
\begin{equation}
\dot{\hat x}(t)=\underbrace{(A-LC)}_{A_{lobs}}\hat x(t)+\underbrace{\left[
\begin{array}{cc}
B&L
\end{array}
\right]}_{B_{lobs}}\left[
\begin{array}{c}
u(t)\\
y(t)
\end{array}
\right]
\end{equation}
\begin{equation}
\hat y(t)=C_{lobs}\hat x(t)
\end{equation}
\subsection{}
The eigenvalues I chose were $2.5\lambda(A)$:
\begin{Verbatim}
array([-0.20846314+0.74651974j, -0.20846314-0.74651974j, -0.20807373+0.j])
\end{Verbatim}
\lstinputlisting[language=Python]{4d.py}
\begin{Verbatim}
Luenberger Observer RMSE: 0.0991557694635
\end{Verbatim}
The result of Luenberger observer is shown in Figure~\ref{fig:4}
\begin{figure}[H]
	\centering
	\includegraphics[width=\linewidth]{4.png}
	\caption{Luenberger observer result}
	\label{fig:4}
\end{figure}
\newpage
\section{Kalman Filter (KF) Design}
\subsection{}
\begin{equation}
\dot{\hat x}(t)=A\hat x(t)+Bu(t)+L(t)(y(t)-\hat y(t))
\end{equation}
\begin{equation}
\hat y(t)=C\hat x(t)
\end{equation}
\begin{equation}
L(t)=\Sigma(t)C^TN^{-1}
\end{equation}
where $\Sigma(t)$ is the solution of the following differential equation:
\begin{equation}
\dot\Sigma(t)=\Sigma(t)A^T+A\Sigma(t)+W-\Sigma(t)C^TN^{-1}C\Sigma(t),\ \ \ \Sigma(0)=\Sigma_0
\end{equation}
where $W$ and $N$ are covariance matrices of $w(t)$ and $n(t)$.
\subsection{}
\lstinputlisting[language=Python]{5b.py}
\begin{Verbatim}
Kalman Filter RMSE: 0.0560275348234 rad/s
\end{Verbatim}
The value of $W$ I tuned is listed as follow. Basically I determined the magnitude first. Then based on the plots in part (c), I tuned the diagonal elements, and finally tuned other elements. It turned out that the RMSE did not change too much, but $\Sigma_{33}$ changed dramatically.
\begin{Verbatim}
W = np.matrix([[0.0006, 0.005, 0.001],
	       [0.02,   0.003, 0.004],
	       [0.004,  0.001, 0.003]])
\end{Verbatim}
\subsection{}
\lstinputlisting[language=Python]{5c.py}
The result of Kalman Filter is shown in Figure~\ref{fig:5}.
\begin{figure}[H]
	\centering
	\includegraphics[width=\linewidth]{5.png}
	\caption{Kalman filter result}
	\label{fig:5}
\end{figure}
\subsection{}
\lstinputlisting[language=Python]{5d.py}
\begin{Verbatim}
array([-0.08707987+0.2849352j, -0.08707987-0.2849352j, -0.34475403+0.j])
\end{Verbatim}
The 1st and 2nd poles are very close to the eigenvalues I chose in the Luenberger observer, but the 3rd one is a bit more different.
\section{Extended Kalman Filter (EKF) Design}
Replace Hooke's law with nonlinear spring torque relationship, and the EKF can be obtained:
\begin{equation}
\underbrace{\frac{\partial}{\partial t}\left[
\begin{array}{c}
\theta\\
\dot\theta_T\\
\dot\theta_B
\end{array}
\right]}_{\hat x(t)}=\underbrace{\left[
\begin{array}{c}
\hat\theta_T - \hat\theta_B\\
\frac{-k_1\theta-k_2\theta^3-b\dot\theta_T+T(t)}{J_T}\\
\frac{k_1\theta+k_2\theta^3-b\dot\theta_B}{J_B}
\end{array}
\right]}_{f(x(t),u(t))}+w(t),\ \ \ x(0)=x_0
\end{equation}
\begin{equation}
\dot\theta_T=\underbrace{C\left[
\begin{array}{c}
\theta\\
\dot\theta_T\\
\dot\theta_B
\end{array}
\right]}_{h(x(t),u(t))}+n(t)
\end{equation}
Hence, $F(t)$ and $H(t)$ can be derived:
\begin{equation}
F(t)=\frac{\partial f}{\partial x}(\hat x(t),u(t))=\left[
\begin{array}{ccc}
0&1&-1\\
\frac{-k_1-3k_2\hat\theta^2}{J_T}&\frac{-b}{J_T}&0\\
\frac{k_1+3k_2\hat\theta^2}{J_B}&0&\frac{-b}{J_B}
\end{array}
\right]
\end{equation}
\begin{equation}
H(t)=\frac{\partial h}{\partial x}(\hat x(t),u(t))=C=\left[
\begin{array}{ccc}
0&1&0
\end{array}\right]
\end{equation}
\lstinputlisting[language=Python]{6.py}
\begin{Verbatim}
Extended Kalman Filter RMSE: 0.0458628285791 rad/s
\end{Verbatim}
The result of EKF is shown in Figure~\ref{fig:6}.
\begin{figure}[H]
	\centering
	\includegraphics[width=\linewidth]{6.png}
	\caption{EKF result}
	\label{fig:6}
\end{figure}
\end{document}

HW2 - State Estimation/ZHAO_FRANKLIN_HW2.tex


\subsection{A Gluing Construction}
In previous parts, we saw that we can make $\log$ is a complete analytic function.
But we are not entirely satisfied, as it is just a bunch of function elements related together, instead of a genuin function.
We shall construct a space as a ``bigger domain'' $R$ at which we can realise $\log$ as a geniune function.\\
The idea is to consider the function elements we defined earlier and glue them together.
We define
$$R=\left( \coprod_{n\in\mathbb Z}U_{I(n)} \middle)\right/\sim$$
where $z_1\in U_{I(m)}$ and $z_2\in U_{I(n)}$ have $z_1\sim z_2$ iff $z_1=z_2$ as elements of $\mathbb C$ and $f_{I(m)}(z_1)=f_{I(n)}(z_2)$.
We give $R$ the quotient topology.
One can imagine $R$ as an ``infinite multi-storey carpark'' that spirals up and down.
\begin{remark}
    Since $F_{I(m)}\approx F_{I(n)}$ for all $m,n\in\mathbb Z$, it follows that $R$ is path-connected by following the sequence of direct analytic continuations.
\end{remark}
Then, with this construction, we can extend all the $f_{I(n)}$ on the $U_{I(n)}$ to a global function $f:R\to\mathbb C$ by $f([z])=f_{I(n)}(z)$ for $z\in U_{I(n)}$.
\begin{proposition}
    $f$ is well-defined.
\end{proposition}
\begin{proof}
    Follows directly from our definition of the equivalence relation $\sim$.
\end{proof}
Similarly, the natural inclusions $U_{I(n)}\hookrightarrow\mathbb C_\star$ can be extended to a global function $\pi:R\to\mathbb C_\star$ with $\pi([z])=z$.
One can also easily verify that $\pi$ is well-defined.
There is a very nice relationship between $f,\pi$ and the usual exponential map.
Indeed, $\exp\circ f=\pi$, which basically tells us $f$ has basically everything we want from $\log$.\\
We can use these global functions together.
Define $\Phi([z])=(\pi([z]),f([z]))$, then $\Phi$ is injective by definition of $\sim$.
Therefore $R$ is Hausdorff as $\mathbb C^2$ is.
\begin{remark}
    $\Phi$ indeed identifies $R$ with the graph $\{(w,z)\in\mathbb C^2:w=\exp(z)\}$.
    So we can view $R$ alteratively as ``flipping'' the graph of $\exp$.
\end{remark}


\section{Empirical relevance}
\begin{frame}{Empirical relevance: definition}
    The \alert{empirical relevance} $r_i(t)$ of node $i$ at time $t$ is defined as:

    \[
        r_i(t) = \frac{n_i(t)}{n_i^{PA}(t)}
    \]

    \begin{itemize}
        \item $n_i(t) = \frac{\Delta k_i^{in}(t, \Delta t)}{L(t, \Delta t)}$: ratio between:

        \begin{itemize}
            \begin{footnotesize}
            \item $\Delta k_i^{in}(t, \Delta t)$: \# of incoming links received by node $i$ in the time window $[t, t+\Delta t]$
            \item $L(t, \Delta t)$: total \# of links created within the same time window
        \end{footnotesize}
        \end{itemize}

        \item $n_i^{PA}(t) = \frac{k_i^{in}(t)}{\sum_j k_j^{in}(t)}$: expected value of $n_i(t)$ according to preferential attachment alone
    \end{itemize}

    $r_i(t) > 1$ \alert{$(< 1)$}: node $i$ at time $t$ outperforms \alert{(underperforms)} in the competition for incoming links with respect to its preferencial attachment weight.
\end{frame}

\section{The Extended Fitness Model}

\begin{frame}{The Extended Fitness Model}
    \begin{itemize}
        \item \textbf{PageRank's under-performance in time-dependent networks is a general feature.}
        \item We can validate this using a model more compatible with the idea that \emph{a node is important if it's pointed by other important nodes.}
    \end{itemize}
    \begin{center}
        \alert{Extended Fitness Model} (EFM)
    \end{center}
    \begin{itemize}
        \item \textbf{High-fitness nodes are more sensitive to fitness} than low-fitness nodes, when choosing their outgoing links.
        \item High-fitness nodes are then more likely to be pointed by other high-fitness nodes than low-fitness nodes.
        \item EFM is \alert{more favorable} to PageRank than RM.
    \end{itemize}
\end{frame}

\begin{frame}{EFM:\@ sensitivity to fitness}
    Probability $\Pi_{i;j}^{in}(t)$ that a link created by node $j$ at time $t$ ends in node $i$:
    \[
        \Pi_{i;j}^{in}(t) \sim (k_i^{in}(t)+1)^{1-\eta_j} \, \eta_i^{\eta_j} \, f_R(t-\tau_i)
    \]
    \begin{itemize}
        \item Fitness $\eta \in [0, 1]$ to prevent negative exponents
        \item $\Pi^{in}$ depends on the fitness of the target \emph{and of the source} nodes (difference with RM).
        \item $k_i^{in}(t)$: indegree of node $i$ at time $t$.
    \end{itemize}
\end{frame}

\begin{frame}{PageRank vs.\ indegree: correlation with fitness in EFM}
    \begin{figure}
        \begin{columns}

        \column{0.6\textwidth}
            \includegraphics[width=1.0\textwidth]{figures/PageRankEFM_heatmap}

        \column{0.4\textwidth}
            \begin{footnotesize}
            \begin{itemize}
                \item $\rho(A) = 2A^{-3}, \; A \in [1, \infty]$
                \item $H$ nodes: high fitness; $\eta \in [10^{-5}, 1]$
                \item $(N-H)$ nodes: low fitness; $\eta \in [0, 10^{-5}]$
                \item $N=10000$, $H=250$
            \end{itemize}
            \end{footnotesize}
        \end{columns}
        \caption{Comparison of performance of PageRank and indegree (EFM data).}
    \end{figure}
\end{frame}


\documentclass[../../main.tex]{subfiles}
\begin{document}
\subsubsection{Append}

The operation $append$ will return a Collection with a Value added at a specified numeric Index.
\begin{schema}{Append[Collection, V, \nat]}
  coll?, coll! : Collection \\
  v? : V \\
  idx? : \nat \\
  append~\_ : Collection \cross V \cross \nat \bij Collection
  \where
  \#~idx? = 1 \\
  coll! = append(coll?, v?, idx?) @ \\
  \t1 let ~ ~ coll' == front(\{~ i : \nat ~| ~ i \in 0~..~idx?\} \extract coll?) \cat v? \\
  \t1 \ \ \ ~~~ coll'' == \{~ j : \nat ~| ~ j \in idx?~..~\#~coll?\} \extract coll? \\
  \t1 = coll' \cat coll'' \implies \\
  \t2 (front(coll')  \cat v? \cat  coll'') ~ \land \\
  \t2 (~ v? \mapsto idx? \in coll!) ~\land \\
  \t2 (~\#~ coll! = \#~ coll? + 1)
\end{schema}
$append$ results in the composition of $coll'$ and $coll''$ such that
$$coll! = coll' \cat coll'' ~ \land ~idx? \mapsto v? \in coll!$$
\begin{itemize}
\item $coll'$ is the items in $coll?$ up to and including $idx?$ but the value at $idx?$ is replaced with $v?$ such that
  $idx? \mapsto coll?_{idx?} \not \in coll'$
\item $coll''$ is the items in $coll?$ from $idx?$ to $\#~coll? \implies coll?_{idx?} \in coll''$
\end{itemize}
 The following example illustrates these properties.
\begin{argue}
  X = \langle x_{0}, x_{1}, x_{2} \rangle \\
  \t1 x_{0} = 0 \\
  \t1 x_{1} = foo \\
  \t1 x_{2} = \langle a, b, c \rangle \\
  \t1 v? = bar \\
  append(X, v?, 0) = \langle bar, 0, foo, \langle a, b, c \rangle \rangle \\
  append(X, v?, 1) = \langle 0, bar, foo, \langle a, b, c \rangle \rangle \\
  append(X, v?, 2) = \langle 0, foo, bar, \langle a, b, c \rangle \rangle \\
  append(X, v?, 3) = \langle 0, foo, \langle a, b, c \rangle, bar \rangle \\
  append(X, v?, 4) = append(X, v?, 3) \iff 3 \not \in \dom X
\end{argue}
\end{document}


\section{Defining data for Oriented TQFT}
This is the defining data for $(d + 1)$-dimensional oriented TQFT.
It Closed $d$-dimensional $\Sigma^d$
has vector space $V(\Sigma^d)$.
When $M^{d+1}$ is closed, meaning it has no boundary,
$Z(M^{d+1})\in\mathbb{C}$ is just a number.
If $M^{d+1}$ has a boundary,
then $Z(M^{d+1})\in V(\partial M^{d+1})$.
This is a state.
$Z$ is the path integral on the spacetime manifold.
You can think of it as a wave function over boundary conditions.

These are the axioms.
All manifolds have orientation.

\begin{enumerate}
    \item $Z$ and $V$ are functorial with respect to orientation-preserving
    diffeomorphisms of $M^{d+1}$ and $\Sigma^d$.
    \item $V$ should be ``involutory'' in the sense that
    $V(\bar{\Sigma^d}) = V(\Sigma^d)^*$.
    Here the $*$ means dual vector space.
    Over bar means hermitian conjugate.
    \item $Z$ and $V$ should be multiplicative.
    \item Non-triviality.
    \item $Z$ is hermitian.
\end{enumerate}

Let's explain the axioms.
\begin{axiom}[Axiom 1]
    Let $f:\Sigma_1^d \to \Sigma_2^d$ be a diffeomorphism.
    This defines an isomorphism
    $\mathcal{U}_f: V(\Sigma_1^d)\to V(\Sigma_2^d)$.
    If we also have $g: \Sigma_2^d \to \Sigma_3^d$,
    then
    $\mathcal{U}_g \mathcal{U}_f = \mathcal{U}_{gf}$.

    If $f$ extends to
    $\tilde{f}: M_1^{d+1}\to M_2^{d+1}$
    where $\partial M_1 = \Sigma_1$
    and $\partial M_2 = \Sigma_2$,
    then
    $\mathcal{U}_f\left( Z\left( M_1^{d+1} \right) \right)
    = Z\left( M_2^{d=1} \right)$.
\end{axiom}
Suppose we have a diffeomorphism from one circle $\Sigma_1 = S^1$ to another
circle.
Then there will be an induced map $\mathcal{U}_f$ from vector spaces between
then too.

\begin{question}
    Isn't $Z$ a path integral that is a number?
    Why is it a vector space?
\end{question}
You can think of $Z\left( M_1^{d+1} \right)$
as a state,
because like in path integrals you have boundary conditions,
which are variable.

\begin{question}
    What is a diffeomorphism?
\end{question}
A diffeomorphism $f:X_1 \to X_2$ is a smooth and continuous map.

\begin{question}
    What is a manifold?
\end{question}
Manifolds can be broken up into charts $\mathcal{U}_\alpha$
with coordinate charts $\phi: \mathcal{U}_\alpha\to \mathbb{R}^d$.

\begin{question}
What is the relation between $\mathcal{U}_{f}$ and
$\mathcal{U}_{\tilde{f}}$?
\end{question}
They are actually the same.

\begin{question}
    The extension from $f$ to $\tilde{f}$ is not unique?
\end{question}
It's not unique.

\begin{question}
    $\mathcal{U}_f$ acting on the partition function
    gives the other partition function.
    This is the sum of all paths.
\end{question}
Don't think of it like that.
This thing is only acting on the partition function.
Now you're saying $Z$ is the sum of something
and how does $U_{f}$ act on terms in the sum.
I'm hesitant to say anything general about that.
There's no general clean answer.
Just think of it as an abstract object $Z$
which gives a state if there is a boundary
and a number otherwise.

Let's talk about axiom 3.
Imagine if I build a manifold by gluing simpler manifolds.
Then I can build my vector spaces by some tensor products and multiplications.
Recall that $\sqcup$ is the disjoint union.
Then we can state the axiom.
\begin{axiom}[Axiom 3]
    Let
    \begin{align}
        V\left( \Sigma_1^d \sqcup \Sigma_2^d  \right)
        =
        V\left( \Sigma_1^d \right) \otimes
        V\left( \Sigma_2^d \right)
    \end{align}
    Suppose the boundary is
    \begin{align}
        \partial M^{d+1} =
        \Sigma_1 \sqcup \bar{\Sigma}_2
    \end{align}.
    Then
    \begin{align}
        Z\left( M_1^{d=1} \right) &\in
        V\left( \Sigma_1 \right) \otimes V\left( \bar{\Sigma}_2 \right)\\
        &\simeq V\left( \Sigma_1 \right) \otimes \overbar{V(\Sigma_2)}
    \end{align}
    so we can think of $Z\left( M^{d+1}_1 \right)$
    as a state in
    $V\left( \Sigma_1 \right)\otimes V\left( \overbar{\Sigma_2} \right)$
    or as a linear map from
    $V\left( \Sigma_2 \right)\to V\left( \Sigma_1 \right)$.
\end{axiom}
I can think of this as a state in a tensor product of two spaces.
\begin{align}
    Z\left( M_1^{d+1} \right)
    = \sum_{n,m} c_{n,m}
    \ket{n}_{V(\Sigma_1)}
    \otimes \ket{m}_{V(\overbar{\Sigma_2})}
\end{align}
or you can think of it as
\begin{align}
    Z\left( M_1^{d+1} \right)
    = \sum_{n,m} c_{n,m}
    \ket{n}_{V(\Sigma_1)}
    \bra{m}_{V(\overbar{\Sigma_2})}
\end{align}

\begin{question}
This only applies when the boundary is a disjoint union of two manifolds?
\end{question}
You can always think of the boundary as $\partial M^{d+1}=\Sigma_1 \sqcup
\emptyset$

\begin{question}
    Why do we need the complex conjugate?
\end{question}
It's a state on $V(\Sigma_1)\otimes V(\ovebar{\Sigma_2})$.
Actually maybe it shouldn't be there,
put a question mark there.
Actually, there should be no complex conjugate.

We can think of the path integral as a state on the vector space
$V(\Sigma_1)\otimes\overbar{V(\Sigma_2)}$
or as a map from
$V(\Sigma_2)\to V(\Sigma_1)$.

I can think of a state as a map from the vector space to the empty manifold.
I'm just moving pieces around no each side.

It's really nothing more than thinking of it as bras and kets.
Just think of this as a definition.
I have a state that lives on a tensor product of two vector spaces.
I can think of it as an operator from one vector space to another.
It's really just that simple.
There's nothing more to that.
Forget that we called it the path integral.
You can think of any tensor product state as an operator from one space to
another.

\begin{question}
    When you have 3 disjoint components of $\Sigma$,
    then how do you define orientation?
    How do you decided which orientation to use?
\end{question}
I don't think you need extra information about what the boundary orientation
looks like.
If this has + orientation, then are you saying that?

Actually I think there should be a star.
\begin{align}
    Z\left( M_1^{d+1} \right)
    = \sum_{n,m} c_{n,m}^*
    \ket{n}_{V(\Sigma_1)}
    \bra{m}_{V(\overbar{\Sigma_2})}
\end{align}
We'll come back to this.

The path integral is a state on the boundary.

We've done the multiplicative axiom for $V$.
We haven't gotten to the multiplicative axiom for $Z$ yet.
We want to glue together $M$s with common boundaries.

Suppose that we have some manifold $M_1$
and another manifold $M_2$.
The boundary $\partial M_1 = \Sigma_1 \sqcup \overbar{\Sigma_2}$
and the boundary
$\partial M_2 = \Sigma_2 \sqcup \overbar{\Sigma_3}$.
Then $M=M_1 \cup_{\Sigma_2} M_2$.
If you think of these $Z$ as linear maps,
then we just compose the maps.
\begin{align}
    Z(M) = Z(M_2)\circ Z(M_1): V(\Sigma_1) \to V(\Sigma_3)
\end{align}
or equivalently,
we can think of $Z(M)$ as the inner product of these two states,
where the inner product
\begin{align}
    Z(M) =
    {\langle Z(M_1), Z(M_1) \rangle}_{V(\Sigma_2)}
    \in V(\Sigma_1) \otimes V(\overbar{\Sigma_3})
\end{align}

\begin{question}
    The bar just means going counterclockwise right?
\end{question}
Yes.

Actually there are many ways of glueing manifolds.
When I identify points together,
I can actually identify with any points I want.
So in general we need a gluing map
$\phi: \Sigma_2 \to \Sigma_2$
with
$M = M_1 \cup_\phi M_2$.

I'm going to use Axiom 1
and this $\phi$ map is going to define a corresponding map
$\mathcal{U}_\phi: V(\Sigma_2)\to V(\Sigma_2)$
so
\begin{align}
    Z\left( M_1 \cup_\phi M_2 \right)
    =
    Z(M_2) \circ \mathcal{U}_\phi \circ Z(M_1).
\end{align}
Or equivalently
$\langle Z(M_2), \mathcal{U}_\phi Z(M_1)\rangle_{V(\Sigma_2)}$.
$\mathcal{U}_\phi$ is an isomorphism between these two vector spaces.

Closed $M^{d=1}$
can be obtained by gluing together
$M_1$, $M_2$ along the their common boundary.
For example,
if you wanted to calculate the path integral on a sphere,
you blue toegether the two on a common boundary.

If you want a torus,
ou glue
\begin{align}
    T^2 = (S_1\otimes I) \union (S^1\times I).
\end{align}

That was the only non-trivial axiom,
the rest are pretty simple.

Funnily enough,.
This is about what happens when we have a non-empty manifold.
the next axiom is called non-triviality

\begin{axiom}[Axiom 4: non-triviality]
    Consider a boundary $\Sigma^d = \emptyset$.
    Then
    V(\emptyset) = V(\emptyset) \otimes V(\emptyset)
    and
    \begin{align}
        V(\emptyset) =
        \begin{cases}
            0 & \text{ordinal}\\
            \mathbb{C} & 1-dimensional
        \end{cases}
        hence we have
        \begin{align}
            V(\phi) = \mathbb{C}
        \end{align}
    \end{align}
    If $M^{d+1}$ is closed,
    $\partial M^{d+1} = \emptyset$,
    then
    $Z(M^{d+1}) \in V(\emptyset) \simeq \mathbb{C}$.
    Consider the case
    \begin{align}
        M^{d+1} = \emptyset.
    \end{align}
    Then
    \begin{align}
        Z(\emptyset) = Z\emptyset) Z(\emptyset)
    \end{align}
    which implies
    \begin{align}
        Z(\emptyset) =
        \begin{cases}
            0 &\\
            1
        \end{cases}.
    \end{align}
    Take $Z(\emptyset)=1$.
\end{axiom}

consider $M^{d+1}= \sigma^d \times I$.
Then
\begin{align}
    Z\left( \Sigma^d \times I \right)
    =
    \begin{cases}
        0\\
        II_{V(\Sigma^d)}
    \end{cases}
\end{align}
Take
$Z(\Sigma^d\times I) = II_{V(\Sigma^d)}
= \sum_{n=1}^{\dim(V(\Sigma^d))}
\ket{n}\bra{n}$
where $\{\ket{n}\}$ is orthonormal basis in $V(\Sigma^d)$.
Or,
\begin{align}
    Z(\Sigma^d\times I) =
    \sum_n
    \ket{n}_{V(\Sigma^d)}
    \otimes
    \ket{n}_{V(\Sigma)}^d.
\end{align}

The last axiom is simple.
\begin{axiom}[Hermitian]
    $Z(\overbar{M^{d+1}}) = Z(M^{d+1})^\dagger$.
\end{axiom}

\section{Non-trivial consequences}
\subsection{Homotopy invariance of maps}
This has some non-trivial consequences.
Consider $M_1^{d+1}=\Sigma^d\times I$
and consider $M_2^{d+1}=\Sigma^2\times I$
with homotopy
$F: M_1\to M_2$.
Then
$\mathcal{F}(\Sigma^d, t)
= f_t(\Sigma^d)$
for $f_t: \Sigma_d\to \Sigma^d$.
This defines a continuous family
$\mathcal{U}_{f_t}: \Sigma^d\to \Sigma^d$ satisfying
$\mathcal{U}_{f_t} = \mathcal{U}_{f_{t'}}$.

[diagram here]

If I have a homotopy group with a homotopy of maps.

\subsection{Mapping Class Group}
That leads us to the important group for a manifold.
For every manifold, there is a mapping class group $\Sigma^d$.
Colloquially, it's the group of ``large'' diffeomorphisms.
You would take all diffeomorphisms and mod out all those that are continuously
\begin{align}
    \mathrm{MCG}(\Sigma^d) = \frac{\mathrm{Diff}(\Sigma^d)}{\mathrm{diff}_0(\Sigma^d)}
\end{align}
Two diffeomorphisms are equivalent if they can be connected to each other.

This is the group taking all possible diffeomorphisms
and modding out all the ones equivalent to the identity.

The mapping class group is related to the homotopy group,
but not the same thing.
You may not understand this,
but
\begin{align}
    \mathrm{MCG}(\Sigma^d)
    = \mathrm{out}(\pi_1(\Sigma^d))
\end{align}
It's isomorphic to the group of all outer maps between loops.
An example of such a map that maps a non-trivial loop on an torus
to the other type of non-trivial loop around the torus.

In TQFT,
all my maps are smooth.
If my maps aren't smooth,
I don't have something that takes me everywhere from the left to everywhere on
the right.
If I have a smooth map,
that's what allows me to take vector spaces on the left to vector space on the
right.
The only reason I can go from left to right is because $\mathcal{F}$ is a
diffeomorphism.

The punchline is this.
For each $g\in \mathrm{MCG}(\Sigma^d)$,
we get some $\mathcal{U}_g: V(\Sigma^d) \to V(\Sigma^d)$.
If you figure out the $\mathcal{U}_g$,
you can completely figure out the TQFT.
These $\mathcal{U}_g$ are very important,
and can basically characterise the TQFT.

Is this the $S$ and $T$ matrices?
Yes.
For every element in the MCG  of a torus,
there is a linear map that acts on the ground state subspace on the system,
hence the $S$ and $T$ matrices on the subspace.
Don't worry if you've never heard of it.

Do you have a concise reference to recommend for the last two lectures?
There's an article from Atiyah 1988.
It's basically called TQFT.
My whole representation is based on this article.

\begin{question}
    Is it readable?
\end{question}
It's as readable as my lectures are understandable.
If you don't understand my lecture,
then I'm not sure.
It's not too bad,
it doesn't assume you know that much.

We're basically done,
but there's one more consequence to talk about.
These TQFTs are very baby versions of TQFTs,
and in real applications, we need some extra structure.