\documentclass[12pt]{article}
\usepackage[a4paper, bindingoffset=0.2in, %
							left=0.5in,right=0.5in,top=0.5in,bottom=0.5in,%
							footskip=.25in]{geometry}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{physics}
\usepackage{hyperref}

\title{Midterm Report}
\author{Ali Abolhassanzadeh Mahani\\ 97110863}

\begin{document}
	\maketitle
	\section{Problem 1 (3D Site Percolation)}
	\subsection{Part a}
	I made a class \texttt{PercMatrix} that takes the length $L$ as the size of the arrays, and $p$, the probability with which I fill the cells with zero and on.
	Then, I go on and generate random binary numbers, and store shared poiters to them in the cells. I use smart pointers (\texttt{std::shared\_ptr<int>}) to make the clustering process faster.
	\subsection{Part b}
	First I make a $L\times L$ matrix called \texttt{frontier} that stores the color codes through out the clustering and coloring process. Then, I add a surface matrix called \texttt{init} to the beginning of the
	cube that contains pointers to the cells in \texttt{frontier}. Then, I go through the cube cells and check if they are adjacent. If so, I form clusters, If more than one neighbors is on, I merge the cluster to the one with the lowest color code, since that cluster was made first.\\
	In the end, I check the ending surface of cube to see if any of the color code are between $1, L\times L +1$. If so, Percolation has occured. Otherwise, no percolation.
	In order to find the gyro radius, first, I find the color code of the biggest non-percolating cluster. Then, I
	find the geometric center of that cluster and then, I can calculate the gyro radius.

	\section{Problem 2 (Erdos Renyi Graph)}
	\subsection{Part a (clustering)}
	I made the graph using \texttt{nx.erdos\_renyi\_graph(n, p)}. Here $p$ is the probablity of an edge to exist which is $\frac{<k>}{n}$ with $n$ being the number of nodes.\\
	I used the \texttt{nx.clustering(G)} module to calculate the clustering of each edge and then
	used a trick to store them in a \texttt{numpy array}. Then I plotted them using \texttt{plt.hist()}. (Fig\ref{fig:clust})
	\begin{figure}[h]
		\centering
		\includegraphics[width=.4\linewidth]{../p2/clust1.jpg}
		\includegraphics[width=.4\linewidth]{../p2/clust8.jpg}
		\includegraphics[width=.4\linewidth]{../p2/clust64.jpg}
		\includegraphics[width=.4\linewidth]{../p2/clust512.jpg}
		\label{fig:clust}
		\caption{clustering distribution for mean degrees of $\{1, 8, 64, 512\}$}
	\end{figure}
	\subsection{Part b (poisson dist. of degrees)}
	First I plot the histogram of degrees using \texttt{plt.hist()}, which gives me \texttt{counts}
	and \texttt{bins}, then I make a function that returns the probability using the poisson distribution. (basically made that function). Then, I found the poisson distribution of the \texttt{bins} and compared them to the actual \texttt{counts}. \\
	Then \texttt{np.sum} over them to find the cultivated error. The error is of order $\mathcal{O}(10^{-5})$ for $<k> = 64$

	This means that the poisson distribution function fits the degree distribution fairly well.
	\section{Problem 3 (Random Generation)}
	\subsection{Part a (generating function)}
	The function that generates this distribution is the inverse of $g(x)$. It is as follows:
	\begin{equation}
		g^{-1}(x) = \left(\frac{\alpha x_m^\alpha}{x}\right)^{\frac{1}{\alpha + 1}}
	\end{equation}
	In order to generate from $x_m$ to $\infty$, we need the input to be a uniform distribution
	from $0$ to $g^{-1}(x_m)$.\\
	I created this function and called it \texttt{target\_dist()}; then I made a random set of size
	$100000$ and plotted the histogram as seen below.(Fig\ref{fig:reverse})
	\begin{figure}[h]
		\centering
		\includegraphics[width=.4\linewidth]{../p3/reverse.jpg}
		\includegraphics[width=.4\linewidth]{../p3/reverseplot.jpg}
		\label{fig:reverse}
		\caption{Distribution of $g(x)$ by 1000 samples and by ideal plot}
	\end{figure}

	\subsection{Part b (metropolis algorithm)}
	In the metropolis algorithm, first, I defined the function $g(x)$ to return it's value as states in the previous
	subsection, and to return $0$ if $x \le 1$. Then, I made a function \texttt{gen\_dist(step, size)} which takes
	in the \texttt{step} and \texttt{size} and returns an array of random numbers as expected.
	Inside the function, I used two counters, one to count acceptances and one to count the whole process.
	Using the first, I can get the acceptance rate. I also stored all the numbers generated (whether or not they
	were same)and using the second counter, I hand-picked the random samples.

	Then, I made a function \texttt{corr\_len()} that takes in all the numbers, calculates the auto correlation
	for values of $j = \{0, 10, 20, 30, \dots, 5000\}$. Then, I look to see at which point, the auto correlation becomes less than $e^{-1}$. That value of $j$ is our correlation (gyro) length.
	The results are in Table\ref{tab:gyro}
	\begin{table}[h]
		\centering
		\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
			\hline
step size& $0.2$& $0.49$ & $0.9$ & $1.5$ & $2.45$ & $4$ & $6.4$ & $13.5$ & $33$ \\
 \hline
acceptance rate & $0.9$ & $0.8$ & $0.7$ & $0.6$ & $0.5$ & $0.4$ & $0.3$ & $0.2$ & $0.1$ \\
 \hline
correlation length & $5000$ & $1190$ & $3530$ & $430$ & $440$ & $290$ & $110$ & $1310$ & $90$ \\
 \hline
		\end{tabular}
	\label{tab:gyro}
	\caption{step size, and gyro length for different values of acceptance rate. The error for gyro length is 10.
	The first with gyro length 5000 suggests that the gyro is actually more for that acceptance rate since 5000
	is the maximum value in my calculations.}
	\end{table}

	\section{Problem 4 (simulation)}
	\textbf{List of group members:} \emph{Ali Abolhassanzadeh Mahani}, \emph{Abbas Shojakani}, \emph{Mohadeseh Asgari}
	\subsection{Stating the Problem}
	A blind /drunk person wants to cross a street using nothing but random process.\\
	We want to find his/her success rate relative to the degree of which how crowded the street is.\\
	We also want to find his/her mean lifetime for the times that he/she does not succeed and dies in a crash.
	\subsection{Simulation}
	The man is on the left side of the street at first, but then, It starts moving randomly with time.
	The boundary conditions are walls on the sides of the street. Meaning that if the man staeps to the left of the street, it stays there. If he reaches the right side, he succeeds.

	We took a mean of the car speeds and decided for it to be twice the speed of the blind/drunk person.
	The width of the street is fixed to size $L$ and the number of cars that are randomly generated and put on
	the street is our independent variable. The length of the cars is 1 unit area and the
	cars move from bottom up. If in a time step, the position of the
	random walker and a car are interfered, the person is rendered dead, and is reset to be on the left side again. This cycle continues for the rest of the ensemble.\\
	We have also omitted the will of the drivers to hit the brake before hitting the person. So we have kind of assumed that the drivers are drunk too, which is a bit absurd or that the drivers' response time is not low enough to avoid a crash, which can be more reasonable.

	A better approximation of to set a condition to lower the speed if the driver sees the drunk person or that it
	shift the car to a new lane. Implementing these conditions can yield better, more accurate results. (We don't have time for that ;-) )
	\subsection{The Simulation Code}
	I made a random walker in 1 Dimension that randomly crosses the street. And every time it makes a choice, its lifetime is incremented by 1 unit time. This method \texttt{next()} that contains the decision making,
	returns 0 if the person is still in the street and returns 1 if the person has reached the right side (success).\\
	\emph{Mohadeseh} made the street and the cars that are randomly generated at the bottom of the street and move upwards with speed 40 unit length.\\
	\emph{Abbas} simulated using our code and got the statistics. The conditions of death and taking the mean are of his work. The precedence of time step evolution is with the random walker. Meaning first, the
	random walker takes a step, then the cars move with their speed. If the cars hit the man in their evolution
	process, the man dies, else, the loop continues until either the man reaches the other side of the street,
	or he dies in a crash.\\
	I also added a some code to get the mean life span for the times that the drunk person dies.
	Afterwards, I spent time refining the code, commenting it and making it pretty :)
	The results are below.

	In our work, the evolution of the cars up the street is a cellular automata and the person crossing the
	street is obviously a random walk simulation.
	\subsection{Results}
	 We simulated for the car speed $v = 40$ and the man's speed $V_p = 2$ and the street width $L = 10$.
	 Then, we plotted the success rate vs. the number of cars and the mean life span vs. the number of cars.
	 Fig\ref{fig:drunk}
	 \begin{figure}[h]
	 	\centering
		\includegraphics[width=.4\linewidth]{../p4/success.jpg}
		\includegraphics[width=.4\linewidth]{../p4/meanlife.jpg}
		\label{fig:drunk}
		\caption{On the left we have the success rate vs. the number of cars, and on the right we have
		the mean life time vs. the number of cars}
	 \end{figure}
 	As one can observe, the \emph{success rate} decays fast with \emph{number of cars}, the mean life time
 	also decays fast but a bit smoother than the success rate. The moral lesson here is that one shall not try crossing a street while drunk.

 	The results show that for example, for a mean degree of crowd about 1 car per 100 unit area, the success rate is about $0.20$ and the mean lifetime for the unsuccessful times, it roughly 8 time steps.
 	for 10 cars per 100 unit area, those numbers decay to about $0$ and $1.5$ respectively.
 	In the ladder case, the drunk person dies after taking about $1.5$ steps in the street.\\
 	This threshold of $10$ can be due to the fact that the width of our street is 10 and since the \texttt{death\_check}'s conditions are perdiodic, for a velocity of 40 unit length per unit time, the entire street becomes full and the drunk person gets hit in just 1 or steps.
\end{document}


\section{MAC network and our proposed simplification}

\begin{wrapfigure}{r}[5pt]{0.4\textwidth}
	\vspace{-15pt}
	\centering
	\includegraphics[width=\textwidth]{../img/mac_cell.pdf}
	\caption{The MAC cell, reproduced on the basis of~\cite{hudson2018compositional}.}
	\label{fig:mac_cell}
	\vspace{-5pt}
\end{wrapfigure}

The MAC network~\cite{hudson2018compositional} is a recurrent model that performs sequential reasoning, where each step involves analyzing a part of the question followed by shifting the attention over the image.
The core of the model is the MAC cell, supported with an input unit that processes the question and image pair, and output unit which produces the answer.
The input unit  uses an LSTM~\cite{hochreiter1997long} to process the question in a word-by-word manner producing a sequence of \emph{contextual words} and a final \emph{question representation}.
Additionally, the input unit uses a pre-trained ResNet~\cite{he2016resnet} followed by two CNN layers to extract a feature map (referred to as \emph{knowledge base}) from the image.

The MAC cell consists of a control unit, a read unit and a write unit (\Fig{fig:mac_cell}).
The control unit is updating the control state $c_i$ and drives the attention over the list of \emph{contextual words} $\cw$ taking into account the \emph{question representation} $q$.
Guided by $c_i$,  the read unit extracts information from the \emph{knowledge base} $\kb$ and combines it with the previous memory state $m_{i-1}$  to produce the \emph{read vector} $r_i$.
Finally, the write unit integrates $r_i$ and $m_{i-1}$ to update the memory state. Detailed equations are described in the next section.

%\begin{figure}[htbp]
%	\centering
%	\includegraphics[width=0.4\textwidth]{img/mac_cell.pdf}
%	\caption{The MAC cell~\cite{hudsonManning18}}
%	\label{fig:mac_cell}
%\end{figure}


\input{mmd-article-header}
\def\mytitle{MultiMarkdown Math Example}
\input{mmd-article-begin-doc}

\part{Math}
\label{math}

An example of math within a paragraph --- ${e}^{i\pi }+1=0$ --- easy
enough.

And an equation on it's own:

\[ {x}_{1,2}=\frac{-b\pm \sqrt{{b}^{2}-4ac}}{2a} \]

That's it.

\chapter{``DollarMath''}
\label{dollarmath}

An example of math within a paragraph --- ${e}^{i\pi }+1=0$ --- easy
enough.

An example of math within a paragraph, ${e}^{i\pi }+1=0$, easy
enough.

And an equation on it's own:

$${x}_{1,2}=\frac{-b\pm \sqrt{{b}^{2}-4ac}}{2a}$$

\chapter{Not Math}
\label{notmath}

An example of math within a paragraph --- \$ \{e\}\textsuperscript{{i}\textbackslash{}pi \}+1=0\$ --- easy
enough.

And an equation on it's own:

\$\$ \{x\}\_\{1,2\}=\textbackslash{}frac\{-b\textbackslash{}pm \textbackslash{}sqrt\{\{b\}\textsuperscript{{2}}--4ac\}\}\{2a\}\$\$

An example of math within a paragraph --- \$\{e\}\textsuperscript{{i}\textbackslash{}pi \}+1=0 \$ --- easy
enough.

And an equation on it's own:

\$\$\{x\}\_\{1,2\}=\textbackslash{}frac\{-b\textbackslash{}pm \textbackslash{}sqrt\{\{b\}\textsuperscript{{2}}--4ac\}\}\{2a\} \$\$

An example of math within a paragraph --- a\$\{e\}\textsuperscript{{i}\textbackslash{}pi \}+1=0\$ --- easy
enough.

And an equation on it's own:

a\$\$\{x\}\_\{1,2\}=\textbackslash{}frac\{-b\textbackslash{}pm \textbackslash{}sqrt\{\{b\}\textsuperscript{{2}}--4ac\}\}\{2a\}\$\$

An example of math within a paragraph --- \$\{e\}\textsuperscript{{i}\textbackslash{}pi \}+1=0\$b --- easy
enough.

And an equation on it's own:

\$\$\{x\}\_\{1,2\}=\textbackslash{}frac\{-b\textbackslash{}pm \textbackslash{}sqrt\{\{b\}\textsuperscript{{2}}--4ac\}\}\{2a\}\$\$b

\input{mmd-memoir-footer}

\end{document}


\documentclass[letter,10pt]{article}
\usepackage{TLCresume}
\begin{document}

%====================
% EXPERIENCE A
%====================
\subsection{{Application Development  \hfill May 2021 --- Present}}
\subtext{BTJN, LLC \hfill Houston, TX}
\begin{zitemize}
    \skilling{\href{https://docs.jnapolitano.io/parts/analysis/business-analysis/project-retail-sales-forecasting/code-base/SalesForcasting.html}{Sales Forecasting}}{Modeled retail sales with SARINA, Holt-Winter, and Exponential Smoothing models with Statsmodels API.}
    \skilling{\href{https://docs.jnapolitano.io/parts/data/web-scraping/index.html}{Data Mining}}{Developed Python application to mine 10,000 unfiltered sales leads weekly.}
    \skilling{\href{https://docs.jnapolitano.io/_modules/csv_to_neo.html\#DataPipelineFunctions}{Data Transformation}}{Transformed results to client's data format with PySpark.}
    \skilling{\href{https://docs.jnapolitano.io/_modules/csv_to_neo.html\#DataUploadFunctions}{Data Warehousing}}{Stored historical data in Neo4j warehouse.}
    \skilling{\href{https://docs.jnapolitano.io/parts/data/web-scraping/general-purpose/docs/realtor_scraper_sheets_4/index.html}{Data Integration}}{Integrated data into client’s existing Google Sheets workflow.}
\end{zitemize}

%====================
% EXPERIENCE B
%====================
\subsection{{Research Lead/Author  \hfill Jan 2017 --- Aug 2019}}
\subtext{University of Central Florida \hfill Orlando, Fl}
\begin{zitemize}
    \skilling{\href{https://www.degruyter.com/document/doi/10.1515/peps-2019-0036/html?lang=en}{Article Publication}}{Led team to publish work in the Journal of Peace Economics, Peace Science and Public Policy.}
    \skilling{Peer Review}{Prepared presentation for peer review at the Peace Science Conference at the Hague.}
    \skilling{\href{https://docs.jnapolitano.io/parts/analysis/political-analysis/terrorism-conflict/project-turkish-kurdish/pdf.html}{Feature Extraction}}{Standardized feature extraction and storage methodology.}
    \skilling{Data Integration}{Warehoused data to mongoDB.}
    \skilling{\href{https://docs.jnapolitano.io/parts/analysis/political-analysis/terrorism-conflict/project-turkish-kurdish/analysis.html}{Inferential Analysis}}{Identified socio-economic features that correlate with increased probability of violence.}
    \skilling{Data Visualization}{Visually conceptualized violent event, socio-economic records, and location data with Matplotlib.}
\end{zitemize}

%====================
% EXPERIENCE C
%====================
\subsection{{Research Assistant \hfill Jan 2017 --- Jan 2018}}
\subtext{University of Central Florida \hfill Orlando, Fl}
\begin{zitemize}
    \skilling{Sentiment Analysis}{Automated sentiment analysis of sentences with Python.}
    \skilling{Data Verification}{Verified user scored sentences against automated scoring with ANOVA testing.}
    \skilling{Data Management}{Standardized data aggregation and storage procedures.}
\end{zitemize}

\end{document}


% !TeX root = ../phd-1st-year-presentation.tex
% !TeX encoding = UTF-8
% !TeX spellcheck = en_GB

\section{A hybrid technique for MRP transient analysis}
  \begin{frame}{A hybrid technique for MRP transient analysis}
    Transient analysis of Markov Regenerative Processes (MRP)\\
    employing different techniques for different regenerative epochs

    \vspace{1em}
    The basics:
    \begin{itemize}
      \item Exact techniques require specific conditions to be met
      \begin{itemize}
        \item different techniques require different conditions
      \end{itemize}
      \item Kernel rows of different epochs can be evaluated independently
    \end{itemize}

    \vspace{1em}
    The idea:
    \begin{itemize}
      \item Evaluate each kernel row with a different technique
      \begin{itemize}
        \item corresponding to the condition enabled in that epoch
        \item eventually with an approximate technique, if no conditions are met
      \end{itemize}
      \item Compute transient probabilities with Markov Renewal Equations
    \end{itemize}
  \end{frame}

  \subsection{Techniques for MRP transient analysis}
    \begin{frame}{Techniques for MRP transient analysis}
      \begin{minipage}{0.6\textwidth}
        \textbf{Analysis under enabling restriction}\footnotemark
        \begin{itemize}
          \item at most one GEN enabled in each state
        \end{itemize}
      \end{minipage}
      \begin{minipage}{0.35\textwidth}
        \begin{center}\scalebox{0.45}{\input{img/pn_enabling_restriction}}\end{center}
      \end{minipage}

      \begin{minipage}{0.6\textwidth}
        \textbf{Analysis with stochastic state classes}\footnotemark
        \begin{itemize}
          \item a regeneration is always reached\\
            within a bounded number of events
          \begin{itemize}
            \item i.e. no cycles without regenerations
          \end{itemize}
          \item a.k.a. bounded regeneration
        \end{itemize}
      \end{minipage}
      \begin{minipage}{0.35\textwidth}
        \begin{center}\scalebox{0.45}{\input{img/pn_bounded_regeneration}}\end{center}
      \end{minipage}

      \begin{minipage}{0.6\textwidth}
        \textbf{Approximate analysis}
        \begin{itemize}
          \item usable when no conditions are met
        \end{itemize}
      \end{minipage}
      \begin{minipage}{0.35\textwidth}
        \begin{center}\scalebox{0.45}{\input{img/pn_no_restriction}}\end{center}
      \end{minipage}

      \addtocounter{footnote}{-1}
      \footnotetext{German, R., Logothetis, D., \& Trivedi, K. S. (1995, October). Transient analysis of Markov regenerative stochastic Petri nets: A comparison of approaches. In Petri Nets and Performance Models, 1995., Proceedings of the Sixth International Workshop on (pp. 103-112). IEEE.}
      \stepcounter{footnote}
      \footnotetext{Horváth, A., Paolieri, M., Ridi, L., \& Vicario, E. (2012). Transient analysis of non-Markovian models using stochastic state classes. Performance Evaluation, 69(7), 315-335.}
    \end{frame}

  \subsection{Classification of epochs}
    \begin{frame}{Classification of epochs}
      Through \textbf{non-deterministic analysis}
      \begin{itemize}
        \item State Class Graphs (SCG) are built
        \item for each regenerative epoch
      \end{itemize}

      \vspace{2em}
      By visiting each SCG, epochs are classified
      \begin{itemize}
        \item enabling restriction
        \begin{itemize}
          \item if at most one GEN is enabled in any state
        \end{itemize}
        \item bounded regeneration
        \begin{itemize}
          \item if no cycle is present
        \end{itemize}
      \end{itemize}
    \end{frame}

  \subsection{Iterative approximate technique}
    \begin{frame}{Iterative approximate technique}
      Based on analysis with stochastic state classes
      \begin{itemize}
        \item truncated after enough precision is met
      \end{itemize}

      \vspace{2em}
      Improvement with heuristics
      \begin{enumerate}
        \item expand at most $\nu_{start}$ nodes for non restricted epochs
        \item identify the truncated node $\Phi$ with highest reaching probability
        \begin{itemize}
          \item based on steady-state analysis of the embedded DTMC
        \end{itemize}
        \item expand at most $\nu_{iter}$ nodes from $\Phi$
        \item if at least $\nu_{max}$ nodes expanded, stop
        \begin{itemize}
          \item otherwise, return to step 2
        \end{itemize}
      \end{enumerate}
    \end{frame}


\documentclass[10pt]{beamer}
\usetheme{metropolis}
% all imports
\input{all_imports}

\AtBeginEnvironment{quote}{\singlespacing}

% new commands
\input{all_new_commands}

% definitions
\input{definitions/colors}
\input{definitions/styles}

\input{header}

\begin{document}

\maketitle

\begin{frame}{Research problem}
\begin{itemize}
\item Create a set of tasks that incorporate logic reasoning to boost performance of the current dialog agents.
\item Perform a stress test in the existing \textit{neural network based end-to-end dialog systems}.
\item Integrate linguistic reasoning with visual references to create a new set of visual question answering (VQA) tasks.
\item Define new models to achieve better results in the tasks proposed above.
\end{itemize}
\end{frame}

\section{Background}

\begin{frame}{Neural network based language model}
We call \alert{language model} a probability distribution over sequences of tokens in a natural language.
\begin{equation}
P(x_1,x_2,x_3,x_4) = p
\end{equation}

Since \cite{Mikolov11}, we use a \alert{Recurrent Neural Network (RNN)} to estimate the probability distribution   \\

\begin{equation}
P(x_{n} = \text{word}_{j^{*}} | x_{1}, \dots ,x_{n-1})
\end{equation}

for any $(n-1)$-sequence of words $x_{1}, \dots ,x_{n-1}$.
\end{frame}

\begin{frame}{Neural network based language model}
\input{tikzfiles/LanguageModelUnfolded}
\end{frame}

\begin{frame}{GRU: Gated Recurrent Units}
\begin{equation}
\vect{\widetilde{h}}^{(t)} = tahn(\vect{W} (\vect{h}^{(t-1)} \odot  \vect{r}^{(t)}) + \vect{U} \vect{x}^{(t)} + \vect{b})
\end{equation}

\begin{equation}
\vect{r}^{(t)} = \sigma(\vect{W}_{r} \vect{h}^{(t-1)} + \vect{U}_{r} \vect{x}^{(t)} + \vect{b}_{r})
\end{equation}

\begin{equation}
\vect{u}^{(t)} = \sigma(\vect{W}_{u} \vect{h}^{(t-1)} + \vect{U}_{u} \vect{x}^{(t)} + \vect{b}_{u})
\end{equation}

\begin{equation}
\vect{h}^{(t)} = \vect{u}^{(t)} \odot \vect{\widetilde{h}}^{(t)} + (1 - \vect{u}^{(t)}) \odot \vect{h}^{(t-1)}
\end{equation}

\end{frame}

\begin{frame}{LSTM: Long Short Term Memory}
\begin{equation}
\vect{f}^{(t)} = \sigma(\vect{W}_{f} \vect{h}^{(t-1)} + \vect{U}_{f} \vect{x}^{(t)} + \vect{b}_{f})
\end{equation}

\begin{equation}
\vect{i}^{(t)} = \sigma(\vect{W}_{i} \vect{h}^{(t-1)} + \vect{U}_{i} \vect{x}^{(t)} + \vect{b}_{i})
\end{equation}

\begin{equation}
\vect{o}^{(t)} = \sigma(\vect{W}_{o} \vect{h}^{(t-1)} + \vect{U}_{o} \vect{x}^{(t)} + \vect{b}_{o})
\end{equation}

\begin{equation}
\tilde{\vect{c}}^{(t)} = tahn(\vect{W} \vect{h}^{(t-1)} + \vect{U} \vect{x}^{(t)} + \vect{b})
\end{equation}

\begin{equation}
\vect{c}^{(t)} = \vect{f}^{(t)} \odot \vect{c}^{(t-1)} + \vect{i}^{(t)} \odot \tilde{\vect{c}}^{(t)}
\end{equation}

\begin{equation}
\vect{h}^{(t)} = \vect{o}^{(t)} \odot tanh(\vect{c}^{(t)})
\end{equation}

\end{frame}

\begin{frame}{Sequence-to-sequence}
\begin{itemize}
\item $\vect{x}^{(1)}, \dots, \vect{x}^{(n)}$, source sentence
\item $\vect{y}^{(1)}, \dots, \vect{y}^{(m)}$, target sentence
\item $f_{enc}$ (the \textit{encoder}), a RNN
\item $f_{dec}$ (the \textit{encoder}), a language model
\end{itemize}

\begin{equation}
\vect{s} = f_{enc}(\vect{x}^{(n)}, \vect{h}^{(n-1)})
\end{equation}

\begin{equation}
\vect{\tilde{h}}^{(t)} = f_{dec}(\vect{y}^{(t)}, \vect{\tilde{h}}^{(t-1)})
\end{equation}

\begin{equation}
p(y_t | y_1, \dots, y_{t-1}, x_1, \dots, x_{n}) = softmax(\vect{W}_{s}  \vect{\tilde{h}}^{(t)} + \vect{b}_s)
\end{equation}

\end{frame}

\begin{frame}{Attention}

\begin{equation}
\vect{a}_{ts} = \frac{exp(score(\vect{\tilde{h}}^{(t)},\vect{h}^{(s)}))}{\sum_j exp(score(\vect{\tilde{h}}^{(t)},\vect{h}^{(j)}))}
\end{equation}

\begin{equation}
score(\vect{\tilde{h}}^{(t)},\vect{h}^{(s)}) = \begin{cases}
{\vect{\tilde{h}}^{(t)}} .^{\top} \vect{h}^{(s)}\\
\vect{\tilde{h}}^{(t)} .^{\top}\vect{W}_a \vect{h}^{(s)}\\
\vect{v}_a ^{\top}tahn(\vect{W}_a[\vect{\tilde{h}}^{(t)};\vect{h}^{(s)}])\\
\end{cases}
\end{equation}

\begin{equation}
\vect{c}^{(t)} = \sum_{s} \vect{a}_{ts}\vect{h}^{(s)}
\end{equation}

\begin{equation}
\vect{\tilde{h}}^{(t)}_{out} = tahn(\vect{W}_c[\vect{c}^{(t)};\vect{h}^{(t)}])
\end{equation}

\begin{equation}
p(y_t | y_1, \dots, y_{t-1}, x_1, \dots, x_{n}) = softmax(\vect{W}_{s}  \vect{\tilde{h}}^{(t)}_{out} + \vect{b}_s)
\end{equation}

\end{frame}

\begin{frame}{Attention general formulation}

\begin{equation}
f(\vect{Q},\vect{K}_i) = \begin{cases}
\vect{Q} .^{\top} \vect{K}_i\;\;\;\; \text{dot}\\
\vect{Q} .^{\top}\vect{W}_{a} \vect{K}_i\;\;\;\; \text{general}\\
\vect{W}_{a}[\vect{Q};\vect{K}_i] \;\;\;\; \text{concat}\\
\vect{v}_{a}.^{\top} tahn(\vect{W}_{a}\vect{Q} + \vect{U}_{a}\vect{K}_i)\;\;\;\; \text{perceptron}\\
\end{cases}
\end{equation}

% softmax figure
\begin{figure}[ht!]
\centering
\scalebox{1.1}{
\begin{tikzpicture}[auto]
\node[textonly] (logits) {$\begin{bmatrix}f(\vect{Q},\vect{K}_1)\\f(\vect{Q},\vect{K}_2)\\\vdots\\f(\vect{Q},\vect{K}_n)\end{bmatrix}$};
\node[textonly, right=60pt of logits] (softmax) {$\begin{bmatrix}a_1\\a_2\\\vdots\\a_n\end{bmatrix}$};
\path[tedge] (logits) edge node[above=1pt] {{\Large softmax}} (softmax);
\end{tikzpicture}
}
\end{figure}

\begin{equation}
Attention(\vect{Q},\vect{K}, \vect{V})= \sum_{i} \vect{a}_{i}\vect{V}_i
\end{equation}

\end{frame}

\section{Neural network based dialog systems}

\begin{frame}{Seq2seq applied to translation}
\input{tikzfiles/Translation}
\end{frame}

\begin{frame}{Seq2seq applied to dialog \cite{DBLP:journals/corr/VinyalsL15}}
\input{tikzfiles/seq2seq_dialog}
\end{frame}

\begin{frame}{MemNN}
\begin{itemize}
\item $U_1, \dots, U_n$ context
\item $q$ question
\item $a$ answer
\end{itemize}

We have $k = 1, \dots, K$ memory layers:

\begin{itemize}
\item $\{ {\vect{m}^{(k)}}_i\}$, memory vectors
\item $\vect{u}^{(k)}$, input vector
\item $\vect{p}^{(k)}$,  match between $\vect{u}^{(k)}$ and each $\vect{m}^{(k)}_i$
\item $\{ {\vect{c}^{(k)}}_i\}$, another representation of the context $U_1, ..., U_n$
\item $\vect{o}^{(k)}$, output.
\item $\vect{\hat{a}}= softmax(\vect{W}(\vect{o}^{K}))$, candidate answer
\end{itemize}
\end{frame}

\section{How to evaluate dialogs?}

\begin{frame}{Human evaluation  \cite{Lowe:2016}}
\begin{center}
\includegraphics[scale=0.2]{images/exemploEval1.png}
\end{center}

\begin{enumerate}
\item \alert{Adequacy}: the meaning equivalence between the generated and control sentence.
\item \alert{Fluency}: the syntactic correctness of the generated sequence.
\item \alert{Readability}: efficacy of the generated sentence in a particular context.
\end{enumerate}

\end{frame}

\begin{frame}{BLEU (bilingual evaluation understudy)}
\begin{equation}
P_n = \frac{\text{number of } n\text{-grams in both } \hat{y} \text{ and } y}{\text{number of } n\text{-grams appearing in } \hat{y}}
\end{equation}
\vspace{0.2cm}

\begin{equation}
BP=
\begin{cases}
1 & \text{if } len(\hat{y}) > len(y) \\
\exp\left( 1 - \frac{len(y)}{len(\hat{y})} \right) & \text{otherwise}
\end{cases}
\end{equation}
\vspace{0.2cm}

\begin{equation}
BLEU = BP \; \exp \left(\frac{1}{N}  \sum_{n=1}^{N} \log P_n \right)
\end{equation}
\end{frame}

\begin{frame}{METEOR (Metric for Evaluation of Translation with Explicit ORdering)}
\begin{equation}
P = \frac{\text{number of } \text{unigrams in both } \hat{y} \text{ and } y}{\text{number of } \text{unigrams appearing in } \hat{y}}
\end{equation}

\begin{equation}
R = \frac{\text{number of } \text{unigrams in both } \hat{y} \text{ and } y}{\text{number of } \text{unigrams appearing in } y}
\end{equation}

\begin{equation}
F_{mean} = \frac{10 P R}{R + 9P}
\end{equation}

\begin{equation}
METEOR = F_{mean} (1 - penalty)
\end{equation}
\end{frame}

\begin{frame}{ROUGE (Recall Oriented Understudy for Gisting Evaluation)}
\begin{equation}
P_{lcs} = \frac{lcs(\hat{y}, y)}{len(\hat{y})}
\end{equation}

\begin{equation}
R_{lcs} = \frac{lcs(\hat{y}, y)}{len(y)}
\end{equation}

\begin{equation}
ROUGE_L = \frac{(1 + \beta^2) P_{lcs} R_{lcs}}{R_{lcs} + \beta^{2}P_{lcs}}
\end{equation}

where $\beta$ is usually set to favour recal ($\beta = 1.2$).
\end{frame}

\begin{frame}{Problems \cite{LiuLSNCP16}}

\begin{table}[h]
\centering
\label{hownottable}
\begin{tabular}{|c|c|c|c|c|}
\hline
\cellcolor{blue!50} metric & \cellcolor{blue!50} Spearman & \cellcolor{blue!50} $p$-value & \cellcolor{blue!50} Pearson &  \cellcolor{blue!50} $p$-value \\ \hline
BLEU   & $0.34$   & $< 0.01$  & $0.14$  & $0.17$ \\ \hline
METEOR & $0.19$   & $0.06$    & $0.19$  & $0.05$ \\ \hline
ROUGE  & $0.12$   & $0.22$    & $0.1$   & $0.34$ \\ \hline
\end{tabular}
\caption{Correlation between automatic metrics and human judgments based on dialog generated on Twitter}
\end{table}

\begin{table}[h]
\centering
\label{hownottable}
\begin{tabular}{|c|c|c|c|c|}
\hline
\cellcolor{blue!50} metric & \cellcolor{blue!50} Spearman & \cellcolor{blue!50} $p$-value & \cellcolor{blue!50} Pearson &  \cellcolor{blue!50} $p$-value \\ \hline
BLEU & $0.12$   & $0.23$    & $0.11$  & $0.26$    \\ \hline
METEOR & $0.06$   & $0.53$    & $0.14$  & $0.16$     \\ \hline
ROUGE & $0.05$   & $0.59$    & $0.06$  & $0.53$    \\ \hline
\end{tabular}
\caption{Correlation between automatic metrics and human judgments based on dialog generated on Ubuntu}
\end{table}
\end{frame}

\section{Creating simplified tasks as tests}

\begin{frame}{bAbI \cite{WestonBCM15}}
One solution is to create a set of QA synthetic tasks to test different capabilities of a dialog agent.

\begin{center}
\includegraphics[scale=0.25]{images/babi.png}
\end{center}
\end{frame}

\begin{frame}{ParlAI \\ \url{https://github.com/facebookresearch/ParlAI}}

\begin{center}
\includegraphics[scale=0.84]{images/parlai.png}
\end{center}

"ParlAI (pronounced 'par-lay') is a framework for dialog AI research, implemented in Python.

Its goal is to provide researchers:

\begin{itemize}
\item a unified framework for sharing, training and testing dialog models
\item many popular datasets available all in one place, with the ability to multi-task over them
\item seamless integration of Amazon Mechanical Turk for data collection and human evaluation"
\end{itemize}

\end{frame}

\begin{frame}{Sanity check experiments}
\begin{center}
\includegraphics[scale=0.34]{images/comparative_results_babi1.png}
\end{center}
\end{frame}

\begin{frame}{Sanity check experiments}
\begin{center}
\includegraphics[scale=0.34]{images/comparative_results_babi2.png}
\end{center}
\end{frame}

\section{Entailment-QA}

\begin{frame}{bAbI: task 15}

\alert{Basic Deduction}

\begin{center}
\includegraphics[scale=0.28]{images/babi15.png}
\end{center}
\begin{quote}
\centering
$P^{1}$ are afraid of $Q^{1}$\\
$P^{2}$ are afraid of $Q^{2}$\\
$P^{3}$ are afraid of $Q^{3}$\\
$P^{4}$ are afraid of $Q^{4}$\\
$c^{1}$ is a $P^{1}$\\
$c^{2}$ is a $P^{2}$\\
$c^{3}$ is a $P^{3}$\\
$c^{4}$ is a $P^{4}$\\
What is $c^j$ afraid of? \alert{A: $Q^j$}\\
\end{quote}

\end{frame}

\begin{frame}{bAbI: task 16}

\alert{Basic Induction}

\begin{center}
\includegraphics[scale=0.28]{images/babi16.png}
\end{center}
\begin{quote}
\centering
$c^{1}$ is a $P^{1}$\\
$c^{1}$ is $C^{1}$\\
$c^{2}$ is a $P^{2}$\\
$c^{2}$ is $C^{2}$\\
$c^{3}$ is a $P^{3}$\\
$c^{3}$ is $C^{3}$\\
$c^{4}$ is a $P^{4}$\\
$c^{4}$ is $C^{4}$\\
$c$ is a $P^{j}$\\
What color is $c$? \alert{A: $C^j$}\\
\end{quote}

\end{frame}

\begin{frame}{Entailment-QA}

\begin{enumerate}
\item \textbf{Boolean Connectives}
\item[]
\item \textbf{First-Order Quantifiers}
\item[]
\item \textbf{Synonymy}
\item[]
\item \textbf{Antinomy}
\item[]
\item \textbf{Hypernymy}
\item[]
\item \textbf{Active/Passive voice}
\end{enumerate}
\end{frame}

\begin{frame}{Entailment-QA: task 1}
\begin{itemize}
\item \alert{Entailment} ($s_1$ implies $s_2$)
\begin{itemize}
\item $\underbrace{P^{1}a^1 \land \dots \land P^{n}a^n}_{s_1}, \underbrace{P^{j}a^j}_{s_2}$
\item $\underbrace{P^{j}a^j}_{s_1}, \underbrace{P^{1}a^1 \lor \dots \lor P^{n}a^n}_{s_2}$
\item $\underbrace{Pa}_{s_1}, \underbrace{\lnot \lnot Pa}_{s_2}$
\end{itemize}

\vspace{0.4cm}
\item \alert{Not entailment} ($s_1$ does not imply $s_2$)
\begin{itemize}
\item $\underbrace{P^{j}a^j}_{s_1}, \underbrace{P^{1}a^1 \land \dots \land P^{n}a^n}_{s_2}$
\item $\underbrace{P^{1}a^1 \lor \dots \lor P^{n}a^n}_{s_1}, \underbrace{P^{j}a^j}_{s_2}$
\item $\underbrace{Pa}_{s_1}, \underbrace{\lnot Pa}_{s_2}$
\end{itemize}
\end{itemize}
\end{frame}

\begin{frame}{Entailment-QA: task 1}
\begin{itemize}
\item[] Ashley is fit
\item[] Ashley is not fit
\item[] The first sentence implies the second sentence? \alert{A: no}
\end{itemize}

\vspace{0.3cm}

\begin{itemize}
\item[]Avery is nice and Avery is obedient
\item[]Avery is nice
\item[]The first sentence implies the second sentence? \alert{A: yes}
\end{itemize}

\vspace{0.3cm}

\begin{itemize}
\item[]Elbert is handsome or Elbert is long
\item[]Elbert is handsome
\item[]The first sentence implies the second sentence? \alert{A: no}
\end{itemize}
\end{frame}

\begin{frame}{Entailment-QA: task 2}

\begin{itemize}
\item \alert{Entailment}
\begin{itemize}
\item $\forall x Px, Pa$
\item $Pa, \exists x Px$
\end{itemize}
\item []
\item \alert{Contradiction}
\begin{itemize}
\item $\forall x Px, \lnot Pa$
\item $\forall x Px, \exists x \lnot Px$
\end{itemize}
\item []
\item \alert{Neutral}
\begin{itemize}
\item $Pa,Qa$
\item $\forall x Px, \lnot Qa$
\end{itemize}
\end{itemize}
\end{frame}

\begin{frame}{Entailment-QA: task 2}

\begin{itemize}
\item[] Every person is lively
\item[] Belden is lively
\item[] What is the semantic relation? \alert{A: entailment}
\end{itemize}

\begin{itemize}
\item[] Every person is short
\item[] There is one person that is not short
\item[] What is the semantic relation?  \alert{A: contradiction}
\end{itemize}

\begin{itemize}
\item[] Every person is beautiful
\item[] Abilene is not blue
\item[] What is the semantic relation? \alert{A: neutral}
\end{itemize}
\end{frame}

\begin{frame}{Entailment-QA: task proxy}

SICK (Sentences Involving Compositional Knowledge) \cite{Marelli14}

\begin{center}
\includegraphics[scale=0.25]{images/sick.png}
\end{center}

\end{frame}

\begin{frame}{Entailment-QA: task proxy}

\begin{itemize}
\item[] There is no dog leaping in the air
\item[] A dog is leaping high in the air and another is watching
\item[] What is the semantic relation? \alert{A: contradiction}
\end{itemize}

\begin{itemize}
\item[] A man is exercising
\item[] A baby is laughing
\item[] What is the semantic relation? \alert{A: neutral}
\end{itemize}

\begin{itemize}
\item[] Some dogs are playing in a river
\item[] Some dogs are playing in a stream
\item[] What is the semantic relation? \alert{A: entailment}
\end{itemize}
\end{frame}

\begin{frame}{Preliminary Results}
\begin{center}
\includegraphics[scale=0.42]{images/comparative_results.png}
\end{center}
\end{frame}

\begin{frame}{Preliminary Results}
\begin{center}
\includegraphics[scale=0.28]{images/training_acc_EntailQA_mem.png}
\end{center}
\end{frame}

\begin{frame}{Preliminary Results}
\begin{center}
\includegraphics[scale=0.42]{images/cm_mem_EntailQA2.png}
\end{center}
\end{frame}

\begin{frame}{Future Steps}
\begin{itemize}
\item Try to overcome the reported overfitting problem.
\item Finish the Entailment-QA corpus.
\item Explore new models not mentioned here, like \alert{Dynamic Memory Networks} \cite{KumarISBEPOGS15} and  \alert{Memory Attention and Composition (MAC) cell} \cite{Manning18}.
\item Create a visual version of the Entailment-QA to test logical inference with images.
\item Check the reinforcement learning on dialog.
\item Review the literature on the theory of comparing models \cite{BenavoliCDZ17}.
\end{itemize}
\end{frame}

\begin{frame}{Schedule}
\begin{center}
\includegraphics[scale=0.24]{images/workplan.png}
\end{center}
\end{frame}

\begin{frame}[allowframebreaks]{References}

  \bibliography{my_references}
  \bibliographystyle{abbrv}

\end{frame}

\end{document}

\end{document}


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\usepackage{amsmath}
\allowdisplaybreaks

\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}

\begin{document}
\title{Notes}
\date{February 19, 2014}
\maketitle

\section*{leftover}
sine integral transform (evaluating integrals explicitly)

find:
\begin{align*}
  I&=\int_0^\infty{\frac{\sin(x\omega }{\omega }e^{-\alpha ^2t\omega ^2}\,\mathrm{d}\omega },& &&x&>0\\
  \intertext{convert to}
  I=I(\beta )&=\int_0^\infty{\frac{\sin(\beta s}{s}e^{-s^2}\,\mathrm{d}s}&&&\text{write }\beta =\frac{x}{\alpha \sqrt{t}}&>0\\
  \intertext{note}
  I(\beta )&=\int_0^\infty{\frac{\sin(s)}{s}e^{-s^2/\beta ^2}\,\mathrm{d}s}
  &\to&& 0 \text{ as }\beta &\to 0\\
  &&\to&&\frac{\pi }{2}\text{ as }\beta &\to+\infty\\
  \intertext{end note}
  I'(\beta )&=\frac{\mathrm{d}}{\mathrm{d}\beta }\int_0^\infty{\cos(\beta s)e^{-s^2}\,\mathrm{d}s}&u&=e^{-s^2}&\mathrm{d}v&=\cos(\beta s)\mathrm{d}s\\
  &&\mathrm{d}u&=-2se^{-s^2}\mathrm{d}s&v&=\frac{1}{\beta }\sin(\beta s)\\
  I'(\beta )&=\left.e^{-s^2}\frac{1}{\beta }\sin(\beta s)\right\rvert_0^\infty-\int_0^\infty{\frac{\sin(\beta s)}{\beta }(-2se^{-s^2})\,\mathrm{d}s}\\
  &=+\frac{2}{\beta }\int_0^\infty{\sin(\beta s)se^{-s^2}\,\mathrm{d}s}\\
  &=\frac{2}{\beta }\left(-I''(\beta )\right)\\
  I''(\beta )&=\int_0^\infty{\sin(\beta s)se^{-s^2}\,\mathrm{d}s}=-\frac{\beta }{2}I'(\beta )\\
  I'(\beta )&=c_1e^{-\beta^2 /4}\\
  I(\beta )&=c_2-c_1\int_\beta^\infty{e^{t^2/4}\,\mathrm{d}t}\text{ note the integration starting at }\beta \\
  &=c_2-0\\
  &=\frac{\pi }{2}-c_1\int_\beta^\infty{e^{-t^2/4}\,\mathrm{d}t}\\
  I(0)&=\frac{\pi }{2}-c_1\int_0^\infty{e^{-t^2/4}\,\mathrm{d}t}\\
  \intertext{fact $\int_0^\infty{e^{-x^2}\,\mathrm{d}x}=\frac{\sqrt{\pi }}{2}$}
  \frac{1}{2}\int_0^\infty{e^{-t^4}\,\mathrm{d}t}&=\frac{\sqrt{\pi }}{2}\\
  I(\beta )&=\frac{\pi }{2}-\frac{\sqrt{\pi }}{2}\int_x^\infty{e^{-x^2/4}\,\mathrm{d}x}\\
  \intertext{note error function (erf)}
  \text{erf}(x)&=\frac{2}{\sqrt{\pi }}\int_0^\infty{e^{-t^2}\,\mathrm{d}t}\to1 \text{ as }x\to\infty\\
  \text{erfc}(x)&=1-\text{erf}(x)=\frac{2}{\sqrt{\pi }}\int_x^\infty{e^{-t^2}\,\mathrm{d}t}\\
  \intertext{graph on page 79}
  \intertext{end note}
  x&=2t&\mathrm{d}x&=2\mathrm{d}t\\
  0&=\frac{\pi }{2}-c_2\sqrt{\pi }&\frac{\sqrt{\pi }}{2}&=c_1\\
  I(\beta )&=\frac{\pi }{2}-\frac{\sqrt{\pi} }{2}\int_{t/2\cdot2t?}^\infty{2e^{-t^2}\,\mathrm{d}t}\\
  &=\frac{\pi }{2}-\frac{\sqrt{\pi }}{2}\cdot2\cdot\frac{\sqrt{\pi }}{2}\left(\frac{2}{\sqrt{\pi }}\int_{2t}^\infty{e^{-u^2}\,\mathrm{d}u}\right)\\
  &=\frac{\pi }{2}-\frac{\pi }{2}\text{erfc}\left(\frac{\beta }{2}\right)\\
  \intertext{solution on p79}
  u(x,t)&=A\text{erfc}\left(\frac{x}{2\alpha \sqrt{t}}\right)
\end{align*}
\section*{last homework problem (hw09)}
we can do this without paying attention to formula's at all because the idea is so simple.
\begin{align*}
  u_x(0,t)&=0=f(t)\\
  u_x(1,t)+hu(1,t)&=1=g(t)
\end{align*}
introduce $u(x,t)=\omega (x,t)+\text{adjustment}$. This adjustment is chosen to obtain hetorgeneous boundary conditions ($f(t)=g(t)=0$). Take adjustment to be $+a(t)+bt)x$ because original boundary values (0 and 1) lie on a line.
\begin{align*}
  u&=\omega +a(t)+b(t)x\\
  \omega_x(0,t)+b(t)+0&=f(t)\\
  (\omega_x(1,t)+b(t))+h(\omega (1,t)+a(t)+b(t)\cdot1)&=g(t)
\end{align*}
\end{document}


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[super,square]{natbib}
\usepackage{tabularx}
\usepackage{parskip}
\usepackage[margin=1.4in]{geometry}
\usepackage{csquotes}
\usepackage{mathrsfs}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{hyperref}
\usepackage{graphicx}
\usepackage{float}
\usepackage{mdframed}
\usepackage[dvipsnames]{xcolor}
\usepackage{subcaption}

% Book headers
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\fancyhead[L]{\rightmark}
\fancyhead[R]{\thepage}
\renewcommand{\headrulewidth}{0pt}

\definecolor{blueish}{HTML}{CAC8FA}

\newcommand{\comment}[1]{}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{corollary}{Corollary}[theorem]
\newtheorem{proposition}{Proposition}[theorem]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{identity}[theorem]{Identity}

\theoremstyle{definition}
\newtheorem{defn}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\newenvironment{definition}
  {\vspace{8pt}\begin{mdframed}[backgroundcolor=blueish,innertopmargin=4]\begin{defn}}
  {\end{defn}\end{mdframed}\vspace{4pt}}

\title{\vspace{-3cm} Fiber Bundles, Gauges, and Connections}
\author{}
\date{}

\begin{document}
\maketitle
\vspace{-1.5cm}
\tableofcontents
\newpage

\section{Bundles}
\subsection{Fiber Bundles}

    % \begin{figure}[h]
    % \begin{subfigure}{0.5\textwidth}
    %     \includegraphics[width=8cm]{fiber-bundle.png}
    % \end{subfigure}

    % \begin{subfigure}{0.5\textwidth}
    %     \includegraphics[width=5cm]{fiber-bundle-mobius.png}
    % \end{subfigure}
    % \end{figure}

    A fiber bundle makes precise the idea of one topological space (called a fiber) being ``parameterized'' by another topological space (called a base). A fiber bundle also comes with a group action on the fiber. This group action represents the different ways the fiber can be viewed as equivalent. Formally, a fibre bundle is a structure ${\displaystyle (E,\,B,\,\pi ,\,F)}$. The topological space $E$ is known as the \textit{total space} of the fibre bundle, $B$ as the \textit{base space}, and $F$ the standard or template \textit{fiber}. The map ${\displaystyle \pi :E\rightarrow B}$, called the \textit{projection map} or \textit{submersion} of the bundle, is a continuous surjection satisfying a \textit{local triviality condition}. This condition enables a local section of a manifold to be interpreted as a trivial i.e. as cartesian product space ($B\times F$), despite global topology possibly being more complicated, i.e. twisted or non-orientable.

    For any $p \in B$, the pre-image ${\displaystyle \pi ^{-1}(\{p\})}$ is homeomorphic to $F$ and is called a \textit{fiber} over $p$. Recall, a homeomorphism is a kind of topological isomorphism, i.e. it is a continuous bijective (invertible) function between topological spaces. Every fibre bundle ${\displaystyle \pi :E\rightarrow B}$ is an open map, since projections of products are open maps. Specifically, we require that for every $p \in B$, there is an open neighborhood ${\displaystyle U\subset B}$ of $p$ (a trivializing neighborhood) such that there is a homeomorphism ${\displaystyle \varphi :\pi ^{-1}(U)\rightarrow U\times F}$ (where ${\displaystyle U\times F}$ is the product space) in such a way that $\pi$ agrees with the projection onto the first factor.

    % Therefore $B$ carries the quotient topology determined by the map $\pi$.

    % the fiber at each point of the base space consists of possible coordinate bases for use when describing the values of objects at that poin

    The canonical example of a nontrivial bundle $E$ is the Möbius strip. It has the circle that runs lengthwise along the center of the strip as a base $B$ and a line segment for the fiber $F$. A neighborhood $U$ of ${\displaystyle \pi (x)\in B}$ (where $x \in E$) is an arc. The preimage $\pi ^{-1}(U)$ is a partially twisted slice of the strip four squares wide and one long. A homeomorphism $\varphi$ exists that maps the preimage of $U$ to a slice of a cylinder: curved, but not twisted. This pair locally trivializes the strip, with the corresponding trivial bundle $\displaystyle B\times F$ being a cylinder whereas the Möbius strip has an overall twist that is only visible globally.

    A \textit{section} (or cross section) of a fiber bundle $E$ is a continuous right inverse of the projection function $\pi$. In other words, if $E$ is a fiber bundle over a base space, $B$, then a section of that fiber bundle is a continuous map, ${\displaystyle \sigma \colon B\to E}$ such that $\pi (\sigma (x))=x$ for all ${\displaystyle x\in B}$.

    Additional structures on $F$ give rise to special types of fiber bundles, e.g. vector bundles or group bundles.  Associated bundles allow derivation of bundles in which the typical fiber of a bundle changes from $F_{1}$ to $F_{2}$, which are both topological spaces with a group action of $G$, e.g. adjoint bundles, frame bundles, determinant bundles, dual bundles.

\subsection{Vector Bundles}

    If $F = V$ is a vector space, one defines a \textit{vector bundle} with standard fiber $V$ to be a fiber bundle $\pi : E \rightarrow B$ where all fibers $\pi^{-1} (b)$ are vector spaces and the local trivializations $\phi_\alpha$ can be chosen to be fiberwise linear. A homomorphism of two vector bundles is a fiber bundle homomorphism that is fiberwise linear. The fibered product of vector bundles $E_1; E_2$ is a vector bundle (also called Whitney sum and denoted $E1 \oplus E2$).

\subsection{Group Bundle}

    Recall, a Lie group is a group that is also a differentiable manifold where points can be multiplied together, they have inverses, and these operations are defined to be smooth (differentiable). If $F = G$ has the structure of a Lie group, one defines a \textit{group bundle}, $\mathcal G \rightarrow B$ with standard fiber $G$ to be a fiber bundle where all fibers carry group structures and the local trivializations can be chosen to be a fiberwise group homomorphisms. A group bundle homomorphism is a fiber bundle homomorphism which is fiberwise a group homomorphism. The fibered product of group bundles is a group bundle. One has natural bundle maps $\mathcal G \times^B \mathcal G \rightarrow \mathcal G$ (fiberwise group multiplication) and $\mathcal G \rightarrow \mathcal G $ (fiberwise inversion). Similarly, one defines algebra bundles, Lie algebra bundle as well as fiberwise linear actions of group or algebra bundles on vector bundle

\subsection{Principal Bundle}
     A principal $G$-bundle (also called a G-torsor over X) share similar properties to the resulting space of a Cartesian product of a space with a group. They are a fiber bundle $\pi: \mathcal P \rightarrow X$ together with a continuous right action $\mathcal P \times G \rightarrow \mathcal P$ such that $G$ preserves the fibers of $\mathcal P$ (i.e. if $y \in \mathcal P_x$ then $yg \in \mathcal P_x$ for all $g \in G$) and acts freely and transitively (i.e. regularly) on them in such a way that for each $x\in X$ and $y \in \mathcal P_x$, the map $G \rightarrow \mathcal P_x$ sending $g$ to $yg$ is a homeomorphism. In particular each fiber of the bundle is homeomorphic to the group $G$ itself. One can also define principal $G$-bundles in the category of smooth manifolds. Here $\pi : \mathcal P \rightarrow X$ is required to be a smooth map between smooth manifolds, $G$ is required to be a Lie group, and the corresponding action on $\mathcal P$ should be smooth.

\subsubsection{Associated Bundles}
    Recall, associated bundles allow derivation of bundles in which the typical fiber of a bundle changes from $F_{1}$ to $F_{2}$, which are both topological spaces with a group action of $G$, e.g. adjoint bundles, frame bundles, determinant bundles, dual bundles.

    Let $\pi : \mathcal P \rightarrow B$ be a principal $G$-bundle. Given a
    $G$-manifold $F$, one defines the associated fiber bundle by
    \[
    F (\mathcal P) \equiv \mathcal P \times_G F := (\mathcal P \times F) / G:
    \]
    The space $\mathcal P \times_G F$ is a fiber bundle over $B = \mathcal P=G$ with standard fiber $F$. The sections  $\Gamma^\infty(\mathcal B, \mathcal P \times_G F )$ of this fiber bundle are naturally identified with the space $C^\infty (\mathcal P, F )^G$ of equivariant maps $\mathcal P \rightarrow F$.

\subsubsection{Adjoint Bundle}

    An adjoint bundle is a vector bundle naturally associated to any principal bundle. The fibers of the adjoint bundle carry a Lie algebra structure making the adjoint bundle into a (nonassociative) algebra bundle. If $V$ is a vector space on which $G$ acts linearly, then $P \times_G V$ is a vector bundle. Taking $V = g$ with the adjoint representation one obtains the adjoint bundle $\mathfrak g(\mathcal P ) := \mathcal P \times_G \mathfrak g$.

    If $K$ is a Lie group on which $G$ acts by automorphisms, $\mathcal P \times_G K$ is a group bundle. Taking $K = G$ with $G$ acting by the adjoint action, one obtains a group bundle $G( \mathcal P) := \mathcal P \times_G G$ which is also called the adjoint bundle. It has $\mathfrak g( \mathcal P)$ as its Lie algebra bundle.

\subsection{Fibrations}
    Fibrations do not necessarily have the local Cartesian product structure that defines the more restricted fiber bundle case, but something weaker that still allows ``sideways'' movement from fiber to fiber. A fibration is like a fiber bundle, except that the fibers need not be the same space, nor even homeomorphic; rather, they are just homotopy equivalent.  Recall, homotopy equivalent implies that if $X$ and $Y$ are a pair of continuous maps $f : X \rightarrow Y$ and $g : Y \rightarrow X$, such that $g \circ f$ is homotopic to the identity map $id_X$ and $f \circ g$ is homotopic to $id_Y$. Intuitively, two spaces $X$ and $Y$ are homotopy equivalent if they can be transformed into one another by bending, shrinking and expanding operations

    A fibration satisfies an additional condition (the homotopy lifting property) guaranteeing that it will behave like a fiber bundle from the point of view of homotopy theory. Weak fibrations discard even this equivalence for a more technical property. Every vector bundle is a fiber bundle with a fiber homotopy equivalent to a point.  Fibrations are dual to cofibrations, with a correspondingly dual notion of the homotopy extension property; this is loosely known as Eckmann–Hilton duality.

\section{Connections}
    The notion of a \textit{connection} defines the idea of transporting data along a curve or family of curves in a parallel and consistent manner. A \textit{covariant derivative} is a linear differential operator which takes the directional derivative of a section of a vector bundle in a covariant manner. It also allows one to formulate a notion of a parallel section of a bundle in the direction of a vector: a \textit{section} $s$ is parallel along a vector $X$ if $\nabla _{X}s=0$.

\subsection{Ehresmann and Principal Connections}
    An \textit{Ehresmann connection} is a connection in a fibre bundle or a principal bundle made by specifying the allowed directions of motion of the field. Specifically, it singles out a vector subspace of each tangent space to the total space of the fiber bundle, called the horizontal space. A section $s$ is then horizontal (i.e., parallel) in the direction $X$ if $\rm {d}s(X)$ lies in a horizontal space.

    For any fiber bundle $\pi : E \rightarrow B$ the tangent bundle $T E$ of the total space has a distinguished subbundle, the vertical bundle $V E \hookrightarrow T E$. The fiber $V_xE$ for $\pi(x) = b$ is the image of $T_x(F_b)$ under the natural inclusion $T F_b \hookrightarrow T E$. An Ehresmann connection on $E$ is the choice of a complementary horizontal subbundle $HE$ such that $T E = V E \oplus HE$. Equivalently, a connection is a bundle projection $T E \rightarrow V E$ which is left-inverse to the inclusion $V E \rightarrow T E$; one defines $HE$ as the kernel of this projection.

    The Ehresmann connection has the immediate benefit of being definable on a much broader class of structures than vector bundles and is well-defined on a general fiber bundle. Many of the features of the covariant derivative still remain: parallel transport, curvature, and holonomy. With the classical covariant derivatives, covariance is an a posteriori feature of the derivative. However, for an Ehresmann connection, it is possible to impose a generalized covariance principle from the beginning by introducing a Lie group acting on the fibers of the fiber bundle. The appropriate condition is to require that the horizontal spaces be equivariant with respect to the group action.

    If the standard object $F$ has additional structure, one is interested in connections such that parallel transport preserves that structure. For example, if $E$ is a vector bundle, each parallel transport operation, $\Pi^\gamma$, should be a linear map, and for group bundles it should be a fiberwise group homomorphism, and so on.
    An important special case of Ehresmann connections are principal connections on principal bundles, which are required to be equivariant in the principal Lie group action.   A principal connection on a fiber bundle is an equivariant Lie algebra valued 1-form $\theta \in \Gamma^1 (\mathcal P, \mathfrak g)^G$ such that $\iota(\xi_{\mathcal P})\theta = \xi$ (where $\iota$ is an inclusion embedding) for all $\xi \in \mathfrak g$, the Lie algebra.

    The space of principal connections will be denoted $\mathcal A(P)$. The space $\mathcal A(P)$  has a natural affine structure, with underlying vector space the space $\Gamma^1 (B, \mathfrak g( \mathcal P))$ of 1-forms on $B$ with values in the adjoint bundle.

\subsection{Cartan Connections}
    TODO

\section{Gauges}

    A gauge can be thought of as a coordinate system that varies depending on one’s location with respect to some base space or parameter space. A gauge transform is a change of coordinates applied to each such location, and a gauge theory is a model for some physical or mathematical system to which gauge transforms can be applied and is typically gauge invariant, in that all physically meaningful quantities are left unchanged or transform naturally under gauge. A principal bundle automorphism is a $G$-equivariant diffeomorphism $\phi : \mathcal P \rightarrow \mathcal P$ taking fibers to fibers. The group of principal bundle automorphisms will be denoted $\hbox{Aut}( \mathcal P)$.

    The space of "coordinate systems" is (non-canonically) identifiable with the isomorphism group $\hbox{Isom}(G)$ of template $G$.  This isomorphism group is called the structure group or gauge group of the class of geometric objects. The gauge group $\hbox{Gau}( \mathcal P) \subseteq \hbox{Aut}(\mathcal P)$ consists of automorphisms $ \phi : \mathcal P \rightarrow \mathcal P$ inducing the identity map on the base $B$. That is, $\hbox{Gau}( \mathcal P)$ is defned by an \textit{exact sequence} of groups
    \[
        1 \longrightarrow \hbox{Gau}( \mathcal P) \longrightarrow \hbox{Aut}( \mathcal P) \longrightarrow \hbox{Diff} B)
    \]
    Let $\theta$ be a principal connection on $\pi : \mathcal P \rightarrow B$. For any path $\gamma : [t_0, t_1] \rightarrow B$, let
    \[
        \Pi^\theta_\gamma : P_{\gamma(t_0)} \rightarrow P_{(t_1)}
    \]
    denote parallel transport with respect to $\theta$. For all $\phi \in \hbox{Gau}(\mathcal P)$,
    \[
        \Pi_{\gamma}^{\phi,\theta} = \phi(\gamma(t_1)) \circ  \Pi_{\gamma}^{\theta} \circ  \phi(\gamma(t_0))^{-1}
    \]
    The group of automorphisms $\hbox{Aut}( \mathcal P)$ acts on the space $\mathcal A( \mathcal P)$ of principal connections by pull-back by the inverse. This can be understood as the gauge transformations of connections. We can interpret $\mathcal A( \mathcal P)$ as an infinite dimensional manifold, equipped with an action of an infinite-dimensional Lie Group.

\section{Moduli Spaces}
    \subsection{Review of differential forms}
    Recall from differential geometry notebook \footnote{\url{https://github.com/lukepereira/notebooks}} the definitions of differential forms, wedge products and the hodge star operations:

        \begin{itemize}
            \item A differential \textit{$k$-form }on an open subset $U \subseteq \mathbb R^m$ is an expression of the form
            \[
                \omega = \sum_{i_1 \dots i_k} \omega i_1\dots i_k dx^{i_1} \wedge \dots \wedge  dx^{i_k}
            \]
            where $\omega_{i_1\dots i_k} \in C^\infty(U)$ are functions, and the indices are numbers $1 \leq i_1 < \dots < i_k \leq m$. The symbol $\wedge$ denotes the exterior product of two differential forms.

            \item The \textit{exterior product} or \textit{wedge product} is the product operator in an exterior algebra. If $\alpha$ and $\beta$ are differential $k$-forms of degrees $p$ and $q$, respectively, then
            \[
                \alpha \wedge \beta=(-1)^{pq} \beta \wedge \alpha.
            \]
            It is not (in general) commutative, but it is associative, and bilinear.

            Let $\alpha, \beta \in \Omega^1(M)$. Then we define a wedge product $\alpha \wedge \beta \in \Omega^2 (M)$, as follows:
            \[
                (\alpha \wedge \beta)(X,Y) = \alpha (X)\beta(Y)-\alpha(Y)\beta(X).
            \]

            \item Let $V$ be an n-dimensional vector space with basis $\{ e_1, \cdots, e_n \}$ and with unit vector given by $\omega := e_1 \wedge \cdots \wedge e_n$ . Note, the dual of $\omega$ is the volume form, $\hbox{det}$.

            An inner product $\langle \cdot,\cdot \rangle$ induces pairs of k-vectors $\alpha, \beta \wedge^k V$ and has the Gram determinant:
            \[
            {\displaystyle \langle \alpha ,\beta \rangle =\det \left(\left\langle \alpha _{i},\beta _{j}\right\rangle \right)_{i,j=1}^{k}}
            \]
            For all pairs of k-vectors, the \textit{Hodge Star operator} can be defined as having property,
            \[
                \alpha \wedge (* \beta ) = \langle \alpha, \beta \rangle \omega.
            \]
            Applying ${\displaystyle \det }$ to the above equation, we obtain the dual definition:
            \[
                {\displaystyle \det(\alpha \wedge {\star }\beta )=\langle \alpha ,\beta \rangle .}
            \]

        \end{itemize}

    \subsection{Moduli Space of Connections}
    The quotient space of the space of principal connections and the Gauge  $\mathcal A(P) / \hbox{Gau}(P)$ is called the moduli space of connections. It is still infnite-dimensional. To obtain a finite dimensional moduli spaces, one has to impose additional gauge-invariant constraints on $\theta$: e.g. that it is a flat connection, or more generally a Yang-Mills connection.

    \subsection{Moduli Space of Yang-Mills Connections}

    On a principal bundle, we want to choose a canonical connection so that curvature $F_\theta$ vanishes. But not every principal bundle can have a flat connection, and the best one can hope for is that the bundle has curvature as small as possible. On a bundle of connections, a connection is defined by its local forms ${\displaystyle \theta_{\alpha }\in \Omega ^{1}(U_{\alpha },\operatorname {ad} (P))}$. The Yang-Mills action functional $YM(\theta)$ is precisely the square of the ${\displaystyle L^{2}}$-norm of the curvature which has critical points, i.e. local minima, that minimize curvature called Yang-mills connections. These Yang-mills connections, $\mathcal M$, are a finite subset of the moduli space of connections that was originally sought after, i.e. $\mathcal M \subset \mathcal A(P) / \hbox{Gau}(P)$. This is more rigoursly described below. As a side note, in physics the gauge field strength is given by curvature of connections, $F^\theta$, and the energy of the gauge field is the Yang-mills functional.

    Suppose $\pi : \mathcal P \rightarrow B$ is a principal $G$-bundle over a compact, oriented, Riemannian manifold $B$. The inner product on $T B$ gives rise to an inner product on $T M$ and on all $ \wedge^k T^*M$. Taking the inner product of the differential form, followed by integration over $B$ with respect to the Riemannian volume form, defines an inner product on $\Omega^* (B)$. In terms of the Hodge star operator,
    \[
        \langle \alpha, \beta \rangle = \int_B \alpha \wedge  * \beta
    \]
    Let $|| \cdot ||$ be the norm corresponding to $\langle \cdot, \cdot \rangle$. The Yang-Mills functional on $\mathcal A(\mathcal P)$ is the functional
    \[
        \hbox{YM}(\theta) = ||F ||^2 = \int_B (F^\theta, * F^\theta)
    \]
    The Yang-Mills functional is invariant under the action of the gauge group, i.e. $YM(\theta) = YM(\phi,\theta)$, hence all its critical points (called Yang-Mills connections) are invariant as well. A connection $\theta$ is a critical point of the Yang-Mills functional if and only if it satisfies the Yang-Mills equation, $d^\theta * F^\theta = 0$. The quotient of the space of Yang-Mills connections by the action of the gauge group is called the Yang-Mills moduli space. In this context, the term ``moduli'' is used synonymously with ``parameter''.

    Moduli of Yang–Mills connections have been most studied when the dimension of the base manifold X is four. Here the Yang–Mills equations admit a simplification from a second-order PDE to a first-order PDE, known as the anti-self-duality equations. The Yang-Mills equations depend upon the Riemannian metric on $B$ only via the star operator on $\Gamma^2 (B)$. The case $\dim B = 4$ is special in that it takes $\Gamma^2 (B)$ to itself, since $4 - 2 = 2$. We mentioned already that in this case the Yang-Mills equations are conformally invariant: Multiplying the metric by a positive function does not change the star operator in middle dimension, hence does not change the Yang-Mills equations. A special type of Yang-Mills connections in $4$ dimensions are those satisfying one of the equations
    \[
        *F^\theta = F^\theta \text{\ \ or \ \  }  *F^\theta = - F^\theta
    \]
    (self-duality and anti-self-duality respectively) because for such connections, the Yang-Mills equations are a consequence of the Bianchy identity $d^\theta F^\theta = 0$. A change of orientation of $B$ changes the sign of the  operator, and therefore exchanges the notion of duality and anti-self duality. For certain principal bundles ($\theta$ is a multiple of the second Chern number), ASD connections give the absolute minimum of the Yang-Mills functional.

    % Anti-self dual connections over S4 are also called instantons.

    % The moduli space for anti-self dual YM-connections for G = SU(2) is the starting point for Donaldson theory of 4-manifolds. As realized by Donaldson, they contain information not only about the topology but also the differentiable structure of 4-manifolds.

\section{Examples}

% \subsection{Review: Lagrangian and Hamiltonian}
%     - The Lagrangian L = K - V is big when most of the energy is in kinetic form, and small when most of the energy is in potential form
%     - Lagrangian measures something we could vaguely refer to as the ‘activity’ or ‘liveliness’ of a system: the higher the kinetic energy the more lively the system, the higher the potential energy the less lively. So, we’re being told that nature likes to minimize the total of ‘liveliness’ over time: that is, the total action. (Principle of Least Action)

%     Hamiltonian:
%     - The simplest interpretation of the Hamilton equations is as follows, applying them to a one-dimensional system consisting of one particle of mass m under time-independent boundary conditions: The Hamiltonian H represents the energy of the system (provided that there are NO external forces, or additional energy added to the system), which is the sum of kinetic and potential energy, traditionally denoted T and V, respectively.
%     - e very careful about making the assumption that H = T + V. You should always default to the definition of H which is H = Sum\_i(qdot\_i * partial L/partial qdot\_i) - L

\subsection{Relativity}

    In gauge theories, physicists begin with a Lagrangian $L[\phi, \phi']$. The claim is that this $L$ is invariant under the action of some group. What is a bit tricky is that to make this statement a bit more precise requires that we have two fiber bundles at once, namely the principal-$G$ bundle and its associated vector bundle. For each patch of spacetime, $U_i$ where $M$ is base manifold, we pick a map $S:U_i \rightarrow G$ (this will later be the gauge group). Then we pick a certain representation of the group, i.e $\rho :G \rightarrow V$ where $V$ is a vector space. We now define what will later be a section $\psi : U_i \rightarrow[x,\phi]$ where $x$ is a point on the manifold. In this context, gauge invariance means that $[x,\phi] \sim [x,\rho (g^{-1})\phi]$.

    Recall, we have the spacetime patch $U_i$ with the map $S$. With this, we construct the cartesian product $U_i \times G$. If we happen to find two overlapping open sets $U_i$ and $U_j$ then for the sets of points in the intersection we have to make sure things are consistent and so we define functions $t_{ij} : U_i \cap U_j \rightarrow G$ that will act on $G$ i.e $(x,G) \rightarrow (x, t_{ij}(x)G)$. Doing this for the whole manifold $M$ gives us another manifold $P$ that is locally $U_i \times G$. This the principal-G bundle.

    Requiring local gauge invariance in the first step meets a snag. The problem involves the map $S$; as we go around on the manifold $M$ we need a method to go from one fiber to another fiber on the principal-G bundle. To do this requires we introduce a connection $\Omega$ on the principal-G bundle. But physicists always work on the base manifold, so we need to pull back $\Omega$ to the base manifold by some section $\sigma$ i.e calculate $\sigma^*\Omega \equiv A$. Since these are locally defined sections, when we are in intersection of two spacetime patches, $U_i \cap U_j$, we will have two sections, $\sigma,\sigma'$. This means we will get $\sigma^*\Omega=A$ and $\sigma'^*\Omega=A'$. The two sections are related by the map $S$ i.e $\sigma'=R_{S(x)}\sigma=\sigma_{S(x)}=\sigma g$. That is, the group acts by a right action.

    To calculate $\sigma^*\Omega$ we note that $\langle \sigma^*\Omega, v \rangle = \langle \Omega, \sigma_* v\rangle$ where $v$ is a vector on the principal bundle. A tricky calculation shows $\sigma'_*v = R_{g*}(\sigma_{*}v)+\eta_x(p)$ where $\eta X$ is a fundamental vector field, $X = \langle S^*\theta,v \rangle$ and $\theta$ is the Maurer-Cartan form and $g$ is the image of $S$. To show this works we do the calculation
    \begin{align*}
            (\sigma'^* \omega)(v) & = <\Omega,\sigma'v> \\
            &= <\Omega,R_{g*}(\sigma_*v)>+<\Omega,\eta_X(pg)\\
            &= <R^*_g\Omega,\sigma_*v> +X \\
            &= <Ad_{g^-1}\Omega,\sigma_*v>+X=<Ad_{g^{-1}}A + S^*\theta,v>
    \end{align*}
    This is the usual transformation rule for the gauge field on the base manifold. We can now state the fact that we have an associated bundle $\mathcal A$ which is $\mathcal{P}\times_G V = (\mathcal{P}\times V)/G$ and is locally $U_i \times V$ with sections as defined in the first step. The sections on this bundle are what physicists call the fields.

\begin{thebibliography}{}

\bibitem[]{}
Tao, T. What is a gauge? (2008).\\ \url{https://terrytao.wordpress.com/2008/09/27/what-is-a-gauge/}
% https://terrytao.wordpress.com/2008/09/27/what-is-a-gauge/

\bibitem[]{}
Eckhard Meinrenken, Principal bundles and connections.\\ \url{http://www.math.toronto.edu/mein/teaching/moduli.pdf}

\bibitem[]{}
Kobayaschi, S. Theory of connections. Annali di Matematica 43, 119–194 (1957).
% file:///home/luke/Downloads/Kobayaschi1957_Article_TheoryOfConnections.pdf

\end{thebibliography}

\end{document}


%\subsubsection{(TODO) The Carnot Engine, Revisited}
%Before moving onto the statistical definition of entropy, let's answer a question about Carnot engines that might have arose as we learned about entropy;
%\textit{How can Carnot Engines run reversibly if macroscopic processes are not truly reversible?}The answer is twofold. First, as we mentioned before the Carnot process is an idealization. It involves two isothermal steps, and as we've discussed no process is truly isothermal, unless it ran for an infinitely long time. Second, you need to be careful what system your considering when you say a process is reversible. The heat engine has zero change in entropy. However, when it interacts with its surroundings it inevitably changes the surrounding's entropy as well. Actually modelling this change in entropy would be fiendishly difficult \textbf{Rio Help I'm not sure this is right}, but we'd expect a high level of idealization to be necessary to actually make it zero, and otherwise for it to be positive. We could try to get around this problem by isolating the engine, but the efficiency of the engine is less than 100\%, so it needs a source of energy outside the system.

\subsubsection{(Optional) The Carnot Efficiency, Revisited}
To finish off this section, I want to show you a neat proof of Carnot's theorem; This theorem tells us that the maximum efficiency of a heat engine is given by $\eta = 1- \frac{T_C}{T_H}$ (the efficiency of the Carnot engine), where $T_C$ is the coldest temperature within the cycle, and $T_H$ is the hottest temperature within the cycle. We will explore in this section how this follows from the second law of thermodynamics. This section is completely optional, and just for your curiosity\footnote{Like seriously, this is completely out of the scope of Science One}, though you have all the tools you need to prove it yourself. To review how a heat engine fundamentally works, it takes in some heat $Q_H$ from a hot reservoir at temperature $T_H$, does some work $W$ with that energy, and then throws away waste heat $Q_C$ into a cold reservoir at temperature $T_C$, before returning to its initial state. This is pictured below:
\begin{center}
    \begin{tikzpicture}[scale=3]
    \filldraw[fill=pink] (0,0) rectangle (1,1);
    \draw (1.75,0.25) rectangle (2.25,0.75);
    \filldraw[fill=blue!50] (3,0) rectangle (4,1);
    \draw[->] (1.15,0.5) -- (1.6,0.5);
    \draw[->] (2.4,0.5) -- (2.85,0.5);
    \draw[->] (2,0.2) -- (2,0);
    \node[above] at (1.375,0.5) {$Q_H$};
    \node[above] at (2.625,0.5) {$Q_C$};
    \node[left] at (2,0.1) {$W$};
    \draw (2,0.5) node {Engine};
    \draw (0.5,0.5) node {Hot bath at $T_H$};
    \draw (3.5,0.5) node {Cold bath at $T_C$};=
    \end{tikzpicture}
\end{center}

Now, let's consider the entropy of the hot reservoir, the engine, and the cold reservoir at the end of one cycle (in other words, we consider the change in entropy of the universe after one cycle). We recall that entropy is a function of state, and therefore at the end of a single cycle, the entropy of the heat engine itself must be the same as when it started. That is, $\Delta S_{engine} = 0$ for one full cycle. By the second law of thermodynamics $dS \geq \frac{Q}{T}$, the entropy of the hot reservoir decreases by the amount of heat $Q_H$ divided by its temperature $T_H$, so the entropy of the hot reservoir decreases by $\Delta S_{hot} \geq \frac{-Q_H}{T_H}$. Conversely, the cold reservoir increases in one cycle by $\Delta S_{cold} \geq \frac{Q_C}{T_C}$ (as it receives $Q_C$ heat at temperature $T_C$. Putting these together, we obtain the change in entropy of the universe for a single cycle:
\[\Delta S_{universe} \geq \frac{Q_C}{T_C} - \frac{Q_H}{T_H} \]
Now, the second law of thermodynamics tells us that the entropy of the universe must increase (or, to phrase it another way, if we treat the two reservoirs and the heat engine as an isolated system, the entropy of the total system must increase. This allows us to conclude that:
\[\Delta S_{universe} \geq \frac{Q_C}{T_C} - \frac{Q_H}{T_H} \geq 0 \]
So we obtain the important inequality:
\[\frac{Q_C}{T_C} - \frac{Q_H}{T_H} \geq 0 \]
Which we can rearrange to obtain:
\begin{equation}
    \label{eqn:(37)}
    \frac{T_H}{T_C} \geq \frac{Q_C}{Q_H}
\end{equation}
Note that to derive inequality \ref{eqn:(37)}, I have made no assumptions whatsoever about what my heat engine actually looks like; this is a completely general statement, based on the amounts of heat gained/lost from the hot/cold reservoirs, and the maximum/minimum hot/cold reservoir temperatures. Now, let us consider out definition of efficiency:
\begin{equation}
    \label{eqn:(38)}
    \eta = \frac{W}{Q_H}
\end{equation}
Where $W$ is the work done by the engine (what we get out), and $Q_H$ is the heat that we inject into the engine in one cycle from the hot reservoir (what we put in). By energy conservation, we find that:
\begin{equation}
    \label{eqn:(39)}
    W = Q_H-Q_C
\end{equation}
This might look like it came out of nowhere, so let's think about it a bit further. Just like entropy, internal energy is also a function of state; the energy something has doesn't care about how that energy got there! With this consideration, since the heat engine returns to the original state at the end of one cycle, just like the entropy change of the heat engine is zero in a single cycle, so must be the total internal energy; in other words, the heat engine must have the same energy it began with. With this consideration, we realize that the sum of the work done on the system and the heat given to the system must be 0, leading to equation \ref{eqn:(39)} above (work this out using the first law of thermodynamics if it's still unclear!). Now, we can substitute equation \ref{eqn:(39)} into equation \ref{eqn:(38)}, giving us:
\begin{align*}
    \eta = \frac{Q_H-Q_C}{Q_H} = 1 - \frac{Q_C}{Q_H}
\end{align*}
And now applying inequality \ref{eqn:(37)}, we have:
\begin{align*}
    1 - \frac{Q_C}{Q_H} \leq 1 - \frac{T_H}{T_C}
\end{align*}
And therefore:
\begin{equation}
    \eta \leq 1 - \frac{T_H}{T_C}
\end{equation}
We have hence proven Carnot's theorem, and can see that for any arbitrary heat engine, the efficiency is bounded by the Carnot efficiency.


\section{Lévy's continuity theorem}


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%       SPECIAL RELATIVITY SECTION      %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Special Relativity}
\label{sec:specrel}
The special relativity subject matter in the qualifying exam is primarily at the level of \cite{ohanianModernPhysics1995}, a common, upper-level undergraduate modern physics text. The main innovation in special relativity is Einstein's second postulate:
\quotebox{
\begin{enumerate}[label=\Roman*.]
	\item Only the relative motion of inertial frames can be measured; the laws of physics are the same in all inertial reference frames. The concept of "absolute rest" is meaningless.
	\item The velocity of light is a universal constant, independent of any relative motion of the source and observer
\end{enumerate}}

\section{Lorentz Invariance}
% Galilean Invariance
Newtonian mechanics, which underlies all of classical mechanics, assumes a universal background space and time, such that the time is shared by all inertial reference frames. Thus a transformation from one Newtonian frame $K$ to another $K'$, separated by velocity $v$ in the $x_1$ dimension, called a \textit{Galilean} transformation takes the form $x_1'=x_1-vt$, $x_2'=x_2$, $x_3'=x_3$, $t'=t$. The fact that Newtonian mechanics is invariant under these transformations is called \textit{Galilean invariance}.\\
% Lorentz Transforms
\indent Galilean invariance isn't compatible with the second postulate of special relativity, so the symmetries of spacetime in special relativity must be different. The \textit{Lorentz transformation} can be derived from the second postulate. Keeping the speed of light $c$ constant, and computing the distance traveled by a pulse of light in two frames, $K$ and $K'$ separated by a boost of speed $v$ on the $x_1$-axis the resulting transformation is defined:
\eqn{\begin{split}
	x_1' &= \gamma\wrap{x_1 - vt} \\
	x_2' &= x_2 \\
	x_3' &= x_3 \\
	t' &= \gamma\wrap{t - x_1}
\end{split}\ \Vast{\}}
	\quad\quad\quad\quad
\gamma=\frac{1}{\sqrt{1-\wrap{\frac{v}{c}}^2}}
}
% Lorentz Group (boosts + rotations)
Any physical system preserved under Lorentz transformations is said to be \textit{Lorentz invariant}. Maxwell's equations of electrodynamics possess Lorentz invariance, and were a major source of motivation for the development of special relativity.

\section{Space-Time and Four-Vectors}
% 4-vectors
Since special relativity deals with transformations in 4-dimensional spacetime, it is helpful to introduce the concept more formally. We adopt the concept of \textit{four-vectors}, such as $x^\mu$ to represent a point in spacetime, where $x^0=ct$, $x^1=x$, $x^2=y$, $x^3=z$. We also define the square of the spacetime interval between two points to be an invariant:
\eqn{\Delta s^2 = -\wrap{c\Delta t}^2 + \wrap{\Delta x}^2 + \wrap{\Delta y}^2 + \wrap{\Delta z}^2}
We can also define the invariant in terms of the \textit{Minkowski metric} $\eta_{\mu\nu}$ which defines a flat Lorentzian manifold. Further, we adopt the \textit{Einstein summation notation} whereby the $\sum$ symbol is omitted in favor of matching a lower and upper index. For instance, computing the invariant interval for two points $x^\mu$ and $x'^\mu$, where $\Delta x^\mu = \wrap{x^0-x'^0, x^1-x'^1, x^2-x'^2, x^3-x'^3}$.
\eqn{\eta_{\mu\nu}=\begin{pmatrix}
	-1 & 0 & 0 & 0 \\
	0 & 1 & 0 & 0 \\
	0 & 0 & 1 & 0 \\
	0 & 0 & 0 & 1
\end{pmatrix}\quad\quad\quad
\Delta s^2=\eta_{\mu\nu}\wrap{\Delta x}^\mu\wrap{\Delta x}^\nu}
% Metric, Causality
At this point it's worth noting that Lorentz transformations $\Lambda$ are those that leave the metric unchanged $\Lambda \eta \Lambda^{T}$, which implies that $\Lambda^T = \Lambda^{-1}$ or that the Lorentz transformations are \textit{orthogonal}. We define causal relationships between two points $x^\mu$ and $x'^\mu$ in the following way:
\eqn{\begin{split}
	\Delta s^2 &< 0\quad \text{\textit{timelike}, possible causal dependence} \\
	\Delta s^2 &= 0\quad \text{\textit{lightlike}, boundary between causal dependence and independence} \\
	\Delta s^2 &> 0\quad \text{\textit{spacelike}, causal independence} \\
\end{split}}
For the case of lightlike separation, we define the \textit{proper time} $\Delta \tau$ to be the negative of the interval such that $\Delta \tau = - \Delta s$. For a parametric path $x^\mu(\lambda)$ in spacetime such that the velocity is timelike at every point, the proper time defines the time experienced by an observer moving along the path:
\eqn{\tau=\int_{\lambda_0}^{\lambda_1}\sqrt{-\eta_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}}d\lambda}

\section{Momentum and Energy}
In special relativity, we extend the relativistic 3-dimensional velocity $\vec{p}=\gamma m \vec{v}$ to be a four-vector, called the \textit{four momentum} $p^\mu$ by adding the \textit{relativistic energy} $E=\gamma mc^2$ as the time coordinate such that $p^\mu=\wrap{E/c, \vec{p}}$. The four-momentum is conserved, as is the relativistic momentum $\vec{p}$. The total energy $E=p^0 c$ is conserved as well. The kinetic energy can be defined:
\eqn{p^\mu = \wrap{\gamma mc, \gamma m v_x, \gamma m v_y, \gamma m v_z}}
\eqn{T=\sqrt{p^2 c^2 + m^2 c^4} - mc^2 = E - mc^2 = \wrap{\gamma - 1}mc^2}
Note that for $v << c$ the classical, Newtonian expressions are recovered for both momentum and kinetic energy. Using the expansion for $1/\sqrt{1-x^2}\approx 1 + x^2/2$, the Lorentz coefficient $\gamma$ can be expanded to $\gamma \approx 1 + v^2/2c^2$, therefore $E\approx mc^2 + mv^2/2$ and $p\approx mv$.

\section{Relativistic Effects}
% Length Contraction & Time dilation
Some common effects of special relativity are the shortening and lengthening of conventional Newtonian measurements due to changes in reference frame. Consider two frames $K$ and $K'$, where $K'$ is related to $K$ by a boost along the $x$ axis. The Lorentz transformation relating $K'$ to $K$ is given:
\eqn{\Lambda^{\mu'}{}_{\nu}=\begin{pmatrix}
	\cosh\phi & -\sinh\phi & 0 & 0 \\
	-\sinh\phi & \cosh\phi & 0 & 0 \\
	0 & 0 & 1 & 0 \\
	0 & 0 & 0 & 1 \\
\end{pmatrix}=\begin{pmatrix}
	\gamma & -\gamma\beta & 0 & 0 \\
	-\gamma\beta & \gamma & 0 & 0 \\
	0 & 0 & 1 & 0 \\
	0 & 0 & 0 & 1 \\
\end{pmatrix}\quad\quad
\beta = \frac{v}{c} = \tanh\phi}
Using this transformation on a set of two points $A^\mu$ and $B^\mu$, we observe two primary phenomena in terms of the spatial separation $\Delta x = B^1 - A^1 = B_x - A_x$ and the temporal separation between $A$ and $B$, $\Delta t = B^0 - A^0 = B_t - A_t$. Applying the transformation matrix we observe the following:
\eqn{\begin{split}
	\Delta x &= \gamma \Delta x'\quad\quad \text{Length Contraction} \\
	\Delta t' &= \gamma \Delta t\quad\quad \text{Time Dilation}
\end{split}}
% Paradoxes? (Bard-pole + Twin Paradox)
% Doppler & Addition of velocities
Let us now say that an object has a velocity $u'$ in frame $K'$, it is natural to ask what the velocity $u$ is measured in frame $K$. For that, we need to use the relativistic addition of velocities formula:
\eqn{u = \frac{v+u'}{1 + \wrap{vu'/c^2}}}
Lastly, note that for an electromagnetic wave propagating at speed $c$ in frame $K$, the Lorentz transformation implies a change in frequency in frame $K'$ according to the below. Note that the sign of the numerator is the same as $\ddt\Delta x$, (positive/negative for diverging/converging observers).
\eqn{\frac{\lambda_r}{\lambda_s}=\frac{f_s}{f_r}=\sqrt{\frac{1+\beta}{1-\beta}}}

%\section{Relativistic Kinematics}
% OMITTED - can include if relevant for qual exams
% Lagrangian
% Collisions

\section{Experimental Verification}
We briefly summarize some of the experimental evidence for the special theory of relativity.
% Muon decay
Muon decay experiments have shown that fastly-moving particles take longer to decay in the LAB frame than stationary ones, but the proper time is equal.
% Atomic clock
Atomic clock experiments have shown measurable differences between fast-moving clocks in airplanes or satellites and slow-moving clocks.